{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Gradient descent" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Simple linear regression\n", "\n", "In a previous notebook, we solved the problem of simple linear regression - finding a straight line that best fits a data set with two variables.\n", "In that case, we were able to find the exact solution.\n", "In this notebook, we'll use a common technique to approximate that solution.\n", "\n", "Why would we want to approximate a solution when we can easily find an exact solution?\n", "We don't - it's just that the technique we discuss here can also be used in situations where we can't find an exact solution or don't want to try, for whatever reason.\n", "The approximation technique is called gradient descent." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Preliminaries" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# numpy efficiently deals with numerical multi-dimensional arrays.\n", "import numpy as np\n", "\n", "# matplotlib is a plotting library, and pyplot is its easy-to-use module.\n", "import matplotlib.pyplot as pl\n", "\n", "# This just sets the default plot size to be bigger.\n", "pl.rcParams['figure.figsize'] = (16.0, 8.0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Simple linear regression model\n", "In simple linear regression, we have some data points $(x_i, y_i)$, and we decide that they belong to a straight line with a little bit of error involved.\n", "Straight lines in two dimensions are of the form $y = mx + c$, and to fit a line to our data points we must find appropriate values for $m$ and $c$.\n", "Numpy has a function called `polyfit` that finds such values for us." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Best fit is m = 5.404076 and c = 7.755549\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd4VGXax/HvbSQGBAtVlEVYXlRqIgYhsmA0wrr2xq6IgoqCZVdRLKCr6FrAig0LgsAKr6iwLCwCCoGxwCySUAxFQYoaQQkgIAQcktzvH8/ACyQhkzCTk5m5P9eVazJnzpn5ieHOw3OeIqqKMcaY6HeU1wGMMcaEhxV0Y4yJEVbQjTEmRlhBN8aYGGEF3RhjYoQVdGOMiRFW0I0xJkZYQTfGmBhhBd0YY2LE0ZX5YXXr1tUmTZpU5kcaY0zUy87O3qyq9co6r1ILepMmTcjKyqrMjzTGmKgnIt+Fcp51uRhjTIwIqaCLyN0iskxElotI/+Cx2iIyS0RWBx9PjGxUY4wxh1NmQReR1sCtwNlAMnCJiDQHBgKZqtocyAw+N8YY45FQ+tBbAP9V1XwAEfkUuBK4HEgPnjMW8AEPljfA3r17yc3NZc+ePeW91MSApKQkGjVqRLVq1byOYkzUC6WgLwOeEpE6wG7gIiALaKCqGwFUdaOI1K9IgNzcXGrVqkWTJk0QkYq8hYlSqsqWLVvIzc2ladOmXscxJuqV2eWiqiuBZ4BZwExgKVAQ6geISF8RyRKRrLy8vGKv79mzhzp16lgxj0MiQp06dexfZ8aESUg3RVV1lKq2U9UuwFZgNfCziDQECD5uKuXaEaqaqqqp9eqVPIzSinn8sv/3xoRPqKNc6gcfGwNXAe8BU4HewVN6A1MiEdAYY6LZ999D//5QEHK/RsWFOg59koisAP4D3KmqvwBDga4ishroGnwelXJzc7n88stp3rw5zZo14+677yYQCJR47oYNG7jmmmvKfM+LLrqIbdu2VSjPY489xvPPP1/meTVr1jzs69u2beP111+vUAZjzJEpKoLXX4dWreDtt2HJksh/ZqhdLp1VtaWqJqtqZvDYFlXNUNXmwcetkY0aGarKVVddxRVXXMHq1atZtWoVO3fu5OGHHy52bkFBASeffDITJ04s832nT5/OCSecEInIIbOCbow3vvkG0tPhzjshLQ2WL4fU1Mh/blTOFPX7/QwZMgS/33/E7zVnzhySkpK46aabAEhISGDYsGG888475OfnM2bMGLp3786ll15Kt27dWL9+Pa1btwYgPz+fP//5z7Rt25a//OUvdOjQYf/SBk2aNGHz5s2sX7+eFi1acOutt9KqVSu6devG7t27AXj77bdp3749ycnJXH311eTn5x8267p160hLS6N9+/Y88sgj+4/v3LmTjIwM2rVrR5s2bZgyxfV+DRw4kDVr1pCSksL9999f6nnGmPAoKIChQyE5GXJyYPRo+PhjqLQlrFS10r7OOussPdSKFSuKHTuc+fPna/Xq1TUhIUGrV6+u8+fPL9f1h3r55Ze1f//+xY6npKTo0qVLdfTo0XrKKafoli1bVFV13bp12qpVK1VVfe6557Rv376qqpqTk6MJCQm6cOFCVVU99dRTNS8vT9etW6cJCQm6ePFiVVXt3r27vvvuu6qqunnz5v2f9/DDD+srr7yiqqqDBw/W5557rlimSy+9VMeOHauqqq+99poee+yxqqq6d+9e3b59u6qq5uXlabNmzbSoqOigrIc7z2vl/RkwpipavFi1XTtVUL3qKtWNG8P33kCWhlBjo66F7vP5CAQCFBYWEggE8Pl8R/R+qlriSIsDj3ft2pXatWsXO+eLL77g2muvBaB169a0bdu2xM9o2rQpKSkpAJx11lmsX78egGXLltG5c2fatGnD+PHjWb58+WGzzps3jx49egBwww03HJT1oYceom3btlxwwQX8+OOP/PzzzyX+N4VynjEmdHv2wMMPuy6VH3+EiRNh0iQ46aTKzxJ1BT09PZ3ExEQSEhJITEwkPT39iN6vVatWxVaA3LFjBz/88APNmjUD4Nhjjy3xWveLs2zHHHPM/u8TEhIoCN7uvvHGG3nttdfIyclh8ODBIY3HLumXz/jx48nLyyM7O5slS5bQoEGDEt8r1POMMaGZPx/OPBOefhpuuAFWrICrr/YuT9QV9LS0NDIzM3niiSfIzMwkLS3tiN4vIyOD/Px8/vnPfwJQWFjIgAEDuPHGG6lRo8Zhr/3DH/7ABx98AMCKFSvIyckp12f/+uuvNGzYkL179zJ+/Pgyz+/UqRMTJkwAOOj87du3U79+fapVq8bcuXP57ju30matWrX49ddfyzzPGFM+O3fCXXfBH/4Au3e7fvLRo6GEf8hXqqgr6OCK+qBBg464mINr8U6ePJkPP/yQ5s2bc9ppp5GUlMTTTz9d5rV33HEHeXl5tG3blmeeeYa2bdty/PHHh/zZTzzxBB06dKBr166cccYZZZ7/8ssvM3z4cNq3b8/27dv3H+/ZsydZWVmkpqYyfvz4/e9Vp04dOnXqROvWrbn//vtLPc8YE7pPPoHWreG11+Cvf4Vly6BbN69TORJqt0E4pKam6qHdGytXrqRFixaVliGcCgsL2bt3L0lJSaxZs4aMjAxWrVpFYmKi19GiSjT/DJj4sXUrDBgAY8bA6afDqFHQqVPlfLaIZKtqmQMfK3XHoliTn5/Peeedx969e1FV3njjDSvmxsSgSZPcmPLNm+Ghh+CRRyApyetUxVlBPwK1atWyLfWMiWE//eS6VSZNcjc/Z86E4IC1KskKujHGBPn9fnw+H+eem86qVWncc4+76Tl0qOtuObqKV8wqHs8YYyqH3+8nIyOD335rCLSnqMiNYhk50vWZR4OoHOVijDHhNmeOjz17bqWoaClFRR247LKP+fTT6CnmYAXdGGNYuRLef/9OVF8GvuCYY1IZOPA4joqyCmldLsaYuLV3Lzz3HDz+ONSseRyPPLKapKTFnHfemLDMc4H/75dPT08P23uWxgq6MSYuLVoEffq4dcr//Gd45RVo0KA5MChsn7GvXz4QCJCYmBiW2e2HE2X/oIichIQEUlJSSE5Opl27dsyfP7/c71HW+uOvvPIKLVq0oGfPnpxzzjkhXXOoJk2a0KZNG1JSUkg9zALLhYWFnHnmmVxyySWh/weE6JtvviElJWX/13HHHcdLL71U7Lxhw4bRqlUrWrduTY8ePWzdGFMl7N4NgwbB2We7YYmTJ8P770ODBuH/rHAvJlimUJZkDNdXOJbPjZR9S9Gqqs6cOVO7dOlS7vc4dLnaQ51++um6du3acl1zqH3L8pblhRde0B49eujFF18c8nvvM3fuXO3du3dI5xYUFGiDBg10/fr1Bx3Pzc3VJk2aaH5+vqq6ZYNHjx5d4ntUlZ8BE/s++0z1tNPcErd9+qhu3RrZzwvXct/E6vK5lWHHjh2ceOKJAIwbN46zzz6blJQU+vXrR2FhIQC7du3i4osvJjk5mdatW/P+++8X21DiQLfddhtr167lsssuY9iwYfu3jzvcNRWVm5vLRx99xC233HLQ8fPOO49Zs2YB8Pe//5277rrriD8rMzOTZs2aceqppxZ7raCggN27d1NQUEB+fj4nn3zyEX+eMVD+TW5+/dXN9OzSBQIBmDXLDUcM/jWPmHAvJliWkPrQReQe4BZAgRzgJqAhMAGoDSwCblDVkjfiDFH//uHfdy8lBUroDShm9+7dpKSksGfPHjZu3MicOXNYuXIl77//PvPmzaNatWrccccdjB8/nl69ejFz5kxOPvlkPvroI8CtZNihQweWLVvGkhL+I958801mzpzJ3LlzqVu37v4dh4YOHVrsmosuuoiRI0eWWABFhG7duiEi9OvXj759+xY7p3///jz77LMHrbQI8Pjjj/Poo4+yadMmFi9ezNSpU8v+gynDhAkT9q/RfqBTTjmF++67j8aNG1O9enW6detGt6qygpGJauXtl54xA/r1g9xcV2OefBJKWRE7ItLS0iJeyPcps4UuIqcAdwGpqtoaSACuBZ4Bhqlqc+AXoE8kg0Za9erVWbJkCV9//TUzZ86kV69ezJ49m+zsbNq3b09KSgqZmZmsXbsWgDZt2jB79mwefPBBPv/883KtsliW6dOnl9qanTdvHosWLWLGjBkMHz6czz777KDXp02bRv369TnrrLOKXdulSxdUlRdffJEJEyaQkJBw0OsdOnQgJSWFW265halTp+7vI//4449LzBIIBJg6dSrdu3cv9tovv/zClClTWLduHRs2bGDXrl2MGzcu1D8CY0oVar/0li3QqxdcdBHUrAnz5sGwYZVbzCtbqKNcjgaqi8heoAawETgfuC74+ljgMeCNIwkTSku6MqSlpbF582a2bt1K7969GTJkSLFzTjvtNLKzs5k+fTqDBg2iW7du9OrVK+LZ9hX6+vXrc+WVV/Lll1/SpUuX/a/PmzePqVOnMn36dPbs2cOOHTu4/vrrGTduHDk5OWzcuJG6detSq1atYu+9YMECwP2FGTNmDGPGjDlslhkzZtCuXTsalHA3afbs2TRt2pR69eoBcNVVVzF//nyuv/76iv6nGwP8/yY3+1roh25yo+p2DfrrX90KiY884nYUOmCfmZhVZgtdVX8Enge+xxXy7UA2sE1VC4Kn5QKnlHS9iPQVkSwRycrLywtP6gj7+uuvKSws5JprrmHixIls2rQJgK1bt+7fFGLDhg3UqFGD66+/nvvuu49FixYV21AiFOW5ZteuXfvP3bVrF5988sn+Dav3GTJkCLm5uaxfv54JEyZw/vnnM27cODZu3EjPnj2ZMmUKxx57bKmt7vJ47733SuxuAWjcuDH//e9/yc/PR1XJzMy0JXJNWByuX3rjRrjqKjcM8Xe/g6ws+Mc/4qOYA2WPcgFOBOYA9YBqwL+BG4BvDzjnd0BOWe9VlUe5HHXUUZqcnKzJycnatm1bnTZtmqqqTpgwQZOTk7VNmzbarl079fv9qupGwrRp00aTk5M1NTV1/+bQPXr00FatWul9991X7DMOHKFy4KiaQ6/505/+pD/++GOx69esWaNt27bVtm3basuWLfXJJ5/c/1pJ18ydO1cvvvhi3bVrl3bs2FE/+eQTVVX99NNPtWPHjqX+WYQyymXXrl1au3Zt3bZt20HHD8zx6KOP6umnn66tWrXS66+/Xvfs2VPie1WVnwETvYqKVEeNUj3+eNWkJNVnn1Xdu9frVOFDiKNcytzgQkS6Axeqap/g815AGtAdOElVC0QkDXhMVf94uPeKtQ0uTHjYz0DsqoxZkuvWQd++MHu2G8UyciQ0bx6Rj/JMODe4+B7oKCI1gN1ABpAFzAWuwY106Q1MqXhcY0ysifQsycJCtw3cQw9BQgK88YYr7NG2/ko4hdKHvgCYiBuamBO8ZgTwIHCviHwL1AFGRTCnMSbKRHKW5IoV0LmzG4aYng7Ll8Ntt8V3MYcQR7mo6mBg8CGH1wJnhyOEqiIi4XgrE2XK6vIz0aus0SgVEQjAs8/CE09ArVowbhxcdx1Y+XA8X5wrKSmJLVu2UKdOHSvqcUZV2bJlC0lVcXNGc8T2jUYJVx96VpZbTOurr+Daa+Hll6F+/TCFjRGeF/RGjRqRm5tLtAxpNOGVlJREo0aNvI5hIiQcsyR374bBg+GFF+Ckk2DKFLjssjAFjDGeF/Rq1arRtGlTr2MYY6qgTz+FW26Bb7+FW2913S0nnOB1qqorzm8hGGOqoh074Pbb3Q3PoiLIzIQRI6yYl8UKujGmSvnoI2jVyhXwe++FnBw4/3yvU0UHK+jGmCph82a4/nq45BI4/niYP9/1m9eo4XWy6GEF3RjjKVWYMAFatIAPPoDHHnPbw3Xo4HWy6OP5TVFjTPz68UfXV/6f/7gt4UaNgkPWmzPlYC10Y0ylU4W334aWLd0aLC+84LpYrJgfGWuhG2Mq1Zo1bgji3Llw3nmusDdr5nWq2GAtdGNMpSgsdC3xNm0gO9sV8sxMK+bhZC10Y0zELVsGN98MCxfCpZe6lRFPKXFLHHMkrIVujImYQMCNWmnXDtavd6NZpkyxYh4p1kI3xkTEl1+6Vvny5dCzp9szuG5dr1PFNmuhG2PCKj8fBgyAtDTYvh2mTXPL3FoxjzxroRtjwmbOHDeCZe1aN7586FA47jivU8UPa6EbY47Ytm2ukGdkuF2DfD54/XUr5pWtzIIuIqeLyJIDvnaISH8RqS0is0RkdfDxxMoIbIypWqZOdYtpvfMOPPCA24Di3HO9ThWfQtlT9BtVTVHVFOAsIB+YDAwEMlW1OZAZfG6MiRObNrmdgy6/HOrUgQUL4JlnoHp1r5PFr/J2uWQAa1T1O+ByYGzw+FjginAGM8ZUHr/fz5AhQ/D7/WWeqwrjx7tp+5Mnu/09s7IgNbUSgprDKu9N0WuB94LfN1DVjQCqulFEStzdT0T6An0BGjduXNGcxpgI8fv9ZGRk7N/MOTMzs9Rt4374wd3s/Ogj6NjRLabVsmUlBzalCrmFLiKJwGXAh+X5AFUdoaqpqppar1698uYzxkSYz+cjEAhQWFhIIBDA5/MVO6eoCN580/WVz53rxpR/8YUV86qmPF0ufwIWqerPwec/i0hDgODjpnCHM8ZEXnp6OomJiSQkJJCYmEh6evpBr69e7XYMuv12t8RtTg7cfTckJHiT15SuPF0uPfj/7haAqUBvYGjwcUoYcxljKklaWhqZmZn4fD7S09P3d7cUFMCwYfDoo3DMMa575aabQMTjwKZUoqplnyRSA/gB+L2qbg8eqwN8ADQGvge6q+rWw71PamqqZmVlHXFoY0xkffUV9OnjbnZecQUMHw4nn+x1qvglItmqWuZt55Ba6KqaD9Q55NgW3KgXY0yM+O03eOopGDIEatd2W8Jdc421yqOFTf03xgDg97tW+cqV0KsXvPiiG19uoodN/Tcmzu3aBffcA506wc6dMH06jB1rxTwaWQvdmDg2e7Zbg2X9erjjDreYVq1aXqcyFWUtdGPi0LZtrnula1eoVg0++8zd+LRiHt2soBsTZ/79bzchaOxYGDgQli6Fzp29TmXCwbpcjIkTP/8Mf/sbfPghpKS4jSfatfM6lQkna6EbE+NU4Z//hBYt3H6eTz3ltoezYh57rIVuTAz7/nvo1w9mzoRzznGzPc84w+tUJlKshW5MDCoqcjc5W7WCzz+HV191j1bMY5u10I2JMd98A7fc4lZD7NYN3noLmjTxOpWpDNZCNyZG7N3rxpEnJ8Py5TBmjOtqsWIeP6yFbkwMWLzYjStfvNitvfLqq3DSSV6nMpXNWujGRLE9e+Chh6B9e9iwASZNcsMSrZjHJ2uhGxOl5s1zrfJvvnHrlL/wApx4otepjJeshW5MlPn1VzdBqHNn10L/+GN45x0r5sYKujFR5eOPoU0bNyTxb3+DZcvcSBZjwAq6MVFh61a48Ua48EKoXt0NSXz5ZahZ0+tkpioJqaCLyAkiMlFEvhaRlSKSJiK1RWSWiKwOPto/+IyJgIkT3bT98ePh4YfdSJZzzvE6lamKQm2hvwzMVNUzgGRgJTAQyFTV5kBm8LkxJkw2boSrr4bu3aFRI1i4EJ58EpKSvE5mqqoyC7qIHAd0AUYBqGpAVbcBlwNjg6eNBa6IVEhj4okqjB7tlrj96CM3WWjBArdCojGHE0oL/fdAHjBaRBaLyEgRORZooKobAYKP9SOY05i4sH49/PGPcPPN7ubnV1/Bgw/C0TbA2IQglIJ+NNAOeENVzwR2UY7uFRHpKyJZIpKVl5dXwZjGRB+/38+QIUPw+/1lnltYCK+8Aq1bu82ahw8Hnw9OOy3yOU3sCOX3fi6Qq6oLgs8n4gr6zyLSUFU3ikhDYFNJF6vqCGAEQGpqqoYhszFVnt/vJyMjg0AgQGJiIpmZmaSlpZV47sqVbjGt+fPdKJa33oLGjSs5sIkJZbbQVfUn4AcROT14KANYAUwFegeP9QamRCShMVHI5/MRCAQoLCwkEAjg8/mKnbN3r9tsIiUFvv7abUIxfboVc1NxofbM/Q0YLyKJwFrgJtwvgw9EpA/wPdA9MhGNiT7p6ekkJibub6Gnp6cf9PqiRa6ffOlS+POfXXdLgwbeZDWxI6SCrqpLgNQSXsoIbxxjYkNaWhqZmZn4fD7S09P3d7fs3g2PPw7PPw/168PkyXCFjQ8zYWL3zo2JkLS0tIP6zT//3PWVr1rlFtV6/nk44QQPA5qYY1P/jYmwHTvgzjuhSxfXbz57NowcacXchJ8VdGMiaMYMNxTxjTegf3/IyYEM66g0EWJdLsZEwJYtcM898O67bsbn/PnQsaPXqUyssxa6MWGkCh984BbTeu89eOQRN6LFirmpDNZCN3HP7/cXG41SERs2uL7yf/8bzjrL9ZW3bRvGoEHhymtijxV0E9fKM6OzNKpux6ABA+C33+DZZ113SyTWXwlHXhO7rMvFxLVQZnQeztq1cMEFbjhiSopbTOv++yO3mNaR5jWxzQq6iWv7ZnQmJCSUOKOzNIWF8NJLbkXEhQvhzTdhzhxo3rxq5jXxwbpcTFwrbUbn4Sxf7iYGLVgAF1/sinmjRpUQlorlNfFDVCtvAcTU1FTNysqqtM8zJpwCAbfZxJNPwnHHufVXevQAEa+TmVgnItmqWtLyKwexFroxIVi40LXKc3Lg2mtdMa9Xz+tUxhzM+tCNOYz8fHeTs2NHN1loyhQ3vtyKuamKrIVuTCl8Pjd6Zc0a6NvXDUc8/nivUxlTOmuhG3OI7duhXz847zz3fM4ct4uQFXNT1VlBN+YA06ZBq1ZuNcQBA9y48n2F3Ziqzgq6MUBeHlx3HVx6KZx4otuo+fnnoUYNr5MZE7qQCrqIrBeRHBFZIiJZwWO1RWSWiKwOPp4Y2ajGhJ+qu8nZsiVMnAiPPQbZ2XD22V4nM6b8ytNCP09VUw4YCzkQyFTV5kBm8LkxUSM3Fy67zLXMf/97tyri4MGQmOh1MmMq5ki6XC4Hxga/HwvYzogmKhQVwYgRrq88MxNeeMGtV966tdfJjDkyoRZ0BT4RkWwR6Rs81kBVNwIEH+tHIqAx4fTtt27HoH793BK3OTlw772QkOB1MmOOXKjj0Dup6gYRqQ/MEpGvQ/2A4C+AvgCNGzeuQERjjty+xbQeeQSqVYO333YzP23avoklIbXQVXVD8HETMBk4G/hZRBoCBB83lXLtCFVNVdXUeja9znggJwfS0uC++9xStytWuAlDVsxNrCmzoIvIsSJSa9/3QDdgGTAV6B08rTcwJVIhjamI335zNznbtYP162HCBDd1/5RTvE5mTGSE0uXSAJgsrjlzNPC/qjpTRBYCH4hIH+B7oHvkYhpTPgsWuC6V5cuhZ0/X3VK3rtepjImsMgu6qq4Fkks4vgXIiEQoYypq1y7XT/7SS64lPm2aW7PcmHhgi3OZmDFnDtx6q9sW7vbb3drlxx3ndSpjKo9N/TdRb9s2V8gzMuCoo9wqia+/bsXcxB8r6CaqTZ3qJgi98w488IBbTOvcc71OZYw3rKCbqLRpk9s56PLL3c3OBQvgmWegenWvkxnjHSvoJqqowrhx0KIFTJ4MTzwBWVmQWuZui8bEPrspaqLGDz/AbbfB9OluS7hRo9wqicYYx1ropsorKoI33nB95T6fG5L4xRdWzI05lLXQTZW2erWbpv/ZZ27a/ogR0LSp16mMqZqshW6qpIICtylz27Zu5Mo778Ann1gxN+ZwrIVuqpylS920/exsuPJKGD4cGjb0OpUxVZ+10E2V8dtvbtp+aqq7AfrhhzBpkhVzY0JlLXRTJfj9rlW+ciX06gUvvgh16nidypjoYi1046mdO6F/f+jUyS2sNWMGjB1rxdyYirAWuvHMrFnQt69bq/yvf4Wnn4ZatbxOZUz0sha6qXS//OK6V7p1g2OOgc8/h1dfLbuY+/1+hgwZgt/vr5ygxkQZa6GbSjV5MtxxB+TlwaBB8OijkJRU9nV+v5+MjAwCgQCJiYlkZmaSlpYW+cDGRBFroZtK8dNP0L07XHUVnHQSfPml62IJpZgD+Hw+AoEAhYWFBAIBfD5fRPMaE41CLugikiAii0VkWvB5UxFZICKrReR9EUmMXEwTrVTdTc6WLeE//3FF/Msv3T6f5ZGenk5iYiIJCQkkJiaSnp4ekbzGRLPytNDvBlYe8PwZYJiqNgd+AfqEM5iJft99B3/6E9x4o1sdcckS181SrVr53ystLY3MzEyeeOIJ624xphQh9aGLSCPgYuAp4F5xO0afD1wXPGUs8BjwRgQymihTVOR2DBo40D1/9VXXb37UEXbwpaWlWSE35jBCvSn6EvAAsG8cQh1gm6oWBJ/nAqeEOZuJQt9840awzJsHf/wjvPUWnHqq16mMiQ9ltplE5BJgk6pmH3i4hFO1lOv7ikiWiGTl5eVVMKap6vbuhSFDIDkZVqyAMWPcJCEr5sZUnlBa6J2Ay0TkIiAJOA7XYj9BRI4OttIbARtKulhVRwAjAFJTU0ss+sY7fr8fn89Henp6hbszFi92rfLFi+Gaa1wXy0knhTmoMaZMZRZ0VR0EDAIQkXTgPlXtKSIfAtcAE4DewJQI5jQRcKRju/fsgX/8wy1zW7euW0jrqqsiGNgYc1hHcpvqQdwN0m9xfeqjwhPJVJYjGdv9xReue2XIELeY1sqVVsyN8Vq5Zoqqqg/wBb9fC5wd/kimsuwb272vhR7K2O5ff3VDD4cPhyZN3KYTXbsWPy8cXTnGmPKxqf9xbN/Y7lAL78cfu8W0fvgB7roLnnoKatYsfp5N0zfGG1bQ41woY7u3boV77oF//hPOOMN1t5xzTunnl9SVYwXdmMiztVzMYU2c6GZ5/u//wsMPu5EshyvmYNP0jfGKtdBNiTZuhDvvdKsjtmvnultSUkK7trxdOcaY8LCCbg6i6iYF3XuvG5b4zDPu+6PL+ZNi0/SNqXxW0M1+69a5m56zZ0PnzjByJJx2mtepjDGhsj50Q2EhvPIKtG4N//2vW1jL57Nibky0sRZ6nFu50k3b9/vdUrdvvgmNG3udyhhTEdZCj1N797oXMGjeAAAMqElEQVRx5CkpboXEd9+Fjz6yYm5MNLMWehzKzoabb4avvoK//MV1t9Sv73UqY8yRshZ6HNm9Gx58EDp0cJs0//vfMGGCFXNjYoW10OPEZ5/BLbfA6tXu8bnn4IQTvE5ljAkna6HHuB073PZv554LBQVuSOLbb1sxNyYWWUGPYdOnu6GIb77p1mLJyYGMDK9TGWMixbpcYtDmza6AjxsHLVvC/PnQsaPXqYwxkWYt9BiiCh984Ir4hAnw6KOwaJEVc2PihbXQY8SGDa6vfMoUSE2FzExo08brVMaYylRmC11EkkTkSxFZKiLLReTx4PGmIrJARFaLyPsikhj5uOZQqm7NlZYt3YqIzz/vZn1aMTcm/oTS5fIbcL6qJgMpwIUi0hF4Bhimqs2BX4A+kYtpSrJ2LVxwAdx6q5vxmZMDAwaUf2VEY0xsKLOgq7Mz+LRa8EuB84GJweNjgSsiktAUU1gIw4a5ESwLF8Jbb8GcOfA//+N1MmOMl0K6KSoiCSKyBNgEzALWANtUtSB4Si5wSmQimgMtXw6dOrk1ys8/H1ascEveHmW3t42JeyGVAVUtVNUUoBFwNtCipNNKulZE+opIlohk5eXlVTxpnAsE4B//gDPPhDVr3JZw//kPNGrkdTJjTFVRrnadqm4DfEBH4AQR2ddb2wjYUMo1I1Q1VVVT69WrdyRZ49bChW7kyuDB0L27a5X36AEiXiczxlQloYxyqSciJwS/rw5cAKwE5gLXBE/rDUyJVMh4lZ8P99/vxpFv3QpTp8L48WC/F40xJQllPERDYKyIJOB+AXygqtNEZAUwQUSeBBYDoyKYM+74fG4RrTVroF8/t7fn8cd7ncoYU5WVWdBV9SvgzBKOr8X1p5sw2r4dHngARoyAZs3c6JXzzvM6lTEmGtjYiCpk2jRo1cpNFLrvPrcBhRVzY0yorKBXAXl5cN11cOmlULu226j5ueegRg2vkxljookVdA+pwnvvuWn7EyfC449DVha0b+91MmNMNLJJ4h7JzYXbb3fdLB06wKhRrrvFGGMqylrolayoyE3Vb9nSrYj44oswb54Vc2PMkbMWeiX69lu3kJbP56btv/02/P73XqcyxsQKa6FXgoICt6xtmzaweLEbxTJ7thVzY0x4WQs9Avx+Pz6fj/T0dGrWTKNPHzd9//LL4fXX4eSTvU5ojIlFVtDDzO/3k5GRwW+/wVFHKaodqF37KN5/363DYuuvGGMixQp6mPl8Pn777UyKikZQVNSKlJRlzJ7dmjp1vE5mjIl11oceRrt2wdKlvSkq+hw4jsTEK3n99V+tmBtjKoW10MMkM9ONYFm37mSuvPIn2rT5gAsvfIC0tDSvoxlj4oQV9CO0bZtb4nbkSGjeHD79FLp0OQkY4HU0Y0ycsS6XIzBlipsgNHo0PPggLF0KXbp4ncoYE6+soFfApk1w7bVwxRVQvz4sWABDh0L16pH7TL/fz5AhQ/D7/ZH7EGNMVIvrLpcDx4uH0tet6nYMuvtu2LkTnnzSrV1erVrkc2ZkZBAIBEhMTCQzM9P65o0xxcRtQS9vkfz+e7jtNpgxA9LS3GJaLUraKjsCfD4fgUCAwsJCAoEAPp/PCroxpphQ9hT9nYjMFZGVIrJcRO4OHq8tIrNEZHXw8cTIxw2fkopkSYqK4I033OJZn34KL78Mn39eecUcID09ncTERBISEkhMTCQ9Pb3yPtwYEzVCaaEXAANUdZGI1AKyRWQWcCOQqapDRWQgMBB4MHJRw2tfkdzXQi+pSK5a5fb1/Pxz6NrVbQvXpEmlRyUtLY3MzMxydQ8ZY+KPqGr5LhCZArwW/EpX1Y0i0hDwqerph7s2NTVVs7KyKhw23ErrQy8ocMvaDh4MSUkwbBj07m3T9o0x3hCRbFVNLeu8cvWhi0gT3IbRC4AGqroRIFjU61cgp6fS0tKKtXaXLoWbb4ZFi+DKK2H4cGjY0KOAxhhTDiEPWxSRmsAkoL+q7ijHdX1FJEtEsvLy8iqSsVLs2QN//zukpsKPP7ot4f71LyvmxpjoEVJBF5FquGI+XlX/FTz8c7CrheDjppKuVdURqpqqqqn16tULR+awmz8fzjwTnnoKevaEFSvg6qu9TmWMMeUTyigXAUYBK1X1xQNemgr0Dn7fG5gS/niRtXOnG1P+hz9Afj7MnAljxkDt2l4nM8aY8gulD70TcAOQIyJLgsceAoYCH4hIH+B7oHtkIkbGrFnQty989x3ceSc8/TTUquV1KmOMqbgyC7qqfgGUNr4jI7xxIu+XX+Dee11L/PTT4bPPXAvdGGOiXVyt5fKvf7nFtN59FwYNgiVLrJgbY2JHXEz9/+kn+OtfYdIkSEmB6dPdTVBjjIklMd1CV4WxY12rfNo010/+5ZdWzI0xsSlmW+jr10O/fvDJJ9Cpk9uA4owzvE5ljDGRE3Mt9KIiePVVaN3ajS9/7TV349OKuTEm1sVUC/3rr91iWvPmwR//CG+9Baee6nUqY4ypHDHRQt+71/WPJye7WZ5jx7p1y62YG2PiSdS30Bctgj593BDEa65xXSwNGnidyhhjKl/UttB373Zjyc8+2w1LnDQJPvzQirkxJn5FZQv9iy9cq3zVKrjpJnjhBTgxqvZLMsaY8IuqFvqvv7oJQp07QyDghiS+844Vc2OMgShpofv9fkaM+J4ZM65g06ZjuPtuePJJqFnT62TGGFN1VPmC7vf76dx5BYWFfRBZyZtvFtC3bxuvYxljTJVT5btcfD4fRUWrgCcQOYstW6Z5HckYY6qkKt9CT09PJykpg0AgQGJiIunp6V5HMsaYKqnKF/S0tDQyMzPx+Xykp6cX29TZGGOMU+ULOriiboXcGGMOr8r3oRtjjAlNKJtEvyMim0Rk2QHHaovILBFZHXy0keDGGOOxUFroY4ALDzk2EMhU1eZAZvC5McYYD5VZ0FX1M2DrIYcvB8YGvx8LXBHmXMYYY8qpon3oDVR1I0DwsX5pJ4pIXxHJEpGsvLy8Cn6cMcaYskT8pqiqjlDVVFVNrVevXqQ/zhhj4lZFhy3+LCINVXWjiDQENoVyUXZ29mYR+a6CnxkpdYHNXocIUTRlhejKG01ZIbryRlNWqJp5Q9qup6IFfSrQGxgafJwSykWqWuWa6CKSpaqpXucIRTRlhejKG01ZIbryRlNWiL68Bwpl2OJ7gB84XURyRaQPrpB3FZHVQNfgc2OMMR4qs4Wuqj1KeSkjzFmMMcYcAZspCiO8DlAO0ZQVoitvNGWF6MobTVkh+vLuJ6rqdQZjjDFhYC10Y4yJEXFZ0EXkdyIyV0RWishyEbnb60yhEJEEEVksIlV6lw8ROUFEJorI18E/4yq9VKaI3BP8OVgmIu+JSJLXmQ4UTesplZL1ueDPwlciMllETvAy4z4lZT3gtftEREWkrhfZKiouCzpQAAxQ1RZAR+BOEWnpcaZQ3A2s9DpECF4GZqrqGUAyVTiziJwC3AWkqmprIAG41ttUxYwhetZTGkPxrLOA1qraFlgFDKrsUKUYQ/GsiMjvcKP3vq/sQEcqLgu6qm5U1UXB73/FFZxTvE11eCLSCLgYGOl1lsMRkeOALsAoAFUNqOo2b1OV6WiguogcDdQANnic5yDRtJ5SSVlV9RNVLQg+/S/QqNKDlaCUP1eAYcADQNTdYIzLgn4gEWkCnAks8DZJmV7C/ZAVeR2kDL8H8oDRwe6hkSJyrNehSqOqPwLP41pjG4HtqvqJt6lCEvJ6SlXMzcAMr0OURkQuA35U1aVeZ6mIuC7oIlITmAT0V9UdXucpjYhcAmxS1Wyvs4TgaKAd8Iaqngnsoup0BxQT7Hu+HGgKnAwcKyLXe5sqNonIw7juzvFeZymJiNQAHgYe9TpLRcVtQReRarhiPl5V/+V1njJ0Ai4TkfXABOB8ERnnbaRS5QK5qrrvXzwTcQW+qroAWKeqeaq6F/gXcI7HmULxc3AdJcqznpJXRKQ3cAnQU6vuWOlmuF/sS4N/1xoBi0TkJE9TlUNcFnQREVwf70pVfdHrPGVR1UGq2khVm+Bu2M1R1SrZilTVn4AfROT04KEMYIWHkcryPdBRRGoEfy4yqMI3cQ+wbz0lKMd6Sl4QkQuBB4HLVDXf6zylUdUcVa2vqk2Cf9dygXbBn+moEJcFHdfivQHX0l0S/LrI61Ax5G/AeBH5CkgBnvY4T6mC/5KYCCwCcnB/J6rUTMFoWk+plKyvAbWAWcG/a296GjKolKxRzWaKGmNMjIjXFroxxsQcK+jGGBMjrKAbY0yMsIJujDExwgq6McbECCvoxhgTI6ygG2NMjLCCbowxMeL/ANDquynrlCP7AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "w = np.arange(1.0, 16.0, 1.0)\n", "d = 5.0 * w + 10.0 + np.random.normal(0.0, 5.0, w.size)\n", "\n", "m, c = np.polyfit(w, d, 1)\n", "print(\"Best fit is m = %f and c = %f\" % (m, c))\n", "\n", "# Plot the best fit line.\n", "pl.plot(w, d, 'k.', label='Original data')\n", "pl.plot(w, m * w + c, 'b-', label='Best fit: $%0.1f x + %0.1f$' % (m,c))\n", "pl.legend()\n", "pl.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Gradient descent\n", "In gradient descent, we select a random guess of a parameter and iteratively improve that guess.\n", "For instance, we might pick $1.0$ as our initial guess for $m$ and then create a `for` loop to iteratively improve the value of $m$.\n", "The way we improve $m$ is to first take the partial derivative of our cost function with respect to $m$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Cost function\n", "Recall that our cost function for simple linear regression is:\n", "\n", "$$\n", "Cost(m, c) = \\sum_i (y_i - mx_i - c)^2\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Calculate the partial derivatives\n", "We calculate the partial derivative of $Cost$ with respect to $m$ while treating $c$ as a constant.\n", "Note that the $x_i$ and $y_i$ values are all just constants.\n", "We'll also calculate the partial derivative with respect to $c$ here.\n", "\n", "$$\n", "\\begin{align}\n", "Cost(m, c) &= \\sum_i (y_i - mx_i - c)^2 \\\\[1cm]\n", "\\frac{\\partial Cost}{\\partial m} &= \\sum 2(y_i - m x_i -c)(-x_i) \\\\\n", " &= -2 \\sum x_i (y_i - m x_i -c) \\\\[0.5cm]\n", "\\frac{\\partial Cost}{\\partial c} & = \\sum 2(y_i - m x_i -c)(-1) \\\\\n", " & = -2 \\sum (y_i - m x_i -c) \\\\\n", "\\end{align}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Code the partial derivatives\n", "Once we've calculated the partial derivatives, we'll code them up in python.\n", "Here we create two functions, each taking four parameters.\n", "The first two parameters are arrays with our $x_i$ and $y_i$ data set values.\n", "The second two are our current guesses for $m$ and $c$." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def grad_m(x, y, m, c):\n", " return -2.0 * np.sum(x * (y - m * x - c))" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "def grad_c(x, y, m , c):\n", " return -2.0 * np.sum(y - m * x - c)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Iterate\n", "Now we can run our gradient descent algorithm.\n", "For $m$, we keep replacing its value with $m - \\eta grad\\_m(x, y, m, c)$ until it doesn't change.\n", "For $c$, we keep replacing its value with $c - \\eta grad\\_c(x, y, m, c)$ until it doesn't change.\n", "\n", "What is $\\eta$? It is called the learning rate and we set it to a small value relative to the data points.\n", "\n", "You can see on each iteration, $m$ and $c$ are getting closer to their true values." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "m: 5.7149506777702488 c: 4.5518695120877757\n", "m: 5.5629397868829846 c: 6.1183980049248472\n", "m: 5.4852588608786546 c: 6.9189286986499470\n", "m: 5.4455621902673004 c: 7.3280175818245992\n", "m: 5.4252763139437015 c: 7.5370710455442040\n", "m: 5.4149097826482908 c: 7.6439019874021357\n", "m: 5.4096122559659570 c: 7.6984949605832087\n", "m: 5.4069051027099357 c: 7.7263931768575951\n", "m: 5.4055216875445105 c: 7.7406497821666767\n", "m: 5.4048147318094859 c: 7.7479352226744274\n", "m: 5.4044534617990276 c: 7.7516582438453998\n", "m: 5.4042688448364773 c: 7.7535607899748742\n" ] } ], "source": [ "eta = 0.0001\n", "m, c = 1.0, 1.0\n", "delta = 0.0000001\n", "\n", "mold, cold = m - 1.0, c - 1.0\n", "i = 0\n", "while abs(mold - m) > delta and abs(cold - c) > delta:\n", " mold, cold = m, c\n", " m = mold - eta * grad_m(w, d, mold, cold)\n", " c = cold - eta * grad_c(w, d, mold, cold)\n", "\n", " i = i + 1\n", " if i % 1000 == 0:\n", " print(\"m: %20.16f c: %20.16f\" % (m, c))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Newton's method for square roots\n", "Newton's method for square roots is a method for approximating the square root of a number $x$.\n", "We begin with an initial guess $z_0$ of the square root - it doesn't have to be particularly good.\n", "We then apply the following calculation repeatedly, to calculate $z_1$, then $z_2$, and so on:\n", "\n", "$$\n", "z_{i+1} = z_i - \\frac{z_i^2 - x}{2z_i}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Coding the calculation\n", "We can create a function that calculates the next value of $z$ based on the current value of $z$ as follows." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "def next_z(x, z):\n", " return z - (z**2 - x) / (2 * z)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Calculating the square root of $x$\n", "Suppose we want to calculate the square root of $x$.\n", "We start with a random guess for the square root, $z_0$.\n", "We then apply the `next_z` function repeatedly until the value of $z$ stops changing.\n", "Let's create a function to do this.\n", "We'll include the next_z function inside the `newtsqrt` function to make it one all-inclusive package." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "3.75\n", "3.341666666666667\n", "3.316718620116376\n", "3.3166247916826186\n", "3.3166247903554\n" ] }, { "data": { "text/plain": [ "3.3166247903554" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def newtsqrt(x):\n", " next_z = lambda x, z: z - (z**2 - x) / (2 * z)\n", " z = 2.0\n", " n = next_z(x, z)\n", " \n", " while z != n:\n", " z, n = n, next_z(x, n)\n", " print(z)\n", " \n", " return z\n", "\n", "newtsqrt(11)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Comparison with the standard library\n", "We can compare our square root method return value to the value calculated by Python's `math` standard library package. It has a `sqrt` function." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "3.3166247903554" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import math\n", "math.sqrt(11)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Being careful\n", "Due to the complexities of floating point numbers, the `nextsqrt` function could get into an infinite loop.\n", "For instance, calculating the square root of 10 gives an infinite loop." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "# Uncommenting will result in infinite loop.\n", "# newtsqrt(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To counteract this problem, the condition of the loop is better written as:\n", "```python\n", "abs(z - n) > 0.001\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Gradient descent for square roots\n", "Newton's method for square roots is efficient, but we can also use gradient descent to approximate the square root of a real number $x$.\n", "Here, we use the following cost function.\n", "\n", "$$\n", "Cost(z \\mid x) = (x - z^2)^2\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example value\n", "Let's use it to calculate the square root of 20, i.e. $x = 20$.\n", "Then the cost function is:\n", "\n", "$$\n", "Cost(z \\mid x=20) = (20 - z^2)^2\n", "$$\n", "\n", "Our goal is to find the $z$ that minimises this." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plotting the cost function\n", "Let's plot the cost function.\n", "Given that we know the best $z$ will be between $4$ and $5$, we'll let $z$ range over 0 to 10." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3X10FOXd//H314QQECUYAgESnlHRqmBjxR+KUimCWoNwK6EgiCjtuVGwtw9Va9XW2mpLrQqKRkDABgF5EEpFBYn1KI9BqLcQFURJAggRCAokQJLr90cmuRMIJIFkJ7v7eZ0zZ2evvXbnO4Gzn5lrZmfMOYeIiISfM/wuQERE/KEAEBEJUwoAEZEwpQAQEQlTCgARkTClABARCVMKABGRMKUAEBEJUwoAEZEwFel3ASfTvHlz1759e7/LEBEJKuvWrfvOORdXVb96HQDt27cnIyPD7zJERIKKmW2rTj8NAYmIhCkFgIhImFIAiIiEqXp9DKAyR48eJScnh4KCAr9LCQnR0dEkJCTQoEEDv0sRkQALugDIycnhrLPOon379piZ3+UENecce/bsIScnhw4dOvhdjogEWNANARUUFBAbG6sv/1pgZsTGxmpvSiRMBV0AAPryr0X6W4qEr6AMABGRUPb888+zYMGCOl+OAkBEpB45cuQIjz32GIsXL67zZSkAgthbb73FXXfdRXJyMu+9994p9xGR+uODDz7g+++/Z8CAAXW+LAXAKcrPz+fqq69m27Zt9O7dm65du3LhhRfy/PPPV+j3zjvvcN5559G5c2eefvrpWq1hwIABvPrqq0ybNo3Zs2cDJVsPvXr1orCw8IR9RKT+WrhwIY0bN6ZPnz51viwFwCmaOnUqAwcOJCoqir/97W9kZmayatUqXnzxRTZt2gRAUVERY8aMYcmSJWzatIk33nij7LXa9Mc//pExY8YAEBUVxbXXXnvcl335PiJSPxUXF7Nw4UL69etHo0aN6nx5CoBTlJaWRnJyMq1ateLSSy8F4KyzzqJr165s374dgDVr1tC5c2c6duxIVFQUKSkpLFy48JSW99Of/pRu3brRrVs3oqOjefPNN3HO8Zvf/Ib+/fuX1QAlW/1paWkAJ+wjIvXPunXr2L59O8nJyQFZXtD9EKw+OHLkCFu3buXYS1V/8803rF+/nssvvxyA7du3k5iYWPZ6QkICq1evPqVlLl++HIBJkyaRnp7OwIEDmTBhAsuWLWP//v1s2bKFX/3qVwD86Ec/Yu3atQAn7CMi9c9bb71FREQEN9xwQ0CWF9QBcO+997Jhw4Za/cxu3brx3HPPnbTPd999R0xMTIW2AwcOMGjQIJ577jnOPvtsoGTr+1jHnnffp08fvv322+P6PfXUU8dtBcyYMYMlS5Ywb948IiIiGDt2LGPHjj3uvREREURFRfHDDz+csI+I1D8LFiygV69exMbGBmR5QR0AfmnUqFGFX88ePXqUQYMGMXToUAYOHFjWnpCQQHZ2dtnznJwcWrduXeGzli1bVq1lvvnmm6SlpbFw4cJqXbfn8OHDREdHV+uzRcR/mZmZZGZmBvRYXVAHQFVb6nWlWbNmFBUVUVBQQMOGDRk1ahRdu3blf/7nfyr0u+yyy9i8eTNff/01bdq0YdasWcycObPGy1u8eDEvvfQSixcvrtaX+p49e4iLi9MF3kSCyLx58wACcvpnqWodBDazGDOba2afm1mmmV1hZueY2VIz2+w9NvP6mpm9YGZbzOxTM7u03OeM8PpvNrMRdbVSgdC3b18++ugjPv74Y15//XWWL19edpD27bffBiAyMpKJEydy3XXX0bVrV2699VYuvPDCGi9rxIgR5OTk0LNnT7p168aUKVNO2j89PZ3rr7/+lNZLRPwxf/58rrjiCtq0aRO4hTrnqpyA6cCd3nwUEAP8BXjIa3sIeMabvx5YAhjQA1jttZ8DbPUem3nzzU623B//+MfuWJs2bTquzQ+ffPKJGzZsmN9lVOrmm292n3/+ebX715e/qUi4+uqrrxzgxo8fXyufB2S4any3V7kHYGZnA72AKV5gHHHO5QHJXjCUBkTpfksyMMOrYxUQY2atgOuApc65vc65fcBSoN8pZFa90L17d3r37k1RUZHfpVRw5MgRBgwYwHnnned3KSJSTfPnzweocAwxEKozBNQRyAVeM7P1ZjbZzM4EWjrndgJ4jy28/m2A7HLvz/HaTtQetO644w4iIiL8LqOCqKgohg8f7ncZIlID8+fPp3v37gG/L0d1AiASuBSY5JzrDhykZMjnRCq7vrA7SXvFN5uNNrMMM8vIzc2tRnkiIsFr+/btrFy5kkGDBgV82dUJgBwgxzlX+gumuZQEwi5vaAfvcXe5/onl3p8A7DhJewXOuVTnXJJzLikuLq4m6yIiEnRKL/tcLwPAOfctkG1mpYPK1wKbgEVA6Zk8I4DSaxwsAoZ7ZwP1APZ7Q0TvAn3NrJl3xlBfr01EJGzNnz+frl27cv755wd82dX9HcA9QJqZRVFy9s5ISsJjjpmNArKAW7y+b1NyJtAW4JDXF+fcXjN7Eljr9fuDc27vqRTtnNOdrGqJq+TXyiISGLm5ufz73//mkUce8WX51QoA59wGIKmSl66tpK8DKv0pm3NuKjC1JgUeKzo6mj179ui+wLXAeTeF1y+GRfyxcOFCiouLfRn+gSD8JXBCQgI5OTnoAHHtiI6OJiEhwe8yRMLS/Pnz6dChA5dccokvyw+6AGjQoEHAT5USEalteXl5LFu2jHHjxvk2mqH7AYiI+GDRokVlF5L0iwJARMQHs2fPpl27dmX3D/GDAkBEJMD27t3Le++9x6233urrySwKABGRAFuwYAGFhYUMHjzY1zoUACIiATZ79mw6derk+326FQAiIgGUm5vL8uXLGTx4sO+/ZVIAiIgE0Lx58ygqKvJ9+AcUACIiATVr1izOP/98LrroIr9LUQCIiATKjh07+PDDD+vF8A8oAEREAmbu3Lk45+rF8A8oAEREAmb27NlcdNFFdO3a1e9SAAWAiEhAZGdns2LFinqz9Q8KABGRgJgzZw6AAkBEJNykpaWRlJRE586d/S6ljAJARKSObdy4kfXr13Pbbbf5XUoFCgARkTqWlpZGREREvRr+AQWAiEidKi4uJi0tjZ/97Ge0bNnS73IqUACIiNShjz/+mKysLIYNG+Z3KcdRAIiI1KF//OMfNG7cmOTkZL9LOY4CQESkjhw+fJg5c+Zw880306RJE7/LOU61AsDMvjGz/zWzDWaW4bWdY2ZLzWyz99jMazcze8HMtpjZp2Z2abnPGeH132xmI+pmlURE6oe3336bvLy8ejn8AzXbA+jtnOvmnEvynj8EvO+c6wK87z0H6A908abRwCQoCQzgceBy4CfA46WhISISitLS0mjRogV9+vTxu5RKnc4QUDIw3ZufDgwo1z7DlVgFxJhZK+A6YKlzbq9zbh+wFOh3GssXEam38vLy+Oc//0lKSgqRkZF+l1Op6gaAA94zs3VmNtpra+mc2wngPbbw2tsA2eXem+O1nai9AjMbbWYZZpaRm5tb/TUREalH5s6dy5EjR+rt8A9AdWOpp3Nuh5m1AJaa2ecn6VvZRa7dSdorNjiXCqQCJCUlHfe6iEgweP311+nSpQtJSUlVd/ZJtfYAnHM7vMfdwAJKxvB3eUM7eI+7ve45QGK5tycAO07SLiISUrZu3cqHH37IiBEj6sWNX06kygAwszPN7KzSeaAv8BmwCCg9k2cEsNCbXwQM984G6gHs94aI3gX6mlkz7+BvX69NRCSkTJs2DTNj+PDhfpdyUtUZAmoJLPBSLBKY6Zx7x8zWAnPMbBSQBdzi9X8buB7YAhwCRgI45/aa2ZPAWq/fH5xze2ttTURE6oHi4mKmT59Onz59SExMrPoNPqoyAJxzW4FLKmnfA1xbSbsDxpzgs6YCU2tepohIcEhPTycrK4unn37a71KqpF8Ci4jUotdee42mTZsyYMCAqjv7TAEgIlJL9u/fz/z58xkyZAiNGjXyu5wqKQBERGrJnDlzyM/PZ+TIkX6XUi0KABGRWvLaa6/RtWtXLrvsMr9LqRYFgIhILfj8889ZuXIlI0eOrNfn/penABARqQXTp08nIiKi3t3392QUACIip6mwsJAZM2bQv39/4uPj/S6n2hQAIiKn6Z133mHHjh1Bc/C3lAJAROQ0paamEh8fz89//nO/S6kRBYCIyGnIycnhX//6F3fccQcNGjTwu5waUQCIiJyGqVOnUlxczJ133ul3KTWmABAROUVFRUVMnjyZvn370qFDB7/LqTEFgIjIKXr33XfJzs5m9OjRVXeuhxQAIiKnKDU1lZYtW3LTTTf5XcopUQCIiJyC7du3s3jxYkaOHBl0B39LKQBERE7Ba6+9RlFRUVAe/C2lABARqaHSg799+vShU6dOfpdzyhQAIiI19N5777Ft2zbuuusuv0s5LQoAEZEaevHFF4mPjw+Ku36djAJARKQGvvrqK95++21Gjx5NVFSU3+WcFgWAiEgNTJo0iYiICH75y1/6XcppUwCIiFTToUOHmDJlCgMHDqR169Z+l3Paqh0AZhZhZuvNbLH3vIOZrTazzWY228yivPaG3vMt3uvty33Gw177F2Z2XW2vjIhIXXrjjTfIy8tjzJgxfpdSK2qyBzAOyCz3/Bng7865LsA+YJTXPgrY55zrDPzd64eZXQCkABcC/YCXzCzi9MoXEQkM5xwTJ07koosu4qqrrvK7nFpRrQAwswTgBmCy99yAnwJzvS7TgdLD4cnec7zXr/X6JwOznHOHnXNfA1uAn9TGSoiI1LUVK1awYcMG7r777qC5529VqrsH8BzwIFDsPY8F8pxzhd7zHKCNN98GyAbwXt/v9S9rr+Q9ZcxstJllmFlGbm5uDVZFRKTuTJw4kaZNmzJ06FC/S6k1VQaAmd0I7HbOrSvfXElXV8VrJ3vP/zU4l+qcS3LOJcXFxVVVnohIndu5cydz585l5MiRnHnmmX6XU2siq9GnJ3CTmV0PRANnU7JHEGNmkd5WfgKww+ufAyQCOWYWCTQF9pZrL1X+PSIi9VZqaiqFhYX893//t9+l1Koq9wCccw875xKcc+0pOYi73Dk3FEgH/svrNgJY6M0v8p7jvb7cOee89hTvLKEOQBdgTa2tiYhIHSgoKOCll16if//+dOnSxe9yalV19gBO5DfALDP7I7AemOK1TwFeN7MtlGz5pwA45zaa2RxgE1AIjHHOFZ3G8kVE6tzMmTPZvXs39913n9+l1Dor2Tivn5KSklxGRobfZYhImHLOcdFFFxEREcGGDRuC5uwfM1vnnEuqqt/p7AGIiIS0pUuXsnHjRqZNmxY0X/41oUtBiIicwLPPPkt8fDwpKSl+l1InFAAiIpX47LPPePfdd7nnnnto2LCh3+XUCQWAiEglnnvuORo1ahQSV/08EQWAiMgxdu3axT/+8Q9uv/12YmNj/S6nzigARESOMWnSJA4fPsy9997rdyl1SgEgIlLOoUOHePHFF7nxxhs599xz/S6nTikARETKmTx5Mt999x0PPfSQ36XUOQWAiIjn6NGjjB8/niuvvJKePXv6XU6d0w/BREQ8M2fOJDs7m5dfftnvUgJCewAiIkBxcTHPPPMMF198Mf379/e7nIDQHoCICLBo0SIyMzOZOXNmSF72oTLaAxCRsOec489//jMdO3bklltu8bucgNEegIiEvfT0dNasWcPLL79MZGT4fC1qD0BEwt7TTz9NfHw8I0aMqLpzCFEAiEhYW7NmDUuXLuXXv/410dHRfpcTUAoAEQlrTzzxBLGxsSF3v9/qUACISNhavXo1S5Ys4YEHHqBJkyZ+lxNwCgARCVu///3viY2NZcyYMX6X4gsFgIiEpXDf+gcFgIiEqXDf+gcFgIiEIW39l6gyAMws2szWmNl/zGyjmf3ea+9gZqvNbLOZzTazKK+9ofd8i/d6+3Kf9bDX/oWZXVdXKyUicjLa+i9RnT2Aw8BPnXOXAN2AfmbWA3gG+LtzrguwDxjl9R8F7HPOdQb+7vXDzC4AUoALgX7AS2YWUZsrIyJSFW39/58qA8CVOOA9beBNDvgpMNdrnw4M8OaTved4r19rJVdWSgZmOecOO+e+BrYAP6mVtRARqQbnHA8//DBxcXFhv/UP1TwGYGYRZrYB2A0sBb4C8pxzhV6XHKCNN98GyAbwXt8PxJZvr+Q95Zc12swyzCwjNze35mskInICy5YtIz09nUcffTTst/6hmgHgnCtyznUDEijZau9aWTfvsbLrqLqTtB+7rFTnXJJzLikuLq465YmIVKm4uJiHH36Ydu3a8ctf/tLvcuqFGl32zjmXZ2YfAD2AGDOL9LbyE4AdXrccIBHIMbNIoCmwt1x7qfLvERGpU/PmzWPdunVMnz6dhg0b+l1OvVCds4DizCzGm28E9AEygXTgv7xuI4CF3vwi7zne68udc85rT/HOEuoAdAHW1NaKiIicyNGjR/ntb3/LhRdeyNChQ/0up96ozh5AK2C6d8bOGcAc59xiM9sEzDKzPwLrgSle/ynA62a2hZIt/xQA59xGM5sDbAIKgTHOuaLaXR0RkeNNmzaNzZs3s3DhQiIidPJhKSvZOK+fkpKSXEZGht9liEgQy8/Pp3PnzrRr146PP/44LG73aGbrnHNJVfULn1vfiEhYmjBhAjt27OCNN94Iiy//mtClIEQkZOXm5vLUU09x/fXX06tXL7/LqXcUACISsp544gkOHjzI+PHj/S6lXlIAiEhI2rRpE6+88gq/+tWv6Nq1sp8uiQJARELS/fffT5MmTXjiiSf8LqXe0kFgEQk57777LkuWLGH8+PE0b97c73LqLZ0GKiIhpbCwkG7dulFQUMDGjRvD8le/Og1URMLS5MmT2bhxI/PmzQvLL/+a0DEAEQkZ+/bt43e/+x29evXi5ptv9rucek8BICIh49FHH2Xv3r288MIL+tFXNSgARCQkrFu3jkmTJnH33XdzySWX+F1OUFAAiEjQKy4uZsyYMbRo0YLf//73fpcTNHQQWESC3tSpU1m9ejUzZswgJibG73KChvYARCSo7dmzh4ceeogrr7ySYcOG+V1OUFEAiEhQ++1vf0teXh4vvfSSDvzWkAJARILW6tWrSU1NZezYsVx00UV+lxN0FAAiEpSOHDnCnXfeSZs2bXS9n1Okg8AiEpT+8pe/8Nlnn/HPf/6Ts88+2+9ygpL2AEQk6Hz++ec8+eSTDB48mBtvvNHvcoKWAkBEgkpxcTF33XUXTZo04YUXXvC7nKCmISARCSqpqal89NFHTJs2jRYtWvhdTlCrcg/AzBLNLN3MMs1so5mN89rPMbOlZrbZe2zmtZuZvWBmW8zsUzO7tNxnjfD6bzazEXW3WiISirKzs3nwwQfp06cPw4cP97ucoFedIaBC4D7nXFegBzDGzC4AHgLed851Ad73ngP0B7p402hgEpQEBvA4cDnwE+Dx0tAQEalKcXExd9xxB8XFxaSmpuqc/1pQZQA453Y65z7x5n8AMoE2QDIw3es2HRjgzScDM1yJVUCMmbUCrgOWOuf2Ouf2AUuBfrW6NiISsiZNmsSyZct49tln6dChg9/lhIQaHQQ2s/ZAd2A10NI5txNKQgIoHYxrA2SXe1uO13aidhGRk9q8eTMPPvgg/fr146677vK7nJBR7QAwsybAPOBe59z3J+taSZs7SfuxyxltZhlmlpGbm1vd8kQkRBUVFXH77bcTFRXF5MmTNfRTi6oVAGbWgJIv/zTn3HyveZc3tIP3uNtrzwESy709AdhxkvYKnHOpzrkk51xSXFxcTdZFRELQ+PHjWbFiBS+++CJt2mjQoDZV5ywgA6YAmc65Z8u9tAgoPZNnBLCwXPtw72ygHsB+b4joXaCvmTXzDv729dpERCq1YcMGHnvsMQYNGsSQIUP8LifkVOd3AD2B24D/NbMNXtsjwNPAHDMbBWQBt3ivvQ1cD2wBDgEjAZxze83sSWCt1+8Pzrm9tbIWIhJyDh48SEpKCs2bN+fll1/W0E8dqDIAnHMfUfn4PcC1lfR3wJgTfNZUYGpNChSR8DRu3Di+/PJL3n//fZo3b+53OSFJl4IQkXpn9uzZTJkyhUceeYTevXv7XU7IUgCISL3y9ddfM3r0aK644goef/xxv8sJaQoAEak3jh49yi9+8QsAZs6cSYMGDXyuKLTpYnAiUm888MADrFq1itmzZ9O+fXu/ywl52gMQkXph9uzZPP/884wbN45bb73V73LCggJARHy3adMmRo0aRc+ePfnrX//qdzlhQwEgIr764YcfGDRoEGeeeSazZ8/WuH8A6RiAiPjGOceoUaPYvHkzy5Yt06UeAkwBICK+eeqpp3jzzTf5y1/+wjXXXON3OWFHQ0Ai4ot58+bxu9/9jmHDhnH//ff7XU5YUgCISMCtX7+e4cOH06NHD1599VVd58cnCgARCahvv/2Wm266idjYWBYsWEB0dLTfJYUtHQMQkYA5dOgQAwYMYO/evXz00UfEx8f7XVJYUwCISEAUFhYyZMgQ1qxZw7x58+jevbvfJYU9BYCI1DnnHHfffTeLFi1iwoQJ3HzzzX6XJOgYgIgEwJ/+9CdeeeUVfvOb33D33Xf7XY54FAAiUqemTZvGo48+ytChQ/nTn/7kdzlSjgJAROrMggULuPPOO7n22muZOnUqZ5yhr5z6RP8aIlInlixZwuDBg7nssstYsGABUVFRfpckx1AAiEit++CDDxg4cCA/+tGPWLJkCWeddZbfJUklFAAiUqtWrVrFjTfeSMeOHXnvvfeIiYnxuyQ5AQWAiNSalStX0q9fP+Lj41m2bBnNmzf3uyQ5iSoDwMymmtluM/usXNs5ZrbUzDZ7j828djOzF8xsi5l9amaXlnvPCK//ZjMbUTerIyJ++fDDD+nbty9xcXGkp6fTqlUrv0uSKlRnD2Aa0O+YtoeA951zXYD3vecA/YEu3jQamAQlgQE8DlwO/AR4vDQ0RCT4LVu2jH79+pGQkMC///1vEhMT/S5JqqHKAHDOfQjsPaY5GZjuzU8HBpRrn+FKrAJizKwVcB2w1Dm31zm3D1jK8aEiIkFoyZIl3HjjjXTq1IkPPviA1q1b+12SVNOpHgNo6ZzbCeA9tvDa2wDZ5frleG0naheRIJaWlsZNN93EBRdcQHp6Oi1btvS7JKmB2j4IXNlFvd1J2o//ALPRZpZhZhm5ubm1WpyI1J6//e1vDBs2jCuvvJL09HQd8A1CpxoAu7yhHbzH3V57DlB+8C8B2HGS9uM451Kdc0nOuaS4uLhTLE9E6kpxcTH33Xcf999/P7fccgvvvPMOTZs29bssOQWnGgCLgNIzeUYAC8u1D/fOBuoB7PeGiN4F+ppZM+/gb1+vTUSCSH5+PkOHDuXZZ5/lnnvuYdasWTRs2NDvsuQUVXk5aDN7A7gGaG5mOZSczfM0MMfMRgFZwC1e97eB64EtwCFgJIBzbq+ZPQms9fr9wTl37IFlEanHdu7cyYABA1i7di3PPPMMDzzwgG7lGOTMuUqH4uuFpKQkl5GR4XcZImFv3bp1JCcnk5eXR1paGsnJyX6XJCdhZuucc0lV9dMvgUXkpObMmcNVV11FREQEK1as0Jd/CFEAiEiljhw5wrhx4xg8eDCXXnopa9eu5eKLL/a7LKlFCgAROU5WVha9evXihRde4N5772X58uW0aNGi6jdKUNE9gUWkgiVLljBs2DAKCwuZO3cugwYN8rskqSPaAxARAA4dOsTYsWO5/vrrSUxMZN26dfryD3EKABEhIyODH//4x0yYMIFx48axcuVKOnfu7HdZUscUACJh7OjRozz55JNcccUVHDhwgKVLl/Lcc8/RqFEjv0uTANAxAJEwtWbNGkaPHs1//vMffvGLXzBx4kSaNdNV2sOJ9gBEwsz333/P2LFj6dGjB7m5ucyfP5+0tDR9+YchBYBImHDOMXfuXC644AImTpzImDFjyMzM5Oabb/a7NPGJAkAkDKxbt46rr76aW265hdjYWFauXMmECRM4++yz/S5NfKQAEAlhO3bs4Pbbb+eyyy7j888/55VXXuGTTz7h8ssv97s0qQdC8iDwwYMH+fLLL4mIiDjpFBkZSXR0NI0aNSIyMiT/FBKmdu/ezTPPPMNLL71EcXExDzzwAI888oiu2y8VhOS33saNG2u8hRMZGUmjRo1o3LgxjRo1qjB/1lln0bRpU5o2bUpMTEzZfPm2mJgY4uLiiI2NVZiIb/bs2cP48eOZMGEC+fn53HbbbTz22GN07NjR79KkHgrJb6ouXbrw1ltvUVRURFFREYWFhWXz5afCwkIKCgrIz88nPz+fQ4cOHTd/6NAhdu/ezebNm9m/fz/79+/nyJEjJ1y2mdGsWTPi4uIqnVq2bEnr1q3LpsaNGwfwLyOhKisri+eff57U1FQOHjzIkCFDePzxxzn33HP9Lk3qMd0P4BQUFBSQl5dXFgj79+9n37595ObmVph2795dNr9nzx6Ki4uP+6yYmBhat25NmzZtykKhdD4xMZF27drRvHlz3XhDKrVhwwbGjx/PrFmzAEhJSeHhhx/mwgsv9Lky8VN17wcQknsAdS06Opr4+Hji4+Or/Z6ioiL27dvHrl272LFjR9m0ffv2ssfMzEx27txJUVFRhfc2btyYtm3b0r59e9q1a0e7du0qzLdq1YozztDx/HCRn5/P3LlzSU1N5aOPPqJJkyaMGzeOcePG0bZtW7/LkyCiAAiQiIgImjdvTvPmzU+6dVZUVERubi7bt28nOzubbdu2lU3ffPMNa9euZc+ePRXe06BBA9q2bUu7du3o2LEjnTp1qjDpwF9o2LhxI6+++iozZsxg3759dOnShb/+9a/ceeedxMTE+F2eBCENAQWhAwcOkJWVxTfffFMhIL7++mu2bt1Kbm5uhf6xsbHHhUJpUGjvoX775ptvmDVrFm+88QaffvopDRo0YNCgQYwePZprrrlGQ4NSqeoOASkAQtD333/P1q1b+eqrr46bsrKyKhyLiI6OrnSvoWPHjrRv356GDRv6uCbh6Yvmre6sAAAG8klEQVQvvmDx4sXMmzePlStXAnDFFVcwZMgQUlJSiIuL87lCqe8UAFKpI0eOsG3bthMGRH5+fllfMyMhIaHCHkP5x3POOUdboLUgPz+fFStW8K9//YvFixezefNmAC6++GJSUlJISUmhQ4cOPlcpwUQBIDXmnOPbb7/lq6++YuvWrWUhUfq4a9euCv2bNm1aaTB06tSJxMRE/R7iBAoKCli9ejXp6el88MEHrFq1isOHDxMVFUXv3r35+c9/zg033ED79u39LlWCVL0NADPrBzwPRACTnXNPn6ivAqB+OXjwYFkwHBsOX3/9NUePHi3rGxkZWeGgdLt27UhMTCyb2rRpQ1RUlI9rExiFhYVs3LiRjIwM1q5dS0ZGBp9++ilHjx7ljDPOoHv37lxzzTVcffXV9O7dmyZNmvhdsoSAehkAZhYBfAn8DMgB1gJDnHObKuuvAAgeRUVFbN++vUIwlB9m2rt3b4X+ZkbLli1p27ZthWBITEwkPj6eFi1a0LJlS5o2bVrvh5mcc+zbt49t27bxxRdfkJmZyaZNm8jMzOTLL78sC8amTZuSlJREUlISPXv25KqrrtLZO1In6msAXAE84Zy7znv+MIBz7s+V9VcAhI4DBw6Qk5NDdnY2WVlZZGdnHzcdPHjwuPc1bNiwLAzKP5ZefqP00hzl55s0aUJ0dDQNGjQ4pfBwzlFQUMChQ4c4cOAAe/bs4bvvvqsw7dq1i6ysrLLpwIEDZe8/44wz6NixIxdccAFdu3bl4osv5rLLLqNTp04640oCor7+EKwNkF3ueQ6gyxKGgSZNmnD++edz/vnnV/q6c468vDyys7PZtWsXu3btYvfu3WXzu3btYufOnfznP/9h9+7dFYabTsTMiI6OrjBFRETgnCubSpddVFRUdumPQ4cOVfm5sbGxtGvXjvPOO4++ffvStm1b2rZty7nnnkuXLl2Ijo6u+R9JJMACHQCVbY5V2AUxs9HAaEC/agwjpddQqs5dqUq30PPy8sqm/fv3l80fPHiQgoICCgoKOHz4cNl8fn5+2SmwZlY2QclWe+PGjY+bzjzzTGJjY8t+xNe8eXOaNWtGREREnf49RAIh0AGQAySWe54A7CjfwTmXCqRCyRBQ4EqTYGFmZVdsbdWqld/liAStQA9IrgW6mFkHM4sCUoBFAa5BREQI8B6Ac67QzO4G3qXkNNCpzrmNgaxBRERKBPyXOs65t4G3A71cERGpSOekiYiEKQWAiEiYUgCIiIQpBYCISJhSAIiIhKl6fTloM8sFtp3GRzQHvqulcoJBuK0vaJ3Dhda5Zto556q8c1C9DoDTZWYZ1bkgUqgIt/UFrXO40DrXDQ0BiYiEKQWAiEiYCvUASPW7gAALt/UFrXO40DrXgZA+BiAiIicW6nsAIiJyAiEZAGbWz8y+MLMtZvaQ3/XUNTNLNLN0M8s0s41mNs7vmgLFzCLMbL2ZLfa7lkAwsxgzm2tmn3v/3lf4XVNdM7Nfe/+vPzOzN8ws5G63ZmZTzWy3mX1Wru0cM1tqZpu9x6rvllRDIRcA3o3nXwT6AxcAQ8zsAn+rqnOFwH3Oua5AD2BMGKxzqXFApt9FBNDzwDvOufOBSwjxdTezNsBYIMk59yNKLiOf4m9VdWIa0O+YtoeA951zXYD3vee1KuQCAPgJsMU5t9U5dwSYBST7XFOdcs7tdM594s3/QMmXQht/q6p7ZpYA3ABM9ruWQDCzs4FewBQA59wR51yev1UFRCTQyMwigcYccxfBUOCc+xDYe0xzMjDdm58ODKjt5YZiAFR24/mQ/zIsZWbtge7Aan8rCYjngAeBYr8LCZCOQC7wmjfsNdnMzvS7qLrknNsOjAeygJ3Afufce/5WFTAtnXM7oWQjD2hR2wsIxQCo8sbzocrMmgDzgHudc9/7XU9dMrMbgd3OuXV+1xJAkcClwCTnXHfgIHUwLFCfeOPeyUAHoDVwppkN87eq0BGKAVDljedDkZk1oOTLP805N9/vegKgJ3CTmX1DyTDfT83sH/6WVOdygBznXOne3VxKAiGU9QG+ds7lOueOAvOB/+dzTYGyy8xaAXiPu2t7AaEYAGF343kzM0rGhTOdc8/6XU8gOOceds4lOOfaU/JvvNw5F9Jbhs65b4FsMzvPa7oW2ORjSYGQBfQws8be//NrCfED3+UsAkZ48yOAhbW9gIDfE7iuhemN53sCtwH/a2YbvLZHvPsvS2i5B0jzNm62AiN9rqdOOedWm9lc4BNKznZbTwj+KtjM3gCuAZqbWQ7wOPA0MMfMRlEShLfU+nL1S2ARkfAUikNAIiJSDQoAEZEwpQAQEQlTCgARkTClABARCVMKABGRMKUAEBEJUwoAEZEw9f8BFTPbUTTmMx4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "i = np.linspace(0.0, 10.0, 1000)\n", "j = (20.0 - i**2)**2\n", "\n", "pl.plot(i, j, 'k-', label='$(20-z^2)^2$')\n", "pl.legend()\n", "pl.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The derivative\n", "Looks like there's a low point at about $4.5$.\n", "Let's take the derivative of the cost function with respect to $z$.\n", "\n", "$$\n", "\\begin{align}\n", "Cost(z) &= ( 20.0 - z^2 )^2 \\\\\n", "\\Rightarrow \\frac{\\partial Cost}{\\partial z} &= 2(20.0 - z^2)(-2z) \\\\\n", " &= 4z(z^2 - 20) \\\\\n", " &= 4z^3 - 80z\n", "\\end{align}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The derivative tells us what the slope of the tangent to the curve is at any point on the cost function.\n", "What does that mean?\n", "It means that if we pick a value of $z$, e.g. $8.0$, that the derivative tells us that a line going through the point $(8.0, (20.0 - (8.0)^2)^2)$ with the slope $4(8.0)^3 - 80(8.0)$ perfectly touches the graph above.\n", "Let's plot that.\n", "\n", "When you simply, the point $(8.0, (20.0 - (8.0)^2)^2)$ becomes $(8,1936)$.\n", "The slope is $4(8.0)^3 - 80(8.0)$ which when simplified becomes $1408$.\n", "So, the claim is that the line with slope $1408$ going through the point $(8,1936)$ touches the graph.\n", "To calculate the equation of the line, we'll use $(y - y_1) = m(x - x_1)$:\n", "\n", "$$\n", "y - 1936 = 1408(x - 8) \\\\\n", "\\Rightarrow y = 1408x - 11264 + 1936 \\\\\n", "\\Rightarrow y = 1408x - 9328\n", "$$\n", "\n", "Let's plot that line and the cost function together." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAD8CAYAAACPWyg8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd4VVXW+PHvIpRQZKiCgryAgoZEQYgM4gygItIkgA1eEX7CiCKKYnkRdAYRwa6UGRhBEQuKiAWEKBCHwQEBCUVpChhaKNIiLYS09ftjn2SSkAQCuTkp6/M858m5++x777oGs+4uZ29RVYwxxpi8KOV3AMYYY4oeSx7GGGPyzJKHMcaYPLPkYYwxJs8seRhjjMkzSx7GGGPyzJKHMcaYPLPkYYwxJs8seRhjjMmz0n4HECg1atTQ+vXr+x2GMcYUKatXrz6kqjXPVq/YJo/69esTHR3tdxjGGFOkiMjOc6ln3VbGGGPyzJKHMcaYPLPkYYwxJs+K7ZhHdpKSkoiNjSUhIcHvUEwOgoODqVu3LmXKlPE7FGNMLkpU8oiNjeWiiy6ifv36iIjf4ZgsVJXDhw8TGxtLgwYN/A7HGJOLEtVtlZCQQPXq1S1xFFIiQvXq1a1laEwRUKKSB2CJo5Cz348xRUOJSx7GGFNcnTp1iiFDhnDo0KGAv5clD2OMKQZUlYceeoiJEycWyA3SljxKqC+//JL777+fiIgIFi5ceN51jDGFw9SpU5k+fTp//etf6dixY+DfUFUDegBVgNnAz8Bm4HqgGrAI2Or9rOrVFWACsA34CWie4XX6efW3Av3O9r4tWrTQrDZt2nRGmR/i4+O1TZs2umPHDm3Xrp1eddVV2qRJEx03blymel9//bU2btxYL7/8cn3xxRcDEsuRI0e0f//+qqp6+vRp/fOf/6xJSUk51ikIheX3ZExR8cMPP2jZsmW1Q4cOmpycfEGvBUTrufxtP5dKF3IA7wF/8c7LesnkFeBpr+xp4GXvvDPwtZdEWgErvfJqQIz3s6p3XjW39y3MyePvf/+7jhs3Tvfu3aurV69WVdVjx45po0aNdOPGjaqqmpycrA0bNtRff/1VT58+rddcc036tfz0+OOPp8egqvrcc8/phx9+mGudQCssvydjioKDBw9qvXr1tF69enro0KELfr1zTR4B7bYSkcpAG+AdAFVNVNXfgQgvqaQll+7eeQTwvvcZVgBVROQS4FZgkaoeUdU4XGulANplgTFjxgwiIiK45JJLaN68OQAXXXQRISEh7NmzB4AffviBK664goYNG1K2bFl69erFnDlzzuv9brrpJpo1a0azZs0IDg7m008/RVUZNmwYnTp1So8BoHv37syYMQMgxzrGmMIhJSWFe+65h/379zN79myqV69eYO8d6JsEGwIHgXdFpCmwGngUqKWq+wBUdZ+IXOzVrwPszvD8WK8sp/JMRGQgMBCgXr16+ftJ8kliYiIxMTFkXS5+x44drF27lj/+8Y8A7Nmzh8suuyz9et26dVm5cuV5vee//vUvACZPnszixYvp2bMnEydOJCoqiqNHj7Jt2zYefPBBAMLCwli1ahVAjnWMMYXDqFGjWLhwIW+99RbXXXddgb53oJNHaaA58IiqrhSR8bhuqpxkN8lfcynPXKA6BZgCEB4efsb1jB577DHWrVuXW5U8a9asGePGjcu1zqFDh6hSpUqmshMnTnD77bczbtw4KleuDJDW5ZdJ1nsg2rdvz/79+8+oN2bMGCIiIjKVvf/++3z99dd89tlnBAUFMWTIEIYMGXLGc4OCgihbtizHjx/PsY4xxn+zZ89m9OjR3Hfffdx///0F/v6BTh6xQKyqpn1lno1LHr+JyCVeq+MS4ECG+pdleH5dYK9X3i5L+b8DGHfAlC9fPtMd1ElJSdx+++3cc8899OzZM728bt267N7938ZWbGwsl156aabXioqKOqf3/PTTT5kxYwZz5sw5pzWjTp8+TXBw8Dm9tjGm4K1bt45+/fpx/fXXM3nyZF9urg1o8lDV/SKyW0SuVNVfgJuBTd7RD3jJ+5nWmT8XeFhEZgJ/BI56CWYBMFZEqnr1OgDDLyS2s7UQAqVq1aqkpKSQkJBAuXLlGDBgACEhITz++OOZ6l133XVs3bqV7du3U6dOHWbOnMlHH32U5/ebN28ekyZNYt68eeeUEA4fPkzNmjVtYUJjCqkDBw4QERFBtWrV+PzzzylXrpwvcRTEwoiPADNEpCxultR9uPtLZonIAGAXcKdXNxI342obEO/VRVWPiMhoYJVX73lVPVIAsQdEhw4dWLp0KcHBwXzwwQdcffXVNGvWDICxY8fSuXNnSpcuzd///nduvfVWUlJS6N+/P6GhoXl+r379+lGtWjVuuOEGAB555BEGDBiQY/3FixfTuXPn8/tgxpiASkxM5Pbbb+fAgQMsXbqU2rVr+xfMuUzJKopHYZ6qu2bNGu3Tp4/fYWSrR48e+vPPP/saQ2H5PRlTmKSmpupf/vIXBfTjjz8O2PtQGKbqmuxde+213HjjjaSkpPgdSiaJiYl0796dK6+80u9QjDFZTJw4kbfffpsRI0bQq1cvv8MpWft5FCb9+/f3O4QzlC1blr59+/odhjEmi7lz5zJ06FAiIiIYPXq03+EAtraVMcYUaqtWraJXr160aNGCjz76iFKlCsef7cIRhTHGmDNs376drl27Urt2bb766isqVKjgd0jpLHkYY0whFBcXR+fOnUlKSiIyMpJatWr5HVImNuZhjDGFzOnTp+nevTsxMTEsWrSIq666yu+QzmDJwxhjCpGUlBT69u3Ld999x0cffUSbNm38Dilb1m1ljDGFhKoyePBgZs2axauvvkrv3r39DilHljyMMaaQ+Otf/8pbb73FsGHDePLJJ8/rNVJT8zmoHFjyMMaYQuDNN99kzJgx3H///bz44ot5fr4qfPEFhITA5s0BCDALSx7GGOOz9957j8cff5zbb7/9vFbJ3bgRbrkFevaEMmXg5MkABZqBJQ8f9O/fn4svvpiwsLAzrqWkpHDttdfStWvX9LJvvvmGK6+8kiuuuIKXXnopU/0333yT0NBQwsLC6N27d6bl3i/U+PHjCQsLIzQ0NNMqxAkJCbRs2ZKmTZsSGhrKyJEjAdi9ezc33ngjISEhhIaGMn78+AKL1Zii6osvvmDAgAG0b9+eGTNmEBQUdM7PjYuDIUOgaVNYswYmToR16yA8PIABpzmXBbCK4lGYF0ZcsmSJrl69WkNDQ8+49vrrr2vv3r21S5cuqpr7XuaxsbFav359jY+PV1XVO++8U9999918iXH9+vUaGhqqJ0+e1KSkJL355pt1y5YtquoWaDt+/LiqqiYmJmrLli11+fLlue7JnpdYC8vvyZhAmzt3rpYpU0ZbtWqV/v/UuUhOVp00SbV6ddVSpVQHDVI9eDB/YsIWRiy82rRpQ7Vq1c4oj42NZf78+fzlL39JLzvbXubJycmcOnWK5ORk4uPj0zeMym7f8rzYvHkzrVq1okKFCpQuXZq2bdvyxRdfAG5Hw0qVKgFuM6ukpCREJNc92XOL1ZiSKDIykjvuuINmzZrxzTffpP8/dTZLlkDz5vDQQ3D11bB2LUyaBDVqBDjgLErsfR6PPeaad/mpWTO4kD2mHnvsMV555RWOHz+eXpbbXuZ16tThySefpF69epQvX54OHTrQoUMHIPt9y/MiLCyMZ555hsOHD1O+fHkiIyMJz9AWTklJoUWLFmzbto3Bgwen772eJuue7LnFakxJs2DBAnr27ElYWBgLFy7kD3/4w1mfs3MnPPUUfPop1Kvnft5+O/iwiSBgYx6Fxrx587j44otp0aJFpnLXiswsbTAtLi6OOXPmsH37dvbu3cvJkyf58MMP0+ul7VuetR+1ffv2hIWFnXFkbNGEhIQwbNgwbrnlFjp27EjTpk0pXfq/3zWCgoJYt24dsbGx/PDDD2zYsCH9WnZ7sp8tVmNKim+//Zbu3bsTEhLCokWLqFKlSq714+Nh5Ei46iqYNw9GjYKff4Y77vAvcUABtDxEJAiIBvaoalcRaQDMBKoBa4B7VTVRRMoB7wMtgMPA3aq6w3uN4cAAIAUYoqoLLjQun3ahzdGyZcuYO3cukZGRJCQkcOzYMfr06cPgwYNz3Ms8KiqKBg0aULNmTQB69uzJ999/T58+fXLdt/xc9z4fMGBA+q6DI0aMoG7dumfUqVKlCu3ateObb74hLCwsxz3Zc4vVmJLi22+/5bbbbqNRo0YsWrQo2+7rNKowa5ZrbezeDXffDa+84lodhcK5DIxcyAE8DnwEzPMezwJ6eef/BAZ55w8B//TOewGfeOdNgB+BckAD4Fcg6GzvW5gHzFVVt2/fnu2Auarq4sWL0wfMk5KStEGDBhoTE5M+YL5hwwZVVV2xYoU2adJET548qampqdq3b1+dMGGCfvXVV9quXTs9ceLEBcX422+/qarqzp079corr9QjR46oquqBAwc0Li5OVVXj4+P1T3/6k3711Veampqq9957rz766KNnvFZOsWanMP2ejMkvX331lZYrV07DwsLS/9/Kydq1qm3aqIJqs2aq331XQEHquQ+YBzpx1AW+BW4C5gECHAJKe9evBxZ45wuA673z0l49AYYDwzO8Znq93I7CnDx69eqltWvX1tKlS2udOnX07bffznQ9Y/JQVZ0/f742atRIGzZsqC+88EKmun/729/0yiuv1NDQUO3Tp48mJCRotWrV9IorrtCmTZtq06ZNz3j9c/WnP/1JQ0JC9JprrtGoqKj08h9//FGbNWumV199tYaGhuqoUaNUVfU///mPAnr11Venv/f8+fNzjTU7heX3ZEx+mTVrlpYuXVrDw8P10KFDOdY7cED1gQfcDKoaNVTfesvNrCpIhSV5zMZ1Q7XzkkcNYFuG65cBG7zzDUDdDNd+9er/HeiTofwd4I4c3m8grossul69emf8R7E/SkWD/Z5McTJ9+nQtVaqU/ulPf9Lff/892zqJiarjxqlWqaIaFKT66KOqXkO/wJ1r8gjYgLmIdAUOqOrqjMXZVNWzXMvtOZkLVaeoariqhqf1rRtjjF8mTZrE//t//4+bbrqJb775JttZVYsWuZmajz0G110HP/3kxmSrVvUh4DwI5GyrG4BuIrIDN0B+EzAOqCIiaQP1dYG93nksriWCd/0PwJGM5dk8xxhjCh1V5fnnn2fw4MHcdtttfPXVV1SsWDFTnV9/he7doUMHSEiAL7+EBQugSROfgs6jgCUPVR2uqnVVtT5uAPxfqnoPsBi4w6vWD0ibHzrXe4x3/V9eE2ou0EtEynkztRoBP1xAXOf7VFMA7PdjirqUlBQGDRrEyJEj6du3L5999hnBwcHp10+cgBEjXJKIioIXX4RNmyAiwt+pt3nlx02Cw4CZIvICsBY3hoH38wMR2YZrcfQCUNWNIjIL2AQkA4NVNeV83jg4OJjDhw9TvXr1PC88ZgJPVTl8+HCm/9GMKUpOnTpF7969mTNnDk8//TRjx45N/1uTmgozZsCwYbBvH9x7L7z0EhTVhRakuH7TCw8P1+jo6ExlSUlJxMbG2oJ8hVhwcDB169Y9494UYwq7I0eOcNttt7F8+XImTJjAww8/nH5t1Sq3gOGKFW5cY8IEaNXKx2BzISKrVfWsSyuWqOVJypQpQ4MGDfwOwxhTzPz666907dqVmJgYZs2axR13uJ75/ftdF9W770KtWu5n375Qqhis7VGikocxxuS37777jp49e5KamsrChQtp27YtiYkwfjyMHu0Gw596Cp59FrzVeoqFYpD/jDHGH++++y7t27enevXqrFy5krZt2zJ/PoSFwf/9H7RpAxs2uGVFilPiAEsexhiTZykpKTz11FP079+ftm3bsmLFClJTG9G5M3Tt6rqlIiPdQoaNG/sdbWBY8jDGmDw4evQoPXr04LXXXuOhhx7i448jeeGFqoSFwbJl8Prr7ka/Tp38jjSwbMzDGGPO0fr16+nZsyc7duxg/PiJVKz4MKGhcPAg9O8PY8a4gfGSwJKHMcacg48++oj777+fypUrM2FCNO+805TVq6F1a9dFlWUrnmLPuq2MMSYXiYmJDBkyhHvuuYfQ0A60bh3DQw81Zf9+d9Pf0qUlL3GAtTyMMSZHu3btonfv3nz//Wpat57PunWd+Okn4Zln4Omn4Ry3HS+WrOVhjDHZ+Oyzz7jmmqasWfM/XHzxIb7/vjO33ips2gQvvFCyEwdYy8MYYzKJj49n6NChTJmylMqVF5CQ0JLLL4ePPoKbb/Y7usLDWh7GGOP56aefuPbam5gyJRSRnwgKuo6JE2HdOkscWVnLwxhT4qWkpPDGG+MZMWI7KSmRlCpVlQceEJ5/HmrU8Du6wsmShzGmRNuyZQs9e45n48aBQFNat05k0iShaVO/IyvcrNvKGFMipaam8re/vUNIyHo2bvwH1atfzqxZytKlZS1xnANreRhjSpyfftrGbbf9h127/pdSpUrx5JPHeP75ypQv73dkRUdAWx4icpmILBaRzSKyUUQe9cqricgiEdnq/azqlYuITBCRbSLyk4g0z/Ba/bz6W0WkX07vaYwxOUlIOM2dd86madNy7Np1Hy1b7icmpiyvvmqJI68C3W2VDDyhqiFAK2CwiDQBnga+VdVGwLfeY4BOuD3KGwEDgcngkg0wEvgj0BIYmZZwjDHmXEyduopq1X5k9uw7qFIlhS++OMTKlQ34n/+xLanPR0CTh6ruU9U13vlxYDNQB4gA3vOqvQd0984jgPfVWQFUEZFLgFuBRap6RFXjgEVAx0DGbowpHjZvPkjjxv9i4MDmnD59OUOGbOTQofp0727TqC5EgQ2Yi0h94FpgJVBLVfeBSzDAxV61OsDuDE+L9cpyKjfGmGydOHGa7t2/pUmTMmzd2oaWLVeye3cFxo8PJSjI7+iKvgJJHiJSCfgMeExVj+VWNZsyzaU86/sMFJFoEYk+ePDg+QVrjCnSVJVRo76nWrVdzJlzMzVqxPDVV7tYubI1l15qAxv5JeDJQ0TK4BLHDFX93Cv+zeuOwvt5wCuPBS7L8PS6wN5cyjNR1SmqGq6q4TVr1szfD2KMKfTmzdvMxRd/z3PPtQbKMXLkGg4caE7Xrg39Dq3YCfRsKwHeATar6hsZLs0F0mZM9QPmZCjv6826agUc9bq1FgAdRKSqN1DewSszxhjWrdtGaOgcbrutIYcONaVbt+XExV3Kc881R2w8PCACfZ/HDcC9wHoRWeeVjQBeAmaJyABgF3Cndy0S6AxsA+KB+wBU9YiIjAZWefWeV9UjAY7dGFPI7dy5m379FrJkSScggrCwNXzySUOaNLne79CKPVE9Y+igWAgPD9fo6Gi/wzDGBMC+ffsYOvQjZs26AdVW1Kq1k2nTLqJz52p+h1bkichqVQ0/Wz27w9wYU2Rs376d5577Jx9+2ITU1CcoX/4oo0Yd4okn/odStthSgbLkYYwp9DZu3MiYMa8yc+bFqD5LqVIVuP/+OF57rSqVK/sdXclkycMYUyipKosXL2b8+PHMnZuMyDhUG3HzzQlMmlSaxo1tkQk/WfIwxhQq8fHxzJgxgwkTJrBhQyJly/4DaE/DhilMnAidOgX7HaLBkocxppD49ddfmTp1KlOnTuXIkWRq1pxIUFAfgoOFF1+Ehx8OomxZv6M0aSx5GGN8c+rUKT7//HPeeecdFi9ejEgQzZtPRHUAhw6VoX9/YcwYqFXL70hNVpY8jDEFSlVZu3Yt06ZNY8aMGfz+++80bNiQgQOns2JFb1avLkvr1jBhArRo4Xe0JieWPIwxBWLLli18/PHHzJw5k59//ply5cpxxx130K3bIObMac2UKcKll8KHH8L//i92Z3ghZ8nDGBMwO3fuZNasWXz88cesXbsWEaFt27Y89thjdOt2F+++W5X+/SE5GZ55Bp5+GipV8jtqcy4seRhj8o2qsmbNGubOncvcuXNZt86tStSyZUveeOMN7rrrLi69tA5ffgk33ADbt0OPHvDaa9DQ1i4sUix5GGMuyMmTJ1myZAnz5s1j7ty57Nmzh1KlSnHDDTfw6quv0qNHDy6//HIANm6Efv3g228hNBSiouDmm33+AOa8WPIwxuRJcnIy0dHRREVFERUVxffff09SUhIVK1bk1ltvpVu3bnTp0oUaNf67U19cHIwcCZMmQeXKMHEiPPgglLa/QEWW/eqMMbk6ffo0a9asYdmyZSxdupR///vfHD16FIBrr72WoUOH0r59e/785z8THJz5Br6UFJgyBf76V5dAHngAnn8eatgOsEWeJY8sjh49yqlTpxARSpUqdc6H2NQQU0z89ttv/PDDDyxbtoxly5axatUqTp8+DcDll1/OnXfeSfv27bnpppvIbdO1JUtgyBD46Sdo2xbGj4emTQvqU5hAs+SRxYgRI5g0adJ5PbdUqVKUKVOGsmXL5njkdr18+fJUqFDhjJ/ZlWVXp0KFCpSypUXNOVJVdu3axdq1a1mzZk36sW/fPgDKlClD8+bNGTx4MDfccAOtW7emdu3aZ33dnTvhqafg00+hXj2YNQvuuMOm3hY3ljyy6NWrF1dffTWpqamoKqmpqed8pKSkkJSURFJSEomJibkep0+f5sSJE5kenzp1ivj4eE6dOpX+TS8vRISKFSty0UUXUalSJS666KI8n6f9rFy5MpUqVbJkVAykpqaya9cuNm/enOnYtGkTcXFxgPvi06RJE9q3b0/z5s1p0aIF4eHhlC9/7nt+x8fDK6/Ayy+7RDFqlEsieXgJU4QUmc2gRKQjMB4IAt5W1Zdyq1/UN4NKSUkhISGB+Pj49ISS28+TJ09y4sQJjh8/nv4zp/OTJ0+ecxwVK1akcuXK6Ukm43nWx2c7L22jowFz8uRJdu7cyfbt29m+fTs7duxg+/btxMTEsGXLFuLj49Pr1qhRg5CQEK666iquvfZamjdvztVXX02FChXO671VXSvjySdh9264+26XROrVy69PZwpSsdoMSkSCgH8AtwCxwCoRmauqm/yNLHCCgoKoWLEiFStWzPfXTk1N5eTJk7kmmuPHj3Ps2LFsz3fu3Jnp8bm2koKDg/OccHJKWOXKlcv3/y6FTWpqKseOHePIkSP89ttv7Nu3L/3Yv39/+vmePXs4cOBApucGBwdTv359GjRoQLt27QgJCUk/auTjaPW6dfDoo/Ddd9Csmbs7vE2bfHt5U4gVieQBtAS2qWoMgIjMBCKAYps8AqlUqVLpf4jzQ2Ji4lkTTk7X9u3bxy+//JJenvEbcm7KlClz1oRTqVIlgoODKVeuXI5H1utly5Y9YyJETpMkUlNTSU5OPuNISUlJP09KSsrUOkxrSWY8P3HiBHFxcenHkSNHiIuL4/fffyc1NTXb31+tWrWoXbs2l1xyCS1atEhPFGlHrVq1AjqJ49AhePZZmDoVqlWDt96CAQMgKChgb2kKmaKSPOoAuzM8jgX+6FMsJouyZctSvXp1qlevfsGvlZycnKkFlFMyyi4xHT58mB07dqRfO3HiBIW9WzZtokPVqlWpVq0a1atX54orrqBatWpUrVo1vTxjsqhZsyZBPv2VTkqCyZPdPRvHj8Mjj7jzqrYvU4lTVJJHdl+hzvirICIDgYEA9azDtUgqXbo0VapUoUqVKhf8WqpKUlISp0+fzvZISEg4oywxMfGMiRLZTZxISUkhKCiI0qVLpx9ZH6cd5cuXp2LFiumJIu28fPnyRWqKd1SU66LatAluuQXGjYMmTfyOyvilqCSPWOCyDI/rAnuzVlLVKcAUcAPmBROaKaxEJH0adH510ZVEv/4KTzwBc+a49ae+/BK6dbOptyVdUZmHuQpoJCINRKQs0AuY63NMxhRrJ07AiBGudREVBS++6FodERGWOEwRaXmoarKIPAwswE3VnaaqG30Oy5hiSRVmzIBhw2DvXrj3XnjpJbj0Ur8jM4VJkUgeAKoaCUT6HYcxxdmqVW5cY/lyuO46mD0brr/e76hMYVRUuq2MMQG0fz/07w8tW0JMDEybBitWWOIwOSsyLQ9jTP5LTHR7hT//PCQkuOVEnn3WLZtuTG4seRhTQs2fD0OHwtat0KULvPEGNG7sd1SmqLBuK2NKmF9+gc6doWtXN2sqMhLmzbPEYfLGkocxJcTRo27xwrAwWLbM7Ru+fj106uR3ZKYosm4rY4q51FSYPh2GD4eDB93A+JgxUKuW35GZosyShzHF2Pffu938Vq+G1q1dF1WLFn5HZYoD67Yyphjaswf69IEbboB9+9xS6UuXWuIw+cdaHsYUIwkJbtbU2LGQnOyWFxk+HCpV8jsyU9xY8jCmGFB1Cxc+8YS7ya9HDzcg3rCh35GZ4sq6rYwp4jZudEuk9+jh9guPioLPP7fEYQLLkocxRVRcnBsMb9rUDYhPmOC2hb35Zr8jMyWBdVsZU8SkpLjtX5991iWQBx5wy4vk49bkxpyVtTyMKUKWLIHmzWHQIHez35o1MGmSJQ5T8Cx5GFME7NwJd90F7drB77/DrFmweLHrsjLGD9ZtZUwhFh8Pr7wCL7/s1qEaNcotMVKhgt+RmZIuYC0PEXlVRH4WkZ9E5AsRqZLh2nAR2SYiv4jIrRnKO3pl20Tk6QzlDURkpYhsFZFPvK1ojSm2VF3r4qqrXMKIiICff4a//c0ShykcAtlttQgIU9VrgC3AcAARaYLbgzwU6AhMEpEgEQkC/gF0ApoAvb26AC8Db6pqIyAOGBDAuI3x1bp1rnvq7ruhWjU3zjFzJtSr53dkxvxXwJKHqi5U1WTv4QqgrnceAcxU1dOquh3YBrT0jm2qGqOqicBMIEJEBLgJmO09/z2ge6DiNsYvhw7Bgw+6JUQ2boR//tNNwW3Txu/IjDlTQQ2Y9we+9s7rALszXIv1ynIqrw78niERpZUbUywkJbl7NBo1grffhkcecRs0PfAABAX5HZ0x2bugAXMRiQJqZ3PpGVWd49V5BkgGZqQ9LZv6SvaJTHOpn108A4GBAPWsjW+KgKgoePRR2LTJ3SU+bhw0aXL25xnjtwtKHqraPrfrItIP6ArcrKppf/BjgcsI82tkAAATZElEQVQyVKsL7PXOsys/BFQRkdJe6yNj/azxTAGmAISHh2ebYIwpDGJi4PHH3XpUDRvCl19Ct25uRpUxRUEgZ1t1BIYB3VQ1PsOluUAvESknIg2ARsAPwCqgkTezqixuUH2ul3QWA3d4z+8HzAlU3MYE0okTbqXbkBDX6hg71o1vRERY4jBFSyDv8/g7UA5Y5Ma8WaGqD6rqRhGZBWzCdWcNVtUUABF5GFgABAHTVHWj91rDgJki8gKwFngngHEbk+9UYcYMGDYM9u6Fe++Fl16CSy/1OzJjzo/8tzepeAkPD9fo6Gi/wzCG6Gi3gOHy5RAe7gbHr7/e76iMyZ6IrFbV8LPVs+VJjAmQ/fvdfuEtW7oxjmnTYOVKSxymeLDlSYzJZ4mJrnXx/PNuZ78nn3Qr4Fau7HdkxuQfSx7G5KPISBg6FLZsgS5d3JawjRv7HZUx+c+6rYzJB7/84pJFly7ucWQkzJtnicMUX5Y8jLkAR4+6bqmwMFi61O0bvn49dOrkd2TGBJZ1WxlzHlJTYfp0GD4cDh50A+NjxkCtWn5HZkzBsORhTB4tX+6m3kZHu5lT8+e7KbjGlCTWbWXMOdqzx93c17q1u9Hvww9h2TJLHKZkspaHMWeRkOBmTY0dC8nJbnmR4cOhUiW/IzPGP5Y8jMmBqlu48Ikn3E1+3bvD66+7hQyNKems28qYbGzcCB06QI8eUL48LFoEX3xhicOYNJY8jMkgLs4Nhjdt6gbEJ0xw28K2z3XzAWNKHuu2MgZISYGpU90yInFxMHAgjB4NNWr4HZkxhZO1PEyJt2SJ2zd80CB3s9+aNTB5siUOY3JjycOUWDt3wl13Qbt2rrUxaxYsXuy6rIwxubNuK1PixMfDK6/Ayy+73fueew6eegoqVPA7MmOKjoC3PETkSRFREanhPRYRmSAi20TkJxFpnqFuPxHZ6h39MpS3EJH13nMmiNiGnSbvVF3rIiQERo1yW7/+/DOMHGmJw5i8CmjyEJHLgFuAXRmKO+H2LW8EDAQme3WrASOBPwItgZEiUtV7zmSvbtrzOgYyblP8rFvnuqfuvhuqVnXjHDNnQr16fkdmTNEU6JbHm8D/ARn3uo0A3ldnBVBFRC4BbgUWqeoRVY0DFgEdvWuVVXW5uj1z3we6BzhuU0wcOgQPPugGxDduhH/+E1avhjZt/I7MmKItYMlDRLoBe1T1xyyX6gC7MzyO9cpyK4/NptyYHCUluXs0GjWCt9+Ghx+GrVvhgQcgKMjv6Iwp+i5owFxEooDa2Vx6BhgBdMjuadmU6XmUZxfPQFz3FvWsP6LEioqCRx+FTZvczX3jxkFoqN9RGVO8XFDLQ1Xbq2pY1gOIARoAP4rIDqAusEZEauNaDpdleJm6wN6zlNfNpjy7eKaoariqhtesWfNCPpopgtLWn7rlFreY4ZdfwsKFljiMCYSAdFup6npVvVhV66tqfVwCaK6q+4G5QF9v1lUr4Kiq7gMWAB1EpKo3UN4BWOBdOy4irbxZVn2BOYGI2xRNJ064lW5DQlyrY+xYN74REeGm4hpj8p8f93lEAp2BbUA8cB+Aqh4RkdHAKq/e86p6xDsfBEwHygNfe4cp4VRhxgwYNsztr3HvvfDii1DHRsSMCbgCSR5e6yPtXIHBOdSbBkzLpjwaCAtUfKboiY52CxguX+42Y5o92+3qZ4wpGLY8iSlSfvsNBgyAli3dGMe0abBypSUOYwqaJQ9TJCQmwmuvuam3H3zgNmjasgXuuw9K2b9iYwqcrW1lCr3ISBg61CWLLl3clrCNG/sdlTElm31nM4VWWrLo0sU9joyEefMscRhTGFjyMIXO0aPw5JNub42lS1131fr10KmT35EZY9JYt5UpNFJTYfp0GD4cDh6E/v1hzBioVcvvyIwxWVnyMIXC8uVu6m10tJs5NX++m4JrjCmcrNvK+Crt5r7Wrd35hx/CsmWWOIwp7KzlYXyRkOBmTY0dC8nJbnmR4cOhUiW/IzPGnAtLHqZAqcKcOe4+jbSFDF9/HRo29DsyY0xeWLeVKTCbNkGHDtCjB5QvD4sWwRdfWOIwpiiy5GECLi7O7a9xzTVuQHzCBLctbPv2fkdmjDlf1m1lAiYlBaZOhWefdQlk4EAYPRpq1PA7MmPMhbKWhwmI775z+4YPGuQ2Y1qzBiZPtsRhTHFhycPkq5074e67oW1b19qYNQv+/W9o2tTvyIwx+cm6rUy+iI+HV16Bl192u/c99xw89RRUqOB3ZMaYQAhoy0NEHhGRX0Rko4i8kqF8uIhs867dmqG8o1e2TUSezlDeQERWishWEflERMoGMm5z7lRd6yIkBEaNgm7d4OefYeRISxzGFGcBSx4iciMQAVyjqqHAa155E6AXEAp0BCaJSJCIBAH/ADoBTYDeXl2Al4E3VbUREAcMCFTc5tz9+CPceKPrpqpaFZYsgU8+gXr1/I7MGBNogWx5DAJeUtXTAKp6wCuPAGaq6mlV3Y7by7yld2xT1RhVTQRmAhEiIsBNwGzv+e8B3QMYtzmLQ4fgwQeheXPYsAH++U9YvRratPE7MmNMQQlk8mgM/NnrbloiItd55XWA3RnqxXplOZVXB35X1eQs5aaAJSW5ezQaNYK334aHH4atW+GBByAoyO/ojDEF6YIGzEUkCqidzaVnvNeuCrQCrgNmiUhDQLKpr2SfyDSX+tnFMxAYCFDP+k7yVVSUu9Fv0yZ3c9+4cW4KrjGmZLqg5KGqOd4jLCKDgM9VVYEfRCQVqIFrOVyWoWpdYK93nl35IaCKiJT2Wh8Z62eNZwowBSA8PDzbBGPyJibGrUP15ZduGZEvv3SD4pJdSjfGlBiB7Lb6EjdWgYg0BsriEsFcoJeIlBORBkAj4AdgFdDIm1lVFjeoPtdLPouBO7zX7QfMCWDcBjhxwq10GxLi1qAaOxY2boSICEscxpjA3ucxDZgmIhuARKCflwg2isgsYBOQDAxW1RQAEXkYWAAEAdNUdaP3WsOAmSLyArAWeCeAcZdoqjBjBgwb5vbX6NMHXnoJ6tgokzEmA3F/z4uf8PBwjY6O9juMIiU62u3mt3y524xpwgS3q58xpuQQkdWqetbt2Gx5EsNvv8GAAdCypRvjmDYNVq60xGGMyZkljxIsMRFee81Nvf3gAzcwvmUL3HcflLJ/GcaYXNjaViVUZCQMHeqSRZcubkvYxo39jsoYU1TY98sSJi1ZdOniHkdGwrx5ljiMMXljyaOEOHbMrXIbFgb/+Y/rrlq/Hjp18jsyY0xRZN1WxVxqKkyfDsOHw8GDbjxj7FioVcvvyIwxRZklj2Js+XI39TY62s2cmj/fTcE1xpgLZd1WxdDevXDvvdC6tTv/8ENYtswShzEm/1jLoxhJSIA334QxY9wKuCNGuO6qSpX8jswYU9xY8igGVGHOHHefRkwMdO8Or7/uFjI0xphAsG6rIm7TJujQAXr0gPLl3SKGX3xhicMYE1iWPIqouDi3v8Y117gB8QkTYN06t9eGMcYEmnVbFTEpKTB1Kjz7rEsgAwfC6NFQo4bfkRljShJreRQh330HLVrAoEFuF781a2DyZEscxpiCZ8mjCNi1C+6+G9q2hSNH4JNP4N//hqZN/Y7MGFNSWbdVIRYfD6++Ci+/7B4/95xbYqRCBV/DMsYYSx6FkSp8+qlLFLt2wV13uSRSr57fkRljjBOwbisRaSYiK0RknYhEi0hLr1xEZIKIbBORn0SkeYbn9BORrd7RL0N5CxFZ7z1ngkjx3UX7xx/hxhtdN1XVqq576pNPLHEYYwqXQI55vAKMUtVmwN+8xwCdgEbeMRCYDCAi1YCRwB+BlsBIEanqPWeyVzfteR0DGLcvDh1yA+HNm8OGDW4gfPVqN85hjDGFTSCThwKVvfM/AHu98wjgfXVWAFVE5BLgVmCRqh5R1ThgEdDRu1ZZVZer23D9faB7AOMuUElJMHGi281v6lR4+GHYuhUefBCCgvyOzhhjshfIMY/HgAUi8houSbX2yusAuzPUi/XKciuPzab8DCIyENdCoV4R6OeJinI3+m3a5G7uGzfOTcE1xpjC7oJaHiISJSIbsjkigEHAUFW9DBgKvJP2tGxeSs+j/MxC1SmqGq6q4TVr1sz7ByogMTFuOZFbboFTp9xyIgsXWuIwxhQdF9TyUNUcF8MQkfeBR72HnwJve+exwGUZqtbFdWnFAu2ylP/bK6+bTf0i58QJePFFt2hh6dJuU6ahQyE42O/IjDEmbwI55rEXSBvuvQnY6p3PBfp6s65aAUdVdR+wAOggIlW9gfIOwALv2nERaeXNsuoLzAlg3PlO1e2pceWVLmHceSf88otbLt0ShzGmKArkmMf9wHgRKQ0k4I1FAJFAZ2AbEA/cB6CqR0RkNLDKq/e8qh7xzgcB04HywNfeUSRER7vd/JYvd5sxzZ7tdvUzxpiiTNwEpuInPDxco6OjfXv/335zmzG9+y5cfLHrrurXD0rZgjDGmEJMRFar6ln3HbU/ZfksMRFee81Nvf3gA7dB05YtcN99ljiMMcWHLU+SjyIj3QD4li3QubPbErZxY7+jMsaY/GffhfPBli3QpYs7AObPd4clDmNMcWXJ4wIcO+YWLwwLg//8x3VXrV/vWh3GGFOcWbfVeUhNhenT3VTbgwfdeMbYsVCrlt+RGWNMwbDkkUfLl7upt9HRbsrt/PluCq4xxpQk1m11jvbuhXvvhdat3fkHH8CyZZY4jDElk7U8ziIhwc2aGjPGrYA7YoTrrqpUye/IjDHGP5Y8cqAKc+a4+zRiYqB7d7cmVcOGfkdmjDH+s26rbGzaBLfe6la+DQ6GRYvcyreWOIwxxrHkkcULL8A118CqVTBhgtsWtn2OawcbY0zJZN1WWTRoAPffD6NHQ40afkdjjDGFkyWPLO65xx3GGGNyZt1Wxhhj8syShzHGmDyz5GGMMSbPLih5iMidIrJRRFJFJDzLteEisk1EfhGRWzOUd/TKtonI0xnKG4jIShHZKiKfiEhZr7yc93ibd73+hcRsjDHmwl1oy2MD0BP4LmOhiDQBegGhQEdgkogEiUgQ8A+gE9AE6O3VBXgZeFNVGwFxwACvfAAQp6pXAG969YwxxvjogpKHqm5W1V+yuRQBzFTV06q6HbdfeUvv2KaqMaqaCMwEIkREgJuA2d7z3wO6Z3it97zz2cDNXn1jjDE+CdSYRx1gd4bHsV5ZTuXVgd9VNTlLeabX8q4f9eqfQUQGiki0iEQfPHgwnz6KMcaYrM56n4eIRAG1s7n0jKrOyelp2ZQp2ScrzaV+bq91ZqHqFGAKQHh4eLZ1jDHGXLizJg9VPZ/FOWKByzI8rgvs9c6zKz8EVBGR0l7rImP9tNeKFZHSwB+AI2cLYPXq1YdEZOd5xA5Qw4upJLHPXDLYZy4ZLuQz/8+5VArUHeZzgY9E5A3gUqAR8AOuFdFIRBoAe3CD6v+rqioii4E7cOMg/YA5GV6rH7Dcu/4vVT1rq0JVa55v8CISraolaqcO+8wlg33mkqEgPvOFTtXtISKxwPXAfBFZAKCqG4FZwCbgG2CwqqZ4rYqHgQXAZmCWVxdgGPC4iGzDjWm845W/A1T3yh8H0qf3GmOM8Yecw5f4Ese+qZQM9plLBvvMgWF3mGdvit8B+MA+c8lgn7lkCPhntpaHMcaYPLOWhzHGmDyz5JFFTmtvFVcicpmILBaRzd46ZY/6HVNB8JbLWSsi8/yOpSCISBURmS0iP3u/6+v9jinQRGSo9296g4h8LCLBfseU30RkmogcEJENGcqqicgib53ARSJSNRDvbckjg7OsvVVcJQNPqGoI0AoYXAI+M8CjuBl/JcV44BtVvQpoSjH/7CJSBxgChKtqGBCEuzWguJmOWz8wo6eBb711Ar8lQDNULXlklu3aWz7HFFCquk9V13jnx3F/VOrk/qyiTUTqAl2At/2OpSCISGWgDd70d1VNVNXf/Y2qQJQGyns3F1fgvzceFxuq+h1n3jSdcT3AjOsE5itLHpnltPZWieAtd38tsNLfSAJuHPB/QKrfgRSQhsBB4F2vq+5tEanod1CBpKp7gNeAXcA+4KiqLvQ3qgJTS1X3gftyCFwciDex5JHZOa+jVdyISCXgM+AxVT3mdzyBIiJdgQOqutrvWApQaaA5MFlVrwVOUsxvtvX6+SOABrhVLiqKSB9/oypeLHlkltuaXMWWiJTBJY4Zqvq53/EE2A1ANxHZgeuWvElEPvQ3pICLBWJVNa1FORuXTIqz9sB2VT2oqknA50Brn2MqKL+JyCUA3s8DgXgTSx6ZrcJbe8vbybAXbm2tYsvbG+UdYLOqvuF3PIGmqsNVta6q1sf9fv+lqsX6G6mq7gd2i8iVXtHNuKWDirNdQCsRqeD9G7+ZYj5JIIO09QAh8zqB+SpQCyMWSaqaLCJpa28FAdMyrL1VXN0A3AusF5F1XtkIVY30MSaT/x4BZnhfimKA+3yOJ6BUdaWIzAbW4GYUrqUY3mkuIh8D7YAa3jqDI4GXgFkiMgCXRO8MyHvbHebGGGPyyrqtjDHG5JklD2OMMXlmycMYY0yeWfIwxhiTZ5Y8jDHG5JklD2OMMXlmycMYY0yeWfIwxhiTZ/8fTcTJEavZa3IAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "i = np.linspace(0.0, 10.0, 1000)\n", "j = (20.0 - i**2)**2\n", "k = 1408 * i - 9328\n", "\n", "pl.plot(i, j, 'k-', label='$(20-z^2)^2$')\n", "pl.plot(i, k, 'b-', label='$1408z - 9328$')\n", "pl.legend()\n", "pl.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Why do we care about the slope?\n", "It's a bit hard to see, but the blue line is perfectly touching the curve.\n", "We care about this because the slope of the blue line tells us in which way to change $z$ in order to make the cost less.\n", "If we increase $z$ the cost goes up.\n", "If we decrease it the cost goes down." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Gradient descent\n", "Let's use the gradient descent algorithm to calculate the best $z$.\n", "We'll start with the guess $z=8$, and then use the derivative to move $z$ ever so slightly in the direction that decreases the cost.\n", "By ever so slightly, we mean $0.001$ times the slope:\n", "\n", "$$\n", "\\begin{align}\n", "z_{i+1} &= z_i - \\eta \\frac{\\partial Cost}{\\partial z} \\\\\n", " &= z_i - (0.001) (4 z_i^3 - 80 z_i)\n", "\\end{align}\n", "$$\n", "\n", "So, for our initial guess $z_0 = 8.0$ we get:\n", "\n", "$$\n", "\\begin{align}\n", "z_1 &= 8.0 - (0.001) (4 (8.0)^3 - 80 (8.0)) \\\\\n", " &= 8.0 - 1.408 = 6.592\n", "\\end{align}\n", "$$\n", "\n", "Let's code it up." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Current: 8.00000000\tNext: 6.59200000\n", "Current: 6.59200000\tNext: 5.97355269\n", "Current: 5.97355269\tNext: 5.59881186\n", "Current: 5.59881186\tNext: 5.34469983\n", "Current: 5.34469983\tNext: 5.16157297\n", "Current: 5.16157297\tNext: 5.02444369\n", "Current: 5.02444369\tNext: 4.91903017\n", "Current: 4.91903017\tNext: 4.83645229\n", "Current: 4.83645229\tNext: 4.77084541\n", "Current: 4.77084541\tNext: 4.71815685\n", "Current: 4.71815685\tNext: 4.67548576\n", "Current: 4.67548576\tNext: 4.64069702\n", "Current: 4.64069702\tNext: 4.61218330\n", "Current: 4.61218330\tNext: 4.58871218\n", "Current: 4.58871218\tNext: 4.56932433\n", "Current: 4.56932433\tNext: 4.55326362\n", "Current: 4.55326362\tNext: 4.53992784\n", "Current: 4.53992784\tNext: 4.52883326\n", "Current: 4.52883326\tNext: 4.51958845\n", "Current: 4.51958845\tNext: 4.51187478\n", "Current: 4.51187478\tNext: 4.50543157\n", "Current: 4.50543157\tNext: 4.50004463\n", "Current: 4.50004463\tNext: 4.49553736\n", "Current: 4.49553736\tNext: 4.49176369\n", "Current: 4.49176369\tNext: 4.48860255\n", "Current: 4.48860255\tNext: 4.48595333\n", "Current: 4.48595333\tNext: 4.48373229\n", "Current: 4.48373229\tNext: 4.48186965\n", "Current: 4.48186965\tNext: 4.48030717\n", "Current: 4.48030717\tNext: 4.47899619\n", "Current: 4.47899619\tNext: 4.47789603\n", "Square root: 4.477896027970839 \tSquared: 20.05155283731702\n" ] } ], "source": [ "def next_z(z, x, eta=0.001):\n", " return z - eta * (4.0 * z**3 - 80 * z)\n", "\n", "def sqrt_grad_desc(x, z, verbose=False):\n", " while abs(z - next_z(z, x)) > 0.001:\n", " if verbose:\n", " print(\"Current: %14.8f\\tNext: %14.8f\" % (z, next_z(z, x)))\n", " z = next_z(z, x)\n", " return z\n", "\n", "ans =sqrt_grad_desc(20.0, 8.0, True)\n", "print(\"Square root:\", ans, \"\\tSquared:\", ans**2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A question\n", "Let's try some other initial guesses: 4.0, 1.0 and -1.0.\n", "Can you explain the square root returned with -1.0?" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "With initial guess 4.00: 4.465951\n", "With initial guess 1.00: 4.465963\n", "With initial guess -1.00: -4.465963\n" ] } ], "source": [ "print(\"With initial guess %6.2f: %10.6f\" % (4.0, sqrt_grad_desc(20.0, 4.0, False)))\n", "print(\"With initial guess %6.2f: %10.6f\" % (1.0, sqrt_grad_desc(20.0, 1.0, False)))\n", "print(\"With initial guess %6.2f: %10.6f\" % (-1.0, sqrt_grad_desc(20.0, -1.0, False)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.7" } }, "nbformat": 4, "nbformat_minor": 2 }