{
 "metadata": {
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%autosave 10"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "javascript": [
        "IPython.notebook.set_autosave_interval(10000)"
       ],
       "metadata": {},
       "output_type": "display_data"
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Autosaving every 10 seconds\n"
       ]
      }
     ],
     "prompt_number": 1
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%matplotlib inline\n",
      "from matplotlib import pyplot"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Linear Regression With One Variable"
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Problem"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Suppose you observe $m$ data points $(x^{(i)}, y^{(i)})$ and you develop the hypothesis that these data points were generated by the following model:\n",
      "\n",
      "$$h_{\\theta}(x) = <\\theta,x> = \\theta_0 + \\theta_1 x_1$$\n",
      "\n",
      "A mathematically tractable measure of how well your model reproduces the observed data is the following cost function\n",
      "\n",
      "$$J(\\theta) = \\frac{1}{2m} \\sum_{i=1}^m \\left( h_\\theta(x^{(i)}) - y^{(i)} \\right)^2$$\n",
      "\n",
      "We now wish to choose the *best* hypothesis from our pool of hypotheses $h_{\\theta}$ as measured by our cost function $J$.\n",
      "To do so we need to modify parameter values $\\theta$ so that $J$ assumes its global minimum.\n",
      "\n",
      "Given the fact that we chose $J$ as a quadratic function we know that there must be one unique minimum\n",
      "(this type of function has a special name ...)."
     ]
    },
    {
     "cell_type": "heading",
     "level": 2,
     "metadata": {},
     "source": [
      "Gradient Descent"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "One way to approximate the minimum of $J$ in $\\theta$-space numerically is to start in a random selected point in $\\theta$-space\n",
      "and then take small successive steps that always move us in the direction of greatest change (the gradient).\n",
      "\n",
      "The gradient of $J$ (with respect to $\\theta$) is\n",
      "\n",
      "\\begin{equation}\n",
      "\\nabla J = \\frac{1}{m} \\sum_{i=1}^m\n",
      "\\begin{pmatrix}\n",
      "\\left( h_{\\theta}(x^{(i)}) - y^{(i)} \\right) \\\\\n",
      "\\left( h_{\\theta}(x^{(i)}) - y^{(i)} \\right) x_1^{(i)}\n",
      "\\end{pmatrix}.\n",
      "\\end{equation}\n",
      "\n",
      "This now allows us to take small successive steps in the direction of $\\nabla J$ and continue doing so until $\\nabla J = (0, 0)$\n",
      "or is numerically close to the zero vector:\n",
      "\n",
      "$$\\theta^{k+1} = \\theta^k - \\alpha \\nabla J.$$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import csv"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 4
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data = []\n",
      "with open('data/ex1data1.txt', 'r') as f:\n",
      "    data_reader = csv.reader(f, delimiter=',')\n",
      "    for row in data_reader:\n",
      "        data.append((float(row[0]), float(row[1])))\n",
      "data = numpy.asarray(data)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 5
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data[:5]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 6,
       "text": [
        "array([[  6.1101,  17.592 ],\n",
        "       [  5.5277,   9.1302],\n",
        "       [  8.5186,  13.662 ],\n",
        "       [  7.0032,  11.854 ],\n",
        "       [  5.8598,   6.8233]])"
       ]
      }
     ],
     "prompt_number": 6
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "pyplot.scatter(data[:,0], data[:,1])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 7,
       "text": [
        "<matplotlib.collections.PathCollection at 0x44e4650>"
       ]
      },
      {
       "metadata": {},
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAD9CAYAAABOd5eOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcVPX6wPHPMAwwA6ICigqKaWaC4K6paVgSgZl75lI3\nl3Zz69deV25dtcxMbb+5tKCZllY3t8xC09Qo9+tyS81cUkBwGQccZub5/QGhXpfYB4bn/Xr5ujNn\nzvl+n3Pu9Mzhe76LQUQEpZRSlZqXuwNQSilVcprMlVLKA2gyV0opD6DJXCmlPIAmc6WU8gCazJVS\nygMUKpnn5OTQrl07WrVqxXXXXce4ceMAyMzMJC4ujpiYGOLj4zl58mSZBquUUuryDIXtZ56dnY3Z\nbMbhcHDjjTcyefJkFi9eTOPGjRk7dizTp0/nwIEDzJgxo6xjVkop9T8K3cxiNpsBsNvtOJ1Oateu\nzbJly7j77rsBGDp0KEuXLi2bKJVSSl1VoZO5y+WiZcuWhIaG0q1bN6KiokhPTyc4OBiAkJAQ0tLS\nyixQpZRSV+Zd2B29vLzYunUrp06dIj4+nu+++65QxxkMhmIHp5RSVVlRZlspcm+W6tWr06NHDzZt\n2kStWrXIyMgAID09ndq1a18xIP0nTJgwwe0xVJR/ei30Wui1uPq/oipUMj9x4gRnzpwB8h6Erlq1\niujoaBITE0lOTgYgOTmZxMTEIgeglFKq5ArVzHL06FHuueceRIScnBwGDx5Mjx496NixIwMHDmTO\nnDnUqVOHhQsXlnW8SimlLqNQyTw6OpotW7Zcsj0oKIhVq1aVelCeKjY21t0hVBh6Lc7Ta3GeXovi\nK3Q/82JXYDAUq/1HKaWqsqLmTh3Or5RSHkCTuVJKeQBN5kop5QE0mSullAfQZK6UUh5Ak7lSSnkA\nTeZKKeUBNJkrpZQH0GSulFIeQJO5Ukp5AE3mSinlATSZK6WUB9BkrpRSHkCTuVJKeQBN5kop5QE0\nmSullAfQZK6UUh5Ak7lSSnkATeZKKeUBNJkrpZQH0GSulFIeQJO5Ukp5gEIl80OHDtG1a1eio6Np\n2rQpU6ZMASApKYnw8HBatWpFq1atWLFiRZkGq5RS6vIMIiJ/tdPx48dJT0+nefPmWK1WWrduzaJF\ni/j888+pVq0a48ePv3IFBgOFqEIppdQFipo7vQuzU2hoKKGhoQAEBAQQExPDkSNHADRRK6VUBVCo\nZH6h3377jdTUVObOnUtqaipvvvkms2bNok2bNsycOZOgoKBLjklKSip4HRsbS2xsbEliVkopj5OS\nkkJKSkqxjy9UM8ufrFYr3bp149lnn6V3795kZGQQHBwM5CXsffv2kZycfHEF2syilFJFVtTcWehk\nnpuby+23385tt93GuHHjLvn86NGjdOvWjb1795YoIKWUUkXPnYXqzSIijBgxgsjIyIsSeVpaWsHr\nzz77jKioqCKEqpRS7rNr1y66d+9NZGRHxo59knPnzrk7pBIp1J35unXr6Nq1KzExMRgMBgAmTZrE\n/Pnz2b59O3a7nYiICGbPnk1YWNjFFeiduVKqgvnjjz9o1qw1p08/jUhrzOaXuf32IBYu/MDdoRUo\ns2aW4tJkrpSqaN5//31GjVrB2bML8rdYMRqDOXfOhtFodGtsfyqTZhallPIkJpMJg8F6wRYrXl7G\ngpaHykiTuVKqyunZsyc1avyCyTQKmIPFksDYsePx8qq8KVGbWZRSVVJGRgYTJ77CoUPHSEi4ieHD\nh1WoO3NtM1dKKQ+gbeZKKVUFaTJXSikPoMlcKVVqcnJy2LNnDydOnHB3KFWOJnOlVKnYunUr4eFN\naNeuJ2FhjZgyZZq7Q6pS9AGoUqpUhIdfx5EjScBg4DAWS0dSUhbTrl07N0dWOekDUKVUucvJyeGP\nPw4Ag/K3hGMw3MyOHTvcGVaVoslcKVVivr6+1KgRCnydvyULWM+1117rxqiqFk3mSqkSMxgMLF48\nj4CAu6levQtmcyQjR/aja9eu7g6tytA2c6VUqcnIyGDHjh3UqVOHZs2aFasMm83Gq69OZ8+eA3Tu\n3JoHH3ygUg+zLy4dAaqUqrQcDgcdO3Zn584QcnK6Y7HMo0+f60lOfs/doZU7TeZKqUpr3bp1JCQ8\njNW6lbxWYCs+PmEcObKPkJAQd4dXrrQ3i1Kq0srJycHLqzrnU5MFo9Gv0q8CVB40mSulKowOHTpg\nNh/BaJwM/ISPz8M0a9aUevXquTu0Ck+TuVKqwqhWrRobN35Lly7rqVatF0FBa7jjjnhcLpe7Q6vw\ntM1cKVWh2Gw2oqM7cPjwjdjtXbBY/kWfPk2q3ENQfQCqlKrUvvrqKwYPfoUzZ1IAA3AGb+9QTp5M\nx9/f383RlR99AKqUqtScTifgS14iB/ABDPnb1ZVoMldKVSixsbFYLL9iNCYBq/HzG0RcXAKBgYHu\nDq1C02YWpVSFc+jQIcaMeZoDBw5z000dmDw5CbPZ7O6wylWZtJkfOnSIIUOGkJWVhd1uZ8SIETzx\nxBNkZmYycOBAjh8/Tt26dfnkk0+oUaNGiQJSSilVRsn8+PHjpKen07x5c6xWK61bt2bRokXMmjWL\nxo0bM3bsWKZPn86BAweYMWNGiQJSSilVTr1Z+vfvz/Dhw3n00Uf58ccfCQ4OJiMjgxtuuIFff/21\nRAEppZQqeu70LmoFv/32G6mpqcyZM4f09HSCg4MBCAkJIS0t7bLHJCUlFbyOjY0lNja2qNUqpZRH\nS0lJISUlpdjHF+nO3Gq1Ehsby3PPPUfv3r0JDAzk9OnTBZ//73vQO3OllCqOMutnnpubS79+/Rgy\nZAi9e/cGoFatWmRkZACQnp5O7dq1ixiuUkqp0lCoZC4ijBgxgsjISMaNG1ewPTExkeTkZACSk5NJ\nTEwsmyiVUkpdVaGaWdatW0fXrl2JiYnBYMgblTV58mTat29f0DWxTp06LFy4ULsmKqVUKdC5WZRS\nygPo3CxKKVUFaTJXSikPoMlcKaU8gCZzpZTyAJrMlVIeQUQ4cuTIFUeiezpN5kqpSu/MmTN07nwr\n117bkvr1r6N//7txOBzuDqtcaTKvBLRrp1JXN3bs02zeHEZOzjHs9qMsX/4Hr702091hlStN5hXY\nqVOnuO22fvj4mKlePZQ5c953d0hKVUgbN27m3LkRgBGwYLPdzfr1m90dVrnSZF6B3XPPQ3z3XSAO\nRwanT6/i0Uef4/vvv3d3WEpVOE2aXIPR+HX+Oxe+vqto1qyRW2MqbzoCtAILDAzlzJktQD0ADIZn\nmTDBlwkT/u7ewJSqYI4cOUKHDt04cyYEERsNG5r54YdVBAQEuDu0Yivz+cxV+alZM4QzZ3aQl8wF\nP7+d1KoV7+6wlLqijRs3smjREgICLDzwwH3Uq1evXOoNCwtj794tbNq0CW9vbzp27IjJZCqXuisK\nvTOvwFauXEnfvkNxufpjNO4jIuIkqakpWCwWd4em1CWWLl3KgAHDyc5+BG/vNAIDP2f79k2EhYW5\nO7RKSSfa8jC7du1i9erV1KhRg/79+1e5FcpV5dGsWQf27Pk70AMAo3EMTzwRyKRJL7o3sEpKm1k8\nTGRkJJGRke4OQ6m/dPbsWf58vgPgdNbj9Onj7guoitHeLEqpUjFoUB8sljHATmA1FssMBgzo5e6w\nqgxtZlFKlQqHw8EzzySRnLwIs9nCSy89y4AB/d0dVqWlbeZKKeUBdHEKpZSqgjSZK6WUB9BkrpRS\nHkCTuVKlxOl08sor04iP78+DD44hPT3d3SGpKkQfgCpVSoYNe4iFC3dhsz2MyfQDdeqsZNeunyr1\n/CDKfbQ3i1JukJOTQ0BADZzONCAQgGrVuvP++w/Tt29f9wanKqUy6c0yfPhwQkNDiY6OLtiWlJRE\neHg4rVq1olWrVqxYsaLo0SrlIc7/R3fh5E4+OJ1Od4SjqqBCJfNhw4ZdkqwNBgPjx49ny5YtbNmy\nhdtuu61MAlSqMjCbzdxxRz/M5juBVRiN/8Bs3kX37t3dHZqqIgqVzLt06ULNmjUv2a7NJ8pTOZ1O\n5s2bx4svvsiyZcsKdczHH89h1KgWtG49mV69fiU1de1l/7tRqiyUaKKtN998k1mzZtGmTRtmzpxJ\nUFDQZfdLSkoqeB0bG0tsbGxJqlWqTIkIvXoNIiXlMDZbLBbLeEaP/pFJk5Kuepyvry9TpvyzfIJU\nHiclJYWUlJRiH1/oB6C//fYbPXv2ZMeOHQBkZGQQHBwM5CXrffv2kZycfGkF+gBUVTKbNm3illvu\n5uzZHYAvkIaPT2PS0g5TvXp1d4enqohyG84fEhKCwWDAYDDwwAMPkJqaWtyilKpQsrKy8PZuQF4i\nB6iFt3c1Tp8+7c6wlLqqYifztLS0gtefffYZUVFRpRKQUu7Wtm1b4D/APCAdo3EideqElNsSaEoV\nR6GaWQYNGsSaNWvIyMggNDSUf/zjH3z33Xds374du91OREQEs2fPvuzyUNrMoiqjzZs3M3jw/Rw6\ntJ/o6NYsXDiHBg0auDssVYXooCGllPIAOgWuKjUiwtSprxESEkFQUDhPPfV3XC6Xu8NSSl2GJnN1\nRcnJ85gw4V+cOPEVWVnf8frrK3nlldfcHVapO3v2LEOH3kdwcAMaN27JypUr3R2SUkWmyVxd0cKF\nS7HZngaigSbYbC+ycOFSd4dV6u6992E+++wUmZkp7N8/mb597y7ogqtUZaHJXF1RSEgNvLz2X7Bl\nH0FBntfPeunSL8jJeRNoBCSQmztI785VpVOiEaDKsz3//OMsWdIZm+0PRPzw9Z3PlClfuzusUmex\nBJKd/TtQCwCT6XcCA7WrrapctDeLuqojR44wf/58nE4n/fv359prr3V3SKXugw8+5KGHniYn5z58\nfPZSr94Otm3bQLVq1dwdmqrCtGuiUsWwZs0aVq5cRUhIECNHjiQwMNDdIakqTpO5Ukp5AO1nrpRS\nVZAmc1UqcnNz2bVrF4cPH3Z3KEpVSZrM1V/KyMjgyy+/ZPXq1Tgcjks+//3332nSpAUdOvTi2mtb\nMHz4w9q0dhUOh4OXXppKQsKdjB79f2RlZbk7JOUBtM3cw/zyyy+sW7eOoKAgevTogbd38XqfHj9+\nnGnTZvLf//7KqlXfYjS2weU6RlRUMGvWLMPX17dg3y5dEtiw4UaczmeBM/j7d+O99x5j0KBBpXRW\nnmXQoOF8+eVBbLaR+Pik0KDBRrZv34jZbHZ3aKoCKXLulDJWDlWofCtWrBCLJUT8/YdKQEB76dw5\nTux2e5HLycjIkNDQhuLt/YhApMDbAiLgELO5h8yYMeOi/WvWDBP4LX8fEXhBnnjiqdI6LY9y6tQp\nMZn8Baz518ol1ap1lOXLl7s7NFXBFDV3ajOLB7n33kew2T7m7NmPsFp/YOtWOwsWLChyOfPnz+fU\nqc44HG8AOcDN+Z8Yyc7uwr59By/av3HjJhgMX+a/y8HffyXNmjUtyal4LKfTCRgAU/4WA+CXv12p\n4tNk7kEyM48B7fLfGbHbW3Ps2LEil5OdnY3DEZz/rh3wOuACMvD3n0+nTu0v2n/evHcJCXmVwMD2\nWCxN6d69AXfffXfxT8SD1axZk27duuPnNwRYjdH4dwICfqdr167uDk1Vctpm7kG6dk1kw4ZIHI6X\ngV+xWG5h1aqFdOrUqUjl7NmzhzZtumCzzQRqYzAMxWCw4uXl5NFHx/Dqq5MwGAwXHWO1Wtm2bRuB\ngYE0b978ks/VednZ2Tz11ATWrv2Rxo0jmD59IuHh4e4OS1UwOmioCktLS+P22wfy88/r8fHx4/XX\npzNy5PBilbV+/XpGj36WrKyT3HFHPI8/Pprq1asTEBBQylErpS5Hk7nCbrdjMpn07lipSkyTuVJK\neQAdzq+UUlWQJnOllPIAmsyVUsoD6EpDSgGZmZn88MMPWCwWunbtWuxpEJRyl0LdmQ8fPpzQ0FCi\no6MLtmVmZhIXF0dMTAzx8fGcPHmyzIJUqizt2bOHJk1iGDJkBr16jadTpzhycnLcHZZSRVKoZD5s\n2DBWrFhx0bYJEybQo0cPtm/fTkJCAhMmTCiTAJUqa8OHjyEr60lOn16F1bqZHTsCeeONN90dllJF\nUqhk3qVLF2rWrHnRtmXLlhUM2R46dChLly4t/eiUKgcHDx5EpFv+Oy9ycmL55ZeDVz1GqYqm2A2D\n6enpBAfnzd8REhJCWlraFfdNSkoqeB0bG0tsbGxxq1Wq1N1wQzu++uot7PY3gFNYLPPo3Hm0u8NS\nVUxKSgopKSnFPr7Qg4Z+++03evbsyY4dOwAIDAzk9OnTBZ//7/uCCnTQkKrgMjMzufXWPuzcuQOX\n6xwjR97Pm29O0xG0yq2KmjuLfWdeq1YtMjIyCAkJIT09ndq1axe3qCrvz+lPjUajmyOpmoKCgkhN\nTSE9PR0/Pz8CAwPdHZJSRVbsfuaJiYkkJycDkJycTGJiYqkFVVU4HA5GjHgEPz9//Pz8uf/+0Tqv\ntZsYDAZq16592USenZ3N9OnTeeyxJ/nyyy8vc7RSFUBhVrC46667pG7dumIymSQ8PFzmzJkjJ06c\nkO7du0t0dLTExcVJVlZWqayWUZW88MJksVhuEsgSOCEWSxeZPPkVd4elLnDu3Dlp0aKT+Pn1Epgo\nFst18sILk90dlqoCipo7daItN+rcOZEffngI6Jm/ZQk33TSXlBS9+6sovvjiC4YOnYLVuo68VYGO\n4O19LTk5Vm0WU2VKJ9qqRMLDQzEafy54bzRupn79Om6MSP0vq9UKhJGXyAFCERHsdrsbo1LqUnpn\n7kYHDx6kbdsuZGe3xWAQzObNbN68XledqUAOHz5Ms2atsVpfAzrg4zOF9u1/5/vvV/zlsUqVhM5n\nXslkZGSwbNkyDAYDPXr0ICgoyN0hqf+RmprKiBFj+eOPo3Tp0pk5c96gRo0a7g5LeThN5kop5QG0\nzVxVSadPn2bw4JHUrx/JDTfEFQxuU6qq0Dtz5TanT59m+fLlOJ1Obr31VkJCQopdVmxsDzZurMW5\nc49hMGygWrW/s3fvVurU0QfKqnLSZhZVKaSlpdG69Y2cOtUY8MPX9yd+/HENjRo1KnJZNpuNwMAg\nnM4zgAmAatX68N57dzFw4MDSDVypcqLNLKpCyc7O5p57HiAoKJyIiOYFIyiTkiaTlpaI1bocq3UJ\nWVkPM3bss8Wqw2QykTeNSlb+FgHSMZvNpXEKSlUKmsxVmbrvvtEsWpROVtb3/P77DO666z5++ukn\nDh78g9zcdgX7uVztOXToj2LVYTKZGD/+cSyW7sB0fH0H0aBBLrfeemspnYVSFZ8m8womOzubceOe\nokOHW7nnngeuOrVwWTp27BhxcX2oVash7dvfzJ49e4pVzpdffklOzkzgGuAWzp27l+XLV3DrrTdi\nsbwJZAJnMZunERd3Y7HjfemlF5g162nuu28fzz/fgo0bV+Pn51fs8pSqdEo2e8BfK4cqPIbL5ZLu\n3e8QP79+AsvEZBonDRtGytmzZ8s1DofDIU2bthZv7ycFfhWD4Q0JDg6XkydPFrmsunWbCKwTEAER\nP7+BMmPGDHE6nfLgg2PE29tXjEYfGTDgHjl37twlx+fm5kpS0kRp3z5O+vQZIr/++mtpnKJSFV5R\nc6cm8wrk2LFj4utbU+BcfvJzSWDgDfLNN9+Uaxy//fabWCz1BFwFSbh69S5XjMPpdMqBAwfk6NGj\nl3z2yScLxWyuIwbDc+LrO0gaNLj+oh+F3NzcyybxPw0f/rBYLLECy8TLa6LUrFlPjh8/XvKTVKqC\nK2ru1GaWCsRgMOBwOABXwTa7PadUFkkQEaZOfY1rrmlBkyZt+OCDj3jlldfo1CmB3r2HsHfv3oJ9\n/f39cTiswKn8Lbk4nccJCAi4pNysrCzatOlKZGQnrrkmigED7rloGt877xzAN998xnPPeTFpUlu2\nb99I9erV2b17N9OmTaNDh5tp0CCKG2+8jV9//fWisl0uFx9+OAebbTGQgMv1DHb7jbpEoVKXUza/\nKeeVQxUeY+/eveLlVUMgUeBTgfsFAi66E7Xb7fLII49JaGhjadSohXz++eeFKnvmzDfF3z9aYIPA\nt+LtXVd8fJoIfCkGwxQJDAyVQ4cOyc6dOyUp6R/Srl0nMZubC0wWi6W73HJLT3E6nZeUO3DgMPHx\neVDAKXBWLJZYmT595lVj+eyzxWI2h4jBUFdgvMBu8fKaKrVrN5TTp08X7Od0OsVkMgukFfyF4O/f\nV+bOnVu4C6pUJVbU3KnJvAL5+uuvJTCwq8ALAr0Exoq/fwP573//W7DPww+PF7O5u8Buga/FYgmV\nH3744S/LbtUqVmBFQVKEWQL9Ct77+g6XsWPHisUSIl5ej4u390NisQTL8OH3yVtvvSV2u/2y5TZu\n3Fpg0wXlvit33TX8qrFUr14n/8eq/kVNOYGBN0hKSspF+44a9ZhYLDcIfCJG49NSu3aEnDhxohBX\nU6nKrai5U5tZKpCoqCgcjv8ACcDnQHe8ve3Ur1+/YJ9PP/2c7OzXgeuBOLKzH+SLL776y7L9/S3A\nhT1j/gDO9/YwGFx88cVKbLapuFxTcDjeIifnQby9/XjooYcwmUyXLbdp08YYjcvz3znx81tJZGTj\nK8bhcrk4cyYdaAecAaz5n+TidGbg7+9/0f4zZkzhn/8cxC23LODuu7PYvHm9Tkam1GUUew1QVfrq\n1avHvHmzGTIkDhE/fH0NLF362UVd7Pz9A4Aj5CVzF0bjQQIDr/vLsidOfJKEhH7YbAcxGGyYTG/h\n5VWPnJxFeHntwc9vJSZTGBBRcIzLFUFm5vqrlvvOO6/SseMtnDmzHJfrNJGRdfi//xt3xf29vLxo\n2bIz27bNwum8E7gFuBM/vxV06HA9rVu3vmT/ceNGM27c6L88R6WqMh3OXwGdO3eO9PR0QkNDL7kj\nXrJkCUOHPozN1gb4BnDSpk0nVq5cTHBw8FXL/fnnn/nww48xmby5775hrFr1LUuWfE3t2kFMnPgs\n8+cvYvLkJdhs7wNWLJaBfPjhK/Tr1/eq5Z49e5affvoJHx8f2rVrh7f31e8Rjhw5Qo8eA9m+fSMm\nky833dSNXr0SuP/++6/4F4BSVY3OzVJBnTp1imPHjtGgQYMSDzOfOXMmjz32Mg7HOqABJtNYunU7\nwsqVi0tUrsvl4plnknjvvbk4nXZ69IhjypSXCQsLK1G5V2K32/OH4pe8t45SnkbnZqmAPvggmTp1\nImjbtgd16lzD999/X6LysrJO4nTeS96oSiO5uc+wYcO6Esfp5eXF6NEPYjJ5k5vblSVLvIiMbMPu\n3btLXPbl+Pj4aCJXqpRoMi9j+/fv56GHxpGTswGr9VdOn/6Anj0HFGsNyblz36dRo5bMnDkXo3EJ\n8Gd/7lRq1apbsF9OTg67d+/mxIkTRa7jhRde5sSJ/thsi8jO/pAzZ55i3Ljni1TGf//7XxYvXszW\nrVuLXH9JOBwODhw4QFZW1l/vrJSH0WRexnbv3o2PTxugWf6WeHJzvfnjj6JNKrV48RIeeSSJAwdm\nkpk5HofjOBCMt3dzLJbhzJ37OpDXLh4Wdi0dOvQiLKwR06bNLFI9R46k43DEFLwXieHYsfRCHz97\n9vu0bHkjw4Z9QOfOt/Pssy8Uqf7iOnDgAI0bR9O8eVfq1InguefKp16lKoxS7Rh5GeVQRYW2a9cu\nMZtDBY7m96f+Sfz8akh2dnaRyrnlll4CswW2CYQI/Ftgj5hMidK79yARyZvbJTS0ocDC/LoOisVS\nT37++edC1/Puu++JxdJK4LDACbFYbpGnn55Q8Pn69evlpptul1atYuW1114Xl8tV8Nnp06fF1zdQ\nYE9+/WliNteRXbt2Felci6NVqy7i5TUlv97j4u/fRFauXFnm9SpVVoqaO0ucaSMiIiQ6Olpatmwp\n7dq1K3FAnuhvfxspUE2gg0CgBAQEX3Yek6sJCKgr8E+BlwTGXTBI54gEBISISF4y9fY2X/CZSEDA\nIPnggw9EJC/Zz537gSQk3Cn33HP/ZSetcrlc8tRTz4uvb4CYTGYZNuyhggFDW7duFYslRGCuwAqx\nWFrIxIkvFxz7yy+/iL9/w4vqr179ZlmxYkVxL12h+foGCGQV1Ovt/ZhMnjy5zOtVqqyUezJv2LDh\nVUfkaTIXadasg8A7At8IHBFv71HyzDPPFfp4u90uYBSoJdBdoM8FCfNHqVWroYjkJeLq1UMFVuff\nWccK+EhoaGNZu3atvPTSVLFYmgl8JF5eSVK9eh05dOjQZet0uVwX3XWLiDz++FMCz19Q908SHt6s\n4POcnBypUaOuwJL8zzeJxRIihw8fLsZVK5pGjWIEFuTXmy3+/m1l4cKFZV6vUmWlqLmzVNrMRbse\nXpXVegZoT94AmXo4HA04edJ60T5nzpxhz549WK3WS443Go2YTD7AQiAa+AEYAkzEbO7DlClJQF5X\npsWL5+PvPxAvr1bADcAxjh9/jYSEvrz00qvYbIuAobhcE8jOvoMFCxZcNmaDwXBJTxOTyRsvr+wL\ntmRjNJ7vU+7r68uKFUsIChqFn18tLJbbmD9/dpl1bbzQJ5/MJjBwDNWrd8PfP5KEhGb069evzOtV\nqqIo8QhQg8FAXFwcDoeD+++/n1GjRl2yT1JSUsHr2NhYYmNjS1ptpXLnnb15++1x2GzvAMexWKbT\nr99HBZ9/8cWXDB58L15ewbhcmSxY8AE9e95e8LmXlxcvvvgiSUnDyckZCbTAYlnGiBGh9O79ITff\nfHPBvjfffDObN68nKqoNLtckwAD0xMvrRuz2NYBPwb4uly87d+6kUaMWnD59kp49e/D229OuuKjD\n8OH38vrrnbBaayBSD4vlBZ577uKeLh06dCAt7SBpaWmEhISU2yCgtm3bsn//f9i8eTNBQUG0bt1a\nuz2qSiUlJYWUlJTiF1DSPwX+nNEvLS1NWrduLatWrSrRnwqeKDc3V8aMeUJq124kERHNZf78jws+\nS09PF4slSODH/CaCDeLvHyyZmZmXlPPvf/9bxo17XKZNm3bVBSvOnTsnJpNF4GB+mXaxWKLkuusi\nxWRqKfB3ZZDMAAAU9ElEQVSRwFPi51dT/PyCBFYK/CJ+fnfIPfc8cNVz2b17t/ztbw9Ir15D5NNP\nPyv+RVFKXVVRc2epjgCdPHkyAE8//XTBNh0BemVbt24lMbEff/zhBfxSsD0wsCXffjubNm3aFLvs\nqVOnM2HCdOz2/nh5fY/dvguwADYAvLyC8fW1kpPTB5H38o/6ncDAdpw6dbygnNmz5zJ58uu4XC7G\njBnJ6NGP6B2vUuWgyLmzJL8cZ8+eLbhDtFqt0rVrV/niiy9K9OtSVZw+fVpq1qwn8LpATYFf8u+i\n94rZXLNUVtP59ttv5dFHHxWDwSTwfwIzBK4v6PVhMLwuBkN4fr1fCtQQ8JaIiEjZtWuXfPLJQrFY\nrhFYI7BeLJZm8q9/zbpifS6XS7KyssThcJQ4dqWquqLmzhJl2v3790tMTIy0aNFCmjRpIs8//3yJ\nA/JUdrtd9u/fX7D4wsaNGyUwsPUFc4sHC7QWX98gmTVrbpHL/+GHH+SWW3pLhw63yjvvvCcul0tO\nnToloaHXCCQIjBHwk7zFIP7sjXJCwFdMpt4C1SVv4QqXwLsSFtZE4uP75zfJ/Ln/F9Kx422XrX/P\nnj0SEdFMTKYA8fMLlAULPin+xVJKFTl3lugB6DXXXMO2bdtKUkSVsG3bNuLi7sBmExyOk0yb9grx\n8d2x2w8BJ4ERQFtMpi6kpHzNDTfccNXyzp49i8PhoHr16kBec0337ndgs00GQtmx40lycnJwuXLJ\nymoDLAK+A2YDy4F/AAEYDItp0iSKDh2q8cknHbDb/6z3fk6ceAYfHyMXz4F+nIAAyyXxiAjx8X34\n/fdRiDxMbu42hg2Lo2XLFjRt2rQkl04pVVhl85tyXjlUUaG5XC6pW7exQHL+3e0+sVjqyJYtW2Tw\n4GFiNl8nvr6PiL9/Exk//umrluV0OmXYsIfE29ssJpO/dO9+h1itVhkz5v8Eki64g/5BGjaMkWee\neU7g7/nbugvMF3hIoI5Ac/H1DZIdO3bIhg0bxN+/kcCZ/H33iK9vgKSmpoq/f4gYDM8JJInFEnLZ\nVY1OnjwpJpP/RYOFqlW7U+bNm1dWl1Upj1fU3Klzs5Qxq9VKevpR8vqFAzTCYLiRfv3u4fPPV+Ny\nnSI4eCn33nsr110XwdGjR69Y1htvvMUnn2zH4ThGbm4m69aZeeyxZ/H2NmIw5F6wpx2j0Uhc3C2Y\nze8C68hb1acO8BawBkigZ8/bad68OR06dGDAgHj8/dsQEDAEi+Um3n77Ddq2bUtq6lrGjbMzZoyV\n9etX0bFjx0viCggIyJ/D/M+/0my4XFvLpX+5UipfGf2oFCiHKio0l8slgYG1BNbm37Vmibd3LfHx\nuUvAkd99sKZ4e/cSk2mQ+PrWkKlTp14y+lJEpHfvoflD6f+8A14rkZEdZdeuXeLvHyLwisCHYrFc\nI5MnvyzXXNNcTKaaAiYxGCxiNEYKpAqsFoulvixduvSiOL/99lt5//33Zfv27UU+zwULPhGLpZZU\nq3an+PtfJ0OH3nfZc1BKFU5Rc6cm83KwYsUK8fcPkerVbxGLJUzq1o0U+Dw/IT+S39PkzwQ9VaCG\nxMf3FJG8ppUXXpgsTZq0ldq1m4i3d7z8uQiy0ThJEhMHiIjItm3b5M4775WEhDtl4cJF0qJFZ/Hy\neil/39/FbK4vI0c+JPXrR0njxq0kObn0m0D27Nkj8+bNkzVr1mgiV6qEipo7daWhcnL06FG2b99O\nWFgY//rXB7z3Xjrnzs0FWgBPcb4Z5pv893tZvfoL1qz5galT/43N9hrwBwbDMMzmhhiNdfHz28Om\nTSlcc801l9Tn7e2L05lFXt9y8PF5lJdfbszYsWPL43SVUiVU1NypCzqXA7vdzquvvsGyZd9Sp04t\nJk16hnXrHmfv3kZkZ6cDrwI3AX7Ac0As0I5Vq1bx0UefY7PNB1oBIPIfbr11J/feO4SbbrqJGjVq\nXLbO0NAIjh79DugBnMNk2kCDBt3K/mSVUm6hybwcjBz5KJ9+epDs7Kns3buD+PjebN++ifXr13P/\n/S9js/UAIslbOSgQmInB8AwREf3x9fUDzq+cYzRmERV1Pb169bpqnR9/PIsePfrj5dUOp/NXundv\nQ+/evcvwLJVS7qTNLGVMRPD19Sc39zAQBIDZfC/Tpt3A0KFDadiwGSdO/BPoC8wDngAaUr/+OXbv\n3syXX/6bkSP/D5ttMLAWH5/dfPnlQuLj4/+y7qNHj5KamkpISAidOnXSYfhKVSK6oHMFZDSagPNT\n23p5ncHHx4eAgAC+/XYp1147HaOxFn5+f8fbuw6+vo05cSKTtWvXMmjQXUye/ATe3u8CCeTmjqNf\nv6H8/PPPf1lvvXr16NWrF507d9ZErpSH02RexgwGA0888TgWSw9gFibTGAIDt9KnTx8AYmJi+OWX\nLSxa9DFGYwMcjp2cO7cEm+1T7r77fgBWrFiHw/EyMAGRJM6efZ4XX5zmvpNSSlU42mZeDpKSnsXl\nsjN9ehLnzp0hPLwlb731FgEBAfTt25f69etz5MgRnM62nJ9vvCNZWX/gcrk4c8YG1LqgxNpYrdmX\nVqSUqrK0zbwcHD16lKZNW2K1vgmEA4lAIkajD2bzUn766XsyMjLo1i2R3Nx25CV0JyEhe0lNTWHN\nmrU8/PA/sdlmA04slhHMmjWRQYPucudpKaXKUFFzpybzcvDpp58yYkQyp09/Tt6kWg2BP1foeZku\nXVLIzc3lxx8NuFzxwAfAOSCHwEA727ZtZPnylbz66rv5zTYPc999I9xzMkqpcqH9zCug6tWr43Id\nJK/rYSZw2wWfRvLzz2/g5VUTl2sLYCQv4dcHojhzpi5vv/0vXn55Eg899ED5B6+UqhT0AWg5uPnm\nm2nVqg4Wy63k3XH/HdgPHAT+gdkMXl61yUvkANXJG0BUCxE/rNZscnJyGDlyFKGhjWnatC1ff/21\nW85FKVUxaTNLOcnNzeXDDz9k2bLlfPHFtzidXoALb29/nnvuAWbOfIfMzCeB7sDbwFIgHThHs2Yt\nuP76a1m+3EpOzhTgFyyW4WzYsJqYmBg3npVSqqxom3kl8PrrbzF58nScTieDB/elfv26WK1WPv/8\nG3bv3oPd7sDlygE+Am7GaHwTl2sKIrvIe4AK3t7jefHF2jz11FOXlH/w4EGWLFmC0WhkwIAB1KlT\np1zPTylVcprMK5HFi5cwYMDfcLkigf/i5+fFTz+tYf/+/QwePA2r9TrgCNAJeAlYQF5PGPDzG8gr\nr3Rh1KhRF5W5Y8cOOne+hXPn+uDldQ6L5Rs2b15PRERE+Z6cUqpENJlXErm5uQQEBGO3rwbaARlA\ncyIignn33WncdtsA4GGgIzAT2ITZHEh29r2YTL9Su/ZWdu788ZKJtm67rT9ff30TIo8C4OX1PH/7\n2wnmzHmrXM9PKVUy2pulAtuzZw8//vgjdevWpVmzZtjtLvK6KNbI/992HDz4DbNnf4DR2Bqn86X8\nI2/BYAiiffsYvv9+JiIm6tZtgZfXpc+v09MzETm/7qbL1ZTjx78qh7NTSrmT9mYpJ4sWfUrr1l15\n5JGV9Okzlqio9kAEMAq4gbxpb1MAfxYt+gIfH59LyvjxRxsu13Ecjgx27GjE6NFPXrJPnz7xWCwv\nAIfIe1D6Mn37/vWkXEqpSq54a2AUXjlUUeG5XC6xWGoIbBawC3wm4C/wywUrDI0QiBb4j8A7Av5i\nND4l8IWYzd2lfv1mArMvWrTZbA6T4cMflmPHjhXU5XA4ZMyYx8XfP1iqVastL7wwSVf9UaoSKmru\n1DbzcmCz2QgMrInT+Rnwt/yt54CfgT+bRP5G3jD+9wDw8+tDy5anMBrNdO/eidxcO9Om/UJOTjJ5\nf1BNAL7D27sdtWt/ye7dPxMYGFgq8X7zzTdMmfI2DoeTceNG0LNnz1IpVylVeEXOnSX99Vi+fLk0\nb95cmjVrJi+99FKJf1081TXXRAlUF9iQf2d9j0BjgU8EXhQIFqgp8IOASwICbpEFCxYUHH/mzBmJ\niekoAQEtBWIEGuUvBi1SrdqtsmjRolKJ89tvvxWzubbA+wLzxGyuJ59//nmplK2UKryi5s4SZdqc\nnBxp2LChHD58WHJzc6Vt27ayefPmEgXkqZKTk8XLq/UFzSQugQCBbgLD85tcpgh0FV/fQXLdda3k\n7NmzF5Vht9vl66+/Fi8vk8AfBWVVq5YoCxcuLJU477hjsMC7F8T5idx4Y2KplK2UKryi5s4SPQDd\ntGkTUVFRhIWF4e3tzcCBA1m6dGlJivRY7du3x9f3MHA8f8uvGAwm8lYWmg1ci8FwlubNbSQlteCn\nn9ZgsVguKsNkMhEXF0e/foMwm4cByzAa/47Fsou4uLhSiTNvEYsL/7Rz6cIWSlUCJeqaePjwYerX\nr1/wPjw8nJSUlEv2S0pKKngdGxtLbGxsSaqtlJo0acITT4zhlVda4+3dltzcjQwdOpDk5JFkZ0/A\nYMjEYnmTBQtSiIqKumpZycnvkZQ0idWrpxMRUY9XX117xYWdi2rcuPtYtepObDYj4IPZ/BRPPPGv\nUilbKXVlKSkpl82fhVWiB6Aff/wxa9eu5e233wZgwYIFpKSk8M4775yvQB+AXmTz5s3s2rWLdu3a\n0bRpU5YtW8bs2Z9gsfjyxBOPEh0d7e4Q+e6773jllXdwOp2MHTuChIQEd4ekVJVTroOGwsPDOXTo\nUMH7Q4cOXXSnri42b958Ro58CIfDQUTEtaxcuZjExEQSExPdHdpFunXrRrdu3dwdhlKqCErUZt6u\nXTt27tzJkSNHyM3NZeHChXoXdwU7d+7kvvvGkZOzHofDyv79fyMhob+7w1JKeYgSJXM/Pz/efvtt\n4uPjadGiBX379qV169alFZtHSU1NxcsrHmgOGBAZx759/yE7W9fyVEqVXInnZklISNC78UIICwsD\nNgM55C088TPgzYkTJwgPD3drbEqpyk/nZikncXFxdOrUmLwRnwOARERup2fPQYUuY8OGDdx6az86\ndUpgzpz39cGyUqqAzppYTgwGA/36JbJ2rZ1z53oDExFpxPbtZnJzczGZTFc9fsuWLXTvfgc22ySg\nFtu2PUlOTg4PP/xgucSvlKrY9M68HNWpUweTKQMYCFwHbMPfvwbe3n/9mzp79kfYbGOA+4De2Gyz\neO2198o2YKVUpaHJvBzdfvvtdOwYRkBARyyWEZjNCcye/XahRlgajV4YDI4Ltjh0ZKZSqoDOmljO\nnE4nX331FcePH6dTp040b968UMf95z//oUOHWM6efQaohcXyPG+8MYFhw+4t03iVUu6hy8Z5sC1b\ntjBx4nSs1mxGjLiTAQO0n7pSnkqTuVJKeYCi5k5tM1dKKQ+gybwcHDx4kCFDRhIbewdTp07H5XK5\nOySllIfRfuZlLD09nTZtbuTkyRE4nYmkpk7l4MHDvP76VHeHppTyINpmXsZmz57N6NGrsNkW5G85\njo9PI3JyrNq1UCl1RdpmXsHk/Z9xYdLWS66UKn2aWcpYz5498fP7Hi+vfwL/xmLpy4gR9+tduVKq\nVGkzSznYv38/Tz75D44eTef227vxxBPjMRqN7g5LKVWBaT9zpZTyANpmrpRSVZAmc6WU8gCazJVS\nygNoMldKKQ+gyVwppTyAJnOllPIAmsyVUsoDFDuZJyUlER4eTqtWrWjVqhUrVqwozbg8UkpKirtD\nqDD0Wpyn1+I8vRbFV+xkbjAYGD9+PFu2bGHLli3cdtttpRmXR9Iv6nl6Lc7Ta3GeXoviK1Ezi47s\nVEqpiqFEyfzNN9+kWbNmDB06lMzMzNKKSSmlVBFddW6WuLg4jh07dsn2iRMn0qlTJ4KDg4G89vN9\n+/aRnJx8aQU6O6BSShVLuU+0dfToUbp168bevXtLWpRSSqliKHYzS1paWsHrzz77jKioqFIJSCml\nVNEVew3Qxx57jO3bt2O324mIiGD27NmlGZdSSqkiKPad+UcffcS2bdvYvXs3K1asICws7KLPV6xY\nQXR0NJGRkbz88sslDrSya9iwITExMbRq1Yr27du7O5xyM3z4cEJDQ4mOji7YlpmZSVxcHDExMcTH\nx3Py5Ek3Rlh+Lnctqup4jUOHDtG1a1eio6Np2rQpU6ZMAarmd+NK16LI3w0pAzk5OdKwYUM5fPiw\n5ObmStu2bWXz5s1lUVWl0bBhQzlx4oS7wyh3a9eulc2bN0vz5s0Lto0aNUpee+01ERF57bXXZPTo\n0e4Kr1xd7lokJSXJq6++6sao3OPYsWOyY8cOERE5c+aMNGnSRLZu3VolvxtXuhZF/W6UyXD+TZs2\nERUVRVhYGN7e3gwcOJClS5eWRVWVilTBfvldunShZs2aF21btmwZd999NwBDhw6tMt+Ny10LqJrf\ni9DQUJo3bw5AQEAAMTExHDlypEp+N650LaBo340ySeaHDx+mfv36Be/Dw8M5fPhwWVRVaRgMhoI/\nH9944w13h+NW6enpBd1aQ0JCLnqYXhVV9fEav/32G6mpqdx4441V/rvx57Xo0qULULTvRpkkc+1b\nfqmNGzeyefNmVq9ezdy5c/nmm2/cHZKqAB555BH27dvHrl27aNy4MaNHj3Z3SOXKarXSv39/ZsyY\nQWBgoLvDcSur1cqAAQOYMWMG1apVK/J3o0ySeXh4OIcOHSp4f+jQoYvu1Kui2rVrA1CrVi369+9P\namqqmyNyn1q1apGRkQHk3aX/eW2qopCQEAwGAwaDgQceeKBKfS9yc3Pp168fQ4YMoXfv3kDV/W78\neS0GDx5ccC2K+t0ok2Terl07du7cyZEjR8jNzWXhwoUkJCSURVWVgs1mw2azAXD27FlWrFhRpfvl\nJyYmFowWTk5OJjEx0c0RuU9VHa8hIowYMYLIyEjGjRtXsL0qfjeudC2K/N0og4ezIiKybNkyiYqK\nkmbNmsmkSZPKqppKYf/+/RITEyMtWrSQJk2ayPPPP+/ukMrNXXfdJXXr1hWTySTh4eEyZ84cOXHi\nhHTv3l2io6MlLi5OsrKy3B1mufjfazF79mwZOnSoxMTEyPXXXy/x8fFy+PBhd4dZLr7//nsxGAzS\nokULadmypbRs2VKWL19eJb8bl7sWy5YtK/J3o1SG8yullHIvXWlIKaU8gCZzpZTyAJrMlVLKA2gy\nV0opD6DJXCmlPIAmc6WU8gD/D9aZp8rWxpfIAAAAAElFTkSuQmCC\n",
       "text": [
        "<matplotlib.figure.Figure at 0x4470110>"
       ]
      }
     ],
     "prompt_number": 7
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "theta = numpy.zeros(shape=(2,))\n",
      "alpha = 0.001\n",
      "\n",
      "m = len(data[:,0])\n",
      "h = lambda theta, x: numpy.add(theta[0], theta[1]*x)\n",
      "J = lambda theta: (1./(2.*m))*numpy.sum(numpy.power(h(theta,data[:,0])-data[:,1], 2))\n",
      "\n",
      "feature_vector = numpy.column_stack((numpy.ones(shape=(m,)), data[:,0]))\n",
      "nabla_J = lambda theta: (1./m)*numpy.sum(numpy.column_stack((h(theta,data[:,0])-data[:,1],h(theta,data[:,0])-data[:,1]))*feature_vector)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 8
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "feature_vector[:5]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 9,
       "text": [
        "array([[ 1.    ,  6.1101],\n",
        "       [ 1.    ,  5.5277],\n",
        "       [ 1.    ,  8.5186],\n",
        "       [ 1.    ,  7.0032],\n",
        "       [ 1.    ,  5.8598]])"
       ]
      }
     ],
     "prompt_number": 9
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print('Before GD, J = %g' % J(theta))\n",
      "for i in range(10000):\n",
      "    theta = theta - alpha*nabla_J(theta)\n",
      "print('After GD, J = %g' % J(theta))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Before GD, J = 32.0727\n",
        "After GD, J = 6.42089"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "theta"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "metadata": {},
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "array([ 0.72088159,  0.72088159])"
       ]
      }
     ],
     "prompt_number": 11
    }
   ],
   "metadata": {}
  }
 ]
}