{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A implementation of Ordinary Least Squares simple and multiple linear regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.regressor import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Illustration of a simple linear regression model:\n",
    "    \n",
    "![](./LinearRegression_files/simple_regression.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Ordinary Least Squares (OLS) Linear Regression, our goal is to find the line (or hyperplane) that minimizes the vertical offsets. Or in other words, we define the best-fitting line as the line that minimizes the sum of squared errors (SSE) or mean squared error (MSE) between our target variable (y) and our predicted output over all samples $i$ in our dataset of size $n$.\n",
    "\n",
    "$$SSE =  \\sum_i \\big(\\text{target}^{(i)} - \\text{output}^{(i)}\\big)^2$$\n",
    "\n",
    "$$MSE = \\frac{1}{n} \\times SSE$$\n",
    "\n",
    "\n",
    "Now, `LinearRegression` implements a linear regression model for performing ordinary least squares regression using one of the following three approaches:\n",
    "\n",
    "- Normal Equations\n",
    "- Gradient Descent\n",
    "- Stochastic Gradient Descent\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normal Equations (closed-form solution)\n",
    "\n",
    "The closed-form solution should be preferred for \"smaller\" datasets where calculating (a \"costly\") matrix inverse is not a concern. For very large datasets, or datasets where the inverse of $[X^T X]$ may not exist (the matrix is non-invertible or singular, e.g., in case of perfect multicollinearity), the gradient descent or stochastic gradient descent approaches are to be preferred.\n",
    "\n",
    "The linear function (linear regression model) is defined as:\n",
    "\n",
    "$$y = w_0x_0 + w_1x_1 + ... + w_mx_m = \\sum_{i=0}^{n} = \\mathbf{w}^T\\mathbf{x}$$\n",
    "\n",
    "where $y$ is the response variable, $\\mathbf{x}$ is an $m$-dimensional sample vector, and $\\mathbf{w}$ is the weight vector (vector of coefficients). Note that $w_0$ represents the y-axis intercept of the model and therefore $x_0=1$.  \n",
    "\n",
    "Using the closed-form solution (normal equation), we compute the weights of the model as follows:\n",
    "\n",
    "$$ \\mathbf{w} = (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^Ty$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Descent (GD)  and Stochastic Gradient Descent (SGD) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See [Gradient Descent and Stochastic Gradient Descent](../general_concepts/gradient-optimization.md) and [Deriving the Gradient Descent Rule for Linear Regression and Adaline](../general_concepts/linear-gradient-derivative.md) for details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Random shuffling is implemented as:\n",
    "\n",
    "- for one or more epochs\n",
    "    - randomly shuffle samples in the training set\n",
    "        - for training sample *i*\n",
    "            - compute gradients and perform weight updates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- F. Galton. [Regression towards mediocrity in hereditary stature](http://www.jstor.org/stable/2841583). Journal of the Anthropological Institute of Great Britain and Ireland, pages 246–263, 1886.\n",
    "- A. I. Khuri. [Introduction to linear regression analysis](http://onlinelibrary.wiley.com/doi/10.1111/insr.12020_10/abstract), by Douglas C. Montgomery, Elizabeth A. Peck, G. Geoffrey Vining. International Statistical Review, 81(2):318–319, 2013.\n",
    "- D. S. G. Pollock. [The Classical Linear Regression Model](http://www.le.ac.uk/users/dsgp1/COURSES/MESOMET/ECMETXT/06mesmet.pdf)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Closed Form Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.25\n",
      "Slope: 0.81\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAFkCAYAAAA9h3LKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xl4VdXZ9/HvEnFWpE6tVauVQLFODVqxVqw1r6FYRXys\nNSBWrcVZGnGs8ljtoB0szk8Hax2iaW0dqlVBsXWWqgkOSDQJ1qFOqGAcEQjr/WOlIlSUk5xk5+zz\n/VxXris5OcPtMeSXe++17xVijEiSpPxYIesCJElScRnukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk\n5YzhLklSzhjukiTljOEuSVLOGO6SJOVMweEeQtgwhHBlCOG1EMK7IYRHQwiV3VGcJEkq3IqF3DmE\nsDZwH3AHUA28BlQAc4tfmiRJ6oxQyMYxIYSzgR1jjLt0X0mSJKkrCj0svyfwcAjhmhDCKyGExhDC\nod1RmCRJ6pxCO/f3gAicA/wF+DJwHnBYjPHKj7j/OqTD988A84pQryRJ5WIVYFNgSozx9UIeWGi4\nvw88GGPc+UO3nQdsF2Pc6SPuPxq4qpCCJEnSEsbEGK8u5AEFLagDXgKalrqtCdhnGfd/BqCuro7B\ngwcX+FL5U1tby6RJk7IuI3O+D4v5XiS+D4v5XiS+D9DU1MQBBxwAHVlaiELD/T5g0FK3DQKeXcb9\n5wEMHjyYykqvluvXr5/vA74PH+Z7kfg+LOZ7kfg+LKHg09qFLqibBAwNIZwSQti847D7ocCFhb6w\nJEnqHgWFe4zxYWAUUAM8DpwKjI8x/rEbapMkSZ1Q6GF5Yoy3ALd0Qy2SJKkInC3fg2pqarIuoVfw\nfVjM9yLxfVjM9yLxfeiagi6FK/jJ08z5hoaGBhdGSJJUgMbGRoYMGQIwJMbYWMhj7dwlScoZw12S\npJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqSc\nMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHc\nJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJ\nyhnDXZKknDHcJUnKGcNdkqScMdwlSb3LM8/A9OlZV1HSDHdJUu+wcCGccw588Ytw4olZV1PSDHdJ\nUvYaGuDLX4YTToBDD4Xrrsu6opJWULiHEE4PISxa6mNmdxUnScq5t9+G445Lwd7eDtOmwXnnwZpr\nZl1ZSVuxE4+ZAewGhI6vFxavHElS2bjlFjjiCJg9G846C2proW9fmpubmTVrFgMGDKCioiLrKktS\nZw7LL4wxvhpjnN3xMafoVUmS8uvll2H//WGPPWDQIJgxA048kTlvvcXw4XswaNAgRowYwcCBAxk+\nfA/mzp2bdcUlpzPhXhFCeCGEMCuEUBdC2LjoVUmS8mfRIrjkEhg8GO64A668EqZMgc03B2D06LFM\nnToNqAOeA+qYOnUaNTUHZFl1SSo03KcBBwHVwOHAZsDdIYTVi1yXJClPnnwSdt0Vvvc9GDkSmprg\ngAMgpDO8zc3NTJlyC+3t5wNjgI2BMbS3n8eUKbfQ0tKSZfUlp6Bz7jHGKR/6ckYI4UHgWWA/4A/L\nelxtbS39+vVb4raamhpqamoKeXlJUql5/3342c/gJz+BTTaBqVNht93+626zZs3q+GzYUt/ZBYDW\n1tZcn3+vr6+nvr5+idva2to6/XydWVD3gRhjWwihGRjwcfebNGkSlZWVXXkpSVKpueceGDcOWlvT\ndeunnQarrvqRd92849A83E3q3P/jLgAGDPjYmCl5H9XwNjY2MmTIkE49X5eucw8hrEEK9pe68jyS\npBx54w047DAYNgzWXhsaG1PnvoxgBxg4cCDV1SPo0+dY0jn354E6+vQZT3X1iFx37d2h0OvcfxFC\nGBZC+FwI4SvA9cACoP4THipJyrsY4c9/Tgvm6uvhoovg3nthq62W6+H19XVUVQ0FxgKbAGOpqhpK\nfX1dd1adS4Uelt8IuBpYB3gVuBcYGmN8vdiFSZJKyHPPwVFHwd/+BqNGwQUXwGc/W9BT9O/fn8mT\nb6alpYXW1lavc++CQhfUuQJOkrRYe3sK8tNOg3790tjYUaO69JQVFRWGehc5W16S1DnTp8PQoWl8\n7EEHpcvbuhjsKg7DXZJUmHfeSavft98e5s2D+++HCy+EtdbKujJ16NKlcJKkMjNlChx+OLz0Evzo\nR3D88dC3b9ZVaSl27pKkTzZ7NowZA8OHw+c/D48/DqecYrD3UnbukqRlixEuuwwmTIAVVoDLL4ex\nYz8YG6veyc5dkvTRmpvh61+HQw6Bb34zLZg78ECDvQQY7pKkJc2fDz/+MWy9dbp+/bbb4IorYL31\nsq5My8nD8pKkxe6/P+3c9tRTcMIJMHEirLZa1lWpQHbukiRoa4Mjj4SddoI11oCGBjjrLIO9RNm5\nS1I5izFNlTvmGHjrLTj//BTyffpkXZm6wM5dksrV88/D3nvDvvumgTQzZ6aQN9hLnuEuSeXmP/Pg\nt9gCHnoIrr0WbrgBNt4468pUJIa7JJWTxx6Dr3wFjj02Xa/e1AT77OPlbTljuEtSOXj3XTj5ZKis\nhLffTvusX3xx2slNueOCOknKu9tvT/PgX3gBzjgjXeK20kpZV6VuZOcuSXn16qtpotzuu8Mmm6RD\n8qeearCXATt3ScqbGNNEuQkTYNEiuPTStN+659XLhp27JOVJaytUVaUwr66GJ5+Egw822MuM4S5J\nebBgQZoot9VW8PTTMHkyXHUVrL9+1pUpAx6Wl6RSN21amgff1ATHHQennw6rr551VcqQnbsklao3\n34Sjj07Xra+8chpI8/OfG+yyc5ekknTDDSnY33gDJk1Knzs2Vh3s3CWplLzwQpooN2oUbLttmgc/\nfrzBriUY7pJUCtrb4aKLYPDgtOf6NdfATTel69elpRjuktTbPf44fPWr6dB7TU1aOPetb3l5m5bJ\ncJek3uq999JEucpKaGuDe+6B3/wG+vfPujL1ci6ok6Te6I470jz4556DiRPhpJPSinhpOdi5S1Jv\n8tprabpcVRVsuCE8+ij87/8a7CqInbsk9QYxpolytbWwcCFcckkaG7uCPZgK50+NJGVt1qw0B37s\n2NSxNzXBd79rsKvT/MmRpKwsWAA/+xlsuSU0N8PNN0N9PXz601lXphLnYXlJysKDD6Z58DNmwPe/\nD2ecAWuskXVVygk7d0nqSW+9lSbKDR0KK66YQv6ccwx2FZWduyT1lBtvhKOOgjlz4Je/hGOPTQEv\nFZmduyR1txdfhH33hZEj037rTzyRtmY12NVNDHdJ6i6LFsGvf53mwd9zD/zxj2nR3KabZl2Zcs5w\nl6Tu8MQTsPPOcMQRsN9+6fK2b3/befDqEYa7JBXTvHlpXOyXvgSvvw533gm/+x186lNZV6Yy4gkf\nSSqWO++EcePgmWfglFPSxyqrZF2VypCduyR11Zw5aaLcrrvCBhvAI4+k69YNdmWkS+EeQjg5hLAo\nhPCrYhUkSSUjRrj6avjCF+Daa9N2rHfdBVtskXVlKnOdDvcQwvbAOODR4pUjSSXiX/+Cb3wDxoyB\nr30tLZgbN8558OoVOvVTGEJYA6gDDgXeKGpFkvQJmpubufXWW2lpaen5F1+4MA2g2XJLmDkTbroJ\nrrkGPvOZnq9FWobO/ol5EXBTjPHvxSxGkj7OnDlzGD58DwYNGsSIESMYOHAgw4fvwdy5c3umgIcf\nhi9/GU46KXXpM2fCN7/ZM68tFaDgcA8h7A9sC5xS/HIkadlGjx7L1KnTSAcOnwPqmDp1GjU1B3Tv\nC7/9dpoot8MOaTDNtGkwaZLz4NVrFXQpXAhhI+BcoCrGuKB7SpKk/9bc3MyUKbeQgn1Mx61jaG+P\nTJkylpaWFioqKor/wjffDEceCa++CmefnXZw69u3+K8jFVGh17kPAdYDGkP4YMxSH2BYCOFoYOUY\nY1z6QbW1tfTr12+J22pqaqipqelEyZLK0axZszo+G7bUd3YBoLW1tbjh/vLLafe2a66B6up0Dftm\nmxXv+aUPqa+vp76+fonb2traOv184SOyeNl3DmF14HNL3XwZ0AScHWNsWur+lUBDQ0MDlZWVnS5S\nkpqbmxk0aBBLdu50fD2W5ubm4oT7okXw+9/DiSemDv3cc6GmxrGx6nGNjY0MGTIEYEiMsbGQxxbU\nuccY3wFmfvi2EMI7wOtLB7skFdPAgQOprh7B1KnH0t4eSR37XfTpM56qqhHFCfamJjjssLTJy8EH\nwy9+Aeus0/XnlXpYMS7IXP7WX5K6oL6+jqqqocBYYBNgLFVVQ6mvr+vaE7//Pvzwh7Dttulw/N//\nDpdearCrZHV5tnyM8evFKESSPkn//v2ZPPlmWlpaaG1tZcCAAV3v2O++O3Xrra1w8slw6qmOjVXJ\nc+MYSSWnoqKi66E+d266Xv13v4Mdd4Tp09NgGikHDHdJ5SXGtAJ+/Hh47z24+OLUuTs2VjniT7Ok\n8vHss2mi3P77w1e/mhbQHXGEwa7c8SdaUv4tXJgmym2xBTz6KNxwA/zlL7DhhllXJnULw11Svk2f\nDkOHwoQJac/1mTNh5Misq5K6leEuKZ/eeQeOPx623x4WLIAHHoDzz4e11sq6MqnbuaBOUv7cems6\nl/7KK/CTn6RNX5wHrzJi5y4pP155BUaPhhEjoKICZsxIl7sZ7Cozdu6SSl+MaaLcCSdAnz5wxRVw\nwAHOg1fZsnOXVNqeegp23RUOPRT22itd3jZ2rMGusma4SypN8+fDj34EW28N//43TJ0Kl10G666b\ndWVS5jwsL6n03HcfjBsHzc3pUPzEibDqqllXJfUadu6SSscbb8Dhh6fpcmuuCY2N8NOfGuzSUuzc\nJfV+McK118Ixx6Tr1y+8MIV8nz5ZVyb1Snbuknq3559PE+W+9a00aW7mTDjqKINd+hiGu6Teqb0d\nzjsvzYNvaIDrroPrr4eNNsq6MqnXM9wl9T6PPJL2WK+the98J3Xro0ZlXZVUMgx3Sb3Hu++miXLb\nbZf2Wr/vvnR+vV+/rCuTSooL6iT1DrfdlhbJvfginHlm2vRlpZWyrkoqSXbukrI1e3YaFVtdDZtt\nBo8/Dj/4gcEudYGdu6RsxAiXX572WYc0Xe7AAx0bKxWBnbukntfSArvtBgcfnHZwe/LJtHDOYJeK\nwnCX1HPmz0/7q2+1FTz7bDrPfuWVsN56WVcm5YqH5SX1jAcegO99L3Xpxx8P//u/sNpqWVcl5ZKd\nu6Tu1daWJsrttFMK84YGOPtsg13qRnbukrrP9dfD0UfDm2/Cuec6NlbqIXbukorv3/9OE+X22QeG\nDEkT5o491mCXeojhLql42tvhoovSPPh//hP+8hf4619h442zrkwqK4a7pOJ4/PF0Xv3oo2HMmNSt\n/8//eHmblAHDXVLXvPdemihXWQlvvQX33gv/93+w9tpZVyaVLRfUSeq8O+6Aww5Le66ffjqceKJj\nY6VewM5dUuFeey1NlKuqSvurP/YYnHaawS71EnbukpZfjFBXl/ZZX7QIfv/7NELW8+pSr2LnLmn5\nzJoFu++eNneprk6T5g45xGCXeiHDXdLHW7AAfvYz2HJLaG2FW2+Fq66C9dfPujJJy2C4S1q2f/4T\nttsurYY/+miYMQOGD8+6KkmfwHCX9N/eeitNlNtxR+jbFx56CH7xC1h99awrk7QcXFAnaUk33phm\nwM+ZA+ecA8ccAyv6q0IqJXbukpIXX0wT5UaOhK23ThPmamsNdqkEGe5SuVu0KE2UGzwY7rsP/vQn\n+Nvf4HOfy7oySZ1kuEvl7IknYOed4cgj4dvfhqYm2G8/L2+TSlxB4R5CODyE8GgIoa3j4/4Qgktn\npVIzbx5MnAhf+lI6t3733fDb30L//llXJqkICj2Z9jxwEtACBOAg4K8hhG1jjE1Frk1Sd/jHP9I8\n+GefhVNPhZNPhpVX/q+7NTc3M2vWLAYMGEBFRUUGhUrqrII69xjjzTHGyTHGWTHG1hjjacDbwNDu\nKU9S0bz+epoo9/Wvw6c/DY88kjZ7WSrY58yZw/DhezBo0CBGjBjBwIEDGT58D+bOnZtR4ZIK1elz\n7iGEFUII+wOrAQ8UryRJRRUjXH11WjB33XXp8Pudd6avP8Lo0WOZOnUaUAc8B9Qxdeo0amoO6MGi\nJXVFwde4hBC2JIX5KsBbwKgY45PFLkxSETz9NBxxBNx2W1owd+65qWtfhubmZqZMuYUU7GM6bh1D\ne3tkypSxtLS0eIheKgGduYD1SWAboB+wL3BFCGHYxwV8bW0t/fr1W+K2mpoaampqOvHykj7RwoUw\naVI67L7eeunStj32+MSHzZo1q+OzYUt9ZxcAWltbDXepG9TX11NfX7/EbW1tbZ1+vhBj7FJBIYTb\ngdYY4xEf8b1KoKGhoYHKysouvY6k5fTQQzBuXNpjffx4OPNMWGON5Xpoc3MzgwYNYsnOnY6vx9Lc\n3Gy4Sz2ksbGRIUOGAAyJMTYW8thiXOe+AvDfS20l9ay3304T5YYOTdepP/gg/OpXyx3sAAMHDqS6\negR9+hxLCvTngTr69BlPdfUIg10qEYVe5/7TEMLOIYTPhRC2DCGcRTpeV9c95UlaLn/7G2yxRVos\n9/Ofp2BPf/EXrL6+jqqqocBYYBNgLFVVQ6mv95+5VCoKPee+PnA58BmgDXgM2D3G+PdiFyZpObz0\nUjr0/uc/p61YL74YNtusS0/Zv39/Jk++mZaWFlpbW73OXSpBBYV7jPHQ7ipEUgEWLYJLLoETT4SV\nVkqXuu2/f1HHxlZUVBjqUolytrxUambOhGHD0pS5ffeFJ5+EmhrnwUv6gOEulYp589KlbdtuC6++\nmsbIXnIJfOpTWVcmqZdxo2apFNx1V+rUn346zYL/wQ9glVWyrkpSL2XnLvVmc+fC974HX/sarLtu\nmgd/5pkGu6SPZecu9UYxwp/+lFbCz5sHv/51CvkV/Htc0ifzN4XU2zzzTBoVW1OTFs41NaVD8ga7\npOXkbwupt1i4ME2U++IX4fHH4a9/Tdevb7hh1pVJKjGGu9QbNDTADjvA8cenw+8zZ8Jee2VdlaQS\nZbhLWXr7bZgwAb78ZWhvh2nT0rasa66ZdWWSSpgL6qSs3HILHHkkzJ4NZ52VNn3p2zfrqiTlgJ27\n1NNeeSUtlttjDxg4EGbMSGNkDXZJRWLnLvWURYvg0kvhhBNgxRWhrg5Gj3ZsrKSis3OXesKTT8Ku\nu6bFcnvvnb4eM8Zgl9QtDHepO73/fpoot8028OKLcMcd8Ic/wDrrZF2ZpBzzsLzUXe65B8aNg9ZW\nOOkkOPVUWHXVrKuSVAbs3KVie+ONNFFu2DBYe22YPh1+/GODXVKPsXOXiiVG+Mtf4Nhj4Z134KKL\n4PDDHRsrqcf5W0cqhueeSxPl9tsPvvKVNA/+yCMNdkmZ8DeP1BXt7Wmi3BZbpMPv118P114Ln/1s\n1pVJKmOGu9RZ06fD0KFw3HFw8MFpHvzee2ddlSQZ7lLB3nknTZTbfvu01/r998MFF8Baa2VdmSQB\nLqiTCjNlSlok9/LLaQX8hAmOjZXU69i5S8tj9uw0UW74cNh887Tf+sknG+ySeiU7d+njxAiXXZY6\n9BVWgMsvh7FjHRsrqVezc5eWpbkZvv51OOQQ2HPPNA/+wAMNdkm9nuEuLW3+/HQ+feut0/Xrt9+e\nOvZ11826MklaLh6Wlz7s/vvTzm1PPZW2Zp04EVZbLeuqJKkgdu4SQFtbmii3006wxhrQ2AhnnWWw\nSypJdu4qbzHCddfBMcfAW2/B+eenkO/TJ+vKJKnT7NxVvp5/Pk2U23ffNJBm5swU8ga7pBJnuKv8\ntLeniXJbbAEPPZRmwd9wA2y8cdaVSVJRGO4qL489lnZtO/bYdL16UxPss4+Xt0nKFcNd5eHdd9NE\nucpKePttuPdeuPhi6Ncv68okqehcUKf8u/32NA/+hRfgjDPSJW4rrZR1VZLUbezclV+vvpomyu2+\nO2yySTokf+qpBruk3LNzV/7ECFdckebBL1oEl14KBx3keXVJZcPOXfnS2gpVVSnMq6vTPPiDDzbY\nJZUVw135sGBBmii31Vbw9NMweTJcdRWsv37WlUlSj/OwvErftGlpHnxTExx3HJx+Oqy+etZVSVJm\n7NxVut58E44+Ol23vvLKaSDNz39usEsqe3buKk033JCC/Y03YNKk9LljYyUJKLBzDyGcEkJ4MITw\nZgjhlRDC9SGEgd1VnPRfXnghTZQbNQq23TbNgx8/3mCXpA8p9LD8zsAFwA5AFdAXuC2EsGqxC1P5\naW5u5tZbb6WlpeW/v9neDhddBIMHpz3Xr7kGbropXb8uSVpCQYflY4wjPvx1COEgYDYwBLi3eGWp\nnMyZM4fRo8cyZcotH9xWXT2C+vo6+vfvD48/DuPGpYVz48bB2WdD//4ZVixJvVtXF9StDURgThFq\nUZkaPXosU6dOA+qA54A6pk6dxnf2q0kT5Soroa0N7rkHfvMbg12SPkGnF9SFEAJwLnBvjHFm8UpS\nOWlubu7o2OuAMR23jmGX9hmcM/VsFt31d1aYOBFOOimtiJckfaKurJa/GNgC2OmT7lhbW0u/pXbf\nqqmpoaampgsvrzyYNWtWx2fDAFiH1/glx3MQl3MX8NKFFzJs3LjM6pOknlBfX099ff0St7W1tXX6\n+UKMsfAHhXAhsCewc4zxuY+5XyXQ0NDQQGVlZaeLVH41NzczaNAg4ErGAJOoZUUWcjz/wx/4PU81\nN1NRUZF1mZLU4xobGxkyZAjAkBhjYyGPLbhz7wj2kcAuHxfs0vIYOHAgB+28C6PvOZj/x0L+yJ58\nn915rc/p7F41wmCXpE4oKNxDCBcDNcBewDshhA06vtUWY5xX7OKUcwsWwK9+xaUP/ZPZq/RlxLyF\n3MpNwE1UV6XV8pKkwhXauR9OWh1/51K3HwxcUYyCVCYefDDNg58xg/D977PBGWdw3ksvcUxrKwMG\nDLBjl6QuKPQ6d2fRq2veegtOOw0uuAC+9KUU8umcEhUVFYa6JBWBs+XVc268EY46CubMgV/+Eo49\nFlb0R1CSis1OXN3vxRdh331h5Mi03/oTT6StWQ12SeoWhru6z6JF8Otfp3nw99wDf/wj3HwzbLpp\n1pVJUq4Z7uoeTzwBO+8MRxwB++0HTU3w7W9DCFlXJkm5Z7iruObNg4kT02K511+HO++E3/0OPvWp\nrCuTpLLhSU8Vz513pl3bnnkGTjklfayyStZVSVLZsXNX182ZA9/9Luy6K2ywATzyCJxxhsEuSRmx\nc1fnxQj19fD978P8+Wk71kMPhRX8m1GSsuRvYXXOv/4F3/gGjBkDX/taWjA3bpzBLkm9gL+JVZiF\nC9MAmi9+EWbOhJtugmuugc98JuvKJEkdDHctv4cfhu23h5NOgsMOS+H+zW9mXZUkaSmGuz7Z229D\nbS3ssEM6zz5tGkyaBGuskXVlkqSP4II6fbybb4Yjj4RXX4Wzz06L5/r2zboqSdLHMNz10V5+GcaP\nT+fTd98d/vEP+Pzns65KkrQcPCyvJS1aBL/9LXzhCynQ6+pg8mSDXZJKiOGuxZqaYJdd0mK5ffZJ\nX48Z4zx4SSoxhrvg/ffhhz+EbbdNh+PvuAMuvRTWWSfryiRJneA593J3992pU29tTZe4nXoqrLpq\n1lVJkrrAzr1czZ2bJsrtsgv07w/Tp8OPf2ywS1IO2LmXmxjTCvjx4+G99+Dii1Pn7thYScoNf6OX\nk2efTRPl9t8fdtopTZg74giDXZJyxt/q5WDhwjRRbost4NFH4YYb4Npr4bOfzboySVI3MNzzbvp0\nGDoUJkxIe67PnAkjR2ZdlSSpGxnuefXOO3D88Wmjl/nz4YEH4PzzYa21sq5MktTNXFCXR7fems6l\nv/JKWgE/YYLz4CWpjNi558krr8Do0TBiBFRUwIwZcPLJBrsklRk79zyIMU2UO+GEtPL9iivggAMc\nGytJZcrOvdQ99RTsuisceijsuSc8+SSMHWuwS1IZM9xL1fz58KMfwdZbw7//DbffDpdfDuuum3Vl\nkqSMeVi+FN13Xxod29ycDsVPnOjYWEnSB+zcS8kbb8Dhh8NXvwprrgkNDfDTnxrskqQl2LmXghjT\nRLljjknXr19wQbrUrU+frCuTJPVCdu693fPPp4ly3/oW7LBDmjB39NEGuyRpmQz33qq9Hc47L82D\nb2iA665LM+E32ijryiRJvZzh3hs98gjsuCPU1sKBB6ZufdSorKuSJJUIw703efddOOkk2G679Pl9\n98FFF0G/fllXJkkqIS6o6y1uuy2thH/xRTjzzLTpy0orZV2VJKkE2blnbfbsNCq2uho23RQefxx+\n8AODXZLUaXbuWYkxTZSbMCF9/Yc/wHe+49hYSVKX2blnoaUFdtsNDj4YvvENaGqCgw4y2CVJRVFw\nuIcQdg4h3BhCeCGEsCiEsFd3FJZL8+fDT34CW20FzzwDkydDXR2sv37WlUmScqQznfvqwCPAkUAs\nbjk59sADMGQInH46jB+f9lqvrs66KklSDhV8zj3GOBmYDBCCx5GXS3s780eP5t2+fWm79lo+N3Jk\n1hVJknLMc+7dbM6cOQzfYy8qnnmGdVpa2HTvvRk+fA/mzp2bdWmSpJwy3LvZ6NFjmTp1Gs9RxyKe\nA+qYOnUaNTUHZF2aJCmnvBSuGzU3NzNlyi1AHTCm49YxtLdHpkwZS0tLCxUVFRlWKEnKox4J99ra\nWvotNUK1pqaGmpqannj5zMyaNavjs2FLfWcXAFpbWw13SRL19fXU19cvcVtbW1unn69Hwn3SpElU\nVlb2xEv1KptvvnnHZ3ezuHMHuAuAAQMG9HRJkqRe6KMa3sbGRoYMGdKp5ys43EMIqwMDgP+slP98\nCGEbYE6M8flOVZFTAwcOpLp6BFOnHkt7eyR17HfRp894qqpG2LVLkrpFZxbUbQdMBxpI17mfAzQC\nZxSxrtymfTCCAAAG+UlEQVSor6+jqmooMBbYBBhLVdVQ6uvrMq5MkpRXnbnO/S5cZb/c+vfvz+TJ\nN9PS0kJraysDBgywY5ckdStXy/eQiooKQ12S1CPswCVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZ\nw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNd\nkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKk\nnDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx\n3CVJyhnDXZKknDHcJUnKGcO9B9XX12ddQq/g+7CY70Xi+7CY70Xi+9A1nQr3EMJRIYR/hRDeCyFM\nCyFsX+zC8sgf1sT3YTHfi8T3YTHfi8T3oWsKDvcQwreBc4DTgS8BjwJTQgjrFrk2SZLUCZ3p3GuB\n38QYr4gxPgkcDrwLHFLUyiRJUqcUFO4hhL7AEOCO/9wWY4zAVGDH4pYmSZI6Y8UC778u0Ad4Zanb\nXwEGfcT9VwFoamoqvLIcamtro7GxMesyMuf7sJjvReL7sJjvReL7sER2rlLoY0NqvJfzziF8BngB\n2DHG+M8P3f4zYFiMccel7j8auKrQoiRJ0gfGxBivLuQBhXburwHtwAZL3b4B8PJH3H8KMAZ4BphX\n4GtJklTOVgE2JWVpQQrq3AFCCNOAf8YYx3d8HYDngPNjjL8otABJklRchXbuAL8CLgshNAAPklbP\nrwZcVsS6JElSJxUc7jHGazquaT+TdDj+EaA6xvhqsYuTJEmFK/iwvCRJ6t2cLS9JUs4Y7pIk5Uy3\nhrsbzEAIYecQwo0hhBdCCItCCHtlXVMWQginhBAeDCG8GUJ4JYRwfQhhYNZ19bQQwuEhhEdDCG0d\nH/eHEIZnXVfWQggnd/z7+FXWtfS0EMLpHf/tH/6YmXVdWQkhbBhCuDKE8FoI4d2Ofy+VWdfVkzpy\nc+mfiUUhhAuW9zm6LdzdYOYDq5MWHR4JlPMCh52BC4AdgCqgL3BbCGHVTKvqec8DJwGVpFHOfwf+\nGkIYnGlVGer4o38c6XdEuZpBWqD86Y6Pr2ZbTjZCCGsD9wHvA9XAYGACMDfLujKwHYt/Fj4N/D9S\nflyzvE/QbQvqlnE9/POk6+F/3i0v2suFEBYBe8cYb8y6lqx1/JE3mzTZ8N6s68lSCOF14PgY4x+y\nrqWnhRDWABqAI4CJwPQY43HZVtWzQginAyNjjGXVnX6UEMLZpAmou2RdS28SQjgXGBFjXO6jnd3S\nubvBjJbD2qS/ROdkXUhWQggrhBD2J82JeCDrejJyEXBTjPHvWReSsYqOU3ezQgh1IYSNsy4oI3sC\nD4cQruk4fdcYQjg066Ky1JGnY4DfF/K47jos/3EbzHy6m15TJaLjKM65wL0xxrI7txhC2DKE8Bbp\n0OPFwKiO7ZPLSscfNtsCp2RdS8amAQeRDkMfDmwG3B1CWD3LojLyedJRnKeA3YH/A84PIYzNtKps\njQL6AZcX8qDOTKiTuupiYAtgp6wLyciTwDakf7D7AleEEIaVU8CHEDYi/YFXFWNckHU9WYoxfnhu\n+IwQwoPAs8B+QLmdqlkBeDDGOLHj60dDCFuS/ui5MruyMnUIcGuM8aP2b1mm7urcC91gRmUihHAh\nMAL4WozxpazryUKMcWGM8ekY4/QY46mkhWTjs66rhw0B1gMaQwgLQggLgF2A8SGE+R1Hd8pSjLEN\naAYGZF1LBl4Clt4jvAnYJINaMhdC2IS0APl3hT62W8K94y/xBmC3/9zW8Y91N+D+7nhN9X4dwT4S\n2DXG+FzW9fQiKwArZ11ED5sKbEU6LL9Nx8fDQB2wTSzj0ZkdiwwHkIKu3NwHDFrqtkGkIxnl6BDS\n6exbCn1gdx6Wd4MZoOO82QDgP53I50MI2wBzYozPZ1dZzwohXAzUAHsB74QQ/nNUpy3GWDbbAYcQ\nfgrcStpJcU3SQpldSOcXy0aM8R1gifUWIYR3gNdjjEt3brkWQvgFcBMpwD4LnAEsAOqzrCsjk4D7\nQginkC772gE4FPheplVloKMhPgi4LMa4qNDHd1u4u8HMB7YD/kFaGR5J1/5DWhxxSFZFZeBw0n//\nnUvdfjBwRY9Xk531Sf/vPwO0AY8Bu7taHCjfORAbAVcD6wCvAvcCQ2OMr2daVQZijA+HEEYBZ5Mu\njfwXMD7G+MdsK8tEFbAxnVx34cYxkiTljLPlJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKk\nnDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJypn/Dw4reuLXj4wNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105e70588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "ne_lr = LinearRegression(minibatches=None)\n",
    "ne_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % ne_lr.b_)\n",
    "print('Slope: %.2f' % ne_lr.w_[0])\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, ne_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Iteration: 100/100 | Cost 0.08 | Elapsed: 0:00:00 | ETA: 0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.82\n",
      "Slope: 0.22\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAFkCAYAAAA9h3LKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xd4V+X9//HnDVJ3EUcdVdsqgR+4iVqssxoFUdtSRw2j\nRYsCggzFIu7RqrV111FbNzXVuq0Mi1b9ogWVqICgCdhK3QqIigMI9++POw4oKp+sk8/J83Fdua7k\n8Blvj0leeZ9zjxBjRJIk5UerrAuQJEkNy3CXJClnDHdJknLGcJckKWcMd0mScsZwlyQpZwx3SZJy\nxnCXJClnDHdJknLGcJckKWcKDvcQwmYhhFtCCO+EED4MITwXQujSGMVJkqTCrVbIg0MI6wGPAw8B\n3YB3gBJgQcOXJkmS6iIUsnFMCOECYLcY496NV5IkSaqPQi/LHwI8HUK4PYTwZgihMoTQvzEKkyRJ\ndVNo5/4REIGLgDuAXYHLgAExxltW8vgNSJfv/wN83AD1SpLUUqwBfBeYEGOcV8gTCw33T4AnY4x7\nfuHYZcDOMcbdV/L4XsBfCilIkiQtp3eM8dZCnlDQgDrgdWDWCsdmAT/9ksf/B2DMmDF06tSpwLfK\nnxEjRnDJJZdkXUbmPA+f81wknofPeS4SzwPMmjWLPn36QG2WFqLQcH8c6LjCsY7Ay1/y+I8BOnXq\nRJcuzpZr27at5wHPwxd5LhLPw+c8F4nnYTkF39YudEDdJUDXEMLoEMLWtZfd+wN/KPSNJUlS4ygo\n3GOMTwM9gXJgOnAqMCzG+NdGqE2SJNVBoZfliTGOBcY2Qi2SJKkBuLZ8EyovL8+6hGbB8/A5z0Xi\nefic5yLxPNRPQVPhCn7xtOb81KlTpzowQpKkAlRWVlJaWgpQGmOsLOS5du6SJOWM4S5JUs4Y7pIk\n5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM\n4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEu\nSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklS\nzhjukiTljOEuSWpeli6FBQuyrqKoGe6SpObj2Weha1fo1y/rSoqa4S5Jyt6HH8KoUbDzzrB4MZxy\nStYVFbXVsi5AktTCPfQQDBgAr7wC55wDJ50EbdpkXVVRK6hzDyGcGUJYtsLHzMYqTpKUY/Pnw9FH\nQ1kZbL45TJuWOnaDvd7q0rnPAPYDQu3XSxuuHElS7sUIt90Gw4bBJ5/AtdfCL38JrVK/WVVVxZw5\nc2jfvj0lJSUZF1uc6nLPfWmM8e0Y41u1H/MbvCpJUj7NnQuHHALl5bDnnjBrFhxzDLRqxfz58+ne\n/SA6duxIjx496NChA927H8QCR84XrC7hXhJCeDWEMCeEMCaEsEWDVyVJypeaGrjiCthmG3jmGbjn\nHrjjDth0088e0qtXXyZOnAyMAeYCY5g4cTLl5X2yqrpoFRruk4F+QDdgIPA94LEQwtoNXJckKS9m\nzIDdd4ehQ6FvX5g5E3784+UeUlVVxYQJY6mpuRzoDWwB9Kam5jImTBhLdXV1FpUXrYLuuccYJ3zh\nyxkhhCeBl4EjgBu+7HkjRoygbdu2yx0rLy+nvLy8kLeXJBWTjz+G3/wGLrgA2reH//s/2GOPlT50\nzpw5tZ/ttcK/7A3A7Nmzc33/vaKigoqKiuWOLVy4sM6vV6+pcDHGhSGEKqD9Vz3ukksuoUuXLvV5\nK0lSMXnsMTj2WHjpJTj1VBg9GlZf/UsfvvXWW3/6RFLn/qlHAWjf/itjpuitrOGtrKyktLS0Tq9X\nr0VsQgjrkIL99fq8jiQpJ959N81Z33tv2GCDtOLcWWd9ZbADdOjQgW7detC69VDSPff/AmNo3XoY\n3br1yHXX3hgKnef+uxDCXiGE74QQfgDcDSwBKr7mqZKkvLvrLujcGSoq4Mor02X4zp1X+ekVFWMo\nK+sK9AW2BPpSVtaViooxjVVxbhV6WX5z4FZgA+BtYBLQNcY4r6ELkyQViVdfhSFD0gj4H/0oBfvm\nmxf8Mu3atWP8+Aeorq5m9uzZznOvh0IH1DkCTpKULFuWFqAZNQrWWgv+9jc49FAI4euf+xVKSkoM\n9Xpy4xhJUuFmzUr31QcNgiOOSNPbDjus3sGuhmG4S5JW3eLFaXOXHXeEN9+Ef/4T/vQnaNcu68r0\nBe4KJ0laNf/6V1oq9sUX4Ve/gtNOgzXXzLoqrYSduyTpq733Xhowt/vu6d761KlpcRqDvdmyc5ck\nfbn774fjjoMFC+Dii+H446F166yr0tewc5ck/a833kgD5X70I9huO3j+eRg+3GAvEnbukqTPxQjX\nXw8jR0KbNnDrrXDkkY6CLzJ27pKkpLoa9t0X+vdPu7bNmpX2XTfYi47hLkkt3ZIlcP756fL7yy/D\ngw/CjTemteFVlLwsL0kt2VNPpU59xgw48cS0yctaa2VdlerJzl2SWqIPPoARI6Br1zRI7qmn4MIL\nDfacsHOXpJZm/HgYOBDeegsuuCCF/GrGQZ7YuUtSS/H229CnDxx4IJSUwPTpcNJJBnsO+X9UkvIu\nRrjlFjjhhPT5jTfCz3/uKPgcs3OXpDx76SXo1g1+8Qs44IA0ve0XvzDYc85wl6Q8WroUfv972Hbb\ntNHL2LFpQZpvfSvrytQEDHdJyptnnoHvfz/t3DZgQFo69sADs65KTchwl6S8+PBDGDUKdtklLUwz\neTJccgmss07WlamJOaBOkvLgoYfg2GPh1VfhnHPSKPg2bbKuShmxc5ekYjZvHhx1FJSVwRZbwLRp\ncMopBnsLZ+cuScUoRrjtNhg6FBYvhj/9CY4+GlrZs8nOXZKKz9y5cPDBace2vfdO09v69zfY9Rm/\nEySpWNTUwOWXQ+fO8NxzcM898Le/waabZl2ZmhnDXZKKwfTpsPvuMHx4WoRm5sy057q0Eoa7JDVn\nH38Mp50GXbrAe+/BpElw5ZXwzW9mXZmaMQfUSVJz9eijaXrbv/+dAv7kk2H11bOuSkXAzl2Smpt3\n302hvs8+sOGG8OyzcOaZBrtWmZ27JDUXMcJdd8GQIbBoEVx1VVo+1lHwKpDfMZLUHLz6KvTsCYcd\nltaFnzkTBg0y2FUnftdIUpaWLYOrr07T26ZMgTvugLvvhs03z7oyFTHDXZKyMmsW7LUXHHcc/Oxn\nqVs/9FD3Wle9Ge6S1NQ++QTOPht23BHeegseeQSuvRbatcu6MuWEA+okqSk98QQccwxUVaXtWU87\nDdZYI+uqlDN27pLUFN57DwYPhj32SPurV1bCr39tsKtR2LlLUmO77750X/3dd+HSS1PIt26ddVXK\nMTt3SWosb7wBhx+e1oDfYQd4/vm0RavBrkZm5y5JDS1GuO46OOkkaNMGbr0VjjzSUfBqMnbuktSQ\nqqth333ToLmf/CRNdysvN9jVpAx3SWoIS5bA+efDdtvB3Lnwj3/ADTfABhtkXZlaIC/LS1J9Pflk\n6tSffx5OOAHOOgvWWivrqtSC1atzDyGcHEJYFkK4uKEKkqSi8cEHMGIE7LZbGiT35JNw4YUGuzJX\n5849hLALcCzwXMOVI0lFYty4tLHLW2/Bb38Lw4fDal4MVfNQp849hLAOMAboD7zboBVJ0teoqqpi\n3LhxVFdXN/2bv/UW9O4NPXpASQnMmAEjRxrsalbqeln+SuD+GOPDDVmMJH2V+fPn0737QXTs2JEe\nPXrQoUMHunc/iAULFjT+m8cIN90EnTrB+PHp8wcfhK22avz3lgpUcLiHEI4EdgRGN3w5kvTlevXq\ny8SJk0kXDucCY5g4cTLl5X0a941fegkOOAD69YMDD4QXXoCf/9zpbWq2CrqOFELYHLgUKIsxLmmc\nkiTpf1VVVTFhwlhSsPeuPdqbmprIhAl9qa6upqSkpGHfdOnStFzsGWfAt74FY8emcJeauUJvEpUC\nGwGVIXz2J2trYK8QwhBg9RhjXPFJI0aMoG3btssdKy8vp7y8vA4lS2qJ5syZU/vZXiv8y94AzJ49\nu2HDvbIyTW979tm0ZOy556YNX6RGUFFRQUVFxXLHFi5cWOfXCyvJ4i9/cAhrA99Z4fCNwCzgghjj\nrBUe3wWYOnXqVLp06VLnIiWpqqqKjh07snznTu3XfamqqmqYcP/wwzRP/eKLoXNn+POfYddd6/+6\nUoEqKyspLS0FKI0xVhby3II69xjjImDmF4+FEBYB81YMdklqSB06dKBbtx5MnDiUmppI6tgfpXXr\nYZSV9WiYYJ84EQYMgFdfTZ36yJFpbXipyDTE8rOr3vpLUj1UVIyhrKwr0BfYEuhLWVlXKirG1O+F\n581Lg+X23x+23BKmTYPRow12Fa16T8yMMe7bEIVI0tdp164d48c/QHV1NbNnz6Z9+/b169hjhIqK\ntADNkiXpEvzRRzsKXkXPVRckFZ2SkpL6X4Z/+eW0wty4cWnP9csvh002aZgCpYy5K5yklqWmBi67\nDLbZBqZPh/vug9tvN9iVK4a7pJZj2jT4wQ/SZi/9+qVd3A45JOuqpAZnuEvKv48/hlNPhdLStJPb\npEnwhz/AN7+ZdWVSo/Ceu6R8e/TRtBjNyy/D6afDqFGw+upZVyU1Kjt3Sfm0YEEK9X32SUvHPvts\nWkbWYFcLYOcuKV9ihDvvhOOPh0WL4Oqr4dhjoZW9jFoOv9sl5cerr0LPnmlq2/e/D7NmwcCBBrta\nHL/jJRW/ZctSh96pE0yZAnfcAXffDd/+dtaVSZkw3CUVt5kzYa+94LjjoLw8deuHHuoqc2rRDHdJ\nxemTT9LubTvuCG+/nUbF//GPsN56WVcmZc4BdZKKz+OPp5Hw1dVw8slpDvsaa2RdldRs2LlLKh7v\nvZcuv++xB6y7LlRWpq1ZDXZpOXbukorDvffC4MHw7rtpbfjBg6F166yrkpolO3dJzdvrr6epbT/5\nCeywQxpAN3SowS59BTt3Sc1TjHDddTByJHzjG2nf9Z/9zFHw0iqwc5fU/FRVwQ9/mAbN9eyZprcd\neaTBLq0iw11S87FkCZx3Hmy/PbzyCkycCDfcABtskHVlUlHxsryk5uHJJ6F//3RPfeTItMnLWmtl\nXZVUlOzcJWXrgw9g+HDo2jXdW3/qKbjgAoNdqgc7d0nZGTsWBg1KK8z97ncwbBis5q8lqb7s3CU1\nvbfegl694KCDoGNHmDEDTjzRYJcaiD9JkppOjHDzzXDCCWnk+803Q58+joKXGpidu6SmMWcOHHAA\n9OsHBx6Yprf17WuwS43AcJfUuJYuTffTt9subfQybhyMGQMbbZR1ZVJuGe6SGk9lJey6a9q5beDA\ndG+9e/esq5Jyz3CX1PA+/BBOOikF+7JlMHkyXHwxrLNO1pVJLYID6iQ1rIkTYcAAeO01+PWv0yj4\nNm2yrkpqUezcJTWMefPSYLn994fvfAemTUuX4w12qcnZuUuqnxjTjm3Dh6e14a+7Do46ylHwUobs\n3CXV3csvp4VoevdOu7jNmgVHH22wSxkz3CUVrqYGLrsMttkGpk+H++6D226DTTbJujJJGO6SCjVt\nGvzgBzBiRLr8PnMmHHJI1lVJ+gLDXdKq+fhjOPVUKC2FRYvg8cfhiitg3XWzrkzSChxQJ+nrPfII\nHHtsusd+xhkwalTanlVSs2TnLunLLVgAxxyTBsttvDE89xycfrrBLjVzdu6S/leMcOedMGQIfPQR\nXHNNCvlW9gNSMfAnVdLyXnkFfvITOPzwNHBu5sy04pzBLhUNf1olJcuWwVVXQefO8NRTqXO/6y74\n9rezrkxSgQx3Sak733NPGDwYevVKX//0p1lXJamODHepJfvkEzjrLNhxx7Q2/GOPpfvr662XdWWS\n6qGgcA8hDAwhPBdCWFj78UQIwc2ZpWL0+OOw005w3nlpg5dnn03du6SiV2jn/l9gFNAFKAUeBu4N\nIXRq6MIkNZL33oPjjoM99oC2baGyEs45B9ZYY7mHVVVVMW7cOKqrqzMqVFJdFRTuMcYHYozjY4xz\nYoyzY4ynAR8AXRunPEkN6t5704C5W25Jq8tNmgTbbrvcQ+bPn0/37gfRsWNHevToQYcOHeje/SAW\nLFiQUdGSClXne+4hhFYhhCOBtYB/NVxJkhrc66/DYYelKW477ZQGzA0ZAq1b/89De/Xqy8SJk4Ex\nwFxgDBMnTqa8vE9TVy2pjgpexCaEsC0pzNcA3gd6xhhfaOjCJDWAZcvS/uonnQSrrw5//SscccSX\nbslaVVXFhAljScHeu/Zob2pqIhMm9KW6upqSkpKmql5SHdVlhboXgB2AtsBhwM0hhL2+KuBHjBhB\n27ZtlztWXl5OeXl5Hd5e0ip58cW0Hvxjj6U91n/3O1h//a98ypw5c2o/22uFf9kbgNmzZxvuUiOo\nqKigoqJiuWMLFy6s8+uFGGO9Cgoh/AOYHWMctJJ/6wJMnTp1Kl26dKnX+0haRYsXpyA/91zYfHO4\n9lrYd99VempVVRUdO3Zk+c6d2q/7UlVVZbhLTaSyspLS0lKA0hhjZSHPbYh57q2A1RvgdSTV15Qp\nsPPOcOaZMHw4TJ++ysEO0KFDB7p160Hr1kNJgf5fYAytWw+jW7ceBrtUJAqd535eCGHPEMJ3Qgjb\nhhDOJ12vG9M45UlaJR98kMJ8t93Sjm1PPw0XXABrrlnwS1VUjKGsrCvQF9gS6EtZWVcqKvwxl4pF\noffcvwXcBGwKLASmAQfEGB9u6MIkraKxY2HQIHjnHfj972HoUFit7hs+tmvXjvHjH6C6uprZs2fT\nvn17O3apyBT0GyDG2L+xCpFUoLfeSt16RQUccEBaNvZ732uwly8pKTHUpSLlfu5SsYkRbroJTjwx\nTWm75Rbo3ftLp7dJanncOEYqJnPmwP77w1FHQY8eMGsW9OljsEtajuEuFYOlS+HCC2G77VLAjx+f\nOvaNNsq6MknNkOEuNXeVlbDrrjB6dBo4N2MGdOuWdVWSmjHDXWquPvwwLRu7yy5pGdkpU+Cii2Dt\ntbOuTFIz54A6qTn6xz9gwIC04ct558EJJ0CbNllXJalI2LlLzcm8efCLX6Spbd/9blphbtQog11S\nQezcpeYgxjRffdgwqKmB66+Hfv0cBS+pTuzcpay9/HKa1ta7N+y3X5redtRRBrukOjPcpazU1MCl\nl8I226QR8Pffn/Zb33jjrCuTVOQMdykLzz2XNnk54YTUpc+cCQcfnHVVknLCcJea0kcfwSmnpG1Z\nP/wQHn8crrgC1l0368ok5YgD6qSm8s9/wrHHwty5ab/1X/0qbc8qSQ3Mzl1qbAsWQP/+sO++sOmm\n6ZL8aacZ7JIajZ271FhihDvugOOPT5fjr7kGjjkGWvk3taTG5W8ZqTG88gr8+MdwxBGw++5petuA\nAQa7pCbhbxqpIS1bBldeCZ07w9NPw113wZ13wmabZV2ZpBbEcJcaysyZsOeeMGQI9OqVvu7ZM+uq\nJLVAhrtUX598AmedBTvumNaGf+yxdH99vfWyrkxSC+WAOqk+Jk1K09tmz4aTT05z2NdYI+uqJLVw\ndu5SXSxcCIMGpcvwbdtCZSWcc47BLqlZsHOXCnXPPTB4MLz3XlpdbtAgaN0666ok6TN27tKqeu01\nOPTQNEiuS5c0YG7IEINdUrNjuEtfZ9kyuPbaNL1t0iS47Ta47z7YYousK5OklTLcpa/y4ovwwx+m\nBWgOPTQtRnPEEe61LqlZM9yllVm8GH7zG9hhB3j1VXjoIbjuOlh//awrk6Sv5YA6aUVTpqSNXmbN\ngpNOgjPOgDXXzLoqSVpldu7Sp95/H4YNg912S1Pann4azj/fYJdUdOzcJYAHHkhT2ubNg4suSju5\nreaPh6TiZOeulu3NN6G8HA4+OI2GnzEDRoww2CUVNX+DqWWKEW68EU48MW3Desst0Lu3o+Al5YKd\nu1qeOXNg//3h6KPhoIPSwLk+fQx2SblhuKvlWLoULrwQtt02Bfz48alj32ijrCuTpAZluKtlmDoV\ndtkFRo+G445L99a7dcu6KklqFIa78m3RIhg5EnbdNX09ZUoaDb/22tnWJUmNyAF1yq8HH4SBA+H1\n1+G88+CEE6BNm6yrkqRGZ+eu/HnnHfj5z9Nl9+99D6ZPh1GjDHZJLYadu/IjRrj1Vhg+HGpq4Prr\noV8/R8FLanHs3JUP//kPHHhgmtK2335pettRRxnsklokw13FraYGLrkEttkGnn8e7r8f/vpX2Hjj\nrCuTpMwY7ipezz0HXbumVeZ++UuYOTMtIytJLVxB4R5CGB1CeDKE8F4I4c0Qwt0hhA6NVZy0Uh99\nlOarl5amz594Ai6/HNZdN+vKJKlZKLRz3xO4Avg+UAa0AR4MIbgnpuqtqqqKcePGUV1d/eUP+uc/\nYfvt4eKL4ayzoLIyde+SpM8UFO4xxh4xxltijLNijNOBfsCWQGljFKeWYf78+XTvfhAdO3akR48e\ndOjQge7dD2LBggWfP2jBgnTpfd99YdNN0yX5006Db3wju8IlqZmq7z339YAIzG+AWtRC9erVl4kT\nJwNjgLnAGCZOnEx5eZ80ve3226FTJ7jjDrjmGnjkEfh//y/boiWpGavzPPcQQgAuBSbFGGc2XElq\nSaqqqpgwYSwp2HvXHu1NTU3k+Ql9+aCsjHUefhh++lO44grYbLMMq5Wk4lCfRWyuAjoDu3/dA0eM\nGEHbtm2XO1ZeXk55eXk93l55MGfOnNrP9vrsWCtqGMS/OR9Y7dln4a67oGfPTOqTpKZQUVFBRUXF\ncscWLlxY59cLMcbCnxTCH4BDgD1jjHO/4nFdgKlTp06lS5cudS5S+VVVVUXHjh35tHPvzPP8mf7s\nxmSuBg6YOpWt/d6R1AJVVlZSWloKUBpjrCzkuQXfc68N9h8DP/yqYJdWRYcOHejWrQdrtjqes+jJ\nM+zEesxln1brcm+3Hga7JNVBofPcryLdGO0FLAohbFz7sUajVKcW4fZhQ3hhzcWM5h4uYAk78Rpr\n7L8nFRVjsi5NkopSoffcB5JGxz+ywvGjgJsboiC1IAsXwskn881rruGbXbvy8umns0sITG/fnpKS\nkqyrk6SiVVC4xxhdrlYN4+67YcgQeO+9NAp+0CC+07o138m6LknKAcNaTeu11+DQQ9PUti5d0nrw\nQ4ZA69ZZVyZJuWG4q2ksWwbXXgudO8OkSXDbbXDffbDFFllXJkm5Y7ir8b34IuyzDwwYkLr2WbPg\niCPca12SGonhrsazeDGce27a6OW11+Chh+C662D99bOuTJJyrT4r1ElfbvJk6N8fXngBTjoJzjgD\n1nTzQElqCnbualjvvw9Dh8IPfpDC/Omn4fzzDXZJakJ27mo4f/87HHcczJsHF12UQt5R8JLU5Ozc\nVX9vvglHHgmHHJJGwz//PIwYYbBLUkbs3FV3McKNN8KJJ6YgHzMGevVyFLwkZczOXXUzezaUlcHR\nR8PBB6fpbb17G+yS1AwY7irMkiXw29/CdtvBSy/BhAlw882w4YZZVyZJqmW4a9VNnQq77gqnnAKD\nB8OMGXDAAVlXJUlageGur7doUbqvvuuu6espU+D3v4e11862LknSSjmgTl9twgQYOBDeeAPOOw9O\nOAHatMm6KknSV7Bz18q98w707Qvdu8NWW8H06TBqlMEuSUXAzl3LixH+8hcYPjzt5Hb99dCvn6Pg\nJamI2Lnrc//+Nxx4YOrY998/TW876iiDXZKKjOEuWLoULr4Ytt0WZs5My8hWVMDGG2ddmSSpDgz3\nlu6552C33WDkyLSL2/PPw0EHZV2VJKkeDPeW6qOPYPRoKC2Fjz+GJ56Ayy6DddfNujJJUj05oK4l\nevhhGDAA5s6Fs86CX/0KvvGNrKuSJDUQO/eWZP78tBb8fvvBZpvBtGlw2mkGuyTljJ17SxAj3H57\n2l/9k0/gj39M99db+bedJOWRv93zbu7ctM/6kUfCHnuk0fDHHmuwS1KO+Rs+r2pq4IorYJttoLIS\n7roL7rwzXY6XJOWa4Z5HM2akLn3oUOjTJy1G07Nn1lVJkpqI4Z4nH38MZ5wBXbrAu+/CY4/B1VdD\n27ZZVyZJakIOqMuL//s/OOYYeOmlNH/9lFNg9dWzrkqSlAE792K3cGHaknWvvWD99eGZZ+Dssw12\nSWrB7NyL2d13w+DB8P778Ic/wKBBjoKXJNm5F6XXXoOf/jR97Lxzmt42eLDBLkkCDPfismwZXHMN\ndOqU1oK//Xa4917YYousK5MkNSOGe7F44QXYe+906f3ww9P0tsMPd691SdL/MNybu8WL4dxzYYcd\n4I030qYvf/4ztGuXdWWSpGbKAXXN2b/+laa3vfBC2rnt9NNhzTWzrkqS1MzZuTdH778Pxx8Pu++e\nwnzqVDjvPINdkrRK7Nybm7//Pd1Xnz8fLrooLSHbunXWVUmSioide3Px5pvws5+lHdy22Qaefx5G\njDDYJUkFs3PPWoxwww0wcmQK8jFjoFcvR8FLkurMzj1L1dWw337wy1+mjn3WLOjd22CXJNVLweEe\nQtgzhHBfCOHVEMKyEMKPGqOwXFuyBC64ALbfHv7zH5gwAW66CTbcMOvKJEk5UJfOfW3gWeA4IDZs\nOS3AU0/BLrvAqaemJWOnT4cDDsi6KklSjhR8zz3GOB4YDxCC149XVfWzz7La2Wfz3fvuI2y/PTz5\nJJSWZl2WJCmHvOfeyObPn0+Pbj34ZKed2OSeexi1bBkHbbQJC7baKuvSJEk55Wj5RtarV18mPjSF\nNRjGNA5lDnNp/fBQysv7MH78A1mXJ0nKIcO9EVVVVTFhwlhgDHfT+7PjNTWRCRP6Ul1dTUlJSXYF\nSpJyqUnCfcSIEbRt23a5Y+Xl5ZSXlzfF22dmzpw5tZ/ttcK/7A3A7NmzDXdJEhUVFVRUVCx3bOHC\nhXV+vSYJ90suuYQuXbo0xVs1K1tvvXXtZ4/BFzp3eBSA9u3bN3VJkqRmaGUNb2VlJaV1HHhdcLiH\nENYG2gNfmuAEAAAHO0lEQVSfjpTfKoSwAzA/xvjfOlWRUx06dKBbtx5MnDiUmppI6tgfpXXrYZSV\n9bBrlyQ1irqMlt8ZeAaYSprnfhFQCZzdgHXlRkXFGMrKugJ9gS2BvpSVdaWiYkzGlUmS8qou89wf\nxSl0q6xdu3aMH/8A1dXVzJ49m/bt29uxS5IalaPlm0hJSYmhLklqEnbgkiTljOEuSVLOGO6SJOWM\n4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEu\nSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklS\nzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y\n7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4d6EKioqsi6hWfA8fM5zkXgePue5SDwP9VOncA8h\nDA4h/DuE8FEIYXIIYZeGLiyP/GZNPA+f81wknofPeS4Sz0P9FBzuIYSfARcBZwI7Ac8BE0IIGzZw\nbZIkqQ7q0rmPAP4YY7w5xvgCMBD4EDi6QSuTJEl1UlC4hxDaAKXAQ58eizFGYCKwW8OWJkmS6mK1\nAh+/IdAaeHOF428CHVfy+DUAZs2aVXhlObRw4UIqKyuzLiNznofPeS4Sz8PnPBeJ52G57Fyj0OeG\n1Hiv4oND2BR4FdgtxjjlC8d/C+wVY9xthcf3Av5SaFGSJOkzvWOMtxbyhEI793eAGmDjFY5vDLyx\nksdPAHoD/wE+LvC9JElqydYAvkvK0oIU1LkDhBAmA1NijMNqvw7AXODyGOPvCi1AkiQ1rEI7d4CL\ngRtDCFOBJ0mj59cCbmzAuiRJUh0VHO4xxttr57SfQ7oc/yzQLcb4dkMXJ0mSClfwZXlJktS8uba8\nJEk5Y7hLkpQzjRrubjADIYQ9Qwj3hRBeDSEsCyH8KOuashBCGB1CeDKE8F4I4c0Qwt0hhA5Z19XU\nQggDQwjPhRAW1n48EULonnVdWQshnFz783Fx1rU0tRDCmbX/7V/8mJl1XVkJIWwWQrglhPBOCOHD\n2p+XLlnX1ZRqc3PF74llIYQrVvU1Gi3c3WDmM2uTBh0eB7TkAQ57AlcA3wfKgDbAgyGENTOtqun9\nFxgFdCEt5fwwcG8IoVOmVWWo9o/+Y0m/I1qqGaQBypvUfuyRbTnZCCGsBzwOfAJ0AzoBJwILsqwr\nAzvz+ffCJsD+pPy4fVVfoNEG1H3JfPj/kubDX9gob9rMhRCWAT+JMd6XdS1Zq/0j7y3SyoaTsq4n\nSyGEecDIGOMNWdfS1EII6wBTgUHA6cAzMcYTsq2qaYUQzgR+HGNsUd3pyoQQLiCtgLp31rU0JyGE\nS4EeMcZVvtrZKJ27G8xoFaxH+kt0ftaFZCWE0CqEcCRpnYh/ZV1PRq4E7o8xPpx1IRkrqb11NyeE\nMCaEsEXWBWXkEODpEMLttbfvKkMI/bMuKku1edobuK6Q5zXWZfmv2mBmk0Z6TxWJ2qs4lwKTYowt\n7t5iCGHbEML7pEuPVwE9a7dPblFq/7DZERiddS0Zmwz0I12GHgh8D3gshLB2lkVlZCvSVZwXgQOA\nq4HLQwh9M60qWz2BtsBNhTypLivUSfV1FdAZ2D3rQjLyArAD6Qf2MODmEMJeLSngQwibk/7AK4sx\nLsm6nizFGL+4bviMEMKTwMvAEUBLu1XTCngyxnh67dfPhRC2Jf3Rc0t2ZWXqaGBcjHFl+7d8qcbq\n3AvdYEYtRAjhD0APYJ8Y4+tZ15OFGOPSGONLMcZnYoynkgaSDcu6riZWCmwEVIYQloQQlgB7A8NC\nCItrr+60SDHGhUAV0D7rWjLwOrDiHuGzgC0zqCVzIYQtSQOQ/1Tocxsl3Gv/Ep8K7Pfpsdof1v2A\nJxrjPdX81Qb7j4EfxhjnZl1PM9IKWD3rIprYRGA70mX5HWo/ngbGADvEFrx0Zu0gw/akoGtpHgc6\nrnCsI+lKRkt0NOl29thCn9iYl+XdYAaovW/WHvi0E9kqhLADMD/G+N/sKmtaIYSrgHLgR8CiEMKn\nV3UWxhhbzHbAIYTzgHGknRTXJQ2U2Zt0f7HFiDEuApYbbxFCWATMizGu2LnlWgjhd8D9pAD7NnA2\nsASoyLKujFwCPB5CGE2a9vV9oD9wTKZVZaC2Ie4H3BhjXFbo8xst3N1g5jM7A/8kjQyPpLn/kAZH\nHJ1VURkYSPrvf2SF40cBNzd5Ndn5Fun//abAQmAacICjxYGWuw7E5sCtwAbA28AkoGuMcV6mVWUg\nxvh0CKEncAFpauS/gWExxr9mW1kmyoAtqOO4CzeOkSQpZ1xbXpKknDHcJUnKGcNdkqScMdwlScoZ\nw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJz5/4VLaJPfiTa/AAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105e66198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "gd_lr = LinearRegression(eta=0.005, \n",
    "                         epochs=100,\n",
    "                         minibatches=1,\n",
    "                         random_seed=123,\n",
    "                         print_progress=3)\n",
    "gd_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % gd_lr.w_)\n",
    "print('Slope: %.2f' % gd_lr.b_)\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, gd_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3X2YXWV57/HvDQyQKDOUBpNDi0VK5KXVSFKq2FZbkUZQ\nQU+lMEhFaH1BPbaxLa3XUVGs4qmXiaCkgrZFBOOhckRR2lDUek4lKSWBtJUQ3wK+YCAIBiSBQHKf\nP9baujPOJGsme2Y9O/P9XNe61uxnP2vte++HML951tprRWYiSZKkXdur7QIkSZL6hcFJkiSpIYOT\nJElSQwYnSZKkhgxOkiRJDRmcJEmSGjI4SZIkNWRwkiRJasjgJEmS1JDBSZIkqaFiglNEvDEi1kfE\nlohYGRHH7aTvyyPixoi4LyI2RcTNEfG7o/Q7LSLW1vtcExEnTe67kCRJe7IiglNEnA58ALgAOBZY\nAyyPiFljbPI84EbgJGA+8GXg+oiY17XP5wKfBD4KPAv4LHBdRBwzWe9DkiTt2aKEm/xGxErg3zLz\nj+vHAXwXuCQz/7rhPv4L+FRm/lX9+FPAzMw8pavPCuC2zHxDr9+DJEna87U+4xQRA8AC4IudtqzS\n3E3A8Q33EcABwANdzcfX++i2vOk+JUmSRmo9OAGzgL2Be0e03wvMabiPPweeBFzT1TZnN/cpSZK0\ng33aLmB3RcSZwNuBUzLz/t3c188DC4G7gEd3vzpJkjTF9gcOA5Zn5g97vfMSgtP9wDZg9oj22cCG\nnW0YEWcAlwOvyMwvj3h6wwT2uRC4elcFS5Kk4r2S6ktiPdV6cMrMxyNiFXAC8Dn4yTlLJwCXjLVd\nRAwDHwNOz8x/GqXLilH2cWLdPpa7AK666iqOPvrocbyLifnRj+CEE+D974cXvGDSX67vLFq0iCVL\nlrRdhmqOR1kcj7I4HuVYu3YtZ511FtS/03ut9eBUWwxcUQeoW4BFwEzgCoCIuAg4JDPPrh+fWT/3\nZuDfI6Izs7QlMx+qf74Y+JeIeAvwBWCY6iT01+ykjkcBjj76aObPn9+zNzeWhx+u1k99KkzBy/Wd\noaGhKRkHNeN4lMXxKIvjUaRJOeWmhJPDycxrgD8DLgRuA54JLMzMjXWXOcChXZu8huqE8kuBe7qW\nD3btcwVwJvBa4HbgvwOnZuYdk/pmxmFgoFpv3dpuHZIkqZlSZpzIzKXA0jGeO2fE499puM9rgWt3\nv7rJ0QlOjz/ebh2SJKmZImacpqu994a99nLGSZKkfmFwatm++zrjNJbh4eG2S1AXx6MsjkdZHI/p\nw+DUsoEBZ5zG4v+IyuJ4lMXxKIvjMX0YnFrmjJMkSf3D4NQyZ5wkSeofBqeWOeMkSVL/MDi1zBkn\nSZL6h8GpZQMDzjhJktQvDE4t23dfZ5wkSeoXBqeWOeMkSVL/MDi1zBknSZL6h8GpZc44SZLUPwxO\nLfNyBJIk9Q+DU8u8HIEkSf3D4NQyZ5wkSeofBqeWOeMkSVL/MDi1zBknSZL6h8GpZc44SZLUPwxO\nLXPGSZKk/mFwapkzTpIk9Q+DU8u8AKYkSf3D4NQyb7kiSVL/MDi1zBknSZL6h8GpZc44SZLUPwxO\nLXPGSZKk/mFwapkzTpIk9Q+DU8uccZIkqX8YnFrmBTAlSeofBqeWdWacMtuuRJIk7YrBqWX77lut\nn3ii3TokSdKuGZxaNjBQrT1BXJKk8hmcWtaZcfI8J0mSymdwapkzTpIk9Q+DU8s6wckZJ0mSymdw\nalnnUJ0zTpIklc/g1LL99qvWjz3Wbh2SJGnXDE4tO+CAav3ww+3WIUmSds3g1LLBwWr90EPt1iFJ\nknbN4NQyg5MkSf3D4NSyTnDatKndOiRJ0q4ZnFo2MAAzZjjjJElSPzA4FWBw0OAkSVI/MDgVwOAk\nSVJ/MDgVYGjIc5wkSeoHBqcCOOMkSVJ/MDgVwOAkSVJ/MDgVwOAkSVJ/MDgVwOAkSVJ/MDgVwJPD\nJUnqDwanAjjjJElSfzA4FaATnDLbrkSSJO2MwakAg4OwbRts2dJ2JZIkaWcMTgUYGqrWnuckSVLZ\nDE4FGBys1p7nJElS2QxOBTA4SZLUHwxOBTA4SZLUHwxOBegEJ89xkiSpbAanAjjjJElSfzA4FWBg\nAGbMMDhJklQ6g1MhvHq4JEnlMzgVwuAkSVL5DE6F8Ea/kiSVz+BUCGecJEkqn8GpEAYnSZLKZ3Aq\nhMFJkqTyGZwK4TlOkiSVz+BUCGecJEkqn8GpEAYnSZLKZ3AqRCc4ZbZdiSRJGovBqRCDg7BtG2ze\n3HYlkiRpLMUEp4h4Y0Ssj4gtEbEyIo7bSd85EXF1RKyLiG0RsXiUPmdHxPb6+e31UmwsGRqq1h6u\nkySpXEUEp4g4HfgAcAFwLLAGWB4Rs8bYZD/gPuDdwO072fUmYE7X8ku9qrnXBgertcFJkqRyFRGc\ngEXAZZl5ZWbeCbwe2AycO1rnzLw7Mxdl5lXAzqJGZubGzLyvXjb2vvTeMDhJklS+1oNTRAwAC4Av\ndtoyM4GbgON3c/dPjoi7IuI7EXFdRByzm/ubNAYnSZLK13pwAmYBewP3jmi/l+rw2kSto5qxOgV4\nJdV7vTkiDtmNfU6azjlOXgRTkqRy7dN2AZMlM1cCKzuPI2IFsBZ4HdW5VGNatGgRQ50kUxseHmZ4\neHgSKq0ccEC1dsZJkqRmli1bxrJly3Zo2zTJMxAlBKf7gW3A7BHts4ENvXqRzHwiIm4DjthV3yVL\nljB//vxevXQjAwMwY4bBSZKkpkab1Fi9ejULFiyYtNds/VBdZj4OrAJO6LRFRNSPb+7V60TEXsAz\ngB/0ap+95tXDJUkqWwkzTgCLgSsiYhVwC9W37GYCVwBExEXAIZl5dmeDiJgHBPBk4OD68dbMXFs/\n/3aqQ3XfBA4EzgeeCnxsit7TuA0Oeo6TJEklKyI4ZeY19TWbLqQ6RHc7sLDr8gFzgENHbHYb0LlB\nyXzgTOBu4PC67eeAy+ttH6Sa1Tq+vtxBkYaGnHGSJKlkRQQngMxcCiwd47lzRmnb6WHGzHwL8Jbe\nVDc1PFQnSVLZWj/HST9lcJIkqWwGp4IYnCRJKpvBqSBDQ54cLklSyQxOBXHGSZKkshmcCmJwkiSp\nbAangnSCU+au+0qSpKlncCrI0BBs2wabN7ddiSRJGo3BqSCDg9Xaw3WSJJXJ4FQQg5MkSWUzOBXE\n4CRJUtkMTgXpBCev5SRJUpkMTgUZGqrWzjhJklQmg1NBDjigWhucJEkqk8GpIAMDMGOGwUmSpFIZ\nnArj1cMlSSqXwakw3uhXkqRyGZwK44yTJEnlMjgVxuAkSVK5DE6FMThJklQug1NhBgc9x0mSpFIZ\nnAozNOSMkyRJpTI4FcZDdZIklcvgVBiDkyRJ5TI4FaYTnDLbrkSSJI1kcCrM0BBs2wabN7ddiSRJ\nGsngVJjBwWrt4TpJkspjcCqMwUmSpHIZnApjcJIkqVwGp8IMDVVrL4IpSVJ5DE6FccZJkqRyGZwK\nc8AB1drgJElSeQxOhRkYgBkzDE6SJJXI4FQgb/QrSVKZDE4F8ka/kiSVyeBUIO9XJ0lSmQxOBTI4\nSZJUJoNTgTzHSZKkMhmcCuQ5TpIklcngVCAP1UmSVCaDU4EMTpIklcngVCCDkyRJZTI4FagTnDLb\nrkSSJHUzOBVoaAi2bYPNm9uuRJIkdTM4FWhwsFp7uE6SpLIYnAr08z9fre+7r906JEnSjgxOBfrl\nX67W3/xmu3VIkqQdGZwKNGtWdZ7TN77RdiWSJKmbwalAETB3rsFJkqTSGJwKZXCSJKk8BqdCGZwk\nSSqPwalQc+fChg3w8MNtVyJJkjoMToWaO7da+806SZLKYXAqVCc4ebhOkqRyGJwKddBB1WJwkiSp\nHAangnmCuCRJZTE4FczgJElSWQxOBTM4SZJUFoNTwebOhY0bYdOmtiuRJElgcCqa36yTJKksBqeC\nGZwkSSqLwalgQ0Nw8MEGJ0mSSmFwKpwniEuSVI4JBaeIeEdEzBylfUZEvGP3y1KHwUmSpHJMdMbp\nAuDJo7TPrJ9TjxicJEkqx0SDUwA5Svs84IGJl6OR5s6FBx6oFkmS1K59xtM5Ih6kCkwJfD0iusPT\n3lSzUB/pXXnq/mbds5/dbi2SJE134wpOwJ9QzTb9HdUhue5LM24F7srMFT2qTcARR1Rrg5MkSe0b\nV3DKzI8DRMR64KuZ+cSkVKWfOOAAmDPH85wkSSrBRM9xehg4uvMgIk6NiOsi4r0RsW9vSlOHJ4hL\nklSGiQany4CnA0TE4cD/BjYDpwF/PZEdRsQbI2J9RGyJiJURcdxO+s6JiKsjYl1EbIuIxWP0Oy0i\n1tb7XBMRJ02ktrYZnCRJKsNEg9PTgdvrn08DvpKZZwKvBn5vvDuLiNOBD1CdN3UssAZYHhGzxthk\nP+A+4N1ddYzc53OBTwIfBZ4FfBa4LiKOGW99besEpxzte4ySJGnK7M7lCDrbvhC4of75u8BYYWdn\nFgGXZeaVmXkn8HqqGaxzR+ucmXdn5qLMvAp4aIx9vhn4x8xcnJnrMvMdwGrgTROor1Vz58KmTXD/\n/W1XIknS9DbR4HQr8LaI+APg+cAX6vanAfeOZ0cRMQAsAL7YacvMBG4Cjp9gfdTb3jSibflu7rMV\n3uxXkqQyTDQ4/QkwH/gw8J7M/Gbd/grg5nHuaxbVNaBGBq57gTkTrI96217vsxXdlySQJEntGe91\nnADIzP8AnjHKU38ObNutivQzZs6EX/gFg5MkSW2bUHDqiIgF/PSyBHdk5uoJ7OZ+qrA1e0T7bGDD\nbpS3YaL7XLRoEUNDQzu0DQ8PMzw8vBvl7B6/WSdJ0o6WLVvGsmXLdmjbtGnTGL17Y0LBKSKeQnUJ\ngucDP6qbD4yILwNnZObGpvvKzMcjYhVwAvC5ev9RP75kIvXVVoyyjxPr9p1asmQJ8+fP342X7r25\nc+HWW9uuQpKkcow2qbF69WoWLFgwaa850XOcPkR1X7pfycyDMvMg4FeBQSYWdhYDr4mIV0XEUVT3\nu5sJXAEQERdFxMe7N4iIeRHxrLqOg+vHR3d1uRh4UUS8JSKOjIh3Up2E/uEJ1Nc6L0kgSVL7Jnqo\n7kXACzNzbachM++IiDcCN453Z5l5TX3NpgupDqfdDizsmrmaAxw6YrPbqG42DNWJ6mcCdwOH1/tc\nERFnAu+pl28Ap2bmHeOtrwRz58KPfwz33FOd7yRJkqbeRIPTXsDjo7Q/zgRnsTJzKbB0jOfOGaVt\nl6+TmdcC106kntL8xm/APvvAtdfCm9/cdjWSJE1PEz1U9yXg4og4pNMQEb8ALKHrekzqnYMPhpe8\nBK64ou1KJEmaviYanN5EdT7TXRHxrYj4FrC+bvsfvSpOOzrnHLjtNlizpu1KJEmaniZ6HafvRsR8\nqtutHFU3r83MkVfqVg+ddFI18/Txj8PiUW9rLEmSJtO4Zpwi4gURcUdEDGblnzPzQ5n5IeDfI+Jr\nEbFwkmqd9gYG4Kyz4Kqr4PHRzjCTJEmTaryH6v4E+Ghm/syNdTNzE3AZHqqbVK9+NWzcCDfcsMuu\nkiSpx8YbnOYB/7ST528EnjnxcrQrz3wmHHusJ4lLktSG8Qan2Yx+GYKOJ4CDJ16OmjjnHPj856uZ\nJ0mSNHXGG5y+T3WF8LE8E/jBxMtRE8PDEAGf/GTblUiSNL2MNzjdALw7IvYf+UREzADeBXy+F4Vp\nbLNmwUtf6uE6SZKm2niD018BBwFfj4jzI+LUevkLYF393Ht6XaR+1qtfDbffXi2SJGlqjCs4Zea9\nwHOB/wIuAj5TL++t236z7qNJ9qIXwVOe4qyTJElTadxXDs/MuzPzZGAW8GzgOcCszDw5M9f3ukCN\nbmAA/uAP4Oqr4ZFH2q5GkqTpYaK3XCEzH8zMf8/MWzLzwV4WpWZe9zp49FE49VTYsqXtaiRJ2vNN\nODipfXPnwhe+ACtWwMtfXoUoSZI0eQxOfe55z4Prr4evfAVOOw22bm27IkmS9lwGpz3AC14A110H\nN94IZ5zhfewkSZosBqc9xMKFcO211RXFzzwTfuBlSCVJ6jmD0x7kJS+Ba66pbgB86KHVeU833ADb\ntrVdmSRJewaD0x7mZS+D738fLrkE1q+HF78YnvY0eNvb4DOfgbVrPQ9KkqSJ2qftAtR7Bx4Ib3gD\nnHce3HorXH45LF0KD9YXjdh7bzj88OpbeQcdVPXvLIODsN9+Oy777gv77FNtN3LZa6+fLhHV0v3z\neNsma5EkqRcMTnuwCDjuuGq5/HK47z64805Yt65af+tbcPfdsGYN/OhH1fLww21XPbl6GcQme9vR\n+o3ndZtu3+t6Smjr/FzqtlPRr9fb7mx/u7uffmgb2d7m/qZi26not7vbtsXgNE1EwOzZ1fL854/d\nb/v26lDe1q3w2GPVsnVrdZ7UE09U686yfTtkVuvO0nncve4sIx/3wwK97be7247Wd3dq7FXbzsZ2\nV9t2fp7M+nr9uk23LaWftCeb6lBlcNIO9toL9t+/WiTtWSYaznbWNtFtJrptk7aR7ZPxGlNRy1Rs\nW1K/Xm17991w0UVMGoOTJE0TJR3ukCbL6tWTG5z8Vp0kSVJDBidJkqSGDE6SJEkNGZwkSZIaMjhJ\nkiQ1ZHCSJElqyOAkSZLUkMFJkiSpIYOTJElSQwYnSZKkhgxOkiRJDRmcJEmSGjI4SZIkNWRwkiRJ\nasjgJEmS1JDBSZIkqSGDkyRJUkMGJ0mSpIYMTpIkSQ0ZnCRJkhoyOEmSJDVkcJIkSWrI4CRJktSQ\nwUmSJKkhg5MkSVJDBidJkqSGDE6SJEkNGZwkSZIaMjhJkiQ1ZHCSJElqyOAkSZLUkMFJkiSpIYOT\nJElSQwYnSZKkhgxOkiRJDRmcJEmSGjI4SZIkNWRwkiRJasjgJEmS1JDBSZIkqSGDkyRJUkMGJ0mS\npIYMTpIkSQ0ZnCRJkhoyOEmSJDVkcJIkSWqomOAUEW+MiPURsSUiVkbEcbvo/9sRsSoiHo2Ir0fE\n2SOePzsitkfEtnq9PSI2T+67kCRJe7IiglNEnA58ALgAOBZYAyyPiFlj9D8M+DzwRWAecDHwsYg4\ncUTXTcCcruWXJqF8SZI0TRQRnIBFwGWZeWVm3gm8HtgMnDtG//OAb2fm+Zm5LjMvBT5d76dbZubG\nzLyvXjZO2juQJEl7vNaDU0QMAAuoZo+AKu0ANwHHj7HZc+rnuy0fpf+TI+KuiPhORFwXEcf0qGxJ\nkjQNtR6cgFnA3sC9I9rvpTq8Npo5Y/QfjIj96sfrqGasTgFeSfVeb46IQ3pRtCRJmn72abuAyZKZ\nK4GVnccRsQJYC7yO6lwqSZKkcSkhON0PbANmj2ifDWwYY5sNY/R/KDMfG22DzHwiIm4DjthVQYsW\nLWJoaGiHtuHhYYaHh3e1qSRJmiLLli1j2bJlO7Rt2rRpUl8zqtOJ2hURK4F/y8w/rh8H8B3gksx8\n/yj93weclJnzuto+CRyYmSeP8Rp7AV8DvpCZfzZGn/nAqlWrVjF//vzdfVuSJGmKrV69mgULFgAs\nyMzVvd5/Cec4ASwGXhMRr4qIo4CPADOBKwAi4qKI+HhX/48Ah0fE/4qIIyPiDcAr6v1Qb/P2iDgx\nIp4WEccCVwNPBT42NW9JkiTtaUo4VEdmXlNfs+lCqkNutwMLuy4fMAc4tKv/XRHxYmAJ8Gbge8Af\nZmb3N+1+Dri83vZBYBVwfH25A0mSpHErIjgBZOZSYOkYz50zStv/pbqMwVj7ewvwlp4VKEmSpr1S\nDtVJkiQVz+AkSZLUkMFJkiSpIYOTJElSQwYnSZKkhgxOkiRJDRmcJEmSGjI4SZIkNWRwkiRJasjg\nJEmS1JDBSZIkqSGDkyRJUkMGJ0mSpIYMTpIkSQ0ZnCRJkhoyOEmSJDVkcJIkSWrI4CRJktSQwUmS\nJKkhg5MkSVJDBidJkqSGDE6SJEkNGZwkSZIaMjhJkiQ1ZHCSJElqyOAkSZLUkMFJkiSpIYOTJElS\nQwYnSZKkhgxOkiRJDRmcJEmSGjI4SZIkNWRwkiRJasjgJEmS1JDBSZIkqSGDkyRJUkMGJ0mSpIYM\nTpIkSQ0ZnCRJkhoyOEmSJDVkcJIkSWrI4CRJktSQwUmSJKkhg5MkSVJDBidJkqSGDE6SJEkNGZwk\nSZIaMjhJkiQ1ZHCSJElqyOAkSZLUkMFJkiSpIYOTJElSQwYnSZKkhgxOkiRJDRmcJEmSGjI4SZIk\nNWRwkiRJasjgJEmS1JDBSZIkqSGDkyRJUkMGJ0mSpIYMTpIkSQ0ZnCRJkhoyOEmSJDVkcJIkSWrI\n4CRJktSQwUmSJKkhg5MkSVJDBidJkqSGDE6SJEkNFROcIuKNEbE+IrZExMqIOG4X/X87IlZFxKMR\n8fWIOHuUPqdFxNp6n2si4qTJewfqtWXLlrVdgro4HmVxPMrieEwfRQSniDgd+ABwAXAssAZYHhGz\nxuh/GPB54IvAPOBi4GMRcWJXn+cCnwQ+CjwL+CxwXUQcM2lvRD3l/4jK4niUxfEoi+MxfRQRnIBF\nwGWZeWVm3gm8HtgMnDtG//OAb2fm+Zm5LjMvBT5d76fjzcA/Zubius87gNXAmybvbUiSpD1Z68Ep\nIgaABVSzRwBkZgI3AcePsdlz6ue7LR/R//gGfSRJkhprPTgBs4C9gXtHtN8LzBljmzlj9B+MiP12\n0WesfUqSJO3UPm0XUJj9AdauXdt2HQI2bdrE6tWr2y5DNcejLI5HWRyPcnT9Dt9/MvZfQnC6H9gG\nzB7RPhvYMMY2G8bo/1BmPraLPmPtE+AwgLPOOmvnFWvKLFiwoO0S1MXxKIvjURbHoziHATf3eqet\nB6fMfDwiVgEnAJ8DiIioH18yxmYrgJGXFvjdur27z8h9nDiiz0jLgVcCdwGPNnsHkiSpIPtThabl\nk7HzqM7DbldE/D5wBdW36W6h+nbcK4CjMnNjRFwEHJKZZ9f9DwP+E1gK/B1VQPogcHJm3lT3OR74\nF+CtwBeAYeAvgfmZeccUvTVJkrQHaX3GCSAzr6mv2XQh1eG024GFmbmx7jIHOLSr/10R8WJgCdVl\nB74H/GEnNNV9VkTEmcB76uUbwKmGJkmSNFFFzDhJkiT1gxIuRyBJktQXDE618d4rT7svIt4aEbdE\nxEMRcW9EfCYinj5Kvwsj4p6I2BwR/xwRR7RR73QTEX8ZEdsjYvGIdsdjikTEIRHxiYi4v/6810TE\n/BF9HI8pEBF7RcS7I+Lb9Wf9zYh42yj9HI9JEBG/FRGfi4jv1/9fOmWUPjv97CNiv4i4tP739HBE\nfDoinjLeWgxOjP9eeeqZ3wI+BDwbeCEwANwYETM6HSLiL6huk/Na4NeBR6jGZt+pL3f6qP9weC3V\nv4XudsdjikTEgcBXgceAhcDRwJ8CD3b1cTymzl8CrwPeABwFnA+cHxE/uY2X4zGpnkR1/vMbgJ85\nx6jhZ/9B4MXA7wHPAw4Brh13JZk57RdgJXBx1+OgOuH8/LZrm04L1VXktwO/2dV2D7Co6/EgsAX4\n/bbr3VMX4MnAOuAFwJeBxY5HK+PwPuAru+jjeEzdeFwPfHRE26eBKx2PKR+L7cApI9p2+tnXjx8D\nXt7V58h6X78+ntef9jNOE7xXnibHgVR/STwAEBFPo/pGZffYPAT8G47NZLoUuD4zv9Td6HhMuZcC\nt0bENfWh7NUR8UedJx2PKXczcEJEzAWIiHnAbwA31I8dj5Y0/Ox/jepKAt191gHfYZzjU8TlCFq2\ns3vlHTn15UxP9UVPPwj8a/70khFzqIKU9xycIhFxBvAsqv/JjOR4TK3DgfOoTiN4D9Xhh0si4rHM\n/ASOx1R7H9WsxZ0RsY3qVJf/mZmfqp93PNrT5LOfDWytA9VYfRoxOKkUS4FjqP6CUwsi4hepwusL\nM/PxtusRewG3ZObb68drIuJXqS4U/In2ypq2TgfOBM4A7qD6A+PiiLinDrKaJqb9oTomdq889VBE\nfBg4GfjtzPxB11MbqM43c2ymxgLgYGB1RDweEY8Dzwf+OCK2Uv1l5nhMnR8AI+84vhZ4av2z/z6m\n1l8D78vMf8jMr2Xm1VQXYX5r/bzj0Z4mn/0GYN+IGNxJn0amfXCq/7Lu3CsP2OFeeT2/OaB2VIem\nU4HfyczvdD+Xmeup/oPuHptBqm/hOTa9dxPwDKq/pOfVy63AVcC8zPw2jsdU+io/e7rAkcDd4L+P\nFsyk+iO723bq36OOR3safvargCdG9DmS6g+Rnd3D9md4qK6yGLiivtlw5155M6nun6dJEhFLqe4h\neArwSER0/lrYlJmdmyx/EHhbRHyT6ubL76b6xuNnp7jcPV5mPkJ1COInIuIR4IeZ2Zn5cDymzhLg\nqxHxVuAaql8CfwS8pquP4zF1rqf6rL8HfA2YT/W74mNdfRyPSRIRTwKOoJpZAji8PkH/gcz8Lrv4\n7DPzoYj4W2BxRDwIPAxcAnw1M28ZVzFtf62wlIXq2hB3UX19cQXwa23XtKcvVH+tbRtledWIfu+k\n+qrpZqq7XR/Rdu3TZQG+RNflCByPKf/8Twb+o/6svwacO0ofx2NqxuJJVH9kr6e6RtA3gHcB+zge\nU/L5P3+M3xl/1/SzB/ajunbg/XVw+gfgKeOtxXvVSZIkNTTtz3GSJElqyuAkSZLUkMFJkiSpIYOT\nJElSQwYnSZKkhgxOkiRJDRmcJEmSGjI4SZIkNWRwkiRJasjgJEmjiIjtEXFK23VIKovBSVJxIuLv\n6+CyrV53fr6h7dokTW/7tF2AJI3hH4FX89O7oQM81k4pklRxxklSqR7LzI2ZeV/Xsgl+chjt9RFx\nQ0RsjohvRcTvdW8cEb8aEV+sn78/Ii6LiCeN6HNuRPxXRDwaEd+PiEtG1HBwRPyfiHgkIr4eES/t\n2vbAiLg6Iu6rX2NdRJw9aZ+GpCIYnCT1qwuBfwCeCVwNfCoijgSIiJnAcuCHwALgFcALgQ91No6I\n84APAx8BfgV4MfD1Ea/xDuBTwDOAG4CrI+LA+rm/Ao4CFtbr84D7e/0mJZUlMrPtGiRpBxHx98BZ\nwKNdzQk646D2AAACD0lEQVS8NzPfFxHbgaWZ+aaubVYAqzLzTRHxGuAi4Bcz89H6+ZOA64H/lpkb\nI+J7wN9m5gVj1LAduDAz31k/ngn8GHhRZt4YEZ8FNmbmH/X23Usqmec4SSrVl4DXs+M5Tg90/bxy\nRP8VwLz656OANZ3QVPsq1Sz7kREBcEj9Gjvzn50fMnNzRDwEPKVu+hvg2ohYANwIXJeZK3b1piT1\nN4OTpFI9kpnrJ2nfWxr2e3zE46Q+xSEz/ykingqcDJwI3BQRl2bm+b0rU1JpPMdJUr96ziiP19Y/\nrwXmRcSMrud/E9gG3JmZPwbuAk7YnQIy84eZ+YnMfBWwCHjt7uxPUvmccZJUqv0iYvaIticy84f1\nz6dFxCrgX6nOhzoOOLd+7mrgncDHI+JdVIfXLgGuzMzOCdzvBP4mIjZSXfpgEHhuZn64SXH1flcB\nXwP2B14C3DHeNympvxicJJXqRcA9I9rWAcfUP18AnAFcCvwAOCMz7wTIzC0RsRC4GLgF2Ax8GvjT\nzo4y88qI2I9qpuj9VN+I+3TXa432zZnsat8KvBc4jOrQ3/8DhifwPiX1Eb9VJ6nv1N94e1lmfq7t\nWiRNL57jJEmS1JDBSVI/cqpcUis8VCdJktSQM06SJEkNGZwkSZIaMjhJkiQ1ZHCSJElqyOAkSZLU\nkMFJkiSpIYOTJElSQwYnSZKkhgxOkiRJDf1/9YCmkE49IDYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105e669e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualizing the cost to check for convergence and plotting the linear model:\n",
    "\n",
    "plt.plot(range(1, gd_lr.epochs+1), gd_lr.cost_)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Cost')\n",
    "plt.ylim([0, 0.2])\n",
    "plt.tight_layout()\n",
    "plt.show()    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3 - Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.24\n",
      "Slope: 0.82\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAFkCAYAAAA9h3LKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xl0VeW9//H3I3VWKPXWoWq9XglcUJc10YpXkWtNQXEq\n1qsGpFpFBSekDpU619r6q7XW2mqdllZT49RaqwWxaJ2oqBBURGgSrEqdBQkWRCE8vz+eVAVROclJ\nds4+79daWSabM3zdkHzy3fsZQowRSZKUH2tkXYAkSSouw12SpJwx3CVJyhnDXZKknDHcJUnKGcNd\nkqScMdwlScoZw12SpJwx3CVJyhnDXZKknCk43EMIXwkh3BxCeDuEsDiE8EwIobIjipMkSYX7QiEP\nDiF8EZgMPAAMBt4GKoB3il+aJElqi1DIxjEhhIuBXWOMAzuuJEmS1B6FXpbfH5gaQrg9hPBGCKE+\nhDCyIwqTJEltU2jn/h4QgUuBO4GvA5cDx8UYb17F4zciXb5/EVhShHolSSoX6wD/CUyMMc4r5ImF\nhvv7wJMxxgEfO3Y5sFOMcbdVPH4Y8LtCCpIkSSsYHmO8pZAnFDSgDngNmLXSsVnAQZ/y+BcBamtr\n6du3b4FvlT9jx47lsssuy7qMzHkePuK5SDwPH/FcJJ4HmDVrFocffji0ZmkhCg33yUCflY71AV76\nlMcvAejbty+Vlc6W69Gjh+cBz8PHeS4Sz8NHPBeJ52EFBd/WLnRA3WVA/xDCuBDCNq2X3UcCvyr0\njSVJUscoKNxjjFOBoUANMAM4CxgTY7y1A2qTJEltUOhleWKM44HxHVCLJEkqAteW70Q1NTVZl9Al\neB4+4rlIPA8f8Vwknof2KWgqXMEvntacnzZt2jQHRkiSVID6+nqqqqoAqmKM9YU8185dkqScMdwl\nScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnK\nGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnD\nXZKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwlScoZw12S\npJwx3CVJyhnDXZKknDHcJUldT4xZV1DSDHdJUtfx7rswZgyMG5d1JSXNcJckdQ333gvbbgvXXQeb\nbpp1NSXNcJckZev11+HQQ2H//aFfP3juOTjllKyrKmkFhXsI4bwQwvKVPp7vqOIkSTkWY+rS+/aF\nBx+E2lqYMAG23jrrykreF9rwnOeAvYDQ+vWy4pUjSSoLf/87HHccPPwwHHEEXHopbLQRAA0NDcyZ\nM4devXpRUVGRcaGlqS2X5ZfFGN+KMb7Z+jG/6FVJkvLpgw/gRz+CHXaAuXPhL3+BG2+EjTZi/vz5\n7L33vvTp04chQ4bQu3dv9t57X955552sqy45bQn3ihDCKyGEOSGE2hDClkWvSpKUP48/DpWVcP75\n6Z76jBlQXf3hHw8bNoJJk6YAtcDLQC2TJk2hpubwjAouXYWG+xTgSGAwMArYGngkhLB+keuSJOXF\nwoVw4omw226w3nowbRpcfHH6vFVDQwMTJ46npeWXwHBgS2A4LS2XM3HieBobG7OqviQVdM89xjjx\nY18+F0J4EngJOAS44dOeN3bsWHr06LHCsZqaGmpqagp5e0lSqfnTn+D442HBAvj5z+Gkk6Bbt088\nbM6cOa2f7bHSnwwEoKmpKdf33+vq6qirq1vhWHNzc5tfry0D6j4UY2wOITQAvT7rcZdddhmVlZXt\neStJUil57bUU5L//PQwZAldeCVtt9akP32abbVo/e4TUuf/bwwD06vWZMVPyVtXw1tfXU1VV1abX\na9c89xDCBqRgf609ryNJyonly+Gaa9L0tkcfhVtvTYvTfEawA/Tu3ZvBg4fQrdvJpHvuc4FaunUb\nw+DBQ3LdtXeEQue5XxJC2COEsFUI4X+Au4ClQN3nPFWSlHezZ8P//m+a4vbtb8OsWWlxmhA+96kA\ndXW1VFf3B0YAXwVGUF3dn7q62g4sOp8KvSy/BXALsBHwFvAY0D/GOK/YhUmSSsT776cBcj/+cerQ\nH3wQ9tyz4Jfp2bMn9933ZxobG2lqanKeezsUOqDOEXCSpI9MngzHHAONjXDGGXD22bDuuu16yYqK\nCkO9nVxbXpJUuOZmGD0adt8duneH+nq46KJ2B7uKo12j5SVJZeiuu9K89YUL4YorUsivYnqbsmPn\nLklaPa+8AkOHwkEHQVUVPP98CnmDvcsx3CVJn2358jRPvW9fmDIF7rgD7r4btnT18a7KcJckfbqZ\nM2HAADjhBDjssNStH3zwak9vUzYMd0nSJy1ZAueeCzvuCPPmpa1Zr7kGevbMujKtBgfUSZJW9Mgj\ncOyx8MILMG5c+lhnnayrUgHs3CVJyYIFKdQHDoSNNoLp0+GCCwz2EmTnLknlLka48044+WRYtCgN\nnjvuOFjD/q9U+TcnSeVs7lw48EA45BDo3z+tBz96tMFe4vzbk6Ry1NKSFqDp1w+mTk1bs951F2y+\nedaVqQgMd0kqNzNmwG67pcvwhx+euvWDDsq6KhWR4S5J5WLJEjjrLKisTEvHPvooXHUV9OiRdWUq\nMgfUSVI5+Otf0yC5l15KO7edeSasvXbWVamD2LlLUp7Nnw9HHw3f+AZssgk8/TScd57BnnN27pKU\nRzHCbbfBmDHpcvxvfpP2XXcUfFnwb1mS8ubll2H//aGmJq0LP2uW89bLjH/TkpQXLS1w+eVpetvT\nT8Mf/5gWp/nKV7KuTJ3McJekPHjmGdh1Vxg7Fo48Mu3eduCBWVeljBjuklTK3nsvbexSVQWLF8Pk\nyfCrX0H37llXpgw5oE6SStUDD6R76XPnwvnnwxlnwFprZV2VugA7d0kqNfPmpUvv1dWwxRbw7LNp\n7rrBrlZ27pJUKmKEujo45RRYuhSuvRaOOspR8PoE/0VIUil48UUYMgSGD4c990zT20aONNi1Sv6r\nkKSubNkyuPRS2HZbmDkT7rknLU6z6aZZV6YuzHCXpK5q+vS0x/rpp6cufeZM2G+/rKtSCTDcJamr\nWbw4BfrOO8MHH8CUKWlxmg03zLoylQgH1ElSV3L//TBqFLz6Klx4IZx2Gqy5ZtZVqcTYuUtSV/DW\nWzBiBAweDFtvDTNmpMVpDHa1gZ27JGUpRrj5Zvje92D5crjhBjjiCAgh68pUwuzcJSkrL7yQOvUj\njoBBg2D27LQ4jcGudjLcJamzLVsGP/0pbLcdNDTA+PFwyy2w8cZZV6acMNwlqTNNnZpGwY8blwbO\nPfcc7LNP1lUpZwx3SeoM//pXuq++yy7pPvuUKfDzn8MGG2RdmXLIAXWS1NEmTIDRo+GNN+AnP0l7\nrjsKXh3Izl2SOsqbb8KwYWlN+IqKdAn+jDMMdnU4O3dJKrYY4cYb4dRT08Yuv/1tmsPuKHh1Ejt3\nSSqmpqa0z/pRR8G++6bd277zHYNdncpwl6RiWLoULr4Ytt8+zV+/7760OM2Xv5x1ZSpDhrsktdeT\nT8JOO8FZZ8GJJ6Z764MHZ12Vyli7wj2EcGYIYXkI4efFKkiSSsa778Ipp6RtWb/wBXjqKbjkElh/\n/awrU5lr84C6EMLOwLHAM8UrR5JKxJ//nKa3zZuXAn3MmBTwUhfQps49hLABUAuMBBYUtSJJ+hwN\nDQ1MmDCBxsbGzn/z11+HQw+F/faDfv3SJfhTTzXY1aW09bL8r4F7YowPFrMYSfos8+fPZ++996VP\nnz4MGTKE3r17s/fe+/LOO+90/JvHCNdfD337woMPQm1tWpxm6607/r2lAhUc7iGEw4CvAeOKX44k\nfbphw0YwadIU0oXDl4FaJk2aQk3N4R37xg0NsOeeMHIkHHhg2r1t+HCnt6nLKug6UghhC+AXQHWM\ncWnHlCRJn9TQ0MDEieNJwT689ehwWloiEyeOoLGxkYqKiuK+6QcfpPvpF14Im28Of/lLmsMudXGF\n3iSqAr4M1Ifw4a+s3YA9QggnAmvHGOPKTxo7diw9evRY4VhNTQ01NTVtKFlSOZozZ07rZ3us9CcD\nAWhqaipuuE+ZAscckxahOe00OPdcWG+94r2+9DF1dXXU1dWtcKy5ubnNrxdWkcWf/uAQ1ge2Wunw\njcAs4OIY46yVHl8JTJs2bRqVlZVtLlKSGhoa6NOnDyt27rR+PYKGhobihPvChfCDH8CVV6a569de\nCzvs0P7XlQpUX19PVVUVQFWMsb6Q5xbUuccYFwHPf/xYCGERMG/lYJekYurduzeDBw9h0qSTaWmJ\npI79Ybp1G0N19ZDiBPuf/gTHHw8LFqTtWE86Cbp1a//rSp2sGCvUrX7rL0ntUFdXS3V1f2AE8FVg\nBNXV/amrq23fC7/2Ghx8cBost8MOMHNmWpzGYFeJavfEzBjjN4pRiCR9np49e3LffX+msbGRpqYm\nevXq1b6OfflyuO66tA3r2mvDrbfCIYc4Cl4lz1UXJJWcioqK9l+Gnz0bjj0WHn007eB2ySXwpS8V\np0ApY24cI6m8vP8+XHBBuvz++utpQZrrrzfYlSt27pLKx+TJaXpbY2O6FH/22bDuullXJRWdnbuk\n/GtuTpu87L47dO8O9fVw0UUGu3LLzl1Svt11V9pjfeFC+OUv01Q3R8Er5+zcJeXTK6/A0KFw0EFQ\nWQnPP++8dZUNw11SvixfnlaX69sXHn8cbr89LU6z5ZZZVyZ1GsNdUn7MnAkDBsAJJ8Bhh6V14f/v\n/5y3rrJjuEsqfUuWpI1ddtwR3n4bHnoIrrkGevbMujIpEw6ok1TaHn00TW974QU488y06cs662Rd\nlZQpO3dJpWnBAjjuONhjj7QAzfTp8MMfGuwSdu6SSk2M8Pvfp5HvixbBr38No0bBGvYq0r/53SCp\ndMydC9/6Vhokt8suaXrb8ccb7NJK/I6Q1PW1tMCvfgX9+sFTT6XO/Y9/hC22yLoyqUsy3CV1bTNm\npGVjTzoJDj88TW876KCsq5K6NMNdUte0ZEna2KWyMq0N/+ijcNVV0KNH1pVJXZ4D6iR1PQ89lPZa\nf+mlFPBnnglrr511VVLJsHOX1HXMnw8jR8Kee8Imm8DTT8N55xnsUoHs3CVlL8a0BvzJJ6fL8b/5\nTVqYxlHwUpv4nSMpWy+/DPvvn9aCHzAgDZg77jiDXWoHv3skZaOlBS6/PE1ve/rpNLXtzjvhK1/J\nujKp5BnukjrfM8/ArrvC2LFw5JFpMZoDD8y6Kik3DHdJnee992DcOKiqgsWLYfLktDhN9+5ZVybl\nigPqJHWOBx5I99LnzoXzz4czzoC11sq6KimX7Nwldax589Kl9+rqtFzss8+muesGu9Rh7NwldYwY\noa4OTjkFli6Fa6+Fo45yFLzUCfwuk1R8L74IQ4bA8OFpQZpZs9LiNAa71Cn8TpNUPMuWwaWXwrbb\nwsyZcM89cNttsOmmWVcmlRXDXVJxTJ8O/fvD6aenLn3mTNhvv6yrksqS4S6pfRYvToG+887wwQcw\nZUpanGbDDbOuTCpbDqiT1Hb33w+jRsGrr8KFF8Jpp8Gaa2ZdlVT27NwlFe6tt2DECBg8GLbeGmbM\nSIvTGOxSl2DnLmn1xQg33wzf+x4sXw433ABHHAEhZF2ZpI+xc5e0el54IXXqRxwBgwbB7NlpcRqD\nXepyDHdJn23ZMvjpT2G77aChAcaPh1tugY03zroySZ/CcJf06aZOTaPgx42D0aPhuedgn32yrkrS\n5zDcJX3Sv/6V7qvvskv6+okn0uI0G2yQbV2SVosD6iStaMKE1KW/8Qb85Cdpz3VHwUslxc5dUvLm\nmzBsWFoTvqIiXYI/4wyDXSpBdu5SuYsRbrwRTj01bexy001w+OGOgpdKmJ27VM6amtI+60cdldaB\nnzUrLU5jsEslraBwDyGMCiE8E0Jobv34Wwhh744qTlIHWbo03U/ffnv4xz9g4sTUsX/5y1lXJqkI\nCu3c5wLfByqBKuBB4O4QQt9iFyapgzzxBFRVwTnnwEknpaVjBw36xMMaGhqYMGECjY2NGRQpqT0K\nCvcY459jjPfFGOfEGJtijGcD/wL6d0x5korm3XdhzBjYddc0SO6pp9LiNOuvv8LD5s+fz95770uf\nPn0YMmQIvXv3Zu+99+Wdd97JqHBJhWrzPfcQwhohhMOA9YDHi1eSpKK7917Ydlu47jr42c9S977j\njqt86LBhI5g0aQpQC7wM1DJp0hRqag7vzIoltUPBo+VDCNuRwnwd4F1gaIxxdrELk1QEr7+euvXb\nb0/rwl91VdrF7VM0NDQwceJ4UrAPbz06nJaWyMSJI2hsbKSioqIzKpfUDm2ZCjcb2AHoARwM3BRC\n2OOzAn7s2LH06NFjhWM1NTXU1NS04e0lfa4Y4frr4fTT0yX43/0Oamo+dxT8nDlzWj/bY6U/GQhA\nU1OT4S51gLq6Ourq6lY41tzc3ObXCzHGdhUUQvgL0BRjHL2KP6sEpk2bNo3Kysp2vY+k1fT3v8Nx\nx8HDD6dd2372M9hoo9V6akNDA3369GHFzp3Wr0fQ0NBguEudpL6+nqqqKoCqGGN9Ic8txjz3NYC1\ni/A6ktrjgw/gRz+CHXaAf/4TJk1K+62vZrAD9O7dm8GDh9Ct28mkQJ8L1NKt2xgGDx5isEslotB5\n7j8OIQwIIWwVQtguhPAT0vW62o4pT9JqefxxqKyE889Pa8HPmAF77dWml6qrq6W6uj8wAvgqMILq\n6v7U1fltLpWKQu+5bwz8FtgMaAaeBQbFGB8sdmGSVsPChfCDH8CVV8JOO8G0aalzb4eePXty331/\nprGxkaamJnr16mXHLpWYgsI9xjiyowqRVKC774YTToAFC+Cyy+DEE6Fbt6K9fEVFhaEulSjXlpdK\nzauvwsEHw7e+lbr0mTPTdLciBruk0ma4S6Vi+XK4+mro1w8efRRuvTUtTrPVVllXJqmLMdylUjBr\nFgwcCKNGwbe/nb4+9FB3b5O0Soa71JW9/z5ccAF87Wvwxhvw4INpcZovfSnryiR1YW1ZoU5SZ3js\nMTj2WGhshDPOgLPPhnXXzboqSSXAzl3qapqbYfRoGDAAuneH+nq46CKDXdJqs3OXupI//CFNaXv3\nXbjiihTyjoKXVCA7d6kreOUVGDo0DZbbaSd4/vmiz1uXVD4MdylLy5en1eX69oUpU+COO9LiNFtu\nmXVlkkqY4S5lZebMdF/9hBPgsMNSt37wwU5vk9RuhrvU2ZYsgXPPhR13hHnz0tas11wDPXtmXZmk\nnHBAndSZHnkkTW974QUYNy59rLNO1lVJyhk7d6kzLFiQQn3gwLS/+vTpaXEag11SB7BzlzpSjHDn\nnXDyybBoURo8d9xxsIa/V0vqOP6EkTrK3Llw4IFwyCHQv39aD370aINdUofzp4xUbC0taQGafv1g\n6tS0MM1dd8Hmm2ddmaQyYbhLxTRjBuy2W7oMP2JE6taHDs26KkllxnCXimHJEjjrLKishIUL06Yv\nV14JPXpkXZmkMuSAOqm9/vrXNEjupZfgnHPg+9+HtdfOuipJZczOXWqr+fPh6KPhG9+ATTaBp59O\ni9MY7JIyZucuFSpGuO02GDMG3n8frr4aRo50FLykLsOfRlIhXnoJ9tsPampgjz3SgLljjzXYJXUp\n/kSSVkdLC/ziF7DttvDMM2nntjvugM02y7oySfoEw136PM88kxah+d734LvfTbu3HXBA1lVJ0qcy\n3KVP8957cOaZUFWVPp88OS1O07171pVJ0mdyQJ20KpMmwahR8M9/pg1eTj8d1lor66okabXYuUsf\nN28eHHkkfPObsMUW8OyzaXEag11SCbFzlyBNb7vlFjjlFFi2DK67Do46CkLIujJJKpidu/SPf8A+\n+8Dhh8Nee6XpbUcfbbBLKlmGu8rXsmVw6aWw3XZpBPy998Ktt8Kmm2ZdmSS1i+Gu8lRfD7vskgbK\njRwJM2fCvvtmXZUkFYXhrvKyaFEK9K9/HZYuhSlT4PLLYcMNs65MkorGAXUqH/ffn6a3vfoqXHgh\nnHYarLlm1lVJUtHZuSv/3noLRoyAwYNh661hxgwYN85gl5Rbdu7Krxjh5pvTsrHLl8MNN8ARRzgK\nXlLu2bkrn+bMgUGDUpgPGgSzZ6fFaQx2SWXAcFe+LFsGP/0pbL89NDbC+PFpcZqNN866MknqNIa7\n8mPqVNh553Q/ffRoeO65tDiNJJUZw12l71//SvfVd9klff3EE2lxmg02yLYuScqIA+pU2iZMSF36\nm2/CxRenteEdBS+pzNm5qzS9+SYMGwZDhkBFRZredvrpBrskUWC4hxDGhRCeDCEsDCG8EUK4K4TQ\nu6OKkz4hxjSl7b//Oy1Kc9NN6b/bbJN1ZZLUZRTauQ8ArgB2AaqBNYH7QwjrFrswlZ+GhgYmTJhA\nY2Pjqh/Q1ATV1Wkr1v32S7u3jRjh9DZJWklB99xjjEM+/nUI4UjgTaAKeKx4ZamczJ8/n2HDRjBx\n4vgPjw0ePIS6ulp69uyZ1oD/2c/ghz+EzTaDiRPT3HVJ0iq19577F4EIzC9CLSpTw4aNYNKkKUAt\n8DJQy6RJU6ipOTyNfK+qgnPOgZNOSvfWDXZJ+kxtHi0fQgjAL4DHYozPF68klZOGhobWjr0WGN56\ndDjrtixmn4nHEu+fQNhxR3jqKdhxxwwrlaTS0Z6pcFcC/YDdPu+BY8eOpUePHiscq6mpoaamph1v\nrzyYM2dO62d7fHhsX+7lSs5nI2D20UfT96qr4AvO2pSUX3V1ddTV1a1wrLm5uc2vF2KMhT8phF8B\n+wMDYowvf8bjKoFp06ZNo7Kyss1FKr8aGhro06cPUMsm7MXljOFQbuc+tmc0M7i/oYGKioqsy5Sk\nTldfX09VVRVAVYyxvpDnFnzPvTXYDwT2/Kxgl1ZH7969GTxoH44JxzKLbdiTBxjGaPZb45/0GTzE\nYJekNijoWmcI4UqgBjgAWBRC2KT1j5pjjEuKXZzKwN//zj2LFrJmXMwNwGksZj5XMfibabS8JKlw\nhd7IHEUaHf/QSse/C9xUjIJUJj74IO3e9qMfseYWW8CkSez+1a9S29REr1697NglqR0KnefucrVq\nv8cfh2OOSXusn346nHsurLsuFWCoS1IRGNbqPAsXwoknwm67wXrrwbRp8JOfwLoucChJxeT8InWO\nu++GE06ABQvgsstSyHfrlnVVkpRLdu7qWK++CgcfDN/6FuywA8ycCWPGGOyS1IEMd3WM5cvh6quh\nXz949FG49Va4917YaqusK5Ok3DPcVXyzZsHAgTBqFHz72+nrQw919zZJ6iSGu4rn/ffhggvga1+D\nN96ABx+E66+HL30p68okqaw4oE7FMXlymt7W2AhnnAFnn+0oeEnKiJ272qe5GUaPht13h+7dob4e\nLrrIYJekDNm5q+3uuitNaVu4EK64IoW8o+AlKXN27ircK6/A0KFw0EFQVQXPP++8dUnqQgx3rb7l\ny+HKK6FvX5gyBe64Iy1Os+WWWVcmSfoYw12rZ+ZMGDAgrTJ32GGpWz/4YKe3SVIXZLjrsy1ZkjZ2\n2XFHmDcPHn4YrrkGevbMujJJ0qdwQJ0+3SOPwLHHwgsvwLhx6WOddbKuSpL0Oezc9UkLFqRQHzgQ\nNtoIpk9Pi9MY7JJUEuzc9ZEY4c474eSTYdGiNHjuuONgDX8HlKRS4k9tJXPnwoEHwiGHQP/+aT34\n0aMNdkkqQf7kLnctLWkBmn79YOpU+MMf0uI0m2+edWWSpDYy3MvZjBmw227pMvyIEalbHzo066ok\nSe1kuJejJUvgrLOgsjItHfvYY+n+eo8eWVcmSSoCB9SVm4ceSiPhX3oJzjkHvv99WHvtrKuSJBWR\nnXu5mD8fRo6EPfeETTaBp59Oi9MY7JKUO3bueRcj3H57uq/+/vtw9dUp5B0FL0m55U/4PHv5Zdh/\n/7QW/B57pAFzxx5rsEtSzvlTPo9aWuDyy9P0tqefTju33XEHbLZZ1pVJkjqB4Z43zzwDu+4KY8fC\nd7+bdm874ICsq5IkdSLDPS/eey9t7FJVBYsXw+TJaXGa7t2zrkyS1MkcUJcHDzyQ1oCfOxfOPx/O\nOAPWWivrqiRJGbFzL2Xz5sGRR0J1NWyxBTz7LJx9tsEuSWXOzr0UxQh1dXDKKbB0KVx7LRx1lKPg\nJUmAnXvpefFFGDIEhg9PC9LMmuW8dUnSCkyEUrFsGVx6KWy7LcycCffcA7fdBptumnVlkqQuxnAv\nBdOnpz3WTz89dekzZ8J++2VdlSSpizLcu7LFi1Og77wzfPABTJmSFqfZcMOsK5MkdWEOqOuq/vKX\nNL3t1VfhwgvhtNNgzTWzrkqSVALs3Luat9+G73wHBg2CrbeGGTPS4jQGuyRpNdm5dxUxQm1tWjZ2\n+XK44QY44ggIIevKJEklxs69K3jhBRg8+KOOffbstDiNwS5JagPDPUvLlsEll8B220FDA4wfD7fc\nAhtvnHVlkqQSZrhnZdq0NAr+zDNh9Gh47jnYZ5+sq5Ik5UDB4R5CGBBC+FMI4ZUQwvIQgvuJFmLR\nIjj1VPj619PXTzyRFqfZYINs65Ik5UZbOvf1gaeB44FY3HJy7r770gpzV10FF18MTz4JO+2UdVWS\npJwpeLR8jPE+4D6AEBzxtVpaWlg4dCjd77mHxf/zP6z3wAOwzTZZVyVJyinvuXew+fPns/e+B3DN\nPffwHWD9v/2NvU84mXfeeSfr0iRJOWW4d7Bhw0YwadIUTqeWm3kZqGXSpCnU1ByedWmSpJxyEZsO\n1NDQwMSJ44FaYHjr0eG0tEQmThxBY2MjFRUVGVYoScqjTgn3sWPH0qNHjxWO1dTUUFNT0xlvn5k5\nc+a0frbHSn8yEICmpibDXZJEXV0ddXV1Kxxrbm5u8+t1SrhfdtllVFZWdsZbdSnbfDho7hE+6twB\nHgagV69enV2SJKkLWlXDW19fT1VVVZter+BwDyGsD/QC/j1S/r9CCDsA82OMc9tURU717t2bwYOH\nMGnSybRE0cGiAAAHHUlEQVS0RFLH/jDduo2hunqIXbskqUO0ZUDdTsB0YBppnvulQD1wQRHryo26\nulqqq/sDI4CvAiOoru5PXV1txpVJkvKqLfPcH8ZR9qutZ8+e3Hffn2lsbKSpqYlevXrZsUuSOpSj\n5TtJRUWFoS5J6hR24JIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhju\nkiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk\n5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM\n4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOHe\nierq6rIuoUvwPHzEc5F4Hj7iuUg8D+3TpnAPIZwQQvhHCOG9EMKUEMLOxS4sj/zHmngePuK5SDwP\nH/FcJJ6H9ik43EMIhwKXAucBOwLPABNDCP9R5NokSVIbtKVzHwtcHWO8KcY4GxgFLAaOKmplkiSp\nTQoK9xDCmkAV8MC/j8UYIzAJ2LW4pUmSpLb4QoGP/w+gG/DGSsffAPqs4vHrAMyaNavwynKoubmZ\n+vr6rMvInOfhI56LxPPwEc9F4nlYITvXKfS5ITXeq/ngEDYDXgF2jTE+8bHj/w/YI8a460qPHwb8\nrtCiJEnSh4bHGG8p5AmFdu5vAy3AJisd3wR4fRWPnwgMB14ElhT4XpIklbN1gP8kZWlBCurcAUII\nU4AnYoxjWr8OwMvAL2OMlxRagCRJKq5CO3eAnwM3hhCmAU+SRs+vB9xYxLokSVIbFRzuMcbbW+e0\n/5B0Of5pYHCM8a1iFydJkgpX8GV5SZLUtbm2vCRJOWO4S5KUMx0a7m4wAyGEASGEP4UQXgkhLA8h\nHJB1TVkIIYwLITwZQlgYQngjhHBXCKF31nV1thDCqBDCMyGE5taPv4UQ9s66rqyFEM5s/f74eda1\ndLYQwnmt/+8f/3g+67qyEkL4Sgjh5hDC2yGExa3fL5VZ19WZWnNz5X8Ty0MIV6zua3RYuLvBzIfW\nJw06PB4o5wEOA4ArgF2AamBN4P4QwrqZVtX55gLfBypJSzk/CNwdQuibaVUZav2l/1jSz4hy9Rxp\ngPKmrR+7Z1tONkIIXwQmA+8Dg4G+wKnAO1nWlYGd+OjfwqbAN0n5cfvqvkCHDaj7lPnwc0nz4X/a\nIW/axYUQlgPfijH+Ketastb6S96bpJUNH8u6niyFEOYBp8UYb8i6ls4WQtgAmAaMBs4BpscYv5dt\nVZ0rhHAecGCMsay601UJIVxMWgF1YNa1dCUhhF8AQ2KMq321s0M6dzeY0Wr4Iuk30flZF5KVEMIa\nIYTDSOtEPJ51PRn5NXBPjPHBrAvJWEXrrbs5IYTaEMKWWReUkf2BqSGE21tv39WHEEZmXVSWWvN0\nOHB9Ic/rqMvyn7XBzKYd9J4qEa1XcX4BPBZjLLt7iyGE7UII75IuPV4JDG3dPrmstP5i8zVgXNa1\nZGwKcCTpMvQoYGvgkRDC+lkWlZH/Il3F+TswCLgK+GUIYUSmVWVrKNAD+G0hT2rLCnVSe10J9AN2\ny7qQjMwGdiB9wx4M3BRC2KOcAj6EsAXpF7zqGOPSrOvJUozx4+uGPxdCeBJ4CTgEKLdbNWsAT8YY\nz2n9+pkQwnakX3puzq6sTB0FTIgxrmr/lk/VUZ17oRvMqEyEEH4FDAH+N8b4Wtb1ZCHGuCzG+EKM\ncXqM8SzSQLIxWdfVyaqALwP1IYSlIYSlwEBgTAjhg9arO2UpxtgMNAC9sq4lA68BK+8RPgv4aga1\nZC6E8FXSAORrC31uh4R762/i04C9/n2s9Zt1L+BvHfGe6vpag/1AYM8Y48tZ19OFrAGsnXURnWwS\nsD3psvwOrR9TgVpgh1jGS2e2DjLsRQq6cjMZ6LPSsT6kKxnl6CjS7ezxhT6xIy/Lu8EM0HrfrBfw\n707kv0IIOwDzY4xzs6usc4UQrgRqgAOARSGEf1/VaY4xls12wCGEHwMTSDspbkgaKDOQdH+xbMQY\nFwErjLcIISwC5sUYV+7cci2EcAlwDynANgcuAJYCdVnWlZHLgMkhhHGkaV+7ACOBYzKtKgOtDfGR\nwI0xxuWFPr/Dwt0NZj60E/BX0sjwSJr7D2lwxFFZFZWBUaT//4dWOv5d4KZOryY7G5P+7jcDmoFn\ngUGOFgfKdx2ILYBbgI2At4DHgP4xxnmZVpWBGOPUEMJQ4GLS1Mh/AGNijLdmW1kmqoEtaeO4CzeO\nkSQpZ1xbXpKknDHcJUnKGcNdkqScMdwlScoZw12SpJwx3CVJyhnDXZKknDHcJUnKGcNdkqScMdwl\nScoZw12SpJz5/7mTdL4Po2m9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c7ea390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "sgd_lr = LinearRegression(eta=0.01, \n",
    "                          epochs=100, \n",
    "                          random_seed=0, \n",
    "                          minibatches=len(y))\n",
    "sgd_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % sgd_lr.b_)\n",
    "print('Slope: %.2f' % sgd_lr.w_)\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, sgd_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XmYHFW9//H3N9tAwACSMCEYDMiVTUEmIITrDhgBhaui\nGOSKouK+xJ9y9XFBcYGrVyKoKIpeXDCK6AWVJQgu7CKZgAghbAkgkJBATCD7cn5/nG6nM8wkNTPd\n053K+/U89fR0dVX1qTrVVZ86VVMVKSUkSZK0aUOaXQBJkqTNhcFJkiSpIIOTJElSQQYnSZKkggxO\nkiRJBRmcJEmSCjI4SZIkFWRwkiRJKsjgJEmSVJDBSZIkqaCWCU4R8YGImBsRKyLi5og4aCPDvj4i\nroqIxyNiSUTcGBGv7mG4N0XE7Mo0b4+IIxs7F5IkqcxaIjhFxPHA14HTgAOA24EZETG6l1FeBlwF\nHAl0AH8EfhsR+9dM81DgZ8D3gRcBlwKXRMQ+jZoPSZJUbtEKD/mNiJuBv6SUPlJ5H8DDwDkppa8W\nnMbfgZ+nlL5Uef9zYGRK6ZiaYW4CZqWU3l/veZAkSeXX9BaniBgOTASuqfZLOc1dDUwqOI0AngU8\nWdN7UmUatWYUnaYkSVJ3TQ9OwGhgKLCgW/8FwNiC0/gEsA1wUU2/sQOcpiRJ0gaGNbsAAxURJwCf\nBY5JKS0a4LR2BCYD84CVAy+dJEkaZFsBE4AZKaUn6j3xVghOi4B1QHu3/u3A/I2NGBFvAb4HHJdS\n+mO3j+f3Y5qTgQs3VWBJktTy3kr+J7G6anpwSimtiYiZwGHAb+Bf1ywdBpzT23gRMQU4Hzg+pXRl\nD4Pc1MM0jqj07808gJ/+9KfsvffefZiL7L//Gzo74Re/6POo6sHUqVOZNm1as4uhCuujtVgfrcX6\naB2zZ8/mxBNPhMo+vd6aHpwqzgIuqASoW4CpwEjgAoCIOAMYl1I6qfL+hMpnHwb+GhHVlqUVKaWl\nlb/PBv4UER8DLgOmkC9Cf/dGyrESYO+996ajo6PPMzF+PNx2G/RjVPVgu+2261c9qDGsj9ZifbQW\n66MlNeSSm1a4OJyU0kXAx4HTgVnAfsDklNLCyiBjgfE1o7ybfEH5t4FHa7pv1EzzJuAE4BTgNuAN\nwLEppbsaNR9tbbBqVaOmLkmSmq1VWpxIKZ0LnNvLZ+/o9v6VBaf5K+BXAy9dMQYnSZLKrSVanMrC\n4CRJUrkZnOrI4FRfU6ZMaXYRVMP6aC3WR2uxPrYcBqc6MjjVlxui1mJ9tBbro7VYH1sOg1MdtbXB\nunW5kyRJ5WNwqqMRI/Lr6tXNLYckSWoMg1MdtbXlV0/XSZJUTganOjI4SZJUbganOjI4SZJUbgan\nOjI4SZJUbganOjI4SZJUbganOjI4SZJUbganOjI4SZJUbganOqoGJ+/jJElSORmc6qh6A0xbnCRJ\nKieDUx15qk6SpHIzONWRwUmSpHIzONWRwUmSpHIzONWRwUmSpHIzONXRkCEwbJjBSZKksjI41Vlb\nm8FJkqSyMjjVmcFJkqTyMjjVWVubN8CUJKmsDE51NmKELU6SJJWVwanOPFUnSVJ5GZzqzOAkSVJ5\nGZzqzOAkSVJ5GZzqzOAkSVJ5GZzqzOAkSVJ5GZzqzOAkSVJ5GZzqzPs4SZJUXganOrPFSZKk8jI4\n1Zk3wJQkqbwMTnVmi5MkSeVlcKozg5MkSeVlcKozg5MkSeVlcKozg5MkSeVlcKozg5MkSeVlcKoz\ng5MkSeVlcKozb4ApSVJ5GZzqzBYnSZLKy+BUZyNGwJo1sH59s0siSZLqzeBUZ21t+dXTdZIklY/B\nqc6qwcnTdZIklY/Bqc4MTpIklZfBqc4MTpIklZfBqc4MTpIklZfBqc4MTpIklZfBqc78rzpJksrL\n4FRntjhJklReBqc6GzEivxqcJEkqH4NTndniJElSeRmc6szgJElSeRmc6szgJElSeRmc6szgJElS\neRmc6szgJElSeRmc6mzo0Nx5HydJksrH4NQAbW22OEmSVEYGpwYwOEmSVE4GpwYYMcLgJElSGRmc\nGsAWJ0mSysng1AAGJ0mSysng1AAGJ0mSysng1AAGJ0mSysng1AAGJ0mSysng1ABtbd4AU5KkMjI4\nNYAtTpIklZPBqQEMTpIklZPBqQG8AaYkSeVkcGoAW5wkSSong1MDGJwkSSqnlglOEfGBiJgbESsi\n4uaIOGgjw46NiAsjYk5ErIuIs3oY5qSIWF/5fH2lW97YucgMTpIklVNLBKeIOB74OnAacABwOzAj\nIkb3Mkob8DjwReC2jUx6CTC2pntuvcq8MQYnSZLKqSWCEzAVOC+l9OOU0t3Ae4HlwMk9DZxSejCl\nNDWl9FNg6Uamm1JKC1NKj1e6hfUv+jMZnCRJKqemB6eIGA5MBK6p9kspJeBqYNIAJ79tRMyLiIci\n4pKI2GeA0yvEG2BKklROTQ9OwGhgKLCgW/8F5NNr/TWH3GJ1DPBW8rzeGBHjBjDNQmxxkiSpnIY1\nuwCNklK6Gbi5+j4ibgJmA+8hX0vVq6lTp7Lddttt0G/KlClMmTKl0HcbnCRJarzp06czffr0Dfot\nWbKkod/ZCsFpEbAOaO/Wvx2YX68vSSmtjYhZwB6bGnbatGl0dHT0+7u8AaYkSY3XU6NGZ2cnEydO\nbNh3Nv1UXUppDTATOKzaLyKi8v7Gen1PRAwBXgg8Vq9p9qZ6jVNKjf4mSZI0mFqhxQngLOCCiJgJ\n3EL+L7uRwAUAEXEGMC6ldFJ1hIjYHwhgW2BM5f3qlNLsyuefJZ+quw/YHjgV2BU4v9Ez09aWX1ev\n7vpbkiRt/loiOKWULqrcs+l08im624DJNbcPGAuM7zbaLKDaptMBnAA8COxe6bcD8L3KuIvJrVqT\nKrc7aKhqWFq1yuAkSVKZtERwAkgpnQuc28tn7+ih30ZPM6aUPgZ8rD6l65va4CRJksqj6dc4lVHt\nqTpJklQeBqcGsMVJkqRyMjg1gMFJkqRyMjg1gMFJkqRyMjg1wIgR+dXgJElSuRicGsAWJ0mSysng\n1AAGJ0mSysng1AAGJ0mSysng1AAGJ0mSysng1ADeAFOSpHIyODWALU6SJJWTwakBhg2DIUMMTpIk\nlY3BqUHa2gxOkiSVjcGpQUaMMDhJklQ2BqcGscVJkqTyMTg1iMFJkqTyMTg1iMFJkqTyMTg1iMFJ\nkqTyMTg1SFubN8CUJKlsDE4NYouTJEnlY3BqEIOTJEnlY3BqEIOTJEnlY3BqEG+AKUlS+RicGsQW\nJ0mSysfg1CAGJ0mSysfg1CAGJ0mSysfg1CAGJ0mSysfg1CDeAFOSpPIxODWILU6SJJWPwalBDE6S\nJJWPwalBDE6SJJWPwalBvAGmJEnlY3BqEFucJEkqH4NTg1SDU0rNLokkSaoXg1ODtLXl1zVrmlsO\nSZJUPwanBqkGJ+/lJElSeRicGqQanLzOSZKk8jA4NYjBSZKk8jE4NYjBSZKk8jE4NUg1OK1c2dxy\nSJKk+jE4NcgOO+TXJ59sbjkkSVL9GJwapL09vy5Y0NxySJKk+jE4Nciznw1DhxqcJEkqE4NTgwwZ\nAjvtZHCSJKlMDE4N1N5ucJIkqUwMTg1kcJIkqVwMTg1kcJIkqVwMTg1kcJIkqVwMTg00dqzBSZKk\nMjE4NVB7Ozz9NCxb1uySSJKkejA4NZA3wZQkqVwMTg1kcJIkqVwMTg1kcJIkqVwMTg204475DuIG\nJ0mSysHg1EBDh8KYMQYnSZLKwuDUYN7LSZKk8jA4NZjBSZKk8jA4NZjBSZKk8jA4NVh7O8yf3+xS\nSJKkejA4NZiPXZEkqTwMTg3W3g5PPQUrVjS7JJIkaaAMTg3mTTAlSSqPfgWniPhcRIzsof/WEfG5\ngRerPAxOkiSVR39bnE4Dtu2h/8jKZ6owOEmSVB79DU4BpB767w882f/ilM/o0RBhcJIkqQyG9WXg\niFhMDkwJuCciasPTUHIr1HfrV7zN37BhOTwZnCRJ2vz1KTgBHyW3Nv2QfEpuSc1nq4F5KaWb6lS2\n0vAmmJIklUOfglNK6UcAETEXuCGltLYhpSoZb4IpSVI59Pcap6eAvatvIuLYiLgkIr4SESPqU7Ty\nsMVJkqRy6G9wOg94PkBE7A78AlgOvAn4an8mGBEfiIi5EbEiIm6OiIM2MuzYiLgwIuZExLqIOKuX\n4d4UEbMr07w9Io7sT9kGyruHS5JUDv0NTs8Hbqv8/SbgzymlE4C3A2/s68Qi4njg6+Trpg4Abgdm\nRMToXkZpAx4HvlhTju7TPBT4GfB94EXApcAlEbFPX8s3ULY4SZJUDgO5HUF13MOByyt/Pwz0FnY2\nZipwXkrpxymlu4H3kluwTu5p4JTSgymlqSmlnwJLe5nmh4ErUkpnpZTmpJQ+B3QCH+xH+QakvR2W\nLIGVKwf7myVJUj31NzjdCnwmIv4TeDlwWaX/bkCf2lYiYjgwEbim2i+llICrgUn9LB+Vca/u1m/G\nAKfZL9WbYD7++GB/syRJqqf+BqePAh3At4Avp5Tuq/Q/Drixj9MaTb4HVPfAtQAY28/yURm33tPs\nF+8eLklSOfT1Pk4ApJT+Brywh48+AawbUIlKyOAkSVI59Cs4VUXERLpuS3BXSqmzH5NZRA5b7d36\ntwMDufvR/P5Oc+rUqWy33XYb9JsyZQpTpkzpV0HGjMmvBidJkupn+vTpTJ8+fYN+S5Ys6WXo+uhX\ncIqInci3IHg58M9K7+0j4o/AW1JKC4tOK6W0JiJmAocBv6lMPyrvz+lP+Spu6mEaR1T6b9S0adPo\n6OgYwFdvaPhw2HFHb4IpSVI99dSo0dnZycSJExv2nf29xumb5OfS7ZtSenZK6dnAC4BR9C/snAW8\nOyLeFhF7kZ93NxK4ACAizoiIH9WOEBH7R8SLKuUYU3m/d80gZwOviYiPRcSeEfF58kXo3+pH+QbM\nWxJIkrT56++putcAh6eUZld7pJTuiogPAFf1dWIppYsq92w6nXw67TZgck3L1VhgfLfRZpEfNgz5\nQvUTgAeB3SvTvCkiTgC+XOnuBY5NKd3V1/LVg8FJkqTNX3+D0xBgTQ/919DPVqyU0rnAub189o4e\n+m3ye1JKvwJ+1Z/y1Ft7Ozz2WLNLIUmSBqK/p+r+AJwdEeOqPSJiF2AaNfdjUhcfuyJJ0uavv8Hp\ng+TrmeZFxP0RcT8wt9LvQ/UqXJl4qk6SpM1ff+/j9HBEdJAft7JXpffslFL3O3Wror0dFi+G1ath\nxIhml0aSJPVHn1qcIuJVEXFXRIxK2e9TSt9MKX0T+GtE3BkRkxtU1s2aj12RJGnz19dTdR8Fvp9S\nesaDdVNKS4Dz8FRdj7x7uCRJm7++Bqf9gSs38vlVwH79L055VYOTN8GUJGnz1dfg1E7PtyGoWguM\n6X9xymunnfKrLU6SJG2++hqcHiHfIbw3+wHeragHI0bADjsYnCRJ2pz1NThdDnwxIrbq/kFEbA18\nAfhdPQpWRt6SQJKkzVtfb0fwJeANwD0R8S1gTqX/XsAHgKHkx5uoBwYnSZI2b30KTimlBRFxKPAd\n4Awgqh8BM4APpJSMBr3w7uGSJG3e+nwDzJTSg8BREbEDsAc5PN2bUlpc78KVTXs73HFHs0shSZL6\nq78P+aUSlP5ax7KUnqfqJEnavPX3WXXqh/Z2eOIJWLOxGzpIkqSWZXAaRNWbYN5/f3PLIUmS+sfg\nNIgOPDDfCHPSJPjhDyGlZpdIjbZiBcyb1+xSSJLqxeA0iMaOhbvugmOOgXe+E444Ah54oNmlGrhV\nq+D22+G66wYvDK5dmy+0/8EP4Ctf2fRF9ynB008PTtkAFi2C00+H5z4X/u3f4Ne/HrzvliQ1Tr8v\nDlf/7Lgj/OhHcMIJ8J73wAteAF/4ApxyCmy3XbNLt2kpQWcnzJgBf/tbDiz33JODDMDrXgfnn9/1\niJl6WbkS/vQn+P3v4ZZbchmWL4chQ2DkSPj0p+GFL4S3vhWmTIFdd4V//AOuvjqPc8018PjjcPDB\ncOyxObzuvTdEbPKr++Tee2HaNLjggvz+5JPz8wmPPx5+/nN44xvr+33r1sGf/wyzZuX1Z4cdurpd\nd83rmySpfiJ5vuhfIqIDmDlz5kw6Ojoa/n1PPw2f/Syccw4MGwaveQ285S05fGy7bQ4pd98N116b\nW3PmzIFDDoEjj4RXvCIHhlpPPZVbfu66K4eKVau6upUrc79ly7peI+DQQ+Hww/NpxGG9xOh16+DG\nG3Orya9/DQ89BKNGwf7757BS7ebPh/e+N4eZH/4Qjj56YMtn/ny47DL43e9y+Fm2DMaPz6c6DzoI\nXvxi6OjIj7O58kq48EL4zW/yvI4fDw8/nOexoyO37u22Ww58M2bkae2xB7z+9bn1b889ey5DSjmo\n3XFHXu677NLzMNdeC9/4Blx6KYwZAx/6ELzvfTm4rF0LJ54IF18Mv/jFwMNTSnDrrfCzn+XpPfZY\nXheWL99wuGHDcoj8+MdhvxI/envRoryOrFoFb3sbbL11sfFSyutIZydss03+HdQ7SG8OFi6E66/P\n61FbW1c3YkQ+sHve8zY+/vLleVlus83Ay/LII/lAcuXK/Js57DAYOnTg0y1i6dJ84DN3bj69Pncu\nPPggTJwI//Vfz9zeDqaU4IYb8rbkwAPz/qEsHnssnzno6ICjjqrPNDs7O5k4cSLAxJRSZ32m2sXg\nVGOwg1PVI4/AL3+Zd4I335w3/AcfDH//e94pDB0KBxyQd+7XX59/zG1t8PKX5/Bw331543/vvfkH\nNmQIbLVV7qobwa22yj/8kSPzBm7kyLxxuv76HLhGjYJXvjJ/79NP59aZanfffbkc48bloPGGN8DL\nXtZz0Jo/PweRyy/PweF//id//8MP54vi778/b6h33jmHm2o3fHhuwfrLX7q6e+7J8zJpErz2tTlQ\n7rPPxnduS5fC//1fboH593+HV73qma0uK1fCH/6QQ9bFF+f/dHzVq3J5jz02l2XRIvjpT/MP+u9/\nz+NFwEtfmluPjjsOtt8+19m0afn79tkHPvpR+M//zMu71tq1uf8vf5lbno47btPrxfz5OTg/+mhX\n99hjOTTdd1/+Z4Pjj8+tly9+MaxfD0uWwOLFubvuuly2hx+GyZPh1FNzHc+bl6dR7ZYuzTuoo47K\ny3r48E2XbTCsXw8zZ+Z1Zscdczd6dH5dsAAuuSQH1euv71rvd9oJPvMZeNe78o6/1ooVueXx2mtz\nfXV2wpNPdn0+aRKceWZet1vFww/nVtOnn8478Be9aGA78JTy9uOGG/JyuPbavI5B/j1XW46rIvJv\n4hOfyAdZtebMgW99K7eurl+fD/pOOSWvi7W/0fXru1qp29rygcNuu204rRUr8rbizDPz9mnHHXO5\nxo3LAeqkk/Lvqx5SyuvPHXfkclW7++7rGmbbbXMZd9kF/vjH/Fs7++y8LPoTrletyst97tx8ycZ+\n+xWfzg035Bb1P/85vx8yJB+oHnJI7g4/HJ7znL6Xqdnmz4evfhW+8528jqxenbeLZ5+d630gDE6D\nqFnBqda8eXDRRfnHst9+eSN+yCHwrGflz1PKG6wrrsitLLNm5UDV0ZG7Aw7Ip6CK7vzWrs07z6uv\nzt1tt+XTPGPG5J3QTjvlH+VRR+UN4pACV8WlBOedBx/7WN5QLlvWdQuGIUPy9J94YsNxhg7NLVvD\nhuWdw8EH5x3Z5Ml5Z9koK1fCr36Vf7w33JA3agceCFddlefj2GPzTvigg+C3v82h5/e/z59V52Py\nZJg6FV796o1vDNeuzS0iF10E3/wm7LVXHr7aLVmSN+AzZ+bu0Ue7xh01Km9Mdt45XzP1pjflVsfe\nWgmr1qzJ3/e1r+XWyNqWqfHj87yOHJnnd+HC/D2vfnVe9ltvncPHiBF5fWpr6wrdtQF8q63ysFtv\n3XvrQPUas8WL4Z//zK9r1nQFodGj8/hLl+ayXHZZXsc3dt+ztra80/iP/8ih+qmncmvFhRfma8s+\n97kciK+8MrdIXXNN3kGPH59DyAEHdP1m7roLPvnJvPyPPBLOOCO3qFYtX56Xz7p1+TexzTYb1vXy\n5V074lmz8rC1y2mbbfI87rFHrr/ddsvlr7V8eV6fbr89r2NXXZXDQ0Re/qtX5+W7zz55fRw/Pver\ndmvW5OHGjs1de3t+Xbx4wwOSxx/P37fPPvlA4GUvy6/jx3ftwKqt1L/7XV535szJwekTn8jlPuec\nvFzHjMmXHIwYkQ8yHnww79RPOSXX6xVX5MD0+ON5G7Z2ba6DAw+EN7857yj/+tcc6h99FD7ykRx8\nR43K26Uf/QimT88Bd++9u8r6spfl8lbXrcWL87Zz3ry8fq1Zk79rzZrcLVjQdeD2wAN5mwQ5IB1w\nQF4fOjryd+y2Gzz72V31e//9uQX5iivydvCcc3puhasGsjvvzAdbd96ZD2YfeCAH4NpdbXt7bgWf\nPDm/Vv/jutasWXlZXH55Xhe/9CWYMCEfXFe7u+7K0500KW8Tjjuua7k8+mjXenTNNfl3Wl3fq687\n79z3IPjkk3l92HrrXJ7tt+95uJUr84He6tV5uz90aH5dvTrvH77znbzeTJ2a6/2KK/KB58qVOUC/\n5z3F9jc9MTgNolYITmUyZw785Cf5x/m85+Xuuc/NP5bVq3NL20MP5Y3KU0/lwHTAAc9srRksd9wB\n3/1uDo/HHZePdMeMeeZwixblU5b33ZePhPfdt/h3rF0Lb3973rn3ZMcduzbiEyfmndAuuwy8aT6l\nHIxvvTVvhCdO3HBjXW3dueKK3P3tb7mOurdAbMrw4V0tPbWbliLTGjmya7h9982neo8+Opd38eIc\nKhYtyq8jR+bQ1NNyufNOOO20HIghb7Bf8pIcrl77Wnj+83veWaxfn8f59KfzDm+//fJOeNGiZ54G\n3XrrroOL5ctzwFm/PgfZfffNIXfFiryDrp4eX7Ag94O8Q9h11xwQnngidytXdk1/woS8Q331q3P4\n23bbPF/VVsK//jVPr3pKrdqtWJH7dz8wGTUqH/gcfHDuDjmk53W7J+vXdwWo66/P/To68s7uzW/u\n+r2uW5fXse99L7fmrl2b6+7II/NlCIcemgPZZZflMH/55V3z/LrXwde/nkNld6tW5WGvvDK3os6e\n3bWMRo3KYWnp0g3HqYbNYcPy6+jRXdugarf33vn7ih4MXnppnucFC/LyW7cuz2O1e/jhruXe1pYP\njPbcM3/X7rvnbrfdcqvTjBk50Nx2Wx5+++27gvbIkXmdnTUrl++LX8yhqKdy/vOfuW4uuihPc/Xq\nXL/LluXwVnupwtq1XS2tiyvP+RgyJC/DZz1rw9fttsuv1X7V1u/Zs/NBQa3tt8/zNWFCrs9HHsld\n93Ww1qhROSRNnbph8HryyXxa9Pzzcxg89tgNz55stVWu02oIGzo0d91/z/fd18mHPmRwGhQGJw2G\nlHJgXLs275RS6rpGZJddWus6m/Xr8xH76tV5o7hixYbXyi1fnvvVdqtXd81D9XX48A0vXN9hh7wB\nrIaGRYtyt9VWeSc7YcLAy37bbTncHnZY/r6i1qzJp59uvTXvcEePziFjzJi8kV64cMNT2cOHdx3F\n77tv78F//fp8BH7vvV3dsmVdrW7Vbo898s52IOvBmjW5bPPn5x3xnnv2/+i91q235sDQ/XRcdwsX\n5vV75517H+bpp3NI32mnfNlBUdVrsq67Lq9vu+3WteOeMCEvw3rMa0+WLYOzzsoHhUOH5nW4uiMf\nNy7X/7775vorcm3W/Pm5NeiRR7p+S9XXQw/NLdSbalWuWrIkt4pfckkOO5Mn53W/e0Cubn9mzcoh\ncOnSfOC6dGnv3ejROWjutVd+ff7zc6CtvR5s3ry87u+yS1c3blw+yFi3Lq//69bl7z/ooI3/Jq+7\nLoequXO7tjt9iyqdgMFpUBicJElqLSnlIL5yZQ5ftUFs3boNhwP42986OfroxgUnb0cgSZJaVvXU\na9Frd6vX8TWKN8CUJEkqyOAkSZJUkMFJkiSpIIOTJElSQQYnSZKkggxOkiRJBRmcJEmSCjI4SZIk\nFWRwkiRJKsjgJEmSVJDBSZIkqSCDkyRJUkEGJ0mSpIIMTpIkSQUZnCRJkgoyOEmSJBVkcJIkSSrI\n4CRJklSQwUmSJKkgg5MkSVJBBidJkqSCDE6SJEkFGZwkSZIKMjhJkiQVZHCSJEkqyOAkSZJUkMFJ\nkiSpIIOTJElSQQYnSZKkggxOkiRJBRmcJEmSCjI4SZIkFWRwkiRJKsjgJEmSVJDBSZIkqSCDkyRJ\nUkEGJ0mSpIIMTpIkSQUZnCRJkgoyOEmSJBVkcJIkSSrI4CRJklSQwUmSJKkgg5MkSVJBBidJkqSC\nWiY4RcQHImJuRKyIiJsj4qBNDP+KiJgZESsj4p6IOKnb5ydFxPqIWFd5XR8Ryxs7F5IkqcxaIjhF\nxPHA14HTgAOA24EZETG6l+EnAL8DrgH2B84Gzo+II7oNugQYW9M9twHFlyRJW4iWCE7AVOC8lNKP\nU0p3A+8FlgMn9zL8+4AHUkqnppTmpJS+DVxcmU6tlFJamFJ6vNItbNgcSJKk0mt6cIqI4cBEcusR\nkNMOcDUwqZfRDql8XmtGD8NvGxHzIuKhiLgkIvapU7ElSdIWqOnBCRgNDAUWdOu/gHx6rSdjexl+\nVES0Vd7PIbdYHQO8lTyvN0bEuHoUWpIkbXmGNbsAjZJSuhm4ufo+Im4CZgPvIV9LJUmS1CetEJwW\nAeuA9m7924H5vYwzv5fhl6aUVvU0QkppbUTMAvbYVIGmTp3Kdtttt0G/KVOmMGXKlE2NKkmSBsn0\n6dOZPn36Bv2WLFnS0O+MfDlRc0XEzcBfUkofqbwP4CHgnJTS13oY/kzgyJTS/jX9fgZsn1I6qpfv\nGALcCVyWUvp4L8N0ADNnzpxJR0fHQGdLkiQNss7OTiZOnAgwMaXUWe/pt8I1TgBnAe+OiLdFxF7A\nd4GRwAUAEXFGRPyoZvjvArtHxH9HxJ4R8X7guMp0qIzz2Yg4IiJ2i4gDgAuBXYHzB2eWJElS2bTC\nqTpSShfBkM95AAAMEUlEQVRV7tl0OvmU223A5JrbB4wFxtcMPy8ijgamAR8G/gG8M6VU+592OwDf\nq4y7GJgJTKrc7kCSJKnPWiI4AaSUzgXO7eWzd/TQ71rybQx6m97HgI/VrYCSJGmL1yqn6iRJklqe\nwUmSJKkgg5MkSVJBBidJkqSCDE6SJEkFGZwkSZIKMjhJkiQVZHCSJEkqyOAkSZJUkMFJkiSpIIOT\nJElSQQYnSZKkggxOkiRJBRmcJEmSCjI4SZIkFWRwkiRJKsjgJEmSVJDBSZIkqSCDkyRJUkEGJ0mS\npIIMTpIkSQUZnCRJkgoyOEmSJBVkcJIkSSrI4CRJklSQwUmSJKkgg5MkSVJBBidJkqSCDE6SJEkF\nGZwkSZIKMjhJkiQVZHCSJEkqyOAkSZJUkMFJkiSpIIOTJElSQQYnSZKkggxOkiRJBRmcJEmSCjI4\nSZIkFWRwkiRJKsjgJEmSVJDBSZIkqSCDkyRJUkEGJ0mSpIIMTpIkSQUZnCRJkgoyOEmSJBVkcJIk\nSSrI4CRJklSQwUmSJKkgg5MkSVJBBidJkqSCDE6SJEkFGZwkSZIKMjhJkiQVZHCSJEkqyOAkSZJU\nkMFJkiSpIIOTJElSQQYnSZKkggxOkiRJBRmcJEmSCjI4SZIkFWRwkiRJKsjgJEmSVJDBSZIkqSCD\nkyRJUkEGJ0mSpIIMTpIkSQUZnCRJkgpqmeAUER+IiLkRsSIibo6IgzYx/CsiYmZErIyIeyLipB6G\neVNEzK5M8/aIOLJxc6B6mz59erOLoBrWR2uxPlqL9bHlaIngFBHHA18HTgMOAG4HZkTE6F6GnwD8\nDrgG2B84Gzg/Io6oGeZQ4GfA94EXAZcCl0TEPg2bEdWVG6LWYn20FuujtVgfW46WCE7AVOC8lNKP\nU0p3A+8FlgMn9zL8+4AHUkqnppTmpJS+DVxcmU7Vh4ErUkpnVYb5HNAJfLBxsyFJksqs6cEpIoYD\nE8mtRwCklBJwNTCpl9EOqXxea0a34ScVGEaSJKmwpgcnYDQwFFjQrf8CYGwv44ztZfhREdG2iWF6\nm6YkSdJGDWt2AVrMVgCzZ89udjkELFmyhM7OzmYXQxXWR2uxPlqL9dE6avbhWzVi+q0QnBYB64D2\nbv3bgfm9jDO/l+GXppRWbWKY3qYJMAHgxBNP3HiJNWgmTpzY7CKohvXRWqyP1mJ9tJwJwI31nmjT\ng1NKaU1EzAQOA34DEBFReX9OL6PdBHS/tcCrK/1rh+k+jSO6DdPdDOCtwDxgZbE5kCRJLWQrcmia\n0YiJR74Ou7ki4s3ABeT/pruF/N9xxwF7pZQWRsQZwLiU0kmV4ScAdwDnAj8kB6RvAEellK6uDDMJ\n+BPwKeAyYArwSaAjpXTXIM2aJEkqkaa3OAGklC6q3LPpdPLptNuAySmlhZVBxgLja4afFxFHA9PI\ntx34B/DOamiqDHNTRJwAfLnS3Qsca2iSJEn91RItTpIkSZuDVrgdgSRJ0mbB4FTR12flaeAi4lMR\ncUtELI2IBRHxfxHx/B6GOz0iHo2I5RHx+4jYoxnl3dJExCcjYn1EnNWtv/UxSCJiXET8JCIWVZb3\n7RHR0W0Y62MQRMSQiPhiRDxQWdb3RcRnehjO+miAiHhpRPwmIh6pbJeO6WGYjS77iGiLiG9Xfk9P\nRcTFEbFTX8ticKLvz8pT3bwU+CZwMHA4MBy4KiK2rg4QEf9FfkzOKcCLgWXkuhkx+MXdclQOHE4h\n/xZq+1sfgyQitgduAFYBk4G9gf8HLK4ZxvoYPJ8E3gO8H9gLOBU4NSL+9Rgv66OhtiFf//x+4BnX\nGBVc9t8AjgbeCLwMGAf8qs8lSSlt8R1wM3B2zfsgX3B+arPLtiV15LvIrwdeUtPvUWBqzftRwArg\nzc0ub1k7YFtgDvAq4I/AWdZHU+rhTODPmxjG+hi8+vgt8P1u/S4Gfmx9DHpdrAeO6dZvo8u+8n4V\n8PqaYfasTOvFffn+Lb7FqZ/PylNjbE8+kngSICJ2I/9HZW3dLAX+gnXTSN8GfptS+kNtT+tj0L0O\nuDUiLqqcyu6MiHdVP7Q+Bt2NwGER8W8AEbE/8O/A5ZX31keTFFz2B5LvJFA7zBzgIfpYPy1xO4Im\n29iz8vYc/OJsmSo3Pf0GcH3qumXEWHKQ8pmDgyQi3gK8iLyR6c76GFy7A+8jX0bwZfLph3MiYlVK\n6SdYH4PtTHKrxd0RsY58qcunU0o/r3xufTRPkWXfDqyuBKrehinE4KRWcS6wD/kITk0QEc8hh9fD\nU0prml0eMQS4JaX02cr72yPiBeQbBf+kecXaYh0PnAC8BbiLfIBxdkQ8Wgmy2kJs8afq6N+z8lRH\nEfEt4CjgFSmlx2o+mk++3sy6GRwTgTFAZ0SsiYg1wMuBj0TEavKRmfUxeB4Duj9xfDawa+Vvfx+D\n66vAmSmlX6aU7kwpXUi+CfOnKp9bH81TZNnPB0ZExKiNDFPIFh+cKkfW1WflARs8K6/uDwfUhiqh\n6VjglSmlh2o/SynNJa/QtXUzivxfeNZN/V0NvJB8JL1/pbsV+Cmwf0rpAayPwXQDz7xcYE/gQfD3\n0QQjyQfZtdZT2Y9aH81TcNnPBNZ2G2ZP8oHIxp5h+wyeqsvOAi6oPGy4+qy8keTn56lBIuJc8jME\njwGWRUT1aGFJSqn6kOVvAJ+JiPvID1/+Ivk/Hi8d5OKWXkppGfkUxL9ExDLgiZRSteXD+hg804Ab\nIuJTwEXkncC7gHfXDGN9DJ7fkpf1P4A7gQ7yvuL8mmGsjwaJiG2APcgtSwC7Vy7QfzKl9DCbWPYp\npaUR8QPgrIhYDDwFnAPckFK6pU+Fafa/FbZKR743xDzyvy/eBBzY7DKVvSMfra3roXtbt+E+T/5X\n0+Xkp13v0eyybykd8AdqbkdgfQz68j8K+FtlWd8JnNzDMNbH4NTFNuSD7LnkewTdC3wBGGZ9DMry\nf3kv+4wfFl32QBv53oGLKsHpl8BOfS2Lz6qTJEkqaIu/xkmSJKkog5MkSVJBBidJkqSCDE6SJEkF\nGZwkSZIKMjhJkiQVZHCSJEkqyOAkSZJUkMFJkiSpIIOTJPUgItZHxDHNLoek1mJwktRyIuJ/K8Fl\nXeW1+vflzS6bpC3bsGYXQJJ6cQXwdrqehg6wqjlFkaTMFidJrWpVSmlhSunxmm4J/Os02nsj4vKI\nWB4R90fEG2tHjogXRMQ1lc8XRcR5EbFNt2FOjoi/R8TKiHgkIs7pVoYxEfHriFgWEfdExOtqxt0+\nIi6MiMcr3zEnIk5q2NKQ1BIMTpI2V6cDvwT2Ay4Efh4RewJExEhgBvAEMBE4Djgc+GZ15Ih4H/At\n4LvAvsDRwD3dvuNzwM+BFwKXAxdGxPaVz74E7AVMrry+D1hU75mU1FoipdTsMkjSBiLif4ETgZU1\nvRPwlZTSmRGxHjg3pfTBmnFuAmamlD4YEe8GzgCek1JaWfn8SOC3wM4ppYUR8Q/gByml03opw3rg\n9JTS5yvvRwJPA69JKV0VEZcCC1NK76rv3EtqZV7jJKlV/QF4Lxte4/Rkzd83dxv+JmD/yt97AbdX\nQ1PFDeRW9j0jAmBc5Ts25o7qHyml5RGxFNip0us7wK8iYiJwFXBJSummTc2UpM2bwUlSq1qWUprb\noGmvKDjcmm7vE5VLHFJKV0bErsBRwBHA1RHx7ZTSqfUrpqRW4zVOkjZXh/Twfnbl79nA/hGxdc3n\nLwHWAXenlJ4G5gGHDaQAKaUnUko/SSm9DZgKnDKQ6UlqfbY4SWpVbRHR3q3f2pTSE5W/3xQRM4Hr\nyddDHQScXPnsQuDzwI8i4gvk02vnAD9OKVUv4P488J2IWEi+9cEo4NCU0reKFK4y3ZnAncBWwGuB\nu/o6k5I2LwYnSa3qNcCj3frNAfap/H0a8Bbg28BjwFtSSncDpJRWRMRk4GzgFmA5cDHw/6oTSin9\nOCLayC1FXyP/R9zFNd/V03/OpJr+q4GvABPIp/6uA6b0Yz4lbUb8rzpJm53Kf7z9R0rpN80ui6Qt\ni9c4SZIkFWRwkrQ5sqlcUlN4qk6SJKkgW5wkSZIKMjhJkiQVZHCSJEkqyOAkSZJUkMFJkiSpIIOT\nJElSQQYnSZKkggxOkiRJBRmcJEmSCvr/YSwKaCdrwd8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c909748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1, sgd_lr.epochs+1), sgd_lr.cost_)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Cost')\n",
    "plt.ylim([0, 0.2])\n",
    "plt.tight_layout()\n",
    "plt.show()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3 - Stochastic Gradient Descent with Minibatches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.24\n",
      "Slope: 0.82\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfcAAAFkCAYAAAA9h3LKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XmcXvPd//HXNxE7kba0Wty920xy2x7IWIIKaioRDU1R\nnSwaJBG7qTXaUu5bqZbU0lS1ahuGUvwokoq1QYRMhUiYmRRB7Ymxk0y+vz++U5E0ItfMNXPmOtfr\n+XjMozNnruXjNDPv+ZzzXUKMEUmSlB/dsi5AkiQVl+EuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklS\nzhjukiTljOEuSVLOGO6SJOWM4S5JUs4UHO4hhK+GEK4OIbwRQng/hDAzhNCvI4qTJEmFW6WQB4cQ\n1gMeBO4GBgJvABXAguKXJkmS2iIUsnFMCOEcYMcY464dV5IkSWqPQi/LDwEeCyH8OYTwagihPoQw\nuiMKkyRJbVNo5/4BEIHzgBuB7YELgMNijFcv5/FfJF2+fw74sAj1SpJULlYHvg5MjjG+WcgTCw33\nj4DpMcZdPnXsAmDbGOPOy3n8MOCaQgqSJElLGR5jvLaQJxQ0oA54GZizzLE5wPc/4/HPAdTW1rLp\nppsW+Fb5U1NTw4QJE7IuI3OehyU8F4nnYQnPReJ5gDlz5jBixAhozdJCFBruDwJ9lznWF3j+Mx7/\nIcCmm25Kv37OluvZs6fnAc/Dp3kuEs/DEp6LxPOwlIJvaxc6oG4C0D+EMD6E8M3Wy+6jgYsLfWNJ\nktQxCgr3GONjwFCgGngS+AlwbIzxug6oTZIktUGhl+WJMd4B3NEBtUiSpCJwbflOVF1dnXUJXYLn\nYQnPReJ5WMJzkXge2qegqXAFv3hac37GjBkzHBghSVIB6uvrqaysBKiMMdYX8lw7d0mScsZwlyQp\nZwx3SZJyxnCXJClnDHdJknLGcJckKWcMd0mScsZwlyQpZwx3SZJyxnCXJClnDHdJknLGcJckKWcM\nd0mScsZwlyQpZwx3SZJyxnCXJClnDHdJknLGcJckKWcMd0mScsZwlyQpZwx3SZJyxnCXJClnDHdJ\nknLGcJckKWcMd0mScsZwlyQpZwx3SZJyxnCXJClnDHdJknLGcJckKWcMd0mScsZwlyQpZwx3SZJy\nxnCXJClnDHdJUteycCG8+WbWVZQ0w12S1HU89hhstx2MGpV1JSXNcJckZe+99+D442GHHaBbNzjj\njKwrKmmrZF2AJKnMTZ4M48bBK6/AOedATQ2sYjy1h527JCkbb7wBBx0EgwbBN78Js2bBiSca7EVQ\nULiHEE4PISxe5mN2RxUnScqhGOGaa2DTTeGvf4XLL4e77koBr6Joy59Hs4A9gND69aLilSNJyrXn\nnkuX4CdPhupq+M1vYIMNlnpIQ0MDc+fOpXfv3lRUVGRTZ4lry2X5RTHG12OMr7V+zC96VZKkfGlp\ngQkTYPPNYfZsuP12uPbapYJ9/vz5DBq0N3379mXw4MH06dOHQYP2ZsGCBRkWXpraEu4VIYSXQghz\nQwi1IYSNi16VJCk/Zs6E/v3TaPjRo+Gpp2Dw4P942LBhI5kyZRpQC8wDapkyZRrV1SM6u+KSV2i4\nTwNGAQOBccB/Aw+EENYqcl2SpFL3wQdw6qlQWQkffggPPQQXXADrrPMfD21oaGDy5DtoabkQGA5s\nDAynpeUCJk++g8bGxs6uvqQVdM89xjj5U1/OCiFMB54HfgBc/lnPq6mpoWfPnksdq66uprq6upC3\nlySVinvvhbFjYd48+PnP4aSTYNVVP/Phc+fObf1swDLf2RWApqamXN9/r6uro66ubqljzc3NbX69\nds03iDE2hxAagN4retyECRPo169fe95KklQKFixI09kuuwwGDEij4fv2/dynffOTkfIPkDr3f7sf\ngN69VxgzJW95DW99fT2VlZVter12zXMPIaxNCvaX2/M6kqQSFyPccEOa3nbjjfD736fufSWCHaBP\nnz4MHDiY7t2PId1zfwGopXv3Yxk4cHCuu/aOUOg891+FEAaEEP4rhLATcDOwEKj7nKdKkvLqxRdh\n333hBz+AnXdOo+HHjk3LyBagrq6Wqqr+wEhgE2AkVVX9qaur7Yiqc63Qy/IbAdcCXwReB6YC/WOM\nbt8jSeVm8WK45BI45RRYe2246SYYOrTNL9erVy8mTbqdxsZGmpqanOfeDoUOqHMEnCQpdedjxqQR\n8IcdltaEX2+9orx0RUWFod5Ori0vSVp5H32URr9vvXXac/2BB1L3XqRgV3G4Or8kaeU8+GDq1hsb\nYfz4NId99dWzrkrLYecuSVqxt9+GI46Ab30L1l0X6uvhzDMN9i7Mzl2S9NluvTUFe3MzXHhh+rx7\n96yr0uewc5ck/adXXoEDDkhT3LbeOq0Hf/TRBnuJsHOXJC0RY1pd7sQT03Kx112X5q+H8PnPVZdh\n5y5JShob4dvfToPmhg6FOXPgwAMN9hJkuEtSuVu4EM4+G7bcMm30ctdd8Kc/wRe+kHVlaiMvy0tS\nOXv00SV7rB9/PJx+Oqy5ZtZVqZ3s3CWpHL37LtTUQP/+sMoqKeR/+UuDPSfs3CWp3EyaBOPGwWuv\nwbnnwrHHpoBXbti5S1K5eP11GDEC9toLKipg1qx0Kd5gzx3/H5WkvIsRamvTZfgY4corYeRIR8Hn\nmJ27JOXZs8/CwIFw0EHpf+fMSZ8b7LlmuEtSHi1aBOefD1tsAc88A3fcAddcAxtskHVl6gSGuyTl\nzeOPp1HwJ5yQFqR56ql0n11lw3CXpLz44AM45RTYdlv4+GN4+GH4zW9g7bWzrkydzAF1kpQH99wD\nY8fCiy+m7VhPPBF69Mi6KmXEzl2SStn8+XDoobDHHrDRRvDEE3DqqQZ7mbNzl6RSFCPccEPahvWj\nj+DSS1PId7Nnk527JJWeF16AffZJO7btskua3jZmjMGuT/gvQZJKRUsLXHwxbLYZ1NfDzTfDjTfC\nhhtmXZm6GMNdkkrBU0/Bt76VLsOPHAmzZ8P3vpd1VeqiDHdJ6so++ghOOw222Qbeegv+/neYOBF6\n9sy6MnVhDqiTpK5q6tR0L33uXBg/Po2CX221rKtSCbBzl6SuprkZDj88DZZbbz34xz/gjDMMdq00\nO3dJ6kpuuQWOPBLefhsuuiiFfPfuWVelEmPnLkldwb/+BfvtB0OHQr9+acDcUUcZ7GoTw12SsrR4\ncVqAZrPN0j3266+HW2+FjTfOujKVMMNdkrLyzDOw++5w2GGpa58zB37wA/daV7sZ7pLU2T7+GM46\nC7baCl56Ce6+Gy67DL7whawrU044oE6SOtMjj6TpbbNnp/3WTz8d1lgj66qUM3buktQZ3n0XjjsO\ndtwRVl0VHnsMzjnHYFeHsHOXpI52xx1pStvrr8Ovfw3HHAOr+OtXHcfOXZI6ymuvwbBhsPfe0Lcv\nzJoFP/6xwa4O578wSSq2GOGqq1KQh5A+HzHCUfDqNHbuklRM//wn7LknjBoFe+2VpreNHGmwq1MZ\n7pJUDIsWpfvpW2wBjY1w551QWwvrr591ZSpDhrsktVd9PeywA5x8Mowbl+6tDxqUdVUqY4a7JLXV\n++/DSSfB9tvDwoXw8MNw/vmw9tpZV6Yy165wDyGcEkJYHEI4v1gFSVJJuPtu2HJLuPBC+N//hRkz\nUshLXUCbwz2EsB0wFphZvHIkqYt78004+GCoqoJNNoEnnoDx46FHj6wrkz7RpnAPIawN1AKjgbeK\nWpEkfY6GhgbuvPNOGhsbO+9NY4TrroNNN4Wbb4Y//hHuuQf69Om8GqSV1NbO/bfAbTHGe4pZjCSt\nyPz58xk0aG/69u3L4MGD6dOnD4MG7c2CBQs69o3nzYPvfheqq2HXXdP0tkMPdXqbuqyCwz2E8ENg\na2B88cuRpM82bNhIpkyZRrpwOA+oZcqUaVRXj+iYN2xpgYsuSnutz5wJt9wCN9wAG27YMe8nFUlB\nK9SFEDYCfgNUxRgXdkxJkvSfGhoamDz5DlKwD289OpyWlsjkySNpbGykoqKieG84axaMHp12cTvi\nCDj7bFh33eK9vtSBCl1+thJYH6gP4ZPrUd2BASGEo4DVYoxx2SfV1NTQs2fPpY5VV1dTXV3dhpIl\nlaO5c+e2fjZgme/sCkBTU1Nxwv3DD9Ne6+ecAxUVMHUq7Lxz+19XWoG6ujrq6uqWOtbc3Nzm1wvL\nyeLPfnAIawH/tczhK4A5wDkxxjnLPL4fMGPGjBn069evzUVKUkNDA3379mXpzp3Wr0fS0NDQ/nB/\n4AEYOzYtIXvqqWkU/Gqrte81pTaqr6+nsrISoDLGWF/Icwvq3GOM7wGzP30shPAe8OaywS5JxdSn\nTx8GDhzMlCnH0NISSR37/XTvfixVVYPbF+xvvZVWl7v00rTf+l/+AptvXqzSpU5XjBXqVr71l6R2\nqKurpaqqPzAS2AQYSVVVf+rqatv+ojffnAbM1dXBb3+bLsMb7Cpx7d7yNcb47WIUIkmfp1evXkya\ndDuNjY00NTXRu3fvtnfs//oXHHVUCvchQ2DiRNhoo+IWLGXE/dwllZyKioq2h/rixfCHP6Q14ddY\nA/78Z9h/f+esK1fcOEZS+Xj6adhtt7Rz2wEHpMVoDjjAYFfuGO6S8u/jj9PmLlttBS+/nJaN/eMf\noVevrCuTOoSX5SXl27RpaTGap59Ol+J/9rN0OV7KMTt3Sfn0zjtwzDGw004pzGfMgF/8wmBXWbBz\nl5Q/t98Ohx+etmc977wU8t27Z12V1Gns3CXlx6uvpp3bvvvdNHd91iyoqTHYVXbs3CWVvhjhiivg\n+OOhWze4+moYPtxR8Cpbdu6SStvcufCd78Ahh8Dee6fpbSNGGOwqa4a7pNK0aBGcey5suWUK+EmT\nUse+/vpZVyZlznCXVHrq62H77dOubYcfnu6tDxyYdVVSl2G4Syod778PJ54I220HLS1pDvt558Fa\na2VdmdSlOKBOUmm46y447LC0wtxZZ6XBcz16ZF2V1CXZuUvq2t58E370I9hzT/j61+GJJ+CUUwx2\naQXs3CV1TTGmPdaPOw4WLoTLLoODD3YUvLQS7NwldT3PP5+mtQ0fDrvvnqa3HXKIwS6tJMNdUtfR\n0gIXXACbbw5PPgm33grXXw9f+UrWlUklxXCX1DU88UTa5KWmBkaNgqeegiFDsq5KKkmGu6Rsffgh\n/OQnUFkJ774LU6fCxRfDuutmXZlUshxQJyk7998PY8fCc8+lfdZPPhlWWy3rqqSSZ+cuqfO99VYK\n9d12S8vFPv44nHaawS4ViZ27pM4TI9x0Exx1FLz3HkycmBam6WafIRWTP1GSOsdLL8HQobD//rDD\nDjB7dloX3mCXis6fKkkda/Fi+N3vYLPN4JFH4MYb4eabYaONsq5Myi3DXVLHmTMHBgyAI46AAw9M\n3fp++7kYjdTBDHdJxffxx3DmmbD11vDaa3DffXDppdCrV9aVSWXBAXWSiuvhh2HMGHjmGTjppDTF\nbfXVs65KKit27pKK4+230yj4nXeGNdeEGTPS1qwGu9Tp7Nwltd9tt6X76gsWwPnnw9FHQ/fuWVcl\nlS07d0lt98oraaDcPvvAllum9eCPO85glzJm5y6pcDHC5ZfD8cfDKqvANddAdbWj4KUuws5dUmGa\nmmCPPeDQQ1PHPmcODBtmsEtdiOEuaeUsXAjnnJMuvz/3HPztb3DllfClL2VdmaRlGO6SPt9jj8F2\n26WtWY86Cp58Er7znayrkvQZDHdJn+2999J99R12SJfdp0+HX/0K1lor68okrYAD6iQt3+TJMG5c\nGhF/9tlQUwM9emRdlaSVYOcuaWlvvAEjR8KgQfCNb6RL8CedZLBLJcTOXVISI1x7bZqn3tICf/oT\njBrlKHipBNm5S0qj3/faC0aMgKqqNL3t4IMNdqlEGe5SOWtpgQkTYPPN03asf/0r1NXBl7+cdWWS\n2sFwl8rVzJnQv38aDX/ooWnp2L33zroqSUVQULiHEMaFEGaGEJpbPx4KIQzqqOIkdYAPPoDx46Gy\nMn3+0ENw4YWwzjpZVyapSAodUPcCcDLQCARgFPD/QghbxxjnFLk2ScV2770wdizMmwc//3kaBb/q\nqst9aENDA3PnzqV3795UVFR0bp2S2qWgzj3GeHuMcVKMcW6MsSnG+FPgXaB/x5QnqSgWLIDRo+Hb\n34YNN0yX5H/60+UG+/z58xk0aG/69u3L4MGD6dOnD4MG7c2CBQsyKFxSW7T5nnsIoVsI4YfAmsDD\nxStJUtHECDfcAJtumv73kkvgvvvgf/7nM58ybNhIpkyZBtQC84BapkyZRnX1iE4qWlJ7FTzPPYSw\nBSnMVwfeAYbGGJ8udmGS2unFF+GII+C222DoULjoIvja11b4lIaGBiZPvoMU7MNbjw6npSUyefJI\nGhsbvUQvlYC2LGLzNLAV0BPYH7gqhDBgRQFfU1NDz549lzpWXV1NdXV1G95e0gotXpw69FNOgbXX\nhr/8Bb7//ZV66ty5c1s/G7DMd3YFoKmpyXCXOkBdXR11dXVLHWtubm7z64UYY7sKCiHcBTTFGA9f\nzvf6ATNmzJhBv3792vU+klbC7NkwZkwaAX/YYWmL1vXWW+mnNzQ00LdvX5bu3Gn9eiQNDQ2Gu9RJ\n6uvrqaysBKiMMdYX8txizHPvBqxWhNeR1FYffZRGv2+9dVob/v77U/deQLAD9OnTh4EDB9O9+zGk\nQH8BqKV792MZOHCwwS6ViELnuf8ihLBLCOG/QghbhBDOJl2vq+2Y8iR9rgcfhG22gbPOgpNPTiPh\nByx7WX3l1dXVUlXVHxgJbAKMpKqqP3V1/phLpaLQe+4bAFcCGwLNwBPAnjHGe4pdmKTP8fbb6b76\n736X9luvr4ctt2z3y/bq1YtJk26nsbGRpqYm57lLJaigcI8xju6oQiQV4NZb00j4t96CCy6AI4+E\n7t2L+hYVFRWGulSiXFteKiWvvAIHHAD77gtbbZUG0B1zTNGDXVJpcz93qRTECJddBieeCD16pJ3b\nDjzQLVklLZedu9TVNTamZWPHjIHvfS/ttf7DHxrskj6T4S51VQsXwtlnp0Fy8+bBXXfB5ZfDF7+Y\ndWWSujgvy0td0aOPpo1ennoKfvzjNId9zTWzrkpSibBzl7qSd9+Fmhro3z8Nkps+Hc4912CXVBA7\nd6mrmDQJxo2D115Ly8bW1MAq/ohKKpydu5S111+HESNgr72gogKefDKNijfYJbWRvz2krMQItbWp\nQ48RrrgCDjrIUfCS2s3OXcrCs8/CwIEpzPfcM01v+9GPDHZJRWG4S51p0SI4/3zYYgt4+mm4/Xa4\n9lrYYIOsK5OUI4a71FkefzyNgj/hhCXT3AYPzroqSTlkuEsd7YMP0u5t226b9l1/+OG02cs662Rd\nmaScckCd1JHuuQfGjoUXX4Qzzkij4FddNeuqJOWcnbvUEebPh0MPhT32gK99DWbOhJ/8xGCX1Cns\n3KViihFuuAGOPjpdgr/00hTy3fw7WlLn8TeOVCwvvAD77JO2Yv3Wt9Je62PGGOySOp2/daT2ammB\niy+GzTaDGTPgppvgL3+Br34168oklSnDXWqPp56CXXZJl+FHjEiL0QwdmnVVksqc4S61xUcfwemn\nwzbbwIIF8MAD8LvfQc+eWVcmSQ6okwo2dWq6lz53bpq/fuqpsPrqWVclSZ+wc5dWVnMzHH54ugy/\n3npQXw9nnmmwS+py7NyllXHLLXDkkfD223DRRSnku3fPuipJWi47d2lFXn4Z9t8/DZLr1y9Nbzvq\nKINdUpdmuEvLs3gx/OEPsOmm8Pe/w3XXwa23wsYbZ12ZJH0uw11a1jPPwO67pzXhv//9NL3twAPd\na11SyTDcpX/7+GM46yzYait46SWYMgX+9Cf4wheyrkySCuKAOgngkUfS9LbZs9N+66edBmuumXVV\nktQmdu4qb+++C8cdBzvumHZse/RROOccg11SSbNzV/m6804YNw5efx1+9Ss49lhYxR8JSaXPzl3l\n57XXYPhwGDwY+vaFWbPg+OMNdkm54W8zlY8Y4aqr4Mc/Tl9feSWMHOkoeEm5Y+eu8vDPf8Kee8Ko\nUbDXXvD003DQQQa7pFwy3JVvixbBr38NW2wBjY1wxx1QWwvrr591ZZLUYQx35dc//gE77AAnnQSH\nHZbure+1V9ZVSVKHM9yVP++/DyefDNttBwsXwrRpMGECrL121pVJUqdwQJ3y5e6707KxL72UtmM9\n8UTo0SPrqiSpU9m5Kx/mz4eDD4aqqrS5yxNPwKmnGuySypKdu0pbjHD99WkBmo8+Sju5HXIIdPPv\nVknly9+AKl3z5sGQIVBdDQMGpN3bRo822CWVPX8LqvS0tMBFF8Hmm6cR8bfcAjfcABtumHVlktQl\nFBTuIYTxIYTpIYS3QwivhhBuDiH06ajipP8waxbsvDMcc0xaXW72bNh336yrkqQupdDOfRfgImAH\noAroAfwthLBGsQtT+WloaODOO++ksbHxP7/54YdpG9ZttoHmZvj732HiROjZs/MLlaQurqABdTHG\nwZ/+OoQwCngNqASmFq8slZP58+czbNhIJk++45NjAwcOpq6ull69eqUgHzMmLSH7k5/A+PGw2moZ\nVixJXVt777mvB0RgfhFqUZkaNmwkU6ZMA2qBeUAtU6ZM49D9D0xbsg4YAF/4Qrq//vOfG+yS9Dna\nPBUuhBCA3wBTY4yzi1eSyklDQ0Nrx14LDG89OpwhLY9y8T0XsPiRteh28cVw+OGOgpekldSeee4T\ngc2AnT/vgTU1NfRc5t5odXU11dXV7Xh75cHcuXNbPxsAwIb8i4s4mv24iVuBdSZOZPeDDsqsPknq\nDHV1ddTV1S11rLm5uc2vF2KMhT8phIuBIcAuMcZ5K3hcP2DGjBkz6NevX5uLVH41NDTQt29fAlcx\nmg84l5P4kNU5mgO4kYtpaGigoqIi6zIlqdPV19dTWVkJUBljrC/kuQV37q3Bvi+w64qCXVoZffr0\n4dBvDeBHUw9hFxZxGQdyArvzTvdTGVg12GCXpDYoKNxDCBOBamAf4L0Qwpdbv9UcY/yw2MUp5z7+\nGM49lz9Mn8bLa6zKtz9YxL1cD1zPwKo0Wl6SVLhCO/dxpNHx9y1z/GDgqmIUpDIxbVqa3jZnDuHE\nE/nqaafx+xdfpKmpid69e9uxS1I7FDrP3eHKap933klz1S++GCor4bHHYOutAaioqDDUJakI3BVO\nnef229OUtjffhPPOg6OPhlX8JyhJxWYnro732mtp57bvfhc22yytD19TY7BLUgfxt6s6Toxw5ZVw\n/PEQAlx9NQwfnj6XJHUYO3d1jLlz4TvfgYMPhsGD017rI0YY7JLUCQx3FdeiRXDuubDlltDUBJMm\npY59/fWzrkySyobhruKpr4ftt0+7to0bl+6tDxyYdVWSVHYMd7Xf++/DiSemYF+8OM1hP/98WHvt\nrCuTpLLkgDq1z5QpcNhh8NJL8H//lwbP9eiRdVWSVNbs3NU2b74Jo0alQXObbAJPPgmnnGKwS1IX\nYOeuwsQIdXVw3HGwcCH88Y9wyCGOgpekLsTOXSvv+edh773TXPXddkvT2w491GCXpC7GcNfna2mB\nCy6AzTdPl99vvRX+/Gf4yleyrkyStByGu1bsySdhp53ScrGjRsFTT8GQIVlXJUlaAcNdy/fhh/DT\nn0K/fmknt6lT005u666bdWWSpM/hgDr9p/vvh7Fj4dlnU8CfcgqstlrWVUmSVpKdu5Z4660U6rvt\nBl/6Ejz+OJx+usEuSSXGzl3JTTfBkUfCe+/BxIlpYZpu/u0nSaXI397l7qWXYOhQ2G8/2GEHmD0b\nDj/cYJekEuZv8HK1eDFccglstllaC/7GG+Hmm2GjjbKuTJLUToZ7OZozB3bdNXXoBx6YuvX99nMx\nGknKCcO9nHz8MZx5Jmy9Nbz6Ktx7L1x6KfTqlXVlkqQickBduXj4YRgzBp55Bk46KU1xW2ONrKuS\nJHUAO/e8e+cdOPpo2HlnWHNNmDEDzjrLYJekHLNzz7O//jXdV1+wAM4/P4V89+5ZVyVJ6mB27nn0\n6qtpoNyQIbDllmk9+OOOM9glqUzYuedJjHD55XDCCSnIr7kGqqsdBS9JZcbOPS+amqCqKu2vPmRI\nmu42bJjBLkllyHAvdQsXwi9/mS6/P/ssTJ4MV16Z1oaXJJUlL8uXshkzYPRoeOKJtN/6GWfAWmtl\nXZUkKWN27qXovffg+ONh++3T19Onw69/bbBLkgA799Lzt7+lHdteeQXOPjt17D16ZF2VJKkLsXMv\nFW+8AQcdBAMHwje+AU8+mVaaM9glScuwc+/qYoRrr03z1Fta4E9/glGjHAUvSfpMdu5d2XPPweDB\nMGIE7LFHmt528MEGuyRphQz3rqilBSZMgM03h1mz4Lbb4Lrr4MtfzroySVIJMNy7mpkzYccd02j4\nQw9Ne61/97tZVyVJKiGGe1fxwQdw6qmw7bbw/vvw0ENw4YWwzjpZVyZJKjEOqOsK7rsPxo6F55+H\n006Dk0+GVVfNuipJUomyc8/SggUwZgzsvnu6nz5zJvzsZwa7JKld7NyzECP85S9w1FHpcvwll6SQ\n7+bfWpKk9is4TUIIu4QQbg0hvBRCWBxC2KcjCsutF1+E730PDjgAdtopDZg77DCDXZJUNG1JlLWA\nx4EjgFjccnJs8WKYOBE22wwefTR17jfdBF/7WtaVSZJypuDL8jHGScAkgBBcTWWltLTw/k47seb0\n6TQfeCA9L7kE1lsv66okSTnlteAONn/+fAbtvQ/jp09nALDe9dcz6IfDWbBgQdalSZJyynDvYMOG\njWTKlGlcSC1/Zx5Qy5Qp06iuHpF1aZKknHK0fAdqaGhg8uQ7gFpgeOvR4bS0RCZPHkljYyMVFRUZ\nVihJyqNOCfeamhp69uy51LHq6mqqq6s74+0zM3fu3NbPBizznV0BaGpqMtwlSdTV1VFXV7fUsebm\n5ja/XqeE+4QJE+jXr19nvFWX8s1vfrP1swdY0rkD3A9A7969O7skSVIXtLyGt76+nsrKyja9XsHh\nHkJYC+jT2Z/4AAAHO0lEQVQN/Huk/DdCCFsB82OML7Spipzq06cPAwcOZsqUY2hpiaSO/X66dz+W\nqqrBdu2SpA7RlgF12wL/AGaQ5rmfB9QDZxSxrtyoq6ulqqo/MBLYBBhJVVV/6upqM65MkpRXbZnn\nfj+Osl9pvXr1YtKk22lsbKSpqYnevXvbsUuSOpSj5TtJRUWFoS5J6hR24JIk5YzhLklSzhjukiTl\njOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5Yzh\nLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5J\nUs4Y7pIk5YzhLklSzhjukiTljOEuSVLOGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOEuSVLO\nGO6SJOWM4S5JUs4Y7pIk5YzhLklSzhjukiTljOHeierq6rIuoUvwPCzhuUg8D0t4LhLPQ/u0KdxD\nCEeGEJ4NIXwQQpgWQtiu2IXlkf9YE8/DEp6LxPOwhOci8Ty0T8HhHkI4EDgPOB3YBpgJTA4hfKnI\ntUmSpDZoS+deA/w+xnhVjPFpYBzwPnBIUSuTJEltUlC4hxB6AJXA3f8+FmOMwBRgx+KWJkmS2mKV\nAh//JaA78Ooyx18F+i7n8asDzJkzp/DKcqi5uZn6+vqsy8ic52EJz0XieVjCc5F4HpbKztULfW5I\njfdKPjiEDYGXgB1jjI986vgvgQExxh2Xefww4JpCi5IkSZ8YHmO8tpAnFNq5vwG0AF9e5viXgVeW\n8/jJwHDgOeDDAt9LkqRytjrwdVKWFqSgzh0ghDANeCTGeGzr1wGYB1wYY/xVoQVIkqTiKrRzBzgf\nuCKEMAOYTho9vyZwRRHrkiRJbVRwuMcY/9w6p/1M0uX4x4GBMcbXi12cJEkqXMGX5SVJUtfm2vKS\nJOWM4S5JUs50aLi7wQyEEHYJIdwaQngphLA4hLBP1jVlIYQwPoQwPYTwdgjh1RDCzSGEPlnX1dlC\nCONCCDNDCM2tHw+FEAZlXVfWQgintP58nJ91LZ0thHB663/7pz9mZ11XVkIIXw0hXB1CeCOE8H7r\nz0u/rOvqTK25uey/icUhhItW9jU6LNzdYOYTa5EGHR4BlPMAh12Ai4AdgCqgB/C3EMIamVbV+V4A\nTgb6kZZyvgf4fyGETTOtKkOtf/SPJf2OKFezSAOUv9L68a1sy8lGCGE94EHgI2AgsClwPLAgy7oy\nsC1L/i18BfgOKT/+vLIv0GED6j5jPvwLpPnw53bIm3ZxIYTFwPdijLdmXUvWWv/Ie420suHUrOvJ\nUgjhTeCEGOPlWdfS2UIIawMzgMOBnwH/iDH+ONuqOlcI4XRg3xhjWXWnyxNCOIe0AuquWdfSlYQQ\nfgMMjjGu9NXODunc3WBGK2E90l+i87MuJCshhG4hhB+S1ol4OOt6MvJb4LYY4z1ZF5KxitZbd3ND\nCLUhhI2zLigjQ4DHQgh/br19Vx9CGJ11UVlqzdPhwGWFPK+jLsuvaIOZr3TQe6pEtF7F+Q0wNcZY\ndvcWQwhbhBDeIV16nAgMbd0+uay0/mGzNTA+61oyNg0YRboMPQ74b+CBEMJaWRaVkW+QruI8A+wJ\n/A64MIQwMtOqsjUU6AlcWciT2rJCndReE4HNgJ2zLiQjTwNbkX5g9weuCiEMKKeADyFsRPoDryrG\nuDDrerIUY/z0uuGzQgjTgeeBHwDldqumGzA9xviz1q9nhhC2IP3Rc3V2ZWXqEODOGOPy9m/5TB3V\nuRe6wYzKRAjhYmAwsFuM8eWs68lCjHFRjPGfMcZ/xBh/QhpIdmzWdXWySmB9oD6EsDCEsBDYFTg2\nhPBx69WdshRjbAYagN5Z15KBl4Fl9wifA2ySQS2ZCyFsQhqA/IdCn9sh4d76l/gMYI9/H2v9Yd0D\neKgj3lNdX2uw7wvsHmOcl3U9XUg3YLWsi+hkU4AtSZflt2r9eAyoBbaKZbx0Zusgw96koCs3DwJ9\nlznWl3QloxwdQrqdfUehT+zIy/JuMAO03jfrDfy7E/lGCGErYH6M8YXsKutcIYSJQDWwD/BeCOHf\nV3WaY4xlsx1wCOEXwJ2knRTXIQ2U2ZV0f7FsxBjfA5YabxFCeA94M8a4bOeWayGEXwG3kQLsa8AZ\nwEKgLsu6MjIBeDCEMJ407WsHYDQwJtOqMtDaEI8CrogxLi70+R0W7m4w84ltgXtJI8Mjae4/pMER\nh2RVVAbGkf7771vm+MHAVZ1eTXY2IP1/vyHQDDwB7OlocaB814HYCLgW+CLwOjAV6B9jfDPTqjIQ\nY3wshDAUOIc0NfJZ4NgY43XZVpaJKmBj2jjuwo1jJEnKGdeWlyQpZwx3SZJyxnCXJClnDHdJknLG\ncJckKWcMd0mScsZwlyQpZwx3SZJyxnCXJClnDHdJknLGcJckKWf+P1M+eDq5SfFoAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c751048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "sgd_lr = LinearRegression(eta=0.01, \n",
    "                          epochs=100, \n",
    "                          random_seed=0, \n",
    "                          minibatches=3)\n",
    "sgd_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % sgd_lr.b_)\n",
    "print('Slope: %.2f' % sgd_lr.w_)\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, sgd_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk4AAAGGCAYAAACNCg6xAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XmYXGWZ9/HvnYQACRIggURElAwSQDaTQQX1VZRF0MEN\nlAADggIKiIZxww0EV+YCBCGKqCCKEUQHRdGwOooE0YSdEIKyiAokAQJkg6Tv94+netJpupPT3dVd\nnarv57rO1V2nzvKc81Sd+p3nbJGZSJIkac2GNLoAkiRJawuDkyRJUkUGJ0mSpIoMTpIkSRUZnCRJ\nkioyOEmSJFVkcJIkSarI4CRJklSRwUmSJKkig5MkSVJFgyY4RcRxEfFARCyJiJsjYtfVDPuuiLg6\nIh6PiIURcVNE7N3FcAdGxOzaNG+PiH37dykkSVIzGxTBKSLeB5wBnAy8CrgdmB4RY7oZ5f8BVwP7\nAhOBG4ArI2LnDtPcHfgxcAGwC/AL4IqI2L6/lkOSJDW3GAwP+Y2Im4E/ZeZHa68D+DtwTmaeXnEa\ndwE/ycwv1V7/BBiRmft3GGYGcGtmHlvvZZAkSc2v4S1OEbEOMAm4rr1fljR3LbBbxWkE8CLgiQ69\nd6tNo6PpVacpSZLUWcODEzAGGAo81qn/Y8C4itP4BDASuKxDv3F9nKYkSdIqhjW6AH0VEQcDnwf2\nz8z5fZzWaGAf4EFgad9LJ0mSBth6wMuB6Zm5oN4THwzBaT6wAhjbqf9Y4NHVjRgRBwHfAQ7IzBs6\nvf1oL6a5D3DJmgosSZIGvUMoF4nVVcODU2Y+HxEzgbcAv4T/O2fpLcA53Y0XEZOB7wLvy8zfdjHI\njC6msVetf3ceBPjRj37EdtttV6n8V10Fn/883HgjrL9+pVFU0ZQpUzjrrLMaXQzVWB+Di/UxuFgf\ng8fs2bM59NBDofabXm8ND041ZwIX1QLULcAUYARwEUBEfBXYPDMPr70+uPbeCcCfI6K9ZWlJZj5d\n+/9s4HcRcSLwa2Ay5ST0o1ZTjqUA2223HRMnTqxU8Llzy9+ddoIXvajSKKpo1KhRletB/c/6GFys\nj8HF+hiU+uWUm8FwcjiZeRnwceBU4FZgJ2CfzJxXG2Qc8NIOoxxFOaH8POCfHbpvdJjmDOBg4Gjg\nNuDdwDsy8556ln3o0PJ3+fJ6TlWSJA1Gg6XFicycCkzt5r0jOr3eo+I0fwb8rO+l696w2hpcsaI/\n5yJJkgaDQdHitDazxUmSpNZhcOojW5z6z+TJkxtdBHVgfQwu1sfgYn20DoNTH9ni1H/cEA0u1sfg\nYn0MLtZH6zA49ZEtTpIktQ6DUx/Z4iRJUuswOPWRLU6SJLUOg1Mf2eIkSVLrMDj1kS1OkiS1DoNT\nH9niJElS6zA49ZEtTpIktQ6DUx/Z4iRJUuswOPWRLU6SJLUOg1Mf2eIkSVLrMDj1kS1OkiS1DoNT\nH9niJElS6zA49ZEtTpIktQ6DUx/Z4iRJUuswOPWRLU6SJLUOg1Mf2eIkSVLrMDj1kS1OkiS1DoNT\nH9niJElS6zA49ZEtTpIktQ6DUx8Nqa1BW5wkSWp+Bqc+iiiH62xxkiSp+Rmc6mDoUFucJElqBQan\nOhg2zBYnSZJagcGpDmxxkiSpNRic6sAWJ0mSWoPBqQ5scZIkqTUYnOrAFidJklqDwakObHGSJKk1\nGJzqwBYnSZJag8GpDmxxkiSpNRic6sAWJ0mSWoPBqQ5scZIkqTUYnOrAFidJklqDwakObHGSJKk1\nGJzqwBYnSZJag8GpDmxxkiSpNRic6sAWJ0mSWoPBqQ5scZIkqTUYnOrAFidJklqDwakObHGSJKk1\nGJzqwBYnSZJag8GpDmxxkiSpNRic6sAWJ0mSWoPBqQ6GDjU4SZLUCgxOdTBsmIfqJElqBQanOrDF\nSZKk1mBwqgNbnCRJag0GpzqwxUmSpNZgcKoDW5wkSWoNBqc6sMVJkqTWYHCqA1ucJElqDQanOrDF\nSZKk1mBwqgNbnCRJag0GpzqwxUmSpNZgcKoDW5wkSWoNBqc6sMVJkqTWYHCqA1ucJElqDQanOrDF\nSZKk1mBwqgNbnCRJag0GpzqwxUmSpNZgcKoDW5wkSWoNBqc6sMVJkqTWYHCqA1ucJElqDQanOrDF\nSZKk1mBwqoNhwyAT2toaXRJJktSfDE51MHRo+WurkyRJzc3gVAfDhpW/nuckSVJzGzTBKSKOi4gH\nImJJRNwcEbuuZthxEXFJRMyJiBURcWYXwxweEW2199tq3eL+KLstTpIktYZBEZwi4n3AGcDJwKuA\n24HpETGmm1HWBR4HTgNuW82kFwLjOnQvq1eZO7LFSZKk1jAoghMwBTg/My/OzHuBDwGLgSO7Gjgz\nH8rMKZn5I+Dp1Uw3M3NeZj5e6+bVv+i2OEmS1CoaHpwiYh1gEnBde7/MTOBaYLc+Tn6DiHgwIh6O\niCsiYvs+Tq9LtjhJktQaGh6cgDHAUOCxTv0foxxe6605lBar/YFDKMt6U0Rs3odpdskWJ0mSWsOw\nRhegv2TmzcDN7a8jYgYwGziGci5Vt6ZMmcKoUaNW6Td58mQmT57c5fC2OEmSNPCmTZvGtGnTVum3\ncOHCfp3nYAhO84EVwNhO/ccCj9ZrJpm5PCJuBbZe07BnnXUWEydOrDxtW5wkSRp4XTVqzJo1i0mT\nJvXbPBt+qC4znwdmAm9p7xcRUXt9U73mExFDgB2Bf9Vrmu1scZIkqTUMhhYngDOBiyJiJnAL5Sq7\nEcBFABHxVWDzzDy8fYSI2BkIYANg09rr5zJzdu39z1MO1d0PbAR8EtgS+G69C2+LkyRJrWFQBKfM\nvKx2z6ZTKYfobgP26XD7gHHASzuNdiuQtf8nAgcDDwHja/02Br5TG/dJSqvWbrXbHdTVOuuUv889\nV+8pS5KkwWRQBCeAzJwKTO3mvSO66Lfaw4yZeSJwYn1Kt3obbVT+PvXUQMxNkiQ1SsPPcWoGo0eX\nv/PnN7YckiSpfxmc6mDjjSECFixodEkkSVJ/MjjVwdChJTwZnCRJam4GpzoZPdrgJElSszM41YnB\nSZKk5mdwqhODkyRJzc/gVCcGJ0mSmp/BqU4MTpIkNT+DU50YnCRJan4GpzoZPRqeeAIy1zysJEla\nOxmc6mT06PKQ34ULG10SSZLUXwxOddL+2BUP10mS1LwMTnVicJIkqfkZnOrE4CRJUvMzONWJwUmS\npOZncKqT9dcv3fz5jS6JJEnqLwanOhozxhYnSZKamcGpjrwJpiRJzc3gVEcGJ0mSmpvBqY4MTpIk\nNTeDUx0ZnCRJam4GpzoyOEmS1NwMTnVkcJIkqbkZnOpo9GhYvBiWLm10SSRJUn8wONWRdw+XJKm5\nGZzqyOAkSVJzMzjVkcFJkqTmZnCqI4OTJEnNzeBUR6NGwZAhBidJkpqVwamOhgwprU7z5ze6JJIk\nqT8YnOrMezlJktS8DE51ZnCSJKl5GZzqzOAkSVLzMjjVmcFJkqTmZXCqM4OTJEnNy+BUZwYnSZKa\nl8GpzkaPhiefhBUrGl0SSZJUbwanOhs9GjLhqacaXRJJklRvBqc687ErkiQ1L4NTnRmcJElqXgan\nOjM4SZLUvAxOdWZwkiSpeRmc6mz4cNhgA4OTJEnNyODUD8aMgfnzG10KSZJUbwanfuBNMCVJak4G\np35gcJIkqTkZnPqBwUmSpOZkcOoHBidJkpqTwakfGJwkSWpOBqd+0B6cMhtdEkmSVE8Gp34wejQs\nWwaLFze6JJIkqZ4MTv3Au4dLktScDE79wOAkSVJzMjj1A4OTJEnNyeDUDwxOkiQ1J4NTP3jRi2DY\nMIOTJEnNxuDUDyK8l5MkSc3I4NRPxowxOEmS1GwMTv1k9GiYP7/RpZAkSfVkcOonHqqTJKn5GJz6\nicFJkqTmY3DqJwYnSZKaT6+CU0R8ISJGdNF//Yj4Qt+LtfYzOEmS1Hx62+J0MrBBF/1H1N5reaNH\nw8KFsHx5o0siSZLqpbfBKYDsov/OwBO9L07zaL97+BOuDUmSmsawngwcEU9SAlMC90VEx/A0lNIK\n9e36FW/ttemm5e/jj8NmmzW2LJIkqT56FJyAj1Fam75POSS3sMN7zwEPZuaMOpVtrTZ+fPl7//2w\nww6NLYskSaqPHgWnzPwBQEQ8APwxMz2Dpxtjx8KGG8J99zW6JJIkqV56e47TM8B27S8i4h0RcUVE\nfCUihtenaGu3CNhmG5gzp9ElkSRJ9dLb4HQ+sA1ARIwHLgUWAwcCp/dmghFxXEQ8EBFLIuLmiNh1\nNcOOi4hLImJORKyIiDO7Ge7AiJhdm+btEbFvb8rWWxMmGJwkSWomvQ1O2wC31f4/EPjfzDwYeD/w\nnp5OLCLeB5xBOW/qVcDtwPSIGNPNKOsCjwOndShH52nuDvwYuADYBfgFcEVEbN/T8vXWNtt4qE6S\npGbSl9sRtI+7J3BV7f+/A92FndWZApyfmRdn5r3AhygtWEd2NXBmPpSZUzLzR8DT3UzzBOA3mXlm\nZs7JzC8As4Dje1G+XpkwAebNgyefHKg5SpKk/tTb4PQX4HMR8Z/AG4Ff1/pvBTzWkwlFxDrAJOC6\n9n6ZmcC1wG69LB+1ca/t1G96H6fZIxMmlL+2OkmS1Bx6G5w+BkwEzgW+nJn31/ofANzUw2mNodwD\nqnPgegwY18vyURu33tPskVe8ovz1PCdJkppDT+/jBEBm3gHs2MVbnwBW9KlETWTkSNhiC4OTJEnN\nolfBqV1ETGLlbQnuycxZvZjMfErYGtup/1jg0T4U79HeTnPKlCmMGjVqlX6TJ09m8uTJPS7EhAke\nqpMkqT9MmzaNadOmrdJv4cKF3QxdH70KThGxGeUWBG8Enqr13igibgAOysx5VaeVmc9HxEzgLcAv\na9OP2utzelO+mhldTGOvWv/VOuuss5g4cWIfZr3SNtvAjTfWZVKSJKmDrho1Zs2axaRJk/ptnr09\nx+mblOfSvTIzN8nMTYAdgA3pXdg5EzgqIg6LiG0pz7sbAVwEEBFfjYgfdBwhInaOiF1q5di09nq7\nDoOcDbw1Ik6MiAkRcQrlJPRze1G+XpswAebOhba2gZyrJEnqD709VPdWYM/MnN3eIzPviYjjgKt7\nOrHMvKx2z6ZTKYfTbgP26dByNQ54aafRbqU8bBjKieoHAw8B42vTnBERBwNfrnVzgXdk5j09LV9f\nbLMNLF0Kf/87vOxlAzlnSZJUb70NTkOA57vo/zy9bMXKzKnA1G7eO6KLfmucT2b+DPhZb8pTL+23\nJJgzx+AkSdLarreH6q4Hzo6Izdt7RMRLgLPocD8mlbA0fLhX1kmS1Ax6G5yOp5zP9GBE/DUi/go8\nUOv3kXoVrhkMHQpbb+2VdZIkNYPe3sfp7xExkfK4lW1rvWdnZuc7dQsf9itJUrPoUYtTRLw5Iu6J\niA2zuCYzv5mZ3wT+HBF3R8Q+/VTWtZYP+5UkqTn09FDdx4ALMvMFD9bNzIXA+Xio7gUmTICHH4Yl\nSxpdEkmS1Bc9DU47A79dzftXAzv1vjjNacIEyCz3c5IkSWuvngansXR9G4J2y4FNe1+c5rTNNuWv\nh+skSVq79TQ4/YNyh/Du7AT8q/fFaU5jxsAmm3iCuCRJa7ueBqergNMiYr3Ob0TE+sAXgV/Vo2DN\nxivrJEla+/X0dgRfAt4N3BcR5wLtUWBb4DhgKOXxJupkm23g3nsbXQpJktQXPQpOmflYROwOfAv4\nKhDtbwHTgeMy87H6FrE5TJgAv/hFOUk8Ys3DS5KkwafHN8DMzIeA/SJiY2BrSniam5lP1rtwzWSb\nbeCpp2DePNhss0aXprkYRiVJA6W3j1whM5/MzD9n5i2GpjVrf9ivV9bVT1sbfOYz5XmA//hHo0sj\nSWoFvQ5O6pmtty6tIp4gXh+LF8N73wtf+1ppyTvxxEaXSJLUCgxOA2S99UrLSLO2OP3jH3DuufD8\n6u7yVSePPgp77AG/+Q1ccQWcdx5cdhlcfXX/z7snMuFb34J3vxuefbbRpZEGv0w49VT4yldKi/JA\naWuD//kf+PWvB3a+VT37LBx7LFx6aaNLIjA4DajBfkuChQvLF3P58p6Nd8cd8JrXwEc+Akce2b8b\nnrvuKvN65BH4wx9g//3h0EPhTW+C446DpUu7Hu+KK2C//cqjbwbCs8/CIYeUjd2vfgUf/GD5UZDU\ntRUr4Jhj4OST4XOfg/e8p/93ODLht7+FiRPLDs7b3w7bbQff/nZp1a63GTPgwQd7Ns4DD8Duu5ed\nsCOP9AkUg4HBaQD1NjgtWVK+NAceWL54/WHuXHjta+Ggg0rAeLLiWWvXXAOvfz2MHVtanC65pBw2\n6y4kzJsHV14Jzz1XvWzz58PPfw4nnFA2IBtvDH/6U9nYQTkEOnUqPPQQfP3rLxz/pz+FAw6AG26A\n3XaD22+vPu/emD0bXv3qspyXXlrWyaWXwje+0b/zldZWy5fD+98P3/seXHRRuQL52mvhda8r3+v+\ncMst8OY3w777woYbwk03lW6nncpO2EtfWgLc/Pl9n1dbG5xyStl+veEN1c/JvP56+Pd/LyHuT3+C\nF78YDj+85zu3qrPMtKt1wEQgZ86cmf3h3HMz11knc8mSzBUr1jz8ggWZp52WuemmmUOGZI4fnxmR\n+dGPZj77bPfjPftsZltb9XJdd13mxhtnbrNN5ve/v/L/OXNWP96FF2YOG5a5336ZzzxT+k2dmgml\n3J1ddllZFijT/+1vu55uW1vmzTeX5dxppzI8ZP7bv2Uef/zKeXV20kmZ666bOXfuyn4/+Unm0KGZ\nBx+c+cgjmZMmZb7oRWWZq1iyJPM73ynzXdP6yMz88Y8zR47MfOUrM2fPXtn/4x8v5fjd76rNt5Ge\nfz5z0aJGl0KtYtmyzPe8p2xLLr10Zf8778zcaquyzfjDH+o3v0WLMg8/vGxTdtgh88orX7i9/Nvf\nyvZngw1KGf76197P76mnMt/+9rLtPumkzJe+NHOXXTKffrr7cdraMs8+u2wz9tqr/BZkZv7xj+W3\n4Mtf7n156uHuuzOffLKxZVidmTNnJuU2SROzP7JCf0x0be36Ozhdf/3KENDeDRuWOWJE+XK+7nWZ\nBx5YvrAf+lD5AV5vvcxjj828//7M5cszzzgjc/31y/DXXlum29aWOXNm5uc/n7njjmW6I0dmbr99\n5lvfmnnMMZmnn575+9+XINDR1Knly7n33iu/CPfdlzlhQuZGG2VeffULl2PRosyTTy7zOfro8kPb\n0WmnlfemTi2vH320bBgh893vzrzhhsw3vam8fsc7Vm6Unn4681vfKhsVyNxii8wjjsi8+OLMhx5a\n8/pdtCjz5S8vy9LWVkLMkCGZhx5a1l1mCV377FMC7I9/3P20FizI/NKXMseOLRu8MWNKXR1/fObj\nj6867PPPZ/7855l77lnKfcghLwy2zz+fuccemZttVgJcPT3wQOYll2T+6Efl7yWXlGW7+uryo1TV\n3/9e6vUlLykB9GMfK3XXKP/8Z/khveSSF35uO7rllrLud9wx81e/Wv0029rK+v/lL8uy/sd/lOUd\nPjxz881LUN9jj/I9/MpXug/p3fnHPzJPPDHzne8s419/fc+nsSaLFmWec07mWWdlLl5c32n3p8ce\nK9upuXPL/0uWlO5tbyvr/xe/eOE48+ZlvvGN5ft69tkrv8e99cADZfsyYkTmBReseXoPPZT5ildk\nvvjFJSz01F13lfE32ijzqqtKvzvuyNxww8x9933htjOzLPMhh5RtyX/91wuHOemksi2aNavn5emr\nFSvK53rIkMwtt8z885/7Ps0bbyzfu6OOyly4sO/TyzQ4NVVwWr488/LLMy+6KPN738s8//wSLs46\nK/NTn8o87LDyA7D99pkve1kJQo899sLpzJ1bNiZQWnu23LL8v9FGmf/5n6XV6IwzMk84oQSTXXYp\ne05QNlCve12Z3wc+UPqdcMILv5xPPlkCxtChpRxf+ELmu95VNgIRZbyvfrXrlq22thL+IsqPyOjR\nJXhceunK4dvaSmvQFluUH+l3vauUcciQUubf/KZaq1xnV15ZynbEEWVahx32wo3jc8+t3OP85CfL\nBvQ73ynd+eeX9TFyZCnXMceUlqbFizO/9rWywdtww7Lx+OtfM7/4xfLDC5m7777qMnb22GNleXfb\nrVqgWb68+x/d554rYW2ffVbWR1fdxhtnHnlk5vTpL6zjtrYSVq+6KnP//cv6GjmybMA+97mynCNG\nlM/K/PlrLm932toyn3gi81//ynzwwbI+77ijbHRnzCiB/rrrSgvkd79b6mb8+FWXY8yYzM9+tgST\ndvfdVwIOlBa+PfYo/++/f2kx6Gjhwszzzlu5YwHlc7n33uWH6JxzMk85pQTjgw7KfMtbSv2PHVvC\n/HPPrX4ZH3qo7OCsu275Hu6xR2nZhLJed965fK5uuKHrH8sqnnkm8+tfL+F76NDy4/myl63+M9ed\nefPKej///PJd3XvvzF13zXztazNf//qyfXnzm8tn4Zprui/zihWlLu+8s/sy3HVX+T4OH/7Cz+eQ\nIWVHcPr07su6bFmpF8icODHzppt6tqztbrihfI622irz9turj/foo+VzM3p05l/+Um2cJUvKNn7k\nyDJux1bwzLJOhw0rO8jt623FivL5Hz06c9SosiPUlWXLyudphx1Wv0NRb088UVrOoLSgv/rVpU4v\nuKB301uyJPMTnyjbr1e9qmz/t9xyZYNAXxicmig41dOKFZnf/nbZkBx3XPmwrW7jvnx55q23lh+I\n97637EENH142nN15/vnMKVPKp2Ts2BLqPvax8uW+4441l+/QQ8u4733vC1tp2j37bOZnPlP2OE4+\nubR69NU731nm+/73d79H2dZWwsHQoSs34hErW5e6C63z5mV+5CNlo9fesnf00WXdVnHzzWW9v/vd\npRXwjDNKcD777BLMjjmmNM3/27+tnMdmm5UfsyOOKGH1M58p9QeZr3lN2UA/+WQJd4sWlXX67LPl\nx+Gzny3Tag8fb397CXjjx5dQ1L7su+xSAkLHPb4FC8q8Ro4sIeCoo8qG7nOfK62Kp5+e+c1vlkO2\nl19egs+NN5bP4tlnZ37wg+WHuD1AVOmGDCmf6RNOKId2//nPEpA+8pGyYR02LHPy5LKehg0rQfTC\nC0s9t7WVcV7yktJSe+qpZX0fdVRZhqFDy2fjpz8tQWdNYePBB8uOSEQ5tPzzn5dxnnuuBLhbby3L\nfNRRpUVkk01KK+VTT5Xxly8vgeKCC0p43WKLlYHtiCNKyF+woPsQvXRp+dG+994y3dGjy3yOProE\nwzlzSkiE8vn4y19K+ebMKfM87LDSAjtsWFn2oUPL+u0YtIcMKTtD++9f6uuII8p4hxxSvrftAXbT\nTcuP/A03ZN52W+Y3vlF2djbZZOW0xo4t4114YebDD5fWtv32K++95CXl83LLLaX/FVeUluRzzy3T\nq2LGjPLZaN8xav9+LllSDmH993+X79UBB5Qdm+nTS+BvayvbvaFDSyDuzU7AggXlu7bhhiVwdqWt\nrdTBsceW8AzlFIHuTqv43vfKMKefXranr3tdeX3ooWtu6b399rId+cQnXvje00+X0wSuvTbzBz8o\n6+L440tAPuWU8t384Q9L6+xtt1Vr5Zk1qwTOjTde2aq7dGn5TED5fLe3fra1Zd5zT5nPu95Vvh8/\n/OGqRw1mziw7O8OHl+3e8uWlNbB95+fYY3veUrtiRZnGr3+d+dGP9m9wikwv9WkXEROBmTNnzmRi\n+5nHTSqz3Dpg+PA1D7t0abmdQk8tX15OlN5xx56P2xfz5pXLig87DIas4fKH7OVdx+fOLSeXvv3t\nMGpUz8b9wQ/gU58qJ8ivWFFOHG1rg3XWga22gvHjV/590Yvgr38t85s7t9zOoq2tXEl49NGwyy5r\nnl8mzJpVTlC/885yIv+4cSu7bbeFnXfufj3Mm1dOur/66vJZWLoUli0r3ZIlXZ/ov+66sP32sMMO\npf632qp8htZdd2U3fHhZ5o7dxhvDBht0XY6FC+HCC+Gb3ywXL3zmM+Uk3vXXX3W4Z5+F006DM88s\nn8EttoCjjoIPfABe8pI1r6/Obrut1NfVV5f6eOaZVd/fbDP4+Mfhwx/uvuxQ6uEvfymXvf/856te\nKDJsGIwcCSNGlOGeemrVK0TXXbdcmfmpT5WTlju65hqYMgXuuQfGjCn1NWRI+Wy84Q3lqQURq3Yb\nb1yuHnvFK8q0V1fmmTPL7T4uu2zlidrDh5cLLd74xtK1tZWTua+9tnzW2n9WdtyxrJuDDqq2rVmT\nFSvgggtK3WeWZZg5s3wGR4woF2W0f97b62ncuHILkylT4PTTy7rujWeegXe8A26+uXz31lln5Xtt\nbaUe7rwTNt+8bHuOOKKs+9X5/OfhS18qZdp663KRyx57VCvP6afDpz9d5vP44+WK4YcfLp+djjbe\nuHzuM+GJJ0q3bNmqw4wZU7Y348fDJpus/K6ut175jp91FrzylXD55eW73NFFF5XP/vbblxPrr7mm\nnPw+fHi54OiJJ8rV0FBuybPLLmX7vMMOcPHFq/4+tLWVdfCpT5V6e+c7S72OHLmyW7q0bAueemrl\n3/vvL7817VdCrrfeLJYunQQwKTNnVVuj1RmcOmil4KS1U/v+/ZoC4UB67jlYtKgElmeeKT8o48fD\n0KH9M7/2212saR3MnVsu/d5jj97/WHZ03XXw5z+XoDR27Mq/L37xqj+iVc2eDXffXTb2ixat/Auw\n0Uardttuu/pHNS1fXn7AHnywXOW6++7lSrF6yizLv2RJCSidA2u7BQvgd78r5X7zm/vncUjz55f7\nPT3+eLnybvfdy492ez20tZUf05kz4dZbS3kPOKDv812ypNwy4U9/WrV/Ztn5OOII2Hvv6p+3TPjs\nZ8vO15QpPQuXK1aUK+zuuquE6S23LH/bu803L4Gpq3pasqQEmkcegb/9bdWuPbR33EF63/vKjkh3\nO9C33VbCYgTsuSfstVcJ7SNHlvcXLIAbb4Tf/74Ezz33LMvd3fLef3+5Onvu3PKdaO+WLSvrdtSo\nVbvx40vMv2pvAAAPT0lEQVRwa+/mzZvFrrsanAaEwUmSpMFpxYqyw7SmMD5r1iwmTeq/4FSH/TBJ\nkqT+1V+t2D01iBr8JUmSBjeDkyRJUkUGJ0mSpIoMTpIkSRUZnCRJkioyOEmSJFVkcJIkSarI4CRJ\nklSRwUmSJKkig5MkSVJFBidJkqSKDE6SJEkVGZwkSZIqMjhJkiRVZHCSJEmqyOAkSZJUkcFJkiSp\nIoOTJElSRQYnSZKkigxOkiRJFRmcJEmSKjI4SZIkVWRwkiRJqsjgJEmSVJHBSZIkqSKDkyRJUkUG\nJ0mSpIoMTpIkSRUZnCRJkioyOEmSJFVkcJIkSarI4CRJklSRwUmSJKkig5MkSVJFBidJkqSKDE6S\nJEkVGZwkSZIqMjhJkiRVZHCSJEmqyOAkSZJUkcFJkiSpIoOTJElSRQYnSZKkigxOkiRJFRmcJEmS\nKho0wSkijouIByJiSUTcHBG7rmH4N0XEzIhYGhH3RcThnd4/PCLaImJF7W9bRCzu36WQJEnNbFAE\np4h4H3AGcDLwKuB2YHpEjOlm+JcDvwKuA3YGzga+GxF7dRp0ITCuQ/eyfii+JElqEYMiOAFTgPMz\n8+LMvBf4ELAYOLKb4T8M/C0zP5mZczLzPODy2nQ6ysycl5mP17p5/bYEkiSp6TU8OEXEOsAkSusR\nUNIOcC2wWzejvbb2fkfTuxh+g4h4MCIejogrImL7OhVbkiS1oIYHJ2AMMBR4rFP/xyiH17oyrpvh\nN4yIdWuv51BarPYHDqEs600RsXk9Ci1JklrPsEYXoL9k5s3Aze2vI2IGMBs4hnIulSRJUo8MhuA0\nH1gBjO3UfyzwaDfjPNrN8E9n5rKuRsjM5RFxK7D1mgo0ZcoURo0atUq/yZMnM3ny5DWNKkmSBsi0\nadOYNm3aKv0WLlzYr/OMcjpRY0XEzcCfMvOjtdcBPAyck5n/3cXwXwP2zcydO/T7MbBRZu7XzTyG\nAHcDv87Mj3czzERg5syZM5k4cWJfF0uSJA2wWbNmMWnSJIBJmTmr3tMfDOc4AZwJHBURh0XEtsC3\ngRHARQAR8dWI+EGH4b8NjI+Ir0fEhIg4FjigNh1q43w+IvaKiK0i4lXAJcCWwHcHZpEkSVKzGQyH\n6sjMy2r3bDqVcsjtNmCfDrcPGAe8tMPwD0bE24CzgBOAR4APZGbHK+02Br5TG/dJYCawW+12B5Ik\nST02KIITQGZOBaZ2894RXfT7PeU2Bt1N70TgxLoVUJIktbzBcqhOkiRp0DM4SZIkVWRwkiRJqsjg\nJEmSVJHBSZIkqSKDkyRJUkUGJ0mSpIoMTpIkSRUZnCRJkioyOEmSJFVkcJIkSarI4CRJklSRwUmS\nJKkig5MkSVJFBidJkqSKDE6SJEkVGZwkSZIqMjhJkiRVZHCSJEmqyOAkSZJUkcFJkiSpIoOTJElS\nRQYnSZKkigxOkiRJFRmcJEmSKjI4SZIkVWRwkiRJqsjgJEmSVJHBSZIkqSKDkyRJUkUGJ0mSpIoM\nTpIkSRUZnCRJkioyOEmSJFVkcJIkSarI4CRJklSRwUmSJKkig5MkSVJFBidJkqSKDE6SJEkVGZwk\nSZIqMjhJkiRVZHCSJEmqyOAkSZJUkcFJkiSpIoOTJElSRQYnSZKkigxOkiRJFRmcJEmSKjI4SZIk\nVWRwkiRJqsjgJEmSVJHBSZIkqSKDkyRJUkUGJ0mSpIoMTpIkSRUZnCRJkioyOEmSJFVkcJIkSarI\n4CRJklSRwUmSJKkig5MkSVJFBidJkqSKDE6SJEkVGZwkSZIqMjhJkiRVZHCSJEmqyOAkSZJUkcFJ\nkiSpIoOTJElSRYMmOEXEcRHxQEQsiYibI2LXNQz/poiYGRFLI+K+iDi8i2EOjIjZtWneHhH79t8S\nqN6mTZvW6CKoA+tjcLE+Bhfro3UMiuAUEe8DzgBOBl4F3A5Mj4gx3Qz/cuBXwHXAzsDZwHcjYq8O\nw+wO/Bi4ANgF+AVwRURs328LorpyQzS4WB+Di/UxuFgfrWNQBCdgCnB+Zl6cmfcCHwIWA0d2M/yH\ngb9l5iczc05mngdcXptOuxOA32TmmbVhvgDMAo7vv8WQJEnNrOHBKSLWASZRWo8AyMwErgV262a0\n19be72h6p+F3qzCMJElSZQ0PTsAYYCjwWKf+jwHjuhlnXDfDbxgR665hmO6mKUmStFrDGl2AQWY9\ngNmzZze6HAIWLlzIrFmzGl0M1Vgfg4v1MbhYH4NHh9/w9fpj+oMhOM0HVgBjO/UfCzzazTiPdjP8\n05m5bA3DdDdNgJcDHHrooasvsQbMpEmTGl0EdWB9DC7Wx+BifQw6LwduqvdEGx6cMvP5iJgJvAX4\nJUBERO31Od2MNgPofGuBvWv9Ow7TeRp7dRqms+nAIcCDwNJqSyBJkgaR9SihaXp/TDzKediNFRHv\nBS6iXE13C+XquAOAbTNzXkR8Fdg8Mw+vDf9y4E5gKvB9SkD6BrBfZl5bG2Y34HfAScCvgcnAp4GJ\nmXnPAC2aJElqIg1vcQLIzMtq92w6lXI47TZgn8ycVxtkHPDSDsM/GBFvA86i3HbgEeAD7aGpNsyM\niDgY+HKtmwu8w9AkSZJ6a1C0OEmSJK0NBsPtCCRJktYKBqeanj4rT30XESdFxC0R8XREPBYR/xMR\n23Qx3KkR8c+IWBwR10TE1o0ob6uJiE9HRFtEnNmpv/UxQCJi84j4YUTMr63v2yNiYqdhrI8BEBFD\nIuK0iPhbbV3fHxGf62I466MfRMQbIuKXEfGP2nZp/y6GWe26j4h1I+K82vfpmYi4PCI262lZDE70\n/Fl5qps3AN8EXgPsCawDXB0R67cPEBGfojwm52jg1cAiSt0MH/jito7ajsPRlO9Cx/7WxwCJiI2A\nPwLLgH2A7YD/Ap7sMIz1MXA+DRwDHAtsC3wS+GRE/N9jvKyPfjWScv7zscALzjGquO6/AbwNeA/w\n/4DNgZ/1uCSZ2fIdcDNwdofXQTnh/JONLlsrdZS7yLcBr+/Q75/AlA6vNwSWAO9tdHmbtQM2AOYA\nbwZuAM60PhpSD18D/ncNw1gfA1cfVwIXdOp3OXCx9THgddEG7N+p32rXfe31MuBdHYaZUJvWq3sy\n/5Zvcerls/LUPzai7Ek8ARARW1GuqOxYN08Df8K66U/nAVdm5vUde1ofA+4/gL9ExGW1Q9mzIuKD\n7W9aHwPuJuAtEfEKgIjYGXgdcFXttfXRIBXX/b9T7iTQcZg5wMP0sH4Gxe0IGmx1z8qbMPDFaU21\nm55+A7gxV94yYhwlSPnMwQESEQcBu1A2Mp1ZHwNrPPBhymkEX6YcfjgnIpZl5g+xPgba1yitFvdG\nxArKqS6fzcyf1N63PhqnyrofCzxXC1TdDVOJwUmDxVRge8oenBogIraghNc9M/P5RpdHDAFuyczP\n117fHhE7UG4U/MPGFatlvQ84GDgIuIeyg3F2RPyzFmTVIlr+UB29e1ae6igizgX2A96Umf/q8Naj\nlPPNrJuBMQnYFJgVEc9HxPPAG4GPRsRzlD0z62Pg/Avo/MTx2cCWtf/9fgys04GvZeZPM/PuzLyE\nchPmk2rvWx+NU2XdPwoMj4gNVzNMJS0fnGp71u3PygNWeVZe3R8OqFXVQtM7gD0y8+GO72XmA5QP\ndMe62ZByFZ51U3/XAjtS9qR3rnV/AX4E7JyZf8P6GEh/5IWnC0wAHgK/Hw0wgrKT3VEbtd9R66Nx\nKq77mcDyTsNMoOyIrO4Zti/gobriTOCi2sOG25+VN4Ly/Dz1k4iYSnmG4P7Aooho31tYmJntD1n+\nBvC5iLif8vDl0yhXPP5igIvb9DJzEeUQxP+JiEXAgsxsb/mwPgbOWcAfI+Ik4DLKj8AHgaM6DGN9\nDJwrKev6EeBuYCLlt+K7HYaxPvpJRIwEtqa0LAGMr52g/0Rm/p01rPvMfDoivgecGRFPAs8A5wB/\nzMxbelSYRl9WOFg6yr0hHqRcvjgD+PdGl6nZO8re2oouusM6DXcK5VLTxZSnXW/d6LK3SgdcT4fb\nEVgfA77+9wPuqK3ru4EjuxjG+hiYuhhJ2cl+gHKPoLnAF4Fh1seArP83dvOb8f2q6x5Yl3LvwPm1\n4PRTYLOelsVn1UmSJFXU8uc4SZIkVWVwkiRJqsjgJEmSVJHBSZIkqSKDkyRJUkUGJ0mSpIoMTpIk\nSRUZnCRJkioyOEmSJFVkcJKkLkREW0Ts3+hySBpcDE6SBp2IuLAWXFbU/rb/f1WjyyaptQ1rdAEk\nqRu/Ad7PyqehAyxrTFEkqbDFSdJgtSwz52Xm4x26hfB/h9E+FBFXRcTiiPhrRLyn48gRsUNEXFd7\nf35EnB8RIzsNc2RE3BURSyPiHxFxTqcybBoRP4+IRRFxX0T8R4dxN4qISyLi8do85kTE4f22NiQN\nCgYnSWurU4GfAjsBlwA/iYgJABExApgOLAAmAQcAewLfbB85Ij4MnAt8G3gl8Dbgvk7z+ALwE2BH\n4CrgkojYqPbel4BtgX1qfz8MzK/3QkoaXCIzG10GSVpFRFwIHAos7dA7ga9k5tciog2YmpnHdxhn\nBjAzM4+PiKOArwJbZObS2vv7AlcCL87MeRHxCPC9zDy5mzK0Aadm5im11yOAZ4G3ZubVEfELYF5m\nfrC+Sy9pMPMcJ0mD1fXAh1j1HKcnOvx/c6fhZwA71/7fFri9PTTV/JHSyj4hIgA2r81jde5s/ycz\nF0fE08BmtV7fAn4WEZOAq4ErMnPGmhZK0trN4CRpsFqUmQ/007SXVBzu+U6vk9opDpn524jYEtgP\n2Au4NiLOy8xP1q+YkgYbz3GStLZ6bRevZ9f+nw3sHBHrd3j/9cAK4N7MfBZ4EHhLXwqQmQsy84eZ\neRgwBTi6L9OTNPjZ4iRpsFo3IsZ26rc8MxfU/j8wImYCN1LOh9oVOLL23iXAKcAPIuKLlMNr5wAX\nZ2b7CdynAN+KiHmUWx9sCOyemedWKVxtujOBu4H1gLcD9/R0ISWtXQxOkgartwL/7NRvDrB97f+T\ngYOA84B/AQdl5r0AmbkkIvYBzgZuARYDlwP/1T6hzLw4ItaltBT9N+WKuMs7zKurK2eyQ//ngK8A\nL6cc+vsDMLkXyylpLeJVdZLWOrUr3t6Zmb9sdFkktRbPcZIkSarI4CRpbWRTuaSG8FCdJElSRbY4\nSZIkVWRwkiRJqsjgJEmSVJHBSZIkqSKDkyRJUkUGJ0mSpIoMTpIkSRUZnCRJkioyOEmSJFX0/wHZ\n6qOUbgP8qgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10cb9ee48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1, sgd_lr.epochs+1), sgd_lr.cost_)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Cost')\n",
    "plt.ylim([0, 0.2])\n",
    "plt.tight_layout()\n",
    "plt.show()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## LinearRegression\n",
      "\n",
      "*LinearRegression(eta=0.01, epochs=50, minibatches=None, random_seed=None, print_progress=0)*\n",
      "\n",
      "Ordinary least squares linear regression.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `eta` : float (default: 0.01)\n",
      "\n",
      "    solver rate (between 0.0 and 1.0)\n",
      "\n",
      "- `epochs` : int (default: 50)\n",
      "\n",
      "    Passes over the training dataset.\n",
      "    Prior to each epoch, the dataset is shuffled\n",
      "    if `minibatches > 1` to prevent cycles in stochastic gradient descent.\n",
      "\n",
      "- `minibatches` : int (default: None)\n",
      "\n",
      "    The number of minibatches for gradient-based optimization.\n",
      "    If None: Normal Equations (closed-form solution)\n",
      "    If 1: Gradient Descent learning\n",
      "    If len(y): Stochastic Gradient Descent learning\n",
      "    If 1 < minibatches < len(y): Minibatch learning\n",
      "\n",
      "- `random_seed` : int (default: None)\n",
      "\n",
      "    Set random state for shuffling and initializing the weights.\n",
      "\n",
      "- `print_progress` : int (default: 0)\n",
      "\n",
      "    Prints progress in fitting to stderr if not solver='normal equation'\n",
      "    0: No output\n",
      "    1: Epochs elapsed and cost\n",
      "    2: 1 plus time elapsed\n",
      "    3: 2 plus estimated time until completion\n",
      "\n",
      "**Attributes**\n",
      "\n",
      "- `w_` : 2d-array, shape={n_features, 1}\n",
      "\n",
      "    Model weights after fitting.\n",
      "\n",
      "- `b_` : 1d-array, shape={1,}\n",
      "\n",
      "    Bias unit after fitting.\n",
      "\n",
      "- `cost_` : list\n",
      "\n",
      "    Sum of squared errors after each epoch;\n",
      "    ignored if solver='normal equation'\n",
      "\n",
      "### Methods\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit(X, y, init_params=True)*\n",
      "\n",
      "Learn model from training data.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "- `y` : array-like, shape = [n_samples]\n",
      "\n",
      "    Target values.\n",
      "\n",
      "- `init_params` : bool (default: True)\n",
      "\n",
      "    Re-initializes model parameters prior to fitting.\n",
      "    Set False to continue training with weights from\n",
      "    a previous model fitting.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `self` : object\n",
      "\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict(X)*\n",
      "\n",
      "Predict targets from X.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `target_values` : array-like, shape = [n_samples]\n",
      "\n",
      "    Predicted target values.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.regressor/LinearRegression.md', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}