{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LinearRegression: An implementation of ordinary least-squares linear regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A implementation of Ordinary Least Squares simple and multiple linear regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.regressor import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Illustration of a simple linear regression model:\n",
    "    \n",
    "![](./LinearRegression_files/simple_regression.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Ordinary Least Squares (OLS) Linear Regression, our goal is to find the line (or hyperplane) that minimizes the vertical offsets. Or in other words, we define the best-fitting line as the line that minimizes the sum of squared errors (SSE) or mean squared error (MSE) between our target variable (y) and our predicted output over all samples $i$ in our dataset of size $n$.\n",
    "\n",
    "$$SSE =  \\sum_i \\big(\\text{target}^{(i)} - \\text{output}^{(i)}\\big)^2$$\n",
    "\n",
    "$$MSE = \\frac{1}{n} \\times SSE$$\n",
    "\n",
    "\n",
    "Now, `LinearRegression` implements a linear regression model for performing ordinary least squares regression using one of the following five approaches:\n",
    "\n",
    "- Normal Equations\n",
    "- QR Decomposition Method\n",
    "- SVD (Singular Value Decomposition) method\n",
    "- Gradient Descent\n",
    "- Stochastic Gradient Descent\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normal Equations (closed-form solution)\n",
    "\n",
    "The closed-form solution should be preferred for \"smaller\" datasets where calculating (a \"costly\") matrix inverse is not a concern. For very large datasets, or datasets where the inverse of $[X^T X]$ may not exist (the matrix is non-invertible or singular, e.g., in case of perfect multicollinearity), the QR, SVD or gradient descent approaches are to be preferred.\n",
    "\n",
    "The linear function (linear regression model) is defined as:\n",
    "\n",
    "$$y = w_0x_0 + w_1x_1 + ... + w_mx_m = \\sum_{i=0}^{m} = \\mathbf{w}^T\\mathbf{x}$$\n",
    "\n",
    "where $y$ is the response variable, $\\mathbf{x}$ is an $m$-dimensional sample vector, and $\\mathbf{w}$ is the weight vector (vector of coefficients). Note that $w_0$ represents the y-axis intercept of the model and therefore $x_0=1$.  \n",
    "\n",
    "Using the closed-form solution (normal equation), we compute the weights of the model as follows:\n",
    "\n",
    "$$ \\mathbf{w} = (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^Ty$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stable OLS via QR Factorization\n",
    "\n",
    "The QR decomposition method offers a more numerically stable alternative to the closed-form, analytical solution based on the \"normal equations,\" and it can be used to compute the inverse of large matrices more efficiently.\n",
    "\n",
    "QR decomposition method decomposes given matrix into two matrices for which an inverse can be easily obtained. For instance, a given matrix $X \\in \\mathbb{R}^{n \\times m}$, the QR decomposition into two matrices is:\n",
    "\n",
    "$$\\mathbf{X = QR},$$\n",
    "\n",
    "where\n",
    "\n",
    "$Q \\in \\mathbb{R}^{n \\times m}$ is an orthonormal matrix, such that $Q^\\top Q = QQ^\\top = I$. \n",
    "The second matrix $R \\in \\mathbf{R}^{m \\times m}$ is an upper triangular matrix.\n",
    "\n",
    "The weight parameters of the ordinary least squares regression model can then be computed as follows [1]:\n",
    "\n",
    "$$\\mathbf{w} = \\mathbf{R}^{-1}\\mathbf{Q}^\\top y.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stable OLS via Singular Value Decomposition\n",
    "\n",
    "Another alternative way for obtaining the OLS model weights in a numerically stable fashion is by Singular Value Decomposition (SVD), which is defined as:\n",
    "\n",
    "$$\\mathbf{X}=\\mathbf{U}\\Sigma \\mathbf{V}^\\top,$$\n",
    "\n",
    "for a given matrix $\\mathbf{X}$.\n",
    "\n",
    "Then, it can be shown that the pseudo-inverse of $X$, $X^+$, can be obtained as follows [1]:\n",
    "\n",
    "$$ X^+ = \\mathbf{U} \\mathbf{\\Sigma}^+  V^{\\top}.$$\n",
    "\n",
    "Note that while $\\Sigma$ is the diagonal matrix consisting of singular values of $\\mathbf{X}$, $\\Sigma^{+}$ is the diagonal matrix consisting of the reciprocals of the singular values.\n",
    "\n",
    "The model weights can then be computed as follows:\n",
    "\n",
    "$$\\mathbf{w} = \\mathbf{X}^+  \\mathbf{y}.$$\n",
    "\n",
    "Please note that this OLS method is computationally most inefficient. However, it is a useful approach when the direct method (normal equations) or QR factorization cannot be applied or the normal equations (via $\\mathbf{X}^T \\mathbf{X}$) are ill-conditioned [3].\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Descent (GD)  and Stochastic Gradient Descent (SGD) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See [Gradient Descent and Stochastic Gradient Descent](https://sebastianraschka.com/faq/docs/gradient-optimization.html) and [Deriving the Gradient Descent Rule for Linear Regression and Adaline](https://sebastianraschka.com/faq/docs/linear-gradient-derivative.html) for details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Random shuffling is implemented as:\n",
    "\n",
    "- for one or more epochs\n",
    "    - randomly shuffle samples in the training set\n",
    "        - for training sample *i*\n",
    "            - compute gradients and perform weight updates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [1] Chapter 3, page 55, Linear Methods for Regression. Trevor Hastie; Robert Tibshirani; Jerome Friedman (2009). [The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd ed.)](https://statweb.stanford.edu/~tibs/ElemStatLearn/download.html). New York: Springer. (ISBN 978–0–387–84858–7)\n",
    "- [2] G. Strang, Linear Algebra and Its Applications, 2nd Ed., Orlando, FL, Academic Press, Inc., 1980, pp. 139-142.\n",
    "- [3] Douglas Wilhelm Harder. Numerical Analysis for Engineering. Section 4.8, [ill-conditioned Matrices](https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/illconditioned/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Closed Form Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.25\n",
      "Slope: 0.81\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAG11JREFUeJzt3WmUlOWZxvH/DdFhWjFgQKNR6XEyiYAIYgeDKCgo7vskxrRLEhVFNGqMDgZHYzwYVERQ2VpAUZugsskadtkFu9k3F5BGRaTVKEoDsjzz4W4nQVmqod56a7l+53Cq6+3qqrs+eJ3b530WCyEgIiKZo1rcBYiISNUouEVEMoyCW0Qkwyi4RUQyjIJbRCTDKLhFRDKMgltEJMMouEVEMoyCW0Qkw3wvijetU6dOyM/Pj+KtRUSyUmlp6SchhLqJvDaS4M7Pz6ekpCSKtxYRyUpmVpboazVUIiKSYRTcIiIZRsEtIpJhFNwiIhlGwS0ikmEU3CIiGUbBLSKSYRTcIiIHauNGuPtuWL8+JR+n4BYRORCjRkGDBvDkkzB+fEo+UsEtIrI/1q+Hq66CSy6B2rVhzhy4/vqUfLSCW0SkKkKAAQOgfn0YMQIefhhKS+HUU1NWQkLBbWZrzGyJmS00M21CIiK56d134eyz4YYboFEjWLQI7r+f4lcPJj8fqlWD/HwoLo62jKpsMnVWCOGTyCoREUlX27ZBt27w5z/DwQdDnz5w001QrRrFxdCuHVRU+EvLyvw5QGFhNOVoqEREZG9KS6FZM+jYEc4/H1asgJtv9vYa6NTpn6H9jYoKvx6VRIM7ABPMrNTM2u3uBWbWzsxKzKykvLw8eRWKiMRh0yb44x89tD/+GIYOhWHD4Oijd3nZ2rW7//M9XU+GRIO7RQihKXA+0MHMWn77BSGEohBCQQihoG7dhPYCFxFJTxMn+hj2E0/AjTfC8uVwxRW7felxx+3+LfZ0PRkSCu4QwrrKxw3AcKBZdCWJiMTk0099Sl/btnDQQTBtGvTtC7Vq7fFPOneGvLxdr+Xl+fWo7DO4zewQM6v5zc9AW2BpdCWJiKRYCDBokE/xGzTIB6gXLYKW3xlc+I7CQigqgnr1wMwfi4qiuzEJic0qORIYbmbfvH5QCOHv0ZUkIpJCZWXQvj2MG+fj2ZMn+zBJFRQWRhvU37bP4A4hrAYap6AWEZHU2bEDnnnmn9M/uneH226D6tXjrSsBkRwWLCKS1pYs8ZuO8+b5FL/evX2MI0NoHreI5I4tW+D++6FpU1i92pc4jhmTUaEN6rhFJFdMn+6rHd9+G667zqf61akTd1X7RR23iGS3zz/3lY6tWvnS9fHjYeDAjA1tUHCLSDYbPtz3yu7Xzw86WLLE52hnOA2ViEj2WbfOZ4gMHw5NmsDIkVBQEHdVSaOOW0Syx86dvvqlfn2fl92li88cyaLQBnXcIpIt3nrL91OdPh3OOssD/Mc/jruqSKjjFpHM9vXXvjFI48aweDH07++rH7M0tEEdt4hksrlzfSHN0qXwy19Cjx7wwx/GXVXk1HGLSOb56iu4805o3tyn+40cCS+/nBOhDeq4RSTTjBsHt9wC778Pt94KjzwChx0Wd1UppY5bRDLDhg2+Bd8FF8Ahh8DMmb5JVI6FNii4RSTdhQAvvOBT/F591Q/sXbAATjst7spio6ESEUlf773ny9UnTvSgfvZZXwmZ49Rxi0j62b7dN4E68UR44w3o2RNmzFBoV1LHLSLpZeFCn+JXWgoXXwy9esExx8RdVVpRxy0i6WHzZujY0Zenf/ABvPIKvPaaQns31HGLSPymTPHl6qtWwQ03wOOPQ+3acVeVttRxi0h8PvvMg7pNG38+ebJvwarQ3isFt4ikXgg+FFK/vh9q0LGj75XdunXclWUEDZWISGp98IGveBw1Ck45xU+kadIk7qoyijpuEUmNnTt9Wl+DBjBpkk/3e+MNhfZ+UMctItFbvtwP6p09G845B/r0geOPj7uqjKWOW0Sis3WrL1Fv0gRWrvSl6+PHK7QPkDpuEYnG7Nm+kGbFCt8c6sknoW7duKvKCuq4RSS5Nm6EDh3g9NNh0yYYOxZeekmhnUQKbhFJnpEj/eZj795wxx2wbBmcf37cVWUdBbeIHLj16/3osEsvhcMPhzlzfGjk0EPjriwrKbhFZP+FAAMG+EKakSP90N7SUjj11Lgry2q6OSki++fdd31/kalToWVLKCqCn/407qpygjpuEamabdugSxdo1Ajmz/fAnjpVoZ1C6rhFJHElJT7Fb9EiuOIKePppOProuKvKOeq4RWTfNm2Cu+/2sesNG2DYMBg6VKEdE3XcIrJ3Eyb4uY9r1vhjly5Qq1bcVeU0ddwisnuffgrXXw/nngv/9m8wfbrvMaLQjp2CW0R2FQIMGgQnnOCP99/v50CecUbclUklDZWIyD+VlUH79jBunI9nP/uszx6RtJJwx21m1c1sgZmNjrIgEYnBjh3Qowc0bOhDIj16wKxZCu00VZWhkjuAFVEVIiJQXAz5+VCtmj8WF6fgQ5csgdNOgzvvhFatfO/s3/8eqldPwYfL/kgouM3sGOBCoF+05YjkruJiX4hYVubDzGVl/jyy8N6yxcevmzaF997z8ezRo+G44yL6QEmWRDvu7sC9wM4IaxHJaZ06QUXFrtcqKvx60k2fDo0b+94ihYW+Z/bVV4NZBB8mybbP4Dazi4ANIYTSfbyunZmVmFlJeXl50goUyRVr11bt+n75/HOfi92qlS9dnzABnn8efvCDJH6IRC2RjrsFcImZrQEGA63N7KVvvyiEUBRCKAghFNTVhukiVbanEYqkjVwMG+Z7ZffrB/fcA0uX+vmPknH2GdwhhPtCCMeEEPKBXwFTQgjXRF6ZSI7p3Bny8na9lpfn1w/IunW+r8iVV8KRR8K8efDYY9/9MMkYWoAjkiYKC32jvXr1fKi5Xj1/Xli4n2+4cyf07et7ZY8bB48+6qF9yilJrVtSz0IISX/TgoKCUFJSkvT3FZEEvfUW3HQTzJgBrVt7gP/4x3FXJXthZqUhhIJEXquOWySbfP21j62cdJKPYQ8YAJMmKbSzjJa8i2SLuXN9r+ylS+Gqq3z145FHxl2VREAdt0im++orP1G9eXOf7jdqFAwerNDOYuq4RTLZ2LG+KdT770OHDvDII1CzZtxVScTUcYtkog0b4Ne/hgsvhEMP9Q2hnn5aoZ0jFNwimSQEGDjQp/gNHQoPPQQLFvgwieQMDZWIZIrVq325+qRJ0KKF75Vdv37cVUkM1HGLpLvt2+GJJ+DEE33mSK9evkmUQjtnqeMWSWcLFvhCmtJSuPRSeOYZOOaYuKuSmKnjFklHmzdDx47ws5/Bhx/CkCEwfLhCWwB13CLpZ8oUP0Fh1SpfUPPYY1C7dtxVSRpRxy2SLj77DG64Adq08V2mpkzxG5AKbfkWBbdI3EKAV17xm40DB8J998HixXDWWXFXJmlKQyUicfpmxeOoUVBQ4CfSNG4cd1WS5tRxi8Rh507o2dNPpJk8Gbp1gzfeUGhLQtRxi6TasmU+xW/OHDj3XOjdG/7jP+KuSjKIOm6RVNm6Ff78Zzj5ZHj7bXjxRT+ZRqEtVaSOWyQVZs3yLnvFCrjmGh8a0aHasp/UcYtEaeNGv/l4+ulQUeEd9osvKrTlgCi4RaIycqTffOzTB+66y0+mOe+8uKuSLKDgFkm29evhl7/0vUUOP9xvQnbr5vtmiySBglskWUKA/v19Ic3IkX4aTWkpNGsWd2WSZXRzUiQZ3nnH98qeOhVatYKiIvjJT+KuSrKUOm6RA7FtG3TpAiedBPPne2BPmaLQlkip4xbZXyUlvnvfokVw5ZV+5uNRR8VdleQAddwiVbVpE9x9N5x6KpSX+z7ZQ4YotCVl1HGLVMX48XDLLbBmjT926QLf/37cVUmOUcctkohPPoHrrvN52DVq+JmPvXsrtCUWCm6RvQkBiot9it/gwfC//+vnQJ5xRtyVSQ7TUInInpSV+XDI3//u49n9+vlJ6yIxU8ct8m07dkD37tCwIcycCU895ZtEKbQlTajjFvlXixf7FL8334QLLvBx7OOOi7sqkV2o4xYB2LIFOnWCU07xGSN/+xuMHq3QlrSkjltk2jRo184PN/jNb6BrV/jBD+KuSmSP1HFL7vr8cw/sM8/0pesTJ8Jzzym0Je0puCU3DRvmU/z694d77vG9ss8+O+6qRBKioRLJLR9+CLfdBiNG+NmPY8ZA06ZxVyVSJeq4JTfs3Okn0TRo4POyH3sM5s1TaEtGUsct2W/lSj+od+ZMaNMG+vaF//zPuKsS2W/77LjNrIaZzTOzRWa2zMweSkVhIgfs66/h4YehcWNYtsxvPE6cqNCWjJfIUMlWoHUIoTHQBDjPzH4ebVkiB+iNN3wY5IEH4IorYMUKn+pntsvLioshPx+qVfPH4uI4ihWpmn0Gd3BfVT49qPJfiLQqkf315Zfw+9/DaafBF1/AqFG+mObII7/z0uJinw1YVuZ7SZWV+XOFt6S7hG5Omll1M1sIbAAmhhDmRluWyH4YM8b3F3nmGZ85snw5XHTRHl/eqRNUVOx6raLCr4uks4SCO4SwI4TQBDgGaGZm39ltx8zamVmJmZWUl5cnu06RPduwAa6+2kP6sMN8Q6innoKaNff6Z2vXVu26SLqo0nTAEMLnwOvAebv5XVEIoSCEUFC3bt0klSeyFyHAwIG+kGbYMPjLX/zA3ubNE/rzPW1Dou1JJN0lMqukrpnVqvz534GzgZVRFyayV6tWQdu2fsOxQQNYuNAPOTj44ITfonNnyMvb9Vpenl8XSWeJdNxHAVPNbDHwJj7GPTraskT2YPt23wSqUSOYO9e3XZ02zbvuKioshKIiqFfPJ5vUq+fPCwsjqFskiSyE5E8QKSgoCCUlJUl/X8lx8+f7Qpr58+HSS6FnT/jRj+KuSiQpzKw0hFCQyGu15F3SX0UF3HsvNGsG69bBkCEwfLhCW3KWlrxLeps82SdXr17t3fajj0Lt2nFXJRIrddySnj77DH77W99qtXp1mDrVB6AV2iIKbkkzIcDLL/vNxpdegj/9CRYt8sMORATQUImkk7Vr4dZbfQXkz34GEyb4BlEisgt13BK/HTt8mXrDhj4k8uSTMGeOQltkD9RxS7yWLYMbb/Td/M491w87yM+PuyqRtKaOW+KxdSs8+KAfH/bOOz6ePW6cQlskAeq4JfVmzvSpfStXwjXXQLduoP1tRBKmjltS54sv/ObjGWfA5s1+9uOLLyq0RapIwS2p8dprvhlU375w112wdKmPaYtIlSm4JVoffQS/+AVcdhnUqeM3Ibt1g0MPjbsykYyl4JZohAD9+vlCmlGj4JFHoKTE52eLyAHRzUlJvrff9v1Fpk3zFY99+8JPfhJ3VSJZQx23JM+2bfDXv8JJJ/nBBv36wZQpCm2RJFPHLcnx5pu+kGbxYvjv//YzH486Ku6qRLKSOm45MJs2wR/+AD//OXzyCYwYAa++qtAWiZA6btl/48fDLbfAmjXQvr0Pk3z/+3FXJZL11HFL1X3yCVx7LZx3HtSoATNmQK9eCm2RFFFwS+JC8D1FTjjB98x+4AG/CXn66XFXJpJTNFQiiVmzxodFxo/38ex+/XwbVhFJOXXcsnc7dvj+2A0bwqxZ8PTTvkmUQlskNuq4Zc8WLfJd/N58Ey66yMexjz027qpEcp46bvmuzZv9rMeCAigrg8GDYeRIhbZImlDHLbt6/XVfrv7OO37KeteucPjhcVclIv9CHbe4f/zDh0XOOsvHtSdNggEDFNoiaUjBnetCgCFDfK/s556De++FJUugTZu4KxORPdBQSS778EPo0MEPOWjaFMaO9TMgRSStqePORTt3Qu/e3mVPmACPPw5z5yq0RTKEOu5cs2KFj2XPmgVnn+17ZR9/fNxViUgVqOPOFV9/DX/5CzRpAsuXw/PPe7et0BbJOOq4c8GcOd5lL1sGV18N3bvDEUfEXZWI7Cd13Nnsyy/h9tuhRQvYuBFGj4ZBgxTaIhlOwZ2tRo/2m489e3p4L1sGF14Yd1UikgQK7mzz8cfwq1/BxRf7/tizZ0OPHlCzZtyViUiSKLizRQi+gKZ+fRg+HB5+GObP9y1YRSSr6OZkNli1Cm6+GSZPhjPOgKIiP+xARLKSOu5Mtn07PPYYNGrkW6/26eObRCm0RbKaOu5MNX8+3HgjLFgAl10GzzwDP/pR3FWJSArss+M2s2PNbKqZrTCzZWZ2RyoKkz2oqIB77oFmzeCjj2DoUB/TVmiL5IxEhkq2A3eHEOoDPwc6mFmDaMvKbcXFkJ8P1ar5Y3Fx5S8mTfJhka5d4YYbfPn6FVfEWKmIxGGfwR1C+CiEML/y5y+BFYDau4gUF/s5BmVlPlGkrAw63vQpq1r+Bs45B6pX93Hsvn2hVq24yxWRGFTp5qSZ5QMnA3OjKEagUycfDXGBqxhM6eb6HDej2I8TW7wYWrWKs0QRiVnCwW1mhwJDgTtDCBt38/t2ZlZiZiXl5eXJrDGnrF3rj8eyllFczGCuZg35FFAKnTtDjRrxFigisUsouM3sIDy0i0MIw3b3mhBCUQihIIRQULdu3WTWmFPyj93BbTzNchpwJq9zJ0/SnDl8Ue+kuEsTkTSxz+mAZmZAf2BFCKFb9CXlsKVLmXfQjdRhLuM4j/b0pox88vK82RYRgcQ67hbAtUBrM1tY+e+CiOvKLVu2wAMPwMknU+eLVcy6tZj2x41lreVTr54vhCwsjLtIEUkX++y4QwgzAUtBLblpxgzfK/utt+Daa6FbN1rUqcOannEXJiLpSkve4/LFF3DLLdCyJWzdCuPHwwsvQJ06cVcmImlOwR2HESN8r+xnn4W774alS6Ft27irEpEMoeBOpXXr4Mor4fLLoW5dP1m9a1c45JC4KxORDKLgToWdO727btAAxo6Fv/7Vd/MrKIi7MhHJQNodMGpvveVr2KdPhzPP9Cki//VfcVclIhlMHXdUtm2DRx6Bxo19mXq/fjBlikJbRA6YOu4ozJvne2UvWQK/+AU89RT88IdxVyUiWUIddzJ99RXcdZef8/jZZ/Daa/DKKwptEUkqddzJMm4ctG/v+7DeeqvfgDzssLirEpEspI77QJWXwzXXwAUXQF4ezJwJPXsqtEUkMgru/RUCvPgi1K/vwyEPPujnP7ZoEXdlIpLlNFSyP957D26+GSZOhObNfY52w4ZxVyUiOUIdd1Vs3w7dusGJJ8KcOX6y+syZCm0RSSl13IlauNCn+JWWwkUXQa9ecOyxcVclIjlIHfe+bN4M993ny9Pffx9efhlGjlRoi0hs1HHvzdSpvlz93Xfhd7+Dxx+Hww+PuyoRyXHquHfnH//wYZHWrX32yKRJ0L+/QltE0oKC+1+FAK++6lP8nn8e/ud/fJ+RNm3irkxE5P9pqOQbH3wAHTr4+HXTpr4S8uST465KROQ71HHv3OkzRBo08HnZXbv6AQcKbRFJU7ndcS9f7gf1zp4N55wDffrA8cfHXZWIyF7lZse9dSs89BA0aQIrV8LAgX5Yr0JbRDJA7nXcs2d7l718Ofz61/Dkk3DEEXFXJSKSsNzpuDduhNtug9NPhy+/hDFjoLhYoS0iGSc3gnvUKL/52KsX3H47LFvm27CKiGSg7A7u9evhqqvgkkugdm3fGKpHD6hZM+7KRET2W3YGdwgwYIAvpBkxAh5+2DeHOvXUuCsTETlg2Xdz8t13fa/sKVPgjDOgqAhOOCHuqkREkiZ7Ou5t2+DRR6FRIygp8TnZr7+u0BaRrJMdHXdpqW8KtXAhXH65H3Bw9NFxVyUiEonM7rg3bYI//hGaNYOPP4ahQ2HYMIW2iGS1zO24J070sez33vM9sx99FGrVirsqEZHIZV7H/emncP310LYtHHQQTJsGffsqtEUkZ2ROcIcAgwb5FL9Bg6BTJ1i0CFq2jLsyEZGUyoyhkrIyaN/e98hu1gwmT/bZIyIiOSi9O+4dO3ylY8OGMH06dO/um0QptEUkh6Vvx71kiU/xmzcPzj8feveGevXirkpEJHbp13Fv2QL33+/Hh61e7Tv4jRmj0BYRqZReHfesWfC738Hbb8N118ETT0CdOnFXJSKSVvbZcZvZADPbYGZLoyykuBhuvnwD7739NdceMZ7itgMV2iIiu5FIx/088AzwQlRFFBf7GpqKissZyPls3VCDYe38d4WFUX2qiEhm2mfHHUKYDnwWZRGdOkFFhf+8lRqAP+/UKcpPFRHJTEm7OWlm7cysxMxKysvLq/S3a9dW7bqISC5LWnCHEIpCCAUhhIK6detW6W+PO65q10VEcllaTAfs3Bny8na9lpfn10VEZFdpEdyFhX5QTb16YOaPRUW6MSkisjv7nFViZn8DzgTqmNkHwIMhhP7JLqSwUEEtIpKIfQZ3COHqVBQiIiKJSYuhEhERSZyCW0Qkwyi4RUQyjIJbRCTDKLhFRDKMhRCS/6Zm5UDZfv55HeCTJJaTCfSds1+ufV/Qd66qeiGEhJadRxLcB8LMSkIIBXHXkUr6ztkv174v6DtHSUMlIiIZRsEtIpJh0jG4i+IuIAb6ztkv174v6DtHJu3GuEVEZO/SseMWEZG9SJvgTtWhxOnEzI41s6lmtsLMlpnZHXHXFCUzq2Fm88xsUeX3fSjumlLFzKqb2QIzGx13LalgZmvMbImZLTSzkrjriZqZ1TKzIWa2svK/5+aRfl66DJWYWUvgK+CFEMKJcdeTCmZ2FHBUCGG+mdUESoHLQgjLYy4tEmZmwCEhhK/M7CBgJnBHCOGNmEuLnJn9ASgADgshXBR3PVEzszVAQQghJ+Zxm9lAYEYIoZ+ZHQzkhRA+j+rz0qbjTsWhxOkmhPBRCGF+5c9fAiuAH8VbVXSC+6ry6UGV/9Kjc4iQmR0DXAj0i7sWST4zOwxoCfQHCCF8HWVoQxoFd64zs3zgZGBuvJVEq3LIYCGwAZgYQsjq71upO3AvsDPuQlIoABPMrNTM2sVdTMSOB8qB5yqHw/qZ2SFRfqCCOw2Y2aHAUODOEMLGuOuJUghhRwihCXAM0MzMsnpYzMwuAjaEEErjriXFWoQQmgLnAx0qh0Kz1feApkDvEMLJwCagY5QfqOCOWeVY71CgOIQwLO56UqXyfyVfB86LuZSotQAuqRzzHQy0NrOX4i0peiGEdZWPG4DhQLN4K4rUB8AH//J/j0PwII+MgjtGlTfr+gMrQgjd4q4namZW18xqVf7878DZwMp4q4pWCOG+EMIxIYR84FfAlBDCNTGXFSkzO6TyZjuVQwZtgaydLRZCWA+8b2Y/rbzUBoh0gsE+z5xMlVQdSpxmWgDXAksqx30B/hRCGBtjTVE6ChhoZtXxpuGVEEJOTI/LMUcCw70v4XvAoBDC3+MtKXK3A8WVM0pWA7+N8sPSZjqgiIgkRkMlIiIZRsEtIpJhFNwiIhlGwS0ikmEU3CIiGUbBLSKSYRTcIiIZRsEtIpJh/g/fd/K2NzDSJgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "ne_lr = LinearRegression()\n",
    "ne_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % ne_lr.b_)\n",
    "print('Slope: %.2f' % ne_lr.w_[0])\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, ne_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - QR decomposition method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.25\n",
      "Slope: 0.81\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAG11JREFUeJzt3WmUlOWZxvH/DdFhWjFgQKNR6XEyiYAIYgeDKCgo7vskxrRLEhVFNGqMDgZHYzwYVERQ2VpAUZugsskadtkFu9k3F5BGRaTVKEoDsjzz4W4nQVmqod56a7l+53Cq6+3qqrs+eJ3b530WCyEgIiKZo1rcBYiISNUouEVEMoyCW0Qkwyi4RUQyjIJbRCTDKLhFRDKMgltEJMMouEVEMoyCW0Qkw3wvijetU6dOyM/Pj+KtRUSyUmlp6SchhLqJvDaS4M7Pz6ekpCSKtxYRyUpmVpboazVUIiKSYRTcIiIZRsEtIpJhFNwiIhlGwS0ikmEU3CIiGUbBLSKSYRTcIiIHauNGuPtuWL8+JR+n4BYRORCjRkGDBvDkkzB+fEo+UsEtIrI/1q+Hq66CSy6B2rVhzhy4/vqUfLSCW0SkKkKAAQOgfn0YMQIefhhKS+HUU1NWQkLBbWZrzGyJmS00M21CIiK56d134eyz4YYboFEjWLQI7r+f4lcPJj8fqlWD/HwoLo62jKpsMnVWCOGTyCoREUlX27ZBt27w5z/DwQdDnz5w001QrRrFxdCuHVRU+EvLyvw5QGFhNOVoqEREZG9KS6FZM+jYEc4/H1asgJtv9vYa6NTpn6H9jYoKvx6VRIM7ABPMrNTM2u3uBWbWzsxKzKykvLw8eRWKiMRh0yb44x89tD/+GIYOhWHD4Oijd3nZ2rW7//M9XU+GRIO7RQihKXA+0MHMWn77BSGEohBCQQihoG7dhPYCFxFJTxMn+hj2E0/AjTfC8uVwxRW7felxx+3+LfZ0PRkSCu4QwrrKxw3AcKBZdCWJiMTk0099Sl/btnDQQTBtGvTtC7Vq7fFPOneGvLxdr+Xl+fWo7DO4zewQM6v5zc9AW2BpdCWJiKRYCDBokE/xGzTIB6gXLYKW3xlc+I7CQigqgnr1wMwfi4qiuzEJic0qORIYbmbfvH5QCOHv0ZUkIpJCZWXQvj2MG+fj2ZMn+zBJFRQWRhvU37bP4A4hrAYap6AWEZHU2bEDnnnmn9M/uneH226D6tXjrSsBkRwWLCKS1pYs8ZuO8+b5FL/evX2MI0NoHreI5I4tW+D++6FpU1i92pc4jhmTUaEN6rhFJFdMn+6rHd9+G667zqf61akTd1X7RR23iGS3zz/3lY6tWvnS9fHjYeDAjA1tUHCLSDYbPtz3yu7Xzw86WLLE52hnOA2ViEj2WbfOZ4gMHw5NmsDIkVBQEHdVSaOOW0Syx86dvvqlfn2fl92li88cyaLQBnXcIpIt3nrL91OdPh3OOssD/Mc/jruqSKjjFpHM9vXXvjFI48aweDH07++rH7M0tEEdt4hksrlzfSHN0qXwy19Cjx7wwx/GXVXk1HGLSOb56iu4805o3tyn+40cCS+/nBOhDeq4RSTTjBsHt9wC778Pt94KjzwChx0Wd1UppY5bRDLDhg2+Bd8FF8Ahh8DMmb5JVI6FNii4RSTdhQAvvOBT/F591Q/sXbAATjst7spio6ESEUlf773ny9UnTvSgfvZZXwmZ49Rxi0j62b7dN4E68UR44w3o2RNmzFBoV1LHLSLpZeFCn+JXWgoXXwy9esExx8RdVVpRxy0i6WHzZujY0Zenf/ABvPIKvPaaQns31HGLSPymTPHl6qtWwQ03wOOPQ+3acVeVttRxi0h8PvvMg7pNG38+ebJvwarQ3isFt4ikXgg+FFK/vh9q0LGj75XdunXclWUEDZWISGp98IGveBw1Ck45xU+kadIk7qoyijpuEUmNnTt9Wl+DBjBpkk/3e+MNhfZ+UMctItFbvtwP6p09G845B/r0geOPj7uqjKWOW0Sis3WrL1Fv0gRWrvSl6+PHK7QPkDpuEYnG7Nm+kGbFCt8c6sknoW7duKvKCuq4RSS5Nm6EDh3g9NNh0yYYOxZeekmhnUQKbhFJnpEj/eZj795wxx2wbBmcf37cVWUdBbeIHLj16/3osEsvhcMPhzlzfGjk0EPjriwrKbhFZP+FAAMG+EKakSP90N7SUjj11Lgry2q6OSki++fdd31/kalToWVLKCqCn/407qpygjpuEamabdugSxdo1Ajmz/fAnjpVoZ1C6rhFJHElJT7Fb9EiuOIKePppOProuKvKOeq4RWTfNm2Cu+/2sesNG2DYMBg6VKEdE3XcIrJ3Eyb4uY9r1vhjly5Qq1bcVeU0ddwisnuffgrXXw/nngv/9m8wfbrvMaLQjp2CW0R2FQIMGgQnnOCP99/v50CecUbclUklDZWIyD+VlUH79jBunI9nP/uszx6RtJJwx21m1c1sgZmNjrIgEYnBjh3Qowc0bOhDIj16wKxZCu00VZWhkjuAFVEVIiJQXAz5+VCtmj8WF6fgQ5csgdNOgzvvhFatfO/s3/8eqldPwYfL/kgouM3sGOBCoF+05YjkruJiX4hYVubDzGVl/jyy8N6yxcevmzaF997z8ezRo+G44yL6QEmWRDvu7sC9wM4IaxHJaZ06QUXFrtcqKvx60k2fDo0b+94ihYW+Z/bVV4NZBB8mybbP4Dazi4ANIYTSfbyunZmVmFlJeXl50goUyRVr11bt+n75/HOfi92qlS9dnzABnn8efvCDJH6IRC2RjrsFcImZrQEGA63N7KVvvyiEUBRCKAghFNTVhukiVbanEYqkjVwMG+Z7ZffrB/fcA0uX+vmPknH2GdwhhPtCCMeEEPKBXwFTQgjXRF6ZSI7p3Bny8na9lpfn1w/IunW+r8iVV8KRR8K8efDYY9/9MMkYWoAjkiYKC32jvXr1fKi5Xj1/Xli4n2+4cyf07et7ZY8bB48+6qF9yilJrVtSz0IISX/TgoKCUFJSkvT3FZEEvfUW3HQTzJgBrVt7gP/4x3FXJXthZqUhhIJEXquOWySbfP21j62cdJKPYQ8YAJMmKbSzjJa8i2SLuXN9r+ylS+Gqq3z145FHxl2VREAdt0im++orP1G9eXOf7jdqFAwerNDOYuq4RTLZ2LG+KdT770OHDvDII1CzZtxVScTUcYtkog0b4Ne/hgsvhEMP9Q2hnn5aoZ0jFNwimSQEGDjQp/gNHQoPPQQLFvgwieQMDZWIZIrVq325+qRJ0KKF75Vdv37cVUkM1HGLpLvt2+GJJ+DEE33mSK9evkmUQjtnqeMWSWcLFvhCmtJSuPRSeOYZOOaYuKuSmKnjFklHmzdDx47ws5/Bhx/CkCEwfLhCWwB13CLpZ8oUP0Fh1SpfUPPYY1C7dtxVSRpRxy2SLj77DG64Adq08V2mpkzxG5AKbfkWBbdI3EKAV17xm40DB8J998HixXDWWXFXJmlKQyUicfpmxeOoUVBQ4CfSNG4cd1WS5tRxi8Rh507o2dNPpJk8Gbp1gzfeUGhLQtRxi6TasmU+xW/OHDj3XOjdG/7jP+KuSjKIOm6RVNm6Ff78Zzj5ZHj7bXjxRT+ZRqEtVaSOWyQVZs3yLnvFCrjmGh8a0aHasp/UcYtEaeNGv/l4+ulQUeEd9osvKrTlgCi4RaIycqTffOzTB+66y0+mOe+8uKuSLKDgFkm29evhl7/0vUUOP9xvQnbr5vtmiySBglskWUKA/v19Ic3IkX4aTWkpNGsWd2WSZXRzUiQZ3nnH98qeOhVatYKiIvjJT+KuSrKUOm6RA7FtG3TpAiedBPPne2BPmaLQlkip4xbZXyUlvnvfokVw5ZV+5uNRR8VdleQAddwiVbVpE9x9N5x6KpSX+z7ZQ4YotCVl1HGLVMX48XDLLbBmjT926QLf/37cVUmOUcctkohPPoHrrvN52DVq+JmPvXsrtCUWCm6RvQkBiot9it/gwfC//+vnQJ5xRtyVSQ7TUInInpSV+XDI3//u49n9+vlJ6yIxU8ct8m07dkD37tCwIcycCU895ZtEKbQlTajjFvlXixf7FL8334QLLvBx7OOOi7sqkV2o4xYB2LIFOnWCU07xGSN/+xuMHq3QlrSkjltk2jRo184PN/jNb6BrV/jBD+KuSmSP1HFL7vr8cw/sM8/0pesTJ8Jzzym0Je0puCU3DRvmU/z694d77vG9ss8+O+6qRBKioRLJLR9+CLfdBiNG+NmPY8ZA06ZxVyVSJeq4JTfs3Okn0TRo4POyH3sM5s1TaEtGUsct2W/lSj+od+ZMaNMG+vaF//zPuKsS2W/77LjNrIaZzTOzRWa2zMweSkVhIgfs66/h4YehcWNYtsxvPE6cqNCWjJfIUMlWoHUIoTHQBDjPzH4ebVkiB+iNN3wY5IEH4IorYMUKn+pntsvLioshPx+qVfPH4uI4ihWpmn0Gd3BfVT49qPJfiLQqkf315Zfw+9/DaafBF1/AqFG+mObII7/z0uJinw1YVuZ7SZWV+XOFt6S7hG5Omll1M1sIbAAmhhDmRluWyH4YM8b3F3nmGZ85snw5XHTRHl/eqRNUVOx6raLCr4uks4SCO4SwI4TQBDgGaGZm39ltx8zamVmJmZWUl5cnu06RPduwAa6+2kP6sMN8Q6innoKaNff6Z2vXVu26SLqo0nTAEMLnwOvAebv5XVEIoSCEUFC3bt0klSeyFyHAwIG+kGbYMPjLX/zA3ubNE/rzPW1Dou1JJN0lMqukrpnVqvz534GzgZVRFyayV6tWQdu2fsOxQQNYuNAPOTj44ITfonNnyMvb9Vpenl8XSWeJdNxHAVPNbDHwJj7GPTraskT2YPt23wSqUSOYO9e3XZ02zbvuKioshKIiqFfPJ5vUq+fPCwsjqFskiSyE5E8QKSgoCCUlJUl/X8lx8+f7Qpr58+HSS6FnT/jRj+KuSiQpzKw0hFCQyGu15F3SX0UF3HsvNGsG69bBkCEwfLhCW3KWlrxLeps82SdXr17t3fajj0Lt2nFXJRIrddySnj77DH77W99qtXp1mDrVB6AV2iIKbkkzIcDLL/vNxpdegj/9CRYt8sMORATQUImkk7Vr4dZbfQXkz34GEyb4BlEisgt13BK/HTt8mXrDhj4k8uSTMGeOQltkD9RxS7yWLYMbb/Td/M491w87yM+PuyqRtKaOW+KxdSs8+KAfH/bOOz6ePW6cQlskAeq4JfVmzvSpfStXwjXXQLduoP1tRBKmjltS54sv/ObjGWfA5s1+9uOLLyq0RapIwS2p8dprvhlU375w112wdKmPaYtIlSm4JVoffQS/+AVcdhnUqeM3Ibt1g0MPjbsykYyl4JZohAD9+vlCmlGj4JFHoKTE52eLyAHRzUlJvrff9v1Fpk3zFY99+8JPfhJ3VSJZQx23JM+2bfDXv8JJJ/nBBv36wZQpCm2RJFPHLcnx5pu+kGbxYvjv//YzH486Ku6qRLKSOm45MJs2wR/+AD//OXzyCYwYAa++qtAWiZA6btl/48fDLbfAmjXQvr0Pk3z/+3FXJZL11HFL1X3yCVx7LZx3HtSoATNmQK9eCm2RFFFwS+JC8D1FTjjB98x+4AG/CXn66XFXJpJTNFQiiVmzxodFxo/38ex+/XwbVhFJOXXcsnc7dvj+2A0bwqxZ8PTTvkmUQlskNuq4Zc8WLfJd/N58Ey66yMexjz027qpEcp46bvmuzZv9rMeCAigrg8GDYeRIhbZImlDHLbt6/XVfrv7OO37KeteucPjhcVclIv9CHbe4f/zDh0XOOsvHtSdNggEDFNoiaUjBnetCgCFDfK/s556De++FJUugTZu4KxORPdBQSS778EPo0MEPOWjaFMaO9TMgRSStqePORTt3Qu/e3mVPmACPPw5z5yq0RTKEOu5cs2KFj2XPmgVnn+17ZR9/fNxViUgVqOPOFV9/DX/5CzRpAsuXw/PPe7et0BbJOOq4c8GcOd5lL1sGV18N3bvDEUfEXZWI7Cd13Nnsyy/h9tuhRQvYuBFGj4ZBgxTaIhlOwZ2tRo/2m489e3p4L1sGF14Yd1UikgQK7mzz8cfwq1/BxRf7/tizZ0OPHlCzZtyViUiSKLizRQi+gKZ+fRg+HB5+GObP9y1YRSSr6OZkNli1Cm6+GSZPhjPOgKIiP+xARLKSOu5Mtn07PPYYNGrkW6/26eObRCm0RbKaOu5MNX8+3HgjLFgAl10GzzwDP/pR3FWJSArss+M2s2PNbKqZrTCzZWZ2RyoKkz2oqIB77oFmzeCjj2DoUB/TVmiL5IxEhkq2A3eHEOoDPwc6mFmDaMvKbcXFkJ8P1ar5Y3Fx5S8mTfJhka5d4YYbfPn6FVfEWKmIxGGfwR1C+CiEML/y5y+BFYDau4gUF/s5BmVlPlGkrAw63vQpq1r+Bs45B6pX93Hsvn2hVq24yxWRGFTp5qSZ5QMnA3OjKEagUycfDXGBqxhM6eb6HDej2I8TW7wYWrWKs0QRiVnCwW1mhwJDgTtDCBt38/t2ZlZiZiXl5eXJrDGnrF3rj8eyllFczGCuZg35FFAKnTtDjRrxFigisUsouM3sIDy0i0MIw3b3mhBCUQihIIRQULdu3WTWmFPyj93BbTzNchpwJq9zJ0/SnDl8Ue+kuEsTkTSxz+mAZmZAf2BFCKFb9CXlsKVLmXfQjdRhLuM4j/b0pox88vK82RYRgcQ67hbAtUBrM1tY+e+CiOvKLVu2wAMPwMknU+eLVcy6tZj2x41lreVTr54vhCwsjLtIEUkX++y4QwgzAUtBLblpxgzfK/utt+Daa6FbN1rUqcOannEXJiLpSkve4/LFF3DLLdCyJWzdCuPHwwsvQJ06cVcmImlOwR2HESN8r+xnn4W774alS6Ft27irEpEMoeBOpXXr4Mor4fLLoW5dP1m9a1c45JC4KxORDKLgToWdO727btAAxo6Fv/7Vd/MrKIi7MhHJQNodMGpvveVr2KdPhzPP9Cki//VfcVclIhlMHXdUtm2DRx6Bxo19mXq/fjBlikJbRA6YOu4ozJvne2UvWQK/+AU89RT88IdxVyUiWUIddzJ99RXcdZef8/jZZ/Daa/DKKwptEUkqddzJMm4ctG/v+7DeeqvfgDzssLirEpEspI77QJWXwzXXwAUXQF4ezJwJPXsqtEUkMgru/RUCvPgi1K/vwyEPPujnP7ZoEXdlIpLlNFSyP957D26+GSZOhObNfY52w4ZxVyUiOUIdd1Vs3w7dusGJJ8KcOX6y+syZCm0RSSl13IlauNCn+JWWwkUXQa9ecOyxcVclIjlIHfe+bN4M993ny9Pffx9efhlGjlRoi0hs1HHvzdSpvlz93Xfhd7+Dxx+Hww+PuyoRyXHquHfnH//wYZHWrX32yKRJ0L+/QltE0oKC+1+FAK++6lP8nn8e/ud/fJ+RNm3irkxE5P9pqOQbH3wAHTr4+HXTpr4S8uST465KROQ71HHv3OkzRBo08HnZXbv6AQcKbRFJU7ndcS9f7gf1zp4N55wDffrA8cfHXZWIyF7lZse9dSs89BA0aQIrV8LAgX5Yr0JbRDJA7nXcs2d7l718Ofz61/Dkk3DEEXFXJSKSsNzpuDduhNtug9NPhy+/hDFjoLhYoS0iGSc3gnvUKL/52KsX3H47LFvm27CKiGSg7A7u9evhqqvgkkugdm3fGKpHD6hZM+7KRET2W3YGdwgwYIAvpBkxAh5+2DeHOvXUuCsTETlg2Xdz8t13fa/sKVPgjDOgqAhOOCHuqkREkiZ7Ou5t2+DRR6FRIygp8TnZr7+u0BaRrJMdHXdpqW8KtXAhXH65H3Bw9NFxVyUiEonM7rg3bYI//hGaNYOPP4ahQ2HYMIW2iGS1zO24J070sez33vM9sx99FGrVirsqEZHIZV7H/emncP310LYtHHQQTJsGffsqtEUkZ2ROcIcAgwb5FL9Bg6BTJ1i0CFq2jLsyEZGUyoyhkrIyaN/e98hu1gwmT/bZIyIiOSi9O+4dO3ylY8OGMH06dO/um0QptEUkh6Vvx71kiU/xmzcPzj8feveGevXirkpEJHbp13Fv2QL33+/Hh61e7Tv4jRmj0BYRqZReHfesWfC738Hbb8N118ETT0CdOnFXJSKSVvbZcZvZADPbYGZLoyykuBhuvnwD7739NdceMZ7itgMV2iIiu5FIx/088AzwQlRFFBf7GpqKissZyPls3VCDYe38d4WFUX2qiEhm2mfHHUKYDnwWZRGdOkFFhf+8lRqAP+/UKcpPFRHJTEm7OWlm7cysxMxKysvLq/S3a9dW7bqISC5LWnCHEIpCCAUhhIK6detW6W+PO65q10VEcllaTAfs3Bny8na9lpfn10VEZFdpEdyFhX5QTb16YOaPRUW6MSkisjv7nFViZn8DzgTqmNkHwIMhhP7JLqSwUEEtIpKIfQZ3COHqVBQiIiKJSYuhEhERSZyCW0Qkwyi4RUQyjIJbRCTDKLhFRDKMhRCS/6Zm5UDZfv55HeCTJJaTCfSds1+ufV/Qd66qeiGEhJadRxLcB8LMSkIIBXHXkUr6ztkv174v6DtHSUMlIiIZRsEtIpJh0jG4i+IuIAb6ztkv174v6DtHJu3GuEVEZO/SseMWEZG9SJvgTtWhxOnEzI41s6lmtsLMlpnZHXHXFCUzq2Fm88xsUeX3fSjumlLFzKqb2QIzGx13LalgZmvMbImZLTSzkrjriZqZ1TKzIWa2svK/5+aRfl66DJWYWUvgK+CFEMKJcdeTCmZ2FHBUCGG+mdUESoHLQgjLYy4tEmZmwCEhhK/M7CBgJnBHCOGNmEuLnJn9ASgADgshXBR3PVEzszVAQQghJ+Zxm9lAYEYIoZ+ZHQzkhRA+j+rz0qbjTsWhxOkmhPBRCGF+5c9fAiuAH8VbVXSC+6ry6UGV/9Kjc4iQmR0DXAj0i7sWST4zOwxoCfQHCCF8HWVoQxoFd64zs3zgZGBuvJVEq3LIYCGwAZgYQsjq71upO3AvsDPuQlIoABPMrNTM2sVdTMSOB8qB5yqHw/qZ2SFRfqCCOw2Y2aHAUODOEMLGuOuJUghhRwihCXAM0MzMsnpYzMwuAjaEEErjriXFWoQQmgLnAx0qh0Kz1feApkDvEMLJwCagY5QfqOCOWeVY71CgOIQwLO56UqXyfyVfB86LuZSotQAuqRzzHQy0NrOX4i0peiGEdZWPG4DhQLN4K4rUB8AH//J/j0PwII+MgjtGlTfr+gMrQgjd4q4namZW18xqVf7878DZwMp4q4pWCOG+EMIxIYR84FfAlBDCNTGXFSkzO6TyZjuVQwZtgaydLRZCWA+8b2Y/rbzUBoh0gsE+z5xMlVQdSpxmWgDXAksqx30B/hRCGBtjTVE6ChhoZtXxpuGVEEJOTI/LMUcCw70v4XvAoBDC3+MtKXK3A8WVM0pWA7+N8sPSZjqgiIgkRkMlIiIZRsEtIpJhFNwiIhlGwS0ikmEU3CIiGUbBLSKSYRTcIiIZRsEtIpJh/g/fd/K2NzDSJgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "qr_lr = LinearRegression(method='qr')\n",
    "qr_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % qr_lr.b_)\n",
    "print('Slope: %.2f' % qr_lr.w_[0])\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, qr_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3 - SVD method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.25\n",
      "Slope: 0.81\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAG11JREFUeJzt3WmUlOWZxvH/DdFhWjFgQKNR6XEyiYAIYgeDKCgo7vskxrRLEhVFNGqMDgZHYzwYVERQ2VpAUZugsskadtkFu9k3F5BGRaTVKEoDsjzz4W4nQVmqod56a7l+53Cq6+3qqrs+eJ3b530WCyEgIiKZo1rcBYiISNUouEVEMoyCW0Qkwyi4RUQyjIJbRCTDKLhFRDKMgltEJMMouEVEMoyCW0Qkw3wvijetU6dOyM/Pj+KtRUSyUmlp6SchhLqJvDaS4M7Pz6ekpCSKtxYRyUpmVpboazVUIiKSYRTcIiIZRsEtIpJhFNwiIhlGwS0ikmEU3CIiGUbBLSKSYRTcIiIHauNGuPtuWL8+JR+n4BYRORCjRkGDBvDkkzB+fEo+UsEtIrI/1q+Hq66CSy6B2rVhzhy4/vqUfLSCW0SkKkKAAQOgfn0YMQIefhhKS+HUU1NWQkLBbWZrzGyJmS00M21CIiK56d134eyz4YYboFEjWLQI7r+f4lcPJj8fqlWD/HwoLo62jKpsMnVWCOGTyCoREUlX27ZBt27w5z/DwQdDnz5w001QrRrFxdCuHVRU+EvLyvw5QGFhNOVoqEREZG9KS6FZM+jYEc4/H1asgJtv9vYa6NTpn6H9jYoKvx6VRIM7ABPMrNTM2u3uBWbWzsxKzKykvLw8eRWKiMRh0yb44x89tD/+GIYOhWHD4Oijd3nZ2rW7//M9XU+GRIO7RQihKXA+0MHMWn77BSGEohBCQQihoG7dhPYCFxFJTxMn+hj2E0/AjTfC8uVwxRW7felxx+3+LfZ0PRkSCu4QwrrKxw3AcKBZdCWJiMTk0099Sl/btnDQQTBtGvTtC7Vq7fFPOneGvLxdr+Xl+fWo7DO4zewQM6v5zc9AW2BpdCWJiKRYCDBokE/xGzTIB6gXLYKW3xlc+I7CQigqgnr1wMwfi4qiuzEJic0qORIYbmbfvH5QCOHv0ZUkIpJCZWXQvj2MG+fj2ZMn+zBJFRQWRhvU37bP4A4hrAYap6AWEZHU2bEDnnnmn9M/uneH226D6tXjrSsBkRwWLCKS1pYs8ZuO8+b5FL/evX2MI0NoHreI5I4tW+D++6FpU1i92pc4jhmTUaEN6rhFJFdMn+6rHd9+G667zqf61akTd1X7RR23iGS3zz/3lY6tWvnS9fHjYeDAjA1tUHCLSDYbPtz3yu7Xzw86WLLE52hnOA2ViEj2WbfOZ4gMHw5NmsDIkVBQEHdVSaOOW0Syx86dvvqlfn2fl92li88cyaLQBnXcIpIt3nrL91OdPh3OOssD/Mc/jruqSKjjFpHM9vXXvjFI48aweDH07++rH7M0tEEdt4hksrlzfSHN0qXwy19Cjx7wwx/GXVXk1HGLSOb56iu4805o3tyn+40cCS+/nBOhDeq4RSTTjBsHt9wC778Pt94KjzwChx0Wd1UppY5bRDLDhg2+Bd8FF8Ahh8DMmb5JVI6FNii4RSTdhQAvvOBT/F591Q/sXbAATjst7spio6ESEUlf773ny9UnTvSgfvZZXwmZ49Rxi0j62b7dN4E68UR44w3o2RNmzFBoV1LHLSLpZeFCn+JXWgoXXwy9esExx8RdVVpRxy0i6WHzZujY0Zenf/ABvPIKvPaaQns31HGLSPymTPHl6qtWwQ03wOOPQ+3acVeVttRxi0h8PvvMg7pNG38+ebJvwarQ3isFt4ikXgg+FFK/vh9q0LGj75XdunXclWUEDZWISGp98IGveBw1Ck45xU+kadIk7qoyijpuEUmNnTt9Wl+DBjBpkk/3e+MNhfZ+UMctItFbvtwP6p09G845B/r0geOPj7uqjKWOW0Sis3WrL1Fv0gRWrvSl6+PHK7QPkDpuEYnG7Nm+kGbFCt8c6sknoW7duKvKCuq4RSS5Nm6EDh3g9NNh0yYYOxZeekmhnUQKbhFJnpEj/eZj795wxx2wbBmcf37cVWUdBbeIHLj16/3osEsvhcMPhzlzfGjk0EPjriwrKbhFZP+FAAMG+EKakSP90N7SUjj11Lgry2q6OSki++fdd31/kalToWVLKCqCn/407qpygjpuEamabdugSxdo1Ajmz/fAnjpVoZ1C6rhFJHElJT7Fb9EiuOIKePppOProuKvKOeq4RWTfNm2Cu+/2sesNG2DYMBg6VKEdE3XcIrJ3Eyb4uY9r1vhjly5Qq1bcVeU0ddwisnuffgrXXw/nngv/9m8wfbrvMaLQjp2CW0R2FQIMGgQnnOCP99/v50CecUbclUklDZWIyD+VlUH79jBunI9nP/uszx6RtJJwx21m1c1sgZmNjrIgEYnBjh3Qowc0bOhDIj16wKxZCu00VZWhkjuAFVEVIiJQXAz5+VCtmj8WF6fgQ5csgdNOgzvvhFatfO/s3/8eqldPwYfL/kgouM3sGOBCoF+05YjkruJiX4hYVubDzGVl/jyy8N6yxcevmzaF997z8ezRo+G44yL6QEmWRDvu7sC9wM4IaxHJaZ06QUXFrtcqKvx60k2fDo0b+94ihYW+Z/bVV4NZBB8mybbP4Dazi4ANIYTSfbyunZmVmFlJeXl50goUyRVr11bt+n75/HOfi92qlS9dnzABnn8efvCDJH6IRC2RjrsFcImZrQEGA63N7KVvvyiEUBRCKAghFNTVhukiVbanEYqkjVwMG+Z7ZffrB/fcA0uX+vmPknH2GdwhhPtCCMeEEPKBXwFTQgjXRF6ZSI7p3Bny8na9lpfn1w/IunW+r8iVV8KRR8K8efDYY9/9MMkYWoAjkiYKC32jvXr1fKi5Xj1/Xli4n2+4cyf07et7ZY8bB48+6qF9yilJrVtSz0IISX/TgoKCUFJSkvT3FZEEvfUW3HQTzJgBrVt7gP/4x3FXJXthZqUhhIJEXquOWySbfP21j62cdJKPYQ8YAJMmKbSzjJa8i2SLuXN9r+ylS+Gqq3z145FHxl2VREAdt0im++orP1G9eXOf7jdqFAwerNDOYuq4RTLZ2LG+KdT770OHDvDII1CzZtxVScTUcYtkog0b4Ne/hgsvhEMP9Q2hnn5aoZ0jFNwimSQEGDjQp/gNHQoPPQQLFvgwieQMDZWIZIrVq325+qRJ0KKF75Vdv37cVUkM1HGLpLvt2+GJJ+DEE33mSK9evkmUQjtnqeMWSWcLFvhCmtJSuPRSeOYZOOaYuKuSmKnjFklHmzdDx47ws5/Bhx/CkCEwfLhCWwB13CLpZ8oUP0Fh1SpfUPPYY1C7dtxVSRpRxy2SLj77DG64Adq08V2mpkzxG5AKbfkWBbdI3EKAV17xm40DB8J998HixXDWWXFXJmlKQyUicfpmxeOoUVBQ4CfSNG4cd1WS5tRxi8Rh507o2dNPpJk8Gbp1gzfeUGhLQtRxi6TasmU+xW/OHDj3XOjdG/7jP+KuSjKIOm6RVNm6Ff78Zzj5ZHj7bXjxRT+ZRqEtVaSOWyQVZs3yLnvFCrjmGh8a0aHasp/UcYtEaeNGv/l4+ulQUeEd9osvKrTlgCi4RaIycqTffOzTB+66y0+mOe+8uKuSLKDgFkm29evhl7/0vUUOP9xvQnbr5vtmiySBglskWUKA/v19Ic3IkX4aTWkpNGsWd2WSZXRzUiQZ3nnH98qeOhVatYKiIvjJT+KuSrKUOm6RA7FtG3TpAiedBPPne2BPmaLQlkip4xbZXyUlvnvfokVw5ZV+5uNRR8VdleQAddwiVbVpE9x9N5x6KpSX+z7ZQ4YotCVl1HGLVMX48XDLLbBmjT926QLf/37cVUmOUcctkohPPoHrrvN52DVq+JmPvXsrtCUWCm6RvQkBiot9it/gwfC//+vnQJ5xRtyVSQ7TUInInpSV+XDI3//u49n9+vlJ6yIxU8ct8m07dkD37tCwIcycCU895ZtEKbQlTajjFvlXixf7FL8334QLLvBx7OOOi7sqkV2o4xYB2LIFOnWCU07xGSN/+xuMHq3QlrSkjltk2jRo184PN/jNb6BrV/jBD+KuSmSP1HFL7vr8cw/sM8/0pesTJ8Jzzym0Je0puCU3DRvmU/z694d77vG9ss8+O+6qRBKioRLJLR9+CLfdBiNG+NmPY8ZA06ZxVyVSJeq4JTfs3Okn0TRo4POyH3sM5s1TaEtGUsct2W/lSj+od+ZMaNMG+vaF//zPuKsS2W/77LjNrIaZzTOzRWa2zMweSkVhIgfs66/h4YehcWNYtsxvPE6cqNCWjJfIUMlWoHUIoTHQBDjPzH4ebVkiB+iNN3wY5IEH4IorYMUKn+pntsvLioshPx+qVfPH4uI4ihWpmn0Gd3BfVT49qPJfiLQqkf315Zfw+9/DaafBF1/AqFG+mObII7/z0uJinw1YVuZ7SZWV+XOFt6S7hG5Omll1M1sIbAAmhhDmRluWyH4YM8b3F3nmGZ85snw5XHTRHl/eqRNUVOx6raLCr4uks4SCO4SwI4TQBDgGaGZm39ltx8zamVmJmZWUl5cnu06RPduwAa6+2kP6sMN8Q6innoKaNff6Z2vXVu26SLqo0nTAEMLnwOvAebv5XVEIoSCEUFC3bt0klSeyFyHAwIG+kGbYMPjLX/zA3ubNE/rzPW1Dou1JJN0lMqukrpnVqvz534GzgZVRFyayV6tWQdu2fsOxQQNYuNAPOTj44ITfonNnyMvb9Vpenl8XSWeJdNxHAVPNbDHwJj7GPTraskT2YPt23wSqUSOYO9e3XZ02zbvuKioshKIiqFfPJ5vUq+fPCwsjqFskiSyE5E8QKSgoCCUlJUl/X8lx8+f7Qpr58+HSS6FnT/jRj+KuSiQpzKw0hFCQyGu15F3SX0UF3HsvNGsG69bBkCEwfLhCW3KWlrxLeps82SdXr17t3fajj0Lt2nFXJRIrddySnj77DH77W99qtXp1mDrVB6AV2iIKbkkzIcDLL/vNxpdegj/9CRYt8sMORATQUImkk7Vr4dZbfQXkz34GEyb4BlEisgt13BK/HTt8mXrDhj4k8uSTMGeOQltkD9RxS7yWLYMbb/Td/M491w87yM+PuyqRtKaOW+KxdSs8+KAfH/bOOz6ePW6cQlskAeq4JfVmzvSpfStXwjXXQLduoP1tRBKmjltS54sv/ObjGWfA5s1+9uOLLyq0RapIwS2p8dprvhlU375w112wdKmPaYtIlSm4JVoffQS/+AVcdhnUqeM3Ibt1g0MPjbsykYyl4JZohAD9+vlCmlGj4JFHoKTE52eLyAHRzUlJvrff9v1Fpk3zFY99+8JPfhJ3VSJZQx23JM+2bfDXv8JJJ/nBBv36wZQpCm2RJFPHLcnx5pu+kGbxYvjv//YzH486Ku6qRLKSOm45MJs2wR/+AD//OXzyCYwYAa++qtAWiZA6btl/48fDLbfAmjXQvr0Pk3z/+3FXJZL11HFL1X3yCVx7LZx3HtSoATNmQK9eCm2RFFFwS+JC8D1FTjjB98x+4AG/CXn66XFXJpJTNFQiiVmzxodFxo/38ex+/XwbVhFJOXXcsnc7dvj+2A0bwqxZ8PTTvkmUQlskNuq4Zc8WLfJd/N58Ey66yMexjz027qpEcp46bvmuzZv9rMeCAigrg8GDYeRIhbZImlDHLbt6/XVfrv7OO37KeteucPjhcVclIv9CHbe4f/zDh0XOOsvHtSdNggEDFNoiaUjBnetCgCFDfK/s556De++FJUugTZu4KxORPdBQSS778EPo0MEPOWjaFMaO9TMgRSStqePORTt3Qu/e3mVPmACPPw5z5yq0RTKEOu5cs2KFj2XPmgVnn+17ZR9/fNxViUgVqOPOFV9/DX/5CzRpAsuXw/PPe7et0BbJOOq4c8GcOd5lL1sGV18N3bvDEUfEXZWI7Cd13Nnsyy/h9tuhRQvYuBFGj4ZBgxTaIhlOwZ2tRo/2m489e3p4L1sGF14Yd1UikgQK7mzz8cfwq1/BxRf7/tizZ0OPHlCzZtyViUiSKLizRQi+gKZ+fRg+HB5+GObP9y1YRSSr6OZkNli1Cm6+GSZPhjPOgKIiP+xARLKSOu5Mtn07PPYYNGrkW6/26eObRCm0RbKaOu5MNX8+3HgjLFgAl10GzzwDP/pR3FWJSArss+M2s2PNbKqZrTCzZWZ2RyoKkz2oqIB77oFmzeCjj2DoUB/TVmiL5IxEhkq2A3eHEOoDPwc6mFmDaMvKbcXFkJ8P1ar5Y3Fx5S8mTfJhka5d4YYbfPn6FVfEWKmIxGGfwR1C+CiEML/y5y+BFYDau4gUF/s5BmVlPlGkrAw63vQpq1r+Bs45B6pX93Hsvn2hVq24yxWRGFTp5qSZ5QMnA3OjKEagUycfDXGBqxhM6eb6HDej2I8TW7wYWrWKs0QRiVnCwW1mhwJDgTtDCBt38/t2ZlZiZiXl5eXJrDGnrF3rj8eyllFczGCuZg35FFAKnTtDjRrxFigisUsouM3sIDy0i0MIw3b3mhBCUQihIIRQULdu3WTWmFPyj93BbTzNchpwJq9zJ0/SnDl8Ue+kuEsTkTSxz+mAZmZAf2BFCKFb9CXlsKVLmXfQjdRhLuM4j/b0pox88vK82RYRgcQ67hbAtUBrM1tY+e+CiOvKLVu2wAMPwMknU+eLVcy6tZj2x41lreVTr54vhCwsjLtIEUkX++y4QwgzAUtBLblpxgzfK/utt+Daa6FbN1rUqcOannEXJiLpSkve4/LFF3DLLdCyJWzdCuPHwwsvQJ06cVcmImlOwR2HESN8r+xnn4W774alS6Ft27irEpEMoeBOpXXr4Mor4fLLoW5dP1m9a1c45JC4KxORDKLgToWdO727btAAxo6Fv/7Vd/MrKIi7MhHJQNodMGpvveVr2KdPhzPP9Cki//VfcVclIhlMHXdUtm2DRx6Bxo19mXq/fjBlikJbRA6YOu4ozJvne2UvWQK/+AU89RT88IdxVyUiWUIddzJ99RXcdZef8/jZZ/Daa/DKKwptEUkqddzJMm4ctG/v+7DeeqvfgDzssLirEpEspI77QJWXwzXXwAUXQF4ezJwJPXsqtEUkMgru/RUCvPgi1K/vwyEPPujnP7ZoEXdlIpLlNFSyP957D26+GSZOhObNfY52w4ZxVyUiOUIdd1Vs3w7dusGJJ8KcOX6y+syZCm0RSSl13IlauNCn+JWWwkUXQa9ecOyxcVclIjlIHfe+bN4M993ny9Pffx9efhlGjlRoi0hs1HHvzdSpvlz93Xfhd7+Dxx+Hww+PuyoRyXHquHfnH//wYZHWrX32yKRJ0L+/QltE0oKC+1+FAK++6lP8nn8e/ud/fJ+RNm3irkxE5P9pqOQbH3wAHTr4+HXTpr4S8uST465KROQ71HHv3OkzRBo08HnZXbv6AQcKbRFJU7ndcS9f7gf1zp4N55wDffrA8cfHXZWIyF7lZse9dSs89BA0aQIrV8LAgX5Yr0JbRDJA7nXcs2d7l718Ofz61/Dkk3DEEXFXJSKSsNzpuDduhNtug9NPhy+/hDFjoLhYoS0iGSc3gnvUKL/52KsX3H47LFvm27CKiGSg7A7u9evhqqvgkkugdm3fGKpHD6hZM+7KRET2W3YGdwgwYIAvpBkxAh5+2DeHOvXUuCsTETlg2Xdz8t13fa/sKVPgjDOgqAhOOCHuqkREkiZ7Ou5t2+DRR6FRIygp8TnZr7+u0BaRrJMdHXdpqW8KtXAhXH65H3Bw9NFxVyUiEonM7rg3bYI//hGaNYOPP4ahQ2HYMIW2iGS1zO24J070sez33vM9sx99FGrVirsqEZHIZV7H/emncP310LYtHHQQTJsGffsqtEUkZ2ROcIcAgwb5FL9Bg6BTJ1i0CFq2jLsyEZGUyoyhkrIyaN/e98hu1gwmT/bZIyIiOSi9O+4dO3ylY8OGMH06dO/um0QptEUkh6Vvx71kiU/xmzcPzj8feveGevXirkpEJHbp13Fv2QL33+/Hh61e7Tv4jRmj0BYRqZReHfesWfC738Hbb8N118ETT0CdOnFXJSKSVvbZcZvZADPbYGZLoyykuBhuvnwD7739NdceMZ7itgMV2iIiu5FIx/088AzwQlRFFBf7GpqKissZyPls3VCDYe38d4WFUX2qiEhm2mfHHUKYDnwWZRGdOkFFhf+8lRqAP+/UKcpPFRHJTEm7OWlm7cysxMxKysvLq/S3a9dW7bqISC5LWnCHEIpCCAUhhIK6detW6W+PO65q10VEcllaTAfs3Bny8na9lpfn10VEZFdpEdyFhX5QTb16YOaPRUW6MSkisjv7nFViZn8DzgTqmNkHwIMhhP7JLqSwUEEtIpKIfQZ3COHqVBQiIiKJSYuhEhERSZyCW0Qkwyi4RUQyjIJbRCTDKLhFRDKMhRCS/6Zm5UDZfv55HeCTJJaTCfSds1+ufV/Qd66qeiGEhJadRxLcB8LMSkIIBXHXkUr6ztkv174v6DtHSUMlIiIZRsEtIpJh0jG4i+IuIAb6ztkv174v6DtHJu3GuEVEZO/SseMWEZG9SJvgTtWhxOnEzI41s6lmtsLMlpnZHXHXFCUzq2Fm88xsUeX3fSjumlLFzKqb2QIzGx13LalgZmvMbImZLTSzkrjriZqZ1TKzIWa2svK/5+aRfl66DJWYWUvgK+CFEMKJcdeTCmZ2FHBUCGG+mdUESoHLQgjLYy4tEmZmwCEhhK/M7CBgJnBHCOGNmEuLnJn9ASgADgshXBR3PVEzszVAQQghJ+Zxm9lAYEYIoZ+ZHQzkhRA+j+rz0qbjTsWhxOkmhPBRCGF+5c9fAiuAH8VbVXSC+6ry6UGV/9Kjc4iQmR0DXAj0i7sWST4zOwxoCfQHCCF8HWVoQxoFd64zs3zgZGBuvJVEq3LIYCGwAZgYQsjq71upO3AvsDPuQlIoABPMrNTM2sVdTMSOB8qB5yqHw/qZ2SFRfqCCOw2Y2aHAUODOEMLGuOuJUghhRwihCXAM0MzMsnpYzMwuAjaEEErjriXFWoQQmgLnAx0qh0Kz1feApkDvEMLJwCagY5QfqOCOWeVY71CgOIQwLO56UqXyfyVfB86LuZSotQAuqRzzHQy0NrOX4i0peiGEdZWPG4DhQLN4K4rUB8AH//J/j0PwII+MgjtGlTfr+gMrQgjd4q4namZW18xqVf7878DZwMp4q4pWCOG+EMIxIYR84FfAlBDCNTGXFSkzO6TyZjuVQwZtgaydLRZCWA+8b2Y/rbzUBoh0gsE+z5xMlVQdSpxmWgDXAksqx30B/hRCGBtjTVE6ChhoZtXxpuGVEEJOTI/LMUcCw70v4XvAoBDC3+MtKXK3A8WVM0pWA7+N8sPSZjqgiIgkRkMlIiIZRsEtIpJhFNwiIhlGwS0ikmEU3CIiGUbBLSKSYRTcIiIZRsEtIpJh/g/fd/K2NzDSJgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "svd_lr = LinearRegression(method='svd')\n",
    "svd_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' %svd_lr.b_)\n",
    "print('Slope: %.2f' % svd_lr.w_[0])\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, svd_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 4 - Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Iteration: 100/100 | Cost 0.08 | Elapsed: 0:00:00 | ETA: 0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.22\n",
      "Slope: 0.82\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAGzJJREFUeJzt3WmUVOW1xvH/Fo0R4pSAs9BJzEArItrggBcFicNVo7n3uoy214mIMWhi1GskRI1JICpRUQaRNA6YVoMIKjSKDAIyNNBMQjOIKI0ihiaiAo1M/d4Pu53Rrh5OnTpVz28tVnUXRdWuDz5r+579vsdCCIiISHLsFncBIiJSNwpuEZGEUXCLiCSMgltEJGEU3CIiCaPgFhFJGAW3iEjCKLhFRBJGwS0ikjC7R/GmzZs3D3l5eVG8tYhIVpo7d+76EEKLVF4bSXDn5eVRVlYWxVuLiGQlM6tI9bVaKhERSRgFt4hIwii4RUQSRsEtIpIwCm4RkYRRcIuIJIyCW0QkYRTcIiINtXEj3HQTrF2blo9TcIuINERJCeTnw733wksvpeUjFdwiIvVRWQkXXwznnAP77gszZsBll6XloxXcIiJ1EQI8/ji0bg0jRsAdd8C8eXDCCWkrIaWzSsxsFbAR2AnsCCEURFmUiEhGqqiAq6+GcePgxBOhqMiXSdKsLh135xDCMQptEck5O3fC/ffDkUfC9OnQvz9Mm/ZJaBcXQ14e7LabPxYXR1tOJKcDiohkjfJy6NYNZs2Cs86CwYOhZctP/rq4GLp3h6oq/72iwn8HKCyMpqRUO+4AvGRmc82sezSliIhkkK1b4fbboV07WLnSE7qk5HOhDdCr16eh/bGqKn8+Kql23B1DCO+Y2QHAeDNbFkKY+tkX1AR6d4CWX/hiIiKJMmMG/OIXsHQpXHIJ3HcfNG++y5euXr3rt/iq5xtDSh13COGdmsd1wCigwy5eMySEUBBCKGjRIqWbOIiIZJaNG+G66+Dkk2HzZhg71idIviK04UsNeK3PN4Zag9vMmpnZ3h//DJwOLI6uJBGRGIwd6xcfBw708F682Ne0a9G7NzRt+vnnmjb156OSSsd9IDDNzBYCs4GSEMKL0ZUkIpJGlZV+FfHss2HvvX1q5P77/ecUFBbCkCHQqhWY+eOQIdFdmASwEEKjv2lBQUHQPSdFJKOF4Bccr78ePvzQrybecgvsuWcs5ZjZ3FTHrTUOKCK5p6ICrrkGXnjBdzwWFfkySUJoy7uI5I6dO+GBBzykp071JZFp0xIV2qCOW0RyRXm5j/iVlsKZZ/pGmlat4q6qXtRxi0h227oV/vhH30izYoWP940dm9jQBnXcIpLNZs70LnvJEj+CtV8/yIJ9Juq4RST7bNoEv/41dOzom2rGjPEJkiwIbVDHLSLZ5sUX/ejVt96CHj2gT5+UZ7KTQh23iGSH9ev9XJGzzoJmzXxapH//rAttUHCLSNKFAE884XekGT4cbrsN5s+Hk06Ku7LIaKlERJJr9WrfSDN2LBx/vG+kOeqouKuKnDpuEUme6moYMMA3zkye7NMi06fnRGiDOm4RSZolS3zEb+ZMOP10eOghv19YDlHHLSLJsG0b/OlPvpFm+XIYNswnSHIstEEdt4gkQWmpd9nl5XDRRb40csABcVcVG3XcIpK5Nm3yY1dPOgk++ABGj/YJkhwObVDHLSKZatw430hTUfHpRpp99om7qoygjltEMsv69XDppX6C3157+UaaAQMU2p+h4BaRzBACPPkk5Of74623+kaajh3jrizjaKlEROL31lvwq1/5YVAdOsDEidCmTdxVZSx13CISn+pqGDTIN9JMmgT33QczZii0a6GOW0TisWyZj/hNnw4/+YlvpPnud+OuKhHUcYtIem3bBn/5C7RtC0uXwmOP+QSJQjtl6rhFJH1mz/Yue9EiuPBCv1nvgQfGXVXiqOMWkeht3gw33AAnngjvvQfPPw9PPaXQrid13CISrZde8o00q1b5Eax33qmZ7AZSxy0i0fj3v+Hyy+GMM2DPPeGVV3yCRKHdYApuEWlcIcA//+kbaYqLoVcvWLAATj457sqyhpZKRKTxvP22b6QZPRrat4fx4+Hoo+OuKuuo4xaRhquuhgcf9C57wgS45x6/0YFCOxLquEWkYZYtg6uu8sOgunb1jTTf+17cVWU1ddwiUj/bt0Pv3r6RprwcHnnEJ0gU2pFTxy0idTdnDnTr5htpLrgAHngADjoo7qpyhjpuEUnd5s1w441wwgk+7vfsszB8uEI7zdRxi0hqJkyA7t3hzTd9Q81dd8G++8ZdVU5Sxy0iX++99+CKK/wEvz32gClTYPBghXaMFNwismsh+DJI69bw+OPQsycsXAidOsVdWc7TUomIfNmaNb6R5vnn4bjjfFqkbdu4q5Ia6rhF5FPV1b4Mkp/vux779oXSUoV2hlHHLSJu+XK/+Dh1KnTpAkOGwPe/H3dVsgspd9xm1sTM5pvZmCgLEpE0274d+vTxrvrVV2HoUJ8gUWhnrLoslfwGWBpVISLih+nl5cFuu/ljcXHEH1hW5odB9eoF557rtxK78kowi/iDpSFSCm4zOww4GyiKthyR3FVc7CsVFRU+0FFR4b9HEt5VVXDTTXD88bBuHYwaBU8/rY00CZFqx90PuBmojrAWkZzWq5fn6WdVVfnzjWriRGjTxk/w+8UvYMkSOP/8Rv4QiVKtwW1m5wDrQghza3lddzMrM7OyysrKRitQJFesXl235+tswwZfBunaFZo0gcmT/SS//fZrpA+QdEml4+4I/NTMVgFPAV3M7B9ffFEIYUgIoSCEUNCiRYtGLlMk+7VsWbfnUxYCjBjhG2mGDYNbbvGNNKec0sA3lrjUGtwhhJ4hhMNCCHnAz4FJIYRLIq9MJMf07g1Nm37+uaZN/fl6W7MGfvYzP8Hv0EP9VL+//hX22qtBtUq8tAFHJEMUFvrodKtWPtTRqpX/XlhYjzerrvZ/nJ8P48bB3XfDrFnQrl2j1y3pZyGERn/TgoKCUFZW1ujvKyIpeO01H0eZMgU6d/YAP+KIuKuSWpjZ3BBCQSqvVcctki22b4c77/T7PC5YAEVFPkGi0M462vIukg3mzvXRvgUL4L//G/r3h4MPjrsqiYg6bpEkq6qCm2+GDh3gX/+CkSN9gkShndXUcYsk1aRJvpa9cqXfZf3uuzWTnSPUcYskzYYNvixy2mk+fjJpkl+AVGjnDAW3SJI884yP+D36KPzud36aX+fOcVclaaalEpEkeOcduPZaPwyqXTsoKYFjj427KomJOm6RTFZdDX//u3fZL7zgd1afPVuhnePUcYtkqhUr/OLj5Mlw6qm+jv2DH8RdlWQAddwimWbHDu+sjz4a5s/3jnvSJIW2fEIdt0gmmTfPJ0bmz/fDoQYMgEMOibsqyTDquEUywZYtPiXSoQOsXeubaEaOVGjLLqnjFonb5Mm+geb1173bvvtu2H//uKuSDKaOWyQu77/vgd25s0+PTJzo69kKbamFglskDqNG+Yjfww/D//0fLFoEXbrEXZUkhJZKRNJp7VrfSDNyJLRtC6NHw3HHxV2VJIw6bpF0CAGGDvUuu6TEbx82Z45CW+pFHbdI1F5/3TfSvPwydOrk69g//GHcVUmCqeMWicqOHdC3L7Rp4zc6eOghD2+FtjSQOm6RKCxYAN26+Yaa886DgQP9LusijUAdt0hj2rIFevaEggJYswaeftonSBTa0ojUcYs0lilTfC57xQq48kpfJvn2t+OuSrKQOm6RhvrgA7j6aj/Bb+dOmDDBJ0gU2hIRBbdIQzz3nI/4FRXBTTf5RprTTou7KslyCm6R+nj3XbjgAjj/fGjeHGbN8qWRpk3jrkxygIJbpC5CgEce8S579Gjo0wfKyvxipEia6OKkSKreeMM30kycCP/xH76R5kc/irsqyUHquEVqs2MH3HMPHHWUb1MfPNiPYlVoS0zUcYt8nYUL/YzssjL46U9h0CDNZEvs1HGL7MpHH0GvXr52vXo1DB8Ozz6r0JaMoI5b5IteecU30ixfDpdf7sskmsmWDKKOW+RjH34I11zjJ/ht2wbjx/sEiUJbMoyCWwR8tC8/H4YMgRtu8I00XbvGXZXILim4Jbf961/w85/7hcfvfAdKS31ppFmzuCsT+UoKbslNIcBjj0Hr1n5631/+4pMj7dvHXZlIrXRxUnLPm2/6oVDjx8PJJ/tGmh//OO6qRFKmjltyx86dcN99vpGmtNRnsqdMUWhL4qjjltzw6qu+kWbOHDj3XA/tww6LuyqRelHHLdnto4/g1lv9buqrVsFTT/lRrAptSbBag9vMvmlms81soZmVm9kd6ShMpMGmTYN27fzCY2EhLF0KF14IZp+8pLgY8vJgt938sbg4tmpFUpZKx70V6BJCaAscA5xpZidEW5ZIA3z4IfTo4Sf4ffQRjBsHjz7q436fUVzsh/1VVPiQSUWF/67wlkxXa3AHt6nm1z1q/oRIqxKprzFj4Mgj/QS/3/4WFi+G00/f5Ut79YKqqs8/V1Xlz4tkspTWuM2siZktANYB40MIs3bxmu5mVmZmZZWVlY1dp8jXW7cOLrrILzzutx/MnAn33vu1G2lWr67b8yKZIqXgDiHsDCEcAxwGdDCzo3bxmiEhhIIQQkGLFi0au06RXQsBhg3zjTQjR8Kf/wxz50KHDrX+05Yt6/a8SKao01RJCOF9YDJwZiTViNTFqlVw5plw2WUe3AsWwB/+AN/4Rkr/vHfvL98ismlTf14kk6UyVdLCzPar+XkvoCuwLOrCRL7Szp3Qr5+vZc+YAQMHwtSpHt51UFjoZ0q1auWDJq1a+e+FhRHVLdJIUtmAczDwmJk1wYN+eAhhTLRliXyFRYt8I83s2XD22fDgg3D44fV+u8JCBbUkT63BHUJ4FWiXhlpEvtrWrb6G8de/wv77w5NPfmkmWyRXaMu7ZL7p073LXrYMLr3Up0W+MJMtkku05V0y18aNcO21vpFmyxZ48UU/ilWhLTlOwS2ZqaTE70gzaBD8+te+keaMM+KuSiQjKLgls1RWwsUXwznnwL77+tRIv37wrW/FXZlIxlBwS2YIAR5/3Ef6RoyAO+6AefPgBB2LI/JFujgp8auo8DvSjBsHJ54IRUW+TCIiu6SOW+Kzcyfcf79vpJk+Hfr396NYFdoiX0sdt8SjvBy6dYNZs+Css/w0Px0SIpISddySXlu3wu23+w0OVq70w69LShTaInWgjlvSZ8YM30izdClcconfuLd587irEkkcddwSvY0b4brr4OSTYfNmGDvWJ0gU2iL1ouCWaI0d6xcfBw708F682Ne0RaTetFQi0aishOuvhyee8CmR6dN91E9EGkwdtzSuEOAf//CNNE8/DX/8o2+kUWiLNBp13NJ4KirgmmvghRd8x2NRkS+TiEijUsctDbdzJzzwgIf01Kn+87RpCm2RiKjjloYpL/cRv9JSv//j4MF+DzARiYw6bqmfrVt9/bpdO1ixwte1x45VaIukgTpuqbuZM73LXrLEb9h4333QokXcVYnkDHXckrpNm/ymBh07+qaakhLvtBXaImml4JbUvPiiX2wcMMBvJ1ZeDv/5n3FXJZKTFNzy9dav93NFzjoLmjXzaZEHHoC99467MpGcpeCWXQvBdz22bg3Dh8Ntt8H8+XDSSXFXJpLzdHFSvmz1at9IM3YsHH+8b6Q56qi4qxKRGuq45VPV1b6GfeSRMHmy36R3+nSFtkiGUcctbskSH/GbORNOPx0eegjy8uKuSkR2QR13rtu2Df70J99Is3w5DBvmEyQKbZGMpY47l5WWepddXg4XXeRLIwccEHdVIlILddy5aNMmPyv7pJPggw9g9GifIFFoiySCOu5cM24cXH21H8Haowf06QP77BN3VSJSB+q4c8X69XDppX6C3157+UaaAQMU2iIJpODOdiHAk0/67cOefBJuvdU30nTsGHdlIlJPWirJZm+95RtpSkqgQweYOBHatIm7KhFpIHXc2ai62u+qnp8PL7/sx67OmKHQFskS6rizzdKlPuI3Ywb85Ce+kea73427KhFpROq4s8W2bfDnP8Mxx8CyZfDYYz5BotAWyTrquLPBrFneZS9eDBdeCPffDwceGHdVIhIRddxJtnkz/Pa3cOKJsGEDPP88PPWUQlsky9Ua3GZ2uJm9bGZLzazczH6TjsKkFi+95Kf29esHv/ylHxJ17rlxVyUiaZBKx70DuDGE0Bo4AehhZvnRlpXbiov9jKfddvPH4uLP/OW//w2XXQZnnAF77gmvvAKDBmkjjUgOqXWNO4SwFlhb8/NGM1sKHAosibi2nFRcDN27Q1WV/15R4b8TAoW7/9Nv1rthA/TqBX/4A3zzm7HWKyLpV6eLk2aWB7QDZkVRjHgefxzaH9u/6m0O7H4NbBkD7dvDhAlw9NHxFCgisUv54qSZfQt4Brg+hPDhLv6+u5mVmVlZZWVlY9aYU1av/vRno5prGMQS8jlpy0S45x6/0YFCWySnpRTcZrYHHtrFIYSRu3pNCGFICKEghFDQokWLxqwxp7Rs6Y8/YhlTOIVB9GAWx3PGIYvhhhugSZN4CxSR2KUyVWLAUGBpCOHe6EvKbX3u2M7te/RmIW05knIu5xHO3+slfnn39+IuTUQyRCpr3B2B/wUWmdmCmud+H0IYG11ZOWrOHC6+pxtsX8SYphdwVdUD7NnqIIb0hsLCuIsTkUyRylTJNMDSUEvu2rwZbrvNZ7IPOgiefZZzzjvPR3lERL5AW97jNmGCz/u9+abfmeauu2DffeOuSkQymLa8x+W99+CKK/wEvz32gClTYPBghbaI1ErBnW4hwPDh0Lo1PP449OwJCxdCp05xVyYiCaGlknRaswZ+9Ss/DOq44/y8kbZt465KRBJGHXc6VFf7Mkh+PowfD337QmmpQltE6kUdd9SWL/eLj1OnQpcuMGQIfP/7cVclIgmmjjsq27dDnz7eVb/6Kgwd6hMkCm0RaSB13FEoK/M70ixcCP/zP9C/v89ni4g0AnXcjamqCm66CY4/Htatg1Gj4OmnFdoi0qjUcTeWiRN9LfuNN/zxrrtgv/3irkpEspA67obasAGuvBK6dvWT+yZPhoceUmiLSGQU3PUVAowY4Rtphg2DW27xNe1TTom7MhHJcloqqY81a6BHD3juOTj2WHjhBWjXLu6qRCRHqOOui+pqn8POz4dx4+Duu2HWLIW2iKSVOu5UvfaaX3ScMgU6d/YAP+KIuKsSkRykjrs227fDnXf6fR4XLIC//90nSBTaIhITddxfZ9486NbNA/u//gsGDICDD467KhHJceq4d6WqCm6+GTp0gHffhWee8T8KbRHJAOq4v2jSJF/LXrnSt6337auZbBHJKOq4P7Zhgwf1aaf575Mm+Xq2QltEMoyCG3wZJD8fHn3Ul0gWLfLJERGRDJTbSyXvvAPXXuuHQR1zDJSU+IYaEZEMlpsddwi+DJKf77se77wTZs9WaItIIuRex/3663DVVX4Y1Kmn+kaaH/wg7qpERFKWOx33jh1+1GqbNjB/vgf2xIkKbRFJnNzouOfP94008+fDz37mG2kOOSTuqkRE6iW7O+4tW+B3v4P27WHtWj+GdeRIhbaIJFr2dtyTJ/ta9uuve7fdty/sv3/cVYmINFj2ddzvv++B3bmzH8M6cSIUFSm0RSRrZFdwjxrlI34PP+w37V20CLp0ibsqEZFGlR1LJWvX+kaakSOhbVsYPRqOOy7uqkREIpHsjjsEGDrUu+ySEujTB+bMUWiLSFZLbse9cqWf4jdpEnTq5Dshf/jDuKsSEYlc8jruHTt8QqRNGygrg8GD4eWXFdoikjOS1XEvWOBHr86dC+edBwMHwqGHxl2ViEhaJaPj3rIFevaEggJ46y0YPtwnSBTaIpKDMr/jnjLF57JXrIArroC//Q2+/e24qxIRiU3mdtwffABXX+0n+O3YAePH+3y2QltEclxmBvdzz/mIX1ER3Hijb6Tp2jXuqkREMkKtwW1mD5vZOjNbHGUhxcXQ/vB3edougPPPZ8PuzaG01JdGmjWL8qNFRBIllY77UeDMKIsoLvaR7I5vP8W5jOb39Cavsozi19pH+bEiIolkIYTaX2SWB4wJIRyVypsWFBSEsrKylIvIy4OKCmjCDvJYxUqOAKBVK1i1KuW3ERFJLDObG0IoSOW1jbbGbWbdzazMzMoqKyvr9G9Xr/bHnez+SWh/9nkREflUowV3CGFICKEghFDQokWLOv3bli3r9ryISC7LiKmS3r2hadPPP9e0qT8vIiKflxHBXVjo9+5t1QrM/HHIEH9eREQ+r9adk2b2JHAq0NzM3gZuDyEMbexCCgsV1CIiqag1uEMIF6WjEBERSU1GLJWIiEjqFNwiIgmj4BYRSRgFt4hIwii4RUQSJqWzSur8pmaVQEU9/3lzYH0jlpME+s7ZL9e+L+g711WrEEJK284jCe6GMLOyVA9ayRb6ztkv174v6DtHSUslIiIJo+AWEUmYTAzuIXEXEAN95+yXa98X9J0jk3Fr3CIi8vUyseMWEZGvkTHBna6bEmcSMzvczF42s6VmVm5mv4m7piiZ2TfNbLaZLaz5vnfEXVO6mFkTM5tvZmPiriUdzGyVmS0yswVmlvp9DBPKzPYzsxFmtqzmv+cTI/28TFkqMbNOwCZgWKr3tkw6MzsYODiEMM/M9gbmAueHEJbEXFokzMyAZiGETWa2BzAN+E0IoTTm0iJnZjcABcA+IYRz4q4nama2CigIIeTEHLeZPQa8EkIoMrNvAE1DCO9H9XkZ03GHEKYC78VdRzqFENaGEObV/LwRWAocGm9V0QluU82ve9T8yYzOIUJmdhhwNlAUdy3S+MxsH6ATMBQghLAtytCGDAruXGdmeUA7YFa8lUSrZslgAbAOGB9CyOrvW6MfcDNQHXchaRSAl8xsrpl1j7uYiH0PqAQeqVkOKzKzZlF+oII7A5jZt4BngOtDCB/GXU+UQgg7QwjHAIcBHcwsq5fFzOwcYF0IYW7ctaRZxxDCscBZQI+apdBstTtwLPBgCKEdsBm4JcoPVHDHrGat9xmgOIQwMu560qXmfyUnA2fGXErUOgI/rVnzfQroYmb/iLek6IUQ3ql5XAeMAjrEW1Gk3gbe/sz/PY7AgzwyCu4Y1VysGwosDSHcG3c9UTOzFma2X83PewFdgWXxVhWtEELPEMJhIYQ84OfApBDCJTGXFSkza1ZzsZ2aJYPTgaydFgshvAu8ZWY/qnnqNCDSAYNa7zmZLum6KXGG6Qj8L7CoZt0X4PchhLEx1hSlg4HHzKwJ3jQMDyHkxHhcjjkQGOV9CbsDT4QQXoy3pMhdBxTXTJS8AVwR5YdlzDigiIikRkslIiIJo+AWEUkYBbeISMIouEVEEkbBLSKSMApuEZGEUXCLiCSMgltEJGH+H+yYASXcfDjCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "gd_lr = LinearRegression(method='sgd',\n",
    "                         eta=0.005, \n",
    "                         epochs=100,\n",
    "                         minibatches=1,\n",
    "                         random_seed=123,\n",
    "                         print_progress=3)\n",
    "gd_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % gd_lr.b_)\n",
    "print('Slope: %.2f' % gd_lr.w_)\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, gd_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X+UHWWd5/H3p3+k88sEElqF/CBxyCwTlR/SRNSVdfBXWJWwMyBhUdBlNytnctRxdYR11WNGzyzHPaKcYRmiIKBIRFY0q2iGAWR3ZgDTgUgIbKSJGWgSoTEBgmCSTr77Rz2drr65nb43dN1b6f68Dvd01VNVTz1Vfcmnq+5zn1JEYGZmVjYtzW6AmZlZNQ4oMzMrJQeUmZmVkgPKzMxKyQFlZmal5IAyM7NSKjSgJC2WtElSj6RLqyz/lKRHJD0k6U5Jx+aWXSTpsfS6KFd+iqQNqc4rJanIYzAzs+ZQUd+DktQK/Bp4N9ALrAXOj4hHcuv8KXB/RLwk6RLgHRFxnqQZQDfQBQSwDjglInZI+iXwCeA+4Hbgyoj4WSEHYWZmTVPkFdQioCciNkfEbmAVsCS/QkTcHREvpdn7gNlp+r3AHRGxPSJ2AHcAiyUdDUyLiHsjS9YbgbMLPAYzM2uStgLrngU8mZvvBd58kPUvBgauhKptOyu9equUH0DSMmAZwJQpU045/vjj62n7qNn09E4mtbcyd8bkpuzfzKxs1q1b92xEdI60XpEBVe2zoar3EyV9iOx23r8ZYdua64yIlcBKgK6uruju7h6pvYV419fuYcGrp3L1h05pyv7NzMpG0r/Usl6Rt/h6gTm5+dnA1sqVJL0L+BxwVkTsGmHbXgZvAw5bZ5m0tYg9ez3eoZlZvYoMqLXAAknzJU0AlgKr8ytIOhm4hiycnsktWgO8R9KRko4E3gOsiYhtwE5Jp6XeexcCPy7wGF6x9tYW+vfta3YzzMwOO4Xd4ouIfknLycKmFbguIjZKWgF0R8Rq4KvAVOAHqbf4ExFxVkRsl/TXZCEHsCIitqfpS4DrgUlkn1mVugdfW6vo9xWUmVndivwMioi4nawreL7sC7npdx1k2+uA66qUdwNvGMVmFqq9pYU9e30FZWZWL48kUbC2VtG/z1dQZmb1ckAVrK21hX5fQZmZ1c0BVbB29+IzMzskDqiCZbf4fAVlZlYvB1TBslt8voIyM6uXA6pg7S1ij6+gzMzq5oAqmK+gzMwOjQOqYO2t7iRhZnYoHFAFa2vxUEdmZofCAVUwD3VkZnZoHFAFa2/1UEdmZofCAVWwthYPdWRmdigcUAVra21h774ge0K9mZnVygFVsPaW7CHA7slnZlYfB1TB2lqzU+yefGZm9XFAFay91VdQZmaHwgFVsLZ0i8+P3DAzq48DqmCDt/h8BWVmVg8HVMEGb/H5CsrMrB4OqIK1taQrKH8GZWZWl0IDStJiSZsk9Ui6tMry0yU9IKlf0jm58j+VtD73+oOks9Oy6yX9JrfspCKP4ZVqS1dQ7sVnZlaftqIqltQKXAW8G+gF1kpaHRGP5FZ7AvgI8On8thFxN3BSqmcG0AP8fW6Vz0TErUW1fTS1p8+g3IvPzKw+hQUUsAjoiYjNAJJWAUuA/QEVEVvSsoNdXpwD/CwiXiquqcUZ7MXngDIzq0eRt/hmAU/m5ntTWb2WAjdXlH1F0kOSrpDUUW0jScskdUvq7uvrO4Tdjo72tuwU73YnCTOzuhQZUKpSVtdlhKSjgTcCa3LFlwHHA6cCM4DPVts2IlZGRFdEdHV2dtaz21E1ub0VgJd3721aG8zMDkdFBlQvMCc3PxvYWmcdHwRui4g9AwURsS0yu4Bvk91KLK0pHdld1Bd39Te5JWZmh5ciA2otsEDSfEkTyG7Vra6zjvOpuL2XrqqQJOBs4OFRaGthpqaA+r0DysysLoUFVET0A8vJbs89CtwSERslrZB0FoCkUyX1AucC10jaOLC9pHlkV2D3VFR9k6QNwAbgKODLRR3DaJg6MQXUbgeUmVk9iuzFR0TcDtxeUfaF3PRaslt/1bbdQpVOFRFxxui2slhTfYvPzOyQeCSJgnW0tdDaIt/iMzOrkwOqYJKYMqGV3+9yLz4zs3o4oBpgakebb/GZmdXJAdUAUzrafIvPzKxODqgGmOIrKDOzujmgGmCqr6DMzOrmgGqAKR3uJGFmVi8HVAP4Fp+ZWf0cUA3gXnxmZvVzQDXAQC++CD8TysysVg6oBpja0Ub/vmBXv58JZWZWKwdUA0yZkD0Tyj35zMxq54BqgCn7H7nhnnxmZrVyQDWARzQ3M6ufA6oB9l9B+ZlQZmY1c0A1gB/7bmZWPwdUA/ix72Zm9XNANcCUDvfiMzOrlwOqAQY7SbgXn5lZrRxQDTDFt/jMzOpWaEBJWixpk6QeSZdWWX66pAck9Us6p2LZXknr02t1rny+pPslPSbp+5ImFHkMo6G9tYUJbS0OKDOzOhQWUJJagauAM4GFwPmSFlas9gTwEeB7Vap4OSJOSq+zcuWXA1dExAJgB3DxqDe+AB4w1sysPkVeQS0CeiJic0TsBlYBS/IrRMSWiHgIqGmQOkkCzgBuTUU3AGePXpOLkz0TygFlZlarIgNqFvBkbr43ldVqoqRuSfdJGgihmcBzETHwL/2wdUpalrbv7uvrq7fto27KhDZ3kjAzq0NbgXWrSlk9z5uYGxFbJb0OuEvSBuCFWuuMiJXASoCurq6mP+fCj303M6tPkVdQvcCc3PxsYGutG0fE1vRzM/AL4GTgWeAISQPBWledzTSlo81DHZmZ1aHIgFoLLEi97iYAS4HVI2wDgKQjJXWk6aOAtwGPRPbEv7uBgR5/FwE/HvWWF8CdJMzM6lNYQKXPiZYDa4BHgVsiYqOkFZLOApB0qqRe4FzgGkkb0+Z/AnRL+hVZIP33iHgkLfss8ClJPWSfSV1b1DGMJneSMDOrT5GfQRERtwO3V5R9ITe9luw2XeV2/wy8cZg6N5P1EDysZI99dycJM7NaeSSJBpmaPoPK7lKamdlIHFANMrWjjQh4abevoszMauGAahCPx2dmVh8HVIP4se9mZvVxQDXI4BWUb/GZmdXCAdUgAw8t9BWUmVltHFAN4se+m5nVxwHVIPtv8Xm4IzOzmjigGsSdJMzM6uOAapCBK6gX/+CAMjOrhQOqQSa3Z50k/BmUmVltHFAN0tIipkxo9UMLzcxq5IBqoCl+aKGZWc0cUA00taONF92Lz8ysJg6oBvIVlJlZ7RxQDeSHFpqZ1c4B1UDZY9/dScLMrBYOqAbyLT4zs9o5oBrIAWVmVjsHVANlt/gcUGZmtSg0oCQtlrRJUo+kS6ssP13SA5L6JZ2TKz9J0r2SNkp6SNJ5uWXXS/qNpPXpdVKRxzCapkxoY1f/Pvr37mt2U8zMSq+tqIoltQJXAe8GeoG1klZHxCO51Z4APgJ8umLzl4ALI+IxSccA6yStiYjn0vLPRMStRbW9KAPPhPr9rr1Mn+yLVzOzgyksoIBFQE9EbAaQtApYAuwPqIjYkpYNuaSIiF/nprdKegboBJ7jMLZ/RPPd/Uyf3N7k1piZlVuRf8bPAp7MzfemsrpIWgRMAB7PFX8l3fq7QlLHMNstk9Qtqbuvr6/e3RZiih9aaGZWsyIDSlXKoq4KpKOB7wAfjYiBq6zLgOOBU4EZwGerbRsRKyOiKyK6Ojs769ltYfxMKDOz2hUZUL3AnNz8bGBrrRtLmgb8FPhvEXHfQHlEbIvMLuDbZLcSDwu+gjIzq12RAbUWWCBpvqQJwFJgdS0bpvVvA26MiB9ULDs6/RRwNvDwqLa6QIOdJBxQZmYjKSygIqIfWA6sAR4FbomIjZJWSDoLQNKpknqBc4FrJG1Mm38QOB34SJXu5DdJ2gBsAI4CvlzUMYy2wVt8Hu7IzGwkRfbiIyJuB26vKPtCbnot2a2/yu2+C3x3mDrPGOVmNoxv8ZmZ1c5fxmkgd5IwM6udA6qBOtpamNrRRt/OXc1uiplZ6TmgGkgSs4+cxJPbX2p2U8zMSs8B1WBzZ0zmCQeUmdmIHFANNhBQEXV9Z9nMbNxxQDXY3JmT2dW/z59DmZmNwAHVYHNmTAbwbT4zsxE4oBpsrgPKzKwmDqgGm3XEJCQHlJnZSBxQDTaxvZXXTpvogDIzG4EDqgnmzJjs70KZmY3AAdUE/i6UmdnIHFBNMHfGZJ5+YRd/2ONRzc3MhuOAaoKBnny9O3wVZWY2HAdUE/i7UGZmI6spoCR9p5Yyq82cGZMAeOJ3Digzs+HUegX1+vyMpFbglNFvzvjQObWDie0tPLH95WY3xcystA4aUJIuk7QTOEHSC+m1E3gG+HFDWjgGSXJPPjOzERw0oCLibyLiVcBXI2Jaer0qImZGxGUNauOYNHfGZHeSMDM7iFpv8f1E0hQASR+S9DVJxxbYrjFvjh+7YWZ2ULUG1NXAS5JOBP4K+BfgxpE2krRY0iZJPZIurbL8dEkPSOqXdE7FsoskPZZeF+XKT5G0IdV5pSTVeAylMnfGZF7avZff/X53s5tiZlZKtQZUf2R/6i8BvhER3wBedbANUkeKq4AzgYXA+ZIWVqz2BPAR4HsV284Avgi8GVgEfFHSkWnx1cAyYEF6La7xGErFo5qbmR1crQG1U9JlwIeBn6bwaR9hm0VAT0RsjojdwCqygNsvIrZExEPAvopt3wvcERHbI2IHcAewWNLRwLSIuDcF5o3A2TUeQ6kMBJTH5DMzq67WgDoP2AX8h4j4LTAL+OoI28wCnszN96ayWgy37aw0PWKdkpZJ6pbU3dfXV+NuG2f2kekKyt+FMjOrqqaASqF0EzBd0vuBP0TESJ9BVftsqNYeAcNtW3OdEbEyIroioquzs7PG3TbOpAmtvPpVHb7FZ2Y2jFpHkvgg8EvgXOCDwP2VnRqq6AXm5OZnA1trbNdw2/am6UOps3TmzpjM5md/3+xmmJmVUq23+D4HnBoRF0XEhWSfL31+hG3WAgskzZc0AVgKrK5xf2uA90g6MnWOeA+wJiK2kX0edlrqvXchh/EXhk973UwefGIHfTt3NbspZmalU2tAtUTEM7n53420bUT0A8vJwuZR4JaI2ChphaSzACSdKqmX7MrsGkkb07bbgb8mC7m1wIpUBnAJ8C2gB3gc+FmNx1A6HzjxGPYF/Ozhbc1uiplZ6aiWL4pK+ipwAnBzKjoPeCgiPltg20ZNV1dXdHd3N7sZVb3ninuYPqmdH3zsrc1uiplZQ0haFxFdI6030lh8x0l6W0R8BriGLKROBO4FVo5KS8e5D5xwDGu37GDrcx441swsb6RbfF8HdgJExA8j4lMR8ZfA7WmZvUIfOPEYAH7y0GHb18PMrBAjBdS89EXaISKiG5hXSIvGmXlHTeGE2dP537/y51BmZnkjBdTEgyybNJoNGc8+cMIxbHjqeX7jLudmZvuNFFBrJf2nykJJFwPrimnS+PO+E44G4Ce/8m0+M7MBbSMs/yRwm6QLGAykLmAC8O+KbNh4cswRkzh13pGs/tVWlp9xHIfpAO1mZqNqpO8yPR0RbwW+BGxJry9FxFvS8Ec2Ss49ZQ6PPfMiX/+Hx5rdFDOzUhjpCgqAiLgbuLvgtoxr53bN5pdbtvONOx/jtdMncv6iuc1ukplZU9UUUFY8SfzNn72RZ1/cxedu20Dn1A7etfA1zW6WmVnT1DrUkTVAe2sLV/37N/GGWdNZfvMD/N09j/PcS37irpmNTzUNdXS4K/NQR9U8++IuPrlqPf/Y8ywT21v4szfNZvHrX8sfv+ZVvGZahztRmNlhrdahjhxQJfbothe4/p+28KP1T7GrP3vo8LSJbRw7cwpHTG5n2qR2pk1sZ2J7Cx1trUxoa6G9RbS0iNYW0apsukXQIiFltxIF2TTZsoHp9N/QddK25MoH6gJy66nqtsPWS2pPqiQ/L+WnsxV0kDoG5MtaRqhjSNtz9VQ7R4N1V29rZV0DBSO1ubL+wXry59B/jNjY44DKOVwDasDzL+/hka0v8Ound/Lrp3fSu+Nlnn95Dy+8vIcX/rCHXXv2sWvvPnanELOxZ39AMjTAoPKPgMGyaqFHZbBWhCj766gIZoYGbL4tQ9pYLdTz+6tow9B9HvhHR+XxDnd85Os/yL6HnLNcW6q1odofQfn25LcZuu8D93ew8zlk3xV/5AzWc+Dx5WuuXE61uoccw9BjHVpnxe+0yjanHHskZ59c6wPSD1RrQLmTxGFg+qR23vJHM3nLH8086HoRwd59wd4I+vcG+yLYF4PlAUSaH5jet386GPhbJQKCSD+zZfsCSGX7BtYbWCeGmWZwX+T2Ua0dQbYghtl/2n3FfrJlaVFq/+B2+yIOXK9q24fZx/52V5Tl2k1u28G6h+5v6LEMzud/b4PrDr+/yvryxz1w/qrvf/j9Dtk+t/7gucqVV9l3vk0M2TYq6ql+nocsr1aWa0fV81uxbv58Dq1n6Pv7gLKB8oBgX9X3V+X5yZ/MA4+12v6Gb1tl3UP2e7DjG66e3O+52jaD+zvwfA9tX/Xlba16RQFVKwfUGCKJtlbRBnT4N2tmhzn34jMzs1JyQJmZWSk5oMzMrJQcUGZmVkoOKDMzK6VCA0rSYkmbJPVIurTK8g5J30/L75c0L5VfIGl97rVP0klp2S9SnQPLXl3kMZiZWXMUFlCSWoGrgDOBhcD5khZWrHYxsCMijgOuAC4HiIibIuKkiDgJ+DCwJSLW57a7YGB5RDxT1DGYmVnzFHkFtQjoiYjNEbEbWAUsqVhnCXBDmr4VeKcOHNvlfODmAttpZmYlVGRAzQKezM33prKq60REP/A8UDlcwnkcGFDfTrf3Pl8l0ACQtExSt6Tuvr6+Qz0GMzNrkiIDqlpwRD3rSHoz8FJEPJxbfkFEvBF4e3p9uNrOI2JlRHRFRFdnZ2d9LTczs6YrMqB6gTm5+dnA1uHWkdQGTAe255YvpeLqKSKeSj93At8ju5VoZmZjTJEBtRZYIGm+pAlkYbO6Yp3VwEVp+hzgrkijGkpqAc4l++yKVNYm6ag03Q68H3gYMzMbcwobUjQi+iUtB9YArcB1EbFR0gqgOyJWA9cC35HUQ3bltDRXxelAb0RszpV1AGtSOLUC/wB8s6hjMDOz5vHzoMzMrKFqfR6UR5IwM7NSckCZmVkpOaDMzKyUHFBmZlZKDigzMyslB5SZmZWSA8rMzErJAWVmZqXkgDIzs1JyQJmZWSk5oMzMrJQcUGZmVkoOKDMzKyUHlJmZlZIDyszMSskBZWZmpeSAMjOzUnJAmZlZKTmgzMyslBxQZmZWSoUGlKTFkjZJ6pF0aZXlHZK+n5bfL2leKp8n6WVJ69Pr73LbnCJpQ9rmSkkq8hjMzKw5CgsoSa3AVcCZwELgfEkLK1a7GNgREccBVwCX55Y9HhEnpdfHcuVXA8uABem1uKhjMDOz5inyCmoR0BMRmyNiN7AKWFKxzhLghjR9K/DOg10RSToamBYR90ZEADcCZ49+083MrNmKDKhZwJO5+d5UVnWdiOgHngdmpmXzJT0o6R5Jb8+t3ztCnQBIWiapW1J3X1/fKzsSMzNruCIDqtqVUNS4zjZgbkScDHwK+J6kaTXWmRVGrIyIrojo6uzsrKPZZmZWBkUGVC8wJzc/G9g63DqS2oDpwPaI2BURvwOIiHXA48Afp/Vnj1CnmZmNAUUG1FpggaT5kiYAS4HVFeusBi5K0+cAd0VESOpMnSyQ9DqyzhCbI2IbsFPSaemzqguBHxd4DGZm1iRtRVUcEf2SlgNrgFbguojYKGkF0B0Rq4Frge9I6gG2k4UYwOnACkn9wF7gYxGxPS27BLgemAT8LL3MzGyMUdYZbmzr6uqK7u7uZjfDzMwASesiomuk9TyShJmZlZIDyszMSskBZWZmpeSAMjOzUnJAmZlZKTmgzMyslBxQZmZWSg4oMzMrJQeUmZmVkgPKzMxKyQFlZmal5IAyM7NSckCZmVkpOaDMzKyUHFBmZlZKDigzMyslB5SZmZWSA8rMzErJAWVmZqXkgDIzs1IqNKAkLZa0SVKPpEurLO+Q9P20/H5J81L5uyWtk7Qh/Twjt80vUp3r0+vVRR6DmZk1R1tRFUtqBa4C3g30AmslrY6IR3KrXQzsiIjjJC0FLgfOA54FPhARWyW9AVgDzMptd0FEdBfVdjMza74ir6AWAT0RsTkidgOrgCUV6ywBbkjTtwLvlKSIeDAitqbyjcBESR0FttXMzEqmyICaBTyZm+9l6FXQkHUioh94HphZsc6fAw9GxK5c2bfT7b3PS1K1nUtaJqlbUndfX98rOQ4zM2uCIgOqWnBEPetIej3Zbb//nFt+QUS8EXh7en242s4jYmVEdEVEV2dnZ10NNzOz5isyoHqBObn52cDW4daR1AZMB7an+dnAbcCFEfH4wAYR8VT6uRP4HtmtRDMzG2OKDKi1wAJJ8yVNAJYCqyvWWQ1clKbPAe6KiJB0BPBT4LKI+KeBlSW1SToqTbcD7wceLvAYzMysSQoLqPSZ0nKyHniPArdExEZJKySdlVa7FpgpqQf4FDDQFX05cBzw+Yru5B3AGkkPAeuBp4BvFnUMZmbWPIqo/Fho7Onq6orubvdKNzMrA0nrIqJrpPU8koSZmZWSA8rMzErJAWVmZqXkgDIzs1JyQJmZWSk5oMzMrJQcUGZmVkoOKDMzKyUHlJmZlZIDyszMSskBZWZmpeSAMjOzUnJAmZlZKTmgzMyslBxQZmZWSg4oMzMrJQeUmZmVkgPKzMxKyQFlZmal5IAyM7NSKjSgJC2WtElSj6RLqyzvkPT9tPx+SfNyyy5L5ZskvbfWOs3MbGwoLKAktQJXAWcCC4HzJS2sWO1iYEdEHAdcAVyetl0ILAVeDywG/qek1hrrNDOzMaDIK6hFQE9EbI6I3cAqYEnFOkuAG9L0rcA7JSmVr4qIXRHxG6An1VdLnWZmNga0FVj3LODJ3Hwv8Obh1omIfknPAzNT+X0V285K0yPVCYCkZcCyNPuipE11tv8o4Nk6txlrfA4yPg8ZnwefgwGv9DwcW8tKRQaUqpRFjesMV17tiq+yzqwwYiWw8mANPBhJ3RHRdajbjwU+Bxmfh4zPg8/BgEadhyJv8fUCc3Lzs4Gtw60jqQ2YDmw/yLa11GlmZmNAkQG1Flggab6kCWSdHlZXrLMauChNnwPcFRGRypemXn7zgQXAL2us08zMxoDCbvGlz5SWA2uAVuC6iNgoaQXQHRGrgWuB70jqIbtyWpq23SjpFuARoB/4i4jYC1CtzoIO4ZBvD44hPgcZn4eMz4PPwYCGnAdlFyxmZmbl4pEkzMyslBxQZmZWSg6oCuN1KCVJcyTdLelRSRslfSKVz5B0h6TH0s8jm93WoqVRSx6U9JM0Pz8NxfVYGpprQrPbWDRJR0i6VdL/S++Jt4zT98Jfpv8fHpZ0s6SJ4+H9IOk6Sc9IejhXVvX3r8yV6d/MhyS9abTa4YDKGedDKfUD/yUi/gQ4DfiLdOyXAndGxALgzjQ/1n0CeDQ3fzlwRToHO8iG6BrrvgH8PCKOB04kOx/j6r0gaRbwcaArIt5A1jFrKePj/XA92TBzecP9/s8k62m9gGxwhKtHqxEOqKHG7VBKEbEtIh5I0zvJ/kGaxdDhqG4Azm5OCxtD0mzgfcC30ryAM8iG4oLxcQ6mAaeT9bIlInZHxHOMs/dC0gZMSt/TnAxsYxy8HyLi/5D1rM4b7ve/BLgxMvcBR0g6ejTa4YAaqtrwTLOGWXfMSqPKnwzcD7wmIrZBFmLAq5vXsob4OvBXwL40PxN4LiL60/x4eE+8DugDvp1udX5L0hTG2XshIp4C/gfwBFkwPQ+sY/y9HwYM9/sv7N9NB9RQtQzPNKZJmgr8L+CTEfFCs9vTSJLeDzwTEevyxVVWHevviTbgTcDVEXEy8HvG+O28atJnLEuA+cAxwBSy21mVxvr7YSSF/T/igBpqXA+lJKmdLJxuiogfpuKnBy7X089nmtW+BngbcJakLWS3d88gu6I6It3igfHxnugFeiPi/jR/K1lgjaf3AsC7gN9ERF9E7AF+CLyV8fd+GDDc77+wfzcdUEON26GU0mct1wKPRsTXcovyw1FdBPy40W1rlIi4LCJmR8Q8st/9XRFxAXA32VBcMMbPAUBE/BZ4UtK/SkXvJBvVZdy8F5IngNMkTU7/fwych3H1fsgZ7ve/Grgw9eY7DXh+4FbgK+WRJCpI+rdkfzUPDKX0lSY3qSEk/Wvg/wIbGPz85b+SfQ51CzCX7H/YcyOi8sPTMUfSO4BPR8T7Jb2O7IpqBvAg8KGI2NXM9hVN0klkHUUmAJuBj5L9QTuu3guSvgScR9bL9UHgP5J9vjKm3w+SbgbeQfZYjaeBLwI/osrvP4X335L1+nsJ+GhEdI9KOxxQZmZWRr7FZ2ZmpeSAMjOzUnJAmZlZKTmgzMyslBxQZmZWSg4oswaRtFfS+txr1EZnkDQvP/K02VhQ2CPfzewAL0fESc1uhNnhwldQZk0maYukyyX9Mr2OS+XHSrozPWPnTklzU/lrJN0m6Vfp9dZUVaukb6bnF/29pElp/Y9LeiTVs6pJh2lWNweUWeNMqrjFd15u2QsRsYjsG/lfT2V/S/YYgxOAm4ArU/mVwD0RcSLZGHkbU/kC4KqIeD3wHPDnqfxS4ORUz8eKOjiz0eaRJMwaRNKLETG1SvkW4IyI2JwG7P1tRMyU9CxwdETsSeXbIuIoSX3A7PzwOukRKXekh8kh6bNAe0R8WdLPgRfJhqr5UUS8WPChmo0KX0GZlUMMMz3cOtXkx4Pby+BnzO8je1L0KcC63EjcZqXmgDIrh/NyP+9N0/9MNqo6wAXAP6bpO4FLACS1pifgViWpBZgTEXeTPYjxCOCAqzizMvJfUmaNM0nS+tz8zyNioKt5h6T7yf5oPD+VfRy4TtJnyJ5w+9FU/glgpaSLya6ULiF74ms1rcB3JU0ne7DcFenx7Wal58+gzJosfQbVFRHPNrstZmXiW3xmZlZKvoIyM7NS8hWUmZmVkgPKzMxKyQFlZmal5IAyM7NSckCZmVkp/X/aSIidJzbOAAAAAUlEQVQtbechnQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualizing the cost to check for convergence and plotting the linear model:\n",
    "\n",
    "plt.plot(range(1, gd_lr.epochs+1), gd_lr.cost_)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Cost')\n",
    "plt.ylim([0, 0.2])\n",
    "plt.tight_layout()\n",
    "plt.show()    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 5 - Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.82\n",
      "Slope: 0.24\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAGxJJREFUeJzt3X2QlfV5//H3hREtocWh0oZ2gzsZ/P3Cgwi4Ai2QwWAlpJLpdEhHsk3EBx5CSswEcapgiygC9QkpLmQBI4GDCslumLDgAArLr7MruNBdtkAKHYXVrj956FB5GgK73/5xLTVGZM/C3uc+9zmf1wxz9tycPec64/iZi+/9fbAQAiIikhwd4i5ARETaRsEtIpIwCm4RkYRRcIuIJIyCW0QkYRTcIiIJo+AWEUkYBbeISMIouEVEEuYLUbzpjTfeGAoLC6N4axGRnLRr165jIYRu6bw2kuAuLCykpqYmircWEclJZnY43ddqqEREJGEU3CIiCaPgFhFJGAW3iEjCKLhFRBJGwS0ikjAKbhGRhFFwi4hcrY8/hmnT4MMPM/JxCm4Rkauxfj306QMvvACbN2fkIxXcIiJX4qOP4J57YMwYuOEGqK6G730vIx+t4BYRaYsQ4JVXoFcvKC+HJ5+EXbtg8OCMlZDWXiVmdgg4CTQBF0IIRVEWJSKSld59FyZNgi1bYNgwWLoUvvrVjJfRlo77jhBCf4W2iOSdCxfgueegb1/YsQMWL4bKyv8N7VQKCguhQwd/TKWiLSeS3QFFRHJGbS08+KAPh3zrW/DSS1BQ8L9/nUrBxIlw5ow/P3zYnwMUF0dTUroddwA2mdkuM5sYTSkiIlnk7Fl49FEoKoIPPoA1a+CXv/xUaAPMmPFJaF905oxfj0q6HffQEEKjmf0RsNnMfh1C2P7bL2gJ9IkAPXr0aOcyRUQyaNs2mDAB/uM/4P774ZlnoGvXS760oeHSb/F519tDWh13CKGx5fEIUA4MusRrSkMIRSGEom7d0jrEQUQku5w44YF9xx3Q3Ow3IZcv/9zQBvi8PjXK/rXV4DazL5rZ71/8GbgL+LfoShIRiUFZmU/x++lP4ZFHoL4eRo5s9dfmzIFOnT59rVMnvx6VdIZK/hgoN7OLr18dQngjupJERDKosRH+7u98TvaAAVBRAQMHpv3rF29AzpjhwyM9enhoR3VjEsBCCO3+pkVFRUFnTopIVmtuhmXLYPp0+M1v4Ikn4Mc/hi/EM9nOzHalO91a0wFFJP/8+7/7nL3t2308u7QUevaMu6q0acm7iOSP8+fh6afh1lthzx6/8fjmm4kKbVDHLSL54p13fCHNnj3w7W/DwoXwpS/FXdUVUcctIrnt9GnfK3vIEDh2zBfRrFmT2NAGddwikss2bfJNoQ4dgu9/H+bOhS5d4q7qqqnjFpHcc/w43HsvjBoF113nNyFLSnIitEHBLSK5JAR49VVfSLN6Ncyc6ZtEDR8ed2XtSkMlIpIbGhp8OGTDBhg0yGeL3HJL3FVFQh23iCRbUxP88z/7uY/btsGCBVBVlbOhDeq4RSTJ9u71KX5vv+3j2UuW+EkGOU4dt4gkz7lzMGuW7y1y8CCsWgUbN+ZFaIM6bhFJmqoq77L37/ednF54AfJsK2l13CKSDB9/7Lv4DRvmi2o2bPBOO89CGxTcIpIE69f7zceSEvjhD31se/TouKuKjYJbRLLXRx/BPffAmDFwww1QXe2zRjp3jruyWCm4RST7hACvvOILacrL4ckn/ZT1wYPjriwr6OakiGSXd9/1/UW2bPHx7KVL4atfjbuqrKKOW0Syw4UL8Nxz0Lcv7Njh49mVlQrtS1DHLSLxq631KX67dvl4dkkJFBTEXVXWUsctIvE5exYefRSKiuD9932f7HXrFNqtUMctIvHYts3PfTx4EO6/H555Brp2jbuqRFDHLSKZdeIETJjgh/Q2NflNyOXLFdptoOAWkcwpK/Mpfi+/DNOnQ309jBwZd1WJo6ESEYleY6MvVy8vh/79oaICBg6Mu6rEUsctItFpbobSUu+yN26E+fNh506F9lVSxy0i0ThwwG8+Vlb6eHZpKfTsGXdVOUEdt4i0r/Pn4emnoV8/qKvzG49vvqnQbkfquEWk/bzzji+k2bMHvv1tWLgQvvSluKvKOeq4ReTqnT4N06bBkCFw7Bj88pe+mEahHQl13CJydTZt8k2hDh2CyZNh3jzo0iXuqnKaOm4RuTLHj8O99/ohvdddB9u3w+LFCu0MUHCLSNuEAK++6lP8Vq+GmTN9k6jhw+OuLG9oqERE0tfQAN//vp/3OGiQzxa55Za4q8o76rhFpHVNTbBokZ/7uG2bn6xeVaXQjok6bhG5vL17fVOo6mofz16yBAoL464qr6njFpFLO3cOZs2CAQN8FeTKlb5sXaEdO3XcIvJZVVW+kGb/figu9qGRbt3irkpaqOMWkU98/LHv4jdsmC+q2bABVq1SaGcZBbeIuPXr/eZjSQlMnepj26NHx12VXELawW1m15jZv5rZ+igLEslnqZQPIXfo4I+pVAY+9MgRGDfOD+nt0sWHSV58ETp3zsCHy5VoS8f9ELA/qkJE8l0q5bugHj7sa1wOH/bnkYV3CLBihS+kKSuD2bNh927fb0SyWlrBbWYFwF8Cy6ItRyR/zZgBZ858+tqZM3693b37Ltx1F4wf78FdWwuPPw4dO0bwYdLe0u24FwCPAM2f9wIzm2hmNWZWc/To0XYpTiSfNDS07foVuXABnnsO+vaFHTt8PHv7dg9vSYxWg9vM7gaOhBB2Xe51IYTSEEJRCKGom+5Ai7RZjx5tu95mtbU+DPLww3DnnbBvny9f76A5CkmTzn+xocC3zOwQ8BrwdTNbFWlVInlozhzo1OnT1zp18utX5exZePRRKCqC99+H11+HdeugoOAq31ji0mpwhxAeDSEUhBAKgXuAt0IIfxt5ZSJ5prjYj2W86SYw88fSUr9+xSor4dZbfY/s733PF9T8zd/4B0hiaeWkSBYpLr7KoL7oxAl45BFYuhS+8hXYsgVGjmyHN5Zs0KbBrRDCthDC3VEVIyLtoKzMbzYuXw7Tp0N9vUI7x6jjFskVjY2+XL28HPr395WQt90Wd1USAd1OFkm65mYfDO/d23fvmz8fdu5UaOcwddwiSXbggC+vrKyEESM8wG++Oe6qJGLquEWS6Px5ePpp6NcP6upg2TJ46y2Fdp5Qxy2SNO+843tl79kDY8fCwoXQvXvcVUkGqeMWSYrTp2HaNF/9eOyY34Rcu1ahnYfUcYskwaZNMGkSHDoEkyf7gpouXeKuSmKijlskmx0/Dvfe64f0duzoG0ItXqzQznMKbpFsFAK8+qovpFm92vd2rauD4cPjrkyygIZKRLJNQwNMmQIVFXD77b5cvV+/uKuSLKKOWyRbNDXBokV+7uPWrX6yenW1Qls+Qx23SDbYuxcmTPCgvusu+MlP/NBJkUtQxy0Sp3PnYNYsGDDAV0GuXAlvvKHQlstSxy0Sl6oqX0izfz985zuwYAHo9ChJgzpukUw7edJ38Rs2zBfVbNjgR7krtCVNCm6RTFq/3nfxKymBqVN9bHv06LirkoRRcItkwpEjMG4cjBnji2eqquDFF6Fz57grkwRScItEKQRYscIX0pSVwezZsHu37zcicoV0c1IkKu++6/uLbNkCQ4f6+Y+9esVdleQAddwi7e3CBXjuOejbF3bs8PHs7dsV2tJu1HGLtKe6Op/iV1Pj49klJVBQEHdVkmPUcYu0h7Nn4bHH/JzHhgZ4/XVYt06hLZFQxy1ytSorfbn6wYNw333w7LPQtWvcVUkOU8ctcqVOnPCDekeM8A2itmyBl19WaEvkFNwiV6KszBfSLF8O06dDfT2MHBl3VZInNFQi0haNjb5cvbwc+vf3lZADB8ZdleQZddwi6WhuhtJS77I3boT582HnToW2xEIdt0hrDhzwsezKSrjjDg/wnj3jrkrymDpukc9z/jzMnesn0NTV+Xj2m28qtCV26rhFLuWdd3whzZ49MHYsLFwI3bvHXZUIoI5b5NNOn4Zp03wTqGPH/Cbk2rUKbckq6rhFLtq0yTeFOnQIJk+GefN8C1aRLKOOW+T4cRg/HkaNguuu8w2hFi9WaEvWUnBL/goBXn3Vd+1LpWDmTKitheHD465M5LI0VCL5qaEBpkyBigq4/XZfrt6vX9xViaRFHbfkl6YmWLQI+vSBrVvhhRegulqhLYmijlvyx969votfdbWPZy9ZAoWFcVcl0mbquCX3nTsHs2bBgAG+CnLlSl+2rtCWhGq14zaz64HtwHUtr/95COEfoy5MpF1UVXmXvW8fFBf70Ei3bnFXJXJV0um4zwFfDyHcCvQHvmFmOqJastvJk76L37BhcOoUbNgAq1Z9JrRTKW+8O3Twx1QqlmpF2qTVjjuEEIBTLU+vbfkToixK5KpUVPgCmv/8T/jhD+Gpp6Bz58+8LJXyvaPOnPHnhw/7c/DmXCRbpTXGbWbXmFktcATYHELYEW1ZIlfgyBEYNw7uvhtuuMFvQi5YcMnQBpgx45PQvujMGb8uks3SCu4QQlMIoT9QAAwys76/+xozm2hmNWZWc/To0fauU+TzhQArVvhCmrIymD0bdu2CwYMv+2sNDW27LpIt2jSrJIRwAtgGfOMSf1caQigKIRR1080fyZT33vOpfePHe3DX1sLjj0PHjq3+ao8ebbsuki1aDW4z62ZmN7T8/HvAncCvoy5M5LIuXIDnn4e+feHtt6GkxPcY6dUr7beYMwc6dfr0tU6d/LpINktnAU53YIWZXYMH/ZoQwvpoyxK5jLo63yu7pgbGjPHQLiho89tcvAE5Y4YPj/To4aGtG5OS7dKZVbIHGJCBWkQu7+xZePJJ+Kd/gj/8Q1izxg85MLvitywuVlBL8mjJuyRDZaUvpDl4EO6/H555Brp2jbsqkVhoybtktxMnfHL1iBG+QdSWLX72o0Jb8piCW7JXWRn07u1BPX061NfDyJFxVyUSOw2VSPZpbPTl6uXl0L8/rF8PAwfGXZVI1lDHLdmjuRmWLvUue+NGmD8fdu5UaIv8DnXckh0OHPCx7MpKuOMOKC2Fnj3jrkokK6njlnidPw9z5/oJNHV1Pp795psKbZHLUMct8ampgQcegD17fD72woXQvXvcVYlkPXXcknmnT8O0ab4J1LFjfhNy7VqFtkia1HFLZm3aBJMmwaFDvmf2vHnQpUvcVYkkijpuyYzjx30Hv1Gj4LrrfEOoxYsV2iJXQMEt0QoBXnvNd+1LpWDmTN96dfjwuCsTSSwNlUh0GhpgyhQ/SmzQIJ8tcsstcVclknjquKX9NTfDokXQpw9s3eonq1dVKbRF2ok6bmlfe/f6Ln7V1T6evWSJH58uIu1GHbe0j3PnYNYsGDDAV0GuXOnL1hXaIu1OHbdcvaoq77L37fNTCV54AXTuqEhk1HHLlTt50nfxGzYMTp2CDRtg1SqFtkjEFNxyZSoqfBe/khKYOtXHtkePjrsqkbyg4Ja2OXIExo2Du+/2xTNVVfDii9C5c9yVieQNBbekJwRYscIX0pSVwezZsHs3DBkSd2UieUc3J6V1773n+4ts3gxDh/phB716xV2VSN5Sxy2f78IFeP556NsX3n7bx7O3b1doi8RMHbdcWl0dPPig75k9ZoyHdkFB3FWJCOq45XedPQuPPQa33eZ7jaxZA+vWKbRFsog6bvlEZaUvpDl4EO6/H555Brp2jbsqEfkd6rgFTpzwg3pHjICmJtiyxc9+VGiLZCUFd74rL/eFNMuXw/TpUF8PI0fGXZWIXIaGSvJVY6OveCwrg/79Yf16GDgw7qpEJA3quPNNc7PPw+7d2/cWmT8fdu5UaIskiDrufHLggI9lV1bCHXdAaSn07Bl3VSLSRuq488H58zB3LvTr5/Ozly/3Y8QU2iKJpI4719XUwAMPwJ49MHYsLFwI3bvHXZWIXAV13Lnq9GmYNg0GD4Zjx3z2yNq1Cm2RHKCOOxdt2uSbQh06BJMnw7x5vgWriOQEddy55PhxGD/eD+nt2NE3hFq8WKEtkmMU3LkgBHjtNd+1L5WCmTP9JuTw4XFXJiIR0FBJ0jU0wJQpfpTYoEE+W+SWW+KuSkQipI47qZqbYdEi6NMHtm71k9WrqhTaInmg1eA2sy+b2VYz229me83soUwUJpexd6+frD51qp9Is3cv/OhHcM01cVcmIhmQTsd9AZgWQugFDAF+YGa9oy0rv6VSUFgIHTr4YyrV8hfnzsGsWTBggK+CXLkSNm70F4lI3mh1jDuE8CHwYcvPJ81sP/CnwL6Ia8tLqZSvSj9zxp8fPuzPbzxYzai1D8K+fVBc7EMj3brFW6yIxKJNY9xmVggMAHZEUYzAjBmfhDZAZ04y78xU/uKJoXDqlG8MtWqVQlskj6Ud3GbWGfgF8KMQwseX+PuJZlZjZjVHjx5tzxrzSkPDJz9/kwr20Zsf8BKLmOpj2aNHx1eciGSFtILbzK7FQzsVQii71GtCCKUhhKIQQlE3dYNXrEcP6MYRVjOOCu7mv+nCn1PF8ze9CJ07x12eiGSBdGaVGLAc2B9CeD76kvJYCKwetYL99OKvKeMfeIKB7Ka+0xDmzIm7OBHJFul03EOB7wJfN7Palj/fjLiu/PPeezBqFH9eOp6m/9OLb/5JLU/ZP/AnN3WktNTvR4qIQHqzSv4FsAzUkp8uXPCtVh9/3Odhl5TwR5Mm8WYHrY0SkUvTkvc41dXBgw/6ntljxkBJCRQUxF2ViGQ5tXVxOHsWHnsMbrvNp5G8/jqsW6fQFpG0qOPOtMpKmDABDh6E++6DZ5+Frl3jrkpEEkQdd6acOOFLIEeMgKYm2LIFXn5ZoS0ibabgzoTycujd2w/pnT4d6uth5Mi4qxKRhNJQSZQaG30Hv7Iy6N8ffvUrH9cWEbkK6rij0NwMS5d6l71hA8yfDzt3KrRFpF2o425vBw74WHZlpY9nl5bCzTfHXZWI5BB13O3l/HmYOxf69fP52cuWwVtvKbRFpN2p424PNTXwwAOwZw+MHesrIbt3j7sqEclR6rivxunT8PDDMHgwHDvms0fWrlVoi0ik1HFfqc2bYdIk3xxq8mSYNw+6dIm7KhHJA+q42+r4cRg/Hu66C669FrZvh8WLFdoikjEK7nSFAK+9Br16+cGQM2b4Tcjhw+OuTETyjIZK0tHQAFOmQEUF3H67L1fv1y/uqkQkT6njvpzmZli0CPr0ga1b/WT16mqFtojESh3359m3z/fKrq728eyf/AQKC+OuSkREHfdnnDsHTzzhe4scOAArV8Ibbyi0RSRrqOP+bdXV3mXv2wff+Q4sWAA6sV5Esow6boCTJ30Xv6FD4dQp3xgqlVJoi0hWUnBXVPjNx5de8vDeuxdGj467KhGRz5W/QyVHjsBDD/nc7D59oKoKhgyJuyoRkVblX8cdAvzsZ76QpqwMZs+G3bsV2iKSGPnVcb/3nu8vsnmzj2cvXeoBLiKSIPnRcV+4AM8/D337wttvQ0mJ7zGi0BaRBMr9jruuzqf41dTAmDF+E/LLX467KhGRK5a7HffZs/DYY1BU5HuNvP46rFun0BaRxMvNjruyEiZMgIMH4b774NlnoWvXuKsSEWkXudVxnzjhNx9HjICmJr8J+fLLCm0RySm5E9zl5dC7tx/SO3061NfDnXfGXZWISLtL/lBJY6OveCwr842hfvUruO22uKsSEYlMcjvuEHwedu/evrfIvHmwc6dCW0RyXjI77gMHYOJEvwk5YgSUlsLNN8ddlYhIRiSr4z5/HubO9RNoamt9PPuttxTaIpJXktNx19T4Qpq6Ohg7FhYuhO7d465KRCTjsr/jPn0aHn4YBg+Go0d99sjatQptEclb2d1xb97s87Ivbg41fz506RJ3VSIiscrOjvv4cRg/3g/pvfZavwm5ZIlCW0SEbAvuEPxgg169/OiwGTN8TPtrX4u7MhGRrNFqcJvZy2Z2xMz+LcpCUimY0/U5GDeO2v8upGL2LnjqKbj++ig/VkQkcdIZ434FWAT8LKoiUimflt35zHf5iI689JsfcP1T11DaA4qLo/pUEZFkshBC6y8yKwTWhxD6pvOmRUVFoaamJu0iCgvh8OHPXr/pJjh0KO23ERFJLDPbFUIoSue17TbGbWYTzazGzGqOHj3apt9taGjbdRGRfNZuwR1CKA0hFIUQirp169am3+3Ro23XRUTyWVbMKpkzBzp1+vS1Tp38uoiIfFpWBHdxse8TddNNYOaPpaW6MSkicimtzioxs1eBEcCNZvYB8I8hhOXtXUhxsYJaRCQdrQZ3CGFcJgoREZH0ZMVQiYiIpE/BLSKSMApuEZGEUXCLiCSMgltEJGHS2qukzW9qdhS4xO4jabkRONaO5SSBvnPuy7fvC/rObXVTCCGtZeeRBPfVMLOadDdayRX6zrkv374v6DtHSUMlIiIJo+AWEUmYbAzu0rgLiIG+c+7Lt+8L+s6RyboxbhERubxs7LhFROQysia4M3UocTYxsy+b2VYz229me83sobhripKZXW9mO82sruX7PhF3TZliZteY2b+a2fq4a8kEMztkZvVmVmtm6Z9jmFBmdoOZ/dzMft3y//OfRfp52TJUYmZfA04BP0v3bMukM7PuQPcQwm4z+31gF/BXIYR9MZcWCTMz4IshhFNmdi3wL8BDIYS3Yy4tcmb2Y6AI+IMQwt1x1xM1MzsEFIUQ8mIet5mtAP5fCGGZmXUEOoUQTkT1eVnTcYcQtgP/FXcdmRRC+DCEsLvl55PAfuBP460qOsGdanl6bcuf7OgcImRmBcBfAsvirkXan5n9AfA1YDlACOE3UYY2ZFFw5zszKwQGADvirSRaLUMGtcARYHMIIae/b4sFwCNAc9yFZFAANpnZLjObGHcxEfsKcBT4actw2DIz+2KUH6jgzgJm1hn4BfCjEMLHcdcTpRBCUwihP1AADDKznB4WM7O7gSMhhF1x15JhQ0MIA4HRwA9ahkJz1ReAgcDiEMIA4DTw91F+oII7Zi1jvb8AUiGEsrjryZSWf0puA74RcylRGwp8q2XM9zXg62a2Kt6SohdCaGx5PAKUA4PirShSHwAf/Na/Hn+OB3lkFNwxarlZtxzYH0J4Pu56omZm3czshpaffw+4E/h1vFVFK4TwaAihIIRQCNwDvBVC+NuYy4qUmX2x5WY7LUMGdwE5O1sshPD/gffN7P+2XBoJRDrBoNUzJzMlU4cSZ5mhwHeB+pZxX4DHQggbYqwpSt2BFWZ2Dd40rAkh5MX0uDzzx0C59yV8AVgdQngj3pIiNxVItcwoeRe4L8oPy5rpgCIikh4NlYiIJIyCW0QkYRTcIiIJo+AWEUkYBbeISMIouEVEEkbBLSKSMApuEZGE+R+EhfZyAyHEYwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "sgd_lr = LinearRegression(method='sgd',\n",
    "                          eta=0.01, \n",
    "                          epochs=100, \n",
    "                          random_seed=0, \n",
    "                          minibatches=len(y))\n",
    "sgd_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % sgd_lr.w_)\n",
    "print('Slope: %.2f' % sgd_lr.b_)\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, sgd_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt0XNWd5v3vU1Uq2ZZ808XG2MY22MSYSwwIh4aEDhAS6MkA0w0BFp2QTGZ4m7eZvmS6J9AzSa9hkpmX1e+apLOaYUJ3yD0hNJNM3A2Jk3BJejpcLHMztnEQxsbyVbZl2bJsSSX95o86MmUh2xI6ZRXy81mrlursc9E+pVI9tffZtUsRgZmZWaXJjHUFzMzMhuKAMjOziuSAMjOziuSAMjOziuSAMjOziuSAMjOzilTWgJJ0taT1klok3TXE+s9IWivpZUmPS5pXsu42Sa8lt9tKyi+UtDo55lckqZznYGZmY0Pl+hyUpCzwG+AqoBVYCdwSEWtLtrkceDYiuiTdAXwwIm6SVAc0A01AAKuACyOiXdJzwB8DzwCPAV+JiJ+U5STMzGzMlLMFtQxoiYgNEdEDPARcV7pBRDwZEV3J4jPAnOT+R4CfR8SeiGgHfg5cLWkWMCUino5isn4LuL6M52BmZmMkV8ZjzwY2lyy3Au87xvafBgZaQkPtOzu5tQ5R/jaSbgduB6ipqblw8eLFI6n7sPT1B2u37WPW1Ak01Fanfnwzs/Fo1apVuyKi8XjblTOghro2NGR/oqTfp9id99vH2XfYx4yIB4AHAJqamqK5ufl49R2xA90Fzv7LFfzF7yzm9svOSP34ZmbjkaRNw9munF18rcDckuU5wNbBG0n6EPAfgWsjovs4+7byVjfgUY95olRliw9fb5/nMzQzS1s5A2olsEjSAkl54GZgeekGks4HvkoxnHaWrFoBfFjSdEnTgQ8DKyJiG7Bf0sXJ6L1PAD8u4zkcU1W22KDrLvSPVRXMzMatsnXxRURB0p0UwyYLPBgRayTdAzRHxHLgr4Ba4O+T0eJvRsS1EbFH0n+hGHIA90TEnuT+HcA3gIkUr1mN2Qg+SeSzGXr7HFBmZmkr5zUoIuIxikPBS8s+X3L/Q8fY90HgwSHKm4FzUqzmqFRlRa9bUGZmqfNMEqNUlcvQ4xaUmVnqHFCj5C4+M7PycECNUlU2Q0/Bo/jMzNLmgBqlvLv4zMzKwgE1SvlsxoMkzMzKwAE1SlU5+RqUmVkZOKBGqSrrLj4zs3JwQI1ScZCEA8rMLG0OqFGqznmYuZlZOTigRsldfGZm5eGAGqXiVEf+HJSZWdocUKOUz2XdxWdmVgYOqFGqyspdfGZmZeCAGqW8R/GZmZWFA2qU8h7FZ2ZWFg6oUarKZvyV72ZmZeCAGiV/UNfMrDwcUKM0MJt5hFtRZmZpckCNUj4rAAr9DigzszQ5oEapKlt8CN3NZ2aWrrIGlKSrJa2X1CLpriHWXybpeUkFSTeUlF8u6cWS2yFJ1yfrviHpjZJ1S8t5DsczEFAeyWdmlq5cuQ4sKQvcB1wFtAIrJS2PiLUlm70JfBL4s9J9I+JJYGlynDqgBfhZySZ/HhGPlKvuI5HPJS0oB5SZWarKFlDAMqAlIjYASHoIuA44HFARsTFZd6xX9xuAn0REV/mq+s7l3cVnZlYW5ezimw1sLlluTcpG6mbg+4PKvijpZUlfklQ91E6SbpfULKm5ra3tHfza4anKFQdJ+LNQZmbpKmdAaYiyEb2KS5oFnAusKCm+G1gMXATUAZ8dat+IeCAimiKiqbGxcSS/dkTy2Szga1BmZmkrZ0C1AnNLlucAW0d4jI8BP4qI3oGCiNgWRd3A1yl2JY6ZqmSYubv4zMzSVc6AWgkskrRAUp5iV93yER7jFgZ17yWtKiQJuB54JYW6vmNVHiRhZlYWZQuoiCgAd1LsnlsHPBwRayTdI+laAEkXSWoFbgS+KmnNwP6S5lNsgf1y0KG/K2k1sBpoAL5QrnMYjuqBYeZuQZmZpaqco/iIiMeAxwaVfb7k/kqKXX9D7buRIQZVRMQV6dZydAZaUB4kYWaWLs8kMUqHZ5Lo6xvjmpiZjS8OqFF663NQbkGZmaXJATVK+cOfg/I1KDOzNDmgRsmTxZqZlYcDapQ8WayZWXk4oEYpn3NAmZmVgwNqlAZaUN3u4jMzS5UDapTyWX8OysysHBxQo+QuPjOz8nBAjVI2IzLyKD4zs7Q5oFJQlc24BWVmljIHVAryuYxnMzczS5kDKgX5bMZdfGZmKXNApcBdfGZm6XNApSCfy3iYuZlZyhxQKajKytegzMxS5oBKQZWvQZmZpc4BlYJiF58DyswsTQ6oFOQ9SMLMLHUOqBS4i8/MLH1lDShJV0taL6lF0l1DrL9M0vOSCpJuGLSuT9KLyW15SfkCSc9Kek3SDyTly3kOw1GVy9DjUXxmZqkqW0BJygL3AdcAS4BbJC0ZtNmbwCeB7w1xiIMRsTS5XVtSfi/wpYhYBLQDn0698iOUz2bodQvKzCxV5WxBLQNaImJDRPQADwHXlW4QERsj4mVgWK/ukgRcATySFH0TuD69Kr8z+ZyHmZuZpa2cATUb2Fyy3JqUDdcESc2SnpE0EEL1wN6IKBzvmJJuT/ZvbmtrG2ndR8QzSZiZpS9XxmNriLKRXKg5LSK2SjodeELSamDfcI8ZEQ8ADwA0NTWV9QKRu/jMzNJXzhZUKzC3ZHkOsHW4O0fE1uTnBuAp4HxgFzBN0kCwjuiY5VLl2czNzFJXzoBaCSxKRt3lgZuB5cfZBwBJ0yVVJ/cbgEuBtRERwJPAwIi/24Afp17zEfJs5mZm6StbQCXXie4EVgDrgIcjYo2keyRdCyDpIkmtwI3AVyWtSXY/C2iW9BLFQPr/ImJtsu6zwGcktVC8JvW1cp3DcHmyWDOz9JXzGhQR8Rjw2KCyz5fcX0mxm27wfr8Gzj3KMTdQHCFYMTxZrJlZ+jyTRAqqshn6+oO+freizMzS4oBKQVW2+DB6qLmZWXocUCmozjmgzMzS5oBKwUALyiP5zMzS44BKwVtdfL4GZWaWFgdUCvLu4jMzS50DKgVV2eKsTt3u4jMzS40DKgV5j+IzM0udAyoF7uIzM0ufAyoFHsVnZpY+B1QKDgeUW1BmZqlxQKXgrS4+DzM3M0uLAyoFeXfxmZmlzgGVgqpccZi5B0mYmaXHAZUCTxZrZpY+B1QKBrr4/EFdM7P0OKBS4M9BmZmlzwGVgsNdfG5BmZmlxgGVAg8zNzNLnwMqBQOTxfqDumZm6SlrQEm6WtJ6SS2S7hpi/WWSnpdUkHRDSflSSU9LWiPpZUk3laz7hqQ3JL2Y3JaW8xyGoyrjz0GZmaUtV64DS8oC9wFXAa3ASknLI2JtyWZvAp8E/mzQ7l3AJyLiNUmnAqskrYiIvcn6P4+IR8pV95HKZERVVh4kYWaWorIFFLAMaImIDQCSHgKuAw4HVERsTNYd8coeEb8pub9V0k6gEdhLharKZtyCMjNLUTm7+GYDm0uWW5OyEZG0DMgDr5cUfzHp+vuSpOqj7He7pGZJzW1tbSP9tSNWlc24BWVmlqJyBpSGKBvRMDdJs4BvA5+KiIFX/7uBxcBFQB3w2aH2jYgHIqIpIpoaGxtH8mvfkXwuQ49H8ZmZpaacAdUKzC1ZngNsHe7OkqYAjwL/KSKeGSiPiG1R1A18nWJX4pjLu4vPzCxV5QyolcAiSQsk5YGbgeXD2THZ/kfAtyLi7wetm5X8FHA98EqqtX6HqrLyMHMzsxSVLaAiogDcCawA1gEPR8QaSfdIuhZA0kWSWoEbga9KWpPs/jHgMuCTQwwn/66k1cBqoAH4QrnOYSSmTKyi42DvWFfDzGzcKOcoPiLiMeCxQWWfL7m/kmLX3+D9vgN85yjHvCLlaqairibPrs7usa6Gmdm44ZkkUlJXk2dPZ89YV8PMbNxwQKWkobaa3Qd6iPBIPjOzNDigUlJXk6e70E9XT99YV8XMbFxwQKWkriYPwJ4D7uYzM0uDAyol9UlAeaCEmVk6HFApcQvKzCxdDqiUNNQWpwTc7YAyM0uFAyolbkGZmaXLAZWSSfks1bmMA8rMLCUOqJRIot6zSZiZpcYBlaK62rxbUGZmKXFApai+ptoBZWaWEgdUiupr8uz2fHxmZqlwQKWorsZdfGZmaXFApaiuNs/B3j66egpjXRUzs3e9YQWUpG8Pp+xkNzDdkbv5zMxGb7gtqLNLFyRlgQvTr867W31NcTYJd/OZmY3eMQNK0t2S9gPnSdqX3PYDO4Efn5AavovU1Xo2CTOztBwzoCLiv0XEZOCvImJKcpscEfURcfcJquO7xuEuPgeUmdmoDbeL7x8l1QBI+n1J/13SvDLW613prfn4PJuEmdloDTeg7ge6JL0X+A/AJuBbx9tJ0tWS1ktqkXTXEOsvk/S8pIKkGwatu03Sa8nttpLyCyWtTo75FUka5jmUXW11jnw240ESZmYpGG5AFSIigOuAv46IvwYmH2uHZCDFfcA1wBLgFklLBm32JvBJ4HuD9q0D/hJ4H7AM+EtJ05PV9wO3A4uS29XDPIeyk0R9bd5dfGZmKRhuQO2XdDfwceDRJHyqjrPPMqAlIjZERA/wEMWAOywiNkbEy0D/oH0/Avw8IvZERDvwc+BqSbOAKRHxdBKY3wKuH+Y5nBD+sK6ZWTqGG1A3Ad3Av46I7cBs4K+Os89sYHPJcmtSNhxH23d2cv+4x5R0u6RmSc1tbW3D/LWjV1fjFpSZWRqGFVBJKH0XmCrpo8ChiDjeNaihrg3FMOt1tH2HfcyIeCAimiKiqbGxcZi/dvTqa/IeJGFmloLhziTxMeA54EbgY8Czgwc1DKEVmFuyPAfYOsx6HW3f1uT+OznmCVFXU80eD5IwMxu14Xbx/Ufgooi4LSI+QfH60ueOs89KYJGkBZLywM3A8mH+vhXAhyVNTwZHfBhYERHbKF4PuzgZvfcJKuwDw/W1eQ709HGot2+sq2Jm9q423IDKRMTOkuXdx9s3IgrAnRTDZh3wcESskXSPpGsBJF0kqZViy+yrktYk++4B/gvFkFsJ3JOUAdwB/B3QArwO/GSY53BC+MO6ZmbpyA1zu59KWgF8P1m+CXjseDtFxGODt4uIz5fcX8mRXXal2z0IPDhEeTNwzjDrfcId/rBuZw+zp00c49qYmb17HTOgJC0EZkbEn0v6XeD9FAcqPE1x0IQNUl870ILyQAkzs9E4Xhffl4H9ABHxw4j4TET8KcVW0ZfLXbl3ozrPaG5mlorjBdT85IO0R0i62eaXpUbvcodbUB7JZ2Y2KscLqAnHWOcLLEOYXJ2jKisPkjAzG6XjBdRKSf92cKGkTwOrylOldzdJyXRHvgZlZjYaxxvF9yfAjyTdyluB1ATkgX9Vzoq9m9XVVPsalJnZKB0zoCJiB3CJpMt5a2j3oxHxRNlr9i5W7/n4zMxGbVifg4qIJ4Eny1yXcaOuJs/m9q6xroaZ2bvacGeSsBGYXz+JTbu7+K+PraO74CmP3qntHYdY3dox1tUwszEy3JkkbATu+OBCdh/o4YFfbeCfXtvFl29ayntOOeb3O6au0NfPazs7WbN1H2u2dtDfH/zJh85kejLTxUjt7uymeVM7z29q5/k325k6Mc/HmuZw+eIZVGUzdBf6eO6NPby0eS8Xzqtj2YI6spl39mXHW/Ye5P6nWnh4ZSuF/n7+x60XcPU5s97Rsbp6CuQyGfI5vxcze7dR8Xv/xrempqZobm4+4b/38XU7+Oz/epl9BwtcdfZMbrhwDh9Y2EChP3h+Uzu/fn03AFeeNYP3zplGJnlB37HvEGu37mN/d4Hu3j66C/0c6u3jQHcfXb0FJlfnuOzMRs45derhfQD6+4OVG/ew/KWtPLZ6G+1dvQBMqMrQ1x/MmDyB+269gKVzpw2r/lv3HuSnr2znJ69so3lTOxGQz2Y4e/YUWtsP0ra/m4baas6ZPYXn3thDV89brcWG2mquOecUPtY0l3PnTD3iuG37u3llawfvW1DHpPxb75He3N3F/b98nUdWFb8K7Mamuby6bR+vbNnHNz51EZcsbBhWvbsLffxi7U7+ftVmfvWbNvoDavJZpk3Kc/Hp9dx5xUIWNNQM61gnQkSweksHO/d1c/niGUcN9kO9fazbto+a6hxnzjyxb3hGqrO7wPOb2jnY20d1rvgG4YzGWmZOefsnV/Yf6qW2Okdx/ufj6+jq5TvPbuK0ukl85OxTRv3mo78/2LL3IBt3H2Dj7i62dxzkmnNmcc7sqcffeQQigpdbO2icXM2pFTANWnehj394aRtN86Yz/wT/P0haFRFNx93OAVVeuzq7+ZsnWvjxi1to7+qlviZPZ3eB7kL/4ReiYnhUc/apU3h1+362dRw66vHyuQy9ff1EQENtnovm19HV00fb/m62dRykvauXiVVZPrRkJlcunsE5s6ewoKGWNVs7uOM7z7Nz/yE+e/ViZk2dyIa2Tt7YfYDqXJY50ycyZ/pEegr9NG9sZ+WmPWxoOwDA4lMmc/U5p/CBRQ2cfepUJlRlKfT189T6Nh5auZkNbZ381hn1XLF4BuefNp2nX9/NY6u38firOzjU28+yBXX8m/cvoGFyNd/69UYeXb2N3r6gJp/ld86dxVVLZvLTV7bz45e2kpX42EVzuOODC5k9bSJ7u3q46avP0Nrexfdvv5jz5rwVrv39wYZdB3hx8142tHWyveMQ2zoOsW77PvZ29TJr6gSuWzqbmnyW9q5e2jq7+fna7fQU+rl+6Ww+tGQma7Z28PymvWzafYBlC+r48Nmn8NtnNlJTnW7nQn9/8Pyb7Ty1vg2peJ2yribPb3bs5x9f3sam3V2HH+vPfXQJly5soC95w7FizXZWbWpn3bZ99PYV/19/9/zZfPaaxUO+4A/W1x/s7ephUj7HhKrMMYMgIni9rZPH1+3ktZ2dnDVrChfOm86SWVPI5zJEBH39QTajI45zoLvAi5v38uwbe/h1yy5e3LyXQv+Rry1VWXFj01z+3w+ewexpE1m5sZ37n2rhyfVtnDVrCp+6ZD7XLj2VbEasfGMPP1u7g32HevnoebP4wKJGchnxDy9v455/WMuuzuLHOOpr8tzYNJffvWA2i2bUHjfkDvb0sXH3Ad7YdYB12/bx4ua9vLR5L/sOFY7YLpsRf/Dbp/NHVy6iOpcd8lgdXb1s2NXJG7sOsGl3F3PrJnH5exqpr61+27avbOngnn9cy3NvFOe8nj1tIssW1PHB9zRy1ZKZR7xR23eol1e2dHB6Qy2nTD3+33ek+vuDf3h5K3+1Yj2t7QeZUJXh7mvO4uMXzzviDW85OaBKjGVADegp9PPEqzt5bPU2GmqruXRhPcsW1NHXHzy5fie/WLuT3+zYz5JTp/DeOdM4d85Upk/KU53LUJ3LMDGfZWJVllw2w+7Obn71WhtPvtrGy617mTqxisbJ1TROrubi0+vf9oQfsLerh3//8Es8/upbE9OfMmUCvX39R4w6nDapiqZ507lofh1XLZnJ6Y217+ic9x/q5QcrN/P1f97Ilr0HgeIHmX/vwjm8f2EDP1u7nUdf3saBnj4mVmW59X2n8W8vO/1tL7o79h3i9+7/NR0Hew+/CPVH8PrOzsMvLLmMmDllArOmTmBefQ3XLj2V9y9seFtrpG1/Nw/86nW+/cwmDvX2k8uIs2ZNYW7dRJ5+fTftXb3ksxnqavLkcxmqsqI6l6WmOsvEfI5JVVkm5rNMqMoWX+wRQRBR7E5s7+qlo6uXvgjqavLU1+Tpj+DJ9W207e8mmxERwcBrdzYjLjmjnn953qlUV2UOv2g0zZvOxt0H2NXZQ3UuwwWnTee9c6fx3jlTeXlLB1/7pzfIZcXHL55HNiN2dXaz50AvkyfkaJxcTUNtnt2dPby4eS+rt3Qcbt1KUJvPMa9hEmc01nJ6Qy1BsOdAD7sP9LC6tYM39xTDcvqkqsOt8KqsyEj0JG+O8rkMMyZXMzN5/qzZuo++/iAjOHf2VC5Z2MAlZ9QzfVKe7kI/3b19PPbKNn6wstg6PqOxlle376euJs/1S2fzzy27WL9jP9MnVdHXH+w7VKA6l2FCVZaOg8U3dvPqJ/H8m3s5b85Uvnj9uezp6uG7z2ziF+t20B8wY3I1ly5s4MyZk9m69yCb9nTR2t5Fd28/vX3F28D5DDz2Z86czNK50zhvzlROb6hhXn0NE6oyfOHRdTyyqpVFM2q59X2nEUChL2jv6uHV7ftZu3Uf2/e9/Y2kBBecNp0LTpvGpHyOifksr+3o5IcvtDJ9Up5/d8VCImDlxj2s3LiHXZ091OSzfOScUzhz5mR+ub6NlRv3HA73mVOqee+caZw6bSKTJ+SSWxWTJ+SYMqEKgJadnfxmx37e2HWAupo88+prmFc/if4Itu49yLa9h+g42IskMoI393Tx6vb9LJk1hX93xUJ+0LyZp9a3cenCev70Q2dSOyFHdS5LPpchK5HJQFZHviGpzmVG9SbOAVWiEgKqUvT3B89t3MPkCTkWNNQcDrKungJb2g8iwekNtam+kyr09fOztTvoPFTgd86bRW3JE7urp8Bzb+zhvDnTDs8EP5SNuw5w709fpbO7QH8UA2FefQ3nz53G0tOmcUZj7Yiuee3q7GbT7i6WzJrCxHz2cD1XbWrnifU7aT/QQ29f0JN0r3b19NHV20dXd4FDhT4O9fZzqKRLE0FNPse0SVVMm1RFNiN2dxZf9Lt7+3j/ogY+cvYpXL54BjX5HHu7ethzoIf62uojzvtQbx/f+PVGHnruTc6ePZVrzjmFy98z420vBpt2H+CLj67jZ2t3kMuI+to80ycVW+dt+7vpLvSTz2Y469QpLJ0zlXn1NXQX+jnYU6DjYC8bd3fRsrPz8BuHqROrqK/Js6ChhssXz+DyxTOYPW0i2zsO8cKb7aze0kFfBPlshqpshgPdBXbu72bHvkNEwIXzptM0fzoXzJt++IVzKFv3HuT+p17nxc17ueHCOXysaS4T81kigqc37Ob7z20mn83w4bNn8oFFDeQyGX75mzZ+9EIrq7d08K8vXcAnfmv+EX/rHfsO8dT6nfyflt38umUXuw/0MGVCjnn1NcyZPpFJ+eLsLrmsmDF5Agsaaji9sYbTG2oP/+2H8uT6nfzFD1cf0aORzYiFjbWcNWsyi2dNYWFjLfMbaphbN5HfbO/kF+t28PirO2jZ2cmh3n6gGO6funQBf3j5QqZOfOuxGfhf/NHzW3hs9Tb2dxc4c2YtV541k2Xz69i4+wAvbd7Ly60d7OrsTp77Q9d1+qQqTm+spf1AD5vbuw63tLMZccqUCUybVEUE9EdQnctw2yXzuX7pbDLJG6bvP7eZLzy69ohu+mO5qWku995w3rC2HYoDqoQDysarrp4CE3LZI95QRAT7uwtJ6/voL8BQDMRsRlRlx8cgkv7+4EBPgcnHCMmR6O3rp+NgL7mMyGbEhKrssB+r/v7gUKEPoWMGIRT/DvsO9jLjGF22EcGBnuJ2+w8V2H+ol96+4IwZNTTWVh9u4fT1F1tOVdkMjZOrh/3GbXvHIdZu6+BQbz/dhT56Cv309UNfBP2DknHhjFouHeY14aEMN6A8is/sXWyorlxJx2zFlJpQdewXznebTEaphRNAVTZDwxDXlIZbl6H+PkMpdhkf+28hidrq3BE9EEPJZsTcuknDrueAU6ZOKMs1r9EYH2+bzMxs3HFAmZlZRXJAmZlZRXJAmZlZRSprQEm6WtJ6SS2S7hpifbWkHyTrn5U0Pym/VdKLJbd+SUuTdU8lxxxYN6Oc52BmZmOjbAElKQvcB1wDLAFukbRk0GafBtojYiHwJeBegIj4bkQsjYilwMeBjRHxYsl+tw6sj4idmJnZuFPOFtQyoCUiNkRED/AQcN2gba4DvpncfwS4Um+fq+QW4PtlrKeZmVWgcgbUbGBzyXJrUjbkNhFRADqA+kHb3MTbA+rrSffe54YINAAk3S6pWVJzW1vbOz0HMzMbI+UMqKGCY/C0FcfcRtL7gK6IeKVk/a0RcS7wgeT28aF+eUQ8EBFNEdHU2Ng4spqbmdmYK2dAtQJzS5bnAFuPto2kHDAV2FOy/mYGtZ4iYkvycz/wPYpdiWZmNs6UM6BWAoskLZCUpxg2ywdtsxy4Lbl/A/BEJJMDSsoAN1K8dkVSlpPUkNyvAj4KvIKZmY07ZZuLLyIKku4EVgBZ4MGIWCPpHqA5IpYDXwO+LamFYsvp5pJDXAa0RsSGkrJqYEUSTlngF8DflusczMxs7Hg2czMzO6GGO5u5Z5IwM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OK5IAyM7OKVNaAknS1pPWSWiTdNcT6akk/SNY/K2l+Uj5f0kFJLya3/1myz4WSVif7fEWSynkOZmY2NsoWUJKywH3ANcAS4BZJSwZt9mmgPSIWAl8C7i1Z93pELE1uf1BSfj9wO7AouV1drnMwM7OxU84W1DKgJSI2REQP8BBw3aBtrgO+mdx/BLjyWC0iSbOAKRHxdEQE8C3g+vSrbmZmY62cATUb2Fyy3JqUDblNRBSADqA+WbdA0guSfinpAyXbtx7nmABIul1Ss6Tmtra20Z2JmZmdcOUMqKFaQjHMbbYBp0XE+cBngO9JmjLMYxYLIx6IiKaIaGpsbBxBtc3MrBKUM6Bagbkly3OArUfbRlIOmArsiYjuiNgNEBGrgNeBM5Pt5xznmGZmNg6UM6BWAoskLZCUB24Glg/aZjlwW3L/BuCJiAhJjckgCySdTnEwxIaI2Absl3Rxcq3qE8CPy3gOZmY2RnLlOnBEFCTdCawAssCDEbFG0j1Ac0QsB74GfFtSC7CHYogBXAbcI6kA9AF/EBF7knV3AN8AJgI/SW5mZjbOqDgYbnxramqK5ubmsa6GmZkBklZFRNPxtvNMEmZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEpbwzgAAAJa0lEQVQcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpEcUGZmVpHKGlCSrpa0XlKLpLuGWF8t6QfJ+mclzU/Kr5K0StLq5OcVJfs8lRzzxeQ2o5znYGZmYyNXrgNLygL3AVcBrcBKScsjYm3JZp8G2iNioaSbgXuBm4BdwL+MiK2SzgFWALNL9rs1IprLVXczMxt75WxBLQNaImJDRPQADwHXDdrmOuCbyf1HgCslKSJeiIitSfkaYIKk6jLW1czMKkw5A2o2sLlkuZUjW0FHbBMRBaADqB+0ze8BL0REd0nZ15Puvc9J0lC/XNLtkpolNbe1tY3mPMzMbAyUM6CGCo4YyTaSzqbY7ff/lKy/NSLOBT6Q3D4+1C+PiAcioikimhobG0dUcTMzG3vlDKhWYG7J8hxg69G2kZQDpgJ7kuU5wI+AT0TE6wM7RMSW5Od+4HsUuxLNzGycKWdArQQWSVogKQ/cDCwftM1y4Lbk/g3AExERkqYBjwJ3R8Q/D2wsKSepIblfBXwUeKWM52BmZmOkbAGVXFO6k+IIvHXAwxGxRtI9kq5NNvsaUC+pBfgMMDAU/U5gIfC5QcPJq4EVkl4GXgS2AH9brnMwM7Oxo4jBl4XGn6ampmhu9qh0M7NKIGlVRDQdbzvPJGFmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhXJAWVmZhWprAEl6WpJ6yW1SLpriPXVkn6QrH9W0vySdXcn5eslfWS4xzQzs/GhbAElKQvcB1wDLAFukbRk0GafBtojYiHwJeDeZN8lwM3A2cDVwP+QlB3mMc3MbBwoZwtqGdASERsiogd4CLhu0DbXAd9M7j8CXClJSflDEdEdEW8ALcnxhnNMMzMbB3JlPPZsYHPJcivwvqNtExEFSR1AfVL+zKB9Zyf3j3dMACTdDtyeLHZKWj/C+jcAu0a4z3jjx6DIj0ORHwc/BgNG+zjMG85G5QwoDVEWw9zmaOVDtfgGH7NYGPEA8MCxKngskpojoumd7j8e+DEo8uNQ5MfBj8GAE/U4lLOLrxWYW7I8B9h6tG0k5YCpwJ5j7DucY5qZ2ThQzoBaCSyStEBSnuKgh+WDtlkO3JbcvwF4IiIiKb85GeW3AFgEPDfMY5qZ2ThQti6+5JrSncAKIAs8GBFrJN0DNEfEcuBrwLcltVBsOd2c7LtG0sPAWqAA/GFE9AEMdcwyncI77h4cR/wYFPlxKPLj4MdgwAl5HFRssJiZmVUWzyRhZmYVyQFlZmYVyQE1yMk6lZKkuZKelLRO0hpJf5yU10n6uaTXkp/Tx7qu5ZbMWvKCpH9MlhckU3G9lkzNlR/rOpabpGmSHpH0avKc+K2T9Lnwp8n/wyuSvi9pwsnwfJD0oKSdkl4pKRvy76+irySvmS9LuiCtejigSpzkUykVgH8fEWcBFwN/mJz7XcDjEbEIeDxZHu/+GFhXsnwv8KXkMWinOEXXePfXwE8jYjHwXoqPx0n1XJA0G/gjoCkizqE4MOtmTo7nwzcoTjNX6mh//2sojrReRHFyhPvTqoQD6kgn7VRKEbEtIp5P7u+n+II0myOno/omcP3Y1PDEkDQH+BfA3yXLAq6gOBUXnByPwRTgMoqjbImInojYy0n2XEjkgInJ5zQnAds4CZ4PEfEriiOrSx3t738d8K0oegaYJmlWGvVwQB1pqOmZZh9l23ErmVX+fOBZYGZEbINiiAEzxq5mJ8SXgf8A9CfL9cDeiCgkyyfDc+J0oA34etLV+XeSajjJngsRsQX4/4E3KQZTB7CKk+/5MOBof/+yvW46oI40nOmZxjVJtcD/Av4kIvaNdX1OJEkfBXZGxKrS4iE2He/PiRxwAXB/RJwPHGCcd+cNJbnGch2wADgVqKHYnTXYeH8+HE/Z/kccUEc6qadSklRFMZy+GxE/TIp3DDTXk587x6p+J8ClwLWSNlLs3r2CYotqWtLFAyfHc6IVaI2IZ5PlRygG1sn0XAD4EPBGRLRFRC/wQ+ASTr7nw4Cj/f3L9rrpgDrSSTuVUnKt5WvAuoj47yWrSqejug348Ymu24kSEXdHxJyImE/xb/9ERNwKPElxKi4Y548BQERsBzZLek9SdCXFWV1OmudC4k3gYkmTkv+PgcfhpHo+lDja33858IlkNN/FQMdAV+BoeSaJQST9DsV3zQNTKX1xjKt0Qkh6P/BPwGreuv7yFxSvQz0MnEbxH/bGiBh88XTckfRB4M8i4qOSTqfYoqoDXgB+PyK6x7J+5SZpKcWBInlgA/Apim9oT6rngqT/DNxEcZTrC8C/oXh9ZVw/HyR9H/ggxa/V2AH8JfC/GeLvn4T331Ac9dcFfCoimlOphwPKzMwqkbv4zMysIjmgzMysIjmgzMysIjmgzMysIjmgzMysIjmgzE4QSX2SXiy5pTY7g6T5pTNPm40HZfvKdzN7m4MRsXSsK2H2buEWlNkYk7RR0r2SnktuC5PyeZIeT75j53FJpyXlMyX9SNJLye2S5FBZSX+bfH/RzyRNTLb/I0lrk+M8NEanaTZiDiizE2fioC6+m0rW7YuIZRQ/kf/lpOxvKH6NwXnAd4GvJOVfAX4ZEe+lOEfemqR8EXBfRJwN7AV+Lym/Czg/Oc4flOvkzNLmmSTMThBJnRFRO0T5RuCKiNiQTNi7PSLqJe0CZkVEb1K+LSIaJLUBc0qn10m+IuXnyZfJIemzQFVEfEHST4FOilPV/O+I6CzzqZqlwi0os8oQR7l/tG2GUjofXB9vXWP+FxS/KfpCYFXJTNxmFc0BZVYZbir5+XRy/9cUZ1UHuBX4P8n9x4E7ACRlk2/AHZKkDDA3Ip6k+EWM04C3teLMKpHfSZmdOBMlvViy/NOIGBhqXi3pWYpvGm9Jyv4IeFDSn1P8httPJeV/DDwg6dMUW0p3UPzG16Fkge9Imkrxi+W+lHx9u1nF8zUoszGWXINqiohdY10Xs0riLj4zM6tIbkGZmVlFcgvKzMwqkgPKzMwqkgPKzMwqkgPKzMwqkgPKzMwq0v8Fn/PcJaMofrMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1, sgd_lr.epochs+1), sgd_lr.cost_)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Cost')\n",
    "plt.ylim([0, 0.2])\n",
    "plt.tight_layout()\n",
    "plt.show()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 6 - Stochastic Gradient Descent with Minibatches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Intercept: 0.24\n",
      "Slope: 0.82\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAGz1JREFUeJzt3Xl01fWZx/H3A9oFXFCh7iRapzUsypKyKhVBq9a2jlMcOmE8ehyRRSuH1tEW65Sx0CpaUUEggIISVxRRlK0sVWRNWGQfRQmKKODCYhAhfOePJ45DZbkh93d/d/m8zslJ7o+be5/rOX54+P6+i4UQEBGRzFEr7gJERKR6FNwiIhlGwS0ikmEU3CIiGUbBLSKSYRTcIiIZRsEtIpJhFNwiIhlGwS0ikmGOiuJF69evH/Lz86N4aRGRrFRWVrY1hNAgkedGEtz5+fmUlpZG8dIiIlnJzMoTfa6GSkREMoyCW0Qkwyi4RUQyjIJbRCTDKLhFRDKMgltEJMMouEVEMoyCW0SkprZvh9/8BjZtSsnbKbhFRGpi0iRo3BgeeACmT0/JWyq4RUSOxEcfQdeu8LOfQb16MG8eXHttSt5awS0iUh0hwJgxUFAAEybA3XdDWRm0bp2yEiLZq0REJCutWwc33QQzZsAFF8DIkXDuuSkvI6HgNrP1wA6gEtgbQiiMsigRkbSydy8MHgx33QVHHQXDhkH37lArnkGL6rxrxxBCM4W2iOSUJUt8GOS22+CSS2D1aujRY7/QLimB/Hy/lJ/vj6OkMW4RkQPZtQvuuAN+9CPYuBGeew5efBFOP32/p5WUePNdXu7D3+Xl/jjK8E40uAMwzczKzKx7dOWIiKSBWbPgvPPgnnvguuu8y/7lL8HsG0/t1w8qKva/VlHh16OSaHC3DyG0AC4HeptZh398gpl1N7NSMyvdsmVLUosUEUmJTz+F//gPuPhib59nzIBRo+CEEw76Kxs2VO96MiQU3CGED6q+bwYmAK0O8JziEEJhCKGwQYOETt8REUkPIcD48T7Fb8wYuP12WL7cA/wwGjas3vVkOGxwm1ldMzv2q5+BS4EV0ZUkIpJCGzfCP/8zdOni49eLFsFf/gLf/W5Cvz5gANSps/+1OnX8elQS6bhPBuaY2TJgIfBKCGFKdCWJiKTAvn0wfDg0agTTpsGgQbBgATRvXq2XKSqC4mLIy/Mh8Lw8f1xUFFHdgIUQkv6ihYWFQYcFi0jaWrsWbrwRXn8dOnWCESPg+9+PtSQzK0t0urWmA4pI7vjySx/DOO88WLECHnvMN4aKObSrS0veRSQ3LFjgM0ZWrIB//Vd48EE4+eS4qzoi6rhFJLvt3Al9+kDbtvDZZ/DSS/D00xkb2qCOW0Sy2ZQpvjy9vBx694aBA+G44+KuqsbUcYtI9tm6Fbp1g8sv97l5c+bAkCFZEdqg4BaRbBKCbxJSUADPPuu7+S1ZAu3bx11ZUmmoRESyQ3m5D4tMmQJt2vhe2U2axF1VJNRxi0hmq6z0GSKNG/u87Ice8qGRLA1tUMctIpls+XKf4rdwIVxxhR9wEOUmIWlCHbeIZJ4vvoA//AFatIB334Unn/TT1nMgtEEdt4hkmtdf9+Xqa9f6qep//SucdFLcVaWUOm4RyQzbtkHPntChA+zeDVOnwtixORfaoOAWkUwwcaLv4ldcDH37+rL1Sy+Nu6rYKLhFJH19+KHvk33VVVC/PsyfD/ffD3Xrxl1ZrBTcIpJ+QoDRo30hzcsv+1L10lI/uFd0c1JE0szbb/sx6bNmwY9/7MMjP/hB3FWlFXXcIpIe9uzxU9WbNoXFiz2wZ85UaB+AOm4RiV9ZmS+kWboUrr4aHn4YTjst7qrSljpuEYlPRQXcdhu0agUffQTPP+9fCu1DUsctIvH429/gppvgnXd8TPuee6BevbirygjquEUktT75BK6/Hi65BGrXhtmz/bBehXbCFNwikhohwDPP+BS/cePg97+HZct85ohUi4ZKRCR6770HvXr5RlCFhTBtGpx/ftxVZSx13CISnX37YOhQX64+c6avepw3T6FdQ+q4RSQaq1b5Ln5z5/q+IsOHw1lnxV1VVlDHLSLJtXs39O8PzZvDmjXw+ON+nJhCO2nUcYtI8syb5wtpVq2Cf/s3eOAB+N734q4q66jjFpGa27EDfv1rP019xw545RU/bV2hHQkFt4jUzCuv+EG9Q4bAzTfDypV+/qNERkMlInJkNm+GPn3gqad81sgbb0DbtnFXlRPUcYtI9YTgNxwLCnxfkf79YckShXYKqeMWkcS9+67vLzJ9OrRrByNHerctKaWOW0QOb+9eP029SRM/PmzoUD9tXaEdC3XcInJoy5b5FL/SUrjySnjkETjzzLirymnquEXkwHbt8o2gWraEDRt8g6iXXlJopwF13CLyTX//uy9Xf+st34L1vvvgxBPjrkqqqOMWka999pkfanDRRVBZ6TchH31UoZ1mFNwi4l54waf4jR7tx4ktXw6dO8ddlRyAhkpEct0HH/iKxwkToFkz3zO7Zcu4q5JDSLjjNrPaZrbEzCZFWZBILispgfx8qFXLv5eURPhm+/ZBcbF32ZMn+5mPCxcqtDNAdTruW4HVwHER1SKS00pKfHi5osIfl5f7Y4CioiS/2dq1/uKvvQYdO3qAn3NOkt9EopJQx21mZwA/BUZFW45I7urX7+vQ/kpFhV9Pmj17YOBAP4HmzTd9PHvGDIV2hkm04x4M/Cdw7MGeYGbdge4ADRs2rHllIjlmw4bqXa+2RYvghhv8pmOXLvDQQ3DKKUl6cUmlw3bcZnYlsDmEUHao54UQikMIhSGEwgYNGiStQJFccbB+p8Z90OefQ9++0KYNfPwxvPgiPPusQjuDJTJU0h74uZmtB54GLjazcZFWJZKDBgyAOnX2v1anjl8/YlOn+v4iDzzgm0OtWgW/+EWN6pT4HTa4Qwi/CyGcEULIB7oCM0MI3SKvTCTHFBX5PcK8PDDz78XFR3hjcutWuPZauOwy+Pa3/SbkI4/A8ccnvW5JPc3jFkkjRUU1nEESgh9scOutvgryzjv97uZ3vpO0GiV+1QruEMJsYHYklYhIzZSXQ8+ePie7dWvfK7tp07irkghoybtIpqus9BkijRv7kMjgwX6MmEI7a2moRCSTrVjhe2UvWODj2cOG+ZJLyWrquEUy0e7dcNdd0KIFvP02jBsHr76q0M4R6rhFMs0bb3iXvWYNdOvmR4pp7UROUcctkim2b4deveCCC/x0msmT4YknFNo5SMEtkgleeskP5h0+HPr08bHtyy6LuyqJiYJbJJ19+CFcc42vdjzxRJg3z1dBHnNM3JVJjBTcIukoBD8yrKAAJk6EP/3JT1lv3TruyiQN6OakSLpZt873FZkxAy680Ne9n3tu3FVJGlHHLZIu9u6FQYN8U6hFi3w8e/ZshbZ8gzpukXSweLFP8VuyxMezhw6F00+PuypJU+q4ReJUUQG33w6tWsGmTTB+vB/aq9CWQ1DHLRKXmTP93Md167zbvvdeOOGEuKuSDKCOWyTVPv3UjxDr1Mk33p4503fyU2hLghTcIqkSAjz3nE/xGzsW7rjDD+zt2DHuyiTDaKhEJBXefx969/YVkC1bwpQp0KxZ3FVJhlLHLRKlfft8q9VGjWD6dLjvPpg/X6EtNaKOWyQqq1fDjTf6bn6dO8OIEXD22XFXJVlAHbdIsn35Jdx9t3fVq1bBmDEwbZpCW5JGHbdIMs2f71P7Vq6Erl39GLGTT467Ksky6rhFkmHnTj9ZvV072LYNXn7ZT1tXaEsE1HGL1NTkydCjB7z3ns8cGTgQjj027qoki6njFjlSW7ZAURFccQXUrQtz5sDDDyu0JXIKbpHqCsGPDCso8AU1f/yjbw7Vrl3clUmO0FCJSHWsX+97ZU+bBm3b+lL1xo3jrkpyjDpukURUVvqRYY0bw9y5MGSID40otCUG6rhFDufNN32K36JF8NOf+krIM8+MuyrJYeq4RQ7miy+gXz/fW2T9enj6aZ/mp9CWmKnjFjmQv//d98r+n/+B667zPUZOOinuqkQAddwi+9u2zW8+XnQR7NnjNyEfe0yhLWlFwS3ylQkTfIrfqFHw29/C8uVwySVxVyXyDRoqEdm0CW6+GV54Ac4/38exW7aMuyqRg1LHLbkrBO+uCwrg1VfhL3/xmSMKbUlz6rglN731lt98nD3bx7OLi+Gf/inuqkQSoo5bcsuePd5ZN23qy9RHjfLDehXakkHUcUvuKC31hTTLlsG//ItvCHXqqXFXJVJt6rgl+33+uc8Sad3ad/SbMAHGj1doS8ZSxy3Zbfp0n5f97rv+/Z574Pjj465KpEYO23Gb2XfMbKGZLTOzlWbWPxWFidTIxx/7isdLL4Wjj/aVkMOHK7QlKyQyVLIbuDiEcD7QDLjMzNpEW5bIEQrB9xQpKICSErjzTh/T7tDhgE8vKYH8fKhVy7+XlKS0WpEjctihkhBCAHZWPTy66itEWZTIEdmwAXr1gldegVatYMYMnz1yECUlPiOwosIfl5f7Y/CDbUTSVUI3J82stpktBTYD00MIC6ItS6QaKit9f+zGjWHWLN83e+7cQ4Y2+MZ/X4X2Vyoq/LpIOksouEMIlSGEZsAZQCsza/KPzzGz7mZWamalW7ZsSXadIge2ciVceCHccgu0b++P+/SB2rUP+6sbNlTvuki6qNZ0wBDCZ8Bs4LID/FlxCKEwhFDYoEGDJJUnchC7d/tZj82b+9arTzzhp63n5yf8Eg0bVu+6SLpIZFZJAzOrV/Xzd4HOwJqoCxM5qLlzPbD794drroHVq6FbNzCr1ssMGAB16ux/rU4dvy6SzhLpuE8FZpnZm8AifIx7UrRliRzA9u2+i98FF/iimldfhXHj4Aj/hVdU5FuU5OV55ufl+WPdmJR0Zz5pJLkKCwtDaWlp0l9XctikSdCzJ2zcCL/+NfzpT3DMMXFXJZI0ZlYWQihM5Lla8i7p7aOPoGtX+NnPoF49mDcPBg9WaEtOU3BLegoBxozxhTQTJsDdd0NZme83IpLjtFeJpJ9163xfkRkzfDx75Eg499y4qxJJG+q4JX3s3eunqTdtCgsXwrBhvseIQltkP+q4JT0sWeJ7ZS9eDD//OQwdCmecEXdVImlJHbfEa9cuuOMO+NGPfMbIc8/Biy8qtEUOQR23xGfWLN/V6e234YYbYNAgOOGEuKsSSXvquCX1Pv3Uh0Uuvthnj8yY4Wc/KrRFEqLgltQJwY8MKyjwqX633w7Ll3uAi0jCNFQiqbFxI/TuDRMnQosWviFU8+ZxVyWSkdRxS7T27fMjwxo1gmnTfBx7wQKFtkgNqOOW6KxZAzfeCHPmQKdOMGIEfP/7cVclkvHUcUvyffmlbwJ1/vl+sMGjj/pp6wptkaRQxy3JtWCBzxhZscL3yn7wQTjllLirEskq6rglOXbu9CPD2rb16X4TJ8Izzyi0RSKgjltqbsoU6NHDj0nv1Qv+/Gc47ri4qxLJWuq45cht3epHhl1+uZ/5NWeO7zGi0BaJlIJbqi8EKCnxhTTPPgt33eWbRLVvH3dlIjlBQyVSPevX+7DI1KnQpo3vld2kSdxVieQUddySmMpKPzKscWMfEnnoIf+u0BZJOXXccnjLl/sUv4UL4Yor/ICDhg3jrkokZ6njloP74gu4807fW+Sdd+DJJ/20dYW2SKzUccuBvf66L1dfuxauvRbuvx/q14+7KhFBHbf8o23boGdP6NABdu/2m5Bjxyq0RdKIglu+NnGi7+JXXAx9+/qy9UsvjbsqEfkHCm6BTZugSxe46irvrOfP96GRunXjrkxEDkDBnctCgNGjvct++WUYOBBKS/3gXhFJW7o5maveegtuuskP7O3QwRfS/OAHcVclIglQx51r9uyBe+6B886DxYt9PHvWLIW2SAZRx51Lysp8Ic3SpXD11fDww3DaaXFXJSLVpI47F1RUwG23QatW8NFH8Pzz/qXQFslI6riz3d/+5mPZ77wD3bv7MEm9enFXJSI1oI47W338MVx/PVxyCdSuDbNn+2G9Cm2RjKfgzjYh+JFhjRrBuHHw+9/DsmXw4x/HXZmIJImGSrLJe+/50WGTJkFhIUyb5ieti0hWUcedDfbt8yPDGjWCmTN91eO8eQptkSyljjvTrVrlu/jNnevj2SNGwFlnxV2ViERIHXem2r0b+veHZs1gzRrfwW/qVIW2SA44bMdtZmcCjwOnAPuA4hDCg1EXJocwb54vpFm1Cn71Kz9S7Hvfi7sqEUmRRDruvcBvQggFQBugt5k1iras3FZSAvn5UKuWfy8pqfqDHTvgllv8NPUdO/wm5JNPKrRFcsxhO+4QwiZgU9XPO8xsNXA6sCri2nJSSYmvk6mo8Mfl5f74tCWv0PGZHrBxI9x8MwwYAMceG2+xIhKLao1xm1k+0BxYEEUxAv36fR3aAA3YzKiKX9Hx/ivhuOPgjTf8hHWFtkjOSji4zewY4HmgTwhh+wH+vLuZlZpZ6ZYtW5JZY07ZsOGrnwLXMpbVFHA1L/Bf9IclS6Bt2zjLE5E0kFBwm9nReGiXhBBeONBzQgjFIYTCEEJhgwYNklljTmnYEM7iHabyE8ZyHWs4l+YsYWzeXfCtb8VdnoikgcMGt5kZMBpYHUL4a/Ql5bC9e3m+/f2soAltmE9vhnAhr1NepxEDBsRdnIiki0Q67vbAvwMXm9nSqq8rIq4r9yxdCm3a0PLJ3/JJ80785PSVDLPeNMyrRXExFBXFXaCIpItEZpXMASwFteSmXbvgv/8bBg2Ck06CZ57hjC5dmGf6Ty4iB6Yl73GaPdvn+r31lm/Bet99cOKJcVclImlOS97j8Nlnvr9Ix45QWQnTp8Ojjyq0RSQhCu5Ue+EFKCjwoL7tNli+HDp3jrsqEckgGipJlQ8+8BWPEyb4xlCTJkHLlnFXJSIZSB131Pbtg+Ji77InT/YzHxcuVGiLyBFTxx2ltWv95uNrr/l4dnExnHNO3FWJSIZTxx2FPXt8E6jzz4c334TRo2HGDIW2iCSFOu5kW7jQ98pevhy6dPENoU45Je6qRCSLqONOls8/h759fROojz+GF1+EZ59VaItI0qnjToapU6FHD1i/Hnr2hD//GY4/Pu6qRCRLqeOuia1b4dpr4bLL4Nvf9puQjzyi0BaRSCm4j0QIfmRYQQE89RT84Q++SdSFF8ZdmYjkAA2VVFd5uQ+HTJ4MrVvDyJHQtGncVYlIDlHHnajKSp8h0rixD4k8+KAfI6bQFpEUU8ediBUrfIrfggU+nj18OOTlxV2ViOQoddyHsns33HUXtGgB69b5EeyvvqrQFpFYqeM+mDlzfOvVNWugWzd44AGoXz/uqkRE1HF/w/bt0KuXzxDZtQumTIEnnlBoi0jaUHD/fy+9BI0a+Rh2nz4+tv2Tn8RdlYjIfhTcAB9+CNdcA7/4hZ9CM2+eD40cc0zclYmIfENuB3cIfhJNQYF32wMGQFmZz88WEUlTuXtzct063yt75kwfzx45En74w7irEhE5rNzruPfuhUGDoEkTKC318ezZsxXaIpIxcqvjXrzYF9IsWeLj2UOHwumnx12ViEi15EbHXVEBt98OrVrBpk0wfrwf2qvQFpEMlP0d98yZPpa9bp132/feCyecEHdVIiJHLHs77k8+gRtugE6dwMwDfORIhbaIZLzsC+4Q4LnnfCHN2LFwxx1+YG/HjnFXJiKSFNk1VPL++9C7t8/JbtnSl6s3axZ3VSIiSZUdHfe+fTBsmHfZ06fDfffB/PkKbRHJSpnfca9e7bv4vfEGdO4MI0bA2WfHXZWISGQyt+P+8ku4+27vqletgjFjYNo0hbaIZL3M7Ljnz/epfStXQteuMHgwnHxy3FWJiKREZnXcO3fCrbdCu3awbRu8/LKfsq7QFpEckjkd9+TJ0KMHvPeezxwZOBCOPTbuqkREUi79O+4tW6CoCK64AurW9SPFHn5YoS0iOSt9gzsEPzKsoMAX1Pzxj745VLt2cVcmIhKr9BwqWb8ebrrJZ4m0bQujRvkcbRERSbOOu7LSjwxr3BjmzoUhQ3xoRKEtIvJ/DhvcZvaomW02sxVRFlJSAvfWvxf69mVG6MiEAav8JmSt9Pq7RUQkbokMlYwBhgCPR1VESYnvvHpURS8WcQ7jd/2SOr8zik/y+5IiIvK1w7azIYTXgE+iLKJfPz/rYDvHM54ugFFR4ddFRGR/SRuHMLPuZlZqZqVbtmyp1u9u2FC96yIiuSxpwR1CKA4hFIYQChs0aFCt323YsHrXRURyWVrc+RswAOrU2f9anTp+XURE9pcWwV1UBMXFkJfnp4zl5flj3ZgUEfmmw84qMbOngIuA+mb2PvBfIYTRyS6kqEhBLSKSiMMGdwjhV6koREREEpMWQyUiIpI4BbeISIZRcIuIZBgFt4hIhlFwi4hkGAshJP9FzbYA5Uf46/WBrUksJxPoM2e/XPu8oM9cXXkhhISWnUcS3DVhZqUhhMK460glfebsl2ufF/SZo6ShEhGRDKPgFhHJMOkY3MVxFxADfebsl2ufF/SZI5N2Y9wiInJo6dhxi4jIIaRNcKfqUOJ0YmZnmtksM1ttZivN7Na4a4qSmX3HzBaa2bKqz9s/7ppSxcxqm9kSM5sUdy2pYGbrzWy5mS01s9K464mamdUzs/Fmtqbq/+e2kb5fugyVmFkHYCfweAihSdz1pIKZnQqcGkJYbGbHAmXAVSGEVTGXFgkzM6BuCGGnmR0NzAFuDSHMj7m0yJlZX6AQOC6EcGXc9UTNzNYDhSGEnJjHbWZjgddDCKPM7FtAnRDCZ1G9X9p03Kk4lDjdhBA2hRAWV/28A1gNnB5vVdEJbmfVw6OrvtKjc4iQmZ0B/BQYFXctknxmdhzQARgNEEL4MsrQhjQK7lxnZvlAc2BBvJVEq2rIYCmwGZgeQsjqz1tlMPCfwL64C0mhAEwzszIz6x53MRE7G9gCPFY1HDbKzOpG+YYK7jRgZscAzwN9Qgjb464nSiGEyhBCM+AMoJWZZfWwmJldCWwOIZTFXUuKtQ8htAAuB3pXDYVmq6OAFsCwEEJz4HPgjijfUMEds6qx3ueBkhDCC3HXkypV/5ScDVwWcylRaw/8vGrM92ngYjMbF29J0QshfFD1fTMwAWgVb0WReh94///963E8HuSRUXDHqOpm3WhgdQjhr3HXEzUza2Bm9ap+/i7QGVgTb1XRCiH8LoRwRgghH+gKzAwhdIu5rEiZWd2qm+1UDRlcCmTtbLEQwofAe2b2w6pLnYBIJxgc9szJVEnVocRppj3w78DyqnFfgN+HEF6NsaYonQqMNbPaeNPwbAghJ6bH5ZiTgQnel3AU8GQIYUq8JUXuFqCkakbJO8D1Ub5Z2kwHFBGRxGioREQkwyi4RUQyjIJbRCTDKLhFRDKMgltEJMMouEVEMoyCW0Qkwyi4RUQyzP8CIO3yfplEP0gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.regressor import LinearRegression\n",
    "\n",
    "X = np.array([ 1.0, 2.1, 3.6, 4.2, 6])[:, np.newaxis]\n",
    "y = np.array([ 1.0, 2.0, 3.0, 4.0, 5.0])\n",
    "\n",
    "sgd_lr = LinearRegression(method='sgd',\n",
    "                          eta=0.01, \n",
    "                          epochs=100, \n",
    "                          random_seed=0, \n",
    "                          minibatches=3)\n",
    "sgd_lr.fit(X, y)\n",
    "\n",
    "print('Intercept: %.2f' % sgd_lr.b_)\n",
    "print('Slope: %.2f' % sgd_lr.w_)\n",
    "\n",
    "def lin_regplot(X, y, model):\n",
    "    plt.scatter(X, y, c='blue')\n",
    "    plt.plot(X, model.predict(X), color='red')    \n",
    "    return\n",
    "\n",
    "lin_regplot(X, y, sgd_lr)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl4HNWd7vHvT63FkmxJtiS8r9jEmM1gYXaSQEjMQDCZQDAh7DfOxk1mmEkGboZkhiRzJzczIcOEBEzYQlhCCAlOIHgI++JNNgYvYBDGi2zJli1rsfaWfvePLom2LFlbl9XI7+d5+lH3qVPVp1otvX2qTp8yd0dERCTZpAx2A0RERLqigBIRkaSkgBIRkaSkgBIRkaSkgBIRkaSkgBIRkaQUakCZ2Twz22hmJWZ2UxfLbzSzDWb2lpk9Z2aT45ZdbWbvBber48rnmNnaYJu3m5mFuQ8iIjI4LKzvQZlZBHgXOA8oBVYCl7v7hrg6nwSWu3u9mX0N+IS7X2Zmo4BioAhwYBUwx933mtkK4FvAMuBp4HZ3/0soOyEiIoMmzB7UXKDE3Te5ezPwKDA/voK7v+Du9cHDZcCE4P5ngGfdvdLd9wLPAvPMbCyQ4+5LPZasvwYuDnEfRERkkKSGuO3xwLa4x6XAKQepfz3Q3hPqat3xwa20i/IDmNlCYCFAdnb2nJkzZ/al7T16p7yW4RmpTBiZmdDtiogMdatWrdrt7oU91QszoLo6N9Tl8UQz+xKxw3kf72HdXm/T3RcBiwCKioq8uLi4p/b2yRn//jynTBvFT78wO6HbFREZ6sxsS2/qhXmIrxSYGPd4ArCjcyUz+xTwXeAid2/qYd1SPjwM2O02D4VIitHWpnkMRUTCEmZArQRmmNlUM0sHFgCL4yuY2YnAXcTCaVfcoiXAp81spJmNBD4NLHH3MqDWzE4NRu9dBTwZ4j50K5JitCqfRERCE9ohPnePmtkNxMImAtzr7uvN7Fag2N0XAz8BhgO/C0aLb3X3i9y90sx+QCzkAG5198rg/teA+4FMYuesBmUEX4qhHpSISIjCPAeFuz9NbCh4fNn34u5/6iDr3gvc20V5MXBsApvZL6kpKUTb2ga7GSIiQ5ZmkuinlBSjVfkkIhIaBVQ/RVKgTRd7FBEJjQKqnyJmtOoclIhIaBRQ/ZSSYupBiYiESAHVT6kpRlTjzEVEQqOA6qcUM1rVgxIRCY0Cqp80k4SISLgUUP0Um0lCASUiEhYFVD+lmHpQIiJhUkD1U2qKEVVAiYiERgHVT7GZJBRQIiJhUUD1U8T0PSgRkTApoPopoh6UiEioFFD9FJtJYrBbISIydCmg+ik2SELTmYuIhEUB1U+xYeaD3QoRkaFLAdVPkRR0DkpEJEQKqH7STBIiIuFSQPWTZpIQEQmXAqqfNJOEiEi4Qg0oM5tnZhvNrMTMbupi+dlmttrMomZ2SVz5J81sTdyt0cwuDpbdb2YfxC2bHeY+dCdFs5mLiIQqNawNm1kEuAM4DygFVprZYnffEFdtK3AN8I/x67r7C8DsYDujgBLgf+KqfNvdHw+r7b0R0fWgRERCFVpAAXOBEnffBGBmjwLzgY6AcvfNwbKDDdi+BPiLu9eH19S+00wSIiLhCvMQ33hgW9zj0qCsrxYAj3Qq+5GZvWVmt5lZRlcrmdlCMys2s+KKiop+PO3BxWaSUECJiIQlzICyLsr69B/dzMYCxwFL4opvBmYCJwOjgH/qal13X+TuRe5eVFhY2Jen7RUNkhARCVeYAVUKTIx7PAHY0cdtfAH4g7u3tBe4e5nHNAH3ETuUeMilmOEOrl6UiEgowgyolcAMM5tqZunEDtUt7uM2LqfT4b2gV4WZGXAxsC4Bbe2zSEqsg6jzUCIi4QgtoNw9CtxA7PDc28Bj7r7ezG41s4sAzOxkMysFLgXuMrP17eub2RRiPbCXOm36ITNbC6wFCoAfhrUPB9MRUOpBiYiEIsxRfLj708DTncq+F3d/JbFDf12tu5kuBlW4+zmJbWX/pFgsoDRhrIhIODSTRD+lBj0oXXJDRCQcCqh+SklRD0pEJEwKqH6KBIPodQ5KRCQcCqh+0ig+EZFwKaD6qeMQn3pQIiKhUED104eDJBRQIiJhUED104fDzBVQIiJhUED1k85BiYiESwHVT5pJQkQkXAqoftIhPhGRcCmg+kmDJEREwqWA6qcUnYMSEQmVAqqfIqbvQYmIhEkB1U8axSciEi4FVD9pJgkRkXApoPqpY5BEqwJKRCQMCqh+ah9mru9BiYiEQwHVTxFdD0pEJFQKqH6KBK+celAiIuFQQPWTZpIQEQlXqAFlZvPMbKOZlZjZTV0sP9vMVptZ1Mwu6bSs1czWBLfFceVTzWy5mb1nZr81s/Qw96E7qSmxl04zSYiIhCO0gDKzCHAHcD4wC7jczGZ1qrYVuAZ4uItNNLj77OB2UVz5j4Hb3H0GsBe4PuGN74WU9kN8CigRkVCE2YOaC5S4+yZ3bwYeBebHV3D3ze7+FtCroQZmZsA5wONB0QPAxYlrcu9F9D0oEZFQhRlQ44FtcY9Lg7LeGmZmxWa2zMzaQygfqHL3aE/bNLOFwfrFFRUVfW17j9qnOlIPSkQkHKkhbtu6KOvLf/NJ7r7DzKYBz5vZWqCmt9t090XAIoCioqKEp4hmkhARCVeYPahSYGLc4wnAjt6u7O47gp+bgBeBE4HdQJ6ZtQdrn7aZSJpJQkQkXGEG1EpgRjDqLh1YACzuYR0AzGykmWUE9wuAM4AN7u7AC0D7iL+rgScT3vJeSE+NvXRNUX1TV0QkDKEFVHCe6AZgCfA28Ji7rzezW83sIgAzO9nMSoFLgbvMbH2w+tFAsZm9SSyQ/t3dNwTL/gm40cxKiJ2TuiesfTiY3Mw0AKobWgbj6UVEhrwwz0Hh7k8DT3cq+17c/ZXEDtN1Xu914LhutrmJ2AjBQZWZFiE9kqKAEhEJiWaS6CczIyczjeqG5sFuiojIkKSAGoC8rDT1oEREQqKAGoDczDSq6hVQIiJhUEANQG6melAiImFRQA1AnnpQIiKhUUANQE5mGjXqQYmIhEIBNQB5WWnUNkWJturLuiIiiaaAGoD2L+vWNEZ7qCkiIn2lgBqAvCzNJiEiEhYF1AC096Cq6vVlXRGRRFNADUBuZuxq8+pBiYgkngJqADRhrIhIeBRQA6BzUCIi4VFADcCH56AUUCIiiaaAGoC0SArZ6RH1oEREQqCAGiBNGCsiEg4F1ADlZqWrByUiEgIF1ADlZqbqooUiIiFQQA1QXqZ6UCIiYVBADZDOQYmIhCPUgDKzeWa20cxKzOymLpafbWarzSxqZpfElc82s6Vmtt7M3jKzy+KW3W9mH5jZmuA2O8x96Iku+y4iEo7UsDZsZhHgDuA8oBRYaWaL3X1DXLWtwDXAP3ZavR64yt3fM7NxwCozW+LuVcHyb7v742G1vS9yMtNoirbR2NLKsLTIYDdHRGTICC2ggLlAibtvAjCzR4H5QEdAufvmYNl+F1Ry93fj7u8ws11AIVBFkomfTUIBJSKSOGEe4hsPbIt7XBqU9YmZzQXSgffjin8UHPq7zcwyullvoZkVm1lxRUVFX5+21zSbhIhIOMIMKOuizPu0AbOxwIPAte7e3su6GZgJnAyMAv6pq3XdfZG7F7l7UWFhYV+etk/yNKO5iEgowgyoUmBi3OMJwI7ermxmOcBTwD+7+7L2cncv85gm4D5ihxIHja4JJSISjjADaiUww8ymmlk6sABY3JsVg/p/AH7t7r/rtGxs8NOAi4F1CW11H2lGcxGRcIQWUO4eBW4AlgBvA4+5+3ozu9XMLgIws5PNrBS4FLjLzNYHq38BOBu4povh5A+Z2VpgLVAA/DCsfeiNHF0TSkQkFGGO4sPdnwae7lT2vbj7K4kd+uu83m+A33SzzXMS3MwBGZGRSoopoEREEk0zSQxQSoqRo9kkREQSTgGVAHmZmk1CRCTRFFAJkJuZRpUCSkQkoRRQCaBrQomIJJ4CKgFyM9Oo1vegREQSSgGVADoHJSKSeAqoBMgNAqqtrU8zOYmIyEEooBIgLyuNNod9zdHBboqIyJChgEqAjtkk9F0oEZGEUUAlQJ6mOxIRSTgFVALkKqBERBJOAZUAeVmxa0JpuiMRkcRRQCWAelAiIomngEqA9mtCVTXoy7oiIomigEqAYWkR0lNT1IMSEUmgXgWUmT3Ym7LDWV5mmoaZi4gkUG97UMfEPzCzCDAn8c356BqVnU5FbdNgN0NEZMg4aECZ2c1mVgscb2Y1wa0W2AU8eUha+BExaVQWWyvrB7sZIiJDxkEDyt3/r7uPAH7i7jnBbYS757v7zYeojR8JUwqy2VJZr/n4REQSpLeH+P5sZtkAZvYlM/upmU0OsV0fOZNGZdEcbWNnbeNgN0VEZEjobUD9Eqg3sxOA7wBbgF/3tJKZzTOzjWZWYmY3dbH8bDNbbWZRM7uk07Krzey94HZ1XPkcM1sbbPN2M7Ne7kOopuRnA7B5tw7ziYgkQm8DKuruDswH/svd/wsYcbAVgoEUdwDnA7OAy81sVqdqW4FrgIc7rTsK+D5wCjAX+L6ZjQwW/xJYCMwIbvN6uQ+hmpyfBcDWyrpBbomIyNDQ24CqNbObgSuBp4LwSethnblAibtvcvdm4FFiAdfB3Te7+1tAW6d1PwM86+6V7r4XeBaYZ2ZjgRx3XxoE5q+Bi3u5D6EamzuM1BRj8x71oEREEqG3AXUZ0ARc5+7lwHjgJz2sMx7YFve4NCjrje7WHR/c73GbZrbQzIrNrLiioqKXT9t/qZEUJo7KYqsCSkQkIXoVUEEoPQTkmtmFQKO793QOqqtzQ70d4tbdur3eprsvcvcidy8qLCzs5dMOzKRRWWzRIT4RkYTo7UwSXwBWAJcCXwCWdx7U0IVSYGLc4wnAjl62q7t1S4P7/dlm6KbkZ7Fldz2xo48iIjIQvT3E913gZHe/2t2vInZ+6ZYe1lkJzDCzqWaWDiwAFvfy+ZYAnzazkcHgiE8DS9y9jNj5sFOD0XtXkURfGJ6Un01tU5S9mvJIRGTAehtQKe6+K+7xnp7WdfcocAOxsHkbeMzd15vZrWZ2EYCZnWxmpcR6ZneZ2fpg3UrgB8RCbiVwa1AG8DXgV0AJ8D7wl17uQ+gmj4qN5Nu8R4f5REQGKrWX9Z4xsyXAI8Hjy4Cne1rJ3Z/uXM/dvxd3fyX7H7KLr3cvcG8X5cXAsb1s9yE1pSAYar6nnpMmjeyhtoiIHMxBA8rMpgOj3f3bZva3wJnEBiosJTZoQuJMGJmFmXpQIiKJ0NMhvp8BtQDu/oS73+juf0+sV/SzsBv3UTMsLcLYnGEaai4ikgA9BdSU4Iu0+wkOs00JpUUfcZPys9iiWc1FRAasp4AadpBlmYlsyFAxeVQ2W3SIT0RkwHoKqJVm9uXOhWZ2PbAqnCZ9tE0uyGL3vmb2NUUHuykiIh9pPY3i+zvgD2Z2BR8GUhGQDnwuzIZ9VE0eFZvVfMueOo4ZlzvIrQmHu+MOKSlJMZG8iAxRBw0od98JnG5mn+TDod1PufvzobfsI6pjVvM99UMyoMqqG7ju/mKmFmTxiyvmDHZzRGQI6+1cfC+4+38HN4XTQbQHVDLNat7Y0sqmin0D3s7m3XVc8sulvFNew9Nry3np3XAn4S3ZtY8l68tDfQ756HF3nnt7J7sSdHHQaGsbm3fXhT5FWWub8/TaMmoaNdNMb/V2JgnppRHD0hiVnd7r60IN5I9ib10zj6zYSmNLa7d1qutbWLBoGZ/66Us8/87Ofj/Xhh01XHLnUhpaWnn8q6czOT+LHz21gWjrh1dK2VSxj68/tIr1O6r7/Tzt/rphJ/N//ipfeXAVz27of7tl6Ln9uRKuf6CYz93xOiW7BvbB6/X3d3PB7a/yif94kfl3vMb/rC+nra1vf5PuzpL15Ww7yOjdxpZWbnh4NV9/aDU3P7F2QG0+nCigQjA5P4stB+lBtbU5f35rB5+57WUuuP1VNpbX9vk5tlc1cMmdr3PzE2u56p4VVDcc+Kls974mFty9jA07apiSn83/fvgN3i6r6VheWdfMt3/3Jr94sYSW1s6X5IKSXbXc8+oHXHPfCj73i9dIixiPfeU05kweyU3zZvLuzn08Vhy7+klZdQNX3rOCp9eW88W7l/Pmtqpu294cbaN0b9evj7uz6OX3+fKDxUwrHM6ssTl85/E32VmTmE/L8tF298ubuO2v7/LpWaNpirZxyZ2vs3rr3j5vp7y6kW88tJov3r2cuuYoN553FFX1LSx8cBV/c/srrNveuw9Z0dY2bnlyHV95cBVf/NUyquqbD6hTVd/Ml361nGfWl3PqtFE89VYZL7yzq4utSWd2OMy8XVRU5MXFxYfs+f7u0Td4Zn05J0zIIzVipEVSOGJEBmNyM8kZlsrvikvZuLOWIwuzqW5ooaYxynf/5miuOm0ypXsb+NNbO3hpYwU5mWlMGpXFpFFZHDs+h+Mn5JEWSWFjeS1X37uCuuYo150xlV+8WMKRhcN54Lq5jM6JfTOgdG89V9+7gu1VDdx1ZREzx4xg/s9fI5Ji/OEbp/Nu+T5ufGwNe+qaaW1zZo4Zwf+75Hhmjc3hmfXl3PPqB7yxNRYy0wqyOfuoQhaePY1xebFvF7g7X7hrKR/sruMPXz+Da+9fSXl1I//5hRP44VMbqKpr4f7rTmbO5FEdr0tztI3fry7l58+XsL2qgXnHjOGWz85ifLDNd8pr+O/nSnhqbRkXHDeW/7j0BHZUN3Dh7a9y0uQ8HrzulD4NzGhtc1Zv3UtdMKLSzDhq9HDG5nb9DYmmaCt/frOMP67ZzvETcvnqx49kxLCersvZe7v3NfFWaRXHjsvliJwPv8HR2uY8vGIrb2zdy43nHcWEkVn7rdccbeOd8hrWbKtizdYqKuubGZWdTsHwDCaOzOTzcyaQld716WR358V3K3hjaxWzJ+YyZ/IocjP7tk+1jS08u2Enp0zL7/hdHQqNLa1sq6xn+LBURgxL449vbOef/7iOC44by38tmM2Oqkauunc55TWN/OyyE5l37JhebXfd9mquvX8ltY0tfP0T01l49jSGpUWItrax+M0d/GTJRuqbW3nky6cya1xOt9upa4pyw8OreWFjBX974nj+/FYZp0wbxf3XziUSvE83Vezjy78uZtveBm77wmzOmzWav7n9FRqaW3n2xrO7/b311SvvVfDPf1zHladO5vozpxKbS/vg9uxr4ra/vsuZ0wuYd+zYhLSjt8xslbsX9VhPAZV4r7xXwZ0vvU9Lq9PW5jRGW9lV00TFvibcYVphNt86dwYXHj+OyrpmvvP4m7ywsYLxeZlsr2oA4JhxObS0trGtsoGG4BBeVnqEoimjWLN1L8PSIvz6+rnMHJPDq+/t5isPFpOXlc6scTls2FHD9qoGhmekcs/VRZwyLR+I/WFeeudSRmalUVbTyLSCbG6//ES2723glifXUVHbRMHwDHbVNjElP4urTpvCebNGM3FUVpf7+ea2Kubf8RpZ6RFa25wHrpvLqdPyKatu4It3L2dnTSMXnxi7nqQ7vPxuBdurGjhhYh6nThvFA69vBuC6M6aydns1r7y3m8y0CF/7xJHc8MnpHWH06Iqt3PTEWm4+fyZf+fiRHc/fHG1ja2U9W/bU0eZwZGE2k0Zl0Rht43fF27j/9c0H9GTN4MzpBXz+pAmcMm0Ue/Y1U17dyLod1Ty0fCsVtU0dv4eC4en8/XlHcVnRRFIjXR9scHfKaxp5u6yGyroWmqNtNEVbaY62EW1zmqNt7K1vZsUHlbwT9JTTU1O4dM4EvnL2kZRVN/Avf9rA22U1RFKMYakp3HT+TK44ZTK7apt4YOlmHl6+taOHXDA8gzG5Geyta2H3viaaom2Myx3GP184i/OPHbPfP6ZVWyr58V82smJzZUeZGRw9JofPHDOGi2aPY2pBdrfv48aWVn6zbAt3vFDC3voWstIj3HjeUVxz+hRSIym4O1sr63m/Yh9tQQe8MdpKya59bCyv5d2dtTRF20hNMVJSjCn52XzuxPGcN2s0w9IiNEfbeGPrXtZsq+JjY0Zw6rR8hqVFqG+O8ptlW1j08iZ279u/R3LOzCO480tzSE+N/T5272vi+vtX8mZpNRfPHsf3P3sMI7PTO343FbVN5A/P6AiMl96t4Ou/WUVeVjr3XXsyR40eccB+b6us57K7ltIYbeO3C09lRqc67s5rJXv4t6ff5p3yGm6dfyxfOnVyx/v06584kn/49Me477UP+MmSjWSmR1h0ZRFzp8Y+rK3cXMmldy7ly2dN5bsXzOrytW9t84429+TBpZv5lz9tICstQm1TlCtOmcS/XHQMacF7tjnaRkNL634fTJZv2sM3H32DnTVNAFxxyiRuuXAWw9IivXrOvrSvKwqoOIc6oLrT0tpGZV0zBXF/MBB7wz+4bAv/s34nZ84o4ILjxnaEQvsf2eqtVbz+/m5eK9nN8GFp/PzyE/cLjnXbq/nmI29gBrPG5XLMuBzOmzWaIwuH79eGJevLueHh1VwyZwK3XDir4xNcTWML/7lkI1sr67l87iTOPXp0r96AN/52DU++uYM7vzSH82aN7ijfVdPI1x9azQe764j9zzSmFmTxjU9O5+NHFWJmbK9q4Ad/2sAz68spHJHBNadP4YpTJpGXlb7fc7g7X39oNUvWl5OXlU5KsL3KuiY6ny5Ij6QQSTEaWlo5aVIeV58+hYmjsnCP/VG9WrKbJ1aXUrq34YB9+fhRhVx/5lTOmlHAW6XV/PCpDazcvJcUi4VKRmqEYWkpZGekMjwjlfRICpt211FZd+BhnXiZaRHmTB7JaUfmc/yEXJ5eW87vV5USbWujzWF8XibfveBojp+Qy81PrOWV93ZzZGE2W/bU0+bOp2eN4cITxjJ7Yh7j8zI7QsjdKd6yl+89uZ63y2o4bVo+Uwqyqahtoqy6gfU7aigckcE3z53B/NnjWLe9mpUf7OW1kt0doXXc+Fw+NmYE2ekRsjJSaXOnpqGFqvoW3txWxY7qRs6aUcB1Z0zlwWVbeP6dXcwam8Ox43N4rWRPxweqeGYwJT+bo0YPJzsjldY2J9oa682WVTcyYlgqx43PZc22KuqbPzx/OiwthVOm5rNuezV76po5c3oBnztxPE3RNmobW0iLpPDFUyYd8E+0KdrKHS+8zy9eKCEvK42rT5vCu7v2sWzTHipqm8hOj3DCxDwm52fzWPE2PjZ6BPdde3LH0YaufLC7jsvuWooD//a548hKjz3nxvJafrNsC5t215Gfnc5/XHoCn5x5RMd6Nz+xlkdWbGXmmBG8U17Lp44ezb997tj9eszt9X67cis/uPhYGppbKd3bQOneBsqqGyirbqSyrpnMtAgjs9IYmZ3OuLxMphZkMyU/m9E5GQxLi5CRmsKf3tzBA0u3cO7MI7htwWx++eL7/PLF9zlrRgGfPX4cL2zcxSvv7WZfU5RZY3M4Y3o+kZQUFr38PpPzs7ntstn8ZW0Zd728iZljRvDtz3yMvKx0sjMipKakUNPYQk1DC5V1zby3ax/vlNXwTnkt844dw/c/e8xB3/cHo4CKkywBlSwaW1p7/UmpJ83RNsqrG5mU33Uvqze2VdZzRE4GGandt6m6oYW7X95EdUMLre64O4XDM5hSkM2UgmyM2Ki/kop91DVF+fxJEzixmxnl29qcFZsreW/XPo4YkcHonGGMz8ukcETGfvXcnb++vYu3SqtoirbFPok2t7KvOUpdU5TGllam5Gcza1wOR4/NYUzOMDJSU0gPbqkpKaRFrMvDLTtrGnlw6RYy0yNcd8ZUMoN/gO7O71aVct9rmzltWj7XnjGl2x5su2hrGw+v2Mrtz5UATsHwDI7IGcap00ZxzelTujyMVFbdwFNvlfH02jLKqxupa26lvjmKYeRkppGXlca4vEy+cvY0zphe0NG2Z9aV869/2kBDSyunTcvnjOn5HDM+l9QUwzAiKcbUguyO/YnX2uYs27SH368uZcOOGoqmjOTM6YWcNDmPDTtqeOGdXbxSspuJI7P45rnT9zs83BsbdtTwnd+/ybrtNYzOyeDUafkcNz6XrZX1vLG1irfLajhrRgH//cWTGJ7R86G1kl21LFi07IBe3EmT8rjytMmcf+zYLsPyi3cv572dtfzr/GO4ePb4Ln//1Q0tfOqnL1FRG+vBDM9IZXxeJuPyhjEuL5OC4RnUBdeW21vfzLbKerZU1tMcPfBc8ZfPmspN5x/d8YHyseJt/J8n1hJtc8bkDOOTMwsZk5PJsk17WLVlL82tbcyfPY4ffe64jtfhxY27+IfH3mTPQT5spUWMIwuHc/TYHM6ZeQSfPWFcj69hdxRQcRRQIomTzF/Ubm1z9uxronBExgHBEG1t6/ZQbXf21jVTEvcVjZFZaUw/4sDDgvGaoq1EW53sHkKwrLqB3bXNTByVSW5mWo/njVrbnB1VDeypa6appZXGaBu5mWnMnph3QN3Nu+uob27l6LEj9ttuQ3MrZdUNTC3IPuD5qhtaeG9nbezDSlOU5tY2cjLTyA1uk0ZldRw2HCgFVBwFlIhI8uhtQGmYuYiIJCUFlIiIJCUFlIiIJCUFlIiIJKVQA8rM5pnZRjMrMbObulieYWa/DZYvN7MpQfkVZrYm7tZmZrODZS8G22xfdkTn7YqIyEdfaAFlZhHgDuB8YBZwuZl1/tr09cBed58O3Ab8GMDdH3L32e4+G7gS2Ozua+LWu6J9ubtrUisRkSEozB7UXKDE3Te5ezPwKDC/U535wAPB/ceBc+3ALwNcDjwSYjtFRCQJhRlQ44FtcY9Lg7Iu67h7FKgG8jvVuYwDA+q+4PDeLV0EGgBmttDMis2suKIi3OsWiYhI4oUZUF0FR+dvBR+0jpmdAtS7+7q45Ve4+3HAWcHtyq6e3N0XuXuRuxcVFhb2reUiIjLowgyoUmBi3OMJwI7u6phZKpALVMYtX0Cn3pO7bw9+1gIPEzuUKCIiQ0yYAbUSmGFmU80snVjYLO5UZzFwdXD/EuB5D+ZeMrMU4FJi564IylLNrCC4nwZcCKxDRESGnMTagmYsAAAMFElEQVRcLasL7h41sxuAJUAEuNfd15vZrUCxuy8G7gEeNLMSYj2nBXGbOBsodfdNcWUZwJIgnCLAX4G7w9oHEREZPJosVkREDilNFisiIh9pCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKCigREUlKoQaUmc0zs41mVmJmN3WxPMPMfhssX25mU4LyKWbWYGZrgtudcevMMbO1wTq3m5mFuQ8iIjI4QgsoM4sAdwDnA7OAy81sVqdq1wN73X06cBvw47hl77v77OD21bjyXwILgRnBbV5Y+yAiIoMnzB7UXKDE3Te5ezPwKDC/U535wAPB/ceBcw/WIzKzsUCOuy91dwd+DVyc+KaLiMhgCzOgxgPb4h6XBmVd1nH3KFAN5AfLpprZG2b2kpmdFVe/tIdtAmBmC82s2MyKKyoqBrYnIiJyyIUZUF31hLyXdcqASe5+InAj8LCZ5fRym7FC90XuXuTuRYWFhX1otoiIJIMwA6oUmBj3eAKwo7s6ZpYK5AKV7t7k7nsA3H0V8D5wVFB/Qg/bFBGRISDMgFoJzDCzqWaWDiwAFneqsxi4Orh/CfC8u7uZFQaDLDCzacQGQ2xy9zKg1sxODc5VXQU8GeI+iIjIIEkNa8PuHjWzG4AlQAS4193Xm9mtQLG7LwbuAR40sxKgkliIAZwN3GpmUaAV+Kq7VwbLvgbcD2QCfwluIiIyxFhsMNzQVlRU5MXFxYPdDBERAcxslbsX9VRPM0mIiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSUkCJiEhSCjWgzGyemW00sxIzu6mL5Rlm9ttg+XIzmxKUn2dmq8xsbfDznLh1Xgy2uSa4HRHmPoiIyOBIDWvDZhYB7gDOA0qBlWa22N03xFW7Htjr7tPNbAHwY+AyYDfwWXffYWbHAkuA8XHrXeHuxWG1XUREBl+YPai5QIm7b3L3ZuBRYH6nOvOBB4L7jwPnmpm5+xvuviMoXw8MM7OMENsqIiJJJsyAGg9si3tcyv69oP3quHsUqAbyO9X5PPCGuzfFld0XHN67xcysqyc3s4VmVmxmxRUVFQPZDxERGQRhBlRXweF9qWNmxxA77PeVuOVXuPtxwFnB7cquntzdF7l7kbsXFRYW9qnhIiIy+MIMqFJgYtzjCcCO7uqYWSqQC1QGjycAfwCucvf321dw9+3Bz1rgYWKHEkVEZIgJM6BWAjPMbKqZpQMLgMWd6iwGrg7uXwI87+5uZnnAU8DN7v5ae2UzSzWzguB+GnAhsC7EfRARkUESWkAF55RuIDYC723gMXdfb2a3mtlFQbV7gHwzKwFuBNqHot8ATAdu6TScPANYYmZvAWuA7cDdYe2DiIgMHnPvfFpo6CkqKvLiYo1KFxFJBma2yt2LeqqnmSRERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQpKaBERCQphRpQZjbPzDaaWYmZ3dTF8gwz+22wfLmZTYlbdnNQvtHMPtPbbYqIyNAQWkCZWQS4AzgfmAVcbmazOlW7Htjr7tOB24AfB+vOAhYAxwDzgF+YWaSX2xQRkSEgzB7UXKDE3Te5ezPwKDC/U535wAPB/ceBc83MgvJH3b3J3T8ASoLt9WabIiIyBKSGuO3xwLa4x6XAKd3VcfeomVUD+UH5sk7rjg/u97RNAMxsIbAweLjPzDb2sf0FwO4+rjPU6DWI0esQo9dBr0G7gb4Ok3tTKcyAsi7KvJd1uivvqsfXeZuxQvdFwKKDNfBgzKzY3Yv6u/5QoNcgRq9DjF4HvQbtDtXrEOYhvlJgYtzjCcCO7uqYWSqQC1QeZN3ebFNERIaAMANqJTDDzKaaWTqxQQ+LO9VZDFwd3L8EeN7dPShfEIzymwrMAFb0cpsiIjIEhHaILzindAOwBIgA97r7ejO7FSh298XAPcCDZlZCrOe0IFh3vZk9BmwAosA33L0VoKtthrQL/T48OIToNYjR6xCj10GvQbtD8jpYrMMiIiKSXDSThIiIJCUFlIiIJCUFVCeH61RKZjbRzF4ws7fNbL2ZfSsoH2Vmz5rZe8HPkYPd1rAFs5a8YWZ/Dh5PDabiei+Ymit9sNsYNjPLM7PHzeyd4D1x2mH6Xvj74O9hnZk9YmbDDof3g5nda2a7zGxdXFmXv3+LuT34n/mWmZ2UqHYooOIc5lMpRYF/cPejgVOBbwT7fhPwnLvPAJ4LHg913wLejnv8Y+C24DXYS2yKrqHuv4Bn3H0mcAKx1+Owei+Y2Xjgm0CRux9LbGDWAg6P98P9xKaZi9fd7/98YiOtZxCbHOGXiWqEAmp/h+1USu5e5u6rg/u1xP4hjWf/6ageAC4enBYeGmY2AbgA+FXw2IBziE3FBYfHa5ADnE1slC3u3uzuVRxm74VAKpAZfE8zCyjjMHg/uPvLxEZWx+vu9z8f+LXHLAPyzGxsItqhgNpfV9Mzje+m7pAVzCp/IrAcGO3uZRALMeCIwWvZIfEz4DtAW/A4H6hy92jw+HB4T0wDKoD7gkOdvzKzbA6z94K7bwf+A9hKLJiqgVUcfu+Hdt39/kP7v6mA2l9vpmca0sxsOPB74O/cvWaw23MomdmFwC53XxVf3EXVof6eSAVOAn7p7icCdQzxw3ldCc6xzAemAuOAbGKHszob6u+HnoT2N6KA2t9hPZWSmaURC6eH3P2JoHhne3c9+LlrsNp3CJwBXGRmm4kd3j2HWI8qLzjEA4fHe6IUKHX35cHjx4kF1uH0XgD4FPCBu1e4ewvwBHA6h9/7oV13v//Q/m8qoPZ32E6lFJxruQd4291/Grcofjqqq4EnD3XbDhV3v9ndJ7j7FGK/++fd/QrgBWJTccEQfw0A3L0c2GZmHwuKziU2q8th814IbAVONbOs4O+j/XU4rN4Pcbr7/S8GrgpG850KVLcfChwozSTRiZn9DbFPze1TKf1okJt0SJjZmcArwFo+PP/yf4idh3oMmETsD/ZSd+988nTIMbNPAP/o7hea2TRiPapRwBvAl9y9aTDbFzYzm01soEg6sAm4ltgH2sPqvWBm/wpcRmyU6xvA/yJ2fmVIvx/M7BHgE8Quq7ET+D7wR7r4/Qfh/XNio/7qgWvdvTgh7VBAiYhIMtIhPhERSUoKKBERSUoKKBERSUoKKBERSUoKKBERSUoKKJFDxMxazWxN3C1hszOY2ZT4madFhoLQLvkuIgdocPfZg90IkY8K9aBEBpmZbTazH5vZiuA2PSifbGbPBdfYec7MJgXlo83sD2b2ZnA7PdhUxMzuDq5f9D9mlhnU/6aZbQi28+gg7aZInymgRA6dzE6H+C6LW1bj7nOJfSP/Z0HZz4ldxuB44CHg9qD8duAldz+B2Bx564PyGcAd7n4MUAV8Pii/CTgx2M5Xw9o5kUTTTBIih4iZ7XP34V2UbwbOcfdNwYS95e6eb2a7gbHu3hKUl7l7gZlVABPip9cJLpHybHAxOczsn4A0d/+hmT0D7CM2Vc0f3X1fyLsqkhDqQYkkB+/mfnd1uhI/H1wrH55jvoDYlaLnAKviZuIWSWoKKJHkcFncz6XB/deJzaoOcAXwanD/OeBrAGYWCa6A2yUzSwEmuvsLxC7EmAcc0IsTSUb6JCVy6GSa2Zq4x8+4e/tQ8wwzW07sQ+PlQdk3gXvN7NvErnB7bVD+LWCRmV1PrKf0NWJXfO1KBPiNmeUSu7DcbcHl20WSns5BiQyy4BxUkbvvHuy2iCQTHeITEZGkpB6UiIgkJfWgREQkKSmgREQkKSmgREQkKSmgREQkKSmgREQkKf1/r6EdopNRh/0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1, sgd_lr.epochs+1), sgd_lr.cost_)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Cost')\n",
    "plt.ylim([0, 0.2])\n",
    "plt.tight_layout()\n",
    "plt.show()  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## LinearRegression\n",
      "\n",
      "*LinearRegression(method='direct', eta=0.01, epochs=50, minibatches=None, random_seed=None, print_progress=0)*\n",
      "\n",
      "Ordinary least squares linear regression.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `method` : string (default: 'direct')\n",
      "\n",
      "    For gradient descent-based optimization, use `sgd` (see `minibatch`\n",
      "    parameter for further options). Otherwise, if `direct` (default),\n",
      "    the analytical method is used. For alternative, numerically more\n",
      "    stable solutions, use either `qr` (QR decomopisition) or `svd`\n",
      "    (Singular Value Decomposition).\n",
      "\n",
      "- `eta` : float (default: 0.01)\n",
      "\n",
      "    solver learning rate (between 0.0 and 1.0). Used with `method =`\n",
      "    `'sgd'`. (See `methods` parameter for details)\n",
      "\n",
      "- `epochs` : int (default: 50)\n",
      "\n",
      "    Passes over the training dataset.\n",
      "    Prior to each epoch, the dataset is shuffled\n",
      "    if `minibatches > 1` to prevent cycles in stochastic gradient descent.\n",
      "    Used with `method = 'sgd'`. (See `methods` parameter for details)\n",
      "\n",
      "- `minibatches` : int (default: None)\n",
      "\n",
      "    The number of minibatches for gradient-based optimization.\n",
      "    If None: Direct method, QR, or SVD method (see `method` parameter\n",
      "    for details)\n",
      "    If 1: Gradient Descent learning\n",
      "    If len(y): Stochastic Gradient Descent learning\n",
      "    If 1 < minibatches < len(y): Minibatch learning\n",
      "\n",
      "- `random_seed` : int (default: None)\n",
      "\n",
      "    Set random state for shuffling and initializing the weights. Used in\n",
      "    `method = 'sgd'`. (See `methods` parameter for details)\n",
      "\n",
      "- `print_progress` : int (default: 0)\n",
      "\n",
      "    Prints progress in fitting to stderr if `method = 'sgd'`.\n",
      "    0: No output\n",
      "    1: Epochs elapsed and cost\n",
      "    2: 1 plus time elapsed\n",
      "    3: 2 plus estimated time until completion\n",
      "\n",
      "**Attributes**\n",
      "\n",
      "- `w_` : 2d-array, shape={n_features, 1}\n",
      "\n",
      "    Model weights after fitting.\n",
      "\n",
      "- `b_` : 1d-array, shape={1,}\n",
      "\n",
      "    Bias unit after fitting.\n",
      "\n",
      "- `cost_` : list\n",
      "\n",
      "    Sum of squared errors after each epoch;\n",
      "    ignored if solver='normal equation'\n",
      "\n",
      "**Examples**\n",
      "\n",
      "For usage examples, please see\n",
      "    [https://rasbt.github.io/mlxtend/user_guide/regressor/LinearRegression/](https://rasbt.github.io/mlxtend/user_guide/regressor/LinearRegression/)\n",
      "\n",
      "### Methods\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit(X, y, init_params=True)*\n",
      "\n",
      "Learn model from training data.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "- `y` : array-like, shape = [n_samples]\n",
      "\n",
      "    Target values.\n",
      "\n",
      "- `init_params` : bool (default: True)\n",
      "\n",
      "    Re-initializes model parameters prior to fitting.\n",
      "    Set False to continue training with weights from\n",
      "    a previous model fitting.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `self` : object\n",
      "\n",
      "\n",
      "<hr>\n",
      "\n",
      "*get_params(deep=True)*\n",
      "\n",
      "Get parameters for this estimator.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `deep` : boolean, optional\n",
      "\n",
      "    If True, will return the parameters for this estimator and\n",
      "    contained subobjects that are estimators.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `params` : mapping of string to any\n",
      "\n",
      "    Parameter names mapped to their values.'\n",
      "\n",
      "    adapted from\n",
      "    https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/base.py\n",
      "    Author: Gael Varoquaux <gael.varoquaux@normalesup.org>\n",
      "    License: BSD 3 clause\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict(X)*\n",
      "\n",
      "Predict targets from X.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `target_values` : array-like, shape = [n_samples]\n",
      "\n",
      "    Predicted target values.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*set_params(**params)*\n",
      "\n",
      "Set the parameters of this estimator.\n",
      "The method works on simple estimators as well as on nested objects\n",
      "(such as pipelines). The latter have parameters of the form\n",
      "``<component>__<parameter>`` so that it's possible to update each\n",
      "component of a nested object.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "self\n",
      "\n",
      "adapted from\n",
      "https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/base.py\n",
      "Author: Gael Varoquaux <gael.varoquaux@normalesup.org>\n",
      "License: BSD 3 clause\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.regressor/LinearRegression.md', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}