{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "按计划这一节是讲线性回归的变分推断，但由于变分分布其实是拟合的后验概率分布，所以这一节先讲线性回归模型的贝叶斯估计，首先线性回归模型大家应该都比较熟悉了，该项目最初便是从线性回归开始的：[《01_线性模型_线性回归》](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/01_%E7%BA%BF%E6%80%A7%E6%A8%A1%E5%9E%8B_%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92.ipynb)以及[《01_线性模型_线性回归_正则化(Lasso,Ridge,ElasticNet)》](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/01_%E7%BA%BF%E6%80%A7%E6%A8%A1%E5%9E%8B_%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92_%E6%AD%A3%E5%88%99%E5%8C%96(Lasso%2CRidge%2CElasticNet)，之前提及的线性回归模型都是如下形式的： \n", "\n", "$$\n", "y(x,w)=w_0+w_1x_1+\\cdots+w_Dx_D\n", "$$\n", "\n", "其中，$w=[w_0,w_1,...,w_D]$为模型参数，$x=[x_1,x_2,...,x_D]$为输入变量，但这样的形式具有很大的局限性，难以拟合复杂的情况，所以我们通常会通过“特征工程”的方式对输入变量做一定的扩展变换，这样我们就有了更加一般的线性回归模型，它被称为线性基函数模型" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 一.线性基函数模型\n", "它的形式如下 \n", "\n", "$$\n", "y(x,w)=w_0+\\sum_{j=1}^{M-1}w_j\\phi_j(x)\n", "$$\n", "这里，$\\phi_j(\\cdot)$表示“特征工程”函数，即基函数，而参数为$w=[w_0,w_1,...,w_{M-1}]$，共有$M$个。显然，我们之前讨论的线性回归模型是$\\phi(x)=x$的特殊情况，可以发现，线性基函数模型具有这样的性质：它关于参数是线性的，关于输入变量是非线性的。为了书写方便，我们会假设$\\phi_0(x)=1$，这样模型方程就可以简便的书写为： \n", "\n", "$$\n", "y(x,w)=w^T\\phi(x)\n", "$$ \n", "\n", "$\\phi_j(\\cdot)$除了我们自定义一些复杂的函数外，还有一些常用的基函数，比如\n", "\n", "#### 多项式基函数\n", "\n", "$$\n", "\\phi_j(x)=x^j\n", "$$ \n", "\n", "#### 高斯基函数\n", "\n", "$$\n", "\\phi_j(x)=exp\\left \\{-\\frac{(x-\\mu_j)^2}{2s^2}\\right \\}\n", "$$ \n", "\n", "其中，$\\mu_j$控制了基函数在输入空间中的位置，超参数$s$控制了基函数在空间中的辐射范围 \n", "\n", "#### sigmoid基函数\n", "\n", "$$\n", "\\phi_j(x)=\\sigma\\left( \\frac{x-\\mu_j}{s}\\right )\n", "$$ \n", "\n", "这里，$\\sigma(\\cdot)$表示sigmoid函数" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 二.最小平方和误差与高斯极大似然估计\n", "上面对“线性”做了扩展，接下来我们从概率的角度去理解一下之前所用的最小平方和误差，我们假设目标变量$t$由确定的函数$y(x,w)$给出，同时这个函数被附加了一个高斯噪声$\\epsilon$，即： \n", "\n", "$$\n", "t=y(w,x)+\\epsilon\n", "$$ \n", "其中$\\epsilon$是一个零均值的高斯随机变量，且精度（方差的倒数）为$\\beta$，因此我们有： \n", "\n", "$$\n", "p(t\\mid x,w,\\beta)=N(t\\mid y(x,w),\\beta^{-1})\n", "$$ \n", "\n", "显然，对于给定的$x$，该分布的最优预测为目标变量的条件均值： \n", "\n", "$$\n", "E[t\\mid x]=\\int tp(t\\mid)dt=y(x,w)\n", "$$ \n", "\n", "对于观测值$X=\\{x_1,x_2,...,x_N\\},t=\\{t_1,t_2,...,t_N\\}$，我们可以写出其似然函数： \n", "\n", "$$\n", "p(t\\mid X,w,\\beta)=\\prod_{n=1}^N N(t_n\\mid w^T\\phi(x_n),\\beta^{-1})\n", "$$ \n", "\n", "那么，其对数似然函数为（为了记号的简洁，省略$X$）： \n", "\n", "$$\n", "ln\\ p(t\\mid w,\\beta)=\\sum_{n=1}^Nln\\ N(t_n\\mid w^T\\phi(x_n),\\beta^{-1})\\\\\n", "=\\frac{N}{2}ln\\ \\beta-\\frac{N}{2}ln(2\\pi)-\\beta E_D(w)\n", "$$ \n", "\n", "这里，$E_D(w)$即是平方和误差函数： \n", "\n", "$$\n", "E_D(w)=\\frac{1}{2}\\sum_{n=1}^N(t_n-w^T\\phi(x_n))^2\n", "$$ \n", "\n", "接下来就可以用极大似然估计的方式求$w,\\beta$了，由于$\\beta>0$，我们可以先关于$w$求最大值，而对对数似然函数求最大，等价于对$E_D(w)$求最小，所以可以得到极大似然估计最优解： \n", "\n", "$$\n", "w_{ML}=(\\Phi^T\\Phi)^{-1}\\Phi^Tt\n", "$$ \n", "\n", "这里$\\Phi$即是$X$通过基函数映射后的$N\\times M$大小的矩阵： \n", "\n", "$$\n", "\\Phi=\\begin{pmatrix}\n", "\\phi_0(x_1) &\\phi_1(x_1) & \\cdots &\\phi_{M-1}(x_1) \\\\ \n", "\\phi_0(x_2) & \\phi_1(x_2) & \\cdots & \\phi_{M-1}(x_2)\\\\ \n", "\\vdots & \\vdots &\\ddots &\\vdots \\\\ \n", "\\phi_0(x_N) &\\phi_1(x_N) & \\cdots & \\phi_{M-1}(x_N)\n", "\\end{pmatrix}\n", "$$ \n", "从上面的推导，可以发现，我们常用的最小均分误差函数从概率的角度来看就是引入了高斯噪声的极大似然估计，接下来再深刻的认识一下偏置参数$w_0$\n", "#### $w_0$的最优解\n", "我们将$E_D(w)$中间项拆开 \n", "\n", "$$\n", "E_D(w)=\\frac{1}{2}\\sum_{n=1}^N(t_n-w_0-\\sum_{j=1}^{M-1}w_j\\phi_j(x_n))^2\n", "$$ \n", "\n", "对$w_0$求导并令其为0，可以解出： \n", "\n", "$$\n", "w_0=\\bar{t}-\\sum_{j=1}^{M-1}w_j\\bar{\\phi_j}\n", "$$ \n", "\n", "这里： \n", "\n", "$$\n", "\\bar{t}=\\frac{1}{N}\\sum_{n=1}^Nt_n,\\bar{\\phi_j}=\\frac{1}{N}\\sum_{n=1}^N\\phi_j(x_n)\n", "$$ \n", "\n", "可以发现，偏置$w_0$补偿了目标值的平均值与基函数的值的平均值的加权求和之间的差 \n", "\n", "#### $\\beta$的最优解\n", "显然，$\\beta$的最优解也可以写出来\n", "\n", "$$\n", "\\frac{1}{\\beta_{ML}}=\\frac{2}{N}E_D(w_{ML})\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 三.$L_2$正则化与最大后验估计\n", "\n", "接下来，我们看看后验概率$p(t\\mid w)$（为了简洁，$X$没有写出来），现阶段我们先将噪声精度$\\beta$看做已知常数，由于似然函数$p(t\\mid w)$是$w$的二次函数的指数形式，所以对应的共轭先验是高斯分布，我们假设参数$w$的先验为： \n", "\n", "$$\n", "p(w)=N(w\\mid m_0,S_0)\n", "$$ \n", "\n", "这里，均值为$m_0$，协方差为$S_0$，所以： \n", "\n", "$$\n", "p(w\\mid t)\\propto p(w)p(t\\mid w)\n", "$$ \n", "\n", "所以后验分布也会是高斯分布（共轭先验选择了高斯分布的缘故），通过配方，我们可以直接得到后验分布的形式： \n", "\n", "$$\n", "p(w\\mid t)=N(w\\mid m_N,S_N)\n", "$$ \n", "\n", "这里： \n", "\n", "$$\n", "m_N=S_N(S_0^{-1}m_0+\\beta\\Phi^Tt)\\\\\n", "S_N^{-1}=S_0^{-1}+\\beta\\Phi^T\\Phi\n", "$$ \n", "\n", "为了简化，我们可以假设先验分布中的超参数$m_0=0,S_0=\\alpha^{-1}I$，这里$I$是单位矩阵，这时\n", "\n", "$$\n", "m_N=\\beta S_N\\Phi^Tt\\\\\n", "S_N^{-1}=\\alpha I+\\beta\\Phi^T\\Phi\n", "$$ \n", "进一步地，对后验概率取对数，我们可以得到： \n", "\n", "$$\n", "ln\\ p(w\\mid t)=-\\frac{\\beta}{2}\\sum_{n=1}^N\\left\\{ t_n-w^T\\phi(x_n) \\right\\}^2-\\frac{\\alpha}{2}w^Tw+const\n", "$$ \n", "\n", "我们可以发现，在这种情况下，如果对后验概率极大化就等价于我们常用到的平方和损失函数+$L_2$正则化项的极小化" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 四.贝叶斯估计\n", "上面对后验概率做了推导，那么我们就可以进入本节的重点了，即对于新的输入$\\hat{x}$如果通过贝叶斯估计求得它的预测$\\hat{t}$，根据定义，我们知道： \n", "\n", "$$\n", "p(\\hat{t}\\mid \\hat{x},X,t,\\alpha,\\beta)=\\int p(\\hat{t}\\mid w,\\beta,\\hat{x})p(w\\mid X,t,\\alpha,\\beta)dw\n", "$$ \n", "\n", "这里，$p(w\\mid X,t,\\alpha,\\beta)$即是我们上面推导出来的后验概率： \n", "\n", "$$\n", "p(w\\mid X,t,\\alpha,\\beta)=N(w\\mid m_N,S_N)\n", "$$ \n", "\n", "而$p(\\hat{t}\\mid w,\\beta,\\hat{x})$即是我们最初带高斯噪声假设的分布： \n", "\n", "$$\n", "p(\\hat{t}\\mid \\hat{x},w,\\beta)=N(\\hat{t}\\mid y(\\hat{x},w),\\beta^{-1})\\\\\n", "=N(\\hat{t}\\mid w^T\\Phi(\\hat{x}),\\beta^{-1})\n", "$$ \n", "\n", "经过化简，我们可以得到： \n", "\n", "$$\n", "p(\\hat{t}\\mid \\hat{x},X,t,\\alpha,\\beta)=N(\\hat{t}\\mid m_N^T\\phi(\\hat{x}),\\sigma_N^2(\\hat{x})))\n", "$$ \n", "\n", "这里，$m_N$的定义与后验概率分布一样，方差$\\sigma_N^2(\\hat{x})$为： \n", "$$\n", "\\sigma_N^2(\\hat{x})=\\frac{1}{\\beta}+\\phi(\\hat{x})^TS_N\\phi(\\hat{x})\n", "$$ \n", "\n", "可以发现，预测数据的方差由噪声项$\\frac{1}{\\beta}$和输入数据$\\hat{x}$相关，当然这一部分其实并不会影响到我们的预测结果，我们的预测结果为均值项： \n", "$$\n", "\\hat{y}(w,\\hat{x})=m_N^T\\phi(\\hat{x})\n", "$$ \n", "\n", "所以： \n", "\n", "$$\n", "w=m_N\n", "$$ \n", "\n", "细心的同学已经发现了，这里贝叶斯估计的结果与极大后验估计的结果是一样的，这是由于共轭先验也是选择的高斯分布造成的，将先验分布换着其他分布就不一定了" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 五.代码实现\n", "这里本质上就是实现一个Ridge回归..." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "线性回归的bayes估计\n", "\"\"\"\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "\n", "class LinearRegression(object):\n", " def __init__(self, basis_func=None, alpha=1, beta=1):\n", " \"\"\"\n", " :param basis_func: list,基函数列表，包括rbf,sigmoid,poly_{num},linear，默认None为linear，其中poly_{num}中的{num}表示多项式的最高阶数\n", " :param alpha: alpha/beta表示理解为L2正则化项的大小，默认为1\n", " :param beta: 噪声，默认为1\n", " \"\"\"\n", " if basis_func is None:\n", " self.basis_func = ['linear']\n", " else:\n", " self.basis_func = basis_func\n", " self.alpha = alpha\n", " self.beta = beta\n", " # 特征均值、标准差\n", " self.feature_mean = None\n", " self.feature_std = None\n", " # 训练参数\n", " self.w = None\n", "\n", " def _map_basis(self, X):\n", " \"\"\"\n", " 将X进行基函数映射\n", " :param X:\n", " :return:\n", " \"\"\"\n", " x_list = []\n", " for basis_func in self.basis_func:\n", " if basis_func == \"linear\":\n", " x_list.append(X)\n", " elif basis_func == \"rbf\":\n", " x_list.append(np.exp(-0.5 * X * X))\n", " elif basis_func == \"sigmoid\":\n", " x_list.append(1 / (1 + np.exp(-1 * X)))\n", " elif basis_func.startswith(\"poly\"):\n", " p = int(basis_func.split(\"_\")[1])\n", " for pow in range(1, p + 1):\n", " x_list.append(np.power(X, pow))\n", " return np.concatenate(x_list, axis=1)\n", "\n", " def fit(self, X, y):\n", " self.feature_mean = np.mean(X, axis=0)\n", " self.feature_std = np.std(X, axis=0) + 1e-8\n", " X_ = (X - self.feature_mean) / self.feature_std\n", " X_ = self._map_basis(X_)\n", " X_ = np.c_[np.ones(X_.shape[0]), X_]\n", " self.w = self.beta * (\n", " np.linalg.inv(self.alpha * np.eye(X_.shape[1]) + self.beta * X_.T @ X_)) @ X_.T @ y.reshape((-1, 1))\n", "\n", " def predict(self, X):\n", " X_ = (X - self.feature_mean) / self.feature_std\n", " X_ = self._map_basis(X_)\n", " X_ = np.c_[np.ones(X_.shape[0]), X_]\n", " return (self.w.T @ X_.T).reshape(-1)\n", "\n", " def plot_fit_boundary(self, x, y):\n", " \"\"\"\n", " 绘制拟合结果\n", " :param x:\n", " :param y:\n", " :return:\n", " \"\"\"\n", " plt.scatter(x[:, 0], y)\n", " plt.plot(x[:, 0], self.predict(x), 'r')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 测试" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "#造伪样本\n", "X=np.linspace(0,100,100)\n", "X=np.c_[X,np.ones(100)]\n", "w=np.asarray([3,2])\n", "Y=X.dot(w)\n", "X=X.astype('float')\n", "Y=Y.astype('float')\n", "X[:,0]+=np.random.normal(size=(X[:,0].shape))*3#添加噪声\n", "Y=Y.reshape(100,1)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "#加噪声\n", "X=np.concatenate([X,np.asanyarray([[100,1],[101,1],[102,1],[103,1],[104,1]])])\n", "Y=np.concatenate([Y,np.asanyarray([[3000],[3300],[3600],[3800],[3900]])])" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHANJREFUeJzt3Xt0VeWd//H3NyFowGoQwdGAgJTlrYjajNAy0wtOxUtbqbUWLyPLMmWtqfUyWixYp95vo1bH1nEN9doZl5daB6l1yjCKusZZokEIF5WrIyTxJ9EQb0RNwvf3x94n2Qnn5JyTnFuyP6+1snL2c/bOeTabPJ+9n/3sJ+buiIhI/JQVuwIiIlIcCgARkZhSAIiIxJQCQEQkphQAIiIxpQAQEYkpBYCISEwpAEREYkoBICISU0OKXYHeHHDAAT5+/PhiV0NEZEBZuXLle+4+Kt16JR0A48ePp7a2ttjVEBEZUMzs7UzWy7gLyMzKzWyVmT0dLk8wsxVmtsnMHjOzoWH5XuHy5vD98ZGfsTAs32BmM7PbJRERyaVs7gFcDLwRWb4FuMPdJwE7gblh+Vxgp7t/EbgjXA8zOxKYDRwFnAT8i5mV96/6IiLSVxkFgJmNAU4F7g2XDZgBPBGu8hAwK3x9WrhM+P4J4fqnAY+6+2fu/hawGTg+FzshIiLZy/QK4E7gcmB3uDwSaHH39nC5HqgOX1cD2wHC9z8I1+8sT7KNiIgUWNoAMLNvAzvcfWW0OMmqnua93raJft48M6s1s9qmpqZ01RMRkT7KZBTQdOC7ZnYKsDewL8EVQZWZDQnP8scAjeH69cBYoN7MhgD7Ac2R8oToNp3cfRGwCKCmpkZ/rUZEYmXxqgZuXbqBxpZWDq6qZP7Mw5h1bH46S9JeAbj7Qncf4+7jCW7iPufu5wDLgTPC1eYAT4Wvl4TLhO8/58GfHVsCzA5HCU0AJgGv5GxPREQGuMWrGlj45FoaWlpxoKGllYVPrmXxqoa8fF5/ngT+OXCpmW0m6OO/Lyy/DxgZll8KLABw9/XA48DrwJ+BC9y9ox+fLyIyqNy6dAOtbd2bxda2Dm5duiEvn5fVg2Du/jzwfPh6K0lG8bj7p8APUmx/A3BDtpUUEYmDxpbWrMr7S3MBiYiUiIOrKrMq76+SngpCRCQOEjd+G1paMboPj6ysKGf+zMPy8rkKABGRIkrc+E30/SfGzDtQnedRQAoAEZEiSnbjN9H4v7RgRl4/W/cARESKqNA3fqMUACIiRVToG79RCgARkSKaP/MwKiu6T4yczxu/UboHICJSRIkbvIWa/iFKASAiUmSzjq0uSIPfk7qARERiSlcAIiJFUMhZP1NRAIiIFFjPh78Ss34CBQ0BdQGJiBRYoWf9TEUBICJSYMV8+CtKASAiUmDFfPgrSgEgIlJgxXz4K0o3gUVECqyYD39FKQBERIqgWA9/RakLSEQkphQAIiIxpQAQEYkpBYCISEwpAEREYkoBICISUxoGKiJSQKUwC2iCAkBEpEBKZRbQBHUBiYgUSKnMApqgABARKZBSmQU0QQEgIlIgpTILaIICQESkQEplFtAE3QQWESmQUpkFNEEBICJSQKUwC2iCuoBERGJKASAiElMKABGRmFIAiIjElAJARCSmFAAiIjGVNgDMbG8ze8XM6sxsvZldE5ZPMLMVZrbJzB4zs6Fh+V7h8ubw/fGRn7UwLN9gZjPztVMiIqVk8aoGpt/8HBMW/InpNz/H4lUNxa4SkNkVwGfADHefAhwDnGRm04BbgDvcfRKwE5gbrj8X2OnuXwTuCNfDzI4EZgNHAScB/2Jm3R+JExEZZBIzgDa0tOJ0zQBaCiGQNgA88HG4WBF+OTADeCIsfwiYFb4+LVwmfP8EM7Ow/FF3/8zd3wI2A8fnZC9EREpUqc0AGpXRPQAzKzez1cAOYBmwBWhx9/ZwlXog8WhbNbAdIHz/A2BktDzJNtHPmmdmtWZW29TUlP0eiYiUkFKbATQqowBw9w53PwYYQ3DWfkSy1cLvluK9VOU9P2uRu9e4e82oUaMyqZ6ISMkqtRlAo7IaBeTuLcDzwDSgyswScwmNARrD1/XAWIDw/f2A5mh5km1ERAalUpsBNCqTUUCjzKwqfF0J/A3wBrAcOCNcbQ7wVPh6SbhM+P5z7u5h+exwlNAEYBLwSq52RESkFM06tpqbTp9MdVUlBlRXVXLT6ZNLYkK4TGYDPQh4KByxUwY87u5Pm9nrwKNmdj2wCrgvXP8+4N/MbDPBmf9sAHdfb2aPA68D7cAF7t6BiMggV0ozgEZZcHJemmpqary2trbY1RARGVDMbKW716RbT08Ci4jElAJARCSmFAAiIjGlABARiSkFgIhITCkARERiSgEgIhJTCgARkZhSAIiIxJQCQEQkphQAIiIxpQAQEYkpBYCISEwpAEREYkoBICISUwoAEZGYUgCIiMSUAkBEJKYUACIiMaUAEBGJKQWAiEhMKQBERGJKASAiElMKABGRmFIAiIjElAJARCSmFAAiIjGlABARiSkFgIhITCkARERiSgEgIhJTCgARkZhSAIiIxJQCQEQkphQAIiIxpQAQEYkpBYCISEwpAEREYiptAJjZWDNbbmZvmNl6M7s4LN/fzJaZ2abw+4iw3MzsLjPbbGZrzOy4yM+aE66/yczm5G+3REQknUyuANqBy9z9CGAacIGZHQksAJ5190nAs+EywMnApPBrHnAPBIEBXAVMBY4HrkqEhoiIFF7aAHD3d9z9tfD1R8AbQDVwGvBQuNpDwKzw9WnA7zzwMlBlZgcBM4Fl7t7s7juBZcBJOd0bERHJWFb3AMxsPHAssAI40N3fgSAkgNHhatXA9shm9WFZqvKenzHPzGrNrLapqSmb6omISBYyDgAz2wf4A3CJu3/Y26pJyryX8u4F7ovcvcbda0aNGpVp9UREJEsZBYCZVRA0/g+7+5Nh8bth1w7h9x1heT0wNrL5GKCxl3IRESmCTEYBGXAf8Ia7/yry1hIgMZJnDvBUpPy8cDTQNOCDsItoKXCimY0Ib/6eGJaJiEgRDMlgnenA3wJrzWx1WHYFcDPwuJnNBbYBPwjfewY4BdgM7ALOB3D3ZjO7Dng1XO9ad2/OyV6IiEjWzH2PbviSUVNT47W1tcWuhojIgGJmK929Jt16ehJYRCSmFAAiIjGlABARiSkFgIhITCkARERiSgEgIhJTCgARkZhSAIiIxJQCQEQkphQAIiIxpQAQEYkpBYCISEwpAEREYkoBICISUwoAEZGYUgCIiMSUAkBEJKYUACIiMaUAEBGJKQWAiEhMKQBERGJKASAiElMKABGRmFIAiIjElAJARCSmFAAiIjGlABARiSkFgIhITCkARERiSgEgIhJTCgARkZhSAIiIxJQCQEQkphQAIiIxpQAQEYkpBYCISEwpAEREYkoBICISU2kDwMzuN7MdZrYuUra/mS0zs03h9xFhuZnZXWa22czWmNlxkW3mhOtvMrM5+dkdERHJVCZXAA8CJ/UoWwA86+6TgGfDZYCTgUnh1zzgHggCA7gKmAocD1yVCA0RESmOtAHg7i8CzT2KTwMeCl8/BMyKlP/OAy8DVWZ2EDATWObuze6+E1jGnqEiIiIF1Nd7AAe6+zsA4ffRYXk1sD2yXn1Ylqp8D2Y2z8xqzay2qampj9UTEZF0cn0T2JKUeS/lexa6L3L3GnevGTVqVE4rJyIiXfoaAO+GXTuE33eE5fXA2Mh6Y4DGXspFRKRI+hoAS4DESJ45wFOR8vPC0UDTgA/CLqKlwIlmNiK8+XtiWCYiIkUyJN0KZvYI8A3gADOrJxjNczPwuJnNBbYBPwhXfwY4BdgM7ALOB3D3ZjO7Dng1XO9ad+95Y1lERArI3JN2xZeEmpoar62tLXY1RETyzx1WrID334e6OvjiF+HMM/v0o8xspbvXpFsv7RWAiIjk2JYt8Pd/D8uWpV7nRz/qcwBkSgEgIpIv7rB0KVx2Gbz+embb/Nd/wfHHw3775bduaC4gEZHcaGmByy8Hs66vsjI4+eTUjf+tt0JraxAUia9vfasgjT/oCkBEJHsvvhic1Wd6j/Kb34R/+ieoSdstX1AKABGRVD75BN56Cx55BG68MbNtrr0WLr0Uhg/Pb91yQAEgIgLw6qswfz688EJm60+dGnTh/PVf57deeaQAEJF4+fRTuOsu+PnPM1t/9my44go49NABcVafDQWAiAxezzwDp56a+fpHHw233RbciI0BjQISkYGvrQ3uvhv23rv7KJzeGv8LL4QdO7qPwKmri03jD7oCEJGB5qOPYO3aoLH+yU8y22bhQrjhhiAUpJMCQERKkzu8/XbQ0K9eHXyvq4OtW1NvM3cuXHMNVCf9cyPSgwJARIpv1y5Yt66rka+rgzVr4MMPg/fNgrlxjjsOzj8/6KufMgUOOURn9f2gABCRwnGHjRth06bujf2mTcF7APvsEzTw55wTNPJTpsCXvhSUS04pAEQkPz79FNav797Q9xxjf+ihQQN/9tnB98mTYcKEYAoFyTsFgIj0jztcfXXwBOwxx8DhhweN/caN0NERrDN8eNC4z50b9Ov/4z8G6+67b1GrHncKABHJXHMzXHcd3Hln8vdXr4adO4MunO9/v6sLZ+JEndWXIAWAiCS3dWvQiK9enfk269bBUUflr06SU4pkkbj78EP4xS+6P0BlFpy1p2r8b7wxmCgt+hCVuxr/AUZXACJx8t57wfDKurqgz76lJfW6l1wCb74Jv/wlfOUrhaujFIwCQGQwam/fc6hlXR00Nqbe5sorg9kwdWM2NhQAIgPd4sXwve8Fr7/2ta6Hqj79NCgbMgSOOAJmzOi6KTtlCoweXbw6S0lQAIgMFLt2wemnB39jNpUXX4QTTgjmyEk09IcfDnvtVbh6yoChABApRS+8AN/4RnbbLFoEf/d3mhpBMqYAECmm9na47z742c/g448z22bYMNiyBf7iL/JbNxn0FAAihfL00/Cd72S3zcUXw69+pYeoimjxqgZuXbqBxpZWDq6qZP7Mw5h1bO+zjRZqm/5SAIjkWkcHXHAB/Ou/Zr7NeecFT9geckj+6hVj0cZ1v8oKzKBlV1u314lGt/btZh5ZsZ2OxOR0EQ0trSx8ci1AysZ58aoGFj65lta2jpTb9Gzsv3n4KP6wsqHXbfLBPMlOloqamhqvra0tdjVEUmtsDOa1uf/+zLeZMwfuvTcYnSN9luqM+crFazsb8HIzph06gte2fdDZuPbGgExaxOqqSl5aMCPpe9Nvfo6GltaU2/QMiN4+t7fP6Y2ZrXT3mnTr6X+gSCbcg6dlb7opu+02boRJk/JTpwGoZ+N81tSxXD9rcrfyhKokZ+eJs+ErF6/l4Ze3dTaaiTPm39du46UtzZ0/o8O923I6mZ4ONyZp4NO9lyi/demGPcIo1ef29jm5oAAQ6em11+DLX858/b33httvhx//GCoq8levHErWBbH8zaZuy0/XvUNLa1vnNlWVFVz93aO6dUkkfk70jDfasEdduXgt//7yts7lDnf+/eVtrNj6Ppt2fLJHHaOfHe0SAbo1/gmtbR1ZNfb9cXBVZa/vJbsCSGyTTaPe2+fkggJA4u3cc+Hhh7Pb5vnn4etfz0t1oPebgdH3qoZV8GlbB61tuwEoM9jtQbdBqm1S9TdHG+aeywktrW3M/30dEPRLJ+vKgK6GHegWAo+s2J50f5M1/sm0tnVw69INQOZn6vlQWVHO/JmHpXx//szD9vh3iW6TKiB6dgOl+5xc0D0AiYfGRpg6Ferrs9vuvfdg5Mh+ffTiVQ1c88f17NwVnNEmO5NO1gWSkGgYqior+Oizdjp2p/+drawo56bTg8Y30/7mTCX6pVP1dSeUm7HlplM6l8cv+FM/PjWQeMKhWK3WiGEVXPWdo/o1oidZcFZWlPP9L1d3uwrrzygg3QOQ+Prtb2HevKw2+e+zf8pVk7+3xyiRg39b1/mLmOmZefS9xasamP9EHW0dXU1WS2sblzy2mkseW011VSXjR1b22nXhke0yFT1bzrS/OVOJLox0XRk9w6zcLGnAZSPRJZLqDPqrE/dP+m85afTwlFcaw4eWs+vzDqqGVfBBaxvJ8jXThj9h1rHVKddNlBd6yGcyugKQgev99+G113j8z6/xV/fezsEfNqXdZPPocZxzxjW8+4UDMv6YinJjSJl1drUk9HaWXVFuDB86JKtGO9fydbbc1yuAnvcAEnprnKPSXdWcM+2QPW4oR+9HLF7VwNVL1ncek2SNeiZXawNBplcACgDJuZ791O7wQeueIzlSbZv0F3Dj/8Ds2VnVY/UVN3PBF/6y86w+0+6TbFT3ckZabL3Vra/dQBVlxq0/mNLrPYCEc8MGOSpXo4CK8dDUQKIAkD5J9osF9DpiJLpOJg1hqrOqxasa+MWjr3LOK0u44vkHMq7zSef/mi0jx9BW3jUCp8xgryHlGY397o9i90mnYsAdPzwG2PNsOVl/8zcPH8Wf1rzTGbzJ9HcUkBSOAiAGshktEj0LT9WARy+Po8rLrNcz54oyA6NbP3c6lRXl/OboCk544Hb4z//MfKdvuYWJzUeR32Y9c/m+AujLmXq0OwSyO1vWmfXgoAAocanGYTe0tHbeLKvucTOx5/qPvbo9aaNbVVnBJ5+3Z9wg96UBz8Ze7Z9zz3/cyIytmR/LFWO/xOUnX8TbIw4GgkbtrZtPBXIzmiQXeuuTzvbnHHfIfry8dWfSfutMr77UaEuCAqCfMj0TymS9nuuMH1nJ/25pzujMLnG5Hh23XcrG7Wxk/gu/49sb/iezDcaPh9tug9NPZ/oty1OeSUcfiZ+48Jm0o0kqK8r2uGmbyohhFZx69EEpu0AS4+t7bhO9gdjziuvjT9tpi2wU7XZJFfIiuVKyAWBmJwH/DJQD97r7zanW7WsARPsl+/KLluzmVmJUR/RmJvTev9rQ0trvMdeQm+FzuVa+u4Ob/vxrzlz731lt960f3c2mUeOAPec5STZkErrfeITUo0mgq/+5Ztz+ex7DMmPokDI++Two661Pu79n2OpKkWIqyQAws3JgI/AtoB54FTjL3V9Ptn5fAqC3kQmJS/Z0v4jphrclftbeFWVJzxhz0eiXkiPf3cozD16U8frNlfty7Qk/5qkjv45b8mmMUx2LTIfhpRpN0vNnqRGWOCrVAPgKcLW7zwyXFwK4e9IZtvoSAOka70xm15uw4E8l1YDn+wqg8x5A+24O+ug9LvzfRzm7rpc/O9jDCw8+xc6jv5z0pnPioaqdu9rU7SFSIKX6JHA1EJ0QpB6YGl3BzOYB8wAO6cPc6OmeTsxkIqZUc3UUy1lTx+5xD6Ci3OjocFL1cleUGfvsPYSWXW0MG1re2e2RsFf750x6bxtf/bieHw5tZr+NrzP09fXs2/pRynp8vu9+zPzZI/zfJ7u7nVEnZsVRgy4ysBQ6AJL9sdJup7buvghYBMEVQLYfkK7xzmR2vWSTOSVTVVnBZ+27czrPSk/TJ+7P9bMmUzNu/177plM+cOUO118P1/2y82e+NXoc496vp6wjrPewYTB5MvztWcEfET/kEJg4EY44oltdhgLLc7hvIlJchQ6AemBsZHkM0JjLD+it8c50dr2ec3WkGtVx9XeP6rZespkWe0qMOEk2lG/hk2u6zex49tSusdyp5hbpVtbYGPz5wIW3w8LU+zdh6tFw9LlBYz9lStDYl5en/XcRkcGl0PcAhhDcBD4BaCC4CXy2u69Ptn6xRgH19jOzeZgmr8P9fvMbuPDC7LZZuhROPDE3ny8iJaskbwIDmNkpwJ0Ew0Dvd/cbUq07mB8Ey1hTE0yfDps2Zb7NDTfAJZcEXTsiEjulehMYd38GeKbQnzsg3HEHXHppdtvcfz+cf35+6iMig5r+HkAxtLbCT34CDz6Y+TYTJ8JLL8GBB+atWiISLwqAfGtuhnXrggnPbk750HN3v/41/PSn+a2XiMSeAiBXdu2C008PbrRmauRIWLkSxo3LX71ERFJQAPTF8uUwo/enibv54x/huOPgoIPAkj0KISJSeAqA3rS3wwMPwGWXwUepn5DtpqwM1q+Hww/Pb91ERPpJAZDwySdBX31dXdfXSy/1vs1FFwUjd8qST3gmIlLK4hcA7rB9e/eGvq4ONm8O3gPYd184+uiubc49N5hOQX31IjKIDO4AaG0NumPq6mDNmq7vO3d2rTNxYjAdwrnnBo3+lCnBHylRX72IDHKDMwDWroUf/hA2bIDd4XyZw4cHDfyZZ3bNgTN5MnzhC8Wtq4hIkQzOADjwQJg0Cc44o6uxP/RQ9dWLiEQMzgAYPRqeeqrYtRARKWk6JRYRiSkFgIhITCkARERiSgEgIhJTCgARkZhSAIiIxJQCQEQkphQAIiIxVfA/Cp8NM2sC3i7CRx8AvFeEzy0U7d/ANZj3DbR/uTLO3UelW6mkA6BYzKzW3WuKXY980f4NXIN530D7V2jqAhIRiSkFgIhITCkAkltU7ArkmfZv4BrM+wbav4LSPQARkZjSFYCISEwpAHows5PMbIOZbTazBcWuT3+Y2VgzW25mb5jZejO7OCzf38yWmdmm8PuIYte1P8ys3MxWmdnT4fIEM1sR7t9jZja02HXsKzOrMrMnzOzN8Dh+ZbAcPzP7h/D/5Toze8TM9h7Ix87M7jezHWa2LlKW9FhZ4K6wnVljZscVo84KgAgzKwfuBk4GjgTOMrMji1urfmkHLnP3I4BpwAXh/iwAnnX3ScCz4fJAdjHwRmT5FuCOcP92AnOLUqvc+Gfgz+5+ODCFYD8H/PEzs2rgIqDG3b8ElAOzGdjH7kHgpB5lqY7VycCk8GsecE+B6tiNAqC744HN7r7V3T8HHgVOK3Kd+szd33H318LXHxE0HtUE+/RQuNpDwKzi1LD/zGwMcCpwb7hswAzgiXCVAbt/ZrYv8DXgPgB3/9zdWxg8x28IUGlmQ4BhwDsM4GPn7i8CzT2KUx2r04DfeeBloMrMDipMTbsoALqrBrZHluvDsgHPzMYDxwIrgAPd/R0IQgIYXbya9dudwOXA7nB5JNDi7u3h8kA+hocCTcADYRfXvWY2nEFw/Ny9AbgN2EbQ8H8ArGTwHLuEVMeqJNoaBUB3lqRswA+TMrN9gD8Al7j7h8WuT66Y2beBHe6+MlqcZNWBegyHAMcB97j7scAnDMDunmTCvvDTgAnAwcBwgm6RngbqsUunJP6fKgC6qwfGRpbHAI1FqktOmFkFQeP/sLs/GRa/m7jcDL/vKFb9+mk68F0z+z+C7roZBFcEVWG3AgzsY1gP1Lv7inD5CYJAGAzH72+At9y9yd3bgCeBrzJ4jl1CqmNVEm2NAqC7V4FJ4UiEoQQ3pZYUuU59FvaH3we84e6/iry1BJgTvp4DPFXouuWCuy909zHuPp7gWD3n7ucAy4EzwtUG8v79P2C7mR0WFp0AvM7gOH7bgGlmNiz8f5rYt0Fx7CJSHaslwHnhaKBpwAeJrqKCcnd9Rb6AU4CNwBbgF8WuTz/35a8ILivXAKvDr1MI+smfBTaF3/cvdl1zsK/fAJ4OXx8KvAJsBn4P7FXs+vVjv44BasNjuBgYMViOH3AN8CawDvg3YK+BfOyARwjuZ7QRnOHPTXWsCLqA7g7bmbUEo6EKXmc9CSwiElPqAhIRiSkFgIhITCkARERiSgEgIhJTCgARkZhSAIiIxJQCQEQkphQAIiIx9f8Bl9b56yhowmwAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "lr=LinearRegression()\n", "lr.fit(X[:,:-1],Y)\n", "lr.plot_fit_boundary(X[:,:-1],Y)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X+YXVV97/H3ZyaTZAIXJoEJJpPQxBhRKBLiCLFYf0QlIahJKSpcrbk+3Jv+gKd6a+lNtBbBWrG0Un0epKVCDVZFRIzhh6QxYOn1PkAmJBJCiIkCySSBjOQHv2J+fu8fZ53kzMw5M+fMrzMz+/N6nvOcs9dee5+1syfru89aa6+tiMDMzLKnptoFMDOz6nAAMDPLKAcAM7OMcgAwM8soBwAzs4xyADAzyygHADOzjHIAMDPLKAcAM7OMGlHtAnTl1FNPjSlTplS7GGZmQ8qaNWt+ExGN3eUb1AFgypQptLS0VLsYZmZDiqTnyslXdhOQpFpJayXdm5anSnpU0mZJ35c0MqWPSstb0vopBftYktI3SZpT2SGZmVlfqqQP4FPAxoLlrwA3RsR0YA9wRUq/AtgTEW8Abkz5kHQmcBlwFjAX+Iak2t4V38zMeqqsACBpEnAx8M20LGA2cFfKshRYkD7PT8uk9e9N+ecDd0TEgYh4BtgCnNcXB2FmZpUr9xfAPwF/BRxNy6cAeyPicFpuBZrS5yZgG0Bavy/lP5ZeZJtjJC2S1CKppa2trYJDMTOzSnQbACR9ANgVEWsKk4tkjW7WdbXN8YSIWyKiOSKaGxu77cQ2M7MeKmcU0AXAhyTNA0YDJ5H7RdAgaUS6yp8E7Ej5W4HJQKukEcDJwO6C9LzCbczMDFi2djs3rNjEjr37mdhQz9VzzmDBuZ0aS/pEt78AImJJREyKiCnkOnEfjIiPAQ8Bl6ZsC4Efp8/L0zJp/YORe+zYcuCyNEpoKjAdeKzPjsTMbIhbtnY7S+5ez/a9+wlg+979LLl7PcvWbu+X7+vNncD/B/gLSVvItfHfmtJvBU5J6X8BLAaIiA3AncBTwAPAlRFxpBffb2Y2rNywYhP7D7WvFvcfOsINKzb1y/dVdCNYRPwM+Fn6/GuKjOKJiN8CHy6x/ZeAL1VaSDOzLNixd39F6b3luYDMzAaJiQ31FaX31qCeCsLMLAvyHb/b9+5HtB8eWV9Xy9VzzuiX73UAMDOronzHb77tPz9mPoCmfh4F5ABgZlZFxTp+85X/zxfP7tfvdh+AmVkVDXTHbyEHADOzKhrojt9CDgBmZlV09ZwzqK9rPzFyf3b8FnIfgJlZFeU7eAdq+odCDgBmZlW24NymAanwO3ITkJlZRjkAmJlllJuAzMyqYCCnfS7FAcDMbIB1vPs3P+0zMKBBwE1AZmYDbKCnfS7FAcDMbIBV8+7fQuU8E3i0pMck/ULSBknXpvRvSXpG0rr0mpHSJenrkrZIekLSzIJ9LZS0Ob0WlvpOM7PhrJp3/xYq5xfAAWB2RJwDzADmSpqV1l0dETPSa11Ku4jc4x6nA4uAmwEkjQOuAc4n9yCZaySN7btDMTMbGqp592+hcp4JHBHxSlqsS6/oYpP5wO1pu0fIPTx+AjAHWBkRuyNiD7ASmNu74puZDT0Lzm3iy5ecTVNDPSI38+eXLzl7cI4CklQLrAHeANwUEY9K+lPgS5L+BlgFLI6IA0ATsK1g89aUVirdzCxzqnX3b6GyOoEj4khEzAAmAedJ+l1gCfAm4G3AOHIPiYfcsww67aKL9HYkLZLUIqmlra2tnOKZmVkPVDQKKCL2knso/NyI2JmaeQ4A/8bxB8S3ApMLNpsE7OgiveN33BIRzRHR3NjYWEnxzMysAuWMAmqU1JA+1wPvA55O7fpIErAAeDJtshz4RBoNNAvYFxE7gRXAhZLGps7fC1OamZlVQTl9ABOApakfoAa4MyLulfSgpEZyTTvrgD9J+e8H5gFbgNeATwJExG5JXwRWp3zXRcTuvjsUMzOrhCK6GtBTXc3NzdHS0lLtYpiZDSmS1kREc3f5fCewmVlGeTI4M7MBNBhmAc1zADAzGyCDZRbQPDcBmZkNkMEyC2ieA4CZ2QAZLLOA5jkAmJkNkMEyC2ieA4CZ2QAZLLOA5rkT2MxsgOQ7ej0KyMwsgwbDLKB5bgIyM8soBwAzs4xyADAzyygHADOzjHIAMDPLKAcAM7OMKueJYKMlPSbpF5I2SLo2pU+V9KikzZK+L2lkSh+Vlrek9VMK9rUkpW+SNKe/DsrMzLpXzi+AA8DsiDgHmAHMTY96/ApwY0RMB/YAV6T8VwB7IuINwI0pH5LOBC4DzgLmAt9ITxkzMxvWlq3dzgXXP8jUxfdxwfUPsmzt9moXCSgjAKQHv7+SFuvSK4DZwF0pfSm55wIDzE/LpPXvTc8Nng/cEREHIuIZco+MzD9I3sxsWMpPAb19736C41NAD4YgUFYfgKRaSeuAXcBK4FfA3og4nLK0Avlb25qAbQBp/T7glML0ItuYmQ1Lg20K6EJlBYCIOBIRM4BJ5K7a31wsW3pXiXWl0tuRtEhSi6SWtra2copnZjZoDbYpoAtVNAooIvYCPwNmAQ2S8nMJTQJ2pM+twGSAtP5kYHdhepFtCr/jlohojojmxsbGSopnZjboDLYpoAuVMwqoUVJD+lwPvA/YCDwEXJqyLQR+nD4vT8uk9Q9GRKT0y9IooanAdOCxvjoQM7PBaLBNAV2onNlAJwBL04idGuDOiLhX0lPAHZL+FlgL3Jry3wp8W9IWclf+lwFExAZJdwJPAYeBKyPiCGZmw9hgmwK6kHIX54NTc3NztLS0VLsYZmZDiqQ1EdHcXT7fCWxmllEOAGZmGeUAYGaWUQ4AZmYZ5QBgZpZRDgBmZhnlAGBmllEOAGZmGeUAYGaWUQ4AZmYZ5QBgZpZRDgBmZhnlAGBmllEOAGZmGeUAYGaWUQ4AZmYZVc4jISdLekjSRkkbJH0qpX9B0nZJ69JrXsE2SyRtkbRJ0pyC9LkpbYukxf1zSGZmVo5yHgl5GPhMRDwu6b8BayStTOtujIh/KMws6Uxyj4E8C5gI/FTSG9Pqm4D3k3tA/GpJyyPiqb44EDMzq0y3ASAidgI70+eXJW0EunqY5Xzgjog4ADyTng18Xlq3JSJ+DSDpjpTXAcDMrAoq6gOQNAU4F3g0JV0l6QlJt0kam9KagG0Fm7WmtFLpHb9jkaQWSS1tbW2VFM/MzCpQdgCQdCLwQ+DTEfEScDMwDZhB7hfCP+azFtk8ukhvnxBxS0Q0R0RzY2NjucUzM7MKldMHgKQ6cpX/dyLiboCIeKFg/b8C96bFVmByweaTgB3pc6l0MzMbYOWMAhJwK7AxIr5akD6hINsfAE+mz8uByySNkjQVmA48BqwGpkuaKmkkuY7i5X1zGGZmVqlyfgFcAPwRsF7SupT2WeBySTPINeM8C/wxQERskHQnuc7dw8CVEXEEQNJVwAqgFrgtIjb04bGYmVkFFNGpGX7QaG5ujpaWlmoXw8xsSJG0JiKau8vnO4HNzDLKAcDMLKMcAMzMMsoBwMwsoxwAzMwyygHAzCyjHADMzDLKAcDMLKMcAMzMMsoBwMwsoxwAzMwyygHAzCyjHADMzDLKAcDMLKMcAMzMMqqcJ4JNlvSQpI2SNkj6VEofJ2mlpM3pfWxKl6SvS9qSHhg/s2BfC1P+zZIW9t9hmZlZd8r5BXAY+ExEvBmYBVwp6UxgMbAqIqYDq9IywEXkHgM5HVhE7uHxSBoHXAOcD5wHXJMPGmZmNvC6DQARsTMiHk+fXwY2Ak3AfGBpyrYUWJA+zwduj5xHgIb0/OA5wMqI2B0Re4CVwNw+PRozMytbRX0AkqYA5wKPAqdFxE7IBQlgfMrWBGwr2Kw1pZVKNzOzKig7AEg6Efgh8OmIeKmrrEXSoov0jt+zSFKLpJa2trZyi2dmZhUqKwBIqiNX+X8nIu5OyS+kph3S+66U3gpMLth8ErCji/R2IuKWiGiOiObGxsZKjsXMzCpQziggAbcCGyPiqwWrlgP5kTwLgR8XpH8ijQaaBexLTUQrgAsljU2dvxemNDMzq4IRZeS5APgjYL2kdSnts8D1wJ2SrgC2Ah9O6+4H5gFbgNeATwJExG5JXwRWp3zXRcTuPjkKMzOrmCI6NcMPGs3NzdHS0lLtYpiZDSmS1kREc3f5fCewmVlGOQCYmWWUA4CZWUY5AJiZZZQDgJlZRjkAmJlllAOAmVlGOQCYmWWUA4CZWUY5AJiZZZQDgJlZRjkAmJlllAOAmVlGOQCYmWWUA4CZWUaV80Sw2yTtkvRkQdoXJG2XtC695hWsWyJpi6RNkuYUpM9NaVskLe77QzEzs0qU8wvgW8DcIuk3RsSM9LofQNKZwGXAWWmbb0iqlVQL3ARcBJwJXJ7ymplZlXT7SMiIeFjSlDL3Nx+4IyIOAM9I2gKcl9ZtiYhfA0i6I+V9quISm5lZn+hNH8BVkp5ITURjU1oTsK0gT2tKK5VuZmZV0tMAcDMwDZgB7AT+MaWrSN7oIr0TSYsktUhqaWtr62HxzMysOz0KABHxQkQciYijwL9yvJmnFZhckHUSsKOL9GL7viUimiOiubGxsSfFMzOzMvQoAEiaULD4B0B+hNBy4DJJoyRNBaYDjwGrgemSpkoaSa6jeHnPi21mZr3VbSewpO8B7wZOldQKXAO8W9IMcs04zwJ/DBARGyTdSa5z9zBwZUQcSfu5ClgB1AK3RcSGPj8aMzMrmyKKNsUPCs3NzdHS0lLtYpiZDSmS1kREc3f5fCewmVlGOQCYmWWUA4CZWUY5AJiZZZQDgJlZRjkAmJlllAOAmVlGOQCYmWWUA4CZWUY5AJiZZZQDgJlZRjkAmJlllAOAmVlGOQCYmWWUA4CZWUY5AJiZZVS3AUDSbZJ2SXqyIG2cpJWSNqf3sSldkr4uaYukJyTNLNhmYcq/WdLC/jkcMzMrVzm/AL4FzO2QthhYFRHTgVVpGeAics8Bng4sAm6GXMAg9yjJ88k9QP6afNAwM7Pq6DYARMTDwO4OyfOBpenzUmBBQfrtkfMI0JAeID8HWBkRuyNiD7CSzkHFzMwGUE/7AE6LiJ0A6X18Sm8CthXka01ppdI7kbRIUouklra2th4Wz8zMutPXncAqkhZdpHdOjLglIpojormxsbFPC2dmZsf1NAC8kJp2SO+7UnorMLkg3yRgRxfpZmZWJT0NAMuB/EiehcCPC9I/kUYDzQL2pSaiFcCFksamzt8LU5qZmVXJiO4ySPoe8G7gVEmt5EbzXA/cKekKYCvw4ZT9fmAesAV4DfgkQETslvRFYHXKd11EdOxYNjOzAaSIok3xg0Jzc3O0tLRUuxhmZkOKpDUR0dxdPt8JbGaWUQ4AZmYZ5QBgZpZRDgBmZhnlAGBmllEOAGZmGeUAYGaWUQ4AZmYZ5QBgZpZRDgBmZhnlAGBmllEOAGZmGeUAYGaWUQ4AZmYZ5QBgZpZRvQoAkp6VtF7SOkktKW2cpJWSNqf3sSldkr4uaYukJyTN7IsDMDOznumLXwDviYgZBQ8fWAysiojpwKq0DHARMD29FgE398F3m5lZD/VHE9B8YGn6vBRYUJB+e+Q8AjTkHyxvZmYDr7cBIID/kLRG0qKUdlp6EDzpfXxKbwK2FWzbmtLMzKwKun0ofDcuiIgdksYDKyU93UVeFUnr9EDiFEgWAZx++um9LJ6ZmZXSq18AEbEjve8CfgScB7yQb9pJ77tS9lZgcsHmk4AdRfZ5S0Q0R0RzY2Njb4pnZjZ4HT0Kq1fDNdfAW98KUu71oQ/BRz8Kixd3v49e6vEvAEknADUR8XL6fCFwHbAcWAhcn95/nDZZDlwl6Q7gfGBfvqnIzGzYOHgQ/uu/4L774N57YfPmyra/5x6YPh1OOql/ylegN01ApwE/kpTfz3cj4gFJq4E7JV0BbAU+nPLfD8wDtgCvAZ/sxXebmQ2sV16BlStzlfq998KuXd1vU64LL4SLL4Z58+ANb+i7/XajxwEgIn4NnFMk/UXgvUXSA7iyp99nZtYvdu2C++8/XrEfONA3+x09Olepf+ADMHcuvO51fbPfPtTbTmAzs8HpkUfg2mvhgQd6v6/3vAceeij3efz4XMV+8cXw/vcPSFNNf3EAMLOhIwL27oWdO3Ovhx+G667rm31Pm5a7Wr/4YnjnO2HUqL7Zb5mWrd3ODSs2sWPvfiY21HP1nDNYcG7/jpR3ADCz6jt6FF588XjFvnMn7NjRfjn/+u1vK9//G9+YG1Xz8Y+z7Mldxyrak+vrkGDva4eOVboAN9z4c7bv3U+txJGIY+9NZVbM3VXmHde/502N/HDNdvYfOgLA9r37WXL3eoB+DQLKNc0PTs3NzdHS0lLtYphZTx0+nGtjL6zUr7kGZs3KVfr59Oefz+Xt6OSTYcKE9q+JE2HCBP7vKyO4aeOrvPDKQQ5MncbVc9/EgnObila+ADes2MT2vfu7LG6N4Gg3VWJ9XS1fvuTskhXzsrXbWXL3+mOVecdtiq0XRW6KApoa6vn54tldF6gISWsKpucpnc8BwMwq9uKL8JOf5IY63nFH+3Xz5h2v7NvachV9MWeffaxC36QT+dHzR9k68iReOHEcu04cx6sN44gxJ7S7Os9Xun+9bD3feWRru0qzvq6WP3xrE99fvY1DR46vqRHU1qhdWm91VTFfcP2DRQNNfptS64sR8Mz1F1dcvnIDgJuAzOy4rVvhvvvY1LKRlc+8xFUPLe1+mw6eXreZhtefzus++NZOV+43PLGPW3/5Gr+tGUGtxOXnT6b5d8Zx9Q9+waHxRSro1w4B7ZtEgE6VP8D+Q0f47qNbO13BHw042oeVP8COLirwUuvy6V1t29HEhvrKClYhBwCzIaycjsNlj7fyjeWPc2T7Ds7kVRZOHcXo3+zi5fse4O2/XF10v2ekV3e2nzyen5/+Fp4dO5Fvz7yYl0edAEBdrbjh0nPaleWvl63n37e0QU2u2jkSwb8/spUfPb6dQ921u5Cr4G9YsQko3lwC3Tff9JWuKuaJDfVFr/Dz25Ra37EZqL6u9ljzVX9xADAbBAor8sKOyYYxdUTAvv2HjnUWPvR0Gzv27ueE2uBtv2zhjpX/zOR9LwBw53ffx7dHj2F2w1GaXtvDq8+1Muf551lwuLKx7aumvY2/f9dCRh4+xFOnvZ4jNbXH1pXTlHHoSHDtPRvaBYDvPbqtaN5XDx4pml5MJVfP/aW7ivnqOWcU7QPIb1Nq/R++tenYufUoILMhpKsr8WVrt3PtPRvYk5oz4PjVXlOq1Avbrfe//Cof3PgwX1pxE6OOHCrybaV9ZP1PeWnkGNpOHEfb6yez9pRpPDfhXHadMI5dJ45lV2pf33XiOF4eOSY390yFym3KKDxeyF3x91b+KrrUFfTouhr2HyrR51BC/l+gYUxdpzIX5smfr+4q5vy6Un8P3a0fSA4AlhmlrrI7XlmXWlc4LLCwIug4qmP73v1cfdcv+MLyDezdf7xC+Z09O/ibVf/Ke3/Vudnlb/vg+P75/D/kprd/5FgzTFNDPTv27i/ZXNJT3TVllJL/t+tIwIgaddsMVHgVXWwUzcdmnX68P6FgX3U14qPnTea+J3Z2quDrasQNHz6nXbD+7N1P8FoKIhJ87PzT+dsFZ5d9nJCr5LsLEtWo8DvyKCAbdCq9IWbZ2u3tKtuxY+q45oNnAbSr8F89eLhPR4Lkh/b92w9+zpsf/y/+x5p7eNNvnuuz/ed995y5fGPWpbQ2VDaVgKi8ku5OYYVZbDhjoYb6OtZdc+Gx5b9etp5/f2Rrp3wfTxV3x4A5pq6GUXW1RUcBdfeLqyfrhhMPA7U+05MKOZ9/zMhaXjt4hIBjoz66uppatnY7V9/1i3YVdakOxe89uq1ks0JtjaiBsjoXi4rgkg0P8tX7buzZ9mX43IV/xt1nzWb/yNH9sv/8r5SOlXRdjUBUHAwb6uv4wofO6nRDU8fmrfx3FF5Z5xWet3L+HqxnHACsRzr+h66vq+Hw0ehUWdTX1TC6w9UZ0Okqrpj6uhq+fMlbigaRc6/7j6LtsGPH1LH2b3JXk6WuJMtRe/QIl697gE+uuYdpu1t7tI9S2k5oIBBfu+ByfnD2+zk4oq5P91+JwqDZ1Y1RhXei5pvAOnY898WdrzawHACGmVL/wcpJbxhTx4FDR461axarvBec29SrirXSq8pSd1NOWXxfyW2eTTfETFtyf6crf8VRLl3/Uz7/4K2cdODVCktfnj//4F9yz5vfSej4c5RKXWVXoq4m1zZe7MfK2ILKuGFMHfteO0SxLs7CIYT5JjBXwNnlADCI5Svnwk7FsR0qaWjfll1s2NjM00/m//1qd6dOvlJX7aXkh6AVu7mmPxW7m7JjADjl1b38r8fu5k8eu7vkfg7V1FJ3tGeV764TxvL59/8pK874vYq37Xh7f2HAfeW3h7tsfuo4qgTKGxXiK20rx6ANAJLmAl8DaoFvRsT1pfJWMwBU8h+t2MROheN5O44wqaQzsq5WnDhqRMnhaX2l1AiN/nLqq3uY/avV/H19a246gV7Mwf7TaW9jb/1JXPrkqmNpd539Pv7l7R9h89iJx9JqoOjV89gxdVz8lglFR4kI+L1p43j2xf0lRwEV44raqmlQBgBJtcAvgfeTe0bwauDyiHiqWP6eBoBiV9jlzuKX375Tx1mtOGHkiE7tot2NhMia6W3P8Y7n1vHBjQ8zc8emvv+CJUv4uzPmcMvGV0pmKTYKqFTbd3ezNLritqFosAaAtwNfiIg5aXkJQER8uVj+ngSArirk7mbxyytnsqb8vsqZYXAo6OoXQG0c5bKnf8aXln+17794+vTjD9f4/d+HUaPKqoQ9msSstMEaAC4F5kbE/0zLfwScHxFXFcvfkwDQXeVdzvSqUxffV1ZbeH/daNNRQ30dBw4fLWv62EoojjLutZeYfGAff/A6sWntZhpeepG/evj2Xu4Znho/lUM1I/jhuz/CzM8sgtpaX1mbDZDBOhtosfvO29VjkhYBiwBOP/30ir+gu9vTy5lLpNybZ/KVWX//AvjChzo3Z3R8gESh0Yd+y7u2PsElT/8nc578z07rn5j4Rsa/sptTXtnTs87TCy7IPWpv9uySUwmcmd4LHxrtCt9scBnoANAKTC5YngTsKMwQEbcAt0DuF0ClX9BdhVzO9KrlDuub2MMhgHU14sTRuY7d7jpfPz7r9ONziDQchL/7O/jKN4GeTx/wlt+dAhPe3u7hGsc+v+51UN+/U9Ca2eAw0AFgNTBd0lRgO3AZ8N/78gu6qpDLnV6142RNxYb15fdVbGKn97ypseiIEigxRvsnP+HI/AXUHjpY6eF2bdSo3OiaT38a/vIvoclX4GZ2XDWGgc4D/oncMNDbIuJLpfJWaxRQV/ssuw07glV3PcjPfvSfbIl6zj60m8/+8B969N1FjRkDn/88/NmfwUkn9d1+zWzIG5SdwJUalDeCHToEt9+eawPfVnx+814ZNw6+9jX46EehrnpTCZjZ0DVYO4EHryNH2j+8eudOWLw49+zT3hg7Fu68EyZNgoYGOO20Hs3BbmbW14Z/ADh4EJ5/vn3FXvjasSP3vmtX6YdXd+WSS+Bzn4OZM/u+7GZm/Wh4BoDdu+Fd78pV7MWu4GtqYPz446NfZs7s9PBqJkzIXa2PGjXw5TczGwDDMwCcdBJMmwbveEf7ij3/Gj8eRgzPQzczK9fwrAVHjIBly6pdCjOzQa2m+yxmZjYcOQCYmWWUA4CZWUY5AJiZZZQDgJlZRjkAmJlllAOAmVlGOQCYmWXUoJ4NVFIb8FwVvvpU4DdV+N6B4uMbuobzsYGPr6/8TkQ0dpdpUAeAapHUUs5UqkOVj2/oGs7HBj6+geYmIDOzjHIAMDPLKAeA4m6pdgH6mY9v6BrOxwY+vgHlPgAzs4zyLwAzs4xyAOhA0lxJmyRtkbS42uXpLUmTJT0kaaOkDZI+ldLHSVopaXN6H1vtsvaUpFpJayXdm5anSno0Hdv3JY2sdhl7SlKDpLskPZ3O4duH2bn73+nv8klJ35M0eiifP0m3Sdol6cmCtKLnSzlfT3XNE5IG/LmyDgAFJNUCNwEXAWcCl0s6s7ql6rXDwGci4s3ALODKdEyLgVURMR1YlZaHqk8BGwuWvwLcmI5tD3BFVUrVN74GPBARbwLOIXecw+LcSWoC/hxojojfBWqByxja5+9bwNwOaaXO10XA9PRaBNw8QGU8xgGgvfOALRHx64g4CNwBzK9ymXolInZGxOPp88vkKpAmcse1NGVbCiyoTgl7R9Ik4GLgm2lZwGzgrpRlKB/bScA7gVsBIuJgROxlmJy7ZARQL2kEMAbYyRA+fxHxMLC7Q3Kp8zUfuD1yHgEaJE0YmJLmOAC01wRsK1huTWnDgqQpwLnAo8BpEbETckECGF+9kvXKPwF/BRxNy6cAeyPicFoeyufw9UAb8G+pieubkk5gmJy7iNgO/AOwlVzFvw9Yw/A5f3mlzlfV6xsHgPZUJG1YDJOSdCLwQ+DTEfFStcvTFyR9ANgVEWsKk4tkHarncAQwE7g5Is4FXmWINvcUk9rC5wNTgYnACeSaRToaquevO1X/W3UAaK8VmFywPAnYUaWy9BlJdeQq/+9ExN0p+YX8z830vqta5euFC4APSXqWXHPdbHK/CBpSkwIM7XPYCrRGxKNp+S5yAWE4nDuA9wHPRERbRBwC7gZ+j+Fz/vJKna+q1zcOAO2tBqanUQgjyXVILa9ymXoltYnfCmyMiK8WrFoOLEyfFwI/Huiy9VZELImISRExhdy5ejAiPgY8BFyasg3JYwOIiOeBbZLOSEnvBZ5iGJy7ZCswS9KY9HeaP75hcf4KlDpfy4FPpNFAs4B9+aaiARMRfhW8gHnAL4FfAZ+rdnn64HjeQe5n5RPAuvSaR66tfBWwOb2Pq3ZZe3mc7wbuTZ9fDzwGbAF+AIyqdvl6cVwzgJZ0/pYBY4fTuQOuBZ4GngS+DYwayucP+B65/oxD5K7wryh1vsg1Ad2U6pr15EZDDWh5fSewmVlGuQnIzCyjHADMzDLKAcDMLKMcAMz4zb3gAAAAH0lEQVTMMsoBwMwsoxwAzMwyygHAzCyjHADMzDLq/wOYYkc/hjIxrgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#增加alpha防止过拟合\n", "lr=LinearRegression(alpha=100)\n", "lr.fit(X[:,:-1],Y)\n", "lr.plot_fit_boundary(X[:,:-1],Y)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3X98VNWd//HXhxAQtRaQ1GpAQcxq8RfYUXHptyq2gKiFdbWl6660tYvfrba13S8t1La22h9arVZ3XR9rq5X+WH/WAl+hKgvYfuuKOhEQEJEgFRKoxEIQK/IjfL5/3DNkkkySmWSSmcl9Px+PeWTuuefOnOvg+dx7zrnnmLsjIiLx06fQBRARkcJQABARiSkFABGRmFIAEBGJKQUAEZGYUgAQEYkpBQARkZhSABARiSkFABGRmOpb6AK0Z8iQIT58+PBCF0NEpKRUV1e/5e4VHeUr6gAwfPhwkslkoYshIlJSzOyNbPJl3QRkZmVmttzMngjbI8zseTNbb2YPm1m/kN4/bNeE/cPTPmN2SF9nZhNzOyUREcmnXPoAvgysTdu+BbjD3auAHcBVIf0qYIe7nwDcEfJhZqOAacDJwCTgP8ysrGvFFxGRzsoqAJjZUOAi4Gdh24DxwGMhyxxgang/JWwT9l8Q8k8BHnL3Pe6+EagBzsrHSYiISO6yvQP4CfA14EDYPhJocPf9YbsWqAzvK4HNAGH/zpD/YHqGYw4ysxlmljSzZH19fQ6nIiIiuegwAJjZxcA2d69OT86Q1TvY194xTQnu97p7wt0TFRUddmKLiEgnZTMKaBzwCTObDBwCHEF0RzDQzPqGq/yhwJaQvxYYBtSaWV/g/cD2tPSU9GNERASYu7yOW59ax5aG3RwzcAAzJ57I1DGtGkvyosM7AHef7e5D3X04USfuEne/AlgKXBayTQfmhffzwzZh/xKPlh2bD0wLo4RGAFXAC3k7ExGREjd3eR2zH19FXcNuHKhr2M3sx1cxd3ldt3xfV54E/jrwVTOrIWrjvy+k3wccGdK/CswCcPc1wCPAK8CTwDXu3tiF7xcR6VVufWodu/c1rxZ372vk1qfWdcv35fQgmLs/AzwT3r9OhlE87v4ecHkbx38f+H6uhRQRiYMtDbtzSu8qzQUkIlIkjhk4IKf0rirqqSBEROIg1fFb17Abo/nwyAHlZcyceGK3fK8CgIhIAaU6flNt/6kx8w5UdvMoIAUAEZECytTxm6r8n501vlu/W30AIiIF1NMdv+kUAERECqinO37TKQCIiBTQzIknMqC8+cTI3dnxm059ACIiBZTq4O2p6R/SKQCIiBTY1DGVPVLht6QmIBGRmFIAEBGJKTUBiYgUQE9O+9wWBQARkR7W8unf1LTPQI8GATUBiYj0sJ6e9rktCgAiIj2skE//pstmTeBDzOwFM1tpZmvM7Lsh/QEz22hmK8JrdEg3M7vLzGrM7GUzOyPts6ab2frwmt7Wd4qI9GaFfPo3XTZ3AHuA8e5+OjAamGRmY8O+me4+OrxWhLQLiZZ7rAJmAPcAmNlg4AbgbKKFZG4ws0H5OxURkdJQyKd/02WzJrC7+zthszy8vJ1DpgC/CMctI1o8/mhgIrDI3be7+w5gETCpa8UXESk9U8dU8sNLT6Vy4ACMaObPH156anGOAjKzMqAaOAG4292fN7N/Ab5vZt8GFgOz3H0PUAlsTju8NqS1lS4iEjuFevo3XVadwO7e6O6jgaHAWWZ2CjAbOAk4ExhMtEg8RGsZtPqIdtKbMbMZZpY0s2R9fX02xRMRkU7IaRSQuzcQLQo/yd23hmaePcDPaVogvhYYlnbYUGBLO+ktv+Ned0+4e6KioiKX4omISA6yGQVUYWYDw/sBwMeAV0O7PmZmwFRgdThkPnBlGA00Ftjp7luBp4AJZjYodP5OCGkiIlIA2fQBHA3MCf0AfYBH3P0JM1tiZhVETTsrgP8d8i8EJgM1wLvAZwHcfbuZ3QS8GPLd6O7b83cqIiKSC3Nvb0BPYSUSCU8mk4UuhohISTGzandPdJRPTwKLiMSUAoCISExpNlARkR5UDNNApygAiIj0kGKZBjpFTUAiIj2kWKaBTlEAEBHpIcUyDXSKAoCISA8plmmgUxQARER6SLFMA52iTmARkR6S6ujVKCARkRgqhmmgU9QEJCISUwoAIiIxpQAgIhJTCgAiIjGlACAiElMKACIiMZXNkpCHmNkLZrbSzNaY2XdD+ggze97M1pvZw2bWL6T3D9s1Yf/wtM+aHdLXmdnE7jopERHpWDZ3AHuA8e5+OjAamBTW+r0FuMPdq4AdwFUh/1XADnc/Abgj5MPMRgHTgJOBScB/hGUmRUR6tbnL6xh38xJGzFrAuJuXMHd5XaGLBGQRADzyTtgsDy8HxgOPhfQ5RAvDA0wJ24T9F4SF46cAD7n7HnffSLRm8Fl5OQsRkSKVmgK6rmE3TtMU0MUQBLLqAzCzMjNbAWwDFgEbgAZ33x+y1AKpR9sqgc0AYf9O4Mj09AzHiIj0SsU2BXS6rAKAuze6+2hgKNFV+4cyZQt/rY19baU3Y2YzzCxpZsn6+vpsiiciUrSKbQrodDmNAnL3BuAZYCww0MxScwkNBbaE97XAMICw//3A9vT0DMekf8e97p5w90RFRUUuxRMRKTrFNgV0umxGAVWY2cDwfgDwMWAtsBS4LGSbDswL7+eHbcL+Je7uIX1aGCU0AqgCXsjXiYiIFKNimwI6XTazgR4NzAkjdvoAj7j7E2b2CvCQmX0PWA7cF/LfB/zSzGqIrvynAbj7GjN7BHgF2A9c4+6NiIj0YsU2BXQ6iy7Oi1MikfBkMlnoYoiIlBQzq3b3REf59CSwiEhMKQCIiMSUAoCISEwpAIiIxJQCgIhITCkAiIjElAKAiEhMKQCIiMSUAoCISEwpAIiIxJQCgIhITCkAiIjElAKAiEhMKQCIiMSUAoCISEwpAIiIxFQ2S0IOM7OlZrbWzNaY2ZdD+nfMrM7MVoTX5LRjZptZjZmtM7OJaemTQlqNmc3qnlMSEZFsZLMk5H7gX939JTN7H1BtZovCvjvc/bb0zGY2imgZyJOBY4D/NrO/CbvvBj5OtED8i2Y2391fyceJiIhIbjoMAO6+Fdga3u8ys7VAe4tZTgEecvc9wMawNvBZYV+Nu78OYGYPhbwKACIiBZBTH4CZDQfGAM+HpGvN7GUzu9/MBoW0SmBz2mG1Ia2tdBERKYCsA4CZHQ78BrjO3d8G7gFGAqOJ7hB+nMqa4XBvJ73l98wws6SZJevr67MtnoiI5CirAGBm5USV/6/d/XEAd3/T3Rvd/QDwU5qaeWqBYWmHDwW2tJPejLvf6+4Jd09UVFTkej4iIpKlbEYBGXAfsNbdb09LPzot298Bq8P7+cA0M+tvZiOAKuAF4EWgysxGmFk/oo7i+fk5DRERyVU2o4DGAf8ErDKzFSHtG8CnzWw0UTPOn4CrAdx9jZk9QtS5ux+4xt0bAczsWuApoAy4393X5PFcREQkB+beqhm+aCQSCU8mk4UuhohISTGzandPdJRPTwKLiMSUAoCISEwpAIiIxJQCgIhITCkAiIjElAKAiEhMKQCIiMSUAoCISEwpAIiIxJQCgIhITCkAiIjElAKAiEhMKQCIiMSUAoCISEwpAIiIxJQCgIhITGWzJOQwM1tqZmvNbI2ZfTmkDzazRWa2PvwdFNLNzO4ysxoze9nMzkj7rOkh/3ozm959pyUiIh3J5g5gP/Cv7v4hYCxwjZmNAmYBi929ClgctgEuJFoHuAqYAdwDUcAAbgDOJlpA/oZU0BARkZ7XYQBw963u/lJ4vwtYC1QCU4A5IdscYGp4PwX4hUeWAQPDAvITgUXuvt3ddwCLgEl5PRsREclaTn0AZjYcGAM8Dxzl7lshChLAB0K2SmBz2mG1Ia2t9JbfMcPMkmaWrK+vz6V4IiKSg6wDgJkdDvwGuM7d324va4Y0bye9eYL7ve6ecPdERUVFtsUTEZEcZRUAzKycqPL/tbs/HpLfDE07hL/bQnotMCzt8KHAlnbSRUSkALIZBWTAfcBad789bdd8IDWSZzowLy39yjAaaCywMzQRPQVMMLNBofN3QkgTEZEC6JtFnnHAPwGrzGxFSPsGcDPwiJldBWwCLg/7FgKTgRrgXeCzAO6+3cxuAl4M+W509+15OQsREcmZubdqhi8aiUTCk8lkoYshIlJSzKza3RMd5dOTwCIiMaUAICISUwoAIiIxpQAgIhJTCgAiIjGlACAiElMKACIiMaUAICISUwoAIiIxpQAgIhJTCgAiIjGlACAiElMKACIiMaUAICISUwoAIiIxpQAgIhJT2SwJeb+ZbTOz1Wlp3zGzOjNbEV6T0/bNNrMaM1tnZhPT0ieFtBozm5X/UxERkVxkcwfwADApQ/od7j46vBYCmNkoYBpwcjjmP8yszMzKgLuBC4FRwKdDXhERKZAO1wR29z+Y2fAsP28K8JC77wE2mlkNcFbYV+PurwOY2UMh7ys5l1hERPKiK30A15rZy6GJaFBIqwQ2p+WpDWltpbdiZjPMLGlmyfr6+i4UT0SkRG3aBK+91u1f09kAcA8wEhgNbAV+HNItQ15vJ711ovu97p5w90RFRUUniyciUqJWrYJzzoFPfhIOHOjWr+pUAHD3N9290d0PAD+lqZmnFhiWlnUosKWddBERSfnlL2H06Kb3fbp3oGaHfQCZmNnR7r41bP4dkBohNB/4LzO7HTgGqAJeILoDqDKzEUAdUUfxP3Sl4CIivYqlNZQ89xwce2y3f2WHAcDMHgTOA4aYWS1wA3CemY0masb5E3A1gLuvMbNHiDp39wPXuHtj+JxrgaeAMuB+d1+T97MRESk1jY3QN60qTiZ7pPIHMPeMTfFFIZFIeDKZLHQxRES6x1/+AkOGNG2/9RYceWSXP9bMqt090VE+PQksIlIIy5Y1r/wbG/NS+edCAUBEpKddemk00ifFvds7fDPpVCewiIh0Unpn79/8DaxbV7Ci6A5ARKQn7N/fvPKfMqWglT8oAIiIdL9166C8vGn76adh7tzClSdQABAR6U4//jGcdFLT9o4d8PGPF648adQHICLSHdxh6FDYEiY9OOMMqK4ubJla0B2AiEi+NTREo3pSlf899xRd5Q+6AxARya/f/x7OO69pe80aGFWcy5/oDkBEJF8++9nmlf/OnUVb+YPuAEREum7vXujfv2l74kT43e+aD/ssQroDEBHpipqa5pX//Pnw5JNFX/mDAoCISOfdeSdUVTVt19fDJZcUrjw5UhOQiEiuDhyAigrYvj3aPvJI2LatIPP5dEVplVZEpNC2b4eysqbK/847o2mcS6zyhywCQFj0fZuZrU5LG2xmi8xsffg7KKSbmd1lZjVhwfgz0o6ZHvKvN7Pp3XM6IiLdaOHC5lM2b9gAX/pS4crTRdmErAeASS3SZgGL3b0KWBy2AS4kWgayCphBtHg8ZjaYaCWxs4nWD74hFTRERIqeO1x0UfRK2b8fjj++cGXKgw4DgLv/AdjeInkKMCe8nwNMTUv/hUeWAQPN7GhgIrDI3be7+w5gEa2DiohI8Xn3XRg8OLr6B5gxIwoIZWWFLVcedLYT+KjUovDuvtXMPhDSK4HNaflqQ1pb6SIixau6GiZMiKZ2APif/2m+kEuJy3evRaaBr95OeusPMJthZkkzS9bX1+e1cCIiWUskotf27bBgQXTV34sqf+h8AHgzNO0Q/m4L6bXAsLR8Q4Et7aS34u73unvC3RMVFRWdLJ6ISCetXx89xJWavO0rX4HJkwtbpm7S2QAwH0iN5JkOzEtLvzKMBhoL7AxNRU8BE8xsUOj8nRDSRESKxyWXRMs0pmzaBLffXrjydLMO+wDM7EHgPGCImdUSjea5GXjEzK4CNgGXh+wLgclADfAu8FkAd99uZjcBL4Z8N7p7y45lEZHC2Lix+Yieww6Dd94pXHl6iLlnbIovColEwpPJZKGLISK92bnnwh/+0LRdUwMjRxauPHlgZtXunugoX+k9uiYikg+rVkVt/anK/4ILoo7eEq/8c6G5gEQkXtxh+PCofT/l9ddhxIiCFalQdAcgIvHxox9Fc/akKv9PfSoKCDGs/EF3ACISBy0XbAH485/hqKMKU54ioTsAEendTj65deXvHvvKHxQARKS3WrYs6uR95ZWmtLfeiip/ARQARKS32b8/qvjTp234z/+MKv70qZxFAUBEepHrroPy8uZp7tEMntKKOoFFpPQtW9Z6orZdu+DwwwtTnhKhACAipWv3bjj00OZp1dVwxhmZ80szagISkdK0eHHzyv8b34iae1T5Z013ACJSWl56CWbPhqefjrbPPDNqAirBRdkLTf/FRKQ01NTAtGnw4Q9DMgm33RY1Ab3wgir/TtIdgIgUt61b4cYb4Wc/g3794PrrYeZMeP/7C12ykqcAICLF6S9/ia7y77ormsphxgz41rfggx8sdMl6DQUAESku27dHq3DdeSf89a9Rs89NN8Vqmuae0qUAYGZ/AnYBjcB+d0+Y2WDgYWA48Cfgk+6+w8wMuJNoxbB3gc+4+0td+X4R6UXeegt+8pPoin/XLvjkJ+Hb347m8pFukY+ek/PdfXTa6jOzgMXuXgUsDtsAFwJV4TUDuCcP3y0ipW7z5ugJ3uOOg+9/HyZOhJUr4eGHVfl3s+7oOp8CzAnv5wBT09J/4ZFlwEAzO7obvj+yf3+3fbSI5MG6dfC5z0VNO//+73DZZbBmDTz6KJx2WqFLFwtdDQAOPG1m1WaWmmzjKHffChD+fiCkVwKb046tDWn5t2sXjBoFP/gB7NnTLV8hIp00bx5MnQof+hA8+CBcfTVs2ABz5kT/30qP6WoAGOfuZxA171xjZh9tJ69lSGs1L6uZzTCzpJkl6+vrO1eq996LriCuvx5OPRUWLNAUsCKFtG9f1KSTSESV/7x50ZO7b7wB//ZvUfOP9LguBQB33xL+bgN+C5wFvJlq2gl/t4XstcCwtMOHAlsyfOa97p5w90RFRUXnClZRAY89Bk8+GW1ffDGcdRbMn69AINKTtm2D730vWoN32jRoaIArroD6+ij9Ax/o8COk+3Q6AJjZYWb2vtR7YAKwGpgPTA/ZpgPzwvv5wJUWGQvsTDUVdZuJE2H1avjpT6MxxVOmwJgxUXA4cKBbv1ok1pJJmD4dhg2Lxu6feio88QS89hr86lcwZEihSyh07Q7gKOCPZrYSeAFY4O5PAjcDHzez9cDHwzbAQuB1oAb4KfCFLnx39vr1g89/PupweuCB6NHxyy+P/kH+139BY2OPFEOk19uzJ2rTP+ecaH6exx+PHt569dXobvyiizRlQ5ExL+ImkUQi4clkMr8f2tgIjzwS3X6+8gpUVUVtkVdc0XohCRHp2OzZcPPNTdtVVXDttfCZz8ARRxSsWHFmZtVpQ/Pbzhe7AJBy4AD89rfRE4YrV8KIETBrVvSPtl+/7vlOkd6irg6GDs28r7FRV/oFlm0AiO+v1KcP/P3fw/LlUefwkCHRcLQTToC7745GEolIkwMH4J//OVpvt2Xlf/XV0QALd1X+JSS+dwAtuUfzi990Ezz7LBx9dDTj4NVXt15xSCROnnkGzj+/dfpHPgJLlvS6ptO5y+u49al11DXspsyMRveDfysHDmDmxBOZOqb9R5hSn7GlYTfHZDgm036g3WNyoSagznKP/sHfeGP0F6L2zS98Ad73vp4ti0ih7NoFt9wSTc3Q0vLlMHp0z5epC9Ir3PcPKMcMGt7d1+z9MQMHMPzIATy7YXu7nzWgvIwfXnpqm5Xz3OV1zH58Fbv3NWY8JtP+8j4GBvsaPeMxuVIAyAdr8ezajTfCF78IAwcWpjwi3em992D8eHjuudb7Zs+OgkHL/ye6UXtX0XOX1/Gd+Wto2L0PgEGHlnPDJSczdUwl35y7igef33zwyn3s8YN4adPOZhVuV1UOHMCzs8Zn3Dfu5iXUNexu85i29uf6Pe3JNgBoOuj2rFwJp5/etP3tb0evr38dvvY1GDy4cGUTyYc9e+CSS2DRotb7fve76Fmadir9TM0l7TWTtKycP332ML439dRWFXpLdQ27mf34qoPbMx9dyb4DTRevO97dx8zHVvJoclOzK/hG9w6v6DtjSzsVeFv7UuntHZvL9+SDAkB7TjstahLatg3OPTcazwzRrfEtt0RPNt55p55mlNKybx//858PM/S7szn2rdpWu//u6w9y8tiTeeL5rTQ8s/Bg+sAB5XznEyc3uwpPb8poDK0JdQ27mfnYSoBmQeCbc1fxq2WbDm43uvOrZZvYWP8OL2zc0axCz2T3vkZufWpddAoZ8u5r7J7KPpNjBg5od1+mK/zUMW3tz/V78kFNQLl4553M/QAVFdHdwtHdN7mpxEdHTR+pfQMPLee9fY3s3hc91d7H4IDT6gp87vI6frJgFSNWLuOyjcs4f91zHPrO282+89wZ9/LGoGM6LFt5H+PWy09n6pjKDpsyBh1azvJvTzi4PXL2woNBorNS9yKFrLV6Ux+A7gBycfjh0R3Bvn3NnxWor4djwv88mzZFj7+LBHOX1/Hd/7uGHe9GzRstr6ShedNIurqG3Xzl4RVc9/AKBg4oZ9ee/TSGq9/U56WkLopTzSWH1P+Zo/64hMMf/i0LNi7nsH3v8Xa/Q3my6mwWnPQR/jh8DHv65vbMy74Dzq1PrWPqmMoOmydalq+rlT80XRFnewWdL0YUdLIZBZTa11YQb2t/e8d023npDqCLZsyI5hpqacmSzEPnpChke5Wdvq+zx8x8bGWzK7t0lVmOPMmG+QFO37qe8ze8yAUbXuSUNzcAUPe+CpaccCZLRp7Js8eNZm/frg3bNGDjzRdl1Zn5p5svOvi+q3cAqStiaN0HAFBeZpw1fFDG/5bjRg5mxead/HVv647g/n37sHf/AY4ZOIDzT6rg8epa3g13VWZwxdnH8r2pp3a63IWgUUA9bckSuOCC1ulf+hLcequeLu5G7Y0IaZkv1WGZuqJLKS8zDuvXN2MnZHmZ0bePHWxqSUmvkFrd0rfzefl06erF3L7gjmZpjdaH6sqTWDryTJaOPJNXhxyX19E7qZEpmZoy0g0cUM6KG5qagFr2AaSMGzm4wz6ATM1a2Y4CSnU0p8rQ1r7eRAGgUPbuhf79W6dfey1ceWU0H3oPDqUrhI4egulINv+TttVkklJeZnzqzGEsfbX+4Njvv+7d3+aVeGdV9nCTxCl/ruGJOde1uf83p4znpvGfp2HAEe2WrWUAzFZ6HwC0bt5qK19KLqOA2grk0jEFgGLw3HPwj/8YTS/x9NPN961dCyedVJhytaMrTyh2NJQvpaNhgpmuEv9x7LHNruIy5Wmps5VcLrq7U/KkbRt58udf7DDfKdc9wjv9m55YN+COT0UPa2XqkPz7D1ceDI6ppo+lr9ZnvDtKydR3kdLVoC/5pQBQbGpr2+4cfuMNOPbYvH5dZ9qrvzl3Fb9etqnV//xlfexgx2PKgPI+HFJedvAJyvNPquA31XVZP2zT1giHttqJy8zY8MPJ7eYphLzeAbgzattG7p73Q0bsaH+pjI/O+CmbBmUedWbAFWkBM9fKWZV56VMAKFa7dkWLX7e8IwiWPPQ039pYllUHJJCxHRRaX/W1J3VFmKnyz1ZnrrYzPeU4fNaCNvOnOhTby9OT2usDyIb5AS5dvZQfL7yjw7yfuew7rP/w/zp4pV7o0SNS3DQMtIi0rLzP/5fbeOJvtzJk8wb++77m6+KMnzaBVJW44MRxzH776yTf2N7s6jr1oE1jo5PeLbnj3X1c9/AKzHJb+XL3vkYefH5zl5oxOnNspmGEqadJM6V3lKezzKCvWZudkKnx9ekytU+nj89/5739zT5vQN8+/J9DtnLVt67quEBnnBGtWTFy5MGkBzo4RBW+dEaP3wGY2STgTqAM+Jm739xW3q7eAXTlVrZVpZ3hyqutJpVs2lNbOm3ra8z/xVfbzXP9hC/w6zGTsyp/Kch0B5CPPoBBh5Zz0WlHt2qS6gO0XAg0/Sq+rSkNUvuy/nfU0MCynz/O2K9mUdkDXHddNM+UJhuUPCnKJiAzKwNeI1oqshZ4Efi0u7+SKX9nA0BbnZHZPlnX0fC29M+Czt3+t6f//r089quZnBrGcbflM5fdwDMjz8zLd+bjqjqXZqD2fotcRwG1lae7p9xl716orobbbouWP8zWD34QLT7Uy0eDSeEUawA4B/iOu08M27MB3P2HmfJ3JgB0VHlnM7tetrP19dQQwGFv1/P/7vlsh/luPvcz3J+YkvODPqk+gLY6ccv7GP369sn4EE3Lz1jw8tZWQwJbynZO9aLhHk318aMfRWve5mLePLj4Yi2SIj2qWPsAKoHNadu1wNn5/IJbn1rX7tV4NrPrZTsDX3fP1Jdy7oQP86Ejf3fwvEa9+ToLH/hSq3yzfv8As37/wMHtnf0P43vjP89jp16AW+YKKL0yThw3uN2ZHduaUz39Sjo1prskOyXdo6G748bldlxlJVx/PVx+OQwaBGVl3VM+kTzr6TuAy4GJ7v75sP1PwFnu/sW0PDOAGQDHHnvsh994442cvmPErAXtNkOU2h3AuJGD+fU/n9PuKKDKjWtZ0M7DQemePe407pp0NVdMn8Anzjmh28pdtHbtgrlzoyk8Orvs56OPRiO5RIpUbJuA2qu8i6EPINVB2fIhnIdf2NxqFEqq8s9GywBxwykDmDB7BqxenfmAPn3guOPg+OObXiNGRK8PfhCOPBIOOyyr7y4ae/fCmjXwzW/CwoUd5+/I5z4Hd9wBRxzR9c8S6UHF2gT0IlBlZiOAOmAa8A/5/IKZE0/MWCHn8lh5ptn62hsF1FbebBfJAA42v3S22WTqmMrW+S9e1Xy7vh7WrYOtW6OKct062LgxuiKur8/6uzIaMgQmTIjWTRg+PJoae8iQqPI89NCoeWXv3uiqe88e2L0b3n4btm+Hv/wler31VtP7DRvgj3/M7rsHDYIdO3Iv8xFHREt9fu1r0WeIxEwhhoFOBn5CNAz0fnfPsOhopCujgEqyDbqQ3nknCgYbN8L990edl6Xiox+NpujmR4tjAAAEdUlEQVRuuZTheefBDTfA2LFwyCEFKZpIIRRlE1CueuWTwKXswAF4+WXYsgWWLo1mQH3ppfx+x2GHRWsu19U1pZWVRWvVjh8PH/kInHhidHehYZQiGRVrE5CUsj59YPTo6DW59zyQJhJXGpwsIhJTCgAiIjGlACAiElMKACIiMaUAICISUwoAIiIxpQAgIhJTCgAiIjFV1E8Cm1k9kNt0oPkxBHirAN/bU3R+pas3nxvo/PLlOHev6ChTUQeAQjGzZDaPUZcqnV/p6s3nBjq/nqYmIBGRmFIAEBGJKQWAzO4tdAG6mc6vdPXmcwOdX49SH4CISEzpDkBEJKYUAFows0lmts7MasxsVqHL01VmNszMlprZWjNbY2ZfDumDzWyRma0Pf0t2TUQzKzOz5Wb2RNgeYWbPh3N72Mz6FbqMnWVmA83sMTN7NfyG5/Sy3+4r4d/lajN70MwOKeXfz8zuN7NtZrY6LS3j72WRu0Jd87KZndHT5VUASGNmZcDdwIXAKODTZjaqsKXqsv3Av7r7h4CxwDXhnGYBi929ClgctkvVl4G1adu3AHeEc9sBXFWQUuXHncCT7n4ScDrRefaK387MKoEvAQl3P4VomdhplPbv9wAwqUVaW7/XhUBVeM0A7umhMh6kANDcWUCNu7/u7nuBh4ApBS5Tl7j7Vnd/KbzfRVSBVBKd15yQbQ4wtTAl7BozGwpcBPwsbBswHngsZCnlczsC+ChwH4C773X3BnrJbxf0BQaYWV/gUGArJfz7ufsfgO0tktv6vaYAv/DIMmCgmR3dMyWNKAA0VwlsTtuuDWm9gpkNB8YAzwNHuftWiIIE8IHClaxLfgJ8DTgQto8EGtx9f9gu5d/weKAe+Hlo4vqZmR1GL/nt3L0OuA3YRFTx7wSq6T2/X0pbv1fB6xsFgOYyrTLeK4ZJmdnhwG+A69z97UKXJx/M7GJgm7tXpydnyFqqv2Ff4AzgHncfA/yVEm3uySS0hU8BRgDHAIcRNYu0VKq/X0cK/m9VAaC5WmBY2vZQYEuBypI3ZlZOVPn/2t0fD8lvpm43w99thSpfF4wDPmFmfyJqrhtPdEcwMDQpQGn/hrVArbs/H7YfIwoIveG3A/gYsNHd6919H/A48Lf0nt8vpa3fq+D1jQJAcy8CVWEUQj+iDqn5BS5Tl4Q28fuAte5+e9qu+cD08H46MK+ny9ZV7j7b3Ye6+3Ci32qJu18BLAUuC9lK8twA3P3PwGYzOzEkXQC8Qi/47YJNwFgzOzT8O02dX6/4/dK09XvNB64Mo4HGAjtTTUU9xt31SnsBk4HXgA3A9YUuTx7O5yNEt5UvAyvCazJRW/liYH34O7jQZe3ieZ4HPBHeHw+8ANQAjwL9C12+LpzXaCAZfr+5wKDe9NsB3wVeBVYDvwT6l/LvBzxI1J+xj+gK/6q2fi+iJqC7Q12zimg0VI+WV08Ci4jElJqARERiSgFARCSmFABERGJKAUBEJKYUAEREYkoBQEQkphQARERiSgFARCSm/j/f3UhbXsO5FgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#换不同的基函数组合\n", "lr=LinearRegression(basis_func=['poly_2','rbf'])\n", "lr.fit(X[:,:-1],Y)\n", "lr.plot_fit_boundary(X[:,:-1],Y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "这里，两个超参数`alpha`和`beta`都需要人工去调试，非常不便，下一节将继续介绍PRML第三章的内容：**证据近似**，它将超参数`alpha`和`beta`的分布也扔到贝叶斯框架中，让训练数据自己去选择合适的`alpha`和`beta`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" } }, "nbformat": 4, "nbformat_minor": 2 }