{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### 一.简介\n", "支持向量机(svm)的想法与前面介绍的感知机模型类似，找一个超平面将正负样本分开，但svm的想法要更深入了一步，它要求正负样本中离超平面最近的点的距离要尽可能的大，所以svm模型建模可以分为两个子问题： \n", "\n", "（1）分的对：怎么能让超平面将正负样本分的开； \n", "（2）分的好：怎么能让距离超平面最近的点的距离尽可能的大。 \n", "\n", "**对于第一个子问题**：将样本分开，与感知机模型一样，我们也可以定义模型目标函数为： \n", "\n", "$$\n", "f(x)=sign(w^Tx+b)\n", "$$ \n", "所以对每对样本$(x,y)$，只要满足$y\\cdot (w^Tx+b)>0$，即表示模型将样本正确分开了 \n", "\n", "**对于第二个子问题**：怎么能让离超平面最近的点的距离尽可能的大，对于这个问题，又可以拆解为两个小问题： \n", "\n", "（1）怎么度量距离？\n", "（2）距离超平面最近的点如何定义？\n", "\n", "距离的度量很简单，可以使用高中时代就知道的点到面的距离公式： \n", "$$\n", "d=\\frac{|w^Tx+b|}{||w||}\n", "$$ \n", "\n", "距离超平面最近的点，我们可以强制定义它为满足$|w^Tx+b|=1$的点（注意，正负样本都要满足），为什么可以这样定义呢？我们可以反过来看，一个训练好的模型可以满足：（1）要使得正负样本距离超平面最近的点的距离都尽可能大，那么这个距离必然要相等，（2）参数$w,b$可以等比例的变化，而不会影响到模型自身，所以$|w^Tx+b|=1$自然也可以满足，所以这时最近的点的距离可以表示为：\n", "\n", "$$\n", "d^*=\\frac{1}{||w||}\n", "$$\n", "\n", "同时第一个子问题的条件要调整为$y\\cdot(w^Tx+b)\\geq1$，而$\\max d^*$可以等价的表示为$\\min \\frac{1}{2}w^Tw$，所以svm模型的求解可以表述为如下优化问题： \n", "\n", "$$\n", "\\min_{w,b} \\frac{1}{2}w^Tw \\\\\n", "s.t.y_i(w^Tx_i+b)\\geq 1,i=1,2,...,N\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 二.原优化问题的对偶问题\n", "对于上面优化问题的求解往往转化为对其对偶问题的求解，首先，构造其拉格朗日函数： \n", "\n", "$$\n", "L(w,b,\\alpha)=\\frac{1}{2}w^Tw+\\sum_{i=1}^N \\alpha_i(1-y_i(w^Tx_i+b)),\\alpha=[\\alpha_1,\\alpha_2,...,\\alpha_N]\n", "$$ \n", "\n", "这时，原优化问题（设为$P$）就等价于： \n", "\n", "$$\n", "\\min_{w,b}\\max_{\\alpha}L(w,b,\\alpha)\\\\\n", "s.t.\\alpha_i\\geq 0,i=1,2,...,N\n", "$$ \n", "\n", "这里简单说明一下为什么等价，首先看里面$\\max$那一层\n", "$$\\max_{\\alpha}L(w,b,\\alpha)\\\\\n", "s.t.\\alpha_i\\geq 0,i=1,2,...,N$$ \n", "\n", "对每个样本都有约束条件$1-y_i(w^Tx_i+b)$，如果满足约束，即$\\leq 0$，必有$\\alpha_i(1-y_i(w^Tx_i+b))=0$，如果不满足，必有$\\alpha_i(1-y_i(w^Tx_i+b))\\rightarrow 正无穷$，所以，（1）如果所有样本均满足约束条件(即$w,b$在可行域内时)，原问题与上面的$\\min\\max$问题等价，（2）如果有任意一个样本不满足约束，这时上面$\\max$问题的函数取值为正无穷，外层再对其求$\\min$会约束其只能在可行域内求最小值，所以两问题是等价的，简单手绘演示一下（两个问题的最优解都是红点标记）： \n", "![avatar](./source/06_原问题与其min-max问题.png) \n", "\n", "假设对于问题$P$我们求得了最优解$w^*,b^*,\\alpha^*$，则必有$L(w^*,b^*,\\alpha^*)=L(w^*,b^*,0)$，所以有： \n", "\n", "$$\n", "\\sum_{i=1}^N\\alpha_i^*(1-y_i({w^*}^Tx_i+b^*))=0(条件1)\n", "$$ \n", "\n", "而最优解自然也满足原始的约束条件，即： \n", "\n", "$$\n", "1-y_i({w^*}^Tx_i+b)\\leq0,i=1,2,...,N(条件2)\\\\\n", "\\alpha_i^*\\geq0,i=1,2,...,N(条件3)\\\\\n", "$$ \n", "\n", "由条件1，2，3，我们可以得出更强地约束条件： \n", "\n", "$$\n", "\\alpha_i^*(1-y_i({w^*}^Tx_i+b^*))=0,i=1,2,...,N(条件4)\n", "$$ \n", "\n", "证明也很简单，由条件2,3可以知道，$\\forall i,\\alpha_i^*(1-y_i({w^*}^Tx_i+b^*))\\leq0$都成立，要使条件1成立，则只能$\\alpha_i^*(1-y_i({w^*}^Tx_i+b^*))=0,i=1,2,...,N$。\n", "\n", "\n", "进一步的，可以推导出这样的关系： \n", "\n", "$$\n", "\\forall \\alpha_i^*>0\\Rightarrow 1-y_i({w^*}^Tx_i+b^*)=0(关系1)\\\\\n", "\\forall 1-y_i({w^*}^Tx_i+b^*)<0\\Rightarrow \\alpha_i^*=0(关系2)\n", "$$ \n", "\n", "所以条件4有个很形象的称呼：**互补松弛条件**，而对于满足关系1的样本，也有个称呼，叫**支持向量** \n", "\n", "好的，我们继续看svm的对偶问题（设为$Q$）的定义： \n", "$$\n", "\\max_{\\alpha}\\min_{w,b}L(w,b,\\alpha)\\\\\n", "s.t.\\alpha_i\\geq 0,i=1,2,...,N\n", "$$ \n", "\n", "很幸运，svm的对偶问题$\\max\\min$与原问题$\\min\\max$等价（等价是指两个问题的最优值、最优解$w,b,\\alpha$均相等，**具体证明需要用到原问题为凸以及slater条件，可以参看《凸优化》**），先看里层的$\\min_{w,b} L(w,b,\\alpha)，$由于$L(w,b,\\alpha)$是关于$w,b$的凸函数，所以对偶问题的最优解必然满足：$L(w,b,\\alpha)$关于$w,b$的偏导为0，即： \n", "\n", "$$\n", "w=\\sum_{i=1}^N\\alpha_iy_ix_i(条件5)\\\\\n", "0=\\sum_{i=1}^N\\alpha_iy_i(条件6)\n", "$$\n", "\n", "消去$w,b$，可得对偶问题关于$\\alpha$的表达式： \n", "\n", "$$\n", "\\max_{\\alpha} \\sum_{i=1}^N\\alpha_i-\\frac{1}{2}\\sum_{i=1}^N\\sum_{j=1}^N\\alpha_i\\alpha_jy_iy_jx_i^Tx_j\\\\\n", "s.t.\\sum_{i=1}^N\\alpha_iy_i=0,\\\\\n", "\\alpha_i\\geq0,i=1,2,...,N\n", "$$ \n", "\n", "显然，等价于如下优化问题（设为$Q^*$）： \n", "\n", "$$\n", "\\min_{\\alpha} \\frac{1}{2}\\sum_{i=1}^N\\sum_{j=1}^N\\alpha_i\\alpha_jy_iy_jx_i^Tx_j-\\sum_{i=1}^N\\alpha_i\\\\\n", "s.t.\\sum_{i=1}^N\\alpha_iy_i=0,\\\\\n", "\\alpha_i\\geq0,i=1,2,...,N\n", "$$\n", "\n", "\n", "该问题是关于$\\alpha$的凸二次规划(QP)问题，可以通过一些优化计算包(比如cvxopt)直接求解最优的$\\alpha^*$，再由条件5，可知：\n", "\n", "$$\n", "w^*=\\sum_{i=1}^N\\alpha_i^*y_ix_i\n", "$$\n", "\n", "\n", "而关于$b^*$，我们可以巧妙求解：找一个样本点$(x_i,y_i)$，它满足对应的$\\alpha_i^*>0$（即支持向量），利用关系1，可知$1-y_i({w^*}^Tx_i+b^*)=0$，所以：$b^*=y_i-{w^*}^Tx_i$ \n", "\n", "这里，条件2,3,4,5,6即是**KKT条件**，而且对于该优化问题，**KKT条件**还是最优解的充分条件（**证明部分...可以参考《凸优化》**），即满足KKT条件的解就是最优解。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 三.SMO求解对偶问题最优解\n", "关于对偶问题($Q^*$)可以使用软件包暴力求解，而且一定能得到最优解，但它的复杂度有点高：（1）变量数与样本数相同，每个变量$\\alpha_i$对应样本$(x_i,y_i)$；（2）约束条件数也与样本数相同；而序列最小最优化化(sequential minimal optimization,SMO)算法是求解SVM对偶问题的一种启发式算法，它的思路是：**每次只选择一个变量优化，而固定住其他变量**，比如选择$\\alpha_1$进行优化，而固定住$\\alpha_i,i=2,3,...,N$，但由于我们的问题中有一个约束：$\\sum_i^N\\alpha_iy_i=0$，需要另外选择一个$\\alpha_2$来配合$\\alpha_1$做改变，当两者中任何一个变量确定后，另外一个也就随之确定了，比如确定$\\alpha_2$后： \n", "\n", "$$\n", "\\alpha_1=-y_i\\sum_{i=2}^N\\alpha_iy_i(关系3)\n", "$$ \n", "\n", "**选择两个变量后，如果优化？** \n", "我们在选择好两个变量后，如何进行优化呢？比如选择的$\\alpha_1,\\alpha_2$，由于剩余的$\\alpha_3,\\alpha_4,...,\\alpha_N$都视作常量，在$Q^*$中可以忽略，重新整理一下此时的$Q^*$： \n", "\n", "$$\n", "\\min_{\\alpha_1,\\alpha_2}\\frac{1}{2}\\alpha_1^2 x_1^Tx_1+\\frac{1}{2}\\alpha_2^2x_2^Tx_2+\\alpha_1\\alpha_2y_1y_2x_1^Tx_2+\\frac{1}{2}\\alpha_1y_1x_1^T\\sum_{i=3}^N\\alpha_iy_ix_i+\\frac{1}{2}\\alpha_2y_2x_2^T\\sum_{i=3}^N\\alpha_iy_ix_i-\\alpha_1-\\alpha_2\\\\\n", "s.t.\\alpha_1y_1+\\alpha_2y_2=-\\sum_{i=3}^N\\alpha_iy_i=\\eta\\\\\n", "\\alpha_1\\geq0,\\alpha_2\\geq0\n", "$$ \n", "\n", "这里求解其实就很简单了，将关系3带入，消掉$\\alpha_1$后可以发现，优化的目标函数其实是关于$\\alpha_2$的二次函数（且开口朝上）： \n", "\n", "$$\n", "\\min_{\\alpha_2}\\frac{1}{2}(x_1-x_2)^T(x_1-x_2)\\alpha_2^2+(-y_2\\eta x_1^Tx_1+y_1\\eta x_1^Tx_2+\\frac{1}{2}y_2x_2^T\\gamma-\\frac{1}{2}y_2x_1^T\\gamma-1+y_1y_2)\\alpha_2\\\\\n", "s.t.\\alpha_2\\geq0,y_1(\\eta-\\alpha_2y_2)\\geq0\n", "$$ \n", "\n", "这里，$\\eta=-\\sum_{i=3}^N\\alpha_iy_i,\\gamma=\\sum_{i=3}^N\\alpha_iy_ix_i$ \n", "\n", "所以该问题无约束的最优解为： \n", "\n", "$$\n", "\\alpha_2^{unc}=-\\frac{-y_2\\eta x_1^Tx_1+y_1\\eta x_1^Tx_2+\\frac{1}{2}y_2x_2^T\\gamma-\\frac{1}{2}y_2x_1^T\\gamma-1+y_1y_2}{(x_1-x_2)^T(x_1-x_2)}(公式1)\n", "$$\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "接下来，我们对上面的表达式做一些优化，大家注意每次迭代时，$\\gamma,\\eta$都有大量的重复计算（每次仅修改了$\\alpha$的两个变量，剩余部分其实无需重复计算），而且对于$\\alpha_1,\\alpha_2$的更新也没有有效利用它上一阶段的取值（记作$\\alpha_1^{old},\\alpha_2^{old}$）： \n", "\n", "我们记： \n", "$$\n", "g(x)=\\sum_{i=1}^N\\alpha_iy_ix_i^Tx+b\n", "$$ \n", "记： \n", "$$\n", "E_i=g(x_i)-y_i\n", "$$ \n", "这里$g(x)$表示模型对$x$的预测值，$E_i$表示预测值与真实值之差，于是我们有： \n", "\n", "$$\n", "x_1^T\\gamma=g(x_1)-\\alpha_1^{old}y_1x_1^Tx_1-\\alpha_2^{old}y_2x_2^Tx_1-b^{old}\\\\\n", "x_2^T\\gamma=g(x_2)-\\alpha_1^{old}y_1x_1^Tx_2-\\alpha_2^{old}y_2x_2^Tx_2-b^{old}\n", "$$\n", "\n", "另外： \n", "\n", "$$\n", "\\eta=\\alpha_1^{old}y_1+\\alpha_2^{old}y_2\n", "$$\n", "带入公式1，可得： \n", "\n", "$$\n", "\\alpha_2^{unc}=\\alpha_2^{old}+\\frac{y_2(E_1^{old}-E_2^{old})}{\\beta}\n", "$$ \n", "\n", "这里$\\beta=(x_1-x_2)^T(x_1-x_2)$，到这一步，可以发现计算量大大降低，因为$E_1^{old},E_2^{old}$可先缓存到内存中，但别忘了$\\alpha_2$还有约束条件$\\alpha_2\\geq0,y_1(\\eta-\\alpha_2y_2)\\geq0$，所以需要进一步对它的最优解分情况讨论： \n", "\n", "当$y_1y_2=1$时，\n", "$$\n", "\\alpha_2^{new}=\\left\\{\\begin{matrix}\n", "0 & \\alpha_2^{unc}<0\\\\ \n", "\\alpha_2^{unc} & 0\\leq\\alpha_2^{unc}\\leq \\alpha_1^{old}+\\alpha_2^{old}\\\\ \n", "\\alpha_1^{old}+\\alpha_2^{old} & \\alpha_2^{unc}>\\alpha_1^{old}+\\alpha_2^{old}\n", "\\end{matrix}\\right.\n", "$$ \n", "\n", "当$y_1y_2=-1$时， \n", "\n", "$$\n", "\\alpha_2^{new}=\\left\\{\\begin{matrix}\n", "\\alpha_2^{unc} & \\alpha_2^{unc}\\geq max\\{0,\\alpha_2^{old}-\\alpha_1^{old}\\}\\\\ \n", "max\\{0,\\alpha_2^{old}-\\alpha_1^{old}\\} & \\alpha_2^{unc}< max\\{0, \\alpha_2^{old}-\\alpha_1^{old}\\}\n", "\\end{matrix}\\right.\n", "$$ \n", "\n", "到这儿，我们可以发现，SMO算法可以极大的方便$Q^*$的求解，而且是以解析解方式，得到$\\alpha_2^{new}$后，由于$\\alpha_1^{new}y_1+\\alpha_2^{new}y_2=\\alpha_1^{old}y_1+\\alpha_2^{old}y_2$，可得到$\\alpha_1^{new}$的更新公式： \n", "$$\n", "\\alpha_1^{new}=\\alpha_1^{old}+y_1y_2(\\alpha_2^{old}-\\alpha_2^{new})\n", "$$\n", "\n", "最后，得到$w$的更新公式： \n", "\n", "$$\n", "w^{new}=w^{old}+(\\alpha_1^{new}-\\alpha_1^{old})y_1x_1+(\\alpha_2^{new}-\\alpha_2^{old})y_2x_2\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "**对$b$和$E$的更新**\n", "\n", "而对于$b$的更新同样借助于$\\alpha_1,\\alpha_2$更新，在更新后，倾向于$\\alpha_1^{new}>0,\\alpha_2^{new}>0$，还记得前面的互补松弛条件吧（条件4），即对于$\\alpha_i>0$的情况，必然要有$1-y_i(w^Tx_i+b)=0$成立，即$w^Tx_i+b=y_i$，所以对$(x_1,y_1),(x_2,y_2)$有如下关系： \n", "\n", "$$\n", "{w^{new}}^Tx_1+b=y_1(关系4)\\\\\n", "{w^{new}}^Tx_2+b=y_2(关系5)\\\\\n", "$$ \n", "对关系4和关系5可以分别计算出$b_1^{new}=y_1-{w^{new}}^Tx_1,b_2^{new}=y_2-{w^{new}}^Tx_2$，对$b$的更新，可以取两者的均值： \n", "\n", "$$\n", "b^{new}=\\frac{b_1^{new}+b_2^{new}}{2}\n", "$$ \n", "\n", "接下来，对于$E_1,E_2$的更新就很自然了： \n", "\n", "$$\n", "E_1^{new}={w^{new}}^Tx_1+b^{new}-y_1\\\\\n", "E_2^{new}={w^{new}}^Tx_2+b^{new}-y_2\n", "$$\n", "那接下来还有一个问题，那就是$\\alpha_1,\\alpha_2$如何选择的问题 " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**如何选择两个优化变量？** \n", "\n", "这可以启发式选择，分为两步：第一步是如何选择$\\alpha_1$，第二步是在选定$\\alpha_1$时，如何选择一个不错的$\\alpha_2$： \n", "\n", "**$\\alpha_1$的选择** \n", "\n", "选择$\\alpha_1$同感知机模型类似，选择一个不满足KKT条件的点$(x_i,y_i)$，即不满足如下两种情况之一的点： \n", "\n", "$$\n", "\\left\\{\\begin{matrix}\n", "\\alpha_i=0\\Leftrightarrow y_i(w^Tx_i+b)\\geq1\\\\ \n", "\\alpha_i>0\\Leftrightarrow y_i(w^Tx_i+b)=1\n", "\\end{matrix}\\right.\n", "$$\n", "\n", "**$\\alpha_2$的选择** \n", "\n", "对$\\alpha_2$的选择倾向于选择使其变化尽可能大的点，由前面的更新公式可知是使$\\mid E_1^{old}-E_2^{old}\\mid$最大的点，所以选择的两个点$(x_1,y_1)$和$(x_2,y_2)$会更倾向于选择异类的点" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 四.代码实现" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import copy\n", "import random\n", "import os\n", "os.chdir('../')\n", "from ml_models import utils\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "#定义一个绘制决策边界以及支持向量的函数（并放到utils中）\n", "def plot_decision_function(X, y, clf, support_vectors=None):\n", " plot_step = 0.02\n", " x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n", " y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n", " xx, yy = np.meshgrid(np.arange(x_min, x_max, plot_step),\n", " np.arange(y_min, y_max, plot_step))\n", "\n", " Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])\n", " Z = Z.reshape(xx.shape)\n", " plt.contourf(xx, yy, Z, alpha=0.4)\n", " plt.scatter(X[:, 0], X[:, 1], alpha=0.8, c=y, edgecolor='k')\n", " # 绘制支持向量\n", " if support_vectors is not None:\n", " plt.scatter(X[support_vectors, 0], X[support_vectors, 1], s=80, c='none', alpha=0.7, edgecolor='red')" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "硬间隔支持向量机的smo实现,放到ml_models.svm模块\n", "\"\"\"\n", "class HardMarginSVM(object):\n", " def __init__(self, epochs=100):\n", " self.w = None\n", " self.b = None\n", " self.alpha = None\n", " self.E = None\n", " self.epochs = epochs\n", " # 记录支持向量\n", " self.support_vectors = None\n", "\n", " def init_params(self, X, y):\n", " \"\"\"\n", " :param X: (n_samples,n_features)\n", " :param y: (n_samples,) y_i\\in\\{0,1\\}\n", " :return:\n", " \"\"\"\n", " n_samples, n_features = X.shape\n", " self.w = np.zeros(n_features)\n", " self.b = .0\n", " self.alpha = np.zeros(n_samples)\n", " self.E = np.zeros(n_samples)\n", " # 初始化E\n", " for i in range(0, n_samples):\n", " self.E[i] = np.dot(self.w, X[i, :]) + self.b - y[i]\n", "\n", " def _select_j(self, best_i):\n", " \"\"\"\n", " 选择j\n", " :param best_i:\n", " :return:\n", " \"\"\"\n", " valid_j_list = [i for i in range(0, len(self.alpha)) if self.alpha[i] > 0 and i != best_i]\n", " best_j = -1\n", " # 优先选择使得|E_i-E_j|最大的j\n", " if len(valid_j_list) > 0:\n", " max_e = 0\n", " for j in valid_j_list:\n", " current_e = np.abs(self.E[best_i] - self.E[j])\n", " if current_e > max_e:\n", " best_j = j\n", " max_e = current_e\n", " else:\n", " # 随机选择\n", " l = list(range(len(self.alpha)))\n", " seq = l[: best_i] + l[best_i + 1:]\n", " best_j = random.choice(seq)\n", " return best_j\n", "\n", " def _meet_kkt(self, w, b, x_i, y_i, alpha_i):\n", " \"\"\"\n", " 判断是否满足KKT条件\n", "\n", " :param w:\n", " :param b:\n", " :param x_i:\n", " :param y_i:\n", " :return:\n", " \"\"\"\n", " if alpha_i < 1e-7:\n", " return y_i * (np.dot(w, x_i) + b) >= 1\n", " else:\n", " return abs(y_i * (np.dot(w, x_i) + b) - 1) < 1e-7\n", "\n", " def fit(self, X, y2, show_train_process=False):\n", " \"\"\"\n", "\n", " :param X:\n", " :param y2:\n", " :param show_train_process: 显示训练过程\n", " :return:\n", " \"\"\"\n", " y = copy.deepcopy(y2)\n", " y[y == 0] = -1\n", " # 初始化参数\n", " self.init_params(X, y)\n", " for _ in range(0, self.epochs):\n", " if_all_match_kkt = True\n", " for i in range(0, len(self.alpha)):\n", " x_i = X[i, :]\n", " y_i = y[i]\n", " alpha_i_old = self.alpha[i]\n", " E_i_old = self.E[i]\n", " # 外层循环：选择违反KKT条件的点i\n", " if not self._meet_kkt(self.w, self.b, x_i, y_i, alpha_i_old):\n", " if_all_match_kkt = False\n", " # 内层循环，选择使|Ei-Ej|最大的点j\n", " best_j = self._select_j(i)\n", "\n", " alpha_j_old = self.alpha[best_j]\n", " x_j = X[best_j, :]\n", " y_j = y[best_j]\n", " E_j_old = self.E[best_j]\n", "\n", " # 进行更新\n", " # 1.首先获取无裁剪的最优alpha_2\n", " eta = np.dot(x_i - x_j, x_i - x_j)\n", " # 如果x_i和x_j很接近，则跳过\n", " if eta < 1e-3:\n", " continue\n", " alpha_j_unc = alpha_j_old + y_j * (E_i_old - E_j_old) / eta\n", " # 2.裁剪并得到new alpha_2\n", " if y_i == y_j:\n", " if alpha_j_unc < 0:\n", " alpha_j_new = 0\n", " elif 0 <= alpha_j_unc <= alpha_i_old + alpha_j_old:\n", " alpha_j_new = alpha_j_unc\n", " else:\n", " alpha_j_new = alpha_i_old + alpha_j_old\n", " else:\n", " if alpha_j_unc < max(0, alpha_j_old - alpha_i_old):\n", " alpha_j_new = max(0, alpha_j_old - alpha_i_old)\n", " else:\n", " alpha_j_new = alpha_j_unc\n", "\n", " # 如果变化不够大则跳过\n", " if np.abs(alpha_j_new - alpha_j_old) < 1e-5:\n", " continue\n", " # 3.得到alpha_1_new\n", " alpha_i_new = alpha_i_old + y_i * y_j * (alpha_j_old - alpha_j_new)\n", " # 4.更新w\n", " self.w = self.w + (alpha_i_new - alpha_i_old) * y_i * x_i + (alpha_j_new - alpha_j_old) * y_j * x_j\n", " # 5.更新alpha_1,alpha_2\n", " self.alpha[i] = alpha_i_new\n", " self.alpha[best_j] = alpha_j_new\n", " # 6.更新b\n", " b_i_new = y_i - np.dot(self.w, x_i)\n", " b_j_new = y_j - np.dot(self.w, x_j)\n", " if alpha_i_new > 0:\n", " self.b = b_i_new\n", " elif alpha_j_new > 0:\n", " self.b = b_j_new\n", " else:\n", " self.b = (b_i_new + b_j_new) / 2.0\n", " # 7.更新E\n", " for k in range(0, len(self.E)):\n", " self.E[k] = np.dot(self.w, X[k, :]) + self.b - y[k]\n", " # 显示训练过程\n", " if show_train_process is True:\n", " utils.plot_decision_function(X, y2, self, [i, best_j])\n", " utils.plt.pause(0.1)\n", " utils.plt.clf()\n", "\n", " # 如果所有的点都满足KKT条件，则中止\n", " if if_all_match_kkt is True:\n", " break\n", " # 计算支持向量\n", " self.support_vectors = np.where(self.alpha > 1e-3)[0]\n", " # 利用所有的支持向量，更新b\n", " self.b = np.mean([y[s] - np.dot(self.w, X[s, :]) for s in self.support_vectors.tolist()])\n", " # 显示最终结果\n", " if show_train_process is True:\n", " utils.plot_decision_function(X, y2, self, self.support_vectors)\n", " utils.plt.show()\n", "\n", " def get_params(self):\n", " \"\"\"\n", " 输出原始的系数\n", " :return: w\n", " \"\"\"\n", "\n", " return self.w, self.b\n", "\n", " def predict_proba(self, x):\n", " \"\"\"\n", " :param x:ndarray格式数据: m x n\n", " :return: m x 1\n", " \"\"\"\n", " return utils.sigmoid(x.dot(self.w) + self.b)\n", "\n", " def predict(self, x):\n", " \"\"\"\n", " :param x:ndarray格式数据: m x n\n", " :return: m x 1\n", " \"\"\"\n", " proba = self.predict_proba(x)\n", " return (proba >= 0.5).astype(int)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 五.效果检验" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import make_classification\n", "# 生成分类数据\n", "data, target = make_classification(n_samples=100, n_features=2, n_classes=2, n_informative=1, n_redundant=0,\n", " n_repeated=0, n_clusters_per_class=1, class_sep=2.0)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd0VNX2wPHvmT6pBELoTUBFEQRRUBQVG9i7qFh5FhTbU58Ve+8FfYpi+9lQsaJPUAERkaogvYk0KSGkT597fn9MCElmQtq0JPuzFmuZO7fskbDnzrnn7K201gghhGg6TIkOQAghRHRJYhdCiCZGErsQQjQxktiFEKKJkcQuhBBNjCR2IYRoYiSxCyFEEyOJXQghmhhJ7EII0cRYEnHR7Oxs3bVr10RcWgghGq2FCxfu1Fq3rmm/Bid2pZQDmAnYy873mdb6/r0d07VrVxYsWNDQSwshRLOilNpQm/2iccfuBYZqrUuUUlZgllLqf1rrOVE4txBCiDpqcGLXoSpiJWU/Wsv+SGUxIYRIkKg8PFVKmZVSi4AdwA9a67nROK8QQoi6i0pi11oHtdYHAx2Bw5RSvavuo5S6Wim1QCm1IDc3NxqXFUIIEUFUpztqrQuAGcCwCK+N11oP0FoPaN26xoe6Qggh6ikas2JaA36tdYFSygkcDzzZ4MgaiWAgyNR3Z/D9W9PQGk66/BiGXTkUs8Wc6NCEEM1UNGbFtAPeVUqZCX0D+ERrPTkK5016WmvuP/tpFk1bitflBWD9kg38+tV8Hp18F0qpBEcohGiOojEr5k+gXxRiaXSWzV7F4ul7kjqAp9TLkpnLWfLLCvoMOSCB0QkhmispKdAAf/68HJ/HH7bd6/KxeMayBEQkhIgVHdyGUfgwxs5TMXZdg/Yl7yLLhJQUaCpa5GRic1jxlHorbbc5bbTIyUxQVEKIaNPBLeidZ4IuBQLAavSu39AZj2BKOT3R4YWRO/YGOPq8QShT+Di6yaw45oIjEhCRECIWdMk40CWEkvpuHih+GK0D1R2WMJLYGyA1M5Unp95Hq3ZZONMcONMctGzbgsf/dy/pWWmJDk8IES3e2UAwfLv2QXBL3MOpiQzFNFCvgT35cNNr/LU4VJtnn75dMJnk81KIJsXUCoytEV4Igin5hl0lA0WByWSiR79u9OjXTZK6EE2QSrsacFbZagP7EJSpRSJC2ivJQkIIUQPlGAZpowEHqDTADrZBqMynEh1aRDIUI4QQtWBKuxadcgkE14EpB2Vum+iQqiWJXQghakmZUsHUJ9Fh1EiGYoQQoomRxC6EEE2MJHYhhGhiJLE3AsFgEHeJm1AXQiGE2DtJ7EksGAzy9tiPOCvrcs5qeTkXdbmWmZ/9luiwhBBJThJ7Eptw1wdMev5b3CUeggGDnZt38dTl41j4w+JEhyaESGKS2JOU1+3l61emVKr1DqGSwO89+EmCohJCNAaS2JNU4c5iqKYD0z9rt8c5GiFEYyKJPUlltcnEbI7819O9b5c4RyOEaEwksScpq83KyPvOxZFir7TdnmLjikcuTFBUQojGQEoKJLFz/30amdkZfPDIJPK25tP94K5c8/Ql7Hdoj0SHJoRIYioRc6MHDBigFyxI3n6BQgiRjJRSC7XWA2raT4ZihBCiiZHELoQQTYwkdiGEaGIksQshRBMjiV0IIZoYSexCCNHENDixK6U6KaWmK6VWKKWWKaVuikZgQggh6icaC5QCwK1a69+VUunAQqXUD1rr5VE4txBCiDpq8B271nqr1vr3sv8uBlYAHRp6XiGEEPUT1TF2pVRXoB8wN8JrVyulFiilFuTm5kbzskIIISqIWmJXSqUBk4CbtdZFVV/XWo/XWg/QWg9o3bp1tC7b6AQDQT54dBIjOl7NGS0u46HznmHr+sZVhtfr9vLO/RO5uMtoRnS8mtdufZfSwtJEhyWEKBOVWjFKKSswGZiitX6upv2bc62YRy98nt++XoDX7QPAZFKkZqXy1vIXaNE6M8HR1Uxrza3H3M+q+WvxefwAWO0W2u3ThtcXPYPFKnXlhIiVuNWKUUopYAKwojZJvTnbun47s7+aX57UAQxD4y318s1/pyYwstpbOmsla37/qzypA/i9AXI35TH7q/kJjEwIsVs0hmIGA5cAQ5VSi8r+nByF8zY56//ciMUWfkfr8/hZ/tuqBERUd6sXrCPoD4Ztd5d4WD5nTQIiEkJU1eDvzVrrWUDkHm6iknbd2xAMhCdFi9VClwM6xi2Ov/7cwFv3fsTKuWto3bEVF997DkeeNbBWx+Z0aY3FbsHvC1Tabk+x06F7m1iEK4SoI1l5GkfdenemZ/99sFa5a7fYzJwxZnhcYli/ZAM3Db6Hed8upDC3iLV/rOeJS17mm9em1Or4Qaf2JyXdiclU+bPcarMw9KIjYxGyEKKOJLHH2SOT7+LIcwZhsVkwW8x07d2JJ3+4j3bd4nO3+/bYj/G6vFR8Zu51eZlw94cE/IHqDyxjtVl5YdYj7D9oXyw2C1abhe59u/LczIdIzUyNYeRCiNqSDkoJ4vf5CfgCONOccb3u+e3+Rf72wrDt9hQ7E5Y9T5sutZ+KWrSrGCNoNIrZPELsprUG/wLwLwFzB7Afi1K2RIdVK7WdFSNz0xLEarNitVnjft02XVpHTOza0GRkp9fpXBkt67a/EImmtRe9axQEloL2g7KBSoGWH6EsnRMdXtTIUEwzM3LsudhTKt+d2FNsnHjZ0ThTHQmKSoj40CXjwb8YtAvwgy4FIw9deGuiQ4sqSezNzMBTDmHMS6NIb5mGzWnD5rRx4mXHcN2LVyQ6NCFiz/054K2y0QD/crSxKxERxYQMxTRDw64cygmXHU3+tgLSW6Zhd9oTHZIQcRI+3ThEgTbiGkksyR17M2U2m8nu0EqSumhenKcCER6UWrqizNlxDydWJLELIZoNlXodWLqC2j011wkqA5X5bMT9dWAzRuFYjNxhGLv+hfY1jrIZMhQjhGg2lCkNWn0B3mlo3yKUpRM4TkWZMsL21YGN6Lyzyh60BiH4F3rXPHTmY5icp8Y/+DqQxC6EaFaUsoLjJJTjpL3up0teDM2aoeLYuweKHkY7hqOUOaZxNoQMxQghRCS+uVRO6mW0G4xtcQ+nLuSOPYHWL9nAHz8tJb1lGoPPOoyU9NqvQi3aVUzAF6Bl26wYRihE06W9M9Elr0LwH7AejEq/EWXpsWcHUzYYOyIcaYBK7tXWktgTQGvNs6NeZcbE2RiGxmI18/INb/L4/+7lwCP22+uxuZvzeOziF1k5dw1KQdtuOdzx3o3sN6B7nKIXovEzXJOg6EHAE9rgnYr2zYSWn6KsPQFQqVehC+8G3BWOtIPjuNBYfRKToZgEmPX5XH7+9De8bh9+rx93iQd3sYf7z3ySYLC6ebYQDAa5ZchYls9eRcAXwO8NsGnlP9x+3IPkby+I4zuoHb/Pz9tjP+LcNqM4LW0k9535JFv/alxtAEXTo3UAip+gPKkDYIB2o0teKN+inKdA2mhCM2fSABvYh6AyHotzxHUniT0B/jfhJzylVVe/gc8bYMVemlX8/uMSivJChbcqCvoDTHlnRrTDbLCHz3+OSc9NpjC3CI/Ly5zJC7n+sDspyA2vVSNE3Bg7QYf/+wMN/j9C/6U12jcPZW4DLd9HtXwf1XoGpqxXUKaU+MZbD5LYEyAYiLzCTSkiNuLYLXfjTnQwvBqnz+Nny9qtUYsvGjav/oeFP/xZqQ2gNjRel5fJrzWONoCiiVIZQDVVbU1t0MFc9M5h6Pyr0UUPwq6L0SUvgym5x9UrksSeACdcejSO1PAVn0opDjh832qP23dAd3SEX0hHmoODjuwV1Rgbav3STVis4dPBfB4/K+etTUBEQoQoUwo4TwOqFL1TTlTaaHThfyC4KTR/XbsAD3h/RZe+nYhw60USewIcO2IwfY85EEdZNUWr3Yo9xc49H92811K+Pfp1o+8xvbE79yyJttgstGyTyTEXHBHzuOuiQ4+2Eb+ZWG0Wuh3UdMqjisZJZTwAzlOAsrK9Kg3SbgXb4eCbB1RtOuMB18fxD7SepNFGgmitWTR9KQt/+JPM7HSOvfBIstu3rPE4v8/PpOcm892bP+Hz+hly7iBGjj03KWuj3zJkLKvmr8Xv3fOPxJnmYMLyF2jdsVUCIxMiRBslYOSBuR1K2dDBPHTu0YAvfGdTNqac2XGPsaLaNtqQxN5M+Lx+pn04i18m/UZaizROG30ivQfvH9NruordvHzDBH6e+CtBf5Ae/btx82vX0LP/PjG9rhANYeQOg+BfVbZawHkepswHExLTbpLYRTm/z8+/j76fv5duxFPqRSmwOe1c9uD5nHfr6TG/fjAYJBgwsNnj3zFKiLrSvsXo/MtABwjduTvB1AKV/QXKVPO36liqbWKXMfZmYPpHv5YndQCtQw2s3xn7MUW7imN+fbPZXJ7UfR4fy+esZtOqLTG/rhD1oWx9UdlTIPUacJwG6Xeisr9LeFKvC1l52gz8+sW8iPPmLTYLS39ZyRFnHBqXOKa+N4NxYyagTIpgIEiHHu14+Os7yOlc+wbaQsSDMrdBpd+Q6DDqTe7Ym4H0lqkokwp/QUNqZnwWW6xasI6XRr+Bu8SDq8iN1+Xj72WbuHPYoyRiOFCIpkwSeyOzeuE6prwznaW/rqx1Qjz12hOxOcLHtx1pDnoftT87Nu3kgXOe5mTnRZyWPpLnr3md0iJXVOP+8uXv8Hn8lbYZQYPcTTtZ83vVB1VCiIaQoZhGwuv2cs8pj7Ny3trQ3bfWdOjZjqd/up/0rL0XJNr/sJ5c9eRIxt/+PhabGa0hJd3B49/fi8/tY8zAuyjMLcIIGvi9MPXdGaz5/S9emfcESkW406+HnVt2RfwgMlvMFOYWReUaQoiQqNyxK6XeUkrtUEotjcb5RLi3x37M8jmr8bq8eEo8eEq9bFi+mRdHv1Gr48+4fjifbB3P3R/cxKOT7+KDDf+lW+/O/PTBLNzF7kr1ZwK+AJtW/cOSX1ZELf5Bp/SvtLBqN7/Xz36H9YhwhBCivqI1FPMOMCxK5xIRTH1nBv4qQxkBX4Bfv5i71/oyFaVmpjLwlEM46KhemM2h5f7rFq2P+GBVBw02LNvU8MDLDP/X8bRqn1VpSMiRaueiu89OysVVQuymffMxdp6Fse0AjB1HYpT+X9I/F4rKUIzWeqZSqms0ziUi83v9EbcbhsYwDMzUr01Xt4O6YE+x43VVTu4ms4mO+7Wv1zkjSUl38uqCJ/nqle+Z9fk8MrPTOfPGkxl4cv+oXUOIaNO+xehdoygv8WvsgOJn0LoAlZa8s2bi9vBUKXW1UmqBUmpBbm5uvC7bZBw6vB8mc+W/LqVg/8N67LW+TE2OG3kUjhQ7pgqzZiw2C226tqbvMQfW+7yRpGamctHd5/Dqgid5/Pt7JamLuNGBjRj5N2NsH4SROwzD9WnYXbcObscovA9jx9EYO89Eu78O9T2tVLcdwA0lb6Ijlv5NDnFL7Frr8VrrAVrrAa1by7zlurr22cvIzE7HkRKqCmlz2kjJSOGW8dc26LypGSm8POcx+h13ECazCYvNwpBzB/HsjAcxmWTSlGj8dHAbOu8s8H4PeleoXEDRI+jiZ/bsY+xC7zwD3J+BsRUCy9GFY8H3e/UnDiZv0xiZFdNI5HTK5u1VLzH13RmsmreWbgd1ZtiVQ8nMzmjwudvt04YnpozFMAyUUlGbCSNEMtClb4QaUFdqTO0G13votKtRpkx06bugS6hc1dFN9TSYk/cGVRJ7nPl9fkwmE2ZL3cfEUzNSOOuGk2MQVYjcoYsmybeQ8DK8gLJBYC3YDgHfHCJWdIxIQcpFKFX75vPxFq3pjh8BvwH7KaU2K6VGReO8TcmmVVu45aixnJo6klNSL+bBc56hcKfM3xYi5sxdgEgrr31gble2T0dqnw41WPtGKbjYiEpi11pfqLVup7W2aq07aq0nROO8TUVxfgk3Db6XZbNXYQQNgv4gcyYv4NZj7scwIrfJE0JEh0q7CqjascwGtkEoc2jml0q9MrSttnxzohVeTMh37zj44b2f8Xl8lZ7CB/xBdmzcyeIZyxIYmRBNn7L2RrV4EUxtCCV4GzhOQLV4ocI+B0Lms2BqCTgBK9WnRxOY28Q87oaQMfY4+HvZJryu8PE7wzDYsmYb/YYeFLdYCncWMeWdGWxatYUDD9+PY0YMLp9pI0RTpRzHgn0mGLmg0kJ9T6swOU9AO4ZCcAuY0tHeOVB4U4Sz2VDOs2MfdAPIHXsc7Dege7XNq/fpE7/+n3/9uYFLe4zh3fsn8v2Eabxy01uMOvBmCnIL4xaDEImilEKZcyIm9T37mFGWzihTFibncMgcByqVUKq0gmqByvovoEOLl4ySeIVfJ5LY42DoxUeRmplSaYGR1W5ln75d6DVo37jF8dTl43AVufG5Q98ePKVe8v7J5617PopbDEI0Jibniaic31HZk1GtPoXsqejSN9G5J6Lzr0DvOAKj5NVEhxlGEnscOFMdjJv3BEefdzjONAdpWamces0JPDFlbNzmjJcWlvJ3hNovQX+QX7+cF5cYhGiMlFIoSw+U9QAouhN88wFv2bx3D5S8jvZ8n+gwK5Ex9jjJbt+Suz+8OWHXr1qOoKLdJQk2r9nK5Nemsn1DLv2P78PxlwzBmeqIV4hCJDVt5IN3FuHz3d3o0jdQjuSpgyiJvZlwpjk5+JgDWTR9KcHAnimWNoeVYVcey/zv/+DBc58h4A8S9AdZMGURnz33Da/Me4K0FqkJjFyIJGEUgrKE5r9XFcyLfzx7IUMxjYTP6+efddtwl+xtmfPe/efdMbTt1gZnugN7ih1Hqp0DB+/PBXeeyVOXjcPr8hH0h0oAe0q95G7K47Nnv47WWxCicTN3JDQNMuwFsA+OdzR7JXfsSU5rzSdPf8X7j0wCwAgEGTZqKNc9f0WdyxK0bJvFWyteYNG0pWxbv4Pu/bqx34DubFi+CU+E6Zh+r5+Zn83h8ocvjMp7EaIxU8qCzrgPCu9lTx0ZK6hUVNqYRIYWRhJ7kvvhvZ95/+HPKjXDmPL2dGwOG9c8fWmdz2cymeh/fJ9K25xpDoxg5GYdzvTkrYchRCxo3x9lD0PNKOepoYemZUzO09Dm9ujSNyG4GWyHo1JHoZJswZIk9iT34WOfh3U48rp8TP7vVP71+MX1KiZWVU7n1nTt3Ym1f/xdqUWeI9XOmWOGN/j8QiQrrTUEVgMaLPuiix8H1yeEarArtOt9dNpoTGmjy49RtkNQtkMSFXKtyBh7ksvfXhBxe8AfwFNatQFA/d3/2W206doaZ5qDlHQnVoeV40cO4fhLhkTtGkIkE8PzI3rHYHTe+ehdI9A7jgDXR4SGWTShMr8eKHkVHYhem8h4kMSe5Hoesk/E7S1yMknJqH4FXV1ZHTb2HdAdvzeA1+Oj/3F9uPTBC8Lm2a+ct4Zbj7mfM1pcypW9bmLah79ELQYh4kEHNmHsPBUKrgO9E3CDdoWacFRXutc7PZ4hNpgk9iR31ZOXYE+xUzG/2lNsjH7+8qgtbgr4A9x0xD3MmjSXgD9QPt3xxsPvJuDfU8d69cJ13Db0Af6cuRxXkZtNq/7huatf5/OXvo1KHELEmtZB9K5LILCmDkeZQrXbGxFJ7EluvwHdefHXRxh02gCyO7aiz9EH8Mg3dzHk3MOjdo3fvllIQW4hwcCeB6jBQJDCnUXM/mp++ba37/04rJiZ1+Xlvfs/qfQBIETS8s0FXUhoqKW2DHCcGKuIYkIenjYC3ft25aEv74jZ+Tcu34ynJLwxr7vEw4blm8t/XvvH+ojHlxa6uLjLaE659kQuvPPMBjXXFiKWdHAH6MgzwEJsgC5biKSAIGQ+hTK1jFOE0SGJXdC5VwccaXbcxZUfxjrTHHTu1RGApbNW7PVh7a5tBXzy5JesnLuGx769O6bxClEfWvvA9QGhGS+RWMF2KGQ8ifLNBGUC+1BQ6WijKFTuVzWOQY7GEWUzk7+9gEXTl7J1fXy6oB9++gAyW2VUmjpptphJb5nGEWcMYM3vf3HnsEfDpl1W5XX7+PPnZaxb/HeMIxai7nTp2xBYVc2rdki7A5U1HpMlB5VyLjjOQrs+RG8/BL1jEHr7IAzXJ3GNub4ksScRwzB48bo3uLjrdTxw9tP868BbuPvkR3FHcVpjJBarhRdnP8IRZwzAYjVjsZo5/PQBvPzbY1htVv7vwU/xufee1HdTSrFu0d8xjVeIenFPIvLdugJzV/BNB+/P5Z3OdMmrUDIOcBFqhl0ARfdjuL+LW8j1JUMxSeSLl77jh/d+xu/14/f6AVg0fRkvXf8md7wT2yXLLdtmcd+nt5X/UleccbNu8d/oWj5rUkrRpmvrWIQoRANV119YQ3AVBFehfbPBPhxaPAel/yWU0CsKQtFD4Dw5xrE2jNyxJ5EvXvwOr6vynbHf6+fnibPxlSX6WFNKhU2j7HJgp1oda7aaad2pFX2GHFDzzkLEm+MMwptaV6XB+x3a9TnVzmnXu6IcWPRJYk8iJYWlEbdrwyjvepQIl9x3HvaUyvN47Sl2jr1wMPv06VI+fNP/+D48M/3BuDUPEaIuVNq/wNIT1O6FfXv5PS19Za/n0rq6u//kIEMxSeTgY3sz+6v5aKPyuEdO59akZkZvlWld9RrYk4e+vINXbnqLjSu2kNoihXNuPoWL7jkHs9lMcX4JFptFmnKIpKaUE1p9Ct6ZaP8i8C6AwPzIOxs7gDSgmp6mwc1giV+/4rqSxJ5ErnpyJIumLcXr9hHwBTCZTVjtVm5+/eqY3QUbhoHJVPMXt/7H92HCshci7p+elRaT2ISIPg32wZgcx2I41kDeKZF3M7cDUzb4f4/woh38S5M6sctQTBLp0KMdby59jjPGDKPXoH054ZIhjJv7OP2GHhT1a8373x9csf+NnGS5gHNaX8HEp74sf3C6N7X5EBAi2WijECP/RvT2PujtfTDyzkOhwX56hL3tkHoj2A4h4r2vMocSfxJTtfnHHG0DBgzQCxYsiPt1Rcjin5dxzymPVSoPYE+xc+6tp3L5gyMSGJkQ0ae1RuedCYG1QIVJCCodWk0Bz0QofSPU8k6lQdotmFIvRAc2o/NOAV2xa5kFLN1QrSYn5FmSUmqh1npATftF5fZLKTVMKbVKKbVWKXVnNM4pYued+yZGrPky6bnJcZt9I0Tc+P+A4AYqJXUIJXLPZ5jSrkflLETlzEHlzMGUGuoYpiwdUVlvlrXEs7N7ZarKejfpJwg0eIxdKWUGXgFOADYD85VSX2utlzf03CI2Nq/6J+L2gD/IdYf8h+JdJRx45P5c8fAIOu3XIc7RCRFlwQ3VvOAtr/KolBlURtgeynYoZP8ExnZQTpQpM4aBRk807tgPA9Zqrf/SWvuAj4EzonBeUQ2/z88f05awYOpifJ66T4PsckDHiNsDvgAblm9m17YCZn0+l+sPu4sta7c2NFwhEsuyH5FX2DnB2ifC9sqUUihz20aT1CE6ib0DULG9yOaybSIGFs9Yxvltr+KBs5/m4fOf5dw2o/jtm7o9r7j84RHYnXuvL60Njdfl5YOHJzUkXCESTlkPANvBVF6cZAJTCsp5NgBau9GB9Wgj8lqS3bT2h4qJJbloJPZIg01hH49KqauVUguUUgtyc3OjcNnmp6SglHtPf4KSglJcRW5cRW7cxR4eHfE8O7fk1fo8vQfvz0Nf30m3Pp0xmU1kZGdgtYeX2jWCBstmr4zmWxAiIVTWeEi9FFRWaIGSfRiq1eeg0jCKX0BvH4TOOwu9YxBG0aPoKqV9tbELI/969Pa+ZbNqRqAD6xL0bmoWjcS+Gai45rwjEDaIq7Uer7UeoLUe0Lq11BKpj1lfzIv4ldIwNNM/+rVO5+p/3EGMX/QsU/wTeXvFC9Xul9NF/q5E46eUHVP67ZjazMXUZhGmrBdQ5nZo1/+B623K2+PhBdcn6JJx5cdqbaDzRpa1xwsABvj/QOddgDYKE/SO9i4aiX0+0FMp1U0pZQNGAF9H4byiitKC0kpdjnbze/0UF1SzQq4WMlqlM/isw7A5Kt+121PsXHT32fU+rxBJr/SNKtMZAdzgenfPug7fXDC2UrkgmAbtQ7u/iFOgddPgxK61DgBjgCnACuATrfWyhp5XhOt/Qp+IC4QcqXYGnHhwg85924TRDDn/CKx2K3anjYzsdG5+/eqYLI4SImkY+ZG361Kg7CYquKGarkueOvZOjZ+olBTQWn8HJH+R4kauW+/OHHfJEKZ98Et50wtHqp0BJx3MQUf1atC57U47d7wzhhvHjaI4v5RW7bMwm801HyhEY2bdH/x/hm83d0GpsvRo2R+UivDk0AmWmmfVJILUimlkbv7v1Rx+6gCmvDONgD/I8SOP5qhzBkZtwYQzzYkzzRmVcwmR7FT63ehdlwNe9mRuBypj7J6drH3B0gv8y8r2AzCDKR3lPC2e4daalBSIIa01pYUuHKl2LFb5DBUiGWn/MnTJy+BfAZbuqLQxKFv/yvtoN7r4eXB/CfjBPhSV/h+UuU1cY61tSQHJNjHy65fzGHfjWxRsL8BkMTN81FCueeZSrLbwaYVCiMRR1gNRWa/VsJcFTBmgbGXj7QblY/BltOEKlS9QTrAenNDG15LYY2DprBU8PvLFPfVY/EG+nzANT6mX2yZcl9jghBB1orUPnTcCAsspb6/n+R/aOxtaT0GZWmC4voKi+0KVH9GgUiHrDZS1Yc++6ktqsMbAB49OCi+y5fYx/aNZlBTsfWVbrP2zbhvvP/IZb975Pstmr6pVqV4hmiutveidZ0NgKZV7phqg3WjXJ2j/GigaS2gufEloRo2xA73rCrROTFE9uWOPgS1rtkXcbrFayPtnF2ktUuMcUcj3b0/j5evfJBg0CAaCfDXue4654Aj+/eboqD183bUtn+L8Ujr2bIfZIrNqROOggzshsArM7VCWffZsL30fguurOcoTWqhk5BJWORIAL/h+A/uQWIS8V5LYY2C/Q7uz/e8dGFVa3AWDBm265tT6PD6vH7PZFJUEWZRXzMvXv4nPs+cX0OPyMuOT2Rx70VH0P65h89ULdxb9zVylAAAdWUlEQVTxyAXPs2z2KsxWM1abhZtevYqjzz+ioaELETNaG+iih8H9adn4eQBtPQjSbofiByCwggjzHMtYwNIDgv9Qdbw9dHIgQStTZSgmBkbedx42Z+Vu6I4UOxf85wwcKTV1SYe//tzAmEF3cVrqxZyaNpLHR75IaTWNrmtrwdTFmK3hHxCeUi8zJtatHEEk9572BEtmrcDv9eMp8VC8q4Snr3yFVQuSt56GaLq01qFhlBqGGrXrY3B/DvhCwyhld+HkX1A2pr634y2olItQ9qFApJ7EAbAdVu/30BCS2GOgS6+OvDDrYQ45sS8pGSl06NGW61+6kpFjz63x2F3b8rllyFhWzVuLYWgCvgC/fDaHO4c92qDx8Oru+pVJYYmQ8Oti48otrP9zA0F/5bsWn9vPpOcnN+jcQtSF1hqj9AP0jkGhgl25R2C4JlZ/gOtdoGpJgQB7T+gACrLeQpnbgePE0EInVXH9hxNSR8V9OuRuMhQTI937duWJ7++t83GTX/8BvzdQaZvfF+DvpRtZ8/tf7HtI93rFc+iwgzGC4b+sNoeN4y85ul7n3G3X1nwsNgted+UHxlprtv+9o0HnFqIutGsiFD9FebI28qDoMQxsmFLOinBAUT2vZAXvT2AfgFJWaPkeuL9GeyaDSkOlXIiyD67v22gwuWNPMuuXbMQfoT2dMpnYvLr+TS9S0p2MnXgLdqcNR6odm8OKzWHlvFtP44BB+zYkZLof3DVizDaHlf4nJOeSa9FElb5M+B24G0pejLy/7SjqlwZ9oXH5MkrZUCnnYmr5DqascQlN6iB37Emn18CezPvfH/iq3P0agSDdDurcoHMPPOUQPtz0GrO/nI/X7eOw4f1ot0/DvyqmZ6Vx3u2nM+m5yeU1bCxWM6ktUjlzzPAGn1+I2tBag1FNrwdje8TNKv1mtHdGWcneOjbQCKsKmTwksSeZYaOG8snTXxHw+stn1dgcVg4acgDdejcssQNktExn2JVDG3yeqi574AK69e7CpOe/oXBnMQNP6c+Fd55FZnZ4H0khYkEphTa1ByNCT19z+L8drY1QGQHboeBfDcZGah5bL78a2AY1KN5YkloxUVSUV8yvX87D7w0w8JT+tKlnk4ptf+/g9VvfY8GURdicNoaPGsqlD16ALUKXIyHEHob7Wyi8C/BU2OpAtXgOlA1d/GyoDK+5M5AKwRVld+t1YQXlQLX6tNKc93ioba0YSexRMuuLuTwx8iWUSYXutLVm5H3nceGdER7YAD6Pj40rt9AiJ5Ps9i3jHK0QTZf2TA0V7ApuBktnVNqtgEIX3ETlhF8bTrAPDTXa0ITmutsGhKY5mrOjH3wNpAhYHJUUlPLEyJfCZoV88PBnHDrsYHoc3K3S9snjf2D8be+hlMLvD3DQkb24d+ItpGelxTNsIZok5TgR5Tix0jYj9yTqntQB/GDpiLKNANthUVuhHWsyKyYK5kxeiMkc/r/S7/Xz0/szK237/aclvPbvd3GXeHAVu/F7/Pw5czkPnfdsvMIVovkJbqjngRpKx6Pzr0Rv74ex/XCM/BvQgb+iGl60SWKPgoA/iI7w0EXr0GsVffrMV3hd3krbAr4Ay2evYsemnTGNU4hmy1SX510V78qDhMZg/IALdB54p6LzzkEH6vthEXuS2KNg4Mn9MAJG2HZ7ii2sVsrOzbsinsNis1CwIzk7ngvR6KWNAap2BqtuMoK97E91NOhSdP5oQi2fq9lLG2jvLHTJ62j3t2hdx+mUDSCJPQqy2rTg2ucuw+awYbaYUUphT7Fz/CVHc+AR+1Xat/8JfSIu4TeCBp17dax3DPk7Cpn1xVz+nLkcwwj/kBGiOVPO8yH9NlAtAAuoTHCeTeQaLx5CZQVqEFyLLn484kvaKEXnnYsuGIMueQFddC869xh0YFMD3kXtyayYKNq0agvTP/oVn8fH4LMG0mtgz7B98rbmc03fWyktdJUP0zhS7Ix6/CLOvOHkel33/x76lI+f+AKLzYLWmrTMVJ768T467tu+Qe9HiKZGayNUL12lgm8muuDfZcW/qrKzp7/p3thRbeajlKPSVqP4KSh9j8qLnkxgPRhTq4/rHb9Md0xieVvzmfjUlyyYsphW7bM4/7bTOXRYv3qda8HUxTx4ztPlKz4htFCjbbcc3l3zcqN5ii9EvGntRu84PHweu3JC6o1QOg60QXiJgoqcqNbfh4qBVWDsGFzNKlgLKmcuypRer5hlumMSa9Uui+uevyIq5/rqlf9VSuoQWlpdsKOQtX+sp2f/+CygcJe4+e6NH5n73R+0ap/FmWOGs9+hPep9Pq01i2csY/lvq2nZLosh5w4iJb3qGKkQ9aeUE535FBTcRKV66qZ2kHI5KuWCUAu84DZwvQ86P8JJLGCKNJ89sZ3JJLHXwi+T5jDxqS/Zta2AfkMP4pL7z6NtHRpmxFJJfuQ67cqkcBXFrpbFplVb+HvZZjru246cztlcN+AO8rbswuv2oUyKXybN4cZXr+LES4+p87l9Xj93D3+UVQvW4XN5sTltvH7ruzw97f6wNQFCNIhvPqE0WCGxB7eC6x1U2ihIOQ8FaMdxob6nlebCOyHtplB1x6ocp4LrAyp3VlJgPaDed+t1IUMxNfj4yS94/+FJ5VMUTWYTKRlOXl/0DDmd4r/yrKovXv6OCXd9ENZj1ZFq59PtE2rV2KMufF4/D5/3LL//tASL1UwwEKRFTib52woqdWcCcKY5+GzHBGwOW52u8dlz3/DO2I/DFny179GWd1a9JMNLIioM/2rIO4OI3Y9MbTDl/FJpk/YvRRc/Df5lYMpBpV2Pcp6y5/XgdvDNA1MG2tIH8i8JrX7VLiAFlB3V6mOUpf43JzIUEwXuUg/vP/RZpQRjBA3cxR4mPvklN4z7VwKjCxk+6ji+f2sa/6zdhqfUi8mksNqtjBk3KupJHeC9+yfy+09L8Ll9+Mq+EGzfkBvxm6cyKdb+sZ4DDt8v/MW9mPLO9LCkDpD3zy62/rWd9t3b1id0Icpp/wrIG0HEpA4RW9opa29Uy3cj7178IpS+SWjGDYAdsiagjK1o/zKUuRM4hqFMkWbhRJ8k9r3YtHJLqJ1clRGNYCDI4hnLajze4/JisZqxWEP/mzes2MzGFVvotF97uh7YKSoxOlLsvPzbY0z7cBazv5pPVtsWnDb6xJgNWXz35k9hJYWrG04MBgzS6lMmobovkZoGdZESYjddsRlHJLaDa38u769Q+hahWTTest/fUii4BlrPxOQ4vmHB1kODErtS6jzgAaAXcJjWOmbjK9s35PLhY5+zZOZycrq0ZsQdZ3Lwsb1jdTkAWrbLwu+LPJ91b02pVy1Yx/NXv8b6JRsxmU0cccahlOSXsPTXVeXDFwccvi8PfXVnVO6qbQ4bw64cGpNyvFVFupOOxGQ20b57Gzrv36HO1zjx8mN4976JYddq1T5L7tZFdPj/rP415USl31XrU+nSN4n4IaFLwb8IbP3rHl8DNXSB0lLgbGBmTTs2xNb127mm321MeXsam1b9w8Kpi7n3tCf48f2fY3lZstu3pN/Q3ljtlT//7Cl2RtxxZsRjdmzM5fahD7Bu0d8YQSPUs3TSHP74aSk+tw9XkRuvy8fSWat4/bb3Yhp/LPQb2htlCh/jzu7YEpvDSmpmCo5UOx16tuWRb+6s1zXOKJtR40hzoJTCkWonNTOFsZ/cKuPrIjpMrap7AVp+hrL2qvEU2ijCyLsQfLOr2UMlrBlHVB6eKqVmALfV9o69rg9Pn7zsZaZ9OAsjWHlFZXpWKp9un1Bto+ZocBW7eeryccz77g/MFhNWu5XrX7qS4y46KuL+b9zxPp+/+C2Bau70K7I7bUwu/SDaIcfUlrVbGTPwrtAYu8eP1W7BYrPy3M8PktM5m1Xz1pLZOoOe/fdpUBLWWrNo+lKW/bqKVu2zOPr8I2S6o4gawzUJih6i8p22A1LOw5QxtnbnyB8N3plUnvlSkRPVZg5KRe/3tkk9PF08fVlYUodQk+ftG3Jj+vU8Jd3JA5Nupzi/hKK8Ytp2zdnrB8mG5ZtqldQhNMNEa92o7kI79GjHWyte5NvXp7Jy3lr26dOF0687iewOoTug+i60qkopRb+hB9Fv6EFROZ8QFSnn2WhjO5S8DsoEOgDOU1Hpd9TqeG2U1JDUHZDxUFSTel3UmNiVUj8CkTLnPVrrr2p7IaXU1cDVAJ07163FW8t2LcjdnBe2PRgwyGgV+zmhEOrrWZt66b0G9eSPn5aETf2LpPfg/RtVUt8tKyeTkWPPS3QYQtSbUgqVdh069QoIbglNXzTVoY2jLqX6kWwHqtUklDW8pEi81DjGrrU+XmvdO8KfWif1svOM11oP0FoPaN26bi3jLrjjLByplR8yWu1WDj/tENJapNbpXLF26jUn4ki1Y6owDm21WzFbzNgc1vKfUzKc3PBK4qdLCtGcKeVEWXrULakDmHLAFKnzmRmcJyc0qUMjGYo56uyBbP1rG+898Clmswm/L8Chww7mtrevj8v1d23L5+tXvmfl/HXs06cLZ94wvNrFSZnZGYyb9wRv/Of/WDBlMY5UO6decwLDyuabr16wjh79unHa6JNo1S6rQXH5vH4+e/Zrvn9rOkbQ4NgLB3PhXWfLWLQQMaIDGyGwAsydUJmPovOvJ1ToKwjYQKWi0m5KcJQNfHiqlDoLeBloDRQAi7TWJ9V0XH1XnnpcXras2UrLti3IatOizsfXx6ZVW7jx8Hvwun34vX4sNgtWu4XnZjxEj36JW96uteb24x5kxdw15fPKrXYrHfdtx38XPhXTB8rJwjAMfv9xCZtX/0PXAzvR95gDG+XQlkg8bRSB9oCpdcTfIa0D6ILbwPsTKCvoIFj3hfR7wP0pBP4Otc5LvQQV8U4+OuLy8FRr/QXwRUPOUReOFDvd+3aN1+UA+O/N71Ba6CpfGBPwBQj4Arwwejzj5kSuxRwPy35dyar5aystFvJ7/Wxbv4PfvlnAkWcNTFhs8VC4s4h/H30fuZvyCPiDWKxmOvRsxzPT7ic1M7mG50Ty0sE8dOHt4JsLKDC3hczHUbZDK+9X+gZ4pxFagFRWdM+/HEonYMp6ee/X0DpUItg1CQignKeD/USUil07DGm0UYPFPy+LuNpx9fx1BAPVLEeOg1Xz14W13QNwl3hY/tvqBEQUXy9d/yb/rN2Gu8SD3+vHXeLh72WbGP+f9xMdmmgktNbo/MvBN4fQ7BYfBDei869CBzZX3tn1IeHNsP3g/REjGLkrWvl1ih9B598I3u/B+yO64A50wY0xXUUtib0G9mpWhlpslogNrOOldadWYQunINSOL1kqT8aKYRjM/nJe2AdbwBdg+kezEhSVaHT8f0JwE2HdkrQf7f6oyrbqFhoFIfc4tG9xxFd1YC24PqHyfHk3+GaBf349A6+ZJPYanHzV8diclasTWh1WTrh0SELHcwedNgBHqiNsFajFamHoRUcmKKr4MYzIdzvBCOsdhIgouIXKjat380NgfeVN9qOoPl2WoguuResI3+C9s4lY/Ei70J7YrZyXxF6Dyx48nwEn9sXmCE1RtDttHHTk/lz73OUJjctmt/LCLw/Ts/8+WO0WrHYrXQ7oyLMzHky6KaDRZjKZ6De0d6UppRCqTzPwlPjX5RCNlLV3aGFSGAdUGWNX6beH+qRG/CAg9ODVvzR8uymNyI8ybVDXKZZ1IPXYa2nL2q1sWL6Zjvu2r1dhq1jK31GIETTCpk9qrZn1xTwmPT+ZorxiDj9tAOfffjqZ2bH7hYqXreu3c+Ogu/G4vHhKveX1ZMbNe4Ls9rGblSCaFqPgVvD8wJ7xcwuYWqKyv0eZKi9I1EY+eue5YERoSK3SUFlvoGyHVDmmBJ17ZHj7PRyo1lPCWurVRHqeCt59YCKfPftNees8q81CZk4G4xc/W6tVtMnOVexm+kezWL90Ez0O7soxIwbHpAa9aLq0DqJd/xfqdqRdYD8OlXYjyhx5nYp2fY4uGktYKQGVgcr5LWI3Je2bh84fzZ4hGQMynsLkPLHO8Upib+aK8ooZ0fEa/N7Kv4A2h5WL7z2Hi+4+J0GRCdF4GZ5foeBKwsbNrX0xtfq02uO09pW14QuE5rvXs4ZMkyoC1ljsrki4ccUWuhzQMaELZtb+sR6bwxqW2H0eP/OnLOaCO87kl8/mMO2jWdhT7AwfdRz9j5OCW0LslWsCER+G+legg7koc+RyKUrZwD44trFVIIk9SorzS7jt2AfY+td2goEgZouZdvu0SdjDzKy2LSLOc1cmReuOrRh7+pMsmbm8fJhmztcLOPPG4Yx67OJ4hypE4xHcGnm7soGRC9Uk9niTWTFR8urNb7Nx5RbcJR58ntCCmY0rt/DqzW8nJJ5uvTvTab/2YaUFbA4r+x/Wo1JSh1C5hs9f+DbUv1QIEZntMCLeD+sgNKBJdbRJYo+Snz/5LawOe8AX4OdPf0tQRPDYd3ez/8Ae2BxWnOkO0lqkcuub17Fp5ZZKSX03k9nE7z/upWWYEM2cSrsGVApQ4YZJOSH9xoTVXo9EhmKiJFIjEAAjgWUHstq04IVfHmHHpp2U5JfSuVcHLFYLfy3ZgNlqJlhlqEaZTKRmxqeLuhCNkTK3h1ZfoktfCS0+MuegUq9GOU5IdGiVyB17lAw4qW9YiQGT2cSAk2rf7TxWcjpls0+fLlisoc/xky47JmL1R6WQBT5C1EBZOmLKfBxTzs+YWn2adEkdJLFHzQ3j/kVGq/TyhiCOVDsZrdK5YdyoBEcWruO+7fn3G9diT7GTkuEkJcNJess0Hv/fPdidMg9ciMZO5rFHkbvEzU8fzOKvJRvo3qcLQy86Emda8oy7VeUucfPnz8uxOmz0GdKr/I5eCJGcZIGSEEI0MbVN7DIUUwvzpyzixiPu4fz2V3Hv6U+wdtH6mg+KEa01u7bl4y6tWhtaCCFC5Lt3DX768Beev/o1vK5Qp6J53y5k0bSlPPfzg+x7SPe4xjJn8kJeHD2ewrxi0Jojzx7ELeOvwZnqiGscQojkJnfse2EYBq/f+m55UgfQGrwuLxPu+iCusaxeuI5HRjzHzi278Hv8+L0BZn0+l0dHPB/XOIQQyU8S+14U7yqhpKA04murFqyLaywTn/oKn7ty3Re/188fPy1hx6adcY1FCJHcZCimCo/Lyw/v/cy8736nZbsW1dbVj3fN7y1rtkbskWi1W8ndlEdOp8hlRoUQzY8k9gpcxW7GDLyLHRt34nV5MZlNKAUWm5mAb88qTXuKnYvvPTeusR10ZC82LNsUVtjL7/XTuVdyNf4QQiSWDMVU8PUr37P97x14XaE6KkbQIBgwAIXVYcVRtqDnyscu5NgR8SvBCXDe7adjT7VXagdnT7Fz5o0nN4mmGUKI6JE79gpmTpqDz+MP226zW3n4mztp06U1Ldu1wGoL75ISazmdsnll3hO8dc9HLJ6+jIzsNM679XSGXTk07rEIIZKbJPYKqiuAFQwGaZGTSZsuia213KFHO8ZO/HdCYxBCJD8ZiqngzDHDy2u97KZMirZdc5KugbUQQlRHEnsFR5xxKGfeMByr3UpKhhNnuoM2XVrz8Dd3Jjo0IYSotQbVilFKPQ2cBviAdcAVWuuCmo5L9loxu7bls2z2arJyMjhw8P4J61sqhBAVxatWzA9Ab611H2A1cFcDz5cUWrbN4qizB9L7yF6S1IUQjU6DHp5qradW+HEOEN/J3U2M1ppfJs3h2zd+JOALcMKlx3D8yKOknK4Qok6imTGuBCZG8XxNXv72At5/+DN++2YBqZkppGaksG7x3+X9SFcvWMf0j2bx+Pf3YDLJ4xAhRO3UmNiVUj8CbSO8dI/W+quyfe4BAkC1lbGUUlcDVwN07ty5XsE2JcX5JVzb/z8U7iwi6A+SuykvbB9PqZflc1azcOpiDh3WLwFRCiEaoxoTu9b6+L29rpS6DDgVOE7v5Ums1no8MB5CD0/rGGeT8+34HynJLw1rKF2Vp8TDwh/+lMQuhKi1Bg3FKKWGAXcAR2utXdEJqXlYNH0pPo+vxv2sdgstcjLiEJEQoqlo6MDtOCAd+EEptUgp9VoUYmoW2u/TBpO55v/9JrOZ40cOiUNEQoimoqGzYnpEK5Dm5owbhjP1vZ/LC44BmM0mUAqb04pSCpPJxN0f3kR2h1YJjFQI0djIPLoE6dKrI/dPuo1nR/2XkoISjKDBQUcdwB3/N4btf+8k6A+w/8CeMtVRCFFnDVp5Wl/JvvI0nrTWbN+QizPNQWa2jKULIapX25WncjuYYEqFiowJIUS0SGJPUl63l+/fns6sz+eSmZ3OaaNPou/RByY6LCFEIyCJPQl53V5uPOIetqzZitcVmhI5Z/LvXP7QBZz779MSHJ0QItnJOvUk9MN7M9myZlt5Ugfwury8fe9HlBSUJjAyIURjIIk9Cc36Ym6laZC7WWwWlv26MgERCSEaE0nsSahFTmbEcsFaa9JbSuNqIcTeSWJPQqdfdxI2Z+WG2UopMlqms//AngmKSgjRWEhiT0IHDNqXq5+6BLvTFmrRl+Ygp0s2T0y5V8r3CiFqJAuUklhpkYuVc9eQmpnCfof2kG5OQjRzskCpCUjNSOGQE/omOgwhRCMj3+uFEKKJkcQuhBBNjCR2IUSzprVGG0Vo7U90KFEjiV0I0Wxp78/o3KHoHYPQ2/tjFD6A1jV3Nkt28vBUCNEsad9idP4NgKdsSwDcn6N1MarFs4kMrcHkjl0I0Szp0teBqqU7POCZgjZ2JSKkqJHELoRongLrgQjreJQNglvjHk40SWIXQjRPtj5ETIHaD+YucQ8nmiSxCyGaJZU6GpQDqLii2wkpl6FMjbvYniR2IUSzpCxdUS0ngu0oUGlg7gTpd6DSb010aA0ms2KEEM2Wsu6HavlmosOIOrljF0KIJkYSuxBCNDGS2IUQoomRxC6EEE2MJHYhhGhiJLELIUQTk5DWeEqpXGBD3C8M2cDOBFw31uR9NS7yvhqXZHpfXbTWrWvaKSGJPVGUUgtq0y+wsZH31bjI+2pcGuP7kqEYIYRoYiSxCyFEE9PcEvv4RAcQI/K+Ghd5X41Lo3tfzWqMXQghmoPmdscuhBBNXrNL7Eqph5VSfyqlFimlpiql2ic6pmhQSj2tlFpZ9t6+UEq1SHRM0aCUOk8ptUwpZSilGtXMhEiUUsOUUquUUmuVUncmOp5oUEq9pZTaoZRamuhYokkp1UkpNV0ptaLsd/CmRMdUW80usQNPa637aK0PBiYD9yU6oCj5Aeitte4DrAbuSnA80bIUOBuYmehAGkopZQZeAYYDBwAXKqUOSGxUUfEOMCzRQcRAALhVa90LGARc31j+vppdYtdaF1X4MZWITQ8bH631VK11oOzHOUDHRMYTLVrrFVrrVYmOI0oOA9Zqrf/SWvuAj4EzEhxTg2mtZwKNu/tzBFrrrVrr38v+uxhYAXRIbFS10ywbbSilHgUuBQqBYxMcTixcCUxMdBAiTAdgU4WfNwMDExSLqAOlVFegHzA3sZHUTpNM7EqpH4G2EV66R2v9ldb6HuAepdRdwBjg/rgGWE81va+yfe4h9BXyg3jG1hC1eV9NhIqwrUl8Y2zKlFJpwCTg5irf+JNWk0zsWuvja7nrh8C3NJLEXtP7UkpdBpwKHKcb0TzWOvx9NXabgU4Vfu4I/JOgWEQtKKWshJL6B1rrzxMdT201uzF2pVTPCj+eDqxMVCzRpJQaBtwBnK61diU6HhHRfKCnUqqbUsoGjAC+TnBMohpKKQVMAFZorZ9LdDx10ewWKCmlJgH7AQahCpPXaq23JDaqhlNKrQXsQF7Zpjla62sTGFJUKKXOAl4GWgMFwCKt9UmJjar+lFInAy8AZuAtrfWjCQ6pwZRSHwHHEKqCuB24X2s9IaFBRYFS6kjgF2AJoXwBcLfW+rvERVU7zS6xCyFEU9fshmKEEKKpk8QuhBBNjCR2IYRoYiSxCyFEEyOJXQghmhhJ7EII0cRIYhdCiCZGErsQQjQx/w/5JUIaVusDaAAAAABJRU5ErkJggg==\n", "text/plain": [ "

" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.scatter(data[:,0],data[:,1],c=target)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "#训练\n", "svm = HardMarginSVM()\n", "svm.fit(data, target)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xd4FVX6wPHvzO33pvdOEtIIoYQuiBQRC4qiAhYsWLCviuuuurZd3bX8XBRdUbGjKNhBUUAR6TX0UIRQUkmvt+SWmd8f2Q2bDUq7cMnN+TyPzyOTmTPvwM17Z86c8x5JVVUEQRAE/yH7OgBBEATBu0RiFwRB8DMisQuCIPgZkdgFQRD8jEjsgiAIfkYkdkEQBD8jErsgCIKfEYldEATBz4jELgiC4Ge0vjhpcGCIGhUZ44tTC8JpVSXV0yXY5OswBD+1dfOBKlVVI4+1n08Se1RkDNP/9q4vTi0Ip9W7xm/pH7ef23te7utQBD8UETjp0PHsJ7piBMGLbnVchsfl8XUYQicnErsgCIKfEYldEATBz4jELgiC4GdEYheE02Bp+UpfhyB0YiKxC4KXrVw2mj3FFb4OQ+jERGIXBC+7LzXH1yEInZxI7IIgCH5GJHZBEAQ/IxK7IAiCnxGJXRBOk7e3zfN1CEInJRK7IJwG0s/X+ToEoRMTiV0QBMHPiMQuCILgZ0RiFwRB8DMisQvCadA7PQaPy8OHu77zdShCJyQSuyCcJvt/GOvrEIROSiR2QRAEP3PKiV2SJKMkSeslSdoqSVK+JEl/9UZggiAIwsnxxpqnzcBIVVWbJEnSASslSfpBVdW1XmhbEARBOEGnfMeutmj69x91//5PPdV2BaGj62oIwG5zitrswhnnlT52SZI0kiRtASqAH1VVXeeNdgWhI+udHsP+H8ZSWFPn61CETsYriV1VVY+qqr2BBGCAJEntClJLkjRFkqSNkiRtrG8QH3RBEITTxaujYlRVrQN+AS46ys9mqqraT1XVfsFBId48rV9yOOys3bSSJSt/oLDkgK/DEQShAznll6eSJEUCLlVV6yRJMgGjgBdOObJO7GBRAa+++SLGZgs6xcB8+Sty+/blxolTkGUxQlUQhN/njVExscCHkiRpaHkC+ExVVTHd7iQpisLMD14jwZlGtDkeAI/qYevGNeR1W0f/3uf4OELhRNltTl+H0CkoioLT6cZg0CFJkq/D8alTTuyqqm4Dcr0QiwAUlx3C0eAgyhjXuk0jaYiVkli7fqVI7B3MVTlpbAa+OLCYq1NG+zocv6SqKl/MXcmXc5bQ0NBEZFQoN0wew6gLe/s6NJ8Rz/VnGVVVAandHYckSSiK4pughFOycplI6KfT53NWsGzRD7z6j1CWf5vO0w+ZmPX2HJYvy/d1aD4jEvtZJjEuGZ1FS5XjcOs2RfVQphQycMAQH0YmCGcfRVH4Ys5PPPVwNF2TTUiSRM/uATx0dzhffPKjr8PzGZHYzzKyLHPbTfdwQN7JLtsm9jbmk2dfSXqPDNENIwj/w+FwYbPaSU02tdmenWmmuLjSR1H5njdengpelp6SxTNPTGPz9g00WRtJT8kiLSWz078QEoT/ZTLpCQkNYuduK9lZltbteVsbSe0a9ztH+jeR2M9SgZYgzht0vq/DELykqqYJUnwdhf+RJInrb76YJ57/gofviaBbppkNmxuZ9kYdjzw13tfh+YxI7IJwmt2XmsPzO6xkJqxkRPS5vg7H71w8ph8mo54ZH/1EaUkxKV3jeOSpq+nTr6uvQ/MZkdgFQejwhp/fk+Hn9/R1GGcN8fJUEATBz4jELgiC4GdEV0wncLCogGWrllBXW0O3bj04d+BwzCbLsQ8UvKqwpg6ifR2F0BmIO3Y/t37zaqZN/wfFG8rw7New7Nufee7lJ7Hamo59sOA1jwQPxG5zsr9pl69DEToBkdj9mMvtYs4XH9JN15dkSzrRpniyLX1Rq2R+Wd15Z+X5yqbKdF+HIHQSIrH7sbLyEiSnTJCubf37KF0c27dv8VFUgiCcbiKx+zGLOQCn2oyiti0e5vDYCQwM8lFUgiCcbiKx+7Hw0AhSU9PYb9v176qRYPfYKOUgw4aO8nF0giCcLiKx+7lbb7gHc7KB9falbHOsZatrNZdcejk5mb18HVqnU1kexeI94uWpcPqJ4Y5+LigwmIfve5KyihIamxpIiO2C2WT2dVid0iPBA9nMKl+HIXQCIrF3ErFR8cRGxfs6DEEQzgDRFSMIguBnRGIXBEHwM6ec2CVJSpQkaakkSbskScqXJOl+bwQmCP7qzbyvfB2C4Oe8ccfuBh5SVbUbMAi4R5KkbC+0Kwh+J7doqq9DEDqBU07sqqqWqaq66d//3wjsAsRbOkEQBB/xah+7JEnJQC6wzpvtCoIgCMfPa8MdJUkKAL4EHlBVteEoP58CTAGIDBe1S73B6Wxm/qIvWLn6F5qdDrpn9eSqy687a4Y1NjY1MO/7z9i4eS2SpGFA/3O4/KLxomSwIJxmXrljlyRJR0tSn62q6lHfDKmqOlNV1X6qqvYLDgo52i7CCXrn49fJW7qR7tIABplG0bDbzkuvPkNDY72vQ8PldvHSv55l79p99JAHkU0/8lfk88qbz6MoyrEb8HNvb5vn6xAEP+aNUTES8C6wS1XVaaceknA8yipK2L0zn2xLX8xaC1pZRxdLGhZ7MCvXL/V1eGzftRl7lYOMgJ4YNWbMWgtZlt5Ul1Sze1++r8PzqdyiqXhcHl+HIfgxb3TFDAFuALZLkvSfWrCPqar6vRfaFn5DReVhAuQQZKntd3OgFEJxcaHXz9fQWM/ytUvYX7CXqOgYhg+5gJiouN/cv/RwMRZ32wqSkiQRoARTVlFMdkYPr8coCEKLU07sqqquBCQvxCKcgKjIGJqUOhTVgyxpWrc3qnXkJHh3tGl1bRUvvPIUxqYAQuVIft27j9VrlnPvnX8kI7Xbb8Zn1za2226TG4kKj/FqfIIgtCVmnnZQsVHxdMvOId+6CZu7Cbfi4qB1L1ZTPUMGDPfquRYs/oqAxlAyLb2IMsXR1dKNFLUbn37+QWs54P+V270fcrDE/qZduBUXLsXJ3qYdmCPMZGf29Gp8giC0JYqAdWC33XBP66gYh91BTree3Dr2SYICg716nh07t5FmbNt1EmmMZV/FDppsjQRa2i/aodPp+eN9T/DZ1x+xdsdPSJJEn9wBTLjiBjSypt3+ndH+pl2kBhz9iaeza2y089EHP7Hi5zwAhgzP5abJFxAYJCqTHg+R2DswnU7PVZdex1WXXndaz2MxW2iucWDRBrZuc6suJBkMOsNvHhcWEs6dkx9oHQUjy+IB8T9WLhuNRreE1J4isf8vRVF49KG3yEiqZcYL4QB8/PkmHpl6gFff+gMajfgcHYv4GxKOacR5ozno2oNLcQKgqB722fLp3+cc9PrfTuz/IcuySOr/Y6gnwtchnLXWrfkV2VPJYw/GkZRgJCnByKMPxKGTqli7Zo+vw+sQxB27cEznDhxBeWUZy5YvIUAOwuppJCs7m4njbvR1aIIf2l9QTv/eelpGUreQJIn+vbUU7D3MkHPFU86xiMQuHJMsy4wfO4nRIy6lrLyEsJBwoiLEyBbh9IiLD+PHje5223ftVRgxJqzNNrvdydbNB5Blid59UtHrRUoDkdiFExAcGEJwoJg1LJxeQ4Z244O3Tbz3cTnXXBmBJMHcr6s4VGpg6LDurfutXL6Tfz4/m4xUDYqi8uLf4c+P30j/gek+jP7sIBK7IPiIx+VhaflKRkSf6+tQzip6vZYXp9/N6698xfsTWhb/7jugGy9OH9d6R15ZWc+05z/itb9H0S2zpfbQ1h1NPPT0+8ya83inHz0jErsg+EDv9Bi+/GEspqsWgqiJ1050dAh/e+4WnE43zc0ubLZmwsICWn/+y887GDHE0JrUAXrlBDCoTz3Lf8lnzNj+vgj7rCESux8oLDnAoiXfUVpaTGJSMheOGEN8bNJv7l9VU0netnU4nc10z+xJSlJamxdVgnA2UBSFWe/9yPfzV2A0qLg8Wq694SKuHD8Em9VBaHD7z2xIkITV1uyDaM8uIrF3cHsKdvL6W9OIVZKI1CZSVlnOi1v/xoP3PkpyYtd2+6/fvJqPPnmHUE8Usqrhp8ULGXjOOVx75WSfJPeqmgrW5a2iydpIdmYPumf2EkMjBQBmvf8Te7etZPYbcURF6jlwyM4jf/uOwEAz/Qdm8PxTP3LTNR7M5pYJb41NbpaucvD8+AwfR+57IrF3cJ9/PZtkNZNoc0sN9mB9GFqbnq++ncPUu//SZl+b3crHc94hR9ufAFPL7FS3ksH6Navp02sAWWk5ZzT2rfl5vPvhDEI9UehUPetXriU5M4W7b3kQjUZ8NDszj0dh/lfL+PC1GKIi9QCkdDEx9a5wXp+1lFffup+4LmlMunM311wZAip88Z2VYaOGkJwS5ePofU/cGnVgHsVDUclBooxtqyxGG+PZf2Bvu/337NuJRQkmQHek5IBW1hGpxLJ528bTHu9/c7ldfDB7JllyLhmWHqQEZNLbNJjC3YfYsHXtGY3Fl+w2p69DOCvZ7U5cTiex0fo227ummCguquCma56jofoAkizzzEulfLs0kDsfuIU7773URxGfXURi78BkScZiDsDmbmqz3epuJOgoi5nIsoxK+6JdCsoZ7/44WFiAzq0jWH9kXLIsyURJCeRt6hwrK16Vk8amynS+OLDY16GcdSwWA+ERoWzZ3vazvWpdPTXVTdw7WctHM7rw5QdpLJiTRXVFGaFhAeJd0b+JxN6BSZLEqOEX82vzdpo9DgAcHhsFznwuGHlJu/0z07rj0FqpdVa1bnN6HFTJZfTPPeeMxQ2g1Wpxq+0noXhUN3q9/ihH+CfbzhRfh3BWkiSJybdfxpMvVLJwSQ2Hihx8Ma+Sf75RQ9dUCxcMP3JDkJRg5MpLLCxeeGafOs9moiOzg7vw/Muw2a0sW7kEncuAW2rmggvHMHzwBe32NRqMTJn8B956dzoWWxAatNRSxUWjLyW1y9EndTgcdjbnb6ShsZ7kxFQyUrt55a6oS0IqxiAjhxuKiDElAuBSnJRTxKUDLzvl9oWOb9jIHlgCbuPLuUt4+5NKUromcu2Ng9mf/0u7fcPDZIoKbGc+yLOUSOwdnEbWcPXY6xkzehx1DbWEBodjNBh/c//sjB489/R0tu3ajMvVTLf0HkSEHf1lU0lZIdNmPIfBbkLvMbBQ8y0pGV2565YH0Wl1pxS3LMvcdeuDvPrmC1TYS9Ghp06p4vyRF9I9Q9RrF1r0G5BGvwFprX+urWniljmLqKh0tr5UVRSVBT/ZGDvRuwvMdGQisfsJk9GMyXh8s+3MJguD+vz+bEdVVXn3ozeIcSQRZ+4CgKIqbN+9nhVrf2bkuReecsyJcV34x5PT2fnrdmx2KzFRsRwq2s/CpfPplp5Dl4RU0WcqtBEaFsA1ky5mytSFXHOFhYAADd8tbsIQmMzQYSKx/4dI7MJRVddWUlVRwQDTkSGQsiQTr0th7fqVXknsADqtjl7ZfdixZysv/+s5gtxhaBUtC+Xv6DdwINdffYtfj2sf6omgqqaJpWZRWuB4Tbx+GFndk/hp4QbsdgcXXN6DkaN6otWKBVz+QyT2TkZRFA5XlqLXGYgIi/ydPX/vTlli97581m1YSbPTSZ/c/uTm9D/plZFcLifvfvg6mVJvQiwtCyu4FTeb162md4++9OiWe1LtdgS902N4fscQMhPaD08Vfluv3in06i1ePP8Wkdg7kZ2/bufDT2bitDpxKy7iExO5/ab7CA9tv+hDeGgEkdFRlJYXEv9fXTGlrgPEmKN5883pRKsJyJKWOds+YkPOGu646f6Turvee2APOpeBEFN46zatrCWKeDZsWuPXiV0QTgevPONKkvSeJEkVkiTt8EZ7gvdVVpfz5juvkGDvSj/TMAaYR+Ipkpj+5gutS9f9N0mSuGXSXVSYCsm3bWBv43Y22VcSkRrBvr176WUYRJIljQRzMr3M57A3/1d27t1+UrHJ8tGfDlRUJMl/u2EE72hudpG/o5DCQ5W+DuWs4a079g+AfwGzvNSe4GWrNywnzBNNmKVlBIwsyXQxp7O5ZhX7Du4hI7X9qjTxsUk8+/jLbMnfSENjHSmJaVTXVlJ/qBG95sjIG1mSCfNEkb9rKzmZvU44tq7JmXgMLmqaKwkztHQPuRUXFVIJY/tefpJXLHQGi3/YzFuvf0lMJNTWuYmIiePxv95EVJR3F3TvaLyS2FVVXS5JUrI32hJOj7q6WoyY2myTJAkjZhqa6n/zOKPB2GYEjS3fiov20+Ddsguz2dJu+/HQaXXcfvN9vPHOywTYgtEoOmrlSs49dxjZnWTo457iCrpYdpEa0LmXffN4FFav2s3u/EIiIoMZNbpXm9rqxUVVrFy+E1WF2LhQ3ntzLjNeiCUtxYSiqMyaU8lfH3uPf739QKceUXXG+tglSZoCTAGIDBcFqM+0zIxsdm7YQYJ6ZAihS3FRr1STkph2jKOPyM7ogcfopNxeQrSppfBYo6ueavkwA/sMOen4stK689xT09m8YyOOZhtZaTnExySedHsdySPBA3m3soLRmb6OxLdstmYefWgmkrucoQP1FGz38MmsH/j7i3eQkRXPV5+v4pMPvmX0cCOSBI+/XMFdk0OJj9bgdLrR67XceE0k3y4+yL5fy0jPjDv2Sf3UGUvsqqrOBGYCpKdmtS9YIpxWfXsM4Of4hewsziNGl4hLcVGiHOC880Ye9eXpb9Hp9Nx3x8PMeGcapdaDaNDg0Ni4adKUU14H1WyyMKT/sFNqQ+i4vpi7ktjQCp55LLH1vcvCJTVMe3EOTz5zM7Pfn8+sGYnERLVMTFq2shK9bKeo8DCKoqLRaklMjCQmWkddndWHV+J7YlRMB+V2u8n/dVtL33dSGgm/s7AGtCTkqfc8zi+rFrNpy0aMBiPXDrmRfr0GnfC5uySk8o8np7P/0F5cLhddkzMw6A0neymCAMDqFZv4812hbV6mjx4RyvSZ+/l+wSbOP8/UmtRXr69nb4GNlesVxl8egNEgU9/gIW9TCfm7PPylW7yvLuOsIBJ7B1RRdZiXZzyHp1HFqJqoUz+hV59cbr72zt8dS240GLlo5FguGjn2lGPQyBrSU7JOuR1B+A+NLONytX2YV1XweECjkVH/60dfza8gJFimvkHh//5VyyUXBFBd7eblt+oIDu3a6dc89dZwx0+BNUCmJEnFkiTd6o12haN79+MZBDVE0Ms0iExzL/qZh7ErbxerNyzzdWjCKVi8Z5evQ/Cp4aMG8uHcGtzuIxn8y/lVJCYncNElfViy3E7p4ZZl78ornYSGaHj35Wgy0ox8+lUjq/OcjBsTCEd5ud/ZeGtUzLXeaEc4turaKkqLihloPr91m0bSkKBNYfWa5QwdONLr53Q0O9iweTX7D+4jKiqGwf2GEhwU6vXzdGa3Oi5jM9N8HYZPXXHVIHZs3cfE2/ZwTj8jB4rcFB828Py0a4mNDWXyHVdy0z1fM3KokcYmKC51I2s03DQxlJsmhuJoVpj9ZT3xiWIFJdEV08F4PO6jTtrRSBrc7vb1zU9VY1MDL0x/Gk8NBBNKAQUs/uk7HrznUZLixZRuwXt0Oi1P/+Mmdu0sZtfOIroNCGLQ4Cz0+pY0ddkVA+k/KIOVy3cxOq6Zd974hr88V8UTUyMwGGDTtmZmfd5M7sAgvvx8NSNG9iAsPNDHV+UbIrF3MJHh0YSEhVBRW9o63FBVVUpchxjZ7/xjHH3iFvz4NXK1nm4BR8aTl9oO8fHc93hs6jNeP5/QuUmSRHb3RLK7H32oa0xMKFdPGAzAxZf04eEHZ3LBhAJCQ7TU1CoEBelIi9lL8e493Pb+dzzy5M0MGNT5FrcWib2DkSSJyZPu5NU3X6TGWoFBMdGgqSYyOYphR1lc41Rt3rqRVEP3NttiTUmsLllMXX0NB4oKsNltpKdknvJwR0E4ERFRwbw/+2EaG2wsWriZnxd8xzuvdMFgaHmiHZvfxJ+e+ZjZXzzZetffWXSuq/UTKUlp/O2xl9iweTW19bWkpWSQk9ULjcb7/5wGvQG3w9Vmm0d14/a4eeb/HkPXbECnGpjDh5x33kiuvuz6Tj3j71R9uOs7buomFmQ+EYFBZnbn72fC5UGtSR2gZ/cAEmJq2bHtEH36dfVhhGeeSOwdVGBAECOHXnTc+3s8bnbvy6fR2khacsZvrpr0v84dPIKf5i8kUDcAjaRBVVUO2PfgcjuJa075r2XtXKxetoLM9Gx6dutzUtfU2Uk/X4d95Ce+DqNDkmUJRWk/79HjUZF+o8icPxOJvRMoqyjh1TdfxN3owYCJOqWK884byfixk455d33+0As5VLSfDVt+IVgOw0Yj5nAzQdXBRBsTWvfTyTpiSGLN2hUisZ+k3ukxbPZ1EB1EfZ0VSZIICm4Zr37eyD58PHMnFwwPxWxumcuxYVMD5dUaevTs4stQfUIkdj+nqiozP3iNkMYoEkwto1hciou1y1eRnppFbo/+v3u8RqPl9hvuo+zCEopLCwkLjUBVFN6a8Wq7LwWNpMHlFmOIBe9TFAVJkig8VMn0lz6jYG8hAJndUnjg4QkMObcbeev7MfH29YwYYqS6VmX9FjdPPHNrp1xZSSR2P1dWUUJNRTX9TT1at+lkHbFSMqvW/nLMxP4fsVHxxEa1jMJxuV14DB7qnNUE68KAltrp5UoxV/W9xuvXIHRe+/aWMuPVeeRvK0DWyDQ1Oph6VyRv/KMriqLy5fxq/vTAG7w3+xHuf+hK9o09h7wN+4jLMHLPn7t32hmoIrH7ObfbhVbStru71kpampsdJ9WmTqtjwrjreX3mP3E2u5BUCYPRQK/efej/79ozFVWH+WbBZ+zYuRWT0cR5557PhSMuQ6sVHznh2FwuN9Nf+orPP/2R5AQNFoPEmNEhVFfDz8urmTQhCq1W5tqrI1mzsYhVK3Yx4vwepKXHkpYe6+vwfU4sT+Pn4mMSkU0Stc1VrdtUVeWwp4i+fQaeVJsut4vvFn5Nqqkb5waPZmDQSOL1yTQ2NIAkUd9Yx4vT/0rFtmpytUNIcXZn2Q9LmfXZ2966LL/29rZ5vg7B59558wcqCzfy9QexLPosiY9nxLA+r57sTD2obtbnNbbum5aipfxwrQ+jPfuIxO7nNBotN19/B7+qW/nVup2DTXvZal9DZGrESZfI3Zq/EXeth4ygHoQFRxIVGkN2SB+aKq3s3LONleuWYrYFkWLJRK8xYtYGEC7HsGzljyxbs+S0zJD1F7lFU/G4PL4Ow6dcLjeLFqzm3ltCCA+VUVGJi9HxwJRQ5i9some2gUNFLU+biqKyNq+ZjMzOXc3xf4nn4k6ge2Yvnnr0BdbmraC+vo6sjIvpld3npMe9l1eUYXIHtNkmSRIWJYjyqsNs3pZHZeNhaq01BOtDOWwrRucxEKpE89Wnc1m6fBEP3fs4gZYgb1ye4Gcqyuupqa5DI2uorVVpbHITHaknuYuBmjqFJcut3DwplF17rMyaW0NgWCK9+4jyFv9NJHY/5fG42VOwC5vDSnpyJuGhEYwZNc4rbcfGJGDT/tJmm6qqNMn1lJYVcXBvAeHOWCxyEKW2g9iw0l/Xj2bJQYwljv3lu5n3/edMGi+KgArtvfziXCwmBasNembrcToVysqdrNrQDOiwuUOY+63E+3MbcLkkKip2ccnIR7jsyiHcOuVijEa9ry/B50Ri90MlZYW8+taLqFbQYqBBreHSi8dx0fmnXocdoEe3XOZFfM6vFdtJMqUBKgftewmMCWDDxrX0Dx5GrbMWnaonga6Uqgcpdu8n0dIVnU5PFzmNDZvWiMQutHPwQAWlRYU885cUnnzxELdPCiIxTsPiX5qY+VEDRnMAU/98EcNG5HDD+L+j1zTSPU1LY1Mj8z77ngP7ynjp1Tt9fRk+JxK7n1EUhdffmUZkUwKx5pZVlZo9Dn744VtSktPI7Jp9yufQaXX88d4n+HrBHDZuWo0kSwwYPJicbr345MMPCdAHoY8yUltbjepWCCWSWm05EeEts11VoPPNBTwx+5s658LW1VUNJCXoGTE0jLBQPW+8V8yKNZWgqmSl6YkIV3n5uQ/56P14VE89/3w2mrp6BavNQ1WNh5dm5FF4qIKkLp27dK9I7H7mQFEBzQ3O1qQOYNAYiVETWbN+uVcSO7SUNLhx4hRunDildVvJ4SIcih1VVdHr9ERHxVJVXUG9tYZAUzCSJKGqKofsexkwZLBX4vBHK5eNRqNbQmrPzpfYu6bFsmefk5paF71yAggO1BIRKvPn+0K5aGQgOp2W/N0Oxt1cyKC+Rp58oYKcLD1BgTKr1jsItCgs+XEbk28b5etL8SmR2P2M09mMVmr/z6qRdDgcJzdu/XjFRScQmxDLgaLdJJszkSUZXYCOMsdBwjWR/Nq0A6tcT0h8CFdcMvG0xtKRDfUc/+Li/iYk1MJl40Zw/2PLmHJjCKvX15OapGX08IDWGaTds4yMu8TCNz9Y+WhGDANyTQDU1LgZc30JVVUNvryEs4JI7H4mJSkNh2yjyVVPgC4YaHmxWamWMKzXqc8Krag6TFVNJXHRCYQEt11FSZIk7pz8IG/Peo0Nh5ail424NS7uvuNBAiyBVFQfJjYqnm4ZPX53bVahc7v2huHY7W6mv7ud8iqJPj206PVHJtm53Ap2ByQnaenTwwCqiqqC2Sxz+cWBVNntPr4C3xOJ3c8YDUaun3gLH3/yLuHNsejQUyOVk5iRRN+eg066XYfDztsf/Yu9e3ZjlgNpUuo5d/BwJlxxA7J8ZDpESHAoD9/3JBVVh7HZrcTHJKLTiVEKwvFZuXwn0174hK5dJEIDITgkgCUraiktdxEXo8PlVCkpc7J1p0p4qJ7iUg9BgRKqArX1KmFhZppqRVoTfwN+aEDuYBLjurBm43KsVisXZ19Mz+w+p3SX/Nm8jyjfVUF/ywhkScaluNi4cj3R0bGMGDK63f5i0Q3hRFVW1jPt+Y949e9RdMu0AJC3pZHrpliZePthJl5uITBAw5KVHlIystlfUER1vRlJVpATNYBbAAAgAElEQVRkiaAQE0vXVDLp9p7HOJP/80pilyTpImA6oAHeUVX1eW+0K5y82Oh4rhzjnTXGXS4n6zeupq95GPK/11vVyTpC1Uhmz32PLVvzyMnpybkDRmAyds6iS97UOz2GzS5Pp1t04+fF2zh/qKE1qQP07R3IFWMiCYjqx4HqOpxlzVw4rhejL+7Dsp+38+dn53LJ+SaCgyQWLq0kNTOHcwZn+vAqzg6nnNglSdIArwMXAMXABkmS5ququvNU2xZOj4bGerbuzMPjcZOd0fOYd9cutwvFo6KTjnSpHGray966nUQQAwd1LC1Ywqo1y/jz/U+L5O4F+38YS/erFvo6jDPKanMQGtx+IGx4mExkYgRXP9R2HsYFF+XSrXsiS37cQoXVwV0PdiO3b6pYwQvv1IoZAOxTVXW/qqpOYA5wuRfaFU6DTdvW85dnprLo8+/5+cufeOb5x1i4ZP7vHmMymomNjafCUQqAW3Hxa90O0sghKSCNKGMc3S39cFa4WbX+lzNwFYI/6jcgnZ+WO2huVlq3NTa5+WWVndSu0cydvYL3Zi5my+YDqGrLakkJiRHcdMsorr5mKMXF1SyYv5HamiZfXcJZwxuJPR4o+q8/F/97WxuSJE2RJGmjJEkb6xvqvHBa4URZbU18MPtNumv60c3ch0xLb3IN5/L9D/MpKj30m8dJksS142/ikLyHAutODjT9iqzI6DR6QoJCWveLkGPZkb/1TFyK4Id69OxCVo9e3PZAId8sqOKLeZXc9kARKRmZPPvEO1QVLsGsrOHVF2by/DOfoigtXwDzv17LHTc+R8HW79mzaQG3Tvo7Py7s3GtReaOP/WjPPe0WH1RVdSYwEyA9Nav94oTCaZf/6zYCPCEEGoJbtxk0RsKVaPK2riMx7reXEEtLzuQvDz/LslU/sf9gAfIemdjQuDaFxJoVO9FBnXcMtnBqJEnij49OYNXKnqxatgWNRubmu3oy7fnZTH82iuyslr73m65RmDJ1ByuX7yS1awyz3vmGD1+PJy7GAMCBQ3amPPQZuX27EhHZOQvNeSOxFwOJ//XnBKDUC+0KXqYqCpLU/iFNQmp9tP09URExjL98Eqqq8sL0pzhUtI8UcyaSJGF1N1ImFTJ+iJh45A1dDQHYbU6+OLCYq1PajzryV7IsM/S8bIae1zJDeuP6faQkyq1JHUCvl7lyTAArftlMcVESF44wtiZ1gJQuJoYO1LNq5S4uH3dyaw50dN7oitkApEuSlCJJkh64Bvj9TlvBJ7IzetJADVb3kUUKXIqTKvnwcS+RB0cmIukSJdbbl7LVsYZ8zwYmjL+etGQxIsEbeqfHsHJZ50nov0WWJTxHKU/vdqtoNBo8HgXdUW5PdToJt7vz1rU/5Tt2VVXdkiTdCyyiZbjje6qq5p9yZILXBQYEcd3Em5k9533CHFFIqkyNpoJR519IcmLqCbUVEhzKIw/8lbKKEqy2JhJiu2A0GE9T5EJn1aNXF8oqZdbnNTCgb0u3SpPVw2fzm7j1nr5ERgXzl4cWMWmCi9AQHQAVlU6WrrTzr8mdr9bOf3hlHLuqqt8D33ujLeHoHM0Olq/5iU2bN2AwGBg6ZAR9ew464aFd5/Q7j4yu2WzesQGXy0nPbrnExyYd+8Df8J8FrgXhdNDptDz29M08/vi75HZvJCxUYtlqO8MuGMLAczKQJImLxo7ksut/IDrMg6JCWaWGKfdcSWxsS8mLxkY7jQ02omNC0Wg6x6JxYuZpB+Byu5j2+rM0FtuI0SXiVlx8UjCLA8MKGD920gm3Fx4awaihF5+GSAXB+3r1TuHDOY+zcvlOrFYHL0xMIzmlpSyvoij8ursEg87F+UNNGAw6Fi33UFRYjs3WzOuvfM2q5ZsJtMi4PHpuueMKRl+c6+MrOv1EYu8A8rato66kgZ6Wga136OFKNMuWLWHk0IsIDz29I1GKSg/xy8ofqaysIC0tg+GDLyAoMPjYBwqnrKqmiaXmlYyIPtfXofhUQICRiy7p02abqqo89vD7rF+5lm8+jMNskqmtV7j4gjDueWQjTzxSTUxICd/M6kJggJbde2386em5REQG0adfVx9dyZkhEnsHsOfXnYQR2abbRSfrCJbDOVC077Qm9h27tzDz3deIVhKwaILYsH8DK1cv5ZEH/0ZYSPgJt1dWUcLaDSuwWpvont3rlGvY+LP7UnN4bRlkXr/F16GcPi43mh1FSFYHSmQwSkYsHKV7cdfOIr6fv4a6ugZ69s7iggtzmTN7OT8uXMekK83Ex+rRaCSCgxUOHGpg5BAdb83azvofu2M2t3y+stLN3H5DMN98uUwkdsH3QkJCOUj7CUTN2AgK+O075wOF+1ixZimNjQ10z+7JoL5DT+gFp6IofPL5B6TJPQg3tTz6RhLL3sYdLFwyn+uumnxC17F+8+qWqpOelqqTW9ZvZkXGUu65depJL6wtdFCqim7BJgzfrEOJC0MJDUBzqBIkcEweiafnkTkVi77fxPtvfcb1VwUQ01vHwp8X8uKzH5GVpmXYIA2HK1wUHLQTH2vAbJKxmGVKy20EB2lak/p/pHYx8tmCmjN9tWec+G3qAAb3P48lPy8krDmKMEMkqqpSaNuHOcL8m8MLV29cxpw5s4hWEzHKJhbuWsDK1Uv5431PHndyr2uopbGugWxzZJvtMYZE8ndug6uO/xoczQ5mz32P7tp+BJhavowS1VS27VnLxq1rGdinc3c1dDb6L9agXbeXny/OZc7PGynP30NGZhK3D8gm49UF2O8fg6dHF5qbXcyc8RVvvBhL12QTbpeH+KgmbI0aggIkdu1zs22ni1HnmXG7FLQ6mYKDLpatdqE3miksdpCUcOTzvmJtI1nde/jwys+MzvGKuIOLDI/mztvup9i0l432Zay3/4ycqPCHO/7cphb6fzQ7m5n7xUd01/Un2ZJBjCmRHHN/rGV21mxcftznNRpMKJKCR3W32e7w2AgMOLEZfQcK92FUzK2LfwDIkkykFM+WrXkn1JbQsUk1TegXbOLrPinMmDuPG8bZeePFMAb1KOUP78xj75i+GD78BVSVfXvLiI2CrsktqyRVVtYjSwpXXxbIgp+s9Mgy0D1Tzxsf1PHPN2v5xyvV3PdoBcMvOIdbplzG1CfK+GlZLb8W2Hh71mHmLXYz/trhPr3+M0HcsXcQ2Rk9ee7JVymrKMGgNxAR9tuL9RaVHsSgmAgwHkm+kiQRKcexdVveUeunH43ZZKZP7/7s25RPhqUHsqSh2ePgkHsv1ww7sdE4ep0e9/98QQC4VRdGoxj//nv2FFcwItrXUXiPbvlOnAPTee+zJUx/Npb01JakffXlkbjdKu9s2sPzTjdyQTmBgSZqat0oioosSzQ12pAkOFTkIjlJy9adTqY/G0FCrI41eQ6Ky9xMmhjPjI8KePSJCURGh/Dll8uoqa4ju0c2r8wY0ToM0p+JxN6ByLJMfEziMfczGc00Kw5UVW3zwtWpNBMREPI7R7Z33dWTedc+g/V7lmKWA7CpTVww+hIG5A45oXZSktIwBOkpqy9sXWi72eOgnCKuGnD1CbXVmdyXmsNmFrO03H9GxkhVDdSGWpDU5tak/h/nDAjiswWFePr1RK5sIOmcDEIjYnj7wxIuv8iC2+2myarw5qx6Joy18Pm3VlK76PEoMLi/mfIqFb3RQvnhMgDOHZrNuUO9s4B7RyIS+1nO0exg7/5daDQa0lO7odPqjnlMXHQCUXHRHCrZSxdzOpIkYXdbKZMOMW7wlSd0fpPRzL23/5GqmgrqGmqJjYrHYg444euQZZl7bpvK9DdfpMJWgg499WoNl14yjsyune8X70Qs2jGEzIS9vg7Da1SLkQCHi2anRFW1i4jwI5/pvfvtaHVG8r7fwNLteyn9OoJ9e0vYua2BD2a7iYrQUF2r4HKpHCzykBinZcU6OwP7mNDpNDgcTrbvsZPZ7eQn3fkDkdjPYnlb1zHrk7cxqQGoqoLb4OTOWx4gPTWrzX5VNZWUlBUSFhpBQmzSv2u5PMCMd6eRd3g5esmATWpi3OUTTjqJRoRF/W73z/GIj03iuSdfYU/BTmwOG+nJmQQH+f9jsdCWe0gWpme/4OKLzuFvL63n8akxREXq2bXHypPPldJVlojXuRg0wcwX320gIljllddTqShvYMt2B19818gr/4jiT09XktJFy1Mv1nD/7WHERGvZsx8++bqRR/966gu3d2TS8VT187b01Cx1+t/ePePn7Uiqair563N/pru2P4H/fuFY3VzBfjmf5596FaPRhEfxMPvz91i/fjVBmlCsSiOJyUncfetUzCYLqqpSVHoIm72JpPhUzCaxslFH9Hz9Ov5w0V6/6YoBME77FkVRmWGQWPDDGrSyB63OjKeijs/TIHB8NBU5GjTYeGtWHSazgRuuNhEVoeOOPx5m3GXRRIR4mPpEMYWlCnqdFpPFyMBzsrjxlovJ6fHbJag7sojASXmqqvY71n5iVMxZauPWNYR6olqTOkC4IQqTO4Dtu1smrCxb/SPb1m2lv2kE3Y396G8aTsP+Jj798gOg5YVpUnwyWWk5IqkLZxXHvRcja2Qe2FbM/PGj+Oi6y3izRwZ/q7YSOCYCdWQYiqKg0cLIoWbmfV/LxNuLGXvDIYpKmikqbkCrceJ0qfToZsRscpOaJLN390G2bNp3XGWo/ZnoijlLORx2tEf559EqOhzNdgCWrVhCsi4DrdyynyRJpJqz2bBlKZMmNGPQG9odL3RM/jYyBr0Wx4OXIhdXo129h+AmB9XpcTyfEsLciyLQAhaLiZoqGzv32AkMgLdeiqOuwcXz02v48NNqvpyvIy7WyMBcHVNuiKL0sBO7Q+ax5+ah1xuYcG37JxxFUcjfUYTN2kz3HkkEBPjniCxxx36W6p7Zixq5Eo96pKa009NMrVRJt/QcoCX56zVtk7dW0qIqKm6367jPZbNbydu2ji35G3E0O7xzAYLXPBI8kE2V6exv2uXrULxOSQjHOWEwzbeMJOCGYURlpjLzw3I8HhVLgJHScpUP5jTwh9vDMJtkDHotj/whEkWB225OoK7OxeRrg6iqcRIdKdMtXcvdN5t45cWP2V9wuM25Duwv55brX+D1l2byzaezuXHC3/hu3jofXfnpJe7Yz1JpKZn07pvLlrxVRKpxKKpChVTC6AvGtL7E7NmjD3tX7yM9IKf1uHJHMXFxCcc9cmVt3kpmz32PQDUEBQWH1sqUyX8gO8P/Z+cJZ59Hn5zEs0/N4oobDxAdqSNvSw3n9g+ke7cgHE6ZmDgz1iYHA/rUU1/vISBAprHJQ1yMBq1GQqeTyexqJDaqgY/eX8xTz94ItNypP/nIO9x+nYYxo1v634tKHNz5x69Jz0wgM8u/yk+LxH6WkiSJGydOYWfuNvK2rEer0TKh7zVtRsSMuWAcL+x8mp31mwiWwrGpjdTqKrl//J+O6xwVVYeZPec9eugGYtEGAlDrrOKt96bz3FOvnpZ+eZfbRUlZIWaThaiImJNrw+VEUVXR1eSHQsMC+Odrd1N4qJLa2iZyN+2ntmQFISGBNDRYaaizojPo2LDFwaABMm6PxM7dDrokWPB4JGRZZvEvTQw5J4T1Wwpa29265SDBAXbGjD4yDDIx3siEsRYW/bBBJHbBu4rLCtman4csyeT26E9MVFzrz2RZJierNzlZvY96bEhwKE88/A9Wb1jGgQMFZEance7AEcdd7TFv2zrCPFFYTIGt20L1EVhsQezYvfmEJyEdjdvtpqGpjkBLEJt3bOTTzz9A49LiVJ3EJcRzx+T7CQ0OO6626upr+eTL99mRvwVVhcz0blw3fvJJf0EIZ6+kLpEkdYmkS5coxo9dQHhQI1ddGojHo/LSq6XUWw28N8dJs93An56poqpWIaOrkRVr61m9wcldt0awr0jf2p7V6iAspH0V0bBQLXtKrGfy0s4Ikdh9aP7CL/jpp4WEeaJQJZXvf5jPleMmHveUfwCzycKo8y6B8078/E5nM/JRPgIaNDhdzhNv8L+oqsrSlYv49oevUZwenEozzQ4nA4KHE2gKRlVVDhbtYca703jswWeOuRKUR/EwbcY/0FYZGWg6H0mSKdq3n5dee5a/PfZSp1iWb0VRAandOtdyb5UV9djtDlZvlJj9ZQMajczwc0PJyZK45pYJxMWGMnPG93z4+VYyu6r07hHEc0+H8Ow/yxk78WoURWHrloM01NnI22rjcIWTmKiWhK8oKt//ZGXUWP+bICcSu48UlxXy008LyTUMaX0Banfb+PLrT+nVve9J1To/UTlZvVn604+4lXS0csvsv2aPg1qq6JZ+an3s6/JWMv+br8jW98ViDmRL9RpMNhVFr4Kupasp2ZzJxtJlFJcVkhj3++OOd+7Zhr3aTm/LkdVvki3p5Fvr2bR9HYP7DTuleM92fbf3xx7pP7NPj0dZWS133/YKQ/pqeOmvkTQ2KdTVKyQmRfLd4no2b9zN8Iev5h8vTeb16d+w/OeN1Da6+Py7Cq665gJyeiRx66QXMekbiY3W4Wxu5rJr9zD1rihCQnR8t7gJVZfAiPP9732SSOy/Q1EUCksOAipJ8SlHraR4srbuzCPME9VmVItJaya0OYL8PVsYOvD8Nvvb7NbWUSvpKVlU11ayfuMaZEmif99z6NEt94TjS+2STv9B57Bh7WoilFhUPFTKZVx6yZWnvHjHoiXfkarNxqJr6eZxKx4CpRAaGusJDmqpVyNJEkbZRJO18ZjtVddWYlbbvxA2eixUVlWcUqwdQe/0GDb7Oogz7OP3FzPsHJnGBonAAC2BARAY4KayopbaerBYWt4B6fVaHnz4am6dcglVVQ3ExoVhNOq469ZpXHOZh6svb7lpqK2LZPJ9B/lxbQQR4WaGXZzD+Rf0QqfzvzR4SlckSdJ44GmgGzBAVdWN3gjqbFBw8FdmfvAabmtLRUKtRcvtN9/7m/XPT5RG0qBK7SdRqJKKLLf9Z9lTsJM33p6GxR2M7NHwvu1NTJKFrpaWx/IPt75Nn0H9mTT+1hOKQZIkrrtqMv1yB7Fp23p0Wh039J5McmLqyV/Yv9XUVpOkPfJ3FW6MpNxRilExtxYnc3hsWNUGuiSkHLO9+JgkGqW6NoXNVFWlSVNHYrx/zjLs7Lbk7eblZ2J44JG9/LLayvDBFgICNGzc2sRXCzy8+GrbpfKCgs0EBbck+09nL6ewYB99Hoyh/HAtYeFBhIbouG1SGL9sNPHEMzf54pLOmFP9qtoBXAm85YVYzho2u5XX3nqJZE8mkaZYACptZfzrrZf4x5OvYDZZTvkcuT36s+D7b7C7UzFpW9prctVTL9fQs9uR7gaX28XM91+lq5pDmDmKOmc1JXWHSCaLiKAYjEYTMUoi69ct57zB55MUn3xCcUiSRGbXbK8X4krp0pWKgsMkmFviSbSksr9xD4XKPszNJpo9Dso4yGWXXX1cf59pKZnEpySQX5BHF0M6MjJFzQUExFro2a3PMY8XOp6AIDMNDR5e/Gsajzy9jw/mNGAxSyxaauXJZ6eQ2vXoL83nzl7GV5/OJzZaIjFOQ2Ojg6JDdpKSowkO0mC328/wlZx5p9S3oKrqLlVV93grmLPF5h0bsbgCiTTGtm6LNMZicQaxeccGr5wjOjKWq6+8li3O1eyybWaXbTP5no3cdP2UNotYFBz8FblZS5ihZex6VXM5oWo0BsmE1dYEgFbWEqJEsKcg3yuxecPlY8ZTKu+n0FpAk6uBckcp5iAT/Uf1x51oI6i7mSl3/IHRw8ccV3uSJHHPbX9kwKiBHDTsZK9uGznn5fDQPY+j1frfo7QAF186lBnvVRMfZ+DLj3py241J2JqNXDlhJGPHDTzqMTZbM3NnL+KtaUlYbXDgkIvICB1BgRI11Y3M+6GRfgN7nuErOfPO2G+EJElTgCnQsiLQ2cxqa0SrtB8jrVX0rcnUG4YNvoCe3fuyY/cWZFlDz2657VYmUlUVWTry/auT9Lil9iNWPLLbK08S3pKSlMZD9/+F7xfP41DRbqKTY7lv1MOn9GRgNBgZd8lExl0y0YuRdhyLdgwBvuLOvidWermjGjtuAKWlFVx58yoyuho4WOgkIzuLBx7+7fr9RYVVxETJdEkwMfWeJB544hBXXmIhLFTm82+rMIakM2Zs/zN4Fb5xzMQuSdJPwNGeef6iquq84z2RqqozgZnQUt3xuCP0gfSULL6X5+NRPWiklrGvHtVDvaaa9JSsYxx9YoIDQ0iM64Kj2YFOp2/387TkDJzaZuqc1YTow4k1JbCnbhuBagjJ5jSgpeqjVVtHbs6JfWAVRWHfwT2UlZcQFRFDZtdsr74gTopP4c7JD3itvd8jO5sJ272ZkIJdyG4n9ohYKnsOxHGW30SciEeCB7KZVb4O44yRZZm77xvLNdeN4OCBCqJjQohP+P3RYqFhFg6Xu3A4FM4fFkZKFxPfLqxi2Q+NuDTJ/Gv63ej1/v+Ed8wrVFV11JkI5GySnNiVnrm92bppDTFSy0y1w2ohPfr0Ijmxq9fOU1ZRwhvvvExTbRM6SY9DsnHt+JsY1G9o6z46nZ5bb7qbme++SlBTGFpVhzHASIlnH81qE9hBMXq46+apJ3THbnfYeG3m/3G4sIxANYQmqYHQmBAeuOtRAiyBx27gLGKqLCP9q/ewxiSwMyqeJlUl0+0i87O3qMgdQtmg84/diHDWCgsPJCw8EEVR2q0KBi1PtV9+toqvP1tCRWUDEvCHRw8w7dlkUpNNjBoWxo/L3fzlmas7RVIHMdzxqCRJ4uZr7ySv+zrWbVgFqFzT/wb69hx4zIk0x0tRFP418/8Irosiw5SLJEk0uRr4ZM4HxMcltRnXnZPZi2cef4lN2zfgaLaTlXYLCbFdOFC0DwmJlKS0E+5nnr/wC+oPNtLHMhRJklBVlb2lO/h83mwmX3enV67xTJCdzaR/9R47cofw9LJF1KxYhV424JBs3HjJVdy0azPNwWHU/NcLaaFjOXSwgrden0fe+j3o9VpGju7H7Xdd2lqZ8ZOPfmbdsiW89HQUaSkxrFxTz8NPH2LkFb8SE2VElYxMuffaNjXa83cUsviH9dhtDvoPymHE+T3QatvPTO2oTnW44zjgNSASWCBJ0hZVVS/0SmQ+Jssy/XufQ//e55yW9gsO/kpzvZP4f48aAQjQBRHpjGf1+mVMvOLGNvsHB4W2m5GakXrysxDXrl9FtrFf6xdVy4ShDDZs+oWbrpni1S6Z0yl85yaaYhN5+pdFaMuN9DMPQ5IkrO5GZn37FRlXX8ugDb9Qk9UbvPSlLJw5dbVW/vTA69w8Xs9Lj6XRZPXw5gfbefqxSv5v+p24XB6+nPszH7waS3ysAbvdSWZXlXtvCebLxYH8/cXbSEgMb/N5/urzVXzxybdMvNxCcJCWBd/s4qdF6/n7i7f6TXI/1VExX6uqmqCqqkFV1Wh/Sepngs1uRS+1nwavx4C1yXsvaH+LR/G0eSkLICOjKMppP7c3hezfya6IGGorakgyp7V+UVm0gUSriXxzcB86WxP6hjofR+o9b2877ldbHd6i7/MY3BcmjIvEYJAJD9Px6ANxVJcXsmd3CXV1Vgw6D/GxBmprmzhcUolB56BfLw2F+w/w+itfoShHXunV11mZ9e63zJwWz6QJ0Vx2UTgz/i8Jt+0Qy5bu8OGVelfHuC3zQ6nJ6TSqtTg8ttZtqqpSxWFyco5e9Mub+vYeQKG9oM22InsBvXL6dJi7dQDZ5aJBUdDLhnbdZAbZSGNTEx6DEdlz/PXpz2a5RVPxuDzH3tFPFBcdJier7Qg1WZbIzjRSVFhFSIgFp1vLoUI7NVX1dEnUER6qY/9BNyPPC8ZtL2yTsLduPUjPbAOx0YY27Y0ZZWHD2rNnuPCp6ji/wX4m0BLEZWOuZlvzOg5Z91FqK2SbbR3RKVEnPLrlZFwxZgJKuIPttnXsb9zNDtsGmoLqmDDuBgD2HdzDB5++yevv/JMV637G6Ww+7TGdDHtEDFkeDzaasLmPVOlTVZVKpZSBqelobU04A0N8GKVwsrqkxLN5R9vPnsejsnWHneSUKPR6LeOvGcVDTxZRWOxEUWDJ8iZef7+OG6+JbZewAyxG6uvbfzHW1XswB/jP8pHi5akPjR4+huSkVFatXYbdbmNwz6sYkDsYnVZ32s8dHBjCk396ns07NlBSWkR0dCx9egzEaDDy88pFzPvmc6LVJPSyge92zWP12uVMvfuxow7J9KXKngNJ//Idrr9oHJ9+P4/o5iQMspFKpZTg+EAuczupycpFOcviFo7PhRfncsfcJbz38WGuGBNOk9XDWx9W0iUtnfSMlhLXE68fxuHDdTzwxALMpkpSu2i548ZgAk2NHC5vm7B75aZQVW9g8dIaRo9oKRddUtbM599aefp5/xnfLvli0df01Cx1+t/ePePnFY7NZrfy56fuo5f2nNZSB6qqst22nrETxzGgzxBqaqsIDAg6ayZExa1aTOj/t3fv4VHVdx7H39+ZJOSeEAKEEG6RBCThIsZUQK0oi+AqWKmtrbcWW9pae9laLy37aLs+dmndarurK2r1ae1q1z6t1i62ClZFeZS7EIPhGrmEexIgmSSTycx8949EIQK5OJmcycn39Tx5Hs7keM4nQ/h65nd+5/fd8T6rxk/mL7t24GvwcWHBOOa1BMjav5ut199GyEXNvN8b8VC/eUgJWld5fPrxl1n77hYSkxK4fPZnuHnhLBITT/7POhAIcvEFt/PA3eksuDoDBLbvbOYLXz/Iv//qDqbPOPn8yc4dB/nJj58iM81PZrqXLdsCfHXRfOZ97kInfrxuyU67cYOqlnS2nxX2GBYOh/nH23/nH2++is9XT8HY8Xzuqi92ez2Y7vhgexnP/OYpJia2f2T7QOMe6gZV46uvhxahRQOUXjCdL117i/NX8aoM+vrY/moAAA/1SURBVGADOetW4m32ExowgPiGemrHTWH/jCtcVdShtbAnJSdwy7lXOR0lZqxdvZ2lDz9FKNhIWoqSmeGlYnuA4cNTueCSOdz0lfbPMny0TntTY4CJk0eRlpbkUPLu6Wpht6GYGPbCsj+w+s13yE+YQPKAFA5tr+KX//UAi394f9S6BiUlJhPQ5tMeBDnSfJDa3UcozbqU1KR0WsIBKtZs5o+e/+GG6xZGJUuXiVBTVELNhPMZcKIGCQYJpA8k7NLWefL6lwlc8bzTMWJKTU09ReOTuPfOfDaX+/A1hJgyMZVXXqtl28ETp+3v8Xg4b2rkq5jGKrt5GqMamxpY+fY/KEoqISNhIPGeBEak5JPdksNrK/8WtfOOyssnbVAa+5oq+ejTXEOwngNNeyhMnkhqfOtaNvGeBAqTJ/Hu2rfxN/ujlqdbRGjOzMafnePaom7OrKh4JGs2NtHcHOa8SWlcPC2T1BQvb77jZ0Kxewv42Vhhj1FHaw6TKEntGnEAZMZls3fvnqid1+Px8O2v3UHz4Ho2NL1FmX815aG1ZGUNIiOxfW/SBM8AvOrt0YXRjPk0Ro4azGcuKuE7P6ri7XePs3FzPf/6syp8gWw+O7PY6Xi9zoZiYlTWwGyatJGWcIB4z8kx7BPBWkbm5kX13EOyc7jvrp+z78BumvxNjMrL5/kXn2Hv2n2kxWeczBKoJSE5gcyMgVHNY0xX/MudC3jlb2P4/V9WE2hu4cKLLuO7i6f3m/VhTtX/fuI+Ii0lnWmlF1P27kbOSSwiyZvC4aYqjsbtZ+GlX4/6+UWEkcNPdjaaO2seS8ruY5dPyU4Yhi94gioqufELX8Xrccdj2H1JqCVEpa+C/NT+1dy6Ix6PhyuvKuHKqzq9t+h6NhQTA1SVM81Ouv7am5k++2I+YD2rfH8nOLyR73zrTnKHRveK/UyGZOdwzw9+Sv60MRzJ3EPSuHhu++b3KZ06o9ez9HdTCnJYtXI2b+/b1fnOpl+yK3YHNTT6+PP/Pcea9e8QDoeZMvF8rrvmBrIyWxtJe71xzJ9zHfOu+DxhDffIlXFjUwMfbC9DFc4tKO7WEr1DsnO63VfVREdu4+nrDBnzESvsDgmHw/xq6RKaq1ooSboEj3jZW7aTB/fez0/u+QUDTpnVISIfN/yIxKYt63n6mcdIDWeAwjOeJ7npS1+j9LzpER/bGBM7bCjGIdsrP6D2YC2FKZNI8CYS54lvHS+t87Dx/bU9fr56Xx1P/+4xzvWcT1FSCUXJJRR7S/n9c7+h5lh1j5/PGOMcK+y9rLr2KMtWvMCLLz9PyB9CaT+2nhxK49Ch/T1+3s0fbCA9PJD0+JOLYaXGp5MZHsymLT3ToNsYExussPei8q2b+Lcl97DulXU071KO1x9j7ZE3CenJ1eYavHUMG9bzN0eDwRY8evpwjifsIRA4vTm2iX1NjQHeOLzK6RgmBllh7yXBYJDfPvs4hZ7JFKZMpCBzAsUDLsDf7KeyfivNIT87fVvwZgpTo7Bs74TCSRyTozSHTj4l2hIOcMx7lOLx0V//3fSsBcVjWbVyNjWNjZ3vbPodu3naS6oO7kH9wsCk7I9fGzp4GA01dexoLKMm4QDnlZRy7T9fT0IUHocfkp3D3DnzeOWVZWSHcwChxnOIS2fOatdf1RjT91lh7yXxcfGENNhucS2Px0NSSjLTz72E2269I+oZrpx1DUXjJ7Fh8xrC4TBTJ91M/qiCqJ/XGNO7rLD3ktycEWRmZ3Kges/HDaxbwgEO6m5uuTD6T5J+ZFRePqPy+t+iSMb0JxGNsYvIgyKyVUTKRORFEbH+Y2chInxj4feoTTvE5qZ3qWh6j/X+t5jx2UuYPOF8p+OZPqq61hZgM6eLqNGGiMwGXlfVoIj8HEBV7+7sv+vPjTZCoSBbd27B11DPOaPHkZ01OCrnOVF/nMo9O0hJTmXs6HF9qkG16br3RjzEuLwhzBx6kdNRTC/olUYbqrr8lM3VwOcjOV5/4PXGUTRuco8f92jNYd5Z9xbHjtVS11DHtootZHiyaMZPUmYi3/3GXVFrzmGc82r5DMbl7XA6hokxPTnGvhA4a1sXEVkELAIYPGhoD57WVOwoZ+mTD5MVysEf8FNV/yETEs5nxNBRiHjYd6ySx556mHvvWtKuK5Ixxp06/XwuIq+JSPkZvuafss9iIAg8e7bjqOoTqlqiqiUZ6TYU31PC4TC/e+4JzpFixqYU0RJqZpQU4g3GU+erAyAvaQzHjh5j/6F9Dqc1xvSGTq/YVXVWR98XkVuAq4DL1YnO2P3ckepD+H1+BiW1fgpqCbeQKvHESwKNjQ1kpGUiIsR7EmiOlRZ2pkfZQ0rmkyKdFTMHuBuYp6r22+WAhIQEQuEgYQ0DMDQ5l6O6n3A4hEda/3rrWo4TjAswMm9MR4cyfVDBvkFU1/qo9FU4HcXEkEinSjwCpAErRGSTiCztgUymG7Iysxk5ejR7G3egquSl5EMCbAmvoz7uGLsaKqgIbuDGL95KfFy803FND1tQPJaNR+0hM9NepLNixvZUEPPp3XrTbfzn47/gvepVJEoypISZMKWIrPRs0tPTmV76DUe6LhljnGFPnrpAVmY29965hMq9OzhRd5zRI85h0MDWNWlUlUNHD3DwyH5yBufarBhj+gEr7C7h8XgYO3pcu9f27t/NU888yona46CQnpXBrTffZksKGONy9jiiS/mb/fz6sSWk1QyiJPFSSpIuJePYEH792M9pbLL73G6zfJvdPDUnWWF3qU1b1jPAn8yw5JGICCJCTlIeic2pvFduHZPc5Fb/1U5HMDHGCrtL1ftOEB86fV33+NAA6n0nHEhkjOktVthdKn9kASe8NYRPabsX1jB13lpbg90Yl7Obpy6VP6qA8UXnUla+llzvKEA4ENxNQVEBBWPGOx3PGBNFdsXuUiLC12/+DvO/eC3BvEaCeQ1cfd01LLrlezbl0aWWbnjB6QgmRtgVu4t5vXFcVDqTi0pnOh3FRNl5+37AeyMecjpG1Mjh43iONRDOTEZzBjodJ+ZZYTfGxCxvRRUD/rAKz4FawkMy8VTXER6SQfP1MwgVj3Q6XsyyoRhjTEzybt5N0i//StWUMfyscAjfPFHN/fmD2DM1n6RfLcO7YZfTEWOWFXZjTOwJh0l8fAUV8y5g0dMvkZNRwbdubGFk9la+9fRLlF1TSuLjKyAY6vxY/ZANxRjjIn/6cDmfHzPb6RgR827eg6Yn8eibG/n2V1OZN7d17aOpk9PIGVrLI6+sZmlONnEbPyRYamsRfpJdsRvjEvL6l6mu9Tkdo0d49tcSKsylbNMuZn22/c3Syy/J5P2yD2kpGIZnf41DCWObFXZjXGJKgYualSfGIz4/AwemsP9QoN23Dh4OkJmZgtfnRwdYj4EzscJujIk5wan5xG2sZP7caTz830eoqw8CUO8L8h+PHuHqudOIW7eT4AU2DHMmNsZujIk5mpVKcFohN9c2Uj22hGu/soa83HiqDgSYObOErzUFCU7NRwenOx01JllhN8bEJP/Cy0j89d+4s/IIi268koNeD0NDysA12wkPj6Pp9rlOR4xZNhRjjMs8WfaS0xF6Rnwc/juupum7V5LS2Ezh3mrSGvz4b5tD013XQIJdl56NvTPGuIi8/mW44nmnY/QcEcKFuTQX5jqdpE+J6IpdRO4XkTIR2SQiy0XE3n1jjHFYpEMxD6rqJFWdAiwD7u2BTMYYYyIQUWFX1bpTNlMAjSyOMcaYSEV881REHhCRfcANdHDFLiKLRGS9iKw/UXc80tMaY84i1BKi0mfNrfuzTgu7iLwmIuVn+JoPoKqLVXUE8Cxw+9mOo6pPqGqJqpZkpGf23E9gjPnYlIIcVq2czdv7bOXD/qzTWTGqOquLx3oOeBm4L6JExpiI5DYmOh3BOCzSWTGndkWeB2yNLI4xxphIRTqPfYmIjAPCwB7gm5FHMsYYE4mICruqLuipIMaYnhNosQYU/ZktKWCMyywoHkuoJcSfPlzudBTjECvsxrjQqpV9v4uS+fSssBtjjMtYYTfGGJexwm6MMS5jhd0YF8ptTKS61scbh1c5HcU4wAq7MS60oHgsr5bPcDqGcYgVdmOMcRkr7MYY4zJW2I0xxmWssBvjYtuqjtja7P2QFXZjXOqejM+w8WhB5zsa17HCbowxLmOF3RhjXEZUe7//tIgcpXX99q7IBqqjGCca+lpmyxtdlje6+lPeUao6uLOdHCns3SEi61W1xOkc3dHXMlve6LK80WV5T2dDMcYY4zJW2I0xxmX6QmF/wukAn0Jfy2x5o8vyRpfl/YSYH2M3xhjTPX3hit0YY0w39KnCLiI/FBEVkWyns3RERO4XkTIR2SQiy0Uk1+lMHRGRB0Vka1vmF0Uk0+lMHRGR60Rki4iERSRmZ0OIyBwR2SYiO0XkHqfzdEZEnhaRIyJS7nSWzojICBF5Q0Qq2n4Xvud0po6ISKKIrBWRzW15fxrN8/WZwi4iI4B/AvY6naULHlTVSao6BVgG3Ot0oE6sAIpVdRKwHfiRw3k6Uw5cC7zldJCzEREv8CgwF5gAfElEJjibqlO/BeY4HaKLgsAdqnoucCHw7Rh/f5uBy1R1MjAFmCMiF0brZH2msAMPA3cBMX9TQFXrTtlMIcYzq+pyVQ22ba4G8pzM0xlVrVDVbU7n6EQpsFNVK1U1APwvMN/hTB1S1beAWqdzdIWqHlTVjW1/rgcqgOHOpjo7beVr24xv+4paXegThV1E5gH7VXWz01m6SkQeEJF9wA3E/hX7qRYCf3c6hAsMB/adsl1FDBeevkxERgPnAWucTdIxEfGKyCbgCLBCVaOWNy5aB+4uEXkNyDnDtxYDPwZm926ijnWUV1VfUtXFwGIR+RFwO3Bfrwb8hM7ytu2zmNaPuM/2ZrYz6UreGCdneC2mP7n1RSKSCvwZ+P4nPinHHFUNAVPa7mG9KCLFqhqV+xkxU9hVddaZXheRicAYYLOIQOswwUYRKVXVQ70YsZ2z5T2D54CXcbiwd5ZXRG4BrgIu1xiYA9uN9zdWVQEjTtnOAw44lMWVRCSe1qL+rKq+4HSerlLV4yLyJq33M6JS2GN+KEZV31fVIao6WlVH0/oPZqqTRb0zInLqItjzgK1OZekKEZkD3A3MU9VGp/O4xDqgQETGiEgCcD3wV4czuYa0XuU9BVSo6kNO5+mMiAz+aLaZiCQBs4hiXYj5wt5HLRGRchEpo3UIKaanYgGPAGnAirYpmkudDtQREfmciFQB04CXReRVpzN9UtvN6NuBV2m9sfdHVd3ibKqOicgfgHeBcSJSJSK3Op2pAzOAm4DL2n5nN4nIlU6H6sAw4I22mrCO1jH2ZdE6mT15aowxLmNX7MYY4zJW2I0xxmWssBtjjMtYYTfGGJexwm6MMS5jhd0YY1zGCrsxxriMFXZjjHGZ/wcZdfnrlaL8PgAAAABJRU5ErkJggg==\n", "text/plain": [ "

" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_decision_function(data, target, svm, svm.support_vectors)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "#可视化训练过程,建议在pycharm中运行，notebook会生成很多张图片\n", "# svm = HardMarginSVM()\n", "# svm.fit(data, target,show_train_process=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 六.问题讨论" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 1.非线可分的情况如何处理？\n", "大家可以将上面的代码多运行几次，可以发现如果有异常点等情况出现时（即线性不可分时），模型训练的结果会很难看，后面小节将会对这种情况做处理，教模型如何去“容忍”这些不好看的点，或者巧妙地通过坐标映射的方式将低维数据映射到高维空间进而可以线性可分 \n", "#### 2.原问题本就是凸优化问题，为何还要转对偶问题求解？\n", "个人觉得更多是为了引入核技巧，因为对偶问题进行计算时，有关于两个点内积的计算：$x_i^Tx_j$，这可以方便的用核函数替代$\\kappa(x_i,x_j)$，便于处理非线性可分的情况" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" } }, "nbformat": 4, "nbformat_minor": 2 }