{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### 一.简介\n", "xgboost分类分两种情况,二分类和多分类: \n", "\n", "(1) 二分类的思路与[logistic回归](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/02_%E7%BA%BF%E6%80%A7%E6%A8%A1%E5%9E%8B_%E9%80%BB%E8%BE%91%E5%9B%9E%E5%BD%92.ipynb)一样,先对线性函数套一个`sigmoid`函数,然后再求交叉熵作为损失函数,所以只需要一组回归树并可实现; \n", "\n", "(2)而多分类的实现,思路同[gbm_classifier](https://nbviewer.jupyter.org/github/zhulei227/ML_Notes/blob/master/notebooks/10_06_%E9%9B%86%E6%88%90%E5%AD%A6%E4%B9%A0_boosting_gbm_classifier.ipynb)一样,即同时训练多组回归树,每一组代表一个class,然后对其进行`softmax`操作,然后再求交叉熵做为损失函数 \n", "\n", "下面对多分类的情况再推一次损失函数、一阶导、二阶导: \n", "\n", "softmax转换: \n", "\n", "\n", "$$\n", "softmax(y^{hat})=softmax([y_1^{hat},y_2^{hat},...,y_n^{hat}])=\\frac{1}{\\sum_{i=1}^n e^{y_i^{hat}}}[e^{y_1^{hat}},e^{y_2^{hat}},...,e^{y_n^{hat}}]\n", "$$ \n", "\n", "交叉熵: \n", "\n", "$$\n", "cross\\_entropy(y,p)=-\\sum_{i=1}^n y_ilog p_i\n", "$$ \n", "\n", "将$p_i$替换为$\\frac{e^{y_i^{hat}}}{\\sum_{i=1}^n e^{y_i^{hat}}}$,得到损失函数如下: \n", "\n", "$$\n", "L(y^{hat},y)=-\\sum_{i=1}^n y_ilog \\frac{e^{y_i^{hat}}}{\\sum_{j=1}^n e^{y_j^{hat}}}\\\\\n", "=-\\sum_{i=1}^n y_i(y_i^{hat}-log\\sum_{j=1}^n e^{y_j^{hat}})\\\\\n", "=log\\sum_{i=1}^n e^{y_i^{hat}}-\\sum_{i=1}^ny_iy_i^{hat}(由于是onehot展开,所以\\sum_{i=1}^n y_i=1)\n", "$$ \n", "\n", "所以一阶导: \n", "\n", "$$\n", "\\frac{\\partial L(y^{hat},y)}{\\partial y^{hat}}=softmax([y_1^{hat},y_2^{hat},...,y_n^{hat}])-[y_1,y_2,...,y_n]\\\\\n", "=softmax(y^{hat})-y\n", "$$ \n", "\n", "二阶导: \n", "\n", "$$\n", "\\frac{\\partial^2 L(y^{hat},y)}{\\partial {y^{hat}}^2}=softmax(y^{hat})(1-softmax(y^{hat}))\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 二.代码实现" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "os.chdir('../')\n", "from ml_models.ensemble import XGBoostBaseTree\n", "from ml_models import utils\n", "import copy\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "\n", "\"\"\"\n", "xgboost分类树的实现,封装到ml_models.ensemble\n", "\"\"\"\n", "\n", "\n", "class XGBoostClassifier(object):\n", " def __init__(self, base_estimator=None, n_estimators=10, learning_rate=1.0):\n", " \"\"\"\n", " :param base_estimator: 基学习器\n", " :param n_estimators: 基学习器迭代数量\n", " :param learning_rate: 学习率,降低后续基学习器的权重,避免过拟合\n", " \"\"\"\n", " self.base_estimator = base_estimator\n", " self.n_estimators = n_estimators\n", " self.learning_rate = learning_rate\n", " if self.base_estimator is None:\n", " self.base_estimator = XGBoostBaseTree()\n", " # 同质分类器\n", " if type(base_estimator) != list:\n", " estimator = self.base_estimator\n", " self.base_estimator = [copy.deepcopy(estimator) for _ in range(0, self.n_estimators)]\n", " # 异质分类器\n", " else:\n", " self.n_estimators = len(self.base_estimator)\n", "\n", " # 扩展class_num组分类器\n", " self.expand_base_estimators = []\n", "\n", " def fit(self, x, y):\n", " # 将y转one-hot编码\n", " class_num = np.amax(y) + 1\n", " y_cate = np.zeros(shape=(len(y), class_num))\n", " y_cate[np.arange(len(y)), y] = 1\n", "\n", " # 扩展分类器\n", " self.expand_base_estimators = [copy.deepcopy(self.base_estimator) for _ in range(class_num)]\n", "\n", " # 第一个模型假设预测为0\n", " y_pred_score_ = np.zeros(shape=(x.shape[0], class_num))\n", " # 计算一阶、二阶导数\n", " g = utils.softmax(y_pred_score_) - y_cate\n", " h = utils.softmax(y_pred_score_) * (1 - utils.softmax(y_pred_score_))\n", " # 训练后续模型\n", " for index in range(0, self.n_estimators):\n", " y_pred_score = []\n", " for class_index in range(0, class_num):\n", " self.expand_base_estimators[class_index][index].fit(x, g[:, class_index], h[:, class_index])\n", " y_pred_score.append(self.expand_base_estimators[class_index][index].predict(x))\n", " y_pred_score_ += np.c_[y_pred_score].T * self.learning_rate\n", " g = utils.softmax(y_pred_score_) - y_cate\n", " h = utils.softmax(y_pred_score_) * (1 - utils.softmax(y_pred_score_))\n", "\n", " def predict_proba(self, x):\n", " # TODO:并行优化\n", " y_pred_score = []\n", " for class_index in range(0, len(self.expand_base_estimators)):\n", " estimator_of_index = self.expand_base_estimators[class_index]\n", " y_pred_score.append(\n", " np.sum(\n", " [estimator_of_index[0].predict(x)] +\n", " [self.learning_rate * estimator_of_index[i].predict(x) for i in\n", " range(1, self.n_estimators - 1)] +\n", " [estimator_of_index[self.n_estimators - 1].predict(x)]\n", " , axis=0)\n", " )\n", " return utils.softmax(np.c_[y_pred_score].T)\n", "\n", " def predict(self, x):\n", " return np.argmax(self.predict_proba(x), axis=1)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "#造伪数据\n", "from sklearn.datasets import make_classification\n", "data, target = make_classification(n_samples=100, n_features=2, n_classes=2, n_informative=1, n_redundant=0,\n", " n_repeated=0, n_clusters_per_class=1, class_sep=.5,random_state=21)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd0FFX7wPHvbN9N7z2hhE5C6FXpolRBQMWCFX3timLF/tp7AUGKgIgiKoqA9C69hN5JSEJ6L9tnfn/kZ2LegBCyyUK8n3M8R4bZO8+GzbN37tz7XElRFARBEISGQ+XuAARBEATXEoldEAShgRGJXRAEoYERiV0QBKGBEYldEAShgRGJXRAEoYERiV0QBKGBEYldEAShgRGJXRAEoYHRuOOiAYFeSlR0kDsuLQiCcNVK3HsmR1GUiyZPtyT2qOgg1mx8wx2XFgRBuGoFet2efCnniaEYQRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGJHZBEIQGRiR2QRCEBkYkdkEQhAZGU9sGJEmKAuYCoYAMTFcU5dPatiu4T1FhGSuW7yEtNZPGTSIYMCgBDw+Du8MSBOESuaLH7gAmKorSCugGPCxJUmsXtCu4QXJSFvePf5eUo6tpFXGUgzuW8cBdH5CdXeju0ARBuES1TuyKoqQrirLn//+/GDgCRNS2XcE9pn2xmLvG6HhlUjijRwTxzsuR3NBHZvb05e4OTRCES+TSMXZJkhoB7YHtrmxXqB92u4M9u44zYnBAleOjhvqz7c8DboqqbsmyTNKZLDIy8t0diiC4TK3H2P8iSZIn8BPwhKIoRef5+wnABIDIqID//WvhCqBWq1CrVVisMnp95Xd+mVlGr9O6MbK6sWPbcT778Ac0kpkys0xkTBSTXhpHaKifu0MThFpxSY9dkiQt5Ul9vqIoP5/vHEVRpiuK0klRlE4Bgd6uuKzgYiqVit79OjB9ThaKogAgywpfz8um78AutWrbbLax7c9j7Nh2HJvN4YpwayUtNZf33pzNy0+a+OmbRiyZ35hrOuQz+dkZyLLs7vAEoVZqndglSZKAmcARRVE+qn1Igjv959HhHE7yZ9wDybz+/jluuusMRdZo7rxnwGW3uXH9Qcbd9BqLF8xn4Zx53Db6dfbsOuXCqGtu+e87GDbQSMcELwA0Gok7bg5CLRdwYH+yW2MThNpyxVBMT+AO4IAkSfv+/9gLiqIsc0HbQj3z8jbx6ZRHObA/mbTUXAbfHEqLlpf/LDwzs4BP3pvPl++E0CLWBMCexGKefWUWc75/CS8vo6tCr5G83AI6t6z68ZckiagILXm5JW6JSRBcxRWzYjYriiIpihKvKErC//8nkvpVTJIk4ts14oYhHWuV1AHWrdrPwN76iqQO0KGdFx3jNGzeeLi2oV621m2bsm6LuWLICaCk1Mme/WZatYl0W1yC4Api5alQp8rKLPh6S9WO+/lKlJVZ3RBRuQGDEsjM9+XV99LYk1jM+s0FPPJsCv2u7ykengpXPZHYhTrVqWtzVm2wYLFUPpAsKnawfouZzl2auS0ug0HHh58/THCj3nz+jYoflnky/OaxPPToMLfFJAiu4rLpjoJwPnHxMbSKT+C+JxIZNcQDhwN+XFLCgBuuJTomyK2xeXoauPPu/tx5d3+3xiEIriYSu1CnJEli4nNj2PpnPFs2JKJWq3h4Ygc6dGrq7tAEocESiV2ocyqVip69WtGzVyt3hyII/wpijF0QBKGBEYldEAShgRGJXRAEoYERY+xCg3L4UArfzl7O8WNnCQnxZ9TY/vS/rp27wxKEeiV67EKDcfRIKi8/O4XreuQyf0oYD98p893sBSz+6U93hyYI9Ur02IUG4/t5q3jwTi+GXV9eFjowQMs7k7U8/PwfDB3RFY1GXXGuLMvs3nmKI4dTCQr2pnfftphMeneFLgguJXrsQoNx8kQKndp7VjnWpJERSbGRn19Z2MtqtfPcU9OZPWU26rIN7NrwK/eMe5vTpzLqO2RBqBOixy40GOERQRw9UUhUROXG25lZNmwOFT4+HhXHFv2wGS9tGl9+GYNKVV7H5rfluXz87vd8Pv2Jeo9bEFxN9NiFBmP0Lf357OsC9u4vRlEUUtIsvPpeOsNG9kanq+zDbN6wm9vG+FUkdYChg/xJT0sXm3YLDYLosQsNRpduzbn/4XG8+env5OZkoNPrGT6qD3fcVbUWjAT8rVqvIDQ4IrELbnXmdCZbtxxFq9PQu29bgoN9atVe3wHx9OkfR2mpFaNRh1pd/ab0mr6dmffjauJae6BWVw7FhEWGExRUu+sLwpVAJHbBbWZNX8HKpesY2NtAnkXhwTlLeOjxWxgwKKFW7UqShKen4YJ/f9PYnry85xi3/yeJnp31nElxcuyUmrc/urVW1xWEK4VI7IJbHD6Uwto/1vHdtGh8vMs/hjffaGbCxB/o0q053j6mi7Rw+XQ6DW9/eD97d5/myOFUerTw5tk32mA06ursmoJQn0RiFy5Lfl4JK5fv4dy5LJrGRjFgUEKN5oFvWr+f4YNMFUkdyqcmdmqnY9vWY1x3ffu6CLuCJEl06NRUlA8WGiQxK0aosZMn0plw13tknllL2+hj7N+2lP/c8yG5OcU1audCDzAlqfpWeoIgXDqR2IUam/LpT/znTiMvPhXOqGFBvPtKJH262pk7e+Ult3Ft33h+W1FGfoG94tipJDO7Em106968LsIWhH8NMRQj1EhZmZXjR5MZ8lZsleMjh/rxn0kHgJsuqZ1WraMYOLQf4x5Yy4BrDZgtChu22nhs4q14edfd+Log/BuIxC7USPn0QQmLVcbzb7VXSstk9AZtjdq6697r6Ns/gW1/HiNQp2H6g20IDPJ2ccSC8O8jEns9k2WZndtPkrj3FL6+nvS/LoGAQC93h3XJ9Hot3XrGM2PeKR5/IBRJknA4FGbMy6HvwD41bi+mUTAxjYJdH6gg/IuJMfZ6ZLc7eHHSTOZ89Q0Bup1kJa1hwvi32bP7lLtDq5FHnhjJ3qM+jHsgmVffPceo8WdQdLHcensfd4cmCAKix16vlv++G8mazDdfxFSseBxwbRFvvDWfeT++hEp1dXzP+vp58Pm0x9mfmET6uXxG3B5GbLMwd4clCML/E4m9Hv25aS9jh3tXJHWAzh288TDmc+J4Oi1aRrgxupqRJIl2CY1pl9DY3aFcElmWMZttmEx6MZ1SaPBEYq9Hao0au6Pq5G1FUbDb5fPWNBFqT5ZlFny7nl9+XIfVYsHP34fx9w4T2+UJDZpI7PWoT//OzF+4kJ5dvDEay2eUrFqfj6TxoWlsqJuja5jmz13L3j/X8vWHoURF6DlwuJTJby/AaNLTo1dLd4cnCHVCJPZ61H9gPPv3nWDsfbvp1dVAeqbMiTMq3nhvghgeuASKorDyj32s/mMrFrOFTt3iuWlsrwsW/HI4nCxetJ5Zn4QRGV5e7iC+jSdPPuhg/ver6yWxK4qC0ylX2ZZPEOqaSOz1SKVSMfHZMZw6eQ37E5No1dmDl3q1RK+v2fzvf6svP/uNY4nbuftWX7y91Py6fCMTH03k06mPolarOHjgLIqiEBcfg1aroaTYguK0VST1v7RsbiI9LbtOY/1rCGjxovUUFZbSqEko90wYQVexqlaoByKxu0HT2FAx9FJD6en5rFvxJz/PaYSnR3nvNyHOk6cmpzJ96go2r99FWJCMhMK5bDWTXryDDp2aoDeYOHayjBaxlatZd+4ppnFsZJ3G+82MlRzavZGp74YQExXBtl1FvPHWLF5680Hi2zWq02sLgkue2EmSNEuSpCxJkg66oj1B+F9HD6fSsZ2xIqlD+cycLu11LFrwB28+682sz6KY+Vk0bz3vzVuvzaao0Mztdw/mxf9msHVnIbl5dpatyuXLWUXcNn5QncVqsdj4ffFG3nwhnMYxRlQqiR5dfHjobh9+XLCmzq4rCH9xVY/9G+ALYK6L2hOEKvwDPDmbZkdRlCrPI9ZtKqJ3dxMdEypX77aP96JnpyLWrzvAjaO6YfIwMnXeKjIz04ltFsXLb42jTdvoOos1P68ED6NCcFDV+u5tWpqYsyijzq4rCH9xSWJXFGWjJEmNXNGWIJxPXHwMTsmXud9nc9uYINRq2L2vhG17LNx1s1+18wP9VZQUWwDo2z+Ovv3j6i3WgEBvSi0q0tKtRIRVju/vSSyhUZO6HQISBBAlBf5VlKt4B2eVSsWb793P5r1+DLn1FKPGn+GNT8p4ctKtbNhqw2x2Vpxrscis2WSmU5fYf2ix7uh0GsbcOoDnXj/H/kMllJQ6Wb46jxnzS7jl9gFuiUn4d6m3h6eSJE0AJgBERgXU12X/9cpnZ2xgyc8byM0rpnWbGO55YNhVs2L07+w2Bx6eBgoKHRSXqBh4QwsGXZ9A8pk0Jjy1lzHDPZEkiR9/Kya+Ywe3ruS95bY+aLU6nnhpKbk5BYSGB/HYM7ddVauLG5pjR9NYs2oPdpud7j3b0rlrswY7zVhyVS/u/4diflcUpe3Fzk3o0ERZs/ENl1xX+GfTvvydU4e28swjwURHGli3OZ8PphTy9oeP0qxFuLvDu2TFRWVMGP8+40aqGTkkALNFZvqcLE6kBvHh5/9hy+ajbFy7B4Br+3WgQ8cmrF6RyMHEY/j4+3LDkC71Ws+mIL+Upx75nMYRZfTsYuDkGTsrN9h47Z0H6nR8Xzi/hd9tZPGPyxh5gwmjQeK3lWXEtmrHMy/cfFUl90Cv23critLpYueJ6Y4NWEmJhT9+38zCGVH4+5XPlR/Yx5/cXAc/LVzPc5PHuTnCS7fyj310iJMZN7o8ORuNap59PJxbJyRz+FAq1/Zuw7W92wDl73vio18SHlDAgN4enEs/ywtPbePhJ8fRu1/9jLV/9+1ausRbeObRyjH1hLb5fPbhQr6aNfGqSiZXu+zsQr6ft4z5X0VWPNAeOVTmrkf3s3d35wa5762rpjsuALYCLSRJSpUk6V5XtCvUTlZmAUEB6oqk/pf4Nh6cTTrnpqguT2pKBvGtqs4ykSSJti31pCRXXWz0689baRpZyPuvRXJ9f3/uuT2Ej14P5stPf8Rud9RLvDu3HmDEYN8qx/pe40t2RhZ5uSX1EoNQbteOk3TvbKgyS8lgUDG4v5FtWw+7MbK646pZMbe6oh3BtUJCfcnOlcnLt1dJ7vsOlhLd6Py9FEVR2LPrFOvX7kWRZXpe245uPVq4vYfZqHE4e3YlMnpE5bETp8pYtiqP26NLsNkc6HTlH+c9Ow5y71jvKjG3auGBn1cOZ05l0vxv49w2m4M/lu5m+9ZEDAYDAwZ1ccn71Ru0lJY6qxyz2RQcTtDpRHmB+mQw6CgplasdLylVMOj153nF1U/MimnAPDwMDB5+Dc+9nsbxU2VYLDLLV+cxZ2Epo2/pc97XfD11GV9+OJNWEceJa3SSb6bO5aP3Frl9Rs2AQQkcOqHl67kZ5OTaeO7Vk9z98GG6d1RzaNca7r71bZLOZAFg8jSRX1i1Z+50KhQWO9HqNWzedJjfFu/g6JFUnp/4NTs2LGFk/3x6tUtlxhdzmD3j0jflvnC8PZgxLw+rtTyhKIrC7O+yaNehpdjTtZ517d6cA0dl9u4vrjiWes7KkpVl9BuY4MbI6o7LHp7WhHh4Wn9kWebHBZv47ZcN5OYU0yauEXdPGErbuJhq5yadyWLSYx/yw4wYvL3Ke79ms5PbHjzLpFf+4/aHfpmZBcz8ainLft9JVKiVqR80IiLCF5VK4tdluSxcpmPqzKfYvOkIc6fNY8p7kfj7aVEUhXk/ZLN8g4GyMgthQVZiItWs2VCIxWJj/ZI4NJryPk5BoYMx9yTz1ZwXCA72uexYnU6ZD97+gd3b99GxnZFTSTY0+iDefO8+/AOunq0QG4o9u07x31dn07yJCpNRYneilfsfGsWQ4V3cHVqNXOrDU5HYrwLFxWZW/bGX5KRzREWHMeiG9nXS61u08E+yz6xk0qNVZ8t8NfscNn0v7r7v8uZgl5RYyMkuIiTUF6OxcpyzIL+UPbtPodVp6NwlFoNB9w+tVJr0xFRuGVJM756VY9iyrDDijjO8++nTREYFMHf2ahb/uIa2LfWkZ9pR6wNwOhTGDrFx0/BAAM6ezeWl/6YwalgENw4Nqmjr+TfS6NpvFP0H1r5m+9nkbI4dTSMk1Je4+Bi3D2n9m1ksNnbuOInd5qBT51i8fa6+OycxK6aBSE/PZ+LDn5PQxklCGx37Dx9gwver+eDzR4iIdO16AJNRR2FR9S/6giIIjK75WKTD4eSrL39n9fKt+PupOZNZRPO+MSTc0JhDG86xceYpvPHDqTix6IsZ8kwbIlv6V2vnuhataOLZquLPTocDna5qgpQk0OlU2O0OJEli/D0DGX5jd44cTsXP3wMfXw+efPBdRg5tAkBWto3Egxau7W5kxbrcKok9K8eJt4u+OKNjgoiOCbr4iUKdMxh0XHNta3eHUS9EYr/CfT1lCTcNlrhrXPkDv5uGw7cLs5j+5W+89vbdLr3WNb1b8/XUn9l3oISEOE8Ajp0sY80mK9PmxNe4vbmzVpN+aic/zY7G10fDoeRsXnu5iHOZMez44xBddIMxacqvk2vOYu1/D/POq5PR6yq/RGYalnBdi6rtduvVnkVLVtC1ozcqVXmC37arCKdiolHj4Irz/Pw9K2qup5zNQatRoSgKb32YzO8rsmnWRMvpZBtFxQqFRQ68vdT8viKPnAJDg5wCJ/x7iMR+hUhLzSUnp4gmTUKqDLNs//MQkx+pOh4+amgAU785XK0gVm15eZt48dV7ePaNOTSOykejkTh60smTk24jKKhm482yLLNk8UbmfhGKr0/5xyw4SMuTD3jz2Au/ECg3qkjqAAH6YM6Zkzh8fD/t23b+x7aHj+zKti0HuP+Js/TtZSA13cm6LXZeeuO+C24IHhkVgM7ow3Ovn+bYsQLmTwkmMEBDYZGDj78qoN+IAzRp7IdK68Ob790rtioUrmoisbtZcbGZt1+fz8mjJ4iK0HM62caNY/pz590DkCQJnV5DaZmz4mEmQGmZE71eUyfjtZ26xDJ/0cvs23MGp1Pm5Y5NqoyLXyqr1YHVYiU0uOpro8KNWKyleJzno6dGg81mvWjber2W9z55gD+3HOVA4mkCo32Ydn8CAYEXfigpSRJPPnsr429+janvBtIoWocsK5iMOiY+HMiKDek8+sx9JHRoLMbBhaueSOxu9sn7i4gMSOWjbxuj1arIzbPz2PNriYgMZsB17eg/qAvTvtnNy89EoFJJyLLCtDlZ9Lvun3u1taHXa2u904/BoCUyOoTtu4vo3rmyt795Rx7RES05ezyDaCUWtVQ+p9viLKOQXFo2u2hFCgDUahXXXNv6vGOmsiyzfetxdu86jpeXiYGD2hMeEUBcfAxqtYroSD2KokKtllCpVYSHyEiSk4gof5HUhQZB3G+6UXFRGbu2H+DR+0LQasv/KQL8tTx4lx/Lf9sEwD33X09mYThj703itffSufm+ZM5mhXLfg4PdGfpFSZLEvQ+M4PUPcvnl9xyOHC/jx5/ymTKrjFGDJ9CuY3v2lm0hqeQ4p0qOkGjdyqjhN+Pj5Xvxxv+B0ynzygtzmDd9LlE+iTgLtvDohPfZuL58Dxid3sjS1aVotWrUahUSsGl7GQ6HmsBAbxe8c0FwP9Fjd6OSUiseJhUmU9WViMFBOgoLyxdTmEx63v34AY4eSSXpTBaDRgfRuk3UVdGz7Nq9Oa+8/RCLvl/HouWZFHkauf+2NwgLiWD8LQ9wpONBEg/uRqvV0qXDPURH1L7i5NrV+ynNP8nsz2PQaMp/RoP6l/HoCwvo2v1VRo7uzbc/raS4WKFnVyMnk2x8Pa+QvoO6X3B8XhCuNiKxu1FIiA8qjUeVWSgAK9cWEt+hcg61JEm0ah1Fq9ZR7gizVtq0jabNm+MB+Gr3z4SllM/ukSSJ1s3jaN3ctUW5tm7ax6ghXhVJHaBlMxONIlUcOnCWR54cTmZmLktWH2TTjmLyCxTaJsTxyuu3uTQOQXAnkdjdSKVS8eCjN/H8m3O5fXQZjWP0bNpWypadKj75qq+7w7sqaXVaLBZnteNmi4xGq8Zg0PH+Jw9w8kQ6yUlZREcHXVXliwXhUojE7ma9rm1NcPCjLFm8ha2JuTRv3Y4vvu6Bn7/nxV8sVNNvYGemf3aQ/r39KmYSrd9cQFGpgbZxlSURYpuF1Wt9dkGoTyKxXwGat4xg4nNj3R1Gg9ClWzMO7r+WMfesp0dnIzl5MieTJd54d4IYQxf+NURiFxqU8tk4NzB4WFf27DpFJ28jXbu3QK/XXvzFLuR0yhw9koqiKLRsFYlGI0r1CvVHJHY32p+YxJwZSzl25CyBQd7cOLofI0Z1uypmvFzpwsL9GTK8et2Z+pC47wzvvj4XHy8bkiSRX6jlmZdup0PHuilTsH7tgfI9bXMLaRMXy6139CcyKrBOriVcHcS9qZscO5rGGy9O46ZBxSz/vhGvP21izdIlzJ+71t2hCbVQXFTG6y/N5IXHTXw7NYZ5U6J5ZaIHb748i8KCUpdfb9EPm5n39XfcfmMZ70/2pEnwcZ58+FPSz+W5/FrC1UMkdjdZ+N0a7rvNk0H9/DGZ1LRp5cHbk8P46Yc1WK12d4dXJ9Ra1wxHKIpCQX7pFflz2rj+EJ3bqaustu3cwZuenTSsW3vApdeyWu18N3c5H70RzrU9fGnSyMhd40IYeb2WhQvWu/RawtVFDMW4SfKZNCbcXHXmS1iIHi8T5GQXXXZJ3rPJ2fz+2zZyMnNp0aoJg4d1qlXtdlmWeezDL9n501nKimyEt/Sh521NCG5U81WaDkVmb9RHNXpNB6CJ56iKP2/78xhfT/mFvNw8ZFlF34GdefCRYZdcy72uFRebCQqoPpQWHChRVGh26bXSUnMJ9IPI8KollXt19eLdqWdcei3h6iISu5tERoVy8HA6sY2NFceysm2UlHHZO+zs2nGSd1+fxcjBBjr00LF5+xkevn8zn0597LKnT3712Qp2zsgkwtaDEA9/MhNTWXb4DM899RphwREXb+BvZhqW8FwvTZXa6jVx7GgaH739Da8+E0CXjk3JL3Dw0dREPnrXwguvXBkLjNp3bMp/X1rOg3fJGAzlN8Q2m8zazVYmvtTEpdcKCPAiO9dBaakTD4/Ku6GTZ8wEh1Sdm5+ens/e3afx9DLQtVvzen+YLNQvMRTjJqNv7ce0uUVs2lqALCucTjLz0lvnGHrjtZdVTVGWZb74eCGvTfLngbtCGdjHn9eejaBXJxvfzVtLcVEZslx9Q99/Ulxs5oe5f9LKMx5PjS9alZZIj8YE2cNZtW5ZjdpSFIWSjHySTuTUOI6/LF60kfFjPejaqXyjan8/LS8+GcaubfvJyS6qdn5mZgFzZq3m4/cXsWL5Xmw2x3lada0WLSOI65DAg0+fZfnqPJavzuPBiWdp1rqty7cW9PH1oHuv9rz1yTkKi8rf28HDpcyYX8SI0b2B8p/7rOkreOTedziycwl//PQDd4x9k6NHUl0ai3BlET12N2kbF8MzL93L118v4ZnXjuPn58GNowdyy229L6u9rKxCLGVFdOlYtd5Kr65a7n1sKauWb8Zk8mDc+OsZOqLrJbWZkZ6PDgM6tQ7L3477agNJSUm65NjOZaay4JePKVSdYc4KNTNt65j4/G3n3czizOlMfl64gdSz6cQ0ieKmsdcSFV0+wyMrI5tm/YxVzjca1USE68jOLiQwqHJ4aM/uU7z1ykwG9dHTOlLDhuWJ/PbzBt775EE8PAyXHPvlmPjcGDasa82atbtRFIVhN99AvwFxdTLb6bGJo5jymYaR43diNEioNSbue+hWEtqXfw527TjJ5rXr+XFWDD7e5b/uG/8s4I2XZjN34Yui7nwDJRK7G3Xu2ozOXZ9CluVaL54xGfWYLTJWq4LBUJ5A8vNKSD9XREJbA99Oj+XoiTImv/0LBoOeAYMuvjt7aKgvVsWC3Vn1IWWhPY/w8EurW2N32Jn53Rtc293M6TQVWkmhVZyNNybPYPrc56ps4HFgfzKvvzCNcaOMjOhjYk/iIZ56aDf//eAhmreMoGmLRmzbtZeOCZVDVTm5dlLPOapM75NlmU/f/55Xn/GreIh503CFyW+n8fOPW7jjrv4XjdvplFn80zbWrdqGw+Gga88ExtxyLZ6ehopr5OWW4O1jQqer+mukUqno2z+Ovv1dWwfnfAwGHU9NGs2DjwwrH98P8q7yWVq7ahe33OhZkdQBru3hy4z5Zzl4IJl2CbUvvCZcecTX9RXAFSsivX1MJHRoxfQ5mchy+b6lqSkF/Ly0hDE3lm8X17KZiUmPBLLoh9VVXmuzOSgttVRr08vbxIixnThachCzswRZkckwp5KlTmVg30srG3z42H7M5ixW/GajeG8MJfsb89t3CmWFJaxcvrfKuTOmLGbiQ97ceUsI7eO9uPeOEB4c78E3M8qHfUaOvoYlqx3MWZBJ6jkrO/cUMfHlNIbf1Bcvr8qefGpKLoqjhG6dKnvwkiRx01AfdvyZeElxv/36d+zauJRHxss8/7CWvJTNPPvEV9hsDpYt2c2QPm8xasAHXNfjdb78eBlO5+UNL7mKyaQnJMS32mfJZrNXjPX/ndEgYbdVr6kjNAyix96APDlpLK+99A033ZVETJSOlWvSmHBnGMOur+zNtmhmIuNcClA+5/rLT39ly8Z9KIpM09hIHnpiNC1aVj4UfWLSMI5ZT7Nl7m5OlNhpFNWER0Y+TWTYpY0XJ6edIfucg+sjuoEKtGoIU4JYe247iftOc9udfYDyHvCRI2fp937VDT49XontAAAgAElEQVQG9PHj42knAQgL8+PjLx9n3uyVLJp0HF8/L4aOGskNQ6tu2q7TazBbZGQZ1H+bYVlS6kSnv/gMoZMn0jm8/yA/zW6ETleeFNu0MvHIsylMm/IHi2btoo0xAV9fX8x2M7/O2g/Aw09eeTXyu/WM55efjjGon39Fxcvjp8o4fVambbxrx/yFK4dbEntOWQFf7//VHZdu8No9HE0v7SgyMws4dXYRvXuZKjZ8Bti+u4gmsREoisKrL35DbEQWv30bg4dJzcp1+bz49FSmzHqG4ODyIQy1WkXPm5vRfqiVPZmxSGoVGznBRk5cUjyntAfRWfyQpfLbQ6eigKKgKw3iUFZyxedAURRsGjsHk7KJCK98eHzyjAWb3lbl8xJ4owd9b2wPQBrnmHHgt2rXtfmo+fr7ZEaPKt+4w2yW+WxOBj5dm1/0s3dwfSqt2jopchbB32Yotmvv5L3PVxPjaI1dhuyyAgBCNTF8PWsVqmutaLT/fPcVGxJA35Be/3iOK/UbEM+m9Xu557ETDOprJDdPZulqM489Ne6KmSIquJ5bErtS5Ie84mZ3XLrBs/T7jtjW5ZULbZYRvPLud0x8yEnrFiZ27Svhs68LeO6VMZw4do6cjBS+ertRReK/YYA/R45bWbZkB3fdO7CizfGthnK65AiDWtY8nu3OID5efIL0czb8fFVIipr8QicW2UFLZRDyin4V58YGe/HZR+uZPDEQD5OGomIHn39SRFP/MVXOuxS9mg5k7pwvWLGsmJhILTv2mgkO6kvz0jHIK/45+XqcPcqZjFkohVVLEpw+YsaeJ+GhiwBn5fRRPaCYvTEvGYyHx4U3/T5lLYHhS+kbUqO3UitqtYpX/zu+YqtAT28Tn05rf9nrJISrg1sSu8mgJaFZqDsuXW/KzGUsXfkz23dtQVGgc8duDL9+NCajR51e9++j1n0HxKM36Ph2wSpSUjJp0jSCF16/mYT2jdmw7iDNY/VVevMALZvp2LI/q1q7lzv3PGJALPMa7cWcbsVeYEAjazCrHEgBCqMH3YCPt1/FuXFNxvPLcoXb/7OeiDAdael2EtoMYvh1t1zGc4hQeiV8wfHTRygqLuCRu5oSHHhpn7m4pkF8MPUXfl2ew+ihIajVEhu35bFlB7Rt1ZaSk7kE6CvjLrTll089jG9GUXEhZ8+dwdfbj6jwRlVnwpzIqOF7cA2VSkX3ni3p3vMyvpmFq5IYY68Dsizz6VfvUJpSRnNDAhIShzYd4sTJo7zw1Buo1fX3Y+/RqyU9elX/hW4aG8qXH5mx2eSKcWSAHXssNGnpurFXvV7LZzPu5YUn53PyVCZGRwAak4aH75xYJakDqNUaRg+9j0F9xpKbn0NQQDAepsuvS69SqWgZ26bGr5Nlmb49b+b7xT/xzfcn0GpV2Kx6CvLUpJYeoLi4CIu5jEjfxhTbCzkrH+fm0XeyZNV37Dv0B3GtDZxNtaOSIhg/dlKt93EVhJoSib0OHD15kNy0HBI8elb02Fp4tCMxcysHjybSrk1HN0cIkVGBdOiSwNOvHOQ/dwXi7a1hyR/57DmkYcJE18YX2yyMH5ZM5L+/zqVF2p1EhEWjVl24boyXpzdenu7ZWPrg0X3MnDcVrV2LrCgoem+6dunJ5vUbaKFLwNfXnwxdComF28m2pNG4USz3DXyYMnMp2XkrWTijMd6eGhRFYeb8c/y4ZCr3jXveLe9F+PcSib0OpGWk4iH7VrkNlyQJT6cP5zJTXJrYLVYLx04dRpadtGhas97p08+PZeGCUCa/vxVzmZXO3dry8ZTralVb5kIkSSIwyotortx50/mFeUyf9TktVAn4GsrHoLMt6fy65Ec6+PbEV+cPKISaovDUenNCvZ+nH5mMSqVi6tyXeXC8P96e5b9SkiRx59gwFv1+kKLiQry9Ljz2LgiuJhJ7HQgKCMasLql23KwuIdDfdU/ODh/fz/TZn2NweCAhUaoqoufjgXCJ3xsajZpxd/Rl3B1if1WAXfu24usMrEjqAEGGMAx2T/JLcrHnyciyjE6nw9fHjxJrCXaHHb1Oj9VWhq931V8nrVbCZFRjsZpFYhfqlVigVAfatkhA76vhVOkRHLIdh+zgTOlRJG9IcFFvvcxcyrSZn9FMiSfO2IW2xs60VnVi9efHycjId8k1zqcgv/S8i5kagjJzKRq5enEsnaQnpzQLIya81D6o7GqSsk/hYfJApy2fMtg0phNLV1f9ue89UIQsexPoH1wv8QvCX1yS2CVJul6SpGOSJJ2UJOk5V7R5NdNoNEx8ZDLBbQPYZl7DNvNq/Fr78PSjk9FqXTN3OPHQbjydvvjqKnuXXlof/J0hrF293yXX+Lsjh1N4+L6PuOvW17h15Mu89uI35OdVvyu5mrVs1pZ8dQ5OpXJFptVhJU/OokRVSK6ShU2xUkwBZ5Xj+Pn7Vwy39e0xhD93ejP5nWRWbchmxrdpvPRODsMGPiD2WhXqXa2HYiRJUgNfAgOBVGCnJEm/KYpyuLZtXymcspODR/Zx4vRRfH386JzQvdqMjv/l6+PHg3c/gcPhQEFBq3FtmVSr3YpKqf4AUi1rMJfZXHqt7OxCJk+axtMPedL/2qZYLDKz5p9l8nMz+Hza4y4vbmWxmMnOy8LPxx9Pj8srYXw5mjdpRav41iQmbiVEikRWnKQ5kzCZPGjv3YOThYdJsZ3ApPEk1qM1sr2yWqSnhxeP3fMW2/ZuYvGyw3h7BvPA7f0ueYqlILiSK8bYuwAnFUU5DSBJ0vfACKBBJHa73can098l40wGPs4AbCorvy/7hYcfmEizxhefF6zR1M1jjNbN4vhJWoDNaUWnLt9owSHbydNkuny+8vLfdzHwWi0D+5Qv2DGZ1Dx8Xwi3Tkjm0MGztI2Lccl1FEVh6aqfWbF6KXoMWBQz3br04paR42v1c0zPSqOouJCI0Kh//KKQJIn7bn+EvQk72bVnG2q1moEJg/hm3lQ8NF50De5TcW5S6TEioqu+b4PBSJ/u1wHXVWvb7rBzLuMM2jNFKHGK2NdWqFOuyDoRQMrf/pwKVKsLK0nSBGACQFBAPS69q6VN29aSczqXBFOPil/GbEs6s7/9ijdf/Mhtt9nBgaFcN3AIq1f9QZAcjoREtnSO5jf4o48u5nTJkQu+tqaLjbIycohrqmCVq46tx8RA4pkDmBqXXdZ7+F+bd6xj7R+rSTD2xKA2YpdtHNi6F6NhITcNG1fj9opLi5g2+1POJiVjVJkoU0oYNHAoQwaOvGBiValUdIzvSsf4yo9wWr+zrF+5hqa61nhovMg0p5GlSeOefhMuKY4DR/byy/Iv8fK34DhTxL0L03jxtbtoGit680LdcEViP99viFLtgKJMB6YDNGvSstrfX6l27dlOqDq6SiII1IeSVHiMzOx0wkJqtouQKw0bdBOtW8Sxc+9WZNnJTe1Gs7nNad7d8s9V+zoE/cx1LVpdcoJP88ghe2sRna+pHH6y2WTW7DAT3sKHDZsvdQOLZrT/h79dvXY5jbUtMajLKzVqVTqaG+PYsGUNNw4eW+OFXXO+m07x6TK6ePRFkiSsTgurV6wgPCySDnFdLrmdodeNIq/RFvYt2URpvpXwBF9uuLUxGY1/4GJrSQuzyli5ehsfvh1MyxbeqFU+bN/sZPKk6cz54QW0WjExTXA9V3yqUoG/F+eOBM65oN0rglqtQVaqJ0oZGbXaNZsz10bTRs1p2qiyImJLa9uLvmZm9hKua1H1WFmZle1bj2O12uncpRkBgZVDFq2viWDu94V8N72Y4dcHUFLqZMb8POIjBjDO5w5w0SSZouJCojVVh5H0KiN2sx273V6jxF5YXMDRY4fo4tGv4ktZrzYQpWrK+o2rapTYVSoV7QZGMfW5xy/5NX/5ZuZqbhsSxDXtK3vnNwyAX/9IYef2k+ddFVwTxUXld0sXW3tgtzvYsf0E+bkltImLpnGTq+euWag5VyT2nUAzSZIaA2nALUDN75uvUD179ObHM98RKIegVpX/uNLMSQSFBl9VQ0r/ZM+uU7z5ymziWqoxGSWmf25h3F1DGX1zeRVCnUFDh74PkZKWyOMvbkevM9Cu9S307lF9LLk2mse2IvNQGjEesRXHcqwZhIaEodfXbNcjs7kMraRFLVX98jWojeSW5Lkk3ktRVFRC06DqHYCQIDWFhaWX3W5aai6ffrCQo4fPAArNWzbm8afHVuw29Xdnk7N54elphAVZiQxT8+0sC526deCpZ0eLGTsNVK0Tu6IoDkmSHgFWAGpglqIoh2od2RWiS0IPjp84zM6dG/AlAKtkQeWl8PidzzWIB2Bms43/vjqb9yb70T6+vJeelW3j7sd+J75dY5r/f212vdGTUYPHA+PrLJbhg0fz/onXcZba8dMGUWQvIEOVzIMjn6jxzzooMAStSUu+NQc/XWWyy7Cl0qHtPw0IuVZCh2b8PG8nY0cqFQXXioodbNtl5rYHLm8VrtVqZ9ITUxg3QsVnr5VvkL14aS7PPjGFmfOfq7JnrqIovPvGt9w1RsXIoeU31haLzMPP7mPF8qbcMMT95S0E13PJAJ+iKMuAmu1ufJVQqVTcefMEBvQZzOnkE3h5+tCmeXydzXapK4ePH2DJsp9Iy0jBFqJi5/MdaHJ9K3ZuP06rWFVFUgcIDtIx8gYTa1fvq0js9SEiNIoXJr7BqvXLSEo+TWhIOLf3vYPoiJonQLVKzbixdzPrm6kEl0RiUnuSK2eCv4MBvetvQ4yevVqxdHE0jz+fwsghXpSZZeYvKmLg4GsIC/e/eAPnsWnDYZpE2bnlpsiKY2NuDGLbnlQ2rj/EoBsqv7jSUnPJy8lkxODKn6HBoGL8zb4s+H27SOwN1NWVndwoPCSS8JDIi594BTp8/ABfTf+EGKkF7fW9SM44y6dPbyJAFQ6Ub5P2v4xGFZl59mrHL4eiKJw4c5TEg3vQaDR0bt/9gjswBQeGctvoe1xy3YQ2nXjmyZfZuGUNuXk59G7Rh15d+9Z56eS/U6tVvPHuvaxcvofFaxPR6XTc+eAIeva6vDLIAJkZBcQ2qj6806yxioz0qqtf7XYnep2K/73hMehV2G2u+fcVrjwisdeBktJijpw4CECbFvH1mkjO59elP9JIakmIsbz37WcMJkSlYepHK5jx3UN8+bGVcxlWwkPL58NbLDK/ryrj/kfLi4pZSu2cOrSVbw+vJzwskq4de11yOV1FUVjw82x2bN1GgByKgsy6tSu5ccQY+l1zfd284b+JCo/htjGu+aK4XDqdhqEjujB0xKU/sP0nzVuEM2uKjYfkyuEdWVbYttvObfdVvcOKaRSELHmwbVdRxcbesqzw42+FdOs5sFrbQsMgEruL7dy3hd9Xf0Xn9noUGX5fbePG6x+hfVvX/FJfjrRzKXTW96lyzN/ox+Gz+/H0MnD3hBu59/HFjLjehIdJYulqM83btKNTl1jS0/OZ9cifSGlhOHVBHJWOsnzVbzzz2MuXtKryVPJxdmzdRntDTzSq8tW3Ec5G/PzrQjrEd8XX559X8ArVdezclAVeEUx+O5XbxwQgSTB/UR4qfShdujWrcq5KpeKp58bx6uQZ9O1ZSnSEmrWbraANY8RN3dz0DoS6JhK7C+UV5LBs7Vd8/WEEMVHlc7FPJZXy0LNf0DTmc7dV+AsKDKYgL49AfeUsniJrEUEhPmg0aoaP7EZcu8asWbmX1PQifAPL2PVnKg/dPR2z1YpPfjihhgR8jeU9+qTS4yxc/C3jb5nAr8t/ZM++HajVarp3vYbBA0Zi+NsMlv2H9uAvh5QndUWh1FyK2WJG7dSycftqhl83pl5/Fk6nA4fTiV6nv6TzZVkmI/scuaoSlA5XxopRlUrFf9+/j+/nr+fl93ehKHBNn5488nyf885ySWjfmK9mP8uqFXtJzi3kxlub0POaVmg07p+uK9QNSVHqf61QRHhT5eH736v369a1vYnraRS5gicmVL0dfvuzNHILhhHXpkedx3C+LQd3JW5j3tyZNNfG46P156wzlUwSGTqxOT2HVk4tLM638MH9azDmBhJsCMXisLAvZT+RflFEqTtWJHaH7ODP0hUEB4WiytESaWiKrDhJtp4gINafp/7zQkUCXLJyEbv/2EOsRxuycjKwW+1o0HJcTsThZeWeOx+iZ+fe//ie9rlgSzmb3cqOnb+QnLIdRXHg5xtDx/ZjuL5XtUXSFZJSTjFjzpeUFJZgUecT2ySYNz8aR4t6fKAsCH8X6HX7bkVROl3sPLf02CXvfFSDfnDHpeuUbDuNSVuK5FN1nrRnQCnZEZtRDUq5wCtdZ6/dSfuUp6oc69SuG45xDn5d+iMHCvJwqHX4XH8tu9QJ7P6jsgd6bt1mSA0h1NQGq7V8SXETSeJk/gFaRnSuOM+pOLA7bdjzHMR5Vs6qaK3pwO7TGzmdfKJi0VSnhO6sXLkMv5Jg7FY7JpUnxXIBVpWZLh69+WHRXDrGdcFgMJ73/bxTuJ1B/bag1taud7nhs93Ety7jgw/98fJUs3FTJu9NeZPtUY/ySkz1+fhl5lI+++o9Im2xtDSEU6bLxJ5exqP3zmTxqmcxmS6txy8I7uCWxB5o8uX++BHuuHSd6mtMZ/Izn/LoeE+8vcp/tHn5dnZsz+WjKePOu3jE1b7a/fN5j3fr2IuuHXpit9vQanXnHVL4IGUlWm00gerKh716r2DOFmopthfgrw9CURROm48QHhqJIbvq9nWSJOGFH+cyUyoSe1hwBGNH3870WZ/jKfuiUlSUSkUkBHbDR+eP0eLJ6bMnad087rxxB4Vk1aj8wfkkJ2WxNmcrb3/UDI2m/H2PucGPpGQncxP3wv8n9tz8HNZu/IPTSadwOK1oSw2EeEdUvLdI70hyCzPZsO6gmCYoXNHEGLsLxTYLo//1vRn/8EaGDzLidMJvK8oYdtPAOk/q+XklHNifRMrZPOI9HOddfi9JErq/jS2fSjrO8lW/ci49jYiIKNRaFaXOYgKpHM7x8vLGWWbnqG0v/nIQJUoRkU2jSGjXiTWLV1VpX1EUSimqtkvUNd36cfTEIY5tP0aoIZIgQxgalRZFUbArVowX6K27SlpqHrGN9RVJ/S+tmuuQtuUCkJmdzrufvIqPORA/bRAniw9BmYRFb8agr4xP4zA0uDr0QsMjEruL3ffgDXTv1YaN6/YjSRKT32pHy1Z1O//9+283MeXDlXjhS44ljz8NT/LohGeIuMBccYAjJw4yZdrHRCpNiNG1JP9wDsnycZyyAx9bAL46fxyyg5PmQ3Tp0oMRg0eTnplGSFAYMZFNsFjNrFi1hKSS40SZysfYk8zH8Q31oUXT1tWu1/ea6ziQmIifPrBidsw5czImPxMxkU3q7GcD0LhpCIePWTCbnRiNlUM6O/dZwC8MgN+WL8LfEkpjz8oiOnvLtpKTl10x594pOynS5BKf0KhO4xWE2hKJvQ60aRtNm7YXTqqutD8xianvr6GjRw+MWiNZ6nwKC6x88fWH/Peljy9YC+SnXxfQRGpFsKl8kZKn1hutWUuO1zmSHEewm204FAfx8e25fex9mIymKitAjQYTTz86mQU/zWHrsZVIkor27Tpxy6jx571mbKMWjBgxmsW/LcQTH+xY0fvqeOT+Z+q8XklYmB/dr+nIpFcTeeieQPz9tCxblc/abQ78BicAsO/gbowWL/ZZthJkDCPUEImv0Z+jZXvRlKmwOQpIth6l28DG9fZvKwiXSyT2q9zSX3YTIkdi1FYOF4SZoskoPsvpsyeIbdSi2msURSElLYle/zNuHaQP42ThQV559h3WbF6B1WwhPq5Dxb6e/ys4MJTHH3gWu8OOSpIuWn2x/zXX07VDT04kHUNxyrRpEV/j4l6X68lJo1m4IJiX3ttCaamFjp1bc8+bfsxPNLF+yyoKcwswOf0wSAZOlx0nVZ9EW79ObNWsQo6xUuqRwmP3DmPw0E5XxJRHQfgnIrFf5UqKLWhV1bfd06DFaj1/PV1JkvDx9qXEVoS31rfieLGjCL3OwFsfvIy/MwSdYuDI3m9Z32gVjz/w7AX3a63Jtn8Hj+7jp18XYCmzIKklru3Zj5FDbq5xrfWaOHM6k+lfrGT3tjMEBHpyxz0DGTaiM+uztuDYZuanxd/R2as3liIrRslEEOEctu4isWgrg/oP5ZZR49kb9RHDOrpvkZkg1ISo2XmV6z2wNVlKOn9fj1DqKKZMVUyTmGYXfN11/YdywnqAMkd56dgyRwknbQcpNZfQQpVAM4+2xHjG0s7YnezTOWzZuaHWsR48lsiC7+bS2NaabqYBtNP0YMf67fyytO6mvqal5nL/uKmcWm2lNZ3wTI/mk8krmTltNQAlKWl4Sr4Eeofg6e1FmVKCWSnFU/FF76fjpmG3ujwmWZbZse04X33+BwvmbyI7u9Dl1xD+3USP/SrXb0A8v3ffzc6tWwkijBxLPkVyKTffPB6j4cKbL/TrNQir1cLKNUuhDCStROee3dizdRe+uoCK8yRJIlQdxd59O+nTo3a1RVasWkK0KhYfXXlVQ4PaSAtjOzZuWcPw60dXmbHzl+zMYFYe2wJceKu/f7L+m2OQ74W3VwAlNiugJkIdy+efL+WeNj3wtLSmQCnf/NvX2w9vTx8cDjsWSwnNO7SvcpdyoamkNaHTaTg8PY+9G1PxcQRhV1uZ9vEqPphyJ526xF68AUG4BCKxX+U0GjUffXk3mzYcZvO6IxwpzeKBZi8TFf7PG0yrVCqGDBzJdX2HUlxSiJenD1k5Gezcth1Fqbp03q7Y8dRXT7pl5lLWbV7J3sRdGI0mevfqR8f4bhccg87JzaaxpuqMGYPaCFYVJWUl+J8nsT/n0xVSLrw69GKW75tMGEGYbJXTTU2At/0s4fvH8kLPaF5ct4pzxWcJN0ajUqlwqOzkatLp0aWyeNj/Lvq6HPtOZLDf+BGH1hfR2a8HKqn8hjmnLIeXn/me39Y8/69d5q8oCon7kjh9KoPIyAA6dYkVm4DUgkjsDYBGo6Zv/zj69o/jq90/E5VSntTNljJOnjmGVqsltlFLNBoNNpuVMnMZ3l4+qFQqtBot/r7lSS88JBK/IH/SspKINJXPgLHLNtKVJO7qVnXjZovVwvufvY4t00moNgqbbGX+6Tkk903ipqHnH75o0jiWrMQMPLSVtd+L7AXojDp8vH3P+xqApJTT7Du4C7VaRYe4LkSERaMoCoeO72fbjk3YbHY6dehCx/iu1cbqQ0PDyDiXjZ++MrE7ZDsW2YKfjz8qlYpHJjzNF9M/ILP4LBpJR549i4iwKL7/aR4tm7eiT6/r8PG6cHw1cXxDFmGaZhVJHSDQFMiZIoljR9P+lTNuysqsvDRpJiUFqXSM17N+uY2Z0/x558MH8PO/tCqiQlUisTdAsizz6x8LWfLHz3ipfdGqtWB00iy2JQcP7QdZwuhhYPSNt9GlfWX9GkmSeOCux/hs2nvkFKWjw0Chkku/foOIb92hyjV27NmCJctGG4/KWSIBcjDr1q+k/zXXn7dq4+DrbuS9w69BKQTpQym2F3JWPs7YUXegVp2/p7p42Q+sXbuKAGcISLBq5XKGDh1FSWkxm9auI4Qo1JKaHw8vYFfrbTx495NVenoD+tzAB/vexGTxJFAfilW2cMJ8gC5du+HlWb5yNiI0iv++9DGnko+z7+AuNq5fiz7TC6PGg91nd/Pn9k288NTr+HjXvhKlSiMhK3KVY4qi4FTkf21vfe6sVYQHZPDKOzGoVBKKovDljAy+/PQXXnrtDneHd1USib2BMRfbePXdSZw5dhp/KZgSCvHQe0GJwq6UnXQL74dBY6TQmse3387Cy9ObVs0qN8AODQ7njRc/5NjJw5SWldAkphkBftVXzR4/cQQ/KajKsItWpcNb5Udy6ml8faovuY8IjWLS4y+zdOViTp85hH9oAPcOfIj4Vh2qnQuQci6ZtWtX0cHQC62qfKw70tmExb/+gIxCN4/+FcdDlSj2HtnC4RMHaNuiXUUb0RGNefD+J/j+pzkcz96PWqvmmt59GTnklirXUqlUNI6K5auZn9Ba16litlCAPoTjRftZtWEZo4fddqn/DBfUql8oew4lEy6Hofn/PXTPFZ/DN1RPs+ZhtW7/arRhzS6+fCewora8JEmMvzWIwbfsx+Fw/mu/8GpDJPYGZsOcE5jPmoiTumFUm1AUhWOWRPKUDOJU3VAcCmjAR+dPpKMpK9f8XiWxQ/m2cheq3fIXPz9/0uSqVRcVRcEsl/5jeeKIsGgmjH/skt7LwSP78HcGVSRvKB+T97D7UEZJleMqSYWfM4gjxw5WSewArZvH8dpz72O2mNFpdRfc1jA7LxPZquBtrDrsEqQL5/CRgzDsksL+R03aB1I6xMK631biSyB2lQWbRyn/mdyLDdl/1v4C/yM9qYCD29JQq1XE9YwkKNzr4i+qZwWWIqwYKLY7Ko5ZFBmzw8y6jM1oalkArqGI8Qi4+En/TyT2BsThcHJyazZt1b1x4ESh/CFosCqCXHsmGrUOWa4cBvDS+nAu99RlXatXt75s2LQWf2sQ/vogZEUmqew4AWEBNIpq6pL3o1KrkaXqZaUVFciKs9pxh2THZDr/TCBJkjAZLzxLCMDD6IldseGUHahVlb8aZkcZPt61r6Wf0CyUdw71gpYQEJBDSUoqGoOBoGZN+fmkFk7W+hJVZP+5GfXxrQwZYMLhUPj08WOo2/cnoMP575DcpTAggdnfn+C+eyqLyi3+LRtLQAumrGnpxsiuXiKxNyBWqx1zoZ1ScynIYHPa0GsMGDHhwEapUkSovvJ2P9eaQZO4C891/ychQWFMuPdR5i2YwanSQzgVB42bNOWe25902crM9m07seT3nylzlGDSlD9EK7IXYNWXYtQZyDCnEmosr8NTaMvj/9q78/ioqruP45/fTCaTPSErgSQssvXagPgAABSbSURBVIOEfZVFRaBqUYtrRWhR0KqP9tHW1mJtq9LH1rWtdWu1RQvYal1wqQgVRGURhACBEIFASEJCIAlJSMJklvP8kYhEAknIJJMZfu/XK68Xd3Lv3O8MyS9nzj33nNKgQ4wedsFZny8qMppBA9PZnbmD3mGDsIqVatcx8s1e5k2+wyuv6efR9SN8ooE2HN1YUHiAV/Zs4ZVn+xETXXcD2S1XOPjBXZ9z67Dr6BR9dgtpt4XK6f15/tVf8/u8I4waZmN7loutmXYWzLqPhOikpp/gHPJpM/fTwh5A/vrsSizuII5JOUmWVFweFw5XDYckD2uwlYP2HKJro4mwRXHYUcgReyE3T7n1rM83qG86//fgHzh0pBB7cAixMc3/qNgcifGdueHa2bz2+iKiauNADJWWMubO/hFxsfE899LTHKzch1WCcNpqueWmO4iPTWjVOWdfP4+/LX6OL3Z9TIglDKc4uOLKqxnUb4iXXlX72Ja1mUunhJwo6gCdE+1MHhdC5q4tTBh9sQ/TNRQZEcWPb3mULTs2kbE9l/jYLtx76+nn6FdN08IeIBwOJ2+9toH0hKFkFX1FtaeSMEskpZ5iyoNKuP+ehxCE5SvfI6csj179+zB32jySE1u3GpDFYiEsJIySsiME24KJCPduH+74UZM5v/9Qdny1DavFwsC+6ScW0l74wFPk5ufgdDnpkdarRVMbnE5YaDh33PITSo8eoaKygs6JXRos9ecvLBYLbvepn5ycLgi1d7w+a5stmFFDxgFtv8rYuUALe4CoOnYctwtiQjsxMXk6BdW5VDsrSZAkbHYL3VJ60ik6lvSB3lsgwulysvj1l9j05QbCrBFUOI/Sp08/pkz8DgP6DD7tRcqWioqMZuzwCac8brFY6JHWNv0ZsTHxJ8b3+6MhA0fylyX/4urvOkhKqLvxKzevhs82OLhn/lAfp1NtTQt7gIjpFE5MXBjlh8tJtiSREtadjJL1HK4pItIexS8fvpexoydw/cw5jY4Zd7lc7Ny9jbKjpXRL6Um3lB5N9pUv+8/r7NqYxYjQSew7lk1RxUGy12eTk5lDWFwod9923xnnhFdtJykhmYmjb2LO/7zKheNDcLrg0/UOLrv4Np8tqq7ajxb2AGGxWLj755fyk7teweOyU1SdR211LQOtI0iOS0EskLFuI0lJnZky8dIGxx4pPcxTzy7EWe4mxBNOhZTSZ0A/5s++67Stbo/Hwyeff8zg0DGUOUvIr8hliGU8FizUOhxIleG5l5/mofsf11vDfWTimGkM6jeczF1bsdss/O/8oV67g1Z1bPobF0CmTE3nsgX9CT4PDjpy6RHej+TOKdiCbARZbHQP7sfqNStPOW7RkhcIL4shPXQsfcMHMzx0Ivsz97N67UenPZfb7aK21kGIJZSCY/tJJg2bBGPBgsfjITk0leqj1Rwo2NeWL9nv5Obn8N6KN1m+6l2KjxQ1fUArxcbEM3HMxVww6kIt6ucQbbEHmNQBsUydezu7du8kKTy5wZwkdmsI1dVVDfavPFZBzr49jAn7ZpSERSyk2s5j7fo1p7Tuv2azBZPatRuHigpwGzdWqevecRonISEhiAhWgnA6nW3wKv2PMYY33l3Mp2tWE+XsRJX7GP9+eylzZs1j/MgLfR1PBRhtsQegEHsIaSndOVST3+DxwppcBg1qOGzP7XYjCPKtHwWLWHC7T70J6GTXzbyJXMlGrIZCzwGOu6txSi0xMbGU15bhstXSPc07Nyu1tbLyUt58fym//8NvWLT0BQ4U7Pfq8+fk7ubTT1YTXRtPXsU+nNVO3BWGPz3/OLn5OV49l1Ja2APUDVfPIS9oD3uqdlBUk0921VYqIkqYMX1mg/1iojuR3KULB2sOnHjMGENB7X5Gjhh7xnP06t6X++99iCEThmFiXOyxb8cZWcN+RzZZ7s3M/v48rwxBbGtHSotZ+PgCtn68DfvBKPI3FfL40w+Tmb31tMdU11RRVHwQp6t5n0gytm/C7gihsPIAgy1j6WsZQrplHF3dPXnmL483WChFqdbSrpgA1SOtFw/8dCFr1n1MUWEB/XqMYcLoixodETH7hvk89effUlFdgt0dRoW1lNjU2NN2w5wsOakrs6+bx43XzCUzK4Od2duJiIhg9LALSIzv3BYvzeveW/4WUVXxnFe/Bmy8PYkIRzSvvb6Ihxc80XBuepeTf731Kuu++BQbwZggDzMunclFE6af8RwWq5VSRwld6EGw1A0/FBESpSs5pZkcOlxI58Qubfci1TlFC3sAS4hLajA3usvlYmPGOnbu2k5ERCTjRk4kOakrqV268cgDT7IxYx2lZSV0T+vJ4P5DW7QOqdViJX3gcK+Ok28vWdmZnBfScCK0uOBEdpdto/JYRYM/hm++u5Tt67YxImwSNkswVa5Klr39JjExsQw7//Rrog5PH80bby7BetKvnNu4MRZDuD2C46dZn1aps9Gqwi4i1wC/BvoDo4wxm7wRSnmf0+XkDy88yqGcQ8SRRK05wCefrGT2jfMYMWQMYaHhTBo7xdcxfSIqIoqa4mrCg765a9ZpasEC9pPuOq2tdfDZutUMDf1mGuHwoEi6ufrw0cr3GxR2Yww7srfy6dpVOBzHGTZkFMOGj2TXhl2EucNBBI+4CI4KBrshRcf7Ky9qbYs9E/ge8IIXsqg2tGHzZxTvPUx6+NgTXQsJzi78458vkz5wWIO1PQOBMYavcrLYuWsbISGhjBgyhoS4xieUumjyNF5fsoRIdzR2awhu42ZPTSajR4/HftJyfTWOGnCDPaThFAPhQZEUlec2eGzZ8jdYvWIlnUnDZgnm/T3L6JQWQ5d+yRTk7yVOOmNsbg5a9/KD62/12l26SkErC7sxJgvw2mx+gczpdPHu21/w2epNGAMXTB7BjKtGYbO1zy/0loxNJFq7Nvi/irLFYKsJZn9+Dr17BM70qB6Ph78vfZ7tW7YS607EZXHxwYfvMGfWfEakjzll/zHDJ3Ck5DAfrXyPEAmnxl3F4MFDufbKhqv3RIZHER4VQVnNEToFfzPdQLGjkN79+57YLisvZcXKDxgeMpFga90fhkTTha0H1vG9G67D43GTlZ1JdFQM48f8iC5JKW30TqhzVbs1E0RkPjAfICXVu7MAdnTGGB765St4qnO4+dq6m0SWvPkBmzdl8cjv5rbLH8bQ0FCqPGUntt3GzVFHCVXOSr8YudISmdkZZG7extCwC06Mr690pvHqkr8yqN+QUyb1EhG+O20mF0+czqHDhcRExzY6ra3FYuGaK29k0at/oauzB5G2GEqchygNLuLmafNO7JeTu5toS+yJov71OWJJYvfeLGZfN5/xoya3zYtXimYMdxSRlSKS2cjXFS05kTHmRWPMCGPMiLj4qKYPCCBbM/ZTlLeHJx9JZczIKMaMjOLJh1M5fHAvWza3zxjmcaMnUcQBHO7jFFbnsargXbYWb+BY5TFeevVZDh7Kb/pJ/MSWjI0k0PVEUYe6RUVCTQS7c7JOe1xYaDg90nqdca7y4emjufP2ewnvb6c4Jpfuo9O4/96HGsySGR4WgcOcejG0luNE6jwtqh002WI3xpybV9S8aMf2A0wYbSco6JuWeVCQMGF0MDu2H2DY8La/iWdAn/OZOv0yln3wBuVl5fSRwYTbokiIT+JIWSF/fP53LHzgqRaNhOmogoJsja6w5DFur7y+Pj3706dn/9N+v3fPfgRH2cgv30fX0O6ICOW1pZRYihg38s5Wn1+ppugNSu0gNi6CAwWeUx4/UOAhLq791qC89JIruWjyVHpE9iEtsSddk1MJtgXTJbQb7mOG7L072y1LWxo1fBzFUkCt23HisSOOQ7jtTnr3bPtrCVaLlbtuvY/jCRVsqvmEjOOfs8e6nR/OuY2khHNzwWrVvlo73PEq4E9AAvC+iGQYY6Z5JVkAmXThIP724jI+/G8pUy/shAh8tKqMjJ3w418OavoJmmCMYc3qHfzzlbVs37+Pfb2Wcsmky4iMOLXLy1nrIsIWRYi94eo0wYRQ9a15ZDoqp8uJxWJpdPphqGsxXzJtOh8uf49OkoBLnBy3VXHHLfe22/WEzoldePC+RzlYlIej1kFq1+4Bdy1DdVytHRXzFvCWl7IErLAwOwsfu5XfL1zMn1+um+0wPCqW3z52G+HhrV+dZ9FLq1j0x89Js/aks6svGbkZbNq8gV/c8/ApKxoN6Hc+W9Z/SZrpdeKiba3HQbkp4bzufVqdpS0VFOXx2r8XsXtvNlarhdEjxnP1jFmNLlJ9+dSZjBkxkey9O7DbQxjU99SLpm1NRHQ+euUT/t+h6id69+nCi3//CQX5JRhTNzLIG6NhKiuq+duzqxgeOZ6QoBCKq8pIlj5kHd3CpxtW8Z2LZjTYf/CAoazutYKtu9fT2ZqCy+OikFwuueRSr69Z6k0VleU88adHSHSkMj58Ki6Pi90bsnjuyJPcc/uCRt/L+NgE4mMnt39YpXxM+9jbkYiQkhpPalq814Y47t5dSKhEEBLUsDUaZ0kiO3vHKftbrUHcOe+nXHn9TIJ7CZ0GR3LLvNuZMe1qr+RpK2s3fkL48RhSw3piESvBVjt9w9PJ23eAvIP7fR1PqQ5FW+x+Lj4+impXFR7jaTD3erXnGKmxjS9UbQuyMX7UZL8aS11UVEgEDa8ZiAgRlmiKSw6R1rWHj5Ip1fFoYfdzad0SOH9EKllfZNEvpm7Ex9HaEookj5sumOPjdC1njGFjxlr+u3o55eVH6dd3AJdNvYq0tO7s/nI38E0B9xg3FZ4yuial+i6wUh2QdsUEgN8+8X16TQpnbcUqNletZX/wLubOuc0vW7EffryMpa++QmhhND0dA8nbWMCjT/2Kvuf1xx1Vy95jOznurqbSWU5m1SYGDBpEclLjn0yUOldpiz0ARMeE88QzP6C0pJLn173N5NpfnHYoYEd2/HgNHyx/hyGh4wix1o106Wnrz+5jHtZuXMN9dz/IOx+8wbbMLwi227ngwslM/9bFYaWUFvaAEhsXSWxyONY8/yvqAMUlRdgJOVHUvxZnS2TPnq+4ZsYsfvj923yUTin/oV0xqsOIjurEcU8NLo+rweOV7nISExufclcpdSot7KrDiI6MYeiQEWRXZ1DrcWCModRRTJHkcvGkMy89p5T6hnbFqA5l1rW38Lr9H6z/Yg3iESJjorh55h10T237idKUChRa2FWHYg+2M+uam7nmilkcP15NZEQ0Fot+sFSqJbSwqw7JHmxvsCydUqr5tCmklFIBRgu7UkoFGC3sSikVYLSwK6VUgNHCrpRSAUYLu1JKBRgt7EopFWC0sCulVIDRwq6UUgFGC7tSSgUYLexKKRVgtLArpVSA0cKulFIBRgu7UkoFGC3sSikVYLSwK6VUgNHCrpRSAUYLu1JKBRgt7EopFWC0sCulVIBpVWEXkcdEZJeIbBORt0QkxlvBlFJKnZ3WtthXAIOMMYOBr4D7Wx9JKaVUa7SqsBtjPjLGuOo31wMprY+klFKqNbzZxz4X+I8Xn08ppdRZCGpqBxFZCXRu5FsLjDHv1O+zAHABi8/wPPOB+QApqXFnFVYppVTTmizsxpgpZ/q+iMwBLgcuNsaYMzzPi8CLAEOG9TztfkoppVqnycJ+JiIyHfgZMMkYU+2dSEoppVqjtX3szwCRwAoRyRCR572QSSmlVCu0qsVujOnlrSBKKaW8Q+88VUqpAKOFXSmlAowWdqWUCjBa2JVSKsBoYVdKqQCjhV0ppQKMFnallAowcoZZANrupCKHgdz6zXjgSLuH8A7N3v78NTdodl/x1+yN5e5mjElo6kCfFPYGAUQ2GWNG+DTEWdLs7c9fc4Nm9xV/zd6a3NoVo5RSAUYLu1JKBZiOUNhf9HWAVtDs7c9fc4Nm9xV/zX7WuX3ex66UUsq7OkKLXSmllBd1iMIuIg+LyLb6Od0/EpEuvs7UXCLymIjsqs//lojE+DpTc4jINSKyQ0Q8IuIXIwZEZLqIZIvIHhH5ua/zNJeIvCwixSKS6essLSEiqSKySkSy6n9W7vZ1puYSkRAR+UJEttZn/42vM7WUiFhFZIuIvNfSYztEYQceM8YMNsYMAd4DHvR1oBZYAQwyxgwGvgLu93Ge5soEvges8XWQ5hARK/Bn4DvAAOAGERng21TN9ndguq9DnAUXcK8xpj8wBrjDj95zB3CRMSYdGAJMF5ExPs7UUncDWWdzYIco7MaYipM2wwG/6fg3xnxkjHHVb64HUnyZp7mMMVnGmGxf52iBUcAeY0yOMaYWeA24wseZmsUYswYo9XWOljLGFBpjNtf/u5K6ItPVt6max9Q5Vr9pq//ym7oiIinAZcBfz+b4DlHYAURkoYjkATfiXy32k80F/uPrEAGqK5B30nY+flJkAoGIdAeGAht8m6T56rsyMoBiYIUxxm+yA08D9wGeszm43Qq7iKwUkcxGvq4AMMYsMMakAouBO9srV3M0lb1+nwXUfXRd7LukDTUntx+RRh7zmxaYPxORCODfwI+/9em6QzPGuOu7d1OAUSIyyNeZmkNELgeKjTFfnu1ztGrN05Ywxkxp5q5LgPeBX7VhnBZpKruIzAEuBy42HWj8aAvec3+QD6SetJ0CHPRRlnOGiNioK+qLjTFv+jrP2TDGHBWR1dRd5/CHC9jjgRkicikQAkSJyD+MMbOa+wQdoitGRHqftDkD2OWrLC0lItOBnwEzjDHVvs4TwDYCvUWkh4gEA9cDy3ycKaCJiAAvAVnGmCd9naclRCTh6xFqIhIKTMFP6oox5n5jTIoxpjt1P+cft6SoQwcp7MCj9V0E24Cp1F0N9hfPAJHAivrhms/7OlBziMhVIpIPjAXeF5Hlvs50JvUXqO8EllN3Ee9fxpgdvk3VPCKyFFgH9BWRfBG52deZmmk8cBNwUf3PdkZ9K9IfJAOr6mvKRur62Fs8bNBf6Z2nSikVYDpKi10ppZSXaGFXSqkAo4VdKaUCjBZ2pZQKMFrYlVIqwGhhV0qpAKOFXSmlAowWdqWUCjD/Dwm3Mw7FJJawAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "classifier = XGBoostClassifier()\n", "classifier.fit(data, target)\n", "utils.plot_decision_function(data, target, classifier)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.8" } }, "nbformat": 4, "nbformat_minor": 2 }