{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Model allocation\n", "> Applying portfolio theory on model ensembling.\n", "\n", "- hide: false\n", "- toc:true\n", "- badges: true\n", "- hide_colab_badge: true\n", "- hide_binder_badge: true\n", "- comments: true\n", "- author: Johannes Tomasoni\n", "- image: images/model_allocation.png\n", "- categories: [Ensembling]\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> This notebook was originally published on https://www.kaggle.com/joatom/model-allocation.\n", "\n", "\n", "In this notebook I experiment with two ensembling strategies.\n", "\n", "There are many ways to combine different models to improve predictions. A common technique for regression tasks is taking a weighted average of the model predictions (`y_pred = (m1(x)*w1 + ... + mn(x)*wn) / n`). Another common technique is building a meta model, that is trained on the models' outputs.\n", "\n", "The first chapter starts with a simple linear combination of two models. And we explore with an simple example, why ensembling actually works. These insights will lead, in the second chapter, to the first technique on how to choose weights for a linear ensemble by using residual variance. In the third chapter an alternative for the weight selection is examined. This second technique is inspired by portfolio theory (a theory to combine financial assets). In the fourth chapter the two techniques are applied and compared on the [Tabular Playground Series (TPS) - Aug 2021](https://www.kaggle.com/c/tabular-playground-series-aug-2021) competition. Finaly cross validation (CV) and leaderboard (LB) Scores are listed in the fith chapter.\n", "\n", "> Note: For the ease of explanation we make some simplifying assumptions, such as equal distribution of the data, same distribution on unseen data, ... (just think of a vanilla world)." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#hide\n", "\n", "import pandas as pd\n", "import numpy as np\n", "from matplotlib import pyplot as plt\n", "import seaborn as sns\n", "\n", "from sklearn.model_selection import StratifiedKFold, cross_val_score\n", "from sklearn.metrics import mean_squared_error, mean_absolute_error\n", "from sklearn.linear_model import LinearRegression, LogisticRegression, BayesianRidge, ARDRegression, PoissonRegressor\n", "from sklearn.ensemble import ExtraTreesRegressor, RandomForestRegressor, StackingRegressor\n", "from sklearn.svm import SVR\n", "from sklearn.neural_network import MLPRegressor\n", "from sklearn.neighbors import KNeighborsRegressor\n", "from sklearn.preprocessing import MinMaxScaler,StandardScaler\n", "from sklearn.pipeline import Pipeline\n", "from sklearn.experimental import enable_hist_gradient_boosting \n", "from sklearn.ensemble import HistGradientBoostingRegressor\n", "\n", "import scipy.optimize\n", "\n", "from xgboost import XGBRegressor\n", "\n", "from tqdm import tqdm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 1. Why ensembling works\n", "\n", "Suppose there are two fitted regression models and they predict values like shown in the first chart." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "#hide\n", "\n", "x = np.linspace(1,7,7)\n", "\n", "y_true = np.array([5, 4, 8, 6, 5, 4, 5])\n", "m1 = np.array([7, 3, 9, 5, 6, 3, 3.5])\n", "m2 = np.array([4, 5, 8, 7, 2, 5, 6])" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#hide\n", "\n", "df = pd.DataFrame(np.array([x, m1, m2, y_true]).T, columns = ['x', 'm1', 'm2', 'y_true']).melt('x')\n", "df.columns = ['x', 'Values', 'y']" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbkAAAFyCAYAAABoe6RfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABT5ElEQVR4nO3dd3hUVfrA8e+Z9AIkoYaaIEgRUCGAFBUpSrCsiK4NFLuuWNgVV1bXXn/YWLFhb9gQuwERRBCk995CKAkhgQTS25zfH2eAAAlpM3NnLu/nefIkc+fOvW+SmXnnnvIepbVGCCGEsCOH1QEIIYQQniJJTgghhG1JkhNCCGFbkuSEEELYliQ5IYQQtiVJTgghhG15LMkppd5XSu1TSq0tty1GKTVTKbXF9T3aU+cXQgghPHkl9yEw9LhtDwGztNbtgVmu20IIIYRHKE9OBldKxQE/aa27uG5vAgZordOUUrHAHK11B48FIIQQ4pQW6OXzNdVapwG4El2TynZUSt0O3A7QuXPnHuvWrfNSiEII4TeU1QH4Op8deKK1nqy1TtBaJ4SFhVkdjhBCCD/k7SSX7mqmxPV9n5fPL4QQ4hTi7ST3A3Cj6+cbge+9fH4hhBCnEE9OIfgc+AvooJTarZS6BXgeGKKU2gIMcd0WQgghPMJjA0+01tdWctcgT51TCCGEKM9nB54IIYQQdSVJTgghhG1JkhNCCGFbkuSEEELYliQ5IYQQtiVJTgghhG1JkhNCCGFbkuSEEELYliQ5IYQQtiVJTgghhG1JkhNCCGFbkuSEEELYliQ5IYQQtiVJTgghhG1JkhNCCGFbkuSEEELYliQ5IYQQtiVJTgghhG1JkhNCCGFbkuSEEELYliQ5IYQQtiVJTgghhG1JkhNCCGFbkuSEEELYliQ54ROSkmDgQIiPN9+TkqyOSAhhB5LkhOWSkmDMGEhLg5gY833MGEl0Qoi6kyQnLDdhAgQFQUQEKGW+Bweb7UIIUReS5ITlkpOhoADWrIGSErMtPBx27LA0LCGEDUiSE5aLjwetwemE7dvN9/x8iIuzOjIhhL+TJCcsN26caa6MjTUJ7uBBKC4224UQoi4kyQnLDRkCL70E7dpB06bQpg1MmgSJiVZHJoTwd4FWByDEX3/BVVfBzJkwYACUlpoBKEIIUVdyJScs16wZPPggnHEGrFhhbv/+u9VRCSHsQK7khOXat4dnnjE/R0bCJZeY+XJCCFFXkuSE5TZvNv1xDgeEhcGHH1odkRDCLqS5Ulhq3z7o0AFeffXY7Xv3msonQghRF5LkhKVCQ+GDD0wT5WF5eWaO3CuvWBaWEMImpLlSWKp+fRg9+thtERHwzjuQkGBJSEIIG5EkJyy1YAG0bWtGVJY3apQ18Qgh7EWaK4VltDbNlP/9b8X3L1hg5s4JIURtyZWcsIzW8PPPUK9exfc/+KAp7zVkiHfjEkLYhyQ5YRmHA/r0qfz+994z9SyFEKK2JMkJy8yaZUZX9utX8f0dOng3HiGE/UifnLDMww/D+PEn3+enn+Chh7wTjxDCfiTJCcv8+CNMnnzyfZYvhy+/NIuqCiFETSmttdUxVCkhIUEvXbrU6jCEBQoLISREViUQohLyyqiCXMkJS8yZYyZ8l5aefL/QUElwQojakyQnLPHFF2Z+XEBA1ftOnw7dukF2tsfDEkLYjCQ5YYk334SVK6t3lRYdDU2aQEaGx8MSQtiMTCEQllDqxFJelendG377zbPxCCHsSa7khNctWwYPPGCW2amJggJTAUUIIapLkpzwutWr4fXXISioZo9p1AiSkjwXlxDCfiTJCa+76SY4dMj0tVVXp05w661mxQIhhKgu6ZMTlqjJVdzh/SdO9EwsQgj7kis54VV79sDf/ga1ndu/bRvs2OHWkIQQNmZJklNKjVVKrVNKrVVKfa6UCrUiDuF9aWmweTM4nTV/bGGhmS/34ovuj0sIYU9eb65USrUA7gU6a60LlFJfAdcAH3o7FuF9CQmwYcOJ25cseZKATa/QWOeQoepR1mEsPXs+esw+oaHw+ecm0QkhRHVY1ScXCIQppUqAcCDVojiED1iy5EmabHySAK04pIKo78wnaONTLIETEt1ll1kToxDCP3m9uVJrvQd4EdgJpAEHtda/ejsO4X1lZdCjB3zyybHbAza9QqETWgaW0kKVUKwCKdEOAja9UuFxZs6EadO8ELAQwu95PckppaKBvwHxQHMgQik1soL9bldKLVVKLc2Qek62cPAgtGwJERHHbm+scyhylgHQwFHGwdISUsscNNa5FR5nwgR4+mlPRyuEsAMrBp4MBpK11hla6xJgGtD3+J201pO11gla64TGjRt7PUjhfjEx8P33cMUVx24/oCLoEnz0dpuAEsbvK2Qfx2VDl/fegwULPBioEMI2rEhyO4FzlFLhSikFDAIqGIog7KaypQvrxQ7GoaBAKwq0onkgfNpM4ez4T7TWHL/mYatWZhCKEEJUxYo+uUXAVGA5sMYVQxXrQws76NsX7rvv2G1O7STnwEoAMnUoB50OtIa4IOgZP5Tn/3yem76/iZKykmMeN20a3HGHlwIXQvgtS+bJaa0f01p31Fp30VqP0loXWRGH8B6tYcCAE4f/J2euo33RdgBaXbGZZqNKUZ3HodCw+FbKygopdZYS6Dh2IPD27fDnn5BbcbedEEIAoI5vCvJFCQkJemltS2QI37brW5h3Bc6YnjiGLjbbSvPhl66Qux3OfBbd+SGUUuzN3cua9DUMOW0IZWXVW3BVCJurxoqMpzYp6yW8Ijf3xD65UmepSXKAo/WIo3cEhkPPt8zPa55A5WwF4NHfH+WKr65gf/7+IwnODz6jCSEsJElOeMWtt0L37sduG//rA+Qkf25utBx+7J2xQyD+BnAWwZI7QWteuegVkq5PomF4QwB++KmEuDhIT/d8/EII/yRJTnjFiBFw++3HbhsY4aCeKoUGnaH+6Sc+6OyXIKQRpM+G7R8SERxB/9b9Afhx04/ct6krHc7ZwaFDXvgFhBB+SZbaEV5x1VUnbksMKTA/tLzixDsBQhtB91fgr1Gw4l/QfBiENQUgOiyaM2Lb8fWnTQmr4bI9QohTh1zJCY/LyIDMzKO3tdbM3DoDvft7s6HV8IofCBB3PcReBMVZsHzskc39W/fnp+t+IiwojPTMIibMe9X08QkhRDmS5ITHTZoETZtCfr65/efOP3n0m6GowjSIaAPRZ1f+YKWg55sQEA4pn0Nq0jF3r18PzQd9x4OzxzI3Za4HfwshhD+SJCc87oor4O23ITzc3O7Tqg8fdXctJ9ByuElkJxMZD92eND8vvhNKjk6O69gR/jv8aqYOWc7A+IEAHCqSTjohhCFJTnjcmWea0ZWHBaoATi9YZ26crKmyvA73QXR3yN8Jq/97ZLPDAY8/DiP6mqvBDRkbiJ8Yz7cbvnVT9EIIfyZJTnjU/v2mmHKRq6bNm0ve5OsFj0LuNghpDI36Ve9AjkDo/Q6oANj8P9i/5MhdWsPKlWYx1qaRTRnWfhi9WvRy/y8jhPA7kuSER82YAf36wcaN5vaPm3+kZOfX5kbLv4GjBmVLYrpDx7GgnbDoNnCaepalpXDBBfDCCxATFsMnwz+hRf0WaK15eu7TpGSnuPm3EkL4C0lywqMuugi+/RbOOMPc/vm6n7m6vmtdnVaVTB04ma5PQEQ8ZK+CjS8DEBRkzvF//3fsrikHU3hxwYtMWTOlDr+BEMKfSe1K4TXFZcUEF+yGH06DwHowIgMCQmp+oLSZ8PuFEBAKw9ZAvXaV7rrz4E5a1m+JQzlIy0mjWWQzVFUDXYTwH/JkroJcyQmPKSmBjz6C1FRYvGcxLV9uScraV82dLS6uXYIDU/IrbhSUFZrRlq4Pat9/b85XXusGrXEoBzlFOfT/oD93/3J37X8hIYTfkSQnPGb1ahg92iyJExoYygXxF9DyoGulgdo0VZbX/SUIaQjpsyDZZLYPP4SJEyvePTI4knt63cOobqPqdl4hhF+R5krhMU4nbNoEzZtDgwZAQRp82wIcwTAiE4Ii63aC5E/grxsgOAYu2UBmbhOio6u3BM+kxZNoGtGUq86ooN6YEP5DmiurIFdywmMcDujUCdYdWkBGXgbs/h7QEHth3RMcQNxIaDYEig/AsrE0alS9BFfmLOOrdV/x5bov8YcPeUKI2pMCzcJjXn4Zevcp45qFV9OtaTd+buaqLXn8sjq1pRT0egt+7gIpUyB+JF/MTWTqVPj668oLqQQ4Avjtht8oLitGKUVmfiZO7aRJRBP3xCWE8BlyJSc8Ij8fxo+H2b8FMGPkDJ4/9yGzZI5yQItL3XeiyLZHS34tuYvc7Fz27KHK5XeCA4KJDDZXkzd9fxP93+9PSVmJ++ISQvgEuZITHhEeDtnZUFwMDRp0huRPQZdC0wvMEjru1OF+2DEFslZwS89HufXOl2v08CcHPMnWA1sJCpA1e4SwG7mSEx6z5dBqHp4/hvTcdNjtqiXprqbK8o6U/HKgNk+E/UspK6v+w8+OPfvIAJQZW2dw2w+3UVha6P44hRBeJ0lOeMSLL8KrXy1jypopBOmSo0vktLzcMyeM6QEdTMmvg7/dRvNmJaTUoprXqvRVLEldQpmzBllSCOGzJMkJj/juOyhedBN7/rmHmOylUFYAMT0hopXnTtrtCYiIo0HZSv73j1coqUUX24P9HmThrQuJCI6gpKyEv3b95f44hRBeI0lOeMRvcwr54AMICwo72lRZ3WV1aiswAnq+BcDVnR+jXdNttTpMaGAoAC8ueJH+H/RnY+ZGt4UohPAuSXJ1lJQEAwdCfLz5npRU9WPsrsxZRtc3u/L0n4+ZlQJ2/2DuqGuVk+pofhHEXQ9lhRTNu4NDB2s/D+6e3vfw0eUf0bFRR5K2JHHBhxcQPzGegR8NJGmL/KOF8AeS5OogKQnGjIG0NIiJMd/HjJFEN/H1Iurv+jsJzXvCvj+gJBvqd4L6HbwTQPdXKAtsSEj2LJZP+6TWh4kMjmRkt5EkbUnijp/uYOGehYQHhpOWk8aYpDGS6ITwA5Lk6mDCBAgONl/5+RARYX6eMMHqyKy1b084jVY9w6UdLoFd08xGb1zFHRbaGEeCmUZwbsQ/oTCjToebsGACDtdLxaEcRARHEOwIZsKCU/wfLYQfkCRXB8nJZj5YcjLs2GG2hYcf/flUlJKdwvAxi5gxA7O46e7vzB2e7o87joofBc2GEFC6H5aPrdOxkrOTaRjekM6NOhMSaFZOCA8KZ0f2DjdEKoTwJElydRAfb67gWrQwP4O5HRdnaViWmrhoIud9eB778/fD/sWmKHN4a4ju7t1AlMKZ8BZlKgx2fAapM2p9qPioePJL8lFKUVBSwMHCg+SX5BMXFee+eIUQHiFJrg7GjTMVPQDCwiAvz9weN87auKzUJfMJOiz9CQoalmuqHF55IUlPimzLMz88YX5ecieU5tXqMOP6jqPYWUxecR57c/eSnJ1MUVkR4/qewv9oIfyEJLk6SEyESZMgNhb27oXSUnM7MdHqyKwTGVSPxjlDiI7SsMuDVU6qweGAi/81lrL6Z0HeDlj9WK2Ok9g+kUmJk4itF0t4UDi9WvTi9WGvk9j+FP5HC+EnZD05N3nuOXj88aMjLU81Wmtu/uFmbuh2AxfEXwDZa+CXbhDSGIangaMaa+B4yoFlMKOX+fmixaY6ihD2IOvJVUGu5Nzkzjth375TM8EB7D60m9+Tf2fXod1mw5GruMusTXDA5zN6sDz/fjMQZtFt4Cyt0/Gmb53O20vfdk9wQgiPkiTnJtHRrtWvT1GtGrTik57beDDxWhYtwrMFmWvo55/hvnefREfEQdYK2PhKnY731bqveO7P52TBVSH8gCQ5N1q+HIYPh/37rY7EuwpKCnBqJ/UjA7hoSCBtmyRD1koIrAfNBlkdHm++CXMXRKB6vmk2rHkMcrfX+ngvXvgiW+/dirJiMI0QokYkybmR0wlLl8LWrVZH4l2PzXmMrm92pVOXYj76CBoXuq7img+DgFBrgwPq1XMN7mw+FNpcZ4pFL74TanklFhMWQ6BDlmIUwh9IknOjHj1g507o3dvqSLyrT8s+XNnpSgrzgs2G3RZUOanCp5/CBReA7v4KBMfA3pmw49NaH2/yssn8d/Z/3RihEMITJMm5kVJHp4OdSt01wzsN556uTxAVBR9PToeMBeAIgea+M8Te4TAl17IKmkB318rhy8fWuuTXirQVzNs5z40RCiE8QZKcm6WkQKdO8M03VkfieVprvl73NQUlBQA89RQMPP17QEOzwRBUz9oAy7nuOpgxwzX6Nf4GE1/Rflj+z1od7/WLX2fO6DlujVEI4X6S5NysZUvo2BHq17c6Es9bvGcxf5/6dz5b8xmNGsHDD0NL7XtNleUVFGAut3u+ZfoLd3wKab/W+DgOJS8dIfyBvFLdLCAAvv0WLrzQ6kg8r1eLXvx+4+9c2+Vatm6F4txsSJ8NygEtLrU6vBP88gs0bAgbNgD1ToOuj5s7Fteu5NfDsx5m9Hej3RmiEMLNJMl5SGEhZNRthRefp5RiQNwAwoMiOOcc+Pj5n80iqY3Pg9DGVod3gjPPhJtvhpAQ14aO/4SoMyEvGdY8XuPjBToCZZSlED5Oynp5gNNpmi0vvRTetmlhjIkLJ5JTnMPD5z6M06mYNg0GBl1Jw/xvoMdE6HCv1SFWz/6l8KtrOOxFSyDGy6slCFE3MlmzCnIl5wEOBzz5pBnsYFfL9y5n0Z5FKKUICICrhhfQsMi1UrYPVDk5mc2by03Yb5gAp9/nKvl1a61KfpU5y9wboBDCbSTJecitt8L551sdhed8dPlHTPu7GWSyfDnsXfkrlOVDTAJEtLI4usolJ0OHDvDZZ+U2dnsSItqYkl+bJtboeCOnjWTYlGHuDVII4TaS5Dxo+3aYM8fqKNzvUNEhAIICggC45x5Y/oOryomXVwCvqfh4+OADuKL84M+gSDhc8mv1o5CbXO3j9W3VlwviLnBvkEIIt5E+OQ+65BJYt84kO7uUOVyaupTzPzyfH6/9kYHxAwFYs7qETuubEujMgos3QIOOFkdZS/Ovg5TPodmFcMF0+/zThJ3Jk7QKciXnQc89B7Nn2+u9Mio0ipFdR9Ij9uiabF2bzDUJrn5Hv0hwTqdZmWDe8QVLerzqKvn1K+z4rKKHVqjMWUZ6brpbYxRCuIckOQ/q2tU0j9lJu5h2vH3p2zQINesKLV0KKfMPN1X65gTw4ykFY8bAxOO730KbQPeXzM/Lx0JhZrWON+jjQfx96t/dG6QQwi0kyXnY4sXw2GNWR+Ees7bPYtuBbcdse/MNJ8HpvrN2XHUoBdOnHzf45LD4G6HpQCjKhBX/qtbxxvQawz297nFvkEIIt5A+OQ/73//gP/+BbdugaVOro6k9rTXtXmtH2+i2zBw188j2vJ2LiPjzHAhvBX9LsUfbbM5W+KUrlBXCBb9C7BCrIxKiMjZ4wXmWXMl52C23QGamfyc4MNVN5o6ey8Shx7bxRex31apsOdzvEtwHH5ii0ieo1w66uC6/l9wJpflVHis1J5XV6avdG6AQos4kyXlYRASEWr9uqFu0qN+Czo07H7m9do0ma41/TB2oyMKFptmywsaMTv8yJb9yt1er5NcVX17BXT/f5fYYhRB1I0nOC5YuhYEDYc8eqyOpnbX71nL11KtJyU45ZvvKueuJDtyCM6gRNO5vUXS199prMH9+JRegjiDo/Y4pNr3xZTiw4qTHmjBkApMSJ3kmUCFErUmS84KICEhPh927rY6kdjZlbmJeyjwigyOP2T7yPNNU6Wh1GfhhoeLg4Cp2aNgTTr8XdBksvu2kJb/ObXMuZ8ee7d4AhRB1JknOCzp1MpPCe/e2OpLaGdF5BDvH7qRheMNj79jtv02Vh334oVmdoLSy/NXtKQhvDQeWwab/nfRY81Lm8eu2mq9NJ4TwHEuSnFIqSik1VSm1USm1QSnVx4o4vE3rk7yZ+qjswmyAE5aUyUhOhqwVlDkizSrbfqphQ2jfHrKzK9mhfMmvFePg25YwayCkJp2w6/hZ43n090c9FmtVkpJMs3h8vPmedGKIohqStiQx8KOBxE+MZ+BHA0naIn9If2bVldxEYLrWuiNwJrDBoji8ZvduaN26krlZPsqpnSRMTuCeX06cA3Zow3cAZIcNMyts+6lLL4WpU6FRo5PspBQERABOKMmB/FRYMuaERPf+395nxsgZHo23MklJZoJ7WhrExJjvY8ZIoquppC1JjEkaQ1pOGjGhMaTlpDEmaYwkOj/m9SSnlKoPnAe8B6C1LtZaZ3s7Dm9r0QKGDDGJzl+UlJVwd8+7SWyfeMJ9pwWZpsroM/2jyklVsrIqGWUJsGEChMWCCoDSQ1By0AxM2TDhmN1Ob3j6kUow3jZhguljDA01OTkiwtyeMKHqx4qjJiyYQLAjmIjgCJRSRARHEOwIZsIC+UP6Kyuu5NoCGcAHSqkVSql3lVIRx++klLpdKbVUKbU0wwZLbCsF778PF/hRwfqQwBDG9hnLsPbHLSVTkA4Zf4IjGEeLExOgv5k+HRo3hmXLKtkhNxmC6pu+OYDCvVCYBjnbTtj1gxUf8OnqTz0XbCWSk+HgQdiwAcpcy9uFh8OOHV4Pxa9tPbCVtNw0CkoKjmwLDwpnR/YO64ISdWJFkgsEugNvaq3PBvKAh47fSWs9WWudoLVOaNy4sbdj9JjsbNi1y+ooqrYjewc/b/4Zp3aecJ9z9w+AJl0NNm/+fq5nTxg3ziS6CkXGm7XyQmIgsp25ois5BEUZkHXsBPCPV3/MZ2u83yYdHw9BQdCgARQXm+bKvDyIi/N6KH6tVf1W5JfkU1haeGRbfkk+cVFx1gUl6sSKJLcb2K21XuS6PRWT9GxPazjjDBg/3upIqvbOsne4/MvL2Ze374T7SrabqQPJpfZoqmzY0KwY0aZNJTt0GgdlxVCaZ5J6RBtwBENZAfx6DiQfTWrfXf0dv1z3i3cCx0xo/+gjk6QDA01/XG6uSXL5+Wa7qNqcHXPQWvPIeY/Qsn5LggOC0VqTV5xHsbOYcX3lD+mvvJ7ktNZ7gV1KqQ6uTYOA9d6OwwpKwSuvwH33WR1J1R4f8DhzR8+lWWSzY+8oPkhI1ixQDs4ZcZk1wXmA1qa5Mi2tgjubJ0LPSaZfrjjLJLl+X0Lb0SbR/TUSlt4HzhIahDZAebG82YQJJkEPGgSTJkFsrHme9e0Lb78Nif7fmuxxv2z5hQs+uoBvNnxDYvtEXh/2OrH1YskqzCK2XiyTEidV2C8t/IMlBZqVUmcB7wLBwHbgJq11VmX7+3OBZtvZ8TksuA6anA+D51gdjdvs2QMtW8Kzz9bgSltr2Po2LLsXnCWm6kv/r3h4wSRiwmL4V9/qrWJQUwUFUFQEUVGm+dvpNFdwlYXoZyVFvUZrjVIKp3byyapPGNltJAGOAKvDqin571bBkikEWuuVrv62blrry0+W4Oxo7Vr48Uero6iY1pqrvr6K7zZ+V/EOu0xT5Yz1/jsBvCItWsC338Jtt9XgQUpB+zth8FwIa24G40zvQUn6H2zev9kjcTqd5qrt+utNAouKqjjBlZbC8OHw5JMeCcPvzUuZxznvncP+/P04lIMbz7rRHxOcqAb/q8VkA08+aWomXnKJ733KzsjPYOuBrRwsPHjinaUF6LQkFLBLX+7t0Dzu8str+cBG58DQ5TD/atj3By+oDNTp13jkMsrhgFtvhSZNTn7owECTAOvVc+vpbSMkMITismKyC7NPrOQjbEXWk7PAtm0QGem7y+9ordFoHOq4C/3dP8Dcv0FMDxhqn//HYVrDl1+aEYq16stylsDKh0xBZ4C4kdDrbQgMr3Nczz8P55zjX1NQfE1OUQ6/7/idyzqYvmSndp74HPc/PvYx2ff4/X/YH512mm8muENFhygqLUIpVfGL/3CtSj9ZAbymlDLry02eXMsDOIKg+0vQ93MKCYAdn8Kvfc1yPXWQnw8ffwzTptX8sVrDqlV1Or1tPDX3Ka76+ip2HzKV0m2Q4EQ1yH/ZIn/+Cffcc5IqGxZ4bt5ztP1fW/JLKlgk1FlqruSA256yx9SBisyYYcp81UncNbwYeSX7AxtC9iqYnlBhrcuqbN1q+tYiIszz5X8nrw9doU8/hbPOMss9napKykoAePT8R5k5aiYt67e0OCLhTZLkLLJxI3zxBaSmWh3JUReediH39LqH8KAKmtf2zYXiA2SVdaBN107eD85LWraEADeMP3jksi9oePk2aHGZmXYw52JY8yRUMLm+Ijt3muT0zDPmdsOGteveu+wyeOMN6Nix5o+1g6f+eIoLP72QUmcpkcGRnNfmPKtDEl4mfXIWKSoygwPc8YbqFUvvgc2ToPNDcNZzVkfjUW++aa6iXnqp7scqLi0keOOLsPpRQEPzS6DvJxAcVeVjX3kF/v53M/JT1M4nqz5hbspcJg2bREhgiNXheIL0yVVBruQsEhLiOwlOa817y99jf/7+SnZwwi7TH1fW3L5NlYdt3Wr6sery+U9rTY/JPRiTdC90eQQG/ALB0ZD6E0zvCdlrTnjMvn0wYoSpQwkwdqz7Etw338Dnn7vnWL5uedpyftv+GwCjzhzFO5e9Y9cEJ6pBkpyFliyBhATzpmqltfvWcuuPt/L1+q8r3mH/UijYw8GSlkS3S6CwsOLd7OLFF+G33+o2+l8pxYhOI442jzUfakakRp8FuVthxjlmYn05OTnw119mHqW7vfkmvPWW+4/ra7TW3P3L3fzr139VWHdVnHpknpyFmjQxTZb790O7dtbF0bVpV1bfuZr46PiKd3CNqsyJGs6//60I9d/l46rlcHKr6zS3/5z7n2M3RLaFIfNhyV2Q/DEsuA69fzELC/+PPv2COO00M70kLKz256zMZ59VsWaenzs8Kjg4IJgvRnxBeFC4jJ4UgPTJiapoDT91hJzNMGg2ND01Jmq9956ZtL95s2larq38knwOFR06tgao1rDlTVh2H+hS/thwHrrflwwY2qzyA7mJHct8FZUWMeCjAXRv1p3XL37d6nC8zWb/TfeTjzo+oLQUy5oAX1zwIvdPv7/ypp1DGyBnM86ghuwsPNenpjx4Unw8DBxo1mirLa017f7XjvGzjiuGqRS6/T9g8B/o0FjO7zSX8wt6QMaCugVdhaVLoUMHzzSHWikkMIQL217IwPiBVocifJAkOYvt3Wsmhn/wgTXnT81JJeVgSuVNO65alTtKL6NNXOApM7F44EDzP2nSpPbHUErx/ODnuemsm47ZPnu2qVySG9YXlbgcGp+LKkiFWQNg8xsemzwZH28GsuRXMA3S32itmbhwIhszNwLwxAVPMKLzCIujEr5IkpzFmjaF0aOha1drzv/yRS/zzd+/qXwH16jKiA7Dee01sx7eqWTPnqMrbdfGDWfecMLcLKcTsrJMXyxhzWDQLOhgluph6d2wcDSUFlR4vLpo2BB+/x169XL7ob0uMz+Tp+c9zbvL37U6FNtQSs1RSl103Lb7lVJvnGT/BO9EV3vSJ3cKS89Np2nkSeqL5aXA93EQGAEjMiHA5iNOjvPrr3DRRfDHH3BeLecQa61Zn7GevDzFoW2dGTzYbC8rq2AKyY7PYdGtZhXy6LPg3GlmVXI3y8+H7duhSxe3H9rj9ubupWlEU5RSJGclExcV59X1+3yQ2355pdQdwDla65vKbVsIjNNaz6tg/znAA1prn35zlis5H5GWZgY5eMvytOW0eLkFP2z6ofKdXFdxzmbD+O33UHJyvBScj+jb16wvd9ppdTvOwI8HMvKt57j8csjMNNsqnCMZdy1ctBAiT4OslTC9B6ROr9vJK3DllWbFBaefjbBfn7GeDpM68P6K9wGIj44/1ROcu00FLlFKhQAopeKA5sB1SqmlSql1SqknKnqgUiq33M9XKqU+dP3cWCn1jVJqieurn2v7+Uqpla6vFUopj62XIVMIfIDW0Ls39Ohh1jTzhmaRzRjXd9zJyxy5pg7s1MMZMsRMJr7mGu/E5wsiI2uwgGolnE7FFyO+IJq2HLq4GsP4o7qa+XQLRpmJ43OGQbcn4Yz/gJuGxP/nPybB+Vt+6NCwA7d1v43BbQdbHYotaa33K6UWA0OB74FrgC+B57TWB5RSAcAspVQ3rfXqah52IvCK1vpPpVRrYAbQCXgAuFtrPV8pFQl4bOidNFf6iKQkaNMGOne2OhKXwn0wrRk4gsgbmsGCJfU5+2x7z7WqSFkZzJtn/jfxNWw5fPhhWL/eVBtx1DQ/aSesfRrWPA5oaHEp9Pm4WuXA7ORAwQHG/zaeF4a8QFRolNXh+CK3flRRSo0ELtZaX6uUWgncDPQCbsdcFMUC92itvyjfXKmUytVaR7qOcSVwidZ6tFJqH1C+Qm9joCNwNzAc+AyYprXe7c7fozxprvQRiYneS3C/bPmFJXuWnHyn3T8AGpoOIiKqPkOGnHoJDswAkUGD4P33a/7YJk2gWTMoLinjm/Xf8OfOP6v/YOWAro/C+T9BUBTs+dFVDsw94/8PHIAnnoAtW9xyOI/ZlLmJz9Z8xsLdC60O5VTxHTBIKdUdCAOyMFddg7TW3YCfgYo658tfLZW/3wH00Vqf5fpqobXO0Vo/D9zqOsdCpZTHSohLkvMhixaZyhSepLXm37/9m3//9u+T73h47bhWVzBlinf7C31Jo0YwaxY89FD19l+yBBa63o/vvdeU0woJdnDv9Ht5a2kt6mq1GGaaL6POdJUD6w0pX9b8OMcpKjL9jbNm1flQHrEpcxMAfVr1IeX+FIa2G2pxRKcGrXUuMAd4H/gcqA/kAQeVUk2BypYTTldKdVJKOTBXaIf9Cow5fEMpdZbr+2la6zVa6xeApZirO8/QWvv8V48ePfSpYPRorZs21bqszLPnOVh4UG/Zv6XyHYoPav15sNZTHDo3M10rpfXjj3s2JjsoK9P6jDO07tNHa6fz2Pu27N+ii0uLa3/wkjyt54/U+jPM19KxWpfV4Xha6/T0Oj3cY95Z9o4OfDJQL0tdZnUo/sDt77euJKWBjq7bHwIbMFdx04DRru1zgATXz1cC21zbJgEfurY3wvTrrQbWA2+5tr8GrAVWYZJpiCd+F6219Mn5ktRUM9ihfn2LA9nxBSy4Fhqfix48l5QUU9oqNtbiuCyiNbz7LkRHm5GJxyu/bNLmzWY+WsOGHgpk8+uwfCzoUmhyPvT7EsLqtsy801mLPkMPOlh4kDeXvskDfR8g0CFj46rgZ8OHvM+HntqieXPPJrj1GesZ/PHgI01BldptqpzQ6gqUgri4UzfBgRmF+M47FS9Vk5sL/fub/i2A00+vOMGVOkt5dt6z/Ljpx7oF0mEMDJ4Doc1g3x8wvTtk1r6/6oUX4JxzrF+hfm7KXG749gbKnGU0CG3AQ/0fkgQn3EKSnI/5/XcYOdIzc5h2H9rNrkO7aBh+ksuMskJITTI/t7ycDz6An392fyz+Zvp0mDr1xO0REdCvH3TvfvLHB6gA3ln+DrOTZ9c9mMb9IHE5NO4PBanw23mm4HMtMlWrVmYFcqtLfW3ev5lFexaxL2+ftYEI25HmyrpKTYINEyA32VSn6DQOmlfWN1u1zz83c7P++MMMW3eHpC1JTFgwgeTsZOIaxPFgvwdJbF9JjHt+gj8uhejukLiMDh3M/L0pU9wTi79KSoIJE8yCpnFx5ortkUdMkqiugpICwoLcuI6OswSWPwCb/2duNx0MuhjydrrluegJ5Z+Lreu35qozrmJMLzMuwe1/n9py82vaw6S5sgqS5OoiNQmWjDHDvQPrgbMAyoqh56RavyjKykz/iLsm6iZtSWJM0hgUivrB9SkoLaDYWcykxEkVJ7qFN8P2D6DbU9DlEUpLTSV+j/Qx+YmkJBgzxjRNlpaavrnt2+Haaz0/GrZakj+DRbeAswgcIRDZDiir0XNx61aoV8/UUvWUw8/FYEcw4UHhbDmwhfySfL6+6muGdxpe9QG84fBr2hEEjmDQJXV+TXuYJLkqVNlcqZQao5SK9kYwfmfDBJPg8lIgP8XUdgwINttrKSDAJDit3dNPMmHBBIJUEHty9pByMIWI4AiCHcFMWFBBjM5S2OMq89XSvOkEBp7aCQ7MFVxwsPnflJVBgwZmYnhqatWPLa+otIgbv7uRj1d97N4A46+HqG6ggkyiy9lo/pfVfC5mZkKnTvDqq+4N63gTFkwg2BFMRHAESilaN2hN88jmvLb4Nc+euCY2TABHABTuNWsouuE1LaxVnT65ZsASpdRXSqmhSorFHZWbDCjQZVB8AA5tAhyQu6NOh12yBDp2hDVr6h5icnYyEcERNI1oSkxYDADhQeHsyK4gxox5ULQf6p0ODTrz8cfwzDN1j8HfJSdDeLgZGNSunfkQEhUFKSk1O05IYAibMjeRnpvu/iCLMqB+Jwiqb56PuVuh+CDkJFf50EaN4JNPzLw+T0rOTiY8KJzswmyc2kloYCiNIxpX/Fy0yqFNprm35KAZwVpWCAHhdX5NC+tUmeS01o8A7YH3gNHAFqXUs0qpOpattYHIePOpr35H00xUVgCHNkJQZJ0O27q16Y8rKqpbeF+u/ZKokCjyS/JpFN6IBqENALNadVxU3IkP2HV4AvhwUIp58+C77+oWgx3Ex584MCM/3/TN1dTCWxcyrt84t8R1jMh40x8X2Q5CXUNhC9OgLNckuypcc43nR9DGR8VzoOAA27K2kZlvKlVX+ly0wo4pULDXXA0HhJkPDYERZlWIyDiro/N7SqmOSqm/lFJFSqkHvHXeao2u1Kbjbq/rqxSIBqYqpf7Pg7H5vk7jTHs9TpPoAiPNz9lrYPXjpv5gLTRtapZ56dmz9qGVlJXw1NynCA0KpdhZTF5xHlpr8orzKHYWM67vcW+0Wh+tctLyCsAMm//rr9rHYBfjxkFxMeTlmT9TXp65Pc4DuarWDj8Xy/IhLBbCWgIOKMqEGb0ge12Vh1i61FR28VQ3/bi+41BK0ap+K6JDoyt/LnqbswSW3Q8Lrgec5nUc3sr0yZXmmb9rJ1/6Z/utA8C9wIvePGl1+uTuVUotA/4PmA901VrfBfQATu2leJsnmg7psFgoOQQxCRB/I6Bg7RNmlGJxVq0Pn59PjZe3ycjLoNRZSlBAEDNGzuCP0X8wKXESsfViySrMIrZebMWDTg4shfzdENYCGh5dBzFQpiqRmAiTJpkrnaws833SJLO9pgpKCujzXh9eXfiqe4Ms/1wszoL67aH3O2ZVg5zN8GtvSPnqpIdYsgQmTzYLxXpCYvtEXh/2Ou1i2pFTnFP5c9GbCvbCrEGwaSKoQEh43UywD29h/o5hsb486KRWlGKoUsxSiu2u73WumaaUilNKbVRKvauUWquU+kwpNVgpNV8ptUUp1UtrvU9rvQQoccOvUX3VKPHyJNCmkvs6eaoUS/kvvyvrlTpD669jTPml79tqfWBljQ+RkaF1eLjWEyZU/zGZeZm6xUst9L9n/rvG59Mrxpt4F9+ttdb6u++0vvZarbOyan4ocXLXTr1Wf7LqE++crCRX6z+vO1oObNm/tC4rqXDXggKt8/I8F8o7y97Rz859Vpc5PVy3rrr2zdd6Wqz5u0yLNbf9Tw1LdumhoLeBXg96qev7NtBDa3qsY49LHKaVryvm4mkZpv6lAv4GfFdu38cxqxd4PHdoravVJ/eo1rrCLnat9YY6ZVi7ir0Qhi4zc81yt8Ovfcww7xpo1Mis+3X++dV/TMPwhtyVcBfXdKnFom/lqpwA7NsHK1aYYeXCvaaMmMLIbiO9c7LACOj7KfRwXalsfAlmDzFLKR0nNNQMsAEzitTdFuxawMztM3G4aV28WjtcHm3WAChIg8bnwtDl0LivtXF5xzigCDjcy5zvuu2O9thkbYouO4F1wCxtstoaTBK0hFQ88ZTIOBjyJ7QdbQak/DUSlt7r6sOrnocfrrpfrqCkgHt+uYftWdvNY857mLOanVWzWA9uMKPKgmOgiVlE9bbbYMOGSlawFnXm1E4KSgq8czKloMO9MGg2hDaFfXMgqTtkLjph16IiU6bME6Nq3//b+/xy/S/uP3BNlBbAwtGwdIzpi+twHwyaBWHNrI3Le+I5muAOy3dtr6vyQ+Wc5W47sXCBbklynhQYBr3fh55vmsmlm1+D2QPNp8dq2rIFVq6s/P70vHSmrJ3CzG0zax/nkQEnl4HUC/S43OJcmkxowsRFE7174iauK5ZGfaFgj6sc2NvHjDQJCYFu3cwIX3fKLzHvq6GBFS1F5iW5yTCzLyR/bKYF9J0CPV41r81TRzIQfty2cNd2W5Ik52lKQfs7YfBcM6gjY775FJ0xv1oPv/hiePDBE7dvzNwIQFxUHJvHbOaOhDtqH+MuV1OlawL4kiWQkACrVtX+kKJykcGR3JlwJz2b12H4bG2FN4dBv8PpY8BZDEvuhEW3mvlgLm+8AaNHu++U6zPW0+zFZvy67Vf3HbSmUqfD9B6QtRIiT4OLFkLctdbFY50JQAhHE12467bHZ7srpZoppXYD/wQeUUrtVkp5fM0VSXLe0ugc00/X5HxTTeG3AbDptSrHa3/wgRnKX97s5Nmc8cYZTNtgktNJCy5XJW8nHFhm+m6aDQGgpMRU9TgVVwL3lqcHPs2gtoOsOXlAMCS8Bn0+NhU9tr8PM/ubyj0uZWWmfqo7hASEMKLzCM5udrZ7DlgT2glrn4Y5w8xoyeaXuBah7er9WHyA1kwH7gbSgBjX97td2+twXL1Da92l3O3RWuup5e/TWu/VWrfUWtfXWke5fj5Ul/NWh9Su9DZnKaz8N2x82dyOGwm93obA41sQKlfqLOX/5v8f9/W+j4jgiLrFs3EiLL8fWl0J535dt2OJGknPTUcpRZOIJtYFkbUS5l4BeckQ0hD6fg6xQ3jrLbjrLli+HM62IDe5RXE2/HUD7PkRUND1cejyiCnFZx9SgaoKtvpv+wVHIHR/Cfp9Ya6ednxqRl/mbKv0IXPmwH//t57hXw4npyiHQEcg/zn3P3VPcHC0P67V0QK5fvC5x+8dLDxI85eb8/bSt60NJPosc2UTO9SUdJszFNY9zzVXa776Crp0qfIIJzV/53xrynZlr4XpPU2CC4qC83+Cro/aLcGJapD/uFXaXA0XLoJ67SF7NUxPgD0VjzybOhVe/2gvi3cvJjnbjf3DhRmmXqUjCJpfDMCBAxATI0vreFqD0AZMvmQyIzr7QD2FkBiTBLr81zTvrRpP1JoRXHX5IYLqMCZDa83tP93O9dOud1+s1ZHyJczobep3Rp1pkniLYd6NQfgMaa60WvFBV5PKD5gmlcfMm41yUOosZcmeJZwe3ofISHA63Lze1rb3zKCD2KFwgVkoNTUVnnoKbr65bmXFhJ/a/SP8NQpKDqLrdeCrtGkUhXbmhhtqd7hdB3exv2B/zae11IazBFb8Gza9Ym7XoivAD0lzZRXkSs5qwQ3gvG+h29Pm9prH4Y+/QXE2j895nAEfDSA3IIWQENy/oOSuE5sqmzeHN9+UBOcNpc5S5qXMOzLH0Se0vBQuWgINuqByNnFpcG/2Lq5gSfRqatWglXcSXEE6zB5sEpwKhB6ugTX2TnCiGiTJ+QLlgC4Pw4AkCI6G1J9gegIPdLqQjy7/iDZRbZg5Ey65xIx8dIuSQ7B3JqCgxd+ObD7k8bFO4rC84jwGfDSAD1d+aHUox6rf3gyxb3MN4cG5PNj3KljxoBk0VU3bs7Zz/bTrvdMfl7kQpneHfXMhtBkMngMdxrhv5WHh1yTJ+ZLmF/FFy3EkEwW524iaO5RrIs1KBvn5Zv2y3bvddK7UJDNPqnE/CDPLQWtt1ksbO9ZN5xAn1SC0ATNHzWTsOT74Bw+MMJOlu78CKsAsGvr7RaYftxpWp69mxtYZBAcEey5GrWHLm2ZSe0EqNO4PicvNc1r4HKXU9Uqp1a6vBUqpM71xXklyPiZDRXKfM4GSNteZcmALrodl93PZJSWsWWPWNnOLI02VVxzZVFoK48fDMOmj95qB8QOJDou2OoyKKQUd72dVo9nsO9QU0mebCdX7l1T50Ms7Xk7qv1JpXq+5Z2IrLYCFN8GSf5i+uNNdZcvCPLwonqiLZOB8rXU34ClgsjdOKgNPfEBKdgrpeen0atHLVM5G40DB1rdg2X3mRdz4XOj/Fc6QZihVx5aYskL4pjGU5sJlybIgpIVyi3P5Yu0X9GrRi25Nu1kdToUyM+Hf9+zhtauvJDx/oVlnLWEStLutwv1zi3OJDK7bwsEnlZsM80ZA1gqzuGmvdyDeyyM4fUeN3wnUE2oopiBzPCbxTNCP6TpNBldKxQHTgT+Bc4BVwAfAE0AT4Hqt9eJy+0cDa7XWLepy3uqQKzmLaa25btp13PDtDZQ5y1BKmSrtSkH7u2DQHxDWHDLmUfxDd4b3X8DChXU86d5ZJsFFn3VMgtuxwywIKrzrrp/v4ruN31kdRqUaNYL3Pm9B+KV/QPt/mGbuxbefUA4MzPO573t9ueunuzwTTNqvZrpN1gqIbAsX/nUqJ7gacyW414FYzCKmscDrru111Q6YCHQDOgLXAf2BB4D/HLfvLUCSG85ZJanGaxGtNU7tJMARwDuXvkOgI5AARwUl/xv3MeXA5l9N8L65TP3HANLyXgH9j9pfzh2pVXnFMZtHjQKnE+ZXr6ymcIPI4Ei23LOFNg3aWB1KlTKzgtmhXifhnF6m5uW29yBrFZz7DUSYis6lzlJGdhvJadGnuffk2gnrnoPV/wU0NB9mlhAK9tGmXt9V0VI7h7fX6WoO11I7AEqpI0vtKKWOWWpHKXUBJsn1r+P5qkWSnAVKykq4eurVdGrUiWcGPUPnxp1P/oCwZjDwN1jxIEGbXqX1vjHw1yLo9VbNh0g7S11z8jhm6gDAf//rmXXExMnFRcVZHUK1XHklpKXBxo03oqK6wbwrzIry03uYCj7NBhEUEMSD/SqoKF4XxQdh4Y2w+3tzu8tjUr2k9uIxV3DleW2pHaVUN+BdIFFrvd8N56ySPEssEBQQRNOIpjQKr0EFZEcQ9HgF+k5BB4TDjk/g175mUdaayJgPRZmm0kqDM46568ILITGxZocTdbc/fz/jfh3H/J2+fQn9f/8H06a5GhBiznaVA7vIPJ9+v5BDKx5h+pYknNrpvpNmr4MZvUyCO1yeq9vjkuBqz7KldpRSrYFpwCit9WZPn+8weaZ40bcbvmXnwZ0AvHHxG4ztU/Oh4wejr6XHfxayv7gdZK8y/ROpNWhlKL+sTrnmzo0bYfVqqVtphbCgMN5e9jar0n17baNeveCM8p+LQhrC+T/DGY+AdlJ/wzPkzh7G7v0b3HPClK/g196Qs9msGjB0KbS42D3HPnVZttQO8CjQEHhDKbVSKeWV0YQyutJL9ufvp+3/2nJtl2t565K36nSsN96A8/pk0+XgKDNxHAXdnoQz/nPyT7haw/dtIH+X6bBvdM6Ru269Fb791oykkzm03ldcVuzZOWVukpYGzz4L998Pp5Xvdtv9PfqvG1Alh6B+Rzj3W2jQsXYncZbCyodg40vmdpvroPdkM3dPHM8nRlf6MklyHpZfkk94kPnQtDxtOWc0PoOQwBD3HFw7Ye0zsOYxQEOLS00po+CoivffvxRm9DSjNS/fdUxCTE6G7dthkEVLnAn/kJoKp58OkyfDddcdd+ehzaaf7uA6CKwHfT48Zh5mtRTugz+vhn1zTHmu7i/B6ffIJ6/KyR+mCtJc6UHbDmyj46SOfLH2CwC6x3Z3W4JbsQLmL3BA1//CgJ9Nf8WeH83yItlrK37Q4WV1Wl5+whVffLwkOCul5qRy6eeXWrt6djU0bw57956Y4G76/ia+S10PFy6E1n+H0hwzl23leHBWczRT5iJI6m4SXGhTM7m7w72S4ESdSJLzoFYNWtG/dX/ax7R3+7Fvvhkeesh1o3kiJC4zy4rkbjXLjOz44sQHHe6PO+7T9dat8M03MkfOSjFhMaRkp5BdmG11KFWKdM3zLi423w8UHGBZ6jL2HNoDQZFmpOXZL5lyYOufN2vUFWZWfkCtYcvbrvJce6BRXxi6HJqc6/lfRtieNFe6WWFpIf83//94oO8DR5opPWHlSmjRAho3LrexNB8W32lGXgJ0GAtnv2BGZh7cCD93MvOKrkg321wmTIAHH4R9+447nhCVeOghmDHDrByulJn3WabLCHSUm5WUPgfmX22aIMNbm/l0DROOPVBZISy5G7a/b26fPsYkSD/on/QRcplbBcvmySmlAoClwB6t9SVWxVFXSVuSmLBgAsnZycRHxTO03VCe+OMJOjfuzJWdr/TYec86q4KNgeHQ5yNo2AuWjzXLjmQtg3Z3uCbRAo4Q2PubufpzufdeGDxYEpyv0FqjfLyJ7uyzTXLLys2jXngwQQFBBKrj3k6aDjCFDOZdCfsXwcz+popP9ipTmiss1hR8zt0KAaHQazLEj7Lk9xH2ZdmVnFLqn0ACUL+qJOerV3JJW5IYkzSGYId5kZeUlVDsLGZ8v/Hc2uNWj59/+nTTNzd+fAV3ZsyHP6+CgjTTbIQDdAmEtTCJruekYxKdsN72rO0M/XQozw9+nis61XDAhkWenvs0by19i/V3r6d+SP2KdyorgmX3m1qsYPqPQxpC3g7QZWZ5nAuSTJk5UVO+/WnIB1jSJ6eUaglcjJn57rcmLJhAsCOYorIi1mWsQylFsCOYKWuneOX8s2eb6QSH+0aO0bif+RQd1MC8kegSwAGhTUxT0AYzLSYjwwwJT0nxSsjiJFrWb0mXJl2ICo2yOpRqi8nrx1Xtb648wQEEhECvN83UAhSUZEPuNvO8DIiAeu0kwZ0ilFJxSqnjx+V6lFUDT14FHsSUe6mQUup2pdRSpdTSjIzqrWHlbcnZyYQHhVMvuB4NwxoSEhBCeFC4dxaKBB591BRVDq6s+yIs1vWp2dUOGRxlRlUGhEOuiXHFCnj4YTeuUydqLTggmGlXT2Ng/ECrQ6mWPXtgzMUXELn4yeo9oKwQ6nUwqxgAhMZCvdMhf4/nghS+Jg5TuPkESh3f3u0eXu+TU0pdAuzTWi9TSg2obD+t9WRc6w0lJCT45OiY+Kh40nLSiAiOoHUDU6A2rzjPa7UID49yO6l6bU2TZWiTo28uZflHVh+48EI4cKCaxxJeUVBSgFKK0MBQq0M5qT8OTGHKtESGDaxmkeTIePNcbHCGWT4qIARK82SpJ2+bcuJkcK6r81I7TwGZWuuJrtvPAOla6/8dt+vzQCel1ErgIyAL06oXCkQopZ4EHjjchaWUmgQs1Vp/qJTqAbwMRAKZwGitdVpVsVlxJdcPuEwptQP4AhiolPrUgjjqbFzfcRQ7i8krzkNrTV5xHsXOYsb1Hee1GJKSoG9fs3J4hTqNg7Ji0zSEMm8qZcVmu0t0NAQFVfJ44VUbMjYQ9UIU32/83upQTmrL/i1cP+169jR5n/onaak8xuHnYlmB+cBVwXNReNiUipfacW2vi/eAGwGUUg7gGuCzCvZ7CJintT5La/2Ka1sf4EatdaVNGEqpIOA14EqtdQ/gfeCZ6gTm9SSntR6vtW6ptY7D/CFma61HejsOd0hsn8ikxEnE1oslqzCL2HqxTEqcRGJ77w3oCA42o9zSKvs80zzRDDIJi4XiLPPdNejE6YQ77oA//vBauKIK7Ru2Z+w5Y+nUuJPVoZxU+4btWXHHCm7pfguzZsGYMdV40Emei8JrKlpqp8i1vda01juA/Uqps4ELgRU1WGVgptb6+JURjtcB6ALMdF0FPgK0rM7BZamdOkpsn+jVpHa8QYOqUamkeWKFbyR795p6lb17w/nneyY+UTOBjkCeH/y81WFUy1nNzgJg0yb48UfTR9ykSRUPquS5KLzGk0vtvAuMBpphrrSqq3wZilKOvfg63GavgHVa6z41DcrSiida6zn+PEfOl5SU1HwtuObNIT0dbrjBMzGJ2tFaszFzIwcLD1odSoXeWvoW/5rxL0qdpYAp7r1tWzUSnPAFnlxq51tgKNATmFHJPjlAvZMcIwXorJQKUUo1AA5/hN8ENFZK9QHTfKmUOqOyg5QnZb1sYPly8wbz++81f6xSECjX8z5lVfoqOr3eiZ82/2R1KBXaemArq9JXHaluEhxsnkNOJxQVVfFgYTWPLbWjtS4Gfge+0lpX9pF7NVCqlFqllDphrTGt9S7gK9d+nwEryh37SuAFpdQqYCXQtzpxSVkvG8jPh3vuMV8VVkKpxG23mcoV//iHx0ITteDUTj5Y8QFD2w2lRf0WVodToTJnGQGOgCO38/IgIQGuvx4eecTCwE49NZ8M7oHRlXBkwMly4Cqt9Za6Hs9d5DO8DYSHw3vv1ewxWpuldVr45nvoKc2hHNzS/Rarw6jQgYIDxITFHJPgACIiYNgw6NLFosBE9ZmE5tb145RSnYGfgG99KcGBXMnZyp49pvmxeXOrIxF1lVucy6zts+jTqg9NInyjs2tj5ka6vdmNz0d8zojOI6wORxg+WdZLKdUV+OS4zUVa697ejkX65GwiNxfatoWJE62ORLhDclYyl395Ob9s+cXqUI6oH1Kfe3vfy7ltKl8Cp6DAjLQUpzat9RrXXLjyX15PcCBXcrYyZYrpFzn99Kr3vfdeOHQIPvzQ42GJWnBqJ/N3zqdXi17uW0neC159FcaOhbVr4YxqjX0TdeSTV3K+RPrkbOT41ZpPJioKAgKq3E1YxKEcJ71i8rbZybOJDo3m7NizT7rfjTfCmWdC585eCkyIKsiVnM3Mm2dGW150kdWRiLram7uXj1d9zLVdrqVVg1aWxpIwOYEARwCLbl1kaRzekJRkFhJOTob4eBg3DhJ9d/66XMlVQfrkbOahh0zliZPxg881AsgqyOLfv/2bBbsWWB0KM0fN5IO/fVCtfZ1OePrpmo/49QVJSaZEWVoaxMSY72PGmO3CP8mVnM1s2QLNmkG9k9QUePpp+PhjWLMGQvynu+eUo7UmPS+dZpHNrA6lxgYMgPbt4Z13rI6kZgYONIktIuLotrw8iI016zf6ILmSq4JcydlM+/YnT3AAnTqZ5kxJcL5NKWV5gkvOSibxs0Q2ZGyo0eOmT/e/BJeWZj74hR9X9Co83KzbKPyTJDkb+v57049QmREj4LXXvBePqL2tB7Zyy/e3sPXAVkvOvy1rG+sz1lMvpIpPTscJdZXVzcs7+X5WKy01zatgRhpnZsLB40qG5udDXJy3IxPuIknOhlauhO++g8LCE+8rLKx4u/Bd32781rIkN7jtYJLvS6Zl/WqtanKMX36Bpk1h40YPBOYGa9ZAmzZHa77efju8+67ps87LO/q9uPjkHxqFb5MkZ0MPPQSbNx/9NF3ejz+a5sx167wfl6i506JPI/PBTIa2q+ualjW3L28fWmscqnZvEwkJcPXVpoCzL3A64euvYYarPn779tCvH0RGmtsNG8Itt8CkSaYPLivLfJ80yadHV4oqyMCTU8yaNWbS+BNP+M6bj/A9WmvOfvtsOjTqwJdXfml1OHWSnW3mhWptJqh36GDWUbQJGXhSBbmSs6mffzYv6OP7F7p2heeekwTnT1akraDf+/1Yt897l99O7eTe3vdyXZcaVBioxPbtMH++G4KqhQcfNM/50lJT13X6dJg61ZpYhDWk4olNNWoErVpBRgY0aGC2lZbC1q2m7JdDPt74jeiwaMqcZRws8t4iqgGOAG4++2a3HOv6601t1dWrTaLxpORkePlleOopc/V24YXmtVBSYta8a93as+cXvkeaK08hq1ebkkuffVazEmDi1JKak8rvyb9zZecr3VI3c+VKaNzYc8s6HTpkkljDhrBsmeln++EHk+BOAdJcWQX5PG9zh0eHgVmC5/33zURd4X+01njjQ+mUNVMY9e0o9uTsccvxzjrLcwkuL89cnb3wgrndowekp58yCU5UgyQ5G1u50jTVTHctj9ioEdx0k6w354/mpsylyYtNWJW+yuPn+meff7LktiW0jW7rtmNu325aD5KT636s996D8ePNzxER8OyzZhTnYYeb54UASXK2dsYZ8I9/mHXmwJQlysiwNiZRO6dFn8aw9sMIcgR5/FwO5aBH8x5uPWZwMMycaZbgqSmnExaVqwu9Zo0ZyFJWZm7/4x/mCk6Iikif3CkiLw/q14eHH4Ynn7Q6GuGr/v713xnWfhijzxrt9mOXlEBQLXL05Mlwxx2mT7lr19ofx6akT64KMrryFLBpkxlNOXeuKd4s/FdmfibRodEEONy/GGBOUQ6Z+ZnkFOW4/dhwNDHl5Jy8vmp6Otx5J9x2GwwbZsrQhYdDu3bHHkeI6pDmSpsrKDAjKi+4AEaONG8csmyIf/p58880ntCY5WnLPXL8eiH1mH3jbMb0GuOR44OpHNK8uakFOXDg0ediSsrRJsmYGNN3t3+/ud2woXnuhoV5LCxhY5LkbG7OHDNfqLTUrDog62P5r54tevL0BU97ZGWCnKKcI1dwykOT2ZKSYPlyk6xiYiA19ehzccQIuOsus19QEKxYAaNGeSQMcYqRPjmbO7w+1vbtZiRa27Y+vz6WsMCz857lhfkvsP3e7TQMb+iRc5Rfq23/fti71xRIbt7cTAFo0sTcFjUifXJVkD45m0tONp+aO3Y8uqSIrI/lv0rKSliSuoSezXsSFOC+zqmh7YbiUA6PJTg4+lwEU30kNNSMutyxA3r29NhpxSlOmittLj7erIcVFHR0kVRZH8t//bDpB/q934+lqe5t2ege252H+j/k1mMe7/BzEcxcttNOMyMl5bkoPEmSnM2NG2cqnsj6WPYwMH4gU6+aSufGnd12zMnLJrPr4C63Ha8y8lwUVpAkZ3OJibI+lp1Eh0UzovMIGoS6p6zHzoM7uevnu/hi7RduOd7JyHNRWEEGngjhZ9Jz0/llyy9c1/U6txRQTslOoUFoA6JCo+oenPA2GXhSBbmSE8LPLNy9kJt/uJnFexa75XhtotpIghO2JUlOCD8zqO0g1ty1hv6t+9fpOG8seYMbv7uR4rJiN0UmhO+RKQRC+JnI4Ei6NOlS5+McLDxIRl4GwQGyTLywL7mSE8IPbczcyPjfxlNQUlDrY4w/dzw/X/ezG6MSwvdIkhPCD207sI0X/3qRtftqsXYNZlQleK6ElxC+QpKcEH5oyGlDyP53Nj1b1LxUyOb9m4l7NY4PV37o/sCE8DHSJyeEHwoOCK51X1qj8EY8M/AZhrYb6uaohPA9ciUnhJ9asGsBI74aQW5xbo0eFxMWw/hzx3tkNQMhfI0kOSH8VG5xLiv3riQlO6Xaj0naksTMbTPxhyIQQriDVDwRwk9prWs8cOT8D88nrziPJbctkUEn9iD/xCpIn5wQfqo2SWrGyBmk5qRKghOnDGmuFMKP/bjpR7q80YVDRYeqtX9oYChto9t6OCohfIckOSH8WFRoFK0atCIzP/Ok+6Vkp9D73d4sS13mpciE8A3SXCmEHzu3zbkktUmqcr/UnFQKSgpoHNHYC1EJ4Ttk4IkQNlBSVkJQQJDVYQjvk87VKkhzpRB+7uNVHxP9QjRZBVkV3p+SnUKps9TLUQnhGyTJCeHnujTpwq3db6WorOiE+7TWDP9yOBdPudiCyISwnvTJCeHnusd2p3ts90rvf3zA4wQ65KUuTk3yzBfCBrTW7Dy4kzZRbY7ZrpTisg6XWRSVENaT5kohbGDioonETYwjIy/jyLa0nDReXfhqtefQCWFHciUnhA0MbTeU0MDQY1Ym+HHzj/xzxj+55PRLqB9S38LohLCOTCEQwsa2HthKu5h2VochPEemEFRBmiuFsIlDRYeYtX3WMdskwYlTnSQ5IWzi7aVvM/iTwaTlpHHp55fyyl+vWB2SEJbzep+cUqoV8DHQDHACk7XWE70dhxB2c02Xa+jRvAcRwRGEBYZJBRQhsGbgSSnwL631cqVUPWCZUmqm1nq9BbEIYRtr961lwoIJJGcnEx8Vz2nRp1kdkhCW83pzpdY6TWu93PVzDrABaOHtOISwk6QtSYxJGsP2rO0UlRaRlpPGmKQxJG2punizEHZmaZ+cUioOOBtYZGUcQvi7CQsmEOwIpqSshLTcNIICggh2BDNhwQSrQxPCUpbNk1NKRQLfAPdrrU+YraqUuh24HaB169Zejk4I/5KcnUxMaAzhQeFEh0UTHBBMkCOIHdk7rA5NCEtZciWnlArCJLjPtNbTKtpHaz1Za52gtU5o3FjWwBLiZOKj4skvyUcpRWhgKAD5JfnERcVZG5gQFvN6klNKKeA9YIPW+mVvn18IOxrXdxzFzmLyivPQWpNXnEexs5hxfcdZHZoQlrLiSq4fMAoYqJRa6foaZkEcQthGYvtEJiVOIrZeLFmFWcTWi2VS4iQS2ydaHZoQlpKyXkII4b+krFcVpOKJEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsy5Ikp5QaqpTapJTaqpR6yIoYhBBC2J/Xk5xSKgB4HUgEOgPXKqU6ezsOIYQQ9mfFlVwvYKvWervWuhj4AvibBXEIIYSwuUALztkC2FXu9m6g9/E7KaVuB2533SxSSq31Qmx10QjItDqIKkiMdefr8YHE6C7+EONarXUXq4PwZVYkOVXBNn3CBq0nA5MBlFJLtdYJng6sLiRG9/D1GH09PpAY3cVfYrQ6Bl9nRXPlbqBVudstgVQL4hBCCGFzViS5JUB7pVS8UioYuAb4wYI4hBBC2JzXmyu11qVKqTHADCAAeF9rva6Kh032fGR1JjG6h6/H6OvxgcToLhKjDSitT+gOE0IIIWxBKp4IIYSwLUlyQgghbMunk5w/lP9SSr2vlNrnq/P4lFKtlFK/K6U2KKXWKaXuszqm4ymlQpVSi5VSq1wxPmF1TJVRSgUopVYopX6yOpaKKKV2KKXWKKVW+urwcqVUlFJqqlJqo+t52cfqmMpTSnVw/f0Ofx1SSt1vdVzlKaXGul4ra5VSnyulQq2OyVf5bJ+cq/zXZmAIZtrBEuBarfV6SwM7jlLqPCAX+NgXJ2UqpWKBWK31cqVUPWAZcLkv/R2VUgqI0FrnKqWCgD+B+7TWCy0O7QRKqX8CCUB9rfUlVsdzPKXUDiBBa+2zk5iVUh8B87TW77pGWIdrrbMtDqtCrvehPUBvrXWK1fEAKKVaYF4jnbXWBUqpr4BftNYfWhuZb/LlKzm/KP+ltZ4LHLA6jspordO01stdP+cAGzBVZ3yGNnJdN4NcXz736Usp1RK4GHjX6lj8lVKqPnAe8B6A1rrYVxOcyyBgm68kuHICgTClVCAQjsw1rpQvJ7mKyn/51Juzv1FKxQFnA4ssDuUErmbAlcA+YKbW2udiBF4FHgScFsdxMhr4VSm1zFUaz9e0BTKAD1zNvu8qpSKsDuokrgE+tzqI8rTWe4AXgZ1AGnBQa/2rtVH5Ll9OctUq/yWqRykVCXwD3K+1PmR1PMfTWpdprc/CVMDppZTyqaZfpdQlwD6t9TKrY6lCP611d8wqH3e7mtN9SSDQHXhTa302kAf4an97MHAZ8LXVsZSnlIrGtGrFA82BCKXUSGuj8l2+nOSk/JebuPq5vgE+01pPszqek3E1Xc0BhlobyQn6AZe5+ry+AAYqpT61NqQTaa1TXd/3Ad9imv19yW5gd7kr9amYpOeLEoHlWut0qwM5zmAgWWudobUuAaYBfS2OyWf5cpKT8l9u4BrU8R6wQWv9stXxVEQp1VgpFeX6OQzzIt5oaVDH0VqP11q31FrHYZ6Ls7XWPvXpWSkV4RpchKsJ8ELAp0b9aq33AruUUh1cmwYBPjMI6jjX4mNNlS47gXOUUuGu1/cgTF+7qIAVqxBUSy3Lf3mdUupzYADQSCm1G3hMa/2etVEdox8wCljj6vMC+I/W+hfrQjpBLPCRaySbA/hKa+2TQ/R9XFPgW/O+RyAwRWs93dqQKnQP8Jnrw+t24CaL4zmBUiocM7L7DqtjOZ7WepFSaiqwHCgFViDlvSrls1MIhBBCiLry5eZKIYQQok4kyQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcENWglOqplFrtWvsuwrWWl0/V1xRCnEgmgwtRTUqpp4FQIAxTf/E5i0MSQlRBkpwQ1eQqQ7UEKAT6aq3LLA5JCFEFaa4UovpigEigHuaKTgjh4+RKTohqUkr9gFlmJx6I1VqPsTgkIUQVfHYVAiF8iVLqBqBUaz3FtVrCAqXUQK31bKtjE0JUTq7khBBC2Jb0yQkhhLAtSXJCCCFsS5KcEEII25IkJ4QQwrYkyQkhhLAtSXJCCCFsS5KcEEII2/p/fNA/4D+MbUQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "#hide_input\n", "\n", "fig = sns.lmplot(x = 'x', y= 'y', data = df, hue='Values', palette=[\"b\", \"g\", \"orange\"], fit_reg=False )\n", "fig.axes[0,0].plot(x, m1, linestyle='dotted', color = 'b')\n", "fig.axes[0,0].plot(x, m2, linestyle='dotted', color = 'g')\n", "fig.axes[0,0].plot(x, y_true, linewidth = 2, color = 'orange')\n", "\n", "plt.xlim([0, 8])\n", "plt.ylim([0, 10])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To get a better intuition on how good the two models fit the ground truth, we plot the residuals `y_true(x)-m(x)`." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#hide\n", "\n", "m1_res = y_true - m1\n", "m2_res = y_true - m2\n", "\n", "df_res = pd.DataFrame(np.array([x, m1_res, m2_res]).T, columns = ['x', 'm1_res', 'm2_res']).melt('x')\n", "df_res.columns = ['x', 'Residuals', 'y']" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAFuCAYAAADtZD0yAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA5HklEQVR4nO3dd5hV1fn28e8zDWZoQxORIigEROxgxIZiA7uJiTXGxJYo1ghRY4kmIa8Zk59JsCOKwRK7Rh1FxQKiSAcRBOlNOsLMwNT1/rHOUGRg2jln77O5P9c118xpez9sZs591tprr2XOOURERKIiLegCRERE4knBJiIikaJgExGRSFGwiYhIpCjYREQkUjKCLqA2+vfv7959992gyxARSRYLuoBUlFIttjVr1gRdgoiIhFxKBZuIiEh1FGwiIhIpCjYREYkUBZuIiESKgk1ERCJFwSYiIpGiYBMRkUhRsImISKQo2EREJFIUbCIiEikKNhERiRQFm4iIRIqCTUREIkXBJiIikaJgExGRSFGwiYhIpCjYREQkUhRsIiISKQo2ERGJFAWbiIhEioJNREQiRcEmIiKRomATEZFIUbCJiEikKNhERCRSFGwiIhIpCjYREYmUwIPNzNLNbIqZvRV0LSIikvoCDzbgRmBW0EWIiEg0BBpsZtYeOAMYFmQdIiISHUG32B4EBgMVAdchIiIREViwmdmZwCrn3KRqnne1mU00s4mrV69OUnUiIpKqgmyxHQOcbWYLgReAfmY28odPcs497pzr5Zzr1bp162TXKCIiKSawYHPO3e6ca++c6wRcCIx2zl0aVD0iIhINQZ9jExERiauMoAsAcM59DHwccBkiIhIBarGJiEikKNhERCRSFGwiIhIpCjYREYkUBZuIiESKgk1ERCJFwSYiIpGiYBMRkUhRsImISKQo2EREJFIUbCIiEikKNhERiRQFm4iIRIqCTUREIkXBJiIikaJgExGRSFGwiYhIpCjYREQkUhRsIiISKQo2ERGJFAWbiIhEioJNREQiRcEmIiKRomATEZFIUbCJpKj8ufn0G9GPzv/sTL8R/cifmx90SSKhoGATSUH5c/MZmD+QFZtW0KJhC1ZsWsHA/IEKNxEUbCIpKW9cHpmWSXZmNmZGo6xGZKVlkTcuL+jSRAKnYBNJQQs2LABg2sppbCreBEBOZg4LNywMsCqRcFCwiaSgzrmdKakoIbdhLkWlRRSXFVNUWkSn3E5BlyYSOAWbSAoadPQgzIzmDZqzdNNSVhWuoqSihEFHDwq6NJHAKdhEUlDvdr2549g76JjbkXZN2tGjdQ+GDhjKgK4Dgi5NJHAZQRcgIrU3YuoIbn3/VhbftJgOzToEXY5IqCjYRFLQzw/8Oa1yWtGhWQeKy4q595N76bVPL35ywE+CLk0kcOqKFElBHZp14JeH/hKArPQsXpn1ChOXTwy4KpFwUItNJMWMWzKODVs20L9Lf9IsDTNjxm9nkJWeFXRpIqGgFptIivnX+H9xzVvXYNjW+xRqItuoxSaSYkacO4L56+djZjvcf83/riEzPZOhpw8NqDKRcFCwiaSYBhkNOKD1ATvd3zirsVpuIijYRFLK/33+fzTOasxVR1y102N/P+3vAVQkEj46xyaSQt6e+zYfLPhgt8/ZWLwxSdWIhJNabCIp5IPLPmBL2ZZdPn7t29cyat4o5l4/d6dzcCJ7CgWbSIppmNFwl48N6DKALi26UO7KyTD9ecueSb/5Iini1P+cytndzmbgkQN3+Zyzup2VxIpEwknn2ERSwObSzTTOakyD9AbVPrekvIRxS8YloSqRcFKwiaSA7MxsXr3g1SpHQ/7Q38f9nWOGH8PyTcuTUJlI+KgrUiQFFJYU0iirUY2ee/FBF3NQm4Nomd0ywVWJhJNabCIht7pwNS3/1pKnpjxVo+fvm7svZ/7oTBpkVN9tKRJFCjaRkKtwFdx81M30bte7xq9ZWbCSf4//NwUlBQmsTCScFGwiIdemcRv+evJf6blXzxq/ZtaaWdzw7g18uujTBFYmEk4KNpEQ21K2hSkrpuCcq9Xrju14LN8M/IbTu56eoMpEwkvBJhJiH8z/gMMfP5zRC0bX6nUZaRn8qOWPElSVSLgp2ERCrE/7Pgw/ezjHdjy21q9dVbiK69+5nvFLxyegMpHwUrCJhFjLnJb86rBf1WmEY05mDiNnjGTGqhkJqEwkvHQdm0hIzV4zm2nfTePsbmeTnZld69c3zmrMqltXkZmemYDqRMJLLTaRkHp+xvNc+tqlFJcX13kbCjXZEynYRELq7r53M+WaKeQ2zK3zNkrLS+k/sj8PjHsgfoWJhFxgwWZmHczsIzObZWYzzezGoGoRCaP0tPRaXbtWlcz0TJpnNycnMydOVYmEn9X2+pi47disLdDWOTfZzJoAk4BznXNf7+o1vXr1chMnTkxajSJBeX7G88xbP487jruDNFPHyh5Mq8XWQWB/Mc65Fc65ybGfNwGzgHZB1SMSJmMWj+HVWa/GLdScc2ws3hiXbYmEXWAtth2KMOsEfAr0dM5t/MFjVwNXA3Ts2PGIRYsWJb9AkQAUlxXHbSLjk545iYYZDXn74rfjsj1JGrXY6iDw4f5m1hh4Bbjph6EG4Jx7HHgcfFdkkssTCUw8Z+e/uOfFZKQF/ucukhSB/qabWSY+1J51zr0aZC0iYfHrN35Ny+yW5J2aF7dtXnH4FXHblkjYBTkq0oAngVnOuX8EVYdI2DTMaEjDjIZx3+6m4k1MWDYh7tsVCZsgR0UeC4wBZgAVsbvvcM69s6vXaFSkSN1d/vrl/G/O/1h560p1S6YOnWOrg8B+u51zY9F/msgOCkoKaJzVOCHbvvmom/nVob/C9GcnEacLZERCoryinK7/7sofPvxDQrZ/yN6H0LdTX9LT0hOyfZGwULCJhERJeQnXH3k9J3Q6IWH7mLduHv8a/69aL1wqkkoUbCIhkZ2ZzR3H3cEp+5+SsH18MP8Dbnz3Ruatn5ewfYgETcEmEhJfLP2CsoqyhO7jwp4XsuyWZXRp0SWh+xEJkoJNJATmrJ1Dnyf78PikxxO6n2YNm7FPk30Sug+RoCnYREKgfdP2vPSzlziv+3kJ39fsNbO5/PXLWb5pecL3JRIEBZtICORk5nB+j/Np26RtwvdVVlHGG9+8wazVsxK+L5EgKNhEAramaA3DJg9j/eb1Sdnfga0PZNWtqzhpv5OSsr8wys+Hfv2gc2f/PT8/6IoknhRsIgEbNW8UV/3vKhZuWJiU/ZkZmemZSdlXGOXnw8CBsGIFtGjhvw8cqHCLEgWbSMAu6nkR038znUP3PjRp+1y2cRlHP3k0r87a8+Yez8uDrCxo1AjM/PesLH+/RIOCTSRgZsZBbQ7CzwueHG0atyE7M5t027NmIXnqKRg3DrKzd7w/JwcWLgykJEkABZtIgD5e+DGD3x/Mhi0bkrrfjLQMPrzsQ87pfk5S95ts5eXw1ltQWOhv5+RA48ZQULDj84qKoFOnpJcnCaJgEwnQ5BWTeWrqU+Rk5gSy//KKcjYVbwpk38kwfjycdRa89JK/fcEF8J//+MArLATn/PeSEhg0KNhaJX4UbCIBuqXPLSy9eSlZ6VlJ33dZRRn7/Ws/7vn4nqTvO1HKy314/fnP/nafPr7Fdskl254zYAAMHQpt28L69f770KH+fokGLcokErAGGQ0C2W9GWgbX9b6Onnv1DGT/8bJmDUyY4IMpPd0PBMmMDfo0gzPO2Pk1AwYoyKIssIVG60ILjUqU3PfJfUxfOZ2XfvZSUgeORIFzPrQAfvtbGDECVq6EJk2CrSsB9ItRB+qKFAlIg/QGNMpqFHiorSpcxbTvpgVaQ21MnAgHHwxz5/rbgwfDl19GMtSkjtQVKRKQ3x/7+6BLAODcF86lrKKML6/6MuhSdmniRD9E/8ADoX17P7Jx3Tr/WOfOwdYm4aNgEwlAQUkBjbMaB10GAPeffH9oatleZXdjcTGcdhr07w/PPgt77w2ffx50dRJmCjaRAPQf2Z+2Tdry0s9eCroUjtv3uKBL2Mn998NHH8G770KDBvDmm9Aztce4SBLpHJtIAC7qeRHndAvPxdFTVkzhqSlPBbb/LVvghRf8cH2Apk2hdWvfWgM45hho1iyw8iTFaFSkiPC7937HY5MeY83gNTTMaJi0/VZ2N772GvzkJ76FdtppSdt9KtCoyDpQi00kySYsm8Dm0s1Bl7GDwccMZsnNS5IWaps2wUknwSOP+NtnngmjR8OppyZl9xJxCrY4yJ+bT78R/ej8z870G9GP/LnhWf9C607VXSKOXVFpEX2f7sttH9xW/43FUZvGbWie3Txu26vq2C1aBO+95x9v0sR3LVZORpyZCSeeuO3aNJH6UFdkPeXPzWdg/kCy0rLIycyhqLSIkooShg4YyoCuwU5tULnuVFaWn/y1qMjPiafpg6qXqGNXVlHGh/M/pH3T9hy414HxKzgOxi4ey6MTH+Xpc58mI63u48q2P3bZ2bB5sz92e+8N8+fD0qV+hhCpEUV9HajFVk954/LISssiKz2LkvISGmU1Iisti7xxwS/uVLnu1ObNsGyZf4PWulM1U3nsyst9SyNexy4jLYPTupwWulADf6H2xws/rveCp5XHzgy+/tr/XHksx49XqEniKdjqacGGBWRnZDN77WwWb1wMQE5mTtJWQ96dBQv8G/KaNX6yV9C6UzVVeewKCmDtWqioqP+xK68o59/j/83SjUvjVmc8ndPtHJbcvIQuLbrUazuVx66iwncxlpf726tWQceOcSpWZDcUbPXUObczm8s206FpBzo07QD48yidcjsFWxj+/EZREXTv7q8BMtO6UzVVeez22QcOP9y3Mup77KZ+N5Ub3r2BsYvHxq3OeEpPS4/L9F6Vx65xY+jWDRo21O+dJJeCrZ4GHT2IkooSMtMyaZDegMKSQorLixl0dPCLOw0a5LshKxdZXLXKf2ndqeqdey5s2LDt2K1f7yfZrc+xO2KfI5h3wzzO+tFZ8SgxISYun8hBjxzEjJUz6ryNX/3K/54VFGi9MwmGgq2eBnQdwNABQ2nbpC3rNq9jU8kmeu7VM/CBIwAdOvgT9RkZ/o25rAwaNfKj1GT3xo6F0lLYay9/7Mz8h4Tu3eu33f2a70ejrEbxKTIB2jVpR6ucVhSVFtV5GytX+lBr3lzrnUkwNCoyzn733u8AeODUBwKftX3ZMnjwQfj976FVK9i40Z/Eb5i8629TVnGxnz2+chqn0lL/ht2+fd22N3ftXP7++d+57djbQtFNnWjz5sH++wddRSRoVGQdKNjizDkXeKBVp6wMZs2Cgw4KupLw2bDBt2orF6qsyvz5sN9+tdvu67Nf5xev/YKZ186kY7Pwj6AoLiumwlWQnZld49ds3uw/PLVpk8DC9jzhfjMJKXVFxlllqM1YOYNXvn4lsDo+/RSmTKn6seuugxNO8G/isqOrr/bzElbOWfhDw4b5ARFffVW77Z7b/VzWDl6bEqG2bOMyWue1ZsS0EbV63V//6rtqV61KUGEiNaTZ/RPkjtF3MHPVTM7pfk69Lnatq9tu8yftp07deTaHG2/08/FpUtmdXXIJLF++62utzjvPd0l2qcOI+Kz0rPoVlyT7NNmH63pfx2F7H1ar111yiZ+8eK+9ElSYSA2pKzJBlm5cSnZGNi1zWgay//Xr/cCR6robKyehlcT5z7T/8NTUp3j55y/TIrtF0OVIatFfZx2oKzJB2jdtT8ucljjnWFmwMun7b968+lB78UXo29cPxd7T5eXBY4/5oK+JadP8JL5r1lT/3DRLIyMtg+YN4zcXYzIs3LCQOWvnVPu8Z5+F3/7Wn2MTCQMFW4INfGcgRw8/Ommzua9dC2ef7d94q5OT41tr69Ylvq4wq6iADz/0Q/xr2nrNyPAzbMyfX/1zLzn4Ekb9YlToBxVtr7yinN5P9Oa+T+6r9rnz5sGMGX5BUJEwUFdkgn288GOmr5zOtb2vTcq5tnHj4Gc/87Oo12TFYXVFes75FkdOTs1fU1bmA253CkoKaJTZKKVCrVL+3Hy6tuxaoym2anIspE5S7xcnBBRsEVReXruJZr//Hp5/Hq65Zs8LufHj/SjH3Ny6vb6iwnfp/vSnVV8icMUbV/Dl8i+Z/pvpKRluuzNtmv89q8kHKKmzaP3SJIm6IpPkg/kfcMUbV5DIDxIbN/qWR21nT3/mGbj2Wpg+PTF1hVVJiR/leNlldd/GmDFw0UX+g0FVBnQdwK8P/XXKhtpHCz7a5WUrt94KZ53lW2siYaLOgySZu3Yuny35jFWFq2jTODFXsP7yl771NXp07V73m9/AccfBIYckpKzQysqCt97y3+uqb18YNQpOPrnqx8/vcX7dNx4Cf//87yzcsJCf9vjpTo89/7w/x6guSAkbdUUmSXlFOWUVZTTISNwZ9qefhi1bfFDV1dq10DKYKxSSKhHnhAoL/awllaZ9N419c/clt2FufHeUREu+X0KrnFY7zECyebOfli1FG6GpRke5DtQVmSTpaek0yGhAaXkpo+aNSsg+Lr+8fqH2xht+4uSod0mWlfnZRR54IH7bnD0bunb1x7DSBS9fwIUvXxi/nQSgQ7MOO02rddll/pxiCn0mlj2Mgi3JHhj3AP1H9mf2mtlx22ZpKfz3v37i3vo47ji/5Mjee8enrrDassUPeIjn+mD77+9XTaicJNk5x/BzhnN337vjt5OAvDrrVQa+MxDwYXbssXD88WqxSXipKzLJNhVv4pNFn3Dmj86M2zb/9z9/7dr//gdnxm+zIgD8dcxfeWb6M0y6ehI5mbW4HkLiQR8f6kAttiRr0qDJ1lArLS+NyzbPOAM++CB+610tXuzn/Vu9Oj7bC5O8PH9hdaJs3gx33QV/fHMYs1bPStyOkmjQMYOYdd0snnw0h3ffDboakeop2ALyztx32P9f+7Ns47J6bystzU/vVNth/rtSUAD5+TBpUny2FxZLlsB99+16aH48FBXBw0+t509TfsOLM19M3I6SKCMtg9JSeOIJeO65oKsRqZ4G6gakW8tuHNTmIEor6tdq+9vf/Oi+W26JU2FAjx4+BBqFd6HnOunQwQ/yaNUqcfto2RK+mdqc8obLSLPofG58e97rbLnq9/zlgglA06DLEdmt6PzlpZj9W+zP2xe/Xe/VlL/4Ar78Mj41ba8y1D79NBqTJFd2P7Zrl/g5DVu1gjaN21C0pnWNJkkOu5kzoUWDvejeqhvlWXv4xKKSEhRsAVu/eT1/+PAPdZ4k+dVX/cwhiTBlir8A+bHHErP9ZPnmG78A5sMPJ35fRaVFXPbaZXwyZxKHHAJ33JH4fSbSpk1w4okw/N6jefOiN+v9QUwkGRRsAZu2chr3f3Y/Hy38qNavLSry3+szc8buHHaYPx911VWJ2X6ydOoE99zjr71KtG/WfEP+t/mUZqznoYfgzjsTv89EatLEn1ur7OreWLyRsgrNoSXhpuH+IbBowyL2zd23Vq+ZNMl/kn7zTTjhhMTUtb3ycj9IRdcuVa+8ohzwF+VXSsVVFH5Y82eLP+PEESfy3qXvcWLnE4MrbM+SYr814aAWWwhUhtq3676t8STJjRvD+ef7VlWiLV0KRxwBr7+e+H3F0/r1cMopMHlycvebnpa+NdRKSuDSS/0gn1RSVuYvON9+FOShex/KjT++kfZN2wdXmEgNKNhCYvzS8Rzw0AE8N6Nm46m7dYPhw6FZswQXhp+JpF27xHV5JsqCBX4RzLQk/ZaPWzKO3k/05uvVX2+9LyvLt3YrKpJTQ7ysX+9baw0bbruvUVYj8k7No2vLrsEVJjViZuVmNtXMvjKz/5lZbh220cvM/rWLxxaaWZ3GF5vZH83s1rq8tqY03D8kerfrzT197+G0LqdV+9zPP/dB07FjEgrDX07w9tvJ2Vc8HX44zJmTvNnni8uKaZDegHZN2u1w/3PPpV43ZOvWflXxH3LO8dWqr2iZ05J9muyT/MKkpjY75w4FMLMRwHXAX2qzAefcRCAlz/0E2mIzs/5m9o2ZfWtmtwVZS9DSLI07j7+TVjm7/xDknB/MccEFSSpsOxUVMGyYX2AyXvLn5tNvRD86/7Mz/Ub0I39ufr23WVYGL7zg603mkiondj6Rsb8eS7OGOzajK0Nt/PiqwyJsnngCNmzwdf8wkFcVruLgRw9m+JThgdQmdfI50A7AzPY3s3fNbJKZjTGz7rH7fxZr3U0zs09j951gZm/Ffm5pZqPMbIqZPUbs3J+ZdTKzryp3ZGa3mtkfYz9fZWYTYtt8xcx2mo/NzG4ws6/NbLqZvRCvf3BgwWZm6cBDwACgB3CRmfUIqp6wWLFpBWc8dwbjl46v8nEz33oaOjTJheEXMr3jDt8FGg/5c/MZmD+QFZtW0KJhC1ZsWsHA/IH1DreXX/aLf37wQXzqrImi0qLdjhZ0zq+8cOed4Z4Vf9YsX+eTT1b9eJvGbXj5Zy9z5eFXJrcwqZPY++xJwJuxux4HrnfOHQHcClReBHM3cJpz7hDg7Co2dQ8w1jl3WGxbNekvetU51zu2zVnAFVU85zbgMOfcwUA91ibZUbWjIs1sIPCsc259vHYa224f4I/OudNit28HcM79dVevCfWoyOdSrK9JRLyLQ/xJo46jIs2sHJgBdAImAacC2cBq4JvtntrAOXeAmT0K7A+8iA+ktWZ2AnCrc+5MM5sK/MQ5Nz+2/XXAj4DGwFvOuZ6x+28FGjvn/mhmfYE/A7mx573nnPtNrEVX4Jx7wMzeBQqA14HXnXMFdfn3/lBNWmx7AxPM7MVY12G83sHbAUu2u700dt8OzOxqM5toZhNXR3FWXhGR+Ks8x7YvkIU/x5YGbHDOHbrd1wEAzrnfAHcCHYCpZlbVcsNVfQIoY8cc2W64EU8DA51zBwH3/uCxSmfge+6OACaZWVxOHlS7EefcnWZ2Fz7xfwUMNbMXgSedc/Pqse+qAnKnA+ecexzffKZXr17h/WiVgE997337HgfudeDW4dX5+XDhhTBxol/UMihbtviZPE44wa/aXRfLNi5j3wf96tIdmnZg3vp5NG/YnJzMHNo2acvoX46u9TbfeQeuuALef9+vtxZWP/kJfPcdjB2bvBGb1Zk1C3r1ghEj/GUku+Oc40dDf8TBbQ7mlZ+/kpwCpU6cc9+b2Q3AG8AjwAIz+5lz7qVYI+Vg59w0M9vfOTceGG9mZ+EDbnufApcAfzazAUDz2P0rgb1iQVgAnAlUrgHRBFhhZpmx1+4w47uZpQEdnHMfmdlY4GJ8y25Dff/dNfqzcr6/8rvYVxn+H/WymdXn6pyl7Hjw2gPL67G9SFlVuIpz/3sueZ/lbb1vwAD/hhhkqIEfAn7HHXDddXXfRrum7bjy8CtplNWILWVb6NK8C9kZ2ZRUlHBu93NZU1T7SRb32cdftP6jH9W9rrr4Zs03rNi0osbPv/DC8M3m0rq1X2S2b9/qn2tmjLp0FP89/7+JL0zqzTk3BZgGXIgPmCvMbBowEzgn9rQ8M5sRGwjyaez527sXON7MJuMbOYtj2y4F7gPGA28B26+gfFfs/vd/cH+ldGCkmc0ApgD/55zbUL9/rVeTc2w3AL8E1gDD8P2gpbG0neuc279OO/ZNzjn4E5vLgAnAxc65mbt6TajPsSXA2MVj6b1PbxpkNKC4OPGT9yZaQUkBN+bfyO3H3U6XFl0AP4Akb1weCzcspFNuJ2768U1c/+71HNDqAN69NDUW/zr7+bOZsWoG82+YT/x66kUAzTxSJzXpz2yFP2m4aPs7nXMVZlbn9Zqdc2WxgSnv4ZN7+O5CbU90bMdjAb8g6Tk/raBVbgNGjgy4qO1UVPg5GJs1g1trcLnlmqI1vDX3LY7f9/itwTag6wAGdN1xhdS2TdrSpEETwHd7VRcWs2f7iaDvvBNyAljgechJQ1i2cVmtQ+2VV/zKDPffn6DCamDdOrj+evjLX/ycmrXx8ISHWVu0lrv63pWQ2kTqqibn2O7ezWP1WiLYOfcO8E59thF1hSWF9HmyD62OOps+zf8cdDk7SEvz52ZaVnWaeTvfrvuWLi260Cm3E3Ovn0vTBrtfz6t3u95bf/7D6D+Qbuncd+J9uwyOd96BRx+Fm24KJth67tWTnnvV/qTepEn+koSiomDqBj/d2DvvwODBtX/thOUT4rJQrki8aRLkFHDbB7dxTIdjOKvbWUGXspPSUsjM3PXjYxeP5cQRJzLyvJFc0LN2V5U757jqf1eRkZbBo2c+utvnrl7tzxMl24szX6Rjs44c1f6oWr+2uNivep7Mi8irUlDg5x6trbKKMjLSNHlRgqkrsg4UbCFXXAxjxvgJacMygq4qixb5yZKPOWbH+8sqyhgyZgg3HXVTtS21qjjnqHAVpKels/j7xRSXFW+dq3D9ej9DRufOcfgH1IFzjo4PduS4jsfx3E9rNsdnVTZvhgkT4Pjj41hcNcrK4LPPajZYRAKlYKuDEL9VCviFRE85BT7+pILHJj7GizNfDLqkKl18sR9qX1HhZ0+5/PXL2Vi8kYy0DO7ue3edQg38CLzKmfKvfPNKTh15KqXlpQDcfTcccog/TxQEM2PmtTO5/+T6nSQbNAj69yepq20PH+4v1xhf9QQ3Nfbv8f/myCeOrPGqFCLJoH6EkDvvPHjpJf9p/g9PP03HZh35+YE/D7qsnTz2GDRt6luVc9bO4fXZr/Prw37N8fvGrxny2JmPsXDDQjLTfd/n4MHQuze0aBG3XdRa0wZN6xzalW67zV/b1qpOc6XXzS9/CY0awZFH1m87rXJa0bVlVwpLC2mcVYf+TJEEUFdkClm3eR3NGzYP7ZDy+evns1/z/di8GYptA7kNcxOyH+fg+a+eI//bfJ446wkaZlQ1oUHi/eat33Be9/NqtCJDTZWVJf6cW3m5P7cnKSGcf+whp67IEPvHP+DZZ7fdbpHdAjNjw5YNzFk7J7jCqjB8ynC6D+3Oxb+bQr9+0DQrN2H7evBB+Nujy1i8YSlpFsyv8NqiteR/m8+89fWZfGdHI0fCQQf5wRyJ8tZbcOih/pxoPK3bHFB/sISGmXU3s8/NrDjR661VR8EWUs7Biy/CqFE/vN9xyn9O4aJXLgrVeY3zup/HncffSf/De3Lqqb7lkSjl5bD/d4P44LL3yUrPorCkkAnLJiRuh1VomdOShTcu5KrD4zeFyP77+wVkCwvjtsmdZGX5tfzato3fNkdMHUHrvNYs3bg0fhuVOjOjvxkfmjE/9r1/kna9DrgBeKA2LzIvrlmkrsgQc86/yf1wKPboBaNp3rA5h7U9LJjCYmaumskjEx/hn/3/uXWAR7I4t22tsFveu4VHJj7C/Bvm07ZJHN+xpUa+Xfctz814jmuOuIY2jdsEXU7U1KorMhZiDwHFQBGQAzQArnOOOk/lY2ad8HNAjgWOwk+59RR+qq29gEucc1/GnvtHYrP3V7O9fOAjoA9wLvDz2FcD4DXn3D1m1gi/4kB7/EQef3LOVTuXm1psIeScb/GYVX19Ub/O/baGWpAfTMYsHsPLX7/M4u8X7/TY+PHwpz/Fd3+ffOJXD4cdF8C86/i7eObcZ5IWaisLVtLjoR58MD8xC76tXAn33utHmMbL11/7AT7x3GalLi26cHffuxVq4TCIbaFG7Htx7P766gL8EzgY6I6ftPhY/Lpud9Rhe92AZ2JrvHUDugJHAocCR5jZ8UB/YLlz7pDY0jg1CmcFWwh9/rm/Nmvy5N0/796P7+XiVy9OTlExzjkWbfAnaK454hpmXTeLzs13vpDszTf9G+n6OK7id/fdcOWVO785N89uzs8O/BkAE5dPZMCzA1hdmLgljtZtXke7pu1onZOYK8JHjfJTXFX3/18bTz0Ft9+euEsjyivKGbdkHAUlCTxBKDXRmW2hVqkodn99LXDOzXDOVeAnUP4wNkF+5bpvtbXIOfdF7OdTY19TgMn44Owa2/bJZna/mR3nnPu+JhtWsIVQVpZfQqS6Weqz0rPIzsjeel1XMgx6fxC9n+jN6sLVmBnNs5tX+bw77/TTbTWv+uE6efttf13f7i5UX7hhIQs3LIzfTqtwQOsDeP8X73PI3ockZPuXXgrffON/B+Llb3/zF4En6pKCz5d+zjHDj6n36udSbwvw3Y/by4ndX1/F2/1csd3tCup26dj2Z5MN+Ot268R1cc496Zybg1+rbQbwVzPb5RSP29N1bCHUqxe89lr1z7vt2NuSPvT/ysOvpH3T9rTK2f07ZHa2/15RATNm+Aup62rdOsjN9d2y3brt/rnn9zifc7qdQ2Z6Js45xi0ZxzEdj9n9i2qhpLyE8opysjOz47bNHzLbNpvKggX1m1llwwb/PTfXD05JlD7t+/DCT1/g1P1PTdxOpCby8OfYYMdzbHm7fEU4vAf8ycyedc4VmFk7oBSfUeuccyPNrAC4vCYbU4stZCZP3vZmVJ3KUJu/fj4jpydu2v/RC0YzZMwQALq36s5NR91U40C991446ihYvPNpuBpxDn76Uzj77Jq/pvIC7qemPsWxTx3L2MVj67bzKrwz9x1a/q0l01dOj9s2d+XNN6FLF39usa4GD/aXECRypCVAelo6F/S8gGYNmyV2R7JbsQEi1wErgBax7/UaOFJTZra3mS0FbgHuNLOlZlaj2Qucc6OA54DPY+uzvYxfqPQg4Eszmwr8AajZTPDOuZT5OuKII1yUlZc7t99+zp16au1ed8UbV7gW97dwG7dsTEhd1751revxUA9XWFJY69cuXerc8OHOVVTUbd8VFc6NGOHcyJG1f+2W0i3uyclPuoq67rwK07+b7m7Kv8mVlJXEbZu7Uljo3N13O/f993XfxsSJzj30UPxq2p2ikiL31JSn3JQVU5Kzwz1D4O+7qfil4f4hM2mSHxH54x/X/DVritZQXFZMu6bt4lZHcVkxG7ZsoE3jNpSUl1BcVrx1jbS6cm7H0YzJtGLTCi57/TIePv3hrZMoS3wVlhTSKq8VNx91M0NOGhJ0OVGhmUfqQF2RIXPEEbULNfDz9VWG2qrCVXGp47z/nsfpz51OWUUZWelZ9Q61UaN8l+SmTTV/zZ13+nky42Hx94uZu3Yum8s213kbS75fwrfrvo1PQbUwbx6cdpo/31ZT//ynHwVZXp64un6oUVYjZl47k7/0+0vydiqhZmYtzWxqFV/VrOJYPwq2kFiyxM/y/t13dd/Gnz75Ez0f7hmX6Y2uP/J6bj/29ritt9WsmR9IsqqGuVtc7MPwiy+qf25N/Lj9j5l7/VwObnMwAJ8v+Zza9lb8a/y/6PFQDzYV1yKd46BBAz9Kcu7cmr/m229h5szkL3W0X/P9QjuXqSSfc26t2zbScfuvtYncr7oiQ+L55+Hyy/0bWKdOddvGtO+m8cY3bzD4mMF1mhj4mWnPkJWexYU9L6xbAdWobVdkeblfyLRhnOc4/mLpF/R5sg+PnfkYVx9xdY1ft2jDIiatmMRPDvhJfAuqgbpMjlzdIrCJ4Jzjro/uolNuJ648/Mrk7jya9CmhDtRiC4mLLvKttbqGGsAhex/C3X3vrlOoVbgKnpzyJCOnj6x1S6amzPyimg8/vPsusrFjoajIz0Af71AD+HG7H/PEWU9w2SGX1ep1++buG0iogQ815/xlILvrzv3ii21dlskONfAjdT9d9ClTv5ua/J2LVAp69EptvqI6KrK8PL7b+2LJF+6ily9ypeWl1T53bdFaV1Bc4Jxzbl3RuoSP9nvxRefAufffr/rx9euda9zYuSuvTGgZW20u3ezOef4cN2bRmN0+792577q357ydnKJ2YeZM58ycu//+qh+vqHCuZ0/njjii7qNQ46GsvCy4nUdP4O+7qfilC7RD4PTToWdPeKBWc2Lv2rJNyxi7eCwL1i/Y7QjALWVb6PNkH45sdyT/Oe8/u5xFJJ7OPx++/NIvEFqV3Fy/tErHjgkvBfCDbWatmVXtoJu8cXms37Ke07uenpzCqtCjhz/veMIJVT9uBvn5sHZtcKNPgaRPiC3yQzrHFrDycj9oZL/9YODA+GzTOceWsi01mh3jkQmPcOjeh9KnQ5/47LwW1q3bcfXrZCyyWZWS8hKy0rMAmLxiMgftddDWi7wrFZcVs2zTMvZrvl/yC6zCli1+6rXKwSGbN2+b7SUMbnnvFhZ9v4hXfv5K0KWkupQ5x2ZmlwC/j90sAH7rnJsWRC06xxaw9HS/oGi8Qg38eY7szGwqXAXvfrvjhAMVroI/f/pnJi73HxB+2/u3gYTamDHQoQN89JG/XVTkp916+umkl7I11FYVrqLv0325ddTOayQ2yGgQmlBbscK38IcN87ed85cDXHddsHVtr02jNrRv0j5h52tl1+xe62/32od2r82PfU/WemwLgL7OuYOBPwGP1+RFiViPTcEWoKIimJ7AmZlGTB3BgGcHcPhjh9P5n53pN6IfL3/9MsMmD+PFmS8mbsc10KsXXHaZn2qrXz/o3h1Wr6755QCJsFejvRh21jB+f6z/0Jk/N59+I/qR+/9y6fFQj9BM8Lv33r47cv16f+w6d/aXiyRioE1dHdzmYGasmsF+/9qPfiP6hebYVar8v638uwhbfXUVC7GHgLb4hT/bAg/VN9zMrJOZzTazYWb2lZk9a2Ynm9lnZjbXzI50zo1zzlWu5/EFfg213W1vlpk9jJ/Nv4OZDTKzCWY23czujT2vkZm9bWbTYvu9oCb1KtgC9MILvpUSz+VJttcyuyV7NdqLopIimmQ2YfnG5dz+4e0M6TeE+0++PzE7raHsbD//4333+RZI69bQsqVf6iY/wPeYC3pewD5N9uGdOe9w4SsX8s2abygtL+W7gu8YmD8wFG+AZn7+zMcf98euZUsfaq+/Huyxq5Q/N5+B+QNZsWkFTbOasmLTitAcO9ixvhYNW4SuvnoKy3psV+AXEt0drccWReeeC088AYclaCHsB8c/SIuGLchMz+TrNV9TUFpAVloWw6YMC8VFtHl5/hzRnDn+nFGjRv68UV4I5iG//7P7KS0vxeE4oPUBdM7tTFZaFnnjQlAc/hhlZvoW79q14Tp2eePyyErLYkPxBmavnU1OZg5ZaVkMzB/IG7Pf2Pq8IWOG8Pact7fe/tMnf9qh6/yej+7h/XnvA7Hr40bfxUcLfN91WUUZd42+i08XfQr4gVB3jb6LzxZ/Bvjpve4afRdfLPVX+G8s3shdo+9i4vKJ5I3LI510vi/+HjOjUVajUP3f1lPg67GZ2Yn4YPt9lVvZRuuxRVGLFn7hzERlzIINC/ybSnoWrRu1JrdhLjmZOQlfr6ymFizwLY2sLD/4ASAnBxYuDLQsABZvXEy3lt1o29ivym1moTt22dn+d6c0thxfWI5d5e9d06ymNG3gJ3fPycxh0YZF5H+77UP8Pz7/B6Pmjdp6O29cHh/O/3Dr7f/32f/jk0XbljYYMnYIYxaPAfzCpkPGDmHcknGAHwA0ZOwQvlz2JQCbyzYzZOwQJi2fBMCm4k0MGTuEqd9NZcGGBWSlZ/FdwbZpfsL0f1tPga7HZmYHA8OAc1z1s4toPbao+fe//dpipyZw+arOuZ1ZsWkFjbIa0a6Jn0uysKSQTrmdErfTWujc2XelHXjgtvuKiup3kXq8bH/sKhWVFoXu2HXvvu2+sB27Jg2abJ1jtLCkkOP3PZ5Hz3x06/PWDF6zw+s23r5xh9vFd257HzUzyu/edlV/g4wGO9xu2qDpDrdb5bTa4Xa7pu223n5uxnOs2LSCw9sevvXxMP3f1lNg67GZWUfgVeAXsUCqDa3HlurKyuDBB+GVBI+EHnT0IEoqSigsKcQ5R2FJISUVJQw6Oh7d7fU3aBCUlPi1wpzz30tK/P1B07GrR21hP3Yhr68+3D2u6vXY/P2JdjfQEng4NtFxja/NcnFej03XsQWktBQKCqB5gq+Jzp+bT964PBZuWEin3E4MOnoQA7oOSOxOayE/358XWrjQtzYGDYIBISlPx67uQn/sQl7fdoI/GZ6CFGxJVnm4QzB2Q0TCT+8UdaCuyCT76CM49NDaLUEiIpKKglqPTYNHkqyiwl931H6Xly6KiERDbGTkocner4ItyU4+2X+JiEhiqCsyiWbN8iPXREQkcRRsSVJW5q9Zu/TSoCsREYk2dUUmSVqanz6radOgKxERiTYFW5KkpUH/ZC0eISKyB1NXZBLMn+8vpP2+RtN3iohIfSjYkiA/H+64w880IiIiiaVgS4LrrvOttnbtgq5ERCT6FGxJ0qFD0BWIiOwZFGwJdsYZ8Le/BV2FiMieQ8GWQMXFkJvrVzcWEZHk0HD/BGrQAJ59NugqRET2LGqxJUhBASxdGnQVIiJ7HgVbgjzzjF/8UcvTiIgkl7oiE+T00/05ti5dgq5ERGTPomBLkE6d4Oabg65CRGTPo67IBHj8cZg8OegqRET2TAq2ONu8GW6/HUaMCLoSEZE9k7oi4yw720+fpQVFRUSCoWBLgGbNgq5ARGTPpa7IOHr3XTjlFFi2LOhKRET2XAq2ONq0ya+51rp10JWIiOy5Agk2M8szs9lmNt3MXjOz3CDqiJf8fOjXDwYPhsaN4cMPg65IRGTPFVSL7X2gp3PuYGAOcHtAddRbfj4MHAiLF0Pz5rBihb+dnx90ZSIie6ZAgs05N8o5Vxa7+QXQPog64iEvDzIzYdEiWLLEz+SfleXvFxGR5AvDObZfA7ts35jZ1WY20cwmrl69Ooll1cyCBX6If/v20LKlvy8nBxYuDLQsEZE9VsKCzcw+MLOvqvg6Z7vn/AEoA3a5uItz7nHnXC/nXK/WIRyV0bkzbNniQ61xY39fUZGfUktERJIvYdexOedO3t3jZvZL4EzgJOecS1QdiTZokD+nBr6lVlTkL84eNCjYukRE9lRBjYrsD/weONs5VxREDfEyYAAMHQpt28L69f770KH+fhERST4LorFkZt8CDYC1sbu+cM79prrX9erVy02cODGhtYmIhIgFXUAqCmRKLeecVikTEZGECMOoSBERkbhRsImISKQo2EREJFIUbCIiEikKNhERiRQFm4iIRIqCTUREIkXBJiIikaJgExGRSFGwiYhIpCjYREQkUhRsIiISKQo2ERGJFAWbiIhEioJNREQiRcEmIiKRomATEZFIUbCJiEikKNhERCRSFGwiIhIpCjYREYkUBZuIiESKgk1ERCJFwSYiIpGiYBMRkUhRsImISKQo2EREJFIUbCIiEikKNhERiRQFm4iIRIqCTUREIkXBJiIikaJgExGRSFGwiYhIpCjYREQkUhRsIiISKQo2ERGJFAWbiIhEioJNREQiRcEmIiKRomATEZFIUbCJiEikKNhERCRSFGwiIhIpCjYREYkUBZuIiESKgk1ERCJFwSYiIpGiYBMRkUhRsImISKQo2EREJFIUbCIiEikKNhERiRQFm4iIRIqCTUREIiXQYDOzW83MmVmrIOsQEZHoCCzYzKwDcAqwOKgaREQkeoJssf0fMBhwAdYgIiIRE0iwmdnZwDLn3LQg9i8iItGVkagNm9kHwN5VPPQH4A7g1Bpu52rgaoCOHTvGrT4REYkmcy65PYFmdhDwIVAUu6s9sBw40jn33e5e26tXLzdx4sQEVygiEhoWdAGpKGEttl1xzs0A9qq8bWYLgV7OuTXJrkVERKJH17GJiEikJL3F9kPOuU5B1yAiItGhFpuIiESKgk1ERCJFwSYiIpGiYBMRkUhRsImISKQo2EREJFIUbCIiEikKNhERiRQFm4iIRIqCTUREIkXBJiIikaJgExGRSFGwiYhIpCjYREQkUhRsIiISKQo2ERGJFAWbiIhEioJNREQiRcEmIiKRomATEZFIUbCJiEikKNhERCRSFGwiIhIpCjYREYkUBZuIiESKgk1ERCJFwSYiIpFizrmga6gxM9sEfBN0HbvRClgTdBG7EObaINz1hbk2CHd9Ya4Nwl9fQ+dcz6CLSDUZQRdQS98453oFXcSumNnEsNYX5tog3PWFuTYId31hrg1So76ga0hF6ooUEZFIUbCJiEikpFqwPR50AdUIc31hrg3CXV+Ya4Nw1xfm2kD1RVJKDR4RERGpTqq12ERERHZLwSYiIpGSEsFmZv3N7Bsz+9bMbgu6nu2Z2XAzW2VmXwVdS1XMrIOZfWRms8xsppndGHRNlcysoZl9aWbTYrXdG3RNP2Rm6WY2xczeCrqWHzKzhWY2w8ymhnFYuJnlmtnLZjY79vvXJ+iaKplZt9hxq/zaaGY3BV1XJTO7OfY38ZWZPW9mDYOuKZWE/hybmaUDc4BTgKXABOAi59zXgRYWY2bHAwXAM2G8kNLM2gJtnXOTzawJMAk4NwzHz8wMaOScKzCzTGAscKNz7ouAS9vKzG4BegFNnXNnBl3P9sxsIdDLORfKC4zNbAQwxjk3zMyygBzn3IaAy9pJ7D1mGfBj59yiENTTDv+30MM5t9nMXgTecc49HWxlqSMVWmxHAt865+Y750qAF4BzAq5pK+fcp8C6oOvYFefcCufc5NjPm4BZQLtgq/KcVxC7mRn7Cs0nLTNrD5wBDAu6llRjZk2B44EnAZxzJWEMtZiTgHlhCLXtZADZZpYB5ADLA64npaRCsLUDlmx3eykheWNONWbWCTgMGB9wKVvFuvqmAquA951zoakNeBAYDFQEXMeuOGCUmU0ys6uDLuYH9gNWA0/FunKHmVmjoIvahQuB54MuopJzbhnwALAYWAF875wbFWxVqSUVgs2quC80n+pThZk1Bl4BbnLObQy6nkrOuXLn3KFAe+BIMwtFd66ZnQmscs5NCrqW3TjGOXc4MAC4LtYtHhYZwOHAI865w4BCIFTnxwFiXaRnAy8FXUslM2uO75XqDOwDNDKzS4OtKrWkQrAtBTpsd7s9apbXSuz81SvAs865V4OupyqxbqqPgf7BVrLVMcDZsfNYLwD9zGxksCXtyDm3PPZ9FfAavts+LJYCS7drgb+MD7qwGQBMds6tDLqQ7ZwMLHDOrXbOlQKvAkcHXFNKSYVgmwB0NbPOsU9XFwJvBlxTyogN0HgSmOWc+0fQ9WzPzFqbWW7s52z8H/TsQIuKcc7d7pxr75zrhP+dG+2cC82nZjNrFBsMRKyL71QgNCNznXPfAUvMrFvsrpOAwAcsVeEiQtQNGbMYOMrMcmJ/vyfhz41LDYV+dn/nXJmZDQTeA9KB4c65mQGXtZWZPQ+cALQys6XAPc65J4OtagfHAL8AZsTOZQHc4Zx7J7iStmoLjIiNSksDXnTOhW5YfUi1AV7z73tkAM85594NtqSdXA88G/tAOh/4VcD17MDMcvCjra8JupbtOefGm9nLwGSgDJiCptaqldAP9xcREamNVOiKFBERqTEFm4iIRIqCTUREIkXBJiIikaJgExGRSFGwiYhIpCjYREQkUhRsIrtgZr3NbHps3bhGsfWxQjGXpYjsmi7QFtkNM/sz0BDIxs99+NeASxKRaijYRHYjNh3UBGALcLRzrjzgkkSkGuqKFNm9FkBjoAm+5SYiIacWm8humNmb+GVrOgNtnXMDAy5JRKoR+tn9RYJiZpcBZc6552IrEIwzs37OudFB1yYiu6YWm4iIRIrOsYmISKQo2EREJFIUbCIiEikKNhERiRQFm4iIRIqCTUREIkXBJiIikfL/ARC6oL0OMUstAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "#hide_input\n", "\n", "fig = sns.lmplot(x = 'x', y= 'y', data = df_res, hue='Residuals', palette=[\"b\", \"g\", \"orange\"], fit_reg=False )\n", "fig.axes[0,0].plot(x, m1_res, linestyle='dotted', color = 'b')\n", "fig.axes[0,0].plot(x, m2_res, linestyle='dotted', color = 'g')\n", "fig.axes[0,0].plot(x, np.zeros(7), linewidth = 2, color = 'orange')\n", "\n", "plt.xlim([0, 8])\n", "plt.ylim([-5, 5])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we had to choose one of the models, which one would we prefer? \n", "Model 2 does better on the first data point and perfect on the third, but it contains an outlier the 5th data point.\n", "\n", "Let's look at the mean and the variance of the residuals." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model #1. mean: 0.0714, var: 1.6020\n", "Model #2. mean: 0.0000, var: 2.0000\n" ] } ], "source": [ "#hide_input\n", "\n", "print(f'Model #1. mean: {m1_res.mean(): .4f}, var: {m1_res.var(): .4f}')\n", "print(f'Model #2. mean: {m2_res.mean(): .4f}, var: {m2_res.var(): .4f}')\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On the long run Model2 has an average residual of 0. Model 1 carries along a residual of 0.0714. So on average Model 2 seams to do better. \n", "\n", "But Model 2 also has a higher variance. That implies we have a great chance to do a great prediction (e.g. x=3) but we also have high risk to screw the prediction (e.g. x=5). \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we build a simple linear ensemble of the two models like `ens = 0.5 * m1 + 0.5 m2`." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#hide\n", "\n", "ens = (m1+m2)/2\n", "\n", "df = pd.DataFrame(np.array([x, m1, m2, ens, y_true]).T, columns = ['x', 'm1', 'm2', 'ens', 'y_true']).melt('x')\n", "df.columns = ['x', 'Values', 'y']\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbkAAAFyCAYAAABoe6RfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABjeElEQVR4nO3dd3hUZfbA8e87KaRASEINRROQjihdEBVQEdRVEdy14NrrxrYurrqKgH3jqvxEwY4FECuokABSBEV6EZBOQEogEBJILzPn98cdekISMjN3MpzP88wzmTu3nEkmc+a+933fY0QEpZRSKhA57A5AKaWU8hZNckoppQKWJjmllFIBS5OcUkqpgKVJTimlVMDSJKeUUipgeS3JGWM+MsakG2PWHLMs1hgz0xizyX0f463jK6WUUt48kxsH9D9h2ZPALBFpAcxyP1ZKKaW8wnhzMLgxJh74UUTaux9vAHqLSJoxJg6YKyKtvBaAUkqpM1qwj4/XQETSANyJrn5ZKxpj7gXuBWjbtm3ntWvX+ihEpZSqNozdAfg7v+14IiLviUgXEekSHh5udzhKKaWqIV8nub3uZkrc9+k+Pr5SSqkziK+T3PfAbe6fbwOm+Pj4SimlziDeHEIwEfgNaGWM2WmMuQt4BbjcGLMJuNz9WCmllPIKr3U8EZGbynjqUm8dUymllDqW33Y8UUoppapKk5xSSqmApUlOKaVUwNIkp5RSKmBpklNKKRWwNMkppZQKWJrklFJKBSxNckoppQKWJjmllFIBS5OcUkqpgKVJTimlVMDSJKeUUipgaZJTSikVsDTJKaWUClia5JRSSgUsTXJKKaUCliY5pZRSAUuTnFJKqYClSU4ppVTA0iSnlFIqYGmSU0opFbA0ySmllApYmuSUUkoFLE1ySimlApYmOeUXkpOhb19ISLDuk5PtjkgpFQg0ySnbJSdDYiKkpUFsrHWfmKiJTilVdZrklO2SkiAkBCIjwRjrPjTUWq6UUlWhSU7ZLjUV8vNh9WooLraWRUTAtm22hqWUCgCa5JTtEhJABFwu2LrVus/Lg/h4uyNTSlV3muSU7YYOtZor4+KsBHfwIBQVWcuVUqoqNMkp211+Ofzvf3DOOdCgAZx9NoweDQMG2B2ZUqq6C7Y7AKV++w1uuAFmzoTevaGkxOqAopRSVaVncsp2DRvCE09Au3awYoX1eM4cu6NSSgUCPZNTtmvRAl580fq5Zk24+mprvJxSSlWVJjllu40bretxDgeEh8O4cXZHpJQKFNpcqWyVng6tWsGbbx6/fM8ea+YTpZSqCk1yylZhYfDxx1YT5WG5udYYuTfesC0spVSA0OZKZauoKLj99uOXRUbC++9Dly62hKSUCiCa5JStFiyAZs2sHpXHuvVWe+JRSgUWba5UthGxmimffbb05xcssMbOKaXU6dIzOWUbEZg6FWrVguTkZJKSkkhNTSUhIYGhQ4fy4osDKCqyZkRRSqnToWdyyjYOB/ToATt2JJOYmMjOnTuJjY0lLS2NxMREbrttHj/9ZHeUSqnqTJOcss2sWfDrr5DkLhy3adMmMjMziYyMJDQ0lIkThxMVZXOQSqlqTZsrlW3+8x+rOOqOHakUuwvJ7dq1i5iYGCIiIti2bRs//gi//AKvvGJzsEqpaknP5JRtfvgB3nsPEhISyMzMxOFwUFRUREZGBnl5ecTHx7N8OUyaZBVVVUqpytIkp2xTrx60bg2JiYnk5eURHR1NREQEu3btorCwkKFDh/LEE1Yh1fBwu6NVSlVHmuSULebOtQZ8l5RAaGgoAE2bNqVWrVqUlJRw5ZVXMmDAAMLCtOyOUur0aZJTtvjiC2t8XFAQpKSkEBERwaJFi0hLS6Nnz55MmTKFgoICAFJSoEMHyMqyN2alVPWjSU7ZYswYWLkSQEhOTqZv377UqFEDYwwjR45k586dfPDBBwDExED9+rBvn50RK6WqI01yyhbGWFN5bd68ma1btzJgwIAjz/Xt25eLL76Yl156ifz8fLp3h59+surOKaVUZWiSUz63bBn8619WmZ2UlBQA+vfvf+T5w2dzaWlpjB079sjy/HwoKvJ5uEqpakyTnPK533+Ht9+GkBBrOq8WLVrQrFmz49a55JJLuPTSS3nllVfIzc3l99+hbl1ITrYpaKVUtaRJTvncHXfAoUMQFpbP3Llzj2uqPNaIESNIT0/nnXfeoU0buPtuq2KBUkpVlCY5ZYuQEJg/fz75+fnHNVUe68ILL+SKK67g1VdfpaAgm1Gj4NxzfRyoUqpa0ySnfGrXLrj2Wli61GqqDAsLo3fv3mWuP3LkSDIyMnjrrbcA2LIFtm3zTaxKqerPliRnjHnMGLPWGLPGGDPRGBNmRxzK99LSYONGcLmsTieXXHIJ4aeYzqRbt25cffXVvPbaa+zde5AOHeC113wYsFKqWvN5kjPGNAYeBrqISHsgCLjR13Eoe3TpAuvWQf3621i/fv2RpsolS0ay/PMYdnwWzPLPY1iyZOSRbUaMGEFmZibvvjuKiROtnplKKVURdjVXBgPhxphgIALYbVMcyiaHhw4MGDCAJUtGUn/9SOq7csgmhChXHvXXP38k0XXq1InrrruO119/nYsuyiQ+3sbAlVLVis+TnIjsAl4D/gTSgIMiMsPXcSjfczqhc2f47DMrycXHx9OyZUuCNrxBgQuaBJfQ2BRTZIIpFgdBG944su2IESM4ePAgr7/+OjNnwrff2vhClFLVhh3NlTHAtUAC0AiINMYMKWW9e40xS40xS/fpfE4B4eBBaNIEQkOLmDVrFv3798cYQz3JptDlBKC2w8nBkmJ2Ox3Uk5wj23bo0IEbbriBN998kxdfzOCFF+x6FUqp6sSO5srLgFQR2ScixcC3QM8TVxKR90Ski4h0qVevns+DVJ4XGwtTpkD9+r+Sk5NzZHzcARNJ+9Cj650dVMxT6QWkE3nc9s899xy5ubm0a/caCxb4MnKlVHVlR5L7E7jAGBNhjDHApcA6G+JQPiZi3aekpBASEkKfPn0AqBV3GQ4D+WLIF0OjYPi8ocHV+p+ICOLesF27dtx444188slbHDqUbtfLUEpVI3Zck1sEfA0sB1a7Y3jP13Eo3+vZEx55xBof16tXL2rVqoVLXGQfWAnAfgnjoMuBCMSHQNeE/rzyyyvcMeUOip3FgHU2l5+fzz33/Jf77rPxxSilqgVbeleKyHMi0lpE2ovIrSJSaEccyndEoHdvaNp0F6tXrz7SVJm6fy0tCrcC0PT6jTS8tQTTdigGgcV343QWUOIqIdgRDECrVq0YMmQI06a9zZw5aeTklHVEpZTSGU+UjxgDL78MsbHTgaNVB5oXbCbCAa7YrhDRxFr53OFQsxlkreaZemF8NvAzjDHsydnDzC0zefbZZxEp5sorX6VmTZtekFKqWtAkp3wiJ8c6m0tOTqZx48a0b9+eElcJ7PgOAMdZg46uHBwBXd0ldlaPwGRvBmDYnGFc/+X1xDSK4bbbbmPs2LHs3LnL1y9FKVWNaJJTPnH33dCxYwkzZ848MnTgqRn/Ijt1orVCk4HHbxB3OST8HVyFsOR+EOGNK94g+ZZk6kTU4ZlnnqGkxEmbNi+xd6/vX49SqnrQJKd8YtAguOyyRRw8ePBIU2XfSAe1TAnUbgtRLU/eqOP/oEZd2Dsbto4jMjSSXmf1AmBN0RrCu0aSm/c+69f/6cuXopSqRjTJKZ+44QYIC0smKCiIyy67DIABNfKtJ5tcX/pGYXWhk3vWkxWPQ/7RU7aY8Bi63dSZkGDDhAkvejN0pVQ1pklOed2+fbB/vzU+rkePHtSuXZuZm6cjO6dYKzQdWPbG8bdA3BVQlAnLHzuyuNdZvZj18CzuuecePvroI56a9Kx1jU8ppY6hSU553ejRUL9+OsuWLaN///788ucvDPumP6YgDSLPhpiOZW9sDHQdA0ERsH0i7E4+7ulBg56ixGl45ZUXmLd9npdfiVKqutEkp7zu+uvhjjusoQMDBgygR9MefNLpGuvJJgOtRHYqNROgg7v0zuL7ofjo4LhLLmlM927/wLE6iKYlTQE4VHjI469BKVU9aZJTXnfeeVBQkEL9+vU5//zzCTZBtMxfaz15qqbKY7V6BGI6Qd6f8PuzRxY7HDB58r+pERrK888/z7p960gYlcB3677zwitRSlU3muSUV2VkwPz5TqZPn84VV1zBu8ve5asFwyBnC9SoB3UvrNiOHMHQ/X0wQbDx/yBjyZGnGjRoyA03/IPx48dzcNdBrmxxJd0ad/PSK1JKVSea5JRXTZ8OF1+8jIyMDAYMGMAPG3+g+M+vrCebXAuOoIrvLLYTtH4MxAWL7gGXNZ9lSQlMnvwEDkc4o14dxWcDP6NxVGNEhBfmvcD2rO1eeGVKqepAk5zyqiuugBtvTMEYw+WXX87Um6fytyh3XZ2mZQwdOJVzR0BkAmStgvWvAxASAlOm1OPBBx9m0qRJrFmzBoDtB7fz2oLXmLB6gqdejlKqmjGHy5j4sy5dusjSpUvtDkOdph49euByuZi/YD6h+Tvh++YQXAsG7YOgGpXfYdpMmNMPgsLgytVQ6xwAMjIySEhI4IorruCrr6yzxT8P/kmTqCY4jIO07DQa1myIKa+ji1LVh76Zy6Fncspriovh7bczWLx4MR0u7ECT15uwfc2b1pONrzq9BAfWlF/xt4KzwOpt6f6i9ssvdejT51G+/vprVq1aBcBZtc/CYRxkF2bT6+Ne/GPaPzzwypRS1YUmOeU1v/8OiYk/4XK5uOSyS+iT0IcmBxdbT55OU+WxOv0PatSBvbMg9RMAxo2D1NTHqF27Ns8999xxq9cMrclD3R7i1g63Vu24SqlqRZsrlde4XDBo0O38/PMP7NuXTlBROnzXGByhMGg/hFSxTk7qZ/Db3yE0Fq5ex/6c+sTEwEsvPc+wYcNYunQpnTt3LnXT0YtH0yCyATe0u6FqMShlL22uLIeeySkvcvHbbyl0vqgTBwoOwM4pgEBcv6onOID4IdDwcig6AMseo25dCAqCRx55hJiYmJPO5g5zupx8ufZLJq2dRHX4kqeUOn2a5JTXDB36O3v37mVZ5DJun3L7kdpxJ5XVOV3GQLexEBQO2yfA7mS++ALuvDOKf/1rKFOnTmXRokUnbRbkCOKnv//EuOvGYYxhf95+0nPTPROTUsqvaJJTXpGXB6NGWfNMfvf0d7xy0ZNWyRzjgMZ/8dyBajY7OuXXkgfIycph1y647baHqFu3LsOGDSt1s9CgUGqGWmeTd0y5g14f9aLYWey5uJRSfkGTnPKKiAjo0SOFDh06ckn7Szi3ZDtICdS/xCqh40mtHrUmec7dzl1dh/Hbb9C4cU3+/e9/M2PGDH755ZdTbj6y90he7PsiIUEhno1LKWU7TXLKKw4ePMhvv/1KcGvD3py9sNPDTZXHOjLllwOzcRRkLMXphAcffJAGDRqUeW3usI5xHY90QJm+eTr3fH8PBSUFno9TKeVzmuSUVzz88CycTicbozcSIsVHS+Q0uc47B4ztDK2sKb8O/nQPjRoWs29fBE8++SSzZ89m7ty5FdrNqr2rWLJ7CU6X0ztxKqV8SocQKK9o2PBeDhyYxP6sHURlzIb5AyG2K/Rf7L2DluTC1PaQu41JG16l861P0LhxPueccw7Nmzfn559/rtBsJwUlBYQFh1HsLGbp7qX0aNrDezErVTU6hKAceianPE5ECA6ZxlVXXUZURNTRpsqKltU5XcGR0HUsAH9r+xznNNhCeHg4Tz/9NPPnz2fWrFkV2k1YcBgAry14jV4f92L9/vVeC1kp5V2a5KooORn69oWEBOs+Obn8bQLd6jWr2bVzFyXNiqxKATu/t56o6iwnFdHoCoi/BZwFFM6/j0MHhbvvvpsmTZowbNiwSo2Le6j7Q3xy3Se0rtua5E3J9BnXh4RRCfT9pC/Jm/QPrVR1oEmuCpKTITER0tIgNta6T0zURPfMsKkADPzL9ZD+MxRnQVQbiGrlmwA6vYEzuA41smax/NvPqFGjBs888wy//fYb06dPr/BuaobWZEiHISRvSua+H+9j4a6FRARHkJadRmJyoiY6paoBTXJVkJQEoaHWLS8PIiOtn5OS7I7MXr+vmEVkZDvu7H0H7PjWWuiLs7jDwurh6GKV4bko8p9QsI877riD+Ph4nn322UrPcpK0IAmH+1/FYRxEhkYS6gglacEZ/odWqhrQJFcFqanWeLDUVNi2zVoWEXH05zPRHzv/YHfaPB54YIBV3HTnZOsJb1+PO4FJuBUaXk5QSQYsf4zQ0FCeffZZli5dyo8//lipfaVmpVInog5t67alRrBVOSEiJIJtWdu8ELlSypM0yVVBQoJ1Bte4sfUzWI/j420Ny1ZPf/Q0xUXF9OzdEzIWQ34aRJwFMZ18G4gxuLqMxWnCYdt42D2dW2+9lebNm1f62lxCdAJ5xXkYY8gvzudgwUHyivOIj473XvxKKY/QJFcFQ4dCUZH1c3g45OZaj4cOtTcuOx2aXw9HUBjdO155TFPlQGueSV+r2YwXvx9h/bzkfkJMEcOGDWPlypVMnjy5wrsZ2nMoRa4icoty2ZOzh9SsVAqdhQzteQb/oZWqJnScXBUlJ1vX4DZssJoq/+//YMAAu6Oyh4jQsGFzXK727N0zBcfUlpCzGS6dCw0usSWmZUtLOH9PV4IOrYTWj1PS4RXatWtHaGgoq1atwuGo2Pe85E3JJC1IYlPGJprWbsqzFz/LgBZn6B9a+RMdJ1cOTXIe8vLLMHz40Z6WZxoRYfDYwXz74Le8/fbbPHjzRTCtA9SoBwPTwBFkX3AHlsH0btbPVyxmQsoGbrnlFiZNmsRf//pX++JSquo0yZVDmys95P77IT39zExwADsP7WTOzDkA9O/f/5iyOtfYm+CAidM7szzvUasjzKJ7+NsNg2jbti3Dhw/H6az89F0pm1N4d+m7ng9UKeVxmuQ8JCYGate2Owr7NK3dlJZp3QkKasm+fc28OyFzJU2dCo98MBKJjIfMFQRt+j+GDx/OunXrmDRpUqX39+XaL3n5l5e14KpS1YA2V3rQ8uXw/PPwwQdQp47d0fhOfnE+rmIXdWLrkpBwL/OmPUq935pBcC0YlA5BYbbGl50NNWuCSUuBuQMgKBzXgN85v9f1FBYWsnbtWoKDgyu8vwP5B4iqEUWwo+LbKOUl2lxZDj2T8yCXC5Yuhc2b7Y7Et56b+xxtn2hLYWEBr7/en3oF7rO4RlfanuAAatVyd+5s1B/Ovhmc+TiWPsiI4cPZuHEj48ePr9T+YsNjNcEpVU1okvOgzp3hzz+he3e7I/GtHk16ELcnjrCwMHr37g07bZjlpByffw59+oB0egNCY2HPTK47P4eOHTsycuRIiosrVxX8vWXv8ezsZ70UrVLKUzTJeZAxR4eDVYNWYI8Z2GYgGauyKCi4hK8+OwT7FoCjBjTyny72Doc15Vpmfn3oZE35ZVb8k1eGP87WrVv59NNPK7W/FWkrmP/nfG+EqpTyIL0m52Hbt0P//ta1ucGD7Y7Gu0SEr//4mvYh7Wnbqi1XXvkG7w6NoMnu+6DRVdC7ctNn+YwIzOkHe35C4ofQ/dENpKens3HjRkJDQyu0C5e4cBj9jqhsp9fkyqH/pR7WpAm0bg1RUXZH4n2Ldy3mr1//lRfHvQjA668PoIn4X1PlsfLzsU63u46FoDDMts95+9lr2b59Ox999FGF96MJTqnqQf9TPSwoCL77Dvr1szsS7+vWuBtzbptD1uosmjSJJ75Rfdg7G4wDGv/F7vBOMm2a1et13TqgVnM4dzgAXVwf0vfi7rz44osUFBRUeH//mfUfbp98u1diVUp5hiY5LykogH377I7Cu4wx9GzUk7lz5rJvX38+e3WaVSS13sUQVs/u8E5y3nlw551Qo4Z7Qet/QvR5mNxUPnrsLHbu3MkHH3xQ4f0FO4K1l6VSfk6vyXmBy2U1W/7lL/BugE6MMWrhKLKLsulZ0pNLL72UJ56YwhM9PqVO3jfQeRS0etjuECsmYynM6I4A93x9HtN+28OWLVsIDw+3OzKlKkKvyZVDz+S8wOGAkSPh5pvtjsR7lu9ZzqJdi0hJSSEkJIRnnrqAOoXuStl+MMvJqWzcCBkZ7gd1ukDLRzDi4o0bc0jfm8bYsWMrtT+nq/JTgymlfEPP5NRpK3YW07ljZ8LC6vH9Ow/TcON1ENsF+i+xO7QypaZCs2YwahQ8fPhkszgHprWH3O2MXXwOz008xNatW4mMjCx3f0O+HcK+vH1MHzLdu4ErVTo9kyuHnsl50datMHeu3VF43qHCQwCk70ln9erVpKf3Z/n37llOfFwBvLISEuDjj+H6Yzt/htSErmMAuKf7TiIknXfeeadC++vZtCd94vt4IVKllCfomZwXXX01rF1rJTs7aoZ6w9LdS7lk3CX8cNMPpM5O5e677+abr5ZxTcllBLsy4ap1ULu13WGenl9vhu0TWba7Dle8IqSmbqNWrVp2R6XUqQTIJ4v36JmcF738MsyeHTgJDiA6LJoh5w6hc1xnUlJSaNy4MQMvPGAluKjW1SLBuVxWZYL5J05Y0vlNCI2lc6MM+rc+wFtvvVWh/TldTvbm7PV4nEqpqtMzOXVaSkpKqFu3Lr17D2bUrWGcXfg2tHsaznvR7tDKJWJdl+vcGb7++oQnt46DhXdwsCCUzs+Fs2zNdmqXU0Op97jeCMLPt//stZiVKkMAfYX2Dj2T87LFi+G55+yOwjNmbZ3FlgNbAFi4cCEHDx4k+1A/Qvf6T+24ijAGUlKg1OIDCbdBg77UDivi2asOMmrUqHL3l9gtkYe6PeT5QJVSVaZJzssWLoT//Q/2VvPWLBHh3h/v5f6p9wOQkpJCUFAQE96KJS56N0Q0hdjONkdZca1aHTMo/FjGQLd3ISiM2y6G5dNeJTMz85T7Gtx2MIPbBvhEpUpVU5rkvOyuu2D/fmjQwO5IqsYYw7zb5zGqv3Vmk5ycTI8ePWhQNNNaocnAanfx8eOPrYm0T1LrHGhvnX7/7295vPXGq+Xua3f2bn7f+7uHI1RKVZUmOS+LjIQw++uGekTjqMa0rdeWvXv3snz5cjp2vILM1dVj6EBpFi60mi1LvSzd5nGIPo/mDSBq++tkHBk9XrrrJ13PA1Mf8E6gSqnTpknOB5Yuhb59YdcuuyM5PWvS1/C3r//G9qztAMyYMQOAs2q3IiZ4E66QulCvl50hnpa33oJffy3jBNQRAt3fR3CQeFkx40f/65T7Sro8idEDRnsnUKXUadMk5wORkdY1uZ077Y7k9GzYv4H52+dTM7QmYDVV1q9fn3/esA4AR9NroBpOVFxu6bg6XTGtHiY4CHqFfEr6nt1lrnrR2RfRMa6jZwNUSlWZJjkfaNPGGhTevbvdkZyeQW0H8edjf1Inog5Op5MZM2ZwxRVX4Ng12VqhGjZVHjZunFWdoKSkjBU6PE9xaCM6xbtY+OmpJyOdv30+M7bM8HiMSqnTZ0uSM8ZEG2O+NsasN8asM8b0sCMOXxM5xYepn8oqyAI4UlJm2bJlZGRk0K9nZ8hcgdNRExpeZmOEVVOnDrRoAVlZZawQUpOQHu8DcGm9n/n9VQfLX4thyZSRJ6361KynGDZnmPeCLUdystUsnpBg3Scn2xZKtZa8KZm+n/QlYVQCfT/pS/Im/UVWZ3adyY0CUkSkNXAesM6mOHxm504466wyxmb5KZe46PJeFx6adnQMWHJyMsYYOtbLBiAr/EoIqr49a/7yF2tAeN26Za+zZOlS0g8ZImsAIkSF5FJ/x/MnJbqPrv3Itomak5MhMRHS0iA21rpPTNREV1nJm5JJTE4kLTuN2LBY0rLTSExO1ERXjfk8yRljooCLgQ8BRKRIRLJ8HYevNW4Ml19uJbrqothZzD+6/oMBLQYcWZaSkkLXrl1pF2U1y8Wcd31Zm1crmZll9LIEgja9QW5RMNkF0OEscBgXxU5D0KY3jluvZZ2W1A479ewo3pKUZF1jDAuzOtJERlqPk5JsCafaSlqQRKgjlMjQSIwxRIZGEuoIJWmB/iKrKzvO5JoB+4CPjTErjDEfGGNOqmlijLnXGLPUGLN0XwCU2DYGPvoI+lSjCetrBNfgsR6PcWWLKwHIyMhg8eLFDL76Itj3CzhCcTQeUM5e/F9KCtSrB8uWlf58vbBs8kuC2bY/BJcLmtV10qCWk/rh2Set+/GKj/n898+9HPHJUlPh4EFYtw6c7vJ2ERGwbZvPQ6nWNh/YTFpOGvnF+UeWRYREsC1rm31BqSqxI8kFA52AMSLSEcgFnjxxJRF5T0S6iEiXevXq+TpGr8nKgh077I6ifNuytjF141Rc4jqybObMmbhcLgZ2CwGEveYyCImyL0gP6doVhg61El1p9hXUIjTISVBwCB/NCyIjG2qFuahbSyDz+AHgn/7+KeNX+75NOiEBQkKgdm0oKrKaK3NzIT7e56FUa02jmpJXnEdBScGRZXnFecRHx9sXlKoSO5LcTmCniCxyP/4aK+kFPBFo1w6eesruSMr3/rL3uW7SdaTnph9ZlpKSQmxsLE1kBQCpJYHRVFmnjlUx4uyzS3/e2eIxQoJchAaV0K1FCDe/A7//CWHBLphxAaQeTWqT/zaZaTdP81Hk1oD2Tz6xknRwsHU9LifHSnJ5edZyVb652+YiIjxz8TM0iWpCaFAoIkJuUS5FriKG9tRfZHXl8yQnInuAHcaYVu5FlwJ/+DoOOxgDb7wBjzxidyTlG957OPNun0fDmg0BcLlcpKSkcO2VvQnLmg3GwQWDrrE5Ss8RsZor09JOfq7rtcNIb/osh4ojiAkv4ba+Nek53PBrWgtw5sNvQ2DpI+AqpnZYbYwPpzdLSrIS9KWXwujREBdnvc969oR334UB1b812eumbZpGn0/68M26bxjQYgBvX/k2cbXiyCzIJK5WHKMHjD7uurSqXmwptWOMOR/4AAgFtgJ3iEiZs+BqqR37rVixgk6dOvHzJ/dzcfBYqH8JXDbX7rA8ZtcuaNIEXnqpYmfaTzzxBElJSWyY+hgtD40GV7E160uvL/nPgtHEhsfyeM/HvRJrfj4UFkJ0tNX87XJZZ3ClEal2U4r6jIhgjMElLj5b9RlDOgwhyBFkd1iVpX/dctgyhEBEVrqvt3UQketOleAC0Zo18MMPdkdROhHhhq9uYPL6ycctT0lJAaBbnDVty/Q/qu8A8NI0bgzffQf33FOx9YcPH05CQgJ/eWwqhRfPhPBGVmeclM4U7/2ZjRkbvRKny2Wdtd1yi5XAoqNLT3AlJTBwIIw8eTifwhq4f8GHF5CRl4HDOLjt/NuqY4JTFaAznthg5Ei4//6yu6zbaV/ePjYf2MzBgoPHLU9OTuaCrudRI3MOADvkOhui867rrjv1eLljRURE8O6777Jx40ZeGPMT9F9und3mp/GqWcS7Lc/1yh/Y4YC774YHHjj1GVpwsJUAa9XyeAgBoUZwDYqcRUcmO1CBSyuD22DLFqhZ03/L74gIguAw1neggwcPUqdOHT56fiB/P/trq25c/8D5exwmApMmWT0UK3ot6+9//zsTJ05kxYoVtG/bClY+Cetft56MH2LVpguOqHJcr7wCF1xQvYag+JvswmzmbJvDNa2sa8kucR15j1dj2lxZjmr/F66Omjf3zwR3qPAQhSWFGGOO++efNWsWTqeTfm1yrQXVpAJ4ZRlj1Zd7772Kb/P6668THR3NPffcg1Mc0Ol/0HMiBQTBts9hRk/I2VqluPLy4NNP4dtvK7+tCKxaVaXDB4zn5z3PDV/dwM5DVpN7ACQ4VQH6V7bJL7/AQw/5V5Ply/Nfptn/NSOvOO+45cnJycRE16JBiTXq457nA2PoQGmmT7em+aqounXr8sYbb7Bw4ULGjBljLYy/kddqDiYjuA5krYKULrC78tNCbd5sXVuLjLTeL//3f5XeBZ9/Duefb5V7OlMVO4sBGHbJMGbeOpMmUU1sjkj5kiY5m6xfD198AbvLrt7ic/2a9+Ohbg8REXK0eU1ESElJ4aG/nYcpOkCmsxVnn9vGxii9q0kTCKpk/4NbbrmFyy+/nKeeeood7pH+z1zzBXWu2wKNr4GiTJh7FaweCccMrj+VP/+0ktOLL1qP69Q5vV6S11wD77wDrVtXfttA8PzPz9Pv836UuEqoGVqTi8++2O6QlI9pkrPJrbfCnj1Wrz5/0SehD0/2On7ymT/++IOdO3fy1x4hAMScO5BnnrEjOt8ZMwYer0Tvf2MMY8eOxel08o9//IMj17lDa1N04STo8Lz1ePVz8PO1UJRV7j7POstqOr377srHf6zata1OKjVrVm0/1VV8dDznxJyD0+W0OxS/Z4yZa4y54oRljxpj3jnF+l18E93p0yRnkxo1Kn/G4C0iwofLPyQjL+Ok56yqA9Aq0ioU4WwUuE2Vh23ebF3HqkxTcrNmzRg5ciQ//PAD33zzDSJC5/c6k5j8MLR/BnpPg9AY2P0jpHSFrNUn7SM9HQYNsuahBHjsMc99CfrmG5g40TP78nfL05bz09afALj1vFt5/5r3qRFcw+aoqoWJwI0nLLvRvbza0iRXRcnJyfTt25eEhAT69u1LciVqmyxZAl26WB+qdlqTvoa7f7ibr/746qTnUlJS+OulCQQX7eFgcRNizulCQUEpOwkgr70GP/1U+ebBRx99lI4dO/LQQw+RlZXFoDaDjjaPNepv9UiNOR9yNsP0C2Db8Z8d2dnw22/WOEpPGzMGxo71/H79jYjwj2n/4PEZjx8376qqkK+Bq40xNQCMMfFAI+Bm92T5a40xI0rb0BiTc8zPg40x49w/1zPGfGOMWeK+XehefokxZqX7tsIY473BLiLi97fOnTuLP5o2bZo0a9ZMWrVqJZ06dZLWrVtLs2bNZNq0aRXafts2ke7dRRYu9HKgFfD7nt8luzD7uGXZ2dkSGhoqs/7XTWQ8smPyQ/LCCzYFaAOXq/LbLF26VBwOh9xzzz2lr1CcK7Lg7yLjERmPuJY+Kgt+KTrydF7eaQZbjj17REpKvLNvf1BQXCCFJYUiIrItc5uk56TbHJHPePSzFpgKXOv++UkgCYh1Pw4C5gId3I/nAl3cP+ccs4/BwDj3zxOAXu6fzwLWuX/+AbjQ/XNNINjTr+XwTc/kqiApKYng4GD27t3LgQMHiIyMJDQ0lKQKFvE6+2xrgt3u3b0caAWc2+BcaoYef+Fmzpw5FBUV0b2x1TumyQUD+c9/7IjO9z780JrBv7Cwctt17tyZxx57jPfff5+ff/6ZvOI89uTsObpCcARcMA66vA0mGLPhTYpSLmNuirVOeLjHXsJxGjSwmsf9qTevpxSWFNL7k948lvIYAGdHn029yMCpXOJjxzZZHm6q/KsxZjmwAmgHtK3E/i4DRhtjVgLfA1Hus7ZfgdeNMQ8D0SJS4qH4T6JJrgpSU1OJiIjA6XSyfft2srOziYiIYFsli3iVlGBbE+BrC17j0ZRHS23aSUlJofM54UQ6d+IKqcOfBRcF5IdkaRISoG9fq0ZbZY0YMYL4+Hjuu+8+mv+vOU/NOmEyTGOQFg/CZT8jYXFc0mYel+R3hn0LPBN8GZYuhVatvNMcaqcawTXo16wffRP62h1KIJgMXGqM6QSEA5nAv4BLRaQD1pleWCnbHfvJcOzzDqCHiJzvvjUWkWwReQW4232MhcYYr/X/1SRXBQkJCRQUFNCsWTNCQ0PZsmULWVlZxFeiiNeePda37I8/9l6cp7I7ezfbD24/aWCsiJCcnMyjg6xS5ttKruHs+OAzZmBx377W36R+/cpvGxkZybvvvsuGDRvontqdO86/47jnZ8+2Zi7JCe+JGbAc6l2Eyd8Ns3rDxne8drqVkGB1ZMnLK39dfycijFo4ivX71wMwos8IBrUdZHNU1Z+I5GA1Q36EdRYXhVXz86AxpgFQ1lxAe40xbYwxDuDY2SJmAImHH7gn58cY01xEVovIq8BSQJOcPxo6dChFRUUUFhbSvHlzRIQdO3Zw//33V3gfDRrA7bfDued6L85Tef2K1/nmr9+ctHzTpk2kpqbSr631iRjZaiBvvWXVwzuT7Np1tNJ2ZfTr148hQ4Yw7eNp1Mmpc9xzLhdkZkJGBhDeEC6dBa2sUj0s/QcsvB1K8kvdb1XUqQNz5kC3bh7ftc/tz9vPC/Nf4IPlH9gdSiCaCJwHfCEiq7CaKddiJb5fy9jmSeBHYDZwbMGqh4EuxpjfjTF/AIc/HB81xqwxxqwC8oHKz5ZQUd662OfJm792PBGxOp/06dNHEhISpGPHjhIcHCx9+/aVoqKi8je22Z7sPWU+N2rUKDmrrtU5QiZFipTk+zAy/zB9ugiI/Pzz6W2fnp4uderUkfO6nCe/bV4tM2cefa7UTiCpE0S+iLB+59POF8neenoHLkdursjq1V7ZtdelZaeJy90jaOuBrUd+PoPZ/vns7zfbA6jIzZ+T3InGjRsngNxzzz2V+gfcvVtkwwYvBnaCZbuXSdCIIJmyfkqpz/fv319evLWeyHjE+fMNMnOmyKFDvovPH2Rni7z0ksjOnae/j8Pvh3qXdZXISJF9+8rZIPN3kSnNrUT3VYzIruTTP3gZBgwQad5cxOn0+K69am36Wol6OUo+WPaB3aH4E9s/n/39ZnsAFblVpyQnIvL0008LIK+99lqF1ne5RJo2FbnuOi8Hdoxdh3bJkzOflMz8zJOey8vLk7CwMNk0ppHIeCR17gQBkYkTfRdfoCgudknnXp0lsmZN+eqrHRXbqDBTZM7V7mEGRmT18yIuz2Wk+fOts9PqdhJU4iyRx6c/Ltsyt9kdij+x/fPZ32+2B1CRW3VLck6nUwYPHizGGJk8eXKFtpk2TWTtWi8HVkEpKSlSLwpxjTciE0MlJ/OgzJhRgbOQAFRSIjJnjsjW02g5fPpp64vLpk1bJDw8XK699tqKn927nCK/j7CS3HhE5v7FSn5nmIy8DLn3+3tL/TKmRMQPPp/9/aYdT7zA4XDwySef0KVLF26++WZWrFhR7jYDBkDbyow+qYJpm6axZNeSMp9PTk5mUPdgDAINLiUyOorLL694QdFAkplpVeL+6KPKb1u/PjRsCE2ans3gBwczZcoUvq1ovRzjgHOHwSU/Qkg07PrBPR2YZ/r/HzgAI0bApk0e2Z3XbNi/gfGrx7Nw50K7Q1HVlBZN9aI9e/bQrVs3XC4XixYtonE5ExEuWmRN8XXLLd6LSUToMLYD9SLqMfu22aWu07p1az69M4NuTfZDt/eZsPhuunSBli29F5c/mzsXuna1St6UZ8kSqzfmBRdYIwGMsX7njV9rTMGYAmrk12DdunVER0dXPIDsLTB/kFW2JygCLvgIzv7b6b4cANLSrMHuo0ZZVer9zYb9G2hVtxUAGXkZ1ImoU84WZywtmloeu08lK3Krbs2Vx1q1apXUrFlTOnXqJDk5Oadc9/bbRRo08H6HgIMFB2VTxqZSn9u6davUCkeKPw8SmeCQnP17xRiR4cO9G1MgcDpF2rUT6dHj5OtdmzI2ycJFC8XhcMi9995b+Z0X54r8OuTIdGCy9DERZ9V68O7dW6XNveb9Ze9L8MhgWbZ7md2hVAe2fz77+832ACpyq85JTkTkxx9/FIfDIdddd504T5HBdu0SOXjQh4GV4p133pG/XeD+IJ1xkbhcIqmpVu/PM5XLJfLeeyJffVX68wUFR4cEbNggsn9/2fv65z//KYDMmzfv9AJZ/5bIhGDr7zPzEpG8soeBVJS/9bLMys+Sl+e/LMXOYrtDqQ5s/3yu6A1rwPdvQCHwL18dV6/J+cBVV13F66+/zuTJk3nqqafKXK9RI4iK8l4cf+z7g8s+vYwN+zeUuU5KSgq39nYXTW16PcZYzVpxcd6Ly98ZA++/X3qpmpwc6NXLur4FVpNunVJa1kpcJbw0/yUuGHIB8fHx3HvvvRRWdmJMY6BVIlw2F8IaQvrPkNIJ9p/+9apXXz3atGqnedvn8ffv/o7T5aR2WG2e7PUkwY5ge4NSnnYAa3D4a748qCY5H3n44Yd54IEH+O9//8uHH35Y5npz5sCQIdasGJ6289BOdhzaUeb1jcLCQn75+ScubVNsLWhyHR9/DFOnej6W6iYlBb7++uTlkZFw4YXQqdOptw8yQby//H0W7F3AmDFjWL9+PS+99NLpBVPvQhiwHOr1gvzd8NPFsGnMaWWqpk2tCuR2T/W1MWMji3YtIj033d5AzgDG0N8YZhnDVvd9/6rv08QbY9YbYz5wz2Qy3hhzmTHmV2PMJmNMNxFJF5ElQLEHXkbF2X0KW5GbXzdX7pom8lMfkcnx1v2ussvsFBcXS79+/SQ4OFhmz55d6joTJoicfbZVhsdTpm2cJn3G9ZH4N+Ol98e9ZdrG0mOcNWuWXNXR3VQ5rZOIiLRsKXLTTZ6LpbqaNk2kTx+R+HiR3r1F7r1X5M8/K7ePvKKjdXRuvvlmCQkJkTVr1px+UM4ikSUPH71O99NlIjMvrtB70S7Hvhcv/uhieWvRW0eeO/b3Y6tK/E/7gUo2GUp/kC0gf4Asdd9vAelf2X0dv1/igRLgXKyTp2VY04AZ4Fpg8jHrDseHzZXau7IqdifDkkSru3dwLXDlg7MIuo6GRqXPY3rw4EF69uzJ7t27WbhwIa1atTrueacTHI7KF+wsS/KmZBKTEzEYokKjyC/Jp8hVxOgBoxnQ4vgYn3jiCdoeeo3bLxbo8Dy0f4aSEmsm/tKa4M4UycmQmGg1TZaUQEwMbN0KN90E48ef3j7T09Np06YNrVu3Zv78+TgcVWhUSR0Pi+4CVyE4akDNcwBnue/FY23eDLVqWXOpesvh92KoI5SIkAg2HdhEXnEeX93wFQPbDCx/B75w+H/aEQKOUJDiSv0ebVCpTwpjmAXEAceeu0cAaSJcetpBWAVWZ4pIC/fjT4HpIjLeGNMM+FZEznc/Nxyr/pxPmi3L/c8yxiQaY2J8EUy1sy7JSnC52yFvOwSFQVCotbwMtWvX5scffyQkJISrrrqKjIyM454PCjrc7dwz10mSFiQRYkLYlb2L7Qe3ExkaSagjlKQFJ8c4I2UaA7u5r4M0sT50goPP7AQHkJQEoaHW38bphNq1rRn9d++u3H4KSwq5bfJtfLrqU+rXr8/rr7/OggULePfdd6sWYMItEN0BTIiV6LLXg6uk3PfiYfv3Q5s28OabVQujPEkLkgh1hBIZGokxhrNqn0Wjmo14a/Fb3j1wZaxLAkcQFOyB7I0V+p+uZhI4PsHhfpzggX0fe5HZdcxjF2DbBdaKfH1sCCwxxnxpjOlvjKfOMQJATipgQJxQdAAObQAckLPtlJslJCQwefJkdu7cyfXXX39SB4QlS6B1a1i9uuohpmalEhkaSYPIBsSGxwIQERLBtqzjY9y5cycxzrXUDiuGWi2hdls+/RRefLHqMVR3qakQEWF1DDrnHOtLSHQ0bN9euf3UCK7Bhv0b2JuzF4C///3vXHbZZfz73/9m165dVQuycB9EtYGQKOv9mLMZig5Cdmq5m9atC599Bg8/XLUQypOalUpESARZBVm4xEVYcBj1Iuud9F601aENkPsnFB8EKQFngTU2sZz/6WokFevM7VgR7uUBqdwkJyLPAC2AD4HbgU3GmJeMMc29HJv/q5lgfeuLam01Eznz4dB6CKlZ7qY9e/bko48+Yt68edx3330c22x81llW1fDKdr470aQ1k4iuEU1ecR51I+pSO6w2AHnFecRHxx+37vTp0xnYxf2g6UAwhvnzYfLkqsUQCBISTu6YkZdn9TqtrIV3L2TohUMBMMYwduxYiouLSUxMLGfLctRMACmymirD3F1hC9LAmWMlu3LceKP3e9AmRCdwIP8AWzK3sD9vP1D6e9E22yZA/h7rbDgo3PrSEBwJzjyoGW93dJ6SBNTgaKKLcD/2+qmqMaahMWYn8E/gGWPMTmOMF/uTWyp0IUCsT+A97lsJEAN8bYz5rxdj839thlrt9bisRBdc0/o5azX8PhxKqbZ9rJtvvpnnnnuOTz75hFdeeeXI8gYNYMYMa5aN01XsLOb5ec8TFhJGkauI3KJcRITcolyKXEUM7Tn0uPWTk6cxuHuQ9aDJ9YDVbf63304/hkAxdCgUFUFurtWEnJtrPR46tPxty9O8eXOGDx/O5MmTKz7lV2kOvxedeRAeB+FNAAcU7ofp3SBrbbm7WLoUnnzSe8MJhvYcijGGplFNiQmLKfO96HOuYlj2KCy4BXBZ/8cRTa1rciW51u+1jc0xeogIKcA/sGq+xbrv/+FeXoX9yjYRaX/M49tF5OtjnxORPSLSRESiRCTa/fOhqhy3osGV12vmYayeMtOBG4AQ93IHsMUXvWOqR+/KBJGZvUUW3HZ0Ut05V4oUHjjl5i6XS2666SYB5KsTRhvn5la+vE16TvqRQbQ7D+6UwpLCIz3aEt5MkD7j+pzUu7KoqEh6nxtpxfxtY4/OeB8oDveuTEiw7qedZoe7vKI8ueCDC+SN3944sqyoqEjOO+88iYuLk8zMzNMP8tj34k99RDZ/KDL13KM1AbdNOuXm77wjEhMjsqOCxRJOR3nvRZ/LSxOZcZH1O5oQLLLhbZGdU4//PQZQ78oz8VaRJDcSOLuM59r4Iki/TnKl2T1d5KtY6x9nSjORAytPuXp+fr706NFDwsLCZPHixSJizfgfESGSlFTxw+7P3S+N/9dY/j3z35UKd/78+fLiX7HiXfwPERGZPNkaOlCVz1xVupu+vkk+W/XZccuWLFkiDodD7r//fs8erDhH5Jebjw4zWPa4SBkzieTnW1+svOX9Ze/LS/NeEqe/fIlK/1Xk2zj3l7s463H1Y3sS8fdbRa7JDRORUi+xi8g6D5xMBp64ftB/GcR0gpytMKOH1c27DGFhYUyePJmGDRtyzTXX8Oeff1K3Ljz9NFxyScUPWyeiDg90eYAb299YqXCTk5O5/nDTaFOrqTI9HVassLqVK8+aMGgCQzoMOW5Zly5deOSRRxg7diy//PKL5w4WHAk9P4fOo8AEw/r/wezLoeDkQddhYVYHG7B6kXragh0LmLl1Jg5j8xwUIrDxbZjVG/LToN5F0H851OsJWP8Pffv2JSEhgb59+5KcnGxvvKpKdJycN5Xkw9IHYes463HLh6Dja1aX5FKsXbuWnj17Eh8fzy+//EKtCmSY/OJ8npj5BI/1eIxmMc1OK8xBl7flmzvWQWgsXL8XdDolr3OJi8KSQsJDwo8sy8nJoX379oSHh7Ny5Upq1Kjh2YOmz4dfboCCvRDeGC76Bup2P26VwkKrtFC/fjBsmGcPD1BQUkBYcJjnd1xRJfmw5H5I/dR63OoR6JhkjYvDSnCJiYmEhoYSERFBXl4eRUVFjB49mgEDqv84uTORTuvlTcHh0P0j6DrG+ifa+BbM7mt9eyxFu3bt+PLLL1m7di033XQTTqeTTZtg5cqyD7E3dy8T1kxg5paZpxXi3r17aRXhPiFvco0mOB/IKcqhflJ9Ri0addzymjVrHpny6+WXX/b8geu7z1jq9oT8Xe7pwN49rqdJjRrQoYPVw9eT8oqt7qm2JricVJjZ00pwQRHQcwJ0fvNIggNISkoiJCSE/Px8du/eTWRkJKGhoSQlBcw4uTOP3e2lFblVu2typdn3m9WpYzwi3zQUSf+lzFXfeecdAeTRRx+VFi1ELr/85HXW7Vt35Of9uaeY9r4cn3zyiSx53n29ZscUERFZvFikc2eRlae+lKiq4D+z/iM/bfmp1OduuukmCQkJkbXeKhVfUiiyJPHodbrf7hQpyffOsURkbfpaqfVSLZm+ebrXjlGuXckiX8W4r5M3F8n8vdTV4uLiJCIiQoAjJbI6deokCQkJPg64wmz/fPb3m57J+UrdC6zrdPUvsWZT+Kk3bHir1P7aDzzwAI888ghvvvkm1147lvffP/752amzafdOO75dZ3U5r0pBySVzv6FLM5DgSGh4OQDFxdasHmdiJXBfeaHvC1zarPRZlN58801q1arFvffei8sbM3UHhUKXt6DHp9aMHls/gpm9rJl73JxO+PlnzxyuRlANBrUdRMeGHT2zw8oQF6x5AeZeCUWZ0Ohq6L8Uos89brW9e/dy5513kpaWRlFREfHx8bRs2RJjDHl5ecSfzqBIdRxjzC3GmN/dtwXGmPN8cmC7s2xFbgFxJneYs1hk2T+Pfov+dYhVEPMEJSUlctVVV0lQUJBMn378N+BiZ7G8OO9FySk8dRHW8pSUlMiTAyOsOOYNrtK+VOXtyd4je3NKr1z68ccfCyBjxozxbhAHVlhd5ccj8nUdkd0zRERkzBgREFm+3LuH96rCTJG5f3H/rxmR30ecNDymqKhI3njjDYmKipKQkBAZPHiwxMfHS+vWraVTp07SunVradasmUw73TEj3mf753NFb0BPIMb98wBgkU+Oa/cLr8gtoJLcYdu+sMYujUdkageRQ5tPWuXQoUPSoUMHiYyMkvuenCLXfXGdHCqo5MC5U1i4cKHM+Y872aaOP7L8xKrWyvOy8rPEMcIhI+eOLPV5l8slffv2laioKNm5c6d3gynIEJnd3z1WzCGy5mXJPOCSL78UKapa8XH5ZfsvkpqZ6pEwKyVztciUc6zX9GW0NfbtBLNmzZK2bdtaU/P37y/r168XEZFp06ZJnz59JCEhQfr06ePPCU7kdD70h9Of4cxiOFvd91WqQCBypArBeuADYA0wHrgM+BXYBHQ7Yf0YYFdVj1uh2HxxkKreAjLJiYhkrhH5vsUp/xG3b98u4eENxBHaUBo810BW7VnlscP/9/mhUvIZ4poQIlKYJSIiGRki0dEi48eXs7Gqsg+WfSBr08u+7rZp0yYJCwuTgQMHej8YZ4nIqmePtjD8PFCkqGpl6l0ul7R9u630/LCnh4KsoG1fiHzhbqGYet5JXyC3b98ugwcPFkCaNWsmU6ZMEVf1/WZ3OgluC8P5g+Esdd9vqWqioxKldtzr/wv4oCrHrOhNr8nZKbodXLEEGl8DxVnw89WwesSR6cBKXCXsMruYMuV7Qh1ZxE+Pp2Xtlh47fNHWrwlygGl4KYRa81oWFFjzGLZo4bHDqDLc1eku2tZrW+bz55xzDs899xzfffcd3333nXeDcQRBh5Fw8fcQUht2foekdGPSe3/w6aent0tjDCm3pPD2lW97NtayuIph2T/h1xut6c3ih0C/BVDLmma3oKCA559/ntatWzN16lSef/551q5dyzXXXMMZNO/8UKzqAIdnY81zP/bEvGWpIrJaRFzAWmCWWBltNVYSBMAY0we4C/i3B45ZLk1ydgutDRd/Bx1esB6vHg4/XwtFWQyfO5zen/SmZdcGfPbZZyxauIg777zz8DehKsnIyOC8Ou6Jx5sereXVqBGMGVO1eTNVxZS4Spi/fT5bM7eWuc7jjz/OeeedR2JiIgcPlj/RcpU1+Yv1xat2e0z2Bv4S2p09i0spiV5BTWs35fyG53suvrLk74XZl8GGN6xB753dHWuCIxARJk+eTNu2bRk2bBhXX30169ev55lnniEszMYhDfawtdSOMaYDVpPmtSJyfJ0xL9Ek5w+MA9r/B3onQ2gM7P4RUrrwrzb9+OS6Tzg7+mxq1x5Mq1YvMnHiREaOHFnlQ86ZMYXL24NgoPG1R5Yf8v50qcottyiX3p/0ZtzKcWWuExISwvvvv09aWhpPPfWUbwKLagFXLISzbyQiNIcnet4AK56watRV0NbMrdzy7S2+KaOzfyGkdIL0eRDWEC6bC60SwRjWr19P//79GThwIBEREcyaNYsvv/ySszw9ELD6sK3UjjHmLOBb4FYR2ejt4x2mSc6fNLqCL5oMJZVoyNlC9Lz+3FjTarrMy4Pg4KcYNOg2hg8fzsSJE6t0qL0rx1EjBGtgcLhVDlrEqpf22GNVfB2qQmqH1WbmrTN57IJT/8K7du3Kww8/zJgxY/j11199E1xwpDVYutMbYIKsoqFzroCCfRXa/Pe9vzN983RCy5jdxyNEYNMYa1B7/m6o1wsGLId6F3Lo0CGGDh3Kueeey6JFi3jzzTdZsWIFffv29V481YNtpXaAYUAd4B1jzEpjjE+msdJpvfzMW4veYuam7/kmoT4h2ydYC1s9gpyfhAkKobCwkMsvv5zFixcze/ZsevbsWeljuFwuvv93BNd1LIROr0Nr60O2uBhGj4b27eHyyz35qlRV5eTk0K5dOyIjI1mxYoXnp/w6hVUz5xG35a/Uj9prlaC56BuoU357dpGzyHtJriQfljwAqZ9Yj1s+DJ1eQ0wwn3/+OU888QR79uzhzjvv5OWXX6Z+/freicN+lb6YaEaY/ljX4BKwzuCS5DmpUqkdv+aL3i1VvQVs70q3bZnbZNHORSJi9UhzupxWP/6N74hMDLF6ic24SCQvTZxOkfT0fdK8eXOpV6+ebN26tdLHW7H0Nzn0gbsXXXaqh1+Nqozswmx5f9n7Feo1O3XqVAFkxIgRPojsqH37RO68cafkfneB9Z6ZGCqy6b0y188uzPZuQNlbRaZ1tGL5Ilxk6+ciIrJs2TLp2bOnANK1a1dZtGiRd+PwD7Z/Pvv7zfYAKnIL5CTncrmk54c9pdVbraTEWXLyCukLRL5tJDIeKfwiTq7p8assWCCyfv16iY6OlrZt20pWVlaljvnF67eKjEeKprQ/bnlqqkhO1caXq0rKLsyW4JHBMmJuxRLXjTfeKKGhofLHH394ObJSlBSKLH7w6DCDhXedNB2Yy+WSc985V+7/wcMlgw4rpYzV/v375b777hNjjNSrV08+/PBDcTr9pJyP99n++ezvN9sDqMgtEJOcy+U6ktTWpq+VDfs3lL1yXprIzIutxPRZiGyfOVrE5ZJZs2ZJcHCw9OvXT4qLS68RVpofn2lgfUj8fvxA5F69RHr6eEiTEknNTK3wOK09e/ZITEyM9OrVy+cf5Pv2iSxZIiJbxol8EWa9h5K7iORsP7JOUUmRvPrLq/L12q89e3CXU2T1C8cVJC7OTZe3335bYmJiJCgoSB599NGqFZ2tnmz/fPb3m+0BVOQWaEmuqKRIBn4xUJ7+6emKb+QsEln66DHTgd0qUpwr77//vgDy4IMPVuiDMuvAfkkf495H5urjnps+/fQrXivf+fDDDwWQsWPH+vS4l1wi0rKle0acjOUik+Pd04HVFUkrfbJpjyjMEvn52qPv/VXPybyf58p5550ngPTt21fWrFnjveP7N9s/n/39pr0rbRASFEKDyAbUjajEDMiOEOj8BvScgARFwLbPYEZP7r6xL//617945513eOutt8rdzcqf3qZeFOQFN4Ha7Y57rl8/8M+SWYEtIy+DoTOG8uufFes5eccdd9CnTx+eeOIJdu/e7eXojvrvf+Hbb8EYILajNdFx3BVQuB/m9OPQimdI2ZSMSzw4qXTWWpjeDXZOgZBo9rf/hJtf2cjFl/TmwIEDfPXVV/z000+0a9eu/H2pM5PdWbYit0A5k/v2j29le5bVtHO60whlZYl0bPa77P/YPS/fVzFSsuNHufbaa8XhcMiPP/54yu1nvtJOZDxSsvTx45avWyeyapXOW2mH3KJcqfVSLXl78dsV3mbjxo1So0YNGTRokBcjqwBnicjKZ46cZX01Ftm+z0NnVdsmHZnf1flDexmT9C+JjIyUGjVqyLPPPiu5uSdPbH4Gsv3z2d9vtgdQkVsgJLn9ufsl6uUoue+H+6q8r7ffFlm9PFNkztVHZlgvXPasdOp4vtSsWVNWrSq9p57L6ZSdo4Osbfb9dtxzd90lEhurSc4uhSWFld7mpZdeEkAmT57shYhKt3u3SGKiyOYT5xPfMVlcX0ZZ760fWotkrSt1+wpxFosse/xI4tz5ZW/p0La5AHLttdfKli1bqvQaAoztn8/+ftNxcl6WV5xHRIg17nJ52nLa1WtHjWAPjXESF6x5EVY/Bwj5dS6n04OryS0OYdGiRcTFxR23+ubFkzhn843kuqKJvCXDmmnFLTUVtm6FS0svcab8UHFxMZ07d+bAgQP88ccfREVFef2Yu3dDy5bw3ntw880nPHloI8y/Hg6uheBa0GMcNL2+cgcoSIdf/gbpcxETzPtLW3Hf62tp2bIlo0aNon///p56KYHijJl083TpNTkv2nJgC61Ht+aLNV8A0Cmuk8cS3IoV8OsCB5z7LPSeCiHRhGfMZMUrITSosY9rr72WvLzjp6jbv+I9AKTxNcclOICEBE1wdtqdvZu/TPwLM7bMqPA2h6f82r17t8+m/GrUCPbsOTnB3THlDibv/gP6LYSz/gol2TB/EKx8ClzOiu18/yJI7gTpc8kujuTSF+Hx97bz3//+l9WrV2uC8xBjTH9jzCxjzFb3vUd+scaYIcaYxe7ZTN41xgQZY3KMMS8aY1YZYxYaYxq4173BGLPGvXyeJ45fFk1yXtS0dlN6ndWLFrGen9L/zjvhySfdDxoNgAHLIPo8wop38NsIoXnwEm677bbjKkvHORcCULPN34/b1+bN8M03kJvr8TBVBcWGx7I9aztZBVmV2q579+489NBDjBkzhgULFngnuBPUrGndFxVZ9wfyD7Bs9zJ2HdoFITXhwi+g4/+s6cD+eAXm9oeC/WXvUAQ2vYv8dDHk72LJtlBaPZpL4443smHDBoYOHUpoqBenBzuDuBPa20AccMB9/3ZVE50xpg3wN+BCETkfcAK3AJHAQhE5D5gH3OPeZBhwhXv5NVU5drnsbi+tyK06XZPLL86XEXNHSG6Rdy+Kr1ghkp5+wsLiXGtogftaxv9uQZ55+t8iIpKze6nIeCT3kzBrOMIx/vtfEShlf6paOHTokDRt2lTatm0rhYWVv7Z3Ov79b5Hzzz96Ddflckmx84SxmnvmiHxT33o/fneWyP4lJ++oJF/ktzuPvGf/7+9Il04d5JdffvH6awgQlfosBWYBfwBLj7n9gVUW57Q/o4FEYDew0n3bAAzHqkRw+LLY33DXkAPGAjPdSa9OVY5d3s22a3LGmCCsX/AuEbn6VOv68zW55E3JJC1IIjUrlYToBPqf05+nZj3FpMGTGNx2sO8DEoGNbyPLH8NICT+vg0V5A7iw9k9ceE4xG/YEc6j5s3S9dtiRTQoL4Y8/oGNH34erTiYila5vNnXqVK6++moSEhIQERISEhg6dCgDvDQmZNIkWLkShj6dS62IUEKCQkpfMW8nzB8MGYvAUYPU4P5kbvuZemHZHCyO5Kz6EUSZPeQXwT+/iOS8617jnnvuISgoyCtxB6BKvVGMMVuxzuBOFCsizU47CGMeAhqJyFMnLM8RkZrunwcDV4vI7e7H3YGrgDuA88VLpXfsbK58BFhn4/GrLHlTMonJiaRlp1ErtBZp2Wm8u+xd3r3qXZ8kuJQUePnlExYaA60SMZfNRcLiuKQN3JKQzFkxxQCEhwr1dzzPkilHy/XUqKEJzh9szdxKy7da8t36yhdIdTgcREZGsm3bNiIiIkhLSyMxMZHk5GQvRAp/+5v13ntnxRskjErgUGEZNZoimsBlP8M594OrkISiKTSPOYRTHDSLPkSU2cPWdHhj7Q28MH47999/vyY47/JWqZ1ZwGBjTH0AY0ysMebsslY2xjQXkUUiMgzYDzSt4vHLFOytHZ+KMaYJVgZ/EfinHTF4QtKCJEIdoRQ6C9l4YCOt6rQi1BHKhDUTuLvz3V4//uzZMHEiPP44nHTJot6FmAHLyJrQlMax1oX//CLILQolJMhJ0KY3gGHs2wfvvw+33AJnl/mWVL7QJKoJ7eu3JzosutLbJiUlERcXR2pqKqmpqdSuXZvi4mIeeOABbr31Vs8H67Z08zYauxJIyiyvUktdmu0PZkjPEmqHu6gdbl0rXrAR0nIiefq1L70WozpOEtY1ObCKpXqk1I6I/GGMeQaYYYxxAMXAP04VhzGmBdaZ6CxgVVWOfyq2NFcaY74GXgZqAf8qrbnSGHMvcC/AWWed1Xn79u2+DbICEkYlEBsWS4mrhLScNBrXaozDOMgsyGTrI2VXe/aUnBwID4dTffHdMToIp8sQX9fJ+jQHLhMGCLVCi2maWMyMGXDFFfDLL3DhhV4PWXlJQkICsbGxZGZmsn379uM6HDkc3muwcbmsxoMKta6Ki07x8NUjEF8PFmxyULtmCFE1nDRNLPZajAGu8qV2TCmldiRwS+34/EzOGHM1kC4iy4wxvctaT0TeA94D65qcb6KrnIToBNKy04gMjeSs2lal4dyiXOKj431y/MO93E5lX0EUUSF5bNoXgrg/iUKDnOwrqElTrKm8Dhyo2L6Ub+QX52OMISw4rMLbJCQkkJaWRmxsLLGxsQDk5uYSFxfH7NmzvRLnhNUTcGwZwJV9Y6jIEL3lr8UQFZJHvjOITfuE6FoOQoNKjrwXlW+4E1rAJrUT2XFN7kLgGmPMNuALoK8x5nMb4qiyoT2HUuQqIrcoFxEhtyiXIlcRQ3sO9VkMycnQs6dVObw0zhaPERLkwuBCgNCgEkKCXDhbHK1GHRMDIWX0G1C+tW7fOqJfjWbK+imV2m7o0KEUFRWRm+t+L+bmUlRUxNCh3nkvbsrYxC3f3sKu+h9VKMHB0fdiSJCTYqcp9b2olKf5PMmJyFMi0kRE4oEbgdkiMsTXcXjCgBYDGD1gNHG14sgsyCSuVhyjB4xmQAvfzXIcGmo1FaWllf5812uHkd70WQ4VR1ArtJhDxRGkN7V6V7pccN998PPPPgtXlaNFnRY8dsFjtKnXplLbDRgwgNGjRxMXF0dmZiZxcXGMHj3aa70rW9RpwYr7VnBXp7uYNQsSE8vf5lTvRaW8xdZpvdzNlaVekzuWPw8hqM5274bzz4dXXrEGlyt1Ot55B159FZYsgfr17Y7mjKPTepVD564MEMXF4HCcuhNKaUTA6YRgW/rZqtKICBsyNhBXM47aYbXtDuckY5eOZVPGJl69/FWCHcEUFVnvPX0P2UKTXDl0Wq8AsHy59Q16zpzKb2uMfjj5m1V7V9Hm7Tb8uPFHu0Mp1eYDm1m1dxXBDuuNExpqvYdcLmtiAaX8iSa5ANC6NVx/PdStRA1WgHvusZqalH/p0KADH/zlA3rH97Y7lFK91u81pg+Zftyy3Fxo1w6SqjTaSgU6Y0y8MebE+hVepd/hA0BEBHz4YeW2EbFK6zRu7J2Y1OlzGAd3dbrL7jBKdSD/ALHhsQQ5jm8Xj4yEK6+E9u1tCkxVF/HAzcCEE58wxgSLSImnD6jX5ALIrl1W82OjRnZHoqoqpyiHWVtn0aNpD+pH+kdvjvX719NhTAcmDprIoLaD7A5HWSp/TW7CyYPBublqg8GNMc8D+0VklPvxi8BeEfm/E9ZbCLRxH/cTIBNr9qswrIoFIzmmM6IxZjSwVETGGWM6A68DNbGmArtdRMroV36UNlcGiJwcaNYMRo2yOxLlCamZqVw36TqmbZpmdyhHRNWI4uHuD3PR2ReVuU5+Pvzwgw+DUpUzofRSO+7lVfEhcBuAe1qvG4Hxpaz3JDBfRM4XkTfcy3oAt4lI37J2bowJAd4CBotIZ+AjrGkhy6XNlQGiZk34+GPo0qVi6z/8MBw6BOPGeTUsdZra1W/HvNvn0a1xN7tDOaJRrUa81u+1U67z7rvw2GOwZo11jU75naFY5W8OTx+Rd8zy0z6bE5FtxpgMY0xHoAGwohJVBWaKSGmVEY7VCmgPzHRX6AgCyj2LA01yAeXEas2nEh1d+eEGynccxnHKMyZfm506m5iwGDrGnbpcxW23wXnnQdu2PgpMVVYCJ5fayXMvr6oPgNuBhlhnWhV1bLnmEo5vYTw8t50B1opIj8oGpc2VAWb+fJg+vfz1Ro6EN94ofz1lnz05e/jvr/9lx8EddofCEzOf4P6p95e7XkwM9OlTwQmb/VRyMvTtCwkJ1r2XqhXZxVuldgC+A/oDXYGyPoWysSbmL8t2oK0xpoYxpjZwqXv5BqCeMaYHWM2XxpgKtRVokgswTz4Jw8qZJaka9DVSQGZ+Jv/+6d8s2LHA7lCYeetMPr724wqt63LBCy9UvsevP0hOtqYoS0uD2FjrPjExoBJdElZpncOJziOldgBEpAiYA3wpIs4yVvsdKDHGrDLGnDRpqYjsAL50rzceWHHMvgcDrxpjVmFVH+9Zkbi0d2WA2bQJGjaEWqf4rvTCC/Dpp7B6tVUwVfknEWFv7l4a1mxodyiV1rs3tGhh1SqsTvr2tRJbZOTRZbm5EBdn1W/0Q37RuxKOdDhZDtwgIpuquj9P0WtyAaZFi/LXadPGqiGnCc6/GWNsT3Cpmak8OO1BXu/3eqUmjU5JgbCKVwryC2lp1he/s846fnlEBGzbZktI3nGz50vtGGPaAj8C3/lTggNtrgxIU6bAqSqsDBoEb73lu3jU6dt8YDN3TbmLzQc223L8LZlb+GPfH9SqcarLKCc7nOByc0+9nt1KSqzmVbB6Gu/fDwcPHr9OXh7Ex/s6supFRP4QkWYi8jiAMeZcY8zKE26L7IhNk1wAWrkSJk+GgoKTnysoKH258l/frf/OtiR3WbPLSH0klSZRTSq97bRp0KABrF/vhcA8YPVqOPvso3O+3nsvfPCBdc06N/fofVHRqb80qpOJyGr3WLhjb93tiEWTXAB68knYuLH05qIffrCu161d6/u4VOU1j2nO/if20/+cqo7Vrbz03HREBIc5vY+JLl3gb3+zJnD2By4XfPXV0d7HLVrAhRdaY0wB6tSBu+6C0aOta3CZmdb96NHgpbJ8yge048kZZvVqmDABRozwnw8f5X9EhI7vdqRV3VZMGjzJ7nCqJCvLGhcqYg1Qb9UKvvvO7qg8phoP1vANPZMLUFOnWv/QJ15fOPdcePllTXDVyYq0FVz40YWsTffd6bdLXDzc/WFubl/1CeO3boVff/VAUKfhiSes93xJiTV2LyUFvv7anliUPbR3ZYCqWxeaNoV9+6C2u+5mSQls3gwtW1pFLlX1EBMeg9Pl5GDhwfJX9pAgRxB3dvRMufhbbrHmVv39d+8PEk9Nhddfh+eft87e+vWz/heKi62adyf2nFSBT5srzyC//25NuTR+fOWmAFNnlt3Zu5mTOofBbQdTI7jq40xWroR69bxX1unQISuJ1akDy5ZZ19m+/95KcGcAba4sh36fD3CHe4eBVYLno4+sgbqq+hERfPGldMLqCdz63a3syt7lkf2df773ElxurnV29uqr1uPOnWHv3jMmwakK0CQXwFautJpqUtzDPuvWhTvu0Hpz1dG87fOo/1p9Vu1d5fVj/bPHP1lyzxKaxTTz2D63brVaD1I9MEPihx/CU09ZP0dGwksvWb04DzvcPK8UaJILaO3awYMPWnXmwJqWaN8+e2NSp6d5THOubHElIY4Qrx/LYRx0btTZo/sMDYWZM60SPJXlcsGiY4YRr15tdWRxumdHfPBB6wxOqdLoNbkzRG4uREXBf/5jVSBQqjR//eqvXNniSm4//3aP77u4GEJOI0e/9x7cd591Tfncc09/PwFKr8mVQ3tXngE2bLB6U86bZ03erKqv/Xn7iQmLIcjh+WKA2YXZ7M/bT3Zhtsf3DUcTU3b2qScQ37sX7r8f7rkHrrzSmoYuIgLOOef4/ShVEdpcGeDy860elX36wJAh1gdHAJUNOaNM3TiVekn1WJ623Cv7r1WjFrNvm01it0Sv7B+smUMaNbLmgjy2Vtv27UebJGNjrWt3Ge660nXqWO/d8HCvhaUCmCa5ADd3rjVeqKTEqjoQgPWxzhhdG3flhT4veKUyQXZh9pEzOOOlwWzJybB8uZWsYmNh9+6j78VBg+CBB6z1QkJgxQq49VavhKHOMHpNLsAdro+1davVE61ZM7+vj6Vs8NL8l3j111fZ+vBW6kTU8coxjq3VlpEBe/ZYEyQ3amQNAahf33qsKkWvyZVDr8kFuNRU61tz69ZHS4oEXH2sM0ixs5glu5fQtVFXQoI8d3Gq/zn9cRiH1xIcHH0vgjX7SFiY1ety2zbo2tVrh1VnOG2uDHAJCVY9rJCQo0VStT5W9fX9hu+58KMLWbrbsy0bneI68WSvJz26zxMdfi+CNZateXOrp6S+F5U3aZILcEOHWjOeaH2swNA3oS9f3/A1beu19dg+31v2HjsO7vDY/sqi70VlB01yAW7AAK2PFUhiwmMY1HYQtcM8M63Hnwf/5IGpD/DFmi88sr9T0feisoN2PFGqmtmbs5dpm6Zx87k3e2QC5e1Z26kdVpvosOiqB6d8TTuelEPP5JSqZhbuXMid39/J4l2LPbK/s6PP1gSnApYmOaWqmUubXcrqB1bT66xeVdrPO0ve4bbJt1HkLPJQZEr5Hx1CoFQ1UzO0Ju3rt6/yfg4WHGRf7j5Cg7RMvApceianVDW0fv96nvrpKfKL8097H09d9BRTb57qwaiU8j+a5JSqhrYc2MJrv73GmvTTqF2D1asSvDeFl1L+QpOcUtXQ5c0vJ+vfWXRtXPmpQjZmbCT+zXjGrRzn+cCU8jN6TU6paig0KPS0r6XVjajLi31fpP85/T0clVL+R8/klKqmFuxYwKAvB5FTlFOp7WLDY3nqoqe8Us1AKX+jSU6paiqnKIeVe1ayPWt7hbdJ3pTMzC0zqQ6TQCjlCTrjiVLVlIhUuuPIJeMuIbcolyX3LNFOJ4FB/4jl0GtySlVTp5Okpg+Zzu7s3Zrg1BlDmyuVqsZ+2PAD7d9pz6HCQxVaPyw4jGYxzbwclVL+Q5OcUtVYdFg0TWs3ZX/e/lOutz1rO90/6M6y3ct8FJlS/kGbK5Wqxi46+yKSz04ud73d2bvJL86nXmQ9H0SllP/QjidKBYBiZzEhQSF2h6F8Ty+ulkObK5Wq5j5d9Skxr8aQmZ9Z6vPbs7ZT4irxcVRK+QdNckpVc+3rt+fuTndT6Cw86TkRYeCkgVw14SobIlPKfnpNTqlqrlNcJzrFdSrz+eG9hxPs0H91dWbSd75SAUBE+PPgn5wdffZxy40xXNPqGpuiUsp+2lypVAAYtWgU8aPi2Ze778iytOw03lz4ZoXH0CkViPRMTqkA0P+c/oQFhx1XmeCHjT/wz+n/5OqWVxNVI8rG6JSyjw4hUCqAbT6wmXNiz7E7DOU9OoSgHNpcqVSAOFR4iFlbZx23TBOcOtNpklMqQLy79F0u++wy0rLT+MvEv/DGb2/YHZJStvP5NTljTFPgU6Ah4ALeE5FRvo5DqUBzY/sb6dyoM5GhkYQHh+sMKEphT8eTEuBxEVlujKkFLDPGzBSRP2yIRamAsSZ9DUkLkkjNSiUhOoHmMc3tDkkp2/m8uVJE0kRkufvnbGAd0NjXcSgVSJI3JZOYnMjWzK0UlhSSlp1GYnIiyZvKn7xZqUBm6zU5Y0w80BFYZGccSlV3SQuSCHWEUuwsJi0njZCgEEIdoSQtSLI7NKVsZds4OWNMTeAb4FEROWm0qjHmXuBegLPOOsvH0SlVvaRmpRIbFktESAQx4TGEBoUS4ghhW9Y2u0NTyla2nMkZY0KwEtx4Efm2tHVE5D0R6SIiXerV0xpYSp1KQnQCecV5GGMICw4DIK84j/joeHsDU8pmPk9yxhgDfAisE5HXfX18pQLR0J5DKXIVkVuUi4iQW5RLkauIoT2H2h2aUray40zuQuBWoK8xZqX7dqUNcSgVMAa0GMDoAaOJqxVHZkEmcbXiGD1gNANaDLA7NKVspdN6KaVU9aXTepVDZzxRSikVsDTJKaWUClia5JRSSgUsTXJKKaUCliY5pZRSAUuTnFJKqYClSU4ppVTA0iSnlFIqYGmSU0opFbA0ySmllApYmuSUUkoFLE1ySimlApYmOaWUUgFLk5xSSqmApUlOKaVUwNIkp5RSKmBpklNKKRWwNMkppZQKWJrklFJKBSxNckoppQKWJjmllFIBS5OcUkqpgKVJTimlVMDSJKeUUipgaZJTSikVsDTJKaWUClia5JRSSgUsTXJKKaUCliY5pZRSAUuTnFJKqYClSU4ppVTA0iSnlFIqYGmSU0opFbA0ySmllApYmuSUUkoFLE1ySimlApYmOaWUUgFLk5xSSqmApUlOKaVUwNIkp5RSKmBpklNKKRWwNMkppZQKWJrklFJKBSxNckoppQKWJjmllFIBS5OcUkqpgKVJTimlVMDSJKeUUipgaZJTSikVsDTJKaWUClia5JRSSgUsTXJKKaUCliY5pZRSAUuTnFJKqYClSU4ppVTA0iSnlFIqYNmS5Iwx/Y0xG4wxm40xT9oRg1JKqcDn8yRnjAkC3gYGAG2Bm4wxbX0dh1JKqcBnx5lcN2CziGwVkSLgC+BaG+JQSikV4IJtOGZjYMcxj3cC3U9cyRhzL3Cv+2GhMWaND2KrirrAfruDKIfGWHX+Hh9ojJ5SHWJcIyLt7Q7Cn9mR5Ewpy+SkBSLvAe8BGGOWikgXbwdWFRqjZ/h7jP4eH2iMnlJdYrQ7Bn9nR3PlTqDpMY+bALttiEMppVSAsyPJLQFaGGMSjDGhwI3A9zbEoZRSKsD5vLlSREqMMYnAdCAI+EhE1paz2Xvej6zKNEbP8PcY/T0+0Bg9RWMMAEbkpMthSimlVEDQGU+UUkoFLE1ySimlApZfJ7nqMP2XMeYjY0y6v47jM8Y0NcbMMcasM8asNcY8YndMJzLGhBljFhtjVrljHGF3TGUxxgQZY1YYY360O5bSGGO2GWNWG2NW+mv3cmNMtDHma2PMevf7sofdMR3LGNPK/fs7fDtkjHnU7riOZYx5zP2/ssYYM9EYE2Z3TP7Kb6/Juaf/2ghcjjXsYAlwk4j8YWtgJzDGXAzkAJ/646BMY0wcECciy40xtYBlwHX+9Hs0xhggUkRyjDEhwC/AIyKy0ObQTmKM+SfQBYgSkavtjudExphtQBcR8dtBzMaYT4D5IvKBu4d1hIhk2RxWqdyfQ7uA7iKy3e54AIwxjbH+R9qKSL4x5ktgmoiMszcy/+TPZ3LVYvovEZkHHLA7jrKISJqILHf/nA2sw5p1xm+IJcf9MMR987tvX8aYJsBVwAd2x1JdGWOigIuBDwFEpMhfE5zbpcAWf0lwxwgGwo0xwUAEOta4TP6c5Eqb/suvPpyrG2NMPNARWGRzKCdxNwOuBNKBmSLidzECbwJPAC6b4zgVAWYYY5a5p8bzN82AfcDH7mbfD4wxkXYHdQo3AhPtDuJYIrILeA34E0gDDorIDHuj8l/+nOQqNP2XqhhjTE3gG+BRETlkdzwnEhGniJyPNQNON2OMXzX9GmOuBtJFZJndsZTjQhHphFXl4x/u5nR/Egx0AsaISEcgF/DX6+2hwDXAV3bHcixjTAxWq1YC0AiINMYMsTcq/+XPSU6n//IQ93Wub4DxIvKt3fGcirvpai7Q395ITnIhcI37mtcXQF9jzOf2hnQyEdntvk8HvsNq9vcnO4Gdx5ypf42V9PzRAGC5iOy1O5ATXAakisg+ESkGvgV62hyT3/LnJKfTf3mAu1PHh8A6EXnd7nhKY4ypZ4yJdv8cjvVPvN7WoE4gIk+JSBMRicd6L84WEb/69myMiXR3LsLdBNgP8KtevyKyB9hhjGnlXnQp4DedoE5wE37WVOn2J3CBMSbC/f99Kda1dlUKO6oQVMhpTv/lc8aYiUBvoK4xZifwnIh8aG9Ux7kQuBVY7b7mBfC0iEyzL6STxAGfuHuyOYAvRcQvu+j7uQbAd9bnHsHABBFJsTekUj0EjHd/ed0K3GFzPCcxxkRg9ey+z+5YTiQii4wxXwPLgRJgBTq9V5n8dgiBUkopVVX+3FyplFJKVYkmOaWUUgFLk5xSSqmApUlOKaVUwNIkp5RSKmBpklNKKRWwNMkppZQKWJrklKoAY0xXY8zv7tp3ke5aXn41v6ZS6mQ6GFypCjLGvACEAeFY8y++bHNISqlyaJJTqoLc01AtAQqAniLitDkkpVQ5tLlSqYqLBWoCtbDO6JRSfk7P5JSqIGPM91hldhKAOBFJtDkkpVQ5/LYKgVL+xBjzd6BERCa4qyUsMMb0FZHZdsemlCqbnskppZQKWHpNTimlVMDSJKeUUipgaZJTSikVsDTJKaWUClia5JRSSgUsTXJKKaUCliY5pZRSAev/AS++h18KUYgCAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "#hide_input\n", "\n", "fig = sns.lmplot(x = 'x', y= 'y', data = df, hue='Values', palette=[\"b\", \"g\", \"black\", \"orange\"], fit_reg=False )\n", "fig.axes[0,0].plot(x, m1, linestyle='dotted', color = 'b')\n", "fig.axes[0,0].plot(x, m2, linestyle='dotted', color = 'g')\n", "fig.axes[0,0].plot(x, ens, color = 'black')\n", "fig.axes[0,0].plot(x, y_true, linewidth = 2, color = 'orange')\n", "\n", "plt.xlim([0, 8])\n", "plt.ylim([0, 10])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The ensemble line is closer to the true values. It also looks smoother then m1 and m2." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "#hide\n", "\n", "ens_res = y_true - ens\n", "\n", "df_res = pd.DataFrame(np.array([x, m1_res, m2_res, ens_res]).T, columns = ['x', 'm1_res', 'm2_res', 'ens_res']).melt('x')\n", "df_res.columns = ['x', 'Residuals', 'y']" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbcAAAFuCAYAAAACplYMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABFUklEQVR4nO3dd3gU5doG8PtJWUIKCaGGABKQQxeQosI5CEEQUMFj74ootoh+CqgoluMRC8eCoIiKoojSRFQgioIFRJRIb9J7qEkgpG2Sfb4/ZhMCpGyyZXaH+3ddc5HdnZ15Msa9d2beIqoKIiIiKwkyuwAiIiJPY7gREZHlMNyIiMhyGG5ERGQ5DDciIrKcELMLqIx+/frpd999Z3YZRES+IGYXEMgC6szt6NGjZpdAREQBIKDCjYiIyBUMNyIishyGGxERWQ7DjYiILIfhRkRElsNwIyIiy2G4ERGR5TDciIjIchhuRERkOQw3IiKyHIYbERFZDsONiIgsh+FGRESWw3AjIiLLYbgREZHlMNyIiMhyGG5ERGQ5DDciIrIchhsREVkOw42IiCyH4UZERJbDcCMiIsthuBERkeUw3IiIyHIYbkREZDkMNyIishyGGxERWY7p4SYiwSKySkTmmV0LERFZg+nhBuARAJvMLoKIiKzD1HATkYYArgDwoZl1EBGRtZh95vYWgJEAHCbXQUREFmJauInIlQAOq+pfFaw3VERSRCTlyJEjPqqOiIgCmZlnbt0BDBSRXQCmA0gUkc/OXElV31fVzqrauU6dOr6ukYiIApBp4aaqT6lqQ1VtAuAmAItV9Taz6iEiIusw+54bERGRx4WYXQAAqOrPAH42uQwiIrIInrkREZHlMNyIiMhyGG5ERGQ5DDciIrIchhsREVkOw42IiCyH4UZERJbDcCMiIsthuBERkeUw3IiIyHIYbkREZDkMNyIishyGGxERWQ7DjYiILIfhRkRElsNwIyIiy2G4ERGR5TDciIjIchhuRERkOQw3IiKyHIYbERFZDsONiIgsh+FGRESWw3AjIiLLYbgRBajkrclI/CQRCeMSkPhJIpK3JptdEpHfYLgRBaDkrclISk5CamYqYsNikZqZiqTkJAYckRPDjSgAjV02FiESgrCQMIgIImwRsAXZMHbZWLNLI/ILDDeiALQzYyfyHflYc2gNcvJzAADhoeHYlbHL3MKI/ATDjSgAJcQkAArUCq+F7PxsFDgKkJ2fjSYxTcwujcgvMNyIAtCIbiMgQYKI0AjsOr4Lh7MOw+6wY0S3EWaXRuQXGG5EAahVnVYY2W0kEmISEBcZhxa1WmBC/wno37y/2aUR+YUQswsgosobt3wcJqZMxLGRxxBhizC7HCK/wzM3ogA0vNtwzL5hNiJsETieexzP//w8ftvzm9llEfkNnrkRBaD4GvGIrxEPAKgWUg1vLn8T4aHh6N64u8mVEfkHhhtRgJm3ZR5UFVe1uAoAEBYShv2P7UekLdLkyoj8B8ONKMC88fsbyM7PLg43AAw2ojMw3IgCzHe3fYeDJw+e9pxDHbhz7p1oW6ctnvjnEyZVRuQ/2KCEKMDYgm1oHN34tOeCJAh5BXmwF9pNqorIv/DMjSiAjPxhJFrVboXBHQef9drM62eaUBGRf2K4EQUIhzqwZM8SONRR7np5BXmoFlLNR1UR+SeGG1GACJIg/D7kdxQ6Cstc59Y5t+JA5gH8dOdPPqyMyP8w3IgCTHBQcJmvJTZJRHpuug+rIfJPbFBCFAAKHAVo824bfLTqo3LXG3LhEAzvNtxHVRH5L4YbUQDIyM1Au7rtUC+iXoXrFjgKsOHwBh9UReS/GG5EAaB2eG1Mv246rvjHFRWu+/j3j+OiDy9CbkGuDyoj8k+850bk51QV6bnpiK0e69L6gzsOxqVNLkWQ8Lsrnbv410/k5zYc2YA6Y+tg7ua5Lq3foX4HXNPqGtiCbd4tjMiPMdyI/FxMWAye/tfTuCj+Ipffc+jkIUxeObnCPnFEVsVwI/JzDWs0xH96/QdxUXEuv2fh9oW459t7sPrgau8VRuTHGG5EfuxY9jH8se+PSp+BDWo5COsfWI+O9Tt6qTIi/8ZwI/JjszbOwsWTL8bmo5sr9b4a1WqgTd02EBEvVUbk3xhuRH7sxjY3Ytb1s9CqdqtKv3fP8T147PvHsCtjl+cLI/JzDDciP1azek1c1/q6Kp2B5Rfm450V72Bl6kovVEbk39jPjchP/b73d2xP344b2txQpWb9zWKbIW1kGiJsEV6ojsi/8cyNyE9NXTsVj3z3iFudsRlsdK5iuBH5qQkDJiDl3hSEBFX9AktaThqu+uIqzFg/w4OVEfk/08JNRBqJyE8isklENojII2bVQuSPgiQICTUT3NpGTFgM0nPSkVOQ46GqiAKDmffcCgA8rqorRSQKwF8i8oOqbjSxJiK/8ObvbyI7PxtP93jare0ESRCW3r3UQ1URBQ7Twk1VUwGkOn/OFJFNAOIBMNzonLfq4Coczzvuse2pKvIK8xAWEuaxbRL5M1FVs2uAiDQB8CuAtqp64ozXhgIYCgCNGzfutHv3bt8XSGSCQkdhubNuu0pV0fmDzujSoAveu/I9D1RGPsIe+G4wvUGJiEQC+BLAo2cGGwCo6vuq2llVO9epU8f3BRL5WNEXTk8EGwCICK5tdS3+1fhfHtkeUSAwtZ+biITCCLZpqjrHzFqI/EXfz/qiW8NueKHXCx7b5qh/jfLYtogCgZmtJQXAZACbVPUNs+og8ieFjkIkxCSgXmQ9j287Oz8bfx/92+PbJfJHpt1zE5F/AlgCYB2AoiHPR6nqgrLe07lzZ01JSfFFeUSWc/lnl+NA5gGse2Cd2aWQa3jPzQ1mtpZcCv7HIzrNsexjqBVeyyvbfrL7kyjUQqgqZwsgyzO9QQkRGTLzMhH/Rjz+t+x/Xtl+r4ReuKzpZQw2Oicw3Ij8hEMdeLHXi+id0Ntr+9ieth2fr/vca9sn8hcMNyI/ER0WjRHdR6BjnPdmz/5o1Ue446s7cDzXcx3EifwRw43IDxQ4CrBoxyLkF+Z7dT9JXZOw45EdiA6L9up+iMzGcCPyA0v3LMVlUy/Dt1u+9ep+4qLi0Di6sVf3QeQPGG5EfqBrfFfMvXEu+jbr6/V9rTm4Bg/MewC5Bble3xeRWRhuRH4gPDQcg1oOQqQt0uv7OpB5ANPWTcPmo5u9vi8iszDciEy2LW0b3l3xrs8aefRp1gdHRhxBh/odfLI/f5ScDCQmAgkJxr/JyWZXRJ7GcCMyWfLWZDy04CFk5Wf5ZH8hQSGoFlLNJ/vyR8nJQFISkJoKxMYa/yYlMeCshuFGZLKkrknYMWwHGkQ18Nk+Nx3ZhJ5TeiLlwLk3nN1rrwE2GxARAYgY/9pswNixZldGnsRwIzKZiCChZoJP91kvsh6O5x1HRm6GT/drtrFjgeXLgfDw058PDwd27TKlJPIShhuRiWZumIlHv3vU5y0XY6vHYtV9q3BZ08t8ul9fy80Fpk8HCgqMx9HRxnLy5OnrZWcDTZr4vDzyIoYbkYk2HdmEH3f8iGrB5twDc6gDeQV5puzbm4omO1m4ELj5ZuD7743HQ4cCH39shF1WlrFeVhZgtwMjRphXL3kew43IRM/1fA5r7l9jymDG6TnpaPRmI0xMmejzfXtLVhbQpw8wYYLxeMAAYNEioH//U+v072+8HhcHpKcb/06YcPo6FPhMnYmb6FxWNPVMcFCwKfuvWb0mbmh9A9rUaWPK/j1lzx5g40agXz+jcUh0NFC9uvFaSIjR1P9M/fszzKzOtMlKq4KTlZKV3PvNvcgtzMXUf081u5SAo2q0dASAG24Afv4ZOHDACDML4dxEbuBlSSKTNIpuhEY1GpldBtJz0rEzfafZZbjsxx+BFi2AQ4eMxy+9BKxYYblgIzfxz4HIJM9e+qzZJUBV0WFSB3SN74pZ188yu5xSqQKLFwONGwPNmwONGhk/p6UB9eoZzxGdieFGZIKj2UdRq3ot02fFFhGM6zcODWs0NLWO0hRdeszIAK68ErjvPuCtt4yzth9/NLs68ncMNyIfU1V0/aArEhMS8eHAD80uB1e3vNrsEs4yfLjRqXr2bKBmTaPF44UXml0VBRLecyPysUItxMjuI3Fd6+vMLqXYqtRV+PZv784lV57jx4GpU0/1T6tbF2jQAHA4jMfdugFhYaaVRwGIrSWJCNfNvA5/7v8Tux/d7dNLpUWXHj/4wOhgnZICdOrks937O7aWdAPP3Ih8bNGORciy+2YGAFe9etmrWHnfSp8FW2oq0KULMHOm8fjmm40Wjww28hSGmwckb01G4ieJSBiXgMRPEpG81X/mzuC8VVXnjWN3IPMALpt6Gcb9Mc4D9SUjMTERCQkJSExMRLIbBTaLbYba4bXdrulUbWcfuw0bTjUEqVfPWKo5Rx2LjAQ6d/bY7ol4WdJdyVuTkZScBFuQDeGh4cjOz4bdYceE/hPQv7m5QyAUzVtlsxmjnmdnG2Pocaihinnr2OUX5mPpnqU4P/Z8NIqueh+35ORkJCUlwWazITw8HNnZ2bDb7ZgwYQL6V7HAxTsXY8HWBfhf3/9VuS6jtlPHrnp1ICfHOHbh4cY9tPXrT3XApnLxKLmBZ25uGrtsLGxBNoQGhyLfkY8IWwRsQTaMXWb+5FBjxxofMFlZxugNnLfKdUXHzm4H9u0zPpg9cexCg0PRK6GXW8Fm1DcWIoKDBw8iLS0NERERsNlsGOtGgWsOrsHUtVORlpPmZm3GsSooMIbFCgs79SXhl18YbOQbDDc37czYibCQMGw+uhl7ju8BAISHhmNXxi5zCwOwc6fxgXLkiNFXSJXzVrmq6NidOAEcPeqZY5eZl4lXl76K/Sf2u13fpk2bsHv3bmRkZGDXrl3Yu3cvqlevjl1uFPhAlwdw4LEDiK0e61ZtRccuKMi47FhYeOrvsLbnrnwSlYvh5qaEmATkFuSicXTj4o6w2fnZaBLTxNzCYNzvyM4G2rQBWrc2vjFz3irXFB27884DOnQwPqjdPXa/7/sdTy56EtvStrlV2/vvv49Dhw4hKCgIrVu3Rt26dXH48GH8/fffiI+Pr/J2w0LCPDKIc9Gxi4oyRg+x2fh3R77HcHPTiG4jYHfYERoUimrB1ZBlz0JeYR5GdDN/cqjhw437HVnOhnmHDhnfnjlvVcUuv9zoe1V07NLT3T92fZv1xZ5H96B74+5Ver/dbseDDz6I++67D506dUJcXBwcDgcaNmyI+vXrIzs7G9u3b8eGDRuqXOPinYvR5YMubl2a/Pe/gcOHjQlBOV8amYXh5qb+zftjQv8JiIuKQ1pOGk7knUD7eu1Nb0wCGJeA9u0zvjmnpxsfMNHRxnxXVL5Fi4yBeOvXN45dfr7xRcHdFn2NohshJKjyAwMdPnwYffr0wcSJE/HEE09g+fLlmDhxIuLi4pCeno5WrVrh9ddfh6ri4osvxtdff12l+qJsUbAF23Dw5MEqvR8w/uZUgTp1OF8amUhVA2bp1KmT+ruHFzysT/7wpDocDrNL0d27VYcPVz1+3Hicnq6am2tqSQEjO1t169ZTjzMzVfftq/r2ft/7u9499249cOJApd+7cuVKbdy4sYaFhennn39e7rr79u3TLl26KAB94YUXtLCwsKolV5nD4d6xomKmf+YG8mJ6AZVZAiHc/CHUKmK3q27YYHYV/unwYdX8/PLX2bOn8tv9dPWnWndsXc3My6zU+6ZPn67Vq1fXhg0bakpKikvvycnJ0dtvv10B6DXXXKOZmZXbp6qqvcCu+YUVHIgzHD5sLOQxpn/mBvJiegGVWQIh3IqsTl2tX236yrT9//ij6urVpb92222q9eurnjzp25r8ncOh2rOnamKi8XNpxo5VjYioWsAVFBa4vm5BgT711FMKQP/5z3/qwYMHK7Uvh8Ohb7zxhgYFBWm7du10+/btLr93depqjX45WhdsWVCpfd5+u2pcnHHWSx5h+mduIC+mF1CZJZDCre/UvvqP8f+o1AeaJ7Vvr3rJJaW/tnKl6tdf+7ScgOBwqM6erVrelb+dO1VffLFyl3crezafkZGhV1xxhQLQoUOHal5eXqXeX9LChQu1Zs2aGhsbq4sWLXLpPTn5OXrvN/fqygMrK7WvDRtUP/mkKlVSGUz/zA3khSOUeMme43sQaYt0u89QVaWlAQcPGl0AyqPKTrXe9srSV5C8LRk/3P4DbMG2ctfdsmULBg4ciO3bt2P8+PG4//773d7/tm3bMGjQIPz999948803kZSUZPo8cuQS/kdyA1tLeknj6MaIrR4LVcXhrMM+339sbMXB9tlnQN++Rifbc91TTwFTpri+/ooVxgSaWS6Mf1wnvA6axDSpMNiSk5PRtWtXHDt2DIsWLfJIsAHA+eefj+XLl+PKK6/EsGHDcM899yAvL6/C9+0/sd+lVpNjxgDDhvHviPwLw83Lhn47FD0+7oG8goo/TDzhwAHgqquM8fsqEhJinLllZHi9LL+Wnw/8/juwdq3r78nJAdatM0bjqMiQC4fgk6s/KfN1VcVrr72GK664AgkJCUhJSUGPHj1cL8YFUVFRmDNnDkaPHo2PPvoIPXv2RGpqapnrH889jibjmuCdP9+pcNtpacYoLsHu9/8m8hyzr4tWZgmke25FFm5bqO/8+Y7P7r399JNqgwanN2Mvi8NRdsOJc43DUfluEq7cCjuadVQLHWU3x8/OztZbbrlFAegNN9ygJ33QymfWrFkaHh6uDRo00D/++KPM9T5d/aluObrFpW2a0OPgXGD6Z24gL6YXUJklEMPNDAWVzNG0NNXJk71Ti7/76adT/QCrorDQaIRS1peE/p/1126Tu5X62u7du/XCCy9UEdExY8b4tBvJmjVrtEmTJlqtWjX9pIqtQBYtUt20ycOFUUmmf+YG8mJ6AZVZAjnckrcm69Bvhnr1Aywjo2pnYi+/rBocrLrFtS/plnHihGqNGqp33FH1bcyebfxfNG9e6a/PWD9DP1396VnPL1myROvWratRUVH67bffVr0ANxw5ckR79eqlAPSxxx7T/DM6+DkcDl28Y7H+tPOns97rcKi2aaN68cU8+/ci0z9zA3kxvYDKLIEcbm/+/qa2fqe1Hs066rV99O2rOmBA5d+Xk1N2nzir++MP1R07qv7+wkLVb7+t3Af8pEmTNDQ0VJs3b64bN26s+s49wG6368MPP6wAtE+fPnrs2LHTXm/7blu97NPLSn3voUOqmzf7ospzlumfuYG8sCuAjxQ4CuBQR4Ut5qpKFfjoI6NZ/913V307aWlGS0urs9uNMTc9KTfXmLusyNI9S9GqdivUCq/l3Kcdjz76KCZOnIh+/frhiy++QExMjGeLqKLJkyfjgQceQOPGjfH111+jTZs2AIBNRzahUXQjRNoii9c9ccIY8Z+9CbyOR9gdZqdrZZZAPnMrkleQpz9s/8HsMkr1xReqkZHWvzyZlaXaooXqO+94bpspKcaoL8uWGY/zC/M19tVYvX3O7aqqeujQIe3Ro4cC0JEjR2pBZW+M+sCyZcu0fv36GhkZqXPnzi11nYIC1S5dVIcM8XFx5ybTP3MDeWFXAB8bs2QMLv/scuxI3+GxbebmAjNmGGcj7rj0UuD224GaNT1Tl7/KywO6dKm4H2BltGgBXHyxMds5AARLML6/7XuM7D4Sq1atQpcuXfDnn39i2rRpePXVVxHsh+3mL7nkEqSkpKBVq1a4+uqr8eKLL8LhcGDG+hl4ZekrAIwrBDfeaEwJROTPeFnSx9Jz0vH7vt8xoPkAj21zxgzgppuMaVoSEz22WfKAGTNmYPDgwahVqxbmzp2LTp06mV1ShXJzczF06FBMnToV1157LaJvjMZfx/7CqvtWcWQT3+LBdgPDzUT5hfkIDQ51ezsOB/Dzz0CvXp65D7JzJ/Dss8YcXNHR7m/PX6gCzz0HDBlizLDtDVlZwEtjFI6LXkXa4h34YNwH6N69O7788kvUq1fPOzv1AlXFW2+9heHDh6N169b4+uuvMXlyUyQmAr17m13dOYPh5g6zr4tWZrHCPbciczfN1fPePE8PZlZutHdfSElRjYkx+oBZycaNquHhqpMmeW8fO3eqVqu3WtEcCkDvvfdetwY+NtupgZdraYMGi3T0aLMrOqe49XkJoBDAagDrAXwLIKYK2+gM4O0yXtsFoHYVa3sewHB3f8fyFt5zM0nL2i3Rtm5b5BW6NyzXiy8C48d7qCinTp2AvXuBnj09u12ztWoFbNkC3HOP9/Zht29Bw8gbEbIzBG+8/QYmTZoEm6ebZfpQnz598Oeff6J6TBgOHOyN6Ji3iz6cAlpycjISExORkJCAxMREJCcnm12SN+SoagdVbQsgDcBDld2Aqqao6jDPl+Z9DDeTtKjdAvNumYfG0Y2rvA1VYPlyYNUqDxbmFOls+f3rr9YYEHfLFuPf+HggyEt/9UUDHx8/fgw//vgjBl3xf8jJCfwrS8eOnY/np76Iuh3qYvjjj7g88LK/Sk5ORlJSElJTUxEbG4vU1FQkJSVZNeCK/A4gHgBEpJmIfCcif4nIEhFp6Xz+ehFZLyJrRORX53M9RWSe8+daIrJQRFaJyCQ4L5uKSBMRKR7NVkSGi8jzzp/vFZEVzm1+KSLhZxYmIsNEZKOIrBWR6Z76hRluJkvLScMzi5+p0sDKIsD8+cCkSV4oDMDSpUYLyqlTvbN9X1mxAmjZEpg2zTvbVz018HF843h0faErCmvVQtu2wMsve2efvrJ/v/E3sPXrwUhdkerywMv+KisrC08//TROnjyJw4cP49ixY4iIiIDNZsPYsWPNLs8rRCQYQG8A3zifeh/Aw6raCcBwAO86n38WwOWq2h7AwFI29RyApara0bktV76Zz1HVLs5tbgIwpJR1ngTQUVUvAOCZqTAAhHhqQ1Q1K/avwCtLX0HPJj1xWdPLXH6fqjEyfXg4EOp+m5RSde8OfPopcMMN3tm+r7RtC7z0EjCwtP9d3ZSTk4N77rkHn3/+OW644QYMfm4w7lpwF+rVFbzxhjFDQyBr0MD4UnDJJUBQUBCeGv0ULrjgAtx5553o3LkzvvrqK3Tt2tXsMkuVk5ODNWvWICUlpXjZtGkTHA4HACA0NBTVq1cHAISHh2PXrl0mVusV1UVkNYAmAP4C8IOIRALoBmBWiZav1Zz//gZgiojMBDCnlO31AHANAKjqfBFJd6GGtiLyXwAxACIBfF/KOmsBTBORuQDmurBN13jzhp6nFys1KClpV/quSr9nyRLV6OhTnYa9zQ/7HJtuz549pQ58XOgo9OkgyN5y5kj/czbO0bD/humOtB0eGXjZk3Jzc3XFihU6ceJEHTJkiLZv316Dg4MVMBr21KlTRwcMGKDPPvustmvXTps1a6adOnUqXlq2bKm9evUy+9c4k7sNSk46/40GsATAMAA1AKSW856LAPwHwF4AtQD0BDDP+dpqAAkl1k0DUBtAQwAbSzz/DIDnnT/vBNDe+fNdAKboGQ1KAAQD6AXgTQB/Awhx93dX5diSfmXbsW0ufyhu2KA6eLCqD2ZI0e3bVdu2VV240Pv78qS9e1V79FBdt87z2y5t4OPS/ttlZqreeKPqlCmer8Gbjh9XbddOteRAJTvSduiwBcOKv4xVNPCyt9jtdl21apV+8MEHet9992mnTp00NDS0OMhiY2O1b9++OmrUKJ0zZ47u2bPntP82CxYs0KZNm2rLli31wgsv1JYtW2rTpk11wYIFPqm/EjwSbs6fOwLYAyAUwDIA1zuflxLh06zE+qsAdDgj3N4G8Izz5/7O413buc2jzjCsBmB5iXA7CqCuc50fzgw3GLfGmjifCwVwCFVo1Vnq7++JjfhqsXK4/bLrFw1+IVhnb5htdilnyclR7d078MJt6VLVZs2McPak999/v9SBj2dvmK3tJ7bX3Rm7i58rLFRNTFR96y3P1uBtu3er9uyp+uef5a9X0cDL7iooKNB169bpxx9/rA899JBedNFFWq1ateIgi46O1sTERB05cqTOnDlTd+zY4dIXxAULFmivXr00ISFBe/Xq5Y/BpurBcHM+/hbA7QASAHwHYA2AjQCedb4+B8A6GF0HxjmDr2S41QKwEMBK51nWbji7AjjPCrcVBViJcHvAefb2M4DxpYRbKIClJfb7pLu/d/Hv66kN+WKxcrjlF+brCz+/oMeyK/5wWLLEOCuhinnycqrdbtcHH3xQAejll1+uaWlpp72+YMsCHTBtgOYXnn4GY7WJPDcf2azHc0+fBO/DDz/U0NBQbdasma5fv75K2y0sLNRNmzbp1KlT9ZFHHtHu3btreHh4cZBFRkZqjx499LHHHtPPP/9ct2zZooVWO7inM/0zN5AXc3cO9HNeY93mSmJbOdxc5XAYZyOu3h4o+obapEkTt7+hFhaqTpzo2YGVF2xZoL2m9NImbzXRXlN66YIt7n+DzspSnTbN/VApeey6d++ubdu2VQA6YsSIKg18vHSp6ooV7tXkbQUFqmPHlj2B68oDKxXPQ6eumXrWa7/99pvWq1dPIyMjdfTo0eX+3TkcDt26dat+8cUX+vjjj+ull16qUVFRxUFWvXp17datmw4bNkw//fRT3bhxo18ONu1lpgdEIC9mBlswgO0AmgKwOU+RW5f3nnMh3PYe36v9PuunKw+sLHOdHTtUV62qeFuevrdw8KAxuedTT1Xp7WfXt2WBNh3XVFuOb6kXvnehthzfUpuOa+p2wE2caPxlV3RJrdzaShy7li1bakhIiIqIjhgxotT103PSNa+g7JFI8vJUGzdWveKKqtfkC7/8oiqiOn166a8XOgp1UsokPXDiQKmv7927V5s3b64AtHbt2tqxY0dt0aKFNmrUSEeNGqVPPPGE9u7dW2NiYoqDzGazadeuXfXBBx/Ujz76SNeuXeuz+3d+zvSACOSlwrElRSQJwDRVdaXZp8tE5BIY12Uvdz5+CgBUtcyeQX49tuTngd9Zl+icdIvfjrjCDxU3uNKJuz6AFSIyU0T6ieeGBY+H0dy0yD7nc6cRkaEikiIiKUeOHPHQromIyMoq7MStqs+IyGgAfQEMBjDB2clvsqpud2PfpYXkWV+hVPV9GD3q0blzZ7/9iuWNb38Lti5Ap7hOqBdpjCb/9dfAHXcA69YBjUsZG+Do0aOYMmUKJk2ahG3btiE0NBRRUVGIi4tDUFAQjh49iiNHjqCgoADx8fG49957cc899yA+/qzvFOXKzDRG/Lj2WuDtt6v2u+1I34Hm45sjNiwW8TXisS1tG2qH10b1kOqIi4rD4jsXV3qbn30GPPEEsGQJ0LRp5d574sQJTJs2De+99x7Wrl2L4OBgREVFoUGDBsUdfbOyshAXF4fFiytfWxFV4LLLjNkW5pTWTdYkixYBAwYACxZUPOp/bkEuar9WG4M7DMb4AWcPbJqYmIjU1FREFE1uB88cO6LKcGn4LTWuXR50LgUAagKYLSKvubHvfQAalXjcEMABN7ZnKftP7MfV06/GG7+/UfzcoEHAoUOnB5uqYunSpbjtttsQHx+PESNGIC4uDtOmTcOsWbMQExODwsJChIaGIjo6Go0aNcLo0aPRrl07vPDCCzjvvPPw73//G99//33xyA0ViYoCRo4EBg+u+u/XtGZTDL1wKCJsEcgryMM/Yv+B6iHVYXfYcXXLq3E893ilt9miBdC/P9CkievvWbVqFe677z40aNAADz74IEJCQvDBBx9g5syZiI2NhcPhgKoiKysLdrsdI0aMOGsbfx34C3uO73FpfyLGl4J//9sIOn/Rti2QlAR061bxumEhYVg2ZBlev/z1Ul8fMWIE7HY7srKyKjx2RF5T0U05GP0X/oIxbMr1AEKdzwcB2F7Vm30wzhp3wOhzUdSgpE157zkXGpSU9MuuX4obKeTmnv5aRkaGjh8/Xtu0aaMAtEaNGvrwww+f1Qy7vP4827dv1yeffFLr1KmjADQhIUFfeeUVPXTokFd+n/ScdL1tzm2njchS1Foy4a0E7TWll87ZOEfrja2n18y4xis1qKpmZWXp5MmTtWvXrsUt84YMGaIrzmjK6GpfqK4fdNUu73fxWr2BKED6kfk70xtlVGYB0BLGAM158PJ0Ni7V40LB/wFwXhmvtXLzYAwAsAVGq8mnK1r/XAu3InkFeXppYp7efbfqihUrdMiQIcX9f7p06aKTJ0/Wk24MVZKXl6fTp0/Xnj17KgANDQ3Vm266SX/++edyO8Ta7apPPKH63nuu7WfL0S1a57U6OmvDrHLX+2nnT7rt2DaX61++XHX0aKOzeXnWr1+vDz/8sEZHRysAbd26tY4fP17T09Nd3ldpNh/ZrMv2VH4ctC++MJrdm2nrVtXrrlPdt6/y733z9zd18srJni+KilTxc1X7AboI0B3Of/tVdVuV2y/qAugC4KXKhBuMW1RBHq/HF7+0p5ZzMdwycjK05esttdWgq/W88zopAA0PD9d7771XU1JSPL6/TZs26aOPPlrcVLtly5b61ltvndVhWdXoc9enj+ojj1SwzSObin8+kXuiUvU8/v3jOva3seWG7PPPq8bHq54oZdO5ubk6bdo0/de//lXc7PzWW2/VJUuWmD7+4513qnbrZu64nV9+qRoXp7p/f+Xfe+nHl+oNs27wfFFUpKrBth3QjYCmOP/d7m7AwRh8eTOAD2GMJDINwGUwBlveCqBriXWfryjcnNvbBGNGglUAzgMwAsAKGAMpv+BcLwLAfOeVvfUAbnS5Znd+YV8v51q4rV27Vh966CG1hdsUgLZt21bfeecdzcjI8Pq+s7OzdcqUKXrxxRcrAA0LC9O77rpLly9fflooVDTJ9Pfbvld5XnTuprmVrqGgsECvn3m9PpL8SIXrnjnq07Zt23TkyJFau3ZtBaDNmjXT1157TQ8fPlzpOsozccXEKp21qRrjTvpDv+SKznjLkm3P9mwhdKaqhNuiEsGWUiLgFlVle6e2iyYw2lu0c96S+gvAR86zrkEA5pZY19VwcwC42Pm4L4yGg+Lc/jwYsxBcC+CDEu+Ldrlmd35hXy/nQrjl5OTo1KlTtXv37gpAq1Wrpn363K5Ll/5m2pnGqlWr9P7779fIyEgFoB06dND33ntPT5Q4Vdq+vfRO0/YCu77060uaZc+q0r4dDocWFBoJsCNth+49fmrcsQMHjDEQi/dlt+ucOXO0b9++CkCDg4P1mmuu0YULF3plmKa8gjyNeSVG/++7/3NrO1lZql44CS/X8eOqixf7dp9UaVUJtx1nBFvRsqMq2zu1XTQBsLXE408B3Or8uSmA1SVeczXcdpZ4/D8Au2DMPLAaxqhVQwD8A8bYlK8C+FelanbnF/b1YuVw+/vvv/Xxxx/X2NhYBaDNmzfX119/XceNO6qA6tJlBTr+j/FVOgPylBMnTuh7772n7du3Lx7r7/7779dVq1brBReoXnihcalyd8ZuvW3ObZqZl+mxfTscDu02uZu2fqd1cdjdeqtq7dqqmzfv0dGjR2uDBg0UgDZs2FD/85//6P6qXGurpJN5J/VI1hG3tnHDDap16hgh5yvPP68aFOT+oNIv/fqS3jz7Zs8URWeqSrh588xtfYnHUwBcV8ZrroZbyfe8DuC+MtaNBXAbjAGWn3W5Znd+YV8vVgs3u92us2bN0t69eysADQkJ0euvv14XLVpUfJaWna06e7aqvSBfO7zXQe/46g6TqzaCZvny5XrXXXdpWFiYAtB27S7WN9+cotnZ2bpw20Kt+UpN/XOfG+NflWLD4Q36y65fVNUYLf7jj+dr585XaVBQkIqIDhgwQL/55puAG7pp3Tpj2Ctfys5W/eYb97cz5tcxevPsm7XQYekBjM1SlXDz5j03b4ZbXwB/AIh0Po6H0UClAYAw53NXl7z8WWHN7vzCvl6sEm67du3Sp59+WuvXr68A9LzzztOXXnpJU1NTy33f0ayjpjeCOFNaWpq+9dZb2rJlSwWgMTExmpT0qK5Y450RglNTU/W//31Ja8XVUgBar149HTVqlO7cudMr+ytLfmG+XjfzOl20Y5FHt+vtQe4LClQDLPvPZVUMIs+3lnQl3GCMZrUPwAkAGc6fa7iyPedzj8CY+mYdjC4FzQBcDqOByWoYjU06u1yzu7+0L5dADreCggL99ttv9YorrlAR0aCgIL3qqqt0/vz5ZY52/vLLpQ9geyz7WKWayvvCuN/Haejdodro/P4qYkwc2bNnT50+fbrmVdTqpAIOh0MXLVqk119/vYaEhCgAjWh4nrZ9qJ1m5fjwWl4JO9J2aLNxzTw6/97EiaoXX+zd8Hn3XdUOHVSPuHcl9SxsXOIVpn/mBvJS4fBb5J7U1FRMnjwZ77//Pvbu3Yu4uDg888wzuOeee9C4tDG0nBwOYOZMoGtX4MYbTz2vqug5pSeiqkVh6eCl8NxQn+655YJbkDM4B9Fdh2PnjmOIifkYH3wwCTfddBPq1KmDIUOG4N5770XTSoyLdezYMXzyySeYNGkStmzZgpo1a2LYsGHIzx+KnJwWeHdcPkKDQ5GZl4ldGbvQrl47L/6Gp0uomYCtD2+Fnj1iXJXVqwfExwMnTwIxMR7b7Gni443RSGrV8tw2/7fsf3hpyUs4+PhBVAup5rkNE7nD7HStzOKvZ25nzpk2b948/eGHH/Taa6/V4OBghXOW4i+//FLtdrvL23U4VEvrm/3d1u90depqD/4GVfPXgb80aX5SmfdbCgsL9bvvvtOrr766+L5Yv379dO7cucX3xc48dvPnz9fffvtNb7/99uIZl7t166affvqpZmefOjsoeXV26DdDtcbLNVya6NUTHA6H310eNtOS3Uv06UVPa0ZOhtmlWI3pn7nuLjBm715dylLL6/s2+5evzOKP4VZy3q8LLrhA69SpU3zprFatWjpixAjdunVrpbbpcLje/8nMD9k3lr2hjd5opPuOnz28xdKlqq+/furx3r179fnnn9f4+HgFoPHx8Xrrrbdq48aNtWXLltq+fXutV6+e2mxGn76oqCh98MEHdc2aNcXbmD9f9Y8/zq4jNTO1wlFPPGndoXWa8FZClfu3VeTAAdU33/TsNn/+WfXDD603K7jFmf6ZG8iL6QVUZvHHcCsaPy82NlZFpHiswlatWmlOFXvH/vijaqNGqmvXlr/eqB9H6d1z767SPqqqoLBAd6bvVFUjWNNz0ktd76GHVJs0OfvMMz8/X+fOnav9+vUrnqwyMjJSg4KCivv1tWjRQjMzT+9G4HCoduyoesklp5+1nem3Pb/ptTOu1eO5ZUwl7QErD6zUAdMG6P4T3ulq8OqrqtWqqW7z4G3Vu+4yZnDP9tKtsUJHoabsT2GrSc8y/TM3kBeXZgWgsu3cuRPVqlVDRkYGateujdatW6NVq1bIzc1FWFhYlbYZEQF06QI0b17+esFBwQgOCkaho7BK+6mKpAVJ6Da5GzJyMyAiiAmLKXW9l182puYpMesJACAkJASDBg1CcnIy4uPjUa9ePRQUFKBmzZpo2bIlWrduDbvdjsjIyNPeJwL8/DMwbZrxc1n+Pvo3NhzZ4NVj0jGuI+bfMh8Nohp4ZfuPPgps2AA0a+a5bU6eDPzyC+CcvcfjZqyfgc4fdMZfB/7yzg6IKqnCmbj9iT/OxF00d1X16tURFGR8V/DV3FWq6vMGJesOrcOyvctwX+f7XFrf4QA2bwZatz77NVfn/Tp8GKhTp/xQK8leaIct2AaHOrAydSU6N+js2htdkGXPgkIRaYuseGUP2L/faARSVQcOAJGRQI0anqupNGk5aZi3ZR4GtRiE6LBo7+7s3OEfrcUCFM/c3FQ0d1VOTg5U3Z+7KiUFOHHCtXWLgm3rsa2YsX5GlfbnigVbF+C134yp+9rVa+dysAHA8OHAJZcApU2i7sq8XwUFwOWXA7fc4nq9tmAbAOCdP9/BRR9ehFWpq1x/cwU+W/sZar9W2+X529zxySfGpKubNlV9G/fcA1x0EVDo5ZP72OqxuKP9HQw28h9mXxetzOKP99xUPTd3VUGBMbr9wIGVe9/Ns2/WemPrea2v0eC5g/XCSRdqbn5uxSufYcsW1alTy75PVtGxKyxUnTxZ9euvK1/3ybyTOillkkcb3axKXaXP/fScTxryHD6s+tRTxgDLVbV8eel9Jb0hMy9Tp62dVmoDI6oS0z9zK7MAuBVGh+u1AJYBaG9mPbws6WdWrDAuv3WuxJW0gycPwqEOj94Dys7PRmZeJupF1kNuQS4c6kB4aLhb21R1/dKip+09vhcPzH8A7135HhrWaGhOERa3PW07zh9/Pib0n4CHuj5kdjlWUKX/W+QF6Qdj+pgEGIMOj9Xn9DtPFlbqfkW6Adikquki0h/A86p6kQvvExi3yByerIeXJf1Mly6VCzYAqB9ZvzjYjmSVcv2vklQV/af1x9UzroZDHQgLCXM72ObNA3r2BHJzXX/PsGHA3Llu7bbYtrRtWH1wNU7kuXjNtxSbj27GxiMb4esvhBs3AgMHAmlprr9n5Ejg+eeNLxS+0iy2GVYOXYkHujzgu53SaZzB9g6AOABpzn/fcT5f9e2KNBGRzSLyoYisF5FpInKZiPwmIltFpKuqLlPVdOdblgMo81ukc3ubRORdACsBNBKRESKyQkTWisgLzvUiRGS+iKxx7vfGsrZ5Joabn9i+HXjySaPxRFU9vehpdJzUEZl5mW7VIiJ4/JLHMbrHaASJZ/5EqlUDcnJKv/dWmsxM4NdfjVaDntAroRe2DduG1nWMli1VadX38tKX8a+P/4VC9V3rVADIzzfuxW7e7Nr6qsbf0ZEjvj9T7hjX0WN/M1QlIwDkAch2Ps52Pq5aI4DTnQ9gHIALALQEcAuAfwIYDmDUGesOAZBcwfZaAPhUVTs6f24OoCuADgA6iUgPAP0AHFDV9qraFoDLZ6AcfstPLFkCvPWWcbZSVQNbDESELaLKQyC9l/Ie6kbUxTWtrsHAFgOrXkgp+vQBLrvM9Q/bqCjjA92TZx5hIUbXjMU7F6P3p73x+TWf4+Z2N7v8/pcSX8IdF9yBkCDf/m/Tvj2waxdgs7m2vggwZYr3G5GUJic/B68sfQUXN7wY/Zv3930BlADjjK2kbOfz7tqpqusAQEQ2AFikqioi62AMhAzna71ghNs/K9jeblVd7vy5r3Mpav0VCSPslgD4n4i8CmCeqi5xtVh+xfITd90FpKYCDdy4bXZRw4sw6l+jilsLVkZ+YT6mrJ6CGRu81+pSBMjOBj78sPzQWrTIuHwZEgKEhnq+jh7n9cD4/uNxbetrK/W+hjUaonfT3p4vyAU2m3HMvvnGaEFalu++A3bvNn4ODvZNbSVVC6mGj1Z/hGV7l/l+5wQY99jOvIcQ7nzeXXklfnaUeOyA80RJRC4A8CGAQap6rILtZZX4WQC8rKodnMv5qjpZVbcA6ARjpoCXReRZV4tluPkBh/M2as2antnesr3LcNfcu+Bw4f7skawjyMnPQWhwKJJvTcbn13zumSLKMG0acO+9wJ9/lv56aiowYADwzDPeqyEkKARJXZNgC7YhOz8b1868FmsOrin3PTPWz8D8LfO9V5QLfvkFGDQI+Oyz0l+3242m/0lJvq2rpCAJwpakLXgx8UXziji3jQVQDacCLtz5eKy3dywijQHMAXC7M5Qq43sAd4tIpHNb8SJSV0QaAMhW1c9gzNZ9oasbZLj5gd69gaee8tz2dqTvwM+7fq6wL9ZJ+0l0+aALhiUb10JrVq+J4CDvft2/+25g+XKj71Vp4uKABQuAKnYTrLS9x/dixf4V2Htib7nrvbbsNbyb8q5viirDpZcaDWxuv73012024LffgLff9mlZZ6ke6qVhUKhCzlaRDwFIhTGDdSqAh3zRWhLAszAGSn5XRFaLiMtN21V1IYDPAfzuvMw5G0AUgHYA/hSR1QCeBvBfV7fJrgAmy883Ojq3bWuc0XiCqiKnIMelFo4T/pyASxpegk4NOnlm55WQkXH61C75+d65DFmRnPyc4g/ktYfWom3dtmc1isgvzMfR7KOIi4rzfYGlsNtPvweXmWncp/QXQ78dipphNfFqn1fNLiWQcYQSN/DMzWShocC4cZ4LNsBo7RgeGo5CRyF+2P7Daa8VOgrxzOJnii/DJXVNMiXYvv8eaNgQ+MvZaDE9HWjRApg+3eelFAfb7ozduPjDi/HcT8+dtU5ocKjfBNvWrcaxmu+8SpqXZ3QhGXVmezUTBUkQW02SqfjXZ6LMTGD9eu9tf2LKRPT9rC+6vN8FCeMSkPhJImZtmIUpq6fgq81feW/HLrjoIuCmm4w+XImJwAUXGMfj6FHzamoc3Rhv938bSV2Nm1bJW5OR+Ekiol6OQpt32iB5a0Utm33jvPOAjh2NrgGJicA//mE0wAl3ryuiRw1qMQh/7P+j+O/OX45dkaL/tv5an1WISC3nJcozFw9Ol1vGvnlZ0jzvvgs89JARcG3aeH77czfNxX3z70PNajURGhyK/MJ85Gs+Xur1Em5qd5Pnd1hJyclG4webzfhgzs42LrdNmAD0N7kV+YItC3DTlzehRrUaSM9JR3hoOGqE1cCE/hP8oom7Px+75K3JSEpOgi3IhrCQMOQW5MLusPvPsStRX3hoOLLzs/2qvhJ4WdINPHMz0Q03GFORlDZivie8/efbiA2LRUhwCDYe3YjsgmzYgmx4f+X73tlhJY0dCwQFAX//bXwwR0QYH9Zjvd6uq2IvL30Z9kI7FIpWdVqhcXRj2IJsGLvMD4qDcYxCQ4EdO4Djx/3r2I1dNha2IBuO5hzFlrQtiLBFwBZkw4PzHzztMvnoxaOxeKcx+0OBowCjF4/Gr7t/BQDkFeRh9OLR+G3PbwCM2RhGLx6N5fuMblEn8k5g9OLRWLF/BQAgPScdoxePxsrUlQCAo9lHMXrxaKw9tBaAMUTd6MWjseHwBoxdNhaigqz8LIhIcX3+8t+WPIPhZqLatY3Wg94aRWJnxk6Eh4ajWnA11I2oi5iwGISHhmNXxi7v7LCSdu40PpBttlPDcoWHGx2WzbYvcx9a1W6F+hH1AZy6j+lvxy4o6FS/N385dkV/d5GhkYiyGa1cwkPDsfv4bvy86+fi9cYsHYMlu40+uYWOQoxZOqa4f5y90I4xS8fgj/1/AAByCnIwZumY4pFlTtpPYszSMVh9cDUA4HjecYxZOgbrDq0DABzLPoYxS8dg45GNAIwuL2OWjsHmo5uxM2MnQoJDcDT71DVwf/pvS57By5ImefNN4z5Tby/2CU78JBGpmamIsJWYL82ehbioOCy+07tzzbkiMdHo11ZyQtOsLKM7gJenwqsQj13V+f2x8/P6SuBlSTfwzM0Edrsx1NbXX3t3PyO6jYDdYTcm2FRFlj0LdocdI7r5qBNZBUaMMI5FVpYx+kZWlvHYV33cyq2Nx67qtfn7sfPz+sgzGG4msNmM5tz/+Y9399O/eX9M6D8BcVFxSM9NR1xUnF/dNO/f32gAERdndAWIi/OPBhEAj51btfn7sfPz+s5FIuLxAVt5WdLHig63WfOaEVHAqNp8blLKfG7q3gglInIbgGEAbAD+APAggOMwZgm4EkAOjPEkD4nI9QCeA1AI4Liq9ihjm3cBuAJAGIAIAFcBGA9jVJIQGPPBfS0ibQB87Nx3EIBrVXVrRTXzzM3HvvsO6NTJaBBARORJzmA7ez43qfp8biLSCsCNALqragcYoXUrjEBarqrtAfwKoGgoimcBXO58vqLpRS4BcKeqJsIYXmuxqnYB0AvAWBGJAHA/gHHOfXcGsM+VujnljY+JALGxQHy82ZUQkQWVNp9b0fNVPXvrDWNk/hXGpNmoDuAwADuAec51/gLQx/nzbwCmiMhMGAMpl+cHVS2aoqcvgIEiMtz5OAxAYwC/A3haRBoCmOPKWRvAcPO5fv2MhYjIC7wxn5sA+ERVTxveXUSG66n7WoVw5omq3i8iF8G45LhaRDqUM/3NmdPeXKuqf5+xziYR+cO5ve9F5B5VrbBZKy9L+tCGDcbgwEREXuKN+dwWAbhOROoCgIjEish5Za0sIs1U9Q9VfRbAUQCNXNzP9wAeFufpoYh0dP7bFMAOVX0bwDcwZgKvEMPNR+x2o0/bkCFmV0JEFubx+dxUdSOAZwAsFJG1AH6AcS+vzBpEZJ2IrIdxL678yRJPeRFAKIC1zvcWTQp4I4D1zmlvWgL41JWNsbWkjxQWGo1J6tY1RnAnIqqA37SWDES85+YjwcHAFVeYXQURWZ0zyM65MDsTL0v6wJYtwBtvACdOmF0JEZFvicjlpUx54/U5t3jm5gPz5gFPPQXcdhtQo4bZ1RAR+Y6qfg+jsYhP8czNBx57DNi2zbjfRkRE3sdw87Ki9jqNXG0MS0REbmO4eVnfvsb0NkRE5DsMNy/KyTEmJC055xYREXkfG5R4UfXqwBdfmF0FEdG5h2duXnLiBHDggNlVEBGdmxhuXvLRR0DjxsCuXWZXQkR07uFlSS8ZONCY3qZJE7MrISI69/DMzUuaNgUeecTsKoiIzk0MNy947z1g9WqzqyAiOncx3Dzs5EngySeBadPMroSI6NzFe24eFhlpNCIpKDC7EiKicxfDzQtiYsyugIjo3MbLkh707bfA5ZcDBw+aXQkR0bmN4eZBJ08anbdr1za7EiKic5sp4SYiY0Vks4isFZGvRCTGjDo8JTkZSEwERo0yhtz64QezKyIiOreZdeb2A4C2qnoBgC0AnjKpDrclJwNJScCePUDNmkBqqvE4OdnsyoiIzl2mhJuqLlTVovaEywE0NKMOTxg7FggJMVpI7t9vzABgsxnPExGROfzhntvdAMo8zxGRoSKSIiIpR44c8WFZrtm5EwgPBxo2BGJjjefCwzmmJBGRmbwWbiLyo4isL2UZVGKdpwEUACizy7Oqvq+qnVW1c506dbxVbpUlJAC5uafP25adzTEliYjM5LV+bqp6WXmvi8idAK4E0FtV1Vt1eNuIEcY9NsA4Y8vOBux243kiIjKHWa0l+wF4AsBAVc02owZP6d8fmDABiIsD0tONfydMMJ4nIiJziBknTSKyDUA1AMecTy1X1fsrel/nzp01JSXFq7UREfkJMbuAQGbK8Fuqer4Z+yUionODP7SWJCIi8iiGGxERWQ7DjYiILIfhRkRElsNwIyIiy2G4ERGR5TDciIjIchhuRERkOQw3IiKyHIYbERFZDsONiIgsh+FGRESWw3AjIiLLYbgREZHlMNyIiMhyGG5ERGQ5DDciIrIchhsREVkOw42IiCyH4UZERJbDcCMiIsthuBERkeUw3IiIyHIYbkREZDkMNyIishyGGxERWQ7DjYiILIfhRkRElsNwIyIiy2G4ERGR5TDciIjIchhuRERkOQw3IiKyHIYbERFZDsONiIgsh+FGRESWw3AjIiLLYbgREZHlMNyIiMhyGG5ERGQ5DDciIrIchhsREVkOw42IiCyH4UZERJbDcCMiIsthuBERkeUw3IiIyHIYbkREZDkMNyIishyGGxERWQ7DjYiILIfhRkRElsNwIyIiy2G4ERGR5TDciIjIckwNNxEZLiIqIrXNrIOIiKzFtHATkUYA+gDYY1YNRERkTWaeub0JYCQANbEGIiKyIFPCTUQGAtivqmvM2D8REVlbiLc2LCI/AqhfyktPAxgFoK+L2xkKYCgANG7c2GP1ERGRdYmqb68Kikg7AIsAZDufagjgAICuqnqwvPd27txZU1JSvFwhEZFfELMLCGReO3Mri6quA1C36LGI7ALQWVWP+roWIiKyJvZzIyIiy/H5mduZVLWJ2TUQEZG18MyNiIgsh+FGRESWw3AjIiLLYbgREZHlMNyIiMhyGG5ERGQ5DDciIrIchhsREVkOw42IiCyH4UZERJbDcCMiIsthuBERkeUw3IiIyHIYbkREZDkMNyIishyGGxERWQ7DjYiILIfhRkRElsNwIyIiy2G4ERGR5TDciIjIchhuRERkOQw3IiKyHIYbERFZDsONiIgsh+FGRESWw3AjIiLLEVU1uwaXiUgmgL/NrqMctQEcNbuIMvhzbYB/1+fPtQH+XZ8/1wb4d31hqtrW7CICVYjZBVTS36ra2ewiyiIiKf5anz/XBvh3ff5cG+Df9flzbYB/1yciKWbXEMh4WZKIiCyH4UZERJYTaOH2vtkFVMCf6/Pn2gD/rs+fawP8uz5/rg3w7/r8uTa/F1ANSoiIiFwRaGduREREFWK4ERGR5QREuIlIPxH5W0S2iciTZtdTkoh8JCKHRWS92bWURkQaichPIrJJRDaIyCNm11RERMJE5E8RWeOs7QWzazqTiASLyCoRmWd2LWcSkV0isk5EVvtjs3ERiRGR2SKy2fn3d4nZNQGAiLRwHrOi5YSIPGp2XSWJyP85/59YLyJfiEiY2TUFGr+/5yYiwQC2AOgDYB+AFQBuVtWNphbmJCI9AJwE8Kk/drgUkTgAcaq6UkSiAPwF4Gp/OH4iIgAiVPWkiIQCWArgEVVdbnJpxUTkMQCdAdRQ1SvNrqckEdkFoLOq+mUnZBH5BMASVf1QRGwAwlU1w+SyTuP8fNkP4CJV3W12PQAgIvEw/l9orao5IjITwAJVnWJuZYElEM7cugLYpqo7VNUOYDqAQSbXVExVfwWQZnYdZVHVVFVd6fw5E8AmAPHmVmVQw0nnw1Dn4jfftkSkIYArAHxodi2BRkRqAOgBYDIAqKrd34LNqTeA7f4SbCWEAKguIiEAwgEcMLmegBMI4RYPYG+Jx/vgJx/OgUZEmgDoCOAPk0sp5rzstxrAYQA/qKrf1AbgLQAjAThMrqMsCmChiPwlIkPNLuYMTQEcAfCx87LuhyISYXZRpbgJwBdmF1GSqu4H8D8AewCkAjiuqgvNrSrwBEK4SSnP+c23+0AhIpEAvgTwqKqeMLueIqpaqKodADQE0FVE/OLSrohcCeCwqv5ldi3l6K6qFwLoD+Ah5yVyfxEC4EIAE1W1I4AsAP52v9wGYCCAWWbXUpKI1IRxdSoBQAMAESJym7lVBZ5ACLd9ABqVeNwQPEWvFOf9rC8BTFPVOWbXUxrnJaufAfQzt5Ji3QEMdN7Xmg4gUUQ+M7ek06nqAee/hwF8BeMSvr/YB2BfiTPx2TDCzp/0B7BSVQ+ZXcgZLgOwU1WPqGo+gDkAuplcU8AJhHBbAaC5iCQ4v2ndBOAbk2sKGM5GG5MBbFLVN8yupyQRqSMiMc6fq8P4n3qzqUU5qepTqtpQVZvA+JtbrKp+8+1ZRCKcDYTgvNzXF4DftNhV1YMA9opIC+dTvQGY3ojpDDfDzy5JOu0BcLGIhDv//+0N4145VYLfzwqgqgUikgTgewDBAD5S1Q0ml1VMRL4A0BNAbRHZB+A5VZ1sblWn6Q7gdgDrnPe2AGCUqi4wr6RicQA+cbZYCwIwU1X9rsm9n6oH4Cvjsw8hAD5X1e/MLeksDwOY5vxSugPAYJPrKSYi4TBaYN9ndi1nUtU/RGQ2gJUACgCsAofiqjS/7wpARERUWYFwWZKIiKhSGG5ERGQ5DDciIrIchhsREVkOw42IiCyH4UZERJbDcCMiIsthuBGVQUS6iMha57xzEc75tfxi7EsiKh87cROVQ0T+CyAMQHUYYyW+bHJJROQChhtROZxDR60AkAugm6oWmlwSEbmAlyWJyhcLIBJAFIwzOCIKADxzIyqHiHwDY8qbBABxqppkcklE5AK/nxWAyCwicgeAAlX93DlzwTIRSVTVxWbXRkTl45kbERFZDu+5ERGR5TDciIjIchhuRERkOQw3IiKyHIYbERFZDsONiIgsh+FGRESW8/8aiKYfrwBt7QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "#hide_input\n", "\n", "fig = sns.lmplot(x = 'x', y= 'y', data = df_res, hue='Residuals', palette=[\"b\", \"g\", \"black\", \"orange\"], fit_reg=False )\n", "fig.axes[0,0].plot(x, m1_res, linestyle='dotted', color = 'b')\n", "fig.axes[0,0].plot(x, m2_res, linestyle='dotted', color = 'g')\n", "fig.axes[0,0].plot(x, ens_res, color = 'black')\n", "fig.axes[0,0].plot(x, np.zeros(7), linewidth = 2, color = 'orange')\n", "\n", "plt.xlim([0, 8])\n", "plt.ylim([-5, 5])\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the residual chart we can see that the ensemble does a bit worse for x=3 compared to Model 2. But it also decreases the residuals for the outliers (points 1, 5, 7).\n", "\n", "Let's check the stats:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Ensemble. mean: 0.0357, var: 0.2219\n" ] } ], "source": [ "#hide_input\n", "\n", "print(f'Ensemble. mean: {ens_res.mean(): .4f}, var: {ens_res.var(): .4f}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We dramatically reduced the variance, hence reduced the risk/chance. The mean value is now in between Model 1 and Model 2.\n", "\n", "Finally let's play around with the model weights in the ensemble and check how mean and variance change. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "# generate weights for w1\n", "weight_m1 = np.linspace(0, 1, 30)\n", "\n", "ens_mean = np.zeros(30)\n", "ens_var = np.zeros(30)\n", "\n", "for i, w1 in enumerate(weight_m1):\n", " # build ensemble for different weights\n", " ens = m1*w1 + m2*(1-w1)\n", " ens_res = y_true - ens\n", " \n", " # keep track of mean and var of the differently weighted ensembles\n", " ens_mean[i] = ens_res.mean()\n", " ens_var[i] = ens_res.var()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAaiElEQVR4nO3dcZCU933f8ffHJxRfFLnXhouNDijKzFUdbCyB1wjZbmondgXE9lGqdpBbq2HaIbTGsSatOrLdcex2Upo4k4lVyyLEIQ6NI5I6kobxYGHPJLKbeJA4JISEZNILY5cDWuHGoEhQK6jf/vE8Jx6W5+6e4/a3++zu5zWzA/s8v+f2e+tH/vLs77O/RxGBmZlZs9d1ugAzM6snNwgzMyvlBmFmZqXcIMzMrJQbhJmZlbqm0wW00sKFC2PZsmWdLsPMrGscOnTo+xExXLavpxrEsmXLGB8f73QZZmZdQ9L3ptvnj5jMzKyUG4SZmZVygzAzs1JuEGZmVsoNwszMSvVUisnMetsjT53ks/uPcersBW4YGuSe229iw8qRTpfVs9wgzKwrPPLUST7+0DNc+OtXATh59gIff+gZADeJRPwRk5l1hc/uP/Zac5hy4a9f5bP7j3Woot7nBmFmXeHU2Qtz2m7zl7RBSFor6ZikCUn3luyXpPvy/Uckrcq33yTpcOHxoqS7U9ZqZvV2w9DgnLbb/CVrEJIGgPuBdcBy4E5Jy5uGrQNG88cW4AGAiDgWEbdExC3A24DzwMOpajWz+rvn9psYXDBw2bbBBQPcc/tNHaqo96W8glgNTETE8Yh4BdgDjDWNGQN2R+YAMCRpUdOYnwH+IiKmXS/EzHrfhpUjbN+4gpGhQQSMDA2yfeMKT1AnlDLFNAKcKDyfBG6tMGYEOF3Ytgl4cLoXkbSF7OqDpUuXzqNcM5tJHSKmG1aOuCG0UcorCJVsi7mMkXQt8EHgv033IhGxMyIaEdEYHi5dsdbM5mkqYnry7AWCSxHTR5462enSLKGUDWISWFJ4vhg4Nccx64AnI+J/J6nQzCpxxLQ/pWwQB4FRSTfmVwKbgL1NY/YCd+VppjXAuYgofrx0JzN8vGRm7eGIaX9KNgcRERclbQP2AwPArog4Kmlrvn8HsA9YD0yQJZU2Tx0v6UeB9wE/n6pGM6vmhqFBTpY0A0dMe1vSpTYiYh9ZEyhu21H4ewAfmebY88CPp6zPzKq55/abLlvmAhwx7Qdei8nMZjWVHOp0isnayw3CrIt0MmrqiGn/cYMw6xJezdTazYv1mXUJR02t3dwgzLqEo6bWbm4QZl3Cq5lau7lBmHUJr2Zq7eZJarMu4aiptZsbhNlV6kTk1FFTayc3CLOr4Mip9QPPQZhdBUdOrR+4QZhdBUdOrR+4QZhdBUdOrR+4QZhdBUdOrR94ktrsKjhyav3ADcJ6Trvip46cWq9zg7Ce4vipWet4DsJ6iuOnZq3jBmE9xfFTs9ZJ2iAkrZV0TNKEpHtL9kvSffn+I5JWFfYNSfqKpO9Iel7SbSlrtd7g+KlZ6yRrEJIGgPuBdcBy4E5Jy5uGrQNG88cW4IHCvs8Bj0bE3wVuBp5PVav1DsdPzVon5ST1amAiIo4DSNoDjAHPFcaMAbsjIoAD+VXDIuBl4KeAnwOIiFeAVxLWam2WKmnk+KlZ66RsECPAicLzSeDWCmNGgIvAGeB3JN0MHAI+FhEvN7+IpC1kVx8sXbq0ZcVbOqmTRo6fmrVGyjkIlWyLimOuAVYBD0TESrIriivmMAAiYmdENCKiMTw8PJ96rU2cNDLrDikbxCSwpPB8MXCq4phJYDIiHs+3f4WsYVgPcNLIrDukbBAHgVFJN0q6FtgE7G0asxe4K08zrQHORcTpiPhfwAlJUzOLP8PlcxfWxZw0MusOyRpERFwEtgH7yRJIfxgRRyVtlbQ1H7YPOA5MAL8F/OvCj/go8GVJR4BbgP+UqlZrLyeNzLqDsgBRb2g0GjE+Pt7pMqyCTtyu08yuJOlQRDTK9nktJptViv8zd9LIrP7cIGxGXvzOrH95LSabkSOpZv3LDcJm5EiqWf9yg7AZOZJq1r/cIGxGjqSa9S9PUtuMvPidWf9yg+hhrYqnOpJq1p/cIHqU46lmNl+eg+hRjqea2Xy5QfQox1PNbL7cIHqU46lmNl9uED3K8VQzmy9PUvcox1PNbL7cIGpuPlFVx1PNbD7cIGrMUVUz6yTPQdSYo6pm1kluEDXmqKqZdVLSBiFpraRjkiYk3VuyX5Luy/cfkbSqsO+7kp6RdFhSX95H1FFVM+ukZA1C0gBwP7AOWA7cKWl507B1wGj+2AI80LT/PRFxy3T3S+11jqqaWSelvIJYDUxExPGIeAXYA4w1jRkDdkfmADAkaVHCmrrKhpUjbN+4gpGhQQSMDA2yfeMKT1CbWVukTDGNACcKzyeBWyuMGQFOAwF8XVIAvxkROxPW2hZXE1l1VNXMOiVlg1DJtpjDmHdGxClJPwF8Q9J3IuJbV7yItIXs4ymWLl06n3qTcmTVzLpNyo+YJoElheeLgVNVx0TE1J8vAA+TfWR1hYjYGRGNiGgMDw+3qPTWc2TVzLpNygZxEBiVdKOka4FNwN6mMXuBu/I00xrgXESclnSdpOsBJF0H/APg2YS1JufIqpl1m2QfMUXERUnbgP3AALArIo5K2prv3wHsA9YDE8B5YHN++BuBhyVN1fj7EfFoqlrb4YahQU6WNANHVs2srpIutRER+8iaQHHbjsLfA/hIyXHHgZtT1tZu99x+02VzEODIqpnVm9dimqeqySSvrmpm3cYNYh7mmkxyZNXMuonXYpoHJ5PMrJe5QcyDk0lm1svcIObBi+mZWS9zg5gHL6ZnZr3Mk9Tz4GSSmfUyN4hZzBZjdTLJzHqVG8QMvMCemfUzz0HMwDFWM+tnbhAzcIzVzPqZG8QMHGM1s37mBjEDx1jNrJ95knoGjrGaWT9zgyiYLtLqhmBm/cgNIudIq5nZ5TwHkXOk1czscm4QOUdazcwu5waRc6TVzOxySRuEpLWSjkmakHRvyX5Jui/ff0TSqqb9A5KekvTVlHWCI61mZs2SNQhJA8D9wDpgOXCnpOVNw9YBo/ljC/BA0/6PAc+nqrFow8oRtm9cwcjQIAJGhgbZvnGFJ6jNrG+lTDGtBiYi4jiApD3AGPBcYcwYsDsiAjggaUjSoog4LWkx8LPALwO/mKrI2VZrNTPrVyk/YhoBThSeT+bbqo75DeDfAf9vpheRtEXSuKTxM2fOzKnAqWjrybMXCC5FWx956uScfo6ZWS9K2SBUsi2qjJH0fuCFiDg024tExM6IaEREY3h4eE4FOtpqZja9yh8xSXoHsKx4TETsnuGQSWBJ4fli4FTFMXcAH5S0Hng98AZJvxcR/6xqvVU42mpmNr1KVxCS/ivwa8C7gLfnj8Yshx0ERiXdKOlaYBOwt2nMXuCuPM20BjgXEacj4uMRsTgiluXH/XGrmwM42mpmNpOqVxANYHk+mVxJRFyUtA3YDwwAuyLiqKSt+f4dwD5gPTABnAc2z6X4+brn9psuW14DHG01M5tStUE8C7wJOD2XHx4R+8iaQHHbjsLfA/jILD/jMeCxubxuVV6t1cxselUbxELgOUlPAD+c2hgRH0xSlZmZdVzVBvHplEV0ildwNTObXqUGERHfTF1IJ8wUc3WDMLN+VzXFtEbSQUkvSXpF0quSXkxdXGqOuZqZTa/qF+U+D9wJ/A9gEPiX+bau5pirmdn0Kn+TOiImgIGIeDUifgd4d7Kq2sQruJqZTa/qJPX5/MtuhyX9Klnc9bp0ZbVHMeZ68uwFBqTLltrwPISZ9bOqVxAfzsduA14mWx7jH6Uqqp02rBx57Uri1fx7gF60z8ysYoOIiO+RLay3KCI+ExG/mH/k1BO8aJ+Z2ZWqppg+ABwGHs2f3yKpeV2lruU0k5nZlap+xPRpshsAnQWIiMNkK7v2BKeZzMyuVLVBXIyIc0kr6SCnmczMrlR5sT5JHwIGJI0CvwB8O11Z7eVF+8zMrlS1QXwU+CTZQn0Pki3h/R9TFdUJG1aOXNEQfL9qM+tnVddiOk/WID6Ztpz68EJ+ZtbvZmwQsyWVenm5by/kZ2b9brYriNuAE2QfKz1O9l2IvuDoq5n1u9lSTG8CPgG8Bfgc8D7g+xHxzV5dAnyKo69m1u9mbBD5wnyPRsQ/B9aQ3Tv6MUkfbUt1HeToq5n1u1m/ByHpRyRtBH6P7P7R9wEPVfnhktZKOiZpQtK9Jfsl6b58/xFJq/Ltr5f0hKSnJR2V9Jm5/Vrzt2HlCNs3rmBkaBABI0ODbN+4wvMPZtY3Zpuk/l2yj5e+BnwmIp6t+oMlDQD3k30sNQkclLQ3Ip4rDFsHjOaPW4EH8j9/CPx0RLwkaQHwp5K+FhEHqv9q81cWfS1yDNbMetlsk9QfJlu99e8AvyC9NkctICLiDTMcuxqYiIjjAJL2AGNAsUGMAbsjIoADkoYkLYqI08BL+ZgF+SOq/1rpOQZrZr1utjmI10XE9fnjDYXH9bM0B4ARsgTUlMl8W6UxkgYkHQZeAL4REY+XvYikLZLGJY2fOXNmlpJaxyvAmlmvq3xHuatQFoltvgqYdkw+QX4LsBhYLektZS8SETsjohERjeHh4fnUOyeOwZpZr0vZICbJbiw0ZTFwaq5jIuIs8BiwtuUVzoNjsGbW61I2iIPAqKQb89uVbgKav5m9F7grTzOtAc5FxGlJw5KGACQNAu8FvpOw1jlzDNbMel3VxfrmLCIuStpGtrDfALArIo5K2prv3wHsA9aTfb/iPLA5P3wR8Lt5Eup1wB9GxFdT1Xo1vAKsmfU6RdQqHDQvjUYjxsfHO13GFRyHNbO6knQoIhpl+5JdQVjGcVgz61Yp5yAMx2HNrHu5QSTmOKyZdSs3iMQchzWzbuUGkZjjsGbWrTxJnZjjsGbWrdwg2mC2VWGbORZrZnXgBlEzjsWaWV14DqJmHIs1s7pwg6gZx2LNrC7cIGrGsVgzqws3iJpxLNbM6sKT1DXjWKyZ1YUbRA3NNRZb5IismbWKG0QPcUTWzFrJcxA9xBFZM2slN4ge4oismbWSG0QPcUTWzFopaYOQtFbSMUkTku4t2S9J9+X7j0halW9fIulPJD0v6aikj6Wss1c4ImtmrZRsklrSAHA/8D5gEjgoaW9EPFcYtg4YzR+3Ag/kf14E/k1EPCnpeuCQpG80HWtNWhmRdRrKzFKmmFYDExFxHEDSHmAMKP6f/BiwOyICOCBpSNKiiDgNnAaIiL+S9Dww0nSslZhPRHaK01BmBmk/YhoBThSeT+bb5jRG0jJgJfB460u0Mk5DmRmkbRAq2RZzGSPpx4A/Au6OiBdLX0TaImlc0viZM2euuli7xGkoM4O0DWISWFJ4vhg4VXWMpAVkzeHLEfHQdC8SETsjohERjeHh4ZYU3u+chjIzSNsgDgKjkm6UdC2wCdjbNGYvcFeeZloDnIuI05IE/DbwfET8esIarYTTUGYGCSepI+KipG3AfmAA2BURRyVtzffvAPYB64EJ4DywOT/8ncCHgWckHc63fSIi9qWq1y7xgoFmBqAsQNQbGo1GjI+Pd7oMK+HYrFk9SToUEY2yfV6sz5JzbNasO3mpDUvOsVmz7uQGYck5NmvWndwgLDnHZs26kxuEJefYrFl38iS1JefYrFl3coOwtmjFIoLTcYTWLA03COtqjtCapeM5COtqjtCapeMGYV3NEVqzdNwgrKs5QmuWjhuEdTVHaM3S8SS1dTVHaM3ScYOwrpcyQlvkOK31GzcIswocp7V+5DkIswocp7V+5AZhVoHjtNaP3CDMKnCc1vqRG4RZBY7TWj9K2iAkrZV0TNKEpHtL9kvSffn+I5JWFfbtkvSCpGdT1mhWxYaVI2zfuIKRoUEEjAwNsn3jCk9QW09LlmKSNADcD7wPmAQOStobEc8Vhq0DRvPHrcAD+Z8AXwI+D+xOVaPZXLQrTjvFsVrrtJRXEKuBiYg4HhGvAHuAsaYxY8DuyBwAhiQtAoiIbwF/mbA+s9qaitWePHuB4FKs9pGnTna6NOsjKRvECHCi8Hwy3zbXMTOStEXSuKTxM2fOXFWhZnXjWK3VQcoGoZJtcRVjZhQROyOiERGN4eHhuRxqVluO1VodpGwQk8CSwvPFwKmrGGPWdxyrtTpI2SAOAqOSbpR0LbAJ2Ns0Zi9wV55mWgOci4jTCWsy6wqO1VodJEsxRcRFSduA/cAAsCsijkramu/fAewD1gMTwHlg89Txkh4E3g0slDQJ/FJE/Haqes3qpJOr1Do9ZVMUMaeP/Gut0WjE+Ph4p8sw61rNixJCduXi73z0LkmHIqJRts/fpDaz1zg9ZUVuEGb2GqenrMgNwsxe4/SUFblBmNlrnJ6yIt9Rzsxe43t8W5EbhJldpt2LEpZx1LYe3CDMrFZ8/+/68ByEmdWKo7b14QZhZrXiqG19uEGYWa04alsfbhBmViuO2taHJ6nNrFYcta0PNwgzq506RG2L+jV26wZhZjaDfo7deg7CzGwG/Ry7dYMwM5tBP8du3SDMzGbQz7FbNwgzsxn0c+zWk9RmZjPo59ht0gYhaS3wOWAA+GJE/Oem/cr3rwfOAz8XEU9WOdbMrF3qFrudkjp+m+wjJkkDwP3AOmA5cKek5U3D1gGj+WML8MAcjjUz61tT8duTZy8QXIrfPvLUyZa9Rso5iNXAREQcj4hXgD3AWNOYMWB3ZA4AQ5IWVTzWzKxvtSN+m7JBjAAnCs8n821VxlQ5FgBJWySNSxo/c+bMvIs2M+sG7YjfpmwQKtkWFcdUOTbbGLEzIhoR0RgeHp5jiWZm3akd8duUDWISWFJ4vhg4VXFMlWPNzPpWO+K3KRvEQWBU0o2SrgU2AXubxuwF7lJmDXAuIk5XPNbMrG9tWDnC9o0rGBkaRMDI0CDbN65oaYopWcw1Ii5K2gbsJ4uq7oqIo5K25vt3APvIIq4TZDHXzTMdm6pWM7NulDp+q4jSj/a7UqPRiPHx8U6XYWbWNSQdiohG2T4vtWFmZqXcIMzMrJQbhJmZlXKDMDOzUj01SS3pDPC9Fv7IhcD3W/jzUnKtrdctdYJrTaFb6oT51fq3I6L0W8Y91SBaTdL4dLP7deNaW69b6gTXmkK31AnpavVHTGZmVsoNwszMSrlBzGxnpwuYA9faet1SJ7jWFLqlTkhUq+cgzMyslK8gzMyslBuEmZmV6tsGIWmtpGOSJiTdW7L/n0o6kj++Lenmwr7vSnpG0mFJyVcHrFDruyWdy+s5LOlTVY9tc533FGp8VtKrkv5Wvq9t76mkXZJekPTsNPsl6b789zgiaVVhX9vez4q11uk8na3Wupyns9VZi/M0f70lkv5E0vOSjkr6WMmYdOdrRPTdg2wJ8b8AfhK4FngaWN405h3A38z/vg54vLDvu8DCGtX6buCrV3NsO+tsGv8B4I879J7+FLAKeHaa/euBr5Hd2XDN1P/27Xw/51BrLc7TirV2/DytUmfT2I6dp/nrLQJW5X+/Hvjzkv/+k52v/XoFsRqYiIjjEfEKsAcYKw6IiG9HxA/ypwfI7mrXCbPWmujY1HXeCTyYqJYZRcS3gL+cYcgYsDsyB4AhSYto7/tZqdYanadV3tfptPV9nWOdHTtPASLidEQ8mf/9r4DngeYbQCQ7X/u1QYwAJwrPJ7nyTS/6F2QdekoAX5d0SNKWBPUVVa31NklPS/qapDfP8dhWqPxakn4UWAv8UWFzO9/T2Uz3u7Tz/bwanTxPq+r0eVpZ3c5TScuAlcDjTbuSna/J7ihXcyrZVpr3lfQesv/w3lXY/M6IOCXpJ4BvSPpO/q+SFKrU+iTZeiovSVoPPAKMVjy2VebyWh8A/iwiiv+Ka+d7Opvpfpd2vp9zUoPztIo6nKdzUZvzVNKPkTWquyPixebdJYe05Hzt1yuISWBJ4fli4FTzIElvBb4IjEXE/5naHhGn8j9fAB4mu5TrWK0R8WJEvJT/fR+wQNLCKse2s86CTTRdtrf5PZ3NdL9LO9/Pympyns6qJufpXNTiPJW0gKw5fDkiHioZku58bddkS50eZFdOx4EbuTR58+amMUvJ7pX9jqbt1wHXF/7+bWBth2t9E5e+9Lga+J9k/3qY9dh21pmP+xtkn/9e16n3NH+dZUw/mfqzXD7p98Rcfsc211qL87RirR0/T6vUWbPzVMBu4DdmGJPsfO3Lj5gi4qKkbcB+spn+XRFxVNLWfP8O4FPAjwNfkARwMbLVEt8IPJxvuwb4/Yh4tMO13gH8K0kXgQvApsjOkNJjO1gnwD8Evh4RLxcOb+t7KulBskTNQkmTwC8BCwp17iNLhkwA54HNM/2OqeqsWGstztOKtXb8PK1YJ9TgPM29E/gw8Iykw/m2T5D9wyD5+eqlNszMrFS/zkGYmdks3CDMzKyUG4SZmZVygzAzs1JuEGZmVsoNwqxA0mOSbm/adrekL1Q8/j9Iem+a6szayzFXswJJPw+siYjNhW0HgHsi4r/PcuxARLyaukazdvEVhNnlvgK8X9KPwGsLpN0AfEjSeL4m/2emBuf3B/iUpD8F/rGkL0m6I9/3KUkH83sK7FT+Dav8KuVXJD0h6c8l/b18+4CkX8vvN3BE0kfz7W+T9M18gbj9+UqdZsm5QZgVRLaW0RNkq3hCth7PHwCfzL+h/Fbg7+frH035vxHxrojY0/TjPh8Rb4+ItwCDwPsL+66JiNXA3WTf5AXYQrYswsqIeCvw5Xwdnv8C3BERbwN2Ab/col/XbEZuEGZXepCsMcClBdv+iaQngaeANwPLC+P/YJqf8x5Jj0t6Bvjp/LgpU4uuHSJbFwjgvcCOiLgIENkqojcBbyFbOfQw8O/p4D0frL/05VpMZrN4BPj1/NaNg8APgH8LvD0ifiDpS8DrC+Nfbv4Bkl4PfAFoRMQJSZ9uOuaH+Z+vcum/Q3HlcswCjkbEbfP5hcyuhq8gzJpEtiT1Y2Qf5zwIvIGsCZyT9EayW3vOZqoZfD9fy/+OCsd8Hdgq6RoAZfdBPgYMS7ot37agcKMds6TcIMzKPQjcDOyJiKfJPlo6StY0/my2gyPiLPBbwDNkVyQHK7zmF8mWwD4i6WngQ5HdKvIO4FfybYfJ7kNtlpxjrmZmVspXEGZmVsoNwszMSrlBmJlZKTcIMzMr5QZhZmal3CDMzKyUG4SZmZX6/7pGZCp63cUfAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "#hide_input\n", "\n", "# plot var vs mean\n", "\n", "fig = plt.figure()\n", "ax = plt.subplot()\n", "plt.scatter(ens_var, ens_mean)\n", "ax.set_xlabel('Variance')\n", "ax.set_ylabel('Mean')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With the previous 50:50 split the variance seems almost at the lowest point. So we only get a reduction of the mean below 0.0357 if we allow the ensemble to have more variance, hence take more risk." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2. Weights by residual variance\n", "\n", "Since the Model 1 and Model 2 are well fitted, their average residuals are pretty close to 0. So let's focus on reducing our variance to avoid surprises on later later predictions.\n", "\n", "We now solve for the optimal weights that minimizes the variance of the residual of our ensemble with this function:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "fun = lambda w: (y_true-np.matmul(w, preds)).var()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We also define a constraint so that the `w.sum() == 0`:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "# w.sum() = 1 <=> 0 = w.sum()-1\n", "cons = ({'type': 'eq', 'fun': lambda w: w.sum()-1})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you want, you can also set bounds, so that the weights want be negative.\n", "\n", "I don't. I like the idea of going *short* with a model. And negative weights really increase the results of TPS predictions in chapter 4. " ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "bnds = ((0,None),\n", " (0,None),\n", " (0,None),\n", " (0,None),\n", " (0,None))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, we are all set to retrieve the optimal weights. " ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Calculated weights: [0.53150242 0.46849758]\n" ] } ], "source": [ "# predictions of Model1 and Model 2\n", "preds = np.array([m1, m2])\n", "\n", "# init weights\n", "w_init = np.ones(preds.shape[0])/preds.shape[0]\n", "\n", "# run optimization\n", "res = scipy.optimize.minimize(fun, w_init, method='SLSQP', constraints=cons) #,bounds=bnds\n", "\n", "# get optimal weights\n", "w_calc = res.x\n", "\n", "\n", "print(f'Calculated weights: {w_calc}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's see how the calculated weights perform." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Ensemble Ex1. mean: 0.0380, var: 0.2157\n" ] } ], "source": [ "ens_ex1 = np.matmul(w_calc, preds)\n", "ens_ex1_res=y_true-ens_ex1\n", "\n", "print(f'Ensemble Ex1. mean: {ens_ex1_res.mean(): .4f}, var: {ens_ex1_res.var(): .4f}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We con compare the results with the first ensemble 50:50 split. \n", "With the calculated weights we could further reduce the variance of the model (0.2219 -> 0.2157). But unfortunately the mean increased a bit (0.0357 -> 0.0380).\n", "\n", "We see the trade off between mean and variance and have to decide if we prefer a more stable model or take some risk for better results." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 3. Portfolio theory for ensembling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In finance different assets are often combined in a portfolio. There are many criteria for the asset selection/allocation. One of them is by choosing a risk strategy. In 1952 the economist Harry Markowitz defined a *Portfolio Selection* strategy which built the foundation of many portfolio strategies to come. There is a great summary on [Wikipidia](https://en.wikipedia.org/wiki/Modern_portfolio_theory), but the original paper can also be found with a google search.\n", "\n", "\n", "So, what it is all about. Let's assume we are living in an easy, plain vanilla world. We want to build a portfolio that yields high return with low risk. That's not easy. If we only buy stocks of our favorite fruit grower, a rainy summer would result in a low return. Wouldn't it be smart to also buy stocks of a raincoat producer, just in case. But what if the summer was sunny, then we would have rather invested the entire money in fruits instead of raincoats. It's clearly a trade off. Either we lower the risk of loosing money in a rainy summer and invest in both (fruits and raincoats). Or we take the risk investing all money in fruits to maybe gain more money. And if we lower the risk, in which raincoat producer should we invest? The one with the bumpy stock price or the one with a steady, but slowly growing stock price.\n", "\n", "Now, we already see the first similarities between our ensemble example above and the Portfolio Theory. Risk can be measured through variance and a good return of our ensemble is results in a low expected residual.\n", "\n", "But there is even more in Portfolio Theory. It also takes dependencies between assets into account. If the summer is sunny the fruit price goes up and the raincoat price goes down, they are somewhat negative correlated. \n", "\n", "Since we expect the average residual of our fitted models to be close to 0 and we build a linear model, we can expect our ensemble average residual also to be close to 0. Therefore, we focus on optimizing the portfolio variance, which can be boiled down to `Var_p = w'*Cov*w`. The covariance measures the dependency between combined models and also considers the variance.\n", "\n", "> What data can we actually use? In the financial example *returns* are the increase or decrease of an asset price (p/p_t-1), hence we are looking on returns for a certain period of time. In ML we can take our **out-of-fold** (oof) predictions and calculate the residuals from the train targets to build a dataset.\n", "\n", "> Can we do this despite we are looking at a time-series in the financial example? Yes, in this *basic* portfolio theory we don't take time dependencies into account. But it's important to keep the same order for the different asset returns for correlation/covariance calculation. We want to compare the residual of model 1 and 2 for always the same data item.\n", "\n", "The optimization function for the second ensemble technique is:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "## following https://en.wikipedia.org/wiki/Modern_portfolio_theory\n", "\n", "# Predictions of Model 1 and Model 2\n", "preds = np.array([m1,m2])\n", "# Residuals of Model 1 and Model 2\n", "preds_res = np.array([m1_res, m2_res])\n", "\n", "# handle residuals like asset returns\n", "R = np.array(preds_res.mean(axis=1))\n", "# factor by which R is considered during optimization. turned off for our example\n", "q = 0 #-1\n", "\n", "# covariance matrix of model residuals\n", "CM = np.cov(preds_res)\n", "\n", "# optimization function\n", "fun = lambda w: np.matmul(np.matmul(w.T,CM),w) - q * np.matmul(R,w)\n", "\n", "# constraint: weights must sum up to 1.0\n", "cons = ({'type': 'eq', 'fun': lambda x: x.sum()-1})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Run the optimization." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Calculated weights: [0.53150242 0.46849758]\n" ] } ], "source": [ "# init weights\n", "w_init = np.ones(preds.shape[0])/preds.shape[0]\n", "\n", "# run optimization\n", "res = scipy.optimize.minimize(fun, w_init, method='SLSQP', constraints=cons) #,bounds=bnds\n", "\n", "# get optimal weights\n", "w_calc = res.x\n", "\n", "print(f'Calculated weights: {w_calc}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The weights are the same as in the first technique. That really surprised me. And I run a couple of examples with different models. But the weights were only slightly different between the two techniques." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "data": { "text/plain": [ "(0.0379644588251949, 0.21567043618739917)" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#hide\n", "\n", "ens_ex2 = np.matmul(w_calc, preds)\n", "ens_ex2_res=y_true-ens_ex2\n", "ens_ex2_res.mean(), ens_ex2_res.var()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 4. Ensembling TPS Aug 2021\n", "\n", "Now that we have to techniques to ensemble, let's try them on the [TPS August 2021](https://www.kaggle.com/c/tabular-playground-series-aug-2021/overview) data.\n", "\n", "We do a 7 kfold split and calculate the residuals on the out-of-fold-predictions, that are used for validation. We train 7 regression models with different architecture so we get some diversity." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "N_SPLITS = 7\n", "SEED = 2021\n", "\n", "PATH_INPUT = '/home/kaggle/TPS-AUG-2021/input/'" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "# load and shuffle\n", "test = pd.read_csv(PATH_INPUT + 'test.csv')\n", "\n", "train = pd.read_csv(PATH_INPUT + 'train.csv').sample(frac=1.0, random_state = SEED).reset_index(drop=True)\n", "\n", "train['fold_crit'] = train.loss\n", "train.loc[train.loss>=39, 'fold_crit']=39\n" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "target = 'loss'\n", "fold_crit = 'fold_crit'\n", "features = list(set(train.columns)-set(['id','kfold','loss','fold_crit']+[target]))" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "kfold\n", "0.0 35715\n", "1.0 35715\n", "2.0 35714\n", "3.0 35714\n", "4.0 35714\n", "5.0 35714\n", "6.0 35714\n", "Name: loss, dtype: int64" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# apply abhisheks splitting technique\n", "skf = StratifiedKFold(n_splits = N_SPLITS, random_state = None, shuffle = False)\n", "\n", "train.kfold = -1\n", "\n", "for f, (train_idx, valid_idx) in enumerate(skf.split(X = train, y = train[fold_crit].values)):\n", " \n", " train.loc[valid_idx,'kfold'] = f\n", "\n", "train.groupby('kfold')[target].count()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "# define models\n", "models = {\n", " 'LinReg': LinearRegression(n_jobs=-1),\n", " 'HGB': HistGradientBoostingRegressor(),\n", " 'XGB': XGBRegressor(tree_method = 'gpu_hist', reg_lambda= 6, reg_alpha= 10, n_jobs=-1),\n", " 'KNN': KNeighborsRegressor(100, n_jobs=-1),\n", " 'BayesRidge': BayesianRidge(),\n", " 'ExtraTrees': ExtraTreesRegressor(max_depth=2, n_jobs=-1),\n", " 'Poisson': Pipeline(steps=[('scale', StandardScaler()),\n", " ('pois', PoissonRegressor(max_iter=100))]) \n", "}\n", "\n" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [], "source": [ "#hide\n", "\n", "#for (k, model) in models.items():\n", "# scores = cross_val_score(model, train[features], train[target], n_jobs=-1, cv=skf, scoring='neg_root_mean_squared_error')\n", "# print(f'{k}:\\t {-scores.mean():0.5f} (+/- {scores.std():0.5f}) rmse')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Fit models and save oof predictions." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# Model:LinReg\n", "\n", "Fold 0 rmse: 7.89515\n", "Fold 1 rmse: 7.90212\n", "Fold 2 rmse: 7.90260\n", "Fold 3 rmse: 7.89748\n", "Fold 4 rmse: 7.89844\n", "Fold 5 rmse: 7.89134\n", "Fold 6 rmse: 7.89643\n", "\n", "Total rmse: 7.89765\n", "\n", "# Model:HGB\n", "\n", "Fold 0 rmse: 7.86447\n", "Fold 1 rmse: 7.87374\n", "Fold 2 rmse: 7.86688\n", "Fold 3 rmse: 7.86255\n", "Fold 4 rmse: 7.86822\n", "Fold 5 rmse: 7.85785\n", "Fold 6 rmse: 7.86566\n", "\n", "Total rmse: 7.86563\n", "\n", "# Model:XGB\n", "\n", "Fold 0 rmse: 7.91179\n", "Fold 1 rmse: 7.92748\n", "Fold 2 rmse: 7.92141\n", "Fold 3 rmse: 7.91901\n", "Fold 4 rmse: 7.91125\n", "Fold 5 rmse: 7.90286\n", "Fold 6 rmse: 7.92340\n", "\n", "Total rmse: 7.91675\n", "\n", "# Model:KNN\n", "\n", "Fold 0 rmse: 7.97845\n", "Fold 1 rmse: 7.97709\n", "Fold 2 rmse: 7.98165\n", "Fold 3 rmse: 7.97895\n", "Fold 4 rmse: 7.97781\n", "Fold 5 rmse: 7.97798\n", "Fold 6 rmse: 7.98711\n", "\n", "Total rmse: 7.97986\n", "\n", "# Model:BayesRidge\n", "\n", "Fold 0 rmse: 7.89649\n", "Fold 1 rmse: 7.90576\n", "Fold 2 rmse: 7.90349\n", "Fold 3 rmse: 7.90007\n", "Fold 4 rmse: 7.90121\n", "Fold 5 rmse: 7.89455\n", "Fold 6 rmse: 7.89928\n", "\n", "Total rmse: 7.90012\n", "\n", "# Model:ExtraTrees\n", "\n", "Fold 0 rmse: 7.93239\n", "Fold 1 rmse: 7.93247\n", "Fold 2 rmse: 7.92993\n", "Fold 3 rmse: 7.93121\n", "Fold 4 rmse: 7.93129\n", "Fold 5 rmse: 7.93247\n", "Fold 6 rmse: 7.93364\n", "\n", "Total rmse: 7.93191\n", "\n", "# Model:Poisson\n", "\n", "Fold 0 rmse: 7.89597\n", "Fold 1 rmse: 7.90240\n", "Fold 2 rmse: 7.90233\n", "Fold 3 rmse: 7.89682\n", "Fold 4 rmse: 7.89873\n", "Fold 5 rmse: 7.89241\n", "Fold 6 rmse: 7.89701\n", "\n", "Total rmse: 7.89795\n", "\n", "# ALL Mean ensemble rmse: 7.88061\n", "\n" ] } ], "source": [ "for (m_name, m) in models.items():\n", " print(f'# Model:{m_name}\\n')\n", " train[m_name + '_oof'] = 0\n", " test[m_name] = 0\n", " \n", " y_oof = np.zeros(train.shape[0])\n", " \n", " for f in range(N_SPLITS):\n", "\n", " train_df = train[train['kfold'] != f]\n", " valid_df = train[train['kfold'] == f]\n", " \n", " m.fit(train_df[features], train_df[target])\n", " \n", " oof_preds = m.predict(valid_df[features])\n", " y_oof[valid_df.index] = oof_preds\n", " print(f'Fold {f} rmse: {mean_squared_error(valid_df[target], oof_preds, squared = False):0.5f}')\n", " \n", " test[m_name] += m.predict(test[features]) / N_SPLITS\n", " \n", " train[m_name + '_oof'] = y_oof\n", " \n", " print(f\"\\nTotal rmse: {mean_squared_error(train[target], train[m_name + '_oof'], squared = False):0.5f}\\n\")\n", "\n", "\n", "oof_cols = [m_name + '_oof' for m_name in models.keys()]\n", "\n", "print(f\"# ALL Mean ensemble rmse: {mean_squared_error(train[target], train[oof_cols].mean(axis=1), squared = False):0.5f}\\n\") " ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": true, "jupyter": { "outputs_hidden": true, "source_hidden": true } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idf0f1f2f3f4f5f6f7f8...f97f98f99LinRegHGBXGBKNNBayesRidgeExtraTreesPoisson
02500000.81266515-1.239120-0.893251295.577015.8712023.043600.94225629.898000...0.5294701.3869508.787677.3452377.6021357.7057257.2557147.5861996.7895887.268488
12500010.190344131-0.5013610.80192164.88663.09703344.805000.80719438.421900...0.2485340.86388111.793905.3767165.2294854.8058006.1585715.6018246.7669835.630479
22500020.91967119-0.0573820.90141911961.200016.39650273.24000-0.00330037.940000...0.9317962.3368709.054007.6512497.4205698.5401917.1228577.6730416.7754517.524152
32500030.86098519-0.5495090.4717997501.60002.8069871.081700.7921360.395235...0.8933481.3594704.848336.6610257.3725557.5242256.9242866.7691126.8257746.652484
42500040.313229890.5885090.1677052931.26004.349861.571871.1183007.754630...0.3619231.5328003.706607.3810467.7050697.1299176.1428577.3491376.9008297.300795
..................................................................
1499953999950.751053770.666725-1.10628021433.300016.58120122.77900-0.319314-82.222900...0.3583541.4349908.554608.1704827.2467726.6613396.5242867.9603426.7926688.062418
1499963999960.734669410.6112250.740177294.185010.7290025.033400.644556-13.998000...0.5634821.87281012.549607.2085727.0492507.3535376.1942867.0974096.8169947.139650
1499973999970.417307142-0.357854-0.8366401215.250010.62460221.604000.875104-41.531500...0.2882362.32366012.338406.5542566.2234666.1154527.1871436.3568666.7710776.560769
1499983999981.023900170.367100-0.9366741832.39008.51691262.79700-0.4740027.162060...0.4538501.65314031.163806.3386625.7673234.3041145.8042866.4587866.8179026.392916
1499993999990.249362490.344670-0.5663632290.930020.80300350.644001.0185609.507420...0.3875681.5612805.696716.6156426.8817296.4250256.3057146.7473496.8193626.610858
\n", "

150000 rows × 108 columns

\n", "
" ], "text/plain": [ " id f0 f1 f2 f3 f4 f5 \\\n", "0 250000 0.812665 15 -1.239120 -0.893251 295.5770 15.87120 \n", "1 250001 0.190344 131 -0.501361 0.801921 64.8866 3.09703 \n", "2 250002 0.919671 19 -0.057382 0.901419 11961.2000 16.39650 \n", "3 250003 0.860985 19 -0.549509 0.471799 7501.6000 2.80698 \n", "4 250004 0.313229 89 0.588509 0.167705 2931.2600 4.34986 \n", "... ... ... ... ... ... ... ... \n", "149995 399995 0.751053 77 0.666725 -1.106280 21433.3000 16.58120 \n", "149996 399996 0.734669 41 0.611225 0.740177 294.1850 10.72900 \n", "149997 399997 0.417307 142 -0.357854 -0.836640 1215.2500 10.62460 \n", "149998 399998 1.023900 17 0.367100 -0.936674 1832.3900 8.51691 \n", "149999 399999 0.249362 49 0.344670 -0.566363 2290.9300 20.80300 \n", "\n", " f6 f7 f8 ... f97 f98 f99 \\\n", "0 23.04360 0.942256 29.898000 ... 0.529470 1.386950 8.78767 \n", "1 344.80500 0.807194 38.421900 ... 0.248534 0.863881 11.79390 \n", "2 273.24000 -0.003300 37.940000 ... 0.931796 2.336870 9.05400 \n", "3 71.08170 0.792136 0.395235 ... 0.893348 1.359470 4.84833 \n", "4 1.57187 1.118300 7.754630 ... 0.361923 1.532800 3.70660 \n", "... ... ... ... ... ... ... ... \n", "149995 122.77900 -0.319314 -82.222900 ... 0.358354 1.434990 8.55460 \n", "149996 25.03340 0.644556 -13.998000 ... 0.563482 1.872810 12.54960 \n", "149997 221.60400 0.875104 -41.531500 ... 0.288236 2.323660 12.33840 \n", "149998 262.79700 -0.474002 7.162060 ... 0.453850 1.653140 31.16380 \n", "149999 350.64400 1.018560 9.507420 ... 0.387568 1.561280 5.69671 \n", "\n", " LinReg HGB XGB KNN BayesRidge ExtraTrees \\\n", "0 7.345237 7.602135 7.705725 7.255714 7.586199 6.789588 \n", "1 5.376716 5.229485 4.805800 6.158571 5.601824 6.766983 \n", "2 7.651249 7.420569 8.540191 7.122857 7.673041 6.775451 \n", "3 6.661025 7.372555 7.524225 6.924286 6.769112 6.825774 \n", "4 7.381046 7.705069 7.129917 6.142857 7.349137 6.900829 \n", "... ... ... ... ... ... ... \n", "149995 8.170482 7.246772 6.661339 6.524286 7.960342 6.792668 \n", "149996 7.208572 7.049250 7.353537 6.194286 7.097409 6.816994 \n", "149997 6.554256 6.223466 6.115452 7.187143 6.356866 6.771077 \n", "149998 6.338662 5.767323 4.304114 5.804286 6.458786 6.817902 \n", "149999 6.615642 6.881729 6.425025 6.305714 6.747349 6.819362 \n", "\n", " Poisson \n", "0 7.268488 \n", "1 5.630479 \n", "2 7.524152 \n", "3 6.652484 \n", "4 7.300795 \n", "... ... \n", "149995 8.062418 \n", "149996 7.139650 \n", "149997 6.560769 \n", "149998 6.392916 \n", "149999 6.610858 \n", "\n", "[150000 rows x 108 columns]" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#hide\n", "\n", "test" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's a look at the correlation heatmap." ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAFCCAYAAACQFIg4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2k0lEQVR4nO3dedytc73/8debTRkyZThCyNkNEpuQShINyMlwjuSHJEPDIdVRR3SKOkWkUkftI5lKSoOSFJIhs83ebKRMFaUMYSf7xL3v9++P67tYlrXuYe9739e6lvfT43rc6/pe0+e+97I+6/u9vtf3K9tEREQ0zSJ1BxARETE/ksAiIqKRksAiIqKRksAiIqKRksAiIqKRksAiIqKRksAiImLMJJ0k6T5JN/XYLklflnS7pBslbdS2bRtJvynbDmkrX0HSBZJuKz+XH0ssSWARETEepwDbjLB9W2BqWfYHvgYgaVHg+LJ9XWA3SeuWYw4BLrQ9FbiwrI8qCSwiIsbM9qXAX0fYZQfgNFeuApaTtCqwKXC77TttPw58p+zbOubU8vpUYMexxJIEFhERE2k14O629XtKWa9ygFVs3wtQfq48lgtNWeBQY7488cCdfTeG19CF36w7hJ6mf+S2ukPoas+X3T36TjVY9pTpdYfQ09D5p46+Uw2mbLtP3SH0tNjKU7Ugx4/n82bxldZ5D1XTX8sJtk8Yx+W6xeoRyudbElhExKAbnjfmXUuyGk/C6nQPsEbb+urAn4DFe5QD/EXSqrbvLc2N943lQmlCjIgYdB4e+7LgzgbeWXojbgY8UpoFrwWmSlpb0uLAO8q+rWP2Kq/3An48lgulBhYRMeiGJyQxASDpDGBLYEVJ9wCfBBYDsD0dOBfYDrgdeAzYu2wbknQAcB6wKHCS7ZvLaY8CzpS0D/AHYJexxJIEFhEx4DwxNatyLu82ynYD/95j27lUCa6z/EFg6/HGkgQWETHo5g3VHcFCkQQWETHoxtGJo0mSwCIiBt0ENiH2kySwiIhBN4GdOPpJElhExICbyE4c/SQJLCJi0KUGFhERjTTvibojWCiSwCIiBt2ANiH2xVBSkh7tUvZeSe8c5bgtJT0iaaakWyV9fuFFOTpJz5H0C0mzJO1aZywREU8aHh770iB9WwMrQ5KMxa9sby9pCWCmpLNsX74wYxvBhsBitqfVdP2IiGdKDWxySTpc0sHl9cWSPifpGkm/lfS6zv1tzwVmUeaXkfRmSVdKul7S9yQtXcq3K7W1y8q01+eMEMMKkn5UpsW+StL6vcolrQx8C5hWamDrdDnf/pJmSJpx4mlnTMBfKSJiDFIDq90U25tK2o5q8Mg3tm+UtDzVFNaXSloR+DjwRtt/l/SfwIclHQ38L7CF7bvKoJQjOQKYaXtHSVsBpwHTupXbniZpX+Bg29t3O1n7NAX9OB9YRAwmDw9mJ46+rYF18cPy8zpgrbby10m6EfgzcI7tPwObAesCl0uaRTU8/5rAS4E7bd9Vjh0tgW0OfBPA9i+B50tadoTyiIj+kxpY7f5Rfs7j6XG37oG9GLhM0llUM39e0DlqsqQNx3nNSZtZNCJiock9sP5m+7fAkcB/AlcBr5X0zwCSliwJ7lbgRZLWKoeN1lPwUmD3co4tgQdszxmhPCKi/wzPG/vSIP1SA1uyTIzW8oX5PM904GBgaeBdwBmSnlO2fdz2byW9H/i5pAeAa0Y53+HAyaWJ8jGemjG0V3lERP8Z0BpYXyQw2yPWBG1v2fb6Aco9MNsXAxe3bZtL6YUI3AVs0uV0F9l+qSQBxwMzRrjuX4EdxlH+tHgiIvpCw+5tjVVfJLBJtp+kvYDFgZlUvRIjIgZXJrQcDLa/CHyxvUzS3sBBHbtebrvrtNgREY2SGtjgsn0ycHLdcURELAx2szpnjFUSWETEoEsNLCIiGim9ECMiopFSA4uIiEZKL8SIiGikNCHGRBq68Jt1h9DVlK33rDuErjaf9x91h9DVYw/25/9CS116Zt0h9OQ5/Tnq2tBl36s7hJ4W2/nQBTtBmhBj0PVr8oqIBZQEFhERjZQmxIiIaKR04oiIiEZKE2JERDTSgDYhDsyElhER0cPw8NiXMZC0jaTfSLpd0iFdti8v6SxJN0q6RtJ6bdsOknSTpJslfbCt/HBJf5Q0qyzbjRZHElhExKCbwAQmaVGquRS3BdYFdpO0bsduhwKzbK8PvBM4rhy7HrAfsCmwAbC9pKltx33R9rSynDtaLElgERGDzh77MrpNgdtt32n7ceA7PHOC33WBC6tL+1ZgLUmrAC8DrrL9mO0h4BJgp/n9tZLAIiIG3dDQmBdJ+0ua0bbs33G21YC729bvKWXtbgB2BpC0KbAmsDpwE7CFpOdLWhLYDlij7bgDSrPjSZKWH+3XSgKLiBh0Hh7zYvsE2xu3LSd0nE3drtCxfhSwvKRZwIHATGDI9q+BzwEXAD+nSnStPv5fA9YBpgH3AseO9mulF2JExKCb2G709/D0WtPqwJ/ad7A9B9gbQJKAu8qC7W8A3yjbPlvOh+2/tI6X9HXgnNECSQ0sImLQTew9sGuBqZLWlrQ48A7g7PYdJC1XtgHsC1xakhqSVi4/X0jVzHhGWV+17RQ7UTU3jig1sIiIQTeBNTDbQ5IOAM4DFgVOsn2zpPeW7dOpOmucJmkecAuwT9spfiDp+cATwL/bfqiUHy1pGlVz5O+A94wWSyMTmKRHbS/dtv4uYGPbB5T1PYCPUv1xh6i+MRxs+2FJFwOrAnOB51B12+xs412Q2HYBPgX82fYbJuq8ERHzbYJH4ihd3M/tKJve9vpKYGrncWXb63qUj3s08UYmsJFI2gb4ELCt7T+WZxb2AlYBHi677W57hqQVgDsknVK6g06EfYD3275ogs4XEbFAPG9e3SEsFIN4D+wwqtrWHwFsz7N9ku3fdNl3aeDvQM9/XUm7SZpdnhz/3Ejlkj4BbA5Ml3RMl3M92T31G7+YsWC/ZUTEWE3wSBz9oqk1sCVK98yWFXjqJuLLgetHOf50Sf+gquJ+0HbXBCbpBVRdPl8JPAScL2lH4Jpu5bY/JWkrqgT6jAxVmipPAJj73SPGdLc0ImKBZSzEvjK3bbiRacAnuu0k6RVlTK07JO3atmn3MsTJC4GDJa3Z4zqbABfbvr88NX46sMUI5RER/WfYY18apKkJbCQ3AxsB2J5dEtzPgCU6d7R9P1Vt7VU9ztXtgb2RyiMi+s+ANiEOYgI7Evi8pNXbyp6RvADKUCYbAnf0ONfVwOslrVg6g+xGNXZXr/KIiP4zb97YlwZp6j2wnmyfK2kl4GcluTxM9UDceW27nS6p1Y3+FNvX9TjXvZI+BlxEVes61/aPAXqVR0T0nYbVrMaqkQms/Rmwsn4KcErb+qnAqT2O3XKc1/o28O1xlI/r/BERC13D7m2NVSMTWEREjMOA9kJMAiskXU3VpNhuT9uz64gnImLCpAY22Gz36okYEdFozj2wiIhopIb1LhyrJLCIiEGXJsSIiGikNCFGREQjpQYWERGNlG70MZGmf+S2ukPo4hNsPu/vdQfR1bQbjq07hK6WeEHXuflq99mPrjr6TjW53A/XHUJXWzKn7hB6+tDOC3iC1MBi0PVr8oqIBeOh9EKMiIgmSg0sIiIaKffAIiKikVIDi4iIJnISWERENFI6cURERCOlBhYREY2UBBYREU1kJ4FFREQTpQYWERGNlAQWERFN5KE8yBwREU00mPmLReoOYH5IWkPSXZJWKOvLl/U1JU2VdI6kOyRdJ+kiSVuU/d4l6X5JsyTdLOn7kpacwLhWknS1pJmS+nOY8oh41vGwx7yMhaRtJP1G0u2SDumyfXlJZ0m6UdI1ktZr23aQpJvKZ/AH28pXkHSBpNvKz+VHi6ORCcz23cDXgKNK0VHACcBfgJ8CJ9hex/YrgQOBF7Ud/l3b02y/HHgc2HUCQ9sauNX2hrZ/NYHnjYiYf8Me+zIKSYsCxwPbAusCu0lat2O3Q4FZttcH3gkcV45dD9gP2BTYANhe0tRyzCHAhbanAheW9RE1MoEVXwQ2Kxl8c+BYYHfgSttnt3ayfZPtUzoPljQFWAp4qNcFSo3uwvIt4kJJL+xVLmkacDSwXanhLTFxv2pExAIYHscyuk2B223faftx4DvADh37rEuVhLB9K7CWpFWAlwFX2X7M9hBwCbBTOWYH4NTy+lRgx9ECaWwCs/0E8BGqRPbB8od8OXD9KIfuKmkW8EdgBeAnI+z7P8Bp5VvE6cCXe5XbngV8gqdqeHM7TyZpf0kzJM248tF+nNAyIgbRBDchrgbc3bZ+TylrdwOwM4CkTYE1gdWBm4AtJD2/3L7ZDlijHLOK7XsBys+VRwuksQms2Ba4F1iv28bSBnuTpB+2FX/X9jTgn4DZVEmwl1cD3y6vv0lV0xupfES2T7C9se2NX7301NEPiIiYAB7ymJf2L9pl2b/jdOp2iY71o4DlS2XhQGAmMGT718DngAuAn1MluqH5/b0am8BKk92bgM2AD0laFbgZ2Ki1j+2dgHdR1bSextWj6T8BthjHZXt9PRnMhywiYjCMowmx/Yt2WU7oONs9PFVrgqpm9af2HWzPsb13qSy8E1gJuKts+4btjWxvAfwVaDVH/aV8jlN+3jfar9XIBCZJVJ04Pmj7D8AxwOepakWvlfS2tt1H6mW4OXDHCNuvAN5RXu8OXDZKeURE3/Hw2JcxuBaYKmltSYtTfRae3b6DpOXKNoB9gUttzynbVi4/X0jVzHhG2e9sYK/yei/gx6MF0tTnwPYD/mD7grL+Vaqa1qbA9sAXJH2Jqlfi34D/bjt2V0mbUyXve8pxvXwAOEnSR4D7gb1HKY+I6D8T+ByY7SFJBwDnAYsCJ9m+WdJ7y/bpVJ01TpM0D7gF2KftFD+Q9HzgCeDfbbc60h0FnClpH+APwC6jxdLIBFaqtCe0rc8DXtm2y3Y9jjsFOGUc1/kdsNU4ysd1/oiIyTDGmtXYz2efC5zbUTa97fWVQNcb/ba7PiNr+0GqR5HGrJEJLCIixs7z3U2ivyWBAZIO45nV1e/Z/kwd8URETKSJroH1iyQwoCSqJKuIGEhJYBER0Uzu9uhW8yWBRUQMuNTAIiKikTycGlhERDTQ8LwksIiIaKA0IUZERCOlCTEm1J4vu3v0nSbZYw/279thiRf05wTXc//Un/OWPvq+d9cdQk8H7tif/5aPn39F3SEsNB7Q4cb79xMrIiImRGpgERHRSOnEERERjZQaWERENJIzEkdERDRRutFHREQjDacGFhERTZQmxIiIaKT0QoyIiEZKL8SIiGik3AOLiIhGyj2wiIhopEEdC3GRugOYX5IebXu9naTbJL1Q0uGSHpO0co99LenYtvWDJR0+gXGtJOlqSTMl9eeopRHxrDJsjXlpksYmsBZJWwNfAbax/YdS/ADwHz0O+Qews6QVF1JIWwO32t7Qdn8OVR4RzyrDwxrz0iSNTmClhvN14K2272jbdBKwq6QVuhw2BJwAfGiM11hT0oWSbiw/X9irXNI04GhgO0mzJC2xIL9fRMRESA2s/zwH+DGwo+1bO7Y9SpXEDupx7PHA7pKWHcN1/gc4zfb6wOnAl3uV254FfAL4ru1ptue2n0jS/pJmSJpx2j33juHSERELztaYlyZpcgJ7ArgC2KfH9i8De0lapnOD7TnAacAHxnCdVwPfLq+/CWw+SnlPtk+wvbHtjd+5+qpjuHRExIJLDaz/DANvBzaRdGjnRtsPUyWY9/c4/ktUyW+pcV63V3+eAe3nExFN53EsTdLkBIbtx4DtqZoDu9XEvgC8hy6PC9j+K3AmvWtwLVcA7yivdwcuG6U8IqKvzBteZMxLkzQr2i5KItoG+LikHTq2PQCcRXW/rJtjgdF6I34A2FvSjcCePHVfrVd5RERfGR7H0iSNfZDZ9tJtr+8G1i6rP+7Y78PAh3sc9xdgyVGu8ztgq3GUnwKcMuovEBExSczE3tuStA1wHLAocKLtozq2L0/VkW4d4P+Ad9u+qWz7ELAvVYvlbGBv2/9XnsfdD7i/nOZQ2+eOFEfja2ARETGyYY99GY2kRal6cm8LrAvsJmndjt0OBWaVXtrvpEp2SFqNqvVqY9vrUSXAd7Qd98XSg3vaaMkLksCeJOmw8uxW+3JY3XFFRCyoYTTmZQw2BW63faftx4HvADt07LMucCFAecxpLUmrlG1TgCUkTaFqAfvT/P5eSWCF7c+0Zf7W8pm644qIWFBGY17GYDXg7rb1e0pZuxuAnQEkbQqsCaxu+4/A54E/APcCj9g+v+24A8rgECeVZsgRJYFFRAy4eWjMS/uAC2XZv+N03bJcZ+PjUcDykmYBBwIzgaGSlHag6rPwAmApSXuUY75Gdc9sGlVyO5ZRNLYTR0REjM14ehfaPoFquL1e7gHWaFtfnY5mwDJYxN4AkgTcVZa3AHfZvr9s+yHwGuBbpVMdpfzrwDmjxZoaWETEgJvgbvTXAlMlrS1pcapOGGe37yBpubINqh6Hl5ak9gdgM0lLlsS2NfDrckz78EQ7ATeNFkhqYBERA24iu9HbHpJ0AHAeVS/Ck2zfLOm9Zft04GXAaZLmAbdQBoywfbWk7wPXUw2sPpOnantHlwHRDfyOahCKESWBRUQMuImeJaV0cT+3o2x62+srgak9jv0k8Mku5XuON44ksIiIATfG7vGNkwRWk2VPmT76TpNsqUvPrDuEnj770f4cvf/R97277hC6WvprJ9UdQk9Dv/xW3SF0teTRX607hIVmXt0BLCRJYBERA25YqYFFREQDNW2alLFKAouIGHBNG2V+rJLAIiIG3ET3QuwXSWAREQNuXnohRkREE6UGFhERjZR7YBER0UjphRgREY2UJsSIiGikNCFGREQjzUsNLCIimmhQa2CjTmgpaZ6kWZJukHS9pNdMRmCSLpb0m3Lda8s8Ma1t50parssxh0s6eDLi60XSMZJulnRMnXFERLRM8ISWfWMsNbC5tqcBSHoLcCTw+oUZVJvdbc+QtDdwDPAmANvbTdL158d7gJVs/6PuQCIiYHB7IY5aA+uwDPAQgKSlJV1YamWzJe1Qyj8t6aDWAZI+I+kD5fVHSm3qRklHlLKlJP201LRukrRrl+teCazWds7fSVqxvD6s1NR+AbykbZ9NynWuLLWim0r5omW9FUfPWT9VOabENbsV2wjlZwNLAVf3+D0iIibdsMa+NMlYamBLSJoFPBdYFdiqlP8fsJPtOSWZXFU+wL8B/BA4TtIiwDuATSW9mWqGzk0BAWdL2gJYCfiT7bcCSFq2SwzbAD/qLJT0ynL+Dcvvcj1wXdl8MrC/7SskHdV22D7AI7Y3kfQc4HJJ59u+q8t1dwamARsAKwLXSroUeE23cttvk/Roq8baJd79gf0Bvnr04ey7x9u77RYRMaGa1jQ4VuNtQnw1cJqk9aiS0GdLEhqmqiGtYvt3kh6UtCGwCjDT9oMlgb0ZmFnOuzRVQvsV8HlJnwPOsf2rtmufLmkpYFFgoy6xvQ44y/ZjJb6zy8/lgOfZvqLs921g+/L6zcD6kv6trC9b4uiWwDYHzrA9D/iLpEuATUYoP3ukP6TtE4ATAJ6499eDWquPiD6TCS0B21eW2tZKwHbl5yttPyHpd1S1NIATgXcB/wS0poYVcKTt/+08b6lJbQccWWpDnyqbdgduAI4CjqeqET0jrC5lI1WEBRxo+7wR9hntPA2raEfEs1nTmgbHalz3wCS9lKo29CBVzeW+krzeAKzZtutZVM1+mwCtRHEe8G5JS5dzrSZpZUkvAB6z/S3g83TUtGw/AXwc2EzSyzpCuhTYSdISkp4H/Es55iHgb5I2K/u9o+2Y84D3SVqsxPHiUsvr5lJg13LfbCVgC+CaEcojIvrOs7kXYuseGFQ1j71sz5N0OvATSTOAWcCtrQNsPy7pIuDh0syG7fNLArpS1fTWjwJ7AP8MHCNpGHgCeF9nALbnSjoWOJjqHlar/HpJ3y3X/z1Vc2TLPsDXJf0duBh4pJSfCKwFXK8qkPuBHXv87mcBr6aqBRr4qO0/S+pa3uMcERG1GtT7FaMmMNuL9ih/gOpD/BlK543NgF06jjkOOK5j9zt4qpbWvu+WHevHtr1eq+31Z4DPdAnjZtvrl3gOAWaU/YeBQ8syItsGPlKWUcvLtqVHO29ExGQaHtAUNuEjcUhaFziHqnPFbRN9/nF4q6SPUf2Ov6e6JxcR8ayTThxjZPsW4EUTfd75iOO7wHfHsq+kVwDf7Cj+h+1XTXhgERGTrGn3tsYqYyECtmdTPdcVETFwBrUXYhJYRMSAyz2wiIhopMFMX0lgEREDL/fAIiKikeYNaB0sCSwiYsANag1svNOpREREwwzjMS9jIWmbMo3V7WWgiM7ty0s6q0xZdU0ZAL617UOqJv29SdIZkp5byleQdIGk28rP5UeLIzWwmgydf2rdITyD58ypO4SeLvfDdYfQ1YE7vq7uELoa+uW36g6hpylb7VF3CF0NXfWjukPoabHtXzL6TiOYyAZESYtSDa7+JuAequmkzi7PALccCsyyvVMZQ/d4YGtJqwEfANYtQwSeSTVW7SnAIcCFto8qSfEQ4D9HiiU1sIiIATfBg/luCtxu+07bjwPfAXbo2Gdd4EIA27cCa0lapWybQjXG7hRgSeBPpXwHoPXN/lR6j1H7pCSwiIgBNw+PeZG0v6QZbcv+HadbDbi7bf2eUtbuBsr0V5I2pZqtZHXbf6SadeQPwL1UkwufX45Zxfa9AOXnyqP9XmlCjIgYcON5kLl94t0euo3r0XmBo4Djykwms6kmMh4q97V2ANYGHga+J2mPMp3WuCWBRUQMuAnuRH8PsEbb+uo81QxYXc+eA+wNUKatuqssbwHusn1/2fZD4DXAt6hmt1/V9r2SVgXuGy2QNCFGRAy4Ce6FeC0wVdLakhan6oRxdvsOkpYr2wD2BS4tSe0PVJMTL1kS29bAr8t+ZwN7ldd7AT8eLZDUwCIiBtxEPgdme0jSAVTzOC4KnGT7ZknvLdunAy8DTpM0D7iFMhGx7aslfR+4HhiialpsNVceBZwpaR+qRPe0+SS7SQKLiBhwnuBGRNvnAud2lE1ve30lMLXHsZ8EPtml/EGqGtmYJYFFRAy4DCUVERGNNKhDSSWBRUQMuGGnBhYREQ00mOkrCSwiYuAN6ozMC/QcmKR5kma1Lc8Ylbhj/0Pn4xpnlXPfLumRtmu9Zv4jX3gk7SLp15IuqjuWiAioeiGO9b8mWdAa2Fzb08ax/6HAZzsLywNtsv2Me422dyr7bAkcbHv7jmOn2B4aRwwL2z7A+20ngUVEXxhqWGIaqwkfiUPSsmWemJeU9TMk7SfpKKoRiGdJOl3SWqWm8lWqh9rWkPS1MnjkzZKOGOEa75L0PUk/Ac6XtJSkkyRdK2mmpB3KfotKOqaU3yjpPaV8VUmXllhuktRzTgxJu0maXfb73Ejlkj4BbA5Ml3TMgv81IyIWXGpg3S1RBmtsOdL2d8tT2qdIOg5Y3vbXASQd0KqxSVoLeAmwt+33l7LDbP+1zDdzoaT1bd/Y49qvBtYv+38W+KXtd0taDrhG0i+A3alGO95E0nOAyyWdTzVK8nm2P1OutWS3C0h6AfA54JXAQ1TJckfgmm7ltj8laSuqmuKMLufbH9gf4Cvvegv7bDltxD9uRMRESDf67ro2Idq+QNIuVJOYbTDC8b+3fVXb+tvLh/wUYFWqOWV6JbALbP+1vH4z8DZJB5f15wIvLOXrS/q3Ur4s1dPh1wInSVoM+JHtWT2usQlwcdvAk6cDW1B16ulW/qMRftenjfI899RDmvVVJyIay+lGP3aSFqEaC2susALV6MXd/L3tmLWBg4FNbD8k6RSqRNTL39teC/hX27/piEPAgbbP6xLjFsBbgW9KOsb2ad1+lR7X7lUeEdF30gtxfD5ENcLwbjxV0wF4ou11p2WoktIjqmbu3HYc1zsPOLAkLCRt2Fb+vtY1Jb243C9bE7ivNG1+A9iox3mvBl4vacXS1LgbcMkI5RERfWc8E1o2yUTfA/s5cBLV8Pmb2v6bpEuBj1MN3ngCcKOk64HD2k9k+wZJM4GbgTuBy8cRx6eBL5VzC/gdsD1wIrAWcH0pv59qmuotgY9IegJ4FHhnt5OWeWk+BlxEVes61/aPAXqVR0T0m0GtgWlQ20b7XT/eA/OcOXWH0NNux/yh7hC6+s6RG46+Ux0W69XQUb8pW+1RdwhdDV31o7pD6GmJ7T+8QLcttl1j2zF/3vzs7p815hZJRuKIiBhw6YU44CRdDTyno3hP27PriCciYqI07fmusUoCK2y/qu4YIiIWhkG9B5YEFhEx4OY9c5S+gZAEFhEx4NKEGBERjZQJLSMiopEGM30lgUVEDLx04oiIiEZKAosJNWXbfeoO4RmGLvte3SH0tCX9OUrI4+dfUXcIXS159FfrDqGnfh3xYspmO9YdwkKTXogREdFI6YUYERGNNKhj3iaBRUQMuNwDi4iIRkoNLCIiGmnegI5HnwQWETHgBnUkjkXqDiAiIhYuj+O/sZC0jaTfSLpd0iFdti8v6SxJN0q6RtJ6pfwlkma1LXMkfbBsO1zSH9u2bTdaHKmBRUQMuImsgUlaFDgeeBNwD3CtpLNt39K226HALNs7SXpp2X9r278BprWd54/AWW3HfdH258caS2pgEREDboJrYJsCt9u+0/bjwHeAHTr2WRe4EMD2rcBaklbp2Gdr4A7bv5/f3ysJLCJiwA3bY17GYDXg7rb1e0pZuxuAnQEkbQqsCazesc87gDM6yg4ozY4nSVp+tECSwCIiBtw8D495kbS/pBlty/4dp1OXS3RmvqOA5SXNAg4EZgJDT55AWhx4G9A+ft3XgHWomhjvBY4d7ffKPbCIiAE3nqGkbJ8AnDDCLvcAa7Strw78qeMcc4C9ASQJuKssLdsC19v+S9sxT76W9HXgnNFira0GJmle6Wlyk6TvSVpyhH3f1q2nSz+S9NLye82UtE7d8URE2MNjXsbgWmCqpLVLTeodwNntO0harmwD2Be4tCS1lt3oaD6UtGrb6k7ATaMFUmcT4lzb02yvBzwOvLfXjrbPtn3U5IW2QHYEfmx7Q9t31B1MRMQwHvMyGttDwAHAecCvgTNt3yzpvZJan+MvA26WdCtVbeug1vGlsvIm4Icdpz5a0mxJNwJvAD40Wiz90oT4K2B9SSsAJwEvAh4D9rd9o6R3ARvbPkDSLsAngXnAI7a3kPRy4GRgcaqk/K+2b5P0YeDd5Ron2v6SpLWAnwGXAa+h6sa5g+253QKTNA2YDiwJ3AG82/ZD3cqBVwMfBOZJ2sL2GybqDxQRMb8meigp2+cC53aUTW97fSUwtcexjwHP71K+53jjqL0Th6QpVBl6NnAEMNP2+lTPEZzW5ZBPAG+xvQHVTUCoam/H2Z4GbAzcI+mVVG2wrwI2A/aTtGHZfypwvO2XAw8D/zpCiKcB/1limk2VPLuWl3/U6VTPMjwjebXfHD3xtO+M8peJiJgYE1kD6yd11sCWKD1UoKqBfQO4mpJMbP9S0vMlLdtx3OXAKZLO5Kkq6JXAYZJWB35Yal+bA2fZ/juApB8Cr6Nqq73Lduva1wFrdQuwXHs525eUolOB7/UqH+0Xbr85+sR9tzXrnRIRjTVvOGMhTrS5pcb0pNJbpdPTPuhtv1fSq4C3ArMkTbP9bUlXl7LzJO1L966eLf9oez0PWGJ+foGIiCYY1Akta29C7HApsDuApC2BBzp6riBpHdtX2/4E8ACwhqQXAXfa/jJVDWv9cq4dJS0paSmqXi2/Gk8wth8BHpL0ulK0J3BJr/Jx/7YREZPA9piXJumXThwthwMnl14ojwF7ddnnGElTqWpYF1I98X0IsIekJ4A/A5+y/VdJpwDXlONOtD2zdOIYj72A6aXnzJ2UZxtGKI+I6CtNu7c1Vmpaxh0U/XgPbOiyUW/j1Wb6B28ZfacavPsN99YdQldLHv3VukPoaejan9YdQldTNtux7hB6WmzFF410S2RUKy7z4jF/3jww57cLdK3J1G81sIiImGDpxDHgJB0PvLaj+DjbJ9cRT0TERBnUJsQksML2v9cdQ0TEwjCot4qSwCIiBtxETmjZT5LAIiIG3KA+B5YEFhEx4FIDi4iIRhoe2zQpjZMEFhEx4NKJIyIiGmlQE1hG4hgAkvYvI933lcQ1fv0aW+Ian36Na9D022C+MX/2rzuAHhLX+PVrbIlrfPo1roGSBBYREY2UBBYREY2UBDYY+rWtPXGNX7/GlrjGp1/jGijpxBEREY2UGlhERDRSElhERDRSElhERDRSEljDSNql/Fy77lg6Sbqw/Pxc3bG0SHpt+fmcumPp1K+xtf79Wu+1ftKP7zHoz/8fnw2SwJrnY+XnD2qNortVJb0eeJukDSVt1L7UFNOXy88ra7r+SPo1tu0kLcZT77V+0o/vMYDvw1MJNiZHxkJsngclXQSsLenszo2231ZDTC2fAA4BVge+0LHNwFaTHhE8IelkYDVJX+7caPsDNcTU0q+x/Rx4AFhK0hxAVP9+qsLyMjXFBf35HgNYRNIngRdL+nDnRtudscYESAJrnrcCGwHfBI6tOZansf194PuS/sv2p+uOp9geeCPVB9t1NcfSqS9js/0R4COSfmx7h7rjaden7zGAdwA7Un2mPq/eUJ498hxYQ0layfb9kp5H9a340bpjaifpbcAWZfVi2+fUHM8Gtm+oM4Ze+jy2VYBNyurVtu+vM552/fYeA5C0re2f1R3Hs0XugTXXKpJmAjcBt0i6TtJ6dQcFIOlI4CDglrIcVMrq9KCksyTdJ+kvkn4gafWaY2rpy9hKJ45rgF2AtwPXSPq3eqOq9Ol7DOAKSV+QNKMsx0patu6gBlVqYA0l6QrgMNsXlfUtgc/afk2dcQFIuhGYZlfTwEpaFJhpe/0aY7oA+DZV0yvAHsDutt9UV0wt/RqbpBuAN9m+r6yvBPzC9gZ1xlVi6bv3WInjB1RfKk8tRXsCG9jeub6oBldqYM21VCt5Adi+GFiqvnCeYbm21/3wDXRl2yfbHirLKcBKdQdV9Gtsi7SSV/Eg/fWZsVzb6354jwGsY/uTtu8syxHAi+oOalClE0dz3Snpv3j6t/a7aoyn3ZHAzNJbUlT3Kerukn2/pD2AM8r6blQfyP2gX2P7uaTzeCquXYFza4ynXT++xwDmStrc9mXw5LN+c2uOaWClCbGhJC0PHAFsTvU/8KXA4bYfqjWwQtKqVDf/RXXz/881x/NC4H+AV5eiy4GDbP++vqgqfR7bzrS9x2yfVXNIT+q39xiApGlUzYfLUsX1V2Av2zfWGdegSgJrOEnLAMN93gvxEts/qTOemD+lF+KmVM9YXdPRpFirfn6Plf8vsT2n7lgGWT+1Z8c4SHpF6YU4G7i5z3ohHsXTe4h9oO4eYpJW78eeftC/sUl6O1UvxH+j6oV4dR/1Quy79xiApGUlfQH4JfDL9EJcuFIDa6j0Qhx3TH3Z0w/6N7b0QpyvuNILcRKlBtZc6YU4Piv1aU8/6N/Y0gtx/NILcRKlF2JzpRfi+DzQpz39oH9jSy/E8UsvxEmUJsSG6uiFCFUvxCOa0AtR0stt3zzJ8bT39DNwBf3T06+fY2tkL8Q63mPluhsAp/FUjfAh0gtxoUkCG1CSvmL7wLrj6EbS9bbrnPriGSR9zHbtnQC66dfYJF1p+9Wj7zn56n6P9eqFKGkv26d2PyrGq5/as2NivbbuAEagugPoou8mb2zTr7E9t+4ARlDre8z2nB5d6A+a9GAGWBJY1KEfq/39mFRb+jW2fvx3bOnX2Pr137KRksAiKv36gQf9HVuMT/4tJ1AS2ODq5296j9cdQBf9/Pfq19j6NS7oz/cY9PffrHGSwAbXcXVcVNIUSSqv15D0b5I2bN/H9mZ1xDaK79UdwAhqi03SmpLeWF4voWoC1ZY9awoLSa+VtFR5vUeZg2vN1vY+fY9BNc5lTJD0QmwoST/hmc0RjwAzgP+1/X81xLQf8DngUeDTwEeA64ENgZNsf66GmJ5L9fzSQ8BPgI8CrwPuAD5t+4HJjqkttk+MsNm2Pz1pwXRR/j33B1awvY6kqcB021vXGRc8ORLHBsD6VM9CfgPY2fbra47rOcC/AmvR9pyt7U/VFdMgSwJrKEnHUY3W0P6Q6Z+BJYBlbE/6t2NJN1M9M/Q84NfAmrYfkLQkcK3tl9cQ05nAE1SjlCxPNczPT0qc02xvP9kxtcX2H12KlwT2BZ5ve+lJDulpJM2iGsj3atsblrLZtl9RZ1wljuttb1S+BPzR9jfq7jpf4vo51RfJ64B5rXLbx9YW1ADLSBzNtaHtLdrWfyLpUttblERSh8fLg9QPSbq9Vbux/Zikuu5JrGt7PUlTgHvavqH/vIz1V5v2D7XSNHcQ8G7gO0A/fOD9w/bjpUWY8jfsl2+8f5P0MapmzNeVsRAXqzkmgNVtb1N3EM8WuQfWXCuVERyAJ0dzWLGs1pUslpC0oaRXAouX1xuV9bqeGXocwPYQ8KeObfOeufvkkrSCpP8GbqT6QrmR7f/sk2lLLpF0KNW/65uo7sX1y5QluwL/AN5dRuBYDTim3pAAuEJS7TXUZ4s0ITaUpO2A6VT3cgSsDbwfuBjYz/aXaojpopG2237DZMXSIuk+qhqNqD70vtPaBLzd9iqTHVNbbMcAOwMnAMf34ZxuiwD7AG+m+nudB5zoPvnQKJ02ptr+RWmmXtT232qO6Rbgn6nGJf0H1d/NdY+SP6iSwBqs3DB+KdX/JLfW0XGj30naa6TtdQ7rI2mY6kNuiKc3zbU+9JapJbAG6NcOJu09Idv1w7iWgygJrKHKN84PU3WU2K/8D/wS2+fUGNMywCq2byvru1B1KgE4z/Zf6ootxk/SbHr3dP1v27WNmN/nHUw2oOrpCvAr27Xeax1kuQfWXCdT3d9pDaZ6D/Df9YUDwOd5+hiMR1KNFr4F1cj5k07S5pLe2bb+fUm/LMtWdcTUID8DfgrsXpafUM168GfglPrCAkoHk9ZKv3QwkXQQcDqwclm+JakvB9UeBKmBNZSkGbY3ljSz7RvoDXXOlitpJlUnBLfW22K7zPbmI55g4cR0IXCg7VvK+mzgXVTd6g+ts8eYpL9Rfei2j85gqs4ci9uutZewpMttv7ZbWd21HUlHAw8D7wQOpLr/e4vtw+qKqcR1I/Bq238v60sBV+Ye2MKRGlhzPS5pCcq3TknrUN1PqdOUjhv87c+iLTfJsbQs00pexW22r7N9KdXzarWx/Tzby5SfzwNeAHyGqoZTy0gqHZaW9KrWiqRNgdazaUP1hPSkQ4D7gdnAe6gm2vx4rRFVxNN7t84jw0ctNHkOrLk+CfwcWEPS6VRNd++qNSIYlvRPrYkFbd8EIGk1YLimmJZrX7G9c9tqbT0Q20laDvggVW3i28Amdd5farMvcJKkpak+hOcA+5ZaRa3zk9kelvQtqkk2f1NnLB1OBq6WdBbV32wHqlFCYiFIE2KDSXo+sBnV/yhXAUvV2dtJ0h5UD+P+BzCzFG9EdW/sy7a/WUNMP6HqnfbTjvLtgffZfutkx9QWw4pUf6tdgZOAr9h+pK54epG0LNVnxcN1x9Ii6W1Uz30tbnttSdOAT9l+W72RgaSNePos1jNHOSTmUxJYA0l6NdWDm5favk/S+lRNKq+zvUbNsW0DHAq0ho26CTjK9s9qimcqcA5wBdW4jACvBF4DbG/7t3XEVWL7O1Uz2MnAM55fsv2FSQ8KkPThkbbXFVc7SdcBWwEXt91nvbHue02lKf8e2/+QtCXVWI2n9VPyHyS5B9Yw5eHXk6gGDP2ppE8CFwBXA1PrjA3A9s9tb2H7+WV5fV3Jq/g/qg+RX1ENsLoWVU+69am/CfEYquQF1f249qXOcRBbMWwMvI/qy9JqwHuBdWuMq91QP9ZWgR8A8yT9M3Ai1QAD3643pMGVe2DN81aqcRD/T9LyVMMjrd969qpOkr7CCF2ZbX9gEsNpuYRqxJIvlOGkkLQK1YfLS6i6+dflRNv3dNsg6V8mO5gW20eUGM6n6lX6t7J+OP0z7cxNkv4fsGipZX+AqpZdt2HbQ5J2Bo6z/ZXSOzcWgtTAmmdua8SNMnDub/oheRUzqEbhvg54W9vr1lKHVwLrADMlbVWe07kGuBJ41YhHLnwXSlqrs1DS3sCXJj2aZ3ohTx9X83GqGmw/OJCqmfofVDWcR6g6wtTtCUm7UXXIaQ0q0A+DDA+k3ANrGEkPUzWBtWzRvt4PN7Hh6c+A9YOSuL5IVWPdrFfNZzKV8SyPA7ZrG73kY8D/A7atO0ZJhwFvB86iqlnvBJxp+7M1x7Uo1cgub6wzjm4krUvV1Hql7TMkrQ3savuomkMbSElgDSNpxAn7bF8yWbGMRH0wN1OJYzmqSTZfRTWZ5XbA1sBBtn9ZY2gASNoa+F9gR6pu65tQdS55qM64WkqPutawSH3To07S2cCefXofDIDSxL+G7RvrjmVQJYHFQtFHCexO4KvAl9rugU0rZb+3vVuN4QHVcFfAj6ju4by97kGZJS1je46kFbptt/3XyY6pRdJmtq9SNVHpZlQdmP7eFlsd91mfJOliqubzKcAsql6ml9gesWdnzJ8ksIaS9FrgcGBNqv9ZWiOYv6jGmFpDI0E1s/BjrU3UNLq6pNVH6Cixn+2vT3ZMbddvH0rqOVQzR7dGbqhtNHpJ59jeXtJddB8lv873WGsm5q6zDLjG2QXgqaZzSftS1b4+2Q/d+wdVElhDSboV+BDPnLq8H0ZwiFgo+qVm30sZa/PNwKnAYbavTQJbeNKNvrkeqfn5qniWkfQS4GDb+9UYxovK/a+u+qAT06eoJv68rCSvFwH90kt44KQG1lCSjgIWBX5I2yC+tq/veVDEGJSRXT5PNbjwj4CvUN0zfBVwrO0v1hjbbVSdXbrql05MMTlSA2uu1jNMG7eVmWp4nYgF8XXga1TPym1DNQTXt4Hd6+5gAvytH5OUpI/aPrrXw/x1dy4ZVElgDWX7DXXHEAPrObZPKa9/I+lg4BDb80Y4ZrL8ru4Aevh1+Tmj1iieZdKE2DCS9rD9rV4DrvbDQKvRbKWD0G48NY/V6VQPVwv6p5la0npUYzM+t1Vm+7T6InqKpOdR9dh8tO5YBllqYM2zVPnZbTLGfBuJiXAv0P5F6M9t633RTF0Gsd6SKoGdC2wLXAbUmsBKUv0msEK1qvuBd9q+uc64BlVqYANE0gdtf6nuOCIWttJdfQNgpu0NWgM0265tEOQS1xVU3ecvKutbAp+1/Zo64xpUGcx3sORp/5gwknYpTWFI+rikH0rql/Et59oeBoYkLQPcB9T2gHWbpVrJC8D2xTzVahITLAlssGj0XSLG7L9s/60MdfUWqodzp9ccU8uMMs7l16ke5r+eapaBut0p6b8krVWWjwN31R3UoEoT4gCR9AfbL6w7jhgMbcMiHQnMtv3tfphlQJKA1W3fXdbXApbph0FzywC+RwCbl6JLgSP6ZXDmQZME1jAd4w0+bROwhO10zIkJIekc4I/AG6nmVZsLXGN7g1oDAyRdZ/uVdcfRIum5VNOo/DMwGzjJ9hP1RjX4ksAioitJS1I9yDzb9m2SVgVeYfv8mkND0vHAKbavrTsWAEnfpRqM+VdUPSJ/Z/uDtQb1LJAEFhE9lftfU22fLGklYGnbtd/TkXQL8GLg91TTqbRGyq9l0FxJs22/oryeQlVT7dtBhwdFmpsioqvyrNXGwEuAk4HFgG8Br60zrmLbugPo8GRzoe2h6jZdLGypgUVEV5JmARsC17c6bvTL1CCSvml7z9HKJjGeeTw1saaAJajmw6t1brdBlxpYRPTyuG1LMoCkfnqe6eXtK5IWpepoUgvbi9Z17WezPAcWEb2cKel/geUk7Qf8guq5q9pI+ljpibu+pDll+RvVg8w95wmLwZQmxIjoSdKbqGYYFnCe7QtqDgkASUfa/ljdcUS90oQYEV1JOgA4vV+SVofb21dKE+LHbR9RUzxRgzQhRkQv/wRcK+lMSduov7rWbS3pXEmrSnoFcBXdZ2iIAZYmxIjoqSStNwN7U3WpPxP4hu07ag0MkLQrcDxVb7/dbF9ec0gxyVIDi4ieXH3D/XNZhoDlge9LOrrOuCRNBQ4CfkA1S/OeZeSQeBZJDSwiupL0AWAv4AHgROBHtp+QtAhwm+11aoztVuDfbV9YaokfBt5t++WjHBoDJAksIrqS9Cmq5sLfd9n2Mtu/riGs1vWXsT2no2yq7dvqiikmX5oQI6Ir25+w/XtJK0t6YWsp22pJXpI+Wq4/R9IuHZv3riGkqFESWER0JelfJN1GNSHjJVT3mn5Wa1DwjrbXnc+BbTOZgUT9ksAiopf/BjYDfmt7bWBroO6efurxutt6DLgksIjo5QnbDwKLSFrE9kXAtJpjco/X3dZjwGUkjojo5WFJSwOXAqdLuo+qK32dNpA0hzLie3lNWX9ufWFFHdILMSK6KqPPz6VqqdkdWJZqaKkHaw0sokgCi4hRSVoReND5wIg+kntgEfE0kjaTdLGkH0raUNJNwE3AXySlp1/0jdTAIuJpJM0ADqVqMjwB2Nb2VZJeCpzRmp05om6pgUVEpym2z7f9PeDPtq8CsH1rzXFFPE0SWER0Gm57PbdjW5psom+kCTEinkbSPODvlK7qVNOVUNafa3uxumKLaJcEFhERjZQmxIiIaKQksIiIaKQksIiIaKQksIiIaKQksIiIaKT/D+jR/X0LDUPwAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "\n", "oof_cols = [m_name + '_oof' for m_name in models.keys()]\n", "\n", "oofs = train[oof_cols]\n", "\n", "oof_diffs = oofs.copy()\n", "for c in oof_cols:\n", " oof_diffs[c] = oofs[c]-train[target]\n", " oof_diffs[c] = oof_diffs[c]#**2\n", "\n", "sns.heatmap(oof_diffs.corr())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "XGB and KNN are most diverse, so I export a 50:50 ensemble. I'll also export an equally weighted ensemble of all models and HGB only because it is the best single model." ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CV: ALL equaly weighted: 7.880605075536334\n", "CV: XGB only: 7.916746570344035\n", "CV: HGB only: 7.865625158180185\n", "CV: XGB and LinReg (50:50): 7.872064005057903\n", "CV: XGB and KNN (50:50): 7.893210466099108\n" ] } ], "source": [ "#hide_input\n", "\n", "# Do some manual equaly wheighted ensemblings and export.\n", "\n", "submission = pd.read_csv(PATH_INPUT + 'sample_submission.csv')\n", "submission['loss']=test[models.keys()].mean(axis=1)\n", "\n", "submission.to_csv('submission_ALL.csv',index=False)\n", "print('CV: ALL equaly weighted:',mean_squared_error(train[target], train[[m_name + '_oof' for m_name in models.keys()]].mean(axis=1), squared = False))\n", "\n", "submission['loss']=test[['XGB']].mean(axis=1)\n", "\n", "submission.to_csv('submission_xgb.csv',index=False)\n", "print('CV: XGB only:',mean_squared_error(train[target], train[['XGB_oof']].mean(axis=1), squared = False))\n", "\n", "submission['loss']=test[['HGB']].mean(axis=1)\n", "\n", "submission.to_csv('submission_hgb.csv',index=False)\n", "print('CV: HGB only:',mean_squared_error(train[target], train[['HGB_oof']].mean(axis=1), squared = False))\n", "\n", "submission['loss']=test[['LinReg', 'XGB']].mean(axis=1)\n", "\n", "submission.to_csv('submission_xgb_lin.csv',index=False)\n", "print('CV: XGB and LinReg (50:50):',mean_squared_error(train[target], train[['LinReg_oof', 'XGB_oof']].mean(axis=1), squared = False))\n", "\n", "submission['loss']=test[['KNN', 'XGB']].mean(axis=1)\n", "\n", "submission.to_csv('submission_xgb_knn.csv',index=False)\n", "print('CV: XGB and KNN (50:50):',mean_squared_error(train[target], train[['KNN_oof', 'XGB_oof']].mean(axis=1), squared = False))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we inspect the variance and mean of the residuals. Means are close to 0, as expected." ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(LinReg_oof 62.373163\n", " HGB_oof 61.868296\n", " XGB_oof 62.675125\n", " KNN_oof 63.678431\n", " BayesRidge_oof 62.412188\n", " ExtraTrees_oof 62.915511\n", " Poisson_oof 62.377910\n", " dtype: float64,\n", " LinReg_oof -0.000055\n", " HGB_oof -0.003314\n", " XGB_oof 0.001392\n", " KNN_oof -0.005395\n", " BayesRidge_oof -0.000024\n", " ExtraTrees_oof -0.000136\n", " Poisson_oof -0.000084\n", " dtype: float64)" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "oof_diffs.var(), oof_diffs.mean()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These are the histograms of the residuals:" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "data": { "text/plain": [ "array([[,\n", " ,\n", " ],\n", " [,\n", " ,\n", " ],\n", " [, ,\n", " ]], dtype=object)" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3oAAANeCAYAAABEbnxYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAACPlUlEQVR4nOz9fbxdZX3n/7/egiKKIKCcAqEGh2jl5ict+VJaZzqp0ZJWa5jvD2o6tIQOU6YOVdvSXwltZ7SdoQOdqVRqsaViCVSFTFoqFVEp9Hw7fgeCYKkUkCFKhJgU5EYkWtDg5/fHuk7ZOZycm+Tc7L3zej4e+3H2/qx1rf1Zm3i5Pmtd61qpKiRJkiRJw+MFC52AJEmSJGl2WehJkiRJ0pCx0JMkSZKkIWOhJ0mSJElDxkJPkiRJkoaMhZ4kSZIkDRkLPU0qyb9Kct9C5zEXkrwhyf1JtiU5ZaHzkSRJkmaLhZ7+WZJNSd7UG6uq/1VVr51m+2VJvtsKp6eS3Jfk5+Ym21nx28AHqmq/qvrLhU5GkgCSvDbJ37V+9F0LnY8kTSXJvkn+KsmTSf7nQuejjoWeZtuWqtoP2B/4ZeBPkkyrUFwArwLuXugkJM2NiU5eJTkzyWd7Pq9KsiHJN5M80t7/xyRpy69I8u2eE1h3JPnXc5z6rwGjVfWyqrpkjr9LUh9Jsl/ru/5tT+xlSR5Mcmr7vDTJJ5I8keTrSe5JckGSA9vyM5M82/qtbUm+nOQdc5z6qcAIcHBVnTbH36VpstDTpNpVus09nzcl+dUkX2hnba5J8uLx7arzSeBx4P/T2r4gyZokX0ryWJJ1SQ7q2fYZSb7Slv2niQ7SJshvnyS/n2RLe/1+kn16lv98ko1JHk9yXZLDWvxLwKuBv2qd4D47+w5JwynJucD7gf8OfA/dQcovAG8AXtSz6u+2E1gHAB8E/iLJXnOYmiehpD1UVW0Dzgben+SVLfy7wO1VtT7JDwOjwP8LfF9VvRxYAWwHXt+zqVvaiKX96Iqw303y/XOY+quA/1NV2+fwOzRDFnraFT9F16kcSVfEnTl+hVbUvQ14BbCxhd8FnAL8a+Aw4AngD9v6RwOXAqcDh9IdUB0+jVx+AzgJOJ6ugzsR+M22zTcC/63leyjwFeBqgKr6F8CDwE+2jvCZae+9pIGX5AC64dv/sarWV9VT7QTV31XV6RP1CVX1XeCjwEF0ReFk239Bkt9sJ68eSXJl+86x5W9Lcnc7Gz+a5HUtfjPwo8AH2kmo18zibksaAFX1GeB64JIky+iOY85pi38X+NOq+m9V9XBb/8Gqek9Vje5ke58H7gVeN9V376xvaste12Jfb+u8rcV/C/jPwNtbv3XWLu66ZpmFnnbFJVW1paoeB/6Krsgac1iSrwP/BFwL/EpV/V1b9h+A36iqze0g6r3AqUn2pjvb9FdV9dmq+jZdh1HTyOV04Ler6pGq+hrwW8DP9iz7cFV9vn3f+cAPJVm8qzsuaWj8ELAP8PHpNmhX8c4AHgAenmL1M9vrR+lGD+wHfKBt5zXAx4BfAl4JfJJudMGLquqNwP8CfrGdhPo/094jScPkl4FlwHrgV6tqa5KX0vVdfz6TDSX5v4DXALdPsd5O+6YkL6Q75vsMcAjwTuAjSV5bVe8Bfge4pvVbl88kP80dCz3tin/sef8tugOYMVvaMIL9gUuAN/YsexVwbTsT9HW6s0vP0p0ZPwx4aGzFqvoW8Ng0cjmM7krdmK+02POWteEQjzG9K4WShsNfjvU5rd+5tMVfATzaO8woyf9u6/1Tkh/p2cavtrbfBH4f+E9V9ewU33s68L6q+nLre84HVrUTW28Hrq+qG6vqO8D/APYFfni391bSUKiqJ+iGcL8E+IsWPpDu2P2fj8OS/G7rt76Z5Dd7NnFSi28DbgOuAu6f4msn65tOojveu7Cqvl1VNwOfAH56d/dVc8dCT3OiXUE7Dzguzz264CHgx6vq5T2vF1fVV4GtwKKx9kn2BQ6exldtoSsgx3xviz1vWTsTdjDw1V3bK0kD6JTePgf4jy3+GPCKVngBUFU/3NZ5jB3///F/tPi+wFLgvyf58Sm+d6KTUHvz3Imt3pNQ36XrHz0JJQmAJD8DLAb+GriohZ8Avkt3OwoAVfVrrX+6lq6PGXNr6/f2o7sH+Ri6q26TmaxvOgx4qMXGfAX7rb5moafxXpjkxWMvduw0ZqQNwfw9umGYAH8EXJDkVQBJXplkZVu2HvjJJD+c5EV0QzAzja/5GPCbbVuvaN/1Z23ZR4GfS3J8m2zld4ANVbVpV/dJ0tC4BXgGWDnVimPaPXz/QDcJwlumWH2ik1Db6YZ8jj8JFeAIPAklCUhyCHAx8PN0t738VJIfqapvAhuA/3sm22v38v058JNTrDpZ37QFOCJJb+3wvdhv9TULPY33Sbr768Ze793N7X0Y+N4kP0k3u911wGeSPAXcCvwgQFXdTTfe+2q6q3tPAY/QHYhN5r/SjTn/AnAX8PkWo6puAv4TXee2FfgXwKrd3B9JQ6Cqvk53QunSJKemm9L8BUmOB166s3ZJvg/4l0w9K+bHgF9OcmSS/Xju/pXtwDrgLUmWt/tezqXr6/737u6XpKHwAeAvq+pvqmor3SNX/qSdtP414N+lm8X8EIAki+gmyJtQkoOBf8PU/dZkfdMGuuHrv5bkhW2SmJ+kTXKn/pSq6cx3Ic2vdmD0dWBJVT2wwOlIGkBJNgH/vqr+uid2Zov9y/b5dODdwLF0BzFfBi4Hrqiqbye5Avi3wLfpRhk8Rjdq4DfHDWEa/90voJsB+OeBFwOfBt7Z7rshyb8BLqAb9nQn3eyfd7dlo8CfVdWHZuFnkDRA2u0ulwJHtxNSY/Gb6IZj/kaSHwTew3P39W6mm1jqfVX1WOvnLqc7YQ/dfAo3Ae+uqkem+P7J+qZjWm7H013J+42qurYtey9wVFX9zK7vvWabhZ76RrvqdxPdwdTv0V3t+4HyH6kkSZI0Iw7dVD9ZSTcGfAuwBFhVVZXkhvZclvGvX1/YdCVJkqT+5BU9SZJ2QZI/AiYapvRnVfUL852PJE2lDVf/4wkWfaWqjpnvfDS3LPQkSZIkacjs8tT5C+0Vr3hFLV68eKHTeJ5vfvObvPSlO52wrW+Z9/wZxJxh5nnfcccdj1bVK+cwpYHTr/0WDNa/y0HKFQYr30HKFWY/X/utib3iFa+oV77ylX37b6Of/92a264xt+mbtN+qqklfwGvpZt0Ze30D+CXgIOBG4P7298CeNucDG4H7gJN74ifQTYG/EbiE564o7gNc0+IbgMVT5XXCCSdUP/qbv/mbhU5hl5j3/BnEnKtmnjdwe03xv+M97dWv/VbVYP27HKRcqwYr30HKtWr287Xf2nnf1c//Nsxt15jbrum33Cbrt6acjKWq7quq46vq+FaofQu4FlgD3FRVS+hmSlwDkORoumeVHQOsoHtG0V5tcx8EzqabaGNJWw5wFvBEVR1F94DIi6bKS5IkSZI0sZnOurkc+FJVfYVuhsS1Lb4WOKW9XwlcXVXPVPf8s43AiUkOBfavqlta9XnluDZj21oPLE+SXdgfSZIkSdrjzfQevVXAx9r7karaClBVW5Mc0uKHA7f2tNncYt9p78fHx9o81La1PcmTwMHAo71fnuRsuiuCjIyMMDo6OsP05962bdv6Mq+pmPf8GcScYfDyTvLLwL8Him7I+M8BL6EbJr4Y2AT8VD33AOvz6UYXPAu8q6o+3eInAFcA+wKfpHvgbCXZh+6E1Ql0D9F+e1Vtmp+9kyRJmty0C70kLwLeRnf/3aSrThCrSeKTtdkxUHUZcBnA0qVLa9myZVOkMv9GR0fpx7ymYt7zZxBzhsHKO8nhwLuAo6vqn5KsoztRdTTdkPMLk6yhG3J+3rgh54cBf53kNVX1LM8NOb+VrtBbAdxAz5DzJKvohpy/fV53VJIkaSdmMnTzx4HPV9XD7fPDbTgm7e8jLb4ZOKKn3SK6B2Bvbu/Hx3dok2Rv4ADg8RnkJknj7Q3s2/qUl9D1Nw45lyRJe4SZFHo/zXPDNgGuA1a396uBj/fEVyXZJ8mRdJOu3NaGeT6V5KR2MHTGuDZj2zoVuLkdVEnSjFXVV4H/ATwIbAWerKrPMG7IOdA75Pyhnk2MDS0/nGkOOQfGhpxLkiQtuGkN3UzyEuDNwH/oCV8IrEtyFt3B1GkAVXV3GyZ1D7AdOKcNfwJ4B8/d63JDewFcDlyVZCPdlbxVu7FPkvZwSQ6ku+J2JPB14H8m+ZnJmkwQm5Uh54NwbzEM1j2Yg5QrDFa+g5QrDF6+kjSfplXoVdW3GHemuqoeo5uFc6L1LwAumCB+O3DsBPGnaYWiJM2CNwEPVNXXAJL8BfDDtCHnbQKp2RpyvnmyIeeDcG8xDNY9mIOUKwxWvoOUKwxevpI0n2b6eAVJGgQPAicleUkbKr4cuBeHnEvqY0lenmR9ki8muTfJDyU5KMmNSe5vfw/sWf/8JBuT3Jfk5J74CUnuassuGbt/uPVx17T4hiSLF2A3Jc2TmT5eQdKAWLzm+mmtt+nCt8xxJvOvqjYkWQ98nm4I+d/RXVXbD4ecS/NuT+6PZuj9wKeq6tQ22/lLgF/H2YKlSdnHTMxCT9JQqqr3AO8ZF34Gh5xL6kNJ9gd+BDgToKq+DXw7yUpgWVttLTAKnEfPbMHAA+2k04lJNtFmC27bHZst+IbW5r1tW+uBDySJoxGk4WShJ0mStPBeDXwN+NMkrwfuAN7NuNmCk/TOFnxrT/uxWYG/wzRnC04yNlvwo+OTGT+RVD9PfGNuu2aYcjv3uO3TWm829reff7fxLPQkSZIW3t7ADwDvbMPP3083THNn5my2YHj+RFL77bdf305808+T8pjbrplpbmdOd+jm6dPf5s708+82npOxSJIkLbzNwOaq2tA+r6cr/B5uswQzi7MFM9lswZKGg4WeJEnSAquqfwQeSvLaFlpON0GUswVL2iUO3ZQkSeoP7wQ+0mbc/DLwc3Qn5Z0tWNKMWehJkiT1gaq6E1g6wSJnC5Y0Yw7dlCRJkqQhY6EnSZIkSUPGQk+SJEmShoyFniRJkiQNGQs9SZIkSRoyFnqSJEmSNGQs9CRJkiRpyFjoSZIkSdKQsdCTJEmSpCFjoSdJkiRJQ8ZCT5IkSZKGjIWeJEmSJA0ZCz1JkiRJGjIWepKGTpLXJrmz5/WNJL+U5KAkNya5v/09sKfN+Uk2Jrkvyck98ROS3NWWXZIkLb5PkmtafEOSxQuwq5IkSROaVqGX5OVJ1if5YpJ7k/yQB0yS+lVV3VdVx1fV8cAJwLeAa4E1wE1VtQS4qX0mydHAKuAYYAVwaZK92uY+CJwNLGmvFS1+FvBEVR0FXAxcNA+7JkmSNC3TvaL3fuBTVfV9wOuBe/GASdJgWA58qaq+AqwE1rb4WuCU9n4lcHVVPVNVDwAbgROTHArsX1W3VFUBV45rM7at9cDysZNXkiRJC23vqVZIsj/wI8CZAFX1beDbSVYCy9pqa4FR4Dx6DpiAB5KMHTBtoh0wte2OHTDd0Nq8t21rPfCBJGkHVpK0O1YBH2vvR6pqK0BVbU1ySIsfDtza02Zzi32nvR8fH2vzUNvW9iRPAgcDj/Z+eZKz6U5wMTIywujo6Ozs1Szbtm1b3+Y23iDlCoOV71zleu5x26e13ky/e5B+W0kLb/Ga66e97qYL3zKHmcyPKQs94NXA14A/TfJ64A7g3XjANKFB/T8d854/85XzbB9YDeJvneRFwNuA86dadYJYTRKfrM2OgarLgMsAli5dWsuWLZsilYUxOjpKv+Y23iDlCoOV71zleuY0D642nT6z7x6k31aS5tt0Cr29gR8A3llVG5K8nzZMcyf26AOmQf0/HfOeP/OV82wfWA3ibw38OPD5qnq4fX44yaHt5NShwCMtvhk4oqfdImBLiy+aIN7bZnOSvYEDgMfnZjckSZJmZjr36G0GNlfVhvZ5PV3h93A7UGIWD5jwgEnSLPppnhu2CXAdsLq9Xw18vCe+qk0MdSTdPcS3tVELTyU5qd1/d8a4NmPbOhW42eHmkiSpX0xZ6FXVPwIPJXltCy0H7sEDJkl9LMlLgDcDf9ETvhB4c5L727ILAarqbmAdXd/2KeCcqnq2tXkH8CG6CVq+RHdfMcDlwMHtPuRfYfKRDpIkSfNqOkM3Ad4JfKTd7/Jl4OfoisR1Sc4CHgROg+6AKcnYAdN2nn/AdAWwL93BUu8B01XtgOlxuskTJGmXVdW36O717Y09RneyaqL1LwAumCB+O3DsBPGnaf2eJElSv5lWoVdVdwJLJ1jkAZMkSZIk9ZnpPkdPkiRJkjQgLPQkSZIkachY6EmSJEnSkLHQkyRJkqQhY6EnSZLUJ5JsSnJXkjuT3N5iByW5Mcn97e+BPeufn2RjkvuSnNwTP6FtZ2OSS9qjrWiPv7qmxTckWTzvOylpXljoSZIk9Zcfrarjq2psxvM1wE1VtQS4qX0mydF0j6Q6BlgBXJpkr9bmg8DZdM8zXtKWA5wFPFFVRwEXAxfNw/5IWgAWepIkSf1tJbC2vV8LnNITv7qqnqmqB4CNwIlJDgX2r6pbqqqAK8e1GdvWemD52NU+ScNlug9MlyRJ0twr4DNJCvjjqroMGKmqrQBVtTXJIW3dw4Fbe9pubrHvtPfj42NtHmrb2p7kSeBg4NHeJJKcTXdFkJGREbZt28bo6Ois7eRsMrddM0y5nXvc9lnPYWff38+/23gWepIkSf3jDVW1pRVzNyb54iTrTnQlriaJT9Zmx0BXYF4GsHTp0tpvv/1YtmzZpIkvlNHRUXPbBcOU25lrrp/1HDadPvH39/PvNp5DNyVJkvpEVW1pfx8BrgVOBB5uwzFpfx9pq28GjuhpvgjY0uKLJojv0CbJ3sABwONzsS+SFpaFniRJUh9I8tIkLxt7D/wY8A/AdcDqttpq4OPt/XXAqjaT5pF0k67c1oZ5PpXkpHb/3Rnj2oxt61Tg5nYfn6Qh49BNSZKk/jACXNvmRtkb+GhVfSrJ54B1Sc4CHgROA6iqu5OsA+4BtgPnVNWzbVvvAK4A9gVuaC+Ay4Grkmyku5K3aj52TNL8s9CTJEnqA1X1ZeD1E8QfA5bvpM0FwAUTxG8Hjp0g/jStUJQ03By6KUmSJElDxkJPkiRJkoaMhZ4kSZIkDRkLPUmSJEkaMhZ6koZSkpcnWZ/ki0nuTfJDSQ5KcmOS+9vfA3vWPz/JxiT3JTm5J35CkrvaskvaVOW06cyvafENSRYvwG5KkiRNyEJP0rB6P/Cpqvo+ulns7gXWADdV1RLgpvaZJEfTTTF+DLACuDTJXm07HwTOpns+1ZK2HOAs4ImqOgq4GLhoPnZKkiRpOiz0JA2dJPsDP0L3vCiq6ttV9XVgJbC2rbYWOKW9XwlcXVXPVNUDwEbgxCSHAvtX1S3tgcJXjmsztq31wPKxq32SJEkLzefoSRpGrwa+BvxpktcDdwDvBkaqaitAVW1Nckhb/3Dg1p72m1vsO+39+PhYm4fatrYneRI4GHi0N5EkZ9NdEWRkZITR0dFZ2sXZtW3btr7NbbxByhUGK9+5yvXc47ZPa72Zfvcg/baSNN8s9CQNo72BHwDeWVUbkryfNkxzJya6EleTxCdrs2Og6jLgMoClS5fWsmXLJklj4YyOjtKvuY03SLnCYOU7V7meueb6aa236fSZffcg/baSNN8cuilpGG0GNlfVhvZ5PV3h93Abjkn7+0jP+kf0tF8EbGnxRRPEd2iTZG/gAODxWd8TSZKkXTCtQi/Jpjbr3J1Jbm8xZ6+T1Jeq6h+Bh5K8toWWA/cA1wGrW2w18PH2/jpgVeuLjqSbdOW2NszzqSQntf7qjHFtxrZ1KnBzu49PkiRpwc3kit6PVtXxVbW0fXb2Okn97J3AR5J8ATge+B3gQuDNSe4H3tw+U1V3A+voisFPAedU1bNtO+8APkQ3QcuXgBta/HLg4CQbgV9h8qGhkiRJ82p37tFbCSxr79cCo8B59MxeBzzQDoJOTLKJNnsdQJKx2etuaG3e27a1HvhAknh2XNKuqqo7gaUTLFq+k/UvAC6YIH47cOwE8aeB03YvS0mSpLkx3UKvgM8kKeCP2+QCzl43gUGdAcy858985Tzbs9wN4m8tSZK0p5puofeGqtrSirkbk3xxknX36NnrBnUGMPOeP/OV82zPcjeIv7UkSdKealr36FXVlvb3EeBa4EScvU6SJEmS+tKUhV6SlyZ52dh74MeAf8DZ6yRJkiSpL01n6OYIcG17EsLewEer6lNJPgesS3IW8CBtUoKqujvJ2Ox123n+7HVXAPvSTcLSO3vdVW3ilsfpZu2UJEmSJO2CKQu9qvoy8PoJ4o/h7HWSJEmS1Hd25/EKkubZ4mlOsCJJkqQ920wemC5JkiRJGgAWepIkSZI0ZCz0JEmSJGnIWOhJkiRJ0pCx0JMkSZKkIWOhJ0mSJElDxkJPkiSpTyTZK8nfJflE+3xQkhuT3N/+Htiz7vlJNia5L8nJPfETktzVll2SJC2+T5JrWnxDksXzvoOS5o2FniRJUv94N3Bvz+c1wE1VtQS4qX0mydHAKuAYYAVwaZK9WpsPAmcDS9prRYufBTxRVUcBFwMXze2uSFpIFnqSJEl9IMki4C3Ah3rCK4G17f1a4JSe+NVV9UxVPQBsBE5Mciiwf1XdUlUFXDmuzdi21gPLx672SRo+FnqSJEn94feBXwO+2xMbqaqtAO3vIS1+OPBQz3qbW+zw9n58fIc2VbUdeBI4eFb3QFLf2HuhE5AkSdrTJXkr8EhV3ZFk2XSaTBCrSeKTtZkon7Pphn8yMjLCtm3bGB0dnUZa88/cds0w5XbucdtnPYedfX8//27jWehJkiQtvDcAb0vyE8CLgf2T/BnwcJJDq2prG5b5SFt/M3BET/tFwJYWXzRBvLfN5iR7AwcAj0+UTFVdBlwGsHTp0tpvv/1YtmzZ7u/lHBgdHTW3XTBMuZ255vpZz2HT6RN/fz//buM5dFPSUEqyqc06d2eS21vM2esk9aWqOr+qFlXVYrpJVm6uqp8BrgNWt9VWAx9v768DVrW+6Ei6SVdua8M7n0pyUuuvzhjXZmxbp7bvmPCKnqTBZ6EnaZj9aFUdX1VL22dnr5M0aC4E3pzkfuDN7TNVdTewDrgH+BRwTlU929q8g25Cl43Al4AbWvxy4OAkG4FfofWBkoaTQzcl7UlWAsva+7XAKHAePbPXAQ+0g6ATk2yizV4HkGRs9robWpv3tm2tBz6QJJ4dl7S7qmqUrn+iqh4Dlu9kvQuACyaI3w4cO0H8aeC0WUxVUh+z0JM0rAr4TJIC/rjdb7LD7HVJemevu7Wn7dgsdd9hmrPXJRmbve7R3iTGT2jQrzdwD9LN5YOUKwxWvnOV63QnSpjpdw/SbytJ881CT9KwekNVbWnF3I1JvjjJunM2e934CQ369QbuQbq5fJByhcHKd65yne5ECTub/GBnBum3laT55j16koZSVW1pfx8BrgVOpM1eBzCLs9cx1ex1kiRJ881CT9LQSfLSJC8bew/8GPAPOHudJEnaQzh0U9IwGgGubU9C2Bv4aFV9KsnngHVJzgIepE1KUFV3JxmbvW47z5+97gpgX7pJWHpnr7uqTdzyON2snZIkSX3BQk/S0KmqLwOvnyDu7HWSJGmP4NBNSZIkSRoy0y70kuyV5O+SfKJ9PijJjUnub38P7Fn3/CQbk9yX5OSe+AlJ7mrLLmn3vNDui7mmxTckWTyL+yhJkiRJe5SZXNF7N3Bvz+c1wE1VtQS4qX0mydF096ocA6wALk2yV2vzQbrnSS1prxUtfhbwRFUdBVwMXLRLeyNJkiRJml6hl2QR8BbgQz3hlcDa9n4tcEpP/OqqeqaqHgA2Aie2qcz3r6pb2sx0V45rM7at9cDysat9kiRJkqSZme5kLL8P/Brwsp7YSJt6nKra2h5KDHA4cGvPeptb7Dvt/fj4WJuH2ra2J3kSOBh4tDeJJGfTXRFkZGSE0dHRaaY/f7Zt29aXeU3FvOfP7uR87nHbZzcZmHYug/hbS5Ik7ammLPSSvBV4pKruSLJsGtuc6EpcTRKfrM2OgarLgMsAli5dWsuWTSed+TU6Oko/5jUV854/u5PzmWuun91kgE2nL5vWeoP4W0uSJO2ppnNF7w3A25L8BPBiYP8kfwY8nOTQdjXvUOCRtv5m4Iie9ouALS2+aIJ4b5vNSfYGDqB7LpUkSZIkaYamvEevqs6vqkVVtZhukpWbq+pngOuA1W211cDH2/vrgFVtJs0j6SZdua0N83wqyUnt/rszxrUZ29ap7Tued0VPkiRJkjS13Xlg+oXAuiRnAQ/SHhxcVXcnWQfcA2wHzqmqZ1ubdwBXAPsCN7QXwOXAVUk20l3JW7UbeUmSJEnSHm1GhV5VjQKj7f1jwPKdrHcBcMEE8duBYyeIP00rFCVJkiRJu2cmz9GTJEmSJA2A3Rm6KUmSJEmzbvEczDS+p/GKniRJkiQNGQs9SZIkSRoyFnqSJEmSNGQs9CRJkiRpyDgZiyRJmjEnSpCk/uYVPUmSJEkaMhZ6koZWkr2S/F2ST7TPByW5Mcn97e+BPeuen2RjkvuSnNwTPyHJXW3ZJUnS4vskuabFNyRZPO87KEmStBMWepKG2buBe3s+rwFuqqolwE3tM0mOBlYBxwArgEuT7NXafBA4G1jSXita/Czgiao6CrgYuGhud0WSJGn6LPQkDaUki4C3AB/qCa8E1rb3a4FTeuJXV9UzVfUAsBE4McmhwP5VdUtVFXDluDZj21oPLB+72idJkrTQnIxF0rD6feDXgJf1xEaqaitAVW1NckiLHw7c2rPe5hb7Tns/Pj7W5qG2re1JngQOBh7tTSLJ2XRXBBkZGWF0dHR392tObNu2rW9zG2+QcoXByncmuZ573PZZ//6Z/k6D9NtOJcmLgb8F9qE7PltfVe9JchBwDbAY2AT8VFU90dqcTze64FngXVX16RY/AbgC2Bf4JPDuqqok+9CdsDoBeAx4e1VtmqddlDTPLPQkDZ0kbwUeqao7kiybTpMJYjVJfLI2OwaqLgMuA1i6dGktWzaddObf6Ogo/ZrbeIOUKwxWvjPJ9cw5mHVz0+nT++4xg/TbTsMzwBuraluSFwKfTXID8H/TDTm/MMkauiHn540bcn4Y8NdJXlNVz/LckPNb6Qq9FcAN9Aw5T7KKbsj52+d3NyXNF4duShpGbwDelmQTcDXwxiR/BjzchmPS/j7S1t8MHNHTfhGwpcUXTRDfoU2SvYEDgMfnYmckDb/qbGsfX9hehUPOJe0ir+hJGjpVdT5wPkC7overVfUzSf47sBq4sP39eGtyHfDRJO+jOzO+BLitqp5N8lSSk4ANwBnAH/S0WQ3cApwK3NwOqiRpl7RJoO4AjgL+sKo2JJn3Iectlx2GnffzMFlz2zX9ntu5xz27oDns7Lfp599tPAs9SXuSC4F1Sc4CHgROA6iqu5OsA+4BtgPntOFPAO/guXtdbmgvgMuBq5JspLuSt2q+dkLScGr9zvFJXg5cm+TYSVafsyHnLZcdhp3vt99+fTtMtp+H8JrbrhkdHeX3PvvNBc1hZ0PJ+/l3G89CT9JQq6pRYLS9fwxYvpP1LgAumCB+O/C8g62qeppWKErSbKqqrycZpbu37uEkh7arebM15HyzQ86l4ec9epIkSQssySvblTyS7Au8Cfgizw0Th+cPOV+VZJ8kR/LckPOtwFNJTmr3350xrs3YthxyLg05r+hJkiQtvEOBte0+vRcA66rqE0luwSHnknaBhZ4kSdICq6ovAN8/Qdwh55J2iUM3JUmSJGnIWOhJkiRJ0pCZstBL8uIktyX5+yR3J/mtFj8oyY1J7m9/D+xpc36SjUnuS3JyT/yEJHe1ZZeMPaSz3Uh8TYtvSLJ4DvZVkiRJkvYI07mi9wzwxqp6PXA8sKI9PHgNcFNVLQFuap9JcjTdzb3H0E0LfGm7sRjgg3QP31zSXita/Czgiao6CrgYuGj3d02SJEmS9kxTFnrV2dY+vrC9ClgJrG3xtcAp7f1K4OqqeqaqHgA2Aie2Z7/sX1W3tKl8rxzXZmxb64HlY1f7JEmSJEkzM6179JLsleROuod03lhVG4CR9qwW2t9D2uqHAw/1NN/cYoe39+PjO7Spqu3Ak8DBu7A/kiRJkrTHm9bjFdpzWY5vD/K8NsnzpuztMdGVuJokPlmbHTecnE039JORkRFGR0cnSWNhbNu2rS/zmop5z5/dyfnc47bPbjIw7VwG8beWJEnaU83oOXpV9fUko3T31j2c5NCq2tqGZT7SVtsMHNHTbBGwpcUXTRDvbbM5yd7AAXQP8hz//ZcBlwEsXbq0li1bNpP058Xo6Cj9mNdUzHv+TJTz4jXXT7P17D/6ctPpy6ZcBwbzt5YkSdpTTWfWzVe2K3kk2Rd4E/BF4DpgdVttNfDx9v46YFWbSfNIuklXbmvDO59KclK7/+6McW3GtnUqcHO7j0+SJEmSNEPTuTxwKLC2zZz5AmBdVX0iyS3AuiRnAQ8CpwFU1d1J1gH3ANuBc9rQT4B3AFcA+wI3tBfA5cBVSTbSXclbNRs7J0mSJEl7oikLvar6AvD9E8QfA5bvpM0FwAUTxG8Hnnd/X1U9TSsUJUmSJEm7Z1qzbkqSJEmSBoeFniRJkiQNGQs9SZIkSRoyFnqSJEmSNGQs9CQNnSQvTnJbkr9PcneS32rxg5LcmOT+9vfAnjbnJ9mY5L4kJ/fET0hyV1t2SXs8DO0RMte0+IYki+d9RyVJknbCQk/SMHoGeGNVvR44HliR5CRgDXBTVS0BbmqfSXI03WNdjgFWAJe2R8oAfBA4m+6ZoEvacoCzgCeq6ijgYuCiedgvSZKkaZnOc/QkaaBUVQHb2scXtlcBK4FlLb4WGAXOa/Grq+oZ4IH2TM8Tk2wC9q+qWwCSXAmcQvcM0JXAe9u21gMfSJL23ZIkaQKL11w/5TrnHrcdy5Td5xU9SUMpyV5J7gQeAW6sqg3ASFVtBWh/D2mrHw481NN8c4sd3t6Pj+/Qpqq2A08CB8/JzkiSJM2QpbKkoVRVzwLHJ3k5cG2SYydZPRNtYpL4ZG123HByNt3QT0ZGRhgdHZ0kjYWzbdu2vs1tvEHKFQYr35nk2p1xn10z/Z0G6beVpPlmoSdpqFXV15OM0t1b93CSQ6tqa5JD6a72QXel7oieZouALS2+aIJ4b5vNSfYGDgAen+D7LwMuA1i6dGktW7ZslvZsdo2OjtKvuY03SLnCYOU7k1zPnMbwq5nadPr0vnvMIP22kjTfHLopaegkeWW7kkeSfYE3AV8ErgNWt9VWAx9v768DVrWZNI+km3Tltja886kkJ7XZNs8Y12ZsW6cCN3t/niRJ6hde0ZM0jA4F1raZM18ArKuqTyS5BViX5CzgQeA0gKq6O8k64B5gO3BOG/oJ8A7gCmBfuklYbmjxy4Gr2sQtj9PN2ilJktQXLPQkDZ2q+gLw/RPEHwOW76TNBcAFE8RvB553f19VPU0rFCVJkvqNQzclSZIkach4RU+SJAFw11efnJNJViRJ888repIkSZI0ZCz0JEmSFliSI5L8TZJ7k9yd5N0tflCSG5Pc3/4e2NPm/CQbk9yX5OSe+AlJ7mrLLmmzBtNmFr6mxTckWTzvOypp3ljoSZIkLbztwLlV9TrgJOCcJEcDa4CbqmoJcFP7TFu2CjiG7jmhl7aZhgE+CJxN96iYJW05wFnAE1V1FHAxcNF87JikhWGhJ0mStMCqamtVfb69fwq4FzgcWAmsbautBU5p71cCV1fVM1X1ALARODHJocD+VXVLe7bnlePajG1rPbB87GqfpOHjZCySJEl9pA2p/H5gAzBSVVuhKwaTHNJWOxy4tafZ5hb7Tns/Pj7W5qG2re1JngQOBh6dIIez6a4KMjIywrZt2xgdHZ2N3Zt15rZrFiq3c4/bPuU6I/tOb725tLPfpp//m45noSdJktQnkuwH/DnwS1X1jUkuuE20oCaJT9bm+cGqy4DLAJYuXVr77bcfy5YtmyTzhTM6Ompuu2ChcpvOzL7nHred37trYcuUTacvmzDez/9Nx3PopiRJUh9I8kK6Iu8jVfUXLfxwG45J+/tIi28GjuhpvgjY0uKLJojv0CbJ3sABwOOzvyeS+oFX9KQ5sngnZ6zOPW67z6mSJO2g3St3OXBvVb2vZ9F1wGrgwvb34z3xjyZ5H3AY3aQrt1XVs0meSnIS3dDPM4A/GLetW4BTgZvbfXyShtCUV/Sc7leSJGnOvQH4WeCNSe5sr5+gK/DenOR+4M3tM1V1N7AOuAf4FHBOVT3btvUO4EN0E7R8CbihxS8HDk6yEfgV2gyekobTdK7ojU33+/kkLwPuSHIjcCbddL8XJllD11mcN26638OAv07ymtb5jE33eyvwSbrpfm+gZ7rfJKvopvt9+2zuqCRJUr+qqs8y8T10AMt30uYC4IIJ4rcDx04Qfxo4bTfSlDRApryi53S/kiRJkjRYZnSP3kJP9zt+qt9+nNp0kKZc7WXes29n0wL3w5TBvab7+/Xzby1JkqQdTbvQ64fpfsdP9duPU5sO0pSrvcx79u1swpV+mDK4186mDx6vn39rSZK0cHY2AZ0W1rQer+B0v5IkSZI0OKYz6+ZU0/3C86f7XdVm0jyS56b73Qo8leSkts0zxrUZ25bT/UqSJEnSbpjO+LGx6X7vSnJni/063fS+65KcBTxIm8Wpqu5OMjbd73aeP93vFcC+dLNt9k73e1Wb7vdxulk7JUmSJEm7YMpCz+l+JUmSJGmwTOsePUkaJEmOSPI3Se5NcneSd7f4QUluTHJ/+3tgT5vzk2xMcl+Sk3viJyS5qy27ZOzRL214+jUtvqHNSixJktQXLPQkDaPtwLlV9TrgJOCcJEcDa4CbqmoJcFP7TFu2CjgGWAFcmmSvtq0P0j3WZUl7rWjxs4Anquoo4GLgovnYMUmSpOnonzneJWmWtMmfxp7z+VSSe+me17kSWNZWWwuMAue1+NVV9QzwQLtf+MQkm4D9q+oWgCRXAqfQ3V+8Enhv29Z64ANJ4kRS6kfTnfr83OPmOBFJ0ryx0JNmyGfFDJY2pPL7gQ3ASCsCqaqtSQ5pqx0O3NrTbHOLfae9Hx8fa/NQ29b2JE8CBwOPjvv+s+muCDIyMtK3D53ftm1b3+Y23iDlCv2R77nHbZ/WeiP7Tn/duTDT36kffltJ6lcWepKGVpL96J4B+ktV9Y12e92Eq04Qq0nik7XZMVB1GXAZwNKlS6tfHzo/OjpKv+Y23iDlCv2R75nTvqK3nd+7a+EODTadvmxG6/fDbytJ/cp79CQNpSQvpCvyPlJVf9HCDyc5tC0/FHikxTcDR/Q0XwRsafFFE8R3aJNkb+AAusfDSJIkLTgLPUlDp82MeTlwb1W9r2fRdcDq9n418PGe+Ko2k+aRdJOu3NaGeT6V5KS2zTPGtRnb1qnAzd6fJ0mS+oVDNyUNozcAPwvcleTOFvt14EJgXZKzgAdpz++sqruTrAPuoZux85yqera1ewdwBbAv3SQsN7T45cBVbeKWx+lm7ZQkSeoLFnqShk5VfZaJ76EDWL6TNhcAF0wQvx04doL407RCUZIkqd84dFOSJEmShoyFniRJkiQNGQs9SZIkSRoy3qMnSdIAWjzNZ+NJkvZMFnqSJEmSduDJpMFnoSdhZyZJkqTh4j16kiRJkjRkLPQkSZIkachY6EmSJEnSkLHQkyRJkqQhY6EnSZLUB5J8OMkjSf6hJ3ZQkhuT3N/+Htiz7PwkG5Pcl+TknvgJSe5qyy5JkhbfJ8k1Lb4hyeJ53UFJ88pCT5IkqT9cAawYF1sD3FRVS4Cb2meSHA2sAo5pbS5Nsldr80HgbGBJe41t8yzgiao6CrgYuGjO9kTSgrPQkyRJ6gNV9bfA4+PCK4G17f1a4JSe+NVV9UxVPQBsBE5Mciiwf1XdUlUFXDmuzdi21gPLx672SRo+PkdPkiSpf41U1VaAqtqa5JAWPxy4tWe9zS32nfZ+fHyszUNtW9uTPAkcDDw6/kuTnE13VZCRkRG2bdvG6OjobO3TrDK3XTNVbucet33+khlnZN+F/X5gp79NP/83HW/KQi/Jh4G3Ao9U1bEtdhBwDbAY2AT8VFU90ZadTzc04FngXVX16RY/gW5Iwr7AJ4F3V1Ul2YfubNMJwGPA26tq06ztoSRJ0vCZ6EpcTRKfrM3zg1WXAZcBLF26tPbbbz+WLVu2C2nOvdHRUXObgcVrrgfg3OOe5fc++81J1ly460HnHred37trYa9HbTp92YTxfvxvujPTGbp5BY4XlyRJWggPt+GYtL+PtPhm4Iie9RYBW1p80QTxHdok2Rs4gOcPFZU0JKYslavqbyeYlWklsKy9XwuMAufRM14ceCDJ2HjxTbTx4gBJxsaL39DavLdtaz3wgSRp48olSdqjjJ1tl5rrgNXAhe3vx3viH03yPuAwupPot1XVs0meSnISsAE4A/iDcdu6BTgVuNnjLWl47eo10b4YL96P42MHadxurz097/kcB94P4857Tff3G9R/I5I0KJJ8jO5E+iuSbAbeQ1fgrUtyFvAgcBpAVd2dZB1wD7AdOKeqnm2begfP3S5zQ3sBXA5c1U7EP043CkvSkJrtwa/zOl68H8fHDtK43V7DmPfMzorP3zjwfhh33mtnY9DHG7R/I95fLGnQVNVP72TR8p2sfwFwwQTx24FjJ4g/TSsUJQ2/XT3afDjJoe1q3myNF9/seHFJs+gK4AN0xdiYsfuLL0yypn0+b9z9xYcBf53kNe3s+Nj9xbfSFXor6M6O//P9xUlW0d1f/PZ52TNJkno45FsT2dVCz/Hikvqa9xdLg2cmB6ubLnzLHGYiSYNvOo9XcLy4pGEx7/cXD8K9xTBY92AOUq4w83z39GdXTdfo6OjA/VuQNDh2duLp3OO2c2bPsn4+6TSdWTcdL66+0vs/vPH/Y5N20ZzdXzwI9xbDYN2DOUi5wszzXcg+rd/uIZ7MptOXDdy/BUmaT9N5jp4kDQufRyVJkvYIg3HaTpJmh/cXa0E4UYIkab5Z6EkaSt5fLEmS9mQWeuoLnu3WbPP+YkmStCez0JMkaRdN5yRVN4ul/3craeY8Ea7d4WQskiRJkjRkLPQkSZIkacg4lkRzyiEHkgaN/ZYkaRh4RU+SJEmShoyFniRJkiQNGYduSpIkSfPE4eGaLxZ60h5uuv+Hc+5x21k2t6lIkiRplljoacY8EyVJkiT1Nws9SdLQ8wSVJGlPY6EnSZIk7aaJTiide9x2zvREkxaIhZ7+mWe8JUmSpOFgoSdJGlieoJI0l+xjNMgs9IbcTGZU9J+DJEmSNBw8sh9Anl2SNMym6uO850WSpKlZ6EmS5oUnqST1A/si7Sks9PqIHY8kSZKk2WChJ0naZZ6gkiSpP1nozTEPgiRJkuaex1zSjvqm0EuyAng/sBfwoaq6cIFTkqQpDWvf5QGTNLyGtd+StKO+KPSS7AX8IfBmYDPwuSTXVdU9C5vZzu3sIMjZ4KQ9x6D1XZMVb/Zd0p5hmPqtMfZfWkgzOTG66cK3zGEmz9cXhR5wIrCxqr4MkORqYCUwr52OZ7AlzVBf9F2SNAP2W9IeIlW10DmQ5FRgRVX9+/b5Z4EfrKpfHLfe2cDZ7eNrgfvmNdHpeQXw6EInsQvMe/4MYs4w87xfVVWvnKtk+sF0+q4B6bdgsP5dDlKuMFj5DlKuMPv52m89t974vusx+vffRj//uzW3XWNu07fTfqtfruhlgtjzKtCqugy4bO7T2XVJbq+qpQudx0yZ9/wZxJxhcPOeY1P2XYPQb8Fg/fcdpFxhsPIdpFxh8PLtE7t0zNXPv7W57Rpz2zX9nNt4L1joBJrNwBE9nxcBWxYoF0maLvsuSYPGfkvaQ/RLofc5YEmSI5O8CFgFXLfAOUnSVOy7JA0a+y1pD9EXQzeranuSXwQ+TTfV74er6u4FTmtX9f0QrZ0w7/kziDnD4OY9Z+y7Fswg5QqDle8g5QqDl++C241+q59/a3PbNea2a/o5tx30xWQskiRJkqTZ0y9DNyVJkiRJs8RCT5IkSZKGjIXeLEjyX5J8IcmdST6T5LCeZecn2ZjkviQnL2Se4yX570m+2HK/NsnLe5b1c96nJbk7yXeTLB23rG/zBkiyouW2Mcmahc5nZ5J8OMkjSf6hJ3ZQkhuT3N/+HriQOWr3DVrfNUh91iD2U/3cP9knLbwkv5qkkryiJ7ag/5b7uQ/r5/6q3/unfuqLBr7vqSpfu/kC9u95/y7gj9r7o4G/B/YBjgS+BOy10Pn25PpjwN7t/UXARQOS9+voHt46Ciztifd73nu1nF4NvKjlevRC57WTXH8E+AHgH3pivwusae/XjP178TW4r0Hruwapzxq0fqrf+yf7pAX//Y+gm7zlK8ArWmzB/y33cx/Wz/1VP/dP/dYXDXrf4xW9WVBV3+j5+FKee/DoSuDqqnqmqh4ANgInznd+O1NVn6mq7e3jrXTP0oH+z/veqrpvgkV9nTddLhur6stV9W3garqc+05V/S3w+LjwSmBte78WOGU+c9LsG7S+a5D6rAHsp/q6f7JPWnAXA7/Gjg9WX/B/y/3ch/Vzf9Xn/VNf9UWD3vdY6M2SJBckeQg4HfjPLXw48FDPaptbrB/9O+CG9n6Q8u7V73n3e35TGamqrQDt7yELnI9mwQD3XYPaZ/Vrrv2a12Tsk+ZBkrcBX62qvx+3qC/+zQxIHzYo/VU/5NYPOUxlYPqevniO3iBI8tfA90yw6Deq6uNV9RvAbyQ5H/hF4D1AJlh/Xp9nMVXebZ3fALYDHxlrNsH6fZf3RM0miPXT80P6PT8NoUHruwapzxqyfqpf89I8mOzfMvDrdMMQn9dsgtis/5vp5z6sn/urAe6f+iGHoWGhN01V9aZprvpR4Hq6jmYz3bj2MYuALbOc2qSmyjvJauCtwPJqg40ZgLx3YsHznkK/5zeVh5McWlVbkxwKPLLQCWlqg9Z3DVKfNWT9VL/mNRn7pFmys3/LSY6ju1fr75NA9+/i80lOZJ7+zfRzH9bP/dUA90/9kMNUBqbvcejmLEiypOfj24AvtvfXAauS7JPkSGAJcNt857czSVYA5wFvq6pv9Szq67wn0e95fw5YkuTIJC8CVtHlPCiuA1a396uBnZ0R1IAYtL5rSPqsfs11EPsn+6Q5VlV3VdUhVbW4qhbTHYT/QFX9I33wb7mf+7AB7a/6IbdB6IsGpu/xit7suDDJa4Hv0s1I9QsAVXV3knXAPXSX7c+pqmcXLs3n+QDdzEo3tjN1t1bVL/R73kn+DfAHwCuB65PcWVUn93veVbU9yS/SzVy2F/Dhqrp7gdOaUJKPAcuAVyTZTHeG9EJgXZKzgAeB0xYuQ82SQeu7BqbPGrR+qt/7J/uk/tMn/5b7uQ/r2/6qn/unfuuLBr3vyXNXkiVJkiRJw8Chm5IkSZI0ZCz0JEmSJGnIWOhJkiRJ0pCx0JMkSZKkIWOhJ0mSJElDxkJPkiRJkoaMhZ4kSZIkDRkLPUmSJEkaMhZ6kiRJkjRkLPQkSZIkachY6EmSJEnSkLHQkyRJkqQhY6EnSZIkSUPGQk+SJEmShoyFniRJkiQNGQs9SZIkSRoyFnqSJEmSNGQs9CRJkiRpyFjoSZIkSdKQsdCTJEmSpCFjoSdJkiRJQ8ZCT5IkSZKGjIWeJEmSJA0ZCz1JkiRJGjIWepIkSZI0ZCz0JEmSJGnIWOhJkiRJ0pCx0JMkSZKkIWOhJ0mSJElDxkJPkiRJkoaMhZ4kSZIkDRkLPUmSJEkaMhZ6kiRJkjRkLPQkSZIkachY6EmSJEnSkLHQkyRJkqQhY6EnSdIMJNmW5NU7WXZmks/Od07Tkc6fJnkiyW0LnY8kaW5Z6GloJXltkr9L8lSSdy10PpImlmRTkn9qBdQTSa5PcsQC5bIsyXdbLk8luS/Jz/WuU1X7VdWXFyK/3fQvgTcDi6rqxIVORpImkuQNSe5v/fApC53PILPQ07S0A7E39Xxe1Q7I/nWSSnL9uPX/LMl72/tlbZ0/HLfOZ5OcOYdp/xowWlUvq6pL5vB7JO2+n6yq/YBDgYeBP1jAXLa0XPYHfhn4kySvXcB8ZsurgE1V9c2FTkQaNuNOWI29PjBFm2VJNu/Gd37vuO+rJN/s+fyvdnXbC+y3gQ+0k2p/udDJDDILPc1YktXAHwJvAb7SwiclecMkzb4JnJFk8Ryn1+tVwN3z+H2SdlNVPQ2sB44GSPKWdmX+G0keGjuB1JZdn+Sdve2TfGHsDHCS70tyY5LH25W5n+pZ7yeS3NOu2n01ya9OkEtV1SeBx4H/T0/bSnJUe39wkutafrcB/2JcPj/WvvvJJJcm+X+S/Pue5f8uyb3txNmnk7xqqt8oyQ8n+Vzb5ueS/HDPssNaPo8n2Zjk51v8LOBDwA+1A8Dfmup7JM3YT7biZOz1i7u7wSR772xZVT3Y+30t/Pqe2P+aznb6kMdvs8RCTzOS5Gzg94CTq+p/9yz6XeC/TtL068AVwHtm+H0vSPKbSb6S5JEkVyY5oGf525LcneTrSUaTvK7FbwZ+FPhAO6h5zUy+V9LCSPIS4O3ArS30TeAM4OV0J5fe0TOUZy3wMz1tXw8cDnwyyUuBG4GPAocAPw1cmuSYtvrlwH+oqpcBxwI3T5DLC5K8DXgFsHEnKf8h8DTdlch/115j7V9BV7SeDxwM3Af0FmWnAL8O/N/AK4H/BXxs578OJDkIuB64pG3zfcD1SQ5uq3wM2AwcBpwK/E6S5VV1OfALwC3tAHBGfbGkXZPkg0nW93y+KMlNrY+6ATis5wrcYUnem2R9Gxn1DeDMJCcmuaUd62xN8oEkL5rie89M8v8muTjJ48B7k+yT5H8keTDJw0n+KMm+PW3emuTO9j3/O0nvCa7z2kmxsSHty6f4/n2S/H6SLe31+0n26Vn+8+1k1OPt5NRhLf4l4NXAX7XfZJ+dfYemoap8+ZryBWwC/pxuSNXre+KLgQL2A74KvKnF/wx4b3u/jO7A43uAbwCvbfHPAmdO8b3/ju4A69XtO/4CuKotew3dQeCbgRfSDdXcCLyoLR8F/v1C/3a+fPma/NX6l210J4S2A1uA43ay7u8DF7f3+9BdbVvSPv8P4NL2/u3A/xrX9o+B97T3DwL/Adh/3DrLgO+2XJ4BngV+adw6BRwF7AV8B/i+nmW/A3y2vT+DrrAaWxbgobF+ie4g76ye5S8AvgW8apLf6meB28bFbgHOBI5o+b6sZ9l/A65o788cy82XL1+z+2r92JsmiL8E+D/tf3//CniU7j7Zfz4+Grf+e1u/ckrrE/YFTgBOAvamO+66d3y/1NoWcFR7f2brT9/Z2u3b+s/rgIOAlwF/Bfy3tv4PAI8AP9j6ttVtn/YBXtv6rsPauouBfzHF7/HbdCfsDqE7kfW/gf/Slr2x/Q4/0Lb/B8DfTvVb+pr5yyt6mok30/2P9q4Jlj0NXMAkV/Wq6h+BP6L7H/90nQ68r6q+XFXb6M6Mr2pDEN4OXF9VN1bVd+gO8val54y5pIFxSlW9nO7/9H8R+H+SfE+SH0zyN0m+luRJuqtSrwCoqmeAdcDPJHkB3VW7q9r2XgX8YDsz/fUkX6frT76nLf//Aj8BfKUNp/yhnly2tFz2p7ty9sad5PxKugOoh3piX+l5f1jvsuqOYHrvx3kV8P6e/B6nKwYP3/nPxGHjvmPsOw9vyx6vqqcmWCZp7v1lb5+T5Oer6lt0Iw/eR3cS/J1VNdV9ebdU1V9W1Xer6p+q6o6qurWqtlfVJrqTVv96Gvlsqao/qKrtdMdpPw/8clWN9RO/A6xq6/488MdVtaGqnq2qtXQnu06iO4G0D3B0khdW1aaq+tIU33068NtV9UhVfQ34LboTVWPLPlxVn2/9+Pl0w8oXT2OfNAMWepqJX6C7ivahJJlg+Z8AI0l+cpJtXASc3IZYTcf4g5qv0B1YjYxfVlXfpTuo8qBGGlDtAOMv6A4s/iXd0MvrgCOq6gC6k0W9/c9auoOG5cC3quqWFn8I+H+q6uU9r/2q6h3tez5XVSvpzjb/JV3BOD6XZ4DzgOMy8cxvX6M7Y947Q+j39rzfCiwa+9D6zUU9yx+iGz7am+O+teOw+PG20BWIvb6XbkTFFuCgJC+bYJmkuXfKuP89/wlAVd0GfJmu73peXzOB3pNHJHlNkk8k+cc2nPN3aCe8ZrCdV9JdXbyj5+TSp1ocun7l3HEnx46gu4q3EfgluquNjyS5emyo5SQmOn47bKJl7UT+Y3j8Nuss9DQTj9AdTP0r4NLxC9tVtd8C/gs7Hoj1rvMY3dCB/zLN7xx/UPO9dAdWD49f1g6ijsCDGmlgpbMSOJBueNLL6K5SPZ3kRODf9q7fCrvv0t07fFXPok8Ar0nys0le2F7/V5LXJXlRktOTHND6rW/QFZbPU1Xfbtv+zxMse5ZuOPl7k7wkydF0w53GXE8rEtsohHN47ooidEXr+WP3DSY5IMlpU/xEn2z79W+T7J3k7XQT13yiqh6iGx7135K8uN1fcxbwkSm2KWkOJTmH7orYFrrbTMbUTpqMj38Q+CLdMPX96e7tnfA4a5LtPAr8E3BMTyF6QD03ictDwAXjCtWXVNXHAKrqo1X1L+mOu4ruxP1kJjp+2zLRsna/4sF4/DbrLPQ0I1W1hW4Y04okF0+wylV0ndmKSTbzPrrhla+bxld+DPjlJEcm2Y/uLNY1bRjCOuAtSZYneSFwLt0wg8nOhkvqT3+VZBtd0XUBsLqq7gb+I/DbSZ6iK7YmOht+JXAc3bAoANqwpB+jG5a0BfhHugOTsRv7fxbY1M6O/wI9k7pM4MPA9+5ktMIv0t0//I90E079aU8OjwKn0U1W9RhdQXY7XT9FVV3bcrq65fEPwI9PksfYybK30vV3j9EdNL61fRd0w1cXt32+lu6exBsn26akuZNuMrj/StfH/Czwa0mOb4sfBg5OzyRzO/Eyur5xW5LvA94x0zzaqKc/AS5OckjL7fAkJ7dV/gT4hTZcPklemm7W45eley7xG9vEKE/TFYwTnhzr8THgN5O8Mt3EVP+Z5/rojwI/l+T4ts3fATa0YamaRYM01ar6RFU9lOSNwN+y49lpqurZJO8Brpmk/TeS/C5Tnw2C7gDrsPZdLwY+TXdjMVV1X5KfobuJ93DgTrqpjb89452StGCqavEky9bTzVw5mQeB/7fGPcS8qu6jm6lzIhOejKqqUXYcXkm7x+YVPZ/T8/5rdIXXhKrqU3RD3mn3EW6m5z69qrqKHa9ETqmqPks3OcNEyzbvLJ+quoKuGJU0N/4qSW8BdCPd8clFVfX3AEl+HbgqydKq+mKSjwFfTrIX7bEyE/hV4DK6Ezt/R3eMtbN7hydzHl3BdWsrvr5Kd7Xw01V1e7rHsXwAWEJXzH2W7vhrH+BCuhP036E7oX72FN/1X+nuc/5C+/w/W4yquinJf6Kb5O/Atr1VE21EuyfdveGSJA2edI9juJluts0rFzqf8drZ8g10B03/P7rhm6+uqn9a0MQkSUPPoZuSpIHUiqiv0Q1/+ugCp7MzPwR8ie7+mJ+km6xh0iIvyb/Kc8/V2uE1HwlLkoaDV/S04JL8ERPfH/NnVfUL852PJEmSJpfkBroJ+sb7nar6nfnOR89noSdJkiRJQ2ZgJ2N5xSteUYsXL17oNP7ZN7/5TV760pcudBozNoh5m/P82N2c77jjjker6pVTr7nn6Ld+Cwbz3yaY93zbU/K235pYv/Vdg/rvcbxh2I9h2AcY7P2YtN+qqoF8nXDCCdVP/uZv/mahU9glg5i3Oc+P3c0ZuL0WqH8AXks3C+vY6xt0D3s9iG4WtPvb3wN72pwPbATuA07uiZ8A3NWWXcJzIyH2oZv5bCPdZBuLp8qr3/qtqsH8t1ll3vNtT8l7Ifutfn71W981qP8exxuG/RiGfaga7P2YrN+a1mQsSV6eZH2SLya5N8kPJTkoyY1J7m9/D+xZ//wkG5Pc1/N8DpKckOSutuyS9oBrkuyT5JoW35Bk8XTykqSJVNV9VXV8VR1PV6h9i+6ZYmuAm6pqCXBT+0x70PUq4Bi6afcvbVNdQzf19Nl0000v4blp+c8Cnqiqo4CLmd7jQiRJkubFdGfdfD/wqar6PuD1wL14wCRpMCwHvlRVXwFWAmtbfC1wSnu/Eri6qp6pqgfortKdmORQYP+quqWdNbtyXJuxba0Hlo+dvJIkSVpoUxZ6SfYHfgS4HKCqvl1VX8cDJkmDYRXwsfZ+pKq2ArS/h7T44cBDPW02t9jh9Dzcuie+Q5uq2g48CRw8B/lLkiTN2HQmY3k13XOK/jTJ64E7gHcz7oApSe8B06097ccOjL7DNA+YkowdMD3am0iSs+muCDIyMsLo6Oj09nIebNu2ra/yma5BzNuc58cg5jxekhcBb6O7/27SVSeI1STxydqMz6Fv+y0Y3P/O5j2/zFuSBs90Cr29gR8A3llVG5K8nzZMcyfm7ICpqi4DLgNYunRpLVu2bJI05tfo6Cj9lM90DWLe5jw/BjHnCfw48Pmqerh9fjjJoe3k1KHAIy2+GTiip90iYEuLL5og3ttmc5K9gQOAx8cn0M/9Fgzuf2fznl/mLUmDZzr36G0GNlfVhvZ5PV3h93A7UGIWD5iY7IBJkmbop3lu2CbAdcDq9n418PGe+Ko2MdSRdPcQ39ZGLTyV5KQ2nPyMcW3GtnUqcHMbli5JkrTgpiz0quofgYeSvLaFlgP34AGTpD6W5CXAm4G/6AlfCLw5yf1t2YUAVXU3sI6ub/sUcE5VPdvavAP4EN39xl8Cbmjxy4GDk2wEfoXJRzpIkiTNq+k+MP2dwEfa/S5fBn6Orkhcl+Qs4EHgNOgOmJKMHTBt5/kHTFcA+9IdLPUeMF3VDpgep5s8QZJ2WVV9i3GTo1TVY3QnqyZa/wLgggnitwPHThB/mtbvSZIk9ZtpFXpVdSewdIJFHjBJkiRJUp+Z7hU9SX1g8Zrrp73upgvfMoeZSNLss4+TNMzmu4+b7gPTJUmSJEkDwkJPkiRJkoaMhZ4kSVIfSPLLSe5O8g9JPpbkxUkOSnJjkvvb3wN71j8/ycYk9yU5uSd+QpK72rJL2mzntBnRr2nxDUkWL8BuSponFnqSJEkLLMnhwLuApVV1LLAX3Szka4CbqmoJcFP7TJKj2/JjgBXApUn2apv7IHA23SOulrTlAGcBT1TVUcDFwEXzsGuSFoiFniRJUn/YG9g3yd7AS4AtwEpgbVu+FjilvV8JXF1Vz1TVA3TP+jwxyaHA/lV1S3sm8ZXj2oxtaz2wfOxqn6ThY6EnSZK0wKrqq8D/oHs28Vbgyar6DDBSVVvbOluBQ1qTw4GHejaxucUOb+/Hx3doU1XbgScZ97xRScPDxytIkiQtsHbv3UrgSODrwP9M8jOTNZkgVpPEJ2szUT5n0w3/ZGRkhNHR0UlSmV/btm3rq3x21TDsxzDsA8zffpx73PZprzsb+VjoSZIkLbw3AQ9U1dcAkvwF8MPAw0kOraqtbVjmI239zcARPe0X0Q313Nzej4/3ttnchoceADw+UTJVdRlwGcDSpUtr2bJlu72Ds2V0dJR+ymdXDcN+DMM+wPztx5kzeY7e6ct2+/scuilJkrTwHgROSvKSdt/ccuBe4DpgdVtnNfDx9v46YFWbSfNIuklXbmvDO59KclLbzhnj2oxt61Tg5nYfn6Qh5BU9SZKkBVZVG5KsBz4PbAf+ju6K2n7AuiRn0RWDp7X1706yDrinrX9OVT3bNvcO4ApgX+CG9gK4HLgqyUa6K3mr5mHXJC0QCz1JkqQ+UFXvAd4zLvwM3dW9ida/ALhggvjtwLETxJ+mFYqShp9DNyVJkiRpyFjoSZIkSdKQsdCTJEmSpCFjoSdJkiRJQ8ZCT5IkSZKGjIWeJEmSJA0ZCz1JkiRJGjIWepIkSZI0ZCz0JEmSJGnIWOhJkiRJ0pCx0JM0lJK8PMn6JF9Mcm+SH0pyUJIbk9zf/h7Ys/75STYmuS/JyT3xE5Lc1ZZdkiQtvk+Sa1p8Q5LFC7CbkiRJE7LQkzSs3g98qqq+D3g9cC+wBripqpYAN7XPJDkaWAUcA6wALk2yV9vOB4GzgSXttaLFzwKeqKqjgIuBi+ZjpyRJkqbDQk/S0EmyP/AjwOUAVfXtqvo6sBJY21ZbC5zS3q8Erq6qZ6rqAWAjcGKSQ4H9q+qWqirgynFtxra1Hlg+drVPkiRpoe290AlI0hx4NfA14E+TvB64A3g3MFJVWwGqamuSQ9r6hwO39rTf3GLfae/Hx8faPNS2tT3Jk8DBwKO9iSQ5m+6KICMjI4yOjs7SLs6Obdu29V1O02He82u+8j73uO3TXnc6+Qzq7y1Js8FCT9Iw2hv4AeCdVbUhyftpwzR3YqIrcTVJfLI2OwaqLgMuA1i6dGktW7ZskjTm3+joKP2W03SY9/yar7zPXHP9tNfddPqyKdcZ1N9bkmaDQzclDaPNwOaq2tA+r6cr/B5uwzFpfx/pWf+InvaLgC0tvmiC+A5tkuwNHAA8Put7IkmStAumVegl2dRmnbszye0t5ux1kvpSVf0j8FCS17bQcuAe4DpgdYutBj7e3l8HrGp90ZF0k67c1oZ5PpXkpNZfnTGuzdi2TgVubvfxSZIkLbiZXNH70ao6vqqWts/OXiepn70T+EiSLwDHA78DXAi8Ocn9wJvbZ6rqbmAdXTH4KeCcqnq2becdwIfoJmj5EnBDi18OHJxkI/ArTD40VJKmlOS17aT62OsbSX7Jk+uSdsXu3KO3EljW3q8FRoHz6Jm9DnigHQSdmGQTbfY6gCRjs9fd0Nq8t21rPfCBJPHsuKRdVVV3AksnWLR8J+tfAFwwQfx24NgJ4k8Dp+1elpL0nKq6j+7EFO0k+VeBa3nu5PqFSda0z+eNO7l+GPDXSV7TTlSNnVy/Ffgk3cn1G+g5uZ5kFd3J9bfP315Kmi/TLfQK+EySAv64TS7g7HU9BnVmr0HMe0/OebZnpJvMIP7OkjRElgNfqqqvJPHkuqQZm26h94aq2tKKuRuTfHGSdffI2esGdWavQcx7T855tmekm8wg/s6SNERWAR9r7+f95LqkwTetQq+qtrS/jyS5FjiRNntd63Bma/a6zc5eJ0mS9mRJXgS8DTh/qlUniM3KyXVHUc29YdiPYdgHGNxnhU5lykIvyUuBF1TVU+39jwG/zXMzzl3I82ev+2iS99GNFx+bve7ZJE8lOQnYQDd73R/0tFkN3IKz10mzYvE0r/5tuvAtc5yJJGmGfhz4fFU93D7P+8l1R1HNvWHYj2HYBxjcZ4VOZTqzbo4An03y98BtwPVV9SmcvU6SJGku/DTPDdsEHw0jaRdMeUWvqr4MvH6C+GM4e50kSdKsSfISuhPo/6EnfCGwLslZwIO0Y6aqujvJ2Mn17Tz/5PoVwL50J9Z7T65f1U6uP053L6CkIbQ7j1eQJEnSLKqqb9FNjtIb8+S6pBmbyQPTJUmSJEkDwEJPkiRJkoaMhZ4kSZIkDRkLPUmSJEkaMhZ6kiRJkjRkLPQkSZIkachY6EmSJEnSkLHQkyRJkqQhY6EnSZIkSUPGQk+SJEmShoyFniRJkiQNGQs9SZIkSRoyFnqSJEmSNGQs9CRJkiRpyFjoSZIkSdKQsdCTJEmSpCFjoSdpKCXZlOSuJHcmub3FDkpyY5L7298De9Y/P8nGJPclObknfkLbzsYklyRJi++T5JoW35Bk8bzvpCRJ0k5Y6EkaZj9aVcdX1dL2eQ1wU1UtAW5qn0lyNLAKOAZYAVyaZK/W5oPA2cCS9lrR4mcBT1TVUcDFwEXzsD+SJEnTYqEnaU+yEljb3q8FTumJX11Vz1TVA8BG4MQkhwL7V9UtVVXAlePajG1rPbB87GqfJO2KJC9Psj7JF5Pcm+SHHIkgaVftvdAJSNIcKeAzSQr446q6DBipqq0AVbU1ySFt3cOBW3vabm6x77T34+NjbR5q29qe5EngYODR3iSSnE13RZCRkRFGR0dnbQdnw7Zt2/oup+kw7/k1X3mfe9z2aa87nXwG8Pd+P/Cpqjo1yYuAlwC/TjcS4cIka+hGIpw3biTCYcBfJ3lNVT3LcyMRbgU+STcS4QZ6RiIkWUU3EuHt87uLkuaLhZ6kYfWGqtrSirkbk3xxknUnuhJXk8Qna7NjoCswLwNYunRpLVu2bNKk59vo6Cj9ltN0mPf8mq+8z1xz/bTX3XT6sinXGaTfO8n+wI8AZwJU1beBbydZCSxrq60FRoHz6BmJADyQZGwkwibaSIS23bGRCDe0Nu9t21oPfCBJ2ogFSUPGQk/SUKqqLe3vI0muBU4EHk5yaLuadyjwSFt9M3BET/NFwJYWXzRBvLfN5iR7AwcAj8/V/kgaeq8Gvgb8aZLXA3cA72YBRiJAf49GGMArtRMahv0Yhn2AwR21MBULPUlDJ8lLgRdU1VPt/Y8Bvw1cB6wGLmx/P96aXAd8NMn76IZALQFuq6pnkzyV5CRgA3AG8Ac9bVYDtwCnAjd7VlzSbtgb+AHgnVW1Icn7aRNG7cScjUSA/h6NMEhXaiczDPsxDPsAgztqYSoWepKG0QhwbZt/YG/go1X1qSSfA9YlOQt4EDgNoKruTrIOuAfYDpzT7nMBeAdwBbAv3dCnG1r8cuCqNlzqcbp7ZSRpV20GNlfVhvZ5PV2h50gESbvEQk/S0KmqLwOvnyD+GLB8J20uAC6YIH47cOwE8adphaIk7a6q+sckDyV5bVXdR9dX3dNejkSQNGMWepIkaU4tnsFwpT3cO4GPtBk3vwz8HN2jsByJIGnGLPQkSZL6QFXdCSydYJEjESTN2LQfmJ5kryR/l+QT7bMP8JQkSZKkPjTtQo9uit97ez6voXuA5xLgpvaZcQ/wXAFcmmSv1mbsAZ5L2mtFi//zAzyBi+ke4ClJkiRJ2gXTKvSSLALeAnyoJ7yS7sGdtL+n9MSvrqpnquoBYOwBnofSHuDZbvy9clybsW2tB5aPXe2TJEmSJM3MdO/R+33g14CX9cTm/QGePrxz9g1i3ntyzjN50OZ07SyvQfydJUmS1Jmy0EvyVuCRqrojybJpbHPOHuDpwztn3yDmvSfnPJMHbU7Xzh7IOYi/syRJkjrTuaL3BuBtSX4CeDGwf5I/wwd4SpIkSVJfmvIevao6v6oWVdViuklWbq6qn+G5h27C8x/guarNpHkkzz3AcyvwVJKT2v13Z4xrM7YtH+ApSZIkSbthd56jdyE+wFOSJEmS+s6MCr2qGgVG2/vH8AGekiRJktR3ZvIcPUmSJEnSALDQkyRJkqQhY6EnSZIkSUPGQk+SJEmShoyFniRJkiQNGQs9SZIkSRoyFnqSJEmSNGQs9CRJkiRpyFjoSZIk9Ykkm5LcleTOJLe32EFJbkxyf/t7YM/65yfZmOS+JCf3xE9o29mY5JIkafF9klzT4huSLJ73nZQ0Lyz0JEmS+suPVtXxVbW0fV4D3FRVS4Cb2meSHA2sAo4BVgCXJtmrtfkgcDawpL1WtPhZwBNVdRRwMXDRPOyPpAVgoSdJktTfVgJr2/u1wCk98aur6pmqegDYCJyY5FBg/6q6paoKuHJcm7FtrQeWj13tkzRc9l7oBCRprrQz27cDX62qtyY5CLgGWAxsAn6qqp5o655Pd6b7WeBdVfXpFj8BuALYF/gk8O6qqiT70B08nQA8Bry9qjbN285JGlYFfCZJAX9cVZcBI1W1FaCqtiY5pK17OHBrT9vNLfad9n58fKzNQ21b25M8CRwMPNqbRJKz6a4IMjIywujo6Kzt4O7atm1bX+Wzq4ZhP4ZhH2D+9uPc47ZPe93ZyMdCT9IwezdwL7B/+zw2/OnCJGva5/PGDX86DPjrJK+pqmd5bvjTrXSF3grgBnqGPyVZRTf86e3zt2uShtQbqmpLK+ZuTPLFSdad6EpcTRKfrM2Oga7AvAxg6dKltWzZskmTnk+jo6P0Uz67ahj2Yxj2AeZvP85cc/201910+rLd/j6HbkoaSkkWAW8BPtQTdviTpL5WVVva30eAa4ETgYdbf0T7+0hbfTNwRE/zRcCWFl80QXyHNkn2Bg4AHp+LfZG0sCz0JA2r3wd+DfhuT2yH4U9A7/Cnh3rWGxvmdDjTHP4EjA1/kqRdkuSlSV429h74MeAfgOuA1W211cDH2/vrgFVtJs0j6SZdua31b08lOamdgDpjXJuxbZ0K3NxOZEkaMg7dlDR0krwVeKSq7kiybDpNJojNyvCnfr7PBQb3/grznl+7m/dM7kuZrunkM4C/9whwbRscsDfw0ar6VJLPAeuSnAU8CJwGUFV3J1kH3ANsB85pQ84B3sFz9xff0F4AlwNXJdlIdyVv1XzsmKT5Z6EnaRi9AXhbkp8AXgzsn+TPaMOf2mQGszX8afNkw5/6+T4XGNz7K8x7fu1u3jO5L2W6pnP/yqD93lX1ZeD1E8QfA5bvpM0FwAUTxG8Hjp0g/jStUJQ03By6KWnoVNX5VbWoqhbTna2+uap+Boc/SZKkPYRX9CTtSS7E4U+SJGkPYKEn9YHFczCsSZ2qGgVG23uHP0mSpD2CQzclSZIkachY6EmSJEnSkLHQkyRJkqQhY6EnSZIkSUPGQk+SJEmShoyFniRJkiQNGQs9SZIkSRoyFnqSJEmSNGSmLPSSvDjJbUn+PsndSX6rxQ9KcmOS+9vfA3vanJ9kY5L7kpzcEz8hyV1t2SVJ0uL7JLmmxTckWTwH+ypJkiRJe4TpXNF7BnhjVb0eOB5YkeQkYA1wU1UtAW5qn0lyNLAKOAZYAVyaZK+2rQ8CZwNL2mtFi58FPFFVRwEXAxft/q5JkiRJ0p5pykKvOtvaxxe2VwErgbUtvhY4pb1fCVxdVc9U1QPARuDEJIcC+1fVLVVVwJXj2oxtaz2wfOxqnyRJkiRpZvaezkrtitwdwFHAH1bVhiQjVbUVoKq2JjmkrX44cGtP880t9p32fnx8rM1DbVvbkzwJHAw8Oi6Ps+muCDIyMsLo6Og0d3Pubdu2ra/yma5BzHsYcz73uO3zl8w4O8trEH9nSZIkdaZV6FXVs8DxSV4OXJvk2ElWn+hKXE0Sn6zN+DwuAy4DWLp0aS1btmySNObX6Ogo/ZTPdA1i3sOY85lrrp+/ZMbZdPqyCeOD+DtLkiSpM6NZN6vq68Ao3b11D7fhmLS/j7TVNgNH9DRbBGxp8UUTxHdok2Rv4ADg8ZnkJkmSJEnqTGfWzVe2K3kk2Rd4E/BF4DpgdVttNfDx9v46YFWbSfNIuklXbmvDPJ9KclK7/+6McW3GtnUqcHO7j0+SJGmPkWSvJH+X5BPts7OcS9ol07midyjwN0m+AHwOuLGqPgFcCLw5yf3Am9tnqupuYB1wD/Ap4Jw29BPgHcCH6CZo+RJwQ4tfDhycZCPwK7QZPCVJkvYw7wbu7fnsLOeSdsmU9+hV1ReA758g/hiwfCdtLgAumCB+O/C8+/uq6mngtGnkK0mSNJSSLALeQncM9SstvBJY1t6vpbuF5jx6ZjkHHmgny09Msok2y3nb5tgs5ze0Nu9t21oPfCBJHEUlDadpTcYiSZKkOff7wK8BL+uJzfss5+BM5/NhGPZjGPYB5m8/ZjLL+mzkY6EnSZK0wJK8FXikqu5Ismw6TSaIzcos5+BM5/NhGPZjGPYB5m8/ZjLL+s5mRZ8JCz1JkqSF9wbgbUl+AngxsH+SP6PNct6u5s3WLOebneVcGn4zeryCJEmSZl9VnV9Vi6pqMd0kKzdX1c/gLOeSdpFX9CRJkvrXhcC6JGcBD9Imr6uqu5OMzXK+nefPcn4FsC/dJCy9s5xf1SZueZyuoJQ0pCz0JEmS+khVjdLNruks51KfWzyD++7mm0M3JQ2dJC9OcluSv09yd5LfanEfPCxJkvYIFnqShtEzwBur6vXA8cCKJCfhg4clSdIewkJP0tCpzrb28YXtVXQPC17b4mvpHiIMPQ8erqoHgLEHDx9Ke/Bwm7DgynFtxra1Hlg+drVPkiRpoXmPnqSh1K7I3QEcBfxhVW1IMu8PHu7nhw7D4D7s1rzn1+7mPZOHBE/XdPIZ1N9bkmaDhZ6kodRmnzs+ycuBa5M8b2KCHnP24OF+fugwDO7Dbs17fk2U98wmIJj9w43pPEx4UH9vSZoNDt2UNNSq6ut0s9etoD14GGAWHzyMDx6WJEn9xkJP0tBJ8sp2JY8k+wJvAr6IDx6WJEl7CIduShpGhwJr2316LwDWVdUnktyCDx6WJEl7AAs9SUOnqr4AfP8EcR88LEmS9ggO3ZQkSZKkIeMVPWmO9M5Id+5x2zlzRjPUSZIkSbvOK3qSJEmSNGQs9CRJkiRpyFjoSZIkSdKQsdCTJEmSpCFjoSdJkiRJQ8ZCT5IkSZKGjIWeJEnSAkvy4iS3Jfn7JHcn+a0WPyjJjUnub38P7GlzfpKNSe5LcnJP/IQkd7VllyRJi++T5JoW35Bk8bzvqKR5Y6EnSZK08J4B3lhVrweOB1YkOQlYA9xUVUuAm9pnkhwNrAKOAVYAlybZq23rg8DZwJL2WtHiZwFPVNVRwMXARfOwX5IWiIWeJEnSAqvOtvbxhe1VwEpgbYuvBU5p71cCV1fVM1X1ALARODHJocD+VXVLVRVw5bg2Y9taDywfu9onafjsvdAJSJIkCdoVuTuAo4A/rKoNSUaqaitAVW1Nckhb/XDg1p7mm1vsO+39+PhYm4fatrYneRI4GHh0glzOprsqyMjICKOjo7Oyj7Nh27ZtfZXPrhqG/RiGfYDd249zj9s+u8k0s/G7TlnoJTmC7mzQ9wDfBS6rqvcnOQi4BlgMbAJ+qqqeaG3Opxse8Czwrqr6dIufAFwB7At8Enh3VVWSfdp3nAA8Bry9qjbt9t5JkiQNiKp6Fjg+ycuBa5McO8nqE12Jq0nik7WZKJfLgMsAli5dWsuWLZsklfk1OjpKP+Wzq4ZhP4ZhH2D39uPMNdfPbjLNptOX7fY2pjN0cztwblW9DjgJOKeNC3fMuCRJ0iyrqq8Do3THSQ+34Zi0v4+01TYDR/Q0WwRsafFFE8R3aJNkb+AA4PG52AdJC2/KQq+qtlbV59v7p4B76S79O2ZckiRpFiR5ZbuSR5J9gTcBXwSuA1a31VYDH2/vrwNWtZk0j6Q7gX5bG+b5VJKT2rHUGePajG3rVODmdkwmaQjN6B69Ng3v9wMbgAUZMy5JkjSEDgXWtlFQLwDWVdUnktwCrEtyFvAgcBpAVd2dZB1wD93oq3Pa0E+Ad/DcrTI3tBfA5cBVSTbSXclbNS97JmlBTLvQS7If8OfAL1XVNya54DZnY8a9MXj2DWLeg5Jz7825I/vO3c26u2tnv+Wg/M6SNAyq6gt0J9PHxx8Dlu+kzQXABRPEbweed39fVT1NKxQlDb9pFXpJXkhX5H2kqv6ihR9Ocmi7mjdbY8Y3TzZm3BuDZ98g5j0oOffenHvucdv5vbv6c5Lbnd3sOyi/syRJkp5vynv02vjuy4F7q+p9PYscMy5JkiRJfWg6lxjeAPwscFeSO1vs14ELccy4JEmSJPWdKQu9qvosE99DB44ZlyRJkqS+0583DUmSpHm3eCcP/j33uO1z9lBgSdLcsNCTZmhnB0KSJEkaDsNwvDflZCySNGiSHJHkb5Lcm+TuJO9u8YOS3Jjk/vb3wJ425yfZmOS+JCf3xE9IcldbdkmbTIo24dQ1Lb6hPWdUkiSpL1joSRpG24Fzq+p1wEnAOUmOBtYAN1XVEuCm9pm2bBVwDLACuLQ9tBjgg3TP71zSXita/Czgiao6CrgYuGg+dkySJGk6LPQkDZ2q2lpVn2/vnwLuBQ4HVgJr22prgVPa+5XA1VX1TFU9AGwETmzPCN2/qm5pj3y5clybsW2tB5aPXe2TJElaaN6jJ2motSGV3w9sAEbaMz2pqq1JDmmrHQ7c2tNsc4t9p70fHx9r81Db1vYkTwIHA4+O+/6z6a4IMjIywujo6Gzt2qzYtm1b3+U0HeY9N849bvuE8ZF9d75soUznd+z331uS5pKFnqShlWQ/4M+BX6qqb0xywW2iBTVJfLI2OwaqLgMuA1i6dGktW7Zsiqzn1+joKP2W03SY99zY2cya5x63nd+7q78OGTadvmzKdfr995akueTQTUlDKckL6Yq8j1TVX7Tww204Ju3vIy2+GTiip/kiYEuLL5ogvkObJHsDBwCPz/6eSJIkzZyFnqSh0+6Vuxy4t6re17PoOmB1e78a+HhPfFWbSfNIuklXbmvDPJ9KclLb5hnj2oxt61Tg5nYfnyRJ0oLrr3EYkjQ73gD8LHBXkjtb7NeBC4F1Sc4CHgROA6iqu5OsA+6hm7HznKp6trV7B3AFsC9wQ3tBV0helWQj3ZW8VXO8T5IkSdNmoSdp6FTVZ5n4HjqA5TtpcwFwwQTx24FjJ4g/TSsUJUmS+o1DNyVJkiRpyHhFTwIW72SmOUmSJGkQWehJkjTkPJnV/5IcAVwJfA/wXeCyqnp/koOAa4DFwCbgp6rqidbmfOAs4FngXVX16RY/gefuLf4k8O6qqiT7tO84AXgMeHtVbZqnXZQ0zyz0JEmSFt524Nyq+nySlwF3JLkROBO4qaouTLIGWAOcl+RoukmgjgEOA/46yWvaRFIfBM4GbqUr9FbQTSR1FvBEVR2VZBVwEfD2ed1LaQ5M92TWpgvfMseZ9BcLPUmSpAXWHueytb1/Ksm9wOHASmBZW20tMAqc1+JXV9UzwANtBuATk2wC9q+qWwCSXAmcQlforQTe27a1HvhAkvhoGO0pdlYQnnvcds4cwpEPFnqSJEl9JMli4PuBDcBIKwKpqq1JDmmrHU53xW7M5hb7Tns/Pj7W5qG2re1JngQOBh6dIIez6a4KMjIywujo6Gzs2qzYtm1bX+Wzq4ZhP/plH849bvtutR/Zd/e3Mdtm43e10JMkSeoTSfYD/hz4par6RrKzJ8VM+AiZmiQ+WZvnB6suAy4DWLp0aS1btmySrOfX6Ogo/ZTPrhqG/eiXfdjdq3HnHred37urv8qiTacv2+1t+HgFSZKkPpDkhXRF3keq6i9a+OEkh7blhwKPtPhm4Iie5ouALS2+aIL4Dm2S7A0cADw++3siqR9Y6EmSJC2wdJfuLgfurar39Sy6Dljd3q8GPt4TX5VknyRHAkuA29owz6eSnNS2eca4NmPbOhW42fvzpOHVX9coJUmS9kxvAH4WuCvJnS3268CFwLokZwEPAqcBVNXdSdYB99DN2HlOm3ET4B0893iFG9oLukLyqjZxy+N0s3ZKGlIWepIkSQusqj7LxPfQASzfSZsLgAsmiN8OHDtB/GlaoShp+Dl0U5IkSZKGjIWeJEmSJA0Zh25KkjSAdvbgX0mSwCt6kiRJkjR0LPQkSZIkachY6EmSJEnSkJnyHr0kHwbeCjxSVce22EHANcBiYBPwU1X1RFt2PnAW8Czwrqr6dIufwHPPdPkk8O6qqiT7AFcCJwCPAW+vqk2ztofaY3n/iiRJkvZU07midwWwYlxsDXBTVS0BbmqfSXI03cM3j2ltLk2yV2vzQeBsYEl7jW3zLOCJqjoKuBi4aFd3RpIkSZI0jUKvqv4WeHxceCWwtr1fC5zSE7+6qp6pqgeAjcCJSQ4F9q+qW6qq6K7gnTLBttYDy5Ps7IGhkiRJkqQp7OrjFUaqaitAVW1NckiLHw7c2rPe5hb7Tns/Pj7W5qG2re1JngQOBh4d/6VJzqa7KsjIyAijo6O7mP7s27ZtW1/lM12DmPd0cz73uO1zn8w0jezbX/n02tlvOYj/NiRJktSZ7efoTXQlriaJT9bm+cGqy4DLAJYuXVrLli3bhRTnxujoKP2Uz3QNYt7TzfnMPrpH79zjtvN7d/XnYys3nb5swvgg/tuQJElSZ1ePPB9Ocmi7mnco8EiLbwaO6FlvEbClxRdNEO9tsznJ3sABPH+oqCTNiBNJSZI0uJxUb/ft6uMVrgNWt/ergY/3xFcl2SfJkXSTrtzWhnk+leSkdv/dGePajG3rVODmdh+fJO2OK3AiKUmStIeastBL8jHgFuC1STYnOQu4EHhzkvuBN7fPVNXdwDrgHuBTwDlV9Wzb1DuAD9FN0PIl4IYWvxw4OMlG4FdoB16StDucSEqSJO3Jphy6WVU/vZNFy3ey/gXABRPEbweOnSD+NHDaVHlI0ixYkImkJEmS5lt/zg4hTeKurz7ZVxOtaCjM2URS/TxbMAzu7KrDnPddX31yWts697hZSGia+nHm4On89x/UfyeSNBss9CTtSeZ9Iql+ni0YBnd21WHOux9PZPXjzME7mzG416D+O5Gk2bCrk7FI0iByIilJfSvJh5M8kuQfemIHJbkxyf3t74E9y85PsjHJfUlO7omfkOSutuySsfuHWx93TYtvSLJ4XndQ0rzqr9Nz2mPNZArd+RyupMHVJpJaBrwiyWbgPXQTR61rk0o9SLs/uKruTjI2kdR2nj+R1BV0j1e4gR0nkrqqTST1ON2snZK0O64APkA38dOYsdmCL0yypn0+b9xswYcBf53kNa3vGpst+Fa6x8KsoOu7/nm24CSr6GYLfvu87JmkeWehJ2koOZGUpEFTVX87wVW2lXQnraCb6XcUOI+e2YKBB9pJpxOTbKLNFgyQZGy24Btam/e2ba0HPpAkjkaQhpOFniRJc2xs1MK5x23vy3vw1NcWZLbgfp5Ialgm2RmG/ZjLfZjPCaAGdcKpqVjoSZIkDZ45my0Y+nsiqWGZZGcY9mMu92E+T4oN6oRTU3EyFkmSpP71cJslmFmcLZjJZguWNBz6q3TV0JnJJCuSNGjs4zQPxmb4vZDnzxb80STvo5uMZWy24GeTPJXkJGAD3WzBfzBuW7fgbMFaIPab88dCT5IkqQ84W7Ck2WShJ0mS1AecLVjSbPIePUmSJEkaMl7RkySph/ePSNLM2G/2J6/oSZIkSdKQ8YqeZsyzNpIkScNtOsd75x63fV6fd6eZ8YqeJEmSJA0Zr+hJkiRJewhHZu05vKInSZIkSUPGK3r6Z57hkTSs7N8kSXsaCz1JkiRpgHkySxOx0JMkSQNnujMCLpv7VCSpL1noDbmp/o/QaXElDTLPYksaZvZx2h0WegPI/9FLkiQNprHjOE+2a65Z6EmS+ooP6ZUkafdZ6EmS5oWjESRJmj8Wen3EgyBJkqTB5HGc+o2FniRpl3lgI2mY2cdpkFnozTE7CEmSpP7hsZn2FH1T6CVZAbwf2Av4UFVduMApSdKUhrXv8kBIGl7D2m9J2lFfFHpJ9gL+EHgzsBn4XJLrquqehc1s58YfBDkDnAbVzg7oJ/o3venCt8xHSgNjEPsuSXu2Ye63PEEl7agvCj3gRGBjVX0ZIMnVwEpg4DsdSUNtoPquiQ6CPEkl7XHst6Q9RKpqoXMgyanAiqr69+3zzwI/WFW/OG69s4Gz28fXAvfNa6KTewXw6EInsQsGMW9znh+7m/OrquqVs5VMP5pO39Xn/RYM5r9NMO/5tqfkbb/13Hr93HcN6r/H8YZhP4ZhH2Cw92On/Va/XNHLBLHnVaBVdRlw2dynM3NJbq+qpQudx0wNYt7mPD8GMecFMGXf1c/9Fgzuf2fznl/mPVQ85uoTw7Afw7APMDz7Md4LFjqBZjNwRM/nRcCWBcpFkqbLvkvSoLHfkvYQ/VLofQ5YkuTIJC8CVgHXLXBOkjQV+y5Jg8Z+S9pD9MXQzaranuQXgU/TTfX74aq6e4HTmqm+HN4wDYOYtznPj0HMeV7Zdy0o855f5j0k7Lf6yjDsxzDsAwzPfuygLyZjkSRJkiTNnn4ZuilJkiRJmiUWepIkSZI0ZCz0dlOS/5LkC0nuTPKZJIf1LDs/ycYk9yU5eSHz7JXkvyf5Ysv72iQv71nWlzkDJDktyd1Jvptk6bhl/Zz3ipbXxiRrFjqfiST5cJJHkvxDT+ygJDcmub/9PXAhc9TsGsS+C+y/FsIg9GFgP7YnGNR+q9eg9mHjDXKf1mtQ+rddVlW+duMF7N/z/l3AH7X3RwN/D+wDHAl8CdhrofNtuf0YsHd7fxFwUb/n3PJ7Hd1DW0eBpT3xvs2b7kb3LwGvBl7U8jx6ofOaIM8fAX4A+Iee2O8Ca9r7NWP/TnwNx2sQ+66Wn/3X/OY9EH1Yy9V+bMhfg9pvjduHgezDJtiPgezTxu3DwPRvu/ryit5uqqpv9Hx8Kc89dHQlcHVVPVNVDwAbgRPnO7+JVNVnqmp7+3gr3TN0oI9zBqiqe6vqvgkW9XPeJwIbq+rLVfVt4Gq6fPtKVf0t8Pi48EpgbXu/FjhlPnPS3BrEvgvsvxbAQPRhYD+2JxjUfqvXoPZh4w1wn9ZrYPq3XWWhNwuSXJDkIeB04D+38OHAQz2rbW6xfvPvgBva+0HJebx+zrufc5vKSFVtBWh/D1ngfDTLBrzvAvuv+dDv+U3FfmzIDEG/1WsY+rDxBmk/BinXXdIXz9Hrd0n+GvieCRb9RlV9vKp+A/iNJOcDvwi8B8gE68/bsyymyrmt8xvAduAjY80mWH9en78xnbwnajZBrF+eG9LPuWnIDWLfBfZf9Fcf0e/5acgMar/Va1D7sPGGtE/rNUi57hILvWmoqjdNc9WPAtfTdTqbgSN6li0Ctsxyajs1Vc5JVgNvBZZXG6jMAucMM/qtey143pPo59ym8nCSQ6tqa5JDgUcWOiHNzCD2XWD/RX/1Ef2e31TsxwbMoPZbvQa1DxtvSPu0XoOU6y5x6OZuSrKk5+PbgC+299cBq5Lsk+RIYAlw23znN5EkK4DzgLdV1bd6FvVtzlPo57w/ByxJcmSSFwGr6PIdBNcBq9v71cDOzt5pAA1i3wX2XwtgkPswsB8bKoPab/Uawj5svEHaj0Hv36bkFb3dd2GS1wLfBb4C/AJAVd2dZB1wD92l+XOq6tmFS3MHH6CbDenGJAC3VtUv9HnOJPk3wB8ArwSuT3JnVZ3cz3lX1fYkvwh8mm52pw9X1d0LnNbzJPkYsAx4RZLNdGdILwTWJTkLeBA4beEy1BwYxL4L7L/m1aD0YWA/tocY1H6r10D2YeMNap/Wa5D6t12V564YS5IkSZKGgUM3JUmSJGnIWOhJkiRJ0pCx0JMkSZKkIWOhJ0mSJElDxkJPkiRJkoaMhZ4kSZIkDRkLPUmSJEkaMhZ6kiRJkjRkLPQkSZIkachY6EmSJEnSkLHQkyRJkqQhY6EnSZIkSUPGQk+SJEmShoyFniRJkiQNGQs9SZIkSRoyFnqSJEmSNGQs9CRJkiRpyFjoSZIkSdKQsdCTJEmSpCFjoSdJkiRJQ8ZCT5IkSZKGjIWeJEmSJA0ZCz1JkiRJGjIWepIkSZI0ZCz0JEmSJGnIWOhJkiRJ0pCx0JMkSZKkIWOhJ0mSJElDxkJPkiRJkoaMhZ4kSZIkDRkLPUmSJEkaMhZ6kiRJkjRkLPQkSZIkachY6EmSJEnSkLHQkyRJkqQhY6EnSZIkSUPGQk+SJEmShoyFnnZJkruTLFvoPGZLkv+a5NEk/7jQuUiSJEm7K1W10DlogSXZBIwAzwLfBD4JvLOqti1kXvMlyRHA/wFeVVWPLHQ+kiRJ0u7yip7G/GRV7Qf8APB/Ab+5wPnMp1cBj1nkSZIkaVhY6GkHVfVV4Abg2CRva0M0v55kNMnrxtZLsinJm9r7E5PcnuQbSR5O8r4Wf3GSP0vyWNvG55KMtGWHJbkuyeNJNib5+Z5tvzfJuiRXJnmq5bB0qtyTvK7l+fXW5m09yw5o2/takq8k+c0kL2j7cCNwWJJtSa6YpZ9SkiRJWjAWetpBG8b4E8BTwMeAXwJeSTec86+SvGiCZu8H3l9V+wP/AljX4quBA4D/f3v3H2x3Xed3/PlqosigKD/kbiSsYYfUFoi6NUNp7bR3m22J1RXagWkclLBNJ6NFx04zs4Y6U3enkxa63dplXZzJrJbIqpihtaRSVrOht45TfggtXQw/SkYQskmhCCJxC3Lx3T/O59aTy8m95+bHved+7/Mxc+Z8z/t8Pt/7/h7CH6/5fM/nnAOcAXwU+L/tva8C+4G3AZcD/yLJur5zfhC4BXgLsAv43Cx9vw74T8C3gLOATwBfTvKONuT3Wi+/BPwN4Crg16vqj4H3AQeq6o1VdfVMf0eSJElaDAx6mvIfk/wI+A7wX4GHgNurandVvQL8a+Bk4K8OmPsKcF6SM6vqUFXd3Vc/Azivql6tqvur6sctTP414FNV9VJVPQD8AfCRvnN+p6r+c1W9CtwMvGuW/i8G3ghcV1U/rao7gW8AH0qyDPj7wLVV9WJVPQH8zrS/J0mSJHWGQU9TLquqt1TV26vqH9FbafvB1JtV9TPgKeDsAXM3AX8eeKTdnvmBVr8Z+CZwS5IDSf5VW3l7G/BcVb3Yd44fTDt3/+6Xfwa8IcnyGfp/G/BU63P6Oc8EXt9/PQP+niRJktQZBj0dyQF6m5QAkCT0bsH80+kDq+qxqvoQvVsmrwduTXJKVb1SVb9VVefTWwn8AL1bJg8Apyd5U99pfnHQuefY7zlJ+v9NT53zWXqri28f8J4kSZLUOQY9HclO4P1J1rVVuC3Ay8B/mz4wyYeTvLWtpv2olV9N8itJ1rRbJ39ML2y9WlVPtfP8y7ZhyzvprQp++Rj6vYfeT0P8RpLXtd/4+zXglnb7505gW5I3JXk78E+APzyGvydJkiSNLIOeBqqqR4EP09vE5Fl6oenXquqnA4avB/YmOURvY5YNVfUS8AvArfRC3sP0vvs3Fa4+BKyitxL3deAzVbX7GPr9Kb0NXN7X+r0RuKqqHmlDPkEvCH6f3vcQvwJ88Wj/niRJkjTK/MF0SZIkSeoYV/QkSZIkqWNm2sVQGilJfpHezz4Mcn5VPTmf/UiSJEmjyls3JUmSJKljFu2K3plnnlmrVq1a6DYO85Of/IRTTjlloduYM/ueP4uxZzi6vu+///5nq+qtJ6glSZIkzWDRBr1Vq1Zx3333LXQbh5mYmGB8fHyh25gz+54/i7FnOLq+k/xg9lGSJEk6EdyMRZIkSZI6xqAnSZIkSR1j0JMkSZKkjjHoSZIkSVLHGPQkSZIkqWMMepIkSZLUMQY9SZIkSeoYg54kSZIkdYxBT5IkSZI6ZvlCNyDpxFi19fahxj1x3ftPcCeSJEmab67oSZIkSVLHGPQkSZIkqWMMepIkSZLUMQY9SZIkSeoYg54kSZIkdYxBT5IkSZI6xqAnSZIkSR1j0JMkSZKkjjHoSZIkSVLHGPQkSZIkqWMMepIkSZLUMUMFvSRvSXJrkkeSPJzkryQ5PcnuJI+159P6xl+bZF+SR5Nc0ld/T5IH23s3JEmrn5Tka61+T5JVx/1KJUmSJGmJGHZF73eBP6qqvwC8C3gY2ArsqarVwJ72miTnAxuAC4D1wI1JlrXzfB7YDKxuj/Wtvgl4vqrOAz4LXH+M1yVJkiRJS9asQS/JqcBfB74AUFU/raofAZcCO9qwHcBl7fhS4JaqermqHgf2ARclWQGcWlV3VVUBX5o2Z+pctwLrplb7JEmSJElzs3yIMb8E/B/g3yV5F3A/8ElgrKoOAlTVwSRntfFnA3f3zd/faq+04+n1qTlPtXNNJnkBOAN4tr+RJJvprQgyNjbGxMTEcFc5Tw4dOjRyPQ3DvufPfPa8Zc3kUOOG6WcxftaSJElL2TBBbznwl4BPVNU9SX6XdpvmEQxaiasZ6jPNObxQtR3YDrB27doaHx+foY35NzExwaj1NAz7nj/z2fPVW28fatwTV47POmYxftaSJElL2TDf0dsP7K+qe9rrW+kFv6fb7Zi052f6xp/TN38lcKDVVw6oHzYnyXLgzcBzc70YSZIkSdIQQa+q/jfwVJJ3tNI64CFgF7Cx1TYCt7XjXcCGtpPmufQ2Xbm33eb5YpKL2/fvrpo2Z+pclwN3tu/xSZIkSZLmaJhbNwE+AXw5yeuB7wO/Ti8k7kyyCXgSuAKgqvYm2UkvDE4C11TVq+08HwNuAk4G7mgP6G30cnOSffRW8jYc43VJkiRJ0pI1VNCrqgeAtQPeWneE8duAbQPq9wEXDqi/RAuKkiRJkqRjM+zv6EmSJEmSFgmDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOMehJkiRJUscY9CRJkiSpYwx6kiRJktQxBj1JkiRJ6hiDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOMehJkiRJUscY9CRJkiSpYwx6kiRJktQxBj1JkiRJ6hiDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOMehJkiRJUscY9CRJkiSpYwx6kiRJktQxQwW9JE8keTDJA0nua7XTk+xO8lh7Pq1v/LVJ9iV5NMklffX3tPPsS3JDkrT6SUm+1ur3JFl1nK9TkiRJkpaMuazo/UpVvbuq1rbXW4E9VbUa2NNek+R8YANwAbAeuDHJsjbn88BmYHV7rG/1TcDzVXUe8Fng+qO/JEmSJEla2pYfw9xLgfF2vAOYAD7V6rdU1cvA40n2ARcleQI4taruAkjyJeAy4I425zfbuW4FPpckVVXH0J/UOau23r7QLUiSJGkRGHZFr4BvJbk/yeZWG6uqgwDt+axWPxt4qm/u/lY7ux1Prx82p6omgReAM+Z2KZIkSZIkGH5F771VdSDJWcDuJI/MMDYDajVDfaY5h5+4FzI3A4yNjTExMTFj0/Pt0KFDI9fTMOx7/hxrz1vWTB6/Zpph+lmMn7UkSdJSNlTQq6oD7fmZJF8HLgKeTrKiqg4mWQE804bvB87pm74SONDqKwfU++fsT7IceDPw3IA+tgPbAdauXVvj4+PDtD9vJiYmGLWehmHf8+dYe776BNy6+cSV47OOWYyftSRJ0lI2662bSU5J8qapY+BvA98DdgEb27CNwG3teBewoe2keS69TVfubbd3vpjk4rbb5lXT5kyd63LgTr+fJ0mSJElHZ5gVvTHg6+2XEJYDX6mqP0ryXWBnkk3Ak8AVAFW1N8lO4CFgErimql5t5/oYcBNwMr1NWO5o9S8AN7eNW56jt2unJEmSJOkozBr0qur7wLsG1H8IrDvCnG3AtgH1+4ALB9RfogVFSZIkSdKxmcvv6EmSJEmSFgGDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOMehJkiRJUscY9CRJkiSpYwx6kiRJktQxBj1JkiRJ6hiDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOMehJkiRJUscY9CRJkiSpYwx6kiRJktQxBj1JkiRJ6hiDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOMehJkiRJUscY9CRJkiSpYwx6kiRJktQxBj1JkiRJ6pihg16SZUn+R5JvtNenJ9md5LH2fFrf2GuT7EvyaJJL+urvSfJge++GJGn1k5J8rdXvSbLqOF6jJEmSJC0pc1nR+yTwcN/rrcCeqloN7GmvSXI+sAG4AFgP3JhkWZvzeWAzsLo91rf6JuD5qjoP+Cxw/VFdjSRJkiRpuKCXZCXwfuAP+sqXAjva8Q7gsr76LVX1clU9DuwDLkqyAji1qu6qqgK+NG3O1LluBdZNrfZJkiRJkuZm+ZDj/i3wG8Cb+mpjVXUQoKoOJjmr1c8G7u4bt7/VXmnH0+tTc55q55pM8gJwBvBsfxNJNtNbEWRsbIyJiYkh258fhw4dGrmehmHf8+dYe96yZvL4NdMM089i/KwlSZKWslmDXpIPAM9U1f1Jxoc456CVuJqhPtOcwwtV24HtAGvXrq3x8WHamT8TExOMWk/DsO/5c6w9X7319uPXTPPEleOzjlmMn7UkSdJSNsyK3nuBDyb5O8AbgFOT/CHwdJIVbTVvBfBMG78fOKdv/krgQKuvHFDvn7M/yXLgzcBzR3lNkiRJkrSkzfodvaq6tqpWVtUqepus3FlVHwZ2ARvbsI3Abe14F7Ch7aR5Lr1NV+5tt3m+mOTi9v27q6bNmTrX5e1vvGZFT5IkSZI0u2G/ozfIdcDOJJuAJ4ErAKpqb5KdwEPAJHBNVb3a5nwMuAk4GbijPQC+ANycZB+9lbwNx9CXJEmSJC1pcwp6VTUBTLTjHwLrjjBuG7BtQP0+4MIB9ZdoQVGSJEmSdGzm8jt6kiRJkqRFwKAnSZIkSR1j0JMkSZKkjjHoSZIkSVLHGPQkSZIkqWMMepIkSZLUMQY9SZIkSeoYg54kSZIkdYxBT5IkSZI6xqAnSZIkSR1j0JMkSZKkjjHoSZIkSVLHGPQkSZIkqWMMepIkSZLUMQY9SZIkSeoYg54kSZIkdYxBT5IkSZI6xqAnSZIkSR1j0JMkSZKkjjHoSZIkSVLHGPQkSZIkqWMMepIkSZLUMQY9SZIkSeoYg54kSZIkdcysQS/JG5Lcm+R/Jtmb5Lda/fQku5M81p5P65tzbZJ9SR5Ncklf/T1JHmzv3ZAkrX5Skq+1+j1JVp2Aa5UkSZKkJWGYFb2Xgb9ZVe8C3g2sT3IxsBXYU1WrgT3tNUnOBzYAFwDrgRuTLGvn+jywGVjdHutbfRPwfFWdB3wWuP7YL02SJEmSlqblsw2oqgIOtZeva48CLgXGW30HMAF8qtVvqaqXgceT7AMuSvIEcGpV3QWQ5EvAZcAdbc5vtnPdCnwuSdrfljpv1dbbF7oFSZIkdchQ39FLsizJA8AzwO6qugcYq6qDAO35rDb8bOCpvun7W+3sdjy9fticqpoEXgDOOIrrkSRJkqQlb9YVPYCqehV4d5K3AF9PcuEMwzPoFDPUZ5pz+ImTzfRu/WRsbIyJiYkZ2ph/hw4dGrmehmHf8+dIPW9ZMzn/zTTDfIaL8bOWJElayoYKelOq6kdJJuh9t+7pJCuq6mCSFfRW+6C3UndO37SVwIFWXzmg3j9nf5LlwJuB5wb8/e3AdoC1a9fW+Pj4XNo/4SYmJhi1noZh3/PnSD1fvYC3bj5x5fisYxbjZy1JkrSUDbPr5lvbSh5JTgZ+FXgE2AVsbMM2Are1413AhraT5rn0Nl25t93e+WKSi9tum1dNmzN1rsuBO/1+niRJkiQdnWFW9FYAO9rOmX8O2FlV30hyF7AzySbgSeAKgKram2Qn8BAwCVzTbv0E+BhwE3AyvU1Y7mj1LwA3t41bnqO3a6ckSZIk6SgMs+vmnwC/PKD+Q2DdEeZsA7YNqN8HvOb7fVX1Ei0oSpIkSZKOzVC7bkqSJEmSFg+DniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOMehJkiRJUscY9CRJkiSpYwx6kiRJktQxBj1JkiRJ6hiDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOMehJkiRJUscY9CRJkiSpYwx6kiRJktQxBj1JkiRJ6hiDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSOWb7QDUhdtWrr7a+pbVkzydUD6pIkSdLx5IqeJEmSJHWMQU+SJEmSOsagJ0mSJEkdM2vQS3JOkv+S5OEke5N8stVPT7I7yWPt+bS+Odcm2Zfk0SSX9NXfk+TB9t4NSdLqJyX5Wqvfk2TVCbhWSZIkSVoShlnRmwS2VNVfBC4GrklyPrAV2FNVq4E97TXtvQ3ABcB64MYky9q5Pg9sBla3x/pW3wQ8X1XnAZ8Frj8O1yZJkiRJS9KsQa+qDlbVf2/HLwIPA2cDlwI72rAdwGXt+FLglqp6uaoeB/YBFyVZAZxaVXdVVQFfmjZn6ly3AuumVvskSZIkSXMzp59XaLdU/jJwDzBWVQehFwaTnNWGnQ3c3Tdtf6u90o6n16fmPNXONZnkBeAM4Nlpf38zvRVBxsbGmJiYmEv7J9yhQ4dGrqdh2PeJsWXN5GtqYycPri+kYT7DUf+sJUmSdLihg16SNwL/HvjHVfXjGRbcBr1RM9RnmnN4oWo7sB1g7dq1NT4+PkvX82tiYoJR62kY9n1iDPq9vC1rJvmdB0fr5yufuHJ81jGj/llLkiTpcEPtupnkdfRC3per6j+08tPtdkza8zOtvh84p2/6SuBAq68cUD9sTpLlwJuB5+Z6MZIkSZKk4XbdDPAF4OGq+jd9b+0CNrbjjcBtffUNbSfNc+ltunJvu83zxSQXt3NeNW3O1LkuB+5s3+OTJEmSJM3RMPeQvRf4CPBgkgda7Z8C1wE7k2wCngSuAKiqvUl2Ag/R27Hzmqp6tc37GHATcDJwR3tAL0jenGQfvZW8Dcd2WZIkSZK0dM0a9KrqOwz+Dh3AuiPM2QZsG1C/D7hwQP0lWlCUJEmSJB2bob6jJ0mSJElaPAx6kiRJktQxo7XPu7QIrBrwswmSJEnSKHFFT5IkSZI6xqAnSZIkSR1j0JMkSZKkjjHoSZIkSVLHGPQkSZIkqWMMepIkSZLUMQY9SZIkSeoYg54kSZIkdYxBT5IkSZI6xqAnSZIkSR2zfKEbkEbBqq23L3QLkiRJ0nHjip4kSZIkdYxBT5IkSZI6xqAnSZIkSR1j0JMkSZKkjjHoSZIkSVLHGPQkSZIkqWMMepIkSZLUMQY9SZIkSeoYg54kSZIkdYxBT5IkSZI6Ztagl+SLSZ5J8r2+2ulJdid5rD2f1vfetUn2JXk0ySV99fckebC9d0OStPpJSb7W6vckWXWcr1GSJEmSlpRhVvRuAtZPq20F9lTVamBPe02S84ENwAVtzo1JlrU5nwc2A6vbY+qcm4Dnq+o84LPA9Ud7MZIkSZKkIYJeVX0beG5a+VJgRzveAVzWV7+lql6uqseBfcBFSVYAp1bVXVVVwJemzZk6163AuqnVPkmSJEnS3C0/ynljVXUQoKoOJjmr1c8G7u4bt7/VXmnH0+tTc55q55pM8gJwBvDs9D+aZDO9VUHGxsaYmJg4yvZPjEOHDo1cT8Poat8P/ukLQ59ry5rj0NAQxk6GLWsm5+ePDWmY//aL9d+IJEnSUnW0Qe9IBq3E1Qz1mea8tli1HdgOsHbt2hofHz+KFk+ciYkJRq2nYXS176u33j5/zQxpy5pJfufB4/2/3bF54srxWccs1n8jkiRJS9XR7rr5dLsdk/b8TKvvB87pG7cSONDqKwfUD5uTZDnwZl57q6gkSZIkaUhHG/R2ARvb8Ubgtr76hraT5rn0Nl25t93m+WKSi9v3766aNmfqXJcDd7bv8UmSJEmSjsKs95Al+SowDpyZZD/wGeA6YGeSTcCTwBUAVbU3yU7gIWASuKaqXm2n+hi9HTxPBu5oD4AvADcn2UdvJW/DcbkySZIkSVqiZg16VfWhI7y17gjjtwHbBtTvAy4cUH+JFhQlSZIkScdutHaFkIawqm2ysmXN5EhuuCJJkiQttKP9jp4kSZIkaUQZ9CRJkiSpY7x1UyNhlbdgSpIkSceNK3qSJEmS1DEGPUmSJEnqGIOeJEmSJHWMQU+SJEmSOsbNWHRCucmKJEmSNP9c0ZMkSZKkjjHoSZIkSVLHGPQkSZIkqWMMepIkSZLUMW7GojlzgxVJkiRptLmiJ0mSJEkdY9CTJEmSpI7x1k39f96SKUmSJHWDK3qSJEmS1DEGPUmSJEnqGG/d7LhhbsfcsmaSq71tU5IkSeoMg94i5HfpJEmSJM3EoCctcXNZ9X3iuvfPQ0eSJEk6Vga9EeJKnSRJkqTjwc1YJEmSJKljXNE7wVylkyRJkjTfRmZFL8n6JI8m2Zdk60L3I0mSJEmL1Uis6CVZBvw+8LeA/cB3k+yqqocWtrMjG7RS588USJIkSRoFIxH0gIuAfVX1fYAktwCXAvMa9LzNUpIkSVIXpKoWugeSXA6sr6p/2F5/BPjLVfXxaeM2A5vby3cAj85ro7M7E3h2oZs4CvY9fxZjz3B0fb+9qt56IpqRJEnSzEZlRS8Daq9JoFW1Hdh+4ts5Oknuq6q1C93HXNn3/FmMPcPi7VuSJGmpGpXNWPYD5/S9XgkcWKBeJEmSJGlRG5Wg911gdZJzk7we2ADsWuCeJEmSJGlRGolbN6tqMsnHgW8Cy4AvVtXeBW7raIzsbaWzsO/5sxh7hsXbtyRJ0pI0EpuxSJIkSZKOn1G5dVOSJEmSdJwY9CRJkiSpYwx6x0GSf57kT5I8kORbSd7W9961SfYleTTJJQvZ53RJfjvJI633ryd5S997I9l3kiuS7E3ysyRrp703kj1PSbK+9bYvydaF7udIknwxyTNJvtdXOz3J7iSPtefTFrJHSZIkzcygd3z8dlW9s6reDXwD+GcASc6nt4PoBcB64MYkyxasy9faDVxYVe8E/hdwLYx8398D/h7w7f7iiPdM6+X3gfcB5wMfaj2PopvofYb9tgJ7qmo1sKe9liRJ0ogy6B0HVfXjvpen8PMfe78UuKWqXq6qx4F9wEXz3d+RVNW3qmqyvbyb3u8Xwgj3XVUPV9WjA94a2Z6bi4B9VfX9qvopcAu9nkdOVX0beG5a+VJgRzveAVw2nz1JkiRpbgx6x0mSbUmeAq6kregBZwNP9Q3b32qj6B8Ad7TjxdT3lFHvedT7m81YVR0EaM9nLXA/kiRJmsFI/I7eYpDkj4FfGPDWp6vqtqr6NPDpJNcCHwc+A2TA+Hn9PYvZ+m5jPg1MAl+emjZg/Lz1PUzPg6YNqI3Sb4eMen+SJEnqEIPekKrqV4cc+hXgdnpBbz9wTt97K4EDx7m1Gc3Wd5KNwAeAdfXzH1Vc0L7n8Fn3W/DPehaj3t9snk6yoqoOJlkBPLPQDUmSJOnIvHXzOEiyuu/lB4FH2vEuYEOSk5KcC6wG7p3v/o4kyXrgU8AHq+rP+t4a6b6PYNR7/i6wOsm5SV5Pb+OYXQvc01zsAja2443AkVZWJUmSNAJc0Ts+rkvyDuBnwA+AjwJU1d4kO4GH6N0aeU1Vvbpwbb7G54CTgN1JAO6uqo+Oct9J/i7we8BbgduTPFBVl4xyzwBVNZnk48A3gWXAF6tq7wK3NVCSrwLjwJlJ9tNbnb4O2JlkE/AkcMXCdShJkqTZ5Od360mSJEmSusBbNyVJkiSpYwx6kiRJktQxBj1JkiRJ6hiDniRJkiR1jEFPkiRJkjrGoCdJkiRJHWPQkyRJkqSO+X/piLlLNd/3/gAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "#hide_input\n", "\n", "oof_diffs.hist(figsize=(15,15),bins=20)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, we apply the two techniques to calculate the ensembling weights" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "R = oof_diffs.mean().values\n", "CM = oof_diffs.cov().values\n", "\n", "q=0\n", "\n", "# Var technique\n", "fun_ex1 = lambda w: (train[target]-np.matmul(oofs.values, w)).var()\n", "# Cov technique\n", "fun_ex2 = lambda w: np.matmul(np.matmul(w.T,CM),w) - q * np.matmul(R,w)\n", "\n", "cons = ({'type': 'eq', 'fun': lambda x: x.sum()-1})\n", "\n", "bnds = ((0,None),\n", " (0,None),\n", " (0,None),\n", " (0,None),\n", " (0,None))" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "# Example 1\n", "\n", "w_init = np.ones((len(models)))/len(models)\n", "\n", "res = scipy.optimize.minimize(fun_ex1, w_init, method='SLSQP', constraints=cons) #,bounds=bnds\n", "\n", "w_calc = res.x" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CV: Ex1 calc weights: 7.85594426240217\n" ] } ], "source": [ "#hide_input\n", "\n", "submission['loss']=(test[models.keys()]*w_calc).sum(axis=1)\n", "\n", "submission.to_csv('submission_ex1_w_calc.csv',index=False)\n", "\n", "print('CV: Ex1 calc weights:',mean_squared_error(train[target], (train[[m_name + '_oof' for m_name in models.keys()]]*w_calc).sum(axis=1), squared = False))" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [], "source": [ "# Example 2\n", "\n", "w_init = np.ones((len(models)))/len(models)\n", "\n", "res = scipy.optimize.minimize(fun_ex2, w_init, method='SLSQP', constraints=cons) #,bounds=bnds\n", "\n", "w_calc = res.x" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "jupyter": { "source_hidden": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CV: Ex2 calc weights: 7.855944262936231\n" ] } ], "source": [ "#hide_input\n", "\n", "submission['loss']=(test[models.keys()]*w_calc).sum(axis=1)\n", "\n", "submission.to_csv('submission_ex2_w_calc.csv',index=False)\n", "\n", "print('CV: Ex2 calc weights:',mean_squared_error(train[target], (train[[m_name + '_oof' for m_name in models.keys()]]*w_calc).sum(axis=1), squared = False))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 5. Results\n", "\n", "The competition metric is root mean squared error (RMSE). These are the scores of the different ensembles:\n", "\n", "|Ensemble| CV|public LB|\n", "|--------|-------|---------|\n", "|HGB only|7.86563|7.90117|\n", "|All weights eq.|7.88061|7.92183|\n", "|XGB and KNN (50:50)| 7.89321|7.91603|\n", "|Ex1 (Var)| 7.85594|7.88876|\n", "|Ex2 (Cov)| 7.85594|7.88876|" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# References\n", "\n", "- Modern Portfolio Theory: https://en.wikipedia.org/wiki/Modern_portfolio_theory\n", "- TPS August 2021 Competition: https://www.kaggle.com/c/tabular-playground-series-aug-2021/overview" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Ressources\n", "- Original notebook: https://www.kaggle.com/joatom/model-allocation\n", "- TPS data: https://www.kaggle.com/c/tabular-playground-series-aug-2021/data" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python [conda env:ml]", "language": "python", "name": "conda-env-ml-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.12" } }, "nbformat": 4, "nbformat_minor": 4 }