{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# StackingRegressor: a simple stacking implementation for regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An ensemble-learning meta-regressor for stacking regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.regressor import StackingRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stacking regression is an ensemble learning technique to combine multiple regression models via a meta-regressor. The individual regression models are trained based on the complete training set; then, the meta-regressor is fitted based on the outputs -- meta-features -- of the individual regression models in the ensemble."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./StackingRegressor_files/stackingregression_overview.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Breiman, Leo. \"[Stacked regressions.](https://link.springer.com/article/10.1023/A:1018046112532#page-1)\" Machine learning 24.1 (1996): 49-64."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Simple Stacked Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mlxtend.regressor import StackingRegressor\n",
    "from mlxtend.data import boston_housing_data\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.svm import SVR\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import warnings\n",
    "\n",
    "warnings.simplefilter('ignore')\n",
    "\n",
    "# Generating a sample dataset\n",
    "np.random.seed(1)\n",
    "X = np.sort(5 * np.random.rand(40, 1), axis=0)\n",
    "y = np.sin(X).ravel()\n",
    "y[::5] += 3 * (0.5 - np.random.rand(8))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Squared Error: 0.1846\n",
      "Variance Score: 0.7329\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD1CAYAAAC1BoUqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAtE0lEQVR4nO3deVxU5f4H8M8AgyCoIxSDIaEoECq5IBeXZBJFS0RM3FArXLtlLnlN0wp30ywts98ttUXvLTRKsSupJCYYLiQuhE6oGSoooyIjAgLDML8/fDEJzLDNOMMcPu9/Ys48Z873gfxweM5zniPSaDQaEBGRIFmZuwAiInp0GPJERALGkCciEjCGPBGRgDHkiYgEjCFPRCRgNuYu4GFpaWnmLoGIyCL5+/vr3N6kQh7QX2hd5HI5fH19jVyNZWDf2ffmhn2v2vfaTpA5XENEJGAMeSIiAWPIExEJGEOeiEjAGPJERALW5GbXEBlCqVRCoVBApVJBLBZDKpVCIpGYuywis2HIk2AolUrk5OSgcvVslUqFnJwcAGDQU7Nl8HDNjRs38OKLL+L5559HaGgotm3bVqONRqPBypUrERISgrCwMJw7d87QwxLVoFAoUP3xCBqNBgqFwkwVEZmfwWfy1tbWeOutt9C1a1cUFhYiIiIC/fv3R+fOnbVtkpOTkZWVhYSEBJw9exZLly5FbGysoYcmqkKlUjVoO1FzYPCZvIuLC7p27QoAcHR0hKenZ40zp8TERIwcORIikQg9evRAQUEBbt68aeihiaoQi8UN2k7UHBh1dk12djbkcjm6d+9eZbtCoYCrq6v2taurK/+EJqOTSqUQiURVtolEIkilUjNVRGR+RrvwWlRUhNmzZ2Px4sVwdHSs8p6ux8hW/8dYSS6XN+r4JSUljd7X0rHvtff9xo0buHHjhokqMh3+3Nn3+jBKyKtUKsyePRthYWEYMmRIjfddXV2Rm5urfZ2bmwsXFxedn9XYRYe4YBH73tyw7+x7pUe6QJlGo8Hbb78NT09PTJ48WWeb4OBgxMXFQaPR4MyZM2jVqpXekCciIuMx+Ew+LS0Ne/bsgbe3N8LDwwEA8+bNw/Xr1wEAkZGRkMlkSEpKQkhICOzt7bF69WpDD0tERPVgcMj37t0bmZmZtbYRiURYsmSJoYciIqIG4to1REQCxpAnIhIwhjwRkYAx5ImIBIwhT0QkYAx5IiIBY8gTEQkYQ56ISMAY8kREAsaQJyISMIY8EZGAMeSJiASMIU9EJGAMeSIiAWPIExEJGEOeiEjAGPJERALGkCciEjCjhPyiRYvQt29fDB8+XOf7J06cgL+/P8LDwxEeHo5NmzYZ47BERFQHg5/xCgCjRo3CpEmTsHDhQr1tevfujc8//9wYhyMionoySsgHBAQgOzvbGB9FtVAqlVAoFFCpVBCLxZBKpZBIJOYui4iaMKOEfH2cOXMGI0aMgIuLCxYuXAgvLy+d7eRyeaM+v6SkpNH7WoKKigpoNBrta5VKhezsbOTk5KCsrEzQfa+N0H/utWHf2ff6MEnId+3aFYcOHYKDgwOSkpIwc+ZMJCQk6Gzr6+vbqGPI5fJG72sJMjMzoVKpamy3sbGBlZWVoPteG6H/3GvDvrPvldLS0vS2N8nsGkdHRzg4OAAAZDIZysvLcefOHVMcWjB0BXxt24mIABOF/K1bt7RDDenp6aioqEDbtm1NcWjBEIvFDdpORAQYabhm3rx5SE1NRX5+PoKCgjBr1iyUl5cDACIjI3HgwAHExMTA2toadnZ2WL9+PUQikTEO3WxIpVLk5ORUGZcXiUSQSqW4ceOGGSsjoqbMKCG/fv36Wt+fNGkSJk2aZIxDNVuVs2h0za5hyBORPiabXUOGk0gknDJJRA3CZQ2IiASMIU9EJGAMeSIiAWPIExEJGEOeiEjAGPJERALGkCciEjCGPBGRgDHkiYgEjCFPRCRgDHkiIgFjyBMRCRhDnohIwBjyREQCxpAnIhIwhjwRkYAZJeQXLVqEvn37Yvjw4Trf12g0WLlyJUJCQhAWFoZz584Z47BERFQHozwZatSoUZg0aRIWLlyo8/3k5GRkZWUhISEBZ8+exdKlSxEbG2uMQ5MZKZXKKo8jdHR0RGFhYY3HE9a1n752RGQ4o5zJBwQEoE2bNnrfT0xMxMiRIyESidCjRw8UFBTg5s2bxjg0mYlSqUROTg5UKhUAQKVSIT8/v8rrnJwcKJXKOvfT1Y6IjMMkY/IKhQKurq7a166urlAoFKY4ND0iCoUCGo2m1jYajabGz1nXfrraEZFxmORB3rrCQCQS6Wwrl8sbdYySkpJG72vpzNF3tVpdr3YqlapKbfr2q96uvvhzZ9+bm4b23SQh7+rqitzcXO3r3NxcuLi46Gzr6+vbqGPI5fJG72vpzNH3zMxM7ZBLbcRiMXx8fOrcr3q72lQf02/Xrl2zHNPn//Pse6W0tDS97U0yXBMcHIy4uDhoNBqcOXMGrVq10hvyZBmkUqnev8YqiUQiSKXSOvfT1U6f6mP6ADimT1QLo5zJz5s3D6mpqcjPz0dQUBBmzZqF8vJyAEBkZCRkMhmSkpIQEhICe3t7rF692hiHJTOqPHNu6OwaXfs1ZHZNbWP6zfFsnqguRgn59evX1/q+SCTCkiVLjHEoagIqKipw/sZ5JF1IQvKFZJy7fg6l5aUoLS9FWXkZRCIRWrVoBUlLCdrYt4GkpeTvr+3//trzcU/0cO8BhxYO9T62viGiylk6+fn5VbZziiY1dyYZk6dHo3JsWq1WIyMjA8CjCTV1hRpnr53VhvqRS0eQV5hX6z65yK31/UpWIit0eaILenv0Ru8OvRHQIQBPt38admI7ne3FYrHeoK8e8MDf4Q+AQU/NEkPeQlWOTVcfujBGqKnKVUi7kobki8lIupCEXy/9ioL7BVXauEncIPOWQeYjQ0CHADi2cIStjS3E1mIAQMH9Aty9fxfKYuWD/95X/v11sRL5xfmQ35Dj95zfkZGTgYycDHx99GsAgI21Dfzc/LTB39ujN7q5dYOtjS2kUmmNfotEolqnc3I4h5ozhryFqm2eekNDrVRVitS/Uh+cqV9MxtE/j6KotKhKm46PdYTMW4Yg7yDIvGXo+FjHWi+8PiF5ol7Hvl92H2evncXJKydxMuskTl45ifM3zuP01dM4ffU0thzZAgBoYdMC3d27o7dHb3Rx6QJ3e3d4tPaAtZU13NzckJ2dXetx6jMTiEiIGPIWqq7Qqu394tJiHLt8DMkXHpypH798HKXlpVXa+Lj6IMgrSBvs7k7uRqm7Ontbe/Tp1Ad9OvXRbissKcTpq6erBP8FxQWk/pWK1L9Ste2cHJwQ1CEIk5+djCfwBOxsdA/xAA+GeYiaI4a8haptbLry/UoF9wuQcilFO/xyMuskVOqq+3Zz6/Zg+MVbhgFeA+DaxrX6R5qMo50jBngPwADvAdptd4vvIu1Kmjb4U7NScSXvCuLOxSHuXBxailvimQ7PYHCnwRjQYQAcWzhq923IFE0ioWHIN1F1LeKla2y6UkFpAbLuZmHz2c1IvpCMU1dPoUJToX3fSmSFXk/20p6lD/AaAGdHZ1N0q9HatGyDYN9gBPsGa7fJb8jx2YHPkJKdgrQraUi4mICEiwkQW4sR1CEIE3tMRL+O/Ti7hpo1hnwTVP2iqq6LqQ/PNy+8X4gjWUfwW/ZvSLuehgu3L1QJfxtrG/zD4x8Phl98ZOjfqT/atNS/oJyl8G3ni38G/hMfR32MK3lXsPv0buw+vRtHLh5B4p+JSPwzEcFPBWN5+HL0l/Q3d7lEZsGQb4Lqe8OPRqzB95e+x4aEDcgr/ntKo62NLQI7BmrP1Pt69oWjnSOEzMPZA3MHz8XcwXNxXXkdW49sxYaDG3Doj0M49MchDO06FMtGLEOgZ6C5SyUyKYZ8E1TbDT8AcPvebWw4uAGbftmkndrYvX13jOo1CjJvGQI9A/XOM28OnpA8geiwaMweNBsbft6ADQc34MC5Azhw7gBC/UKxPHw5enn0MneZRCbBkDeT2sbc9V1ULSwvxILvF+D/Dv+fdopj8FPBmNB1AqYMnVLnWjLNjaSlBMvCl2H2oNn4MOFDbDy0EfG/xyP+93iM7DESy8KX4en2T5u7TKJHis94NYO6HpxRfRGv+6r72PrbVgz5YgjWHViHotIiPN/teRx96ygS/5WIfh79GPC1cHZ0xupRq/HXe39h/pD5sLe1R9yZOHRf1h1jPxuL89fPm7tEokeGIW9iSqUS2dnZtT44QyKRwM3NDSJrEWJ/j0XotlB8fPRj3Cu5h5AuIUhdnIqf5vyEvp36mqMLFuvxVo9j3Zh1uLz6MuYMmoMWNi0QmxaLbku7YeKWibiQe8HcJRIZHUPehCrP4PV5eIjmRPYJRHwbgeWHluNW0S34e/jj5zd+RsIbCQjoGGCKcgXLtY0rPhr/ES6tuoTXnn0NNlY2+Db1W/hG+yLqyyhcvnXZ3CUSGQ1D3oTqemSeWCzG1byriPh3BJ77+Dlk5mai0+OdsGPGDqQuTsXgLoNNWK3wtXdqj08nfoqLqy5i+oDpsLKywrZj2+Dzrg+mb5+OK3lXzF0ikcEY8iZU2x2q5RXl+Pbcg7PJXad2waGFA94f/T7OLz+PcQHjYGXFH9Wj4uHsgc0vbUbmikxE9YtCRUUFth7ZCq+3vfDaN68hJ1//X19ETR2Tw4T0rZ9y7OoxjN0xFsv3LUdxWTHG9h6LP5b/gTeHvglbG1sTV9l8eT7uia8mfwX5cjkmBk5EeUU5/n343+i0uBPm7JiD3Lv1Wz6ZqClhyJtQ9VkzikIF5v80HzN2z8DFWxfh4+qDn9/4GTtf2Yn2Tu3NWGnz5u3qjf9O+y8ylmZgbO+xKC0vxcbEjfBc7In5sfNx694tc5dIVG8MeROqnDVjbWON737/DiP/MxIHLh5AS9uWWDNqDdKXpHPcvQnp8kQX7HxlJ84uOYsXer6A+2X38WHCh+i4qCMW7VpU54NTiJoCo4R8cnIyhg4dipCQEGzevLnG+ydOnIC/vz/Cw8MRHh6OTZs2GeOwFqmwohCzf5qNFYdWoLCsECO6j8AfK/7AwucXcmimiXq6/dPY9doupL2ThuFPD0dRaRHW7FuDjos6InpPNJTFSnOXSKSXwSGvVquxfPlybN26FfHx8di7dy8uXbpUo13v3r2xZ88e7NmzB6+//rqhh7VIO1J3wG+ZHw7KD+Ixx8ewc8ZOxM2Me2RrtZNx9fLohf/N+h+OLzqOoV2H4l7JPazYuwId3uqAFXtX1Hh6FlFTYHDIp6enw8PDA+7u7rC1tUVoaCgSExONUVuTp1QqkZmZiYyMDGRmZmrvWK0uvygfE7dMROSWSCiLlRj+9HBkLMvA2ICxvFPVAgV6BmL/3P04suAIgp8Kxt37dxG9JxodF3XEmn1rUFhSaO4SibREmtombtfD/v37ceTIEaxatQoAEBcXh/T0dERHR2vbnDhxArNnz4ZUKoWLiwsWLlwILy+vGp+VlpaGli1bNqqOkpIS2NmZblGuiooKnXPeRSJRlemOx68ex+L9i5FbmAt7G3ssfHYhxviNMWq4m7rvTUlT6HvqtVRsPLoRp3JOAQCc7J0wNWAqxncfjxbWLWo8j9ZY02GbQt/NhX2v2vfi4mL4+/vrbG/wAmX6gu5hXbt2xaFDh+Dg4ICkpCTMnDkTCQkJOj/P19e3UXXI5fJG79sYmZmZOue929jYwMfHByWqEizetRgbDm4AAAR2DMR/p/0XnV06G70WU/e9KWkKfff19cVLIS/hoPwg3o17Fyf+OoF1yeuw/fR2TO41GaO7jUYLmxYA/v730q5dO4MfZNIU+m4u7HvVvqelpeltb/AphaurK3Jz/54/rFAo4OLiUqWNo6MjHBwcAAAymQzl5eW4c+eOoYc2uYeHZ2pbDvjstbPovbI3NhzcAGsraywbsQy/Lvz1kQQ8NQ0ikQghXUJwbNExxM+Oh7+HPxT3FFiTtAbDtg3D7nO7tQGvVqurLEhH9CgZHPJ+fn7IysrCtWvXUFZWhvj4eAQHB1dpc+vWLe3/4Onp6aioqEDbtm0NPbRJVV85Uhd1hRrbTm9DwKoAnLt+Dl4uXjj61lFEh0XDxpqrOjcHIpEIw/yG4be3f8PHwz+G92PeuFl4E9EHozFt1zRcU14DUHVBOqJHyeDksbGxQXR0NKZNmwa1Wo2IiAh4eXkhJiYGABAZGYkDBw4gJiYG1tbWsLOzw/r16y3ugmNd687kFOTgnYR3cDLnJADg1WdfxbrR6+DQwsFUJVITIhKJMPSpoXjW81n8lPkT3k9+H6nZqYj4JgJz+s9BZPfIWk8YiIzFKKeXMpkMMpmsyrbIyEjt15MmTcKkSZOMcSiz0fcPUqPR4H9//A+rD69GUVkRXNu44suXv8Tzfs+buEJqaioftj78qeHo79Ef7x1+D/su7MOapDU4eOkg3nv+PXRDN3OXSQLHMYR60vW0poLSAqz8ZSX2Ze4DALzQ8wVsfnEzHmv1mDlKpCam8sLq9evX0da+Ld5//n0M8RqCFYdW4GTOSYzcPhJLbi/B8E7DoS5X13hCGJExcFmDeqq+7kx6bjrGfjsW+zL3wbGFI76K+go/vPoDA56qkEgk6NKlC9q3bw+xWIzBnQcjfnI8Xuj+AorKirAgbgGidkYhpyCnxhPCiIyBIV9PD687s+3UNrwc+zJyCnLg7+GP09GnEdU/yuKuM5DpSCQS+Pj4oFu3bujTsw92vb4LH4V9BCd7J6Rmp2L0N6Nx4OIBXpAlo2PIN4DaRo03E97EB0c+QHlFOeYMmoOUhSmcGkmNMshzEHZP2o1BnQahsKwQ83+aj5WHVqLwPu+YJeNhyNfT0UtH0WN5D8T/Ho+2Ldtiz8w9+Gj8R2ghbmHu0shCicViOLV0wobQDVj07CKIrcXY+ftOTPpuEp83S0bDkK9DRUUF3t//PoLWBSE7Pxt9O/XFmegzGNFjhLlLIwtXeZ1HJBJhQvcJ+M+Y/8C9jTv+uPUH/Ff649sT35q7RBIAhnwtbt+7jbBNYVj4w0KoK9R4c+ibSJqfhCednzR3aSQAldd5Kp8Y1qN9D/w6/1eMCxiHwtJCTNw6EdO3T0dxabGZKyVLximUeqRcSsH4zeORnZ8NJwcnbJ+yHaFPh5q7LBIYiURSY8pkzPQYDPQZiDk75mDrka04fvk4vnvlO/i2a55rtZBhmuWZfG1LBFdUVGDtvrWQrZMhOz8b/Tr1w5noMwx4MhmRSIRXZK/gxOIT8JZ6IyMnA71X9sa2o9vMXRpZoGYX8tXXoHl4bvLte7cx/JPheGvXW1BXqLFg6AIcnn+YD/Ugs+ju3h1p76RhUp9JKC4rRtRXUYj6MgpFpUXmLo0sSLMLeV1r0Gg0GvyU9hN6LO+BfRn74OTghL2z9mLt6LUQ24jNVCkR4GjniO1TtuOLl7+Ava09th17sABeRk6GuUsjC9EsxuSVSiUUCoXO9WcqNBX48uSX2HRsE9QaNfp16ocdM3bw7L0RHv4+8xZ94xGJRJjyzBQEdgzE2M/H4vyN8whYFYDFzy7GO0+9w5vwqFaCP5OvbYng/Pv5eG3Pa/j46MdQa9RY+NxCDs80Um3DYGQcXd26IvXtVEzuPxklqhJE/xyNSVsn4V7JPXOXRk2YYEO+8uJqdna2ziWCT+WcwuhvRyPlSgokdhLsnLITayLWcHimkfQNg/EWfeNyaOGAL6O+xPYp22Evtse3qd+i98reOHP1jLlLoyZKkCFf29m7ukKNLalbMOWHKbhZeBO93HrhyL+OYGzfsWaoVDhqe1IWGd+LfV9E7MRY+Ln54YLiAvq81wf/PvzvWp95QM2TIENe3wM+bhbexIzdM7Dx2Ebt8Mzxd46jWweu6W2oyht66rudDOfp5IkTi09gRtAMlJaX4rVvXsP4zeNxt/iuuUujJkSQIa/r7DH5r2SM/nY0UrNT4dTSCd9P+57DM0ZUfSlm4MEFQ6lUaqaKmgd7W3t8/uLniJkeA8cWjvju5HfotbIXTmadNHdp1EQYJeSTk5MxdOhQhISEYPPmzTXe12g0WLlyJUJCQhAWFoZz584Z47B6PXz2WFpeirVJazHzx5nIv5+Pfh79cPRfRxERGPFIa2huqt+iLxaL4ebmxtk1JjL+H+Nx6t1T6PlkT1y+dRn91vTDxsSNHL4hw0NerVZj+fLl2Lp1K+Lj47F3715cunSpSpvk5GRkZWUhISEBK1aswNKlSw09bK0qzyr/zPsTE3ZOwH/P/Bc2VjZYMmwJjiw+Ap8nfR7p8Zurh9dM9/HxYcCbmJf0wYPjZw6cCZVahTk75iDi3xHIL8o3d2lkRgaHfHp6Ojw8PODu7g5bW1uEhoYiMTGxSpvExESMHDkSIpEIPXr0QEFBAW7evGnoofVq3bo1frz8I8bGjMWF2xfgIfHAgdcPYOkLS2FlJcgRKiIAgJ3YDpsmbML3//webezbYPfp3ei5oidOXD5h7tLITAxOPIVCAVdXV+1rqVRaY9pc9Taurq6PbGrd1byrCNkQgsU/LkaZugxT+k9BxooMBPsFP5LjETVFEf4ROPXuKQR0CMCVvCt45v1n8GHChxy+aYYMvuNV1/801S/A1adNJblc3qg67t+/j7U/rMWqX1bhXuk9ONk7YVnIMgzqPAjX/rrWqM+0FCUlJY3+vlk69r32vm8J34L1R9Zj+6ntmB87H3vT9mL10NWQ2EtqtK2oqKjyb1UkEjXZv3z5c69/3w0OeVdXV+Tm5mpfKxQKuLi41NomNze3RptKvr4NX071TtEdRH4aiYSLCQCAEd1HYMtLW+DSWvcxhEYulzfq+yYE7Hvdfd/WbRsizkQg6qsoHL58GGN3jMWOGTvQv3N/bZvKe0uqa9euXZO8tsKfe9W+p6Wl6W1v8K9pPz8/ZGVl4dq1aygrK0N8fDyCg6sOjQQHByMuLg4ajQZnzpxBq1at9IZ8Y7wb9y4SLibAsYUjvnj5C8TNjGs2AU9UHyN6jMDp6NPo26kvsvOzIVsnw5p9a1BRUQGAdywLmcFn8jY2NoiOjsa0adOgVqsREREBLy8vxMTEAAAiIyMhk8mQlJSEkJAQ2NvbY/Xq1QYX/rDxAeORdycPq8evhufjnkb9bCKh8HD2QNL8JLwT9w7eP/A+Fu1ahKQLSdg+ZTvvWBYwo6xCKZPJIJPJqmyLjIzUfi0SibBkyRJjHEqnAd4D8Jj6MQY8UR3ENmKsHb0WMh8ZXvryJezP2I8ey3tgzdA16Onas2Z73rFs8ZrmVRUieqSG+Q3DmXfPYIDXAFxXXkfUd1H4PPVzqCvU2ja8Y1kYGPJEzVR7p/Y49K9DeHvY29BAg03HNuHVPa/idtFt3rEsIAx5ombMxtoGK19YiQNzD8CllQuOXT2G8d+Nx3XNdQa8QDDkiQghXUJwJvoMBvoMhKJAgcHrB2NV/Crt7BuyXAx5IgIAtJO0w8/zfsa7w9+FBhq8E/cOQj8Jxe17t81dGhmAIU9EWtZW1lgevhw/zf4Jzo7O2J+xHz1X9MTRS0fNXRo1EkOeiGp4rttzOP3uQzdPfSDDBwc+4PCNBWLIE5FO7k7u+HbCt3ip50soV5fjze/fxJAPhnD4xsIw5Buh8iHhGRkZyMzMhFKpNHdJREaXk5ODwoJCvBn0JjaGbUTrFq2ReDERXd/tir1pe81dHtUTQ76Bqj8kXKVSIScnh0FPgpOf//fDRgZ6DkTshFj0bNcTN4tuYsRnI7Bg5wKUq8vNWKHxCfEEjiHfQFzIiZqrJ1o/gS9Hf4kZ/5gBAFh3cB0GfjAQ1+4IYylvoZ7AMeQbiAs5UXNmY2WDWX1nYcuoLXjc4XH8eulXdF/WHXGn48xdmsGEegLHkG8gfQs2cSEnEpq2bdvqfS/QPRDfT/gew/yGIb84Hy/83wt4/dvXUaIqMWGFxiXUEziGfANVPiT8YVzIiYTIzc2t1qB/vNXj2DtrL9aPXQ+xtRif/vIpAlcHQn7DMp/YJNQTOIZ8A0kkEri5uWl/8FzIiYTMzc0N7du313li065dO4hEIrwR8gaOLTqGzi6dkZ6djl4reuGjgx9Z3Jx6oZ7AMeQbQSKRwMfHB926dYOPjw8DngStPic2/h7+OPXuKUT1i0KJqgRv7HwDg9YPwpW8K2aquuGEegJnlIeGEJGwSSSSOsOulV0rfDX5K4zsORIzts/A4czD8Fvqh4/Hf4yoflE1zpKbovr009LwTJ6IjCq8RzgylmXghZ4v4F7JPUz5egpGfjoSigLLnqViqQwKeaVSicmTJ2PIkCGYPHky7t69q7NdcHAwwsLCEB4ejlGjRhlySCKyAI+3ehw/vPoDtk/Zjtb2rfHj2R/RbUk37Dq1y9ylNTsGhfzmzZvRt29fJCQkoG/fvti8ebPettu2bcOePXuwaxd/yETNgUgkwot9X0TG0gwM8h2E24W3EfHvCLz0xUtQFivNXV6zYVDIJyYmYuTIkQCAkSNH4uDBg8aoiYgExN3JHQlzE/BJ5Cewt7XHf47/B12XdMWO1B01bj4i4xNpDPgu9+7dGydPntS+DggIwG+//VajXXBwMNq0aQORSIRx48Zh3LhxOj8vLS0NLVu2bFQtJSUlsLOza9S+lo59Z98tRVZ+Ft7a9xbSc9MBAAHtA/D2wLfh/bh3gz7HEvtuLLr6XlxcDH9/f53t65xdExUVhdu3ay4tOnfu3HoXFRMTA6lUiry8PEyePBmenp4ICAjQ2dbX17fen/swuVze6H0tHfvOvluKdsp22OG0A9+d+Q4fHf0Iv2X/hohvIjBz4EwsG7EMkpaSen2OJfbdWHT1PS0tTW/7OkP+66+/1vues7Mzbt68CRcXF9y8eRNOTk4621XeTODs7IyQkBCkp6frDXkiEqbKBcA0Gg0iukVgcOfB+PT4p9iZvhMbEzciJjUGa0atQVS/KFhZceKfsRj0nQwODkZcXBwAIC4uDoMGDarRpri4GIWFhdqvU1JS4OXlZchhicgCVV8ArI1dGyx+djF2vbgLz3R+Brfu3cLUbVPRd01f/PZXzWFfahyDQn7GjBlISUnBkCFDkJKSghkzHixBqlAoMH36dABAXl4eJkyYgBEjRmDMmDGQyWQICgoyvHIisij6Fvrq1LYTkhck45tp36Bdm3ZI/SsVge8FYvr26bh175aJqxQeg+54bdu2LbZt21Zju1QqxZYtWwAA7u7u+PHHHw05DBEJgFgs1hn0YrEYIpEIEwInIKx7GFbuXYkNBzdg65Gt+D7te6wIX4F/yv4JG2veoN8YHPgiIpOozwJgrexaYe3otfh96e8Y0mUIlMVKzIqZhV4reiH5QrKpSxYEhjwRmURDFgDzcfXB/rn7sfu13ejg3AG/5/wO2ToZJmyZgJz8HBNXbtkY8kRkMg1ZwVUkEmFkz5E4v/w8loYthZ3YDjGpMfB51wefnfgMd4t1L6NCVTHkiahJs7e1x5IRSyBfLscLPV9AUWkRNqZshPtCdyz4fgGuK6+bu8QmjSFPRBahw2MdsOu1XTg47yAC3QNxr+Qe1h1Yh46LOmLatmnIzM00d4lNEkOeiCyGUqlEe6v22DpqK2InxmKE3wio1Cp88esX8I32xaj/G4UTl0+Yu8wmhSFPRBah8o7ZymmYTz32FFYPWo3UN1MxfcB0iK3F2H16N/q81wfPrnsW+37fxwXQwJAnIgtR/Y5ZANBoNGhV0QqbX9qMrPeysPC5hWht3xpJF5IwbOMw9FjeA5sObcIN5Q0zVW1+DHkisgj67pit3N5O0g5rItbg6pqrWBuxFq5tXJGenY5ZMbPgtsANQe8H4ZPET5rdhVqGPBFZhMr59XVtb9OyDRY8twBZ72XhP1P/g7DuYRBbi3Hk4hHM3jEb7Re0b1aBz5AnIotQnztmH3a/6D4C2gZg9bOrkfJqCj4b/xlGdB8BW2vbKoE/YO0AbEzcKNibrLgYBBFZhMobpxQKBVQqFcRiMaRSqc4bqh5e1hgA7Kzs8IzrMxjnPw5WLazwv7P/Q2xaLPZn7Mevl37Fr5d+xZwdc9C/c3+M8R+D0KdD0enxTjV+qVgihjwRWQyJRAKJRAK5XA4fHx+97fRdpFUoFPDx8cHEPhMxsc9EFNwvwN70vYg9GYt9GfuQcikFKZdSMHfnXDzm+Bj6ePZBYMdA9PHsg4AOAWjTss2j7qLRMeSJSHDqukhbqbV9a0wInIAJgRNwr+Qe4tPj8cOpH5B0IQm37t3C3vS92Ju+V9u+pW1LODk4wcnBCc4OztqvnRyc4OzoDKeWTjW3OTjBTmy+RxUy5IlIcGpb1lifVnatMP4f4zH+H+Oh0Wjw1+2/cOLyCRz/6ziOXz6OM9fOoLisGMVlxcjOz25QPfa29nBq+XfoOzk4wbGFIwpLC3H3/l2UlZcheng0BncZ3OC+1oUhT0SCI5VKq4zJA7VfpK2Uk5OD/Px87eug9kGIDIwE8GC4p7C0EHeK7uBO0R3kFeZpv75TdAd3ivVvu192HzllOchR6r+4m3QhiSFPRFQfDblIW6l6wAPQvnZwcKjyWR2lHdHzyZ71qkWj0aCotKhK+OcV5aGwtBCt7FqhtV1rODs6o6d7/T6voQwK+X379mHTpk34888/ERsbCz8/P53tkpOTsWrVKlRUVGDMmDHaxwQSET0qlRdp66t6wD+8XalUav8qUKlUyMnJ0R6jLiKRCI52jnC0c8STzk/WeF+pVEKhUOD8+fP1+mXUUAaFvLe3Nz755BMsWbJEbxu1Wo3ly5fjq6++glQqxejRoxEcHIzOnTsbcmgiIpPRN1OnMWFcGeoqlQpWVlbQaDSN/gVSHwbdDNWpUyd4enrW2iY9PR0eHh5wd3eHra0tQkNDkZiYaMhhiYjMTt8MntpUX2StoqJC7y8QY3nkd7wqFAq4urpqX0ulUqN2gIjIGNq2batzu5WV7pisbaaOPrrm7+vSmF8g+tQ5XBMVFYXbt2/X2D537lwMHlz3lWBdHartLjK5XF7nZ+pSUlLS6H0tHfvOvjc3puy7vlAuLy9vcA1qtbrebfV9dkP7XmfIf/311/X+MF1cXV2Rm5urfa1QKODi4qK3va+vb6OOI5fLG72vpWPf2ffmxtR9f3gc3ZCLo5mZmXWepYtEIr0POAd09z0tLU3v5z3yKZR+fn7IysrCtWvXIJVKER8fjw8//PBRH5aIyGgaOlNHH13z9wHA2toaarX6kcyuMWhM/ueff0ZQUBBOnz6NV155BVOnTgXw4Gx9+vTpAAAbGxtER0dj2rRpGDZsGJ5//nl4eXkZXjkRkYWRSCRwc3PTjueLxWK0b98evr6+6NatG3x8fIwa8ICBZ/IhISEICQmpsV0qlWLLli3a1zKZDDKZzJBDEREJgrH+KqgvridPRCRgDHkiIgFjyBMRCRhDnohIwBjyREQCxpAnIhIwhjwRkYAx5ImIBIwhT0QkYAx5IiIBY8gTEQkYQ56ISMAY8kREAsaQJyISMIY8EZGAMeSJiASMIU9EJGAWH/JKpRKZmZlQq9XIzMyEUqk0d0lERE2GQY//27dvHzZt2oQ///wTsbGx8PPz09kuODgYDg4OsLKygrW1NXbt2mXIYbWUSmWVh+KqVCrk5OQAgEkfr0VE1FQZFPLe3t745JNPsGTJkjrbbtu2DU5OToYcrgaFQlHjqecajQYKhYIhT0QEA0O+U6dOxqqjUVQqVYO2ExE1NwaFfENMnToVIpEI48aNw7hx4/S2k8vlRjmesT7HEpSUlDSr/j6MfWffm5uG9r3OkI+KisLt27drbJ87dy4GDx5cr4PExMRAKpUiLy8PkydPhqenJwICAnS29fX1rddnAjXH5AFAJBLBzc2tWQ3XyOXyBn3fhIR9Z9+bG119T0tL09u+zpD/+uuvDS5KKpUCAJydnRESEoL09HS9Id8QlUGuUCigUqkgFoshlUqbVcATEdXmkU+hLC4uRmFhofbrlJQUeHl5Ge3zJRIJfHx8YG1tDR8fHwY8EdFDDAr5n3/+GUFBQTh9+jReeeUVTJ06FcCDM+vp06cDAPLy8jBhwgSMGDECY8aMgUwmQ1BQkOGVExFRnQy68BoSEoKQkJAa26VSKbZs2QIAcHd3x48//mjIYYiIqJFMNruGiIRNqVTy+lgTxJAnIoPx7vOmy+LXriEi86vt7nMyL4Y8ERmMd583XQx5IjKYWCxu0HYyHYY8ERlMKpVCJBJV2SYSibQ3QpL58MIrERmMd583XQx5IjIKiUTCUG+COFxDRCRgDHkiIgFjyBMRCRhDnohIwBjyREQCJtJUvxfZjGp7ugkREenn7++vc3uTCnkiIjIuDtcQEQkYQ56ISMAEccdrcnIyVq1ahYqKCowZMwYzZswwd0kmsWjRIhw+fBjOzs7Yu3evucsxqRs3bmDBggW4ffs2rKysMHbsWLz88svmLuuRKy0txcSJE1FWVga1Wo2hQ4di9uzZ5i7LpNRqNSIiIiCVSvH555+buxyTCQ4OhoODA6ysrGBtbY1du3bVb0eNhSsvL9cMGjRIc/XqVU1paakmLCxMc/HiRXOXZRKpqamajIwMTWhoqLlLMTmFQqHJyMjQaDQazb179zRDhgxpFj/3iooKTWFhoUaj0WjKyso0o0eP1pw+fdq8RZnYl19+qZk3b55mxowZ5i7FpAYOHKjJy8tr8H4WP1yTnp4ODw8PuLu7w9bWFqGhoUhMTDR3WSYREBCANm3amLsMs3BxcUHXrl0BAI6OjvD09GwWD6gQiURwcHAAAJSXl6O8vLzG6o9Clpubi8OHD2P06NHmLsViWHzIKxQKuLq6al9LpdJm8Y+d/padnQ25XI7u3bubuxSTUKvVCA8PR79+/dCvX79m028AWL16Nd58801YWVl8dDXK1KlTMWrUKOzcubPe+1j8d0qjYwZoczqzae6Kioowe/ZsLF68GI6OjuYuxySsra2xZ88eJCUlIT09HRcuXDB3SSbxyy+/wMnJCd26dTN3KWYRExOD3bt3Y8uWLfjmm2/w22+/1Ws/iw95V1dX5Obmal8rFAq4uLiYsSIyFZVKhdmzZyMsLAxDhgwxdzkm17p1awQGBuLIkSPmLsUkTp06hUOHDiE4OBjz5s3D8ePHMX/+fHOXZTKVD2BxdnZGSEgI0tPT67WfxYe8n58fsrKycO3aNZSVlSE+Ph7BwcHmLoseMY1Gg7fffhuenp6YPHmyucsxmTt37qCgoAAAUFJSgqNHj8LT09PMVZnGv/71LyQnJ+PQoUNYv349+vTpgw8++MDcZZlEcXExCgsLtV+npKTAy8urXvta/BRKGxsbREdHY9q0adqpVfXtvKWbN28eUlNTkZ+fj6CgIMyaNQtjxowxd1kmkZaWhj179sDb2xvh4eEAHnw/ZDKZmSt7tG7evIm33noLarUaGo0Gzz33HAYOHGjusugRy8vLw8yZMwE8uCYzfPhwBAUF1WtfLmtARCRgFj9cQ0RE+jHkiYgEjCFPRCRgDHkiIgFjyBMRCRhDnohIwBjyREQCxpAnIhKw/we+hSci4cAd7QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Initializing models\n",
    "\n",
    "lr = LinearRegression()\n",
    "svr_lin = SVR(kernel='linear')\n",
    "ridge = Ridge(random_state=1)\n",
    "svr_rbf = SVR(kernel='rbf')\n",
    "\n",
    "stregr = StackingRegressor(regressors=[svr_lin, lr, ridge], \n",
    "                           meta_regressor=svr_rbf)\n",
    "\n",
    "# Training the stacking classifier\n",
    "\n",
    "stregr.fit(X, y)\n",
    "stregr.predict(X)\n",
    "\n",
    "# Evaluate and visualize the fit\n",
    "\n",
    "print(\"Mean Squared Error: %.4f\"\n",
    "      % np.mean((stregr.predict(X) - y) ** 2))\n",
    "print('Variance Score: %.4f' % stregr.score(X, y))\n",
    "\n",
    "with plt.style.context(('seaborn-whitegrid')):\n",
    "    plt.scatter(X, y, c='lightgray')\n",
    "    plt.plot(X, stregr.predict(X), c='darkgreen', lw=2)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "StackingRegressor(meta_regressor=SVR(),\n",
       "                  regressors=[SVR(kernel='linear'), LinearRegression(),\n",
       "                              Ridge(random_state=1)])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stregr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Stacked Regression and GridSearch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this second example we demonstrate how `StackingCVRegressor` works in combination with `GridSearchCV`. The stack still allows tuning hyper parameters of the base and meta models!\n",
    "\n",
    "For instance, we can use `estimator.get_params().keys()` to get a full list of tunable parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.082717 using {'lasso__alpha': 0.1, 'meta_regressor__C': 1.0, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 10.0}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import Lasso\n",
    "\n",
    "# Initializing models\n",
    "\n",
    "lr = LinearRegression()\n",
    "svr_lin = SVR(kernel='linear')\n",
    "ridge = Ridge(random_state=1)\n",
    "lasso = Lasso(random_state=1)\n",
    "svr_rbf = SVR(kernel='rbf')\n",
    "regressors = [svr_lin, lr, ridge, lasso]\n",
    "stregr = StackingRegressor(regressors=regressors, \n",
    "                           meta_regressor=svr_rbf)\n",
    "\n",
    "params = {'lasso__alpha': [0.1, 1.0, 10.0],\n",
    "          'ridge__alpha': [0.1, 1.0, 10.0],\n",
    "          'svr__C': [0.1, 1.0, 10.0],\n",
    "          'meta_regressor__C': [0.1, 1.0, 10.0, 100.0],\n",
    "          'meta_regressor__gamma': [0.1, 1.0, 10.0]}\n",
    "\n",
    "grid = GridSearchCV(estimator=stregr, \n",
    "                    param_grid=params, \n",
    "                    cv=5,\n",
    "                    refit=True)\n",
    "grid.fit(X, y)\n",
    "\n",
    "print(\"Best: %f using %s\" % (grid.best_score_, grid.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-9.810 +/- 6.86 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 0.1, 'svr__C': 0.1}\n",
      "-9.591 +/- 6.67 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 0.1, 'svr__C': 1.0}\n",
      "-9.591 +/- 6.67 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 0.1, 'svr__C': 10.0}\n",
      "-9.819 +/- 6.87 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 1.0, 'svr__C': 0.1}\n",
      "-9.600 +/- 6.68 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 1.0, 'svr__C': 1.0}\n",
      "-9.600 +/- 6.68 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 1.0, 'svr__C': 10.0}\n",
      "-9.878 +/- 6.91 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 10.0, 'svr__C': 0.1}\n",
      "-9.665 +/- 6.71 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 10.0, 'svr__C': 1.0}\n",
      "-9.665 +/- 6.71 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 0.1, 'ridge__alpha': 10.0, 'svr__C': 10.0}\n",
      "-4.839 +/- 3.98 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 0.1}\n",
      "-3.986 +/- 3.16 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 1.0}\n",
      "-3.986 +/- 3.16 {'lasso__alpha': 0.1, 'meta_regressor__C': 0.1, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 10.0}\n",
      "...\n",
      "Best parameters: {'lasso__alpha': 0.1, 'meta_regressor__C': 1.0, 'meta_regressor__gamma': 1.0, 'ridge__alpha': 0.1, 'svr__C': 10.0}\n",
      "Accuracy: -0.08\n"
     ]
    }
   ],
   "source": [
    "cv_keys = ('mean_test_score', 'std_test_score', 'params')\n",
    "\n",
    "for r, _ in enumerate(grid.cv_results_['mean_test_score']):\n",
    "    print(\"%0.3f +/- %0.2f %r\"\n",
    "          % (grid.cv_results_[cv_keys[0]][r],\n",
    "             grid.cv_results_[cv_keys[1]][r] / 2.0,\n",
    "             grid.cv_results_[cv_keys[2]][r]))\n",
    "    if r > 10:\n",
    "        break\n",
    "print('...')\n",
    "\n",
    "print('Best parameters: %s' % grid.best_params_)\n",
    "print('Accuracy: %.2f' % grid.best_score_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Squared Error: 0.1845\n",
      "Variance Score: 0.7330\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD1CAYAAAC1BoUqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAtTklEQVR4nO3deUBUVf8/8PcAo6wySjL4KKIkEuGWQG4JiqEVIoSQoVSSW4v6LXLP1NwqTVu0p0dt0bKwMIWSVBQVzF0UER0pTVJQBgVGBESGYX5/+INkGbYZZpjL+/WPzJ1z534O6JvjufeeK1Kr1WoQEZEgmRi6ACIiaj4MeSIiAWPIExEJGEOeiEjAGPJERALGkCciEjAzQxfwsOTkZEOXQERklDw8PGrd3qJCHtBcaH1kMhnc3Nx0XI1xYN/Z99aGfa/a97oGyJyuISISMIY8EZGAMeSJiASMIU9EJGAMeSIiAWtxV9cQaUOhUEAul0OpVEIsFkMqlUIikRi6LCKDYciTYCgUCmRlZaFi9WylUomsrCwAYNBTq6X1dM3Nmzfx0ksv4dlnn4W/vz+2bNlSo41arcby5cvh5+eHgIAAXLhwQdvDEtUgl8tR/fEIarUacrncQBURGZ7WI3lTU1PMmzcP7u7uKCwsxNixYzFkyBD06NGjsk1SUhIyMjIQHx+Pc+fOYcmSJYiOjtb20ERVKJXKRm0nag20Hsnb29vD3d0dAGBtbQ1nZ+caI6eEhAQEBQVBJBKhX79+KCgoQE5OjraHJqpCLBY3ajtRa6DTq2syMzMhk8nQt2/fKtvlcjkcHBwqXzs4OPC/0KRzUqkUIpGoyjaRSASpVGqgiogMT2cnXouKijBz5kwsWLAA1tbWVd6r7TGy1f8xVpDJZE06fklJSZP3NXbse919v3nzJm7evKmnivSHP3f2vSF0EvJKpRIzZ85EQEAARo4cWeN9BwcHZGdnV77Ozs6Gvb19rZ/V1EWHuGAR+97asO/se4VmXaBMrVbj3XffhbOzMyIiImpt4+vri5iYGKjVaqSkpMDGxkZjyBMRke5oPZJPTk5GbGwsevbsicDAQABAZGQkbty4AQAICwuDj48PEhMT4efnBwsLC6xcuVLbwxIRUQNoHfKenp5IT0+vs41IJMLixYu1PRQRETUS164hIhIwhjwRkYAx5ImIBIwhT0QkYAx5IiIBY8gTEQkYQ56ISMAY8kREAsaQJyISMIY8EZGAMeSJiASMIU9EJGAMeSIiAWPIExEJGEOeiEjAGPJERALGkCciEjCGPBGRgOkk5OfPn49BgwZh9OjRtb5/4sQJeHh4IDAwEIGBgVi/fr0uDktERPXQ+hmvABAcHIzw8HDMnTtXYxtPT09s2LBBF4cjIqIG0knIe3l5ITMzUxcfRXVQKBSQy+VQKpUQi8WQSqWQSCSGLouIWjCdhHxDpKSkYMyYMbC3t8fcuXPh4uJSazuZTNakzy8pKWnyvsagvLwcarW68rVSqURmZiaysrJQWloq6L7XReg/97qw7+x7Q+gl5N3d3XHgwAFYWVkhMTERb775JuLj42tt6+bm1qRjyGSyJu9rDNLT06FUKmtsNzMzg4mJiaD7Xheh/9zrwr6z7xWSk5M1ttfL1TXW1tawsrICAPj4+KCsrAx5eXn6OLRg1BbwdW0nIgL0FPK3bt2qnGpITU1FeXk52rdvr49DC4ZYLG7UdiIiQEfTNZGRkTh58iTy8/Ph7e2NGTNmoKysDAAQFhaGvXv3IioqCqampjA3N8fatWshEol0cehWQyqVIisrq8q8vEgkglQqxc2bNw1YGRG1ZDoJ+bVr19b5fnh4OMLDw3VxqFar4iqa2q6uYcgTkSZ6u7qGtCeRSHjJJBE1Cpc1ICISMIY8EZGAMeSJiASMIU9EJGAMeSIiAWPIExEJGEOeiEjAGPJERALGkCciEjCGPBGRgDHkiYgEjCFPRCRgDHkiIgFjyBMRCRhDnohIwBjyREQCppOQnz9/PgYNGoTRo0fX+r5arcby5cvh5+eHgIAAXLhwQReHJSKieujkyVDBwcEIDw/H3Llza30/KSkJGRkZiI+Px7lz57BkyRJER0fr4tBkQAqFosrjCK2trVFYWFjj8YT17aepHRFpTycjeS8vL9ja2mp8PyEhAUFBQRCJROjXrx8KCgqQk5Oji0OTgSgUCmRlZUGpVAIAlEol8vPzq7zOysqCQqGod7/a2hGRbuhlTl4ul8PBwaHytYODA+RyuT4OTc1ELpdDrVbX2UatVtf4Ode2X23tiEg39PIg79rCQCQS1dpWJpM16RglJSVN3tfYGaLvKpWqQe2USmWV2jTtV71dQ/Hnzr63No3tu15C3sHBAdnZ2ZWvs7OzYW9vX2tbNze3Jh1DJpM1eV9jZ4i+p6enV0651EUsFsPV1bXe/aq3q0v1Of1OnTq1yjl9/p1n3yskJydrbK+X6RpfX1/ExMRArVYjJSUFNjY2GkOejINUKtX4v7EKIpEIUqm03v1qa6dJ9Tl9AJzTJ6qDTkbykZGROHnyJPLz8+Ht7Y0ZM2agrKwMABAWFgYfHx8kJibCz88PFhYWWLlypS4OSwZUMXLOzs5GXlEeVFDBrK0Z8gvyUaosha2lLZy7OMOmnU2t+zX16pq65vRb42ieqD46Cfm1a9fW+b5IJMLixYt1cSgyMLVajau3r+LgpYM49OchHLx0EFmKrDr3sW5rjXYW7dDOvB1sLWwrv+72SDd4OnnCs70nbNW29f7PAIDGKaKKq3Ty8/OrbOclmtTa6WVOnppHxdy0SqVCWloagOYJtYzbGTiYfhCH0g/hYPpBXM+7XuV9q7ZWsDG3gbmZOdqK20IEEQrvF6KgpAB3S+6i8H4hCu8X4gZuaDyGxFICDycPeDp5Vv7Z7ZFuNYJfLBZrDPrqAQ/8G/4AGPTUKjHkjVTF3HT1qQtdhNq13GuVgX4o/RAycjOqvN/BqgN8evpguOtwDHMdBvf/uMPEpPbTO+Xl5Q8C/14BCkoKUHCvAHfu3YHingKXbl7C6X9O4/Q/p5F9JxsJsgQkyBKqHMfTyROe3f4Nfnt7e9y4caNKv0UiUZ2Xc3I6h1ozhryRqus69caGWmZeZuXUy6E/D+HvW39XeV9iKYFPTx8Mcx2G4a7D0btzb42hXp2JicmD6RmLdnW2u6G4gdMZDwK/4s9bd28h/mI84i/GV7braNMRfTv3RU9JT/R16AuvLl7o7tQdmZmZdX5+Q64EIhIihryRqi+06nr/huJGlZH65ZzLVd63tbCFd09vDOs5DMMfG44+XfrA1MRUJ3Vr8h/JfzCm3xiM6TcGwINfVNfzriP5n+Qawb//0n7sx34AQLu27RD4RCA8H/HEYKfBMDczr/XzxWJxs9ZP1FIx5I1UXXPTFe9XyL6TjUPphyqD/U/5n1Xa2pjbwNvF+8FI/bHh6OfYr9lDvT4ikQhd7bqiq11XPN//eQAPgj/jdgaSryXj5NWT2J22G2lZafj++Pf4Ht/DwswCT3V7Ck/3eBrDnIfBUmxZ+VkNvUSTSGgY8i1UfYt4SaXSWufkASDvXh6u5F7BZ6c/w8FLB3Ep+1KV963aWmGoy9DKOfX+XfvDzLTl/1UQiUTo3rE7unfsjhCPEKwKWYXdR3fj3J1z2HFmB05lnMK+y/uw7/I+WIgtEOwejNcGvYZezr04H0+tVsv/l90KVT+pWtvJ1IevNy+6V4SkjCScuH4Cp7NO43Ju1ekXyzaWeKrHUxj+2HAM6zkMHk4eEJsJY/qiW/tueHbws5j37Dxcy72GmJQYbDu1DceuHMMPKT9gx8UdmDF8BuY8Mwd21naGLpdI7xjyLVBDb/gRtRFh5987sWbvGtwuul253aKNBYY8OqTyRKlnN0+0MWujr/INpqtdV8wcMRMzR8zEuevnsPjXxYhNicWqvavwZeKXePvptxHpFwlbS80rphIJDUO+Barrhh8AyC/Kx+cJn+PThE+hKFYAAPp06YMQjxAMdx0Or25eaCtuq69yW6S+jn0R82YMTl09hfdi38PeC3uxdNdSrDuwDrNHzcYM3xmwNrc2dJlEzY4hbyB1zblrOqlaUl6CRbGL8FnCZyi4VwAA8Onpg/Be4Zj0zKQG3THa2nh198Ket/bg8J+HsTB2IZL+TMKCnQvwyf5PMP/Z+XjN5zVYtLEwdJlEzYbPeDWA+h6cUX0Rr9KyUmw9uxWjvh6FZbuWoeBeAUa4jUDi7EQcmn0IQ7oNYcDXY2jPoTg06xD2vb0PA7oPwK27txD5cyR6vNsDXx76EqVlpYYukahZMOT1TKFQIDMzs84HZ0gkEnTu3BmmZqbYdWkXxnw/Bh8lfYS84jwMdRmKP+b+gf2R++Hd09sQXTBaIpEITz/+NI7NP4bfpv+Gfo79cENxA2/88AZcF7ri2yPfokxVZugyiXSKIa9HFSN4TR6eojmfcx7jfx6P+XvnI6sgC+7/ccdv039D4uxEDOkxRB/lCpZIJMLovqORvDAZ0a9Fw62TGzJyM/Dq5lfx+KLHEXUiCuXl5YYuk0gnGPJ6VN8j88RiMbLvZOPlr1+G92pvpFxPQZf2XfDNxG9wbvE5jO47mtMyOmRiYoIQjxCcX3Ie30/6Ho92fBR/5fyF8V+NR9+lfbHzzM56H3FI1NIx5PWorjtUVWoVYv6Kget7rvj++Pdoa9YWiwMW48/lfyJiSITB70AVMlMTU4QPDIdsqQybXt4Exw6OSMtKQ/CXwfBa4YXd53cz7MloMeT1SNP6KSk3UhAeHY55sfNQcK8Az/V+Dhfev4AlY5bwyg89EpuJMXnoZPy1/C+sC1sHB1sHJP+TjOc+fw5PffQUDl46aOgSiRqNIa9H1a+aySvOw3v73sNL0S8h7WYanOycEPNmDHbN2IVH7R81YKWtW1txW0z3nY4rK65gdchq2Fnb4eiVo/Bd44sRa0bg2JVjhi6RqMEY8npUcdWMmZkZfrv0GwK3BiLmYgzamLXBQv+FuPj+RQT2C+S8ewth2dYSs0bNwtUPrmJZ4DLYWtjiwKUDGPzhYPh/7o8z/5wxdIlE9dJJyCclJWHUqFHw8/PDxo0ba7x/4sQJeHh4IDAwEIGBgVi/fr0uDmuU7pvcx+x9s7Fg7wIo7ikwwm0E0pakYVnQMli2tTR0eVQLG3MbLBy9EFc/uIqF/gth3dYav5//HR7LPTD2y7FIy0ozdIlEGmkd8iqVCkuXLsVXX32FuLg47Nq1C5cvX67RztPTE7GxsYiNjcX06dO1PaxR2nlmJ3ot7oVdqbtga2GLbyZ+g31v74OL1MXQpVEDtLdqj2VBy/D3B39j1shZMBebY8eZHejzfh9M2DQBf8n/MnSJRDVoHfKpqalwcnKCo6Mj2rRpA39/fyQkJNS/owAoFAqkp6cjLS0N6enplXesVldwrwCvbn4VwV8G43bh7crRe8SQCE7NGKGONh2xOnQ1rqy8gunDp8PMxAw/nvwRbovcMGnzJGTczjB0iUSVRGotrw3bs2cPDh8+jBUrVgAAYmJikJqaikWLFlW2OXHiBGbOnAmpVAp7e3vMnTsXLi41R6/JycmwtGzalEVJSQnMzWt/KlBzKC8vr/WyOpFIVOXReKczT2P+ngc3NLU1bYvIoZGY8MQEmIh0dzpE331vSVpC37MKsrDh+AbsvLATKrUKZiZmCO0dimkDpuERy0dqPI+2oY9OrE9L6LuhsO9V+15cXAwPD49a22u9QJmmoHuYu7s7Dhw4ACsrKyQmJuLNN99EfHx8jf0AwM3NrUl1yGSyJu/bFOnp6bVe925mZgZXV1fcV97HothFWB2/Gmq1Gv279sfWyVvh1kn3Neq77y1JS+i7G9zw9ICncTnnMt7/7X38cOIHRJ2Lws4LOzGuzzhEeETAzvLBWvYV/146deqk9YNMWkLfDYV9r9r35ORkje21HlI4ODggOzu78rVcLoe9vX2VNtbW1rCysgIA+Pj4oKysDHl5edoeWu8enp6pazng1MxUPLnySazauwoiiLDQfyGOzT/WLAFPLUcP+x74ftL3OL/4PEI8QlBSVoItZ7bg2c3P4ssTX0KpevB3RqVSVVmQjqg5aR3yvXv3RkZGBq5fv47S0lLExcXB19e3Sptbt25VjmBSU1NRXl6O9u3ba3tovaq+cmRtVOUqfHf2O3it8EJqZip62PfAH3P/wLKgZa3ioR30gHtnd0S/Fo2fw36GT3cf3FPew3+P/xcvRL2AtOwHV+I8vCAdUXPSerrGzMwMixYtwuTJk6FSqTB27Fi4uLggKioKABAWFoa9e/ciKioKpqamMDc3x9q1a43uhGN9685kFWRhYfxCnM46DQCY5j0NH4d+zAdTtGJ9OvfB+jHrcSrzFN5PeB+Xcy8j/OdwvOr5Kl578jVDl0ethE4eGuLj4wMfH58q28LCwiq/Dg8PR3h4uC4OZTCaRvBqtRqxslh8mPghikqLIG0nxdevfA3/Pv56rpBamoqHrXt18cL2Cdux/th6fHfmO2w6tQmH/j6EVf6r0Au9DF0mCRyfDNVAtT2t6U7JHSw9sBTxfz04iRzcPxgbwjfgEZtHDFEitTAVJ1Zv3LgBczNzzBo6C77Ovli4byH+yv0LoVtDMTt7Nsa7jwfKUeMJYUS6wGUNGqj6ujNnb5xFyI8hiP8rHjbmNtgSsQXbX9vOgKcqJBIJHn/8cXTp0gVisRj9O/dH7CuxmDJkCsrKy/BB/Ad44YcXcDn3co0nhBHpAkfyDVQ5Krt5A/879j/89/h/oVKr8GT3JxE1JQrOHZ0NWyC1aBKJpMoIfWPvjRjSeQgW7F4AWY4ML0a9iPnD5iPYPRhyuZyjedIZjuQboURUgpm7Z2LdsXVQqVWYPWo2/pjzBwOemsSjkwd2hO9A0ONBuK+6jyUJSzBv7zwoihSGLo0EhCP5Btp/cT8mfDUBOXdz0NGmI75/9XuM6jXK0GWREROLxbCCFZb5LcOTjk9i2YFl+D39d1zMuYiYjjHo69jX0CWSAHAkX48yVRkW7lyIkZ+ORM7dHAx3HY5zi84x4ElrD5/nCXgsANte3AaXR1yQkZ+BASsHYEPiBj6RirTGkK9DZl4mhn88HCt+XwERRHh/zPvYF7kPnSSdDF0aCUDF8wUqnhjmKnVFYmQipgydgvtl9/Ha1tcQtikMBfcKDFwpGTNO12gQlxqHV759BbmFuehk2wk/TvkRw1yHGbosEpjqJ2QBYOPLGzHMdRimfT8NP536Ccn/JOPnaT/jia5PGKZIMmqtciRf1xLBpWWlmBU9C6PXjUZuYS6e6fUMzi0+x4AnvRo/YDySFyajb5e+uJxzGQM/GIgvD33J6RtqtFYX8tXXoHn42uSrt65i6KqhWBO/BqYmpvho7EeImxGHjjYdDVw1tUY9HXri2PxjmOY9DaVlpXjjhzcwbsM43Cm+Y+jSyIi0upCvbQ0atVqN7w5/hyeWPYGTV0+ia4euODznMOY8M0dna38TNYVFGwv876X/IWpKFGzMbRCdHA2P5R5I/kfz0rJED2sVc/IKhQJyubzW9Wful93Hx4c/xrbUbQCAoH5B+Hri1+hg1UHfZRq9h7/PvEVft1588kV4OHnghQ0vIOV6CgZ/OBizh87GsseWGd1if6Rfgh+m1rVEcEZ+BsJ/Dse21G0Qm4rx+YufY8cbOxjwTVDXNBjphovUBcfmH8Mbw95AaVkpVhxcgdD/hUJRrDB0adSCCTbkK06uZmZm1nqyKu5SHMZFjcOlW5fgaOuI+OnxmDFiBkdFTaRpGoxrpuuWudgcX0z4Aj9N/QlWbazwy5lf0H9Zf5zOOG3o0qiFEmTI1zV6Ly4txnv73sO8vfNQrCzGc67P4Y9Zf2BYr2H6L1RA6npSFuneC14vYPuE7ejftT+u3r6KwR8Oxmf7P+PVN1SDIENe0wM+LuZcxLiocYi5GIO2Zm2x4aUN2PXOLnR16GqAKoWl4oaehm4n7Tm1d8LReUcxffh0KFVKvPXTWwj+bzDyi/INXRq1IIIM+eqjx3J1Ob478x0m/DQBGYoMuDzigoP/dxBTvadyekZHqi/FDDx4oLtUKjVQRa1DW3FbrBu/Dttf2452Fu0QkxKD/sv64+TVk4YujVoInYR8UlISRo0aBT8/P2zcuLHG+2q1GsuXL4efnx8CAgJw4cIFXRxWo4dHj7eKbuH1mNex+vBqlJWXYXy/8UiMTMSgxwY1aw2tTfVb9MViMTp37syra/RkrMdYnH3vLDydPJGRm4GnPnoKn+z7hNM3pH3Iq1QqLF26FF999RXi4uKwa9cuXL58uUqbpKQkZGRkID4+HsuWLcOSJUu0PWydKkaVB64cQPDWYBy9dhQScwl+eOUH/PDmD+jUkWvPNAeJRAJXV1f06tULrq6uDHg9c+7ojD/m/oGZI2ZCqVIi8udIBH0RhLyiPEOXRgakdcinpqbCyckJjo6OaNOmDfz9/ZGQkFClTUJCAoKCgiASidCvXz8UFBQgJydH20NrZNLWBB8e+RD/t+v/oChRYLDTYBybfQzjnxrfbMckagnaitvisxc/w47Xd8DWwha/nvsVTyx9AsevHDd0aWQgWoe8XC6Hg4ND5WupVFrjsrnqbRwcHJrt0rrE9ET0eb8Ptp7airZmbfHJuE9weMFhPNb1sWY5HlFL9Hz/53H2vbPw6uaFa3nXMHT1g+U6OH3T+mh9x2ttf2mqn4BrSJsKMpmsSXXcKbyDif+biO+Sv4MaarhL3fHBMx+gh10PpKenN+kzjUVJSUmTv2/Gjn2vu++bAjdh7eG1+O7Md5gVPQu7kndh5aiVkFhIarQtLy+v8m9VJBK12GU9+HNveN+1DnkHBwdkZ2dXvpbL5bC3t6+zTXZ2do02Fdzc3BpdQ3p2OsK3hONK7hWYmphiof9CvPvcuxCbtY7L92QyWZO+b0LAvtff9y29tiA4JRgTv52IQ38fwrifxmHblG0Y3GNwZZuKe0uq69SpU4s8t8Kfe9W+JydrXstI61/TvXv3RkZGBq5fv47S0lLExcXB19e3ShtfX1/ExMRArVYjJSUFNjY2GkO+KVbtWYUruVfg6uCKY/OOYcmYJa0m4IkaIrBfIFIWpWBA9wG4nncd3qu9sXrvapSXlwPgHctCpvVI3szMDIsWLcLkyZOhUqkwduxYuLi4ICoqCgAQFhYGHx8fJCYmws/PDxYWFli5cqXWhT/svdHvobtld0QGRsKyraVOP5tIKJzsnJA0JwkLdi7Amvg1mLN9Dg6lH8KWiC28Y1nAdLIKpY+PD3x8fKpsCwsLq/xaJBJh8eLFujhUrbo90g1je49lwBPVo41ZG3wc+jF8evrglW9ewe/nf0e/pf2w6plV6CPtU6M971g2fi3zrAoRNauAvgFIWZSCQY8OQpYiCy///DK+Pv01ytXllW14x7IwMOSJWqmudl2ROCsRc0bNgapchU+PfIrpv05HXnEe71gWEIY8USsmNhPjo5CPsGvGLnSw6oDDGYcRFh2GWya3GPACwZAnIvj38UfKohQM6TEENxQ3MOzjYVi9dzVvnhIAhjwRAQAcOzji4DsHMXvUbKjKVZizfQ6Cvgji0sVGjiFPRJXEZmKsClmF2DdjIbGU4Ndzv6L/sv44dfWUoUujJmLIE1ENY/qNwZmFZyqXLh7y0RCsS1jH6RsjxJAnolp179gdP730E8L6hkGpUmLmtpnw/8Sf0zdGhiHfBBUPCU9LS0N6ejoUCoWhSyLSuaysLBTfLcaCYQuw5rk1sG5jjd2y3XB/zx17zu4xdHnUQAz5Rqr+kHClUomsrCwGPQlOfv6/I/aRLiPx8/if0VvaGzfv3sToL0dj4faFUJWrDFih7glxAMeQbyQu5EStlaOtI7aEbkGERwRUahVW7F0Bv7V+uKG4YejSdEKoAziGfCNxISdqzcSmYkQ+FYkNQRvQwaIDDqYfRN/3+yIuNc7QpWlNqAM4hnwjaVqwiQs5kdC0b99e43uDnQbjlwm/wO9xP9wuvI3R60bj7Z/exn3lfT1WqFtCHcAx5Bup4iHhD+NCTiREnTt3rjPope2k2PN/e/DR2I9gZmqGT/d/isEfDsaf2X/qsUrdEeoAjiHfSBKJBJ07d678wXMhJxKyzp07o0uXLrUObDp16gQTExPMeWYO/pjzB7rZdcOZa2fwxLIn8MXBLyofSGIshDqAY8g3gUQigaurK3r16gVXV1cGPAlaQwY2A5wHIGVRCiYMmIDi0mJM/3E6nvnsGWTmZRqo6sYT6gBOJw8NISJhk0gk9YadraUttk7eiuefeB7Ttk7Dvov70GtJL6wLW4fwgeE1RsktUUP6aWw4kicinRrrMRZpS9IQ0DcAd+7dwcvfvIyQ/4Xg1t1bhi6tVdIq5BUKBSIiIjBy5EhERETgzp07tbbz9fVFQEAAAgMDERwcrM0hicgIONg6IPbNWHz9ytewMbfBjjM70GtxL8SmxBq6tFZHq5DfuHEjBg0ahPj4eAwaNAgbN27U2HbLli2IjY3Fjh07tDkkERkJkUiEV596FamLU+HT0wc5d3MQ9EUQIr6NwJ3i2geEpHtahXxCQgKCgoIAAEFBQdi/f78uaiIiAen2SDcceOcAPhn3CczF5th8dDN6LemF6NPRXNVSD0RqLb7Lnp6eOH36dOVrLy8vnDpVc91pX19f2NraQiQSYdy4cRg3blytn5ecnAxLS8sm1VJSUgJzc/Mm7Wvs2Hf23Vhcyb2CBXsW4Lz8PABggOMALBi+AC6PuDTqc4yx77pSW9+Li4vh4eFRa/t6r66ZOHEibt++XWP7W2+91eCioqKiIJVKkZubi4iICDg7O8PLy6vWtm5ubg3+3IfJZLIm72vs2Hf23Vh0UnTCNrtt2JayDZ8f/Rwnrp9A8NZgzPCdgSUBS2BradugzzHGvutKbX1PTk7W2L7ekN+8ebPG9+zs7JCTkwN7e3vk5OSgQ4cOtbaruJnAzs4Ofn5+SE1N1RjyRCRMFQuAqdVqvND7BYzsMRLrj69H9PlofLr/U/x44kd8OPZDvDLoFZiY8MI/XdHqO+nr64uYmBgAQExMDEaMGFGjTXFxMQoLCyu/PnLkCFxcGvdfMyIyftUXAJNYSLBw+EL88tIvGNJjCHLu5uDVza9i8IeD+bhBHdIq5KdOnYojR45g5MiROHLkCKZOnQrgwQ9zypQpAIDc3FyMHz8eY8aMQWhoKHx8fODt7a195URkVDQt9NWjfQ8cnnMYWydtRSfbTjhx9QQGfDAAU76bwmvrdUCrO17bt2+PLVu21NgulUqxadMmAICjoyN+/fVXbQ5DRAIgFotrDXqxWAyRSIQJAydgTL8xWLZrGT7d/ym+OvwVtidvx9IxS/H6sNdhZsob9JuCE19EpBcNWQDMxtwGq0JW4fyS8xj5+EgoihWYuW0m+i/rj8T0RH2XLAgMeSLSi8YsAObq4Io9b+3Bzjd2optdN5zPOo9hHw9D2MYwo1r0rCVgyBOR3jRmBVeRSISgJ4JwcelFvD/mfZiLzbHt1DY8tugxfHn8S+QX5Wvcl/7FkCeiFs2ijQUWBSzCpWWXMLb/WBTdL8K6o+vQdW5XzIqexZF9PRjyRGQUnOycsP317Tg46yAGdR2EwvuFWBO/Bs4LnBHxbQRkN2WGLrFFYsgTkdFQKBTohE7Y+PxGbA/fjuf7Pg9VuQqbj27G44seR+D6QBy9fNTQZbYoDHkiMgoVd8xWXIbpaueKZcOX4fSc03h92OswF5vj13O/YshHQ/DUR0/ht3O/Gd0jCJsDQ56IjEL1O2YBQK1Ww1Jlif9O+C/++fAfvPvcu5BYSnDk8hGMWT8Gfd7vg0/3f9qq5+0Z8kRkFDTdMVux3b6dPZY/vxzXPrqGtS+sRZf2XXDhxgW8/dPbcJzriMEfDsYn+z7B9bzr+izb4BjyRGQUKq6vr2+7jbkN3vZ7G1dWXsG2qdsQ3D8Y5mJzHLtyDJE/R6Lr3K4Y9MEgrI1fi2u51/RRukHxPmEiMgpSqbRyFcsK1e+YfVhxYTH6tesH96HuePepd3Eu7xx+l/2OuPNxOP73cRz/+zjeiX4HT3Z/EqEeoQjxCEG3R7rpqTf6w5AnIqNQceOUXC6HUqmEWCyGVCqt9Yaqh5c1BoA2ojZ48pEn8fyLz0McIcbv539H9OloxJ2Pw8mrJ3Hy6knM3j4bXt28EOoZCv/e/njM4TFBLHnMkCcioyGRSCCRSCCTyeDq6qqxnaaTtHK5HK6urgj1DEWoZyiK7hdhd9puRJ+Oxq7UXTiVcQqnMk5hzvY5sLWwxYDuAzDQeSAGOg+EVzcv2Fnb1Vh/p6VjyBOR4NR3kraCVVsrhHiEIMQjBMX3i7Hnwh78kvwLEv9MRJYiC/EX4xF/Mb6yvdhUDDtrO3Sw7PDgT6sOsLOyq/J1B6ua75mLDfeoQoY8EQlOXcsaa2LZ1hLB/YMR3D8YAJCZl4kTV09Uzt+nXE9B4f1CZN/JRvad7EbVY9HG4t9fAP//TxtzGxTdL8Ld+3dRWlaKhf4LMfyx4Y3raAMw5IlIcBp7krZCVlYW8vP/XfhsoMNAjPUYW/m6RFmCvKI85BbmIrcot/LrvKI85BblVvn64W33Su8hszQTmfmar9cf6DyQIU9E1BCNOUlboXrAA6h8bWVlVeWzHKWO6N2ld4NqUavVKLpfVOOXwt2Su7A2t0Y783boYNUBXt2a57nXWoX87t27sX79ely5cgXR0dHo3bv2TiclJWHFihUoLy9HaGho5WMCiYiaS8VJ2oaqHvAPb1coFJX/K1AqlcjKyqo8Rn1EIhGsza1hbW4NJzunGu8rFArI5XJcvHixQb+MGkurkO/ZsyfWrVuHxYsXa2yjUqmwdOlSfPvtt5BKpQgJCYGvry969OihzaGJiPRG05U6TQnjilBXKpUwMTGBWq1u8i+QhtDqItBHH30Uzs7OdbZJTU2Fk5MTHB0d0aZNG/j7+yMhIUGbwxIRGZymK3jqUn2RtfLyco2/QHSl2a/0l8vlcHBwqHwtlUp12gEiIl1o3759rds13RBV15U6mtR2/X5tmvILRJN6p2smTpyI27dv19j+1ltv4emnn673ALV1qK6bCWSypi38X1JS0uR9jR37zr63Nvrsu6ZQLisra3QNKpWqwW01fXZj+15vyG/evLnBH1YbBwcHZGf/e02pXC6Hvb29xvZubm5NOo5MJmvyvsaOfWffWxt99/3heXRtTo6mp6fXO0oXiUQaH3AO1N735ORkjZ/X7JdQ9u7dGxkZGbh+/TqkUini4uKwZs2a5j4sEZHONPZKHU1qu34fAExNTaFSqZrl6hqt5uT37dsHb29vnD17FtOmTcOkSZMAPBitT5kyBQBgZmaGRYsWYfLkyXjuuefw7LPPwsXFRfvKiYiMjEQiQefOnSvn88ViMbp06QI3Nzf06tULrq6uOg14QMuRvJ+fH/z8/Gpsl0ql2LRpU+VrHx8f+Pj4aHMoIiJB0NX/ChrK+NfRJCIijRjyREQCxpAnIhIwhjwRkYAx5ImIBIwhT0QkYAx5IiIBY8gTEQkYQ56ISMAY8kREAsaQJyISMIY8EZGAMeSJiASMIU9EJGAMeSIiAWPIExEJGEOeiEjAjD7kFQoF0tPToVKpkJ6eDoVCYeiSiIhaDK0e/7d7926sX78eV65cQXR0NHr37l1rO19fX1hZWcHExASmpqbYsWOHNoetpFAoqjwUV6lUIisrCwD0+ngtIqKWSquQ79mzJ9atW4fFixfX23bLli3o0KGDNoerQS6X13jquVqthlwuZ8gTEUHLkH/00Ud1VUeTKJXKRm0nImpttAr5xpg0aRJEIhHGjRuHcePGaWwnk8l0cjxdfY4xKCkpaVX9fRj7zr63No3te70hP3HiRNy+fbvG9rfeegtPP/10gw4SFRUFqVSK3NxcREREwNnZGV5eXrW2dXNza9BnAjXn5AFAJBKhc+fOrWq6RiaTNer7JiTsO/ve2tTW9+TkZI3t6w35zZs3a12UVCoFANjZ2cHPzw+pqakaQ74xKoJcLpdDqVRCLBZDKpW2qoAnIqpLs19CWVxcjMLCwsqvjxw5AhcXF519vkQigaurK0xNTeHq6sqAJyJ6iFYhv2/fPnh7e+Ps2bOYNm0aJk2aBODByHrKlCkAgNzcXIwfPx5jxoxBaGgofHx84O3trX3lRERUL61OvPr5+cHPz6/GdqlUik2bNgEAHB0d8euvv2pzGCIiaiK9XV1DRMKmUCh4fqwFYsgTkdZ493nLZfRr1xCR4dV19zkZFkOeiLTGu89bLoY8EWlNLBY3ajvpD0OeiLQmlUohEomqbBOJRJU3QpLh8MQrEWmNd5+3XAx5ItIJiUTCUG+BOF1DRCRgDHkiIgFjyBMRCRhDnohIwBjyREQCJlJXvxfZgOp6ugkREWnm4eFR6/YWFfJERKRbnK4hIhIwhjwRkYAJ4o7XpKQkrFixAuXl5QgNDcXUqVMNXZJezJ8/H4cOHYKdnR127dpl6HL06ubNm5gzZw5u374NExMTvPDCC3jllVcMXVazu3//PiZMmIDS0lKoVCqMGjUKM2fONHRZeqVSqTB27FhIpVJs2LDB0OXoja+vL6ysrGBiYgJTU1Ps2LGjYTuqjVxZWZl6xIgR6mvXrqnv37+vDggIUP/111+GLksvTp48qU5LS1P7+/sbuhS9k8vl6rS0NLVarVbfvXtXPXLkyFbxcy8vL1cXFhaq1Wq1urS0VB0SEqI+e/asYYvSs2+++UYdGRmpnjp1qqFL0avhw4erc3NzG72f0U/XpKamwsnJCY6OjmjTpg38/f2RkJBg6LL0wsvLC7a2toYuwyDs7e3h7u4OALC2toazs3OreECFSCSClZUVAKCsrAxlZWU1Vn8UsuzsbBw6dAghISGGLsVoGH3Iy+VyODg4VL6WSqWt4h87/SszMxMymQx9+/Y1dCl6oVKpEBgYiMGDB2Pw4MGtpt8AsHLlSsyePRsmJkYfXU0yadIkBAcH46effmrwPkb/nVLXcgVoaxrZtHZFRUWYOXMmFixYAGtra0OXoxempqaIjY1FYmIiUlNT8eeffxq6JL04ePAgOnTogF69ehm6FIOIiorCzp07sWnTJvzwww84depUg/Yz+pB3cHBAdnZ25Wu5XA57e3sDVkT6olQqMXPmTAQEBGDkyJGGLkfv2rVrhwEDBuDw4cOGLkUvzpw5gwMHDsDX1xeRkZE4fvw4Zs2aZeiy9KbiASx2dnbw8/NDampqg/Yz+pDv3bs3MjIycP36dZSWliIuLg6+vr6GLouamVqtxrvvvgtnZ2dEREQYuhy9ycvLQ0FBAQCgpKQER48ehbOzs4Gr0o933nkHSUlJOHDgANauXYuBAwfi448/NnRZelFcXIzCwsLKr48cOQIXF5cG7Wv0l1CamZlh0aJFmDx5cuWlVQ3tvLGLjIzEyZMnkZ+fD29vb8yYMQOhoaGGLksvkpOTERsbi549eyIwMBDAg++Hj4+PgStrXjk5OZg3bx5UKhXUajWeeeYZDB8+3NBlUTPLzc3Fm2++CeDBOZnRo0fD29u7QftyWQMiIgEz+ukaIiLSjCFPRCRgDHkiIgFjyBMRCRhDnohIwBjyREQCxpAnIhIwhjwRkYD9P4wgC6TAxxJ5AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Evaluate and visualize the fit\n",
    "print(\"Mean Squared Error: %.4f\"\n",
    "      % np.mean((grid.predict(X) - y) ** 2))\n",
    "print('Variance Score: %.4f' % grid.score(X, y))\n",
    "\n",
    "with plt.style.context(('seaborn-whitegrid')):\n",
    "    plt.scatter(X, y, c='lightgray')\n",
    "    plt.plot(X, grid.predict(X), c='darkgreen', lw=2)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**\n",
    "\n",
    "The `StackingCVRegressor` also enables grid search over the `regressors` and even a single base regressor. When there are level-mixed hyperparameters, `GridSearchCV` will try to replace hyperparameters in a top-down order, i.e., `regressors` -> single base regressor -> regressor hyperparameter. For instance, given a hyperparameter grid such as\n",
    "\n",
    "    params = {'randomforestregressor__n_estimators': [1, 100],\n",
    "    'regressors': [(regr1, regr1, regr1), (regr2, regr3)]}\n",
    "    \n",
    "it will first use the instance settings of either `(regr1, regr2, regr3)` or `(regr2, regr3)` . Then it will replace the `'n_estimators'` settings for a matching regressor based on `'randomforestregressor__n_estimators': [1, 100]`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## StackingRegressor\n",
      "\n",
      "*StackingRegressor(regressors, meta_regressor, verbose=0, use_features_in_secondary=False, store_train_meta_features=False, refit=True, multi_output=False)*\n",
      "\n",
      "A Stacking regressor for scikit-learn estimators for regression.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `regressors` : array-like, shape = [n_regressors]\n",
      "\n",
      "    A list of regressors.\n",
      "    Invoking the `fit` method on the `StackingRegressor` will fit clones\n",
      "    of those original regressors that will\n",
      "    be stored in the class attribute\n",
      "    `self.regr_`.\n",
      "\n",
      "\n",
      "- `meta_regressor` : object\n",
      "\n",
      "    The meta-regressor to be fitted on the ensemble of\n",
      "    regressors\n",
      "\n",
      "\n",
      "- `verbose` : int, optional (default=0)\n",
      "\n",
      "    Controls the verbosity of the building process.\n",
      "    - `verbose=0` (default): Prints nothing\n",
      "    - `verbose=1`: Prints the number & name of the regressor being fitted\n",
      "    - `verbose=2`: Prints info about the parameters of the\n",
      "    regressor being fitted\n",
      "    - `verbose>2`: Changes `verbose` param of the underlying regressor to\n",
      "    self.verbose - 2\n",
      "\n",
      "\n",
      "- `use_features_in_secondary` : bool (default: False)\n",
      "\n",
      "    If True, the meta-regressor will be trained both on\n",
      "    the predictions of the original regressors and the\n",
      "    original dataset.\n",
      "    If False, the meta-regressor will be trained only on\n",
      "    the predictions of the original regressors.\n",
      "\n",
      "\n",
      "- `store_train_meta_features` : bool (default: False)\n",
      "\n",
      "    If True, the meta-features computed from the training data\n",
      "    used for fitting the\n",
      "    meta-regressor stored in the `self.train_meta_features_` array,\n",
      "    which can be\n",
      "    accessed after calling `fit`.\n",
      "\n",
      "\n",
      "**Attributes**\n",
      "\n",
      "- `regr_` : list, shape=[n_regressors]\n",
      "\n",
      "    Fitted regressors (clones of the original regressors)\n",
      "\n",
      "\n",
      "- `meta_regr_` : estimator\n",
      "\n",
      "    Fitted meta-regressor (clone of the original meta-estimator)\n",
      "\n",
      "\n",
      "- `coef_` : array-like, shape = [n_features]\n",
      "\n",
      "    Model coefficients of the fitted meta-estimator\n",
      "\n",
      "\n",
      "- `intercept_` : float\n",
      "\n",
      "    Intercept of the fitted meta-estimator\n",
      "\n",
      "\n",
      "- `train_meta_features` : numpy array,\n",
      "\n",
      "    shape = [n_samples, len(self.regressors)]\n",
      "    meta-features for training data, where n_samples is the\n",
      "    number of samples\n",
      "    in training data and len(self.regressors) is the number of regressors.\n",
      "\n",
      "\n",
      "- `refit` : bool (default: True)\n",
      "\n",
      "    Clones the regressors for stacking regression if True (default)\n",
      "    or else uses the original ones, which will be refitted on the dataset\n",
      "    upon calling the `fit` method. Setting refit=False is\n",
      "    recommended if you are working with estimators that are supporting\n",
      "    the scikit-learn fit/predict API interface but are not compatible\n",
      "    to scikit-learn's `clone` function.\n",
      "\n",
      "**Examples**\n",
      "\n",
      "For usage examples, please see\n",
      "    https://rasbt.github.io/mlxtend/user_guide/regressor/StackingRegressor/\n",
      "\n",
      "### Methods\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit(X, y, sample_weight=None)*\n",
      "\n",
      "Learn weight coefficients from training data for each regressor.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "\n",
      "- `y` : numpy array, shape = [n_samples] or [n_samples, n_targets]\n",
      "\n",
      "    Target values. Multiple targets are supported only if\n",
      "    self.multi_output is True.\n",
      "\n",
      "\n",
      "- `sample_weight` : array-like, shape = [n_samples], optional\n",
      "\n",
      "    Sample weights passed as sample_weights to each regressor\n",
      "    in the regressors list as well as the meta_regressor.\n",
      "    Raises error if some regressor does not support\n",
      "    sample_weight in the fit() method.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `self` : object\n",
      "\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit_transform(X, y=None, **fit_params)*\n",
      "\n",
      "Fit to data, then transform it.\n",
      "\n",
      "    Fits transformer to `X` and `y` with optional parameters `fit_params`\n",
      "    and returns a transformed version of `X`.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : array-like of shape (n_samples, n_features)\n",
      "\n",
      "    Input samples.\n",
      "\n",
      "\n",
      "- `y` :  array-like of shape (n_samples,) or (n_samples, n_outputs),                 default=None\n",
      "\n",
      "    Target values (None for unsupervised transformations).\n",
      "\n",
      "\n",
      "- `**fit_params` : dict\n",
      "\n",
      "    Additional fit parameters.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `X_new` : ndarray array of shape (n_samples, n_features_new)\n",
      "\n",
      "    Transformed array.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*get_params(deep=True)*\n",
      "\n",
      "Return estimator parameter names for GridSearch support.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict(X)*\n",
      "\n",
      "Predict target values for X.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `y_target` : array-like, shape = [n_samples] or [n_samples, n_targets]\n",
      "\n",
      "    Predicted target values.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict_meta_features(X)*\n",
      "\n",
      "Get meta-features of test-data.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : numpy array, shape = [n_samples, n_features]\n",
      "\n",
      "    Test vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `meta-features` : numpy array, shape = [n_samples, len(self.regressors)]\n",
      "\n",
      "    meta-features for test data, where n_samples is the number of\n",
      "    samples in test data and len(self.regressors) is the number\n",
      "    of regressors. If self.multi_output is True, then the number of\n",
      "    columns is len(self.regressors) * n_targets\n",
      "\n",
      "<hr>\n",
      "\n",
      "*score(X, y, sample_weight=None)*\n",
      "\n",
      "Return the coefficient of determination :math:`R^2` of the\n",
      "    prediction.\n",
      "\n",
      "    The coefficient :math:`R^2` is defined as :math:`(1 - \\frac{u}{v})`,\n",
      "    where :math:`u` is the residual sum of squares ``((y_true - y_pred)\n",
      "** 2).sum()`` and :math:`v` is the total sum of squares ``((y_true -\n",
      "y_true.mean()) ** 2).sum()``. The best possible score is 1.0 and it\n",
      "\n",
      "can be negative (because the model can be arbitrarily worse). A\n",
      "\n",
      "constant model that always predicts the expected value of `y`,\n",
      "    disregarding the input features, would get a :math:`R^2` score of\n",
      "    0.0.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : array-like of shape (n_samples, n_features)\n",
      "\n",
      "    Test samples. For some estimators this may be a precomputed\n",
      "    kernel matrix or a list of generic objects instead with shape\n",
      "    ``(n_samples, n_samples_fitted)``, where ``n_samples_fitted``\n",
      "    is the number of samples used in the fitting for the estimator.\n",
      "\n",
      "\n",
      "- `y` : array-like of shape (n_samples,) or (n_samples, n_outputs)\n",
      "\n",
      "    True values for `X`.\n",
      "\n",
      "\n",
      "- `sample_weight` : array-like of shape (n_samples,), default=None\n",
      "\n",
      "    Sample weights.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `score` : float\n",
      "\n",
      "    :math:`R^2` of ``self.predict(X)`` wrt. `y`.\n",
      "\n",
      "**Notes**\n",
      "\n",
      "The :math:`R^2` score used when calling ``score`` on a regressor uses\n",
      "    ``multioutput='uniform_average'`` from version 0.23 to keep consistent\n",
      "    with default value of :func:`~sklearn.metrics.r2_score`.\n",
      "    This influences the ``score`` method of all the multioutput\n",
      "    regressors (except for\n",
      "    :class:`~sklearn.multioutput.MultiOutputRegressor`).\n",
      "\n",
      "<hr>\n",
      "\n",
      "*set_params(**params)*\n",
      "\n",
      "Set the parameters of this estimator.\n",
      "\n",
      "    Valid parameter keys can be listed with ``get_params()``.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "self\n",
      "\n",
      "### Properties\n",
      "\n",
      "<hr>\n",
      "\n",
      "*coef_*\n",
      "\n",
      "None\n",
      "\n",
      "<hr>\n",
      "\n",
      "*intercept_*\n",
      "\n",
      "None\n",
      "\n",
      "<hr>\n",
      "\n",
      "*named_regressors*\n",
      "\n",
      "None\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.regressor/StackingRegressor.md', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}