{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# H2O.ai XGBoost GPU Benchmarks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In this notebook, we benchmark the latest version of [XGBoost](https://github.com/h2oai/xgboost), the well-known Kaggle-winning gradient boosting algorithm, and in particular, the [XGBoost GPU plugin](https://github.com/h2oai/xgboost/blob/master/plugin/updater_gpu/README.md). We also showcase the integration of XGBoost (incl. the GPU version) into H2O."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "## For comparison between 1 GPU and 1 CPU, we use only 1 CPU:\n",
    "#numactl -C 0 -N 0 -m 0 jupyter notebook\n",
    "\n",
    "## This will ensure that we only use the first CPU on multi-CPU systems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "![1CPU](./1CPUonNUMA.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "## First time only: install xgboost and H2O, and restart the kernel afterwards\n",
    "if False:\n",
    "    ## Build XGBoost from source and install its Python module\n",
    "    import os\n",
    "    os.system(\"mkdir -p tmp && cd tmp && git clone https://github.com/h2oai/xgboost --recursive && cd xgboost && mkdir build && cd build && cmake .. -DPLUGIN_UPDATER_GPU=ON -DCUB_DIRECTORY=../cub -DCUDA_NVCC_FLAGS=\\\"--expt-extended-lambda -arch=sm_30\\\" && make -j; make; cd ../python-package && python3.6 setup.py install\")\n",
    "\n",
    "    ## Download and install H2O and its Python module\n",
    "    os.system(\"cd tmp && wget http://h2o-release.s3.amazonaws.com/h2o/rel-vajda/1/h2o-3.10.5.1.zip && unzip h2o-3.10.5.1.zip\")\n",
    "    os.system(\"python3.6 -m pip install h2o-3.10.5.1/python/h2o-3.10.5.1-py2.py3-none-any.whl --upgrade\")\n",
    "    \n",
    "    ## restart the kernel!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import xgboost as xgb\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import scipy as sp\n",
    "import os\n",
    "import time\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "path = \"/opt/higgs_head_2M.csv\"\n",
    "if not os.path.exists(path):\n",
    "    os.system(\"cd /opt/ && wget https://s3.amazonaws.com/h2o-public-test-data/bigdata/laptop/higgs_head_2M.csv\")\n",
    "num_class = 2\n",
    "num_round = 100\n",
    "learn_rate = 0.02\n",
    "max_depth = 10\n",
    "\n",
    "## Parse data into a Pandas Frame\n",
    "df = pd.read_csv(path, header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2000000, 27)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_target = df.iloc[:,0]\n",
    "df.drop(df.iloc[:,0], axis=1, inplace=True)\n",
    "cols = df.columns.values\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "train = df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "train_target = df_target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2000000, 27)\n"
     ]
    }
   ],
   "source": [
    "print(train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Architecture:          x86_64\r\n",
      "CPU op-mode(s):        32-bit, 64-bit\r\n",
      "Byte Order:            Little Endian\r\n",
      "CPU(s):                80\r\n",
      "On-line CPU(s) list:   0-79\r\n",
      "Thread(s) per core:    2\r\n",
      "Core(s) per socket:    20\r\n",
      "Socket(s):             2\r\n",
      "NUMA node(s):          2\r\n",
      "Vendor ID:             GenuineIntel\r\n",
      "CPU family:            6\r\n",
      "Model:                 79\r\n",
      "Model name:            Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz\r\n",
      "Stepping:              1\r\n",
      "CPU MHz:               2201.000\r\n",
      "CPU max MHz:           2201.0000\r\n",
      "CPU min MHz:           1200.0000\r\n",
      "BogoMIPS:              4391.41\r\n",
      "Virtualization:        VT-x\r\n",
      "L1d cache:             32K\r\n",
      "L1i cache:             32K\r\n",
      "L2 cache:              256K\r\n",
      "L3 cache:              51200K\r\n",
      "NUMA node0 CPU(s):     0-19,40-59\r\n",
      "NUMA node1 CPU(s):     20-39,60-79\r\n",
      "Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts\r\n"
     ]
    }
   ],
   "source": [
    "!lscpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MemTotal:       528278376 kB\r\n"
     ]
    }
   ],
   "source": [
    "!cat /proc/meminfo | grep MemTotal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPU 0: Tesla P100-SXM2-16GB (UUID: GPU-77864ed9-e817-3463-e5be-bf4c4c563a4c)\r\n",
      "GPU 1: Tesla P100-SXM2-16GB (UUID: GPU-e8ece262-f677-038f-5b40-6cc86f978487)\r\n",
      "GPU 2: Tesla P100-SXM2-16GB (UUID: GPU-7d996bf3-876b-1439-e447-ec8fb235fd98)\r\n",
      "GPU 3: Tesla P100-SXM2-16GB (UUID: GPU-151287c0-bb97-6f82-fedb-a88273ee5447)\r\n",
      "GPU 4: Tesla P100-SXM2-16GB (UUID: GPU-99c5cdb5-54b9-ccd2-eb51-a32850d37ebf)\r\n",
      "GPU 5: Tesla P100-SXM2-16GB (UUID: GPU-d2d22332-f462-061e-606a-3d6a5ea89848)\r\n",
      "GPU 6: Tesla P100-SXM2-16GB (UUID: GPU-5cc8b757-ee04-3bbb-5866-1f9f8c404a54)\r\n",
      "GPU 7: Tesla P100-SXM2-16GB (UUID: GPU-3016922d-1da7-46cd-17f3-cb6b3ab17bc3)\r\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi -L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def runXGBoost(param):\n",
    "    have_updater = \"updater\" in param.keys()\n",
    "    label = \"XGBoost \" \\\n",
    "        + (\"GPU hist\" if have_updater and param[\"updater\"]==\"grow_gpu_hist\" else \"GPU exact\" if have_updater and param[\"updater\"]==\"grow_gpu\" else \"CPU\") \\\n",
    "        + \" \" + (param[\"tree_method\"] if \"updater\" not in param.keys() else \"\")\n",
    "    print(label)\n",
    "    print(\"=====================\")\n",
    "    for k, v in param.items():\n",
    "        print(k, v)\n",
    "    print(\"=====================\")\n",
    "    \n",
    "    t_start = time.time()\n",
    "    dtrain = xgb.DMatrix(train.values, label = train_target.values, feature_names=[str(c) for c in cols])\n",
    "    tt = time.time() - t_start\n",
    "    print(\"Time to create DMatrix (sec): \", tt)\n",
    "    dmatrix_times.append(tt)\n",
    "    \n",
    "    t_start = time.time()\n",
    "    bst = xgb.train(param, dtrain, num_round)\n",
    "    tt = time.time() - t_start\n",
    "    print(\"Time to train (sec): \", tt)\n",
    "    train_times.append(tt)\n",
    "\n",
    "    t_start = time.time()\n",
    "    preds = bst.predict(dtrain)\n",
    "    tt = time.time() - t_start\n",
    "    print(\"Time to predict (sec): \", tt)\n",
    "    score_times.append(tt)\n",
    "\n",
    "    labels = dtrain.get_label()\n",
    "    auc = metrics.roc_auc_score(labels, preds)\n",
    "    print(\"Training AUC:\", auc)\n",
    "    valid_aucs.append(auc)\n",
    "    plot_labels.append(label)\n",
    "    \n",
    "    fs = bst.get_fscore()\n",
    "    \n",
    "    # Optional: Uncomment to show variable importance\n",
    "    #varimp = pd.DataFrame({'Importance': list(fs.values()), 'Feature': list(fs.keys())})\n",
    "    #varimp.sort_values(by = 'Importance', inplace = True, ascending = False)\n",
    "    #varimp.head(10).plot(label='importance',kind=\"barh\",x=\"Feature\",y=\"Importance\").invert_yaxis()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "valid_aucs = []\n",
    "dmatrix_times = []\n",
    "train_times = []\n",
    "score_times = []\n",
    "plot_labels = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost CPU exact\n",
      "=====================\n",
      "objective reg:logistic\n",
      "max_depth 10\n",
      "eta 0.02\n",
      "tree_method exact\n",
      "subsample 0.7\n",
      "colsample_bytree 0.9\n",
      "min_child_weight 5\n",
      "seed 12345\n",
      "=====================\n",
      "Time to create DMatrix (sec):  1.2587285041809082\n",
      "Time to train (sec):  160.4102680683136\n",
      "Time to predict (sec):  0.009805917739868164\n",
      "Training AUC: 0.814825032969\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"objective\":('reg:logistic' if num_class>1 else 'reg:linear')\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"eta\":learn_rate\n",
    "    , \"tree_method\":\"exact\"\n",
    "    , \"subsample\":0.7\n",
    "    , \"colsample_bytree\":0.9\n",
    "    , \"min_child_weight\":5\n",
    "    , \"seed\":12345\n",
    "}\n",
    "runXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost CPU approx\n",
      "=====================\n",
      "objective reg:logistic\n",
      "max_depth 10\n",
      "eta 0.02\n",
      "tree_method approx\n",
      "subsample 0.7\n",
      "colsample_bytree 0.9\n",
      "min_child_weight 5\n",
      "seed 12345\n",
      "=====================\n",
      "Time to create DMatrix (sec):  1.2151424884796143\n",
      "Time to train (sec):  96.42245173454285\n",
      "Time to predict (sec):  0.003930091857910156\n",
      "Training AUC: 0.812860299622\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"objective\":('reg:logistic' if num_class>1 else 'reg:linear')\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"eta\":learn_rate\n",
    "    , \"tree_method\":\"approx\"\n",
    "    , \"subsample\":0.7\n",
    "    , \"colsample_bytree\":0.9\n",
    "    , \"min_child_weight\":5\n",
    "    , \"seed\":12345\n",
    "}\n",
    "runXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost CPU hist\n",
      "=====================\n",
      "objective reg:logistic\n",
      "max_depth 10\n",
      "eta 0.02\n",
      "tree_method hist\n",
      "subsample 0.7\n",
      "colsample_bytree 0.9\n",
      "min_child_weight 5\n",
      "seed 12345\n",
      "=====================\n",
      "Time to create DMatrix (sec):  1.0287103652954102\n",
      "Time to train (sec):  73.14299082756042\n",
      "Time to predict (sec):  0.004008293151855469\n",
      "Training AUC: 0.813941675296\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"objective\":('reg:logistic' if num_class>1 else 'reg:linear')\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"eta\":learn_rate\n",
    "    , \"tree_method\":\"hist\"\n",
    "    , \"subsample\":0.7\n",
    "    , \"colsample_bytree\":0.9\n",
    "    , \"min_child_weight\":5\n",
    "    , \"seed\":12345\n",
    "}\n",
    "runXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost GPU exact \n",
      "=====================\n",
      "objective reg:logistic\n",
      "max_depth 10\n",
      "eta 0.02\n",
      "tree_method exact\n",
      "updater grow_gpu\n",
      "subsample 0.7\n",
      "colsample_bytree 0.9\n",
      "min_child_weight 5\n",
      "seed 12345\n",
      "=====================\n",
      "Time to create DMatrix (sec):  1.0280582904815674\n",
      "Time to train (sec):  73.03138780593872\n",
      "Time to predict (sec):  0.007105827331542969\n",
      "Training AUC: 0.814253361342\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"objective\":('reg:logistic' if num_class>1 else 'reg:linear')\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"eta\":learn_rate\n",
    "    , \"tree_method\":\"exact\"\n",
    "    , \"updater\":\"grow_gpu\"\n",
    "    , \"subsample\":0.7\n",
    "    , \"colsample_bytree\":0.9\n",
    "    , \"min_child_weight\":5\n",
    "    , \"seed\":12345\n",
    "}\n",
    "runXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost GPU hist \n",
      "=====================\n",
      "objective reg:logistic\n",
      "max_depth 10\n",
      "eta 0.02\n",
      "tree_method exact\n",
      "updater grow_gpu_hist\n",
      "n_gpus 1\n",
      "subsample 0.7\n",
      "colsample_bytree 0.9\n",
      "min_child_weight 5\n",
      "seed 12345\n",
      "=====================\n",
      "Time to create DMatrix (sec):  1.0200557708740234\n",
      "Time to train (sec):  5.28957724571228\n",
      "Time to predict (sec):  0.00944375991821289\n",
      "Training AUC: 0.813648642801\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"objective\":('reg:logistic' if num_class>1 else 'reg:linear')\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"eta\":learn_rate\n",
    "    , \"tree_method\":\"exact\"\n",
    "    , \"updater\":\"grow_gpu_hist\"\n",
    "    , \"n_gpus\":1\n",
    "    , \"subsample\":0.7\n",
    "    , \"colsample_bytree\":0.9\n",
    "    , \"min_child_weight\":5\n",
    "    , \"seed\":12345\n",
    "}\n",
    "runXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>algorithm</th>\n",
       "      <th>dmatrix time</th>\n",
       "      <th>scoring time</th>\n",
       "      <th>training AUC</th>\n",
       "      <th>training time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>XGBoost GPU hist</td>\n",
       "      <td>1.020056</td>\n",
       "      <td>0.009444</td>\n",
       "      <td>0.813649</td>\n",
       "      <td>5.289577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>XGBoost GPU exact</td>\n",
       "      <td>1.028058</td>\n",
       "      <td>0.007106</td>\n",
       "      <td>0.814253</td>\n",
       "      <td>73.031388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>XGBoost CPU hist</td>\n",
       "      <td>1.028710</td>\n",
       "      <td>0.004008</td>\n",
       "      <td>0.813942</td>\n",
       "      <td>73.142991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>XGBoost CPU approx</td>\n",
       "      <td>1.215142</td>\n",
       "      <td>0.003930</td>\n",
       "      <td>0.812860</td>\n",
       "      <td>96.422452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>XGBoost CPU exact</td>\n",
       "      <td>1.258729</td>\n",
       "      <td>0.009806</td>\n",
       "      <td>0.814825</td>\n",
       "      <td>160.410268</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            algorithm  dmatrix time  scoring time  training AUC  training time\n",
       "4   XGBoost GPU hist       1.020056      0.009444      0.813649       5.289577\n",
       "3  XGBoost GPU exact       1.028058      0.007106      0.814253      73.031388\n",
       "2    XGBoost CPU hist      1.028710      0.004008      0.813942      73.142991\n",
       "1  XGBoost CPU approx      1.215142      0.003930      0.812860      96.422452\n",
       "0   XGBoost CPU exact      1.258729      0.009806      0.814825     160.410268"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.DataFrame({'algorithm'  :plot_labels,\n",
    "                     'dmatrix time':dmatrix_times,\n",
    "                     'training time':train_times,\n",
    "                     'scoring time':score_times,\n",
    "                     'training AUC' :valid_aucs}).sort_values(by=\"training time\")\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2ab48bd1d320>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAD8CAYAAAAyun5JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm4FdWZ7/HvT1ARRWRIlCvEgzQxCugJkxhwaKM4Ns7B\nXHJbTIvRq7aalg4ZGodORxK8atCojbZKGzUoalqh4+MQnFFzDoKAOBFphMcoTqhxAn3vH7UObrbn\nnL0Zdu3t8fd5nvNQtWpV1VsLDu9eq1bVVkRgZmZmlbdZtQMwMzP7snDSNTMzy4mTrpmZWU6cdM3M\nzHLipGtmZpYTJ10zM7OcOOmamZnlxEnXzMwsJ066ZmZmOWlf7QCstnTv3j3q6uqqHYaZ2RdKY2Pj\n6xHxlVL1nHRtHXV1dTQ0NFQ7DDOzLxRJ/1NOPQ8vm5mZ5cRJ18zMLCdOumZmZjlx0jUzM8uJk66Z\nmVlOnHTNzMxy4qRrZmaWEyddMzOznDjpmpmZ5cRvpLJ1LFixiroJs6odhplZrpZOOiyX87ina2Zm\nlhMnXTMzs5w46ZqZmeXESdfMzCwnTrpmZmY5cdI1MzPLSe5JV1IvSS9J6prWu6T1urTeV9JMSUsk\nNUqaLWmftG2spJWS5klaJGmGpI6bMLZ6SYe2sn2opIckPSfpKUnXSOpYFNczksal+udJOqfoGEsl\ndd9UMZej1HWZmVk+ck+6EfEycCUwKRVNAqZGxFJJHYBZab1PRAwCzgB2LjjE9Iioj4h+wMfA6E0Y\nXj3QbHKStD1wK/CjiNglIr4J3A10KowL2A/4RapfK1q8LjMzy0+1hpcvAYZJOgsYAVyUyscAcyLi\nzqaKEbEwIq4vPoCk9sDWwFtpvU7SHyU9Lel+SV8rUX6cpIWS5qfe6xbABcDo1GMtTuanAdMiYk5B\nbDMi4tXCShHxGrAE2Gl9GkTSSElzJM2VdKukbSR1Tr3qXVKdmwt60VdKakg9/vMLjjNE0mPpup6U\n1LnEdZmZWU6qknQjYjUwniz5npXWAfoBc0vsPlrSPGAF0BW4K5VfRpYUdwduBKaUKJ8IHBQRewCj\nIuLjVNbUk55edN7+QGOpa5O0M1nP/MVSdQv26Q78DDggIgYCDcAPI2IVcDpwvaTjgS4RcXXa7acR\nMRjYHdhX0u7pg8N04Mx0XQcAfy1xXWZmlpNqTqQ6BHiFLJk1S9IdqTd6e0Fx0zDuDsACsuQNsBdw\nU1q+gawH3Vr5o2TJbBzQbiOvBT77MHAz8IOIeBOIFuoWlw8DdgMeTcc4gdRTjoh7ya7zN8BJBft8\nR9Jc4CmyDyu7AbsAr0TEn9K+70TEmlKBSzo59ZobPnl/VXlXa2Zm660qSVdSPXAgWbI5W1KPtGkR\nMLCpXkQcBYwl69GuIyKCrJe7z4bEEBGnkPUuewGNkrqV2GURMKiV7U09yT0j4o5U9gbQpaheJ+Dt\nojIB96b96yNit4j4BwBJmwG7Au83HUtSb+Ac4NupBz8L6FAi/hZFxNSIGBwRg9t17LyhhzEzsxKq\nMXtZZBOpzoqIZcBkPrunexMwXNKogl1am508guz+KcBjwPFpeQzwcGvlkvpExBMRMRFYSZZ83+Wz\niVHFLgdOkLRnwbUcXWLC1EPAKEmdmuoD8yPik6J6j5Nd99+keltL+nradjawGPjfwHWSNge2JRs2\nXpXOf0iq+xzQQ9KQdJxO6d53a9dlZmY5qca3DI0DlqVhU4ArgBMl7RsRD0o6HLhY0qXAq2QJ4+cF\n+4+WNILsA8Nysp4wZLOcr5M0niyJnliifLKkvmS9zPuB+cAyYEIa4r2w8P5nRLya7qteJOmrwKdk\nSfXuli40Ip6WdDnwiKQAXmPdIeKmeisljQVulrRlKv5Z+oByEjA0It6V9BDws4g4V9JTwLPAy2RD\n5UTEx2mi1GWStgI+ILuvO7ul6zIzs/woG6U1y2zZo2/0OOHSaodhZparjf1qP0mNaXJrq/xGKjMz\ns5w46ZqZmeXESdfMzCwnTrpmZmY5cdI1MzPLiZOumZlZTqrxnK7VsAE7dqZhI6fOm5lZ89zTNTMz\ny4mTrpmZWU6cdM3MzHLipGtmZpYTJ10zM7OcOOmamZnlxEnXzMwsJ066ZmZmOXHSNTMzy4mTrpmZ\nWU6cdM3MzHLipGtmZpYTJ10zM7OcOOmamZnlxEnXzMwsJ066ZmZmOXHSNTMzy4mTrpmZWU6cdM3M\nzHLipGtmZpYTJ10zM7OctK92AFZbFqxYRd2EWdUOw6wqlk46rNohWBvnnq6ZmVlOnHTNzMxy4qRr\nZmaWEyddMzOznDjpmpmZ5cRJ18zMLCcVSbqSekl6SVLXtN4lrdel9b6SZkpaIqlR0mxJ+6RtYyWt\nlDRP0iJJMyR13ISx1Us6tJXtQyU9JOk5SU9JukZSx6K4npE0LtU/T9I5RcdYKqn7porZzMzahook\n3Yh4GbgSmJSKJgFTI2KppA7ArLTeJyIGAWcAOxccYnpE1EdEP+BjYPQmDK8eaDbpStoeuBX4UUTs\nEhHfBO4GOhXGBewH/CLVrypJftbazOwLopLDy5cAwySdBYwALkrlY4A5EXFnU8WIWBgR1xcfICWU\nrYG30nqdpD9KelrS/ZK+VqL8OEkLJc1PvdctgAuA0anHWpzMTwOmRcScgthmRMSrhZUi4jVgCbDT\n+jSIpCslNaQe/PkF5Usl/UrSAklPSvqbVH69pKvSPs9LOjyVj5V0p6Q/AvcrMzld64Km65J0VGoP\nSeqRjrHD+sRsZmabTsWSbkSsBsaTJd+z0jpAP2Buid1HS5oHrAC6Anel8svIkuLuwI3AlBLlE4GD\nImIPYFREfJzKmnrS04vO2x9oLHVtknYm65m/WKpukZ9GxGBgd2BfSbsXbFsVEQOAy4FLC8rrgKHA\nYcBVaaQAYCBwbETsCxxN1oPfAzgAmCypR0TcAbxC9mHiauDciPjLesZsZmabSKUnUh1C9p9+/5Yq\nSLoj9dBuLyhuGsbdAVhAlrwB9gJuSss3kPWgWyt/FLg+3X9tt5HXAp99GLgZ+EFEvAlEC3WbK/+O\npLnAU2QfPnYr2HZzwZ97FZTfEhGfRsQLwJ+Bb6Tye9P5IbvemyPik9QrfxAYkradAfwY+CgibqYZ\nkk5OvemGT95f1cLlmJnZxqpY0pVUDxwIDAPOltQjbVpE1ksDICKOAsaS9WjXERFB1svdZ0NiiIhT\ngJ8BvYBGSd1K7LIIGNTK9qYe8p6pFwnwBtClqF4n4O3CAkm9gXOAb6ce+SygQ0GVKGO5cP2vrcRZ\nqCfwKbC9pGb/viNiakQMjojB7Tp2LvOwZma2vio1e1lkE6nOiohlwGQ+u6d7EzBc0qiCXVqbnTyC\n7P4pwGPA8Wl5DPBwa+WS+kTEExExEVhJlnzf5bOJUcUuB06QtGfBtRxdYsLUQ8AoSZ2a6gPzI+KT\nonrbkiXKVel4hxRtH13w55yC8uMkbSapD9mQ9nPNxPAwWS+8naSvkH1IeTLdE78W+C6wGPhhK9dh\nZmYVVqmZr+OAZRFxb1q/AjhR0r4R8WCaEHSxpEuBV8kS4c8L9h8taQTZh4LlZD1hyIZKr5M0niyJ\nnliifLKkvoCA+4H5wDJgQhomvrDwvm5EvCrpeOAiSV8l6yE+RDaDuVkR8bSky4FHJAXwGnBSM/Xm\nS3oKeBZ4mWzou1AXSU8DH5ElySbLgCfJkvYpEfFh9plmHXeQDUnPJ+sJ/3NE/EXSRODhiHhE0nzg\nT5JmRcTilq7HzMwqR9kIrlWTpKXA4Ih4vaj8emBmRMzIK5Yte/SNHidcWrqiWRvkr/azDSWpMU2U\nbZXfSGVmZpYTv1ihBkREXQvlY/ONxMzMKsk9XTMzs5w46ZqZmeXESdfMzCwnvqdr6xiwY2caPIPT\nzKwi3NM1MzPLiZOumZlZTpx0zczMcuKka2ZmlhMnXTMzs5w46ZqZmeXESdfMzCwnTrpmZmY5cdI1\nMzPLiZOumZlZTpx0zczMclLWu5cltQMOA+oK94mIiysTlpmZWdtT7hce3AV8CCwAPq1cOGZmZm1X\nuUm3Z0TsXtFIzMzM2rhy7+n+QdLIikZiZmbWxpXb030cuEPSZsBqQEBExLYVi8zMzKyNKTfpXgzs\nBSyIiKhgPGZmZm1WucPLLwMLnXDNzMw2XLk93T8DD0j6A/BRU6EfGTIzMytfuUn3pfSzRfoxMzOz\n9VRW0o2I8ysdiJmZWVtX7hupvg6cw+ffSLV/ZcKyalmwYhV1E2ZVOwz7Als66bBqh2BWs8odXr4V\nuAq4BvikcuGYmZm1XeUm3TURcWVFIzEzM2vjWk26krqmxbsk/V/gDtadvfxmBWMzMzNrU0r1dBuB\nIHsDFcD4gm0B7FyJoMzMzNqiVpNuRPQGkNQhIj4s3CapQyUDMzMza2vKfSPVY2WWmZmZWQtaTbqS\ndpA0CNhK0jclDUw/+wEdKxWUpF6SXmq6pyypS1qvS+t9Jc2UtERSo6TZkvZJ28ZKWilpnqRFkmZI\n2mSxSqqXdGgr24dKekjSc5KeknSNpI5FcT0jaVyqf56kc4qOsVRS92aO/V4L5zxF0t+3EtN+kr5V\n/lWamVkllLqnexAwFuhJ9qUHTd4FflKhmIiIlyVdCUwCTk5/To2IpWlYexZwTkTcCSCpPzAYeCgd\nYnpEnJ623QSMBq7bROHVp3P9d/EGSduTPV51fETMSWXHAp0K45L0VWCRpDs3RUARcVWJKvsB7+HR\nCTOzqip1T3caME3SMRFxW04xNbkEaJR0FjACOD2VjwHmNCXcFOdCYGHxASS1B7YG3krrdcC1QHdg\nJXBiRCxrpfw44FyyZ5NXAQcAF5D1/EcAF0bE9IJTngZMa0q4KbYZ6dwUlL0maQmw0/o2iqR/Aw4H\nPgCOiIhXJZ0HvBcRF0n6R+AUYA3wDDAhrX8i6XvAGRHx8Pqe18zMNl6pR4a+FxG/Beok/bB4eyW/\n8CAiVksaD9wNjIyI1WlTP2Buid1Hp6TYA3geuCuVX0aWFKdJ+j4wBTiylfKJwEERsULSdhHxsaSJ\nwOCmnnSR/sC0UtcmaWeymd8vlqpbZGvg8Yj4qaRfAeOAnxfVmQD0joiPUsxvS7qKlJTX83xmZrYJ\nlZpItXX6cxuyIdLin0o7BHiFLJk1S9IdkhZKur2geHpE1AM7AAv47FGnvYCb0vINZD3o1sofBa5P\n91/bbeS1QPZhYB5wM/CD9JxzS1+X2Fz5x8DMtNxI9lrOYk8DN6Ze7ZpygpJ0sqQGSQ2fvL+qnF3M\nzGwDlBpe/ndJ7YB3IuKSnGICsglLwIHAMOARSb+LiFeARcA+BTEeJWkw8LleXESEpLuAM8juC6+X\niDhF0p7AYWRD3YNK7LIIGAT8VwvbpzfTQ36DrEdeqBPwdjP7ry74TuNPaP7v7zCy9vk74KeSBpSI\nmYiYCkwF2LJHX39nsplZhZR8ZCgiPgG+m0Msaym7AXolcFZELAMm81lSvQkYLmlUwS6tzU4eASxJ\ny48Bx6flMcDDrZVL6hMRT0TERLJ7vb3IJpG11Mu/HDghJeqmazk6TbBqyUPAKEmdmuoD81O7rxdJ\nmwG9ImI28COgM9koRWsxm5lZTsp99/Kjki4HpgN/bSqMiFL3VjfUOGBZRNyb1q8ATpS0b0Q8KOlw\n4GJJlwKvkiWVwnubTfd0NwOWk83AhqzHe126V7wSOLFE+WRJfcneyHU/MB9YBkxIw8TrTKRKk5qO\nBy5KM5Q/JUuqd7d0oRHxdGrbRyQF8Bpw0vo0VoF2wG8ldU4xT0n3dO8CZkg6Ak+kMjOrGn02WtlK\nJWl2M8Xhr/Zre7bs0Td6nHBptcOwLzB/tZ99GUlqjIjBpeqV+yX2f7vxIZmZmX25lfUaSEmdJV3c\nNMNV0v9LQ5hmZmZWpnLfvXwt2X3T76Sfd9h0b3gyMzP7Uih3IlWfiDimYP38NJHIzMzMylRuT/eD\nNBsYAEnDyV5DaGZmZmUqt6d7Ktk7mJseRXmTzx7DMTMzszKU9cjQ2srStgAR8U7FIrKqGjx4cDQ0\nNFQ7DDOzL5RN+shQ8ZcdpG/MWQU0RoTv7ZqZmZWh3Hu6g8m+Hm7H9PMD4GDgakn/XKHYzMzM2pRy\n7+n2BAZGxHsAks4l+yL5fci+7eZXlQnPzMys7Si3p/tV4KOC9dXA9hHxQVG5mZmZtaDcnu6NwBOS\nmr6y7u+AmyRtDTxTkcjMzMzamHLfvfyvkv4ADE9Fp0RE0xTXMRWJzMzMrI1pNelK6lqw+uf0s3Zb\nRLxZqcDMzMzamlI93UYgyF6IQVomrQewc4XiMjMza3NaTboR0btpOfV6+wIdKh2UmZlZW1TuyzFO\nAs4ke3RoHjAMeAz4duVCMzMza1vKfWToTGAI8D/pC+2/SfZGKjMzMytTuUn3w4j4EEDSlhHxLLBL\n5cIyMzNre8p9Tne5pO2A3wP3SnoL+J/KhWVmZtb2lPuc7lFp8TxJs4HOwN0Vi8rMzKwNKrenu1ZE\nPFiJQMzMzNq6cu/pmpmZ2UZy0jUzM8uJk66ZmVlOnHTNzMxy4qRrZmaWEyddMzOznKz3I0PWti1Y\nsYq6CbOqHYZ9gS2ddFi1QzCrWe7pmpmZ5cRJ18zMLCdOumZmZjlx0jUzM8uJk66ZmVlOnHTNzMxy\nUvGkK6mXpJckdU3rXdJ6XVrvK2mmpCWSGiXNlrRP2jZW0kpJ8yQtkjRDUsdNGFu9pENb2T5U0gOS\nXpA0V9IsSQPStvMkrUixLZQ0KpVfL+nYouO8t6liLpek/SR9K+/zmplZyyqedCPiZeBKYFIqmgRM\njYilkjoAs9J6n4gYBJwB7FxwiOkRUR8R/YCPgdGbMLx6oNmkK2l74BbgJxHRNyIGAhcCfQqqXRIR\n9cBxwLWSamnkYD/ASdfMrIbklSQuAYZJOgsYAVyUyscAcyLizqaKEbEwIq4vPoCk9sDWwFtpvU7S\nHyU9Lel+SV8rUX5c6pHOl/SQpC2AC4DRqbdanMxPB6ZFxGMFsT0SEb8vji0iFgNrgO7r0yiSvifp\nyXT+f5fUTtJOqWfdXdJmkh6WNDLV/30aDVgk6eSC4xyceuLz0zXXAacAZ6dj770+cZmZWWXk8kaq\niFgtaTxwNzAyIlanTf2AuSV2Hy1pBNADeB64K5VfRpYUp0n6PjAFOLKV8onAQRGxQtJ2EfGxpInA\n4Ig4vZnz9gOmlXN9kvYEPgVWllM/7bMrWa99eGqfK4AxEfGfkn5JNjrwJPBMRNyTdvt+RLwpaSvg\nT5JuI/vgdDWwT0S8JKlrqnMV8F5EXPT5s5uZWTXkORx6CPAK0L+lCpLuSL3R2wuKp6ch3B2ABcD4\nVL4XcFNavoGsB91a+aPA9ZLGAe3WN3hJT0haLOnXBcVnS5pH1nMfHREBRDO7N1f2bWAQWfKcl9Z3\nBoiIa4BtyXqr5xTs84+S5gOPA72AvsAw4KGIeCnt++YGXNvJkhokNXzy/qr13d3MzMqUS9KVVA8c\nSJYgzpbUI21aBAxsqhcRRwFjga7Fx0gJ7S5gnw2JISJOAX5GlqwaJXUrsUtxbHsC/wJ0LqhzSbrf\nvHdEPJzK3gC6NFVIE8heb+b4IuuR16efXSLivLRPR6BnqrdNKtsPOADYKyL2AJ4COpS88DJExNSI\nGBwRg9t17Fx6BzMz2yB5zF4W2VDpWRGxDJjMZ/d0bwKGN838TVqbnTwCWJKWHwOOT8tjgIdbK5fU\nJyKeiIiJZMPAvYB3gU4tnOs3wNiiGcDlzJx+gGxIfIu0PhaY3Uy9+4FjJX01xddV0k5p2y+BG8mG\nxK9OZZ2BtyLifUnfIPsAA1mvdx9JvZuOk8pbuzYzM6uCPO7pjgOWRcS9af0K4ERJ+0bEg5IOBy6W\ndCnwKlmy+HnB/k33dDcDlpMlMchmOV+X7hWvBE4sUT5ZUl+yHub9wHxgGTAhDe9eGBHTm04aEX9J\nk6t+KWlH4DWyHusFrV1sRMyUNIisN/0J2YeEU5qp94yknwH3pFnPq4HT0iSoIWT3ej+RdIykE8k+\noJwiaTHwHFmyJSJWpklVt6fjvEY2qnAXMEPSEcAZBT1xMzOrEmWjtmaZLXv0jR4nXFrtMOwLzF/t\nZ19GkhojYnCperX0XKmZmVmb5qRrZmaWEyddMzOznDjpmpmZ5cRJ18zMLCe5vAbSvjgG7NiZBs8+\nNTOrCPd0zczMcuKka2ZmlhMnXTMzs5w46ZqZmeXESdfMzCwnTrpmZmY5cdI1MzPLiZOumZlZTpx0\nzczMcuKka2ZmlhMnXTMzs5w46ZqZmeXESdfMzCwnTrpmZmY5cdI1MzPLiZOumZlZTpx0zczMcuKk\na2ZmlhMnXTMzs5w46ZqZmeXESdfMzCwn7asdgNWWBStWUTdhVsl6SycdlkM0ZmZti3u6ZmZmOXHS\nNTMzy4mTrpmZWU6cdM3MzHLipGtmZpYTJ10zM7Oc1ETSldRL0kuSuqb1Lmm9Lq33lTRT0hJJjZJm\nS9onbRsraaWkeZIWSZohqeMmjK1e0qGtbB8q6QFJL0iaK2mWpAFp23mSVqTYFkoalcqvl3Rs0XHe\na+bYdZIWtnDeCyQd0EpcR0rardzrNDOzyquJpBsRLwNXApNS0SRgakQsldQBmJXW+0TEIOAMYOeC\nQ0yPiPqI6Ad8DIzehOHVA80mXUnbA7cAP4mIvhExELgQ6FNQ7ZKIqAeOA66VtEnaPCImRsR9rVQ5\nEnDSNTOrITWRdJNLgGGSzgJGABel8jHAnIi4s6liRCyMiOuLDyCpPbA18FZar5P0R0lPS7pf0tdK\nlB+XeqTzJT0kaQvgAmB06q0WJ/PTgWkR8VhBbI9ExO+LY4uIxcAaoPt6tks7SVenXvw9krZKsa7t\nLUuaJOmZdD0XSfoWMAqYnOLu09oJzMwsHzXzRqqIWC1pPHA3MDIiVqdN/YC5JXYfLWkE0AN4Hrgr\nlV9GlhSnSfo+MIWsB9hS+UTgoIhYIWm7iPhY0kRgcESc3sx5+wHTyrk+SXsCnwIry6lfoC/w3YgY\nJ+kW4BjgtwXH7QYcBXwjIiLF/bakO4GZETFjPc9nZjVk9erVLF++nA8//LDaoRjQoUMHevbsyeab\nb75B+9dM0k0OAV4B+gP3NldB0h1kiej5iDg6FU+PiNMlCfgNMJ5siHovoKnODcCv0nJL5Y8C16fk\ndvv6Bi/pCWBb4J6IODMVny3pe8C7wOiUGKOZ3ZsrA3gpIual5Uagrmj7KuBD4D8kzQRmbkDcJwMn\nA7Tb9ivru7uZVdDy5cvp1KkTdXV1ZP/FWbVEBG+88QbLly+nd+/eG3SMmhlellQPHAgMI0tUPdKm\nRcDApnoRcRQwFuhafIyICLJe7j4bEkNEnAL8DOgFNKZeZGuKY9sT+Begc0GdS9L95r0j4uFU9gbQ\npalCmkD2egvn+Khg+ROKPihFxBpgKDADOJxspGC9RMTUiBgcEYPbdexcegczy82HH35It27dnHBr\ngCS6deu2UaMONZF0Uw/1SuCsiFgGTOaze7o3AcObZv4mrc1OHgEsScuPAcen5THAw62VS+oTEU9E\nxESyYeBeZD3UTi2c6zfA2HQPtZzYmjxANiS+RVofC8wuY7/PkbQN0Dki/hs4G9gjbWotbjP7AnHC\nrR0b+3dRE0kXGAcsi4imIeUrgF0l7RsRH5D14E6R9GdJc8h6oz8v2L9potPTwDeBf03lZwAnpvL/\nA5xZonyypAXpMZ3HgPlkyXC35iZSRcRfyGZKXyjpRUmPAccCl7d2sRExkyzRN0qaBwwHflRmWxXr\nBMxM1/II8MNU/jtgvKSnPJHKzDbU22+/zRVXXLFB+x566KG8/fbbrdaZOHEi993X2oMY5fvFL36x\nzvq3vvWtFmpWj7IRWbPMlj36Ro8TLi1Zz1/tZ5aPxYsXs+uuu65dL+erN9dHqd/lpUuXcvjhh7Nw\n4edfGbBmzRrat6+dqUHbbLMN7733uVcebHLFfycAkhojYnCpfWulp2tmZjVowoQJLFmyhPr6esaP\nH88DDzzA3nvvzahRo9htt+xVAEceeSSDBg2iX79+TJ06de2+dXV1vP766yxdupRdd92VcePG0a9f\nP0aOHMkHH3wAwNixY5kxY8ba+ueeey4DBw5kwIABPPvsswCsXLmSAw88kH79+nHSSSex00478frr\nr38uzg8++ID6+nrGjBkDZEkY4IEHHmDffffliCOOYOedd2bChAnceOONDB06lAEDBrBkyZK15znm\nmGMYMmQIQ4YM4dFHH93k7emka2ZmLZo0aRJ9+vRh3rx5TJ48GYC5c+fy61//mueffx6Aa6+9lsbG\nRhoaGpgyZQpvvPHG547zwgsvcNppp7Fo0SK22247brvttmbP1717d+bOncupp57KRRdlU3vOP/98\n9t9/fxYtWsSxxx7LsmXLmo1zq622Yt68edx4442f2z5//nyuuuoqFi9ezA033MDzzz/Pk08+yUkn\nncRll10GwJlnnsnZZ5/Nn/70J2677TZOOumkDWu0VtTOuICZmX0hDB06dJ1HZqZMmcIdd9wBwMsv\nv8wLL7xAt27rPvzRu3dv6uvrARg0aBBLly5t9thHH3302jq33549ufnII4+sPf7BBx9Mly5dmt23\nNUOGDKFHj+yhmD59+jBy5EgABgwYwOzZ2TzW++67j2eeeWbtPu+88w7vvffe2h7zpuCka2Zm62Xr\nrbdeu/zAAw9w3333MWfOHDp27Mh+++3X7CM1W2655drldu3arR1ebqleu3btWLNmzSaLufD8m222\n2dr1zTbbbO15Pv30Ux5//HE6dOiwyc5bzMPLZmbWok6dOvHuu++2uH3VqlV06dKFjh078uyzz/L4\n449v8hiGDx/OLbfcAsA999zDW2+91Wy9zTffnNWrVze7rRwjR45cO9QMMG/evFZqbxgnXTMza1G3\nbt0YPnw4/fv3Z/z48Z/bfvDBB7NmzRp23XVXJkyYwLBhwzZ5DOeeey733HMP/fv359Zbb2WHHXag\nU6fPv4bg5JNPZvfdd187kWp9TZkyhYaGBnbffXd22203rrrqqo0N/XP8yJCtY/DgwdHQ0FDtMMws\nae7xlC8axqFTAAAF+klEQVSbjz76iHbt2tG+fXvmzJnDqaeeWpFeaLk25pEh39M1M7OatmzZMr7z\nne/w6aefssUWW3D11VdXO6QN5qRrZmY1rW/fvjz11FPVDmOT8D1dMzOznDjpmpnVOM+9qR0b+3fh\npGtmVsM6dOjAG2+84cRbA5q+T3djnuP1PV0zsxrWs2dPli9fzsqVK6sdipF9COrZs+cG7++ka2ZW\nwzbffPN1XrloX2weXjYzM8uJk66ZmVlOnHTNzMxy4tdA2jokvQs8V+04WtEdeL1kreqo5djA8W2s\nWo6vlmODL0d8O0XEV0pV8kQqK/ZcOe8PrRZJDbUaXy3HBo5vY9VyfLUcGzi+Qh5eNjMzy4mTrpmZ\nWU6cdK3Y1GoHUEItx1fLsYHj21i1HF8txwaOby1PpDIzM8uJe7pmZmY5cdI1ACQdLOk5SS9KmlAD\n8fSSNFvSM5IWSTozlXeVdK+kF9KfXaocZztJT0mamdZ7S3oiteN0SVtUMbbtJM2Q9KykxZL2qpX2\nk3R2+ntdKOlmSR2q2XaSrpX0mqSFBWXNtpUyU1KcT0saWKX4Jqe/26cl3SFpu4JtP07xPSfpoGrE\nV7DtnySFpO5pvSbaL5WfkdpwkaRfFZRXrP2cdA1J7YDfAIcAuwHflbRbdaNiDfBPEbEbMAw4LcU0\nAbg/IvoC96f1ajoTWFyw/kvgkoj4G+At4B+qElXm18DdEfENYA+yOKvefpJ2BP4RGBwR/YF2wPFU\nt+2uBw4uKmuprQ4B+qafk4ErqxTfvUD/iNgdeB74MUD6PTke6Jf2uSL9jucdH5J6ASOBZQXFNdF+\nkv4WOALYIyL6ARel8oq2n5OuAQwFXoyIP0fEx8DvyP4xVk1EvBIRc9Pyu2QJY8cU17RUbRpwZHUi\nBEk9gcOAa9K6gP2BGalK1eKT1BnYB/gPgIj4OCLepnbarz2wlaT2QEfgFarYdhHxEPBmUXFLbXUE\n8J+ReRzYTlKPvOOLiHsiYk1afRxo+uqbI4DfRcRHEfES8CLZ73iu8SWXAP8MFE4eqon2A04FJkXE\nR6nOawXxVaz9nHQNsmT2csH68lRWEyTVAd8EngC2j4hX0qa/ANtXKSyAS8n+Q/k0rXcD3i74j7Ca\n7dgbWAlcl4a/r5G0NTXQfhGxgqxXsYws2a4CGqmdtmvSUlvV4u/L94E/pOWaiE/SEcCKiJhftKkm\n4gO+Duydbmk8KGlIKq9ofE66VtMkbQPcBpwVEe8Ubots6n1Vpt9LOhx4LSIaq3H+MrQHBgJXRsQ3\ngb9SNJRcrfZL90aPIPtg8L+ArWlmaLKWVPPfWimSfkp2O+bGasfSRFJH4CfAxGrH0or2QFey21fj\ngVvSaFVFOekawAqgV8F6z1RWVZI2J0u4N0bE7an41aahqPTnay3tX2HDgVGSlpINx+9Pdg91uzRk\nCtVtx+XA8oh4Iq3PIEvCtdB+BwAvRcTKiFgN3E7WnrXSdk1aaqua+X2RNBY4HBgTnz3/WQvx9SH7\nUDU//Y70BOZK2qFG4oPsd+T2NMz9JNmIVfdKx+ekawB/Avqm2aNbkE0iuLOaAaVPnP8BLI6Iiws2\n3QmckJZPAP4r79gAIuLHEdEzIurI2uuPETEGmA0cWwPx/QV4WdIuqejbwDPURvstA4ZJ6pj+npti\nq4m2K9BSW90J/H2ahTsMWFUwDJ0bSQeT3d4YFRHvF2y6Ezhe0paSepNNWHoyz9giYkFEfDUi6tLv\nyHJgYPp3WRPtB/we+FsASV8HtiD70oPKtl9E+Mc/AIeSzYBcAvy0BuIZQTac9zQwL/0cSnbf9H7g\nBeA+oGsNxLofMDMt75x+QV8EbgW2rGJc9UBDasPfA11qpf2A84FngYXADcCW1Ww74Gay+8uryRLE\nP7TUVoDIZvsvARaQzcKuRnwvkt17bPr9uKqg/k9TfM8Bh1QjvqLtS4HuNdZ+WwC/Tf8G5wL759F+\nfiOVmZlZTjy8bGZmlhMnXTMzs5w46ZqZmeXESdfMzCwnTrpmZmY5cdI1MzPLiZOumZlZTpx0zczM\ncvL/ARbuOokXHsYwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2ab52e046278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAEICAYAAAAazi7yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu4XVV97vHvSyhgMKRcVHIgsiGNF4IQIXKRCPSIIqBR\nFAwWW0IVTatYsFCpUkTKKVEoUAWh4NFQDmA0FUuI8hQBCXdMIBcCYkViIIdLRAwgyiW8/WOO3aws\n995r7ZA192Lzfp5nPZlzzDHn/M2VwG/+xhxrLdkmIiIiOm+DoQ4gIiLilSJJNyIioiZJuhERETVJ\n0o2IiKhJkm5ERERNknQjIiJqkqQbER0h6QJJ/7C++0a8nCmf042IZpKWAR+3/aOhjmV9kPRq4BHg\nRtsHNm0zMN72zxvaTgH+xPZHy/pmwKnAB4EtgEeBOcBptn9Vy0XEsJBKNyIGTdKGQx3DIH0IeBZ4\nl6StB7OjpI2Aa4EJwHuAzYC9gMeB3ddznDHMJelGxFokXQK8Hpgj6WlJfyepR5IlfUzScuC60ve7\nkh6RtErSPEkTGo4zU9JpZXk/SQ9J+ltJj0l6WNJR69h3S0lzJD0p6SeSTpN0U4vLOhK4AFgMfHSQ\nb8lflPfjENv32H7R9mO2/9H2DwZ5rHiFS9KNiLXY/nNgOfA+26+2/ZWGzfsCbwYOKOs/BMYDrwXu\nBC4d4NBbA6OBbYCPAedJ2nwd+p4H/Lb0ObK8+iVpO2C/EtulVEl0MPYHrrb99CD3i/gDSboRMRin\n2P6t7d8B2P6m7adsPwucAuwiaXQ/+z4PnGr7+VIhPg28cTB9JY2gGir+ou1nbN8DXNwi5j8HFpe+\n3wYmSHpr+5fMlsDDg+gf0a8k3YgYjAd7FySNkDRD0v2SngSWlU1b9bPv47ZfaFh/Bnj1IPu+Btiw\nMY6m5b78BaUCt70CuIG1q+PVwB817fNHVIkfqme3Y1qcI6ItSboR0Zf+PtbQ2P5nwPuphl9HAz2l\nXZ0Li5XAC8C2DW1j++ss6e1Uw99/X549PwLsAfxZw2Sw5ayJvdf2wC/L8o+AAyRt+tLDj1e6JN2I\n6MujwA4t+oyimhH8ODAS+KdOB2V7NfA94BRJIyW9iYGf0R4JXAPsCEwsr52AVwG9Hx2aBZwkaVtJ\nG0jaH3gfMLtsv4Sqmv53SW8qfbaU9HlJB63va4zhLUk3IvpyOlUi+o2k4/vp829U1eAK4B7gtppi\n+zRVZf0IVUK8nCr5r0XSJsCHga/ZfqTh9UDZr3eI+VTgFuAm4AngK8ARtu8GKM+r9wd+SpXAnwTu\noBpGv71TFxnDU74cIyJe1iR9Gdja9oCzmCO6QSrdiHhZKUO8O6uyO9VHiq4Y6rgi2vFy+1aZiIhR\nVEPK/4vq2fM/A/8xpBFFtCnDyxERETXJ8HJERERNMrwca9lqq63c09Mz1GFERLysLFiw4Fe2X9Oq\nX5JurKWnp4f58+cPdRgRES8rkn7ZuleGlyMiImqTpBsREVGTJN2IiIiaJOlGRETUJEk3IiKiJkm6\nERERNUnSjYiIqEmSbkRERE2SdCMiImqSb6SKtSxZsYqeE+cOdRgREbVaNuPgWs6TSjciIqImSboR\nERE1SdKNiIioSZJuRERETZJ0IyIiapKkGxERUZMk3YiIiJrUnnQljZX0gKQtyvrmZb2nrI+XdJWk\n+yUtkHS9pH3KtmmSVkpaKGmppNmSRq7H2CZKOmiA7btLmifpPkl3SfqGpJFNcd0j6ejS/xRJxzcd\nY5mkrdZXzO1odV0REVGP2pOu7QeB84EZpWkGcKHtZZI2AeaW9XG2dwOOAXZoOMQs2xNtTwCeA6au\nx/AmAn0mJ0mvA74LfM72G22/FbgaGNUYF7Af8E+lf7fo97oiIqI+QzW8fDawp6RjgcnAmaX9COBW\n21f2drR9t+2ZzQeQtCGwKfBEWe+RdJ2kxZKulfT6Fu2HSbpb0qJSvW4EnApMLRVrczL/FHCx7Vsb\nYptt+9HGTrYfA+4HthvMGyLp3ZJulXSnpO9KerWk0aWqfmPpc3lDFX2+pPml4v9Sw3HeJumWcl13\nSBrd4roiIqImQ5J0bT8PnECVfI8t6wATgDtb7D5V0kJgBbAFMKe0f40qKe4MXAp8tUX7ycABtncB\npth+rrT1VtKzms67E7Cg1bVJ2oGqMv95q74N+2wFnATsb3tXYD7wWdurgE8DMyUdDmxu+6Ky2xds\nTwJ2BvaVtHO5cZgF/E25rv2B37a4LiR9oiTw+aufWdVu2BERMUhDOZHqQOBhqmTWJ0lXlGr0ew3N\nvcO4WwNLqJI3wF7AZWX5EqoKeqD2m6mS2dHAiJd4LbDmZuBy4JO2fw24n77N7XsCOwI3l2McSamU\nbV9DdZ3nAR9v2OfDku4E7qK6WdkReCPwsO2flH2ftP1Cq8BtX2h7ku1JI0aObu9qIyJi0IYk6Uqa\nCLyLKtkcJ2lM2bQU2LW3n+1DgGlUFe1abJuqyt1nXWKwPZ2quhwLLJC0ZYtdlgK7DbC9t5Lcw/YV\npe1xYPOmfqOA3zS1Cbim7D/R9o62PwYgaQPgzcAzvceStD1wPPDOUsHPBTZpEX9ERAyxoZi9LKqJ\nVMfaXg6cwZpnupcBe0ua0rDLQLOTJ1M9PwW4BTi8LB8B3DhQu6Rxtm+3fTKwkir5PsWaiVHNzgWO\nlLRHw7V8sMWEqXnAFEmjevsDi2yvbup3G9V1/0npt6mkN5RtxwH3An8GfEvSHwGbUQ0bryrnP7D0\nvQ8YI+lt5TijyrPvga4rIiJqMhQ/7Xc0sLwMmwJ8HThK0r62b5D0XuAsSecAj1IljNMa9p8qaTLV\nDcNDVJUwVLOcvyXpBKokelSL9jMkjaeqMq8FFgHLgRPLEO/pjc8/bT9anqueKem1wItUSfXq/i7U\n9mJJ5wI3STLwGGsPEff2WylpGnC5pI1L80nlBuXjwO62n5I0DzjJ9hcl3QX8FHiQaqgc28+ViVJf\nk/Qq4HdUz3Wv7++6IiKiPqpGaSMqG48Z7zFHnjPUYURE1Oql/p6upAVlcuuA8o1UERERNUnSjYiI\nqEmSbkRERE2SdCMiImqSpBsREVGTofjIUHSxt2wzmvkvcRZfRET0LZVuRERETZJ0IyIiapKkGxER\nUZMk3YiIiJok6UZERNQkSTciIqImSboRERE1SdKNiIioSZJuRERETZJ0IyIiapKkGxERUZMk3YiI\niJok6UZERNQkSTciIqImSboRERE1SdKNiIioSZJuRERETZJ0IyIiapKkGxERUZMk3YiIiJok6UZE\nRNRkw6EOILrLkhWr6Dlx7lCHERFRm2UzDq7tXKl0IyIiapKkGxERUZMk3YiIiJok6UZERNQkSTci\nIqImSboRERE16UjSlTRW0gOStijrm5f1nrI+XtJVku6XtEDS9ZL2KdumSVopaaGkpZJmSxq5HmOb\nKOmgAbbvLmmepPsk3SXpG5JGNsV1j6SjS/9TJB3fdIxlkrZaXzFHRMTw0JGka/tB4HxgRmmaAVxo\ne5mkTYC5ZX2c7d2AY4AdGg4xy/ZE2xOA54Cp6zG8iUCfSVfS64DvAp+z/UbbbwWuBkY1xgXsB/xT\n6T+kJOWz1hERLxOdHF4+G9hT0rHAZODM0n4EcKvtK3s72r7b9szmA5SEsinwRFnvkXSdpMWSrpX0\n+hbth0m6W9KiUr1uBJwKTC0Va3My/xRwse1bG2KbbfvRxk62HwPuB7YbzBsi6XxJ80sF/6WG9mWS\nviJpiaQ7JP1JaZ8p6YKyz88kvbe0T5N0paTrgGtVOaNc65Le65J0SHk/JGlMOcbWg4k5IiLWn44l\nXdvPAydQJd9jyzrABODOFrtPlbQQWAFsAcwp7V+jSoo7A5cCX23RfjJwgO1dgCm2nyttvZX0rKbz\n7gQsaHVtknagqsx/3qpvky/YngTsDOwraeeGbatsvwU4Fzinob0H2B04GLigjBQA7Aocantf4INU\nFfwuwP7AGZLG2L4CeJjqZuIi4Iu2HxlkzBERsZ50eiLVgVT/09+pvw6SrigV2vcamnuHcbcGllAl\nb4C9gMvK8iVUFfRA7TcDM8vz1xEv8Vpgzc3A5cAnbf8acD99+2r/sKQ7gbuobj52bNh2ecOfezW0\nf8f2i7b/C/gF8KbSfk05P1TXe7nt1aUqvwF4W9l2DPD3wLO2L6cPkj5Rqun5q59Z1c/lRETES9Wx\npCtpIvAuYE/gOEljyqalVFUaALYPAaZRVbRrsW2qKnefdYnB9nTgJGAssEDSli12WQrsNsD23gp5\nj1JFAjwObN7UbxTwm8YGSdsDxwPvLBX5XGCThi5uY7lx/bcDxNloW+BF4HWS+vz7tn2h7Um2J40Y\nObrNw0ZExGB1avayqCZSHWt7OXAGa57pXgbsLWlKwy4DzU6eTPX8FOAW4PCyfARw40DtksbZvt32\nycBKquT7FGsmRjU7FzhS0h4N1/LBFhOm5gFTJI3q7Q8ssr26qd9mVIlyVTnegU3bpzb8eWtD+2GS\nNpA0jmpI+74+YriRqgofIek1VDcpd5Rn4t8EPgLcC3x2gOuIiIgO69TM16OB5bavKetfB46StK/t\nG8qEoLMknQM8SpUIT2vYf6qkyVQ3BQ9RVcJQDZV+S9IJVEn0qBbtZ0gaDwi4FlgELAdOLMPEpzc+\n17X9qKTDgTMlvZaqQpxHNYO5T7YXSzoXuEmSgceAj/fRb5Gku4CfAg9SDX032lzSYuBZqiTZazlw\nB1XSnm7799U9zVquoBqSXkRVCf+d7UcknQzcaPsmSYuAn0iaa/ve/q4nIiI6R9UIbgwlScuASbZ/\n1dQ+E7jK9uy6Ytl4zHiPOfKc1h0jIoaJ9fHTfpIWlImyA8o3UkVERNQkX6zQBWz39NM+rd5IIiKi\nk1LpRkRE1CRJNyIioiZJuhERETXJM91Yy1u2Gc389TCTLyIi/lAq3YiIiJok6UZERNQkSTciIqIm\nSboRERE1SdKNiIioSZJuRERETZJ0IyIiapKkGxERUZMk3YiIiJok6UZERNQkSTciIqImbX33sqQR\nwMFAT+M+ts/qTFgRERHDT7s/eDAH+D2wBHixc+FEREQMX+0m3W1t79zRSCIiIoa5dp/p/lDSuzsa\nSURExDDXbqV7G3CFpA2A5wEBtr1ZxyKLiIgYZtpNumcBewFLbLuD8URERAxb7Q4vPwjcnYQbERGx\n7tqtdH8B/FjSD4FnexvzkaGIiIj2tZt0HyivjcorIiIiBqmtpGv7S50OJCIiYrhr9xup3gAczx9+\nI9X/7kxYMVSWrFhFz4lzhzqMiIjaLJtxcG3nand4+bvABcA3gNWdCyciImL4ajfpvmD7/I5GEhER\nMcwNmHQlbVEW50j6a+AK1p69/OsOxhYRETGstKp0FwCm+gYqgBMathnYoRNBRUREDEcDJl3b2wNI\n2sT27xu3Sdqkk4FFREQMN+1+I9UtbbZFREREP1o9090a2AZ4laS3smaYeTNgZIdji4iIGFZaVboH\nAGcC21L96ME/l9dngc93KihJYyU90DuRS9LmZb2nrI+XdJWk+yUtkHS9pH3KtmmSVkpaKGmppNmS\n1tsNgqSJkg4aYPvukuZJuk/SXZK+IWlkU1z3SDq69D9F0vFNx1gmaas+jv10P+ecLukvBohpP0lv\nb/8qIyKiE1o9070YuFjSh2z/e00xYftBSecDM4BPlD8vtL2sPEueCxxv+0oASTsBk4B55RCzbH+6\nbLsMmAp8az2FN7Gc6wfNGyS9juozzYfbvrW0HQqMaoxL0muBpZKuXB8B2b6gRZf9gKfJI4GIiCE1\nYKUr6aNlsUfSZ5tfHY7tbGBPSccCk6kqboAjgFt7Ey6A7bttz+wj/g2BTYEnynqPpOskLZZ0raTX\nt2g/TNLdkhaV6nUj4FRgaqlYpzad8lPAxb0Jt8Q22/ajjZ1sPwbcD2w32DdF0v8p8dxWkvxa1bKk\nz5RKerGkb5fRgenAcSXmdwz2nBERsX60Gl7etPz5aqpqrfnVMbafp/qI0tnAsWUdYAJwZ4vdp0pa\nCKwAtgDmlPavUSXFnYFLga+2aD8ZOMD2LsAU28+Vtlm2J9qe1XTenag+ZjUgSTtQfdzq5636NtkU\nuK3EMw84uo8+JwJvLdcy3fYyqm8TO7vEfGMf8XxC0nxJ81c/s2qQIUVERLsGTLq2/1XSCOBJ219q\nftUQ34HAw1TJrE+SrijV6PcammfZnghsDSxhzeeL9wIuK8uXUFXQA7XfDMwsz19HvMRrgTU3A5cD\nnyxfLtLfbxT31f4ccFVZXkD1XdjNFgOXllGKF9oJyvaFtifZnjRi5Oh2domIiHXQ8iNDtlcDH6kh\nlrVImgi8C9iTamh0TNm0FNi1Ib5DgGlUFe1abJuqyt1nXWKwPR04CRgLLJC0ZYtdlgK7DbC9t0Le\nw/YVpe1xYPOmfqOA3/Sx//PlmqD6Duy+nskfDJxH9R79pAyxR0REF2j3c7o3SzpX0jsk7dr76lRQ\nkgScTzWsvBw4gzXPdC8D9pY0pWGXgWYnT6Z6fgrVRKLDy/IRwI0DtUsaZ/t22ycDK6mS71P0P7R+\nLnCkpD0aruWDvc9e+zEPmCJpVG9/YFG52RkUSRsAY21fD3wOGE31aGCgmCMioibtVkETy5+nNrQZ\n6NRP+x0NLLd9TVn/OnCUpH1t3yDpvcBZks4BHqVKKqc17D9V0mSqm4qHqCphgGOAb0k6gSqJHtWi\n/QxJ46k+n3wtsAhYDpxYholPb3yua/tRSYcDZ5YZyi9SJdWr+7tQ24slnQvcJMnAY8DHB/NmNRgB\n/D9Jo0vMX7X9G0lzgNmS3g8c09dz3YiI6DytGa2MgI3HjPeYI88Z6jAiImqzPn5PV9IC25Na9Wtr\neFnSaEln9c5wlfTPpZqKiIiINrX7TPebVEO4Hy6vJ1l/XzYRERHxitDuM91xtj/UsP6l8kwzIiIi\n2tRupfu7MjEJAEl7A7/rTEgRERHDU7uV7l9RfQdz76zYX7NmRnBERES0YVCzlyVtBmD7yY5FFENq\n0qRJnj9//lCHERHxstLu7OW2Kt3mHzeovruCVcAC23m2GxER0YZ2n+lOovqlmm3K65PAe4CLJP1d\nh2KLiIgYVtp9prstsKvtpwEkfZHqN233ofri/a90JryIiIjho91K97XAsw3rzwOvs/27pvaIiIjo\nR7uV7qXA7ZL+o6y/D7hM0qbAPR2JLCIiYphpK+na/kdJPwT2Lk3TbfdOcT2iI5FFREQMMwMmXUmN\nv1H7i/L6n23lR9gjIiKiDa0q3QVUP+Gnst77oV6V5R06FFdERMSwM2DStb1973KpescDm3Q6qIiI\niOGo3S/H+DjwN1QfHVoI7AncAryzc6FFREQML+1+ZOhvgLcBv7T9p8Bbqb6RKiIiItrUbtL9ve3f\nA0ja2PZPgTd2LqyIiIjhp93P6T4k6Y+B7wPXSHoC+GXnwoqIiBh+2v2c7iFl8RRJ1wOjgas7FlVE\nRMQw1G6l+z9s39CJQCIiIoa7dp/pRkRExEuUpBsREVGTJN2IiIiaJOlGRETUJEk3IiKiJkm6ERER\nNRn0R4ZieFuyYhU9J84d6jAiImq1bMbBtZwnlW5ERERNknQjIiJqkqQbERFRkyTdiIiImiTpRkRE\n1CRJNyIioiYdT7qSxkp6QNIWZX3zst5T1sdLukrS/ZIWSLpe0j5l2zRJKyUtlLRU0mxJI9djbBMl\nHTTA9t0l/VjSf0m6U9JcSW8p206RtKLEdrekKaV9pqRDm47z9PqKuV2S9pP09rrPGxER/et40rX9\nIHA+MKM0zQAutL1M0ibA3LI+zvZuwDHADg2HmGV7ou0JwHPA1PUY3kSgz6Qr6XXAd4DP2x5ve1fg\ndGBcQ7ezbU8EDgO+KambRg72A5J0IyK6SF1J4mxgT0nHApOBM0v7EcCttq/s7Wj7btszmw8gaUNg\nU+CJst4j6TpJiyVdK+n1LdoPKxXpIknzJG0EnApMLdVqczL/NHCx7VsaYrvJ9vebY7N9L/ACsNVg\n3hRJH5V0Rzn/v0oaIWm7UllvJWkDSTdKenfp//0yGrBU0icajvOeUokvKtfcA0wHjivHfsdg4oqI\niM6o5RupbD8v6QTgauDdtp8vmyYAd7bYfaqkycAY4GfAnNL+NaqkeLGkvwS+CnxggPaTgQNsr5D0\nx7afk3QyMMn2p/s47wTg4nauT9IewIvAynb6l33eTFW1713en68DR9j+N0lfphoduAO4x/Z/lt3+\n0vavJb0K+Imkf6e6cboI2Mf2A5K2KH0uAJ62feYfnj0iIoZCncOhBwIPAzv110HSFaUa/V5D86wy\nhLs1sAQ4obTvBVxWli+hqqAHar8ZmCnpaGDEYIOXdLukeyX9S0PzcZIWUlXuU20bcB+799X2TmA3\nquS5sKzvAGD7G8BmVNXq8Q37fEbSIuA2YCwwHtgTmGf7gbLvr9fh2j4hab6k+aufWTXY3SMiok21\nJF1JE4F3USWI4ySNKZuWArv29rN9CDAN2KL5GCWhzQH2WZcYbE8HTqJKVgskbdlil+bY9gD+ARjd\n0Ofs8rz5HbZvLG2PA5v3digTyH7Vx/FFVZFPLK832j6l7DMS2Lb0e3Vp2w/YH9jL9i7AXcAmLS+8\nDbYvtD3J9qQRI0e33iEiItZJHbOXRTVUeqzt5cAZrHmmexmwd+/M32Kg2cmTgfvL8i3A4WX5CODG\ngdoljbN9u+2TqYaBxwJPAaP6Odd5wLSmGcDtzJz+MdWQ+EZlfRpwfR/9rgUOlfTaEt8WkrYr274M\nXEo1JH5RaRsNPGH7GUlvorqBgarq3UfS9r3HKe0DXVtERAyBOp7pHg0st31NWf86cJSkfW3fIOm9\nwFmSzgEepUoWpzXs3/tMdwPgIaokBtUs52+VZ8UrgaNatJ8haTxVhXktsAhYDpxYhndPtz2r96S2\nHymTq74saRvgMaqK9dSBLtb2VZJ2o6qmV1PdJEzvo989kk4C/rPMen4e+FSZBPU2qme9qyV9SNJR\nVDco0yXdC9xHlWyxvbJMqvpeOc5jVKMKc4DZkt4PHNNQiUdExBBRNWobUdl4zHiPOfKcoQ4jIqJW\nL/Wn/SQtsD2pVb9u+lxpRETEsJakGxERUZMk3YiIiJok6UZERNQkSTciIqImtXwNZLx8vGWb0cx/\nibP4IiKib6l0IyIiapKkGxERUZMk3YiIiJok6UZERNQkSTciIqImSboRERE1SdKNiIioSZJuRERE\nTZJ0IyIiapKkGxERUZMk3YiIiJok6UZERNQkSTciIqImSboRERE1SdKNiIioSZJuRERETZJ0IyIi\napKkGxERUZMk3YiIiJok6UZERNQkSTciIqImGw51ANFdlqxYRc+Jc4c6jIiI2iybcXBt50qlGxER\nUZMk3YiIiJok6UZERNQkSTciIqImSboRERE1SdKNiIioSZJuRERETboi6UoaK+kBSVuU9c3Lek9Z\nHy/pKkn3S1og6XpJ+5Rt0yStlLRQ0lJJsyWNXI+xTZR00ADbd5f0Y0n/JelOSXMlvaVsO0XSihLb\n3ZKmlPaZkg5tOs7TfRy7R9Ld/Zz3VEn7DxDXByTt2O51RkRE53VF0rX9IHA+MKM0zQAutL1M0ibA\n3LI+zvZuwDHADg2HmGV7ou0JwHPA1PUY3kSgz6Qr6XXAd4DP2x5ve1fgdGBcQ7ezbU8EDgO+KWm9\nvOe2T7b9owG6fABI0o2I6CJdkXSLs4E9JR0LTAbOLO1HALfavrK3o+27bc9sPoCkDYFNgSfKeo+k\n6yQtlnStpNe3aD+sVKSLJM2TtBFwKjC1VKvNyfzTwMW2b2mI7Sbb32+Ozfa9wAvAVoN8X0ZIuqhU\n8f8p6VUl1v+pliXNkHRPuZ4zJb0dmAKcUeIeN9AJIiKiHl2TdG0/D5xAlXyPLesAE4A7W+w+VdJC\nYAWwBTCntH+NKinuDFwKfLVF+8nAAbZ3AabYfq609VbSs5rO205sAEjaA3gRWNlO/wbjgfNKFf8b\n4ENNx90SOASYUK7ntHITcCVwQon7/haxfULSfEnzVz+zapDhRUREu7om6RYHAg8DO/XXQdIVpRr9\nXkPzrDKEuzWwhCp5A+wFXFaWL6GqoAdqvxmYKeloYMRgg5d0u6R7Jf1LQ/Nx5YbgTGCqbQPuY/e+\n2gAesL2wLC8Aepq2rwJ+D/xfSR8Enhls3LYvtD3J9qQRI0cPdveIiGhT1yRdSROBdwF7UiWqMWXT\nUmDX3n62DwGmUVW0aykJbQ6wz7rEYHs6cBIwFlhQqsiBNMe2B/APQGPmOrtUm++wfWNpexzYvLdD\nmUD2q37O8WzD8mqafqTC9gvA7sBs4L3A1S1ijoiIIdIVSVeSqCZSHWt7OXAGa57pXgbs3Tvztxho\ndvJkoHc49Rbg8LJ8BHDjQO2Sxtm+3fbJVMPAY4GngFH9nOs8YFp5htpObL1+TDUkvlFZnwZc38Z+\nf0DSq4HRtn8AHAfsUjYNFHdERAyBbvlpv6OB5bavKetfB46StK/tGyS9FzhL0jnAo1QJ5bSG/adK\nmkx1E/EQVRKDapbztySdQJVEj2rRfoak8YCAa4FFwHLgxDJEfHrjc13bj5TJVV+WtA3wGFXFeupA\nF2v7Kkm7UVXTq6luEqa3+2Y1GQX8R5nlLeCzpf3bwEWSPgMc2uq5bkREdJ6qEdmIysZjxnvMkecM\ndRgREbVZH7+nK2mB7Umt+nXF8HJERMQrQZJuRERETZJ0IyIiapKkGxERUZMk3YiIiJp0y0eGoku8\nZZvRzF8PM/kiIuIPpdKNiIioSZJuRERETZJ0IyIiapKkGxERUZMk3YiIiJok6UZERNQkSTciIqIm\nSboRERE1SdKNiIioSZJuRERETfIj9rEWSU8B9w11HP3YCvjVUAfRj8S2bro5Nuju+BLbuulUbNvZ\nfk2rTvnu5Wh2n+1JQx1EXyTNT2yDl9jWXTfHl9jWzVDHluHliIiImiTpRkRE1CRJN5pdONQBDCCx\nrZvEtu66Ob7Etm6GNLZMpIqIiKhJKt2IiIiaJOlGRETUJEn3FUrSeyTdJ+nnkk7sY/vGkmaV7bdL\n6umi2PYrdpLYAAADt0lEQVSRdKekFyQdWldcbcb2WUn3SFos6VpJ23VRbNMlLZG0UNJNknbsltga\n+n1IkiXV9pGONt63aZJWlvdtoaSPd0tspc+Hy7+5pZIuqyu2duKTdHbD+/YzSb/potheL+l6SXeV\n/14PqiUw23m9wl7ACOB+YAdgI2ARsGNTn78GLijLhwOzuii2HmBn4N+AQ7vsfftTYGRZ/qsue982\na1ieAlzdLbGVfqOAecBtwKRuiQ2YBpxb17+zQcY2HrgL2Lysv7ab4mvqfwzwzW6JjWpC1V+V5R2B\nZXXElkr3lWl34Oe2f2H7OeDbwPub+rwfuLgszwbeKUndEJvtZbYXAy/WEM9gY7ve9jNl9TZg2y6K\n7cmG1U2BumZRtvPvDeAfgS8Dv68prsHENhTaie1o4DzbTwDYfqzL4mv0EeDyWiJrLzYDm5Xl0cD/\nryOwJN1Xpm2ABxvWHyptffax/QKwCtiyS2IbKoON7WPADzsa0RptxSbpU5LuB74CfKZbYpO0KzDW\n9tyaYurV7t/ph8oQ5GxJY+sJra3Y3gC8QdLNkm6T9J6aYoNB/PdQHrNsD1xXQ1zQXmynAB+V9BDw\nA6pKvOOSdCM6QNJHgUnAGUMdSyPb59keB3wOOGmo4wGQtAFwFvC3Qx1LP+YAPbZ3Bq5hzQhQN9iQ\naoh5P6pK8iJJfzykEfXtcGC27dVDHUiDjwAzbW8LHARcUv4tdlSS7ivTCqDxbn3b0tZnH0kbUg2/\nPN4lsQ2VtmKTtD/wBWCK7We7KbYG3wY+0NGI1mgV2yhgJ+DHkpYBewJX1jSZquX7Zvvxhr/HbwC7\n1RBXW7FRVXBX2n7e9gPAz6iScLfE1+tw6htahvZi+xjwHQDbtwKbUP0YQmfV9dA9r+55Ud0d/4Jq\nuKd3ksGEpj6fYu2JVN/pltga+s6k3olU7bxvb6WawDG+C/9Oxzcsvw+Y3y2xNfX/MfVNpGrnfRvT\nsHwIcFsXxfYe4OKyvBXVkOqW3RJf6fcmYBnly5i6JTaqRz/TyvKbqZ7pdjzGWt6AvLrvRTWc8rOS\nIL5Q2k6lqs6guuv7LvBz4A5ghy6K7W1Ud/i/paq+l3ZRbD8CHgUWlteVXRTbvwBLS1zXD5T46o6t\nqW9tSbfN9+308r4tKu/bm7ooNlENzd8DLAEOryu2dv9eqZ6dzqgzrjbfux2Bm8vf60Lg3XXEla+B\njIiIqEme6UZERNQkSTciIqImSboRERE1SdKNiIioSZJuRERETZJ0IyIiapKkGxERUZP/BgVci9kA\nBXKHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2ab52e046e10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data.plot(label=\"training time\",kind='barh',x='algorithm',y='training time')\n",
    "data.plot(title=\"training AUC\",kind='barh',x='algorithm',y='training AUC',legend=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Now call XGBoost from H2O"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking whether there is an H2O instance running at http://localhost:54321..... not found.\n",
      "Attempting to start a local H2O server...\n",
      "  Java Version: java version \"1.8.0_131\"; Java(TM) SE Runtime Environment (build 1.8.0_131-b11); Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)\n",
      "  Starting server from /usr/local/lib/python3.6/dist-packages/h2o/backend/bin/h2o.jar\n",
      "  Ice root: /tmp/tmpbz9cy2k1\n",
      "  JVM stdout: /tmp/tmpbz9cy2k1/h2o_nimbix_started_from_python.out\n",
      "  JVM stderr: /tmp/tmpbz9cy2k1/h2o_nimbix_started_from_python.err\n",
      "  Server is running at http://127.0.0.1:54321\n",
      "Connecting to H2O server at http://127.0.0.1:54321... successful.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div style=\"overflow:auto\"><table style=\"width:50%\"><tr><td>H2O cluster uptime:</td>\n",
       "<td>01 secs</td></tr>\n",
       "<tr><td>H2O cluster version:</td>\n",
       "<td>3.10.5.1</td></tr>\n",
       "<tr><td>H2O cluster version age:</td>\n",
       "<td>6 days </td></tr>\n",
       "<tr><td>H2O cluster name:</td>\n",
       "<td>H2O_from_python_nimbix_6vso2s</td></tr>\n",
       "<tr><td>H2O cluster total nodes:</td>\n",
       "<td>1</td></tr>\n",
       "<tr><td>H2O cluster free memory:</td>\n",
       "<td>26.67 Gb</td></tr>\n",
       "<tr><td>H2O cluster total cores:</td>\n",
       "<td>40</td></tr>\n",
       "<tr><td>H2O cluster allowed cores:</td>\n",
       "<td>40</td></tr>\n",
       "<tr><td>H2O cluster status:</td>\n",
       "<td>accepting new members, healthy</td></tr>\n",
       "<tr><td>H2O connection url:</td>\n",
       "<td>http://127.0.0.1:54321</td></tr>\n",
       "<tr><td>H2O connection proxy:</td>\n",
       "<td>None</td></tr>\n",
       "<tr><td>H2O internal security:</td>\n",
       "<td>False</td></tr>\n",
       "<tr><td>Python version:</td>\n",
       "<td>3.6.1 final</td></tr></table></div>"
      ],
      "text/plain": [
       "--------------------------  ------------------------------\n",
       "H2O cluster uptime:         01 secs\n",
       "H2O cluster version:        3.10.5.1\n",
       "H2O cluster version age:    6 days\n",
       "H2O cluster name:           H2O_from_python_nimbix_6vso2s\n",
       "H2O cluster total nodes:    1\n",
       "H2O cluster free memory:    26.67 Gb\n",
       "H2O cluster total cores:    40\n",
       "H2O cluster allowed cores:  40\n",
       "H2O cluster status:         accepting new members, healthy\n",
       "H2O connection url:         http://127.0.0.1:54321\n",
       "H2O connection proxy:\n",
       "H2O internal security:      False\n",
       "Python version:             3.6.1 final\n",
       "--------------------------  ------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parse progress: |█████████████████████████████████████████████████████████| 100%\n",
      "Time to parse by H2O (sec):  3.3415310382843018\n"
     ]
    }
   ],
   "source": [
    "import h2o\n",
    "from h2o.estimators import H2OXGBoostEstimator\n",
    "h2o.init()\n",
    "\n",
    "t_start = time.time()\n",
    "df_hex = h2o.import_file(path)\n",
    "print(\"Time to parse by H2O (sec): \", time.time() - t_start)\n",
    "\n",
    "trainhex = df_hex\n",
    "trainhex[0] = (trainhex[0]).asfactor()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def runH2OXGBoost(param):\n",
    "    label = \"H2O XGBoost \" \\\n",
    "    + (\"GPU\" if \"backend\" in param.keys() and \"gpu\"==param[\"backend\"] else \"CPU\") \\\n",
    "    + (\" \" + param[\"tree_method\"] if \"tree_method\" in param.keys() else \"\")\n",
    "    print(label)\n",
    "    print(\"=====================\")\n",
    "    for k, v in param.items():\n",
    "        print(k, v)\n",
    "    print(\"=====================\")\n",
    "        \n",
    "    t_start = time.time()\n",
    "    model = H2OXGBoostEstimator(**param)\n",
    "    model.train(x = list(range(1,trainhex.shape[1])), y = 0, training_frame = trainhex)\n",
    "    tt = time.time() - t_start\n",
    "    print(\"Time to train (sec): \", tt)\n",
    "    h2o_train_times.append(tt)\n",
    "\n",
    "    t_start = time.time()\n",
    "    preds = model.predict(trainhex)[:,2]\n",
    "    tt = time.time() - t_start\n",
    "    print(\"Time to predict (sec): \", tt)\n",
    "    h2o_score_times.append(tt)\n",
    "\n",
    "    preds = h2o.as_list(preds)\n",
    "    labels = train_target.values\n",
    "    auc = metrics.roc_auc_score(labels, preds)\n",
    "    print(\"Training AUC:\", auc)\n",
    "\n",
    "    h2o_valid_aucs.append(auc)\n",
    "    h2o_plot_labels.append(label)\n",
    "    \n",
    "    #pd.DataFrame(model.varimp(),columns=[\"Feature\",\"\",\"Importance\",\"\"]).head(10).plot(label='importance',kind=\"barh\",x=\"Feature\",y=\"Importance\").invert_yaxis()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "h2o_valid_aucs = []\n",
    "h2o_train_times = []\n",
    "h2o_score_times = []\n",
    "h2o_plot_labels = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "H2O XGBoost CPU exact\n",
      "=====================\n",
      "ntrees 100\n",
      "max_depth 10\n",
      "eta 0.02\n",
      "subsample 0.7\n",
      "colsample_bytree 0.9\n",
      "min_child_weight 5\n",
      "seed 12345\n",
      "score_tree_interval 100\n",
      "backend cpu\n",
      "tree_method exact\n",
      "=====================\n",
      "xgboost Model Build progress: |███████████████████████████████████████████| 100%\n",
      "Time to train (sec):  192.16322207450867\n",
      "xgboost prediction progress: |████████████████████████████████████████████| 100%\n",
      "Time to predict (sec):  7.761167287826538\n",
      "Training AUC: 0.819896987781\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"ntrees\":num_round\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"eta\":learn_rate\n",
    "    , \"subsample\":0.7\n",
    "    , \"colsample_bytree\":0.9\n",
    "    , \"min_child_weight\":5\n",
    "    , \"seed\":12345\n",
    "    , \"score_tree_interval\":num_round\n",
    "    , \"backend\":\"cpu\"\n",
    "    , \"tree_method\":\"exact\"\n",
    "}\n",
    "runH2OXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "H2O XGBoost CPU approx\n",
      "=====================\n",
      "ntrees 100\n",
      "max_depth 10\n",
      "eta 0.02\n",
      "subsample 0.7\n",
      "colsample_bytree 0.9\n",
      "min_child_weight 5\n",
      "seed 12345\n",
      "score_tree_interval 100\n",
      "backend cpu\n",
      "tree_method approx\n",
      "=====================\n",
      "xgboost Model Build progress: |███████████████████████████████████████████| 100%\n",
      "Time to train (sec):  166.70660161972046\n",
      "xgboost prediction progress: |████████████████████████████████████████████| 100%\n",
      "Time to predict (sec):  7.733324766159058\n",
      "Training AUC: 0.818158055647\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"ntrees\":num_round\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"eta\":learn_rate\n",
    "    , \"subsample\":0.7\n",
    "    , \"colsample_bytree\":0.9\n",
    "    , \"min_child_weight\":5\n",
    "    , \"seed\":12345\n",
    "    , \"score_tree_interval\":num_round\n",
    "    , \"backend\":\"cpu\"\n",
    "    , \"tree_method\":\"approx\"\n",
    "}\n",
    "runH2OXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "H2O XGBoost CPU hist\n",
      "=====================\n",
      "ntrees 100\n",
      "max_depth 10\n",
      "eta 0.02\n",
      "subsample 0.7\n",
      "colsample_bytree 0.9\n",
      "min_child_weight 5\n",
      "seed 12345\n",
      "score_tree_interval 100\n",
      "backend cpu\n",
      "tree_method hist\n",
      "=====================\n",
      "xgboost Model Build progress: |███████████████████████████████████████████| 100%\n",
      "Time to train (sec):  114.9106297492981\n",
      "xgboost prediction progress: |████████████████████████████████████████████| 100%\n",
      "Time to predict (sec):  7.72308611869812\n",
      "Training AUC: 0.819105186679\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"ntrees\":num_round\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"eta\":learn_rate\n",
    "    , \"subsample\":0.7\n",
    "    , \"colsample_bytree\":0.9\n",
    "    , \"min_child_weight\":5\n",
    "    , \"seed\":12345\n",
    "    , \"score_tree_interval\":num_round\n",
    "    , \"backend\":\"cpu\"\n",
    "    , \"tree_method\":\"hist\"\n",
    "}\n",
    "runH2OXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "H2O XGBoost GPU exact\n",
      "=====================\n",
      "ntrees 100\n",
      "max_depth 10\n",
      "learn_rate 0.02\n",
      "sample_rate 0.7\n",
      "col_sample_rate_per_tree 0.9\n",
      "min_rows 5\n",
      "seed 12345\n",
      "score_tree_interval 100\n",
      "backend gpu\n",
      "tree_method exact\n",
      "=====================\n",
      "xgboost Model Build progress: |███████████████████████████████████████████| 100%\n",
      "Time to train (sec):  85.09905505180359\n",
      "xgboost prediction progress: |████████████████████████████████████████████| 100%\n",
      "Time to predict (sec):  7.728891849517822\n",
      "Training AUC: 0.819862038742\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"ntrees\":num_round\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"learn_rate\":learn_rate\n",
    "    , \"sample_rate\":0.7\n",
    "    , \"col_sample_rate_per_tree\":0.9\n",
    "    , \"min_rows\":5\n",
    "    , \"seed\":12345\n",
    "    , \"score_tree_interval\":num_round\n",
    "    , \"backend\":\"gpu\"\n",
    "    , \"tree_method\":\"exact\"\n",
    "}\n",
    "runH2OXGBoost(param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "H2O XGBoost GPU hist\n",
      "=====================\n",
      "ntrees 100\n",
      "max_depth 10\n",
      "learn_rate 0.02\n",
      "sample_rate 0.7\n",
      "col_sample_rate_per_tree 0.9\n",
      "min_rows 5\n",
      "seed 12345\n",
      "score_tree_interval 100\n",
      "backend gpu\n",
      "tree_method hist\n",
      "=====================\n",
      "xgboost Model Build progress: |███████████████████████████████████████████| 100%\n",
      "Time to train (sec):  17.322280406951904\n",
      "xgboost prediction progress: |████████████████████████████████████████████| 100%\n",
      "Time to predict (sec):  7.722412109375\n",
      "Training AUC: 0.819402793669\n"
     ]
    }
   ],
   "source": [
    "param = {\n",
    "    \"ntrees\":num_round\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"learn_rate\":learn_rate\n",
    "    , \"sample_rate\":0.7\n",
    "    , \"col_sample_rate_per_tree\":0.9\n",
    "    , \"min_rows\":5\n",
    "    , \"seed\":12345\n",
    "    , \"score_tree_interval\":num_round\n",
    "    , \"backend\":\"gpu\"\n",
    "    , \"tree_method\":\"hist\"\n",
    "}\n",
    "runH2OXGBoost(param)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## H2O GBM (CPU)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gbm Model Build progress: |███████████████████████████████████████████████| 100%\n",
      "Time to train (sec):  50.22540855407715\n",
      "gbm prediction progress: |████████████████████████████████████████████████| 100%\n",
      "Time to predict (sec):  3.781240463256836\n",
      "AUC: 0.818376121653\n"
     ]
    }
   ],
   "source": [
    "from h2o.estimators.gbm import H2OGradientBoostingEstimator\n",
    "param = {\n",
    "      \"ntrees\":num_round\n",
    "    , \"max_depth\":max_depth\n",
    "    , \"learn_rate\":learn_rate\n",
    "    , \"sample_rate\":0.7\n",
    "    , \"col_sample_rate_per_tree\":0.9\n",
    "    , \"min_rows\":5\n",
    "    , \"seed\":12345\n",
    "    , \"score_tree_interval\":num_round\n",
    "}\n",
    "\n",
    "t_start = time.time()\n",
    "model = H2OGradientBoostingEstimator(**param)\n",
    "model.train(x = list(range(1,trainhex.shape[1])), y = 0, training_frame = trainhex)\n",
    "tt = time.time() - t_start\n",
    "print(\"Time to train (sec): \", tt)\n",
    "h2o_train_times.append(tt)\n",
    "\n",
    "t_start = time.time()\n",
    "preds = model.predict(trainhex)[:,2]\n",
    "tt = time.time() - t_start\n",
    "print(\"Time to predict (sec): \", tt)\n",
    "h2o_score_times.append(tt)\n",
    "\n",
    "preds = h2o.as_list(preds)\n",
    "labels = train_target.values\n",
    "auc = metrics.roc_auc_score(labels, preds)\n",
    "print(\"AUC:\", auc)\n",
    "\n",
    "h2o_valid_aucs.append(auc)\n",
    "h2o_plot_labels.append(\"H2O GBM CPU\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>algorithm</th>\n",
       "      <th>scoring time</th>\n",
       "      <th>training AUC</th>\n",
       "      <th>training time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>H2O XGBoost GPU hist</td>\n",
       "      <td>7.722412</td>\n",
       "      <td>0.819403</td>\n",
       "      <td>17.322280</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>H2O GBM CPU</td>\n",
       "      <td>3.781240</td>\n",
       "      <td>0.818376</td>\n",
       "      <td>50.225409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>H2O XGBoost GPU exact</td>\n",
       "      <td>7.728892</td>\n",
       "      <td>0.819862</td>\n",
       "      <td>85.099055</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>H2O XGBoost CPU hist</td>\n",
       "      <td>7.723086</td>\n",
       "      <td>0.819105</td>\n",
       "      <td>114.910630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>H2O XGBoost CPU approx</td>\n",
       "      <td>7.733325</td>\n",
       "      <td>0.818158</td>\n",
       "      <td>166.706602</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>H2O XGBoost CPU exact</td>\n",
       "      <td>7.761167</td>\n",
       "      <td>0.819897</td>\n",
       "      <td>192.163222</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                algorithm  scoring time  training AUC  training time\n",
       "4    H2O XGBoost GPU hist      7.722412      0.819403      17.322280\n",
       "5             H2O GBM CPU      3.781240      0.818376      50.225409\n",
       "3   H2O XGBoost GPU exact      7.728892      0.819862      85.099055\n",
       "2    H2O XGBoost CPU hist      7.723086      0.819105     114.910630\n",
       "1  H2O XGBoost CPU approx      7.733325      0.818158     166.706602\n",
       "0   H2O XGBoost CPU exact      7.761167      0.819897     192.163222"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.DataFrame({'algorithm'  :h2o_plot_labels,\n",
    "                     'training time':h2o_train_times,\n",
    "                     'scoring time':h2o_score_times,\n",
    "                     'training AUC' :h2o_valid_aucs}).sort_values(by=\"training time\")\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2ab5a77bc898>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfkAAAD8CAYAAAB0DN3jAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2cVWW9///X2wEF5cbAO5KTo2beADIiKiKahpqeDG/y\nZKGFHf35VaO8SYrCyspTBGamaIrKQY3UI5lxvCHNQE2yuBscRDBRMrEjikcUQY7o5/fHugYX272Z\nDczMHrbv5+Mxj9nrWtda67MWm/ms62btrYjAzMzMqs9WlQ7AzMzMWoaTvJmZWZVykjczM6tSTvJm\nZmZVyknezMysSjnJm5mZVSkneTMzsyrlJG9mZlalnOTNzMyqVLtKB2AfPjvssEPU1tZWOgwzsy3K\n7NmzX42IHTdmGyd5a3W1tbXMmjWr0mGYmW1RJP19Y7dxd72ZmVmVcpI3MzOrUk7yZmZmVcpJ3szM\nrEo5yZuZmVUpJ3kzM7Mq5SRvZmZWpfycvLW6hqUrqB15X6XDMDNrVUtGf6bVj+mWvJmZWZVykjcz\nM6tSTvJmZmZVyknezMysSjnJm5mZVamKJHlJKwuWz5Q0Lr2+WNICSU9KeljSbrl6vST9UdIiSX+T\n9F1JKrL/UyQ9nFseJKleUru0fJykv0pamMrvlPSxtG6ipOdT+UJJ32/mcz9J0n4bWP9lSfMlNUia\nK+mSInHNkXRoKp8uqX9u+1pJ85sz5nI0dV5mZtb62mJLfi7QPyL2ByYDYwAkdQSmAKMjYm+gLzAQ\nOL9wBxFxN7BG0lBJ7YHrgPMjYq2k3sA1wLCI2Cci6oBJQG1uFyNSeR0wTNLuzXh+JwFFk6Gk44EL\ngWMjog8wAFhRJK6RwA3NGFNzKHleZmZWGW0uyUfEtIhYlRafAHqm10OBxyPiwVRvFTCcLOEVMxy4\nHLgMmBkRM1L5t4AfR8TTuWNOiYhHi+yjQ/r9FoCkwal13SBpgqRtmigfneuVuELSQGAIMDa1yPcs\nON63gUsi4qUU15qIuLFIXI8CHy9x3iVJGiFpZornB6nsoLTcQdJ2kp6S1FtSp9STMied14m5/Xw5\nbTNP0m1lnJeZmVVApT4Mp6Ok+txyN7JWeqGzgAfS617A7PzKiFicklGXiHijYN1zku4kS/b5pNML\nuKKJ+MZKupQskV4dEcskdQAmAoMj4hlJtwLnSbq+RPltwMnAPhERkraPiNclTQHujYjJRY7bu/Ac\nS/gs0FBGvXUkHQvsBRwMCJgi6YiIeDTFdDnQEfhVRMxPQxsnR8QbknYAnkj19gMuBQZGxKuSukXE\na02cF5LOAc4BqOmy48aEbmZmm6hSLfnVEVHX+AN8r7CCpDOA/sDYTTmApBrgGGAlsFuJOt1Ty/OZ\nxrHvpLFbfBdgcGqp7g08HxHPpDq3AEdsoHwF8DZws6RTgFVsvrHp5ugcshsggChSr1jZselnLjAH\n2Ics6QP8kOxa9ScNj5DdCPxY0pPAH4BdgZ2BTwF3RcSrABHxWjmBR8T4iOgfEf1rtu1aziZmZraZ\n2lx3PYCko4FRwJCIWJOKFwAHFtTbA1hZ2IpPzidr7Z4FXJuboPcU0A8gIpanZD4e6FS4g4hYCUwH\nBm3sOUTEWrJW82TgBGBqGZs9RcE5FhiRboyOiYjGyXXLgY/k6nQDXi2yrYCf5G6uPh4RN6d13cnO\nvzPvD1GcDuwIHJiu0cu5dWZmtgVoc0le0gFkk8qGRMSy3KpJwKB0A9A4Ee9q3m955vexC3Ax8M2I\nmAosBc5Oq8cAoyTtm9tk2xKxtAMOARYDi4BaSY1j4V8CHilVLqkT0DUi7gcuIpsoCPAmWTIt5idk\nrfVd0vG3lnR2ibqNpgNn5G5ihgHTitT7PfDvKS4k7Sppp7TuBuC7ZNf4p6msK7AsIt6RdBTv94b8\nEfg3Sd3TfrqVcV5mZlYBbfELasaStSrvSnnrhYgYEhGr0+SvayRdC9QAtwHjiuzjSmBMRLySli8E\nHpP0m4hokHQBcKukLmSt3heA/KNyjWPyWwMPA3encfWvpLjaATOB6yNiTbFyshb179JYvshuOgDu\nAG6U9HXg1IhY3HjQiLhf0s7AH1LSDmBCE9drPFnX+zxJAcwim8C3noh4MN3Y/Dld15VkNwfHAe9E\nxK/TEMcMSZ8iS/j/Lakh7XNh2s9Tkv6D7EbmXbLu/zM3dF5mZlYZiig2fGvWcrbpsVf0GHZVpcMw\nM2tVm/stdJJmR0T/pmu+r81115uZmVnzcJI3MzOrUk7yZmZmVcpJ3szMrEq1xdn1VuX67NqVWZs5\nAcXMzJrmlryZmVmVcpI3MzOrUk7yZmZmVcpJ3szMrEo5yZuZmVUpJ3kzM7Mq5SRvZmZWpZzkzczM\nqpSTvJmZWZVykjczM6tSTvJmZmZVyknezMysSjnJm5mZVSkneTMzsyrlJG9mZlalnOTNzMyqlJO8\nmZlZlWpX6QDsw6dh6QpqR95X6TDMrAosGf2ZSofQprklb2ZmVqWc5M3MzKqUk7yZmVmVcpI3MzOr\nUi2W5CWtLFg+U9K49PpiSQskPSnpYUm75er1kvRHSYsk/U3SdyWpyP5PkfRwbnmQpHpJ7dLycZL+\nKmlhKr9T0sfSuomSnk/lCyV9v5nP/SRJ+21g/ZclzZfUIGmupEuKxDVH0qGpfLqk/rntayXNb86Y\nzcys+lSqJT8X6B8R+wOTgTEAkjoCU4DREbE30BcYCJxfuIOIuBtYI2mopPbAdcD5EbFWUm/gGmBY\nROwTEXXAJKA2t4sRqbwOGCZp92Y8v5OAokle0vHAhcCxEdEHGACsKBLXSOCGZoxpkzXeOJmZ2Zal\nIkk+IqZFxKq0+ATQM70eCjweEQ+mequA4WQJr5jhwOXAZcDMiJiRyr8F/Dgins4dc0pEPFpkHx3S\n77cAJA1OresGSRMkbdNE+ehcr8QVkgYCQ4CxqUW+Z8Hxvg1cEhEvpbjWRMSNReJ6FPh4ifMuSlKn\n1DMyJ8V5YiqvTT0WkyQ9LWmypG3TuiWSxqT6f5X08VQ+UdL1kv4CjJHUTdI96TyfkLR/qvcLSd9L\nrz8t6VFJHgYyM2sDWvKPcceU5Ool1QM/LFHvLOCB9LoXMDu/MiIWA50kdSncMCKeA+4kS/bfyq3q\nBcxpIr6xKa4XgTsiYpmkDsBE4LTUym4HnLeB8u7AyUCv1CtxebrRmEJqkaf483oXnmMJnwUayqiX\n9zZwckT0A44CfpYb6tgbuC4i9gXeYP3ekRXpvMYBV+XKewIDI+Ji4AfA3HSe3wFuTXW+DZwm6Sjg\nauArEfHeRsZtZmYtoCWT/OqU5OpS9/P3CitIOgPoD4zdlANIqgGOAVYCu5Wo0z3daDzTOPadNHaL\n7wIMTi3wvYHnI+KZVOcW4IgNlK8gS6w3SzoFWMXma7z5OIfsBgggitQrVibgx5KeBP4A7ArsnNb9\nIyIeT69/BQzKbXd77vehufK7IuLd9HoQcBtARPwR6C6pS+pt+f+Ah4BxRW5qssCkcyTNkjTr3VUr\nilUxM7NmVrFuVUlHA6OAIRGxJhUvAA4sqLcHsDIi3iiym/PJWrtnAdfmWq1PAf0AImJ5SubjgU6F\nO4iIlcB01k96ZYmItcDBZPMKTgCmlrHZUxScY4HGHoBjIqJxct1y4CO5Ot2AV4tsezqwI3BgOueX\neX84ovCmIMp4/dYG4szrk2L8aKkKETE+IvpHRP+abbuWuVszM9scFUnykg4gm1Q2JCKW5VZNAgal\nG4DGiXhXkybmFexjF+Bi4JsRMRVYCpydVo8BRknaN7fJtiViaQccAiwGFgG1jePSwJeAR0qVS+oE\ndI2I+4GLyCYKArwJdC5x+j8ha63vko6/taSzS9RtNB04I3cTMwyYVqReV2BZRLyTus/zvRsfa5yt\nTzb34U+5daflfv+5RAyPkd1EIOlI4NWIeEPZkxHfAA4Ajpd0SBPnYmZmraRSs6bHkrWq70p564WI\nGBIRq9NksWskXQvUkHURjyuyjyuBMRHxSlq+EHhM0m8iokHSBcCtaSz/VeAFIP+o3FhJlwJbAw8D\nd0dESPpKiqsdMBO4PiLWFCsna1H/Lo3Zi+ymA+AO4EZJXwdOzXdhR8T9knYG/pCSdgATmrhe44F9\ngHmSAphFNhZeaBLw35IaUp2FuXWLgK9KmkDWY/LL3LqPpC7+NcAXS8RwGTAh1VtF9kSCgJtJEwkl\nnQVMlHRQRLzdxDmZmVkLU0SxoV2rJpJqgXsjoneRdUvIHmcs1v3fIrbpsVf0GHZV0xXNzJrwYfqC\nGkmzI6J/0zXf50edzMzMqpQ/5ORDICKWkD26V2xdbasGY2ZmrcYteTMzsyrlJG9mZlal3F1vra7P\nrl2Z9SGaLGNmViluyZuZmVUpJ3kzM7Mq5SRvZmZWpZzkzczMqpSTvJmZWZVykjczM6tSTvJmZmZV\nyknezMysSjnJm5mZVSkneTMzsyrlJG9mZlalnOTNzMyqlJO8mZlZlXKSNzMzq1JO8mZmZlXKSd7M\nzKxKOcmbmZlVqXaVDsA+fBqWrqB25H2VDsOsVS0Z/ZlKh2AfQm7Jm5mZVSkneTMzsyrlJG9mZlal\nnOTNzMyqVJtN8pJWFiyfKWlcen2xpAWSnpT0sKTdcvV6SfqjpEWS/ibpu5JUZP+nSHo4tzxIUr2k\ndmn5OEl/lbQwld8p6WNp3URJz6fyhZK+38znfpKk/Taw/suS5ktqkDRX0iVF4poj6dBUPl1S/9z2\ntZLmF9nvkZLuLXHMm5qI6UxJH92Y8zQzs5ZVVpKXVCNpiKSvpwR7saSLWzq4DZgL9I+I/YHJwJgU\nZ0dgCjA6IvYG+gIDgfMLdxARdwNrJA2V1B64Djg/ItZK6g1cAwyLiH0iog6YBNTmdjEildcBwyTt\n3ozndxJQNKFKOh64EDg2IvoAA4AVReIaCdzQXAFFxNkRsWADVc4EnOTNzNqQclvy/032R7w70Dn3\nUxERMS0iVqXFJ4Ce6fVQ4PGIeDDVWwUMJ0t4xQwHLgcuA2ZGxIxU/i3gxxHxdO6YUyLi0SL76JB+\nvwUgaXBqXTdImiBpmybKR+d6Ja6QNBAYAoxNLfI9C473beCSiHgpxbUmIm4sEtejwMdLnPeGdJI0\nOfVQTGrsBWnsDUg3fBNzPQkXSToV6A9MSjF33ITjmplZMyv3OfmeqdXcmjpKqs8tdyNrpRc6C3gg\nve4FzM6vjIjFkjpJ6hIRbxSse07SnWTJPp9MewFXNBHfWEmXkiXSqyNimaQOwERgcEQ8I+lW4DxJ\n15covw04GdgnIkLS9hHxuqQpwL0RMbnIcXsXnmMJnwUayqhX6ACy838JeBw4DPhTbn0dsGtE9AbI\nxTyc7OZj1iYc08zMWkC5LfkHJB3bopF80OqIqGv8Ab5XWEHSGWQtyLGbcgBJNcAxwEpgtxJ1uqfW\n6TONY99JY7f4LsDg1ALfG3g+Ip5JdW4BjthA+QrgbeBmSacAq9h8Y9PN0TlkN0AAUaResTKAv0bE\nixHxHlDP+kMUAM8Be0i6RtJxwBuFOyhG0jmSZkma9e6qFU1vYGZmm63cJP8E8FtJqyW9IelNSWX9\ncW8pko4GRgFDImJNKl4AHFhQbw9gZWErPjmfrLV7FnBtboLeU0A/gIhYnpL5eKBT4Q4iYiUwHRi0\nsecQEWuBg8nmFZwATC1js6coOMcCI9KN0TER0Ti5bjnwkVydbsCrJbZfk3v9LgW9PRHxv2RzHaYD\n5wI3lREzETE+IvpHRP+abbuWs4mZmW2mcpP8lcChwLYR0SUiOkdElxaMa4MkHUA2qWxIRCzLrZoE\nDEo3AI0T8a4mTcwr2McuwMXANyNiKrAUODutHgOMkrRvbpNtS8TSDjgEWAwsAmolNY6Ffwl4pFS5\npE5A14i4H7iILHkCvEnpOQ8/IWut75KOv7Wks0vUbTQdOCN3EzMMmNbENkVJ2gHYKiJ+A1xKuhlq\nImYzM6uAcsfk/wHMj4hSXbytbSxZq/qulLdeiIghEbFa0onANZKuBWqA24BxRfZxJTAmIl5JyxcC\nj0n6TUQ0SLoAuFVSF7JW7wtA/lG5xjH5rYGHgbvTuPpXUlztgJnA9RGxplg5WYv6d2ksX2Q3HQB3\nADdK+jpwakQsbjxoRNwvaWfgDylpBzChies1HtgHmCcpgFlkE/g2xa7Af0pqvEFs3M9E4HpJq4FD\nI2L1Ju7fzMyaicrJ25ImAnuQTXBb150bEVe2WGRWtbbpsVf0GHZVpcMwa1X+ghrbXJJmR0T/pmu+\nr9yW/PPpZ+v0Y2ZmZm1cWUk+In7Q0oGYmZlZ8yoryUv6BHAJ2eNU67aJiE+1TFhmZma2ucrtrr+L\nbKLYTWSPVZmZmVkbV26SXxsRv2zRSOxDo8+uXZnlSUhmZi1ug0leUrf08r8lnQ/8lvVn17/WgrGZ\nmZnZZmiqJT+b7Dnsxg9RGZFbF2SP1ZmZmVkbtMEkHxG7A0jqEBFv59elD3AxMzOzNqrcj7WdUWaZ\nmZmZtRFNjcnvQvYxph3T58U3dtt3ocRnuZuZmVnb0NSY/KeBM4GeZJ/13uhN4DstFJOZmZk1g6bG\n5G8BbpH0ufStY2ZmZraFaKq7/oyI+BXZ16ReXLjeX1BjZmbWdjXVXb9d+t2ppQMxMzOz5tVUd/0N\nkmqANyLi560Uk5mZmTWDJh+hi4h3gS+2QixmZmbWjMr97PrHJY0D7gTeaiyMiDktEpWZmZlttnKT\nfF36/cNcWQD+qlkzM7M2qqwkHxFHtXQgZmZm1rzK+lhbSV0lXSlpVvr5maSuLR2cmZmZbbpyP7t+\nAtmn3H0+/bwB/GdLBWVmZmabr9wx+T0j4nO55R9Iqm+JgMzMzKx5lJvkV0saFBF/ApB0GLC65cKy\natawdAW1I++rdBi2BVky+jOVDsFsi1Rukj+P7DPsu5J9E91rZF9cY2ZmZm1UubPr64G+krqk5Tda\nNCozMzPbbGUl+cIvp5EEsAKYnW4AzMzMrI0pd3Z9f+BcYNf08/+A44AbJX2zhWIzMzOzzVDumHxP\noF9ErASQ9H3gPuAIYDYwpmXCMzMzs01Vbkt+J2BNbvkdYOeIWF1Q3iRJKwuWz0yfi4+kiyUtkPSk\npIcl7Zar10vSHyUtkvQ3Sd9VGjco2N8pkh7OLQ+SVC+pXVo+TtJfJS1M5XdK+lhaN1HS86l8YbqZ\naTaSTpK03wbWn5HO/SlJ8yTdJGn7tG56Ovd5kh6XtHcqXyJph9w+jpR0b3PGXY707/jR1j6umZmV\nVm6SnwT8RdL3U+J7HPi1pO2ABc0Yz1ygf0TsD0wm9RBI6ghMAUZHxN5AX2AgcH7hDiLibmCNpKGS\n2gPXAedHxFpJvYFrgGERsU9E1KVzq83tYkQqrwOGSdq9Gc/vJKBokpd0HHARcHxE9AL6ATOAnXPV\nTo+IvsAtwNhmjKs5nAk4yZuZtSFlJfmI+BFwDvB6+jk3In4YEW9FxOnNFUxETIuIVWnxCbJhAoCh\nwOMR8WCqtwoYDowssavhwOXAZcDMiJiRyr8F/Dgins4dc0pEPFpkHx3S77cAJA2WNFdSg6QJkrZp\nonx0rlfiCkkDgSHA2NRTsGfB8UYBl0TE0hTXuxExISIWFYntUeDjJc69KEk1ksZKmpli+n+p/OTU\nayJJPSQ9I2kXSbWSHpM0J/0MzO3rW+l856XzPJVs3sakdG4dNyY2MzNrGRsck5fULbf4XPpZty4i\nXtuEY3Ys+LS8bmSt9EJnAQ+k173Ixv7XiYjFkjpJ6lL4SF9EPCfpTrJkn0+mvYArmohvrKRLyZLo\n1RGxTFIHYCIwOCKekXQrcJ6k60uU3wacDOwTESFp+4h4XdIU4N6ImFzkuL2Acr+697NAQ5l1G50F\nrIiIg9KNyOOSHoyI30r6HPBVssmU34+I/5G0LXBMRLwtaS/gdqC/pOOBE4FDImJV4/tA0nCym5RZ\nxQ4u6RyyG0Vquuy4kaGbmdmmaGri3Wyyr5RtHPuO9Fvp9R6bcMzVqTs825F0JlkrkFzZGansk5uw\nfyTVAMcAK4HdgFeL1OkOPAxsC4yPiMbkPyIiJkvqBDycWrBvAc9HxDOpzi1kSXFaifJxwNvAzWl8\nfKPGyCX1AW4DOgPfiYg706pJklYDS4CvpbL44B6Klh0L7J9a3QBdgb2A59O+5gNPRMTtaX17YJyk\nOuBd4BOp/GjgPxt7XMq90YuI8cB4gG167FUsPjMza2YbTPIRsW48OrXq9+L9buwWIelosq7rT0ZE\n46S+BWQz+fP19gBWlvhgnvPJWrqXAtdKOjQiAniKbKx7XkQsB+okXQJ0KtxBRKyUNB0YBPx+Y84h\njf8fDAwGTiXrUfhUE5s1xjYtIhpSbOOAfNf36UVaysuBj/D+jUw3itzUkN2YfS0iip1LT+A9YGdJ\nW0XEe2TzA14mm/+wFdlNi5mZbUHK/arZs4FHgKlk49xTge81dzCSDgBuAIZExLLcqknAoHQD0DgR\n72qKPLonaRfgYuCbETEVWAqcnVaPAUZJ2je3ybYlYmkHHAIsBhYBtZIax8G/RHY9ipanXoCuEXE/\nWbLsm9a/SdY6L+YnwBWSeubKyhnbnp6O29iDcQZZD0Oh35MNJbRPdT8habt0nhOALwJPk107yFr6\n/0wJ/0tATSp/CPhK6s7PD+ls6NzMzKwCyp1dfwFwEPD3iDgKOIDsE++a21iyVvVdaQLXFID0qN6J\nwKWSFpG10meSdYsXuhIYExGvpOULyRJ7t9RCvgC4VdnjaI8D+wK/zseQ5gw8mY5zd0S8DXwlxdVA\n1uq9vlQ5WbK7V9KTwJ94P3HeAYxIE/XWm3iXbgiuBh5IE/ZmkHWTN9WL8CPg45LmkT2d8CzwqyL1\nbiLrEZkjaT7ZzVQ74DvAY+nLhy4Gzk43QdeRPV0wD9iHNAEx3ThNAWal63RJ2v9E4HpPvDMzazuU\n9WI3UUmamSZs1ZNNuFoj6an0qJfZRtmmx17RY9hVlQ7DtiD+FjozkDQ7Ivo3XfN95X7i3YvKPpTl\nHuAhSf8L/H1jAzQzM7PWU+630J2cXl4maRrZeO3UFovKzMzMNlu5Lfl1IuKRlgjEzMzMmtdGJ3mz\nzdVn167M8hirmVmLK3d2vZmZmW1hnOTNzMyqlJO8mZlZlXKSNzMzq1JO8mZmZlXKSd7MzKxKOcmb\nmZlVKSd5MzOzKuUkb2ZmVqWc5M3MzKqUk7yZmVmVcpI3MzOrUk7yZmZmVcpJ3szMrEo5yZuZmVUp\nJ3kzM7Mq5SRvZmZWpdpVOgD78GlYuoLakfdVOoyqsGT0Zyodgpm1YW7Jm5mZVSkneTMzsyrlJG9m\nZlalnOTNzMyqlJO8mZlZlXKS3wySVhYsnylpXHp9saQFkp6U9LCk3XL1ekn6o6RFkv4m6buSVOIY\nB0uanurNkXSfpD5p3WWSlkqql7RQ0i8lbZXWTZS0SlLn3L6ukhSSdihynE6SbpC0WNLsdMxD0rp3\n0zHmS7pL0raSaiXNL9jHZZIu2fQramZmzclJvuXMBfpHxP7AZGAMgKSOwBRgdETsDfQFBgLnF+5A\n0s7AfwHfiYi9IqIf8BNgz1y1n0dEHbAf0Af4ZG7ds8CJaV9bAZ8ClpaI9ybgNWCviDgQ+ArQeDOw\nOiLqIqI38H/AuRtzIczMrDKc5FtIREyLiFVp8QmgZ3o9FHg8Ih5M9VYBw4GRRXYzHLglImbk9vun\niLinSN2tgQ7A/+bK7gBOS6+PBB4H1hZuKGlP4BDg0oh4Lx3n+Ygo9jD7Y8DHi5SbmVkb4yS/eTqm\nbux6SfXAD0vUOwt4IL3uBczOr4yIxUAnSV0KtusFzGkihovSsf8JPBMR9bl1zwA7SvoI8EWypF9M\nL6A+It7d0IEktQOOBxqaiKnYtudImiVp1rurVmzs5mZmtgmc5DdPYzd2Xeoy/15hBUlnAP2BsZt7\nMEl/kfS0pF/kihu763cCtpP0hYLN7ga+QNZSf2wTD90x3UjMAl4AbgaiRN2i5RExPiL6R0T/mm27\nbmIYZma2Mfyxti1I0tHAKOCTEbEmFS8AjiiotwewMiLeKNjFU0A/4HcAEXGIpFOBEwqPFRHvSJqa\n9p1vsd9J1nNwS0S8V2J+31NAX0k1JVrzq9ONRD7m5cBHCup1A54vdgAzM2t9bsm3EEkHADcAQyJi\nWW7VJGBQugFonIh3NWliXoFrgTMlDcyVbVvieAIOAxbnyyPi72Q3GteVijUNF8wCftA4yz/Nni/5\nwegRsRL4p6RPpfrdgOOAP5XaxszMWpeTfMsZC3QC7kpj9lMAImI12Yz3SyUtIhvfngmMK9xBRPwP\n2cS5n0h6VtIM4NSCuo1j8vOBGook84i4ISXyDTkb2Bl4Nj0aNxFYtsEt4MvAd9Px/wj8oIzjmJlZ\nK1FEqaFVs5axTY+9osewqyodRlXwt9CZfXhImh0R/TdmG7fkzczMqpSTvJmZWZVykjczM6tSTvJm\nZmZVys/JW6vrs2tXZnnCmJlZi3NL3szMrEo5yZuZmVUpJ3kzM7Mq5SRvZmZWpZzkzczMqpSTvJmZ\nWZVykjczM6tSTvJmZmZVyknezMysSjnJm5mZVSkneTMzsyrlJG9mZlalnOTNzMyqlJO8mZlZlXKS\nNzMzq1JO8mZmZlXKSd7MzKxKtat0APbh07B0BbUj72u14y0Z/ZlWO5aZWVvilryZmVmVcpI3MzOr\nUk7yZmZmVcpj8mZmZXrnnXd48cUXefvttysdilWxDh060LNnT9q3b7/Z+2qzSV7SyojolFs+E+gf\nEcMlXQycDawFXgH+PSL+nur1Aq4BdiXrqbgVuDwiomD/pwBfjYjBaXkQMC4dY62k44AfAl2At4FF\nwIiIeEHSROCTwAqgA3B7RPygGc/9JOCZiFhQYv0ZwDeBmnQNZgKXRMTrkqYDPVLMK8muzSJJS9K5\nvZr2cWTa5oSCfZ+Z6g0vctz7gaER8XqJuC4ExkfEqo0+abMtwIsvvkjnzp2pra1FUqXDsSoUESxf\nvpwXX3yR3XfffbP3t6V2188lS0T7A5OBMQCSOgJTgNERsTfQFxgInF+4g4i4G1gjaaik9sB1wPkp\nwfcmu1H2A7OWAAAQbUlEQVQYFhH7REQdMAmoze1iRCqvA4ZJ2vx/jfedBOxXbEW6+bgIOD4iegH9\ngBnAzrlqp0dEX+AWYGxzBRUR/1oqwScXAts21/HM2pq3336b7t27O8Fbi5FE9+7dm623aItM8hEx\nLddafALomV4PBR6PiAdTvVXAcGBkiV0NBy4HLgNmRsSMVP4t4McR8XTumFMi4tEi++iQfr8FIGmw\npLmSGiRNkLRNE+WjJS2Q9KSkKyQNBIYAYyXVS9qz4HijyFrgS1Nc70bEhIhYVCS2R4GPlzj3Dfmo\npKmS/iZpTGOhpCWSdpC0naT7JM2TNF/SaZK+DnwUmCZp2iYc02yL4ARvLa0532NtOcl3TEmuXlI9\nWdd5MWcBD6TXvYDZ+ZURsRjoJKlL4YYR8RxwJ1my/1ZuVS9gThPxjU1xvQjcERHLJHUAJgKnRUQf\nsuGQ8zZQ3h04GeiVeiUuTzcaU0g9BSn+vHJia/RZoKHMunl1wGlAH+A0Sf9SsP444KWI6BsRvYGp\nEXE18BJwVEQctQnHNLMy1NTUUFdXR69evejbty8/+9nPeO+99wCYPn06krjpppvW1a+vr0cSV1xx\nxQb3e88997BgQdERQgCuv/56br311uY5iQJLlizh17/+9SZtO3DgwCbrnH322Rs8t3K9/vrrXHfd\ndeuWX3rpJU499dTN3m9LarNj8sDq1B0OvD9WnK+Qxqb7k42PbzRJNcAxZGPXuwGvFqnTHXiYrBt6\nfEQ0/k8ZERGTJXUCHk4t8LeA5yPimVTnFuCrwLQS5ePIxs5vlnQvcO9Gxt8HuA3oDHwnIu5MqyZJ\nWg0sAb6WyuKDeyhaBvBwRKxIx1hAdm3+kVvfAPxM0k+BeyPisTJiPQc4B6Cmy45NVTfbIjT3hzqV\n88FNHTt2pL6+HoBly5YxdOhQ3njjDX7wg2xaUO/evfmv//ovzj77bABuv/12+vbt2+R+77nnHk44\n4QT22++DI4Vr167l3HPP3ZhTWU9tbS1Lliwpub4xyQ8dOrTosdu1K52qZsyYUXJdo/xNz+ZoTPLn\nn5+NAH/0ox9l8uTJzbLvltKWW/IbJOlosq7rIRGxJhUvAA4sqLcHsDIi3iiym/PJEtZZwLV6v4/k\nKbKxbiJiebrZGA90KtxBRKwEpgODNvYcImItcDDZvIITgKllbJaPrSHF9gDQMVfn9NQLcFJENCbn\n5cBHcnW6UeSmJlmTe/0uBTeD6WalH9m1u1zS95oKOiLGR0T/iOhfs23XpqqbWRl22mknxo8fz7hx\n42icW7zbbrvx9ttv8/LLLxMRTJ06leOPP37dNjfeeCMHHXQQffv25XOf+xyrVq1ixowZTJkyhREj\nRlBXV8fixYs58sgjufDCC+nfvz+/+MUvuOyyy7jiiitYu3YtBx10ENOnTwfg29/+NqNGjdqs8xg5\nciSPPfYYdXV1/PznP2fixIkMGTKET33qUwwePJiVK1cyePBg+vXrR58+ffjd7363bttOnbI/y9On\nT+fII4/k1FNPZZ999uH0009fd02OPPJIZs2ata7+qFGj6Nu3LwMGDODll18GYPHixQwYMIA+ffpw\n6aWXrttvYZyLFy+mrq6OESNGsGTJEnr37g3AxIkTOemkkzjmmGOora1l3LhxXHnllRxwwAEMGDCA\n1157bd1xjjvuOA488EAOP/xwFi5cuFnXrilbZJKXdABwA1mCX5ZbNQkYlG4AGifiXU2amFewj12A\ni4FvRsRUYCnZjH1S/VGS9s1tUnRCmaR2wCHAYrIZ+LWSGsfBvwQ8Uqo89QJ0jYj7ySbTNd5uv0nW\nOi/mJ8AVknrmyjqWqJs3PR23sQfjDLIeho0m6aPAqoj4FdnEvn5p1YbiNrMWsMcee/Duu++ybNn7\nfwpPPfVU7rrrLmbMmEG/fv3YZptt1q075ZRTmDlzJvPmzWPffffl5ptvZuDAgQwZMoSxY8dSX1/P\nnntmU4H+7//+j1mzZvGNb3xj3fbt2rVj4sSJnHfeefzhD39g6tSpfP/739+scxg9ejSHH3449fX1\nXHTRRQDMmTOHyZMn88gjj9ChQwd++9vfMmfOHKZNm8Y3vvGNdQk8b+7cuVx11VUsWLCA5557jscf\nf/wDdd566y0GDBjAvHnzOOKII7jxxhsBuOCCC7jgggtoaGigZ8+eH9iuMc4999yT+vp6xo794Jzm\n+fPnc/fddzNz5kxGjRrFtttuy9y5czn00EPXDXWcc845XHPNNcyePZsrrrhiXa9AS2nL3fUbMpas\nVX1Xany/EBFDImK1pBOBayRdS/aI2W1k3eKFrgTGRMQraflC4DFJv4mIBkkXALemsfxXgReA/Dt5\nrKRLga3JuvPvjoiQ9JUUVzuyR9uuj4g1xcrJWtO/S2P2IrvpALgDuDFNZjs1Py4fEfdL2hF4ICXr\n14H5wO+buGY/An4paV461lTgV01sU0qfdP7vAe8A56Xy8cBUSS95XN6scj7/+c9z2mmnsXDhQr74\nxS+u16U9f/58Lr30Ul5//XVWrlzJpz/96ZL7Oe2004qW9+rViy996UuccMIJ/PnPf2brrbf+QJ3/\n+I//4K677gKyseu6umz09bDDDuPaa69t8hyOOeYYunXrBmSPlX3nO9/h0UcfZauttmLp0qW8/PLL\n7LLLLuttc/DBB69L0HV1dSxZsoRBg9bvZN1666054YTsyeEDDzyQhx56CIA///nP3HPPPQAMHTqU\nSy65pMkYCx111FF07tyZzp0707VrVz772c8C0KdPH5588klWrlzJjBkz+Ld/+7d126xZs6bU7ppF\nm03y+Wfk0/JEsslrRMTRG9iuATiyjP0PLVj+B7lH5CLiPqDogFtEnLmB/T4MHFBm+T/JuusL6z5O\niUfo0vpbyMb1i607skT5CrKnDzYof53T8gm517Xp5e8pclMREdeQPXpoZq3kueeeo6amhp122omn\nn84eCNpll11o3749Dz30EL/4xS/WS/Jnnnkm99xzD3379mXixInrut2L2W677Uqua2hoYPvtt1+v\nByFv1KhR67rxa2tr180jKFf+2JMmTeKVV15h9uzZtG/fntra2qKPmOV7LGpqali7du0H6rRv337d\n7PVSdTZV/vhbbbXVuuWtttqKtWvX8t5777H99ttv9LXYHFtkd72ZmcErr7zCueeey/Dhwz/w2NUP\nf/hDfvrTn1JTU7Ne+ZtvvkmPHj145513mDRp0rryzp078+abb5Z13LvvvpvXXnuNRx99lK997Wu8\n/vqGPj6jaU0de8WKFey00060b9+eadOm8fe//32zjlfMgAED+M1vfgPAHXfcsUlxNqVLly7svvvu\n63o4IoJ58+Zt8v7K4SRvZrYFWb169bpH6I4++miOPfbYomPiAwcO5KSTTvpA+Y9+9CMOOeQQDjvs\nMPbZZ5915V/4whcYO3YsBxxwAIsXFz65+75XX32VkSNHctNNN/GJT3yC4cOHc8EFF2zWOe2///7U\n1NTQt29ffv7zn39g/emnn86sWbPo06cPt95663pxN5errrqKK6+8kv33359nn32Wrl0/OEG4e/fu\nHHbYYfTu3ZsRI0Zs0nEmTZrEzTffTN++fenVq9d6kwhbgopNXjBrSdv02Ct6DLuq1Y7n75O35vL0\n00+z7777Nl3RtjirVq2iY8eOSOKOO+7g9ttvb/EEvCHF3muSZkdE/xKbFNVmx+TNzMxay+zZsxk+\nfDgRwfbbb8+ECRMqHVKzcJK3Vtdn167McuvazNqQww8/vMXHxyvBY/JmZmZVyknezGwjeB6TtbTm\nfI85yZuZlalDhw4sX77cid5aTOP3yXfo0KHpymXwmLyZWZl69uzJiy++yCuvvNJ0ZbNN1KFDh5If\nrbuxnOTNzMrUvn17dt9990qHYVY2d9ebmZlVKSd5MzOzKuUkb2ZmVqX8sbbW6iS9CSyqdBxN2IHs\nK4bbui0hzi0hRtgy4twSYoQtI84tIUZYP87dImLHjdnYE++sEhZt7OcvtzZJs9p6jLBlxLklxAhb\nRpxbQoywZcS5JcQImx+nu+vNzMyqlJO8mZlZlXKSt0oYX+kAyrAlxAhbRpxbQoywZcS5JcQIW0ac\nW0KMsJlxeuKdmZlZlXJL3szMrEo5yVurkXScpEWSnpU0stLxNJL0L5KmSVog6SlJF6TyyyQtlVSf\nfv61wnEukdSQYpmVyrpJekjS39Lvj1Q4xr1z16te0huSLqz0tZQ0QdIySfNzZUWvnTJXp/fpk5L6\nVTjOsZIWplh+K2n7VF4raXXuml5fwRhL/vtK+na6loskfbo1YtxAnHfmYlwiqT6VV+palvrb03zv\nzYjwj39a/AeoARYDewBbA/OA/SodV4qtB9Avve4MPAPsB1wGXFLp+HJxLgF2KCgbA4xMr0cCP610\nnAX/5v8D7FbpawkcAfQD5jd17YB/BR4ABAwA/lLhOI8F2qXXP83FWZuvV+EYi/77pv9H84BtgN3T\n34CaSsVZsP5nwPcqfC1L/e1ptvemW/LWWg4Gno2I5yLi/4A7gBMrHBMAEfHPiJiTXr8JPA3sWtmo\nynYicEt6fQtwUgVjKTQYWBwRf690IBHxKPBaQXGpa3cicGtkngC2l9SjUnFGxIMRsTYtPgE0z9eT\nbaIS17KUE4E7ImJNRDwPPEv2t6DFbShOSQI+D9zeGrGUsoG/Pc323nSSt9ayK/CP3PKLtMFEKqkW\nOAD4SyoanrrFJlS6KxwI4EFJsyWdk8p2joh/ptf/A+xcmdCK+gLr/xFtS9cSSl+7tvxe/Xeyllyj\n3SXNlfSIpMMrFVRS7N+3rV7Lw4GXI+JvubKKXsuCvz3N9t50kjdLJHUCfgNcGBFvAL8E9gTqgH+S\nde9V0qCI6AccD3xV0hH5lZH157WJx2UkbQ0MAe5KRW3tWq6nLV27UiSNAtYCk1LRP4GPRcQBwMXA\nryV1qVB4bfrft4gvsv4NaEWvZZG/Pets7nvTSd5ay1LgX3LLPVNZmyCpPdl/skkRcTdARLwcEe9G\nxHvAjbRSN2MpEbE0/V4G/DbF83Jjd136vaxyEa7neGBORLwMbe9aJqWuXZt7r0o6EzgBOD390Sd1\ngS9Pr2eTjXd/ohLxbeDfty1ey3bAKcCdjWWVvJbF/vbQjO9NJ3lrLTOBvSTtnlp5XwCmVDgmYN34\n3M3A0xFxZa48P9Z1MjC/cNvWImk7SZ0bX5NNxppPdg2HpWrDgN9VJsIPWK+l1JauZU6pazcF+HKa\nyTwAWJHrOm11ko4DvgkMiYhVufIdJdWk13sAewHPVSjGUv++U4AvSNpG0u5kMf61teMrcDSwMCJe\nbCyo1LUs9beH5nxvtvZsQv98eH/IZoY+Q3aXPKrS8eTiGkTWHfYkUJ9+/hW4DWhI5VOAHhWMcQ+y\nWcrzgKcarx/QHXgY+BvwB6BbG7ie2wHLga65sopeS7Ibjn8C75CNY55V6tqRzVy+Nr1PG4D+FY7z\nWbJx2Mb35vWp7ufSe6EemAN8toIxlvz3BUala7kIOL6S1zKVTwTOLahbqWtZ6m9Ps703/Yl3ZmZm\nVcrd9WZmZlXKSd7MzKxKOcmbmZlVKSd5MzOzKuUkb2ZmVqWc5M3MzKqUk7yZmVmVcpI3MzOrUv8/\n7VjI9nE2858AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2ab4a1bf70b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAEICAYAAACtc9bVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8XVV97v/PYyKQGBK5CQiVDYIg4RIwSqUcb4DFquEi\nv6oYDR44HEmpIgVFQUXkJ2lCreViAZUGMBUOkdYokCKBCEJREgiERIIGEI0XBDUYEnIAn/PHHFsm\ni7WzV3b2jcnzfr3WK3OOMeac37kS+M5xWWvJNhEREdEMLxnqACIiIqL/JLFHREQ0SBJ7REREgySx\nR0RENEgSe0RERIMksUdERDRIEntEvGBJulDSZ/q7bcQLmfI59ogYCpIeAo61fcNQx9IfJI0Bfg3c\nYvsdLXUGdrH901rZGcDOtieX/bHAmcARwObAb4DvAGfZfnRQbiIaIT32iBiWJI0c6hjW03uAtcDB\nkrZZnwMlbQTMA8YDhwBjgTcCjwFv6Oc4o+GS2CNi0Em6HHgV8B1JqyR9QlKXJEs6RtLDwI2l7VWS\nfi1ppaSbJY2vnWempLPK9lsk/ULSP0h6RNKvJH24j223kPQdSY9LukPSWZJ+0MttTQEuBO4BJq/n\nW/Kh8n4cbnup7T/ZfsT2F2xfu57nihe5JPaIGHS2Pwg8DLzb9hjb02vVbwZeC/x12b8O2AV4BXAn\nMGsdp94GGAdsBxwDXCBpsz60vQB4orSZUl49krQD8JYS2yyqRL0+DgLm2l61nsdFPE8Se0QMN2fY\nfsL2GgDbl9j+o+21wBnA3pLG9XDsU8CZtp8qPd1VwK7r01bSCKph9c/ZXm17KXBpLzF/ELintL0C\nGC9pn85vmS2AX61H+4geJbFHxHDz8+4NSSMkTZO0XNLjwEOlassejn3M9tO1/dXAmPVsuxUwsh5H\ny3Y7H6KMJNheAXyf5/bynwFe2nLMS6keLqCaS9+2l2tEdCSJPSKGSk8fyamXHwUcSjVUPQ7oKuUa\nuLD4LfA0sH2t7C96aixpf6qpgk+VtQC/BvYDjqotAHyYZ2PvtiPws7J9A/DXkl624eHHi10Se0QM\nld8AO/XSZlOqleaPAaOBLw50ULafAa4GzpA0WtJurHvOfArwPWB3YEJ57QGMAro/9nYlcLqk7SW9\nRNJBwLuB2aX+cqpRgW9J2q202ULSpyX9TX/fYzRbEntEDJWzqZLdHySd3EOby6h6tSuApcDtgxTb\nCVQjBL+mSrrfpHrAeA5JmwB/C5xn+9e114PluO7h+DOB24AfAL8HpgMfsH0vQFk/cBBwH9VDwuPA\nj6imHH44UDcZzZQvqImI6IWkfwS2sb3O1fERw0F67BERLcpw+F6qvIHq43D/MdRxRXTihfbNThER\ng2FTquH3V1KtBfgn4NtDGlFEhzIUHxER0SAZio+IiGiQDMXHoNtyyy3d1dU11GFERLygLFy48FHb\nW/XWLok9Bl1XVxcLFiwY6jAiIl5QJP2s91YZio+IiGiUJPaIiIgGSWKPiIhokCT2iIiIBklij4iI\naJAk9oiIiAZJYo+IiGiQfI49Bt3iFSvpOvWaoQ4jImJQPTTtnYNynfTYIyIiGiSJPSIiokGS2CMi\nIhokiT0iIqJBktgjIiIaZEgSu6RVLftHSzq/bJ8kaamkeyTNk7RDrd14STdKWibpJ5I+I0ltzn+E\npHm1/QMkLZI0suwfIulHku4r5VdKelWpmynpwVJ+n6TP9fO9HyZp93XUf0jSvZIWS7pL0slt4rpT\n0htL+XxJE2vHd0m6tz9j7kRv9xUREYNjOPbY7wIm2t4LmA1MB5A0CpgDTLO9K7A3sD8wtfUEtq8G\n1ko6StJLga8AU20/LWkP4Dxgiu3dbE8AZgFdtVOcUsonAFMk7diP93cY0DYBSnoHcCLwdtt7An8J\nrGwT16nARf0YU3/o8b4iImLwDLvEbvsm26vL7u3A9mX7KOBW29eXdquBE6iSXDsnAGcBZwB32L6t\nlH8S+KLtH9euOcf2zW3OsUn58wkASQeWXvRiSZdI2riX8mm10YdzJO0PTAJmlJ73q1uu9yngZNu/\nLHGttf3VNnHdDOzcw333SNIpku4o8Xy+lL2+7G8i6WWSlkjaQ9KYMmJyZ7mvQ2vn+VA55m5Jl3dw\nXxERMUiG6gtqRklaVNvfnKo33uoY4LqyPR5YWK+0vbwkoLG2H2+pe0DSlVQJvp5oxgPn9BLfDEmn\nUyXPc20/ImkTYCZwoO37JV0GHC/pwh7KLwcOB3azbUkvt/0HSXOA79qe3ea6e7TeYw/eDSzuoN2f\nSXo7sAvwBkDAHElvsn1zieksYBTwDdv3lmmLw20/LmlL4PbSbnfgdGB/249K2tz273q5LyQdBxwH\nMGLsVusTekRErIeh6rGvsT2h+wV8trWBpMnARGBGXy4gaQRwMLAK2KGHNluUHub93XPZRfeQ9zbA\ngaVHuivwoO37S5tLgTeto3wl8CTwdUlHAKvZcDPKA9FxVA89AG7Trl3Z28vrLuBOYDeqRA9wJtV7\nNZEy9UGV/L8o6R7gBmA7YGvgbcBVth8FsP27TgK3fbHtibYnjhg9rpNDIiKiD4bdUDyApIOA04BJ\ntteW4qXA61ra7QSsau2tF1OperXHABfUFtktAfYFsP1YSeAXA2NaT2B7FTAfOGB978H201S949nA\nu4C5HRy2hJZ7bHFKeRg62Hb3ArnHgM1qbTYHHm1zrICzaw9UO9v+eqnbgur+N+XZ6YcPAFsBryvv\n0W9qdRERMUwNu8QuaR+qhWGTbD9Sq5oFHFCSfvdiunN5todZP8c2wEnAJ2zPBVYAx5bq6cBpkl5b\nO2R0D7GMBPYDlgPLgC5J3XPbHwS+31O5pDHAONvXAh+nWuwH8EeqBNrO2VS98m3K9TeSdGwPbbvN\nBybXHlymADe1afdfwP8scSFpO0mvKHUXAZ+heo//sZSNAx6x/ZSkt/LsqMeNwP8naYtyns07uK+I\niBgkw/FHYGZQ9R6vKrnqYduTbK8pC7jOk3QBMAK4HDi/zTm+BEy3/duyfyJwi6Rv2V4s6WPAZZLG\nUvVuHwbqH2vrnmPfCJgHXF3myT9c4hoJ3AFcaHttu3KqnvO3y9y8qB40AK4Avirpo8CRtpd3X9T2\ntZK2Bm4oidrAJb28XxdTDavfLcnAAqpFeM9h+/ryMPPf5X1dRfVAcAjwlO1/L9MXt0l6G1WS/46k\nxeWc95XzLJH0/1M9vDxDNbR/9LruKyIiBo/sdtOxEQNn42138bZTvjzUYUREDKoN/XU3SQttT+yt\n3bAbio+IiIi+S2KPiIhokCT2iIiIBklij4iIaJDhuCo+Gm7P7caxYAMXkURERHvpsUdERDRIEntE\nRESDJLFHREQ0SBJ7REREgySxR0RENEgSe0RERIMksUdERDRIEntERESDJLFHREQ0SBJ7REREgySx\nR0RENEgSe0RERIMksUdERDRIEntERESDJLFHREQ0SBJ7REREgySxR0RENMjIoQ4gXnwWr1hJ16nX\nDHUYERGD5qFp7xy0a6XHHhER0SBJ7BEREQ2SxB4REdEgSewRERENksQeERHRIAOW2CWtatk/WtL5\nZfskSUsl3SNpnqQdau3GS7pR0jJJP5H0GUlqc/4jJM2r7R8gaZGkkWX/EEk/knRfKb9S0qtK3UxJ\nD5by+yR9rp/v/TBJu6+j/kOS7pW0WNJdkk5uE9edkt5YyudLmlg7vkvSvf0Zc0RENMNQ9djvAiba\n3guYDUwHkDQKmANMs70rsDewPzC19QS2rwbWSjpK0kuBrwBTbT8taQ/gPGCK7d1sTwBmAV21U5xS\nyicAUyTt2I/3dxjQNrFLegdwIvB223sCfwmsbBPXqcBF/RhTn3U/LEVExPA3JInd9k22V5fd24Ht\ny/ZRwK22ry/tVgMnUCW5dk4AzgLOAO6wfVsp/yTwRds/rl1zju2b25xjk/LnEwCSDiy96MWSLpG0\ncS/l02qjD+dI2h+YBMwoPe9Xt1zvU8DJtn9Z4lpr+6tt4roZ2LmH+25L0pgyAnJnifPQUt5VRiZm\nSfqxpNmSRpe6hyRNL+1/JGnnUj5T0oWSfghMl7S5pP8s93m7pL1Ku3+R9Nmy/deSbpaUKZ6IiCEy\nkP8DHlUS2yJJi4Aze2h3DHBd2R4PLKxX2l4OjJE0tvVA2w8AV1Il+E/WqsYDd/YS34wS1y+AK2w/\nImkTYCbw3tKbHgkcv47yLYDDgfFl9OGs8nAxh9LzLvHX7dF6jz14N7C4g3Z1TwKH294XeCvwT7Vp\njF2Br9h+LfA4zx0FWVnu63zgy7Xy7YH9bZ8EfB64q9znp4HLSptPAe+V9FbgXODDtv/UGpik4yQt\nkLTgmdUrW6sjIqKfDGRiX1MS24QytPzZ1gaSJgMTgRl9uYCkEcDBwCpghx7abFEeLu7vnssuuoe8\ntwEOLD3tXYEHbd9f2lwKvGkd5SupkunXJR0BrGbDdT9wHEf10APgNu3alQn4oqR7gBuA7YCtS93P\nbd9atr8BHFA77pu1P99YK7/K9jNl+wDgcgDbNwJbSBpbRlX+F/A94Pw2DzKUYy62PdH2xBGjx7Vr\nEhER/WDIhkwlHQScBkyyvbYULwVe19JuJ2CV7cfbnGYqVa/2GOCCWu90CbAvgO3HSgK/GBjTegLb\nq4D5PDfRdcT208AbqNYJvAuY28FhS2i5xxbdPf2DbXcvkHsM2KzWZnPg0TbHfgDYCnhdueff8OxU\nQ+uDgDvYfmIdcdbtWWJ8ZYftIyJigAxJYpe0D9XCsEm2H6lVzQIOKEm/ezHduZTFdS3n2AY4CfiE\n7bnACuDYUj0dOE3Sa2uHjO4hlpHAfsByYBnQ1T3PDHwQ+H5P5ZLGAONsXwt8nGqxH8AfgU17uP2z\nqXrl25TrbyTp2B7adpsPTK49uEwBbmrTbhzwiO2nytB4fRTjVd2r7KnWMvygVvfe2p//3UMMt1A9\nOCDpLcCjth9X9YmGfwD2Ad4hab9e7iUiIgbQUK12nkHVe76q5KqHbU+yvaYs+DpP0gXACKrh3/Pb\nnONLwHTbvy37JwK3SPqW7cWSPgZcVubmHwUeBuofa5sh6XRgI2AecLVtS/pwiWskcAdwoe217cqp\nes7fLnPwonrQALgC+KqkjwJH1oenbV8raWvghpKoDVzSy/t1MbAbcLckAwuo5rZbzQK+I2lxaXNf\nrW4Z8HeSLqEaGfnXWt1mZfh+LfD+HmI4A7iktFtN9UkCAV+nLAaUdAwwU9LrbT/Zyz1FRMQAkN1u\nqjaaRFIX8F3be7Spe4jqo4fthvYHxMbb7uJtp3y594YREQ3RH7/uJmmh7Ym9tcvHkiIiIhokXzzy\nImD7IaqP2bWr6xrUYCIiYkClxx4REdEg6bHHoNtzu3Es6If5poiIeL702CMiIhokiT0iIqJBktgj\nIiIaJIk9IiKiQZLYIyIiGiSJPSIiokGS2CMiIhokiT0iIqJBktgjIiIaJIk9IiKiQZLYIyIiGiSJ\nPSIiokGS2CMiIhokiT0iIqJBktgjIiIaJIk9IiKiQZLYIyIiGmTkUAcQLz6LV6yk69RrhjqMiIhB\n89C0dw7atdJjj4iIaJAk9oiIiAZJYo+IiGiQJPaIiIgGSWKPiIhokI4Su6QRkiZJ+qikk7pfAxmY\npFUt+0dLOr9snyRpqaR7JM2TtEOt3XhJN0paJuknkj4jSW3Of4SkebX9AyQtkjSy7B8i6UeS7ivl\nV0p6VambKenBUn6fpM/1870fJmn3ddR/SNK9khZLukvSyW3iulPSG0v5fEkTa8d3Sbq3zXnfIum7\nPVzza73EdLSkV67PfUZERP/rtMf+HeBoYAtg09prqNwFTLS9FzAbmA4gaRQwB5hme1dgb2B/YGrr\nCWxfDayVdJSklwJfAabaflrSHsB5wBTbu9meAMwCumqnOKWUTwCmSNqxH+/vMKBtEpX0DuBE4O22\n9wT+EljZJq5TgYv6KyDbx9peuo4mRwNJ7BERQ6zTz7FvX5LosGD7ptru7cDksn0UcKvt60u71ZJO\nAOYDF7Q51QnADcB44A7bt5XyTwJftP3j2jXn9BDOJuXPJwAkHQicQ/Xe3gEcb3vtOsqnAZOAp4Hr\ngavL/pslnQ68x/by2vU+BZxs+5clrrXAV9vEdTOwcw8xr8sYSbOBPYCFwGTbljQfOJnqoerrwETA\nwCXAz8v+LElrgDfaXtOHa0dExAbqtMd+naS3D2gkzzeqDCkvkrQIOLOHdscA15Xt8VTJ6M9KUhwj\naWzrgbYfAK6kSvCfrFWNB+7sJb4ZJa5fAFfYfkTSJsBM4L2lNz0SOH4d5VsAhwPjy4PTWeXhYg6l\n592S1OHZhNubdwOLO2jXah+qEYHdgZ2Av2qpnwBsZ3uPci//Zns2sAD4QIn5eUld0nGSFkha8Mzq\nla3VERHRTzpN7LcD/yFpjaTHJf1R0uMDGRiwpiSJCWVo+bOtDSRNpuopzujLBSSNAA4GVgE79NBm\ni/JwcX/3XHbRPeS9DXCgpP2BXYEHbd9f2lwKvGkd5SuBJ4GvSzoCWN2X+2jR/cBxHNVDD1Q961bt\nygB+ZPsXtv8ELOK50w8ADwA7STpP0iFAR/8ObF9se6LtiSNGj+vkkIiI6INOE/uXgDcCo22Ptb2p\n7ef1gAeTpIOA04BJZTgaYCnwupZ2OwGrbLdLQFOperXHABfUFtktAfYFsP1YSeAXA2NaT2B7FdVQ\n/wHrew+2nwbeQLVO4F3A3A4OW0LLPbbo7ukfbLt7gdxjwGa1NpsDj/Zw/Nra9jO0TNfY/j3V2oX5\nwEeAr3UQc0REDJJOE/vPgXtt99TLG1SS9qFaGDbJ9iO1qlnAASXpdy+mO5eyuK7lHNsAJwGfsD0X\nWAEcW6qnA6dJem3tkNE9xDIS2A9YDiwDuiR1z21/EPh+T+WSxgDjbF8LfJwqYQL8kZ4XJ55N1Svf\nplx/I0nH9tC223xgcu3BZQpwU8/NeyZpS+Altr8FnE55AOol5oiIGCSdLp57AJgv6TpqPTrbXxqQ\nqHo3g6r3fFXJVQ/bnmR7jaRDgfMkXQCMAC4Hzm9zji8B023/tuyfCNwi6Vu2F0v6GHBZmZt/FHgY\nqH+sbUZZ3LYRMA+4uiwy+3CJq3uR3IVlkdzzyql6zt8uc/CietAAuAL4qqSPAkfW59ltXytpa+CG\nkqi7F7Cty8XAbsDdkkw1H/6pXo7pyXbAv0nqfijsPs9M4MIsnouIGFrqpBPe0+e0bX++3yOKxtt4\n21287ZQvD3UYERGDpj9+3U3SQtsTe2vXUY89CTwiIuKFoaPELuk1VJ9h7qofY/ttAxNWRERE9EWn\nc+xXUc0Jf41qpXREREQMQ50m9qdt/+uARhIvGntuN44F/TDfFBERz7fOxC5p87L5HUlTgf/guavi\nfzeAsUVERMR66q3HvpDq41Tdn38+pVZnqq8cjYiIiGFinYnd9o4Akjax/WS9rnz2OiIiIoaRTr95\n7rYOyyIiImII9TbHvg3VN42NKl/j2j0kP5YevmI1IiIihk5vc+x/DRwNbE/1Fazd/gh8eoBiioiI\niD7qbY79UuBSSe8pP/oRERERw1hvQ/GTbX+D6pfJTmqtH8IfgYmIiIg2ehuKf1n583m/Qx4RERHD\nT29D8RdJGgE8bvufBymmiIiI6KNeP+5m+xng/YMQS0RERGygTr8r/lZJ5wNXAk90F9q+c0CiioiI\niD7pNLFPKH+eWSszkJ9tjYiIGEY6Suy23zrQgURERMSG6+grZSWNk/QlSQvK658kjRvo4CIiImL9\ndPpd8ZdQfdvc35bX48C/DVRQERER0TedzrG/2vZ7avufl7RoIAKKiIiIvus0sa+RdIDtHwBI+itg\nzcCFFU22eMVKuk69ZqjDiIgYVA9Ne+egXKfTxH481XfGj6P6hbffUf04TERERAwjna6KXwTsLWls\n2X98QKOKiIiIPukosbf+AIwkgJXAwpL0IyIiYhjodFX8ROAjwHbl9b+BQ4CvSvrEAMUWERER66nT\nOfbtgX1trwKQ9DngGuBNwEJg+sCEFxEREeuj0x77K4C1tf2ngK1tr2kp75WkVS37R5fvoUfSSZKW\nSrpH0jxJO9TajZd0o6Rlkn4i6TMqcwIt5ztC0rza/gGSFkkaWfYPkfQjSfeV8islvarUzZT0YCm/\nrzzA9BtJh0nafR31k8u9L5F0t6SvSXp5qZtf7v1uSbdK2rWUPyRpy9o53iLpu/0ZdyfK3+MrB/u6\nERHxXJ0m9lnADyV9riS7W4F/l/QyYGk/xnMXMNH2XsBsykiApFHAHGCa7V2BvYH9gamtJ7B9NbBW\n0lGSXgp8BZhq+2lJewDnAVNs72Z7Qrm3rtopTinlE4Apknbsx/s7DGib2CUdAnwceIft8cC+wG3A\n1rVmH7C9N3ApMKMf4+oPRwNJ7BERQ6yjxG77C8BxwB/K6yO2z7T9hO0P9Fcwtm+yvbrs3k41BQBw\nFHCr7etLu9XACcCpPZzqBOAs4AzgDtu3lfJPAl+0/ePaNefYvrnNOTYpfz4BIOlASXdJWizpEkkb\n91I+rTb6cI6k/YFJwIwyIvDqluudBpxse0WJ6xnbl9he1ia2m4Gde7j3tiSNkDRD0h0lpv9dyg8v\noyOStK2k+yVtI6lL0i2S7iyv/Wvn+mS537vLfR5JtQ5jVrm3UesTW0RE9J91zrFL2ry2+0B5/bnO\n9u/6cM1RLd9atzlVb7zVMcB1ZXs81Vz+n9leLmmMpLGtH7+z/YCkK6kSfD2BjgfO6SW+GZJOp0qc\n59p+RNImwEzgQNv3S7oMOF7ShT2UXw4cDuxm25JebvsPkuYA37U9u811xwOd/gzuu4HFHbbtdgyw\n0vbry8PHrZKut/0fkt4D/B3VgsjP2f61pNHAwbaflLQL8E1goqR3AIcC+9le3f3vQNIJVA8mC9pd\nXNJxVA+HjBi71XqGHhERnept8dxCqp9n7Z7LdvlTZXunPlxzTRnqrk4kHU3V26NWNrmUvbkP50fS\nCOBgYBWwA/BomzZbAPOA0cDFtrsT/im2Z0saA8wrPdUngAdt31/aXEqVCG/qofx84Eng62W+e73m\nvCXtCVwObAp82vaVpWqWpDXAQ8DflzI//wxty94O7FV61wDjgF2AB8u57gVut/3NUv9S4HxJE4Bn\ngNeU8oOAf+seWen04c72xcDFABtvu0u7+CIioh+sM7Hb/vP8cum978KzQ9QDQtJBVMPSb7bdvTBv\nKdUK/Hq7nYBVPXxZzlSqHu3pwAWS3mjbwBKqueu7bT8GTJB0MjCm9QS2V0maDxwA/Nf63EOZz38D\ncCBwJNXIQW+/Xd8d2022F5fYzgfqw9ofaNMjfgzYjGcfXjanzYMM1cPY39tudy/bA38Ctpb0Ett/\noprv/w3VeoaXUD2oRETEMNfpz7YeC3wfmEs1bz0X+Gx/ByNpH+AiYJLtR2pVs4ADStLvXkx3Lm0+\nZidpG+Ak4BO25wIrgGNL9XTgNEmvrR0yuodYRgL7AcuBZUCXpO557Q9SvR9ty0tvf5zta6kS5N6l\n/o9UvfB2zgbOkbR9rayTuer55brdIxWTqUYSWv0X1TTBS0vb10h6WbnPS4D3Az+meu+g6tH/qiT5\nDwIjSvn3gA+Xofr6dM267i0iIgZJp6viPwa8HviZ7bcC+1B981x/m0HVe76qLMKaA1A+VncocLqk\nZVS98TuohrxbfQmYbvu3Zf9EqmS+eekJfwy4TNVHx24FXgv8ez2GsgbgnnKdq20/CXy4xLWYqnd7\nYU/lVAnuu5LuAX7As8nyCuCUstjuOYvnykPAucB1ZdHdbVRD4L2NFnwB2FnS3VSfKvgp8I027b5G\nNfJxp6R7qR6gRgKfBm4pP/BzEnBsefD5CtWnAu4GdqMsIiwPS3OABeV9OrmcfyZwYRbPRUQMLVUj\n1L00ku4oi64WUS2aWitpSflYVsR62XjbXbztlC8PdRgREYNqQ3/dTdJC2xN7a9fpN8/9QtUXpfwn\n8D1Jvwd+tiEBRkRERP/r9NfdDi+bZ0i6iWr+de6ARRURERF90mmP/c9sf38gAomIiIgNt96JPWJD\n7bndOBZs4FxTRES01+mq+IiIiHgBSGKPiIhokCT2iIiIBklij4iIaJAk9oiIiAZJYo+IiGiQJPaI\niIgGSWKPiIhokCT2iIiIBklij4iIaJAk9oiIiAZJYo+IiGiQJPaIiIgGSWKPiIhokCT2iIiIBkli\nj4iIaJAk9oiIiAYZOdQBxIvP4hUr6Tr1mqEOIyJi0Dw07Z2Ddq302CMiIhokiT0iIqJBktgjIiIa\nJIk9IiKiQZLYIyIiGiSJfQNIWtWyf7Sk88v2SZKWSrpH0jxJO9TajZd0o6Rlkn4i6TOS1MM13iBp\nfml3p6RrJO1Z6s6QtELSIkn3SfpXSS8pdTMlrZa0ae1cX5ZkSVu2uc4YSRdJWi5pYbnmfqXumXKN\neyVdJWm0pC5J97ac4wxJJ/f9HY2IiA2VxD5w7gIm2t4LmA1MB5A0CpgDTLO9K7A3sD8wtfUEkrYG\n/g/wadu72N4XOBt4da3ZP9ueAOwO7Am8uVb3U+DQcq6XAG8DVvQQ79eA3wG72H4d8GGg+wFgje0J\ntvcA/i/wkfV5IyIiYvAksQ8Q2zfZXl12bwe2L9tHAbfavr60Ww2cAJza5jQnAJfavq123h/Y/s82\nbTcCNgF+Xyu7Anhv2X4LcCvwdOuBkl4N7AecbvtP5ToP2m73YfNbgJ3blEdExDCQxL5hRpUh6kWS\nFgFn9tDuGOC6sj0eWFivtL0cGCNpbMtx44E7e4nh4+XavwLut72oVnc/sJWkzYD3UyX6dsYDi2w/\ns64LSRoJvANY3EtM7Y49TtICSQueWb1yfQ+PiIgOJbFvmO4h6gllOPyzrQ0kTQYmAjM29GKSfijp\nx5L+pVbcPRT/CuBlkt7XctjVwPuoeuS39PHSo8rDwwLgYeDrgHto27bc9sW2J9qeOGL0uD6GERER\nvclXyg4gSQcBpwFvtr22FC8F3tTSbidgle3HW06xBNgX+DaA7f0kHQm8q/Vatp+SNLecu94zv5Jq\nhOBS238QgNegAAALOElEQVTqYY3eEmBvSSN66LWvKQ8P9ZgfAzZrabc58GC7C0RExOBIj32ASNoH\nuAiYZPuRWtUs4ICS9LsX051LWVzX4gLgaEn718pG93A9AX8FLK+X2/4Z1cPFV3qKtUwFLAA+3706\nv6x67/HLjW2vAn4l6W2l/ebAIcAPejomIiIGXhL7wJkBjAGuKnPwcwBsr6FaqX66pGVU89V3AOe3\nnsD2r6kWv50t6aeSbgOObGnbPcd+LzCCNgnc9kUlea/LscDWwE/Lx9hmAo+s8wj4EPCZcv0bgc93\ncJ2IiBhAsnuaKo0YGBtvu4u3nfLloQ4jImLQ9Mevu0laaHtib+3SY4+IiGiQJPaIiIgGSWKPiIho\nkCT2iIiIBsnn2GPQ7bndOBb0w0KSiIh4vvTYIyIiGiSJPSIiokGS2CMiIhokiT0iIqJBktgjIiIa\nJIk9IiKiQZLYIyIiGiSJPSIiokGS2CMiIhokiT0iIqJBktgjIiIaJIk9IiKiQZLYIyIiGiSJPSIi\nokGS2CMiIhokiT0iIqJBktgjIiIaZORQBxAvPotXrKTr1GuGOoyIiEHz0LR3Dtq10mOPiIhokCT2\niIiIBklij4iIaJAk9oiIiAZJYo+IiGiQYZvYJa1q2T9a0vll+yRJSyXdI2mepB1q7cZLulHSMkk/\nkfQZSWpz/iMkzavtHyBpkaSRZf8QST+SdF8pv1LSq0rdTEkPlvL7JH2un+/9MEm7r6N+crn3JZLu\nlvQ1SS8vdfPLvd8t6VZJu5byhyRtWTvHWyR9t825//w+t6m7tvs6PdSfKGn0+txrRET0r2Gb2Htx\nFzDR9l7AbGA6gKRRwBxgmu1dgb2B/YGprSewfTWwVtJRkl4KfAWYavtpSXsA5wFTbO9mewIwC+iq\nneKUUj4BmCJpx368v8OAtold0iHAx4F32B4P7AvcBmxda/YB23sDlwIz+iso239j+w/raHIikMQe\nETGEXpCJ3fZNtleX3duB7cv2UcCttq8v7VYDJwCn9nCqE4CzgDOAO2zfVso/CXzR9o9r15xj++Y2\n59ik/PkEgKQDJd0labGkSyRt3Ev5tNrowzmS9gcmATPKiMCrW653GnCy7RUlrmdsX2J7WZvYbgZ2\n7uHe1+WVkuaWEY/p3YXdvX5JL5N0TRkVuFfSeyV9FHglcJOkm/pwzYiI6AfD+QtqRklaVNvfnKo3\n3uoY4LqyPR5YWK+0vVzSGEljbT/eUveApCupEnw9gY4HzuklvhmSTqdKnOfafkTSJsBM4EDb90u6\nDDhe0oU9lF8OHA7sZtuSXm77D5LmAN+1PbvNdccDd/YSW7d3A4s7bFs3AdgHWAssk3Se7Z/X6g8B\nfmn7nQCSxtleKekk4K22H209oaTjgOMARozdqg8hRUREJ4Zzj32N7QndL+CzrQ0kTQYm0sfhZkkj\ngIOBVcAOPbTZovSc75d0cq2qeyh+G+DA0tPeFXjQ9v2lzaXAm9ZRvhJ4Evi6pCOA1awHSXuW2JZL\nem+talZ5KPoroDtmtzlFuzKAebZX2n4SWMrz35vFwMGS/lHS/7C9srdYbV9se6LtiSNGj+uteURE\n9NFwTuzrJOkgqmHpSbbXluKlwOta2u0ErGrtrRdTqZLUMcAFtUV2S6jmrrH9WEngFwNjWk9gexUw\nHzhgfe/B9tPAG6jWCbwLmNvBYfXYFpfYrgNG1dp8oDwQHVbraT8GbFZrsznwvJ51sba2/QwtIzvl\nAWVfqvfuLEnPe+iKiIih8YJM7JL2AS6iSuqP1KpmAQeUpN+9mO5cyuK6lnNsA5wEfML2XGAFcGyp\nng6cJum1tUPaLgorq+j3A5YDy4AuSd3z2h8Evt9TuaQxwDjb11ItiNu71P8R2LSH2z8bOEfS9rWy\nUT20rZtfrts9UjEZ6NNcuKRXAqttf4NqtGTfUrWuuCMiYhAM5zn2dZlB1Xu+qnSyH7Y9yfYaSYcC\n50m6ABgBXA60+/jWl4Dptn9b9k8EbpH0LduLJX0MuEzSWKqe7cNA/WNt3XPsGwHzgKvLPPmHS1wj\ngTuAC22vbVdO1Wv+dpmbF9WDBsAVwFfLgrQjbS/vvqjtayVtBVxXEvQfgHuB/+rlPfsC8K+S7i7X\nmgt8o5djerJnuf8/AU8Bx5fyi4G5kn5p+619PHdERGwA2T1Ns0YMjI233cXbTvnyUIcRETFo+uPX\n3SQttD2xt3YvyKH4iIiIaC+JPSIiokGS2CMiIhrkhbp4Ll7A9txuHAv6Yb4pIiKeLz32iIiIBkli\nj4iIaJAk9oiIiAZJYo+IiGiQJPaIiIgGSWKPiIhokCT2iIiIBklij4iIaJD8CEwMOkl/pPop2+Fo\nS3r+nfqhNFzjgsTWV4mtb17Mse1ge6veGuWb52IoLOvkF4qGgqQFwzG24RoXJLa+Smx9k9h6l6H4\niIiIBklij4iIaJAk9hgKFw91AOswXGMbrnFBYuurxNY3ia0XWTwXERHRIOmxR0RENEgSe0RERIMk\nsceAkHSIpGWSfirp1Db1G0u6stT/UFLXMIrtTZLulPS0pCMHK64OYztJ0lJJ90iaJ2mHYRTbRyQt\nlrRI0g8k7T5cYqu1e48kSxq0jyR18L4dLem35X1bJOnY4RJbafO35d/cEkn/Plxik/TPtffsfkl/\nGEaxvUrSTZLuKv+t/s1gxQaA7bzy6tcXMAJYDuwEbATcDeze0mYqcGHZfh9w5TCKrQvYC7gMOHKY\nvW9vBUaX7eOH2fs2trY9CZg7XGIr7TYFbgZuByYOl9iAo4HzB+vf2XrGtgtwF7BZ2X/FcImtpf3f\nA5cMl9ioFtEdX7Z3Bx4azL/b9NhjILwB+KntB2z/X+AK4NCWNocCl5bt2cCBkjQcYrP9kO17gD8N\nQjzrG9tNtleX3duB7YdRbI/Xdl8GDNbK3E7+vQF8AfhH4MlBimt9YhsKncT2v4ALbP8ewPYjwyi2\nuvcD3xyUyDqLzcDYsj0O+OUgxQZkKD4GxnbAz2v7vyhlbdvYfhpYCWwxTGIbKusb2zHAdQMa0bM6\nik3S30laDkwHPjpcYpO0L/AXtq8ZpJi6dfp3+p4yZDtb0l8MTmgdxfYa4DWSbpV0u6RDhlFsAJTp\nqB2BGwchLugstjOAyZJ+AVxLNaIwaJLYI16AJE0GJgIzhjqWOtsX2H418Eng9KGOB0DSS4AvAf8w\n1LH04DtAl+29gO/x7EjWcDCSajj+LVS94q9KevmQRvR87wNm235mqAOpeT8w0/b2wN8Al5d/h4Mi\niT0Gwgqg3uvYvpS1bSNpJNVw1WPDJLah0lFskg4CTgMm2V47nGKruQI4bEAjelZvsW0K7AHMl/QQ\n8JfAnEFaQNfr+2b7sdrf49eA1w1CXB3FRtUbnWP7KdsPAvdTJfrhEFu39zF4w/DQWWzHAP8HwPZ/\nA5tQ/UDMoEhij4FwB7CLpB0lbUT1H96cljZzgCll+0jgRpeVJsMgtqHSa2yS9gEuokrqgzXf2Wls\n9f/hvxP4yXCIzfZK21va7rLdRbU2YZLtBUMdG4CkbWu7k4AfD0JcHcUG/CdVbx1JW1INzT8wTGJD\n0m7AZsB/D0JM6xPbw8CBJcbXUiX23w5ahIO5Ui+vF8+LavjpfqrVo6eVsjOp/odK+Yd+FfBT4EfA\nTsMottdT9VSeoBpFWDKMYrsB+A2wqLzmDKPY/gVYUuK6CRg/XGJraTufQVoV3+H7dnZ53+4u79tu\nwyg2UU1jLAUWA+8bLrGV/TOAaYMV03q8b7sDt5a/00XA2wczvnylbERERINkKD4iIqJBktgjIiIa\nJIk9IiKiQZLYIyIiGiSJPSIiokGS2CMiIhokiT0iIqJB/h+c+5OR2Zu8SgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2ab4a1b3c240>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data.plot(label=\"DMatrix + training time\",kind='barh',x='algorithm',y='training time')\n",
    "data.plot(title=\"training AUC\",kind='barh',x='algorithm',y='training AUC',legend=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Summary: Fastest GPU algorithm (XGBoost histogram) takes 5s, fastest CPU algorithm (H2O) takes 50s\n",
    "\n",
    "##### Note: H2O's XGBoost integration has some internal overhead still (DMatrix creation is single-threaded, and some parameters have different default values, hence the slightly slower training speed and slightly higher training accuracy) - this doesn't affect the summary conclusion"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}