{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Lesson 7 - Model interpretability\n", "\n", "> How to interpret the predictions from Random Forest models and use these insights to prune the feature space." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/lewtun/dslectures/master?urlpath=lab/tree/notebooks%2Flesson07_model-interpretation.ipynb) \n", "[![slides](https://img.shields.io/static/v1?label=slides&message=2021-lesson07.pdf&color=blue&logo=Google-drive)](https://drive.google.com/open?id=1Tib76a0leS8xhuAlni2WUhqlcITGEf38)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Learning objectives" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* Understand how to interpret feature importance plots for Random Forest models.\n", "* Know how to drop uninformative features to build simpler models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## References" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This lesson is adapted (with permission) from Jeremy Howard's fantastic online course [_Introduction to Machine Learning for Coders_](https://course18.fast.ai/ml), in particular:\n", "\n", "* [3 — Performance, validation and model interpretation](https://course18.fast.ai/lessonsml1/lesson3.html)\n", "\n", "Below are a few relevant articles that may be of general interest:\n", "\n", "* [Explaining Feature Importance by example of a Random Forest](https://towardsdatascience.com/explaining-feature-importance-by-example-of-a-random-forest-d9166011959e)\n", "* [Beware Default Random Forest Importances](https://explained.ai/rf-importance/index.html)\n", "* [Explainable AI won’t deliver. Here’s why.](https://hackernoon.com/explainable-ai-wont-deliver-here-s-why-6738f54216be)\n", "* [Confidence Intervals](https://dfrieds.com/math/confidence-intervals.html)\n", "* [Reading and Writing Files in Python (Guide)](https://realpython.com/read-write-files-python/)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Homework\n", "\n", "* Solve the exercises included in this notebook" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this lesson we will analyse the preprocessed table of clean housing data and their addresses that we prepared in lesson 3:\n", "\n", "* `housing_processed.csv`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## What is model interpretability?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n", "

Figure reference: https://bit.ly/3djjWc6

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A nice explanation for what it means to interpret a model's predictions is given in the _Beware Default Random Forest Importances_ article:\n", "\n", "> Training a model that accurately predicts outcomes is great, but most of the time you don't just need predictions, you want to be able to interpret your model. For example, if you build a model of house prices, knowing which features are most predictive of price tells us which features people are willing to pay for.\n", "\n", "In this lesson we will focus on one specific aspect of interpretability for Random Forests, namely _feature importance_ which is a technique that (with care) can be used to identify the most informative features in a dataset." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import libraries" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# reload modules before executing user code\n", "%load_ext autoreload\n", "# reload all modules every time before executing Python code\n", "%autoreload 2\n", "# render plots in notebook\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# uncomment to update the library if working locally\n", "# !pip install dslectures --upgrade" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# data wrangling\n", "import pandas as pd\n", "import numpy as np\n", "from dslectures.core import get_dataset, convert_strings_to_categories, rmse, fill_missing_values_with_median\n", "from pathlib import Path\n", "\n", "# data viz\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "sns.set(color_codes=True)\n", "sns.set_palette(sns.color_palette(\"muted\"))\n", "\n", "# ml magic\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.ensemble import RandomForestRegressor\n", "from sklearn.metrics import r2_score\n", "import scipy\n", "from scipy.cluster import hierarchy as hc" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load the data" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Dataset already exists at '../data/housing_processed.csv' and is not downloaded again.\n" ] } ], "source": [ "get_dataset('housing_processed.csv')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We also make use of the `pathlib` library to handle our filepaths:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "autos.csv housing_gmaps_data_raw.csv\n", "\u001b[1m\u001b[36mcar-challenge\u001b[m\u001b[m housing_merged.csv\n", "\u001b[1m\u001b[36mcats_vs_dogs\u001b[m\u001b[m housing_processed.csv\n", "churn.csv imdb.csv\n", "fine_tuned.pth \u001b[1m\u001b[36mspotify\u001b[m\u001b[m\n", "housing.csv word2vec-google-news-300.pkl\n", "housing_addresses.csv\n" ] } ], "source": [ "DATA = Path('../data/')\n", "!ls {DATA}" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
longitudelatitudehousing_median_agetotal_roomstotal_bedroomspopulationhouseholdsmedian_incomemedian_house_valuecitypostal_coderooms_per_householdbedrooms_per_householdbedrooms_per_roompopulation_per_householdocean_proximity_INLANDocean_proximity_<1H OCEANocean_proximity_NEAR BAYocean_proximity_NEAR OCEANocean_proximity_ISLAND
0-122.2337.8841.0880.0129.0322.0126.08.3252452600.069947056.9841271.0238100.1465912.55555600100
1-122.2237.8621.07099.01106.02401.01138.08.3014358500.0620946116.2381370.9718800.1557972.10984200100
2-122.2437.8552.01467.0190.0496.0177.07.2574352100.0620946188.2881361.0734460.1295162.80226000100
3-122.2537.8552.01274.0235.0558.0219.05.6431341300.0620946185.8173521.0730590.1844582.54794500100
4-122.2537.8552.01627.0280.0565.0259.03.8462342200.0620946186.2818531.0810810.1720962.18146700100
\n", "
" ], "text/plain": [ " longitude latitude housing_median_age total_rooms total_bedrooms \\\n", "0 -122.23 37.88 41.0 880.0 129.0 \n", "1 -122.22 37.86 21.0 7099.0 1106.0 \n", "2 -122.24 37.85 52.0 1467.0 190.0 \n", "3 -122.25 37.85 52.0 1274.0 235.0 \n", "4 -122.25 37.85 52.0 1627.0 280.0 \n", "\n", " population households median_income median_house_value city \\\n", "0 322.0 126.0 8.3252 452600.0 69 \n", "1 2401.0 1138.0 8.3014 358500.0 620 \n", "2 496.0 177.0 7.2574 352100.0 620 \n", "3 558.0 219.0 5.6431 341300.0 620 \n", "4 565.0 259.0 3.8462 342200.0 620 \n", "\n", " postal_code rooms_per_household bedrooms_per_household \\\n", "0 94705 6.984127 1.023810 \n", "1 94611 6.238137 0.971880 \n", "2 94618 8.288136 1.073446 \n", "3 94618 5.817352 1.073059 \n", "4 94618 6.281853 1.081081 \n", "\n", " bedrooms_per_room population_per_household ocean_proximity_INLAND \\\n", "0 0.146591 2.555556 0 \n", "1 0.155797 2.109842 0 \n", "2 0.129516 2.802260 0 \n", "3 0.184458 2.547945 0 \n", "4 0.172096 2.181467 0 \n", "\n", " ocean_proximity_<1H OCEAN ocean_proximity_NEAR BAY \\\n", "0 0 1 \n", "1 0 1 \n", "2 0 1 \n", "3 0 1 \n", "4 0 1 \n", "\n", " ocean_proximity_NEAR OCEAN ocean_proximity_ISLAND \n", "0 0 0 \n", "1 0 0 \n", "2 0 0 \n", "3 0 0 \n", "4 0 0 " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "housing_data = pd.read_csv(DATA/'housing_processed.csv'); housing_data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With the data loaded, we can recreate our train/validation splits as before:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X = housing_data.drop('median_house_value', axis=1)\n", "y = housing_data['median_house_value']" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "15554 train rows + 3889 valid rows\n" ] } ], "source": [ "X_train, X_valid, y_train, y_valid = train_test_split(X, y, test_size=0.2, random_state=42)\n", "print(f'{len(X_train)} train rows + {len(X_valid)} valid rows')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To simplify the evaluation of our models, we'll reuse our scoring function from lesson 5:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def print_rf_scores(fitted_model):\n", " \"\"\"Generates RMSE and R^2 scores from fitted Random Forest model.\"\"\"\n", "\n", " yhat_train = fitted_model.predict(X_train)\n", " R2_train = fitted_model.score(X_train, y_train)\n", " yhat_valid = fitted_model.predict(X_valid)\n", " R2_valid = fitted_model.score(X_valid, y_valid)\n", "\n", " scores = {\n", " \"RMSE on train:\": rmse(y_train, yhat_train),\n", " \"R^2 on train:\": R2_train,\n", " \"RMSE on valid:\": rmse(y_valid, yhat_valid),\n", " \"R^2 on valid:\": R2_valid,\n", " }\n", " if hasattr(fitted_model, \"oob_score_\"):\n", " scores[\"OOB R^2:\"] = fitted_model.oob_score_\n", "\n", " for score_name, score_value in scores.items():\n", " print(score_name, round(score_value, 3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Confidence intervals" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Recall that to make predictions with our Random Forest models, we take the _average value_ in each leaf node as we pass each row in the validation set through the tree. However, we would also like to estimate our _**confidence**_ in these predictions - how can we achieve this?\n", "\n", "One way to do this is to calculate the _**standard deviation of the predictions**_ of the trees. Conceptually, the idea is that if the standard deviation is high, each tree is generating very different predictions and may indicate the model has not learnt the most important features of the data.\n", "\n", "To get started, let's use our baseline model from the previous lesson:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RMSE on train: 16520.37\n", "R^2 on train: 0.971\n", "RMSE on valid: 42727.043\n", "R^2 on valid: 0.81\n", "OOB R^2: 0.791\n" ] } ], "source": [ "model = RandomForestRegressor(n_estimators=40, max_features='sqrt', n_jobs=-1, oob_score=True, random_state=42)\n", "model.fit(X_train, y_train)\n", "print_rf_scores(model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As before, we concatenate all the predictions from each tree into a single array:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(137125.0, 41845.23120978064)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "preds = np.stack([t.predict(X_valid) for t in model.estimators_])\n", "# calculate mean and standard deviation for single observation\n", "np.mean(preds[:,0]), np.std(preds[:,0])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(40, 3889)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "preds.shape" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, let's create a copy of the validation dataset and add the mean predictions and their standard deviation as new columns." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "#### Exercise #1\n", "\n", "* Combine `X_valid` and `y_valid` into a single array called `valid_copy`. You may find the `DataFrame.join(DataFrame)` method from pandas useful here.\n", "* Create two new columns `preds_mean` and `preds_std` that are the mean and standard deviation of the predictions for each tree in `preds`\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can use these new columns to drill-down into the predictions of each individual, categorical feature. Let's examine `ocean_proximity_INLAND` which denotes whether a housing district is inland or not:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEJCAYAAACaFuz/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAXi0lEQVR4nO3de1AV5+HG8ecgSCQxVQwY6yWt8TYx463emyqYAVRERyUxagRLTdTqpGE0RkeJxpjYMUYbtSaxNhMsaLUqMRirGI31AhmLbbRx1DqJdy1EBBUVwuHs749O+JUIxxV24bB8P3+5u+zheXdnnllf9uy6DMMwBABwLL/aDgAAsBdFDwAOR9EDgMNR9ADgcBQ9ADgcRQ8ADkfRA4DD+dd2gMrk59+Sx1N/bvFv1uwh5eUV1naMGsWY6wfGXDP8/Fxq2vTBCrf5bNF7PEa9KnpJ9W68EmOuLxhz7WLqBgAcjqIHAIej6AHA4Sh6AHA4ih4AHI6iBwCHo+gBwOEoegBwOBdvmAIA31BU7NbNG3eqtK+fn0vNmj1U4Taf/WbsS8u+0NWCotqOAQA1Zv3CMN204XOZugEAh6PoAcDhKHoAcDiKHgAcjqIHAIej6AHA4Sh6AHA4ih4AHI6iBwCHo+gBwOEoegBwOIoeAByOogcAh6PoAcDhKHoAcDiKHgAcjqIHAIej6AHA4Sh6AHA4ih4AHI6iBwCHo+gBwOEoegBwOIoeAByOogcAh6PoAcDhKHoAcDiKHgAcjqIHAIej6AHA4fzv9QPHjh1TcnKy/v3vf+uBBx5Qhw4dFB8frw4dOtREPgBANXm9ot+/f79+/etfq0OHDpo5c6amT5+uFi1aKCEhQYcPH66pjACAavB6Rb969WqtXbtWnTp1Kls3cOBADRgwQO+884569+5te0AAQPV4vaK/detWuZL/XpcuXXTz5k3bQgEArOO16P39K7/gNwzD8jAAAOtx1w0AOJzXOfpvvvlGMTExFW67cOGCLYEAANbyWvR/+MMfaioHAMAmXou+unfVpKen67333lNJSYkmTpyo8ePHV+vzAAD3z2vRDxo0SC6Xq8JtLpdLn332WaX75uTkaPny5dq6dasaNmyo5557Tn369FG7du2qlxgAcF+8Fv2KFSvuWpedna1ly5YpKirK6wdnZmaqb9++atKkiSQpKipKO3fu1PTp06sRFwBwv7wW/ZNPPln2b4/HoxUrVmjDhg1atGiRhg8f7vWDc3NzFRISUrYcGhqqY8eOVTMuAOB+3fNZN5J06dIlzZgxQ5K0efNmtW7d+p77VHSffWXTQAAA+9zzPvpPPvlEo0aNUt++fZWammqq5CWpefPmunr1atlybm6uQkNDq54UAFAlXq/oX3nlFWVkZCgxMVG9evXSyZMny23v3Llzpfv2799fK1eu1LVr19SoUSNlZGTojTfesCY1AMA0r0V/5MgRNWvWTOvWrdO6devKbXO5XNqzZ0+l+zZv3lyJiYmKi4tTSUmJYmNj1aVLF2tSAwBM81r0e/furXRbQUHBPT88Jiam0m/WAgBqxn0/6+brr7/Wa6+9prCwMBviAACsZuquG0k6cOCAPvroI2VmZupnP/uZ3nvvPTtzAQAs4rXoi4uLlZaWpnXr1ikvL09Dhw7V6dOnlZKSUlP5AADV5LXow8LC1LVrV7388ssKCwtTw4YNtX///prKBgCwgNei79mzp7Kzs/Xggw8qMDBQv/jFL2oqFwDAIl6LfuXKlcrNzdWmTZuUlJQkj8ej4uJiXbx4Ua1ataqpjACAarjnXTehoaGaPn26Pv/8cyUlJenJJ5/U4MGD9fLLL9dEPgBANZm+66ZBgwaKiopSVFSUzpw5o40bN9qZCwBgkSq9M/anP/2pjh49anUWAIANqvxy8FOnTlmZAwBgkyoXPQCgbqDoAcDhvP4xdtGiRRWuNwxDJSUltgQCAFjLa9F//77XikyePNnyMAAA63ktel7kDQB1n9einzNnTqXbXC6X3nrrLcsDAQCs5bXo27dvf9e6/Px8JScnq2XLlraFAgBYx2vRJyQklFvOzMzUq6++qpiYGM2bN8/WYAAAa5h6BILb7dY777yjtLQ0LViwQIMHD7Y7FwDAIvcs+nPnzikxMVFBQUFKS0tTixYtaiIXAMAiXr8wtXnzZj3zzDOKiIhQSkoKJQ8AdZDLMAyjso2dOnWSn5+fAgMD5XK5ytYbhiGXy6V//OMftgV7adkXulpQZNvnA4CvWb8wTN9+e7NK+/r5udSs2UMVbvM6dbNnz54q/UIAgO/wWvTcQgkAdZ/Xou/evXu5KZvv1cTUDQDAGl6Lfvv27TWVAwBgk2pP3Tz//PNKSUmxLBAAwFrVfh59YWGhFTkAADapdtFXNIcPAPAdvGEKAByOogcAh6PoAcDhql30Xp6gAADwAaaKfuTIkfrLX/6iO3fu3LUtNTXV8lAAAOuYKvqkpCRlZ2crIiJCCxcu1OnTp8u2Pfjgg7aFAwBUn6kXj/To0UM9evTQjRs3lJ6erqlTpyo0NFQTJkzQkCFD7M4IAKgG03P0N27c0LZt27Rp0yY1btxYQ4YM0bZt2zRr1iw78wEAqsnUFf2MGTO0f/9+hYWFacGCBerevbskaezYserfv7+tAQEA1eP1xSPfe//99/Xss88qODj4rm1ff/21Hn/8cVvCAUB9UlTs1s0bd9/0YkaVXzzyvezsbE2ZMqXcumeffVabNm2yreTz8grl8dSfWzdDQhpX+c0ydRVjrh8Yc+3zWvQvvfSSzpw5owsXLigmJqZsvdvtlp8f37UCgLrAa9HPmjVLly5dUlJSkpKSksrWN2jQQO3bt7c9HACg+rwWfatWrdSqVSvt2rWLp1QCQB3ltejHjh2rDRs2qEePHuWKnlcJAkDd4bXo3333XUm8UhAA6jKvRR8aGirpv68U/PLLL3X9+vVy2828ahAAULtM3V6ZmJio7OzssuKX/vtmqYEDB9oWDABgDVNF/9VXX2nPnj1q2LCh3XkAABYzdTN827Zt5Xa77c4CALCBqSv6MWPGaPjw4erevbv8/f9/l8WLF9sWDABgDVNFv2TJEj311FNq06aN3XkAABYzVfT+/v5asGCBzVEAAHYwNUffrVs37du3z+YoAAA7mLqi/+KLL7R582YFBAQoICCAb8YCQB1iquiTk5PtzgEAsInXos/KylK/fv10/PjxCrfzzVgA8H1ei/7TTz9Vv3799Kc//emubS6XS5GRkbYFAwBYw2vRL1q0SJK0du1aBQYGltuWk5NjXyoAgGVM3XXzzDPP6Ny5c2XLe/fu1ahRo2wLBQCwjqk/xsbFxWn8+PGaNWuWjh49qoMHD2rVqlW2BqvsJbdOFhLSuLYj1DhfGLP7u2LlX/+utmMAtjFV9LGxsQoJCdGUKVP0yCOP6JNPPlHTpk1tDfbNmply37hq6+8AJKnDzI8kUfRwLlNTNxs2bNDs2bM1d+5chYeHa9y4cfrXv/5ldzYAgAVMXdGnpqYqOTlZHTp0kCRlZGRoypQpOnTokK3hAADVZ6rot2zZosDAQLndbhmGocjISHXp0sXubAAAC5iauiksLNSkSZPUrVs3denSRRMmTCj3snAAgO8yVfQLFy5Ut27dlJmZqczMTPXq1YunWQJAHWGq6M+ePavp06fr4YcfVtOmTfXSSy/p/PnzdmcDAFjAVNG73W4VFxeXLd+5c4epGwCoI0z9MXbo0KGaOHFi2bdht27dqqioKFuDAQCsYarop02bpkcffVQHDhyQx+PRqFGjFBsba3c2AIAFTBV9fHy8kpOTNXr0aLvzAAAsZmqO/ubNm7p9+7bdWQAANjB1Rd+oUSOFh4erY8eOCgoKKlv//vvv2xYMAGAN0w81AwDUTaaKfuTIkcrPz1d2drb8/PzUu3dvNW5c+4+XBQDcm6k5+t27dysyMlLJyclau3atIiIi9MUXX9idDQBgAVNX9MuXL1dKSoo6duwoSTp+/LjmzZuntLQ0W8MBAKrP1BX9Aw88UFbyktS5c2e+GQsAdYSpou/Tp4/WrFmj27dvq7i4WBs3blT79u11/fp1FRQU2J0RAFANpqZu1q1bp9LSUi1btqzc+m3btsnlcunEiRO2hAMAVJ+poj9+/Hil29LT0y0LAwCwnqmpG28+/PBDK3IAAGxS7aI3DMOKHAAAm1S76Ln7BgB8W7WLHgDg2yh6AHA45ugBwOGqXfQxMTFW5AAA2MTUffSHDx/WypUrdf369XJX8Onp6frVr35lWzgAQPWZKvqFCxdq9OjReuKJJ7jLBgDqGFNFHxAQoF/+8pd2ZwEA2MDUHH379u116tQpu7MAAGxg6or+woULGj16tH784x8rMDCwbD3PuQEA32eq6BMTE+3OAQCwiami7927twoKCnTnzh0ZhqHS0lKdP3/e7mwAAAuYKvp3331Xa9askSQ1aNBAJSUlateuHVM3AFAHmPpj7LZt2/T5558rKipKGRkZ+u1vf6t27drZnQ0AYAFTRR8cHKzQ0FC1bdtWJ0+e1IgRI3Tu3Dm7swEALGCq6P39/XX+/Hm1bdtW2dnZcrvdunHjht3ZAAAWMFX0kydPVlJSksLCwrR7926FhYWpb9++99yvsLBQw4YN08WLF6sdFABQNab+GBseHq7w8HBJ0scff6xz586pY8eOXvc5evSo5s2bp7Nnz1Y7JACg6kxd0d+6dUuvv/664uPjVVxcrPXr1+v27dte99m0aZPmz5+v0NBQS4ICAKrGVNEvWrRIDz/8sPLy8hQYGKjCwkK99tprXvd588031bNnT0tCAgCqzlTRnzhxQomJifL391ejRo20dOlSnThxwu5sAAALmCp6P7/yP1ZaWnrXOgCAbzL1x9hevXrp7bffVlFRkQ4cOKCUlBT16dPH7mwAAAuYuiyfOXOmgoKC1LhxYy1fvlydOnXSrFmz7M4GALCA6ReP9O7dW9OmTVNBQYGys7PLPa7Ym71791YrIACgekxd0S9fvlwrVqyQJBUVFWnNmjVavXq1rcEAANYwVfR79uzRhx9+KEl69NFHlZKSoh07dtgaDABgDVNFX1JSooCAgLLlgIAAXhIOAHWEqTn6Hj16aMaMGYqNjZXL5VJaWpq6du1qdzYAgAVMXdEnJSXpkUce0fTp0/Wb3/xGISEhmjt3rt3ZAAAWMFX0ubm5OnjwoFwul9xut3bu3KnLly/bnQ0AYAFTRf/GG29o0qRJys7O1pEjRzR16lS9/vrrdmcDAFjAVNHn5eVp5MiRZcujR49Wfn6+baEAANYxVfSlpaUqKCgoW7527ZptgQAA1jJ1183zzz+vMWPGaMiQIZKkv/71r4qPj7c1GADAGqaKfsyYMWrTpo0OHjwoj8ej+fPnq3///nZnAwBYwFTRS1K/fv3Ur18/O7MAAGzAQ+UBwOEoegBwOIoeAByOogcAh6PoAcDhKHoAcDiKHgAcjqIHAIej6AHA4Sh6AHA4ih4AHI6iBwCHo+gBwOEoegBwOIoeAByOogcAh6PoAcDhKHoAcDiKHgAcjqIHAIej6AHA4Sh6AHA4ih4AHI6iBwCHo+gBwOEoegBwOP/aDlCZti8ure0IqCfc3xXXdgTAVj5b9Hl5hfJ4jNqOUWNCQhrr229v1naMGlUfxwzUBqZuAMDhKHoAcDiKHgAcjqIHAIej6AHA4Sh6AHA4ih4AHI6iBwCH89kvTPn5uWo7Qo1jzPUDY64fanrM3n6fyzCM+vP1UwCoh5i6AQCHo+gBwOEoegBwOIoeAByOogcAh6PoAcDhKHoAcDiKHgAcjqIHAIfzqaJPT0/X0KFDFRERodTU1NqOY6m4uDhFR0drxIgRGjFihI4ePVrpeDMzMxUTE6PIyEgtX768FlNXTWFhoYYNG6aLFy9Kqnw8J06c0OjRoxUVFaW5c+fK7XZLki5fvqzx48dr8ODBmjp1qm7dulUr47gfPxzznDlzFBkZWXa+d+/eLen+j4WvWrVqlaKjoxUdHa0lS5ZIcv55rmjMdeY8Gz7iP//5jxEeHm7k5+cbt27dMmJiYozTp0/XdixLeDwe4+c//7lRUlJStq6y8d65c8cYOHCgcf78eaOkpMRISEgw9u3bV4vp78+XX35pDBs2zOjcubNx4cIFr+OJjo42/vnPfxqGYRhz5swxUlNTDcMwjBdffNHYvn27YRiGsWrVKmPJkiW1MxiTfjhmwzCMYcOGGTk5OeV+rirHwhcdOnTIGDNmjFFcXGx89913RlxcnJGenu7o81zRmDMyMurMefaZK/rMzEz17dtXTZo0UVBQkKKiorRz587ajmWJb775Ri6XSy+88IKGDx+ulJSUSsd77NgxPfbYY2rdurX8/f0VExNTp47Dpk2bNH/+fIWGhkpSpeO5dOmSioqK1K1bN0nSqFGjtHPnTpWUlOjvf/+7oqKiyq33ZT8c8+3bt3X58mUlJSUpJiZGK1askMfjue9j4atCQkI0e/ZsNWzYUAEBAXr88cd19uxZR5/nisZ8+fLlOnOefebplbm5uQoJCSlbDg0N1bFjx2oxkXVu3Lihfv36acGCBSoqKlJcXJyGDBlS4XgrOg45OTm1EbtK3nzzzXLLlY3nh+tDQkKUk5Oj/Px8PfTQQ/L39y+33pf9cMx5eXnq27evFi5cqKCgIE2ePFmbN29WUFDQfR0LX9W+ffuyf589e1Y7duzQhAkTHH2eKxrz+vXrdfjw4Tpxnn3mit6o4CGaLpczHm3avXt3LVmyREFBQQoODlZsbKxWrFhx18+5XC7HHYfKxnO/6+uS1q1b6/e//72aNWumRo0aacKECfrb3/7muDGfPn1aCQkJevXVV9WmTZu7tjvxPP/vmNu2bVtnzrPPFH3z5s119erVsuXc3Nyy/wrXddnZ2crKyipbNgxDLVu2rHC8TjsOlY3nh+u//fZbhYaGKjg4WIWFhSotLS23vi45deqUdu3aVbZsGIb8/f3v+1j4siNHjmjixImaMWOGRo4cWS/O8w/HXJfOs88Uff/+/ZWVlaVr167pzp07ysjI0IABA2o7liVu3rypJUuWqLi4WIWFhUpLS9Pbb79d4Xi7du2qM2fO6Ny5cyotLdX27dvr9HGobDwtW7ZUYGCgjhw5Ikn6+OOPNWDAAAUEBKhnz57asWNHufV1iWEYeuutt3T9+nWVlJRo48aNioiIuO9j4auuXLmiadOmaenSpYqOjpbk/PNc0Zjr0nn2qRePpKen64MPPlBJSYliY2P1wgsv1HYky/zud7/Trl275PF4NG7cOMXHx1c63qysLC1evFjFxcUaOHCg5syZUyf+W/u/Bg0apHXr1qlVq1aVjufkyZOaN2+ebt26pSeeeEKLFy9Ww4YNdenSJc2ePVt5eXlq0aKFli1bph/96Ee1PaR7+t8xp6amKjU1VW63W5GRkZo5c6akys9tZcfCFy1atEhbtmwpN13z3HPP6Sc/+Yljz3NlY/Z4PHXiPPtU0QMArOczUzcAAHtQ9ADgcBQ9ADgcRQ8ADkfRA4DDUfSAjRISEnTt2rXajoF6jqIHbHTo0KHajgBQ9Ki/Nm/erOjoaMXExCguLk5XrlzRxo0bNWzYMA0fPlwJCQk6c+aMJGn27Nn64x//WLbv/y4PGjRIK1eu1Lhx4xQeHl7uWeWSFB8frytXrtTw6ID/5zNPrwRq0smTJ7V06VKlpaWpRYsW+uijjzRx4kR5PB5t3LhRwcHB2rp1q6ZNm6ZPP/30np93+/ZtrV+/Xjk5OYqIiNDYsWO1ePFibd26VcnJyQoODq6BUQEV44oe9VJWVpaeeuoptWjRQpI0ceJEPf300xo6dGhZKY8aNUo5OTllb43y5umnn5b034e4NWvWTNevX7cvPHCfKHrUSw0aNCj3/KCioqIKC90wDLnd7rseM1tSUlLu5wIDA8v+XdkjaYHaQtGjXurTp4+ysrKUm5srSfrzn/+sffv2aceOHWV3yWzZskVNmjTRY489pqZNm+qrr76SJF27dk3Z2dmmfk+DBg18/v2vcD7m6FEvdezYUa+88oomTZok6b9v+9m9e7c+++wzxcfHy+PxKDg4WB988IH8/Pw0YcIEzZw5U1FRUWrVqpV69+5t6vdERERo3LhxWr16tTp06GDnkIBK8fRKAHA4pm4AwOEoegBwOIoeAByOogcAh6PoAcDhKHoAcDiKHgAcjqIHAIf7Pzwbt48Rn6E8AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sns.countplot(y='ocean_proximity_INLAND', data=valid_copy);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can calculate the predictions and standard deviation per category by applying a _**group by**_ operation in pandas, followed by taking the mean." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ocean_proximity_INLANDmedian_house_valuepreds_meanpreds_std
00227863.194178227319.69934951913.127856
11123654.224570124241.48816535858.872619
\n", "
" ], "text/plain": [ " ocean_proximity_INLAND median_house_value preds_mean preds_std\n", "0 0 227863.194178 227319.699349 51913.127856\n", "1 1 123654.224570 124241.488165 35858.872619" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cols = ['ocean_proximity_INLAND', 'median_house_value', 'preds_mean', 'preds_std']\n", "preds_quality = valid_copy[cols].groupby('ocean_proximity_INLAND', as_index=False).mean()\n", "preds_quality" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the table, we can see that the predictions and ground truth are close to each other on average, while the standard deviation varies somewhat for each category. We can visualise this table in terms of bar plots as follows:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA18AAAGeCAYAAABrbbtRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3daWAUdb718dNJOmGLLCFhxxFkGVHZ1+uFoMMmCV42EUFwYBzkwqhcFFGIIKIoIsiiKPB4CSaCyGIMKqAgAgJq4ArKIDCyBRgBCZCELCTpel4wZoyQTkG6qknl+3llV6erT+dnC8d/LS7DMAwBAAAAACwV4O8AAAAAAFAaUL4AAAAAwAaULwAAAACwAeULAAAAAGxA+QIAAAAAG1C+AAAAAMAGlC8AAAAAsEGQvwMU5ty5i/J4uAVZSRYWVkFnz6b7OwaKiTk6A3N0DmbpDMzR/wICXKpcuby/Y6CUuWHLl8djUL4cgBk6A3N0BuboHMzSGZgjUPpw2CEAAAAA2IDyBQAAAAA2oHwBAAAAgA0oXwAAAABgA8oXAAAAANiA8gUAAAAANqB8AQAAAIANKF8AAAAAYAPKFwAAAADYgPIFAAAAADagfAEAAACADShfAAAAAGADyhcAAAAA2IDyBQAAAAA2oHwBAAAAgA0oXwAAAABgA8oXAAAAANggqKgf2LNnj2JjY3XgwAGVKVNGDRs21NChQ9WwYUM78gEAAACAI3gtX5s3b9azzz6rhx56SL169ZIkff/99xo2bJhmzpypNm3aWBYsLKyCZfuGfcLDQ/0dAT7gxDlmZecpLTXD3zEAAEAp4rV8vfnmm1q0aJEaN26cv61Tp07q2LGjXnvtNUvL19BX/q7T53Ms2z+A0u3TaU2V5u8QAACgVPF6ztfFixcLFK9f3XnnnUpL468tAAAAAGCW1/IVFFT4wphhGD4PAwAAAABOxdUOAQAAAMAGXs/5OnTokKKjo6/6XHJysiWBAAAAAMCJvJavhQsX2pUDAAAAABzNa/my8mqGAAAAAFCaeC1fd999t1wu11Wfc7lc+vzzzy0JBQAAAABO47V8zZkz54ptSUlJmjlzprp162ZZKAAAAABwGq/l6/bbb8//Z4/Hozlz5mjp0qWaOnWqevXqZXk4AAAAAHAKr+XrVydOnNDYsWMlSStWrFCdOnUsDQUAAAAATlPkfb4++ugj9enTR+3atVN8fDzFCwAAAACug9eVr6eeekrr16/XmDFj1Lp1a/34448Fnm/SpIml4QAAAADAKbyWr507dyosLExLlizRkiVLCjzncrm0YcMGS8MBAAAAgFN4LV8bN24s9Lnz58/7PAwAAAAAOFWR53z93k8//aTnnntOkZGRFsQBAAAAAGcydbVDSdqyZYsWL16sbdu2qWXLlpo/f76VuQAAAADAUbyufGVnZ2vZsmW699579eSTT6pu3boKDw9XXFyc2rdvX+TO09PTFRUVpePHj/ssMAAAAACURF5XviIjI9W0aVM98cQTioyMVHBwsDZv3mxqx7t379bEiRN15MgRX+QEAAAAgBLN68pXq1attHv3bq1bt07bt2+Xx+MxvePly5dr0qRJioiIKHZIAAAAACjpvK58zZ07V6dPn9by5csVExMjj8ej7OxsHT9+XLVr1/a64xdffNGnQQEAAACgJCvyghsREREaPXq0Ro4cqc8//1zLli1T9+7d9ac//Umvv/66HRkBwBLh4aH+jmCr0vZ5nYxZOgNzBEof01c7DAwMVLdu3dStWzcdPnxY77//vpW5AMByZ86k+TuCbcLDQ0vV53UyZukMzNH/AgJcCgur4O8YKGWu+T5fknTLLbdo9+7dvs4CAAAAAI51XeVLkvbv3+/LHAAAAADgaKYPO7xeGzdutPotAAAAAOCGd90rXwAAAAAA87yufE2dOvWq2w3DUE5OjiWBAAAAAMCJvJavSpUqFfrciBEjfB4GAAAAAJzKa/kaPXq0XTkAAAAAwNG8lq9nnnmm0OdcLpdeeuklnwcCAAAAACfyWr4aNGhwxbZz584pNjZWtWrVsiwUAAAAADiN1/I1bNiwAo+3bdump59+WtHR0Zo4caKlwQAAAADASUzd5ys3N1evvfaaVq9ercmTJ6t79+5W5wIAAAAARymyfB09elRjxoxRuXLltHr1atWoUcOOXAAAAADgKF5vsrxixQr1799fXbp0UVxcHMULAAAAAK6T15WviRMnKiAgQAsWLNDChQvztxuGIZfLpV27dlkeEAAAAACcwGv52rBhg105AAAAAMDRvJYvLicPAAAAAL7htXw1b95cLpfriu0cdggAAAAA18Zr+VqzZo1dOQAAAADA0Yp92OHgwYMVFxfns0AAAAAA4EReLzVvRnp6ui9yAAAAAICjFXmT5aJc7ZwwX4h9+jZL9gsAkpSVnefvCAAAoJQpdvmyytmz6fJ4DH/HQDGEh4fqzJk0f8dAMTFHAAAA3yj2YYcAAAAAgKJRvgAAAADABsUuX4bBoYEAAAAAUBRT5at379764IMPlJmZecVz8fHxPg8FAAAAAE5jqnzFxMQoKSlJXbp00ZQpU3Tw4MH858qXL29ZOAAAAABwClNXO2zRooVatGih1NRUJSYmauTIkYqIiNBDDz2kHj16WJ0RAAAAAEo80+d8paamKiEhQcuXL1doaKh69OihhIQEjRs3zsp8AAAAAOAIpla+xo4dq82bNysyMlKTJ09W8+bNJUkDBw5Uhw4dLA0IAAAAAE5gqnw1aNBAEyZMUJUqVQq+OChIS5cutSQYAAAAADiJqcMOk5KSrihe999/vySpfv36vk8FAAAAAA7jdeXrscce0+HDh5WcnKzo6Oj87bm5uQoI4P7MAAAAAGCW1/I1btw4nThxQjExMYqJicnfHhgYqAYNGlgeDgAAAACcwmv5ql27tmrXrq1169bJ5XLZlQkAAAAAHMdr+Ro4cKCWLl2qFi1aFChfhmHI5XJp165dlgcEAAAAACfwWr5mz54tSVqzZo0tYQAAAADAqbyWr4iICElSrVq19N133+nChQsFnq9Vq5Z1yQAAAADAQUzd52vMmDFKSkrKL2OS5HK51KlTJ8uCAQAAAICTmCpfP/zwgzZs2KDg4GCr8wAAAACAI5m6WVe9evWUm5trdRYAAAAAcCxTK18DBgxQr1691Lx5cwUF/fsl06ZNsywYAAAAADiJqfI1ffp03XXXXapbt67VeQAAAADAkUyVr6CgIE2ePNniKAAAAADgXKbO+WrWrJk2bdpkcRQAAAAAcC5TK187duzQihUr5Ha75Xa7ZRiGXC6Xdu3aZXU+AAAAAHAEU+UrNjbW6hwAAAAA4Ghey9f27dvVvn177d2796rP16pVy5JQAAAAgL/k5OQoOTlZmZlZ/o6CEqZs2TKqU6eO3G73VZ/3Wr4+/vhjtW/fXu++++4Vz7lcLnXt2tU3KQEAAIAbRHJysoKCQlSjRoRcLpe/46CEMAxDaWkXlJycrHr16l31Z7yWr6lTp0qSFi1apJCQkALPnTp1ykcxAQAAgBtHZmYWxQvXzOVyKTS0ov75z2OF/oypqx32799fR48ezX+8ceNG9enTp/gJAQAAgBsQxQvXo6h/b0xdcGPIkCEaNGiQxo0bp927d2vr1q2aN2+eTwICAAAAN7Ky5cuoTHCgz/ebdSlPmRc5r6w0MVW++vXrp/DwcD366KOqWrWqPvroI1WuXNnqbAAAAIDflQkOVI9ndvt8v59Oa6rMiz7fbaF27kzSokVva/78hXrxxSnq06ef/vjH23y2///6r556882Fqlmzps/2aZff/m6sZOqww6VLl2r8+PGaMGGCOnfurAcffFDff/+9pcEAAAAAWGPChOd8WrxgjqmVr/j4eMXGxqphw4aSpPXr1+vRRx/VV199ZWk4AAAAoLTbuTNJixf/PxmGoRMnjuvuu+9R+fKh2rz5CxmGoZkz5+rAgR+1cOFbys3NVY0aNfXsszGqWLGSvv56u15//TUFBwfr5ptvyd/nyJGP6C9/GaGmTZtp+vRpOnToH0pJSVHdujfr5ZdnKCUlRePHj1W9evV14MB+ValSRS++OF0VK1b0mvWddxbowIH9ysrK0nPPTdHtt9+hY8eOatq0qUpNvaCyZcvqf/5nnG67rYmmTJmkFi1aKiqqlySpXbsW2rFjl7799mvNmzf7XxewuEkvvPCSKlWqrE8+WaNly96TYXjUuPEf9eST46+4KOCvNm/+UgkJK/Xaa3MkSR98sEzHjh3To4/+t158cYpOnz6lX345o2bNWmjSpBcKvPbX303Llq108uRJ/fd/P6IPP/xYZ8+e1SuvvKhTp35WQECARo78m9q0aXtNszS18rVy5Uo1bNhQubm5ysnJUdeuXbVy5cpreiMAAAAA12fv3h8UEzNZS5d+oFWrVqhy5UpavDhet97aQKtXr9Cbb87V66+/oSVLlqpdu/aaN2+OLl26pClTJumll6YrNva9qxaV77/fLbc7SIsWxWrFigRlZ2dr27bLCywHDx7QwIGD9d57H6hChVCtW/dpkTlvuaWelixZqv79B+i99y7frmry5Im6//4HFB+/XE88MVbPPvuULl26VOg+/vd//5+efnqCFi+O1113ddT+/T/q0KGflJCwSgsX/q/efXeZKleuovj4K2+H9asOHTpo//4flZqaKklav36dune/V199tVUNGjTUokWx+uCDBP3wwx7t37+vyM8lSbNmvaqoqPsUG/ueXn11ll555UVdvHhtx42aWvlKT0/XqFGjtGPHDuXl5alVq1aaMWPGNb0RAAAAgOtTv359VatWXZJUsWIltWrVRpJUvXoNbd26WadO/axRo/4qSfJ4PLrpppv000//UNWqVXXLLZfvOdWzZ5Tefnt+gf02b95SFStW0ooV7+vIkSM6fvyYMjMzJEmVK1dRo0aN898/NfVCkTk7doyUJNWrV19ffLFRGRkZOn48WZ073yNJuv32O3XTTRV19OiRQvfxn//ZUU8/PVadOkXqP/8zUm3bttMHHyxTcnKy/vKXoZIu3wj712xXExTkVmTk3friiw1q06atUlMvqEmT29Wkye3au/cHLVsWryNHDuvChQvKyMgs8nNJ0rfffqOjR49o4cLLv8Pc3FydOHFcDRs2MvV6yWT5mjJlipo1a6aZM2cqLy9P7777riZPnqz58+cX/WIAAAAAxRIU5C7wODDw33+N93g8uvPOZpox43VJUnZ2tjIyMnTq1D9lGMZvXnPlFRs3b/5SCxfO14ABAxUV1UsXLpzPf01wcPBvftJVYF+F+Xeuyz9vGJ4rXmcYhvLy8nT5quyXn8vNzcl/fuDAwbrrro766qsteuON2fr7339QuXLldM89XTR27DhJUkZGhvLy8rxm6d79Xr399nylpaWqa9fukqTly5fpiy8+13339VH//m31008/XZHv8uXif82Vm7/d48nTvHlv5x96eebMGVWpUqXI38lvmTrs8MiRIxo9erRuuukmVa5cWY899piOHSv85mEAAAAA7NGkye364Yc9Onbs8n1533lnoebOfV233tpA586l6ODBA5IuH3r3e99++7XuuaeLoqLuU1hYVX333S55PB6fZStfvoJq1aqtL77YIEn64Yc9Onv2rOrXv1WVKlXSoUOHJElffrkp/zXDhg1RRkaGHnhgkB54YJD27/9RLVq00pdffqGUlBQZhqHp01/SsmXxXt/79tvv1C+/nNGnn36s7t3vlSR9880O/dd/9VX37vfK5XLp4MH98ngKlrjLuX6SJG3e/EX+9pYtW2vlyg8kSYcPH9LgwQOUlXVttwowtfKVm5ur7Ozs/ONEMzMzufEcAAAASoWsS3n6dFpTS/brC1WqhGnChEmaMOFpeTwehYdH6PnnpyooyK0pU17S5MkTFRgYdNXD9O67r7cmTZqgjRs/l9vtVpMmd+jkyRM+yfWryZOn6pVXXtKiRW/L7Xbr5ZdnyO12q0+f/po4cbwGDbpfrVq1VtWqVSVJI0eO0gsvTFJgYKBCQsro6aefVf36t2r48L9q9OgR8ng8atiwsYYM+XOR7/2nP3XVjh3bVKtWbUnSAw88qOnTpyk+/l2VL19Od9zRVCdPnlTt2nXyXzN48FBNmfKcEhM/UqdOkfnbx459Wi+//IIGDbpfkjRp0gsqX778Nf0uXIaJ9cM33nhDW7duVZ8+fSRJq1atUocOHfS3v/3tmt7sWpw9my6Pp+ilTdy4wsNDdeZMmr9joJiYozMwR+dgls7AHP0vIMClsLAKV31u796/q2bNm21OBKc4efKomjS5+mX8Ta18jRo1StWrV9eWLVvk8XjUp08f9evXz6chAQAAANzY/vu//6q0tNQrtvfu3U99+tjfD260PEUxVb6GDh2q2NhY9e3b1+o8AAAAAG5Qb765wN8RCrjR8hTFVPlKS0tTRkaGypUrZ3WefIUtA6NkCQ8P9XcE+ABzdAbm6ByldZZZ2XlKS83wdwyUEoZhcI0DXLOizugyVb7Kli2rzp07q1GjRgUK2FtvvVW8dF4MfeXvOn0+p+gfBAAApcKn05qKs6Rgh8DAQOXl5V5xeXegKJf/vbnykv6/MlW+OL8LAAAApUWVKpWVmnpOlStXlctl6s5MgAzDowsXzqly5cqF/oyp8tW7d2+dO3dOSUlJCggIUJs2bRQaWjoPeQAAAICzVa1aVZmZmfr552SZuK8wIElyuaTy5cvnXzL/akyVr88++0zPPvusGjVqpLy8PE2YMEGvv/662rVr57OwAAAAwI0gICBAN9/Mpebhe6bK16xZsxQXF6dGjRpJkvbu3auJEydq9erVloYDAAAAAKcwdRBrmTJl8ouXJDVp0oSrvwAAAADANTBVvtq2basFCxYoIyND2dnZev/999WgQQNduHBB58+ftzojAAAAAJR4pg47XLJkifLy8jRz5swC2xMSEuRyubRv3z5LwgEAAACAU5gqX3v37i30ucTERJ+FAQAAAACnKvaNC9555x1f5AAAAAAARyt2+TK4+QEAAAAAFKnY5YurHgIAAABA0YpdvgAAAAAARaN8AQAAAIANOOcLAAAAAGxQ7PIVHR3tixwAAAAA4Gim7vP1zTffaO7cubpw4UKBla7ExEQNHz7csnAAAAAA4BSmyteUKVPUt29f3Xbbbdd0dcPExETNnz9fOTk5evjhhzVo0KDrDgoAAAAAJZmp8uV2u/XnP//5mnZ86tQpzZo1S6tWrVJwcLAeeOABtW3bVrfeeut1BQUAAACAkszUOV8NGjTQ/v37r2nH27ZtU7t27VSpUiWVK1dO3bp109q1a68rJAAAAACUdKZWvpKTk9W3b1/VrFlTISEh+dsTExMLfc3p06cVHh6e/zgiIkJ79uwpRlQAAAAAKLlMla8xY8Zc846vdgn6azlfDAAA4PfCw0P9HcFnnPRZAJhjqny1adNG58+fV2ZmpgzDUF5eno4dO+b1NdWqVVNSUlL+49OnTysiIqJ4aQEAQKl25kyavyP4RHh4qGM+S0kVEOBSWFgFf8dAKWOqfM2ePVsLFiyQJAUGBionJ0e33nqr18MOO3TooLlz5yolJUVly5bV+vXr9cILL/gmNQAAAACUMKbKV0JCgr744gu9/PLLGjdunL7++mtt2rTJ62uqVaumMWPGaMiQIcrJyVG/fv105513+iIzAAAAAJQ4pspXlSpVFBERoXr16unHH3/Ufffdp9jY2CJfFx0drejo6GKHBAAAAICSztSl5oOCgnTs2DHVq1dPSUlJys3NVWpqqtXZAAAAAMAxTJWvESNGKCYmRpGRkfrss88UGRmpdu3aWZ0NAAAAABzD1GGHnTt3VufOnSVJH374oY4ePapGjRpZGgwAAAAAnMTUytfFixf1/PPPa+jQocrOztZ7772njIwMq7MBAAAAgGOYKl9Tp07VTTfdpLNnzyokJETp6el67rnnrM4GAAAAAI5hqnzt27dPY8aMUVBQkMqWLasZM2Zo3759VmcDAAAAAMcwVb4CAgr+WF5e3hXbAAAAAACFM3XBjdatW+vVV19VVlaWtmzZori4OLVt29bqbAAAAADgGKaWr5588kmVK1dOoaGhmjVrlho3bqxx48ZZnQ0AAAAAHMPUypfb7VabNm00atQonT9/XklJSQoJCbE6GwAAAAA4hqmVr1mzZmnOnDmSpKysLC1YsEBvvvmmpcEAAAAAwElMla8NGzbonXfekSRVr15dcXFx+uSTTywNBgAAAABOYqp85eTkyO125z92u91yuVyWhQIAAAAApzF1zleLFi00duxY9evXTy6XS6tXr1bTpk2tzgYAAAAAjmFq5SsmJkZVq1bV6NGj9fjjjys8PFwTJkywOhsAAAAAOIap8nX69Glt3bpVLpdLubm5Wrt2rU6ePGl1NgAAAABwDJdhGEZRPzR8+HBFRUWpd+/ekqSVK1cqISFBS5YssTwgAACAJGVl5yktNcPfMXwiPDxUZ86k+TtGqRYQ4FJYWAV/x0ApY+qcr7Nnz+YXL0nq27evFi9ebFWmf71nujyeInshbmD8weIMzNEZmKNzMEsAKLlMHXaYl5en8+fP5z9OSUmxLBAAAAAAOJGpla/BgwdrwIAB6tGjhyTp008/1dChQy0NBgAAAABOYqp8DRgwQHXr1tXWrVvl8Xg0adIkdejQwepsAAAAAOAYpsqXJLVv317t27e3MgsAAAAAOJapc74AAAAAAMVD+QIAAAAAG1C+AAAAAMAGlC8AAAAAsAHlCwAAAABsQPkCAAAAABtQvgAAAADABpQvAAAAALAB5QsAAAAAbED5AgAAAAAbUL4AAAAAwAaULwAAAACwAeULAAAAAGxA+QIAAAAAG1C+AAAAAMAGQf4OUJiAAJe/I8AHmKMzMEdnYI7OwSydgTn6F79/+IPLMAzD3yEAAAAAwOk47BAAAAAAbED5AgAAAAAbUL4AAAAAwAaULwAAAACwAeULAAAAAGxA+QIAAAAAG1C+AAAAAMAGlC8AAAAAsAHlCwAAAABsQPkCAAAAABtQvgAAAADABpQvAAAAALAB5QsAAAAAbED5AgAAAAAbUL4AAAAAwAaULwAAAACwAeULAAAAAGxA+QIAAAAAG1C+AAAAAMAGlC8AAAAAsAHlCwAAAABsQPkCAAAAABtQvgAAAADABpQvAAAAALAB5QsAAAAAbED5AgAAAAAbUL4AAAAAwAaULwAAAACwAeULAAAAAGxA+QIAAAAAG1C+AAAAAMAGlC8AAAAAsAHlCwAAAABsQPkCAAAAABsE+TtAYc6duyiPx/B3DBRDWFgFnT2b7u8YKCbm6AzM0TmYpTMwR/8LCHCpcuXy/o6BUuaGLV8ej0H5cgBm6AzM0RmYo3MwS2dgjkDpw2GHAAAAAGADyhcAAAAA2IDyBQAAAAA2oHwBAAAAgA0oXwAAAABgA8oXAAAAANiA8gUAAAAANqB8AQAAAIANKF8AAAAAYAPKFwAAAADYgPIFAAAAADagfAEAAACADShfAAAAAGADyhcAAAAA2IDyBQAAAAA2oHwBAAAAgA0oXwAAAABgA8oXAAAAANggqKgf2LNnj2JjY3XgwAGVKVNGDRs21NChQ9WwYUM78gEAAACAI3gtX5s3b9azzz6rhx56SL169ZIkff/99xo2bJhmzpypNm3aWBYs2xOoPI9h2f5hvVPnsnXJE+jvGCgmp88xJEhyefL8HQMAAJQCXsvXm2++qUWLFqlx48b52zp16qSOHTvqtddes7R8fbTtjFIv5lq2f1ivTFm3sjJz/B0DxeT0OQ68p7rKcAA2AACwgde/cly8eLFA8frVnXfeqbS0NMtCAQAAAIDTeC1fQUGFL4wZBocEAgAAAIBZHGwDAAAAADbwes7XoUOHFB0dfdXnkpOTLQkEAAAAAE7ktXwtXLjQrhwAAAAA4Ghey5eVVzMEAAAAgNLEa/m6++675XK5rvqcy+XS559/bkkoAAAAAHAar+Vrzpw5V2xLSkrSzJkz1a1bN8tCAQAAAIDTeC1ft99+e/4/ezwezZkzR0uXLtXUqVPVq1cvy8MBAAAAgFN4LV+/OnHihMaOHStJWrFiherUqWNpKAAAAABwmiLv8/XRRx+pT58+ateuneLj4yleAAAAAHAdvK58PfXUU1q/fr3GjBmj1q1b68cffyzwfJMmTSwNBwAAAABO4bV87dy5U2FhYVqyZImWLFlS4DmXy6UNGzZYGg4AAAAAnMJr+dq4cWOhz50/f97nYQAAAADAqYo85+v3fvrpJz333HOKjIy0IA4AAAAAOJOpqx1K0pYtW7R48WJt27ZNLVu21Pz5863MBQAAAACO4nXlKzs7W8uWLdO9996rJ598UnXr1lV4eLji4uLUvn37Ineenp6uqKgoHT9+3GeBAQAAAKAk8rryFRkZqaZNm+qJJ55QZGSkgoODtXnzZlM73r17tyZOnKgjR474IicAAAAAlGheV75atWql3bt3a926ddq+fbs8Ho/pHS9fvlyTJk1SREREsUMCAAAAQEnndeVr7ty5On36tJYvX66YmBh5PB5lZ2fr+PHjql27ttcdv/jiiz4NCqCgjxePs+V9AgJc8ngMW97LH75NCFaAnPv5fhUcHKRLl3It2/+8eQss2zcAAE5R5AU3IiIiNHr0aI0cOVKff/65li1bpu7du+tPf/qTXn/9dcuChZQJUhmPy7L9wx5lyrr9HcGxAgLs+37Y+V52C3C5FOwO9HcMWwQHm77G0jULDw+1bN+4Er9vZ2COQOlj+k/iwMBAdevWTd26ddPhw4f1/vvvW5lL2Vm5ysq07v/SwnplyrqVlZnj7xiO1WPIK7a8j9PnOPCe6ioTkOfvGJYLDw/VmTNplu3fyn2jIKtnCXswR/8LCHApLKyCv2OglLnm+3xJ0i233KLdu3f7OgsAAAAAONZ1lS9J2r9/vy9zAAAAAICjWXcCwL9s3LjR6rcAAAAAgBveda98AQAAAADM87ryNXXq1KtuNwxDOTnOPQEfAAAAAHzNa/mqVKlSoc+NGDHC52EAAAAAwKm8lq/Ro0fblQMAAAAAHM1r+XrmmWcKfc7lciAxsAQAABkiSURBVOmll17yeSAAAAAAcCKv5atBgwZXbDt37pxiY2NVq1Yty0IBAAAAgNN4LV/Dhg0r8Hjbtm16+umnFR0drYkTJ1oaDAAAAACcxNR9vnJzc/Xaa69p9erVmjx5srp37251LgAAAABwlCLL19GjRzVmzBiVK1dOq1evVo0aNezIBQAAAACO4vUmyytWrFD//v3VpUsXxcXFUbwAAAAA4Dp5XfmaOHGiAgICtGDBAi1cuDB/u2EYcrlc2rVrl+UBAQAAAMAJvJavDRs22JUDAAAAABzNa/nicvIAAAAA4Btey1fz5s3lcrmu2M5hhwAAAABwbbyWrzVr1tiVAwAAAAAcrdiHHQ4ePFhxcXE+CwQAAAAATmTqJsvepKen+yLHFXp1CFeex7Bk37BHcHCgLl3K83cMFJPT5xgSJMnj7xQAAKA0KHb5uto5Yb4QEpAnjyhfJVl45XI6cybN3zFQTI6fI8ULAADYxOtNlgEAAAAAvkH5AgAAAAAbUL4AAAAAwAbFLl+GwXlZAAAAAFAUU+Wrd+/e+uCDD5SZmXnFc/Hx8T4PBQAAAABOY6p8xcTEKCkpSV26dNGUKVN08ODB/OfKly9vWTgAAAAAcApTl5pv0aKFWrRoodTUVCUmJmrkyJGKiIjQQw89pB49elidEQAAAABKPNPnfKWmpiohIUHLly9XaGioevTooYSEBI0bN87KfAAAAADgCKZWvsaOHavNmzcrMjJSkydPVvPmzSVJAwcOVIcOHSwNCAAAAABOYKp8NWjQQBMmTFCVKlUKvjgoSEuXLrUkGAAAAAA4ianDDpOSkq4oXvfff78kqX79+r5PBQAAAAAO43Xl67HHHtPhw4eVnJys6Ojo/O25ubkKCOD+zAAAAABgltfyNW7cOJ04cUIxMTGKiYnJ3x4YGKgGDRpYHg4AAAAAnMJr+apdu7Zq166tdevWyeVy2ZUJAAAAABzHa/kaOHCgli5dqhYtWhQoX4ZhyOVyadeuXZYHBAAAAAAn8Fq+Zs+eLUlas2aNLWEAAAAAwKm8lq+IiAhJUq1atfTdd9/pwoULBZ6vVauWdckAAAAAwEFM3edrzJgxSkpKyi9jkuRyudSpUyfLggEAAACAk5gqXz/88IM2bNig4OBgq/MAAAAAgCOZullXvXr1lJuba3UWAAAAAHAsUytfAwYMUK9evdS8eXMFBf37JdOmTbMsGAAAAAA4ianyNX36dN11112qW7eu1XkAAAAAwJFMla+goCBNnjzZ4igAAAAA4Fymzvlq1qyZNm3aZHEUAAAAAHAuUytfO3bs0IoVK+R2u+V2u2UYhlwul3bt2mV1PgAAAABwBFPlKzY21uocAAAAwA3B4/EoOTlZFy9elGH4O03p5nJJ5cuXV506dRQQYOqgvRua1/K1fft2tW/fXnv37r3q87Vq1bIkFAAAAOAvv/zyi3JzPapevY5crpL/F/6SzDA8Skn5Rb/88osiIiL8HafYvJavjz/+WO3bt9e77757xXMul0tdu3a1LBgAAADgDykp5xQWVo3idQNwuQJUsWJlpaSccn75mjp1qiRp0aJFCgkJKfDcqVOnrEsFAAAA+EleXp4CA02dnQMbBAYGKTc3z98xfMJUne/fv7+OHj2a/3jjxo3q06ePZaEAAAAAf3K5XP6OgH9x0ixMVfohQ4Zo0KBBGjdunHbv3q2tW7dq3rx5VmcDAAAA/C5PAcrK8f1+y7ilQHl8v2MvpkyZpBYtWioqqpet74vLTJWvfv36KTw8XI8++qiqVq2qjz76SJUrV7Y6GwAAAOB3WTnS0o0/+3y/A++urvJun+8WNzBT5Wvp0qWaM2eOJkyYoAMHDujBBx/U9OnTdccdd1idDwAAACjVdu5M0qJFbykwMEinT5/Sbbc10Z///BeNG/c/qlixkoKDgzV79huaO/d17dq1Ux5Pnnr2jNbAgYNlGIZmz56pr77aoqpVw+Xx5KlFi5a6eDFdMTHP6uzZXyRJw4ePUMeOnQrNMGXKJJUtW0a7d3+n9PQ0PfHEk/r004/1j38cVMeOkXr88f9RXl7eVTPk5uZq+vRpOnToH0pJSVHdujfr5ZdnKCUlRePHj1W9evV14MB+ValSRS++OF0VK1a061drO1PlKz4+XrGxsWrYsKEkaf369Xr00Uf11VdfWRoOAAAAgPT3v+/VkiVLVbfuzZow4Wl99dVWHT16RKtWrVHNmjW1atUKSdKSJe/p0qVLevzxUfrjH29TSkqKDhzYr6VLP1BaWroGDx4gSdq06QvVqFFDM2fO0eHDh7RmTYLX8iVdvgR/XNz7+vjjRE2dOlnLl3+okJAQRUd31/Dhj2j9+nVXzWAYhtzuIC1aFCuPx6NRo0Zo27av1LjxH3Xw4AFNmDBJjRo11vjxT2rduk91//0PWPmr9CtT5WvlypUKCQlRbm6uDMNQ165ddeedd1qdDQAAAICkZs2a6+ab/yBJ6t69pxISVqpy5SqqWbOmJOnbb7/WwYP7tXPnt5KkzMwM/eMf/9CRI4cUGXm3goLcqly5sjp0+A9J0h13NNVbb83TmTNn1KHDXRo27JEiM7Rv30GSVL16DdWrd6uqVKkiSbrpppuUmppWaIZ+/e5XxYqVtGLF+zpy5IiOHz+mzMwMSVLlylXUqFFjSVL9+vWVmnrBR7+xG5Op8pWenq5Ro0Zpx44dysvLU6tWrTRjxgyrswEAAACQClz63jA8CgwMKnArqMsrSo+rc+d7JEnnz59TmTJlNW/ebBmG5zf7CZQk1a1bV8uWrdKOHdu0detmLV0ap2XLVnq9smBQ0L9PUPt1P79VWIbNm7/UwoXzNWDAQEVF9dKFC+dlGIYkKTg4+Dd7cOVvdypTl5qfMmWKmjVrpm3btmnbtm1q3bq1Jk+ebHE0AAAAAJK0e/d3On36tDwejz75ZE3+KtSvWrZsrYSE1crNzVFGRoZGjBiuvXt/UJs2bbVhw+e6dOmSUlNTtWPHNknSBx8s08KFb+mee7roqaee0blzKUpPTy9WxsIyfPvt17rnni6KirpPYWFV9d13u+Tx2HuVxxuFqZWvI0eOaPbs2fmPH3vsMfXs2dOyUAAAAAD+rWrVqpoyJUZnzpxR69Zt1bp1W8XG/m/+83369FVy8jENGfKg8vLy1LNnL7Vs2UrS5fPFHnywv8LCwvSHP9STJN17b5RiYp7VoEH3KzAwSMOHj1BoaGixMhaWoWLFipo0aYI2bvxcbrdbTZrcoZMnTxTrvUoql2Fiba9nz55atWpV/tJmZmam+vfvrzVr1lgW7OzZdHk8zl52dLrw8FCdOZPm7xgoJuboDMzROZilMzBH/wsIcCksrMJVn9u79++qWfPm/Mf+vs/X5asdvq358xf6PkQJcfLkUTVpcpu/YxSbqZWve++9Vw8//LD69OkjSVq1apW6detmaTAAAADgRhAoT6m4H9fcubP0zTdfX7G9cePbNGHCc35I5DymyteoUaNUvXp1bdmyRR6PR3369FG/fv2szgYAAACUei1btso/hNBKf/vbGMvfo7QzVb6GDh2q2NhY9e3b1+o8+bI9gcrjsMMS7dS5bF3yXHklHJQszNEZmKNzlNZZhgRJLk+ev2MAQLGYKl9paWnKyMhQuXLlrM6T76NtZ5R6Mde294PvlSnrVlamBQdIw1bM0RmYo3OU1lkOvKe6ypi6RjPgG4ZheL3sOuzjpMvPmypfZcuWVefOndWoUaMCBeytt96yLBgAAADgD2XLllFa2gWFhlakgPmZYRhKS7ugsmXL+DuKT5gqX5zfBQAAgNKiTp06Sk5O1j//eczfUaDLZbhOnTr+juETpspX7969de7cOSUlJSkgIEBt2rQp9n0AAAAAgBuR2+1WvXr1/B0DDmTq6OnPPvtMXbt2VWxsrBYtWqQuXbpox44dVmcDAAAAAMcwtfI1a9YsxcXFqVGjRpKkvXv3auLEiVq9erWl4QAAAADAKUytfJUpUya/eElSkyZNOPkQAAAAAK6BqfLVtm1bLViwQBkZGcrOztb777+vBg0a6MKFCzp//rzVGQEAAACgxDN12OGSJUuUl5enmTNnFtiekJAgl8ulffv2WRIOAAAAAJzCVPnau3dvoc8lJib6LAwAAAAAOFWx7xX/zjvv+CIHAAAAADhascuXYRi+yAEAAAAAjlbs8sVVDwEAAACgaMUuXwAAAACAolG+AAAAAMAGnPMFAAAAADYodvmKjo72RQ4AAAAAcDRT9/n65ptvNHfuXF24cKHASldiYqKGDx9e6OsSExM1f/585eTk6OGHH9agQYOKnxgAAAAASiBT5WvKlCnq27evbrvtNtNXNzx16pRmzZqlVatWKTg4WA888IDatm2rW2+9tViBAQAAAKAkMlW+3G63/vznP1/Tjrdt26Z27dqpUqVKkqRu3bpp7dq1Gj169LWnBAAAAIASzlT5atCggfbv369GjRqZ3vHp06cVHh6e/zgiIkJ79uy59oQAAJj08eJx/o5guYAAlzye0nexq28TghUg53zu4OAgXbqU6+8YN4R58xb4OwJgG1PlKzk5WX379lXNmjUVEhKSvz0xMbHQ11ztKojXckPmkDJBKuPhBs4lXZmybn9HgA8wR2coDXMMCCgdf26Uls/5WwEul4Ldgf6O4VPBwab+GuZ44eGh/o4A2MbUt37MmDHXvONq1aopKSkp//Hp06cVERFh+vXZWbnKyuT/CJVkZcq6lZWZ4+8YKCbm6AylZY49hrzi7wiWKy2z/L2B91RXmYA8f8fwmfDwUJ05k+bvGDcEf/0eAgJcCgur4Jf3Rull6lLzbdq0UcOGDVWnTh3Vrl1bNWrUUE6O9//wd+jQQdu3b1dKSooyMzO1fv16dezY0SehAQAAAKCkMbXyNXv2bC1YcPl43MDAQOXk5OjWW2/1ethhtWrVNGbMGA0ZMkQ5OTnq16+f7rzzTt+kBgAAAIASxlT5SkhI0BdffKGXX35Z48aN09dff61NmzYV+bro6GhuwgwAAAAAMnnYYZUqVRQREaF69erpxx9/1H333aejR49anQ0AAAAAHMNU+QoKCtKxY8dUr149JSUlKTc3V6mpqVZnAwAAAADHMFW+RowYoZiYGEVGRuqzzz5TZGSk2rVrZ3U2AAAAAHAMU+d8de7cWZ07d5Ykffjhhzp69Og13XAZAAAAAEo7UytfFy9e1PPPP6+hQ4cqOztb7733njIyMqzOBgAAAACOYap8TZ06VTfddJPOnj2rkJAQpaen67nnnrM6GwAAAAA4hqnytW/fPo0ZM0ZBQUEqW7asZsyYoX379lmdDQAAAAAcw1T5Cggo+GN5eXlXbAMAAAAAFM7UBTdat26tV199VVlZWdqyZYvi4uLUtm1bq7MBAAAAgGOYWr568sknVa5cOYWGhmrWrFlq3Lixxo0bZ3U2AAAAAHAMUytfbrdbbdq00ahRo3T+/HklJSUpJCTE6mwAAAAA4BimVr5mzZqlOXPmSJKysrK0YMECvfnmm5YGAwAAAAAnMVW+NmzYoHfeeUeSVL16dcXFxemTTz6xNBgAAAAAOImp8pWTkyO3253/2O12y+VyWRYKAAAAAJzG1DlfLVq00NixY9WvXz+5XC6tXr1aTZs2tTobAAAAADiGqZWvmJgYVa1aVaNHj9bjjz+u8PBwTZgwwepsAAAAAOAYpsrX6dOntXXrVrlcLuXm5mrt2rU6efKk1dkAAAAAwDFchmEYRf3Q8OHDFRUVpd69e0uSVq5cqYSEBC1ZssSyYCfPZCrPU2Q03MCCgwN16VKev2OgmJijMzBH5yitswwJklwe53zu8PBQnTmT5u8YpVpAgEthYRX8HQOljKlzvs6ePZtfvCSpb9++Wrx4sVWZJEkhAXnyiPJVkoVXLscfLA7AHJ2BOTpHqZ2lx98BAKD4TB12mJeXp/Pnz+c/TklJsSwQAAAAADiRqZWvwYMHa8CAAerRo4ck6dNPP9XQoUMtDQYAAAAATmKqfA0YMEB169bV1q1b5fF4NGnSJHXo0MHqbAAAAADgGKbKlyS1b99e7du3tzILAAAAADiWqXO+AAAAAADFQ/kCAAAAABtQvgAAAADABpQvAAAAALAB5QsAAAAAbED5AgAAAAAbUL4AAAAAwAaULwAAAACwAeULAAAAAGxA+QIAAAAAG1C+AAAAAMAGlC8AAAAAsAHlCwAAAABsQPkCAAAAABsE+TtAYQICXP6OAB9gjs7AHJ2BOToHs3QG5uhf/P7hDy7DMAx/hwAAAAAAp+OwQwAAAACwAeULAAAAAGxA+QIAAAAAG1C+AAAAAMAGlC8AAAAAsAHlCwAAAABsQPkCAAAAABtQvgAAAADABpQvAAAAALDBDVW+EhMTde+996pLly6Kj4/3dxz8y5AhQ9SzZ0/dd999uu+++7R79+5CZ7Vt2zZFR0era9eumjVrVv72ffv2qW/fvurWrZsmTJig3NxcSdLJkyc1aNAgde/eXSNHjtTFixdt/3xOl56erqioKB0/flyS72aUmpqqv/71r+rRo4cGDRqkM2fOSJIuXbqkp556Sj169FDv3r31008/2fyJnen3c3zmmWfUtWvX/O/lZ599Jsn6+aJ45s2bp549e6pnz56aPn26JL6TJdHV5sh3EoApxg3i559/Njp37mycO3fOuHjxohEdHW0cPHjQ37FKPY/HY/zHf/yHkZOTk7+tsFllZmYanTp1Mo4dO2bk5OQYw4YNMzZt2mQYhmH07NnT+L//+z/DMAzjmWeeMeLj4w3DMIy//vWvxpo1awzDMIx58+YZ06dPt/kTOtt3331nREVFGU2aNDGSk5N9OqPnn3/eePvttw3DMIzVq1cbjz/+uGEYhrFo0SIjJibGMAzD+Oabb4x+/frZ94Ed6vdzNAzDiIqKMk6dOlXg5+yYL67fV199ZQwYMMDIzs42Ll26ZAwZMsRITEzkO1nCXG2O69ev5zsJwJQbZuVr27ZtateunSpVqqRy5cqpW7duWrt2rb9jlXqHDh2Sy+XSI488ol69eikuLq7QWe3Zs0c333yz6tSpo6CgIEVHR2vt2rU6ceKEsrKy1KxZM0lSnz59tHbtWuXk5Ojbb79Vt27dCmyH7yxfvlyTJk1SRESEJPl0Rps2bVJ0dLQkKSoqSps3b1ZOTo42bdqkXr16SZJat26tc+fO6eTJk3Z/dEf5/RwzMjJ08uRJxcTEKDo6WnPmzJHH47Flvrh+4eHhGj9+vIKDg+V2u1W/fn0dOXKE72QJc7U5njx5ku8kAFOC/B3gV6dPn1Z4eHj+44iICO3Zs8ePiSBdPsyhffv2mjx5srKysjRkyBD16NHjqrO62gxPnTp1xfbw8HCdOnVK586dU4UKFRQUFFRgO3znxRdfLPDYlzP67WuCgoJUoUIFpaSkXHVfP//8s2rWrGnZ53S638/x7NmzateunaZMmaJy5cppxIgRWrFihcqVK2f5fKtVq2b1x3WsBg0a5P/zkSNH9Mknn+ihhx7iO1nCXG2O7733nr755hu+kwCKdMOsfBmGccU2l8vlhyT4rebNm2v69OkqV66cqlSpon79+mnOnDlX/JzL5Sp0hte6HdaxekYBAVf/T0ph23F96tSpozfeeENhYWEqW7asHnroIX355Zd+my+uzcGDBzVs2DA9/fTTqlu37hXP850sGX47x3r16vGdBGDKDfOtrVatmn755Zf8x6dPn84/xAb+k5SUpO3bt+c/NgxDtWrVuuqsCpvh77efOXNGERERqlKlitLT05WXl1dgO6zjyxlFRETkvyY3N1fp6emqVKmSIiIiCpwIzlx9b//+/Vq3bl3+Y8MwFBQUZMt8UTw7d+7Uww8/rLFjx6p37958J0uo38+R7yQAs26Y8tWhQwdt375dKSkpyszM1Pr169WxY0d/xyr10tLSNH36dGVnZys9PV2rV6/Wq6++etVZNW3aVIcPH9bRo0eVl5enNWvWqGPHjqpVq5ZCQkK0c+dOSdKHH36ojh07yu12q1WrVvrkk08KbId1fDmjTp066cMPP5QkffLJJ2rVqpXcbrc6deqkhIQESZfLe0hICIc3+ZhhGHrppZd04cIF5eTk6P3331eXLl1smS+u3z//+U+NGjVKM2bMUM+ePSXxnSyJrjZHvpMAzHIZV1vj9pPExES9/fbbysnJUb9+/fTII4/4OxIkvf7661q3bp08Ho8efPBBDR06tNBZbd++XdOmTVN2drY6deqkZ555Ri6XSz/++KMmTpyoixcv6rbbbtO0adMUHBysEydOaPz48Tp79qxq1KihmTNnqmLFin7+xM5z9913a8mSJapdu7bPZnT+/HmNHz9eycnJCg0N1YwZM1S7dm1lZ2frueee0w8//KDg4GBNnTpVTZo08fevwBF+O8f4+HjFx8crNzdXXbt21ZNPPinJd9/BwuaL6zd16lStXLmywKGGDzzwgP7whz/wnSxBCpujx+PhOwmgSDdU+QIAAAAAp7phDjsEAAAAACejfAEAAACADShfAAAAAGADyhcAAAAA2IDyBQAAAAA2oHwBAAAAgA0oXwAAAABgA8oXAAAAANjg/wMtTshTR9BrwgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# prepare figure \n", "fig, (ax0, ax1) = plt.subplots(nrows=2, ncols=1, figsize=(12,7), sharex=True)\n", "\n", "# plot ground truth\n", "preds_quality.plot('ocean_proximity_INLAND', 'median_house_value', 'barh', ax=ax0)\n", "# put legend outside plot\n", "ax0.legend(loc='upper left', bbox_to_anchor=(1.0, 0.5))\n", "\n", "# plot preds\n", "preds_quality.plot('ocean_proximity_INLAND', 'preds_mean', 'barh', xerr='preds_std', alpha=0.6, ax=ax1)\n", "# put legend outside plot\n", "ax1.legend(loc='upper left', bbox_to_anchor=(1.0, 0.5))\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The error bars in the plot indicate how confident the model is at predicting each category. Alternatively, we can compare the _distribution_ of values to inspect how close the predictions match the ground truth. For example, for housing districts where `ocean_proximity_INLAND` is 0 we have:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/leandro/git/dslectures/env/lib/python3.7/site-packages/seaborn/distributions.py:2557: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).\n", " warnings.warn(msg, FutureWarning)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhAAAAEJCAYAAADFMR5HAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3de1xUdfoH8M8wM8yAgCgOIoiWl0S0xJQUNUlXEUXE25aXlZ+5qWVlauv9klkaubaaq+5aZm6pm5cUjYyyDLfCFMlL3k0EEZT7VZhhmPn+/iAmkIFhuA0Dn/fr5UvmzDPnPOcwMA/nfM/3kQghBIiIiIjMYGPpBIiIiMj6sIAgIiIis7GAICIiIrOxgCAiIiKzsYAgIiIis7GAICIiIrOxgCAiIiKzySydwMOysh5Ar7f+qSlcXByQkZFv6TQAADsv7UF87h084tQBM3pObfDt18WxWLx4AX799QIef7wX3n33H3WUWcP7/uVI3P35Ltr2aYeAD4IsnY7FNaafE0ur6bGwsZGgVasW9ZARUdUaXQGh14smUUAAaDT7ka3ORVpBJpxtnS2WU223m5qairt376Jt23aN5rjWRP79fOQk5KCFh6NV70dd4nH4A48FWRNewiAiIiKzsYAgIiIiszW6SxhU9/q164uuzp3Q2q61pVOpsTFjxqFv36fg7u5h6VRqxWe6D1S+7eDUoaWlU6FmRq/XIzExEQ8ePAA7IJEpEgnQokULeHp6wsbG+LkGFhDNgF+7vpZOodZCQsZbOoU64TPdBx5peZZOg5qh9PR0FBfr4ebmCYmEJ5+pakLokZmZjvT0dLi6uhqN4buIiKgZyMzMgpNTKxYPVC0SiQ1atmyFrKysSmP4TiIiagZ0Oh2kUp50puqTSmUoLtZV+jzfTc3AqXtnkVmYidZ2ra32csaRI4eQnJwEd3cPq76ccX7XeSRdToFTh5bwmtTD0ulQMyORSCydAlkRU+8XnoFoBn6+dxbH4r9FdNJZ5Behyn/FonH+gjl69DC2b9+Ko0cPWzqVWjm/6zzObvgZ1z67bOlUiCwmOTkZ/fs/ibCwt8stv3HjOvr3fxIREUcbPKcHD/IxffpUTJs2CXfuJNTbdl56aSZiY8/i6tUrWLt2TaVx+fl5WLRoAQAgLS0N8+e/Wm851RTPQDQDen3J/7kFxYi5lltlrK+XExxsGyApImrWWrZ0xs8/R/9+aUUKAPj222/QqlUri+Rz48Z1yOVy7Nq1p0G21727N5YvX1Xp87m5ebhx4zoAQKVSYePGfzZIXuZgAUFERA3Ozs4Ojz3WDefP/4I+fXwBAKdPn4Kvbz9DzKlTP+HDD/+N4uJitGvnjmXLVqJlS2d8991x7N37KTQaDTQaDZYtW4nevfvgpZdmwtu7By5cOIfs7CwsWLAYAwYMLLfdjIwMrFv3Ju7fvw+pVIaXXnoZ3bp1x9q1byIjIwN/+9s8bNiwyRAfEXEUUVEnkJubi8zMDAwaNBivvbYAv/wSi61b34dOp0Pnzl3wt78twYYNYbh16zfo9XpMmzYdAQGBKCoqwrp1a3D16hW0a+eOnJxsAEBs7Fns2LEd//rXh7hx4zrCwtZCo1HDyckJb765Fv/4x3qkp6dh8eLX8dprr2POnJkID//SaP5+fgPx4Yf/RlpaGhIT7+D+/XsYM2Ysnn/+Bdy8eQNhYW9Dp9PB1laBFStWo0OHDnXyPWQBQUTUDIWHH0J4+KEqY7y8umPJkuWGx9euXUFY2LoqXzN27HiMHVu9cUp/+tNwnDjxLfr08cWVK5fRpUtXwxwVWVlZ2Lbtn9i69QM4OTnh8OGD2LJlM5YuXYHDhw/ivffeh7NzK3zxRTh27/4EvXv3AQAUF2uxY8d/8MMPJ7F9+9YKBcQ//rEeffo8hSlT/oKkpLuYPXsG/vOf/2Lp0lXYsWN7ueKh1NWrV/DJJ/+Fk5MT5syZiaioE3Byaok7dxIQHv4lHBwcsXXrZnTr1h2rVq3Bgwf5mDnzefTo0RNRUScAAPv2HcKdO3cwbdpzFdb/xhvL8fLLczFo0GB8/vkB7Nu3FwsWLMKcOTPx7rvvITk52WT+APDbbzexfftHyMvLw8SJYzBx4nP47LM9mDJlGv70p+E4fvxrXL58kQUEERHVXFLSXZw9e8as1+Tm5pl8ja/vU9Ve36BBg7F9+zbo9Xp8++03GDYsAMePfwMAuHz5V6Sk3MfLL88CUDIRlpOTE2xsbPDuu+/hhx/+hzt3EvDLL2dhYyM1rLN//wEAgM6duyA3t+Il27NnY7B06QoAgIdHe/To8TguX76EFi0qb0j29NOD4eLiAgAYNmwEYmNjMGTIMHTo8AgcHBwBADExp6FWqxERcQQAUFhYiLi4W/jll7MYO3YCAKBDhw54/PEnyq07OzsLGRnpGDRoMABgwoQ/A0C5oqE6+QNAnz59IZfL0bp1azg5OSE/Pw8DBz6Nv/89DD//HI2BA5/G0KHDKt1Pc7GAICJqhjw82qNv36o/7L28upd77OTkaPI1Hh7tq51DixYt0LXrY7hw4RzOno3BnDmvGgoIvV6PJ57wMZwR0Gg0KCgoQEFBAZ5/fhoCA4Pg4/MkunTpigMH9hnWaWur+P0rCYSRKTeF0D/0WECnK64yz9IxGqWvL32sUCgMy/V6PVavfttwzDIyMtCypRPCww+Vy6PsugBAJiv/MazRaJCenlbpfB1V5W9rW3YAmwRCAEOHDkPPnk/gp5/+h3379iI6+icsW7ayyv2tLhYQRETNkDmXGkp5eXlj167ddZrHn/40HNu2/RPdu3cv92Hao0dPrFu3BnfuJKBDh47YufNDpKWl4dlnJ0EikWD69BkAgHfeeQt6feVzFTysTx9fHD16xHAJ4OLF81i0aCkSEiq/8+LUqWjk5+dBLrfFN998jVmzXjS63kOHDmLZspVIT0/DtGmT8eGHH+Opp/rh66+/wqBBg5GSch+//nqx3OscHBzh6toWp0//jH79+uOrr77EuXOxePnludDpKu5XZfn/9ttNo7kvX74Yw4ePwLhxE/HII49i06b3qn2sTGEBQUREFjNo0GCsXbsGs2a9VG65i0sbLF/+BpYvXwy9Xg+VyhVvvvk2HBwc8dhj3fDcc+OhVCrRu3cf3L9/v9rbW7BgEcLC3sKXX5bcKrps2Sq0aaOqsoBo1aoV5s+fi5ycbAQGBqF//wGIjT1bLuaFF2Zh/fp3MGXKn6HT6fDKK6+hfXtPTJjwZ9y6dQuTJk2Am1s7dOrUucL6V69+G+vXv4MtWzahZUtnrF79FpydneHm5oY5c2ZhxYrVJvOvzPTpf8W6dWuwc+cOyGRSvPba69U+VqZIhLFzPA/ZsmULvvrqKwCAv78/Fi1ahKVLlyI2NhZ2dnYAgFdeeQXDhw9HdHQ03nnnHWg0GowcORLz5883K6GMjHzo9dbf6UWlckRaI+l5sPfaUVxPvQNnmRt8HAKqjK2P2zjr4lisX78O169fRbdu3bFo0bI6yqzhxa79CYkxSWjTU4VBbw+xdDoW15h+TiytpsfCxkYCFxcHk3GXL1+Bu3vHmqTWrEVEHMUvv8Ri1ao3LZ2KRSQnJ6BHD2+jz5k8AxEdHY0ff/wRhw8fhkQiwQsvvIDjx4/j0qVL2L17d7kmG2q1GsuWLcOnn36Kdu3aYfbs2Th58iT8/f3rbm/IbGM6jUFMUdXzPzR21lw0lBW4KZAfmETUJJiciVKlUmHJkiWwtbWFXC5H586dkZycjOTkZKxcuRLBwcHYvHkz9Ho9Ll68iI4dO8LT0xMymQzBwcGIjIxsiP0gIiKqc6NHj2m2Zx9MMXkGomvXroav4+PjcezYMezduxdnzpzBmjVrYG9vj9mzZ+PgwYOwt7eHSvXHtRhXV1ekpKTUT+ZERERkMdUeRHnz5k3Mnj0bixcvRqdOnbB161bDc9OmTUN4eDgCAwMrvM7c5i3VuZZnLVQqR0unAAC4k3ALebJM2NrYwUXRrspYe3tbqFopqoypidoei0uXLiE3NxdOTk7o2bNnHWXV8O6fvw91thpKZyXcfNwsnU6j0Fh+ThoDHguyJtUqIGJjYzF37lwsW7YMQUFBuH79OuLj4zFixAgAJfehymQytG3bFunp6YbXpaamlhsjUR0cRFn3Dlw5hLjcOLSRdcAzzqFVxhYWKnC7oKja61bKJZBJqv5+1cWxWLZsBWJjY9Cnjy8++ujTWq3LkiLnRSLhZALcB7TH2PBnLZ2OxTWmnxNLq+9BlER1zWQBce/ePbz88svYuHEj/Pz8AJQUDOvWrUP//v1hb2+Pffv2Ydy4cejVqxdu376NhIQEtG/fHhEREZgwYUK97wTVnaJigfM3qz/gks23iIiaJ5MFxEcffQSNRoOwsDDDskmTJmHWrFmYPHkyiouLERAQgNGjRwMAwsLC8Oqrr0Kj0cDf39/oZQ0iIiKybiYLiBUrVmDFihVGn5s6dWqFZX5+fjh6tOF7uRMRkXUo24myOeXQv/+T+PnnX8otE0Lg4493ICrqBNRqNZ5//q8YObLkD/Ldu/+DiIiSz9MxY8ZhypS/AABWrlyKW7d+w6BBgzFnzqsAgE8/3YXOnbtgwIBBRrednp6Gf/5zE27cuA6pVIq2bdtiwYJF8PBoj+TkZEO3T3OYvI2TiIiI6kdk5DGcOfMzPvroE/zrXx9i8+ZNyMvLQ2LiHXz++QHs2rUHH3+8G/v370Vi4h3cvHkD+fn52Lv3AH766Qfk5+chJycHly79WmnxUFhYiJdemonevZ/E3r0HsHv3PgwfHoi5c+eguFhb49w5lTURUTMUnRSDU8kxVca0d/TAc14hhseJuUnYf/1Ila/xc/fFAA/fGue1a9dHiIw8BqlUiqee6o9XXnkNixYtwIQJz2LAgIH417+24Pr1a9i0aQvS09Mwd+4c7N17AMeOReCzz/ZCCD28vLrjb39bAoVCgcDAofDy6o6MjAx8/PGnkMnkhm3l5GRj3ryXkZaWBm/vnli4sGTOox9//J+hS6iHR3ssXrwcLi4uGDs2CNu2fQh3d/dyZzBeemkmvL174MKFc8jOzsKCBYsxYMBAJCcnY/XqFSgsLEDPno8b3d9vv/0GU6aEQi6Xw8WlDbZv3wmFQgEhBIqLtSgqKoIQAkKUNN6SyWTQaotQXFyM4uJiSKUy7Nz5b/zf/82o9JgeP/412rRRGbqCAkBg4CjY2tqiqIgFBBERmSGjMBM3suLMek1BcaHJ1zzWqmKvh+qKjv4RP/xwErt27YFMJsPSpQtx+PBBDBz4NM6ePYMBAwbi/PlfkJKSAp1Oh59/joaf30DExd3CkSOH8OGHH0OhUGDbtn9iz55PMWPGC8jOzsa0ac+jT5++FbaXnJyEsLD34OnpiRUrluDw4YMYPjwQ7767Ftu3fwx3d3fs3v0fvPfeu1i3bn2VuRcXa7Fjx3/www8nsX37VgwYMBDvvReGoKBghISMw1dfReDw4c8rvO7u3UTEx9/GJ5/sRFGRFqGh09GhQwd06NARw4cHYuzYIAACY8aMRbt27gCALl26Yvr0qRg/fiKysrKQlZUJb+8eleZ248Y19OhR8fb30tbe2dnZVe5bZVhAEBE1Qy52rfFYq05VxrR39Cj32F5mZ/I1Lnata5zT2bMxCAgIhFKpBAAEB4fgyy+/wIIFC7Fw4Xw8ePAAANC162O4fv0aTp2KxsSJzyI2NgaJiYl44YX/AwBotVp06+ZlWK+xD08A8PF5Eh06dAAAjBgxCl9+eRQeHu3h7d0T7u4lH9Zjx47HJ598bDL3/v0HAAA6d+6C3NySO9l++SUWa9a8Y1j/2rVrKrxOp9Pht99u4t//3oGMjAzMmjUDjz3mhaSkRFy7dhUREV9DCD3mz5+Lb7/9BsOGBWDevL8ZXr9mzRuYMWMmDh7cj1OnfkSPHk9gxowXym1DIrEx2tq8tlhAEBE1QwM8zL/U4Onkgdd959RTRoBery/3WAgBnU6Htm3doNfr8f333+GJJ3zQunVrnD17BteuXcUTT/jgxo3r+NOfhuP11xcBAAoKCsq1wi4tSB4mlUrLbg0ymazCPERCAMXFJesqmRix5Pni4uJycba2pRPwSQwf1hKJBELoDV/b2FQcduji4oKhQ/8EmUyOtm3d0LPn47hx4xrOnYvFkCFDYW9vDwAICBiBc+diMWzYHw0Rr127ghYtWqB1axccOnQAe/bsx/z5rxhaoJfq3t3b0L2zrLVr12Dy5KlQKu2MHh9TOIiyGZj9xIuY2GaFyUmkGrOPPvoU589fs+pJpABgetR0zEldwEmkiIzo29cX33wTCbVajeLiYkREHDVcevDzG4hdu3bgySf7oG/fp3DgwGfo0aMnpFIpnnyyL06e/B6ZmZkQQmD9+nX47LM9Jrd38eJ53L9/D3q9Hl9++QV8fZ9Cjx49cenSr0hOTgYAhId/bsjB2dkZcXG3AAA//BBlcv2+vv0QGXkMAPD99ydQVFRxkr5Bgwbju++OQwiBnJxsXL78Kx57rBu6dn0MP/30A3Q6HYqLtTh16id0717+MsXOnTswY8bMckWKRGIDjUZTLm7o0GG4d+8ejh4NNyyLiDiCc+fOon17T5P7URmegSAiogZ34cI5DBky0PA4MHAUFi9ejhs3buD55/8CnU6Hfv388Oc/TwIADBw4CHv37kavXr1hZ2cHrVaLgQOfBlBySeOvf52FV16ZDb1ej8ce80Jo6PMmc3j00c54++03kZGRjj59fBEcPBZSqRRLlizHkiWvQ6vVws2tHZYvXwUAmDnzRbz33np89NEH6NfPz+T6X399Md58cwUOH/4c3t49YG/fokLM5MlTsWXL+5gy5c/Q6/V4/vmZ6NChI9q390R8fDymTPkzpFIpBgwYhKCgYMProqN/hJdXd7Rq1QoA8NRT/TF58kR07+6Nrl0fK7cNpVKJf/7zX9i06T3897+7IZFI4O7ugfff3wZb25rPBCgR9XFhpBY4lXXdyy8CYq5Vb3ZJn65OdT4TZWM6FpbGY1Eej8cf6nsq68uXr8DdvaPJOKKykpMT0KOHt9HneAmDiIiIzMYCohnYfvHfOJj+NqKyP7F0KjX2179Og4+PF/7612mWTqVWdj2zC9tc/4HwsfstnQoRUa2wgCAiIiKzsYAgIiIis7GAICIiIrOxgCAiIiKzsYAgIqIGlZyc/HuPh/L693+ywXKIjT2Ll16a2WDbA4zvnxACO3d+iNDQyXj22ZKeGaV27/4PJk2agEmTJmDv3t2G5StXLsWUKX/Gtm3/NCz79NNdiI7+sdJtp6en4Y03lmPy5In4y1+ew+uvz0VS0l0AlX8/TOFEUtQgFFBDolWbjBNyJTQwPu0sEVFTU7add25uDv7yl0kYNMgf2dlZ+PzzA/jvfw9CCIHJkyfg6acHQ61WG9p5T536LEJDp0On0+PSpV8xbdp0o9sobec9deo0rF79NiQSCSIjj2Hu3DnYt69ig6/qYgFBDUKiVUPz21mTcYoufQE5Cwii5iwi4iiiok4gNzcXmZkZGDRoMF57bQF++SUWO3b8G1KpDKmpKfD27oFly1bB1taW7bzZzpuIiBrClb2XcPW/l6qMUfV0xeB3hhoep/2aiv8tO1Hla7pP7gnvKca7X5rj6tUr+OST/8LJyQlz5sxEVNQJODm1xJUrl/HJJ/9Fhw4dsXz5Yhw8uB/9+/uxnTfbeVN9CO40Bufi0mFrY71/2S9cuAx5eblwdHSydCq1ErgpEPfjM6FwUpgOJqpHuXdykPTTXbNeo8lRm3yNx0DTzZlsbCQVlgkhfu92WeLppwfDxcUFADBs2AjExsZgyJBh8PHpjY4dHwEABAYG4ciRzyGXy9jOm+28qT64O7gjydb0XPkP6+QioJRoqoxxEnrIUf/jFry8utfr+huKm48bpB4VG+oQNTSnDi3hMbB9lTGqnq7lHitaKk2+xqlDS5PbdnR0woMH+eWWZWZmlvsDoWyrbSH0hsdSqeyh5TLo9Xq287ZAO28WEFQppUSDnCtnqoxp5aaEbTeOWyCyNt5TzL/UoHrcFRO+mFTrbbdo0QKenh1w4sR3GDr0TwCAI0cOwdf3KUPMqVPRyM/Pg1xui2+++RqzZr0IALhw4TxSU1PRpk0bHDsWAT+/AejZ8wns2fMpnn/+BbRq1Qrr16+Dh0d7zJz5YpV5lLbzdnVtiy+//AJ+fgPQo0dPvPvuWiQnJ8Pd3d1oO293dw+z2nlPnPicyXbe/v5DkJubg8uXf8Xs2XOQn5+HqKgTGD/+zxBCj1OnfsKQIcPKvXbnzh1YunRltdp579ixHUePhmPMmLEA/mjnvXDhEqSnp5vcF2NYQBARUYNbvfptrF+/Djt3fgCtVosuXbpi4cIlhudbtWqF+fPnIicnG4GBQejffwBiY8+iTZs2WLNmJdLS0uDr2w9jxoyDVCplO2+282Y77/qw99pRXE+9A2eZG3wcAqqMLdvO27uN2uQZiEfclHDq1hdquXOlMSqVI/KSE6t9F4axda1fvw7Xr19Ft27dsWjRMpPraaxi1/6ExJgktOmpwqC3h1g6HYtrTD8nlsZ23n+IiDiKX36JxapVb5ZbXvbOB2oYVbXz5hmIZuBefjLSi+9YOo1auX79KmJjYyydRq3dP38fydHmDVwjImqMWEAQEVGjMnr0GIwePabC8j59+hq9HZMsgwUE1ZrURkCprfw+Yl12AaTQVfo8ERFZHxYQVGuSYg00cZVPSGPrqIRE1aUBMyIiYx6ea4GoKqaGSLKZFhFRM2Bnp0ReXk69TChETY8QAnl5ObCzq/wWfZ6BICJqBjw9PZGYmIh796x7QDU1HDs7JTw9K59ZlAUEEVEzIJfL0alTJ0unQU1ItS5hbNmyBUFBQQgKCsL69SUNRaKjoxEcHIyAgABs3LjREHv16lVMmDABI0aMwPLlyytM90lERETWz+QZiOjoaPz44484fPgwJBIJXnjhBURERGDDhg349NNP0a5dO8yePRsnT56Ev78/Fi5ciLfffhs+Pj5YtmwZ9u/fjylTpjTEvlAl+rTtC4XWAy2klU/21NiNGTMOffs+BXd3D0unUis+032g8m1XrX4BRESNmckCQqVSYcmSJYbpLjt37oz4+Hh07NjRcG0kODgYkZGR6NKlC9RqNXx8fAAA48ePx+bNm1lAWFjftn0hsnItnUathISMt3QKdcJnug88OPMiETUBJi9hdO3a1VAQxMfH49ixY5BIJFCpVIYYV1dXpKSkIDU1tdxylUqFlJSUekibiIiILKnagyhv3ryJ2bNnY/HixZDJZLh9+3a550valla8Pcjce46rM6e7tVCpHC2dAgBAn6WBg2P1umXK5FJDrK2tFgqFvMp4W1sZ5HIpHE2svzoxACCzt4Wjc+M4bvWlsbwvGgsejz/wWJA1qVYBERsbi7lz52LZsmUICgrCmTNnyrX/TE1NhaurK9q2bVtueVpaGlxdXY2tslJsplX3/pd4Fpfu30MLqTMeUfaqMrZYa4v8PDUAoEihg0ajrTK+qEgKrVaHB7+/xhhHR6XJmFKKgiKotRWP25Ejh5CcnAR3dw+rvpyR9OUtJF1OgVOHlvCa1MPS6VhcY/o5sbT6bqZFVNdMXsK4d+8eXn75ZWzYsAFBQUEAgF69euH27dtISEiATqdDREQEBg8eDA8PDygUCsTGxgIAwsPDMXjw4PrdAzIpNuUsrhb+gHj1BUunUmNHjx7G9u1bcfToYUunUivnd53H2Q0/49pnly2dChFRrZg8A/HRRx9Bo9EgLCzMsGzSpEkICwvDq6++Co1GA39/fwQGBgIANmzYgBUrVuDBgwfw9vZGaGho/WVPREREFmGygFixYgVWrFhh9LmjR49WWObl5YWDBw/WPjMiIiJqtDgTZTPUyUVAKdEYfa4V9PBuUwQAcLYTyGnIxIiIyGqwgGiGlBINcq6cMfqck6sSOaklgx3b9OndkGlVUCwkUGtLBtTqfh9XqxNAfpHxeKVcApnE+gfgEhFZAxYQ1GiptQIx10omwMor0Bn+L132MF8vJzjYNlh6RETNGgsIqiUJdALQ6CqPEAXFsPs9Ri6VwAY8S0BEZO1YQFCt6PQC+YU6xN+vfI4HhUKO9qqSmEfclFBIK1+f1EZAqc0GANgIGMZjtJDrDf97t1FDLRSIyzBvkjIiIqo7LCCagXYO7sgtKIazzM3SqZgkKdZAE3cJQMkZi5zfC5P2zo4o7tQJ7Z0dkXPlDFp6PwWgerNrNiZuPm7QanVo01NlOpiIqBFjAdEMjOk0BjFF1t1Ma3pIiKVTqBOBmwI58yIRNQksIMgqObewgTfKXzZxEnrYPjTztpArobHCMxVERI0dCwiySjK9BjlXzpVb1spNCfHQ+ApFl76AnAUEEVFdYwHRDCTnJyO1KB22NkqrGAdhTHxSEh4UFqKFnR0e8fCwdDo1dv/8fdyPz4TCSYE2j5vXaI6IqDFhAdEMfBF3FHG5cWgj64BnnK2zN8muI0dwJS4O3p06YfWcOZZOp8Yi50Ui4WQC3Ae0x9jwZy2dDhFRjZnsxklERET0MBYQREREZDYWEERERGQ2FhBERERkNhYQREREZDYWEERERGQ23sZJDUwCja7ybpzKMp09hWCzLCKixooFBDUonV4gMbXyzp2dPf7o7OnpyhkkiYgaKxYQzcDsJ15EzDXrbqZlzZNHlTU9ajqbaRFRk8AxEERERGQ2FhBERERkNl7CsELFQgK1tvKBiA/TczAiERHVMRYQVkitFWaNaThbtBfxVt5Ma/W2bU2imdauZ3axmRYRNQm8hEFERERm4xkIakIqzjEh0QP5RcajlXIJZJLqXwoiIqI/sICgJsPYHBMtWxfhSrrxyz2+Xk5wsG2IzIiImh4WENSkObewgTeMT1zlJPSw1ZZ8LeRKaMCJq4iIqosFBDVpMr0GOVfOGX2ulZsSQlrytaJLX0DOAoKIqLo4iJKIiIjMVu0CIj8/H6NHj8bdu3cBAEuXLkVAQABCQkIQEhKC48ePAwCio6MRHByMgIAAbNy4sX6yJiIiIouq1iWMCxcuYMWKFYiPjzcsu3TpEnbv3g1XV1fDMjrEGdkAAB5bSURBVLVajWXLluHTTz9Fu3btMHv2bJw8eRL+/v51njgRERFZTrUKiP379+ONN97AokWLAAAFBQVITk7GypUrkZycjOHDh+OVV17BxYsX0bFjR3h6egIAgoODERkZyQLCwoIeGYNf49Nha2O91/inh4TgQWEhWtjZWTqVWgncFIj78ZlQOCksnQoRUa1Uq4BYu3ZtuccZGRno378/1qxZA3t7e8yePRsHDx6Evb09VCqVIc7V1RUpKSl1mzGZrV0Ld6TYOlg6jVp5xMPD0inUCTcfN0g9Wlg6DSKiWqvRXRienp7YunWr4fG0adMQHh6OwMDACrESiXl9GFxcrPuDriyVyrFe1qvP0sDBsfpnE2Ryabl4W1stFAq50VgbqY3hOWmZrytjI7WpVlxpjI2J2LLrqirW2DaNxVeVm0wmhbApeX8KqRR6WeWTQtgrpHC0r5ublurrfWGteDz+wGNB1qRGvxGvX7+O+Ph4jBgxAgAghIBMJkPbtm2Rnp5uiEtNTS03RqI6MjLyoddb/+yAKpUj0tLy6mXdBUVAfp7xuQ2MKdbalosvUuig0WiNxup1UsNzOp2+0riy8abiFAq5Iabs+o0pu66qYo1t01h8VblpiqSGiadaOhTiSpn37sN8vZygflDp09VWn+8La8Tj8YeaHgsbG0mT+sOLrEeNCgghBNatW4f+/fvD3t4e+/btw7hx49CrVy/cvn0bCQkJaN++PSIiIjBhwoS6zpkq0clFQCnRVFj+3e2vEffgHtyUrhjRdiic7QRyLJBfbew6cgTxSUl4xMMD00NCLJ1OjUXOi0RiTBLa9FRh0NtDLJ0OEVGN1aiA8PLywqxZszB58mQUFxcjICAAo0ePBgCEhYXh1VdfhUajgb+/v9HLGlQ/lBINcq6cqbA8wfY3JBRnobggDzkZDmjTp7cFsqud+KQkXImLs3QatXb//H0kR9+1dBpERLVmVgFx4sQJw9dTp07F1KlTK8T4+fnh6NGjtc+MiIiIGi3ORElERERmYwFBREREZmMBQURERGZjAUFERERmYwFBREREZmMBQURERGarm7l5qVHzVbpDlecIJ2G9zbSe8fWFd+fOcG3d2tKp1IrPdB+ofNvBqUNLS6dCRFQrLCCaAV+lB9xyqz/1dWP0jK+vpVOoEz7TfeDBqZuJqAngJQwiIiIyG89AEAFwbmEDb1R+lsZJ6GGrBYRcCQ2s91IQEVFdYQHRDMSokxAvyYOTUMIbbpZOp0aiYmKQmpkJ19at6+VyhkyvQc6Vc5U+38pNCSEFFF36AvKaFxDnd51H0uUUOHVoCa9JPWq8HiIiS+MljGYgRp2M05I7uCJJsXQqNRYVE4ODx48jKibG0qnUyvld53F2w8+49tllS6dCRFQrLCCIiIjIbLyEQWQGqY2AUpttMo5jJYioqWMBQWQGSbEGmrhLJuNqO1aCiKix4yUMIiIiMhsLCCIiIjIbCwgiIiIyG8dANALFQgK1VlQ7Xi8k9ZgNERGRaSwgGgG1ViDmWm614326Opm1fg+ZIzRaPVRwMDe1RuMRD49y/1srNx83aLU6tOmpsnQqRES1wgKiGQhx8EJigXU305oeEmLpFOpE4KZApLGZFhE1ARwDQURERGbjGQiiapFAoxNQCkCjMx0tB8epEFHTxgKiGUgqzsVdFEABmdWOg4hPSsKDwkK0sLOzyDgInV4gMVWNzh46xN83fTnIq6vxQbH3z9/H/fhMKJwUaPO4a12nSUTUYFhANANH8q8jziYLHqIlJopelk6nRnYdOYIrcXHw7tQJq+fMsXQ6NRY5LxIJJxPgPqA9xoY/a+l0iIhqjGMgiIiIyGwsIIiIiMhsLCCIiIjIbCwgiIiIyGwsIIiIiMhs1Sog8vPzMXr0aNy9excAEB0djeDgYAQEBGDjxo2GuKtXr2LChAkYMWIEli9fjuLi4vrJmoiIiCzKZAFx4cIFTJ48GfHx8QAAtVqNZcuWYdu2bTh27BguXbqEkydPAgAWLlyIlStX4uuvv4YQAvv376/X5ImIiMgyTBYQ+/fvxxtvvAFX15JJby5evIiOHTvC09MTMpkMwcHBiIyMRFJSEtRqNXx8fAAA48ePR2RkZP1mT0RERBZhciKptWvXlnucmpoKleqPToKurq5ISUmpsFylUiElJaUOU6WamuPsi8RU626mZc2TR5U1PWo6m2kRUZNg9kyUQlScolcikVS63FwuLtY51bIxKpVjteL0WRo4OCqrvV6ZXGo03tZWC4VCXmG5jdSm3HLpQ48ri60qrmx8deJKYx7OpbI4Y3lXFmcs96rijMWb2ofS2OrsKwDI5VI4VfL9r+77orng8fgDjwVZE7MLiLZt2yI9Pd3wODU1Fa6urhWWp6WlGS57mCMjIx96vfE+AtZEpXKs9l+aBUVAfl7lZwg6uQgoJRrDY0dtATooiirE2UsF0jTaCsv1Oik0ZZbrdPpyjyuLrSqubLypOIVCboh5OJeHlV1XVbHGtmksvq72tTS2OscEALRandHvvznvi+aAx+MPNT0WNjaSJvWHF1kPswuIXr164fbt20hISED79u0RERGBCRMmwMPDAwqFArGxsejTpw/Cw8MxePDg+si50SsWEqRkaVBQ8TPeKL2o+kyNUqJBzpUzhsdOrkrkGLkk0aZPb7PypPpjKxWANrvCcl12AZTaP94YQq6EBtU/+0RE1FiYXUAoFAqEhYXh1VdfhUajgb+/PwIDAwEAGzZswIoVK/DgwQN4e3sjNDS0zhO2BmqtwNX4/CrPKpTl09WpXvPZlh1j9c20Vm/bZlXNtCTFGqhvX6qw/MCC20iMzYO7TwuMe78zFF36AnIWEERkfapdQJw4ccLwtZ+fH44ePVohxsvLCwcPHqybzBqZYiGBWlu9SyumzigQERFZO7bzria1ViDmWm61Yuv7jAIREZGlcSprIiIiMhsLCCIiIjIbCwgiIiIyG8dAEDVx5gwABgClXAKZxPrnYiGi+sUCgqheSKDRVVyq05V8MOsFoNEBEj2QX1S/H9rmDAAGAF8vJzjY1ksqRNSEsIAgqgdCCMTfrzgPiFqrL/m/SI/4+2q0bF2EK+m5/NAmIqvDAqIZCHHohjuZBVBY8bd7ekgIHhQWooWdnaVTqRXv591QkF0EuX39DD9SQA2JtnzhYiMA7zZ/zH6pFgrEZXCuEiKqHev9RGkCSntctIK+3C/4hznbCeTUYjseMifoYd1/3j7i4WHpFOpEy0ftoNT88WPn3MIG3lDDSehhW0WLjepOeS3RqqH57Wy5ZRodkFPmbEhL76cATp9NRLXEAsKCSntcVNbbohR7XDRdMr0GOVfOoZWbEkJaeRynvCaixoYFBJEVquzOCpvfB2eWJTi1OhHVAxYQzcCR/GuIk+RABQf4i86WTqdGdh05gvikJDzi4YHpISGWTqfGLu+8h+y4Qjh2VKBbaNl29xJodJXfhVF6t0YpvQBir1e8s8K7TVG5yxUA4OnKMxdEVPdYQDQDScV5SJLkAFZ8a398UhKuxMVZOo1ay41XI/tqYYXlOr1AYhWXsUrv1ijFfitEZGmciZKIiIjMxgKCiIiIzMZLGERWoPR2z1KV3fpb21t+iYiqiwUEkRUovd2zVGW3/vKWXyJqKLyEQURERGZjAUFERERmYwFBREREZuMYiGbAV+kOVZ4jnIT1Tij0jK8vvDt3hmvr1pZOpVbaP+OMlt2UsFPJLZ0KEVGtsIBoBnyVHnDLrXySImvwjK+vpVOoE55DW0GjqaJrFhGRleAlDCIiIjIbCwgiIiIyGy9h1AMnmRpdHB+gSKGrMq6hJv2JUSchXpIHJ6GEN9waYIt1LyomBqmZmXBt3dqqL2cknshC3j017FRyuPu3tHQ6REQ1xgKiHkiL1ci9etbkte6GmvQnRp2MOEkWPNAS3sJ6C4grcXHw7tTJqguIu1HZyLxcAOfudiwgiMiqsYAgonIkEgnyi6rfulUpl0AmseJWr0RUIywgiKicomKB8zdzTQf+ztfLCQ629ZgQETVKHERJREREZuMZCCIyqpOLgFKiMRnnILUFYL2TlBFRzbCAIGpmHm4N/rDSVuHOdgIJsTEm19fOZRD0UhYQRM1NrQqI0NBQZGRkQCYrWc2aNWtw584d/Otf/4JWq8X06dMxderUOkmUiOrGw63BH1baKpytwYmoKjUuIIQQiIuLQ1RUlKGASElJwfz583Ho0CHY2tpi0qRJ6NevH7p06VJnCRMREZHl1biAiIuLg0QiwcyZM5GRkYFnn30WLVq0QP/+/eHs7AwAGDFiBCIjI/HKK6/UWcJkPg+ZIzRaPVRwsHQqNfaIh0e5/62V0yNK6PUCjh0Vlk6FiKhWalxA5Obmws/PD6tXr4ZarUZoaChGjhwJlUpliHF1dcXFixfNWq+LS+P8kNNnaeDgWL3rvDY2hQAAhaLqjotSqQ0UCjlsfv/fVJxh/ZXEPxxXalxLb9zXaU3GPbzuquLKxlcnrib7WjZ29rMTK40zlntVccbiTe1DaWx19hUomUvBWFyPGe2M5lef74GHY+t6X+VyKZxUjibjKqOqxWubGh4LsiY1LiB69+6N3r1LrpHa29tj4sSJeOedd/Diiy+Wi5NIJGatNyMjH3p945uUpqAIyM+rXkfL0vxNzUSp0+mh0Wih10mrjC2NM6y/kviH4yqLryzu4diq4srGm4pTKOQ12teqYo1t01h8Xe1raWx1jglQconPWJxCITf6vajP94A53/+y8dXdV61Wh7S0PJNxxqhUjjV+bVNT02NhYyNptH94UdNW43kgzp49i1OnThkeCyHg4eGB9PR0w7LU1FS4urrWLkMiIiJqdGpcQOTl5WH9+vXQaDTIz8/H4cOH8fe//x2nTp1CZmYmCgsL8c0332Dw4MF1mS/VQFJxLu4iG2nIt3QqNRaflITLv/2G+KQkS6dSKzm3C5F5pQB58dU7m0VE1FjV+BLGkCFDcOHCBYwdOxZ6vR5TpkxBnz59MH/+fISGhkKr1WLixIl44okn6jJfqoEj+dcRZ5MFD9ESE0UvS6dTI7uOHDE001o9Z46l06mxKx/fNzTT6rvS09LpEBHVWK3mgZg3bx7mzZtXbllwcDCCg4NrlRQRERE1buyFQURERGZjAUFERERmYwFBREREZmMBQURERGZjN04iarSKhQRqbfUnllPKJZBJGt9EdERNEQsIM3RyEVBKNCbjFDbSBsiGqOlTawViruVWO97XywkOtvWYEBEZsIAwg1KiQc6VMybjJAP7NkA2REREltNsCwhzT43qhXk9PRqTOc6+SEy17pkPrXnyqLL81jxarf4S1kWC/KLqRfISA1HT0WwLCHNPjfp0darHbIisl9xGjwdpqSbj1EIBF1VLXmIgaiKabQFBRHVDUqxBzpWzJuNaej9V/7lIJMgv4qBLoobAAoKImoyiYoHzNznokqghsIBoBrZlx1h9M63V27Y1iWZap1bdbtbNtB4+Q6DP0qCgivET1jz2iKipYwFBRA3m4TMEDo5K5OdVPsCXY4+IGi8WEETUbJkzZoLjJYjKYwFBRA3CuYUNFMiGd5s/rlnY2mpRpNCVi1MLBeIyGubShTljJjhegqg8FhBE1CBkeg0Kb55HTpk5SRQKeYV5MUru1lA2cHZEZC420yIiIiKzsYAgIiIis/ESBhFRPTC7k2hBcT1mQ1T3WEAQEVWDubNc6gUQe736k1r5P2nLU8JkVVhANAMhDt1wJ7MACiv+dk8PCcGDwkK0sLOzdCq14v28GwqyiyC350eFtTF3lkvOYUFNnfV+olC1ecicoId133/2iIeHpVOoEy0ftYNSwx87IrJ+/E0GoJOLgFKiqTKmFfTQ2AnkNFBOREREjRkLCABKiQY5V85UGePkqoStZ/cGyoiIiKhxYwHRDBzJv4Y4SQ5UcIC/6GzpdGpk15EjiE9KwiMeHpgeEmLpdGrs8s57yI4rhGNHBbqFulo6HSKiGmMB0QwkFechSZIDWPE0/vFJSbgSF2fpNGotN16N7KuFlk6jUXNuYQNvlMxW2Qr6clNfl9WQU14TUUUsIIioUZHpNci5cg5AyaXDslNfl8Upr4ksq8kUEOZO2qIX/MuFyJqVPVNRytgZC56pIKofTaaAUGsFYq7xHm2i5qLsmYpSxs5Y8EwFUf3gbDZERERktnopIL744guMGjUKw4cPx549e+pjE0RERGRBdX4JIyUlBRs3bsShQ4dga2uLSZMmoV+/fujSpUtdb4qIiIgspM4LiOjoaPTv3x/Ozs4AgBEjRiAyMhKvvPJKtV5vY1OzwU4yKWCvlJoRLzHEy+UyKFq0qDJeqlRAKpObjAMAiVQGW3t7QFZ1d73S9UmVCihaVJ77w9utLL6y/FrJnaEqlqAVnKAQLarcj7Lrrs7+Vue42NrKarSvZWPburkht6gIbd3cStZjZJvG1l1X+1oaa857wFico7sTdPlyOLory+1Hfb4HHo5tqH01lptUaV8uF1tbWYWfk8reA8bWV533gFwuq/R3Q9nfA6aYE1uTeBubmv3+q+nvTKLakggh6nR2gO3bt6OgoADz588HABw4cAAXL17EW2+9VZebISIiIguq8zEQxuoRiYQVMhERUVNS5wVE27ZtkZ6ebnicmpoKV1dO2UtERNSU1HkBMWDAAJw6dQqZmZkoLCzEN998g8GDB9f1ZoiIiMiC6nwQZdu2bTF//nyEhoZCq9Vi4sSJeOKJJ+p6M0RERGRBdT6IkoiIiJo+zkRJREREZmMBQURERGZjAUFERERmYwFBREREZmMBYUR+fj5Gjx6Nu3fvAiiZnjs4OBgBAQHYuHGjIe7q1auYMGECRowYgeXLl6O4uGRK3uTkZEydOhWBgYF46aWX8ODBAwBAbm4uZs2ahZEjR2Lq1KlIS0sDABQVFWHhwoUYOXIkxo0bh1u3bjXwHlduy5YtCAoKQlBQENavXw+geR+P999/H6NGjUJQUBA+/vhjAM37eADAu+++iyVLlgCo/30WQuDdd99FYGAgRo0ahdjYWAvscUWhoaEICgpCSEgIQkJCcOHChUqbCtb3+4WowQgq5/z582L06NGiR48eIjExURQWFgp/f39x584dodVqxYwZM0RUVJQQQoigoCBx7tw5IYQQS5cuFXv27BFCCDFr1iwREREhhBBiy5YtYv369UIIId58802xfft2IYQQhw8fFq+99poQQogdO3aIlStXCiGEOHPmjJg4cWLD7XAVfvrpJ/Hcc88JjUYjioqKRGhoqPjiiy+a7fE4ffq0mDRpktBqtaKwsFAMGTJEXL16tdkeDyGEiI6OFv369ROLFy8WQtT/Pn/11Vdi5syZQqfTibi4ODFs2DCh1WobaG+N0+v1YuDAgeXyuH//vhgyZIjIysoSDx48EMHBweLmzZsN8vuEqKHwDMRD9u/fjzfeeMMwe+bFixfRsWNHeHp6QiaTITg4GJGRkUhKSoJarYaPjw8AYPz48YiMjIRWq0VMTAxGjBhRbjkAREVFITg4GAAwevRo/O9//4NWq0VUVBTGjBkDAPD19UVWVhaSk5MbetcrUKlUWLJkCWxtbSGXy9G5c2fEx8c32+Px1FNP4ZNPPoFMJkNGRgZ0Oh1yc3Ob7fHIzs7Gxo0b8eKLLwJAg+zzyZMnMWrUKNjY2ODRRx+Fu7s7zp0719C7Xk5cXBwkEglmzpyJMWPGYPfu3eWaCtrb2xuaCjbE7xOihsIC4iFr165F3759DY9TU1OhUqkMj11dXZGSklJhuUqlQkpKCrKysuDg4ACZTFZu+cPrkslkcHBwQGZmptF13b9/v173szq6du1q+IUWHx+PY8eOQSKRNNvjAQByuRybN29GUFAQ/Pz8mvX7Y9WqVZg/fz6cnJwAVPxZqY99fnhq/MZwLHJzc+Hn54etW7di165d+Oyzz5CcnFyt90V9vF+IGgoLCBNEJc3BzF1eGRsb49+CypZbws2bNzFjxgwsXrwYHTp0qPB8czsec+fOxalTp3Dv3j3Ex8dXeL45HI8DBw6gXbt28PPzMyxriH02ti5LH4vevXtj/fr1sLe3R+vWrTFx4kRs3ry5QlxN3hd1deyI6gPfbSZU1hzs4eVpaWlwdXVF69atkZ+fD51OV245UPLXRulriouLkZ+fD2dnZ7i6upYbAFX2NZYWGxuL6dOn4/XXX8e4ceOa9fG4desWrl69CgCws7NDQEAATp8+3SyPx7Fjx/DTTz8hJCQEmzdvxokTJ3DgwIF63+e2bds2umNx9uxZnDp1yvBYCAEPD49qvS/q4/1C1FBYQJjQq1cv3L59GwkJCdDpdIiIiMDgwYPh4eEBhUJhGAUeHh6OwYMHQy6Xo2/fvjh27Fi55QDg7++P8PBwACW/gPv27Qu5XA5/f38cOXIEQMkvI4VCAXd3dwvsbXn37t3Dyy+/jA0bNiAoKAhA8z4ed+/exYoVK1BUVISioiJ89913mDRpUrM8Hh9//DEiIiJw5MgRzJ07F0OHDsU777xT7/s8ePBgfPHFF9DpdEhISEB8fDwef/xxCxyBP+Tl5WH9+vXQaDTIz8/H4cOH8fe//91oU8GG+PkhajCWGLlpDYYMGSISExOFECUjzYODg0VAQIBYu3at0Ov1Qgghrl69KiZMmCACAwPFggULhEajEUIIcffuXfGXv/xFjBw5UsyYMUNkZ2cLIYTIysoSs2fPFqNGjRLPPfecYf1qtVosWrRIjBo1SowdO1ZcunTJAntc0VtvvSV8fHzEmDFjDP/27t3bbI+HEEK8//77YuTIkWL06NFi8+bNQojm+/4o9fnnnxvuwqjvfdbr9SIsLEyMGjVKjBo1Svzwww8W2OOKNm7cKAIDA0VAQIDYtWuXEEKIo0ePiqCgIBEQECA++OADQ2x9v1+IGgqbaREREZHZeAmDiIiIzMYCgoiIiMzGAoKIiIjMxgKCiIiIzMYCgoiIiMzGAoKard69exs6rhIRkXlYQBAREZHZZJZOgKg6Tp8+jfXr16Nt27ZITEyEUqlEWFgYPvzwQ2RnZyMxMRHPPPMMXnvtNWzYsAExMTHQ6XTw9vbGihUr4ODggLNnz+Ktt96CRCLB448/Dr1eDwB48OABli5dioSEBNjY2KBHjx5Ys2ZNlX0FlixZAoVCgV9//RXp6ekYOXIkWrduje+//x5paWl4++234efnh6Kiokrz+f7777F9+3YUFRUhMzMTY8eOxbx583D69Gls3LgRnp6euHnzJoqKirBq1Sr079+/oQ43EZFJPANBVuPKlSuYMWMGvvjiC4wfPx4LFy4EAKjVanz55ZdYuHAhPvjgA0ilUhw6dAhHjx6Fq6srNmzYgKKiIrz22mtYsmQJwsPD0a9fP6jVagDA8ePH8eDBAxw5cgQHDx4EACQmJprM5+rVq9i3bx8+//xz7Nq1C/b29vjss88QGhqKDz/8EAAqzUcIgZ07dyIsLAyHDh3Cvn378MEHHxi6KV68eBEzZsxAeHg4Jk6ciC1bttTHISUiqjGegSCr4eXlZWi1PmHCBKxZswaurq7o06ePISYqKgp5eXmIjo4GAGi1Wri4uODGjRuQyWSG7pGjR4/GqlWrAAB9+vTBxo0bMW3aNAwYMAD/93//h44dO5rMZ8iQIZDL5VCpVLC3t8fTTz8NAOjQoQOys7OrzEcikeDf//43oqKiEBERgVu3bkEIgcLCQgCAu7s7unfvDgDw9vbG4cOHa338iIjqEgsIshpSqbTcYyEEbGxsYG9vb1im1+uxbNky+Pv7Ayi5PKHRaHDv3r0KrZFlspK3v6enJ44fP47Tp0/j559/xvPPP48VK1YgMDCwynxsbW2Nrq+syvIpKCjAuHHjMGzYMPTt2xcTJkzAt99+a8hRqVQa1lFZW2ciIkviJQyyGteuXcO1a9cAAPv27cOTTz4JJyencjGDBg3Cnj17UFRUBL1ej5UrV+If//gHHnvsMQghcPLkSQDAd999h5ycHADA3r17sXTpUgwaNAgLFy7EoEGDcPPmzTrJubJ8EhISkJ+fj3nz5mHo0KE4c+aMIYaIyBqwgCCr0aZNG2zatAnBwcH49ttvsX79+goxc+bMgYeHB8aNG4dRo0ZBCIElS5ZALpdj69ateP/99xESEoLjx4/DxcUFADB27FjodDqMGjUK48ePR35+PkJDQ+sk58ry6datG5555hmMHDkS48aNw4kTJ9ClSxckJCTUyXaJiOobu3GSVTh9+jTeeustREREWDoVIiICx0AQGRUXF4f58+cbfe7RRx/Fpk2bGjgjIqLGhWcgiIiIyGwcA0FERERmYwFBREREZmMBQURERGZjAUFERERmYwFBREREZmMBQURERGb7fx1+ORyP/8YaAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "sample = valid_copy.copy().loc[valid_copy[\"ocean_proximity_INLAND\"] == 0]\n", "sample_mean = sample['preds_mean'].mean()\n", "sample_std = sample['preds_mean'].std()\n", "lower_bound = sample_mean - sample_std\n", "upper_bound = sample_mean + sample_std\n", "\n", "sns.distplot(\n", " sample[\"median_house_value\"], kde=False,\n", ")\n", "sns.distplot(sample[\"preds_mean\"], kde=False)\n", "plt.axvline(\n", " x=sample_mean,\n", " linestyle=\"--\",\n", " linewidth=2.5,\n", " label=\"Mean of predictions\",\n", " c=\"k\",\n", ")\n", "plt.axvline(\n", " x=lower_bound,\n", " linestyle=\"--\",\n", " linewidth=2.5,\n", " label=\"Lower bound 68% CI\",\n", " c=\"g\",\n", ")\n", "plt.axvline(\n", " x=upper_bound,\n", " linestyle=\"--\",\n", " linewidth=2.5,\n", " label=\"Upper bound 68% CI\",\n", " c=\"purple\",\n", ")\n", "plt.legend(bbox_to_anchor=(1.01, 1), loc=\"upper left\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In general, we expect our models to perform best on categories that are most frequent in the data. One way to validate this hypothesis is by calculating the ratio of the standard deviation of the predictions to the predictions themselves:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "ocean_proximity_INLAND\n", "1 0.288622\n", "0 0.228371\n", "dtype: float64" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "preds_quality = valid_copy[cols].groupby(\"ocean_proximity_INLAND\", as_index=True).mean()\n", "(preds_quality[\"preds_std\"] / preds_quality[\"preds_mean\"]).sort_values(ascending=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What the above tells us is that our predictions are less confident (i.e. higher variance) for housing districts that are inland - indeed looking at our bar plot we see these categories are under-represented in the data!\n", "\n", "In general, confidence intervals serve two main purposes:\n", "\n", "* We can identify which categories the model is less confident about and investigate further\n", "* We can identify which rows in the data the model is not confident about. This is particularly important when deploying models to production, where e.g. we need to decide how to evaluate the model's predictions for a _single_ housing district." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Feature importance" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One drawback with the confidence interval analysis is that we need to drill-down into each feature to see where the model is making mistakes. In practice, we can get a global view by ranking each feature in terms of its importance to the model's predictions. In scikit-learn, the Random Forest model has an attribute called `feature_importances_` that we can use to rank each feature:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def rf_feature_importance(fitted_model, df):\n", " return pd.DataFrame(\n", " {\"Column\": df.columns, \"Importance\": fitted_model.feature_importances_}\n", " ).sort_values(\"Importance\", ascending=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's use this function to calculate the feature importance for our fitted model:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ColumnImportance
7median_income0.248165
14ocean_proximity_INLAND0.135266
13population_per_household0.088856
9postal_code0.088172
0longitude0.080466
12bedrooms_per_room0.070059
1latitude0.066439
10rooms_per_household0.048305
2housing_median_age0.030146
8city0.022991
\n", "
" ], "text/plain": [ " Column Importance\n", "7 median_income 0.248165\n", "14 ocean_proximity_INLAND 0.135266\n", "13 population_per_household 0.088856\n", "9 postal_code 0.088172\n", "0 longitude 0.080466\n", "12 bedrooms_per_room 0.070059\n", "1 latitude 0.066439\n", "10 rooms_per_household 0.048305\n", "2 housing_median_age 0.030146\n", "8 city 0.022991" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# expected shape - (n_features, 2)\n", "feature_importance = rf_feature_importance(model, X)\n", "\n", "# peek at top 10 features\n", "feature_importance[:10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the table we see that `median_income`, `ocean_proximity_INLAND`, and `population_per_household` are the most important features - this is not entirely surprising since income and house location seem to be good indicators of house value. We can also plot the feature importance to gain a visual understanding:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def plot_feature_importance(feature_importance):\n", " return sns.barplot(y=\"Column\", x=\"Importance\", data=feature_importance, color='b')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhQAAAEJCAYAAADM2r49AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdfVzPd////9u7UzmraDEn25zPyWKOIadbMtaUVKw4FL5OR04O0xRJJJMYKnR0fMw2YzlLJ7IMGRspMyczJ2sHSc5aJKTzd78/+vU6pBNvKmc9rpfLLpf1fr/ez9fz+bLp+X6+nq/7Q1VYWFiIEEIIIUQlaD3vDgghhBDi5ScTCiGEEEJUmkwohBBCCFFpMqEQQgghRKXJhEIIIYQQlSYTCiGEEEJUmkwohBBCCFFpOs+7A0IIIYR4NnJy8rh/P4eGDety69b9J/qslpYKY+M65b4vEwpRY03/Ipy09Mzn3Q0hhHhmNi/7J3fvZgOgVldtrqXc8qgB4uPjcXZ2BmDevHn8/vvvVdr+hAkTuHnzZpW2KYQQ4uUiKxQ1jK+vb5W3+Z///KfK2xRCCPFykQnFCyw+Pp7g4GAKCwtJTk5m0KBB1KtXj3379gEQEhLC2bNnCQgIID8/n2bNmuHj44OxsTG//PILX3zxBfr6+rRo0UJp09nZGVdXV/7xj3/g7e1NYmIiaWlptGjRgqCgINLS0nB1daVNmzacO3eOhg0bsnr1aoyMjMrtZ//+/fn2229JSEjg559/JiMjgytXrtC7d2+8vb0pLCxk+fLl7Nu3D21tbRwdHRk9ejSXLl3Cy8uLO3fuULt2bebNm4eZmRnu7u4YGBhw/Phx7t27x9y5c4mIiOD8+fMMGDAAd3d3CgoKWLZsGQkJCRQUFGBvb8+YMWOq+49ECCFEOeSWxwvu1KlTfPHFF0RHRxMaGkqDBg0ICwujXbt2hIaGsmLFCtavX094eDh9+vRh+fLl5Obm4u7uTkBAAGFhYdSqVatUuydOnEBXV5ctW7awd+9ecnJyOHjwIADnz59n7Nix7Nq1i/r16xMVFaVxf0+cOEFAQACRkZEcOHCACxcuEBMTw2+//UZUVBTbtm0jLCyMv//+Gzc3N5ydnYmKisLDw4MZM2aQm5sLQGpqKpGRkUyfPh0PDw8WLlxIeHg4W7du5d69e2zduhWAnTt3sn37dvbv38+vv/5aBVdcCCHE05AVihdc27Ztef311wEwNjamZ8+eADRp0oTY2FiuX7+Oi4sLAGq1GkNDQy5cuICpqSmtWrUCwM7OjtWrV5dot1u3bhgZGbFp0yYuXrxIUlISDx48AKBhw4Z06NABgDZt2pCRkaFxf999913q1q0LQPPmzcnIyODYsWNYWVmhp6eHnp4eERERZGZmkpyczMCBAwHo0qULhoaGXLx4EYB+/fop42zTpg0NGzYEwMjIiIyMDOLi4jh37hxHjx4F4MGDB1y4cIH33nvvSS6vEEKIKiITihecrq5uiZ+1tbWVf1er1XTt2pXg4GAAcnJyyMzM5Nq1a6jV6jI/U2z//v0EBATg4uKCvb096enpFFey19fXV45TqVQ8SYX7sj6ro1PyP7OUlBQMDQ1LtVtYWEhBQUGpcT/6eYCCggLc3NyUCcnt27epXbu2xv0UQghRteSWx0vMzMyMkydPcunSJQDWrl3LsmXLaNeuHbdu3eL8+fMAREdHl/psXFwcVlZWODg4YGJiwrFjx5Rf5lWtW7du7N27l7y8PLKyshg/fjxpaWk0b96cH3/8EYCTJ0+SlpZGmzZtNGrT3NycrVu3kpeXR2ZmJiNHjuTUqVPV0n8hhBCPJysUL7HXXnuNJUuWMHPmTNRqNY0aNcLf3x9dXV2+/PJL3Nzc0NHRUW5fPGz48OHMnj2bmJgY9PT06NKlCykpKdXSzw8//JAzZ85gb2+PWq3GxcWFFi1a4O/vj7e3N4GBgejq6hIYGIienp5GbTo5OXH58mXs7OzIz8/H3t6eHj16PFG/AjyGPs1whBDipZWdk1dtbasKn2Q9W4hXyK1b96s82OVl8dpr9fj773vPuxvPVU2/BjJ+Gf+Tjl9LS0XDhnXLfV9WKIRGnJ2duXv3bqnXnZycGDFixHPoUeVV9D9GTfDaa/Wedxeeu5ftGmTn5HHv/085FOJFIxOKl9jq1avp1KkTlpaWGh0/YcIEFi9eTG5uLuvWrWPJkiUan2vjxo0AhIWFkZCQwNKlSwkMDGTDhg3Y2dkpj6bGx8cTFBTExo0bcXd3p3v37tjb25fZpp+fHzt37uTQoUPKrY6UlBQsLS356quv6N27t3JscdYFwEcffaQ8wZKdnU27du3w8vLCxMRE4/GARG+Ll8/mZf/kHjKhEC8m2ZT5EpsxY4bGkwkoSrRs1KgR165d48qVK1XSh2vXrvHll18+8efy8/P54YcfePfdd4mJiSnxnq6uLvPnz+f+/bIL15iamhIREUFERAQxMTG8+eabTJ8+/an6L4QQomq80hOK4OBgPv74Y2xsbFi6dCkFBQV8/fXXDBo0iI8//hh/f38A0tLSmDJlCvb29jg4OHDkyBEAbt68ybhx4/jkk0+wsLBg+fLlQNG39H/961/8v//3//jwww/x9vausB8pKSnY2NgwZcoUBg8ezIQJE7hz5w5Q9LTCuHHjsLW1JS8vr8w+79+/n4EDB5KTk8Ply5fp168fN27cwN3dnbCwMFJSUrC1tcXV1ZWBAwcya9YsQkNDcXR05KOPPuK///0vUPQtPyUlhcWLF3PmzBkWLlyIm5sbW7ZsUfrq7Oz8RE9LODk5sXv37icOlTp48CDNmzdn6NChJc4PRROGXr164efn99h2VCoV06ZNIzExUXmqRQghxLP3yk4oDh48SGxsLGFhYezcuZPLly+zbt06Nm/ezPbt24mMjOSPP/7gzJkz+Pr64uDgQFhYGOvWrcPLy4v79++za9curK2t2bp1K5GRkWzevJnbt28DZSdCVuTPP/9k9OjRREdH06pVK4KCggBIT09n4sSJREREcOTIkVJ9Dg0NxdLSknfffZd169bh4eHBnDlzaNy4cYn2L1y4wJQpU4iJieH333/n6tWrbNmyBWtr61K/sD09PenUqRMLFizAwcGByMhIAK5evcrt27fp3LmzxtfZ0NAQb29v5s2bR3a25kuxYWFhfPTRR7z//vucO3eOv/76q8T77u7u/PLLLxw+fPixbenp6fHmm28qoVhCCCGevVd2QnH06FEGDx5MrVq10NHRwcHBgaCgICwsLKhXrx46Ojp8/fXXdOrUiSNHjhAQEICtrS0TJkwgPz+fK1euMG7cOF5//XXWr1+Pr6+vkqMA/0uENDAwUBIhK/LWW28pjzUOHTpUSXgElF/gZfU5Li4OKKoSun37dkxMTBg8eHCp9k1MTOjQoQNaWlo0bty4RKJmWZspi/Xo0YPU1FRSUlIIDw/H1tb2Ca5ykQEDBtCpUyeNb33cvn2bX375BSsrK2rVqoWFhQWhoaEljqlbty4+Pj4V3vp4mEqlKjNiXAghxLPxym7KfDgpstijT8jevHkTAwMD1Go133zzjVIA6+bNm5iYmLB06VKuXLmCtbU1AwYM4MiRI0+dJvlw2mNhYWGJ9MriX4Rl9Tk/Px8oui2jra3NpUuXyM3NLZXX8OjPZaVjlkWlUjF06FCio6OJiYnh//7v/zT63KPmz5+PtbV1hUXEikVGRlJYWMiwYcOAoo2VeXl5zJ49u8Rxffr00ejWR25uLpcuXaJ169ZP1XchhBCV98quUJibmxMdHU12djb5+fns2LGDOXPmcOjQITIzM8nPz+ezzz7jzJkzmJubs3nzZgD++usvhgwZQlZWFocPH2bcuHFYWVlx/fp1bt68WeYvfU1cunSJc+fOAbBjxw6lVsXj+mxubk5BQQEeHh7MmzePbt26sWrVqqe/MBRNNoonKgD29vaEhobSuHFjGjVq9FRtGhkZ4e3tzdq1ax977I4dO1i6dCmxsbHExsbyyy+/YGhoyO7du0sdW3zrIzU1tcy21Go1gYGBdO7cmTfeeOOp+i6EEKLyXtkVCgsLC86dO4eDgwP5+fn07dsXFxcX9PX1cXJyQq1W8+GHH9KrVy9atWqFl5cXNjY2ACxbtoy6desyadIkPv/8c+rXr0/Dhg3p1KnTU6dJGhoaEhAQQHJyMu3atWPx4sUa9XnUqFF89dVXNGzYkIEDB9KrVy+sra2VGhZPo1WrVty7dw83Nzf8/f15/fXXef3117Gzs3vqNqHo1segQYNK/PJfsGABPj4+ys///ve/SU9P58MPP1Re09LSYvTo0YSGhtK9e/cSbRbf+hg3bpzyWmpqqnJrRq1W0759e1asWFGpvgshhKgcScp8BlJSUnBxcSE2NvZ5d6WUwsJCUlNTcXZ2ZteuXRpHX78KJIdCvGw2L/tnlaU7SlKkjF+SMl9QycnJTJs2rcz3ylqNeFHs2bMHb29vvL29lcmEn5+f8ujswzp16oSvr++z7mK1kVoe4mVTnXUYhKgsWaEQNZbU8qi5385AroGMX8YvKxRCVBGp5fFy1bEoi9S2EOLFIROKJ+Ds7Iyrq2uFZbKdnZ2Vuhe2trZEREQ8q+5VSrt27R4bzlXVnmZvSXFNj2bNmpV4XZM/m0fJHoqXn9S2EOLF8co+Nvq8JCQkKP/+skwmhBBCiMp6pSYU8fHxjBo1ijFjxjBo0CDc3NzIzc1lx44dWFtbY2Njg7u7O5mZRd9Kzc3N8fDwwMbGBicnJ+WR0OKaF8VtOjs7lzhPfn4+np6eODo6Ymlpyfjx48nOzlY2Xw4fPhwo+tYPkJWVxWeffab0ITw8HKjamiCHDh1i2LBhDB06FFdXV9LT05WxzJw5k0GDBnHr1q0K2/fy8mLIkCEMGTKEy5cvA3Dy5EmGDx/OkCFDGD16tPK6s7Mz8fHxSr/69+8PQFRUFLa2ttjb2zN9+nRycnIACAkJwc7OjiFDhrBs2TIlCCw7O5t//etfWFtbM3LkSKXfBw4cwNbWVhlvWlpaib7m5ubi5uaGlZUV48ePVz4nhBDi+XilJhQAp0+fxsvLi5iYGHJycggJCSE4OJiNGzcSFRWFgYFBiToa3bt3JyoqisGDB2v8NMaJEyfQ1dVly5Yt7N27l5ycHA4ePIinpycA27ZtK3F8YGAgxsbG7Nq1i2+++YbAwEClkFVV1AS5ffs2K1asYP369YSHh9OnTx+lkBlAv3792LNnDw0bNqyw7V69ehEZGUnv3r0JDQ0lNzeXWbNmMX/+fCIjI3FycmLWrFkVtrFq1Sq++uorwsLCaNGiBRcvXuTQoUOcOXOG7du3Ex4ezs2bN5X6Ibdv32bs2LHs2rULExMTdu/eza1bt/Dy8mLNmjVERUXRtWtXFi1aVOI8xbeVfvjhBzw9PUlOTq6wX0IIIarXKzeh6NatGy1btkSlUmFra8vatWuxsLDA2NgYAEdHR6WOhr6+PkOHFj06aGdnp3zj1uQcI0eOZNOmTfj6+pKUlMSDBw/KPf7o0aNKzHSDBg2wtLRUbo1URU2QU6dOcf36dVxcXLC1tWXTpk3KSgKgcbGvAQMGANC6dWvu3LlDUlIS9evXx8zMDAArKyuSk5O5d6/8ncEWFhaMGDECPz8/LCwsaN++PXFxcZw+fRp7e3vs7Ow4c+aMUgzM1NRUab9169akp6dz+vRpzMzMlH0SD/+ZFUtISMDKykq5Ju+++65GYxRCCFE9XrlNmQ/XsCgsLCwVlV1YWKjETmtpaaFSqYCixMVHPwuUiKgutn//fgICAnBxccHe3p709PQKa3k8+l5hYSEFBQVA1dQEKSgooGvXrgQHBwOQk5Oj3NZ59ByatF3cj/LqoRT3vaxr5Onpyfnz5zl48CBubm64urpSUFDA6NGjGTt2LAB3795FW1ub9PT0EuMp77wP/5k9fOzDxz3cjhBCiGfvlVuhOH78uFJzIzw8HA8PD2JjY5W9Blu3blW+4WdlZSlPGISFhSn1NYyNjZVv0Pv37y91jri4OKysrHBwcMDExIRjx44pv2QfrZMBRXs1tm/fDhQt8e/fv79UxLSmyqoJ0rlzZ06ePMmlS5cAWLt2LcuWLXuq9h/WsmVL7ty5w+nTpwHYvXs3TZo0wcjIqMQ12rdvH1A0sRg4cCDGxsZMmjQJW1tbzp07h7m5OREREUoNlalTp7Jnz55yz9u5c2dOnTql7GPZsmVLqac3evbsya5du1Cr1Vy9epXffvut0uMVQgjx9F65r3WmpqZ8/vnn3Lx5k969ezNq1Chq166Ns7MzeXl5dOzYkYULFyrHx8TEsHLlSkxNTZWqltOnT8fHx4egoCD69OlT6hzDhw9n9uzZxMTEoKenR5cuXZRffpaWltja2hIWFqYcP3XqVLy9vbGxsaGgoIDJkyfTsWPHp3pMs6yaILVr12bJkiXMnDkTtVpNo0aN8Pf3f+K2H6Wnp8fKlSvx8fEhKysLQ0NDVq5cCcD48eNxd3dnx44dWFpaAkWrBNOnT2fs2LHUqlWL+vXr4+fnR6NGjTh//jyffPIJBQUF9O3bFzs7O65evVrmeU1MTFi0aBGurq7k5eXRpEmTUgmdI0eOJDExESsrK5o2bUrbtm0rPV4hhBBP75VKyoyPjycoKEjZsPc4zyN7oTJe5JogQjwPlQm2kqREGb+MX5IyX2nVVRMkOzsbR0fHMt+bPn26sspQk0j0ds39y1QIUfVeqRUKIUTNUdnY7Zo+qZLxy/hlhUI8N7GxsVy+fFl5WqMsYWFhJCQksHTp0mrpQ2BgIEC5qzhPQqK3X24Suy3Ei0UmFEJjf/zxx/PughBCiBeUTChqiPj4eAIDA9HR0eH69euYmZnh6+tLVFQUGzZsQKVS0bFjR+bPn4+enh5z584lMTERKHqiomvXroSGhgLQpEkT+vTpw9y5c7l37x5///03gwcPZvbs2Rr15dy5c3h5eZGdnY2hoSHLly+ncePGBAcHExkZiba2Nr1798bNzQ1tbW3+7//+j61bt2JsbFwiaOvQoUMEBASQn59Ps2bN8PHxUQLMhBBCPFuvXA6FKJ+mseQnTpwgIyOD8PBwNmzYwG+//Ubr1q1xcnLCyckJBwcHdu3ahbW1NVu3biUyMpLNmzdz+/Ztjfoxe/ZspkyZQlRUFB9//DHffPMNBw8eJDY2lrCwMHbu3Mnly5cJDQ3l999/Z8eOHezcuZMNGzZw48YNgMfGjQshhHi2ZIWiBimOJYei0urTpk1j1KhRJWLJPTw8mDhxIpcuXWLcuHH069evzJWHcePGcfToUdavX09iYiJ5eXlkZWU9tg+3b9/m77//xsLCAiha/QDw8/Nj8ODB1KpVCwAHBwfCw8PJzs7m/fffp06dOgB89NFHqNXqEnHjUJR0amhoWMkrJIQQ4mnJhKIG0TSW3NjYmOjoaA4fPszBgwexs7MjOjq6xLFLly7lypUrWFtbM2DAAI4cOfLY2HAAXV3dEj/n5OSQmppaZsx3fn5+mRHbubm5j40bF0II8WzJLY8aRNNY8v379zN79mw++OADPD09qV27NtevXy8RK3748GHGjRuHlZUV169fV9p9nHr16tG4cWMOHz4MQEREBKtXr8bc3Jzo6Giys7PJz89nx44dmJub07NnT3766Sfu3btHTk4Oe/fuBai2uHEhhBBPR1YoahBNY8n19fXZs2cPgwcPRl9fn4EDB9KuXTvu3r3LnDlzMDExYdKkSXz++efUr1+fhg0b0qlTJyV+/HH8/f3x9vZm2bJlGBsbs2zZMkxNTTl37hwODg7k5+fTt29fRo0ahY6ODqNHj2bYsGHUr1+fJk2aAPDaa69VS9y4EEKIpyPBVjXEk8aSC/Gik2CrypHxy/gl2Eq8FD777DOlGunD+vfvz4wZM55Dj0qT6O2a+5epEKLqyQqFqLFkQlGzJxQ1/RrI+GX8skIhRBWp6H+MmuC11+o97y6UqbK3MoQQz4dMKESFqnrvxf79+zlz5gwzZswgICCAXr168d5772n8+aosOS+1PF5MUqNDiJeTPDYqnilLS0tlD8WxY8coKCh4zj0SQghRFWSFQmjk0qVLeHl5cefOHWrXrs28efMwMzPD3d2dunXr8scff3Dz5k2mTp2Kg4MD9+7d4/PPPyc5OZnmzZtz48YNgoKCSEhIICEhAXNzc86cOYOnpydBQUEsXrwYV1dXevToQUpKCi4uLsTGxpKSkoKbmxsPHjygc+fOSn8yMzNZtGgRiYmJFBQUMGHCBKytrZ/jFRJCiJpNViiERtzc3HB2diYqKgoPDw9mzJhBbm4uADdu3GDz5s2sW7dOCZdas2YNLVq0IDo6mqlTp5a6TTF06FA6derE4sWLadeuXbnn9fHxwd7enoiICLp27aq8vm7dOjp27EhYWBibNm0iODiYK1euVMPIhRBCaEImFOKxMjMzSU5OZuDAgQB06dIFQ0NDLl68CEDv3r1RqVS0bdtWSd08fPgwtra2ALzzzjsVThoqkpCQgJWVFQBDhgxRoruPHDlCaGgotra2/POf/+TBgwdKdVQhhBDPntzyEI9VWFhYqk5HYWGhsv9BX18fAJVKpbyvra2tUW2PR9sElHjvR19XqVTKOdRqNf7+/nTs2BGAtLQ0KQ4mhBDPkaxQiMeqW7cuzZs358cffwTg5MmTpKWl0aZNm3I/06tXL6KiogC4cOECiYmJJSYcUDTpKJ6UGBsbK0FY+/btK9FOZGQkAD/++KNym8Xc3Jzvv/8egNTUVIYMGcL169erYrhCCCGegkwohEb8/f3ZuHEjNjY2LFq0iMDAQPT09Mo9fsqUKSQnJ2NjY0NAQAAmJiZKafJiffv2ZcGCBfz222+MHz+ezZs3Y2dnR3b2/x4Z9PLyYs+ePdjY2HDw4EGljLmrqyvZ2dlYW1szevRo3NzceOONN6pn8EIIIR5LkjJFtYiIiKBZs2b84x//4Nq1a4waNYp9+/ahpSVzWFGxZxVsJUmJMn4ZvyRlipdAy5YtWbBgAWq1Gi0tLRYtWvTCTSYkervm/mUqhKh6MqEQ1eKdd94hLCzseXejQhK9Xf3R2xKjLUTNIROKSqhMLLWUE3/+JHq7+kmMthA1x4u1Bi2EEEKIl5KsUFRSeno648aNIzU1FTMzMxYsWMDRo0cJCAggPz+fZs2a4ePjg7GxMb/88gtffPEF+vr6tGjRQmnD2dkZQ0NDEhMTWbVqFTdu3GDVqlWo1WqaN2/OokWLMDEx4eTJk/j6+pKTk4OxsTGLFi3izTffxNnZmfbt2xMXF0d2djaenp5s3LiRv/76izFjxjBmzBji4uLw9/cHwNDQkBUrVtCgQYNyx2Vubo6FhQVnzpyhTp06LF++nGbNmnH69Gm++OILsrOzMTY2ZuHChTRv3rzUGNq3b19mu/3798fMzIxz586xefNmfvrpJzZs2IBKpaJjx47Mnz+fOnXqcODAgTKvQf/+/bGysuKnn35CW1ubWbNm8dVXX3H58mXmzJnDxx9/XLV/wEIIITQiKxSVlJKSwvz584mMjCQzM5OQkBBWrFjB+vXrCQ8Pp0+fPixfvpzc3Fzc3d0JCAggLCys1COU7dq1Y8+ePZiamuLl5cWaNWuIioqia9euLFq0iNzcXGbNmqWcy8nJiVmzZpVoIyoqCltbWxYvXkxgYCCbNm1izZo1AKxduxZvb2/CwsKwsLDg7NmzFY4rPT2d7t27ExUVxeDBg1m8eDG5ubl4enqyYsUKdu7cydixY5k/f36pMZQ3mSjWr18/9uzZQ1paGsHBwWzcuJGoqCgMDAwICgri1q1bZV6DYqampkRHR9OxY0dCQkL46quv8Pf3JyQkRKM/MyGEEFVPVigq6b333uOtt94CwMbGBnd3d1QqFS4uLkBRoqOhoSEXLlzA1NSUVq1aAWBnZ8fq1auVdszMzAA4ffo0ZmZmNGvWDABHR0dCQkJISkqifv36ynFWVlZ4eXlx717RTv1+/foB0KRJEzp37oyBgQFNmzbl7t27QFGVT1dXVwYMGIClpSW9e/eucFz6+voMHTpU6euXX35JUlISV65c4dNPP1WOu3//fqkxPE5xka9jx45hYWGBsbGxMlYPDw+6d+9e5jUo9vBYTU1N0dHRoUmTJspYhRBCPHsyoagkHZ3/XcLiSI+uXbsSHBwMQE5ODpmZmVy7dg21Wq0cq62tXaKd4hWLh48pbjM/P7/U68XvFSdNFte4eLRPxcaMGYOFhQUHDhzA39+f06dPl5gYPEpLS6tEzLW2tjZqtZpmzZoREREBQEFBAWlpaaXG8DjFUd2ajrX49WKPG6sQQohnT255VNLx48eVyUJ4eDijR4/m5MmTXLp0CSi61bBs2TLatWvHrVu3OH/+PADR0dFltte5c2dOnTpFSkoKAFu2bKFHjx60bNmSO3fucPr0aQB2795NkyZNMDIy0qifw4cPJzMzU9lT8bhbHllZWcTGxgIQFhZGv379aNmyJRkZGfz6668A7Nixg9mzZ2t0/rJ0796d2NhYpaDY1q1b6dGjR7nXQAghxItLvt5VUuvWrZk7dy5///035ubmfPrpp3To0IGZM2eiVqtp1KgR/v7+6Orq8uWXX+Lm5oaOjg4dOnQosz0TExMWLVqEq6sreXl5NGnSBF9fX/T09Fi5ciU+Pj5kZWVhaGjIypUrNe7nrFmzcHd3R0dHB319fRYuXPjYz8TExLBy5UpMTU3x8/NDT0+P1atXKxtD69ati5+fn8Z9eNTbb7/NpEmTcHZ2Ji8vj44dO7Jw4ULq1q1b5jUQQgjx4pLobVGmdu3aceHChefdDfGSe5GDrWp6WqiMX8Yv0duiSmRnZ+Po6Fjme9OnT69U287OzmVukHRycmLEiBGVarsqSfR2zf3LVAhR9WRCUUPVqlVL2VxZlsqsTkj6pxBC1DwyoRA1ltTyqNpaHi/y7Q0hRPWTCYWoEo+rTRIbG8vly5cZO3Ys33//PQAjRozAw8MDV1dXmjZtqtF5UlJScHFxUZ5AqQyp5VG1pG6HEDWbPDYqnok//vhDCcEaMWKEspciPj4e2RcshBAvP1mhEFUqISGBlStXkhidMLkAACAASURBVJ2dTUZGBm5ubrRp04bQ0FCgKN3y2rVrQFHAVWpqKhMnTmTTpk04ODjw7bff0qxZsxIrHmfPnmXevHlA0aOmxdLS0vDy8uLGjRuoVCo+++wzevXq9ewHLYQQQlYoRNX67rvvWLx4MTt37sTX15e1a9fSunVrnJyccHJywsHBQTl24sSJmJqaEhISosRvl2XOnDm4ubmxc+dOJY4bwNfXFwcHB8LCwli3bh1eXl4losCFEEI8O7JCIaqUv78/Bw4cICYmhlOnTpGZWbk9Crdv3yY1NVVZebC3t2fHjh0AHDlyhIsXLxIQEABAfn4+V65ceWxxMiGEEFVPJhSiSo0cOZIePXrQo0cPevbs+cTR3MX7KYprd6hUqhJ7LB6ugaJWq/nmm2+U+PGbN29iYmJS2SEIIYR4CnLLQ1SZO3fukJSUxIwZM3j//fc5fPiwUrxMW1u7RIGvYtra2soxxsbG/PXXXwDs379fea1Jkyb89NNPAOzatUv5rLm5OZs3bwbgr7/+YsiQIWRlZVXb+IQQQpRPJhSiyhgZGTF8+HAGDx7M0KFDuXXrFtnZ2Tx48IBu3boRFRVV6rHSDz74gIkTJ3LlyhWmT5+u7IuoV+9/GQn+/v4EBQUxdOhQkpOTldc9PT05deoUNjY2/Otf/2LZsmXUrVuzsyWEEOJ5kVoeQogq8bIFW9X0+HEZv4xfankIUUWklkfN/ctUCFH1ZEIhaqyaGL39sq0iCCFeHjKhEACEhYWRkJDA0qVLn+l5AwMDAZg2bZpGx5cX8f00kdw1MXpb4rGFENVFNmUKIYQQotJkheIx4uPj8ff3R61W06xZM3R1dblw4QIqlYpx48YxdOhQ1Go1S5YsIS4uDpVKxZAhQ5g4cSLx8fEEBwdTWFhIcnIygwYNol69euzbtw+AkJAQDA0NmTt3LomJiUBRjsMnn3xSbn8CAwNJSkoiOTmZO3fu4OjoyPjx4ykoKGDZsmUkJCRQUFCAvb09Y8aMKdH/Nm3a4OfnV27bly9fxtnZmWvXrtGzZ08WL14MQHBwMJGRkWhra9O7d2/c3Ny4fv16iRWB4pWGyZMnlzmeimKyT58+jZOTEzdv3sTe3p5p06aVe00fVl4ktxBCiGdPJhQaSEpK4sCBA6xbt47c3Fx27drF7du3GT58OG+//TbHjx/n+vXrREZGkpubi7OzM23btsXAwIBTp04RHR2NkZERvXr1Ys6cOYSFheHh4UF0dDTt27cnIyOD8PBw0tPT8fPzq3BCAfDnn38SGhqKWq3G3t6enj17cvr0aQB27txJbm4u48aNo1OnTiX6//CjmGW5fv064eHh1K5dmwEDBpCYmMi1a9eIjY0lLCwMHR0dpk2bRmhoKO+//36ZbZw4caLM8RQ/DmppaUlqaiojR44kPDwcgFu3bhEaGsr9+/fp378/Y8eOJSIiotxrWmzOnDl4eHjQq1cv1qxZQ3x8vMZ/pkIIIaqW3PLQQIsWLahXrx5Hjx5l2LBhADRo0ABLS0sSEhKIj4/Hzs4ObW1tDAwMsLGxIS4uDoC2bdvy+uuvY2BggLGxMT179gSKimTdvXuXNm3acOnSJcaNG0dkZKRGyZLW1tbUqVOHevXq0b9/f44ePUpcXByxsbHY2toyfPhwbty4wYULF0r0/3Hee+89jIyM0NPT44033iA9PZ2jR48yePBgatWqhY6ODg4ODsrYylLeeI4cOUJAQAC2trZMmDBBickG6Nu3L3p6ejRo0ABjY2MyMjIqvKZQdiS3EEKI50dWKDRQq1YtgFJltgsLCykoKECtVpf5OoCurm6J9x6OjoaiJMjo6GgOHz7MwYMHsbOzIzo6mvr165fbn0fjp4vTJt3c3Bg4cCBQ9Au3du3anDp1Sun/4+jo/O8/h+LI60fHBkWx2I9GYufn56Ojo1PueMqLyd63b59G5334mj58XFnXRAghxLMnKxRPwNzcnO3btwNFv7D3799P9+7dMTc3Jzw8nIKCArKysoiKiqJHjx4atbl//35mz57NBx98gKenJ7Vr1+b69esVfmbfvn3k5uaSkZHBgQMH6NOnD+bm5mzdupW8vDwyMzMZOXIkp06dqpIxR0dHk52dTX5+Pjt27MDc3Jz69euTkZHB7du3yc3N5eeff65wPE8ak/24a1pRJLcQQohnT1YonsDUqVPx9vbGxsaGgoICJk+eTMeOHWnbti1JSUnY2tqSl5fHkCFD+PDDDzW6p9+vXz/27NnD4MGD0dfXZ+DAgbRr167Cz+jr6zNy5Eju37/PpEmTaN26NW+++SaXL1/Gzs6O/Px87O3t6dGjR6X3FVhYWHDu3DkcHBzIz8+nb9++jBo1Ch0dHcaNG8ewYcNo3Lgx77zzToXj8fT0xMvLCxsbG4DHxmQ7Ojo+9pr6+/vj4eHBqlWr6NKlS6XGKYQQonIkevsl86S5DUI8rDjYSpIyJS1Uxi/jfy7R21lZWcTExJCRkVHivvXYsWOfqDNCM19//TU7d+4s9bqpqSlmZmZP3e7u3bv597//XeZ7ERERT93uy6omR28LIURV02iFYvLkyaSmptK2bVtUKpXy+hdffFGtnROiOtXkCUVN/3YGcg1k/DL+57JCcfHiRXbv3l1iN35NVV70c1X4/vvvARgxYkSVt10V3N3d6d69O71798bT05P//Oc/z7tLlfKi1vKQehtCiJeRRjOExo0bV3c/BC/uROJRjRo1euknE/Di1vKQehtCiJeRRhOKtm3b4uLiQt++fUtkGtTUPRS3b99mwoQJJCcn06JFCwICAoiKimLDhg2oVCo6duzI/PnzqVOnDu3atVMCph4uwOXn58fhw4fR1tbG0tISV1fXEhsu+/Tpw6BBgzh+/Dja2tqsWrWK5s2bEx8fz+LFi9HW1qZLly7897//rXC1xN3dHQMDA44fP869e/eYO3cuERERnD9/ngEDBuDu7l5ubHdhYSFLly7lp59+wtTUlIKCArp3716iENeff/6Jj48PDx484Pbt24wdOxYXFxcCAwO5efMmly9f5urVqwwfPpxPP/203H7m5+fj7e1NYmIiaWlptGjRgqCgIGrVqsW3337Ld999R7169WjZsiVvvPEG06ZN49ChQwQEBJCfn0+zZs3w8fHB2Ni4av+whRBCaESjHIrMzEzefPNNkpOT+fPPP5V/aqpr167h5eXFDz/8QFpaGt9//z3BwcFs3LiRqKgoDAwMCAoKKvfzV69e5dChQ0RGRhIaGkpSUhI5OTkljvn777/p2bMn4eHhdOvWjU2bNpGXl8fnn3+Ov78/4eHhGt+CSk1NJTIykunTp+Ph4cHChQsJDw9n69at3Lt3j61btwJFsd3bt29n//79/Prrr+zZs4ezZ8+ya9cuVq9eTXJycqm2t23bxpQpU9ixYwfffvstK1euVN67cOEC69evZ9u2bYSEhHD37t1y+3jixAl0dXXZsmULe/fuJScnh4MHD3L+/Hk2bdpEWFgYmzdv5vLly0DRpG7FihWsX7+e8PBw+vTpw/LlyzW6HkIIIaqeRr+RZPNlSW+//TbNmzcHoFWrVty7dw8LCwvl27GjoyMeHh7lfr5Ro0bo6+vj5OSEhYUFM2fORF9fv9Rxffv2BYrirH/99Vf+/PNPGjZsqBTCGjZsGL6+vo/tb79+/YCiuO82bdrQsGFDAIyMjMjIyCAuLo5z585x9OhRAB48eMCFCxf473//y8CBA9HV1aVBgwZKOw9zd3fn559/5t///jcXLlzgwYMHyns9evRAT0+Phg0bYmRkxL1798pNAO3WrRtGRkZs2rSJixcvkpSUxIMHD4iLi8PCwkLJrBg8eDB3797l1KlTSoEyKEoMNTQ0fOy1EEIIUT00mlDEx8cTEhJCRkZGideLUyNrmkejouvXr1/i23dhYSH5+fklflapVMprOjo6bNu2jYSEBA4dOoSTk1OZty2KJxnFMdPa2tplRmE/zsPx32WtapQX211cpbSiz86cOZP69etjYWHBxx9/THR0dKn+PzyG8uzfv5+AgABcXFywt7cnPT2dwsJCtLS0yhxzQUEBXbt2JTg4GICcnBwyM1+8/RBCCFFTaHTLw9PTk/fffx9XV9cS/4j/iY2N5c6dOwBs3bpViYk2NjYmMTGRwsJCpdT32bNnGTVqFN26dWPOnDm0atWKS5cuPfYcLVu25O7du8qejKioqCrpe3mx3T179iQmJkaJ+S6O137Y4cOHmT59OgMGDODYsWMAJWpuaCouLg4rKyscHBwwMTHh2LFjFBQU0LNnTw4ePMj9+/fJzc3lxx9/RKVS0blzZ06ePKlct7Vr17Js2bLKXQghhBBPTaMVioYNGypLy6K0unXrMmnSJJydncnLy6Njx44sXLgQgM8++4zJkydjYmLCP/7xD9LT0+nQoQNdunTB2toaAwMD2rdvT79+/fjjjz8qPI+enh7Lli1jzpw5aGlp0aJFC40Lf1XEycmpzNhugN9//x1ra2tMTExo1apVqc9OmzaNkSNHUr9+fVq0aEHTpk1JSUl54j4MHz6c2bNnExMTg56eHl26dCElJYXhw4fj4uKCo6MjtWvXxtjYGH19fV577TWWLFnCzJkzUavVNGrUCH9//0pfCyGEEE9Ho2CrkJAQ6tSpQ9++fUssezdp0qRaOydKUqvVLF++HFdXV2rXrs2GDRu4efMm7u7uz7tr1ebSpUscPHiQMWPGAPDpp58yfPhw+vfvX+m2X+THRqs7cKemh/qAXAMZv4z/uQRb3b59my+//BIDAwPlNZVKxW+//fZEnRGVo6WlhZGREcOGDUNXV5emTZvi6+uLn58fR44cKXV8p06dNNq0+az8+uuv+Pj4lPleSEgIjRo1KvV606ZNlVUSlUpFnz59sLCwqJL+BHgMrZJ2qlp2Tt7z7oIQQjwxjVYoPvjgA7Zv346Jicmz6JMQz4REb9fcb2cg10DGL+N/LisUDRs2pEGDBk90YiFedM8jeltitYUQryqNJhTvvPMOI0eOxMLCAj09PeX1mpqUKZ6tefPm4eTkxFtvvcWcOXNYu3ZtlbT7PPZQSKy2EOJVpdGEIicnhxYtWpCUlFTN3RGitOJ9ICkpKZw/f/4590YIIURZJClTvFAKCwtZvnw5+/btQ1tbG0dHR/bt24erqysbNmwgNTWVqVOn0qZNG9RqNbNmzQLAw8ODvn378vHHHz/nEQghRM2k0YRi8uTJZb5enFIoRFWJiYnht99+Iyoqiry8PEaOHKnUOfH09MTFxYU1a9Zw5coVRo8ezb/+9S+ysrKIi4tTsj+EEEI8expNKAYNGqT8e15eHrGxsbRr167aOiVqrmPHjmFlZYWenh56enpERETg7Oxc6rjmzZvTtGlTjh07xrVr13j//fdL7O8RQgjxbGk0obCzsyvxs729PaNGjaqWDoma7dF6ISkpKSUKjj3MwcGBXbt2ce3aNaZNm/YsuieEEKIcGtXyeJRarSY1NbWq+yIE3bp1Y+/eveTl5ZGVlcX48eO5efMmUDTZeLjo2kcffURcXBxpaWl07tz5eXVZCCEET7mH4s8//6R79+7V0iFRs3344YecOXMGe3t71Go1Li4u/PDDD0BRHkqTJk1wdnZm48aN1KpViy5dutC2bdvn3GshhBAaJWXu3Lnzfx9QqTA2NqZPnz5oa2tXa+eEKE9hYSGZmZk4Ojry9ddf89prrz3vLmnkRQm2qukpgSDXQMYv43+mSZnF5bjLqp1w7949jIyMnqgzQlSV33//nfHjxzN16tSnnkzU5OhtIYSoahVOKMzNzVGpVEDRN8KHqVQqzp07V309E6ICZmZmJCQkVKqNZxW9/aKsSgghRHWqcEIhqYTiVfasorclblsIURNo9JSHWq3mP//5D87OzowYMYKgoKASu+3Fy+HevXtMmTKlwmM8PDy4evVqhcc4OzsTHx9flV0TQgjxktNoQrFixQqOHj3K6NGjGTt2LCdOnMDPz6+6+yaqWEZGxmNXneLj40vd3hJCCCEeR6PHRn/++Wd27NiBrq4uAB988AFDhgyp1o6Jqrd48WKlFkb//v3ZsGEDKpWKjh07Mn/+fDZt2kRqaioTJ05k06ZNHD16lA0bNpCdnU1OTg6LFy+mW7dujz1PSkoK48ePx9jYGH19fb766iuWLFlCXFwcKpWKIUOGMHHiRKAovj0yMhJtbW169+6Nm5sb169fZ+rUqTRv3pw///yTTp060b17d3bu3ElGRgZr1qyhVatW+Pn5cfjwYbS1tbG0tMTV1bW6L6EQQohyaLRCUVhYqEwmAPT09Er8LF4Onp6emJqaMn36dIKDg9m4cSNRUVEYGBgQFBTExIkTMTU1JSQkBENDQ0JDQ5Vf+BMmTGD9+vUan+vSpUv4+/vz9ddf8/3333P9+nUiIyPZtm0bP/74Iz/99BMHDx4kNjaWsLAwdu7cyeXLlwkNDQXgwoULTJkyhZiYGH7//XeuXr3Kli1bsLa2ZsuWLVy9epVDhw4RGRlJaGgoSUlJSs0PIYQQz55GKxRvv/02S5YsUeK2v/vuOwkTeokdO3YMCwsLjI2NAXB0dMTDw6PEMVpaWqxZs4bY2FguXbpEQkICWlqaB6s2bNiQZs2aAUW3Uezs7NDW1sbAwAAbGxvi4uLQ0tJi8ODB1KpVCyiK0g4PD+f999/HxMSEDh06ANC4cWN69uwJQJMmTUhJSaFRo0bo6+vj5OSEhYUFM2fORF9fv9LXRgghxNPR6DfEggULuHv3Lk5OTnzyySekp6czf/786u6bqCZqtbrEz4WFhaU22WZmZuLg4EBKSgrdunUrs0BXRYonCeWdr6CgoNTrgNKPRwt9PRqipqOjw7Zt25gxYwZ37tzBycmJS5cuPVEfhRBCVJ0KJxS5ubnMmTOHo0ePsnTpUo4cOYKZmRna2trUrftsnuEXVae4Fkb37t2JjY1Vgsu2bt1Kjx49gKJf3AUFBSQlJaGlpcXkyZMxNzfn0KFDFBQUPNV5zc3NCQ8Pp6CggKysLKKioujRowfm5uZER0eTnZ1Nfn4+O3bswNzcXKM2z549y6hRo+jWrRtz5syhVatWMqEQQojnqMIJRUBAAPfv3+fdd99VXvPx8eHu3bsEBgZWe+dE1SquheHr68ukSZNwdnbmo48+4u7du8ycORMo2nA7ceJE6tWrR/v27bGyssLOzo7atWtz7dq1pzqvo6MjjRs3xtbWlqFDh9K/f38+/PBDLCws+OCDD3BwcGDw4ME0bdpU4yq2HTp0oEuXLlhbW2NnZ0fTpk3p16/fU/VPCCFE5VVYy8Pa2prt27eXWL4GuH//Po6OjkRHR1d7B4V42b2ISZk1vY4ByDWQ8cv4n2ktD11d3VKTCYC6deuWusctapbk5GSmTZtW5nuLFy/mnXfeecY9enJSy0MIIapOhRMKLS0t7t+/X2q/xP379yUps4Z74403iIiIeN7dqJTqruXxIq5MCCFEdalwQmFtbY2npydLliyhdu3aADx48ABPT08GDhz4TDr4NOLj4wkKCmLjxo3P9LMvs/79+/Ptt98qj3o+K+3atePChQsaH+/s7Iyrq6uyibSYu7s73bt3x97eXuO2qruWh9TwEELUJBVuyhw9ejT16tWjd+/efPLJJwwbNozevXtTv359pk6d+qz6KIQQQogX3GNvefj4+DBp0iTOnj2LlpYW77zzDo0aNXpW/Xtq6enpjBs3jtTUVMzMzFiwYAFHjx4lICCA/Px8mjVrho+PD8bGxvzyyy988cUX6Ovr06JFC6UNZ2dnDA0NSUxMZNWqVdy4cYNVq1ahVqtp3rw5ixYtwsTEhJMnT+Lr60tOTg7GxsYsWrSIN998E2dnZ9q3b09cXBzZ2dl4enqyceNG/vrrL8aMGcOYMWOIi4vD398fAENDQ1asWEGDBg3KHZe5uTkWFhacOXOGOnXqsHz5cpo1a8bp06f54osvyM7OxtjYmIULF9K8efNSY2jfvn25ba9Zs4Zz586RlZXFsmXL6Ny5M5cuXcLLy4s7d+5Qu3Zt5s2bh5mZWakVgeKVhvLGEx4ezjfffINaraZjx44sWLBACaLy8vLi5MmTAAQGBvLmm2+We02LFRYWsnTpUn766SdMTU0pKCige/fuT/lfixBCiMrSKNiqWbNmDBw4kAEDBrwUkwkoqicxf/58IiMjyczMJCQkhBUrVrB+/XrCw8Pp06cPy5cvJzc3F3d3dwICAggLCyu1CbVdu3bs2bMHU1NTvLy8WLNmDVFRUXTt2pVFixaRm5vLrFmzlHM5OTkxa9asEm1ERUVha2vL4sWLCQwMZNOmTaxZswaAtWvX4u3tTVhYGBYWFpw9e7bCcaWnp9O9e3eioqIYPHgwixcvJjc3F09PT1asWMHOnTsZO3ZsieCx4jFUNJkAaN26NeHh4Tg7Oysx225ubjg7OxMVFYWHhwczZswgNze33DbKGk9iYiJbt24lNDSUiIgIGjZsWCLGu1evXkRGRtK7d29CQ0M1uqZ79uzh7Nmz7Nq1i9WrV5OcnFzh2IQQQlQvjaK3X0bvvfceb731FgA2Nja4u7ujUqlwcXEBitIbDQ0NuXDhAqamprRq1QoAOzs7Vq9erbRjZmYGwOnTpzEzM1P2GDg6OhISEkJSUhL169dXjrOyssLLy4t794oexynORmjSpAmdO3fGwMCApk2bcvfuXQClqNWAAQOwtLSkd+/eFY5LX1+foUOHKn398ssvSUpK4sqVK3z66afKcffv3y81hscZMGAAUDSx2LNnD5mZmSQnJyv7Zbp06YKhoSEXL14st42yxvPdd99x+fJlPvnkEwDy8vKUWO1Hz/vrr78+9poCJCQkMHDgQHR1dWnQoIFkUAghxHP2yk4odHT+N7TiqI2uXbsSHBwMQE5ODpmZmVy7dq1EBPSjEc/FKxblxVWXFR9dHC0NlCii9nCfio0ZMwYLCwsOHDiAv78/p0+fLjExeJSWlhYqlUrpk7a2Nmq1mmbNmilPXRQUFJCWllZqDI9TPPbi9gsLC0uVMi8em0qlUt7Ly8urcDy1a9fGysoKT09PoCjW++HUzeLrUtzm465p8bEPH1fWtRVCCPHsaF7t6SVz/PhxZbIQHh7O6NGjOXnypBLPvHbtWpYtW0a7du24desW58+fByg3rKtz586cOnWKlJQUALZs2UKPHj1o2bIld+7c4fTp0wDs3r2bJk2aYGRkpFE/hw8fTmZmprKn4nG3PLKysoiNjQUgLCyMfv360bJlSzIyMvj1118B2LFjB7Nnz9bo/BWpW7cuzZs358cffwTg5MmTpKWl0aZNG4yMjPjrr78A2LdvX4Xj6dGjB3v37uXWrVsUFhbi7e3NN998U+55NbmmPXv2JCYmhtzcXDIyMvj5558rPV4hhBBP75X9Wte6dWvmzp3L33//jbm5OZ9++ikdOnRg5syZqNVqGjVqhL+/P7q6unz55Ze4ubmho6NTYin+YSYmJixatAhXV1fy8vKUCGs9PT1WrlyJj48PWVlZGBoasnLlSo37OWvWLNzd3dHR0UFfX5+FCxc+9jMxMTGsXLkSU1NT/Pz80NPTY/Xq1comxrp16+Ln56dxHyri7++Pt7c3gYGB6OrqEhgYiJ6eHiNHjmTmzJnY2Nhgbm7Oa6+9Vu542rZti6urK6NHj0atVtO+fXsmTpxY7jk1uaYDBgzg999/x9raGhMTE+WWlRBCiOejwuht8eJ50twG8fy8yMFWNT12GOQayPhl/M80els8e9nZ2Tg6Opb53vTp0yvVtrOzs7IZ9GFOTk6MGDGiUm2/jCR6Wwghqo6sUAhRTWSF4sVW06+BjF/GLysU4qVUXmT2o8cUR57b2tpWe60Qid4WQoiq88o+5SFePgkJCcq/v+yFx4QQoqaRFQoBFBVFCwwMREdHh+vXr2NmZoavry9RUVFs2LABlUpFx44dmT9/PnXq1Ck3AvzhImNlFVrLz8/H29ubxMRE0tLSaNGiBUFBQSxfvhwoeux027ZtyubTrKwsPD09uXDhAiqVinHjxjF06FDCwsL4+eefycjI4MqVK/Tu3Rtvb+/ndPWEEELICoVQnD59Gi8vL2JiYsjJySEkJITg4GA2btxIVFQUBgYGBAUFAWVHgGvixIkT6OrqsmXLFvbu3UtOTg4HDx5UQq+2bdtW4vjAwECMjY3ZtWsX33zzDYGBgUpmyIkTJwgICCAyMpIDBw7I0y9CCPEcyYRCKLp160bLli1RqVTY2tqydu1aLCwsMDY2Borixo8ePQqUjgCPj4/X+BwjR45k06ZN+Pr6kpSUxIMHD8o9/ujRowwbNgyABg0aYGlpqdwaeffdd6lbty4GBgY0b96cjIyMpx67EEKIypEJhVA8HDteVgR2cdw4lB0B/vBxgHLsw/bv38/s2bOpVasW9vb2dOvWrVS896PnfPTn4gju4mqlQIkocCGEEM+eTCiE4vjx49y8eVOJK/fw8CA2NpY7d+4AsHXrVuUpjbIiwAGMjY2VSO79+/eXOkdcXBxWVlY4ODhgYmLCsWPHlAmCtrZ2qUmIubk527dvB+D27dvs379fypQLIcQLSCYUQmFqasrnn3/Oxx9/TKNGjRg1ahSTJk3C2dmZjz76iLt37zJz5kzl+JiYGGxsbPj555+ZO3cuUBS+5evri4ODA/Xq1St1juHDhxMdHc3QoUOZNm0aXbp0UeqjWFpaYmtrS05OjnL81KlTuXPnDjY2NowaNYrJkyfTsWPHar4SQgghnpQEWwmAMp/IqIhEgD+eBFu92Gr6NZDxy/gl2EqIKiLR20IIUXVkQiEA6NGjR4Uplo+S1QkhhBAPkwmFqLEqWrqrrBf5docQQlQHmVDUQE+6X6IqhIWFkZCQwNKlSzU6PiUlBRcXF+VJkodV1f6N6qzlIXU8hBA1jTzlIYQQQohKkxWKunqtzAAAIABJREFUGur27dtMmDCB5ORkWrRoQUBAQLl1Ox5eEXh4pcHPz4/Dhw+jra2NpaUlrq6uZGZmsmjRIhITEykoKGDChAlYW1sDcPnyZZydnbl27Ro9e/ZU4rqDg4OJjIxEW1ub3r174+bmVqKvKSkpuLm58eDBAzp37qy8HhcXh7+/PwCGhoasWLGCBg0aPIvLJ4QQ4hGyQlFDXbt2DS8vL3744QfS0tL4/vvvy63bUZarV69y6NAhIiMjCQ0NJSkpiZycHNatW0fHjh0JCwtj06ZNBAcHc+XKFQCuX79OYGAgP/zwA4cOHSIxMZGDBw8SGxtLWFgYO3fu5PLly4SGhpY4l4+PD/b29kRERNC1a1fl9bVr1+Lt7U1YWBgWFhacPXu2ei6WEEKIx5IJRQ319ttv07x5c7S0tGjVqhX37t0rt25HWRo1aoS+vj5OTk58/fXXzJw5E319fY4cOUJoaCi2trb885//5MGDByQmJgLw3nvvYWRkhN7/196dR2VVdQ8c/z6AA4TibJJDikkaOfzUIlMLQiZBTNJ4VZwhxBGUVwglRDGcokDTTMskjVRIQU1dBYqzluaQYzliCqGoiCLT/f3B4r4+MogMorA/a7GW3PHse596Dufcu3fNmrRs2ZLU1FT2799P3759qV27Nnp6ejg7O7Nv3z6tcx08eBA7OzsA+vXrR40aNQDUUZGgoCBMTEzo2bNnRVwqIYQQJSBTHtWUnt7/br1Go6Fu3brcuXNHXfZw3Y783zUajbpMT0+PdevWcfDgQRISEnBxcSEiIoLc3Fzmz5+vZrNMSUnByMiI2NjYAucsrF4IFF4DJD//mkajUWuIjBgxAgsLC+Lj45k/fz7Hjh1j7NixZbksQgghSklGKISqqLod9evX59y5cyiKor51cfLkSYYOHUr37t2ZNm0aJiYmXLhwAXNzc3744QcAkpOT6devH9euXSvynObm5mzevJmMjAyys7OJiorC3Nxca5sePXoQExMDwPbt28nMzATy0ninp6czYsQIRowYIVMeQghRiWSEQgBgaGio1u3IysritddeY+bMmQBMmTIFDw8PGjVqRNeuXUlNTaVDhw507twZBwcH9PX1ad++Pb179+aNN94gMDAQBwcHcnJy8PHxoWXLlvz222+FntfCwoJTp07h7OxMdnY2vXr1YujQoVy/fl3dJiAgAB8fHyIjI3n99dd54YUXAPD29sbX1xc9PT1q1aqltlcIIcTTJ7U8hKgAz3piq+pexwDkGkj8Er/U8hCinEgtDyGEKD/SoRDVVkWk3n7WRyaEEKKiSIdCVFsVkXpbUm4LIaorecvjGZeWloanp2ex2/j5+XH16tVit3F1deXAgQNFrk9MTMTS0rJUbSyP/YUQQjzfpEPxjLt9+zanT58udpsDBw4gz9YKIYSoTDLl8YybPXs2ycnJjBs3DktLywK1NlavXk1ycjLu7u6sXr2a/fv38+2335KRkcGDBw+YPXs23bt3L9G5Hjx4wKRJk7hw4QItW7YkODgYIyMjjh07xqeffkpGRgb169dn5syZtGjRgpMnT+Lv7w/kZd7M5+vry61bt7h06RI+Pj40aNCA4OBgHjx4QP369QkKCqJVq1ZcuHCBgIAAbt26hYGBAf7+/nTs2BFfX1/09fX5/fffSUtL4+OPP2bjxo2cPn0aKysrfH19OX36NAEBAWRnZ1OrVi0+/fRTXn755Yq4BUIIIUpARiiecdOnT6dJkyZMnDix0Fob7u7uNGnShGXLlmFkZERkZKRabMvNzY0VK1aU+Fw3btzA1dWVmJgYWrZsyeLFi8nMzGT69OksXLiQn376iZEjRzJjxgwApk2bho+PDz/99BPNmzfXOla9evX4+eef6dmzJ97e3syYMYOYmBhcXFzw9vYGwMfHB1dXV2JjY/Hz82PSpElq0qrk5GRiYmKYOHEifn5+zJw5kw0bNrB27VrS0tL47rvvGDlyJNHR0bi6uvLHH3+U0xUXQghRGjJC8Zw4dOhQgVobfn5+Wtvo6OiwePFi4uLiuHDhAgcPHkRHp+R9xtatW9OtWzcgr2aGr68vFy9e5MqVK1opre/evcvNmzdJTk6mR48eAAwYMICoqCh1m44dOwJw8eJF6tatq/5uZ2dHQEAAaWlpXL58GWtrawA6d+6MkZER58+fB6B3794AGBsb88orr9CwYUMgr6Ny+/Zt3nnnHYKCgti1axcWFhbY2NiUOE4hhBDlTzoUz4lHa148WmsDID09HWdnZ5ycnOjevTumpqasXr26xOd4uNZG/u+5ubk0b96cjRs3ApCTk0NKSopaiyOfrq6u1r61a9cutN35bU9LSyvw3IeiKOTk5ACoBcAKaxeAra0tXbp0IT4+nu+++46dO3eq5dCFEEI8fTLl8YzT09MjOzubN954o8haG7q6uuTk5HDx4kV0dHTw8PDA3NychIQE9Qu6JP7++2+1Hsb69evp0aMHbdq04fbt22rq7KioKKZOnUr9+vUxNjZmx44dAGzatKnQY7Zp04Zbt25x7NgxALZs2YKxsTHGxsa0aNGC7du3A/DHH3+QkpLCK6+8UqK2Tp48mWPHjuHi4sKkSZOkjocQQlQyGaF4xjVs2BBjY2OCg4OLrLXx7rvv4u7uztdff0379u2xs7Ojdu3adO/enX/++afE58p/buLy5cu0a9cOLy8vatasyRdffKE+VGloaMjcuXMBmD9/Pn5+fnz++ed07ty50GPWrFmT0NBQZs2axf379zEyMiI0NFTdPzAwkPDwcGrUqEF4eDg1a9YsUVs9PDzw9/fnyy+/RFdXF19f3xLHmS/Mr/8T7/M4GQ+yyv2YQgjxPJBaHqLaqs6pt6t7HQOQayDxS/xSy0OU2uXLl5kwYUKh62bPns3rr7/+lFskhBCiqpAORTXSsmVL9eFKUT61PKR2hxBC5JEOxVPwxRdfYGZmxnvvvVei7d3c3Jg9ezaZmZksWbKEOXPmlHub7t69i4uLC0uXLlVzSJiamnLmzBmt7SwtLVm1alWBPBPp6eksWLCA3bt3o6+vj6GhIRMmTOCtt95St9mxYwdLly7l3r175ObmYmVlxcSJE9HR0cHX15f9+/djZGSkddzo6Gj1jZHvv/+ekJAQ4uPjady4sbqNqakps2bNYtCgQeoyV1dXxo8frz6oWhLlUctDancIIUQe6VA8BZMmTXqi7b/++msgL6X2lStXynz+nJwcfvnlFwwMDOjVqxdHjx5l+vTpXLx4sVTHUxQFDw8P2rdvz+bNm6lZsyYnT57E3d2dhQsX8uabb5KQkEBQUBArVqygdevWZGRkMHnyZMLCwpg8eTIAEydOZMCAAUWeJzo6GktLS9avX6+VBwMgNDSUXr160axZs1LFIIQQonxV+mujS5cuxd7eHkdHR0JCQtTXHFeuXImNjQ329vbMnz8fgJSUFDw9PRkwYADOzs7s3bsXgKSkJEaPHs2gQYOwsLBgwYIFQN4XkpeXF6NGjaJPnz4EBgYW25bExEQcHR3x9PSkb9++uLm5qa9pmpubM3r0aJycnMjKyiq03b/++ivW1tY8ePCAS5cu0bt3b65fv46vry/R0dEkJibi5OTE+PHjsba2xtvbm8jISD788ENsbW35+++/gbxRgcTERGbPns2JEyeYOXMmPj4+/Pjjj2pbXV1dOXr0aLHxpKWl8c0332Bvb8/27dvVUYa1a9fyySef0KRJkye8W3kOHjzIP//8g5+fn/pWRocOHRg7dixffvklkHdfx48fT+vWrYG8vBSBgYG88cYbJTrH6dOnuXXrFu7u7qxbt65APothw4Yxffr0UrVfCCFE+avUDsXOnTuJi4sjOjqan376iUuXLhEZGcmxY8dYs2YN69evJyYmhj///JMTJ04QHByMs7Mz0dHRLFmyhICAAO7evcumTZtwcHBg7dq1xMTEsGbNGm7evAnAkSNHCAsLIyYmhvj4+AJD+o86e/Ysw4cPZ/PmzZiYmLBo0SIAUlNTcXd3Z+PGjezdu7fQdr/33nt06dKFJUuW4Ofnx7Rp03jxxRe1jn/mzBk8PT3ZunUrx48f5+rVq/z44484ODhodRggL+22mZkZn3zyCc7OzsTExABw9epVbt68SadOnYqM46uvvsLZ2Zn09HQiIiJYuHCh+uUeHBysZsR8lJOTk9ZPcnJygW2OHz+OmZkZGo1Ga3n37t05fvw4AKdOnSrQvhdffFHNrAkQFhamda7812AhrzNoa2uLmZkZurq67Nq1S+tYbm5upKamsnbt2iKvgRBCiKenUqc89u/fT9++fdWsis7OzmzYsIGMjAwsLCyoU6cOkDdaAbB3717Onz9PWFgYANnZ2Vy5coXRo0ezf/9+VqxYwblz58jKyuL+/fsAdOnSBUPDvIfvWrRowe3bt4tt08svv6zOw/fv35+pU6eq6/K/IItq95AhQ/D398fe3p7/+7//o2/fvgWO36hRIzp06ADkfcHmP3NgbGxMYmJike168803mTFjBomJiWzcuBEnJ6di49BoNOrPk6TffvShzcJKkms0mkITZmVlZWlt87g3koua8sjKyiI2NlatQ2Jvb09kZCTvvPOOuo2enh4hISEMHz6cXr16FR+UEEKIClepHYrC0jJnZ2cXSLWclJSEvr4+ubm5fPfdd9SrV09d3qhRI0JCQrhy5QoODg5YWVmxd+9e9cusVq1a6nFK8iX38LkVRdFKKV1cOun8NNgpKSno6upy4cIFMjMzCyRqevT3R1NWF0Wj0dC/f382b97M1q1bWb58ebHbu7u785///Id169YxZMgQOnXqxLhx42jVqlWJzlecTp06ERERQVZWllaK7D/++EN99dTMzIwTJ07Qtm1bdf2FCxdYsmQJ8+bNK/b4O3bs4M6dO4wfPx7I62DcuHGD69eva434tGvXTqY+hBDiGVGpUx7m5uZs3ryZjIwMsrOziYqKwtzcnG7dupGQkEB6ejrZ2dlMmTKFEydOYG5uzpo1awD466+/6NevH/fv32fPnj2MHj0aOzs7rl27RlJSUqFf+iVx4cIFTp06BeSlmc4vUlWSdufk5ODn54e/vz/du3fn888/L/3FIa+z8XC9jgEDBhAZGcmLL75I06ZNH7t/nTp1GDVqFFu2bOHdd98t9UOYj+rWrRtt27Zlzpw56qjEiRMnWLJkCZ6engCMGTOGRYsWqedMT08nJCSkRA9RRkVFMWnSJOLi4oiLi2PXrl107dqVdevWFdg2f+rjyJEj5RKbEEKI0qnUEQoLCwtOnTqFs7Mz2dnZ9OrVi6FDh6Knp8fQoUNxcXEhNzeXPn360KNHD0xMTAgICMDR0RGAefPmYWhoyEcffcR///tf6tatS8OGDTEzMyt2+qA4RkZGhIWFcfnyZUxNTQstOFVUu7/55hsaNmyItbU1PXr0wMHBQa2mWRomJiakpaXh4+PD/PnzadasGc2aNeP9999/ouPo6upib29f6nYUZtGiRYSGhuLg4ICuri5GRkbMnz9fnS7q3bs3Xl5eeHl5kZOTQ3Z2Nra2tuqoA+Q9Q/Hdd99pHdff358DBw4UeFV25MiRBAYGqh2WfPlTH8W9LSKEEKLiSerthyQmJjJs2DDi4uIquykFKIpCcnIyrq6ubNq0qcQ1L0TFel4TW1X3tMMg10Dil/gl9XYZPS799LNq27ZtBAYGEhgYqHYm5s6dq746+zAzMzOCg4OfdhOfO9W5locQQpQ3GaEQogjP6+hDSVT3v85AroHEL/HLCMUzpDJSaoeHhxMbG0tMTIz61smBAwdYtGgRERERhIeHExkZSaNGjbT2W7p0qfpAZHx8PB4eHkRFRWFmZqZuY2lpSe3atdU3N9LS0jAzMyMkJAQDA4MCbVm5ciU//vgjurq66OrqMmjQIIYMGaKuP3/+PPPmzePq1atA3lsZ/v7+NGjQgOjo6EIf0gwKClJfzz179iyOjo6EhYVhY2OjbuPq6kqTJk1YuHCh1nUBihx9KszjUm9LWm0hhCg56VCUQWWl1P7nn3/47LPP+Pjjjwtd7+LiUuwXa3R0NDY2NkRGRhaY5lm2bJmaUTMzM5PBgwezYcMGBg8erLVdeHg4hw4dIiIigkaNGnHz5k08PT25desW48aNIykpiWHDhhEUFISlpSWKovDVV18xfvx49U0dS0tLQkJCStTOhzsUkDcFZGdnh5WVVdEXSgghxFNTYa+NSkrtikup7eLiwpYtW/jtt9+e+L7cvHmTffv28d///petW7dy9+7dIrdNS0sjLS1NzfuR7/79+6xYsYI5c+aoIyENGjRg9uzZLF++nPv37/PDDz/Qs2dPNTGWRqPBzc2NwYMHa70KW5Ts7GxiYmLw8vLi5MmTXL58WWv92LFjmTlzpnofhRBCVK4K6VBISu2KS6kNea+2BgYG4u/vT0ZGwSH5yMhIrZTW48aNU9fFxsby9ttv07x5c8zMzApkxnR3d8fR0ZEePXrg5ubG0KFDsbOz09rm3Llz6OvrF6hA2rZtW2rWrMn58+c5deoUHTt21Fqvq6uLg4ODmjwsLi5Oq50DBw5Ut92xYwfGxsa0bt0aKysrIiMjtY7VrVs3bG1tmTVrVrHXSgghxNNRIVMeklK74lJq57OysuLnn3/ms88+K/AMR3FTHtHR0WouCHt7e77//nut5x7ypzy2bdvGp59+iqWlZYGaHUWl3ob/ZQwtSVbS4qY8oqOjcXBwUNs5depUJk+erPW6rLe3N05OTvzyyy/FnkcIIUTFq5AOhaTUrriU2g+bMWMGDg4OBaYkinLy5EnOnj1LcHAwn376KTk5OSQnJ3PkyBG6dOmita2NjQ179uwhICBAramRr23btmRlZXH+/HnatGmjLj937hy5ubm0adNGTb39sNzcXCZOnPjYKaobN26QkJDAiRMnWLVqFYqicOfOHbZv3652MgD09fWZM2cOXl5e2NjYYGRkVKLrIIQQovxVyJSHpNQuXllTauerV68egYGBasnwx4mOjmbQoEHs2LGDuLg4du7ciZOTU4EpmXyTJk3iyJEjxMfHay3X19dn7Nix+Pv7c+PGDSCvEzBjxgzGjBmDvr4+H374ITt37mTnzp1AXifuyy+/5MaNGwXeQHlUTEwM5ubmJCQkEBcXp76VUlg786c+Hp0SEUII8XRVyAiFpNQuXnml1Ia8qQ8bGxutMuORkZEFpgF8fHyIjY1l1apVWstHjBjBhx9+iJ+fX4FjN2zYEDc3N+bNm0evXr20Rnnc3d2pU6cOI0aMQFEUNBoNLi4u6vRJ48aN+frrr5k3bx4LFiwgJyeHDh06sHjxYvUY+c9QPGzkyJHqQ7cPGzx4MMuXL1cfcH2Yt7e32nERQghROapFYitJqS1KQxJbVW3V/RpI/BK/JLYqgqTUFk9KUm8LIUT5qRYjFEIUpjp3KKr7X2cg10Dil/jLe4SiwhJbiZL54osv+PXXX0u8vZubG0lJSVy5cqXITJmPEx4ejrW1tVYOiwMHDuDq6qquf/vtt7VyRDg5OXHt2jV1+/j4eExNTQu8yWFpaYm9vb26j6WlJRMnTuTevXsF2mFqaqpuZ29vz5QpUwrk1Zg7dy7m5uZkZmaqy4KCgpg4caLWdrt37+a9994rNlHXoxo2NKRx4zpaP3Xq1i7x/kIIIf6nykx5PK+qc/puQE2spSgKEyZMICoqSn2wMzs7m59//pkuXbqwdetW+vXrB8CUKVNwdHQkLi4OS0tL7t27R2BgIHPmzFFzk5REYbU8pH6HEEKUzjM9QiHpu6tu+u5H5Scte/iV0p07d9KiRQv69++vFd8LL7zA7NmzCQoK4t69e4SFhWFpaakmLhNCCPH0PbMdCknfXbXTd+fLP36vXr34999/1QyjkNfps7W15Z133uHUqVP89ddf6roePXrQs2dP/Pz82LNnD97e3sXGK4QQomI9sx2Kh9Ng6+np4ezszL59+zh06JCavltPT4+VK1diZmbG3r17CQsLw8nJCTc3N6303c2aNWPFihUEBwcXmr5bX1+/VOm79+/fr64rLH33w+0G8Pf3Z/369TRq1KjY9N06OjoF0nffuXOnyHa9+eabJCcnk5iYyIYNG54ofbeZmRmfffZZgXUuLi5s3LhR/Xk4f8SjabEf7ewsW7aM2NhYPvnkE27evFlo+u58+cfft28fvXr1UvNP3Lx5k927d2NnZ0ft2rWxsLAokLzK19eXPXv2MH36dDXbqRBCiMrxzD5DIem7q3b67kfp6Ojg6OioZkyNiYlBURQ++OADADIyMsjKymLq1KnqtTY0NKRu3bq89NJLJY5XCCFExXhmRygkfXfxnvf03YXZt2+fWmAtKiqKkJAQ4uLiiIuLY/fu3RgZGbFly5YSxyaEEOLpeWZHKCR9d/GqQvpuQJ2eyc7Opn79+gQFBXHixAlSU1Pp06ePup2Ojg7Dhw8nMjKSAQMGPHGMQgghKpYktiohSd9d9RT12mh1SHZT3ZP6gFwDiV/il9TbFUjSd1cvYX79CyzLeJBVCS0RQojnn4xQiGpLUm9X37/OQK6BxC/xS+ptIYQQQjxzpEPxnKuMWiDR0dH4+voCeam1Z86ciYODA46OjgwZMoRjx44B2vVBCnP27FlMTU3Ztm2b1nJXV1emTJmitSw8PJzw8HB1fZ8+fXBycsLBwYEBAwawc+fOUsUihBCifMgzFM+5yqoFkm/lypXk5uYSGxuLRqPh999/x9PTs0SviT5cD8TGxkZr3bZt27Czs8PKyqrQfWfPnq0mGTt+/Dhjxoxh9erVtG3btswxCSGEeHJVfoSisLoaUguk7LVA8qWkpJCVlUVWVt7DjF27dmXOnDmPzfWRnZ1NTEwMXl5enDx5ksuXL2utHzt2LDNnzlSvUXFef/117OzsWLduXYnaLIQQovxV6Q5FYfVAlixZIrVAKHstkHzDhg3j6NGjvPXWW4wdO5ZVq1bRpUsXrSykhdmxYwfGxsa0bt0aKyurAmm1u3Xrhq2tLbNmzSpRO1555RXOnz9fom2FEEKUvyrdoSisrsaiRYukFsgjSlsLBKB58+Zs2rSJb7/9lk6dOqn7F3c+KFgP5KeffiIzM1NrG29vb44fP14gwVZhNBqN1PMQQohKVKWfoShs2P3Rt2SlFkjZaoF89tlnDBkyhI4dO9KxY0c8PDxwcXFhz549NGjQoNB9bty4QUJCAidOnGDVqlUoisKdO3fYvn272skA0NfXZ86cOXh5eWFjY4ORkVGR7Thz5gwmJiYlbrcQQojyVaVHKAqrqzFt2jSpBUL51QJJSkpi8eLF6ujCrVu3SE1NpV27dkXuExMTg7m5OQkJCcTFxREfH4+Hh0eh9UDypz4enRJ52LFjx9i2bZtaSEwIIcTTV6VHKAqrqzFs2DBq1aoltUDKqRbIjBkzmDt3Lra2tujr61OjRg2mTJmCiYkJKSkp/Pbbb1pVSB0dHTly5Ihapjzf4MGDWb58ufrw6MO8vb0LvBY6ffp0DAwM0Gg06OvrExoaSvPmzZ+o7UIIIcqPZMp8SqQWyLMnNTW92mbKbNjQkBs37lZ2MypVdb8GEr/E/6Tx6+hoqF//hSLXV+kRiqdNaoE8X4r7D6M6KC6FbnVR3a+BxC/xlycZoRBCCCFEmVXphzKFEEII8XRIh0IIIYQQZSYdCiGEEEKUmXQohBBCCFFm0qEQQgghRJlJh0IIIYQQZSYdCiGEEEKUmXQohBBCCFFm0qEQVU5sbCz29vb06dOH1atXF1ifXyvFxsYGf39/tUjaP//8w5AhQ7C1tWXs2LGkp6c/7aaXi9LGv2HDBnr27ImTkxNOTk6EhoY+7aaXi8fFn2/atGlER0erv1eV+w+lvwbV5TPwyy+/4OTkRL9+/fD09OT27dtA1fkMlDb+Mt9/RYgq5Pr164qFhYWSmpqqpKenK46Ojsq5c+e0tunbt69y5MgRRVEUxc/PT1m9erWiKIri7u6ubNq0SVEURVm0aJEyb968p9v4clCW+IOCgpTY2Nin3ubyVJL4r1+/rnz00UdKx44dlaioKHV5Vbj/ilK2a1AdPgNpaWnK22+/rVy/fl1RFEX5/PPPlVmzZimKUjU+A2WJv6z3X0YoRJWyd+9ezM3NqVevHgYGBtjY2LB161Z1/dWrV8nIyKBz585AXtn2rVu3kpWVxaFDh7CxsdFa/rwpbfwAx48fZ8OGDfTr14+pU6eqf7U8Tx4XP+T99fbee+9hZ2enLqsq9x9Kfw2genwGsrKyCAwMpGnTpgCYmppy7dq1KvMZKG38UPb7Lx0KUaUkJyfTuHFj9fcmTZqQlJRU5PrGjRuTlJREamoqhoaG6OnpaS1/3pQ2/vx/T5gwgY0bN9KsWTOCgoKeXsPLyePiBxgzZgwDBw7UWlZV7j+U/hpA9fgM1K9fHysrKwAyMjJYtmwZVlZWVeYzUNr4oez3X6qNiipFKaTWnUajeez6x+33vCht/ACLFy9Wl40ZM0b9n8zzpLT3sarcfyhbLNXpM5CWloanpyevvvoq77//fqGdh+fxM1Da+KHs919GKESV0rRpU1JSUtTfk5OTadKkSZHr//33X5o0aUKDBg24e/cuOTk5WsufN6WNPy0tjZUrV6rLFUVR/1J7njwu/qJUlfsPpb8G1ekzkJyczODBg3n11VcJDg4Gqs5noLTxl8f9lw6FqFJ69OjBvn37uHnzJvfv32f79u307t1bXf/SSy9Rq1Ytfv/9dyDvqebevXtTo0YNunXrxpYtW7SWP29KG7+BgQHLly/n6NGjAHz//ff06dOnUmIoi8fFX5Sqcv+h9NegunwGcnJy8PDwwM7ODn9/f/Wv96ryGSht/OVy/0v9OKcQz6iYmBilb9++irW1tbJs2TJFURRlzJgxyrFjxxRFUZRTp04pzs7Oiq2treLt7a08ePBAURRFSUxXj5WFAAAEtklEQVRMVIYOHarY2dkpo0aNUm7dulVpMZRFaeM/dOiQ0r9/f8XW1lbx8PBQ7ty5U2kxlMXj4s83bdo0rTccqsr9V5TSX4Pq8BnYvn27YmpqqvTr10/9+fjjjxVFqTqfgdLGX9b7r1GUQiZchBBCCCGegEx5CCGEEKLMpEMhhBBCiDKTDoUQQgghykw6FEIIIYQoM+lQCCGEEKLMpEMhhBDFsLS05Pjx40/1nDt27OCLL754qucUoqykQyGEEM+Y48ePP5eFuUT19vzlVRVCiErw+uuvM2LECHbs2MHdu3fx8fFh69atnD17liZNmrB06VIMDAzo0KEDw4cP58CBA9y7dw9vb2+sra2BvFoJmzdvRldXl9atWzNjxgwaN26Mq6srRkZGnD9/Hnt7eyIjI8nJyaFOnTp89NFHBAYGcvHiRW7fvs0LL7zAggULaNOmDa6urnTu3JnDhw9z7do1unbtyty5c9HR0SE+Pp7PP/+c3NxcDAwMmDlzJq+++iqHDx9mwYIF3L9/H41Gw4QJE7CwsKjkqyuqAulQCCFECWRmZtK4cWNiY2NZtmwZ06dP5+eff6Zx48Z88MEH/Prrrzg6OpKTk4ORkRHR0dGcPn2aoUOH0q1bN+Lj49m1axfr16/HwMCA8PBwfH19WbFiBQB169ZV0z4rikJqaipeXl5s3bqVunXrsnbtWgACAgJYvXo1M2bMAODy5ctERERw79497OzsOHjwIG3btsXHx4eIiAjat2/P9u3bWbBgAQsXLsTPz48VK1bQvHlzkpKSGDRoEKamphgbG1fOhRVVhnQohBCihGxsbABo2bIl7dq1o2nTpgA0b95ca4pi6NChALz66qu0a9eOQ4cOkZCQwIABAzAwMABg2LBhLF26lMzMTAC6detW6DltbW1p0aIFERERXLp0iYMHD9KlSxd1vYWFBTo6OhgaGtKqVStu377N4cOHeeWVV2jfvj0A1tbWWFtbs3PnTv7991/GjRun7q/RaDhz5ox0KESZSYdCCCFKqEaNGoX++1G6urrqv3Nzc9HV1S1QVjo3N5fs7Gz19/yOxqPWrFnD2rVrGTJkCI6OjtSrV4/ExER1fe3atdV/azQatUrko2Xrz5w5Q05ODiYmJqxbt05dl5SURIMGDYoLW4gSkYcyhRCinG3YsAGAP//8kwsXLtC9e3d69uxJdHQ09+7dAyAiIoLu3btTs2bNAvvr6uqqnY3du3fz/vvvM3DgQFq3bk1cXJxaYrsonTp14u+//+bcuXMA/Prrr/j4+NC5c2cuXbrEoUOHADh16hQ2NjYkJyeXW+yi+pIRCiGEKGeHDx9m7dq15ObmEhoaipGRER988AHXrl1j4MCB5Obm0qpVKxYsWFDo/m+99RYTJkygRo0ajBo1ioCAAKKjo9HV1eW1117j7NmzxZ6/UaNGLFiwgGnTppGTk4OhoSGhoaE0aNCAsLAw5s2bx4MHD1AUhXnz5vHSSy9VxGUQ1YxUGxVCiHJkamrKvn37ZBpBVDsy5SGEEEKIMpMRCiGEEEKUmYxQCCGEEKLMpEMhhBBCiDKTDoUQQgghykw6FEIIIYQoM+lQCCGEEKLMpEMhhBBCiDL7f/NTQQWicBsGAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_feature_importance(feature_importance);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In nearly every real-world dataset, this is what the feature importance looks like: a handful of columns are very important, while most are not. The powerful aspect of this approach is that is _focuses our attention_ on which features we should investigate further and which ones we can safely ignore." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Warning: The feature importance analysis above can be biased and has a tendency to inflate the importance of continuous features or categorical features with high cardinality (i.e. many unique categories). See the Beware Default Random Forest Importances article in the references for more information." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Drop uninformative features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the feature importance plot above, we can see there are only a handful of informative features - let's use this insight to make a simpler model by dropping uninformative columns from our data:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "9" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "feature_importance_threshold = 0.03\n", "cols_to_keep = feature_importance[\n", " feature_importance['Importance'] > feature_importance_threshold\n", "]['Column']\n", "\n", "len(cols_to_keep)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# create a copy of the data with selected columns and create new train / test set\n", "X_keep = X.copy()[cols_to_keep]\n", "X_train, X_valid = train_test_split(X_keep, test_size=0.2, random_state=42)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
median_incomeocean_proximity_INLANDpopulation_per_householdpostal_codelongitudebedrooms_per_roomlatituderooms_per_householdhousing_median_age
08.325202.55555694705-122.230.14659137.886.98412741.0
18.301402.10984294611-122.220.15579737.866.23813721.0
27.257402.80226094618-122.240.12951637.858.28813652.0
35.643102.54794594618-122.250.18445837.855.81735252.0
43.846202.18146794618-122.250.17209637.856.28185352.0
\n", "
" ], "text/plain": [ " median_income ocean_proximity_INLAND population_per_household \\\n", "0 8.3252 0 2.555556 \n", "1 8.3014 0 2.109842 \n", "2 7.2574 0 2.802260 \n", "3 5.6431 0 2.547945 \n", "4 3.8462 0 2.181467 \n", "\n", " postal_code longitude bedrooms_per_room latitude rooms_per_household \\\n", "0 94705 -122.23 0.146591 37.88 6.984127 \n", "1 94611 -122.22 0.155797 37.86 6.238137 \n", "2 94618 -122.24 0.129516 37.85 8.288136 \n", "3 94618 -122.25 0.184458 37.85 5.817352 \n", "4 94618 -122.25 0.172096 37.85 6.281853 \n", "\n", " housing_median_age \n", "0 41.0 \n", "1 21.0 \n", "2 52.0 \n", "3 52.0 \n", "4 52.0 " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_keep.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As a sanity check, let's ensure the model's prediction have not gotten worse with the reduced data (recall we had $R^2 = 0.91$ on the validation set):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RMSE on train: 16622.874\n", "R^2 on train: 0.97\n", "RMSE on valid: 43873.09\n", "R^2 on valid: 0.8\n", "OOB R^2: 0.789\n" ] } ], "source": [ "model = RandomForestRegressor(\n", " n_estimators=40, min_samples_leaf=1, n_jobs=-1, oob_score=True, random_state=42\n", ")\n", "model.fit(X_train, y_train)\n", "print_rf_scores(model)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAEJCAYAAABmLwVwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzde1zO9//H8cfVkYaKZDO2L0Y2hvmOctxyTiUVY6Ywx5nTTFPEctooZtLM+n7t5MtipIMsGzlsDjHfYTbCzyE5dFCS6HB1fX5/uLm+msolXV2Xa6/77eZ2q8/h/Xl+3un26v3+fK7PR6UoioIQQgghTIKZoQMIIYQQoupIYRdCCCFMiBR2IYQQwoRIYRdCCCFMiBR2IYQQwoRIYRdCCCFMiBR2IYQQwoRYGDqAEAA5OfloNMbzSIV69Wpx/fotQ8coxRgzgXHmkky6M8ZckunhzMxU2Ns/VeY6lTygRgghhKh+BYXF5N0sqNS+ZmYq6tWrVeY6GbELozDl4xiycvINHUMIIarN+tC3yKNyhb0ico1dCCGEMCFS2IUQQggTIoVdCCGEMCFS2IUQQggTIoVdCCGEMCFS2KtZcnIyfn5+AMyePZvff/+9StsfO3Ys6enpVdqmEEKIJ4d83M2AFi1aVOVt/utf/6ryNoUQQjw5pLDrKDk5mdWrV6MoCqmpqfTt25fatWuzY8cOACIjI/nzzz8JDw9HrVbTqFEjFixYgL29Pb/88gsff/wx1tbWNGnSRNumn58fkyZN4p///CchISGcOXOGrKwsmjRpQkREBFlZWUyaNInmzZtz8uRJ6tWrx4oVK7Czsys3Z48ePfj22285dOgQP//8M7m5uVy6dIkuXboQEhKCoigsXbqUHTt2YG5uzpAhQxgxYgTnz59n7ty53LhxAxsbG2bPnk2bNm0IDAykZs2aHDlyhLy8PGbNmkVsbCynTp2iV69eBAYGUlJSQmhoKIcOHaKkpAQfHx9Gjhyp7x+JEEKIMshU/CM4duwYH3/8MQkJCURFRVG3bl2io6NxcnIiKiqKZcuWsWbNGmJiYujatStLly6lqKiIwMBAwsPDiY6OpkaNGg+0+9tvv2FpacmGDRv46aefKCwsZM+ePQCcOnWKUaNGsXXrVurUqUN8fLzOeX/77TfCw8OJi4tj165dpKSkkJiYyH//+1/i4+P5/vvviY6OJjMzk4CAAPz8/IiPjycoKIipU6dSVFQEQEZGBnFxcUyZMoWgoCDmzZtHTEwMGzduJC8vj40bNwKwZcsWNm3axM6dO/n111+roMeFEEI8KhmxP4IWLVrwzDPPAGBvb0+nTp0AaNiwIUlJSVy9ehV/f38ANBoNtra2pKSk4OjoSLNmzQDw9vZmxYoVpdrt0KEDdnZ2rFu3jnPnznHhwgVu374NQL169XjppZcAaN68Obm5uTrnfeWVV6hV6+4jBxs3bkxubi6HDx/Gzc0NKysrrKysiI2NJT8/n9TUVPr06QNAu3btsLW15dy5cwB0795de57NmzenXr16ANjZ2ZGbm8uBAwc4efIkBw8eBOD27dukpKTw6quvPkr3CiGEqAJS2B+BpaVlqe/Nzc21X2s0Gtq3b8/q1asBKCwsJD8/nytXrqDRaMrc556dO3cSHh6Ov78/Pj4+5OTkcO8R/tbW1trtVCoVj/Jo/7L2tbAo/SNPS0vD1tb2gXYVRaGkpOSB8/7r/gAlJSUEBARo/zDIzs7GxsZG55xCCCGqjkzFV5E2bdpw9OhRzp8/D8CqVasIDQ3FycmJ69evc+rUKQASEhIe2PfAgQO4ubnh6+uLg4MDhw8f1hbVqtahQwd++ukniouLuXPnDmPGjCErK4vGjRvz448/AnD06FGysrJo3ry5Tm26uLiwceNGiouLyc/PZ9iwYRw7dkwv+YUQQlRMRuxVpH79+nz00UdMmzYNjUZDgwYNCAsLw9LSkk8++YSAgAAsLCy00+r3Gzx4MDNmzCAxMRErKyvatWtHWlqaXnL27t2bEydO4OPjg0ajwd/fnyZNmhAWFkZISAgrV67E0tKSlStXYmVlpVObQ4cO5eLFi3h7e6NWq/Hx8cHZ2Vkv+YUQQlRMXtsqjIK83U0I8XezPvQtMjPzKrWvvLbVxPj5+XHz5s0Hlg8dOpQ333zTAImEEEIYCynsT6C1a9caOoIQQggjJTfPCSGEECZECrsQQghhQuTmOSGEEMIACgqLybtZUKl95eY5YfSuX7+FRmM8f2PWr1+70ner6osxZgLjzCWZdGeMuSTT45GpeCGEEMKESGEXQgghTIhMxQujUN61IkOqX7+2oSM8wBgzQdm5Huf6oRCi8qSwC6MgT54zPetD3yIPKexCVDeZihdCCCFMiBR2IYQQwoRIYRdCCCFMiBR2IYQQwoRIYa8iK1asYOfOnTpvP3bsWNLT07l06RKzZs2q1DGjo6MJDAwEYOXKlfTp04eCgv/drJScnIyfnx8AgYGBREdHl9vWkiVLcHFxoaioSLssLS0NJycn9u3bV2rbHj16kJaWRlpaGq1bt8bLywsvLy/69u3LlClTyMrKqtT5CCGEeHxS2KvI1KlT6dmzp87b/+tf/6JBgwZcuXKFS5cuVUmGK1eu8Mknnzzyfmq1mh9++IFXXnmFxMTEUussLS2ZM2cOt27dKnNfR0dHYmNjiY2NJTExkeeff54pU6ZUKr8QQojHV22FffXq1fTv3x9PT08WL15MSUkJX3/9NX379qV///6EhYUBkJWVxcSJE/Hx8cHX15f9+/cDkJ6ezujRo3njjTdwdXVl6dKlwN1R63vvvcfbb79N7969CQkJqTBHWloanp6eTJw4EXd3d8aOHcuNGzcAcHFxYfTo0Xh5eVFcXFxm5p07d9KnTx8KCwu5ePEi3bt359q1a9oRcVpaGl5eXkyaNIk+ffowffp0oqKiGDJkCP369eP//u//gP+NehcuXMiJEyeYN28eAQEBbNiwQZvVz8+PY8eO6dzHQ4cOZdu2bfz666867wOwZ88eGjduzMCBA0sdH+4W7s6dO7NkyZKHtqNSqZg8eTJnzpzh1KlTj5RBCCFE1aiWwr5nzx6SkpKIjo5my5YtXLx4kc8//5z169ezadMm4uLi+OOPPzhx4gSLFi3C19eX6OhoPv/8c+bOncutW7fYunUrHh4ebNy4kbi4ONavX092djYAv/32G+Hh4cTFxbFr1y5SUlIqzHP69GlGjBhBQkICzZo1IyIiAoCcnBzGjRtHbGws+/fvfyBzVFQUPXv25JVXXuHzzz8nKCiImTNn8vTTT5dqPyUlhYkTJ5KYmMjvv//O5cuX2bBhAx4eHg8UzuDgYFq3bs2HH36Ir68vcXFxAFy+fJns7Gzatm2rcz/b2toSEhLC7NmzS03JP0x0dDT9+vXjtdde4+TJk5w9e7bU+sDAQH755ZcHpuTLYmVlxfPPP8+5c+d0Pr4QQoiqUy2F/eDBg7i7u1OjRg0sLCzw9fUlIiICV1dXateujYWFBV9//TWtW7dm//79hIeH4+XlxdixY1Gr1Vy6dInRo0fzzDPPsGbNGhYtWkRxcTF37twB4JVXXqFWrVrUrFmTxo0bk5ubW2Gef/zjHzg7OwMwcOBADh48qF13r5CWlfnAgQMAzJ49m02bNuHg4IC7u/sD7Ts4OPDSSy9hZmbG008/TadOnQBo2LAhN2/eLDeXs7MzGRkZpKWlERMTg5eX1yP08l29evWidevWOk/JZ2dn88svv+Dm5kaNGjVwdXUlKiqq1Da1atViwYIFFU7J30+lUlGjRo1Hzi6EEOLxVcuT5zQazQPL/vq22PT0dGrWrIlGo+Gbb77Bzs5Ou9zBwYHFixdz6dIlPDw86NWrF/v379e2YW1trW1HpVI90PZfWVj877QVRcHc3Fz7/b2CVFZmtVoN3L1cYG5uzvnz5ykqKsLKyqrUdn/9/v72K6JSqRg4cCAJCQkkJiby73//W6f9/mrOnDl4eHho+7AicXFxKIrCoEGDACgoKKC4uJgZM2aU2q5r1646TckXFRVx/vx5XnjhhUplF0II8XiqZcTu4uJCQkICBQUFqNVqNm/ezMyZM9m7dy/5+fmo1Wref/99Tpw4gYuLC+vXrwfg7NmzDBgwgDt37rBv3z5Gjx6Nm5sbV69eJT09vcziq4vz589z8uRJADZv3kz37t11yuzi4kJJSQlBQUHMnj2bDh068Omnn1a+Y7hb9O/9wQDg4+NDVFQUTz/9NA0aNKhUm3Z2doSEhLBq1aqHbrt582YWL15MUlISSUlJ/PLLL9ja2rJt27YHtr03JZ+RkVFmWxqNhpUrV9K2bVuee+65SmUXQgjxeKplxO7q6srJkyfx9fVFrVbTrVs3/P39sba2ZujQoWg0Gnr37k3nzp1p1qwZc+fOxdPTE4DQ0FBq1arF+PHj+eCDD6hTpw716tWjdevWpKWlVSqPra0t4eHhpKam4uTkxMKFC3XKPHz4cL788kvq1atHnz596Ny5Mx4eHvTp06fSfdOsWTPy8vIICAggLCyMZ555hmeeeQZvb+9Ktwl3p+T79u1bqgh/+OGHLFiwQPv9F198QU5ODr1799YuMzMzY8SIEURFRdGxY8dSbd6bkh89erR2WUZGhvaSgUaj4cUXX2TZsmWPlV0IIUTlqZSHzVubmLS0NPz9/UlKSjJ0lAcoikJGRgZ+fn5s3br1gSl9UyYvgTE960PfIjMzzyDHrl+/tsGOXR5jzATGmUsyPZyZmarct2Ka5NvdUlNTmTx5cpnryhqdG4vt27cTEhJCSEiItqgvWbJE+5G/+7Vu3ZpFixZVd0QhhBBGziQL+3PPPUdsbGy5641xtA7Qr18/+vXrV2rZzJkzDZRGCCHEk0iePCeEEEKYECnsQgghhAmRwi6EEEKYEJO8xi6ePOFBAw0dQVSxgsJiQ0cQ4m9JCrswCtev30KjMZ5PXhrbR1vAODOB8eYS4u9KpuKFEEIIEyKFXQghhDAhUtiFEEIIEyLX2IVRKO/RiIZUv35tQ0d4QGUzFRQWk3ezoIrTCCGMkRR2YRTkWfH6tT70LfKQwi7E34FMxQshhBAmRAq7EEIIYUKksAshhBAmRAq7EEIIYUKe2MLu5+dHcnLyQ7e5x8vLS9+RqoyTk1O1HzMtLY0ePXo80j49evQgLS3tgeW6/GyEEELoxxNb2HVx6NAh7dcVvZ9dCCGEMBV6K+zJyckMHz6ckSNH0rdvXwICAigqKmLz5s14eHjg6elJYGAg+fl3P+Lk4uJCUFAQnp6eDB06VDsSvH9UmJycXGoUDqBWqwkODmbIkCH07NmTMWPGUFBQwMKFCwEYPHgw8L9R8J07d3j//fe1GWJiYgCIjo7mvffe4+2336Z3796EhIRUeH5paWl4enoyceJE3N3dGTt2LDdu3ABg7969DBo0iIEDBzJp0iRycnK05zJt2jT69u3L9evXK2x/7ty5DBgwgAEDBnDx4kUAjh49yuDBgxkwYAAjRozQLr9/hHz/yDs+Ph4vLy98fHyYMmUKhYWFAERGRuLt7c2AAQMIDQ1FUe4+o72goID33nsPDw8Phg0bps29a9cuvLy8tOeblZVVKmtRUREBAQG4ubkxZswY7X5CCCGqn15H7MePH2fu3LkkJiZSWFhIZGQkq1evZu3atcTHx1OzZk0iIiIAyMnJoWPHjsTHx+Pu7q4tzA/z22+/YWlpyYYNG/jpp58oLCxkz549BAcHA/D999+X2n7lypXY29uzdetWvvnmG1auXMmpU6e0bYWHhxMXF8euXbtISUmp8NinT59mxIgRJCQk0KxZMyIiIsjOzmbZsmWsWbOGmJgYunbtytKlS7X7dO/ene3bt1OvXr0K2+7cuTNxcXF06dKFqKgoioqKmD59OnPmzCEuLo6hQ4cyffr0Ctv49NNP+fLLL4mOjqZJkyacO3eOvXv3cuLECTZt2kRMTAzp6enExcUBkJ2dzahRo9i6dSsODg5s27aN69evM3fuXD777DPi4+Np37498+fPL3WctWvXAvDDDz8QHBxMampqhbmEEELoj14Le4cOHWjatCkqlQovLy9WrVqFq6sr9vb2AAwZMoSDBw8CYG1tzcCBd1/d6e3trfM12g4dOjBs2DDWrVvHokWLuHDhArdv3y53+4MHDzJo0CAA6tatS8+ePbVT9q+88gq1atWiZs2aNG7cmNzc3AqP/Y9//ANnZ2cABg4cyMGDBzl27BhXr17F398fLy8v1q1bpx1ZA7Rt21an8+rVqxcAL7zwAjdu3ODChQvUqVOHNm3aAODm5kZqaip5eeW/VcvV1ZU333yTJUuW4OrqyosvvsiBAwc4fvw4Pj4+eHt7c+LECc6ePQuAo6Ojtv0XXniBnJwcjh8/Tps2bWjUqBFQ+md2z6FDh3Bzc9P2ySuvvKLTOQohhKh6en3ynLm5ufZrRVHQaDSl1iuKglqtBsDMzAyVSgWARqN5YF9Au+39du7cSXh4OP7+/vj4+JCTk6Pdvix/XacoCiUlJcDdPy7uUalUFbYDYGHxv+5TFAVzc3NKSkpo3749q1evBqCwsFB7ueGvx9Cl7Xs5/tp3f81eVh8FBwdz6tQp9uzZQ0BAAJMmTaKkpIQRI0YwatQoAG7evIm5uTk5OTmlzqe8497/M7t/2/u3u78dIYQQ1UuvI/YjR46Qnp6ORqMhJiaGoKAgkpKStNeiN27cqB3x3rlzh6SkJODu9e7u3bsDYG9vrx1R7ty584FjHDhwADc3N3x9fXFwcODw4cPaYmdubv5AEXJxcWHTpk3A3annnTt30rFjx0qd3/nz5zl58iQAmzdvpnv37rRt25ajR49y/vx5AFatWkVoaGil2r9f06ZNuXHjBsePHwdg27ZtNGzYEDs7u1J9tGPHDuBuge/Tpw/29vaMHz8eLy8vTp48iYuLC7GxseTn56NWq3n33XfZvn17ucdt27Ytx44d097nsGHDBu3P7J5OnTqxdetWNBoNly9f5r///e9jn68QQojK0evQytHRkQ8++ID09HS6dOnC8OHDsbGxwc/Pj+LiYlq1asW8efO02ycmJrJ8+XIcHR1ZsmQJAFOmTGHBggVERETQtWvXB44xePBgZsyYQWJiIlZWVrRr105bhHr27ImXlxfR0dHa7d99911CQkLw9PSkpKSECRMm0KpVq4deTy+Lra0t4eHhpKam4uTkxMKFC7GxseGjjz5i2rRpaDQaGjRoQFhY2CO3/VdWVlYsX76cBQsWcOfOHWxtbVm+fDkAY8aMITAwkM2bN9OzZ0/g7qh5ypQpjBo1iho1alCnTh2WLFlCgwYNOHXqFG+88QYlJSV069YNb29vLl++XOZxHRwcmD9/PpMmTaK4uJiGDRuyaNGiUtsMGzaMM2fO4ObmxrPPPkuLFi0e+3yFEEJUjkp52HxzJSUnJxMREaG9sephnJycKlVcDSUtLQ1/f3/tLIN4PPISGP1aH/oWmZnl34/xOOrXr623titLMunOGHNJpoczM1OV+1ZMuRhagdTUVCZPnlzmOl3v2i9LQUEBQ4YMKXPdlClTtKNuIYQQ4lHprbA7Ozs/cC22IsY4Wn/uuecqfLBNZUfrNWrUkAfmCCGE0AuTfvKcEEII8XcjhV0IIYQwIXKNXRiF8KCBho5g0goKiw0dQQhRTaSwC6Nw/fotNBq9fECjUoztDlgwzkxCCOMjU/FCCCGECZHCLoQQQpgQKexCCCGECZFr7MIolPcEJUOqX792pfYrKCwm72ZBFacRQgjdSGEXRsGUHim7PvQt8pDCLoQwDJmKF0IIIUyIFHYhhBDChEhhF0IIIUyIFHYhhBDChEhhf4IkJyfj5+dXZe3t3LmTFStWABAeHs6vv/76SPs7OTlVWRYhhBBVQwr731jPnj2ZOnUqAIcPH6akpMTAiYQQQjwu+bjbE+j8+fPMnTuXGzduYGNjw+zZs2nTpg2BgYHUqlWLP/74g/T0dN599118fX3Jy8vjgw8+IDU1lcaNG3Pt2jUiIiI4dOgQhw4dwsXFhRMnThAcHExERAQLFy5k0qRJODs7k5aWhr+/P0lJSaSlpREQEMDt27dp27atNk9+fj7z58/nzJkzlJSUMHbsWDw8PAzYQ0II8fclI/YnUEBAAH5+fsTHxxMUFMTUqVMpKioC4Nq1a6xfv57PP/+c0NBQAD777DOaNGlCQkIC7777LikpKaXaGzhwIK1bt2bhwoUVTq8vWLAAHx8fYmNjad++vXb5559/TqtWrYiOjmbdunWsXr2aS5cu6eHMhRBCPIwU9idMfn4+qamp9OnTB4B27dpha2vLuXPnAOjSpQsqlYoWLVpw48YNAPbt24eXlxcAL7/8cqWvjR86dAg3NzcABgwYgKWlJQD79+8nKioKLy8v3nrrLW7fvs2ZM2ce6zyFEEJUjkzFP2EURUFRlAeW3bs+bm1tDYBKpdKuNzc3f2AfXY4DoFary1yuUqm0x9BoNISFhdGqVSsAsrKysLW1faTjCSGEqBoyYn/C1KpVi8aNG/Pjjz8CcPToUbKysmjevHm5+3Tu3Jn4+HgAUlJSOHPmTKnCD3eL/70/Duzt7Tl79iwAO3bsKNVOXFwcAD/++KN2+t/FxYXvvvsOgIyMDAYMGMDVq1er4nSFEEI8IinsT6CwsDDWrl2Lp6cn8+fPZ+XKlVhZWZW7/cSJE0lNTcXT05Pw8HAcHByoUaNGqW26devGhx9+yH//+1/GjBnD+vXr8fb2pqDgf888nzt3Ltu3b8fT05M9e/bw1FNPATBp0iQKCgrw8PBgxIgRBAQE8Nxzz+nn5IUQQlRIpTzqHK144sTGxtKoUSP++c9/cuXKFYYPH86OHTswMzOev+tM7SUwmZl5Vd5u/fq19dLu4zLGXJJJd8aYSzI9nJmZqty3Yso19r+Bpk2b8uGHH6LRaDAzM2P+/PlGVdSFEEJUHZ0K+507d0hMTCQ3N7fUTVijRo3SWzBRdV5++WWio6MNHUMIIUQ10Kmwv/fee2RkZNCiRYsHbroSQgghhPHQqbCfO3eObdu2YWEhM/dCCCGEMdOpUj/99NP6ziH+5sKDBho6QpUpKCw2dAQhxN+YToW9RYsW+Pv7061bt1Ifk5Jr7KKqXL9+C43GeD6gYWx3wAohhK50Kuz5+fk8//zzpKam6juPEEIIIR6DToX9448/1ncOIYQQQlQBnQp7cnIykZGR5Obmllq+adMmvYQSQgghROXoVNiDg4Px8/OTx4QKvSnvCUqGVL9+7QrXFxQWk3ezoMJthBCiuulU2OvVq4e/v7++s4i/sSfxkbLrQ98iDynsQgjjolNh79GjB+vWraNbt26lPsvesGFDvQUTQgghxKPTqbBnZ2fzySefULNmTe0ylUrFf//7X70FE0IIIcSj06mwJyYm8ssvv+Dg4KDvPEIIIYR4DDq94qtevXrUrVtX31mEEEII8Zh0GrG//PLLDBs2DFdXV6ysrLTL5clzhpOUlMTFixcr/BlER0dz6NAhFi9erJcMK1euBGDy5Ml6aV8IIcSj06mwFxYW0qRJEy5cuKDnOEJXf/zxh6EjCCGEMELy5DkDSE5OZuXKlVhYWHD16lXatGnDokWLiI+P56uvvkKlUtGqVSvmzJmDlZUVs2bN4syZMwAMGzaM9u3bExUVBdz9ZELXrl2ZNWsWeXl5ZGZm4u7uzowZM3TKcvLkSebOnUtBQQG2trYsXbqUp59+mtWrVxMXF4e5uTldunQhICAAc3Nz/v3vf7Nx40bs7e2pU6cObdq0AWDv3r2Eh4ejVqtp1KgRCxYswN7eXj8dKIQQolw6FfYJEyaUuXz16tVVGubv5Pjx48TExNCkSROmTp1KZGQksbGx2qI5b948IiIicHV1JTc3l5iYGHJycliyZAlvvPEGQ4cOBcDX15c1a9bg4eGBt7c3eXl5vPbaa7z99ts65ZgxYwYzZszA1dWV9evX88033+Di4kJSUhLR0dFYWFgwefJkoqKiaNOmDZs3b2bLli2oVCqGDBlCmzZtyM7OZtmyZXz77bfY2toSFRXF0qVLWbRokT67UAghRBl0Kux9+/bVfl1cXExSUhJOTk56C/V30KFDB5o2bQqAl5cXkydPZvjw4dpR7pAhQwgKCmLcuHGcP3+e0aNH07179zJH4qNHj+bgwYOsWbOGM2fOUFxczJ07dx6aITs7m8zMTFxdXYG7swEAS5Yswd3dXfsmP19fX2JiYigoKOC1117jqaeeAqBfv35oNBqOHTvG1atXtQ8x0mg02NraPmYPCSGEqAydCru3t3ep7318fBg+fLheAv1dmJuba79WFAWNRlNqvaIoqNVq7O3tSUhIYN++fezZswdvb28SEhJKbbt48WIuXbqEh4cHvXr1Yv/+/SjKw1+BamlpWer7wsJCMjIyHsgCoFarUalUpdZZWFhQVFRESUkJ7du3187gFBYWkp//ZD1FTgghTIVOH3f7K41GQ0ZGRlVn+Vs5cuQI6enpaDQaYmJiCAoKIikpiRs3bgCwceNGnJ2d2blzJzNmzOD1118nODgYGxsbrl69irm5OWq1GoB9+/YxevRo3NzcuHr1qrbdh6lduzZPP/00+/btAyA2NpYVK1bg4uJCQkICBQUFqNVqNm/ejIuLC506dWL37t3k5eVRWFjITz/9BEDbtm05evQo58+fB2DVqlWEhobqo9uEEEI8RKWusZ8+fZqOHTvqJdDfhaOjIx988AHp6el06dKF4cOHY2Njg5+fH8XFxbRq1Yp58+ZhbW3N9u3bcXd3x9ramj59+uDk5MTNmzeZOXMmDg4OjB8/ng8++IA6depQr149WrduTVpamk45wsLCCAkJITQ0FHt7e0JDQ3F0dOTkyZP4+vqiVqvp1q0bw4cPx8LCghEjRjBo0CDq1KmjfaRw/fr1+eijj5g2bRoajYYGDRoQFhamz+4TQghRDpWiw5ztli1b/reDSoW9vT1du3YtNZ0sdJecnExERARr1641dBSj8aS+BCYzM6/ajle/fu1qPZ6ujDGXZNKdMeaSTA9nZqYq962YFY7Y700L37u56n55eXnY2dlVQTyhb++//z5nz559YKBZAIwAACAASURBVHmPHj2YOnWqARIJIYTQlwoLu4uLCyqVCuCBm7FUKhUnT57UXzIT5uzsjLOzc7Udb9myZdV2LCGEEIZVYWE/depUdeUQQgghRBXQ6eY5jUbDmjVr2Lt3L2q1mi5dujBhwoRS72YXQgghhOHpVJmXLVvGqVOnGDFiBBqNhg0bNrBkyRJmz56t73zibyI8aKChIzyygsJiQ0cQQogH6FTYf/75ZzZv3qx9oMnrr7/OgAED9BpM/L1cv34LjebhD9WpLsZ2B6wQQuhKpwfUKIpS6illVlZWDzy1TAghhBCGp1Nhb9myJR999BGpqamkpqby0Ucf0aJFC31nE0IIIcQj0qmwf/jhh9y8eZOhQ4fyxhtvkJOTw5w5c/SdTQghhBCPqMJr7EVFRcyZM4fevXuzePFiAMaNG4e5uTm1apX9xBshKqO8JygZUv36tUt9X1BYTN7NAgOlEUII3VRY2MPDw7l16xavvPKKdtmCBQuYN28eK1eu5L333tN7QPH38CQ8UnZ96FvkIYVdCGHcKpyK3717N8uWLaNevXraZQ0aNCA0NJQdO3boPZwQQgghHk2Fhd3S0pIaNWo8sLxWrVpYWVnpLZQQQgghKqfCwm5mZsatW7ceWH7r1i3tu8CFEEIIYTwqLOweHh4EBwdz+/Zt7bLbt28THBxMnz599B5OCCGEEI+mwsI+YsQIateuTZcuXXjjjTcYNGgQXbp0oU6dOrz77rvVlVE8guTkZPz8/Mpdn5SUxFdffQXAd999x3fffQdAUFAQly9f1vk4aWlp9OjR4/HCCiGEqHIV3hVvZmbGggULGD9+PH/++SdmZma8/PLLNGjQoLryiSr2xx9/aL9+8803tV8nJyfLH2tCCGECdHpWfKNGjWjUqJG+s4gqdOjQIZYvX05BQQG5ubkEBATQvHlzoqKiAGjYsCFXrlwBwNramoyMDMaNG8e6devw9fXl22+/pVGjRiQnJxMREcHatWv5888/tS/+admypfZYWVlZzJ07l2vXrqFSqXj//ffp3Llz9Z+0EEII3Z48J548//nPf1i4cCFbtmxh0aJFrFq1ihdeeIGhQ4cydOhQfH19tduOGzcOR0dHIiMjsbe3L7fNmTNnEhAQwJYtW0r9obdo0SJ8fX2Jjo7m888/Z+7cuWXedCmEEEL/5IXqJiosLIxdu3aRmJjIsWPHyM9/vIe/ZGdnk5GRoR2J+/j4sHnzZgD279/PuXPnCA8PB0CtVnPp0iVefPHFxzsJIYQQj0wKu4kaNmwYzs7OODs706lTJ2bMmPFI+yvK3Veo3vtYo0ql0i4DMDc3136t0Wj45ptvsLOzAyA9PR0HB4fHPQUhhBCVIFPxJujGjRtcuHCBqVOn8tprr7Fv3z5KSkqAuwW5rGcQmJuba7ext7fn7NmzAOzcuVO7rGHDhuzevRuArVu3avd1cXFh/fr1AJw9e5YBAwZw584dvZ2fEEKI8klhN0F2dnYMHjwYd3d3Bg4cyPXr1ykoKOD27dt06NCB+Ph41q5dW2qf119/nXHjxnHp0iWmTJmivW5eu/b/XoQSFhZGREQEAwcOJDU1Vbs8ODiYY8eO4enpyXvvvUdoaKi8JEgIIQxEpdw/vyqEgTwpL4HJzMwz2PHr169t0OOXxxhzSSbdGWMuyfRwZmaqct+KKSN2IYQQwoRIYRdCCCFMiBR2IYQQwoRIYRdCCCFMiHyOXRiF8KCBho7wUAWFxYaOIIQQDyWFXRiF69dvodEYzwc0jO0OWCGE0JVMxQshhBAmRAq7EEIIYUKksAshhBAmRK6xC6NQ3hOU9KmgsJi8mwXVflwhhNAnKezCKBjikbLrQ98iDynsQgjTIlPxQgghhAmRwi6EEEKYECnsQgghhAmRwi6EEEKYkCe6sCcnJ+Pn56eXtr/77ju+++47vbRdFQIDA4mOjiY9PZ2xY8caOo4QQggjIXfFl+PNN980dASdNGjQgH/961+GjiGEEMJIPPGFPTs7m7Fjx5KamkqTJk0IDw8nPj6er776CpVKRatWrZgzZw5PPfUUTk5OpKSkABAdHc2hQ4dYvHgxS5YsYd++fZibm9OzZ08mTZrEypUrAZg8eTJdu3alb9++HDlyBHNzcz799FMaN25McnIyCxcuxNzcnHbt2vF///d/rF27ttysgYGB1KxZkyNHjpCXl8esWbOIjY3l1KlT9OrVi8DAQEpKSggNDeXQoUOUlJTg4+PDyJEjURSFxYsXs3v3bhwdHSkpKaFjx46kpaXh7+9PUlISp0+fZsGCBdy+fZvs7GxGjRqFv78/K1euJD09nYsXL3L58mUGDx7MO++8U25OtVpNSEgIZ86cISsriyZNmhAREUGNGjX49ttv+c9//kPt2rVp2rQpzz33HJMnT2bv3r2Eh4ejVqtp1KgRCxYswN7evmp/2EIIIR7qiZ6KB7hy5Qpz587lhx9+ICsri++++47Vq1ezdu1a4uPjqVmzJhEREeXuf/nyZfbu3UtcXBxRUVFcuHCBwsLCUttkZmbSqVMnYmJi6NChA+vWraO4uJgPPviAsLAwYmJisLDQ7W+kjIwM4uLimDJlCkFBQcybN4+YmBg2btxIXl4eGzduBGDLli1s2rSJnTt38uuvv7J9+3b+/PNPtm7dyooVK0hNTX2g7e+//56JEyeyefNmvv32W5YvX65dl5KSwpo1a/j++++JjIzk5s2b5Wb87bffsLS0ZMOGDfz0008UFhayZ88eTp06xbp164iOjmb9+vVcvHgRuPvH1bJly1izZg0xMTF07dqVpUuX6tQfQgghqtYTP2Jv2bIljRs3BqBZs2bk5eXh6uqqHS0OGTKEoKCgcvdv0KAB1tbWDB06FFdXV6ZNm4a1tfUD23Xr1g2A5s2b8+uvv3L69Gnq1atHy5YtARg0aBCLFi16aN7u3bsD0LBhQ5o3b069evUAsLOzIzc3lwMHDnDy5EkOHjwIwO3bt0lJSeH//u//6NOnD5aWltStW1fbzv0CAwP5+eef+eKLL0hJSeH27dvadc7OzlhZWVGvXj3s7OzIy8ujTp06ZWbs0KEDdnZ2rFu3jnPnznHhwgVu377NgQMHcHV1pVatu0+Jc3d35+bNmxw7doyrV6/i7+8PgEajwdbW9qF9IYQQouo98YX9/pGySqWiTp06pUajiqKgVqtLfa9SqbTLLCws+P777zl06BB79+5l6NChZU6n3yv2KpUKRVEwNzdHo9E8cl5LS8sys99TUlJCQEAAffr0Ae6Ohm1sbAgLCyt1vLL2nTZtGnXq1MHV1ZX+/fuTkJDwQP77z6E8O3fuJDw8HH9/f3x8fMjJyUFRFMzMzMo855KSEtq3b8/q1asBKCwsJD+/ep8iJ4QQ4q4nfiq+LElJSdy4cQOAjRs34uzsDIC9vT1nzpxBURSSkpIA+PPPPxk+fDgdOnRg5syZNGvWjPPnzz/0GE2bNuXmzZvaa/bx8fFVkt3FxYWNGzdSXFxMfn4+w4YN49ixY3Tq1InExESKiorIzc3l559/fmDfffv2MWXKFHr16sXhw4eBu0X3UR04cAA3Nzd8fX1xcHDg8OHDlJSU0KlTJ/bs2cOtW7coKirixx9/RKVS0bZtW44ePartt1WrVhEaGvp4HSGEEKJSnvgR+1/VqlWL8ePH4+fnR3FxMa1atWLevHkAvP/++0yYMAEHBwf++c9/kpOTw0svvUS7du3w8PCgZs2avPjii3Tv3p0//vijwuNYWVkRGhrKzJkzMTMzo0mTJtSoUeOx8w8dOpSLFy/i7e2NWq3Gx8dH+4fJ77//joeHBw4ODjRr1uyBfSdPnsywYcOoU6cOTZo04dlnnyUtLe2RMwwePJgZM2aQmJiIlZUV7dq1Iy0tjcGDB+Pv78+QIUOwsbHB3t4ea2tr6tevz0cffcS0adPQaDQ0aNCAsLCwx+4LIYQQj06lVDQnK8ql0WhYunQpkyZNwsbGhq+++or09HQCAwMNHU1vzp8/z549exg5ciQA77zzDoMHD6ZHjx6P3bahXgKTmZlX5rr69WuXu85QjDETGGcuyaQ7Y8wlmR7OzExV7lsxTW7EXl3MzMyws7Nj0KBBWFpa8uyzz7Jo0SKWLFnC/v37H9i+devWOt1cV11+/fVXFixYUOa6yMhIGjRo8MDyZ599VjtroFKp6Nq1K66urvqOKoQQ4hHIiF0YBRmxP5wxZgLjzCWZdGeMuSTTw1U0YjfJm+eEEEKIvysp7EIIIYQJkcIuhBBCmBC5eU4YhfCggdV+zILC4mo/phBC6JsUdmEUrl+/hUYj93EKIcTjkql4IYQQwoRIYRdCCCFMiEzFC6NQ3ucxq1JBYTF5Nwv0fhwhhDAkKezCKFTHA2rWh75FHlLYhRCmTabihRBCCBMihV0IIYQwIVLYhRBCCBMihV0IIYQwIVLYjUx0dLRB3um+cuVKVq5cqfP2ycnJ+Pn5PbA8LS2tSt7PLoQQonKksAshhBAm5In5uFtycjJhYWFoNBoaNWqEpaUlKSkpqFQqRo8ezcCBA9FoNHz00UccOHAAlUrFgAEDGDduHMnJyaxevRpFUUhNTaVv377Url2bHTt2ABAZGYmtrS2zZs3izJkzAAwbNow33nij3DwrV67kwoULpKamcuPGDYYMGcKYMWMoKSkhNDSUQ4cOUVJSgo+PDyNHjiyVv3nz5ixZsqTcti9evIifnx9XrlyhU6dOLFy4EIDVq1cTFxeHubk5Xbp0ISAggKtXr+Lv709SUpI2F8CECRPKPJ+srCzmzp3LtWvXUKlUvP/++3Tu3BmA48ePM3ToUNLT0/Hx8WHy5Mnl9un9/vzzT2bPng1Ay5YtH/lnK4QQouo8MYUd4MKFC+zatYvPP/+coqIitm7dSnZ2NoMHD6Zly5YcOXKEq1evEhcXR1FREX5+frRo0YKaNWty7NgxEhISsLOzo3PnzsycOZPo6GiCgoJISEjgxRdfJDc3l5iYGHJycliyZEmFhR3g9OnTREVFodFo8PHxoVOnThw/fhyALVu2UFRUxOjRo2ndunWp/LVr166w3atXrxITE4ONjQ29evXizJkzXLlyhaSkJKKjo7GwsGDy5MlERUXx2muvldnGb7/9Vub5LFq0CF9fX3r27ElGRgbDhg0jJiYGgOvXrxMVFcWtW7fo0aMHo0aNIjY2ttw+vWfmzJkEBQXRuXNnPvvsM5KTk3X+mQohhKhaT9RUfJMmTahduzYHDx5k0KBBANStW5eePXty6NAhkpOT8fb2xtzcnJo1a+Lp6cmBAwcAaNGiBc888ww1a9bE3t6eTp06AdCwYUNu3rxJ8+bNOX/+PKNHjyYuLo4ZM2Y8NI+HhwdPPfUUtWvXpkePHhw8eJADBw6QlJSEl5cXgwcP5tq1a6SkpJTK/zCvvvoqdnZ2WFlZ8dxzz5GTk8PBgwdxd3enRo0aWFhY4Ovrqz23spR3Pvv37yc8PBwvLy/Gjh2LWq3m0qVLAHTr1g0rKyvq1q2Lvb09ubm5FfYpQHZ2NhkZGdpRv4+Pz0PPTwghhP48USP2GjVqAKAopd8CpigKJSUlaDSaMpcDWFpallpnbm5e6nt7e3sSEhLYt28fe/bswdvbm4SEBOrUqVNunvvb0Gg0mJubU1JSQkBAAH369AHuFj4bGxuOHTumzf8wFhb/+7GoVCoURXng3ADUarV2/f3LLCwsyj0fjUbDN998g52dHQDp6ek4ODiwY8cOnY57f5/ev11ZfSKEEKL6PVEj9ntcXFzYtGkTcLdw7ty5k44dO+Li4kJMTAwlJSXcuXOH+Ph4nJ2ddWpz586dzJgxg9dff53g4GBsbGy4evVqhfvs2LGDoqIicnNz2bVrF127dsXFxYWNGzdSXFxMfn4+w4YN49ixY1VyzgkJCRQUFKBWq9m8eTMuLi7UqVOH3NxcsrOzKSoq4ueff67wfFxcXFi/fj0AZ8+eZcCAAdy5c6fC41bUp/b29jRs2JDdu3cDsHXr1sc+VyGEEJX3RI3Y73n33XcJCQnB09OTkpISJkyYQKtWrWjRogUXLlzAy8uL4uJiBgwYQO/evXW65tu9e3e2b9+Ou7s71tbW9OnTBycnpwr3sba2ZtiwYdy6dYvx48fzwgsv8Pzzz3Px4kW8vb1Rq9X4+Pjg7Oz82NedXV1dOXnyJL6+vqjVarp168bw4cOxsLBg9OjRDBo0iKeffpqXX365wvMJDg5m7ty5eHp6AhAaGkqtWuW/gGXIkCEP7dOwsDCCgoL49NNPadeu3WOdpxBCiMejUv46ry10cu/u88mTJxs4iWmorpfAZGbm6bRt/fq1dd62uhhjJjDOXJJJd8aYSzI9nJmZqty3Yj6RI/bq8vXXX7Nly5YHljs6OtKmTZtKt7tt2za++OKLMtfFxsZWul0hhBBCCnsFRo4cyciRI6u83f79+9O/f/8qb1cIIYR4Im+eE0IIIUTZpLALIYQQJkQKuxBCCGFC5Bq7MArhQQP1foyCwmK9H0MIIQxNCrswCtev30KjkU9eCiHE45KpeCGEEMKESGEXQgghTIgUdiGEEMKEyDV2YRTKezTi4ygoLCbvZkGVtyuEEMZMCrswCvp4Vvz60LfIQwq7EOLvRabihRBCCBMihV0IIYQwIVLYhRBCCBMihV0IIYQwISZR2JOTk/Hz86v2fYUQQghjYxKFXQghhBB3mczH3XJychg9ejQZGRm0adOGDz/8kIMHDxIeHo5araZRo0YsWLAAe3t7fvnlFz7++GOsra1p0qSJtg0/Pz9sbW05c+YMn376KdeuXePTTz9Fo9HQuHFj5s+fj4ODA0ePHmXRokUUFhZib2/P/Pnzef755/Hz8+PFF1/kwIEDFBQUEBwczNq1azl79iwjR45k5MiRHDhwgLCwMABsbW1ZtmwZdevWLfe8XFxccHV15cSJEzz11FMsXbqURo0acfz4cT7++GMKCgqwt7dn3rx5NG7c+IFzePHFF8tst0ePHrRp04aTJ0+yfv16du/ezVdffYVKpaJVq1bMmTOHp556il27dpXZBz169MDNzY3du3djbm7O9OnT+fLLL7l48SIzZ86kf//+VfsDFkIIoROTGbGnpaUxZ84c4uLiyM/PJzIykmXLlrFmzRpiYmLo2rUrS5cupaioiMDAQMLDw4mOjqZGjRql2nFycmL79u04Ojoyd+5cPvvsM+Lj42nfvj3z58+nqKiI6dOna481dOhQpk+fXqqN+Ph4vLy8WLhwIStXrmTdunV89tlnAKxatYqQkBCio6NxdXXlzz//rPC8cnJy6NixI/Hx8bi7u7Nw4UKKiooIDg5m2bJlbNmyhVGjRjFnzpwHzqG8on5P9+7d2b59O1lZWaxevZq1a9cSHx9PzZo1iYiI4Pr162X2wT2Ojo4kJCTQqlUrIiMj+fLLLwkLCyMyMlKnn5kQQoiqZzIj9ldffZV//OMfAHh6ehIYGIhKpcLf3x8AjUaDra0tKSkpODo60qxZMwC8vb1ZsWKFtp02bdoAcPz4cdq0aUOjRo0AGDJkCJGRkVy4cIE6depot3Nzc2Pu3Lnk5eUBd4slQMOGDWnbti01a9bk2Wef5ebNmwD07NmTSZMm0atXL3r27EmXLl0qPC9ra2sGDhyozfrJJ59w4cIFLl26xDvvvKPd7tatWw+cw8O0bdsWgMOHD+Pq6oq9vb32XIOCgujYsWOZfXDP/efq6OiIhYUFDRs21J6rEEKI6mcyhd3C4n+noih3X//Zvn17Vq9eDUBhYSH5+flcuXIFjUaj3dbc3LxUO/dG8Pdvc69NtVr9wPJ760pKSgCwtLQsM9M9I0eOxNXVlV27dhEWFsbx48dLFei/MjMzQ6VSaTOZm5uj0Who1KgRsbGxAJSUlJCVlfXAOTyMtbX1I53rveX3POxchRBCVD+TmYo/cuSItmjHxMQwYsQIjh49yvnz54G7U+ChoaE4OTlx/fp1Tp06BUBCQkKZ7bVt25Zjx46RlpYGwIYNG3B2dqZp06bcuHGD48ePA7Bt2zYaNmyInZ2dTjkHDx5Mfn6+9pr7w6bi79y5Q1JSEgDR0dF0796dpk2bkpuby6+//grA5s2bmTFjhk7HL0vHjh1JSkrixo0bAGzcuBFnZ+dy+0AIIYTxMplh1gsvvMCsWbPIzMzExcWFd955h5deeolp06ah0Who0KABYWFhWFpa8sknnxAQEICFhQUvvfRSme05ODgwf/58Jk2aRHFxMQ0bNmTRokVYWVmxfPlyFixYwJ07d7C1tWX58uU655w+fTqBgYFYWFhgbW3NvHnzHrpPYmIiy5cvx9HRkSVLlmBlZcWKFSu0N/DVqlWLJUuW6Jzhr1q2bMn48ePx8/OjuLiYVq1aMW/ePGrVqlVmHwghhDBeKuXevLUwSk5OTqSkpBg6ht7p6yUwmZl5ldq3fv3ald5XX4wxExhnLsmkO2PMJZkezsxMVe5bMU1mxP6kKigoYMiQIWWumzJlymO17efnV+aNbEOHDuXNN998rLaFEEIYJynsBlajRg3tTXBleZzR+tq1ayu9rxBCiCeTydw8J4QQQggp7EIIIYRJkal4YRTCgwZWeZsFhcVV3qYQQhg7KezCKOTk5KPRVP0HNMzMVAbZV1+MMRMYZy7JpDtjzCWZKlZRFvm4mxBCCGFC5Bq7EEIIYUKksAshhBAmRAq7EEIIYUKksAshhBAmRAq7EEIIYUKksAshhBAmRAq7EEIIYUKksAshhBAmRAq7EEIIYUKksAu9iY+Pp3///vTu3Zt169Y9sP7kyZP4+vrSt29fZs+ejVqtBuDKlSu89dZb9OvXj3feeYf8/HyjyBUTE0PXrl3x8vLCy8uL5cuXV1ume2bOnEl0dLT2e332VWUz6bOfdMm1Y8cOvLy8GDBgABMnTiQ3NxcwbF+Vl8mQ/6d++uknPD09cXd3JzAwkKKiIsDwv3/l5TKG37/du3fTo0cP7fc3b95k3LhxuLm58dZbb5GZmVllmR6LIoQeXLt2TXF1dVVycnKU/Px8xdPTUzlz5kypbdzd3ZXffvtNURRFCQoKUtatW6coiqKMGzdO2bp1q6IoihIREaGEhoYaRa758+cr8fHxVZblUTJdu3ZNGT9+vNKmTRtl8+bN2uX66qvHyaSvftIlV15entKlSxfl2rVriqIoyqeffqosWLBAURTD9VVFmQz1fyo/P1/p2rWrkpmZqSiKokybNk2JiopSFMWwv38V5TLk75+iKEpmZqbSr18/xdXVVbts3rx5yhdffKEoiqJs2bJFmTp1apXnqwwZsQu92L9/Py4uLtjZ2WFjY0Pfvn1JTEzUrr98+TIFBQW0a9cOAB8fHxITEykuLubw4cP07du31HJD5wL4/fffiYmJYcCAAcyYMUM76tJ3Jrg7oujZsydubm7aZfrsq8pmAv31ky65iouLCQkJoUGDBgA4OTlx9epVg/ZVeZnAcP+nbGxsSEpKwsHBgdu3b3P9+nXq1Klj8N+/8nKBYX//AIKDg5k0aVKpZbt378bT0xMADw8P9u7dS3Gx4d8qKYVd6EVGRgb169fXfu/o6Eh6enq56+vXr096ejo5OTnUqlULCwuLUssNneve15MnTyY2NpZnnnmG+fPnV0smgDFjxjB48OBSy/TZV5XNdC+HPvpJl1z29vb06tULgIKCAiIjI+nVq5dB+6q8TPdyGOr/lKWlJXv27MHV1ZWcnBy6du1q8N+/8nLdy2Kovvr222956aWXaNu2bbn7WlhYUKtWLbKzs6sk1+OQwi70QinjpYEqleqh6x+2n6FyAXz22We0bdsWlUrFmDFj2Lt3b7Vkqur99JkJ9NdPj5IrLy+PsWPH0rJlS7y9vY2ir/6aCQz/f+q1114jOTkZV1dXQkJCDP77V14uMFxfnT59mh9//JGJEyfq1J6ZmeHLquETCJPUoEEDsrKytN9nZGTg6OhY7vrMzEwcHR2pW7cut27doqSkpNRyQ+fKy8vj66+/1i5XFEU7qtF3pvLos68qm0mf/aRrroyMDIYNG0bLli1ZtGgRYPi+KiuTIf9P3bhxg19++UX7vaenJykpKQb//SsvlyH7KjExkczMTHx9fRk3bpz2Zwl3R/f39lWr1dy6dQs7O7sqyfU4pLALvejcuTMHDhwgOzubO3fu8OOPP9K9e3ft+meffRZra2uOHDkC3L3jtXv37lhaWvLqq6+ybdu2UssNncvGxoZ///vfHDt2DID//Oc/9O7du1oylUeffVXZTPrsJ11ylZSUMGHCBNzc3Jg9e7Z25GXIviovkyH/TymKQkBAAFeuXAHghx9+oH379gb//SsvlyH7asqUKWzfvp3Y2FgiIyNxdHRk/fr1wN2ZhZiYGAC2bdvGq6++iqWlZZXkeizVeque+FuJi4tT3N3dlT59+iiRkZGKoijKmDFjlOPHjyuKoignT55UfH19lX79+inTp09XCgsLFUVRlLS0NGX48OGKm5ub8vbbbys3btwwilyHDx9WBg4cqPTr10+ZMGGCcvPmzWrLdM/MmTNL3YGuz76qbCZ99tPDcv3444+Kk5OTMmDAAO2/WbNmKYpiuL6qKJMh/0/99NNPioeHh+Lp6am899572mMb+vevvFzG8Pt36dKlUnfF5+TkKOPHj1f69++vDBkyRLl06VKVZXocKkUp4wKDEEIIIZ5IMhUvhBBCmBAp7EIIIYQJkcIuhBBCmBAp7EIIIYQJkcIuhBBCmBAp7EKIJ0KPHj34/fffq/WYu3fvZsWKFdV6TCEelxR2IYQox++//16lL7ERojpU3bMehRCiGrz88suMHDmS3bt3c+vWLQICAkhMTOT06dM4OjqyevVqbGxseOmllxgxYgTJycncvn2b6dOn06dPH+Duc8cTEhIwNzenSZMmzJkzh/r16+Pn54etrS3nzp2jf//+REVFUVJSQu3au1oxugAAA4NJREFUtRk/fjwhISH8f3v3DtJKFsdx/GuiF4khiphGo+IrGiyMmBSCjY2PIoVibHwUtmJhEVTQgKVhJKKNCKmCFlHSWCiCig8QDKQTX4hGBFELSRMxmMmtDLvrXVnYXS6O/081w+EwzGl+nMMwv5ubG+LxOHl5eSiKQmVlJQMDA9jtdqLRKPf39zQ1NTEzM4NOp2N3d5e5uTlUVcVgMDA9PU1dXR3RaBRFUXh5eSErK4uRkRFaW1t/8+oKLZBgF0J8KclkErPZzPr6OktLS0xOTrKxsYHZbKanp4ft7W1cLhepVIr8/HzC4TBnZ2f09/fjcDjY3d3l4OCAtbU1DAYDCwsLjI+PEwgEADCZTJlfqqbTaZ6fnxkdHWVzcxOTyUQoFALA6/WyvLzM1NQUALe3twSDQRKJBJ2dnRwfH1NdXY3H4yEYDGKz2dja2kJRFGZnZ5mYmCAQCGCxWHh4eKC3t5fa2lqKi4t/z8IKzZBgF0J8Oe994WVlZVit1kzXucVi+dPReX9/PwB1dXVYrVYikQj7+/t0d3djMBgAGBwcZHFxkWQyCYDD4fjlMzs6OigtLSUYDBKLxTg+PqaxsTEz3traik6nw2g0Ul5eTjweJxqNUlNTg81mA6CtrY22tjb29vZ4enpieHg4Mz8rK4vz83MJdvGvSbALIb6cPxZtfFa6odfrM9eqqqLX6z/UdKqqytvbW+b+PfD/amVlhVAoRF9fHy6Xi4KCAu7u7jLjubm5mev3CuLs7OwPtcDn5+ekUimqqqpYXV3NjD08PFBYWPjZawvxj8jHc0IIzXpv3jo5OeH6+hqn00lLSwvhcJhEIgFAMBjE6XTy48ePD/P1en0m9A8PD+nq6sLtdlNRUcHOzk6m3vTvNDQ0cHV1xeXlJQDb29t4PB7sdjuxWIxIJALA6ekp7e3tPD4+/mfvLr4v2bELITQrGo0SCoVQVRW/309+fj49PT3c39/jdrtRVZXy8nIURfnl/ObmZkZGRsjJyWFoaAiv10s4HEav11NfX8/FxcWnzy8qKkJRFMbGxkilUhiNRvx+P4WFhczPz+Pz+Xh9fSWdTuPz+SgpKfk/lkF8M9LuJoTQpNraWo6OjuR4W3w7chQvhBBCaIjs2IUQQggNkR27EEIIoSES7EIIIYSGSLALIYQQGiLBLoQQQmiIBLsQQgihIRLsQgghhIb8BNqDMTM0+OTbAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "feature_importance = rf_feature_importance(model, X_keep)\n", "plot_feature_importance(feature_importance);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We've now got a model that isn't really more predictive than our baseline, but it's much simpler - it has just 9 features instead of 19 and we now know that `median_income` and `ocean_proximity_INLAND` are particularly important features to focus on." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "#### Exercise #2\n", "\n", "Go through the top 3 features and use plots with `seaborn` to gain insights into things like the following:\n", "\n", "* What is the relationship between each feature and the target `median_house_value`?\n", "* What do the distribution / bar plots look like for continuous / categorical columns?\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Removing redundant features" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Looking at our feature importance plots, we can see that there are some features that seem to be related to each other (e.g. the square footage features like `rooms_per_household` and `bedrooms_per_room`). Features like this are potentially measuring the same thing, so we can use a special technique called _**hierarchical clustering**_ to produce something called a _**dendogram**_ that will tell us which pairs of features are similar:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def plot_dendogram(X):\n", " \"\"\"Plots a dendogram to see which features are related.\"\"\"\n", " # calculate correlation coefficient\n", " corr = np.round(scipy.stats.spearmanr(X).correlation, 4)\n", " # convert to distance matrix\n", " corr_condensed = hc.distance.squareform(1 - corr)\n", " # perform clustering\n", " z = hc.linkage(corr_condensed, method=\"average\")\n", " # plot dendogram\n", " fig = plt.figure(figsize=(16, 10))\n", " dendrogram = hc.dendrogram(\n", " z, labels=X.columns, orientation=\"left\", leaf_font_size=16\n", " )\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABFEAAAJBCAYAAACOHyoGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOzdeVhV5d7/8Q+jA6AIipnmlLkzBQEV0lQU8+BjZjmVT0p5UnEIBzTUcyozJ1KRQUJTMU3lJD2GlZVNmuNRpEw7p9LK4ZTlgCgiOCHs3x/+WMcdyl444fB+XZfXtdda91r3d619+8f+sNa9HKxWq1UAAAAAAAAolWN5FwAAAAAAAHA7IEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARnew1OnMhXUZH1ZtRy2/H2dld2dl55l4FbGGME9jBGYA9jBKVhfMAexghK4+jooGrV3Mq7DOC2YjdEKSqyEqKUgmsDexgjsIcxAnsYIygN4wP2MEYA4PrhcR4AAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAADAbcNqtZZ3CbiLEaIAAAAAAEplsVi0aNGim97vhAkT1K1bN2P53XffVUJCwjUfNykpSQEBAdd8HNx9nMu7AAAAAAAALmf48OE6ffq0sfzmm2+qQ4cO5VcQ7nqEKAAAAACAW1LdunXLuwTABo/zAAAAAADK5ODBgxo1apRat26tgIAADRs2TAcOHDC2JyUlqWfPnvroo48UFhYmX19f9erVSzt27LA5zpo1a9StWzf5+fmpd+/e+vLLL2WxWJSRkSHJ9nGe0NBQ/f7770pNTZXFYimxvVjxMQ4ePGisW7RokTp27Ch/f39FR0fr7NmzJc7po48+0uOPP65mzZrp0Ucf1bJly67LtcKdhRAFAAAAAGDa4cOH1adPH/3nP//RpEmTFBMTo4MHD+qZZ57RkSNHjHYHDhzQnDlzFBkZqaSkJJ07d06jRo3ShQsXJEkbN25UVFSUfH19lZycrDZt2mjs2LFX7PeNN95QjRo1FBYWprS0NNP1Llq0SLNnz1aPHj00Z84cFRQU6O2337Zps2rVKo0dO1atWrXSm2++qSeffFIxMTFKSUkp49XBnY7HeQDgOvph5x/6+fujptu7uDqp4HzhDawItzvGCErD+IA9jBGUxr1KBfV9PqjM+y1ZskRnz57VW2+9JS8vL0lSUFCQHn30US1evFgTJkyQJOXn52vJkiXy8/OTJBUWFmr48OHavXu3mjVrprlz56pVq1aKiYmRJLVr1075+flavnz5Zft96KGH5OrqqurVq8vf399UrUVFRVq4cKH69OmjkSNHGv088cQT+u2334w2cXFxevzxxzVx4kRJUtu2beXg4KC5c+fqmWeeUeXKlct8nXBnshuieHu734w6bls1aniUdwm4xTFG7i4Hfs5Wdla+7qldxfQ+Lq5ON7Ai3AkYIygN4wP2MEZwJc4uVzc2MjMzFRwcbAQokuTl5aXWrVtr+/bt/z2+s7OaNWtmLN9zzz2SpDNnzujcuXPatWuXxo8fb3PsLl26XDFEuRr79+/XiRMn1L59e2Odg4OD/vKXvxhvG9q/f7+OHj2qDh06GHfJSFL79u01Z84cfffdd3r44YevW024vdkNUbKz81RUxHu4L6dGDQ9lZZ0q7zJwC2OM3H0KzhfKu4abuvbxNdWeMQJ7GCMoDeMD9jBGUBpHR4er2i83N1dNmjQpsd7b21u//PKLsezq6ipHx//OIFH8uaioSCdPnlRRUZFNEFN8jOvp5MmTkqRq1arZrK9evbrxOScnR5I0duzYyz5OlJWVdV1rwu2Nx3kAAAAAAKZVrVpVx44dK7H+2LFj8vT0NHUMb29vubi46Pjx4zbr/7xsj4ODg4qKimzW5efnG5+L6/nzcYuDE0ny8Lh45/jEiRONR48uVadOnTLVhDsbE8sCAAAAAExr0aKFMjIybIKJ48ePa+vWrQoMDDR1DCcnJ/n7+2vdunU269euXVvqfpfe2SJJbm5uys7OtglSvvnmG+NzgwYN5OPjo88//9xmvw0bNhifGzZsKE9PTx05ckS+vr7Gv5ycHCUmJiovL8/UOeHuwJ0oAAAAAADTBgwYoFWrVun555/XsGHDJEnz5s2Tq6urnnvuOdPHeeGFF/TXv/5VL7/8srp06aKdO3ca86H8OSwpVqVKFX3//ffavn27WrVqpfbt22vZsmV67bXX1LVrV23btk1ffvml0d7BwUEjR47UK6+8Im9vbz3yyCNas2aNvv/+ezk5XZwTxtnZWSNGjNDrr78uSWrdurUOHjyo2bNnq379+tyJAhvciQIAAAAAMK1WrVpKTU2Vj4+PJkyYoJdeekm1a9fWihUrjMljzWjdurVmzpypzMxMDR06VBs3bjTmJLnS23CGDBmi//znPxo8eLCOHDmi9u3bKyoqSmvXrlVERIR+/PFHIwwp1qdPH02ZMkVffvmlhg8fruzsbA0dOtSmTf/+/TVp0iStW7dOgwcPVmJiorp06aL58+fLweHq5o7BncnBarWWOmssE8teGRN1wR7GyN3ng9SdkqQn+pl77R5jBPYwRlAaxgfsYYygNI6ODuX6NtYvv/xSdevWVePGjY11aWlpmjRpkjIyMlSlivm3HQI3C4/zAAAAAABuuq+++kqbN2/W2LFjVatWLe3du1fx8fHq3r07AQpuWYQoAAAAAICb7u9//7tmz56t2bNnKzs7Wz4+Purbt69eeOGF8i4NuCJCFAAAAADATefm5qaJEydq4sSJ5V0KYBoTywIAAAAAAJhAiAIAAAAAAGACIQoAAAAAAIAJhCgAAAAAAAAmEKIAAAAAAACYQIgCAAAAAABgAiEKAAAAAACACYQoAAAAAAAAJhCiAAAAAAAAmECIAgAAAAAAYAIhCgAAAAAAgAmEKAAAAAAAACYQogAAAAAAAJhAiAIAAAAAAGACIQoAAAAAAIAJhCgAAAAAAAAmEKIAAAAAAACYQIgCAAAAAABgAiEKAAAAAACACc7lXQBwo63f+bsyvj9SLn27uDqp4HxhufSN8nHsaJ7a+lQp7zIAAAAA3ADciYI7Xsb3R/Tr0bzyLgMAAAAAcJvjThTcFer6uGt8v8Cb3m+NGh7Kyjp10/tF+fkgdWd5lwAAAADgBuFOFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAADc8axWa3mXgDsAIQoAAAAA4LaRlJSkgICAMu3z9ddfa+TIkcZyenq6LBaLjh8/fr3Lu+VZLBYtWrSovMsw5Wq+68sJDw/XkCFDSm0TGhqqyZMn2z0WIQoAAAAA4I62cuVK7d+/31ju0KGD0tLSVKVKlXKsCrcj5/IuAAAAAACAm8nLy0teXl7lXQZuQ9yJAgAAAAAolcVi0YoVKzRs2DA1b95coaGhSk1NtWmTn5+vGTNmKDQ0VH5+furdu7c2b95sbM/IyJDFYtHmzZvVo0cP+fn5qWfPntq6davR5nKP2eTm5spisSg9Pf2ytRUUFGjOnDkKCwtTs2bN1KpVK0VGRurQoUOSpAkTJmjVqlX6+eefZbFYlJGRUaIfq9Wqd999V48//rj8/Pz0l7/8RUuWLClxDdLT0xUVFaWAgAAFBwdr2rRpunDhgunrePDgQVksFq1Zs0b9+/eXn5+funbtqk8++cSm3enTpzVlyhS1adNGfn5+Cg8P1w8//GBznYKDg5WSkqLg4GCFhITo9OnTpuvIycnRmDFjjPOYPn26CgoKjO1mv8t//etfNsdt2bKlkpKSjOWUlBR17txZvr6+evTRR5WcnKyioiJje3Z2tsaNG6egoCAFBARo6NCh+u2330rU+8knnygsLEy+vr7q2bOnduzYYbM9MzNT/fr1U2BgoNq0aaPJkycrPz//iueflZWlkSNHqkWLFmrXrp3ef/9909eOO1EA4Do7djRPH6TuNNXWxdVJBecLb3BFuJ0xRlAaxgfsYYygNO5VKqjv80Gm28fGxiokJERJSUnasmWLJk+eLBcXFz311FMqKirSoEGDtG/fPkVFRalWrVp67733FBERofnz56tdu3bGccaOHavw8HCNHj1ay5Yt0+DBg5Wenq7GjRtf1XnExMToo48+0rhx41S3bl39/PPPiouL0/Tp05WUlKThw4fr+PHj2rdvn2JjY9WoUSP9/vvvNseIi4vTokWLNHjwYLVq1Urbt2/XjBkzdOLECUVFRRntpk+frieeeEJz585VZmamkpOT1aBBAz3zzDNlqvmVV17Rk08+qYiICK1evVpjxoxRlSpV1LZtW1mtVg0bNkx79uzRmDFjVKNGDS1fvlzh4eFatWqV6tatK0k6deqUVq9erdjYWOXn56ty5cqm+09JSdEzzzyjuXPnatu2bXrzzTd13333KTw8vEzfZWk++OADJSYmasKECXrggQf07bffKj4+Xt7e3urbt6/Onj2rZ599VmfPntXLL7+sSpUqaf78+erfv78+/PBDVa1aVZJ05swZxcfHa/To0apUqZJmzZqlESNGaMOGDXJ2dtaGDRs0dOhQhYWFKSIiQn/88Yfi4+P1008/aenSpXJ0tL13pLCwUAMHDlReXp6mTJkiq9Wq2bNn68iRI6bOy26I4u3tbupAd6saNTzKuwTY4eLqJKn8vivGyN0lILiu/v3t7/YbXqJ4jAJXwhhBaRgfsIcxgitxdinb2GjYsKFmz54tSWrfvr0OHTqk+fPn66mnntL69eu1Y8cOpaSkGD+yQ0JC9PTTTys+Pt7mh3efPn0UGRkpSWrdurUeffRRLV68WDExMVd1HsePH9e4cePUu3dvSVJQUJD279+v1atXS5Lq1q0rLy8v/fHHH/L39y+x/4kTJ7R48WINHDjQCEyKw4xFixbpueeeMx79CQgI0CuvvGLU/tVXX2njxo1lDlHatWunl19+WdLFa7l//37Nnz9fbdu21ebNm7Vt2zYtXrxYbdq0Mdo/9thjmjdvnnGdCgsLFRkZaTrUuFSbNm1szmPdunXKyMhQeHh4mb7L0nzzzTeqXbu2nnnmGTk4OCgoKEjOzs7y8fGRJL3//vvG93T//fcbtXTs2FHLli0zxojVatWsWbOM7+7ChQsaMWKEfvnlFz344INKTEyUn5+fEhISjL7r1KmjQYMGaf369QoNDbWpa/369dqzZ4/S0tKMY9avX189e/Y0dV52Q5Ts7DwVFfEqqMupUcNDWVmnyrsM2FH815fy+K4YI3efuo28VLeR+edrGSOwhzGC0jA+YA9jBKVxdHQoU/uuXbvaLHfq1EmfffaZDh8+rMzMTLm5uZX4gd21a1fFxMQoLy/PWPfYY48Zn11dXdWuXTtt3779Ks7gouIfz0eOHNG+ffu0b98+7dixQ+fPnze1/65du1RQUKAuXbrYrH/ssce0YMEC7dq1Sx07dpQkNW/e3KZNzZo1y/QYzaXHvlRoaKjmzp2roqIiZWRkqFKlSmrVqpXNo0Jt27bVunXrbPZr0KBBmfuWVOKNN7Vr11Zubq4klem7LE3Lli2VlpamXr16qUuXLurQoYMGDhxobM/IyFC9evVUr1494zwrVqyoFi1aaNu2bUaI4uTkJD8/P5tapYt34uTn5+uHH37Q+PHjbfpu166dqlatqszMzBIhyo4dO1S1alWbQK1p06bGce3hcR4AAAAAgF3FdxAUK747IycnR7m5uapevXqJfapXry6r1WozP0WNGjVKHOfkyZNXXdeOHTs0adIk7dmzRx4eHmrSpIkqVKhgev/ivv9cv7e3tyTZhAaVKlWyaePo6Cirtew3HVzuGhQUFOj06dPKycnRmTNn1KxZsxL7ubi4lNjvapR2HmX5LkvTvXt3FRYWKjU1VXFxcZo9e7YsFoumTZsmX19f5eTkaN++fWratGmJfevXr298rlChgs0jOcWfi4qKdOrUKVmtVuO7upSXl9dlA5/c3FxVq1atxPo/fydXQogCAAAAALDrxIkTNsvZ2dmSLv5YrVq1qo4dO1Zin6ysLEmSp6ensS4nJ8fmR3p2drYRBjg4XLw75tJgorQ7PU6dOqWhQ4cqMDBQSUlJqlevniRp5syZ2r17t6nzKq7t2LFjqlmzprG++Hwurf16ycnJsVnOzs5WhQoV5ObmJg8PD3l7e2v+/PnXvV8zzHyXl/uerFarzpw5Y7NPjx491KNHD2VnZ2vdunVKTk7WuHHjtGbNGnl4eOjBBx/U1KlTS/Tl6upqqlYPDw85ODgYY/FSx44du+x35+npedn2f/5OroS38wAAAAAA7Fq/fr3N8tq1a9WwYUP5+PioRYsWys/P16ZNm2zarFmzRk2bNrW5M+Srr74yPp8/f14bN25UcHCwJMnd/eKcnEePHjXafP3111esad++fTp58qSee+45I0ApKirSP//5T5sf+H+eXPRSvr6+cnFx0aeffmqz/pNPPpGzs7PNoyTXy6XXQLp4LYOCguTg4KAWLVro+PHjqly5snx9fY1/q1ev1ocffnjda/kzM9/l5b6nnTt32jx+9NJLL2nkyJGSLt7V06dPH/Xu3dt4a1JgYKAOHjyo2rVrG+fYrFkzLVmypMRYuxI3Nzc1adKkxHe3adMmnTp1SoGBgSX2CQ4O1qlTp2zeCrVv3z79+uuvpvrkThQAAAAAgF2bNm3S5MmTFRoaqvXr1+uLL74w5iPp0KGDmjdvrujoaOONLunp6dq1a5fmzZtnc5y5c+fKxcVFDRo00NKlS3X69GkNGjRI0sUfuBUqVNC0adM0bNgw/fHHH5o3b94V70xo2LCh3NzcjPlEzp49q3/84x/avXu3HBwcZLVa5eDgoCpVqujw4cPasmVLicdkvLy8FB4erkWLFsnJyUmtWrVSZmamFi1apAEDBhhvibme/u///k9eXl4KCAjQ+++/rz179mj58uWSpI4dO8rX11cRERGKjIxUrVq19Nlnn+kf//iHXnvttetey5+Z+S4tFotq1qypxMREOTs7Ky8vT3PmzJGHx39fqtGqVSuNHz9ecXFxatOmjQ4fPqx33nlHnTt3liT17t1by5Yt0/PPP6+IiAh5enoqLS1Nn3/+ubp372663hEjRmj48OEaPXq0evbsqUOHDikuLk4BAQFq3759ifaPPPKIWrVqpejoaL344ouqXLmyEhISSjwqdSWEKAAAAAAAuwYNGqQff/xRw4cPV926dRUfH29Mxurk5KSUlBTFxsYqPj5eZ86cUZMmTbRgwYISP2THjRun5cuX6+DBg/Lz81Nqaqruu+8+SVKVKlWUkJCg2NhYDRkyRA888IBmzpypF1544bI1eXh4KCkpSTNnztSwYcNUrVo1tWzZUomJiRo5cqR27dolf39/Pf300/rqq680ZMgQzZw5s8RxoqOjVa1aNaWlpSklJUW1a9fW+PHj9eyzz17nq3jR6NGj9cUXXyglJUWNGzdWSkqKMdmrk5OTFi1apNjYWM2aNUt5eXmqV6+eYmJiTL9B5lqY+S6dnJyUkJCgadOmKTIyUnXq1FF0dLRNYPbkk08qLy9PqampWrJkiTw8PBQWFqaxY8dKunjXUWpqqmbOnKlJkybp/PnzeuCBBzR37lyFhISYrjc0NFTJyclKTk7W8OHD5enpqW7duikqKkpOTiXfQOXg4KB58+Zp+vTpmjZtmpydnfX888/riy++MNWfg9XOLDi8nefKmO389jAjdYckaXy/krdy3WiMEdjDGIE9jBGUhvEBexgjKI2jo4O8vd1NtbVYLBo3bpzN21XKKiMjQ88++6xWrlwpX1/fqz7O7ezgwYPq1KmTEhMTS7wNCLcH7kQBAAAAAOAaWK1WFRYW3vL9OzsTAVwrriAAAAAAANdg1apV+tvf/ma33dq1a29I/9u3bzf16NHatWtVp06dG1LD3YIQBQAAAABQqj179lzzMYKDg6/LcW5FHTt21MqVK+22q1Onzg25Bk2bNjXVv4+Pz3Xv+25DiAIAAAAAwDWoVq2aqlWrVm79u7u737XzzNxsV35ZNgAAAAAAAAyEKAAAAAAAACYQogAAAAAAAJhAiAIAAAAAAGACIQoAAAAAAIAJhCgAAAAAAAAmEKIAAAAAAACYQIgCAAAAAABgAiEKAAAAAACACYQoAAAAAAAAJhCiAAAAAAAAmECIAgAAAAAAYAIhCgAAAAAAgAmEKAAAAAAAACYQogAAAAAAAJhAiAIAAAAAAGACIQoAAAAAAIAJzuVdwPWwfufvyvj+yE3v18XVSQXnC296vyibX4+eUl0fj/IuAwAAAABwm7sj7kTJ+P6Ifj2aV95lAAAAAACAO9gdcSeKJNX1cdf4foE3tc8aNTyUlXXqpvaJspuRuqO8SwAAAAAA3AHuiDtRAAAAAAAAbjRCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAACUymKxaNGiReVdhtLT02WxWHT8+PHyLuWGmjBhgrp162Ys3yrXH5JzeRcAAAAAAIAZHTp0UFpamqpUqVLepdxUaWlpuvfee8u7DIgQBQAAAABwm/Dy8pKXl1d5l3HT+fv7l3cJ+P94nAcAAAAAYFdOTo7GjBmjgIAABQcHa/r06SooKDC25+fna8aMGQoNDZWfn5969+6tzZs3G9szMjJksVj0r3/9y+a4LVu2VFJSkrGckpKizp07y9fXV48++qiSk5NVVFQkqeTjPKGhoVq4cKFeffVVBQUFKTAwUOPHj1deXp5xvHPnzmnq1Klq3bq1AgMD9dJLLykuLk6hoaFlOn+LxaKVK1dqxIgR8vf3V9u2bfWPf/xDR44cUUREhPz9/RUWFqYNGzbY7Ldlyxb16dNHfn5+at++vRITE1VYWGhsv3DhgmJjY/XII48oMDBQMTExNtuL+770cZ5Nmzapf//+CggIkK+vr5544gl9/vnnxvakpCT17NlTH330kcLCwuTr66tevXppx44dZTpnSfrwww/Vq1cvNW/eXM2bN1ffvn2VmZlp02bNmjXq1q2b8b1/+eWXslgsysjIMNr8+9//1nPPPafmzZvr4Ycf1pQpU3TmzJky11PeCFEAAAAAAHalpKSoWrVqmjt3rvr27au3335bK1askCQVFRVp0KBBSk9PV0REhJKSknTvvfcqIiJCmzZtMt3HBx98oMTERA0YMECLFi1Snz59lJSUpHffffeK+8yfP1+5ubmKi4vT6NGj9fHHH2vevHnG9r///e9KT09XZGSkZs+erV9//VWLFy++qmsQExOjevXqad68eQoICNCUKVM0YMAABQYGau7cufLw8FB0dLQRDmzdulWDBw9WnTp19MYbb2jgwIFavHixpk6dahxz+vTpWrZsmQYPHqy4uDjt3r1ba9asuWIN3333nSIiIvTAAw9o7ty5io+PV6VKlTR27FibuWIOHDigOXPmKDIyUklJSTp37pxGjRqlCxcumD7fTz/9VOPGjVOHDh20YMECxcTEKDc3V1FRUTp//rwkaePGjYqKipKvr6+Sk5PVpk0bjR071uY4v/zyi/r37y8HBwclJCToxRdf1CeffKLRo0ebruVWweM8AHCdbT96UruOnzLV1mWvkwoKCu03xF2LMYLSMD5gD2MEpalawUUveDcy3b5NmzZ65ZVXJEmtW7fWunXrlJGRofDwcK1fv147duxQSkqK2rVrJ0kKCQnR008/rfj4eGOdPd98841q166tZ555Rg4ODgoKCpKzs7N8fHyuuM8999yjuLg4OTg4qG3bttq+fbs2btyo6Oho7d+/Xx999JFiYmLUs2dPSdLDDz+sTp06mT7vSwUEBOjFF1+UJNWsWVOff/65/P39NXToUEmSg4ODBgwYoAMHDqhJkyZKSEhQ8+bNFR8fL0lq3769qlatqr/97W8aOHCg3N3dtWLFCo0ePVoDBgwwrm3Hjh2vWMPPP/+szp0769VXXzXW3XvvverRo4d27dpl7Jufn68lS5bIz89PklRYWKjhw4dr9+7datasmanz/fXXX9WvXz+NGDHCWOfi4qLIyEgdOHBAjRs31ty5c9WqVSvFxMRIktq1a6f8/HwtX77c2Gfu3LmqXr26FixYIFdXV0lS/fr11a9fP2VmZqpVq1am6rkV2A1RvL3db0Yd18TF1UmSVKOGx03vuzz6RNmU5/goz35Rfn7Ye0iHz5zXfVUqmWrv4uJ0gyvC7Y4xgtIwPmAPYwRX4uxStgcTAgICbJZr166t3NxcSVJmZqbc3NxKhCVdu3ZVTEyMzeM1pWnZsqXS0tLUq1cvdenSRR06dNDAgQNL3cfX11cODg7G8j333KMff/zRqEuSHn30UWN7pUqVFBISYvOoiVnFgYQkVa9eXZJsAglPT09JUm5urs6cOaPvvvtOUVFRNnd/tNXiGH8AACAASURBVG/fXkVFRcrIyFD16tVVWFio9u3bG9srVKigkJCQEo89FevVq5d69eql06dPa+/evTpw4IC2bdsmScbdIZLk7OxsU9s999wjSWV6hCYiIsI4n3379mn//v1at26d0de5c+e0a9cujR8/3ma/Ll262IQoGRkZ6tSpkxwdHY1r4e/vL3d3d23duvXOClGys/NUVGS9GbVctYLzF9P1rCxzf/m9XmrU8LjpfaLsymt8SIyRu1VBQaHuqeSqAffXstuWMQJ7GCMoDeMD9jBGUBpHRwf7jS5RqZLtH4gcHR1ltV78rZibm2uECpeqXr26rFar8vPzTfXRvXt3FRYWKjU1VXFxcZo9e7YsFoumTZsmX19fU3U5ODgYdZ04cUIuLi4l3uZzuVrNcHNzs9t/sdzcXBUVFWn27NmaPXt2ie1ZWVnGXRnVqlUzXd/p06c1ceJE45GfBg0a6MEHH5Qk47wlydXVVY6O/w3Kij8Xzy9jRlZWll566SVt3LhRLi4ueuCBB1S7dm2jr5MnT6qoqKjEZL/e3t42yzk5OUpLS1NaWtpl+7id8DgPAAAAAOCaVK1aVceOHSuxvvgHsqenp3G3yKU/9K1Wa4k7I3r06KEePXooOztb69atU3JyssaNG1fqPCFXUrNmTRUUFCg3N9cmSLl07pAbpThwGTZs2GUfH/Lx8dFPP/1k1FOzZk1jW05OzhWPO2XKFG3ZskULFixQq1at5Orqql9++UWrV6++zmcgjR07VkeOHFFaWpqaNm0qZ2dnbdiwwZjE1tvbWy4uLiWu55+X3d3d1alTJ/3v//5viT7+HCDd6phYFgAAAABwTVq0aKH8/PwSk8iuWbNGTZs2VYUKFeTufnGqiKNHjxrbd+7cafOoy0svvaSRI0dKuvgDvU+fPurdu7cOHTp0VXUFBATI0dHReARFuvgYSlkmu71a7u7uevDBB/Xbb7/J19fX+Ofi4qK4uDgdPnxYAQEBcnV1tXmzzoULF7Rly5YrHnfnzp1q166dHnnkEeNOluLzuTSguh527typrl27qnnz5nJ2di7Rl5OTk/z9/W2uryStXbvWZrlFixbat2+fmjVrZlyHWrVqafbs2fr555+va803GneiAAAAAACuSYcOHdS8eXNFR0crKipKtWrVUnp6unbt2mW8KcdisahmzZpKTEyUs7Oz8vLyNGfOHHl4/HcOwVatWmn8+PGKi4tTmzZtdPjwYb3zzjvq3LnzVdVVr149Pf7445o6dapOnz6t2rVra+nSpcrKytK99957Xc69NCNHjtQLL7wgd3d3de7cWSdOnFBCQoIcHR3VuHFjVapUSQMHDtTChQtVsWJFNWnSRO+8846OHTumunXrXvaYvr6+WrdunVatWqVatWpp27ZtxuuPz549e13r9/X11apVq2SxWFS1alV98cUXeuedd2z6euGFF/TXv/5VL7/8srp06aKdO3ca86EUP0I0fPhw9e3bV6NGjVKvXr10/vx5zZ07V4cOHdJDDz10XWu+0QhRAAAAAADXxMnJSSkpKYqNjVV8fLzOnDmjJk2aaMGCBcakqU5OTkpISNC0adMUGRmpOnXqKDo62uZ1xE8++aTy8vKUmpqqJUuWyMPDQ2FhYSVemVsWkyZNUsWKFZWQkKALFy6oW7du6tKli3755ZdrPm97OnXqpLlz5yo5OVnp6elyd3dXmzZt9OKLLxpzqYwaNUoVK1ZUamqqcnNz9Ze//EVPPfWUMVnsn02YMEFnz57V9OnTJUn333+/3njjDU2fPl3ffvutevTocd3qj4mJ0Wuvvaa//e1vqlChgiwWi5YuXaqIiAjt3LlTQUFBat26tWbOnKnk5GS9//77euihhzR27FjFxMSocuXKki5Ovvv2228rISFBI0eOVIUKFRQYGKiZM2faPMZ0O3Cw2rnf53aYWHZG6g5J0vh+gTe1Xybquj2U1/iQGCN3q4W7D0qSBj9Yx25bxgjsYYygNIwP2MMYQWkcHR1ui7exXovjx49ry5Yt6tixo/E4kST17dtX1atX1xtvvFGO1d0ZvvzyS9WtW1eNGzc21qWlpWnSpEnKyMgoManv7Y47UQAAAAAAd6SKFSvqtdde06effqq+ffvK2dlZa9as0c6dO7V48WJZrVYVFhbaPU7xfCB3isLCQrvzpzg6Otq83edKvvrqK23evFljx45VrVq1tHfvXsXHx6t79+53XIAiEaIAAAAAAO5QlStX1ltvvaX4+HiNGTNGBQUFslgsevPNN9W6dWtlZGTo2WeftXuctWvXqk4d+3cZ3y46d+6s33//vdQ2kZGRGjFihN1j/f3vfzde45ydnS0fHx/17dtXL7zwwvUq95ZCiAIAAAAAuGP5+flp8eLFl93WtGlTrVy50u4xfHx8rndZ5WrevHk6f/58qW3MnrObm5smTpyoiRMnXo/SbnmEKAAAAACAu5K7u7t8fX3Lu4ybzmKxlHcJty37DzgBAAAAAACAEAUAAAAAAMAMQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABOfyLgC4GX49mqcZqTtuer8urk4qOF940/tF+Tp0+pyatruvvMsAAAAAcJ0RouCOF9y0ZnmXAAAAAAC4AxCi4I7Xwb+2OvjXLpe+a9TwUFbWqXLpG+Vn4e6D5V0CAAAAgBuAOVEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAQKksFosWLVp02x0buN4IUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAYNeJEyc0fPhw+fn5KTQ0VEuWLLHZfuHCBSUmJqpDhw7y9fVVz549tXXrVps2+/fv18CBAxUQEKDOnTtr06ZNNtvT09MVHByslJQUBQcHKyQkRKdPn1ZBQYEWLFigsLAw+fr66vHHH9fq1att9s3Pz9eMGTMUGhoqPz8/9e7dW5s3bza2Z2RkyGKxaNu2berTp4/8/PzUrVs3ff311/r666/15JNPqnnz5nrmmWf0n//8x9hv165d6tevnwICAhQUFKSRI0fq999/L9O1s1gsWrFihYYNG6bmzZsrNDRUqampZbp+xfWvWLFCjzzyiIKDg/Xbb7+Z6j80NFSxsbF66qmn5Ofnp5SUFEnS7t27NWjQIAUFBSkoKEjR0dE6duyYzb6ZmZnq16+fAgMD1aZNG02ePFn5+fnG9vDwcE2bNk2zZs3Sww8/rMDAQL366qs6ffq0pk6dqpYtW6pt27aaP39+ma7Zrcq5vAsAgDvRodPntHD3QbvtXPY6qaCg8CZUhNsVYwSlYXzAHsYISlO1gote8G5kuv1bb72lbt26KTk5Wf/85z8VExMjZ2dn9e/fX5L0yiuvaM2aNRo5cqQaNWqkDz/8UIMHD9bSpUsVGBiovLw8hYeHq3r16po1a5ZOnDihCRMmlOjn1KlTWr16tWJjY5Wfn6/KlStrzJgxWrdunUaMGCGLxaLPP/9cL774os6ePas+ffqoqKhIgwYN0r59+xQVFaVatWrpvffeU0REhObPn6927doZx4+OjlZkZKRq1aql119/XaNHj1blypU1fPhwubu766WXXtLkyZO1aNEinTp1ShEREXrkkUcUGRmp3NxczZo1S2PGjFFaWlqZrndsbKxCQkKUlJSkLVu2aPLkyXJxcdFTTz1l6voVW7hwoaZMmaLc3Fzdd999pvtfvHixRo4cqWHDhqlevXr68ccf1bdvX/n7++v1119Xbm6uEhMT1b9/f6Wnp6ty5crasGGDhg4dqrCwMEVEROiPP/5QfHy8fvrpJy1dulSOjhfvy3jvvfcUFBSkuLg47dixQ0lJSdq2bZv8/f01Z84crV69WnFxcQoKClJAQECZrtutxm6I4u3tfjPquCYurk6SpBo1PG563+XRJ24vjJG7T9szNZTxx3HT7V1cnG5gNbgTMEZQGsYH7GGM4EqcXcr2YEJAQIBmzpwpSWrXrp0OHTqkhQsXqn///tq7d6/S09M1depU9enTR5LUvn17ZWVlKSEhQUuXLlV6erpOnDihlStX6p577pEkVa1aVSNGjLDpp7CwUJGRkUbwsWfPHn388cd67bXX1LdvX0lS27ZtlZeXp7i4OPXs2VMbNmzQjh07lJKSYuwXEhKip59+WvHx8TYhSnh4uJ5++mlJ0h9//KFXX31VM2bM0JNPPilJ+ve//63ly5dLkvbu3aucnByFh4cbP/6rVaumbdu2qaioyAgRzGjYsKFmz55tXJtDhw5p/vz5euqpp0xdv0vrDw0NNd1vsfvvv19DhgwxlkeMGCEvLy8tXLhQrq6ukqRmzZrp8ccf13vvvafw8HAlJibKz89PCQkJxn516tTRoEGDtH79eqMOZ2dnJSQkqGLFimrTpo1WrFihoqIixcTEyNHRUS1bttQHH3ygXbt23fkhSnZ2noqKrDejlqtWcP5iup6Vdeqm9lujhsdN7xO3F8bI3alJpQpqcn8tU20ZI7CHMYLSMD5gD2MEpXF0dChT+86dO9ssd+zYUWvWrNHBgwe1fft2SRd/+F+4cMFoExISori4OJ0/f147duxQ48aNjQBFkjp16iQnp5JBX4MGDYzPX3/9tSSpS5cuNm26du2qjz/+WHv37lVmZqbc3NxswpLiNjExMcrLyzPW+fn5GZ+rV68u6WJ4UMzT01OnTl38f9OoUSN5enpq6NCheuyxxxQSEqLWrVsrKCiotEt1WV27drVZ7tSpkz777DMdPnzY1PUrdum1KYs/75eZmalu3boZAYp08XwtFosyMzPVs2dP/fDDDxo/frzNfu3atVPVqlWVmZlphCiNGzdWxYoVjTbe3t6qX7++ETK5urqqcuXKxnW9nfE4DwAAAADAruLAoZiXl5ckKS8vTzk5OZIuhgCXc+LECeXm5qpatWo2652cnIzjXO7YknTy5Ek5OzvL09PzsvXk5eUpNze3RH3FbaxWq80cHm5ubiXaXRoAXMrd3V3Lly9XcnKyVq1apdTUVFWpUkUREREaPHjwZfe5Eh8fH5vl4nPMyckxdf3+vF9ZeXt72yzn5uaWWFfcLi8vT6dOnZLVar1sGy8vL5tg6nLXtFKlSldV562OEAUAAAAAYNfJkydtlrOzsyVdfLzFw8NDDg4OWrFixWXvLKlWrZo8PT21d+9em/VWq7XEcf+satWqunDhgnJycmyClOIJUD09PVW1atUSE6JKUlZWltHmaj3wwANKSEjQ+fPn9c033+jtt99WbGysgoKC1Lx5c9PHuTQIkf57/by8vExdvwMHDlz1OVxO1apVjRoudezYMd1///1GTVdqcy3X9HbG23kAAAAAAHb9+U06n332merXr6+aNWuqRYsWslqtysvLk6+vr/Fv69atWrJkiZydnRUcHKyff/7ZJgzYunWrzaMql9OiRQtJ0qeffmqz/pNPPjEeG2nRooXy8/NL1LhmzRo1bdpUFSpUuKpz3rhxo1q3bq3jx4/L1dVVrVu31iuvvCLp4nwqZbF+/Xqb5bVr16phw4by8fExdf2utxYtWmjt2rU213/v3r366aefFBgYKDc3NzVp0qTEdd+0aZNOnTplM9nt3YQ7UQAAAAAAdm3dulUzZsxQ+/bt9cUXX2jdunWKj4+XJDVp0kRhYWHGm2/uv/9+bd++XfPmzdOgQYPk6OioJ598Um+99ZaGDh2qqKgonT17VvHx8XJxcSm13wcffFBhYWF6/fXXlZ+fL4vForVr1+rjjz/WxIkT5ejoqA4dOqh58+aKjo423s6Tnp6uXbt2ad68eVd9zn5+frJarYqMjNTgwYPl4uKit99+W1WqVFFwcHCZjrVp0yZNnjxZoaGhWr9+vb744gtjwlYz1+96Gzp0qPr27avBgwdrwIABOnXqlBISElS7dm1jkt0RI0Zo+PDhGj16tHr27KlDhw4pLi5OAQEBV3z06E5HiAIAAAAAsCsqKkqbNm3S8uXLVatWLcXGxtpMlhobG6vExEQtWLBA2dnZql27tsaOHauBAwdKkipUqKC3335bU6ZM0YQJE1SlShVFRUVp1qxZdvsuPvaSJUuUk5Ojhg0batasWerevbuki3OrpKSkKDY2VvHx8Tpz5oyaNGmiBQsWXNOPfU9PT6WkpGj27NkaN26cCgoK5Ofnp8WLF5d5bpJBgwbpxx9/1PDhw1W3bl3Fx8fbTJZr7/pdb82aNdPbb7+tuLg4jRo1SpUqVVJISIiio6Pl7n7xLb2hoaFKTk5WcnKyhg8fLk9PT3Xr1k1RUVGXfezobuBgtVpLffXO7fB2nhmpOyRJ4/vd3NuJmO0c9jBGYA9jBPYwRlAaxgfsYYygNI6ODvL2di/vMu4KFotF48aNu2GBCG4e7kQBAAAAAKCMrFarCgsL7ba7EfOZFLv0dchX4uTkJAeHsr3OGldGiAIAAAAAQBlt375dzz77rN12a9euvSH9Hzx4UJ06dbLbbunSpWWevwVXRogCAAAAAEAZNW3aVCtXrrTbzsfHR3v27Lnu/fv4+Jjqv0GDBte977sZIQoAAAAAAGXk7u4uX1/fcuvf1dW1XPu/W13/9yQBAAAAAADcgQhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAAADCBEAUAAAAAAMAEQhQAAAAAAAATCFEAAAAAAABMIEQBAAAAAAAwgRAFAAAAAADABEIUAAAAAAAAEwhRAAAAAAAATCBEAQAAAAAAMIEQBQAAAAAAwARCFAAAAAAAABMIUQAAAAAAAEwgRAEAAAAA3BBWq/WatgO3GkIUAAAAAECpLBaLFi1aZLr9+fPnNXXqVK1du9ZYFxoaqsmTJ19x+9UKDw/XkCFDrvk4gBmEKAAAAACA6+ro0aNatmyZLly4YKx744039Pzzz19xO3A7cC7vAgAAAAAAd76HHnqovEsArhl3ogAAAAAAyuS7777T4MGD1bJlSzVr1kxhYWFasWKFJOngwYPq1KmTJGnUqFEKDw+X9N/Heextv9S0adMUGhpqLOfn5+vll19WcHCwgoODtWDBghK1XbhwQYmJierQoYN8fX3Vs2dPbd269fpfBNyVuBMFAAAAuAPlbFivw99mquA8j0vg8lyrecr77+PKvN8ff/yhZ599ViEhIUpMTNSFCxf0zjvv6NVXX1VAQIAaNGigN954Q5GRkRozZowRmBTz8fEpdXtpxowZo507dyo6Olqenp5KSkrS3r179cgjjxhtXnnlFa1Zs0YjR45Uo0aN9OGHH2rw4MFaunSpAgMDy3y+wKXshije3u43o45r4uLqJEmqUcPjpvddHn3i9sIYgT2MEdjDGEFpGB+4ksPfZip//365NWhQ3qXgFuXscnV/U//555/l7++v2NhYubi4SJL8/f0VFBSkzMxMWSwWNWnSRJJUr149NWrUyGZ/V1fXUrdfye7du7V+/XrFx8era9eukiQ/Pz+bEGbv3r1KT0/X1KlT1adPH0lS+/btlZWVpYSEBC1duvSqzhkoZvd/TXZ2noqKbu3XThWcL5QkZWWduqn91qjhcdP7xO2FMQJ7GCOwhzGC0jA+UJqC8xfk1qCB7hkdXd6l4Bbl6OhwVfuFhIQoJCRE586d0+7du3XgwAF99913ki6+dedG2bFjh6SLoUgxHx8f+fv7G8vbt2832lw6aW1ISIji4uJ0/vx5ubq63rAacefjcR4AAAAAgGmFhYV6/fXXlZaWpoKCAtWtW1ctW7aUJFmtN+4P8Lm5uXJxcZG7u+3TEjVq1FB+fr4kKScnR5Jt0HKpEydOqGbNmjesRtz5CFEAAAAAAKbNmzdP7777rmbMmKGQkBBVrlxZZ86c0cqVK6/52EVFRTbLp0+fNj57enqqoKBAubm5qlKlirE+JyfHeKzIw8NDDg4OWrFihZycnEocv1q1atdcI+5uvJ0HAAAAAGDazp071axZM/3P//yPKleuLEnatGmTpP/eiXK5AONSl9vu7u6uo0ePGstFRUX69ttvjeWgoCBJ0ueff26sO3nypHbu3Gkst2jRQlarVXl5efL19TX+bd26VUuWLJGzM/cR4NowggAAAAAApvn6+mrhwoVavny5GjdurH/9619KTk6Wg4ODzp49K+niHSGS9M9//lP169fXgw8+aHOMy21v3769Fi9erGXLlqlRo0ZasWKFsrOz5ebmJklq2LChunfvrunTp+vcuXO69957NX/+fJu5T5o0aaKwsDBFR0crMjJS999/v7Zv36558+Zp0KBBcnTkPgJcG0IUAAAAAIBpERERysrK0htvvKFz586pfv36mjhxolavXm3cOeLu7q7Bgwdr+fLl+vbbb7V69WqbY1xu+9ChQ5WVlaX4+Hg5Ozure/fuGjJkiJYvX27sN23aNHl5eSkpKUkFBQXq3bu3atasaYQ3khQbG6vExEQtWLBA2dn/r707D6/p3P8+/skcJBIihkbLwRFTIgmlpoSoRo01lbb46Ym5CVLUPJTW0JKhESFVohWHDinFUapoaU0V1Vlb6pS2KmLIIJppP3947CMN2StKduj7dV2uK3uv4f6ula/Y+VjrXmny8vLS+PHjFRoaWjonCPc0G5OFmX/uhqfzLEy6OkvzpKdK95nfzIgPS+gRWEKPwBJ6BMWhP1CcUy/Nl4OjPU/nwU3Z2trIw8PF8ooAzLiWCQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAADCBEAQAAAAAAMIAQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAA+ytXQAAACjq4ke7lXFgn8442is3J8/a5aCMoj9QnD9O/SyHunWsXQYA3FO4EgUAgDIo48A+/XHqZ2uXAQAAgOtwJQoAAGWU0/0PyOfFOUpNzbB2KSijPD1d6Q/c1KmX5lu7BAC453AlCgAAAAAAgAGEKAAAAAAAAAYQogAAAAAAABhAiAIAAAAAAGAAIQoAAAAAAIABhCgAAAAAAAAGEKIAAAAAAAAYQIgCAAAAAABgACEKAAAAAACAAYQoAAAAAAAABhCiAAAAAAAAGECIAgAAAAAAYAAhCgAAAAAAgAGEKAAAAAAAAAYQogAAAAAAABhAiAIAAAAAAGAAIQoAAAAAAIABhCgAAAAAAAAGEKIAAAAAAAAYQIgCAAAAAABgACEKAAAAAACAAYQoAAAAAAAABhCiAAAAAADKDJPJZO0SgJsiRAEAAAAAlAlLlizR2rVrS7TNoEGDNGLEiDtUUcl4e3vrtddes3YZuIMIUQAAAAAAZUJsbKyuXLli7TKAmyJEAQAAAAAAMIAQBQAAAABQLG9vb61bt06jRo1S06ZNFRwcrKSkpELrZGVlaeHChQoODpavr6/69u2rvXv3Flrn3XffVdeuXeXj46PAwEDNmzdPf/zxh3kMSXrppZcUHBws6er8KKtXr1b37t3l4+Mjf39/Pf300zp27NhfOp4rV65o4cKFCgwMlL+/vwYMGKDPPvusRMfy008/KTQ0VP7+/urUqZP27NlTZJy0tDQ999xzatGihfz9/TVy5EidOnXqL9UO67K3dgEAAAAA7oysn37SqZfmW7sMlFGOldzlMfU5w+svWrRIQUFBio2N1SeffKI5c+bIwcFBjz/+uAoKCjR06FCdOHFCERERqlGjht555x0NHz5cy5cvV7t27XTo0CFNnTpVY8aMUbNmzfTjjz9qwYIFcnJy0vjx47V+/Xr1799fgwYNUu/evSVJK1euVHR0tCZMmKCGDRvq9OnTioqK0pQpU5ScnHzLxz5u3DgdOnRIY8eOVb169ZSUlKRhw4Zpw4YNuv/++y0eS2ZmpgYNGqQqVaro5Zdf1oULFzR58uRCY1y5ckWDBw/WlStXNH36dJUrV07Lly/XwIED9d5778nNze2W64f1WAxRPDxcSqOOv8TB0U6S5OnpWupjW2NM3F3oEVhCj+BGzjj+759oegTFoT9wM/kPt1fqx3str4i/LXuHkv2fep06dbR48WJJUmBgoH777TctX75cjz/+uHbv3q2UlBStWLFC7dq1kyQFBQWpf//+ioqKUrt2cAjDSAAAIABJREFU7XTkyBGVK1dOoaGhcnR0VIsWLeTg4CAHBwdJkp+fnySpRo0aatSokSTpt99+0+jRo/V///d/kqQWLVooPT1d8+fPV1ZWlipUqFDi4/7uu++0a9cuLVy4UI899pgkqXnz5urVq5dSUlJ0/Phxi8eSnJysCxcu6O2331b16tUlSW5ubgoPDzePs2HDBv3000/atGmT6tatK0lq1aqVOnTooDfeeENhYWElrh3WZ/FvTVpapgoKyvYjpnJz8iVJqakZpTqup6drqY+Juws9AkvoEdxMbk6e+Wt6BDfDzxAUxy6glXxCHqFHcFO2tjYlWr9Lly6FXnfs2FHbtm3TmTNndOjQIVWoUMEcOly/zfz585WZmamAgABdvnxZPXr00KOPPqoOHTqob9++srG5eR3Tp0+XJJ0/f14nTpzQiRMntHPnTklSTk7OLYUoKSkpkmS+ZUiSHB0dtWXLFknSwoULLR5LSkqK6tevbw5Qrp0POzs78+sDBw6oVq1aqlWrlvLyrv677uzsrGbNmmn//v2EKHcpbucBAAAAAFhUtWrVQq8rV64sSbp48aLS09NVpUqVIttUqVJFJpNJWVlZat68uZYuXapVq1YpISFBS5cuVc2aNTV79uwigcU1x48f14wZM3T48GGVK1dODRo0MAcnJtOt/Wf/pUuX5ODgoIoVK95wuZFjSU9PV6VKlQott7OzM58T6ep5OXHihBo3blxkX7Vr176l2mF9hCgAAAAAAIsuXLhQ6HVaWpqkq2GKm5ubzp07V2Sb1NRUSZK7u7ukq1d/BAcHKyMjQx9//LHi4+MVERGhTz/9VI6OjoW2LSgo0KhRo+Tu7q5NmzapXr16srW1VVJSUpFJXkvC1dVVubm5ysjIkKvr/26JPHLkiCpWrGjoWNzd3XX8+PFCy00mky5dulRonAYNGuiFF14osq8/HyvuHjydBwAAAABg0e7duwu9/vDDD1WnTh1VrVpVzZo1U1ZWVpEn1GzdulWNGzeWk5OTYmNj9fjjj0u6GjB07dpVoaGhysjIUGZmpiTJ1vZ/v6KeP39e//3vf/X444+rfv365mU3egpOSfj7+0uSdu3aZX4vJydH48aN08aNGw0dS8uWLfXDDz/o5MmT5uX79u1TTk6O+XVAQIBOnz4tLy8v+fj4yMfHR02aNFFiYmKRc4m7B1eiAAAAAAAs2rNnj+bMmaPg4GDt3r1bH3zwgaKjoyVJ7du3V9OmTTVx4kTzE22Sk5N19OhRxcfHS5JatmypuLg4TZ8+XV27dtWlS5e0bNkyNWvWzHwbTMWKFXX48GE1b95cvr6+uu+++7R69Wp5eHjIzs5OGzZsMAcQ2dnZt3QcjRs3VocOHTR37lxlZmaqVq1aWrdunbKzs9W/f39Vr17d4rE89thjWrlypUaOHKmIiAhduXJFUVFR5klyJalv375644039K9//UvDhw+Xu7u71q9fr+3bt6tHjx63+m2AlXElCgAAAADAoqFDh+q///2vRo8erf379ysqKkqdO3eWdHU+kBUrVuiRRx5RVFSUwsPDdebMGSUkJKhDhw6Srj5ZJzIyUl999ZVGjRqlWbNmydfXV7GxseYxwsLCdODAAQ0bNkz5+fmKjY1VhQoVFBERoalTpyo7O1urVq2SJH3++ee3fCxRUVHq2bOn4uLiFBYWposXLyoxMVFeXl6GjsXJyUmrV69W3bp1NXnyZEVGRmrcuHGFHlvs4uKipKQk1alTR7Nnz9bo0aP166+/aunSpQoKCrrl2mFdNiYLs/HcDU/nWZh0dXblSU8FlOq4zIgPS+gRWEKP4GZOvTRfkhTw8jx6BDfFzxBYQo+gOLa2NvLwcDG0rre3t5577jmFhobe4aqAso3beQAAAAAAdz2TyaT8/HyL69nb82swbh3dAwAAAAC467377ruaMmWKxfWOHTtWCtXgXkWIAgAAAAAo1t0QPHTo0EFvv/22tcvAPY4QBQAAAABw16tUqZIqVapk7TJwj+PpPAAAAAAAAAYQogAAAAAAABjA7TzAPWz357/owNe/W7sMFMPB0U65OZZnkcffzx8FDfW07bfWLgMAAADX4UoU4B524Ovf9fPZTGuXAQAAAAD3BK5EAe5xD1R10aSnAqxdBm7C09NVqakZ1i4DZdCpl7ZZuwQAAAD8CVeiAAAAAAAAGECIAgAAAAAAYAAhCgAAAAAAgAGEKAAAAAAAAAYQogAAAAAAABhAiAIAAAAAAGAAIQoAAAAAAIABhCgAAAAAAAAGEKIAAAAAAAAYQIgCAAAAAABgACEKAAAAAACAAYQoAAAAAAAABhCiAAAAAAAAGECIAgAAAAAAYAAhCgAAAAAAgAGEKAAAAAAAAAYQogAAAAAAABhAiAIAAAAAAGAAIQoAAAAAAIABhCgAAAAAAAAGEKIAAAAAAAAYQIgCAAAAAABgACEKAAAAAACAAYQoAAAAAAAABhCiAAAAAAAAGECIAgAAAAAAYAAhCgAAAAAAgAGEKAAAAAAAAAYQogAAAAAAABhAiAIAAAAAAGAAIQoAAAAAAIABhCgAAAAAAAAGEKIAAAAAAAAYQIgCAAAAAABgACEKAAAAAACAAYQoAAAAAAAABhCiAAAAAAAAGECIAgAAAADAXc5kMlm7hL8FQhQAAAAAAErJgQMH5O3trS+//PKW9xEbGyt/f3/z6x07dmjWrFm3o7xCvL299dprr5lfBwcHq1WrVrpw4UKRdXfs2CFvb2+dPn1akpScnCxvb2+dP3/e0Fjz5s2Tt7e3Xn311RsuL8nY187x9X98fX3VuXNnLV68WJmZmYZquhFCFAAAAAAASknjxo21fv161a1b95b30a9fP61evdr8evXq1fr9999vR3kWnT9/XgsXLryt+8zLy9OWLVv0z3/+U2+//fZtG3v+/Plav3691q1bp/j4ePXo0UNr1qzR4MGDdfny5VuqlRAFAAAAAIBS4uLiIj8/P5UvX/6W91G9enX5+vrexqqMc3V11bvvvqtPP/30tu1z7969On/+vKZNm6aTJ0/q0KFDt2Xsf/7zn/Lz85O/v7/atGmj0aNHKy4uTt98881Nr3ixhBAFAAAAAGBRbm6uEhISFBISIh8fH3Xv3l2bNm0yL79y5YoWLlyowMBA+fv7a8CAAfrss8/My/Py8hQTE6P27dvLx8dHvXv31r59+wqNceLECY0ZM0YPPfSQmjRpouDgYMXFxZnn+7h2m8Znn32mAQMGyMfHRx07dtRbb71VomNJTk6Wv7+/9u7dq86dO8vPz08DBw7Ut99+a14nNjZWvXv31rx58xQQEKDHHntMkpSVlaWFCxcqODhYvr6+6tu3r/bu3Wve7vnnn1fjxo31/fffm9+bOXOm/P39derUqSK38wwaNEgvvviiXn75ZT300EMKCAjQrFmzdPnyZb3wwgtq3ry52rZtq+XLlxeq7drtPIMGDdLBgwe1e/dueXt766uvvlKTJk0K3YYjSSkpKfL29taxY8dKdK7+rEuXLmrYsKFmzpypK1eu/KV9XbNhwwb5+vqqVatWqlOnzk2/n7dj7NatW6tZs2Yl7plr7G9pKwAAcMf9cepnfTltpnJz8qxdCsqoM4729AeKRY+gOI6V3OUx9TnD60+aNEk7d+5UeHi4vL29tX37dk2YMEFXrlxRv379NG7cOB06dEhjx45VvXr1lJSUpGHDhmnDhg2qVauWZsyYoa1bt2rMmDGqV6+e3nvvPQ0bNkyvv/66AgIClJWVpcGDB6tOnTpauHCh7O3ttXnzZr3yyitq2LChgoODzbVEREToX//6l8aOHau1a9dq+vTp8vf3V7169QwfT05OjiZMmKDRo0erZs2aWrp0qQYPHqz3339fHh4ekqRjx47JxcVFcXFx+uOPP1RQUKChQ4fqxIkTioiIUI0aNfTOO+9o+PDhWr58udq1a6fx48dr9+7dmjNnjtasWaP9+/frzTff1OzZs3X//ffr119/LVLLO++8oxYtWigyMlIpKSmKjY3V/v375efnp1deeUWbNm1SZGSkWrRoUWguFEmaNWuWJk6cKGdnZ02aNEn169dXUFCQtmzZotDQUPN6mzZtUsOGDeXt7W34HN2Ivb295s6dq/79+ysmJkaTJk36S/vLyMjQzp079dxzV3uxZ8+eio+P14wZM+Tq6npHxm7VqpU+++wznT59WjVr1izRthZDFA8Pl1sqqjQ5ONpJkjw9XS2seftZY0zcXazZI9b8uwHj+P7gRvIfbq/Uj6/+r5aDI//ngZujP2AJPYKbsXcw3hvHjh3Tli1b9Pzzz2vAgAGSpLZt2yozM1ORkZHy8fHRrl27tHDhQvMVG82bN1evXr2UkpKivLw8JScn64UXXlC/fv0kSYGBgUpNTVV0dLRef/11/fTTT3rggQcUHR2typUrS7r6y+6OHTt06NChQiHK4MGD9fTTT0u6OsfIBx98oI8//rhEIUpeXp7Gjh2rJ554QpLk5+en4OBg/fvf/1ZYWJh5ncmTJ6tRo0aSpJ07dyolJUUrVqxQu3btJElBQUHq37+/oqKi1K5dO7m4uGj27NkaPny43n77bS1fvlxt2rQxn7cbsbe3V3R0tJydndW6dWutW7dOBQUFmj9/vmxtbdW8eXNt3LhRR48eLRKi1KtXTy4uLipfvrz8/PwkSb169dIzzzyjEydOqE6dOsrLy9PWrVs1YsQIw+enOD4+Pho0aJBWr16tbt26qXHjxre8r61bt6qgoEBdu3aVJPXo0UPR0dF677339NRTT92Rsa/1V1pa2u0PUdLSMlVQULYflZSbky9JSk3NKNVxPT1dS31M3F2s3SPW+rsB46zdIyi77AJaqXpAK3oExaI/YAk9guLY2toYXvfabTmdO3cu9H6XLl20ZcsW8xwW1wcdjo6O2rJliyTp3//+t6SrwUle3v+ujgoKClJkZKRycnLUpEkTrV27Vrm5ufrxxx918uRJffPNN8rLy1NOTk6hca+FBZJUsWJFlS9f/pYmCr32i7t09RdrPz+/QrcgSVLt2rXNXx86dEgVKlQwByjXn4f58+crMzNTLi4uCgoKUs+ePTVjxgy5uLhozZo1xdZRv359OTs7m197eHiodu3asrW9OgOHo6Ojypcvr4wMY3+fg4KC5O7uri1btig8PFx79uxRenq6unXrZmh7I8aOHasPPvhA06dPL3YyWEs2bNigli1bys7OTunp6XJxcZG/v7/efvvtG4Yot3PsW0EsDQAAAAAo1qVLl2Rvby93d/dC71epUkWSlJmZKQcHB1WsWPGG21+8eFHS1RDlRi5cuKBq1aopPj5er732mjIyMuTl5SV/f3/Z29ub50S55vrAQZJsbW2LrGOJk5NTkXorV66sn376yfy6fPnyhSaATU9PNx/z9apUqSKTyaSsrCy5uFy9m6NHjx7auHGjGjRooGrVqhVbS4UKFYq8V65cuRIdz/UcHBzUrVs3bd68WeHh4dq0aZNatWolT0/PW97nn5UvX16zZs3S8OHDtWrVqkJhk1GnTp1SSkqKTCaTHnzwwSLLv/766xteafJXx772JCNL35cbIUQBAAAAABTLzc1NeXl5unjxYqEg5dy5c5Ku/lKbm5urjIyMQvNYHDlyRBUrVpSrq6tsbGy0bt062dnZFdl/pUqVtGHDBsXExGjWrFnq1q2beT+tWrW6I8f0xx9/KDs7u1BYkZaWZr7V40bc3NzMx3y91NRUSTKfm5ycHC1YsED169fXoUOHtGHDBvNtTqWlZ8+eWrNmjb788kvt2rVLzz///G0fIygoSN26dVNsbKzGjRtX4u03btwoJycnLVu2zHzVjSTl5+dr5MiReuutt256u85fGfvAgQPy8vJS9erVS1wzT+cBAAAAABSrWbNmkqT333+/0Pv/+c9/5OHhoebNm0uSdu3aZV6Wk5OjcePGaePGjWrWrJlMJpMyMzPl4+Nj/rNv3z4lJibK3t5eR44cUfXq1fXEE0+YA5Svv/5a58+fL/FVJkZdX29aWpo+//xztWzZ8qbrN2vWTFlZWdqzZ0+h97du3arGjRvLyclJkrRs2TL9/PPPiouLU+/evTV//nylpaXdkWOQVCiAuMbX11d169bVggULJEmdOnW6I2NPnTpVzs7OWrJkSYm33bhxowIDA9WqVSu1bNnS/Kd169Zq3769Nm/eXOxTeG5l7AMHDujIkSPmuXlKiitRAAAAAADFatCggUJCQrRgwQJlZWXJ29tbH374obZs2aKZM2eqcePG6tChg+bOnavMzEzVqlVL69atU3Z2tvr37y8vLy+FhIRo4sSJCgsLU926dXXw4EHFx8dr6NChsrW1lY+Pj9atW6clS5aoRYsWOn78uOLi4mRjY3PbHqX7Z3PmzFFWVpYqV66suLg4ubm5mSeavZH27duradOmmjhxovnpPMnJyTp69Kji4+MlSd9//70SEhI0cuRIPfDAA5owYYI+/PBDzZkzRzExMXfkOCpWrKhvv/1WBw4cUNOmTc23Oz322GNavHixHnvssb90e1BxPDw8NGnSJE2ZMuWGy9evX19k7Jo1a6pSpUr6+eefFRERccPtevTooW3btun999+/6VU8lsb+4YcflJ9/dZ7IrKwsff7551q5cqV8fX3NExOXFCEKAAAAAMCiRYsWKSYmRomJibp48aLq1Kmjl19+WT169JAkRUVFafHixYqLi9Ply5fVpEkTJSYmysvLq9D2CQkJSktLk5eXl8aPH29+DG/v3r118uRJrVu3TitWrJCXl5dCQ0N1/PhxHT58+I4c0+TJkxUbG6vz58/roYce0iuvvFLksbrXs7Oz04oVK7Ro0SJFRUUpOztbDRs2VEJCggIDA5Wfn69p06bJy8tLw4cPl3R1npVnn31WM2fO1I4dO4rd/60aMmSIIiIiNHToUK1evVoBAQGSrs5Bs3jxYvXs2fO2j3m93r1767333tO+ffuKLIuOji7yXtu2beXl5SVnZ2e1b9/+hvsMDAyUm5ub3nrrrWJvhSpu7OvDFScnJ91///0aMmSIQkNDi8yrY5SNycJ1UXfD03kWJqVIkiY9FVCq4zLbOSyxdo9Y6+8GjLN2j6Dso0dQHPoDltAjKI6trY08PFysXYZVJCcna8qUKdq3b1+xc6Dc7V599VUlJSVp586dN7zlByXHlSgAAAAAgHtCQUGBCgoKil3Hxsb4o53vVtu2bdMXX3yhpKQkhYeHF5m01dIcM7a2toQuN0GIAgAAAAC4J0ydOlXvvvtuseu0aNFCvXr1KqWKrOO///2v1qxZo44dO2rw4MGFlg0ZMkQHDx4sdvtevXqZJ6RFYYQoAAAAAIB7QlhYmJ566qli16lQoYLq1Kmj3r17l1JVpW/48OHmOVn+7Pnnn1dWVlax21eqVOlOlHVPIEQBAAAAANwTatasqZo1a1q7jDKtTp061i7hrsZNTgAAAAAAAAYQogAAAAAAABhAiAIAAAAAAGAAIQoAAAAAAIABhCgAAAAAAAAGEKIAAAAAAAAYQIgCAAAAAABgACEKAAAAAACAAYQoAAAAAAAABhCiAAAAAAAAGGBv7QJul5/PZmphUkqpjungaKfcnPxSHRN3F2v3yM9nM/RAVVerjQ8AAAAA95J7IkRp2biatUsAAAAAAAD3uHsiRGnv56X2fl6lPq6np6tSUzNKfVzcPazdI6V9dRYAAAAA3MuYEwUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAADCBEAQAAAAAAMIAQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAADCBEAQAAAAAAMIAQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAoZbGxsfL397d2GYZ5e3vrtdde+0v7OHDggLy9vfXll1/edJ3k5GR5e3vr/Pnzf2msO4UQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAxfL29tayZcvUtWtX+fn56T//+Y8k6dChQ3rqqacUEBCg1q1ba86cOcrKyiq07QcffKA+ffrIz89PQUFBio6OVl5ennl5cHCwEhISNH36dDVr1kwtW7bUK6+8ooyMDE2YMEH+/v7q0KGDkpOTzdtcvnxZ06ZNU9u2beXr66tevXpp+/btJTqmyZMna8SIEVq5cqVat26t5s2ba/z48bp48WKh9T755BP169dPvr6+CgwMVExMjPLz8wvVv2jRIj3++OPy9fXVihUrSlTHf/7zH4WEhMjHx0e9e/dWSkpKoeWWzvGgQYM0YsSIQtskJibK29vb/PrEiRMaOnSomjdvroCAAIWGhuq7774rtM3mzZvVvXt3NWnSRA8//LDeeOONIrVevHhRzz77rPz9/dWyZUvNmzdPubm55uVZWVlauHChgoOD5evrq759+2rv3r3FHv+GDRsUEhIiX19fDRs2rMj5L2sIUQAAAAAAFsXHx2vw4MFasGCBWrRooY8++kiDBw+Wp6enoqKiFB4ers2bN2vEiBEqKCiQJK1fv15hYWHy9fXVkiVLNHDgQK1cuVKTJ08utO9ly5YpPz9fS5Ys0aOPPqq4uDj17dtXnp6eWrp0qf75z39q5syZ+vXXXyVJL774ovbv369p06YpISFBdevW1dixY3X8+PESHdPhw4e1du1azZgxQ9OnT9enn36qUaNGmZfv27dPw4YNU82aNbVkyRKFhoZq1apVeuGFFwrtZ9WqVerYsaNiYmIUHBxsePzs7GxFRUVpzJgxiomJUXZ2tsLDw80hk5FzbElBQYFGjRql/Px8RUVFKSoqShcuXNCIESPMYdC7776r8ePH68EHH9SyZcv02GOPaf78+UUCoRUrVqhSpUpaunSpBgwYoNWrV2vdunXmcYYOHark5GQNHz5csbGxuu+++zR8+HDt2bPnhrVt3bpVkyZNUtu2bRUXF6eaNWsqMjLS8PmzBntrFwAAAADr2vvLfn32++fWLgN3gIODnXJz8y2viL8ld+eKmhg0wvKK/1/r1q3Vv39/8+uYmBj5+voqOjra/F7NmjU1dOhQ7d6923zVSdeuXTVr1ixJUtu2beXq6qpZs2Zp6NChatCggSSpevXqmjdvnmxsbOTv76/169erWrVqmjRpkiTJy8tLnTp10jfffKP77rtPhw8fVps2bfToo49KkgICAlSlSpVCV7gYkZmZqXXr1qlevXpXz4m7u0aMGKGDBw+qRYsWio6OVtOmTRUVFSVJCgwMlJubm6ZMmaLQ0FDVrFlTklS3bt0iV4MYYTKZ9PLLL8vPz0+SlJeXp/DwcP34449q0KCBxXNsJLBJS0vTyZMnFR4ernbt2kmSatSooc2bN+vy5cuqUKGCIiMj1b17d82cOVPS1e+TjY2Nli5dqieffFLly5eXdLUHZsyYIUlq1aqVdu7cqQMHDmjQoEHavXu3UlJStGLFCvM4QUFB6t+/v6KioszvXS8hIUHt2rUz77Ndu3b67bfftGvXrhKfy9JiMUTx8HApjTruWp6ertYuAWWcNXvEwdHO6jXAMr4/sIQeQXFuR38c/fJL/ZL1m2q717wNFaGscXCws3YJKKNK2hv/+Mc/zF9nZWXpm2++MYcc17Rr105ubm46dOiQ7r//fp0/f16dO3cutM61UOWzzz4zhyg+Pj6ysbGRJDk7O6tChQpq0qSJeRt3d3dJUnp6uiSpefPmevPNN3X27Fl16NBB7du3L3J1ixHe3t7mAEW6+ku/g4ODPvvsM/n4+OiLL75QREREoXAmMDBQBQUFOnDggDlEuf7clISdnZ18fX3Nr728vCRJGRkZhs6xkRDFw8NDtWvX1owZM/Tpp58qKChIbdu21bPPPitJOn78uM6ePav27dsXOc5XXnlFX3zxhR566CFJKvI0IS8vL/P35NChQ6pQoUKRsKRLly6aP3++MjMzC72fnZ2tb7/9VlOnTi30fkhIyN0doqSlZaqgwFQatdx1PD1dlZqaYe0yUIZZu0dyc67+zxN9WnZZu0dQ9tEjKM7t6o/c3Hx5VaihZ3yG3YaqUJbwMwTFsbW1KdH6Hh4e5q8zMjJkMpkKvXdN5cqVlZmZqUuXLhXZTpJcXV3l6OhY6JfqChUqFNlPuXLlblrL9OnTVbVqVW3cuFG7du2Sra2tgoKCNG/ePFWuXNnwMXl6ehZ6bWNjI3d3d126dEnp6ekqKCjQ4sWLtXjx4iLbpqammr++0XkwwsnJSba2/5tl49rXBQUFhs6xEba2tkpMTFRsbKw+/PBDvfPOO3J2dtaAAQM0adIk8xwk48eP1/jx44tsf/1x/vl7YmtrK5Ppal6Qnp6uKlWqFNm+SpUqMplMRebKSU9Pl8lkUqVKlYqsX5ZxOw8AAAAAoERcXV1lY2OjtLS0IsvOnTsnd3d389Ujf14nPT1dOTk55uW3wtnZWWPGjNGYMWN04sQJbdu2TUuXLlVMTIyef/55w/v58ySmBQUFunDhgjw8PMzBzqhRo9SxY8ci21atWvWW6zfCyDm+5s/zo1y+fLnQ6xo1amjevHkqKCjQ559/rrfeekuJiYlq2rSp+UqcmTNnFroq5pprV9tY4ubmpnPnzhV5/1oI8+fvt5ub2w2Pj4llAQAAAAD3lAoVKqhhw4Z6//33C72/Z88eZWRkKCAgQP/4xz9UqVKlIutce7JPQEDALY2dn5+vbt26KTExUZJUp04djRo1Sn5+fvrtt98cdVtkAAAVL0lEQVRKtK/vvvtOZ86cMb/evXu38vLy1LJlS7m4uKhBgwY6deqUfHx8zH8cHBwUGRlZaLs7wcg5liQXFxedPXu20DqHDx82f/3dd9+pbdu2+vrrr2Vra6uAgAC98MILsre316+//qo6derI3d1dv//+e6HjvHjxomJiYgxf8dKsWTNlZWUVmUR269ataty4sZycnAq97+zsLD8/P+3YsaPQ+x999JGh8ayFK1EAAAAAACUWHh6u0aNHa9y4cerdu7d+++03RUZGyt/fX4GBgbKzs1NYWJjmzp0rNzc3dezYUceOHVNsbKw6d+6s+vXr39K41+YRiYuLk5OTk+rUqaOjR4/q8OHDJboKRbo6kevIkSMVFhamS5cuadGiRWrfvr2aNm0qSRozZoyeeeYZubi4qFOnTrpw4YKio6Nla2t7y/WXhKVzLF2du2T27NmKjY3Vgw8+qG3btumrr74y76NevXqqUKGCJk2apLCwMLm5uWnDhg2ysbFR+/btZW9vr/DwcC1YsEDS1QljT58+rcWLF6t27dqGr0S5dt4mTpyoiIgI1ahRQ8nJyTp69Kji4+NvenxDhw7VlClT1KVLF+3fv79IqFLWEKIAAAAAAEosODhYcXFxiouL0+jRo+Xu7q5u3bopIiJCdnZXJ60dOHCgnJ2dtXLlSr311luqWrWqnn76aY0ePfovjT19+nSVL19ey5YtU1pamry8vDRp0iT169evRPupV6+eHn30UU2dOlU2Njbq3r27JkyYYF7esWNHLV26VHFxcUpOTpaLi4tat26tCRMmFDtny+1i5Bz369dPP/30k9asWaOVK1fq4Ycf1tSpU/Xcc89Jkuzt7fXqq69q4cKFmj17ti5fvixvb28tX77cfCvPte9TYmKiVq5cKXd3d3Xu3FkRERHmCX8tsbOz04oVK7Ro0SJFRUUpOztbDRs2VEJCgjnw+bM2bdooNjZWMTEx2rx5s3x8fDRx4kTNmTPnNpy9O8PGdG0WmJtgYtmbY6IuWGLtHlmYlCJJmvTUrV0qiTvP2j2Cso8eQXFuV39EpyyTJI0LGPmX94WyhZ8hKI6trc3f+mmskydP1ldffaXNmzdbuxTcRbgSBQAAAABwz8jPz5eFawUKPRHHWuPfyRpw5xCiAAAAAADuGZ06ddIvv/xS7DphYWF3bPwhQ4bo4MGDxa7Tq1cv8xwkuLsQogAAAAAA7hnx8fHKyckpdp2qVauqWrVqd2T8559/XllZWcWuU6lSpTsyNu48QhQAAAAAwD3D29vbquPXqVPHquPjzuImLAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAADCBEAQAAAAAAMIAQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMsLd2AQDurJ/PZmphUoq1y8BNODjaKTcn39ploAyjR1Cc29UfpzNrqu6Dp29DRQAA3NsIUYB7WMvG1axdAgAAAADcMwhRgHtYez8vtffzsnYZKIanp6tSUzOsXQbKMHoExbld/RGdcvA2VAMAwL2POVEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAADCBEAQAAAAAAMIAQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAADCBEAQAAAAAAMIAQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAQJkxefJkdevWzfza29tbr732mtXrACTJ3toFAAAAAABwM+vXr9d9991X6uOOHj1aly9fLvVxUbYRogAAAAAAyiw/Pz+rjPvAAw9YZVyUbdzOAwAAAAAolre3t95++22Fh4fLz89Pbdu21dq1a/X7779r+PDh8vPzU0hIiD766KNC233yySfq16+ffH19FRgYqJiYGOXn55uX5+XladGiRWrTpo0CAgI0f/78QsuvjX397Tx79uzRwIED5e/vLx8fH/Xs2VPbt283L4+NjVXv3r21efNmhYSEyMfHR3369FFKSkqJjvn623lOnz4tb29v7dy5U6GhoWratKnatWun+Pj4QttcvHhR06ZNU+vWrRUQEKB//etfOnbsmHn5+fPnNX36dAUGBqpp06YaPHiwvvzyS/Py5ORktWzZUh9//LG6d+8uHx8f9e3bV8ePH9cHH3ygkJAQ+fv7a8SIEUpLSyt0HmNiYtS+fXv5+Piod+/e2rdvX4mOF8ZwJQoAAAB0OvNXRacss3YZuM0cHOyUm5tveUX8Lbk7V9TEoBGG158/f76eeOIJPfnkk1q7dq3mzp2rN954Qz179tSQIUMUGRmpiRMn6qOPPlK5cuW0b98+DRs2TCEhIQoPD9dPP/2kqKgoXbx4UbNmzZIkzZs3T++8844iIiJUu3ZtrVq1SocPH1bt2rVvWMMXX3yh4cOHa8CAAXrmmWeUlZWlFStWaPz48froo49UuXJlSdLJkyf1yiuvKDw8XK6urlq0aJHGjh2rXbt2yd7+1n8NnjJlip566ikNHTpUW7duVXR0tBo1aqSgoCDl5eXp6aefVmpqqp599llVq1ZN8fHxCg0N1ZYtW2Rvb68nnnhCubm5Gj9+vFxdXbVq1SoNHDhQb775pry9vSVJWVlZmj17tiIiIlSuXDnNnj1bw4cPl5OTk5599lllZ2drxowZiomJ0Zw5cyRJM2bM0NatWzVmzBjVq1dP7733noYNG6bXX39dAQEBt3y8KMpi93h4uJRGHXctT09Xa5eAMo4egSX0CCyhR1Cc29EfHeo9pL3/PXQbqkFZ5OBgZ+0SUEaVtDf8/f01YcIESVK1atW0fft2+fn5aeTIkZIkGxsbDRkyRCdPnlTDhg0VHR2tpk2bKioqSpIUGBgoNzc3TZkyRaGhoXJxcdG6des0btw4DRkyRJLUqlUrdejQ4aY1/PDDD+rUqZM5hJGk++67T7169dLRo0fN22ZlZSkxMVG+vr6SpPz8fI0ePVrfffedmjRpUqLjvt6jjz6qMWPGSJJatmypbdu26eOPP1ZQUJB2796tb775RklJSWrevLkkqVGjRurXr5+++uornThxQj///LM2bdqkevXqSZLatm2rkJAQLVmyRLGxsZKk3NxcTZgwQV26dJEkHT16VAkJCVqzZo0efPBBSVev8Dl69Kgk6fjx40pOTtYLL7ygfv36mc91amqqoqOj9frrr9/y8aIoiyFKWlqmCgpMpVHLXcfT01WpqRnWLgNlGD0CS+gRWEKPoDi3qz+aVvRTUx/rzDmAO4ufISiOra1Nida/FkhIUpUqVSSpUCDh7u4uSUpPT1d2dra++OILRUREKC8vz7xOYGCgCgoKdODAAVWpUkX5+fkKDAw0L3dyclJQUFChW1yu16dPH/Xp00eXL1/W8ePHdfLkSe3fv1+SlJOTY17P3t6+UG3Vq1eXJGVnZ5fomP/s+vlZbG1tVbVqVfPks0eOHJGrq6s5QJEkDw8P7dy5U9LVCXLr1atnDlAkydHRUZ06ddLGjRsLjePj41NoH1LRc52RcfXv9sGDByVdPbfXn+ugoCBFRkYqJydHjo6Of+m48T/czgMAAAAAsKhChQpF3itXrtwN101PT1dBQYEWL16sxYsXF1memppq/sW+UqVKhZZdC2hu5PLly5o5c6a2bt0qSfrHP/6hBg0aSJJMpv/957+jo6Nsbf83Bei1rwsKCm66byOcnZ0Lvba1tTWPe+nSJXPgcSPp6ek3PLYqVaooKyur0HslOdcXL16UpEJh1PUuXLigatWq3bQulAwhCgAAAADgtroWAowaNUodO3Yssrxq1ar6/vvvJV2dbPX6X/KvhQI3MnfuXH3yySdKSEjQgw8+KEdHR/3444/atGnTbT6CknN1ddX58+eLvL9//37VrFlTbm5uOnHiRJHlqamp5qt4bnVcGxsbrVu3TnZ2RW/R+nNIhb+Gp/MAAAAAAG4rFxcXNWjQQKdOnZKPj4/5j4ODgyIjI3XmzBn5+/vL0dGx0JN18vLy9Mknn9x0v59//rnatWunNm3amK9k2bNnj6TCV6JYg7+/v9LT0ws9BejSpUsaNmyYPvnkEzVr1kw//vijjh8/bl6ek5OjHTt2/KXJX5s1ayaTyaTMzMxC53rfvn1KTEz8SxPpoijOJgAAAADgthszZoyeeeYZubi4qFOnTrpw4YKio6Nla2ur+vXrq1y5cgoNDdWrr74qZ2dnNWzYUP/+97917tw5PfDAAzfcp4+Pj3bu3Kl3331XNWrU0P79+82PP75y5UppHl4RHTp0UKNGjRQREaGIiAhVqlRJr776qqpWraouXbrIxsZGq1ev1rBhwzRu3Di5uroqMTFR586dM0/OeysaNmyokJAQTZw4UWFhYapbt64OHjyo+Ph4DR06tNBtTfjrCFEAAAAAALddx44dtXTpUsXFxSk5OVkuLi5q3bq1JkyYYJ7fY+zYsXJ2dlZSUpLS09P1yCOP6PHHHzdPFvtnkydP1pUrVzRv3jxJUt26dbVkyRLNmzdPR44cUa9evUrt+P7MwcFBr732ml566SXNmzdPBQUFat68uRITE+XqevVJaklJSVq4cKHmzJmj/Px8+fn5KSkpSY0aNfpLYy9atEgxMTFKSEhQWlqavLy8NH78eIWGht6OQ8N1bEwWrnni6Tw3x2znsIQegSX0CCyhR1Ac+gOW0CMojq2tjTw8XKxdBnBX4UoUAAAAAMDfRn5+vsX5U2xtbbkNBjdEiAIAAAAA+Nvo1KmTfvnll2LXCQsLU3h4eClVhLsJIQoAAAAA4G8jPj5eOTk5xa5TtWrVUqoGdxtCFAAAAADA34a3t7e1S8BdjJu8AAAAAAAADCBEAQAAAAAAMIAQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAwgRAEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAADCBEAQAAAAAAMIAQBQAAAAAAwABCFAAAAAAAAAMIUQAAAAAAAAywt7SCra1NadRx1+L8wBJ6BJbQI7CEHkFx6A9YQo/gZugNoORsTCaTydpFAAAAAAAAlHXczgMAAAAAAGAAIQoAAAAAAIABhCgAAAAAAAAGEKIAAAAAAAAYQIgCAAAAAABgACEKAAAAAACAAYQoAAAAAAAABhCiAAAAAAAAGECIAgAAAAAAYAAhikGZmZnq1q2bTp8+XWTZjh071LNnT/Xo0UOjR4/WpUuXrFAhrK24Hvn222/Vp08fhYSEaNq0acrLy7NChbCmTZs2qUuXLurUqZOSkpKKLP/666/Vp08f9ejRQyNGjFB6eroVqoS1WOqPEydOaNCgQerRo4dCQ0P5d+ZvyFKPXLN7924FBweXYmUoKyz1CJ9XYalH+LwKGGSCRZ9//rmpW7dupsaNG5tOnTpVaFlGRoapTZs2pjNnzphMJpMpOjraNHfuXGuUCSsqrkdMJpOpa9eupiNHjphMJpNpypQppqSkpNIuEVZ05swZU4cOHUwXLlwwZWVlmbp372764YcfCq3zxBNPmHbv3m0ymUym+fPnmyIjI61RKqzAUn8UFBSYHnnkEdNHH31kMplMppdfftn00ksvWatcWIGRnyEmk8mUmppq6ty5s6lDhw5WqBLWZKlH+LwKIz9H+LwKGMOVKAa8+eabmjVrlqpWrVpkWW5urmbPnq1q1apJkry9vfXbb7+VdomwsuJ65JdfftGVK1fk5+cnSerdu7fef//90i4RVvTpp5/qoYcekru7u8qXL6+QkJAiPVBQUKCsrCxJUnZ2tpydna1RKqzAUn98/fXXKl++vAIDAyVJI0eO1FNPPWWtcmEFRn6GSNL06dMVFhZmhQphbZZ6hM+rsNQjfF4FjCNEMeDFF19U8+bNb7isUqVKevjhhyVJV65cUUJCgvk1/j6K65GzZ8/K09PT/NrT01O///57aZWGMuDPPVC1atUiPTB58mRNmzZNbdu21aeffqoBAwaUdpmwEkv98fPPP6tKlSqaNGmSunfvrlmzZql8+fLWKBVWYuRnyOuvv65GjRqpadOmpV0eygBLPcLnVVjqET6vAsYRotwmGRkZGjZsmBo0aKBevXpZuxyUISaTqch7NjY2VqgE1mKpB65cuaJp06Zp9erV2rt3r5588klNmjSpNEuEFVnqj7y8PB08eFADBw7Upk2bdP/992vBggWlWSKszFKPfP/999q+fbtGjx5dmmWhDDH6WYPPq39flnqEz6uAcYQot8HZs2f15JNPqkGDBnrxxRetXQ7KmGrVquncuXPm16mpqTe87Qf3rj/3wNmzZwv1wPfffy8nJyf5+vpKkvr376+DBw+Wep2wDkv94enpqVq1asnHx0eS1K1bN33xxRelXiesx1KPvP/++0pNTVWfPn00fPhw8+cS/H1Y6pFr7/F59e/LUo/weRUwjhDlL8rPz9fIkSP16KOPatq0aSS2KMLLy0tOTk46fPiwJGnDhg3muQ3w99C6dWvt27dP58+fV3Z2trZv316oB2rVqqUzZ87oxIkTkqQPP/zQ/Asz7n2W+sPf31/nz5/Xd999J0nauXOnGjdubK1yYQWWemTMmDHatm2bNm7cqISEBFWtWlVr1661YsUobZZ6hM+rsNQjfF4FjLO3dgF3q2HDhmnMmDE6c+aMvvnmG+Xn52vbtm2SpCZNmpDww9wjPj4+WrRokaZPn66srCw1atRIgwcPtnZ5KEXVqlVTRESEBg8erNzcXPXt21e+vr6FemT+/PkaN26cTCaTPDw8NG/ePGuXjVJipD/i4uI0ffp0ZWdnq3r16nrppZesXTZKkZEewd+bpR7h8yqM/Bzh8ypgjI3pRjfAAQAAAAAAoBBu5wEAAAAAADCAEAUAAAAAAMAAQhQAAAAAAAADCFEAAAAAAAAMIEQBAAAAAAAwgBAFAAAAAADAAEIUAAAAAAAAAwhRAAAAAAAADPh/4PZENratMwgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_dendogram(X_keep)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the plot we see that quantities like `latitude` and `postal_code` are grouped together and similar (as we might expect). Note that we used Spearman's rank correlation coefficient to calculate notions of similarity - this is useful for finding non-linear correlations that may be missed by Pearson's correlation coefficient.\n", "\n", "To examine these correlations a bit deeper, let's create a function that trains a Random Forest on subsets of the data where one of the columns is removed and see in which cases the OOB score does not get worse:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def get_oob(df):\n", " model = RandomForestRegressor(\n", " n_estimators=40, max_features='sqrt', n_jobs=-1, oob_score=True, random_state=42\n", " )\n", " X, _ = train_test_split(df, test_size=0.2, random_state=42)\n", " model.fit(X, y_train)\n", " return model.oob_score_" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8021265907435767" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# calculate reference value\n", "get_oob(X_keep)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "longitude 0.7728875237498377\n", "population_per_household 0.7901969073954147\n", "housing_median_age 0.7921744780471269\n", "bedrooms_per_room 0.8033772050439794\n", "latitude 0.7764470270681797\n", "postal_code 0.7817819559028422\n", "rooms_per_household 0.8039031326542978\n", "median_income 0.7947438916177811\n" ] } ], "source": [ "for column in (\n", " \"longitude\",\n", " \"population_per_household\",\n", " \"housing_median_age\",\n", " \"bedrooms_per_room\",\n", " \"latitude\",\n", " \"postal_code\",\n", " \"rooms_per_household\",\n", " \"median_income\",\n", "):\n", " print(column, get_oob(X_keep.drop(column, axis=1)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we're looking for columns where the OOB score did not drop much, say around the third decimal place. Let's see what happens to our OOB score when we drop these candidate columns:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.805703437136301" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "cols_to_drop = [\n", " 'bedrooms_per_room',\n", " \"rooms_per_household\",\n", "]\n", "get_oob(X_keep.drop(cols_to_drop, axis=1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The OOB score increases fractionally which is good since we're looking for a simpler model - let's drop these columns and run the full model again:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "X_keep.drop(cols_to_drop, axis=1, inplace=True)\n", "X_train, X_valid = train_test_split(X_keep, test_size=0.2, random_state=42)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RMSE on train: 15876.383\n", "R^2 on train: 0.973\n", "RMSE on valid: 41855.649\n", "R^2 on valid: 0.818\n", "OOB R^2: 0.806\n" ] } ], "source": [ "model = RandomForestRegressor(n_estimators=40, max_features='sqrt', n_jobs=-1, oob_score=True, random_state=42)\n", "model.fit(X_train, y_train)\n", "print_rf_scores(model)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfoAAAEJCAYAAAB8E+X+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deVSV1f7H8fcBQSVkCNAyrGtmWpqaZWKiXdRQc0Ahy0ywrtmoNklCKpFDJWgWUHm9+fOWaUSJCFo0SGk5oOVQlpLliAM4ICLIeJ7fHy7PDRU9Kgjn9Hmt5VrwDPv5nE2t79n7POfZJsMwDERERMQuOdR2ABEREak5KvQiIiJ2TIVeRETEjqnQi4iI2DEVehERETumQi8iImLHVOhFRETsWL3aDiACkJdXiNlse4908PJy5ciRE7Ud46LZam5Q9tqi7LWjquwODiY8Pa+yqg0VeqkTzGbDJgs9oNy1QNlrh7LXjsvNbtKT8URERK684pIyCo4Xn/cYH59GHDpUcNZ2BwcTXl6uVl1HI3qpE8a+nsLhvMLajiEicsUsjHmYAs5f6KuDbsYTERGxYyr0IiIidkyFXkRExI6p0IuIiNgxFXoRERE7pkIvIiJix1ToRURE7JgKvYiIiB1Tob/CMjMzCQ0NBWDChAn88ssv1dr+qFGjyMnJqdY2RUTEdunJeLVo2rRp1d7mf/7zn2pvU0REbJcKvZUyMzOZPXs2hmGwZ88eevfuTaNGjfjmm28AmDNnDr/99htxcXGUl5fj6+vLlClT8PT05IcffuD111+nfv36NG/e3NJmaGgoo0eP5o477iA6Oprt27dz+PBhmjdvTkJCAocPH2b06NG0bNmSrVu34uXlxdtvv42Hh0eVOXv06MGHH37IunXr+P7778nPz2fv3r107dqV6OhoDMNgxowZfPPNNzg6OvLggw8yYsQIdu7cSVRUFMeOHcPFxYUJEybQrl07IiIiaNiwIT/99BMFBQW8/PLLLFmyhG3bttGrVy8iIiKoqKggJiaGdevWUVFRQXBwMI888khN/0lERMQKmrq/CJs3b+b1119n2bJlJCYmcvXVV5OcnEyrVq1ITExk5syZzJ07l5SUFPz9/ZkxYwalpaVEREQQFxdHcnIyDRo0OKvdjRs34uTkxCeffMLXX39NSUkJK1asAGDbtm08+uijLF26FDc3N9LS0qzOu3HjRuLi4khNTeXbb78lKyuL9PR0NmzYQFpaGp9++inJyckcOnSI8PBwQkNDSUtLIzIykmeffZbS0lIAcnNzSU1NZezYsURGRvLqq6+SkpJCUlISBQUFJCUlAbB48WI+++wzli9fzo8//lgNPS4iIpdLI/qLcPPNN3PttdcC4OnpSZcuXQBo2rQpGRkZHDhwgLCwMADMZjPu7u5kZWXRuHFjWrRoAcDgwYN5++23K7XbqVMnPDw8WLBgATt27GDXrl0UFRUB4OXlxa233gpAy5Ytyc/Ptzrv7bffjqvrqdWNmjVrRn5+PuvXr6dv3744Ozvj7OzMkiVLKCwsZM+ePQQGBgLQoUMH3N3d2bFjBwDdu3e3vM6WLVvi5eUFgIeHB/n5+axZs4atW7eydu1aAIqKisjKyuLOO++8mO4VEZEaoEJ/EZycnCr97ujoaPnZbDbTsWNHZs+eDUBJSQmFhYXs378fs9l8znNOW758OXFxcYSFhREcHExeXh6nVw+uX7++5TiTycTFrCp8rnPr1av8J8/Ozsbd3f2sdg3DoKKi4qzXfeb5ABUVFYSHh1veKBw9ehQXFxerc4qISM3R1H01adeuHZs2bWLnzp0AvPvuu8TExNCqVSuOHDnCtm3bAFi2bNlZ565Zs4a+ffsSEhKCt7c369evtxTZ6tapUye+/vprysrKOHnyJI899hiHDx+mWbNmfPXVVwBs2rSJw4cP07JlS6va9PPzIykpibKyMgoLCxk2bBibN2+ukfwiInJxNKKvJj4+Prz22ms899xzmM1mmjRpQmxsLE5OTrz55puEh4dTr149yzT8Xw0ZMoRx48aRnp6Os7MzHTp0IDs7u0Zy3nvvvWzZsoXg4GDMZjNhYWE0b96c2NhYoqOjiY+Px8nJifj4eJydna1qc+jQoezevZvBgwdTXl5OcHAwnTt3rpH8IiJycUzGxcwFi9SQsa+ncDivsLZjiIhcMQtjHubQoYLzHuPj0+icxzg4mPDycrXqOhrR26DQ0FCOHz9+1vahQ4fy0EMP1UIiERGpq1TobdD8+fNrO4KIiNgI3YwnIiJix1ToRURE7JgKvYiIiB1ToRcREbFj+nqdiIhILSguKaPgePF5j9HX68RuHDlyArPZ9t5zVvU/YV1nq7lB2WuLstsuTd2LiIjYMRV6ERERO6ZCLyIiYsdU6EVEROyYbsaTOsHau0frIh+fRrUd4ZLU9dzW3JEsIhemQi91glavkzMtjHmYAlToRS6Xpu5FRETsmAq9iIiIHVOhFxERsWMq9CIiInZMhV5ERMSOqdCLiIjYMRV6ERERO6ZCX03efvttli9fbvXxo0aNIicnh7179/Lyyy9f0jWTk5OJiIgAID4+nsDAQIqL//e948zMTEJDQwGIiIggOTm5yramT5+On58fpaWllm3Z2dm0atWKVatWVTq2R48eZGdnk52dTdu2bQkKCiIoKIjevXszduxYDh8+fEmvR0REqp8KfTV59tln6dmzp9XH/+c//6FJkybs37+fvXv3VkuG/fv38+abb170eeXl5XzxxRfcfvvtpKenV9rn5OTEpEmTOHHixDnPbdy4MUuWLGHJkiWkp6dzww03MHbs2EvKLyIi1e+KFfrZs2dz3333MWDAAN544w0qKir473//S+/evbnvvvuIjY0F4PDhwzz99NMEBwcTEhLC6tWrAcjJyWHkyJE88MADBAQEMGPGDODUqPb555/nX//6F/feey/R0dHnzZGdnc2AAQN4+umn6devH6NGjeLYsWMA+Pn5MXLkSIKCgigrKztn5uXLlxMYGEhJSQm7d++me/fuHDx40DJizs7OJigoiNGjRxMYGMgLL7xAYmIiDz74IH369OHPP/8E/jcqnjp1Klu2bOHVV18lPDycTz75xJI1NDSUzZs3W93HQ4cO5fPPP+fHH3+0+hyAFStW0KxZMwYNGlTp+nCqkN99991Mnz79gu2YTCbGjBnD9u3b2bZt20VlEBGRmnFFCv2KFSvIyMggOTmZxYsXs3v3bt577z0WLlzIZ599RmpqKr/++itbtmxh2rRphISEkJyczHvvvUdUVBQnTpxg6dKl9O/fn6SkJFJTU1m4cCFHjx4FYOPGjcTFxZGamsq3335LVlbWefP8/vvvjBgxgmXLltGiRQsSEhIAyMvL4/HHH2fJkiWsXr36rMyJiYn07NmT22+/nffee4/IyEjGjx/PNddcU6n9rKwsnn76adLT0/nll1/Yt28fn3zyCf379z+rkE6cOJG2bdvyyiuvEBISQmpqKgD79u3j6NGjtG/f3up+dnd3Jzo6mgkTJlSawr+Q5ORk+vTpwz333MPWrVv5448/Ku2PiIjghx9+OGsK/1ycnZ254YYb2LFjh9XXFxGRmnNFCv3atWvp168fDRo0oF69eoSEhJCQkEBAQACNGjWiXr16/Pe//6Vt27asXr2auLg4goKCGDVqFOXl5ezdu5eRI0dy7bXXMnfuXKZNm0ZZWRknT54E4Pbbb8fV1ZWGDRvSrFkz8vPzz5vnH//4B507dwZg0KBBrF271rLvdGE9V+Y1a9YAMGHCBD777DO8vb3p16/fWe17e3tz66234uDgwDXXXEOXLl0AaNq0KcePH68yV+fOncnNzSU7O5uUlBSCgoIuopdP6dWrF23btrV6Cv/o0aP88MMP9O3blwYNGhAQEEBiYmKlY1xdXZkyZcp5p/D/ymQy0aBBg4vOLiIi1e+KLGpjNpvP2mYYRqXfc3JyaNiwIWazmQ8++AAPDw/Ldm9vb9544w327t1L//796dWrF6tXr7a0Ub9+fUs7JpPprLbPVK/e/162YRg4Ojpafj9doM6Vuby8HDj18YKjoyM7d+6ktLQUZ2fnSsed+ftf2z8fk8nEoEGDWLZsGenp6bz//vtWnXemSZMm0b9/f0sfnk9qaiqGYXD//fcDUFxcTFlZGePGjat0nL+/v1VT+KWlpezcuZObbrrpkrKLiEj1uiIjej8/P5YtW0ZxcTHl5eUsWrSI8ePHs3LlSgoLCykvL+fFF19ky5Yt+Pn5sXDhQgD++OMPBg4cyMmTJ1m1ahUjR46kb9++HDhwgJycnHMWY2vs3LmTrVu3ArBo0SK6d+9uVWY/Pz8qKiqIjIxkwoQJdOrUibfeeuvSO4ZTbwJOv4EACA4OJjExkWuuuYYmTZpcUpseHh5ER0fz7rvvXvDYRYsW8cYbb5CRkUFGRgY//PAD7u7ufP7552cde3oKPzc395xtmc1m4uPjad++Pddff/0lZRcRkep1RUb0AQEBbN26lZCQEMrLy+nWrRthYWHUr1+foUOHYjabuffee7n77rtp0aIFUVFRDBgwAICYmBhcXV154okneOmll3Bzc8PLy4u2bduSnZ19SXnc3d2Ji4tjz549tGrViqlTp1qVefjw4fzf//0fXl5eBAYGcvfdd9O/f38CAwMvuW9atGhBQUEB4eHhxMbGcu2113LttdcyePDgS24TTk3h9+7du1JRfuWVV5gyZYrl93//+9/k5eVx7733WrY5ODgwYsQIEhMTueuuuyq1eXoKf+TIkZZtubm5lo8YzGYzt9xyCzNnzrys7CIiUn1MxoXmue1MdnY2YWFhZGRk1HaUsxiGQW5uLqGhoSxduvSsjwDsmdajlzMtjHmYQ4cKztru49PonNttgbLXDnvM7uBgwsvL1ao2rsiI/krbs2cPY8aMOee+c43e64ovv/yS6OhooqOjLUV++vTplq8Y/lXbtm2ZNm3alY4oIiI2xi4L/fXXX8+SJUuq3F8XR/MAffr0oU+fPpW2jR8/vpbSiIiIPdCT8UREROyYCr2IiIgdU6EXERGxYyr0IiIidkyFXkRExI7Z5V33YnviIgfVdgSpY4pLymo7gohdUKGXOuHIkROYzbb37CZbfRCHreYWkYunqXsRERE7pkIvIiJix1ToRURE7JgKvYiIiB3TzXhSJ1i7ClNd5OPTqLYjXJIrmbu4pIyC48VX7Hoi8j8q9FInaJla+7Yw5mEKUKEXqQ2auhcREbFjKvQiIiJ2TIVeRETEjqnQi4iI2DEVehERETumQi8iImLHVOhFRETsmAq9jcrIyGDevHnnPSY5OZmIiIgayxAfH098fHyNtS8iIpdPD8yxUb/++mttRxARERugQl8LMjMziY+Pp169ehw4cIB27doxbdo00tLSmDdvHiaTiTZt2jBp0iScnZ15+eWX2b59OwDDhg2jY8eOJCYmAtC0aVP8/f15+eWXKSgo4NChQ/Tr149x48ZZlWXr1q1ERUVRXFyMu7s7M2bM4JprrmH27Nmkpqbi6OhI165dCQ8Px9HRkffff5+kpCQ8PT1xc3OjXbt2AKxcuZK4uDjKy8vx9fVlypQpeHp61kwHioiI1TR1X0t+/vlnoqKiSE9Pp6SkhDlz5jB79mzmz59PWloaDRs2JCEhgY0bN5Kfn09KSgrz5s1jw4YN3HTTTQwdOpShQ4cSEhLC0qVL6d+/P0lJSaSmprJw4UKOHj1qVY5x48bx9NNPk5aWxn333ccHH3zAihUryMjIIDk5mcWLF7N7924SExP55ZdfWLRoEYsXL2bevHkcPHgQgKNHjzJz5kzmzp1LSkoK/v7+zJgxoya7T0RErKQRfS3p1KkTN954IwBBQUGMGTOG4cOHW0bBDz74IJGRkTz++OPs3LmTkSNH0r1793OO1EeOHMnatWuZO3cu27dvp6ysjJMnT14ww9GjRzl06BABAQHAqdkCgOnTp9OvXz8aNGgAQEhICCkpKRQXF3PPPfdw1VVXAdCnTx/MZjObN2/mwIEDhIWFAWA2m3F3d7/MHhIRkeqgQl9LHB0dLT8bhoHZbK603zAMysvL8fT0ZNmyZaxatYoVK1YwePBgli1bVunYN954g71799K/f3969erF6tWrMQzjghmcnJwq/V5SUkJubu5ZWQDKy8sxmUyV9tWrV4/S0lIqKiro2LEjs2fPtrRTWKgFakRE6gJN3deSn376iZycHMxmMykpKURGRpKRkcGxY8cASEpKonPnzixfvpxx48bxz3/+k4kTJ+Li4sKBAwdwdHSkvLwcgFWrVjFy5Ej69u3LgQMHLO1eSKNGjbjmmmtYtWoVAEuWLOHtt9/Gz8+PZcuWUVxcTHl5OYsWLcLPz48uXbrw3XffUVBQQElJCV9//TUA7du3Z9OmTezcuROAd999l5iYmJroNhERuUga0deSxo0b89JLL5GTk0PXrl0ZPnw4Li4uhIaGUlZWRps2bXj11VepX78+X375Jf369aN+/foEBgbSqlUrjh8/zvjx4/H29uaJJ57gpZdews3NDS8vL9q2bUt2drZVOWJjY4mOjiYmJgZPT09iYmJo3LgxW7duJSQkhPLycrp168bw4cOpV68eI0aM4P7778fNzY2mTZsC4OPjw2uvvcZzzz2H2WymSZMmxMbG1mT3iYiIlUyGNXO8Uq0yMzNJSEhg/vz5tR2lztB69PZtYczDHDpUUC1t+fg0qra2rjRlrx32mN3BwYSXl6tVbWhE/zfw4osv8scff5y1vUePHjz77LO1kEhERK4UFfpa0LlzZzp37nzFrjdz5swrdi0REalbdDOeiIiIHVOhFxERsWMq9CIiInZMhV5ERMSOqdCLiIjYMd11L3VCXOSg2o4gNai4pKy2I4j8bVlV6E+ePEl6ejr5+fmVnqH+6KOP1lgw+Xs5cuQEZrPtPbvJVh/EYau5ReTiWVXon3/+eXJzc7n55psxmUw1nUlERESqiVWFfseOHXz++efUq6eZfhEREVti1c1411xzTU3nEBERkRpg1RD95ptvJiwsjG7dutGgQQPLdn1GLyIiUrdZVegLCwu54YYb2LNnT03nkb8pa1dhqot8fBrVdoRLUl25i0vKKDheXC1tiUj1s6rQv/766zWdQ/7mtEyt7VoY8zAFqNCL1FVWFfrMzEzmzJlDfn5+pe2fffZZjYQSERGR6mFVoZ84cSKhoaFcf/31NZ1HREREqpFVhd7Ly4uwsLCaziIiIiLVzKpC36NHDxYsWEC3bt0qfZe+adOmNRZMRERELp9Vhf7o0aO8+eabNGzY0LLNZDKxYcOGGgsmIiIil8+qQp+ens4PP/yAt7d3TecRERGRamTVk/G8vLy4+uqrazqLiIiIVDOrCv1tt93GsGHD+Pe//828efMs/2pTaGgomZmZFzzmtKCgoJqOVG1atWp1xa+ZnZ1Njx49LuqcHj16kJ2dfdZ2a/42IiJyZVg1dV9SUkLz5s3ZtWtXDcepXuvWrbP8vGTJklpMIiIiUjtq7Ml4mZmZxMfHU69ePQ4cOEC7du2YNm0aaWlpzJs3D5PJRJs2bZg0aRJXXXUVfn5+BAQEsGXLFq666ipmzJiBr68vPXr04MMPP8TX15fMzEwSEhKYP3++5Trl5eVER0ezfft2Dh8+TPPmzUlISGDGjBkADBkyhE8//ZRWrVqRlZXFyZMnmThxIllZWZhMJkaOHMmgQYNITk7m+++/Jz8/n71799K1a1eio6OrfH3Z2dk89dRTNGvWjN27d9O0aVNiY2Px8PBg5cqVxMXFUV5ejq+vL1OmTMHT05MePXrQrl07tm7dysKFC/Hy8qqy/aioKDZt2gRAfHw8N9xwA5s2bWLatGmUlJTg6enJ5MmTueGGGwgNDWX06NF07tyZ7OxswsLCyMjIIC0tjffffx9HR0d8fX2JjY2lfv36zJkzhy+++IKKigr8/f0JDw8HoLi4mOeff57t27fj5ubGO++8g6enJ99++y1vvfUWZrOZZs2aMXny5Er3a5SWljJhwgS2bNnCddddR15e3kX/9yIiIjXDqqn7J5988pz/LuTnn38mKiqK9PR0SkpKmDNnDrNnz2b+/PmkpaXRsGFDEhISAMjLy+Ouu+4iLS2Nfv36MXXqVKtewMaNG3FycuKTTz7h66+/pqSkhBUrVjBx4kQAPv3000rHx8fH4+npydKlS/nggw+Ij49n27Ztlrbi4uJITU3l22+/JSsr67zX/v333xkxYgTLli2jRYsWJCQkcPToUWbOnMncuXNJSUnB39/f8qYDoHv37nz55ZfnLfIAd999N6mpqXTt2pXExERKS0t54YUXmDRpEqmpqQwdOpQXXnjhvG289dZb/N///R/Jyck0b96cHTt2sHLlSrZs2cJnn31GSkoKOTk5pKamAqe+XfHoo4+ydOlSvL29+fzzzzly5AhRUVG88847pKWl0bFjRyZPnlzpOqffeH3xxRdMnDhRayKIiNQhVo3oe/fubfm5rKyMjIwMqz5H7tSpEzfeeCNw6jPyMWPGMHz4cDw9PQF48MEHiYyMBKB+/foMGjQIgMGDB/Pmm29a9QI6deqEh4cHCxYsYMeOHezatYuioqIqj1+7di2vvfYaAFdffTU9e/Zk3bp1uLq6cvvtt+PqempxlWbNmp31yN8z/eMf/6Bz584ADBo0iHHjxtG1a1cOHDhgecCQ2WzG3d3dck779u2tel29evUC4KabbuLHH39k165duLm50a5dOwD69u1LVFQUBQUFVbYREBDAQw89RM+ePenduze33HILqamp/PzzzwQHBwOnRvFNmzbljjvuoHHjxpb2b7rpJvLy8vj5559p164dvr6+wKm/2Zw5cypdZ926dTz44IOWPrn99tuteo0iIlLzrCr0gwcPrvR7cHAww4cPv+B5jo6Olp8Nw8BsNlfabxgG5eXlADg4OGAymYBTxfHMcwHLsX+1fPly4uLiCAsLIzg4mLy8PMvx53LmPsMwqKioAE692TjNZDKdtx2g0sODDMPA0dGRiooKOnbsyOzZs4FT9zcUFv5vsZa/XsOatk/nOLPvzsx+rj6aOHEi27ZtY8WKFYSHhzN69GgqKioYMWKEZYnh48eP4+joSF5eXqXXU9V1//o3++uxfz3ur+2IiEjtsmrq/kxms5nc3NwLHvfTTz+Rk5OD2WwmJSWFyMhIMjIyOHbsGABJSUmWEfHJkyfJyMgAIDk5me7duwPg6enJH3/8AZwq6mdas2YNffv2JSQkBG9vb9avX28pfo6OjmcVJT8/P8tiPEePHmX58uXcddddl9IN7Ny5k61btwKwaNEiunfvTvv27dm0aRM7d+4E4N133yUmJuaS2v+rG2+8kWPHjvHzzz8D8Pnnn9O0aVM8PDwq9dE333wDnCr4gYGBeHp68sQTTxAUFMTWrVvx8/NjyZIlFBYWUl5ezjPPPMOXX35Z5XXbt2/P5s2bLXfXf/LJJ5a/2WldunRh6dKlmM1m9u3bpwcpiYjUIVYNvc78PP7333+3qjg2btyYl156iZycHLp27crw4cNxcXEhNDSUsrIy2rRpw6uvvmo5Pj09nVmzZtG4cWOmT58OwNixY5kyZQoJCQn4+/ufdY0hQ4Ywbtw40tPTcXZ2pkOHDpai1LNnT4KCgkhOTrYc/8wzzxAdHc2AAQOoqKjgySefpE2bNhf8PP5c3N3diYuLY8+ePbRq1YqpU6fi4uLCa6+9xnPPPYfZbKZJkybExsZedNtncnZ2ZtasWUyZMoWTJ0/i7u7OrFmzAHjssceIiIhg0aJF9OzZEzg1qh47diyPPvooDRo0wM3NjenTp9OkSRO2bdvGAw88QEVFBd26dWPw4MHs27fvnNf19vZm8uTJjB49mrKyMpo2bcq0adMqHTNs2DC2b99O3759ue6667j55psv+/WKiEj1MBkXmp8GFi9e/L8TTCY8PT3x9/evNL1+pnPdIX8+p++KtxV/vbtdLp/Wo7ddC2Me5tChqu8VqW4+Po2u6PWqk7LXDnvM7uBgwsvL1ao2zjuiPz3FHhAQcNa+goICPDw8rLqIrdqzZw9jxow55z5rvxVwLsXFxZab1840duxYy6hcRETkcp230Pv5+VlukDtz4G8ymSyfT59L586dz/os93zq4mj++uuvP++Ddi51NN+gQQM9wEdERK6I8xb6098vFxEREdtk1c14ZrOZuXPnsnLlSsrLy+natStPPvmkvkYlIiJSx1n19bqZM2eydu1ay/evN27caLkrXkREROouq4bk33//PYsWLcLJyQmAf/7znwwcOLBGg4mIiMjls2pEbxiGpcjDqe90//V3ERERqZusGtG3bt2a1157zfLY248++kgPRZFqFRc5qLYjyCUqLimr7Qgich5WFfpXXnmFqVOnMnToUMxmM926dWPSpEk1nU3+Ro4cOYHZfMFnN9U5tvogDlvNLSIX77xT96WlpYwfP561a9fyxhtvsHr1atq1a4ejo6NllTcRERGpu85b6OPi4jhx4kSlZUenTJnC8ePHiY+Pr/FwIiIicnnOW+i/++47Zs6ciZeXl2VbkyZNiImJsaySJiIiInXXeQu9k5MTDRo0OGu7q6srzs7ONRZKREREqsd5b8ZzcHDgxIkTZ30ef+LEibPWeRe5HNauwlQX+fg0qu0Il+RicheXlFFwvLgG04hITTlvoe/fvz8TJ07ktddew8XFBYCioiImTpxIYGDgFQkofw9aprZuWxjzMAWo0IvYovNO3Y8YMYJGjRrRtWtXHnjgAe6//366du2Km5sbzzzzzJXKKCIiIpfoglP3U6ZM4YknnuC3337DwcGB2267jSZNmlypfCIiInIZrHpgjq+vL76+vjWdRURERKqZVc+6FxEREdukQi8iImLHVOhFRETsmAq9iIiIHVOhtyGZmZmEhoZWW3vLly/n7bffBk6ta/Djjz9e1PmtWrWqtiwiIlIzVOj/xnr27Mmzzz4LwPr166moqKjlRCIiUt2s+nqd1C07d+4kKiqKY8eO4eLiwoQJE2jXrh0RERG4urry66+/kpOTwzPPPENISAgFBQW89NJL7Nmzh2bNmnHw4EESEo3trXUAABeLSURBVBJYt24d69atw8/Pjy1btjBx4kQSEhKYOnUqo0ePpnPnzmRnZxMWFkZGRgbZ2dmEh4dTVFRE+/btLXkKCwuZPHky27dvp6KiglGjRtG/f/9a7CERETlNI3obFB4eTmhoKGlpaURGRvLss89SWloKwMGDB1m4cCHvvfceMTExALzzzjs0b96cZcuW8cwzz5CVlVWpvUGDBtG2bVumTp163un4KVOmEBwczJIlS+jYsaNl+3vvvUebNm1ITk5mwYIFzJ49m71799bAKxcRkYulQm9jCgsL2bNnj2WtgQ4dOuDu7s6OHTsA6Nq1KyaTiZtvvpljx44BsGrVKoKCggC47bbbLvmz9XXr1tG3b18ABg4ciJOTEwCrV68mMTGRoKAgHn74YYqKiti+fftlvU4REakemrq3MYZhYBjGWdtOf75ev359AEwmk2W/o6PjWedYcx3grFUKT283mUyWa5jNZmJjY2nTpg0Ahw8fxt3d/aKuJyIiNUMjehvj6upKs2bN+OqrrwDYtGkThw8fpmXLllWec/fdd5OWlgZAVlYW27dvr/RGAE69GTj9ZsHT05M//vgDgG+++aZSO6mpqQB89dVXlo8L/Pz8+PjjjwHIzc1l4MCBHDhwoDperoiIXCYVehsUGxvL/PnzGTBgAJMnTyY+Ph5nZ+cqj3/66afZs2cPAwYMIC4uDm9vbxo0aFDpmG7duvHKK6+wYcMGHnvsMRYuXMjgwYMpLv7f0qRRUVF8+eWXDBgwgBUrVnDVVVcBMHr0aIqLi+nfvz8jRowgPDyc66+/vmZevIiIXBSTcbFzumJzlixZgq+vL3fccQf79+9n+PDhfPPNNzg41J33eVqPvm5bGPMwhw4V1HYMAHx8GtWZLBdL2WuHPWZ3cDDh5eVqVRv6jP5v4MYbb+SVV17BbDbj4ODA5MmT61SRFxGRmqNC/zdw2223kZycXNsxRESkFmhYJyIiYsdU6EVEROyYCr2IiIgdU6EXERGxYyr0IiIidkx33UudEBc5qLYjyHkUl5TVdgQRuUQq9FInHDlyArPZ9p7dZKsP4rDV3CJy8TR1LyIiYsdU6EVEROyYCr2IiIgdU6EXERGxY7oZT+oEa1dhqot8fBrVdoRL4uPTiOKSMgqOF1/4YBGxWSr0UidomdrasTDmYQpQoRexZ5q6FxERsWMq9CIiInZMhV5ERMSOqdCLiIjYMRV6ERERO6ZCLyIiYsdU6EVEROyYCr2dyczMJDQ0tMr9GRkZzJs3D4CPP/6Yjz/+GIDIyEj27dtn9XWys7Pp0aPH5YUVEZEap0L/N/Prr79y4sQJAB566CEeeugh4NQbBMOwvWViRUTk/PRkPDu1bt06Zs2aRXFxMfn5+YSHh9OyZUsSExMBaNq0Kfv37wegfv365Obm8vjjj7NgwQJCQkL48MMP8fX1JTMzk4SEBObPn89vv/3GhAkTAGjdurXlWocPHyYqKoqDBw9iMpl48cUXufvuu6/8ixYRkbNoRG+nPvroI6ZOncrixYuZNm0a7777LjfddBNDhw5l6NChhISEWI59/PHHady4MXPmzMHT07PKNsePH094eDiLFy/G19fXsn3atGmEhISQnJzMe++9R1RUlGXWQEREapdG9HYqNjaWb7/9lvT0dDZv3kxh4eU9R/7o0aPk5uZaRurBwcEsWrQIgNWrV7Njxw7i4uIAKC8vZ+/evdxyyy2X9yJEROSyqdDbqWHDhtG5c2c6d+5Mly5dGDdu3EWdf/rz+vLycgBMJlOlz/AdHR0tP5vNZj744AM8PDwAyMnJwdvb+3JfgoiIVANN3duhY8eOsWvXLp599lnuueceVq1aRUVFBXCqQJ8u3n/l6OhoOcbT05M//vgDgOXLl1u2NW3alO+++w6ApUuXWs718/Nj4cKFAPzxxx8MHDiQkydP1tjrExER66nQ2yEPDw+GDBlCv379GDRoEEeOHKG4uJiioiI6depEWloa8+fPr3TOP//5Tx5//HH27t3L2LFjLZ+7N2r0v7XWY2NjSUhIYNCgQezZs8eyfeLEiWzevJkBAwbw/PPPExMTg6ur7a4vLyJiT0yGvlMldYDWo68dC2Me5tChgtqOcVF8fBrZXObTlL122GN2BwcTXl7WDag0ohcREbFjKvQiIiJ2TIVeRETEjqnQi4iI2DEVehERETumQi8iImLHVOhFRETsmB6BK3VCXOSg2o7wt1RcUlbbEUSkhqnQS51w5MgJzGbbe3aTrT6Iw1Zzi8jF09S9iIiIHVOhFxERsWMq9CIiInZMhV5ERMSO6WY8qROsXYWpLvLxObWUb3FJGQXHi2s5jYhIZSr0UifYwzK1C2MepgAVehGpWzR1LyIiYsdU6EVEROyYCr2IiIgdU6EXERGxYyr0IiIidkyFXkRExI6p0IuIiNgxmy70mZmZhIaG1kjbH3/8MR9//HGNtF0dIiIiSE5OJicnh1GjRtV2HBERqaP0wJwqPPTQQ7UdwSpNmjThP//5T23HEBGROsrmC/3Ro0cZNWoUe/bsoXnz5sTFxZGWlsa8efMwmUy0adOGSZMmcdVVV9GqVSuysrIASE5OZt26dbzxxhtMnz6dVatW4ejoSM+ePRk9ejTx8fEAjBkzBn9/f3r37s1PP/2Eo6Mjb731Fs2aNSMzM5OpU6fi6OhIhw4d+PPPP5k/f36VWSMiImjYsCE//fQTBQUFvPzyyyxZsoRt27bRq1cvIiIiqKioICYmhnXr1lFRUUFwcDCPPPIIhmHwxhtv8N1339G4cWMqKiq46667yM7OJiwsjIyMDH7//XemTJlCUVERR48e5dFHHyUsLIz4+HhycnLYvXs3+/btY8iQITz11FNV5iwvLyc6Oprt27dz+PBhmjdvTkJCAg0aNODDDz/ko48+olGjRtx4441cf/31jBkzhpUrVxIXF0d5eTm+vr5MmTIFT0/P6v1ji4jIRbPpqXuA/fv3ExUVxRdffMHhw4f5+OOPmT17NvPnzyctLY2GDRuSkJBQ5fn79u1j5cqVpKamkpiYyK5duygpKal0zKFDh+jSpQspKSl06tSJBQsWUFZWxksvvURsbCwpKSnUq2fde6bc3FxSU1MZO3YskZGRvPrqq6SkpJCUlERBQQFJSUkALF68mM8++4zly5fz448/8uWXX/Lbb7+xdOlS3n77bfbs2XNW259++ilPP/00ixYt4sMPP2TWrFmWfVlZWcydO5dPP/2UOXPmcPz48Sozbty4EScnJz755BO+/vprSkpKWLFiBdu2bWPBggUkJyezcOFCdu/eDZx6szVz5kzmzp1LSkoK/v7+zJgxw6r+EBGRmmXzI/rWrVvTrFkzAFq0aEFBQQEBAQGW0eSDDz5IZGRklec3adKE+vXrM3ToUAICAnjuueeoX7/+Wcd169YNgJYtW/Ljjz/y+++/4+XlRevWrQG4//77mTZt2gXzdu/eHYCmTZvSsmVLvLy8APDw8CA/P581a9awdetW1q5dC0BRURFZWVn8+eefBAYG4uTkxNVXX21p568iIiL4/vvv+fe//01WVhZFRUWWfZ07d8bZ2RkvLy88PDwoKCjAzc3tnBk7deqEh4cHCxYsYMeOHezatYuioiLWrFlDQEAArq6nFqDp168fx48fZ/PmzRw4cICwsDAAzGYz7u7uF+wLERGpeTZf6P86kjaZTLi5uVUarRqGQXl5eaXfTSaTZVu9evX49NNPWbduHStXrmTo0KHnnH4/XfxNJhOGYeDo6IjZbL7ovE5OTufMflpFRQXh4eEEBgYCp0bLLi4uxMbGVrreuc597rnncHNzIyAggPvuu49ly5adlf+vr6Eqy5cvJy4ujrCwMIKDg8nLy8MwDBwcHM75misqKujYsSOzZ88GoKSkhMJC216gRkTEXtj81P25ZGRkcOzYMQCSkpLo3LkzAJ6enmzfvh3DMMjIyADgt99+Y/jw4XTq1Inx48fTokULdu7cecFr3HjjjRw/ftzymX9aWlq1ZPfz8yMpKYmysjIKCwsZNmwYmzdvpkuXLqSnp1NaWkp+fj7ff//9WeeuWrWKsWPH0qtXL9avXw+cKsIXa82aNfTt25eQkBC8vb1Zv349FRUVdOnShRUrVnDixAlKS0v56quvMJlMtG/fnk2bNln67d133yUmJubyOkJERKqFzY/oz+Tq6soTTzxBaGgoZWVltGnThldffRWAF198kSeffBJvb2/uuOMO8vLyuPXWW+nQoQP9+/enYcOG3HLLLXTv3p1ff/31vNdxdnYmJiaG8ePH4+DgQPPmzWnQoMFl5x86dCi7d+9m8ODBlJeXExwcbHmj8ssvv9C/f3+8vb1p0aLFWeeOGTOGYcOG4ebmRvPmzbnuuuvIzs6+6AxDhgxh3LhxpKen4+zsTIcOHcjOzmbIkCGEhYXx4IMP4uLigqenJ/Xr18fHx4fXXnuN5557DrPZTJMmTYiNjb3svhARkctnMs43hytVMpvNzJgxg9GjR+Pi4sK8efPIyckhIiKitqPVmJ07d7JixQoeeeQRAJ566imGDBlCjx49Lrtte1mP/tChgtqOYRUfn0Y2k/VMyl47lL12VJXdwcGEl5erVW3Y3Yj+SnFwcMDDw4P7778fJycnrrvuOqZNm8b06dNZvXr1Wce3bdvWqpv1rpQff/yRKVOmnHPfnDlzaNKkyVnbr7vuOsusgslkwt/fn4CAgJqOKiIil0EjeqkTNKK/suxxhGMLlL122GP2ixnR2+XNeCIiInKKCr2IiIgdU6EXERGxYyr0IiIidkyFXkRExI7p63VSJ8RFDqrtCJetuKSstiOIiJxFhV7qhCNHTmA22943PW35azsi8vegQi91goODqbYjXDJbzW6ruUHZa4uy145zZb+Y16MH5oiIiNgx3YwnIiJix1ToRURE7JgKvYiIiB1ToRcREbFjKvQiIiJ2TIVeRETEjqnQi4iI2DEVehERETumQi8iImLHVOilxqSlpXHfffdx7733smDBgrP2b926lZCQEHr37s2ECRMoLy8HYP/+/Tz88MP06dOHp556isLCwisd/ZKzp6Sk4O/vT1BQEEFBQcyaNetKR79g9tPGjx9PcnKy5Xdb6PfTzsxuC/3+zTffEBQUxMCBA3n66afJz88HbKPfq8pe2/1+odxff/01AwYMoF+/fkRERFBaWgrYRp9Xlf2S+twQqQEHDx40AgICjLy8PKOwsNAYMGCAsX379krH9OvXz9i4caNhGIYRGRlpLFiwwDAMw3j88ceNpUuXGoZhGAkJCUZMTIzNZJ88ebKRlpZ2RfP+lTXZDx48aDzxxBNGu3btjEWLFlm220K/V5W9rvd7QUGB0bVrV+PgwYOGYRjGW2+9ZUyZMsUwjLrf7+fLXpv9fqHchYWFhr+/v3Ho0CHDMAzjueeeMxITEw3DqPt9fr7sl9LnGtFLjVi9ejV+fn54eHjg4uJC7969SU9Pt+zft28fxcXFdOjQAYDg4GDS09MpKytj/fr19O7du9J2W8gO8Msvv5CSksLAgQMZN26cZeRTV7LDqZFEz5496du3r2WbLfR7Vdmh7vd7WVkZ0dHRNGnSBIBWrVpx4MABm+j3qrJD7fb7hXK7uLiQkZGBt7c3RUVFHDlyBDc3N5vo86qyw6X1uQq91Ijc3Fx8fHwsvzdu3JicnJwq9/v4+JCTk0NeXh6urq7Uq1ev0vYr6VKzn/55zJgxLFmyhGuvvZbJkydfueDnyHZmdoDHHnuMIUOGVNpmC/0O584Odb/fPT096dWrFwDFxcXMmTOHXr162US/V5X9dN7a6ndr/ntxcnJixYoVBAQEkJeXh7+/v030eVXZT+e92D5XoZcaYZxjUUSTyXTB/Rc670q41OwA77zzDu3bt8dkMvHYY4+xcuXKmgt6Dpfaf7bQ7+djK/1eUFDAqFGjaN26NYMHD7apfj8zO9Ruv1ub+5577iEzM5OAgACio6Ntqs/PzA6X1ucq9FIjmjRpwuHDhy2/5+bm0rhx4yr3Hzp0iMaNG3P11Vdz4sQJKioqKm2/ki41e0FBAf/9738t2w3DsIwarpQLZa+KLfR7VWyl33Nzcxk2bBitW7dm2rRpgO30+7my13a/Xyj3sWPH+OGHHyy/DxgwgKysLJvo86qyX2qfq9BLjbj77rtZs2YNR48e5eTJk3z11Vd0797dsv+6666jfv36/PTTT8CpO0m7d++Ok5MTd955J59//nml7baQ3cXFhffff5/NmzcD8NFHH3HvvffWqexVsYV+r4ot9HtFRQVPPvkkffv2ZcKECZbRmy30e1XZa7vfL5TbMAzCw8PZv38/AF988QUdO3a0iT6vKvsl9/ml3TMocmGpqalGv379jMDAQGPOnDmGYRjGY489Zvz888+GYRjG1q1bjZCQEKNPnz7GCy+8YJSUlBiGYRjZ2dnG8OHDjb59+xr/+te/jGPHjtlM9vXr1xuDBg0y+vTpYzz55JPG8ePH61z208aPH1/pznVb6PfTzsxe1/v9q6++Mlq1amUMHDjQ8u/ll182DKPu9/v5std2v1/ov5evv/7a6N+/vzFgwADj+eeft+Sr631+vuyX0ucmwzjHhwUiIiJiFzR1LyIiYsdU6EVEROyYCr2IiIgdU6EXERGxYyr0IiIidkyFXkRsQo8ePfjll1+u6DW/++473n777St6TZHqpkIvIlKFX3755YovkCNS3a7scyJFRC7TbbfdxiOPPMJ3333HiRMnCA8PJz09nd9//53GjRsze/ZsXFxcuPXWWxkxYgSZmZkUFRXxwgsvEBgYCJx6XviyZctwdHSkefPmTJo0CR8fH0JDQ3F3d2fHjh3cd999JCYmUlFRQaNGjXjiiSeIjo5m165d5Ofnc9VVVzFjxgxuvPFGQkND6dChAxs2bODAgQPccccdTJ8+HQcHB7799lveeustzGYzLi4uvPrqq7Ru3ZoNGzYwY8YMTp48iclkYsyYMQQEBNRy74o9UqEXEZtSWlqKj48PaWlpzJkzh4kTJ/LFF1/g4+PD/fffz/LlyxkwYAAVFRW4u7uTnJzMtm3bGD58OHfeeSfffvst33//PZ999hkuLi7Ex8cTERHB3LlzAXBzc7M8HtUwDPLy8nj++edJT0/Hzc2NpKQkAKKioliwYAGTJk0CYM+ePcyfP5+ioiL69u3LunXruOmmmwgPD2f+/PnccsstfPXVV8yYMYOZM2cSGRnJ3Llz8fX1JScnhwceeIBWrVrRtGnT2ulYsVsq9CJic06vJX799ddz8803W9ZK9/X1rTTVPnz4cABat27NzTffzPr161m5ciXBwcG4uLgAEBYWxuzZsyktLQXgzjvvPOc1+/TpQ7NmzZg/fz67d+9m3bp13H777Zb9AQEBODg44Orqyg033EB+fj4bNmygZcuW3HLLLQAEBgYSGBjIihUrOHToEM8884zlfJPJRFZWlgq9VDsVehGxOU5OTuf8+UyOjo6Wn81mM46OjmctEWo2mykvL7f8fvoNwJkWLlxIUlISDz/8MAMGDMDDw4Ps7GzL/gYNGlh+Pr3kcr169c5a4jgrK4uKigpatGjBp59+atmXk5PD1Vdffb6XLXJJdDOeiNitlJQUAH799Vd27txJp06d8Pf3Jzk5maKiIgDmz59Pp06dcHZ2Put8R0dHy5uAH374gcGDBzNkyBCaN29ORkaGZanTqrRv354///yT7du3A7B8+XLCw8Pp0KEDu3fvZv369QBs3bqV3r17k5ubW22vXeQ0jehFxG5t2LCBpKQkzGYzs2bNwt3dnfvvv58DBw4wZMgQzGYzN9xwAzNmzDjn+V26dGHMmDE4OTnxr3/9i6ioKJKTk3F0dKRNmzb8/vvv572+t7c3M2bMYPz48VRUVODq6sqsWbO4+uqriYuLIyYmhpKSEgzDICYmhuuuu64mukH+5rR6nYjYpVatWrFmzRpNh8vfnqbuRURE7JhG9CIiInZMI3oRERE7pkIvIiJix1ToRURE7JgKvYiIiB1ToRcREbFjKvQiIiJ27P8BHafFYWm/cNIAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "feature_importance = rf_feature_importance(model, X_keep)\n", "plot_feature_importance(feature_importance);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Our final model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we have simplified our model, let's create a huge Random Forest to see if we can squeeze a bit more performance:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RMSE on train: 14963.277\n", "R^2 on train: 0.976\n", "RMSE on valid: 41200.693\n", "R^2 on valid: 0.823\n", "OOB R^2: 0.823\n" ] } ], "source": [ "model = RandomForestRegressor(\n", " n_estimators=1000, max_features='sqrt', n_jobs=-1, oob_score=True, random_state=42\n", ")\n", "model.fit(X_train, y_train)\n", "print_rf_scores(model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our final model achieves around 5% better performance compared to our naive model with 10 trees from lesson 4, but it is _much simpler_ - we've reduced the features from 19 down to 7! Moreover we've identified which features are most important, let's plot them out for the final visual comparison:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfwAAAEJCAYAAABxDZW5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deVRV9f7/8ecBQSVkCNAyrGtmWpqaZWKiXdRQc0Ahy1SwrtmoNklCKpFDJWgWUHm9+fWWaUSJCFo0SGk5oOVQlpLliAOIIiLIePbvD3+eG46IzOf1WMu1ztnDZ7/2h1rv8/mcffY2GYZhICIiIvWaTU0HEBERkaqngi8iImIFVPBFRESsgAq+iIiIFVDBFxERsQIq+CIiIlZABV9ERMQKNKjpACIA2dl5mM1175YQbm6OHDt2qqZjVIiy1wxlrxn1LbuNjQlX12uuqB0VfKkVzGajThZ8oM7mBmWvKcpeM6w9u0l32hMREal+BYXF5J4suOx2Hh5NOHo0t8wyGxsTbm6OV3Q8jfClVpjwRgJZ2Xk1HUNEpNosiRhJLpcv+JVFF+2JiIhYARV8ERERK6CCLyIiYgVU8EVERKyACr6IiIgVUMEXERGxAir4IiIiVkAFX0RExAqo4Fez1NRUAgMDAZg8eTK//vprpbY/duxYMjIyKrVNERGp+3SnvRo0c+bMSm/zP//5T6W3KSIidZ8KfjmlpqYyb948DMNg//799O3blyZNmvDtt98CMH/+fH7//XeioqIoKSnB09OT6dOn4+rqyo8//sgbb7xBw4YNadmypaXNwMBAxo0bx1133UV4eDi7du0iKyuLli1bEhMTQ1ZWFuPGjaN169bs2LEDNzc33nnnHVxcXC6as1evXnz00Uds3LiRH374gZycHA4cOED37t0JDw/HMAxmz57Nt99+i62tLQ8//DCjR49mz549hIWFceLECRwcHJg8eTIdOnQgJCSExo0b8/PPP5Obm8srr7zC8uXL2blzJ3369CEkJITS0lIiIiLYuHEjpaWl+Pv78+ijj1b1n0RERK6ApvSvwLZt23jjjTdYuXIlsbGxXHvttcTHx9OmTRtiY2OZM2cOCxYsICEhAW9vb2bPnk1RUREhISFERUURHx9Po0aNzmt3y5Yt2NnZ8emnn/LNN99QWFjI6tWrAdi5cyePPfYYK1aswMnJiaSkpHLn3bJlC1FRUSQmJvLdd9+RlpZGcnIymzdvJikpic8++4z4+HiOHj1KcHAwgYGBJCUlERoaynPPPUdRUREAmZmZJCYmMmHCBEJDQ3nttddISEggLi6O3Nxc4uLiAFi2bBmff/45q1at4qeffqqEHhcRkcqiEf4VuPXWW7n++usBcHV1pVu3bgA0b96clJQUDh8+TFBQEABmsxlnZ2fS0tJo2rQprVq1AmDo0KG88847Zdrt0qULLi4uLF68mN27d7N3717y8/MBcHNz4/bbbwegdevW5OTklDvvnXfeiaPjmacptWjRgpycHDZt2kT//v2xt7fH3t6e5cuXk5eXx/79+/H19QWgU6dOODs7s3v3bgB69uxpOc/WrVvj5uYGgIuLCzk5Oaxfv54dO3awYcMGAPLz80lLS+Puu+++ku4VEZEqpIJ/Bezs7Mq8t7W1tbw2m8107tyZefPmAVBYWEheXh6HDh3CbDZfcJ+zVq1aRVRUFEFBQfj7+5Odnc3ZpxY3bNjQsp3JZOJKnmZ8oX0bNCj7J09PT8fZ2fm8dg3DoLS09LzzPnd/gNLSUoKDgy0fGI4fP46Dg0O5c4qISNXTlH4l6dChA1u3bmXPnj0AvPfee0RERNCmTRuOHTvGzp07AVi5cuV5+65fv57+/fsTEBCAu7s7mzZtshTbytalSxe++eYbiouLOX36NI8//jhZWVm0aNGCr7/+GoCtW7eSlZVF69aty9Wml5cXcXFxFBcXk5eXx4gRI9i2bVuV5BcRkYrRCL+SeHh48Prrr/P8889jNptp1qwZkZGR2NnZ8dZbbxEcHEyDBg0s0/N/N2zYMCZOnEhycjL29vZ06tSJ9PT0Ksl5//33s337dvz9/TGbzQQFBdGyZUsiIyMJDw8nOjoaOzs7oqOjsbe3L1ebw4cPZ9++fQwdOpSSkhL8/f3p2rVrleQXEZGKMRlXMkcsUkUmvJFAVnZeTccQEak2SyJGcvRo7mW38/Boct52NjYm3Nwcr+h4GuHXQYGBgZw8efK85cOHD+eRRx6pgUQiIlLbqeDXQYsWLarpCCIiUsfooj0REREroIIvIiJiBVTwRURErIAKvoiIiBXQz/JERERqQEFhMbknCy67nX6WJ/XKsWOnMJvr3mfPC/2PWFcoe81Q9ppRl7NXFk3pi4iIWAEVfBERESuggi8iImIFVPBFRESsgC7ak1rhSq82rU08PJrUdIQKq0vZy3tFs4hcmAq+1Ap6Wp5czpKIkeSigi9SUZrSFxERsQIq+CIiIlZABV9ERMQKqOCLiIhYARV8ERERK6CCLyIiYgVU8EVERKyACn4leeedd1i1alW5tx87diwZGRkcOHCAV155pULHjI+PJyQkBIDo6Gh8fX0pKPjf75RTU1MJDAwEICQkhPj4+Iu2NWvWLLy8vCgqKrIsS09Pp02bNqxdu7bMtr169SI9PZ309HTat2+Pn58ffn5+9O3blwkTJpCVlVWh8xERkaqjgl9JnnvuOXr37l3u7f/zn//QrFkzDh06xIEDByolw6FDh3jrrbeueL+SkhK+/PJL7rzzTpKTk8uss7OzY+rUqZw6deqC+zZt2pTly5ezfPlykpOTuemmm5gwYUKF8ouISNWptoI/b948HnjgAQYNGsSbb75JaWkp//3vf+nbty8PPPAAkZGRAGRlZfHMM8/g7+9PQEAA69atAyAjI4MxY8bw0EMP4ePjw+zZs4Ezo9wXXniBf/3rX9x///2Eh4dfMkd6ejqDBg3imWeeYcCAAYwdO5YTJ04A4OXlxZgxY/Dz86O4uPiCmVetWoWvry+FhYXs27ePnj17cuTIEcsIOj09HT8/P8aNG4evry8vvvgisbGxPPzww/Tr14+//voL+N8oecaMGWzfvp3XXnuN4OBgPv30U0vWwMBAtm3bVu4+Hj58OF988QU//fRTufcBWL16NS1atGDIkCFljg9nCvq9997LrFmzLtuOyWRi/Pjx7Nq1i507d15RBhERqVrVUvBXr15NSkoK8fHxLFu2jH379vH++++zZMkSPv/8cxITE/ntt9/Yvn07M2fOJCAggPj4eN5//33CwsI4deoUK1asYODAgcTFxZGYmMiSJUs4fvw4AFu2bCEqKorExES+++470tLSLpnnjz/+YPTo0axcuZJWrVoRExMDQHZ2Nk888QTLly9n3bp152WOjY2ld+/e3Hnnnbz//vuEhoYyadIkrrvuujLtp6Wl8cwzz5CcnMyvv/7KwYMH+fTTTxk4cOB5BXXKlCm0b9+eV199lYCAABITEwE4ePAgx48fp2PHjuXuZ2dnZ8LDw5k8eXKZqf3LiY+Pp1+/ftx3333s2LGDP//8s8z6kJAQfvzxx/Om9i/E3t6em266id27d5f7+CIiUvWqpeBv2LCBAQMG0KhRIxo0aEBAQAAxMTH4+PjQpEkTGjRowH//+1/at2/PunXriIqKws/Pj7Fjx1JSUsKBAwcYM2YM119/PQsWLGDmzJkUFxdz+vRpAO68804cHR1p3LgxLVq0ICcn55J5/vGPf9C1a1cAhgwZwoYNGyzrzhbYC2Vev349AJMnT+bzzz/H3d2dAQMGnNe+u7s7t99+OzY2Nlx33XV069YNgObNm3Py5MmL5uratSuZmZmkp6eTkJCAn5/fFfTyGX369KF9+/blnto/fvw4P/74I/3796dRo0b4+PgQGxtbZhtHR0emT59+yan9vzOZTDRq1OiKs4uISNWplofnmM3m85YZhlHmfUZGBo0bN8ZsNvPhhx/i4uJiWe7u7s6bb77JgQMHGDhwIH369GHdunWWNho2bGhpx2Qyndf2uRo0+N9pG4aBra2t5f3ZQnWhzCUlJcCZrx1sbW3Zs2cPRUVF2Nvbl9nu3Pd/b/9STCYTQ4YMYeXKlSQnJ/PBBx+Ua79zTZ06lYEDB1r68FISExMxDIMHH3wQgIKCAoqLi5k4cWKZ7by9vcs1tV9UVMSePXu45ZZbKpRdRESqRrWM8L28vFi5ciUFBQWUlJSwdOlSJk2axJo1a8jLy6OkpISXXnqJ7du34+XlxZIlSwD4888/GTx4MKdPn2bt2rWMGTOG/v37c/jwYTIyMi5YlMtjz5497NixA4ClS5fSs2fPcmX28vKitLSU0NBQJk+eTJcuXXj77bcr3jGc+TBw9oMEgL+/P7GxsVx33XU0a9asQm26uLgQHh7Oe++9d9ltly5dyptvvklKSgopKSn8+OOPODs788UXX5y37dmp/czMzAu2ZTabiY6OpmPHjtx4440Vyi4iIlWjWkb4Pj4+7Nixg4CAAEpKSujRowdBQUE0bNiQ4cOHYzabuf/++7n33ntp1aoVYWFhDBo0CICIiAgcHR158sknefnll3FycsLNzY327duTnp5eoTzOzs5ERUWxf/9+2rRpw4wZM8qVedSoUfzf//0fbm5u+Pr6cu+99zJw4EB8fX0r3DetWrUiNzeX4OBgIiMjuf7667n++usZOnRohduEM1P7ffv2LVOcX331VaZPn255/+9//5vs7Gzuv/9+yzIbGxtGjx5NbGws99xzT5k2z07tjxkzxrIsMzPT8tWD2WzmtttuY86cOVeVXUREKp/JuNz8dz2Tnp5OUFAQKSkpNR3lPIZhkJmZSWBgICtWrDjvq4H6bMIbCWRl59V0DKnFlkSM5OjRXAA8PJpYXtc1yl4z6lt2GxsTbm6OV9ROtYzwq9v+/fsZP378BdddaDRfW3z11VeEh4cTHh5uKfazZs2y/DTx79q3b8/MmTOrO6KIiNRR9bLg33jjjSxfvvyi62vj6B6gX79+9OvXr8yySZMm1VAaERGpT3SnPRERESuggi8iImIFVPBFRESsgAq+iIiIFVDBFxERsQL18ip9qXuiQofUdASp5QoKi2s6gkidpoIvtcKxY6cwm+vePaDq2808RKT+0pS+iIiIFVDBFxERsQIq+CIiIlZABV9ERMQK6KI9qRWu9KlPtYmHR5OajlBhVZm9oLCY3JMFVda+iFwZFXypFfR43PpnScRIclHBF6ktNKUvIiJiBVTwRURErIAKvoiIiBVQwRcREbECKvgiIiJWQAVfRETECqjgi4iIWAEV/DoqJSWFhQsXXnKb+Ph4QkJCqixDdHQ00dHRVda+iIhUHt14p4767bffajqCiIjUISr4NSA1NZXo6GgaNGjA4cOH6dChAzNnziQpKYmFCxdiMplo164dU6dOxd7enldeeYVdu3YBMGLECDp37kxsbCwAzZs3x9vbm1deeYXc3FyOHj3KgAEDmDhxYrmy7Nixg7CwMAoKCnB2dmb27Nlcd911zJs3j8TERGxtbenevTvBwcHY2trywQcfEBcXh6urK05OTnTo0AGANWvWEBUVRUlJCZ6enkyfPh1XV9eq6UAREblimtKvIb/88gthYWEkJydTWFjI/PnzmTdvHosWLSIpKYnGjRsTExPDli1byMnJISEhgYULF7J582ZuueUWhg8fzvDhwwkICGDFihUMHDiQuLg4EhMTWbJkCcePHy9XjokTJ/LMM8+QlJTEAw88wIcffsjq1atJSUkhPj6eZcuWsW/fPmJjY/n1119ZunQpy5YtY+HChRw5cgSA48ePM2fOHBYsWEBCQgLe3t7Mnj27KrtPRESukEb4NaRLly7cfPPNAPj5+TF+/HhGjRplGRU//PDDhIaG8sQTT7Bnzx7GjBlDz549LzhyHzNmDBs2bGDBggXs2rWL4uJiTp8+fdkMx48f5+jRo/j4+ABnZg8AZs2axYABA2jUqBEAAQEBJCQkUFBQwH333cc111wDQL9+/TCbzWzbto3Dhw8TFBQEgNlsxtnZ+Sp7SEREKpMKfg2xtbW1vDYMA7PZXGa9YRiUlJTg6urKypUrWbt2LatXr2bo0KGsXLmyzLZvvvkmBw4cYODAgfTp04d169ZhGMZlM9jZ2ZV5X1hYSGZm5nlZAEpKSjCZTGXWNWjQgKKiIkpLS+ncuTPz5s2ztJOXpwfhiIjUJprSryE///wzGRkZmM1mEhISCA0NJSUlhRMnTgAQFxdH165dWbVqFRMnTuSf//wnU6ZMwcHBgcOHD2Nra0tJSQkAa9euZcyYMfTv35/Dhw9b2r2cJk2acN1117F27VoAli9fzjvvvIOXlxcrV66koKCAkpISli5dipeXF926deP7778nNzeXwsJCvvnmGwA6duzI1q1b2bNnDwDvvfceERERVdFtIiJSQRrh15CmTZvy8ssvk5GRQffu3Rk1ahQODg4EBgZSXFxMu3bteO2112jYsCFfffUVAwYMoGHDhvj6+tKmTRtOnjzJpEmTcHd358knn+Tll1/GyckJNzc32rdvT3p6erlyREZGEh4eTkREBK6urkRERNC0aVN27NhBQEAAJSUl9OjRg1GjRtGgQQNGjx7Ngw8+iJOTE82bNwfAw8OD119/neeffx6z2UyzZs2IjIysyu4TEZErZDLKM/crlSo1NZWYmBgWLVpU01FqjQlvJJCVra8B6pMlESM5ejS3Str28GhSZW1XNWWvGfUtu42NCTc3xytqRyN8K/DSSy/x559/nre8V69ePPfcczWQSEREqpsKfg3o2rUrXbt2rbbjzZkzp9qOJSIitZMu2hMREbECKvgiIiJWQAVfRETECqjgi4iIWAEVfBERESugq/SlVogKHVLTEaSSFRQW13QEEfmbchX806dPk5ycTE5OTpl7tD/22GNVFkysy7FjpzCb6949oOrbzTxEpP4qV8F/4YUXyMzM5NZbb8VkMlV1JhEREalk5Sr4u3fv5osvvqBBA30DICIiUheV66K96667rqpziIiISBUq15D91ltvJSgoiB49etCoUSPLcn2HLyIiUjeUq+Dn5eVx0003sX///qrOI1bqSp/6VJt4eDSp6QgVVhnZCwqLyT1ZUAlpRKQqlavgv/HGG1WdQ6ycHo9bdy2JGEkuKvgitV25Cn5qairz588nJyenzPLPP/+8SkKJiIhI5SpXwZ8yZQqBgYHceOONVZ1HREREqkC5Cr6bmxtBQUFVnUVERESqSLkKfq9evVi8eDE9evQo81v85s2bV1kwERERqTzlKvjHjx/nrbfeonHjxpZlJpOJzZs3V1kwERERqTzlKvjJycn8+OOPuLu7V3UeERERqQLlutOem5sb1157bVVnERERkSpSroJ/xx13MGLECP7973+zcOFCy7+aFBgYSGpq6mW3OcvPz6+qI1WaNm3aVPsx09PT6dWr1xXt06tXL9LT089bXp6/jYiIVK9yTekXFhbSsmVL9u7dW8VxKtfGjRstr5cvX16DSURERGpWld1pLzU1lejoaBo0aMDhw4fp0KEDM2fOJCkpiYULF2IymWjXrh1Tp07lmmuuwcvLCx8fH7Zv384111zD7Nmz8fT0pFevXnz00Ud4enqSmppKTEwMixYtshynpKSE8PBwdu3aRVZWFi1btiQmJobZs2cDMGzYMD777DPatGlDWloap0+fZsqUKaSlpWEymRgzZgxDhgwhPj6eH374gZycHA4cOED37t0JDw+/6Pmlp6fz9NNP06JFC/bt20fz5s2JjIzExcWFNWvWEBUVRUlJCZ6enkyfPh1XV1d69epFhw4d2LFjB0uWLMHNze2i7YeFhbF161YAoqOjuemmm9i6dSszZ86ksLAQV1dXpk2bxk033URgYCDjxo2ja9eupKenExQUREpKCklJSXzwwQfY2tri6elJZGQkDRs2ZP78+Xz55ZeUlpbi7e1NcHAwAAUFBbzwwgvs2rULJycn3n33XVxdXfnuu+94++23MZvNtGjRgmnTppW5nqOoqIjJkyezfft2brjhBrKzs6/4vxcREala5ZrSf+qppy7473J++eUXwsLCSE5OprCwkPnz5zNv3jwWLVpEUlISjRs3JiYmBoDs7GzuuecekpKSGDBgADNmzCjXCWzZsgU7Ozs+/fRTvvnmGwoLC1m9ejVTpkwB4LPPPiuzfXR0NK6urqxYsYIPP/yQ6Ohodu7caWkrKiqKxMREvvvuO9LS0i557D/++IPRo0ezcuVKWrVqRUxMDMePH2fOnDksWLCAhIQEvL29LR8+AHr27MlXX311yWIPcO+995KYmEj37t2JjY2lqKiIF198kalTp5KYmMjw4cN58cUXL9nG22+/zf/93/8RHx9Py5Yt2b17N2vWrGH79u18/vnnJCQkkJGRQWJiInDm1xiPPfYYK1aswN3dnS+++IJjx44RFhbGu+++S1JSEp07d2batGlljnP2A9iXX37JlClT9MwFEZFaqFwj/L59+1peFxcXk5KSUq7vmbt06cLNN98MnPkOffz48YwaNQpXV1cAHn74YUJDQwFo2LAhQ4YMAWDo0KG89dZb5TqBLl264OLiwuLFi9m9ezd79+4lPz//ottv2LCB119/HYBrr72W3r17s3HjRhwdHbnzzjtxdDzzEJcWLVqcdyvhc/3jH/+ga9euAAwZMoSJEyfSvXt3Dh8+bLlRkdlsxtnZ2bJPx44dy3Veffr0AeCWW27hp59+Yu/evTg5OdGhQwcA+vfvT1hYGLm5uRdtw8fHh0ceeYTevXvTt29fbrvtNhITE/nll1/w9/cHzozqmzdvzl133UXTpk0t7d9yyy1kZ2fzyy+/0KFDBzw9PYEzf7P58+eXOc7GjRt5+OGHLX1y5513luscRUSk+pSr4A8dOrTMe39/f0aNGnXZ/WxtbS2vDcPAbDaXWW8YBiUlJQDY2NhgMpmAM0Xy3H0By7Z/t2rVKqKioggKCsLf35/s7GzL9hdy7jrDMCgtLQXOfOg4y2QyXbIdoMxNiAzDwNbWltLSUjp37sy8efOAM9c/5OX976Ewfz9Gedo+m+Pcvjs3+4X6aMqUKezcuZPVq1cTHBzMuHHjKC0tZfTo0ZZHG588eRJbW1uys7PLnM/Fjvv3v9nft/37dn9vR0REaodyTemfy2w2k5mZedntfv75ZzIyMjCbzSQkJBAaGkpKSgonTpwAIC4uzjJCPn36NCkpKQDEx8fTs2dPAFxdXfnzzz+BM8X9XOvXr6d///4EBATg7u7Opk2bLEXQ1tb2vOLk5eVleejP8ePHWbVqFffcc09FuoE9e/awY8cOAJYuXUrPnj3p2LEjW7duZc+ePQC89957REREVKj9v7v55ps5ceIEv/zyCwBffPEFzZs3x8XFpUwfffvtt8CZwu/r64urqytPPvkkfn5+7NixAy8vL5YvX05eXh4lJSU8++yzfPXVVxc9bseOHdm2bZvlavxPP/3U8jc7q1u3bqxYsQKz2czBgwd1QyYRkVqoXEOxc7+v/+OPP8pVJJs2bcrLL79MRkYG3bt3Z9SoUTg4OBAYGEhxcTHt2rXjtddes2yfnJzM3Llzadq0KbNmzQJgwoQJTJ8+nZiYGLy9vc87xrBhw5g4cSLJycnY29vTqVMnS3Hq3bs3fn5+xMfHW7Z/9tlnCQ8PZ9CgQZSWlvLUU0/Rrl27y35ffyHOzs5ERUWxf/9+2rRpw4wZM3BwcOD111/n+eefx2w206xZMyIjI6+47XPZ29szd+5cpk+fzunTp3F2dmbu3LkAPP7444SEhLB06VJ69+4NnBllT5gwgccee4xGjRrh5OTErFmzaNasGTt37uShhx6itLSUHj16MHToUA4ePHjB47q7uzNt2jTGjRtHcXExzZs3Z+bMmWW2GTFiBLt27aJ///7ccMMN3HrrrVd9viIiUrlMxuXmrYFly5b9bweTCVdXV7y9vctMu5/rQlfUX8rZq+jrir9fDS9Xb8IbCWRl511+Q6l1lkSM5OjRi19LUhU8PJpU+zEri7LXjPqW3cbGhJub4xW1c8kR/tmpdx8fn/PW5ebm4uLickUHq2v279/P+PHjL7iuvL8iuJCCggLLRW7nmjBhgmWULiIiUlkuWfC9vLwsF9KdOxFgMpks319fSNeuXc/7rvdSauPo/sYbb7zkDXsqOrpv1KiRbgQkIiLV6pIF/+zv00VERKRuK9dFe2azmQULFrBmzRpKSkro3r07Tz31lH5+JSIiUkeU62d5c+bMYcOGDZbfb2/ZssVyFb2IiIjUfuUaov/www8sXboUOzs7AP75z38yePDgKg0mIiIiladcI3zDMCzFHs78Jvzv70VERKR2K9cIv23btrz++uuW2+l+/PHHurmKVKqo0CE1HUEqqKCwuKYjiEg5lKvgv/rqq8yYMYPhw4djNpvp0aMHU6dOrepsYkWOHTuF2XzZe0DVOvXtZh4iUn9dckq/qKiISZMmsWHDBt58803WrVtHhw4dsLW1tTxVTkRERGq/Sxb8qKgoTp06VeZxp9OnT+fkyZNER0dXeTgRERGpHJcs+N9//z1z5szBzc3NsqxZs2ZERERYnsomIiIitd8lC76dnR2NGjU6b7mjoyP29vZVFkpEREQq1yUv2rOxseHUqVPnfV9/6tSp854zL3I1rvSpT7WJh0eTmo5QYeXJXlBYTO7JgmpIIyJV6ZIFf+DAgUyZMoXXX38dBwcHAPLz85kyZQq+vr7VElCsgx6PW3stiRhJLir4InXdJaf0R48eTZMmTejevTsPPfQQDz74IN27d8fJyYlnn322ujKKiIjIVbrslP706dN58skn+f3337GxseGOO+6gWbNm1ZVPREREKkG5brzj6emJp6dnVWcRERGRKlKue+mLiIhI3aaCLyIiYgVU8EVERKyACr6IiIgVUMGvQ1JTUwkMDKy09latWsU777wDnHluwk8//XRF+7dp06bSsoiISNVSwbdivXv35rnnngNg06ZNlJaW1nAiERGpKuX6WZ7ULnv27CEsLIwTJ07g4ODA5MmT6dChAyEhITg6OvLbb7+RkZHBs88+S0BAALm5ubz88ju1RboAABehSURBVMvs37+fFi1acOTIEWJiYti4cSMbN27Ey8uL7du3M2XKFGJiYpgxYwbjxo2ja9eupKenExQUREpKCunp6QQHB5Ofn0/Hjh0tefLy8pg2bRq7du2itLSUsWPHMnDgwBrsIREROZdG+HVQcHAwgYGBJCUlERoaynPPPUdRUREAR44cYcmSJbz//vtEREQA8O6779KyZUtWrlzJs88+S1paWpn2hgwZQvv27ZkxY8Ylp+mnT5+Ov78/y5cvp3Pnzpbl77//Pu3atSM+Pp7Fixczb948Dhw4UAVnLiIiFaWCX8fk5eWxf/9+y7MMOnXqhLOzM7t37wage/fumEwmbr31Vk6cOAHA2rVr8fPzA+COO+6o8HfvGzdupH///gAMHjwYOzs7ANatW0dsbCx+fn6MHDmS/Px8du3adVXnKSIilUtT+nWMYRgYhnHesrPfvzds2BAAk8lkWW9ra3vePuU5DnDeUxHPLjeZTJZjmM1mIiMjadeuHQBZWVk4Oztf0fFERKRqaYRfxzg6OtKiRQu+/vprALZu3UpWVhatW7e+6D733nsvSUlJAKSlpbFr164yHwjgzIeCsx8aXF1d+fPPPwH49ttvy7STmJgIwNdff235GsHLy4tPPvkEgMzMTAYPHszhw4cr43RFRKSSqODXQZGRkSxatIhBgwYxbdo0oqOjsbe3v+j2zzzzDPv372fQoEFERUXh7u5Oo0aNymzTo0cPXn31VTZv3szjjz/OkiVLGDp0KAUF/3ssalhYGF999RWDBg1i9erVXHPNNQCMGzeOgoICBg4cyOjRowkODubGG2+smpMXEZEKMRlXOtcrdc7y5cvx9PTkrrvu4tChQ4waNYpvv/0WG5va83lvwhsJZGXn1XQMuYAlESM5ejS3pmOU4eHRpNZlKi9lrxn1LbuNjQk3N8crakff4VuBm2++mVdffRWz2YyNjQ3Tpk2rVcVeRESqngq+FbjjjjuIj4+v6RgiIlKDNMwTERGxAir4IiIiVkAFX0RExAqo4IuIiFgBFXwREREroKv0pVaICh1S0xHkIgoKi2s6gohUAhV8qRWOHTuF2Vz37gFV327mISL1l6b0RURErIAKvoiIiBVQwRcREbECKvgiIiJWQBftSa1wpU99qk08PJrUdIQKu1D2gsJick8WXGBrEanLVPClVtDjcWuPJREjyUUFX6S+0ZS+iIiIFVDBFxERsQIq+CIiIlZABV9ERMQKqOCLiIhYARV8ERERK6CCLyIiYgVU8OuZ1NRUAgMDL7o+JSWFhQsXAvDJJ5/wySefABAaGsrBgwfLfZz09HR69ep1dWFFRKTaqOBbmd9++41Tp04B8Mgjj/DII48AZz4oGEbdezytiIiUj+60V09t3LiRuXPnUlBQQE5ODsHBwbRu3ZrY2FgAmjdvzqFDhwBo2LAhmZmZPPHEEyxevJiAgAA++ugjPD09SU1NJSYmhkWLFvH7778zefJkANq2bWs5VlZWFmFhYRw5cgSTycRLL73EvffeW/0nLSIiF6URfj318ccfM2PGDJYtW8bMmTN57733uOWWWxg+fDjDhw8nICDAsu0TTzxB06ZNmT9/Pq6urhdtc9KkSQQHB7Ns2TI8PT0ty2fOnElAQADx8fG8//77hIWFWWYRRESkdtAIv56KjIzku+++Izk5mW3btpGXd3X3qT9+/DiZmZmWkbu/vz9Lly4FYN26dezevZuoqCgASkpKOHDgALfddtvVnYSIiFQaFfx6asSIEXTt2pWuXbvSrVs3Jk6ceEX7n/0+v6SkBACTyVTmO35bW1vLa7PZzIcffoiLiwsAGRkZuLu7X+0piIhIJdKUfj104sQJ9u7dy3PPPcd9993H2rVrKS0tBc4U6rNF/O9sbW0t27i6uvLnn38CsGrVKsuy5s2b8/333wOwYsUKy75eXl4sWbIEgD///JPBgwdz+vTpKjs/ERG5cir49ZCLiwvDhg1jwIABDBkyhGPHjlFQUEB+fj5dunQhKSmJRYsWldnnn//8J0888QQHDhxgwoQJlu/lmzT53/PSIyMjiYmJYciQIezfv9+yfMqUKWzbto1BgwbxwgsvEBERgaNj3X2+vYhIfWQy9FssqQUmvJFAVvbVXWcglWNJxEiOHs2t6RiX5OHRpNZnvBhlrxn1LbuNjQk3tysbWGmELyIiYgVU8EVERKyACr6IiIgVUMEXERGxAir4IiIiVkAFX0RExAqo4IuIiFgB3VpXaoWo0CE1HUH+v4LC4pqOICJVQAVfaoVjx05hNte9e0DVt5t5iEj9pSl9ERERK6CCLyIiYgVU8EVERKyACr6IiIgV0EV7Uitc6VOfahMPjyYUFBaTe7KgpqOIiFyUCr7UCnX98bhLIkaSiwq+iNRemtIXERGxAir4IiIiVkAFX0RExAqo4IuIiFgBFXwREREroIIvIiJiBVTwRURErECdLvipqakEBgZWSduffPIJn3zySZW0XRlCQkKIj48nIyODsWPH1nQcERGp5XTjnYt45JFHajpCuTRr1oz//Oc/NR1DRERquTpf8I8fP87YsWPZv38/LVu2JCoqiqSkJBYuXIjJZKJdu3ZMnTqVa665hjZt2pCWlgZAfHw8Gzdu5M0332TWrFmsXbsWW1tbevfuzbhx44iOjgZg/PjxeHt707dvX37++WdsbW15++23adGiBampqcyYMQNbW1s6derEX3/9xaJFiy6aNSQkhMaNG/Pzzz+Tm5vLK6+8wvLly9m5cyd9+vQhJCSE0tJSIiIi2LhxI6Wlpfj7+/Poo49iGAZvvvkm33//PU2bNqW0tJR77rmH9PR0goKCSElJ4Y8//mD69Onk5+dz/PhxHnvsMYKCgoiOjiYjI4N9+/Zx8OBBhg0bxtNPP33RnCUlJYSHh7Nr1y6ysrJo2bIlMTExNGrUiI8++oiPP/6YJk2acPPNN3PjjTcyfvx41qxZQ1RUFCUlJXh6ejJ9+nRcXV0r948tIiIVVqen9AEOHTpEWFgYX375JVlZWXzyySfMmzePRYsWkZSUROPGjYmJibno/gcPHmTNmjUkJiYSGxvL3r17KSwsLLPN0aNH6datGwkJCXTp0oXFixdTXFzMyy+/TGRkJAkJCTRoUL7PTpmZmSQmJjJhwgRCQ0N57bXXSEhIIC4ujtzcXOLi4gBYtmwZn3/+OatWreKnn37iq6++4vfff2fFihW888477N+//7y2P/vsM5555hmWLl3KRx99xNy5cy3r0tLSWLBgAZ999hnz58/n5MmTF824ZcsW7Ozs+PTTT/nmm28oLCxk9erV7Ny5k8WLFxMfH8+SJUvYt28fcOZD15w5c1iwYAEJCQl4e3sze/bscvWHiIhUjzo/wm/bti0tWrQAoFWrVuTm5uLj42MZXT788MOEhoZedP9mzZrRsGFDhg8fjo+PD88//zwNGzY8b7sePXoA0Lp1a3766Sf++OMP3NzcaNu2LQAPPvggM2fOvGzenj17AtC8eXNat26Nm5sbAC4uLuTk5LB+/Xp27NjBhg0bAMjPzyctLY2//voLX19f7OzsuPbaay3t/F1ISAg//PAD//73v0lLSyM/P9+yrmvXrtjb2+Pm5oaLiwu5ubk4OTldMGOXLl1wcXFh8eLF7N69m71795Kfn8/69evx8fHB0fHMg24GDBjAyZMn2bZtG4cPHyYoKAgAs9mMs7PzZftCRESqT50v+H8fWZtMJpycnMqMXg3DoKSkpMx7k8lkWdagQQM+++wzNm7cyJo1axg+fPgFp+XPfggwmUwYhoGtrS1ms/mK89rZ2V0w+1mlpaUEBwfj6+sLnBk9Ozg4EBkZWeZ4F9r3+eefx8nJCR8fHx544AFWrlx5Xv6/n8PFrFq1iqioKIKCgvD39yc7OxvDMLCxsbngOZeWltK5c2fmzZsHQGFhIXl5dfdBOCIi9VGdn9K/kJSUFE6cOAFAXFwcXbt2BcDV1ZVdu3ZhGAYpKSkA/P7774waNYouXbowadIkWrVqxZ49ey57jJtvvpmTJ09arglISkqqlOxeXl7ExcVRXFxMXl4eI0aMYNu2bXTr1o3k5GSKiorIycnhhx9+OG/ftWvXMmHCBPr06cOmTZuAM8X4Sq1fv57+/fsTEBCAu7s7mzZtorS0lG7durF69WpOnTpFUVERX3/9NSaTiY4dO7J161ZLv7333ntERERcXUeIiEilqvMj/HM5Ojry5JNPEhgYSHFxMe3ateO1114D4KWXXuKpp57C3d2du+66i+zsbG6//XY6derEwIEDady4Mbfddhs9e/bkt99+u+Rx7O3tiYiIYNKkSdjY2NCyZUsaNWp01fmHDx/Ovn37GDp0KCUlJfj7+1s+sPz6668MHDgQd3d3WrVqdd6+48ePZ8SIETg5OdGyZUtuuOEG0tPTrzjDsGHDmDhxIsnJydjb29OpUyfS09MZNmwYQUFBPPzwwzg4OODq6krDhg3x8PDg9ddf5/nnn8dsNtOsWTMiIyOvui9ERKTymIxLze3KRZnNZmbPns24ceNwcHBg4cKFZGRkEBISUtPRqsyePXtYvXo1jz76KABPP/00w4YNo1evXlfd9oQ3EsjKrrtfAyyJGMnRo7k1HeOKeHg0qXOZz1L2mqHsNeNC2W1sTLi5OV5RO/VuhF9dbGxscHFx4cEHH8TOzo4bbriBmTNnMmvWLNatW3fe9u3bty/XRX3V5aeffmL69OkXXDd//nyaNWt23vIbbrjBMstgMpnw9vbGx8enqqOKiEgl0AhfagWN8KtffRvx1BXKXjPqW/aKjPDr5UV7IiIiUpYKvoiIiBVQwRcREbECKvgiIiJWQAVfRETECuhneVIrRIUOqekIV6WgsLimI4iIXJIKvtQKx46dwmyue78Qrcs/9RER66KCL7WCjY2ppiNUmLLXDGWvGcpeM87NXpFz0Y13RERErIAu2hMREbECKvgiIiJWQAVfRETECqjgi4iIWAEVfBERESuggi8iImIFVPBFRESsgAq+iIiIFVDBFxERsQIq+FJlkpKSeOCBB7j//vtZvHjxeet37NhBQEAAffv2ZfLkyZSUlABw6NAhRo4cSb9+/Xj66afJy8ur7uhAxfMnJCTg7e2Nn58ffn5+zJ07t7qjXzb7WZMmTSI+Pt7yvjb0fUWz14V+//bbb/Hz82Pw4ME888wz5OTkAHWj3y+WvS70+zfffMOgQYMYMGAAISEhFBUVAXWj3y+WvUL9bohUgSNHjhg+Pj5Gdna2kZeXZwwaNMjYtWtXmW0GDBhgbNmyxTAMwwgNDTUWL15sGIZhPPHEE8aKFSsMwzCMmJgYIyIionrDG1eXf9q0aUZSUlK1Zz6rPNmPHDliPPnkk0aHDh2MpUuXWpbXdN9fTfba3u+5ublG9+7djSNHjhiGYRhvv/22MX36dMMwan+/Xyp7be/3vLw8w9vb2zh69KhhGIbx/PPPG7GxsYZh1P5+v1T2ivS7RvhSJdatW4eXlxcuLi44ODjQt29fkpOTLesPHjxIQUEBnTp1AsDf35/k5GSKi4vZtGkTffv2LbO8ruQH+PXXX0lISGDw4MFMnDjRMhKqLdnhzKiid+/e9O/f37KsNvR9RbND7e/34uJiwsPDadasGQBt2rTh8OHDdaLfL5Ydan+/Ozg4kJKSgru7O/n5+Rw7dgwnJ6c60e8Xyw4V63cVfKkSmZmZeHh4WN43bdqUjIyMi6738PAgIyOD7OxsHB0dadCgQZnl1a2i+c++Hj9+PMuXL+f6669n2rRp1Rf8AtnOzQ7w+OOPM2zYsDLLakPfVzQ71P5+d3V1pU+fPgAUFBQwf/58+vTpUyf6/WLZz+atzf0OYGdnx+rVq/Hx8SE7Oxtvb+860e8Xy34275X2uwq+VAnjAg9hNJlMl11/uf2qS0XzA7z77rt07NgRk8nE448/zpo1a6ou6AVUtA9rQ99fTYa60u+5ubmMHTuWtm3bMnTo0DrV7+dmh7rT7/fddx+pqan4+PgQHh5ep/r93OxQsX5XwZcq0axZM7KysizvMzMzadq06UXXHz16lKZNm3Lttddy6tQpSktLyyyvbhXNn5uby3//+1/LcsMwLCOI6nK57BdTG/q+otnrSr9nZmYyYsQI2rZty8yZM4G60+8Xyl4X+v3EiRP8+OOPlveDBg0iLS2tTvT7xbJXtN9V8KVK3Hvvvaxfv57jx49z+vRpvv76a3r27GlZf8MNN9CwYUN+/vln4MwVpz179sTOzo67776bL774oszyupLfwcGBDz74gG3btgHw8ccfc//999eq7BdTG/q+otnrQr+Xlpby1FNP0b9/fyZPnmwZydWFfr9Y9rrQ74ZhEBwczKFDhwD48ssv6dy5c53o94tlr3C/V+zaQpHLS0xMNAYMGGD4+voa8+fPNwzDMB5//HHjl19+MQzDMHbs2GEEBAQY/fr1M1588UWjsLDQMAzDSE9PN0aNGmX079/f+Ne//mWcOHGiTuXftGmTMWTIEKNfv37GU089ZZw8ebLWZT9r0qRJZa50rw19X9Hstb3fv/76a6NNmzbG4MGDLf9eeeUVwzBqf79fKntt73fDMIxvvvnGGDhwoDFo0CDjhRdesGSs7f1+qewV6XeTYVzgSwQRERGpVzSlLyIiYgVU8EVERKyACr6IiIgVUMEXERGxAir4IiIiVkAFX0TqhF69evHrr79W6zG///573nnnnWo9pkhVUcEXEbmIX3/9tdofBiNSVar3HogiIlfpjjvu4NFHH+X777/n1KlTBAcHk5yczB9//EHTpk2ZN28eDg4O3H777YwePZrU1FTy8/N58cUX8fX1Bc7ch3zlypXY2trSsmVLpk6dioeHB4GBgTg7O7N7924eeOABYmNjKS0tpUmTJjz55JOEh4ezd+9ecnJyuOaaa5g9ezY333wzgYGBdOrUic2bN3P48GHuuusuZs2ahY2NDd999x1vv/02ZrMZBwcHXnvtNdq2bcvmzZuZPXs2p0+fxmQyMX78eHx8fGq4d6U+U8EXkTqlqKgIDw8PkpKSmD9/PlOmTOHLL7/Ew8ODBx98kFWrVjFo0CBKS0txdnYmPj6enTt3MmrUKO6++26+++47fvjhBz7//HMcHByIjo4mJCSEBQsWAODk5GS53aphGGRnZ/PCCy+QnJyMk5MTcXFxAISFhbF48WKmTp0KwP79+1m0aBH5+fn079+fjRs3cssttxAcHMyiRYu47bbb+Prrr5k9ezZz5swhNDSUBQsW4OnpSUZGBg899BBt2rShefPmNdOxUu+p4ItInXP2GeY33ngjt956q+U57Z6enmWm4EeNGgVA27ZtufXWW9m0aRNr1qzB398fBwcHAIKCgpg3bx5FRUUA3H333Rc8Zr9+/WjRogWLFi1i3759bNy4kTvvvNOy3sfHBxsbGxwdHbnpppvIyclh8+bNtG7dmttuuw0AX19ffH19Wb16NUePHuXZZ5+17G8ymUhLS1PBlyqjgi8idY6dnd0FX5/L1tbW8tpsNmNra3veI0nNZjMlJSWW92c/CJxryZIlxMXFMXLkSAYNGoSLiwvp6emW9Y0aNbK8Pvuo5wYNGpz3WOW0tDRKS0tp1aoVn332mWVdRkYG11577aVOW+Sq6KI9Eam3EhISAPjtt9/Ys2cPXbp0wdvbm/j4ePLz8wFYtGgRXbp0wd7e/rz9bW1tLR8GfvzxR4YOHcqwYcNo2bIlKSkplkerXkzHjh3566+/2LVrFwCrVq0iODiYTp06sW/fPjZt2gTAjh076Nu3L5mZmZV27iLn0ghfROqtzZs3ExcXh9lsZu7cuTg7O/Pggw9y+PBhhg0bhtls5qabbmL27NkX3L9bt26MHz8eOzs7/vWvfxEWFkZ8fDy2tra0a9eOP/7445LHd3d3Z/bs2UyaNInS0lIcHR2ZO3cu1157LVFRUURERFBYWIhhGERERHDDDTdURTeIAKCn5YlIvdSmTRvWr1+vaXKR/09T+iIiIlZAI3wREREroBG+iIiIFVDBFxERsQIq+CIiIlZABV9ERMQKqOCLiIhYARV8ERERK/D/ADe81VGX3qTFAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "feature_importance = rf_feature_importance(model, X_keep)\n", "plot_feature_importance(feature_importance);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Save model to disk" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally let's save this model to disk for later use or if we want to deploy it to production:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pickle\n", "\n", "with open(DATA/'housing_model.pkl', mode='wb') as file:\n", " pickle.dump(model, file)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# sanity check\n", "with open(DATA/'housing_model.pkl', mode='rb') as file:\n", " model = pickle.load(file)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RMSE on train: 14963.277\n", "R^2 on train: 0.976\n", "RMSE on valid: 41200.693\n", "R^2 on valid: 0.823\n", "OOB R^2: 0.823\n" ] } ], "source": [ "print_rf_scores(model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> Note: We often use the `with` statement to read or write data in Python as this automatically takes care of closing the file once we have finished with it. The `mode` argument controls how we want to open the file, where `'rb'` (read) and `'wb'` (write) correspond to opening in binary mode." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "#### Exercise #3\n", "\n", "By this stage you've now learned all the key steps needed to clean data, train a machine learning model, and interpret the results! Try applying the techniques you have learned to Kaggle's [House Prices: Advanced Regression Techniques](https://www.kaggle.com/c/house-prices-advanced-regression-techniques) competition. Don't spend too much time trying to build a perfect model - the purpose of this exercise is to get familiar with downloading a new dataset, understanding the evaluation metric, and submitting your predictions to the Kaggle platform. To get started:\n", "\n", "* Create an account on [Kaggle](https://www.kaggle.com/)\n", "* Download the data from [here](https://www.kaggle.com/c/house-prices-advanced-regression-techniques/data)\n", "\n", "When you have your predictions ready, submit them [here](https://www.kaggle.com/c/house-prices-advanced-regression-techniques/submissions) for evaluation on the public leaderboard!\n", "\n", "Hint #1: Kaggle provides 2 datasets called `train.csv` and `test.csv`, where the latter is missing the target column `SalePrice`. Your task is to build a regression model on `train.csv` and then use that model to make _predictions_ on `test.csv`. \n", "\n", "Hint #2: A closer look at Kaggle's evaluation metric shows that we actually want the RMSE between the _logarithms_ of the predicted and actual house prices:\n", "\n", "> Submissions are evaluated on Root-Mean-Squared-Error (RMSE) between the logarithm of the predicted value and the logarithm of the observed sales price. (Taking logs means that errors in predicting expensive houses and cheap houses will affect the result equally.)\n", "\n", "How might you handle this? You may find NumPy's [log function](https://docs.scipy.org/doc/numpy/reference/generated/numpy.log.html) to be useful here.\n", "\n", "Hint #3: You may find the `convert_strings_to_categories` function from lesson 3 to be useful. You can import it from the `dslectures` library as follows\n", "\n", "```python\n", "from dslectures.core import convert_strings_to_categories\n", "```\n", "\n", "Hint #4: When you use the median to fill missing values in the training set, you should use the _same_ median for the test set. See p.61 of _Hands-On Machine Learning with Scikit-Learn and TensorFlow_ by A. Geron.\n", "\n", "---\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 4 }