{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "acceptable-netherlands", "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "Today we will look at ensemble learning, which is an often undervalued part of machine learning. The way it works is\n", "by taking multiple different classifiers, training them, and then averaging their predictions into one \"ensemble\" prediction. \n", "You often want to have a diverse set of classifiers so you can aggregate predictions. The reason this works is because even if you have", "subpar classifiers with a 60% chance of predicting correctly on their own - as a group together it will increase more \n", "and more as long as these classifiers all have different learnt parameters. This can be suprisingly effective. \n", "The two algorithms we will look at are Random Forests and Ensemble Classifiers. \n", "\"\"\"\n", "\n", "#let's get started\n", "import sealion as sl \n", "from sealion.ensemble_learning import RandomForest, EnsembleClassifier" ] }, { "cell_type": "code", "execution_count": 2, "id": "aquatic-great", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PassengerIdSurvivedPclassNameSexAgeSibSpParchTicketFareCabinEmbarked
0103Braund, Mr. Owen Harrismale22.010A/5 211717.2500NaNS
1211Cumings, Mrs. John Bradley (Florence Briggs Th...female38.010PC 1759971.2833C85C
2313Heikkinen, Miss. Lainafemale26.000STON/O2. 31012827.9250NaNS
3411Futrelle, Mrs. Jacques Heath (Lily May Peel)female35.01011380353.1000C123S
4503Allen, Mr. William Henrymale35.0003734508.0500NaNS
.......................................
88688702Montvila, Rev. Juozasmale27.00021153613.0000NaNS
88788811Graham, Miss. Margaret Edithfemale19.00011205330.0000B42S
88888903Johnston, Miss. Catherine Helen \"Carrie\"femaleNaN12W./C. 660723.4500NaNS
88989011Behr, Mr. Karl Howellmale26.00011136930.0000C148C
89089103Dooley, Mr. Patrickmale32.0003703767.7500NaNQ
\n", "

891 rows × 12 columns

\n", "
" ], "text/plain": [ " PassengerId Survived Pclass \\\n", "0 1 0 3 \n", "1 2 1 1 \n", "2 3 1 3 \n", "3 4 1 1 \n", "4 5 0 3 \n", ".. ... ... ... \n", "886 887 0 2 \n", "887 888 1 1 \n", "888 889 0 3 \n", "889 890 1 1 \n", "890 891 0 3 \n", "\n", " Name Sex Age SibSp \\\n", "0 Braund, Mr. Owen Harris male 22.0 1 \n", "1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 \n", "2 Heikkinen, Miss. Laina female 26.0 0 \n", "3 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 \n", "4 Allen, Mr. William Henry male 35.0 0 \n", ".. ... ... ... ... \n", "886 Montvila, Rev. Juozas male 27.0 0 \n", "887 Graham, Miss. Margaret Edith female 19.0 0 \n", "888 Johnston, Miss. Catherine Helen \"Carrie\" female NaN 1 \n", "889 Behr, Mr. Karl Howell male 26.0 0 \n", "890 Dooley, Mr. Patrick male 32.0 0 \n", "\n", " Parch Ticket Fare Cabin Embarked \n", "0 0 A/5 21171 7.2500 NaN S \n", "1 0 PC 17599 71.2833 C85 C \n", "2 0 STON/O2. 3101282 7.9250 NaN S \n", "3 0 113803 53.1000 C123 S \n", "4 0 373450 8.0500 NaN S \n", ".. ... ... ... ... ... \n", "886 0 211536 13.0000 NaN S \n", "887 0 112053 30.0000 B42 S \n", "888 2 W./C. 6607 23.4500 NaN S \n", "889 0 111369 30.0000 C148 C \n", "890 0 370376 7.7500 NaN Q \n", "\n", "[891 rows x 12 columns]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"\"\"\n", "Random Forests are just a collection of decision trees that's predictions are aggregated. The way this works\n", "is by having each decision tree trained on a different set of data - so you can have a diverse set of trees that's predictions\n", "will then be averaged and given to the user. Each tree was trained on a different set of data, so they won't be the same. We also \n", "have some added functionality as well which you will see in a bit. We are going to use the titanic dataset with the \n", "same preprocessing steps as the decision trees examples, so feel free to skip down until this \"X_train, X_test = new_X_train, new_X_test\" cell.\n", "These random forests can be trained in parallel on multiple CPU cores, which is exactly what SeaLion does. \n", "\"\"\"\n", "\n", "import pandas as pd\n", "# first we can load in the dataset\n", "titanic_dataframe = pd.read_csv(\"titanic_dataset.csv\") # of my local computer\n", "titanic_dataframe # print it out" ] }, { "cell_type": "code", "execution_count": 3, "id": "proud-story", "metadata": {}, "outputs": [], "source": [ "# looks like it has 891 rows and 12 columns. First we can delete some of the features we know we won't use. \n", "titanic_dataframe = titanic_dataframe.drop(['Name', 'Ticket', 'Cabin'], axis = 1) # non-numeric\n", "titanic_dataframe = titanic_dataframe.fillna(0)" ] }, { "cell_type": "code", "execution_count": 4, "id": "backed-latter", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PassengerIdSurvivedPclassSexAgeSibSpParchFareEmbarked
0103022.0107.25000
1211138.01071.28331
2313126.0007.92500
3411135.01053.10000
4503035.0008.05000
..............................
88688702027.00013.00000
88788811119.00030.00000
8888890310.01223.45000
88989011026.00030.00001
89089103032.0007.75002
\n", "

891 rows × 9 columns

\n", "
" ], "text/plain": [ " PassengerId Survived Pclass Sex Age SibSp Parch Fare Embarked\n", "0 1 0 3 0 22.0 1 0 7.2500 0\n", "1 2 1 1 1 38.0 1 0 71.2833 1\n", "2 3 1 3 1 26.0 0 0 7.9250 0\n", "3 4 1 1 1 35.0 1 0 53.1000 0\n", "4 5 0 3 0 35.0 0 0 8.0500 0\n", ".. ... ... ... .. ... ... ... ... ...\n", "886 887 0 2 0 27.0 0 0 13.0000 0\n", "887 888 1 1 1 19.0 0 0 30.0000 0\n", "888 889 0 3 1 0.0 1 2 23.4500 0\n", "889 890 1 1 0 26.0 0 0 30.0000 1\n", "890 891 0 3 0 32.0 0 0 7.7500 2\n", "\n", "[891 rows x 9 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# next we can change the sex column. Female will be 0 and Male will be 1. \n", "import numpy as np # we'll need this too\n", "sex_col = np.array(titanic_dataframe['Sex'])\n", "sex_col[np.where(sex_col == \"male\")] = 0\n", "sex_col[np.where(sex_col == \"female\")] = 1\n", "titanic_dataframe[\"Sex\"] = sex_col\n", "\n", "# we can also change the embarked column - we will make it one-hot-encoded\n", "from sealion.utils import one_hot\n", "embarked_col = np.array(titanic_dataframe[\"Embarked\"])\n", "embarked_col[np.where(embarked_col == \"S\")] = 0\n", "embarked_col[np.where(embarked_col == \"C\")] = 1\n", "embarked_col[np.where(embarked_col == \"Q\")] = 2 \n", "titanic_dataframe[\"Embarked\"] = embarked_col\n", "titanic_dataframe" ] }, { "cell_type": "code", "execution_count": 5, "id": "lonely-building", "metadata": {}, "outputs": [], "source": [ "# looks like we are all set. Time to get the labels and the training and testing data\n", "y = np.array(titanic_dataframe['Survived'])\n", "titanic_dataframe = titanic_dataframe.drop('Survived', axis = 1)\n", "X = np.array(titanic_dataframe)\n", "\n", "from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.15, random_state = 42)" ] }, { "cell_type": "code", "execution_count": 6, "id": "anonymous-discipline", "metadata": {}, "outputs": [], "source": [ "#we'll need to one_hot_encode the last column of X_train and X_test (Embarked)\n", "embarked_train = one_hot(X_train[:, -1], depth = 3)\n", "embarked_test = one_hot(X_test[:, -1], depth = 3)" ] }, { "cell_type": "code", "execution_count": 7, "id": "located-parking", "metadata": {}, "outputs": [], "source": [ "# we'll have to use a bit of the long route to avoid typical rules of numpy\n", "new_X_train, new_X_test = [], []\n", "X_train, X_test = np.array(X_train).tolist(), np.array(X_test).tolist() # turn them into regular python lists\n", "for row in range(len(X_train)) : \n", " observation = X_train[row] # get the row\n", " observation[-1] = embarked_train[row].tolist() # .tolist() helps make sure it can be interpreted (only needed as of v3.0.8 if you are using one_hot_encoded data) \n", " new_X_train.append(observation)\n", " \n", "for row in range(len(X_test)) : \n", " observation = X_test[row] # get the row\n", " observation[-1] = embarked_test[row].tolist()\n", " new_X_test.append(observation)" ] }, { "cell_type": "code", "execution_count": 8, "id": "relative-lighter", "metadata": {}, "outputs": [], "source": [ "X_train, X_test = new_X_train, new_X_test # just change the name" ] }, { "cell_type": "code", "execution_count": 25, "id": "conceptual-teach", "metadata": {}, "outputs": [], "source": [ "# yay now we have our data. We can now apply the random forest algorithm. \n", "\n", "rf = RandomForest(num_classifiers = 20, replacement = True, min_data = 50)\n", "\n", "# a quick word on the arguments\n", "# num_classifiers is just the amount of trees to be made\n", "# max_branches and min_samples are the same arguments as in decision trees\n", "# replacement is whether or not to boostrap. When deciding the dataset for each of the 20 trees, you may \n", "# want to not have any data points shared across the datasets or be fine with some. In general using bootstrapping\n", "# does better on testing data at the expense of training data. \n", "# min_data is simply the minimum amount of data points you need each set to have. If you set replacement = True, \n", "# then what will happen is that each decision tree will get a different amount of data, one may get 5 samples and \n", "# another may get 500. So if you want to ensure that at least all datasets get some X amount of data you can set that there. \n", "\n", "rf.fit(X_train, y_train) # let's train it!" ] }, { "cell_type": "code", "execution_count": 28, "id": "preliminary-lawrence", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Validation accuracy : 0.96\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxoklEQVR4nO3deXxU5fn38c9FCGIQkQL2kS2BSllkCRhBHqhLlUVqtXX5qQ9WcUsLLl2pWH+2tpVf7WMflyqiaYu0mrrXSq0iVbHWuoZFRBQXBEy0ElBQQCQJ1/PHOYmTcJKZycxkBvm+X695Zc597nPf1zknmStnmXObuyMiItJUu2wHICIiuUkJQkREIilBiIhIJCUIERGJpAQhIiKRlCBERCSSEoTkNDObb2ZXhe+/YmarW9nOLWZ2RXqjyx1m9qSZnd/Wy7bQppvZwelsU9qeEoSkzMzWmtknZrbVzN4PP9T3S3c/7v4vdx+YQDzTzOzpJst+x91/me6Y4sSxj5n9ycw+NLMNZnZDAsusNbNj2yK+VJhZUZgEtoavtWY2qxXt7LavJHcoQUi6fN3d9wNGASXAfzetYGbt2zyq7JpGsD36A/2Av2YzmAw5INzvZwA/NbPJ2Q5I0kcJQtLK3auAR4Ch0HCq4UIzewN4Iyw73syWm9lmM3vGzIbXL29mI81sqZl9bGZ3Ax1j5h1lZpUx033M7C9mVm1mm8zsJjMbDNwCjA3/s90c1m04VRVOX2Bmb5rZB2a2wMx6xsxzM/uOmb0RxjjHzCycd7CZ/dPMtpjZxjDG5tQAW9z9Q3ff5u6LW7tdzayrmT0UruuH4fveTap9ycxeMLOPzOxBM/tCzPKHh9t6s5m9ZGZHNdNPMuvXwN2fBV4h3O9N2uwSHklVm9k6M/tvM2vX3L6S3KEEIWllZn2AKcCymOJvAGOAIWY2EpgHfBvoBtwKLAhPx3Qg+C/7duALwL3Ayc30kwc8BKwDioBewF3u/irwHeBZd9/P3Q+IWParwK+A/wIOCtu4q0m144HDgOFhvUlh+S+BRUBXoDdwYwubYwlwuJn9ooU6iWoH3AYUAn2BT4CbmtQ5CziXYJ1qgd8CmFkv4O/AVQTb9UfA/WbWI6KfZNaPsH0zs3HAITTe7/VuBLoQHEkdGcZ5TiL7SrJLCULS5a/hf4BPA/8E/idm3q/c/QN3/wQoBW519+fdvc7d/wh8ChwevvKB6929xt3vA15spr/RQE9gZvjf+Q53T/Rc9lRgnrsvdfdPgcsI/ostiqlztbtvdvf1wGKgOCyvIfiQ7tlSn+F/738DvgZMMrMrY+ZVmtmwBGMFwN03ufv97r7d3T8GZhN82Ma63d1Xuvs24Argv8JEeibwsLs/7O673P0fQAVBIm8qofWLsRH4APg9MMvdH4+dGfZ/OnCZu3/s7muB/wd8K4nVlyxRgpB0+Ya7H+Duhe4+I0wG9d6JeV8I/DA81bE5TCp9CD7sewJV3vgJkuua6a8PsM7da1sRa8/Ydt19K7CJ4Cik3n9i3m8H6i+6/xgw4AUze8XMzm2mj1OBV919IcEH8almdmWYhNoDK5MJ2MwKzOzW8BTNR8BTwAHhB3C92O28jiDZdifY5qc22ebjCY40mkp0/ep1d/eu7j7Y3X8bNT+MI3Y/rqPxtpYctbddNJTsiP3AfweY7e6zm1YysyOBXmZmMUmiL/BWRJvvAH3NrH1Ekoj3iOJ3CT406/vtRHC6qyrOcrj7f4ALwuXGA4+Z2VPu/maTqu0JPhhx901mNgH4N8HF3N80SYKJ+CEwEBjj7v8xs2KC0zkWU6dPzPu+BEcDGwm21e3ufkEa1y9RG/nsqGRVTGz121qPk85hOoKQtvY74DtmNiY8d93JzL5mZp2BZwnOnV9iZvlmdhLBqaQoLwDvAVeHbXQMz4MDvA/0Dq9pRLkTOMfMis1sH4LTYc+Hpz9aZGanxlwc/pDgA25XRNWHgcPM7Ntmlk/wIfkM8GWCI5KW5IfrU/9qD3QmuO6wOTx99bOI5c40syFmVgD8ArjP3euAO4Cvm9kkM8sL2zwq4iJ3MuuXkLD/e4DZZtbZzAqBH4QxQfx9JVmkBCFtyt0rCP5DvYngA+hNgttBcfedwEnh9AfAacBfmmmnDvg6cDCwHqgM6wM8QXBHzX/MbGPEso8RnKO/nyDJfIngPHkiDgOeN7OtwALgu+6+JqKPt4HjCC7IbgJeIvgwPBr4tbV8O+jDBMmg/nUlcD2wL8F/5M8BCyOWux2YT3B6rCNwSRjLO8CJwE+AaoIjiplE//0ntH5JuhjYBqwhuEb1Z4IbFSDOvpLsMg0YJCIiUXQEISIikZQgREQkkhKEiIhEUoIQEZFIn6vvQXTv3t2LioqyHYaIyB5jyZIlG9096rErn68EUVRUREVFRbbDEBHZY5hZc08r0CkmERGJpgQhIiKRlCBERCSSEoSIiERSghARkUgZSxAWDAe52MxWhc+V/25EHTOz31ow9OMKMxsVM+9sC4Z8fMPMzs5UnPHMmFtO+5lF2JXtaD+ziBlzy9u8j/JyKCqCdu2Cn+XlyS2fSH+ZXs+m7R9y6Yyk+0s1xnTEkKxU900iMaa6XZr+fh37g/THUP5yOUXXF9Hu5+0our6IY69tuc10/E6nWzr6aIs40yljD+szs4OAg9x9afgo5yUEg8qsiqkzheBJj1MIhqS8wd3HhI8zrgBKCB43vAQ41N0/bKnPkpIST+dtrjPmljO3qhTyY57OXFPA9F5l3Dx9apv0UV4OpaWwPWZ2QQGUlcHUqcnHGFm/Nh8waL8zI+sZ2afTeCSDOP2lui/SEUOy0rJv4sSY6nbZ7fdrWDl8vRQ6pC+G8pfLKf1bKdtrEmszLdsty/syU21kgpktcfeSyHlt9TRXM3sQuCkc7rC+7FbgSXe/M5xeDRxV/3L3b0fVa066E0T7mUXU7bf7LcJ5WwupvWZtm/RRVATrIu5SLiyEtWuTj7G5+lHStZ6J9tlSf6nui3TEkKxM7ZvY5VPdLrv9fn2vCA5IbwxF1xexbkvibaZru2VzX2aqjUxoKUG0yTWIcJjFkcDzTWb1ovEwiZVhWXPlUW2XmlmFmVVUV1enLWaAuk7rkyrPRB/rm+mqvjzZGJOJPV3rmWg7LdVLdX3SXS+VttK5Lqm2tdvvV5f0x7B+S3Jtpmu7ZXNfZqqNtpbxBGFm+xEMzPI9d/8o3e27e5m7l7h7SY8ekd8Wb7W8bX2TKs9EH32b6aq+PNkYk4k9XeuZaDst1Ut1fdJdL5W20rkuqba12+/XlvTH0LdLcm2ma7tlc19mqo22ltEEEQ61eD9Q7u5RI4NV0Xgc3d5hWXPlbaq0/2yoKWhcWFMQlLdRH7NnB9ccYhUUBOWtiTGyfm0+1DYZ8TGN6xnZZ9Mzm3H6S3VfpCOGZKVl38SJMdXtstvv1+OzYWd6Y5h9zGwK8hNvMy3bLcv7MlNttLVM3sVkwB+AV9392maqLQDOCu9mOhzY4u7vAY8CE82sq5l1BSaGZW3q5ulTmd6rjLytheBG3tbCtF9QitfH1KnBBenCQjALftZfoG5NjJH1e97G9J7zMraeUX0O+WR6Uv2lui/SEUOy0rFv4sWY6nbZ7ffro6kc80l6Y5g6bCplXy+jsEshhlHYpZBj9m++zbT8Tmd5X2YrznTL5F1M44F/AS/z2aDnPwH6Arj7LWESuQmYTDCQ+znhmMWY2blhfYDZ7n5bvD7TfZFaROTzrqWL1Bl7mqu7P03jG9mi6jhwYTPz5vHZwOYiItLG9E1qERGJpAQhIiKRlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIJCUIERGJpAQhIiKRlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIpIwNGGRm84DjgQ3uPjRi/kygfqy99sBgoIe7f2Bma4GPgTqgtrnRjkREJHMyeQQxn2Ao0Ujufo27F7t7MXAZ8E93/yCmytHhfCUHEZEsyFiCcPengA/iVgycAdyZqVhERCR5Wb8GYWYFBEca98cUO7DIzJaYWWmc5UvNrMLMKqqrqzMZqojIXiXrCQL4OvDvJqeXxrv7KOA44EIzO6K5hd29zN1L3L2kR48emY5VRGSvkQsJ4nSanF5y96rw5wbgAWB0FuISEdmrZTVBmFkX4EjgwZiyTmbWuf49MBFYmZ0IRUT2Xpm8zfVO4Cigu5lVAj8D8gHc/Zaw2jeBRe6+LWbRLwIPmFl9fH9294WZilNERKJlLEG4+xkJ1JlPcDtsbNkaYERmohIRkUTlwjUIERHJQUoQIiISSQlCREQiKUGIiEgkJQgREYmkBCEiIpGUIEREJJIShIiIRFKCEBGRSEoQIiISSQlCREQiKUGIiEgkJQgREYmkBCEiIpGUIEREJJIShIiIRMpYgjCzeWa2wcwihws1s6PMbIuZLQ9fP42ZN9nMVpvZm2Y2K1MxiohI8zJ5BDEfmBynzr/cvTh8/QLAzPKAOcBxwBDgDDMbksE4RUQkQsYShLs/BXzQikVHA2+6+xp33wncBZyY1uBERCSubF+DGGtmL5nZI2Z2SFjWC3gnpk5lWBbJzErNrMLMKqqrqzMZq4jIXiWbCWIpUOjuI4Abgb+2phF3L3P3Encv6dGjRzrjExHZq2UtQbj7R+6+NXz/MJBvZt2BKqBPTNXeYZmIiLShrCUIM/tfZmbh+9FhLJuAF4EBZtbPzDoApwMLshWniMjeqn2mGjazO4GjgO5mVgn8DMgHcPdbgFOA6WZWC3wCnO7uDtSa2UXAo0AeMM/dX8lUnCIiEs2Cz+TPh5KSEq+oqMh2GCIiewwzW+LuJVHzsn0Xk4iI5CglCBERiaQEISIikZQgREQkkhKEiIhEUoIQEZFIShAiIhJJCUJERCIpQYiISCQlCBERiaQEISIikZQgREQkkhKEiIhEUoIQEZFIShAiIhIpYwnCzOaZ2QYzW9nM/KlmtsLMXjazZ8xsRMy8tWH5cjPTAA8iIlmQySOI+cDkFua/DRzp7sOAXwJlTeYf7e7FzQ1kISIimZWxIUfd/SkzK2ph/jMxk88BvTMVi4iIJC9XrkGcBzwSM+3AIjNbYmalLS1oZqVmVmFmFdXV1RkNUkRkb5KxI4hEmdnRBAlifEzxeHevMrMDgX+Y2Wvu/lTU8u5eRnh6qqSk5PMzwLaISJZl9QjCzIYDvwdOdPdN9eXuXhX+3AA8AIzOToQiInuvrCUIM+sL/AX4lru/HlPeycw6178HJgKRd0KJiEjmJHSKyczGAVcCheEyBri7929hmTuBo4DuZlYJ/AzIJ1jwFuCnQDfgZjMDqA3vWPoi8EBY1h74s7svbMW6iYhIChK9BvEH4PvAEqAukQXc/Yw4888Hzo8oXwOM2H0JERFpS4kmiC3u/kj8aiIi8nmRaIJYbGbXEFwz+LS+0N2XZiQqERHJukQTxJjwZ+y3mh34anrDEZFMqqmpobKykh07dmQ7FGljHTt2pHfv3uTn5ye8TEIJwt2PbnVUIpIzKisr6dy5M0VFRYQ3gshewN3ZtGkTlZWV9OvXL+HlErrN1cy6mNm19d9YNrP/Z2ZdWh2tiGTFjh076Natm5LDXsbM6NatW9JHjol+D2Ie8DHwX+HrI+C2pHoSkZyg5LB3as1+T/QaxJfc/eSY6Z+b2fKkexMRkT1GokcQn5hZw7OSwi/OfZKZkEREEvPkk09y/PHHA7BgwQKuvvrqZutu3ryZm2++uWH63Xff5ZRTTkl7TM899xwXXHBB2ttNxrRp07jvvvtSbifRBDEdmBMO5LMOuAn4Tsq9i0hOKy+HoiJo1y74WV7eNv3W1SX0fdxGTjjhBGbNmtXs/KYJomfPnmn5EG3qkUceYfLklobCaZ3a2tq0txlPQgnC3Ze7+whgODDM3Ue6+0uZDU1Esqm8HEpLYd06cA9+lpamniTWrl3LoEGDmDp1KoMHD+aUU05h+/btFBUVcemllzJq1CjuvfdeFi1axNixYxk1ahSnnnoqW7duBWDhwoUMGjSIUaNG8Ze//KWh3fnz53PRRRcB8P777/PNb36TESNGMGLECJ555hlmzZrFW2+9RXFxMTNnzmTt2rUMHToUCC7en3POOQwbNoyRI0eyePHihjZPOukkJk+ezIABA/jxj38MBAls2rRpDB06lGHDhnHdddc1xPH4449z7LHHcsQRR7B8+fKG8vHjx/PSS9Efm1deeSXf+ta3GDt2LAMGDOB3v/sdEBwhfeUrX+GEE05gyJAh1NXVMXPmTA477DCGDx/OrbfeCgR3KV100UUMHDiQY489lg0bNqS2k0ItXoMwszPd/Q4z+0GT8vqgrk1LFCKScy6/HLZvb1y2fXtQPnVqam2vXr2aP/zhD4wbN45zzz234T/7bt26sXTpUjZu3MhJJ53EY489RqdOnfj1r3/Ntddey49//GMuuOACnnjiCQ4++GBOO+20yPYvueQSjjzySB544AHq6urYunUrV199NStXrmz40F67dm1D/Tlz5mBmvPzyy7z22mtMnDiR118PniG6fPlyli1bxj777MPAgQO5+OKL2bBhA1VVVaxcGTxHdPPmzQBs3LiR/Px8unTpwnnnncf8+fO5/vrref3119mxYwcjRjT/FKEVK1bw3HPPsW3bNkaOHMnXvvY1AJYuXcrKlSvp168fZWVldOnShRdffJFPP/2UcePGMXHiRJYtW8bq1atZtWoV77//PkOGDOHcc89NZRcB8Y8gOoU/O0e89ku5dxHJWevXJ1eejD59+jBu3DgAzjzzTJ5++mmAhg/85557jlWrVjFu3DiKi4v54x//yLp163jttdfo168fAwYMwMw488wzI9t/4oknmD59OgB5eXl06dLyXflPP/10Q1uDBg2isLCwIUEcc8wxdOnShY4dOzJkyBDWrVtH//79WbNmDRdffDELFy5k//33B2DRokVMnDgRgFNPPZWHHnqImpoa5s2bx7Rp01qM4cQTT2Tfffele/fuHH300bzwwgsAjB49uuG7C4sWLeJPf/oTxcXFjBkzhk2bNvHGG2/w1FNPccYZZ5CXl0fPnj356lfT8x3mFo8g3P3W8O1j7v7v2HnhhWoR+Zzq2zc4rRRVnqqmt1zWT3fqFPxP6u5MmDCBO++8s1G92FM2bWWfffZpeJ+Xl0dtbS1du3blpZde4tFHH+WWW27hnnvuYd68eTzyyCP84AfBCZeCggImTJjAgw8+yD333MOSJUta7CfeNoFgu9x4441MmjSpUd2HH344pXVsTqIXqW9MsExEPidmz4aCgsZlBQVBearWr1/Ps88+C8Cf//xnxo8f32j+4Ycfzr///W/efPNNALZt28brr7/OoEGDWLt2LW+99RbAbgmk3jHHHMPcuXOB4HrBli1b6Ny5Mx9//HFk/a985SuUhxdXXn/9ddavX8/AgQObjX/jxo3s2rWLk08+mauuuoqlS5fi7qxYsYLi4uKGeueffz6XXHIJhx12GF27dm1xmzz44IPs2LGDTZs28eSTT3LYYYftVmfSpEnMnTuXmpqahli3bdvGEUccwd13301dXR3vvfdewzWUVLWYIMxsrJn9EOhhZj+IeV0J5KUlAhHJSVOnQlkZFBaCWfCzrCz16w8AAwcOZM6cOQwePJgPP/yw4XRQvR49ejB//nzOOOMMhg8fztixY3nttdfo2LEjZWVlfO1rX2PUqFEceOCBke3fcMMNLF68mGHDhnHooYeyatUqunXrxrhx4xg6dCgzZ85sVH/GjBns2rWLYcOGcdpppzF//vxGRw5NVVVVcdRRR1FcXMyZZ57Jr371K5YsWcLIkSMbHQkceuih7L///pxzzjlxt8nw4cM5+uijOfzww7niiivo2bPnbnXOP/98hgwZwqhRoxg6dCjf/va3qa2t5Zvf/CYDBgxgyJAhnHXWWYwdOzZuf4kw9+aHcTazIwkG/fkOcEvMrI+Bv7n7G2mJIk1KSkq8oqIi22GI5KxXX32VwYMHZzWGtWvXcvzxxzdc4P28uOqqqzj44IM5/fTTG8reffddjjrqKF577TXatWv+//Err7yS/fbbjx/96EcZjTFq/5vZknCwtt3EuwbxT+CfZjbf3SPORrbMzOYBxwMb3H1oxHwDbgCmANuBafWPEDezs4H/Dqte5e5/TLb/RMyYW07Zmsup67SevG19GdhuCqt3PdwwXdp/NjdPb/2/TE3bL+0fHJ83LWupj/KXy7n88ctZv2U9fbv0ZfYxs5k6rPn6UX0muw7xtku86VS3W2ukY72TbTPVPluzfLpj2rR9E1UfV7Gzbicd8jrQq3MvuhV0S3gdoqzbsInqT6sgbyfUdaDHPr0oPLD1baajvXTH1NS3SqdT/WkVFe9WQF0Hnnrwaa779a+49tprm00O9TG9+/G77FvbmXUbNqU1plS1eATRUMnsH8Cp7r45nO4K3OXuk+IsdwSwFfhTMwliCnAxQYIYA9zg7mPM7AtABcHjxZ1gJLtD3f3DlvpL9ghixtxy5laVQn7MvXxOMKBqvZoCpvcqa9UHTWT7tflBB+13JtRH+cvllP6tlO01n7VRkF9A2dfLIpNEZJ9JrkNC2yXedArbrTXSsd7Jtplqn61ZPtWYmv4HuWn7JtZtWccu39VQ1s7aUdilsNVJYt2GTVTXrAP7rE28HT3yC1v14ZeO9tIdU2vbv+2227jhhhsA2FlbR63vZMRhw7n0fy5Ne0xRkj2CSDRBLHP3kfHKmlm2CHiomQRxK/Cku98ZTq8mOKV1FHCUu387ql5zkk0Q7WcWUbdf/AOjvK2F1F6zNuF2k22/pT6Kri9i3Zbd2yjsUsja7+1ev7k+k1mHZOJuSWu3W2ukY72TbTPVPluzfKoxNf2AWPH+CnbW7dytfoe8Dgz/4vC46xCl4p0VwX/pTdV1oKRP8m2mo710x5SO9jMdU5RkE0SidzHtMrOGm9vMrJDgf8ZU9QLeiZmuDMuaK9+NmZXWP4a8uro6qc7rOiV2Q3ei9VJZrrm667ckV95cO+mIJVnpaieVvlKJIV6bqfbZmuXTHVNUcmipPCFRH3otlbdFe+mOKR3tZzqmNEg0QVwOPG1mt5vZHcBTwGWZCytx7l7m7iXuXtKjR4+kls3bltgN3YnWS2W55ur27ZJceXPtpCOWZKWrnVT6SiWGeG2m2mdrlk93TB3yOiRVnpC6ZpZtrrwt2kt3TOloP9MxpUGiz2JaCIwC7gbuIrge8Gga+q8C+sRM9w7LmitPq9L+s6GmyY3eTY+LagoaLiynpf3afKht8gvQQh+zj5lNQX7jNgryC5h9THT9yD6TXIeEtku86RS2W2ukY72TbTPVPluzfLpj6tW5F+2s8cdAO2tHr86RB+wJ6bFPL/AmHy3eLijPUnvpjikd7Wc6pnSI9z2IQeHPUUBf4N3w1TcsS9UC4CwLHA5scff3gEeBiWbWNbwgPjEsS6ubp09leq8y8rYWght5WwsZ8sn0RtOpXOSMan96z9uY3nNewn1MHTaVsq+XUdilEMMo7FLY7AXqZvtMch0S2S7xptvyAnVzMacaQ7w2U+2zNcunO6ZuBd0o7FLYcMTQIa9DSheoAQoP7EaP/MLP/hOu65DShdd0tJfumNLRfqZjSgt3b/YF/C78uTji9URLy4bL3Qm8B9QQXEc4j+A7Fd8J5xswB3gLeBkoiVn2XODN8HVOvL7cnUMPPdRFpHmrVq3KdgjeqVOnFue//fbbfsghhyTV5tlnn+333ntvi3XeffddnzBhQlLtLlu2zP/+978ntUy9Dz/80OfMmdOqZTMlav8DFd7MZ2q870FcEP48upXJ54w48x24sJl58wiGOhWRLEn2Ozi5bOHChbs9wyie5cuXU1FRwZQpU5Lur378iRkzZiS9bK6Id4rppJZebRWkiLS9+u/grNuyDsdZt2UdpX8rpfzl9IwatHXrVo455hhGjRrFsGHDePDBBxvm1dbW7jZeBMCSJUs48sgjOfTQQ5k0aRLvvffebu3OmjWLIUOGMHz48EbfTF64cCHHHXccZ511Fn/9618byqdOndqo73o7d+7kpz/9KXfffTfFxcXcfffdbNu2jXPPPZfRo0czcuTIhuVeeeUVRo8eTXFxMcOHD+eNN97YbfyJPVJzhxbBP/fcFr7+DnwI3B++PiD4bkPc0z5t+dIpJpGWJXOKqfC6QudKdnsVXleYUgz1p5hqamp8y5Yt7u5eXV3tX/rSl3zXrl3+9ttvO+BPP/20u7ufc845fs011/jOnTt97NixvmHDBnd3v+uuu/ycc85x989OMW3cuNG//OUv+65du9w9OM3j7l5bW+sjRoxwd/cnn3zSTzzxRHd337x5sxcVFXlNTU1krLfddptfeOGFDdOXXXaZ33777Q1tDxgwwLdu3eoXXXSR33HHHe7u/umnn/r27dtbdaos09J9iukcADNbBAzx4AIyZnYQMD8zKUtEckGy38FJlrvzk5/8hKeeeop27dpRVVXF+++/D+w+XsRvf/tbJk+ezMqVK5kwYQIQPKX1oIMOatRm/bgN5513Hscff3zDeNXPP/88Y8aMAeDII49kxowZVFdXc//993PyySfTvn2LH4UNFi1axIIFC/jNb34DBCPRrV+/nrFjxzJ79mwqKys56aSTGDBgQOobKAcktlWgT31yCL1PcFeTiHxO9e3SN/Jb/M19BydZ5eXlVFdXs2TJEvLz8ykqKmLHjh1A9NgI7s4hhxzS8JjwKO3bt+eFF17g8ccf57777uOmm27iiSee2G2c6LPOOos77riDu+66i9tuuy3hmN2d+++/f7dHgQ8ePJgxY8bw97//nSlTpnDrrbfSv3//hNvNVYl+Ue5xM3vUzKaZ2TSCU06PZS4sEcm2ZL+Dk6wtW7Zw4IEHkp+fz+LFi1kXMzpR1HgRAwcOpLq6uqG8pqaGV155pVGbW7duZcuWLUyZMoXrrruuYQzo+nGi602bNo3rr78egCFDhjQbY9MxJCZNmsSNN95YfwqeZcuWAbBmzRr69+/PJZdcwoknnsiKFStaHH9iT5HoF+UuInjc94jwVebuF2cyMBHJrmS/g5N0+1OnUlFRwbBhw/jTn/7EoEGDGuZFjRfRoUMH7rvvPi699FJGjBhBcXExzzzzTKM2P/74Y44//niGDx/O+PHjufbaa6murqZjx4507ty5od4Xv/hFBg8eHHechqOPPppVq1Y1XKS+4oorqKmpYfjw4RxyyCFcccUVANxzzz0MHTqU4uJiVq5cyVlnndXi+BN7ioQe1gcNz18a4O6PmVkBkOfuOZUeNR6ESMtyYTyItnbHHXdQWVnJrFmzGsq2b9/OsGHDWLp0adzxqj9P0joeREwDFwClwBeALxE8OO8W4JiUohURybAzzzyz0fRjjz3Geeedx/e///29Kjm0RqIXqS8ERgPPA7j7G2YWPdafiEgOO/bYYxtd7wB49NFHufTSSxuV9evXjwceeKAtQ8s5iSaIT919Z/2dBWbWnvQ87ltE2pi773aX0N5u0qRJSX/Lek+T6OWEWInexfRPM/sJsK+ZTQDuBf6WdG8iklUdO3Zk06ZNrfqwkD2Xu7Np0yY6duyY1HKJHkFcCpxP8EC9bwMPA79PqicRybrevXtTWVlJsoNryZ6vY8eO9O7dO6ll4iYIM8sDXnH3QcDvWhmbiOSA/Px8+vXrl+0wZA8R9xSTu9cBq2OHHBURkc+/RE8xdQVeMbMXgG31he5+QkaiEhGRrEs0QVyR0ShERCTntJggzKwjwQhwBxNcoP6Du9cm2riZTQZuAPKA37v71U3mXwfUD0ZUABzo7geE8+rCPgHW62hFRKRtxTuC+CPBcKH/Ao4DhgDfTaTh8OL2HGACwXCjL5rZAndfVV/H3b8fU/9iYGRME5+4e3EifYmISPrFSxBD3H0YgJn9AXghibZHA2+6+5pw+buAE4FVzdQ/A/hZEu2LiEgGxbuLqab+TTKnlkK9gHdipivDst2EDwLsBzwRU9zRzCrM7Dkz+0ZznZhZaVivQvd2i4ikT7wjiBFm9lH43gi+Sf1R+N7dff80xXE6cF94S229QnevMrP+wBNm9rK7v9V0QXcvA8ogeJprmuIREdnrxRtyNC+FtquAPjHTvcOyKKcTPBAwtu+q8OcaM3uS4PrEbglCREQyI9FnMbXGi8AAM+tnZh0IksCCppXMbBDB9yyejSnramb7hO+7A+No/tqFiIhkQKLfg0iau9ea2UXAowS3uc5z91fM7BdAhbvXJ4vTgbu88dPDBgO3mtkugiR2dezdTyIiknkJjyi3J9CIciIiyWlpRLlMnmISEZE9mBKEiIhEUoIQEZFIShAiIhJJCUJERCIpQYiISCQlCBERiaQEISIikZQgREQkkhKEiIhEUoIQEZFIShAiIhJJCUJERCIpQYiISCQlCBERiZTRBGFmk81stZm9aWazIuZPM7NqM1sevs6PmXe2mb0Rvs7OZJwiIrK7jI0oZ2Z5wBxgAlAJvGhmCyJGhrvb3S9qsuwXgJ8BJYADS8JlP8xUvCIi0lgmjyBGA2+6+xp33wncBZyY4LKTgH+4+wdhUvgHMDlDcYqISIRMJohewDsx05VhWVMnm9kKM7vPzPokuSxmVmpmFWZWUV1dnY64RUSE7F+k/htQ5O7DCY4S/phsA+5e5u4l7l7So0ePtAcoIrK3ymSCqAL6xEz3DssauPsmd/80nPw9cGiiy4qISGZlMkG8CAwws35m1gE4HVgQW8HMDoqZPAF4NXz/KDDRzLqaWVdgYlgmIiJtJGN3Mbl7rZldRPDBngfMc/dXzOwXQIW7LwAuMbMTgFrgA2BauOwHZvZLgiQD8At3/yBTsYqIyO7M3bMdQ9qUlJR4RUVFtsMQEdljmNkSdy+Jmpfti9QiIpKjlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIJCUIERGJpAQhIiKRlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIJCUIERGJlNEEYWaTzWy1mb1pZrMi5v/AzFaZ2Qoze9zMCmPm1ZnZ8vC1oOmyIiKSWRkbctTM8oA5wASgEnjRzBa4+6qYasuAEnffbmbTgf8LnBbO+8TdizMVn4iItCyTRxCjgTfdfY277wTuAk6MreDui919ezj5HNA7g/GIiEgSMpkgegHvxExXhmXNOQ94JGa6o5lVmNlzZvaN5hYys9KwXkV1dXVKAYuIyGcydoopGWZ2JlACHBlTXOjuVWbWH3jCzF5297eaLuvuZUAZQElJibdJwCIie4FMHkFUAX1ipnuHZY2Y2bHA5cAJ7v5pfbm7V4U/1wBPAiMzGKuIiDSRyQTxIjDAzPqZWQfgdKDR3UhmNhK4lSA5bIgp72pm+4TvuwPjgNiL2yIikmEZO8Xk7rVmdhHwKJAHzHP3V8zsF0CFuy8ArgH2A+41M4D17n4CMBi41cx2ESSxq5vc/SQiIhlm7p+f0/YlJSVeUVGR7TBERPYYZrbE3Uui5umb1CIiEkkJQkREIilBiIhIJCUIERGJpAQhIiKRlCBERCSSEoSIiERSghARkUhKECIiEkkJQkREIilBiIhIJCUIERGJpAQhIiKRlCBERCSSEoSIiERSghARkUgZTRBmNtnMVpvZm2Y2K2L+PmZ2dzj/eTMripl3WVi+2swmZTLOZMyYW077mUXYle1oP7OIQy6d0eL0jLnlKffRtI1485NtLxNa02e8bZvsdsjEdkq1z1zYF6n+jrZmHcrLoagI2rULfpY3WSQX9lWyf8uZ2NfJxpSOz5uWZGxEOTPLA14HJgCVBGNUnxE7dKiZzQCGu/t3zOx04JvufpqZDQHuBEYDPYHHgC+7e11LfWZ6RLkZc8uZW1UK+ds/K3TAaH66poDpvcq4efrU1vcR00a8+cm2lwmt6TOhbZvEdsjEdkq1z5zZFyn8jrZmHcrLobQUtscsUlAAZWUwdWrybWZsXyXxt5yJfd2qmFL8vIGWR5TLZIIYC1zp7pPC6csA3P1XMXUeDes8a2btgf8APYBZsXVj67XUZ6YTRPuZRdTtty7p5fK2FlJ7zdqU+qhvI978ZNvLhNb0mei2TXQ7ZGI7pdpnLu2L1sbQmnUoKoJ1ESEUFsLatcm3mcl9FU8m93VrY0qmjyjZGnK0F/BOzHRlWBZZx91rgS1AtwSXBcDMSs2swswqqqur0xR6tLpO6zO+XHN168vjzU+1PB1a02ei8SS6HTKxnVLtM5f2RbrrtbT8+mZm1Zfn0r6KJ5P7Ol2/B+n8fdrjL1K7e5m7l7h7SY8ePTLaV962vhlfrrm69eXx5qdang6t6TPReBLdDpnYTqn2mUv7It31Wlq+bzOz6stzaV/Fk8l9na7fg3T+PmUyQVQBfWKme4dlkXXCU0xdgE0JLtvmSvvPhpqCxoVNz9A1na4pCJZLpY+YNuLNT7a9TGhNnwlt2yS2Qya2U6p95sy+SOF3tDXrMHt2cM0hVkFBUN6aNjO2r5L4W87Evm5VTCl+3sSTyQTxIjDAzPqZWQfgdGBBkzoLgLPD96cAT3hwUWQBcHp4l1M/YADwQgZjTcjN06cyvVcZeVsLwY28rYUM+WR6i9PJXjCK6iO2jXjzk20vE1rTZyLbNpntkIntlGqfubIvUvkdbc06TJ0aXJAuLASz4Gf9BerWtJmpfZXM33Im9nVrYkr18yaejF2kBjCzKcD1QB4wz91nm9kvgAp3X2BmHYHbgZHAB8Dp7r4mXPZy4FygFvieuz8Sr79MX6QWEfm8ycpdTNmgBCEikpxs3cUkIiJ7MCUIERGJpAQhIiKRlCBERCTS5+oitZlVA639rnp3YGMaw8kExZgeijE99oQYYc+IM5sxFrp75LeMP1cJIhVmVtHclfxcoRjTQzGmx54QI+wZceZqjDrFJCIikZQgREQkkhLEZ8qyHUACFGN6KMb02BNihD0jzpyMUdcgREQkko4gREQkkhKEiIhE2usThJlNNrPVZvammc3Kdjz1zGyemW0ws5UxZV8ws3+Y2Rvhz65ZjK+PmS02s1Vm9oqZfTfXYgzj6WhmL5jZS2GcPw/L+5nZ8+F+vzt8JH1WmVmemS0zs4dyMUYzW2tmL5vZcjOrCMtybX8fYGb3mdlrZvaqmY3NpRjNbGC4/epfH5nZ93Ipxlh7dYIwszxgDnAcMAQ4w8yGZDeqBvOByU3KZgGPu/sA4PFwOltqgR+6+xDgcODCcNvlUowAnwJfdfcRQDEw2cwOB34NXOfuBwMfAudlL8QG3wVejZnOxRiPdvfimHv2c21/3wAsdPdBwAiC7ZkzMbr76nD7FQOHAtuBB3Ipxkbcfa99AWOBR2OmLwMuy3ZcMfEUAStjplcDB4XvDwJWZzvGmNgeBCbkeIwFwFJgDMG3VttH/R5kKbbeBB8MXwUeAiwHY1wLdG9SljP7m2BEyrcJb77JxRibxDUR+Hcux7hXH0EAvYB3YqYrw7Jc9UV3fy98/x/gi9kMpp6ZFREM+vQ8ORhjeOpmObAB+AfwFrDZ3WvDKrmw368HfgzsCqe7kXsxOrDIzJaYWWlYlkv7ux9QDdwWnqr7vZl1IrdijHU6cGf4Pidj3NsTxB7Lg381sn6PspntB9xPMOrfR7HzciVGd6/z4JC+NzAaGJTdiBozs+OBDe6+JNuxxDHe3UcRnJK90MyOiJ2ZA/u7PTAKmOvuI4FtNDlVkwMxAhBeTzoBuLfpvFyJEZQgqoA+MdO9w7Jc9b6ZHQQQ/tyQzWDMLJ8gOZS7+1/C4pyKMZa7bwYWE5yuOcDM2oezsr3fxwEnmNla4C6C00w3kFsx4u5V4c8NBOfNR5Nb+7sSqHT358Pp+wgSRi7FWO84YKm7vx9O52KMe32CeBEYEN4t0oHgkG9BlmNqyQLg7PD92QTn/bPCzAz4A/Cqu18bMytnYgQwsx5mdkD4fl+C6ySvEiSKU8JqWY3T3S9z997uXkTwO/iEu08lh2I0s05m1rn+PcH585Xk0P529/8A75jZwLDoGGAVORRjjDP47PQS5GaMe/dF6uBIjinA6wTnpS/Pdjwxcd0JvAfUEPxndB7BeenHgTeAx4AvZDG+8QSHwSuA5eFrSi7FGMY5HFgWxrkS+GlY3h94AXiT4DB/n2zv8zCuo4CHci3GMJaXwtcr9X8rObi/i4GKcH//FeiagzF2AjYBXWLKcirG+pcetSEiIpH29lNMIiLSDCUIERGJpAQhIiKRlCBERCSSEoSIiERSgpC9gpnVhU/PfCV8susPzazF338zKzKz/9NWMcZjZk+aWc4NbC+fX0oQsrf4xIOnaB5C8GW544CfxVmmCMiZBJGKmG9kiyRMCUL2Oh48KqIUuMgCRWb2LzNbGr7+d1j1auAr4ZHH91uo1yCs86qZ/S48WlkUfoO70RGAmXUPH62BmU0zs7+G4wCsNbOLzOwH4QPnnjOzL8R08a0wnpVmNjpcvpMF44e8EC5zYky7C8zsCYIvYYkkRQlC9kruvgbIAw4keO7NBA8eRHca8Nuw2izgX+GRx3Ut1GtqADAnPFrZDJycQEhDgZOAw4DZwHYPHjj3LHBWTL0CDx48OAOYF5ZdTvB4jtHA0cA14eMwIHgW0SnufmQCMYg0osNOEcgHbjKzYqAO+HKK9d529+Xh+yUEp6riWezuHwMfm9kW4G9h+csEjwupdyeAuz9lZvuHz5maSPCwvx+FdToCfcP3/3D3DxLoX2Q3ShCyVzKz/gQf8hsIrkW8TzACWTtgRzOLfT/Bep/GvK8D9g3f1/LZUXvHFpbZFTO9i8Z/p02fjeMEgwud7O6rY2eY2RiCR16LtIpOMclex8x6ALcAN3nwMLIuwHvuvgv4FsGpJ4CPgc4xizZXL1FrCYaZhM+e0pqs0wDMbDywxd23AI8CF4dP2MXMRraybZFGlCBkb7Fv/W2uBE/LXAT8PJx3M3C2mb1EMJhQ/X/dK4C68LbY77dQL1G/Aaab2TKgeyvXY0e4/C18Nkb1LwlOf60I1++XrWxbpBE9zVVERCLpCEJERCIpQYiISCQlCBERiaQEISIikZQgREQkkhKEiIhEUoIQEZFI/x9mmjoctWPX8QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# as usual we'll want to evaluate it and visualize its evaluation\n", "print(\"Validation accuracy : \", rf.evaluate(X_test, y_test))\n", "rf.visualize_evaluation(rf.predict(X_test), y_test)" ] }, { "cell_type": "code", "execution_count": 29, "id": "lasting-transsexual", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.9733333333333334" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# 96% accuracy is really good compared to what we were getting before with a decision tree (78%) \n", "# but here's where the cool part comes in!\n", "\n", "# You know - I bet you that one of those trees in the random forests is better than all of them, so \n", "# why don't we just go find that tree, get it, and then use it? Seriously. \n", "\n", "from sealion.decision_trees import DecisionTree\n", "best_tree = rf.give_best_tree(X_test, y_test) # get the best tree of off the data you give it\n", "dt = DecisionTree()\n", "dt.give_tree(best_tree) # enter the best tree trained from the random forests\n", "dt.evaluate(X_test, y_test) # see how well that best tree in random forests did" ] }, { "cell_type": "code", "execution_count": 18, "id": "primary-belgium", "metadata": {}, "outputs": [], "source": [ "#WOW! 97%? Moving from 96% to 97% is a big deal - it gets exponentially harder to make the model near perfect. \n", "\n", "# That's mostly it for random forests. Onto ensemble classifiers!\n", "# the way ensemble classifiers work is super simple. All it does is just take in a bunch of predictors, \n", "# train all of them, and then average all of their predictions. It's basically a random forest but for other classifiers. \n", "\n", "# for this we can use the blobs dataset \n", "from sklearn.datasets import make_blobs\n", "from sklearn.model_selection import train_test_split\n", "X, y = make_blobs(500, random_state = 2, centers = 3)\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.15, random_state = 3)" ] }, { "cell_type": "code", "execution_count": 19, "id": "egyptian-archives", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEWCAYAAACNJFuYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABBGklEQVR4nO2df5hUd33v359dBhiIMsSgJpNswB8Fg1tAMKbS+1hQgzUmWYkRbay/WlO9tZoY124aa8BiQbcp6W17n5pb89hrUoWEuJLQPsQUqk9RoosLiRi4NcYkDrFiYdGwC8zufu4fM9/lzJnv93u+58w5c87Mfl7PkyfszJlzPnNm5vP5fj8/iZkhCIIgCF460hZAEARByB5iHARBEIQ6xDgIgiAIdYhxEARBEOoQ4yAIgiDUIcZBEARBqEOMg9DWENGXiWij47FMRK9IWiZBaAXEOAgtDRH9lIhGieh5IjpBRDuJ6JIU5ZlfNTLPV//7LyJ6iIjeHOIc7yei/0hSzmZeR2hNxDgI7cDVzHwegAsB/BeAv01ZHgAoVGVaAuCbAL5ORO9PVyRBcEeMg9A2MPNpAPcDuMx0DBF9iIh+TETHiWgHEV3kO+StRPQTIvolEfUTUUf1da8gom8R0cnqc1sdZfo5M/8NgPUAPu85Xx8RPUlEvyaiHxHR26uPvwrAPwD4rerOY7j6+FVENEREvyKiZ4lovec9zSSie4jov4lomIi+T0QvqT43h4i+RETPEVGJiDYSUafpOoKgEOMgtA1ENAvAOgD7DM+vBrAJwDtR2WU8DeBrvsPeDmAFgNcAuBbAB6uP/wWAhwHMBXAxwu9OHgDwYgALq38/CeB/AJgDYAOAe4joQmZ+AsCHAXyXmc9j5kL1+FMA3gugAOAqAB8hop7qc++rnucSAC+qvn60+tyXAYwBeAWAZQCuBPCHlusIAgAxDkJ7MFBd+Z4E8GYA/YbjbgBwNzP/gJnPALgVlZXzfM8xn2fm48z8DIA7Aby7+ngZwKUALmLm08wc1ld/tPr/8wGAme9j5qPMPMHMWwH8J4DLTS9m5n9n5serxz8G4KsA3uCR7UUAXsHM48y8n5l/Vd09vBXATcx8ipl/AWALgHeFlF2YgohxENqBnurKdyaAjwL4FhG9VHPcRajsFgAAzPw8gP8GUPQc86zn309XXwMAnwJAAL5HRIeI6IMIh7rGcQAgovcS0YGqG2gYwKsBXGB6MRG9joj2ENExIjqJyqpfHf8VALsAfI2IjhLRF4goh4oxywF4znOdL6KygxEEK2IchLahump+AMA4gN/WHHIUFYUJACCi2aisuEueY7yZTl3V16jYwYeY+SIAfwTgf4dMe307gF8AOEJElwL4P6gYshdVDdsPUTE+AKBrlfzPAHYAuISZ56ASL6CqbGVm3sDMlwF4PYC3oeKCehbAGQAXMHOh+t8LmXmx5TqCAECMg9BGUIVrUYkLPKE55KsAPkBES4loBoC/BPAoM//Uc0wvEc2tpsN+HMDW6rmvJ6KLq8ecQEWxTjjI9BIi+iiA2wHcyswTAGZXX3+seswHUNk5KP4LwMVENN3z2AsAHGfm00R0OYDf81xjFRF1E1EngF+h4maaYObnUImT3EFELySiDiJ6ORG9wXIdQQAgxkFoDx4koudRUYyfA/A+Zj7kP4iZHwHw5wC2A3gOwMtR73//BoD9AA4A2AngS9XHXwvg0ep1dgD4ODP/xCLTMBGdAvA4Kn7/65n57qocPwJwB4DvoqKguwHs9bx2N4BDAH5ORL+sPvY/AXyWiH4N4DMAtnmOfykqWVq/QsUofgsVVxNQ2UFMB/AjVIza/agE403XEQQAAMmwH0EQBMGP7BwEQRCEOsQ4CIIgCHVk0jgQ0SXVtL0fVdMGP562TIIgCFOJTMYciOhCABcy8w+I6AWoBAh7qoE8QRAEIWGmpS2AjmoK3nPVf/+aiJ5ApYhIaxwuuOACnj9/fvMEFARBaAP279//S2aep3suk8bBS7W1wTIAj/oevxHAjQDQ1dWFwcHB5gsnCILQwhDR06bnMhlzUBDReajkpN/EzL/yPsfMdzHzCmZeMW+e1vAJgiAIEcmscaj2htkO4N5qSwRBEAShSWTSOBARoVKZ+gQz/3Xa8giCIEw1MmkcAKwE8PsAVlc7Vx4goremLZQgCMJUIZMB6WqvfAo8UBAEQUiETBoHQWglBoZK6N91BEeHR3FRIY/eNQvRs6wY/EJByDBiHAShAQaGSrj1gccxWh4HAJSGR3HrA48DgBgIoaUR4yAIDdC/68ikYVCMlsfRv+uIGIcGkN1Y+ohxEIQGODo8GupxIRjZjWWDrGYrCUJLcFEhH+pxIRjbbkxoHmIcBKEBetcsRD7XWfNYPteJ3jULU5Ko9ZHdWDYQt5IgNIByc4h/PD4uKuRR0hiCoN2YxCniRYyDIDRIz7KiKKEY6V2zsCbmAATvxiROET/iVhIEIVP0LCti09puFAt5EIBiIY9Na7utSl7iFPEjOwdBSBlxh1Ro5D5InCJ+xDgIQoqIO6RCo/chapxCMCNuJUFIEXGHVGj0PkjWWPzIzkEQUqRd3SFhXUSN3gfJGosfMQ6CkCLt6A6J4iKK4z5I1li8iFtJEFJE5w7JdRBGzo5hQd9OrNy8GwNDJes5BoZKWLl5t/PxSRPFRSRuoewhOwdBSBG/O2ROPodTZ8dwYqQMIHjVncWAdhQXkbiFsocYB0FIGa87ZOXm3RgeLdc8b+vyun7Hocx1hY3qIhK3ULYQ4yAIKeIP3OqUKqBfdQ8MleoMie34JPG+j8KsHHIdhPIETz7vdRFJXUdrIMZBEFJC5xIyUZiVq3vM5sNvZkDb/z5OjJSR6yQU8jmcHC3XGIA43WBiZJJFjIOQCabiD10XuDXBXP+YbXfQzECu7n2UxxmzZ0zDgduvDDw2ihssi7GWdkOylYTUUT/00vAoGOd+6Gln3SRNGNfPSY37yLQ7mDsr11QFGSYAHVddR1BGVNYyuFqRzBoHInoLER0hoh8TUV/a8gjJMVWrhMO4fnTHmtI/b796ccOyhSHMwKO4hiPZjMxUXWzETSaNAxF1Avh7AL8L4DIA7yaiy9KVSkiKVq4SbmSFqq1x6CTkOqjmMVO+f5TupUkQpkYhrnqGOfn6GIx6fKouNuImqzGHywH8mJl/AgBE9DUA1wL4UapSCYnQqlXCjfq9dbn9qxbNw87Hnpuscyjkc1h/zWLj+bKQ/hmmRiGuegYi8+OtvNjIElk1DkUAz3r+/hmA13kPIKIbAdwIAF1dXc2TTIidKMNdskAcwVWvcvcbGwA4MzYRn8AJEsZIxWHQhkf0KbzDI+WWXWxkjUy6lVxg5ruYeQUzr5g3b17a4ggNkBX3SFjiXqGKO0SPznVni11IK454yOrOoQTgEs/fF1cfE9qULLhHwhL3ClXcIfWYXHfXLS9i+/6SdreZZiuOdkrJzqpx+D6AVxLRAlSMwrsA/F66IglCLXG7w0zGRlcAFwdhFFlaSs+0m9pz+Bg2re02ypTGYqPdai+IddU1GYCI3grgTgCdAO5m5s+Zjl2xYgUPDg42SzRBmCROpTkwVELv/QdRHq/9TeY6CP3XL4lVwejiG/lcJzat7QZQHyT3r9KB4GB5HCzo2wmdhiIAT22+qu7xNFfuKzfv1hr3YiGPvX2rmyJDWIhoPzOv0D6XVeMQBjEOQlaIqpzU60wtNOJWMCZFVsjncGZswrlymwBwVb5Vi+Zhz+Fjkd+77nVhFK7N4DXDQIQ1ZFnAZhxaNiAtCFkjavGV93Um4o47mM43PFp2NgwAJpVhaXgU9+x7pua937T1AJZ99mHr+w+6Z2GCy0kH9INqWuIq8MsKYhwEISaiKieXHktxK5hmKawTI2WrgQy6Z2Ey2ZIM6LsY/nbLkspqQFpoE9opeyOIqMrJtmNQxK1gTMH0M2PjmIjZ02yr/XC5Z67B5STrG1xqWtptYJEYByEx2i17I4gg5WQylJ1EGLfE/pJopGdSZDdtPRDrdRQmIxCnQk+ymNLV8LdiSrYJMQ5CYsTVnrlVsCknm6G0GYY4G+npjJM/qGsLijfCnHwOKzfvrjOMunsGVJTu/L6dKIZYfatj1u84NDkEaWYuHs/5VKy6FuMgJMZULOqameuYVHTeVM+Vm3cbDWXRMgHO73+Pij9NtjQ8it77D06e15stpTKQwqCylR46+FzddLpcB+HU2bHJx9W11+84hJOjZczJ59BBwKmz5+6PN9CtjCjg5rLxthxRMQ/1Pr33I4z7Z9Wiebhn3zPax9sVMQ5CYrT7ass/GvP502M1ozFPjpYx+PRx9CwrWg3llnVLtatnRRzuuA0PHqqrnyiPM/7sgccAoOb6jHMpqsrlZXN9edNKN/Z01ynekbNjk40EvddWxmJ4tAxDHz0AFQO5fsehmhRb0z1x2a1GcXfuOXxM+/jOx57Dxp5ui/Sti2QrCYnRbtkbXvzZKydGyjWGAago13v3PRPYC8ifkdOpaTnqkvVkS7X0K2fFSHkC63ccqlOoqnbhjncuQbGQxwQz5lZnQ3vRfZ49y4rY27caT22+Cnv7Vhub5PmvZ0OXYqu7Jy671ShZZabznhgpt+2cCDEOQmI0q6FeGlO/XEd8cvXYIEPpVagThhW6zR3XyIAbvxtIURoexc1bD9QYQFDFXUao/H9mrgM3bz1gve9J7hT998Sl1iCKu9P2Htq1MaIYByFR/KvIJAxDGlO/wsRNjg6PhjKUUYqpTKvhDQ8ewsrNu51l9eM3U2o29JZ1S3FmbAInRsqB911nGHWYXEsEwBRX9g/90V2LqvIpA2YbFGTCtttt1xiaGAehpUmrzXWUEZ/KUG5ZtxQAjCtukzIdOTtWd6zaNZkC2idGyrFnHx0dHg113/2G0eSeuuGKLhQ195UBlA1jLfweOO+1ANQE11XVtmmnZBogpM5bMBiPdomh+RHjEDMy2Ly5pJURpVPgnR167eJV6i47HaXg/MrIX23s0nYjCS4q5I33t1Sd4Wxj1vRpWHf5JTXGgsCT7TdswWk/uniGMsLFQj5U1lVQbGT9NYtDxdBaXReIcYgRGWyeDLYfmWnVxkDsP0ivHP27juC65cUaN9Ed1y/BneuWWpW664q7Z1kRs2fUJxN6j93wYH0gOWmUMrStlv3fed3vYvv+EnrXLMSWdUvx/JkxjHi2BmEUuk2OsAuEoB1AGNdgO+gC6coaI63YsjfrBHXa1D3vJa6unGE6fto6nhpdGjjXuTOoQysB2LJuqbWauVjI49SZMeP1TOQ6CWDUZF55O6+qeoCg+95JhAlmXGSRo5MIM6ZRjWEIy3uu6DKmktrcbX7i7t7aKrpAurI2ialY9JU0YRqz6Ygr/hDGx27reGrC22IjyFV0USFvfU+dRDg6PAoi1Pn2dVTcOpX/z54+DeUJnkynLRby2LJuKX7qSyhQ993EOPPkitn0vseZGzIMgLn+AAguUOskSiyLrh10gRiHGGm3lr1ZwLUx296+1UZfdRw/SJOy1p077Oed66BJv3VQiqxy69jek1LMqvYiH9BCYtb0SvbR6fLEpCIfZ64bu+lH9YVKCge7NpmFpHM5PnTwOePr8rlO3PHOJYFZdFHjBu2gC8Q4xEgWi75aPSgW5keW1A9yYKhkNDy6c7umbirOmzltUjkFGbLXdM1Bz7JiqPd0ujyB91zRZXw+bPaRYmCoZO0L1Qi5DsILZwaPR1Vpqjq/vm2n5rJTaCRukEVdEBYxDjHSrKIvV8J8ubNqRML8yJL6QfbvOmKc8KU7t+57MNcyB9qbJROk9L/z5PFKn6Q1C52zehgV94tJhsKsXKidEXDuu5UU/dcvwcmAeImuB5SrG9EURPb+BnQB/zDnT1oXJP2bld5KMZOllr2uXVHjbq0d5wyHMD3yo/TTd5HVpCAZ5vvj/x4MDJVw89YDWiPjNQimLqXea/bvOoK9fatDtdc29XDKdRKePz1mfJ3JWCWdKdWzrBjYIda0Z1Gf19xZOW3bEJ2R1P0GTLi6KRvVBbbvZjPa4YtxaGNcg2JxttZO4ksb5kcW5lhXWU0NBE1BcJNcg08fx737nqlRav6djbruJ7YdMA7dUZ+frZurH9XDCag1nraMJtOua2CoZOzVBFQyiNQs6ShOJ3Vfg+ZLmLrHKoN2+9WLazrRAhVjqGuB7toOxXv+IBpZJAV9N5vRDl/cShkgqe2hqw8+zsyKZlcsN3LvXGWNy121sacbW9YtdXI12AK9F3mUp0tsw9TDaW/faqvr5rrlekNr+yyLhTw29nRPXiOMAVWoLCNbVTJwrnusF/977X/Hkpr73f+OJaEm0vlx/dxVi3SvS7f3/oPO30/Td3P9jkNWeePMhsrczoGI+gFcDeAsgCcBfICZh1MVKkGS3B66TsaKs7V2M1P4Gr13YaZ7AXZ3lesq0WVn07/rSF2HV4U3swmonR+ho5PI6us2ffaAOU3U9ln6v1tBbrKg666/ZnGgm61YrdjW3fdGR4wW8jnMnjEt9Orf1CJ9w4OHGvpuDo+WJ7v8Jt0OP3PGAcA3AdzKzGNE9HkAtwL405RlSowkt4euPvg4xys2c4ZDo/cujKw2JRPGSDUS4wDOZTYFFaEpJpit98LmujlabYXhl9emSP3X0n0Hh0fO1gz20V3X//pbth3UZkbFVVRm+g2oYU1hMbndbO44Lzajrbr8JjUSVZE548DMD3v+3AfgHWnJ0gySWGmH9XVGHYyuu04zvrSKRu9dXLLGHfi3KQaV2eTqI3dpCbHhwUNapTUnn9PKe93yIrbvL9Vcn1BZ1a7cvBurFs2bjDnoxpEu3fAwALPsfpnVvUnyexX1N5AUNqNdqqYeX7e8WHef45Q3c8bBxwcBbNU9QUQ3ArgRALq6zDncWSfulXZUV0vYzArTdTat7camtd2BP7I4MpoavXdxuYviDvz3rlkYmNnkYgBdleftV9e7bnKdhJOny/Av1kfL49hz+NjkZ+wfK1oaHq0Zp6n7/tniHCaZm6G848w0NLVKscVQ/LKYjDZwrj9VkqnyqRgHInoEwEs1T93GzN+oHnMbgDEA9+rOwcx3AbgLqPRWSkjUxAkaSh/2x9CMLIag6wTNbYgrzrJq0bzA7J8gTAphYKhUM6jeJqerkQoT4wjKbDJd09vTyFV5+hVvYVYOJ6tzGkzyqvvm0r/I//2zyW5TdmEz0eKIEUVl/TWL0XvfwZrYUa6DsP6a+kwpEzqj7SWJ37WXVIwDM7/J9jwRvR/A2wC8kduhM6AF9cF6VwkzpnVg8OnjNVt3VwXarIBwI9eJw4ANDJWwfX+pRoERzBk2pnPoFITNn29a6bu4PAqGvPuCJu9+Y083Vlx6vlGBma4ZdiXpvwdb1i3F+h2HYOt45DJVzY/3uLCyh1XkQYsP2/NAPLuTOHY63nOELVKMg8y5lYjoLQA+BeANzDyStjzN4rSnAdnwaLlu1Qi4KVDTqqyDCANDpdhWGY24dOIwYDoDoyqBXbApiCB/fpRsJgB1LhrFiZGKrz5Mpk0cykd3D4IK6/xGzxYf8eL9XoSRPcouM2jxYUsTPTM2EVvmYBxuqqAdWpK9mjJnHAD8HYAZAL5JlVzvfcz84XRFShaTotMRpEBNqYPjzLGkyHrbSfuLkFxdOnHEWaIYmE8PPI6vPvqssR+QUiBB9zhsNpPC5muPoogaVT5hCr8U/o64LqmquU7CqTNjWNC3s8YQuMgeZZcZ9N0I0zU3adeNK81M9FBkzjgw8yvSlqHZhFkxu2SfAPrUP+8XPYrP1b+KU0VI/l7/Qei+6ITgFsteOTqItEredH8+PfB4TaDUhLofptVwIz/IoFV2GEUUh8886gQ5nSHzyuLNVirMyuH50+eqsJOqRfFiW3zYvjthZWgmaWRTZc44ZJmkglimL3PUlXnPsiJuDshdjxIQNu1wwuaa6wKuDGD7/hJWXHq+kz9Z9+O23Z+vPvqsk2zqc9WthmdP78Tn3h49O8Rlle2iiOIK6HeGVJJevIbMtgtYuXl3XZwlqVoUhWmVvWrRPOt3Z2auQxsTykqb7TizqVyQ9hmOJDn2z9Se4YYrumpS32Zq+vKb2kfYWmdEbXERZ7B7z+FjoTtqDgyVcMu2g1rlqjJdAGjvh4sS9M4vuG55sa41g6nXkSs9y+wDcgA3RWT6/G7Z5t6eAQi+J2oYjgmXzz2OWpSwrUvUffa3Kdlz+Jj1u3P71eFmRLc7snNwJI1KZqCymlaoWcTqNbYVpM1HadtV2AizigvaZYVVGrYdA1CpBAZgvB+2VTIBdTLajFcjn7cKiJp2ii6KyHSPdHEl2+dga9xHAO5455JQwdAw1dRx1qKYXuc/xvS991eRZ6UQLm3EOGjQfcnjWjWbfqy6L/PKzbsjZV2oWgNA/0U3KScGML9vJ4BKsY7KyVbH6yZzeVdWpmC1zu0RVmkEBU+DdkTvft0l2piDaQZxkinBppjLDVd0NeRqAerjSjb3U++ahXW5+AqvLC7BUNO1dNXUcdWiBOH/rc0xFKb5M6mmqjHwI8bBh+lL7vLFCjqvv+IxyFccNetCPW76orv4vodHy/jEtgPoJJpUHn4dQjinjPx1GUGrbp1i8jeV070nHS47ImUAVLZSJxHe/bpLjMPpk+wR1WhwMejzU/cqaLerruct9ps7K4fbr67tJ+Qir+la3mrqZq7Gdb/jXCch10E137mp7DYKQoyDD9OX3DTP1iXDJmxRlSJIQUVVYC7FNUDFGExY/NLenYGuLsNP3bX899Ti4LatllUsJuh+bOzpNhoDP0mnDjayQrVlpAHBLTZK1aQEdZ7ZM6bh5Gi5oW6ytoVKGqtx3e+4PM6YOyuHWdPru6wmXTHdikhA2ofpS27qIulSdBW2qEoRFIxrZM5Az7JKX/+4xsO7xGq9Mwr6dx3RtjQ2BaR71yxEzmChVSxm1aJ5sQUUTUHNZimMoDkVPcuKuOOdS6zv17ZIuPWBx/Hpgcfrkixu3noA8yPMxnCdHeJCHPNNjLUMI+WaeRZe91sSySatjOwcfLhWfCoaydjwXlNH0HY+jtznsO+3EbyrXNuqVlULA7X9fmzZNUm4MLwrXrWyvHnrgcRXlq6pqkGfv839NFoe1xYE2uJENuLaacWVphtmV92sfmSthhgHH6Yv+YxpHZFjDo0UVQVtyf0Kwlu96oItKBkW09hGhXcqmO2elIZH0XvfQYAwubtw6YOflAujGb14vIRRVrb3qx43tcQISmUNoyDVMf4eYWGJS1GHMVbN6kfWaohbyYfJnbD+mug50Ka4RD7X0bCrotEtcc+yIvqvX+LcShioBC39Lp58rhOvf/n5RjeV/17pXGJeyhNc53YKQvWPAqK5JkyvsfXiScIdEaey6llWNI7qtI0ijXpNf4+wsPcjrvcexi0Yp0usnZCdgwbbaizKKtEUlzh/9oymrjJN+N+vSmfVodpkqGt770X/riPanQMBdT9MXZV0o6g8/zAdbV3Sb5vdi8fFJRImgGpKnb3iZXPxnSePW+9/GAUZx3cxziwx111kGn2LWgFqh47YK1as4MHBwbTFMLKgb6dRaT61+arMnfvlt/6L1eVgaq9sMypAvWFJKtZhcm9523zoUotNhG0z0ejnqstu895z3fO5TsLs6easo08PPK6dD2FLlAjbAjyO76LpvSc99SyObKVWzHgiov3MvEL3nOwcmkCSOfOu5w7zxTUVjClGy+PY8OChumZrQTEHXSzBRK6D6o5TClC3YvcS1NHWdf6yotm9eIICzaY0TVtzO1PFt8nwBQ3e0RHH91z33lctmhdptkkYGo1VxRVIzxISc2gCjaScxnHusHGJjT3deM8VXVaf9ImRcs35XN1DLrGETiL0X78E/e9YUuMz7n/HklCTtPwoJRWlVbUXNVQoyV48KtXYm3KpcPG/+/tU2Vpu6N6Dap0Rhri+5/73ruuJ5NILrJlE7VeWZWTn0ATiSDlt5NxRfMGqYMxlDCTgVufggt+VoWspEpWRs2NG10cY1FAhVVDXbFeCa/qx1yCYXlP0xIuyMP1MRytkE7WCjGER49AkkqwSbaR61YQ3UNssXGZCNCKPS3zBlaAWJXHidwn63SwmvO4cW9A1zveQxP1I0i0bF60gY1jErZRR4qgSVYRN1fO6oZKgU5MGe+e6pXXuE+1rHdIvm0GzfvQ6l+D2/SVct7w46XIzpRZ73TlxVnzH+d10IUm3bFy0goxhkZ1DBok7uBU2Va9Rn3wQL5gxDbNn1Pe3cSHqcJooqAB71KFLjaLmV+gm+u05fKxmwJJLwkEcq/o0Aq9JumXjohVkDIuksmYQk58/7MQ1L2GyleLwy9toJNVz6YaHA7OVomJKB/W62FR2T5ixqFEIyqiKIw06Ckl8N4X0kFTWFsO1m6YrYfOvo/ZbKuRzODM2EcoXHpakvEo2Za8ei7Jijpr77jK/Ig3aMfAq6JGYQwYJ6qYZxscbpb1GUGsLHflcJ9Zfsxib1nZb4wKNumSGYwwqKwgIjHdESVVspLWJy/yKNJBWE1OHzBoHIrqFiJiILkhblmahAn2qlYMOnUKyBQijKDVv8NKGMgLe4GbPsqJ1BkSjvaSiKCFCZeKb6f1cVMgHBlmjrJgbyX03vc8oxWl+Ggkot2PgVdCTSbcSEV0C4EoA5jLdNsPvY7b5/L0KKShAGNUNoBR9FB+zyS3VqEdoYKiEU2fGnI7toMqwIq+7yNRCYtWieYEuoyipio24YExJBHE1aowaUG7HwKugJ5PGAcAWAJ8C8I20BQGa0zMlTIaQVyEFrU47DO0RXFfgpqZt3rkL/nuhRnb6r8pVeZXcYe6na8sLNfvae76BoZI2kK0qnW0VuLbZCEEr5kZy35NSwkk0ahTak8wZByK6FkCJmQ+SxXdNRDcCuBEAurq6EpOnWal7rgE9v0KyBa9vfeBxY18gVzeAV0nZOpf6O66aZgio13jv581bD+CmrQesQWFX4zl7xrQ6w2AyKqrS2WWFH0VZN9rtMwklLAFlwZVUjAMRPQLgpZqnbgPwZ6i4lKww810A7gIqqayxCljFlmce95Qo0yqzkM9N1gTMyedABNy89QD6dx1B75qF1swinUKM4rO2uZhGy+O4ySOPOm/RIFcnUZ1cLtPHXJWX/ziXEa2uK/ywyjqLLph2rOQVkiEV48DMb9I9TkTdABYAULuGiwH8gIguZ+afN1HEyRWnqegq7pWWaZWpXCSmHcx1y4tOrRQUE8x1K2sX5TUwVLKmtyp5Bp8+jj2Hj9XtMhRRp4+5ptf6lZzLiNZVi+Zpu9CahjSFIWsuGJldILiSqWwlZn6cmV/MzPOZeT6AnwF4TbMNA9D8PPOg9gYmX7GamxyUWaST2zXVUh0XxGh5HPfse2ZSiUfdzumMQO+ahch1Boe0j586UyP/HMuEO6UUTcOYTI+3MnG20RDam8zFHLJCGnnmtlWmzVesXhdU2ewPJLsGJ5Nup6Hj0wOPT3Y9BSr3Zv2OQ4HV0aPlicrMiOprTGGrDjqXVnuzIT4Sx+4wiwNgsrabEbJJpnYOfqo7iF+mce0k88zjlMf7uG03owskm9w0fqWYRrDynn3PYL4vD/+kY9uM8gRPZkWZiuaYa9NUdTS6O2x0vrcgpEmmjUOamIp9ogxBSVIe7w7GdEwhnzNOAdPhV4pJBCuLhTzuXLc08LjS8Ch67z+IgaFSKDmUQQtS/KbaCdPuMEwBWTsOgBGmDoHGgYi+QEQvJKIcEf0bER0jovc0Q7g0yZpv1kUe0zGmFbdpCphfKdoCs1FaaCv3lsmd46c8zvjEtgOh+j0p5W8zqmpl73dVzZ2V037WYXcCkjYqtDIuMYcrmflTRPR2AD8FsBbAtwHck6RgWSBrvlkXeXTHmIb2dBIFDm4fGCph+3698lMVuwCMdQ3ea40z17i3wgSsJ0IcnOugSQNnSydduXm3NpYya/o07X0OW0AWJW00izGKZjBV33eWcTEO6pirANzHzCdtxWlC9tClLwKVncP2/SXrjsgUjPbGXgaGSugAMGG4vjIizZgsp6uQNhnVsCv7oMddJrZ5d2ZBx7fDkHoX0pgRIQTjEnN4iIgOA1gO4N+IaB6A08mKlS7NnnSVNMrdpHMBBfnATQrRWy/Rv+uI0TCo3Ymtz1McFAt5/HTzVThw+5XOCiVsINr2uMvENq8rUHf8vfuemZIxConNZJNA48DMfQBeD2AFM5cBnAJwbdKCpUW7ZpjYuqXalLaLorS9Xu1OXALK+Vwn3nNFV93IyyCiphaH7TBqO95Wh7K3bzWe2nxVTVtw3fEmz1m7xygkNpNNjMaBiFZX/78WwO8AuLb677egYizaknZexURJ2dQpxFwn4dSZscmdVWGWudAMOHf/etcsNHZmVW6qjT3d6L9+yeRqu5DP1RW/5ToJhXyu4USBsEkHtuPjclHpaPfWFjIjIpvYYg5vALAbwNWa5xjAA4lIlDLtvIqJ0jrBH9AtzMrh+dNjkxk+peFR5DoIuU5CedwcNVbFeoNPH9e2zfYqZX+MIMlgZVCQX3dtXavysMFnW1vzNOZVp4m09MgmRuPAzLdX//+B5omTPu3cmCxqIzivAl25eTdO+ArLyhM82SDQFHBW929jTzdWXHp+KBnSyhoLEygNq+BMxwdlj7UjWWxQKADEAY3QiOgrAD7KzCerf18K4G5mfmMT5HNixYoVPDg4GMu5dC2e4xiy0i6YWnSogfftdP/CDjoKu8OR9E0hbYhoPzOv0D3nksr6HwAeJaJPACgC6AVwS4zyZQpZxdgJ2lm10/0L62KM0tK7Fe+LMDUINA7M/EUiOgRgD4BfAliWRpfUZiI/WjO9axai976DKHuq0rxFZ0D73L92djEKQhAu7TN+H8DdAN4L4MsA/oWIliQsl5Bl/ClHbVoTGTbVVRDaCZciuOsA/DYzf5WZbwXwYQD/lKxYQlbp33WkLiupPM5tkerrJ2v9tQShmbi4lXp8f3+PiC5PTCIh07Rzqq+OdnGRCUJYAo0DEc0E8AcAFgOY6Xnqg0kJJWQXFz+8ZOEIQuvj4lb6CoCXAlgD4FuozHX+dZJCCdklyA/fru1Hmk279fcSWg8X4/AKZv5zAKeY+Z9Q6c76umTFErJK1FnX7RiTSAoxsEIWcKlzUOWww0T0agA/B/Di5EQS0sTFJRR11rXgRti5EYKQBC7G4S4imgvg0wB2ADgPwJ8nKpWQCnH01ZfagMYRAytkAZeW3f/IzCeY+dvM/DJmfjEzf7EZwgnNJQ6XkNQGNI50KRWygEvMYRIieigpQXzX+RMiOkxEh4joC824phDPilVqAxpHDKyQBVzcSl4S/4UT0SpUhgktYeYzRCTxjSYRl0tIagMao536Uwmti0udw58A+AozDwMYSlwi4CMANjPzGQBg5l804ZoCpK9+lhADK6SNi1vpJQAGiWgbgG1EmkHE8fIbAP4HET1KRN8iotfqDiKiG4lokIgGjx07lrBIUwNxCQmCoAic5wAAVYNwJYAPAFgBYBuALzHzk5EuSvQIKoV1fm4D8DlUOsB+DMBrAWwF8DK2CBrnPAchWaR6WhCyQ6PzHMDMTEQ/R6XGYQzAXAD3E9E3mflTYQVi5jdZhP0IgAeqxuB7RDQB4AIAsj1oceJIlRUEoTm4tOz+OBHtB/AFAHsBdDPzRwAsR6Vja9wMAFhVvfZvAJiOyhwJocWR6mlBaB1cdg7nA1jLzE97H2TmCSJ6WwIy3Q3gbiL6IYCzAN5ncykJrYMUdwlC6+DSsvt2y3NPxCsOwMxnAbwn7vMK6SPV04LQOoQqghOERpDiLkFoHcIWwQlCZKS4SxBaBzEOQlOR4i5BaA3ErSQIgiDUITsHITWkIE4QsosYByEVpCBOELKNuJWEVJCCOEHINmIchFSQgjhByDZiHIRUkGlngpBtxDgIqaAriAOAkbNjGBgqNV2egaESVm7ejQV9O7Fy8+5UZBCELCEBaSEVVNB5/Y5DGB4tTz5+YqTc9MC0BMcFoR7ZOQip8uvTY3WPNTswLcFxQahHjIOQCmq1Pm5ouNvMwLQExwWhHjEOQiroVutemhmYluC4INQjxkFIBduqPO5OrUHBZukWKwj1SEBag7R1SB7TbIdOImxa2x3b/XYJNku3WEGoh9phyNqKFSt4cHAwlnP5lQlQWUXGqbDSIGsGr1n3eeXm3VojVCzksbdvdWzXEYRWhIj2M/MK3XOyc/Bhy1yJW5k2S2FnMVWzWat1CTYLQjTEOPholjJppsJupsELQzNmO8hoUkGIhgSkfTQrc6WZufUmw1YaHm37qmAJNgtCNDJnHIhoKRHtI6IDRDRIRJc38/rNUibNdHeYDBuhYiAY53YuWTEQcbWz6FlWxKa13SgW8iBUYg2tHj8ShGaQRbfSFwBsYOZ/JaK3Vv/+nWZdvFm+8Ga6O3rXLKwL/hIAfypCFlxNQPwuNxlNKgjhyaJxYAAvrP57DoCjzRagGcpEp7CTcnfoDJ7OMAHRdi5xB9YbiZHYZGl2xlbWMsQEIQxZNA43AdhFRH+Fitvr9emKkwzNzq33GzxTimfYnUsSgfWoLjebLACamrGVxQwxQQhDKsaBiB4B8FLNU7cBeCOAm5l5OxG9E8CXALxJc44bAdwIAF1dXQlKmxxpuDvUarY0PFrnWoqyc0kiEyqqyy0oyK977pZtBwG4K+yBoVJNJ9m5s3K4/erFda/PaoaYILiSinFg5jplryCi/wvg49U/7wPwj4Zz3AXgLqBSBBe3jO2IfzXLOBd7KEbcuSQRWDe53FYtmoeVm3cbd1pRZBlndl7RDwyV0HvfQZQnzn3dToyU0Xt/vYFpZPcjrighC2QuWwmVGMMbqv9eDeA/U5SlrdCtZpVh2Nu3OpISMq3mO4hizTC6bnkR2/eXrNlVtjRk267DNYW4f9eRGsOgKI9z3eujpEQr453VDDJhapFF4/AhAHcQ0UEAf4mq62iqkOREsqRW+bqJbmpF3oiB2Nu3Gk9tvgp7+1Zjz+FjgXUhtjRkk5wKl3tgO8b/XJSUaJkrIWSJzAWkmfk/ACxPW440SDqImUT6rJLrlm0H62YzRPGxm9wqLobNJch/87YD0LUTc7kHtiwv/+ujJBxIq4/oiDsufjJnHKYySQQxvT+awqwcch1U4xqJI322Z1kRN289oH0ujGKzGUdXwxYU5J9GhLLPOuQ6yeke9K5ZWBdzsL3ebyDUDsAkn7T6iIZkhiVDFt1KU5awK8cgF5Tfh31ipAwQUMjnYq0WHhgqoYNI+1wYxWYzjnFUrptiBrOnT3O6Bz3Liui/fgkK+dzkY3Nn5dD/jiXa14eNIUirj2iIOy4ZZOeQIcKsHF1WS7ofTXmcMXvGNBy4/cpYZLaN+wxSbH5XgK0wz+amcXUpmM5/spqW6oJtZ+KXY+TsWKidoMyViEaW3HHt5N4S45AhwlRNu7igmvGjMY37DBraozNuupYewDnjqFPMri6FgaGS8fxz8jltiqxrTYNJDhO2+y+tPsKTFXdcu7m3xDhkiDArRxfF34wfjUmOCWbrD8KUVhumMG9gqOQcCO/fdURrGADg1NmxSQOgftCDTx/H1u89a6xpUOdUn9OpM/W7BBMSQ4iXZraisdFuhY9iHDKG68rRRfG7/Gga3QZHNUAmo6LqLoLksbmzdOe3reTL4/XG5auPPqs9d3mcseHBQzhdnnDaJfiRGEL8ZMUdlyX3VhyIcWgR/Ep81aJ52L6/ZFX8QT+aOLbBpo6vqxbNM8reu2ah0agU8jn0rllozPDxtv+wQsCyzz6M4ZEyCrNy9mM1mIwOUA3sO1LI5zB7xrS28EFnmSy447Li3ooLmSHdApjmLV+3vIg9h4+FVjxBCraTCHe8sz4Dx7TL+PTA47h33zN17qBNa7sBQCv7a7rmYO+Tx+uv3UHoAOrSbU3nahQiaOse4iDt2ePtFBxtBVpx/rzMkG5xTL7MPYePYW/f6lDn0n2B/ej6Ddl2GXsOHzPOhlD/9j/3HY1hAIDxCYZfMtu5GoW58gN2PW+ukzB7+rTJGIUOArS7NNfgdhy0W3C0FciKeysuxDikjMvqLk5fpim7yI8/kGYLtpl2IDbXT9jFehi/fhhUw0EXV5VS6ABw89YD2veg+lR5cWnYl6WZGEJ0suDeigsxDk3A9MN3Xd3F6csMY1C8x9rmUDcDUxqqopPIGifQoSqb1Q/aNONCXX/oM+dqQwafPq51pZnSjoMa9mVlJoYgKKRCOmFsVbKulZ1xVs6GMSjeY5sVVMt1EnIdtdXWQYahkM/h3a+7xNpYL9dJyOfOfd11lc29axZCX+d97v2rqvR79z2DOfkc5s4KrjYPatiXRIVvlK6wguBFdg4BNLrdt/3wXVd3cfoyddlFuQ4CqDal0298dK+LG6/bxqVyWjE8Wsb2/aWaAP2cfA5EwPBIOTAl1nut17/8fHznyePaHYF/pzc8WkY+14kt65ZaP4ughn22XdnAUCm2z1nSaIUwiHGwoHP73LT1ANbvOIT117gFE20tIcK4i+LyZZoMje4x//Vm5jpiNQ7KFaQbNOQy0tRLlAC97vM9fuosXv/y87HvJycwzoxOIly3/JzbKYofP6hhny3eEdW91G7BUaH5iHGwYAreDo+WnX60tpYN6seaxurOZGhcW13ExTjz5PsNUq4u1w/rTzft6rwptuPM2L6/hBWXnh/Zj6/emy1byfT+Ggkit1NwVGg+U9o4BLmMbD96lx+tqWUDATXX0q3ibeMw48TFbeaa4aRqLx46+FyNErzqNy/EnsPHtKtj/320yaMe7zAEn009kky4GhMlYyOJATZFrR6/KYa254IQF1PWOLhkCgX5u4N+tLYWEeoafqURJT89alzE5VoDQyWnjCRvo72NPd3aa5mUnzp/kDwqw2vDg4fqqpRzHaTtkeR/L977VJiVc652Pjo8ii3rlia20+tZVjS6lySILKTBlM1WcskQCRotGfSjNT1ftLwubOZKI3OHg66lzu2CrdGey3lcsrfUefwKvZDP4byZ07Q9kryv7b3vYM19OjlSRq7TlJ9Uy5x8blK+zursirjmYShknoOQJabsziHM2EndStXlRxslphDWr91IsVPQtVzdSYDdULqcxxZTKA2PYkHfTqM7SfUu0qEeX7/jUF1AeALAjA7Ci18wE0erRsOEd1fiGitRuO7sJIgsZIkpaxzCjp2M4rqJ8mMP69eOUp3sei1XX7c31TPM/GcvakVuKmRjmJvhuWR+mdpdjJYnAjOGOknfudVmgE39q0rDozWV0X50bsZmxZ8EwcuUNQ5hV/VRMz/Cvi6MXDbXUadhbKdXgc/J55DrJGN9g0nhzp2Vw6zp0+qC6GHnP/tRK/KwWVGNZn7p0kyByj284mVztQ0CAbPx1DUi9KLafgd9L6L2R5KGe0IcpGIciOh6AOsBvArA5cw86HnuVgB/AGAcwMeYeVcSMmR1Cx9GLlsFrW6V7Vdaw6Nl5DoIc6uB2U6iGj+9SeHqGsbZagBcU1GL1VbkNsXqx+/eMd23uZbgs84wqB5JKzfvNl7bNL7VRX6XQHhYl6G/uR8gDfeE6KS1c/ghgLUAvuh9kIguA/AuAIsBXATgESL6DWZOpCw3q3ngrnLZ3DX+oLdJaZUnuK4zqVIom9Z2Y9Pa7oYn0/kVd2FWDidHypjwHTtydgwPHXwuULF2EmGCuU4e2327/erFxmwpk9ze/+sw9VGKqwN4mPiTrRZFGu4JUUjFODDzEwBA9a6PawF8jZnPAHiKiH4M4HIA322uhK2ByV2j6ii82JSWzh8/Wh7HhgcP1bmPTGmhc/I57Xl0858Hhkrovf8gJnx+fJfVNAHaWRNB9Cwr1q2qbczJVwYEGe8xVTqzqp2Rksc1TlPIBw8gChN/Cgr6S62EEJaspbIWATzr+ftn1cemLCoguaBvJ1Zu3l0TZ9ClPhKAG67ocm77bePESFmbIqtLnz11dqyuYZ61S+l4tPW1t0bEj+1eAcD6axbX3a9cJ2l/BKfOjlWMmCGdmRna1GGXmoRcB2H9NYsDjwuT2hr0+UqthBCWxIwDET1CRD/U/HdtTOe/kYgGiWjw2LFjcZwycwTVMPQsK2LT2m4UC/nJzqBb1i3VFqHFoRyUe0K3Si2PM86bOa1GlihdSoMw1Yi41Hvo7lf/O5ZgjmaMqGqn7X+NLtDvj9PojEmuA+eueb3bzkcnr+me2j5fqZUQopDqmFAi+ncAn1QB6WowGsy8qfr3LgDrmdnqVmrXMaGmhnO6gTJBxNUfSalGU1uQpzZfFXgOl0Z6HQT448S2kYtR75Wtclv3fhb07Qx877axqUm2QdF9vklPnBNaG9uY0Ky5lXYAeBcRzSCiBQBeCeB7KcuUGnEObFGrUFdMPvEOIhQ0K23AfXeyatE849wEBTNw57qlTqtmW4sP270KqtzWvU+XOQlBY1OTQLfLuHPdUgx95koxDEIk0kplfTuAvwUwD8BOIjrAzGuY+RARbQPwIwBjAP44qUylViDOCXCAvX+PF7Xa1K1Ex5nx/OmxuvoIQkXpBzEwVML2/aXAjJ6LCnmngjAAVgXfSOW2blPtUk+R1hS2rGbfCa1JWtlKXwfwdcNznwPwueZKlE2SaOntUnPAfC7oe8u2g3U1E+UJRj7XgbFxnlTyDEy2tvYrdG9W08jZsUDXlu49mgrCbDMmqHrcys27Q83mVpz0ZDb5iwdn5jqMg4TiNuqCkAZZcysJHsIEJKOc04RSij3LisaWFaPliUDXiS5IbEtXtb1HU0GY7XxKPlMzQtfGif73MTxaxunyBG64ogtAJaXVmx2V1QZ6QdlcguBlyrbPaBWScBWoc5qCuF6laet3pMO7Gg/TuC8ocNyoS0ZXCGbbRXmVuckweYPOukrkLFXfR23FIUxdZOcwhXFZ4YYxDEAlYK1WpGEb99mIwyWjm8193fJiXXqqf/dim8vhxbtz6llWxN6+1Xhq81XY27c6dQUcthW8IIhxmMK4uK1M7qdCPqfN5x9nnnThmBR6IZ8L7SoLmq3RSTR5PlOmlV8eFRz3GkBdK+4whimrlchpBcmF1kWMwxQnaIVr2l2sv2YxNq3tthaFmbKX3rbkQuztW40t65YCqPfZm+S0peKOM0++B10ltG534rqaNlWi68hq0NklBVcQvEjMQbAS5D+3zT3ec1hfuX7Pvmew87Hn8PzpscmOqMoHPvj0cew5fEx7LVsqLqGyE/DGaIJ8/q6rad35Vi2ah+37S4mMDE2CJDLfhPYm1QrpuGjXCumsMzBUws1bD2hrFoqFfOB0NR0EWCuLg66pC2qb5hs0WoHeanMTmiFvq92TqY6tQlqMgxAZk3IlAFvWLXUquHPBr6zn9+3UHqdrd6FrK6EMDlA/njTpNhdRaBWFa7vXWZRXaK32GUILYcvi6VlWDAwiR72Oa8AZCB6YE7aOpNm1Ai4NBbOCZES1FxJzECJjqgRWGU5eX73LDsLvUvJeRzEwVMKps2N1x+Q6KFQra/V4mDqSNGoFwk6DSxPJiGovZOcgRMalTkJlQ925bql2lkIhn5tctd9wRVfg+UyzIM6bOS1UK+soWTpRV8aN7DZaSeFKRlR7ITsHITJhKoFdj11x6fnWY0xKcdjQRiOOLB3l82+k82vU3UYr9WmSjKj2QgLSQkthmwVRrKaY+lNhgeitLFzmYNiym+LIiGqlIG+rBM+FCraAtOwchJbC1g+pNDyKe/Y9U/P3rQ88jk1ru0MPR1IE9YcKWhk36hbKYp8mG9I2vH0Q49CmtOsKLmyQu9Hgre0aRYf7GodbyG8gvP2bBCEpxDi0GC5KvxU7cIYxZmp1ahrZ6Sdq8HZgqGTMoHJ1C8UV82i1z1NofSRbqYVwzXlvtXzzqLn8rqvvqMHb/l1HjPOiXZV7HDM5Wu3zFNoD2Tm0EK45762U/ghEz+V3mWrXSLZMUJGfK4364Vvt8xTaA9k5tBCuSqLV8s2jKj/dqvw9V3TFNjnPdL9sU/SSoNU+T6E9kJ1DC+Ea3Gy1fPNGgrZJZsckcR+jJAq02ucptAeyc2ghXGcTJzF7OkmyOnM57vsYNbbSap+n0B6kUgRHRNcDWA/gVQAuZ+bB6uNvBrAZwHQAZwH0MvPuoPNNpSK4dk1Rbdf35aXRgjhBiJssFsH9EMBaAF/0Pf5LAFcz81EiejWAXQDaS0M0SLsWGbXr+/IigWWhlUjFODDzEwBAvhGTzDzk+fMQgDwRzWDmM00UTxASoZX6JAlClmMO1wH4gRgGoV3IamxFEHQktnMgokcAvFTz1G3M/I2A1y4G8HkAV1qOuRHAjQDQ1dXVgKSC0BxarU+SMLVJtSsrEf07gE+qgHT1sYsB7AbwAWbe63KeqRSQFgRBiIuWGRNKRAUAOwH0uRoGQRAEIX5SMQ5E9HYi+hmA3wKwk4h2VZ/6KIBXAPgMER2o/vfiNGQUBEGYyqSVrfR1AF/XPL4RwMbmSyQIgiB4yZRbSRAEQcgGYhwEQRCEOtpihjQRHQPwdMiXXYBKRXaWERnjQWSMB5ExHrIk46XMPE/3RFsYhygQ0aAphSsriIzxIDLGg8gYD60gIyBuJUEQBEGDGAdBEAShjqlsHO5KWwAHRMZ4EBnjQWSMh1aQcerGHARBEAQzU3nnIAiCIBgQ4yAIgiDUIcYBABHdQkRMRBekLYsfIvoLInqs2mfqYSK6KG2Z/BBRPxEdrsr59WoDxUxBRNcT0SEimiCiTKUREtFbiOgIEf2YiPrSlscPEd1NRL8goh+mLYsJIrqEiPYQ0Y+qn/PH05bJDxHNJKLvEdHBqowb0pbJxpQ3DkR0CSpzI55JWxYD/cz8m8y8FMBDAD6Tsjw6vgng1cz8mwD+H4BbU5ZHhxpN++20BfFCRJ0A/h7A7wK4DMC7ieiydKWq48sA3pK2EAGMAbiFmS8DcAWAP87gfTwDYDUzLwGwFMBbiOiKdEUyM+WNA4AtAD4FIJOReWb+lefP2cignMz8MDOPVf/cB+DiNOXRwcxPMPORtOXQcDmAHzPzT5j5LICvAbg2ZZlqYOZvAziethw2mPk5Zv5B9d+/BvAEMjZ/nis8X/0zV/0vc79nxZQ2DkR0LYASMx9MWxYbRPQ5InoWwA3I5s7BywcB/GvaQrQQRQDPev7+GTKm1FoNIpoPYBmAR1MWpQ4i6iSiAwB+AeCbzJw5GRWptOxuJrZxpQD+DJZRpM0iaKQqM98G4DYiuhWVmRe3N1VAuI19JaLbUNne39tM2RSNjKYV2gMiOg/AdgA3+XbdmYCZxwEsrcblvk5Er2bmTMZy2t44MPObdI8TUTeABQAOEhFQcYX8gIguZ+afN1FEo4wa7gXwL0jBOATJSETvB/A2AG/klIpnQtzHLFECcInn74urjwkhIaIcKobhXmZ+IG15bDDzMBHtQSWWk0njMGXdSsz8ODO/mJnnM/N8VLbzr2m2YQiCiF7p+fNaAIfTksUEEb0FlbjNNcw8krY8Lcb3AbySiBYQ0XQA7wKwI2WZWg6qrPC+BOAJZv7rtOXRQUTzVCYfEeUBvBkZ/D0rpqxxaCE2E9EPiegxVFxgmUvRA/B3AF4A4JvVlNt/SFsgP5bRtKlSDeR/FMAuVIKo25j5ULpS1UJEXwXwXQALiehnRPQHacukYSWA3wew2jNi+K1pC+XjQgB7qr/l76MSc3goZZmMSPsMQRAEoQ7ZOQiCIAh1iHEQBEEQ6hDjIAiCINQhxkEQBEGoQ4yDIAiCUIcYB0FICSL6MBG9N205BEGHpLIKgiAIdcjOQRAcIKLXVudVzCSi2dV+/K/2HXM1ET1KRENE9AgRvaT6+N8Q0Weq/15DRN8mog4iWk9En6w+/rHqLILHiOhrzX+HglCL7BwEwREi2ghgJoA8gJ8x8ybf83MBDDMzE9EfAngVM99CRLNQqYj9KIB/APBWZn6SiNYDeJ6Z/4qIjgJYwMxniKjAzMNNfGuCUEfbN94ThBj5LCpK/jSAj2mevxjAViK6EMB0AE8BADOPENGHUBk0dDMzP6l57WMA7iWiAQAD8YsuCOEQt5IguPMiAOeh0kdqZnXOxoFqf34A+FsAf8fM3QD+CJVdhqIbwH8DMI15vQqViXCvAfB9IpKFm5AqYhwEwZ0vAvhzVFqnf56Zb2PmpdURrgAwB+fabb9PvYiILgVwCyoDaH6XiF7nPSkRdQC4hJn3APjT6nnOS/KNCEIQsjoRBAeqKadlZv7n6tzn7xDRambe7TlsPYD7iOgEgN0AFnhaSX+SmY9WO5p+mYhe63ldJ4B7iGgOAALwvyTmIKSNBKQFQRCEOsStJAiCINQhxkEQBEGoQ4yDIAiCUIcYB0EQBKEOMQ6CIAhCHWIcBEEQhDrEOAiCIAh1/H/KEY1To5oR2QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# as usual we can visualize our dataset\n", "import matplotlib.pyplot as plt\n", "fig = plt.figure() \n", "ax = fig.add_subplot()\n", "ax.scatter(X[:, 0], X[:, 1])\n", "plt.title(\"Blobs Dataset\")\n", "plt.xlabel(\"x-axis\")\n", "plt.ylabel(\"y-axis\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 20, "id": "improved-intake", "metadata": {}, "outputs": [], "source": [ "# well the first thing we have to do is setup our classifiers. We are going to be using k-nearest-neighbors, \n", "# gaussian naive bayes, and a decision tree.\n", "\n", "from sealion.naive_bayes import GaussianNaiveBayes\n", "from sealion.nearest_neighbors import KNearestNeighbors\n", "\n", "knn = KNearestNeighbors(k = 5) \n", "gnb = GaussianNaiveBayes()\n", "dt = DecisionTree(min_samples = 5, max_branches = 25)" ] }, { "cell_type": "code", "execution_count": 30, "id": "fewer-thomas", "metadata": {}, "outputs": [], "source": [ "# then we can setup our classifiers dict\n", "\n", "classifiers_dict = {\"k-nearest-neighbors\" : knn, \"gaussian_nb\" : gnb, \"decision_trees\" : dt} # give a name to each of your classifiers\n", "\n", "ec = EnsembleClassifier(classifiers_dict, classification = True) # set it up (note : can't use this for neural nets)\n", "\n", "# classification = True because we are using classification, but set it False for regression. The default is True,\n", "# and an easy way to remember this is because its an EnsembleCLASSifier. \n", "\n", "ec.fit(X_train, y_train) # we can now train the all classifiers - will train all algos on all CPU cores available in parallel" ] }, { "cell_type": "code", "execution_count": 31, "id": "metallic-therapy", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.9866666666666667" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# well .... let's evaluate it\n", "ec.evaluate(X_test, y_test)" ] }, { "cell_type": "code", "execution_count": 32, "id": "weighted-german", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "k-nearest-neighbors : 0.9866666666666667\n", "gaussian_nb : 0.9866666793823242\n", "decision_trees : 0.9733333333333334\n" ] } ], "source": [ "# looks like it did well, but even at the top there's a hierarchy. Let's see which one did best. \n", "ec.evaluate_all_predictors(X_test, y_test)" ] }, { "cell_type": "code", "execution_count": 27, "id": "compatible-grass", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/plain": [ "array([ True, True, True, True, True, True, True, True, True,\n", " True, True, True, True, True, True, False, True, True,\n", " True, True, True, True, True, True, True, True, True,\n", " True, True, True, True, True, True, True, True, True,\n", " True, True, True, True, True, True, True, True, True,\n", " True, True, True, True, True, True, True, True, True,\n", " True, True, True, True, True, True, True, True, True,\n", " True, True, True, True, True, True, True, True, True,\n", " True, True, True])" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Looks like its the Gaussian Naive Bayes (I was expecting the KNN to do best)\n", "# we could also just get that class like : \n", "\n", "best_predictor = ec.get_best_predictor(X_test, y_test)\n", "print(best_predictor)\n", "\n", "# and we can use it...\n", "y_pred = best_predictor.predict(X_test)\n", "y_pred == y_test # ... do more things ... " ] }, { "cell_type": "code", "execution_count": null, "id": "incorporate-tattoo", "metadata": {}, "outputs": [], "source": [ "# well why is this useful? Well for one thing imagine if you knew you were going to use a KNN but wasn't sure\n", "# about the k-value. Well you could just add multiple KNN classes here - all w/ different k-values - and just \n", "# get the class that works best and use it. That way hyperparameter tuning is a breeze. \n", "# That too this module trains algorithms extremely fast, so you could take advantage of that. \n", "\n", "# If you think you have a better way to use it, please let me know @anish.lakkapragada@gmail.com or on GitHub. \n", "# In the meantime, thank you!" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.7" } }, "nbformat": 4, "nbformat_minor": 5 }