{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%matplotlib inline\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import texttable as tt\n", "from sklearn import preprocessing\n", "from sklearn import model_selection \n", "from sklearn import linear_model \n", "from sklearn import svm\n", "from sklearn import neural_network" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Loading the data" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [], "source": [ "data = pd.read_csv('data/train.csv', index_col='PassengerId')\n", "del data.index.name # lets also remove this row with just the name on it to make things easier later\n", "data_test = pd.read_csv('data/test.csv', index_col='PassengerId')\n", "del data_test.index.name # lets also remove this row with just the name on it to make things easier later" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SurvivedPclassNameSexAgeSibSpParchTicketFareCabinEmbarked
103Braund, Mr. Owen Harrismale22.010A/5 211717.2500NaNS
211Cumings, Mrs. John Bradley (Florence Briggs Th...female38.010PC 1759971.2833C85C
313Heikkinen, Miss. Lainafemale26.000STON/O2. 31012827.9250NaNS
\n", "
" ], "text/plain": [ " Survived Pclass Name \\\n", "1 0 3 Braund, Mr. Owen Harris \n", "2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... \n", "3 1 3 Heikkinen, Miss. Laina \n", "\n", " Sex Age SibSp Parch Ticket Fare Cabin Embarked \n", "1 male 22.0 1 0 A/5 21171 7.2500 NaN S \n", "2 female 38.0 1 0 PC 17599 71.2833 C85 C \n", "3 female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data[0:3]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So, what is the data we have to work with:\n", "\n", "Survivied is our target variable we wish to predict, a value of 1 means survival, a value of 0 means death.\n", "\n", "Pclass is a proxy for socio-economic status the values are as so:\n", "- 1st = Upper\n", "- 2nd = Middle\n", "- 3rd = Lower\n", "\n", "SibSp quantifies how many siblings/spouses were aboard with the passenger.\n", "\n", "Parch quantifies how many parents/children were aboard with the passenger.\n", "\n", "Embarked codifies the port at which the passenger embarked as so:\n", "- C = Cherbourg\n", "- Q = Queenstown\n", "- S = Southampton\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Feature Engineering / data Pre-processing" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to use many machine learning algorithms we first have to do some preprocessing to get it into an appropriate format. We'll go through the features one at a time." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 1: Pclass\n", "\n", "First lets check for missing data entries" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Num of missing entries in training data = 0\n", "Num of missing entries in test data = 0\n" ] } ], "source": [ "print('Num of missing entries in training data = {}'.format(sum(np.isnan(data.Pclass))))\n", "print('Num of missing entries in test data = {}'.format(sum(np.isnan(data_test.Pclass))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are no missing entries here so we move on." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Pclass is a categorical variable, these need to be encoded into binary variables. This is nessesary because of the way the algorithm interprets numbers. If we have a categorical feature that takes values 0, 1, 2, 3, 4 it assumes the higher numbers are 'better' (e.g. 4>3) even though they are arbitrary encodings, because ultimately it is calculating values/weights/parameters to be multiplied by these feature variables to give a term which enters into the linear regression. One common way to deal with this is one-hot-encoding, where a feature N takes values 0, 1, 2 for example we would generate 3 features which takes binary values 0 or 1. An example is shown below\n", "\n", "We have the original feature data:\n", "\n", "| Entry | N |\n", "| ------------ |:----------:|\n", "| 0 | 1 | \n", "| 1 | 2 |\n", "| 2 | 0 |\n", "| 3 | 1 |\n", "| 4 | 2 |\n", "| 5 | 0 |\n", "\n", "Which when encoded becomes:\n", "\n", "| Entry | N==0 | N==1 | N==2 |\n", "| ------------ |:----------:|:----------:|:----------:|\n", "| 0 | 0 | 1 | 0 | \n", "| 1 | 0 | 0 | 1 | \n", "| 2 | 1 | 0 | 0 | \n", "| 3 | 0 | 1 | 0 | \n", "| 4 | 0 | 0 | 1 | \n", "| 5 | 1 | 0 | 0 | " ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# on training data\n", "\n", "Pclass_lb = preprocessing.LabelBinarizer()\n", "Pclass_one_hot_encoded = Pclass_lb.fit(data['Pclass']) # one-hot encoding\n", "Pclass_one_hot_encoded = Pclass_lb.transform(data['Pclass']) # one-hot encoding\n", "\n", "dfOneHot_Encoded = pd.DataFrame(\n", " Pclass_one_hot_encoded, \n", " columns = [\"Pclass_\"+str(int(i+1)) for i in range(Pclass_one_hot_encoded.shape[1])],\n", " index=data.index\n", " ) # we now construct a dataframe out of this one-hot-encoded data\n", "\n", "# we now add our one-hot-encoded Embarked features\n", "data = pd.concat([data, dfOneHot_Encoded], axis=1)\n", "del(data['Pclass']) # and delete the original feature\n", "\n", "# on testing data\n", "\n", "Pclass_one_hot_encoded = Pclass_lb.transform(data_test['Pclass']) # one-hot encoding\n", "dfOneHot_Encoded = pd.DataFrame(\n", " Pclass_one_hot_encoded, \n", " columns = [\"Pclass_\"+str(int(i+1)) for i in range(Pclass_one_hot_encoded.shape[1])],\n", " index=data_test.index\n", " ) # we now construct a dataframe out of this one-hot-encoded data\n", "\n", "# we now add our one-hot-encoded Embarked features\n", "data_test = pd.concat([data_test, dfOneHot_Encoded], axis=1)\n", "del(data_test['Pclass']) # and delete the original feature" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 2: Name\n", "\n", "We will drop this feature as names are unlikely to effect survivability." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [], "source": [ "data.drop(labels=['Name'], axis=1, inplace=True)\n", "data_test.drop(labels=['Name'], axis=1, inplace=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 3: Sex\n", "Sex can be encoded easily as a single binary feature as it only has 2 values, male or female." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "array(['male', 'female'], dtype=object)" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data.Sex.unique()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "le_sex = preprocessing.LabelEncoder()\n", "le_sex.fit(data.Sex) # fits a value to each unique integer value of the feature variable sex\n", "data.Sex = le_sex.transform(data.Sex) # transform the data from labels to numeric\n", "\n", "data_test.Sex = le_sex.transform(data_test.Sex) # transform the data from labels to numeric" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 4: Age\n", "\n", "Lets first check for missing entries" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Num of missing entries in training data = 177\n", "Num of missing entries in test data = 86\n" ] } ], "source": [ "print('Num of missing entries in training data = {}'.format(sum(np.isnan(data.Age))))\n", "print('Num of missing entries in test data = {}'.format(sum(np.isnan(data_test.Age))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have quite a lot of missing age data, lets just try replacing it with the median of the age data we have." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [], "source": [ "imputer = preprocessing.Imputer(strategy=\"median\", axis=0)\n", "data['Age'] = imputer.fit_transform(data['Age'].values.reshape(-1, 1))\n", "data_test['Age'] = imputer.transform(data_test['Age'].values.reshape(-1, 1))" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Num of missing entries in training data = 0\n", "Num of missing entries in test data = 0\n" ] } ], "source": [ "print('Num of missing entries in training data = {}'.format(sum(np.isnan(data.Age))))\n", "print('Num of missing entries in test data = {}'.format(sum(np.isnan(data_test.Age))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Age is our first continous feature. There are 2 approaches one can take to continuous data such as this, one is to leave it as continous data but perform feature scaling (make it a similar scale to other parameters), another is to discretise it into bins. The second approach *can* help the learning algorithm to pick up on general trends, such as young people and old people perhaps (don't know if this is true for this data) being more likely to die. However we don't know for sure which approach is best for this particular data and should test it rather than picking one based on a *hunch* or preconcieved notions of what approach is best." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For now I will leave this as a continous variable and perform feature scaling, later I will try the binning method and compare the results with the different learning algorithms to decide which approach is best for this data.\n", "\n", "Feature scaling makes it so the mean is 0 and the variance one, applying this to all continous variables means they are all on the same scale and thus will be weighted similarly and the parameter space can converge in minimising the cost function." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [], "source": [ "scaler_age = preprocessing.StandardScaler().fit(data['Age'].values.reshape(-1, 1))\n", "data['Age'] = scaler_age.transform(data['Age'].values.reshape(-1, 1))\n", "data_test['Age'] = scaler_age.transform(data_test['Age'].values.reshape(-1, 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 5 & 6: SibSp and Parch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "These 2 parameters can be combined to parameterise the number of relatives aboard the ship, which seems like a sensible combination to make." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [], "source": [ "data['NumRels'] = data['SibSp'] + data['Parch']\n", "data_test['NumRels'] = data_test['SibSp'] + data_test['Parch']\n", "data.drop(labels=['SibSp', 'Parch'], axis=1, inplace=True)\n", "data_test.drop(labels=['SibSp', 'Parch'], axis=1, inplace=True)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Num of missing entries in training data = 0\n", "Num of missing entries in test data = 0\n" ] } ], "source": [ "print('Num of missing entries in training data = {}'.format(sum(np.isnan(data.NumRels))))\n", "print('Num of missing entries in test data = {}'.format(sum(np.isnan(data_test.NumRels))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will now treat this new feature as a continuous variable rather than treating it as a catagorical variable which takes on discrete values, which is another valid approach. Therefore we perform feature scaling on this new continous variable." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Users/ajs3g11/anaconda/envs/python3/lib/python3.6/site-packages/sklearn/utils/validation.py:444: DataConversionWarning: Data with input dtype int64 was converted to float64 by StandardScaler.\n", " warnings.warn(msg, DataConversionWarning)\n" ] } ], "source": [ "scaler_NumRels = preprocessing.StandardScaler().fit(data['NumRels'].values.reshape(-1, 1))\n", "data['NumRels'] = scaler_NumRels.transform(data['NumRels'].values.reshape(-1, 1))\n", "data_test['NumRels'] = scaler_NumRels.transform(data_test['NumRels'].values.reshape(-1, 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 7: Ticket" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We just delete this information since they are unique ticket numbers and contain no relevant information." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "data.drop(labels=['Ticket'], axis=1, inplace=True)\n", "data_test.drop(labels=['Ticket'], axis=1, inplace=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 8: Fare" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Fare is another continous feature we want to feature scale, or could discretise alternatively, here we feature scale and treat it as continous." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Num of missing entries in training data = 0\n", "Num of missing entries in test data = 1\n" ] } ], "source": [ "print('Num of missing entries in training data = {}'.format(sum(np.isnan(data.Fare))))\n", "print('Num of missing entries in test data = {}'.format(sum(np.isnan(data_test.Fare))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "However before we do that we need to take care of the missing entry in the testing data. We will use the median of the training data to replace the missing entry." ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": true }, "outputs": [], "source": [ "imputer = preprocessing.Imputer(strategy=\"median\", axis=0)\n", "imputer.fit(data['Fare'].values.reshape(-1, 1))\n", "data_test['Fare'] = imputer.transform(data_test['Fare'].values.reshape(-1, 1))" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false }, "outputs": [], "source": [ "scaler_Fare = preprocessing.StandardScaler().fit(data['Fare'].values.reshape(-1, 1))\n", "data['Fare'] = scaler_Fare.transform(data['Fare'].values.reshape(-1, 1))\n", "data_test['Fare'] = scaler_Fare.transform(data_test['Fare'].values.reshape(-1, 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 9: Cabin" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Num of missing entries in training data = 687\n", "Num of missing entries in test data = 327\n" ] } ], "source": [ "print('Num of missing entries in training data = {}'.format(sum(pd.isnull(data['Cabin']))))\n", "print('Num of missing entries in test data = {}'.format(sum(pd.isnull(data_test['Cabin']))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There may well be some useful information here about the position on the ship of the passanger's cabins but there are so many missing values that we will just delete this feature for now." ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": true }, "outputs": [], "source": [ "data.drop(labels=['Cabin'], axis=1, inplace=True)\n", "data_test.drop(labels=['Cabin'], axis=1, inplace=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Feature 10: Embarked" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As with the PClass data we want to encode this categorical feature into many binary features, we once again you one-hot encoding." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Num of missing entries in training data = 2\n", "Num of missing entries in test data = 0\n" ] } ], "source": [ "print('Num of missing entries in training data = {}'.format(sum(pd.isnull(data.Embarked))))\n", "print('Num of missing entries in test data = {}'.format(sum(pd.isnull(data_test.Embarked))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since there's only 2 null values in the training data we'll drop them." ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SurvivedSexAgeFareEmbarkedPclass_1Pclass_2Pclass_3NumRels
101-0.565736-0.502445S0010.059160
2100.6638610.786845C1000.059160
5010.433312-0.486337S001-0.560975
\n", "
" ], "text/plain": [ " Survived Sex Age Fare Embarked Pclass_1 Pclass_2 Pclass_3 \\\n", "1 0 1 -0.565736 -0.502445 S 0 0 1 \n", "2 1 0 0.663861 0.786845 C 1 0 0 \n", "5 0 1 0.433312 -0.486337 S 0 0 1 \n", "\n", " NumRels \n", "1 0.059160 \n", "2 0.059160 \n", "5 -0.560975 " ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data[0:5].drop([3, 4])" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[62, 830]\n" ] } ], "source": [ "indexes_of_nulls = [i for i, x in zip(data.index, pd.isnull(data.Embarked)) if x]\n", "print(indexes_of_nulls)\n", "data.drop(np.array(indexes_of_nulls), inplace=True) # remove examples with null values for embarked" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# on training data\n", "\n", "Embarked_lb = preprocessing.LabelBinarizer()\n", "Embarked_one_hot_encoded = Embarked_lb.fit(data['Embarked']) # one-hot encoding\n", "Embarked_one_hot_encoded = Embarked_lb.transform(data['Embarked']) # one-hot encoding\n", "\n", "dfOneHot_Encoded = pd.DataFrame(\n", " Embarked_one_hot_encoded, \n", " columns = [\"Embarked_\"+str(int(i)) for i in range(Embarked_one_hot_encoded.shape[1])],\n", " index=data.index\n", " ) # we now construct a dataframe out of this one-hot-encoded data\n", "\n", "# we now add our one-hot-encoded Embarked features\n", "data = pd.concat([data, dfOneHot_Encoded], axis=1)\n", "del(data['Embarked']) # and delete the original feature\n", "\n", "# on testing data\n", "\n", "Embarked_one_hot_encoded = Embarked_lb.transform(data_test['Embarked']) # one-hot encoding\n", "dfOneHot_Encoded = pd.DataFrame(\n", " Embarked_one_hot_encoded, \n", " columns = [\"Embarked_\"+str(int(i)) for i in range(Embarked_one_hot_encoded.shape[1])],\n", " index=data_test.index\n", " ) # we now construct a dataframe out of this one-hot-encoded data\n", "\n", "# we now add our one-hot-encoded Embarked features\n", "data_test = pd.concat([data_test, dfOneHot_Encoded], axis=1)\n", "del(data_test['Embarked']) # and delete the original feature" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
SurvivedSexAgeFarePclass_1Pclass_2Pclass_3NumRelsEmbarked_0Embarked_1Embarked_2
101-0.565736-0.5024450010.059160001
2100.6638610.7868451000.059160100
310-0.258337-0.488854001-0.560975001
4100.4333120.4207301000.059160001
5010.433312-0.486337001-0.560975001
\n", "
" ], "text/plain": [ " Survived Sex Age Fare Pclass_1 Pclass_2 Pclass_3 NumRels \\\n", "1 0 1 -0.565736 -0.502445 0 0 1 0.059160 \n", "2 1 0 0.663861 0.786845 1 0 0 0.059160 \n", "3 1 0 -0.258337 -0.488854 0 0 1 -0.560975 \n", "4 1 0 0.433312 0.420730 1 0 0 0.059160 \n", "5 0 1 0.433312 -0.486337 0 0 1 -0.560975 \n", "\n", " Embarked_0 Embarked_1 Embarked_2 \n", "1 0 0 1 \n", "2 1 0 0 \n", "3 0 0 1 \n", "4 0 0 1 \n", "5 0 0 1 " ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data[0:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### We can now seperate our feature and target variables for the training data and begin to train some machine learning classifiers!" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false }, "outputs": [], "source": [ "X = data.drop([\"Survived\"], axis=1)\n", "y = data[\"Survived\"]\n", "\n", "X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size=0.10, random_state=3000) # we will also hold back 10% of our data as test data to appraise the fitted and tuned models after validation before applying them to our Kaggle test data" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "39.0\n", "31.4606741573\n" ] } ], "source": [ "print(sum(y_train)/len(y_train)*100)\n", "print(sum(y_test)/len(y_test)*100)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we do not stratify y then we can get a very uneven distribution of survivers in our train and test data. Below I stratify on y so that we get an even distribution of survivors in both." ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": true }, "outputs": [], "source": [ "X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size=0.10, stratify=y, random_state=3000) # we will also hold back 10% of our data as test data to appraise the fitted and tuned models after validation before applying them to our Kaggle test data" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "38.25\n", "38.202247191\n" ] } ], "source": [ "print(sum(y_train)/len(y_train)*100)\n", "print(sum(y_test)/len(y_test)*100)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Validation Methods\n", "\n", "Before we start training models it's important to consider how we will assess the accuracy of the predictions it makes. Assessing this on the data we used to train it is a very bad practise and is called the *resubstitution error*. This is because you cannot know if you are overfitting or not if you assess your model on the training data.\n", "\n", "So how do we assess the quality of our learning algorithms? There are a few approaches called Validation (because they validate our model). It is also important to have a 3rd set of test data. The purpose of this is that we train many learning algorithms on the trianing data and validate the performance of them on the validation data to pick which one to use. The performance of the algorithm on the validation data is no-longer a fair assessment of the accuracy, as the algorithm was picked for the fact it performed well on that data. We now assess the accuracy of the model fitted with training data and tuned (as in, we pick the algorithm we are using and the hyperparameters for that algorithm) by validation data on data it has never seen, i.e. test data. This should give us a fair assessment of the actual accuracy of the algorithm.\n", "\n", "#### Holdout\n", "The first approach is simple, we simply split the data into a training set and validation (also called cross-validation) set, also called the holdout set (as we hold it out from the training set). We train the model on the training data and validate the model on the valdiation data. In this case it is important to be careful that the sets contain similar occurances of different classes in the data, this process of equal distribution of classes is called *stratification*.\n", "\n", "#### K-Fold Cross-Validation\n", "The problem with this first approach is that we lost part of the training data we'd like our model to learn from, by reducing the data in this way we are forced to use a lower bias model (more data => higher variance models can be fit => higher accuracy can be achieved).\n", "\n", "In K-Fold cross validation the data is divided into k subsets and we perform the hold-out method k times, such that each time, one of the k subsets is used as the validation set and the other k-1 subsets are put together to form a training set. The error estimation is averaged over all k trials to get total effectiveness of our model. In this scheme every data point gets to be in a validation set exactly once, and every data point gets to be in a training set k-1 times. \n", "\n", "#### Stratified K-Fold Cross Validation\n", "This is the same as above but care is taken such that each fold contains approximately the same percentage of samples of each target class as the complete set, or in case of prediction problems, the mean response value is approximately equal in all the folds.\n", "\n", "#### Leave-$P$-out Cross Validation (or Leave-One-Out Cross Validation [LOOCV])\n", "This approach is similar to k-folds cross validation. We leave $P$ points out of the training data and validate with these p points. We then repeat this for all possible combinations. Commonly $P$ is set to $1$ and this becomes ***Leave-one-out Cross Validation***. This is generally preferred as it is easy to compute all combinations (there are $m$, where m = number of training examples)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Logistic Regression" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# We now use scikit learn to fit a regularised logistic regression \n", "# model with each of the feature variables being linear\n", "# setting fit_intercept=True fits the Theta_0 term - an intercept term\n", "\n", "lr_model = linear_model.LogisticRegression(fit_intercept=True)\n" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# lets use stratified K-folds cross validation with 10 folds to evaluate our algorithm - can use sklearn.model_selection.cross_val_score for this" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false }, "outputs": [], "source": [ "N_splits = 10\n", "kfold = model_selection.KFold(n_splits=N_splits)\n", "\n", "scoring = 'accuracy'\n", "score = model_selection.cross_val_score(lr_model, X_train, y_train, cv=kfold, n_jobs=1, scoring=scoring)\n" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "80.0" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "score.mean()*100 # we get an average accuracy of ~80% over 10 folds with logistic regression" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.56249999999999989" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "score.var()*100 # we get a variance of ~0.1% so the accuracy is also pretty consistant" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n", " intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n", " penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n", " verbose=0, warm_start=False)" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lr_model = linear_model.LogisticRegression(fit_intercept=True)\n", "# we now train our model, which we've confirmed should give ~80% accuracy, on our entire training data set \n", "lr_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.7865168539325843" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# and test it on our test data we have held back from the training/validation set\n", "lr_model.score(X_test, y_test) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We get ~78% accuracy, which is sligtly lower accuracy. This suggests the model is fitted correctly and is performing well, we now predict on Kaggle's test data and save the prediction to submit to Kaggle." ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "collapsed": true }, "outputs": [], "source": [ "prediction = lr_model.predict(data_test)" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "collapsed": true }, "outputs": [], "source": [ "with open('prediction_submission_LR.csv', 'w') as file:\n", " print('PassengerId,Survived', file=file)\n", " for i, id_ in enumerate(data_test.index):\n", " print('{},{}'.format(id_, prediction[i]), file=file)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When we submit this result to Kaggle we get a reported accuracy of **~76.5%** which is pretty similar to when we evaluated on our held back test data." ] }, { "cell_type": "code", "execution_count": 40, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4VFX6wPHvmWTSK2lAQggl9E7oiiAq2BsoFhR1Ze2u\n7Krs6tq2uerPtSyuYi8oIjZUREREVGoo0mtooYaE9J6c3x9nJplMZpJJyJAA7+d58kzmzr13zqTc\n9572HqW1RgghhKiLpbkLIIQQouWTYCGEEKJeEiyEEELUS4KFEEKIekmwEEIIUS8JFkIIIeolwUII\nIUS9JFgIIYSolwQLIYQQ9fJt7gI0lejoaJ2UlNTcxRBCiFPK6tWrj2mtY+rb77QJFklJSaSmpjZ3\nMYQQ4pSilNrryX7SDCWEEKJeEiyEEELUS4KFEEKIep02fRZCCO8oKysjPT2d4uLi5i6KOAEBAQEk\nJCRgtVobdbwECyFEndLT0wkNDSUpKQmlVHMXRzSC1prMzEzS09Pp0KFDo84hzVBCiDoVFxcTFRUl\ngeIUppQiKirqhGqHEiyEEPWSQHHqO9HfoQSLomxY/DQcWN3cJRFCiBZLggXA4n/B3qXNXQohhAvZ\n2dm88sorDT7uoosuIjs72wslOjNJsAgIB2sw5B5s7pIIIVxwFywqKirqPG7evHlERER4q1hnHBkN\npRSEtYXcA81dEiGEC9OmTWPXrl3069cPq9VKSEgIbdq0Yd26dWzevJkrrriC/fv3U1xczP3338+U\nKVOA6hRA+fn5XHjhhZx11lksXbqU+Ph4vvzySwIDA5v5k51aJFiALVhIzUKI+jz51SY2H8xt0nP2\naBvG45f2dPv6008/zcaNG1m3bh2LFy/m4osvZuPGjVVDQN966y1atWpFUVERgwYN4uqrryYqKqrG\nOXbs2MFHH33E66+/zjXXXMOnn37KjTfe2KSf43QnzVAAYfESLIQ4RQwePLjGXIGXXnqJvn37MnTo\nUPbv38+OHTtqHdOhQwf69esHwMCBA9mzZ8/JKu5pQ2oWYGoWeYehohx85EcihDt11QBOluDg4Krv\nFy9ezMKFC1m2bBlBQUGMGjXK5VwCf3//qu99fHwoKio6KWU9nUjNAkyw0BVQcLS5SyKEcBIaGkpe\nXp7L13JycoiMjCQoKIitW7eyfPnyk1y6M4fcRgOEJ5jH3IMmcAghWoyoqChGjBhBr169CAwMJC4u\nruq1cePG8eqrr9KnTx+6du3K0KFDm7GkpzcJFlAdIHIPACnNWhQhRG0ffvihy+3+/v58++23Ll+z\n90tER0ezcePGqu1/+tOfmrx8ZwJphgLTwQ3SyS2EEG5IsAAIjATfAJlrIYQQbkiwAIeJeVKzEEII\nVyRY2MlcCyGEcEuChZ2k/BBCCLckWNiFtYXcQ1BZ2dwlEUKIFserwUIpNU4ptU0ptVMpNc3F63co\npTYopdYppX5RSvVweO3PtuO2KaXGerOcgGmGqiyDggyvv5UQ4vSzePFiLrnkEgDmzp3L008/7XK/\nkJCQOs/jnGX34MGDjB8/vukK2kheCxZKKR9gOnAh0AO4zjEY2Hyote6tte4HPAM8bzu2BzAR6AmM\nA16xnc97asy1EEKcSsrLy5u7CDVcdtllTJtW6/7YI87Bom3btsyZM6epitZo3qxZDAZ2aq3TtNal\nwCzgcscdtNaO6SuDAW37/nJglta6RGu9G9hpO5/3VAUL6eQWoiV677336NOnD3379mXSpElMnjyZ\nqVOnMnr0aB5++GGysrK44oor6NOnD0OHDmX9+vUA/PTTT/Tr149+/frRv39/8vLyOHToECNHjqRf\nv3706tWLn3/+udb7DRkyhE2bNlU9HzVqFKtXr2blypUMHz6c/v37M3z4cLZt21br2HfeeYd77rkH\ngN27dzNs2DAGDRrEX//616p98vPzGTNmDAMGDKB37958+eWXQM2U7A8++CB79uyhV69egFkP/ZZb\nbqF3797079+fH3/8ser9rrrqKsaNG0dycjIPPfRQE/3Uq3lzBnc8sN/heTowxHknpdTdwFTADzjX\n4VjHJC/ptm3Ox04BpgAkJiaeWGllYp4Q9ft2Ghze0LTnbN0bLnTdZGO3adMm/vGPf/Drr78SHR1N\nVlYWU6dOZfv27SxcuBAfHx/uvfde+vfvzxdffMGiRYu46aabWLduHc899xzTp09nxIgR5OfnExAQ\nwIwZMxg7diyPPPIIFRUVFBYW1nrPiRMnMnv2bJ588kkOHTrEwYMHGThwILm5uSxZsgRfX18WLlzI\nX/7yFz799FO3Zb///vu58847uemmm5g+fXrV9oCAAD7//HPCwsI4duwYQ4cO5bLLLquRkh2okSHX\nfvyGDRvYunUrF1xwAdu3bwdg3bp1rF27Fn9/f7p27cq9995Lu3btPP411MebNQtXq4PrWhu0nq61\n7gQ8DDzawGNnaK1TtNYpMTExJ1RYgqLBYpVmKCFaoEWLFjF+/Hiio6MBaNWqFQATJkzAx8e0UP/y\nyy9MmjQJgHPPPZfMzExycnIYMWIEU6dO5aWXXiI7OxtfX18GDRrE22+/zRNPPMGGDRsIDQ2t9Z7X\nXHMNn3zyCQCzZ89mwoQJgEleOGHCBHr16sUDDzxQo/bhyq+//sp1110HUFU+AK01f/nLX+jTpw/n\nnXceBw4c4MiRI3Wey/EzduvWjfbt21cFizFjxhAeHk5AQAA9evRg7969dZ6robxZs0gHHMNaAlDX\nbfss4H+NPPbEWSwQ1kZqFkLUpZ4agLdorVGq9j2kY7pyrWvdT6KUYtq0aVx88cXMmzePoUOHsnDh\nQkaOHMmSJUv45ptvmDRpEg8++CChoaE8+eSTALzxxhukpKQQFRXF+vXr+fjjj3nttdcA+Otf/8ro\n0aP5/PPP2bNnD6NGjaq3/K7KPnPmTDIyMli9ejVWq5WkpCSX6dWdfw7uOKdhb+p+HG/WLFYByUqp\nDkopP0yH9VzHHZRSyQ5PLwbsq5bMBSYqpfyVUh2AZGClF8tqyMQ8IVqkMWPGMHv2bDIzMwHIysqq\ntc/IkSOZOXMmYEYmRUdHExYWxq5du+jduzcPP/wwKSkpbN26lb179xIbG8vtt9/Obbfdxpo1a7jy\nyitZt24d69atIyXFJBSdOHEizzzzDDk5OfTu3RswNYv4eNNs/c4779Rb9hEjRjBr1iyAqvLZzxMb\nG4vVauXHH3+sqgnUlZLd8TNu376dffv20bVr13rL0BS8Fiy01uXAPcB3wBZgttZ6k1LqKaXUZbbd\n7lFKbVJKrcP0W9xsO3YTMBvYDMwH7tZa1706e1OQiXlCtEg9e/bkkUce4ZxzzqFv375MnTq11j5P\nPPEEqamp9OnTh2nTpvHuu+8C8MILL9CrVy/69u1LYGAgF154IYsXL67q8P7000+5//77Xb7v+PHj\nmTVrFtdcc03Vtoceeog///nPjBgxgoqK+i9LL774ItOnT2fQoEHk5ORUbb/hhhtITU0lJSWFmTNn\n0q1bN6BmSvYHH3ywxrnuuusuKioq6N27N9deey3vvPNOjRqFN6m6qjWnkpSUFJ2amnpiJ1nwKKyY\nAY8eMfmihBBs2bKF7t27N3cxRBNw9btUSq3WWte7NoPM4HYUFg8VJVBYu4orhBBnMgkWjmRinhBC\nuCTBwpHMtRDCpdOlufpMdqK/QwkWjqRmIUQtAQEBZGZmSsA4hWmtyczMJCAgoNHnkDW4HYXEgfKR\nYCGEg4SEBNLT08nIkCSbp7KAgAASEhIafbwEC0cWHwgIh6Ls5i6JEC2G1WqlQ4cOzV0M0cykGcqZ\nNQjK655FKYQQZxoJFs6sAVBWO6mYEEKcySRYOPMNhDKpWQghhCMJFs6sgVBe1NylEEKIFkWChTNr\nAJRJsBBCCEcSLJz5BkqwEEIIJxIsnFkDZTSUEEI4kWDhzCo1CyGEcCbBwpmv9FkIIYQzCRbOZFKe\nEELUIsHCmX1SniRNE0KIKhIsnFkDQVdCRVlzl0QIIVoMCRbOfAPNo0zME0KIKhIsnFlt+d6lk1sI\nIapIsHBmDTKPEiyEEKKKBAtnvraahYyIEkKIKhIsnFltfRbOaco3z4Ulz5788gghRAsgwcJZVbBw\nqlls/BRWvn7yyyOEEC2ABAtn7kZDlRbIcqtCiDOWBAtn7kZDlRZARYl0fAshzkheDRZKqXFKqW1K\nqZ1KqWkuXp+qlNqslFqvlPpBKdXe4bUKpdQ629dcb5azhqrRUE7NUKX55lFqF0KIM5DXgoVSygeY\nDlwI9ACuU0r1cNptLZCite4DzAGecXitSGvdz/Z1mbfKWUvVaCgXNQuAYgkWQogzjzdrFoOBnVrr\nNK11KTALuNxxB631j1pr+7Cj5UCCF8vjmaoObqdgYR8dJTULIcQZyNeL544H9js8TweG1LH/bcC3\nDs8DlFKpQDnwtNb6i6YvogvugsWJ1Cyy98PSl6rzTfkFw+hHwC+o8eUUQoiTyJvBQrnY5jKVq1Lq\nRiAFOMdhc6LW+qBSqiOwSCm1QWu9y+m4KcAUgMTExKYpddVoKIc+C61PrM9i02ewcgYEx5gkhYWZ\n0G4w9Li8/mOFEKIF8GYzVDrQzuF5AnDQeSel1HnAI8BlWusS+3at9UHbYxqwGOjvfKzWeobWOkVr\nnRITE9M0pbZYwMev5qS88mJzkYfG1SzyjoA1GB7cCVO3go8/pK9qmvIKIcRJUGfNQikVAFwCnA20\nBYqAjcA3WutN9Zx7FZCslOoAHAAmAtc7nb8/8BowTmt91GF7JFCotS5RSkUDI6jZ+e1d1sCao6Hs\nTVDQuJpF/mEIjTPf+/pBm76QnnpiZRRCiJPIbbBQSj0BXIq5q18BHAUCgC7A07ZA8ket9XpXx2ut\ny5VS9wDfAT7AW1rrTUqpp4BUrfVc4FkgBPhEKQWwzzbyqTvwmlKqElP7eVprvbkJPq9nfANrjoZy\nDBaNrVmEtK5+njAIUt80fRg+1saXUwghTpK6ahartNZPuHnteaVULFBnR4HWeh4wz2nbYw7fn+fm\nuKVA77rO7VVWp3W4m6Jm0drh4ySkwPLpcGQjtK3VuiaEEC2O2z4LrfU3AEqpXm5eP6q1Pj3bUqxB\n7oNFo2sWcdXPEwaZR2mKEkKcIjzp4H5VKbVSKXWXUirC6yVqCXwDao6Gso+EslgbXrMoLYDSvJrB\nIjzBNEtJJ7cQ4hRRb7DQWp8F3IAZ2ZSqlPpQKXW+10vWnKyBrmsWYW0aXrPIO2weQx36LJQyTVES\nLIQQpwiPhs5qrXcAjwIPY+ZCvKSU2qqUusqbhWs2zsHCPow2LKHhNYv8I+bRsWYBpikqKw0KMhtf\nTiGEOEnqDRZKqT5Kqf8AW4BzgUu11t1t3//Hy+VrHu6aocLjm6ZmAdX9Fgek30II0fJ5UrP4L7AG\n6Ku1vltrvQaqJs096s3CNRtrUM1JeVXNUPEmiDhnpK1Lvm36SIhTsGjbD5SPNEUJIU4JnqT7uAiT\nAbYCQCllAQK01oVa6/e9WrrmYg1wPSkvLN48FmeDtXXt41zJP2w6xoNa1dzuFwxxPSVYCCFOCZ7U\nLBYCgQ7Pg2zbTl+1JuXlm6Yp+wW/If0W9mGzykWqrIRBkL4aKitOrLxCCOFlngSLAK11vv2J7fvT\nO11qrdFQhaYmEGgbOezYb5F/FPavqv4qzKp5LsdUH84SBplhtce2N235hRCiiXnSDFWglBpg76tQ\nSg3E5Ig6fVkDoaLU3PFbfEwzlF8wBESa1x1rFu9cXPNinzgcbnXItJ53BCKTXL9PQop5TE+F2O5N\n+hGEEKIpeRIs/oDJ3WTPGNsGuNZ7RWoBqlbLKzZBojQf/EJq1yxKbLWC/pOgxxWw9n3YsQAqK032\nWjA1i0Q3y3i06mSy0R7Z6N3PI4QQJ6jeYKG1XqWU6gZ0xaxRsVVrXeb1kjWnqnW4i2zBwl6zsAUL\ne83i6Bbz2O1iSD4P8g7B5i/g+G6I6gTlpWbtCueRUHYWC8T1gMMSLIQQLZun61l0xayj3R+zlvZN\n3itSC2C11Szs/RalBSaABISb5/aaxRFblvZY29LicbbHo7YEuQX2YbOx7t8rrhcc2WAWWBJCiBbK\nk0l5jwMv275GY9aVuMzL5WpezqvllRWaZigfX/ALdahZbDbbw21rPMV0AxQcsQWLPNvsbecJeY5a\n94LiHMg90OQfQwghmoonNYvxwBjgsNb6FqAv4O/VUjW3qnW4bRPzSvNNMxSYfouqmsVm0zFt75/w\nCzad2UdtNY582+xt51QfjuJsSX2lKUoI0YJ5EiyKtNaVQLlSKgyzCFJH7xarmVU1Q9lqFvY+CzD9\nFkXZptno6ObqJii7uJ4ONQs3qT6c9wfTFCWEEC2UJ8Ei1Zaa/HVgNSb1x0qvlqq5VTVDOfRZONcs\n8o9AUVb1xd4utgdk7TL9HflHAQXBdfRZ+Iea2ojULIQQLVh9a3Ar4F9a62zMuhbzgTB3S6meNqqa\noYrMMNiyQoeaRbjJFuvcuW0X1wN0JWRsM81QwdGmr6Mucb2qzyeEEC1QnTULrbUGvnB4vue0DxRQ\nM1jY+y2cm6HsI55q1Sxsz49uqb32tjtxvUxtpLSw/n2FEKIZeNIMtVwpNcjrJWlJHCfl2ZMIOjdD\nHdlsAoFzgsBWHcHH33Ry15Xqw1HrXqY2Yp+3IYQ4JZSUnzl53TwJFqOBZUqpXUqp9UqpDUqp07t2\nUTUpr7B6LQu/EPMYEGG2H1rnOkWHjy/EdDHBpCE1C5BObiFOIe8t20PfJxewdNex5i7KSeFJuo8L\nvV6KlsZxNJSrmgWYZqhO57o+PrYnpP1oZm97UrOIaG+CkfRbCHFK+GHLEZ6YuwkN/PmzDcy/fySB\nfj7NXSyv8qRmod18nb4cR0PZg4W9tmFP+QG1O7ft4nqY0VKV5Z7VLCwW0/chI6KEaPE2Hsjh3o/W\n0rNtOG/enMLezEJe+KH+zNH7Mgv517wtHMw+NfOwelKz+AYTHBQQAHQAtgE96zrolOZjNavYlRVB\nmb1mYWuGCnQIFnFugkWsw4/Gk5oFmKao9bOh6DgERja8zEIIrzucU8xt764iItDKmzenEBsWwMRB\n7Xjj591c2qctveLDXR73+dp0/vrFJvJLyvnqt4O8d9sQOseGnOTSn5h6axZa695a6z62x2RgMPCL\n94vWjJSyrWlRuxmqzBoGQCUWW3oPFxyDSF2ztx31nWg61D+eBOUljS25EMJLtNY88vkGcovKeXPy\nIGLDTHP1ny/qTlSwH/d9tJbDOTWXXM4rLuMPs9bywMe/0b1NKG/enEJphWbCq0tZt78Bi6gBlZUa\n3Yw55DypWdSgtV5zRoyO8g2o2QxlCxa5KpgoINM/nhhroOtjQ9uY5qribM+DRbvBcMUr8Nnt8OXd\ncNXrrlfXE0J4ZPXe4+QWldEmIoA2YYGEBfqiTuB/asHmI/yw9SiPXNSd7m3CqraHB1qZfsMAbnl7\nFRNeW8rM24aSGBXEuv3Z3PvRGg5mFzP1/C7cPbozPhbFp3eGMOnNlVz/+nKmXdiNa1LaEWCtu79j\n+5E8bnpzJVmFpcSE+BMb5k90iD8xoeaxU0wwl/eLb/Rn80S9wUIpNdXhqQUYAGR4rUQthTXINEOV\n1myGyq40wWKfTxIx7o5VyvRn7Ftad6oPZ32ugex9sOhvEBRtUp8D+AVB2wESPITwQGWl5v++38b0\nH3fV2D62Zxz/vX4AVh9Pk21XKygp58m5m+jWOpTJI5JqvT4oqRUzfzeEm99eyfhXlzJ+YAIzlqQR\nFxbA7N8PZWD76iH27aOCmXPnMO77aC2PfbmJ6T/u5M5zOjFxcKLLoLE/q5BJb65Aa5g8PIljeSUc\nzSthf1Yha/YeJ6uwlIGJkc0fLIBQh+/LMX0Yn3pycqXUOOBFwAd4Q2v9tNPrU4Hf2c6bAdyqtd5r\ne+1m4FHbrn/XWr/ryXs2GWuALVjYh86aDu6syiBidSAbVGcG1nV8Qgpk762e4Oeps/9oAsaK/5kv\nu2H3wNh/NOxcQpyG9hwrYFdGPud0icHX6cJfUFLO1Nnr+G7TESYOaseElAQO55TwW3o2M5ak8efP\nNvDs+D4NrmG89MMODuYU89J1/d0Gm77tIpj9+2Hc+MYKXlm8iwt7tebpq/oQHmSttW9saAAf3T6U\nZWmZvLBwB098tZnpi3fx+5EduX5IIkF+5tKckVfCpDdXUFRawew7htGtdVitc5VXVFJQ6v35Hp4s\nfvRkY06slPIBpgPnA+nAKqXUXK31Zofd1gIpWutCpdSdmPTn1yqlWgGPAymYzvXVtmOPN6YsjWIN\ndJiUp6pGSB0v1lxQ8gx+AXFMruv4UX+GYXc3/H2VgktegAE3Va+nsXEOLPuvGWI7ZErDzynEKa64\nrIKvfjvIJ6nprNxj1rnvEhfC45f2ZETnaApLy/l2w2FmLEljx9E8Hr+0B5OHJ1UFhYv7tCHIz4cX\nFu4gJtSfP57fpVagcWfb4Tze/GU316a0IyWpVZ37dokL5Yu7R7DlUC7ndoutMygppRjeKZrhnaJZ\ntiuTlxft4O/fbOGVxbsY0TkaHwXrD+RwJLeED343xGWgAPD1sRAe2PDaUkN50gz1PTDBlh8KpVQk\nMEtrPbaeQwcDO7XWabbjZgGXA1XBQmv9o8P+y4Ebbd+PBb7XWmc5lGEc8JEnH6pJ+AbaJuXZ8kLZ\n0pBnF5VxiCj88ivRWrv/Y/ALqqqNNJjFUr0+N0D74ZCfAfMfhvD46uYpIc4AR/OKufWdVWw8kEvH\n6GAeGteV+IhAnluwjRveWEFK+0i2HMqloLSC9lFBvH3LYM7pUruR+P4xyRzJLeF/i3fx6k+7aBXk\nR0yof42vy/vG06Nt9UW5oKSc+2etJTTAl2kXuhnQ4qRtRCBtIxrWojCsUxTDOkWxem8W/1u8iw3p\n2WjAz8fCa5MGMrB984+Q9KQZKsYeKAC01seVUnWkUa0SD+x3eJ4OuFmMGoDbgG/rONa7DXLOrAEm\nUDiuZQHkFJoVZUsrKskpKiMiyM/7ZbH4wNVvwLuXwJzb4LYF0KZP/cdVVsKSZ6DLWGjb3/1+e5fC\nqjdNyhEwnfLnPgr+p9bQvlNRZaXmfz/tYkz3WLd3jmeSmSv28vIPO7ntrA5MHpHE/qxCbn57Jcfy\nSnn1xgGM7dm66gZtbM/WvL4kjTlr0rm4TxvGD2zHoKRItzdwSin+dnlPBneIZM+xQo7mlZCRV0xG\nfilpGQUczSvmg2V7efuWwQzu0IrKSs2fPvmN7UfyeOeWwUQGe/9/fWD7Vrxxc921l+biSbCoUEol\naq33ASil2uPZpDxXvzGXxymlbsQ0OZ3TkGOVUlOAKQCJiYkeFKkBrEFmBrZjenIgu6i06vuMvJKT\nEyzA1FKu+xhe6g+pb8KlL9Z/zLKXYfG/TB/IFa+43ufgOvhgPPj6Q1CU2ZZlW0f82pn1Z8wVJ2TB\n5sM8+9025qxOZ959Z5/2s4Dd0Vrz8qKdPP/9dtqGB/CPeVv4OHU/mfklWJRi1pSh9G0XUeOYAKsP\n945J5t4xyR6/j6+PhSv7J7h87UhuMde9vpyb31rJmzenkLr3ON9uPMyjF3dnpIuaypnGk4auR4Bf\nlFLvK6XeB5YAf/bguHSgncPzBOCg805KqfNs73GZ1rqkIcdqrWdorVO01ikxMU38y/QNqB4N5RAs\njttqFgBH89zPh8grLmNfZhNnkQ2JgU6jYcf39a/Znb4afnjK9v0q1/tk74MPrzHJEO9cCvemmq+L\n/w+2z4dvH5K1wb2oslLzwsIdRIf4sftYAf9ZWP8s4NPF1sO5zP3tIEt3HmP7kTyemLuJ57/fzlUD\n4vnpodG8cVMKJeUVhAda+fTO4bUChTfEhQXw8ZRhtGsVyOR3VlWV57azOnj9vU8FnnRwz1dKDQCG\nYu74H9Bae5I5axWQrJTqABwAJgLXO+6glOoPvAaM01ofdXjpO+Cftv4RgAvwLEA1napJeflgrdkM\n5WtRlFdqMuoIFv/9cSezVu4n9dHzGjVUz63kC2DLXDiyEVr3dr1PcS58equZ79HzClj6cu2Z4UXH\nYeYE8xlv+hLC2lS/lnKrCSS//Aci2sFZDzRd+UWVBZuPsPVwHv+5ti+r9hznjZ/TuLBXa/on1t0+\nvW5/NjlFZbQJD6BNeAChAbVH25xM+7MKuWvmGq4eEM+kYUn4WEzDgNaa9ek5FJSUEx3qT1SwHyt3\nZ/H20j2s3J1V6zy3n92BP1/YHYtFcV6POEZ3i6VS66b9/6lHTKg/s6YM45a3V+Lna+GfV/Y+obkZ\npxNPOrivBBZprb+2PY9QSl2htf6iruO01uVKqXswF34f4C2t9Sal1FNAqtZ6LvAsEAJ8YvuF7NNa\nX6a1zlJK/Q0TcACesnd2nzTWQDMpr6ywRj6o7KJSOkQHs+Nofp3BIv14ETlFZWw8kFPvP3+DJF9g\nHncscB8sfngKsvfDLd+aEV1LX4YDq6HzedX7zP8LZO6CSZ+5zp577mPmHAufgA4jIb7OgcKigSor\nNS/+sIOO0cFc2qct53WPY/HWozw4Zz1f33uW20laezMLGP+/pZRXVtf4bh7Wnicv73Wyil7LRyv3\nseFADhsO5PD5uoP8/fJepB3L542fd7PhQE6t/RMiA/nLRd0Y2SWGrIJSjuWXEhrgy6guMTUuzD4W\nhY/LFmnvahXsxxd3j0BrsFgkUNh50iD9uNb6c/sTrXW2UupxHBZFckdrPQ+Y57TtMYfvz6t1UPVr\nbwFveVC+JlVWUYlFKXwcm6HCqvvWswvLaNcqiP3HCzmaV+z2PJn5JpAsT8tq2mARGgdt+sH2BWZO\nhrPCLFj7AfS7HhKHQEkeoCA9tTpYVJTB1m9MipEOI12/j8UClzwPm7+ALV9JsGhiCzYfYcuhXJ6/\npi++PhZCfSz886reTH57FY98vpF/XdUbP9/ad9TPf78dXx/FGzenkFtczvyNh3hv+V4mDWtP59hQ\nF+/kXRWVms/WHGB01xiu6B/PU19t5tL/mmxAHWOC+fsVvegYE8yx/FKO5ZWQ2CqI0d1iq2ofLZVS\nSubAOvEe/6RGAAAgAElEQVQkWLiqA562vZ4XvfgzV/SP525roAkWJfnVSQQxwaJr61BiQv3rrFlk\n5puO8BW7M7lzVKemLWSXsbDkWRMYnBdfWv22qRENvcs89w81s8n3Oyybvn8FlOSY89QlIBwSh5nA\ndN4TTfkJzmj2WkWH6GAu69u2avuorrHcNyaZl37YQdqxfP53w0BahwdUvb75oGnnv+OcTozqagYk\nntU5msXbMnjph528dF0dI9685NedxzicW8xjl/bgot5tGJkcw4cr99G9TSijusTKnflpxJPGwFSl\n1PNKqU5KqY5Kqf8Aq71dsOayL6uQ9enZtpnX2rTtO46GKiwlItCP2NAAMvLrCBYFJlis2p1FeUVl\n0xYy+QIzzHXnDzW3V5TByteh46iayQwTUuBAqhlKC7D9O7BYzX716TLWrPqXvb/+fYVHlqdlsuVQ\nLneP7lxrYtjU87sw/foBbDucxyUv/8KyXZlVrz23YBuh/r7cMbL65qNVsB83DUviq/UH2Xk0/6R9\nBrs5q9MJD7QyprsJXpHBftw9ujPndouTQHGa8SRY3AuUAh8DnwDFQCOmJrd8WmtKyis5kF1UvaZF\nWUHV5LrScjOtPiLISkyIP0dzXQeLikrN8cJSOsUEU1BawcaDuU1b0LYDTO6oHQtqbt/8JeQdqq5V\n2CUMguIcyNxpnu9YYCb6+XvQbOHYRyKaxOdrDxDi78slfdq4fP3iPm348u4RhAX6csMby5n+405W\n7s5i0daj3Dmqc630Ebef3YEAXx/+u2jHySh+lZyiMr7bdJjL+7XF3/fMHPJ7JvFkNFQBMO0klKXZ\nlVWYTsMDx4uqV8uDqmaonCIzbDYiyEpMqD/Ld2fWOgfA8cJStIaLe7fhpUU7WZGWSb9GDv3LLynn\nue+2cffozsSE+puNFgskn29qCJUVZtKe1rBsOrTqBJ3Pr3mSdoPNY/oqM58iY6tJJ+KJ6C4mzciO\n72HQbY36DKJacVkF8zceZmzP1nVmGk2OC2XuPWeZXEbfbSPAaiE21J/Jw5Nq7RsV4s9Nw9rz+s9p\n3DsmmU4xJzaZctvhPOb+dgB/Xx8ig6yEB/kRHmglPNBKhO0xLNDKN+sPUVJeyfiBructiNOLJ6Oh\nYoCHMIsdVV1BtdZu1hQ9ddkXXz9eWEaJ8sff/oKtGSrHNiEvPNBKbKg/2YVllJRX1LqrsvdXdG0d\nRqeYYJanZfL7cxrXb/H+sr28s3QPwf4+PDjWId1A8gXw20em4zpxiAkEB9fARc9VpSapEpUM/uFm\nnzLb3I/k+rK12ChlmqLWfmCG2ToGUdFgi7YeJa+knCv6t6133xB/X16a2I/BSZH8/ZstPDi2q9tJ\ne7eP7Mh7y/Zy30drSWkfSUSQH51jQ7i4dxuPm4PSMvJ5YeEOvlp/EAVU1jPFxsei6BIXQm83C/6I\n04snHdUzMU1QlwB3ADdzmqYoLymv7lvIKrFQ1UhgCxbZhfaahR9FtiyPx/JLiXfKA2MfCRUV4sfQ\njlF8ue4g5RWVHicusystr+SdpbsBmJ2azh/O61I95rzTuWY1v48mmrQcxTkmIPS9rvaJLBZIGGgC\nS+4BaNURojt7XpDksbByBuz5BZLPg01fmI70q16HEA8yv5QVw+e/N6sBnvOg5+97Gvpi7QFiQv0Z\n3inao/2VUkwalsTEwYl1zjeIDvHnr5f04NWfdvH52gPkFpcDZljrsxP61vobdXQgu4gXF25nzup0\n/H19uOOcTkw5uyPB/r7kFJWRU1RKTlEZ2YXmy2wrI7e4rEb6DeFF5aXwwVUmk7Vdvxth1MMnrQie\nBIsorfWbSqn7tdY/AT8ppX7ydsGag2OwyCh2DBa2tSxswSIyyFrVaZ2RV1LrH/GYrXM72hYsZq7Y\nx+ZDufRJaFhT1Fe/HeRIbgmThrbn/eV7+WHLUcb1sq2PERgB4542tQm7LmPd53NKGGRGUGXugIGT\nG1QOkkaYPpwd30FkEnxxl+nL+fBamPx1jQEAtVRWmkCx+QvzFRB+xmbOzSksY/G2DG4c2r7BQ0c9\nmZh2/ZBErh9i0t5UVGpmp+7nb19vZtx/lvDA+V2ICLKSV1xOpdaEBlgJDfBl5e4s3l++FzRMHt6B\nu0Z3Ijqkqk5dlWBPNLNNn8Oen6HbJaavMScdFv/T/D8NveOkFMGTYGHPb3FIKXUxJu3GadlIWVJW\nnRP+SJHDP6fVdHAfLzRBICLQD2WbLHQ0t/ZcC3vNolWwP0M6ms7I5WmZDQoWWmte/zmN5NgQHru0\nB99vPsKsVfuqgwU07KKbMMiMoCovru609pQ1EDqeY1KA7Ftu+j0u+BvM+xN8+ju49gPTb+LKwsdM\nkDjvCdi/6ozOnDtv4yFKKyo9aoI6UT4WxXWDExnRKZo/ffIbT3292eV+FgVXD0jgD+d3qbP2IZqR\n1rB8uuk/vOZ901JQWWGWYJ4/DcIToPslXi+GJ8Hi70qpcOCPwMtAGHBa5n9wrFkcdkzrVNVnYeJm\neJC1asKUq+GzmfmlWBREBFqxWBQdY4JZnpbFlJGe91v8ujOTrYfzeObqPlh9LFwzqB0vL9pB+vFC\nEiIbkfrcPqnOGgxJZzX8+OQLTLBgH1w3C7peaILPvD/BV/ebtCLO0leb2eODbocRfzDzVuyZcy97\nqXqOSERSw5rFmkBOURlT3kulvFITEWglIsiPiCBbJ25QdSdueKCVzrEhhDmn1MjeZxIv1lWrcvLF\n2gN0jAk+qW38iVFBfDRlKDuO5uHv60NYgFlaNK+4jLziciKCrI37exINp7XJmuDqb/3YjppNTPED\nq9Pz7FsOh36Di5+v7o90zET96W1w89fQzrurXXsyGupr27c5wGivlqaZOQaLA45D1h2aoSwKQv19\nCfLzQSlcTszLLCihVbB/Vcfi0I5RzF13kI9X7SMlqRUdo4NrtfPmFJWxN7OATjEhBPv7MuPnNKJD\n/Lncdhd6TUoCLy/awexV+5l6QdeGf7igVhDX2/yh+jaiWaHLWPjWFwb9zgQKgMG3mz/wpS/D2vdd\nH9f1Irjw36aj3J45983zzFrjdoGR8OAu97UTL5i77gArdmcxuEMrDucWs/VwHtmFpS5XHIsO8eON\nmwdVj2g7uhVmjILoZLhlXo0hyO8v38sHy/YS7O9DiK2pJ9TflwCrDyt2ZzH1/C4nvY3fx6JqpT9v\ndRLSbQsnP/7TLBkw/m3odVX19qLj5u+p1OGiE9kBfr8EAsJMrSIgonZ/pOP/05d3w13LvPo/1KCZ\n2EqpNVrrAd4qTHOzN0P5WBT78xyGgtg7uItKCbfVFiwoWgX5ucw8m5lfSnRI9T/j5X3bMn/jYR7+\ndANg/lF7x4fTKz6MpKhgftqewYLNRyi1BavEVkHsyyrkTxd0qRpplRAZxDldYvg4dT/3jUlucGc5\nADfPBZ9GXiTCE+C+tRDm1AJ5/t+gz8TqUVaOLD4mNYnjH3BIDNzxCxzdYp7v+dnksjq8Adr2a1zZ\n6pBTVEbqnizGdI+rsX12ajo92oQx+/fDamwvLa+s6sDNKSrjWH4Jf/9mMxNnLOPFif0Z2yUc5txi\nAu6RTfDJZPMP6+PLhvQcnpi7iS5xoQT5mc7h9OOFFJSUk19cTliAL1f2P7nLsogWYs17JlAoi1n1\nsueVVOUTWf2uCRQT3oWwtqbW+tnt8M0f4dxHTGqeEfe7XkwtJAZu+NScy8s3Ww1N23FaD3sorXC4\nWOflVb/gMBrKcf0Kdyk/MgtKiXIIFkM6RrH60fPYlVHA6r1ZpO45zsaDubz2U5ppBgmyMnFQOwZ3\naMXujAK2Hs4jsVUQNw5tX+O8EwclcscHq1m8LYPzesQ5v239nFODNFSEizVDlILWDUxi5x9aPfcj\nLN4Ei71LvRIs/m/BNt5btpcPbhvCWclmBNKWQ7lsOJDD45f2qLW/n6+lVqfuwPaR/O7dVO74YDUf\nxX/C0GOb+U/sPzkrroRBG56Ebx6g5ML/8MdP1hEd4ses24e6XHdZnKF2LISv/gCdxpj5UfOnmWHs\n7Qbbsi7MgKSzq5ty2w2GrDT48R+2myplmnLdOUlNuA0NFt94pRQtREmZCRYdo4NZs01TNdHCoc8i\nwuEi4DZY5JfU6sxWStE5NoTOsSFcO8hcdIvLKtibWUhSdJBHM2DHdI/F6qNYtTerccGiJQqPNyOs\n9v4Kwxxmnq9+xzRP9bi80afOLynnszUHAPj3/K0M7zQCi0XxSWo6fj4Wrujn5i5/2XQTvGyigU8j\nNVsKjtHr2AreV5fyWX53XtxXxNuJNzF6zXsc3rmJPx5XDEiMJHzu29XnikiEsf9EstJ5SfZ+WPqS\nSaMfdgIDByor4efnTCZnezOrK+mpptm1stzzc6ctNul3rnkXUGZBsuWvmKCw5SsznP2i52oec/Yf\nYfcSU/PudbX5P2lmDQoWWutHvVWQlsDeZ9ExJphftjq069uCxfHCUmKchhWmZRTUOk9mfs2ahTsB\nVh+6tvY8U6jVx0JSVLDL9zyltT8Lts0z/7AWC+QdgXkPmqavHpezZHsG3248RIDVhyA/H4L8fPH3\ntRDo50Og1XwF+PkQGeRH34Twqj6Bz9ekk19SXjX0+JsNhxjbszWfr03n/B5xrpfJzN4HCx6F0LZm\neLKND9ArFCo7TmDSFa9wvcXKcwu2ccvisTwfWUD37FX0DrYSU14Mx20HleTB1q9NBmB36eRF4xUd\nhw+uhmPbYO8y038U0MilaRc9ZdZvsVjhxk/N6D9nGdvN+ylLwwJTwiC44n/VfVsDbjY3JNn7Yfn/\nTP+Ec1JPiw9cNcMsQHbOyZtLUZdGZY9VSm3QWp92f/32GdwdY0IowVaDsPhWtfNnF5aR7JAG2l6z\n0FpXXaCKyyrIKymvMVa9KXWMMWtpnFbaD4d1H5h/+tjukPoWVJRCVhqlR7bx0Jx0sotKsVosFJZV\nUFHH1OKJg9rZFqyBd5ftpU9COE9c1pNVe7J4bsE2NGaG/vgUN6O/V74OKLh1vln4yYm9p8gHeHhc\nN7q1DuWhOT60Cr6B7/4wEhxHTeUdgf/rYtKySLBoWuUlZuhoVhqM+gv89G/45Ga4fjb4NLAJcNWb\nJlD0u9Gs+/LxJPP7d0zGmX8UZl5tzv27haY23FiDp5hgMfceSF8J4/7tur8hrK0Zlt5CuA0WSqmr\n3L0EtHbz2imtqmYRHQwoyi3++PoFVDUh5BSWER7o0AwV4k9pRaWtecoElCzbhDxvjTbpFBPCD1uO\nUlZReVJXEPOq9sPN455fzF1W6pvQpi8c+o2Niz/hcG5/3rt1cNU6yKXllRSVVVBcVkFhqXksKqvg\nu42HeW1JGmUVmiv7x7PzaD7Pju+Dj0Xx8Lhu3PLOKv71+Qp6hFYyso2uvXpgST6seRd6XOYyULhy\neb94+rWLwNfHUnt4rX3tkR0LYOSfmuInJcDUQL+82zTRXPUG9JlgVnqce6/pG7j8v3U3+5XkmWHc\nAPuWmeHfyWPNuvZ5h+CN88wqkpM+N7XLilL4+EYoOAaTvzmxQAHmb6v7pWb+kX8Y9L/hxM53ktRV\ns/gYk+rD1W3caZkgyD4aqkN0MEpBmSUAX9uw2bKKSvJKymv0WcSGmR9DRl5JVbCw54WK8lKw6BgT\nQnmlZl9W4QknjGsxIpNMs8/epWYCYEEGXP0met6DVG5bQO/4UZydXJ0ew8/Xgp+vpUbgBhiQGEmw\nvy/Pf7+d+RsPERFk5VLbehGjusbw57gV/C77JXzKNDwPoMxkwbP+YE7w20cmbYpz1t56tI+qY65F\nXWuPiMZZ9DfY8AmMecwECjCJMbP3mxFHSWdBPxdpbwB2/WhS5JQ7TKZt0w/GvwU+vuZCfsNsePsi\nmO4wb0FZYOKHEN9Eg0GH3mWCRf9JnmV/bgHqChbrgee01hudX1BKuV3h7lRmr1kE+/sSFxpASaUf\ngbbZ27lF9lQfDqOhbE1NGXklJMeZX/ixAnteKO80Q3WKMRemtIyC0ydYKGVSiuxeAse2Q2xP6DCS\nXREj6JPxLvedFefx3IT7xiRj9bHw7/lbuWNYUlVmV7Xje6bkvsw6ax/an3UdrYKtsHMRLHzcjMjq\ndbVpP44faNqYm0ryBaaJZOcP1Rc20Xipb8Evz5uUNWdNrfna6L+YYaZLXzarQDr/zeQfhc+mmCzK\n9uwHFqu5y3dMk9OmL9z2PeyrHuRAXC9IHNp0nyNxCNwwp2nP6WV1BYs/AO4WYrjSC2VpdvZg4e9r\nIT4ykKJjfkRUdW5Xpye3iw0zAcFxroW9ZhHtQQd3Y3S0BYhdGfmcz2kyIgpMU9SGTyD/CFz2XzQw\n41BnnlEVjPHfAiR7fKo7R3Xi7ORoutgCOAfXwieTUa1703/yvOoLQ/9J8P5V8MWdcGQjZO2Cq99s\n2pFLVWuPfCfB4kRt/87MPUi+AC76v9q/J6Vg6J2mL2D3kpqd1JWV8PkdUJILN30BcT3rfq+4HjX7\nLLwh+fz692lB3AYLrfXPdbyW6p3iNK/S8kp8LApfHwvxEYEUHLU6rGVRnZ7czj4W33H4bJaXaxbh\ngVaiQ/zZddp1co8wj0FR0HsCP247ymeZCfwjJATrjgWmHwHg+B4zfLHX1XVe1HvZU2pk74OZ15gm\noOtn17yD9PWHiR/Am2Ph1xcgtM0JDdV1qWrtkfnVa480pV0/mhFA3lojvaLMDGMudL12ywlp27/+\npX3tbAGf1r3NDGgfN5eu3hNg4ROmlugYLJb9F3b9ABf/X/2BQrhUVwf3o8ArWussN6+fCwQ5pAM5\n5Zm1KUyncXxkIKvKO9ExvhcWaqYntwv1N0M4HfNDZeaX4u9rIdjNugNNoVNMMGnHTq/hs5Wtktnl\nm8ycvKG8/tgPVGqIjwjFp8MYs/CS1qbW8c4lkLPfjE0fcX/dJ60og09uMe3TN38FoS7GZQRGwo1z\n4IPxZp5HQ0fSeCL5fNvaI6uattlh23yYdR34BpiO16ZqT7fTGr7+g1nLxFvGv2UCf12O77UF/Gi4\n/hP3mZXBrLcy6Db46RmThymqE2z71jQ3dr8UUmQBr8aqqxlqA/CVUqoYWINZwyIA0x7QD1gI/NPr\nJTyJSsorq4NFRCB/LruNUUPOpQ0OwcKhZqGUonV4APuzqlNdHMsvJTrE36v5fzrGhPDtxkNeO39z\nWL0/mwn5T3Jp37bcHRWERSlGdY3BknkItnxpRq18+7DpKO50Lnz/mJmHUdeF5sd/mLXHx78Nsd3c\n7xeRCPesbPoPZddpjFl7ZMeCpgsWB9aYtCNxvaAo26SL/91CiGxf/7GeWvKsCRQjH4TRjzTdecEE\n8PevNE1DoW2qR8Q5KzoOM8dDRYlJhx/qQdNrym3w8/Ow4jXoey3MudX0Q1z5mkyOPAFux15qrb/U\nWo/ALHi0CTO0PBf4ABistX5Aa31aLYJUUlZZNZM6PtKkaz5w3Ayxyy6q3WcB0K9dBKv2HEdrM2gs\ns6DEowl5J6JTTDDZhWVVw3Q9VVpeWVXOlubr3w7i72vhX1f15o8XdOWB87vQPzGyul33w2tNLqYJ\n78DEjyBxuLnQ7PnV9Ql3/Qi/vGBGyfRyNwr8JAmMMEFiexOtY358j/l5BEWbTtIb55iL6czxJpg2\nhXUfmmDb9zoTKJRq2i9roBldFNEeProOMrbVLkN5Ccy6wXzeiR9BjIcJNEPjoPd4E+g+vBaCo00T\nZAMyBIvaPMk6uwM4uSvBN5OS8gr8rSZ+Jthy+x/ILiIFyCksRSlqjaUf0sGshJd2zIxOck4i6A2d\nYqs7uVsFezYcs6yikium/0p8ZCAzJg1sUaubVVRq5m08zLndYgnxd/qTDIk1bdsH18IlL0AX21oc\nE2fCmxfA+1fUnCthV5Rt8v+P+7f3P4Anks83benPdTnxc5XkmzZ7+512aJy5mL5/BbzY11yIT1RB\nBnQ4By59yXt340GtTKB74zx47Zzas6/LS6A42ww6SBrRsHMPvdM0/flYYfI8z1Z0FHVq1Azu01WN\nZihbzSL9eBFaaw7lFFdlnHU0pKO5WK9Iy7IFi5LqUThe0inaBIu0jHwGJdUOFou2HmHjgVzuG1M9\ngmjO6nQ2H8pl86Fcvlp/iMv6en8BHk+t3J1FRl4JF/dp43qHC581qdB7j6/eFtTKTJpaNh3Ki2of\n4+MHQ+5wnamzOfSfBLmHTA3gRCmLOZ/jnXbSCHP3vPmLEz8/mJTYZ08FXy+nMo9MgpvmmomYrvIt\ndRhZf5+GK236mkDXth/ENEGAFhIsHJWUV1YtahTk50urYD++Xn+Iz9ceYOfRfPom1F60pmN0MNEh\n/qzYncl1g9txrMD7NYv4yED8fC3scpEjqrS8kkc/38jBnGL6J0ZwdnIMxWUVvLhwB/0TI6io1Dz1\n1WbOSY5p8syo2w7ncdfM1fzpgq5c2NvNhd+Fr9cfJNDqw7nd3Nz9tRvkemGXiHZw4dONLO1JFhwN\nFz3j3ffoNNp8nWriephRSk1t4M1Nf84zWJ35IpRSPkqp03JVPFfMaKjqUUydYoLZciiXiEArT1/V\nmw9+N6TWMUophnRsxYq0LPJLyiktr/R6n4WPRdEhKpi0jNrDZ79Yd4CDOcWE+vvy1FebKa+o5P1l\nezmcW8xDY7vxzyt7k1VQwr+/29qkZSouq+C+j9ayK6OAP3y8jjX7jtd/EFBeUcn8jYc5t3ssQX5y\n7yJES1VnsNBaVwCNHniulBqnlNqmlNqplJrm4vWRSqk1SqlypdR4p9cqlFLrbF9zG1uGhjAd3NU/\nkunXD+CnB0cx587hTBycSKhz7h+bobbV1tbuywYgKtj7C9x3ig2uVbOoqNS8ungXPduG8dw1fdlx\nNJ/XlqTxyuKdnJ0czbBOUfSKD+fWER34cMU+vtt0mJ93ZPDhin18s/7ERlc9/e1Wth3J4/lr+hIb\n5s+U91JrjBJzZ3laFpkFpVzqrglKCNEieHIr96tS6r+YXFFVVyet9Zq6DlJK+QDTgfOBdGCVUmqu\n1tpx5fh9wGTAVZa1Iq1106+GU4fSikpCA6p/JPbcT/UZ0jEKoGo4q7drFmASCn63yayuZ286m7/x\nMGnHCnjlhgFc0COOEZ2jePY7M8rkwbHV7dsPnN+FeRsO8fv3V9c4p79vSqPWyfhx21HeWbqHycOT\nuGpAAn0SwrnylaXc+s4qPr1reO0EezblFZX898cdhPj7MqqrdEAK0ZJ5EizsA6CfctimgXPrOW4w\nsFNrnQaglJqFqaVUBQut9R7ba5WuTnCyOQ6dbYjk2BBaBfsxf+NhAK+lJ3fUMSaYikrNvqwCOseG\norXmlcU76RgdzNierVFK8dglPbnopZ+5oEdcjcWYgv19effWwfyWnkNCZCBtwwO544PVPPzpeua3\nG1ljlThXKio124/ksS+rkP1Zhbz60y66xoUy7UIzl6FzbCiv3TiQSW+t5IXvd/CYixXpwCxItDwt\ni2fH96nK4SSEaJk8GTrb2B6zeGC/w/N0oHajv3sBSqlUoBx4WmvdRMM83HMcOtsQSikGJ7Vi/iYT\nLE5WzQJg51ETLH7ansGmg7k8Y0vJDdC1dSjz7jubxFa1RwQlx4VWJT8EeHFiPy55+RcemvMbb00e\n5HJobVZBKR+v2s8Hy/dyILt6BFJsqD8vXtevxgV/eOdoLunThtmp+3ng/ORaTXhzfzvI6z/v5qZh\n7ZmQ4lk6cCFE86k3WCilwoHHgZG2TT8BT2mtc+o71MW2hswIS9RaH1RKdQQW2RZc2uVUtinAFIDE\nRBfrQzeQ49DZhhrSsTpYeGstC0cdos0Eo9d/TmPGkl1sPJBL2/CAWkuFeroSX3JcKH+5qDuPz93E\nGz/vpn9iBNuP5LP9SB67MvJJyyioChDDOkbxxwu60Dk2hHaRQUQEWV0Gl1tGdODLdQeZszqdW0Z0\nqNq++WAuD835jUFJkTx6sZeTtQkhmoQnzVBvARuBa2zPJwFvA/VNi00HHG8ZE4CDnhZMa33Q9pim\nlFoM9Ad2Oe0zA5gBkJKScsJTk02waFxzyJAOpt8iNMC30edoiNAAK13iQthwIIc+8eFMHpHE+IEJ\nVf0XjXHTsPYs2nqUf8zbUrUtyM+HTjEhDEqKZGJMO8b2au3xPJJ+7SIYkBjBO0v3cPOwJCwWRV5x\nGXfOXE14oJXpNww4ofIKIU4eT4JFJ62146yYJ5VS6zw4bhWQrJTqABwAJgLXe1IopVQkUKi1LlFK\nRQMjAC8PUjeLHzW2ZtGtdSjhgVYim3juQl2+vvdsgCa74CqleOHafnyx7gDto4JIjg0lPiKw1kTE\nhrhlRAfu/Wgti7YeZUz3WKZ9toH040XMmjKU2NDTcg0tIU5LngSLIqXUWVrrXwCUUiMAF1Nma9Ja\nlyul7gG+w+SVektrvUkp9RSQqrWeq5QaBHwORAKXKqWe1Fr3BLoDr9k6vi2YPovNbt6qyThOymso\ni0VxUe/WFJRUNHGp3PPGXXlksF+NJqMTNa5Xa9qEB/D20t0czi3mm/WHeGhcV5czz4UQLZcnweIO\n4D1b3wXAccCjqZFa63nAPKdtjzl8vwrTPOV83FLgpK5wX1GpKa/UJ9SE9K+r+jRhiU4PVh8Lk4a1\n55n521i1+zjndInhjpGdmrtYQogGqm8GtwXoqrXuC/QB+mit+2ut15+U0p1EpfZV8hoxGkrU7bpB\niQRYLUQGW3n+mr4n1KwlhGgeddYstNaVtqak2Vprd0usnhZKyk3zUWP7LIR7kcF+zPzdEKJD/L22\ngqAQwrs8aYb6Xin1J2rP4G6ixPktQ/X62zI5zBsGtpc+CiFOZZ4Ei1ttj3c7bNNAx6YvTvMpKbMH\nC6lZCCGEszqDha3P4kattZvlyE4fVc1Q0mchhBC11Jd1thJ47iSVpVlJM5QQQrjnyW30AqXU1aol\nrcPpBfaahcwoFkKI2jzps5gKBAMVSqkiTM4nrbUOq/uwU4v0WQghhHueZJ317oLSLURJhQQLIYRw\np7CMpU8AAA7mSURBVN4rozJuVEr91fa8nVJqsPeLdnJV1yykz0IIIZx5chv9CjCM6iSA+ZgV8E4r\nMhpKCCHc86TPYojWeoBSai2A1vq4Usr7CzacZNWjoSRYCCGEM0+ujGW29bQ1gFIqBmgRy6A2JRk6\nK4QQ7nkSLF7CpBGPVUr9A/gF+KdXS9UMSsqkGUoIIdzxZDTUTKXUamAMZtjsFVrrLfUcdsqx1yz8\nfCRYCCGEM0/6LNBabwW2erkszUr6LIQQwj25MtqUlFfg52vhNJ+oLoQQjSLBwqa0vFJqFUII4YZc\nHW1KyitlJJQQQrghwcKmpExqFkII4Y5cHW1Kyitk2KwQQrghV0cbaYYSQgj3JFjYlJRXyloWQgjh\nhlwdbUrKKqTPQggh3JCro02JDJ0VQgi35OpoUyp9FkII4ZYECxsZDSWEEO559eqolBqnlNqmlNqp\nlJrm4vWRSqk1SqlypdR4p9duVkrtsH3d7M1ygjRDCSFEXbx2dbStgTEduBDoAVynlOrhtNs+YDLw\nodOxrYDHgSHAYOBxpVSkt8oKMnRWCCHq4s1b6cHATq11mta6FJgFXO64g9Z6j9Z6PbUXUxoLfK+1\nztJaHwe+B8Z5sawyGkoIIergzatjPLDf4Xm6bZu3j20UaYYSQgj3vHl1dJXrWzflsUqpKUqpVKVU\nakZGRoMKV+PEWkuwEEKIOnjz6pgOtHN4ngAcbMpjtdYztNYpWuuUmJiYRhe0tMK28JFV+iyEEMIV\nbwaLVUCyUqqDUsoPmAjM9fDY74ALlFKRto7tC2zbvKJUVskTQog6ee3qqLUuB+7BXOS3ALO11puU\nUk8ppS4DUEoNUkqlAxOA15RSm2zHZgF/wwScVcBTtm1eIUuqCiFE3Txag7uxtNbzgHlO2x5z+H4V\nponJ1bFvAW95s3x21cFCmqGEEMIVuZXGDJsFZAa3EEK4IVdHpBlKCCHqI1dHpBlKCCHqI8GC6mYo\nWfxICCFck6sj0gwlhBD1kasj0gwlhBD1kWCBw6Q8GQ0lhBAuydURs/ARSDOUEEK4I1dHpBlKCCHq\nI8ECh0l5UrMQQgiX5OqIQ81C+iyEEMIluTpSHSz8fOTHIYQQrsjVEdPB7WNR+EqwEEIIl+TqCJSU\nySp5QghRF7lCYlbKk2AhhBDuyRUSe81Chs0KIYQ7EiwwfRYyEkoIIdyTKyRmNJQ0QwkhhHtyhcQe\nLKQZSggh3JFggWmGkrUshBDCPblCIkNnhRCiPnKFRPoshBCiPnKFxKxnIX0WQgjhngQLZOisEELU\nR66QSDOUEELUR66QyNBZIYSojwQLzOJHUrMQQgj3vHqFVEqNU0ptU0rtVEpNc/G6v1LqY9vrK5RS\nSbbtSUqpIqXUOtvXq94sZ0l5pcyzEEKIOvh668RKKR9gOnA+kA6sUkrN1VpvdtjtNuC41rqzUmoi\n8G/gWttru7TW/bxVPrvyikrKK7U0QwkhRB28eTs9GNiptU7TWpcCs4DLnfa5HHjX9v0cYIxSSnmx\nTLWUVsiSqkIIUR9vXiHjgf0Oz9Nt21zuo7UuB3KAKNtrHZRSa5VSPymlzvZWIUvKbMFCmqGEEMIt\nrzVDAa5qCNrDfQ4BiVrrTKXUQOALpVRPrXVujYOVmgJMAUhMTGxUIS0WxcV92tAxJqRRxwshxJnA\nm7fT6UA7h+cJwEF3+yilfIFwIEtrXaK1zgTQWq8GdgFdnN9Aaz1Da52itU6JiYlpVCHDA61Mv34A\n53Rp3PFCCPH/7Z17rB1VFYe/n1Qoj0KhBVIeobRUHgq0cEXKu1gJEnkYS0JtFKRKiCAvwVBRFOID\nJOHhI4CpSGKqEFoepUahTyUIbW9rb3uhRaoUaai2Bqi8xJYu/9jrlPFk7pkL3HPOnWF9yeTMXnvP\nzPqdO/esmT2z1/4g0MxgsRgYJWl/SdsC5wAz69rMBM719QnAPDMzSbv7A3IkjQBGAX9roq9BEARB\nA5rWDWVmmyVdDDwCbAPcZWZPSboe6DSzmcAvgF9JWg28RAooACcA10vaDLwNXGhmLzXL1yAIgqAx\nMqt/jFBOOjo6rLOzs91uBEEQlApJS8yso6hdvAIUBEEQFBLBIgiCICgkgkUQBEFQSASLIAiCoJAI\nFkEQBEEhlXkbStIG4Pk681DgX21wp5lUTVPV9ED1NFVND1RP0/vRs5+ZFY5KrkywyENSZ29eCSsT\nVdNUNT1QPU1V0wPV09QKPdENFQRBEBQSwSIIgiAopOrB4uftdqAJVE1T1fRA9TRVTQ9UT1PT9VT6\nmUUQBEHQN1T9ziIIgiDoAyobLCSdKukZSaslXd1uf3qDpLskrZfUnbHtJmm2pGf9c1e3S9KPXd9y\nSUe0z/N8JO0rab6klZKeknSp28usaaCkRZK6XNN1bt9f0kLXdK+n5UfSdl5e7fXD2+l/T0jaxmem\nnOXlsutZI2mFpGWSOt1W5vNusKTpklb5/9PYVuupZLDwuTB+BnwaOASYKOmQ9nrVK+4GTq2zXQ3M\nNbNRwFwvQ9I2ypcLgNtb5OO7YTPwdTM7GDgauMj/DmXW9BZwspkdDowGTpV0NHAjcItrehmY7O0n\nAy+b2QHALd6uP3IpsDJTLrsegHFmNjrzSmmZz7vbgN+b2UHA4aS/VWv1mFnlFmAs8EimPAWY0m6/\neun7cKA7U34GGObrw4BnfP1OYGJeu/66AA8Bn6qKJmAHYCnwCdKAqAFu33r+keZzGevrA7yd2u17\nnY59SD82JwOzSNMdl1aP+7YGGFpnK+V5B+wMPFf/PbdaTyXvLIC9gRcy5bVuKyN7mtk6AP/cw+2l\n0ujdFWOAhZRck3fZLAPWA7NJ0/6+YmabvUnW762avH4jMKS1HhdyK/ANYIuXh1BuPQAGPCppiaQL\n3FbW824EsAH4pXcVTpW0Iy3WU9VgoRxb1V77Ko1GSTsBM4DLzOzfjZrm2PqdJjN728xGk67IjwIO\nzmvmn/1ak6TPAOstzXW/1ZzTtBR6MhxrZkeQumQuknRCg7b9XdMA4AjgdjMbA7zOO11OeTRFT1WD\nxVpg30x5H+DFNvnyfvmnpGEA/rne7aXQKOnDpEAxzczud3OpNdUws1eABaTnMYMl1aYpzvq9VZPX\n70KaQri/cCxwhqQ1wD2krqhbKa8eAMzsRf9cDzxACuplPe/WAmvNbKGXp5OCR0v1VDVYLAZG+Rsd\n25Lm9p7ZZp/eKzOBc339XFK/f83+RX/z4WhgY+2WtL8gSaR51lea2c2ZqjJr2l3SYF/fHhhPetg4\nH5jgzeo11bROAOaZdyT3B8xsipntY2bDSf8n88xsEiXVAyBpR0mDauvAKUA3JT3vzOwfwAuSDnTT\nJ4GnabWedj+8aeJDodOAv5D6k69ptz+99Pk3wDpgE+nqYDKpP3gu8Kx/7uZtRXrj66/ACqCj3f7n\n6DmOdPu7HFjmy2kl13QY8GfX1A1c6/YRwCJgNXAfsJ3bB3p5tdePaLeGBtpOAmaVXY/73uXLU7X/\n/5Kfd6OBTj/vHgR2bbWeGMEdBEEQFFLVbqggCIKgD4lgEQRBEBQSwSIIgiAoJIJFEARBUEgEiyAI\ngqCQCBZBqZA0RtLUBvV7SZrexOOf7Vk/59fZh0v6/Hvc55960WZqM5JhSvqupCsL2pzVm2NLuljS\nl/rOu6A/EcEiKBvfBH7SU6WZvWhmE3qq7wMmA181s3F19uFAbrDIjITOxcyOKTqomX3ZzJ7urZN9\nzFmk7M1F3AVc0mRfgjYRwSJoC34lvsqvmLslTZM0XtLjnp//qJxtBgGHmVmXl0/0+QqWeYK1Qb7f\nbq+fmqnfIOk7br9K0mLP9X9dD/5NVJoPoVvSjW67ljTQ8A5JN9VtcgNwvB/rcknnSbpP0sOkhHY7\nSZoraanv98zMsV7zz5MkLdA78xZM81HwuL2j1l7S95Xm1HhS0p5uH+nlxZKur+03R9s1SnO9zAEO\nzNi/4tt2SZohaQdJxwBnADe5tpF57QDM7A1gTd7fLqgA7R6ZGMsHcyFdiW8GDiVdtCwhXZkKOBN4\nMGebccCMTPlhUsI4gJ1ICdeGk0nx7nX7Aav88xTSfMXy484CTqhrvxfwd2B33+c84CyvW0DOiFgy\no5+9fB5pFH5tVO0AYGdfH0oaAV0bFPtaZh8bSbl8PgQ8ARxXf1zSqPjTff1HwLd8fRaemhq4sLbf\nOj+PJI3q3YGU+no1cKXXDcm0+x7wNV+/G5iQqctt5+VrSHOYtP0ci6Vvl7izCNrJc2a2wsy2kNIy\nzLX0i7OC9KNfzzBSquYajwM3S7oEGGzvpNTeiqRaeoqLzex5UrA4hZSyYylwEGmSmCwfBxaY2Qbf\n5zSgUdbSnphtZrUkewJ+IGk5MIeUMnrPnG0Wmdla/06Wkf89/JcUGCAF2VqbsSStAL/uwafjgQfM\n7A1LGYCzOdM+JukxSSuAScBHe9hHo3brScE2qBgN+1KDoMm8lVnfkilvIf/cfJOUmwgAM7tB0m9J\n+aaelDQe+E/dNncA95vZHC8L+KGZ3dnAr7wUz++F1zPrk0h3Kkea2SalLK8Dc7bJfidvk/89bPKg\n2qhNI3rK8XM36Q6qS9J5pDudd9tuIOnvFFSMuLMIysRK4IBaQdJIvzO5kZRk7aBsY0kXAYPM7IaM\n+RHgfKU5NpC0t6Q9+H8WAidKGqo0Re9E4A8Fvr0KDGpQvwtp3ohNksaRusT6mieBz/n6OT20+SPw\nWUnb+zOg0zN1g4B1SmnlJ2Xs9dp6agfwEVKCxaBiRLAISoOZrQJ28R85gMv8AXQX6Wr2d3WbXAkc\nmnnIfaGZPUrqonnCu1GmU/cjbymd8xRSmu4uYKmZPURjlgOb/aHv5Tn104AOSZ2kH9hVvdX9LrgM\nuELSIlKX3cb6Bma2FLiX1MU1A3gsU/1tUqCcXeffPcBV/hLByAbtIM2PMYegckTW2aBU+A/xq2bW\n41iLDyr+VtKbZmaSziE97D6zaLs+PP4Y4Aoz+0Krjhm0jnhmEZSN24Gz2+1EP+VI4Kf+uu0rwPkt\nPv5Q0l1HUEHiziIIgiAoJJ5ZBEEQBIVEsAiCIAgKiWARBEEQFBLBIgiCICgkgkUQBEFQSASLIAiC\noJD/AV3FX5BbjlCXAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# first we split the training set into a training / validation set (as we are using k-folds cross validation elsewhere and don't have an explicit training and validation set.)\n", "X_train_train, X_cv, y_train_train, y_cv = model_selection.train_test_split(X_train, y_train, test_size=0.25, stratify=y_train)\n", "\n", "# set the model to calculate learning curves for\n", "#model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", "\n", "m_array = np.round(np.linspace(20, X_train_train.shape[0], 100)).astype(int)\n", "train_acc_array = []\n", "cv_acc_array = []\n", "\n", "for m in m_array:\n", " model = linear_model.LogisticRegression(fit_intercept=True)\n", " # we now fit to the training data\n", " model.fit(X_train_train.head(m), y_train_train.head(m)) # training on the first m training data examples\n", " train_accuracy = model.score(X_train_train.head(m), y_train_train.head(m))\n", " train_acc_array.append(train_accuracy)\n", " cv_accuracy = model.score(X_cv, y_cv)\n", " cv_acc_array.append(cv_accuracy)\n", "\n", "train_acc_array = np.array(train_acc_array)\n", "cv_acc_array = np.array(cv_acc_array)\n", "\n", "fig, ax = plt.subplots()\n", "ax.plot(m_array, 1-train_acc_array, label='train')\n", "ax.plot(m_array, 1-cv_acc_array, label='cross-validation')\n", "ax.legend()\n", "ax.set_xlabel('m (size of training data)')\n", "ax.set_ylabel('error (1-accuracy)')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This learning curve is typical of a high bias algorithm. Getting more data will not help improve the fitting, adding more features might." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Support Vector Machine" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Linear Kernel" ] }, { "cell_type": "code", "execution_count": 80, "metadata": { "collapsed": true }, "outputs": [], "source": [ "C = 1 # start with penalty parameter equal to 1 as we don't know what value this should take yet, larger C -> lower bias, higher variance, smaller C -> higher bias, lower varaince. Since we don't have that much data a lower bias algorithm is probably best to avoid overfitting\n", "\n", "kernel = 'linear' # we'll start with a simple linear kernal to see how ths performs\n", "\n", "svm_model = svm.SVC(C=C, kernel=kernel)" ] }, { "cell_type": "code", "execution_count": 81, "metadata": { "collapsed": true }, "outputs": [], "source": [ "N_splits = 10\n", "kfold = model_selection.KFold(n_splits=N_splits)\n", "\n", "scoring = 'accuracy'\n", "score = model_selection.cross_val_score(svm_model, X_train, y_train, cv=kfold, n_jobs=1, scoring=scoring)\n" ] }, { "cell_type": "code", "execution_count": 82, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "78.625000000000014" ] }, "execution_count": 82, "metadata": {}, "output_type": "execute_result" } ], "source": [ "score.mean()*100 # we get an average accuracy of ~78.5% over 10 folds with logistic regression" ] }, { "cell_type": "code", "execution_count": 83, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.54203125000000019" ] }, "execution_count": 83, "metadata": {}, "output_type": "execute_result" } ], "source": [ "score.var()*100 # we get a variance of ~0.2% so the accuracy is also pretty consistant" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0 0.003\n", "1 0.01\n", "2 0.02\n", "3 0.03\n", "4 0.05\n", "5 0.1\n", "6 0.3\n", "7 1.0\n", "8 3.0\n", "9 10.0\n" ] } ], "source": [ "C_array = np.array([0.003, 0.01, 0.02, 0.03, 0.05, 0.1, 0.3, 1, 3, 10])\n", "acc_mean_array = np.zeros_like(C_array)\n", "acc_variance_array = np.zeros_like(C_array)\n", "for i, C in enumerate(C_array):\n", " print(i, C)\n", " svm_model = svm.SVC(C=C, kernel=kernel) \n", " score = model_selection.cross_val_score(svm_model, X_train, y_train, cv=kfold, n_jobs=1, scoring=scoring)\n", " acc_mean_array[i] = score.mean()*100\n", " acc_variance_array[i] = score.var()*100" ] }, { "cell_type": "code", "execution_count": 85, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 85, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEACAYAAACkvpHUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4lOX18PHvIWHfCYoIKGhBtiSEHRWhRCluIAICtRWw\nSF1wrQv6c6Fa31qlpVptXVBEpcoIgqgoSmEQUZCgEQUEUVAie9gDgSzn/eOexCHMZGYgk8lyPtc1\nV/LsZx6GnHnuVVQVY4wxJlxVYh2AMcaY8sUShzHGmIhY4jDGGBMRSxzGGGMiYonDGGNMRCxxGGOM\niYglDmOMMRGxxGGMMSYiljiMMcZExBKHMcaYiMSX5sWqVKmiNWvWLM1LGmNMuXfo0CFV1TLzRb9U\nE0fNmjXJysoqzUsaY0y5JyKHYx2DvzKTwYwxxpQPljiMMcZExBKHMcaYiJRqHUcgOTk5ZGRkkJ2d\nHetQjClVNWrUoHnz5lStWjXWoRgTkZgnjoyMDOrWrUvLli0RkViHY0ypUFUyMzPJyMigVatWsQ7H\nmIjEvKgqOzubhIQESxqmUhEREhIS7EnblEsxf+IALGnESk4OHDoENn1wdFSpAvHxv7yqHPs9rax9\n7vfvh8WL7eMQLT17wqmnRufcIjIAeBKIA6ao6mNFto8GngB+9q16WlWn+G2vB6wFZqvq+FDXKxOJ\no7y75JJL+O9//0uDBg1K9bpjx47ljjvuoH379qF3VoWjR+HgQThwwP0M8W335XfeIW3tWp6++26e\nnTWLWjVqcM2llx6zz6YtW7js9tv5ZsaMoOfZtGULn65axW8HDAAgbc0aXpk3j6fuvDN03BVJlSpQ\nteqxyWTPHvjb3+CUU6Bx42N/1q8PpZRccnPhwgthxYpSuVyl9P774PsvUKJEJA54BrgIyABWiMhc\nVV1TZNcZxSSFR4DF4V7TEsdJUFVUlXnz5sXk+lOmTAm+UdUlhoIkcfCgSxwAcXFQpw4kJLifVYKU\nWK5cCdu2Qbt2XH///YH3qVULqleHdu2ChrJp507+u3Qpv739dgC6tmtH1yFDwnmLZUZubi7x8RH+\nd8nPd3+RC145Ocf+fviw+/eZMCHw8fHxLokUJJJAycX/Z+PGUK3aCb2/yZNd0nj6aejV64ROYUI4\n++yonbo7sEFVfwAQkTeAQUDRxBGQiHQBmgAfAF3DumLBH7/SeNWqVUuLWrNmzXHrStPdd9+tzzzz\nTOHyQw89pJMmTdIDBw5ov379NCUlRTt27Khz5sxRVdWNGzdq27Zt9YYbbtBOnTrppk2b9Mwzz9Sd\nO3eqquqgQYO0c+fO2r59e33uuecKz1u7dm297777NCkpSXv06KHbtm1TVdVt27bpFVdcoUlJSZqU\nlKRLly5VVdVXX31Vu3XrpsnJyTpu3DjNzc09LvY+ffroihUrfjn/nXdqUvv22iM5WbctWKC6YoV7\npadr3vr1emaLFrrn559V8/NVVfXss8/Wbdu26dy5c7V79+7aqVMnTU1NLYxt6tSpetNNNxXelyee\neEJVVdPS0jQpKUl79uypd955p3bo0KHw3px//vmakpKiKSkphe+lR48eWq9ePU1OTtZ//OMfumjR\nIr300ktVVTUzM1MHDRqkiYmJ2qNHD/3qq68KrzdmzBjt06ePtmrVSp988smA/37XX3+9dunSRdu3\nb68PPvhg4frPP/9ce/XqpUlJSdqtWzfdv3+/5ubm6p/+9Cft2LGjJiYm6lNPPaWqesy/34oVK7RP\nnz6FMVx33XV60UUX6ciRI4O+P1XVv/3tb9qxY0dNSkrSe+65Rzds2KApKSmF29evX6+dO3c+Lv41\na9aoHjyoummT+7eaN0/1lVdU//531XvvVR07VvWKK1TPO0/1nHNUGzVSdV8LAr/q1VM9+2zVnj1V\nL7tMdcwY1bvuUn38cdWXXlJ95x3Vzz5T3bBBde9e1fx8/fZb1erV3WV8Hw1TxgBZGuTvKjAUVzxV\nsPx7XFGU/z6jga3AKmAm0MK3vgrgBVr49nk62HX8X2XqieO22yA9vWTP2akT/POfwbePGDGC2267\njRtvvBEAj8fDBx98QI0aNZg9ezb16tVj165d9OzZk4EDBwKwbt06pk6dyr///e/jzvfSSy/RqFEj\nDh8+TLdu3RgyZAgJCQlkZWXRs2dPHn30Ue6++25eeOEF7r//fm655Rb69OnD7NmzycvL4+DBg6xd\nu5YZM2awdOlSqlatyo033sj06dO55pprjn8De/fCunXu/E2b8ui0adz9zDO8MG8e9997r3uiqF6d\nKiIMGjyY2fPnM2bMGJYvX07Lli1p0qQJ559/PsuWLUNEmDJlCo8//jh///vfg96zMWPG8K9//Ys+\nffpw1113Fa4/9dRT+eijj6hRowbfffcdI0eOJC0tjccee4xJkybx7rvvAuD1eguPeeihh0hJSWHO\nnDksXLiQa665hnTfh+Dbb79l0aJFHDhwgHPOOYcbbrjhuKarjz76KI0aNSIvL4/U1FRWrVpF27Zt\nGT58ODNmzKBbt27s37+fmjVr8vzzz7Nx40a+/PJL4uPj2b17d/APhs/KlSv55JNPqFmzJocOHQr4\n/t5//33mzJnD8uXLqVWrFrt376ZRo0bUr1+f9PR0OnXqxNSpUxk9enTgi9Su7V5nnhkyHsA9seze\nDTt3wq5dwX9mZLj/UDt3wpEjAU+VX7U6f5CPqZnXln/vGotcmVNqxWOVzgMPQErKiR4dLyJpfsvP\nq+rzvt8D/YMVral6B3hdVY+IyPXANKAfcCMwT1U3R1LnVqYSRyykpKSwY8cOtmzZws6dO2nYsCFn\nnHEGOTk53HfffXz88cdUqVKFn3/+me3btwNw5pln0rNnz4Dne+qpp5g9ezYAmzdv5rvvviMhIYFq\n1apx2WWXAdClSxc++ugjABYuXMgrr7wCQFxcHPXr1+fVV19l5cqVdOvWDYDDhw9zqn+tWn6++8OR\nlQVbtsBpp7nzjxoF9erRZcAAd/7GjY+Jbfjw4Tz88MOMGTOGN954g+HDhwOuSfTw4cPZunUrR48e\nLbZ56L59+9i7dy99+vQB4Pe//z3vv/8+4PrkjB8/nvT0dOLi4li/fn3I+//JJ58wa9YsAPr160dm\nZib79u0D4NJLL6V69epUr16dU089le3bt9O8efNjjvd4PDz//PPk5uaydetW1qxZg4jQtGnTwvtX\nr149ABYsWMD1119fWOTUqFGjkPENHDiQgoE5g72/BQsWMGbMGGrVqnXMeceOHcvUqVP5xz/+wYwZ\nM/j8889DXi8s8fGuljXcmlZVV1QZILk880Frli7ozsud/knT/J/h+wMlE6M53smN05erqsGKkTJw\nTwwFmgNb/HdQ1Uy/xReAv/l+7wX0FpEbgTpANRE5qKpByk+dMpU4insyiKahQ4cyc+ZMtm3bxogR\nIwCYPn06O3fuZOXKlVStWpWWLVsWNp2sXbt2wPN4vV4WLFjAZ599Rq1atejbt2/hMVWrVi1sRRMX\nF0dubm7QeFSVUaNG8de//vXYDfn5kJkJW7e6+goRaN4cOnRw509IKPb8vXr1YsOGDezcuZM5c+Zw\nv6/e4uabb+aOO+5g4MCBeL1eJk6cWGxswb6ZTJ48mSZNmvDVV1+Rn59PjRo1gp7H/3xFFZy/evXq\nhesCvaeNGzcyadIkVqxYQcOGDRk9ejTZ2dlBYwy2Pj4+nvz8fIDjmsf6/1sHe3/BzjtkyBD+/Oc/\n069fP7p06UKC79+n1IlA3bru5felYONGmPCgq7C9Zt5tILfFJj5zslYArUWkFa7V1Ajgt/47iEhT\nVd3qWxyIa0GFql7tt89ooGuopAFloB9HWTBixAjeeOMNZs6cydChQwH3zfrUU0+latWqLFq0iB9/\n/DHkefbt20fDhg2pVasW3377LcuWLQt5TGpqKv/5z38AyMvLY//+/aSmpjJz5kx27NgBwO5du/hx\n5Ur45hv48UfXMudXv3IV0/XqhV20ICIMHjyYO+64g3bt2hX+Idu3bx/NmjUDYNq0acWeo0GDBtSv\nX59PPvkEcAnW//03bdqUKlWq8Oqrr5KXlwdA3bp1OXAg8DfZCy64oPAcXq+Xxo0bFz4hhLJ//35q\n165N/fr12b59e+GTT9u2bdmyZQsrfE2EDhw4QG5uLv379+fZZ58tTEAFRVUtW7Zk5cqVAIVPP4EE\ne3/9+/fnpZde4tChQ8ect0aNGvzmN7/hhhtuYMyYMWG9p9KiCtdd59pJPPeclU6VZ6qaC4wH5uMS\ngkdVV4vIwyIy0LfbLSKyWkS+Am7B1WecMEscQIcOHThw4ADNmjWjadOmAFx99dWkpaXRtWtXpk+f\nTtu2bUOeZ8CAAeTm5pKUlMQDDzwQtDjL35NPPsmiRYtITEykS5curF69mvbt2/OXv/yF/v37k9S+\nPRddcAFbV61yLWZat4a2beEEm/4OHz6c1157rbCYCmDixIkMGzaM3r1707hI8VYgU6dO5aabbqJX\nr174z69y4403Mm3aNHr27Mn69esLv60nJSURHx9PcnIykydPPuZcEydOJC0tjaSkJCZMmBAycflL\nTk4mJSWFDh06cO2113LeeecBUK1aNWbMmMHNN99McnIyF110EdnZ2YwdO5YzzjiDpKQkkpOT+e9/\n/wu4epZbb72V3r17ExcXF/R6wd7fgAEDGDhwIF27dqVTp05MmjSp8Jirr74aEaF///5hv6/S8OKL\n8L//wRNPwBlnxDoac7JUdZ6qtlHVs1X1Ud+6B1V1ru/3e1W1g6omq+qvVfXbAOd4WcPowwEggYoK\noqV27dpadD6OtWvX0q6YppyVUl4e7NgB27e7itC6deH0091PU65MmjSJffv28cgjjwTcHovPf0YG\ndOgAXbrAggXBW2ObskNEDqlq4DLyGAhZxyEi5wD+vbvOAh7ENeF6FqgB5AI3qmoJ1f5VUrm5vySM\nvDzXAaxpU9cyypQ7gwcP5vvvv2fhwoWxDqWQKvzxj+6j9sILljTMiQmZOFR1HdAJCnso/gzMxtXM\n/1lV3xeRS4DHgb7RC7UCy8lxCWPHDpcwGjRwCSNIJbwpHwpa15Ul06fDvHmuw18UO6SZCi7SVlWp\nwPeq+qOIKFBQi1mfIs2/TBhyclzP7J07XYuphg1dwvA16zSmJG3bBrfc4nqG33xzrKMx5VmkiWME\n8Lrv99uA+SIyCVfJfu6JBlFcE88K6ejRXxKGKjRq5BKGX0WzqfhKs34RYPx4N6blSy+51lTGnKiw\nE4eIVMO1/73Xt+oG4HZVnSUiVwEvAhcGOG4cMA5ca5eiatSoQWZmZuUYWv3IEZcwdu1yywkJcNpp\nEEZ/B1OxqG8+jnD6upSEmTNh1iz4619dozxjTkbYrapEZBBwk6r29y3vAxqoqor7i79PVYttgB+o\nVVWlmAEwJ8eNWX3woFuuU8dVfEc6aJ6pUEprBsDMTGjfHlq0gGXL7GNXHpW7VlV+RvJLMRW4Oo0+\nuNZV/YDvTiSAqlWrVtwZ0NauhUcfhddfd30wrrsO7r7b9fY2ppTcdpsboeajjyxpmJIR1sdIRGrh\nxnr/o9/q64AnRSQeyMZXHGV85s+Hiy929RZ33AF/+pMrljKmFL37Lrz2Gjz0ECQlxToaU1HEvANg\nhXXttfD22/Dtt26+BGNK2d69rqNfo0ZuapUTnKrDlAHluajKRMLrhT59LGmYmLnrLtcWY84cSxqm\nZFm/0Wj46Sc39GjfvrGOxFRSH30EU6bAnXeCb3R5Y0qMJY5oWOybutc3Z4UxpengQdcOo00bKGaE\nfGNOmBVVRYPX63qBJybGOhJTCd17r3voXbLE+pSa6LAnjmgoqN+wEeRMKVuyBJ5+2g0p4htl3pgS\nZ3/ZStpPP8EPP1j9hil1hw/DH/7gJvn7f/8v1tGYiswSR0mz+g0TIw89BN9954ZLt4GVKxcRGSAi\n60Rkg4gcN/WriIwWkZ0iku57jfWt7yQin/lmB1wlIsOPP/vxrI6jpC1e7Oo3rLeVKUWffw5//7ur\nFE9NjXU0pjT5prt4BtdJOwNYISJzVXVNkV1nBJjh7xBwjap+JyKnAytFZL6q7i3umvbEUdK8Xrjg\nAqvfMKXmyBHX37RpUzcVrKl0ugMbVPUHVT0KvAEMCudAVV2vqt/5ft8C7ABCdj6zv24lafNm+P57\nq98wperRR2H1anjuOTd2pqmQ4kUkze/lP8RTM2Cz33KGb11RQ3zFUTNFpEXRjSLSHagGfB8ymAiD\nN8UpqN+wxGFKSXq6Gyr997+HSy+NdTQminJVtWuQbYHmoyg6ltQ7wOuqekRErgem4QandScQaQq8\nCoxS1fxQwdgTR0lavNhN+2r9N0wpyMlxRVQJCfDPf8Y6GhNDGYD/E0RziszIqqqZqnrEt/gC0KVg\nm4jUA94D7lfVZeFc0BJHSSqo37Dp1UwpeOIJ+PJL+Pe/3UCGptJaAbQWkVa+CfdGAHP9d/A9URQY\nCKz1ra8GzAZeUdU3w72gJY6SkpEBGzZYMZUpFWvXwp//DMOGwZVXxjoaE0uqmguMB+bjEoJHVVeL\nyMMiMtC32y2+JrdfAbcAo33rrwIuAEb7NdXtFOqaNqx6SZk+HX73O/jiC0hJiXU0pgLLy4Pzz3d9\nNlavhiZNYh2RiTYbVr2iWrzYNWmx/hsmyp56yk0B+9prljRMbNgTR0lp0wbatoW5c0Pva8wJ2rDB\nfTdJTXUfNQnUnsZUOGXticPqOErCzz+7cgOr3zBRlJ8PY8dC1arw7LOWNEzsWFFVSbD+G6YUPPec\n+6hNmQLNAnXvMqaU2BNHSfB6Xf1GcnKsIzEV1I8/wt13w4UXur4bxsRSyMQhIuf4NdNKF5H9InKb\nb9vNvhEZV4vI49EPt4xavBh697b+GyYqVGHcOPfzhResiMrEXsiiKlVdB3SCwlEYfwZmi8ivcQNp\nJfm6sZ8a1UjLqi1bYP169z/bmCiYNg0+/NBN0NSyZayjMSbyoqpU4HtV/RG4AXisoBu7qu4o6eDK\nBavfMFG0ZQvcfrt7oL3hhlhHY4wTaeIYAbzu+70N0FtElovIYhHpVrKhlRNeL9SrB51CdrY0JiKq\nLllkZ8OLL9pI/absCLtVlW9Mk4HAvX7HNgR6At0Aj4icpUU6hviG/x0HUK1atZKIuWyx+g0TJTNm\nuL4aTzwBrVvHOhpjfhHJd5iLgS9UdbtvOQN4S53PgXygcdGDVPV5Ve2qql3j4ytY69+tW2HdOium\nMiVu5064+Wbo3t0VVRlTlkSSOEbySzEVwBx847mLSBvcBCC7Si60csDqN0yU3Hwz7NsHL71kD7Om\n7AkrcYhILdx8tm/5rX4JOEtEvsFNVTiqaDFVhWf1GyYKZs92xVQPPggdOsQ6GmOOZ2NVnYx27eCs\ns+C992Idiakg9uyB9u3htNPg88/d8CLG2FhVFcW2bfDtt1ZMZUrUHXe4+o2XXrKkYcouSxwnyuo3\nTAn74AN4+WW45x6b0sVERkQG+Ebx2CAiEwJsHy0iO/1GABnrt22UiHzne40K63pWVHWCbrjBTd60\nezdUtNZiptTt3w8dO0KdOm4usBo1Yh2RKUuKK6ryjeixHlcPnYGbSnakqq7x22c00FVVxxc5thGQ\nBnQFFFgJdFHVPcXFY08cJ2rxYjcNmyUNUwLuucfNPvzSS5Y0TMS6AxtU9QdVPYprrDQozGN/A3yk\nqrt9yeIjYECogyxxnIjt292kz1ZMZUrAokVufo3bb4eePWMdjSmHmgGb/ZYzfOuKGiIiq0Rkpoi0\niPDYY1jiOBFWv2FKSFaWm5zp7LPhkUdiHY0pw+JFJM3v5T+qaqDxkovWQbwDtFTVJGABMC2CY48P\nJpyITRFeryuM7tw5pmHs2AGvvAKrVsHkyZCQENNwzAm4/3744Qf3kapVK9bRmDIsV1W7BtmWAbTw\nW24ObPHfQVUz/RZfAP7md2zfIsd6QwVjieNEeL0xq9/Iy4OPPnKzwL39NuTmusHvvv4aFiyw5FGe\nfPopPPkk3Hgj9OkT62hMObYCaC0irXDTXowAfuu/g4g0VdWtvsWBwFrf7/OB/yciDX3L/fllPMKg\nrKgqUjt2xKR+Y9MmeOghaNUKLr7YlZbdeiusWQPz5rmQLrrINfIyZV92NvzhD9CiBTz2WKyjMeWZ\nquYC43FJYC3gUdXVIvKwiAz07XaLb8K9r4BbgNG+Y3cDj+CSzwrgYd+6Yllz3Ei9+SZcdRUsWwY9\nekT1UkeOuNFRp0xxTxkA/fu7MvGBA8F/sOEPPoBBg9wQFQsWQKNGUQ3NnKT77oO//hXmz3f/psYU\np6z1HLfEEambbnIVC7t3R61r75o1bv6FV16BXbvct9Jrr4UxY+DMM4Mf9/77cMUVkJjoEk3DhsH3\nNbGzcqX7znHNNa75rTGhWOIo74mjQwf3l/yDD0r0tAcPgsfjni4++8zlpEGD3NPFhReGP0LqvHkw\neDAkJbnk0aBBiYZpTtLRo9CtmxtWZPVqS+4mPGUtcVgdRyR27HCPAyVUv6HqBrIbNw6aNnVl3nv2\nwKRJrjPYm2/Cb34T2bDal1wCs2bBV1+5IpC9e0skVFNCHnvMtYJ79llLGqb8slZVkfj4Y/fzJBNH\nZia89porjvr6a9cMc/hw93TRqxdIoJbVEbjsMpc8hgxxiefDD6F+/ZM7pzl5X38Nf/kLjBzp6qiM\nKa+sqCoS48e7Uej27Im4fiM/3/UQnjIF3nrrlyKLsWNhxAg3rUdJmzsXhg513U3mz7fkEUu5ue5L\nwY8/uofWxsfNlWlMcGWtqMqeOCJR0H8jgqTx888u17z4Imzc6Ion/vhHVyyVnBy1SAH3rfbNN13y\nGDDAJY9oJCgT2j/+AWlpboImSxqmvLM6jnAV1GaG2VNryxZXZHTGGa53cKtW8N//uvVPPRX9pFFg\n0CCXPNLSXPLYv790rmt+sW6dm81v8GAYNizW0Rhz8ixxhCvC+o3HH3etmiZMgA0b4H//c2XbsRj5\n9IorXIutFStc58EDB0o/hsoqP989XdasCc88c/L1V8aUBZY4wlUwmFDXYMPF/CI/333Lv+QSePRR\nN4BdrA0eDG+8AcuXW/IoTc88A0uXwj//6VrOGVMRWOIIVwT1G0uXuiKpq66KfliRGDLEJY9ly1xS\nO3gw1hFVbBs3uifOAQNcZz9jKoqQiUNEzvGbbjBdRPaLyG1+2+8UERWRilvlt2sXfPNN2PUbHo8r\nkrr88ijHdQKGDoXXX3edDC15RI+qazEXFwfPPWdFVKZiCdmqSlXXAZ2gcIrCn4HZvuUWuOkKf4pi\njLEXQf1GXh7MnAmXXupGXi+Lhg1zxWlXX+2KrUaMiHVEFc+6dbBwoevod8YZsY7GmJIVaXPcVOB7\nVf3RtzwZuBt4u0SjKmsiqN9YsgS2bSt7xVRFDR/ufl5zDXzySWxjqaguuQSuuy7WURhT8iJNHCOA\n1wF8w/X+rKpfSUV/Dvd64bzzjh2ONgiPx+WYSy+Nflgna/hwF+fhw7GOpGJq3NiKqEzFFHbiEJFq\nuAlA7hWRWsD/4Sb9CHXcOGAcQLUw/vCWObt2ubEiCr6iFyM31xVTXXYZ1C4zfTyLV6dO2S1SM8aU\nTZG0qroY+EJVtwNnA62Ar0RkE266wS9E5LSiB6nq86raVVW7xsdgxryTtmSJ+xlG/cbixa6fYFkv\npjLGmJMRSeIYia+YSlW/VtVTVbWlqrbEzVvbWVW3RSHG2PJ6Xe+tbt1C7urxuCeNSy6JfljGGFNA\nRAaIyDoR2SAiE4rZb6ivFWxX33JVEZkmIl+LyFoRCTltLISZOHxFUxcBb4Wzf4USZv1GTo4bkXbg\nQJdnjDGmNPhauz6DKxVqD4wUkfYB9quLmzZ2ud/qYUB1VU0EugB/FJGWoa4ZVuJQ1UOqmqCq+4Js\nb6mqu8I5V7mSmekmTwijmGrRIre7FVMZY0pZd2CDqv6gqkeBN4BBAfZ7BHgcyPZbp0BtEYkHagJH\ngZAj2lnP8eIU1G+E0fHP44G6dV0vYWOMKWHxIpLm9xrnt60ZsNlvOcO3rpCIpAAtVPXdIuedCWQB\nW3H98Sap6u6QwZzIO6g0wqzfOHrUzbExaFBsBjE0xlR4uaoarCNZoEbfhRMtiUgVXJ+70QH26w7k\nAacDDYElIrJAVX8oLhhLHMXxeuHcc6F69WJ3+9//3NxOVkxljImBDKCF33JzYIvfcl2gI+D19bk7\nDZjr64v3W+ADVc0BdojIUqArUGzisKKqYHbvDrt+w+Nxs+v1D9mrxRhjStwKoLWItPL1txsBzC3Y\nqKr7VLWxXyvYZcBAVU3DFU/1E6c20BP4NtQFLXEE8/HHbqS6EPUbR47A7NluzosQDybGGFPiVDUX\nGA/MB9YCHlVdLSIP+54qivMMUAf4BpeApqrqqlDXtKKqYBYvdhUW3bsXu9tHH8G+fVZMZYyJHVWd\nB8wrsu7BIPv29fv9IK5JbkTsiSOYMOs3PB43j/iFF5ZOWMYYE2uWOALZvRu++ipk/UZ2NsyZ42bX\nK4/DcBljzImwxBHIkiVh1W/Mn++mYLViKmNMZWKJI5Aw6zc8HkhIgH79SikuY4wpAyxxBOL1Qq9e\nxfbmO3wY5s6FK68MaxpyY4ypMCxxFLVnD6Snh6zfeP99N1+3FVMZYyobSxxFhVm/4fHAKaeE1T/Q\nGGMqFEscRS1e7Jrg9ugRdJesLHjnHRgyBMrj3FTGGHMyLHEUFUb9xrx5cOiQFVMZYyonSxz+9u6F\nL78MWf7k8UCTJnDBBaUTljHGlCWWOPwV1G8UkzgOHoT33oOhQyEurvRCM8aYssIShz+vN2T9xrvv\nuqa4VkxljKmsLHH4W7wYevYstn7D44GmTeH880sxLmOMKUMscRQIo35j/35XMT5sGFSxO2eMqaRC\nNiYVkXOAGX6rzgIexM1pezlucvPvgTGqujcaQZaKTz6B/PxiE8c777j5N6yYyhhTmYX83qyq61S1\nk6p2AroAh4DZwEdAR1VNAtYD90Y10mjzet0Qt8XUb3g80Ly5a61rjDFlhYgMEJF1IrJBRCYUs99Q\nEVER6erkKloNAAAc50lEQVS3LklEPhOR1SLytYgEL6v3ibTAJRX4XlV/VNUPfTNPgZuKsHmE5ypb\nCuo3atYMuHnvXvjgAyumMsaULSISh5vJ72KgPTBSRNoH2K8ucAuw3G9dPPAacL2qdgD6Ajmhrhnp\nn8ARwOsB1l8LvB/hucqOffvgiy+KLaaaOxeOHrViKmNMmdMd2KCqP6jqUeANYFCA/R4BHgey/db1\nB1ap6lcAqpqpqnmhLhh24vBNgj4QeLPI+v8DcoHpQY4bJyJpIpKWm5sbaJfYC6N+w+OBM84otiTL\nGGNioRmw2W85w7eukIikAC1U9d0ix7YBVETmi8gXInJ3OBeMZKSli4EvVHW7XzCjgMuAVFXVQAep\n6vPA8wC1a9cOuE/MFdRv9OwZcPOePfDhh3DrrSBSuqEZYwwQLyJpfsvP+/62AgT6q1T4t1ZEqgCT\ngdGBzgucD3TD1V//T0RWqur/ig0mgsBH4ldMJSIDgHuAPqp6KILzlD2LF7tHiSD1G3PmQE6OFVMZ\nY2ImV1W7BtmWAbTwW24ObPFbrgt0BLzivvmeBswVkYG+Yxer6i4AEZkHdAaKTRxhFVWJSC3gIuAt\nv9VP+wL6SETSReTZcM5V5uzfDytXhiymatUKugb7ZzPGmNhZAbQWkVa+KoURwNyCjaq6T1Ubq2pL\nVW2Ja8w0UFXTgPlAkojU8lWU9wHWhLpgWE8cvieKhCLrfhXmmyrbQtRvZGbCggXwpz9ZMZUxpuxR\n1VwRGY9LAnHAS6q6WkQeBtJUdW4xx+4RkX/gko8C81T1vVDXtNkkQtRvzJ4NublWTGWMKbtUdR4w\nr8i6B4Ps27fI8mu4Jrlhsx4JXi907w61agXc7PHAr34FKSmlG5YxxpRVlTtx7N9fbP+NnTth4UL3\ntGHFVMYY41TuxLF0KeTlBU0cb73lNlsxlTHG/KJyJw6vF6pWDTr4lMcD55wDSUmlG5YxxpRlljiC\n1G9s3+42WzGVMcYcq/ImjgMHiu2/MWuWa6VrxVTGGHOsyps4QtRveDzQvj107Fi6YRljTFlXeRNH\nMfUbW7bAxx/b04YxxgRSuRNHt25Qu/Zxm2bNAlVLHMYYE0jlTBwHD0JaWrHFVImJ0K5d6YZljDHl\nQeVMHMXUb2RkuOGr7GnDGGMCq5yJw+uF+Hg499zjNs2c6X5a4jDGmMAkyPxLUVG7dm3NysoqtesF\n1auX65zx6afHbTr3XDh8GL78MgZxGWNMACJySFWPr5CNkcr3xFFM/cZPP8Fnn9nThjHGFKfyJY5P\nP3XjpAdIHG/6ZlO3xGGMMcFVvsRRTP2GxwNdusDZZ5d+WMYYU15UzsTRrRvUqXPM6o0b4fPP7WnD\nGFP+iMgAEVknIhtEZEIx+w0VERWRrkXWnyEiB0XkznCuV7kSR1YWrFgBffoct6mgmGrYsFKOyRhj\nToKIxAHPABcD7YGRItI+wH51gVuA5QFOMxl4P9xrVq7EUUz9hsfjBspt1ar0wzLGmJPQHdigqj+o\n6lHgDWBQgP0eAR4Hsv1XisgVwA/A6nAvGDJxiMg5IpLu99ovIreJSCMR+UhEvvP9bBjuRWPG64W4\nODjvvGNWb9jgBsq1YipjTBkVLyJpfq9xftuaAZv9ljN86wqJSArQQlXfLbK+NnAP8OeIggm1g6qu\nAzr5LhIH/AzMBiYA/1PVx3xlahN8AZRdQeo3rJjKGFPG5apq1yDbAs0YVNhBT0Sq4IqiRgfY78/A\nZFU9KBFMPBQycRSRCnyvqj+KyCCgr2/9NMBLWU4cWVmu9vtPfzpuk8fj+gSecUYM4jLGmJOTAbTw\nW24ObPFbrgt0BLy+5HAaMFdEBgI9gKEi8jjQAMgXkWxVfbq4C0aaOEYAr/t+b6KqWwFUdauInBrh\nuUrXZ58FrN9Yvx7S02Hy5NiEZYwxJ2kF0FpEWuFKhEYAvy3YqKr7gMYFyyLiBe5U1TSgt9/6icDB\nUEkDIqgcF5FqwEDgzXCP8R03rqBcLjc3N5JDS1aQ+g2Px/0cOrT0QzLGmJOlqrnAeGA+sBbwqOpq\nEXnY91RR4sIeq8pXNHWTqvb3La8D+vqeNpoCXlU9p7hzxHSsqvPPd08cy5YdszopCerXhyVLYhOW\nMcaEUp7HqhrJL8VUAHOBUb7fRwFvl1RQJe7QIVe/UaT/xtq18PXX1prKGGMiEVbiEJFawEXAW36r\nHwMuEpHvfNseK/nwSshnn0FOznH1Gx6PGyR3yJDYhGWMMeVRWJXjqnoISCiyLhPXyqrsK6Z+44IL\n4PTTYxOWMcaUR5Wj57jX60YvrFevcNU338CaNVZMZYwxkar4iePQIVi+/Lj6DY8HqlSxYipjjIlU\nxU8cAeo3VF3i6NsXmjSJWWTGGFMuVfzEsXixe7Q4//zCVatWwbp1VkxljDEnouInjgD1Gx6Pqyu/\n8srYhWWMMeVVxU4cBfUbAYqp+vWDU06JXWjGGFNeVezEsWwZHD16TMX4l1+6YdStmMoYY05MxU4c\nAeo3PB435fjgwTGMyxhjyrGKnTi8Xujc2Q1GxS/FVBdeCAkJxR9qjDEmsIqbOA4fdkVVfvUbaWmw\ncaMVUxljzMmouIkjQP2GxwNVq8IVV8QwLmOMKecqbuIoUr9RUEzVvz80LPuzoxtjTJlVcROH1wsp\nKdCgAeBa5f70kxVTGWMqHhEZICLrRGSDiEwoZr+hIqIi0tW3fJGIrBSRr30/+4VzvYqZOLKzj6vf\n8HigWjUYNCh2YRljTEkTkTjgGeBioD0wUkTaB9ivLnALsNxv9S7gclVNxM2r9Go416yYiWPZMjhy\npLB+Iz8f3nwTBgwobGBljDEVRXdgg6r+oKpHgTeAQF+RHwEeB7ILVqjql6q6xbe4GqghItVDXbBi\nJg6v183Q1NvNw/7ZZ5CRYcVUxpgKqRmw2W85w7eukIikAC1U9d1izjME+FJVj4S6YFgTOZU7ixcf\nU7/h8UD16nD55TGOyxhjTky8iKT5LT+vqs/7fpcA+2vBLyJSBZgMjA52chHpAPwN6B9WMOHsVK5k\nZ7tHjJtuAn4pprrkkmPGOTTGmPIkV1W7BtmWAbTwW24ObPFbrgt0BLwiAnAaMFdEBqpqmog0B2YD\n16jq9+EEU/GKqpYvd/UbvorxTz6BrVutmMoYU2GtAFqLSCsRqQaMAOYWbFTVfaraWFVbqmpLYBlQ\nkDQaAO8B96rq0nAvWPESR5H6DY8HataEyy6LbVjGGBMNqpoLjAfmA2sBj6quFpGHRWRgiMPHA78C\nHhCRdN/r1FDXFFUNtQ++rDQF97ijwLXAYeBZoAaQC9yoqp8Xd57atWtrVlZWyOudlH79YO9e+OIL\n8vKgWTOXQ958M7qXNcaYaBGRQ6paO9ZxFAj3ieNJ4ANVbQsk47La48CfVbUT8KBvObYK6jd8xVQf\nfwzbt1sxlTHGlKSQleMiUg+4AF+NvK+d8FERUaCgurk+x1bGxMbnn7vk4UscHg/UquUqxo0xxpSM\ncFpVnQXsBKaKSDKwErgVuA2YLyKTcE8u50YtynD51W/k5sKsWa4Jbu0y84BnjDHlXzhFVfFAZ+A/\nqpoCZAETgBuA21W1BXA78GKgg0VknIikiUhabm5uCYUdxOLFkJwMDRvi9cLOnVZMZYwxJS2cxJEB\nZKhqwfgmM3GJZBTwlm/dm7hu78dR1edVtauqdo2Pj2K3kSNH4NNPjymmqlMHLr44epc0xpjKKGTi\nUNVtwGYROce3KhVYg6vTKJjsoh/wXVQiDJdf/UZODrz1Fgwc6JriGmOMKTnhPgLcDEz3dS75ARgD\nvA08KSLxuEGzxkUnxDD51W8sXAiZmVZMZYwx0RBWP46SEtV+HKmpLlukp/OHP8DMma4pbo0a0bmc\nMcaUlvLaj6NsO3KksP/G0aMwe7abd8OShjHGlLyKkThWrIDDh6FvXxYsgD17rJjKGGOipWIkDq/X\n/ezdG4/HTdZ00UUxjcgYYyqsipM4kpI4UieBOXNg8GA3/4YxxpiSV/4Tx9Gjhf03PvwQ9u2zYipj\njImm8p84/Oo3PB5o2NA1sDLGGBMd5T9x+Oo3srtfwNtvw5VXQrVqsQ3JGGMqsoqROBIT+WBFAgcO\nWDGVMcZEW/lOHH71Gx4PJCTAr38d66CMMaZ0icgAEVknIhtEZEIx+w0VERWRrn7r7vUdt05EfhPO\n9cp34khLg0OHONQrlblzYcgQqFo11kEZY0zpEZE44BngYqA9MFJE2gfYry5wC7Dcb1173BzlHYAB\nwL995ytW+U4cvvqN97N/TVaWFVMZYyql7sAGVf3BN9HeG8CgAPs9gpupNdtv3SDgDVU9oqobgQ0E\nGencX/lPHB074vmgHqecAn36hDzCGGPKo/iCeY18L/9BZZsBm/2WM3zrColICtBCVd8tct6QxwYM\nJqLQy5KcHFi6lKzfX8+7r8KoURDN6T6MMSaGclW1a5BtEmBd4ei1IlIFmIxv+u9Ijg2m/P6p9dVv\nvFd7GIcOWTGVMabSygBa+C03x82XVKAu0BHwigjAacBcERkYxrEBld+iKl/9hmd9CqedBr17xzYc\nY4yJkRVAaxFp5ZszaQQwt2Cjqu5T1caq2lJVWwLLgIGqmubbb4SIVBeRVkBr4PNQFyy/TxxeLwfa\nduO9BdUZOxbiQrYDMMaYikdVc0VkPDAfiANeUtXVIvIwkKaqc4s5drWIeHCzuuYCN6lqXqhrls+J\nnHJyoGFDXj/3X/z2ozF8/LE9cRhjKi6byKkkrFwJWVl49vbn9NPhvPNiHZAxxlQe5TNxeL3spy7v\nrzqdYcOgSvl8F8YYUy6Vzz+5Xi9zT7+BI0fEWlMZY0wpCytxiEgDEZkpIt+KyFoR6eVbf7NvfJPV\nIvJ4dEP1ycmBTz7BU+1qmjeHnj1L5arGGGN8wm1V9STwgaoO9TX3qiUiv8Z1V09S1SMicmrUovT3\nxRfszYrng+yO3HyrFVMZY0xpC5k4RKQecAG+Xoe+sVCOisgNwGOqesS3fkcU4/yF18vbDCInr4oV\nU5mTlpOTQ0ZGBtnZ2aF3NibKatSoQfPmzalaxkdrDeeJ4yxgJzBVRJKBlcCtQBugt4g8ihs0605V\nXRG1SAt4vXjqTODMBOgeciguY4qXkZFB3bp1admyJb5etcbEhKqSmZlJRkYGrVq1inU4xQqnoCce\n6Az8R1VTgCxggm99Q6AncBfgkQD/80RkXMHAXLm5uScXbW4uu5es5sND53PVVWD/z83Jys7OJiEh\nwZKGiTkRISEhoVw8/YaTODKADFUtGMN9Ji6RZABvqfM5kA80Lnqwqj6vql1VtWv8yY5C+MUXzMm6\nkNz8OCumMiXGkoYpK8rLZzFk4lDVbcBmETnHtyoV1z19DtAPQETaANWAXVGK0/F68XAVZ52ZS5cu\nUb2SMSaA0aNHM3PmTADGjh3LmjVrjtvn5ZdfZvz48cWex+v18umnnxYuP/vss7zyyislG6yJmnAf\nAW4GpvtaVP0AjMEVWb0kIt8AR4FRGuXxS3Z9+AULuIO7RsZbMZUxMTZlypQTPtbr9VKnTh3OPfdc\nAK6//vqSCqvU5ObmctKlKOVUWI1ZVTXdV9yUpKpXqOoeVT2qqr9T1Y6q2llVF0Y10txcZn9yCnnE\nWzGVqTA2bdpE27ZtGTt2LB07duTqq69mwYIFnHfeebRu3ZrPP3cDlWZlZXHttdfSrVs3UlJSePvt\ntwuP7927N507d6Zz586F3+K9Xi99+/Zl6NChtG3blquvvpqi3+vWrl1Ld78WJps2bSIpKQmAhx9+\nmG7dutGxY0fGjRt33LEAffv2JS0tDYCpU6fSpk0b+vTpw9KlSwv3eeedd+jRowcpKSlceOGFbN++\nnU2bNvHss88yefJkOnXqxJIlS5g4cSKTJk0CID09nZ49e5KUlMTgwYPZs2dP4fXuueceunfvTps2\nbViyZMlxMR08eJDU1FQ6d+5MYmJi4X0CeOWVV0hKSiI5OZnf//73AGzfvp3BgweTnJxMcnIyn376\nKZs2baJjx46Fx02aNImJEycWxnDffffRp08fnnzyyYDvryCOMWPGkJiYSFJSErNmzeLFF1/k9ttv\nLzzvCy+8wB133BHkk1HGqWqpvWrVqqUnbPlyvZAP9Ven7df8/BM/jTH+1qxZE9Prb9y4UePi4nTV\nqlWal5ennTt31jFjxmh+fr7OmTNHBw0apKqq9957r7766quqqrpnzx5t3bq1Hjx4ULOysvTw4cOq\nqrp+/Xrt0qWLqqouWrRI69Wrp5s3b9a8vDzt2bOnLlmy5LjrJycn6/fff6+qqo899pg+8sgjqqqa\nmZlZuM/vfvc7nTt3rqqqjho1St98801VVe3Tp4+uWLFCt2zZoi1atNAdO3bokSNH9Nxzz9WbbrpJ\nVVV3796t+b7/sC+88ILecccdqqr60EMP6RNPPFF4Df/lxMRE9Xq9qqr6wAMP6K233lp4vYLj33vv\nPU1NTT3u/eTk5Oi+fftUVXXnzp169tlna35+vn7zzTfapk0b3blz5zHv76qrrtLJkyerqmpubq7u\n3btXN27cqB06dCg85xNPPKEPPfRQYQw33HBD4bZg7+/uu+8ujLtgv4MHD+pZZ52lR48eVVXVXr16\n6apVq457D4E+k0CWluLf6lCvcvOcteO9FSzkeu4dftiKqUx03HYbpKeX7Dk7dYJ//rPYXVq1akVi\nYiIAHTp0IDU1FREhMTGRTZs2AfDhhx8yd+7cwm/l2dnZ/PTTT5x++umMHz+e9PR04uLiWL9+feF5\nu3fvTvPmzX1hdGLTpk2cf/75x1z7qquuwuPxMGHCBGbMmMGMGTMAWLRoEY8//jiHDh1i9+7ddOjQ\ngcsvvzxg/MuXL6dv376ccsopAAwfPrwwjoyMDIYPH87WrVs5evRoyGam+/btY+/evfTxzQM9atQo\nhg0bVrj9yiuvBKBLly6F98afqnLffffx8ccfU6VKFX7++We2b9/OwoULGTp0KI0bu/Y7jRo1AmDh\nwoWFdStxcXHUr1+/8AknmOHDhxf+Huz9LViwgDfeeKNwv4YNGwLQr18/3n33Xdq1a0dOTk7hv3t5\nU276Xb81S8knjquurRPrUIwpUdWrVy/8vUqVKoXLVapUoaAJu6oya9Ys0tPTSU9P56effqJdu3ZM\nnjyZJk2a8NVXX5GWlsbRo0cDnjcuLo5AzeGHDx+Ox+Nh/fr1iAitW7cmOzubG2+8kZkzZ/L1119z\n3XXXhWwiGqw10M0338z48eP5+uuvee655066qWnBewr2fqZPn87OnTtZuXIl6enpNGnShOzsbFQ1\n7BZL8fHx5OfnFy4Xjbl27V9GNw/2/oJdb+zYsbz88stMnTqVMWPGhBVPWVQ+njhyc/F8m8g5DbaR\nmHharKMxFVWIJ4NY+s1vfsO//vUv/vWvfyEifPnll6SkpLBv3z6aN29OlSpVmDZtGnl5IefgOcbZ\nZ59NXFwcjzzySOE36YI/fo0bN+bgwYPMnDmToUOHBj1Hjx49uPXWW8nMzKRevXq8+eabJCcnA+4J\nolmzZgBMmzat8Ji6deuyf//+485Vv359GjZsyJIlS+jduzevvvpq4dNHOPbt28epp55K1apVWbRo\nET/++CMAqampDB48mNtvv52EhAR2795No0aNSE1N5T//+Q+33XYbeXl5ZGVl0aRJE3bs2EFmZiZ1\n6tTh3XffZcCAAUGvF+j99e/fn6effpp/+j5Te/bsoWHDhvTo0YPNmzfzxRdfsGrVqrDfVygiMgA3\nNFQcMEVVHyuy/XrgJiAPOAiMU9U1IlIVmILrYhEPvKKqfw11vXLxxLHto69ZnHc+w/vtsmIqUyk9\n8MAD5OTkkJSURMeOHXnggQcAuPHGG5k2bRo9e/Zk/fr1x3wbDtfw4cN57bXXuMrX6qRBgwZcd911\nJCYmcsUVV9CtW7dij2/atCkTJ06kV69eXHjhhXTu3Llw28SJExk2bBi9e/cuLCYCuPzyy5k9e3Zh\n5bi/adOmcdddd5GUlER6ejoPPvhg2O/l6quvJi0tja5duzJ9+nTatm0LuCLA//u//6NPnz4kJycX\nVko/+eSTLFq0iMTERLp06cLq1aupWrUqDz74ID169OCyyy4rPEcgwd7f/fffz549e+jYsSPJycks\nWrSocNtVV13FeeedV1h8dbJEJA54BrgYaA+MFJH2RXb7r6omqmon4HHgH771w4DqqpoIdAH+KCIt\nQ15Ty8EMgM8MXsD4ORfyzaKddOh7ShQiM5XV2rVradeuXazDMJXIZZddxu23305qamrA7YE+k8XN\nAOgbrXyiqv7Gt3wvQLAnBxEZCVyjqhf7fv8tMBioD3wG9FTV3cW9h3LxxLEjM44utdZY0jDGlFt7\n9+6lTZs21KxZM2jSOEHNgM1+yxm+dccQkZtE5HvcE8ctvtUzcX3ytgI/AZNCJQ0oJ3Ucf/741zyU\nV3pPRsYYU9IaNGhwTKu3CMWLSJrf8vOq+rzv90AF+Mf9wVTVZ4BnROS3wP3AKKA7rt7jdNzYg0tE\nZIGq/lBsMCfwBmKiSpxVbhhjKq1cVe0aZFsG0MJvuTmwpZhzvQH8x/f7b3FzLeUAO0RkKdAVN0JI\nUOWiqMqYaCrNej5jinOCn8UVQGsRaeUbFmoEMNd/BxFp7bd4KfCd7/efgH7i1MaNdv5tqAta4jCV\nWo0aNcjMzLTkYWJOffNx1KhRI9LjcoHxwHxgLeBR1dUi8rCIDPTtNt43xXc6cAeumApca6w6wDe4\nBDRVVUO2Ey4XraqMiRabAdCUJcFmACyuVVUsWOIwxpgyrqwlDiuqMsYYExFLHMYYYyJiicMYY0xE\nSrWOQ0TygcMneHgcrqNKSexb3PZg2wKtL7qu6HI8cPwQniUvkntzMseHs1+k9/ZE15WHe2uf2eDK\nyn0tbp+Tua+B1p3Mva2pqmXni35pTv6B6+0Y9WND7Vvc9mDbAq0vui7AclpZv6+RHB/OfpHe2xNd\nVx7urX1my/59jfT+hXtfY3lvS+NV2hnsnVI6NtS+xW0Pti3Q+qLrTub9nYyTvW64x4ezX6T39mTW\nlQb7zEZHWbmvxe1zMvc13GuXS6VaVFXZiEiaBh8mwJwEu7fRYfc1eirSvS07ZWYV0/OhdzEnyO5t\ndNh9jZ4Kc2/ticMYY0xE7InDGGNMRCxxGGOMiYglDmOMMRGxxBEjInKFiLwgIm+LSP9Yx1NRiMhZ\nIvKiiMyMdSwVgYjUFpFpvs/q1bGOp6Io759TSxwnQEReEpEdIvJNkfUDRGSdiGwQkQnFnUNV56jq\ndcBoYHgUwy03Sui+/qCqf4hupOVbhPf5SmCm77M68LiTmUKR3Nfy/jm1xHFiXgYG+K8QkTjcpCgX\nA+2BkSLSXkQSReTdIq9T/Q6933ecKdn7aoJ7mTDvM24a0s2+3U5mWJvK4GXCv6/lWrmZc7wsUdWP\nRaRlkdXdgQ3qm+RdRN4ABqnqX4HLip5DRAR4DHhfVb+IbsTlQ0ncVxNaJPcZN591cyAd+6JZrAjv\n65rSja5k2Qeh5DTjl29m4P7DNStm/5uBC4GhInJ9NAMr5yK6ryKSICLPAikicm+0g6tAgt3nt4Ah\nIvIfKvAQGlEU8L6W98+pPXGUHAmwLmjvSlV9CngqeuFUGJHe10zAEnHkAt5nVc0CxpR2MBVIsPta\nrj+n9sRRcjKAFn7LzYEtMYqlIrH7WjrsPkdHhbyvljhKzgqgtYi0EpFqwAhgboxjqgjsvpYOu8/R\nUSHvqyWOEyAirwOfAeeISIaI/EFVc4HxwHxgLeBR1dWxjLO8sftaOuw+R0dluq82yKExxpiI2BOH\nMcaYiFjiMMYYExFLHMYYYyJiicMYY0xELHEYY4yJiCUOY4wxEbHEYYwxJiKWOIwxxkTEEocxxpiI\n/H/1Wx+h96/vygAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots()\n", "ax.plot(C_array, acc_mean_array, label='mean validation accuracy', color='red')\n", "ax2 = ax.twinx()\n", "ax2.plot(C_array, acc_variance_array, label='variance in validation accuracy', color='blue')\n", "ax.semilogx()\n", "ax.legend()\n", "ax2.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From this we can conclude that a value of C=0.1 is best for a linear kernel SVM." ] }, { "cell_type": "code", "execution_count": 86, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mean = 78.62500000000001\n", "variance = 0.5420312500000002\n" ] } ], "source": [ "N_splits = 10\n", "kfold = model_selection.KFold(n_splits=N_splits)\n", "\n", "scoring = 'accuracy'\n", "score = model_selection.cross_val_score(svm_model, X_train, y_train, cv=kfold, n_jobs=1, scoring=scoring)\n", "\n", "print(\"mean = {}\".format(score.mean()*100)) # we get an average accuracy of ~78.5% over 10 folds with logistic regression\n", "\n", "print(\"variance = {}\".format(score.var()*100)) # we get a variance of ~0.2% so the accuracy is also pretty consistant" ] }, { "cell_type": "code", "execution_count": 87, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "SVC(C=0.1, cache_size=200, class_weight=None, coef0=0.0,\n", " decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',\n", " max_iter=-1, probability=False, random_state=None, shrinking=True,\n", " tol=0.001, verbose=False)" ] }, "execution_count": 87, "metadata": {}, "output_type": "execute_result" } ], "source": [ "bestC = 0.1\n", "svm_model = svm.SVC(C=bestC, kernel=kernel)\n", "# we now train our model\n", "svm_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 88, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.7640449438202247" ] }, "execution_count": 88, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# and test it on our test data we have held back from the training/validation set\n", "svm_model.score(X_test, y_test) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We actually get 76% accuracy slightly lower than for the validation data, this is expected as we have tuned the hyperparameter C to perform best on the validation data. This is also similar to that we got with the logistic regression, not suprising since a SVM with a linear kernal is a very similar algorithm. This suggests the model is fitted correctly and is performing well, we now predict on Kaggle's test data and save the prediction to submit to Kaggle." ] }, { "cell_type": "code", "execution_count": 89, "metadata": { "collapsed": false }, "outputs": [], "source": [ "prediction = svm_model.predict(data_test)\n", "\n", "with open('prediction_submission_SVM_linear_kernel.csv', 'w') as file:\n", " print('PassengerId,Survived', file=file)\n", " for i, id_ in enumerate(data_test.index):\n", " print('{},{}'.format(id_, prediction[i]), file=file)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When we submit this result to Kaggle we get a reported accuracy of **~76.5%** which is again very similar to when we evaluated on our held back test data." ] }, { "cell_type": "code", "execution_count": 96, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 96, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4lGXat887kzLpIY2SQu8EQkdRpKnYEAv23uvq8r2r\nvpZddV9dV113LVh3bbs2EFFcwYKKKIJIk94SIAkESCE9mZS5vz/umWSSTJJJmRS4zuPIMTNPm2sS\neH5z1VtprREEQRCExvDpaAMEQRCEzo+IhSAIgtAkIhaCIAhCk4hYCIIgCE0iYiEIgiA0iYiFIAiC\n0CQiFoIgCEKTiFgIgiAITSJiIQiCIDSJb0cb0FZER0frPn36dLQZgiAIXYr169dna61jmjruuBGL\nPn36sG7duo42QxAEoUuhlDrgyXEShhIEQRCaRMRCEARBaBIRC0EQBKFJjpuchSAI3qGiooKMjAzK\nyso62hShFVitVuLj4/Hz82vR+SIWgiA0SkZGBqGhofTp0welVEebI7QArTU5OTlkZGTQt2/fFl1D\nwlCCIDRKWVkZUVFRIhRdGKUUUVFRrfIORSwEQWgSEYquT2v/hiIWddnzDeTu62grBEEQOhUiFnVZ\neB389FxHWyEIgoO8vDxefvnlZp939tlnk5eX5wWLTkxELFwpL4HyIshJ7WhLBEFw0JBYVFVVNXre\n0qVLiYiI8JZZJxxSDeVKaa55zBWxEITOwgMPPEBKSgrJycn4+fkREhJCz5492bRpE9u3b2fOnDmk\np6dTVlbGPffcwy233ALUjAAqKirirLPO4pRTTuHnn38mLi6Ozz77jMDAwA7+ZF0LEQtXSnLMY+Eh\n42X4B3WsPYLQyXjs821sP1TQptcc1iuMP503vMH9Tz31FFu3bmXTpk2sWLGCc845h61bt1aXgL75\n5ptERkZSWlrK+PHjueiii4iKiqp1jT179vDBBx/wxhtvcMkll7Bo0SKuuuqqNv0cxzsShnLFKRYA\nxyTJLQidkQkTJtTqFXjhhRcYNWoUkyZNIj09nT179tQ7p2/fviQnJwMwduxY9u/f317mHjeIZ+FK\nSW7N89xU6N7wtx1BOBFpzANoL4KDg6ufr1ixguXLl7N69WqCgoKYOnWq216CgICA6ucWi4XS0tJ2\nsfV4QjwLV4qza55L3kIQOgWhoaEUFha63Zefn0+3bt0ICgpi586drFmzpp2tO3EQz8KVkhxAQWCE\niIUgdBKioqKYPHkyI0aMIDAwkO7du1fvmzVrFq+++iojR45k8ODBTJo0qQMtPb4RsXClJAcCu0HU\nABELQehEvP/++263BwQEsGzZMrf7nHmJ6Ohotm7dWr39f/7nf9rcvhMBCUO5UpIDwdEQ2U96LQRB\nEFwQsXClJAeCooxYFGRAhSTBBEEQQMSiNiW5NWIBcMyjpWkFQRCOe0QsXCnJgaDIGrGQvIUgCAIg\nYlGD1i5hKEfDT25Kx9okCILQSfCqWCilZimldiml9iqlHnCz/zal1Bal1Cal1E9KqWGO7X2UUqWO\n7ZuUUq96004AbIVgrzBiERRpqqLEsxAEQQC8KBZKKQswHzgLGAZc7hQDF97XWidprZOBpwHX2eAp\nWutkx89t3rKzGueojyDHTJnIfiIWgiB4zIoVKzj33HMBWLJkCU899ZTb40JCQhq9Tt0pu4cOHeLi\niy9uO0NbiDc9iwnAXq11qta6HPgQON/1AK2160SyYEB70Z7GEbEQhC5LZWVlR5tQi9mzZ/PAA/WC\nKR5RVyx69erFxx9/3FamtRhvikUckO7yOsOxrRZKqTuVUikYz+J3Lrv6KqU2KqV+UEqd6kU7DdVi\nEW0eI/tBfgZU2rz+1oIgNM27777LyJEjGTVqFFdffTXXXXcd8+bNY9q0adx///3k5uYyZ84cRo4c\nyaRJk9i8eTMAP/zwA8nJySQnJzN69GgKCwvJzMxkypQpJCcnM2LECH788cd67zdx4kS2bdtW/Xrq\n1KmsX7+etWvXcvLJJzN69GhOPvlkdu3aVe/ct99+m7vuuguAffv2cdJJJzF+/HgeeeSR6mOKioqY\nMWMGY8aMISkpic8++wyoPZL9D3/4A/v372fEiBGAWQ/9+uuvJykpidGjR/P9999Xv9+FF17IrFmz\nGDhwIPfdd18b/dZr8GYHt7sFX+t5Dlrr+cB8pdQVwMPAtUAmkKi1zlFKjQU+VUoNr+OJoJS6BbgF\nIDExsXXWVotFpHmM7AfabspnYwa17tqCcLyw7AE4vKVtr9kjCc5yH7Jxsm3bNp544glWrVpFdHQ0\nubm5zJs3j927d7N8+XIsFgt33303o0eP5tNPP+W7777jmmuuYdOmTTz77LPMnz+fyZMnU1RUhNVq\n5fXXX+fMM8/koYceoqqqipKSknrvedlll7FgwQIee+wxMjMzOXToEGPHjqWgoICVK1fi6+vL8uXL\nefDBB1m0aFGDtt9zzz3cfvvtXHPNNcyfP796u9VqZfHixYSFhZGdnc2kSZOYPXt2rZHsQK0Juc7z\nt2zZws6dOznjjDPYvXs3AJs2bWLjxo0EBAQwePBg7r77bhISEjz+MzSFNz2LDMDV0njgUCPHfwjM\nAdBa27TWOY7n64EUoN4dW2v9utZ6nNZ6XExMTOusdReGAglFCUIn4LvvvuPiiy8mOtp4/pGR5kvd\n3LlzsVgsAPz0009cffXVAEyfPp2cnBzy8/OZPHky8+bN44UXXiAvLw9fX1/Gjx/PW2+9xaOPPsqW\nLVsIDQ2t956XXHIJCxcuBGDBggXMnTsXMMML586dy4gRI/j9739fy/twx6pVq7j88ssBqu0D0Frz\n4IMPMnLkSGbOnMnBgwc5cuRIo9dy/YxDhgyhd+/e1WIxY8YMwsPDsVqtDBs2jAMH2rZPzJuexa/A\nQKVUX+AgcBlwhesBSqmBWmvn8PlzgD2O7TFArta6SinVDxgIePeuXZIDPn4Q4PhHE9nfPIpYCEIN\nTXgA3kJrjVL1gxWu48q1rp/yVErxwAMPcM4557B06VImTZrE8uXLmTJlCitXruSLL77g6quv5g9/\n+AOhoaE89thjAPzzn/9k3LhxREVFsXnzZj766CNee+01AB555BGmTZvG4sWL2b9/P1OnTm3Sfne2\nv/fee2RlZbF+/Xr8/Pzo06eP2/HqdX8PDVF3DHtb53G85llorSuBu4CvgB3AAq31NqXU40qp2Y7D\n7lJKbVNKbQLmYUJQAFOAzUqp34CPgdu01rl4E2ePhfOPGhQJAeEiFoLQCZgxYwYLFiwgJ8dEAHJz\n698OpkyZwnvvvQeYyqTo6GjCwsJISUkhKSmJ+++/n3HjxrFz504OHDhAbGwsN998MzfeeCMbNmzg\nggsuYNOmTWzatIlx48YBJhT19NNPk5+fT1JSEmA8i7g4k359++23m7R98uTJfPjhhwDV9jmvExsb\ni5+fH99//321J9DYSHbXz7h7927S0tIYPHhwkza0BV6dOqu1XgosrbPtjy7P72ngvEVAw0FAb+Ac\n9eFEKYjqD4c3t6sZgiDUZ/jw4Tz00EOcdtppWCwWRo8eXe+YRx99lOuvv56RI0cSFBTEO++8A8A/\n/vEPvv/+eywWC8OGDeOss87iww8/5Jlnnqle0/vdd991+74XX3wx99xzT63E9H333ce1117Lc889\nx/Tp05u0/fnnn+eKK67g+eef56KLLqrefuWVV3Leeecxbtw4kpOTGTJkCFB7JPtZZ53FnXfeWX3O\nHXfcwW233UZSUhK+vr68/fbbtTwKb6Iac2u6EuPGjdPr1q1r+QXenAU+vnDdf2u2/fA0fP8E3LsV\nItouUSQIXYkdO3YwdOjQjjZDaAPc/S2VUuu11uOaOlfGfThxhqFcSXI0wmxtXydHEAShsyFi4cS5\nloUrkf0gbhxs6fiGGEEQhI5ExALAXlU/Z+Fk5CVwZAsc3dH+dglCJ+F4CVefyLT2byhiAVCaB2j3\nYjH8AlAW2LKw3c0ShM6A1WolJydHBKMLo7UmJycHq9Xa4mvIGtxQvyHPlZBY6DfViMX0R2pKawXh\nBCE+Pp6MjAyysrI62hShFVitVuLj41t8vogF1B/1UZekufDpbZC+FhIntp9dgtAJ8PPzo2/fvh1t\nhtDBSBgKGvcsAIaeC75WCUUJgnDCImIBTYtFQCgMmAl7vmo/mwRBEDoRIhbQtFgAxAyB/INQ1bnm\n5guCILQHIhZgxMIvGPwCGz4mPB50FRQdbj+7BEEQOgkiFtBwj4Ur4Y5xH/kZ3rdHEAShkyFiAY5R\nHw1UQjkJd5SciVgIgnACImIBUJLtgWfhWBE2P73x4wRBEI5DRCzA/RDBugSEgjVCPAtBEE5IRCzA\ns5wFmDHlIhaCIJyAiFhUloOtwDOxCE+APAlDCYJw4iFiUepYnrGpBDeYJLd4FoIgnIDIbKjgWLhv\nH1j8mz42PB5s+VCWD9Zw79smCILQSRDPwsfHeBUBIU0fW10+e9C7NgmCIHQyRCyaQ3iieZRQlCAI\nJxgiFs2h2rOQJLcgCCcWIhbNIaQ7+PiJZyEIwgmHiEVz8PGBsF7iWQiCcMIhYtFcwqUxTxCEEw8R\ni+YivRaCIJyAiFg0l4gEKDgkiyAJgnBCIWLRXGQRJEEQTkBELJqLrGshCMIJiIhFc5EV8wRBOAER\nsWguYY5FkPLSOtYOQRCEdkTEorkEhEBgN/EsBEE4oRCxaAlSPisIwglGoyPKlVJW4FzgVKAXUAps\nBb7QWm/zvnmdlPBEOLa/o60QBEFoNxr0LJRSjwKrgJOAX4DXgAVAJfCUUuobpdTI9jCy0yGehSAI\nJxiNeRa/aq0fbWDfc0qpWCCx7U3qAsgiSIIgnGA06Florb8AUEqNaGD/Ua31usYurpSapZTapZTa\nq5R6wM3+25RSW5RSm5RSPymlhrns+1/HebuUUmd6/pHaAVkESRCEEwxPEtyvKqXWKqXuUEpFeHph\npZQFmA+cBQwDLncVAwfva62TtNbJwNPAc45zhwGXAcOBWcDLjut1Dqp7LWT6rCAIJwZNioXW+hTg\nSiABWKeUel8pdboH154A7NVap2qty4EPgfPrXLvA5WUwoB3Pzwc+1FrbtNb7gL2O63UOZBEkQRBO\nMBqthnKitd6jlHoYWAe8AIxWSingQa31Jw2cFge43k0zgIl1D1JK3QnMA/yB6S7nrqlzbpwntrYL\nsgiSIAgnGE16FkqpkUqpvwM7MDfz87TWQx3P/97YqW626XobtJ6vte4P3A883JxzlVK3KKXWKaXW\nZWVlNfFJ2pDqRZBELARBODHwJGfxErABGKW1vlNrvQFAa32Impu7OzIwoSsn8cChRo7/EJjTnHO1\n1q9rrcdprcfFxMQ0+UHalIhEEQtBEE4YPBGLszGJ6FIApZSPUioIQGv970bO+xUYqJTqq5TyxySs\nl7geoJQa6PLyHGCP4/kS4DKlVIBSqi8wEFjryQdqN6TXQhCEEwhPchbLgZlAkeN1EPA1cHJjJ2mt\nK5VSdwFfARbgTa31NqXU48A6rfUS4C6l1EygAjgGXOs4d5tSagGwHdMEeKfWuqrZn86bhMfXLIJk\ncfwaty+BihIYdVnH2iYIgtDGeCIWVq21UyjQWhc5PYum0FovBZbW2fZHl+f3NHLuE8ATnrxPh+Bc\nBKkw06yeB/Ddn6E0D0ZeCspd2kUQBKFr4kkYqlgpNcb5Qik1FjMj6sSm7iJIxdmQvRuKj0JuasfZ\nJQiC4AU88SzuBRYqpZwJ5p7Apd4zqYtQdxGkNJdK37TVENW//W0SBEHwEk2Khdb6V6XUEGAwpqR1\np9a6wuuWdXbqNualrQaLP/gFmeejr+o42wRBENoYj5ryMEIxDLBiGvLQWr/rPbO6AP7BEBhZ27OI\nGwvWiNpehiAIwnGAJ015fwJedPxMw8xwmu1lu7oGzvLZ8hLI3ASJJ0HvkyBnLxQd7WjrBEEQ2gxP\nEtwXAzOAw1rr64FRQIBXreoqhCeYMNTBdWCvNGKReJLZJ96FIAjHEZ6IRanW2g5UKqXCgKNAP++a\n1UVwehZpawAFCeOh5yjwtYpYCIJwXOFJzmKdYzT5G8B6THNe5+qm7ijC48FWALu/gthhENjNbI8b\na5LcgiAIxwmNehaOybJ/0Vrnaa1fBU4HrnWEowRnRdTBdSZX4STxJMj8DWxF7s8TBEHoYjQqFlpr\nDXzq8nq/1nqz163qKoS7zDpMrCMWusqIiCAIwnGAJzmLNUqp8V63pCvi9CwAEifVPE8YDyjJWwiC\ncNzgSc5iGnCrUuoAUIxpzNNa65Fetawr4FwEKbRnbeGwhkOPEXDg546zTRAEoQ3xRCzO8roVXRUf\nH4gdCnFj6u9LmAi/fQRay1BBQRC6PJ6IRb0V6gQXrvvCjPmoS/QgKC+E4iwIiW1/uwRBENoQT8Ti\nC4xgKMy4j77ALmC4F+3qOljD3G+PdLSi5KaKWAiC0OXxZJBgkutrx7jyW71m0fGCq1i4Jr8FQRC6\nIJ5UQ9XCsQa3VEc1RUQiKAvkpHS0JYIgCK2mSc9CKTXP5aUPMAbI8ppFxwsWPyMYshCSIAjHAZ7k\nLEJdnldichiLvGPOcUZkPxELQRCOCzzJWTzWHoYcl0T2g4x1Uj4rCEKXx5P1LL5xDBJ0vu6mlPrK\nu2YdJ0T2A1s+lOR2tCWCIAitwpMEd4zWOs/5Qmt9DDhua0FfWZHC2n1tdHOvroiSJLcgCF0bT8Si\nSimV6HyhlOrNcdyo99J3e1i8MaNtLhbV3zxK3kIQhC6OJwnuh4CflFI/OF5PAW7xnkkdi63STl5J\nRdtcLCIRlI+IhSAIXR5PEtxfOhrxJmG6uH+vtc72umUdQJVdU2nXbScWvgFmwKCIhSAIXRxPEtwX\nABVa6/9qrT/HLK86x/umtT/llXYA8krbSCxAymcFQTgu8CRn8Setdb7zhSPZ/SfvmdRx2CqrAMgv\nKW+7i0b2ky5uQRC6PJ6IhbtjPMl1dDmcnkV+m3oW/aEsT8pnBUHo0ngiFuuUUs8ppforpfoppf4O\nrPe2YR2BzSEWxeVV1cLRaqrLZ/e1zfUEQRA6AE/E4m6gHPgIWAiUAXd606iOwhmGgjb0LlynzwqC\nIHRRPKmGKgYeaAdbOhybizeRX1pOTGhA6y/arQ+gRCwEQejSeDJ1Nga4D7PYkdW5XWs93Yt2dQiu\nYtFm5bN+VgiLky5uQRC6NJ6Eod4DdmJWyHsM2A/86kWbOoxyb4gFQJSUzwqC0LXxRCyitNb/wvRa\n/KC1vgHToHfcUcuzaOtei5wUKC8xP5VtWJorCILQDnhSAuu8a2Yqpc4BDgHx3jOp47BV1CS489qy\n1yJqAJTmwpM9zWtLANyxumZ2lCAIQifHE7H4P6VUOPD/gBeBMOD3XrWqgyivck1wt6FnkXylmRFV\nVQGFh+GXVyB7t4iFIAhdBk+qof7reJoPTGvOxZVSs4DnAQvwT631U3X2zwNuwqzAlwXcoLU+4NhX\nBWxxHJqmtZ7dnPduCbYKL+UsgiLhJEe1cV66EYuio213fUEQBC/jSc6iGqXUhmYcawHmA2cBw4DL\nlVLD6hy2ERintR4JfAw87bKvVGud7PjxulBAjWdh9fNpW8/CleAY81gsy5gLgtB1aJZYYKbOesoE\nYK/WOlVrXQ58CJzveoDW+nutdYnj5Ro6OBfizFnEhlrbNsHtip8VAsJELARB6FI0Vyy+aMaxcUC6\ny+sMx7aGuBFY5vLaqpRap5Ra09CUW6XULY5j1mVltf7m6/QsYkMD2naYYF2CYyQMJQhCl6JZYqG1\nfrgZh7vzQtyusKeUugoYBzzjsjlRaz0OuAL4h1KqXjZYa/261nqc1npcTExMM0xzjzNn0T3Mi54F\nGLEQz0IQhC5Ecz0LAJRSW5o+igwgweV1PKbstu61ZmJW45uttbY5t2utDzkeU4EVwOiW2NocbJV2\nfBREhfi3bYK7LiEiFoIgdC0arIZSSl3Y0C6ghwfX/hUYqJTqCxwELsN4Ca7vMRp4DZiltT7qsr0b\nUKK1timlooHJ1E5+e4XyKjv+vj5EBPlTUFZBlV1j8WlOmsZDgmPgwM9tf11BEAQv0Vjp7EeYUR/u\nQkdWN9tqobWuVErdBXyFKZ19U2u9TSn1OLBOa70EE3YKARYqpaCmRHYo8JpSyo7xfp7SWm9vxudq\nEbaKKgJ8LUQE+qE1FJZVEBHk3/ZvFBxr1reoqgTLcbk0iCAIxxmN3ak2A89qrbfW3eEIHTWJ1nop\nsLTOtj+6PHd7Ha31z0CSJ+/RltR4Fn6A6bXwiliExAAaSrIh1BMnTRAEoWNpLGdxL1DQwL4LvGBL\nh2OrsBPgKhbSayEIggA04llorX9sZN8675jTsdgqjViEBxpvok3nQ7kSHGsepXxWEIQuQoOehVLq\nYaVUZCP7pyulzvWOWR2DrdKOv6+l2rPwWhd3iEMsirO9c31BEIQ2prGcxRbgc6VUGbABM7vJCgwE\nkoHlwJNet7AdsVVWmTBUoJfFIjjaPBaLZyEIQtegsTDUZ8BnSqmBmNLVnpgcxn+AW7TWpe1jYvtR\nXmkS3OGBNQlurxAQZsaUSxhKEIQugidTZ/cAe9rBlg7HVmkn1OqLr8WH0ABf74mFUo4ubglDCYLQ\nNWhRB/fxiklwWwAIC/Qjr9SL86FCYiQMJQhCl0HEwoVyR84CICLIj3xvjvwIjpUwlCAIXYZGxUIp\nZVFKHZer4rnDWToLRiy8P0xQwlCCIHQNGhULrXUVddagOJ5xJrgBIgL93fZZVFbZmf7sCj7bdLB1\nb+YcJqjdDuIVBEHoVHgShlqllHpJKXWqUmqM88frlnUArp5FeJCf29LZzPwyUrOL2ZFZ2Lo3C44F\newWU5bXuOoIgCO2AJ1PsTnY8Pu6yTQPT296cjqW80k6An0lwRwT6kVdSgdYax5BDANJzzcJ+xbbK\n1r2Zc+RHURYEdmvdtQRBELyMJ6Wz09rDkI5Ga42tsgp/S03OotKuKS6vIiSg5td0oK3EIsQ5H+oo\nxAxq3bUEQRC8TJNhKKVUuFLqOefypUqpvymlwtvDuPak0q6xa2oS3I75UHVDUWkOsShqtWfhHPkh\nwwQFQej8eJKzeBMoBC5x/BQAb3nTqI6gvNIsqervkrOA+sMEnWJRXN6GYShBEIROjic5i/5a64tc\nXj+mlNrkLYM6CptDLGo8C8d8qDq9FunVnkVV694wKBKUj3gWgiB0CTzxLEqVUqc4XyilJgPH5Vwo\noCbB7Vj0qG6vxYGcNspZ+FggKEq6uAVB6BJ44lncBrzrkqc4BlzrPZM6Blul8RRcE9xQe5hgfklF\ndQ6j1WIBji5u8SwEQej8NCoWSikfYLDWepRSKgxAa93Q6nldmuowlJ8jZ+GcPOsyHyr9mPEqeoZb\nW5/ghprGPEEQhE5OUx3cduAux/OC41UowCXB7fAsrH4WrH4+tXIWzuT20J5hFNsq0a3tvg6WYYKC\nIHQNPMlZfKOU+h+lVIJSKtL543XL2hlnGMqZswDjXbiGoZz5iqE9Q7FrKKuwt+5Ng2Pdz4fK2gXv\nzgFbUeuuLwiC0EZ4krO4wfF4p8s2DfRre3M6DlsdzwJMr0WuS+lsWm4JUcH+dA+zAqbXItDfQosJ\niYHyIigvAf+gmu07v4DU7+HINkic2PLrC4IgtBFNTZ31Aa7SWvet83NcCQXUz1kADO8VxpqUnOr8\nRHpuCQmRQQT7G41ts5EfdUNRR7ebx/z01l1fEAShjfAkZ/FsO9nSodgqavdZAFxzch8KbZUsWp8B\nGM8iMTKIEKsRi7br4q4TijqyzTyKWAiC0EnwJGfxtVLqIuU6Te84pLyqvlgkJ0SQnBDBOz/vp7zS\nzsG8UiMWAW3lWUSbR9dFkCrLIXu3eZ6f0brrC4Jw3LMx7Rjf7Tzi9ffxRCzmAQuBcqVUgVKqUCl1\n3FVF2SocCW7f2jmI6yf3ITW7mI9+TaPKrkmMCiLYKRatHfkR1ss8unoQOXvA7riuiIUgnBBkF9nI\nOFZClb15FZYfr8/g0tfW8MxXu7E389zm4snU2VCvWtBJcHoW/r619fOsET35v9Ad/H35HgCHZ2EE\npbCstZNnu5u8Rebmmm1HHPmKiN4iFoLQDNJySvjfxZvx9fFhYGwIA7uHcPqwHkQG+zd5bl5JOV9u\nPczA7qGM7d2+SwYs25LJvR9twlZpx9/iQ3xkINEhAQT6WQjyt9AzPJDeUUEkRgaREBlEfLdAfH0U\nTy7dyZur9jF5QBQvXT4GHx/vBn+aFAtH+OlKoK/W+s9KqQSgp9Z6rVcta2fc5SzAiMdVE3vz9+Um\nNJQYGYQzIFfc2vlQSkHPUZD5W822I1vBxw/6T4Nti1t3fUE4QUjPLeHyN9ZQWFZBfLcg1qTmYKu0\n8+zXu/n7JcmcMjC63jlaazakHeO9X9L47+ZMyivtKAW3nNqPeWcMqhdlaGu01ry+MpW/LNvJmMQI\n5o5L4EBOCQdyisktLievpJyMY5Ws2JVFaUXte02o1ZfCskpumNyXB88egq/FkyBR6/CkdPZlwI5Z\n7OjPQBEwHxjvRbvanZpBgvX/gVwxMZGXvt+DQtE9zFodfmqTkR89R0Hq81BRBn5WUwkVPQi69YWy\nfCgrAGtY699HEI5TDuWVcsU/jVC8f/MkRsSFU2XXbD2Yz/9b+BtXv/kLt07pz70zB2L1s1Bsq+Sz\nTYf495oD7MgsICTAl0vGxXPRmHgWrMvgtZWprNyTzT8uTWZwj7YLrHy/6yh//u92QgJ8SYgMwlZh\nZ/mOI5w7sifPzh2F1c+9OGmtySqykZZTQvqxEtJzS8k4VsLkAdGcnxzXZvY1hSdiMVFrPUYptRFA\na31MKdW0X9fFqDui3JWY0AAuGZfArsOFWHxUdelsm4z86DnK5CiOboO4sSYM1fskCI83+wsOuheL\nilJ4/xI4uLFm27Dz4fyXwBu1CGX58Pq02sl4J9ZwuGVFzYJOgtBO5BTZuPyNNeQVV/CfmyYyIs6M\nsLP4KEYlRPD5Xafw+H+38eoPKbz6QwoWH4XCrF8ztGcYT16QxPnJvarzkKMTuzFzaCz3L9rMeS/+\nxLwzBnHdnKr0AAAgAElEQVTzqf2wtDLE89GvaTy4eCv9ooOJCPJn+6ECsgpt3DVtAPNOH9RoCEkp\nRWyoldhQK+P6dFw/tCdiUaGUsmAa8VBKxWA8jeMKW2UVvj6qwX8Ufz5/RPVzi48i0PENpdX0TDaP\nmb9BZH8oyIDuwyEi0WzPS4fYofXPW/4o7FsJY68HvyAoPASb/gNxY2D8ja23qy7bl0BuCoy5Bvxd\nvm0VHYGtH5vwWcgJsaii0Il4culODuWV8uEtJzEqIaLe/kB/C3+5cCRnDu/Blox8yiqrsGuYOTSW\nMYndcFfkOWNod766dwoPLd7KU8t28s32Izx+/nAGdQ/Fz+JDZZWdH3Zn8cHaNDak5XHjKX25dUo/\nfB37vt+VxY7MAnpFBJLQLZBVKTm88O0epgyK4eUrx9RaebMr4YnVLwCLgVil1BPAxcDDXrWqAyiv\ntLv1KpzUVf7gAN/WV0OBEQVrhBGLmCFmW+zwGs/CXa/F3uXwy6sw8TY4669mm91uvv1/9RD0nQLR\nA1tvmytbFkBkPzjvhdqey7EDRiwkGS+44UhBGaFWX4L82/4GuXZfLos2ZHD71P5NJqWnDo5l6uBY\nj68dFRLAK1eN4dNNB/njZ9s454Wf8FHQKyKQ8ko7RwttxIQGMKRHKM98tYtlWzM5ZUAMn2zI4Gih\nrd71LhoTz1MXJeHXDrkFb+FJNdR7Sqn1wAxAAXO01ju8blk7Y6u010tuN0ZIgKX1CyBBTZL70Cbo\n7vBeug8zlVI+vvVvwsXZ8OkdEDMUZj5as93HB85/GV45GRbdBDd+A75tFC0syIR9P8Jp99UPcYX1\nApSIhVCP7CIbM//2A1Eh/rx+zTgGda/xSLMKbaQfK6F3ZBCRwf5uv+E3RkWVnUc+3UpcRCB3Tx/Q\n1qYDJvxzweh4JvePZsXuLDJyS0jLLaG8ys7sUXHMGBqLn8WHpVsy+eNnW3l9ZQpTB8dy2fgEThkY\nzZECG+m5Jdi15rRBMc3+jJ0Nj+Rea70T2OllWzqU8kp7s6ofggN82yYMBUYsfnnVeBcB4RAWZ27K\nYb3q34Q/vwdKj8FVn4BfYO19YT1h9gvw0VXwyU1GfJQPjLwUIhJqH3t4C+xa5t6e0B6QfJURIIBt\nnwAakubWP9biB6E93XtA2z+DhEkQ2t2jXwM7vzAht8g602SObIed/615HTUARlzo2TXbkmP7ze9t\n6HkNH5O123xuHDXvEb1h1KXtYV2n44Vv91BSUUWArZIL5q/iuUuTSU6I4JUVKXywNq26qCTM6kvf\nmBD6RgXRJzqYIT3COGNY90bj+O/8vJ9dRwp57eqxXvFaXIkNs3LJuIQG95+d1JOpg2Moq7DXKtPt\nG+1L3+hgr9rWnnTN4JkXsFVWNRqGqktwgG/bJLgBeiVDVTns+Nx4Fc5vIOEJtcXi2H5z05z6IPQY\n4fZSDD0PJt0Ja+Y7blrAlo9NAtrPDECkIBPeOc+ITkOU5sHk35nnmxeY3EpDoa3w+PpiUZoHC66B\n0VebpHtTVFXCgmuhW2+4dSX4O/6TFWXBu7Pdr/vR3oLx9cPmb3TD1+4HPJbkGlsLM2tv7zvFCPlx\nhta6wW/L+7KLef+XNC4bn8Bd0wdw27/Xc+u/1+Pv60OVXXPh6DhOH9adjGOl7MsuJjW7iF/3H+Oz\n3w6hNZw2KIa/X5pMZLA/e48W8doPKWw5mE9cRCAJkUEsXJfO9CGxnDHMwy8iXibI35eg467spzZe\nFQul1CzgecAC/FNr/VSd/fOAm4BKIAu4QWt9wLHvWmpyI/+ntX7Hm7Y2Pwzly5GCsla/7+4jhfTv\nPgoLQFmeSW47CY+HA6trXjufDz238YvOehLO+LN5nvIdvHexSYif9ZTJbXx6O1Ta4M5fIap/7XO1\nhoXXwrePQ7+pxnvJ3ARnPtnw+0UkwKGNtbflpprH7UvgnL+Bb0DjNucdAHsF5Ow1eZfz/mFsWXKX\nKR++bZXxOuyV8NZZ8N97IWFCTW7H25Tmwe6vzPNl98HN35mlcZ1obby+4mwjzD1GmtDiP6dD2uqO\n8YS8zBVv/ALAi1eMJjqk9t/32a924e/rwz0zBxIbauWjW0/iqWU7sVVWcftpA0iMCnJ3Scoqqli4\nPoM/f76dc174kZHx4Xy9/QgBvj5M7BvFwbxSVqfm4KMUj543vMuHdroSXhMLRwXVfOB0IAP4VSm1\nRGu93eWwjcA4rXWJUup24GngUsd6GX8CxmH8+fWOcxv5Ktw6mkpw16UtwlAH80qZ9Y+VPDlnOJf5\nh0J5IcQOqzkgPN6UztqrzI0pbbUpU41xUx1VF+eNbODpMOFW+OUV8zxrlxl/fu7fIWaQ+3PPewFe\nOcnkPgadCSgY3sjNLjwedvzXCJEzdOUUC1s+7Pm68dANQE6KeexzKqx/CwaeYb6h7/4SZv21xpPy\nscCFb8Crp8Li2+CaJTXv6U12fG68v0l3wJqXYeO/Yex1Nfs3vQc7lsDMx6DXaLOt5yjwC4a0Nced\nWKTnlrA6NQeA819axT+vHcfQnqbEe2PaMb7Yksk9M4xQgFlM7NHZwxu8nhOrn4WrJ/VmdEIEd7y3\ngZ/35nDn1AFcP7kPUQ5B0lpTadddOlncFfGmZzEB2Ku1TgVQSn0InA9Ui4XW+nuX49cAVzmenwl8\no7XOdZz7DTAL+MBbxnZEgnv3kULsGn5KyeWyniPhwKo6nkUC6Cpz0wyPNzedhEnNvzme/pgps118\nq/mWPvhsU3LbEMFRMOcV+M+FkL0L+p7WeBglPAGqbFCSDSGOihOnWARFwZaFHojFXvN44eumf+Sz\nO00vSf8ZMPHW2sdG9Tde0pK7YfWLMPme+terqjSeCpiOeEsT/9TtVUYMnNTNBzmrwc580ngM3z4O\ng84yPTB56bDsfiN0J99dc47FF+LHQdrPbuyrqJ4B1mA4x8fX5ITagspy82+pLpaAFont8h1HUNh5\n6ZLh/HXZDubO/4EhcVEkRgWx/VAB0SH+3Dyln/G4Gvr2X1Ha4PVHxPrzzd3jsdtxrBljrz5eAX5w\nHBbwtwZVE2b2Et4UizjANZCdATS2ks+NgDPj6u5cr7YqNjvB7d96z2J/djFgSgD12FGoAz9D7FB+\n2pPNPR9u5Ps5PQgDk7fwDTQ37lGXNf+N/ALhojfgjemmTHf2i0037g2YARNvNx6Ju8S2K85QUF56\nbbEIi4Mh58L6t01ZrzW84Wvk7DX7Q3vCRf+C16aYvMWcl93bOvpqExb69s/G63FN4JfmwYtjjXiB\nqSy74cv6iXMnhUfgX6ebUJiTUVfUvHd1Ndj95vXZTxv7/ubimVnD4YJXa4emAHqfDD/8tfbnz02F\nVyZDhVl5scG/hF8wXP6+CQe2ht8+hM/uqhFPV2KHmd9NY38bN/y4LY2lQY8xdMkezgHK/a08W/4Q\nS/YO50hhGU9dmESIvRBenmU82jP+r+ZkreHz38GGdxt9jyYCl4IrcePg5m+9+hbeFAt3/wfcjkVU\nSl2FCTmd1pxzlVK3ALcAJCYmtsxKB7bKKiKakaEKDvCltKKKKrtucXenUyyOFtpIG3YrvftPB2s4\nb/+8jpzicvZXRjMSjFiUGJefxJNa9F70SDIhm6DImtHoTXH6Y9BzJIy4uPHjXHtC4sea57mp5uac\nNBfWvmbCVKOvbPgauSmmykkpiBkM134O/iGmMssdSsGZT5iE/9aP4ZTf1+zb8bkRisn3mJvgT8/D\nJ7fC9cvqexhaw2d3mObC6Q+bb/NZu+G396HPZBh9FWxdhKkGc/weeiSZarTMTTXX6T/Dff4kcRJo\nO2T8CgNmmm2/fWS+JU9/mE83H2VnphniPLFfFFMHxdRo48b30ItvY8lJC/nzd0ew+vkwIDaE/jEh\n9IsJpl90CP1jgokJDWg4dp+TAv+dZ0JidXNdFaWw8hnjFV3wqvvz3VBYVsHU9JcYatljfu/WcPx/\n+4gHS17gwd+tpiooGosCPr4Bsnaan96nwOBZ5gKbFxihGHmp+4ZTofmEer+AwptikQG41pvFA4fq\nHqSUmgk8BJymtba5nDu1zrkr6p6rtX4deB1g3LhxrZrP25IEN5gx5WHWloUK9uWUEB7oR35pBWuO\n+NB7/OnkFNlYscuM1EivinSIRboRC0uA6dBuKb2bKTS+AZB8RdPHVYuFS+VWbqoJd8WPg259TBin\nMbHISTHfwp0kTGj6fbv1gfgJsHlhbbFwhoxmPmZEJaI3LLoRfvwbTL2/9jXWvmGaHM9+FibcbLbZ\nq8zvfNn9xqYtC+tXg/WfZn6aIm4cKIspThgw04jTlgXQdwrfd7+WezN+5fczB5FVVMb1a9L4XeJA\n5p1uPJbcHqcQ9p9Z+C2bR++ejxLXLYiUrCLWpObUWv892N9C35hg+kQFM21wLBeNdfw9qirgk1uM\nQF7yTsPFAD/81eSIPMyr7Fj5CddYvubw0BvoMfNRs3HQLHjtNPjsTixXfASbPzIl16fdDzuXmrDi\nHauNQC39HxNOnfNKfU9M6LR4Uyx+BQYqpfoCB4HLgFp3HqXUaOA1YJbW2nXo0FfAk0opZ1vmGcD/\netHWZie4navlFdtaLhb7s4s5ZUA0v+zL4ZfUXC4dn8hnmw5R6ZhLf7DEx4SN8jNMnDxuTNNVRR2B\nNcJ4AU6xKCswpa6R/czNOmmuuVEXHnbvKVSUmptzZP/6+5pi5CXm5nNkm8n3uGsgTLrYhKx++Cv0\nnw4JjhmYR3fCN4+YG+X4m2qu6WMx37RfORnev9QsRtVYNVhjBISYb/Vpa8zrQxsgNxXbSffy8OKt\n9I8J5rap/fDz8aGiUvPCt3tYtiUTDRzOL+Na+yX8wfI+syal4jPmagDsdk1mQRmpWUWkZhWzL9v8\nbEzL47+bM9l9tJAHZg1BrXwGDq5j56kv8O/vjvH702PqVS0x5T7Y+63n1WVFWQz55QF2k0i/OU/U\nbI8dCqc/Dl/eb/I5a98wXvBp98PwC+D1qUYwbIVGMC98TYSii+E1sdBaVyql7sLc+C3Am1rrbUqp\nx4F1WuslwDNACLDQ4Uanaa1na61zlVJ/xggOwOPOZLe3sLWgKQ9aPnm2vNJOxrESZo/qhUbzyz7z\n8RZtyCApLpy9R4s4nG8zyePs3Sbk4Zo87Uwo5egJcaSZnMltZ44gaa4Jd2z9BE66o/75zuPrlvF6\nwrA5xgPYstCIRXXIqE6e5ZxnzQ374xscFV5A6gojcufPr58XCY83FWMf30CT1WBNkXgSrPuXKVfe\nvBAsAcw/PIyDeUf56JZJ1f/u/nJhEt3DrezMLMDP4sOYxAguOOUp+DIdn2X3O8JeCh9MAi8OOBXM\n/67uYO8Oa/flsGtVIb/stjIx7wvWhZ/J3G+igTR+3JPNOzdMqN0oZvE1RQWvngofXGa+8TeCPriB\ngKpCPu37LPcF1Cl/nXirqXz76TkICIMLHILgFJJl95njLnjNeIVCl8KrfRZa66XA0jrb/ujyfGYj\n574JvOk962pjq7QT4Ne8aiigxRVR6cdKsGvoEx1MdIg/S7cc5tsdR9h2qIBHzxvGu6sPmD6O8Hjz\nH1BXtTxf0R6Ex9d4FnXFImaw6TvYstC9WDjLZqNaMLYhJMZ4C1sWwfQ/ug8ZgcldXPRP+ORmh6Bg\nBjBe+FpNUr4uIy6C9LVQXty6prrESaZJ8uAG2LqI4t4zmL86i0vHJTCxX1T1YT4+qjoEVYsLXjUe\nztZPGn0bH0wFyShrFaW5VazTA7k56xLunNafUwfGcMd7G7jw5VU8Ons4/WNCiIsIJCLIDxXVH85/\nEb58sOZ30wCV2odHKq5jymg3/xaVMkUBH11lvth0612zb8ItZkKBf4jJVQhdDungdmCrrMK/GXXb\nzjHlLfUsnMntvtFB1eMK/rRkG34WxezkOL7adoTDBWXQ21E+i/Isjt9RhMebEAu4iEXfmv0jLzEd\n0Dkp9T0IZ9lsSzwLMF7E4lvM1N3GGggTJ8K9m93vawjnoMbW4BT5H/4KxUf52nIqAPPOaKDPpS7h\n8XD7Ko8OVUAgsPiXNJbvOMIHZw6u7n/45PaTue6ttdzzYU1i3qzEZqVXRDzDhn3K72YMrPaa3fHs\nsh0s/mkfDw9qYBx9aA+4abkbwxxCInRZRCwcNNezcP6HaunIj30OsegTFUy3IH/CrL5kHCvljGHd\niQz2p0e4lbX7cmtiyLHDILB9l3tsFuHxJglfXgK5+yCkR83IDjDf0r9+xHzzn/pA7XNzUkx5a0AL\nF5oZcrYpLV72AK0OGXmDkBjjNaV+jw4I5ZnUPkwbHEv3MO/VxV8xMZErJtauEOwTHcyX905hz5Ei\nDuaVcDCvjIPHSsnML+VQXilv/GgW/XnjmrHEdwvCbtcsXJ/OKytS8LX40CPMyo7MAib2jWpxnk7o\nuohYYJqiyivtBDTDswhpZc5if04xoVbf6ombE/pGsnzH0epKlu5hVo4WlmEPi8cHTCijMxPuKHwr\nOOgog63jJYT1gj6nmLJJZ7+CE2fZbEsJCDWCsXVR0w2EHUXiJMjZy6Gep3Nop+bx8Q0PpvMmVj8L\nSfHhJMXX76v4YXcWd72/gTnzV/Hg2UP5z5oDbEjLIzkhgh5hVg4XlBHob+HyCa0rUxe6JiIWQHmV\nY0nVBpY1dEdrE9z7s0voGx1cXR9/fnIcOcXlTHPM3O8RFkBFlaYgMJ4IMDX/nRnXXovcVNOIVZek\nuaYZ69DG2iXAOXtN6WVrSLrEiMXIS1p3HW/R+xTY+B8+KJ1EbGgAUwd3vlUFTxsUw+I7JnPTO78y\nb8FvRAX78+zcUVw0Jk5mMAkiFuCypGoLPIuWJrj3ZRfXWrDlvFG9OG9Ur+rXPcJNiCLDOpiIa5aY\nURKdGadYZO0yDW7uuqWHzTZlrls+rhGL0jxTZtsazwJMhdPVi41n4QFFtkoeWryFu6cPZEBsSOve\n2xOS5pJrieLl98u5fWo8vp10rtGA2BA+vXMyn2/OZPbIXoQHSbhJMHTOf7HtjHOufnNyFlY/H3wU\nFNncjFBogrKKKg7ll9KnkVn3znj24QIb9DutfYbleUhFlZ3M/DpzfcJ6mbUz9v1oXrsTi8Bupqdh\n6yLT+AYmBAWtFwulTFWUh7X7/159gM82HeL5b/e07n09xeLL+1l9sWvV6NoInYGIIH+untRbhEKo\nRee5A3Ug1WLRjKY8pZRj8mzzPYv03BK0NpVQDdEz3AyyO9wGY9DbmrdX7ee0p1dUJ+mBmkWQ9v9k\nXjc0hylpLhQdhv0OUclx9lh4Z7Uzd5RVVPGvn1Lx9VEs3ZLJobyGB9q1FXa75qN16ZzcP4reUcfP\ngjjCiYOIBS5hqGaIBZhQVEuqoVwroRoiOsQfH0WbrJnR1qxKyaa8ys6zX+2qvSM83owkh4bFYtCZ\n4B8KS+8ziyP99HdAtWuT1odr08guKufpi0eitead1fu9/p6rU3NIzy3l0g5KbAtCaxGxwPRYAM3q\n4IaWr2mxP8fZY9GwWPhafIgJDeBwfucSC7tds+HAMUICfPliSyab0vNqdjrzFsGxDZfB+gXCqfNM\n2Chrl+khSb7S6+OVnZRX2nltZSrj+3TjwjHxzBrRgw9+SWu7JXIbYOG6dEKtvpw5vIHBiILQyRGx\noGUJbmj50qr7skuICPJrcsqts1yxM7HnaBEFZZXcN2sw0SH+/GXpDrR2zHB0ikVDXoWTU+fBnb/U\n/MyZ712jXVi8MYPM/DLunGbCXjee0peCskoWbcho4syWU1BWwZfbDjN7VC+szai4E4TOhIgFLUtw\ngxn54fqN9K1V+3jRg4Tp/uziRkNQTrqHWTtdGGr9AbNY4ZSBMdwzYyC/7MtlxS7H+tjOXoumxMIL\nlFVUsWBdeqPiXVll55UVKSTFhXOaowN5TGI3RiVE8Naq/djtrRpc3CBLN2dSVmHnYuc0WEHogohY\nUONZNDsM5V87wf3vNQd4++f9Nd+0G2B/TnGjISgnPcKtnS4Mte5ALtEh/vSOCuKyCYn0jQ7mqWU7\nqbLrDhOLKrvm3g83cd/Hm/n9R5savOkv23qY/Tkl3Dmtf3XfgFKKG0/py77sYr7efrjeOVprimyV\nTf5NG+Pj9RkMiA0hOSGixdcQhI5GxIKanEVrEtzFtkr2ZReTU1zeaOiotLyKzPwyjz2LgrJKSsq9\nG09vDusPHGNs724opfCz+PD70wex60ghq1Nyaiqaug9r/CJtiNaaxz7fxpfbDjNlUAzfbD/CKz+k\nuD3u9ZWp9IsJ5oxhtfMGZ43owYDYEO5ftIW9Rwurt+eVlHPhKz8z4k9fMejhZUx4Yjk3vbOOwjLP\ny6X3ZRez7sAxLh4bL41tQpdGxAKwVTS/dBYcCW7HjXzn4QKcXz63ZOQ3eM6BXEclVCNls056Ohrz\nOot3kVVo40BOSa1mwtOHdifA14flO45A9AC44xez6FE78fKKFN5dfYBbp/TjnevHM3tUL579ehc/\n7M6qddzq1By2HMzn5lP74VNnZUM/iw9vXTceP4sP1775K0cKysgusnHZ62vYdqiA300fwE2n9uO0\nQTGs2HWUG99eR2m5ZyXTH69Px0fBBaO9uiqwIHgdEQtqxn0017NwrYbafqigevvWgw2LhfO4gbFN\nD83rUd2Y1znEYv0Bs+bG2N6R1dsC/S2cOjCa5TuOmFBN7JCm1/duI1buzuKZr3YxJ7kX988aglKK\npy5KYnD3UH73wUbSc0uqj33th1SiQwIavGknRAbx1nXjOVZSznVv/cqlr61mf04xb147nnlnDOb+\nWUN4Zu4o/nFZMusO5HLrf9Y3GZ6qsms+2XCQKYNivDo0UBDaAxn3Qcs9i1CrLxVVGltlFdsOFdAt\nyI+Y0AC2ughHXVan5BAR5MeQHk2LRXeHZ9FZktzrDxzD39eHEXFhtbbPGNqd5TuOsvtIEYM9+Fxt\nxT9/2kePMCtPXzyq2lsI8vfltavHct6LP3HTO+tYdMfJZBwr4YfdWfzPGYMarUZKig/n5SvHcNM7\n67D6WXj3holM6BtZ65hzR/aixFbFfYs2M+JPX+FnUYQE+BJi9SUkwI/QAF+CAywEB/hi15rM/DIe\nOkfWmRa6PiIWgK2qpQluc3yxzYjFsF5hdA+z8uOe7AbPWZ2aw8S+kfVCIe6o9izybU0c2Xxe+m4P\nH6xNJzo0gO6hAcSGBRATYiUmNICY0ACiQvyJDg4gMsSfYH8LSinWHTjGqPjwer+nGUPM8MPlO460\nm1jszy5m5e4s7p05sJ5H2DsqmPlXjuG6t37l3g83EWr1JcjfwlWTejdwtRqmDo7lo1snER7o3+DM\nqEvGJ9AzwsqWg/kUlVVSWFZJsa2SQlslRWWVZBXZ2J9TQpGtkiE9Qpk5tHubfGZB6EhELABbRcsS\n3M7Js3kl5ew6XMh1k/vQI8zKJxsOcqSgrF7oIT23hIxjpdx0Sl93l3N7/dAAX694Fl9uO4xda8Ks\nvuzPKeaXfbnkl7pP3Pr7+hAV7M+RgjJunlK/0ik2zMqo+HCW7zhS3b9QVlHFQ4u3cuGYOCYPiG6V\nrfuyi8ktLq+VK3l/bRoWH9XguOxTB8bw8DlDeezz7QBcP7lPk30tTlzDbA1x6sAYTh3Y+SbHCoK3\nELGgZbOhoGby7OaMfMqr7AzvFUavCDPTaevB/HpisTo1B4CT+nt+8+zuhfJZu12TcrSYyyck8sfz\naiqXbJVV5BSVk1VoI7e4nOwiGznF5RwrLie3uJwiWyUXj3HfKzBjaHf+vnw3WYU2YkIDeP7bPSza\nkMHKPVks//1pLR5Kd6SgjLmvria/tJyFt51MckIEZRVVLFyXzhnDujeaC7ju5D7sOVrE4g0HuWGy\nZwItCIJ7RCxoXQc3wNr9JvE7rKcRC6Vgy8F8ZtQJP6xJySEq2J9B3T0fie2NLu7MgjJKK6roH1u7\nfDfA10KviMBqwWsOM4bG8tw3u/l+51GG9Azl9ZWpnDIgmtWpOTyxdDtPXzyq2dcsr7Rzx3sbKCmv\nJCYkgDvf28AXvzuF73Ye5VhJRZNhJaUUT8wZwQNnDZGV3QShlUg1FMaz8Lf4eJRHcKVaLPblYvXz\noV9MCMEBvvSLDmbrwdpJbq01q1NzmNQvqln19j3C276LO+VoEQD9Y9puHYdhPcPoFW5l2dZM7vt4\nM9Eh/sy/cgw3n9qPBesyWLW34TxOQzzxxXbWHzjG0xeP5JWrxpJVaGPegt/495oD9IsO5uT+UU1e\nQyklQiEIbYCIBeYbbHNDUFAThtp7tIghPcKwOMQmKS68XvnsgZwSMvPLmOTBDc6VHmFWjhbaTId0\nG5GS1fZioZRixtDufL8ri52HC3nygiTCA/24d+ZA+kQF8b+fbPG4NwHMDKd3Vh/g5lP7cu7IXoxK\niODhc4fy3c6jbEzL44qJidLkJgjtiIgFJlbf3OQ2QHBATVXQ8F415aQj4sI5XFBGVmFNFVN1vqJf\n88Sie7iVKrsmu6jtKqJSsooID/QjOsSzhK+nzBhqqqLmJPeqDsFZ/Sz85cKRpOWW8LevdzV2ejWV\nVXae+GIn43p34/5ZQ6q3Xz2pN3OSexEe6CdzlgShnRGxwIShWuNZAAyrIxYAWw/VeBerU3KICQ2g\nf0zzFr6pKZ/1PBSVW1zOXkeoyR0pR4vpHxPc5t/MpwyM4S8XJvH4nBG1tp/UP4orJybyr1X7+MUh\nmo3x455ssots3DylX63lR5VS/P3SZH68f5rHlU2CILQNIhaYMFTLPIsasRjeK9zluRGOrY6xH1pr\nfk7J4aRm5isAEiJNsnnn4YYb/epy38ebuez1NQ12F+/NKmrTEJQTH0cpq7scwYNnDyWhWxD/b+Fv\nTY51X7Qhg25BfkwbHFtvn+QgBKFjELHAhKGa25AHZqaQv69Zi9u1IzvU6kff6GC2HMzHbtdsO1RA\ndpGNk5qZrwAY3D2U3lFBLPntkEfHH8wr5budR8guspFxrP5yofmlFWQV2ujfQMOZtwgO8OW5S0Zx\nMJhHAYMAAA7hSURBVK+UJ77Y3uBx+aUVfL39CLNH9WqRgAuC4B3kfyOOBHcz17JwEhLgS/+YkHpj\nJEbEhfPNjiMMeGgp575o1qX2pHqnLkop5iTH8XNKDpn5Ta8V/dHaNJy58C1uZlSleiG57Snj+kRy\ny5R+fLA2ne93HnV7zNItmZRX2rmwgX4OQRA6BhELakpnW0J8t8B684MAbp3Sj2tP6sNd0wbwyLnD\neOu68fT2YCy5Oy4YHYfWsGRT495FZZWdj9alc3L/KPwsyq1YpGSZqbfNzZ20FfNOH8SQHqH84ePN\ntQoAnHyyIYP+McGMjA93c7YgCB2FNOXROs/ivZsm4udGaEbEhVcnultLn+hgRidGsHjjQW49rX+D\nx3278yhHCmz8+fwR5JdWuJ1+m5JVhJ9FkRjZ9Ih0bxDga+H5y0Yz+6Wf+H8Lf+Pt68ZX97ccyCnm\n1/3H+MOZg6UsVhA6GeJZ0DrPItTq1y7rKl8wOo6dhwvZkdlwovv9X9LoEWZl+pBYkuLC2ZyRXy/J\nvfdoEX2igmtVGbU3g3uE8si5w1i5O4t//bSvevvijQdRsvaDIHRKRCxoeYK7PTl3ZC98fRSfbjzo\ndn96bgkr92Rx6fgEfC0+JMWHk19aUS/JneKlSqjmcuXERM4c3p2nv9rJl1sP89J3e3h39QFO7h/V\nonEjgiB4FxELWl46255EBvszdXAMn2065Lab+8Nf01DAZRPMOthJjhDYZpdV+yqq7KTllNSbCdUR\nKKX460UjiQkJ4Lb/rOfZr3fTLzqY/z1L1n4QhM6I5CxoeVNee3PB6HiW79jAmtScWmO/q+yaj9dn\nMG1wLD3DzbfywT1Cq5Pc54zsCZiRI5V23Sk8C4CIIH/evH48v6Tmcvqw7uJRCEInpvPfIduB1iS4\n25MZQ2MJs/ry0a/ptbavSc3hSIGtVrlpgK+FwT1C2XIwr3qbcyZUQ4v6dARDeoRx7cl9RCgEoZPT\n+e+Q7YBJcHfunAWYOUsXjY1n2dZMclxmRX2y4SChVt/q2UxOkuLC2eKS5HaKRb9O4lkIgtB1ELHA\nkeDuAp4FmMRwRZUJOwGUllfx5dZMzknqWa8qKykugoKyStJySwBTCdUjzFprppUgCIInePUOqZSa\npZTapZTaq5R6wM3+KUqpDUqpSqXUxXX2VSmlNjl+lnjLRrtdU1GlW1w6294MiA1lQt9I3l+bht2u\n+Xr7YYrLq5jjptzUmeSe//1e7vv4N77dcbRTJLcFQeh6eO0OqZSyAPOBs4BhwOVKqWF1DksDrgPe\nd3OJUq11suNntrfsLK9yLKnaRTwLMN7FgZwSVqVk88mGg8RFBDKhT/0u8sE9QgkN8GXBugyW7zjK\nyPhwbjq1/hragiAITeHNeMQEYK/WOhVAKfUhcD5QPUVOa73fsc/uRTsapWb97c6fs3Aya0QPIoP9\nefHbvaw7kMvtU/u7XeXP39eHL38/BR9lRp1LV7QgCC3Fm1+n4wDXsp0MxzZPsSql1iml1iil5rg7\nQCl1i+OYdVlZWS0y0lZpVm/r7H0WrgT4Wpg7Np61+3Oxa1NS2xBxEYH0DA8UoRAEoVV48w7p7u7U\nnLVBE7XW44ArgH8opeoNRdJav661Hqe1HhcTE9MiI2NDraQ8eTaXj09o0fkdxeUTEgEYGR/eqUph\nBUE4PvFmGCoDcL0DxwOeLcoAaK0POR5TlVIrgNFASlsa6OT/t3fuwVbVVRz/fJMUHwjKVQcf4xUk\n0VJB0cQ3Zo5ZKk0wAzGliTlOmK+0kTRLpwfmjNpr0IaMfyhN8ElTyLMcU+CCXLgIJo2YjBY0KfnK\nQFZ//NbB7encs6967zl3b9dnZs/ev/X77b3X95x9ztr7t/devzR2drHOvFtbdueGzx7+rhH6giAI\neoqevLJYBgyVdIiknYHxQJeeapK0l6RdfLkFOInMvY4gcfEpgzlxSEt+wyAIgg9IjwULM9sGXAbM\nBdYCvzWzNZJulnQegKTjJG0ExgF3SVrjqx8OtElqBxYBU80sgkUQBEGTUGfjNBeNkSNHWltbW7Pd\nCIIgKBSSlvv94boU5xGgIAiCoGlEsAiCIAhyiWARBEEQ5BLBIgiCIMglgkUQBEGQSwSLIAiCIJfS\nPDoraTPwfJW5BfhnE9zpScqmqWx6oHyayqYHyqfpg+g52Mxy8yWVJljUQlJbV54fLhJl01Q2PVA+\nTWXTA+XT1Ag90Q0VBEEQ5BLBIgiCIMil7MHiF812oAcom6ay6YHyaSqbHiifph7XU+p7FkEQBEH3\nUPYriyAIgqAbKG2wkHS2pGckrZd0XbP96QqS7pa0SVJHxra3pHmSnvX5Xm6XpJ+4vlWSjmme57WR\ndJCkRZLWSloj6Qq3F1lTX0lLJbW7ppvcfoikJa7pXh/DBUm7eHm917c20//OkLSTpKckzfFy0fVs\nkLRa0kpJbW4r8nE3QNIsSev89zSq0XpKGSwk7QT8HPgMcAQwQdIRzfWqS8wAzq6yXQcsMLOhwAIv\nQ9I21KdLgGkN8vG9sA34hpkdDpwATPbvocia3gLOMLOjgeHA2ZJOAG4BbndNLwOTvP0k4GUzOxS4\n3dv1Rq4gjTtToeh6AEab2fDMI6VFPu5+DPzBzIYBR5O+q8bqMbPSTcAoYG6mPAWY0my/uuh7K9CR\nKT8DDPLlQcAzvnwXMKFWu946AQ8Bny6LJmA3YAXwSdILUX3cvuP4Iw3+NcqX+3g7Ndv3Kh0Hkv5s\nzgDmkMYYLqwe920D0FJlK+RxB+wJPFf9OTdaTymvLIADgBcy5Y1uKyL7mdlLAD7f1+2F0ujdFSOA\nJRRck3fZrAQ2AfNIY8O/Yml0SHi33zs0ef0WYGBjPc7lDuCbwHYvD6TYegAMeFTSckmXuK2ox91g\nYDPwK+8qnC5pdxqsp6zBQjVsZXvsqzAaJe0BzAauNLN/12taw9brNJnZ22Y2nHRGfjxpGOD/a+bz\nXq1J0ueATWa2PGuu0bQQejKcZGbHkLpkJks6tU7b3q6pD3AMMM3MRgCv806XUy16RE9Zg8VG4KBM\n+UDgxSb58kH5h6RBAD7f5PZCaJT0UVKgmGlm97u50JoqmNkrwGLS/ZgBkvp4VdbvHZq8vj/wr8Z6\nWpeTgPMkbQDuIXVF3UFx9QBgZi/6fBPwACmoF/W42whsNLMlXp5FCh4N1VPWYLEMGOpPdOwMjAce\nbrJP75eHgQt8+QJSv3/F/mV/8uEEYEvlkrS3IEnAL4G1ZnZbpqrImvaRNMCXdwXOJN1sXASM9WbV\nmipaxwILzTuSewNmNsXMDjSzVtLvZKGZTaSgegAk7S6pX2UZOAvooKDHnZn9HXhB0mFu+hTwNI3W\n0+ybNz14U+gc4C+k/uTrm+1PF33+DfASsJV0djCJ1B+8AHjW53t7W5Ge+PorsBoY2Wz/a+g5mXT5\nuwpY6dM5Bdd0FPCUa+oAbnT7YGApsB64D9jF7X29vN7rBzdbQx1tpwNziq7HfW/3aU3l91/w4244\n0ObH3YPAXo3WE29wB0EQBLmUtRsqCIIg6EYiWARBEAS5RLAIgiAIcolgEQRBEOQSwSIIgiDIJYJF\nUCgkjZA0vU79/pJm9eD+x3nWz0VV9lZJX3yf2/xzF9pM74lkmJK+K+manDZjurJvSZdJ+kr3eRf0\nJiJYBEXjW8BPO6s0sxfNbGxn9d3AJOBrZja6yt4K1AwWmTeha2JmJ+bt1MwuNrOnu+pkNzOGlL05\nj7uBy3vYl6BJRLAImoKfia/zM+YOSTMlnSnpcc/Pf3yNdfoBR5lZu5dP8/EKVnqCtX6+3Q6vn56p\n3yzpO26/VtIyz/V/Uyf+TVAaD6FD0i1uu5H0ouGdkm6tWmUqcIrv6ypJF0q6T9IjpIR2e0haIGmF\nb/f8zL5e8/npkhbrnXELZvpb8Lh9ZKW9pO8rjanxpKT93D7Ey8sk3VzZbg1t1yuN9TIfOCxj/6qv\n2y5ptqTdJJ0InAfc6tqG1GoHYGZvABtqfXdBCWj2m4kxfTgn0pn4NuBI0knLctKZqYDzgQdrrDMa\nmJ0pP0JKGAewBynhWiuZFO9edzCwzudnkcYrlu93DnBqVfv9gb8B+/g2FwJjvG4xNd6IJfP2s5cv\nJL2FX3mrtg+wpy+3kN6ArrwU+1pmG1tIuXw+AjwBnFy9X9Jb8ef68o+AG3x5Dp6aGri0st0qP48l\nvdW7Gyn19XrgGq8bmGn3PeDrvjwDGJupq9nOy9eTxjBp+jEWU/dOcWURNJPnzGy1mW0npWVYYOkf\nZzXpT7+aQaRUzRUeB26TdDkwwN5Jqb0DSZX0FJeZ2fOkYHEWKWXHCmAYaZCYLMcBi81ss29zJlAv\na2lnzDOzSpI9AT+QtAqYT0oZvV+NdZaa2Ub/TFZS+3P4LykwQAqylTajSFoBft2JT6cAD5jZG5Yy\nAGdzpn1C0mOSVgMTgY93so167TaRgm1QMur2pQZBD/NWZnl7pryd2sfmm6TcRACY2VRJvyPlm3pS\n0pnAf6rWuRO438zme1nAD83srjp+1Urx/H54PbM8kXSlcqyZbVXK8tq3xjrZz+Rtan8OWz2o1mtT\nj85y/MwgXUG1S7qQdKXzXtv1JX1PQcmIK4ugSKwFDq0UJA3xK5NbSEnWhmUbS5oM9DOzqRnzXOAi\npTE2kHSApH15N0uA0yS1KA3ROwH4Y45vrwL96tT3J40bsVXSaFKXWHfzJPAFXx7fSZs/AZ+XtKvf\nAzo3U9cPeEkprfzEjL1aW2ftAD5GSrAYlIwIFkFhMLN1QH//kwO40m9At5POZn9ftco1wJGZm9yX\nmtmjpC6aJ7wbZRZVf/KW0jlPIaXpbgdWmNlD1GcVsM1v+l5Vo34mMFJSG+kPdl1Xdb8HrgSulrSU\n1GW3pbqBma0A7iV1cc0GHstUf5sUKOdV+XcPcK0/RDCkTjtI42PMJygdkXU2KBT+R/yqmXX6rsWH\nFX8q6U0zM0njSTe7z89brxv3PwK42sy+1Kh9Bo0j7lkERWMaMK7ZTvRSjgV+5o/bvgJc1OD9t5Cu\nOoISElcWQRAEQS5xzyIIgiDIJYJFEARBkEsEiyAIgiCXCBZBEARBLhEsgiAIglwiWARBEAS5/A8S\ny9sgW76glQAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# first we split the training set into a training / validation set (as we are using k-folds cross validation elsewhere and don't have an explicit training and validation set.)\n", "X_train_train, X_cv, y_train_train, y_cv = model_selection.train_test_split(X_train, y_train, test_size=0.25, stratify=y_train)\n", "\n", "# set the model to calculate learning curves for\n", "#model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", "\n", "m_array = np.round(np.linspace(20, X_train_train.shape[0], 100)).astype(int)\n", "train_acc_array = []\n", "cv_acc_array = []\n", "\n", "for m in m_array:\n", " model = svm.SVC(C=bestC, kernel=kernel)\n", " # we now fit to the training data\n", " model.fit(X_train_train.head(m), y_train_train.head(m)) # training on the first m training data examples\n", " train_accuracy = model.score(X_train_train.head(m), y_train_train.head(m))\n", " train_acc_array.append(train_accuracy)\n", " cv_accuracy = model.score(X_cv, y_cv)\n", " cv_acc_array.append(cv_accuracy)\n", "\n", "train_acc_array = np.array(train_acc_array)\n", "cv_acc_array = np.array(cv_acc_array)\n", "\n", "fig, ax = plt.subplots()\n", "ax.plot(m_array, 1-train_acc_array, label='train')\n", "ax.plot(m_array, 1-cv_acc_array, label='cross-validation')\n", "ax.legend()\n", "ax.set_xlabel('m (size of training data)')\n", "ax.set_ylabel('error (1-accuracy)')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This learning curve is also typical of a high bias algorithm. Getting more data will not help improve the fitting, adding more features might." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Gaussian Kernel" ] }, { "cell_type": "code", "execution_count": 97, "metadata": { "collapsed": true }, "outputs": [], "source": [ "C = 1 # start with penalty parameter equal to 1 as we don't know what value this should take yet, larger C -> lower bias, higher variance, smaller C -> higher bias, lower varaince. Since we don't have that much data a lower bias algorithm is probably best to avoid overfitting\n", "\n", "kernel = 'rbf' # we'll use the radial basis function (Gaussian) kernal to see how ths performs\n", "\n", "gamma = 'auto' # a second hyperparameter to tune for the Guassian kernal - the width of the Gaussian function used\n" ] }, { "cell_type": "code", "execution_count": 98, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mean score: 0.82\n", "variance in score: 0.003506250000000001\n" ] } ], "source": [ "svm_model = svm.SVC(C=C, kernel=kernel, gamma=gamma)\n", "\n", "N_splits = 10\n", "kfold = model_selection.KFold(n_splits=N_splits)\n", "\n", "scoring = 'accuracy'\n", "score = model_selection.cross_val_score(svm_model, X_train, y_train, cv=kfold, n_jobs=1, scoring=scoring)\n", "\n", "print(\"mean score: {}\".format(score.mean()))\n", "print(\"variance in score: {}\".format(score.var()))" ] }, { "cell_type": "code", "execution_count": 99, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0 0.003\n", "1 0.01\n", "2 0.02\n", "3 0.03\n", "4 0.05\n", "5 0.1\n", "6 0.3\n", "7 1.0\n", "8 3.0\n", "9 10.0\n", "10 30.0\n" ] } ], "source": [ "C_array = np.array([0.003, 0.01, 0.02, 0.03, 0.05, 0.1, 0.3, 1, 3, 10, 30])\n", "acc_mean_array = np.zeros_like(C_array)\n", "acc_variance_array = np.zeros_like(C_array)\n", "for i, C in enumerate(C_array):\n", " print(i, C)\n", " svm_model = svm.SVC(C=C, kernel=kernel, gamma=gamma) \n", " score = model_selection.cross_val_score(svm_model, X_train, y_train, cv=kfold, n_jobs=1, scoring=scoring)\n", " acc_mean_array[i] = score.mean()*100\n", " acc_variance_array[i] = score.var()*100" ] }, { "cell_type": "code", "execution_count": 100, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 100, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEACAYAAAB/BTv2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4VNXWh9+VhA4iRZGmFFEBk1AiTWmCiIooiqBYEINc\nFbDgVbGjeL0KeLEXropcRQHxQzGiggqKSiQJVXqXJiAldEhZ3x97EidhJjOBJJOy3ueZZ+acs/c5\n66TM7+y11l5bVBXDMAzDyEvCQm2AYRiGUfwwcTEMwzDyHBMXwzAMI88xcTEMwzDyHBMXwzAMI88x\ncTEMwzDyHBMXwzAMI88xcTEMwzDyHBMXwzAMI88xcTEMwzDynIhQG+CLsLAwLVeuXKjNMAzDKDIc\nPnxYVbXQDBiCEhcReQAYCCiwFBgAvAfEACnAfOAfqprio2+apw/AH6raM9D1ypUrx6FDh4K6AcMw\nDANE5EiobfBGAhWuFJHawM9AE1U9IiJTgBnATuBrT7OPgZ9U9S0f/Q+qasXcGFWhQgU1cTEMwwge\nETmsqhVCbUcGwbrFIoByIpIClAe2qerMjIMiMh+okw/2GYZhGEWQgP45Vd0KjAH+ALYDydmEpRRw\nK/CNn1OUFZFEEYkXkWvzwGbDMAyjkBNQXESkCnANUB+oBVQQkVu8mryJc4nN9XOKs1U1BugHvCwi\nDf1cZ5BHhBJTU1NzdROGYRhG4SKYzIKuwAZV3eUJ2P8f0A5ARJ4GzgCG+eusqts87+uBOUBzP+3G\nqWqMqsZERBTKJDbDMAwjSIIRlz+ANiJSXkQE6AKsEJGBwOXATaqa7qujiFQRkTKez9WBi4HleWO6\nYRiGUVgJOERQ1d9EZCqwAEgFFgLjgEPAJmCe0xz+T1WfFZEY4C5VHQg0Bt4RkXSckL2gqiYuRpHk\njz+gTBmoUSPUlvggNRXi42HPnlBbUrQ46yxo0gQq5iqh1QiCgKnIocBSkY3ChCq88Qb8858QGQkJ\nCaG2yENaGsydC1OmwGefwc6dobao6HLOOdC0adZX48ZQodBk9gakqKYiG0aJZNcuGDAAvvoK6tWD\nxERYuBCa+4wcFgDp6fDLL05Qpk6FP/+E8uWhRw+44QZo0CBEhhVBVN1wdNmyv1/ffQfHj7vjIu6X\n7kt0rIJIQGzkYhh+mDULbrsN9u6F0aOhXz+oXRsGDoTXXy9AQ9LTnctryhT49FPYtg3KloWrroI+\nfdx7EXrCLtSkpsLatVkFZ9kyWL0aUjwFSESciGcXnQsucL+XEFHYRi4mLoaRjePH4fHHYcwY547/\n5BOIinLH+vWDr7923+/5+vCqCvPn/y0omze7gM8VVzhBufpqixMUJCkpsGbNiaKzZo0TJICwMGjY\n8ETROf9897vLZ0xcgsDExQgVq1Y5AVmwAO65xwmMt4h8/z107QoTJ7p2eYoqJCU5QZkyBTZtglKl\noHt3Jyg9e8Jpp+XxRY1T4vhxN6rJLjpr17qYGEB4OJx77omic955ULp0npli4hIEJi5GQaMK48fD\n0KHOs/H++3DNNSe2S0933xP16zuhyZMLL14Mkyc7QVm/HiIioFs3JyjXXAOnn54HFzIKlGPH3JNK\ndtFZt879EYH7PZ93nm/3msvAzRUmLkFg4mIUJHv3wj/+4bxPl14K//ufi634Y+RIeOop9z1xUvFz\nVfj9978FZc0a93TbpQv07QvXXgtVq570/RiFmCNHfIvO+vXu76JqVfjrLxOX/MLExSgo5s6Fm2+G\n7dvhX/9y6cZhAaYWb97sMlcff9wJTdAsX/63oKxc6S7UubMTlF69oHr1U7oXowhz+LD7m9i507lB\nTwITlyAwcTHym9RUJwzPPedGHx9/DBddFHz/K65wg4+NG92gwy+rVjkxmTzZPaGKQMeOTlCuuw7O\nPPNUb8UwgMInLjbPxShxbNjgRivz5sHtt8Orr0KlSrk7R2ysm1Yyc6YTmiysXfu3oCxZ4gTlkktc\n/vL117tZ4YZRzLGRi1Gi+OQTuOsu9/mdd+DGG0/uPMePu7hMx45uLiPr17ugzeTJbpYlQLt2Lijf\nu3fOQRzDyANs5GIYIeDAARgyxAXr27VzqcT16p38+UqXhluv2c9r4yuws9kVnLl4ljvQujW89JIb\n1tStmye2G0ZRJJiqyIZRpJk/35Vr+egjePpp+PHHUxCWzZth7Fho04bY99qSmh7Oh7uvhFGjnL8t\nPh6GDTNhMQolItJdRFaJyFoRGe7j+O0isktEFnleA72O9ReRNZ5X/4DXMreYUVxJS3NlW558EmrV\ncqOVSy7JRef167Omi/7+Oyxd6o43bw59+tB2yv0kHy2bGas3jFARyC0mIuHAauAyYAuQgFsyZblX\nm9uBGFUdkq1vVSARiAEUSAJaqupef9czt5hRLNm6FW69FWbPdh6qd96BKlV8NExPdyOO7PMOVq6E\no0f/bpdRNbdvX3fC884DILY63HmnG7C0bVsw92YYJ0krYK1n4UZEZBJuleFglkG5HJilqns8fWcB\n3YFP/HUwcTGKHZ9/7rK5jh1zM+1vvx1E02HDphNFZMUKN7Etg7p1nYh06fL3jOkc1vvo2xfuvx/e\ne8/ExSj01AY2e21vAVr7aHe9iHTAjXIeUNXNfvrmmKUSlLiIyAPAQNxwaCkwAKgJTAKq4hYSu1VV\nj/vo+ygQC6QB96rqt8Fc0zByy+HD8OAw5e13hJbnJvNxr08578ef4U2PiHi7WmvXdsJx111ZRSSX\ntbsqVXIJYZMnw8svWy1JI6REiEii1/Y4VR3nte3LcZs9LvIl8ImqHhORu4AJwKVB9s1qTCBrRaQ2\ncC/QRFWPiMgU4EbgSmCsqk4SkbdxAvJWtr5NPG2bArWA70TkPFVNC3Rdw8gRVdiyJXMEsmRuMjfN\nvJ3lRxrwEKN4bu0TlB6dAjVrOuEYODCriORhva7YWFeXbMoUuOOOPDutYeSWVFWNyeH4FsA706QO\nsM27garu9tr8L/CiV99O2frOycmYYN1iEUA5EUkBygPbcWqWURd2AjCCbOKC8+dNUtVjwAYRWYvz\n+80L8rpGSUfV1bfP7s5avhz270eB1xjKw4yiSulDzLzmDS67ojI0/cEJic9AS97Srp2rqv7eeyYu\nRqEmAWgkIvWBrbgH/yy1vUWkpqpu92z2BFZ4Pn8LPC8iGf9Q3YBHc7pYQHFR1a0iMgb4AzgCzMRl\nCuxTVc9CBn79b7WBeK/tgH46wwDcLMVevdyqi8nJf+8/4wwnGrfeys6zYxgw7WpmxFejRw94//2y\nnHHG4AI3VcSNXh5+2HnfGjcucBMMIyCqmioiQ3BCEQ68r6rLRORZIFFVpwP3ikhPIBXYA9zu6btH\nREbiBArg2Yzgvj+CcYtVwY1A6gP7gE+B7AUvwLf/LWg/nYgMAgYBlM7DNQ6MIsqcOTBjhouYt2//\nt0vrjDMA+PZb6N8f9u1zVVXuuSe0qcC33QaPPeYSCEaPDp0dhpETqjoDmJFt31Nenx/Fz4hEVd8H\n3g/2WsG4xboCG1R1F4CI/B/QDjhdRCI8o5cTfHceAvr4vAwfB4wDN88l2BswiilxcW6VrvHjs6zW\ndeyY+xL/z3+c1syaBZGRIbTTQ40abhn7CRNcdWV7PjJKOsHM0P8DaCMi5UVEgC64vOjZQG9Pm/7A\nFz76TgduFJEyHj9fI2D+qZttFGtUnbh07ZpFWFauhDZtnLAMHgwJCYVDWDIYOBB27XKmG0ZJJ6C4\nqOpvwFRcuvFST59xwCPAME+QvhrwHoCI9PT48FDVZcAUnBh9Awy2TDEjICtWuImNPXoATmvefRda\ntnTVV6ZPd66wfF3D/iS4/HJXCeC990JtiWGEHiv/YhQ+XnwRhg+HLVvYU642gwbBZ5+5eY3/+5/7\nAi+sPP44vPAC/PGHFUI2CpbCVhXZClcahY+4OGjRgp/W1SY6Gr74wtWFnDmzcAsLuFTk9HT44INQ\nW2IYocXExShc7N4Nv/7KX5f2oVs35/qaNw8eeijw8sOFgYYNoVMnlzWWnh5qawwjdBSBf1ejRPH1\n15CezvzavTh2zMVaYnKac1wIiY11BZV//DHUlhhG6DBxMQoXcXFw1lnM39uIsDBo0SLUBuWe66+H\nypUtsG+UbExcjMJDSgp88w1cdRUJiZJTMeJCTbly0K+fS0LYty/U1hhGaDBxMQoPnlIvelUPEhLg\nootCbdDJExvrloP5+ONQW2IYocHExSg8fPkllCnDpvMuY9cuaNUq1AadPC1aQHS0ucaMkouJi1F4\niIuDzp1JWO5S9YvyyEXEzdhfsAAWLgy1NYZR8Ji4GIWD1avdq4dziZUuXbhKu5wMN98MZcrY6MUo\nmZi4GIWDjIJcPXowfz40b170iz9WqQLXXQcTJ2ZdSdkwSgImLkbhIC4OIiNJq3MOSUlF2yXmTWys\nyxibNi3UlhhGwWLiYoSefftg7lzo0YNVq+DgweIjLp07Q/365hozSh4mLkbomTkTUlMzXWJQtDPF\nvAkLgwED4Icf3Kx9wygpmLgYoefLL6F6dWjdmoQEOO00OO+8UBuVd9x+u8seGz8+1JYYRsFh4mKE\nlrQ0t5zxlVdCeDjz57t1W4pCkcpgqVvXrfXywQfudg2jJFCM/oWNIkl8POzZAz16cOwYLF5cfFxi\n3sTGwpYtzgNoGCWBgOIiIueLyCKv134RuV9EJnvt2ygii/z03ygiSz3tEvP+FowiTVwcRERAt24s\nWeLKixWXYL43PXs6z58F9o1QIiLdRWSViKwVkeE5tOstIioiMZ7teiJyxOs7/+1A14oI1EBVVwHN\nPBcIB7YC01T1ZS9DXgKSczhNZ1X9K9C1jBLIl19Chw5QuXJmML84ikvp0nDbbfDqq7BzJ5x5Zqgt\nMkoanu/vN4DLgC1AgohMV9Xl2dpVAu4Ffst2inWq2izY6+XWLdbFc4FNXoYI0Af4JJfnMko6GzbA\nsmVw9dUAJCRAjRouRlEciY11SXEffhhqS4wSSitgraquV9XjwCTgGh/tRgKjgKOncrHcisuNnCgi\n7YEdqrrGTx8FZopIkogM8ndiERkkIokikpiamppLs4wiyVdfufcePQAyKyGLhNCmfKRJE2jTxrnG\nVENtjVECqQ1s9tre4tmXiYg0B+qqapyP/vVFZKGI/Cgi7QNdLGhxEZHSQE/g02yHbiLnUcvFqtoC\nuAIYLCIdfDVS1XGqGqOqMRERAb11RnHgyy/h/PPh3HM5cABWrCieLjFvYmPdfcbHh9oSoxgSkfGA\n7nllf5j39diW+ZgjImHAWOBBH+22A2eranNgGPCxiJyWkzG5GblcASxQ1R1exkQA1wGT/XVS1W2e\n953ANNzQzCjpHDgAc+ZkusSSktzTfHHMFPOmb1+oUMEC+0a+kJrxgO55jct2fAvg7XSuA2zz2q4E\nXAjMEZGNQBtguojEqOoxVd0NoKpJwDogx9louREXXyOUrsBKVd3iq4OIVPAEhxCRCkA34PdcXNMo\nrnz3HRw/nsUlBhATE0KbCoBKlaBPH5g82ZW5MYwCJAFoJCL1PZ6oG4HpGQdVNVlVq6tqPVWtB8QD\nPVU1UUTO8CQEICINgEZAjjUnghIXESmPyzD4v2yHTojBiEgtEZnh2awB/Cwii4H5wFeq+k0w1zSK\nOXFxcPrp0K4dAPPnuxpc1auH2K4CIDbWCcuUKaG2xChJqGoqMAT4FlgBTFHVZSLyrIj0DNC9A7DE\n810+FbhLVffk1EG0EEYWK1SooIcOHQq1GUZ+kZ4OtWq5qo6fuGeTevVcsHvSpNCaVhCoQuPGUK2a\nW9nZMPICETmsqhVCbUcGNkPfKHgSE2HHjsx4y86dsGlT8Q/mZyDiRi+//uqC+4ZRHDFxMQqeuDhX\nPKx7d+DveEtJERdwEyojIuD990NtiWHkDyYuRsHz5Zdw8cVQtSrgxCUsDFq0CLFdBUiNGm7gNmGC\ny2swjOKGiYtRsGzZAosWZWaJgROXJk2gYsUQ2hUCYmNh166/V3g2jOKEiYtRsGTMyvfEW1RdplhJ\ncollcPnlLq/B5rwYxRETF6NgiYuDBg3gggsAF8j/66/iP3nSFxERbiGxb76BrVtDbY1h5C0mLkbB\ncfiwmzzZo0dmAbGSGMz35o47XGb2Bx+E2hLDyFtMXIyC44cf4OjRTJcYOJdY6dIQGRlCu0JIw4bQ\nqZPLGktPD7U1hpF3mLgYBUdcnIvad/i7dmlCAjRv7gSmpBIbC+vXw48/htoSw8g7TFyMgkHVicvl\nl2cqSVqaK1hZUl1iGVx/PVSubIF9o3hh4mIUDIsWuai1VwryypWuxlZJF5dy5aBfP/jsM9i3L9TW\nGEbeYOJiFAxxcS6If+WVmbsygvklMVMsOwMHunDUxx+H2hLDyBtMXIyCIS4OWrfOsnj8/Plw2mlw\nXo6rQpQMWrSAZs3g3XdDbYlh5A0mLkb+8+efTkm8XGLgRi4tW7rSL4YL7C9c6F6GUdSxf2sj/5nh\nWd7HS1yOHYPFi80l5s3NN0OZMhbYN4oHAcVFRM4XkUVer/0icr+IjBCRrV77r/TTv7uIrBKRtSIy\nPO9vwSj0xMVB3boQFZW5a/FiSEmxYL43VarAddfBxIlw5EiorTGMUyOguKjqKlVtpqrNgJbAYWCa\n5/DYjGOqOiN7X8+ymG8AVwBNgJtEpEnemW8Ueo4ehZkzs8zKB5uZ74/YWJcxNm1a4LaGUZjJrVus\nC7BOVTcF2b4VsFZV16vqcWAScE0ur2kUZX78EQ4d8hlvqVHDDWiMv+nc2S33bK4xo6iTW3G5EfjE\na3uIiCwRkfdFpIqP9rWBzV7bWzz7jJJCXByULw+XXppld0YlZK/BjIFLbhgwwFXKWb8+1NYYxskT\ntLiISGmgJ/CpZ9dbQEOgGbAdeMlXNx/71M/5B4lIoogkpqamBmuWUZjJmJXftSuULZu5+8ABN4HS\nXGK+uf12J7rjx4faEsM4eXIzcrkCWKCqOwBUdYeqpqlqOvBfnAssO1sAb8dHHWCbr5Or6jhVjVHV\nmIiIiFyYZRRali2DjRtPcIklJTndsUwx39St66rkfPCBK5FjGHlFsAlWItJbRFREYrz2Perpt0pE\nLg90rdyIy014ucREpKbXsV7A7z76JACNRKS+Z+RzIzA9F9c0ijIZSyxedVWW3fPnu/eYGAw/DBzo\nFu2cOTPUlhjFhWATrESkEnAv8JvXvia47++mQHfgTc/5/BKUuIhIeeAy4P+8do8SkaUisgToDDzg\naVtLRGYAqGoqMAT4FlgBTFHVZcFc0ygGxMW5WZK1amXZnZDggtbVq4fIriLA1VfDGWdYYN/IU4JN\nsBoJjAKOeu27BpikqsdUdQOwFt/eqkyCEhdVPayq1VQ12WvfraoaqapRqtpTVbd79m9T1Su92s1Q\n1fNUtaGq/iuY6xnFgL/+gnnzTnCJgRMXc4nlTOnScOut8MUXsHNnqK0xigkBE6xEpDlQV1Xjcts3\nOzZD38gfvv7arX6VTVx27nRLG1swPzCxsZCaCh9+GGpLjCJCREZSlOc1KNvxHBOsRCQMGAs86KNd\n0MlZmcYEstYwToq4ODjrLFeR0QubPBk8TZpAmzbONTZsmKVtGwFJVdWcIpmBEqwqARcCc8T9sZ0F\nTBeRnkH0PQEbuRh5T0oKfPONG7Vkq0qZkOB2ZdMcww+xsbBiBcTHh9oSoxiQY4KVqiaranVVraeq\n9YB4oKeqJnra3SgiZUSkPtAImJ/TxUxcjLxn7lzYv99nvGX+fPdEXrFiCOwqgvTtCxUqWGDfOHX8\nJViJyLOe0UlOfZcBU4DlwDfAYFXNMVFeVHN0m4WEChUq6KFDh0JthnGyDBsGb74Ju3e7b0YPqm45\nl6uvhvffD6F9RYw77oBPP4Xt202UDf+IyGFVrRC4ZcFgIxcj74mLc+VeKmT9O9+0ySWRWaZY7oiN\ndctBT5kSaksMI3hMXIy8ZdUqWLPGr0sMLJifW9q1g/PPN9eYUbQwcTHyFj+z8sEF80uXhsjIArap\niCPiZuz/+qsL7htGUcDExchb4uKcepxzzgmHEhKgeXMnMEbuuO02iIiwWJVRdDBxMfKOfftcptjV\nV59wKC0NEhPNJXayZCRCTJgAx4+H2hrDCIyJi5F3fPONUxEf8ZaVK92aYSYuJ09sLOza9bfn0TAK\nMyYuRt4RF+eqUfpIB8sI5lum2Mlz+eWuBqgF9o2igImLkTekprp6YldeCeEnVuJOSIDTToPzzguB\nbcWEiAi3kNg338DWraG2xjByxsTFyBvi42HPHp/xFnDi0rLlCdVgjFxyxx2uHugHH4TaEsPIGftX\nN/KGL790j9bdup1w6NgxWLzYXGJ5QcOG0KmTyxpLTw+1NYbhn4DiIiLni8gir9d+EblfREaLyEoR\nWSIi00TkdD/9N3oWFVskIol5fwtGoSAuDjp2dL6vbCxe7GpZWjA/b4iNhfXr4ccfQ22Jf1TdaHXp\n0lBbYoSKXNUW8yxruRVoDZwP/KCqqSLyIoCqPuKjz0YgRlX/CvY6vmqLpaSksGXLFo4ePeqnlxEy\nUlNdEKBKFZ/icuCA85jVru0GN0buKFu2LHXq1KFUqVIAHDkCNWu6pLyPPgqxcdlYtQo+/ti91q6F\n8uWdCNqS1vlPYastltt/9S7AOlXdBGzy2h8P9M4zq3ywZcsWKlWqRL169RBb2KJwsWOH831deCGU\nLXvC4Q0boEwZd9h+dblDVdm9ezdbtmyhfv36AJQrBzff7Fxjr78Op/v0GRQc27bBpElOUJKS3O+4\nc2d48EF48UUngr/95nNerVGMyW3M5UbgEx/77wC+9tNHgZkikuRjZbSgOXr0KNWqVTNhKYwkJztR\n8SEs4Oa3lC9vwnIyiAjVqlU7YcQeGwtHj7ov9FCQnOzErUsXqFPHCQnASy/Bli3w/fdw110wY4Z7\n7rjySjfH1ig5BC0unsVlegKfZtv/OJAKTPTT9WJVbQFcAQwWkQ5+zj8oY3nO1NRUfzYEa65RUKSl\nOb9X5cp+Dx89ekKBZCMX+Pq7b9ECmjWDd98tODuOHoX/+z+4/nqoUcMJ3KZN8OSTruZZYqJbbaFW\nrb/7NG7s+qxZA9ddZ9UFShK5GblcASxQ1R0ZO0SkP9ADuFn9BG9UdZvnfScwDfCZM6Sq41Q1RlVj\nIoqpY/7KK69kXwge3wYOHMjy5cvz5+T79/PB9OkMGTkSgLfffpv//e9/mYczQmd//bWRCy+8MMdT\nbdy4kY+9HsUTExO59957897mYkJsLCxc6F75RVqaG4XExrpVq6+/Hn75xY1KfvvNicYzz8AFF/g/\nR+fObuLn7Nlw550u2G8Uf3LzLX4TXi4xEekOPAJ0VNXDvjqISAUgTFUPeD53A549BXuLJKqKqjJj\nxoyQXP/d/Hy83bfPTV7xBJvvuuuuLIcPe/4yypULfKoMcenXrx8AMTExxBSxSHBqaioF9XB0883w\nz3+6L+7XX8+786rCggUwcaKLpWzfDpUquZHHzTc7scjtLd56q4u9Pf00NGjg3o3iTVAjFxEpD1wG\n/J/X7teBSsAsT5rx2562tUQk41u0BvCziCzGrbf8lap+k2fWFyCPPPIIb775Zub2iBEjeOmllzh4\n8CBdunShRYsWREZG8sUXXwDui7Jx48bcc889tGjRgs2bN1OvXj3++sslzV177bW0bNmSpk2bMm7c\nuMzzVqxYkccff5zo6GjatGnDjh1uoLhjxw569epFdHQ00dHR/PrrrwB89NFHtGrVimbNmvGPf/yD\ntLQTVx7t1KkTiYmJOZ4/g/T0dOrVq5dlhHXuueeyY8cOvvzyS1q3bk3z5s3p2rUrO/780znfy5XL\nDKiMGDGCMWPGAJCUlETnztHExrZl3Lg3Ms+3ceNG2rdvT4sWLWjRokXmvQwfPpy5c+fSrFkzxo4d\ny5w5c+jhqVO2Z88err32WqKiomjTpg1LlizJvN4dd9xBp06daNCgAa+++qrP39/dd99NTEwMTZs2\n5Wmvb7aEhATatWtHdHQ0rVq14sCBA6SlpfHPf/6TyMhIoqKieO211wCy/P4SExPp1KlTpg2DBg2i\nW7du3HbbbX7vD2DUqFFERkYSHR3N8OHDWbduHS1atMg8vmbNGlq2bOnzHrJTpYr7wp840WWQnSpr\n18KzzzpXVkyME6zWrd0iZTt2uImbl1128hl/Tz7pKgyMGOEKcBrFnIyn6sL0Kl++vGZn+fLlJ+wr\nSBYsWKAdOnTI3G7cuLFu2rRJU1JSNDk5WVVVd+3apQ0bNtT09HTdsGGDiojOmzcvs88555yju3bt\nUlXV3bt3q6rq4cOHtWnTpvrXX3+pqiqg06dPV1XVhx56SEeOHKmqqn369NGxY8eqqmpqaqru27dP\nly9frj169NDjx4+rqurdd9+tEyZMOMH2jh07akJCQo7n9+bee+/V999/X1VV4+PjtUuXLqqqumfP\nHk1PT1dV1f/+97867N57VRMSdPyrr+rgwYNVVfXpp5/W0aNHq6pqZGSkvvfeHF27VvWf//ynNm3a\nVFVVDx06pEeOHFFV1dWrV2vLli1VVXX27Nl61VVXZdrhvT1kyBAdMWKEqqp+//33Gh0dnXm9tm3b\n6tGjR3XXrl1atWrVzJ+HNxk/79TUVO3YsaMuXrxYjx07pvXr19f58+erqmpycrKmpKTom2++qddd\nd52mpKRk6ev9+0tISNCOHTtm2tCiRQs9fPhwjvc3Y8YMbdu2rR46dCjLeTt16qQLFy5UVdVHH31U\nX3311RPs9/f3/913qqA6caLPwwHZvl315ZdVW7Vy5xFR7dRJddw41T17Tu6cOXHsmGqXLqoREarf\nf5/35y/JAIe0EHx/Z7yKZHDj/vth0aK8PWezZvDyy/6PN2/enJ07d7Jt2zZ27dpFlSpVOPvss0lJ\nSeGxxx7jp59+IiwsjK1bt2aOBs455xzatGnj83yvvvoq06ZNA2Dz5s2sWbOGatWqUbp06cyn9ZYt\nWzJr1iwAfvjhh8xYRnh4OJUrV+bDDz8kKSmJizyzE48cOcKZZ56Z4336O783ffv25dlnn2XAgAFM\nmjSJvn2YlxvBAAAgAElEQVT7Ai4dvG/fvmzfvp3jx49Tv3Zt16FMmRPOkZyczL59+4iK6kiFCnDr\nrbfy9dcuoTAlJYUhQ4awaNEiwsPDWb16dY42A/z888989tlnAFx66aXs3r2b5ORkAK666irKlClD\nmTJlOPPMM9mxYwd16tTJ0n/KlCmMGzeO1NRUtm/fzvLlyxERatasmfnzO80zR+e7777jrrvuynRv\nVa1aNaB9PXv2pJzH9+fv/r777jsGDBhA+fLls5x34MCBjB8/nv/85z9MnjyZ+RlVPoOgc2eoX9+5\nxjzexIDs3w/TprkRz/ffu5n+zZvD6NFw440u+yu/KF0apk6FSy5xo65ff4UmTfLvekboKJLiEip6\n9+7N1KlT+fPPP7nxxhsBmDhxIrt27SIpKYlSpUpRr169zLTRCn5SpObMmcN3333HvHnzKF++PJ06\ndcrsU6pUqczsoPDwcPxlzoEbdfbv359///vfQd9DMOdv27Yta9euZdeuXXz++ec88cQTAAwdOpRh\nw4bRs2dP5syZw4jhw12RSk+8JbttquL5OWQ9NnbsWGrUqMHixYtJT0+nrJ8U5uzny07GfZTxEjdf\n97RhwwbGjBlDQkICVapU4fbbb+fo0aOoqs9MLH/7IyIiSPfUXMmeGuz9u/Z3f/7Oe/311/PMM89w\n6aWX0rJlS6pVq+b355CdsDAYMACeesrN2m/QwHe7Y8dcXdGPP3aVeo4edW0fe8yJUuPGQV/ylDn9\ndPjqK2jTxqUox8e7ZAGjeFEka4u9/DLMmZO3r5xGLRnceOONTJo0ialTp9K7t5szmpyczJlnnkmp\nUqWYPXs2mzZtCnAW16dKlSqUL1+elStXEh8fH7BPly5deOuttwBIS0tj//79dOnShalTp7Jz507A\nxSWCuX4gRIRevXoxbNgwGjdunPlll5ycTG3PaGXChAnukbdCBZ8TWE4//XQqVqzMokU/U768E2Hv\n+69ZsyZhYWF8+OGHmXGiSpUqceDAAZ82dejQIfMcc+bMoXr16pkjjUDs37+fChUqULlyZXbs2JE5\ngrrgggvYtm0bCQkJABw4cIDU1FS6devG22+/nSlSe/bsAVzMJSkpCSBzFOULf/fXrVs33n//fQ57\nshwyzlu2bFkuv/xy7r77bgYMGBDUPXlz++1OZMaPz7o/Pd39bd95p/vy7tXLbQ8cCPPmuRjLyJEF\nKywZnHOOqxi0a5ebZJmtIIeRT4hIdxFZJSJrRWS4j+N3eZXr+llEmnj21xORI15lwN4OdK0iKS6h\nomnTphw4cIDatWtTs2ZNAG6++WYSExOJiYlh4sSJXJBTTqaH7t27k5qaSlRUFE8++aRf15k3r7zy\nCrNnzyYyMpKWLVuybNkymjRpwnPPPUe3bt2IiorisssuY/v27ad8n+BcYx999FGmSwxc4PqGG26g\nffv2VK9a1eWp5jCB5fnnxzN69GAuuaRtpssI4J577mHChAm0adOG1atXZz71R0VFERERQXR0NGPH\njs1yrhEjRpCYmEhUVBTDhw934hYk0dHRNG/enKZNm3LHHXdw8cUXA85FOHnyZIYOHUp0dDSXXXYZ\nR48eZeDAgZx99tlERUURHR2dmR799NNPc99999G+fXvCfSwrEOj+unfvTs+ePYmJiaFZs2aZiQ/g\n/o5EhG4+Cn8Gom5dt9bLBx+4X8nChfDQQ3D22c5tNmmSK1b99dduNv1rr7lRQ6injbVsCZMnO3tv\nusnZbuQfnvJdb+CmlTQBbsoQDy8+VtVIVW0GjAL+43Vsnao287zuIhChDvr4ehXGgL6Rjf37VRMS\nVPft83k4PV114ULVDRsK1qyiyujRo/WJJ57wezzQ3//UqS4gX7eue4+IUL36atVJk1Q9+QOFljfe\ncDYPGeL+boyTgwABfaAt8K3X9qPAozm0vwn42vO5HvB7TufP/rKYi3FyHDzo3v2MXI4fd/UsPbFr\nIwd69erFunXr+OGHH076HFdf7WbtV6gAjz8OvXtDLkI3IeWee1y86KWX3JIC998faouKLbWBzV7b\nW3BFiLMgIoOBYUBp4FKvQ/VFZCGwH3hCVefmdDETF+PkOHTIZYn5mfSQ4UO3si+BycgaPBVKl3ZF\nI4sqo0bBxo2ufMw557j4kJFrIrItazJOVcd5bftyhJ6QKaOqbwBviEg/4AmgP7AdOFtVd4tIS+Bz\nEWmqqvv9GnNSt2CUbFSdeuQQUD90yPn0g5mZbxhhYfDhh27lhn79XOJB6xOeqY0ApKpqTiUttgB1\nvbbrANtyaD8JeAtAVY8Bxzyfk0RkHXAe4HeNriIV0Fcf6ahGCDh+3K3+lcOwJKMSsi1rfOqUlL/7\ncuXgiy9c4curr3auMiNPSQAaiUh9TyHiG4Hp3g1EpJHX5lXAGs/+MzwJAYhIA6ARkONvqMj865ct\nW5bdu3eXmH+0Qk2Gz6tiRZ+HVV1NMXOJnTqqbj2XYOYCFQfOPNNltaWluTkwnmxtIw9Q1VRgCPAt\nsAKYoqrLRORZEenpaTZERJaJyCJc3KW/Z38HYImnlNdU4C5VzfG3k6uVKAsKW4mykLNnjwvo163r\nM5/1+HFX7LBaNb/6Y+SC7CtRlgTmzoWuXV3K9MyZPotAGNkobCtRFhlxMQoRrVs7H8acOT4Pjx8P\nd9zh1vgIYtqPYfjkk09c/KVfP7ecc6jn5RR2Cpu4FBm3mFFIOHrUzXrLYeJnQoKL9Z93XgHaZRQ7\nbroJnn/elax56qlQW2PkFssWM3LHwoUumJ+DuMyf72ZfWzDfOFWGD3eB/eeecwU677gj1BYZwWL/\n/kbumDfPvfvJEz12DJYsgVY+1xs1jNwhAm++6crbDBrk4i9G0cDExcgd8fFulpuntlp2Fi92AxtP\nFXvDOGVKlXILljVt6ioPLF0aaouMYAgoLiJyvlclzEUisl9E7heRqiIyS0TWeN6r+Onf39NmjYj0\n99XGKELEx0Pbtn4PZyxFYuJi5CWnnebK9Feq5FKUt+U09c8oFAQUF1VdpZ5KmEBL4DAwDRgOfK+q\njYDvPdtZEJGqwNO4+jWtgKf9iZBRBNi6FTZvDhjMr1HDZSkbRl5Sp44TmH374KqrwM/qDEYhIbdu\nsS64ssubgGuAjLrnE4BrfbS/HJilqntUdS8wC+h+ssYaIea339x7AHG56CJLGzXyh2bNnIts6VK3\namYOa+kZISa34nIj8Inncw1V3Q7gefe1vq6vKpy1c2ukUUiYN89VSGzWzOfh/fth5UpziRn5yxVX\nuCD/jBkwdKirCGEUPoJORfbUoumJWwMg6G4+9vn8UxCRQcAgcIs4GYWQ+HhX193PdOmkJPePbpli\nRn4zaBBs2AAvvOCWa37ooVBbZGQnNyOXK4AFqrrDs71DRGoCeN53+ugTdBVOVR2nqjGqGhPhp4y7\nEUJSUiAxMcdgvme1YGJyqstqGHnEv/4FffvCww/Dp5+G2hojO7kRl5v42yUGrppmRvZXf+ALH32+\nBbqJSBVPIL+bZ59RCNi/32lGUCxZ4mbnB5g8Wb8+VK+eN/YZRk6EhbmlnS++GG69FX79NdQWGd4E\nJS4iUh64DPg/r90vAJeJyBrPsRc8bWNE5F0AT9XMkbhSzwnAs4EqaRoFR4cOEB0NmzYF0Tg+3r0H\nCOabS8woSMqWhc8/h7PPhp49Ye3aUFtkZGCFK0sou3f/PcKoWdMFR/3E6R233AI//ODSkX2kgu3Y\nAWedBWPGwIMP5o/NhuGPtWudx/b0013eSUkcPVvhSqNQsHChe3/5ZQgPd6OYWbNy6JAxedJPjnFG\nvMUyxYxQcO65MH26m4Z17bXOg2uEFhOXEkrGeuu33uqe9OrVczOfP/zQR+Ndu2DduoAusbAwl0xm\nGKGgbVtXmv+XX6B/f0hPD7VFJRsTlxLKggVOUKpWdTOf586F9u3httvg3//ONncgyMmTTZrY4mBG\naOndG0aPdhMtH3ss1NaUbExcSihJSa4sfgaVK7vlZfv1c/+Ugwe7pWYBN7QJD8/awQtVlylmLjGj\nMPDgg3D33fDii/DOO6G2puRiE0pKIPv2OS9X9rUxypRxbrG6dd0/5tatbjXA8vHxLq2sfHmf59u4\n0SUIWKaYURgQgVdfhT/+cA9JZ5/tZvUbBYuNXEogGcF8XwORsDA36/n11+HLL6HLpcpfv60LavKk\njVyMwkJEBEyaBFFR0KcPLFoUaotKHiYuJZCMYH5OwffBg+Gzz2DRIqXdoZmsq9/Vb9v5813JscjI\nPDbUME6BihUhLg6qVHFVlDdvDtynuCMi3UVklYisFRFflezvEpGlnuVVfhaRJl7HHvX0WyUilwe6\nlolLCWTBAuf6OuOMnNv16gXf3/8lu6lGu3/3yByhZCchAZo3dwJjGIWJWrVcmf6DB53A7N8faotC\nh4iEA2/gSnk1AW7yFg8PH6tqpGeJlVHAfzx9m+AKFzfFVbZ/03M+v5i4lECyB/Nzot3OL/j19Kso\nXymcTp3cZEtv0tLc+cwlZhRWIiPdKHzFCrjhhlyUPCp+tALWqup6VT0OTMItnZKJqnrLbwX+LjR8\nDTBJVY+p6gZgred8fjFxKWHs3w+rV+diPkp8POdfXJ1584QLLnAlNt599+/DK1bAoUMmLkbhpmtX\nGDcOZs6Ee+4psWX6g1oCRUQGi8g63Mjl3tz09cbEpYSREdgMauSyb59Tj7ZtOess+PFHuOwyuPNO\nePpp9w+a4SqzTDGjsDNgADz5pHs4euGFUFuTL0SISKLXa1C240EtgaKqb6hqQ+AR4Inc9M1iTDAW\nG8WHBQvce1Ajl/nz3btn8mTFiq7Exj/+Ac8+6wKk4eFuffPzzssfew0jL3nmGbcOzGOPuUnEN90U\naovylFRVzWnBi6CXQPEwCXjrJPuauJQ0kpJckPOss4JoHB/vJg14+bxKlYL33nNzB555xu3r3Nml\nMBtGYUfEjVw2b4bbb3eJLZdcEmqrCowEoJGI1Ae24gL0/bwbiEgjVV3j2bwKyPg8HfhYRP4D1AIa\nAfNzuph9JZQwkpJyEW+ZNw+aNnVDEy9EYMQI+O9/3cilU6e8ttIw8o8yZWDaNPeANGAAHDsWaosK\nBlVNBYbg1tRaAUxR1WUi8qyI9PQ0GyIiy0RkETAMz5pdqroMmAIsB74BBqtq2gkX8cJK7pcgDh2C\nSpXgqaecOORIerqrW967t4uE+mHrVpfSbGnIRlFj5ky4/HJXS2/4CTM+ih5FsuS+iJwuIlNFZKWI\nrBCRtiIy2TPRZpGIbPQona++G70m5STmrflGbli0yAXhgxq5rFkDe/fmWKwSoHZtExajaNKtmyvP\n/9xz7iHJyFuCdYu9AnyjqhcA0cAKVe2rqs08k20+I+sqldnp7Glrq6uHkIxgflCZYkGsPGkYRZ2X\nXoLUVHjkkVBbUvwIKC4ichrQAXgPQFWPq+o+r+MC9AE+yS8jjbwhKQlq1HAB/YDMm+diLRdckO92\nGUaoaNAAHnoIJk5068AYeUcwI5cGwC5gvIgsFJF3RcTbr9ce2OGVYZAdBWaKSJKPvGujAFmwwLnE\n/CwmmZX4eGjd2tLAjGLP8OFuTaOhQ72WmTBOmWC+OSKAFsBbqtocOAR4h79uIudRy8Wq2gJXz2aw\niHTw1UhEBmVM/klNTQ3OeiNojhyB5cuDdIkdPAhLl+ZYCdkwigsVKsCYMa5auHf1CePUCEZctgBb\nVNWzHCFTcWKDiEQA1wGT/XVW1W2e953ANPzUo1HVcaoao6oxERE2/SavWbzYPZUFFcxPTHTZYhZv\nMUoIffpAx47w+OOwZ0+orSkeBBQXVf0T2Cwi53t2dcHlOgN0BVaq6hZffUWkgohUyvgMdAN+P2Wr\njVxzUsF8q+lilBAyFhjbu9eVNjJOnWAd6kOBiSKyBGgGPO/ZfyPZXGIiUktEMmrn1gB+FpHFuNmc\nX6nqN6dutpFbkpKgWjU3Izkg8+a5ei7VquW7XYZRWIiKcssjv/mm8wobp4ZNoiwhNG8OZ54J334b\noKGqqw1zxRXwwQcFYZphFBr27IFGjZzQ/PBDkMkvhYQiOYnSKNocPQq//x5kvGXjRti50+ItRomk\nalX4179gzhyYOjXU1hRtTFxKAL//7iaK2eRJwwjMnXdCdDQ8+KArmWScHCYuJYCkJPce1MglPh7K\nl4cLL8xXmwyjsBIeDq+95ionv/hiqK0pupi4lAAWLIAqVaB+/SAaz5vnSuxbOrhRgmnfHvr1g1Gj\n3PovRu4xcSkBZJTZDxicPHLEzSSzyZOGwahR7hnrwQdDbUnRxMSlmHP8uEurDMoltnChC85YvMUw\nqF3bTaqcNg1mzQq1NUUPE5dizrJlTmByFcxv3TpfbTKMosIDD0DDhnDffZCSEmprihYmLsWcXAXz\n581zC4sHtQayYRR/ypaFsWNhxQp4/fVQW1O0MHEp5ixY4CrnN2wYROP4eHOJGUY2evSA7t3d6q07\ndoTamqKDiUsxJynJzc4PWDl/yxb3smC+YWRBBF5+2eW7PPZYqK0pOpi4FGNSUlw15KDiLb95il7b\nyMUwTuD88+H+++H992H+/FBbUzQwcSnGrFgBx47lYvJkmTLQrFm+22UYRZEnnnDhyHvvdStSFEVE\npLuIrBKRtSIy3MfxYSKyXESWiMj3InKO17E0EVnkeU0PdC0Tl2JMRjA/qJHLvHlOhUqXzlebDKOo\nctppbsb+b7/Bhx+G2prcIyLhwBu4hRubADeJSJNszRYCMaoahVu7a5TXsSOq2szz6hnoeiYuxZgF\nC6BiRVc9P0eOH3dKZPEWw8iRW25xnuNHHoHk5FBbk2taAWtVdb2qHgcmAdd4N1DV2ap62LMZD9Q5\n2YuZuBRjkpKclytgMH/JElc62eIthpEjYWGu7tjOnTByZKityTW1gc1e21s8+/wRC3zttV3WsxR9\nvIhcG+hiJi7FlLQ0WLTIKiEbRl4TEwOxsfDKK7ByZaityUKE58s/4zUo23FfBaB8LuglIrcAMcBo\nr91nq2oM0A94WURynOAQlLiIyOkiMlVEVorIChFpKyIjRGSrV4DnSj99cwwgGfnDypUudTLoyZO1\nakGdkx4BG0aJ4l//ggoV3Mz9QrTeYqqqxni9xmU7vgXwXou2DrAt+0lEpCvwONBTVY9l7FfVbZ73\n9cAcoHlOxgQ7cnkF+EZVLwCigRWe/WO9AjwzsncKMoBk5AMLFrj3oEcubdoUrWX3DCOEnHkmPPMM\nzJwJX34ZamuCJgFoJCL1RaQ0bpn6LFlfItIceAcnLDu99lcRkTKez9WBi4HlOV0soLiIyGlAB+A9\nAFU9rqr7gryZgAEkI39ISoJy5Vx+fo7s3Anr11sw3zByyT33QJMmrv7Y0aOhtiYwqpoKDAG+xQ0Q\npqjqMhF5VkQysr9GAxWBT7OlHDcGEkVkMTAbeEFVcxSXYBbtaADsAsaLSDSQBNznOTZERG4DEoEH\nVXVvtr6+Akg+qyJ6/IODAEpbOuwps2CBC+YHXJbFJk8axklRqpSLu1x2Gbz0kqugXNjxeJhmZNv3\nlNfnrn76/QpE5uZawbjFIoAWwFuq2hw4BAwH3gIaAs2A7cBLPvoGHUBS1XEZvsIIW6jqlEhPd9Xz\ng548GRERZGPDMLzp2hWuuw6ef96tXGn8TTDisgXYoqqeR1ymAi1UdYeqpqlqOvBfnAvMV9+AASQj\nb1m9Gg4ezMXkyehot7SxYRi55qWX3APdww+H2pLCRUBxUdU/gc0ikuG97wIsF5GaXs16Ab/76B4w\ngGTkPUEH89PSXKEkc4kZxklTr56bVDlpEvz0U6itKTwEmy02FJgoIktwbrDngVEistSzrzPwAICI\n1BKRGeA/gJTH92BkIynJlQlr3DhAw2XL4NAhC+Ybxiny8MNw9tkwdKhbzNUA0UKUpJ1BhQoV9NCh\nQ6E2o8jSuTMcPvx3rN4v48bBP/4Ba9cGueCLYRj+mDoVbrgB3njDZZIVNCJyWFUrFPyVfWMz9IsZ\n6enOLRZ0ML96dWjQIN/tMozizvXXuwe7J56A3btDbU3oMXEpZqxfD/v3BxFvSU2FH36wyZOGkUeI\nwKuvuv+/J58MtTWhx8SlmJFRZj/gyOXdd2HTJhg4MN9tMoySwoUXwuDB8M47rrZfScZiLsWMhx92\nS7IePJjD0iwHDsC557rp+z/+aCMXw8hD9u51y1w0blyw/14WczHylQULIDIywJpfY8a4si+jR5uw\nGEYeU6WKm1Q5dy5Mnhxqa0KHjVyKEapQrRr07u0SwXyybRs0agQ9epTsv3zDyEfS0qBVK9ixw1Uo\nr1gx/69pIxcj39i40Q3JcwzmjxgBKSnu0cowjHwhPNwF97duhX//O9TWhAYTl2JEwGD+smXw3nsu\nCd/mtRhGvnLxxW5Z5DFjYN26UFtT8Ji4FCMWLHA1KCP91S4dPhwqVbI8ScMoIF580cU/hw0LtSUF\nj4lLMSIpCZo2hbJlfRycMwfi4uDRR11gxjCMfKdWLfcsN306fPNNqK0pWCygX0xQhTPOgGuucZ6v\nLKSnQ+vWLrq4apVbRcwwjALh2DHnTQgLgyVLAmRyngIW0Dfyhc2bXckJn/GWyZMhMRGee86ExTAK\nmDJl3NyzVatckL+kYCOXYsK0aW7RonnzslXQP3YMLrgATj/d+c3C7HnCMEJBjx5uUuXq1VCzZuD2\nuaWwjVxsycdiwoIFTjeiorIdeOMNl6M8a5YJy0mSkpLCli1bOFoUFko3Ci2DB5di5swG3HPPfp5/\nfvtJn6ds2bLUqVOHUqVK5aF1eY+NXIoJV17pXGNLl3rt3LvXpRy3alXyool5yIYNG6hUqRLVqlVD\nrKKBcQo8+ii88IIPD0OQqCq7d+/mwIED1K9fP8uxwjZyCepRVkROF5GpIrJSRFaISFsRGe3ZXiIi\n00TkdD99N3oWFVskIol5a74BLpiflORj8uTzz8O+fTBqVEjsKi4cPXrUhMXIEx5/3GWQDR3q8mxy\ni4hQrVq1IjGKDtYt9grwjar29ixXXB6YBTyqqqki8iLwKPCIn/6dVfWvUzc3Zz791JVdKGkcPOhK\nhWUJ5m/c6KKH/fv78JUZucWExcgLKlZ0z3q33AIffAB33JH7c5zK36KIdMd9n4cD76rqC9mODwMG\nAqnALuAOVd3kOdYfeMLT9DlVnZDjxVQ1xxdwGrABjwvNT5tewEQ/xzYC1QNdx/tVvnx5PRnKl1d1\nz/El85WU5PXDuPlm1XLlVDdvPqmfpfE3y5cvD7UJRYr+/fvrp59+qqqqsbGxumzZshPajB8/XgcP\nHpzjeWbPnq2//PJL5vZbb72lEyZMyFtjQ0B6umq7dqpnnKG6d+/JncPX3yRwSHP+Lg8H1gENgNLA\nYqBJtjadgfKez3cDkz2fqwLrPe9VPJ+r5HS9YEYuDTwKNl5EooEk4D5V9Q6K3AH4q4KowEwRUeAd\nVfVXUvGUWbDAfc2WRCpWhDp1PBtJSTBxIjz2mNdOwyh43n333ZPuO2fOHCpWrEi7du0AuOuuu/LK\nrAIjNTWViIisX7Mi8NprEBMDzzwDY8cWmDmtgLWqut7ZIZOAa4DlGQ1UdbZX+3jgFs/ny4FZqrrH\n03cW0B34xO/VclIej2LF4IZIrT3brwAjvY4/DkzDz8gGqOV5PxOnlB38tBsEJAKJpUuXPjk5N9xj\nUefOqtWrq+7bF2prigWhHrls2LBBzz//fI2NjdWmTZtqv379dNasWdquXTs999xz9bffflNV1YMH\nD+qAAQM0JiZGmzVrpp9//nlm/0suuUSbN2+uzZs3zxwNzJ49Wzt27KjXX3+9nn/++dqvXz9NT0/P\ncu3ly5frRRddlMWWyMhIVVV95plnNCYmRps2bap33nlnZl/vkUvHjh01ISFBVVXff/99bdSokXbo\n0EEHDhyYOXKZPn26tmrVSps1a6ZdunTRP//8Uzds2KA1atTQWrVqaXR0tP7000/69NNP6+jRo1VV\ndeHChdq6dWuNjIzUa6+9Vvfs2ZN5vYcfflgvuugibdSokf70008n/DwPHDigl156qTZv3lwvvPDC\nzJ+TquqECRM0MjJSo6Ki9JZbblFV1T///FOvvfZajYqK0qioKP3ll190w4YN2rRp08x+o0eP1qef\nfjrThkcffVQ7dOigY8aM8Xl/qqoDBhxXkVQ999yeGhkZqVOnTtV3331X77///szzjhs3Th944IET\n7uEkRy69ca6wjO1bgddzaP868ITn8z8zPnu2nwT+meP1cjroOclZwEav7fbAV57P/YF5eIZRQZxr\nRCCDVE/eLWao6ldfuV/ra6+F2pJiQ2EQl/DwcF2yZImmpaVpixYtdMCAAZqenq6ff/65XnPNNaqq\n+uijj+qHH36oqqp79+7VRo0a6cGDB/XQoUN65MgRVVVdvXq1tmzZUlWduJx22mm6efNmTUtL0zZt\n2ujcuXNPuH50dLSuW7dOVVVfeOEFHTlypKqq7t69O7PNLbfcotOnT1dV3+Kybds2rVu3ru7cuVOP\nHTum7dq1yxSXPXv2ZArTf//7Xx02bJiqahYxyb4dGRmpc+bMUVXVJ598Uu+7777M62X0/+qrr7RL\nly4n3E9KSoomJyerququXbu0YcOGmp6err///rued955umvXriz316dPHx07dqyqqqampuq+ffsC\nisvdd9+deczf/Q0ZMkLLlDmkXbq4Z8I9e/bowYMHtUGDBnr8+HFVVW3btq0uWbLkhHvwIy7HMh7Q\nPa9BmvX79wYf4vKa+v6uvsUzcinj2X7Ih7g86KtvxiugW0xV/xSRzSJyvqquAroAyz2BoUeAjqp6\n2FdfEakAhKnqAc/nbsCzga5pnCRpaW4pynPPhUGDQm1N8eT++/N+/dpmzdwU7hyoX78+kZ6KpE2b\nNqVLly6ICJGRkWzcuBGAmTNnMn36dMaMGQO4LLc//viDWrVqMWTIEBYtWkR4eDirV6/OPG+rVq2o\n4/rs9HgAAAjaSURBVHGdNmvWjI0bN3LJJZdkuXafPn2YMmUKw4cPZ/LkyUz2rAM0e/ZsRo0axeHD\nh9mzZw9Nmzbl6quv9mn/b7/9RqdOnTjjjDMA6Nu3b6YdW7ZsoW/fvmzfvp3jx4+fkGKbneTkZPbt\n20fHjh0B6N+/PzfccEPm8euuuw6Ali1bZv5svFFVHnvsMX766SfCwsLYunUrO3bs4IcffqB3795U\nr14dgKpVqwLwww8/8L///Q+A8PBwKleuzN69e3O0sW/fvpmf/d3fr79O5+GH72bkyPJ8/jn06lUF\ngEsvvZS4uDgaN25MSkpK5u89CFJVNSaH41uAul7bdYBt2RuJSFecR6qjqh7z6tspW985ORkT7Ky6\nocBEEVkCNAOexw2ZKgGzPGnGb3sMqyUiMzz9agA/i8hiYD5uxGMTLvKLDz5wZfVfeCH/ChgZIaFM\nmTKZn8PCwjK3w8LCSE1NBdyX5meffcaiRYtYtGgRf/zxB40bN2bs2LHUqFGDxYsXk5iYyPHjx32e\nNzw8PPNc3vTt25cpU6awevVqRIRGjRpx9OhR7rnnHqZOncrSpUu58847A6bH+styGjp0KEOGDGHp\n0qW88847p5xmm3FP/u5n4sSJ7Nq1i6SkJBYtWkSNGjU4evQoqhp0JlZERATpXrnE2W2uUOHv6Sb+\n7k9V6ddvPxde6KomHzni2g8cOJAPPviA8ePHM2DAgFzdewASgEYiUt+T9XsjMN27gYg0B94Beqrq\nTq9D3wLdRKSKiFTBDRS+zeliQaUiq+oiXOzFm3P9tN0GXOn5vB6IDuYaxily6BA89RS0bevqwBj5\nQ4ARRii5/PLLee2113jttdcQERYuXEjz5s1JTk6mTp06hIWFMWHCBNJyma/fsGFDwsPDGTlyZOYT\necYXZPXq1Tl48CBTp06ld+/efs/RunVr7rvvPnbv3s1pp53Gp59+SnS0+2pITk6mdu3aAEyY8Hd2\na6VKldi/f/8J56pcuTJVqlRh7ty5tG/fng8//DBzFBMMycnJnHnmmZQqVYrZs2ezadMmALp06UKv\nXr144IEHqFatGnv27KFq1ap06dKFt956i/vvv5+0tDQOHTpEjRo12LlzJ7t376ZixYrExcXRvXt3\nv9fzdX/dunXj7bdf59VXX+bSS+HZZ4/w73+Xo3Xr1mzevJkFCxawZMmSoO8rEOqmjQzBiUI48L6q\nLhORZ4FEVZ0OjAYqAp96hPYPVe2pqntEZCROoACeVU9w3x9WD6S4MHasW8J4zBiXjmKUOJ588klS\nUlKIioriwgsv5EnPuj333HMPEyZMoE2bNqxevTrLU3Ww9O3bl48++og+ffoAcPrpp3PnnXcSGRnJ\ntddey0UXXZRj/5o1azJixAjatm1L165daeE1KWvEiBHccMMNtG/fPtMlBXD11Vczbdo0mjVrxty5\nc7Ocb8KECTz00ENERUWxaNEinnrqqaDv5eabbyYxMZGYmBgmTpzIBRdcADh34+OPP07Hjh2Jjo5m\nmGcRlldeeYXZs2cTGRlJy5YtWbZsGaVKleKpp56idevW9OjRI/McvvB3f0888QR79+5l6NALqVz5\nW/7zn9J4dI4+ffpw8cUX8//t3c+LG2Ucx/HPhxWJeFoEPTQWf6yWFATXg//AlqVCqKKCihelFTzo\nYW8K4k30XChKxbqelKW16ori1cseLOJBXYTSQw0eXPboxRa/HhJsiJNkJjyTmUzeL5hDnsyTfPfD\nLN99EvaZ1dXV3D9XHhHxTUQ8HBEPRsQ7g7G3B41FEXEiIu6JiEcHx6mhuRciYm1wfDztvZq1/cuZ\nM9LQkn+pXL4sbW5Kly5VXUnj7O/vq9PpVF0GGuz69f7+st2utLMjdbtdbW1taWNjI/P8rGuybtu/\nNGvjyr29Wx9cLptOh21egAV19Gh/37Hd3ZtaW3tM6+vHxjaWRdGslQtQAlYumIcbN6SVlXybl7Ny\nAQDkUvMd9AvjC30ghzqu8LGcFuVapLkAU7RaLR0eHi7MLzWaK6J/P5dWq1V1KVPxsRgwRbvdVq/X\n08HBQdWlAP/dibLu+EIfABqgbl/o87EYACA5mgsAIDmaCwAguVp+52L7H0l/Syq2w17fSo55084Z\n93zW+OjYtMe3qX/ztdTy/NyzzmtiXlnvlWoOeRWfM+k88sp33h0RUZ8Fw6SbvVR5SDpf1rxp54x7\nPmt8dCzH4yvkVX1es2ZGXuXMmXQeeaV57Xkf9ely/7db4rxp54x7Pmt8dGza47KQV3GzvBd5lTNn\n0nnklea156qWH4s1me0rMflucRhCXsWQVzHkVZ46r1ya6nzVBSwY8iqGvIohr5KwcgEAJMfKBQCQ\nHM0FAJAczQUAkBzNpSZsP2X7Q9tf2t6sup66s/2A7Y9sX6y6lrqyfaftTwbX1YtV17MIuK7Sobkk\nYPuC7T9t/zwyftL2b7av2n5j0mtExBcR8YqklyQ9V2K5lUuU17WIOF1upfVTMLunJV0cXFen5l5s\nTRTJbFmvqzLQXNLYlnRyeMD2iqRzkp6QdFzSC7aP237E9tcjx91DU98azGuybaXLa9lsK2d2ktqS\nfh+cNsvWQE2xrfyZIRFuFpZARHxv+76R4cclXY2Ia5Jk+zNJT0bEu5K6o69h25Lek/RtRPxYbsXV\nSpHXsiqSnaSe+g3mJy3xH5IFM/t1vtU119JecHNwRLf+apT6v+hHJpz/uqQTkp61/WqZhdVUobxs\n32X7A0nrtt8su7iaG5fd55Kesf2+FmC7kDnLzIzrKh1WLuVxxtjY/1iNiLOSzpZXTu0VzetQ0jI2\n4SyZ2UXEX5JenncxC2JcZlxXibByKU9P0r1Dj9uS/qiolkVAXrMju+LIrGQ0l/L8IOkh2/fbvl3S\n85K+qrimOiOv2ZFdcWRWMppLArY/lbQn6Zjtnu3TEXFT0muSvpO0L2knIn6pss66IK/ZkV1xZFYN\nNq4EACTHygUAkBzNBQCQHM0FAJAczQUAkBzNBQCQHM0FAJAczQUAkBzNBQCQHM0FAJDcvxfF3XB7\ni2nNAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig, ax = plt.subplots()\n", "ax.plot(C_array, acc_mean_array, label='mean validation accuracy', color='red')\n", "ax2 = ax.twinx()\n", "ax2.plot(C_array, acc_variance_array, label='variance in validation accuracy', color='blue')\n", "ax.semilogx()\n", "ax.legend()\n", "ax2.legend()" ] }, { "cell_type": "code", "execution_count": 101, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "1.0" ] }, "execution_count": 101, "metadata": {}, "output_type": "execute_result" } ], "source": [ "C_array[np.argmax(acc_mean_array)]" ] }, { "cell_type": "code", "execution_count": 102, "metadata": { "collapsed": true }, "outputs": [], "source": [ "bestC = 1" ] }, { "cell_type": "code", "execution_count": 104, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "SVC(C=1, cache_size=200, class_weight=None, coef0=0.0,\n", " decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n", " max_iter=-1, probability=False, random_state=None, shrinking=True,\n", " tol=0.001, verbose=False)" ] }, "execution_count": 104, "metadata": {}, "output_type": "execute_result" } ], "source": [ "svm_model = svm.SVC(C=bestC, kernel=kernel, gamma=gamma)\n", "svm_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 105, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.8202247191011236" ] }, "execution_count": 105, "metadata": {}, "output_type": "execute_result" } ], "source": [ "svm_model.score(X_test, y_test)" ] }, { "cell_type": "code", "execution_count": 106, "metadata": { "collapsed": true }, "outputs": [], "source": [ "prediction = svm_model.predict(data_test)\n", "\n", "with open('prediction_submission_SVM_gaussian_kernel.csv', 'w') as file:\n", " print('PassengerId,Survived', file=file)\n", " for i, id_ in enumerate(data_test.index):\n", " print('{},{}'.format(id_, prediction[i]), file=file)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When we submit this result to Kaggle we get a reported accuracy of ~78.5% which is quite a bit less than the accuracy we got when we evaluated it on our held back test data, but still the best score we have got with any learning algorithm so far." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "I'm not sure why the accuracy on the held-out test data and Kaggle test data is so different. Possibly there are examples in the Kaggle data that are unlike any in the training data given by Kaggle (i.e. my train/validation and test data I use here with known values of the target variable) so my learning algorithm can't categorise them correctly. Or maybe there is a data leak from the held-out test data into the train data (imputing the average on all the Kaggle training data for example may be an issue?) causing it to perform better for my held-out test data than Kaggle's test data." ] }, { "cell_type": "code", "execution_count": 113, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 113, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd4nOWV8OHfUe+W1VzkIjfcu1zA4NgYiGmmmd5D4Ast\nJN5QAiGUXbIsyZLAhoTiACFxMC0QJxCqC2DASC64d8uWLBcVq3fpfH+8M/JIHkkjW6Pmc1+Xrpl5\n2zyvJc+Zp51HVBVjjDGmOQEdXQBjjDGdnwULY4wxLbJgYYwxpkUWLIwxxrTIgoUxxpgWWbAwxhjT\nIgsWxhhjWmTBwhhjTIssWBhjjGlRUEcXoK0kJCRoSkpKRxfDGGO6lNWrV+eqamJLx3WbYJGSkkJ6\nenpHF8MYY7oUEdnry3HWDGWMMaZFFiyMMca0yIKFMcaYFnWbPgtjjH9UV1eTlZVFRUVFRxfFnICw\nsDD69etHcHDwcZ1vwcIY06ysrCyio6NJSUlBRDq6OOY4qCp5eXlkZWUxaNCg47qGNUMZY5pVUVFB\nfHy8BYouTESIj48/odqhBQtjTIssUHR9J/o7tGBRlg/L/wcOfNfRJTHGmE7LgkVAIKz4H9i8pKNL\nYozxoqCggD/84Q+tPu+8886joKDADyU6OVmwCOsB/abArs86uiTGGC+aCha1tbXNnvfBBx8QGxvr\nr2KddCxYAAydA9nroDSvo0tijGnkgQceYNeuXUyYMIEpU6Ywe/ZsrrnmGsaOHQvAxRdfzOTJkxk9\nejQvvvhi/XkpKSnk5uaSkZHByJEjufXWWxk9ejTnnHMO5eXlHXU7XZYNnQUYMgeWPQG7l8HY+R1d\nGmM6rcf+uYnN2UVtes1RfWN45MLRTe5/8skn2bhxI+vWrWP58uWcf/75bNy4sX4I6Msvv0xcXBzl\n5eVMmTKFyy67jPj4+AbX2LFjB6+//jovvfQSV1xxBe+88w7XXXddm95Hd2c1C4C+EyC8J+xa2tEl\nMca0YOrUqQ3mCjz77LOMHz+e6dOnk5mZyY4dO445Z9CgQUyYMAGAyZMnk5GR0V7F7TasZgFOJ/fg\nWU6wUAUbJmiMV83VANpLZGRk/fPly5fz6aef8vXXXxMREcGsWbO8ziUIDQ2tfx4YGGjNUMfBahZu\nQ+ZA8QE4vLmjS2KM8RAdHU1xcbHXfYWFhfTs2ZOIiAi2bt3KN998086lO3lYzcJtyJnO466l0Kvj\nvz0ZYxzx8fHMmDGDMWPGEB4eTq9ever3zZ07l+eff55x48YxfPhwpk+f3oEl7d5EVf13cZG5wDNA\nILBQVZ9stP9HwJ1ALVAC3Kaqm137fg7c4tr3Y1X9qLn3Sk1N1RNe/Oi5aRDdB25478SuY0w3smXL\nFkaOHNnRxTBtwNvvUkRWq2pqS+f6rRlKRAKB54BzgVHA1SIyqtFhf1PVsao6AXgKeNp17ijgKmA0\nMBf4g+t6/jVkDuz9CqrK/P5WxhjTlfizz2IqsFNVd6tqFbAYuMjzAFX1HIMXCbirORcBi1W1UlX3\nADtd1/OvoWdCbaUTMIwxxtTzZ7BIBjI9Xme5tjUgIneKyC6cmsWPW3Numxs4AwJDIOMLv7+VMcZ0\nJf4MFt7Gnx7TQaKqz6nqEOB+4BetOVdEbhORdBFJz8nJOaHCAhAcDpFJUHL4xK9ljDHdiD+DRRbQ\n3+N1PyC7meMXAxe35lxVfVFVU1U1NTEx8QSL6xIeCxWFbXMtY4zpJvwZLNKAYSIySERCcDqsG6R2\nFZFhHi/PB9xTL5cAV4lIqIgMAoYB3/qxrEeF9bBgYYwxjfgtWKhqDXAX8BGwBXhTVTeJyOMiMs91\n2F0isklE1gELgBtd524C3gQ2Ax8Cd6pq8ykm24oFC2PMcVi+fDkXXHABAEuWLOHJJ5/0elxUVFSz\n12mcZTc7O5v58zs+Z51fJ+Wp6gfAB422/dLj+T3NnPsE8IT/SteEsFio2NDub2uMOX41NTUEBXWe\nOcbz5s1j3rx5LR/ohTtY3HHHHQD07duXt99+uy2Ld1ws3UdjVrMwplN67bXXGDduHOPHj+f666/n\npptuYsGCBcyePZv777+f/Px8Lr74YsaNG8f06dNZv349ACtWrGDChAlMmDCBiRMnUlxczIEDB5g5\ncyYTJkxgzJgxfPHFsSMgp02bxqZNm+pfz5o1i9WrV/Ptt99y2mmnMXHiRE477TS2bdt2zLmvvvoq\nd911FwB79uzh1FNPZcqUKTz88MP1x5SUlDBnzhwmTZrE2LFj+cc//gE0TMl+7733kpGRwZgxYwBn\nPfSbb76ZsWPHMnHiRJYtW1b/fpdeeilz585l2LBh3HfffW30r35U5wnFnUVYD6gsgrpaJ8GgMeao\nfz8AB9u45t17LJzrvcnGbdOmTTzxxBOsXLmShIQE8vPzWbBgAdu3b+fTTz8lMDCQu+++m4kTJ/Le\ne++xdOlSbrjhBtatW8dvfvMbnnvuOWbMmEFJSQlhYWG8+OKLfP/73+ehhx6itraWsrJjJ+JeddVV\nvPnmmzz22GMcOHCA7OxsJk+eTFFREZ9//jlBQUF8+umnPPjgg7zzzjtNlv2ee+7h9ttv54YbbuC5\n556r3x4WFsa7775LTEwMubm5TJ8+nXnz5jVIyQ40yJDrPn/Dhg1s3bqVc845h+3btwOwbt061q5d\nS2hoKMOHD+fuu++mf3/PcUInxmoWjYW7Vtay2oUxncbSpUuZP38+CQkJAMTFxQFw+eWXExjofKn7\n8ssvuf766wE488wzycvLo7CwkBkzZrBgwQKeffZZCgoKCAoKYsqUKbzyyis8+uijbNiwgejo6GPe\n84orruCtt94C4M033+Tyyy8HnOSFl19+OWPGjOGnP/1pg9qHNytXruTqq68GqC8fgKry4IMPMm7c\nOM466yz279/PoUOHmr2W5z2OGDGCgQMH1geLOXPm0KNHD8LCwhg1ahR79+5t9lqtZTWLxsJ6OI8V\nhRAR17FlMaazaaEG4C+qinhZOsAzXbm3PHciwgMPPMD555/PBx98wPTp0/n000+ZOXMmn3/+Oe+/\n/z7XX3899957L9HR0Tz22GMALFy4kNTUVOLj41m/fj1vvPEGL7zwAgAPP/wws2fP5t133yUjI4NZ\ns2a1WH5vZV+0aBE5OTmsXr2a4OBgUlJSvKZXb/zv0JTGadhrampaLFdrWM2iMc9gYYzpFObMmcOb\nb75JXp6z9HF+fv4xx8ycOZNFixYBzsikhIQEYmJi2LVrF2PHjuX+++8nNTWVrVu3snfvXpKSkrj1\n1lu55ZZbWLNmDZdccgnr1q1j3bp1pKY6efWuuuoqnnrqKQoLC+uXcS0sLCQ52Uko8eqrr7ZY9hkz\nZrB48WKA+vK5r5OUlERwcDDLli2rrwk0l5Ld8x63b9/Ovn37GD58eItlaAsWLBoLczdDFXRsOYwx\n9UaPHs1DDz3E9773PcaPH8+CBQuOOebRRx8lPT2dcePG8cADD/DnP/8ZgN/97neMGTOG8ePHEx4e\nzrnnnsvy5cvrO7zfeecd7rnH+8DM+fPns3jxYq644or6bffddx8///nPmTFjBrW1LY/of+aZZ3ju\nueeYMmUKhYVHv4Ree+21pKenk5qayqJFixgxYgTQMCX7vffe2+Bad9xxB7W1tYwdO5Yrr7ySV199\ntUGNwp/8mqK8PbVJinKAgxvh+RlwxWsw6qKWjzemm7MU5d1Hp0xR3mVZM5QxxhzDgkVj7tFQ5dYM\nZYwxbhYsGguJAgmwmoUxHrpLc/XJ7ER/hxYsGhOxWdzGeAgLCyMvL88CRhemquTl5REWFnbc17B5\nFt6ExdpoKGNc+vXrR1ZWFm2yZozpMGFhYfTr1++4z7dg4Y3VLIypFxwczKBBgzq6GKaDWTOUNxYs\njDGmAQsW3oTH2mgoY4zxYMHCG6tZGGNMAxYsvLFgYYwxDViw8CYsFmrKoaayo0tijDGdggULbyzl\nhzHGNGDBwpswWwDJGGM8WbDwxvJDGWNMAxYsvLFmKGOMacCChTf1wcJqFsYYAxYsvLPV8owxpgEL\nFt5YM5QxxjRgwcKb4DAIDLVgYYwxLhYsmmL5oYwxpp4Fi6ZYyg9jjKlnwaIpFiyMMaaeX4OFiMwV\nkW0islNEHvCyf4GIbBaR9SLymYgM9NhXKyLrXD9L/FlOryxYGGNMPb+tlCcigcBzwNlAFpAmIktU\ndbPHYWuBVFUtE5HbgaeAK137ylV1gr/K16KwWMjf3WFvb4wxnYk/axZTgZ2qultVq4DFwEWeB6jq\nMlUtc738Bjj+BWLbWnM1i5pKqKtr3/IYY0wH8mewSAYyPV5nubY15Rbg3x6vw0QkXUS+EZGL/VHA\nZrmDhWrD7bXV8NvRsObVdi+SMcZ0FH8GC/GyTb1sQ0SuA1KBX3tsHqCqqcA1wO9EZIiX825zBZT0\nnJyctijzUeGxUFcDVaUNtxfsg9Ic2L+6bd/PGGM6MX8Giyygv8frfkB244NE5CzgIWCeqtavNqSq\n2a7H3cByYGLjc1X1RVVNVdXUxMTEti19U7O48/c0fDTGmJOA3zq4gTRgmIgMAvYDV+HUEuqJyETg\nBWCuqh722N4TKFPVShFJAGbgdH63H89g0cOj9eyIBQsqS2D3Mqirbf64gCAYMhtCItunXMYYv/Fb\nsFDVGhG5C/gICAReVtVNIvI4kK6qS3CanaKAt0QEYJ+qzgNGAi+ISB1O7efJRqOo/K+pZILuEVLF\n2VBdDsHh7VqsTuHzp2DlM74dO/B0uP5dCArxb5mMMX7lz5oFqvoB8EGjbb/0eH5WE+d9BYz1Z9la\n1FIzFMCRDEga2W5F6hTq6mDDOzB4Fnz/v5s/dt/X8P4C+OBncOEzIN66sYwxXYFfg0WX1lSwOLIH\novs6NYv8PSdfsMj6Foqy4KxHoNeo5o/tNQoKs+DLp6HXaJj2/9qnjMaYNtdssBCRMOAC4AygL1AO\nbATeV9VN/i9eBwrv6Tx6JhOsq3MCxLjLYe1ffZ+0V5oH2z+Eide2fTnb24a3ISgMhp/r2/FnPgw5\n2+DDB5x/r6AwZ/ugM2Co14pl55WzDb57/djh1ADDzoaU09u/TMa0kyaDhYg8ClyIMxJpFXAYCANO\nAZ50BZL/UNX1/i9mBwiNcR49axbFB6C2EvpOgi3/PNrZ3ZK0hbD8V05nb0zfti9re6mtgc3vwSlz\nITTat3MCAuDSF2DRFZD+irOtrgbW/BkWbOlafT7/vh92L4fARv0vWgvf/BFu/gD6pXZI0Yzxt+Zq\nFmmq+mgT+54WkSRgQNsXqZMIDIKQqIbBwh0c4gZBz0G+j4hyz8koOdS1g0XGF84ckzGXte680Gj4\ngcd8yz2fw58vhE3vwYSr27aM/pK70xkBNvsX8L17G+4ry4eXZsPrV8Nty6BH50lEYExbaXKehaq+\nDyAiY5rYf1hV0/1VsE4hLLbhaCh3s1PcYOfHl2YoVche4zwvOdz8sZ3dxrchJNppcjkRKWdAwilO\njaurSH/ZGQo86YZj90XEwdWLndFxr1997EROY7oBXzq4nxeREOBV4G+qevKsCBTWA8qPHH2dv8f5\nwIjp59QutixxmmYCm/lnLMxyvo2DU7PoKKqw/ElIHA5jLm36uJxt8NFDUFd97L7Mb2HURSfedCQC\nU34I/74PstdC32PmW/pHUTZ8+hjM/W/nA95XVWWw7q8wch5E9/J+TNJImP8y/O0KeO92mP+q0wRn\nTDfR4l+zqp4OXIszGztdRP4mIif41bKL6DUaMlYe/aaYvxtiBzrBIW6w0/ZemNn8Ndy1CujYmsVX\nz8KKJ+Hr3zd/3Ia3Yddnzrfkxj99J8K0H7VNecZfBcERkPantrmeL75+DtYvhtWvtO68je84zZFT\nftj8caecA+f8J2z+B6z4n+MvpzGdkE9DZ1V1h4j8AkgHngUmijOL7kFV/bs/C9ihUn8AG950PkAn\n3+j0WcQNcvb1dD3m7z66zZv9ayAg2OkU7ahgse1D+OQR58P54AaoqWp6klz2GkgcCbd87N8yhfWA\ncVfAd284H7Du0Wf+UlXmjGADp6N9xk8gILDl81Qh7SXn32TgaS0ff+pdcHiLE5hbqsUZ04W0WLMQ\nkXEi8ltgC3AmcKGqjnQ9/62fy9exBkyHpNHOh4Uq5GccDRLuANHSiKjsNU4NJaYvlHZAsDi0Gd65\nBfqMg/N+A7VVcLiJUc+qTnBLbqdmodRboKYc1v3N/++16e9O/9O0Hzm1wR0+BsP9a+DAdzDlFt8m\nFYrABb+F/tPgvTucZjZjugFfaha/B17CqUWUuzeqararttF9iTgfEu8vgO0fQWWh0/wEENUbgsKb\nHxFVVwfZ62DsfKcv4ERrFpUl8KezoWi/7+dUl0N4HFz1+tF+iP1rvPcTFOyF8nxnaHB76DPO+VBd\n/Sqceqd/3yttISSOgHP+y2kmSlt4dK5IaS4suhzydx17Xk2lMypu3JXH7mtKUChc+Vd46UxYfB38\neK2lOzFdni/B4jycVetqAUQkAAhT1TJV/YtfS9cZjLvSacL5xJWlxF2jCAiAninNB4v8XVBZ5Hz4\nlh9xmoBOxIa34PBmmHid8wHmCwmESdc7yRBVncCRvQZn+ZBG9rv6V5LbKViA02H+0YNQdABi+vjn\nPfavdr7hn/cbCAyGyTc7817ydzuDFd64zvl3nXQDiJfK9sAZEBbTuveMSnIC01s3QlYapMxom3sx\npoP4Eiw+Bc4CSlyvI4CPAR8acLuB0ChnLsC3Lzqv3TULcAJHc81Qnh++BzdAyQmsuaHqfBvuPRbm\n/f748iyJOGXZ30TTSPYap28lafTxl7O1+k93HjO/gdGX+Oc90v7UsHYw6QYnGWLan5ymqX1fOyOZ\nWjt/pCVDZjvBetdnFixMl+fL2L4wVXUHClzPI/xXpE4o1f0tXJzRUG5xg52ahbf0D+B8+AZHQMJw\n55tmZSFUVxxfGTK/hUMbnRE5J5KQr+8kyNnifS7A/rVOMGrPJpM+45zmvH2r/HP9snxnNNO4K4/W\nDmL6wIgLYNXzTqf3zPvaPlCA04nfbwrs/Kztr21MO/MlWJSKSH27hIhMxskRdfJIGuFMJIsdAMFh\nR7f3THE6aIsPej9v/xroM94ZahvlGp9/vJ3caQudFCRjLz++892SJ4HWwYFGWVrqauHAuvbrr3AL\nDIbkyU7Nwh++ehZqKpy+J09Tb3WGPo+cB7N+7p/3Bhg6x+kgL83133sY0w58CRY/wVlv4gsR+QJ4\nA7jLv8XqhC5bCNe80XCbu0nKW1NUbTUcXH/0wzcqyXk8nk7ukhwnJ9OEa058ISF3eTznfwDk7oCq\nkvbtr3AbMM0JXm0983nfKmfdjQnXOSPSPKWcDj/8DC59yb+T54bMAdTJKWVMF+bLpLw0YARwO3AH\nMFJVT74FqKN7H5uOPM5jrkVjh7c432iT2yBYrP2LM+Q11UundGtF94KY5KP9KW7u4NHeNQtwRkRp\nbduua15ZAu/+PydP09wm1t3ol9qwpugPfSc4c0isKcp0cb6uZzEcGIWTdXaiiKCqr/mvWF1Ej/5O\nB+aOT5y+CU/7vnYe3UNUI93BopUpP+pqnUlkg2ZC4iknVl63vhOPrVnsX+N0AicMa5v3aI1+U5zH\nfauc+wRXOvPw4x8h9cnDzuJUN73f+pFMbSkg0FkoatdSp2+rLRaAOrgRYvsfXXPFmHbQYrAQkUeA\nWTjB4gPgXOBLwIJFYDAkjXKaiDa/d+z+6D5Hm6oiE53H1tYsPv81FO6Dub86sbJ6Sp4EW//lDOd1\nz5zOXgN9Jvg2q7mtRcQ5cyDc/RaHtzrzSUIi4dZlrQ8Y373hJP477e7OMQppyBzY9K4zPLdxc1hr\n7fwMFs13BiLc/CGEnFxjTUzH8aVmMR8YD6xV1ZtFpBfQhdKF+tnN7ztzBLyJSjr6TTIoxJnj0JoO\n7k3vwvL/hgnXOqN32kp9v8VaGHKmk/7j4IaOXcmu/zQnZXlpHrx+lTOEt7IYFl/jrBPha/LCzDRY\ncrczIGHOI/4ts6+GnOk87vzsxIJF7g5462anae3Aeidh4eWv2nK1pl340rNXrqp1QI2IxOAsgjS4\nhXNOHmE9nNFS3n4aZzaNSvK9GSp7Lbx7u/MhesFv2/YDwd005u63OLzJ6RPpiP4KtwHTnaHFr57n\nzFC/+nWn8zl7LfzjzqaHJ3sqzHKCS0wfuOI1p+bXGfRIdnJL7TqBfouyfPjblc493fQ+nP24U5td\n8VTbldOYZvhSs0gXkViclB+rcSbnfevXUnVXUUm+TcwrPgivXwORCU7aiKDQti1HeCzEDXESJFYU\nOqlIoGNGQrn1n+Y85myFS16A/lOd13N+CZ895gz37dG/+Wvs+MRJb3LjP1uXgrw9DJ0D374EHz98\nfOfv/QoK9jn3FjvAaWI7vMWZiZ44HEZf3Lbl7epUYd2io3/bnkIiYeptne9vpJNraQ1uAf7btYbF\n8yLyIRDTbZdS9bfIpJZH/FSXO9+OKwqczK/uUVRtbfTF8PUfnA8wcOY6eE44bG9xg53Z3ENmO+nL\n3U7/qVPTWLuo5WuERMLlrzi1us5m9CXOqDb3v3drBYXARb+Hgac6r0Xgwt85KWXe/ZEzMq/P+LYr\nb1e36gX48H4IDD02hUtNBez5Aq5/13J2tYJoC9V7EVmtqpPbqTzHLTU1VdPTO/nCfR8+6Kw9/WAT\niQBV4e+3OWnRr/wrjLywfctnup6Sw/DibEDh1qXOEO+T3c5PncSQw8+DK/5y7Dya9W/C32+FyTfB\nBb876ft8XJ/xLS4e70ufxTciMqUNymSiEp2Jb5Ul3vd/+bQTKM78hQUK45uoJKd/p/wILL72+NPJ\ndBc52+GtHzijFC95wfuEy3FXwOkLnGzHx1vTOwn50mcxG/h/IrIXKAUEUFUd59eSdUeeKT9CG2WN\n3bUUPnscxsyHM37W/mUzXVefcXDpi0723BdnOV9KTlY5252mpatfP/b/mKczH3b6Mz58wFkeuavX\nLhKGw/m/8etb+BIszvVrCU4m9bO4cxpmrwVnbkBEgtMu3dX/cE37G3khXPgMfLfYSTVzskoa4QSC\n2AHNHxcQAJe+AP9a0PLSyF1Bnf9/574ECx/GLBqfNDeLO/MbZ/ior/MJjGls8k3Oj/FNaDRcZs1Q\nvvIlWLyPEzAEJ93HIGAb0I6LHnQTTWWeLT7kpKaY8sN2L5IxxviixWChqmM9X7vSlXfgVN8uLCIe\nkGNTfrjTXLgXAjLGmE6m1bmZVXUN4NPoKBGZKyLbRGSniDzgZf8CEdksIutF5DMRGeix70YR2eH6\nubG15eyUAoOciXaNm6H2rYKgMBsnb4zptHxJJLjA42UAMAlocRqyiAQCzwFnA1lAmogsUdXNHoet\nBVJVtUxEbgeeAq4UkTjgESAVpwlstevcIz7eV+cV6WUWd+Y3TqoNmyBkjOmkfKlZRHv8hOL0YVzk\nw3lTgZ2qultVq4DFjc9T1WWqWuZ6+Q3Qz/X8+8AnqprvChCfAHN9eM/Or3F+qKoyZyW1AdM6rkzG\nGNMCX/osHjvOaycDnmPSsoDmPhFvAf7dzLnJx1mOziWql5OiwS17jbO8p/VXGGM6sRZrFiLyiSuR\noPt1TxH5yIdre5ss4HUYrohch9Pk9OvWnCsit4lIuoik5+T4kKCvM4hKdDq43WlW9rk7t6d2XJmM\nMaYFvjRDJboSCQLgahbyJbtdFuCZJrQfkN34IBE5C3gImKeqla05V1VfVNVUVU1NTOwis1ajejmJ\nzCqLndeZq5zZl5YB0xjTifkSLGpFpH46pGvEki8T9dKAYSIySERCgKuAJZ4HiMhE4AWcQOE5nvQj\n4BxXLaYncI5rW9fnnpi3bpGTIjxzlfVXGGM6PV8m5T0EfCkiK1yvZwK3tXSSqtaIyF04H/KBwMuq\nuklEHgfSVXUJTrNTFPCWkw2dfao6T1XzReQ/cQIOwOOqmt+qO+us4oc6jx96jCQePKsjSmKMMT5r\nMUU5gIgkANNx+hK+VtVcfxestbpEinK3wixnFBQ4K5/1TLF8UN3EuswCth4o4sop/RH7nZouwNcU\n5b7Ms7gEWKqq/3K9jhWRi1X1vTYo58mpR7+WjzFdjqryH2+uY1dOKesyC/ivi8cQFNjqea/GdEq+\n/CU/oqqF7heuzu5H/FckY7qmL3fmsiunlOmD41iclskdi9ZQUV3b0cUypk34Eiy8HeNLX4cxJ5VX\nV2aQEBXCn38wlUcvHMUnWw5xw5++paCsqqOLZswJ8yVYpIvI0yIyREQGi8hvgRYWkjbm5LI3r5Sl\n2w5zzdQBhAYFctOMQTx71UTWZRZw2R+/IjO/rOWLGNOJ+RIs7gaqgDeAt4AK4E5/FsqYrua1r/cS\nKMK10+tzYXLh+L785Zap5BRXcskfVrI+q6CZKxjTubUYLFS1VFUfcE1+m6yqP1fV0vYonDFdQWll\nDW+mZXL+uD70iglrsG/a4Hj+fsdphAUHcuUL37DjUHEHldKYE+PLaKhE4D6cxY7q/yeo6pl+LJcx\nncKXO3L5YkcO8VEhxEeGEhocQH5pFXklVYQEBRAXGcL2Q8UUV9Zw02kpXq8xNCmad24/je//7nMe\nfHcDb9x2KgEBNqzWdC2+dFQvwmmCugD4EXAjPqQoN6aryyup5I5FqymqqGnx2IkDYpk4oGeT+3vF\nhPHgeSO57+31vJmeyVVTW1gj2phOxpdgEa+qfxKRe1R1BbDCYza3Md3Wbz7eRllVLZ/8dCZ9YsPJ\nK6mkorqOuMgQekYEU1On5JdWkV9aRb+eLa+dfvnkfry9OotffbCFOSN7kRgd2g53YUzb8KWDu9r1\neEBEznflc7JZZaZb+y6zgMVpmdx0WgrDekUTFRrEwPhIhveOJjE6lKDAAMKCA+kbG86Y5B7ERrS8\ncJWI8KtLxlJeXcsT729u8XhjOhNfgsV/iUgP4D+AnwELgZ/6tVTGdKC6OuWXSzYRHxnKPWcNa9Nr\nD02K4vZZQ3lvXTavrNyDL+l2jOkMfFn86F+up4XAbP8Wx5iO9/bqLL7LLODpK8YTHRbc5te/c/YQ\nNmcX8tjsCsQzAAAgAElEQVQ/N7NhfyG/umQsYcGBbf4+xrSlViWuEZE1/iqIMZ1BcUU1T320lckD\ne3LJRP8szhgaFMiL16fyk7OG8fc1+5n//FdkHbFJe6Zza22WMxvvZ7q13y/bSW5JFY9cOMqvWWMD\nAoSfnHUKL92Qyt7cMi74vy9Ztu1wyyca00FaGyze90spjGkHGbml3PpaOlsPFnndvy+vjFe+zOCy\nSf0Y1y/W6zFt7exRvVhy9+n0jgnjB6+m8fTH26ita74f43BxBdsOFrM7p4TM/DJqauvapazm5Naq\nhICq+gt/FcQYf3vy31v5ZPMh0jLy+est0xiT3KPh/g+3EBgg3Dd3eLuWa1BCJO/eMYNf/mMjzy7d\nSfreI/zuygkkNZoNXlunvPD5Ln77yXaqa7XB+U9fMb5+nkd+aRX/+a/NrN13hEfnjWbWcF9WQTam\neT4tfnTMSSIbVHWsH8pz3LrU4kem3X2XWcBFz63k6qn9+Xx7LkUV1bx681QmD3Q+YFftzuPKF79h\nwdmn8OM5bTsCqjXeTM/kl//YSFRoEL++fDwT+8cSECAcKqzgvnfWs3ZfAeeO6c0F4/pSU1dHUUUN\nf1y2k0PFldw5eyhDEiN57J+bKa6opnePMDLzy7kitR+/uGAUEcGBHCyqoGdECJGhvn9PVNVWNcll\n5JZSWF5NYIAQGCAMTowkNKjpDnxVpaDMGaEfECDU1Sm5JZXkFFfSJzacQQmRPr93U6pq6ggQbH0R\nL3xd/KjJYCEilzZ1DvC8qiaeQPnanAUL05zr/7SKTdlFfH7fbArLq7n2pW84XFzJmL49CAiAPbml\nBIiw9D9mER7SsSOTth8q5q6/rWH7oZIG22PCgvjPi8cwb3zfBh/eheXVPLZkE39fux+A8f168D/z\nx5ESH8kzn+3ghRW7CA0KpLKmljqFuMgQfnnBKC6a0LfZIFBRXcvzK3bx0ue7OW9sH+4/dwQJUU1P\nJMzML+PJD7fy/voDDbYnx4Zz/7kjuHBcn2Pe79s9+Tz57y2s2ec9yaIIXDIxmZ+edQr94yKafO+m\n5JdW8fKXe/jzVxmEhwRy28zBXDttYIf/jjuTtggW1TipPrwdMF9Vo0+siG3LgoVpyte78rj6pW/4\nxfkj+eEZgwE4XFTB4//aTF5JFbWu/wM/mTOM04YmdGRR65VX1fLP9dmUVtZQW6cEiHhNVOjp400H\nySmp5KopAwj0yD21dt8R3l6dRVxkCL1iwnhnTRZr9xXwvVMSeeKSMfTreeyH8NKth3h0yWb25Zcx\nbVAcq/ceISIkkHu/P5xrpg1scP2K6lqe/WwHC7/cQ4DAbWcMZnz/WGrrlNKqGl76fA+bDxQxcUAs\nl05MJiBAEIRPtxxi6dbD9IoJ5frpA4kMDaq/1/ioEBKjQlmxPYdXv8qgTpVrpg7gztlDj2me8yan\nuJKXvtjNX77eS0VNLXNH96agrJqvd+eREBXCqzdPPaYZ8mTVFsFiNXCjqm70si9TVfufeDHbjgUL\n442qctkfvyK7oILl986y+Qw4fR9/+TqDpz7ahiosOPsUbp6RQlBgANkF5Ty6ZBMfbz7E0KQoHr9o\nNKcNSWDn4WIefm8TX+/OY3TfGB6dN5opKXEcKqrgtr+s5rvMAi6dmMy9c4fTp0f4Me/3zuosfv3x\nNnKKK+u3x4QFcfusodx0Wkqz3/QPFlbwzGc7eDM9k+BA4YZTU7j1jMHERgSjCori/hg7UlbFwi/2\nsGjVXqpq6rhoQjJ3zh7C0CTnu21aRj53/20tkaGB/OvuM3yqYWQdKSOvpIpx/Xr4ZYRc1pEy3kzP\nIqe4on7bsKRoLpvUjx4RbT/Pp7G2CBZnAHtVdZ+Xfamq2qk+mS1YdC91dUpZdS21tUqtKlGhQYQE\n+d7evCm7kK935ZGecYQPNx3kV5eM5ZpplrzPU9aRMh5dsolPtxxmZJ8Yzh7Vi4Vf7KZOlXvmnMIt\npw9q8G+uqvxr/QF+9cEWDhRWcN7Y3qRnHKGksoanr5jA3DG9m32/qpo6Csqq6psqeoQHtyp4Z+SW\n8uxnO3h33X6a62oNDBAunpDMXWcO9drfsXJnLtcuXMVNp6Xw6LzRTV5nc3YRz6/YxfsbDlBbp4xJ\njuHWMwZz/tg+Dfo+yqtqySmuJDosiJ6RLad9cUvLyOdPX+zh480HAeqb+OoUcksqCQsO4MJxfblu\n+kC/BSpog2DR1Viw6D5KK2u4duEq1mUebcce0TuaD358hk+pvd9Kz+Tet9cD0LdHGN8bnsjjF40h\n2Do3j6GqfLTpEI8u2cTBogrmjEji0Xmjm+0fKKuq4Y/Ld/HC57vpFRPKSzekMqJ3TLuVecehYj7e\nfOiYjncRCA4I4OxRvUhpoVP88X9u5uWVe3jtB1M5Y1gCX+7M5cXPd5N1pBxVpaZOyTpSTmRIINdO\nH8iAuAheXrmH3Tml9V9c6lSprqmjtMpZZz04ULhwfF9+MGNQs01cqsr/Ld3J059sp2dEMFdNHcB1\n0weSHHu0RrYpu5C/frOP99bup7y6ljHJMVwzdSCDEyOpU6cmNaxXFEnRLTfJtaQtaha/AP6gqvlN\n7D8TiPBIB9KhLFh0D7V1ym2vpbNs22Humj2UHhEhZOaX8epXGTx/3eQWv73+Y91+fvLGOmYMSeDp\nK8e3yX+mk0FJZQ07D5cwvhXfYHOKK4kICWzVyKrOoqK6lgv/70sKy6vpExvOd5kF9OkRxuSBPQkM\nEAQ4pXc0104dWN8UVFenfLb1MF/syEEVAgQCAwKc/pXoUDZnF/FmeiZlVbUMTogkKFCorVMiQ4O4\neEIy81P7ER4cyIN/38Bbq7O4dGIyv7q0+VQvRRXV/GPtfhat2sfWgw0XzgoMEOaMSOLqqQOYeUpi\ng36k1miLYHERzqJHFcAanDUswoBhwATgU+BXqtop1rawYNE9/Ne/NrPwyz08ftFobjg1BYCa2jrm\nPL2C2IgQ3rvjtCY/zP694QB3vb6WyQN78uebp9qIF9OsjfsLufSPX9ErJpQ7Zg3l0knJzQ7x9UVR\nRTVvpmWSlpFPgAgBImQVlPNdZgERIYH07xnBtkPF3DNnGD85a5jPgVlV2XygiMLyagJEqFNlxfYc\n3lmdRa6rP+Ufd844rqaqNmuGEpFhwAygD1AObAE+V9XyVpfKjyxYdG1VNXW89nUG//X+Fq9tyYtW\n7eWhdzfyt1uncdoQZ8TSZ1sO8eg/N1FYVk1FdR1VtXVMGhDLa7dMI6oLfts17S+3pJLY8GC/z7/Y\nkFXIa19nsGzbYe6fO4LLU9tmfFBVTR2fbTlESWXNcV/T+ixMp1BXpxRX1FBQXkVheTVF5TXOY0U1\nxRXO6+2Hilm5M5fSqlpmDU9k4Q2px/znraiu5YynljGidzR/uWUaq/fmc81LqxiUEMn0wfGEhwTW\nt//G+CFTrDHdla/Bwr5+Gb/IL63iuoWr2HKwqNmRKwECfXqEc9HEZM4cnsT3hid6/ZYXFhzILacP\n4sl/b+W9tft5ZMkm+saGs+iH04hvZqKYMaZtWLAwbU5VefDvG9hxuJjbvzeE+KhQeoQHExseTEx4\nMD3Cg4kOC6JHeDARIYE+t7NeO20Azy3byU/eWEdidCiv/WCqBQpj2kmzwUJEAoEfq+pv26k8pht4\nZ81+Ptx0kPvnjuD2WUPa7LrRYcH8v5mDefHz3bxy05TjSv9gjDk+zfbqqGotcNHxXlxE5orINhHZ\nKSIPeNk/U0TWiEiNiMxvtK9WRNa5fpYcbxla67xnvmDhF7vb6+26ncx8Z6LX1JQ4bps5uM2vf+fs\noaT94ixL1WBMO/OlGWqliPweeAModW9U1WZXzXPVSp4DzgaygDQRWaKqnivV7wNuwlnbu7FyVZ3g\nQ/naTG2dsuVgETsP2wfR8aitUxa8uQ4B/veK8cc97rs5InLCwxuNMa3nS7A4zfX4uMc2Bc5s4byp\nwE5V3Q0gIotxain1wUJVM1z7OsXqLYXl1ahCmWtGpmmdV1buIS3jCP97+XhrIjKmm2kxWKjq7OO8\ndjKQ6fE6C5jWivPDRCQdqAGeVNX3jrMcPssvrQKcdAamdfbmlfKbj7cxZ0QSl07yz9rVxpiO02Kw\nEJEewCPATNemFcDjqlrY0qletrVmUscAVc0WkcHAUteCS7sale024DaAAQNOPElcQZkTLEorrWbR\nGnV1yv3vrCc4IIAnLhnr17WrjTEdw5dpiy8DxcAVrp8i4BUfzssCPKcU9gOyfS2Yqma7HncDy4GJ\nXo55UVVTVTU1MfHE12Kqr1lUW7BojdfT9vHN7nweOn8kvXtYLiZjuiNf+iyGqOplHq8fE5F1PpyX\nBgwTkUHAfuAq4BpfCiUiPYEyVa0UkQScdCNP+XLuiTjiqlmUVVozVEsOF1ewObuIzQeK+MOyXcwY\nGs+VUzrVEifGmDbkS7AoF5HTVfVLABGZgZMjqlmqWiMidwEfAYHAy6q6SUQeB9JVdYmITAHeBXoC\nF4rIY6o6GhgJvODq+A7A6bPY3MRbtZkjrnWArYO7eZ4pwAGGJUXx5KXjrPnJmG7Ml2DxI+A1V98F\nwBHgRl8urqofAB802vZLj+dpOM1Tjc/7Chjry3u0pSPWwe2TTzYfom+PMJ6+cgIje8e0y2pexpiO\n1dIM7gBguKqOF5EYAFUtapeSdQB3n0Wp1SyaVFenfJuRz9kjezF9cHxHF8cY005amsFdB9zlel7U\nnQMFHO2zqKqpo6a2U0z96HR2HC6hoKyaqYPiOrooxph25MtoqE9E5Gci0l9E4tw/fi9ZB3D3WYCN\niGrKqj15AFarMOYk40ufxQ9cj3d6bFOg7RP/dDB3nwVAWWVtt1sX4d8bDhAfFXpCtYJVe/Lp0yOM\nfj3DWz7YGNNt+NJncZ2qrmyn8nSo/LIqYsKCKKqo6Xad3HV1ygN/30C/nuG8/+MzfDpHVSkqr6nv\nwFZVVu3O5/Sh8TbyyZiTTLPBQlXrROQ3wKntVJ4OU1unFJZXM6J3DEUHirrd8Nnth4spLK+msLya\nrCNl9OvpPXeTqvLN7nw+2XyIz7YeYm9eGU9dNo4rpvRnT24puSWVTLMmKGNOOr70WXwsIpdJN/8q\n6U4i6G5e6W7BIm1Pfv3zTzYfavK4177ey9UvfcNfV+1lcEIkkwf25KH3NpCWkc8q1zWsc9uYk48v\nfRYLgEigVkTKcXI+qarG+LVk7cw9bDY51gkWpd2sGerbjCP0jgkjKiyIjzcd4uYZg445pqK6lt8v\n28m0QXG8cvMUIkKCKCyr5uI/rORHf1nN8N7RJESFMjghsgPuwBjTkVqsWahqtKoGqGqwqsa4Xner\nQAFHh83W1yy6UTJBVSVtTz5TBsVxzqhefJuRX5800dPfVu0jp7iSBWefQkSI8z2iR0QwL92QSlVN\nHV/tymPaoDjrrzDmJNRisBDHdSLysOt1fxGZ6v+itS/3SKijzVDdp2aRdaScg0UVTE3pyTmje1Nb\npyzderjBMRXVtTy/YhfTB8cd0ycxNCmKZ6+ZSIDAGcMS2rPoxphOwpc+iz/gdHC7kwCW4KyA1624\naxbJsU7Hb3fqs/jW1dcwZVAc45J70CsmlI83Ney3WPztPg4XV/LjOcO8XmP28CS+/vkcLk+1ZIHG\nnIx8CRbTVPVOoAJAVY8AIX4tVQfIL3Um5CX37H59FmkZ+cSEBXFKUjQBAcJZI3uxYnsOFa6JhxXV\ntfxxxS6mpsRxajMjnXrFhPllqVRjTOfnS7Codq2nrQAikgh0u1wYBWVVhAYF0DMiGBEo7041i4x8\nUlPiCHB90J8zujfl1bWs3JnLjkPF3Pv2eg4VVXLPWcOsP8IY45Uvo6GexUkjniQiTwDzgV/4tVQd\nIL+0irjIEESEyJCgbrNaXm5JJbtzSrl88tHmo1MHxxMdGsS9b68nv9QJkrfNHMxpQ2z+hDHGO1/W\n4F4kIquBOTjDZi9W1S1+L1k7O1JWRWyE07oWERJIeXX3aIZKz3DPjehZvy0kKIBLJyXz6ZbD3Pv9\n4Vw9dQBxkd2uZdEY04Z8qVmgqluBrX4uS4dyahZOWouIkMBuU7P4ds8RQoMCGJsc22D7YxeN4bGL\nOqhQxpgux5c+i5NCQVk1PetrFkHdZuhsWkY+E/rHEhJkv2pjzPGzTxCX/LIqj2AR2C2Gzi78Yjcb\n9hfa3AhjzAnzqRmqu3MnEezparePCA2isLy6hbM6t98v3cFvPt7OeWN7c9vMIR1dHGNMF2fBgqNJ\nBONcqbgjQwI5UFDewaU6PiWVNTz57y389Zt9XDIxmV/PH0dQoFUgjTEnxoIFR5MIumsW4V20Geqz\nLYd4+L2NHCiq4IenD+Ln5420SXTGmDZhwYKjqT7cfRaRXbCD+38/3sb/Ld3J8F7R/P7aSUwa0LPl\nk4wxxkcWLDiaRNA916ArdnC/vTqLM4Yl8Kcbp9jIJ2NMm7NPFTxqFpFHh85W1tRRU9s1sprklVRy\noLCCM4YlWKAwxviFfbJwNIlgT3cHd2ggAGXVXaN2sSm7CIAxfXt0cEmMMd2VBQucmkVoUADhwU6Q\nCA9xHrtKMkF3sBjVt9utSWWM6SQsWOD0WbiTCILTwQ1QWtk1Ork3ZhfSr2d4fW4rY4xpaxYsaJhE\nEI7WLDw7uatq6ijppMFjc3YRo61WYYzxIwsWNEwiCEdrFp7B4lcfbOHSP6xEVdu9fM0prqhmT26p\n9VcYY/zKr8FCROaKyDYR2SkiD3jZP1NE1ohIjYjMb7TvRhHZ4fq50Z/l9EwiCBDh6uD2XC1vx+Fi\nth8qYXduqT+L0mpbDhQDMDrZahbGGP/xW7Bwra73HHAuMAq4WkRGNTpsH3AT8LdG58YBjwDTgKnA\nIyLit1lm+WVVDdZziPDSwZ1TXAnAim05bfreJ9ovsnF/IWAjoYwx/uXPmsVUYKeq7lbVKmAx0GAF\nBVXNUNX1HLtM6/eBT1Q137Xm9yfAXH8Usqa2jsLy6gZ9Ft46uA+7gsXy7W0XLPJLq5j4n5/wl2/2\nHvc1NmUXkRAVSlJMWJuVyxhjGvNnsEgGMj1eZ7m2+fvcVmmcRBCO1izcfRaVNbUUlFUTEhjAqt15\nVLTR/IvM/DKqaur4zUfb6meRt9am7ELGWBOUMcbP/BksvGWw87V32KdzReQ2EUkXkfScnOP7xh8Z\nGsQL109m9oik+m0RjTq43U1QZ41KorKmjm925x3XezWWV+pct7C8mt99ur3V51dU17LjcImNhDLG\n+J0/g0UW0N/jdT8guy3PVdUXVTVVVVMTExOPq5BhwYF8f3RvBsZHemwLQIT6ZILuJqgLx/UlNCiA\nFW3UFJVb7NQmZg9P5K+r9rHjUHGrzt9+qJjaOrX+CmOM3/kzWKQBw0RkkIiEAFcBS3w89yPgHBHp\n6erYPse1rV2ICBHBgcfULPrHRTB9cHybdXLnlDjXfeKSsUSEBPJf729p1fkb9zszt0dbsDDG+Jnf\ngoWq1gB34XzIbwHeVNVNIvK4iMwDEJEpIpIFXA68ICKbXOfmA/+JE3DSgMdd29pNRGjQMTWLxOhQ\nvndKIrtzS9mXV3bC75FbUklUaBB9Y8O59YzBrNiew6GiigbH1NUpH2w44LWfZFN2IdFhQfSPCz/h\nshhjTHP8Os9CVT9Q1VNUdYiqPuHa9ktVXeJ6nqaq/VQ1UlXjVXW0x7kvq+pQ188r/iynN5EhgZRW\numoWRRWIQHxkCLOGO81dy7cfPuH3yC2pIiHKGYU18xTnumkZDWPiyl253LFoDb94b+Mx52/YX8jo\nvjH1aUqMMcZfbAZ3E8JDguqboQ4XVxIfGUpQYACDEiIZmhTFknW+dr80Lbe4koSoUABG940hPDiQ\ntD0Ng8WXO3MBZ72Kt9KPDhB7btlO1mcV1gcZY4zxJwsWTYgMCaxvhsopriQp2vlQFxEumZhM+t4j\nPjdF1dYp+72s6Z1XWkm8q2YRHBjAxAGxpGUcaXDM17vymDQgllMHx/PwPzay/VAxr6zcw68/2sYl\nE5P50cwhJ3KbxhjjEwsWTQgPCaTUo2aRFBNav+/iic6Uj3fX7vfpWm+vzmT2b5ZTUNZwLoXTDHX0\nulNS4thysIiiCmd9jcKyajbuL+SMYYk8c/UEokKDuXbhKh7752a+P7oXv54/jgBbY9sY0w4sWDQh\nMiSI8voO7or6mgVAcmw40wfH8e7aLJ8SC67LLKCqpo69HjWRmto6jpQ1DBZTB8WhCqv3OrWLb/bk\nUadw2pB4kqLDeOaqCeSWVDLzlESevXoiQYH26zPGtA/7tGlCRKjTwV1bp+SWVJHoESwALp3Yj4y8\nMtZmFrR4rW0HnfkT2R5NUfmlVahCgsd1Jw6IJTBASHd1cn+9K4+w4AAmDnDSYs0YmsCKn83mTzem\nEhoUeML3aIwxvrJg0YSIkEDKq2vJL62itk5Jim6Ye+ncsb0JDQrg3TXNN0WpKtsPlQA06Ldwz7FI\njPJMYBjEmL4xpO1xahYrd+YyJSWuwbraA+IjCLYahTGmndmnThMiQ4Ioraypn5CX1KhmER0WzNmj\nevHP9dlU1TTOg3jU/oLy+kWTPINFbonTf+HZDAVOv8W6rAKyjpSx43AJM4YmtMn9GGPMibBg0YTw\nkEAqa+o4WOR8wHt2cLtdOimZgrJqlm9res7FdlcKjwCB/UeOBos8V80ivnGwGBRHVU0dL6zYDTj9\nFcYY09EsWDTBnaY8I9fplG7cDAVwxrBEEqJCWJyWecw+t20HnSaoiQN6kl3oWbNwgkVCVMN1s6ek\nxAHwRlomMWFBlsrDGNMpWLBognu1vIw8Z2W8xh3c4MyNuH56Cku3Hq7vxG5s28Ei+vYIY3jv6AY1\ni9ySKkKDAogKDWpwfFxkCEOToqiqrWP64HgCbWisMaYTsGDRBPeaFhl5ZUSHBREW7H300Q2nDiQ8\nOJAXPt/ldf+2QyWc0jua5NhwjpRV10/0c8/e9paqY0qKM/rJmqCMMZ2FBYsmuNe02JtXekzntqee\nkSFcNbU/S9ZlHzNLu6a2jl2HSxjeywkWcHT4bE5JZYNhs55mDkskMEAslYcxptOwYNEEd80i60i5\n1/4KTz88YzAAf/piT4PtGXllVNXWMbx3NMk9nWCxv8DJKptXUtVg2KynuWN689UDZzI4MeqE7sEY\nY9qKBYsmuGsWtXXqdSSUp+TYcOZN6Mvr3+5rsDyqux/jlF7R9HXVLNz9FrklTnJCb0SEXramtjGm\nE7Fg0YTI0KN9FM01Q7n96HtDKK+u5bWv99Zv23aomACBoUlR9IoOJTBA2F9QRl2dkldaRUK095qF\nMcZ0NhYsmhARfHSUkreRUI2d0iuas0b2YuGXu+uHxW4/WExKfCRhwYEEBQbQOyaM7IIKCsqrqa3T\nYybkGWNMZ2XBogkRDWoWvjUJPXDuCMqravnfj7cBzoS84b2j6/cnx4az/0i5xxwLCxbGmK7BgkUT\n3B3c4FszFDjNTTeelsLitExW780nI6+UU3odDRZ9Y8PYX1BObrEFC2NM12LBoglhQYG4p0C01MHt\n6cdzhtEzIoS7/7aWOqVhzaJnOAeLKjhU7IyISrQ+C2NMF2HBogkBAUKEayJeoo/NUAA9woP52TnD\nyS50AkLDmkU4tXXK5uwigCZHQxljTGdjwaIZ4SFBhAYFEBMW1PLBHq6c0p9RfWIIDQogJT6ifrt7\nYt53WYUEBQg9woPbtLzGGOMvrfsUPMlEhgYSFhzgNSVHcwIDhOevm8yu3JIGq9m5g8XG/YXER4XY\nkqjGmC7DgkUzwoMDG3R0t8aA+AgGeNQqgPqJeWVVtQxKiDzh8hljTHuxYNGM+ZP7EX6cwcKbyNAg\nYiOCKSirtpFQxpguxYJFM9w5n9pScmy4BQtjTJdjHdztzN0UZak+jDFdiQWLdubu5E6wYbPGmC7E\ngkU7S7aahTGmC7Jg0c7c61pYn4Uxpivxa7AQkbkisk1EdorIA172h4rIG679q0QkxbU9RUTKRWSd\n6+d5f5azPZ0xLIFbzxhE6sC4ji6KMcb4zG+joUQkEHgOOBvIAtJEZImqbvY47BbgiKoOFZGrgP8B\nrnTt26WqE/xVvo4SHRbMQ+eP6uhiGGNMq/izZjEV2Kmqu1W1ClgMXNTomIuAP7uevw3MkdZOlzbG\nGON3/gwWyUCmx+ss1zavx6hqDVAIxLv2DRKRtSKyQkTO8GM5jTHGtMCfk/K81RDUx2MOAANUNU9E\nJgPvichoVS1qcLLIbcBtAAMGDGiDIhtjjPHGnzWLLKC/x+t+QHZTx4hIENADyFfVSlXNA1DV1cAu\n4JTGb6CqL6pqqqqmJiYm+uEWjDHGgH+DRRowTEQGiUgIcBWwpNExS4AbXc/nA0tVVUUk0dVBjogM\nBoYBu/1YVmOMMc3wWzOUqtaIyF3AR0Ag8LKqbhKRx4F0VV0C/An4i4jsBPJxAgrATOBxEakBaoEf\nqWq+v8pqjDGmeaLauBuha0pNTdX09PSOLoYxxnQpIrJaVVNbOs5mcBtjjGlRt6lZiEgOsLfR5gQg\ntwOK40/d7Z662/1A97un7nY/0P3u6UTuZ6CqtjhCqNsEC29EJN2X6lVX0t3uqbvdD3S/e+pu9wPd\n757a436sGcoYY0yLLFgYY4xpUXcPFi92dAH8oLvdU3e7H+h+99Td7ge63z35/X66dZ+FMcaYttHd\naxbGGGPaQLcNFi0tvNQZicjLInJYRDZ6bIsTkU9EZIfrsadru4jIs677Wy8ikzqu5N6JSH8RWSYi\nW0Rkk4jc49rele8pTES+FZHvXPf0mGv7INcCXjtcC3qFuLZ7XeCrsxGRQFeW53+5Xnf1+8kQkQ2u\nxdPSXdu68t9drIi8LSJbXf+fTm3v++mWwUKOLrx0LjAKuFpEusKKQ68CcxttewD4TFWHAZ+5XoNz\nb8NcP7cBf2ynMrZGDfAfqjoSmA7c6fo9dOV7qgTOVNXxwARgrohMx1m467euezqCs7AXeCzwBfzW\nddMpgLYAAAfHSURBVFxndA+wxeN1V78fgNmqOsFjSGlX/rt7BvhQVUcA43F+V+17P6ra7X6AU4GP\nPF7/HPh5R5fLx7KnABs9Xm8D+rie9wG2uZ6/AFzt7bjO+gP8A2flxG5xT0AEsAaYhjMhKsi1vf7v\nDyc32qmu50Gu46Sjy97oPvrhfNicCfwLZ+mALns/rrJlAAmNtnXJvzsgBtjT+N+5ve+nW9Ys8G3h\npa6il6oeAHA9Jrm2d6l7dDVXTARW0cXvydVksw44DHyCk0K/QJ0FvKBhuZtb4Kuz+B1wH1Dneh1P\n174fcNbF+VhEVouz7g103b+7wUAO8IqrqXChiETSzvfTXYOFLwsvdXVd5h5FJAp4B/iJNlrAqvGh\nXrZ1untS1Vp11ofvh7N88Ehvh7keO/U9icgFwGF11o2p3+zl0C5xPx5mqOoknCaZO0VkZjPHdvZ7\nCgImAX9U1YlAKUebnLzxy/1012Dhy8JLXcUhEekD4Ho87NreJe5RRIJxAsUiVf27a3OXvic3VS0A\nluP0x8SKs4AXNCy31wW+2rekzZoBzBORDGAxTlPU7+i69wOAqma7Hg8D7+IE9a76d5cFZKnqKtfr\nt3GCR7veT3cNFr4svNRVeC4QdSNOu797+w2ukQ/TgUJ3lbSzEBHBWbNki6o+7bGrK99ToojEup6H\nA2fhdDYuw1nAC469p2MW+Gq/EjdPVX+uqv1UNQXn/8lSVb2WLno/ACISKSLR7ufAOcBGuujfnaoe\nBDJFZLhr0xxgM+19Px3deePHTqHzgO047ckPdXR5fCzz6zjrj1fjfDu4Bac9+DNgh+sxznWs4Iz4\n2gVsAFI7uvxe7ud0nOrvemCd6+e8Ln5P44C1rnvaCPzStX0w8C2wE3gLCHVtD3O93unaP7ij76GZ\ne5sF/Kur34+r7N+5fja5//938b+7CUC66+/uPaBne9+PzeA2xhjTou7aDGWMMaYNWbAwxhjTIgsW\nxhhjWmTBwhhjTIssWBhjjGmRBQvTpYjIRBFZ2Mz+viLyth/f/3JX1s9ljbaniMg1x3nNr3w4ZqE/\nkmGKyKMi8rMWjrnYl/cWkbtE5Oa2K53pTCxYmK7mQeD/mtqpqtmqOr+p/W3gFuAOVZ3daHsK4DVY\neMyE9kpVT2vpTVX1h6q62ddCtrGLcbI3t+Rl4Md+LovpIBYsTIdwfRPf6vrGvFFEFonIWSKy0pWf\nf6qXc6KBcar6nev191zrFaxzJViLdl13o2v/Qo/9OSLyiGv7vSKS5sr1/1gT5btanPUQNorI/7i2\n/RJnouHzIvLrRqc8CZzheq+fishNIvKWiPwTJ6FdlIh8JiJrXNe9yOO9SlyPs0RkuRxdt2CRaxY8\nru2p7uNF5Alx1tT4RkR6ubYP+f/tnV2IVkUYx3//8EKlTcHNSC8KNkowL2LrojBFCO/UwgJjCUQI\nhDJM9CKsmxDzA7oSrDtvFAU3E40o17Ak/KqldS/ci6AUIaibxNRqdR8vnjk2Hc57Tsq6274+Pzi8\nM+88M2fOeV/mOTNz5j8pfkbS+0W5Fde2Ub7XSx/wRPb96ynvgKReSVMlPQcsBbana+uqsgMws6vA\nz1W/XdAGjPfKxDjuzQN/Er8OzMMfWr7Hn0wFLAM+rcizCOjN4odwwTiA+3HBtUfJJN5T2iPAUPpc\njO9XrHTew8CCkv0s4ALwYCrzK+DFlHaMihWxZKufU3wlvgq/WFU7CXgghTvxFdDFotg/sjIu4Vo+\n9wEngPnl8+Kr4pek8Dbg3RQ+TJKmBlYX5Zbq2Y2v6p2KS1//CKxPaTMyu03AmhTeBbycpVXapfhG\nfA+Tcf+PxTG6R/QsgvHkJzMbNLMRXJbhqHmLM4g3+mUexqWaC74FPpT0FjDd/pHUvoWkQp7iTTM7\njzuLxbhkRz8wB98kJucZ4JiZ/ZbK3A3UqZa24oiZFSJ7AjZLOgv04ZLRD1XkOW1mF9M9+YHq+/A3\n7hjAnWxh8yx+rQB7WtTpeeCAmV01VwDONdOelHRc0iDQA8xtUUad3a+4sw3ajNqx1CC4y/yVhUey\n+AjV/81ruDYRAGa2RdJnuN7USUkvAH+W8nwEfGJmfSku4AMz+7imXlUSz3fClSzcg/dUus1sWK7y\nOrkiT35PblB9H4aTU62zqaOVxs8uvAc1IGkl3tO5XbvJ+O8UtBnRswgmEueAx4qIpK7UM9mKi6zN\nyY0lvQF0mNmW7OsvgFXyPTaQNFvSTP7NKWChpE75Fr2vAl831O0y0FGTPg3fN2JY0iJ8SGy0OQks\nT+EVLWy+AV6SNCXNAS3J0jqAX+Sy8j3Z9+Vra2UH8DgusBi0GeEsggmDmQ0B01IjB7A2TUAP4E+z\nn5eyrAfmZZPcq83sS3yI5kQaRtlPqZE3l3N+B5fpHgD6zewg9ZwFrqdJ37cr0ncDT0v6Dm9gh/7r\ndd8Ga4F1kk7jQ3aXygZm1g/sw4e4eoHjWfJ7uKM8UqrfXmBDeomgq8YOfH+MPoK2I1RngwlFaogv\nm1nLtRb3KumtpGtmZpJW4JPdy5ryjeL5nwLWmdlrY3XOYOyIOYtgorETeGW8K/E/pRvYkV63/R1Y\nNcbn78R7HUEbEj2LIAiCoJGYswiCIAgaCWcRBEEQNBLOIgiCIGgknEUQBEHQSDiLIAiCoJFwFkEQ\nBEEjNwGCwMbE0+n7TgAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# first we split the training set into a training / validation set (as we are using k-folds cross validation elsewhere and don't have an explicit training and validation set.)\n", "X_train_train, X_cv, y_train_train, y_cv = model_selection.train_test_split(X_train, y_train, test_size=0.25, stratify=y_train)\n", "\n", "# set the model to calculate learning curves for\n", "#model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", "\n", "m_array = np.round(np.linspace(20, X_train_train.shape[0], 100)).astype(int)\n", "train_acc_array = []\n", "cv_acc_array = []\n", "\n", "for m in m_array:\n", " model = svm.SVC(C=bestC, kernel=kernel, gamma=gamma)\n", " # we now fit to the training data\n", " model.fit(X_train_train.head(m), y_train_train.head(m)) # training on the first m training data examples\n", " train_accuracy = model.score(X_train_train.head(m), y_train_train.head(m))\n", " train_acc_array.append(train_accuracy)\n", " cv_accuracy = model.score(X_cv, y_cv)\n", " cv_acc_array.append(cv_accuracy)\n", "\n", "train_acc_array = np.array(train_acc_array)\n", "cv_acc_array = np.array(cv_acc_array)\n", "\n", "fig, ax = plt.subplots()\n", "ax.plot(m_array, 1-train_acc_array, label='train')\n", "ax.plot(m_array, 1-cv_acc_array, label='cross-validation')\n", "ax.legend()\n", "ax.set_xlabel('m (size of training data)')\n", "ax.set_ylabel('error (1-accuracy)')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This learning curve suggests that the Gaussian SVM is a high variance algorithm for this problem and amount of data. Getting more data will help improve the fitting, adding more features will just increase the variance and amount of data needed to fit. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Neural Network" ] }, { "cell_type": "code", "execution_count": 136, "metadata": { "collapsed": false }, "outputs": [], "source": [ "hidden_layer_sizes=(10,10)\n", "alpha=1e-1\n", "\n", "nn_model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)" ] }, { "cell_type": "code", "execution_count": 137, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mean score: 0.7925\n", "variance in score: 0.003131250000000002\n" ] } ], "source": [ "N_splits = 10\n", "kfold = model_selection.KFold(n_splits=N_splits)\n", "\n", "scoring = 'accuracy'\n", "score = model_selection.cross_val_score(nn_model, X_train, y_train, cv=kfold, n_jobs=1, scoring=scoring)\n", "\n", "print(\"mean score: {}\".format(score.mean()))\n", "print(\"variance in score: {}\".format(score.var()))" ] }, { "cell_type": "code", "execution_count": 138, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "MLPClassifier(activation='relu', alpha=0.1, batch_size='auto', beta_1=0.9,\n", " beta_2=0.999, early_stopping=False, epsilon=1e-08,\n", " hidden_layer_sizes=(10, 10), learning_rate='constant',\n", " learning_rate_init=0.001, max_iter=200, momentum=0.9,\n", " nesterovs_momentum=True, power_t=0.5, random_state=1, shuffle=True,\n", " solver='lbfgs', tol=0.0001, validation_fraction=0.1, verbose=False,\n", " warm_start=False)" ] }, "execution_count": 138, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", "\n", "nn_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 139, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.7865168539325843" ] }, "execution_count": 139, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_model.score(X_test, y_test)" ] }, { "cell_type": "code", "execution_count": 140, "metadata": { "collapsed": true }, "outputs": [], "source": [ "prediction = nn_model.predict(data_test)\n", "\n", "with open('prediction_submission_nn.csv', 'w') as file:\n", " print('PassengerId,Survived', file=file)\n", " for i, id_ in enumerate(data_test.index):\n", " print('{},{}'.format(id_, prediction[i]), file=file)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For this we get 72% performance on the Kaggle test data. Lets take a look at the learning curve for this neural network to see if we can figure out what's going on." ] }, { "cell_type": "code", "execution_count": 141, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 141, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXl4VNXd+D8newhZyAaBEDbDDmEJuxugiBt1X6q4Vlu3\n2tq61VqXvq22tbbaH75uVV+tG4oLLqigIEpBCVvYd0hCEggJ2chClvP74zs3c2cyk0zCDAlwPs+T\nZ2buPffcMwmc7/3uSmuNwWAwGAwtEdTRCzAYDAZD58cIC4PBYDC0ihEWBoPBYGgVIywMBoPB0CpG\nWBgMBoOhVYywMBgMBkOrGGFhMBgMhlYxwsJgMBgMrWKEhcFgMBhaJaSjF+AvEhMTdd++fTt6GQaD\nwXBcsWrVqoNa66TWxp0wwqJv375kZWV19DIMBoPhuEIptdeXccYMZTAYDIZWMcLCYDAYDK1ihIXB\nYDAYWuWE8VkYDIbAUFdXR15eHjU1NR29FMNREBERQWpqKqGhoe263ggLg8HQInl5eURHR9O3b1+U\nUh29HEM70FpTXFxMXl4e/fr1a9ccxgxlMBhapKamhoSEBCMojmOUUiQkJByVdmiEhcFgaBUjKI5/\njvZvaIRFTRksfgLyVnX0SgwGg6HTYoSF1vDtk5C7oqNXYjAYPFBaWspzzz3X5uvOO+88SktLA7Ci\nkxMjLCJiITgMKg909EoMBoMHvAmLhoaGFq/7/PPPiYuLC9SyTjqMsFAKopLgcFFHr6TzUVUC714L\nhw929EoMJzEPPPAAO3fuZNSoUYwbN46pU6fy05/+lBEjRgBw0UUXMXbsWIYNG8aLL77YdF3fvn05\nePAge/bsYciQIdxyyy0MGzaMGTNmUF1d3VFf57jFhM6CCAujWTQnfw1s/gRGXQuDZnb0agydgMc+\n2cim/HK/zjm0ZwyPXDjM6/knn3ySDRs2sHbtWpYsWcL555/Phg0bmkJAX3nlFeLj46murmbcuHFc\neumlJCQkuMyxfft23n77bV566SWuuOIK5s2bx7XXXuvX73GiY4QFQNdkqNzf0avofByplNeaso5d\nh8FgY/z48S65As8++ywffvghALm5uWzfvr2ZsOjXrx+jRo0CYOzYsezZs+eYrfdEIaDCQik1E3gG\nCAZe1lo/6Xb+F8AdQANQCdyqtd6klOoLbAa2Ooau0Fr/ImALjUqGwg0Bm/64pbbC8erfJ0nD8UtL\nGsCxIioqqun9kiVLWLRoEcuXL6dLly6ceeaZHnMJwsPDm94HBwcbM1Q7CJiwUEoFA3OAs4E8YKVS\nar7WepNt2Fta6+cd42cBTwOWvWOn1npUoNbnQleHz0Jr8WEYhFpLszARJYaOIzo6moqKCo/nysrK\n6NatG126dGHLli2sWGGiGgNFIDWL8cAOrfUuAKXUO8BPgCZhobW2P7JGATqA6/FOVDI01kH1IegS\n3yFL6JRYmoUxQxk6kISEBKZMmcLw4cOJjIyke/fuTedmzpzJ888/z8iRIxk0aBATJ07swJWe2ARS\nWPQCcm2f84AJ7oOUUncA9wBhwDTbqX5KqTVAOfB7rfV3Hq69FbgVIC0trf0r7Zosr4eLjLCwc8QS\nFsYMZehY3nrrLY/Hw8PDWbBggcdzll8iMTGRDRucZubf/va3fl/fyUAgQ2c92XOaaQ5a6zla6wHA\n/cDvHYcLgDSt9WhEkLyllIrxcO2LWutMrXVmUlKrXQG9E+W41kREuWI0C4PB4CCQwiIP6G37nArk\ntzD+HeAiAK11rda62PF+FbATGBigddo0CyMsXKg10VAGg0EIpLBYCaQrpfoppcKAq4D59gFKqXTb\nx/OB7Y7jSQ4HOUqp/kA6sCtgK7U0C5N85oqJhjIYDA4C5rPQWtcrpe4EvkRCZ1/RWm9USj0OZGmt\n5wN3KqXOAuqAQ8D1jstPBx5XStUjYbW/0FqXBGqtRMaDCjZmKHdMnoXBYHAQ0DwLrfXnwOdux/5g\ne3+3l+vmAfMCuTYXgoIgKtGYodyxNAojLAyGkx5TG8oiKhkqTX0oF2pNNJTBYBCMsLDommQ0C3cs\nB3dDLdSZ/ssGQ0ssWbKECy64AID58+fz5JNPehzXtWvXFudxr7Kbn5/PZZdd5r+FthMjLCyMZtGc\n2goIiXS8N9qFofNSX1/f0UtwYdasWTzwwAPtutZdWPTs2ZP333/fX0trN0ZYWFiahe6YJPJOR0M9\n1FdDbKp8Nn4LQwfz+uuvM3LkSDIyMpg9ezY33HAD99xzD1OnTuX++++npKSEiy66iJEjRzJx4kSy\ns7MB+Pbbbxk1ahSjRo1i9OjRVFRUUFBQwOmnn86oUaMYPnw4333XLOeXCRMmsHHjxqbPZ555JqtW\nreLHH39k8uTJjB49msmTJ7N169Zm17722mvceeedAOzevZtJkyYxbtw4Hn744aYxlZWVTJ8+nTFj\nxjBixAg+/vhjwLUk+7333suePXsYPnw4IP3Qb7zxRkaMGMHo0aNZvHhx0/0uueQSZs6cSXp6Ovfd\nd5+ffutOTNVZi6hkqK+Rp+mIZvl/nYfVb8ApZ0FMSmDvY2Vvx/aC4u1GWBiEBQ9A4Xr/ztljBJzr\n2WRjsXHjRv70pz+xbNkyEhMTKSkp4Z577mHbtm0sWrSI4OBg7rrrLkaPHs1HH33EN998w3XXXcfa\ntWt56qmnmDNnDlOmTKGyspKIiAhefPFFzjnnHB566CEaGhqoqqpqds+rrrqKuXPn8thjj1FQUEB+\nfj5jx46lvLycpUuXEhISwqJFi/jd737HvHne43HuvvtubrvtNq677jrmzJnTdDwiIoIPP/yQmJgY\nDh48yMSJE5k1a5ZLSXbApUKudf369evZsmULM2bMYNu2bQCsXbuWNWvWEB4ezqBBg7jrrrvo3due\n6nZ0GM3Cwl7yo7NyaC/MvxPWvR34e1n+ihijWRg6nm+++YbLLruMxMREAOLjpSzP5ZdfTnBwMADf\nf/89s2fPBmDatGkUFxdTVlbGlClTuOeee3j22WcpLS0lJCSEcePG8eqrr/Loo4+yfv16oqOjm93z\niiuu4L333gNg7ty5XH755YAUL7z88ssZPnw4v/71r120D08sW7aMq6++GqBpfQBaa373u98xcuRI\nzjrrLPbt28f+/S23SrB/x8GDB9OnT58mYTF9+nRiY2OJiIhg6NCh7N27t8W52orRLCzsJT8SBnTs\nWrxRsE5ej0UV2FqbZgFGWBiEVjSAQKG1RnmoCG0vV649mJCVUjzwwAOcf/75fP7550ycOJFFixZx\n+umns3TpUj777DNmz57NvffeS3R0NI899hgAL7/8MpmZmSQkJJCdnc27777LCy+8AMDDDz/M1KlT\n+fDDD9mzZw9nnnlmq+v3tPY333yToqIiVq1aRWhoKH379vVYXt399+AN9zLs/vbjGM3C4ngo+WEJ\ni+pjICyshDzjszB0AqZPn87cuXMpLi4GoKSkeY7u6aefzptvvglIZFJiYiIxMTHs3LmTESNGcP/9\n95OZmcmWLVvYu3cvycnJ3HLLLdx8882sXr2aiy++mLVr17J27VoyMzMBMUX99a9/paysrKmNa1lZ\nGb16yUPUa6+91urap0yZwjvvvAPQtD5rnuTkZEJDQ1m8eHGTJtBSSXb7d9y2bRs5OTkMGjSo1TX4\nAyMsLI6HYoKF4rA7Jhu3Ff0U08v1s8HQAQwbNoyHHnqIM844g4yMDO65555mYx599FGysrIYOXIk\nDzzwAP/3f/8HwD//+U+GDx9ORkYGkZGRnHvuuSxZsqTJ4T1v3jzuvttjfjCXXXYZ77zzDldccUXT\nsfvuu48HH3yQKVOm0NDQ0Oran3nmGebMmcO4ceMoK3P+373mmmvIysoiMzOTN998k8GDBwOuJdnv\nvfdel7luv/12GhoaGDFiBFdeeSWvvfaai0YRSFRLas3xRGZmps7Kymr/BA318MdEOON+mPqg/xbm\nT54aKO1f+58J130c2Htt/Ajeux5+sQxeOB1O/RVM/0Pr1xlOODZv3syQIUM6ehkGP+Dpb6mUWqW1\nzmztWqNZWASHSC+LzmqGqih09gk/JpqFQw2OiIGIWGOGMhhOcoywsBOV3HnNUAUOE1RMr2Prswjr\nKgLDlPwwGE5qjLCwY/Xi7ggOF8Pu5olBTVjO7b6nHlvNIjzaaBaGFqNwDMcHR/s3NMLCTkdqFj++\nCK/PgiovldgL10F8f4jtLRt3oP/z1lZASAQEhxphcZITERFBcXGxERjHMVpriouLiYiIaPccJs/C\nTtfkjtMsKgtBN0LujzBoZvPzBeug11jZuHWDmInCmycS+Y3aCuf84TFwOHC9pwydm9TUVPLy8igq\n6sQJq4ZWiYiIIDU1td3XG2FhJypJNuEjVRDW5dje2+rSl7uiubCoPgSlOZB5E0TGOY6VBlZYHKkU\nfwVARJzRLE5iQkND6devX0cvw9DBGDOUnY5MzKuSZCNyVjQ/Zzm3e4wUzQICv3nbNQtjhjIYTnqM\nsLAT5RAWHVGq3BIW+1ZDfa3rOSsZLyVDnvIh8CU/am1mrogY0TQaOlcZaIPBcOwIqLBQSs1USm1V\nSu1QSjUr7q6U+oVSar1Saq1S6nul1FDbuQcd121VSp0TyHU2ESVFyjpEszh8ELr2kEZD+WtdzxWs\nk4J+UYnHULMod9UsrGMGg+GkJGDCQikVDMwBzgWGAlfbhYGDt7TWI7TWo4C/Ak87rh0KXAUMA2YC\nzznmCyyWGepYR0Q1NohfYtC58jlnuev5gmzRKsDVZxFIaitsPotjJKAMBkOnJZCaxXhgh9Z6l9b6\nCPAO8BP7AK21/VE1CrBi834CvKO1rtVa7wZ2OOYLLFZ9KMvZ7I3CDXDkcPPjlQegPL/t960+BGhI\nHgLxAyD3B+e5I4fh4DZIGSmfj9XGbY+2Cnf09wiEZlFTDiW7/T+vwWDwK4EUFr2AXNvnPMcxF5RS\ndyildiKaxS/beO2tSqkspVSWX8L6QsIhshuU7/M+Zu1b8PwUWPFc83Mf3wlvXtH8eGtYwqlLAqRN\nEie3FdO+5TNAS9gsQHgsoI6Bz6ICwo+BZrH4T/Da+f6f12Aw+JVACovmBdydmoPzgNZztNYDgPuB\n37fx2he11pla68ykpKSjWmwTiQPlSd4T69+Hj++Q9we2ND9/YBPsXy9hrm2hyiEsohIhbQJUl8DB\n7aJVLHwEUkbBgOkyJihInvQDaYZqqJOugZZGEUhhkb9WtLHGRv/PbTAY/EYghUUeYO/plwq0ZKN5\nB7iondf6j6TBUORBEGz7Cj64VZ780yZByU7X83XVUJYn77d/1bZ7WpFQlmYB4rf47mmoyIdz/ypC\nwiKynaGsmz+BvctbH2eV+mjyWTiEhr/rQ2nt+F1rqHX7Ptu+hD3L/Hs/g8HQbgIpLFYC6Uqpfkqp\nMMRhPd8+QCmVbvt4PrDd8X4+cJVSKlwp1Q9IB34M4FqdJA2Wzds9fPaH56UR0E/fhe7DoXina8mN\nkt00KT/b2igsmsxQiZBwigiN9e/Bf/8FIy4XbcNORGz7zFBfPAiLHm19nFVE0D0ayt+aReV+5/eo\nPuR67quHYckT/r2fwWBoNwHL4NZa1yul7gS+BIKBV7TWG5VSjwNZWuv5wJ1KqbOAOuAQcL3j2o1K\nqbnAJqAeuENr3XqXEX+QLA1IKNoihQVlQRK+OnCmbKAJA8TZe/igc4ylaaSOh91LRdMIjfTtnnbN\nQinoPRG2fgahXeCsx5qPb09GdUO9mHsq90NdDYS2UCOmqYigQ7OwzFH+FhZ2Dc7drFa5X8qaGAyG\nTkFA8yy01p9rrQdqrQdorf/kOPYHh6BAa3231nqY1nqU1nqq1nqj7do/Oa4bpLVeEMh1upBkExYW\nFQXiV7DCV+MdPbrtpqhix/uJt0F9Nez53vd7VhXLhhwSJp/TJsrrafc4e2DbiYhtu8+islA234Yj\nkL+m5bG1bppFUDCERfs/Gsru97FrFvVHROMoLwh8wUSDweATJoPbnegUiTiyCwurPLgVvprgEBbF\nNmFRslPMSIPOg5DItvktDh8UrcIi42o44wGYdJfn8ZHt0CwsfwpI/amWaPJZ2GpPBaLkR5EXYWEV\nc6w7bBIBDYZOghEW7igFSYNcn3oLsgElvgqAuDRQwW6axS4RIqER0P8McdD6+lRcddCZPQ5i2pr6\noHdTUURc230WlrAIiWxef6qh3nWzPmLrZdF0zwAJC0tLcxEWtqTI8gL/3tNgMLQLIyw8kTy4uWaR\nmO604QeHQre+ULzDOaZ4hzinAdJnQOleCX/1hapiV82iNSLioK5KzDW+YgmLgedI0p89VPXrx+D/\njXcec/dZgKNbnh+FhdZwYLPT5GYXfvbggopjEwRnMBhaxggLTyQNlqd9K0qpYJ1UfLWTMEC0CRAb\nf2WhNCcCSD9bXn01RR0uFhOWr7QnOqksT65LP1ue4q1ckvpaWPOGPM2XOfJD3H0W1j39KSwqD4iA\n6DESQqNcfTBGszAYOh1GWHjC7uQ+XAzleU7ntkX8ACjZJU/IJQ6hYfky4tIgaQhs/7L1e2ntMEO1\nQbOw6kO1VVjE9pZIK3D6LbZ86jQBFW2V12Phs7A0t6RBkjVvN0PZa3MZzcJg6BQYYeEJS1gc2Czt\nTMHp3LZIGCAO2IpCp+/Csr+DFAXcswwq9rd8ryOVEqHUJjOUpVm0wW9Rlid5IgkDRIux/BarX4eu\n3eX9gc2ONVWIbyPYFlkdHuNfZ7MlLJKHiPBz8VkclLDhyG5GszAYOglGWHgipqdsjkVbXRsP2Umw\nhc9aUVGWGQokokk3wLq3W76XPSHPV9rT06IsV4SFUuInyFkBh/bCriWQebOUR7drFu5d+CJiJYPb\nX6GsRVtkzq7dHZqFmxkqKgmie0rYssFg6HCMsPCEFRFVtEX8FXFp0CXedYylRRTvEGERneLqEE4a\nKKU71rzR8gZrJeRFtcNn4WuuRW2lCJYYR85G2kQ4tBu+fxpQMOqnDqf+Zud4+3ex7qkbPFfbbQ8H\ntoipTqnmmkXlASkXH5PSviq+BoPB7xhh4Y2kwWKW8eTcBnlKDw4TQVGy09UEZTF6tggT9/4Udpqy\nt9sgLNrqs7Cq6MY6ym1Z9adWvQYDpkFcb0dNrG0SEeVRs/BjFrfWIpiSBslnd5/F4SLpWhidYjQL\ng6GTYISFN6yIqJKdUvXVnaBg6NZPnNvFOyGhf/Mxwy4SJ/HqN7zfp8kMFe99jDtt9VmUOaq9x6bK\na4+REOLI4RgzW16TBosPpixX/ChhHsxQ4B9hcbhIhEPyEMfcnjSLJDEHVh6QKrgGg6FDMcLCG1aN\nKGju3LZIGCClM6oOetYswqJg+CWw6SPvm6y9PLmvhEZCcLjvG7eVY2EJi5AwSB0HkfGScQ62CLCt\nri1VLXxprVq2D55Ia5705449EgpEs2iolXpaDfWibVmaBVrqRFlkvQrPZEh3QYPBcMwwwsIbSXZh\nkeF5TMIAp4nHSshzZ8z1kkC3YZ7n81XFsvGHdfV83httqQ9Vtg9UkGPzdXD+03Dt+9LwCZwbd9Fm\n18ZHFuE+aBY5y6XU+M7FLa/Hyo5PcmgWkd3ktfqQwyynHT6LnnLcHhG1YxEc2uN7wqPBYPALAas6\ne9wT00tMMWFdILqH5zF2bSLBg2YB0GsMJA+FH192Vm+N7Ca+AqUcCXmOarNtoS31ocryRFDYQ2GT\nBrqO6RIvkUkHtjgc3O0wQxVmu756o2iLCB/r92oXFtqRRR6V5BRu9lwL+z3s2p8/Kdoq398SVq1x\naI/4r3wdbzAchxjNwhtKQe/x0PdU72OaBIQS/4W3eTJvggMbYd7N8vOfSyDrFTnf1oQ8i7b0tLDC\nZlvDavxUW9Fc07E2dMsh7wmr4KL16o0Dm2SjtwSk5bCvLnUm5HnSLKpKnF0IW7tHezl8EF4+G+bd\n4tv4jR/CnAnSUtdgOIExmkVLXPWWmG+8YWkWsakt94cY9zM4ZbrTzv7Jr+Cb/xF/RlUbS31YRMQ5\nq7O2RlmeaDitkTRYkvQaap1akEWXeBEgh/Z4vtbq+REUKqa5wwc9+2EaG6FwveShWNg1Cys0NypZ\nNK7gMKepr3C9vAaFBk5YfP24mNL2fu8IXPCiMWoNy56BRY8AynsrXoPhBKFFzUIpFaGUukwp9YxS\n6j2l1OtKqfuUUsOO1QI7lNAIZ48JT0SnSKaxtw3FQilJ2EtMl59z/yJawZInm5cn9xVfy280Nkqu\ngi+aRfJg6cUBzX0WSkF8P2dpE3fK8mSzH3y+fPa2mR/aLdFWdj+QXVhYdaGiEuWe0T2c4bPWnIPP\nk2RJb/krWsM3f2q7QMlfK8Jy+GXykLCmhSi2Lx4UQTHsEph8lwg0E7VlOIHxKiyUUo8Cy4BJwA/A\nC8BcpHPdk0qphUopL2FCJwlBQTD2Rml92hZ6DJfrfnxJNpm2REJZRPpYprzqoGgKMT6aoSzcfRbg\nCBXe7flaa2MePdv1c7Nxa+XVLiysjPTqQ2KGCg5z+kiiezrNUAXr5HsMmCZP/960nOIdsPSv8H+z\nnBn4raE1fPGACO7z/w7p58DatyU6y52KQmmzO3o2XPpvCQ7Qjc4QZYPhBKQlzWKl1nqs1vo3Wuu3\ntNaLtNafaq2f1lpfCFwDtPDYfZIw888w+tq2Xzf1IXl6bzjSfjNUTVnr5Tfccyxawi4sPEVnxfeX\nDdpT2GrBOnka7zMZ4vq0ICwcpip3waSCRfhZCXmWPyOmp9PBXZgtQsZKkmzpHiC/mzcucta8aokN\n8ySaa/ofRBCPmS2VhHcsbD52y6eAhkl3yANDXB85fmhv6/cxGI5TvAoLrfVnAEqp4V7OH9BaZ7U0\nuVJqplJqq1Jqh1LqAQ/n71FKbVJKZSulvlZK9bGda1BKrXX8zPf9Kx0nRCWIwLDet5WIWHmatSrE\nesM9x6IlusTLRg2eNYv4ftBY5/Qh2CnMhsSBEj2WkuE9IqogG7oPdTXvKeXM4rYS8ixiHJpFbaWE\ny6aMlOiyoJAW7rFOtJOfLRTB9MbF0nfcG42NsOhREUKW4E+fIb8LTwmVm+ZLqLQl8Lo5/tmWGmFh\nOHHxJRrqeaXUj0qp25VScb5OrJQKBuYA5wJDgauVUkPdhq0BMrXWI4H3gb/azlU7enOP0lrP8vW+\nxxWZN8M5T8CQdnw9X0t+tEVYgDMc1ZsZCjz7LQrWOU1LKSNljPvaLCe4p/IplrA4fMApsED8QvXV\nsHcZoOUeoRGyUbekWSQPFfPQuU+Kz2P/Rs9jQfwoZbkw7mbJzAdpcDXqatj2hWvl4KoS6a8+ZJZN\n++klwstoFoYTmFaFhdb6VMTk1BvIUkq9pZQ624e5xwM7tNa7tNZHgHeAn7jNvVhrXeX4uALwcUc7\nQQgOgUm3t89n4WvJj7J90lzIciK3RlILwsKqquvut6g8IBuyJQSs8ihW9FLTWvKgusRzkqNVTLCy\nyE2zcORabPtCXpvukSFCwd0Mp7XTXAXQ0xEFVtiCs9vSUNzLuoye7agc/Jbz2NbP5dhQm4APChZh\nbDQLwwmMT3kWWuvtwO+B+4EzgGeVUluUUpe0cFkvwO7xy3Mc88bNwALb5wilVJZSaoVS6iJf1nlS\nEeGrZpELsb18T/rrOUb8B1FJzc/F9BTzjrtmYTmRmzSLDNfjFt42ZRBhVlXi9FlYRDtyLbZ9Kb4d\nK/ciJUPGVhS6zlOWK0LHWkNcmvyuWoqMKlgnmoFVq8oiMR36TIHv/wG7v5Njm+bLnO7fIa6P0SwM\nJzStCgul1Eil1D+AzcA04EKt9RDH+3+0dKmHYx69sUqpa4FM4G+2w2la60zgp8A/lVLN4lOVUrc6\nBEpWUZGPOQcnCr6WKbeaHvnKyCvhjh89aztBwdJ7/JCbZmFFOPUYIa9dk6U/hvsGbTnBu3uIvI7s\nJs5z3SDXW1iaRfk+EQCW0PPm5LY+W8JCKTGLtSYskoc4S5/Yueh/5bu8cbFEr+1a7GqCsujWx2gW\nhhMaXzSL/wesBjK01ndorVcDaK3zEW3DG3mI6coiFWjWnEApdRbwEDBLa11rHXfMj9Z6F7AEGO1+\nrdb6Ra11ptY6MynJw5PwiUxbfBZtERZBQZDopc4VeA6fLcwWIRJpc2lZZiI7BTYnuDsRcc4ihXat\nxl7Pyl7QscdwQDV3chdki2ZkF0gpGbB/k+c8CK3lmh5e6n916wM3fwV9JsHnv5XoNU8+prg+oul4\n6/dxpEpKwjc2ej5vMHRyfBEW5wFvaa2rAZRSQUqpLgBa6xayllgJpCul+imlwoCrAJeoJqXUaCR/\nY5bW+oDteDelVLjjfSIwBdjk+9c6CfDFZ3G4WBzG9g5+R0u8Q1jYfQV257ZFykg4uFU2yZbGWdh9\nKnbNIiTcmbRovzY8WpIhPWkWiQOlMq9FjwzJNbE6Adopz5dcFG/rAhGC18yTsi1pk6Rirzvd+sqr\nVY7Enc2fwCd3O3ufGwzHGb4Ii0WA7X8eXRzHWkRrXQ/cCXyJmLDmaq03KqUeV0pZj2Z/A7oC77mF\nyA5BnOnrgMXAk1prIyzshMcCqvUqsOBsduQP4vtL3wurhlN1qZiPmgmLDAntPeD4s1UWSb6Ep0go\ncBUW7v6SaJufwv0e7sLC7ty2j7POuVPo5m/xRkgYXPAPuOkL0b7caS3XoizANa0MhgDjS22oCK11\npfVBa11paRatobX+HPjc7dgfbO/P8nLdf4ERvtzjpCUoSOo3rfy3ROgADLsYTvuNc8ze/0qTo57N\nLHjtxwqfPbQbortD/mr57G7GsYTCti8gNdMZjeSLZmF3cIP4LUr3Qlzf5vfYME9CW6O7O6Oy3O+R\nMEDKshSskxaydgrWAcqzH6UtWLkW3rLKrRBmXzPKTyb2fA9r/gM/ec6zIDZ0Cnz5yxxWSjVVoVNK\njQWqA7ckg8+c9mupjBvbG+qPwLd/c9U09i4Tk4knx217aQqf3SWmqCV/ETNR2gTXcXFp0lhp6d8k\n4S1/jRzv4eUZwPJ3qODmYb4Tfg5nP9Z8Ixl0noz/9i/yuSkqy017CQqW+3p6qi9YJ1FP7rWw2kpU\nkggkb04Fy3QnAAAgAElEQVTuJmFhNItmrH4d1r0N5XkdvRJDC/iiWfwKMRNZzukU4MrALcngM6f+\n2vk+bxW8PA3Wvy/JZTXlYmI5/V7/3jMuTSKaSnbLU33uCrjwmeZ5GUrBFW+IU/j7f4iG4+4Et2MJ\niKik5kLhFI8KqPTkGPczWPkSZN7YPCrLTkoGrH1LHMz2+QuyxXl9tCglvxuvZihH1nvRFukIaPep\n2KmrcbaYDY3wPT/G3zTUSWJiu66td+2d0hqWubRoq/wODZ0SX5LyVgKDgduA24EhWutVgV6YoY30\nGgPJw5yVUnN/FJ9Bn8n+vU9ImERX7d8IC//gKJEx2/PY4BCx88/4H6ivdSbIecLaFLu2Mapt6oMS\nSbXgAXlq79bP6fy302OkVLu154gcPihPs978KG0lzkv4rNaS/xGTKqHBB7y437SGF8+ApwfLz1/6\nwb7V/llbW8jLgj/3cnY0bAtVJfBUOnx4m/zNW6M83xkU4EsNL0OH4auBcBBSsmM0UrbjusAtydAu\nlJLid/lroHCDmKCCQjxH7hwt3frB1s8k9+HcvzpLZHhb1+S74NYlMPMJ7+OaNItk72O8XTft99J/\nYusC7z6RpkTBtc5j7jkZR0u3vqJZuGeV15SJoBp0rut93SnaKprH6NlS+VYFSRTVsSZnuUSPtefe\n2xdKlv66t+CNS0R4tHgvKzpMeY5WM3QafEnKewT4l+NnKlK/6cSs1XS8M/JKybBe84Y4t3uOhrAo\n/98n3uHkHn6p7yacnqO8t6cFpzbQtY3CAmDsDdB9uBQ59LbxJw2WooL2iCjrvTc/Slvp1geOVDjN\nSBaWv6LPZPme3pzc27+S1zPuF/Na2kTZfNtDVYlol+3B0iis9bSF7V+JwL/4Bcj7Ef59dsuZ7bk/\niK8nbZL0fzd0WnzRLC4DpgOFWusbgQzAjx5Tg9/oEg+DL4B178C+Vf43QVn0Giumn7Mf99+cwaFS\nybW7xyLHLRMU7NBwQqHvaZ7HhIRJtVv7U33BOrGRd4lv35rdifMSEWUJi7g0z+G+Ftu/ElNinCOX\nNX0G7F/v9He0he/+Dq+eK76rtlLkEBZ5KyVXx1ca6mHHIkg/GzKugus+lki1b/7o/Zqc5fLvqcdw\n0SxaK7lv6DB8ERbVWutGoF4pFQMcAPyY5WXwK2NmS6JeY53UNQrIPa6D325rW2a4L9yxUnpEtIe+\nU+B3+6B3C2a3lAzXDnsFHnIyjgZvpcrLbZV/e4wUf497NnlNmWyc6bYanekz5NVTT43WyF8DjfWy\n4bcFrWXT7jUW0LDza9+v3Zcl//as79BnMgy5QASIpx4otZViMk2bKJrfkUqnYDV0OnwRFlmO0uQv\nAauQ0h/t1G8NAaffmRCbBijoPaG10e3Hn+G4FkFBvhc89ERra+oxUuzpb10Bb18NJTu9l/loD94S\n88ryROuJSpYChA21zXt271oim/vAc5zHkodIWPS2NpqDrD7nIGYeb+xaAsuedT1Wvk9MaRlXS2Ra\nW0xR27+SUOb+U53H0s8Ws1yeh9Y3+7LE4W8JC3BqNYZOR2s9uBXwhNa6VGv9PHA2cL3DHGXojAQF\nwfSHYeJt3sNUT1bSZ4jDv7wASnMlOmvwef6bPyJGHO7umkVZnlTLDQpy5oC4m6K2fyX+jNTxzmNK\nyWa7a4lvkUUWh3Y762xZYanuaA1f/E5yYGorncctf0XyUDjlbO9agSe2fSW+B/u/uwHTRIB4Ejo5\nK8SJnzreWfHXCItOS4vCQmutgY9sn/dorU0Kamdn5BUtRx6drHTrAz9bBLd9Lz+3Lj76zG13PJUq\nL8sTDQHELxPaxdXJ3dgojuwB05vnJ6TPkPIqe//r+xosx33vCZJ/46mAYv5qOLBRnuz32SLhrc06\naXDLWoE75fniX0l3a3UT2U3W4U1YJA8TIdslXjSZ9oTrGo4JvpihViilAhB/aTCcgHTr07yEe9k+\np38nKFic+HbNojAbKvc7fRR2+p0OweFtMwdZ/TnG3iiCxr0JFUjWdEgkoGzhq0hEUlSStPptSStw\nxxrj6TsMnCHfsbzAeazB4U9Jm+g8ljQ4cJrF+zfDYvMAdTT4IiymAsuVUjsdvbLXK6WMdmEweCIl\nQxL/rPyCxgbxA8T2ch1TmO0sV759IaA8Z6qHRUHfU9suLJKHiKABV2EAUkZ9/TwYdpFoVvZKuEVb\nnf6DyDhH+O6Xrd9z+0LRntwbSIFnR/2BjeLQbiYsAhARdbhYqg3s/ta/855k+CIszgUG4Gh8BFzg\neDUYDO70dmx+lmO5olBMPfbIsZQM2SjXvwc7v4HNH0sGvrfs9YHnQPEOKN7Z/FxFoWvim70/R2wv\nCXZwL4u+6WNxYo+eLSai3B/lSd+KhLKEBYhZqXC9mJma3Xu/rH/H1+JXST/bc4BC8lDJXrcLPEuA\n2YVF8mBZV3k7QoVbYufXgPb/vCcZvggL7eXHYDC402uMRD5Zm6EVChpr6wNmZdV/eKt04CtcL0UR\nvWH5AdwzqhsbJZdirq2ggnt/jrSJshb70/rqNyB+gIS2pk0SwXVgo1xbWw5Jg5xjB86U1zX/cb13\n9SH438my/v9c4shQP9/z+i1H/c4lUvBy/0ZY9oxUArALUUtI+dtvYQmpikLTfOoo8KXa12eIcFBA\nBNAP2Ar42TNoMJwAhEZKtrqlWZQ52tDbN8XkwXDbcmfEkgpuOd8jvr9oAGv+A1Pudj6971kqJq+S\nXaJ1JAxo3p8jbQKsnyuJgvH94OAOyPkvnPWozGNVC875ARIc6VN2U1LyEOkM+P0/YNQ1TnPakr9I\nGPLl/yeZ+SERLX+H9Bmw6lVY8oS0pw3vClf8n+uYJFtEVLqX4pFtpbFBIrqCw6TLYVVx2+uPGQDf\nCgmO0FqPdLymA+OB7wO/NIPhOCVtohQArK91mj5iermO6T5UxqVNlETCkLCW5xw9G4q3u+ZNrH5D\nepqoYGcBSff+HFbjK0vTWfWqjM9w9PWI7S1ry1nurM1kN0OBFILUjVI4EuTJ/8cXYcz14vdImygC\nsqUcmf5nyIb9/dNSQ+tnXzcXLlEJ0CXRv2U/8rJECxrisJxXeDCn+eU+q2Djhy2Pqa8Vjcpb691O\nTps7jTh6cJvoKIPBG70nSuJd/loxQ4XHSnjo0TDsYgjrKlFMIH6KzZ9IWY30GVJ+vaHe0efc1p8j\naYjcP3cFZM+F5XOkpld0dzmvHMmbOSuk6muXRIhKdL13tz4w+Zew4X3Yuxy+eEDmn/Z739cfFiVt\naYdfCjctcHX420ke4t+Cglai4Khr5LMn34s/WPgwvHcDfH6v/B08sWORCNxVrwVmDQGmVTOUUuoe\n28cgYAxQFLAVGQzHO5bTNme5I8fCD2VRwruKwNgwD2Y+Kc7xhlrROMpyYdsC2RgL1rkWdwwKkgZZ\nGz8SQdP3VOk/4rLeSbDxA3FWu2sVFqf+Cta+CXNnw+EiWYO7UGmNc//S+pikQSLUtD66bH6L7V+K\nMEweKp8D4eRubBQhHd1TNK5De+GyV5o31LLCpVe/ARNv98/3O4b4ollE237CER/GT3yZXCk1Uym1\nVSm1Qyn1gIfz9yilNjlCcr9WSvWxnbteKbXd8XO9b1/HYOgERCVK8l3uD7KR+6uG1pjroK5KNvbV\nb4gZJ2WkaBZRyWLi8NSfI22i1GxKHQ9XvwNhbl2RLb9F+T7xp3giLEoKRx4uEoEy7mf++U7uJA0W\nX44/NvXyfAkeGDhDqhmrYNdcD39xaLdEcU19UErL71goTn/3EGBLWBRt9i3RsZPRqmahtX6sPRMr\npYKBOUiJkDxgpVJqvtba3vllDZCpta5SSt2GlD+/UikVDzwCZCLO9VWOa91qPxsMnZS0ibDlc7H1\n20t4HA2p42QzXfKk9Bo/7yk5HhwKo64WYQHNfQFjrhd7+eQ7PbePTR4GYdGy4XnTLEBMSGV5kg/S\n3i56rZEwQF4P7Tl6Ibtjkbymz5BkyOgegTFD2fuipGSIT2LhH6TsS7e+tnHZEvW2awmseb3loped\nEF/6WSx0FBK0PndTSvmQpcN4YIfWepfW+gjwDm4aidZ6sda6yvFxBWD96zgHWKi1LnEIiIXATB/u\naTB0DnpPlGihmlLv9vm2opSYnSoKJPpoxOXOc6Nt4bPu/Tm6JsG0hzx3EAQpMZKaKe9bEhZKiTmq\nRzvKyPtKrKOtamnu0c+17UvJ77BMUDE9mzu4aytaTwJsbJBx3ihYJ+HSVjTXgGnymmMLRqg8IPfu\nM8VhTvzAtSaXL9SUd2jory9mqCStdan1wbF5+9Khphdg/4vnOY5542ZgQTuvNRg6F2k2v4E9x+Jo\nybhKooqGXuRasC/xFNmI4vu3rz9H31PFTGNtrB2FpU2UHaWwaKiHXd9KCK7lG4jp6apZ1JTD00Nh\n5cve56nYDy9Ph38Mg93feR5jZcxbEW3JQyVKzV7EscAW0jzmOslL2fRR87m8caQK/jkC/vtM62MD\nhC/CokEp1dRF3eFX8CUpz5P3xuN1SqlrEZPT39pyrVLqVqVUllIqq6jI+NwNnYiEARJZBP7t+xGV\nKMUQz32y+blL/w0/ndu+eSfdAT9bKOGrHUlohPhfrL7c7WX/BjGr2ZthRbsJi7wfxT+y6WMvc2wS\nQVG0Vf6Wb1wsUWd2tJbcFrvpLyhYggrsYc6FDlNVjxHicE9Id0a2+UL+atFSV/7b9yrAfsYXYfEQ\n8L1S6g2l1BvAUuBBH67LA+yPVKlAM4OhUuosxz1maa1r23Kt1vpFrXWm1jozKckk2hg6EUo5o6L8\n3SQqJcPZs9xOTIqEzbaH0EhHw6NOQFzv5ppFeT7sWeb7HE3lRGwaXkxPeaK3ugdaY3KWS/MpO7uX\nwivnSMXeGxfALd9IxvtHt7kWJCzfJ4l+7n6i3hPhwCZni92CdeK/iIyTfxtjZosw8TVM2NJSynLF\n59EB+JKU9wUSLvsuMBcYq7X2xWexEkhXSvVTSoUBVwHz7QOUUqOBFxBBccB26ktghsM/0g2Y4Thm\nMBw/DD5fTFDRKR29kuOL2N7NfRZLnpAn+7oa3+bIWS7z2P1FMT3l1dIuclZAaJQ0ndq52DmusRE+\nugO6dodbvpaEw8g4uOZ9SWb89kmphwWuzm071oOC1QfdvStjxtVSTXjp3/CJnB+kREtkvDMB8xjj\ni4P7YqBOa/2p1voTpL3qRa1dp7WuB+5ENvnNwFyt9Ual1ONKqVmOYX8DugLvKaXWKqXmO64tAf6I\nCJyVwOOOYwbD8cOon8KvNwQucuhEJa63RF3Znbn5axyJjmtav15reWq3FykEp7CoyBeNIS9Losgi\nYh2Vfx3sXgJlOXDmA65aYUgYXPhP8Qt98aDMUZAtDZzc+6L0Gitl4nNWQHWphNfaQ5q7JsPkuyRf\nxr0qsDuNjSJ0+p0GI6+EzZ82741+DHqX+2KGekRr3aSjOZzdj/gyudb6c631QK31AK31nxzH/qC1\ntoTCWVrr7lrrUY6fWbZrX9Fan+L4ebVtX8tgMBy3xKaJYDjs8EPW1zqLC7pX0PVE6V6JGHMXFpaG\nV54vfob6agkKGDBdciMs4bT6DYiIg8EXNJ87JBzO+TMc3Co1rgrWif8hLMp1XFgX0SRyVjj7iaSM\nch1z2j3iR1lwv/PeDfXw7V8lMdGiaDPUlolJbcxsaKyD7HflXGMjfP1HWHBfwAWGL8LC0xhfChAa\nDAZD24lzuCstv8WBzbJBQutP4eAMWe3dgrCwl0gfeI40nypcJ2VUtnwqT/ChEZ7nHzhTBMySJ8VJ\n7q2AYu+J4pjOWymfU9ySJa1Ex4K1sPY/Ep77ztWw+E/w1cNOAWL5K3pPEA2m5xgxRdXVwAc/g++e\ngvoayekJIL4Iiyyl1NNKqQFKqf5KqX8Aq1q9ymAwGNqDFWpsRURZfoE+p8om31quQc5yqYfl3ogp\nNAK6JDiFRVwfMU0NmA4oMUVlz5XqtGOu8zg1IA7qmU9KF8Kq4uZCwCJtomzia/4jgqqrh4yDEZeJ\nUPn6cSk3v+Nr0WgqC51CJucH6NrDmeA35jpxnr9wupR/OesxuPBZicIKIL4Ii7uAI4iD+z2gBrgj\nkIsyGAwnMe6aRWG25C1kXCnhowe3tXx97g+SHe1p87RyLXJWOM1UXZOkD8m2LyWctefo1hMPkwbC\n+J/Le2+ahTV/yU7vY5SSMOjDB6FkD1wzFy56TpL8NjvigXJWSEkWK19k+KXSx710r5SIP/VXx6TO\nlC/lPg4Dzeo6GQwGQ0CIiBXNwIqIKlgnzuG0yfI5Z7n3GlbVh+Spe/glns9H9xTTUfUhV59G+jmw\n5M/y/vynfVvntIdEqPQ51fP5rsniDC/Z1bxel52eo+Ga90RzsEKfB0yFTfOl4GBZDky63Tk+IkYi\nsyLjmjvWA4gv0VBJSqm/KaU+V0p9Y/0ci8UZDIaTFCvXorEBCjeIqcdKdLQnu7ljhaq6+yssYno6\ncx/sY6xuhCGRYhryhbAoiXgLamEbte7RUmMo6/72HJkhs0RI/PC8Y54JruP7TjmmggJ8M0O9CWxB\nOuQ9BuxBwlkNBoMhMFi5Fge3S9RSSoYz0dFeRsOdnBUSsuotwdAKn42Ida2DlTIK4tJg5BXea2i1\nh1OmS3mWtiY8DjpPyq/88LzkgrSkmRwjfBEWCVrrfyO5Ft9qrW8CvIhtg8Fg8AOWZmE5t63NMm2i\nVKStKPR8Xc4KESzuZdgtLGHRe4KrRhAUBD//Ds7zMUnOV4ZfCvdsluz6thCVINpDwxFIHSvFHjsY\nX1bgiFmjQCl1PlJ2w8/1CwwGg8FGbG+p27RnqVTYTRwox+1tYoddBGX74L/POluV7lsF42/xPq8V\nPuuegwGuhRn9hVJtbxJlMWSWlB2xlyzpQHwRFv+jlIoFfgP8C4gBfh3QVRkMhpMbKyJq6wLoPtz5\nZN1jpAiPnBUQ3w/eulJyI7o4CiBG95BN1hs9R0t/kZbGdBaGXSyhvEN96jUXcHyJhvrU8bYMmBrY\n5RgMBgPOvhZVxa6bZUgY9MqU8t6rX5eCircu9t3Z2yVequseD0Qldqq1+uKzaEIptTpQCzEYDIYm\n4mxFp5sV6Zsg5TwS06XQ3zGOCjpZaavX5PjqMG4wGI5PopLE3FRf0zwSaPzPIawrTPh585pMhoDR\nJs0C+CwgqzAYDAY7SknF16CQ5t37ortLET4jKI4pbdIstNa/D9RCDAaDwYXEgVLmw1tBP8MxpV3B\nu0qp9VrrEa2PNBgMhnZy4TPSmMjQKfAqLJRSXoqroIAegVmOwWAwOPBUpdXQYbSkWbyLlPrw1FHD\n6IUGg8FwEtGSsMgGntJab3A/oZQ6K3BLMhgMBkNno6VoqF8B5V7OXezL5EqpmUqprUqpHUqpZmXO\nlVKnK6VWK6XqlVKXuZ1rcPTlburNbTAYDIaOwatmobX+roVzWa1NrJQKBuYAZwN5wEql1Hyt9Sbb\nsBzgBuC3Hqao1lqP8nDcYDAYDMcYr5qFUur3Sqn4Fs5PU0p56GjexHhgh9Z6l9b6CPAO4FLkRGu9\nR2udDQS2eazBYDAYjoqWfBbrgU+UUjXAaqAIcWynA6OARcCfW7i+F5Br+5wHTPAy1hMRSqksoB54\nUmv9URuuNRgMBoMfackM9THwsVIqHZgCpCA+jP8At2qtq1uZ21NpEE+RVd5I01rnK6X6A984cjt2\nutxAqVuBWwHS0tLaMLXBYDAY2oIvVWe3A9vbMXceYKsGRirSC8MntNb5jtddSqklwGhgp9uYF4EX\nATIzM9siiAwGg8HQBtpaG6otrATSlVL9lFJhwFWAT1FNSqluSqlwx/tERLPZ1PJVBoPBYAgUARMW\nWut64E7gS2AzMFdrvVEp9bhSahaAUmqcUioPuBx4QSm10XH5ECBLKbUOWIz4LIywMBgMhg5Cae3d\neuMIf/2l1vofx25J7SMzM1NnZbUa0WswGAwGG0qpVVrrzNbGtahZaK0bcAt3NRgMBsPJhy9VZ5cp\npf4fUivqsHVQa2265hkMBsNJgi/CYrLj9XHbMQ1M8/9yDAaDwdAZ8SV0duqxWIjBYDAYOi+tRkMp\npWKVUk8rpbIcP39XSsUei8UZDAaDoXPgS+jsK0AFcIXjpxx4NZCLMhgMBkPnwhefxQCt9aW2z48p\npdYGakEGg8Hgb17+bheJXcO5aHSvjl7KcYsvmkW1UupU64NSagrQWl0og8Fg6BRorfnXNzt4/tud\nrQ/uIMqq6qhr6NzFt33RLH4BvG7zUxwCrg/ckgwGg8F/FJTVUFZdR3lNHRU1dURHhDadK6qoJSEq\njKAgT3VPjw2VtfVM+/sSZgzrwROXjOiwdbRGi5qFUioIGKS1zgBGAiO11qMdPSgMBoPBr9TWN7Bg\nfQHulSVaqjTRGpvyyx1zwJqc0qbjRRW1nPbXb3huyQ6f56qpayC/1L+Glbkrcyk+fIT3snLJLany\nOq6grJr6DtQ+WsvgbkTqO6G1Ltdae2uzajAYDEfNR2v2cdubq1m195DL8ReW7iLzfxbx/qq8NguO\nTQXlKAVBCpd5P8vOp6aukVeX7aGmrqHFOfYWH+ZPn21iwp+/5synlnCwsrZNa/BGfUMjryzbzeAe\n0QQFKeYs9iy4KmrqmPbUtzz11Ta/3Lc9+OKzWKiU+q1SqrdSKt76CfjKDAbDSYf15L9yj6uw+HD1\nPg5VHeG3763jmpd/YPfBw54u98im/HL6JkQxqEcMq3Oc8368Lp/oiBCKDx9h/lrP3ROqjzTwwLxs\nznxqCa8s28OIXrEcqW9k6bYin+5dVlXH3KxcGhs9C7gvNhaSd6iae84eyNXjevP+qjyP2sXKPSVU\n1zXw5oq9VNTU+XRvf+OLsLgJuANYCqxy/JiKfQbDSciqvSWUVQVus1qbW9p0H4t9pdVs3V/BfecM\n4n8uGs76vDKm/X0J5z3zHX/8dBNLth5oUdvYVFDO0JQYxvaJY01OKQ2NmpziKtbklHLbmQMY3COa\nV5btbjbHrqJKLn5uGe+szOXmKf1Ydv80Xr9pPIldw/lmywGfvs/zS3dy3/vZfLR2X7NzWmteWrqL\nfolRnDWkO784cwBBSnk0iy3fWUyQgoraeuZm5fl0b3/ji8/iWq11P7ef/sdofQaDoZOwbMdBLv3f\n5fxjUftMIQ9+sJ6Z/1zK459s4pst+6k+4mr6qTpSz7b9FSiHucjavJdslY15+pBkrp3Yh0W/OYNf\nnzWQ2MhQ3lixlxteXcnL3+32eM+KmjpySqoY2jOGsX26UVlbz9bCCuavk817VkZPbj61H1sKK1i2\no7jpus+yC7jwX9+zv7yG124cx+8vGEqP2AiCghRnDkpi6baiVv0H9Q2NzFslG/tfvtjC4dp6l/M/\n7i5hXV4ZN5/aj6AgRUpsJFeO6817WXnkHXLVLpbvKmZc33jG943nle93d4jvwhefxVPHaC0Gg6GT\nUlp1hN/MXQfAlxsL2+w3KDl8hLlZuVTW1vPmD3u56bUsrnpphcs86/PKaNQwY2h3DlXVsbNITE1L\nthaR2i2SAUldAegeE8Evp6fz9q0TyX5kBjOH9eDJL7aQtaek2X23FFYAMDQlhsw+Yj1ftbeEj9fm\nM65vN1K7dWHWqJ4kdg3n39/voq6hkcc/2cQdb61mYI9oPvvlaZw5KNllzmmDkymvqWdNbmmz+9n5\ndlsRBypq+fkZ/dlfXssLbqG7L323i/ioMC4dk9p07DaHdmEP8y2rqmNjfjmTBiRw82n92FdazZcb\n97f6O/c3vpihvlJKXaqU6rjYMoPB0GForXnoww0UH67lhsl9KSirYcO+tsW6LNhQQEOj5sXZmax7\nZAa/nHYK63JLmwQCwLo82Xx/dpoYLlbtLaG2voFlOw5y5qAkPG1BEaHB/PXykaR2i+TOt9ZQ7OZ4\ntiKhhqTEkNotkqTocN7+MZftByr5yShJ0AsPCWb2xD4s3lrEpf/7X15ZtpsbJvfl3Vsn0TMustk9\nT01PJCRItWqKmpuVS2LXMH47YxAXZvTkhaW72FdaTU1dA499spFFmw8we2IfIsOCm67pGRfJeSN6\n8Mm6gqa8ixW7i9EaJg9I5Kwh3emb0IWXvtt1VBFi7cEXYXEP8B5wRClVrpSqUEqZqCiD4SThg9X7\n+Gx9AfecPYhfTk8nSMFXmwrbNMcn6/LpnxTFkJRoIkKDuXJ8GgBfb3Y+Ia/LLaN3fCSZfboRHxVG\n1p5DrNx9iKojDUx1e7q3ExMRynPXjKGk6gi/enctDTZn8qb8cuKjwugeE45SirFp3dhUUE5IkOK8\nESlN466dmEZYSBDb91fyzFWjeHTWMMJCPG+PMRGhZPbtxmKbsHhm0XbufmcNR+plgz9YWcvXmw9w\nyZhUQoODuH/mIEBMcRfNWcary/Zww+S+3D51QLP5zxuRQll1Hf/dKWax5TuLiQgNIqN3LMFBiptO\n7cfa3FLeXZnLx2v38eLSnbz1Q06rf4OjpVVhobWO1loHaa1DtdYxjs8xAV+ZwWDocA4dPsIj8zcy\nvl88t57en/ioMMb1jeerNphBDpTX8MPuEi4c2bNJO+gVF8mQlBi+3uzccNfmlpKRGodSijFp3cja\ne4glWw8QFhzEpAEJLd5jWM9YHps1jO+2H+StH/Y2Hd9cKM5t676ZfbsBcPrAJOKjwprGJXQN5+1b\nJvD53ac1aRwtMXVQMlsKKygoq2bDvjL++fU2Pl6bz4MfrEdrzUdr9lHfqLl8rJiYUrt14ZbT+rN0\nWxFFFbW8esM4Hp01jPCQ4GZznz4wiaiwYBasLwBgxa5iMvvEN429bGwqcV1CeeCD9dz9zlr+/PkW\n5mbltrrmo8WXqrNKKXWtUuphx+feSqnxvkyulJqplNqqlNqhlHrAw/nTlVKrlVL1SqnL3M5dr5Ta\n7vgxGeMGQwfw454SKmvrufecQQQ7spxnDOvB1v0V7C0WE1JuSRVXvbicPV7CWT9bX4DWcGFGisvx\n6V7uJdYAACAASURBVIOTydpbQmnVEQ5U1LCvtJpRveMA2dR3HzzMJ9n5TOgfT5ew1otNXDWuN+P7\nxvP/Fu+gpq6B+oZGthRWMLSn89l2Yn8ROpeMaS4QxvaJp19ilA+/FZg6WDSdb7Yc4LFPNtKtSxi3\nnt6feavzeObr7by7MpfRaXGkd49uuuaOqafwhwuGsuBXpzVd74mI0GCmD+nOlxsLOVBRw5bCChdh\n2SUshPd/MZm3bpnAonvOYP2jM/jw9sle5/MXvpihngMmAT91fK4E5rR2kaN/9xzgXGAocLVSaqjb\nsBzgBuAtt2vjgUeACcB44BGlVDcf1mowGPzI6pxDhAYrRvRydiWYMbQ7AAs3iXbxx083sWJXCS8s\n9Vx76dPsAgb3iOaU5GiX49OHJNOoxYGdnVsG4BQWfeS/+/7y2hZNUHaUUvzqrHT2l9fyzo857Dp4\nmCP1jQxJcd53eK9Yvv7NGZw/IqWFmVonPbkrveIi+cfC7azcc4h7zxnEg+cO5rKxqfxz0Xa2H6jk\niszeLtdEhgVz06n9SI6OaHX+80akcKiqjn8u2g7QTLM6JbkrkwckckpyV6IjQj36c/yNL8Jigtb6\nDqAGQGt9CAhr+RJANvkdWutdWusjwDu49fPWWu9xlA5xjwM7B1iotS5x3G8hMNOHexoMJxWPfLyB\nZ7/eHrD51+wtZWjPWCJCneaS3vFdGJISw1cb97Nsx0G+2rSfxK7hfLB6HyWHj7hcn3eoilV7D3Fh\nRs9mc2ekxpHYNZxFm/ezNreU4CDFsJ4ilIb3iiUsWLanMwcl+bzeSQMSGN8vnueW7GS1I1t7aIpr\n+50BSV2PenNVSjF1cBIHK2sZ1jOGKzJ7o5TizxePYMopCcRGhnLByPYLpDMHJdElLJi3f8whKizY\nRVh3FL4IizqHlqABlFJJNN/cPdELsBvS8hzHfOForjUYTgrqGxqZm5XHi0t3NctZ8Ad1DY1k7ytl\nTFpcs3MzhnZn5d4Sfv/RBnrHR/LajeOorW/k7R9dHa2fZYvd/cKRzYVFUJBi2uAkvt1WRNbeEgb3\niG6KDIoIDWZU7zj6JnTx2TQEson/+qyBHKio5e8LtxEWEkT/JN+vbwvnj+hJWHAQj80a1mSiCwsJ\n4vWbJvDNb85wKVjYViJCg5k6OBmtYVy/eEKDfdmqA4svK3gW+BBIVkr9Cfge+LMP13kS3b7Gevl0\nrVLqVquDX1GRb+n3BsOJwtb9FVTXNVBZW8+XG9sWneQLmwvKqalrZExacwvwjGHd0Rp2HzzMQ+cN\nYXivWE5LT+T15XuaIoJq6xv4YPU+MlJjSUvo4vEe04d0p6KmnhW7Ssjo7SqUnro8g5evz2yzFjBp\nQAIT+8dTVFHLoO7RAdtoJw1IIPvRGWT2da1+FBykSOgaftTzW6aySf1bdu4fK3yJhnoTuA94AigA\nLtJav+fD3HmA3WiXCnguwNLOa7XWL2qtM7XWmUlJvquqBsOJgFUaIzYylPdX+b8EhFWnaUyf5sJi\naEoM/RKjmHJKAucM6wHATaf2Y395LQs2FFBT18Ctr69i6/6KprwJT5yWntgUojoq1VVYpCV0aebn\n8JVfnzUQwMVfEQjs5jl/M31IMndNO4XLxqa2PvgY4Es/C7TWW4AtbZx7JZCulOoH7AOuwukkb40v\ngT/bnNozgAfbeH+D4YRmTU4p8VFhzJ7Yh2e/2U5+aXVTEpnWmsra+qMyhazOOUT3mHB6xjZ3yCql\nmHfbZCJCg5qe/M9IT6J/YhQvfbeL97LyWLbzIE9eMsKjv8KiS1gIkwcksGRrEaM8mLvay4T+CTxy\n4dCm6KfjkfCQYH4zY1BHL6OJgBnCtNb1SHnzL4HNwFyt9Ual1ONKqVkASqlxSqk84HLgBaXURse1\nJcAfEYGzEnjcccxgMDhYm1vKqN5xXDomFa3hwzVS76ixUfPLd9Zyxt+WHFX3tdU5hxjdu5tXM1B8\nVJhLSGtQkOLGKX3ZsK+cZTsP8rfLMrjKkXzXErMn9uH0gUlN5Tz8xY1T+jEkxaSE+QufNIv2orX+\nHPjc7dgfbO9XIiYmT9e+ArwSyPUZDMcr5TV17CyqZFZGT9ISujChXzzvr8rj9jMH8PeFW/lknVht\nt++vdMkz+DQ7n4ZG3WriWVFFLbkl1cye2KdN67p0bCrfbiviJ6N6tahR2Jk+pDvTh3Rv030Mx56O\nd7EbDIY2k51bhtYw2mG6uXRsKrsPHuYPH29kzuKdTeGm2Xmuxe7+uWg7j8zf2KrGYfV98OTcboku\nYSG8fP04nwWF4fjBCAuD4Thkba5s5iMdTuHzRqQQGRrMGyv2clp6Ii/OziQ6IoTsfWVN11QfaWBX\nUSWlVXV8v+Ngi/OvySklNFgxvBPE9xs6B0ZYGAydiP+s2MsPu4pbHbc2t5QBSVHERooDu2t4CNdO\nTGN0WhxzrhlDWEgQI1NjXTSLLYXlWDX2Pl1X0OL8q3MONUvGM5zcGGFhOCH5fvtBKt2azXR2ckuq\nePjjDbz8ffNGPiWHjzSVpNZasyanlFG9XU1EvztvCB/ePoUYRwTUiF5xbC2saOovvalAikVP7B/P\nVxsLXfpOl9fU8fn6AnYVVXKkvpHsvFJG9/ZfdJLh+McIC8MJx+6Dh7n23z/wrwCWwTga8kurOfUv\n37B8p6sG8eYPOWgNO4sqXY4fKK9hwp8X8cQCiV7PO1RN8eEjzUJN3aOWMlJjqWvQbHU0ANqUX050\nRAg/P2MAFbX1Ln2k738/m9vfXM20v3/LmD8ulGQ8D/kVhpMXIywMJxyLHAXu5q/Lp7Gx9aIBDY2a\n15btZreXqqn+Zv66fPIOVfPHTzc1ra+mroF3V+agFOQUV7k4oFfnlFLXoHlx6S4+XJPX1KGttSf/\nEanib7BMURvzpVz3qack0q1LKJ86SnF8tbGQBRsKufX0/vzl0hFcMDKFMwclcXp6ot+/u+H4xQgL\nwwnHwk37CQlSFJTVsNJDq007DY2ae99fx6OfbOIvC9qad9o+FqwvoEtYMJsKyvnY0Qv6k3X5HKqq\n47IxqdQ3avYWO3swZ+eVEhKkGN83nvvnrefdlTlEhAYxqEfL2cm94iJJiAojO6+MhkbNlsJyhvWM\nJTQ4iJnDU1i0eT9FFbX84eONDO4Rzb3nDOLKcWk8eelIXrtxPHFdfKkXajhZMMLCcEJRcvgIWXtL\nuGFyX7qEBfPxOu8VZhobNffPy+aD1fvok9CFb7YcoKy6zm9rKauuayasckuqWJdXxp3TTmF4rxie\n+nIbtfUNvL58L+nJXbnGkddgN0Wt31fGwO7R/O+1Y0jqGs6yHcWM6BXbas0jpRQjUmPJzitj98HD\n1NQ1NuVcXDgyhaojDVz78g/sr6jhiUtGdIpidYbOi/nXYTihWLzlAI0aZo3qyYyh3fl8fUFTYTs7\nWmse/GA976/K4+7p6Txz1WiONDTy5Qb/FORrbNT84o1VXP788qYaTgBfOOY/f0QKD8wcwr7Sau57\nP5v1+8q4blIfBjgqpFrCQmtNdl4ZGb1jSegazkvXZRIZGsyEfr6VsRiZGsf2AxVkOYTWMIewmNA/\ngcSu4WzdX8H1k/oyuo35FIaTDyMsDCcUizbvp3tMOMN7xvKTUb0orapzceRarNp7iHezcvnFGQP4\n1VnpZKTG0jehCx+t3eeXdbz23z0s31VMaLDib186zVufrS9gWM8Y+iREcWp6IqelJ/Lx2ny6hodw\n8ZhUoiNC6R4Tzs4D4j/ZW1zF/2/vzMOrKq+F/1snAyGBBELCkJAQSMIQBhkiyCQgVECvYi32E62K\nxVq/6qfWOnbQ6m1va3uv3ra29fYTqnWoA1jFCQRUBEQgAUIGpgAhhAwQhiQQMr/3j70TTk5OBiAn\nyTmu3/Oc5+z97vfde62Tnb32O6y1Ss5VMzramp9Iigplw2OzuH92YpvkGBMdRp2Bd1LzCPRzNITU\n8HMINyUPZFCfYB6e23XiDyldFzUWis9QUV3L+n3HmTOiHw6HMC3Rmsh1NxT1SUYhgX4O7p0Vj4gg\nIlw/NprNB09QVFpxSXJkHzvDs6v2cNXwvjw+fwSbsk+wKbuYo6fPsfPIaa5xytL22LzhiFh5lXt0\ns6LvxEf2aOhZ1DvVjRl43jkuoke3hkitrVHfLvXwKRL79WjU7tG5w1j30IyG6ypKS6ixUDqEJ97d\nxatfH76oti9tOMjd/0ihvKplv4nNB09QXlXLHDvtZ4Cfg2vHDGBNViGlFefnIowxrMooZHpiRKOo\nrAvGRmEMDXGV3FHv69AcNbV1/OTtnXQP9OO3N47m1kmxRIUF8bvVe/kk3Vp95GwsRkWH8f69U3nE\n6e2+3lgYY9h15DSB/q1PZjdH39Ag+odaUWNHRjUOqici+Os8hdJG9E5RPE5FdS1vp+Tx+kUai1c2\n5/BpVhE/fDWVyprmM8KtzSoiONCvUbKY/5McS0V1He+knM/3kH60hKOnzzFvVP9G7eMjezA6Ooz3\nd7o3Fi9tOMis//yCmhbiKi3deIi0vBJ+fcNo+oYGERTgx4NzhpJ25DR/XLefEXYeCGfGDOxFiNPb\nfXxkCGUVNRw/U8muoyUkDQi9pMnn+t5FkkZgVS4BNRaKx9lXVGYv3SzjxJnKC2p7+MRZjpw8x9SE\nPmzYX8xDb6VRW2dIzyvh3te3M/qXq7l92VaWbjzE2t1FzBga2ShExeiBYSQP6s0rX+VQa/s0fJJR\niJ9D+FZS00inC8ZGkX60pIljHMDG7GJyTpSz9VDzy3GXp+YxcXA41zrlX75xfDTxkSGUVtRw7ej+\nzbatJ76vNa+wv+gMGUdLuGzgpcVnajAWURrnSbl41FgoHiczv7Rh++uDF5aWZMN+K+DdMwtG8fNr\nR/BRegFX/dcXXPfCRr605yfyTpXz7x9mUVRa6dYA3Dl1MLkny/lsz7GGIajJQ/q49SO47rIoRHDb\nu6jXY1UzKUxzis+y/9gZ5o1sbBD8/Rw8MX8EIYF+bYrGmmAbizVZRZRX1TJ64KWF3VgwNppFE2O5\nLEaNhXLx6MyW4nEy80vo2c0fA2w6UNzorbs1Nu4vJiosiCERIcRPH8KZyhqWp+bxxPzh3DIptmHO\n4cjJcjLzS5kzom+Tc1w9sh8DwoL4+6ZDxIYHc6j4LEumDXZ7vX6hQYyP7c2m7GIe+tbQhvJjZRUc\nL6vE3yGsyijkl9eNxOFoHF5j7W7Lc9ydwZqT1I/0X85t0sYd/UODCA70a5g7udSeRUx4ML+5cfQl\nnUNRtGeheJzM/FJGRIUyaXB4k3hILVFbZ/jqQDHTEiMa4h49OGcoGx+7ih/OiG80OR0THsy8Uf3d\nTtgG+Dm4bfIgvjpwgj+s24eIZUCaY3xsL9KPljTyz9hdYMVXuil5IMfKKtlhhwh35tOsIob370lM\neLDb87bFUIA18Rwf2YMTZ6sIDvRjSDtnkFOUi0GNheJRausMewrKGBkVyuT4PhwqPkv+6XNtarsr\n7zSlFTVMT4y8ZDkWXR5LUICDj9MLuXxQOH17Ns0rXc+42N5U1dSxu+D88FmWPQT1o5kJBPgJn6Q3\nHoo6dbaKlJyTbnsVF0O9c96o6DD82mhkFMWTeNRYiMg8EdkrItki8rib491E5C37+BYRibPL40Tk\nnIjstD8velJOxXMcKj7DuepaRkaFMSXeCkzX1t7Fhv3FiMDUhEsPaNc7JJBvj7NSibqugnKlPvvc\njtzzvYesglKie3UnJjyYaQkRrMosbLSM9vO9luf4nHZKD1rvPDdGkw8pXQSPGQsR8QP+DMwHkoBF\nIpLkUm0JcMoYkwA8DzzrdOyAMWas/bnHU3L6CodPnGVfUdklnycl5yTXv7CRgpK2vf23Rv2k8Mio\nUIb370l4SCBfNWMsXt2c05DOE6z5ipFRoYSHtE9Au3tmxDNzWCTXj215knlAWHf6hwY1RHcFyMov\naYirNH/UAPJOnWs0cb8my/IcH91OD/f6FVFjNKeE0kXwZM9iIpBtjDlojKkC3gQWuNRZALxiby8H\nZotrUH6lVfYUlnLdnzZy9fNf8r2XtrB+3/FWncea45OMQnbllfDYivSLPoczmfmlBPo7SOjbA4dD\nmDykD5sPFDc595f7jvOL9zP53ktb2J57ijOVNWzPPcW0hEsfgqpnUJ8QXr5zIhE9urVad1xsL3bk\nWsaivKqGg8VnG5za5iT1w88h/GHdfs5V1TZ4js+2PcfbgyuHRvLDK4cwe3jTCXtF6Qw8aSyigSNO\n+3l2mds6xpgaoASo96gaLCI7RGS9iEz3oJxezZGT5dy+dCvdA/348Zyh7Csq445lW/nBP1Iv6nyp\nh0/RPcCPL/cd542tuZcsX2Z+CcP69WxwKpsc34f8kopGIbhrauv49Ue7iQnvTmTPbixetpVXvsqh\nps4wvZNyKoyP7U3uyXKKz1Syp7AMY847tYWHBPLI3GGs3V3Et/+yiX9uzaW8qrbd5ivASpP6xDUj\nGjnrKUpn4klj4e4Vy/VVtbk6BUCsMWYc8BDwhog0cT8VkbtFJEVEUo4fbxosztc5XlbJbUu3UFlT\nx6tLJvHAnEQ2PnYVi6fEsXZ30QXHOKqoriUzv4TbpwxiWkIEv/5oN4dPXHxCIGMMmfmljcJMTIm3\n3gU2HShuKHs7JY+9RWX8dP4IXlsyieBAf36/ei/d/B1M6KRsbfXzFjtzTzdMbic56XHPjHheuXMi\nRaUVPP1BVhPPcUXxNTxpLPKAGKf9gYCrp1NDHRHxB8KAk8aYSmPMCQBjTCpwABjq0hZjzN+MMcnG\nmOTIyPYbrvAGjDHc81oqRaWVLFt8OUP7WbGDAv0d3JQ8ELDG/C+E9KMlVNcakgeF87uFY/AT4ZF3\ndjV4Pl8o+SUVnC6vbmQsBkeE0D80iNe+zuXtlCNk5Zfy3Jq9TIwLZ96o/sSEB/PaXZMIDwnkShdv\n7I5kVHQY/g5he+4psgpKCQ3yJ7pX90Z1rhwayUf3T2d6YgS3TIztNFkVpSPwZB93G5AoIoOBo8DN\nwC0udVYCdwCbgYXAZ8YYIyKRWEajVkSGAInAQQ/K6nXsLSoj9fApnrouqcnb94j+ofQJCWRjdjHf\nmTCwzedMPWxNLo+P7UWfHt146vqRPPxOGi+uP8C9sxIuWMZMO2Kqc5gJEeGu6YN54fNsHl2+q6F8\n2eIRDb4UCX178MUjM3F04vRVUIAfSVGh7Mg9zbnqWpKiQpvkuAaI6tWdV5dM6gQJFaVj8ZixMMbU\niMh9wGrAD1hmjMkUkWeAFGPMSmAp8KqIZAMnsQwKwJXAMyJSA9QC9xhjLixOhI/zQVo+fg5xGz7C\n4RCmJESwMduaSG7rmoHUw6cYHBFCH3sC+Dvjo/li7zGeW7OPSYPDSY4LvyAZM/NLEYERAxpHTL1r\n+hC+P3UwB4vPsC3HmiMZ4xLSItTJ4a6zGBfTi3dS86gzhlsmDupscRSlU/Ho7Jkx5mPgY5eyJ522\nK4Cb3LRbAazwpGzejDGGD9IKmBLfp9mVPdMTIvggLZ+9RWUM7996tFFjDNsPn2LmsPOrb0SE/7hx\nNLvySnjgzZ18dP+0VvMyl5RX0zPIH4dDyMwvZUhECMGBTW8zh0NI6NuThL4XF3q7IxgX25tXNluR\ncpOiNGKr8s1GPbi9kF15JeSeLG8xKN00exVRW+ctDp8o58TZqiZDWqFBAfxp0TiKSit4bMWuFpfT\n7i8qY8Kv1nDZ059y29ItbMs5yUgvjnRaP8kNTXNBKMo3DTUWXsjKtHwC/RzMHdm8J3JUr+4MiQxp\niNrqyud7jrHSKclP/XyFu9VHl8X04rF5w1mdWcTs/1rPo8vTWGcHzXNmeaqVM+LfLoui+EwVZypr\nmNYO3tedRWx4MOEhgY3SkSrKNxVdxO1l1NUZPtyVz4xhkYR1b3lcf3pCBG+lHKGyppZu/udX6nyQ\nls8Db+4AILpXEBMGhbM99xQ9u/mT2Nf9Q3HJtMF0C3Cwfu9xPs0q4u2UPD6+f3rD8ExtneG9nUeZ\nOSyyIcJpbZ3x6rhGIsLUhAiKSiranMZUUXwV/Q/wEBlHS3hx/QFOna1q1/NuyzlJUWllm/IiTEuM\npKK6jtSc8yE0VmUU8OBbO0keFM6AsO785O00yqtqSD18inGDejfrgexwCLdPjmPp4stZ//AsggP9\neGnj+QVqXx88QVFpJTeMO+936c2Gop7fLxzDsjsv72wxFKXT0Z6Fh3jmgyy25pzkhc+yuXNqHHdN\nG0JY8IWv8DlWVsEv3stgxIBQrk7qz8q0fLoH+LnN2+DKFUPC6ebvYPHftzE2thdJA0J5fcthLhsY\nxrI7L2dX3mlu+f9b+Pl7GewtKmP+qLblmQgLDuC7yTG89vVhHp07nP5hQby7/Sg9u/m3WyC9roL6\nTiiKhfYsPED2sTK25pzktisGMWNoJH/6LJvZz62npLz6gs/11PuZrN19jD+s2881f9zA61tymT2i\nr9sVRq70DArgnXsms3hqHBXVtfxjcw6josN4+fsT6dHNnynxEdw5NY53tx/FGBg/qO1B65ZMG0yd\nMbz8VQ7nqmpZlVHA/NH99eGqKD6K9iw8wD+3HiHAT3hgTiIRPbqxaH8x31u6hY/SC7hlUmybz7Mq\no5BPMgp5ZO4wvpscw9rdRWzKLuaHV8a3+RxjBvZq8GEor6qhe4BfI7+LR+cOZ/3e4+ScOMvYC4hw\nGhMezPxRA3hjy2Fiw4M5W1XLt8e13QFQURTvQo1FO1NRXcuK7XlcPbJ/gw/E1IQ+xEeG8N6Oo202\nFiXnqnny/QySBoRy95VDCPBzsGhiLIsmtt3YuOKuN9I90I+liy8n42hJo8xzbeGu6YP5KL2Af/8w\ni6iwICYNvjCnPUVRvAcdhmpnVmcWcrq8mlucHuoiwo3jB7I15yRHTpa7bbct5yT3vbGdf2zO4VDx\nWX7z8W6Kz1Ty7HfGNERs9RSDI0LaNGHuyrjY3lwe15tz1bUsGBfdbuG5FUXpeqixaGfe2JLLoD7B\nTSKQXm8/jJ19G5z5zce7+Ti9gCffz2TWf37Bm9uO8IPpQxg9sGs7td13VSLdA/xYeAExqBRF8T50\nGKodOXD8DFsOneSxecObvGXHhAczMS6cd7fn8aOZ8Y3mDXYeOc323NM8dV0Ss4b1ZcP+4xw5dY4H\n5zQJtNvlmDE0koyn5/rEMllFUZpHjUU78ubWXPwd0uxb9g3jovnpv9LJzC9llFP6zb9vOkSPbv4s\nnDCQnkEBxEWEdJTI7YIaCkXxfXQYqp2oq7OC+80c1pfInu6D+107egCBfg7e3X60oayotIKPdhXw\n3eSYC55gVhRF6SjUWLQTu46WUFhawTWjm4/XFBYcwKzhkaxMy6f4TCUAr24+TK0xLJ4S10GSKoqi\nXDg6DNVOrMooxN8hzB7esgfzd5NjWJ1ZRPKv1hLRoxtnK2uYPbwfsX2CO0hSRVGUC0eNRTtgjGFV\nRgGT4/u0GtJj9oh+rPi/k9mRe5o9hWUcPnGWB2YndpCkiqIoF4cai3ZgX9EZck6Uc9f0IW2qP2FQ\nOBMGqQOboijeg85ZtAOrMgoRgauTfCuInqIoSj0eNRYiMk9E9opItog87uZ4NxF5yz6+RUTinI49\nYZfvFZG5npTzUlmdWciE2N70DQ3qbFEURVE8gseMhYj4AX8G5gNJwCIRSXKptgQ4ZYxJAJ4HnrXb\nJgE3AyOBecBf7PN1OXJPlJNVUMq8Uc2vglIURfF2PNmzmAhkG2MOGmOqgDeBBS51FgCv2NvLgdli\nuTYvAN40xlQaYw4B2fb5uhyrMwsBWkxxqiiK4u14coI7GjjitJ8HTGqujjGmRkRKgD52+dcubaPx\nAKfLq7jpxc0X3b6wpIKRUaHEhOvSV0VRfBdPGgt3MSBMG+u0pS0icjdwN0Bs7MWF7nY4hMR+7vNO\nt4XEfj24KTnmotsriqJ4A540FnmA81N0IOAacrW+Tp6I+ANhwMk2tsUY8zfgbwDJyclNjElbCA0K\n4C+3TriYpoqiKN8YPDlnsQ1IFJHBIhKINWG90qXOSuAOe3sh8JkxxtjlN9urpQYDicBWD8qqKIqi\ntIDHehb2HMR9wGrAD1hmjMkUkWeAFGPMSmAp8KqIZGP1KG6222aKyNtAFlAD3GuMqfWUrIqiKErL\niPUi7/0kJyeblJSUzhZDURTFqxCRVGNMcmv11INbURRFaRU1FoqiKEqrqLFQFEVRWkWNhaIoitIq\naiwURVGUVvGZ1VAichw47FIcARR3gjiexNd08jV9wPd08jV9wPd0uhR9BhljIlur5DPGwh0iktKW\nJWHehK/p5Gv6gO/p5Gv6gO/p1BH66DCUoiiK0ipqLBRFUZRW8XVj8bfOFsAD+JpOvqYP+J5OvqYP\n+J5OHtfHp+csFEVRlPbB13sWiqIoSjvgs8ZCROaJyF4RyRaRxztbnrYgIstE5JiIZDiVhYvIGhHZ\nb3/3tstFRP5o67dLRMZ3nuTuEZEYEflcRHaLSKaIPGCXe7NOQSKyVUTSbJ2etssHi8gWW6e37LD8\n2GH237J12iIicZ0pf3OIiJ+I7BCRD+19b9cnR0TSRWSniKTYZd583/USkeUissf+f5rc0fr4pLEQ\nET/gz8B8IAlYJCJJnStVm3gZmOdS9jiwzhiTCKyz98HSLdH+3A38tYNkvBBqgJ8YY0YAVwD32n8H\nb9apErjKGHMZMBaYJyJXAM8Cz9s6nQKW2PWXAKeMMQnA83a9rsgDwG6nfW/XB2CWMWas05JSb77v\n/gCsMsYMBy7D+lt1rD7GGJ/7AJOB1U77TwBPdLZcbZQ9Dshw2t8LDLC3BwB77e3/ARa5q9dVP8D7\nwLd8RScgGNiOlVu+GPC3yxvuP6x8LpPtbX+7nnS27C56DMR62FwFfIiV1thr9bFlywEiXMq88r4D\nQoFDrr9zR+vjkz0LIBo44rSfZ5d5I/2MMQUA9ndfu9yrdLSHK8YBW/Bynewhm53AMWANcAA4AclQ\nhAAABuVJREFUbYypsas4y92gk328BOjTsRK3yn8DjwJ19n4fvFsfAAN8KiKpInK3Xeat990Q4Djw\nd3uo8CURCaGD9fFVYyFuynxt2ZfX6CgiPYAVwIPGmNKWqrop63I6GWNqjTFjsd7IJwIj3FWzv7u0\nTiLyb8AxY0yqc7Gbql6hjxNTjTHjsYZk7hWRK1uo29V18gfGA381xowDznJ+yMkdHtHHV41FHhDj\ntD8QyO8kWS6VIhEZAGB/H7PLvUJHEQnAMhSvG2PetYu9Wqd6jDGngS+w5mN6iUh9mmJnuRt0so+H\nYaUQ7ipMBa4XkRzgTayhqP/Ge/UBwBiTb38fA/6FZdS99b7LA/KMMVvs/eVYxqND9fFVY7ENSLRX\ndARi5fZe2ckyXSwrgTvs7Tuwxv3ry2+3Vz5cAZTUd0m7CiIiWHnWdxtjnnM65M06RYpIL3u7OzAH\na7Lxc2ChXc1Vp3pdFwKfGXsguStgjHnCGDPQGBOH9X/ymTHmVrxUHwARCRGRnvXbwNVABl563xlj\nCoEjIjLMLpoNZNHR+nT25I0HJ4WuAfZhjSf/rLPlaaPM/wQKgGqst4MlWOPB64D99ne4XVewVnwd\nANKB5M6W340+07C6v7uAnfbnGi/XaQyww9YpA3jSLh8CbAWygXeAbnZ5kL2fbR8f0tk6tKDbTOBD\nb9fHlj3N/mTW//97+X03Fkix77v3gN4drY96cCuKoiit4qvDUIqiKEo7osZCURRFaRU1FoqiKEqr\nqLFQFEVRWkWNhaIoitIqaiwUr0JExonISy0cjxKR5R68/k121M/PXcrjROSWizznV22o85IngmGK\nyC9F5OFW6tzQlmuLyH0icmf7Sad0JdRYKN7GT4E/NXfQGJNvjFnY3PF2YAnwI2PMLJfyOMCtsXDy\nhHaLMWZKaxc1xtxljMlqq5DtzA1Y0ZtbYxlwv4dlUToJNRZKp2C/ie+x35gzROR1EZkjIpvs+PwT\n3bTpCYwxxqTZ+zPsfAU77QBrPe3zZtjHX3I6flxEnrLLHxGRbXas/6ebkW+RWPkQMkTkWbvsSSxH\nwxdF5PcuTX4LTLev9WMRWSwi74jIB1gB7XqIyDoR2W6fd4HTtc7Y3zNF5As5n7fgddsLHrs8ub6+\niPxarJwaX4tIP7s83t7fJiLP1J/XjW4/EyvXy1pgmFP5D+y2aSKyQkSCRWQKcD3we1u3eHf1AIwx\n5UCOu7+d4gN0tmeifr6ZH6w38RpgNNZLSyrWm6kAC4D33LSZBaxw2v8AK2AcQA+sgGtxOIV4t48N\nAvbY31dj5SsW+7ofAle61I8CcoFI+5yfATfYx77AjUcsTt7P9v5iLC/8eq9afyDU3o7A8oCud4o9\n43SOEqxYPg5gMzDN9bpYXvHX2du/A35ub3+IHZoauKf+vC5yTsDy6g3GCn2dDTxsH+vjVO9XwP+z\nt18GFjodc1vP3v8ZVg6TTr/H9NO+H+1ZKJ3JIWNMujGmDisswzpjPXHSsR76rgzACtVczybgORG5\nH+hlzofUbkBE6sNT3GeMOYxlLK7GCtmxHRiOlSTGmcuBL4wxx+1zvg60FLW0OdYYY+qD7AnwHyKy\nC1iLFTK6n5s2W40xefZvshP3v0MVlmEAy8jW15mMpSvAG83INB34lzGm3FgRgJ1jpo0SkQ0ikg7c\nCoxs5hwt1TuGZWwVH6PFsVRF8TCVTtt1Tvt1uL83z2HFJgLAGPNbEfkIK97U1yIyB6hwafMi8K4x\nZq29L8BvjDH/04Jc7kI8XwxnnbZvxeqpTDDGVIsV5TXITRvn36QW979DtW1UW6rTEs3F+HkZqweV\nJiKLsXo6F1ovCOvvpPgY2rNQvIndQEL9jojE2z2TZ7GCrA13riwi9wI9jTG/dSpeDXxfrBwbiEi0\niPSlMVuAGSISIVaK3kXA+lZkKwN6tnA8DCtvRLWIzMIaEmtvvga+Y2/f3EydL4Fvi0h3ew7oOqdj\nPYECscLK3+pU7qpbc/UAhmIFWFR8DDUWitdgjNkDhNkPOYAH7QnoNKy32U9cmjwMjHaa5L7HGPMp\n1hDNZnsYZTkuD3ljhXN+AitMdxqw3RjzPi2zC6ixJ31/7Ob460CyiKRgPWD3tFXvC+BB4CER2Yo1\nZFfiWsEYsx14C2uIawWwwenwL7AM5RoX+d4EHrEXEcS3UA+s/BhrUXwOjTqreBX2g7jMGNOsr8U3\nFXtV0jljjBGRm7Emuxe01q4drz8OeMgYc1tHXVPpOHTOQvE2/grc1NlCdFEmAC/Yy21PA9/v4OtH\nYPU6FB9EexaKoihKq+ichaIoitIqaiwURVGUVlFjoSiKorSKGgtFURSlVdRYKIqiKK2ixkJRFEVp\nlf8FfkFIwbPNpq4AAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# first we split the training set into a training / validation set (as we are using k-folds cross validation elsewhere and don't have an explicit training and validation set.)\n", "X_train_train, X_cv, y_train_train, y_cv = model_selection.train_test_split(X_train, y_train, test_size=0.25, stratify=y_train)\n", "\n", "# set the model to calculate learning curves for\n", "#model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", "\n", "m_array = np.round(np.linspace(20, X_train_train.shape[0], 100)).astype(int)\n", "train_acc_array = []\n", "cv_acc_array = []\n", "\n", "for m in m_array:\n", " model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", " # we now fit to the training data\n", " model.fit(X_train_train.head(m), y_train_train.head(m)) # training on the first m training data examples\n", " train_accuracy = model.score(X_train_train.head(m), y_train_train.head(m))\n", " train_acc_array.append(train_accuracy)\n", " cv_accuracy = model.score(X_cv, y_cv)\n", " cv_acc_array.append(cv_accuracy)\n", "\n", "train_acc_array = np.array(train_acc_array)\n", "cv_acc_array = np.array(cv_acc_array)\n", "\n", "fig, ax = plt.subplots()\n", "ax.plot(m_array, 1-train_acc_array, label='train')\n", "ax.plot(m_array, 1-cv_acc_array, label='cross-validation')\n", "ax.legend()\n", "ax.set_xlabel('m (size of training data)')\n", "ax.set_ylabel('error (1-accuracy)')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This learning curve suggests that the neural network is a high variance algorithm for this problem and amount of training data. Getting more data will help improve the fitting, adding more features will just increase the variance and amount of data needed to fit. More data would cause this gap between the 2 error curves to drop such that the accuracy improved overall." ] }, { "cell_type": "code", "execution_count": 142, "metadata": { "collapsed": true }, "outputs": [], "source": [ "hidden_layer_sizes=(3,3)\n", "alpha=1e-1\n", "\n", "nn_model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)" ] }, { "cell_type": "code", "execution_count": 143, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mean score: 0.8175000000000001\n", "variance in score: 0.00350625\n" ] } ], "source": [ "N_splits = 10\n", "kfold = model_selection.KFold(n_splits=N_splits)\n", "\n", "scoring = 'accuracy'\n", "score = model_selection.cross_val_score(nn_model, X_train, y_train, cv=kfold, n_jobs=1, scoring=scoring)\n", "\n", "print(\"mean score: {}\".format(score.mean()))\n", "print(\"variance in score: {}\".format(score.var()))" ] }, { "cell_type": "code", "execution_count": 144, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 144, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnWd4lFXagO+T3nsgoSX0mtA7IgoqKGLDtop117WthVUX\n1+63rq7uuuouu6JrFwsWFBEbShdQSggloQUSQhIgPSE9Od+PM2+mZGYyk2SSAOe+rrnezFvPZJLz\nnKcLKSUajUaj0TjDq6MHoNFoNJrOjxYWGo1Go2kWLSw0Go1G0yxaWGg0Go2mWbSw0Gg0Gk2zaGGh\n0Wg0mmbRwkKj0Wg0zaKFhUaj0WiaRQsLjUaj0TSLT0cPoK2IiYmRiYmJHT0MjUajOaXYunVrvpQy\ntrnzThthkZiYyJYtWzp6GBqNRnNKIYTIdOU8bYbSaDQaTbNoYaHRaDSaZtHCQqPRaDTNctr4LDQa\njWeora0lOzubqqqqjh6KphUEBATQo0cPfH19W3S9FhYajcYp2dnZhIaGkpiYiBCio4ejaQFSSgoK\nCsjOzqZ3794tuoc2Q2k0GqdUVVURHR2tBcUpjBCC6OjoVmmHWlhoNJpm0YLi1Ke136EWFlUlsPo5\nyN7a0SPRaDSaTosWFrIBVj8LRzZ19Eg0Go0diouL+c9//uP2dRdeeCHFxcUeGNGZiRYWARHg5Qsn\nT3T0SDQajR0cCYv6+nqn161YsYKIiAhPDeuMQ0dDCQHBsVCuhYVG0xlZsGABBw8eZMSIEfj6+hIS\nEkJ8fDwpKSns2bOHSy+9lCNHjlBVVcW9997LbbfdBphLAJWXlzNr1iymTJnCzz//TPfu3fnyyy8J\nDAzs4E92aqGFBUBwjNYsNBoXeOqr3ezJKW3Tew7pFsYTFw91ePy5555j165dpKSksHr1ai666CJ2\n7drVGAL65ptvEhUVRWVlJWPHjuWKK64gOjra6h779+/nww8/5PXXX+eqq67is88+4/rrr2/Tz3G6\no4UFKM1CCwuN5pRg3LhxVrkCr7zyCkuXLgXgyJEj7N+/v4mw6N27NyNGjABg9OjRHD58uN3Ge7qg\nhQUoYZG/v6NHodF0epxpAO1FcHBw48+rV69m5cqVbNy4kaCgIKZNm2Y3l8Df37/xZ29vbyorK9tl\nrKcT2sENZjOUlB09Eo1GY0NoaChlZWV2j5WUlBAZGUlQUBDp6els2qSjGj2F1ixAaRZ1lVBzEvxD\nOno0Go3GgujoaCZPnsywYcMIDAyka9eujcdmzpzJq6++SnJyMgMHDmTChAkdONLTGy0sQAkLUNqF\nFhYaTafjgw8+sLvf39+fb775xu4xwy8RExPDrl27Gvc/8MADbT6+MwFthgII6aK22smt0Wg0dtHC\nApTPArSw0Gg0GgdoYQHWZiiNRqPRNEELC4AgrVloNBqNM7SwAPANAP8wOJnf0SPRaDSaTokWFga6\n5IdGo9E4RAsLg+AuUH68o0eh0WhOUVavXs3s2bMBWLZsGc8995zd80JCnIfn21bZzcnJYe7cuW03\n0BbiUWEhhJgphNgrhDgghFhg5/jtQoidQogUIcR6IcQQi2MPm67bK4S4wJPjBEyahTZDaTSnInV1\ndR09BCvmzJnDggVNpjyXsBUW3bp149NPP22robUYjwkLIYQ3sBCYBQwBrrUUBiY+kFImSSlHAM8D\nL5quHQJcAwwFZgL/Md3Pc+highpNp+bdd98lOTmZ4cOHM2/ePG666Sbmz5/POeecw5/+9CcKCwu5\n9NJLSU5OZsKECaSmpgKwZs0aRowYwYgRIxg5ciRlZWXk5uYydepURowYwbBhw1i3bl2T540fP57d\nu3c3vp82bRpbt27ll19+YdKkSYwcOZJJkyaxd+/eJte+/fbb3H333QAcOnSIiRMnMnbsWB577LHG\nc8rLy5k+fTqjRo0iKSmJL7/8ErAuyf7ggw9y+PBhhg0bBqh+6DfffDNJSUmMHDmSVatWNT7v8ssv\nZ+bMmfTv35+HHnqojX7rZjyZwT0OOCClzAAQQnwEXALsMU6QUlrWOg4GjOJMlwAfSSmrgUNCiAOm\n+2302GiDY6GiABrqwcuzckmjOWX5ZgHk7Wzbe8YlwSz7JhuD3bt388wzz7BhwwZiYmIoLCxk/vz5\n7Nu3j5UrV+Lt7c0f/vAHRo4cyRdffMFPP/3EDTfcQEpKCn//+99ZuHAhkydPpry8nICAAF577TUu\nuOACHnnkEerr66moqGjyzGuuuYYlS5bw1FNPkZubS05ODqNHj6a0tJS1a9fi4+PDypUr+fOf/8xn\nn33mcOz33nsvd9xxBzfccAMLFy5s3B8QEMDSpUsJCwsjPz+fCRMmMGfOHKuS7IBVhVzj+p07d5Ke\nns7555/Pvn37AEhJSWH79u34+/szcOBA/vCHP9CzZ0+Xv4bm8KQZqjtwxOJ9tmmfFUKIu4QQB1Ga\nxT3uXNumBMcCEioKPfoYjUbjPj/99BNz584lJkaFuUdFRQFw5ZVX4u2tFnfr169n3rx5AJx77rkU\nFBRQUlLC5MmTmT9/Pq+88grFxcX4+PgwduxY3nrrLZ588kl27txJaGhok2deddVVfPLJJwAsWbKE\nK6+8ElDFC6+88kqGDRvG/fffb6V92GPDhg1ce+21AI3jA5BS8uc//5nk5GRmzJjB0aNHOXbsmNN7\nWX7GQYMGkZCQ0Cgspk+fTnh4OAEBAQwZMoTMzEyn93IXT2oWws6+JmVdpZQLgYVCiN8AjwI3unqt\nEOI24DaAXr16tWqwVlncIbGtu5dGc7rSjAbgKaSUCNF0WrAsVy7tVI0WQrBgwQIuuugiVqxYwYQJ\nE1i5ciVTp05l7dq1fP3118ybN48HH3yQ0NBQnnrqKQD+97//MWbMGKKjo0lNTeXjjz9m0aJFADz2\n2GOcc845LF26lMOHDzNt2rRmx29v7IsXL+bEiRNs3boVX19fEhMT7ZZXt/09OMK2DHtb+3E8qVlk\nA5Y6UA8gx8n5HwGXunOtlPI1KeUYKeWY2NhWTvC6PpRG02mZPn06S5YsoaCgAIDCwqYWgKlTp7J4\n8WJARSbFxMQQFhbGwYMHSUpK4k9/+hNjxowhPT2dzMxMunTpwu9+9ztuvfVWtm3bxmWXXUZKSgop\nKSmMGTMGUKao559/npKSEpKSkgClWXTvrgwdb7/9drNjnzx5Mh999BFA4/iM+3Tp0gVfX19WrVrV\nqAk4K8lu+Rn37dtHVlYWAwcObHYMbYEnhcWvQH8hRG8hhB/KYb3M8gQhRH+LtxcBRgeiZcA1Qgh/\nIURvoD/wiwfHeuaV/CjNhcKMjh6FRuMSQ4cO5ZFHHuHss89m+PDhzJ8/v8k5Tz75JFu2bCE5OZkF\nCxbwzjvvAPDSSy8xbNgwhg8fTmBgILNmzWL16tWNDu/PPvuMe++91+5z586dy0cffcRVV13VuO+h\nhx7i4YcfZvLkydTX1zc79pdffpmFCxcyduxYSkpKGvdfd911bNmyhTFjxrB48WIGDRoEWJdkf/DB\nB63udeedd1JfX09SUhJXX301b7/9tpVG4UmEM7Wm1TcX4kLgJcAbeFNK+YwQ4mlgi5RymRDiZWAG\nUAsUAXdLKXebrn0EuAWoA+6TUtqvQ2xizJgxcsuWLS0fbEUhPN8bZj4HE+5o+X1OFd67HMqPwR0b\nOnokmk5OWloagwcP7uhhaNoAe9+lEGKrlHJMc9d6tJ+FlHIFsMJm3+MWP9sX5+rYM8AznhudDQER\nILzPDM2ivhayNkF9tfrZ27ejR6TRaDo5OoPbwMvrzCn5kZcKtSehoQ4KD3X0aDQazSmAFhaWBMee\nulncy+6BrxwqatZkWqSr5O9z/1m7v4BXRkF958qa1XgOT5qrNe1Da79DLSwsOZWzuI9uU6YlV8ja\nCCFx6uf8ptmnzXJsFxQehMoi96/VnHIEBARQUFCgBcYpjJSSgoICAgICWnwP3YPbkuDYUzdCqLII\nqkubP09KJSz6XwAZq+FECzSL6nLzM3VOymlPjx49yM7O5sSJU3QhpQGU0O/Ro0eLr9fCwpJT2QxV\nVQw15Woi93dS1TJ/nyprkjARynJaZoaqLjM/05LaKvjmQZj2MIR1c/++mk6Jr68vvXv37uhhaDoY\nbYayJDhGOX5rTnb0SNyjvlYJCoCyXOfnZv6str0mQcxAyN+vtA13MDQYWzPUsd2w7V3Y86V799No\nNJ0eLSwsaUzMO8W0i0qLFX6psyR5lAkqOBai+0JMf6gpa17A2GIIpkobzaLSlFWbm+re/TQaTadH\nCwtLTlVhYWkOsp34D65S2doGmRuh10QQAmJNZQJOuOnkdmSGMjSNPC0sNJrTDS0sLAk5RUt+WJqD\nLDWL+lpYfCW8eYHaX5INJVmQMEkdjxmgtu76LSwd3JYYFXtPpENdtXv31Gg0nRotLCxp1CxOsfaq\nlQ40i7JcaKiF4kx49xJIW67295qotiFdwT+8BcKirOlzwWyGaqiD42nu3VOj0XRqtLCwJMiiTPmp\nhLHC9/a31ixKjqrtWX+E4iPw7QLwC4GuquuWMkUNcN8MVWMICweaBTg3RdXXQUODe8/UaDQdihYW\nlvgFgV+oMtecShi+g9gB1ppFqUlYJF0F17wPXj5Kq/C2iJiOGeCeZiGlc59FZG/1O3Tm5F56G/x3\nIpSfYhqcRnMGo4WFLYmTYf8P7oeTdiSGOajLEGtntiH0wrtDvxlw+3qY8y/ra2MGqOqztiYlR9RW\ngjRpBbaaRWUhBEVD3DDHmkVlMexZpvwa712mOxNqNKcIWljYMmg2lByB3B0dPRLXqSxSq/nwnmri\nbzDV2C89qnwS/qaWkV0GQVi89bWNTu79uES1RVMWWwFTUQiBkaqnct4u+6am/d8rP8o5jyqNZvFc\n63tqNJpOiRYWtgycBcIL0pe33T1zd8CntzovvJf5Myz7Q8s0mqpiCIxQgkDWm807JUeVVuEMI3zW\nVVOUkWPhG2TfDBUUBXHJKrnRXumUtK9UXaqz/ghXvgM5KbBwPLx6FiyaCvu+c20cGo2mXdHCwpbg\nGEiYrCa1tmLPl7DrUyh14gvZ963Kfm5JjkdlkerHEWYSDIaTuzTbvM8REQng7ed6QUEjezu8p3qu\npXCrLILAKIhPVu/zbLSz2ko4sBIGXaRKwg+6EK5ZDPHD1TiPpykToEaj6XRoYWGPQbOVTT3/QNvc\nz4g2ciYIqkztFota0F+i0qRZhJpMTGUmYeGKZuHtA1F9XS8oaJiMInpCfY0SAKByOqpLlRkqdjB4\n+TZ1ch/8CWorYPBs876Bs+DaD+E3HymNw9BcNBpNp0ILC3sMukht013QLg78qGoiOcPwBziL/qky\nrdiLDjf/zCbXGmYoU/G+0lxV1K8iH8JcqDIZ6yAiqr4Otr2nBIGBkZAX3lNtDSe3sQ2KAh8/iB3U\n1MmdthwCwiHxLPvj8A/V/guNppOihYU9InpC/AhzEpsjaqvg43nKH+Eob6C+VvV+AOf5G4Zm0ZLO\ndZVFakUfFKNW9GU55rDZ5jQLUFFURYfMAstg7wpYdjdkrDHvs9QswOy3MIRFYKTaxicrzcIwU9XX\nwb5vYMBMx21c/UO0sNBoOilaWDhi8MVwdIvzwnwZq5Qj90SamgjtUXRYZTSDa8LCXTOUlMoMFRCh\n/AChcUqzMIRFcz4LgJ7jVTjskV+s92eZOupZ5m4YCXmNmoVJWBghsEFRahuXrDSbsjz1PnODEiiD\nLExQtmjNQqPptGhh4YjBF6tt+teOz0lbDv5hEJkI6/5hP5LJMjvamc/CcBy7q1nUVkJ9tTJDgfJb\nlOWYs7fDXTBD9RgLwhuyfrbeb5QzLz9mMU5bYWGYoUzCwlKzABVVlrsDUj4An0DoN93xOPy0ZqHR\ndFa0sHBE7ECI7uc4lLO+TplpBlwAk++Fo1vh0Nqm5xlRRs21bG2pZmGYgYxJOizepFmYIq9caULk\nH6Iikix7c1eXmX0OVsKiXIUWG/kaVTaaRaBJs+g6TJnEVjygQmJTP4L+M8Av2Mk4QrWDW6PppOhO\nec7oOd5xKGfWz2o1Pfhi1aJ09d+UdtHnbOvz8ver1X5Er2aERamahMuPQU2FKj3iCoYZKMDQLLop\np3vJUZVN7Rvo2n0SJsEvr6tqsT7+yiRlZGrbahZ+oWbhZM/BDRAQBr/9wdqM13O88zFoM5RG02nR\nmoUz4pJVBVrD7m5J2nLwCVBlNHwDYNLdcGgNZG+xPu/EXpUl7UyzqKuGukoVQQTuRUTZOpbD4tXq\n/MRe1/wVBr0mKnPW0W3qfdZGJby6jbSO4qopV5O6f5gyXRnCqrJQaRJ+Fi1du41UkWXGKzjG+RgM\nzUIXGdRoOh1aWDgjLkltbfMFpFS+jL7nms0qo29Wq/sNL1mfl79fmbSCYxwLCyMKKX642rpjimo0\nQ1loFgA5213zVxgYZcsNv0XmRiUso/paC8vqUmW2EkKFwRrCyij1IYTrz7TFKEuiTVEaTafDo8JC\nCDFTCLFXCHFACLHAzvH5Qog9QohUIcSPQogEi2P1QogU02uZJ8fpEENY2GYi52xXPgHLyB7/EBhx\nnfJxGJN/Wa6KHooZAMFdoKLAXLfJEsO5HT9Cbd1xchuTtWGGMnwJdZXuaRbB0aond+ZGpekc3aJM\nUyFdrTWL6nLzpB4YaR06a5igWoqhlWhhodF0OjwmLIQQ3sBCYBYwBLhWCDHE5rTtwBgpZTLwKfC8\nxbFKKeUI02uOp8bplIAwVXI7b6f1/rSvlAlm4Czr/YNnq6zmAyY/h5HoZpihZEPTSq1gnnAjE9Rq\n3R3NotLGwR1qUSjQlRwLSxImwpHNyhRVV6W0jdCuKjzYSMarLrMQFhEWZqgis3O7pRj31X4LjabT\n4UnNYhxwQEqZIaWsAT4CLrE8QUq5SkpZYXq7CXDDbtJOGMlllqQvV6XMbVfSPcerxDgjme+EpbBw\n0ljJ0EQCwpVwckezqCoGhPIhgHX0kyvZ25b0mqS0nF//Z3o/UWkWYHZyV5eZNYCACGszVGs1i0Zh\noTULjaaz4Ulh0R04YvE+27TPEbcClpltAUKILUKITUKIS+1dIIS4zXTOlhMnPNTdLi7JlN1shLYe\nVhrDwAubnuvlrRy5+79X2d35+9QkHhpn0bLVnrAw3dvI2XDXwR1oSsgDFf1kmKTc1Sx6TVDb3Z9D\ndH/Vkzyki9pnCIuacrNgsjJDFZr9Ji2lUViUOj9Po9G0O54UFvY8nXbrbwshrgfGAC9Y7O4lpRwD\n/AZ4SQjRt8nNpHxNSjlGSjkmNja2LcbclDiT0zlvl9oapS/6nmv//MEXqwn10BqVYxEzQDl9XREW\nAeEQ1RuKs+z7NuxhZG9bYmgX7vgsQIX3hnVX5jJDcITEqa2lZuFv0iwCI6xDZ1trhtI+C42m0+JJ\nYZEN9LR43wNoUjtDCDEDeASYI6WsNvZLKXNM2wxgNTDSg2N1TGO5bZPfImO1mkCNpkG29J6q8hDS\nvlKRUMZ5hrAotyMsqm3MUA21rrd2NTQLS0LjAeFaQp4lQpijohImqa1hhio7Zm6pauXgLoGak8rH\n0WZmKO2z0Gg6G06FhRAiQAgxVwjxshDiEyHEu0KIh4QQQ124969AfyFEbyGEH3ANYBXVJIQYCSxC\nCYrjFvsjhRD+pp9jgMnAHvc+WhsRGqcimfJSVfz/obXQZ5rjEFEffxhwPqQtU9FQsSZhERipnOIO\nNQuhVtZRvdU+V53cVcVm57ZB7ECI7uu4YJ8z+p6renUnTjGP28vXlCx4EpDWPgvZAEWZ5nNbgxYW\nGk2nxaGwEEI8CWwAJgKbUZP6EqAOeE4I8YMQItnR9VLKOuBu4DsgDVgipdwthHhaCGFEN70AhACf\n2ITIDga2CCF2AKuA56SUHSMsQPktclPh+G5VHK/PNOfnD5ptNi0ZmoWXl+Nci6pSFXnl5aU0C3Dd\nyW3PDHXuY3BLCzvODb8W7tmuTFLGuEO6qPBZwzxkGQ0F5o54OhpKozltcVbu41cp5ZMOjr0ohOgC\n9HJ2cynlCmCFzb7HLX6e4eC6n4EkZ/duV+KT4ed/m0t/2Jb0sKX/eeDtrzKiYwaa9wfH2i8mWFWi\nTFCgTEdevq47uY3y5Jb4BbleLsQWLy+zoDAI6QLleeZJ3NIMBWZh0VozlI+/+uxaWGg0nQ6HmoWU\n8msAIcQwB8ePSym32Dt22hGXrPwIv76hNIXmfAH+odD3HDXxRSaa9zvULErA3yQsvLxVvoUrZigp\n1bWtjUJqjpA4ZYayFRYBbaxZGPfWDm6NptPhioP7VSHEL0KIO4UQHp6VOilxJmtbaTb0bkarMDjv\nabjif6ptqYGj+lDVpWbNAlzPtaguA1nf1AzV1oR0UQ5uQ1j4WURDgbm5U2t9FqCLCWo0nZRmhYWU\ncgpwHSqyaYsQ4gMhxHkeH1lnIqqPeYLsM821a2IHwlCb9BCnZqgwi+f1VmYoe/0xLLEtIugpQroq\nX01jPoitGcok2FprhjLurZPyNJpOh0uhs1LK/cCjwJ+As4FXhBDpQojLPTm4ToOXl+rPILzMUUIt\nIThG1YqqrbTeX2VHs6guNfeIcIRtEUFPEdrVFPVkEgq2ZqiSbPANVj6H1uIfqpPyNJpOSLPCQgiR\nLIT4Jyqi6VzgYinlYNPP//Tw+DoPI6+Hcbe1bmIONmVD25qiLB3cAHEmN5Ft5zpb2lOzACg4oLaG\nsPANVI58ZNtoFaA0OO2z0Gg6Ha5oFv8GtgHDpZR3SSm3QWPS3KOeHFynYtQ8mPW31t3DXhZ3Q4Op\n7LeFGarXRLVqN2pMOcK28ZGnaBQWJt+EISyEMAvPttJutM9Co+mUuCIsLgQ+kFJWAgghvIQQQQBS\nyvc8ObjTjkZhYeG3qCkDpLVm4e2rKtru+wbqa63vkb1VddKD9jNDWWoWwls1fTIwBFVbREKBKiWi\nfRYaTafDFWGxErDszRlk2qdxF3uVZy3rQlky+GJ17PB68768XfC/c+HHp9T7djNDWRQT9A+1zl43\nnt1WZij/MK1ZaDSdEFeERYCUsnGpZ/q5hRlfZzhOhUWY9bl9zwXfIFVjymC9yUW09R1VY6qyWOVy\n+Hr46/ANNOeBGCYog8A21iz8QlT/DFcLKWo0mnbBFWFxUggxyngjhBgNVDo5X+MIv2AVNWRphrLs\nZWGJbyD0m67atzY0KH/B7s+VxlFXBZv/a87ebk0rU1cJNZmimgiLSOtta9GtVTWaTomzch8G96Fq\nNxkVY+OBqz03pNOckFjrNqWWvSxsGXSx0iyOboXt7ykt4sJ/KL/BL69D99Ge91c0jrur6s9h5JsY\nGD6LNjNDme5fXdZUgGo0mg6jWWEhpfxVCDEIGIjqUZEupaxt5jKNI2yzuKsdaBagqtd6+cAvr8Hu\npTD6RrXCP2s+7PkCMlZBj3HtM27Db+FpM5TulqfRdEpc7WcxENVHeySql/YNnhvSaY5tFnejz8KO\nhhAYqfpj7FyikuIm3aP2xw+HfueZz2kPjCZI/jaaRZuboUwalnZyazSdCleS8p4A/mV6nQM8D8xx\nepHGMbbFBB05uA0GzVbb5KtUgUGDs/6otu1mhnKgWbS1GaqxW95pJixKjsKyPzTN3tdoThFc0Szm\nAtOBPCnlzcBwoA3qOpyhBMeqOksNDep9VYmKZnLUqGjoZTB4Dpz9J+v9CRNh4t3qeHsQatIs/GyE\nRe+zYNhc6OpKPywXOF17Wmx5E7a9C0d+6eiRaDQtwhUHd6WUskEIUSeECAOOA308PK7Tl+BYaKhT\nCXVBUU1LfdgSFAVXO8h9vOAZz4zRHo40i7BuMPeNtntOo4P7NPNZpJuy8fP3Nd8PRaPphLiiWWwx\nlSZ/HdiKKv2hl0ctxTaL27bUR2fFyOK29Vm0NW3lszj4k9nE19HkH4AT6aaf93XsWFyhrkZF4dlW\nD2hvCg7C8bSOHYOmkeZ6cAvgWSllsZTyVeA84EaTOUrTEoxmSEaRwOY0i85CRC8V8RQ7yLPPaQuf\nxZFf4L3LYFsnqUaTbkqsDOsBJ/Z27FhcYf938PH18MUdHZsc+c1DsPT3Hfd8jRVOhYWUUgJfWLw/\nLKVM9fioTme6j4b4EbDhZfWPaNvLorPiHwp/OgQDLvDsc3z8VCXb1mgW6/6htqVH22ZMrSVtOXQb\nqfw7+fs7ejTNY1QX3vkJLL+/+b4qnqIkW2llHfV8jRWumKE2CSHGenwkZwpCqEimwgyVK2Hby0Jj\nKibYQmGRtwv2fat+LsttuzG1lNIcOLpFRbXF9IeyHHPWfmel8BAERcOU+bDtHfjuEfU5SnNUx8T2\noixXlX6x112yraiv1cLIRVxxcJ8D/F4IkQmcRCXmSSllskdHdjozaLbq5b3uReXo1sLCmtZ0y1v/\nT2XKikxs34nNEelfq+3gOWZ/Rf5+6DG648bUHEWHVAOu6Y9DzUnYtFC9DJKvhkv/q/rFe4raSrPP\nqfCQOcCiTZ9RBS8OghlPqYRXjVNcERazPD6KMw0vL7Vq++J29f5UcHC3J34t7Glh1M+aeLcyQeVs\nb/uxuUvaV2phEDvAXMMrf18nFxaHoed4Nd6ZzynzWUWBOnY8DTa/qroiXvyK5+qSlVsI+qLD0Gt8\n2z+jOEvVV9vxoRYWLuCKsNA6midImgur/golWVqzsMU/tGWFBDe8rOpnTbwLNrwCZXnKxNAehRbt\nUVGoSsxPvle9j0xU5VvyO7GTu65G+QqSTeXfvLxU8UpL/IKVX8gvBC74q2d+v2V55p+Ndr5tTckR\ntc3apOq1eUJ7OY1wxWfxNbDctP0RyAC+8eSgzgi8fWGyqXzHqeDgbk9a0oe7/DikfKDa34bGqVdt\nRccm9+37DmQ9DDZl4Xv7QlTf1jm5q8vhs9/Cvu+bHtvxEax5vuX3BjWBygZlhnLEuY/B+Nth039g\n86LWPc8RlsKi0MPCAgl7V3jmGa2hugw+us7cobKDaVZYSCmTpJTJpm1/YBywvrnrAIQQM4UQe4UQ\nB4QQC+wcny+E2COESBVC/CiESLA4dqMQYr/pdXrqiCOvh3G/N9d50iha0i0vbyc01MKwK9R7I+O8\nvAP9Fnls4uUoAAAgAElEQVSpKjs/fqR5X0z/lofP1lbCh9eoKKUdHzY9/svrsPHfrXPYGhNzlBNh\nIQRc8CzEJZmTDdsa43vrMsSDmkU2CC8VFm7ZN6azkLlR/X63vNnRIwFcLyTYiKkHd7PRUUIIb2Ah\nyucxBFWAcIjNaduBMSZn+aeoulMIIaKAJ4DxKOH0hBCinSrmtSO+gXDh89Y1nzQt68NtRD6FdVNb\nI4mwIyOiCg+ZTE8W/2axA1UkXF2Ne/eqq4ElNyqzVlgPJYgsaaiHY7uVU7g10UPGxOxMswD1meKG\ney7JsCxXmey6jfKgZpENod1U8EHGms6TxGmQm6K26cs7RcSWK4UE51u8HhBCfAC48tc4DjggpcyQ\nUtYAHwGXWJ4gpVwlpTQ1lGYT0MP08wXAD1LKQillEfADMNPFz6Q51fELcd9nUWoSCqHx1tuOjIgy\nooosiRmoTFPurJalhKW3qWS52S8qZ2zBQWvtK38/1JmKFLYm8a/osOqxbmhmzogdoDSAymLr/Ts/\nVf6a1lB2TFU6juoNJ4+3vvxL1mY4tsd6X/ERiOipfDINtbD/B/fuWVmsulZ6KnExxyQsig6rhUAH\n44pmEWrx8kf5Li5xeoWiO3DE4n22aZ8jbsXsC3HpWiHEbUKILUKILSdOeDAWW9O++Icpf0N9nevX\nlOWoDHPfAPU+tIM1i4YG9U9ua86J6a+27kzoJ/aqfiZTH4IxtyjzDxKO7TKfk7fT/HNrHOiGNuSK\n0zpmgOl5FtpF/n747NbW+zLKctV3aPz+ijNbd7+lt8EPj1nvKzkC4T1UT5iQru6ZoqrLYfFc+Ooe\nyP61dWNzRG4K9D4bEJ4z97mBKz6Lpyxez0gpF0spq1y4t72/Nru6lBDiemAM8II710opX5NSjpFS\njomNjXVhSJpTAqP+lDvaRWmu2QQFSuD4BHacz6I8T7W/Ncq7GDROsG5M6Lk71NaoMBxnSnGyFBB5\nO1Tmu29w6xzo9rQhR9gTFpkb1NYoZ9NSyk2ahTGW1piiaiuhKNP6Hg31Krw6vKcyqQ28UGkWrpSQ\nN3xHhpDwhJms/IQaX//zoNeETuFTccUM9YOpkKDxPlII8Z0L984Gelq87wHk2J4khJgBPALMkVJW\nu3Ot5jSlJWXKy3LMpidQK+PQOOuomvak6LDa2k68/iEQ1t29CT0vVZmGjMk5rJvKsDaECEBuKnQd\n0joHupT2tSFHRCaCt5/18zI3qm32ltYVIizLU9+fMZbWOLkLDgBS5VUYJqPyY6r6c7jJ8j14tsoW\nz1jt/F51NbDkBuU7mvNvQJi/67bE+G7jR6gk3mO7POe7cRFXzFCxUspGo6TJh+BKQPKvQH8hRG8h\nhB9wDbDM8gQhxEhgEUpQWDSm5jvgfJNgigTON+3TnAn4WfThdpXSHGvNAjpWWDiLKooZ4N6EnrtD\nRQV5m9KihFCmKMPJLaXSMuKSlQO9pZpF+XFl/nNVs/Dyhuh+1ppF1s9mM6KlMHOHumqoLFTfX2Ck\narDVmonS+F031Kq/E1D+ClCRUACJU8E/XNXxcsaWN2D/9zD7nzBqnhL8HhEWpoTS+GRz6HUHm6Jc\nERb1QohexhtTeGuzrnkpZR1wN2qSTwOWSCl3CyGeFkIYnfZeAEKAT4QQKUKIZaZrC4H/QwmcX4Gn\nTfs0ZwJGRrurZqi6GhUBZCssQroqc1BHUHRIhWWG92x6zJjQjQZYzmgUBEnW++OSVTZ1fa0yV1QW\nqnNiBkBpdsscwkVOBJwjYgaYhUXJUbV6H3urep/ZQlOUYTo0ItoiE1unWVgKM8P3YeRYGJqFj5/q\neb93hXNfWcYaJSDH3GweW2v9KfbISYGoPiphNzJRfbfNCTIP44qweARYL4R4TwjxHrAWeNiVm0sp\nV0gpB0gp+0opnzHte1xKaQiFGVLKrlLKEabXHItr35RS9jO93nL/o2lOWRobILmYmGcIBEszlPG+\no6KhCg+picjHr+mxmP7K5FHmgmW15IiqHxZvU4otfjjU16g+Gbmp5n32/AjujBma+lmcETNAraxr\nqyDLZIIacqma6Iz37mJ8Z8b3GdW7dZpF/j7lvwKzFmArLECZeyoLHftbGhrUZ+o10bwvMtFDmkWq\nMkE1ju1iOLK5Q6P7XHFwfwuMAj4GlgCjpZTaJKTxHI0+CxdXx6U2ORYGoV1VX4yO6LrnzFEcM1Bt\nXTFFGYIgbrj1fksnd14qIFRr21jTve2ZoqSEbxbAP5PMr4+uM8fwFx1S94no1fRaR8QOVBnfhQeV\nJuEXqlbBvSapidUV7ckWI4LNiGiL7K0m9+ai46pK4e3ZcHSb9f4T+yBhktL0GoVFtjJvWXZ+7DdD\n+YYcreBPpCvBnTDJvC8y0VQdtw17q1cUqjJA3SyExeDZgIS0ZQ4v8zSuOLgvA2qllMullF8BdUKI\nSz0/NM0Zi7sObmOFbk+zgI6JiCo85Nic08WUm2oZ+uqIvFQ1ydn2OI/uq7LDc1OVwIjup2o2RfYG\n4W0/2mrjQtj8XzXBJ05R2/TlcGCleczhPVSRQFex1GSyNkLPccqXkTBRFelriYbTaIYy5XpE9VbO\n6NJs59ft+xYOr1N5HgYN9crB3XWo+myGsDByLCzxD4G+56pKwfaS4AyNw0qzMCXUFme59NFcwiiA\naalZdBmiNMdN/+2whlSumKGekFI2pjaanN1PeG5ImjMePzdDZx1pFo1Z3O3st6gqUeYMR+ac4Gjl\nGLUMfXVEoyAIst7v5a0mwLxUk8nCpGn4+CkTkO0knbEGfnhcJaBd9wlc9l+45gOVDW40iyo67J4J\nCtTYEKo74fE95onU2LYkhLYsTwm84Bj13tXwWSO81PKZxZlQX62EmqXJqCTbvj9p0GwllOxVLM7c\nqBYglr8j4+e2NEUZmduWpsfGPjgHYc+XbfcsN3BFWNg7x5VqtRpNy3CmWRzbDVtsXFhlOSrHINCm\nIoyRhdzeiXmOwmYtiUs2m5ickZtqNjnZu8fRbcpkYXlO7EBlejEoOQqf3qwm9kv/a0648/FTxSyz\nNioTUpETbcgRfkFqhb7jI/U+wSQkovooYZ3ZAr9FWZ6qAGv0y3AlfLa2UmlI3n7qd2aYHo3fQ+xA\nk7CwcHBb+isMBs5Sgso28khKs7/CMmHRI8Jih7qv7d/zoIvNfXA6oPyHK8JiixDiRSFEXyFEHyHE\nP4Gtnh6Y5gzG21fZjm2FxbE98PZFsPw+63ISpbkQFt8067ijigm6UowvPhkK9kNNheNzKgrVKtfW\nuW0Ql2Qu8WF5Tkx/tQI18hzWPq9+l9cstrbRA4ycB0Ex8OP/qYgydzULUD6YykJVHr67qU+HEGpi\nbYmTuzzPutxIaLwSAs40i4OrVLjuuNtUORUjYc7QsGL6q8928rj6e6kuta9ZBEVB4uSmSXDFWSrq\nzNJfARAcq8yBRW0UEdVQr3I4eoxreszLCybfB8d2mk2H7YgrwuIPQA3Kwf0JUAXc5clBaTRNigkW\nHIT3LjVPrpYmnLJcVRDOloAIpXG0u2bhQjG+uCTlGHZW88fIU3CkWVgKCMtzYgYqG3/RYZVXYJRu\nN0qNWOIXBBPuMJtuXM2xsMTwW3QfpYpjGiRMUiv44iP2r3NEWZ7ZXwFKw4hIcK5ZpC9XYaZn/VH5\neAwhlb8XgruoVbohCA+bimbb0yxAreDz91lrZ8b9LP0VoIRiW0ZEZW1UjaYGXWT/ePJVSsgZpsN2\nxJVoqJNSygWmshqjpZQPSylPtsfgNGcwfiGqOmvGGlWG4Z05agKct1QdtxQWpTlKs7BFCBVRY4Qb\nStl2K0BnGD2snfUpaYxmcmKKMj6jI2HRZagymYR2M9v3QRX4AxVttXGhWq1Ousfxc8b+1pzb4q4Z\nyvJ5thNpo9/CjnZRXaa+W+OVf8B8rCzPHAllENVb5ZUY5x9PNx+rr1P5EQNmKs0gLsmc43FinzlC\nLCJRbQ+vM713EPVlTNTpFtpF5s8qaa/L4Kbnt6WwSFuuFjj9Ztg/7u2rvsusjaqIofH7sI0A8wDN\n+h6EELHAQ8BQIMDYL6U814Pj0pzphMbBoTXqBeof9aavVERIaDfr7OWy3KaRUI33iVdmDSlhxQPw\n6//g5m/NtnVP4IqjOKKX0nycCotU5QgPjrZ/3DdAhVfamlOiTRrEkU3Kv5M017kQCIyA8b9X3QWj\n+jgftz2MqJ2+51jv7zpUFXf8+RXof756Dqi6R2/NUmY4A+EFV76tajRV5Df9PmMHqczpd+eY9835\nt8qiztygIq8GmTKde02CrW+rZM38fTDscrXfVc0ivDv0GAubXlU5I9F9Tf6K8fb7jkckqAm7tV0Z\npVQaUt9zzblG9hg1D9a+oIoYGnQfA7/7seXPdgFXHNWLUSao2cDtwI24VqJco2k5V71nHdET3c+8\n2oy3cA5XFqmCfbaRUAYhXVV8/A+PK0EBqoKrR4XFIfs2Z0uMkh3OnNzOnNsGv/nEXAbEICBMCdTN\ni1Ti3pT77V66PauI9Lwyrh3XC6Y9rExVLWnx220E3JPSVCB5ecPlr8GH18IHVymtsL4G3rtMRSNd\n8YZJKEj48Wn49Fa40FRLNMRGs5j2sHI+G47ddf+AZX9QZq8jm1XSXb/p6ljCRBUifGClyosw8lqC\nolQeSOFB5QMJdlK1aM6/4e0L4d1L4Or31d/i8GvtnxuZqJIsT+ZDSCsKmuamKLPdtCZ94qzxDYTb\nVllryc6ESxvhirCIllK+IYS4V0q5BlgjhFjj6YFpznBCYh3/48UlmSuEljrIsTAIjVOJTPn7YMyt\nSgtJ/xpm/c0zvaNte1g7Iy5Z1Rqqr2s64ddUqJX30GZSmhxpHbEDICMHBl5k33QCLFqTwY/px7h8\nVHf8fbxb5tw2cKS59D8P5r4Bn9ykhEbNSeVHuPYj8+QO8Jsl8M7FKngBmvbT8Auydi53G6lKhC/9\nvaq022+6yjMBs/lr69tqa/hqDP/CsZ1KY/NyYoXvMgiu/1yZP9+6UO2zdW4bWEZEtUZYpC1XGtaA\nWc2fG97DsWbkIVxxcBulI3OFEBeZiv+17yg1GkviklXEy/E9TTvk2WJMOsnXwIV/V3kGtnH0tVXw\nvxmw42PHz5QSVv0V3r/CediiKz2sDeKTlVZUYCfbOneHuo9tTShXMVbTZ813eEpaXim19ZL0XA/3\nKR9yCVzyH2VSzNmuzE2WggKUiWreF8rcBM03X/ILUgInLhmqS8wmKFBht1F94YCpmZHhswBzEp1t\nQp49uo1QOSmg/AjdRto/zxAWxZnqb2P1c/DWRc67IVYUwmvTrMPA05dDwmTHC4AOxhXN4i9CiHDg\nj8C/gDDAvl6r0bQHRhRQrim7GRxrFsnXqJXn2N+qleSAmeY4+u6j1Dkpi1WopfCC4Q40gjV/Uy9Q\njuMug+yf50rYrEGcxeewXf2nfqzChxPPav4+9phwu/Lv9Bhj9/DJ6joyC1RkWerREob3jLA63tAg\n8fJqQ81rxLXKxOUboGzy9giOhhuWqR7jzZnfQJnbrv8Mdn1m9ksYJExU5ibfYKVFGBgTu72wWXv0\nGg83f62c7o4y2w1HedEhWPt3WP2sep/6sfIv2GPzIiU4c7ar+/YYq8ylY25xbVw2SCkRntCULXAl\nGmq5lLJESrlLSnmOKSKq4wqUaDQRCcrhnZdqUUfIgbAI764mTsPM0xhHb0q6qq+DDS/RmIVsr1Db\nz/9SE8AAU2dfZz0PXO1hDSrk1Nu/qZO7ulxNmEMvNzuF3SWqD4y8zuHhvcfM2sTObOu2qNuyihj2\n5HfszmnjntSDLrQrKH5MO8YNb/5CbX2D8ktNutu+I9keQVEw7ndNJ/JeJpNRTH/7SXSuCgtQGsVA\nJ6YhvyAV6vvrm7DqL2qBEpes/q7sleaoLoPNr0K/86DPOfDlXbDiQXXMUcisA6rr6nl2RRqPf+n5\ntquumKEaEUJ4Pj5Lo2mOxn4OO5XPIijGfnVXRwy6WNnNT+xTq9LiLDj3UUDC3q+tz92+GL5/VHWp\nu+YDJQQOOXHZGT2sbR209vD2UQ2LbIXF7s9VqZPRN7r+mdzEMD31jQ0mNdtaKHy3K4+KmnoWrjpg\n79IWk5pdzJ6cppWEF63NYO2+E6zfn992DzMCGCxNUGAhLNrYkh6ZoCoJDJ4DlyxU+R4FB+wX/tvy\nlnK8T1ugEiV7jIOMVUoouTGu9LxSLvn3BhatzaBeShoaPJvV7ZawwH67U42m/YlPVgltJdn2cyyc\nYaze0pbB+n+qIm1T5qvVuGXF0ZqTSlAkTIHLXlOr3T7T4NA6+xVQ62thzzKVxezMeWqJUfbD0g+y\n9W1lu+853r3P5QZpuaWE+Pswa1g8+4+XU1ljXgGvP6Am7W925XHwRNtU7JVScufibdz+/larSS2n\nuJJfDqls/C9SjrbJswAl1IddoUJfLek+GnpPhd4tNO85YujlSqO44g21CBh8sQphXvcP6++2tkrl\nvvQ+W5kI/YLhuiVKyDiIWrPH+5symfOvDeSXV/PmTWP462VJbWs2tIO7wuLr5k/RaNqBuGRV3uHI\nZmubtCuEd1eTxvqX4ESa+if18lJO0kNrodJkltn6jipjMf1xs+bSZ5oqe55jR8ne+amq0+QsAc6W\n+GS1yjT6K+TthKNbYdSNLkVr7Tpa0qIJPT2vlEFxoST3CKe+QbInV2kXhSdr2J1Tyk2TEvHz9uLV\n1Qcbr6mqref/lu9ha6b7fcj2Hy8nu6iSrMIK1uwzR95/tUNFs53VP4bvdudRXt1MGXJXEQLmvqlM\nX5YERcGNX7Uu8sseE26HyxeZ/068vGHKfer7tCzNseMDlfdjGXgQEA5Xv6cCAVxg/7Eynli2m/F9\novjuvqmcO8gFLbYNcEtYSCkf9dRANBq3MKKEasod+yucMWi2mvQjEtSqENRqsKFWJX/VVStfRcIU\n5eQ06D0VECoJy5KGBlj/InQdBgMucONzmPpUbH1b5X+s+ZvyYwy/xqXL7/s4hfs/TnH9eahVfnpu\nGYPiQxsd24Yp6ueDSqu4ZEQ3rhnbk6Xbj5JTXElNXQN3Lt7GG+sPcdfi7ZRWuddf+6d01TU5IsiX\ndzcebtz/ZUoOI3pGcO/0/lTVNvD97g7qbOgJkq5SVX1X/VV9t7uXqgVK99FKs2ghf12RRpCfNy9f\nM5LoEDfKybcSdzULAIQQLtRW1mg8SOxANamC47BZZwy5BLx8YOqDZud39zHKUZn2laqiWpbTNPQ0\nKEppA7ZO7vTlKpdjyv3u5W90HaJKm6z7h8pFSPtKmU+Copq9tL5BkllwktTsEg7nu16B52hxJWXV\ndQyOD6NrWABdQv3ZaRIWGw7kExrgQ1L3cH43VWVzL1pzkPs/TuGn9OPcMrk3x8uqeHZFmuufEfgp\n7ThD4sO4YWIiq/edILPgJPuPlbEnt5RLRnRjdEIkPSID+SLFhe6Bpwo+fkq7yNmmvttPblLhtVMf\nanGOz7r9J1i19wR/OLcfUcFu+OnaAIehs0KIyx0dApoJgtZoPIy3rwo3zU1pmWYR3Rf+uM86pt3L\nS/kzdnyonM7xI+yHefaZBhv/o3wafsHKJr3uH8rnMfQyt4ZRJQK4XL7CvWdHccFQo9mPayU3coor\nqa1X9vCvd+Zy1zn9XLrOcG4PilP1oJJ7hLPDFBG1/kA+E/tE4+PtRY/IIOaM6MY7G1Wm8KMXDea3\nZ/XB11uwaG0Gs5O7MblfjP2HWFBSUcvWrCLuOLsvvxnXi4WrDvD+pkz8fbzxEnBRcjxCCC4d0Z3/\nrD7A8bIquoQGNHvfU4Kxv1URTw0mTcwnoGX1t1CLg2e+TqNnVCA3TkpsuzG6iDPN4mNgDnCxzWs2\nFjWiNJoOw8i3cNfBbWAv+WnwbOULKTpsqmBqZwXY+2z1z394vUq8OrBSCa3J97ke8mliW2YRe8oC\n+OhwsBJ+XQa73Kkuq1DlSQT7eTfa/l0hLVdFJA2MU+XKk7pHkJF/kt05JRwprGRKf7MAuHNaP0ID\nfHjg/AH89iwlxO4/bwC9Y4JZ8HkqJ13wMazZf4L6Bsk5g2KJCw9g5tA4lmzJZun2o0zuF9MoGC4d\n2Y0GCV/taOcqwZ5ECIjpZ/5uWygoAD7deoT0vDIWzBysMu7bGWdJeanA36WUTXo/CiEclETUaNqR\neJO9P6wNwyATz1IOx5Cu1lnBlvSaqExgH1xl3hca77KfwZJNGQUAbD5USE1dA34+rluGjaS66yck\nsGhtBvuPldG/a6jdc2vrG/D1VvdOzyujV1QQIf7q3z+5ZzhSwmtrMwCstIV+XULY/th5+HibxxXg\n683zc5O5atFG/vH9Ph6/eIjTca5KP05kkC8jeqpmPvMmJvD1zlxKKmu5b4a5bHq/LqEM6x7GF9uP\ncusU60n1aHElB4+XM3VAK8ppnGLsOlrCA5/soOBkDfUNkrKqWkYnRHJhUscYdpz9Zd4HNA2KVrin\na2s0nmD4b1Soom0sfWvw9lX5FFe+DV5eLFpzkDve30puSaX5HL8guOodOPcx8+vq993rXW1iY0YB\nvt6Cipr6RlOQq2QWnsTP24tbpvTGS8BXqfZX5P9dfZBR//cD+0yJeGmmSCiDpO6qeOBXO3KIDw+g\nT0yw1fWWgsJgbGIUvxnXi3c2HmZvnuNyIfUNktV7j3P2gFi8TaGd43tHMaBrCH4+XlwwzHriu3RE\nd3YeLSGzwNoH84/v93LTW7+QnudoSmrK7pwSSirdc8R3FrZmFnHt65soq6pjxuCuXJQUz40TE3np\n6hEez9R2hENhIaVcJ6W024VcSrnFc0PSaJxTV9+gfvALUuW32/qfJ3EKdB1KXX0Dr645yDe78pj5\n0jq+tpyMB86CqQ/A1AfIHX4Xx8OHuf2Yypp6Uo4UM3d0T7wEbielZeZX0DMqkK5hAUzoE83yHTlI\nm7pVX6fm8rdv0ymrquP5b/dSWVPP4fyTDI4399qICfGne0QgDVJpFa5ORg+cP5DQAB+eWLbL6rll\nVbWNuRQpR4opqqjlnEHmCq9CCF6YO5xXrhlBWICv1T0NrWZbVpHV/s0ZhTRI+MvytCaf0R4F5dVc\ntvBnnvsmvdlz3WFrZiH3fbSd6jo7mdluUlBebfezbMooYN4bm4kO9uOT2yfy7OVJ/N+lw3h09hB6\nRgXZuVP74FBYCCEeFUI4DMkQQpwrhHCgp2s0bU99g+TBT3Yw+W8/UVBe7fHn/XK4kKKKWhbMGkRi\nTDB3fbCNR7+wDgSsq2/gqkUbufhf690e09bMImrrJRcM7UpS9/DGsFVXySysICFaaQEXD+9m8juY\nV97bs4qYvySF0QmR3HNuP1amHePDX7JokDA43tpcZWgXU1xwWBtEBvvxwPkD2ZRRyPLUXLKLKpi/\nJIXkp75nxotr+GBzFt/uysVLwNk25qPhPSOYOaypr6l/lxACfL3YccScVX60uJKjxZUMjg9j/YH8\nxjBcZ3yRkkNNfQPf7MpVZUTaiH/+sJ8vUnJY8qt197+XV+7n4c9d6KluIj2vlAnP/sizNsLseFkV\nt7+/lW4RgSz5/US6RQQ6uEP748wMtRP4SgjxoxDiBSHEQ0KIx4UQ75lCZy8GNju7uRBiphBirxDi\ngBCiSZF2IcRUIcQ2IUSdEGKuzbF6IUSK6aVrUZ3hNDRIHv48lU+2ZnOstJqXVtqp1NrGfL/7GP4+\nXtwwMYFPb5/IvAkJvL8pi11HzRPZd7uPcaSwkmOl1fzxkx1W2cnZRRW8uuYgz3+bzuNf7uKpr3Zb\nZUpvzMjH20swJjGKSf1i2J5V7JLDGFSuRFbBSXqZVpozh8bh4yV4c8Mhvt2Vx6dbs/ndu1vpEubP\na/NGc/u0vsSG+vPct2pyMiKhDMYkRuLrLZjUz72Kp9eO68XQbmE8snQn5/59DctTc7lufC+C/L35\n89KdvL7uEKMTIokIci3M08fbi2Hdwkm1MMn9asrw/tsVSfSJDeaZr9OoqXMsAKSUfLLlCMF+3hRX\n1DZmpLeWjBPlrD+Qj5+3F//66QBVteq7zC6q4N+r9vPhL0dcNpP9/bt91NZLXl+XwcaDBY3jfmTp\nLipr6lk0bzRdwjpXHJEzM9SXUsrJqIZHuwFvlA/jfWCclPJ+KaXDJkhCCG9gITALGAJcK4Sw9YRl\nATcBH9i5RaWUcoTpNcfOcc0ZgpSSR7/cxZIt2dw7vT83TEzgg1+yGm3wtlTW1HPta5tYuOpAi+vl\nNDRIvt2Vx9kDYgny88HX24sHLhhIgK8XizdnNo7r9XUZJEYH8cTFQ1i99wT/W6+cxN/vzuPCl9fx\n3DfpvLY2g2U7cnhrw2He/vlw4zM2ZRSS3COcEH8fpvSLoa5BNpa+sKSipo77P06x+rz55TWcrKkn\nIVoJi8hgP6YNjOXzbUe5/f2tPPDJDuobGnjrprFEh/gT5OfDPdP7U1PXQJCfd6OQMZg3MYFv75vq\ndsiqt5fg6UuGIaWKZlr9wDT+cmkSX909hQ9+O56LkuO5bWpft+6Z3COC3TmljRrB5kOFhAb4MLRb\nOI9cOJiM/JO8v8lxe9xdR0tJzytjvslMtryNoqsWb87Cx0vw4tXDOV5W3TiGhasOIhAE+Hrx1vrD\nzd5nW1YRK9OOcee0viRGB/PAJzsorarly5QcfthzjAfOH0jfWM83M3KXZkuUSyn3Ay1Zxo0DDkgp\nMwCEEB8BlwB7LO592HSs7fREzWmFlJKnl+/hg81Z3DmtL/fN6E9RRS1fbD/KX75O491bmnak+35P\nHhszCtiYUcC2zCJevGoEYYE+lFTWkl9eQ+HJGgpPVlNTLzl/SFcCfJuGIaYeLSGvtIqHhpmd5+GB\nvlwyvDtfbM9hwazB7D9WRsqRYp6+ZCjzJiSwKaOA57/dS3puGZ9vP0pS93D+de1IEqKDEEJw01u/\n8Oqag1w3oRfeQrDjSHFj4tvohEj8fLzYcCDfyr4P8PbPh1m6/Shdwvx5eJYqZZ5VqBzAidFmZ/Q/\nrgpJqP8AACAASURBVBzBvuNlBPl5E+znQ5cwJSQMrhnbkzfXHyI2xL9JHSF/H+8WT1CjEyJJffJ8\nK1+HEIJJ/WKY5IZZy2B4z3De3NDAvmNlDO0Wzi+HChibGIW3l+DcQV2Y0i+GF3/YR1lVHVeN7UF8\nuLWp5pOtR/Dz8WLuqB7sySnl+z15VNcNa1W4aWVNPZ9uzWbmsDhmJ3fj41+P8J/VBzmrfyyfbDnC\nb8b3oq5B8unWbB6cOZAYB5nVUkpe+HYvMSF+3HVOP2YM6crc//7Mg5/sYFNGIaMTIrllSsvDaz1J\nizK4XaQ7YGnYyzbtc5UAIcQWIcQmIYTddmFCiNtM52w5cUJ3ej0deeXHA7y14TC3TO7NgxcMRAhB\nVLAf90zvz9p9J1i1t6n9+vNtR+keEciTFw9h7f4TTHj2R/o/8g0jnv6BGS+u4apFG7n9/W3c8+F2\nHvuiSWQ4AN/uysPHSzDdpu7OvIkJVNbW8/m2bF5fl0F4oC9zR/dACMHzVwyna1gAn28/yo0TE/j0\njokkxgQ3TqIPnD+Qkspa/rfuEFsyi6hrkEzso8w+Ab7ejE2MbGIyKamsZdEapa38aqF1GGGzvaLN\nGkJ4kC9jE6MY2i2cxJhgK0EB4OvtxUe3TeCVax008WkFbRmhk9zDXIIkv7yagydOMq53VONznr08\niZG9Ivjnyn1Mfu4nfvvOFrKL1O+jqraeL1NymDk0jvAgX2YPj6esqo51+5o3RaXnlVpHvVnwVWoO\nJZW1XD9BNU+af94ACk/WcN3/NuElBHdM68stkxOpqWvgg81244IA2HBALWLuOqcfwf4+jOoVyZ3T\n+vHd7mNU1dbzwtzkxqixzoYrzY9air1P7I5NoJeUMkcI0Qf4SQixU0p50PIEKeVrwGsAY8aM8Wx9\nXk278+7Gw/xz5T6uGNWDRy8abDUh3TAxkcWbs/jL8j1M6RfTmENwvLSKdftPcMe0vtw0uTfDe0bw\n6dZswgJ9iQnxJzrYj+gQP6KC/fhi+1FeX3eIC5PirVbzUkq+253HxL7RhAdZR+sM6x7OiJ4RvLY2\ng7zSKu6c1rdxUg4P8uXD303gaHElE/s2tf0P6x7OrGFxvLEug+Ol3fDxEoxOiGw8PqlvDC98t5f8\n8urGlekb6w9RUlnL2QNi2XAgn8qaegL9vMksqEAI6BHpngO0ayezg9sjMTqIsAAfUrNLiAhUv39D\nWAD0jArivVvHk1VQwUe/ZvHexkxVqnveaPJKqyiprOXKMSr3Zkq/GCKCfFmemsOMIY4L7lXV1jP3\nvxsRAv5+5XBzNr2JxZsy6d8lhPGmcYzsFcmMwV1YmXacGycmNGo3Zw+I5b1Nmfz+7D5NNJmGBskL\n36XTPSKQ34zv1bj/nun9ySqsYNrAWPp0QvOTgVPNQgjhLYRoaVe8bMCyw0gPwOU0UylljmmbAawG\n2n45pOm0fLUjhyeW7ea8IV352xVNyy/7+XjxyIWDOXjiJO9Y+AGW7cihQcJlI9VkMbJXJM9clsSf\nZg7i1im9uXRkd87qH8vQbuE8cMFABnQN4eHPd1rF4+8/Xs6h/JNNJgyDeRMSyC2pwsdLcOPERKtj\nvaKD7AoKg/nnDaCitp6Pfj3C8J4RBPub12tG2OjPJodn4cka3liXwYVJcdw0KZG6Bsn2IyqkNLPg\nJN3CAzskk9fTCCFI7hFBanYxmw8VEujrzbBu4U3O6xUdxEMzB7H0rsmEBfpy7eubeP7bvXQLD2BS\nX/W79PX2YubQOH7Yo1buvx4u5K4PtjVqIgar0o9TXl1HWIAvv39vK8+uSKO2voGKmjo2HixgR3YJ\n109IsFqwLJg1iLP6x1iVWbllSm9OlFVb+UnKqmp5e8MhZry4hh3ZJdw7o7/V9+bn48Ur147k8lGd\nu1u1U2EhpaxH+Rlawq9AfyFEbyGEH3AN4FJUkxAiUgjhb/o5BpiMha9Dc/rz8o/7GdotjH9dO9Ju\nUhjA9MFdOGdgLC+t3M+x0ioAPtt2lOE9I+jXpfkVmr+PN3+/cjgnyqv5y3Lzn9e3u/IQAs53sBK9\nKDme2FB/rhjVw+2Ilf5dQ7lshLLGGiYog6Tu4UQE+fLCd+ms2JnLq2sOUllbz/zzBjAqIRIh4NdD\nJmFRWNHo3D4dSe4Rzt68MtYfyGdkrwinme39uoTwxZ2TmdAnmqzCCq4Y3cPKlDM7uRsna+q5+a1f\nufLVjXydmsvrpmx1g+WpuUQH+/HD/KlcN74Xi9Zm0P+Rbxjy+Hdc+/omgvy8uWxUd5vnhvLereOt\n/gam9o+hX5cQ/roijUsWbmDGi2sY/9cfefKrPYQH+fLyNSO4cnTnFgqOcMUMtUEI8W9UrajGtEop\npdOueVLKOiHE3cB3qEiqN6WUu4UQTwNbpJTLhBBjgaVAJHCxEOIpKeVQYDCwyOT49gKek1JqYXGG\nUN8gySqo4ObJiXadzwZCCJ64eCjn/3Mtf12Rxu1n9yUtt5Sn5gx1+VnJPSL4/dQ+/Gf1QfYdLye/\nrJq80ipG9Yp0KAgCfL354f6pTXwCrnL/eQPYk1vKLJuyDd5egv9eN5onlu3izsXq3+vykd3p10Xl\nRAyKC+OXwwVAf7IKKjh/aPv0MegIkntEUNcgOXC8nNnJzdf+Cg/y5a2bxvLd7mOcM8g6p2NCnyhi\nQvzYdKiAmycnkldSxefbjvLQzEEE+/tQUVPHj+nHmDu6B0F+PjxzWRJnD4hlR3YxIf6+hAT4MKxb\nWJMEQnsIIXh41iBeXXOQQD8fukcEMLlvNFeM7tHoizlVceWv3dTMlqct9knAQdd1i5OkXAGssNn3\nuMXPv6LMU7bX/QwkuTA2zWlIXmkVNfUNVs5bRyTGBPP7s/vwr58OkGcyDV083L2S5ffO6M+B4+WU\nVdUxvncUceEBzd7D1bwBe/SMCuLb+6baPTaxbzTf3DuVz7Zm81VqDvPPH9B4bHzvKD7+9QhFJ2so\nOFnTmJB3OjK8p9nsZOmvcIaPtxcX2REsPt5evHXTOIRQfqMthwv5Zlcey3bkcO24XvyYdpyq2gZm\nJ5u/8/OHxnG+AzNkc0wf3JXpg08/Qe5K6Ow57TEQjcYgM79pWKgz7pzWj8+3HWXzoUJmDO7qdp1/\nfx9vXrthjNvj9BTeXoKrxvbkqrE9rfaPTYzi7Z8Ps2KXsocndGDpB08TFxZAbKg/xRU1jOwZ2fwF\nzZDUwyx8RidEMigulPc3ZXLN2J4sT80hNtSfsYmuCaUzlWZDZ4UQ4UKIF40QVSHEP4QQTb1NGk0b\nkWkqvW2bOOaIQD9vnjBVPr3aZoI9nRjbW02an27NBnBJ8zpVEUJw9oBYzuofS6Bf2zrxhRBcNyGB\n3TmlbDhQwKq9J7goKb7Thqx2FlwxQ70J7AKMeszzgLcAR82RNJpWcbjgJL7ewq26OOcPjWPDgnPp\n3olq6bQ1XUID6B0TzPYsVQrjdDZDAbwwNxkXaga2iMtGdue5FWn88ZMUauoaXPKLnOm4kpTXV0r5\nhJQyw/R6CnCtlZdG0wKyCiroGRnk9krvdBYUBmMTlXYRE+LX2I/idEUI0SRkuq0I8ffh0pHdOVZa\nTXx4AKN6td7UdbrjirCoFEJMMd4IISYD9tMcNZo2ILOg4rQ2sbQGw67uqolO4xgjG/vCpHiPCaXT\nCVeWJrcD71r4KYqAGz03JM2ZjJSSzIKTLkfAnGmM761yM053E1R7MDg+jHduGceIUzyktb1wKiyE\nEF7AQCnlcCFEGICU0vVWVRqNmxScVNVU9crZPj2jArkoKZ4LTuMci/bEts+GxjFOhYWUssGUWLdE\nCwlNe2C000yM0cLCHkIIFl43qqOHoTkDccVn8YMQ4gEhRE8hRJTx8vjINGckjdVUo7SZRaPpTLji\ns7jFtL3LYp9ER0RpPIBRTbVn1Okf2aTRnEq44rO4Xkq5oZ3GoznDOZ2rqWo0pzLNVZ1tAP7eTmPR\naMgsrNDObY2mE+KKz+J7IcQVoi1bYWk0DsgqqNDObY2mE+KKz2I+EAzUCyEqUR3wpJQyzKMj05xx\nlFXVUnCyRju3NZpOiCtVZ0PbYyCazs/R4kriwgI8VnDNiIQ6nZv6aDSnKq5UnRVCiOuFEI/9f3v3\nHR1ndSZ+/Puo2+pWcZOFbdnY2AYXjOnFCaEsNQROcEgCLFl+nMCPJdnAwmYDSXYhkOzZTVkSIKT9\nshCSQAg9LBhMSSgGVxlJLrKxZctWl0aS1Z/fH+8daUYaNTMjacbP5xwdzXvf9525VxrNo9vd8SwR\nWRX5rJmJpNrXzuofrOO5zQN3xu3pCc9qbxYsjJm4RtJn8VPgVOAL7rgZeDBiOTITUkllEx3dPeyr\nC967eOPeeo67+y/sb/jky4V9XOdNyLOlLIyZeEYSLE5W1ZuBNgBVrQeOfJswE5W2H/IBUNfaEZT+\nUWUT7V09bK1o/MSvsbe2lZzU2F9N1ZhoNJJg0Ski8XgT8RCRPKAnorkyE07ZQS9YNLR2BqVX+9oB\n2F3TMuCe0dpT22JNUMZMUCMJFj8GngbyReRe4G3gvojmykw4vTWLluCaRVVvsGj+RM/vrTbbak1Q\nxkxQIxkN9ZiIfAh8Gm/Y7OWqWhLxnJkJo6dH2X7ICwb1/ZqhwlWzeK20isrGNtsH2ZgJakSNw6pa\nCpRGOC9mgqqoP8zhzm4S4mRAsKgKQ7Do6u7hvhdLmJObylUrCz5RXo0xkTGSZihzlCtzTVDHF2RS\n3xLcZ1HjgkVNcweNhzsH3DsST6zfx67qFu68cCGJ8faWNGYisr/MGPHoW+X8/M3yI7q3vLqZP2/c\nP+h5f3/FqjlTaG7voqPLG9+gqlT72pmb5/Uz7DmC2oWvrZMfvrqdVbOncN4i29DHmIkqosFCRC4Q\nkTIR2Skid4Y4f5aIbBCRLhG5st+5a0Vkh/uybVyHoKo88mY5P3x1O4c7ukd9/0Nv7OJrf9hEW2fo\ne8sO+piZNYlZ2d5IpQbXFNV4uJOO7h5OdlugjqYpqrO7hxe3VnLdr9ZT09zBv1x0HLb8mDETV8SC\nhRtu+yBwIbAIWCMii/pdthe4Dni8371TgHuAk4FVwD0ikh2pvEa7ysY2qnzttHR081pp1ajvL97f\nhKo3dDWU7Yd8LJiWzpRUb3qNf66Fv3P7xGOmECdQPsJg8cym/Zz6vdf46mMbONjYxr9fvoRls2wf\nZGMmskjWLFYBO1W1XFU7gCeAywIvUNU9qrqFgfM2zgdeUdU6NwnwFeCCCOY1qm3a1wBAYrzwzKbB\nm5NCae/qZkeV18xUXj3ww76zu4dd1c0cOzWdrMmJAL39Fv5gMTNrEgXZk0dUs+jo6uE7z31EXnoy\nv7ruJN68YzVfPOWYUeXZGDP2IhksZgL7Ao4rXFrY7hWRG0XkAxH5oLq6+ogzGu027q0nKSGOq08q\nZF1Z9ag6mnccaqaz21vbqbx64FyJPTUtdHYrC6al9dYs/COi/COh8tKTmZObOqK5Fq+VHqKupYM7\nzl/A6oX5EVuU0BgTXpEMFqE+BUa64tyI7lXVR1R1paquzMvLG1XmYsmmfQ0smZHBlScW0NHdw8vF\nB0d877YD3jIdSfFxIWsW/pFQx05NZ8pk1wzVEtwMlZ/hgkV1C6pD/4r/+EEF+enJnDk/d8R5NMaM\nv0gGiwpgVsBxATBwydLw33tU6ezuYUtFI8sLszmhIJPZOZN5ZvP+3nPf/0spb+0YvNZVvL+JtOQE\nVs7OZleIZqTtB33ECRTlpZHlgkVDb82ijeSEONKTE5ibl0pLR3dvbSOUKl8b67ZXc8WKAhJsiKwx\nUSWSf7HrgfkiMkdEkoCrgWdHeO/LwHkiku06ts9zaaafsoM+2rt6WDYrCxHh0qUzeGdXLYea2rjj\nyS38dN0urv/V+kGHxm470Mii6RkU5aVRXtU8oGaw/VAzs3NTSUmMJykhjrTkBOoC+izyM5IREebk\nesNnQ9VO/J7esJ/uHrWJd8ZEoYgFC1XtAm7B+5AvAf6gqttE5LsicimAiJwkIhXAVcDDIrLN3VsH\n/BtewFkPfNelmX427q0HYHmhN5ro0mUz6FH4ws/f5emN+7ll9TxOmj2F236/id/8bU/Qvd09Skml\nj8UzM5ibl4qvvYvq5uCawfZDPhZM7dv/KmtyYm+fRXVzO3lpyQC9wSKwk3tnVTPN7V2AN7z3jx9W\nsKIwi6K8tDD+BIwxYyGia0Gr6ovAi/3S7g54vB6viSnUvb8EfhnJ/IVbTXM73T3K1IyUMXvNjXsb\nyE1LZmbWJADm5aezeEYG2w40cf3ps/mn846lvauHW3+3kXue3Yaqct3pcwBv8b/Dnd0snpFJfrr3\noV9e3UJ+upf/ts5u9tS2cMnSGb2vNyU1qa+Du6lvQt6MzEkkJcT1dnKXVDZx8U/eJjctiW9dvIiZ\nWZPYWdXM9644fmx+MMaYsLKG4zC6609bufV3G8f0NTfta2B5YVbQhLZvXnQct5+/gG9dtAgRISUx\nnp9es4Iz5+fyo7U7eiffFe9vAmCJq1lAcDPSjkPN9KjXue2XPTmJ+pa+moU/sMTFCXNyUtld43Vy\nf/vZbWSkJJCblswtj2/k+l+vJyUxjotPmB7ZH4gxJiIsWITRvrrWsOwYN1INrR2U17QMmNB2WlEu\nN6+eR1zAsNSE+DhuOruI+tZOXthSCXj9FckJcczLS2NG5iRSEuOChs++vbMGgJWz++ZDZk9OpL61\nk/aubhpaO8lzNRLwmqLKa1p4YWsl7+2u4xvnL+CZm0/nnksW0dWtfHb5TNJTEiPyszDGRJZtSRZG\n1b52Dg+yZEYk+Cfj+fsrhnNaUQ5Fean8v3f28LkTCyje38TCaem9I5Nm56QGzcJ+vayKRdMzgprV\nslO9mkVNs1e7CAoWeam8WnKI+14oYdH0DK4+qZD4OOH60+ewZlWhzakwJopZzSJMOrt7qG3poLWj\ne9A1lsJt494GROCEgpEFCxHh2tNms7mikU37Gth2oJHFMzN7zxflpfXWLJraOvnw43pWLwyev5I9\nOQlfexcHXA0qPyBYzM1NpatHOdDYxncuWxwUHFIS421FWWOimP31hkltc98+D7X9dpP7pEoqm0JO\ndtu0r4EFU9NHtWf1FSsKSEtO4L4XS2hq62LxjIzec0V5qeyta6W9q5u3d9TQ3aOsXpAfdH+2m8Xt\nX4k2sGbh7/e4bNkM28TImBhjwSJMqgMmo9U1hy9YFO9v5MIfvcXrZcELBKoqmysaWDrCWoVfWnIC\nV6yYyfu7vZHIS2b01Szm5qXRo7C3tpXXS6vInJQ4oD/EP4t7+8GBwWJpQRa3n7+Auy/uv16kMSba\nWbAIkypfW+/j2pbBZzGPVvF+bzmODR83BKXvbzhMQ2snxxdkhrptSF8+1Vu4Lz5OWDCtb6STv2aw\nq7qZddurOevYvAEzrbPdYoL+ZUBy0/qCRUJ8HDevnkdOQJoxJjZYB3eYBNUswtgM5f9Q9q/h5Nc3\n7HX0wWJefjpnHZtH4+FOUhLje9P9E+ue21xJta+d1QsGrrfV1wzVzJTUJOuHMOYoYcEiTCIVLPx9\nA8UHmoLSPzrQSHycsDCgZjAaP7tmBV3dwf0g6SmJ5Kcn81KxN7T2rGNDBIuAxQSP9LWNMdHH/i0M\nkypfO5mTEkmMl7B2cJcdbCYxXqj2tVPV1NfUVXygiXl5aUE1g9FITU4gc/LAOQ9Frt9iaUFmUBOT\nX1bAPYH9FcaY2GbBIkyqfe3kpyeTPTkpbB3cdS0d1DS386mF3oik4oCmqOL9jSyemTHYrUfM329x\nTr9RUH4pifGkJnkBKs/6Jow5aliwCJPq5nby0pOZkpoUtpqFvwnq8mUzEYFtrp+iyudto7p4xuj7\nK4YzL99b5O+cEP0Vfv6lyvMyLFgYc7SwPoswqfK1cWJhNiJQdwSjoepaOiipbOL0eX2bAvmDxfLC\nbObkpPbWLLa5/oslM8Jfs/jciQVkTR44ZDbQlNQk9jcctpqFMUcRq1mEgapS7fPXLJKPqIP7X/+8\nlS/+4r2gIbhlB31kpCQwNSOZRTMyekdAbXPDaRdFIFhkpCTy2eUFQQsT9ucfEWV9FsYcPSxYhEFz\nexdtnT3kpSeTcwTNUDsO+Xip+CCq8Hpp3+S77Yd8LJiWjoiwZGYm+xsOU9/SQfH+Jubkpo7bonz+\nuRb+FWeNMbHPgkUY+LcSzU9PYUpqEr62Ljq6ekZ8/4Ov72RSYjz56cm8WuIFC1Wl7KCvd3lw/0zr\njyqb2FbZGJFaxUj5h89azcKYo4cFizDwz7Hwd3ADvRsEDWd3TQvPbj7AF085hvMXT+PtHTW0dXZz\nqKmdprau3hnW/jWc/rqzhn11h4OW6RhruWleGfOtg9uYo4Z1cIdBYLBoOuztT13b3DGiHfN++vpO\nEuPj+MqZc9h2oInfvvsx75TXEu/6DPw1i+zUJGZmTeLJDysAb8Oi8XL1qkLm5aeTYXtTGHPUsJpF\nGPQ1Q/XVLEbSyf1xbQtPb9zPmlWF5KencOrcHCYnxbO25FDvSKjAXeoWz8jofa1IDJsdqdy0ZC5Y\nMm3cXt8YM/YsWIRBta+dxHghc1IiOa6JZiSLCd7/UilJCXF89ZwiwJvwdsa8XF4rqaL0oC+oWQv6\nAsSMzJSgdGOMiTQLFmFQ7WsnLy0ZEWFKqteOH1iz2F3Twn0vltDa0dWb9v7uOl4qPshNZxeRH9Bc\nde5xUznQ2MYrHx1iwdTgtZf8TU+Lj2DxQGOM+SQsWIRBla+NPPeBnzUpkTgJDhZPb9zPI2+W85Xf\nfEBbZzc9Pcq9L3zEtIwU/uHMuUHPdY7bma7xcCfzp6YFnfOvMHu8BQtjzBizYBEG/poFQFyckD05\neK5FaWUTqUnxvFNey//57Yc8+WEFmysauf38BUxKCl4IMD89haVu9nT/msXUjBT+54aTue702ZEt\nkDHG9GOjocKgprmd5YXZvcdTUoMXEyw75OPsBXmcfWwe//zUVt7YXs2SmRl8dvnMkM937sJ8Nu9r\nYP7UgUuAnzE/N8QdxhgTWRGtWYjIBSJSJiI7ReTOEOeTReT37vx7IjLbpc8WkcMissl9PRTJfH4S\nXd091LZ0kB8wQW1KalJvM1RrRxd761pZMDWDz59UyL9dvoTUpHi+ddEi4uJCL6nxxVOO4c4LFw65\nPpMxxoyliNUsRCQeeBD4DFABrBeRZ1X1o4DLbgDqVXWeiFwNPAB83p3bparLIpW/0frJ2h28vbOG\n//nKyUG7w9W2dKAaPJs5Jy2JMrdH9fZDzajSO7nuS6ccw5qTZg3YrjRQdmoSN51dFKGSGGPM6EWy\nZrEK2Kmq5araATwBXNbvmsuA37jHTwKflqFWsIsAVWX9njpqm4ce6vrC1kre213Hw2/sCkoPnJDn\nF1izKK30Fv87bnpfk9JQgcIYYyaiSH5qzQT2BRxXuLSQ16hqF9AI5Lhzc0Rko4i8ISJnRiqTH9e2\nctVD7/DE+n2DXuNr66TskI9JifH8eO1Odlb5es/5V4kNboZKpuFwJ909SulBH5OT4pmVPTlSRTDG\nmIiLZLAIVUPQEV5TCRSq6nLg68DjIjJgfQsRuVFEPhCRD6qrq48ok7NzUzmtKIfH39tLd0//7Hk2\n72tEFe67YgmTk+O548ktvdeGqlnkpCah6q0PVXbQx/yp6YP2TxhjTDSIZLCoAGYFHBcABwa7RkQS\ngEygTlXbVbUWQFU/BHYBx/Z/AVV9RFVXqurKvLzBd3YbzpdOOYb9DYdZV1YV8vyGvfWIwKePm8rd\nFy9iw94GHnmzHOgLFoH7VQcu+VF6sInjpg0c1WSMMdEkksFiPTBfROaISBJwNfBsv2ueBa51j68E\nXlNVFZE810GOiMwF5gPlkcrouYumkp+ezG/f/Tjk+Q1765mfn+Y2BprJRcdP5/svl7K25BBVvnYy\nJyWSktg3XyLHBYvSgz7qWzt7O7eNMSZaRSxYuD6IW4CXgRLgD6q6TUS+KyKXust+AeSIyE685ib/\n8NqzgC0ishmv4/smVa2LVF4T4+NYs6qQN7ZXs7e2NehcT4+ycW8DK9w8ChHhP65aypIZmdz6u42s\n31M/YF+HKW59qHd21QBYsDDGRL2IDstR1RdV9VhVLVLVe13a3ar6rHvcpqpXqeo8VV2lquUu/SlV\nXayqS1V1hao+F8l8AqxZVUicCI+9H1y7KK9pofFwZ2+wAJiUFM/Pv7yS1OQESiqbBuxF7W+G+tuu\nWgAWThu/5cSNMSYcbAynMy0zhc8cN5U/rN9HW2d3b/qGvfUALC/MGnD9o9euJDkhjpnZk4LO+XeS\n+7i2NWjZcmOMiVYWLAJ86dRjqG/t5Pktlb1pG/fWk5GSQFFe2oDrTyjI4oVbz+CfL1gYlJ4YH0dG\nijff0ZqgjDGxwIJFgNOKclg4LZ3/eLmMpjZvx7sNHzewrDB70KGv8/LTQ+5FneOaphZasDDGxAAL\nFgFEhAc+dwJVvjbufb6EprZOtlf5WFE4+jWa/E1P1l9hjIkFtupsP0tnZXHjWUU89MYusiYnokpQ\n5/ZI+YOFNUMZY2KB1SxCuO3c+RTlpfLwm+WIwLIjqFnkpiURHyfMyx/Y12GMMdHGgkUIKYnxfP/K\npYjQOxlvtNasKuTblywKmqxnjDHRypqhBnHiMdncf8XxZE4afaAAb6TUCQW2H4UxJjZYsBjC508q\nHO8sGGPMhGDNUMYYY4ZlwcIYY8ywLFgYY4wZlgULY4wxw7JgYYwxZlgWLIwxxgzLgoUxxphhWbAw\nxhgzLFHV8c5DWIhINdB/E+1coGYcshNJsVamWCsPxF6ZYq08EHtl+iTlOUZV84a7KGaCRSgi8oGq\nrhzvfIRTrJUp1soDsVemWCsPxF6ZxqI81gxljDFmWBYsjDHGDCvWg8Uj452BCIi1MsVaeSD2yhRr\n5YHYK1PEyxPTfRbGGGPCI9ZrFsYYY8IgZoOFiFwgImUislNE7hzv/IyEiPxSRKpEpDggbYqIHp++\nhQAACGxJREFUvCIiO9z3bJcuIvJjV74tIrJi/HIemojMEpHXRaRERLaJyD+69GguU4qIvC8im12Z\nvuPS54jIe65MvxeRJJee7I53uvOzxzP/gxGReBHZKCLPu+NoL88eEdkqIptE5AOXFs3vuywReVJE\nSt3f06ljXZ6YDBYiEg88CFwILALWiMii8c3ViPwauKBf2p3AWlWdD6x1x+CVbb77uhH42RjlcTS6\ngH9S1eOAU4Cb3e8hmsvUDnxKVZcCy4ALROQU4AHgv1yZ6oEb3PU3APWqOg/4L3fdRPSPQEnAcbSX\nB2C1qi4LGFIaze+7HwF/UdWFwFK839XYlkdVY+4LOBV4OeD4LuCu8c7XCPM+GygOOC4DprvH04Ey\n9/hhYE2o6ybqF/AM8JlYKRMwGdgAnIw3ISrBpfe+/4CXgVPd4wR3nYx33vuVowDvw+ZTwPOARHN5\nXN72ALn90qLyfQdkALv7/5zHujwxWbMAZgL7Ao4rXFo0mqqqlQDue75Lj6oyuuaK5cB7RHmZXJPN\nJqAKeAXYBTSoape7JDDfvWVy5xuBnLHN8bB+CNwB9LjjHKK7PAAK/K+IfCgiN7q0aH3fzQWqgV+5\npsJHRSSVMS5PrAYLCZEWa8O+oqaMIpIGPAXcpqpNQ10aIm3ClUlVu1V1Gd5/5KuA40Jd5r5P6DKJ\nyMVAlap+GJgc4tKoKE+A01V1BV6TzM0ictYQ1070MiUAK4CfqepyoIW+JqdQIlKeWA0WFcCsgOMC\n4MA45eWTOiQi0wHc9yqXHhVlFJFEvEDxmKr+ySVHdZn8VLUBWIfXH5MlIgnuVGC+e8vkzmcCdWOb\n0yGdDlwqInuAJ/Caon5I9JYHAFU94L5XAU/jBfVofd9VABWq+p47fhIveIxpeWI1WKwH5rsRHUnA\n1cCz45ynI/UscK17fC1eu78//ctu5MMpQKO/SjpRiIgAvwBKVPU/A05Fc5nyRCTLPZ4EnIvX2fg6\ncKW7rH+Z/GW9EnhNXUPyRKCqd6lqgarOxvs7eU1VryFKywMgIqkiku5/DJwHFBOl7ztVPQjsE5EF\nLunTwEeMdXnGu/Mmgp1Cfwdsx2tP/uZ452eEef4dUAl04v13cANee/BaYIf7PsVdK3gjvnYBW4GV\n453/EOU5A6/6uwXY5L7+LsrLdAKw0ZWpGLjbpc8F3gd2An8Ekl16ijve6c7PHe8yDFG2c4Dno708\nLu+b3dc2/99/lL/vlgEfuPfdn4HssS6PzeA2xhgzrFhthjLGGBNGFiyMMcYMy4KFMcaYYVmwMMYY\nMywLFsYYY4ZlwcJEFRFZLiKPDnF+hog8GcHXv8qt+vl6v/TZIvKFI3zOv43gmkcjsRimiHxbRL4x\nzDWXj+S1ReQWEbk+fLkzE4kFCxNt/gX4yWAnVfWAql452PkwuAH4qqqu7pc+GwgZLAJmQoekqqcN\n96Kq+hVV/WikmQyzy/FWbx7OL4FbI5wXM04sWJhx4f4TL3X/MReLyGMicq6I/NWtz78qxD3pwAmq\nutkdn+32K9jkFlhLd89b7M4/GnC+WkTucem3i8h6t9b/dwbJ3xrx9kMoFpEHXNrdeBMNHxKRH/S7\n5X7gTPdaXxOR60TkjyLyHN6CdmkislZENrjnvSzgtZrd93NEZJ307VvwmJsFj0tf6b9eRO4Vb0+N\nd0VkqksvcsfrReS7/ucNUbZvirfXy6vAgoD0f3D3bhaRp0RksoicBlwK/MCVrSjUdQCq2grsCfW7\nMzFgvGcm2tfR+YX3n3gXcDzePy0f4v1nKsBlwJ9D3LMaeCrg+Dm8BeMA0vAWXJtNwBLv7twxQKn7\nfh7efsXiXvd54Kx+188A9gJ57jlfAy5359YRYkYsAbOf3fF1eLPw/bNqE4AM9zgXbwa0f1Jsc8Bz\nNOKt5RMHvAOc0f918WbFX+Iefx/4V/f4edzS1MBN/uftl88T8Wb1TsZb+non8A13Lifgun8H/q97\n/GvgyoBzIa9zx9/E28Nk3N9j9hXeL6tZmPG0W1W3qmoP3rIMa9X7xNmK96Hf33S8pZr9/gr8p4jc\nCmRp35LavUTEvzzFLar6MV6wOA9vyY4NwEK8TWICnQSsU9Vq95yPAUOtWjqYV1TVv8ieAPeJyBbg\nVbwlo6eGuOd9Va1wP5NNhP45dOAFBvCCrP+aU/HKCvD4IHk6E3haVVvVWwE4cM20JSLylohsBa4B\nFg/yHENdV4UXbE2MGbIt1ZgIaw943BNw3EPo9+ZhvLWJAFDV+0XkBbz1pt4VkXOBtn73PAT8SVVf\ndccCfE9VHx4iX6GWeD4SLQGPr8GrqZyoqp3irfKaEuKewJ9JN6F/Dp0uqA51zVAGW+Pn13g1qM0i\nch1eTWe016Xg/Z5MjLGahYkmJcA8/4GIFLmayQN4i6wtDLxYRG4G0lX1/oDkl4G/F2+PDURkpojk\nE+w94GwRyRVvi941wBvD5M0HpA9xPhNv34hOEVmN1yQWbu8Cn3OPrx7kmjeBz4rIJNcHdEnAuXSg\nUrxl5a8JSO9ftsGuAzgWb4FFE2MsWJiooaqlQKb7kAO4zXVAb8b7b/alfrd8Azg+oJP7JlX9X7wm\nmndcM8qT9PuQV28557vwluneDGxQ1WcY2hagy3X6fi3E+ceAlSLyAd4HbOlIyz0KtwFfF5H38Zrs\nGvtfoKobgN/jNXE9BbwVcPpbeIHylX75ewK43Q0iKBriOvD2x3gVE3Ns1VkTVdwHsU9VB51rcbRy\no5IOq6qKyNV4nd2XDXdfGF9/OfB1Vf3SWL2mGTvWZ2Gizc+Aq8Y7ExPUicB/u+G2DcDfj/Hr5+LV\nOkwMspqFMcaYYVmfhTHGmGFZsDDGGDMsCxbGGGOGZcHCGGPMsCxYGGOMGZYFC2OMMcP6/7oZ6Rvw\njrL0AAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# first we split the training set into a training / validation set (as we are using k-folds cross validation elsewhere and don't have an explicit training and validation set.)\n", "X_train_train, X_cv, y_train_train, y_cv = model_selection.train_test_split(X_train, y_train, test_size=0.25, stratify=y_train)\n", "\n", "# set the model to calculate learning curves for\n", "#model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", "\n", "m_array = np.round(np.linspace(20, X_train_train.shape[0], 100)).astype(int)\n", "train_acc_array = []\n", "cv_acc_array = []\n", "\n", "for m in m_array:\n", " model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", " # we now fit to the training data\n", " model.fit(X_train_train.head(m), y_train_train.head(m)) # training on the first m training data examples\n", " train_accuracy = model.score(X_train_train.head(m), y_train_train.head(m))\n", " train_acc_array.append(train_accuracy)\n", " cv_accuracy = model.score(X_cv, y_cv)\n", " cv_acc_array.append(cv_accuracy)\n", "\n", "train_acc_array = np.array(train_acc_array)\n", "cv_acc_array = np.array(cv_acc_array)\n", "\n", "fig, ax = plt.subplots()\n", "ax.plot(m_array, 1-train_acc_array, label='train')\n", "ax.plot(m_array, 1-cv_acc_array, label='cross-validation')\n", "ax.legend()\n", "ax.set_xlabel('m (size of training data)')\n", "ax.set_ylabel('error (1-accuracy)')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This learning curve looks much better, the validation and training error are much closer for larger values of m. This means this much simpler neural network with much less hidden units in each layer is a much lower variance learning algorithm, which makes sense as it is less complex." ] }, { "cell_type": "code", "execution_count": 145, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "MLPClassifier(activation='relu', alpha=0.1, batch_size='auto', beta_1=0.9,\n", " beta_2=0.999, early_stopping=False, epsilon=1e-08,\n", " hidden_layer_sizes=(3, 3), learning_rate='constant',\n", " learning_rate_init=0.001, max_iter=200, momentum=0.9,\n", " nesterovs_momentum=True, power_t=0.5, random_state=1, shuffle=True,\n", " solver='lbfgs', tol=0.0001, validation_fraction=0.1, verbose=False,\n", " warm_start=False)" ] }, "execution_count": 145, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_model = neural_network.MLPClassifier(solver='lbfgs', activation='relu', alpha=alpha, hidden_layer_sizes=hidden_layer_sizes, random_state=1)\n", "\n", "nn_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": 146, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "0.8314606741573034" ] }, "execution_count": 146, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nn_model.score(X_test, y_test)" ] }, { "cell_type": "code", "execution_count": 147, "metadata": { "collapsed": true }, "outputs": [], "source": [ "prediction = nn_model.predict(data_test)\n", "\n", "with open('prediction_submission_nn.csv', 'w') as file:\n", " print('PassengerId,Survived', file=file)\n", " for i, id_ in enumerate(data_test.index):\n", " print('{},{}'.format(id_, prediction[i]), file=file)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We now with this much smaller hidden layer neural network get 77% accuracy, much better than with the higher variance neural network model." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.0" } }, "nbformat": 4, "nbformat_minor": 2 }