{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%reload_ext autoreload\n", "%autoreload 2\n", "%matplotlib inline\n", "import os\n", "os.environ[\"CUDA_DEVICE_ORDER\"]=\"PCI_BUS_ID\";\n", "os.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0\";" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Text Regression with Extra Regressors: An Example of Using Custom Data Formats and Models in *ktrain*\n", "\n", "This notebook illustrates how one can construct custom data formats and models for use in *ktrain*. In this example, we will build a model that can predict the price of a wine by **both** its textual description and the winery from which it was produced. This example is inspired by [FloydHub's regression template](https://github.com/floydhub/regression-template) for wine price prediction. However, instead of using the wine variety as the extra regressor, we will use the winery.\n", "\n", "Text classification (or text regression) with extra predictors arises across many scenarios. For instance, when making a prediction about the trustworthiness of a news story, one may want to consider both the text of the news aricle in addition to extra metadata such as the news publication and the authors. Here, such models can be built.\n", "\n", "The dataset in CSV format can be obtained from Floydhub at [this URL](https://www.floydhub.com/floydhub/datasets/wine-reviews/1/wine_data.csv). We will begin by importing some necessary modules and reading in the dataset." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Unnamed: 0countrydescriptiondesignationpointspriceprovinceregion_1region_2varietywinery
8295682956SpainSpiced apple and dried cheese aromas are simul...Mercat Brut8412.0CataloniaCavaNaNSparkling BlendEl Xamfrà
6076760767USA little too sharp and acidic, with jammy cher...NaN829.0CaliforniaCaliforniaCalifornia OtherShirazWoodbridge by Robert Mondavi
123576123576SpainStarts out rustic and leathery, with hints of ...Selección 12 Crianza8915.0LevanteJumillaNaNRed BlendBodegas Luzón
7100371003ChileRipe to the point that it's soft and flat. Big...NaN828.0Maule ValleyNaNNaNChardonnayMelania
7816878168ItalyFrom one of the best producers in the little-t...Contado Riserva8817.0Southern ItalyMoliseNaNAglianicoDi Majo Norante
\n", "
" ], "text/plain": [ " Unnamed: 0 country description \\\n", "82956 82956 Spain Spiced apple and dried cheese aromas are simul... \n", "60767 60767 US A little too sharp and acidic, with jammy cher... \n", "123576 123576 Spain Starts out rustic and leathery, with hints of ... \n", "71003 71003 Chile Ripe to the point that it's soft and flat. Big... \n", "78168 78168 Italy From one of the best producers in the little-t... \n", "\n", " designation points price province region_1 \\\n", "82956 Mercat Brut 84 12.0 Catalonia Cava \n", "60767 NaN 82 9.0 California California \n", "123576 Selección 12 Crianza 89 15.0 Levante Jumilla \n", "71003 NaN 82 8.0 Maule Valley NaN \n", "78168 Contado Riserva 88 17.0 Southern Italy Molise \n", "\n", " region_2 variety winery \n", "82956 NaN Sparkling Blend El Xamfrà \n", "60767 California Other Shiraz Woodbridge by Robert Mondavi \n", "123576 NaN Red Blend Bodegas Luzón \n", "71003 NaN Chardonnay Melania \n", "78168 NaN Aglianico Di Majo Norante " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# import some modules and read in the dataset\n", "import pandas as pd\n", "from tensorflow import keras\n", "import numpy as np\n", "import math\n", "path = 'data/wine/wine_data.csv' # ADD path/to/dataset\n", "data = pd.read_csv(path)\n", "data = data.sample(frac=1., random_state=42)\n", "data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cleaning the Data\n", "\n", "We use the exact same data-cleaning steps employed in [FloydHub's regression example](https://github.com/floydhub/regression-template) for this dataset." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train size: 95612\n", "Test size: 23904\n" ] } ], "source": [ "# Clean it from null values\n", "data = data[pd.notnull(data['country'])]\n", "data = data[pd.notnull(data['price'])]\n", "data = data.drop(data.columns[0], axis=1) \n", "variety_threshold = 500 # Anything that occurs less than this will be removed.\n", "value_counts = data['variety'].value_counts()\n", "to_remove = value_counts[value_counts <= variety_threshold].index\n", "data.replace(to_remove, np.nan, inplace=True)\n", "data = data[pd.notnull(data['variety'])]\n", "data = data[pd.notnull(data['winery'])]\n", "\n", "# Split data into train and test\n", "train_size = int(len(data) * .8)\n", "print (\"Train size: %d\" % train_size)\n", "print (\"Test size: %d\" % (len(data) - train_size))\n", "\n", "# Train features\n", "description_train = data['description'][:train_size]\n", "variety_train = data['variety'][:train_size]\n", "\n", "# Train labels\n", "labels_train = data['price'][:train_size]\n", "\n", "# Test features\n", "description_test = data['description'][train_size:]\n", "variety_test = data['variety'][train_size:]\n", "\n", "# Test labels\n", "labels_test = data['price'][train_size:]\n", "\n", "x_train = description_train.values\n", "y_train = labels_train.values\n", "x_test = description_test.values\n", "y_test = labels_test.values\n", "\n", "# winery metadata to be used later\n", "winery_train = data['winery'][:train_size]\n", "winery_test = data['winery'][train_size:]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Building a Vanilla Text Regression Model in *ktrain*\n", "\n", "We will preprocess the data and select a `linreg` model for our initial \"vanilla\" text regression model." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "using Keras version: 2.2.4-tf\n" ] } ], "source": [ "import ktrain\n", "from ktrain import text" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "task: text regression (supply class_names argument if this is supposed to be classification task)\n", "language: en\n", "Word Counts: 30807\n", "Nrows: 95612\n", "95612 train sequences\n", "train sequence lengths:\n", "\tmean : 41\n", "\t95percentile : 62\n", "\t99percentile : 74\n", "Adding 3-gram features\n", "max_features changed to 1765149 with addition of ngrams\n", "Average train sequence length with ngrams: 120\n", "train (w/ngrams) sequence lengths:\n", "\tmean : 121\n", "\t95percentile : 183\n", "\t99percentile : 219\n", "x_train shape: (95612,200)\n", "y_train shape: 95612\n", "23904 test sequences\n", "test sequence lengths:\n", "\tmean : 41\n", "\t95percentile : 62\n", "\t99percentile : 74\n", "Average test sequence length with ngrams: 111\n", "test (w/ngrams) sequence lengths:\n", "\tmean : 112\n", "\t95percentile : 171\n", "\t99percentile : 207\n", "x_test shape: (23904,200)\n", "y_test shape: 23904\n" ] } ], "source": [ "trn, val, preproc = text.texts_from_array(x_train=x_train, y_train=y_train,\n", " x_test=x_test, y_test=y_test,\n", " ngram_range=3, \n", " maxlen=200, \n", " max_features=35000)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "fasttext: a fastText-like model [http://arxiv.org/pdf/1607.01759.pdf]\n", "linreg: linear text regression using a trainable Embedding layer\n", "bigru: Bidirectional GRU with pretrained word vectors [https://arxiv.org/abs/1712.09405]\n", "standard_gru: simple 2-layer GRU with randomly initialized embeddings\n", "bert: Bidirectional Encoder Representations from Transformers (BERT) [https://arxiv.org/abs/1810.04805]\n", "distilbert: distilled, smaller, and faster BERT from Hugging Face [https://arxiv.org/abs/1910.01108]\n" ] } ], "source": [ "text.print_text_regression_models()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "maxlen is 200\n", "done.\n" ] } ], "source": [ "model = text.text_regression_model('linreg', train_data=trn, preproc=preproc)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Adding an Extra Regressor to Our Model\n", "\n", "Next, we will add an extra regressor to our model, thereby, creating a new, augmented model. We choose the winery as the extra regressor, which is a categorical variable. Instead of representing the winery as a typical one-hot-encoded vector, we will learn an embedding for the winery during training. The embedding module will then be concatenated with our `linreg` text regression model forming a new model. The new model expects two distinct inputs. The first input is an integer representing the winery. The second input is a sequence of word IDs - standard input to neural text classifiers/regressors." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "50\n" ] } ], "source": [ "extra_train_data = winery_train\n", "extra_test_data = winery_test\n", "\n", "# encode winery as integers\n", "from sklearn.preprocessing import LabelEncoder\n", "encoder = LabelEncoder()\n", "encoder.fit(data['winery'])\n", "extra_train = encoder.transform(extra_train_data)\n", "extra_test = encoder.transform(extra_test_data)\n", "no_of_unique_cat = np.max(extra_train) + 1\n", "embedding_size = min(np.ceil((no_of_unique_cat)/2), 50 )\n", "embedding_size = int(embedding_size)\n", "vocab = no_of_unique_cat+1\n", "print(embedding_size)\n", "extra_train = np.expand_dims(extra_train, -1)\n", "extra_test = np.expand_dims(extra_test, -1)\n", "\n", "# winery module\n", "extra_input = keras.layers.Input(shape=(1,))\n", "extra_output = keras.layers.Embedding(vocab, embedding_size, input_length=1)(extra_input)\n", "extra_output = keras.layers.Flatten()(extra_output)\n", "extra_model = keras.Model(inputs=extra_input, outputs=extra_output)\n", "extra_model.compile(loss='mse', optimizer='adam', metrics=['mae'])\n", "\n", "# Combine winery module with linreg model\n", "merged_out = keras.layers.concatenate([extra_model.output, model.output])\n", "merged_out = keras.layers.Dropout(0.25)(merged_out)\n", "merged_out = keras.layers.Dense(1000, activation='relu')(merged_out)\n", "merged_out = keras.layers.Dropout(0.25)(merged_out)\n", "merged_out = keras.layers.Dense(500, activation='relu')(merged_out)\n", "merged_out = keras.layers.Dropout(0.5)(merged_out)\n", "merged_out = keras.layers.Dense(1)(merged_out)\n", "combined_model = keras.Model([extra_model.input] + [model.input], merged_out)\n", "combined_model.compile(loss='mae',\n", " optimizer='adam',\n", " metrics=['mae'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Wrapping our Data in an Instance of `ktrain.Dataset`\n", "To use this custom data format of two inputs in *ktrain*, we will wrap it in a `ktrain.Dataset` instance. There are two ways to do this. \n", "\n", "The first is to represent our datasets as `tf.data.Dataset` instances and then wrap each in a `ktrain.TFDataset` instance, which is a wrapper to a `tf.data.Dataset`. Use of `tf.data.Dataset` instances can potentially [yield certain performance improvements](https://www.tensorflow.org/guide/data_performance). See [this example notebook](https://github.com/amaiya/ktrain/blob/master/examples/vision/mnist-tf_workflow.ipynb) for a demonstration of using the `ktrain.TFDataset` class. For this example, one can make us of `ktrain.TFDataset` instances as follows:\n", "\n", "```python\n", "import tensorflow as tf\n", "from ktrain.data import TFDataset\n", "BATCH_SIZE = 256\n", "\n", "trn_combined = [extra_train] + [trn[0]] + [trn[1]]\n", "val_combined = [extra_test] + [val[0]] + [val[1]]\n", "\n", "def features_to_tfdataset(examples):\n", " \n", " def gen():\n", " for idx, ex0 in enumerate(examples[0]):\n", " ex1 = examples[1][idx]\n", " label = examples[2][idx]\n", " x = (ex0, ex1)\n", " y = label\n", " yield ( (x, y) )\n", "\n", " tfdataset= tf.data.Dataset.from_generator(gen,\n", " ((tf.int32, tf.int32), tf.int64),\n", " ((tf.TensorShape([None]), tf.TensorShape([None])), tf.TensorShape([])) )\n", " return tfdataset\n", "train_tfdataset= features_to_tfdataset(trn_combined)\n", "val_tfdataset= features_to_tfdataset(val_combined)\n", "train_tfdataset = train_tfdataset.shuffle(trn_combined[0].shape[0]).batch(BATCH_SIZE).repeat(-1)\n", "val_tfdataset = val_tfdataset.batch(BATCH_SIZE)\n", "\n", "train_data = ktrain.TFDataset(train_tfdataset, n=trn_combined[0].shape[0], y=trn_combined[2])\n", "val_data = ktrain.TFDataset(val_tfdataset, n=val_combined[0].shape[0], y=val_combined[2])\n", "learner = ktrain.get_learner(combined_model, train_data=train_data, val_data=val_data)\n", "```\n", "\n", "\n", "\n", "The second approach is to wrap our datasets in a subclass of `ktrain.SequenceDataset`. We must be sure to override and implment the required methods (e.g., `def nsamples` and `def get_y`). The `ktrain.SequenceDataset` class is simply a subclass of `tf.keras.utils.Sequence`. See the TensorFlow documentation on the [Sequence class](https://www.tensorflow.org/api_docs/python/tf/keras/utils/Sequence) for more information on how Sequence wrappers work. \n", "\n", "We employ the second approach in this tutorial. Note that, in the implementation below, we have made `MyCustomDataset` more general such that it can wrap lists containing an arbitrary number of inputs instead of just the two needed in our example. " ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "class MyCustomDataset(ktrain.SequenceDataset):\n", " def __init__(self, x, y, batch_size=32, shuffle=True):\n", " # error checks\n", " err = False\n", " if type(x) == np.ndarray and len(x.shape) != 2: err = True\n", " elif type(x) == list:\n", " for d in x:\n", " if type(d) != np.ndarray or len(d.shape) != 2:\n", " err = True\n", " break\n", " else: err = True\n", " if err:\n", " raise ValueError('x must be a 2d numpy array or a list of 2d numpy arrays')\n", " if type(y) != np.ndarray:\n", " raise ValueError('y must be a numpy array')\n", " if type(x) == np.ndarray:\n", " x = [x]\n", "\n", " # set variables\n", " super().__init__(batch_size=batch_size)\n", " self.x, self.y = x, y\n", " self.indices = np.arange(self.x[0].shape[0])\n", " self.n_inputs = len(x)\n", " self.shuffle = shuffle\n", "\n", " # required for instances of tf.keras.utils.Sequence\n", " def __len__(self):\n", " return math.ceil(self.x[0].shape[0] / self.batch_size)\n", "\n", " # required for instances of tf.keras.utils.Sequence\n", " def __getitem__(self, idx):\n", " inds = self.indices[idx * self.batch_size:(idx + 1) * self.batch_size]\n", " batch_x = []\n", " for i in range(self.n_inputs):\n", " batch_x.append(self.x[i][inds])\n", " batch_y = self.y[inds]\n", " return tuple(batch_x), batch_y\n", "\n", " # required for instances of ktrain.Dataset\n", " def nsamples(self):\n", " return self.x[0].shape[0]\n", "\n", " #required for instances of ktrain.Dataset\n", " def get_y(self):\n", " return self.y\n", "\n", " def on_epoch_end(self):\n", " if self.shuffle: np.random.shuffle(self.indices)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that, you can also add a `to_tfdataset` method to your `ktrain.SequenceDataset` subclass. The `to_tfdataset` method is responsible for converting your dataset to a `tf.Dataset` and, if it exists, will be called by *ktrain* just prior to training. We have not done this here.\n", "\n", "\n", "\n", "## Using the Custom Model and Data Format\n", "\n", "Once we wrap our data in a `ktrain.SequenceDataset` instance, we can wrap the model and datasets in a `Learner` object and use *ktrain* normally." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "train_data = MyCustomDataset([extra_train] + [trn[0]], trn[1], shuffle=True)\n", "val_data = MyCustomDataset([extra_test] + [val[0]], val[1], shuffle=False)\n", "learner = ktrain.get_learner(combined_model, train_data=train_data, val_data=val_data, batch_size=256)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Estimate Learning Rate\n", "\n", "We'll choose a learning rate where the loss is falling. As shown in the plot, *1e-3* seems to be a good choice in this case." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "simulating training for different learning rates... this may take a few moments...\n", "Train for 373 steps\n", "Epoch 1/1024\n", "373/373 [==============================] - 9s 24ms/step - loss: 34.1117 - mae: 34.1153\n", "Epoch 2/1024\n", "373/373 [==============================] - 8s 20ms/step - loss: 28.8677 - mae: 28.8826\n", "Epoch 3/1024\n", "373/373 [==============================] - 8s 20ms/step - loss: 13.2890 - mae: 13.2908\n", "Epoch 4/1024\n", "373/373 [==============================] - 8s 21ms/step - loss: 20.4389 - mae: 20.4431\n", "Epoch 5/1024\n", "359/373 [===========================>..] - ETA: 0s - loss: 17.9780 - mae: 17.9780\n", "\n", "done.\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEKCAYAAAAVaT4rAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU1f3/8ddnsu97QkgIAcIiAoIgAloFxd3WWq3WLj+tWrt826+11mqrVvutrX5rrVbbWq32q7WtKy6tqIiIxQ0hIBB2ImuArEDIQraZ8/vj3oQEEsgyN3eWz/PxyIOZO3dmPrkk75w599xzxBiDUkqp8OFxuwCllFKDS4NfKaXCjAa/UkqFGQ1+pZQKMxr8SikVZjT4lVIqzES6XUBvZGZmmsLCQrfLUEqpoLJixYpqY0zWkduDIvgLCwspLi52uwyllAoqIrKju+3a1aOUUmHGseAXkVgRWSYiq0VknYj8wt7+lIhsE5FV9tdkp2pQSil1NCe7epqBs4wx9SISBXwgIm/aj91ijHnJwfdWSinVA8eC31iTANXbd6PsL50YSCmlXOZoH7+IRIjIKqASWGiM+cR+6FciskZEHhSRGCdrUEop1ZWjwW+M8RpjJgP5wHQRmQD8FBgHnAKkA7d291wRuUFEikWkuKqqyskylVIqrAzKqB5jzAFgMXC+MWavsTQD/wdM7+E5jxtjphljpmVlHTUMVfXDmrID1De3uV2GUsplTo7qyRKRVPt2HHAOsFFEcu1tAnwRWOtUDZ21eX04vfZAc5uXP//ns4AM17fXlfOFP3zIj19Y7XYpSimXOdnizwUWi8gaYDlWH//rwD9EpAQoATKBexysAYCa+maKbn+Tyx79iNrG1l49x+vr2x+JpVtruPzRj7nvzY3M/PWi/pTpiH0NLRTeNp8bnlkBwFvryrnp+VUuV6WUcpNjwW+MWWOMmWKMmWSMmWCM+R97+1nGmIn2tq8bY+qP91oD9VlVAwArdx7gisc+Pu7+jyzawujb3+BAY8tx9/336j389OU1fOXxpZTsrgWgrrmNVz4tc/wTRk+aWr0A1De3cfIvF3ZsnzwsFYBXPt3Nog0VrtSmlHJfUEzZMFCdA3xTRR3V9c1kJlqDiWrqmznY1MaIzATACssHFm4G4K8fbuemuaM50NhKXHQEsVERHa9XfrCJN9bs5eF3S7u8V3pCNPsaWrjp+dX8av4Grp5ZSG5qHJdPzXf8+zzU4uWSP37A5oqj/5Z+euc5pCVE89yyndz2cgnXPV3MwpvOYHROUsc+zW1eWr2GxJiw+LFQKmyFxW/4waaufe4L11dw1fQC9jW0MPWedwDYdM/5xERGdGkJP7xoCw8v2gLAsPQ4HrpyCpc9+lGP75ObEsvL35vFY//ZylMfbae6vqXjj8gnW2v4xSUnEunxsGt/I6OyEnt8HWMMNz63iuY2L49+bSoejxz1OIB1msS6v2vfIc64f3G3r/f+T+aQlhANwFemF3DbyyUAnPPgEgCKshPJSoyh4mATDS1tPHn1KZw4NLnj9ZVSoUWCYbH1adOmmYFM0vbXD7bxP6+v567Pj+fhRVuIj45k8Y9nM+aONzv2Ob0ok+/OHsXXnrAuNbjz4vH88vX1x33tiyfl8shVUwC6BOVnVfWc/cB/enzeP791KrNGZdLm9bG3tolfv7GBVq/hz18/mXkry7h1nhXOz90wgxkjM6iqa2ZLZR13vbaOLZX15KfF8c6PzqSqrpnP/aZr4P/q0gkcaGzl/gWb+ONXT+aiSbldHv9kaw1XPr70mN/Xry6dwNdOHX7U9pY2Hyt37mfGyIxjHxillOtEZIUxZtpR28Mh+C/904d8uvMApb+6gEfeLeX3i7YwKT+FNWW1xEVFcMjuE2+XlRTD8tvnsrmijnMfXMIdF53A7xZuprHF2m/zPRfQ3Oalqq6ZvLQ4YiIjun3f//twG/sbWnhvcxVrymqPenzRzWce849DuykFqXy680C325tbfazfe7BjW3uXTm94fYZnl+3kheJdR9U3PCOed2+ezX8/+ynzS/Yy77uz8PoMT3+8nflr9pIQHUFDi5ffXD6JM8dkkZMc26v3VEoNnrAN/uLt+7j8z9YJ3e33XcSybfs6TvB6BNb+4jz+78Pt3L9gU8dz3rzxc5yQmwxY3SjtLflWr48IkaO6Xo7H5zNU1zcTFeEhPiaC2+aV8Mqnu7vskxofxYFOI47OP3EI+xpaWLZ931Gvd88XJ/DBlmreWlcOwIS8ZJ6/YSb7GloYlh7fp9ratXp9VNc3ExcVwfySvdz+ylrOHpfNoo2VvX6NZT87m2z9A6BUwOgp+EO6j98Yw7yVXQN2Un5Kx+3bLxpPfHQk/zWniDNGZ/H5P3zAHRed0BH60LX7Jiqif4OgPB7pEogPfPmkLsG//b6LANhSUcfD75bS3Orlt1ecBMA76yso2V1LYkwkC9aVM6Ugla9OL+CCCUPYVFFHY0sbv/3ySSTERJIwgJOyUREeclPiALhoYi63v7K2I/RHZSWQlRTD0q3WH6FbzhtLcmwkyXFR3Pjc4aGh03+9iMU/nt1xolwpFZhCusX/s1dK+OcnOwG49fxxfHf2KADeKNnLgws389J3Z5ESF9Wxv9dniOhja76/ymubeGbpdn5w1uiO0UKBpKnVy+T/eZumVh+rf34uKfFR+HyGNbtrOSk/pcunoOXb9/H3pTt4o6ScxJhISu4+V08Mq375+LMa5pfs4ZeXTNCfIT8Iy66ee9/YwGNLtjIyM4F3fzzb/4WFuCNHDx3Pqb9+h4qDzTx05WS+OCXPydJUiJrx60WUH2ziw9vOIi81zu1ygl5PwR/SK3A1tFjDOLdWN7hcSXASkT61uuZ9dxYAzyzdQUMATluhAl9WknV9zQ79nXVUSAd/U6vP7RLCSn5aPBPzUlixYz8n3rWAwtvmU3mwye2yVBAZmmqdCyvbf8jlSkJbSAf/fV+a6HYJYec7Z47qcv+Pi0t72FMpS3ltE4W3zeeF5bsYYg+C2LW/0eWqQltIB39khIf3fzKH938yx+1SwsaUgtSO2+eMz+HlT3cH5GylKnBsr7G6dX4yb03HUGlt8TsrpIdzAv0e1676Z2hqHE/8v2mMH5rM7gOHWLi+gheLd/HN00a4XZoKUJ3nhmofa7Jrn7b4nRTSLX7ljrnjcxiaGsfUgjRGZCbwRslet0tSAczXaWThIfvqeG3xO0uDXznG4xEunZLH8u372VmjLTjVvc5LXxTvsC4SrKhrornN28Mz1EBp8CtHnXtiDgAPLdrsciUqUHVe9Kh97QxjtNXvJA1+5ahxQ6zpL15eubvjY7xSnfV0Eem2Kh3L7xQNfuW4i+1poT8srXa5EhWIelrltKq+eXALCSMa/Mpxv7tiMvHREXygwa+60d7Vk5kY3eXfGg1+x2jwK8dFR3oYNySJDZ3WDVCqXXtXT/vcPDGRESTFRlJdf/w1r1X/aPCrQTF2SDIby+tcW4BeBa72rp6hdvDXNbWSmRhDtbb4HaPBrwbFlIJUag+1srKblcRUePPajYH24D/Y1EZmYrQGv4M0+NWgOHtcNgDLth29opgKb74jgh8gIyGGGu3qcYwGvxoUGYkxjMpKYNm2GrdLUQHmcB//4VXqMpO0xe8kDX41aGaOymDZtn20enW6bHVY+49DTqflSTMSYtjf2Eqb/qw4QoNfDZrTi7JoaPHy3qYqt0tRAaS+uRWwRn+1y7QXZNnXoN09TtDgV4Nm9tgsAN5eV+5yJSqQ3PT8agA8nVZ7y0ywxvLrkE5naPCrQRMbFcGQ5FheXFGGr6fLNVXY8ojw12um8eCVJ3W0+LWf3xka/GpQXTW9AICN5XUuV6ICTXObl7PG5XDplHwy7BZ/TYMGvxM0+NWgunDiEAA2VehVvMqSFh8FwIShKR3bOlr8ddrV44SQX4FLBZbhGQlEeITSynq3S1EBIiEmkjnjsjuWXQRIiokkOsJDtbb4HaEtfjWooiM9DM+I1+BXHYzpemIXQESsq3e1xe8IDX416MZkJ7Fs2z4do60A68pd6WZ7RmKM9vE7RINfDbpZRRnsb2xl9wFdYUl13+IHdL4eB2nwq0HXvirXtmpdYUlZLX5PN0mUkajz9ThFg18NusLMeAC2a/ArrGmZpdsWvxX8OpW3/zkW/CISKyLLRGS1iKwTkV/Y20eIyCciUioiz4tItFM1qMCUlRhDZmI0xTv2u12KCgDGGDzddPJnJkbT4vVxsKlt8IsKcU62+JuBs4wxJwGTgfNFZAbwv8CDxpgiYD9wnYM1qAAkIpxSmM66PTqWX9ldPT20+EGv3nWCY8FvLO1j9qLsLwOcBbxkb38a+KJTNajANXZIEjtqGqg91Op2KcplPkMPo3ra197Vfn5/c7SPX0QiRGQVUAksBD4DDhhj2j+7lQF5TtagAtPU4Wn4DKzbXet2Kcplxpge+/hBW/xOcDT4jTFeY8xkIB+YDozr7XNF5AYRKRaR4qoqncY31BRmJACwa3+jy5Uot/U0nPNwi1+D398GZVSPMeYAsBiYCaSKSPtUEfnA7h6e87gxZpoxZlpWVtZglKkGUW5KLBEeYec+Df5w5+vh5G56fDQiUKVdPX7n5KieLBFJtW/HAecAG7D+AFxu73Y18JpTNajAFRnhYWhqLLv26UVc4c5n6DJPT7vICA9p8XoRlxOcnKQtF3haRCKw/sC8YIx5XUTWA8+JyD3Ap8CTDtagAlhBery2+JU1ZUN3Z3eBjIRo7epxgGPBb4xZA0zpZvtWrP5+FeYK0uN5e12F22Uol/XUxw/WCV5dhcv/9Mpd5Zr8tHhqGlpoaNYLdMJZT5O0gXWCV1v8/qfBr1xTkG5N3aAje8KbQVv8g02DX7mmI/j1BG9Y62lUD0B2cgz1zW36qdDPNPiVa4bZwa8neMNTm9fHHa+WYIw1sqc7OUmxAFTWaXePP2nwK9ekxUeRGBPJLg3+sFRaVc/fl+4EYFR2Qrf7ZCdbV+9WHmwatLrCgQa/co2IUJgZz8ZynawtHLV33/z3WUVcOiW/231ykq0Wf4W2+P1Kg1+5alJ+KpvK69wuQ7lg4fpKAM4Y0/OV+R1dPdri9ysNfuWq/LQ49je26sm7MNPY0saf//MZALFRET3ulxwXSUykR/v4/UyDX7kqP806wavr74aXe9/Y2HF73JCkHvcTEbKTY6jQFr9fafArV+WnxQFQpmP5w8reWivIV9wxl8iIY8dQTlKsBr+fafArV+Wntge/tvjDSVOrl5MLUsmw59w/lpzkWCoPalePP2nwK1dlJsYQHelhtwZ/WNlYXkekp3fxk5Mcy97aJnw9DfZXfebk7JxKHZfHI+SnxmmLP4wYY6iub+71dMujshM41Opl78Em8uxPiGpgtMWvXJeXFqd9/GGkqdUHwJnHGMbZWXvYl9dq48BfNPiV6wrS49lW3YAx+lE+HNTbQ3fnnpDdq/1zU6zgbz8hrAZOg1+5rig7kYNNbVTp9Lth4e315QC9nnVziH31brkGv99o8CvXFWUnAlBaWe9yJWow1DdZLf5zxuf0av/kuEjioiI0+P1Ig1+5rjDDmqBrZ43284eDuqY2PAIn5Cb3an8RITcllr06lt9vNPiV69pnYPzL+1tdrkQNhmeX7SQmMoKInibh70ZOciwV2uL3Gw1+5bqYSGuuFh2mHR56Wlj9WHJTYvXkrh9p8KuA8MXJQ2n1+twuQw2CNp/h8qndT8Pck5wUa9oGvYjLPzT4VUDISbEuy2/T8A959U1tJMX27drR3JRY2nyG6gYd+eUPGvwqIEzKS6XF62PN7lq3S1EOa/OZ407MdqSh9lj+PQe0u8cfNPhVQJiUnwLAhr26Glcoa79Irw/ndQHr6m7QWVz9RYNfBYT8tDjioyN0LH+Ia++i9/TxDG978O/RdRv8QoNfBQQRoSA9nl379Bc7lPn62eJPiokkPjqC8lrt4/cHDX4VMPLT4tm1Tz/Kh7L24Jc+tvhFxBrLX6d9/P6gwa8CRkF6PLv2N+pkbSHM9LOrByAnOUanbfATDX4VMArS42hs8VLT0LvJu1Tw6W9XD0Bearye3PUTDX4VMAoyrIXXd2p3T8jq78ldsD4RVhxsZtLdC/xcVfjR4FcBY1iaFfzazx+6vL72Pv6+P3dYujWy52BTG02tXn+WFXY0+FXAGJZuBf8OnaUzZB0ex9+/Fn87nbdnYDT4VcCIjYpgSHKsBn8IO9zV0/fndg5+Hc8/MBr8KqAUZMSzc1+D22Uoh7Sf3O3LlMztspJiOm7v3q/BPxAa/CqgFKTHa4s/hPV3HP+Rz9mtLf4B0eBXAWV4ejyVdc0catGTd6FoIOP4O9PgHxjHgl9EhonIYhFZLyLrRORGe/vdIrJbRFbZXxc6VYMKPjqkM7QNZBw/wCNXTQG0j3+g+jYpdt+0ATcbY1aKSBKwQkQW2o89aIz5rYPvrYLUcHv93W3VDYwdkuRyNcrfBjKOH+DzJw3l7fUVrCk74Meqwo9jLX5jzF5jzEr7dh2wAchz6v1UaBibY4X9zS+scrkS5QTfAMbxtxuaGsveA7oa10AMSh+/iBQCU4BP7E3fF5E1IvJXEUnr4Tk3iEixiBRXVVUNRpkqAMRFW+vvNmgff0jyRx9/fmocLV6frsY1AI4Hv4gkAvOAHxpjDgKPAqOAycBe4IHunmeMedwYM80YMy0rK8vpMlUAufPi8QBU1+svdqjp6OMfQPIMTbWu4NUhnf3naPCLSBRW6P/DGPMygDGmwhjjNcb4gL8A052sQQWf0dmJAHymi7KEnFZ7TeWIASR/e/DrMoz95+SoHgGeBDYYY37XaXtup90uBdY6VYMKTqPs4C+t0uAPNW+tLQcgNS6q36/RvhrX7gM68qu/nBzVcxrwDaBERNrP1P0MuEpEJgMG2A5828EaVBDKTY4lKSaS9Xt0/d1Q835pNQDjBjBiKzk2iqSYSG3xD4BjwW+M+QDo7gzOG069pwoNHo8wuSCVFTv2u12K8rP46Agm5CWTnRw7oNfJS4vTi7gGoFddPSJyo4gki+VJEVkpIuc6XZwKXycXpLGpoo66pla3S1F+VFPfQmZizPF3PI6hqXF6cncAetvHf609IudcIA2rC+c+x6pSYW/ysFSMQbt7Qkx1fbNfgr8gPZ7PquppbGnzQ1Xhp7fB395lcyHwjDFmHd134yjlFyfkJgOwsbzO5UqUvxhj/NbinzEyneY2H1sqdABAf/Q2+FeIyNtYwb/AnoLB51xZKtzlJMeQnRTDyp3azx8qDja10eL1kZkYPeDXGm1f4V2qQ377pbcnd6/DuuBqqzGmUUTSgW86V5YKdyLCKSPSWbZtH8aYfk3jqwJL+wV5/mjxD0+PJypC2KLB3y+9bfHPBDYZYw6IyNeBO4Ba58pSCmaNymBvbROf6Xj+kLClwuq280fwR0Z4GJGZoC3+fupt8D8KNIrIScDNwGfA3xyrSilg5sgMAFbu0JkYQ0H7+ZrhGfHH2bN3irITtVHQT70N/jZjrZJ8CfAHY8wfAZ0zVzmqID2e5NhIlm6rcbsU5Ud59pQLA1WUlciOmgaaWnVCv77qbfDXichPsYZxzhcRD9D/a66V6oXICA8T81P4dKe2+EOB12fwiHWBnj8U5SThM7C9Rtdo7qveBv+VQDPWeP5yIB+437GqlLLNGpXJtuoGnakzBLT5DJEDmZbzCEVZ1pxOOqSz73r1v2CH/T+AFBG5GGgyxmgfv3Lc5GGpAGzS8fxBz+szRPiptQ8wMisBj+iQzv7o7ZQNVwDLgC8DVwCfiMjlThamFByezGvDXr2CN9i1eQ2Rfgz+2KgIhqXHa/D3Q2/H8d8OnGKMqQQQkSzgHeAlpwpTCiAjMYaspBi9gjcEeH0+IiL8ez1GUVaiBn8/9LbDzdMe+raaPjxXqQHJS43jpRVlusZqkGvzGSL8fCFeUU4i26obaPPqRAJ90dvwfktEFojINSJyDTAfnV5ZDZKJeSkALN++z+VK1ED4u48frBZ/i9fHzn26KEtf9Pbk7i3A48Ak++txY8ytThamVLubzx0DwCfbNPiDmTWqx7/Br3P29E+vF2IxxszDWj9XqUGVGh/NuCFJfLKtBhjtdjmqn3w+4/c+/lFZCQBsqazn3BP9+tIh7ZjBLyJ1WEskHvUQYIwxyY5UpdQRZozM4LnlO2lp8xEdqaeXglFDSxtxURF+fc2k2CiGJMfymbb4++SYv0HGmCRjTHI3X0ka+mowzRiZTlOrjzVlehVvsNrX0EJ6wsCnZD7S6JxESnXOnj7RppMKCtNHWBO2Fes6vEHLseDPTmJzRZ2O7OkDDX4VFNITohmSHMvmCh3PH6ycCv4Jeck0tfrYWq1z9vSWBr8KGqNzEnVeliDV2NLG/sZWspNi/f7a7cN9S8p0iZDe0uBXQWNsjvWR/lCLTsMbbMprmwAYlu6fKZk7G5mVSFxUBCW7Nfh7S4NfBY2ZozJobvOxdo/+ggebFrv/PSbSv6N6ACI8wvihyazV4O81DX4VNCbYH+nX6S940Glps4I/OsKZyDm5IJU1ZbXUHmp15PVDjQa/ChrZSdZarXf/ez3WgnAqWHQEv0PXYFw0aSgtXh/vbqxw5PVDjQa/ChrSaYKvF4vLXKxE9ZXTwT8pL4XEmEhdra2XNPhVUFlyyxwAXlqhwR9Mmr3OBr/HI0welsr7W6r102AvaPCroFKQEc93zhzFyp37aWhuc7sc1UtO9/EDXDJ5KNuqG1i8qfL4O4c5DX4VdE4vyqTNZ1ims3UGjfYhuPHR/h/V0+4Lk4cCcOer67TVfxwa/CroTCtMIzrSwwel1W6XonqpocX6dJYQ0+sJgfssJjKCiXkp7D5wiFW7tK//WDT4VdCJjYpgemE6H2rwB42H3tkCONviB/j79afiEXhzbbmj7xPsNPhVUDqtKJON5XW6AEeQqG+yWvyJDrb4AVLiopiYn8rjS7bqUp3HoMGvgtJlU/MAeGvtXpcrUb2REhfFZSfndxmS65Rx9qpc6/cedPy9gpUGvwpK2UmxDEuPY8Nena0z0BljqKpvZkhKzKC8383nWUt1aldgzzT4VdCaPCyNpVtrdB72ANfqNXh9hvhoZ7t52rU3CvQEb88cC34RGSYii0VkvYisE5Eb7e3pIrJQRLbY/6Y5VYMKbRdOGEJNQwsfb61xuxR1DO1DOWP9vOzisZxckMby7fvwaj9/t5xs8bcBNxtjxgMzgP8SkfHAbcAiY8xoYJF9X6k+mzMuG4BvPLlMx20HsMZW68Suv9fbPZazT8ihur6F4u16rUd3HAt+Y8xeY8xK+3YdsAHIAy4BnrZ3exr4olM1qNAWGxXBGWOyAFi6VX/BA9WSzVUA5Kb4fxGWnpw9LpuYSA/zS/Tkf3cGpY9fRAqBKcAnQI4xpv1/oxzI6eE5N4hIsYgUV1VVDUaZKgg9ctUUAJ78YJvLlaievFFSTmZiDGfaf6QHQ0JMJHPH5/Daqj00terCPUdyPPhFJBGYB/zQGNNlfJWxPp93+xndGPO4MWaaMWZaVtbg/cCo4JISF8V5J+bw7sYKXZkrANU1tbJ0aw2zx2bh8Tg/lLOzr51aQO2hVhas04u5juRo8ItIFFbo/8MY87K9uUJEcu3HcwGdUUkNyKVT8vAZWLhB52IPNJV1zTS3+ZhemD7o7z1jRAaZidH88vUNOvLrCE6O6hHgSWCDMeZ3nR76F3C1fftq4DWnalDh4bSiTAA9kReAmlutwE2Oixr09/Z4hAsn5lJd38zj728d9PcPZE62+E8DvgGcJSKr7K8LgfuAc0RkCzDXvq9UvyXFRnFaUQYrdux3uxR1hOY2q/stJsqdS4Z+cv44AFbu2E/hbfMpvG0+Wyr0oj8nR/V8YIwRY8wkY8xk++sNY0yNMeZsY8xoY8xcY4w209SATS1IY2N5HfU6R39AaW5rX2TdneBPjInkGzOG886Gwz3K33xqOa1h3vWjV+6qkDBjZAZen+HWeWvcLkV10r4UYkzk4I3hP9IV04Z13P7SyXmU7T/E6Nvf5Pqni8N2IjcNfhUSThlhnTxcsV27ewJJXVMrAMMz4l2rYWJ+Cv+8/lSumVXIry+d2LH9nQ0VvFC8y7W63KTBr0JCVISHL0/Np6nN29GvrNxX39xGSlwUmYmDM0FbT2YVZXL3F04kNiqC/71sYsfFZLe9XMKiMBwNpsGvQsYFE4dwoLGVj0p17p5AUXmwmczEaLfL6OLKUwr4+Kdn89J3ZgLw+JLwG/Gjwa9CxsyRmUR4REf3BIhWr48F68sZkZngdindmlaYzo/PHcMn2/axa1+j2+UMKg1+FTLioiMYn5uswR8g3t1YiTFw6ogMt0vp0SWTrQV9Hl60hbW7aztO9lYebKK0sp6lITrz6+BMkK3UIJk6PI3nl++izesjMkLbNW5qXxbzq6cWuFxJz4alxzNnbBYvrijjxRVlAMw9IbvL8M8bzhjJzy48wa0SHaG/GSqkTMpP4VCrl23VDW6XEva2VjWQkxxDgsPr7A7Uj84Z2+V+59AH6xzA7a+UDGZJjtPgVyFljL3e6ia9OtN1O2oaKMwIzP79zibmp7Dq5+ew9hfnccnkoQCMG5LE6rvOZcUdcwH4xyc7WbwxdKYV0+BXIaUoOxGPQElZrdulhL2DTa2kxQfWiJ6epMZHkxgTye+/MoUVd8zlrR+eQUpcFBmJMbz6X6cB1hW/H5ZWU3jbfL8NAd1e3UCr14fPZ7osJrSjpoH9DS1+eY/uBPZnMKX6KDYqgukj0nlzbTm3XTAOa65A5YbGFi/xMe5dsdtfGUdcczB5WCqzx2bx3qYqvvbEJwBc93QxANefPoKYKA8XTMhleEY8SbFRlFbWkZsSd8wuLp/PsKWynvMeWtKx7arpw7j3S5No8/o48/73ANh+30V+/u4sGvwq5Fw0MZc7X1vHtuoGRmYlul1O2Gps8RIfHXzB352/Xn0KI3/2xlHbn7AXAPrj4s8AOGd8DgvXV3D51Hzuv3wSK3bsZ0pBGhH2WgStXh9l+w8x57fvHfVazy7bxbPLul5J/MzH2/nGzEK/fi+gwa9C0Oyx2cA6/rV6DxocRAcAABIMSURBVD+cO8btcsJSaWUd+xpaSI0Ljq6e4/F4hFU/P4e311fw5an5ANwzfwO1h1p5yR4NBLBwvdUF9NKKsi7bv3pqAVmJMfx+0ZYurzshL5kRmYnkpsR2eyHZna+t46RhqUzKT/Xr96PBr0LOsPR4ThyazOtr9mrwu2SRPTLmc6MzXa7Ef1Ljo7tM+HbnxeMBuPX8cSxYV05zm49fvr6+2+f+85OdXe7PGZvF12cM5+wTDq88e/3pI/jWMys4c0wWPzpnDDX1zdz0wmpHRkVp8KuQdOHEXO5fsInKg01kJw/eIt/KUmxfROfvlmogykqK4eszhgNWP31ji5f3t1SRGBPFWeOyaWr1cuJdCwD4xRdOZHROIrNGHf0HMTs5ltfsE8lgnWv427XTHalZg1+FpPNOzOH+BZv495q9XHf6CLfLCTtenyEvNY64EOnj76346EjioyO5dEp+x7aEmEi23Xsh6/ceZHxuckAMONDhnCokFWUnMTEvhVc/3e12KWGp9lArBenuTcUcaESEE4emBETogwa/CmHnTxhCye5aquqa3S4l7OyoaWRYepzbZageaPCrkHX2CdkAzFtZdpw9lT/VNrZSXd/MKB1KG7A0+FXIGjckmZMLUnlrbbnbpYSV0iprcraibA3+QKXBr0LapPxUtlTUdbkcXjnrM3tWTm3xBy4NfhXSxuQk0dDipWz/IbdLCRulVfVER3gYpid3A5YGvwppk/JTAPjos2qXKwkPa8oO8PiSrYzLTeqYpkAFHg1+FdJOHJoMwK3zSqiu19E9TvL5DF/4w4cATMxLcbkadSwa/CqkiQi3XTAOgDf1JK+j9tQe7k77+efHu1iJOh4NfhXyvn3GSIalx/GfTaGzkEYgaj+P8sx104mJDK8rdoONBr8KeSLC7DHZfPRZDc1tXrfLCVm77eDPT9OTuoFOg1+Fhdljs2hs8VK8fb/bpYSs9hb/0FSdFC/QafCrsDBzVAbRER7e0+4ex5TtbyQnOUa7eYKABr8KC/HRkUwfkc57m6rcLiVkle0/pN08QUKDX4WN2WOz2FJZz0elOqbf36rqmvl4a03ILLUY6jT4Vdg4a5w1adttL5e4XEnoeaHYWiu2/RirwKbBr8LGyKxETi/KZOe+RtbtqXW7nJDh8xneXlfO2JwkvnmaLnoTDDT4VVj5ynRrzdTLHv3I5UpCx5/eK2V1WS1fnpZ//J1VQNDgV2Hl4klDyU6KoaXNR1OrjukfKGMMb64tZ3R2oi5xGUQ0+FXY+c3lk/AZ+Plra90uJei9vb6CdXsOMmdcdsAsK6iOz7HgF5G/ikiliKzttO1uEdktIqvsrwuden+lejJ7bDZTClJ5obiMz+xFQ1Tf7Wto4ZYXV5MUE8kt5411uxzVB062+J8Czu9m+4PGmMn21xsOvr9SPfr9lVMA+OFzq2hsaXO5muD04MLNHGxq47rPjSAqQjsPgolj/1vGmCXAPqdeX6mBaF8IvGR3Ld/7x0qXqwlOpfZKW9+dPcrlSlRfufFn+vsissbuCkrraScRuUFEikWkuKpKr7ZU/iUiHYH13qYqGpq11d9Xu/Y3csnkoTpFQxAa7OB/FBgFTAb2Ag/0tKMx5nFjzDRjzLSsrKzBqk+FkVvPH8fzN8wA0AXZ+6jN62NvbRPDdIqGoDSowW+MqTDGeI0xPuAvwPTBfH+ljjStMJ3oCA83v7iaQy06vLM3/r16D0W3v4nXZzq6zFRwGdTgF5HcTncvBXQ8nXJVhEf41hnW+PP2aQdU9yoPNlF423x+8OynHdvGDUl2sSLVX5FOvbCIPAvMBjJFpAy4C5gtIpMBA2wHvu3U+yvVWz8+dyzvrK9k3soyrp5V6HY5Aam0sp65v/tPx/0/fvVkphWmkZOsc+8HI8eC3xhzVTebn3Tq/ZTqLxHhkilD+c1bm9hbe4jcFO2+6Gz9noNc+PD7AERHelj0ozMZlq59+8FMB98qBZx34hAAZt77LmX7G12uJrC0h/6pI9LZ9MvzNfRDgAa/UsCorETG51r91af/72Iq65pcrigwPP3R9sO3r52u0zKECA1+pWyPfWNqx+3pv1rEs8t2hvVEbhv2HuSuf60D4K/XTCM2SsfrhwoNfqVsw9Lj2XbvhcwYmQ7AT18u4bqnl7tclTsWb6zkZ69YC9Y8fNUUZo/RBVZCiQa/Up2ICH+79lQunmSNPP6wtIat9kRuu/Y1csVjH/OlP33oZomOW7FjP998ajmf7jzA9aeP4AsnDcXj0S6eUCLGGLdrOK5p06aZ4uJit8tQYaa8tokZ9y4C4PtzivjD4tKOx35wVhE3nxt6M1LW1Dcz9Z53APj2GSP5yfnjiNDQD1oissIYM+3I7driV6oHQ1JimTwsFaBL6AM88m4p9SE4v8+d9hoF18wq5KcXnqChH6I0+JU6hle+N4tTR1h9/s/fMIMlt8zh6WutmUYm3LWAuqZWN8vrVm1jKz9/bW2fRya9t6mSN0qsOYvu+vx4J0pTAcKxC7iUCgUiwvPfntllW0FGPIUZ8WyvaeT55bu4/nMjAWt++rT4KL4yvYAXi3dx5SkFREcObttqW3UDc377HgA79zXy1DenY4zhX6v3EBsVwd+X7uC+yyYxNCWWv328gy2VdXxvdhFp8dFc83/WieznbpihwzZDnPbxK9UPLW0+zv/9ErZWNZCZGENuSiwlu2u77HPzOWP4wdmjAdhz4BDr9hxk9tgsRxctuerxpXy8teaY++SlxnHFtGE8+M7mox67ae4Ybpw72qny1CDTPn6l/Cg60sOP7ZO71fXNR4U+wGNLtrK39hA+n+Hap5bzrb8VM/WXCxlIY2tnTSNPvL+VO14t4Y+LS5l9/2I2lddhjOGZj7d3hP76/zmv2+dPHZ5GVV1zR+hfNCm3y+Pfm6OLqoQDbfEr1U/GGP6+dAfRkR521DTy/2YWkhofxYHGVuqbWznvoffx+gxnjMliyebDiwkdb0TQR6XVbK9p5CunDMPjEZrbvFz71HK2VTWwp/b4/fanFWXwj+tn8MzH27nztXWcNCyV2y88gYSYCIqyE9lZ08hFj3zAl6bkcd9lk/jP5iq8Ph9njsnWk7khpqcWvwa/Ug55fc0evv/Pw1MYv/XDz3H+Q++TmRjDe7fMJibSQ6RHaPMZHnpnM0+8v41Hv34y1z51+Gf9vBNzWLCuosf3uGDCEN7stIhMTnIMb990JilxUc58UyqoaPAr5YKPPqvmpeIyLp+az6yiTD4qrearT3wCQHSEh+S4KKrrm4/7OllJMTz2jakcPNTK8IwECtLju7TOSyvrGZmZoBdaqS56Cn4d1aOUg2aNymTWqMzD94syOWd8DgvXV9Di9XUJ/aumF/Dssp0AlP7qAh5etIX6Zi/nnZjDqSMzjvk+RdmJznwDKiRpi18pF7QPsfzzf7bS0ublyatPoTAzgb21h0hPiNYFzJVfaItfqQAiIlwyOY9LJud12a6LwKjBoMM5lVIqzGjwK6VUmNHgV0qpMKPBr5RSYUaDXymlwowGv1JKhRkNfqWUCjMa/EopFWaC4spdEakCdrhdhx9lAtVuFxFk9Jj1jR6vvgvFYzbcGJN15MagCP5QIyLF3V1GrXqmx6xv9Hj1XTgdM+3qUUqpMKPBr5RSYUaD3x2Pu11AENJj1jd6vPoubI6Z9vErpVSY0Ra/UkqFGQ1+pZQKMxr8SikVZjT4A4yIeETkVyLyiIhc7XY9wUBEEkSkWEQudruWYCAiXxSRv4jI8yJyrtv1BCr75+pp+1h9ze16/EmD349E5K8iUikia4/Yfr6IbBKRUhG57TgvcwmQD7QCZU7VGgj8dLwAbgVecKbKwOKPY2aMedUY8y3gO8CVTtYbaPp4/L4EvGQfqy8MerEO0lE9fiQiZwD1wN+MMRPsbRHAZuAcrCBfDlwFRAD3HvES19pf+40xj4nIS8aYywer/sHmp+N1EpABxALVxpjXB6d6d/jjmBljKu3nPQD8wxizcpDKd10fj98lwJvGmFUi8k9jzFddKtvvdLF1PzLGLBGRwiM2TwdKjTFbAUTkOeASY8y9wFFdEyJSBrTYd73OVes+Px2v2UACMB44JCJvGGN8TtbtJj8dMwHuwwq1sAl96Nvxw/ojkA+sIsR6RzT4nZcH7Op0vww49Rj7vww8IiKfA5Y4WViA6tPxMsbcDiAi12C1+EM29I+hrz9jPwDmAikiUmSM+bOTxQWBno7fw8AfROQi4N9uFOYUDf4AY4xpBK5zu45gY4x5yu0agoUx5mGsUFPHYIxpAL7pdh1OCKmPLwFqNzCs0/18e5vqnh6vvtNjNjBhd/w0+J23HBgtIiNEJBr4CvAvl2sKZHq8+k6P2cCE3fHT4PcjEXkW+BgYKyJlInKdMaYN+D6wANgAvGCMWedmnYFCj1ff6TEbGD1+Fh3OqZRSYUZb/EopFWY0+JVSKsxo8CulVJjR4FdKqTCjwa+UUmFGg18ppcKMBr/yOxGpH4T3+EIvp2z253vOFpFZ/XjeFBF50r59jYj8wf/V9Z2IFB45PXE3+2SJyFuDVZMaHBr8KmDZ0+V2yxjzL2PMfQ6857Hmr5oN9Dn4gZ8RpHPjGGOqgL0icprbtSj/0eBXjhKRW0RkuYisEZFfdNr+qoisEJF1InJDp+31IvKAiKwGZorIdhH5hYisFJESERln79fRchaRp0TkYRH5SES2isjl9naPiPxJRDaKyEIReaP9sSNqfE9EHhKRYuBGEfm8iHwiIp+KyDsikmNP5fsd4CYRWSUin7Nbw/Ps7295d+EoIknAJGPM6m4eKxSRd+1js0hECuzto0Rkqf393tPdJyixVoeaLyKrRWStiFxpbz/FPg6rRWSZiCTZ7/O+fQxXdvepRUQiROT+Tv9X3+708KtASK1AFfaMMfqlX379Aurtf88FHgcEq5HxOnCG/Vi6/W8csBbIsO8b4IpOr7Ud+IF9+3vAE/bta4A/2LefAl6032M81tzqAJcDb9jbhwD7gcu7qfc94E+d7qdx+Kr264EH7Nt3Az/utN8/gdPt2wXAhm5eew4wr9P9znX/G7javn0t8Kp9+3XgKvv2d9qP5xGvexnwl073U4BoYCtwir0tGWsG3ngg1t42Gii2bxcCa+3bNwB32LdjgGJghH0/Dyhx++dKv/z3pdMyKyeda399at9PxAqeJcB/i8il9vZh9vYarMVn5h3xOi/b/67AWg6vO68aay7+9SKSY287HXjR3l4uIouPUevznW7nA8+LSC5WmG7r4TlzgfHWuiYAJItIojGmcws9F6jq4fkzO30/zwC/6bT9i/btfwK/7ea5JcADIvK/wOvGmPdFZCKw1xizHMAYcxCsTwdY88pPxjq+Y7p5vXOBSZ0+EaVg/Z9sAyqBoT18DyoIafArJwlwrzHmsS4brVWz5gIzjTGNIvIe1tKJAE3GmCNXHmu2//XS889sc6fb0sM+x9LQ6fYjwO+MMf+ya727h+d4gBnGmKZjvO4hDn9vfmOM2SwiJwMXAveIyCLglR52vwmowFqm0gN0V69gfbJa0M1jsVjfhwoR2sevnLQAuFZEEgFEJE9EsrFak/vt0B8HzHDo/T8ELrP7+nOwTs72RgqH52O/utP2OiCp0/23sVazAsBuUR9pA1DUw/t8hDUFMFh96O/bt5dideXQ6fEuRGQo0GiM+TtwP3AysAnIFZFT7H2S7JPVKVifBHzAN7DW4j3SAuC7IhJlP3eM/UkBrE8Ixxz9o4KLBr9yjDHmbayuio9FpAR4CSs43wIiRWQD1tqvSx0qYR7WMnrrgb8DK4HaXjzvbuBFEVkBVHfa/m/g0vaTu8B/A9Psk6HrsfrjuzDGbMRa4jDpyMew/mh8U0TWYAXyjfb2HwI/srcX9VDzRGCZiKwC7gLuMca0AFdiLd25GliI1Vr/E3C1vW0cXT/dtHsC6zittId4PsbhT1dzgPndPEcFKZ2WWYW09j53EckAlgGnGWPKB7mGm4A6Y8wTvdw/HjhkjDEi8hWsE72XOFrksetZgrV4+363alD+pX38KtS9LiKpWCdpfznYoW97FPhyH/afinUyVoADWCN+XCEiWVjnOzT0Q4i2+JVSKsxoH79SSoUZDX6llAozGvxKKRVmNPiVUirMaPArpVSY0eBXSqkw8/8BuCDySBEyxXsAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learner.lr_find(show_plot=True, restore_weights_only=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train the Model\n", "\n", "We will now train the model using the estimated learning rate from above for 12 epochs using the [1cycle learning rate policy](https://arxiv.org/pdf/1803.09820.pdf)." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "begin training using onecycle policy with max lr of 0.001...\n", "Train for 374 steps, validate for 94 steps\n", "Epoch 1/12\n", "374/374 [==============================] - 9s 23ms/step - loss: 22.8788 - mae: 22.8866 - val_loss: 13.7107 - val_mae: 13.7028\n", "Epoch 2/12\n", "374/374 [==============================] - 9s 23ms/step - loss: 12.2521 - mae: 12.2531 - val_loss: 10.8341 - val_mae: 10.8276\n", "Epoch 3/12\n", "374/374 [==============================] - 9s 23ms/step - loss: 9.9158 - mae: 9.9183 - val_loss: 9.9131 - val_mae: 9.9106\n", "Epoch 4/12\n", "374/374 [==============================] - 8s 23ms/step - loss: 8.9252 - mae: 8.9264 - val_loss: 9.4691 - val_mae: 9.4692\n", "Epoch 5/12\n", "374/374 [==============================] - 8s 23ms/step - loss: 8.3064 - mae: 8.3072 - val_loss: 9.1714 - val_mae: 9.1709\n", "Epoch 6/12\n", "374/374 [==============================] - 8s 22ms/step - loss: 7.9027 - mae: 7.9037 - val_loss: 9.0367 - val_mae: 9.0353\n", "Epoch 7/12\n", "374/374 [==============================] - 9s 23ms/step - loss: 7.4723 - mae: 7.4741 - val_loss: 8.6807 - val_mae: 8.6820\n", "Epoch 8/12\n", "374/374 [==============================] - 9s 23ms/step - loss: 6.9741 - mae: 6.9762 - val_loss: 8.3878 - val_mae: 8.3916\n", "Epoch 9/12\n", "374/374 [==============================] - 8s 22ms/step - loss: 6.4518 - mae: 6.4508 - val_loss: 8.2264 - val_mae: 8.2321\n", "Epoch 10/12\n", "374/374 [==============================] - 8s 23ms/step - loss: 5.9795 - mae: 5.9803 - val_loss: 7.8524 - val_mae: 7.8609\n", "Epoch 11/12\n", "374/374 [==============================] - 8s 23ms/step - loss: 5.7376 - mae: 5.7394 - val_loss: 7.8682 - val_mae: 7.8760\n", "Epoch 12/12\n", "374/374 [==============================] - 8s 23ms/step - loss: 5.5266 - mae: 5.5273 - val_loss: 7.8161 - val_mae: 7.8243\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learner.fit_onecycle(1e-3, 12)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our final validation MAE is **7.82**, which means our predictions are, on average, about $8 off the mark, which is not bad considering our model only looks at the textual description of the wine and the winery.\n", "\n", "### Plot Some Training History\n", "\n", "The validation loss is still decreasing, which suggests we could train further if desired. The second and third plot show the learning rate and momentum schedules employed by `fit_onecycle`." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxdZb3v8c8vczPvjB0ytaVjOieUIrNwsCIyKFMBBQSrXL3IwXOOiN6jx6PnctXD4XhFsQiC0hZqAeEcUUAEgWtbaeg8UVqaNkmbpGkzN/Pv/rFWkt2wk2bYOzvZ+/d+vfZr7732Wiu/XTHfPM+z1vOIqmKMMcb0FRHsAowxxoxNFhDGGGN8soAwxhjjkwWEMcYYnywgjDHG+GQBYYwxxicLCGOGSUQKRERFJGoQ+94uIu+MRl3G+IsFhAkLInJIRNpEJKPP9i3uL/mC4FQ2tKAxZjRZQJhw8iGwovuNiMwH4oNXjjFjmwWECSe/AT7v9f424NfeO4hIioj8WkSqRaRURL4tIhHuZ5Ei8mMROS4iB4FP+Tj2cRE5KiLlIvJ9EYkcScEiEisiD4tIhft4WERi3c8yROS/RaRWRE6IyNtetX7DraFBRPaJyKUjqcOEJwsIE042AskiMsf9xX0T8HSfff4vkAJMAy7CCZQ73M++CFwJLAaKgev6HPsk0AGc5e5zOXDXCGv+FrAMWAQsBJYC33Y/+zpQBmQC2cADgIrILOCrwNmqmgR8Ajg0wjpMGLKAMOGmuxXxd8AeoLz7A6/Q+KaqNqjqIeDfgc+5u9wAPKyqR1T1BPC/vY7NBq4A7lXVJlWtAv7DPd9I3AJ8T1WrVLUa+BevetqBSUC+qrar6tvqTK7WCcQCc0UkWlUPqeqBEdZhwpAFhAk3vwFuBm6nT/cSkAFEA6Ve20qBKe7rycCRPp91y3ePPep2+dQCvwCyRljvZB/1THZf/wj4AHhVRA6KyP0AqvoBcC/wXaBKRJ4RkckYM0QWECasqGopzmD1FcDzfT4+jvNXeb7Xtjx6WxlHgdw+n3U7ArQCGaqa6j6SVbVwhCVX+Kinwv0uDar6dVWdBlwF3Nc91qCqa1T1fPdYBf7PCOswYcgCwoSjO4GPq2qT90ZV7QTWAT8QkSQRyQfuo3ecYh1wj4jkiIgHuN/r2KPAq8C/i0iyiESIyHQRuWgIdcWKSJzXIwJYC3xbRDLdS3T/ubseEblSRM4SEQHqcLqWukRkloh83B3MbgFOAV1D/DcyxgLChB9VPaCqm/v5+H8CTcBB4B1gDfCE+9ljwCvANuA9PtoC+TwQA+wGTgLrccYIBqsR55d59+PjwPeBzcB2YIf7c7/v7j8D+JN73AbgZ6r6Bs74w4M4LaJjON1c3xxCHcYAILZgkDHGGF+sBWGMMcYnCwhjjDE+WUAYY4zxyQLCGGOMTyE1e2RGRoYWFBQEuwxjjBk3SkpKjqtqpq/PQiogCgoK2Ly5v6sXjTHG9CUipf19Zl1MxhhjfLKAMMYY45MFhDHGGJ9CagzCl/b2dsrKymhpaQl2KSEhLi6OnJwcoqOjg12KMSbAQj4gysrKSEpKoqCgAGdOMzNcqkpNTQ1lZWVMnTo12OUYYwIs5LuYWlpaSE9Pt3DwAxEhPT3dWmPGhImQDwjAwsGP7N/SmPARFgExkC5VqhpaaGhpD3YpxhgzpoR9QAhwvKGN2ubABERtbS0/+9nPhnzcFVdcQW1tbQAqMsaYwQlYQIhIroi8ISK7RWSXiHzN3f4jEdkrIttF5AURSe3n+EMiskNEtopIwG6PFhHiYyJpbusMyPn7C4iOjo4Bj3v55ZdJTfX5T2OMMaMikC2IDuDrqjoXWAZ8RUTmAq8B81R1AfA+A690dYmqLlLV4gDWSXxsJK0dnXR0+n9Vxvvvv58DBw6waNEizj77bC644AKuuuoq5s6dC8A111xDUVERhYWFrFq1que4goICjh8/zqFDh5gzZw5f/OIXKSws5PLLL+fUqVN+r9MYY/oK2GWu7hq9R93XDSKyB5iiqq967bYRuC5QNfT1L/+1i90V9R/Z3qXKqbZO4qIjiYwY2iDs3MnJfOfT/a9L/+CDD7Jz5062bt3Km2++yac+9Sl27tzZc5noE088QVpaGqdOneLss8/ms5/9LOnp6aedY//+/axdu5bHHnuMG264geeee45bb711SHUaY8xQjcoYhIgUAIuBTX0++gLwh34OU+BVESkRkZWBqw4iRECgcxSWX126dOlp9xD85Cc/YeHChSxbtowjR46wf//+jxwzdepUFi1aBEBRURGHDh0KeJ3GGBPwG+VEJBF4DrhXVeu9tn8LpxtqdT+Hnq+q5SKSBbwmIntV9S0f518JrATIy8sbsJaB/tL/oKoRAaZnJQ78hUYoISGh5/Wbb77Jn/70JzZs2EB8fDwXX3yxz3sMYmNje15HRkZaF5MxZlQEtAUhItE44bBaVZ/32n47cCVwi6rvP9tVtdx9rgJeAJb2s98qVS1W1eLMTJ9Tmg9KfEwkze2ddPm5FZGUlERDQ4PPz+rq6vB4PMTHx7N37142btzo159tjDEjEbAWhDh3VD0O7FHVh7y2Lwf+CbhIVZv7OTYBiHDHLhKAy4HvBapWgITYSI43OmMRCbH++2dJT0/nvPPOY968eUyYMIHs7Oyez5YvX86jjz7KnDlzmDVrFsuWLfPbzzXGmJGSfv6AH/mJRc4H3gZ2AN2XBz0A/ASIBWrcbRtV9csiMhn4papeISLTcFoN4ITYGlX9wZl+ZnFxsfZdMGjPnj3MmTPnjPW2d3ax52g9k1ImkJkUe8b9w9lg/02NMWOfiJT0d6VoIK9iegfnPrS+Xu5n/wrgCvf1QWBhoGrzJToygpioCJrbOnDyyxhjwlvY30ntLSEmiqa2TgLVqjLGmPHEAsJLfEwkHZ1dtAXghjljjBlvLCC8xMc4PW6BmnbDGGPGEwsIL3HREUSK0Nw68DxJxhgTDiwgvIgIE2IiabIWhDHGWED0lRAbRUt7J51dwRmHSEx07uSuqKjguut8T1N18cUX0/dy3r4efvhhmpt7bzOx6cONMUNlAdFHfEwkEPxxiMmTJ7N+/fphH983IGz6cGPMUFlA9BEfE4WA37qZ7r//fh555JGe99/97nf5/ve/z6WXXsqSJUuYP38+L7744keOO3ToEPPmzQPg1KlT3HTTTcyZM4drr732tLmY7r77boqLiyksLOQ73/kO4EwAWFFRwSWXXMIll1wC9E4fDvDQQw8xb9485s2bx8MPP9zz82xacWOMt4BP1jem/OF+OLZjwF0igentHQgC0ZFnPufE+fDJB/v9+MYbb+Tee+/lK1/5CgDr1q3jlVde4Z577iE5OZnjx4+zbNkyrrrqqn7Xe/75z39OfHw8e/bsYfv27SxZsqTnsx/84AekpaXR2dnJpZdeyvbt27nnnnt46KGHeOONN8jIyDjtXCUlJfzqV79i06ZNqCrnnHMOF110ER6Px6YVN8acxloQPkSK0KWKMvIb5hYvXkxVVRUVFRVs27YNj8fDxIkTeeCBB1iwYAGXXXYZ5eXlVFZW9nuOt956q+cX9YIFC1iwYEHPZ+vWrWPJkiUsXryYXbt2sXv37gHreeedd7j22mtJSEggMTGRz3zmM7z99tuATStujDldeLUgBvhL39up5jYOn2hmRlYiE2JG/k90/fXXs379eo4dO8aNN97I6tWrqa6upqSkhOjoaAoKCnxO830mH374IT/+8Y9599138Xg83H777cM6TzebVtwY481aED50D1T7axzixhtv5JlnnmH9+vVcf/311NXVkZWVRXR0NG+88QalpaUDHn/hhReyZs0aAHbu3Mn27dsBqK+vJyEhgZSUFCorK/nDH3rXXupvmvELLriA3/3udzQ3N9PU1MQLL7zABRdc4JfvaYwJLeHVghik6MgIoiMj/HYlU2FhIQ0NDUyZMoVJkyZxyy238OlPf5r58+dTXFzM7NmzBzz+7rvv5o477mDOnDnMmTOHoqIiABYuXMjixYuZPXs2ubm5nHfeeT3HrFy5kuXLlzN58mTeeOONnu1Llizh9ttvZ+lSZ3mNu+66i8WLF1t3kjHmIwI23XcwjGS6775Ka5o41dbJ7EnJ/iovZNh038aEjoGm+7Yupn7Ex0TR1tlFe4dN3GeMCU8WEP1IiO0eh7B5mYwx4SksAmI43Whx0ZFEiAT9juqxJpS6JI0xAwv5gIiLi6OmpmbIv9gi3In7mq0F0UNVqampIS4uLtilGGNGQchfxZSTk0NZWRnV1dVDPrbuVDuNLR20HY/r9y7ncBMXF0dOTk6wyzDGjIKQD4jo6GimTp06rGP/vLeSu9Zt5pmVy1g2Ld3PlRljzNgWsC4mEckVkTdEZLeI7BKRr7nb00TkNRHZ7z57+jn+Nnef/SJyW6DqHMiSPKe0ktKTwfjxxhgTVIEcg+gAvq6qc4FlwFdEZC5wP/C6qs4AXnffn0ZE0oDvAOcAS4Hv9BckgZQaH8NZWYkWEMaYsBSwgFDVo6r6nvu6AdgDTAGuBp5yd3sKuMbH4Z8AXlPVE6p6EngNWB6oWgdSlOehpPQkXV129Y4xJryMylVMIlIALAY2AdmqetT96BiQ7eOQKcARr/dl7jZf514pIptFZPNwBqLPpKjAQ92pdg4eb/T7uY0xZiwLeECISCLwHHCvqtZ7f6bOtacj+tNcVVeparGqFmdmZo7kVD4V5Ts9W5sPWTeTMSa8BDQgRCQaJxxWq+rz7uZKEZnkfj4JqPJxaDmQ6/U+x9026qZlJOCJj7ZxCGNM2AnkVUwCPA7sUdWHvD56Cei+Kuk24KPrbcIrwOUi4nEHpy93t406EaEo32MBYYwJO4FsQZwHfA74uIhsdR9XAA8Cfyci+4HL3PeISLGI/BJAVU8A/wq86z6+524LiqL8NA4eb+JEU1uwSjDGmFEXsBvlVPUdoL/bjy/1sf9m4C6v908ATwSmuqHpHocoKT3J3831NaZujDGhJ+TnYvKHBTkpREeKdTMZY8KKBcQgxEVHUjg5hZLSoPVyGWPMqLOAGKTifA/byuposwWEjDFhwgJikIryPbR1dLGzoi7YpRhjzKiwgBik7oHq92wcwhgTJiwgBikrOY7ctAl2R7UxJmxYQAxBcX4aJYdP2rKbxpiwYAExBEvyPVQ3tHLkxKlgl2KMMQFnATEExd03zB22y12NMaHPAmIIZmYnkRQbZeMQxpiwYAExBJERwqK8VLuj2hgTFiwghqgo38O+ygbqW9qDXYoxxgSUBcQQFeenoQpbD9cGuxRjjAkoC4ghWpSXSoTAZutmMsaEOAuIIUqMjWL2xGS7o9oYE/IsIIahKN/DlsMn6ei0ifuMMaHLAmIYigs8NLV1sq+yIdilGGNMwFhADMOSvN4V5owxJlRZQAxDjmcC2cmxFhDGmJAWsDWpReQJ4EqgSlXnudueBWa5u6QCtaq6yMexh4AGoBPoUNXiQNU5HCJCUb7H7qg2xoS0QLYgngSWe29Q1RtVdZEbCs8Bzw9w/CXuvmMqHLoV5adRXnuKY3UtwS7FGGMCImABoapvAT5ntRMRAW4A1gbq5wda9wJC1s1kjAlVwRqDuACoVNX9/XyuwKsiUiIiK0exrkErnJxMXHSEBYQxJmQFbAziDFYwcOvhfFUtF5Es4DUR2eu2SD7CDZCVAHl5ef6vtB/RkREsyEmlpNSm/jbGhKZRb0GISBTwGeDZ/vZR1XL3uQp4AVg6wL6rVLVYVYszMzP9Xe6AivM97Kqo51Rb56j+XGOMGQ3B6GK6DNirqmW+PhSRBBFJ6n4NXA7sHMX6Bq0o30NHl7KtzCbuM8aEnoAFhIisBTYAs0SkTETudD+6iT7dSyIyWURedt9mA++IyDbgb8DvVfWPgapzJOyGOWNMKAvYGISqruhn++0+tlUAV7ivDwILA1WXP3kSYpiemWABYYwJSXYn9QgV56fx3uGTdHVpsEsxxhi/soAYoaJ8D7XN7Rw83hjsUowxxq8sIEaoqMDGIYwxockCYoSmZSTgiY+2eZmMMSHHAmKEuifuKzlsAWGMCS0WEH6wJN/DweomTjS1BbsUY4zxGwsIPyjOTwOwdaqNMSHFAsIPFuSkEB0pbLaAMMaEEAsIP4iLjqRwcoq1IIwxIcUCwk+K8j1sK6ulraMr2KUYY4xfWED4SXG+h9aOLnZV1AW7FGOM8QsLCD+xFeaMMaHGAsJPspLjyE2bYAFhjAkZFhB+VJTnYXPpSVRt4j5jzPhnAeFHRQVpVDe0UnbyVLBLMcaYEbOA8KMidwGhzbZOtTEmBFhA+NGsiUkkxkbZOIQxJiRYQPhRZISwOC/VZnY1xoQECwg/K8r3sK+ygYaW9mCXYowxI2IB4WdF+R5UYcvh2mCXYowxIxKwgBCRJ0SkSkR2em37roiUi8hW93FFP8cuF5F9IvKBiNwfqBoDYVFuKhFiN8wZY8a/QLYgngSW+9j+H6q6yH283PdDEYkEHgE+CcwFVojI3ADW6VdJcdHMmphsAWGMGfcCFhCq+hYwnOs9lwIfqOpBVW0DngGu9mtxAVac72HL4ZN0dtkNc8aY8SsYYxBfFZHtbheUx8fnU4AjXu/L3G0+ichKEdksIpurq6v9XeuwFOV7aGrrZO+x+mCXYowxwzbaAfFzYDqwCDgK/PtIT6iqq1S1WFWLMzMzR3o6v+ieuM/WhzDGjGejGhCqWqmqnaraBTyG053UVzmQ6/U+x902buR4JpCVFGsrzBljxrVBBYSIfE1EksXxuIi8JyKXD/WHicgkr7fXAjt97PYuMENEpopIDHAT8NJQf1YwiQjFBR4bqDbGjGuDbUF8QVXrgcsBD/A54MGBDhCRtcAGYJaIlInIncAPRWSHiGwHLgH+3t13soi8DKCqHcBXgVeAPcA6Vd019K8WXEvyPJSdPEVlfUuwSzHGmGGJGuR+4j5fAfxGVXeJiAx0gKqu8LH58X72rXDP3f3+ZeAjl8COJ8UFaYBzP8QV8yedYW9jjBl7BtuCKBGRV3F+ib8iIkmALb48gLmTkomNirB5mYwx49ZgWxB34lx5dFBVm0UkDbgjcGWNfzFRESzMTaXksAWEMWZ8GmwL4lxgn6rWisitwLeBusCVFRqK8j3sKq/jVFtnsEsxxpghG2xA/BxoFpGFwNeBA8CvA1ZViCjO99DRpWwvs4n7jDHjz2ADokOdhZavBn6qqo8ASYErKzQs6VlhzrqZjDHjz2DHIBpE5Js4l7deICIRQHTgygoNnoQYpmcm2B3VxphxabAtiBuBVpz7IY7h3N38o4BVFUKK8j2UHD5Jl03cZ4wZZwYVEG4orAZSRORKoEVVQ2cMoqESOjsCcuri/DRqm9s5eLwpIOc3xphAGexUGzcAfwOuB24ANonIdYEsbNQ0n4BfXAh/DMy6REvciftKSocz87kxxgTPYLuYvgWcraq3qerncSbZ+1+BK2sUxafBguvh3cdg0yq/n356ZgKp8dE2L5MxZtwZ7CB1hKpWeb2vIZTWs77sX6DmAPzxG5A2DWZc5rdTiwhFeR67kskYM+4M9pf8H0XkFRG5XURuB37POJ8r6TQRkfCZxyC7EH57O1Tu9uvpiwo8HKxu4kRTm1/Pa4wxgTTYQep/BFYBC9zHKlX9RiALG3WxibDiWYhJgDU3QqP/VqcryrMFhIwx48+gu4lU9TlVvc99vBDIooImZQqsWAtN1fDMzdDun6m6F+amEhUhNi+TMWZcGTAgRKRBROp9PBpEJDQXXJ6yBK59FMr+Bi99FXTk9y/ERUdSOCWFEpvZ1RgzjgwYEKqapKrJPh5Jqpo8WkWOusJr4NJ/hh2/hb/80C+nLM73sK2slrYOmyXdGDM+hM6VSP52/n2wcAW8+W+wY/2IT1eU76G1o4tdFTYJrjFmfLCA6I8IfPo/Ie9c+N3/gCPvjuh0RT03zFk3kzFmfLCAGEhULNy4GpInwTMroPbwsE+VnRxHjmeCBYQxZtwIWECIyBMiUiUiO722/UhE9orIdhF5QURS+zn2kIjsEJGtIrI5UDUOSkI63LwOOtpgzU3QMvyx+aJ854Y59cPAtzHGBFogWxBPAsv7bHsNmKeqC4D3gW8OcPwlqrpIVYsDVN/gZc6CG56E6r3w3J3QNbwV4orzPVQ3tFJ28pR/6zPGmAAIWECo6lvAiT7bXlXV7mlTN+JMGz4+TP84XPEj2P8qvPrtYZ1iiY1DGGPGkWCOQXwB+EM/nynwqoiUiMjKUaxpYGffCefcDRt/Bu8+PuTDZ09MJiEmks02s6sxZhwY7GR9fiUi3wI6cNaY8OV8VS0XkSzgNRHZ67ZIfJ1rJbASIC8vLyD1nuYTP4ATB+Hlf4S0qU7LYpAiI4TFeR5KSm2NamPM2DfqLQh3sr8rgVu0n9FaVS13n6uAF3CmF/dJVVeparGqFmdmZgag4j4iIuG6xyFzNqy7Har3DenwonwP+47VU9/SHpj6jDHGT0Y1IERkOfBPwFWq2tzPPgkiktT9Grgc2Olr36CJTYKbn4GoGFhzAzTVDPrQC2dm0KXw989spaV9eIPdxhgzGgJ5metaYAMwS0TKRORO4KdAEk630VYRedTdd7KIdE8fng28IyLbcFax+72q/jFQdQ5bah7ctBbqj8Kzt0JH66AOK8pP41+vLuT1vVXc9dRmmtsCs9SpMcaMlITSNfnFxcW6efMo3zax8zlY/wVnWo5rfu7cgT0I6zYf4f7ntlOcn8YTd5xNYmxQhoOMMWFOREr6u53A7qQeqXmfhYsfgG1r4Z2HBn3YDcW5PHzTYkoOn+SWX26irtnGJIwxY4sFhD9c9E8w7zp4/Xuw+8VBH3bVwsn87JYl7KmoZ8VjG6lpHFw3lTHGjAYLCH8QgasfgZyl8PyXoPy9QR/6icKJrPp8EQeqG7lp1Uaq6v2zSJExxoyUBYS/RMfBTWsgMRPWroC68kEfevGsLJ68Yynltae44RcbKK+1qTiMMcFnAeFPiZnOutZtTbD2RmhtHPSh505P5zd3nkNNYxs3PLqB0pqmABZqjDFnZgHhb9lz4fpfQeUueP6LQ5rYryjfw5ovLqOprYMbfrGBD6oGHzDGGONvFhCBMOPvYPmDsO9l+NN3h3To/JwUnl15Lp1dcOMvNrC7IjSX/jbGjH0WEIFyzpfg7C/CX38CJU8N6dBZE5N49kvLiI6MYMVjG9l2xOZuMsaMPguIQFr+oDOZ3+/vgw99zjXYr+mZifz2y+eSFBfFLb/cxLuHbAZYY8zosoAIpMgouP5JSD8Lnv0cHP9gSIfnpsXz2y+fS1ZSLJ9//G/89YPjganTGGN8sIAItLgUWPGMMwvsmhugeWgtgUkpE3jmS8vIS4vn9iff5Y29VQEq1BhjTmcBMRrSpjr3SNQdgXWfd9a3HoKspDjWrlzGzOxEVv5mM3/ceTRAhRpjTC8LiNGStwyu+ikceht+//fQObS5l9ISYlh91zLmT0nhK2u28OLWwd+IZ4wxw2EBMZoW3ggX/iNseRoemguvfAuq9gz68JQJ0fzmznM4u8DDvc9u5dl3DwewWGNMuLOAGG2XfMu52zp3KWx6FH62DFZd4qxxferMl7MmxEbxq9uXcsGMTL7x3A6e+uuhwNdsjAlLth5EMDVWw451sGU1VO2CqDiYfSUsvgWmXuQMbPejtaOTr67Zwmu7K7n/k7P58kXTR7FwY0yoGGg9CAuIsUAVjm51gmLHb6GlFpJzYNEKWHQzpE3zeVh7Zxf3rdvGf22r4GuXzuDey2Ygg1ywyBhjwAJifGlvcabo2LoaDvwZtAvyz4NFt8DcqyE28bTdO7uUbzy3nfUlZXzpwmnc/8nZFhLGmEGzgBiv6iucleq2rIYTByA6AQqvdbqg8s7tWd60q0v555d28vTGw3z+3Hy+++lCIiIsJIwxZzZQQNhCyGNZ8mS44Otw/n1wZJNz9dOuF2Dr006306KbYeEKIlJy+Ner5zEhOpLH3v6Q1vYu/u0z84m0kDDGjEBAr2ISkSdEpEpEdnptSxOR10Rkv/vs6efY29x99ovIbYGsc8wTce6juPqn8A/vwzWPQvIU+PP34T/mwW+uRXY+xwOXT+Wej5/Fs5uPcN+6rbR3dgW7cmPMOBbQLiYRuRBoBH6tqvPcbT8ETqjqgyJyP+BR1W/0OS4N2AwUAwqUAEWqenKgnxdyXUxncuJDpwtqq3uXdlwKzLuO9V0X8Q9/jeQThRP5vyuWEBNlVzMbY3wL6hiEiBQA/+0VEPuAi1X1qIhMAt5U1Vl9jlnh7vMl9/0v3P3WDvSzwi4gunV1waG3nLGKPS9BRwsnE6bzSO0yqqZezQ9vu4y46P4vmTXGhK+xNgaRrardkwkdA7J97DMFOOL1vszd9hEishJYCZCXl+fHMseRiAiYdrHzaPkx7Hwez9bVfLtpNe1HnmHXD+eRWFBEwdylRE1eABkzITI6uDUbY8a8oA5Sq6qKyIiaMKq6ClgFTgvCL4WNZ3EpUHyH86jex+HXVhG//8/kvv9rovY/AYBGxCBZsyB7PmQXwsR5zuuE9CAXb4wZS4IREJUiMsmri8nX/NXlwMVe73OAN0ehttCSOYvpN/87XV3KW/sq+PM7G6g/9B6z5TDn1R5lZu2fiN22pnf/pElOYGTPg4nznef0s5x1LYwxYScY/89/CbgNeNB9ftHHPq8A/+Z1hdPlwDdHp7zQExEhXDxnChfPuY6K2k/xzN8Oc+e7R6hqaKUwpY0vzmzi0rRqkk7uhcpdcPAv0OXONhsZC1mznRbGxHlOaGQXQnxacL+UMSbgAn0V01qclkAGUAl8B/gdsA7IA0qBG1T1hIgUA19W1bvcY78APOCe6geq+qsz/bywHaQehvbOLv60u5KnN5Xy/z6oISpC+EThRG45J49zC5KQ4/uhcicc2+GERuVOaKruPUHyFLel4QZG9nxInz7g/FHGmLHH7qQ2AzpY3ciaTYf5bUkZdafamZaRwM3n5HFdUQ6p8TG9OzZUQuUOOLazNzSq94F2Op9HTYCsOU63VGoupORCap7zSMmB6AnB+YLGmH5ZQJhBaWnv5NSq5YgAABNUSURBVPfbj7J6UynvHa4lNiqCKxdM5tZleSzKTfU9x1NHK1Tv9QqNHXDyENSV9wZHt4QsH8HR/ToXYpNG5XsaY3pZQJgh211Rz+pNpfxuSzlNbZ0UTk7mlnPyuXrRZBJiBzF01dkBDUedG/hqD0PtEag77PW6DDpbTz9mgsdHeOT2vp7g6Zl/yhjjHxYQZtgaWzv43ZZynt5Yyt5jDSTGRnHt4incsiyP2ROTh3/iri5oqnLCorbUDRI3TLpftzedfkxM4keDI3kKTEiF2GSnBdL9HJPo3B9ijBmQBYQZMVXlvcO1rN5Yyn/vOEpbRxfF+R5uXZbP8nkT/X+ntio0n3BbHX2Co/aws72lbuBzxCQ5YRHXHR7ej2SvUEk6PVy8949JtIF3E9IsIIxfnWxqY31JGas3lXKophlPfDQ3FOeyYmkeBRkJo1dIS50zJXpLPbQ2QGu9+2jwerjvW/pub4C2hsH9nJjE3vBIyHBm0k0/y7lqK/0s8EyF6LjAfldjAsQCwgREV5fy1wM1rN5Uyqu7K+nsUi6YkcGn5k/iY9MzyE2bMLYXL+rqhLbGjwaKrzDpDp+GSmdtDu9LfhGn2yvdDY606b0BkppvNxqaMc0CwgRcZX0Lz757hGf+dpiKuhYApqROYNm0dD42PZ1zp6czOTWELnNtqYOaA3DiINR84Lzufm716vqKiAJPgVdoeIVI8hQbJzFBZwFhRo2q8kFVIxsO1rDhQA0bD9Zwstm5Kzs/PZ5zpzlhce60dLKSQ7BbRhWaa3oD40R3cBx0Xrc39+4bFed2V00/vdWRfhYkZNoVW2ZUWECYoOnqUvYea+gJjE0f1tDQ0gHA9MwEzp2ezsemZ7BsWjppCTFnONs4p+pc+uvd4uhugZz4sHd6E3AG2NOngye/99Lf7qu3UnKdK7eM8QMLCDNmdHYpuyrq2HCghg0Ha3j3wxM0tTk31M2emMQyt4WxbGo6KfFhNCV5Z4dzldaJA254uAHSffVWR8vp+8cmO3ene4dGai6kuHetJ2Zb95UZFAsIM2a1d3axo9wNjAM1bC49QUt7FyJQODm5p0vq7II0kuLCKDC8qULT8d5Lfrsv960r693WUnv6MZExzhhHd2icFiI5kJwDUSHeYjODYgFhxo3Wjk62HanjrweOs+FADVsO19LW2UVkhDB/SkrP+EVxgYf4GLs6qEdrg1d4HHbD40jvtoZjOKv3dhNImnh6CyQlx7lbfUIqxHU/pzprjNiVWCHLAsKMWy3tnbxXepK/ul1S247U0tGlREcKi3JTWTYtndkTk5mZnUhBRgLRkdat4lNHG9SX9QZGXZnX9Cfue+8xkL5iknoDY4IbGqe9T3XC5bT33eESpi2/ccICwoSMptYONpeedLukjrOjvI4u9z/hqAhhakYCM7OTOCsrkZnZSczITqQgPYGYKAuOAXV1Ofd2nDrpdFedqh38c8epgc8dk+gjOFIhMcu5YitjJmSc5QSMGXUWECZknWrr5EB1I/urGthf2cj7lY18UNVA6Ylm1Cs4CjISmJmdyFlZSczMTmRGVhJTMyw4/KKj1bkvZCih0lILjZXQ1dF7nviM3rDImAnpMyBjht1sGGAWECbstLS7wVHphIcTHI2U1jT1tDgiI4SC9HinpZGVyAy3xTE1I4HYKJt/KeA6O5yJGo+/D8f3O881Hzivm4/37hcR7dwvkuEGRk94WKvDHywgjHG1tHdysLrJq8XRwAdVjRzqExz56fHMzHICY4YbINMyLThGTfMJNyzc8Oh+feLg6a2OhMzelkbGDGt1DIMFhDFn0NLeyYfHm3oC4/3KBvZXNVJa00ynmxwRAgUZCcyemMSs7GRmT0pi9sQkcj3xRETYXc+jorMdTpZCzf4+rY73nTvYu/VtdaTPcBekUtAu96HO4yPb3NfD2tdrv4goZ4A+MsZ9HeO+j3bqi4xxQiwypp/30b6Pj4jy6132FhDGDFNrR3dwNPJBZQP7KhvYd+z0MY74mEhmZCcxZ2ISs9zH7InJoX9n+FjTfMJtbXSHh/u6b6sjFHgHSEQ0JE2Cu98Z1qkGCohRb4OJyCzgWa9N04B/VtWHvfa5GHgR+NDd9Lyqfm/UijTGFRsVyeyJyR9ZHKm5rYP3KxvZd6yevcca2Hu0gVd3V/LMu0d69slKinXDIolZE5OZPdG5usrva2cYR3wa5J3jPLx1tjv3hrQ1gUS4D+l9jXi977vd1/4McB6vfbo6oLOt97mz3cf7dufy4s42Z0yms8193z6I/b3eR8cH5J901ANCVfcBiwBEJBIoB17wsevbqnrlaNZmzGDFx0SxKDeVRbm9cyKpKtWNrew96rQy9h5rYF9lPU9tKKWtowvoHRif7QZGd2sjxzPBuqkCJTLamddq1H9u1LhfJyTYoziXAgdUtTTIdRgzYiJCVlIcWUlxXDgzs2d7R2cXh2qa2XesgX3H6tlzrIEd5XX8fsfRnn0SYiKZ2d3ayE5i9iQnQFLjrZvKBE9QxyBE5AngPVX9aZ/tFwPPAWVABfAPqrqrn3OsBFYC5OXlFZWWWtaY8aGptYP3K92WxrEG9rrdVbXNvXc0T0mdwIKcFObnpLAwJ5V5U1JImWB3Jhv/GZOD1CISg/PLv1BVK/t8lgx0qWqjiFwB/KeqzjjTOW2Q2ox3qkpVQ6s7rlHPzop6tpfVUlrTu47E1IwE5k9JYUFOCgtyUimcnExCbLA7A8x4NaYGqb18Eqf1UNn3A1Wt93r9soj8TEQyVPV4332NCSUiQnZyHNnJcVzk1U1V29zGjvI6tpfVsb2sls2HTvDStgrAufz2rKxE5k9JZWFuCvOnpDBnUrINhpsRC2ZArADW+vpARCYClaqqIrIUiABqfO1rTDhIjY/hghmZXDCjNzSqG1rZUV7LtiN17Civ4y/vV/Hce2WAM73IrIlJLMhJdbqopqQwa2KSTWZohiQoXUwikgAcBqapap277csAqvqoiHwVuBvoAE4B96nqX890XutiMuFMVTla18L2slq2l9X1tDjqTjljGjFREcydlMzCnBTm56SyMCeFaZmJRNrVU2FtTI5BBIIFhDGnU1UOn2hmW1kdO8pq2VZWx87yOprdVfwSYiIpnJLCginOQPjiXA+5aRMQWw87bIzVMQhjTICJCPnpCeSnJ3DVwsmAs+zrwerGnvGM7eV1/Hpj770a6QkxPfd4LM7zsCA3heRwXc0vzFlAGBNmIiPEnbk2ic8W5QDO0q/7jjWw9UgtW4/UsuXwSV7fWwU4NwtPz0xkcW4qi/Kc4JiVnUSUjWeEPOtiMsb4VHeqne1ltWw5XNsTHCea2gCYEB3pdkmlsjgvlUW5HiamjO+7hsOVjUEYY0asezzDaWHUsuVILXsq6mnrdLqmJibHuWHhPObnpNi64eOAjUEYY0bMezzj6kVTAGe2290V9ae1Mv6w8xjgdGXNyk5iUV5qT0tjWkaizTk1jlhAGGOGLTYqksV5Hhbn9a7sVtPY6jWWUct/ba1gzabDACTF9U5y2D11SHZyrF01NUZZF5MxJqC6upSDxxt7uqW2Hq5l77H6nhX8MhJjmDs5hXmTkymcnMK8KcnkpcVbaIwSG4MwxowpzW0d7K6oZ1dFPTvL69hZUc/+ygY63NRIio1i7uRk5k1JodB9npaRYFdOBYCNQRhjxpT4mCiKC9IoLkjr2dba0cn7xxrZVVHHzoo6dpbXs3pTKS3tziB4bFQEcyYlM2+K29KYnMLMiYm2TngAWUAYY8aE2Cjn0tn5OSk92zo6uzh4vMkJjXKntfHilgqe3uiMaUS593Q43VNOS2POJJvd1l+si8kYM650dSlHTjaf1j21q7yOGvceDRFnSvR5k3u7pwonJ9viS/2wLiZjTMiIiOi93PaK+ZMA5x6NyvrW3pZGRR0lpSd7pkQHSJkQTVZSLFnJsWQnxZHpPmclx5KVFEe2+zwhxrqsullAGGPGPRFhYkocE1PiuHROds/2E01t7mB4HeW1p6iqb6WyoYVNH56guqG15yY/b0mxUaeHRnKcGyzuc1Is2clxYdGNFfrf0BgTttISYjh/Rgbnz8j4yGeqSm1zO5UNLVTVt1LV0EplfQvV7nNVQyslh09SWd/aM5Ght4SYSLKT48h0wyPbbZ0465LHkpEUS0ZiLKkTosftzYEWEMaYsCQieBJi8CTEMHti//upKvWnOryCpIVK97n7/fayWirrW3quuPIWGSF44mPISIwhIzGWjMQY0hNjSfd+n+AESnpCzJhaCdACwhhjBiAipMRHkxIfzczspH73U1UaWjuoqneC43hTGzWNrRxvbKWmsY3jjW0cb2yl9EQTNY1tPWty9JUYG9UbIgkxTkskwXmf0SdYUiZEB/SGQgsIY4zxAxEhOS6a5LhozsrqP0i6Nbd1uMHRyvFGJ0xqmtqobnCeaxpbOVTTREnpSU40t+HrgtOoCCE9MYa8tHh+++WP+f07WUAYY0wQxMdEEZ8WRW5a/Bn37exSTjS1UdPU+pFQOd7YSkSAWhEWEMYYM8ZFRgiZSbFkJsWO6s+1iU2MMcb4FLSAEJFDIrJDRLaKyEdufxbHT0TkAxHZLiJLglGnMcaEq2B3MV2iqsf7+eyTwAz3cQ7wc/fZGGPMKBjLXUxXA79Wx0YgVUQmBbsoY4wJF8EMCAVeFZESEVnp4/MpwBGv92XuttOIyEoR2Swim6urqwNUqjHGhJ9gBsT5qroEpyvpKyJy4XBOoqqrVLVYVYszMzP9W6ExxoSxoAWEqpa7z1XAC8DSPruUA7le73PcbcYYY0ZBUAJCRBJEJKn7NXA5sLPPbi8Bn3evZloG1Knq0VEu1RhjwlZQFgwSkWk4rQZwrqRao6o/EJEvA6jqo+JMMPJTYDnQDNyhqgOuBiQi1UDpMMvKAPq7omq8s+82foXy97PvNjbkq6rP/vmQWlFuJERkc3+rKo139t3Gr1D+fvbdxr6xfJmrMcaYILKAMMYY45MFRK9VwS4ggOy7jV+h/P3su41xNgZhjDHGJ2tBGGOM8ckCwhhjjE9hHxAislxE9rnTit8f7Hr8SURyReQNEdktIrtE5GvBrsnfRCRSRLaIyH8HuxZ/EpFUEVkvIntFZI+InBvsmvxJRP7e/W9yp4isFZG4YNc0XCLyhIhUichOr21pIvKaiOx3nz3BrHG4wjogRCQSeARnPqi5wAoRmRvcqvyqA/i6qs4FluHMeRVK3w/ga8CeYBcRAP8J/FFVZwMLCaHvKCJTgHuAYlWdB0QCNwW3qhF5EueGXm/3A6+r6gzgdff9uBPWAYEz/9MHqnpQVduAZ3CmGQ8JqnpUVd9zXzfg/JL5yIy445WI5ACfAn4Z7Fr8SURSgAuBxwFUtU1Va4Nbld9FARNEJAqIByqCXM+wqepbwIk+m68GnnJfPwVcM6pF+Um4B8SgphQPBSJSACwGNgW3Er96GPgnoCvYhfjZVKAa+JXbffZLd86ykOBO1Plj4DBwFGeetVeDW5XfZXvNHXcMyA5mMcMV7gERFkQkEXgOuFdV64Ndjz+IyJVAlaqWBLuWAIgClgA/V9XFQBPjtIvCF7c//mqcIJwMJIjIrcGtKnDUuZdgXN5PEO4BEfJTiotINE44rFbV54Ndjx+dB1wlIodwugY/LiJPB7ckvykDylS1u7W3HicwQsVlwIeqWq2q7cDzwMeCXJO/VXavgOk+VwW5nmEJ94B4F5ghIlNFJAZnoOylINfkN+6MuI8De1T1oWDX40+q+k1VzVHVApz/3f6sqiHxV6iqHgOOiMgsd9OlwO4gluRvh4FlIhLv/jd6KSE0CO96CbjNfX0b8GIQaxm2qGAXEEyq2iEiXwVewbmS4glV3RXksvzpPOBzwA4R2epue0BVXw5iTWZw/iew2v3D5SBwR5Dr8RtV3SQi64H3cK6028I4nppCRNYCFwMZIlIGfAd4EFgnInfiLEFwQ/AqHD6basMYY4xP4d7FZIwxph8WEMYYY3yygDDGGOOTBYQxxhifLCCMMcb4ZAFhzBggIheH2oy0ZvyzgDDGGOOTBYQxQyAit4rI30Rkq4j8wl2PolFE/sNd3+B1Ecl0910kIhtFZLuIvNC9JoCInCUifxKRbSLynohMd0+f6LUGxGr3LmNjgsYCwphBEpE5wI3Aeaq6COgEbgESgM2qWgj8BedOWoBfA99Q1QXADq/tq4FHVHUhzhxE3bN+LgbuxVmbZBrOnfDGBE1YT7VhzBBdChQB77p/3E/AmYStC3jW3edp4Hl3TYdUVf2Lu/0p4LcikgRMUdUXAFS1BcA9399Utcx9vxUoAN4J/NcyxjcLCGMGT4CnVPWbp20U+V999hvu/DWtXq87sf9/miCzLiZjBu914DoRyYKedYfzcf5/dJ27z83AO6paB5wUkQvc7Z8D/uKu7FcmIte454gVkfhR/RbGDJL9hWLMIKnqbhH5NvCqiEQA7cBXcBb0Wep+VoUzTgHONM+PugHgPSPr54BfiMj33HNcP4pfw5hBs9lcjRkhEWlU1cRg12GMv1kXkzHGGJ+sBWGMMcYna0EYY4zxyQLCGGOMTxYQxhhjfLKAMMYY45MFhDHGGJ/+P2tcvdlPpB2JAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learner.plot('loss')" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEWCAYAAABBvWFzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3gVZfbA8e9JpyYEQg0QSiihpoC96wI2LKggIAhrQVjrqqiru/pzV1lXsYHKLggKCIgtay/YC6TQe+idACHUhJTz++MOa4whELg3k3tzPs+Th7nvvPPOmdHkZGbenBFVxRhjjDlVQW4HYIwxJjBYQjHGGOMVllCMMcZ4hSUUY4wxXmEJxRhjjFdYQjHGGOMVllCMqQJEJE5EVERCvDTeehG52Nt9jSmPJRRjOPYPVRE5X0SKReSAiOwXkZUicvNxxhouIiuc/jtE5GMRqeO76I2pGrzy25AxAW6rqsaKiAB9gFQR+UlVV5buKCLnAf8AeqvqfBGJBq6o5HiNcYVdoRhzgtTjY2AP0PUY3XoAP6vqfGebPao6RVX3A4hIDRF5VkQ2iEiuiPwgIjVKbD9QRDaKyC4ReeRoo4gEichoEVkjIrtFZJaTrI6uH+yMubvkds66ySLyZInP54vI5rKCP95+jCmPJRRjTpDzw/ZKoAGQdYxuc4FeIvK4iJwlIuGl1v8LSAbOBKKBB4DiEuvPBtoDFwGPiUhHp/1PwFXAeUBTIAcY58SVALwCDHbW1QdiT/Iwj7kfY47HEooxx9dURPYCh4H3gHuPXoGUpqrfA9cAScBHwG4ReU5EgkUkCBgG3KWqW1S1SFV/UtX8EkM8rqqHVXUhsBDo5rTfDjyiqpud/n8D+jkP8fsBH6rqd866R/ltkqqI8vZjTLnsfxJjju/oM5Rw4GngQuD5Y3VW1U+AT5wEcgHwNrASTzKKANaUs6/tJZYPAbWd5ZbAeyJSMlEUAY3wXElsKrH/gyKy+wSPrbTy9rPlJMc01YRdoRhzgpzf2B8EuojIVSfQv1hVvwLmAJ2BXUAe0OYkdr8J6KOqUSW+IlR1C7ANaH60o4jUxHPb66iDQM0Snxuf5H6MKZclFGN+FSoiESW+fncFr6pHgGeBx8oaQET6ikh/EaknHj3xPI/4RVWLgUnAcyLS1LkNdkYZz1nK8irwdxFp6ewnRkT6OutmA5eLyNkiEgY8wW+/txcAl4pItIg0Bu4+yf0YUy5LKMb86mM8z0mOfv3tGP0mAS1EpKzpwDnALcBqYB8wFXhGVac56/8MLAbS8MwWG8OJfR++AKQCn4vIfuAX4DQAVV0KjASm47layQFKzuJ6E8/zmPXA58DMk9mPMccj9oItY4wx3mBXKMYYY7zCEooxxhivsIRijDHGKyyhGGOM8Ypq/YeNDRo00Li4OLfDMMYYv5KRkbFLVWNKt1frhBIXF0d6errbYRhjjF8RkQ1ltdstL2OMMV5hCcUYY4xXWEIxxhjjFZZQjDHGeIUlFGOMMV7h04QiIr1FZKWIZInI6DLWh4vITGf9XBGJK7HuIad9pYj0KtE+SUR2isiSUmNFi8gXIrLa+beeL4/NGGPMb/ksoYhIMJ5Xh/YBEoABzqtKSxoO5KhqW2AsnsqrR19p2h/oBPQGxjvjAUx22kobDXylqvHAV85nY4wxlcSXVyg9gSxVXeu8Q2IGUPq9Cn2BKc7ybOAiERGnfYaq5qvqOjzv7+4JoKrf4Sn7XVrJsabgeS+2MX5n055DfLBgC1YJ3PgbX/5hYzNKvJYUz/sZSr9X4X99VLVQRHLxvGmuGZ73MJTcttlx9tdIVbc5y9vxvLL0d0TkVuBWgBYtWhz/KIypRLsO5HPZi9+zL6+QA/mFDDytpdshGXPCAvKhvHp+tSvz1ztVnaCqKaqaEhPzu8oBxrgmr6CIW99IJ6+wmKiaoTyeuoxFm/e6HZYxJ8yXCWULJd5zDcQ6bWX2cV63GgnsPsFtS9shIk2csZoAO086cmMqWXGx8ue3F5K5cS8v3NCdOfedT4PaYYyYmsneQ0fcDs+YE+LLhJIGxItIK+c91/3xvFq0pFRgiLPcD5jjXF2kAv2dWWCtgHhg3nH2V3KsIcAHXjgGYyrF2C9X8eGibYzu04E+XZoQXSuM8YOS2bk/j3tmLqC42J6nmKrPZwlFVQuBUcBnwHJglqouFZEnRORKp9tEoL6IZAH34szMct6RPQtYBnwKjFTVIgAReQv4GWgvIptFZLgz1tPAJSKyGrjY+WxMlTc7YzMvzcmif4/m3HZu6/+1d28exaOXJ/D1ymzGf5PlYoTGnJhq/U75lJQUtWrDxk0/r9nNTZPm0rNVNJNv7klo8G9/x1NV7pqxgA8XbeXN4adxVtsGLkVqzK9EJENVU0q3B+RDeWP8wZrsA9w+NYOW9WsxfmDy75IJgIjw1DVdaB1Tmzvfms/23DwXIjXmxFhCMcYFew4eYdjkNEKChNeH9iCyRugx+9YKD+HVQUkcLihi5PRMCoqKKzFSY06cJRRjKll+YRG3vZnOttw8JtyUQvPomsfdpm3DOoy5tisZG3J46uMVlRClMRVnCcWYSqSqjH5nMWnrc3j2um4ktzzxknNXdGvK0DPjmPTjOj5atO34GxhTySyhGFOJXvwqi/fmb+HPf2jHFd2aVnj7hy/tSGKLKB6YvZCsnQd8EKExJ88SijGV5IMFWxj75SquTYpl5AVtT2qMsJAgxt2YRHhoMHdMy+DQkUIvR2nMybOEYkwlSFu/h/vfXsRpraJ56poueGqgnpymUTV4oX93Vu88wMPvLrYikqbKsIRijI+t33WQW99IJ7ZeDV4bnExYyKl/250TH8M9F7fj/QVbmTp3oxeiNObUWUIxxodyDxUwbHIaCkwa2oOommFeG3vUBW05v30M//ffZSzcZEUkjfssoRjjI0cKi7ltajqbcw4zYXAKcQ1qeXX8oCBh7PXdiakTzh3TMsk5aEUkjbssoRjjA6rKw+8t5pe1exjTrws9W0X7ZD/1aoUxfmAS2fvzuWeWFZE07rKEYowPjP9mDbMzNnPXRfFcnRjr0311ax7Fo1ck8M3KbF7+2opIGvdYQjHGyz5ctJVnPltJ3+5Nufvi+ErZ56DTWnBV96aM/XIV36/OrpR9GlOaJRRjvChzYw73zlpISst6jLm26ylND64IEeEf13QhvmFt7pqxgK17D1fKfo0pyRKKMV6yac8hbpmSTpPICCbclEJEaHCl7r9mWAivDEom3ykieaTQikiaymUJxRgvyD1cwM2T0ygoKmbS0B5E1/Le9OCKaBNTm3/268b8jXv5x8fLXYnBVF+WUIw5RQVFxYyclsn6XQd5dXAybWJquxrPZV2bcPNZcUz+aT3/XbjV1VhM9WIJxZhToKo89sESfsjaxVPXdOHMNlXjjYoP9elIUosoRr+zyIpImkpjCcWYU/Dv79fy1rxNjLygDdelNHc7nP8JCwli3EBPEckRUzM4mG9FJI3vWUIx5iR9umQ7T32ygsu6NuG+S9q7Hc7vNImswYv9E8nKPsDD71kRSeN7llCMOQkLN+3l7pnz6d48imev60ZQUOVMD66os+MbcO/F7fhgwVam/rLB7XBMgLOEYkwFbdl7mD++kU6D2uH824XpwRU18oK2XNA+hic+XMYCKyJpfMgSijEVsD+vgOGT08g7UsTrQ3vQoHa42yEdV1CQMPaG7jSsE8FIKyJpfMgSijEnqLComFHT57N65wHGD0oivlEdt0M6YVE1w3hlkKeI5N0zrYik8Q1LKMacAFXl8f8u49tV2Tx5VWfOiY9xO6QK6xobxWNXJPDtqmxemmNFJI33WUIx5gRM+nE9b/6ygdvObc2Ani3cDuekDTytBVcnNuP5r1bx3SorImm8yxKKMcfx5bIdPPnRMnp3asyDvTu4Hc4pERH+fnVnp4jkfCsiabzKEoox5ViyJZc7Z8ynS7NIxt7QvcpOD66Io0UkC4qUO6ZZEUnjPZZQjDmGbbmHGT4ljagaofznphRqhFXt6cEV4Ski2ZUFm6yIpPEeSyjGlOFgfiHDJ6dzML+ISTf3oGHdCLdD8rpLuzRh+NmtmPzTelKtiKTxAksoxpRSVKzcNWM+K7bv46UbE+nQuK7bIfnM6D4dSGlZj9HvLGL1jv1uh2P8nE8Tioj0FpGVIpIlIqPLWB8uIjOd9XNFJK7Euoec9pUi0ut4Y4rIRSKSKSILROQHEWnry2MzgevJj5bx5fKdPH5lJy5o39DtcHwqNDiIl29MomZYMCOmZVoRSXNKfJZQRCQYGAf0ARKAASKSUKrbcCBHVdsCY4ExzrYJQH+gE9AbGC8iwccZ8xVgoKp2B6YDf/HVsZnA9cbP63n9x/UMO6sVg8+IczucStE4MoIX+yeyNvsAo9+1IpLm5PnyCqUnkKWqa1X1CDAD6FuqT19girM8G7hIPC/h7gvMUNV8VV0HZDnjlTemAkfvTUQCdlPYVMjXK3byt9SlXNyxIY9c1tHtcCrVmW0bcN8f2vPfhVt542crImlOTogPx24GbCrxeTNw2rH6qGqhiOQC9Z32X0pt28xZPtaYfwQ+FpHDwD7g9LKCEpFbgVsBWrTw3z9QM961fNs+Rk3PpGOTurzQP5HgAJgeXFEjzmtD5oYcnvxoGV1iI0lqUc/tkIyfCaSH8vcAl6pqLPA68FxZnVR1gqqmqGpKTIz/lc8w3rdzXx7DJ6dRJyKUiUN6UCvcl79nVV1BQcJz13enUd0IRk3LZI8VkTQV5MuEsgUo+Qq7WKetzD4iEoLnVtXucrYts11EYoBuqjrXaZ8JnOmdwzCB7NCRQoZPSWfv4QImDk2hcWTgTQ+uiMiaobwyMJldB45w14z5FFkRSVMBvkwoaUC8iLQSkTA8D9lTS/VJBYY4y/2AOep5IpgK9HdmgbUC4oF55YyZA0SKSDtnrEsA+2stU67iYuWemQtYujWXlwYk0qlppNshVQldYiP525Wd+H71Ll78arXb4Rg/4rNre+eZyCjgMyAYmKSqS0XkCSBdVVOBicCbIpIF7MGTIHD6zQKWAYXASFUtAihrTKf9FuAdESnGk2CG+erYTGB4+tMVfLZ0B49dnsBFHRu5HU6VMqBnc9I37OHFOatJbBHF+QE+fdp4h1TnKYIpKSmanp7udhjGBdPnbuTh9xZz0xktefzKTngmF5qSDh8p4urxP7J9Xx4f3XkOzaJquB2SqSJEJENVU0q3B9JDeWNOyPers3n0gyWc3z6Gxy5PsGRyDDXCghk/MIlCp4hkfmGR2yGZKs4SiqlWVu3Yzx1TM4lvWJuXBiQSEmzfAuVpHVObf13XlYWb9vL3j+yxpCmffTeZaiN7fz43v55GRFgwE4f2oE5EqNsh+YXenZvwx7Nb8cbPG/hgQemJmsb8yhKKqRbyCoq45Y10dh/MZ+KQFHseUEEP9ulAj7h6jH5nsRWRNMdkCcUEvOJi5b5ZC1m4eS8v9E+ka2yU2yH5naNFJGuFB3P71AwOWBFJUwZLKCbg/evzlXy0eBsP9elAr06N3Q7HbzWqG8GLAxJZt+sgo99ZZEUkze9YQjEBbVb6JsZ/s4YBPVtwyzmt3Q7H753ZxlNE8sNF25jy03q3wzFVjCUUE7B+WrOLh99dzDnxDXiir/2tibeMOK8NF3VoyN8/Xk7mxhy3wzFViCUUE5Cydh7g9jczaNWgFuMGJhFq04O95mgRycaREYyclsnuA/luh2SqCPsuMwFn94F8hk1OIywkiElDe1DXpgd73dEikrsPHuHumQusiKQBLKGYAJNXUMStb2awY18e/74phebRNd0OKWB1bhbJ404RyResiKTBEooJIKrKA7MXkbEhh+eu706ivSDK5/r3aE6/5FhemrOab1budDsc4zJLKCZgjP1yNakLt3J/r/Zc1rWJ2+FUCyLC//XtTPtGdbh75gI25xxyOyTjIksoJiC8m7mZF79azfUpsdxxfhu3w6lWaoQF8+qgZIqKlJFWRLJas4Ri/N7ctbt58J1FnNG6Pk9e1cWmB7sgrkEtnrmuGws35/Lkh1ZEsrqyhGL82rpdB7ltagbNo2vy6qBkwkLsf2m39O7cmFvPbc2bv2zg/flWRLI6su8+47dyDh5h2OQ0gkR4fWgPImva9GC3PdCrPT3jonno3cWssiKS1Y4lFOOX8guLuG1qBltyDjNhcDIt69dyOyQDhAQH8fKNidQKD7EiktWQJRTjd1SVh95dzLx1e3jmuq6kxEW7HZIpoWHdCF4akMj6XQd5cLYVkaxOLKEYv/PynCzezdzCPRe3o2/3Zm6HY8pwRpv63N+rAx8t3sbrP653OxxTSSyhGL/ywYItPPvFKq5JbMadF7V1OxxTjtvPa83FHRvxj4+Xk7Fhj9vhmEpgCcX4jYwNe7h/9iJ6xkXz1LU2PbiqExGevb4bTaNqMHLafHZZEcmAZwnF+IWNuw9xyxsZNI2M4LXByYSHBLsdkjkBkTVCGT8wiT2HjnDXjPlWRDLAWUIxVV7uoQJunjyPYlUmDe1BvVphbodkKqBzs0j+r28nfszazfNfrnI7HONDllBMlXaksJgR0zLYuOcQrw1KpnVMbbdDMifhhh4tuC45lpfmZPH1CisiGagsoZgqS1V59P0l/LRmN09f05XTWtd3OyRzCv7vqs50bFKXu2cuYNMeKyIZiCyhmCrr1W/XMjN9E3+6sC3XJse6HY45RRGhwbwyMIniYmXkdCsiGYgsoZgq6ePF2xjz6Qqu6NaUey9p53Y4xkviGtTiX9d3Y9HmXJ747zK3wzFeZgnFVDnzN+Zwz8wFJLWI4pl+XW16cIDp1akxt53bmmlzN/Le/M1uh2O8yBKKqVI27TnELW+k07BuOP++KYWIUJseHIju79Wenq08RSRXbrcikoHCpwlFRHqLyEoRyRKR0WWsDxeRmc76uSISV2LdQ077ShHpdbwxxePvIrJKRJaLyJ2+PDbjffvyChg+JY38wmJeH9qD+rXD3Q7J+EhIcBAvD0ikdngoI6ZmsD+vwO2QjBf4LKGISDAwDugDJAADRCShVLfhQI6qtgXGAmOcbROA/kAnoDcwXkSCjzPmUKA50EFVOwIzfHVsxvsKiooZOS2TtdkHeW1QMm0b1nE7JONjDetG8PKNiWzYc4gH37EikoHAl1coPYEsVV2rqkfw/IDvW6pPX2CKszwbuEg8N8z7AjNUNV9V1wFZznjljTkCeEJViwFU1Sa7+wlV5a+pS/l+9S7+fnVnzmzbwO2QTCU5vXV97u/Vno8Xb2eSFZH0e75MKM2ATSU+b3bayuyjqoVALlC/nG3LG7MNcIOIpIvIJyISX1ZQInKr0yc9Ozv7pA7MeNfEH9Yxfe5Gbj+vDTf0aOF2OKaS3XZuay5JaMRTHy8nfb0VkfRngfRQPhzIU9UU4N/ApLI6qeoEVU1R1ZSYmJhKDdD83mdLt/P3j5dzaZfGPNCrvdvhGBeICP+6rhvN6tVg5PRMKyLpx3yZULbgeaZxVKzTVmYfEQkBIoHd5Wxb3pibgXed5feArqd8BManFm/O5e4ZC+gaG8Vz13cnKMimB1dXkTVCeWVgMnsPFVgRST/my4SSBsSLSCsRCcPzkD21VJ9UYIiz3A+Yo54nc6lAf2cWWCsgHph3nDHfBy5wls8DrApdFbZ172GGT0kjulYY/74p2aYHGxKa1uX/rurMj1m7GfuFffv6oxBfDayqhSIyCvgMCAYmqepSEXkCSFfVVGAi8KaIZAF78CQInH6zgGVAITBSVYsAyhrT2eXTwDQRuQc4APzRV8dmTs2B/EKGTU7j8JEi3hxxGg3rRLgdkqkirk9pTsb6HF7+OoukllFc2KGR2yGZCpDqPFUvJSVF09PT3Q6jWiksKuaWN9L5bvUuXh/ag3Pb2XMs81t5BUVcM/4ntuw9zId/Opvm0TXdDsmUIiIZzvPq3wikh/LGDzz50XK+XpnNE307WTIxZYoIDebVQckUq3LHtEzyCqyIpL+whGIqzeQf1zH5p/X88exWDDytpdvhmCqsRf2aPHd9dxZvyeWJD62IpL+whGIqxZwVO3jiw2VcktCIhy7t6HY4xg9cktCI289rw/S5G3knw4pI+gNLKMbnlm7NZdT0+SQ0rcsL/bsTbNODzQn68x/acXrraB55fzErtu9zOxxzHJZQjE/t2JfH8MnpRNYIZeKQHtQM89nEQhOAQoKDeHFAInUjQhkxNZN9VkSySrOEYnzm0JFChk9JY39eAROH9KBRXZsebCquYZ0IXr4xiY17DvHA21ZEsiqzhGJ8oqhYufOtBSzbuo+Xb0wioWldt0Myfqxnq2ge7N2eT5duZ+IP69wOxxyDJRTjE099vJwvl+/gr1d04oIODd0OxwSAW85pTa9OjXjqkxWkWRHJKskSivG6qb9s4D8/rGPomXEMOTPO7XBMgBARnrmuG83r1WDktEyy91sRyarGEorxqm9XZfPX1KVc2KEhj15e+n1qxpyauhGhjB+YTO7hAu58az6FRcVuh2RKsIRivGbF9n2MnJZJu0Z1eHFAok0PNj6R0LQuT17VmZ/X7uY5KyJZpVhCMV6xc79nenDNsGAmDU2hdrhNDza+c11Kc/r3aM74b9bw5bIdbodjHJZQzCk7fKSIW6aks+fgESYO6UGTyBpuh2Sqgb9d2YlOTety76wFbNx9yO1wDCeQUEQkWERWVEYwxv8UFyv3zlrAoi25vNC/O11iI90OyVQTEaHBvDIwGYA7pmdYEckq4LgJxXkPyUoRsZd9m98Z89kKPlmynUcu7cgfOjV2OxxTzRwtIrlkyz4e/+/S429gfOpEb3TXA5aKyDzg4NFGVb3SJ1EZvzBj3kZe+3YtA09rwfCzW7kdjqmmLk5oxIjz2/DKN2tIbhlNv+RYt0Oqtk40oTzq0yiM3/kxaxd/eX8J57aL4fErOyFiM7qMe+67pB0LNu7lkfcW06lpXTo2scoMbjihh/Kq+m1ZX74OzlRNq3fs5/apGbSJqc24GxMJCba5HcZdR4tIRtYIZcTUDCsi6ZJyfxKIyH4R2VfG134RsVrS1dCuA/kMm5JGeEgwE4emUCci1O2QjAEgpk444wYmsSnnsBWRdEm5CUVV66hq3TK+6qiqXVNWM3kFRdz6RjrZ+/P5z5AUYuvZu75N1dIjLpqH+nTg06Xb+c/3VkSystm9CnNCiouVP7+9kMyNexl7fXe6N49yOyRjyjT87Fb06dyYpz9dwbx1VkSyMllCMSfkuS9W8eGibYzu04E+XZq4HY4xxyQi/LNfV1pE12TU9Ex27s9zO6RqwxKKOa7ZGZt5+essbkhpzm3ntnY7HGOOq05EKK8MSmJfnhWRrEyWUEy5fl6zm4feXcRZbevz5NWdbXqw8RsdGtfl71d14Ze1e3jWikhWCkso5pjWZB/g9qkZtIiuyfiByYTa9GDjZ65NjmVAzxa88s0avrAikj5nPyFMmfYcPMKwyWmEBAmvD+1JZA2bHmz801+vSKBzMysiWRksoZjfyS8s4rY309mWm8eEm1JoUd+mBxv/dbSIZJAII6ZZEUlfsoRifkNVGf3OYtLW5/Dsdd1IblnP7ZCMOWXNo2sy9oZuLN26j7+lWhFJX7GEYn7jha9W8978Lfz5D+24oltTt8Mxxmsu7NCIkRe0YUbaJmalb3I7nIBkCcX8z/vzt/D8l6u5NimWkRe0dTscY7zu3kvac2ab+jz6/hKWbs11O5yAYwnFAJC2fg8PzF7Eaa2ieeqaLjY92ASk4CDhxQGJRNUM5Y5pmeQetiKS3uTThCIivUVkpYhkicjoMtaHi8hMZ/1cEYkrse4hp32liPSqwJgvisgBXx1TIFq/6yC3vpFOs3o1eG1wMmEh9nuGCVwNaocz7sYktuQc5v63F1oRSS/y2U8OEQkGxgF9gARggIgklOo2HMhR1bbAWGCMs20C0B/oBPQGxjuvIi53TBFJwfMyMHOCcg8VMGxyGgpMGtqDqJphbodkjM+lxEUzuk8HPl+2gwnfrXU7nIDhy19FewJZqrpWVY8AM4C+pfr0BaY4y7OBi8Rzr6UvMENV81V1HZDljHfMMZ1k8wzwgA+PKaAcKSzmtqnpbM45zITBKbRqUMvtkIypNMPPbsWlXRrzz89WMnftbrfDCQi+TCjNgJJTKTY7bWX2UdVCIBeoX8625Y05CkhV1W3lBSUit4pIuoikZ2dnV+iAAomq8vB7i/ll7R7G9OtCz1bRbodkTKUSEcZc25WW0TUZ9dZ8du6zIpKnKiBulotIU+A64KXj9VXVCaqaoqopMTExvg+uihr/zRpmZ2zmroviuTrR3sFtqqc6EaGMH5TE/rwCRlkRyVPmy4SyBWhe4nOs01ZmHxEJASKB3eVse6z2RKAtkCUi64GaIpLlrQMJNB8u2sozn62kb/em3H1xvNvhGOOqDo3r8o+ruzBv3R6e+Xyl2+H4NV8mlDQgXkRaiUgYnofsqaX6pAJDnOV+wBz1TLlIBfo7s8BaAfHAvGONqaofqWpjVY1T1TjgkPOg35SSuTGHe2ctJKVlPcZc29WmBxsDXJMUy42nteC1b9fy+dLtbofjt0J8NbCqForIKOAzIBiYpKpLReQJIF1VU4GJwJvO1cQePAkCp98sYBlQCIxU1SKAssb01TEEmk17DnHLlHQa141gwk0pRIQGux2SMVXGY5cnsHhzLve9vZAPG9ehZX2bpFJRUp3nYKekpGh6errbYVSK3MMFXPvKT+zcl8e7d5xF24a13Q7JmCpn055DXP7SDzSNqsF7d5xpv3Qdg4hkqGpK6faAeChvyldQVMzIaZms33WQVwcnWzIx5hiOFpFcvm0fj32wxO1w/I4llACnqjz2wRJ+yNrFU9d04cw2DdwOyZgq7cIOjfjThW2Zlb6ZWWlWRLIiLKEEuAnfreWteZsYeUEbrktpfvwNjDHcfXE7zm7bgEc/sCKSFWEJJYB9umQbT3+6gsu6NOG+S9q7HY4xfiM4SHihf3fq1QxjxFQrInmiLKEEqIWb9nL3zAV0bx7Fs9d3IyjIpgcbUxH1a4czbmASW/ce5s9WRPKEWEIJQFv2HuaPb6TToFTWtVIAABOYSURBVHY4Ewbb9GBjTlZyy3o8fGlHvli2g9esiORxWUIJMPvzChj2ehp5R4p4fWgPYuqEux2SMX7t5rPiuKxrE/756Qp+sSKS5bKEEkAKi4oZNX0+WdkHGD8oifhGddwOyRi/d7SIZFyDWoyabkUky2MJJUCoKo//dxnfrsrmyas6c0589S18aYy31Q4P4dVByRzML7QikuWwhBIgJv24njd/2cBt57ZmQM8WbodjTMBp16gOT13jFJH8zIpIlsUSSgD4ctkOnvxoGb06NeLB3h3cDseYgHVVYjMGnd6C175by2dWRPJ3LKH4uSVbcrlzxny6NIvk+RsSbXqwMT726OUJdIuN5M+zFrJ+10G3w6lSLKH4sW25hxk+JY2oGqH856YUaoTZ9GBjfC08JJhxA5MIDhZGTMskr6DI7ZCqDEsofupgfiHDJ6dzML+IiUN70LBuhNshGVNtxNarydgburNi+z4efd+KSB5lCcUPFRUrd741nxXb9/HSjYl0bFLX7ZCMqXYuaN+QP13QlrczNjMzbaPb4VQJllD80JMfLeOrFTt5/MpOXNC+odvhGFNt3XVxO86Jb8CjHyxlyRYrImkJxc+88fN6Xv9xPcPOasXgM+LcDseYai04SHj+hu7UrxXGiGkZ5B6q3kUkLaH4ka9X7ORvqUu5uGNDHrmso9vhGGPwFJF8+cYktu3N4763F1BcXH2LSFpC8RPLt+1j1PRMOjapywv9Ewm26cHGVBnJLevxyGUd+XL5Tl79bo3b4bjGEoof2Lkvj+GT06gTEcrEIT2oFR7idkjGmFKGnukpIvmvz1by05pdbofjCksoVdyhI4UMn5LO3sMF/GdICo0jbXqwMVXR0SKSrRrU4s635rOjGhaRtIRShRUXK3fPWMDSrbm82D+Rzs0i3Q7JGFOO2uEhvDIomYP5RYyanklBNSsiaQmlCnv60xV8vmwHf7ksgYsTGrkdjjHmBLRrVIenr+1C2voc/vnpCrfDqVSWUKqo6XM3MuG7tdx0RktuPivO7XCMMRXQt3szBp/ekn9/v45Pl2xzO5xKYwmlCvpuVTaPfrCE89vH8NjlCYjYjC5j/M1fLu9It+ZR3P/2ItZVkyKSllCqmFU79jNyWibxDWvz0oBEQoLtP5Ex/ig8JJjxA5MICRZGTM3g8JHALyJpP62qkOz9+dz8ehoRYcFMHNqDOhGhbodkjDkFzaJq8Hz/RFbu2M9f3l+CamD/0aMllCoir6CIW95IZ/fBfCYOSaFZVA23QzLGeMF57WK488J43snczIy0TW6H41OWUKqA4mLlvlkLWbh5L8/fkEjX2Ci3QzLGeNGdF8VzTnwD/poa2EUkLaFUAf/6fCUfLd7GQ3060LtzY7fDMcZ4WXCQ8EL/RBrUCuP2qYFbRNKnCUVEeovIShHJEpHRZawPF5GZzvq5IhJXYt1DTvtKEel1vDFFZJrTvkREJomIXzyAmJW+ifHfrGFAzxbcck5rt8MxxvhIdK0wxg1MYse+PO6dFZhFJH2WUEQkGBgH9AESgAEiklCq23AgR1XbAmOBMc62CUB/oBPQGxgvIsHHGXMa0AHoAtQA/uirY/OWn7J28fC7izknvgFP9O1k04ONCXCJLerxl8sS+GrFTl75NvCKSPryCqUnkKWqa1X1CDAD6FuqT19girM8G7hIPD9V+wIzVDVfVdcBWc54xxxTVT9WBzAPiPXhsZ2yrJ0HuH1qBq0a1GLcwCRCbXqwMdXCTWe05IpuTXn288ArIunLn2LNgJJTGjY7bWX2UdVCIBeoX862xx3TudU1GPj0lI/AR3YfyGfY5DTCQoKYNLQHdW16sDHVhojw9DVdaB1Tmzvfms/23MApIhmIvxaPB75T1e/LWikit4pIuoikZ2dnV3JonunBt76ZwY59eUy4KYXm0TUrPQZjjLtqhYfw6qAkDh0JrCKSvkwoW4DmJT7HOm1l9hGRECAS2F3OtuWOKSJ/BWKAe48VlKpOUNUUVU2JiYmp4CGdGlXlgdmLyNiQw3PXdyepRb1K3b8xpupo27AOT1/blfQNOYz5JDCKSPoyoaQB8SLSSkTC8DxkTy3VJxUY4iz3A+Y4z0BSgf7OLLBWQDye5yLHHFNE/gj0AgaoapVM92O/XE3qwq3c36s9l3Vt4nY4xhiXXdmtKUPOaMl/fljHJ4v9v4ikz179p6qFIjIK+AwIBiap6lIReQJIV9VUYCLwpohkAXvwJAicfrOAZUAhMFJViwDKGtPZ5avABuBnZ7bUu6r6hK+Or6LezdzMi1+t5rrkWO44v43b4RhjqohHLktg4eZc7p+9iPaN69A6prbbIZ00CfTaMuVJSUnR9PR0n+9n7trdDJo4l5SW0UwZ1pOwkEB8dGWMOVlb9h7m8he/p1HdCN674yxqhAW7HVK5RCRDVVNKt9tPNh9bt+sgt03NoHl0TV4dlGzJxBjzO82iavCCU0TykfcX+20RSfvp5kM5B48wbHIaQSK8PrQHkTVterAxpmzntovhrovieTdzC2/N888ikpZQfCS/sIjbpmawJecwEwYn07J+LbdDMsZUcXdeGM+57WL4W+pSFm3e63Y4FWYJxQdUlYfeXcy8dXt45rqupMRFux2SMcYPBAUJz9/QnQa1wxgxNZO9h464HVKFWELxgZfnZPFu5hbuubgdfbuXLg5gjDHHFl0rjPGDktm5P497ZvpXEUlLKF72wYItPPvFKq5ObMadF7V1OxxjjB/q3jyKRy9P4OuV2Yz/JsvtcE6YJRQvytiwh/tnL6JnXDRPX9vFqgcbY07a4NNbcmW3pjz3xSp+zPKPIpKWULxk4+5D3PJGBk0jI3htcDLhIVV7HrkxpmoTEZ7ysyKSllC8IPdQATdPnkexKpOG9qBerTC3QzLGBABPEclk8gqKGOkHRSQtoZyiI4XFjJiWwcY9h3h1ULJfl00wxlQ9bRvWZky/rmRsyOGpj6t2EUlLKKdAVfnL+4v5ac1unr6mK6e3ru92SMaYAHR516YMPTOOST+u46NFVbeIpCWUU/Dqt2uZlb6ZP13YlmuTq/QLIo0xfu7hSzuS1CKKB2YvZE32AbfDKZMllJP08eJtjPl0BVd0a8q9l7RzOxxjTIALCwli3MAkwkODGTE1g0NHCt0O6XcsoZyE+RtzuGfmApJaRPFMv642PdgYUymaRNbghf7dWb3zAI+8t6TKFZG0hFJBqso/Pl5Ow7rh/PumFCJCbXqwMabynBMfwz0Xt+O9+VuYNnej2+H8hs9esBWoRITXBqeQe7iA+rXD3Q7HGFMNjbqgLZkbc3jiv8voGhtJ19got0MC7ArlpETXCqNVA6sebIxxR1CQMPb67sTUCWfE1ExyDlaNIpKWUIwxxg/VqxXG+IFJZO/P555ZVaOIpCUUY4zxU92aR/HoFQl8szKbcV+7X0TSEooxxvixQae14KruTXnuy1X8sNrdIpKWUIwxxo+JCP+4pgvxDWtz54z5bMs97FosllCMMcbP1QwL4ZVByeQXFDFyWiZHCt0pImkJxRhjAkCbmNr8s183Mjfu5alPlrsSgyUUY4wJEJd1bcLNZ8Xx+o/r+XDR1krfvyUUY4wJIA/18RSRfHD2IrJ2Vm4RSUsoxhgTQEoWkbxjWuUWkbSEYowxAaZJZA1e7J/I6p0HePjdxZVWRNISijHGBKCz4xtw78XteH/BVqZWUhFJSyjGGBOgRl7Qlgvax/B//13Ggk17fb4/SyjGGBOggoKEsTd4ikiOnOb7IpKWUIwxJoBF1QzjlUGeIpJ3z/RtEUmfJhQR6S0iK0UkS0RGl7E+XERmOuvnikhciXUPOe0rRaTX8cYUkVbOGFnOmGG+PDZjjPEXXWOj+OuVCXy7KpuX5viuiKTPEoqIBAPjgD5AAjBARBJKdRsO5KhqW2AsMMbZNgHoD3QCegPjRST4OGOOAcY6Y+U4YxtjjAFu7NmCaxKb8fxXq/huVbZP9uHLK5SeQJaqrlXVI8AMoG+pPn2BKc7ybOAi8bygvS8wQ1XzVXUdkOWMV+aYzjYXOmPgjHmVD4/NGGP8iojw96u70K5hHe6aMZ+te71fRNKXCaUZsKnE581OW5l9VLUQyAXql7PtsdrrA3udMY61LwBE5FYRSReR9Oxs32RpY4ypimqEBfPKoCS6xEYRJOL18avdQ3lVnaCqKaqaEhMT43Y4xhhTqVrH1OaNYT1pHBnh9bF9mVC2AM1LfI512srsIyIhQCSwu5xtj9W+G4hyxjjWvowxxviQLxNKGhDvzL4Kw/OQPbVUn1RgiLPcD5ijnhoBqUB/ZxZYKyAemHesMZ1tvnbGwBnzAx8emzHGmFJCjt/l5KhqoYiMAj4DgoFJqrpURJ4A0lU1FZgIvCkiWcAePAkCp98sYBlQCIxU1SKAssZ0dvkgMENEngTmO2MbY4ypJFJZRcOqopSUFE1PT3c7DGOM8SsikqGqKaXbq91DeWOMMb5hCcUYY4xXWEIxxhjjFZZQjDHGeEW1figvItnAhpPcvAGwy4vh+Ds7H79l5+O37Hz8KhDORUtV/d1fhlfrhHIqRCS9rFkO1ZWdj9+y8/Fbdj5+Fcjnwm55GWOM8QpLKMYYY7zCEsrJm+B2AFWMnY/fsvPxW3Y+fhWw58KeoRhjjPEKu0IxxhjjFZZQjDHGeIUllJMgIr1FZKWIZInIaLfj8QURmSQiO0VkSYm2aBH5QkRWO//Wc9pFRF50zsciEUkqsc0Qp/9qERlS1r78gYg0F5GvRWSZiCwVkbuc9mp5TkQkQkTmichC53w87rS3EpG5znHPdF4zgfMqiplO+1wRiSsx1kNO+0oR6eXOEZ06EQkWkfki8qHzufqdC1W1rwp84SmbvwZoDYQBC4EEt+PywXGeCyQBS0q0/RMY7SyPBsY4y5cCnwACnA7MddqjgbXOv/Wc5XpuH9tJno8mQJKzXAdYBSRU13PiHFdtZzkUmOsc5yygv9P+KjDCWb4DeNVZ7g/MdJYTnO+hcKCV870V7PbxneQ5uReYDnzofK5258KuUCquJ5ClqmtV9QgwA+jrckxep6rf4XlHTUl9gSnO8hTgqhLtb6jHL3jentkE6AV8oap7VDUH+ALo7fvovU9Vt6lqprO8H1gONKOanhPnuA44H0OdLwUuBGY77aXPx9HzNBu4SETEaZ+hqvmqug7IwvM95ldEJBa4DPiP81mohufCEkrFNQM2lfi82WmrDhqp6jZneTvQyFk+1jkJyHPl3KJIxPNbebU9J84tngXATjyJcQ2wV1ULnS4lj+1/x+2szwXqEzjn43ngAaDY+VyfanguLKGYk6Kea/RqN+dcRGoD7wB3q+q+kuuq2zlR1SJV7Q7E4vlNuoPLIblCRC4HdqpqhtuxuM0SSsVtAZqX+BzrtFUHO5zbNjj/7nTaj3VOAupciUgonmQyTVXfdZqr9TkBUNW9wNfAGXhu7R19tXjJY/vfcTvrI4HdBMb5OAu4UkTW47kFfiHwAtXwXFhCqbg0IN6ZwRGG56FaqssxVZZU4OispCHAByXab3JmNp0O5Dq3gT4D/iAi9ZzZT39w2vyOc497IrBcVZ8rsapanhMRiRGRKGe5BnAJnudKXwP9nG6lz8fR89QPmONc0aUC/Z2ZT62AeGBe5RyFd6jqQ6oaq6pxeH4ezFHVgVTDc+H6rAB//MIzg2cVnnvGj7gdj4+O8S1gG1CA517ucDz3eb8CVgNfAtFOXwHGOedjMZBSYpxheB4uZgE3u31cp3A+zsZzO2sRsMD5urS6nhOgKzDfOR9LgMec9tZ4fghmAW8D4U57hPM5y1nfusRYjzjnaSXQx+1jO8Xzcj6/zvKqdufCSq8YY4zxCrvlZYwxxissoRhjjPEKSyjGGGO8whKKMcYYr7CEYowxxissoRhzkkTkJ+ffOBG50ctjP1zWvoypymzasDGnSETOB/6sqpdXYJsQ/bXOU1nrD6hqbW/EZ0xlsSsUY06SiByttvs0cI6ILBCRe5yiic+ISJrzLpTbnP7ni8j3IpIKLHPa3heRDOedIrc6bU8DNZzxppXcl/OX98+IyBIRWSwiN5QY+xsRmS0iK0RkmvPX/YjI0+J5j8siEflXZZ4jU72EHL+LMeY4RlPiCsVJDLmq2kNEwoEfReRzp28S0Fk95ckBhqnqHqd8SZqIvKOqo0VklHoKL5Z2DdAd6AY0cLb5zlmXCHQCtgI/AmeJyHLgaqCDqurRcinG+IJdoRjjfX/AU8drAZ4S9/Xx1GUCmFcimQDcKSILgV/wFAaMp3xnA2+pp9LvDuBboEeJsTerajGe0jBxeEqj5wETReQa4NApH50xx2AJxRjvE+BPqtrd+WqlqkevUA7+r5Pn2cvFwBmq2g1PbayIU9hvfonlIuDoc5qeeF7kdDnw6SmMb0y5LKEYc+r243kt8FGfASOccveISDsRqVXGdpFAjqoeEpEOeF6he1TB0e1L+R64wXlOE4PnVc3HrEjrvL8lUlU/Bu7Bc6vMGJ+wZyjGnLpFQJFz62oynndhxAGZzoPxbH59/WtJnwK3O885VuK57XXUBGCRiGSqpxT6Ue/hee/IQjzVjx9Q1e1OQipLHeADEYnAc+V078kdojHHZ9OGjTHGeIXd8jLGGOMVllCMMcZ4hSUUY4wxXmEJxRhjjFdYQjHGGOMVllCMMcZ4hSUUY4wxXvH/c++bV5d/Q/MAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learner.plot('lr')" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3hUdfbH8fdJSOidUAOEDqGl2QtWROwond/adl0pNmyw1tXF3mmuu+66SqiKiBVRsTfSqYHQi0AooUNCcn5/zI2OOMAAmdzJzHk9zzyZ+d57537mKjm5Zc4VVcUYY4w5XITbAYwxxgQnKxDGGGN8sgJhjDHGJysQxhhjfLICYYwxxicrEMYYY3yyAmFMBScib4jIP8rovW4QkW/Lel5TMVmBMOVGRFaLSKGINDhsPFNEVETi3EnmW1n+4vVzfbEi8o6IbBWRnSKyUERuKK/1G3M4KxCmvK0CBpa+EJGuQDX34gSVt4B1QEugPvB/wGZXE5mwZgXClLe3gD95vb4eeNN7BhGpLSJviki+iKwRkQdFJMKZdoOIfCciL4pIgYisFJEznfF1IrJFRK73eq/KIvKciKwVkc0i8qqIVHWmnSci60Xkbme5X0TkRmfaLcBg4D4R2SMi7zvjKiJtvd7/170Mr/e7z+v9rhaR3iKyTES2i8jfjrJtTgHeUNW9qnpIVTNV9WOvdZ0tIt87n3vdYXsXdUXkQxHZLSI/iUgbr+U6ishcZ/25ItLPa1p9EZktIrtE5GfAe7k45/NW8hr7UkT+7Cv80dZjKiYrEKa8/QjUEpFOIhIJDAAmHTbPWKA20Brogaeg3Og1/TQgB89f2ZOBqXh+ubYFhgDjRKSGM+9TQHsgwZneDHjY670aO+tqBtwMjBeRuqr6GpAKPKOqNVT1Cj8/X2Ogitd6/uVkSgbOAR4SkVZHWPZHZ/0DRKSF9wQRaQl87GybGOfzZHnNMgD4O1AXyAPGOMtVB+bi2U4NnfkmiEi8s9x44ADQBLjJeRw3P9ZjKiArEMYNpXsRFwNLgA2lE7yKxmhV3a2qq4Hn8RxuKbVKVf+rqsXANKA58JiqHlTVT4FCoK2ICHALcJeqblfV3cATzvuXKnKWLVLVj4A9QIeT+GxFwBhVLcJTuBoALzufZRGwGOh+hGX7At8ADwGrRCRLRE5xpg0CPlPVKU7WbarqXSDeVdWfVfUQnsKW4IxfDqx2ttchVc0E3gH6Otv6WuBhZ69lIfC/E/zcR1zPCb6fCQKVjj2LMWXuLeBroBWHHV7C8ws1CljjNbYGz1/kpbyPy+8HUNXDx2rg+Uu7GpDuqRUACBDpNe8255dqqX3Osidqm1O4fs3mI6/P91fVHcAoYJRzIv85YJaIxOIpgiuOst5NXs+9P0NL4DQRKfCaXgnPf4MY5/k6r2ne2/14HG09poKyAmHKnaquEZFVQG88h3W8bcXzV3hLPH9tA7TAay/jOGzF8wu5s6qeyPK+Wh3v4/cn1RsD60/gvY++YtWtIvIcnnM09fD8Ej/1BN5qHfCVql58+ARnD+IQnuKz1Bn2PrS11/lZDdjlPG98vOsxFZcdYjJuuRm4QFX3eg86f31PB8aISE3n2PtI/nie4phUtQTPOYAXRaQhgIg0E5FL/HyLzXjOg3jLAgaJSKSI9MJzjqRMiMjTItJFRCqJSE1gKJCnqtvwHDa6SET6OdPri0jC0d8RgA+A9iLyfyIS5TxOEZFOzraeCTwqItWc8wW/nuBX1Xw8hXmI83lvwusktr/rOfEtYtxmBcK4QlVXqGraESbfhuev15XAt3hOfP7nBFd1P56Ttj+KyC7gM/w/x/A6EO9cNTTLGbsDuAIowHOV06wjLXwCqgHvOu+9Es9e1JUAqroWzx7X3cB2PIXqSOcyfuWcd+mJ57zLRjyHop4GKjuzjMBzOGoT8Abw38Pe4i/AvcA2oDPw/Qmux1RAYjcMMsYY44vtQRhjjPHJCoQxxhifrEAYY4zxyQqEMcYYn0LmexANGjTQuLg4t2MYY0yFkp6evlVVY3xNC5kCERcXR1raka6aNMYY44uIHPHb83aIyRhjjE9WIIwxxvhkBcIYY4xPViCMMcb4ZAXCGGOMT1YgjDHG+GQFwhhjjE9hXyBUlZkZ69l1oMjtKMYYc9y+X7GVjLU7AvLeYV8gVm7dy71v53DfjBys9bkxpiLJWlfAoH/9xPWv/0xJSdn//gr7AtEmpgajenXkk0WbeP3bVW7HMcYYv+zYW8jw1AyiK0Xw0R3nEBEhx17oOIV9gQD48zmt6NW5MU9+vJSfV213O44xxhxVSYly57Qs8ncf5O1bz6B5vWrHXugEWIEARIRn+najRb1qjJicwZbdB9yOZIwxRzT2izy+WpbPw1fE0y22TsDWYwXCUatKFBOHJLHrQBG3T8nkUHGJ25GMMeYPvl6Wz0ufL6NPYjMGn9YioOuyAuGlY+NajLm6Kz+u3M7zc5e5HccYY35nQ8F+7piaSfuGNRlzTVdEyv68gzcrEIe5NjmWgae2YOKXK5i7eLPbcYwxBoDCQyUMT82gqFiZOCSJqtGRAV+nFQgfHrkini7NajFyehZrt+1zO44xxjDmw8VkrSvgmeu60TqmRrms0wqED1WiIpk4OJkIEYampnOgqNjtSMaYMDY7eyP/+2ENN5/dit5dm5Tbeq1AHEHzetV4sX93Fm3cxaOzF7kdxxgTppZv3s2od3JIaVmXUZd2LNd1W4E4igs6NmL4+W2YOn8d09PWuR3HGBNm9h48xNDUDKpFRzJuUBJRkeX7K9sKxDGMvLgDZ7Wtz0OzFrJo40634xhjwoSqMmrmAlbm7+GVAYk0rl2l3DNYgTiGyAjh5QGJ1K0WzbDUDHbut6Z+xpjAe/OHNbyfvZG7e3bgzLYNXMlgBcIPDWpUZvzgRDbs2M+9M7KtqZ8xJqAy1u7gHx8u5sKODRnao41rOaxA+Cm5ZT1G9+7Ep4s389rXK92OY4wJUdv2HGR4agaNa1fhhX4JAWnC5y8rEMfhprPiuKxrE56Zk8tPK7e5HccYE2KKnSZ82/YWMnFwMrWrRbmaxwrEcRARnrq2Ky3rVWPElEy27LKmfsaYsvPy58v5ZvlW/n5lZ7o0q+12nMAWCBHpJSK5IpInIqN8TG8pIp+LSI6IfCkisYdNryUi60VkXCBzHo+aVaKYOCSZPQcOMcKa+hljysiXuVsY+8Vyrk2KZcApzd2OAwSwQIhIJDAeuBSIBwaKSPxhsz0HvKmq3YDHgCcPm/448HWgMp6oDo1r8kSfLvy8ajvPzsl1O44xpoJbv2Mfd07LokOjmvzj6i4Bb8Lnr0DuQZwK5KnqSlUtBKYCVx02TzzwhfN8nvd0EUkGGgGfBjDjCbsmMZYhp7fgn1+vZM6iTW7HMcZUUAcPFTM8NYPiYmXikORyacLnr0AWiGaA99eP1ztj3rKBPs7za4CaIlJfRCKA54F7jrYCEblFRNJEJC0/P7+MYvvvocvj6R5bm3umZ7N6695yX78xpuL7xwdLyF6/k2f7dqdVg+pux/kdt09S3wP0EJFMoAewASgGhgEfqer6oy2sqq+paoqqpsTExAQ+7WEqV4pk/OAkIiOFoakZ1tTPGHNcZmVu4K0f13DLua3p1aWx23H+IJAFYgPgfaYl1hn7lapuVNU+qpoIPOCMFQBnACNEZDWe8xR/EpGnApj1hMXWrcaL/RNYumkXD81a6HYcY0wFsWzzbkbPXMCpcfW475IObsfxKZAFYj7QTkRaiUg0MACY7T2DiDRwDicBjAb+A6Cqg1W1harG4dnLeFNV/3AVVLA4v0NDbju/LTPS1zNt/lq34xhjgtyeg4e4dVI61StXYtygRCqVcxM+fwUslaoeAkYAc4AlwHRVXSQij4nIlc5s5wG5IrIMzwnpMYHKE2h3XNSec9o14KH3FrFwgzX1M8b4pqrc/3YOq7fuZezARBrWKv8mfP6SUOkrlJKSomlpaa5m2LbnIJeP/ZZKkcIHI85x/VuQxpjg859vV/HYB4u5v1dHhp7nXp+lUiKSrqopvqYF535NBVW/RmXGDUril4ID3D0ji5KS0Ci+xpiykb5mO098tISLOjXi1h6t3Y5zTFYgylhyy7o8eFknPluyhVe/XuF2HGNMkNi65yDDUzNpWqcqz/frHjRfhjsaKxABcP2ZcVzRvSnPzcnl+xVb3Y5jjHFZcYlyx9RMduwrZOKQJGpXrRiHn61ABICI8FSfrrRqUJ3bp2Sy2Zr6GRPWXpy7jO/ytvH4VV3o3NT9Jnz+sgIRINUrV+LVIcnsKyxmxOQMiqypnzFh6Yulmxk3L49+KbH0C5ImfP6yAhFA7RrV5Mk+XZm/egfPfLLU7TjGmHK2bvs+7pqWTXyTWjx2VRe34xw3KxABdlVCM/50Rkv+9c0qPln4i9txjDHl5EBRMcNSMyhRZeKQJKpEBU8TPn9ZgSgHD1zWie7N63DPjBxW5u9xO44xphw89sFiFmzYyfN9u9OyfnA14fOXFYhyULlSJBMGJxEVKQxLzWB/oTX1MyaUzcxYz+Sf1vLXHq3p2Tn4mvD5ywpEOWlWpyovDUgkd/NuHpi1gFD5Brsx5veWbtrF395dwGmt6nFvz+BswucvKxDlqEf7GO64sB0zMzYw5ed1x17AGFOh7D5QxNBJGdSqEsXYIG7C56+Knb4Cuv2CdpzbPoZHZy9iwXpr6mdMqFBV7p2Rw9rt+xg3KImGNYO3CZ+/rECUs4gI4aX+CTSoEc3Q1HQK9hW6HckYUwZe/3YVnyzaxP29OnBqq3puxykTViBcUK96NBOGJLN51wFGTs+2pn7GVHDzV2/nyY+XcknnRvzlnOBvwucvKxAuSWheh4cuj+eLpVuY+JU19TOmosrffZDhqRk0r1uVZ/tWjCZ8/rIC4aL/O70lV3ZvyvOf5vJdnjX1M6aiOVRcwu1TMtm5v4gJg5OpVaViNOHzlxUIF4kIT/bpSuuYGtw+JZNNO62pnzEVyQtzl/HDym384+ouxDet5XacMmcFwmWlTf0OFBUz3Jr6GVNhfLZ4MxO+XMHAU5vTN6ViNeHzlxWIINC2YQ2evq4b6Wt28ORH1tTPmGC3dts+Rk7PokuzWjxyRWe34wSMFYggcXm3ptxwZhz/+W4VH+ZYUz9jgtWBomKGpqYDMHFwcoVswucvKxBB5G+9O5HUog73vZ3NCmvqZ0xQenT2IhZt3MWL/RNoXq+a23ECygpEEImuFMH4wUlUjopk6KR09hUecjuSMcbLjLR1TJ2/jmHnteHCTo3cjhNwViCCTJPaVXl5QALLt+zhbzOtqZ8xwWLxxl08OGshZ7Suz8iL27sdp1xYgQhC57SL4a6L2jMrayOTflrrdhxjwt6uA0UMS02ndtUoXhlY8Zvw+Ss8PmUFNOL8tpzXIYbH319M9roCt+MYE7ZUlXumZ7N+x34mDE4ipmZltyOVGysQQSoiQnixXwIxNSszLDWDHXutqZ8xbvjXNyv5dPFmRl3akZS40GjC5y8rEEGsbvVoJgxOIn/3Qe6anmVN/YwpZz+t3MbTn+TSu2tjbj67ldtxyp0ViCDXvXkdHr4ini9z8xk3L8/tOMaEjS27DjBiSiYt61Xj6Wu7hVQTPn9ZgagABp/WgmsSm/HiZ8v4Znm+23GMCXmHiksYMSWT3QeKmDAkiZoh1oTPX1YgKgARYcw1XWjXsAZ3TM1iY8F+tyMZE9Ke/TSXn1dt54lrutKxceg14fNXQAuEiPQSkVwRyRORUT6mtxSRz0UkR0S+FJFYZzxBRH4QkUXOtP6BzFkRVIuuxMQhyRQeKmFYagaFh6ypnzGB8OmiTfzzq5UMOq0FfZJi3Y7jqoAVCBGJBMYDlwLxwEARiT9stueAN1W1G/AY8KQzvg/4k6p2BnoBL4lInUBlrSjaxNTgmeu6kbWugCc+WuJ2HGNCzppte7l7RjZdm9Xm4csP/3UVfgK5B3EqkKeqK1W1EJgKXHXYPPHAF87zeaXTVXWZqi53nm8EtgAxAcxaYfTu2oSbzmrFG9+vZnb2RrfjGBMyDhQVc+ukDCJEmDA4KaSb8PkrkAWiGbDO6/V6Z8xbNtDHeX4NUFNE6nvPICKnAtGA3ZfTMbp3R5Jb1mXUOznkbdntdhxjQsLD7y1kyS+7eCkMmvD5y+2T1PcAPUQkE+gBbACKSyeKSBPgLeBGVf3DQXcRuUVE0kQkLT8/fK7uiYqMYPygJKpGRXLrpAz2HrSmfsacjGnz1zI9bT23XdCW8zs2dDtO0AhkgdgAeN9mKdYZ+5WqblTVPqqaCDzgjBUAiEgt4EPgAVX90dcKVPU1VU1R1ZSYmPA6AtW4dhXGDkxkZf4eRltTP2NO2MINO3novUWc3bYBd14UHk34/BXIAjEfaCcirUQkGhgAzPaeQUQaiEhphtHAf5zxaOBdPCew3w5gxgrtzLYNuLtnB2Znb+StH9e4HceYCmfn/iKGpWZQr1o0Lw9IIDIi/L4MdzQBKxCqeggYAcwBlgDTVXWRiDwmIlc6s50H5IrIMqARMMYZ7wecC9wgIlnOIyFQWSuyoT3acGHHhjz+wWIy1u5wO44xFUZJiXL39Gw2Fuxn/OAk6tcInyZ8/pJQOTSRkpKiaWlpbsdwxc59RVw29htKSpQPbj+HetWj3Y5kTNCb+OUKnv5kKQ9fHs9NYdhnqZSIpKtqiq9pbp+kNmWgdrUoJg5OZuueQu6YmkmxNfUz5qh+WLGNZ+cs5bJuTbjxrDi34wQtKxAhomtsbR69sjPfLN/KK58vdzuOMUFry64D3DYlk7gG1cO2CZ+/rECEkIGnNqdPUjNe+WI5X+ZucTuOMUGnqLiE4ZM9l4a/OiSZGpUruR0pqFmBCCEiwpiru9KhUU3unJbFBmvqZ8zvPPPJUuav3sFT13alfaOabscJelYgQkzV6EgmDkmmuFgZlprBwUPFx17ImDDwycJf+Nc3q/i/01tyVcLhTR2ML1YgQlCrBtV5tm83stcVMOZDa+pnzKqte7l3Rg7dm9fhwcs7uR2nwrACEaJ6dWnCX85pxZs/rOG9rA3HXsCYELW/sJihk9KJjBTGD0qkciVrwucvKxAh7L5eHTklri6j3lnAss3W1M+EH1XlwVkLyd28m5f6JxBb15rwHQ8rECEsKjKCcYOSqF65ErdOSmePNfUzYWbq/HW8k7Ge2y5ox3kdrAnf8fKrQIhIioi8KyIZzh3eFohITqDDmZPXqJanqd/qrXu5/50ca+pnwsbCDTt5ZPYizmnXgDsubOd2nArJ34uAU4F7gQWA3euygjmjTX3uuaQDz3ySS0rLutx4Vvi2FTDhoWBfIbdOSqdB9WheHpBoTfhOkL8FIl9VZx97NhOsbj23DRlrPFc1dYutQ3LLum5HMiYgSkqUkdOz2bzrANP/eob1JjsJ/p6DeERE/i0iA0WkT+kjoMlMmYqIEJ7v152mdaoyYnIG2/YcdDuSMQEx8asVfLF0Cw9eFk9iC/tD6GT4WyBuBBKAXsAVzuPyQIUygVG7ahQTBiexbW8hd0zNsqZ+JuR8l7eV5z/N5YruTfnTGS3djlPh+XuI6RRV7RDQJKZcdGlWm8ev6sz97yzgpc+WcXdP+89qQsOmnQe4fUomrWNq8FSfrtaErwz4uwfxvYjEBzSJKTf9T2lB3+RYxn6Rx7yl1tTPVHxFxSWMmJzB/qJiXh3iubTbnDx/C8TpQJaI5NplrqHh8au70KlJLe6clsW67fvcjmPMSXnq46WkrdnBU9d2o21Da8JXVvwtEL2AdkBPfjv/cEWgQpnAqxIVycTBSZSUKMMnW1M/U3F9tOAXXv92FTecGceV3Zu6HSek+Fsg9AgPU4HFNajOc/26k7N+J4+9v9jtOMYctxX5e7jv7RwSW9Thb72tCV9Z8/dA3Yd4CoIAVYBWQC7QOUC5TDm5pHNj/tqjNf/8aiUpcXW5JjHW7UjG+GVf4SGGTkonulIE4wclEV3JOgeVNb8KhKp29X4tIknAsIAkMuXu3p4dyFpbwOiZC+jUpBYdG9dyO5IxR6WqPPDuQpZv2cObN51K0zpV3Y4Ukk6o5KpqBnBaGWcxLqkUGcHYQYnUrBLF0EkZ7D5Q5HYkY44q9ae1vJu5gTsvbM857WLcjhOy/G3WN9LrcY+ITAY2BjibKUcNa1Zh3MBE1m7fx31vW1M/E7xy1hfw2PuL6dE+htsuaOt2nJDm7x5ETa9HZTznJK4KVCjjjtNa1+e+Szrw8cJNvP7tKrfjGPMHO/YWMnRSBjE1K/NS/wQirAlfQPl7knqxqs7wHhCRvsCMI8xvKqhbzm1N+podPPXxUhKa1yElrp7bkYwBPE347pqeRf7ug8y49QzqWhO+gPN3D2K0n2OmghMRnu3bnWZ1qzJ8cgZbramfCRLj5+XxZW4+D10RT/fmddyOExaOWiBE5FIRGQs0E5FXvB5vAHZ7shBVu2oUEwcnU7CviNunZFpTP+O6b5bn88Jny7g6oSlDTmvhdpywcaw9iI1AGnAASPd6zAYuCWw046b4prX4x9Vd+H7FNl6Ym+t2HBPGNhbs546pWbRrWIMnrAlfuTrqOQhVzQayRWSyqtq1j2Gmb0pz0tfsYPy8FSS1qMuFnRq5HcmEmcJDJZ5WMEXFTBySTLVoa8JXnvw9B3GqiMwVkWUislJEVonIyoAmM0Hh0Ss707lpLe6alsXabdbUz5SvJz5aQubaAp65rjttYmq4HSfs+FsgXgdeAM4GTgFSnJ8mxHma+iUDMGxyOgeKrKmfKR/vZ2/kje9Xc+NZcVzWrYnbccKSvwVip6p+rKpbVHVb6eNYC4lIL6dFeJ6IjPIxvaWIfO60EP9SRGK9pl0vIsudx/XH8ZlMGWtRvxov9Etg4YZd/P39RW7HMWEgb8seRr2TQ1KLOoy+1JrwucXfAjFPRJ4VkTNEJKn0cbQFRCQSGA9cCsQDA33cdOg54E1V7QY8BjzpLFsPeARPO49T8dwT224u66KL4hsx9Lw2TPl5HW+nr3c7jglhew96mvBViYpk/GBrwucmf8/4lPZdSvEaU+CCoyxzKpCnqisBRGQqnm9fe/eVjgdGOs/nAbOc55cAc1V1u7PsXDz3pJjiZ14TAHdf3J6stQU88O4COjetRacm1tTPlC1VZfTMBazI38NbN59Gk9rWhM9NfpVmVT3fx+NoxQGgGbDO6/V6Z8xbNtDHeX4NUFNE6vu5LCJyi4ikiUhafn6+Px/FnIRKkRG8MjCR2lWjGDopnV3W1M+Usbd+XMPs7I2MvLg9Z7Vt4HacsOdvs75GIvK6iHzsvI4XkZvLYP33AD1EJBPoAWwA/D4LqqqvqWqKqqbExFhHx/IQU7My4wcnsW7Hfu6dkW1N/UyZyVy7g8c/WMwFHRsy7DxrwhcM/D249wYwByi9n98y4M5jLLMBaO71OtYZ+5WqblTVPqqaCDzgjBX4s6xxzylx9Rh9aUfmLNrMv76xq53Nydu+t5DhqRk0qlWFF/p1tyZ8QcLfAtFAVacDJQCqeohj/6U/H2gnIq1EJBoYgOcb2L8SkQYiUpphNPAf5/kcoKeI1HVOTvd0xkyQuPnsVlzapTFPf5LLTyuPeUGbMUdUXKLcOS2LrXsKmTA4iTrVrAlfsPC3QOx1zg0ogIicDuw82gJOERmB5xf7EmC6qi4SkcdE5EpntvOAXBFZBjQCxjjLbgcex1Nk5gOPlZ6wNsFBRHjmum60qFeNEVMy2bL7gNuRTAU19ovlfL0sn0eujKdbrDXhCybizzFk55LWsUAXYCEQA1ynqjmBjee/lJQUTUtLcztG2Fm6aRdXj/+O7rF1SP3zaVSKtEsSjf++WpbPDf/9mWsSm/F83+7WZ8kFIpKuqim+pvl7FVMGnpPIZwJ/BToHU3Ew7unYuBZjru7KT6u289yny9yOYyqQDQX7uXNqJh0a1WTM1daELxj59T0I50tvvYE4Z5meIoKqvhDAbKaCuDY5lrQ1O3j1qxUkt6zLxfHW1M8c3cFDxQxLzaCoWJkwOImq0ZFuRzI++Hs84H3gBqA+v7/9qDEAPHJFPF2a1WLk9CzWbNvrdhwT5MZ8uITsdQU817cbra0JX9Dy95vUsU47DGN8Km3qd/nYbxk6KYOZw86kSpT9VWj+6L2sDbz5wxr+fHYrenWxJnzBzN89iI9FpGdAk5gKr3m9arzYvzuLf9nFw+8tdDuOCULLN+9m1DsLOCWuLvdf2tHtOOYY/C0QPwLvish+EdklIrtFZFcgg5mK6YKOjRhxflump61n+vx1x17AhI09Bw9x66R0qleOZNygJKLsireg5+9/oReAM4BqqlpLVWuqqnVqMz7ddXF7zmpbn4feW8iijUf9uowJE6rKqHdyWLV1L68MTKRRrSpuRzJ+8LdArAMWqjXeMX6IjBBeHpBI3WrRDJ2Uwc791tQv3P3v+9V8kPML91zSgTPbWBO+isLfArES+FJERovIyNJHIIOZiq1BjcqMH5zIxoL93GNN/cJaxtodjPloCRd1asit57ZxO445Dv4WiFXA50A0dpmr8VNyy3qM7t2JuYs388+vralfONq25yDDUzNoXLsKz/dNsCZ8FYxfl7mq6t8BRKSG83pPIEOZ0HHTWXFkrNnBM58sJaF5HU5vXd/tSKacFJcod0zNYtveQmYOPZPa1aLcjmSOk7/3g+ji3LNhEbBIRNJFpHNgo5lQICI8dW1X4hpUZ8TkTLbssqZ+4eLlz5bxbd5WHruyM12a1XY7jjkB/h5ieg0YqaotVbUlcDfwr8DFMqGkZpUoXh2SzN6DhxgxOZOi4hK3I5kAm5e7hVe+yOO65Fj6n9L82AuYoORvgaiuqvNKX6jql0D1gCQyIal9o5o82acrP6/ezrNzct2OYwJo/Y593DUti46Na/L4VV2sCV8F5vdVTCLykIjEOY8H8VzZZIzfrk5sxpDTW/Da1yv5ZOEmt+OYAChtwldcrLw6JNma8FVw/haIm/DcA+Id59EAuDFQoUzoeujyeLrH1ubeGdms2mpN/ULN4x8sJmf9Tp7r1524BnaQoaLzt0C0wXOP6Ag8l7peCHwdqFAmdFWuFMn4wUlERuCYyicAABZCSURBVApDJ6Wzv/BYd641FcWszA1M+nEtfz23NZd0bux2HFMG/C0QqXjuF90HuNx5XBGoUCa0xdatxov9E8jdvJuH3ltoX6ILAbmbdjN65gJObVWPey/p4HYcU0b8LRD5qvq+qq5S1TWlj4AmMyHt/A4Nue38trydvp5p1tSvQtt9oIihk9KpXrkS4wYm2m1nQ4i/94N4RET+jefb1AdLB1V1ZkBSmbBwx0XtyVxXwMOzF9GlWW27Vr4CUlXufyeHNdv3kfrn02hoTfhCir+l/kYgAeiF59DSFXgOMxlzwkqb+tWvHs2tk9LZuc+a+lU0//luNR8t2MS9l3Swb8mHIH8LxCmqmqKq16vqjc7jpoAmM2GhXvVoxg9OYvOuA4ycnkVJiZ2PqCjSVm/nyY+WcHF8I/56bmu345gA8LdAfC8i8QFNYsJWUou6PNC7E58v3cLEr1a4Hcf4YeuegwyfnEGzulV5rm93+zJciPL3HMTpQJaIrMJzDkIAtftUm7Jy/ZlxpK8t4PlPc0lsUcfuGRDEPE34MinYV8TMYadQu6o14QtV/haIXgFNYcKeiPBUn64s3riT26dk8sFt59C4tp3wDEYvzl3Gd3nbeOa6bnRuahcWhDK/DjF5X9pql7maQKleuRKvDklmX2ExIyZnWFO/IPT5ks2Mm5dH/5Tm9EuxJnyhzi5YNkGlndPUL23NDp7+eKnbcYyXdds9Tfjim9Ti71dZt/9wYAXCBJ2rEppx/Rkt+fe3q/howS9uxzHAgaJihqamo8CrQ5KpEmVN+MKBFQgTlB64LJ6E5nW47+0cVubbDQzd9vf3F7Nwwy5e6JdAi/rV3I5jyokVCBOUoitFMH5wElGRwtBJGewrPOR2pLD1Tvp6pvy8llt7tOHi+EZuxzHlKKAFQkR6iUiuiOSJyCgf01uIyDwRyRSRHBHp7YxHicj/RGSBiCwRkdGBzGmCU7M6VXl5QCLLtuzmwXetqZ8blm7axQOzFnB663rc07O923FMOQtYgRCRSGA8cCkQDwz08WW7B4HpqpoIDAAmOON9gcqq2hVIBv4qInGBymqC17ntY7jjwnbMzNzA5J/Xuh0nrOw6UMTQSRnUqhLF2IFJ1oQvDAXyv/ipQJ6qrlTVQmAqcNVh8yhQy3leG9joNV5dRCoBVYFCYFcAs5ogdvsF7Ti3fQx/n72YnPUFbscJC6rKfTNyWLt9H+MGJRFTs7LbkYwLAlkgmgHefZzXO2PeHgWGiMh64CPgNmf8bWAv8AuwFnhOVbcHMKsJYhERwkv9E2hQI5qhkzIo2FfodqSQ9+9vVvHJok2M6tWRU1vVczuOcYnb+4wDgTdUNRboDbwlIhF49j6KgaZAK+BuEflDNzARuUVE0kQkLT8/vzxzm3JWr3o0E4Yks2X3Ae6aZk39AunnVdt56pOl9OrcmD+f08rtOMZFgSwQG/DcprRUrDPm7WZgOoCq/gBUwXO/60HAJ6papKpbgO+AlMNXoKqvOV1mU2JiYgLwEUwwSWheh4cvj2debj7j5+W5HSckbdl9gBGTM2hetyrP9O1mTfjCXCALxHygnYi0EpFoPCehZx82z1o897dGRDrhKRD5zvgFznh1PM0C7Wu1hiGnt+SqhKa88Nkyvl2+1e04IeVQcQm3T8lk14EiJg5JplYVa8IX7gJWIFT1EDACmAMswXO10iIReUxErnRmuxv4i4hkA1OAG9RzLeN4oIaILMJTaP6rqjmBymoqDhHhyT5daRtTg9unZvLLzv1uRwoZz89dxo8rt/OPq7vSqUmtYy9gQp6EyrXlKSkpmpaW5nYMU07ytuzhqnHf0qFxTabecgbRldw+nVaxzV28mb+8mcbAU1vwZJ+ubscx5UhE0lX1D4fwwf2T1MackLYNa/D0dd3IWFvAkx8vcTtOhbZ22z5GTs+iS7NaPHKF3RfM/MYKhKmwLu/WlBvOjOO/363mg5yNx17A/EFpE74IESYOtiZ85vesQJgK7W+9O5HUog73v51D3hZr6ne8HnlvEYs27uLF/t1pXs+a8JnfswJhKrTSpn6VoyIZOimdvQetqZ+/pqetY1raOoaf34YLOloTPvNHViBMhdekdlVeGZBIXv4e/vbuAmvq54dFG3fy0KyFnNmmPiMv7uB2HBOkrECYkHB2uwaMvKg972VtZNKPdjfco9m5v4hhqRnUqRbFKwMTiYywL8MZ36xAmJAx/Py2nN8hhsc+WEzWOmvq54uqcu+MbDbs2M/4QUk0qGFN+MyRWYEwISMiQnixfwINa1ZheGoGO/ZaU7/Dvfb1Sj5dvJnRvTuREmdN+MzRWYEwIaVOtWgmDkkif/dB7rSmfr/z08ptPDMnl8u6NuGms+LcjmMqACsQJuR0i63Dw1fE89WyfMZ+YU39ALbsOsCIKZm0rFeNp67tak34jF+sQJiQNPi0FlyT2IyXPl/G18vCuxX8oeISRkzJZM+BQ0wckkxNa8Jn/GQFwoQkEWHMNV1o17AGd0zNZENB+Db1e3ZOLj+v2s4TfbrQoXFNt+OYCsQKhAlZ1aIrMXFIMkXFyvDUDAoPlbgdqdzNWbSJf3690tmjinU7jqlgrECYkNYmpgbPXNeNrHUFjPlwsdtxytXqrXu5Z3o23WJr87A14TMnwAqECXm9uzbh5rNb8b8f1jA7Ozya+nma8GUQGSmMH5RE5UrWhM8cPysQJiyMurQjKS3rMuqdHJZv3u12nIB7aNZClm7axYv9E6wJnzlhViBMWIiKjGDcoCSqRUcyNDUjpJv6TZu/lhnp67nt/Lac36Gh23FMBWYFwoSNxrWr8MqARFbm72HUzNBs6rdww04eem8R57RrwB0XtXc7jqngrECYsHJm2wbc3bMD72dv5H/fr3Y7Tpnaua+Ioanp1K8ezUv9E6wJnzlpViBM2Bnaow0XdmzImI+WkLF2h9txykRJiXL3jCx+KTjAuEFJ1LcmfKYMWIEwYSciQnihXwKNanma+m3bc9DtSCft1a9X8NmSLTxwWSeSW9Z1O44JEVYgTFiqXS2KiYOT2ba3kDunZVFcgZv6fb9iK8/NyeWybk244cw4t+OYEGIFwoStrrG1+fuVnflm+VZe/ny523FOyOZdB7h9SiatGlTn6Wu7WRM+U6asQJiwNuCU5lybFMvYL5bzZe4Wt+Mcl6LiEkZMzmBfYTGvDkmmRuVKbkcyIcYKhAlrIsI/ru5Ch0Y1uXNaFut37HM7kt+e+WQp81fv4Mk+XWnXyJrwmbJnBcKEvarRkUwckkyx09Tv4KFityMd08cLfuFf36ziT2e05KqEZm7HMSHKCoQxQKsG1Xm2bzey1+/k8Q+Cu6nfyvw93Pt2Dt2b1+GByzq5HceEMCsQxjh6dWnCX85pxaQf1zIrc4PbcXzaX1jMsNQMoiKFCYOtCZ8JLCsQxni5r1dHTomry+iZC1gWZE39VJUHZi0gd/NuXhqQSLM6Vd2OZEKcFQhjvJQ29ateuRK3TkpnTxA19Zvy8zpmZmzg9gva0aN9jNtxTBiwAmHMYRrVqsLYgYms3rqX+9/OCYqmfgvW7+TR2Ys4t30Mt1/Yzu04JkwEtECISC8RyRWRPBEZ5WN6CxGZJyKZIpIjIr29pnUTkR9EZJGILBCRKoHMaoy3M9rU595LOvLhgl/473erXc1SsK+QoanpNKhhTfhM+QpYgRCRSGA8cCkQDwwUkcPve/ggMF1VE4EBwARn2UrAJOBWVe0MnAcUBSqrMb7c2qM1F3VqxBMfLSF9zXZXMpSUKCOnZ7N51wEmDEmmXvVoV3KY8BTIPYhTgTxVXamqhcBU4KrD5lGglvO8NlB6P8ieQI6qZgOo6jZVDf6L001IERGe79edpnWqMjw1k60uNPWb8GUeXyzdwkOXx5PQvE65r9+Et0AWiGbAOq/X650xb48CQ0RkPfARcJsz3h5QEZkjIhkicp+vFYjILSKSJiJp+fn5ZZveGKB21SgmDE5i+75C7piaWa5N/b7L28oLc5dxZfem/N/pLcttvcaUcvsk9UDgDVWNBXoDb4lIBFAJOBsY7Py8RkQuPHxhVX1NVVNUNSUmxq7qMIHRpVltHr+qM9/lbePFucvKZZ2bdnqa8LWOqcGTfbpaEz7jikAWiA1Ac6/Xsc6Yt5uB6QCq+gNQBWiAZ2/ja1Xdqqr78OxdJAUwqzFH1f+UFvRNjmXcvDy+WLo5oOsqKi5h+OQM9hcV8+oQzyW3xrghkAViPtBORFqJSDSek9CzD5tnLXAhgIh0wlMg8oE5QFcRqeacsO4BBHf/AxPyHr+6C52a1OKuadms2x64pn5PfrSU9DU7ePrabrRtaE34jHsCViBU9RAwAs8v+yV4rlZaJCKPiciVzmx3A38RkWxgCnCDeuwAXsBTZLKADFX9MFBZjfFHlahIXh2SRIkqw1IzOFBU9tdNfJjzC//5bhU3nBnHFd2blvn7G3M8JBi+BFQWUlJSNC0tze0YJgx8umgTt7yVzqDTWvDENV3L7H1X5O/hyrHf0qFxTabecgbRldw+RWjCgYikq2qKr2n2f6Axx6ln58b8tUdrJv+0lnfS15fJe+4rPMTQSelUjopk/OAkKw4mKNj/hcacgHt7duC0VvV4YNYClm7adVLvpar8beYClm/Zw8sDEmhS25rwmeBgBcKYE1ApMoKxgxKpWSWKoZMy2HXgxL/oP+mntczK2shdF7XnnHZ2ubYJHlYgjDlBDWtWYdzARNZu38d9M06sqV/2ugIef38x53WIYcT5bQOQ0pgTZwXCmJNwWuv63HdJBz5ZtInXv111XMvu2FvIsNQMYmpW5sV+CURYEz4TZKxAGHOSbjm3NT3jG/Hkx0uZv9q/pn4lJcpd07PI332QCYOTqGtN+EwQsgJhzEkSEZ7r153mdasyPDWD/N3Hbuo3bl4eX+bm8/AV8XS3JnwmSFmBMKYM1KoSxYTByezcX8TtUzI5VFxyxHm/WZ7Pi58t45rEZgw+rUU5pjTm+FiBMKaMxDetxT+u7sIPK7fxwhGa+m0s2M8dU7No17AGY67pYk34TFCzAmFMGeqb0pwBpzRnwpcrmLv49039Cg+VMCw1g8JDJUwckky1aGvCZ4KbFQhjytijV3amc9NajJyexdptvzX1e+KjJWStK+CZ67rRJqaGiwmN8Y8VCGPKWJWoSCYOTkaAoanpHCgqZnb2Rt74fjU3ndWK3l2buB3RGL9YgTAmAFrUr8YL/RJYtHEXt03JZNQ7OSS3rMvo3h3djmaM36xAGBMgF8U3Yth5bZi7eDNVoyIZPyiJqEj7J2cqDjtLZkwAjby4PZUihPM7NqRx7SpuxzHmuFiBMCaAKkVGMLJnB7djGHNCbH/XGGOMT1YgjDHG+GQFwhhjjE9WIIwxxvhkBcIYY4xPViCMMcb4ZAXCGGOMT1YgjDHG+CQncqP1YCQi+cCak3iLBsDWMooTCmx7/Ma2xe/Z9vi9ir49WqpqjK8JIVMgTpaIpKlqits5goVtj9/Ytvg92x6/F8rbww4xGWOM8ckKhDHGGJ+sQPzmNbcDBBnbHr+xbfF7tj1+L2S3h52DMMYY45PtQRhjjPHJCoQxxhifwr5AiEgvEckVkTwRGeV2nkARkf+IyBYRWeg1Vk9E5orIcudnXWdcROQVZ5vkiEiS1zLXO/MvF5Hr3fgsZUFEmovIPBFZLCKLROQOZzzstomIVBGRn0Uk29kWf3fGW4nIT85nniYi0c54Zed1njM9zuu9RjvjuSJyiTufqGyISKSIZIrIB87r8Nseqhq2DyASWAG0BqKBbCDe7VwB+qznAknAQq+xZ4BRzvNRwNPO897Ax4AApwM/OeP1gJXOz7rO87puf7YT3B5NgCTneU1gGRAfjtvE+Uw1nOdRwE/OZ5wODHDGXwWGOs+HAa86zwcA05zn8c6/ocpAK+ffVqTbn+8ktstIYDLwgfM67LZHuO9BnArkqepKVS0EpgJXuZwpIFT1a2D7YcNXAf9znv8PuNpr/E31+BGoIyJNgEuAuaq6XVV3AHOBXoFPX/ZU9RdVzXCe7waWAM0Iw23ifKY9zsso56HABcDbzvjh26J0G70NXCgi4oxPVdWDqroKyMPzb6zCEZFY4DLg385rIQy3R7gXiGbAOq/X652xcNFIVX9xnm8CGjnPj7RdQnJ7OYcEEvH85RyW28Q5nJIFbMFT5FYABap6yJnF+3P9+pmd6TuB+oTItnC8BNwHlDiv6xOG2yPcC4RxqGefOOyueRaRGsA7wJ2qust7WjhtE1UtVtUEIBbPX7kdXY7kGhG5HNiiquluZ3FbuBeIDUBzr9exzli42OwcJsH5ucUZP9J2CantJSJReIpDqqrOdIbDepuoagEwDzgDz2G0Ss4k78/162d2ptcGthE62+Is4EoRWY3nsPMFwMuE4fYI9wIxH2jnXJ0QjecE02yXM5Wn2UDpVTfXA+95jf/JuXLndGCnc9hlDtBTROo6V/f0dMYqHOcY8evAElV9wWtS2G0TEYkRkTrO86rAxXjOycwDrnNmO3xblG6j64AvnL2t2cAA56qeVkA74Ofy+RRlR1VHq2qsqsbh+Z3whaoOJhy3h9tnyd1+4Lk6ZRmeY64PuJ0ngJ9zCvALUITnWOjNeI6Tfg4sBz4D6jnzCjDe2SYLgBSv97kJz8m2POBGtz/XSWyPs/EcPsoBspxH73DcJkA3INPZFguBh53x1nh+oeUBM4DKzngV53WeM72113s94GyjXOBStz9bGWyb8/jtKqaw2x7WasMYY4xP4X6IyRhjzBFYgTDGGOOTFQhjjDE+WYEwxhjjkxUIY4wxPlmBMMYhIt87P+NEZFAZv/fffK3LmGBml7kacxgROQ+4R1UvP45lKulvfXp8Td+jqjXKIp8x5cX2IIxxiEhpR9OngHNEJEtE7nIa2T0rIvOde0H81Zn/PBH5RkRmA4udsVkiku7cV+EWZ+wpoKrzfqne63K+mf2siCwUkQUi0t/rvb8UkbdFZKmIpDrf/kZEnhLPfSxyROS58txGJrxUOvYsxoSdUXjtQTi/6Heq6ikiUhn4TkQ+deZNArqop50zwE2qut1pWTFfRN5R1VEiMkI9zfAO1wdIALoDDZxlvnamJQKdgY3Ad8BZIrIEuAboqKpa2iLDmECwPQhjjq0nnj5MWXhagtfH01cH4Gev4gBwu4hkAz/iadTWjqM7G5iinm6qm4GvgFO83nu9qpbgaQUSh6eV9AHgdRHpA+w76U9nzBFYgTDm2AS4TVUTnEcrVS3dg9j760yecxcXAWeoanc8/Y2qnMR6D3o9LwZKz3OciufGNJcDn5zE+xtzVFYgjPmj3XhuQ1pqDjDUaQ+OiLQXkeo+lqsN7FDVfSLSEc9tO0sVlS5/mG+A/s55jhg8t4Y9YsdP5/4VtVX1I+AuPIemjAkIOwdhzB/lAMXOoaI38NwLIA7IcE4U5/Pb7Sa9fQLc6pwnyMVzmKnUa0COiGSop3V0qXfx3HshG0932ftUdZNTYHypCbwnIlXw7NmMPLGPaMyx2WWuxhhjfLJDTMYYY3yyAmGMMcYnKxDGGGN8sgJhjDHGJysQxhhjfLICYYwxxicrEMYYY3z6f99VbO9gM/+YAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learner.plot('momentum')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### View Top Losses\n", "\n", "Let's examine the validation examples that we got the most wrong. Looks like our model has trouble with expensive wines." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "----------\n", "id:21790 | loss:1042.46 | true:1100.0 | pred:57.54)\n", "\n", "----------\n", "id:13745 | loss:1014.34 | true:1400.0 | pred:385.66)\n", "\n", "----------\n", "id:11710 | loss:884.58 | true:980.0 | pred:95.42)\n", "\n" ] } ], "source": [ "learner.view_top_losses(n=3)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Wet earth, rain-wet stones, damp moss, wild sage and very ripe pear make for a complex opening. Further sniffs reveal more citrus: both juice and zest of lemon. The palate still holds a lot of leesy yeast flavors but its phenolic richness is tempered by total citrus freshness. This is still tightly wound; leave it so it can come into its own. The warming resonance on the palate suggests it has a long future. Drink from 2019.\n" ] } ], "source": [ "print(x_test[21790])" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "A wine that has created its own universe. It has a unique, special softness that allies with the total purity that comes from a small, enclosed single vineyard. The fruit is almost irrelevant here, because it comes as part of a much deeper complexity. This is a great wine, at the summit of Champagne, a sublime, unforgettable experience.\n" ] } ], "source": [ "print(x_test[13745])" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "preds = learner.predict(val_data)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([385.65793], dtype=float32)" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "preds[13745]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Making Predictions\n", "\n", "Lastly, we will use our model to make predictions on 5 randomly selected wines in the validation set." ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TEXT:\n", "Relatively full-bodied and muscular as well as dry, this new effort from winemaker Steve Bird features plenty of brawny citrus and spice flavors that finish long. There's no real track record, so it's probably best to drink now.\n", "\n", "\tpredicted: 18.009167\n", "\tactual: 17.0\n", "----------------------------------------\n", "TEXT:\n", "Very tart and spicy, with distinct notes of clove and orange peel. Citrus and apple flavors crop up unexpectedly, and the tannins have a hint of green tea about them.\n", "\n", "\tpredicted: 20.4764\n", "\tactual: 20.0\n", "----------------------------------------\n", "TEXT:\n", "Dusty apple aromas are given lift courtesy of citrus notes. This feels good on the palate, with zesty acidity. Flavors of stone fruits, tropical fruits, apple and citrus meld together well, while the finish is pure and long.\n", "\n", "\tpredicted: 15.768029\n", "\tactual: 17.0\n", "----------------------------------------\n", "TEXT:\n", "Smoky and savory on the nose, with saucy fruit sitting below a veil of firm oak. Runs a bit tart and racy in the mouth, where cherry and plum flavors are boosted by blazing natural acidity. Not a sour wine, but definitely crisp and racy.\n", "\n", "\tpredicted: 15.798236\n", "\tactual: 24.0\n", "----------------------------------------\n", "TEXT:\n", "Textbook Gewurztraminer, done well, starting with scents of rose petals and lychees, and moving through pear and melon flavors into a finish that shows a hint of bitterness. Medium-weight and just slightly off-dry.\n", "\n", "\tpredicted: 23.65241\n", "\tactual: 16.0\n", "----------------------------------------\n" ] } ], "source": [ "# 5 random predictions\n", "val_data.batch_size = 1\n", "for i in range(5):\n", " idx = np.random.choice(len(x_test))\n", " print(\"TEXT:\\n%s\" % (x_test[idx]))\n", " print()\n", " print(\"\\tpredicted: %s\" % (np.squeeze(learner.predict(val_data[idx]))))\n", " print(\"\\tactual: %s\" % (y_test[idx])) \n", " print('----------------------------------------')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's look at our most expensive prediction. Our most expensive prediction (`$404`) is associated with an expensive wine priced at `$800`, which is good. However, we are `~$400` off. Again, our model has trouble with expensive wines. This is somewhat understandable since our model only looks at short textual descriptions and the winery - neither of which contain clear indicators of their exorbitant prices." ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "highest-priced prediction: 404.31885\n", "actual price for this wine:800.0\n", "TEXT:\n", "The palate opens slowly, offering an initial citrus character, followed by wood and then, finally, wonderfully rich, but taut fruit. There is still a toast character here, with apricots and pear on top of the citrus, but it is still only just developing. In 10–15 years, it will be a magnificent wine.\n" ] } ], "source": [ "max_pred_id = np.argmax(preds)\n", "print(\"highest-priced prediction: %s\" % (np.squeeze(preds[max_pred_id])))\n", "print(\"actual price for this wine:%s\" % (y_test[max_pred_id]))\n", "print('TEXT:\\n%s' % (x_test[max_pred_id]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Making Predictions on Unseen Examples\n", "\n", "In the example above, we made predictions for examples in the validation set. To make predictions for an arbitrary set of wine data, the steps are as follows:\n", "1. Encode the winery using the same label encoder used above for validation data\n", "2. Preprocess the wine description using the `preprocess_test` method. In this example, you will use `preproc.preprocess_test`.\n", "3. Combine both into a `ktrain.Dataset` instance, as we did above." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 2 }