{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%reload_ext autoreload\n", "%autoreload 2\n", "%matplotlib inline\n", "import os\n", "os.environ[\"CUDA_DEVICE_ORDER\"]=\"PCI_BUS_ID\";\n", "os.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0\";" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "using Keras version: 2.2.4-tf\n" ] } ], "source": [ "import ktrain\n", "from ktrain import text" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Predicting Wine Prices from Textual Descriptions\n", "\n", "This notebook shows an example of **text regression** in *ktrain*. Given a textual description of a wine, we will attempt to predict its price. The data is available from FloydHub [here](https://www.floydhub.com/floydhub/datasets/wine-reviews/1/wine_data.csv).\n", "\n", "## Clean and Prepare the Data\n", "\n", "We will simply perform the same data preparation as performed by the [original FloydHub example notebook](https://github.com/floydhub/regression-template) that inspired this exmaple." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Unnamed: 0countrydescriptiondesignationpointspriceprovinceregion_1region_2varietywinery
84868486ItalyMade entirely from Nero d'Avola, this opens wi...Violino8920.0Sicily & SardiniaVittoriaNaNNero d'AvolaPaolo Calì
148584148585PortugalWarre's seems to have found just the right for...Otima 20-year old tawny9042.0PortNaNNaNPortWarre's
1835318353ItalyA more evolved and sophisticated expression of...Campogrande8723.0VenetoSoave SuperioreNaNGarganegaSandro de Bruno
52815281SpainRed-fruit and citrus aromas create an astringe...NaN8412.0Northern SpainRibera del DueroNaNTempranilloCondado de Oriza
8776887768USLightly funky and showing definite signs of ea...Lia's Vineyard8935.0OregonChehalem MountainsWillamette ValleyPinot NoirSeven of Hearts
\n", "
" ], "text/plain": [ " Unnamed: 0 country \\\n", "8486 8486 Italy \n", "148584 148585 Portugal \n", "18353 18353 Italy \n", "5281 5281 Spain \n", "87768 87768 US \n", "\n", " description \\\n", "8486 Made entirely from Nero d'Avola, this opens wi... \n", "148584 Warre's seems to have found just the right for... \n", "18353 A more evolved and sophisticated expression of... \n", "5281 Red-fruit and citrus aromas create an astringe... \n", "87768 Lightly funky and showing definite signs of ea... \n", "\n", " designation points price province \\\n", "8486 Violino 89 20.0 Sicily & Sardinia \n", "148584 Otima 20-year old tawny 90 42.0 Port \n", "18353 Campogrande 87 23.0 Veneto \n", "5281 NaN 84 12.0 Northern Spain \n", "87768 Lia's Vineyard 89 35.0 Oregon \n", "\n", " region_1 region_2 variety winery \n", "8486 Vittoria NaN Nero d'Avola Paolo Calì \n", "148584 NaN NaN Port Warre's \n", "18353 Soave Superiore NaN Garganega Sandro de Bruno \n", "5281 Ribera del Duero NaN Tempranillo Condado de Oriza \n", "87768 Chehalem Mountains Willamette Valley Pinot Noir Seven of Hearts " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "import numpy as np\n", "path = 'data/wine/wine_data.csv' # ADD path/to/dataset\n", "data = pd.read_csv(path)\n", "data = data.sample(frac=1., random_state=0)\n", "data.head()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train size: 95646\n", "Test size: 23912\n" ] } ], "source": [ "# this code was taken directly from FloydHub's regression template for\n", "# wine price prediction: https://github.com/floydhub/regression-template\n", "\n", "# Clean it from null values\n", "data = data[pd.notnull(data['country'])]\n", "data = data[pd.notnull(data['price'])]\n", "data = data.drop(data.columns[0], axis=1) \n", "variety_threshold = 500 # Anything that occurs less than this will be removed.\n", "value_counts = data['variety'].value_counts()\n", "to_remove = value_counts[value_counts <= variety_threshold].index\n", "data.replace(to_remove, np.nan, inplace=True)\n", "data = data[pd.notnull(data['variety'])]\n", "\n", "# Split data into train and test\n", "train_size = int(len(data) * .8)\n", "print (\"Train size: %d\" % train_size)\n", "print (\"Test size: %d\" % (len(data) - train_size))\n", "\n", "# Train features\n", "description_train = data['description'][:train_size]\n", "variety_train = data['variety'][:train_size]\n", "\n", "# Train labels\n", "labels_train = data['price'][:train_size]\n", "\n", "# Test features\n", "description_test = data['description'][train_size:]\n", "variety_test = data['variety'][train_size:]\n", "\n", "# Test labels\n", "labels_test = data['price'][train_size:]\n", "\n", "x_train = description_train.values\n", "y_train = labels_train.values\n", "x_test = description_test.values\n", "y_test = labels_test.values" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## STEP 1: Preprocess the Data" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "task: text regression (supply class_names argument if this is supposed to be classification task)\n", "language: en\n", "Word Counts: 30953\n", "Nrows: 95646\n", "95646 train sequences\n", "train sequence lengths:\n", "\tmean : 41\n", "\t95percentile : 62\n", "\t99percentile : 74\n", "Adding 3-gram features\n", "max_features changed to 1769319 with addition of ngrams\n", "Average train sequence length with ngrams: 120\n", "train (w/ngrams) sequence lengths:\n", "\tmean : 121\n", "\t95percentile : 183\n", "\t99percentile : 219\n", "x_train shape: (95646,200)\n", "y_train shape: 95646\n", "23912 test sequences\n", "test sequence lengths:\n", "\tmean : 41\n", "\t95percentile : 62\n", "\t99percentile : 73\n", "Average test sequence length with ngrams: 111\n", "test (w/ngrams) sequence lengths:\n", "\tmean : 112\n", "\t95percentile : 172\n", "\t99percentile : 207\n", "x_test shape: (23912,200)\n", "y_test shape: 23912\n" ] } ], "source": [ "trn, val, preproc = text.texts_from_array(x_train=x_train, y_train=y_train,\n", " x_test=x_test, y_test=y_test,\n", " ngram_range=3, \n", " maxlen=200, \n", " max_features=35000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## STEP 2: Create a Text Regression Model and Wrap in Learner" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "fasttext: a fastText-like model [http://arxiv.org/pdf/1607.01759.pdf]\n", "linreg: linear text regression using a trainable Embedding layer\n", "bigru: Bidirectional GRU with pretrained word vectors [https://arxiv.org/abs/1712.09405]\n", "standard_gru: simple 2-layer GRU with randomly initialized embeddings\n", "bert: Bidirectional Encoder Representations from Transformers (BERT) [https://arxiv.org/abs/1810.04805]\n", "distilbert: distilled, smaller, and faster BERT from Hugging Face [https://arxiv.org/abs/1910.01108]\n" ] } ], "source": [ "text.print_text_regression_models()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "maxlen is 200\n", "done.\n" ] } ], "source": [ "model = text.text_regression_model('linreg', train_data=trn, preproc=preproc)\n", "learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=256)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lower the `batch size` above if you run out of GPU memory." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## STEP 3: Estimate the LR" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "simulating training for different learning rates... this may take a few moments...\n", "Train on 95646 samples\n", "Epoch 1/1024\n", "95646/95646 [==============================] - 8s 81us/sample - loss: 2627.6407 - mae: 34.2769\n", "Epoch 2/1024\n", "95646/95646 [==============================] - 7s 70us/sample - loss: 2610.0313 - mae: 34.0299\n", "Epoch 3/1024\n", "95646/95646 [==============================] - 7s 70us/sample - loss: 2148.5174 - mae: 26.8848\n", "Epoch 4/1024\n", "95646/95646 [==============================] - 7s 71us/sample - loss: 1158.6146 - mae: 15.1160\n", "Epoch 5/1024\n", "15360/95646 [===>..........................] - ETA: 5s - loss: 4022.5116 - mae: 36.6476\n", "\n", "done.\n", "Please invoke the Learner.lr_plot() method to visually inspect the loss plot to help identify the maximal learning rate associated with falling loss.\n" ] } ], "source": [ "learner.lr_find()" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEKCAYAAADXdbjqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dd3gc5bW437Or3otlWZbcuw3GGNNLIIAxLZByAyQBkpAQEnLTbpJL2oUQIPxyE0KANDo3QBKSAKH3XoyxjXHFvVuyLBf1ttrv98fM7M6udlcraVdaSed9nn088007O1rPmfOdJsYYFEVRFKU/eAZbAEVRFGXoo8pEURRF6TeqTBRFUZR+o8pEURRF6TeqTBRFUZR+o8pEURRF6Tdpgy1AMhg1apSZOHHiYIuhKIoypFi2bFmdMaasL8cOS2UyceJEli5dOthiKIqiDClEZHtfj9VpLkVRFKXfqDJRFEVR+o0qE0VRFKXfqDJRFEVR+o0qE0VRFKXfqDJRFEVR+o0qE0VRlGHCsu0HWLL1wKBcW5VJGM+vqeHBxX0OtVYURRk0bn1pIzc9s25Qrj0skxb7w9f+sgyAzx87HhEZZGkURVHip62zi+x076BcWy2TKHR0+QdbBEVRlF7R2tlFToYqk5Sis0vbGSuKMrRo7egiS5VJatHpU8tEUZShRVunX6e5Uo1OneZSFGWI0dLhU2WSaqjPRFGUoUZrZxfZOs2VWqjPRFGUoYTfb2jr9JOllklqodNciqIMJdptP69Oc6UYHeqAVxRlCNHa2QVAdvrgPNZVmURBLRNFUYYSAWWiPpPUQn0miqIMJVo7LGWiPpMUQy0TRVGGEm22ZZKTMThVspKmTEQkS0SWiMiHIrJGRH5uj08SkfdEZJOI/F1EMuzxTHt9k719outcP7LH14vIWcmSGSDDa90SDQ1WFGUoEfSZDD/LpB34uDHmCGAesEhEjgP+H/BbY8xU4CBwhb3/FcBBe/y39n6IyGzgYmAOsAj4g4gk7W6lea3ijpoBryjKUKKlw/GZDDMHvLFoslfT7Y8BPg780x5/ALjQXr7AXsfefrpYZXsvAP5mjGk3xmwFNgHHJEtur8dWJuozURRlCDGsfSYi4hWRFUAt8CKwGThkjPHZu+wCKu3lSmAngL29Hih1j0c4JuGk29Nc6jNRFGUo0TaMp7kwxnQZY+YBVVjWxMxkXUtErhSRpSKydN++fX0+T5ptmYwEn8ndb27h3yt2D7YYiqIkgBERGmyMOQS8ChwPFImIE25QBThPs93AOAB7eyGw3z0e4Rj3Ne40xiwwxiwoKyvrs6yOMlmx81CfzzFUuOHpdXz7bysGWwxFURKAM8017CwTESkTkSJ7ORs4E1iHpVQ+Y+92OfBve/kJex17+yvGGGOPX2xHe00CpgFLkiV3pv2HePi9Hcm6RMph3WZFUYYyjmUyWD6TZAYkVwAP2JFXHuARY8xTIrIW+JuI3AB8ANxj738P8BcR2QQcwIrgwhizRkQeAdYCPuBqY0xXsoTOTBt5qTf7GtsZXZA12GIoitIPWju6EBm8Z1jSlIkxZiVwZITxLUSIxjLGtAH/EeVcNwI3JlrGSDihwSOJjbVNA6JM2jq7mPmz57j2/Nl86cRJSb+eoowkmtp95GWmYQXBDjwj7zW8B0bSjE+u7ajbsLdxQK7340dXAfDzJ9cOyPUUZSTRbCuTwUKVSRj+EaRMSvMyAcsyGQje3bI/sNzuS9pMpaKMSJrafeSqMkkd3M7o4V6G3mB91017B0aZVNe3BZZrG9qTdp13NtUx8Zqn2ThAFpeipAJNapmkLk6o3XDFb+vKddUNdA2wSVbT0NbzTn3kjY11ADy3uiZp11CUVEOVSYrhd1kmLZ2+GHumBu2+rn6F9uZmeGls9/H6htoEStUzbisl0VQWWcEE1UlUWIqSaqjPJMVwP5dbUtwy6fD5mfHT57j52Y/6dLzfGM6aM4bcDC9PflidYOm6M3lULh+bbiWU3vT0uqTltxRkpwOwvyl5U2mKkmo0t3epzySV8BsTKEOfatNcbZ1dvLO5LrDulHz58xtb+nQ+Y6xQ6OaOLh77YDcvrd2bEDmj0dHlpyQ3A7CmuTYl2fG/v6kjqedXlFSisa2TvMzBSVgEVSbdMECO/QdJNcvk5mc/4nN3vcfaPQ0AdLkqG/flLd9g8Lhi0pfvONh/IaPwlQeWsutgK+muPJ4dB1qSci1nqrJOLRNlhGCMobmji7wstUxSBwO5dqeylo7U8pnstB++uw5a//r8wWizvkx1+Q2IwFlzygH4w2ubA5VHE81L6/YGrnnrRfMA2JukiC7ntuxrVGWijAzaOv10+Y1Oc6USfmPItS2TVJvmKrR9Adv2NzP/Fy/y1yXB+mF9meoyBkSEP1+6IDD29/d3xjgCvv7gMv73+b75aAA27m3knMMrAKhtTI6D3LFMmlPs76coyaKp3XrxVQd8CmEI9lBOtWmusgIryfD1Dfs40NzBr1/YELK9ppcRUsYYwgsvxPrOfr/h2dU1/P7Vzb26jpuahjYy0jyU5mZQmyTLwT3j90gPylFRhgPOLMpg9X8HVSbdMIaAZdKSpCmfvlJVlA3A25v2h4zbVfO56sFlvTqfgYDP5FzbWvDH8L10ubbVt3b26lqTRuUCUJxjOeDL8jOTlrjolvPJlXuSco1wDjR38N2/r6CxrXf3RVESgfMSmDNIvUxAlUk3/MYEfCatKeYzifaYd7pD9rYHi98YHP/7rRdbfoyGGA9Dd2LjtrrmXl0rzSNkpHm470tHAzC6IIt9SZ7mAnhzY92AJGQ+8M42HvtgN/e8tTXp11KUcIL931WZpAzGBLV7qk1z+aM8FK85O9jAcmcvIqSMCVom6V4PFYVZ1DVGD6d1tzLeebB3kVhdfsPC2eVUFFrW1ZiCTPYkKXEx/Db97qUNkXdMIGX51hRkb6caFSUROIEzOYPUywRUmXTDGEOa10O6V7j1pY0pFREU7f26ojCLP33hKIBeteENn9IalZcZM5zW/YZf14v78vqGfew61IrXE/TQTCjNZV9jO83tibf+nDDp6y+YA8C6muTX6HKUSTIz+xUlGsFpLvWZpAwGEKDTzuH473+tHNDr//n1zUy85umIUzPuoVsvmhdoMez1eALhvb9+YUP80zp2aLBDWX4me2OUIPG5znvdk2vZeaCFK+5/n18+Y2Wzt3T4IhbHvPzeJXT4/HhdF5tYavlQtu3v3XRZPDgW3Plzx3J4ZSEHmpOfvOhYeP21TFbvrmfiNU/zUU1DIsRSRgiOA16nuVII99QPEPPhmgzuetMK8Y1kETlv3F89eRLnzq3guMmlgGXiuhviXPvE6rhyZNwOeIDp5flsqm0KTGfd/eYWJl7zNL9+fj1ANyV1/h1v8fJHtfz5jS1s2NvEvJ+/yAW/fzvq9dyWyayKfAA+3Fnfo5y9xRHTI8LxU0pZtv0g25OgtNw4f5v+JmKed/tbAHz7ryv6LZMycmhVB3zq4XZKQ2yHdDIotzse7qlv7bbNmZX69hnTSfd6mDnGeiDvPmTte/OnDgfgwcU7OPe2t3q8lj8sNHhcSTY+v2Gr7Vy/4el1ANzx6iaMMSGWCcChluC9Wbr9AB1dftZVN0S9Z25lMmlULplpnuRYJvaNEg+cfdgYAD5K8lSXc2taO7uiKvJDLT1bSONLcgAozk1PmGzK8CfggFefSepgsKZ+zp1rhco2tA5sRNcYW5lUH+puEQUekvb61z42hY9NL+MzR1UBsMh+cAJsrWsOcZhHwhjwuB7wo/Otay/87Ru8s6mOSjsUGSxLySnf8v2F00PO4/UID78XTKB019xyBw24ryUilBdkJcXyc+6TR4Spo/MA2Lwv2T1bgt9z+/7u1sn2/c3Mu/5F7ns7drTXefbvriw/+W2UleFDa6dGc6UcTlb4/5w3G4CTpo4a0OuXF9rKJJJlYv/rTE2V5WfywJePYZTdMbHIzuFweGHNXv7w2qao1wq3TBwnMsDn7n6P5g5fIIfllY9q6bTrlIwryQnkpQCcOr2MNXuCc/yf+sM7gWV3zocvTLmNKchKaPSTM9Xk6C+vCPlZ6YzOz2TLvuROc7mNtkhh07sPWn/Ph1xKN9Z5arV8vtILWuz/q5lpg/dIV2UShpMVXl6QxYzy/JD6VwNBvl0OYU8syyQ8bT0KVz+8nF89tz5qvS3LCguezLGKHA61dLJwtmXtXPPoKny2ZeL1CJXFQatlTmVhVBncfpbwWlyjCzJ7nQW/Zk99xKm0L9//PnN//gLQ/T5NGpUbmLpLFu7AuK0Rpu4ceXtK9nRkT1Z1AGV40tLRRU5GWsj/54Fm8OLIUhRnmgsgLystUPOmr/i6/BiCiYXxXB9gz6HoPpPe/l7W1zRyxLiiCOcL9Q+NKczirssWMH98EUff+BJ+E6wHBsFaWmkeT8gU2GQ7ux1g5ph8ml0+A7efJfw7leRmBKbj4r0/bl/QtpvPDSy/8lFt4Ds598mx4CqLs1m8ObRqQKJxh1lHskwa2qx7crCHyDJnWnCgAz+UoU1rR9egTnGBWibdMCZYlj0vM42mtp6VydMrqzni5y/Q4fNT39IZEol13u1vccYtr8d9fedhEnGay+ULiMYk14Pd4a1NdRH2tKf0wsbOnF1OaV4mk8ssX4P7B+q83ad5hIuOHhcYH2c7jQGmlOWx51BbYErLXSZ/d5gyca79+vp9Ub9PLMKnzZxr/K8dfeZM0VUVZVPT0NajD6k/ON+yJDeDbXVBn8lf3t3GrS9tCFzb5zcxQ7edTS0dXf1+kVFGDpZlosokpfC7HrD5WWk0xqFMfvnsOupbO/nWXz/gpF+9wtE3vhTY9lFNYzeH7Ic7D0WNeHIeJpGyw/1hb9yR+NuVx/GnL8wPGYtWZsVvTNRzOUqpLD+TX9pRYs5D0usVslxRI+NDlEkuXX4TSN5z+0zcfhaAS4+fCMD+5vimdN4Nsy4iRWgt3nIgsOx8t6riHPzG6nWfLBxFP3lUbmCaq93Xxc/+vYZbX9pIW2dQkdXEsDrcFo5aJ0q8tHR0DWokF6gy6YY19WM9hPKz0mh0vR3ua2zntpc3ditrUpBlTQU9t6YmqvJxMr2NMVzw+7e5+M+LI+7nbuwUngAYHs0VifKCLBYdVhGIRgPYsDdyWKx7Si+cXPstJy8zjU8cMRYgkKvhJEseNaEYgFF5Qcf/zIoCIJjs6ficrjl7ZkApOThKqKa+HWNMj5bDJXeF3rOv/SVY2NJxPH7/Hx8GxpzvdvwUKx9n5a7E57Q4ODpgWnk++xrbmXjN09z+cjD4YZer/MyKHdFrqLmbnCWrEKYy/GjrVMsk5TCurPDwaa6fPr6KW17cwOKtoW/IY4u6h3GGT1HssqN5nCmOtVHekh2FYUx3JdAbn8nHZ4wOLO880BLRCe9ErkXi8hMmkp3u5fRZo8nNTGNUXkYgvNbJF/n7lcex4YazQ87hKBgnPNj5vkXZ6aSF+UWcUvR7G9v4zt9XMPt/novZMXJcSXbIenlBMPrMaQfsxpFrbFE2aR4J/A16yy0vbuCWF9bH3Mf5uy2wvz9Y+TkOO1zW6dUPL49xnuDya+treyuqMkJp6fANaikVUGXSDauciuMzSae1syswN+/8R3fPiQPMtt/G3by+fh8/fXxVYN1piet2SE+85umQN1brGsHtTnfCgGyBKKWetUm5KzLLb4IO6m7ninL8keOLWfeLRVQVW9bDpFG5bLMfiI6zPM3rISMsFDE/K42Tpo6isjib6vrWgKXgTlh0M7ogi731bfx7xR46u0zUvJ4Pdhyk3Z4qSvMIJ0wpZfmOQ9S3dvKl+5Z0q4l1zKSSwLLXI1QUZXXz2cTLbS9v5LZXoodYQ1DRHzEucmRb+MtDNCvMbwyluRnMH1/En9/YklQ/jzJ8aFEHfOphOeCtZaefsmNlONNZP35sVcgxXYGHfHBs2/5mHlwczClw8gzCs8h/9vjqkHW/sQouHlZZwOItoRaQVf4kvu9x/JRSvn7qFO653Oqi+I2Hgm/Dm2obA8ollv/FjVPtFyIrhqtPm8LsigKy072U5mXwwY5D3Pf2tkDvlTRv5OuMKcgM8SFE85988g/vBMJlf/PZI/jY9DIAfvviBl61HfhXnjI5MOX25RMnhRxfVZTD7l5WOg4nVtFP5yUgM83L3ZctCNlWlJPeTdkt334wynms5M4CO4pu2k+e7Y/IygihtVN9JimH3zXNlW8rE8cPMrmse6QUWAoiIy00XHZ9mHPYsUDc0U3WeOjbsqPMjplovXn7/YYVOw9x1xtbYjrMw/F6hP9eNJOPzxxtvZnbyZCPvL+TM255gyseWArEH2Y8pjBo6aRFUCY/OGsmT3/rJEQkUNDRHQablRb5hz6mMCtkOi9Sfk14bsYF8ypZOMfKf7n/nW2B8ca2Tn56rpVsGj4lVlmc3WfLxCFWrxL3X/WM2eV854xpgfUqOyfH6xFe+/6pQHdLMXAe++/vLoqpDbeUntBorhTEEHTAF9lvh07VWfcfyx2W6vcb0jzC4a7kvSc+DO3w9+TKatZVNwQc0sdOKuGMWaO7PSj9fstamDgqhw6fn7qmdi6/dwk3PrOOQy2dvc4xERGuPGUy1fVtdPkNt7+6MWR7vJaOO6Ex2pSVc98+d+x4AP6xbFdg22kzR0c8ZnR+VqBCM8Da6u5O8keX7+o25o4gCxy7p4ELj6zkg5+dyZyxodNNlUXZ1DZ2D2qIhwx7Wi9msUgn0s6+N5fbkWrOtcFSwhNH5TKhNIe/L43cTth5Yfieq2TNAy6FqSiR0DyTFMTtgB9faj2wnEqw7hmqhjYfdU3t/HPZLnx+g1ckJN8inC6/4ezfvRkobHjBvErmjSuiNqynh99+M3Ue3sfc9HJAiW3Z19ynDFfnYfbSur3dLJt4zxdqmcT+2cytCk2Q/Nopk0NCiaOdF2DV7u6BCZHyMrwe4fnvnBIydsXJkwEojuCMryzOxpjI+Ts9YWxN8ezqmqgdJsMj7YpzMzj7sDFcdvwEKous30W7rci272/hUEsnD723vdt389tVq+eMLeSp/zwJ6H8lYmV447R/UMskxbAS+ezaV3bNq/12wyh3SHB9ayffe+RDvv+PD7nv7W14vRKIZArnbFcBxl8/b3X9c95SIbSnR5cdmjzWNWU22lYs727ZHzMsOBrn2Pkd66obmDUmNFgg3lo+5XFYJg4ZaZ6A72JWRQE/OmdWjPNmhqw/+eEequtbec/2F9U2tvH6hmBS4yxXsMOMMfk8+c2TuO9LR7Pt5nMDIcyRqLLv5/19eMt3P+/fjJYAav/rVtZ//MJRXH/BYSGlZwB+Yt+Pnzy2mvve3srq3UFrzF21+rDKQo6ZVJL0umLK0Kbd58dvBrcxFqgy6YYh6IB3Sok4jl93Al5DaycHXM5irwgnTxvFzDH53G/3OQe4+7IFXH3aVAps/8u79kPS65FggyhXdJgx1jb3G3uGy3nd3odpmpLcDCaW5vDEij0caO5g0qhcvn7qFMCK2oqHCpc86VGc6W7u/aJ1D6762OSY+zlv7WCFYgMc/8tXuOjOxTS1+7jsniW8ubGOrHQPS35yOg995diQ4w+vKuS0GZGn0ML3g8jVmHvCbwzfPG0quRle1u6JHdIdydBz5+EAfHxWUN4bnl7Hebe/xSPvW9Ne4f10ppTlDkDF477z2vpazrv9zYjVCJSBoTUFys+DKpNuuB3wTl7EH17bzNJtB0LCdutbO5lilxwBSwHkZKTx3HdO4dQZo/noF4vYfNM5nDG7nMMqC1l53Vkh10nzRrZMnGmuElcF4NURpn56y7TyfLbUNbNk2wFKcjP470Uz+eBnZ0a1psJxVxTuyTIBa6rrg5+dGdNagFDfxw8XzQjZ9tr62kCWe1unn9H5WRHzSeIhPyuduVWFtEQpehkLp1T/lNF5/HXJDh56b3vEfSCyMnEi4b53puUHcQdqOPzQTvL0u6IJwSpPc7Cls8eaXoPFjx5dxerdDezVwpSDhvObHrbTXCIyTkReFZG1IrJGRL5tj18nIrtFZIX9Ocd1zI9EZJOIrBeRs1zji+yxTSJyTbJkBqdqcPcnwoqdh0KmuXYebGFccfBBGD6rn5Xu7fbQPXFqaWDZ6xHyMtMoy88MKfPhzJl7PMIvLjwMsML++ltauso11eKIFcm3EI10r4eTp1nl+J2S9z1RnJvRo0/G7TR030+Abz78QdzyxcPYwuxehwcH66ERCLD43Usbo+4X6bdzzKQS/vrV47j6tKkAUf1HEPz7OzgRhNGSXAcbx3qvb9GIs8EiYJkMV2UC+ID/MsbMBo4DrhaR2fa23xpj5tmfZwDsbRcDc4BFwB9ExCsiXuD3wNnAbOAS13kSTrRcjn1N7TiWfHa6lw01jSEJZfH0Gb/z0mD+gdPH49TpZTy1spqr/rKM6vrWkDnzS4+bEGju5EyJ9RV3NNb72yLnOPTEX644li03nRPzYdgXnDf1jDQPv/zU4fzknFmBLpKJZFZFAZv3NfcqOspdD+0HZ1mWk9sidQj6TCKf5/gppSEvF7/6zNxuIdbtvq5unT4PsxXY5+9+L26ZBxInH6an0vpK8gi27B2mPhNjTLUxZrm93AisAypjHHIB8DdjTLsxZiuwCTjG/mwyxmwxxnQAf7P3TZLchMxVXHrcBMB6+DvTXGle4YF3t/PnN7YE9otVCdYhNzONI+y5e6eQ4qfmW10Sn1tTw5m3vEFXV2guyWFjLYdzf3MNTnfN03/D9pf0BU+8scS9wJlqy0zzcMkx4/nqKZMD3SMBTp42KhDZ1B+crPhrn1gTs2yLG7/LMinKyeATR4xl16Hu1o1jtcYbHffZBeO44qTQxMrHP9gdUrUagt0vgah9acAq7//YB91DqJNNoSqTQcdpEz3Y01wDospEZCJwJPAecCLwTRG5DFiKZb0cxFI07kp+uwgqn51h46Fe2AQRqcTILy48jHXVDextaGN8SQ4iRCzm6CiJnvj3N0/iUEtHoCuiuxRLU7uPLXVNIQ+TT8wby+Mr9kSsItwbpo7OZ8tN5yRFGfSXmz51OMdMKgkJBjh5WhnW+4flzI+330ksppcHLYo3N9Zxip1FH4ugY926b+NKsnlmVTW+Ln9IrbGeLJNIHD2xJOSF5L//tYqq4mxyo7xh1tS3Bfxs4Zxw8ysAzBtXHLENQbJwlEmDKpNB42CLNSuSmzlMLRMHEckD/gV8xxjTAPwRmALMA6qB3yToOleKyFIRWbpvX9/6Y4Q3VXIozs3gUEsnXXY+yVfC3ihXXreQh756XNzXcbfXLcxJD9m2YW9TiBUyuyKopO743JF88YSJcV8nnFRUJGBFcX3huAkh00DuB38iFAlAqcvXs8oVjvvqR7Ws2RO5onD4b2LSqDx8fsMbG0N/Y45hGslnEo0zZpd3G9t1sLWbE/+v9m8rUsO0cKJ9j2ThlBhSy2TwWLunARGYVZH4qeHekFRlIiLpWIrkIWPMowDGmL3GmC5jjB+4C2saC2A3MM51eJU9Fm08BGPMncaYBcaYBWVlPb9xRiJaeGdxTjqHWjoDdZO+f1Yw6qiqOJuCrPRAWGtf+Mk5s7j+gjmB9QMtQf+LO0T4vLljue4TcxgJiAh3X7aA685PrHvsSydOBOBfrqz6L93/Pufe9lbEHjPuaS6wKhcAfPn+pSH7BazaXv6P+t6Z07n8+AkhY+F9Whyf0q44lMnm2oHNSXGmVg62pGa02Uhgf3MHxTkZZEYpWTRQJDOaS4B7gHXGmFtc4+4OSZ8EnEqHTwAXi0imiEwCpgFLgPeBaSIySUQysJz0TyRD5mhTFUU5GdQ0tPGn1zfT4fOTle4NVMv94aKZ/b7uV0+ZHPDNABwzqTRk+7KfnsHyn53Z7+sMNc6YXc4Xwwo29pdrz5/Dd8+Yzta6Zlo7ukLyI+Ze90I3X0p4Q7JxJTnk2y8O7mMDocG9lOdbp0/j5xccFnOfMYVZiMRnmfz2pQ28szlyYmUycKoDHNRorkHjQHNHn0PmE0kyLZMTgUuBj4eFAf9KRFaJyErgNOC7AMaYNcAjwFrgOeBq24LxAd8EnseaRH/E3jfhhM+PO7j7oDtMtEutJCpRSET451XHM2lULjdeGPpwKc3LTIkfy3BhyuhcjLHaEIdbAbVh+RKRrNXPLLCCA9y1x5yHaryFOMP54+fnB0Kuw0PKM9I8FGan89zqmqjHO0mxAJ+7a+AivxwlWt+qlslgsb+5IyQvbbBImsfGGPMWkV/UnolxzI3AjRHGn4l1XKKIFuBTlNNdmRw2tpANe5sYnR9fzkU8LJhYwqt2VVkleTgO6nNue5Pz5oa2Et6yrzmkdEwwGTH4U/726dO47+1tbK1rZsXOQ2SmeYI+kz66pc4+vIJJZbl8+b73uePz87ttz81I46OaRstvF8H3FU80YTJwLnuwWS2TweJgc0fEcPWBRjPgI9DNAR9B6//0vNn85j+OYG6cUVxK6uDOun9qZTUA/7jqeAC21IWWLjFhPhOwpj2nlOWyfX8zl9y5mLN/9yY3P/sR0DsHfDgzxxTwzo9OZ36EEjdX2eHc0Yo++vyGC+cFqw0MlEPcuT/vbtkfyJ1SBpaGtk4Ksgc3kgtUmYQQzQFf5JrmcraV5Gbw6aOq+lTFVxlc8rPSuzn254wtICvdw9awoorhPhOHiaW5bN/fQmtY7keyfg7T7OTVnQda2H2olbqm0Ok4n99QWZzNnZceBVhTeAOBu8TQA+9uG5BrKqG0dnQNesIiqDIJIZoT1R2++9g3Thw4gZSk8cUTJwVyg+aPLyInI42Jpbk89sHukDLz4dFcbsL9LZA8ZeK0N7jv7a2cePMrLLjhpcA2Y4w9/eVhsj3dsWWAikO6Z9diJVUqyaOt009m+uA/ygdfghQiUhlxgNLcoF8knoq5ytDg2MlW1NzyHYcAq5jl/uYOzr/jrcA+0YIy5kcpkNlXB3xPOOVwnBbFAHe9sYVdB1sC/pJ0jwSm8L73yIexm3klCLdlEmdRASWBdPkNHV3+Qa8YDKpMQog2zVXqKiHenzlxJbX41HyrwMJVH7P8EU4dtMY2XyAMN1oiq7skzbVcUQYAACAASURBVI/Pmcm3Pj6VP196VMISLMPxeqRbsMCNz6zjpP/3aqD1s9crgZB1iN4aOJEYA/mZaYwtzOrWglpJPo41mArKZPAn2lKISJE7EJqBrZbJ8GHmmAK23XxuYP07Z0yns8vPg4t3cMLNr7D2+rOiTnOJCF87ZTLvbT3AZcdPTHjxy0jMqigIBAy4Of92y5IKLxwZ7ldJBsYYvF7h6EklLO1jAVGl7zjKZCB+fz2hlomLSLW5HDbccDYPf+VYppUPbskCJXkUZqfzySODBSZn/8/zLNtuPSAjTV/96JxZPH71iQP2H/mIsHbIDo1222ev3U7Zqbg8EJaCUzJ/bFE2exvaQkKUb372o5AumUriaU0hy0SViYtYDY4y0jycMHXUwAqkDDhHjisKaRj2uuOjSAGD9IQppZw3tyKkk6cbx2r+25XHUZqbwfoIAQKJxmnmNbYwC5/fhFhDf3p9M5ffuyTpMoxkHMtEHfApRjQHvDJy8HisSgQOK3dZhRNT4Tfh8Qh3fG4+p84YHbGNgFMfrigng0/MG8uOAy1xl9rvK5YhIoFukvGUfFESR1unVdJHLZMUI1Yfb2XkIK5GWOv3Wm/3qVZw+YeLZrLt5nMpL4jcTnlCSQ4tHV3sS7LfxNiWSUWRFW1WHSFxsbZBkxmTRav6TFKTvMw0br/kSLuXhjKSufq0qTz0lWDbnFSwTCJx56ULAnXifF1BK2SCXTJmx/7etSnuLcb2mTiWSaQs+P724lGikyote0GVSQhZ6V7OP2LsgDYXUlKXeeOCDu/9cbRlHgyOGFfEs98+hR+cNYPzjwiWU3HaPG9LsjJxfCZF2emkeyWiJbSvMflRZSOVQ3bZnKIIxWgHGlUmihKF3My0gG/C6WOSimRneLn6tKkhOSaVRdl4BHYkOXHRb6xpQY/Hsk5W7+7enGsgQpRHKgftl5xUqCqueSaKEoMfLpqZkJ41A01GmofK4uykWybGGOyIZD42vYy/LN5OW2dXyBx+nVomSWP3oVYy0jwh3VsHC7VMFGWYYhWjTLZlYgJVIeaMLQAsS8QdRbYqgrWiJIYd+1sYX5ITsS3BQKPKRFGGKRNKc9gepWR9ojAEI92c5l51TR0hBSBfWLs3qTKMZFo7u8hNAec7qDJRlGHLxNJcDrV0ciiJ/dmdDHiAUXajuLrG9pACkADtPq0onAzaOrvITIGwYFBloijDlsMqrRL7b25MXk94vzGBvKxRdkHUuqagMnGqHR9I0Wi4oU6bz58SOSagykRRhi3zxxeT5hHWVTck7RpW0qJtmQSmudoDpYlm2HXCIvV+UfpPe2cXWWmp8RhPDSkURUk4GWkeJpflsmFv8hpl+f3Baa6sdC/5mWnUNXUECj46OVvVhzRxMRmER84NJqpMFGUYM708nw17k2cVuKe5wPKb7HNNc5XZfpQDzRoenAzaOv1kqmWiKEqymVGez44DLTy4eHtSzm8I7f8zKi+D/U3tgWiurHQveba1oiSeNp9aJoqiDABO/52fPr46KRWEnUKPDqPyMqlr6ghcyyNWp1J1wCcHa5orNR7jqSGFoihJ4czZ5UywC0H+/Mm1CT+/OzQYLGWyr7Gdu9/cCjhFILNYvkO7MCYaYwxtnRrNpSjKAOD1CH/8/FEA/Gv5roSf3x/BMqlv7eSOVzcBlmVyzMQSdh1spcPnT/j1RzLt9v1UZaIoyoAwe2wBlx43ISll9J1Cjw6j8kNrRIkIlcVWefq92tckobR0pE7LXlBloigjgsribOpbO2ls60zoeU1YNFdFYVbIdo8IYwpVmSQDp6tl+D0fLFSZKMoIoMq2DnYnuK2uCfOZzK0qCtnukWAWfKQujErf2WnXXRtXkjPIkljEpUxE5NsiUiAW94jIchFZmGzhFEVJDJVFtjI5mFhlEu4zKQ4rhV7f2skY+805UhdGpe84LwbOi8JgE69l8mVjTAOwECgGLgVuTppUiqIkFMdvsSsJysTtMwkvhV5d30ZBVhrZ6V5qdJoroTS0diIChSnQZRHiVybOL+Qc4C/GmDWuMUVRUpyyvEwy0zwJn+ayQoNDx1Zet5BffXouAIdaOhA7PFgtk8TS2O4jLzMtRJkPJvEqk2Ui8gKWMnleRPIBjfNTlCGCiFBZlM2ug8H+Js+uqub3dghvX3EXenQoyEpntt0oa+roPADKC7LUMkkwTW0+8jNTp1luvJJcAcwDthhjWkSkBPhS8sRSFCXRbKlrZktdM41tneRnpfP1h5YDcPZhYyjLz2RbXQuHVxX26pxWaHD38cMqC3n22ycz3c7AryjM4r2tB/r9HZQgTe0+8rKGnjI5HlhhjGkWkS8A84HfJU8sRVGSxaPLd4eUN/n4b17nP46q4h/LdvGnLxzFosPGxH2uSJaJw6yKgsByeWEWexva8PsNnhRoMTscaLKnuVKFeKe5/gi0iMgRwH8Bm4H/i3WAiIwTkVdFZK2IrBGRb9vjJSLyoohstP8ttsdFRG4TkU0islJE5rvOdbm9/0YRubxP31RRRjjPf+cUAK59Yg2/e3ljyLYddpjpVQ8u69U5w5MWo1FRmIXPb6jT6sEJo7HNR15WajjfIX5l4jNW5bYLgDuMMb8H8ns6BvgvY8xs4DjgahGZDVwDvGyMmQa8bK8DnA1Msz9XYikw7Cm1a4FjgWOAax0FpChK/Dj+C4ey/Exy7P7h7imoZdvjn44KL/QYjXI712RvvSqTRNHY1kleZmpkv0P8yqRRRH6EFRL8tIh4gJgq0RhTbYxZbi83AuuASiyF9IC92wPAhfbyBcD/GYvFQJGIVABnAS8aYw4YYw4CLwKL4v6GiqIAVtjumz88jamj83j86hNZ8uPTeeKbJ3Xb79N/fDfuc4YXeoyGk6W9Yteh+AVWYnKwpZOS3Iyedxwg4lUmFwHtWPkmNUAV8L/xXkREJgJHAu8B5caYantTDVBuL1cCO12H7bLHoo2HX+NKEVkqIkv37dsXr2iKMqIYV5LDS9/7GPPGFSEiTLY7IYZT3xpf2RW/MXHlCDhZ8D97fHW8oiox6Ozyc6C5g7K81CilAnEqE1uBPAQUish5QJsxJqbPxEFE8oB/Ad+xEx/d5zVY/XX6jTHmTmPMAmPMgrKyskScUlGGPR6PcOtF87qNv7WxLq7j4/WZlNr94ZXEcKjFUvYluUPMZyIinwWWAP8BfBZ4T0Q+E8dx6ViK5CFjzKP28F57+gr731p7fDcwznV4lT0WbVxRlARw4ZFBQ/8n58wCYH1NQ7TdQ4jXZ+L1CF87ZTIAT63c03shlRCcgp0FKZL9DvFPc/0EONoYc7kx5jIsR/jPYh0g1uvKPcA6Y8wtrk1PAE5E1uXAv13jl9lRXccB9fZ02PPAQhEpth3vC+0xRVESxEvf+xhPfPNEvnrKZCaU5nDf29u4/sm11DbGTjQML/QYi5kVVszONx/+gEW3vtFvmUcyDW0+wEoQTRXiDVL2GGNqXev76VkRnYjlsF8lIivssR9j1fR6RESuALZjWToAz2Bl2G8CWrCTIo0xB0TkF8D79n7XG2M0+0lREog70svrERrbfdz79lb2NbVz+yVHRj3ObwyeOF9JJ5YG/TMf1TRS39JJYU7qPAyHEg22Tyt/CCYtPicizwN/tdcvwnr4R8UY8xbR63edHmF/A1wd5Vz3AvfGKauiKP3gV5+ey2f+ZEV0rdldH3Pf8EKPsZgU5ux/7INdfPHESX0TcoTTMFSnuYwxPwDuBObanzuNMf+dTMEURRkcFkwsobzAcphvqWvmzjc2R93XmPgrvhblZIRUuL0uCT3pRwoNrak3zRV3cyxjzL+MMd+zP48lUyhFUQaXf151AodVWuVQbnrmI5bvOBhxP3+MciqR+P7C6SHr+xo1ibEvBB3wqTPNFVOZiEijiDRE+DSKSHzhHoqiDDnGleTw1H+ezNWnTQFg5c7QZMPaxjZO/81rbNvfQkZa/A1bJ5SGTnWtrdbHSF9oaOvE65GU6f8OPSgTY0y+MaYgwiffGFMQ61hFUYY+3184g5wML9sPtISMP7+6hs37moHeTbWE+0021zb1X8gRSEOrj4Ks1OllAtoDXlGUGIgI40ty2LE/VJlUFAZbxaanxf9AqyzK5rQZZfzpC0cBcP1T6jfpCzUNbRTlpE4pFYg/mktRlBHKhNKcgBXi4PUGFUinL/4iFh6PcN+XjkmYbCOVVbvqOX5K6WCLEYJaJoqixGRCaS47DrTg9weVRleXtTyuJJtv2H6V3vK9My1nfIdPm7b2hg6fn72NbYwvyRlsUUJQZaIoSkzGl+TQ4fOHOMt9fksB3HnpAkb1se5WaZ41TXOwpaOHPRU3+5vbMSZY1j9VUGWiKEpMHGVx3u1vBcZ8tpWS1o+uiaV2+fS6Jg0P7g37myzl6yjjVEGViaIoMTlhqjU37y5X32UrE29/lImtpNwthJWe2Wcr31GqTBRFGUoUZKWzaM6YkFT3Tttnku7t+yPEsUycN20lPpz71dfpxWShykRRlB6ZVJbLln3N7LTzTbpsn0l/LJOy/ExENHGxt9TUtwLW/UslVJkoitIjTtvdk3/1KuDymXj7rkzys9I5clwRy7dHLtWiRGbb/hbKCzLJyUitzA5VJoqi9EhZ2JSKr8txwPfvETJ7bAHr9zZiFQ1X4qG+tZPiFEtYBFUmiqLEwVlzxpCfab0J+7r8rN1jTU2l98MyAZhRnk9jm4+ahthNuJQgjW2dKdXHxEGViaIoPeLxCNdfOAewGlv9felOAPIy+/dQm1JmNeXaEpZhr0Snqd1HfgqVnndQZaIoSlxMG2213XWXo+9vocFy2xfTU3tgJUhLexfZGalTLdhBlYmiKHExpSwPEfhgh1WO/ry5Ff0+p5PFXdugiYvx0u7zk5WmykRRlCFKdoaX8SU5LLOjrz5xxNh+nzMvM42cDC97VZnETVtnF1npqffoTj2JFEVJWdK9HnbYuSajEpTnMK44h837tK9JvLT7/GSqZaIoylDm5GmjAsszyvMTcs4FE4tZtv1goERLIthU24iva3hWI1bLRFGUIc+XT5wUWM7tZySXw6yKAprafQnrB1/b0MYZt7zBD/65MiHnSyV8XX58fqOWiaIoQ5vKIqvD4riS7B727MU5i61z7TrY0sOeoWyqbeSdzXXdxhvafAA89sHu/guXYnTY1lYqWiapl/miKErK4vEIb/zgNLIyEvcwG2crk92HWlnQi+POuOUNAF763ilMHR2ccnN6rQB8sOMgR44vToicqUBbp/XdMtNST5mknkSKoqQ040tzGJ2fuMZMY4scy6S1T8ev2RNaKNIp9QJQN8wqErf7ugDIStdpLkVRlBByMtIozc3otTI5aaoVDBDua+l0Od4b2zr7L2AKEbBMUnCaK/UkUhRlxFFZnN1rn4lTn2pvWF0vnysqbLg54QOWiTrgFUVRuuM3hjc31vUqosspNFzTEN0ySWS4cSqglomiKEoM5lYVAfCPZTtDxjt8fupbIk9VGSxFsbc+zDKxfSZzxhYARD1+KNLQan0XLfSoKIoSgRsuOIzKomx+9dx6Wju6AuPffWQFR1z/Av4IFoY/YJmEKpOrHlwGwMnTygDY2cvps1SmLtD/PbW6LIIqE0VRUgCPR/jMUVUAPPDutsD486trACtsOBzjUibu5lottjKaXJYLECj/MhxwlElpnjbHUhRFichXTray61fuOhQYqyiyQpD3RFAm2NNcHT4/hyJMZR1eWQjANx5aHlI2fyhT19RBRpon0KgslVBloihKSpCflc4p08t4ZlUNL67dC0BVUQ4AF925uNv+7pkv91TXsZNKGFuYxayKgsDYjx9dlSSpB5a6xnbK8jL73UcmGagyURQlZZg8ypqa+ur/LQVgTGEwOTLckW6MIc1jPVQ37G0MjmMlVgIU2OHDH9U0Mhyoa+5gVApOcUESlYmI3CsitSKy2jV2nYjsFpEV9ucc17YficgmEVkvIme5xhfZY5tE5JpkyasoyuAzviQnZN2dM/LQku0h2/wGppfn4xHYXBssYW+MwWO/uRfmBKOe2jq7GOrsb2qnNAWd75Bcy+R+YFGE8d8aY+bZn2cARGQ2cDEwxz7mDyLiFREv8HvgbGA2cIm9r6Iow5AyV4+Uzi4/vi4/E20ro/pQaNSWATLSPFQUZrPTlT3vNwSUSVF28C3+qZXVSZR8YKhrah95lokx5g3gQJy7XwD8zRjTbozZCmwCjrE/m4wxW4wxHcDf7H0VRRmGzK0qDCxv2NuIz2/Izkjj6InFrK0OrcFljEEEqoqz2emK2PLb4wBfP3VKYPz7//gwucInGWMM+5s6RqRlEo1vishKexrMKedZCbizlXbZY9HGuyEiV4rIUhFZum/fvmTIrShKkplQmsvvLp4HwLm3vUVLh490rzBnbCHrqhtCMtqNAQGqinNC6nq5LZNzDq9g803WbLpbUQ1FGtt9+PyGkpwRZplE4Y/AFGAeUA38JlEnNsbcaYxZYIxZUFZWlqjTKooywMwbVxRYfnvTfnYdbGXmmHxaOrpCQoQNlm+ksjibvY1tgTIqls8keD6vRzh3bkUge3yocqjZkr8oJ/Wy32GAlYkxZq8xpssY4wfuwprGAtgNjHPtWmWPRRtXFGWYMqE0lwxXv44DzR2U21FdtY1Bv4nfjzXNVZSNMVBjl1XxuxzwDpVF2eypD01uHGocbLHK6RerZQIiUuFa/STgRHo9AVwsIpkiMgmYBiwB3gemicgkEcnActI/MZAyK4oy8HzlpEkh604Drdtf2RQYMxhEJNAPZeFvrWZZlpIJVSYVhVl0+Pw88M62mNdduesQs//nuYBiSiUO2ZZVce4Is0xE5K/Au8AMEdklIlcAvxKRVSKyEjgN+C6AMWYN8AiwFngOuNq2YHzAN4HngXXAI/a+iqIMY76/cAbnHzEWgIWzy5lSlgfAkq3BmB6/7TMZXWA5pFvt0F9/2DQXBBtwXffk2pjXvf+dbbR0dPHI0p3dStsPNodsy6Qwe4RZJsaYS4wxFcaYdGNMlTHmHmPMpcaYw40xc40xnzDGVLv2v9EYM8UYM8MY86xr/BljzHR7243JkldRlNTB4xFOmFIKQEF2OiLCt06fRmtnV7DEvLGmuRxFAVYuiXE54B2qioP7+Fwl6sPJ8FqPxFte3MCxN72cqK+TEJzy/KW5I0yZKIqi9IejJljBnsdNtpRKuG/EccDnZabxySOtIM9dB1styyTsyTajPNgjvjZGzxSnkKJDKvlYtu9voSArTR3wiqIovWF6eT4r/udMPj3fUhSVtnXhVAH225YJwOePHQ9Y5eatPJNQyyTN6+G+Lx0NRCsaadHuC7VaetOsK9nUt3ZSkpuRknW5QJWJoigpTFFO8OF52NhC0jzCO5vrgNCyKePsMizLth2MOM0FVkQXwJ4YznV/mCWSSuXrWzp85GSkXrVgB1UmiqIMCQpz0pkxJp+Vu+qB0KrBZXZW+B2vborogAcrogtiWyb+MHdKKjXWam7vIjcz9Xq/O6gyURRlyDC3qpCVu+oxxmra61ggHpf2aPf5I1om+Vnp5GelUR1DmXSFWSY7D0Tfd6Bp6fCRm4J9TBxUmSiKMmSYW1VEfWsnW+qaA7W5HJwyLNX1bURzK1QWZYeUXgnHGEOJK1pqVypZJh1d5Oo0l6IoSv85ZlIJYOWbOLW5HGa7mmFFskwAJo3KZWtdc9Tz+40V+XXhPCvHJaUsk3YfORmpO82VumpOURQljMmjchmVl2kpE0LLpkwaZZVh6fD5I/pMAKaU5fHC2r10+PwhJVsc/MaQ5hVuvfhIAJZuT512v80dXTrNpSiKkghEhGMnlfDu5v2B2lwOaV4P08utTPlolsmU0bl0+Q07DkS2Tvz+0Aix6vq2mEmOA4kVzZW6lokqE0VRhhSnTB9FTUMbH9U0dMu5mG4nJza2+SIeO3mUpWw21UZRJoaAVTO+JIcuv2HJtnjbMiWP5nYfnV2GguzUTFgEVSaKogwxTp0xGgjW5nLj+E1W7DwU8diJdo/551ZH7rroNwavrU0WzhmDR+CfS3clQOr+4dQJG52fmo2xQJWJoihDjPKCLOaPt3qehM9mXWiXVdkdJfy30H6zf3zFnoilUqysegnse+LUUWze19Rtv4HGKQFTXpA1yJJER5WJoihDjmmjremscN+IUwTxuMklUY+dbFsnr3xU222b5TMJrlcV54T0lx8sHGWilomiKEoCmTDKKp/S0tEVMi4iLP7R6dz7xaOjHvv8d08hK93DO5v3d9sW3lhrXEk2B5o7aGqP7IMZKGqdaS61TBRFURLHhBLLuohU7mRMYVbMGlbpXg9tnX7ueWtrsJy9jVVxOKhM5oy1+sa/t6W74hlIahvbyUzzUJClocGKoigJY+poKyqrtqFvVX2doo/h2fDhRSKPtH0zG2sH12+yt6GN8oKslK0YDKpMFEUZgkwdnceUslyuPGVyn46/1S69ssmlJP78+ma21DWH+EwKstIpzc1g+/7oWfMDQW1De0r7S0Az4BVFGYJ4PcLL/3Vqn4+fW1VIuldYvuMgZ84uB+CXz35knTvs7X9CaU7MEiwDwZ76Vg6vLBxUGXpCLRNFUUYcmWleppTlsb6mESDEwR4+lTRtdH6IBTPQdPkNuw+2Mt7u2ZKqqDJRFGVEMmNMPh9VNwDQ2NYZGG/rDI0Qm1aeR11TBweaOwJjtQ1t1Ld2MhBU17fi8xtVJoqiKKnIjDH57Km3lIKvK5jAGJ6kOMV29v/40VUYY2jr7OKYm17mc3ctHhA5d9tBAk7b4lRFlYmiKCOSWWOs0ivrqhvocIUIh09pTbOVyXNrali9u4FVu61Oj2v2NAxIv5MaO8ekolCViaIoSsoxb5wV9rtk64EQy6QoJyNkv7Guh/ja6no6fUHFs3hL8otAVts968cUpm7CIqgyURRlhFJsl1655cUNgeTFL54wkUe/fkLIfh6PsOWmc8jPSuOpldX4XM3n1+ypT4psb22sY+cBy+rZdbCF/Kw08lK4lwmoMlEUZQQzy64y/PtXNwHwselljC/t7uj2eITDKwt5b+sBWm0HvdcjrN6deGVS19TOF+55j5N/9Sp3v7mFD3fWc9jY1A4LBlUmiqKMYC4/fgIAz66uAaxSK9H4zFFVdPj8gZyTI8cVsWZPA13+7tWH+8OhlmDU2A1Pr2PV7nrmTyhK6DWSgSoTRVFGLBcdPY50bzCvJM0bvVzJBNticRTP7LEFtHR08fyamoTK1NnVXTktmBi9CnKqoMpEUZQRi4jw5RMnBdbTYyiTeeOKKcnN4EO78dbxk0sB+MZDy/En0Dpx/Dc/PXcWv/zU4fzsvNmcMq0sYedPFqpMFEUZ0RzteuvPTo/u5PZ6hIV26RWAmba/BeD9BLb2dSyTaeX5XHLMeK44aVKg+2Mqo8pEUZQRzRkuBVFVEjuXY0pZXmA5zSPc/KnDgehtgvuCY5nEspJSEVUmiqKMeE6bUUZBVhoFWekx95tclhtY9nqEi48ZT1VxNit39S2qq8tv+PFjqwI1wiCoTDJiBAOkIkNLWkVRlCRw9+VHs/SnZ/a4X4hlYlsOR1QV8fSqanxhjbbioaahjYff28FFd74bGLv0niVA7MiyVGRoSasoipIEvB4hI63nx+E4V7FFp1T9zDFWP/q3NtVFPKbLb6IqmtYOq1rxoZbuRSOHgp/EjSoTRVGUOPF6hMe+cQLnHzE2UHblshMmArDWrkAczqm/fpXTfvNaxG1N7cEKxa+trw1ROtkZ3sQIPUAkTZmIyL0iUisiq11jJSLyoohstP8ttsdFRG4TkU0islJE5ruOudzef6OIXJ4seRVFUeLhyPHF3H7JkQHLoTA7nXEl2azZE1mZ7DzQys4DrRjTPXz4xbXBHJUnVuyhucNSLpccMy5kSm0okEzL5H5gUdjYNcDLxphpwMv2OsDZwDT7cyXwR7CUD3AtcCxwDHCto4AURVFShdkVBayNokwcnIKNbn7/6mYAFkwo5tEPdvPEit0AzK1K/Yz3cJKmTIwxbwDhwdcXAA/Yyw8AF7rG/89YLAaKRKQCOAt40RhzwBhzEHiR7gpKURRlUJkztpBt+5tDOjaGEyvi6ysnW4mTP/v3GoCU7/ceiYH2mZQbY6rt5RrACfCuBHa69ttlj0Ub74aIXCkiS0Vk6b59+xIrtaIoSgxmVxRgDIHOjW4KsqxEyKseXMa2sF7yWekevnryJM6aM4YzZgXzXeYMgcKO4QyaA95YE4gJq0FgjLnTGLPAGLOgrCz1Sw8oijJ8mFNpZcOHO+G7/IaGtqC1cv1TawPLVtdGP9npXkSE2y6Zx5iCLL57xvSU710SiYEukL9XRCqMMdX2NFatPb4bGOfar8oe2w2cGjb+2gDIqSiKEjdjCrIozklnze5QZbJ4y/6Qdb/LCd9uN9nKzrAewzkZaSz+8elJljR5DLRl8gTgRGRdDvzbNX6ZHdV1HFBvT4c9DywUkWLb8b7QHlMURUkZRIQ5Ywu7WSZfeWApADd98nDOm1sR0l++xY7cyk4fHhkayQwN/ivwLjBDRHaJyBXAzcCZIrIROMNeB3gG2AJsAu4CvgFgjDkA/AJ43/5cb48piqKkFLPHFrC+pjFQDqW+tTPQSOuEKaVMLstj54FWlu84CBDYNtTySaKRtGkuY8wlUTZ1s+Ns/8nVUc5zL3BvAkVTFEVJOHPGFtDR5WdTbROzKgr4r0dWBLZNHJXLJ4+s5LaXN/Lkh3uYP76YVscyyUjtdrzxMjzsK0VRlEFmzljLCe9YHi+ts1zCXznJCvudNCqXj88czQtr9uL3G9ocyyRdLRNFURTFZkpZHpVF2by0di91jcHWuz9YNCOwfP4RFbzyUS0//fdqHn5vB6DKRFEURXEhIhw5voinVlbz6nor1+1T8yvJTAsqi4WzxwAfBhQJ3UfrSwAACe9JREFUQHFu7LL3QwWd5lIURUkQZ7oabQEs3XYwZD03M43PLqgKrGene5lenj8gsiUbtUwURVESxCeOGEtpbiYHWjp44J1t/OfHp3bb56ZPHs600fnsqW/l2vPnDIKUyUEiVbIc6ixYsMAsXbp0sMVQFEUZUojIMmPMgr4cq9NciqIoSr9RZaIoiqL0G1UmiqIoSr9RZaIoiqL0G1UmiqIoSr9RZaIoiqL0G1UmiqIoSr9RZaIoiqL0m2GZtCgi+4DtrqFCoD5sOdLYKKCul5dzn6c3+/Q01tNyf2SOdv14toePx1of6nIn6jcSS65Y2xP1G4Hk/Lb1NzK0fiPu9Why5xpj+tb33Bgz7D/AneHLUcaW9ufcvdmnp7GelvsjczxyR9sePh5rfajLnajfSF/lTtRvpK9y629keP1GkiW38xkp01xPRliONNbfc/dmn57Gelruj8zxHB9te/h4rPWhLneifiPxHK+/kaEp91D6jbjXEyk3MEynufqKiCw1faxLM1gMRZlB5R5ohqLcQ1FmGLlyjxTLJF7uHGwB+sBQlBlU7oFmKMo9FGWGESq3WiaKoihKv1HLRFEURek3qkwURVGUfqPKRFEURek3qkziQEROFpE/icjdIvLOYMsTLyLiEZEbReR2Ebl8sOWJFxE5VUTetO/5qYMtT7yISK6ILBWR8wZblngRkVn2ff6niHx9sOWJFxG5UETuEpG/i8jCwZYnXkRksojcIyL/HGxZYmH/lh+w7/Hn4zlm2CsTEblXRGpFZHXY+CIRWS8im0TkmljnMMa8aYy5CngKeCCZ8rrk67fcwAVAFdAJ7EqWrG4SJLcBmoAsBkDuBMkM8N/AI8mRsjsJ+m2vs3/bnwVOTKa8LvkSIffjxpivAlcBFyVTXpd8iZB7izHmiuRKGpleyv8p4J/2Pf5EXBfoT8bjUPgApwDzgdWuMS+wGZgMZAAfArOBw7EUhvsz2nXcI0D+UJEbuAb4mn3sP4eQ3B77uHLgoSEi85nAxcAXgfOGyr22j/kE8CzwuaEkt33cb4D5Q1DuAfn/2A/5fwTMs/d5OJ7zpzHMMca8ISITw4aPATYZY7YAiMjfgAuMMb8EIk5RiMh4oN4Y05hEcQMkQm4R2QV02KtdyZM2SKLut81BIDMZcrpJ0L0+FcjF+o/YKiLPGGP8qS63fZ4ngCdE5Gng4eRJHLheIu63ADcDzxpjlidXYosE/7YHnN7IjzUjUAWsIM4ZrGGvTKJQCex0re8Cju3hmCuA+5ImUXz0Vu5HgdtF5GTgjWQK1gO9kltEPgWcBRQBdyRXtKj0SmZjzE8AROSLQF2yFUkMenuvT8Wa0sgEnkmqZLHp7W/7P4EzgEIRmWqM+VMyhYtBb+93KXAjcKSI/MhWOoNJNPlvA+4QkXOJs9zKSFUmvcYYc+1gy9BbjDEtWEpwSGGMeRRLEQ45jDH3D7YMvcEY8xrw2iCL0WuMMbdhPfCGFMaY/Vh+npTGGNMMfKk3xwx7B3wUdgPjXOtV9liqo3IPHENRZlC5B5qhKrdDwuQfqcrkfWCaiEwSkQwsx+kTgyxTPKjcA8dQlBlU7oFmqMrtkDj5BzqiYBAiGP4KVBMMj73CHj8H2IAVyfCTwZZT5VaZVW6VeyjLr4UeFUVRlH4zUqe5FEVRlASiykRRFEXpN6pMFEVRlH6jykRRFEXpN6pMFEVRlH6jykRRFEXpN6pMlJRARJoG4BqfiLOUfCKveaqInNCH444UkXvs5S+KyGDVKAtBRCaGlzCPsE+ZiDw3UDIpqYEqE2VYISLeaNuMMU8YY25OwjVj1bg7Fei1MgF+zBCsPQVgjNkHVIvIgPRHUVIDVSZKyiEiPxCR90VkpYj83DX+uIgsE5E1InKla7xJRH4jIh8Cx4vINhH5uYgsF5FVIjLT3i/whi8i94vIbSLyjohsEZHP2OMeEfmDiHwkIi+KyDPOtjAZXxORW0VkKfBtETlfRN4TkQ9E5CURKbfLfV8FfFdEVojVsbNMRP5lf7/3Iz1wRSQfmGuM+TDCtoki8op9b162WyMgIlNEZLH9fW+IZOmJ1T3vaRH5UERWi8hF9vjR9n34UESWiEi+fZ037Xu4PJJ1JSJeEflf19/qa67NjwNxdehThgmDneKvH/0YYwCa7H8XAncCgvWy8xRwir2txP43G1gNlNrrBvis61zbgP+0l78B3G0vfxG4w16+H/iHfY3ZWD0dAD6DVYrdA4zB6qnymQjyvgb8wbVeDIGKEl8BfmMvXwd837Xfw8BJ9vJ4YF2Ec58G/Mu17pb7SeBye/nLwOP28lPAJfbyVc79DDvvp4G7XOuFWA2RtgBH22MFWNXEc4Ase2wasNRenojdXAm4EvipvZwJLAUm2euVwKrB/l3pZ+A+WoJeSTUW2p8P7PU8rIfZG8C3ROST9vg4e3w/VuOvf4WdxylhvwyrX0ckHjdW35G1IlJuj50E/MMerxGRV2PI+nfXchXwdxGpwHpAb41yzBnAbBFx1gtEJM8Y47YkKoB9UY4/3vV9/gL8yjV+ob38MPDrCMeuAn4jIv8PeMoY86aIHA5UG2PeBzDGNIBlxWD1s5iHdX+nRzjfQmCuy3IrxPqbbAVqgbFRvoMyDFFloqQaAvzSGPPnkEGridMZwPHGmBYReQ2rRzxAmzEmvJNku/1vF9F/5+2uZYmyTyyaXcu3A7cYY56wZb0uyjEe4DhjTFuM87YS/G4JwxizQUTmYxX2u0FEXgYei7L7d4G9wBFYMkeSV7AswOcjbMvC+h7KCEF9Jkqq8TzwZRHJAxCRShEZjfXWe9BWJDOB45J0/beBT9u+k3IsB3o8FBLsA3G5a7wRyHetv4DVJRAA+80/nHXA1CjXeQerTDhYPok37eXFWNNYuLaHICJjgRZjzIPA/2L1A18PVIjI0fY++XZAQSGWxeIHLsXqFR7O88DXRSTdPna6bdGAZcnEjPpShheqTJSUwhjzAtY0zbsisgr4J9bD+DkgTUTWYfX+XpwkEf6FVZ57LfAgsByoj+O464B/iMgyoM41/iTwSccBD3wLWGA7rNcSoeueMeYjrHa0+eHbsBTRl0RkJdZD/tv2+HeA79njU6PIfDiwRERWANcCNxhjOoCLsNo7fwi8iGVV/AG43B6bSagV5nA31n1abocL/5mgFXga8HSEY5RhipagV5QwHB+GWP26lwAnGmNqBliG7wKNxpi749w/B2g1xhgRuRjLGX9BUoWMLc8bwAXGmIODJYMysKjPRFG685SIFGE50n8x0IrE5o/Af/Ri/6OwHOYCHMKK9BoURKQMy3+kimQEoZaJoiiK0m/UZ6IoiqL0G1UmiqIoSr9RZaIoiqL0G1UmiqIoSr9RZaIoiqL0G1UmiqIoSr/5/03V0tkb+BK2AAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learner.lr_plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## STEP 4: Train and Inspect the Model" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "begin training using onecycle policy with max lr of 0.03...\n", "Train on 95646 samples, validate on 23912 samples\n", "Epoch 1/10\n", "95646/95646 [==============================] - 8s 79us/sample - loss: 1556.0435 - mae: 19.2369 - val_loss: 984.7442 - val_mae: 15.1122\n", "Epoch 2/10\n", "95646/95646 [==============================] - 8s 79us/sample - loss: 1052.5454 - mae: 13.0505 - val_loss: 808.2142 - val_mae: 12.5382\n", "Epoch 3/10\n", "95646/95646 [==============================] - 7s 76us/sample - loss: 809.7949 - mae: 9.4578 - val_loss: 695.8532 - val_mae: 10.8098\n", "Epoch 4/10\n", "95646/95646 [==============================] - 8s 80us/sample - loss: 616.9707 - mae: 6.6427 - val_loss: 621.5498 - val_mae: 9.9253\n", "Epoch 5/10\n", "95646/95646 [==============================] - 7s 78us/sample - loss: 471.5737 - mae: 4.8021 - val_loss: 582.4865 - val_mae: 9.9948\n", "Epoch 6/10\n", "95646/95646 [==============================] - 8s 79us/sample - loss: 369.3043 - mae: 4.1017 - val_loss: 572.5836 - val_mae: 10.4219\n", "Epoch 7/10\n", "95646/95646 [==============================] - 8s 79us/sample - loss: 304.9035 - mae: 3.7351 - val_loss: 563.6406 - val_mae: 10.3136\n", "Epoch 8/10\n", "95646/95646 [==============================] - 8s 79us/sample - loss: 257.2997 - mae: 2.8500 - val_loss: 562.7244 - val_mae: 10.0789\n", "Epoch 9/10\n", "95646/95646 [==============================] - 8s 79us/sample - loss: 226.9375 - mae: 1.9855 - val_loss: 559.9848 - val_mae: 9.7024\n", "Epoch 10/10\n", "95646/95646 [==============================] - 8s 79us/sample - loss: 211.9495 - mae: 1.3842 - val_loss: 561.2627 - val_mae: 9.6977\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learner.fit_onecycle(0.03, 10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our MAE is roughly 10, which means our model's predictions are about $10 off on average. This isn't bad considering there is a wide range of wine prices and predictions are being made purely from text descriptions. \n", "\n", "Let's examine the wines we got the most wrong." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "----------\n", "id:6695 | loss:675000.75 | true:980.0 | pred:158.42)\n", "\n", "this was a great vintage port year and this white port which was bottled in 2015 has hints of the firm tannins and structure that marked out the year it also has preserved an amazing amount of freshness still suggesting orange marmalade flavors these are backed up by the fine concentrated old wood tastes the wine is of course ready to drink\n", "----------\n", "id:19469 | loss:524528.9 | true:775.0 | pred:50.76)\n", "\n", "perfumed florals mingle curiously with deep dusty mineral notes on this bracing tba sunny nectarine and tangerine flavors are mouthwatering and juicy struck with acidity then plunged into of sweet honey and nectar it's a delightful sensory roller coaster that feels endless on the finish\n", "----------\n", "id:3310 | loss:400394.03 | true:848.0 | pred:215.23)\n", "\n", "full of ripe fruit opulent and concentrated this is a fabulous and impressive wine it has a beautiful line of acidity balanced with ripe fruits the wood aging is subtle just a hint of smokiness and toast this is one of those wines from a great white wine vintage that will age many years drink from 2024\n" ] } ], "source": [ "learner.view_top_losses(n=3, preproc=preproc)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It looks like our model has trouble with expensive wines, which is understandable given the descriptions of them, which may not differ much from less expensive wines." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## STEP 5: Making Predictions" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "predictor = ktrain.get_predictor(learner.model, preproc)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's make a prediction for a random wine in the validation set." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Description: This Millesimato sparkling blend of Pinot Nero and oak-aged Chardonnay delivers a generous and creamy mouthfeel followed by refined aromas of dried fruit and baked bread. This is a beautiful wine to serve with tempura appetizers.\n", "Actual Price: 52.0\n" ] } ], "source": [ "idx = np.random.randint(len(x_test))\n", "print('Description: %s' % (x_test[idx]))\n", "print('Actual Price: %s' % (y_test[idx]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our prediction for this wine:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([52.698753], dtype=float32)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "predictor.predict(x_test[idx])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using the Transfomer API for Text Regression\n", "\n", "*ktrain* includes a simplified interface to the Hugging Face transformers library. This interface can also be used for text regression. Here is a short example of training a [DistilBERT model](https://arxiv.org/abs/1910.01108) for a single epoch to predict wine prices." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "preprocessing train...\n", "language: en\n", "train sequence lengths:\n", "\tmean : 41\n", "\t95percentile : 61\n", "\t99percentile : 73\n" ] }, { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "preprocessing test...\n", "language: en\n", "test sequence lengths:\n", "\tmean : 41\n", "\t95percentile : 62\n", "\t99percentile : 73\n" ] }, { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "begin training using onecycle policy with max lr of 0.0001...\n", "Train for 748 steps, validate for 187 steps\n", "748/748 [==============================] - 310s 415ms/step - loss: 1443.0076 - mae: 18.3470 - val_loss: 879.1416 - val_mae: 13.9600\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "MODEL_NAME = 'distilbert-base-uncased'\n", "t = text.Transformer(MODEL_NAME, maxlen=75)\n", "trn = t.preprocess_train(x_train, y_train)\n", "val = t.preprocess_test(x_test, y_test)\n", "model = t.get_regression_model()\n", "learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=128)\n", "learner.fit_onecycle(1e-4, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### For the Prediction part as discussed in the [issue](https://github.com/amaiya/ktrain/issues/417) ###" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "p = ktrain.get_predictor(learner.model, t)\n", "p.predict(['This is first document.', 'This is second document.', 'This is third document.'])" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 2 }