{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%reload_ext autoreload\n", "%autoreload 2\n", "%matplotlib inline\n", "import os\n", "os.environ[\"CUDA_DEVICE_ORDER\"]=\"PCI_BUS_ID\";\n", "os.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0\"" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tensorflow version 2.1.0\n" ] } ], "source": [ "import numpy as np\n", "import tensorflow as tf\n", "print(\"Tensorflow version \" + tf.__version__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Using *ktrain* to Facilitate a Normal TensorFlow Workflow\n", "\n", "This example notebook simply illustrates how *ktrain* can be used in a **minimally-invasive** way within\n", "a normal TensorFlow workflow. In this notebook, we will store our datasets in the form of `tf.Datasets` and build our own `tf.Keras` model following the example of TensorFlow's [Keras MNIST TPU.ipynb](https://colab.research.google.com/github/tensorflow/tpu/blob/master/tools/colab/keras_mnist_tpu.ipynb#scrollTo=cCpkS9C_H7Tl). We will then simply use **ktrain** as a lightweight wrapper for our model and data to estimate a learning rate, train the model, inspect the model, and make predictions." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Detect Hardware: CPU vs. GPU vs. TPU" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Running on single GPU /device:GPU:0\n", "Number of accelerators: 1\n" ] } ], "source": [ "# Detect hardware\n", "try:\n", " tpu = tf.distribute.cluster_resolver.TPUClusterResolver() # TPU detection\n", "except ValueError:\n", " tpu = None\n", " gpus = tf.config.experimental.list_logical_devices(\"GPU\")\n", " \n", "# Select appropriate distribution strategy\n", "if tpu:\n", " tf.tpu.experimental.initialize_tpu_system(tpu)\n", " strategy = tf.distribute.experimental.TPUStrategy(tpu, steps_per_run=128) # Going back and forth between TPU and host is expensive. Better to run 128 batches on the TPU before reporting back.\n", " print('Running on TPU ', tpu.cluster_spec().as_dict()['worker']) \n", "elif len(gpus) > 1:\n", " strategy = tf.distribute.MirroredStrategy([gpu.name for gpu in gpus])\n", " print('Running on multiple GPUs ', [gpu.name for gpu in gpus])\n", "elif len(gpus) == 1:\n", " strategy = tf.distribute.get_strategy() # default strategy that works on CPU and single GPU\n", " print('Running on single GPU ', gpus[0].name)\n", "else:\n", " strategy = tf.distribute.get_strategy() # default strategy that works on CPU and single GPU\n", " print('Running on CPU')\n", "print(\"Number of accelerators: \", strategy.num_replicas_in_sync)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prepare Training and Validation Data as `tf.Datasets`\n", "\n", "Download the dataset files from [LeCun's website](http://yann.lecun.com/exdb/mnist/)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "BATCH_SIZE = 64 * strategy.num_replicas_in_sync # Gobal batch size.\n", "training_images_file = 'data/mnist_lecun/train-images-idx3-ubyte'\n", "training_labels_file = 'data/mnist_lecun/train-labels-idx1-ubyte'\n", "validation_images_file = 'data/mnist_lecun/t10k-images-idx3-ubyte'\n", "validation_labels_file = 'data/mnist_lecun/t10k-labels-idx1-ubyte'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that, if training using a TPU, these should be set as follows:\n", "\n", "```python\n", "training_images_file = 'gs://mnist-public/train-images-idx3-ubyte'\n", "training_labels_file = 'gs://mnist-public/train-labels-idx1-ubyte'\n", "validation_images_file = 'gs://mnist-public/t10k-images-idx3-ubyte'\n", "validation_labels_file = 'gs://mnist-public/t10k-labels-idx1-ubyte'\n", "```\n", "\n", "You may need to authenticate:\n", "```python\n", "IS_COLAB_BACKEND = 'COLAB_GPU' in os.environ # this is always set on Colab, the value is 0 or 1 depending on GPU presence\n", "if IS_COLAB_BACKEND:\n", " from google.colab import auth\n", " # Authenticates the Colab machine and also the TPU using your\n", " # credentials so that they can access your private GCS buckets.\n", " auth.authenticate_user()\n", "```" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "def read_label(tf_bytestring):\n", " label = tf.io.decode_raw(tf_bytestring, tf.uint8)\n", " label = tf.reshape(label, [])\n", " label = tf.one_hot(label, 10)\n", " return label\n", " \n", "def read_image(tf_bytestring):\n", " image = tf.io.decode_raw(tf_bytestring, tf.uint8)\n", " image = tf.cast(image, tf.float32)/255.0\n", " image = tf.reshape(image, [28*28])\n", " return image\n", " \n", "def load_dataset(image_file, label_file):\n", " imagedataset = tf.data.FixedLengthRecordDataset(image_file, 28*28, header_bytes=16)\n", " imagedataset = imagedataset.map(read_image, num_parallel_calls=16)\n", " labelsdataset = tf.data.FixedLengthRecordDataset(label_file, 1, header_bytes=8)\n", " labelsdataset = labelsdataset.map(read_label, num_parallel_calls=16)\n", " dataset = tf.data.Dataset.zip((imagedataset, labelsdataset))\n", " return dataset \n", " \n", "def get_training_dataset(image_file, label_file, batch_size):\n", " dataset = load_dataset(image_file, label_file)\n", " dataset = dataset.cache() # this small dataset can be entirely cached in RAM\n", " dataset = dataset.shuffle(5000, reshuffle_each_iteration=True)\n", " dataset = dataset.repeat() # Mandatory for Keras for now\n", " dataset = dataset.batch(batch_size, drop_remainder=True) # drop_remainder is important on TPU, batch size must be fixed\n", " dataset = dataset.prefetch(-1) # fetch next batches while training on the current one (-1: autotune prefetch buffer size)\n", " return dataset\n", " \n", "def get_validation_dataset(image_file, label_file):\n", " dataset = load_dataset(image_file, label_file)\n", " dataset = dataset.cache() # this small dataset can be entirely cached in RAM\n", " dataset = dataset.batch(10000, drop_remainder=True) # 10000 items in eval dataset, all in one batch\n", " dataset = dataset.repeat() # Mandatory for Keras for now\n", " return dataset\n", "\n", "def load_label_dataset(label_file):\n", " labelsdataset = tf.data.FixedLengthRecordDataset(label_file, 1, header_bytes=8)\n", " labelsdataset = labelsdataset.map(read_label, num_parallel_calls=16)\n", " return labelsdataset \n", "\n", "# instantiate the datasets\n", "training_dataset = get_training_dataset(training_images_file, training_labels_file, BATCH_SIZE)\n", "validation_dataset = get_validation_dataset(validation_images_file, validation_labels_file)\n", "\n", "# exract ground truth labels\n", "training_labels = np.vstack(list(load_label_dataset(training_labels_file).as_numpy_iterator()))\n", "validation_labels = np.vstack(list(load_label_dataset(validation_labels_file).as_numpy_iterator()))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Build a Model" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# This model trains to 99.4% accuracy in 10 epochs (with a batch size of 64) \n", "\n", "def make_model():\n", " model = tf.keras.Sequential(\n", " [\n", " tf.keras.layers.Reshape(input_shape=(28*28,), target_shape=(28, 28, 1), name=\"image\"),\n", "\n", " tf.keras.layers.Conv2D(filters=12, kernel_size=3, padding='same', use_bias=False), # no bias necessary before batch norm\n", " tf.keras.layers.BatchNormalization(scale=False, center=True), # no batch norm scaling necessary before \"relu\"\n", " tf.keras.layers.Activation('relu'), # activation after batch norm\n", "\n", " tf.keras.layers.Conv2D(filters=24, kernel_size=6, padding='same', use_bias=False, strides=2),\n", " tf.keras.layers.BatchNormalization(scale=False, center=True),\n", " tf.keras.layers.Activation('relu'),\n", "\n", " tf.keras.layers.Conv2D(filters=32, kernel_size=6, padding='same', use_bias=False, strides=2),\n", " tf.keras.layers.BatchNormalization(scale=False, center=True),\n", " tf.keras.layers.Activation('relu'),\n", "\n", " tf.keras.layers.Flatten(),\n", " tf.keras.layers.Dense(200, use_bias=False),\n", " tf.keras.layers.BatchNormalization(scale=False, center=True),\n", " tf.keras.layers.Activation('relu'),\n", " tf.keras.layers.Dropout(0.4), # Dropout on dense layer only\n", "\n", " tf.keras.layers.Dense(10, activation='softmax')\n", " ])\n", "\n", " model.compile(optimizer='adam', # learning rate will be set by LearningRateScheduler\n", " loss='categorical_crossentropy',\n", " metrics=['accuracy'])\n", " return model\n", " \n", "with strategy.scope():\n", " model = make_model()\n", "\n", "\n", "# set up learning rate decay [FROM ORIGINAL EXAMPLE BUT NOT USED]\n", "# NOT NEEDED: we will use ktrain to find LR and decay learning rate during training\n", "LEARNING_RATE = 0.01\n", "LEARNING_RATE_EXP_DECAY = 0.6 if strategy.num_replicas_in_sync == 1 else 0.7\n", "lr_decay = tf.keras.callbacks.LearningRateScheduler(\n", " lambda epoch: LEARNING_RATE * LEARNING_RATE_EXP_DECAY**epoch,\n", " verbose=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use *ktrain* With Our Model and Data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Wrap tf.Datasets in a `ktrain.TFDataset` wrapper and create `Learner`" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/amaiya/projects/ghub/ktrain/ktrain/data.py:86: UserWarning: batch_size parameter is ignored, as pre-configured batch_size of tf.data.Dataset is used\n", " warnings.warn('batch_size parameter is ignored, as pre-configured batch_size of tf.data.Dataset is used')\n" ] } ], "source": [ "import ktrain\n", "trn = ktrain.TFDataset(training_dataset, n=training_labels.shape[0], y=training_labels)\n", "val = ktrain.TFDataset(validation_dataset, n=validation_labels.shape[0], y=validation_labels)\n", "learner = ktrain.get_learner(model, train_data=trn, val_data=val)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Find Learning Rate" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "simulating training for different learning rates... this may take a few moments...\n", "Train for 937 steps\n", "Epoch 1/1024\n", "937/937 [==============================] - 8s 8ms/step - loss: 1.8162 - accuracy: 0.4173\n", "Epoch 2/1024\n", "604/937 [==================>...........] - ETA: 2s - loss: 0.2286 - accuracy: 0.9345\n", "\n", "done.\n", "Visually inspect loss plot and select learning rate associated with falling loss\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU9b3/8dcnG1nIQiCBkAQiO0EWMSBoVXBFtLUudW+1tRepXrdut3axq9d6+6u2LnWttVit+4IK7htWRMIeFjFsEggQtiSEhGzf3x8zYsQEEpKTM5N5Px+PeXDmLDNvhjDvnDlzvsecc4iISOSK8juAiIj4S0UgIhLhVAQiIhFORSAiEuFUBCIiEU5FICIS4WL8DtBWvXr1cnl5eX7HEBEJKwsWLNjunMtoblnYFUFeXh6FhYV+xxARCStmtqGlZfpoSEQkwqkIREQinIpARCTCqQhERCKcikBEJMKpCEREIlxEFcGGHVVU1tT5HUNEJKRETBE0NjpO/NO7XHj/RwA8t7CE/5pRyGPzWvxqbaut217Fxp17m122Z189y0rKD7p9fUNjuzOIiByusDuh7HDNmLsegBWlFfz8+WU8Pu8zAN5YsZXfv7yCWdcdz4CM7gA88P4aHv5gPUdmp/KNMX0Z0TeFvJ5J1DU08uOnl5AYF81Zo/oSGx3Fvz7awCvLSgFIjo/htPw+JMRFcXT/HkwemsnJf36PHVW19O+ZSEH/dG47byQx0V/07/rtVZx11wc0OsdPTx9KQlw0RZsqGJmdynGDe5GdloBzDjPr1NdLRCKHhdsVygoKCtzhnFlcXdvA8JtfbddzJ8VFU1Xb0Obt8nomckSvJN75pKzN28ZFR1Eb3GP478mDuGbyIBLiotv8OCIS2cxsgXOuoNllkVIEAKu2VDDlL3MAuHh8LreeO4p126v41QtFbKusYfXWPQDkZ6Vw32VHM6e4jPnrdvL2qm1U1NQD8I3RfRmTm8ayTeXk9kjgjJFZDM9Koa6hkdr6Rh75cD0Zyd0o2bmXlVsqOW9sDlOO7INzjsv+Po//FO/4Sq6bzhhGQV4PVpRW0ispjlPye/PSks08u7CE9dv3sml39f510xJjefnar1Gyq5qC/j2+tHchItISFUETrxaVMia3B31S47+yrKHRsW57Ff17JhLb5A3WOUdFTT3vrNrG6SP6HPZv5Luqalm0cRcTBvSkrsGRmhBLY6MjKurgH/s0Njoqa+q5dfZKnpi/sdl1+qbG88fzRnHCkGbHlBKRCKci6EKeX1TC/e+tJTk+htLyGkp2BfYWkrvFUF3XwNWTBpLfN4XTR/TRcQUR2U9F0IU55yir3Ed9Y+Cjp7VlVQD06h7HDyYN4syRWc3u/YhIZFERRIidVbXc+danbNy5l4/W7th/YPu0/N5MHZnFmNw0zKB/zySfk4pIZ1MRRCDnHM8t3MSPnl7ypflpibG895PJpCbE+pRMRPygIohgW8prWFlaweyiUnZW1fHmyq2kJ8URHWXcfsFojh+sg8sikUBFIPu9tnwLf3nzU1aWVgDQMymOaScMYHhWCgMyksjpkehzQhHxgopAvmJN2R5O/vN7X5k/56eTyU1XGYh0NSoCaVZdQyMrNlfwydZKfvH8MuoavvhZePA7BZya39vHdCLSkQ5WBDotNYLFRkcxOjeNCwpy+fSWqdx0xrD9y65+bAHF2yp9TCcinUVFIPtddeJAFv7qVP526VjiY6OZ9ugCyqs1bLdIV6cikC9JT4pj6sgs7rlkLGvLqhj929cpq9zndywR8ZCKQJp1wpAMLpvQD4Dz7/uQFxdv8jmRiHhFRSAt+sM3R3LfZUezYcdern9iMS8sUhmIdEUqAjmoKUf2Yc5PJ5OeFMcNTy7mo7VfHUZbRMKbikAOKTc9kUe+Ow6Ayx6ax/3vrfE5kYh0JBWBtMqonDTe/fEk6hsdt85exfH/9zYlu5q/TrOIhBfPisDMcs3sHTNbYWbLzez6ZtaZZGblZrY4eLvZqzzSfnm9kvj45yfTLz2RjTur+dpt7/D2qq1+xxKRdvJyj6Ae+JFzLh+YAFxjZvnNrDfHOTcmePudh3mkA2SmxPP+Tyfz4jXHkRAbzfceKeS15Vv8jiUi7eBZETjnSp1zC4PTlcBKINur55PONTo3jSevmgDAVY8u4PF5n/mcSEQOV6ccIzCzPOAoYF4ziyea2RIzm21mI1rYfpqZFZpZYVlZmYdJpS1G5aTxzPSJAPz8+WXUNTT6nEhEDofnRWBm3YFngRuccxUHLF4I9HfOjQbuAl5o7jGccw845wqccwUZGRo/P5QU5KXzh28eCcAT8zf6nEZEDoenRWBmsQRK4DHn3HMHLnfOVTjn9gSnZwGxZtbLy0zS8S49ph+jclL51QtF/Om1VX7HEZE28vJbQwb8HVjpnLu9hXX6BNfDzMYH8+iMpTBjFrjaGcA976zZf9EbEQkPXu4RHAd8GzipyddDp5rZdDObHlznfKDIzJYAdwIXuXC7QIIAMCgzmcU3n0paYiw3PbeMxkb9M4qEixivHtg59wFgh1jnbuBurzJI50pLjOOXZ+bz46eX8Ni8DXx7Yp7fkUSkFXRmsXSob47pS0H/HvzqxeW8uUInm4mEAxWBdKiY6CjuuXQssdHG92cU8tzCEr8jicghqAikw/VOieeR744H4Dczl7OtosbnRCJyMCoC8cRxg3rxxo0nUFFTzz3vFPsdR0QOQkUgnhncO5ljB/bkn3M3MOdTnREuEqpUBOKpuy4+CoB739U1DERClYpAPNWzezduOmMYH67ZQdGmcr/jiEgzVATiuYuP6UdaYiyXP/wx2/fs8zuOiBxARSCeS4mP5dZzRrKjqpYL7p+LTh4XCS0qAukUZ4zMYurIPqwtq+LpBTq3QCSUqAik09x23ihG56bxi+eXsXl3td9xRCRIRSCdJjk+lnsuOYqGRse0Rwv1EZFIiFARSKfK6ZHIuWNzKNpUwR1vfup3HBFBRSA+uPXckQzK7M59765h3fYqv+OIRDwVgXS62OgoHv/+McTFRHHtvxfq2gUiPlMRiC8yU+L53nF5FG2q4KEP1vodRySiqQjENzeeOoRjjkjnzreKqalr8DuOSMRSEYhvzIyrJw9iz756Xlu+xe84IhFLRSC+OnZgT3p1j+P6Jxazemul33FEIpKKQHwVGx3FP64IXMTmyfkbfU4jEplUBOK7kTmpTB3Zh2cWlFBRU+d3HJGIoyKQkHD1pEGUV9fxz/+s9zuKSMRREUhIODI7lVOG9+bBOWspr9ZegUhnUhFIyLjx1MHs2VfP/e/pamYinUlFICFjRN9Uju7fg7dWbtPZxiKdSEUgIeXCcf34ZGslLy8r9TuKSMRQEUhIOXtMX7LTErht9irtFYh0EhWBhJTY6Ch+cvpQNu2uZu7aHX7HEYkInhWBmeWa2TtmtsLMlpvZ9c2sY2Z2p5kVm9lSMxvrVR4JH1OO7ENqQiz3vFPsdxSRiODlHkE98CPnXD4wAbjGzPIPWOcMYHDwNg2418M8EibiY6P5/teO4MM1O5i7RnsFIl7zrAicc6XOuYXB6UpgJZB9wGpnAzNcwEdAmplleZVJwsflx+XRMymOO95c7XcUkS6vU44RmFkecBQw74BF2UDTAWZK+GpZYGbTzKzQzArLysq8iikhJCU+lh9MGsjH63ay8LNdfscR6dI8LwIz6w48C9zgnKs4nMdwzj3gnCtwzhVkZGR0bEAJWReP70dqQiz3vasTzES85GkRmFksgRJ4zDn3XDOrbAJym9zPCc4TIalbDJdP7M/rK7ZSvE1DVIt4xctvDRnwd2Clc+72FlabCXwn+O2hCUC5c05nEsl+lx+bR7eYKGbM3eB3FJEuy8s9guOAbwMnmdni4G2qmU03s+nBdWYBa4Fi4EHgag/zSBjq2b0bJw7J4PXlW6lvaPQ7jkiXFOPVAzvnPgDsEOs44BqvMkjXcN7ROby+Yiuvr9jK1JH6UplIR9OZxRLyThnem17du/HKUn1qKOIFFYGEvOgo4/QRvXl71Tb21tb7HUeky1ERSFg4a1RfqusaeH6RvlQm0tFUBBIWjjkinfFHpHPb7FVsq6jxO45Il6IikLAQFWXcdt4oauoauVdXMBPpUCoCCRtH9EpiwsCevLe6jMAXzkSkI6gIJKyclt+btWVVLCkp9zuKSJehIpCw8vVRfYmNNp4u3HjolUWkVVQEElZSE2M5/+gcni4soby6zu84Il2CikDCzkXj+lHb0MhrRVv8jiLSJagIJOyMykklr2cij83boIPGIh1ARSBhx8y48mtHsKSknKU6aCzSbioCCUtnH5VNt5gonl6gg8Yi7aUikLCUEh/L1JFZPDW/hI079/odRySsqQgkbP3w1CE0OMeMuev9jiIS1lQEErZy0xOZPDSTWcu26KCxSDuoCCSsnTg0g027q1m/Qx8PiRwuFYGEteMH9QLg9eU6p0DkcKkIJKz175nI6JxU/v3xZ/p4SOQwqQgkrJkZF47rx/ode1lZWul3HJGwpCKQsHf6iN5ERxmzlumaxiKHQ0UgYa9n924cO7Anrywr1cdDIodBRSBdwpkjs1i3vYrlmyv8jiISdlQE0iWcPqIP0VHGK/p4SKTNVATSJfRIigt8PLRUHw+JtJWKQLqMKUf24bOde1m1Rd8eEmmLVhWBmV1vZikW8HczW2hmp3kdTqQtTsvvQ1JcNP87a6XfUUTCSmv3CL7nnKsATgN6AN8G/niwDczsYTPbZmZFLSyfZGblZrY4eLu5TclFDpCR3I0bTx3CnE+3s+izXX7HEQkbrS0CC/45FXjUObe8ybyWPAJMOcQ6c5xzY4K337Uyi0iLLhrfj+T4GB76YJ3fUUTCRmuLYIGZvU6gCF4zs2Sg8WAbOOfeB3a2M59Im3TvFsMl4/sxe1kppeXVfscRCQutLYIrgZ8B45xze4FY4Lsd8PwTzWyJmc02sxEd8HgiXDy+H40OZi7e7HcUkbDQ2iKYCHzinNttZpcBvwTae7HYhUB/59xo4C7ghZZWNLNpZlZoZoVlZWXtfFrp6vJ6JTE6N40XVAQirdLaIrgX2Gtmo4EfAWuAGe15YudchXNuT3B6FhBrZr1aWPcB51yBc64gIyOjPU8rEeKcMX1ZWVrB6q36KqnIobS2COpd4Cyds4G7nXP3AMnteWIz62NmFpweH8yyoz2PKfK5s0b3JTrK9PGQSCu0tggqzewmAl8bfcXMoggcJ2iRmf0bmAsMNbMSM7vSzKab2fTgKucDRWa2BLgTuMjplFDpIL26d2N8XrpGJBVphZhWrnchcAmB8wm2mFk/4E8H28A5d/Ehlt8N3N3K5xdps6kj+/CrF5fz+vItnDaij99xREJWq/YInHNbgMeAVDM7C6hxzrXrGIGI1y4a34+M5G78a95nfkcRCWmtHWLiAuBj4FvABcA8Mzvfy2Ai7RUbHcXF4/sx59My1pbt8TuOSMhq7TGCXxA4h+By59x3gPHAr7yLJdIxLjumH3HRUfzzw/V+RxEJWa0tgijn3LYm93e0YVsR32SmxHP84AzeXLlNw1OLtKC1b+avmtlrZnaFmV0BvALM8i6WSMc5Lb83m3ZX6+plIi1o7cHinwAPAKOCtwecc//jZTCRjnJKfm9io41H9PGQSLNa/fGOc+5Z59wPg7fnvQwl0pHSk+I4/+hcXl66mT376v2OIxJyDloEZlZpZhXN3CrNTPvZEjbOPzqHmrpGZusEM5GvOGgROOeSnXMpzdySnXMpnRVSpL3G9ksjr2cizy3c5HcUkZCjb/5IRDAzzh2bw9y1O9i0W9cpEGlKRSAR45yjsgF4YZH2CkSaUhFIxMhNT2R8XjrPLizROQUiTagIJKJ8qyCHtWVVvLtaFzgS+ZyKQCLKN4/KJjc9gTveWK29ApEgFYFElNjoKK49aTBLS8p5c+W2Q28gEgFUBBJxzj0qm7yeidz+xmoaG7VXIKIikIgTEx3F9acMZmVpBa8u3+J3HBHfqQgkIn1jdDYDM5K4++1iv6OI+E5FIBEpOsq4bEJ/VpRWsEYXrZEIpyKQiHXGkVkAzFqq8YcksqkIJGL1SY1nwoB0nizcSF1Do99xRHyjIpCI9l/HD6BkVzUz5m7wO4qIb1QEEtFOGpbJ8YN7cffbn1JT1+B3HBFfqAgkopkZ008cyK69dcwu0rECiUwqAol4Ewf0ZEBGEg/NWadhJyQiqQgk4kVFBfYKlm+uYO7aHX7HEel0KgIR4Ouj+pIQG83sZTrTWCKPikAESIiLZvKwDF5dvoUGjT8kEcazIjCzh81sm5kVtbDczOxOMys2s6VmNtarLCKtcdaovpRV7uPBOWv9jiLSqbzcI3gEmHKQ5WcAg4O3acC9HmYROaRThvdmZHYqt7+xmu179vkdR6TTeFYEzrn3gZ0HWeVsYIYL+AhIM7Msr/KIHEpcTBR/uWgMtfWN/OsjnWAmkcPPYwTZwMYm90uC877CzKaZWaGZFZaV6RKD4p2BGd2ZNDSDx+d9RtW+er/jiHSKsDhY7Jx7wDlX4JwryMjI8DuOdHFXnTCQbZX7eKpw46FXFukC/CyCTUBuk/s5wXkivpowIJ2j+/fgnneKqayp8zuOiOf8LIKZwHeC3x6aAJQ753SOv/jOzPjZGcPYvqeWR/6z3u84Ip7z8uuj/wbmAkPNrMTMrjSz6WY2PbjKLGAtUAw8CFztVRaRthqXl86JQzKY8dEGaus1RLV0bTFePbBz7uJDLHfANV49v0h7XXFcHt/9x3yeX1TCheP6+R1HxDNhcbBYxA+ThmQwKieVu94u1l6BdGkqApEWmBk3nDKYkl3VvLBI32OQrktFIHIQk4dmMjwrhfvfX6MxiKTLUhGIHISZce1Jg1hTVqW9AumyVAQihzBlRB+G9O7OPe8Ua69AuiQVgcghfH7hmrXbq3j3k21+xxHpcCoCkVaYOjKL/j0TufnF5foGkXQ5KgKRVoiPjeY3Xx/Bpt3VPLOgxO84Ih1KRSDSSpOGZjC2Xxp/fWu1RiaVLkVFINJKZsbPpw6nrHIfP3tuGYGT40XCn4pApA0K8tL50WlDeWnJZp7WR0TSRagIRNroBycO5Oj+PfjpM0tZtaXC7zgi7aYiEGmjqCjj+pMHAzDlL3N4c8VWnxOJtI+KQOQwnDAkg4evKADghicXs6uq1udEIodPRSBymE4a1psXrzmOmroGbp653O84IodNRSDSDqNz07jhlMG8tGQzLy/d7HcckcOiIhBpp+knDmR0Tiq/fKFIHxFJWFIRiLRTTHQUfzxvFLv31vHYvA1+xxFpMxWBSAcYnpXCCUMy+Md/1muvQMKOikCkg/zPlKHsrq7jr2996ncUkTZREYh0kBF9U7mgIIfH5m1g4869fscRaTUVgUgHuu7kwXSLiebGJxdT36DhqiU8qAhEOlBWagK3nHMkhRt28cOnlvgdR6RVVAQiHezsMdlcNqEfM5ds5sM12/2OI3JIKgIRD/zyzHyy0xL4w8srdUUzCXkqAhEPxMdG84szh7OitIKbXyzStQskpKkIRDwydWQWVxybxxPzN/L/Xv/E7zgiLYrxO4BIV3bzWflU1tRz/3trGZPbg1Pze/sdSeQrPN0jMLMpZvaJmRWb2c+aWX6FmZWZ2eLg7fte5hHpbFFRxs1n5TOkdzI/+NcCFmzY6Xckka/wrAjMLBq4BzgDyAcuNrP8ZlZ90jk3Jnh7yKs8In5JTYzlke+NIy0xliv/WUjRpnK/I4l8iZd7BOOBYufcWudcLfAEcLaHzycSsjKT43n0ymOIj4nmgvvn8tryLX5HkjDinOO91WWefQPNyyLIBjY2uV8SnHeg88xsqZk9Y2a5zT2QmU0zs0IzKywrK/Miq4jnhmelMPO/jyO3RyJXPbqAp+ZvPPRGIsAdb6zm8oc/5n9nrfTk8f3+1tBLQJ5zbhTwBvDP5lZyzj3gnCtwzhVkZGR0akCRjpSZEs9zVx/LhAHp/PKFIoq37fE7koSwPfvqOe6Pb3Pn28UATBzY05Pn8bIINgFNf8PPCc7bzzm3wzm3L3j3IeBoD/OIhISkbjHcdfFYErtFM21GIeXVdX5HkhD1+LwNbNpdTVx0FJ/ecganj+jjyfN4WQTzgcFmdoSZxQEXATObrmBmWU3ufgPwZr9HJMRkJHfj3kuP5rOdezn77g8oq9x36I0kojjnuPvtYnJ6JLDq91OIjfbu7dqzR3bO1QP/DbxG4A3+KefccjP7nZl9I7jadWa23MyWANcBV3iVRyTUTBzYk4evGMem3dVc8uBH7NlX73ckCSF3vLGaipp6LizIJSrKPH0uC7dT3wsKClxhYaHfMUQ6zDufbOPKR+Zz9phsbr9gNGbe/qeX0OecY9wtb7J9Ty0rfnc6iXHtP/fXzBY45wqaW+b3wWKRiDd5aCbXnTyY5xdt4tGPdM3jSFdd28C37pvL9j213HruyA4pgUNREYiEgOtOGszJwzL53UsrmL9eZx9HsqcXbKRwwy4S46I5a1TWoTfoACoCkRAQFWXcfuEYcnokcPVjC/lPsa5jEIkWbNjFHW+sZmy/NJb/9nSS42M75XlVBCIhIjUhlr9edBTOOS59aJ7OPo4wiz7bxUUPzKWipp6fThnWqceKVAQiIWR0bhqzrj+eARlJXPv4It5audXvSOIh5xzOOR6as5Zz/vYh9Y2Ot390IhMGeHPiWEtUBCIhJjM5nienTeSIXkl8f0Yhv31pOTuranVxmy7oNzOXc8RNs/jDK4FTqP560VH075nU6Tn09VGREFVd28APn1rM7KLAR0TD+iRz9yVjGZTZ3edk0l7vrS7jqcKNvLK0dP+8D392En3TEjx7zoN9fVRFIBLCGhsdLy8r5XcvLWf7nlqS4qL55Vn5nDkqi5ROOpAo7bO1oob3V5dxVL80BmUm85/i7Vz60DwAstMSmHHleHJ7JBIX4+0HNCoCkS6gZNdern5sIUtLAtczyE5L4Ndfz+fU/N46CS1E3fbqKu59dw0AcTFRTBnRh5eWbiY1IZanr5rIoMzunfZvpyIQ6SLqGxp5ZVkpK0ormLl4M6XlNfRNjefmr49gypHeDEgmbVfX0Mgtr6zkkQ/Xc+bILI4ZkM4zC0pYWlLO4Mzu3HvZ0Z3+EZ+KQKQLqqlr4G/vFO8fovjCglyunjyQfumJANpL8MGW8hrufbeYf84NnCE+Pi+dGVeOJz42GoDNu6tJT4rbf78zqQhEurCaugZ+/eJynl6wkUYHiXHR5PRI4LyxOZwzNpvM5Hi/I3ZJ9Q2NrNtexbX/XkRKfCwOx9KScvYFryJ2YUEut50/yueUX1ARiESATburmb2slFeLtrBo424aGh3J8TH86sx8LhjX7MX/pI327KvnR08t5p1Pmr9s5KDM7pwwOIOrJw+kV/duPiRsmYpAJAIVb6vk588X8fG6naTEx3DtSYO5aHxupw1b0JW8uHgTj330GR8fMA7U2H5pjMtL54enDWFr+T5y0xNC9iM5FYFIhKpvaOT+99cyu6iUok0VdO8WwynDM5ldtIVrTxrEkdmpnDgkI2TfvPy2Y88+pv9rAfPX7wKgX3oik4ZmcPNZ+cR4eKEYL6gIRISlJbt5aM46ZheVUtfw5f/3A3olkZoYy6jsVCYO7MmwPink9er8M1xDSXVtAzc8uYjXlgeG+fjbpWOZOrJzRgP1gopARPZraHTUNTRSvG0PT87fyNaKGsqr65i37ssfexw3qCffPfYIJg/LJNrjK2SFEuccb6zYyrRHFwBwzeSB3HDKEE8vFdkZDlYE3l/xQERCSnSUER0VzZHZqRyZnbp/vnOOqtoGZi8r5aWlpRRvreT7MwpJiotm4sBeXHJMLmbG+Lx0krqF31tHY6PDLPC1WucC3/DJSounuraBwvW7yEqN5443V1O8bQ+79tYB8PtvHsm3J/T3Obn3tEcgIs2qqWtg5uLNLPxsF68u38Lu4JsjBD4rP3l4JlNG9GFon2R2VNVSW9/IgIwk9u5roGhzObv21lFdW88HxTvITkugtr6R6roGYqKM4VkpZPdIYPPuamrqGrjkmH50i2nbd+udc2zfU8u7n2xjZ1UtqQmxlOyqZmBmEgX90/n7B+uYu2YHWytriI2OoqxyX6set1f3OM4/Opevj85iRN/UQ28QJvTRkIi0S0VNHW+v3Maasj3s2VfP+u1VvP/pdhoaO+b9IyU+hiuOzWPqqCz6piV8aRylxkbHhp17yUzuxu7qOnomxXHLKyv598efUd/K58/pkUBNXQP5fVOJjTKq6xoYl5dOo3OUV9cxeVgmy0rKmTw0k5E5XefNvykVgYh0uG2VNXy0dielu6uJj41m9946oqOgqraBMblpLC3ZTW6PRAb3TmZw7+4kxEbT6Bx1DY7tlftYWVpBXEwURZsqWPjZLt5bXQZAlMHwrBRKy2voFhNFRXUdVbUNX3n+hNhojsxOYfqJAxmWlcLKzRWMyE5hbVkV76zaxsicVM4ek93ZL0vIUhGISMgr3lbJh2t28NmOvSwpCZwQl5WaQH1jI1sq9nH8oF5EWaBovjaoF5OHZfodOazoYLGIhLxBmckMykz2O0ZECu/vQ4mISLupCEREIpyKQEQkwqkIREQinKdFYGZTzOwTMys2s581s7ybmT0ZXD7PzPK8zCMiIl/lWRGYWTRwD3AGkA9cbGb5B6x2JbDLOTcIuAO4zas8IiLSPC/3CMYDxc65tc65WuAJ4OwD1jkb+Gdw+hngZNN4uCIincrLIsgGNja5XxKc1+w6zrl6oBzo6WEmERE5QFicUGZm04Bpwbt7zOyT4HQqgfJoafrzP3sB2w/jqZs+ZluWNzf/UFmbTit325Z3VG44vOxdNXdrMjY3ryvkbilve37G/c7d8jCqzjlPbsBE4LUm928CbjpgndeAicHpGAIvrLXhOR442HSTPwsP8+/wwOEsb27+obIqt/+5Dzd7V83dmoxdNfdB8h72z3go5G7p5uVHQ/OBwWZ2hJnFARcBMw9YZyZweXD6fOBtF0zdSi8dYrrpvMNxqO1bWt7c/ENlbTqt3G1brtyHpzXbHypjc/O6Qu6m9w+2TvTmgkwAAAg/SURBVFuEbG5PB50zs6nAX4Bo4GHn3C1m9jsCbTrTzOKBR4GjgJ3ARc65tR7kKHQtDLYUypS784VrduXuXOGauyWeHiNwzs0CZh0w7+Ym0zXAt7zMEPRAJzyHF5S784VrduXuXOGau1lhNwy1iIh0LA0xISIS4VQEIiIRTkUgIhLhIr4IzOx4M7vPzB4ysw/9ztNaZhZlZreY2V1mdvmhtwgNZjbJzOYEX/NJfudpCzNLMrNCMzvL7yytZWbDg6/1M2b2A7/ztJaZfdPMHgwOSnma33lay8wGmNnfzewZv7O0RVgXgZk9bGbbzKzogPkHHfW0KefcHOfcdOBlvhj3yFMdkZvAOE05QB2B4Ts810G5HbAHiCe8cgP8D/CUNym/qoN+vlcGf74vAI7zMm+TfB2R+wXn3H8B04ELvczbJF9H5F7rnLvS26QeaOvZcaF0A04AxgJFTeZFA2uAAUAcsITA6KcjCbzZN71lNtnuKSA5XHIDPwOuCm77TBjljgpu1xt4LIxyn0rgpMgrgLPCJXdwm28As4FLwil3cLs/A2PDMHen/J/sqFtYjDXUEufc+81cw2D/qKcAZvYEcLZz7lag2V16M+sHlDvnKj2Mu19H5DazEqA2eLfBu7Rf6KjXO2gX0M2LnAfqoNd7EpBE4E2g2sxmOecaQz138HFmAjPN7BXgce8S73++jni9DfgjMNs5t9DbxAEd/PMdVsK6CFrQ3KinxxximyuBf3iWqHXamvs54C4zOx5438tgh9Cm3GZ2LnA6kAbc7W20g2pTbufcLwDM7Apgu9clcBBtfb0nAecSKN1ZLa3XCdr6830tcAqQamaDnHP3eRnuINr6evcEbgGOMrObgoUR8rpiEbSZc+7XfmdoK+fcXgIFFlacc88RKLGw5Jx7xO8MbeGcexd41+cYbeacuxO40+8cbeWc20HguEZYCeuDxS3YBOQ2uZ8TnBfqlLtzKXfnUu4Q1hWLoDWjnoYi5e5cyt25lDuU+X20uj034N9AKV98hfLK4PypwGoCR/t/4XdO5VZu5VbuUL5p0DkRkQjXFT8aEhGRNlARiIhEOBWBiEiEUxGIiEQ4FYGISIRTEYiIRDgVgXjOzPZ0wnN8o5VDSXfkc04ys2MPY7ujzOzvwekrzMzPMZf2M7O8A4dgbmadDDN7tbMySedQEUjYMLPolpY552Y65/7owXMebDyuSUCbiwD4OWE4jg6Ac64MKDWzTrm2gXQOFYF0KjP7iZnNN7OlZvbbJvNfMLMFZrbczKY1mb/HzP5sZkuAiWa23sx+a2YLzWyZmQ0Lrrf/N2sze8TM7jSzD81srZmdH5wfZWZ/M7NVZvaGmc36fNkBGd81s7+YWSFwvZl93czmmdkiM3vTzHoHhyueDtxoZostcKW7DDN7Nvj3m9/cm6WZJQOjnHNLmlmWZ2ZvB1+bt4LDo2NmA83so+Df9w/N7WFZ4Oppr5jZEjMrMrMLg/PHBV+HJWb2sZklB59nTvA1XNjcXo2ZRZvZn5r8W13VZPELwKXN/gNLePL71Gbduv4N2BP88zTgAcAI/BLyMnBCcFl68M8EoAjoGbzvgAuaPNZ64Nrg9NXAQ8HpK4C7g9OPAE8HnyOfwHjyAOcTGIo5CuhD4JoI5zeT913gb03u94D9Z+F/H/hzcPo3wI+brPc48LXgdD9gZTOPPRl4tsn9prlfAi4PTn8PeCE4/TJwcXB6+uev5wGPex7wYJP7qQQupLIWGBecl0JgxOFEID44bzBQGJzOI3hRFmAa8MvgdDegEDgieD8bWOb3z5VuHXfTMNTSmU4L3hYF73cn8Eb0PnCdmZ0TnJ8bnL+DwEV3nj3gcT4fxnoBgbH2m/OCC1wzYIWZ9Q7O+xrwdHD+FjN75yBZn2wynQM8aWZZBN5c17WwzSlAfuCaKgCkmFl351zT3+CzgLIWtp/Y5O/zKPB/TeZ/Mzj9OPD/mtl2GfBnM7sNeNk5N8fMRgKlzrn5AM65CgjsPQB3m9kYAq/vkGYe7zRgVJM9plQC/ybrgG1A3xb+DhKGVATSmQy41Tl3/5dmBi6ecgow0Tm318zeJXBNY4Aa59yBV2DbF/yzgZZ/hvc1mbYW1jmYqibTdwG3O+dmBrP+poVtooAJzrmagzxuNV/83TqMc261mY0lMEDaH8zsLeD5Fla/EdgKjCaQubm8RmDP67VmlsUT+HtIF6FjBNKZXgO+Z2bdAcws28wyCfy2uStYAsOACR49/3+A84LHCnoTONjbGql8MQb95U3mVwLJTe6/TuDKWgAEf+M+0EpgUAvP8yGBYY4h8Bn8nOD0RwQ++qHJ8i8xs77AXufcv4A/Ebj27idAlpmNC66THDz4nUpgT6ER+DaB6/Ie6DXgB2YWG9x2SHBPAgJ7EAf9dpGEFxWBdBrn3OsEPtqYa2bLgGcIvJG+CsSY2UoC16n9yKMIzxIYXngF8C9gIVDeiu1+AzxtZguA7U3mvwSc8/nBYuA6oCB4cHUFzVypyjm3isDlF5MPXEagRL5rZksJvEFfH5x/A/DD4PxBLWQeCXxsZouBXwN/cM7VAhcSuKTpEuANAr/N/w24PDhvGF/e+/ncQwRep4XBr5Tezxd7X5OBV5rZRsKUhqGWiPL5Z/YWuLbsx8BxzrktnZzhRqDSOfdQK9dPBKqdc87MLiJw4PhsT0MePM/7BC7gvsuvDNKxdIxAIs3LZpZG4KDv7zu7BILuBb7VhvWPJnBw14DdBL5R5AszyyBwvEQl0IVoj0BEJMLpGIGISIRTEYiIRDgVgYhIhFMRiIhEOBWBiEiEUxGIiES4/w9xIcywpuHf8AAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learner.lr_find(show_plot=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train the Model Using a Cosine Annealing LR Schedule" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Train for 938 steps, validate for 1 steps\n", "Epoch 1/10\n", "938/938 [==============================] - 7s 8ms/step - loss: 0.1176 - accuracy: 0.9641 - val_loss: 0.0513 - val_accuracy: 0.9825\n", "Epoch 2/10\n", "938/938 [==============================] - 6s 7ms/step - loss: 0.0504 - accuracy: 0.9844 - val_loss: 0.0375 - val_accuracy: 0.9874\n", "Epoch 3/10\n", "938/938 [==============================] - 6s 6ms/step - loss: 0.0413 - accuracy: 0.9875 - val_loss: 0.0336 - val_accuracy: 0.9888\n", "Epoch 4/10\n", "938/938 [==============================] - 6s 6ms/step - loss: 0.0327 - accuracy: 0.9899 - val_loss: 0.0388 - val_accuracy: 0.9891\n", "Epoch 5/10\n", "938/938 [==============================] - 6s 6ms/step - loss: 0.0268 - accuracy: 0.9918 - val_loss: 0.0278 - val_accuracy: 0.9906\n", "Epoch 6/10\n", "938/938 [==============================] - 6s 6ms/step - loss: 0.0186 - accuracy: 0.9943 - val_loss: 0.0254 - val_accuracy: 0.9921\n", "Epoch 7/10\n", "938/938 [==============================] - 6s 6ms/step - loss: 0.0135 - accuracy: 0.9955 - val_loss: 0.0224 - val_accuracy: 0.9933\n", "Epoch 8/10\n", "938/938 [==============================] - 6s 6ms/step - loss: 0.0083 - accuracy: 0.9974 - val_loss: 0.0191 - val_accuracy: 0.9937\n", "Epoch 9/10\n", "938/938 [==============================] - 6s 6ms/step - loss: 0.0044 - accuracy: 0.9988 - val_loss: 0.0190 - val_accuracy: 0.9943\n", "Epoch 10/10\n", "938/938 [==============================] - 6s 6ms/step - loss: 0.0035 - accuracy: 0.9992 - val_loss: 0.0190 - val_accuracy: 0.9943\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learner.fit(5e-3, 1, cycle_len=10, checkpoint_folder='/tmp/mymodel')" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd5xU5dn/8c81s426tKWDINWlw9LtFWyoQQVL7FggtjzJo8kvPjF5nsQUYwWRiAUFgZAQ0agowUaVpalLXQQEpEtvy+5evz/moJt1KYs7O7s73/frNS9mzrnPmesMs/vdc+5zzm3ujoiIyIkKxboAEREpXxQcIiJSLAoOEREpFgWHiIgUi4JDRESKRcEhIiLFouAQKUVm1szM3MwSSmh9a8zs/JJuK3IsCg6JK0f75WlmZ5tZvpntNbM9ZrbczG45zrpuM7NlQfvNZva2mVWLXvUiZUOJ/NUjUkF87e6NzcyA/sAUM5vl7ssLNzSzs4DfAf3cfaGZ1QIuK+V6RWJCexwihXjE28A3QMejNOsOzHb3hcEy37j7K+6+B8DMKpnZ42a21sx2mdkMM6tUYPnrzewrM9tmZr88MtHMQmb2kJmtMrPtZjYxCKUj828M1rm94HLBvJfN7H8LvD7bzNYXVfzx3kfkWBQcIoUEv1QvB+oA2UdpNhe4yMweNbO+ZpZcaP6fgW5AH6AW8HMgv8D804E2wHnAI2Z2WjD9J8AVwFlAQ2AHMDyoKx14DrgxmFcbaHySm3nU9xE5HgWHyHcamtlO4AAwGXjwyB5FYe7+CXAV0BX4F7DdzP5iZmEzCwG3Ave5+wZ3z3P3We5+qMAqHnX3A+6+GFgMdAqm3wX80t3XB+1/DQwMOtMHAm+5+8fBvF/xn2FUHMd6H5Fj0pdE5DtH+jiSgceAc4Enj9bY3d8B3gmC4hzgb8ByIqGTAqw6xnttKvB8P1A1eH4KMNnMCgZCHlCPyJ7BugLvv8/Mtp/gthV2rPfZcJLrlDihPQ6RQoK/wP8b6GBmV5xA+3x3/zcwHWgPbAMOAi1O4u3XAf3dvUaBR4q7bwA2Ak2ONDSzykQOVx2xD6hc4HX9k3wfkWNScEg8SjSzlAKP7+15u3sO8DjwSFErMLMBZjbIzGpaRA8i/QVz3D0feBH4i5k1DA5f9S6iH6QoI4H/M7NTgvdJM7MBwbxJwKVmdrqZJQG/4T9/hhcBF5tZLTOrD9x/ku8jckwKDolHbxPpxzjy+PVR2r0INDWzok6z3QHcAawEdgOvAX9y97HB/P8CPgfmETk76w+c2M/bU8AU4D0z2wPMAXoCuHsWMBQYR2TvYwdQ8KypV4n0l6wB3gMmnMz7iByPaSAnEREpDu1xiIhIsSg4RESkWBQcIiJSLAoOEREplri4ALBOnTrerFmzWJchIlJuzJ8/f5u7pxU1Ly6Co1mzZmRmZsa6DBGRcsPM1h5tng5ViYhIsSg4RESkWBQcIiJSLAoOEREpFgWHiIgUS1SDw8z6mdlyM8s2s4eKmJ9sZhOC+XPNrFmBeQ8H05eb2UUFpq8xs8/NbJGZ6VQpEZFSFrXTcc0sTGQoyguI3MFznplNcfclBZrdBuxw95ZmNojIHUSvDYbIHAS0IzJ4zTQza+3uecFy57j7tmjVLiIiRxfN6zh6ANnu/iWAmY0HBgAFg2MA393SehLwrJlZMH18MKDOajPLDtY3O4r1fs/T/16JOyQmGEnhEEkJIZITQqRWSqRG5SRqVE6kZuUkalVJIjGso34iEh+iGRyNKDDMJZG9jsL3+/+2jbvnmtkuIiOaNSIyPkDBZRsFz53IGAIOPO/uo4p6czMbAgwBaNq06UltwMiPVrE/J++47UIG9aqn0KhGJRrWqETjmpVokVaV1vWq0bJuVSolhU/q/UVEyqLyeOX46e6+wczqAu+b2TJ3/7hwoyBQRgFkZGSc1KAjS37Tj7x853BePofz8snJzedgbj679h9m5/4cdh44zI79OWzefYgNOw6wYed+Fq7bwdufbyQ3P/KWZtC0VmXa1q9G5yY16dykBh0bp1IluTx+9CIi0Q2ODRQYHxloHEwrqs36YPjOVGD7sZY9Miayu28xs8lEDmF9LzhKSjhkhENhUhK/22toVKPSMZfJzctnzfb9rNy8h+Wb97By816yvt7F1KzNQGQPpU396nRvVpO+LevQ69TapFZKjNYmiIiUqGgGxzyglZk1J/JLfxBwXaE2U4CbiPRdDASmu7ub2RRgnJn9hUjneCvgUzOrAoTcfU/w/EIi4y6XKQnhEC3rVqVl3ar079Dg2+k79uWwaP1OFn61k4Vf7WDS/PWMmb2WkEHHxjU4vWUdzmlbly5NahAKWQy3QETk6KIWHEGfxTBgKhAGXnT3LDP7DZDp7lOA0cCrQef3N0TChaDdRCId6bnAUHfPM7N6wORI/zkJwDh3fzda21DSalZJ4pw2dTmnTV0AcnLzWfjVDmZmb2NG9jae+2gVz36QTVq1ZC5Ir8eF6fXo06IOSQnqeBeRsiMuxhzPyMjw8nB33F0HDvPh8i1MzdrEh8u3sj8nj2rJCfRrX58ruzaiV/Pa2hMRkVJhZvPdPaPIeQqOsung4TxmrdrGvz7bxLtfbGRfTh4NUlMY0LkRV3VtROt61WJdoohUYAqOchgcBR3IyeP9pZuZvGA9H6/cRl6+0+2UmtzQqyn92zf4j457EZGSoOAo58FR0La9h5i8YANj565lzfb91KqSxNXdGnN9z1NoWrtyrMsTkQpCwVGBguOI/Hxn5qptvDZnLdOWbiHfnYvS6zPkrFPp2rRmrMsTkXLuWMGhq9DKqVDIOKNVGme0SmPTroO8OmcNr85ey7tZm+jerCZDzmzBeW3rqjNdREqc9jgqkH2Hcpkwbx2jZ6xmw84DtEirwr3nteLSjg0JK0BEpBh0qCpOguOIw3n5vP35RkZ8sIrlm/fQsm7VSIB0aKA9EBE5IQqOOAuOI/LznXe+2MST01awcsteWteryn3ntaZ/+/oKEBE5pmMFhy5JrsBCIeOSjg149/4zeXpwF/LynaHjFnDFiJnM+XJ7rMsTkXJKwREHwiHj8k4Nee+Bs3j86k5s3XOIQaPmcPsr88jesifW5YlIOaNDVXHo4OE8Xpy5muc+WMX+w3kM6t6E+89vTVq15FiXJiJlhA5VyX9ISQxzz9kt+fBnZ3NDz6ZMmLeOcx//kJdnriY3Lz/W5YlIGafgiGO1qybz6ID2TH3gTDo3qcGv31zCpc/MYN6ab2JdmoiUYQoOoUVaVcbc2oORN3Rl94HDXD1yNg9OXMTWPYdiXZqIlEEKDgHAzOjXvgHTfnoWQ89pwZuLv+bcxz9k3NyvyM+v+P1gInLiFBzyHyonJfCzi9oy9f4zad8wlV9M/pzrXpjDmm37Yl2aiJQRCg4p0qlpVRl3R08eu6oDWV/v5qInP2bkR6vUeS4iCg45OjNjUI+mTHvwLM5qncZj7yzjihEzyfp6V6xLE5EYUnDIcdWrnsLzN3ZjxPVd2bTrEFcMn8nwD7K19yESpxQcckLMjIs7NOD9B87kwnb1+dPU5Vzz/Gz1fYjEIQWHFEvNKkk8O7gLTw3qTPaWvfR/6hPGzl1LPNyBQEQiFBxSbGbGgM6NmPrAmXQ7pSa/nPwFt7w8jy27D8a6NBEpBQoOOWkNUisx5tYe/PqydGav2k7/pz7hw+VbYl2WiESZgkN+kFDIuLlvc976yenUqZrMzS/N4/fvLOWwOs5FKiwFh5SIVvWq8cawvlzfsynPf/QlV4+czbpv9se6LBGJAgWHlJiUxDD/d2UHhl/XlVVb93Lx05/w9ucbY12WiJQwBYeUuEs6NuDte8+gRVpV7hm7gP/3z885lJsX67JEpIQoOCQqmtSqzN/u6s2QM0/ltTlfcc3zc/h654FYlyUiJUDBIVGTGA7xi4tP47nru7Jqy14ufWYGM7O3xbosEfmBFBwSdf07NOCNYX2pXSWJG0fPZcSH2bpgUKQcU3BIqWiRVpV/Du3LxR0a8Md3lzPk1fnsPng41mWJyEmIanCYWT8zW25m2Wb2UBHzk81sQjB/rpk1KzDv4WD6cjO7qNByYTNbaGZvRbN+KVlVkhN4ZnAXHrk0nQ+WbWHAszPJ3rIn1mWJSDFFLTjMLAwMB/oD6cBgM0sv1Ow2YIe7twSeAP4QLJsODALaAf2AEcH6jrgPWBqt2iV6zIxbT2/OuDt6sefgYa4cPosPlulqc5HyJJp7HD2AbHf/0t1zgPHAgEJtBgCvBM8nAeeZmQXTx7v7IXdfDWQH68PMGgOXAC9EsXaJsh7Na/HGsNNpUqsyt74yj+c/WqV+D5FyIprB0QhYV+D1+mBakW3cPRfYBdQ+zrJPAj8HjnlPCzMbYmaZZpa5devWk90GiaJGNSox6e7eXNy+Ab9/Zxk/nbiYg4d1vYdIWVeuOsfN7FJgi7vPP15bdx/l7hnunpGWllYK1cnJqJyUwLPXdeHBC1rzj4UbuHbUHN1lV6SMi2ZwbACaFHjdOJhWZBszSwBSge3HWLYvcLmZrSFy6OtcM3stGsVL6TEz7j2vFSNv6MqKTXu47NkZfLZ+Z6zLEpGjiGZwzANamVlzM0si0tk9pVCbKcBNwfOBwHSPHOieAgwKzrpqDrQCPnX3h929sbs3C9Y33d1viOI2SCnq174Bf7+7DwmhENc8P5upWZtiXZKIFCFqwRH0WQwDphI5A2qiu2eZ2W/M7PKg2WigtpllAw8CDwXLZgETgSXAu8BQd9fB7ziQ3rA6/xzalzb1q3PXa/N54ZMv1WkuUsZYPPxQZmRkeGZmZqzLkGI4kJPHAxMW8W7WJn7c+xQeuTSdhHC56pITKdfMbL67ZxQ1Tz+JUiZVSgoz4vquDDnzVMbMXsuQV+ez71BurMsSERQcUoaFQsYvLj6N317Rng+Xb+Ga52ezWWdcicScgkPKvBt7ncLom7uzZts+rhg+k6Ubd8e6JJG4puCQcuGcNnWZeFdv3OGakbOZtUq3ZxeJFQWHlBvtGqYyeWgfGtRI4eYX5/GvzzQsrUgsKDikXGmQWom/3dmHTk1SGfb6Al6ZtSbWJYnEHQWHlDuplRN59baenH9aPf5nShZ/mrpM13qIlCIFh5RLKYlhnru+K4N7NGX4B6v4+aTPyM075n0vRaSEJMS6AJGTlRAO8bsr21O3WjJP/Xsl2/fl8Ox1XaicpK+1SDRpj0PKNTPjgQta87/BtR7X/XUuO/blxLoskQpNwSEVwg29TmHE9d1YsnE31zw/m027dKGgSLQoOKTC6Ne+Pq/c0oOvdx7g6udn8dX2/bEuSaRCUnBIhdK7Re1gPPNcBo6cxfJNe2JdkkiFo+CQCqdTkxpMvLM3ANeOms3idRoUSqQkKTikQmpdrxqT7upDtZQErvvrHGav2h7rkkQqDAWHVFhNa1dm0l19aFijEje99Cn/Xro51iWJVAgKDqnQ6lVPYeKdvWlbvxp3vjqfNxYVHvZeRIpLwSEVXs0qSYy9vSfdTqnJ/RMWMXbu2liXJFKuKTgkLlRLSeSVW3twTpu6/HLyF4yesTrWJYmUWwoOiRspiWFG3tCN/u3r89u3lvDch6tiXZJIuaTgkLiSlBDimcFduLxTQ/7w7jKenLZCd9YVKSbdDU7iTkI4xBPXdiYpIcST01aSk5vPzy5qg5nFujSRckHBIXEpHDL++KOOJIZDjPhwFTm5+fzyktMUHiInQMEhcSsUMn53ZXuSE0K8MGM1h3LzefTydoRCCg+RY1FwSFwzM/7nsnSSEkKM+vhLDufl87srOyg8RI5BwSFxz8x4uH9bkhNCPDM9m5zcfP44sCMJYZ07IlIUBYcIkfD46YVtSAqHePz9FeTk5fPEtZ1JVHiIfI+CQ6SAn5zXiqSEEL9/Zxl5+c7Tg7soPEQK0U+ESCF3ntWCX12azjtfbOLe1xdyOC8/1iWJlCkKDpEi3HZ6c4WHyFFENTjMrJ+ZLTezbDN7qIj5yWY2IZg/18yaFZj3cDB9uZldFExLMbNPzWyxmWWZ2aPRrF/im8JDpGhRCw4zCwPDgf5AOjDYzNILNbsN2OHuLYEngD8Ey6YDg4B2QD9gRLC+Q8C57t4J6Az0M7Ne0doGkYLh8ZNxCg8RiO4eRw8g292/dPccYDwwoFCbAcArwfNJwHkWuXR3ADDe3Q+5+2ogG+jhEXuD9onBQzcakqi67fTmPHJpOu9mKTxEILrB0QhYV+D1+mBakW3cPRfYBdQ+1rJmFjazRcAW4H13n1vUm5vZEDPLNLPMrVu3lsDmSDy7VeEh8q1y1znu7nnu3hloDPQws/ZHaTfK3TPcPSMtLa10i5QKSeEhEhHN4NgANCnwunEwrcg2ZpYApALbT2RZd98JfECkD0SkVBQMj2HjFig8JC5FMzjmAa3MrLmZJRHp7J5SqM0U4Kbg+UBgukcGR5gCDArOumoOtAI+NbM0M6sBYGaVgAuAZVHcBpHvufX05vzPZelMzdqs8JC4FLUrx90918yGAVOBMPCiu2eZ2W+ATHefAowGXjWzbOAbIuFC0G4isATIBYa6e56ZNQBeCc6wCgET3f2taG2DyNHc0rc5AI++uYRh4xbw7HVddYW5xA2Lh9HPMjIyPDMzM9ZlSAX00szVPPrmEi5qV0/hIRWKmc1394yi5ulbLvID3NL3u8NW941fSK4OW0kc0E0ORX6gW/o2Jy/f+d9/LSUcWswT13TSLdmlQlNwiJSA2884lbx85/fvLCNs8Pg1nQlrMCipoBQcIiXkzrNakJvv/GnqcsKhEH8a2FEjCUqFpOAQKUFDz2lJbp7zxLQVJISM31+lYWil4lFwiJSw+85vRV5+Pk9PzyYcNv7vivZEbsEmUjEoOESi4IELWpOb74z4cBUJIePRy9spPKTCUHCIRIGZ8bOL2pCb74z6+EvCIeORS9MVHlIhKDhEosTMeLh/W3LznBdnriYhZPzi4tMUHlLuKThEosjM+NWlp5GXn89fP1lNOBTiv/u1UXhIuabgEIkyM+PXl7cjN98Z+VGkz+OnF7ZWeEi5peAQKQVmxm8HtCcv33n2g2wSwsb957eOdVkiJ0XBIVJKQiHjd1d2IDffeXLaShJCxrBzW8W6LJFiO25wBLcwz3L3tqVQj0iFFgoZf/hRR/LynT+/t4KEcIi7zmoR67JEiuW4wRGMg7HczJq6+1elUZRIRRYOGX++uhN5+c5j7ywjIWTcfsapsS5L5ISd6KGqmkCWmX0K7Dsy0d0vj0pVIhVcOGT85ZpOBe6qa98ODiVS1p1ocPwqqlWIxKGEcIgnB3UmNz+fR99cQkLIuLF3s1iXJXJcJxQc7v5RtAsRiUeJ4RDPDO7KPWMX8Ks3sgiHQlzXs2msyxI5pmOONmNme8xsdxGPPWa2u7SKFKnIkhJCDL++C+e0SeMXkz9n4rx1sS5J5JiOGRzuXs3dqxfxqObu1UurSJGKLjkhzHM3dOPM1mn89z8+Y9L89bEuSeSoNL6lSBmRkhhm1I3d6NuiDj+btJh/LtwQ65JEiqTgEClDUhLD/PXHGfRqXpsHJy7ijUUKDyl7FBwiZUylpDCjb86ge7NaPDBhEVMWfx3rkkT+g4JDpAyqnJTAS7d0J6NZLe4fv5A3FR5Shig4RMqoykkJvHRzdzJOqcX9Exbxr882xrokEUDBIVKmVUmO7Hl0bVqDe8cv5J3PFR4SewoOkTIuEh496NykBj95fSHvfqHwkNhScIiUA1WTE3j5lu50bJzKsHELmZq1KdYlSRxTcIiUE9VSEnnl1h50aJzK0LELeE/hITGi4BApR46ER7tGqQwdt4BpSzbHuiSJQwoOkXKmekoiY27tQXqD6tw9dj7/XqrwkNIV1eAws37BIFDZZvZQEfOTzWxCMH+umTUrMO/hYPpyM7somNbEzD4wsyVmlmVm90WzfpGyKrVSImNu68lpDapz92sL+GDZlliXJHEkasERDDk7HOgPpAODzSy9ULPbgB3u3hJ4AvhDsGw6MAhoB/QDRgTrywV+6u7pQC9gaBHrFIkLqZUSefXWnrSuX5U7X53PB8sVHlI6ornH0QPIdvcv3T0HGA8MKNRmAPBK8HwScJ6ZWTB9vLsfcvfVQDbQw903uvsCAHffAywFGkVxG0TKtNTKibx2W09a1YuEx4cKDykF0QyORkDBgQXW8/1f8t+2cfdcYBdQ+0SWDQ5rdQHmFvXmZjbEzDLNLHPr1q0nvREiZV2NykmMvb0nLdOqMuTV+Xy8Qt93ia5y2TluZlWBvwP3u3uRA0q5+yh3z3D3jLS0tNItUKSUHQmPFmlVuWNMJp+sVHhI9EQzODYATQq8bhxMK7KNmSUAqcD2Yy1rZolEQmOsu/8jKpWLlEM1q0TCo3mdKtz+Sqb2PCRqohkc84BWZtbczJKIdHZPKdRmCnBT8HwgMN3dPZg+KDjrqjnQCvg06P8YDSx1979EsXaRcqlWwfAYk6mzrSQqohYcQZ/FMGAqkU7sie6eZWa/MbPLg2ajgdpmlg08CDwULJsFTASWAO8CQ909D+gL3Aica2aLgsfF0doGkfKodtVkXr+jF63qRjrM39dFglLCLPIHfsWWkZHhmZmZsS5DpFTt2n+YH784l6yvd/PM4C7079Ag1iVJOWJm8909o6h55bJzXESOL7VyIq/e3jNyY8TXNRiUlBwFh0gFVj0lcoV5t6Y1uW/8QiYvXB/rkqQCUHCIVHBVkxN4+dbu9GxemwcnLuZvmeuOv5DIMSg4ROJA5aQEXry5O6e3rMPPJn3GuLlfxbokKccUHCJxolJSmL/+OINz2qTxi8mfM2b2mliXJOWUgkMkjqQkhhl5YzfOP60ej7yRxegZq2NdkpRDCg6ROJOcEGbE9V3p374+v31rCSM/WhXrkqScUXCIxKGkhBDPDO7CZZ0a8tg7y3j8veXEwzVdUjISYl2AiMRGQjjEk9d2pkpSmGemZ7PnYC6PXJpOKGSxLk3KOAWHSBwLh4zfX9WBKskJjJ6xmn2HcnnsRx0JKzzkGBQcInHOzPh/l5xGtZQEnpy2kn05uTx5bReSEnQkW4qm4BARzIz7z29N1eQE/vdfS9l3KJORN3SjUlI41qVJGaQ/KUTkW7efcSq/v6oDH6/cyk0vfsqeg4djXZKUQQoOEfkPg3s05alBXVjw1Q6uf2EuO/blxLokKWMUHCLyPZd3asioH3dj2aY9XDtqNlt2H4x1SVKGKDhEpEjntq3Hy7d0Z8OOA/xo5CxWb9sX65KkjFBwiMhR9WlRh7F39GLvwVwGPjeLz9fvinVJUgYoOETkmDo3qcGku/uQkhhm0KjZzFi5LdYlSYwpOETkuFqkVeUf9/ShSa3K3PLypxpNMM4pOETkhNSrnsKEO3vTpUlN7h2/kJdn6s668UrBISInLLVSImNu68H5p9Xj128u4U9Tl+nmiHFIwSEixZKSGOa567syuEcThn+wiof+/jm5efmxLktKkW45IiLFlhAO8bsrO5BWNZmnp2ezfd8hnh7chcpJ+pUSD7THISInxcx48MI2/HZAO6Yv28K1z89hyx5dKBgPFBwi8oPc2LsZo27MIHvLXq4cPosVm/fEuiSJMgWHiPxg56fXY+KdvcnJy+dHz81iVrau9ajIFBwiUiI6NE5l8j19aJCawo9f/JRJ89fHuiSJEgWHiJSYxjUrM+nuPvQ8tRb/9bfF/OX9FTpdtwJScIhIiaqekshLN/fg6m6NefrfK/npxMUcys2LdVlSgnTunIiUuKSEEH8c2JGmtSrz+PsrWPvNfkbe0I20asmxLk1KQFT3OMysn5ktN7NsM3uoiPnJZjYhmD/XzJoVmPdwMH25mV1UYPqLZrbFzL6IZu0i8sOYGT85rxXDr+tK1te7GPDsDLK+1t11K4KoBYeZhYHhQH8gHRhsZumFmt0G7HD3lsATwB+CZdOBQUA7oB8wIlgfwMvBNBEpBy7p2IBJd/XBgYHPzeadzzfGuiT5gaK5x9EDyHb3L909BxgPDCjUZgDwSvB8EnCemVkwfby7H3L31UB2sD7c/WPgmyjWLSIlrH2jVN4Y1pe2Dapx99gFPDVtpTrNy7FoBkcjYF2B1+uDaUW2cfdcYBdQ+wSXPSYzG2JmmWaWuXXr1mKWLiIlrW61FF6/oxdXdW3EE9NWMGzcQvbn5Ma6LDkJFfasKncf5e4Z7p6RlpYW63JEhMgNEh+/uhO/uLgtb3+xkatHzmbdN/tjXZYUUzSDYwPQpMDrxsG0ItuYWQKQCmw/wWVFpBwyM4ac2YLRN2Xw1Tf7uezZGXy0QkcFypNoBsc8oJWZNTezJCKd3VMKtZkC3BQ8HwhM98iBzynAoOCsq+ZAK+DTKNYqIqXs3Lb1mDLsdOpXT+Hmlz7lmX+vJD9f/R7lQdSCI+izGAZMBZYCE909y8x+Y2aXB81GA7XNLBt4EHgoWDYLmAgsAd4Fhrp7HoCZvQ7MBtqY2Xozuy1a2yAi0dW8ThX+cU8fLu/UkMffX8EdYzLZdeBwrMuS47B4OLMhIyPDMzMzY12GiByFuzNm9lp++9YSGtWsxMgbunFag+qxLiuumdl8d88oal6F7RwXkfLDzLipTzMm3NmLg4fzuHLETP6umySWWQoOESkzup1Sizd/cjqdGtfgp39bzIMTF7HvkE7ZLWsUHCJSptStlsK4O3px33mtmLxwA5c9o1uVlDUKDhEpc8Ih44ELWjPu9l7sy8nlyhGzGDN7ja42LyMUHCJSZvVuUZu37z2Dvi1q88gbWdz12nx27ddZV7Gm4BCRMq121WRG39Sd/3fJaUxftoX+T33MTA1NG1MKDhEp80Ih4/YzTmXSXX1ISQxz/QtzefTNLA4e1gBRsaDgEJFyo1OTGvzr3jO4qfcpvDRzDZc8/Qmfrd8Z67LijoJDRMqVSklhHh3Qnldv68G+Q3lcNWIWT01bSW5efqxLixsKDhEpl85olcbU+8/kko4NeGLaCq56bhZLN+6OdVlxQcEhIuVWauVEnhrUheHXdWXDjgNc9swMHn9vOYdy1fcRTQoOESn3LunYgGkPnsXlnRvyzPRsLn7qEzLXaKDQaFFwiEiFUFaZr5UAAAwFSURBVLNKEn+5pjOv3NqDg4fzGThyNr/65xfsOajrPkqagkNEKpSzWqfx3gNnckvfZrw2dy3nPf4RbyzaoKvOS5CCQ0QqnCrJCfzPZe2YfE9f6qemcN/4RQwaNYflm/bEurQKQcEhIhVW5yY1mHxPX353ZQeWb97DxU9/wqNvZrFbh69+EAWHiFRo4ZBxXc+mfPDTs7m2exNenrWGc//8IWPnrtW1HydJwSEicaFmlSR+d2UH3hjal2a1q/DLyV9w0ZMf8/6Szer/KCYFh4jElY6Na/C3u3oz8oZuuMMdYzK5dtQcFq/TrUtOlIJDROKOmdGvfX2mPnAmvx3QjlVb9jJg+Ezufm2+rj4/ARYPu2gZGRmemZkZ6zJEpIzac/Awf/1kNS/OWM3eQ7lc3KE+957Xirb1q8e6tJgxs/nunlHkPAWHiEjEzv05vDhjNS/OXMPeQ7lc0qEBw85tyWkN4i9AFBwKDhEphp37cxg9YzUvBQFyZus0hpxxKn1b1sbMYl1eqVBwKDhE5CTs3J/D2Llf8fKsNWzdc4i29asx5MxTubRjQ5ISKnYXsYJDwSEiP8Ch3DzeWPQ1f/34S1Zu2UvdaskM6t6Ea3s0pVGNSrEuLyoUHAoOESkB+fnORyu2Mmb2Gj5csRUDzm1bl+t6NuWs1nUJhyrOYaxjBUdCaRcjIlJehULGOW3rck7buqz7Zj/j533FhHnrmbY0kwapKVzeuSFXdG5U4TvTtcchIvIDHM7L5/0lm5k0fz0fr9hKbr7Tpl41BnRpyGUdG9KkVuVYl3hSdKhKwSEipWD73kO8/flG/rnoa+av3QHAaQ2qc0F6PS5Mr0e7htXLzVlZCg4Fh4iUsq+27+fdrI28v2Qz89fuIN+hQWoKZ7dJo0+LOvRuUZs6VZNjXeZRKTgUHCISQ9v3HmL6si28v2Qzs1dtZ8+hXADa1q9G7xa16XZKTTo1rkHjmpXKzB5JzILDzPoBTwFh4AV3f6zQ/GRgDNAN2A5c6+5rgnkPA7cBecC97j71RNZZFAWHiJQVuXn5fPH1bmat2sas7O3MW/MNh3Ijt3evXSWJTk1q0KFRKm3qV6Nl3ao0q10lJteMxCQ4zCwMrAAuANYD84DB7r6kQJt7gI7ufpeZDQKudPdrzSwdeB3oATQEpgGtg8WOuc6iKDhEpKzKyc1n+aY9LFq/k8XrdvLZ+p2s3LKXI7+awyHjlNqVaV67Cg1qpNCwRiUaplaiQWoKtaokkVopkeqVEklJDJdoXbE6HbcHkO3uXwZFjAcGAAV/yQ8Afh08nwQ8a5H9tAHAeHc/BKw2s+xgfZzAOkVEyo2khBAdGqfSoXEqN/Y6BYADOXms2rqX7C2Rx8ote1i7fT+Za3ew60DRoxcmJYSolpxAYjhEQthICoeoUy2ZiXf2LvGaoxkcjYB1BV6vB3oerY2755rZLqB2MH1OoWUbBc+Pt04AzGwIMASgadOmJ7cFIiIxUCkpTPtGqbRvlPq9eftzcvl650E27TrIjv057D54mF0HIo+9B3PJzXMO5+dzOM+pmlyyeyFHVNgLAN19FDAKIoeqYlyOiEiJqJyUQMu6VWlZt2rMaohmj8sGoEmB142DaUW2MbMEIJVIJ/nRlj2RdYqISBRFMzjmAa3MrLmZJQGDgCmF2kwBbgqeDwSme6S3fgowyMySzaw50Ar49ATXKSIiURS1Q1VBn8UwYCqRU2dfdPcsM/sNkOnuU4DRwKtB5/c3RIKAoN1EIp3eucBQd88DKGqd0doGERH5Pl0AKCIi33Os03Er9kgkIiJS4hQcIiJSLAoOEREpFgWHiIgUS1x0jpvZVmDtSS5eB9hWguWUR/oM9BmAPoN42/5T3D2tqBlxERw/hJllHu3Mgnihz0CfAegziPftL0iHqkREpFgUHCIiUiwKjuMbFesCygB9BvoMQJ9BvG//t9THISIixaI9DhERKRYFh4iIFIuC4yjMrJ+ZLTezbDN7KNb1lCQza2JmH5jZEjPLMrP7gum1zOx9M1sZ/FszmG5m9nTwWXxmZl0LrOumoP1KM7vpaO9ZVplZ2MwWmtlbwevmZjY32NYJwe37CW7xPyGYPtfMmhVYx8PB9OVmdlFstuTkmFkNM5tkZsvMbKmZ9Y6374GZPRD8HHxhZq+bWUq8fQ+Kzd31KPQgcsv2VcCpQBKwGEiPdV0luH0NgK7B82rACiAd+CPwUDD9IeAPwfOLgXcAA3oBc4PptYAvg39rBs9rxnr7ivlZPAiMA94KXk8EBgXPRwJ3B8/vAUYGzwcBE4Ln6cH3IxloHnxvwrHermJs/yvA7cHzJKBGPH0PiAxJvRqoVOD//+Z4+x4U96E9jqL1ALLd/Ut3zwHGAwNiXFOJcfeN7r4geL4HWErkB2gAkV8kBP9eETwfAIzxiDlADTNrAFwEvO/u37j7DuB9oF8pbsoPYmaNgUuAF4LXBpwLTAqaFP4Mjnw2k4DzgvYDgPHufsjdVwPZRL4/ZZ6ZpQJnEhkXB3fPcfedxNn3gMi4RJWCUUgrAxuJo+/ByVBwFK0RsK7A6/XBtAon2NXuAswF6rn7xmDWJqBe8Pxon0d5/5yeBH4O5AevawM73T03eF1we77d1mD+rqB9ef4MmgNbgZeCw3UvmFkV4uh74O4bgD8DXxEJjF3AfOLre1BsCo44ZmZVgb8D97v77oLzPLL/XWHP1TazS4Et7j4/1rXEUALQFXjO3bsA+4gcmvpWHHwPahLZW2gONASqUL72lmJCwVG0DUCTAq8bB9MqDDNLJBIaY939H8HkzcGhB4J/twTTj/Z5lOfPqS9wuZmtIXIo8lzgKSKHX44MqVxwe77d1mB+KrCd8v0ZrAfWu/vc4PUkIkEST9+D84HV7r7V3Q8D/yDy3Yin70GxKTiKNg9oFZxZkUSkE2xKjGsqMcEx2dHAUnf/S4FZU4AjZ8TcBLxRYPqPg7NqegG7gkMZU4ELzaxm8JfbhcG0Ms/dH3b3xu7ejMj/73R3vx74ABgYNCv8GRz5bAYG7T2YPig426Y50Ar4tJQ24wdx903AOjNrE0w6D1hCHH0PiByi6mVmlYOfiyOfQdx8D05KrHvny+qDyBkkK4icHfHLWNdTwtt2OpHDD58Bi4LHxUSO1f4bWAlMA2oF7Q0YHnwWnwMZBdZ1K5GOwGzgllhv20l+Hmfz3VlVpxL5gc8G/gYkB9NTgtfZwfxTCyz/y+CzWQ70j/X2FHPbOwOZwXfhn0TOioqr7wHwKLAM+AJ4lciZUXH1PSjuQ7ccERGRYtGhKhERKRYFh4iIFIuCQ0REikXBISIixaLgEBGRYlFwiByHmc0K/m1mZteV8Lp/UdR7iZRlOh1X5ASZ2dnAf7n7pcVYJsG/u+dRUfP3unvVkqhPpLRoj0PkOMxsb/D0MeAMM1sUjOEQNrM/mdm8YHyKO4P2Z5vZJ2Y2hchVyJjZP81sfjDuw5Bg2mNE7sq6yMzGFnyv4OrsPwVjRHxuZtcWWPeH9t0YGmODK54xs8csMsbKZ2b259L8jCS+JBy/iYgEHqLAHkcQALvcvbuZJQMzzey9oG1XoL1HbrENcKu7f2NmlYB5ZvZ3d3/IzIa5e+ci3usqIld1dwLqBMt8HMzrArQDvgZmAn3NbClwJdDW3d3MapT41osEtMchcvIuJHLvpkVEbktfm8g9igA+LRAaAPea2WJgDpGb4bXi2E4HXnf3PHffDHwEdC+w7vXunk/kdjHNiNze+yAw2syuAvb/4K0TOQoFh8jJM+An7t45eDR39yN7HPu+bRTpGzkf6O3unYCFRO55dLIOFXieBxzpR+lB5A63lwLv/oD1ixyTgkPkxO0hMtTuEVOBu4Nb1GNmrYOBkApLBXa4+34za0tk2NUjDh9ZvpBPgGuDfpQ0IiP1HfVuq8HYKqnu/jbwAJFDXCJRoT4OkRP3GZAXHHJ6mcj4Hc2ABUEH9Va+G2K0oHeBu4J+iOVEDlcdMQr4zMwWeOS27kdMBnoTGcfagZ+7+6YgeIpSDXjDzFKI7Ak9eHKbKHJ8Oh1XRESKRYeqRESkWBQcIiJSLAoOEREpFgWHiIgUi4JDRESKRcEhIiLFouAQEZFi+f9qwQ6Pha4UhAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# cosine annealed LR schedule\n", "learner.plot('lr')" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU9b3/8dcnmewbIQtbAmFP2AQMAoJiwIW626pgrVZvW6699rb9tb232va21mtv7e/ntd7eaxdbtXWrtaitt9ViUUBZBVzYl7AEEpYsQMhCtpnP749zEiZhgJBkMknm83w88sjMWWY+k9Z5813O94iqYowxxrQVEeoCjDHG9EwWEMYYYwKygDDGGBOQBYQxxpiALCCMMcYEZAFhjDEmIAsIYzpIRHJEREXE045j7xGRld1RlzFdxQLChAUR2S8iDSKS3mb7R+6XfE5oKruwoDGmO1lAmHCyD7ij+YmITATiQ1eOMT2bBYQJJ88Dd/s9/zzwnP8BIpIiIs+JSJmIFInI90Qkwt0XKSKPiUi5iOwFrgtw7tMiclhESkTkERGJ7EzBIhIjIk+IyCH35wkRiXH3pYvIX0TkhIgcE5H3/Wr9tltDlYjsFJF5nanDhCcLCBNO1gLJIpLnfnEvBF5oc8x/AynACGAOTqDc6+77EnA9MAXIB25tc+5vgSZglHvM1cAXO1nzd4EZwGTgIuAS4Hvuvm8CxUAGMAD4DqAiMhb4CjBNVZOAa4D9nazDhCELCBNumlsRVwHbgZLmHX6h8aCqVqnqfuA/gbvcQ24HnlDVg6p6DPix37kDgGuBr6tqjaqWAj91X68z7gQeVtVSVS0DfuhXTyMwCBimqo2q+r46i6t5gRhgnIhEqep+Vd3TyTpMGLKAMOHmeeCzwD206V4C0oEooMhvWxEwxH08GDjYZl+zYe65h90unxPAr4DMTtY7OEA9g93H/w8oBN4Wkb0i8gCAqhYCXwceAkpF5GURGYwxF8gCwoQVVS3CGay+Fnitze5ynH+VD/PbNpTTrYzDQHabfc0OAvVAuqr2c3+SVXV8J0s+FKCeQ+5nqVLVb6rqCOBG4BvNYw2q+pKqznbPVeAnnazDhCELCBOOvgDMVdUa/42q6gVeAX4kIkkiMgz4BqfHKV4BvioiWSKSCjzgd+5h4G3gP0UkWUQiRGSkiMy5gLpiRCTW7ycC+D3wPRHJcKfofr+5HhG5XkRGiYgAlThdSz4RGSsic93B7DrgFOC7wL+RMRYQJvyo6h5V3XCW3f8M1AB7gZXAS8Az7r5fA0uAT4APObMFcjcQDWwDjgOLccYI2qsa58u8+Wcu8AiwAdgEbHbf9xH3+NHAUve8NcDPVXUZzvjDozgtoiM43VwPXkAdxgAgdsMgY4wxgVgLwhhjTEAWEMYYYwKygDDGGBOQBYQxxpiA+szqkenp6ZqTkxPqMowxplfZuHFjuapmBNrXZwIiJyeHDRvONnPRGGNMICJSdLZ91sVkjDEmoKAGhIjMd5caLmxeJ6bN/stF5EMRaRKRW/22TxaRNSKyVUQ2iciCYNZpjDHmTEELCHdlzCeBTwHjgDtEZFybww7gLJr2UpvttcDd7jo284EnRKRfsGo1xhhzpmCOQVwCFKrqXgAReRm4CWcZAgDc5ZQRkVbrxKjqLr/Hh0SkFGfN+xMXUkBjYyPFxcXU1dV19DOYNmJjY8nKyiIqKirUpRhjgiyYATGE1ksjFwPTL/RFROQSnPVtzljPXkQWAYsAhg4d2nY3xcXFJCUlkZOTg7OemekMVaWiooLi4mKGDx8e6nKMMUHWowepRWQQzvr996rqGatRqupTqpqvqvkZGWfO0qqrqyMtLc3CoYuICGlpadYiMyZMBDMgSmi9dn4WfnfvOh8RSQb+CnxXVdd2tAgLh65lf09jwkcwA2I9MFpEhotINM6tF99oz4nu8a8Dz6nq4iDWSJPXx9GTdZxqaArm2xhjTK8TtIBQ1SacG6cvwbn37yuqulVEHhaRGwFEZJqIFAO3Ab8Ska3u6bcDlwP3iMjH7s/koBQqUHqynspTjUF5+RMnTvDzn//8gs+79tprOXHigsbkjTGmS/WZ+0Hk5+dr2yupt2/fTl5e3nnP3VNWjdenjBmQ1OV17d+/n+uvv54tW7a02t7U1ITH0zsvZG/v39UY0/OJyEZVzQ+0r3d+Q3WxpFgPRyrraGjyEe3p2kbVAw88wJ49e5g8eTJRUVHExsaSmprKjh072LVrFzfffDMHDx6krq6Or33tayxatAg4vXRIdXU1n/rUp5g9ezarV69myJAh/PnPfyYuLq5L6zTGmLbCJiB++L9b2XboZMB9PlVONXiJiYrAE9H+gBg3OJkf3HDue9I/+uijbNmyhY8//pjly5dz3XXXsWXLlpZpos888wz9+/fn1KlTTJs2jc985jOkpaW1eo3du3fz+9//nl//+tfcfvvtvPrqq3zuc59rd53GGNMRYRMQ5xIhgojQ5FW6uAFxhksuuaTVNQQ/+9nPeP311wE4ePAgu3fvPiMghg8fzuTJzhDMxRdfzP79+4NbpDHGEEYBcb5/6Zccr+V4bSPjBicTEcSpnAkJCS2Ply9fztKlS1mzZg3x8fFcccUVAa8xiImJaXkcGRnJqVOnglafMcY069EXynWnpNgofKrU1HftdNekpCSqqqoC7qusrCQ1NZX4+Hh27NjB2rUdvtzDGGO6XNi0IM4nIcaDiFBV10RSbNetM5SWlsasWbOYMGECcXFxDBgwoGXf/Pnz+eUvf0leXh5jx45lxowZXfa+xhjTWTbN1c++8hoamnyMHdj10137Epvmakzfca5prtbF5Ccp1kN9k5f6Rm+oSzHGmJCzgPCTFOP0uFV18TiEMcb0RhYQfmKiIonxRFJVZwFhjDEWEG0kxXqorm/C6+sbYzPGGNNRFhBtJMV60CBMdzXGmN7GAqKNhBgPESJU1QVndVdjjOktLCDaiBAhMcZDVV0ToZgCnJiYCMChQ4e49dZbAx5zxRVX0HZKb1tPPPEEtbW1Lc9t+XBjzIWygAggKc5Dg9dHfdMZdzntNoMHD2bx4o7fK6ltQLz55pv069evK0ozxoQJC4gAkmKcK6lPdkE30wMPPMCTTz7Z8vyhhx7ikUceYd68eUydOpWJEyfy5z//+Yzz9u/fz4QJEwA4deoUCxcuJC8vj1tuuaXVWkxf/vKXyc/PZ/z48fzgBz8AnAUADx06REFBAQUFBYCzfHh5eTkAjz/+OBMmTGDChAk88cQTLe+Xl5fHl770JcaPH8/VV19taz4ZE+bCZ6mNtx6AI5vbdWg0MKqxCUEgKvLsBw6cCJ969JyvtWDBAr7+9a9z//33A/DKK6+wZMkSvvrVr5KcnEx5eTkzZszgxhtvPOv9nn/xi18QHx/P9u3b2bRpE1OnTm3Z96Mf/Yj+/fvj9XqZN28emzZt4qtf/SqPP/44y5YtIz09vdVrbdy4kWeffZZ169ahqkyfPp05c+aQmppqy4obY1qxFsRZeCIEr09ROjcOMWXKFEpLSzl06BCffPIJqampDBw4kO985ztMmjSJK6+8kpKSEo4ePXrW13jvvfdavqgnTZrEpEmTWva98sorTJ06lSlTprB161a2bdt2znpWrlzJLbfcQkJCAomJiXz605/m/fffB2xZcWNMa+HTgjjPv/TbaqxvYm9ZNUP7x9MvPrpTb33bbbexePFijhw5woIFC3jxxRcpKytj48aNREVFkZOTE3CZ7/PZt28fjz32GOvXryc1NZV77rmnQ6/TzJYVN8b4sxbEWcRHRxIZIV1yVfWCBQt4+eWXWbx4MbfddhuVlZVkZmYSFRXFsmXLKCoqOuf5l19+OS+99BIAW7ZsYdOmTQCcPHmShIQEUlJSOHr0KG+99VbLOWdbZvyyyy7jT3/6E7W1tdTU1PD6669z2WWXdfozGmP6nvBpQVwgESEpJqpluuvZxgfaY/z48VRVVTFkyBAGDRrEnXfeyQ033MDEiRPJz88nNzf3nOd/+ctf5t577yUvL4+8vDwuvvhiAC666CKmTJlCbm4u2dnZzJo1q+WcRYsWMX/+fAYPHsyyZctatk+dOpV77rmHSy65BIAvfvGLTJkyxbqTjDFnsOW+z+F4TQMHj9cyKjOR+GjL0ma23LcxfYct991BSbHu6q62eJ8xJgxZQJyDJzKC+GiPBYQxJiz1+YDobBdaUqyH2oYmGr2hu6q6J+krXZLGmPPr0wERGxtLRUVFp77UmruZqq0VgapSUVFBbGxsqEsxxnSDoI68ish84L+ASOA3qvpom/2XA08Ak4CFqrrYb9/nge+5Tx9R1d9d6PtnZWVRXFxMWVlZRz8CqlB+so6qIxH0T+jc9RB9QWxsLFlZWaEuwxjTDYIWECISCTwJXAUUA+tF5A1V9b/U9wBwD/CtNuf2B34A5AMKbHTPPX4hNURFRTF8+PCOfwjXM3/8hLe3HuHDf7sKT2SfbnQZY0yLYH7bXQIUqupeVW0AXgZu8j9AVfer6iagbQf/NcDfVfWYGwp/B+YHsdZzmpubycm6Jj46aMtlG2PCRzADYghw0O95sbuty84VkUUiskFENnSmG+l8Zo9OJzJCeHdHadDewxhjeppe3V+iqk+par6q5mdkZATtfZJjo8gflsoyCwhjTBgJZkCUANl+z7PcbcE+Nyjm5may40gVhyttATtjTHgIZkCsB0aLyHARiQYWAm+089wlwNUikioiqcDV7raQKcjNBGDZjuB1ZRljTE8StIBQ1SbgKzhf7NuBV1R1q4g8LCI3AojINBEpBm4DfiUiW91zjwH/jhMy64GH3W0hMzozkSH94li207qZjDHhIajXQajqm8CbbbZ93+/xepzuo0DnPgM8E8z6LoSIUJCbwWsfllDf5CXGc447zRljTB/Qqwepu1vB2ExqG7x8sC+kjRljjOkWFhAXYObINKI9ETYOYYwJCxYQFyA+2sPMEWkst3EIY0wYsIC4QAVjM9hbXsP+8ppQl2KMMUFlAXGBWqa7WivCGNPHWUBcoGFpCYzISGDZThuHMMb0bRYQHVAwNpO1eyuobbB7RBhj+i4LiA4oGJtJQ5OP1YUVoS7FGGOCxgKiA6YNTyUhOtLGIYwxfZoFRAfEeCKZNSqdZTtK7R7Nxpg+ywKigwpyMzlUWceuo9WhLsUYY4LCAqKDCsY6013tJkLGmL7KAqKDBqbEkjco2cYhjDF9lgVEJxSMzWBj0XEqTzWGuhRjjOlyFhCdMDc3E69PeX+3XTRnjOl7LCA6YXJ2P1Liomx1V2NMn2QB0QmeyAguH5PBil2l+Hw23dUY07dYQHTS3NwMyqsb2FxSGepSjDGmS1lAdNLlozMQsdVdjTF9jwVEJ6UlxnBRVj9b3dUY0+dYQHSBubmZbCo+QXl1fahLMcaYLmMB0QUKxmaiCiusFWGM6UMsILrA+MHJZCTF2DiEMaZPsYDoAhERwhVjMnhvVxlNXl+oyzHGmC5hAdFFCnIzOVnXxIcHToS6FGOM6RJBDQgRmS8iO0WkUEQeCLA/RkT+4O5fJyI57vYoEfmdiGwWke0i8mAw6+wKs0en44kQ62YyxvQZQQsIEYkEngQ+BYwD7hCRcW0O+wJwXFVHAT8FfuJuvw2IUdWJwMXAPzaHR0+VHBtFfk4qy2z5b2NMHxHMFsQlQKGq7lXVBuBl4KY2x9wE/M59vBiYJyICKJAgIh4gDmgATgax1i5RMDaTHUeqOHTiVKhLMcaYTgtmQAwBDvo9L3a3BTxGVZuASiANJyxqgMPAAeAxVT3W9g1EZJGIbBCRDWVloZ9iOjfXuYnQcpvuaozpA3rqIPUlgBcYDAwHvikiI9oepKpPqWq+quZnZGR0d41nGJWZyJB+cXaXOWNMnxDMgCgBsv2eZ7nbAh7jdielABXAZ4G/qWqjqpYCq4D8INbaJUSEgtwMVhWWU9/kDXU5xhjTKcEMiPXAaBEZLiLRwELgjTbHvAF83n18K/CuqipOt9JcABFJAGYAO4JYa5eZm5vJqUYvH+w7o0fMGGN6laAFhDum8BVgCbAdeEVVt4rIwyJyo3vY00CaiBQC3wCap8I+CSSKyFacoHlWVTcFq9auNHNEOtGeCOtmMsb0ep5gvriqvgm82Wbb9/0e1+FMaW17XnWg7b1BXHQkM0eksXxnGT+4IdTVGGNMx/XUQepebW5uJvvKa9hXXhPqUowxpsMsIIKgYKwz3dUumjPG9GYWEEEwNC2eERkJtuyGMaZXs4AIkrljM1m39xi1DU2hLsUYYzrEAiJICnIzafD6WFVYEepSjDGmQywggmRaTn8SoiOtm8kY02tZQARJtCeC2aPTWb6jFOfaP2OM6V0sIIKoYGwmhyrr2Hm0KtSlGGPMBbOACKIrWqa72uquxpjexwIiiAamxDJuULJdD2GM6ZUsIIKsIDeDjQeOU1nbGOpSjDHmglhABFnB2Ey8PuX9QutmMsb0LhYQQTZlaCr94qNsdVdjTK9jARFkkRHC5aMzWLGzDJ/PprsaY3oPC4huUJCbQUVNA5tLKkNdijHGtJsFRDeYMyYTEaybyRjTq1hAdIP+CdFMzu7Hclt2wxjTi1hAdJO5YzP5pLiSsqr6UJdijDHtYgHRTQpynauqV+yy6a7GmN7BAqKbjBuUTEZSjK3uaozpNSwguklEhFAwNoP3dpXR5PWFuhxjjDkvC4huVDA2k6q6JjYWHQ91KcYYc17tCggR+ZqIJIvjaRH5UESuDnZxfc2s0el4IoRlO20cwhjT87W3BfEPqnoSuBpIBe4CHg1aVX1UcmwU03L623RXY0yv0N6AEPf3tcDzqrrVb5u5AAW5Gew4UkXJiVOhLsUYY86pvQGxUUTexgmIJSKSBNhIawcUuDcRslaEMaana29AfAF4AJimqrVAFHDv+U4SkfkislNECkXkgQD7Y0TkD+7+dSKS47dvkoisEZGtIrJZRGLbWWuPNiozkazUOLvLnDGmx2tvQMwEdqrqCRH5HPA94Jwrz4lIJPAk8ClgHHCHiIxrc9gXgOOqOgr4KfAT91wP8AJwn6qOB64A+sQdd0SEgrGZrCosp67RG+pyjDHmrNobEL8AakXkIuCbwB7gufOccwlQqKp7VbUBeBm4qc0xNwG/cx8vBuaJiOAMhm9S1U8AVLVCVfvMt2lBbganGr18sO9YqEsxxpizam9ANKmq4nyh/4+qPgkkneecIcBBv+fF7raAx6hqE06rJA0YA6iILHGn1P5roDcQkUUiskFENpSV9Z4um5kj0onxRNhV1caYHq29AVElIg/iTG/9q4hE4IxDBIsHmA3c6f6+RUTmtT1IVZ9S1XxVzc/IyAhiOV0rLjqSmSPTWGbLfxtjerD2BsQCoB7neogjQBbw/85zTgmQ7fc8y90W8Bh33CEFqMBpbbynquXuoPibwNR21torFIzNZH9FLfvKa0JdijHGBNSugHBD4UUgRUSuB+pU9XxjEOuB0SIyXESigYXAG22OeQP4vPv4VuBdtytrCTBRROLd4JgDbGvXJ+olmqe7WivCGNNTtXepjduBD4DbgNuBdSJy67nOcccUvoLzZb8deEVVt4rIwyJyo3vY00CaiBQC38CZSouqHgcexwmZj4EPVfWvF/rherKhafGMzEiwcQhjTI/laedx38W5BqIUQEQygKU4M4/OSlXfxOke8t/2fb/HdTihE+jcF3CmuvZZBWMzeW5NETX1TSTEtPd/CmOM6R7tHYOIaA4HV8UFnGvOYm5uJg1eH6v3VIS6FGOMOUN7v+T/5k45vUdE7gH+SpuWgblw+Tn9SYiO5F0bhzDG9EDt6tdQ1X8Rkc8As9xNT6nq68ErKzxEeyKYPTqd5TtLUVWcawSNMaZnaHfHt6q+CrwaxFrC0tzcTJZsPcqOI1XkDUoOdTnGGNPinAEhIlWABtoFqKraN1onXdE83XVnqQWEMaZHOecYhKomqWpygJ8kC4euMSA5lnGDklluq7saY3oYm4nUA8zNzWTjgeNU1vaJBWuNMX2EBQSABupF6z4FuRl4fcp7u60VYYzpOSwgGmrht9fBzrdCVsLk7FT6xUfZVdXGmB7FAqK2AupOwu8Xwl++4QRGN4uMEOaMyWDFzjJ8vtC2ZowxppkFRL9s+NI7cOk/w4an4ak5cOjjbi9jbm4mFTUN3P/Sh2wpOefN+owxpltYQAB4YuDqR+CuP0F9FfzmSlj5BPh83VbCdRMHcX/BSFbuLuf6/17JXU+vY1VhORri8RFjTPiSvvIFlJ+frxs2bOj8C9Ueg//9Kmz/X8i5DG75FaS0vRFe8Jysa+TFtQd4ZtU+yqrqmZSVwpfnjOTq8QOJjLArrY0xXUtENqpqfsB9FhABqMJHL8Bb34bIKLjhCRh/S9e8djvVNXp57cMSfvXeHooqahmRnsCiy0dwy9QhxHgiu7UWY0zfZQHRURV74LUvQclGmHwnfOonEHO+W3F3La9P+duWI/xiRSFbSk6SmRTDF2YP57PTh5IUG8y7vhpjwoEFRGd4G2HFT+D9/4R+Q+HTv4HsaV3/PuehqqwqrOAXKwpZVVhBUqyHu2cO455Lh5ORFNPt9Rhj+gYLiK5QtAZeWwQnS2DOt+Gyb0JkaG7ys6n4BL9csYe3thwhKjKC2/OzWHTZSIamxYekHmNM72UB0VXqKuGv34LNr0D2dPj0U5CaE9z3PIe9ZdX8+v29vLqxhCafj+smDea+OSMYPzglZDUZY3oXC4iutumP8NdvOIPZ1z0GkxZACO/lcPRkHc+s3MeL6w5QXd/E5WMy+PKckcwY0d/uMWGMOScLiGA4XgSv/yMcWAMTPgPXPQ5x/brv/QOoPNXIC2uLeHbVPsqrG7gou58zRXbcACJsiqwxJgALiGDxeWHl47Dsx5A82LlmImfW+c8LsrpGL4s3FvPUe3s5cKyWERkJ3Hf5SG6eMoRoj10baYw5zQIi2Io3wmtfhGP7YPb/gSseBE90aGrx0+T18eaWI/xy+R62HT7JwORYvjB7OHdMH0piTGgG2I0xPYsFRHeor4a/PQAfPQ+DJsNnfgPpo0NXjx9V5b3d5fxy+R7W7K0gOdbD3TNzuGdWDumJNkXWmHBmAdGdtr3hLNXRVA/zfwxTPx/SAey2Pj54gl8u38OSbUeIjoxgwbRsvnTZCLL72xRZY8KRBUR3O3kIXr8P9q2A3Ovhhp9BQlqoq2qlsLSap97bw+sfleBTuH7SIO6bM9Lui21MmLGACAWfD9Y+Ce88DHH94eafw6h5oa7qDEcq63h65V5eWneAmgYvV4zN4L45I5k+3KbIGhMOzhUQQZ3SIiLzRWSniBSKyAMB9seIyB/c/etEJKfN/qEiUi0i3wpmnUEREeHcY+KL70BsCrzwafjbg9BYF+rKWhmYEst3rxvH6gfm8a2rx7C5uJKFT63lhv9Zyasbi6lv8oa6RGNMiAStBSEikcAu4CqgGFgP3KGq2/yO+SdgkqreJyILgVtUdYHf/sWAAutU9bFzvV+Pa0H4azwFf/8+fPAUZI53BrAHjAt1VQGdavDy+kclPLNqH4Wl1aQnxnDXjGHcOWOoDWgb0weFpItJRGYCD6nqNe7zBwFU9cd+xyxxj1kjIh7gCJChqioiNwOzgBqgulcHRLNdS+DP9zu3OL363+GSRT1qANufqrKysJxnVu5j2c4yoiMjuHHyYO6dlWNLeRjTh4Sqi2kIcNDvebG7LeAxqtoEVAJpIpIIfBv44bneQEQWicgGEdlQVlbWZYUHzZhr4MurYcQceOtf4cVboepoqKsKSES4bHQGz957Ce98cw4LpmXz102Hue5nK1n41Bre3noEbzDvn+1tcn6MMSHTUy+rfQj4qapWn+sgVX1KVfNVNT8jI6N7KuusxEz47Ctw7WOwfyX8YibsfCvUVZ3TyIxE/v3mCax9cB7fuTaXg8dOsej5jRQ8tpynV+6jqq6x82+iCuW7Yd1T8PvPwv8dDo+NgrW/dJZcN8Z0ux7ZxQS8B2S7h/UDfMD3VfV/zvZ+vaKLqa3SHfDqF+HoZsj/gnNf7Oiefz1Ck9fH29uO8uyqfazff5zEGA+35Wdxz6U5DEtLaP8LVZfC3hWwdxnsXe4spQ7QbxiMLHCuTN+3AtJGO3+bMdf02C45Y3qrUI1BeHAGqecBJTiD1J9V1a1+x9wPTPQbpP60qt7e5nUeoq+MQQTSVA/v/jus/m/oPxLyrneuxB50EaQOd2ZD9WCbik/w7Kr9/GXTIZp8yrzcAfzD7Bxmjkg7c5psQw0UrXbCYO9yOLrF2R6XCsMvhxEFMOIK6D/c2a4Ku/4Gb38PKgqdfdf8BwwY310fz5g+L2TXQYjItcATQCTwjKr+SEQeBjao6hsiEgs8D0wBjgELVXVvm9d4iL4cEM32LoelP4Qjm8HndqnEpMCgSU5YNIdG2qgeGRpHT9bxwtoiXlx3gGM1DeQOTOILl2ZzY2YpMUXvOZ/v4Drns0XGwNAZzhf+yAIYOAkiznGfbW8jrH8alv8Y6k/C1Luh4LtOd50xplPsQrnepKkBSrfB4U/cn4/hyBbw1jv7oxNh4MTWoZE+JmR3t2tFlfqju9jy/hvU7XqHiQ2fkCy1ADRmTiRq9FwnFIbOhKi4C3/92mOw4v/C+l+DJw4u+wbM+CeIiu3Sj2FMOLGA6O28jVC2s01obIZG58sXTxwMnNA6NDJyu2dF2eoyZ5xg7zJnPKHSmbimKdmUZszk9cqx/Lp4CCcjUrhh0mDunTWciVmdnCZbvtu5rmTnm859wq/8IYy/xcYnjOkAC4i+yOd1+uUPfXw6NA5vgoYqZ39ktNNX7x8aA8aDp5MXuzXUwgF3HGHPcmeAHZyrxYfPcVoII66A/iNavrD3l9fw29X7+eOGg9Q0eJmWk8o/zBrOVeMG4InsRHfZ3uWw5LvOWEb2dLjmx5B1cWc+nTFhxwIiXPh8cHwfHPrILzQ+ce6lDRDhgcw8v9CY7LQ8ztXd4/M6IdQ80+jgOvA2OAGUPd0ZQxhxhfNa5xpHAE7WNfLHDcX8dvU+Dh47xZB+cdxzaQ63T8smJS6qg5/ZCx+9AO8+AjWlMDA/tDEAABOJSURBVPF2uPIHkJLVsdczJsxYQIQzVThR5NfScIOjtsLZL5GQMbZ1SyO+v3ONxt5lsO+90wEzcOLpFsLQSzs8JdfrU5Zud6bJrt17jPjoSG692JkmOyIjsWOfs74K3n8c1jwJ4q6DNetrENPB1zMmTFhAmNZUnWsO2oZGdZurulOyTwfC8DmQ2PUXI249VMmzq/bzxseHaPD6KBibwT/MHs7sUekdW032xAFY+hBseRUSB8K878NFd/TImV/G9AQWEKZ9qo44YVFdCsMubTWOEGxlVfW8tO4Az68tory6njEDErlz+jCuHj+AQSkdmPF08ANn9dySDc402vk/hpzZXV+4Mb2cBYTpNeqbvPzlk8M8s2ofWw+dBGD84GSuzBvAlXkDmDAkuf0tC5/PaUksfQhOFjs3b7rqYUgbGbwPYEwvYwFheh1VZU9ZDUu3H+Wd7UfZWHQcn8LA5Fjm5WVyZd4AZo5MIzbq3APjgLPc+pr/gfd/6gywT/9HuPxfIK5f8D+IMT2cBYTp9Sqq61m2s4x3th9lxa4yahu8xEdHctnodOblDWBubub571dRdcRZ1uSjF53lPQq+Axff2zMuMjQmRCwgTJ9S1+hl7d4K3tleytLtRzlcWYcITB2ayry8TK7KG8CozMSzd0Ud/sS5fmL/+5A+Fq75EYy+qns/hDE9hAWE6bNUla2HTraExeYSZ0rusLR4rswbwLy8TKbl9Ceq7QV5qrDjr/D3f4Nje2HkPCcoMvNC8CmMCR0LCBM2jlTW8c6OoyzddpRVeypoaPKRHOuhIDeTeXkDmDMmo/VFeU0NztpOK37iXEtx8T3OQoAJ6SH7DMZ0JwsIE5Zq6ptYWVjO0m1HeXdHKRU1DXgihOkj+jMvdwBXjRtAdn/3Yr+aCljxqLNqbHQCXP4tmH5f55cmMaaHs4AwYc/rUz4+eIKl253Wxe5S52aFYwckObOixg1gclY/Iip2Ofef2P02pOY402LzbrSFAE2fZQFhTBtFFTUs3V7K0m1H+WD/Mbw+JT0xhrm5GVyZN4DLIzcT++73naXXEzKdVWP7DYV+2c4V5v2Gur+zISYp1B/HmA6zgDDmHCprG1m+q5Sl20tZvrOUqromYjwRXDYylS8mr2Oy7iC2psRZyryy2LmWwl9capvQaBMkcanWAjE9lgWEMe3U6PWxft8x/r79KEu3H+XgsVNERQrXThzE3TNzmJqdjFSXOmFx4sDp3ycOuo8PQmNN6xeNSmgTGs2/hzmPEzJtrSgTMhYQxnSAqrLzaBUvf3CQVzcWU1XfxPjBydw9cxg3XjSEuOgAV3Grwqnjzgq6/qHREiQHoO5E63Mio53lyVtaH0NbB0nyELuYzwSNBYQxnVRT38SfPi7hudVF7DxaRUpcFLddnMXnZgwjJz3hwl6svurM0PAPkrar6kokJA92Fk9MH+P8ZLi/kwZZ95XpFAsIY7qIqvLBvmM8t7aIJVuO0ORT5ozJ4POXDmPOmEwiI7rgy7qxzlmOvVUr5ABU7IHyXVB/8vSx0YmQPtoNjtHOleHpY5ww6Y5bzppezwLCmCA4erKO339wgJfWHaC0qp7s/nF8bvowbs/PJjUhSF/Oqk4Lo3yX+7PbuV95+W5nxdpmEulM020Ojoyxpx/HpQanNtMrWUAYE0SNXh9vbz3Kc2v2s27fMWI8Edxw0WDunjmMSVnduGJsfbVzn/Ly3W54uMFRUdh65lVCxpktjvTRzniHDZaHHQsIY7rJziNVPLdmP69/VEJtg5eLsvtx94xhXDdpUPuWJg8Gn9fprmppbbgtj/KdzoB6M08cpI06Pb7R3HWVNurc9y03vZoFhDHd7GRdI69tLOa5tUXsLauhf0I0C6Zlc+f0oWSlduxe3kFRU9G6tdHcdXW8CGj+bhBnRlX6GKfFkTEGMvKcbiu7p0avZwFhTIioKqv3VPDcmv38fZszO2lu7gDunjmM2aPSieiKQe1gaDx1elDcPzjKd0PTqdPHJQ2CjFznJzPXgqMXCllAiMh84L+ASOA3qvpom/0xwHPAxUAFsEBV94vIVcCjQDTQAPyLqr57rveygDA9XcmJU7y0roiXPzhIRU0Dw9MT+NyMYdx6cVbrFWZ7Mp8PKg9A6Q4oc39Ktzvh0Vh7+rikQU5QZOS5weH+WHD0OCEJCBGJBHYBVwHFwHrgDlXd5nfMPwGTVPU+EVkI3KKqC0RkCnBUVQ+JyARgiaoOOdf7WUCY3qK+yctbm4/w3Jr9fHjgBHFRkdw8ZQh3zxxG3qDkUJfXMe0NjsSBrQMjs7nFYTOrQiVUATETeEhVr3GfPwigqj/2O2aJe8waEfEAR4AM9StKnNuCVQCDVLX+bO9nAWF6oy0llTy/pog/f1JCXaOPaTmp3DUzh/njBxLt6QMzipqDo2ynExjN4VG288zgyBjrBoZfl5UFR9CFKiBuBear6hfd53cB01X1K37HbHGPKXaf73GPKW/zOvep6pUB3mMRsAhg6NChFxcVFQXlsxgTbCdqG/jjhmJeWFdEUUUtGUkx3DEtm89OH8bAlNhQl9f1fD7nAsDmlkbZTijbHiA4BrRuaTSPccT3D13tfUyvDQgRGQ+8AVytqnvO9X7WgjB9gc+nrNhdxvNrili2s5QIEa4eN4C7Z+YwY0T/s99nu6/wD46yHW6X1XYo29V6EcS4VGcNKwQkwlluRCLc59Lmedv97vMLOVbc1lyg96IH/G+SPgbm/0eHTj1XQARzBbASINvveZa7LdAxxW4XUwpOdxIikgW8Dtx9vnAwpq+IiBAKxmZSMDaTAxW1vLiuiD9sOMhbW46QnhjDrFFpzBqZzqWj0nrWdNmuEhEBqcOcnzHXnN7eEhxuS+N4EagX1OdcXa4KqN9z33metzn+fMeq7+z7e4K2C0B2kWC2IDw4g9TzcIJgPfBZVd3qd8z9wES/QepPq+rtItIPWAH8UFVfa8/7WQvC9FV1jV7e2nKY5TvLWFVYQXm1MxSXkxbPpaPSmT0qnZkj0oK3vIfp00I5zfVa4Amcaa7PqOqPRORhYIOqviEiscDzwBTgGLBQVfeKyPeAB4Hdfi93taqWnu29LCBMOFBVdh2tZlVhOasKy1m37xjV9U2IwPjByW7rIp1LcvoHXo7cmDbsQjlj+qhGr49NxZUtgfHhgeM0epXoyAimDO3H7FFOYFyUlYInsg/MijJdzgLCmDBR29DE+v3HWV1YzsrCcrYdPokqJMV4mD6iP5eOTGf26HRGZyb2/QFv0y6hGqQ2xnSz+GgPc8ZkMGdMBgDHahpYs6eCVXvKWV1YztLtTi9tRlIMs0amcemodGaNSmdIP1uMz5zJWhDGhJHi47WsLqxgZWE5q/eUU17tLAM+PD2BS0emOQPeI9PoF28D3uHCupiMMWdovuf2qsIKVheWs3ZvBTUNXkRgwuAULnWn1E6zAe8+zQLCGHNezoD3CVbudrqkPvIb8J467PSA96QhNuDdl1hAGGMuWG1DEx/sO8bqPRWs3O0MeAMkxXqYOSKN2aOdazCGpyfYgHcvZoPUxpgLFh/t4YqxmVwxNhOAiup61uytYFVhOe/vLudt9/4Wg1NimT3aGeyeNSqd9MSYUJZtupC1IIwxF0xVOXCslvd3l7dcg3GyrgmAvEHJXOYGhl2w1/NZF5MxJqi8PmVLSSUrC8tZubucjUXHafD6iI6M4OJhqS3dUROGpBDZU++iF6YsIIwx3ar5gr2Vu8tYWVjBdnf8IiUuiktHpjFrVDqXjU5naP94G78IMRuDMMZ0q7YX7JVV1bN6j9O6WFlYzltbjgCQlRrX0h01a2S6LTjYw1gLwhjTrVSVveU1LYPda/dUUOW34ODsURnMHpVOfk4qsVE2fhFs1sVkjOmxmrw+PnEXHFy521lwsMmnxHgimJbTv2X8YtygZCJs/KLLWUAYY3qNmvom1u2rYOXuClYWlrHraDUAqfFRLfe/mDUynez+cTZ+0QVsDMIY02skxHiYmzuAubkDACg9WefMjnJbGH/ddBhwAmPCkBQmuj8ThqSQlWqh0ZWsBWGM6TVUlcLSatburWBzSSWbS06y+2gVTT7ne6xffBQTBqe0Cg5raZybtSCMMX2CiDB6QBKjByS1bKtr9LLjSBWbSyrZUlzJlkOV/Ob9vS2hkRIXxYQhya1Cw6bXto8FhDGmV4uNimRydj8mZ/dr2Vbf5GVnc2iUVLK5pJJnVu6j0euERnKspyUwmn8PS7PQaMsCwhjT58R4IpmU1Y9JWa1DY9eRardrygmOZ1ftp8HrA5xFCCcMTmFill9o9I8P65lTFhDGmLAQ44lkYpYTAM0amnzsOlrV0srYUlLJb1fvp6HJDY0YD+MGJztdU25wDE9LCJvQsIAwxoStaE8EE9xupoXutkZv69DYXHKS59YWtYRGol9ojM5MJDHWQ3x0JPHRHhKiPcTHRJIQ7SEuOpKE6Mhefe8MCwhjjPETFRnB+MEpjB+cwoJpzrZGr4/dR6vZcuj0mMYLa4uod0PjXKI9ESQ0B0hMJHHRnlbP41ueRxIf4zn/sTGRxHoiu6UVYwFhjDHnERUZwbjByYwbnMzt+dmAcwX4kZN1nGrwUtPgpba+yfnd0ERtg5eaevd3Q5NzTL2zr/nYE7WnWj2vbfRyIVcdxPsFx6Ssfvz3HVO6/HNbQBhjTAd4IiPISo3vstdTVeoafdQ0NFFb7wTL6bBpHS61Da2fD0mN67I6/FlAGGNMDyAixEVHOjdYSgx1NY7eO3pijDEmqIIaECIyX0R2ikihiDwQYH+MiPzB3b9ORHL89j3obt8pItcEs05jjDFnClpAiEgk8CTwKWAccIeIjGtz2BeA46o6Cvgp8BP33HHAQmA8MB/4uft6xhhjukkwWxCXAIWquldVG4CXgZvaHHMT8Dv38WJgnjjXut8EvKyq9aq6Dyh0X88YY0w3CWZADAEO+j0vdrcFPEZVm4BKIK2d5yIii0Rkg4hsKCsr68LSjTHG9OpBalV9SlXzVTU/IyMj1OUYY0yfEsyAKAGy/Z5nudsCHiMiHiAFqGjnucYYY4IomAGxHhgtIsNFJBpn0PmNNse8AXzefXwr8K46dzB6A1joznIaDowGPghircYYY9oI2oVyqtokIl8BlgCRwDOqulVEHgY2qOobwNPA8yJSCBzDCRHc414BtgFNwP2q6j3X+23cuLFcRIo6UXI6UN6J8/sS+1u0Zn+P1uzvcVpf+FsMO9uOPnPL0c4SkQ1nu+1euLG/RWv292jN/h6n9fW/Ra8epDbGGBM8FhDGGGMCsoA47alQF9CD2N+iNft7tGZ/j9P69N/CxiCMMcYEZC0IY4wxAVlAGGOMCSjsA+J8S5KHExHJFpFlIrJNRLaKyNdCXVOoiUikiHwkIn8JdS2hJiL9RGSxiOwQke0iMjPUNYWSiPwf97+TLSLyexGJDXVNXS2sA6KdS5KHkybgm6o6DpgB3B/mfw+ArwHbQ11ED/FfwN9UNRe4iDD+u4jIEOCrQL6qTsC5GHhhaKvqemEdELRvSfKwoaqHVfVD93EVzhfAGavohgsRyQKuA34T6lpCTURSgMtxVj9AVRtU9URoqwo5DxDnriMXDxwKcT1dLtwDol3Liocj9+5+U4B1oa0kpJ4A/hXwhbqQHmA4UAY863a5/UZEEkJdVKioagnwGHAAOAxUqurboa2q64V7QJgARCQReBX4uqqeDHU9oSAi1wOlqrox1LX0EB5gKvALVZ0C1ABhO2YnIqk4vQ3DgcFAgoh8LrRVdb1wDwhbVrwNEYnCCYcXVfW1UNcTQrOAG0VkP07X41wReSG0JYVUMVCsqs0tysU4gRGurgT2qWqZqjYCrwGXhrimLhfuAdGeJcnDhnu716eB7ar6eKjrCSVVfVBVs1Q1B+f/F++qap/7F2J7qeoR4KCIjHU3zcNZbTlcHQBmiEi8+9/NPPrgoH3QlvvuDc62JHmIywqlWcBdwGYR+djd9h1VfTOENZme45+BF91/TO0F7g1xPSGjqutEZDHwIc7sv4/og8tu2FIbxhhjAgr3LiZjjDFnYQFhjDEmIAsIY4wxAVlAGGOMCcgCwhhjTEAWEMb0ACJyha0Ya3oaCwhjjDEBWUAYcwFE5HMi8oGIfCwiv3LvF1EtIj917w3wjohkuMdOFpG1IrJJRF531+9BREaJyFIR+UREPhSRke7LJ/rdb+FF9wpdY0LGAsKYdhKRPGABMEtVJwNe4E4gAdigquOBFcAP3FOeA76tqpOAzX7bXwSeVNWLcNbvOexunwJ8HefeJCNwrmw3JmTCeqkNYy7QPOBiYL37j/s4oBRnOfA/uMe8ALzm3j+hn6qucLf/DvijiCQBQ1T1dQBVrQNwX+8DVS12n38M5AArg/+xjAnMAsKY9hPgd6r6YKuNIv/W5riOrl9T7/fYi/33aULMupiMab93gFtFJBNARPqLyDCc/45udY/5LLBSVSuB4yJymbv9LmCFe6e+YhG52X2NGBGJ79ZPYUw72b9QjGknVd0mIt8D3haRCKARuB/n5jmXuPtKccYpAD4P/NINAP/VT+8CfiUiD7uvcVs3fgxj2s1WczWmk0SkWlUTQ12HMV3NupiMMcYEZC0IY4wxAVkLwhhjTEAWEMYYYwKygDDGGBOQBYQxxpiALCCMMcYE9P8BjaslVEbVDpMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# training vs. validation loss\n", "learner.plot('loss')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Inspect Model\n", "\n", "#### Evaluate as Normal" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\r", "1/1 [==============================] - 0s 57ms/step - loss: 0.0186 - accuracy: 0.9943\n" ] }, { "data": { "text/plain": [ "[0.018631214275956154, 0.9943]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learner.model.evaluate(validation_dataset, steps=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Validation Metrics" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", " 0 0.99 1.00 1.00 980\n", " 1 1.00 1.00 1.00 1135\n", " 2 0.99 1.00 1.00 1032\n", " 3 0.99 0.99 0.99 1010\n", " 4 0.99 0.99 0.99 982\n", " 5 0.99 0.99 0.99 892\n", " 6 0.99 0.99 0.99 958\n", " 7 1.00 0.99 1.00 1028\n", " 8 0.99 0.99 0.99 974\n", " 9 0.99 0.99 0.99 1009\n", "\n", " accuracy 0.99 10000\n", " macro avg 0.99 0.99 0.99 10000\n", "weighted avg 0.99 0.99 0.99 10000\n", "\n" ] }, { "data": { "text/plain": [ "array([[ 979, 0, 0, 0, 0, 0, 0, 1, 0, 0],\n", " [ 0, 1135, 0, 0, 0, 0, 0, 0, 0, 0],\n", " [ 0, 1, 1028, 0, 0, 0, 0, 3, 0, 0],\n", " [ 0, 0, 2, 1003, 0, 4, 0, 0, 1, 0],\n", " [ 0, 0, 0, 0, 975, 0, 4, 0, 0, 3],\n", " [ 1, 0, 0, 7, 0, 883, 1, 0, 0, 0],\n", " [ 3, 1, 0, 0, 0, 1, 951, 0, 2, 0],\n", " [ 0, 2, 2, 0, 0, 0, 0, 1022, 0, 2],\n", " [ 2, 0, 2, 1, 0, 0, 0, 0, 967, 2],\n", " [ 0, 0, 0, 0, 5, 2, 0, 0, 2, 1000]])" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learner.validate(class_names=list(map(str, range(10))))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### View Top Losses" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "----------\n", "id:1014 | loss:7.4 | true:6 | pred:5)\n", "\n" ] } ], "source": [ "learner.view_top_losses(n=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Making Predictions" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "preds = learner.predict(val)\n", "preds = np.argmax(preds, axis=1)\n", "actual = learner.ground_truth(val)\n", "actual = np.argmax(actual, axis=1)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PredictedActual
077
122
211
300
444
\n", "
" ], "text/plain": [ " Predicted Actual\n", "0 7 7\n", "1 2 2\n", "2 1 1\n", "3 0 0\n", "4 4 4" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "df = pd.DataFrame(zip(preds, actual), columns=['Predicted', 'Actual'])\n", "df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Save Model and Reload Model" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "learner.save_model('/tmp/my_tf_model')" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "learner.load_model('/tmp/my_tf_model')" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\r", "1/1 [==============================] - 0s 176ms/step - loss: 0.0190 - accuracy: 0.9943\n" ] }, { "data": { "text/plain": [ "[0.018986882641911507, 0.9943]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learner.model.evaluate(validation_dataset, steps=1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 2 }