{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Basic training functionality" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [], "source": [ "from fastai.basic_train import *\n", "from fastai.gen_doc.nbdoc import *\n", "from fastai import *\n", "from fastai.vision import *" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[`basic_train`](/basic_train.html#basic_train) wraps together the data (in a [`DataBunch`](/basic_data.html#DataBunch) object) with a pytorch model to define a [`Learner`](/basic_train.html#Learner) object. This is where the basic training loop is defined for the [`fit`](/basic_train.html#fit) function. The [`Learner`](/basic_train.html#Learner) object is the entry point of most of the [`Callback`](/callback.html#Callback) functions that will customize this training loop in different ways (and made available through the [`train`](/train.html#train) module), notably:\n", "\n", " - [`Learner.lr_find`](/train.html#lr_find) will launch an LR range test that will help you select a good learning rate\n", " - [`Learner.fit_one_cycle`](/train.html#fit_one_cycle) will launch a training using the 1cycle policy, to help you train your model fast.\n", " - [`Learner.to_fp16`](/train.html#to_fp16) will convert your model in half precision and help you launch a training in mixed precision." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

class Learner[source]

\n", "\n", "> Learner(`data`:[`DataBunch`](/basic_data.html#DataBunch), `model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `opt_func`:`Callable`=`'Adam'`, `loss_func`:`Callable`=`None`, `metrics`:`Collection`\\[`Callable`\\]=`None`, `true_wd`:`bool`=`True`, `bn_wd`:`bool`=`True`, `wd`:`Floats`=`0.01`, `train_bn`:`bool`=`True`, `path`:`str`=`None`, `model_dir`:`str`=`'models'`, `callback_fns`:`Collection`\\[`Callable`\\]=`None`, `callbacks`:`Collection`\\[[`Callback`](/callback.html#Callback)\\]=``, `layer_groups`:`ModuleList`=`None`)\n", "\n", "Train `model` using `data` to minimize `loss_func` with optimizer `opt_func`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner, title_level=2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The main purpose of [`Learner`](/basic_train.html#Learner) is to train `model` using [`Learner.fit`](/basic_train.html#Learner.fit). After every epoch, all *metrics* will be printed, and will also be available to callbacks.\n", "\n", "The default weight decay will be `wd`, which will be handled using the method from [Fixing Weight Decay Regularization in Adam](https://arxiv.org/abs/1711.05101) if `true_wd` is set (otherwise it's L2 regularization). If `bn_wd` is False then weight decay will be removed from batchnorm layers, as recommended in [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://arxiv.org/abs/1706.02677). You can ensure that batchnorm layer learnable params are trained even for frozen layer groups, by enabling `train_bn`.\n", "\n", "To use [discriminative layer training](#Discriminative-layer-training) pass an [`nn.Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) for each layer group to be optimized with different settings.\n", "\n", "Any model files created will be saved in `path`/`model_dir`.\n", "\n", "You can pass a list of [`callbacks`](/callbacks.html#callbacks) that you have already created, or (more commonly) simply pass a list of callback functions to `callback_fns` and each function will be called (passing `self`) on object initialization, with the results stored as callback objects. For a walk-through, see the [training overview](/training.html) page. You may also want to use an `application` to fit your model, e.g. using the [`create_cnn`](/vision.learner.html#create_cnn) method:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Total time: 00:09\n", "epoch train_loss valid_loss accuracy\n", "1 0.067861 0.032277 0.992149 (00:09)\n", "\n" ] } ], "source": [ "path = untar_data(URLs.MNIST_SAMPLE)\n", "data = ImageDataBunch.from_folder(path)\n", "learn = create_cnn(data, models.resnet18, metrics=accuracy)\n", "learn.fit(1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Model fitting methods" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

fit[source]

\n", "\n", "> fit(`epochs`:`int`, `lr`:`Union`\\[`float`, `Collection`\\[`float`\\], `slice`\\]=`slice(None, 0.003, None)`, `wd`:`Floats`=`None`, `callbacks`:`Collection`\\[[`Callback`](/callback.html#Callback)\\]=`None`)\n", "\n", "Fit the model on this learner with `lr` learning rate, `wd` weight decay for `epochs` with `callbacks`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.fit)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Uses [discriminative layer training](#Discriminative-layer-training) if multiple learning rates or weight decay values are passed. To control training behaviour, use the [`callback`](/callback.html#callback) system or one or more of the pre-defined [`callbacks`](/callbacks.html#callbacks)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

fit_one_cycle[source]

\n", "\n", "> fit_one_cycle(`learn`:[`Learner`](/basic_train.html#Learner), `cyc_len`:`int`, `max_lr`:`Union`\\[`float`, `Collection`\\[`float`\\], `slice`\\]=`slice(None, 0.003, None)`, `moms`:`Point`=`(0.95, 0.85)`, `div_factor`:`float`=`25.0`, `pct_start`:`float`=`0.3`, `wd`:`float`=`None`, `callbacks`:`Optional`\\[`Collection`\\[[`Callback`](/callback.html#Callback)\\]\\]=`None`, `kwargs`)\n", "\n", "Fit a model following the 1cycle policy. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.fit_one_cycle)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Uses the [`OneCycleScheduler`](/callbacks.one_cycle.html#OneCycleScheduler) callback." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

lr_find[source]

\n", "\n", "> lr_find(`learn`:[`Learner`](/basic_train.html#Learner), `start_lr`:`Floats`=`1e-07`, `end_lr`:`Floats`=`10`, `num_it`:`int`=`100`, `stop_div`:`bool`=`True`, `kwargs`:`Any`)\n", "\n", "Explore lr from `start_lr` to `end_lr` over `num_it` iterations in `learn`. If `stop_div`, stops when loss explodes. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.lr_find)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Runs the learning rate finder defined in [`LRFinder`](/callbacks.lr_finder.html#LRFinder), as discussed in [Cyclical Learning Rates for Training Neural Networks](https://arxiv.org/abs/1506.01186). " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### See results" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

get_preds[source]

\n", "\n", "> get_preds(`ds_type`:[`DatasetType`](/basic_data.html#DatasetType)=``, `with_loss`:`bool`=`False`, `n_batch`:`Optional`\\[`int`\\]=`None`, `pbar`:`Union`\\[`MasterBar`, `ProgressBar`, `NoneType`\\]=`None`) → `List`\\[`Tensor`\\]\n", "\n", "Return predictions and targets on the valid, train, or test set, depending on `ds_type`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.get_preds)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

validate[source]

\n", "\n", "> validate(`dl`=`None`, `callbacks`=`None`, `metrics`=`None`)\n", "\n", "Validate on `dl` with potential `callbacks` and `metrics`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.validate)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Test time augmentation" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

TTA[source]

\n", "\n", "> TTA(`learn`:[`Learner`](/basic_train.html#Learner), `beta`:`float`=`0.4`, `scale`:`float`=`1.35`, `ds_type`:[`DatasetType`](/basic_data.html#DatasetType)=``, `with_loss`:`bool`=`False`) → `Tensors`" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.TTA, full_name = 'TTA')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Applies Test Time Augmentation to `learn` on the dataset `ds_type`. We take the average of our regular predictions (with a weight `beta`) with the average of predictions obtained thourh augmented versions of the training set (with a weight `1-beta`). The transforms decided for the training set are applied with a few changes `scale` controls the scale for zoom (which isn't random), the cropping isn't random but we make sure to get the four corners of the image. Flipping isn't random but applied once on each of those corner images (so that makes 8 augmented versions total)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Mixed precision training" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

to_fp16[source]

\n", "\n", "> to_fp16(`learn`:[`Learner`](/basic_train.html#Learner), `loss_scale`:`float`=`512.0`, `flat_master`:`bool`=`False`) → [`Learner`](/basic_train.html#Learner)\n", "\n", "Transform `learn` in FP16 precision. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.to_fp16)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Uses the [`MixedPrecision`](/callbacks.fp16.html#MixedPrecision) callback to train in mixed precision (i.e. forward and backward passes using fp16, with weight updates using fp32), using all [NVIDIA recommendations](https://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html) for ensuring speed and accuracy." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Discriminative layer training" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When fitting a model you can pass a list of learning rates (and/or weight decay amounts), which will apply a different rate to each *layer group* (i.e. the parameters of each module in `self.layer_groups`). See the [Universal Language Model Fine-tuning for Text Classification](https://arxiv.org/abs/1801.06146) paper for details and experimental results in NLP (we also frequently use them successfully in computer vision, but have not published a paper on this topic yet). When working with a [`Learner`](/basic_train.html#Learner) on which you've called `split`, you can set hyperparameters in four ways:\n", "\n", "1. `param = [val1, val2 ..., valn]` (n = number of layer groups)\n", "2. `param = val`\n", "3. `param = slice(start,end)`\n", "4. `param = slice(end)`\n", "\n", "If we chose to set it in way 1, we must specify a number of values exactly equal to the number of layer groups. If we chose to set it in way 2, the chosen value will be repeated for all layer groups. See [`Learner.lr_range`](/basic_train.html#Learner.lr_range) for an explanation of the `slice` syntax).\n", "\n", "Here's an example of how to use discriminative learning rates (note that you don't actually need to manually call [`Learner.split`](/basic_train.html#Learner.split) in this case, since fastai uses this exact function as the default split for `resnet18`; this is just to show how to customize it):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# creates 3 layer groups\n", "learn.split(lambda m: (m[0][6], m[1]))\n", "# only randomly initialized head now trainable\n", "learn.freeze()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Total time: 00:08\n", "epoch train_loss valid_loss accuracy\n", "1 0.036884 0.023377 0.993621 (00:08)\n", "\n" ] } ], "source": [ "learn.fit_one_cycle(1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Total time: 00:11\n", "epoch train_loss valid_loss accuracy\n", "1 0.025823 0.008318 0.997547 (00:11)\n", "\n" ] } ], "source": [ "# all layers now trainable\n", "learn.unfreeze()\n", "# optionally, separate LR and WD for each group\n", "learn.fit_one_cycle(1, max_lr=(1e-4, 1e-3, 1e-2), wd=(1e-4,1e-4,1e-1))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

lr_range[source]

\n", "\n", "> lr_range(`lr`:`Union`\\[`float`, `slice`\\]) → `ndarray`\n", "\n", "Build differential learning rates. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.lr_range)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Rather than manually setting an LR for every group, it's often easier to use [`Learner.lr_range`](/basic_train.html#Learner.lr_range). This is a convenience method that returns one learning rate for each layer group. If you pass `slice(start,end)` then the first group's learning rate is `start`, the last is `end`, and the remaining are evenly geometrically spaced.\n", "\n", "If you pass just `slice(end)` then the last group's learning rate is `end`, and all the other groups are `end/3`. For instance (for our learner that has 3 layer groups):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(array([1.e-05, 1.e-04, 1.e-03]), array([1.e-04, 1.e-04, 3.e-04]))" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.lr_range(slice(1e-5,1e-3)), learn.lr_range(slice(3e-4))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

unfreeze[source]

\n", "\n", "> unfreeze()\n", "\n", "Unfreeze entire model. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.unfreeze)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sets every layer group to *trainable* (i.e. `requires_grad=True`)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

freeze[source]

\n", "\n", "> freeze()\n", "\n", "Freeze up to last layer. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.freeze)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Sets every layer group except the last to *untrainable* (i.e. `requires_grad=False`)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

freeze_to[source]

\n", "\n", "> freeze_to(`n`:`int`)\n", "\n", "Freeze layers up to layer `n`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.freeze_to)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

split[source]

\n", "\n", "> split(`split_on`:`SplitFuncOrIdxList`)\n", "\n", "Split the model at `split_on`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.split)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A convenience method that sets `layer_groups` based on the result of [`split_model`](/torch_core.html#split_model). If `split_on` is a function, it calls that function and passes the result to [`split_model`](/torch_core.html#split_model) (see above for example)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Saving and loading models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Simply call [`Learner.save`](/basic_train.html#Learner.save) and [`Learner.load`](/basic_train.html#Learner.load) to save and load models. Only the parameters are saved, not the actual architecture (so you'll need to create your model in the same way before loading weights back in). Models are saved to the `path`/`model_dir` directory." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

load[source]

\n", "\n", "> load(`name`:`PathOrStr`, `device`:[`device`](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device)=`None`)\n", "\n", "Load model `name` from `self.model_dir` using `device`, defaulting to `self.data.device`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.load)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

save[source]

\n", "\n", "> save(`name`:`PathOrStr`, `return_path`:`bool`=`False`) → `Union`\\[`NoneType`, `str`\\]\n", "\n", "Save model with `name` to `self.model_dir`, and return path if `return_path`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.save)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### See results" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

show_results[source]

\n", "\n", "> show_results(`ds_type`=``, `rows`:`int`=`5`, `kwargs`)\n", "\n", "Show `rows` result of predictions on `ds_type` dataset. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.show_results)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

predict[source]

\n", "\n", "> predict(`img`:[`ItemBase`](/core.html#ItemBase), `pbar`:`Union`\\[`MasterBar`, `ProgressBar`, `NoneType`\\]=`None`, `kwargs`)\n", "\n", "Return prect class, label and probabilities for `img`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.predict)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

validate[source]

\n", "\n", "> validate(`dl`=`None`, `callbacks`=`None`, `metrics`=`None`)\n", "\n", "Validate on `dl` with potential `callbacks` and `metrics`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.validate)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Segmentation model" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

Learner_create_unet[source]

\n", "\n", "> Learner_create_unet(`data`:[`DataBunch`](/basic_data.html#DataBunch), `arch`:`Callable`, `pretrained`:`bool`=`True`, `split_on`:`Union`\\[`Callable`, `Collection`\\[`ModuleList`\\], `NoneType`\\]=`None`, `kwargs`:`Any`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.create_unet, doc_string=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Build a Unet [`Learner`](/basic_train.html#Learner) for segmentation tasks from [`data`](/vision.data.html#vision.data), using `arch` that may be `pretrained` if that flag is `True`. `split_on` will overwrite the default way the layers are split for differential learning rates. `kwargs` are passed to the [`Learner`](/basic_train.html#Learner) constructor." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Other methods" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

init[source]

\n", "\n", "> init(`init`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.init)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Initializes all weights (except batchnorm) using function `init`, which will often be from PyTorch's [`nn.init`](https://pytorch.org/docs/stable/nn.html#torch-nn-init) module." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

mixup[source]

\n", "\n", "> mixup(`learn`:[`Learner`](/basic_train.html#Learner), `alpha`:`float`=`0.4`, `stack_x`:`bool`=`False`, `stack_y`:`bool`=`True`) → [`Learner`](/basic_train.html#Learner)\n", "\n", "Add mixup https://arxiv.org/abs/1710.09412 to `learn`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.mixup)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Uses [`MixUpCallback`](/callbacks.mixup.html#MixUpCallback)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

pred_batch[source]

\n", "\n", "> pred_batch(`ds_type`:[`DatasetType`](/basic_data.html#DatasetType)=``, `pbar`:`Union`\\[`MasterBar`, `ProgressBar`, `NoneType`\\]=`None`) → `List`\\[`Tensor`\\]\n", "\n", "Return output of the model on one batch from valid, train, or test set, depending on `ds_type`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.pred_batch)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Get the first batch of predictions. Mainly useful for debugging and quick tests." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

create_opt[source]

\n", "\n", "> create_opt(`lr`:`Floats`, `wd`:`Floats`=`0.0`)\n", "\n", "Create optimizer with `lr` learning rate and `wd` weight decay. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.create_opt)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You generally won't need to call this yourself - it's used to create the [`optim`](https://pytorch.org/docs/stable/optim.html#module-torch.optim) optimizer before fitting the model." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

dl[source]

\n", "\n", "> dl(`ds_type`:[`DatasetType`](/basic_data.html#DatasetType)=``)\n", "\n", "Return DataLoader for DatasetType `ds_type`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.dl)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

class Recorder[source]

\n", "\n", "> Recorder(`learn`:[`Learner`](/basic_train.html#Learner)) :: [`LearnerCallback`](/basic_train.html#LearnerCallback)\n", "\n", "A [`LearnerCallback`](/basic_train.html#LearnerCallback) that records epoch, loss, opt and metric data during training. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder, title_level=2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A [`Learner`](/basic_train.html#Learner) creates a [`Recorder`](/basic_train.html#Recorder) object automatically - you do not need to explicitly pass to `callback_fns` - because other callbacks rely on it being available. It stores the smoothed loss, hyperparameter values, and metrics each batch, and provides plotting methods for each. Note that [`Learner`](/basic_train.html#Learner) automatically sets an attribute with the snake-cased name of each callback, so you can access this through `Learner.recorder`, as shown below." ] }, { "cell_type": "markdown", "metadata": { "hide_input": true }, "source": [ "### Plotting methods" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

plot[source]

\n", "\n", "> plot(`skip_start`:`int`=`10`, `skip_end`:`int`=`5`)\n", "\n", "Plot learning rate and losses, trimmed between `skip_start` and `skip_end`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.plot)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is mainly used with the learning rate finder, since it shows a scatterplot of loss vs learning rate." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VFX+x/H3N52EkBAINQmhdwgQQGygoiK7wtph7bq6q2LXXXWb666ubV11sfxYe8Uu4qLYsSBI6L0jHUJNKOnn90fGGCMkAXJzM5PP63nmeWbunLnzPQyZz9x7zz3XnHOIiIgAhPldgIiI1B0KBRERKaNQEBGRMgoFEREpo1AQEZEyCgURESmjUBARkTIKBRERKeNpKJjZMDNbamYrzOy2Azzfxsw+NbN5ZvaFmaV4WY+IiFTOvDqj2czCgWXAycB6YAYw2jm3qFybN4D3nXPPm9mJwKXOuQsrW2/Tpk1denq6JzWLiISqmTNnbnPOJVfVLsLDGgYAK5xzqwDMbDwwElhUrk034KbA/c+Bd6taaXp6OllZWTVcqohIaDOz76vTzsvdR62BdeUerw8sK28ucGbg/hlAvJk1qbgiM7vSzLLMLCs7O9uTYkVExP8DzbcAg81sNjAY2AAUV2zknBvnnMt0zmUmJ1e59SMiIofJy91HG4DUco9TAsvKOOc2EthSMLOGwFnOuV0e1iQiIpXwckthBtDRzNqaWRQwCnivfAMza2pmP9RwO/CMh/WIiEgVPAsF51wRMAaYDCwGXnfOLTSzu8xsRKDZEGCpmS0DmgN3e1WPiIhUzbMhqV7JzMx0Gn0kInJozGymcy6zqnZ+H2gWEZE6xMsDzXIY8gqLeW3GOpxzNGsUQ7P4aJrFx5Ca1AAz87s8EQlxCgWPPfb5CjbvzqNvm0T6pDamTZPYg365F5c4bhg/hw8Xbv7Zc33TEnngnN60T27odckiUo8pFDw0fdV2Hpi8lMhw48VppScTNomL4qSuzbj9tK40josqa+uc40/vLuDDhZv50y+68qs+rdmak0/2nnxWbN3Do58uZ/gjX3HLKZ257Ni2hIdpq0FEap4ONHukuMQxYuzX7NxbwMc3DWbtjn3MXruLrDU7eG/uRhrHRXH/Wb04oUszAP710VL+89kKrh7Snt8P6/Kz9W3NyeOOdxbwyeIt9GvTmL+N6E73Vo20S0lEqqW6B5oVCh55fcY6fv/WPB4ZlcHIjJ/O7rFw425uem0uS7fkMqp/KulN47j3gyWcl5nKvWf1POgXvXOOCXM28tf3FrJ7fyFpSbEM7dqck7s1p396YyLCNW5ARA5MoeCj3LxCTnhwCmlJDXjrqqMP+CWfX1TMvz9ezrgvV1Li4JRuzXn8/L7V+mLfsbeADxZs4pNFW/hm5XYKikqICDPioiNoGB1BXHQ4DSLDKXaOwiJHYXEJxc6R2SaJczJTGNg2SVsYIvWMQsFH9324hCe+WMmEa46hd2pipW1nfr+DL5Zmc80JHYiJDD/k99qbX8RXy7cxb/0u9uYXsSe/mL35RewvLCYizIgMDyMyIozikhK+XLaNPflFpCXFcna/FEYPSCM5PvpwuykiQUSh4JO12/cx9KEp/LJ3Sx46N8Pvcn5if0ExHy7cxBtZ65m6cjvNG0XzwmUD6dwivkbfp6i4hGmrdvDBgk0s2ZxLWlIs7ZPjaJfckE7NG9I+uaG2VERqmULBJ797cSZTlmXz+S1DaJEQ43c5B7Vw424ufXYG+UUlPHNJJv3aJB3xOuev382L09bw8aIt7NxXSIPIcLq3asSGXfvZtDuvrF3P1glcfmxbhvdsSVTEj7vLnHNszc0noUHkYW01icjBKRRqWWFxCX+buJCXpq3lllM6MebEjn6XVKV1O/Zx4dPT2ZyTxxMX9OOEzs0Oaz3FJY4np6zkoY+X0SAynKFdmzGsR0sGd0qmQVTpl/ve/CJWb9vLrLU7eX7qGlZm76V5o2guGNiGsDBjzrpdzF23i625+bRMiGHchZn0TEmoye6K1GsKhVq0c28BV788i29Xbee3g9vx+1O7BM15BNm5+Vzy7Hcs3ZzLP8/sydn9Ug5p186m3fu58bU5TFu1g1/2asndZ/QkoUFkpa8pKXFMWZ7NM1+v5qvl2wBo1zSO3qmJdG0Zz/NTv2fbnnzuP7vXz0ZuicjhUSjUkmVbcrn8+RlsycnnvrN6ckafFL9LOmQ5eYVc+UIW01btYEB6Enf8oisZ5Q6Qr9+5j6e/Xs3bszbQMDqCtk3jSG8aS9OG0Tz7zZrSraQR3Q85UAA27NpPw6gIEmJ/DJJte/K5+qVZfLdmB78b3J5bT+0cNCErUlcpFDxWWFzCK9PXcv+HS4iNjmDchf3ok9bY77IOW1FxCeNnrOPhT5axbU8Bp/duxTn9Unhr1nren7cJA4b1aEF4mLFm215Wb9tLTl4RvVISeGRUH9o2javRegqKSrhz4kJemb6Wo9s3YcyJHRjUrskBQ2dfQRENIsN18FqkEgoFD32xdCv/+N9iVmzdw9Htm/Cvc3vTMqGBrzXVlD35RfzflJX896tV5BWW0DA6gtEDUrn0mLa0Svyxj845cvKKaBQT4emX8cvTv+fByUvZua+QTs0bctGgdIb3bMnCjbv5avk2vlyWzZLNuWS2aczNp3RmUPufXeJbRFAo1Jj8omI2785j0+48Nu3ez7uzNzJlWTbpTWK5Y3hXTu7WPCR/oW7encf01dsZ0rlZlccIvJZXWMx7czfy/NQ1LNyYU7Y8KjyMzPTG9GydwLtzNrAlJ59jOzTlllM7/2T3l4goFI7Yuh37uPmNuXy3esdPlsfHRHD9SR25aFD6T4ZTivecc8xau5Ovl2+nV2oCA9smERtVOqdjXmExL037nse/WMmOvQWMzGjF30Z0JzE2qoq1itQPCoUKnHPV/kU/eeFmbn1jLs7Bpcekk9YkjpYJMbRIiKF1YgONoa/D9uQXMW7KSh7/YiVNGkZx/9m9Gdwp2e+yRHynUKjgqa9W8dDHy0hsEElCbBSNYyNpHBdFl+bx9EhJoGfrBOJjIvjnpCU8N3UNvVISGDu6L2lNYj3ohXhtwYbd3PjaHJZv3cMFR6Vxx/CuZVsVIvWRQqGCaau288miLezaX8iufQXs2ldI9p58vt++r6xNXFQ4ewuKueyYtvzhtM5ER2iLIJjlFRbz4OSlPP3Nato2iWPsr/vSrVUjv8sS8YVCoZpy8wpZuDGHBRt2s2xLLqd2b8FJXZvX2PrFf1NXbuOG8XPYvb+QO0d0Z1T/1JAcHCBSGYWCSDnb9uRz42tz+Gr5NkZmtOKeM3oSF63dSVJ/VDcUNHxG6oWmDaN5/tIB3HJKJybO3cjpY79mUbnhrSJSSqEg9UZYmDHmxI68/Juj2JNXxK8e/4YXvl1DsG0ti3hJoSD1zqD2Tfjg+uM4pn0T/jJhIb99cSa79hX4XZZInaBQkHqpScNonr64P3/6RVc+X7qV4Y98xdLNuX6XJeI7hYLUW2Fhxm+Oa8dbVx1NUYnjomems27HvqpfKBLCFApS7/VKSeTFyweyv6CYC5+eTnZuvt8lifhGoSACdG4Rz7OXDmBLTulFh3LyCv0uScQXCgWRgH5tGvPEBX1ZujmX3zyfxf6CYr9LEql1CgWRcoZ0bsa/zu3NjDU76PePj7nsuRk8/fVqlm3J1dBVqRd0SqdIBSMzWpMcH80H8zfzzYptfLZkKwB90hIZd2EmyfHRPlco4h1NcyFShQ279vPp4i38c9ISmsZH8dylA2if3NDvskQOiaa5EKkhrRMbcNGgdF777VHsLyjmzMen/uziSyKhQqEgUk29UhJ55+pjaNowiguems47s9frOIOEHIWCyCFITYrlrauOJiMtkRtfm8vpY7/mf/M2UVyicJDQoFAQOUSJsVG8/JuB3H9WL/blF3PNK7MY+tAUxn+3lqLiEr/LEzkiCgWRwxAZHsa5/VP5+KbBPH5+X+Kiw7nt7fmcPvYbZq3d6Xd5IofN01Aws2FmttTMVpjZbQd4Ps3MPjez2WY2z8yGe1mPSE0LDzOG92zJxDHH8vj5fdm5t4CznpjK7W/P18yrEpQ8G5JqZuHAMuBkYD0wAxjtnFtUrs04YLZz7gkz6wZMcs6lV7ZeDUmVumxPfhEPf7yMZ6euIbFBJMN6tKBH6wR6tEqgU4uGuu63+Ka6Q1K9PHltALDCObcqUNB4YCSwqFwbB/xwJfUEYKOH9Yh4rmF0BH/6ZTfO7JvCA5OX8N6cjbw8fS0AEWHG+QPTuHNEd10jWuosL0OhNbCu3OP1wMAKbe4EPjKza4E4YKiH9YjUmm6tGvHspQMoKXGs27mPBRty+HTJFp7/9nsax0Vxw9BOfpcockB+T3MxGnjOOfcvMxsEvGhmPZxzPxnCYWZXAlcCpKWl+VCmyOEJCzPaNImjTZM4hvdsQZgZD3+ynJTGsZzdL8Xv8kR+xssDzRuA1HKPUwLLyrsceB3AOfctEAM0rbgi59w451ymcy4zOTnZo3JFvGVm3HNGT47t0JTb3prH18u3+V2SyM94GQozgI5m1tbMooBRwHsV2qwFTgIws66UhkK2hzWJ+CoqIozHL+hLh2YNueqlmSzZnON3SSI/4VkoOOeKgDHAZGAx8LpzbqGZ3WVmIwLNbgauMLO5wKvAJU7zBkiIaxQTybOX9icuOoLLnp3Bzr0auip1h2ZJFfHJ/PW7OeuJqRzXsSlPXZypEUniKc2SKlLH9UxJ4I7hXfh0yVae+mq13+WIAAoFEV9dfHQ6p3Zvzn0fLtH0GFInKBREfGRm3H9Wb1okxHDtK7PZva/Q75KknlMoiPgsITaS/4zuw5acPG55c66u0SC+UiiI1AF90hpz22ld+HjRFh7/YqXf5Ug9plAQqSMuP7YtIzNa8cDkpXy0cLPf5Ug9pVAQqSPMjPvO6kWvlARufG0OSzfn+l2S1EMKBZE6JCYynHEXZhIbHcFvXtCJbVL7FAoidUyLhBjGXdiPLTn5XP3yLAp1iU+pRQoFkTqoT1pj7j2zJ9+u2s6f312gEUlSa/yeOltEDuLMvimsyt7L2M9XkNK4AWNO7Oh3SVIPKBRE6rCbT+nExl37efCjZbRKbMCZfXUNBvGWQkGkDjMz7j2rF5tz8vj9m/No3iiGYzr87JIjIjVGxxRE6rioiDCevLAf7ZMb8rsXZ7Joo67BIN5RKIgEgfLXYDjnyal8MH+T3yVJiFIoiASJVokNeOeao+nUIp6rXp7FvR8soUjDVaWGKRREgkjLhAaMv/Iozh+YxpNTVnLxs9+xQye4SQ1SKIgEmeiIcO4+oyf3n9WLGWt2MnrcNAqKtMUgNUOhIBKkzu2fyuO/7svSLbn83xTNrCo1Q6EgEsSGdmvOL3q15D+frWBl9h6/y5EQoFAQCXJ/Pb0b0ZFh3PH2fE2HIUdMoSAS5JrFx3DH8K5MX72DN7LW+12OBDmFgkgIOC8zlQHpSdw9aTHZufl+lyNBTKEgEgLCwox7zuzB/oJi7np/kd/lSBBTKIiEiA7N4rnmhA5MnLuRJzUaSQ6TJsQTCSFjTuzA8q253PvBEpJiozi3f6rfJUmQUSiIhJDwMOOhczPYvb+Q296eR0JsJKd2b+F3WRJEtPtIJMRERYTxfxf2o3dqIte+OptvV273uyQJIgoFkRAUGxXBs5f0p01SLFe8kMU3K7b5XZIECYWCSIhKjI3ixcsH0ioxhgufns5/v1ylk9ukSgoFkRDWIiGGd64+hmE9WnD3pMVcN34O+wqK/C5L6jCFgkiIi4uO4LFf9+UPw7rwv3kbOfPxqWzYtd/vsqSOUiiI1ANmxlVD2vPcpQNYv3M/d7w93++SpI5SKIjUI8d3SuaGoR2ZsiybL5dl+12O1EEKBZF65sJBbUhNasA9kxZTXKIDz/JTCgWReiY6Ipw/DOvCks25vD1Ls6rKTykUROqhX/RsSUZqIg9+tJT9BcV+lyN1iEJBpB4yM/70i65sycnnqa9W+V2O1CGehoKZDTOzpWa2wsxuO8Dz/zazOYHbMjPb5WU9IvKjzPQkTuvRgiemrGRrbp7f5Ugd4VkomFk48BhwGtANGG1m3cq3cc7d6JzLcM5lAP8B3vaqHhH5ud8P60JBUQn//ni536VIHeHllsIAYIVzbpVzrgAYD4yspP1o4FUP6xGRCto2jeOiQemMn7GWBRt2+12O1AFehkJrYF25x+sDy37GzNoAbYHPPKxHRA7g+qEdSYqN4s73FmpuJKkzB5pHAW865w44DMLMrjSzLDPLys7WCTciNSmhQSR/GNaFrO93MmHORr/LEZ95GQobgPKXfUoJLDuQUVSy68g5N845l+mcy0xOTq7BEkUE4Ox+KfRKSeCeSYvZk68J8+ozL0NhBtDRzNqaWRSlX/zvVWxkZl2AxsC3HtYiIpUICzP+NqI7W3PzGfvZCr/LER9VKxTMrL2ZRQfuDzGz68wssbLXOOeKgDHAZGAx8LpzbqGZ3WVmI8o1HQWMd9qZKeKrPmmNObtfCk9/vYrV2/b6XY74xKrzXWxmc4BMIB2YBEwAujvnhnta3QFkZma6rKys2n5bkXpha24eJz44hf7pjXn20gF+lyM1yMxmOucyq2pX3d1HJYFf/mcA/3HO3Qq0PJICRaTuaRYfw3UndeDzpdm6hGc9Vd1QKDSz0cDFwPuBZZHelCQifrpoUDqtExtw7wdLKNEsqvVOdUPhUmAQcLdzbrWZtQVe9K4sEfFLTGQ4N5/SifkbdvP+/E1+lyO1rFqh4Jxb5Jy7zjn3qpk1BuKdc/d5XJuI+GRkRmu6tIjnwclLKSgq8bscqUXVHX30hZk1MrMkYBbwXzN7yNvSRMQv4WHGbad1Ye2Ofbw8/Xu/y5FaVN3dRwnOuRzgTOAF59xAYKh3ZYmI3wZ3SmZQuyb857MV5OYV+l2O1JLqhkKEmbUEzuXHA80iEsLMjNuHd2HH3gLGfalrLtQX1Q2Fuyg9CW2lc26GmbUDNNeuSIjrlZLIL3u15KmvVrN5t665UB9U90DzG865Xs65qwKPVznnzvK2NBGpC35/aheKnePuSYv9LkVqQXUPNKeY2TtmtjVwe8vMUrwuTkT8l9YklqsGt2fi3I1M1QltIa+6u4+epXQyu1aB28TAMhGpB64a0p7UpAb85b2FGqIa4qobCsnOuWedc0WB23OA5rAWqSdiIsP524jurNi6h2e+We13OeKh6obCdjO7wMzCA7cLgO1eFiYidcuJXZoztGtzHv10ORt37fe7HPFIdUPhMkqHo24GNgFnA5d4VJOI1FF/Pb0bxSWOu/+ng86hqrqjj753zo1wziU755o5534FaPSRSD2TmhTLmBM68L/5m5iyTJfGDUVHcuW1m2qsChEJGlcc3472yXHc/tY8nekcgo4kFKzGqhCRoBETGc4D5/Rmc04e90xa4nc5UsOOJBQ00bpIPdU3rTFXHNeOV79by1fLtRsplFQaCmaWa2Y5B7jlUnq+gojUUzee3In2yXH84U3tRgollYaCcy7eOdfoALd451xEbRUpInXPT3cjaTRSqDiS3UciUs/9uBtpHV9qNFJIUCiIyBH5YTfSbW/NI0e7kYKeQkFEjkhMZDgP/rAbSSe1BT2FgogcsT5pjbni+HaMn6HdSMFOoSAiNeLGoZ3o0KyhdiMFOYWCiNQI7UYKDQoFEakxGamJXHl8e8bPWKe5kYKUQkFEatQNQzvSsVlDbnljLpt2a4rtYKNQEJEaFRMZzthf92V/QTGXP5fF3vwiv0uSQ6BQEJEa17lFPGN/3Yclm3O4fvxsiks0VVqwUCiIiCeGdG7GnSO688nirfxT02AEDc1fJCKeuWhQOquy9/LU16tpmxzH+QPb+F2SVEFbCiLiqT//shsndE7mLxMWMmvtTr/LkSooFETEU+FhxiOj+9AyIYbrx8/WNNt1nEJBRDzXKCaSh8/LYMPO/fx1wkK/y5FKKBREpFZkpidx7YkdeXv2BibM2eB3OXIQCgURqTXXntiBfm0a86d3FrBuxz6/y5EDUCiISK2JCA/j4fMyALjxtTkUFZf4XJFU5GkomNkwM1tqZivM7LaDtDnXzBaZ2UIze8XLekTEf6lJsfz9Vz3I+n4nj32+0u9ypALPQsHMwoHHgNOAbsBoM+tWoU1H4HbgGOdcd+AGr+oRkbrjV31aMzKjFY9+tpw563b5XY6U4+WWwgBghXNulXOuABgPjKzQ5grgMefcTgDn3FYP6xGROuSukT1oHh/NDeNna36kOsTLUGgNrCv3eH1gWXmdgE5m9o2ZTTOzYR7WIyJ1SEKDSB46L4Pvd+zjH/9b5Hc5EuD3geYIoCMwBBgN/NfMEis2MrMrzSzLzLKyszVHu0ioOKpdE357fHte/W4dHy3c7Hc5grehsAFILfc4JbCsvPXAe865QufcamAZpSHxE865cc65TOdcZnJysmcFi0jtu+nkTnRr2Yjb3p7P1tw8v8up97wMhRlARzNra2ZRwCjgvQpt3qV0KwEza0rp7qRVHtYkInVMVEQYj47OYG9+EWNemU1BkYap+smzUHDOFQFjgMnAYuB159xCM7vLzEYEmk0GtpvZIuBz4Fbn3HavahKRuqlDs3juO6sX363ewd8mahoMP3k6dbZzbhIwqcKyv5S774CbAjcRqcd+1ac1Szbn8uSUlXRpEc+Fg9L9Lqle8vtAs4hImVtP7cxJXZpx58RFTF2xze9y6iWFgojUGeFhxsOjMmjXNI6rX5nF99v3+l1SvaNQEJE6JT4mkqcuzgTgyhdmsr+g2OeK6heFgojUOW2axPHoqD4s25rLnycs8LucekWhICJ10vGdkrn2xI68OXM9r2etq/oFUiMUCiJSZ11/UkeO6dCEP7+7gMWbcvwup15QKIhInRUeZjx8Xh8SGkRyzcuzdH3nWqBQEJE6LTk+mv+M7sOa7Xu5/e35lJ7eJF5RKIhInTewXRNuPqUz78/bxMR5m/wuJ6QpFEQkKPxucHt6pyby1wkL2LYn3+9yQpZCQUSCQniY8eDZvdibX8xfJ2h+JK8oFEQkaHRsHs/1Qzvyv/mb+GC+diN5QaEgIkHlyuPb0aN1I/48YQE79xb4XU7IUSiISFCJDA/jgbN7s2tfoabZ9oBCQUSCTteWjbjmhA68O2cjb85c73c5IUWhICJBacyJHTimQxNuf3se01bp2lw1RaEgIkEpMjyMx3/dj7SkWH774kxWZe/xu6SQoFAQkaCVEBvJs5cMIDzMuOy5GTrwXAMUCiIS1NKaxPLfi/qxcXcev31xJvlFuv7CkVAoiEjQ69cmiQfO7sV3a3bwz0lL/C4nqCkURCQkjMxozcWD2vDc1DV8u1IHng+XQkFEQsYfTutCepNYbn1zLnvyi/wuJygpFEQkZMRGRfDgOb3ZsGs/90xa7Hc5QUmhICIhJTM9iSuOa8cr09cyZVm23+UEHYWCiIScm07uRIdmDfnDm/PYvV9XazsUCgURCTkxkeH865zeZO/J5y8TFuhqbYdAoSAiIal3aiI3nNSRCXM28tL0tX6XEzQUCiISsq45oQNDOidz18SFzFm3y+9ygoJCQURCVliY8fB5GTSLj+Hql2ayQ9NgVEmhICIhLTE2iicv6Me2PQVcP342xSU6vlAZhYKIhLyeKQncOaI7Xy3fxu1vz2Pi3I3MWLODdTv2UVBU4nd5dUqE3wWIiNSG0QNSWbhxNy9PX8vrWT9emCc+JoJnLulP//QkH6urOyzYhmplZma6rKwsv8sQkSC1e18hm3Py2JyTx5bdeTw5ZSXb9uTzxu+OpnOLeL/L84yZzXTOZVbVTruPRKReSYiNpHOLeAZ3Subc/qk8f9kAYiLDueiZ6azfuc/v8nynUBCRei01KZbnLxvAvoJiLnrmu3o/QkmhICL1XteWjXj64v6s37mfS5+bwb6C+jvDqkJBRAQY0DaJsaP7MH/9Lm5+fS4l9XToqkJBRCTglO4tuGN4Vz5YsJlHP1vudzm+0JBUEZFyLj+2LYs35fLwJ8vp0iKeYT1alj1XUuJ4f/4mIsOM03q2rGQtwcvTLQUzG2ZmS81shZnddoDnLzGzbDObE7j9xst6RESqYmbcfUYPMlITufG1uSzamAPAtFXbGfnYN1z36myueWUWWWt2+FypNzw7T8HMwoFlwMnAemAGMNo5t6hcm0uATOfcmOquV+cpiEht2JqTx+ljvyYiLIyuLRvxyeIttEqI4YahnRj7+QqKSxyTrj+OhAaRfpdaLXXhPIUBwArn3CrnXAEwHhjp4fuJiNSYZo1iGHdhJtl78pm2aju3ntqZz24Zwrn9U3lkVAabc/L44zvzQ+5aDV4eU2gNrCv3eD0w8ADtzjKz4yndqrjRObeuYgMzuxK4EiAtLc2DUkVEfq53aiKTbzieRjERNGkYXba8T1pjbhzakQc/WsaQzs04u1+Kj1XWLL9HH00E0p1zvYCPgecP1Mg5N845l+mcy0xOTq7VAkWkfmvbNO4ngfCDq4Z0YEDbJP4yYQFrtu31oTJveBkKG4DUco9TAsvKOOe2O+fyAw+fAvp5WI+ISI0JD1yrISLMuPbV2ezJD40T3rwMhRlARzNra2ZRwCjgvfINzKz8mK4RwGIP6xERqVGtEhvw4Dm9WbQph/Ofms7OEJgiw7NQcM4VAWOAyZR+2b/unFtoZneZ2YhAs+vMbKGZzQWuAy7xqh4RES+c0r0FT5zfl8Wbcjhv3Ldsycnzu6QjoqmzRURqwNSV27ji+SySGkbx0uUDaRYfw+y1O5m2ajuz1+3iwqPacEr3Fr7VV90hqQoFEZEaMnfdLi5+9juKih0FRSUUFJcQZtCoQSRFxY73rz2W9KZxvtSmUBAR8cHyLaVTZKQ0bsBR7ZrQL70xuXlFDH/kK1KTGvDWVUcTHRFe63UpFERE6pCPF23hiheyuOTodO4c0b3W378unNEsIiIBJ3drzqXHpPPc1DVMXrjZ73IOSqEgIlJLbjutCz1bJ3DrG3Pr7KU/FQoiIrUkOiKcsb/uQ4mDc578lrdmrq/WxXxKShwPTl5aK8NdFQoiIrWoTZM4Xrh8AE0bRnPzG3MRGLqBAAAIoklEQVQ5fezXfLNi20HbO+e46/1FjP18BR/Vwm4nhYKISC3rm9aYCdccwyOjMti1r5Dzn5rO5c/NOOCWwGOfr+C5qWv4zbFtueCoNp7XplAQEfFBWJgxMqM1n948mNtP68I3K7dxyr+/5P15G8vavPrdWh78aBln9GnNHcO7Ymae16XLcYqI+CgmMpzfDm7Pyd2ac+Prcxnzymw+XrSF4zsm88d35jO4UzL3n92LsDDvAwF0noKISJ1RVFzC41+s5JFPl1Nc4shITeSVKwYSG3Xkv9+re56CthREROqIiPAwrjupI0M6J/P2rA1cf1LHGgmEQ6qhVt9NRESq1CslkV4pib68tw40i4hIGYWCiIiUUSiIiEgZhYKIiJRRKIiISBmFgoiIlFEoiIhIGYWCiIiUCbppLswsG/i+wuIEYHcVyyp7/MP98suaAgefz7ZqB6rpUNqoT9W7fyR9qk5/KmtXnf5UXFbV/dr4jCprF6x9qo2/pfL3g7FPHZ1zCVVW5pwL+hswrqpllT3+4X6FZVk1XdOhtFGfqn3/sPtUnf5U1q46/TnUPtXGZxSKfaqNv6VQ6lNlt1DZfTSxGssqezzxIG2ORHXWVVkb9al6949EdddzsHbV6U/FZerToasrf0vVraU6/O7TQQXd7qPaYmZZrhozCgYT9anuC7X+gPoUbEJlS8EL4/wuwAPqU90Xav0B9SmoaEtBRETKaEtBRETK1ItQMLNnzGyrmS04jNf2M7P5ZrbCzB61chdJNbNrzWyJmS00s/trtupKa6rx/pjZnWa2wczmBG7Da77ySuvy5DMKPH+zmTkza1pzFVerLi8+p7+b2bzAZ/SRmbWq+corrcuLPj0Q+DuaZ2bvmFmtXkjAoz6dE/heKDGz4Dr2cCTDqoLlBhwP9AUWHMZrvwOOAgz4ADgtsPwE4BMgOvC4WZD3507gllD6jALPpQKTKT23pWmw9wloVK7NdcCTIdCnU4CIwP37gPtCoE9dgc7AF0BmbfbnSG/1YkvBOfclsKP8MjNrb2YfmtlMM/vKzLpUfJ2ZtaT0j3CaK/2kXwB+FXj6KuBe51x+4D22etuLH3nUH1952Kd/A78Hav3gmRd9cs7llGsaRy33y6M+feScKwo0nQakeNuLn/KoT4udc0tro/6aVi9C4SDGAdc65/oBtwCPH6BNa2B9ucfrA8sAOgHHmdl0M5tiZv09rbZqR9ofgDGBTfhnzKyxd6VW2xH1ycxGAhucc3O9LvQQHPHnZGZ3m9k64HzgLx7WWl018X/vB5dR+ovbbzXZp6BSL6/RbGYNgaOBN8rtfo4+xNVEAEmUbjr2B143s3aBXwy1qob68wTwd0p/ef4d+Belf6C+ONI+mVkscAeluybqhBr6nHDO/RH4o5ndDowB/lpjRR6imupTYF1/BIqAl2umusNTk30KRvUyFCjdQtrlnMsov9DMwoGZgYfvUfpFWX5TNgXYELi/Hng7EALfmVkJpfOhZHtZ+EEccX+cc1vKve6/wPteFlwNR9qn9kBbYG7gDzsFmGVmA5xzmz2u/WBq4v9deS8Dk/AxFKihPpnZJcAvgZP8+GFVQU1/TsHF74MatXUD0il3IAmYCpwTuG9A74O8ruKBpOGB5b8D7grc7wSsI3DeR5D2p2W5NjcC44P9M6rQZg21fKDZo8+pY7k21wJvhkCfhgGLgOTa7ovX//cIwgPNvhdQSx/4q8AmoJDSX/iXU/or8kNgbuA/5F8O8tpMYAGwEhj7wxc/EAW8FHhuFnBikPfnRWA+MI/SX0Eta6s/XvWpQptaDwWPPqe3AsvnUTqXTesQ6NMKSn9UzQncantElRd9OiOwrnxgCzC5Nvt0JDed0SwiImXq8+gjERGpQKEgIiJlFAoiIlJGoSAiImUUCiIiUkahIEHPzPbU8vs9ZWbdamhdxYEZTxeY2cSqZgg1s0Qzu7om3lvkQDQkVYKeme1xzjWswfVFuB8naPNU+drN7HlgmXPu7krapwPvO+d61EZ9Uv9oS0FCkpklm9lbZjYjcDsmsHyAmX1rZrPNbKqZdQ4sv8TM3jOzz4BPzWyImX1hZm8G5vp/udxc+V/8MEe+me0JTFA318ymmVnzwPL2gcfzzewf1dya+ZYfJ/NraGafmtmswDpGBtrcC7QPbF08EGh7a6CP88zsbzX4zyj1kEJBQtUjwL+dc/2Bs4CnAsuXAMc55/pQOsPoPeVe0xc42zk3OPC4D3AD0A1oBxxzgPeJA6Y553oDXwJXlHv/R5xzPfnpTJoHFJhX5yRKzyYHyAPOcM71pfTaHf8KhNJtwErnXIZz7lYzOwXoCAwAMoB+ZnZ8Ve8ncjD1dUI8CX1DgW7lZrlsFJj9MgF43sw6UjojbGS513zsnCs/r/53zrn1AGY2h9L5cb6u8D4F/Dh54Ezg5MD9Qfx4XYdXgAcPUmeDwLpbA4uBjwPLDbgn8AVfEni++QFef0rgNjvwuCGlIfHlQd5PpFIKBQlVYcBRzrm88gvNbCzwuXPujMD++S/KPb23wjryy90v5sB/L4XuxwNzB2tTmf3OuYzAVN+TgWuARym9VkIy0M85V2hma4CYA7zegH865/7vEN9X5IC0+0hC1UeUziIKgJn9MA1yAj9Ob3yJh+8/jdLdVgCjqmrsnNtH6eU1bzazCErr3BoIhBOANoGmuUB8uZdOBi4LbAVhZq3NrFkN9UHqIYWChIJYM1tf7nYTpV+wmYGDr4soneoc4H7gn2Y2G2+3lG8AbjKzeUAHYHdVL3DOzaZ09tPRlF4rIdPM5gMXUXosBOfcduCbwBDWB5xzH1G6e+rbQNs3+WloiBwSDUkV8UBgd9B+55wzs1HAaOfcyKpeJ+I3HVMQ8UY/YGxgxNAufLy0qcih0JaCiIiU0TEFEREpo1AQEZEyCgURESmjUBARkTIKBRERKaNQEBGRMv8PqUdaE0MCJTwAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn = create_cnn(data, models.resnet18, metrics=accuracy)\n", "learn.lr_find()\n", "learn.recorder.plot()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

plot_losses[source]

\n", "\n", "> plot_losses(`last`:`int`=`None`)\n", "\n", "Plot training and validation losses. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.plot_losses)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that validation losses are only calculated once per epoch, whereas training losses are calculated after every batch." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Total time: 00:15\n", "epoch train_loss valid_loss accuracy\n", "1 0.110780 0.047797 0.981845 (00:07)\n", "2 0.046339 0.038317 0.987733 (00:07)\n", "\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VPW9//HXJ5N9X1kDJOyEPUQQUQG1iraFqmiheqtWpbX1+mttvaW9vdZ6r7feLhZ7621rrVtbpWqlta2WuuC+EfYdIgQIWxYgJGRPPr8/5hDHkI2QkzPJfJ6Pxzwy58x3znnPCeQzZ/t+RVUxxhhjAMK8DmCMMSZ4WFEwxhjTzIqCMcaYZlYUjDHGNLOiYIwxppkVBWOMMc2sKBhjjGlmRcEYY0wzKwrGGGOahXsd4Eylp6drVlaW1zGMMaZXWbNmTamqZnTUrtcVhaysLPLz872OYYwxvYqI7O1MOzt8ZIwxppkVBWOMMc2sKBhjjGnW684pGGP6jvr6eoqKiqipqfE6Sp8RHR1NZmYmERERXXq/FQVjjGeKiopISEggKysLEfE6Tq+nqpSVlVFUVER2dnaXlmGHj4wxnqmpqSEtLc0KQjcREdLS0s5qz8uKgjHGU1YQutfZbs+QKgrbD5/gg91lXscwxpigFVJFYd6yt/j8w+97HcMYEwTKysqYMmUKU6ZMYcCAAQwePLh5uq6urlPLuOmmm9ixY4fLSXtWSJ5orqprIDYyJD+6McaRlpbG+vXrAbjnnnuIj4/nW9/61ifaqCqqSlhY69+fH3vsMddz9rSQ2lM4ZXfJSa8jGGOCVEFBATk5OVx33XWMHz+eQ4cOsWTJEvLy8hg/fjz33ntvc9vzzz+f9evX09DQQHJyMkuXLmXy5MnMnDmT4uJiDz9F14XU12URUIWPSiqZMDjJ6zjGmAA/+OsWth480a3LzBmUyPc/O/6M37d9+3aefPJJ8vLyALj//vtJTU2loaGBuXPnsnDhQnJycj7xnvLycmbPns3999/PnXfeyaOPPsrSpUu75XP0pJDaU+ifEA3AR7anYIxpx4gRI5oLAsDTTz9Nbm4uubm5bNu2ja1bt572npiYGC6//HIApk2bRmFhYU/F7VYhtafQ0KQA7D9a5XESY0xLXflG75a4uLjm57t27eLBBx/kww8/JDk5meuvv77V+wAiIyObn/t8PhoaGnoka3cLqT2F6jr/L6nsZOeuLDDGmBMnTpCQkEBiYiKHDh1i5cqVXkdyVcjsKagqVfWNAByzomCM6aTc3FxycnIYO3Ysw4YNY9asWV5HcpWoqnsLF5kHPAj4gEdU9f4Wrw8FngCSnTZLVfXF9paZl5enXRlkp7qukXF3/wOAwckxvLP0ojNehjGme23bto1x48Z5HaPPaW27isgaVc1r4y3NXDt8JCI+4CHgciAHWCwiOS2afQ94RlWnAouA/3MrT5Vz6CgqPIyyk7VurcYYY3o1N88pTAcKVHW3qtYBy4EFLdookOg8TwIOuhWmqs5/6CgzJYaa+qbmImGMMeZjbhaFwcD+gOkiZ16ge4DrRaQIeBH419YWJCJLRCRfRPJLSkq6FObjohALQFmlnVcwxpiWvL76aDHwuKpmAlcAvxOR0zKp6sOqmqeqeRkZGV1a0ak9gyGpMQActZPNxhhzGjeLwgFgSMB0pjMv0M3AMwCq+h4QDaS7Eaa6xZ6CFQVjjDmdm0VhNTBKRLJFJBL/ieQXWrTZB1wMICLj8BeFrh0f6sCpw0dDnKKwcsth3LzyyhhjeiPXioKqNgC3AyuBbfivMtoiIveKyHyn2TeBW0VkA/A0cKO69Jf61D0KYwYksHBaJstX7+fDPUfdWJUxppeYO3fuaTejLVu2jNtuu63N98THxwNw8OBBFi5c2GqbOXPm0NGl88uWLaOq6uPeFa644gqOHz/e2eiucfWcgqq+qKqjVXWEqt7nzLtbVV9wnm9V1VmqOllVp6jqP93KUlXrP6cQG+nje5/2X79rRcGY0LZ48WKWL1/+iXnLly9n8eLFHb530KBBPPfcc11ed8ui8OKLL5KcnNzl5XUXr08095hTh49iI30kx0Yypn8CHxZaUTAmlC1cuJC///3vzYPqFBYWcvDgQaZOncrFF19Mbm4uEydO5C9/+ctp7y0sLGTChAkAVFdXs2jRIsaNG8eVV15JdXV1c7vbbrutudvt73//+wD8/Oc/5+DBg8ydO5e5c+cCkJWVRWlpKQAPPPAAEyZMYMKECSxbtqx5fePGjePWW29l/PjxXHrppZ9YT3cJmW4uoiN8DE6OISbSB8C5w1N54r29zPzhq3zninHMnzzI44TGhLiXlsLhTd27zAET4fL723w5NTWV6dOn89JLL7FgwQKWL1/OtddeS0xMDCtWrCAxMZHS0lLOPfdc5s+f3+b4x7/85S+JjY1l27ZtbNy4kdzc3ObX7rvvPlJTU2lsbOTiiy9m48aN3HHHHTzwwAOsWrWK9PRPXluzZs0aHnvsMT744ANUlRkzZjB79mxSUlLYtWsXTz/9NL/5zW+49tpr+dOf/sT111/fPdvKETJ7Cl+YMZR3ll5EVLi/KPzrxaOYnp3KofIaXtl6xON0xhivBB5COnXoSFX57ne/y6RJk7jkkks4cOAAR460/XfizTffbP7jPGnSJCZNmtT82jPPPENubi5Tp05ly5YtrXa7Hejtt9/myiuvJC4ujvj4eK666ireeustALKzs5kyZQrgXvfcIbOn0FJ6fBTPfHkmX/jN++w/Zl1pG+O5dr7Ru2nBggV84xvfYO3atVRVVTFt2jQef/xxSkpKWLNmDREREWRlZbXaXXZH9uzZw09+8hNWr15NSkoKN954Y5eWc0pUVFTzc5/P58rho5DZU2jLsLQ49pZZUTAmVMXHxzN37ly+9KUvNZ9gLi8vp1+/fkRERLBq1Sr27t3b7jIuvPBCnnrqKQA2b97Mxo0bAX+323FxcSQlJXHkyBFeeuml5vckJCRQUVFx2rIuuOAC/vznP1NVVcXJkydZsWIFF1xwQXd93A6F7J7CKVlpsRw9WUd5dT1JMRFexzHGeGDx4sVceeWVzYeRrrvuOj772c8yceJE8vLyGDt2bLvvv+2227jpppsYN24c48aNY9q0aQBMnjyZqVOnMnbsWIYMGfKJbreXLFnCvHnzGDRoEKtWrWqen5uby4033sj06dMBuOWWW5g6dWqPjeTmatfZbuhq19lt+cfmw3zl92v46+3nMzHTxm02pidZ19nuCMqus3uL7HT/sHuFZTZuszHGhHxRGJrq7/ZirxUFY4yxohAT6WNAYjSFdrLZGE/0tkPYwe5st2fIFwWAYWmxFJbanoIxPS06OpqysjIrDN1EVSkrKyM6OrrLywj5q48AstLieHV7sdcxjAk5mZmZFBUV0dXBs8zpoqOjyczM7PL7rSgAw9JjKa2spaKmnoRouyzVmJ4SERFBdna21zFMADt8BEwZ4u+Z8BerCthdUulxGmOM8Y4VBWB6VioAv35jN//y2w89TmOMMd5xtSiIyDwR2SEiBSKytJXXfyYi653HThHxZISJcF8YX5w5DIADx6s5XmVDdRpjQpNrRUFEfMBDwOVADrBYRHIC26jqN5zBdaYA/ws871aejnz/s+N56tYZALyx0056GWNCk5t7CtOBAlXdrap1wHJgQTvtF+MfktMTvjBhRnYaMRE+1u/3fkg8Y4zxgptFYTCwP2C6yJl3GhEZBmQDr7mYp0O+MGF0/3h2HD6950JjjAkFwXKieRHwnKo2tvaiiCwRkXwRyXf7euYxAxKsKBhjQpabReEAMCRgOtOZ15pFtHPoSFUfVtU8Vc3LyMjoxoinGzMgkbKTdZRU1Lq6HmOMCUZuFoXVwCgRyRaRSPx/+F9o2UhExgIpwHsuZum0sQMSAGxvwRgTklwrCqraANwOrAS2Ac+o6hYRuVdE5gc0XQQs1yDp/GSMUxS2Hz7hcRJjjOl5rnZzoaovAi+2mHd3i+l73MxwptLjo0iPj7Q9BWNMSAqWE81BZcyABLZbUTDGhCArCq0Y0z+RnUcqqG9s8jqKMcb0KCsKrZienUJtQxP5hce8jmKMMT3KikIrLhiVQWR4GK9sO+J1FGOM6VFWFFoRFxXOucPTeKeg1OsoxhjTo6wotGHsgAR2l56kqSkorpQ1xpgeYUWhDdnpcdQ1NHGwvNrrKMYY02OsKLQhOz0OgN0lJz1OYowxPceKQhuGO0VhT6kVBWNM6LCi0IaMhCjio8IpKLYxm40xocOKQhtEhMlDklhdeNTrKMYY02OsKLTjvBHpbD9cQVmldaNtjAkNVhTaMXNEGgDvfFTmcRJjjOkZVhTaMTkzmYyEKP664aDXUYwxpkdYUWiHL0xYMHkQr+8o5tjJOq/jGGOM66wodODK3MHUNyp/23TI6yjGGOM6V4uCiMwTkR0iUiAiS9toc62IbBWRLSLylJt5uiJnYCJj+iewYm2R11GMMcZ1rhUFEfEBDwGXAznAYhHJadFmFPAdYJaqjge+7laerhIR5k8ZxNp9xymuqPE6jjHGuMrNPYXpQIGq7lbVOmA5sKBFm1uBh1T1GICqFruYp8vmjMkA4K2d1muqMaZvc7MoDAb2B0wXOfMCjQZGi8g7IvK+iMxrbUEiskRE8kUkv6SkxKW4bcsZmEhGQhSv7+z5dRtjTE/y+kRzODAKmAMsBn4jIsktG6nqw6qap6p5GRkZPRzRfwhp1og03vuoDFXrStsY03e5WRQOAEMCpjOdeYGKgBdUtV5V9wA78ReJoDM9O43SylrrIM8Y06e5WRRWA6NEJFtEIoFFwAst2vwZ/14CIpKO/3DSbhczddmM4akAXPTTN7jlidXU1Dd6nMgYY7qfa0VBVRuA24GVwDbgGVXdIiL3ish8p9lKoExEtgKrgLtUNSj7lBieHscVEwcwpn8Cr2wrtrucjTF9kvS2Y+R5eXman5/v2fpVlcuWvUlMhI+/3H6+ZzmMMeZMiMgaVc3rqJ3XJ5p7HRFhwZTBbCgqt/sWjDF9jhWFLpg92n8F1Nu77L4FY0zfYkWhC3IGJpIeH8lr24PyXjtjjOkyKwpdEBYmXD5hIK9sO0JFTb3XcYwxpttYUeiiq3IHU1PfxEubD3sdxRhjuo0VhS6aMiSZ4elx/GmN9Z5qjOk7rCh0kYhwVe5gPthzlP1Hq7yOY4wx3cKKwln43FR//35/Xtey9w5jjOmdrCichcyUWM4dnsof8/dz5ITds2CM6f2sKJyl2+aMpKSiltufWut1FGOMOWtWFM7S7NEZ3HJBNuv2Hae6zjrJM8b0blYUusHUISk0NCmbD5Z7HcUYY86KFYVuMGWof1yg/MJjHicxxpizY0WhG6THRzE5M4nH391jdzgbY3o1Kwrd5AcLJlBSUcu3nt1gQ3YaY3otV4uCiMwTkR0iUiAiS1t5/UYRKRGR9c7jFjfzuGnKkGT+bd5YVm45wtp9dhjJGNM7uVYURMQHPARcDuQAi0Ukp5Wmf1TVKc7jEbfy9ITF5wwlTOD1HSVeRzHGmC5xc09hOlCgqrtVtQ5YDixwcX2eS4qNIHdoCm/stKJgjOmd3CwKg4H9AdNFzryWrhaRjSLynIgMcTFPj5g9OoONReWUVtZ6HcUYY86Y1yea/wpkqeok4GXgidYaicgSEckXkfySkuD+Fj5nTD8A3rS9BWNML+RmUTgABH7zz3TmNVPVMlU99ZX6EWBaawtS1YdVNU9V8zIyMlwJ213GD/KPyvby1iNeRzHGmDPmZlFYDYwSkWwRiQQWAS8ENhCRgQGT84FtLubpEWFhwtXTMvnHlsPsOFzhdRxjjDkjrhUFVW0AbgdW4v9j/4yqbhGRe0VkvtPsDhHZIiIbgDuAG93K05O+cuEIosN9PPleoddRjDHmjEhvu9EqLy9P8/PzvY7Rodt+v4Y1e4/x/ncuJixMvI5jjAlxIrJGVfM6atepPQURGSEiUc7zOSJyh4gkn23IvuzS8f0prqhldeFRr6MYY0yndfbw0Z+ARhEZCTyM/wTyU66l6gMuzRlAalwkv1hV4HUUY4zptM4WhSbnHMGVwP+q6l3AwA7eE9LiosJZcuFw3tpVSkFxpddxjDGmUzpbFOpFZDFwA/A3Z16EO5H6jiunDkYEXtx0yOsoxhjTKZ0tCjcBM4H7VHWPiGQDv3MvVt/QPzGac4al8tu39/DadrtvwRgT/DpVFFR1q6reoapPi0gKkKCq/+Nytj7hvisn0C8hiu88v4maehuu0xgT3Dp79dHrIpIoIqnAWuA3IvKAu9H6hlH9E7hn/niOnKjliXcLvY5jjDHt6uzhoyRVPQFcBTypqjOAS9yL1becNyKNT+X056cv72TzARvH2RgTvDpbFMKdLimu5eMTzaaTRIQfXjWR9LhIbn0yn+ITNV5HMsaYVnW2KNyLv7uKj1R1tYgMB3a5F6vvSY+P4pEbzqG8up5bn8y38wvGmKDU2RPNz6rqJFW9zZnerapXuxut78kZlMiDi6ay8UA5P/rHDq/jGGPMaTp7ojlTRFaISLHz+JOIZLodri/6VE5/FuZm8ocP9lJSYQPxGGOCS2cPHz2Gv9vrQc7jr8480wW3zRlBY5Py3y/2+p7CjTF9TGeLQoaqPqaqDc7jcSC4R7sJYsMz4vnq3JGsWHfABuMxxgSVzhaFMhG5XkR8zuN6oMzNYH3d7XNHMnZAAt9dsYnyqnqv4xhjDND5ovAl/JejHgYOAQvpIwPieCUyPIyfXDOZoyfr+Mk/7aSzMSY4dPbqo72qOl9VM1S1n6p+Dujw6iMRmSciO0SkQESWttPuahFREelwAIi+ZMLgJBZMGcSf1x2gtsEuUTXGeO9shuO8s70XRcQHPARcDuQAi0Ukp5V2CcD/Az44iyy91oIpg6mobWDV9hKvoxhjzFkVhY7GmJwOFDj3NNQBy4EFrbT7T+B/gJC8zXfWiDQGJ8fw4Ku7aGzqXUOjGmP6nrMpCh39BRsM7A+YLnLmNRORXGCIqv69vQWJyBIRyReR/JKSvvWNOtwXxnevGMe2Qye4/am1VNfZYSRjjHfaLQoiUiEiJ1p5VOC/X6HLRCQMeAD4ZkdtVfVhVc1T1byMjL53JewVEwfwvU+P4x9bDvPl36/xOo4xJoSFt/eiqiacxbIP4B/L+ZRMZ94pCcAE4HURARgAvCAi81U1/yzW2+uICLdcMJyGJuX+l7azqaiciZlJXscyxoSgszl81JHVwCgRyRaRSGAR/ruiAVDVclVNV9UsVc0C3gdCriAE+sKMocRG+vjOio0csZ5UjTEecK0oqGoDcDv+3lW3Ac+o6hYRuVdE5ru13t4sMTqCny+ayp6Skyz81bs8/eE+603VGNOjRLV3XfGSl5en+fl9e2di3b5j3PXcRgqKK5mcmcQfvzyT6Aif17GMMb2YiKxR1Q7vBXPz8JHpoqlDU3j5Gxfy4KIpbCgq54GXd3odyRgTIqwoBCkRYcGUwSyclsnj7xZSdKzK60jGmBBgRSHIfeNTowkTuPsvW6hraPI6jjGmj7OiEOQGJ8dw12VjeW17Mdf86l0aGq0wGGPcY0WhF/jSrCx+eNVENhSV8/SH+7yOY4zpw6wo9AIiwqJzhjBrZBr/+fdtrNt3zOtIxpg+yopCLyEiPPSFXJJjIuxqJGOMa6wo9CLJsZHcOCuLt3aV8sMXt9FkvaoaY7pZu30fmeBz43lZ7Dxcwa/f3A3A7ReNJCE6wuNUxpi+wopCLxMbGc7PPj+F+kbl12/u5sPCozx967l2x7MxplvY4aNeSET4xRem8tNrJrNu33Fy//NlHnlrtw3SY4w5a1YUeikR4eppmTx16wxmDk/jv/6+jat++S7bD5/wOpoxphezotDLnTcinUduyOPni6dSdLSKz/z8bb7z/CbrXdUY0yV2TqEPEBHmTx7E+SPT+dnLO/nd+3s5UVPPLxZPxRnAyBhjOsWKQh+SGhfJf35uAgOTo/nRP3YwOTOJW84fTliYFQZjTOe4evhIROaJyA4RKRCRpa28/hUR2SQi60XkbRHJcTNPqPjyhSOYPTqD/35xO99dsYneNmaGMcY7rhUFEfEBDwGXAznA4lb+6D+lqhNVdQrwI+ABt/KEEl+Y8NiN5/DVOSNYvno/Nz+RT1llrdexjDG9gJt7CtOBAlXdrap1wHJgQWADVQ28VCYOsK+03SQsTLjrsjHc89kc3i4o5Zpfv8fJ2gavYxljgpybRWEwsD9gusiZ9wki8jUR+Qj/nsIdLuYJOSLCjbOyefzGcygsPckPX9rmdSRjTJDz/JJUVX1IVUcA3wa+11obEVkiIvkikl9SUtKzAfuA80am88WZWTz1wT42FZV7HccYE8TcLAoHgCEB05nOvLYsBz7X2guq+rCq5qlqXkZGRjdGDB1fv2QU/ROjueXJ1ZRV1lJVZ4eSjDGnc7MorAZGiUi2iEQCi4AXAhuIyKiAyU8Du1zME9KSYyN55IY8jpyoZdp/vULO3SutC25jzGlcKwqq2gDcDqwEtgHPqOoWEblXROY7zW4XkS0ish64E7jBrTwGxg9K4oaZw0iKiWByZhI/f3UXr20/4nUsY0wQkd52DXteXp7m5+d7HaPXUlXqGptQhSv/710Ol1fz9zsuYFByjNfRjDEuEpE1qprXUTvPTzSbniUiRIX7iI7w8dAXplLX0MRtv19DeVW919GMMUHAikIIG54Rz7JFU9l2qIJrf/0eu45UeB3JGOMxKwoh7lM5/XnspnM4cLyay5a9yTsFpV5HMsZ4yIqCYdbIdN64aw5ZaXF885kNPL+2iMPlNV7HMsZ4wIqCASAtPopli6ZQUVPPnc9s4JIH3mDN3mNexzLG9DArCqbZpMxkVnxtFj9eOImUuAi+/sd1NpKbMSHGioL5hNH9E7gmbwgPLppKRU0D83/xDn/beNDrWMaYHmJFwbQqd2gKr945m8mZSdz+1DoeWlVApfWyakyfZ0XBtCktPorf3TyD+ZMH8eOVOzjvh69SUGyXrRrTl1lRMO2KjvCx7PNTePhfpuELE772h3VU1zV6HcsY4xIrCqZDYWHCpeMHsGzRVHYcqeAbf1xPTb0VBmP6IisKptNmj87gPz6Twz+2HObSn73J5gM2NoMxfY0VBXNGbj4/m9/fPIOGxiYWP/w++49WeR3JGNONrCiYM3b+qHT++OWZILDwV++y8JfvMvXef9qegzF9gBUF0yVDUmN54kvTGTcwker6Ro5V1XPzE6s5eLza62jGmLNg4ymYbrHt0Amu/uW71NQ38oP54/mXmVleRzLGBAiK8RREZJ6I7BCRAhFZ2srrd4rIVhHZKCKvisgwN/MY94wbmMiKr85i1sh07v3bVm5+fDV3PbuB2ga7SsmY3sS1oiAiPuAh4HIgB1gsIjktmq0D8lR1EvAc8CO38hj3jRmQwC8W53LJuP68XVDKs2uK+MYf19Pb9kaNCWVu7ilMBwpUdbeq1gHLgQWBDVR1laqeunzlfSDTxTymByTFRvDL66ex478u57tXjOXFTYf52lNrKa+2kd2M6Q3CXVz2YGB/wHQRMKOd9jcDL7X2gogsAZYADB06tLvyGZfdesFwGpvgp//cweYDb3P5xAE0NCo3zMxiaFqs1/GMMa1wsyh0mohcD+QBs1t7XVUfBh4G/4nmHoxmzoKIcNucEUwblsJX/7CGX7+xG4Bn8vdzzbQh3DQriyGpVhyMCSZuFoUDwJCA6Uxn3ieIyCXAvwOzVbXWxTzGI9OzU3nr3y6iqq6BipoGfrxyB797v5Dn1xXxldkjOHKihrsuG0NsZFB8RzEmpLl2SaqIhAM7gYvxF4PVwBdUdUtAm6n4TzDPU9VdnVmuXZLaN+wtO8ktT+Szq7gSgHOyUnj8punERVlhMMYNnb0k1dX7FETkCmAZ4AMeVdX7ROReIF9VXxCRV4CJwCHnLftUdX57y7Si0HfU1DfyTkEp5dX1fPPZDajConOGMHVoMnPH9qNfQrTXEY3pM4KiKLjBikLf9NauElasO8CKdQdQhcTocO6/ehKXTxiAiHgdz5hez4qC6ZWq6xrZXVrJd5/fxIaicpJjIxjVL56vXzKaWSPTvY5nTK9lRcH0avWNTTy/toj1+8t5Y0cxB8trmDk8jU/l9Gd6dioTBid5HdGYXsWKgukzauob+cMH+/jtW7s5WF5DeJhwz/zxXDdjqB1aMqaTrCiYPkdVKTpWzd1/2cyqHSUMSY3hc1MGc/tFI4kK93kdz5igZkXB9FmNTcpza/bzj82HWbWjhPiocHIGJfLVOSOYOSLNCoQxrbCiYELCGztLeHXbEV7bXkzRsWoGJEbzzUtH89nJg4iOsOJgzClWFExIqapr4JVtxSx7eSe7S08yflAi//GZHCZlJtmd0sZgRcGEqKYm5eVtR1j6p40cq6onOTaCeeMHMGVIMldPyyTCZ4MNmtBkRcGEtPLqet77qJTn1x5gdeFRjlXVEx0RRr+EaG4+P5t/OXcYYWF25ZIJHVYUjHGoKiu3HOGNnSV8VFLJh3uOMjg5hqunZXLdjKH0T7TuNEzfZ0XBmFY0NSl/23SIZ/P383ZBKTERPj5/zhCumzGMkf3ivY5njGusKBjTgb1lJ3ng5Z28uOkQ9Y3KJeP6ERsZzmXjB3DZ+P6E2/kH04dYUTCmk0oqannqg308tKoAgLrGJtLiIrnu3GHcNnsEMZF2aavp/awoGHOGSipqiYvy8ebOEp5fe4B/bj1CcmwEd1w0igtHpzM8Pd5OTptey4qCMWcpv/AoP165gw/2HAVgcmYSn508iFkj00mPj6K0spboCB/Z6XEeJzWmY0FRFERkHvAg/kF2HlHV+1u8fiH+QXgmAYtU9bmOlmlFwfQkVWXnkUo+3FPGr97YzYHj1QCIwKn/Ov0SohgzIIFvXTqGyUOSPUxrTNs8Lwoi4sM/HOengCL8w3EuVtWtAW2ygETgW8ALVhRMsNt++ARbDpzgo5JKBqfEUFpRx/bDJ8jfe4zSylrOGZbKlKHJfHriQCsQJqh0tii4ef//dKBAVXc7gZYDC4DmoqCqhc5rTS7mMKbbjB2QyNisNRkbAAARaElEQVQBiafNP1FTz6Nv7+G17cU8/m4hD7+5mwtHZzB+UCLnj0xnYmYSidERHiQ25sy4WRQGA/sDpouAGS6uzxjPJEZH8PVLRvP1S0ZTWdvAI2/tZsW6A7xTUMovX/+I2EgfV04dTGJMBCdrG4j0hTFnTD+mDE0mLtJHZW0D/9h8mO2HK8gZmMi8CQOIjvARJtiYEaZH9YqewkRkCbAEYOjQoR6nMaZ98VHhzQXi4PFqdhVX8sL6gzy3pojGJiUuKpzq+kYeeXsPABkJUfhEOHyihgifUN+ofO/Pm/GFCf0So/j6JaOZMyaDkopaYiJ89E+MxmdXQRmXuFkUDgBDAqYznXlnTFUfBh4G/zmFs49mTM8YlBzDoOQYZo/O4L4rJ1Df2ERCdAQ19Y28tr2YwrKTbDtUwZHyGh74/GTOzU5j7b5jrFh3gOr6RjYfKOeOp9d9Ypm+MGFAYjSDU2IYkhJLdnosSbGRzB6VwZDUGNuzMGfFzaKwGhglItn4i8Ei4Asurs+YoBYd4Wse4yE6wscVEwe22i4vK5W8rFTAP6DQ6zuK2XGkgoFJ0VTVNXLweDUHjlVz4Hg1r+8o5k9r65rfmx4fxYTBiQxMiuHc4anMHJ5GP+vbyZwBty9JvQL/Jac+4FFVvU9E7gXyVfUFETkHWAGkADXAYVUd394y7eojYz7W1KRU1Teyt+wkG4vKeX93GQXFlew7WkVFTQMAKbERDEuL46Kx/fj8OUOsA8AQ5fklqW6xomBMxxqblK0HT/De7lIKy6rYevAEG4qOowqj+8dzbd4Qrs7NJCUusrn9jsMV9E+MIi0+yuP0xg1WFIwxn1BYepKXNh/mn1sPs27fccIExgxI5LOTB/LeR2W8tauUCJ9w3oh0BiZFEx3hIystlpkj0hmWFmvDm/ZyVhSMMW3afvgEL206zJu7Sli37ziRvjC+MmcElTUNvLe7jKMna6mqbaSitqH5PcMz4rg0ZwBpcZFsPXSCvKwUIn1hjO6fwKDkGDISbA8jmFlRMMZ0ysHj1STGRBAfdfp1JwXFlWw5WE5haRX5e4/y7kdlNDYp8VHhVAYUDIDs9DjmjunH8Iw4YiN9lFbWMnVoCmMHJJBgN+55LhjuaDbG9AKDkmPafG1kv/hPDD5UXlVPQUkFkzOT2Xu0ioZGZVdxBYeO1/DuR6U8+V4hDU2nf9EcmhrL9OxURmTEMyjZf6I7ITqcCYOTSI6JJDLcxq4IFranYIzpNnUNTRwur6GytoGBSdF8sKeMPaVVrC48yuYD5RRX1J72nuiIMM4dnkZaXBQ1DY2M7Z9AfWMT5dX1RPjCOG9kGnlZqTQ2KsmxEXYfRhfZ4SNjTNA5UVNP8YlawsQ/fsXWQyfYW1bFOwWlnKipR5XmwpEYHU5NQxN1DR93jZYUE+Hfe8nw78Ecr66joUkZnBzDwKQYRvePZ2hqbJuFo66hibd2lVDf2ER6vL9328jwMKrrGlGFmEgfG/Yf54M9R3l9RzEjMuIZlhZLXUMTr24vprq+kcyUWIamxjA0NZZ9R6s4XlXP1KEpVNc1MCkzmXEDEzs8v9LQ2MTRk3VkJET1WJGzomBMT6qthBMHICwcwnzOz/DWp8UHYXa4pDWqSl1jE5G+MESEmvpG1uw9xvr9/pPhhWUnKSiu5KOSSkor6wgPE8JEqGv8uHBkJESRNyyFipoGFGXa0BRyh6UwdWgKv317Dz9/dVensozuH8+RE7WUV9cDMHVoMgMSoyk6Vs2+o1WUV9cT6QsjOTbitD2giYOTGNkvniGpsWSmxFBaWcuR8hpO1jWSGhfJK9uOsLvkJOnxkQxPj6e2sYmstFgEqKlvYtzARKYMTWbS4KRu2zuyomBMTyp4FX5/1Rm8QdovHJ+Y9rXyejvvEV/nilNYuL84ddimo+W2Nq/lclsrjmH+gSm66NjJOsJ9QlxkOKUnazl4vIYtB8vJLzxG/t6jhIeFERvpY9uhEzSpf1VhIswdk8GdnxrDwePVFJRU0tikzZ0PVtc3khwTyacnDSQpJgJVpbahiSZVYiM/eQq2vMpfLBJjwjl8oobocB/bD1eQX+g/Ib/vaBWHyqs5dYolOiKMSF8YJ+saifAJX5k9gqJj1RSWniQqIozC0irCwiAiLIw9ZSebx+sQ8Y/ZsXj6UL5+yeguby8rCsb0pIrDUPg2NDWCNkJTg/MIfN5yXsuf7bxHmzpoc4bL1Eavt5jfJ4pNZwpZywLZ8V5ZPWGUVTVypLKBsqpGpg3PICk2ppX3nNlyO5O1Tn2UVNYTFRlJcnwM4ougXoUm8REbHe0s6/TCWFFTz8aicrYcLKeipoH9R6uYlJnMl87P7vKmtquPjOlJCQNg4kKvU3SeaicKzal5nSxy2lZBam1ey3W3UcjaLbCN0FgP9dVtZPXPi2hqYIDzoKkRNgW0UXeHconEP4ZAoNNuAZSw04pNQlg4s5xHc7EpDYfkb8OEq13NbEXBmFAk8vG3XUL4prOmphaFp2XxOYM9rw73+DrbJnCZLebFpLi+SawoGGNCV1gYEAY+u7nuFLsEwhhjTDMrCsYYY5pZUTDGGNPMioIxxphmrhYFEZknIjtEpEBElrbyepSI/NF5/QMRyXIzjzHGmPa5VhRExAc8BFwO5ACLRSSnRbObgWOqOhL4GfA/buUxxhjTMTf3FKYDBaq6W1XrgOXAghZtFgBPOM+fAy4W6wLRGGM842ZRGAzsD5gu4vSb+5rbqGoDUA6ktVyQiCwRkXwRyS8pKXEprjHGmF5x85qqPgw8DCAiJSKyt4uLSgdKuy1Y9wvmfJatayxb1wVzvt6YbVhn3uxmUTgADAmYznTmtdamSETCgSSgrL2FqmpGVwOJSH5nOoTySjDns2xdY9m6Lpjz9eVsbh4+Wg2MEpFsEYkEFgEvtGjzAnCD83wh8Jr2tm5bjTGmD3FtT0FVG0TkdmAl/o4BH1XVLSJyL5Cvqi8AvwV+JyIFwFH8hcMYY4xHXD2noKovAi+2mHd3wPMa4Bo3M7TwcA+uqyuCOZ9l6xrL1nXBnK/PZut1g+wYY4xxj3VzYYwxplnIFIWOutzwIE+hiGwSkfUiku/MSxWRl0Vkl/PT/RE1/Ot9VESKRWRzwLxWs4jfz53tuFFEcj3Kd4+IHHC233oRuSLgte84+XaIyGUu5hoiIqtEZKuIbBGR/+fMD4pt106+YNh20SLyoYhscLL9wJmf7XR5U+B0gRPpzO+xLnHayfa4iOwJ2G5TnPle/J/wicg6EfmbM919201V+/wD/4nuj4Dh+EfI2wDkeJypEEhvMe9HwFLn+VLgf3ooy4VALrC5oyzAFcBLgADnAh94lO8e4FuttM1xfr9RQLbze/e5lGsgkOs8TwB2OusPim3XTr5g2HYCxDvPI4APnG3yDLDImf8r4Dbn+VeBXznPFwF/dHG7tZXtcWBhK+29+D9xJ/AU8Ddnutu2W6jsKXSmy41gENjtxxPA53pipar6Jv6rvzqTZQHwpPq9DySLyEAP8rVlAbBcVWtVdQ9QgP/370auQ6q61nleAWzDf5d+UGy7dvK1pSe3napqpTMZ4TwUuAh/lzdw+rbrkS5x2snWlh79vYpIJvBp4BFnWujG7RYqRaEzXW70NAX+KSJrRGSJM6+/qh5ynh8G+nsTrd0swbQtb3d21x8NONTmST5nt3wq/m+VQbftWuSDINh2ziGQ9UAx8DL+PZPj6u/ypuX6O9UljlvZVPXUdrvP2W4/E5FTg1v39O91GfBvQJMznUY3brdQKQrB6HxVzcXfi+zXROTCwBfVv78XFJeGBVOWAL8ERgBTgEPAT70KIiLxwJ+Ar6vqicDXgmHbtZIvKLadqjaq6hT8vR1MB8Z6kaM1LbOJyATgO/gzngOkAt/u6Vwi8hmgWFXXuLWOUCkKnelyo0ep6gHnZzGwAv9/iiOndjudn8XeJWwzS1BsS1U94vzHbQJ+w8eHOXo0n4hE4P+D+wdVfd6ZHTTbrrV8wbLtTlHV48AqYCb+Qy+n7p8KXH9zNulklzjdnG2eczhOVbUWeAxvttssYL6IFOI/DH4R8CDduN1CpSh0psuNHiMicSKScOo5cCmwmU92+3ED8BdvEkI7WV4AvuhccXEuUB5wqKTHtDhmeyX+7Xcq3yLnqotsYBTwoUsZBP9d+dtU9YGAl4Ji27WVL0i2XYaIJDvPY4BP4T/nsQp/lzdw+rbrkS5x2si2PaDQC/5j9oHbrUd+r6r6HVXNVNUs/H/HXlPV6+jO7eb2WfJgeeC/QmAn/uOW/+5xluH4r/LYAGw5lQf/sb5XgV3AK0BqD+V5Gv9hhHr8xyNvbisL/issHnK24yYgz6N8v3PWv9H5hz8woP2/O/l2AJe7mOt8/IeGNgLrnccVwbLt2skXDNtuErDOybAZuDvg/8aH+E9yPwtEOfOjnekC5/XhHmR7zdlum4Hf8/EVSj3+f8JZ7xw+vvqo27ab3dFsjDGmWagcPjLGGNMJVhSMMcY0s6JgjDGmmRUFY4wxzawoGGOMaWZFwQQdEWl0eqHcICJrReS8Dtoni8hXO7Hc10UkKMfV9YrT8+fCjluaUGFFwQSjalWdoqqT8Xct8MMO2ifj7w0yKAXcaWpM0LOiYIJdInAM/H34iMirzt7DJhE51dPt/cAIZ+/ix07bbzttNojI/QHLu0b8feXvFJELnLY+EfmxiKx2Ojv7sjN/oIi86Sx386n2gcQ/LsaPnHV9KCIjnfmPi8ivROQD4EfiH2Phz87y3xeRSQGf6THn/RtF5Gpn/qUi8p7zWZ91+i9CRO4X//gIG0XkJ868a5x8G0TkzQ4+k4jIL8Q/XsIrQL/u/GWZ3s++wZhgFCP+Hiqj8Y8JcJEzvwa4UlVPiEg68L6IvIB/zIIJ6u/ADBG5HH+XwTNUtUpEUgOWHa6q08U/sMz3gUvw3yFdrqrniL/ny3dE5J/AVcBKVb1PRHxAbBt5y1V1ooh8EX8Plp9x5mcC56lqo4j8L7BOVT8nIhcBT+LvkO4/Tr3fyZ7ifLbvAZeo6kkR+TZwp4g8hL9birGqqqe6YgDuBi5T1QMB89r6TFOBMfjHTugPbAUe7dRvxYQEKwomGFUH/IGfCTwp/l4qBfhv8fco24S/W+DWuhe/BHhMVasAVDVwLIZTndatAbKc55cCkwKOrSfh7/dnNfCo+DuV+7Oqrm8j79MBP38WMP9ZVW10np8PXO3keU1E0kQk0cm66NQbVPWY+HvCzMH/hxz8A0O9h7/b4xrgt+IfcetvztveAR4XkWcCPl9bn+lC4Gkn10ERea2Nz2RClBUFE9RU9T3nm3MG/n57MoBpqlov/p4io89wkbXOz0Y+/vcvwL+q6sqWjZ0C9Gn8f3QfUNUnW4vZxvOTZ5itebX4+/Bf3Eqe6cDF+Ds3ux24SFW/IiIznJxrRGRaW59JAobeNKY1dk7BBDURGYt/ONUy/N92i52CMBcY5jSrwD/c5CkvAzeJSKyzjMDDR61ZCdzm7BEgIqPF35PtMOCIqv4G/yhXbY29+/mAn++10eYt4Dpn+XOAUvWPbfAy8LWAz5sCvA/MCjg/EedkigeSVPVF4BvAZOf1Ear6gareDZTg7yq51c8EvAl83jnnMBCY28G2MSHG9hRMMDp1TgH833hvcI7L/wH4q4hsAvKB7QCqWiYi74jIZuAlVb1L/IOq54tIHfAi8N121vcI/kNJa8V/vKYEf9fIc4C7RKQeqAS+2Mb7U0RkI/69kNO+3TvuwX8oaiNQxcfdGf8X8JCTvRH4gao+LyI3Ak/Lx6N7fQ9/8fuLiEQ72+VO57Ufi8goZ96r+Hvf3djGZ1qB/xzNVmAfbRcxE6Ksl1RjzoJzCCtPVUu9zmJMd7DDR8YYY5rZnoIxxphmtqdgjDGmmRUFY4wxzawoGGOMaWZFwRhjTDMrCsYYY5pZUTDGGNPs/wN4JQHK58a64QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit_one_cycle(2)\n", "learn.recorder.plot_losses()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

plot_lr[source]

\n", "\n", "> plot_lr(`show_moms`=`False`)\n", "\n", "Plot learning rate, `show_moms` to include momentum. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.plot_lr)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAAD8CAYAAABJhAMxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8VNX9//HXJ/tKgCQQyMoOYYcIBNzqClYFURRUwKW17dettda6Vauttdqq3dRWK7KoICoqKorWHQhLWENYA9kJISxZyJ6Z8/sjo7+UoiRkOTOTz/PxyKOTO/fevG8xk8/cOedzxBiDUkoppZRSqm352A6glFJKKaWUN9JCWymllFJKqXaghbZSSimllFLtQAttpZRSSiml2oEW2koppZRSSrUDLbSVUkoppZRqB1poK6WUUkop1Q600FZKKaWUUqodaKGtlFJKKaVUO/CzHaAtREVFmaSkJNsxlFKqxTZu3HjYGBNtO0dH0tdspZQna8nrtlcU2klJSaSnp9uOoZRSLSYiubYzdDR9zVZKebKWvG7r0BGllFJKKaXagRbaSimllFJKtQMttJVSSimllGoHWmgrpZRSSinVDrTQVkoppZRSqh00q9AWkckisltEskTk3pM8Hygir7ueXyciSU2eu8+1fbeIXOzaFiQi60Vkq4hkisgjTfbv4zpHluucAa2/TKWUUkoppTrWKQttEfEFngWmAMnALBFJPmG3m4Fjxpj+wDPAE65jk4GZwFBgMvCc63y1wHnGmJHAKGCyiExwnesJ4BnXuY65zq2UUkoppZRHaU4f7XFAljFmP4CILAGmAjua7DMV+K3r8ZvAP0REXNuXGGNqgWwRyQLGGWPSgOOu/f1dX8Z1zHnAta7nFrjO+/xpXZ3qEEs35HOoooYgf1+CA3zpER5Er4ggencNpnuofiChlGq95VsPUFXbwMxxCbajKKU8TFVdA9mHKykqreFYVR2lVfVU1jUwqX8UZyR1b9ef3ZxCOxbIb/J9ATD+u/YxxjSISBkQ6dq+9oRjY+HbO+Ubgf7As8aYdSISBZQaYxpO3P9EInILcAtAQoK+8NqyOe8Y97y17TufjwoLZEivcIbFRjChbyRnJHUjJMAr1klSSnWg97YeYFPuMa4YE0ugn6/tOEopN1VT72BT3jE255WyOe8Y2wvLOVhec9J9A/183aLQbhfGGAcwSkS6Am+LyDDgYAuOfwF4ASAlJcW0T0p1Ki+vziE80I+vf/0DfHyEqloHhypqKCqrIf9oFbsOVrCzqJwXv9rP81/sw99XGJvYjSnDejF5WAw9uwTZvgSllAeYk5rIJzuKWZFRxBWj42zHUUq5kdKqOj7afpBPdx1i1d7DVNc7AOgbHUpqv0j6RYfSJyqM2G7BdA8JoGuoP6EBfvj6SLtna06hXQjEN/k+zrXtZPsUiIgfEAEcac6xxphSEfmcxjHcTwFdRcTPdVf7ZD9LuYmismpWZBRxw8QkuoY0DhHpEuRPTEQQI074O1hV10B6zjFW7zvM57sO8fDyTH77Xibj+3Rn1rgEJg+L0btUSqnvNKlfFH2jQ1mwJlcLbaUUDqfh812HeHNjAZ/tOkSdw0ls12CuHBvLuQN7kJLU7dvaxKbmFNobgAEi0ofGoncm/38M9TeWA3OBNOAq4DNjjBGR5cBrIvI00BsYAKwXkWig3lVkBwMXAk+4jvncdY4lrnO+2+qrVO1iUVouTmOYOzHplPuGBPhx9sBozh4YzX1ThpB1qIIPth3krU0F3LlkC91C/Jk1LoGbzuxDVFhg+4dXSnkUHx9h9oREHnlvBxkFZQyPi7AdSSllQXlNPUs35LMgLYf8o9VEhQVw/YREpo+JZWjvLjRO93Mfpyy0XWOubwNWAr7APGNMpog8CqQbY5YDLwGLXJMdj9JYjOPabymNEycbgFuNMQ4R6QUscI3T9gGWGmPed/3IXwNLROT3wGbXuZWbqa5z8Nr6PC5KjiG+e0iLj+/fI5w7Lwjn9vP6s3rfYV5dm8fzX+7jpVXZXHNGPD87tx+9IoLbIblSylNdOTaOP63czcK0HP40Y6TtOEqpDlRRU8/Lq3N48ev9VNQ0MC6pO/dPGcKFyT3x83XfZWGaNUbbGLMCWHHCtoeaPK4BZnzHsY8Bj52wbRsw+jv2309jpxPlxt7ZUkhpVT03ndmnVefx8RHOGhDNWQOi2VdynH99uY/F6/N4fUM+N5/Zh5+d24/wIP82Sq2U8mRdgvy5YnQsb24s4P5LhtBNuxop5fVq6h0sWJPDP7/cx7Gqei5K7snt5w3wmE+13PctgHJbxhjmrcpmaO8unJHUrc3O2y86jCevGslnvzyXKcNieO6LfZzzpy9YmJaDw6nzXZVSMCc1idoGJ0vT80+9s1LKo32yo5gLnv6Sxz/cxfC4rrx76yRemJPiMUU2aKGtTsOqrMPsPXScmyb1aZexUPHdQ/jLzNG8d9uZDOoZzkPvZjL9udXsOFDe5j9LKeVZBsWEM75PdxatzdU34Ep5qbwjVdw8fwM/XphOsL8vr/14PAtvGsfI+K62o7WYFtqqxeatyiYqLJBLR/Zq158zPC6C1348nr/NGk1haTWX/WMVT3y0ixpX2x6lVOc0JzWJgmPVfLH7kO0oSqk25HQaXl6dzYXPfMna/Ud44JIhrLjzLCb2i7Id7bTpyiGqRfaVHOfz3SX84oKBHdKOT0S4fGRvzh4QxWMf7OT5L/bxyY5i/jZzNMm9u7T7z1dKuZ+LhvakZ5dAFqblcv6QnrbjKKXaQGFpNb96Yytr9h3hB4OieXz6CGIiPH+tDb2jrVpk/uocAnx9uG5Cx67G2TUkgD/NGMnCm8ZRXl3PtGdXM29VNsboR8dKdTb+vj5cOy6RL/eUkH240nYcpVQrvbulkMnPfMXW/FL+OH048244wyuKbNBCW7VAWVU9b24sYOqo3tZ6XZ89MJoP7zyLswdG8ej7O7hx/gaOVtZZyaKUsmfWuHj8fIRX1ubajqKUOk21DQ4eeDuDO5dsYVBMOB/eeTYzxyW4XS/s1tBCWzXbkg15VNc7uHFS61r6tVZkWCAvzknhd1OHsmbfES77+yoyD5RZzaSU6lg9ugQxZXgvlqbnU1XXYDuOUqqF8o9WMeOfaby6Lo+fnN2XJbdMICGy5etyuDsttFWzNDicLFiTQ2rfSLcYGy0izE5N4o2fpOJwGq58fg3vbim0HUsp1YHmpCZSUdPAu1sO2I6ilGqBNfsOc9k/VpF9uJJ/zR7LfZcMcetFZ1rDO69KtbmPdxRzoKyGGycl2Y7yX0bGd+W9289kRGxX7lyyhcdX7MSpLb+U6hRSErsxOCacBWtydL6GUh7i9Q15zHlpPdFhgbx325lcPDTGdqR2pYW2apZ5q7JJ6B7iljP8o8MDefXH45k9IZF/fbWf2xdv1haASnUCIsLciUnsOlhBeu4x23GUUt/D6TQ8vmInv34rg9R+kbz1fxNJigq1HavdaaGtTmlrfinpuce4YWISvj7uOUHB39eHR6cO5YFLhvBBRhGzX1pHaZVOklSquURksojsFpEsEbn3JM8nisinIrJNRL4QkbgTnu8iIgUi8o+OSw1TR/UmPMiPhWk6KVIpd1VT7+D/Xt3Ev77az+wJibx8wxl0CfK3HatDaKGtTunl1dmEBfoxIyXu1DtbJCL8+Oy+/H3WaLbmlzH9+TUUHKuyHUsptycivsCzwBQgGZglIskn7PZnYKExZgTwKPD4Cc//DviqvbOeKCTAj6tT4vkwo4hD5TUd/eOVUqdwvLaBm+Zv4KPMgzz4wyE8OnWo147HPpnOc6XqtBSX1/D+tiKuTokn3EPefV42sjeLbh7H4Yparv5nmvbZVerUxgFZxpj9xpg6YAkw9YR9koHPXI8/b/q8iIwFegIfd0DW/3H9hEQanIbF6/Nt/Hil1Hc4cryWa19cy7rsozxzzUh+dFZfr2rd1xxaaKvvtSgtF4cx3DAxyXaUFhnfN5LFt0ygpsHJjH+msftghe1ISrmzWKBplVrg2tbUVmC66/EVQLiIRIqID/AUcHe7p/wOfaJCOXtgNK+uy6Xe4bQVQynVRFFZNVf/q/Hv7wuzx3LFaPf+VLy9aKGtvlNNvYNX1+Vy4ZCeHtnbcmjvCJb+ZAI+Ate8kEZGgfbaVqoV7gbOEZHNwDlAIeAA/g9YYYwp+L6DReQWEUkXkfSSkpI2Dzc3NZFDFbV8nFnc5udWSrXMwbIaZr2wluLyWhbeNM4tGyl0FC201Xd6Z3Mhx6rquelMuwvUtEb/HuG88dNUQgP8uPbfa7XYVurkCoH4Jt/HubZ9yxhzwBgz3RgzGnjAta0USAVuE5EcGsdxzxGRP574A4wxLxhjUowxKdHR0W1+AecO6kFct2AWpuW0+bmVUs13sKyGWS+u5fDxOhbePI7xfSNtR7JKC211UsYYXl6dQ3KvLozv0912nFZJjAxl6U9T6RLkz+x569hZVG47klLuZgMwQET6iEgAMBNY3nQHEYlyDRMBuA+YB2CMuc4Yk2CMSaLxrvdCY8z/dC1pb74+wuwJiazLPsqug/o7rpQNxeWNRfah8hoW3DSOMQndbEeyTgttdVJr9h1hd3EFN05K8oqJC7Fdg3ntx+MJ8vPl+n+vI+uQjtlW6hvGmAbgNmAlsBNYaozJFJFHReRy127nArtFZA+NEx8fsxL2e1ydEk+gnw+LtNWfUh2uuLxxuMih8hoW3jyOsYlaZIMW2uo7zFuVTVRYAJeN7G07SptJjAzl1R+PR0S49sV15Gg3EqW+ZYxZYYwZaIzpZ4x5zLXtIWPMctfjN40xA1z7/MgYU3uSc8w3xtzW0dm/0S00gMtH9ubtzYWU19TbiqFUp1NaVcf1/15HsetO9thEz/4kvC1poa3+R/bhSj7ddYjrxicS5O9rO06b6hcdxqs/Gk+9w8m1L67lQGm17UhKqTY0JzWJqjoHb2383rmZSqk2UlXXwI3zN5B7pIoX56aQkqRFdlNaaKv/MX91NgG+Plw3IcF2lHYxKCacRTePp6Kmgbnz1usKkkp5keFxEYyK78qitFycTmM7jlJera7ByU9f2cTW/FL+Nms0E/tF2Y7kdrTQVv+lrLqeNzYWcNnI3vQID7Idp90Mi43ghTkp5B6p4scL06mpd9iOpJRqI3MnJrL/cCWr9x22HUUpr+V0Gu5+Yytf7SnhD1cMZ/KwGNuR3JIW2uq/LN2QT1WdgxsnJdmO0u5S+0Xy9DUjSc89xh2LN+PQu19KeYVLhvciMjSAhTopUql2YYzh0fd3sHzrAe6ZPIiZ47zzE/C2oIW2+laDw8n8NTmM79OdYbERtuN0iEtH9OahS5P5eEcxDy/fjjFabCvl6QL9fJk5Lp5PdxZTcKzKdhylvM5Lq7KZvyaHm8/sw8/O6Wc7jltrVqEtIpNFZLeIZInI//RHFZFAEXnd9fw6EUlq8tx9ru27ReRi17Z4EflcRHaISKaI3Nlk/9+KSKGIbHF9XdL6y1TN8Z+dxRSWVnv0AjWn48ZJffjpOf14ZW0ez32xz3YcpVQbuHZ8IgCvrsuznEQp77Iy8yCPrdjJ5KExPHDJEK9oAdyeTlloi4gv8CwwBUgGZolI8gm73QwcM8b0B54BnnAdm0zjwgdDgcnAc67zNQC/NMYkAxOAW0845zPGmFGurxWtukLVbPNW5RDfPZgLOuFSqb+ePIipo3rzp5W7+Wh7ke04SqlWiu3a+Fq2ZH2ezsFQqo1kFJTx8yVbGBEbwTPXjMLHR4vsU2nOHe1xQJYxZr8xpg5YAkw9YZ+pwALX4zeB86XxLc5UYIkxptYYkw1kAeOMMUXGmE0AxpgKGhdIiG395ajTlVFQxvqco8xNTcK3E/7iiAhPXDmC0Qld+cXrW9leqEu1K+Xp5k5M4lhVPR9s0zfPSrVWYWk1Ny3YQPfQAF6cm0JwgHe1/20vzSm0Y4H8Jt8X8L9F8bf7uFYYKwMim3Osa5jJaGBdk823icg2EZknIiddWkhEbhGRdBFJLykpacZlqO/z8upsQgN8ufqMeNtRrAny9+WF2Sl0C/Hn5gUbKC6vsR1JKdUKE/tF0i86lIVrdVKkUq1xvLaBm+dvoKbOwbwbzvDqrmRtzepkSBEJA94Cfm6MKXdtfh7oB4wCioCnTnasMeYFY0yKMSYlOjq6Q/J6q0PlNby37QAzUuLpEuRvO45V0eGBvHTDGVTUNPCjBelU1+lHzkp5KhFhTmoSW/NL2ZpfajuOUh7J6TTc9foW9hRX8Ox1YxgUE247kkdpTqFdCDS9zRnn2nbSfUTED4gAjnzfsSLiT2OR/aoxZtk3Oxhjio0xDmOME3iRxqErqh29sjaXBqfhholJtqO4hSG9uvC3maPZfqCMu9/Yqp1IlPJg08fEEhrgq63+lDpN//g8i493FHP/JUM4e6De2Gyp5hTaG4ABItJHRAJonNy4/IR9lgNzXY+vAj4zjdXJcmCmqytJH2AAsN41fvslYKcx5ummJxKRXk2+vQLY3tKLUs1XU+/glXV5nD+4J0lRobbjuI0Lknvy68mD+SCjiBe+2m87jlLqNIUH+XPFmFje23aAo5W6CqxSLfGfHcU88589XDE6lps7WUeytnLKQts15vo2YCWNkxaXGmMyReRREbnctdtLQKSIZAF3Afe6js0ElgI7gI+AW40xDmASMBs47yRt/J4UkQwR2Qb8APhFW12s+l/LtzT+8bnpzCTbUdzOT87uyyXDY3jio12sydIV5pTyVHNSk6hrcPL6hvxT76yUAmBfyXF+8foWhvbuwuPTh2sbv9Mk3vCxeEpKiklPT7cdw+MYY5jy168B+PDOs/SX6CSO1zZwxbOrOVJZx3u3n0ls12DbkZSXEZGNxpgU2zk6ko3X7JkvpJF/tJqv7vlBp+yspFRLlNfUM+3Z1ZRV1bNc//b9j5a8buvKkJ1Y2v4j7DpYwU2T+miR/R3CAv345+yx1Dc4+dkrG7Ufr1Ieam5qEoWl1Xy265DtKEq5tcbJj1vJPVLFs9eN0SK7lbTQ7sTmrcqhe2gAl4/qbTuKW+sXHcZTV49kW0EZD72ry7Qr5YkuTO5JTJcgFqbl2I6ilFt7/st9/GdnMb/54RAm9I20HcfjaaHdSeUcruTTXcVcPz6BIH9tOn8qFw2N4fbz+rM0vYDF63Wcp1Kexs/Xh2vHJ/D13sPsLzluO45Sbmnt/iM89fFuLh/Zm7naiaxNaKHdSc1fk4Ofj3D9hETbUTzGzy8YyNkDo/nt8kwyD+jKkUp5mpnj4vH3FRbpAjZK/Y+SilpuX7yZpMhQ/qCTH9uMFtqdUHlNPW+k53PZiN706KKrOzWXr4/wl2tG0S3Un9te28zx2gbbkZRSLdAjPIgpw3rx5sYCKvX3V6lvOZyGO5dspry6nueuH0NYoJ/tSF5DC+1OaOmGfCrrHNw4SXtitlT30AD+NnM0uUcquX9Zho7XVsrDzJ2YSEVNA+9sOXHdNaU6r799upc1+47wu6nDGBzTxXYcr6KFdifjcBrmr8lhXFJ3hsdF2I7jkcb3jeSuCweyfOsB7curlIcZk9CN5F5dWJSWq2+UlQK+3lvC3z7by5Vj4piREmc7jtfRQruT+WRHMQXHqnWBmlb6v3P7c9aAKB5ensmug+W24yilmklEmDsxkV0HK1iffdR2HKWsOlRew8+XbKF/dBi/mzZUx2W3Ay20O5mXV2cT2zWYC5NjbEfxaD4+wtNXj6JLsD+3vrpJx3sq5UEuHxlLRLA/C3VSpOrEnE7DXUu3UlnXwHPXjSEkQMdltwcttDuR7YVlrMs+yg0Tk3RltDYQHR7IX68Zxf7DlTz0bqbtOEqpZgoO8GXG2DhWbj9IcXmN7ThKWfHvVftZlXWYhy8byoCe4bbjeC0ttDuRl1fnEBLgy9VnxNuO4jUm9o/i9h/0561NBby/7YDtOEqpZrp+QiIOY3htXZ7tKEp1uO2FZfxp5W4mD41hptYE7UoL7U7iUEUN7209wIyxcUQE+9uO41XuOH8Ao+K7cv+yDA6UVtuOo5RqhqSoUM4ZGM1r6/Ooa3DajqNUh6mqa+COxZuJDA3kj1dqv+z2poV2J/Hq2jzqHE5u0JZ+bc7P14e/zhyFw2m4a+kWHE7tZKCUJ5ibmkRJRS0rMw/ajqJUh3n0vR1kH6nk6WtG0jUkwHYcr6eFdidQU+/g1XW5nD+4B32iQm3H8UqJkaE8fPlQ1u4/yotf77cdRynVDOcMjCahewiL0nRSpOocPswoYsmGfH52Tj8m9ouyHadT0EK7E3hv6wEOH6/jpjP1bnZ7mjE2jinDYnjq491sL9Ql2pVydz4+wvUTElifc5SdRdqmU3m3A6XV3Lssg5FxEfziwoG243QaWmh7OWMM81bnMKhnOBP7RdqO49VEhMenDycyNJA7lmymus5hO5JS6hSuTokn0M+HhXpXW3kxp2toY4PDyV9njsbfV8u/jqL/T3u5tfsb79TcOClJJzx0gK4hATx19Uj2l1Ty2IodtuMopU6ha0gAU0f15p3NhZRV19uOo1S7mLc6m7X7j/Lw5UNJ0iGkHUoLbS/38upsuoX4M210rO0oncak/lH8+Kw+vLI2j093FtuOo1SziMhkEdktIlkicu9Jnk8UkU9FZJuIfCEica7to0QkTUQyXc9d0/HpW2dOahLV9Q7e3FhgO4pSbS7rUAVPrtzNBUN6MmOsLrHe0bTQ9mJ5R6r4ZGcx141PJMjf13acTuXuiwcxOCacX7+VwbHKOttxlPpeIuILPAtMAZKBWSKSfMJufwYWGmNGAI8Cj7u2VwFzjDFDgcnAX0Ska8ckbxvDYiMYk9CVV9bm4tSuQcqL1Duc3LV0K2GBfjw+XVv52aCFthebvyYHXxFmpybajtLpBPr58vTVoyirruOh5bpqpHJ744AsY8x+Y0wdsASYesI+ycBnrseff/O8MWaPMWav6/EB4BAQ3SGp29Cc1CSyD1fyddZh21GUajPPfb6PbQVlPDZtGNHhgbbjdEpaaHupipp6lqbnc+mIXvTsEmQ7TqeU3LsLd5w3gPe2HuCDbUW24yj1fWKB/CbfF7i2NbUVmO56fAUQLiL/NcNaRMYBAcC+dsrZbqYMjyEqLIBFaTm2oyjVJjIKyvj7Z3uZNqo3U4b3sh2n09JC20u9kV7A8doGbeln2c/O7ceIuAgefCeDkopa23GUao27gXNEZDNwDlAIfNtaR0R6AYuAG40x/7PUoojcIiLpIpJeUlLSUZmbLdDPl5lnJPDprkPkH62yHUepVqmpd3DX0i1EhQXyyOXDbMfp1LTQ9kIOp2H+mhxSErsxIs6jhkp6HT9fH56aMZLKOgcPvJ2BMTr+U7mlQiC+yfdxrm3fMsYcMMZMN8aMBh5wbSsFEJEuwAfAA8aYtSf7AcaYF4wxKcaYlOho9xxZcu34BHxEeGWdtvpTnu3pT/aw99BxnrhqBBEh/rbjdGrNKrSbMRs9UERedz2/TkSSmjx3n2v7bhG52LUtXkQ+F5EdrpnqdzbZv7uIfCIie13/2631l9m5fLqzmLyjVXo3200M6BnO3RcN5OMdxbyzpfDUByjV8TYAA0Skj4gEADOB5U13EJEoEfnmb8Z9wDzX9gDgbRonSr7ZgZnbXO+uwVw4pCdLN+RTU6998JVnWp/duELxteMTOGege76p7UxOWWg3czb6zcAxY0x/4BngCdexyTS+YH8zG/051/kagF8aY5KBCcCtTc55L/CpMWYA8Knre9UC81ZnE9s1mIuSe9qOolxuPrMvKYndePjdTA6W1diOo9R/McY0ALcBK4GdwFJjTKaIPCoil7t2OxfYLSJ7gJ7AY67tVwNnAzeIyBbX16iOvYK2Myc1kWNV9by39YDtKEq1WGVtA3e/sZX4biE8cMkQ23EUzbuj3ZzZ6FOBBa7HbwLnS2MPmanAEmNMrTEmG8gCxhljiowxmwCMMRU0vrDHnuRcC4Bpp3dpndOOA+Ws3X+UOamJ+OnKT27D10f404yR1Dmc/PqtbTqERLkdY8wKY8xAY0w/Y8xjrm0PGWOWux6/aYwZ4NrnR8aYWtf2V4wx/saYUU2+tti8ltZI7RdJ/x5hLFqrw0eU53nio13kH6vizzNGEhroZzuOonmFdnNmo3+7j+vOSBkQ2ZxjXcNMRgPrXJt6GmO+adFwkMY7J6qZXl6dTbB/46Qe5V76RIVy7+TBfLmnhNc35J/6AKVUhxMR5qQmsq2gjC35pbbjKNVs6/YfYWFaLjdO7MO4Pt1tx1EuVm95ikgY8Bbwc2NM+YnPm8bbfie99efuM9htOHy8lne3HOCqsXE6+cFNzUlNIrVvJL//YCcHSqttx1FKncT0MXGEBfqxcE2O7ShKNUt1nYNfv7WNhO4h3H3xQNtxVBPNKbRPORu96T4i4gdEAEe+71gR8aexyH7VGLOsyT7FrjZR37SLOnSyUJ4wg72jvbo2jzqHkxsmJdmOor6Dj4/w5FUjcDgN92sXEqXcUligH9PHxPL+tiKOHNe2nMr9Pf3JbnKOVPHHK4cTEqBDRtxJcwrtU85Gd30/1/X4KuAz193o5cBMV1eSPsAAYL1r/PZLwE5jzNPfc665wLstvajOqLbBwaK1ufxgUDT9osNsx1HfI757CL+6eBBf7C7h7c3ahUQpdzQnNZE6h5MlOsxLubnNecd4aVU2141PYGK/KNtx1AlOWWg3czb6S0CkiGQBd+HqFGKMyQSWAjuAj4BbjTEOYBIwGzivySz1S1zn+iNwoYjsBS5wfa9O4f2tRRw+Xqst/TzE3IlJjEnoyqPv79CFbJRyQ/17hDOxXySvrcvD4dRPnpR7qm1wcM+b24jpEsS9UwbbjqNOolljtJsxG73GGDPDGNPfGDPOGLO/ybGPuY4bZIz50LVtlTFGjDEjmsxSX+F67ogx5nzX7PYLjDFH2/6yvYsxhnmrsxnQI4wz++u7WU/g6xpCUlXr4LfLM23HUUqdxJzURApLq/l0Z7HtKEqd1D8+y2LvoeP8YfpwwoN0bpY70v5vXmB99lEyD5Rz05l9aByVozxB/x7h3HF+fz7IKOKj7Qdtx1FKneCCIT0NvJt/AAAgAElEQVTpFRHEwjRt9afcz/bCMp77Yh9Xjonj3EE9bMdR30ELbS8wb3U2XUP8mTbqxK6Lyt395Jx+JPfqwm/e3U5ZVb3tOEqpJvx8fbhufAKrsg6Tdei47ThKfave4eSeN7fRPTSA31yqC9O4My20PVz+0So+2VHMteMSCA7wtR1HtZC/rw9PXjWCo5V1/P6DHbbjKKVOMHNcAgG+PryiC9goN/KvL/exo6ic308bRteQANtx1PfQQtvDLViTg48Is1MTbUdRp2lYbAQ/Obsvb2ws4Ks92hNeKXcSFRbIJcNjeGtjAcdrG2zHUYo9xRX87dMsLh3Ri4uHxtiOo05BC20Pdry2gdc35HPJ8F70igi2HUe1wh3nD6BvdCj3LcugUv+YK+VWZqcmUVHboO04lXVOp+Het7YRGujLI5cPtR1HNYMW2h7szfR8KmobtKWfFwjy9+XJK0dwoKyaJz/aZTuOUqqJMQldGRbbhUVpObrIlLLq1fV5bMor5TeXJhMZFmg7jmoGLbQ9lNNpeHlNDmMSujIqvqvtOKoNpCR1Z25qEgvSctmYq10tlXIXIsKcCUnsKT7Oumz93VR2HCyr4ckPd3Fm/yiuGK3NDzyFFtoe6rNdh8g9UqV3s73Mry4eRGzXYO59K4O6BqftOEopl8tH9aZriD8L03JsR1Gd1G+XZ1LncPLYFcO0la8H0ULbQ81bnU3viCAm60QIrxIa6Mfvpg1l76Hj/OvLfbbjKKVcgvx9uTolnpWZxRwsq7EdR3UyH2ce5KPMg9x5wQASI0Ntx1EtoIW2B9pZVM6afUeYnZqEn6/+E3qb8wb35IcjevH3z7LYV6K9e5VyF9ePT8RpDK+t01Z/quNU1NTz0LuZDI4J58dn9bUdR7WQVmkeaP7qHIL8fZg1Lt52FNVOHr4smUB/H+5flqGTr5RyEwmRIfxgUA9eW5+vQ7tUh3nq4z0UV9Tw+PTh+OvNNY+j/2Ie5sjxWt7eUsiVY+K0Sb0X6xEexP2XDGFd9lHeSC+wHUcp5TI7NZHDx2v5KPOg7SiqE9iSX8qCtBzmTEhkdEI323HUadBC28O8ti6PugYnN05Ksh1FtbNrUuIZl9Sdx1bspKSi1nYcpRRwzoBoEiNDWLgmx3YU5eXqHU7ufWsbPcODuPviQbbjqNOkhbYHqWtwsnBtLucMjKZ/j3DbcVQ78/ER/jB9ONV1Dn73vi7PrpQ78PERZk9IJD33GJkHymzHUV7s319ns+tgBY9MHUp4kL/tOOo0aaHtQT7IOEBJRa229OtE+vcI49Yf9Gf51gN8vvuQ7ThKKWDG2HiC/H1YlKaTIlX7yD1SyV8/3cPFQ3vqMuseTgttD2GM4aVV2fTvEcbZA6Jsx1Ed6Kfn9qV/jzAefHs7VXW6PLtStkWE+DNtVCzvbCmkrKredhzlZYwxPPjOdvx8fHjk8mG246hW0kLbQ6TnHmN7YTk3TkrSRvWdTKCfL49PH05haTXPfLLHdhylFI2TImvqnbyxMd92FOVl3t1ygK/3HuaeyYOIiQiyHUe1khbaHmLeqmwigv2ZPjrOdhRlwRlJ3bl2fAIvrcpme6GOC1XKtqG9I0hJ7Maitbk4ndqCU7WNY5V1PPr+DkYndOW68Ym246g2oIW2Byg4VsXKzIPMGpdAcICv7TjKkl9PHkxkWCD3LttGg0N7+Cpl2+zURHKPVPHV3hLbUZSXeGzFTsqr63l8+nB8ffTTa2+ghbYHWJiWi4gwJ1Xf3XZmEcH+PHL5ULYXljNfW4spZd2UYb2ICgtkoU6KVG1gTdZh3txYwC1n92VwTBfbcVQb0ULbzVXWNrB4fR5ThsXQu2uw7TjKsinDYrhgSA+e+ngP+UerbMdRqlML8PPh2nHxfL77EHlH9PdRnb6aegf3v51BYmQId5w/wHYc1Ya00HZzb20qoKKmQVv6KQBEhEenDsNH4MF3tuvy7EpZdu34RHxEeGWd3tVWp+8fn2WRc6SKP1wxnCB/HSLqTbTQdmNOp+Hl1TmMiu/KGF16Vbn07hrM3RcP4ss9Jby3rch2HKU6tZiIIC4e2pOl6fnU1Dtsx1EeaPfBCv755T6mj4llUn9t3+tttNB2Y1/sOUT24Uq9m63+x5zUJEbGRfDoe5mUVtXZjqNUpzZ7QhKlVfUs33rAdhTlYZxOw33LthEe5MeDP0y2HUe1g2YV2iIyWUR2i0iWiNx7kucDReR11/PrRCSpyXP3ubbvFpGLm2yfJyKHRGT7Cef6rYgUisgW19clp395nm3eqhxiugQxZZiuCqX+m6+P8Pj0ERyrqufxFbtsx1FeoBmv84ki8qmIbBORL0Qkrslzc0Vkr+trbscmt29C3+4M7BnGwrQcHc6lWuTV9XlsyivlwR8m0z00wHYc1Q5OWWiLiC/wLDAFSAZmiciJb7tuBo4ZY/oDzwBPuI5NBmYCQ4HJwHOu8wHMd207mWeMMaNcXytadkneYffBClZlHWbOxET8ffWDB/W/knt34Udn9eH19HzW7j9iO47yYM18nf8zsNAYMwJ4FHjcdWx34GFgPDAOeFhEOtVYNxFhdmoS2wvL2ZxfajuO8hDF5TU8+eEuJvWPZPqYWNtxVDtpTgU3Dsgyxuw3xtQBS4CpJ+wzFVjgevwmcL40Ll84FVhijKk1xmQDWa7zYYz5CjjaBtfglV5enU2Qvw+zzkiwHUW5sZ+fP5D47sHc/3YGtQ06PlSdtua8zicDn7kef97k+YuBT4wxR40xx4BP+O6bKF7ritGxhAX6sVBbb6pm+u3yTOocTh6bNlxXfPZizSm0Y4Gma8wWuLaddB9jTANQBkQ289iTuc318eS8znZnBOBoZR1vby7kitFxdNOPktT3CA7w5bFpw9lfUslzn++zHUd5rua8Vm8FprseXwGEi0hrXue9SligH1eNjWNFxkEOH6+1HUe5uU92FPPh9oPccf4AkqJCbcdR7cgdxyQ8D/QDRgFFwFMn20lEbhGRdBFJLynxrlW5Fq/Po7bByU2TkmxHUR7g7IHRTBvVm+e+yCLrUIXtOMp73Q2cIyKbgXOAQqDZH6N482v2N66fkEidw8nrG/JPvbPqtI7XNvDQu9sZ1DOcW87uazuOamfNKbQLgfgm38e5tp10HxHxAyKAI8089r8YY4qNMQ5jjBN4EddQk5Ps94IxJsUYkxIdHd2My/AMdQ1OFqblcNaAKAb0DLcdR3mIBy9NJjTQj/uWZeB06mQs1WKnfK02xhwwxkw3xowGHnBtK23Osa59vfI1u6n+PcI4s38Ur6zNpcHhtB1Huak/r9zNwfIaHr9yuM7B6gSa8y+8ARggIn1EJIDGyY3LT9hnOfDNTPOrgM9M49Tr5cBMV1eSPsAAYP33/TAR6dXk2yuA7d+1rzf6cHsRxeW12tJPtUhUWCD3XzKEDTnHeD1d76apFjvl67yIRInIN38z7gPmuR6vBC4SkW6uoX4XubZ1SrNTEykqq+E/Ow/ZjqLc0Jb8Uhak5TB7QqKuj9FJnLLQdo25vo3GF86dwFJjTKaIPCoil7t2ewmIFJEs4C7gXtexmcBSYAfwEXCrMcYBICKLgTRgkIgUiMjNrnM9KSIZIrIN+AHwiza6VrdnjOGlVdn0jQ7lnAHeecdHtZ8ZY+OY0Lc7f1ixk0MVNbbjKA/SzNf5c4HdIrIH6Ak85jr2KPA7Gov1DcCjrm2d0vmDe9A7IoiFaTm2oyg3U+9wct+yDHqGB/GriwfZjqM6iHhDz8+UlBSTnp5uO0arbcw9ypXPp/G7acOYPSHRdhzlgfaXHGfyX7/mouSe/OPaMbbjqGYQkY3GmBTbOTqSt7xmf5dnP8/iTyt385+7zqZ/Dx0CqBr988t9/PHDXfzz+rFM1vUxPFpLXrd1cJAbmbcqhy5Bflyp/TTVaeobHcbtP+jP+9uK+GxXse04SnVKM8+IJ8DXh0VpubajKDeRd6SKv/xnDxcl99Qiu5PRQttNFJZW81HmQWaNSyAkwM92HOXBfnJOPwb0COM372RSWdtgO45SnU5kWCCXjujFW5sKOa6/g52eMYYH3snAz8eHR6YOtR1HdTAttN3EN+P55kxMshlDeYEAPx8enz6cwtJqnvlkj+04SnVKs1MTOV7bwNubCmxHUZa9s6WQr/ce5p7Jg+gVEWw7jupgWmi7gaq6Bhavy2Py0Bhiu+ovoWq9lKTuXDc+gXmrs8koKLMdR6lOZ1R8V4bHRrAgLRdvmAulTs/Ryjp+9/5ORid05brxOveqM9JC2w28tamQ8poGbjozyXYU5UXumTyYyLBA7nt7m/b0VaqDiQhzUhPJOnSctP1HbMdRljz2wU7Kq+t5fPpwfH10mfXOSAtty5xOw8ursxkZF6E9NVWbigj255HLh7K9sJz5a3Jsx1Gq07lsZG+6hvjrpMhOanXWYd7aVMBPzunL4JgutuMoS7TQtuzLvSXsL6nkpjP7IKLvdlXbmjIshguG9OCpj/eQf7TKdhylOpUgf1+uOSOej3cUU1RWbTuO6kA19Q7ufzuDpMgQbj9vgO04yiIttC2btyqbnl0CmTKs16l3VqqFRIRHpg5DBH7z7nYdK6pUB7t+fCJOY3htXZ7tKKoD/e3TveQeqeIPVwwnyN/XdhxlkRbaFu0pruDrvYeZk5pEgJ/+U6j2Eds1mLsvGsQXu0t4f1uR7ThKdSrx3UM4b1APFq/Po7bBYTuO6gA7i8p54av9XDU2jon9o2zHUZZpdWfRy6tzCPTzYda4BNtRlJebOzGJEXERPPLeDsqq6m3HUapTmTMxicPH6/ho+0HbUVQ7czgN9y3LoEuwPw9cMsR2HOUGtNC25FhlHcs2FXDF6Fi6hwbYjqO8nK+P8IcrhnOsqo4/frTTdhylOpWz+keRFBnCAp2U7PVeWZvLlvxSHro0mW76t12hhbY1izfkUdvg5MZJfWxHUZ3EsNgIfnRmHxavz2d99lHbcZTqNHx8hNmpSWzKK2V7ofa191YHSqt58qNdnDUgiqmjetuOo9yEFtoW1DucLFyTy5n9oxgUE247jupE7rxgAHHdgrlv2TYdL6pUB7pqbBzB/r7a6s9LGWN46N1MHMbw2LTh2kVMfUsLbQs+3H6Qg+U1ukCN6nAhAX78ftow9pVU8vwX+2zHUarTiAj2Z9roWN7ZUkhpVZ3tOKqNrcw8yH92FvOLCwaSEBliO45yI1poWzBvVTZ9okI5d2AP21FUJ3TuoB5cPrI3z32+j6xDx23HUarTmJOaSG2DkzfSC2xHUW2ovKaeh97NJLlXF24+U4eDqv+mhXYH25R3jC35pdw4KQkfXY5VWfKbS5MJDvDl/mUZOJ3aW1upjjCkVxfOSOrGorW5+nvnRZ78aBeHj9fy+PTh+PlqWaX+m/4X0cHmrcomPMiPK8fE2Y6iOrHo8EAeuGQI63OOsjQ933YcpTqNOalJ5B2t4ss9JbajqDawIecor6zN44aJfRgZ39V2HOWGtNDuQAdKq/lw+0FmnhFPaKCf7Tiqk5uREsf4Pt35w4qdlFTU2o6jVKdw8dAYosMDWZiWYzuKaqWaege/fnMbcd2C+eVFA23HUW5KC+0OtDAtF2MMc1KTbEdRChHhD9OHU1Pv5LfvZdqOo1SnEODnw7XjEvhiTwm5Ryptx1Gt8Jf/7GX/4Uoenz5cb56p76SFdgeprnOweH0eFw+NIb67zkhW7qFfdBi3n9efD7YVsTJTV61TqiNcOz4BXxFeWaut/jxVRkEZL369n6tT4jhrQLTtOMqNaaHdQZZtLqCsup6bdEaycjM/Pbcfyb268OA723V5dqU6QM8uQVw8NIbXN+RTXaf97D1NvcPJPW9tIzI0gAd+mGw7jnJzWmh3AKfTMG9VNsNjI0hJ7GY7jlL/xd/XhyevGsHRyjp+98EO23GU6hTmpCZSXtPA8q2FtqOoFvrnF/vYWVTO76cNIyLY33Yc5ea00O4AX2cdZl9JJTedmaSrRSm3NCw2gp+d0483Nxbwxe5DtuMo5fXG9enOoJ7hLFjTOHdHeYa9xRX8/bMsfjiiFxcNjbEdR3kALbQ7wLxV2USHB/LD4b1tR1HqO91+fn/69wjj/mUZVNToEBKl2pOIMGdiIjuKytmUd8x2HNUMDqfhnre2ERroyyOXD7UdR3mIZhXaIjJZRHaLSJaI3HuS5wNF5HXX8+tEJKnJc/e5tu8WkYubbJ8nIodEZPsJ5+ouIp+IyF7X/3r0WIusQxV8uaeEORMSCfDT9zXKfQX6+fKnq0ZwsLyGP364y3YcpbzetFGxhAf6sTBNJ0V6gvlrcticV8rDlw0lKizQdhzlIU5Z+YmIL/AsMAVIBmaJyImj/28Gjhlj+gPPAE+4jk0GZgJDgcnAc67zAcx3bTvRvcCnxpgBwKeu7z3Wy6tzGts5jU+wHUWpUxqd0I2bz+zDq+vyWLPvsO04Snm10EA/rhwbx4qMIu1l7+byjlTx55W7OW9wD6aO0k+nVfM15xbrOCDLGLPfGFMHLAGmnrDPVGCB6/GbwPnSOBh5KrDEGFNrjMkGslznwxjzFXD0JD+v6bkWANNacD1upbSqjrc2FTBtVG8i9d2v8hB3XTiIpMgQ7n0rg6q6BttxlPJqs1MTqXcYlqzPsx1FfQdjDPcu24avj/DYFcN0rpVqkeYU2rFA0zWaC1zbTrqPMaYBKAMim3nsiXoaY4pcjw8CPZuR0S0t2ZBPTb2TGydpSz/lOYIDfHniyhHkHa3izyv32I6jlFfrFx3GWQOieHVdHg0Op+046iQWr89nzb4j3HfJYHpFBNuOozyMWw8aNo1TsU86HVtEbhGRdBFJLykp6eBkp1bvcLJgTQ4T+0UypFcX23GUapHxfSOZk5rIy2uy2Zh7sg+elFJtZU5qEgfLa/hkR7HtKOoE+Uer+P0HO5jUP5JZZ+gQUNVyzSm0C4H4Jt/HubaddB8R8QMigCPNPPZExSLSy3WuXsBJe40ZY14wxqQYY1Kio91vVaaVmQcpKqvhJr2brTzUPZMH0zsimF+9sU0X1VCqHZ03uAexXYN1UqSbcToNd7+xFR8RnrxqJD4+OmREtVxzCu0NwAAR6SMiATROblx+wj7Lgbmux1cBn7nuRi8HZrq6kvQBBgDrT/Hzmp5rLvBuMzK6nXmrskmMDOG8wT1sR1HqtIQF+vHkVSPYf7iSJ1dqFxKl2ouvj3DdhATS9h9hT3GF7TjKZUFaDuuyj/LQpcnEdtUhI+r0nLLQdo25vg1YCewElhpjMkXkURG53LXbS0CkiGQBd+HqFGKMyQSWAjuAj4BbjTEOABFZDKQBg0SkQERudp3rj8CFIrIXuMD1vUfZnHeMTXml3DgxSd8BK482qX8Uc1MTeXl1jnYh8XLNaOOaICKfi8hmEdkmIpe4tvuLyAIRyRCRnSJyX8en93zXpMQT4OfDIr2r7Rb2lxzniY928YNB0cxIibMdR3kwv+bsZIxZAaw4YdtDTR7XADO+49jHgMdOsn3Wd+x/BDi/Obnc1curcwgP9OOqlPhT76yUm7t3yhC+2nuYX72xjY9+fhbhQbrksLdp0sb1QhonrW8QkeXGmB1NdnuQxhstz7tat64Akmh87Q80xgwXkRBgh4gsNsbkdOhFeLjIsEAuHdGLZZsKuGfyIP09s8jhGjIS6OfLH68coV1GVKu49WRIT1RUVs2KjCKuOSOesMBmvY9Ryq0FB/jy5xkjKSqr5nfv7zj1AcoTNaeNqwG+mdkdARxosj3UNT8nGKgDyts/sveZm5pEZZ2DZZtONZVJtacXv97PprxSHp06lJ5dgmzHUR5OC+02tigtF6cxzJ2YZDuKUm1mbGI3fnJOP5amF/Af7YzgjZrTivW3wPUiUkDj3ezbXdvfBCqBIiAP+LMxRlvVnIaR8V0ZGRfBwrQcGqc5qY62p7iCpz/ew+ShMVw+UhemUa2nhXYbqq5z8Nr6PC5M7kl89xDbcZRqUz+/YACDY8K5d1kGxyrrbMdRHW8WMN8YEwdcAiwSER8a74Y7gN5AH+CXItL3xIPdvSWru5iTmsS+kkrW7DtiO0qnU+9w8sulWwkL8uP3ujCNaiNaaLehd7YUUlpVry39lFcK9PPl6atHUVZdx4PvbrcdR7Wt5rRivZnGye0YY9KAICAKuBb4yBhTb4w5BKwGUk78Ae7ektVd/HBEL7qHBrAwLcd2lE7nuc/3kVFYxh+uGEaUruas2ogW2m3EGMO8VdkM7d2FcX26246jVLtI7t2Fn18wkA+2FbF864FTH6A8RXPauObhmqguIkNoLLRLXNvPc20PBSYA2g/yNAX5+3J1Sjyf7CimsLTadpxOY3PeMf722V6mjerN5GG9bMdRXkQL7TayKuswew8d56ZJffTjJuXVfnJ2X0YndOU372ynuLzGdhzVBprZxvWXwI9FZCuwGLjBtV7Cs0CYiGTSWLC/bIzZ1vFX4T2uG9+4AuFr67TVX0eorG3gF69vIaZLEI9MHWY7jvIyWmi3kXmrsokKC+TSkfpOWHk3P18fnpoxktoGB3ct3YLTqZO2vIExZoUxZqAxpp+rLSvGmIeMMctdj3cYYyYZY0YaY0YZYz52bT9ujJlhjBlqjEk2xvzJ5nV4g/juIZw3uCdL1udT26Crsra3R9/bQe7RKp6+eiQRwdpWUbUtLbTbwL6S43y+u4TZExIJ9PO1HUepdtc3OoyHLxvK6qwj/HvVfttxlPI6cycmcqSyjhUZRbajeLWPthfxeno+PzunH+P7RtqOo7yQFtptYP7qHAJ8fbhuQoLtKEp1mJlnxDN5aAx/Wrmb7YVltuMo5VUm9Yuib1QoC3WlyHZzsKyGe5dlMCIugp9fMNB2HOWltNBupbKqet7cWMDlo3rrLGXVqYgIf7xyOJGhgdyxeDNVdQ22IynlNXx8hOsnJLI5r5SMAn0j29acrtUfa+ud/OWaUQT4aTmk2of+l9VKSzbkUV3v4MZJSbajKNXhuoYE8PQ1I8k+Usmj7+mqkUq1pSvHxhES4Kut/trBvNXZrMo6zG8uTaZvdJjtOMqLaaHdCg0OJwvTcpnQtztDe0fYjqOUFRP7RfGzc/qxZEM+H+p4UqXaTESwP9NGx7J86wFdJKoN7ThQzpMf7ebC5J7MGhd/6gOUagUttFvhY1efU12gRnV2v7hwICPjIrh3WQYHtPevUm1mTmoitQ1Olqbn247iFSprG7ht8SYiQvz54/Th2o5XtTsttFth3qpsErqHcP6QnrajKGWVv68Pf505mnqHk1+8vgWHtvxTqk0MjmlcBO2Vdbn6e9VKxhgefGc7OYcr+evMUUTqvCrVAbTQPk1b80tJzz3GDROT8PXRd8RKJUWF8rupw1iXfZS/frrXdhylvMac1ETyj1bzxe5DtqN4tDc2FvD25kLuOH8AE/tF2Y6jOgkttE/Ty6uzCQv0Y0ZKnO0oSrmNK8fGMWNsHH//bC9f7SmxHUcpr3Dx0Bh6hAdqq79W2FtcwUPvbmdiv0huP2+A7TiqE9FC+zQUl9fw/rYirk6JJzxIV5FSqqlHpw5jYI9wfvH6Fg6W6RLtSrWWv68P145P4Ms9JWQfrrQdx+NU1zm49bVNhAX68ZeZo/RTaNWhtNA+DYvScnEYww0Tk2xHUcrtBAf48ux1Y6iud3D74k00OJy2Iynl8a4dl4Cfj/DKWr2r3VIPL9/O3kPH+cs1o+kRHmQ7jupktNBuoZp6B6+uy+WCIT1JiAyxHUcpt9S/RxiPTx/Ohpxj/PnjPbbjKOXxenQJYvKwGN5Iz9fFoVpg2aYClqYXcOu5/TlzgI7LVh1PC+0WemdzIceq6rWln1KnMHVULNeOT+CfX+7j053FtuMo5fHmpCZRXtPAu1sO2I7iEbYXlnHfsgwm9O3Ozy/QcdnKDi20W8AYw8urcxjSqwsT+na3HUcpt/fQpckk9+rCXUu3knekynYcpTzaGUndGBwTzsK0XIzRVn/fp7Sqjp++spFuIQH849ox+PlquaPs0P/yWmDNviPsLq7gpklJ2uReqWYI8vfl+evHAHDLonT9yFupVhAR5qQmsbOonPTcY7bjuC2H03DHki0cKq/l+evHEKX9spVFWmi3wLxV2USFBXDZyN62oyjlMRIjQ/nbrNHsKa7gV29u0ztxSrXCtNG9CQ/y01Z/3+OZT/bw1Z4SHpk6lNEJ3WzHUZ2cFtrNlH24kk93HeK68YkE+fvajqOURzlnYDT3TB7MB9uK+OeX+23HUcpjhQT4MWNsPB9mFHGoXNtnnmhl5kH+8XkW16TEM2tcgu04SjWv0BaRySKyW0SyROTekzwfKCKvu55fJyJJTZ67z7V9t4hcfKpzish8EckWkS2ur1Gtu8S2MX91NgG+Plw3QX9xlTodPzm7L5eO6MWTK3fpCndKtcLs1EQanIbF6/NtR3Er+0qO88ulWxkRF8EjU4fajqMU0IxCW0R8gWeBKUAyMEtEkk/Y7WbgmDGmP/AM8ITr2GRgJjAUmAw8JyK+zTjnr4wxo1xfW1p1hW2grLqeNzYWcOnIXtqDU6nTJCI8edUIBvUM547Fm8nRhTeUOi19okI5e2A0r63PpV771AONkx9/tCCdAD8fnr9+rH7yrNxGc+5ojwOyjDH7jTF1wBJg6gn7TAUWuB6/CZwvjbMFpwJLjDG1xphsIMt1vuac020s3ZBPVZ1DW/op1UohAX68OCcFHx/hlkXpVNTU246klEeaMyGR4vJaPs7U1pn1Dic/e2UThceqeWH2WGK7BtuOpNS3mlNoxwJNP58qcG076T7GmAagDIj8nmNPdc7HRGSbiDwjIlanCzc4nMxfk8O4Pt0ZFhthM4pSXiG+ewjPXjuGfSWV3PbaZl05UqnT8IPBPYjrFszCtBzbUawyxvDQu9tJ23+Ex6cPJyVJW+8q9+KOkyHvAwYDZwDdgV+fbCcRuUVE0q6zmEQAABYrSURBVEUkvaSkpN3C/GdnMYWl1Xo3W6k2NKl/FL+fNowv95Tw2/cytROJUi3k6yNcPyGRddlH+X/t3Xl0VeW5x/Hvk5AQEkgYEhAzMYNMMoQwaUWsilhFlAoiImq1xWHp1Wq1vbe1XWqvbdW2t04ooyjIIBbrgANWkSEQhjCjDAEShkAYAoEkJHnuH2ejacwkZJ99kvN81srKPvvss9/feXPy5sketx044XUcz0xZmsmslXuZOKQ9N/VN8DqOMd9Tk0I7G0gs8zjBmVfhMiLSAIgBcqt4baXrVNX96lMITMV3mMn3qOokVU1R1ZS4uLgavI1zM+WrTBKaNeLKrq1ca8OYYHRLahI/v6wdM1fsYfJXu7yOY0ydMzolkYYNQoJ2q/bnW3N4+v3NXN2tFY9e1dnrOMZUqCaF9iqgo4i0FZFwfCc3Liy3zELgdmd6FLBYfZuoFgJjnKuStAU6AiurWqeItHa+C3ADsPF83uD52JB1nJWZR5gwqA2hIXaDGmNq26+u7sI13S/g6Q+2sGjTAa/jGFOnNIvy3ddhwdps8oLsfIf1Wce47601XNQ6mhdG9yLE/kabAFVtoe0cc30/sAjYAsxR1U0i8gcRud5ZbDLQQkS2Aw8Djzuv3QTMATYDHwH3qWpJZet01vWmiGwANgCxwFO181Z/uKlLdxEVHsrN/RKrX9gY84OFhAjP39yLnglNeWj2OtbtPeZ1JGPqlPEDkzlVVML81VleR/GbzMP53DF1Fc0iw5k6oR+R4Q28jmRMpaQ+HBuZkpKi6enptbrOnLwCBj+7mFv7J/Pk9XY9TmPcdOhEITe+vJSTBcXM/cUgOrRs7HUkvxGR1aqa4nUOf3JjzA5mN7y4lLyCM3z28GX4dgbXX4dPFnLTy8vIO32GeRMH0T4ueMYKEzh+yLgdiCdDBoSZK3ZTXKpMGNTG6yjG1HtxTRryxp39CQ0JYfzkNPYdO+11JGPqjPEDk9l5KJ+l23O9juKq/MJi7pi6ioN5BUye0M+KbFMnWKFdgYIzJcxM28MVXVrSJjbK6zjGBIU2sVFMv7MfJwqKGT9lJUfzi7yOZEydMLxHa1pEhTN9eabXUVxTVFzKxDfXsHl/Hi+O7UOfpGZeRzKmRqzQrsDCdfs4kl9kl/Qzxs+6XRjD67ensPfIKSZMW0V+YbHXkYwJeBFhoYzul8hnWw6SdfSU13Fq3ZmSUh6YtYYvvz7EMyO7c8VFdhUwU3dYoV2OqjJl6S66XNCEge1beB3HmKDTv10L/jG2Dxuzj3P3jHROF5V4HcmYgHfrgGQA3kzb43GS2lVSqjw8J4NFmw7yu+u6MrpfkteRjPlBrNAuZ/nOXLYeOMGdg9vW+5NKjAlUV3ZtxXM/vZgVO3P52YxVVmz7gYgME5FtIrJdRB6v4PkkEflcRNY6d+4dXua5niKyXEQ2icgGEYnwb3oT37QRP76oFW+v2kvBmfrx+1Jaqjw2bz3vZezjiWu6cIftZTZ1kBXa5Uz5KpPmUeFc3+tCr6MYE9Ru6B3PczdfzLIdudw9I73eFA+BSERCgReBa4CuwC0i0rXcYv+N71KsvfHd++Al57UNgJnAL1S1GzAECK6LOgeI8QPbcCS/iPfX7/c6ynlTVX7z7kbmr8ni4Ss78fPL2nsdyZhzYoV2GZmH8/ls60HG9U8iIizU6zjGBL2RvRP4y6iLWbrjMD+bbsW2i1KB7aq6U1WLgNnAiHLLKBDtTMcA+5zpq4D1qpoBoKq5qmo/KA8M7tCCdnFRzFix2+so56W0VPn1go3MWrmHe4e054GhHbyOZMw5s0K7jGnLMmkQIoxzjnUzxnjvpr4J/LlMsW0nSLoiHthb5nGWM6+sJ4FxIpIFfAA84MzvBKiILBKRNSLymNthTcVEhPEDksnYe4yMOnrzp+KSUh6Zm/Ftkf3o1Z3tME5Tp1mh7cgrOMPc9L1c1/NCWkbb4YXGBJJRTrG9bMdhxk1O49gpu/SfB24BpqlqAjAceENEQoAGwCXArc73kSJyRfkXi8g9IpIuIumHDh3yZ+6gcmPfBCLDQ5mxvO5t1S4qLuX+t9ayYG02j17dmceGdbEi29R5Vmg75qzaS35RiZ1sYUyAGtU3gZdu7cum7DxGv7qCg3kFXkeqT7KBxDKPE5x5Zd0FzAFQ1eVABBCLb+v3l6p6WFVP4dva3ad8A6o6SVVTVDUlLi7OhbdgAKIjwrixTzzvrfddprauyC8s5u4Z6Xy06QD/85Ou3He5HS5i6gcrtPFdPmjaskz6tWlGj4QYr+MYYyoxrPsFTLujH1lHT3HTy8vYdTjf60j1xSqgo4i0FZFwfCc7Liy3zB7gCgARuQhfoX0IWAT0EJFI58TIy4DNfktuvmf8wDYUFZfy9qq91S8cAHJOFDB60nKWfHOI/72xB3ddYhu8TP1hhTbwyeaDZB09bTeoMaYOGNQhlln3DOBUUQkjX1pK2s76fdtpf1DVYuB+fEXzFnxXF9kkIn8QkeudxR4B7haRDGAWMEF9jgLP4yvW1wFrVPV9/78Lc1anVk0Y0K45M1fspqRUvY5Tpe05Jxj54jJ25OTz+u0pjEm162Sb+sUKbWDq0l3EN23ElV3tblPG1AU9E5qy4N5BtIgKZ9zkNOatzvI6Up2nqh+oaidVba+qTzvzfquqC53pzao6WFUvVtVeqvpxmdfOVNVuqtpdVe1kyAAwfmAbso+d5vOtOV5HqdTyHbnc9PJyCotLePvnAxjaxf4Gm/on6AvtjdnHSdt1hAmD2tAgNOi7w5g6I7lFFO9MHEy/Ns355dwM/vTR1oDfemeMv1zZtRUXREcwfXmm11G+R1WZunQX4yanEds4nAX3DqZnQlOvYxnjiqCvLKcuzSQyPJSb+yVWv7AxJqDERIYx/c5UbklN5KV/72DC1JV16gQwY9wSFhrC2P5JLPnmMDsPnfQ6zrcKzpTwyNwMfv/eZi7v3JJ37xtMYvNIr2MZ45qgLrRzThTwXsY+fto3gZhGYV7HMcacg7DQEJ4Z2YM/3tiDtF1HuPbvS1iz56jXsYzx3JjURMJChTcC5AY2mYfz+ekry3lnTTYP/bgjk27rS5MI+9tr6regLrTfXLGHopJSJthJkMbUaSLCLalJzP/FIEJDhNGvLuf1JTsptUNJTBBr2SSCa7q3Zt7qLE4VeXejJ1Vl/uosrv37Enbn5vPa+BQe+nEnQkLsGtmm/gvaQrvgTAlvpu1maJeWtI2N8jqOMaYW9EiI4f0HLmVI55Y89f4Wxr6+gqyjp7yOZYxnxg9M5kRBMe+u3edJ+3kFZ3hw9joemZtBt/gYPnroR3bhARNUgrbQfi9jH4dPFtkl/YypZ2Iiw5h0W1/+dFNPNmbnMeyvS5iTvhdV27ptgk/f5GZ0bR3NjOWZfv8d+HjTAa56/kve37CfX17ViVl3D+DCpo38msEYrwVloa2qTFmaSadWjRncoYXXcYwxtUxEuLlfIh8+eCndLozmsXnrGftaGt8cPOF1NGP8SkQYPzCZrQdOsCrTP+cu5OQVMHHmau55YzVNI8N4Z+Ig7h/akVA7VMQEoaAstE8UFtMquiF3XdIWEfvFN6a+Smweyay7B/DUDd3ZvD+Pa/62hD9+sIWThd4dr2qMv43oFU90RAPXL/VXcKaEV7/YwRXPf8HirTk8Nqwz7z1wCRcn2qX7TPBq4HUAL0RHhDHtjlTblWxMEAgJEcYNSGZ4j9Y8++FWXv1yJ/PXZHPf5e25JTWJiLBQryMa46pG4aHcnJLItGWZHMwroFV0RK2uv7RUWZixjz8v2kb2sdNc3jmO317Xzc5/MoYg3aJ9lm3NNiZ4NI8K59lRPVlw7yA6tmzsu47vX/7Nm2m7KSwu8TqeMa4aNyCZ4lLlrbQ9tbbOEqfAvvb/vuKht9fRNDKMt37Wn6l3pFqRbYwjKLdoG2OCV++kZsy6ZwDLth/mLx9v4zcLNvLCJ99w24Bkbh2QRGzjhl5HNKbWtYmNYkjnOGat3MP9QzsQdh53Qj5++gzvrs1mytJd7M49Rfu4KF4YfTEjLo63S/YZU06NftNEZJiIbBOR7SLyeAXPNxSRt53n00SkTZnnnnDmbxORq6tbp4i0ddax3Vln+Pm9RWOM+b5BHWKZP3EQb9yVSo/4aF749GsG/XExE2eu5uNNBygqLvU6ojG1avzAZHJOFLJo04Ef/NqSUmX5jlwenZtB/2c+5XcLN9G0URivjOvDJ/91GSN7J1iRbUwFqt2iLSKhwIvAlUAWsEpEFqrq5jKL3QUcVdUOIjIGeBYYLSJdgTFAN+BC4FMR6eS8prJ1Pgu8oKqzReQVZ90v18abNcaYskSESzvGcWnHOHYcOsnMFbt5L2MfH248QEyjMIZ0jmNol5Zc2jGO5lH2P7+p2y7r1JLE5o2YsWw3P+l5YbXLHz91hhW7cvni60Ms2niA3PwiIsNDGdk7nrGpyfRIiPFDamPqtpocOpIKbFfVnQAiMhsYAZQttEcATzrT84B/iO8A6BHAbFUtBHaJyHZnfVS0ThHZAgwFxjrLTHfWa4W2McZV7eMa87vruvHr4Rex5JtD/Gv9fr7Ydoh/rvPd6KNdbBS9k5rRPT6atrFRtIttzAUxEYQ3COpTXUwdEhoi3DYgmWc+2MqW/Xlc1DoagMLiEnLyCsk+dpptB06wZX8eG7KPs3l/HqoQGR7K5V1acm2P1gzpHEdkuB11akxN1eS3JR7YW+ZxFtC/smVUtVhEjgMtnPkryr023pmuaJ0tgGOqWlzB8sYY47qw0BCGdmnF0C6tKC1VMrKOsXxnLmt2H+Pf23KYvybrP5aPCg+laWQ4jcJD+evoXnSPt618JnDdnJLIcx9/zdjXVhDeIITTRSXkFfzn5S6bRYZxUetoHryiI4Pax9Irsan9Q2nMOaqz/5aKyD3APQBJSUkepzHG1EchIULvpGb0TmoG+G52dfhkEbsO57Pr8EkOnSjk6KkzHD1VRMGZEiLD7VKBJrA1jQzn99d3Y/nOXCIahBIRFkKLxg25ICaC1jERdGzZhFbRDe2qXMbUkpoU2tlAYpnHCc68ipbJEpEGQAyQW81rK5qfCzQVkQbOVu2K2gJAVScBkwBSUlLsgtjGGNeJCHFNGhLXpCGpbZt7HceYczImNYkxqbaByhh/qMm+oFVAR+dqIOH4Tm5cWG6ZhcDtzvQoYLH67gazEBjjXJWkLdARWFnZOp3XfO6sA2ed/zz3t2eMMcYYY4w3qt2i7RxzfT+wCAgFpqjqJhH5A5CuqguBycAbzsmOR/AVzjjLzcF34mQxcJ+qlgBUtE6nyV8Bs0XkKWCts25jjDHGGGPqFKkPtyFPSUnR9PR0r2MYY8wPJiKrVTXF6xz+ZGO2MaYu+yHjtp1GbIwxxhhjjAus0DbGGGOMMcYFVmgbY4wxxhjjAiu0jTHGGGOMcYEV2sYYY4wxxrigXlx1REQOAbvP4aWxwOFajnM+AilPIGUBy1OdQMoTSFkg8PMkq2qcV2G8YGO2awIpTyBlActTnUDKE0hZoOI8NR6360Whfa5EJD2QLqsVSHkCKQtYnuoEUp5AygKWpz4JtL6zPJULpCxgeaoTSHkCKQucfx47dMQYY4wxxhgXWKFtjDHGGGOMC4K90J7kdYByAilPIGUBy1OdQMoTSFnA8tQngdZ3lqdygZQFLE91AilPIGWB88wT1MdoG2OMMcYY45Zg36JtjDHGGGOMK4Ky0BaRYSKyTUS2i8jjHmXIFJENIrJORNKdec1F5BMR+cb53szF9qeISI6IbCwzr8L2xefvTn+tF5E+fsrzpIhkO320TkSGl3nuCSfPNhG5upazJIrI5yKyWUQ2iciDznxP+qeKPF71T4SIrBSRDCfP7535bUUkzWn3bREJd+Y3dB5vd55v44cs00RkV5m+6eXMd/2z7LQTKiJrReRfzmO/90194/W4bWN2jfLYmF11nqAfs6vJ49m47eqYrapB9QWEAjuAdkA4kAF09SBHJhBbbt6fgMed6ceBZ11s/0dAH2Bjde0Dw4EPAQEGAGl+yvMk8MsKlu3q/NwaAm2dn2doLWZpDfRxppsAXzttetI/VeTxqn8EaOxMhwFpzvueA4xx5r8CTHSm7wVecabHAG/7Ics0YFQFy7v+WXbaeRh4C/iX89jvfVOfvgiAcRsbs2uSx6sxycbsqvMEzJhdTZ5peDRu4+KYHYxbtFOB7aq6U1WLgNnACI8znTUCmO5MTwducKshVf0SOFLD9kcAM9RnBdBURFr7IU9lRgCzVbVQVXcB2/H9XGsry35VXeNMnwC2APF41D9V5KmM2/2jqnrSeRjmfCkwFJjnzC/fP2f7bR5whYiIy1kq4/pnWUQSgGuB153Hggd9U88E6rhtY3bN2JhtY3ZN8lTG1Z+X22N2MBba8cDeMo+zqPoXwC0KfCwiq0XkHmdeK1Xd70wfAFr5OVNl7XvZZ/c7u4qmyHe7Zf2Wx9kt1Bvff9ye90+5POBR/zi72dYBOcAn+LbAHFPV4gra/DaP8/xxoIVbWVT1bN887fTNCyLSsHyWCnLWlr8CjwGlzuMWeNQ39UggjNs2ZteMjdmV5wEbsyvM4/G47eqYHYyFdqC4RFX7ANcA94nIj8o+qb79Ep5dEsbr9h0vA+2BXsB+4Dl/Ni4ijYH5wEOqmlf2OS/6p4I8nvWPqpaoai8gAd+Wly7+aru6LCLSHXjCydQPaA78yh9ZROQnQI6qrvZHe8avbMyuno3ZVeexMbuSPF6N2/4Ys4Ox0M4GEss8TnDm+ZWqZjvfc4AF+D74B8/uDnG+5/g5VmXte9JnqnrQ+WUsBV7ju11prucRkTB8A+SbqvqOM9uz/qkoj5f9c5aqHgM+Bwbi253XoII2v83jPB8D5LqYZZiz61ZVtRCYiv/6ZjBwvYhk4ju8YSjwNzzum3rA83Hbxuzq2ZhddR4bs6vM49W47fqYHYyF9iqgo3NGaTi+g9kX+jOAiESJSJOz08BVwEYnx+3OYrcD//RnriraXwiMd878HQAcL7M7zjXljsEaia+PzuYZ45z92xboCKysxXYFmAxsUdXnyzzlSf9UlsfD/okTkabOdCPgSnzHIH4OjHIWK98/Z/ttFLDY2brkVpatZf64Cr5j68r2jWs/K1V9QlUTVLUNvrFlsareigd9U894Om7bmF0zNmZXncfG7CrzeDJu+2XM1lo+c7MufOE7g/VrfMco/caD9tvhO8M4A9h0NgO+43w+A74BPgWau5hhFr5dV2fwHX90V2Xt4zvT90WnvzYAKX7K84bT3nrnw926zPK/cfJsA66p5SyX4NvFuB5Y53wN96p/qsjjVf/0BNY67W4Eflvmc70S34k8c4GGzvwI5/F25/l2fsiy2OmbjcBMvjvD3fXPcplsQ/juDHa/9019+8LDcRsbs2uax8bsqvME/ZhdTR5Px21cGrPtzpDGGGOMMca4IBgPHTHGGGOMMcZ1VmgbY4wxxhjjAiu0jTHGGGOMcYEV2sYYY4wxxrjACm1jjDHGGGNcYIW2McYYY4wxLrBC2xhjjDHGGBdYoW2MMcYYY4wL/h8o2FKypN/VXAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.recorder.plot_lr(show_moms=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

plot_metrics[source]

\n", "\n", "> plot_metrics()\n", "\n", "Plot metrics collected during training. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.plot_metrics)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that metrics are only collected at the end of each epoch, so you'll need to train at least two epochs to have anything to show here." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4VPX5/vH3Q0hYw74vYd8CRMQAAipKXXAXsK3WutRWbKvfLv6sgFtxK0itrVZbixYrbd0KiEEFRAVBRSWoJCRsYSfsIPua5Pn9MSd2pCgBJjOT5H5dVy7PnPM5M885THLPWebR3B0REZFKsS5ARETigwJBREQABYKIiAQUCCIiAigQREQkoEAQERFAgSAiIoESBYKZDTKzpWaWZ2YjjrG8lZm9a2ZZZjbbzFqELRtrZjlmttjMnjQzC+Zfa2bZwTrTzaxB5DZLRERO1HEDwcwSgKeBi4FU4FozSz1q2GPABHdPAx4ERgfr9gP6A2lAN6AXMMDMKgNPAOcF62QBt0dki0RE5KRULsGY3kCeu68EMLOXgSuB3LAxqcAdwfQsYEow7UBVIAkwIBHYHEwbUMPMtgO1gLzjFdKgQQNv3bp1CUoWEZFiCxYs2ObuDY83riSB0BxYF/Z4PdDnqDELgSGEPvUPBpLNrL67zzOzWcBGQgHwlLsvBjCznwHZwD5gOXDbsV7czIYBwwBSUlLIzMwsQckiIlLMzNaUZFykLirfSehU0OfAACAfKDSz9kAXoAWhYBloZmebWSLwM+B0oBmhU0Yjj/XE7j7O3dPdPb1hw+MGnIiInKSSHCHkAy3DHrcI5n3F3TcQOkLAzGoCQ919p5ndAnzs7nuDZdOAvsDBYL0VwfxXgf+5WC0iItFTkiOE+UAHM2tjZknANUBG+AAza2Bmxc81EhgfTK8luIgcHBUMABYTCpRUMyv+yH9BMF9ERGLkuEcI7l5gZrcDM4AEYLy755jZg0Cmu2cA5wKjzcyBOfz3esBEYCChawUOTHf3qQBm9gAwx8yOAGuAmyK5YSIicmKsLP3/ENLT010XlUVEToyZLXD39OON0zeVRUQEUCCIiEhAgSAiEseWbtrD2OlLiMbp/ZLcdioiIlF2uKCIv8zO4+lZeSRXTeSHZ7aiWZ1qpfqaCgQRkTizcN1O7pqYxdLNe7iyRzPuvyyV+jWrlPrrKhBEROLEgcOFPD5zKX//YBWNkqvy9xvT+U6XxlF7fQWCiEgc+GjFNkZMymbtjv38oE8KIy7uTK2qiVGtQYEgIhJDuw8eYfRbS3jp07W0ql+dl245k77t6sekFgWCiEiMvJO7mXumZLN1zyGGndOWX5/fkWpJCTGrR4EgIhJl2/ce4oGpuWQs3EDnJsmMuz6d01rWiXVZCgQRkWhxdzIWbmBURg57DxVwxwUd+emAdiRVjo+vhCkQRESiYMPOA9w7ZRHvLdlCj5Z1GHt1Gh0bJ8e6rK9RIIiIlKKiIuel+WsZ/dYSCouc+y5L5aZ+rUmoZLEu7X8oEERESsmqbfsYMSmLT1btoH/7+owenEZK/eqxLusbKRBERCKsoLCI8R+u4g9vLyOpciUeHdqd76W3xCz+jgrCKRBERCJo8cbdDJ+URdb6XVyQ2piHr+pG41pVY11WiSgQREQi4FBBIU+/l8dfZq+gTvVEnv5BTy7p3iTujwrCKRBERE7RZ2u/ZPjELJZv2cuQ05tz32Wp1K2RFOuyTpgCQUTkJO0/XMBjM5bx/EeraFqrKs//qBfndWoU67JOmgJBROQkfJi3jRGTs1i34wDXn9mKuwZ1IjnKzegiTYEgInICdh04wu/eXMwrmeto06AGrww7kz5tY9OMLtIUCCIiJfR2zibunbKI7fsO89MB7fjV+R2omhi7ZnSRpkAQETmOrXsOMWpqDm9mbaRL01r8/cZedG9RO9ZlRZwCQUTkG7g7r32ez4Nv5LL/UCG/uagTw85pS2JCfDSji7QSBYKZDQKeABKA59x9zFHLWwHjgYbADuCH7r4+WDYWuBSoBMwEfgnUBOaGPUUL4F/u/qtT2hoRkQjJ33mAe17LZvbSrfRMCTWja98ovprRRdpxA8HMEoCngQuA9cB8M8tw99ywYY8BE9z9BTMbCIwGrjezfkB/IC0Y9wEwwN1nAz3CXmMBMDkC2yMickqKipx/f7KGMdOW4MCoy1O5vm98NqOLtJIcIfQG8tx9JYCZvQxcCYQHQipwRzA9C5gSTDtQFUgCDEgENoc/uZl1BBrx9SMGEZGoW7l1LyMmZfPp6h2c3aEBvxvcnZb14rcZXaSVJBCaA+vCHq8H+hw1ZiEwhNBppcFAspnVd/d5ZjYL2EgoEJ5y98VHrXsN8Iq7+8lsgIjIqSooLOLZuav44zvLqFq5Er+/Oo2rz2hRptpOREKkLirfCTxlZjcBc4B8oNDM2gNdCF0jAJhpZme7e/jRwDXA9d/0xGY2DBgGkJKSEqFyRURCcjbsYvikLBbl72ZQ1yY8eFVXGiWXjWZ0kVaSQMgHWoY9bhHM+4q7byB0hICZ1QSGuvtOM7sF+Njd9wbLpgF9CU4PmdlpQGV3X/BNL+7u44BxAOnp6TqKEJGIOHikkD+/t5xn3l9J3epJ/PW6nlzcvWmsy4qpktw7NR/oYGZtzCyJ0Cf6jPABZtbAzIqfayShO44A1gIDzKyymSUCA4DwU0bXAi+dygaIiJyoBWt2cOmTc3l61gqu6tGcd+44p8KHAZTgCMHdC8zsdmAGodtOx7t7jpk9CGS6ewZwLjDazJzQKaPbgtUnAgOBbEIXmKe7+9Swp/8ecEmkNkZE5NvsO1TA72cs5YV5q2lWuxov3NybAR0bxrqsuGFl6Vpuenq6Z2ZmxroMESmD5izbysjJ2WzYdYAb+7bmzos6UbNKxfhurpktcPf0442rGHtDRCqsnfsP8/Cbi5m4YD1tG9bgP7f2Jb11vViXFZcUCCJSbk3L3sh9r+fw5f7D3HZeO/5vYPlqRhdpCgQRKXe27DnIb1/PYdqiTXRtVosXbu5F12blrxldpCkQRKTccHcmLljPw28u5sCRQu4a1Ilbzi6/zegiTYEgIuXCuh37ufu1bOYu30av1nUZMzSNdg1rxrqsMkWBICJlWlGRM2HeasbOWIoBD13Zlev6tKJSBWhGF2kKBBEps/K27GH4pGwWrPmSAR0b8sjgbrSoW3Ga0UWaAkFEypwjhUWMm7OSJ95ZTvUqCTz+vdMYfHrzCteMLtIUCCJSpizK38VdE7PI3bibS7s3ZdQVXWmYXCXWZZULCgQRKRMOHinkiXeXM27OSurVSOKZH57BoG5NYl1WuaJAEJG4N3/1DoZPzGLltn18L70F91ySSu3qibEuq9xRIIhI3Np7qICx05cwYd4aWtStxr9+3IezOjSIdVnllgJBROLSrKVbuGdyNht3H+Tm/m2486KOVE/Sn6zSpL0rInHly32HeeiNXCZ/nk/7RjWZ+NN+nNGqbqzLqhAUCCISF9ydt7I38duMRezcf4RfDGzPbQPbU6WymtFFiwJBRGJuy+6D3DtlEW/nbqZ789pMuLkPqc1qxbqsCkeBICIx4+78J3M9D72Zy+GCIkZe3Jkfn9WGympGFxMKBBGJibXb9zPytSw+zNtO7zb1eHRoGm0a1Ih1WRWaAkFEoqqwyPnHR6t5bMZSEioZD1/VjR/0TlEzujigQBCRqFm+eQ93Tcri87U7Oa9TQx4Z3J1mdarFuiwJKBBEpNQdLijimfdX8NR7edSoksCfvt+DK3s0UzO6OKNAEJFSlbV+J3dNzGLJpj1cflozfnt5Kg1qqhldPFIgiEipOHC4kD+9s4xn566kYXIVnr0hnQtSG8e6LPkWCgQRibiPV25nxKQsVm/fz7W9WzLyki7UqqpmdPFOgSAiEbPn4BHGTFvCvz9ZS0q96rz4kz70a69mdGVFib79YWaDzGypmeWZ2YhjLG9lZu+aWZaZzTazFmHLxppZjpktNrMnLbiKZGZJZjbOzJaZ2RIzGxq5zRKRaHtvyWYu/OMcXvp0LT85qw0zfnWOwqCMOe4RgpklAE8DFwDrgflmluHuuWHDHgMmuPsLZjYQGA1cb2b9gP5AWjDuA2AAMBu4B9ji7h3NrBJQL0LbJCJRtGPfYR6cmsOULzbQsXFN/nJdP05PUTO6sqgkp4x6A3nuvhLAzF4GrgTCAyEVuCOYngVMCaYdqAokAQYkApuDZTcDnQHcvQjYdtJbISJR5+5MzdrIqIwc9hw8wi+/04HbzmtPUmW1nSirSvIv1xxYF/Z4fTAv3EJgSDA9GEg2s/ruPo9QQGwMfma4+2IzqxOMfcjMPjOz/5jZMW8/MLNhZpZpZplbt24t4WaJSGnatOsgt0xYwC9e+pyWdasx9f/O4tcXdFQYlHGR+te7ExhgZp8TOiWUDxSaWXugC9CCUIgMNLOzCR2ZtAA+cveewDxCp53+h7uPc/d0d09v2LBhhMoVkZPh7rz06VouePx9Psjbyr2XdmHyz/vTuYk6k5YHJTlllA+0DHvcIpj3FXffQHCEYGY1gaHuvtPMbgE+dve9wbJpQF9C1xL2A5ODp/gP8ONT2A4RKWVrtu9jxKRs5q3cTt+29RkztDut6qsZXXlSkiOE+UAHM2tjZknANUBG+AAzaxBcGAYYCYwPptcSOnKobGaJhI4eFru7A1OBc4Nx3+Hr1yREJE4UFjnPzV3JRX+aw6L8XYwe0p0Xb+mjMCiHjnuE4O4FZnY7MANIAMa7e46ZPQhkunsGoT/so83MgTnAbcHqE4GBQDahC8zT3X1qsGw48E8z+xOwFfhR5DZLRCJh6aZQM7qF63ZyfpdGPHxVd5rUrhrrsqSUWOjDetmQnp7umZmZsS5DpNw7XFDE07Py+MvsPJKrJjLqiq5cntZUzejKKDNb4O7pxxunbyqLyNd8sW4nd01cyLLNe7mqRzPuv7wr9WokxbosiQIFgogAoWZ0f3h7KeM/XEXjWlUZf1M6AzurGV1FokAQET5asY0Rk7JZu2M/1/VJYcTFnUlWM7oKR4EgUoHtPniE0W8t5qVP19G6fnVeHnYmZ7atH+uyJEYUCCIV1Du5m7lnSjZb9xzi1nPa8qvzO1ItKSHWZUkMKRBEKphtew/xwNRcpi7cQOcmyTx7QzppLeocf0Up9xQIIhWEu/P6Fxt4YGoOew8VcMcFHfnpgHbqPyRfUSCIVAAbdh7g3imLeG/JFk5PqcPYoWl0aJwc67IkzigQRMqxoiLnxU/XMmbaEgqLnPsvS+XGfq1JqKQvmMn/UiCIlFOrtu1jxKQsPlm1g/7t6zN6cBop9avHuiyJYwoEkXKmoLCIv3+wisdnLiOpciXGDk3ju+kt1HZCjkuBIFKO5G7YzfBJWWTn7+LC1MY8dFU3GtdSMzopGQWCSDlwqKCQp97L46+zV1CneiJP/6Anl3RvoqMCOSEKBJEybsGaLxk+KYu8LXsZ0rM5912aSl01o5OToEAQKaP2Hy7g9zOW8o+PVtO0VlWe/1EvzuvUKNZlSRmmQBApgz5Yvo0Rk7NY/+UBbujbirsGdaZmFf06y6nRO0ikDNm1/wiPvJXLq5nradOgBq/e2pfeberFuiwpJxQIImXE9EWbuO/1RezYd5ifnduOX36nA1UT1YxOIkeBIBLntu45xKiMHN7M3khq01o8f1MvujWvHeuypBxSIIjEKXdn8mf5PPhGLgcOF/Kbizox7Jy2JCaoGZ2UDgWCSBzK33mAuydn8/6yrZzRqi6PDk2jfaOasS5LyjkFgkgcKSpy/vXJGh6dtgQHRl2eyg19W1NJzegkChQIInFixda9jJiUxfzVX3J2hwb8bnB3WtZTMzqJHgWCSIwdKSzi2bkr+dM7y6mWmMBj3z2NoT2bq+2ERF2JAsHMBgFPAAnAc+4+5qjlrYDxQENgB/BDd18fLBsLXApUAmYCv3R3N7PZQFPgQPA0F7r7llPeIpEyZFH+LoZPyiJnw24u7taEB67sSqNkNaOT2DhuIJhZAvA0cAGwHphvZhnunhs27DFggru/YGYDgdHA9WbWD+gPpAXjPgAGALODx9e5e2ZEtkSkDDl4pJA/v7ecZ95fSd3qSfz1up5c3L1prMuSCq4kRwi9gTx3XwlgZi8DVwLhgZAK3BFMzwKmBNMOVAWSAAMSgc2nXrZI2ZW5egd3Tcpi5dZ9XH1GC+69tAt1qqsZncReSQKhObAu7PF6oM9RYxYCQwidVhoMJJtZfXefZ2azgI2EAuEpd18ctt7zZlYITAIednc/ye0QiXv7DoWa0b0wbzXNaldjws29Oadjw1iXJfKVSF1UvhN4ysxuAuYA+UChmbUHugAtgnEzzexsd59L6HRRvpklEwqE64EJRz+xmQ0DhgGkpKREqFyR6Hp/2VbunpzNhl0HuLFva35zUSdqqBmdxJmSvCPzgZZhj1sE877i7hsIHSFgZjWBoe6+08xuAT52973BsmlAX2Cuu+cH6+4xsxcJnZr6n0Bw93HAOID09HQdQUiZsnP/YR56YzGTPltPu4Y1+M+tfUlvrWZ0Ep9K8h34+UAHM2tjZknANUBG+AAza2Bmxc81ktAdRwBrgQFmVtnMEgldUF4cPG4QrJsIXAYsOvXNEYkf07I3cv7jc5jyRT63n9eeN39xtsJA4tpxjxDcvcDMbgdmELrtdLy755jZg0Cmu2cA5wKjzcwJnTK6LVh9IjAQyCZ0gXm6u081sxrAjCAMEoB3gGcju2kisbFl90Hufz2H6Tmb6NqsFi/c3IuuzdSMTuKflaXruOnp6Z6ZqbtUJT65OxMXrOehN3I5WFDEr8/vyC1nt6GymtFJjJnZAndPP944XdUSiYB1O/Zz92vZzF2+jV6t6zJmaBrtGqoZnZQtCgSRU1BY5EyYt5rfz1iKAQ9d2ZXr+rRSMzopkxQIIicpb8sehk/KZsGaLxnQsSG/G9Kd5nWqxboskZOmQBA5QUcKi/jb+yt48t08qldJ4PHvncbg09WMTso+BYLICViUv4vfTMxi8cbdXJrWlFGXd6VhcpVYlyUSEQoEkRI4eKSQP72znGfnrqRejST+dv0ZXNS1SazLEokoBYLIcXyycjsjJmezats+vp/ekrsv6ULt6omxLksk4hQIIt9gz8EjjJ2+lH9+vIaW9arxrx/34awODWJdlkipUSCIHMOspVu4Z3I2G3cf5Ob+bbjzoo5UT9Kvi5RveoeLhPly32EeeiOXyZ/n06FRTSb9rB89U+rGuiyRqFAgiBBqO/Fm9kZ++3oOuw4c4RcD23PbwPZUqZwQ69JEokaBIBXe5t0HuXfKImbmbqZ789r86yd96NK0VqzLEok6BYJUWO7Oq5nrePjNxRwuKGLkxZ358VlqRicVlwJBKqS12/czYnIWH63YTp829RgzNI02DWrEuiyRmFIgSIVSWOT846PVPDZjKQmVjEcGd+PaXilqRieCAkEqkGWb93DXxCy+WLeTgZ0b8cjgbjStrWZ0IsUUCFLuHS4o4q+zV/DUrOXUrFKZJ67pwRWnNVMzOpGjKBCkXFu4bifDJ2WxZNMeLj+tGaMuT6V+TTWjEzkWBYKUSwcOF/LHd5bx3NyVNEyuwrM3pHNBauNYlyUS1xQIUu7MW7GdkZOzWL19P9f2TmHkJZ2pVVXN6ESOR4Eg5cbug0cYM20JL36yllb1q/PiLX3o107N6ERKSoEg5cJ7SzZz9+RFbNlzkFvObsMdF3SiWpLaToicCAWClGnb9x7iwTdyef2LDXRqnMwz159Bj5Z1Yl2WSJmkQJAyyd3JWLiBB6bmsufgEX51fgd+fm57kiqr7YTIyVIgSJmzcdcB7n1tEe8u2cJpLeswdmganZokx7oskTKvRB+nzGyQmS01szwzG3GM5a3M7F0zyzKz2WbWImzZWDPLMbPFZvakHfVtIDPLMLNFp74pUt4VFTkvfrKWCx+fw4crtnHvpV2Y/LN+CgORCDnuEYKZJQBPAxcA64H5Zpbh7rlhwx4DJrj7C2Y2EBgNXG9m/YD+QFow7gNgADA7eO4hwN4IbYuUY6u37WPE5Cw+XrmDvm3rM2Zod1rVVzM6kUgqySmj3kCeu68EMLOXgSuB8EBIBe4IpmcBU4JpB6oCSYABicDm4HlqBusMA149pa2QcqugsIjnP1zNH2YuJbFSJcYM6c73e7VU2wmRUlCSQGgOrAt7vB7oc9SYhcAQ4AlgMJBsZvXdfZ6ZzQI2EgqEp9x9cbDOQ8AfgP2nUL+UY0s27Wb4xCwWrt/F+V0a8fBV3WlSu2qsyxIptyJ1UflO4CkzuwmYA+QDhWbWHugCFF9TmGlmZwN7gHbu/msza/1tT2xmwwgdRZCSkhKhciWeHSoo5OlZK/jLrDxqV0vkz9eezmVpTXVUIFLKShII+UDLsMctgnlfcfcNhI4Qik8FDXX3nWZ2C/Cxu+8Nlk0D+hIKhHQzWx3U0MjMZrv7uUe/uLuPA8YBpKen+wltnZQ5n6/9kuGTsli2eS9X9WjG/Zd3pV6NpFiXJVIhlOQuo/lABzNrY2ZJwDVARvgAM2tgZsXPNRIYH0yvBQaYWWUzSyR0QXmxu//V3Zu5e2vgLGDZscJAKo79hwt46I1chvz1I/YcLGD8Ten86ZrTFQYiUXTcIwR3LzCz24EZQAIw3t1zzOxBINPdM4BzgdFm5oROGd0WrD4RGAhkE7rAPN3dp0Z+M6Qs+yhvGyMmZ7N2x35+eGYKwwd1JlnN6ESiztzLzlmY9PR0z8zMjHUZEiG7Dhxh9FuLeXn+OlrXr86YoWmc2bZ+rMsSKXfMbIG7px9vnL6pLDHxds4m7p2yiG17D3HrgLb8+vyOVE1UMzqRWFIgSFRt23uIURk5vJG1kc5NknnuxnTSWqgZnUg8UCBIVLg7U77I54Gpuew/VMj/u6Ajtw5op2Z0InFEgSClbsPOA9zzWjazlm7l9JRQM7oOjdV/SCTeKBCk1BQVOf/+dC2PTltCYZFz/2Wp3NivNQmV9AUzkXikQJBSsXLrXkZMzubTVTs4q30DRg/pTst61WNdloh8CwWCRFRBYRHPfbCKP85cRlLlSowdmsZ301uo7YRIGaBAkIjJ3bCbuyYtZFH+bi5MbcxDV3WjcS01oxMpKxQIcsoOFRTy1Ht5/HX2CupUT+Qv1/Xk4m5NdFQgUsYoEOSULFgTakaXt2UvQ3o2575LU6mr/kMiZZICQU7KvkMFPPb2Uv7x0Wqa1a7GP37Ui3M7NYp1WSJyChQIcsLmLt/KyMnZrP/yADf0bcVdgzpTs4reSiJlnX6LpcR27T/Cw2/m8p8F62nboAav3tqX3m3qxbosEYkQBYKUyPRFm7jv9UXs2HeYn5/bjl98p4Oa0YmUMwoE+VZb9hxkVEYOb2VvIrVpLZ6/qRfdmteOdVkiUgoUCHJM7s7kz/J58I1cDhwp5DcXdWLYOW1JTFAzOpHySoEg/2P9l/u5+7VFzFm2lTNa1eXRoWm0b1Qz1mWJSClTIMhXioqcf368hkenLwHggSu6cv2ZraikZnQiFYICQQBYsXUvwydmkbnmS87u0IDfDVYzOpGKRoFQwR0pLGLcnJU88e5yqiUm8Nh3T2Noz+ZqOyFSASkQKrBF+bsYPimLnA27uaR7E0Zd0ZVGyWpGJ1JRKRAqoINHCnny3eX8bc5K6lZP4pkf9mRQt6axLktEYkyBUMHMX72D4ROzWLltH989owX3XppK7eqJsS5LROKAAqGC2HuogLHTlzBh3hqa16nGhJt7c07HhrEuS0TiSIkCwcwGAU8ACcBz7j7mqOWtgPFAQ2AH8EN3Xx8sGwtcClQCZgK/dHc3s+lA06CGucBt7l4Yka2Sr3l/2VbunpzNhl0HuKlfa35zUSdqqBmdiBzluF87NbME4GngYiAVuNbMUo8a9hgwwd3TgAeB0cG6/YD+QBrQDegFDAjW+Z67nxbMbwh895S3Rr5m5/7D3PHqF9w4/lOqJlZi4k/7MuqKrgoDETmmkvxl6A3kuftKADN7GbgSyA0bkwrcEUzPAqYE0w5UBZIAAxKBzQDuvjushqRgrETIW9kbuf/1Rezcf4Tbz2vP7QPbqxmdiHyrkjSmaQ6sC3u8PpgXbiEwJJgeDCSbWX13n0coIDYGPzPcfXHxSmY2A9gC7AEmntQWyNds2X2QW/+Zyc///RlNalfl9dv7c+dFnRQGInJckepUdicwwMw+J3RKKB8oNLP2QBegBaEQGWhmZxev5O4XEbqOUAUYeKwnNrNhZpZpZplbt26NULnlj7vzauY6zn/8fWYt3crwQZ2Z8vP+dG2mzqQiUjIlOWWUD7QMe9wimPcVd99AcIRgZjWBoe6+08xuAT52973BsmlAX0IXkYvXPWhmrxM6DTXz6Bd393HAOID09HSdVjqGdTv2M3JyNh/kbaN363qMGdqdtg3VjE5ETkxJjhDmAx3MrI2ZJQHXABnhA8ysgZkVP9dIQnccAawldORQ2cwSCR09LDazmmbWNFi3MqG7kJac+uZULIVFzvMfruLCP87h87Vf8tBV3Xh52JkKAxE5Kcc9QnD3AjO7HZhB6LbT8e6eY2YPApnungGcC4w2MwfmALcFq08kdCoom9BF4+nuPtXMGgMZZlaFUCjNAp6J7KaVb3lb9nDXxCw+W7uTczs15JHB3Wlep1qsyxKRMszcy85ZmPT0dM/MzIx1GTF1pLCIZ2av4M/v5VG9SgK/vTyVq3qoGZ2IfDMzW+Du6ccbpxvSy5Ds9bv4zcSFLNm0h0vTmvLAFV1pULNKrMsSkXJCgVAGHDxSyB/fWcazc1bSoGYV/nb9GVzUtUmsyxKRckaBEOc+WbmdEZOzWbVtH99Pb8ndl3ahdjU1oxORyFMgxKk9B4/w6PQl/OvjtbSsV41//6QP/ds3iHVZIlKOKRDi0KwlW7jntWw27j7Ij89qw/+7sCPVk/RPJSKlS39l4siOfYd56I1cXvs8nw6NajLpZ/3omVI31mWJSAWhQIgD7s4bWRsZlZHDrgNH+MV3OnDbee2oUln9h0QkehQIMbZ590HueW0R7yzeTFqL2vzrJ33o0rRWrMsSkQpIgRAj7s4r89fxyFuLOVy2vXzBAAAK30lEQVRQxN2XdObm/m2onBCpfoMiIidGgRADa7fvZ8TkLD5asZ0+berx6NA0WjeoEeuyRKSCUyBEUXEzusfeXkrlSpX43eDuXNOrJZUqqe2EiMSeAiFKlm7aw/BJWXyxbicDOzfikcHdaFpbzehEJH4oEErZ4YIi/jI7j6dn5ZFcNZEnrunBFac1UzM6EYk7CoRStHDdTu6amMXSzXu44rRm/PbyVOqrGZ2IxCkFQik4cLiQx2cu5e8frKJRclWeuyGd81Mbx7osEZFvpUCIsHkrtjNichZrtu/nB31SGHFxZ2pVVTM6EYl/CoQI2X3wCKPfWsJLn66lVf3qvHhLH/q1UzM6ESk7FAgR8E7uZu6Zks3WPYcYdk5bfn1+R6olqe2EiJQtCoRTsH3vIR6YmkvGwg10apzM365Pp0fLOrEuS0TkpCgQToK7k7FwA6Mycth7qIBfn9+Rn53bjqTKajshImWXAuEEbdx1gHtfW8S7S7bQo2Udxl6dRsfGybEuS0TklCkQSqioyHlp/lpGv7WEgqIi7r20Cz/q34YEtZ0QkXJCgVACq7ftY8TkLD5euYN+7eozZkgaKfWrx7osEZGIUiB8i4LCIsZ/uIo/vL2MpIRKjBnSne/3aqm2EyJSLikQvsHijbsZPimLrPW7OL9LYx6+qhtNaleNdVkiIqWmRLfFmNkgM1tqZnlmNuIYy1uZ2btmlmVms82sRdiysWaWY2aLzexJC6luZm+a2ZJg2ZhIbtSpOFRQyOMzl3H5nz8g/8sD/Pna03n2hjMUBiJS7h33CMHMEoCngQuA9cB8M8tw99ywYY8BE9z9BTMbCIwGrjezfkB/IC0Y9wEwAPgUeMzdZ5lZEvCumV3s7tMitmUn4bO1XzJ8YhbLt+xl8OnNue+yVOrVSIplSSIiUVOSU0a9gTx3XwlgZi8DVwLhgZAK3BFMzwKmBNMOVAWSAAMSgc3uvj8Yh7sfNrPPgBbEyP7DBfzh7WWM/3AVTWpV5fmbenFe50axKkdEJCZKEgjNgXVhj9cDfY4asxAYAjwBDAaSzay+u88zs1nARkKB8JS7Lw5f0czqAJcH60bdh3nbGDE5i3U7DvDDM1MYPqgzyWpGJyIVUKQuKt8JPGVmNwFzgHyg0MzaA13476f/mWZ2trvPBTCzysBLwJPFRyBHM7NhwDCAlJSUCJULuw4c4XdvLuaVzHW0aVCDV4adSZ+29SP2/CIiZU1JAiEfaBn2uEUw7yvuvoHQEQJmVhMY6u47zewW4GN33xssmwb0BeYGq44Dlrv7n77pxd19XDCO9PR0L8lGHc/bOZu4d8oitu09xK0DQs3oqiaqGZ2IVGwluctoPtDBzNoEF4CvATLCB5hZAzMrfq6RwPhgei0wwMwqm1kioQvKi4N1HgZqA7869c0oma17DnHbi58x7J8LqFcjiSm39WfkxV0UBiIilOAIwd0LzOx2YAaQAIx39xwzexDIdPcM4FxgtJk5oVNGtwWrTwQGAtmELjBPd/epwW2p9wBLgM+CL3o95e7PRXTr/rsNTPkinwem5rL/UCF3XtiRWwe0IzFBzehERIqZe0TOwkRFenq6Z2ZmntA6RwqLGDYhk1lLt9IzJdSMrn0jNaMTkYrDzBa4e/rxxpX7byonJlSibcOanNOxITf0ba1mdCIi36DcBwLAfZelxroEEZG4p5PoIiICKBBERCSgQBAREUCBICIiAQWCiIgACgQREQkoEEREBFAgiIhIoEy1rjCzrcCaUnyJBsC2Unz+UxXP9am2k6PaTl481xdvtbVy94bHG1SmAqG0mVlmSfp9xEo816faTo5qO3nxXF881/ZtdMpIREQABYKIiAQUCF83LtYFHEc816faTo5qO3nxXF881/aNdA1BREQAHSGIiEigQgWCmbU0s1lmlmtmOWb2y2B+PTObaWbLg//WDeabmT1pZnlmlmVmPWNQ2+/NbEnw+q+ZWZ1gfmszO2BmXwQ/z8SgtlFmlh9WwyVh64wM9ttSM7soBrW9ElbXajP7Ipgftf0WvF5VM/vUzBYG9T0QzG9jZp8E++iV4P9XjplVCR7nBctbx6C2fwf/bovMbHzw/0PHzM41s11h++7+GNT2DzNbFVZDj2B+NH9Xv6m2uWF1bTCzKcH8qO23U+buFeYHaAr0DKaTgWVAKjAWGBHMHwE8GkxfAkwDDDgT+CQGtV0IVA7mPxpWW2tgUYz32yjgzmOMTwUWAlWANsAKICGatR015g/A/dHeb8HrGVAzmE4EPgneS68C1wTznwF+Fkz/HHgmmL4GeCUGtV0SLDPgpbDazgXeiPF++wdw9THGR/N39Zi1HTVmEnBDtPfbqf5UqCMEd9/o7p8F03uAxUBz4ErghWDYC8BVwfSVwAQP+RioY2ZNo1mbu7/t7gXBsI+BFqXx+idT27esciXwsrsfcvdVQB7QOxa1mZkB3yP0hy3qgvfO3uBhYvDjwEBgYjD/6Pdc8XtxIvCdYBuiVpu7vxUsc+BTYvOe+6b99k2i+bv6rbWZWS1C/75TSuP1S1OFCoRwwaH46YTSvbG7bwwWbQIaB9PNgXVhq63n2/8QlkZt4W4m9CmoWBsz+9zM3jezs0u7rm+o7fbgEH28BafaiK/9djaw2d2Xh82L6n4zs4TglNUWYCahI6adYUEfvn++2nfB8l1A/WjV5u6fhC1LBK4Hpoet0jc4VTLNzLqWVl3Hqe2R4D33RzOrEsyL6nvu2/YboXB/1913h82L2n47FRUyEMysJqFDul8d9Y9G8KkoZrdefVNtZnYPUAD8O5i1EUhx99OBO4AXg08m0aztr0A7oEdQzx9K8/VPsLZi1/L1o4Oo7zd3L3T3HoQ+afcGOpfm652Io2szs25hi/8CzHH3ucHjzwi1QDgN+DOl/An4G2obSWj/9QLqAcNLs4YTrK3Y0e+5qO63U1HhAiH41DMJ+Le7Tw5mby4+vAz+uyWYnw+0DFu9RTAvmrVhZjcBlwHXBYFFcDpmezC9gNCnzo7RrM3dNwe/GEXAs/z3tFC87LfKwBDgleJ50d5v4dx9JzAL6EvolEblYFH4/vlq3wXLawPbo1jboOC1fws0JBSaxWN2F58qcfe3gEQzaxDN2oJThO7uh4DnidF77li1AQT7ozfwZtiYmOy3k1GhAiE4F/t3YLG7Px62KAO4MZi+EXg9bP4NwR0MZwK7wk4tRaU2MxsE3AVc4e77w+Y3NLOEYLot0AFYGeXaws/RDgYWBdMZwDUWumOmTVDbp9GsLXA+sMTd14eNj9p+C3u94jvDqgEXELrOMQu4Ohh29Huu+L14NfBe8YeAKNW2xMx+AlwEXBuEffH4JsXXM8ysN6G/H6USVt9SW/EHNyN0aib8PRet39Vj1hYsvprQBeSDYeOjtt9OmUf5KnYsf4CzCJ0OygK+CH4uIXSO9l1gOfAOUM//ezfB04Q+RWYD6TGoLY/QudHiecV3oAwFcoJ5nwGXx6C2fwb7JYvQL2TTsHXuCfbbUuDiaNcWLPsH8NOjxkdtvwWvlwZ8HtS3iP/e7dSWUEjmAf8BqgTzqwaP84LlbWNQW0Hwb1e8P4vn3x7su4WEbnDoF4Pa3gvec4uAf/Hfu32i+bt6zNqCZbMJHcmEj4/afjvVH31TWUREgAp2ykhERL6ZAkFERAAFgoiIBBQIIiICKBBERCSgQBAREUCBICIiAQWCiIgA8P8BqX9Iu2aZ+RQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.recorder.plot_metrics()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Callback methods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You don't call these yourself - they're called by fastai's [`callback`](/callback.html#callback) system automatically to enable the class's functionality." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_backward_begin[source]

\n", "\n", "> on_backward_begin(`smooth_loss`:`Tensor`, `kwargs`:`Any`)\n", "\n", "Record the loss before any other callback has a chance to modify it. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.on_backward_begin)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_batch_begin[source]

\n", "\n", "> on_batch_begin(`train`, `kwargs`:`Any`)\n", "\n", "Record learning rate and momentum at beginning of batch. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.on_batch_begin)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_epoch_end[source]

\n", "\n", "> on_epoch_end(`epoch`:`int`, `num_batch`:`int`, `smooth_loss`:`Tensor`, `last_metrics`=`'Collection'`, `kwargs`:`Any`) → `bool`\n", "\n", "Save epoch info: num_batch, smooth_loss, metrics. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.on_epoch_end)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_train_begin[source]

\n", "\n", "> on_train_begin(`pbar`:`PBar`, `metrics_names`:`StrList`, `kwargs`:`Any`)\n", "\n", "Initialize recording status at beginning of training. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.on_train_begin)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Module functions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Generally you'll want to use a [`Learner`](/basic_train.html#Learner) to train your model, since they provide a lot of functionality and make things easier. However, for ultimate flexibility, you can call the same underlying functions that [`Learner`](/basic_train.html#Learner) calls behind the scenes:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

fit[source]

\n", "\n", "> fit(`epochs`:`int`, `model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `loss_func`:`LossFunction`, `opt`:[`Optimizer`](https://pytorch.org/docs/stable/optim.html#torch.optim.Optimizer), `data`:[`DataBunch`](/basic_data.html#DataBunch), `callbacks`:`Optional`\\[`Collection`\\[[`Callback`](/callback.html#Callback)\\]\\]=`None`, `metrics`:`OptMetrics`=`None`)\n", "\n", "Fit the `model` on `data` and learn using `loss` and `opt`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(fit)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that you have to create the `Optimizer` yourself if you call this function, whereas [`Learn.fit`](/basic_train.html#fit) creates it for you automatically." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

train_epoch[source]

\n", "\n", "> train_epoch(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `dl`:[`DataLoader`](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader), `opt`:[`Optimizer`](https://pytorch.org/docs/stable/optim.html#torch.optim.Optimizer), `loss_func`:`LossFunction`)\n", "\n", "Simple training of `model` for 1 epoch of `dl` using optim `opt` and loss function `loss_func`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(train_epoch)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You won't generally need to call this yourself - it's what [`fit`](/basic_train.html#fit) calls for each epoch." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

validate[source]

\n", "\n", "> validate(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `dl`:[`DataLoader`](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader), `loss_func`:`OptLossFunc`=`None`, `cb_handler`:`Optional`\\[[`CallbackHandler`](/callback.html#CallbackHandler)\\]=`None`, `pbar`:`Union`\\[`MasterBar`, `ProgressBar`, `NoneType`\\]=`None`, `average`=`True`, `n_batch`:`Optional`\\[`int`\\]=`None`) → `Iterator`\\[`Tuple`\\[`IntOrTensor`, `Ellipsis`\\]\\]\n", "\n", "Calculate loss and metrics for the validation set. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(validate)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is what [`fit`](/basic_train.html#fit) calls after each epoch. You can call it if you want to run inference on a [`DataLoader`](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader) manually." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

get_preds[source]

\n", "\n", "> get_preds(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `dl`:[`DataLoader`](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader), `pbar`:`Union`\\[`MasterBar`, `ProgressBar`, `NoneType`\\]=`None`, `cb_handler`:`Optional`\\[[`CallbackHandler`](/callback.html#CallbackHandler)\\]=`None`, `activ`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)=`None`, `loss_func`:`OptLossFunc`=`None`, `n_batch`:`Optional`\\[`int`\\]=`None`) → `List`\\[`Tensor`\\]\n", "\n", "Tuple of predictions and targets, and optional losses (if `loss_func`) using `dl`, max batches `n_batch`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(get_preds)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

loss_batch[source]

\n", "\n", "> loss_batch(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `xb`:`Tensor`, `yb`:`Tensor`, `loss_func`:`OptLossFunc`=`None`, `opt`:`OptOptimizer`=`None`, `cb_handler`:`Optional`\\[[`CallbackHandler`](/callback.html#CallbackHandler)\\]=`None`) → `Tuple`\\[`Union`\\[`Tensor`, `int`, `float`, `str`\\]\\]\n", "\n", "Calculate loss and metrics for a batch, call out to callbacks as necessary. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(loss_batch)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You won't generally need to call this yourself - it's what [`fit`](/basic_train.html#fit) and [`validate`](/basic_train.html#validate) call for each batch. It only does a backward pass if you set `opt`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Other classes" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

class LearnerCallback[source]

\n", "\n", "> LearnerCallback(`learn`:[`Learner`](/basic_train.html#Learner)) :: [`Callback`](/callback.html#Callback)\n", "\n", "Base class for creating callbacks for a [`Learner`](/basic_train.html#Learner). " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(LearnerCallback, title_level=3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Undocumented Methods - Methods moved below this line will intentionally be hidden" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

_tta_only[source]

\n", "\n", "> _tta_only(`learn`:[`Learner`](/basic_train.html#Learner), `ds_type`:[`DatasetType`](/basic_data.html#DatasetType)=``, `scale`:`float`=`1.35`) → `Iterator`\\[`List`\\[`Tensor`\\]\\]\n", "\n", "Computes the outputs for several augmented inputs for TTA " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.tta_only)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

get_preds[source]

\n", "\n", "> get_preds(`ds_type`:[`DatasetType`](/basic_data.html#DatasetType)=``, `with_loss`:`bool`=`False`, `n_batch`:`Optional`\\[`int`\\]=`None`, `pbar`:`Union`\\[`MasterBar`, `ProgressBar`, `NoneType`\\]=`None`) → `List`\\[`Tensor`\\]\n", "\n", "Return predictions and targets on the valid, train, or test set, depending on `ds_type`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.get_preds)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

_TTA[source]

\n", "\n", "> _TTA(`learn`:[`Learner`](/basic_train.html#Learner), `beta`:`float`=`0.4`, `scale`:`float`=`1.35`, `ds_type`:[`DatasetType`](/basic_data.html#DatasetType)=``, `with_loss`:`bool`=`False`) → `Tensors`" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.TTA)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

format_stats[source]

\n", "\n", "> format_stats(`stats`:`MetricsList`)\n", "\n", "Format stats before printing. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.format_stats)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

add_metrics[source]

\n", "\n", "> add_metrics(`metrics`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.add_metrics)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

add_metric_names[source]

\n", "\n", "> add_metric_names(`names`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Recorder.add_metric_names)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## New Methods - Please document or move to the undocumented section" ] } ], "metadata": { "jekyll": { "keywords": "fastai", "summary": "Learner class and training loop", "title": "basic_train" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 2 }