{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## List of callbacks" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "hide_input": true }, "outputs": [], "source": [ "from fastai.gen_doc.nbdoc import *\n", "from fastai.vision import *\n", "from fastai.text import *\n", "from fastai.callbacks import * \n", "from fastai.basic_train import * \n", "from fastai.train import * \n", "from fastai import callbacks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "fastai's training loop is highly extensible, with a rich *callback* system. See the [`callback`](/callback.html#callback) docs if you're interested in writing your own callback. See below for a list of callbacks that are provided with fastai, grouped by the module they're defined in.\n", "\n", "Every callback that is passed to [`Learner`](/basic_train.html#Learner) with the `callback_fns` parameter will be automatically stored as an attribute. The attribute name is snake-cased, so for instance [`ActivationStats`](/callbacks.hooks.html#ActivationStats) will appear as `learn.activation_stats` (assuming your object is named `learn`)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [`Callback`](/callback.html#Callback)\n", "\n", "This sub-package contains more sophisticated callbacks that each are in their own module. They are (click the link for more details):" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`LRFinder`](/callbacks.lr_finder.html#LRFinder)\n", "\n", "Use Leslie Smith's [learning rate finder](https://www.jeremyjordan.me/nn-learning-rate/) to find a good learning rate for training your model. Let's see an example of use on the MNIST dataset with a simple CNN." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "path = untar_data(URLs.MNIST_SAMPLE)\n", "data = ImageDataBunch.from_folder(path)\n", "def simple_learner(): return Learner(data, simple_cnn((3,16,16,2)), metrics=[accuracy])\n", "learn = simple_learner()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The fastai librairy already has a Learner method called [`lr_find`](/train.html#lr_find) that uses [`LRFinder`](/callbacks.lr_finder.html#LRFinder) to plot the loss as a function of the learning rate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n" ] } ], "source": [ "learn.lr_find()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8nGW99/HPb7bszdKka9KmK7SF0iVUSqWUrVRUFhEsuICoeFTk0XP0eenR53gEj8cjetRzxAU5IC6IiKAgetgsUKDQpkApbelC17TQJk2bNs06M9fzx9wt0zTJpO1MJpn5vl+veXXmmuue+5fpZL657uW6zTmHiIhIb3zpLkBERAY+hYWIiCSksBARkYQUFiIikpDCQkREElJYiIhIQgoLERFJSGEhIiIJKSxERCShQLoLSJby8nJXXV2d7jJERAaVlStXNjjnKhL1S2lYmNki4EeAH7jTOfedLs//ADjPe5gPDHPOlXjPXQd83XvuW865e3pbV3V1NbW1tcksX0Qk45nZtr70S1lYmJkfuB24CKgDVpjZw865tYf7OOe+GNf/88BM734Z8A2gBnDASm/ZfamqV0REepbKfRZzgE3Ouc3OuQ7gPuCyXvpfA/zOu38x8IRzrtELiCeARSmsVUREepHKsBgN7Ih7XOe1HcPMxgLjgL8f77IiIpJ6qQwL66atp/nQFwMPOOcix7Osmd1oZrVmVltfX3+CZYqISCKpDIs6oCrucSWwq4e+i3lnE1Sfl3XO3eGcq3HO1VRUJNyZLyIiJyiVYbECmGRm48wsRCwQHu7aycxOAUqBZXHNjwELzazUzEqBhV6biIikQcqOhnLOhc3sJmJf8n7gLufcGjO7Bah1zh0OjmuA+1zcJfucc41mdiuxwAG4xTnXmKpaRUSkd5Ypl1WtqalxmXSeRVtnhDfrm9m0p5mtDS1EncPvM3wGoYCPwpwghbkBinIC+HyGcw7nIOocnRFHZyRKZyQKQG7QT17QT07Qh88MA8wMvw+Cft+RW07AR37IT17IT27Aj8/X3a6j4xONOjoiUdo6I7SHYzUF/T4CPiPg9xGJOto6I7R2Rmg7covS2hEh4hw5AR85AT+hgI+i3ABDcoMMyQuQF/RjdvL1iWQ7M1vpnKtJ1C9jzuA+Uc45DrSGGZIXSPjl45xjX0sn9QfbqT/YTkNzO42HOmjpCNPcHuFQe5h9LR3sbe5g76F2mtvClBflMGJILiOKcwn5fTS1dtLU2smBtk5aO6O0e1+iHeEokagj6mK3xkMdRNOc4yG/j5ygj9ygn5DfR8BvsS95n49wNEpHJEp7Z6xuv89iIeA3whFHa2eElo4wbZ3RlNU2bEgOI4tzGT4kl5L8ICF/LFRCXugVhPwU5MSC5XBtQb+P3KCPvGDgSDDG/7fnBv0U5ST+LIhkm6wPi6bWTmbc8gQhv4+hhSEqinIozQ+R631J5gR87G/pZHtjCzsaWzjUEen2dUJ+H/k5fkrzQwwtCDGuvICCnAANzR1s3XuIZZv30hmJUpIXojgvSHFekJK8IDlFOeR6X2Z+H/h9hplRUZjDpOGFTBpWRHV5PiHvr/Cog/ZwhEPtEZrbOznYFibqHOaNGHwW+0IMBWL/Ogdt4QitHbG/2A+PJB0Qib4zAmkPx25tnRFaOmL9Dz9uD8fuR6KOcNQRjkQJ+GJfyodDJPZajkg0is+MvJDf+zIOxN7LgJ/coJ+AzwjHrTfgs9jIJ+QnJxD7Ny/oJ9cbBR0OpLZwLIwPtIY50NbJvpYOdje18faBNl7f2cSBtjAd4ViAdYRPLqBCAR/lBSHKi3IozgseGc0MyQtSXpDD0MIQQwtzKMkLUpQboMh7PifgP6n1igxkWR8Wfp/x9fdOoaG548hoYV9Lx5EvqLbOCEW5QcaW5XPW+KFUleUzfEgOFYU5lBflUJYfoiAnQCiQ+jkZA/7YX7uxTTJBIDfl6xyMolHnhUssYFo6IoSjUTojsaBrC0dp7Yi1t3ZGiN8S29oRoeFQOw0HO2hobqeptZNd+1s50BamqaWTjkjPQVRRlEP10HzGDi2gsjSPoQUhSvJDlBWEGF2SR2VpHgG/5u6UwSnrw6IoN8gnzxmf7jIkiXw+Iz8UID8UoKIoJ2mv65yjuT3M3uZYkOxv6eSgN7rb39JJ3b4Wtu5tYenGenYfaD9m+ZDfR3V5PhOHFTJtVDFnVJZwemUxxXnBpNUokipZHxYifWVmFOUGKcoNUl1e0GvfzkiU/S2dR/Zh7djXwpv1zby55xCv7zzAX1e/faTv9MpiPj6vmvdNH0VQIw8ZoHQ0lEga7G/p4LW6Jlbt2M+fV+1i055mRgzJ5bqzq/nwWWMYkqvRhvSPvh4NpbAQSbNo1PHMhnrufG4zz2/ay5DcAJ86ZzzXz6v29k2JpI7CQmQQen1nEz98cgNPrttDSX6QG+eP57q51RTkaIuxpIbCQmQQW7VjPz98cgNL1tdTVhDiH84dz0fPqiYvpMNzJbkUFiIZ4OXt+/jBExtYurGB8sIcvvqeU7lydmW6y5IM0tew0KEXIgPYrDGl/PoT7+L+T89l7NB8/ukPq7jtsTfIlD/yZPBQWIgMAnPGlXHfjWdxzZwqbl/yJl/4/au0h7ufTUAkFbTXTGSQCPp9fPuK06kqy+e7/7uet5ra+MXHanRSn/QLjSxEBhEz47MLJvKjxTN4Zfs+rr97Oc3t4XSXJVlAYSEyCF02YzQ/vnYWr9U18YlfrqC1hwkuRZJFYSEySF08bQT/efUZLN/ayI2/rtU+DEkphYXIIHbZjNH8xwems3RjAzf/7hWi6b4IimQshYXIIHf1mVV87ZIpPLZmN795aVu6y5EMpbAQyQCfPGcc8ydX8O2/rmNzfXO6y5EMpLAQyQBmxnevnE7I7+Of/rCKcC8XaRI5EQoLkQwxojiXWy8/jVe27+fnz25OdzmSYRQWIhnk0jNG8d7TR/LDJzewZldTusuRDKKwEMkgZsatl5/GkNwg3/nbG+kuRzKIwkIkw5QVhLjh3eNYurGBtbsOpLscyRAKC5EM9JF3jSU/5OfOpdp3IcmhsBDJQMX5QT50ZhUPr9rFW02t6S5HMoDCQiRD3TBvHA64+/mt6S5FMoDCQiRDVZXlc8npI7n3pe0caOtMdzkyyCksRDLYjeeMp7k9zH3Lt6e7FBnkFBYiGez0ymLmjh/KXc9tpSOss7rlxCksRDLcp+aP4+0DbTy25u10lyKDmMJCJMMtmDyMMWX5/OZFzUgrJ05hIZLhfD7j2neN4aUtjWzYfTDd5cggpbAQyQJXza4k5PfxW40u5AQpLESywNDCHN47fSQPvryTQ+3hdJcjg1BKw8LMFpnZejPbZGZf6aHP1Wa21szWmNm9ce0RM3vVuz2cyjpFssFHzhrDwfYwD6/ale5SZBAKpOqFzcwP3A5cBNQBK8zsYefc2rg+k4CvAvOcc/vMbFjcS7Q652akqj6RbDNrTCmnjiji18u2sfjMKsws3SXJIJLKkcUcYJNzbrNzrgO4D7isS59PAbc75/YBOOf2pLAekaxmZnzkrLGsfesAr+zYn+5yZJBJZViMBnbEPa7z2uJNBiab2fNm9qKZLYp7LtfMar32y1NYp0jWuHzmaApzAjqMVo5bKsOiuzGu6/I4AEwCFgDXAHeaWYn33BjnXA1wLfBDM5twzArMbvQCpba+vj55lYtkqMKcAJfOGMXfVr9NS4d2dEvfpTIs6oCquMeVQNc9a3XAn51znc65LcB6YuGBc26X9+9m4GlgZtcVOOfucM7VOOdqKioqkv8TiGSg908fRWtnhL+/oa2+0nepDIsVwCQzG2dmIWAx0PWopj8B5wGYWTmxzVKbzazUzHLi2ucBaxGRkzZnXBkVRTk8+tpb6S5FBpGUhYVzLgzcBDwGrAPud86tMbNbzOxSr9tjwF4zWwssAb7snNsLTAFqzWyV1/6d+KOoROTE+X3GJaeN4O9v7KFZ51xIH5lzXXcjDE41NTWutrY23WWIDAortjZy1c+W8aPFM7hsRtfjTiSbmNlKb/9wr3QGt0gWmj2mlBFDcnlklTZFSd8oLESykM9nXHL6SJ7dUK+r6EmfKCxEstT7zhhJRyTKE2t2p7sUGQQUFiJZamZVCaNL8vjLa5orShJTWIhkKTPjvdNHsnRjA/tbOtJdjgxwCguRLPbe00cSjjoe16YoSUBhIZLFplcWM7okT9fnloQUFiJZzMxYOG04Szc16KJI0iuFhUiWWzh1BB3hKM9u0GSc0jOFhUiWO7O6lJL8II+v1X4L6ZnCQiTLBfw+Ljh1OE+t201nJJrucmSAUliICAunDedAW5jlWxrTXYoMUAoLEWH+pApygz4e11FR0gOFhYiQF/Izf1IFj6/dTabMRC3JpbAQEQAWThvBW01tvL7zQLpLkQFIYSEiAFxw6jB8Bo+v1aYoOZbCQkQAKC0IMWdcmc7mlm4pLETkiIVTR7BhdzPb9h5KdykywCgsROSIC6cMB+DJdXvSXIkMNAoLETlizNB8Jg0r5Kl1OptbjqawEJGjXDBlOMu3NOpyq3IUhYWIHOXCKcMIRx3PrNfEgvIOhYWIHGXmmFLKCkLaFCVHUViIyFH8PmPBKRUsWV9PWBMLikdhISLHuHDKcJpaO1m5bV+6S5EBQmEhIsc4Z1I5Qb/x1Bs6hFZiFBYicoyi3CBnjR/Kk9pvIR6FhYh064JTh7G5/hBbGnQ2tygsRKQHF3hnc+uoKAGFhYj0oKosn1NHFPHQKzt1jQtRWIhIzz4+r5o1uw6wdGNDukuRNFNYiEiPrphZyYghudy+ZFO6S5E0U1iISI9CAR+fmj+el7Y0snJbY7rLkTRSWIhIr66ZU0VpfpCfLHkz3aVIGiksRKRX+aEAN8wbx1Nv7GHtLl2fO1ulNCzMbJGZrTezTWb2lR76XG1ma81sjZndG9d+nZlt9G7XpbJOEendx+ZWU5gT4KfPaHSRrVIWFmbmB24H3gNMBa4xs6ld+kwCvgrMc85NA77gtZcB3wDeBcwBvmFmpamqVUR6V5wf5MNnjeHR13bpJL0slcqRxRxgk3Nus3OuA7gPuKxLn08Btzvn9gE45w5PRHMx8IRzrtF77glgUQprFZEEPvHucQT8Pu54dnO6S5E0SGVYjAZ2xD2u89riTQYmm9nzZvaimS06jmVFpB8NK8rlqtmV/HFlHXsOtKW7HOlnqQwL66at62mgAWASsAC4BrjTzEr6uCxmdqOZ1ZpZbX29ruolkmqfnj+BcDTK/zy3Jd2lSD9LZVjUAVVxjyuBXd30+bNzrtM5twVYTyw8+rIszrk7nHM1zrmaioqKpBYvIscaMzSf900fxW9e3EZTi67RnU1SGRYrgElmNs7MQsBi4OEuff4EnAdgZuXENkttBh4DFppZqbdje6HXJiJp9g/nTuBQR4TfvLQt3aVIP0pZWDjnwsBNxL7k1wH3O+fWmNktZnap1+0xYK+ZrQWWAF92zu11zjUCtxILnBXALV6biKTZ1FFDWHBKBXc9t4W2zki6y5F+Ypkym2RNTY2rra1NdxkiWWH5lkau/vkybr1sGh+dW53ucuQkmNlK51xNon46g1tEjtuZ1aXMHlvKHUs3a/ryLKGwEJHjZmYsPrOKHY2trNEUIFmhT2FhZhPMLMe7v8DMbvYOcRWRLHXeqcMwQ9fpzhJ9HVn8EYiY2UTgf4BxwL29LyIimay8MIdZY0p5at2exJ1l0OtrWES9o5uuAH7onPsiMDJ1ZYnIYHDBlGGs3tnE2006ozvT9TUsOs3sGuA64C9eWzA1JYnIYHHhlOEAPPWGNkVlur6GxceBucC/Oee2mNk44DepK0tEBoNJwwqpKsvTpqgsEOhLJ+fcWuBmAO+M6iLn3HdSWZiIDHxmxgWnDufe5dtp6QiTH+rTV4oMQn09GuppMxviXWdiFXC3mf1naksTkcHgoqnD6QhHeW5jQ7pLkRTq62aoYufcAeADwN3OudnAhakrS0QGizOryyjKCWhTVIbra1gEzGwkcDXv7OAWESEU8DH/lAqeemMP0ajO5s5UfQ2LW4hN+vemc26FmY0HNqauLBEZTC6cMoyG5nZW1e1PdymSIn0KC+fcH5xz051zn/Eeb3bOXZna0kRksFgweRi+Pp7N3doR4fzvPc2jr73VD5VJsvR1B3elmT1kZnvMbLeZ/dHMKlNdnIgMDqUFIc6sLuPJtYn3W6zY2sjmhkPc/byutjeY9HUz1N3ELlw0iti1sB/x2kREgNhRUet3H2Tb3kO99lu2eS8Atdv2JewrA0dfw6LCOXe3cy7s3X4J6DqmInLEwqkjAHhibe+bopa9uZfqofmYwUOv7OyP0iQJ+hoWDWb2ETPze7ePAHtTWZiIDC5jhuZz6ogiHl/Tc1g0t4dZvbOJ900fxdzxQ3nolZ26HsYg0dewuIHYYbNvA28BHyQ2BYiIyBELpw6ndlsje5vbu31+xZZGIlHH3AlD+cCsSrbtbeHl7fv6uUo5EX09Gmq7c+5S51yFc26Yc+5yYifoiYgcsXDaCKIOnnqj+x3dL7zZQMjvY/bYUhadNoLcoI8HX9amqMHgZK6U949Jq0JEMsK0UUMYVZzb436LZZv3MmNMCblBP4U5AS6eNoK/vPYW7eFIP1cqx+tkwsKSVoWIZAQz46Kpw1m6sZ7WjqMDoKmlkzW7DnD2hKFH2j4wq5Km1k6W9DASkYHjZMJCe6VE5BgXTR1BW2eUpRvrj2p/actenIO5498Ji3kThlJRlNPjpijnHH9b/RZtnRp5pFuvYWFmB83sQDe3g8TOuRAROcq7xpdRlBvg8S6bopZt3ktOwMeMMSVH2gJ+H5fPGMWS9Xtoau085rVeeHMvn/nty/z33zW7ULr1GhbOuSLn3JBubkXOOU1cLyLHCPp9nH/qMJ5at5twJHqkfdmbe6mpLiUn4D+q/6LTRtAZcd1Ocf70+tjmqTuXbmHX/tbUFi69OpnNUCIi3brk9JHsa+nkk7+qZc/BNvY2t/PG2weP2gR12IyqUkrygyxZf+x+i2c21HPqiCIAbntsfcrrlp4pLEQk6RZOHc43L53Gsjf3cvEPnuX7T2wAYO6E8mP6+n3G/EkVPL2+/qgpznftb2XD7maunFXJJ949jode2cnquqZ++xnkaAoLEUk6M+O6s6t59OZzqCzN596XtpMf8jO9srjb/gtOqaChuZ01uw4caXt2Q2wH+fzJFXxmwQSGFoT41qNrdcZ3migsRCRlJg4r5MHPns2XLz6FLy08haC/+6+c+ZMrMOOoTVHPbKhnxJBcJg8vpCg3yBcumsxLWxoTzj0lqaGwEJGUCvp9fO68idzw7nE99ikvzGF6ZcmRsAhHojy3qYFzJ1dgFjul65ozq5hQUcAtf1mrnd1poLAQkQHhvFMqeHXHfhoPdfDqjv0cbAtz7invTG4d8Pu47aozaGrp5MqfvsDG3QePWn51XRP//dRGnZORIgoLERkQzjtlGM7B0o31PLOhHr/PmDfx6B3is8aU8vtPzyUcdXzwZ8tY6V0T4/O/e4X3//g5vv/EBv6wsi5NP0FmU1iIyIBw+uhihhaEWPLGHp7ZUM/MqhKK84LH9Js6aggPfuZsSvODXPuLF7nwP5/hybW7+fz5E5leWcxdz2056qgqSQ6FhYgMCD6fce4pFTy1bg+rdzZx7uSer69WVZbPA585m7PGD+Xqmiqe+fIC/mnhKdw4fzxbGg716Vrgcnx0FraIDBgLThl2ZJ6o+b2EBcR2it9zw5yj2hZNG8HokjzuXLqFhdNGpKzObKSRhYgMGPMnleMzKCsIcfro7s/J6E3A7+OGd49j+dZGVu3Yn4IKs1dKw8LMFpnZejPbZGZf6eb5682s3sxe9W6fjHsuEtf+cCrrFJGBoSQ/xPvPGMVVNZX4fCd2FYQPnVlFUW6AXyzdnOTqslvKNkOZmR+4HbgIqANWmNnDzrm1Xbr+3jl3Uzcv0eqcm5Gq+kRkYPrR4pkntXxhToBr54zhzue2sKOxhaqy/CRVlt1SObKYA2xyzm12znUA9wGXpXB9IiIAXD+vGgPufn5rukvJGKkMi9HAjrjHdV5bV1ea2Wtm9oCZVcW155pZrZm9aGaXd7cCM7vR61NbX1/fXRcRyUIji/N4/xmjuG/FdvYd6kh3ORkhlWHR3QbHrgc/PwJUO+emA08C98Q9N8Y5VwNcC/zQzCYc82LO3eGcq3HO1VRU9H7khIhkl88smEBLR4S7n9+S7lIyQirDog6IHylUArviOzjn9jrn2r2HvwBmxz23y/t3M/A0cHIbMkUkq0weXsTF04bzyxe2crDt2KvwyfFJZVisACaZ2TgzCwGLgaOOajKzkXEPLwXWee2lZpbj3S8H5gFdd4yLiPTqpvMmcaAtzK9f3JbuUga9lIWFcy4M3AQ8RiwE7nfOrTGzW8zsUq/bzWa2xsxWATcD13vtU4Bar30J8J1ujqISEenV6ZXFzJ9cwf8s3UJrhyYYPBmWKRcSqampcbW1tekuQ0QGmOVbGrn658v4xvun8vF5PU+Tnq3MbKW3f7hXOoNbRDLanHFlzKku445nN9Me1ujiRCksRCTjfe78ibzV1MYfV+5MdymDlsJCRDLe/EnlnFFVwu1LNtERjqa7nEFJYSEiGc/M+MKFk9i5v5UHdHGkE6KwEJGssGByBTM0ujhhCgsRyQpmxhcvmszO/a3cX7sj8QJyFIWFiGSN+ZPKmTmmhJ8s2aQjo46TwkJEsoaZ8cULJ7OrqY37a7Xv4ngoLEQkq5wzqZzZY0u5/e+b2NvcnngBARQWIpJlzIyvv3cK+1s7+PCdL9GoKcz7RGEhIlln5phS7vzYmWxpOMS1v3hR17zoA4WFiGSld08q587ratjccIgP3/mSNkkloIkERSSrPbOhnk/9qpZI1DFlZBGzxpQyZ1wZi6aNIODP/L+nNZGgiEgfnDu5goc+ezafWzCB4rwgf1xZx033vsK3Hl2X7tIGlEC6CxARSbdpo4qZNqoYgEjU8a1H13L381s5fXQxV86uTHN1A4NGFiIicfw+42uXTGHu+KH880OrWV3XlO6SBgSFhYhIFwG/jx9fO5Pywhw+/evao3Z+Z8p+3uOlzVAiIt0YWpjDzz86myt/+gKLfrSUkN/HgbZOWjoifORdY/jXS6dhZukus98oLEREenDa6GJ++pFZ/G75DopyAgzJC7L3UAf3LNtGbtDPV95zatoD4+fPvElLR4QvXjQ5petRWIiI9OL8U4dz/qnDjzx2zlGSF+Tnz25mSF6Qz503MW21dUai/GLpFmaNKUn5uhQWIiLHwcz45qXTONDWyW2Prac4L8hHzhqbllqeXl9PQ3M7V9dUpXxdCgsRkePk8xnfu+oMDraF+X9/fp3x5QWcPbG83+u4v3YH5YU5LDilIuXr0tFQIiInIOgdMVU9tIAvP/AaB9s6+3X99QfbWfLGHq6cNbpfzjRXWIiInKD8UIDvXXUGbzW1cutf1vbruv/0yk7CUcdVNf1z0qDCQkTkJMweW8qnz53A/bV1PLVud7+s0znH/bU7mDmmhInDivplnQoLEZGT9IULJ3HqiCK+8uDqfpnufFVdExv3NPfLju3DFBYiIicpJ+Dn+1efwf6Wjn7ZHHV/7Q5ygz7eN31kytd1mMJCRCQJpo0q5uPzxvGnV3eyc39rytbT2hHhkVd3cclpIynKDaZsPV0pLEREkuRjc2PnW/zmxW0pW8ezG+s52B7mg/08G67CQkQkSSpL87lwynDuW76dts5IStaxpeEQAKdXFqfk9XuisBARSaLrz65mX0snj6zalZLXr9vXQkl+sF83QYHCQkQkqeZOGMrk4YXcs2xrSqYz39HYSmVpXtJfNxGFhYhIEpkZH5tbzes7D/Dy9n1Jf/26fS1UleYn/XUTUViIiCTZFTNHU5Qb4JcvJHdHt3OOun0aWYiIZISCnABX11Txt9VvsftAW9Jet6G5g/ZwlMpMG1mY2SIzW29mm8zsK908f72Z1ZvZq97tk3HPXWdmG73bdamsU0Qk2T42dyzhqOO+5TuS9pp1+1oAMmtkYWZ+4HbgPcBU4Bozm9pN198752Z4tzu9ZcuAbwDvAuYA3zCz0lTVKiKSbGOHFvDuieXcX7uDSDQ5O7p37Iud7FdVllkjiznAJufcZudcB3AfcFkfl70YeMI51+ic2wc8ASxKUZ0iIimxeE4VO/e38tymhqS83uGRxeiSDBpZAKOB+PFXndfW1ZVm9pqZPWBmh2fF6tOyZnajmdWaWW19fX2y6hYRSYqLpg6nrCDEfcu3J+X16va1UlYQoiCn/69bl8qw6O4q5l3HYo8A1c656cCTwD3HsSzOuTucczXOuZqKitRfKUpE5HjkBPx8YOZonli7m/qD7Sf9euk6EgpSGxZ1QPz8uZXAUac0Ouf2OucOv4O/AGb3dVkRkcFg8ZwqwlHHgy/XnfRr1e1ryciwWAFMMrNxZhYCFgMPx3cws/j5dS8F1nn3HwMWmlmpt2N7odcmIjKoTBxWRM3YUn6/YsdJndEdjcbOsUjHCXmQwrBwzoWBm4h9ya8D7nfOrTGzW8zsUq/bzWa2xsxWATcD13vLNgK3EgucFcAtXpuIyKCzeM4YNjccYvmWE/8aa2hupyMcTdvIIqV7SZxzfwX+2qXtX+LufxX4ag/L3gXclcr6RET6w3tPH8k3H17DfSt28K7xQ0/oNQ4fNpuOE/JAZ3CLiKRcXsjPZTNH8dfVb53wju50npAHCgsRkX5xw7xxdEai/PyZN09o+TpvZDFaYSEikrnGVxRyxcxKfv3iNvacwHxRdftaKS8MkR/q/3MsQGEhItJvbr5gIuGo4ydPH//oom5fC6PTtL8CFBYiIv1m7NACPjirknuXb+ftpuMbXaTzhDxQWIiI9Kubzp9INOr4ydOb+rxMNOrYqbAQEckeVWX5XFVTxX3Ld7Bzf2uflqlvbqcjEk3bCXmgsBAR6Xc3nT8Rh+OnfRxdpPuwWVBYiIj0u9EleVwxczQPrKxjf0tHwv47GtN7Qh4oLERE0uLj88bR1hnl9ysSX0lPIwsRkSw1ZeQQzhqPlz05AAAJuklEQVRfxq+WbSMcifbaN3aORQ65QX8/VXcshYWISJpcf/Y4du5v5cl1u3vtV7evlaqy9I0qQGEhIpI2F00dzuiSPO5+fmuv/WLXsUjf/gpQWIiIpI3fZ1x39lhe2tLI2l0Huu2z52AbO/a1Mm6owkJEJGt9qGYMeUE/v3xhS7fP3/vSdiJRx+UzR/dzZUdTWIiIpFFxfpArZo3mT6/uYm/z0dOXd4Sj/Pal7Zw7uYLxFYVpqjBGYSEikmY3zBtHJOq47bH1R7X/7fXY9S+un1ednsLiKCxERNJs4rBCPvnucdy3Ygcvbt57pP2eF7YyrryAcydVpLG6GIWFiMgA8IULJ1NVlsc/P7iats4Iq+uaeHn7fj561lh8Pkt3eQoLEZGBIC/k59tXnM7mhkP8+O+b+OULW8kP+flgTWW6SwMgPZdcEhGRY5wzqYIPzBzNz555E58ZHzqziiG5wXSXBWhkISIyoHz9fVMpyg3QEYly3dlj013OERpZiIgMIGUFIf7rmpms3XWAicOK0l3OEQoLEZEB5pxJFZwzAI6AiqfNUCIikpDCQkREElJYiIhIQgoLERFJSGEhIiIJKSxERCQhhYWIiCSksBARkYTMOZfuGpLCzOqB/UBTN08Xd2nv7fHh+921lQMNx1la13X19fkTqTn+/snU3FtdvT2fqG0g1txduz4fiWXL52Mw1txde2+PJznnihNW4pzLmBtwR1/ae3t8+H4PbbXJqikVNXdX/4nUfKJ1J2obiDXr86HPR6bVfDKfj95umbYZ6pE+tvf2+JFe2pJZU6LnT6Tm+PsnU3Nflu/u+URtA7Hm7tr1+UgsWz4fg7Hm7tr7+vnoUcZshuoPZlbrnKtJdx3HQzX3n8FYt2ruH4Ox5q4ybWSRaneku4AToJr7z2CsWzX3j8FY81E0shARkYQ0shARkYSyMizM7C4z22Nmr5/AsrPNbLWZbTKz/zIzi3vu82a23szWmNl3k1t1auo2s381s51m9qp3u2Sg1xz3/JfMzJlZefIqTtn7fKuZvea9x4+b2ahk1pzCum8zsze82h8ys5JBUPNV3u9g1MyStp/gZGrt4fWuM7ON3u26uPZeP/dpcyKHcw32GzAfmAW8fgLLLgfmAgb8DXiP134e8CSQ4z0eNkjq/lfgS4PpvfaeqwIeA7YB5QO9ZmBIXJ+bgZ8NhvcaWAgEvPv/AfzHIKh5CnAK8DRQk+5avTqqu7SVAZu9f0u9+6W9/VzpvmXlyMI59yzQGN9mZhPM7H/NbKWZLTWzU7suZ2Yjif3SL3Ox/9VfAZd7T38G+I5zrt1bx55BUndKpbDmHwD/F0j6TrdU1OycOxDXtWAQ1f24cy7sdX0RqBwENa9zzq1PZp0nU2sPLgaecM41Ouf2AU8Ai9L5u5pIVoZFD+4APu+cmw18CfhJN31GA3Vxj+u8NoDJwDlm9pKZPWNmZ6a02necbN0AN3mbGe4ys9LUlXrESdVsZpcCO51zq1JdaJyTfp/N7N/MbAfwYeBfUlhrvGR8Pg67gdhfuqmWzJpTrS+1dmc0sCPu8eH6B8rPdQxdgxsws0LgbOAPcZsHc7rr2k3b4b8QA8SGk2cBZwL3m9l476+DlEhS3T8FbvUe3wp8n9iXQkqcbM1mlg98jdjmkX6RpPcZ59zXgK+Z2VeBm4BvJLnUo4tJUt3ea30NCAO/TWaNxxSSxJpTrbdazezjwP/x2iYCfzWzDmCLc+4Keq4/7T9XTxQWMT5gv3NuRnyjmfmBld7Dh4l9scYPwyuBXd79OuBBLxyWm1mU2Hww9QO5bufc7rjlfgH8JYX1wsnXPAEYB6zyfkErgZfNbI5z7u0BWnNX9wKPkuKwIEl1eztf3wdckMo/fjzJfq9TqdtaAZxzdwN3A5jZ08D1zrmtcV3qgAVxjyuJ7duoI/0/V/fSvdMkXTegmrgdVcALwFXefQPO6GG5FcRGD4d3Pl3itf8DcIt3fzKxIaYNgrpHxvX5InDfQK+5S5+tJHkHd4re50lxfT4PPDBIPteLgLVARSrqTeXngyTv4D7RWul5B/cWYlsjSr37ZX393KfjlvYC0vJDw++At4BOYkn+CWJ/rf4vsMr75fiXHpatAV4H3gR+zDsnNoaA33jPvQycP0jq/jWwGniN2F9sIwd6zV36bCX5R0Ol4n3+o9f+GrG5eEYPks/HJmJ/+Lzq3ZJ6FFeKar7Ce612YDfwWDprpZuw8Npv8N7fTcDHj+dzn46bzuAWEZGEdDSUiIgkpLAQEZGEFBYiIpKQwkJERBJSWIiISEIKC8loZtbcz+u708ymJum1IhabpfZ1M3sk0YyvZlZiZp9NxrpFutKhs5LRzKzZOVeYxNcLuHcm1kup+NrN7B5gg3Pu33rpXw38xTl3Wn/UJ9lFIwvJOmZWYWZ/NLMV3m2e1z7HzF4ws1e8f0/x2q83sz+Y2SPA42a2wMyeNrMHLHath98evuaA117j3W/2Jg9cZWYvmtlwr32C93iFmd3Sx9HPMt6ZSLHQzJ4ys5ctdt2Dy7w+3wEmeKOR27y+X/bW85qZfTOJb6NkGYWFZKMfAT9wzp0JXAnc6bW/Acx3zs0kNivst+OWmQtc55w733s8E/gCMBUYD8zrZj0FwIvOuTOAZ4FPxa3/R976E877482LdAGxM+wB2oArnHOziF1H5fteWH0FeNM5N8M592UzWwhMAuYAM4DZZjY/0fpEuqOJBCUbXQhMjZspdIiZFQHFwD1mNonYTJ/BuGWecM7FX8tguXOuDsDMXiU2Z9BzXdbTwTsTM64ELvLuz+WdaxTcC3yvhzrz4l57JbFrHkBszqBve1/8UWIjjuHdLL/Qu73iPS4kFh7P9rA+kR4pLCQb+YC5zrnW+EYz+29giXPuCm/7/9NxTx/q8hrtcfcjdP+71One2SnYU5/etDrnZphZMbHQ+RzwX8Suh1EBzHbOdZrZViC3m+UN+Hfn3M+Pc70ix9BmKMlGjxO7ngQAZnZ4iuliYKd3//oUrv9FYpu/ABYn6uycayJ2KdYvmVmQWJ17vKA4DxjrdT0IFMUt+hhwg3fdBcxstJkNS9LPIFlGYSGZLt/M6uJu/0jsi7fG2+m7ltj08gDfBf7dzJ4H/Cms6QvAP5rZcmAk0JRoAefcK8RmNl1M7AJENWZWS2yU8YbXZy/wvHeo7W3OuceJbeZaZmargQc4OkxE+kyHzor0M+9qf63OOWdmi4FrnHOXJVpOJJ20z0Kk/80GfuwdwbSfFF7GViRZNLIQEZGEtM9CREQSUliIiEhCCgsREUlIYSEiIgkpLEREJCGFhYiIJPT/ASHFM/jclx2tAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this example, a learning rate around 2e-2 seems like the right fit." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "lr = 2e-2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`OneCycleScheduler`](/callbacks.one_cycle.html#OneCycleScheduler)\n", "\n", "Train with Leslie Smith's [1cycle annealing](https://sgugger.github.io/the-1cycle-policy.html) method. Let's train our simple learner using the one cycle policy." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 00:07

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
00.1094390.0593490.98086400:02
10.0395820.0231520.99214900:02
20.0190090.0212390.99165900:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit_one_cycle(3, lr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The learning rate and the momentum were changed during the epochs as follows (more info on the [dedicated documentation page](https://docs.fast.ai/callbacks.one_cycle.html))." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuoAAAEKCAYAAABEy3C5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd8lfX5//HXlU0gCZAFhBFG2FOWIsuFYhUEJ45Ka4ttv9ph7a9oq7VaqtY66qh11lUnikVFUZG9BGSPkBD2yGCEJJB9/f44N5rGQA4hJ/c5Odfz8bgfOec+933nfRh3rtznc18fUVWMMcYYY4wx/iXE7QDGGGOMMcaY77NC3RhjjDHGGD9khboxxhhjjDF+yAp1Y4wxxhhj/JAV6sYYY4wxxvghK9SNMcYYY4zxQ1aoG2OMMcYY44esUDfGGGOMMcYPWaFujDHGGGOMHwpzO4CbEhISNDU11e0Yxhhz2latWpWnqolu52hIds42xgSyupy3g7pQT01NZeXKlW7HMMaY0yYiO93O0NDsnG2MCWR1OW/b0BdjjDHGGGP8kBXqxhhjjDHG+CEr1I0xxhhjjPFDVqgbY4wxxhjjh6xQN8YYY4wxxg/5tFAXkUtEJF1EMkVkag2vR4rIO87ry0Uk1Vl/kYisEpH1ztfzq+wz0FmfKSJPiog461uKyBcikuF8beHL92aMMcYYY4wv+axQF5FQ4BlgLNATmCQiPattdgtwWFW7AI8DDzvr84DLVbUPcDPwepV9ngWmAGnOcomzfiowR1XTgDnOc2OMMcYYYwKSL/uoDwEyVTULQETeBsYDm6psMx64z3k8HXhaRERVV1fZZiMQJSKRQEsgVlWXOsd8DbgC+NQ51mhnn1eBecDv6/1dmTOiqmw5UMD6vfkcyC8mNEQICxHimoQT3yyShGYRdEpoRlx0uNtRjTGNwOcbD5CRU0ir2CjO7hxPSvMmbkcyxgSIo8VlbM8tIreghINFJRw+VkZxWQW928RxYc/kBsngy0I9Bdhd5fkeYOjJtlHVchHJB+LxXFE/4UpgtaqWiEiKc5yqx0xxHier6n7nWPtFJKmmUCIyBc8Vedq3b1+X92XqoLJSmbF6L/+av42MnMJat0+MiaRrcjPOat+CQaktOat9c2KirHg3xpyeuek5vPX1dz+KBqe24DcXdmVYlwQXUxlj/E1RSTmrdx1hxY5DfLPrMOkHCsgpKKlx2xuGtm8UhbrUsE5PZxsR6YVnOMyY0zjmKanq88DzAIMGDTqtfU3d7D50jF++vZrVu47Qq00s0yb05tzOCbRp3gRFqahUDh8r42BhCdlHS8jKLSQzp5DNB47yzNxMKhXCQoShnVpyUY9kxvRqRRu7KmaM8cKDE/vyp8t7sevQMeZszuHVJTu4/sXlXD2wLX8a14tmkUE9QbcxQS37aDGfbzzAF5tzWLotj7IKJUSgW6tYRqQl0iWpGZ0Sm9IqNor4ZhG0bBpBVFgoISE1laO+4csz1B6gXZXnbYF9J9lmj4iEAXHAIQARaQvMAH6oqtuqbN/2JMfMFpHWztX01kBOfb4ZUzdLtx3k1tdXosBj1/Tjiv4pNf4Dj44Iq/KR9He/pRaWlLNm1xEWZuby5aZs7vtoE3/+eBPndk7g6kFtubhXK6LCQxvmzRhjAlJUeChdk2PomhzDj85N5emvMnlmXibr9+bz6o+HkBwb5XZEY0wDKS2vZM7mbN5ZuZsFW3OpVEiNj2bysFSGpyUyoH1zYv3oE3xfFuorgDQR6QjsBa4Drq+2zUw8N4suBa4CvlJVFZHmwCfAXaq6+MTGThFeICJnA8uBHwJPVTvWQ87X//rsnRmvzN2Sw61vrKJDy2heunkw7eOjT/sYzSLDGJ6WwPC0BO4a24Os3EJmrt3Heyv38Ku319CyaQQ3nd2BH57TgfhmkT54F8aYxiQqPJQ7L+7GkI4t+dkbq7jy2SW897NzaB1nn9IZ05jlHyvjtaU7eHXpDvIKS2kVG8UvRnfhigFt6JzYDKeJoN8RVd+N/hCRS4EngFDgZVWdJiL3AytVdaaIROHp6DIAz5X061Q1S0T+CNwFZFQ53BhVzRGRQcArQBM8N5He7hT38cC7QHtgF3C1qh46Vb5BgwbpypUr6/MtG8eqnYe4/oXldE2O4bUfD6FF04h6PX5lpbIs6yAvL97Ol5tziAoP4eqB7bjt/C52dcwEBRFZpaqD3M7RkOr7nL129xFueHE5bZpH8d6tw+wmdmMaoZyCYl5YkMWby3dRVFrBed0S+eE5qYzsmkhoAw5hgbqdt31aqPs7K9R9Y/ehY1z+9CJaREcw/Wfn+PxKd2ZOAS8s2M4Hq/cQGiLcMrwjt47q7FcfXRlT36xQrx9LMvOY/O8VnN05nn9PHtzgP7iNMb5RVFLO8wuyeGFhFiXllVzetzW3jupMj9axrmWqy3nbZiY19aq0vJLb3lpNRYXy78mDG2Q4SpekGB6+qi9z7hjNxb1a8czcbYz821xeX7aTysrg/UXUGFO7YV0SuG9cLxZszeWprzJq38EY49cqK5V3Vuxi9N/n8Y85GYzulsicO0bxxHUDXC3S68oKdVOv/vbZFtbuPsLfrupLakLTBv3e7eOj+cd1A/j49uH0aBXLPR9uYOKzS9i4L79BcxhjAsukIe2YOCCFJ+dksGb3EbfjGGPqKCO7gGueW8rv319P+5bRvP/zYfzzhoENXo/UJyvUTb1ZlnWQFxdt56azOzC2T2vXcvROiePNnw7l8Wv7sfvQMcY9vZgHZ22mpLzCtUzGGP8lItw3vhfJsVH87r21dq4wJsCUllfy2BdbufTJhWTmFvLIVX2Z/rNzGNihhdvRzpgV6qZeFJdVcPeM9bRr2YS7Lu3udhxEhAkD2jLnt6O4emBbnluQxfinF7PlwFG3oxlj/FBsVDh/ndCHjJxCnpqT6XYcY4yXMnMKmPDPxTw5J4PL+rZhzh2juHpQO7/t4nK6rFA39eKf87aRlVvEtCv6EB3hPxOINI+O4KEr+/LvyYPJKyxl3FOLeWFBlo1dN8Z8z3ndk5g4IIXnFmxjR16R23GMMaegqryxbCeXPbWI/fnFPH/TQB6/tn+ja9Vshbo5Y7sOHuNf87ZxRf82jOya6HacGp3XPYnZvx7B6G6JTJu1mSmvryT/WJnbsYwxfmbq2O6Eh4bw4Keb3Y5ijDmJguIyfv7GN/zxww0MTm3JZ78awZherdyO5RNWqJsz9rfZWwgNEaaO7eF2lFOKbxbJczcN5M/jejEvPZfLn15kN5oaY/5HUmwUvxjdmdkbs1myLc/tOMaYajJzChj/zGK+2JzN3Zd259UfDSGpEc+fYoW6OSOrdx3m43X7+emIjrSK8///KCLCzcNSeefWcygtr2TiP5fw4eq9bscyxviRn4zoRErzJjw4awvBPNeIMf7m0/X7Gf/0Yo4eL+M/PxnKlJGdCWnkcx9YoW7qTFV58NMtJDSLZMqozm7HOS0DO7Tg418Op3+75vz6nTU8/sVW+4FsjAEgKjyUX12Yxvq9+Xy1JcftOMYEPVXlmbmZ/Pw/39C1VQwf3z6CszvFux2rQVihbupsWdYhvt5+iNvP70KzSP+5gdRbCc0ief2WoVw1sC3/mJPBHe9aWzZjjMeEASm0bxnNE19m2C/xxriorKKSqe+v55HZ6Yzv34a3p5wdEJ/g1xcr1E2dPT03g8SYSK4d3M7tKHUWERbCI1f15XcXd2PG6r3c+OJyDheVuh3LmIAhIpeISLqIZIrI1Bpe7yAic0RknYjME5G21V6PFZG9IvJ0w6WuXXhoCLed38WuqhvjoqPFZfz4lRW8s3I3t5/fhSeu7U9kWKjbsRqUFeqmTlbtPMzizINMGdGJqPDA/k8jIvzfeV14atIA1u7J59rnl5J9tNjtWMb4PREJBZ4BxgI9gUki0rPaZn8HXlPVvsD9wIPVXn8AmO/rrHVx4qr6U19ZX3VjGlpeYQnXPreMpdsO8rcr+/LbMd0aTW/002GFuqmTp7/KoEV0ODec3d7tKPXm8n5teOVHg9l7+DhX/2spuw8dczuSMf5uCJCpqlmqWgq8DYyvtk1PYI7zeG7V10VkIJAMfN4AWU9beGgItwzvyJrdR1i187DbcYwJGvuOHOeafy1le14hL00ezDUB/Mn9mbJC3Zy2jfvymZuey09GdPKryY3qw7DOCfznp2dztLiMK59dwtbsArcjGePPUoDdVZ7vcdZVtRa40nk8AYgRkXgRCQEeBX7n85Rn4KqBbYmNCuPlRdvdjmJMUNh5sIir/7WU3IISXr9lKKP8dH6WhmKFujltLy/aQXREKDee3cHtKD7Rv11z3plyDgDXPLfUeq0bc3I1fQ5d/c7LO4FRIrIaGAXsBcqBXwCzVHU3pyAiU0RkpYiszM3NrY/Mp6VpZBjXD+3Apxv226dsxvhYRnYBV/9rKcdKy3nzp2czOLWl25Fc59NC3YubjCJF5B3n9eUikuqsjxeRuSJSWPUGIxGJEZE1VZY8EXnCeW2yiORWee0nvnxvwSqvsISP1u7jqoFtiWsS7nYcn+nWKobpPxtGdHgoN764nM37j7odyRh/tAeo+pl0W2Bf1Q1UdZ+qTlTVAcAfnHX5wDnAbSKyA8849h+KyEPVv4GqPq+qg1R1UGKiO1fWbh7WgRARXlmyw5Xvb0wwyMwpZNILywB459Zz6NM2zuVE/sFnhbqXNxndAhxW1S7A48DDzvpi4B48V2K+paoFqtr/xALsBD6ossk7VV5/sf7flXlz+S5KKyq5eViq21F8rn18NG9NOZvIsFBueHE56QdsGIwx1awA0kSko4hEANcBM6tuICIJzjAXgLuAlwFU9QZVba+qqXjO9a+p6vcu6PiD1nFN+EHf1ryzYjeFJeVuxzGm0dmeV8T1LywDhLemnE3X5Bi3I/kNX15R9+Ymo/HAq87j6cAFIiKqWqSqi/AU7DUSkTQgCVhY/9FNTUrLK3l92U5GdU2kc2Izt+M0iA7xTXlrytmEhQg3vLiMzBwr1o05QVXLgduA2cBm4F1V3Sgi94vIOGez0UC6iGzFc+PoNFfCnqGbh6VSWFLOzDX7at/YGOO1XQePcf0LyyivVN786dCgqS+85ctC3ZubjL7dxjnh5wPeTjU1Cc8V9KrjIa90evVOF5HgvUXYRz7dsJ/cghImn5vqdpQG1THBU6yLCJNeWM6ugzZO1ZgTVHWWqnZV1c6qOs1Zd6+qznQeT1fVNGebn6hqSQ3HeEVVb2vo7KdjQLvmdG8Vw1tf73I7ijGNxt4jx5n0wjKOl1Xwxi1D7Up6DXxZqHtzk5E325zMdcBbVZ5/BKQ6vXq/5Lsr9f/7DV2+MSmQvb50Jx0TmjIqLfjuwO6c2Iw3fzKUsopKbnp5ObkF36s1jDGNmIgwaUh71u/NZ/0eu8HcmDN1qKiUm15aztHiMt64ZSg928S6Hckv+bJQr/Umo6rbiEgYEAccqu3AItIPCFPVVSfWqerBKldqXgAG1rSvP9yYFIgycwpYufMwk4a0IyQk+CYcAEhLjuHlyYPJOVrCzS9/zdHiMrcjGWMa0BUDUogMC+GtFXZV3Zgzcay0nB+/soI9h4/z0s2D6Z1iN46ejC8L9VpvMnKe3+w8vgr4qtpQlpOZxP9eTUdEWld5Og7PeElTT95ZsZuwEGHiWW1r37gRO6t9C5698Sy2Zhcw5bWVFJdVuB3JGNNA4pqEc1nfNsxcs48iu6nUmDopq6jk5298w7o9R3hq0gCGdLQWjKfis0Ldy5uMXgLiRSQTuAP49o5/p2XXY8BkEdlTrWPMNVQr1IFfishGEVkL/BKY7IO3FZRKyit4/5u9XNQzmYRmkW7Hcd3obkk8ek0/lmUd4ldvr6ai0tvRWsaYQHf90HYUlpTz8Tq7qdSY01VZqfy/6euYvzWXaRP6cHGvVm5H8ns+nVZSVWcBs6qtu7fK42Lg6pPsm3qK43aqYd1deFp/mXr25aYcDhWVcm0QT+Fb3fj+KRwsLOX+jzfxwMebuG9cL7cjGWMawFntW9A5sSnvr9rLtYPbux3HmIDy8OwtzFi9l99e1JVJQ+z/jzdsZlJTq7dX7KJNXBQjgvAm0lP58fCO/GR4R15ZsoNXFtv04sYEAxFhwoAUvt5xyGYqNeY0vP31Lp6bn8WNZ7fntvO7uB0nYFihbk5p35HjLMrM46pB7QgN0ptIT+WuS3twYY9k7v94E19tyXY7jjGmAYzv7+k0/N81e11OYkxgWJyZxx8/3MCoroncd3kvRKye8JYV6uaUZq7dhypceVb1FvgGIDREeHJSf3q0juX2N1ezad9RtyMZY3ysXctohnRsyQer9+Jd/wNjgldmTiE/f2MVnRKb8tT1AwgLtdLzdNifljmlD1fv5az2zekQ39TtKH4rOiKMl24eTExUOLe8uoLsoyedUNcY00hMGJBCVm4R66ynujEndaiolFteXUF4aAgv3TyY2KhwtyMFHCvUzUlt3n+ULQcKuGKAXU2vTau4KF6aPIj842VMeX2VtW00ppG7tE9rIsJCmLHahr8YU5PS8kp+9sYq9ucX8/wPB9GuZbTbkQKSFermpD5cs5ewEOEHfVrXvrGhV5s4HrumP2t3H+He/26wj8SNacTimoRzYY8kPl63j/KKSrfjGON3/vLJJr7efohHrurLwA4t3I4TsKxQNzWqrFRmrtnHqK6JxFvvdK9d0rsVt5/fhXdX7uGNZTvdjmOM8aHL+rYhr7CUr7fXOqG2MUHl3ZW7eW3pTqaM7PTtzdembqxQNzVavv0Q+/OLGW/DXk7bby7syvndk/jzR5vsB7gxjdh53ZJoEh7KJ+v3ux3FGL+xbs8R/vjhBs7tEs//u7ib23ECnhXqpkYfrt5L04hQLuqR7HaUgBMSIjx+bX/atYzmF/9Zxf78425HMsb4QJOIUM7vkcRnGw7Y8BdjgLzCEn72+ioSm0Xy1KSzrMNLPbA/QfM9peWVfLphPxf3bkWTiFC34wSkuCbhPH/TQI6XVvAzu7nUmEbrB31ac7DIhr8YU1ZRyf/95xsOFpXy3E0Dadk0wu1IjYIV6uZ7Fm/L42hxOZf3beN2lICWlhzDY9f2Z+2efP780Sa34xhjfODE8JePbfiLCXIPfbqF5dsP8eDEPvROiXM7TqNhhbr5nlnr9hMTFcawLvFuRwl4F/dqxc9Gdeatr3fxobVxM6bROTH8ZbYNfzFB7LMNB3hp0XYmD0tl4llt3Y7TqFihbv5HWUUln2/K5qIeyUSG2bCX+nDnmK4MSW3J3TPWk5lT4HYcY0w9u8wZ/rLchr+YILTr4DF+N30t/drGcfelPdyO0+hYoW7+x9JtB8k/XsZY651eb8JCQ3hy0gCahIfyi/98w7HScrcjGWPq0Wjr/mKCVEl5Bbe99Q0CPH39WUSEWVlZ3+xP1PyPTzfsp2lEKCPSEtyO0qi0iovi8Wv7k5FTyL3/3eh2HGNMPWoSEcr53ZP4fGM2lZU20ZkJHg/O2sK6Pfk8cnU/m3nUR6xQN98qr6hk9sZsLuiRTFS4DXupbyO7JnL7eV2YvmoP767c7XYcY0w9GtMrmbzCElbvPuJ2FGMaxGcb9vPKkh38+NyOXNyrldtxGi2fFuoicomIpItIpohMreH1SBF5x3l9uYikOuvjRWSuiBSKyNPV9pnnHHONsySd6ljGe19vP8SholIu7WP/4XzlVxd25ZxO8dzz4QbSD9h4dWMai9HdkggLEb7YlO12FGN8zjMufR392jVn6tjubsdp1HxWqItIKPAMMBboCUwSkZ7VNrsFOKyqXYDHgYed9cXAPcCdJzn8Dara31lyajmW8dIn6/fTJDyUUV2T3I7SaIWGCP+Y1J+YqDB++dZq669uTCMR1yScoZ1a8sWmA25HMcanSssrvxuXPmmAjUv3MV/+6Q4BMlU1S1VLgbeB8dW2GQ+86jyeDlwgIqKqRaq6CE/B7q0aj1X3+MGlolKZvfEA53dPskmOfCwpJopHrupHenYBD326xe04xph6clGPZLblFpGVW+h2FGN85tEv0m1cegPyZaGeAlQdiLvHWVfjNqpaDuQD3jTv/rcz7OWeKsV4XY9lgJU7DpFXWMpYG/bSIM7rnsSPzk3llSU7+GqLfVRuTGNwYc9kABv+YhqtJZl5PL8gi+uHtrdx6Q3El4V6TVezq98O78021d2gqn2AEc5y0+kcS0SmiMhKEVmZm5tby7cKHl9syiYiNITR3WzYS0P5/SXd6d4qhjvfW0fO0dP58MgY44/atoimZ+tYK9RNo3S4qJQ73l1Lx4Sm3POD6iOZja/4slDfA7Sr8rwtsO9k24hIGBAHnHLGCFXd63wtAN7EM8TG62Op6vOqOkhVByUmJp7mW2qcVJUvNmdzTud4mkWGuR0naESFh/LUpAEUlZTz2/fWWls3YxqBi3oms2rXYfIKS9yOYky9UVXunrGeg0UlPHndABsi24B8WaivANJEpKOIRADXATOrbTMTuNl5fBXwlaqetFoRkTARSXAehwOXARvqcizznW25hew8eIyLnI9tTcNJS47hnst6sjAjj5cXb3c7jjHmDF3UMxlV+GpzTu0bGxMg3l25m083HODOMd3onRLndpyg4rNC3RknfhswG9gMvKuqG0XkfhEZ52z2EhAvIpnAHcC3LRxFZAfwGDBZRPY4HWMigdkisg5YA+wFXqjtWObUPnc+pr2ghw17ccMNQ9szpmcyD3+2hQ17892OY4w5A73axJLSvMm351VjAl1WbiH3zdzEsM7x/HREJ7fjBB2fjnNQ1VnArGrr7q3yuBi4+iT7pp7ksANPsv1Jj2VO7ctN2fRJiaN1XBO3owQlEeHhK/ty8RMLuOPdNcy8bbhNOGVMgBIRLuyRxDsrd3O8tMKGCJiAVlZRya/fWUNEWAiPXtOPkBBrptfQrPllkMst8Mykd2EPG/biphZNI/jbVX3Zml3IY19sdTuOMeYMXNgzmeKySpZm5bkdxZgz8sSXW1m3J5+HJvaxi3kusUI9yM3dkoMqNj7dD4zulsQNQ9vzwsIslmUddDuOMV7xYgbqDiIyR0TWOTNLt3XW9xeRpSKy0Xnt2oZP7xtDOrakSXgo89Kts5gJXKt2HubZedu4ZlBbxvZp7XacoGWFepD7YnM2Kc2b0KN1jNtRDHD3pT1o3zKaO99bS0FxmdtxjDklL2eg/jvwmqr2Be4HHnTWHwN+qKq9gEuAJ0SkecMk963IsFDO7RLPV1tysJ4GJhAdL63gzvfW0jquCfdcZq0Y3WSFehA7XlrBwoxcLuyRhE3i6h+aRobx6NX92HfkOH/5eLPbcYypjTczUPcE5jiP5554XVW3qmqG83gfkAM0mp65o7slsefwcbblFrkdxZjT9vBnW9ieV8QjV/UlJirc7ThBzQr1ILY4M4/iskou6mmzi/mTQaktuXVUZ95ZuZsvrXOE8W/ezEC9FrjSeTwBiBGR/5k1WkSGABHANh/lbHCju3l+55iXbm0aTWBZsi2PV5bsYPKwVIZ1SXA7TtCzQj2Ifbk5m5jIMIZ0bOl2FFPNry9Mo3urGKZ+sI6DNnGK8V/ezAh9JzBKRFYDo/C01S3/9gAirYHXgR+pauX3vkGAzibdtkU0XZObMdcKdRNACkvK+d1760iNj+b/XdLN7TgGK9SDVmWl8uXmHEZ1SyQizP4Z+JvIsFAev7Y/+cfL+MOMDTbO1firWmegVtV9qjpRVQcAf3DW5QOISCzwCfBHVV1W0zcI5Nmkz+uWxNfbD1FYUl77xsb4gWmfbGJ//nEevaYf0RE2U7k/sAotSK3fm09eYYm1ZfRjPVrHcsdF3fhs4wFmrt1X+w7GNLxaZ6AWkQQROfGz5i7gZWd9BDADz42m7zVg5gYzulsSZRXK4kxr02j837z0HN76ejc/HdmJgR3sk3Z/YYV6kJq/NRcRGJFm48/82ZSRnejfrjn3zdxIboENgTH+xcsZqEcD6SKyFUgGpjnrrwFG4pl9eo2z9G/Yd+Bbg1Jb0CwyzMapG7+Xf6yM37+/jrSkZvzmwq5uxzFVWKEepOZvzaVvShzxzSLdjmJOITREeOSqvhSVVHDvfze4HceY71HVWaraVVU7q+o0Z929qjrTeTxdVdOcbX6iqiXO+jdUNVxV+1dZ1rj5XupbeGgIw7skMHdLrg1fM37tvo82kldYymPX9LeZsf2MV4W6iAwXkR85jxNFpKNvYxlfyj9WxupdhxnVLcntKMYLackx/OrCND7dcIBP1u13O44x5jSc1z2RA0eLSc8ucDuKMTWavfEAM1bv5bbzutCnbZzbcUw1tRbqIvIn4Pd4xhYChANv+DKU8a2FmblUKozqGlg3ZgWzW0d2ok9KHPf+d4N1gTEmgIx2LojM3RI4HWtM8Mg/XsY9H26gR+tYbju/i9txTA28uaI+ARgHFMG3E1PYNJYBbH56LnFNwulnvzkHjLDQEB65ui9Hi8u476NNbscxxngpOTaKnq1jmbvFxqkb//O3z7aQV1jCw1f2ITzURkP7I2/+VkrVM7hOAUSkqW8jGV9SVeZvzWV4WgJh9p8yoHRvFcvt56fx0dp9zN54wO04xhgvje6WyKpdhykoLnM7ijHfWrHjEP9ZvosfnduRvm2bux3HnIQ3ldq7IvIc0FxEfgp8Cbzo21jGVzbvLyCnoITRNuwlIP18dGd6to7lDzM2cORYqdtxjDFeGNk1kYpKZem2g25HMQaAkvIKpr6/jpTmTbjjIuvy4s9qLdRV9e/AdOB9oBtwr6o+6etgxjfmb/WMk7Tx6YEp3BkCc+RYKffbEBhjAsJZ7VsQHRHKggwbp278w7PztrEtt4i/TOhN00ib2MifeXMz6cOq+oWq/k5V71TVL0TkYW8OLiKXiEi6iGSKyNQaXo8UkXec15eLSKqzPl5E5opIoYg8XWX7aBH5RES2iMhGEXmoymuTRSS3Sj/en3iTMdjM35pDj9axJMVGuR3F1FGvNnH8YnRnPli9lzmbs92OY4ypRURYCOd0imdhhk18ZNyXmVPAP+duY1y/Npxn3d/8njdDXy6qYd3Y2nYSkVDgGWfbnsAkEelZbbNskxggAAAgAElEQVRbgMOq2gV4HDjxC0AxcA9wZw2H/ruqdgcGAOeKSNUs71Tpx2vDc6opLCln5Y7DjO5mV9MD3W3np9G9VQx3z1jPURv3aozfG5GWwM6Dx9h5sMjtKCaIVVYqd32wniYRodx7efWSzPijkxbqIvJzEVkPdBORdVWW7cA6L449BMhU1SxVLQXeBsZX22Y88KrzeDpwgYiIqhap6iI8Bfu3VPWYqs51HpcC3wBtvchigMWZeZRXqg17aQQiwkJ4+Mq+5BaU8LfPtrgdxxhTi5HOedeuqhs3vbViFyt2HOYPP+hBgk14GBBOdUX9TeByYKbz9cQyUFVv9OLYKcDuKs/3OOtq3MaZijofiPcmuIg0d/LMqbL6SueXieki0u4k+00RkZUisjI3N7jGC87fmkuzyDDOat/C7SimHvRr15wfnduRN5btYsWOQ27HMQFORAaJyAwR+cY5j64XEW8uyhgvdExoSkrzJiy0cerGJdlHi3lo1haGdY7n6oF2jTNQnLRQV9V8Vd2hqpNUdSdwHE+LxmYi0t6LY0tNh63DNt8/sEgY8BbwpKpmOas/AlJVtS+ezjSv1rSvqj6vqoNUdVBiYvBcWVZV5qfnMqxzPBFh1paxsfjtmK60bdGEqe+vo6S8wu04JrD9B/g3cCWeiyCXOV9NPRARRnZNYEnmQcorKt2OY4LQfTM3UlpRyV8n9EGkpvLL+CNvbia9XEQygO3AfGAH8KkXx94DVL2q3RbYd7JtnOI7DvDm0uDzQIaqPnFihaoeVNUTUza+AAz04jhBY1tuEXuPHP92ljzTOERHhDFtQh+25RbxzNxtbscxgS1XVWeq6nZV3XlicTtUYzIiLZGCknLW7D7idhQTZD7feIBPNxzglxekkZpg0+EEEm8urf4FOBvYqqodgQuAxV7stwJIE5GOIhIBXIdnGE1VM4GbncdXAV85kyudlIj8BU9B/+tq61tXeToO2OxFxqAxL90zK97IrgkuJzH1bVTXRCYMSOHZeZmkHyhwO44JXH8SkRdFZJKITDyxuB2qMTm3cwIhAgtsnLppQAXFZdz73410bxXDlJGd3I5jTpM3hXqZqh4EQkQkxLmZs39tOzljzm8DZuMpmt9V1Y0icr+IjHM2ewmIF5FM4A7g2xaOIrIDeAyYLCJ7RKSniLQF/oCni8w31dow/tJp2bgW+CUw2Yv3FjTmb82lS1Iz2raIdjuK8YF7LutJTFQ4Uz9YR0VlraPHjKnJj/Cc2y/hu3uSLnM1USMTFx1Ov3bNbZy6aVCPzE4nu6CYh67sS7jNSB5wvOlyf0REmgELgP+ISA5Q7s3BVXUWMKvaunurPC4Grj7JvqknOWyNA6tU9S7gLm9yBZvjpRUs336Im87u4HYU4yMtm0Zwz2U9+M07a3lj2U5uHpbqdiQTePqpah+3QzR2I9ISefqrDPKPlREXHe52HNPIrdp5mNeX7eTmc1Lp366523FMHXjzq9V44BjwG+AzYBt2g1FAWbb9IKXlldY/vZG7on8KI7sm8rfPtrD3yHG345jAs6yGuS5MPRuZlkClwpJtNvzF+FZpeSV3fbCO1rFR3HlxN7fjmDqqtVB3eppXqmq5qr6KZxKjS3wfzdSX+em5RIWHMDi1pdtRjA+JCNOu6E2lwj0fbqCW2z2MqW44sMaZTdraM/pIv3bNiYkMs3Hqxueem7+NrdmFPHBFb5pFejOAwvijU014FCsid4nI0yIyRjxuA7KAaxouojlT87fmck6neKLCQ92OYnysXctofjumK19tyeHjdfvdjmMCyyVAGjAGa8/oM+GhIQzrEs+Crbn2y7TxmW25hTz1VSY/6NuaC3okux3HnIFTXVF/HegGrAd+AnyOZzz5eFWtPsOo8VM7DxaxPa/IZiMNIj86tyP92sZx38yNHC4qdTuOCRx6ksXUsxFpiew9cpzteUVuRzGNUGWlcvcH64kKD+FPl9totkB3qkK9k6pOVtXngEnAIOAyVV3TMNFMfViw1dNdwPqnB4/QEOHBiX3JP17GtFnWpdR47RPgY+frHDyfnnozZ4Y5TSPTPBdOFtrwF+MD763azfLth7j70h4kxUS5HcecoVMV6mUnHqhqBbBdVa1Jc4CZl55Lh/hom+AgyPRsE8uUkZ2YvmoPi6wYMF5Q1T6q2tf5mgYMARa5nasxah8fTYf46G8vpBhTX3IKipn2yWaGdmzJtYPb1b6D8XunKtT7ichRZykA+p54LCJHGyqgqbuS8gqWbDtow16C1C8vSKNjQlPunrGe46UVbscxAUZVvwEGu52jsRqRlsCyrIOUVVS6HcU0In/+aBPF5ZX8dWIfRGrsZm0CzEkLdVUNVdVYZ4lR1bAqj2MbMqSpm5U7DnO8rMIK9SAVFR7KtAm92XXoGE9+leF2HOPnROSOKsudIvImYJd8fWR4l0SKSitYveuI21FMIzFnczafrNvP7ed1oXNiM7fjmHpiU1Q1YvPSc4gIDeGczvFuRzEuGdY5gWsGteX5BVls2mcfhJlTiqmyROIZq26NA3zknM7xhAgssllKTT0oLCnnng830DW5GbeO6ux2HFOPrFBvxOZvzWVIx5ZER1j/1GB296U9aN4knLs+WEdFpTXxMCe1SVX/7CzTVPU/WHtGn4lrEk7/ds2tn7qpF49+ns7+o8U8OLEvEWFW2jUm9rfZSO07cpyt2YU27MXQPDqCey/vydo9+by2dIfbcYz/usvLdaaeDE9LZN2eI+QfK6t9Y2NOYs3uI7yyZAc3nd2BgR1auB3H1DMr1Bup79oyWqFuYFy/Nozulsgjs9PZe+S423GMHxGRsSLyFJAiIk9WWV4Byl2O16iNTEugUmFpll1VN3VTVlHJ1PfXkRwTxe8u7uZ2HOMDtRbqJ7q8VFt2i8gMEenUECHN6ZuXnkubuCi6JNkNJQZEhAfG90YV7vlwg82IaKraB6wEioFVVZaZwMUu5mr0+rVrTrPIMBv+YurshYVZbDlQwP3jexETFe52HOMD3gxefgzPifxNQIDrgFZAOvAyMNpX4UzdlFVUsjgzj8v6tbb2TOZb7VpG89sxXfnLJ5uZtf4AP+jb2u1Ixg+o6lpgrYi8qao2BqMBhYeGcHaneJvrwNTJjrwi/vFlBmN7t2JMr1ZuxzE+4s3Ql0tU9TlVLVDVo6r6PHCpqr4D2GAoP7R61xEKSsptfLr5nsnDUumTEsefZm60cbGmuiEi8oWIbBWRLBHZLiJZbodq7EZ2TWDXoWPsPFjkdhQTQFSVu2esJyIshPvG9XI7jvEhbwr1ShG5RkRCnOWaKq+d8vNzEblERNJFJFNEptbweqSIvOO8vlxEUp318SIyV0QKReTpavsMFJH1zj5PinPJWERaOj9kMpyvQftLxPytOYSFCMO6JLgdxfiZsNAQHpzYh8PHSnnos81uxzH+5SU8n6AOxzPR0SC8mPDIi/N8BxGZIyLrRGSeiLSt8trNzjk7Q0Rursf3EjCGO+fphXZV3ZyG6av2sGTbQaaO7U5ybJTbcYwPeVOo3wDcBOQA2c7jG0WkCXDbyXYSkVDgGWAs0BOYJCI9q212C3BYVbsAjwMPO+uLgXuAO2s49LPAFCDNWS5x1k8F5jhTX89xngeleem5nNWhBbE2Xs3UoHdKHLcM78hbX+9medZBt+MY/5Gvqp+qao6qHjyxnGoHL8/zfwdeU9W+wP3Ag86+LYE/AUOBIcCfgvECS8eEpqQ0b2LDX4zX8gpLmDZrM4NTWzBpcHu34xgfq7VQV9UsVb1cVRNUNdF5nKmqx1V10Sl2HQJkOvuXAm/z/ckzxgOvOo+nAxeIiKhqkXPs4qobi0hrIFZVl6rnbrjXgCtqONarVdYHlZyCYjbuO2rDXswp/frCNNq2aMJdM9ZTUl7hdhzjH+aKyCMico6InHViqWUfb87zPfFcPAGYW+X1i4EvVPWQqh4GvuC7Cy9BQ0QYkZbA4m15lFdUuh3HBIAHPt7EsZIKHpzYh5AQuw+tsfOm60uiiNwtIs+LyMsnFi+OnQLsrvJ8j7Ouxm1UtRzIB041jWaKc5yajpmsqvudY+0HkrzI2Ogs3Oq5KmOFujmV6Igwpk3oQ1ZuEc/M3eZ2HOMfhuIZ7vJX4FFn+Xst+3hznl8LXOk8ngDEiEi8l/sGheFpCRQUl7Nub77bUYyfm5uew3/X7OMX53WmS1KM23FMA/Cm68t/gYXAl8DpXHqr6de86mPavdnmTLb//gFEpuAZOkP79o3vI6P5W3NJjImkV5tYt6MYPzeqayJX9G/Ds/Myubxva9KS7aQfzFT1vDrs5s05+U7gaRGZDCwA9uLpz+7V+byxn7MBzu2cgAgsysjjrPZBN/rHeOlYaTl/nLGBLknN+Pnozm7HMQ3EmzHq0ar6e1V9V1XfP7F4sd8eoF2V523xtHmscRsRCQPigEO1HLNtledVj5ntDI05MUQmp6YDqOrzqjpIVQclJjauq84VlcqCjFxGpiVaW0bjlT9e1pOmkWHc9cF6Kiutt3owE5FkEXlJRD51nvcUkVtq2a3W87yq7lPViao6APiDsy7fm32dbRvtOfuEFk0j6JMSx8KMXLejGD/22Odb2XvkOA9O7ENkWKjbcUwD8aZQ/1hELq3DsVcAaSLSUUQi8PRfn1ltm5nAiTv9rwK+0lPMxOIMaSkQkbOdbi8/xHPFv/qxbq6yPmis23OEI8fKGGWzkRovJTSL5I8/6MnKnYd58+tdbscx7noFmA20cZ5vBX5dyz61nudFJEFETvysuQvP/Bs432uMiLRwbiId46wLSsO7JHha6xZb21Tzfev2HOHlxdu5YWh7Bqe2dDuOaUDeFOq/wlOsH3dmJS0QkaO17eSMOb8Nz4l3M/Cuqm4UkftFZJyz2UtAvIhkAndQpVOLiOzA0ypssojsqdJJ4OfAi0AmsA341Fn/EHCRiGQAFznPg8r8rbmECIywtozmNFx5VgrDOsfz8KdbyD5aXPsOprFKUNV3gUr49hx+yuGOXp7nRwPpIrIVSAamOfseAh7AU+yvAO531gWlEWmJlFcqy7KC9o/AnER5RSVT319PQrNIfj+2u9txTAOrdYy6qtZ54KqqzgJmVVt3b5XHxcDVJ9k39STrVwK9a1h/ELigrlkbg3npufRr15wWTSPcjmICiIjw1wl9uPiJBdw3cyPP3jjQ7UjGHUXOTZ4KICJn47nB/5S8OM9Px9PVq6Z9X+a7K+xB7awOzWkSHsqijFwu6pnsdhzjR15atJ1N+4/yrxvPsrbLQeikV9RFpLvz9ayaloaLaLxxuKiUtXuOWLcXUyepCU355QVpfLrhAJ9vPOB2HOOOO/AMW+ksIovxtL+93d1IwSMyLJShnVqyMNP6qZvv7Dp4jMe/3MqYnslc0ru123GMC051Rf0OPHfaP1rDawqc75NEpk4WZuaham0ZTd1NGdmJj9bu497/buSczvHE2JWboKKq34jIKKAbno4s6apqA6Yb0Ii0RB74eBN7jxwnpXkTt+MYl6kqf/hwPWEhIdw//nsDCUyQOOkVdVWd4nw9r4bFinQ/Mz89lxbR4fRt29ztKCZAhYeG8ODEPmQXFPP32eluxzENzJll9FI8QwjHALeLyB3upgouI9I89xctsu4vBpixei8LM/L4/SXdaBUX5XYc4xJv+qgjIsOA1Krbq+prPspkTlNlpTJ/ay4j0hIJtVnKzBkY0L4FN5+TyqtLdzB+QIr1dA4uH+GZDXo9zg2lpmGlJTUjOTaShRl5XGtTwwe1Q0WlPPDxJgZ2aMENQzu4Hce4qNZCXUReBzoDa/iuA4DiGb9o/MCm/UfJKyyxYS+mXtx5cTdmbzzAXe+v56PbhxMR5k1zKNMItFXVvm6HCGYiwvAuiXy1JZvKSrXp4YPYXz7eRGFJOQ9O7GP/DoKcNz+BBwHnquovVPV2Z/mlr4MZ781L98ztNNIKdVMPmkWG8cD43qRnF/DCwiy345iG86mIjHE7RLAbkZbA4WNlbNxXaxdk00gtzMjlg9V7+fmoznS1GaODnjeF+gagla+DmLqbl55Ln5Q4EmMi3Y5iGokLeyZzaZ9W/GNOBtvzityOYxrGMmDG6c6ZYerXuc48GAtsnHpQOl5awR9mbKBTYlN+cV4Xt+MYP+BNoZ4AbBKR2SIy88Ti62DGO/nHyvhm12FG22ykpp7dd3kvIsNCuPuD9ZxiwmDTeDwKnANEq2qsqsaoaqzboYJNYkwkPVrHsijD2jQGoyfmbGXXoWM8OKEPUeGhbscxfsCbm0nv83UIU3cLM3OpVKxQN/UuKTaKu8b24O4Z63lv1R6uGdTO7UjGtzKADWq/lbluRFoCryzewbHScqIjvOr5YBqBDXvzeXHhdiYNacfQTvFuxzF+4pRnAKdd1z2qemED5TGnaV56LnFNwunfzrpzmPp33eB2zFi9h2mfbOb87kkkNLPhVY3YfmCeiHwKlJxYqaqPuRcpOI1IS+D5BVks336I87oluR3HNIDyikru+mA9LaIjmHpJD7fjGD9yyqEvqloBHBORuAbKY07Dd20ZE6wto/GJkBDhwYl9OF5awf0fbXI7jvGt7cAcIAKIqbKYBjY4tSURYSE2/CWI/HvxDtbvzef+8b2Ii7bJ5sx3vPlMrRhYLyJfAN/eVWadX9y3af9RcgtK7IqL8akuSTH84rzOPPFlBhPOSrF/b42Uqv4ZQERiPE+10OVIQSsqPJShHVtaoR4kdh08xqNfpHNRz2TG9rbeHeZ/eXMz6SfAPcACYFWVxbhs/lZPVwBry2h87eejO9M5sSl/nLGBY6XlbscxPiAivUVkNZ5OXxtFZJWI9HI7V7Aa3iWB9OwCso8Wux3F+JCqcveM9YSHhPDA+N6I2Kfj5n/VWqir6qs1LQ0Rzpza3C051pbRNIjIsFAenNiXvUeO8/gXW92OY3zjeeAOVe2gqh2A3wIvuJwpaA1P87RptKvqjdv73+xlUWYevx/bnVZxUW7HMX6o1kJdRNJEZLqIbBKRrBNLQ4QzJ2dtGU1DG9KxJZOGtOelRdtZvyff7Tim/jVV1bknnqjqPKCpe3GCW49WsSQ0i2Ch9VNvtHILSnjg400MTm3B9UPaux3H+Clvhr78G3gWKAfOA14DXvdlKFM7a8to3DB1bHfim0Uy9YN1lFdUuh3H1K8sEblHRFKd5Y94bjA1LggJEc7tksCizIM2j0Ej9eePNnK8tIIHJ/YlxBpCmJPwplBvoqpzAFHVnap6H3C+NwcXkUtEJF1EMkVkag2vR4rIO87ry0Uktcprdznr00XkYmddNxFZU2U5KiK/dl67T0T2VnntUm8yBipry2jcENcknD+P68XGfUf59+Idbscx9evHQCLwATDDefwjVxMFueFdEsgrLGHLgQK3o5h69uWmbD5et5/bz+9Cl6Rmbscxfsyrri8iEgJkiMhtwF6g1rYPTg/2Z4CLgD3AChGZqapVe7zdAhxW1S4ich3wMHCtiPQErgN6AW2AL0Wkq6qmA/2rHH8vnh8oJzyuqn/34j0FNGvLaNw0tncrLuyRxGNfbOWS3q1o1zLa7UimHqjqYcC6efmREWmeT0wXZuTSo7VNEttYFBSXcc9/N9AtOYZbR3V2O47xc95cUf81EI3nBD4QuBG42Yv9hgCZqpqlqqXA28D4atuMB07cmDoduEA8tzyPB95W1RJV3Q5kOser6gJgm6ru9CJLo3KiLeNoa5NnXCAi3D++NyECf/hwg30sH+BEZOapFrfzBbNWcVGkJTVjod1Q2qg8MjudA0eLeejKPkSEeVOGmWBW6xV1VV0BICKqqqfzMWgKsLvK8z3A0JNto6rlIpIPxDvrl1XbN6XavtcBb1Vbd5uI/BBYCfzWuULU6JxoyzjK2jIal7Rp3oTfXdyN+z7axMy1+xjfv/p/TxNAzsFzHn4LWA7Yx3R+ZERaIv9ZvpPisgqiwkPdjmPO0Modh3h92U4mD0tlQHsbumpq503Xl3NEZBOw2XneT0T+6cWxazrZV7/0drJtTrmviEQA44D3qrz+LNAZz9CY/cCjNYYSmSIiK0VkZW5uYN5NPy89h94psdaW0bjqpnNS6deuOfd/tInDRaVuxzF11wq4G+gN/APPcMU8VZ2vqvNdTWYYkZZASXklK3c0yutOQaWkvIKpH6ynTVwT7hzTze04JkB485nLE8DFwEEAVV0LjPRivz1AuyrP2wL7TraNiIQBccAhL/YdC3yjqtknVqhqtqpWqGolnt6/1YfKnNjueVUdpKqDEhMD74r04aJSVu08zPk27MW4LDREeGhiH/KPlzFt1ma345g6cs6bn6nqzcDZeIYazhOR212OZoChnVoSHioszAzMC0vmO8/M3UZmTiF/mdCbppHe3CJojHeFOqq6u9qqCi92WwGkiUhH5wr4dUD18Y4z+W68+1XAV+oZ8DoTuM7pCtMRSAO+rrLfJKoNexGR1lWeTsAzu16jMzc9h0qFC3okux3FGHq0jmXKyE5MX7WHJZk2jjZQOefaicAbwP8BT+Lp/mJcFh0RxsAOLVi41f5/BbJN+47yz7mZXNG/DefZhTZzGrwp1HeLyDBARSRCRO7EGQZzKqpaDtwGzHa2f1dVN4rI/SIyztnsJSBeRDKBO4Cpzr4bgXeBTcBnwP+pagWAiETj+Wi2+g+Rv4nIehFZh6ff+2+8eG8BZ87mHBJjIumTEud2FGMA+OUFaXSIj+buGespLvPmd3jjT0TkVWAJcBbwZ1UdrKoPqOpel6MZx4i0RDbtP0peYYnbUUwdlFVU8rvpa2keHc6fLu/ldhwTYLwp1H+G5wpLCp4hKf2BX3hzcFWdpapdVbWzqk5z1t2rqjOdx8WqerWqdlHVIaqaVWXfac5+3VT10yrrj6lqvKrmV/teN6lqH1Xtq6rjVHW/NxkDSWl5JfO35nJB9ySbHMH4jajwUP46oQ87Dh7jyTkZbscxp+8moCvwK2CJMz/FUREpEJGjLmczeMapAyy2T60C0nPzt7Fx31H+ckVvWjSNcDuOCTC1FuqqmqeqN6hqsqomqeqNwA8bIJup5uvthygsKbdhL8bvnNslgasGtuX5BVls2me1XSBR1RBVjXGW2CpLjKpa824/0KtNHM2jw61NYwBKP1DAP+ZkcFnf1lzSu3XtOxhTTV0beN5RrymMV77cnE1kWAjDuyS4HcWY7/nDpT1oHh3Bne+tpbS80u04xjQaoSHCuZ0TWJSRZ/MWBJByZ8hLbJRnRmdj6qKuhbqNu2hgqsqcLdmc2yWBJhHWS9f4nxZNI/jrhN5s2n+UZ+Zmuh3HmEZlRFoCB44Wsy230O0oxkvPL8xi3Z587h/fm/hm1k7Z1E1dC3X7lb6BZeQUsvvQcS7oYXeLG/81plcrJgxI4Zm5mWzYm1/7DsYYrwx3xqkvsO4vASEju4AnvshgbO9W/KCvDXkxdXfSQv3EjUQ1LAVAmwbMaPAMewG4oLuNTzf+7U+X96RlU88QmJJy6wJjTH1o2yKaTglNv52Z2vivikrld9PX0TQylPvH93Y7jglwJy3Ua7ixqOoNRtapv4HN2eyZjbRVXJTbUYw5pebRETw4sQ9bDhTw1BwbAmNMfRndLYmlWQc5VlrudhRzCi8tymLN7iPcN66XzSBuzlhdh76YBnSwsIRvdh22q+kmYFzQI5mrBrbl2fnbWLv7iNtxjA+JyCUiki4imSIytYbX24vIXBFZLSLrRORSZ324iLzqzH+xWUTuavj0geX87kmUlleyJPOg21HMSWzLLeTvn29lTM9kxvWzwQfmzFmhHgDmpueiChdaW0YTQO65rCeJzSK58721NhFSIyUiocAzwFigJzBJRHpW2+yPeCa8G4Bnhup/OuuvBiJVtQ8wELhVRFIbInegGtKxJU0jQvkqPcftKKYG5RWV3PHuWpqEh/KXCb0Rsb4b5sxZoR4AvtyUTXJsJL1TrKWxCRxxTcJ56Mo+ZOQU8sSXNhFSIzUEyFTVLFUtBd4GxlfbRoETJ684YF+V9U1FJAxoApQC1oT/FCLCQhielsDcLTnWptEPPTvP8wniX67oTVKMDVM19cMKdT93vLSCeVtzGNOzlf12bgLO6G5JTBrSjucXbOObXYfdjmPqXwqwu8rzPc66qu4DbhSRPcAs4HZn/XSgCNgP7AL+rqqHfJq2ETi/exL784vZcqDA7SimivV78vnHnAzG9WvD5TbkxdQjK9T93PytORSXVTK2dyu3oxhTJ3df2oPWcU248921HC+1ITCNTE1XD6pf6p0EvKKqbYFLgddFJATP1fgKPF3EOgK/FZFO3/sGIlNEZKWIrMzNtY4n53XztOj9aosNf/EXxWUV/ObdNcQ3i+D+8TaxkalfVqj7uc82HKB5dDhDOrZ0O4oxdRITFc4jV/UlK6+IBz/d7HYcU7/2AO2qPG/Ld0NbTrgFeBdAVZcCUUACcD3wmaqWqWoOsBgYVP0bqOrzqjpIVQclJib64C0ElqTYKHqnxDLXCnW/8cjsdDJzCnnkqn40j45wO45pZKxQ92Ol5ZXM2ZzDRT2SCQu1vyoTuIZ1SeAnwzvy2tKdVmA0LiuANBHpKCIReG4WnVltm13ABQAi0gNPoZ7rrD9fPJoCZwNbGix5ADu/WxLf7DrM4aJSt6MEvSXb8nhp0XZ+eE4HRna1XyRN/bPqz48t3pZHQUk5Y/vYsBcT+O68uBvdW8Xwu+nrOFhY4nYcUw9UtRy4DZgNbMbT3WWjiNwvIuOczX4L/FRE1gJvAZPVcyfkM0AzYAOegv/fqrquwd9EADqvexKVik1+5LKjxWXc+e5aOiY0ZerY7m7HMY2UFep+bPaGAzSLDGNY5wS3oxhzxqLCQ3niuv4cPV7G1A/WW9eKRkJVZ6lqV1XtrKrTnHX3qupM5/EmVT1XVfupan9V/dxZX6iqV6tqL1XtqaqPuN8/PfcAACAASURBVPk+Akm/ts2Jbxph49Rddt/MjRw4Wsxj1/QjOsLmgTS+YYW6nyqvqOTzTdmc1z2JqPBQt+MYUy+6t4rl/13SjS82ZfPOit3/v737js+qvP8//vokgYQRAoQZhuxtGCKCiOICnKDFCtZVB1pxa622/frrt7b9dautrdXWXQfDhUUFRStVFAiBsEeQvcLekPX5/nEfNKYBw7hz7jt5Px+P+5H7Pvc5J+8Tbq58cs51ruvbNxCR/5KQYJzVsSGfLN1MYVFx2HGqpPfmbeCN7HWMPrsdPVvWCzuOVGJRLdTLMWNdspmNCd6fXnKyCzN7KFi+xMwGl1i+MpjJbo6ZZZVYXt/MPjCzZcHXuP6fM3PldrbtzWdIV3V7kcrlhv6t6d8unf99ZyErtuwNO45IXDqnUyN27i9gtmb+rXDrduznR6/PpXvzNO44p33YcaSSi1qhXs4Z624Etrt7O+BR4DfBtl2I3JTUFRgC/DXY3yFnB5dQS44Q8CAwxd3bA1OC13Fr0oKNJCclMLCjbk6RyiUhwfj9Fd2pnpTA3WPmUKAzgiJHbUD7hiQlGB8u2hR2lCqlqNi557U5FBU7fxrZk+pJ6pgg0RXNT1h5ZqwbCrwQPB8PnGuRWX2GAq+5+0F3XwHkBvs7kpL7egEYdgKOIRTFxc778zdyZoeG1EpWvzepfJqm1eBXl51Mzpod/Pmj3LDjiMSdtBrV6Nc2nckLNul+jwr0xEe5zFi5jUeGdeOk9Fphx5EqIJqFenlmrPtqnWD0gJ1A+rds68BkM5tlZqNKrNPY3TcE+9oANDpBx1HhslZtZ+OuA1yo0V6kErsosynf6dWcJz5axufLt4YdRyTuDOrSmBVb9pKbtyfsKFVC1sptPD5lKcN6ZHB5r+Zhx5EqIpqFenlmrDvcOkfatr+79yLSpWa0mZ15VKHiYJa7d3LWk5yUwPldVKhL5fbzoV1p1aAWd702W0M2ihylQ78jJi3YGHKSym/n/gLuem0OzevV5JFh3cKOI1VINAv18sxY99U6ZpYEpAHbjrStux/6mge8ydddYjaZWdNgX02BMsetivVZ7gqLinl33gbO69yY2ur2IpVcreQknhjZix37C7hvXA7FxbqEL1JeTdJS6NGiLpMWqJ96NLk7P35zHpt2HeDxET1ITakWdiSpQqJZqJdnxroJwHXB8+HAR8FEGBOAEcGoMK2B9sAMM6tlZqkAwUx2g4hMllF6X9cBb0fpuKLqs+Vb2bo3n0u6Z4QdRaRCdMmow/9c3IV/L9nMPz79Muw4InFlcNcmzFu3k3U79ocdpdIam7WGiXM3cM/5HTQUo1S4qBXq5Zyx7hkg3cxygXsJRmpx9wXAWGAh8D4w2t2LgMbAp8EMdzOAie7+frCvXwPnm9ky4PzgddyZMGc9qclJGu1FqpSrT2vJBd2a8Nv3lzB79faw44jEjcFdGwPwgbq/RMWiDbt4+O0F9G+Xzq1ntQ07jlRBUe1b4e7vAu+WWvZwiecHgCsOs+0vgV+WWvYl0P0w628Fzj3OyKE6UFDE5AUbGdytiSY5kirFzPj1dzKZt+4/3PHqbCbeOYC0Grq8LPJt2jSsTftGtZm0YBPX928ddpxKZfeBAm57OZu0GtV47MqeJCaUdfucSHRpANAY8u8leew+WMil6vYiVVBajWr8eWRPNu48wIOvz9WQcyLlNKhrY2as3Mb2vflhR6k03J0HX5/H6m37+PPInjRMTQ47klRRKtRjyISc9TSoXZ3T26aHHUUkFD1b1uNHQzrx3vyNPPPpirDjiMSFwV2bUFTsTFlc5hgKcgxe/HwVE+dt4P5BHTmtjX4nS3hUqMeInfsK+HBRHhdnZpCUqH8WqbpuGtCaIV2b8P/fW8wXX2p8dZFvc3KzNDLSUnh/vvqpnwhz1uzgFxMXcm6nRtxyZpuw40gVp4owRkyYu578wmKGn6JJFKRqMzN+d0UmJ6XX5PZXstm480DYkURimpkxuFsTpi7bzK4DBWHHiWs79uUz+uVsGqWm8IfvdidB/dIlZCrUY8T4rDV0apJK14w6YUcRCV1qSjWeuvoU9uUXMfqVbPILi8OOJBLTLumeQX5hMR9oTPVjVlhUzB2vziZv9wGeuKondWtWDzuSiAr1WLB0025y1u5k+CnNMdNf7yIA7Run8tvhmcxatZ1fvbso7DgiMa1ni7o0q1uDd+aWnldQyut3k5bwn2VbeGRoN42XLjFDhXoMGJe1hqQE47KezcKOIhJTLs7M4KYzWvP8tJW8NXtd2HFEYpaZcUn3DD5dtoVtGv3lqL09Zx1PTf2Sq/u2ZESflmHHEfmKCvWQFRQV8+bs9ZzTqRHptTX8k0hpP7qgE31a1+dHr88lZ82OsOOIxKxLujelsNh1U+lRmr9uJw+Mn0ufVvV5+OKuYccR+QYV6iH7eHEeW/Yc5IreLcKOIhKTqiUm8OT3etEwNZmbX8zSzaUih9GlaR3aNKzFOznq/lJeW/cc5JaXZlG/VnX+8r1eVE9SWSSxRZ/IkL30xSqapqVwdseGYUcRiVnptZN55rpT2XuwkJtfzGJ/flHYkURijplxSWYGX6zYSt4u/UH7bQqKihn9Sjab9xzkqWtO0aRGEpNUqIdoxZa9/GfZFq7q01Jjp4t8i45NUvnTyJ7MX7+T+8flUFysmUtFSruke1PcYeK8DWFHiWnuzk/fnM8XX27j15efTGbzumFHEimTqsMQvfzFKpISjCv7qNuLSHmc27kxD13QiYnzNvD4lGVhxxGJOe0apdK5aR3emqPuL0fy5CfLGZO1hjvOacflvTR/icQuFeoh2Z9fxLhZaxncrQmNUlPCjiMSN24e0IYrTmnO41OWMUF9cUX+y3d6NSNnzQ6WbdoddpSY9K+56/nt+0u4tHsG957fIew4IkekQj0k78xdz879BVzT96Swo4jEFTPjF5d1o0+r+tw/Nodpy7eEHUkkpgzr2YykBGN89tqwo8ScWau2c+/YHHqfVI/fDs/U3CUS81Soh8DdefbTFXRoXJvTWtcPO45I3ElOSuTv1/bmpPSa3PLiLBZv3BV2JJGY0aB2MgM7NuLN7HUUFmlW30NWb93HqBezaJqWwtPX9ialWmLYkUS+lQr1EHyydDOLN+7m5gFt9Ne8yDFKq1mNF27oQ63kJK57dgbrduwPO5JIzBh+SnPydh/kP7m64gSQt/sA1zw7nSJ3nrv+VOrXqh52JJFyiWqhbmZDzGyJmeWa2YNlvJ9sZmOC96ebWasS7z0ULF9iZoODZS3M7GMzW2RmC8zsrhLr/8zM1pnZnOBxYTSP7Xg89cmXNKmTwtAemolU5Hhk1K3B8zecyr78Iq5/dgY79mlGRhGAczo1ol7NaozPUveXnfsLuO7ZmWzefZDnrj+VNg1rhx1JpNyiVqibWSLwF+ACoAsw0sy6lFrtRmC7u7cDHgV+E2zbBRgBdAWGAH8N9lcI3OfunYG+wOhS+3zU3XsEj3ejdWzHI2fNDj7/cis3ntFaEyuInACdmtTh6Wt6s2rrPm56IYt9+YVhRxIJXfWkBIb2aMYHCzexfW/V/QN2f34RN70wk9y83fzt6lPo2bJe2JFEjko0K8U+QK67f+nu+cBrwNBS6wwFXgiejwfOtUhfkKHAa+5+0N1XALlAH3ff4O7ZAO6+G1gExNVp6aemLqdOShIjT2sZdhSRSqNf23QeG9GD7NXbufnFLA4UaEIkkRF9WpBfVMy4WWvCjhKKgqJibn8lm6xV23n0yh6c2UETC0r8iWah3gwo2Tqs5b+L6q/WcfdCYCeQXp5tg24yPYHpJRbfbmZzzexZMyvzz2YzG2VmWWaWtXnz5qM9puOybNNu3pu/kWv6nUTt5KQK/d4ild2FJzfl91d0Z9ryrdz2cjb5hbqJTqq2Tk3q0KdVff75xeoqN0FYUbHzw3E5TFmcxyNDu3FxZkbYkUSOSTQL9bLukizdUhxunSNua2a1gdeBu9390HAPTwJtgR7ABuAPZYVy96fdvbe7927YsGL/uv7jB0upVT2Jm85oU6HfV6SquLxXc3457GQ+WpzHna/O1ogXFaAc9yK1DO4tmh2cSLmwxHuZZvZ5cM/RPDPTpBIn2NX9TmL1tn1MXVaxJ6bCdKhIf2vOen44uCNXaxhkiWPRLNTXAiWn3GwOlJ6d5Kt1zCwJSAO2HWlbM6tGpEh/2d3fOLSCu29y9yJ3Lwb+TqTrTcyYt3Yn783fyI1ntKae7jYXiZqrTmvJwxd34f0FG7lvXA5FVexMYkUq571IPwXGuntPIvce/TXYNgn4J3Cru3cFBgIFFRS9yhjStQkNalfnn1+sCjtKhSgqdn44Poc3Zq/j/kEdGH12u7AjiRyXaBbqM4H2ZtbazKoTaaAnlFpnAnBd8Hw48JG7e7B8RDAqTGugPTAj6L/+DLDI3f9Yckdm1rTEy8uA+Sf8iI7D7ycvoW7Natw0oHXYUUQqvRvOaM0DQzry9pz13D1mDgU6sx4t5bkXyYE6wfM0vj5hMwiY6+45AO6+1d11c8EJVj0pgRGntmTK4jzWbNsXdpyoKip2Hhg/lzey13Hv+R24/Zz2YUcSOW5RK9SDPue3A5OI3PQ51t0XmNnPzezSYLVngHQzywXuBR4Mtl0AjAUWAu8Do4MGvD9wDXBOGcMw/ja4dDoXOBu4J1rHdrQ+XpLHJ0s3c9vAtqSmVAs7jkiVcNvAdjx0QSfeyVnPbS9nc7BQNWAUlOdepJ8BV5vZWuBd4I5geQfAzWySmWWb2QPRDltVjTytJQaV+qx6QVEx94/L4fXstdxzXgfuPFdFulQOUb2jMRgi8d1Syx4u8fwAcMVhtv0l8MtSyz6l7P7ruPs1x5s3Gg4WFvHzdxbSpkEtrj9dZ9NFKtItZ7WlRvVEHn57ATe9kMXT1/SmRnXNRngCledepJHA8+7+BzPrB7xkZt2I/P45AzgV2AdMMbNZ7j7lG9/AbBQwCqBlS42WdSya1a3BRZkZvDx9Nbed3Y60GpXrhNGBgiJufyWbDxflcf8gnUmXykUDeUfZc5+tZMWWvTx8SReNmy4Sgmv7teK3wzP5LHcL1z03g5371Q36BCrPvUg3ErlCirt/DqQADYJtP3H3Le6+j8hJnV6lv0GYAwBUJree1YY9Bwsr3Vn1XQcKuPbZGZHRXYZ1U5EulY4qxyhasWUvj3+4jPM6N2Zgx0ZhxxGpsr7buwWPj+jJ7NXbGf7kNNZur9x9dStQee5FWg2cC2BmnYkU6puJdIvMNLOawY2lZxHp7ihR0DUjjTM7NOS5z1ZWmnkGNu8+yMinvyB71XYeu7IH12h0F6mEVKhHSWFRMfeNnUO1ROMXw7qFHUekyrukewYv3NCHjbsOcNlfpzF/3c6wI8W9ct6LdB9ws5nlAK8C13vEduCPRIr9OUC2u0+s+KOoOn5wVlu27DnI+Flrw45y3JZs3M2wv3zG8s17+Pu1vRnaI67mPhQpNxXqUfLU1C/JXr2DR4Z1o0mahgYWiQWnt23A6z84neqJCXz3qc/5eHFe2JHinru/6+4d3L1tcG8R7v6wu08Ini909/7u3t3de7j75BLb/tPdu7p7N3fXzaRR1rdNfXq0qMtTU5fH9YRgHy/J4ztPTqOgqJixt/Tj7E66Yi2Vlwr1KPgsdwt/mLyEizKbcml3zYYmEks6NE7lzdtOp3WDWtzwwkz+8nEukVFhRSo3M+Ou89qzZtt+xsxcHXaco+buPP/ZCm58fiYt69fk7dv7k9m8btixRKJKhfoJtmLLXm5/JZu2DWvzm+9kEhn6XURiSaM6KYy7tR8XZ2bwu0lLuPWfs9hzsDDsWCJRN7BDQ/q0rs/jU3LZlx8/n/n9+UXcP24uP3tnIed0asy4W/vRNK1G2LFEok6F+gm0fsd+rv7HdMyMp6/tTe3kqI5+KSLHoWb1JP40ogc/vagzHy7KY+gTn5KbtyfsWCJRZWb8aEhHtuw5yLOfrgg7Trks37yHYX/5jDdmr+XOc9vz1DWnUEu/X6WKUKF+gizbtJsr/vY5u/YX8OINfWjdoFbYkUTkW5gZNw1ow0s39mH7vgIu+fOnvDpjtbrCSKV2ykn1Ob9LY57893I27jwQdpzDcnfemr2OS//8KXm7D/DC9/tw7/kdSEzQlWqpOlSoH6dDDcnlT04jv6iYV27uS7dmaWHHEpGjcHrbBrx75wBOOakeD70xj1EvzWLb3vywY4lEzf9c1IXCYueRibE5Iub2vfnc/sps7h4zh05N6zDxzgGc2UHj6EvVo0L9GBUUFTN16Wau+vt07h4zh/aNavPmbadzcnMV6SLxqElaCi/e0IefXtSZT5ZsZtCjU5k4d4POrkul1DK9JqPPbsfEuRuYunRz2HG+4aPFmxj02FQmL9zIDwd3ZMyovmTUVX90qZrUyeso3Tc2hyWbdrFqyz52HyykYWoy/++SLlzbr5Uux4nEuYSESFeY09s24IHXcxj9SjbndmrEz4d1o5kKBalkbjmrDW/OXsdDb8zj3bsGkFajWqh5NuzczyP/Wsi78zbSsXEqz3//VLpm6OSXVG0q1I9SUXExDWsn06NFXc5o15CBHRuSUi0x7FgicgJ1yajDW7f15/lpK/nD5KWc/8dPGH12O27o35oa1fX/XSqH5KRE/vjd7gz/2+f8z1vzeXxEj1BGKssvLOaFaSt59MOlFBU79w/qwM1ntiE5Sf/XRFSoH6XHRvQMO4KIVICkxARuGtCGwV2b8PN/LeR3k5bw0ueruG9QBy7v1VxX0KRS6NmyHvec157fT17KgPYNuKJ3iwr73kXFzttz1vHoh0tZs20/53ZqxM8u7UqL+jUrLINIrFOhLiJyBC3q1+Tv1/Zm+pdb+dV7i/nh+Lk8NfVLbj2rLUN7ZFAtUbf6SHz7wcB2fJa7lZ+8OZ/WDWrRu1X9qH6/wqJi3l+wkT9NWcbSTXvomlGH57/fjYEdNcOoSGlWlW+U6t27t2dlZYUdQ0TihLszcd4Gnvgol8Ubd5ORlsINZ7Rm+CnNqVuzeoVmMbNZ7t67Qr9pyNRmR8+Offlc9tdp7NiXz8s39aVLRp0T/j12HyhgzMw1PPfZStbt2E+bBrW4b1BHLujWhARdoZIq4Fja7aieCjKzIWa2xMxyzezBMt5PNrMxwfvTzaxVifceCpYvMbPB37ZPM2sd7GNZsM+K/a0pIpWemXFxZgbv3TWA564/leb1avKLiYvo86sp3PHqbP6zbDNFxVX35IfEr7o1q/P8908lpVoiV/3jC7JXbz8h+y0udqblbuHesXM47VdT+MXERTSrV4OnrzmFD+49i4sym6pIFzmCqJ1RN7NEYClwPrAWmAmMdPeFJda5Dch091vNbARwmbtfaWZdgFeBPkAG8CHQIdiszH2a2VjgDXd/zcz+BuS4+5NHyqizMyJyvBas38m4rLW8OXsdO/cXUL9Wdc7p1IjzOjemX5t00mpGZyQNnVGXaFizbR/f+8d0Nuzcz08u7Mw1xzCi2e4DBUxbvpWPFuXx8ZI88nYfJDU5iYsym3LVaS3JbF43SulFYtuxtNvR7KPeB8h19y8BzOw1YChQcnaFocDPgufjgScscsv5UOA1dz8IrDCz3GB/lLVPM1sEnANcFazzQrDfIxbqIiLHq2tGGl0vTePBCzoxZVEekxduZPKCjYyftRaAjo1T6XVSPTo0rk37Rqm0alCTBrWTNVqUxKQW9Wsy4fb+3DNmDj97ZyHjZq1l1JltOK9zY2olf7NkyC8sJm/3AVZu2ceKLXtYuGEX2at2sDRvN+6QmpzEmR0aMrhbEwZ1aazPvMgxiGah3gxYU+L1WuC0w63j7oVmthNID5Z/UWrbZsHzsvaZDuxw98Iy1hcRibqUaolclNmUizKbUlBUzKxV28lauY2ZK7czce56Xj1Q+I31U5OTqFOjGinVEvjxhZ05t3PjkJKLfFPdmtV59vpTeWfuBn4/aQl3vTYHM8hIq0FSolFY5OzYl8/e/KJvbFcnJYkeLetxwclNOK11Or1b1dPN1iLHKZqFelnXykr3szncOodbXtb/+COt/9+hzEYBowBatmxZ1ioiIselWmICfduk07dNOhC5CXXznoPk5u1h9dZ9bN2bz+bdB9m1v4CDRcXUCXmiGZHSzIxLu2dw8clNmb5iGzNWbGPl1r0Uu5OYYNStUZ16NauRXjuZ1g1q0aZhLRqlJocyDrtIZRbNQn0tUHJA1ubA+sOss9bMkoA0YNu3bFvW8i1AXTNLCs6ql/W9AHD3p4GnIdLf8egPS0Tk6JgZjVJTaJSawultw04jUn4JCUa/tun0a5sedhSRKima16RmAu2D0ViqAyOACaXWmQBcFzwfDnzkkbtbJwAjglFhWgPtgRmH22ewzcfBPgj2+XYUj01EREREJKqidkY96HN+OzAJSASedfcFZvZzIMvdJwDPAC8FN4tuI1J4E6w3lsiNp4XAaHcvAihrn8G3/BHwmpn9Apgd7FtEREREJC5pwiMN9SUicUjDM4qIxJeYm/BIRERERESOjQp1EREREZEYpEJdRERERCQGqVAXEREREYlBKtRFRERERGJQlR71xcw2A6uOYdMGRCZZilfKH654zh/P2aFy5T/J3RuGGaaiqc2OW8ofLuUPT+nsR91uV+lC/ViZWVY8D4um/OGK5/zxnB2Uv6qK95+b8odL+cMVz/lPRHZ1fRERERERiUEq1EVEREREYpAK9WPzdNgBjpPyhyue88dzdlD+qiref27KHy7lD1c85z/u7OqjLiIiIiISg3RGXUREREQkBqlQPwpmNsTMlphZrpk9GHaespjZs2aWZ2bzSyyrb2YfmNmy4Gu9YLmZ2Z+C45lrZr3CS/5V1hZm9rGZLTKzBWZ2V7A8Lo7BzFLMbIaZ5QT5/zdY3trMpgf5x5hZ9WB5cvA6N3i/VZj5g0yJZjbbzP4VvI6b7ABmttLM5pnZHDPLCpbFy+enrpmNN7PFwf+BfvGSPVap3Y4utdnWKsz8h8Rzux3PbXaQKarttgr1cjKzROAvwAVAF2CkmXUJN1WZngeGlFr2IDDF3dsDU4LXEDmW9sFjFPBkBWU8kkLgPnfvDPQFRgc/53g5hoPAOe7eHegBDDGzvsBvgEeD/NuBG4P1bwS2u3s74NFgvbDdBSwq8Tqesh9ytrv3KDEsVrx8fh4H3nf3TkB3Iv8O8ZI95qjdrhBqs2NDvLfb8dpmQ7TbbXfXoxwPoB8wqcTrh4CHws51mKytgPklXi8BmgbPmwJLgudPASPLWi9WHsDbwPnxeAxATSAbOI3IhAdJpT9LwCSgX/A8KVjPQszcPGhUzgH+BVi8ZC9xDCuBBqWWxfznB6gDrCj9M4yH7LH6ULsdynGoza743HHdbsdrmx18/6i32zqjXn7NgDUlXq8NlsWDxu6+ASD42ihYHtPHFFyS6wlMJ46OIbgEOQfIAz4AlgM73L0wWKVkxq/yB+/vBNIrNvE3PAY8ABQHr9OJn+yHODDZzGaZ2ahgWTx8ftoAm4HngkvY/zCzWsRH9lgVzz+juPt3V5sdmnhvt+O1zYYKaLdVqJeflbEs3ofMidljMrPawOvA3e6+60irlrEs1GNw9yJ370HkLEcfoHNZqwVfYya/mV0M5Ln7rJKLy1g15rKX0t/dexG5xDjazM48wrqxdAxJQC/gSXfvCezl68ulZYml7LGqMv6MYvKY1GaHo5K02/HaZkMFtNsq1MtvLdCixOvmwPqQshytTWbWFCD4mhcsj8ljMrNqRBr8l939jWBxXB0DgLvvAP5NpN9mXTNLCt4qmfGr/MH7acC2ik36lf7ApWa2EniNyGXUx4iP7F9x9/XB1zzgTSK/eOPh87MWWOvu04PX44n8AoiH7LEqnn9GcfPvrjY71HYv7tvtOG6zoQLabRXq5TcTaB/cSV0dGAFMCDlTeU0ArgueX0ekD+Gh5dcGdyH3BXYeulQTFjMz4Blgkbv/scRbcXEMZtbQzOoGz2sA5xG5seRjYHiwWun8h45rOPCRBx3XKpq7P+Tuzd29FZHP90fu/j3iIPshZlbLzFIPPQcGAfOJg8+Pu28E1phZx2DRucBC4iB7DFO7HWVqs8Nt9+K93Y7nNhsqqN0OqwN+PD6AC4GlRPqv/STsPIfJ+CqwASgg8pfbjUT6n00BlgVf6wfrGpEREZYD84DeMZD/DCKXgeYCc4LHhfFyDEAmMDvIPx94OFjeBpgB5ALjgORgeUrwOjd4v03Y/wZBroHAv+Ite5A1J3gsOPT/NI4+Pz2ArODz8xZQL16yx+pD7XbUs6vNjoHPUJAt7trteG+zg0xRbbc1M6mIiIiISAxS1xcRERERkRikQl1EREREJAapUBcRERERiUEq1EVEREREYpAKdRERERGRGKRCXao8M9sTfG1lZled4H3/uNTraSdy/yIiVY3abKlKVKiLfK0VcFSNvpklfssq32j03f30o8wkIiJla4XabKnkVKiLfO3XwAAzm2Nm95hZopn9zsxmmtlcM7sFwMwGmtnHZvYKkQkLMLO3zGyWmS0ws1HBsl8DNYL9vRwsO3QmyIJ9zzezeWZ2ZYl9/9vMxpvZYjN7OZj5DzP7tZktDLL8vsJ/OiIisUVttlR6SWEHEIkhDwL3u/vFAEHjvdPdTzWzZOAzM5scrNsH6ObuK4LXN7j7tmAK6plm9rq7P2hmt7t7jzK+1+VEZjPrDjQItpkavNcT6AqsBz4D+pvZQuAyoJO7+6Epr0VEqjC12VLp6Yy6yOENAq41sznAdCJTArcP3ptRosEHuNPMcoAvgBYl1jucM4BX3b3I3TcBnwCnltj3WncvJjIddytgF3AA+IeZXQ7sO+6jExGpXNRmS6WjQl3k8Ay4w917BI/W7n7o7Mzer1YyGwicB/Rz9+7AbCClHPs+nIMlnhcBSe5eSOSMb5brmAAAAQpJREFU0OvAMOD9ozoSEZHKT222VDoq1EW+thtILfF6EvADM6sGYGYdzKxWGdulAdvdfZ+ZdQL6lniv4ND2pUwFrgz6VDYEzgRmHC6YmdUG0tz9XeBuIpdgRUSqMrXZUumpj7rI1+YChcHl0OeBx4lcwswObg7aTOTMSGnvA7ea2VxgCZFLqYc8Dcw1s2x3/16J5W8C/YAcwIEH3H1j8EujLKnA22aWQuTMzj3HdogiIpWG2myp9Mzdw84gIiIiIiKlqOuLiIiIiEgMUqEuIiIiIhKDVKiLiIiIiMQgFeoiIiIiIjFIhbqIiIiISAxSoS4iIiIiEoNUqIuIiIiIxCAV6iIiIiIiMej/AOvEZiRMhTG5AAAAAElFTkSuQmCC\n", "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot_lr(show_moms=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`MixUpCallback`](/callbacks.mixup.html#MixUpCallback)\n", "\n", "Data augmentation using the method from [mixup: Beyond Empirical Risk Minimization](https://arxiv.org/abs/1710.09412). It is very simple to add mixup in fastai :" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy]).mixup()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`CSVLogger`](/callbacks.csv_logger.html#CSVLogger)\n", "\n", "Log the results of training in a csv file. Simply pass the CSVLogger callback to the Learner." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy, error_rate], callback_fns=[CSVLogger])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 00:07

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_ratetime
00.1272590.0980690.9695780.03042200:02
10.0846010.0680240.9749750.02502500:02
20.0550740.0472660.9833170.01668300:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can then read the csv." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "

\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_rate
000.1272590.0980690.9695780.030422
110.0846010.0680240.9749750.025025
220.0550740.0472660.9833170.016683
\n", "
" ], "text/plain": [ " epoch train_loss valid_loss accuracy error_rate\n", "0 0 0.127259 0.098069 0.969578 0.030422\n", "1 1 0.084601 0.068024 0.974975 0.025025\n", "2 2 0.055074 0.047266 0.983317 0.016683" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.csv_logger.read_logged_file()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`GeneralScheduler`](/callbacks.general_sched.html#GeneralScheduler)\n", "\n", "Create your own multi-stage annealing schemes with a convenient API. To illustrate, let's implement a 2 phase schedule." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def fit_odd_shedule(learn, lr):\n", " n = len(learn.data.train_dl)\n", " phases = [TrainingPhase(n).schedule_hp('lr', lr, anneal=annealing_cos), \n", " TrainingPhase(n*2).schedule_hp('lr', lr, anneal=annealing_poly(2))]\n", " sched = GeneralScheduler(learn, phases)\n", " learn.callbacks.append(sched)\n", " total_epochs = 3\n", " learn.fit(total_epochs)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 00:07

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
00.1766070.1572290.94602500:02
10.1409030.1336900.95436700:02
20.1309100.1311560.95682000:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn = Learner(data, simple_cnn((3,16,16,2)), metrics=accuracy)\n", "fit_odd_shedule(learn, 1e-3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZoAAAEKCAYAAAArYJMgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd8VVW2wPHfSocQAoQQQgiEEnooEhiwgo5UFbtgGfQ54zwHns5Ycd4443Mcy9i7YtdREXFUVAQVC6i0gNJb6KHX0EtgvT/uiWZiyk0599yyvp/P/eTec8/Zd20Nd+Wss8/eoqoYY4wxbonyOgBjjDHhzRKNMcYYV1miMcYY4ypLNMYYY1xlicYYY4yrLNEYY4xxlSUaY4wxrrJEY4wxxlWWaIwxxrgqxusAvNS4cWPNysryOgxjjAkpc+fO3aGqqf7uH9GJJisri7y8PK/DMMaYkCIi66qyv5XOjDHGuMoSjTHGGFdZojHGGOMqSzTGGGNcZYnGGGOMq1xNNCIySESWi0i+iIwp4/14EXnHeX+WiGSVeO8OZ/tyERlYYvvLIrJNRBaVaquRiHwuIiudnw3d7Jsxxhj/uJZoRCQaeBoYDHQCRohIp1K7XQvsVtW2wKPAA86xnYDhQGdgEPCM0x7Aq8620sYAU1U1G5jqvDbGGOMxN89oegP5qrpaVY8C44BhpfYZBrzmPJ8AnCUi4mwfp6pHVHUNkO+0h6pOA3aV8Xkl23oNOL82O1PS+z8U8PRX+bw1az0/rN/N0aITbn2UMcaEPDdv2MwANpR4XQD8qrx9VLVIRAqBFGf7zFLHZlTyeWmqutlpa7OINClrJxG5DrgOoEWLFv71pJSP5m/my2XbfnqdFB/DkJx0rj4li47p9avVpqm5gt0HWb5lH2d1TPM6FGNMCW4mGiljm/q5jz/HVouqjgXGAuTm5larzZev7sXhY8fZvu8IizYW8sXSbXy0YBPj527g3K7N+Ms5HWmSlFAb4ZoqeGPGOsZOX82kG06zhG9MEHGzdFYAZJZ43RzYVN4+IhIDJOMri/lzbGlbRSTdaSsd2FbJ/jWSEBtNZqO6DM5J5+FLuzFjzFn8oV8bpizewtmPTOOTBZvd/HhThmPHFVV4+LMVXodijCnBzUQzB8gWkVYiEofv4v7EUvtMBEY6zy8GvlRVdbYPd0altQKygdmVfF7JtkYCH9ZCH/yWXDeWWwd2YNKNp9GqcSKj3prHA5OXcfxErZyImSr4YulW5q7b7XUYxhiHa4lGVYuA0cAUYCkwXlUXi8jdInKes9tLQIqI5AM34YwUU9XFwHhgCTAZGKWqxwFE5G1gBtBeRApE5FqnrfuBs0VkJXC28zrg2qTWY/zv+3L5r1rw7NeruPXd+ZZsAkRREmKjaFwvjgenLMP3N4sxxmuuzt6sqpOASaW2/bXE88PAJeUc+w/gH2VsH1HO/juBs2oSb22Ji4ni3gtySK+fwMOfr+DYCeXRS7sRE233x7pJFWKjoxjdvy13fbSEb/N3cFq23zOZG2NcYt98Lvqfs7K5fVAHPpq/ib98sMj+wg4AAUb8qgUZDerw4JTl9t/cmCBgicZl1/drw+j+bRk3ZwPPT1vtdTgRIT4mmht/nc2CgkKmLN7idTjGRDxLNAFw09ntOLdbM+7/dBmTF9kXn1tUFd/9vnBhjwzapCby0Gcr7BqZMR6zRBMAUVHCgxd3pVtmA259dz7rdx70OqSw5eQZYqKjuHlAe/K37ef9HzZ6G5QxEc4STYAkxEbz1IgeIPA/b8+zaWtcUPq8ZXCXpuRkJPPo5ys4UnTck5iMMZZoAiqzUV0evLgr8wsKefiz5V6HE3ZU/3NKCRHhloHt2bjnEO/M2VDuccYYd1miCbBBXdIZ0bsFL0xfzQ/r7abC2qT8fI2m2OnZjendqhFPTM3n4NEijyIzJrJZovHAn4d0IK1+ArdNWGAlHZeJCLcNbM+O/Ud49fu1XodjTESyROOBpIRY7rswh5Xb9vPUl/lehxM2SpfOiuVmNeLMDk147utVFB48FvC4jIl0lmg80q99Ey7skcHz36xmzY4DXocTFpSfR52VdsuA9uw7UsSz36wKaEzGGEs0nhozpANxMVHc/dFir0MJI2Vnmk7N6nN+9wxe+W4Nm/YcCnBMxkQ2SzQeapKUwB9/nc1Xy7czdelWr8MJeZXNNnPzgHaowiOf2zICxgSSJRqPjTw5i7ZN6vH3j5dw7LjdW1MzWm7pDKB5w7qMPLkl780rYNmWvYELy5gIZ4nGY7HRUdwxuANrdx5knN3rUWMV5BkARvVvS1J8DA98uiwg8RhjLNEEhTM7NKFXVkOemLrS7vWoAX8mam5QN45R/dvy1fLtfL9qh/tBGWMs0QQDEWHM4A5s33eEV75b63U4IUu1/FFnJY08OYtmyQnc/+kyTtiEm8a4zhJNkOjZshG/7pjGc1+vYveBo16HE7Kk0uKZb965mwa0Z0FBIZ8s3ByAqIyJbJZogshtg9qz/2iRrVtTTfqLaTXLd0GPDDo0TeLBKcttglNjXGaJJoi0S0tiaE46b8xYy56DdlZTVf6WzgCio3zlyvW7DvLmrHXuBmZMhLNEE2RGn9mWA0eP27WaavIzzwBwRrtUTm6TwpNf5rP3sE1NY4xbLNEEmQ5N6zOgUxqvfLeGffblVyVVvawvItwxuCO7DhzleZuaxhjXWKIJQqPPbMvew0W8MdNKOlXhK51V5ZwGcponc163Zrz07Ro2F9rUNMa4wRJNEOravAFntEvlxelrOHTUlhHwV1UGA5R068D2nDgBD06xxeiMcYMlmiD1h35t2HXgKO/NK/A6lLCX2agu15yaxb/nbWRBwR6vwzEm7FiiCVK9WzWia/NkXv52jd1U6K8qjDorbXT/tqQkxnHPx0tRf6YYMMb4zRJNkBIRrj21Fat3HOCr5du8DickVLQeTWWSEmK5aUA7Zq/dxZTFW2o1LmMinSWaIDYkJ5305ARemG43cPrLn5kBynNZbibt0upx76RltsS2MbXIEk0Qi42O4ppTspi5eheLNhZ6HU7Qq2nJKyY6ir8M7cT6XQd5/Xsb8WdMbbFEE+Qu69WCxLhoXvp2jdehBL2alM6Knd4ulX7tU3niy5Xs3H+kVuIyJtJZoglyyXViubhncz5ZsNm++PxQwzwDwP8O6cjBo8d5fOrKWmjNGONqohGRQSKyXETyRWRMGe/Hi8g7zvuzRCSrxHt3ONuXi8jAytoUkbNEZJ6I/Cgi34pIWzf7FkhX9mnJ0eMnGJ9nQ50rUluDxbLTkri8dwvenLWelVv31U6jxkQw1xKNiEQDTwODgU7ACBHpVGq3a4HdqtoWeBR4wDm2EzAc6AwMAp4RkehK2nwWuEJVuwNvAX9xq2+Blp2WRJ/WjXhz1jqO21DncvlKZ7VxTgN/OrsddeOiuXfS0lppz5hI5uYZTW8gX1VXq+pRYBwwrNQ+w4DXnOcTgLPE900xDBinqkdUdQ2Q77RXUZsK1HeeJwObXOqXJ67qk0XB7kN8s8KGOlekdtIMNEqM44Yzs/lq+XamrdheS60aE5ncTDQZwIYSrwucbWXuo6pFQCGQUsGxFbX5W2CSiBQAVwH310ovgsSAzmk0SYrn9Rk2Gqo8tX2j5W9ObknLlLrc88kSio7bmjXGVJebiaasPy5LfxOUt09VtwP8CRiiqs2BV4BHygxK5DoRyRORvO3bQ+cv1djoKEb0bsE3K7azbucBr8MJSgq1d0oDxMdEc8fgjqzYup83Z62vvYaNiTBuJpoCILPE6+b8spz10z4iEoOv5LWrgmPL3C4iqUA3VZ3lbH8HOLmsoFR1rKrmqmpuampqdfrlmRG9WxAlwlv2pVeuWswzAAzsnMapbRvz8GfL2WVLbBtTLW4mmjlAtoi0EpE4fBf3J5baZyIw0nl+MfCl+uofE4Hhzqi0VkA2MLuCNncDySLSzmnrbCDsruI2TU7g7I5pjM/bYMsPl8WFcRIiwt/O7cSBo8d56DOb3dmY6nAt0TjXXEYDU/B96Y9X1cUicreInOfs9hKQIiL5wE3AGOfYxcB4YAkwGRilqsfLa9PZ/jvgPRGZj+8aza1u9c1Lw3tnsvvgMb5YutXrUIKOorU26qyk7LQkRvbN4u3Z622GBmOqQSJ5ptrc3FzNy8vzOowqOX5COfWBL2nfNIlXr+ntdThB5fp/zSV/234+v+mMWm+78NAxznzoa1o1TuTd/+7rSkIzJlSIyFxVzfV3f5sZIMRERwkX92zOtBXbbUXIAEquE8ttg9qTt243E+eH1ch5Y1xniSYEXdIzkxMKE2ymgP+gNViPxh+X9Myka/Nk7p20lANHitz7IGPCjCWaENQipS59W6fw7twCWxStBEVrtExAZaKihL+d25mte4/w1Ff5rn2OMeHGEk2IurRXc9bvOsjMNTu9DiWouH3ppGfLhlx4UgYvTV/Dmh12P5Mx/rBEE6IGd0knKSGGd6189pNAjWsZM6gDsdHCPR8vCcwHGhPiLNGEqITYaM7r1oxJCzdTeOiY1+EEhUAVEZvUT+CGs7KZumwbXy2zueeMqYwlmhB2Wa9MjhSdsFFQJQRq2PE1p7SideNE7v54iS37bEwlLNGEsJyMZNqnJfH+PCufQeBKZwBxMVHcdV5n1uw4wNhvVgfug40JQZZoQpiIcH6PDOat32MTbQKgLo45+6XT26UyNCedp77KZ/3OgwH8ZGNCiyWaEDesezNE4P0fNnodSlAI9A37d57TiZgo4W8TF9X6MgXGhAtLNCGuWYM69GmVwgc/bIz4Lzovut80OYE/nd2Or5Zv57MlNv+cMWWxRBMGLuiRwdqdB/lxwx6vQ/GUbynnwH/uyJOz6NA0if+buJiDR23GAGNKs0QTBgblNCU+JsrKZ+DqzADliY2O4p7zu7Cp8DBPTLUZA4wpzRJNGKifEMuvO6Xx0fxNHIvgJYe9LB3mZjXi0tzmvDh9NSu37vMsDmOCkSWaMHFB9wx2HzzGtBWhszx1bfOqdFZszOCO1EuI4S8f2MAAY0qyRBMmTm+XSsO6sRFdPlOt/aWcq6JRYhy3D+rArDW7+ODHyP3/YExplmjCRFxMFOd2a8bnS7ay97BNSeOVy3Iz6Z7ZgH98stSmBjLGYYkmjJzfI4MjRSeYvGiL16F4QsHb2hm+pQTuOb8Luw4c5cEpyzyNxZhgYYkmjPTIbEBWSl0+iNDymWpgZwYoT5eMZEaenMWbs9Yzd91ur8MxxnOWaMKIiHBet2bMXL2TbfsOex2OJzw+ofnJLQPa0yy5Dnf8ewFHiyJ3JKAx4GeiEZFTReQa53mqiLRyNyxTXed2a8YJhU8XRmb5LFgkxsfw9/M7s2Lrfp7/ZpXX4RjjqUoTjYj8DbgduMPZFAv8y82gTPVlpyXRPi2JjyJw6QCvR52VdmaHNM7pms6TX+azavt+r8MxxjP+nNFcAJwHHABQ1U1AkptBmZo5t1s6eet2s2nPIa9DCbhArUfjr7+e24mE2Cju+PdCTpywe2tMZPIn0RxV391nzqAeSXQ3JFNT53RtBsAnCzZ7HElgacDW2PRfk6QE/ndoR2av2cX4vA1eh2OMJ/xJNONF5HmggYj8DvgCeNHdsExNZDVOJCcjmY8WRFb5LNhKZ8Uuzc2kT+tG3DtpacQO0jCRrdJEo6oPAROA94D2wF9V9Qm3AzM1c263dBYUFEbcgmhBVjkDfOW8ey/I4XDRCf7voyVeh2NMwPkzGOABVf1cVW9V1VtU9XMReSAQwZnqG+qUzz6OoPJZME8v1jq1Hjec2ZZPFmxm6lJbt8ZEFn9KZ2eXsW1wbQdialdGgzr0bNkwokafKerJMgH+uu70NrRPS+IvHyxi/xFbt8ZEjnITjYhcLyILgfYisqDEYw2wIHAhmuo6t2s6y7bsi6xp64M3zxAXE8V9F+WwZe9h/jnZpqcxkaOiM5q3gHOBic7P4kdPVb0yALGZGhqSk44IfBQh5bNgLp0VO6lFQ0b2zeL1GeuYuXqn1+EYExDlJhpVLVTVtao6QlXXAYfwDXGuJyItAhahqbYm9RPo0yqFjxdsioj1UZSgPqH5yW2D2tOiUV1uf28Bh44e9zocY1znz2CAc0VkJbAG+AZYC3zqT+MiMkhElotIvoiMKeP9eBF5x3l/lohklXjvDmf7chEZWFmb4vMPEVkhIktF5AZ/Ygx353ZrxurtB1iyea/XoQREMI46K61uXAwPXNSVdTsP8uCU5V6HY4zr/BkMcA/QB1ihqq2As4DvKjtIRKKBp/ENHOgEjBCRTqV2uxbYraptgUeBB5xjOwHDgc7AIOAZEYmupM2rgUygg6p2BMb50bewN6hLU2KiJDJGn4XQSVvfNilc1aclr3y/hry1u7wOxxhX+ZNojqnqTiBKRKJU9Sugux/H9QbyVXW1qh7F98U/rNQ+w4DXnOcTgLPEN4fIMGCcqh5R1TVAvtNeRW1eD9ytqicAVHWbHzGGvUaJcfRtk8KnCzeHffks2EedlTZmcAcyGtThtgkLOHzMSmgmfPmTaPaISD1gGvCmiDwO+DM2MwMoOedGgbOtzH1UtQgoBFIqOLaiNtsAl4lInoh8KiLZZQUlItc5++Rt377dj26EviE56azdeTDsy2eqoVE6K5YY7yuhrd5xgEc+X+F1OMa4xp9EMww4CPwJmAyswjf6rDJl/ZMv/Sd1eftUdTtAPHBYVXOBF4CXywpKVceqaq6q5qamppYZeLgZ2Lkp0VESEUsHhFKiATilbWNG9G7Bi9NXM2+9LZJmwpM/U9AcUNUTqlqkqq/hu0YyyI+2C/BdMynWHCh99+BP+4hIDJAM7Krg2IraLMA3TQ7A+0BXP2KMCI0S4+jTuhGTwrx8Fqo9+/OQDjStn8Ct7863EpoJSxXdsFnfGfn1lIgMcEZ1jQZWA5f60fYcIFtEWolIHL6L+xNL7TMRGOk8vxj40pkpeiIw3BmV1grIBmZX0uYHwJnO8zMAq0WUMLhLOqt3HGB5GN+86VvKOcROaYCkhFjuu6grq7Yf4LEvVnodjjG1rqIzmjfwTaK5EPgt8BlwCTBMVUtf1P8F55rLaGAKsBQYr6qLReRuETnP2e0lIEVE8oGbgDHOsYuB8cASfOW6Uap6vLw2nbbuBy5yZjO4z4nZOAZ2bkqUwKQwL5+FWums2BntUrksN5Ox01Yxd52V0Ex4kfJKKSKyUFVznOfRwA6ghaqGzZ/Eubm5mpeX53UYATN87Ax27D/KFzed4XUorrjgme+oFx/DG9f+yutQqmXf4WMMemw6sdHCpBtPo25cjNchGVMmEZnrXA/3S0VnNMeKn6jqcWBNOCWZSDQ0J538bfvDdu6zUL/8lJQQy8OXdmPdroPcO2mp1+EYU2sqSjTdRGSv89gHdC1+LiLhPU42TA3s0hQR+GRh+N68GWxLOVdVn9YpXHtKK/41cz3frIiM4fcm/FU011m0qtZ3HkmqGlPief1ABmlqR5OkBHplNQrbYc4hfkLzk1sGtie7ST1ufXc+ew4e9TocY2rMn/toTBgZ0qUpy7fuI3/bfq9DqX2qITjm7JcSYqN59LLu7DpwlDs/XFz5AcYEOUs0EWZQl3QAPg3T8lmIV85+0iUjmRvPyuaj+ZuYGEGL15nwZIkmwjRNTiC3ZUMmLQq/8lm4lM6KXd+vDd0zG3DnB4vYUnjY63CMqTZLNBFocE46SzfvZc2OA16HUqtUQ2M9Gn/FREfxyKXdOFJ0nNveWxDWszqY8ObPejT7Sow+K35sEJH3RaR1III0tWtwl6YATArD8lmojzorrXVqPf53SEemrdjO6zPWeR2OMdXizxnNI8Ct+GZJbg7cgm/SynGUM3GlCW7NGtShR4sGYZdoNOyKZz5X9mlJ//ap/GPSUpZtsTsLTOjxJ9EMUtXnVXWfqu5V1bHAEFV9B2jocnzGJUNz0lm8aS/rdoZP+SzcSmfFRIQHL+lG/YRYbnj7B5t404QcfxLNCRG5VESinEfJCTXD80/ICDDIKZ99GkaDAkJtPZqqaFwvnocv7caKrftt1gATcvxJNFcAVwHbgK3O8ytFpA6+CS5NCGresC7dMsOvfBbOzmiXym9PbcXrM9bxxZKtXodjjN/8WY9mtaqeq6qNVTXVeZ6vqodU9dtABGncMaRLUxYUFLJh10GvQ6kVvtPrMD2lcdw6qD2dm9Xn1gnz2brXhjyb0ODPqLNUEfmziIwVkZeLH4EIzrhrSI5z8+ai8DirUdWwLZ0Vi4+J5okRPTh87AQ3jf+REyesem2Cnz+lsw/xrXz5BfBJiYcJcZmN6pKTkcwnYTT3WZjnGQDapNbjb+d24rv8nbwwfbXX4RhTKX8WvKirqre7HonxxOCcpvxz8nI27jlERoM6Xodj/HRZr0y+WbGdB6csp2+bFLo2b+B1SMaUy58zmo9FZIjrkRhPDM0Jn7nPwnnUWWkiwn0X5tAkKZ7Rb/3A3sPHKj/IGI/4k2huxJdsDtl6NOGnZUoinZvVD5s1aiQiimc+DerG8eTlPdi45xC3T7Apakzw8mfUWZKqRqlqHVuPJjwNyUnnh/V72LTnkNeh1Ei4zgxQkZ4tG3HbwPZ8umiLTVFjgla5iUZEOjg/TyrrEbgQjduKR5+F+j01kVQ6K+l3p7XmrA5NuOeTJSwo2ON1OMb8QkVnNDc5Px8u4/GQy3GZAGrVOJGO6fVDPtFAZCaaqCjhoUu6kVovnlFvzaPwkF2vMcGloqWcr3N+9i/jcWbgQjSBMDSnKfPW72FzYeiWzyKvcPazholxPHn5SWzec5jbJsy36zUmqPi1Ho2InCwil4vIb4ofbgdmAuunmzdD+J4aVY2owQCl9WzZkNsHdWDK4q28+v1ar8Mx5if+zAzwBr5S2alAL+eR63JcJsBap9ajQ9Ok0C+fRW6eAeC3p7Xi1x2bcO+kpfy4wa7XmODgzxlNLnCKqv5BVf/HedzgdmAm8IbmpJO3bnfILhtsxSLf/TUPXdKNJkkJjHpzHrsPHPU6JGP8SjSLgKZuB2K8N6RriM99Fqbr0VRVg7pxPHPFSWzfd4Qbxv3AcZsPzXjMn0TTGFgiIlNEZGLxw+3ATOC1Sa1H+7TQLZ8p4beUc3V1y2zA3cM6M33lDh7+bLnX4ZgI589cZ3e5HYQJHkNy0nls6gq27j1MWv0Er8MxNTC8dwvmF+zhma9X0S2zAQM7W2HCeKPCMxoRiQbuVNVvSj8CFJ8JsKFdm6IKk0Nw5U3fqDNT0l3ndaZb82RuHj+fVdv3ex2OiVAVJhpVPQ4cFJHkAMVjPNa2SRLt0uqF5NxnvtKZ11EEl/iYaJ69sifxMVH8/o257D9S5HVIJgL5c43mMLBQRF4SkSeKH/40LiKDRGS5iOSLyJgy3o8XkXec92eJSFaJ9+5wti8XkYFVaPNJEbE/3WpgSE46c9buYlsIruBoeeaXmjWow5OX92D19v12M6fxhD+J5hPgTmAaMLfEo0JO2e1pYDDQCRghIp1K7XYtsFtV2wKPAg84x3YChgOdgUHAMyISXVmbIpIL2MIcNTQ0J91XPlscWuUz+/4s38ltGjNmcAcmLdzC2Gm2WJoJrEoHA6jqa9VsuzeQr6qrAURkHDAMWFJin2H8PNhgAvCU+IYNDQPGqeoRYI2I5DvtUV6bThJ6ELgcuKCaMRsgOy2Jtk3q8cmCzfymb5bX4fhNURt1VoHfndaa+RsKeWDyMjqm1+f0dqleh2QihD8zA2SLyAQRWSIiq4sffrSdAWwo8brA2VbmPqpaBBQCKRUcW1Gbo4GJqhp6FxeC0JCcdGav3cW2faFVPrM0Uz4R4Z8Xd6VdWhKj3ppngwNMwPhTOnsFeBYoAvoDrwNv+HFcWf/mSxc3ytunSttFpBlwCfBkpUGJXCcieSKSt3379sp2j1jF5bMpITT6zEpnlUuMj+HFkbnERUfxu9fyKDxoMz0b9/mTaOqo6lRAVHWdqt4F+DN7cwGQWeJ1c2BTefuISAyQDOyq4NjytvcA2gL5IrIWqOuU235BVceqaq6q5qamWumgPO3S6tEmNZFJITTJppb3p4j5D80b1uW5q3qyYfdBRr89j6LjJ7wOyYQ5v0adiUgUsFJERovIBUATP46bA2SLSCsRicN3cb/0jAITgZHO84uBL9U3JGYiMNwZldYKyAZml9emqn6iqk1VNUtVs4CDzgADU00iwtCcdGat2cmO/Ue8DsdvkTx7c1X0ymrEP87PYfrKHdzzyVKvwzFhzp9E80egLnAD0BO4kp+TQ7mcay6jgSnAUmC8qi4WkbtF5Dxnt5eAFOfs4yZgjHPsYmA8voEDk4FRqnq8vDb97aypmiFd0zkRojdvmspd2iuTa09txavfr+Xt2eu9DseEMX9Gnc0BEBFV1Wuq0riqTgImldr21xLPD+O7tlLWsf8A/uFPm2XsU68qcZqytU9LonXjRCYt3MyVfVp6HU6lVNVu2KyiOwZ3IH/bfu78YBGtGifSp3WK1yGZMOTPqLO+IrIE3xkEItJNRJ5xPTLjORFhSE46M1eHTvnM8kzVxERH8cSIHrRIqcv1/5rL+p0HvQ7JhCF/SmePAQOBnQCqOh843c2gTPAY6pTPPg2B8pkNOque5DqxvDSyFycUrnl1to1EM7XOr6WcVXVDqU3HXYjFBKEOTX03b370Y+kBg8FH1eY6q65WjRN5/qqebNh1iOveyONIkf0TN7XHn0SzQUROxne/SpyI3IJTRjPhT0Q4r1szZq/dxaY9h7wOp0KK2qizGujTOoUHL+nKrDW7uH3CApsTzdQafxLNfwOj8N2BXwB0B/7gZlAmuJzXrRkAHy8I/rMaUzPDumdwy4B2fPDjJh75fIXX4ZgwUWmiUdUdqnqFqqapahNVvRL4TQBiM0Eiq3EiXZsnM3F+cCcaK53VjlH923JZbiZPfpnP+Dmlq+bGVJ1f12jKcFOtRmGC3nndmrFo415WB/H8WLYeTe0QEe65oAunZTfmz+8vZPpKm6rJ1Ex1E439c44w53RthghBf1Zjv5q1IzY6iqevOIm2Tepx/b/msWTTXq9DMiGsuonGrhJGmKbJCfTOasTE+ZvLUOWtAAAYkklEQVSC9iJxkIYVsuonxPLy1b1ISohh5Cuz7R4bU23lJhoR2Scie8t47AOaBTBGEyTO696M1dsPsDho/7q1mQFqW7MGdXj9v3pztOgEV708i+37QuPGXRNcyk00qpqkqvXLeCSpaqVT15jwM6RLOjFRwkdBXD6zPFP7stOSePnqXmzde5irX5nNvsN2Q6epmuqWzkwEapgYx2nZjflo/iZOnAi+OpWVztzTs2VDnr2yJ8u37OO61+dy+Jjd0Gn8Z4nGVMl53ZuxqfAwc9fv9jqUX7BRZ+7q374JD17SlRmrd/Knd37keBD+sWGCkyUaUyVnd2pKfEwUE4N0ShqbGcBdF/Rozl+GduTTRVu488NFQTswxAQXSzSmSurFx3BWxyZMWriZY0G2MqN96QXGb09rzfX92vDWrPXc9+ky++9uKmWJxlTZ+d0z2HngaNDdyGels8C5bWB7ftO3JWOnreaxL1Z6HY4JcjZ6zFRZv/ZNaFg3lvfmbuTMDmleh/MfLM8Ehohw17mdOXzsOI9PXUlCbDTX92vjdVgmSFmiMVUWFxPFsO4ZvDV7PYUHj5FcN9brkAAbdRZoUVHCfRd25fCxEzwweRl1YqO4+pRWXodlgpCVzky1XHhSBkeLTvDxwuAZFOBbytnOaQIpOkp4+NJuDOycxl0fLWHc7PVeh2SCkCUaUy05GclkN6nHv+dt9DoU47FYZznofu1TueP9hbz/Q4HXIZkgY4nGVIuIcOFJzZm7bjdrdhzwOhzAJuDzUnxMNM9d2ZO+rVO4efx8SzbmP1iiMdV2QY8MROD9eUHypWLr0XgqITaaF0fm0qd1CjeNn8+EuUHye2E8Z4nGVFvT5ARObduY9+ZtDIopaRS7YdNrdeNieGlkL05p05hbJ8znnTl2zcZYojE1dNFJzdm45xCz1+7yOhTAzmiCQZ0435nNadmp3P7eQt6aZckm0lmiMTUyoHMaiXHRvBcEZRK7Qz14JMRGM/aqnvRvn8qf31/IGzPWeh2S8ZAlGlMjdeNiGJKTzqSFmzl01NsZfX2lMxMsEmKjee6qnvy6YxPu/HAxL3+7xuuQjEcs0Zgau6hncw4cPc4nCzd7HYqVzoJMfEw0z1zRk4Gd07j74yU8+vkKO/OMQJZoTI39qlUjWjVO9PxmPfv+Ck5xMVE8fflJXNyzOY9PXcn/fbQkKAaPmMCxRGNqTEQY3iuTvHW7Wbl1n2dxKDYzQLCKiY7inxd15dpTW/Hq92u55d35QTf7t3GPJRpTKy7q2ZzYaOHt2Rs8jcPSTPCKihL+MrQjtwxox79/2Mj1/5pnK3VGCEs0plY0rhfPgM5NeW9egWdfHlY6C34iwugzs/n7sM5MXbaVq1+Zzb7Dx7wOy7jM1UQjIoNEZLmI5IvImDLejxeRd5z3Z4lIVon37nC2LxeRgZW1KSJvOtsXicjLIhIcUwpHkMt7t6Dw0DEmL9riyecr2ClNiLiqbxaPXdadvLW7ueS5GWwpPOx1SMZFriUaEYkGngYGA52AESLSqdRu1wK7VbUt8CjwgHNsJ2A40BkYBDwjItGVtPkm0AHIAeoAv3Wrb6ZsfVun0DKlLm97OCjAZgYIHcO6Z/Dy1b0o2H2IC575jmVb9nodknGJm2c0vYF8VV2tqkeBccCwUvsMA15znk8AzhLf1dxhwDhVPaKqa4B8p71y21TVSeoAZgPNXeybKUNUlHBZr0xmrdnFqu37Ax+Alc5CzuntUhn/+76owiXPzuDblTu8Dsm4wM1EkwGUvDJc4Gwrcx9VLQIKgZQKjq20TadkdhUwuaygROQ6EckTkbzt24NrKeJwcHHP5sREiSdDnX2jzgL+saaGOjWrz/ujTiajYR2ufmU27+Z5O6DE1D43E01Z/+RL/81Z3j5V3V7SM8A0VZ1eVlCqOlZVc1U1NzU1taxdTA00SUrg7E5pTJjrzaAAyzOhKT25DuP/uy99Wqdw64QFdmNnmHEz0RQAmSVeNwdKL8f40z4iEgMkA7sqOLbCNkXkb0AqcFOt9MBUy1V9W7L74DEmzg/s6pv2vRTa6ifE8so1vX66sfOGcT96Pq2RqR1uJpo5QLaItBKROHwX9yeW2mciMNJ5fjHwpXONZSIw3BmV1grIxnfdpdw2ReS3wEBghKranWAe6ts6hfZpSbzy3dqA/lWq2BQ0oS42OooHL+7K7YM68PGCTVz6/Aw2Fx7yOixTQ64lGueay2hgCrAUGK+qi0XkbhE5z9ntJSBFRPLxnYWMcY5dDIwHluC71jJKVY+X16bT1nNAGjBDRH4Ukb+61TdTMRHh6lOyWLp5L7PXBG75AFW1UWdhQES4vl8bXrgql9Xb93PeU98xb/1ur8MyNSCRXAfNzc3VvLw8r8MIS4eOHqfv/VPp2zqFZ6/sGZDPbH3HJ4zq35abB7QPyOcZ963Yuo/fvpbHlr2Hue+CHC7qaYNJg4GIzFXVXH/3t5kBjCvqxEUzvFcLpizeQsHugwH5zMj9kyl8tUtL4sNRp9CzRUNufnc+d3+0xOZIC0GWaIxrrurbEhHhjZnrAvJ5qjbqLBw1TIzj9Wt7c80pWbz83RqGj51pMwmEGEs0xjUZDeowsHMa42Zv4ODRosB8qI0GCEux0VH87dzOPDmiB8s272XoE9P5Lt9u7gwVlmiMq649tRWFh47xzhy7Cc/U3LndmvHh6FNplBjHVS/N4qkvV9raNiHAEo1xVc+Wjeid1YgXpq12tbZePKjFzmfCX9sm9fhg1Cmc07UZD322gv96bQ479h/xOixTAUs0xnXX92vDpsLDTPzR/Rs4rXIWGRLjY3h8eHf+Pqwz36/ayaDHpvPNCptSKlhZojGu69c+lQ5Nk3jum1WulTkieJR+xBIRruqbxcTRp9AoMZaRL8/mno+XcKTIZhMINpZojOuKb8BbuW0/U5dtc+UzivOM3bAZeTo0rc/E0adyVZ+WvPjtGi585ntvZg835bJEYwJiaE46mY3q8NSXK12dlsZKZ5EpITaav5/fhbFX9WTjnkMMfWI6r363xgYKBAlLNCYgYqKjGN2/LfMLCpm6tPbPaiJ5hgvzswGdmzL5xtPp0zqFuz5awogXZrJ+Z2BuGDbls0RjAubCk5rTMqUuj3y+otb/0vy5dGYiXdPkBF65uhf/vKgrSzbtZdDj03hjxlo7u/GQJRoTMLHRUdx4VjZLNu9lyuItrnyGlc4M+K4LXtorkyl/Op2eLRty54eLufKlWazbecDr0CKSJRoTUMO6Z9AmNZFHPl/B8Vr8C9MqZ6YszRrU4fX/6s39F+awoKCQAY9O48mpK21kWoBZojEBFR0l/PHX7Vi5bT8f/LCx1tpVp3gmdkpjShERhvduwdSbz+DXHdN4+PMVDHl8OjNW7fQ6tIhhicYE3NCcdLo2T+bBKctrbQ40O6MxlUmrn8DTV5zEK9f04ujxE4x4YSY3j59vswoEgCUaE3BRUcJfz+nElr2Hee6b1bXatp3QmMr0b9+Ez/54Bn/o14YPf9xI/we/Zuy0VVZOc5ElGuOJ3KxGnNM1nee/WcXGPbZUrwmsOnHR3DaoA5P/eBq5WQ25d9IyBjw6jSmLt9hQeRdYojGeGTO4AwAPfLqsxm0VfzfYzACmKto2SeKVa3rz6jW9iI2O4vdvzGXECzNZtLHQ69DCiiUa45nmDevy+9NbM3H+Jr5dWTtri1jpzFRHv/ZNmHzjadw9rDPLt+zjnCe/ZdRb88jfZlPZ1AZLNMZTf+jflqyUuvz5/YUcOlr9GrnaQs6mhmKio/hN3yy+vrU/o/u35atl2xjw6Dfc8u58Nuyy2QVqwhKN8VRCbDT3XpjD+l0HeWzqimq383PpzJiaSa4Tyy0D2zPttv5cc0orJs7fxJkPf81fPlhoCaeaLNEYz53cpjHDe2Xy4vQ1zFu/u0ZtWenM1JbG9eK585xOTLu1P5fmZvLOnA30e+hr/vTOjyzfss/r8EKKJRoTFP48tCPpyQncOO4H9h4+VuXjrXBm3NI0OYF/XJDjO8M5OYspi7cw8LFpXPvqHPLW7vI6vJBgicYEhfoJsTw+vAeb9hzmrx8sqvLxPy/lbKc0xh3pyXX4yzmd+H7Mmdx0djvmrd/Nxc/N4LynvuXdvA0cPmb34ZTHEo0JGj1bNuSPZ2XzwY+beGPmumq1YaUz47YGdeO44axsvhtzJn8f1pmDR49z64QF9LlvKvdNWmrXccoQ43UAxpT0h/5t+WHDHu6auJg2jRM5uW1jv46z0pkJtLpxMVzVN4sr+7RkxuqdvDFjHS9+u4ax01dzatvGXNyzOQM6NaVOXLTXoXrOEo0JKtFRwuPDu3PhM99z/ZvzeO/6k2nbpF6lx9nN3MYrIsLJbRpzcpvGbC48xNuzN/De3AJuHPcjSfExDMlJ56Kezclt2ZCoqMg85bbSmQk6SQmxvDSyF7HRwhUvzmTtDv/XELHZm42X0pPrcNPZ7Zh+W3/e+t2vGNC5KR8t2MSlz8+g7/1T+duHi5ixametLpERCiSS5/XJzc3VvLw8r8Mw5Vi+ZR/Dx86gTmw0b1/Xh5YpieXuW3jwGN3u/ow7z+nEtae2CmCUxlTs4NEiPlu8lUkLN/PNiu0cKTpBSmIcZ3dKo1/7JpzSNoWkhFivw6wSEZmrqrn+7m+lMxO02jdN4l+//RVXvDiLC575nhd+05OeLRuVue9P69EEMkBj/FA3Lobze2Rwfo8MDhwp4uvl2/l00WY+XrCZcXM2EBMlnNSiIWe0T+WUto3p3Kw+sdHhVWxytTciMkhElotIvoiMKeP9eBF5x3l/lohklXjvDmf7chEZWFmbItLKaWOl02acm30zgdG5WTL/vv5k6ifEMGLsLF6cvrrCtd+tcmaCWWJ8DEO7pvPU5Scx786zGXddH647vTUHjhbx4JTlnP/0d3S96zMuf2Emj3y+gukrt1N4qOr3lQUb10pnIhINrADOBgqAOcAIVV1SYp8/AF1V9b9FZDhwgapeJiKdgLeB3kAz4AugnXNYmW2KyHjg36o6TkSeA+ar6rMVxWils9Cx+8BRbntvAZ8v2UrPlg356zmd6JbZ4D/e7/H3z/nbuZ245hQrnZnQs33fEWav2cWctbvIW7eLJZv2Uvw3VfOGdeiUXp/OzZLpmJ5E69R6tGhUl7gYb858gql01hvIV9XVACIyDhgGLCmxzzDgLuf5BOAp8V3NHQaMU9UjwBoRyXfao6w2RWQpcCZwubPPa067FSYaEzoaJsYx9qqeTJhbwAOTlzPs6e/o07oRl/TM5LTsxsQ4pQY7oTGhKjUpnqFd0xnaNR2A/UeK+GH9bhZt3MviTYUs2byXz5du/WmEZZRARsM6ZKUk0qJRXdLqJ5BWP54mSQmkJsWTUi+OxPgYEuNiiPZ4tJubiSYD2FDidQHwq/L2UdUiESkEUpztM0sdm+E8L6vNFGCPqhaVsb8JEyLCJbmZDM5J540Z6/jXzHXc/O78X+xjTDioFx/DadmpnJad+tO2A0eKWL51H2t3HGDtjgOs2XmQtTsOsHDjZvYcLL/EVjcumsT4GOKio4iNFmKio3hpZG6FA2xqk5uJpqx/8aXrdOXtU972ss4TK9r/l0GJXAdcB9CiRYuydjFBrl58DNf3a8PvT2/Nks17mbl6J+t3HWTvoWOc0S618gaMCVGJ8TGc1KIhJ7Vo+Iv3Dh87zvZ9R9i27wjb9x1m98FjHDhSxL7DRRw4UsT+I0UcPX6CouNK0YkTxMcE7kZSNxNNAZBZ4nVzYFM5+xSISAyQDOyq5Niytu8AGohIjHNWU9ZnAaCqY4Gx4LtGU/VumWARFSV0yUimS0ay16EY47mE2GgyG9Uls1Fdr0P5BTevJM0Bsp3RYHHAcGBiqX0mAiOd5xcDX6pvdMJEYLgzKq0VkA3MLq9N55ivnDZw2vzQxb4ZY4zxk2tnNM41l9HAFCAaeFlVF4vI3UCeqk4EXgLecC7278KXOHD2G49v4EARMEpVjwOU1abzkbcD40TkHuAHp21jjDEes5kBbHizMcZUSVWHN4fX7afGGGOCjiUaY4wxrrJEY4wxxlWWaIwxxrjKEo0xxhhXRfSoMxHZDlRvcXpojO9G0XASbn0Kt/6A9SkUhFt/4Jd9aqmqfk/DEdGJpiZEJK8qw/tCQbj1Kdz6A9anUBBu/YGa98lKZ8YYY1xlicYYY4yrLNFU31ivA3BBuPUp3PoD1qdQEG79gRr2ya7RGGOMcZWd0RhjjHGVJZpqEJFBIrJcRPJFZIzX8fhDRF4WkW0isqjEtkYi8rmIrHR+NnS2i4g84fRvgYic5F3k5RORTBH5SkSWishiEbnR2R6S/RKRBBGZLSLznf78n7O9lYjMcvrzjrNEBs4yGu84/ZklIllexl8REYkWkR9E5GPndUj3SUTWishCEflRRPKcbSH5ewcgIg1EZIKILHP+PfWtzf5YoqkiEYkGngYGA52AESLSyduo/PIqMKjUtjHAVFXNBqY6r8HXt2zncR3wbIBirKoi4GZV7Qj0AUY5/y9CtV9HgDNVtRvQHRgkIn2AB4BHnf7sBq519r8W2K2qbYFHnf2C1Y3A0hKvw6FP/VW1e4lhv6H6ewfwODBZVTsA3fD9v6q9/qiqParwAPoCU0q8vgO4w+u4/Iw9C1hU4vVyIN15ng4sd54/D4woa79gfuBb7O7scOgXUBeYB/wK341yMc72n37/8K3L1Nd5HuPsJ17HXkZfmjtfVGcCH+Nbej3U+7QWaFxqW0j+3gH1gTWl/zvXZn/sjKbqMoANJV4XONtCUZqqbgZwfjZxtodcH50SSw9gFiHcL6fE9COwDfgcWAXsUd8S5fCfMf/UH+f9QiAlsBH75THgNuCE8zqF0O+TAp+JyFwRuc7ZFqq/d62B7cArTnnzRRFJpBb7Y4mm6qSMbeE2dC+k+igi9YD3gD+q6t6Kdi1jW1D1S1WPq2p3fGcBvYGOZe3m/Az6/ojIOcA2VZ1bcnMZu4ZMnxynqOpJ+MpIo0Tk9Ar2DfY+xQAnAc+qag/gAD+XycpS5f5Yoqm6AiCzxOvmwCaPYqmprSKSDuD83OZsD5k+ikgsviTzpqr+29kc8v1S1T3A1/iuPTUQkeJl10vG/FN/nPeT8S2JHkxOAc4TkbXAOHzls8cI7T6hqpucn9uA9/H9URCqv3cFQIGqznJeT8CXeGqtP5Zoqm4OkO2MmokDhgMTPY6puiYCI53nI/Fd4yje/htndEkfoLD4FDqYiIgALwFLVfWREm+FZL9EJFVEGjjP6wC/xndR9ivgYme30v0p7ufFwJfqFM2DhareoarNVTUL37+VL1X1CkK4TyKSKCJJxc+BAcAiQvT3TlW3ABtEpL2z6SxgCbXZH68vRIXiAxgCrMBXP/9fr+PxM+a3gc3AMXx/kVyLr/Y9FVjp/Gzk7Cv4RtatAhYCuV7HX06fTsV3yr4A+NF5DAnVfgFdgR+c/iwC/upsbw3MBvKBd4F4Z3uC8zrfeb+1132opH/9gI9DvU9O7POdx+Li74BQ/b1zYuwO5Dm/ex8ADWuzPzYzgDHGGFdZ6cwYY4yrLNEYY4xxlSUaY4wxrrJEY4wxxlWWaIwxxrjKEo0x1SQi+52fWSJyeS23/edSr7+vzfaNCSRLNMbUXBZQpUTjzAJekf9INKp6chVjMiZoWKIxpubuB05z1ib5kzMx5oMiMsdZr+P3ACLST3zr57yF70Y3ROQDZ2LGxcWTM4rI/UAdp703nW3FZ0/itL3IWQ/lshJtf11iTZE3nZkTEJH7RWSJE8tDAf+vYyJeTOW7GGMqMQa4RVXPAXASRqGq9hKReOA7EfnM2bc30EVV1ziv/0tVdzlTzswRkfdUdYyIjFbf5JqlXYjvLu5uQGPnmGnOez2AzvjmnfoOOEVElgAXAB1UVYunuDEmkOyMxpjaNwDfXFA/4lu2IAXfIlEAs0skGYAbRGQ+MBPfRIXZVOxU4G31zfK8FfgG6FWi7QJVPYFvOp4sYC9wGHhRRC4EDta4d8ZUkSUaY2qfAP+jvtUXu6tqK1UtPqM58NNOIv3wTZzZV32rav6Ab66vytouz5ESz4/jW1isCN9Z1HvA+cDkKvXEmFpgicaYmtsHJJV4PQW43lnCABFp58zyW1oyvmWLD4pIB3xLAhQ7Vnx8KdOAy5zrQKnA6fgmnyyTs1ZPsqpOAv6Ir+xmTEDZNRpjam4BUOSUwF7Ft/56FjDPuSC/Hd/ZRGmTgf8WkQX4lsOdWeK9scACEZmnvmn1i72Pb+nj+fhmrr5NVbc4iaosScCHIpKA72zoT9XrojHVZ7M3G2OMcZWVzowxxrjKEo0xxhhXWaIxxhjjKks0xhhjXGWJxhhjjKss0RhjjHGVJRpjjDGuskRjjDHGVf8Pj9aCYq2p0VkAAAAASUVORK5CYII=\n", "text/plain": [ "

" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot_lr()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`MixedPrecision`](/callbacks.fp16.html#MixedPrecision)\n", "\n", "Use fp16 to [take advantage of tensor cores](https://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html) on recent NVIDIA GPUs for a 200% or more speedup." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`HookCallback`](/callbacks.hooks.html#HookCallback)\n", "\n", "Convenient wrapper for registering and automatically deregistering [PyTorch hooks](https://pytorch.org/tutorials/beginner/former_torchies/nn_tutorial.html#forward-and-backward-function-hooks). Also contains pre-defined hook callback: [`ActivationStats`](/callbacks.hooks.html#ActivationStats)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`RNNTrainer`](/callbacks.rnn.html#RNNTrainer)\n", "\n", "Callback taking care of all the tweaks to train an RNN." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`TerminateOnNaNCallback`](/callbacks.tracker.html#TerminateOnNaNCallback)\n", "\n", "Stop training if the loss reaches NaN." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`EarlyStoppingCallback`](/callbacks.tracker.html#EarlyStoppingCallback)\n", "\n", "Stop training if a given metric/validation loss doesn't improve." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`SaveModelCallback`](/callbacks.tracker.html#SaveModelCallback)\n", "\n", "Save the model at every epoch, or the best model for a given metric/validation loss." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 00:07

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
00.6791890.6465990.80422000:02
10.5274750.4972900.90824300:02
20.4647560.4624710.91707600:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn = Learner(data, simple_cnn((3,16,16,2)), metrics=accuracy)\n", "learn.fit_one_cycle(3,1e-4, callbacks=[SaveModelCallback(learn, every='epoch', monitor='accuracy')])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "best.pth\t bestmodel_2.pth model_1.pth model_4.pth stage-1.pth\r\n", "bestmodel_0.pth bestmodel_3.pth model_2.pth model_5.pth tmp.pth\r\n", "bestmodel_1.pth model_0.pth\t model_3.pth one_epoch.pth trained_model.pth\r\n" ] } ], "source": [ "!ls ~/.fastai/data/mnist_sample/models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`ReduceLROnPlateauCallback`](/callbacks.tracker.html#ReduceLROnPlateauCallback)\n", "\n", "Reduce the learning rate each time a given metric/validation loss doesn't improve by a certain factor." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`PeakMemMetric`](/callbacks.mem.html#PeakMemMetric)\n", "\n", "GPU and general RAM profiling callback" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`StopAfterNBatches`](/callbacks.misc.html#StopAfterNBatches)\n", "\n", "Stop training after n batches of the first epoch." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`LearnerTensorboardWriter`](/callbacks.tensorboard.html#LearnerTensorboardWriter)\n", "\n", "Broadly useful callback for Learners that writes to Tensorboard. Writes model histograms, losses/metrics, embedding projector and gradient stats. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [`train`](/train.html#train) and [`basic_train`](/basic_train.html#basic_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`Recorder`](/basic_train.html#Recorder)\n", "\n", "Track per-batch and per-epoch smoothed losses and metrics." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`ShowGraph`](/train.html#ShowGraph)\n", "\n", "Dynamically display a learning chart during training." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`BnFreeze`](/train.html#BnFreeze)\n", "\n", "Freeze batchnorm layer moving average statistics for non-trainable layers." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`GradientClipping`](/train.html#GradientClipping)\n", "\n", "Clips gradient during training." ] } ], "metadata": { "jekyll": { "keywords": "fastai", "summary": "Callbacks implemented in the fastai library", "title": "callbacks" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.2" } }, "nbformat": 4, "nbformat_minor": 2 }