{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## List of callbacks" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [], "source": [ "from fastai.gen_doc.nbdoc import *\n", "from fastai.vision import *\n", "from fastai.text import *\n", "from fastai.callbacks import * \n", "from fastai.basic_train import * \n", "from fastai.train import * \n", "from fastai import callbacks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "fastai's training loop is highly extensible, with a rich *callback* system. See the [`callback`](/callback.html#callback) docs if you're interested in writing your own callback. See below for a list of callbacks that are provided with fastai, grouped by the module they're defined in.\n", "\n", "Every callback that is passed to [`Learner`](/basic_train.html#Learner) with the `callback_fns` parameter will be automatically stored as an attribute. The attribute name is snake-cased, so for instance [`ActivationStats`](/callbacks.hooks.html#ActivationStats) will appear as `learn.activation_stats` (assuming your object is named `learn`)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [`Callback`](/callback.html#Callback)\n", "\n", "This sub-package contains more sophisticated callbacks that each are in their own module. They are (click the link for more details):" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`LRFinder`](/callbacks.lr_finder.html#LRFinder)\n", "\n", "Use Leslie Smith's [learning rate finder](https://www.jeremyjordan.me/nn-learning-rate/) to find a good learning rate for training your model. Let's see an example of use on the MNIST dataset with a simple CNN." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "path = untar_data(URLs.MNIST_SAMPLE)\n", "data = ImageDataBunch.from_folder(path)\n", "def simple_learner(): return Learner(data, simple_cnn((3,16,16,2)), metrics=[accuracy])\n", "learn = simple_learner()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The fastai librairy already has a Learner method called [`lr_find`](/train.html#lr_find) that uses [`LRFinder`](/callbacks.lr_finder.html#LRFinder) to plot the loss as a function of the learning rate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n" ] } ], "source": [ "learn.lr_find()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAH7dJREFUeJzt3Xl8XGXZ//HPlUyWNunepC1NN0pLLWVtqJR9p/anIujjQx9FFIGfPoILgj/94UsRVFweF1BBkU1cQBERigqF57FAZWsLdKFsXYCm0KaF7rTNzJzr+WPOTKZp0qY0Z9bv+/WaV+acc8+caybJXHMv577N3REREQGoyHcAIiJSOJQUREQkQ0lBREQylBRERCRDSUFERDKUFEREJENJQUREMpQUREQkQ0lBREQyYvkOYG8NHjzYR48ene8wRESKyvz589e5e8OeyhVdUhg9ejTz5s3LdxgiIkXFzF7rTjk1H4mISIaSgoiIZCgpiIhIhpKCiIhkKCmIiEiGkoKIiGQoKYiISIaSgohIEfjpwy/z2CtrIz+PkoKISBH4xT+X8sSytyI/j5KCiEiBc3fiSSdWGf1HtpKCiEiBSwQOQFWFRX4uJQURkQIXTwYAVMVUUxARKXvxZKqmEFNNQUREEmFNoVo1BRERaa8pKCmIiJS9TJ9CpZqPRETKXntSUE1BRKTspYekxlRTEBGRtoRqCiIiEspcvKaagoiIJNSnICIiaW1hUtCQVBERIZFU85GIiIQ0JFVERDIyVzSrpiAiIokgnPtINQUREUk3H2mRHRERyTQfqaNZRETU0SwiIu0SWmRHRETStByniIhkZPoUdEWziIgktMiOiIikpZuPKtWnICIi8cCpqjTMlBRERMpePBHkZDgqRJgUzOwWM2s1s8VdHP+YmS0Mb4+b2aFRxSIiUswSgedkOCpEW1O4DZi2m+MrgBPc/RDgauDGCGMRESlabcmA6hwMRwWIRfXE7v6omY3ezfHHszafBJqiikVEpJglkkFOFtiBwulT+DTwj3wHISJSiBJJpyqWm+ajyGoK3WVmJ5FKCsfupsxFwEUAI0eOzFFkIiKFoS0Z5OTCNchzTcHMDgFuAs5097e6KufuN7p7s7s3NzQ05C5AEZECkEh6ThbYgTwmBTMbCfwFONfdX85XHCIihS6ezN2Q1Miaj8zsDuBEYLCZtQDfBKoA3P2XwDeAQcD14QUZCXdvjioeEZFiFQ88JwvsQLSjj2bs4fgFwAVRnV9EpFQkkgHVpd58JCIi3RMvwyGpIiLShXg5dDSLiEj3xJMB1cU+95GIiPSMshiSKiIi3ZPLIalKCiIiBS4eKCmIiEgokfScLMUJSgoiIgUvngxydvGakoKISIGLJ52qElhkR0REeoA6mkVEJCM1JFVJQUSk7Ll7ajlOdTSLiEgycADVFEREBBJhUlCfgoiI0JYMAHSdgoiIpDqZAWIakioiIvF0TSGm5iMRkbKXSQpaZEdERNLNR1UxNR+JiJS9dE1By3GKiAjxdE1Bo49ERCTTp6DrFEREJBGEzUdKCiIi0pZQ85GIiITSNQU1H4mISPuQVCUFERFpywxJVfORiEjZU01BREQy4polVURE0nSdgoiIZMTVfCQiImntF6+p+UhEpOxlagqaEE9ERNoX2VFNQUSk7CU0dbaIiKS1lcrU2WZ2i5m1mtniLo6bmV1nZkvNbKGZHRFVLCIixSqRDIhVGGZFnhSA24Bpuzn+PmBceLsIuCHCWEREilIi8JwNR4UIk4K7Pwq8vZsiZwK3e8qTQH8zGxZVPCIixagtEeRsOCrkt09hOLAya7sl3CciIqFEEJRGTaEbOkt93mlBs4vMbJ6ZzVu7dm3EYYmIFI54wnPWyQz5TQotwIis7Sbgjc4KuvuN7t7s7s0NDQ05CU5EpBDEgyBnw1Ehv0nhPuAT4Siko4CN7v5mHuMRESk48aRTHcvdR3Usqic2szuAE4HBZtYCfBOoAnD3XwJ/B6YDS4F3gE9FFYuISLFKD0nNlciSgrvP2MNxBz4X1flFREpBPOnEyqSjWURE9iCeDKguk45mERHZg0QQqKYgIiIp5TQkVURE9iBeRheviYjIHsSTSgoiIhJKJD2nQ1KVFERECphqCiIikhFPFmBHs5mNNbOa8P6JZvZ5M+sfbWgiIpJIFuaQ1LuBpJkdANwMjAH+EFlUIiICpJbjLMTmo8DdE8BZwE/d/UuAFsQREYlYaj2FAms+AuJmNgM4D7g/3FcVTUgiIpKWKNCawqeAqcB33H2FmY0BfhddWCIiAtCWzO1ynN2aJdXdlwCfBzCzAUAfd/9elIGJiEiqo7mq0BbZMbPZZtbXzAYCC4BbzezH0YYmIlLekoETOAXZfNTP3TcBZwO3uvtk4NTowhIRkXgyAMhp81F3k0LMzIYBH6W9o1lERCKUTgrVBVhTuAp4EFjm7nPNbH/glejCEhGRRNKB3NYUutvRfBdwV9b2cuDDUQUlIiKpabOBwrui2cyazOweM2s1szVmdreZNUUdnIhIOYuHNYVCXI7zVuA+YD9gODAz3CciIhFJpDuaC21IKtDg7re6eyK83QY0RBiXiEjZS3c0V8UKLymsM7OPm1llePs48FaUgYmIlLt081FVAS6ycz6p4airgTeBj5Ca+kJERCKSHn1UcBevufvr7v5Bd29w90Z3/xCpC9lERCQibQV88VpnLu2xKEREZBfpjuaCqyl0IXepS0SkDMULtfmoC95jUYiIyC7aL14rkCuazWwznX/4G9ArkohERASAeCL3cx/tNim4e59cBSIiIjtLBLmf+yh36UdERPZKvICvaBYRkRxrn/tISUFEpOwliuw6BRERiVC8yK5TEBGRCLVfp6CagohI2Su5moKZTTOzl8xsqZl9tZPjI83sn2b2rJktNLPpUcYjIlJMSmpIqplVAr8A3gdMBGaY2cQOxb4O/MndDwfOAa6PKh4RkWKTqSmUyJDUKcBSd1/u7m3AncCZHco40De83w94I8J4RESKSjwZUFlhVORwPYXdXtG8j4YDK7O2W4D3dihzJTDLzC4B6oBTI4xHRKSoJJJOLIcJAaKtKXT2SjrOozQDuM3dm4DpwG/NbJeYzOwiM5tnZvPWrl0bQagiIoWnLRnk9MI1iDYptAAjsrab2LV56NPAnwDc/QmgFhjc8Ync/UZ3b3b35oYGLQ0tIuUhkfScdjJDtElhLjDOzMaYWTWpjuT7OpR5HTgFwMzeQyopqCogIgIkgoBYqdQU3D0BXAw8CLxAapTR82Z2lZl9MCz2ZeBCM1sA3AF80t21ToOICNCW8Jw3H0XZ0Yy7/x34e4d938i6vwQ4JsoYRESKVaqmUDrNRyIisg/iySCnVzODkoKISMGKl9iQVBER2QfxZEB1TDUFERGh9C5eExGRfRBPltCQVBER2TfxEruiWURE9kEiKK0rmkVEZB+0JTQkVUREQonAc7oUJygpiIgULF28JiIiGakhqUoKIiJCuqag5iMREUHNRyIikqXUFtkREZF9UGrLcYqIyD7QxWsiIgJAEDjJIPejjyJdea2QLG3dzIPPryEZOIE7QeAkAieeDIgnnbZkQPtKoIYZJJOpMskgIOkQqzAqzFI/K1I/K8N9FQZmYJbK6snwF5oMnKQ7qafuaqXR9m8CZqktM8LnDc9TaVSGzx04uIevwyEIn9/dqQjjqawwOn6/sOw4w9do4Ukt6/zhO7DLa+r4uPR26pWlX2MYd4VRaVBRYZnYdrfOavv528+V2Z/et9N7tGsclRU73yzrNe90zAyn/X0zg0rb+bGxiopdni/9+66OVVBVWUFN+LMyx7NYSnmIBwFAzqfOLpuk8NLqLfzwwZcy22apD/nqygpilRVUVaY+TB0yH+DpD4dYeCz9IZ8IApJB6kMlkQx2/mAOP/piFRVUGMQqUz/J/hDO0vGD0jPPQyZ5JYN0cvLMh3RF1gdhOoGQ/ZgOS12nn9PDRJL+kNaK2PuuMvw7qo6lEkVtVSW1VamfddUx6moqqauJUVcTo09NjPqaGPW1Mfr3rmJgXQ0De1czqL6ahj41OR9pIoUrkUx/luT2S0fZJIVpk4by0renZb4Rpr+RSrt0TWmXBJJVC/Cs7XQZ6/BNPpWYIBnWZtLf2NPf6nc9b/izYxw7HWsvlN6fHVf6nKmEvXMNKvD2Gls6sVdYe5J2UjW7dO0x6Z5VS0w/LvVFIJ4MSCQDdiQC2pIB8YTTlkzSlghoS6T270gEbI8n2RZP8s6OJG9s2M7WtgRbdyTYvD3BjkTQ6ftvBoPraxjWr5Zh/WoZMaA3Iwb2ZuTA3owf2of9+tXq77aMxJOpv5Ncf1Eom6SQagKozHcYBa1j082u9RrpCfFkwJbtCda/08b6d9p4a0sbb21tY/XG7azeuJ03N21n2dqtzH5p7U4JZHB9DYeN6MdhI/pz9AGDObSpv5quSlg8rCnk+uK1skkKIoWiqrKCAXXVDKir3m05d2ft5h28/vY7vPDmJp5duYEFKzfw8AutMOtlBvSu4vjxDZw8oZEzDhpKbZW+9JQS1RREZCdmRmPfWhr71tI8eiDnTk3tX7+1jceWrmP2S6088tJa7n3uDfrUxjjr8OGcc+RIJu7XN7+BS4/I9CkoKYjI7gyoq+aDh+7HBw/djyBwnlrxNn+c+zp3zl3J7U+8xpTRA/na9AkcPnJAvkOVfZAefaTmIxHptooKY+rYQUwdO4hvvRPn7mdauH72Ms66/nE+cOh+fOWMAxkxsHe+w5R3IV/NRxr/JlIi+vWu4vxjxzD78hP5/MkH8NCS1Zzyo0e49uFXMh8wUjzyNSRVSUGkxNTXxLj09AOZfdlJnDFpKD95+GU+fMPjLG3dnO/QZC+0pWsKOb54TUlBpEQN7VfLz2YczvUfO4KVb7/D9OvmcNNjywkCXbFYDNI1hSotsiMiPWn6wcOY9aUTOH5cA9/+2wucd+vTtG7enu+wZA/STX6aEE9EelxDnxp+/YnJXHP2wTy94m2mX/sYj7y8Nt9hyW6oo1lEImVmzJgykpmXHMuguhrOu+Vpvn3/Era1JfMdmnQikacrmpUURMrM+CF9uPfiYzj3qFHcNGcF0659lDmvrMt3WNKBagoikjO1VZVc/aFJ3HHhUVSY8fGbn+LLf1rA+q1t+Q5NQvFANQURybGpYwfxjy8cx8UnHcC9z61i+nWPMf+19fkOS4B4QjUFEcmD2qpKLjvjQP76uWOoqqzg33/1BDc9tjxr0SnJh0SQHn2kuY9EJA8mDe/HzEuO5fK7FvDtv73AUyveZvrBQ2nsU0tjnxqGD+hF72p9ZORKZursUlpkx8ymAdcClcBN7v69Tsp8FLiS1FonC9z9P6KMSUS61q9XFb86dzI3z1nB9x94kYeWrMkc61VVyeVnHMh5R4/WOg45UHJTZ5tZJfAL4DSgBZhrZve5+5KsMuOArwHHuPt6M2uMKh4R6R4z44Lj9mfGlJGs3rSdtZt30Lp5B/c808JV9y/h/oVv8IOPHMIBjX3yHWpJa586u3RqClOApe6+HMDM7gTOBJZklbkQ+IW7rwdw99YI4xGRvVBXE2NsQz1jG+oB+MAhw/jrc6v41swlTL92DucdPYqPNo9g3BAlhyi0lVpNARgOrMzabgHe26HMeAAz+xepJqYr3f2BCGMSkXfJzDjr8CaOG9fA1fcv4ZZ/vcqvH1vBoU39OPuIJo4cPZAxg+voVa0V4HpC+8VrpZMUOqvzdBzOEAPGAScCTcBjZjbJ3Tfs9ERmFwEXAYwcObLnIxWRbhtcX8O15xzO1//PRO59bhV3P7OKb973fOb4fv1qGT+0Dx8+oolpk4bm/EOtVMSTAWbkvP8myqTQAozI2m4C3uikzJPuHgdWmNlLpJLE3OxC7n4jcCNAc3OzxsmJFICGPjVccNz+XHDc/ixt3cKLqzexfO1WVqzbytxX3+aSO56lsU8NH3vvKGa8dwSNfWrzHXJRiQdBXhJqlElhLjDOzMYAq4BzgI4ji/4KzABuM7PBpJqTlkcYk4hE4IDGeg5orM9sJwPnkZdb+c3jr/GTh1/m+tlL+cTUUXzmhLEMqq/JY6TFI5H0nA9HhQiTgrsnzOxi4EFS/QW3uPvzZnYVMM/d7wuPnW5mS4AkcLm7vxVVTCKSG5UVxskThnDyhCEsX7uFX/xzGTfPWcHvn3qd848Zw4XH70+/XlX5DrOgxZNBzhfYAbBiu2qxubnZ582bl+8wRGQvLW3dwk8ffpn7F75JY58avv2hSZx+0NB8h1WwvvaXRTy0ZA3zvn5qjzyfmc139+Y9lVMPkIjkxAGN9fz8P47gvouPYVB9DRf9dj4X/+EZ1m3Zke/QClI8GVCd42sUQElBRHLskKb+3HfxMXz5tPHMen4Np/34ER7OunJaUhLJIOfzHoGSgojkQVVlBZecMo6/ff5Y9uvfiwtun8f3H3iRRHjBlqTmPsr11cygpCAieTRuSB/u/uzRzJgyghtmL+Pcm59m7WY1JwG805agNpb7CwE15aGI5FVtVSXXnH0Ik0cN5Ip7FnHSf83msBH9OaSpH4eO6M9R+w8qy5FKL7y5mffuPzDn51VSEJGC8JHJTUwa3pfbn3iNBSs3cOOjy0kEzuD6aq45+xBOmzgk3yHmzNrNO1i9aTsHD++X83MrKYhIwZgwtC/fPetgALbHkzz7+gauun8JF94+j482N/GNDxxEfU3pf2wtXrURIC9JQX0KIlKQaqsqmTp2EPd+7hj+88Sx/Hl+C9N++ih/nPs6G7fF8x1epBat2ogZHKSkICKys+pYBV+ZNoG7PjOV3tWV/L+7F3Hkdx7ms7+bz6znVxMExXUBbncsbNnImMF1eakVlX49TERKwuRRA3nwi8ezsGUjf31uFTMXvME/Fq9mwtA+XH7GgZw8oRGz0lgRbvGqjRyVh05mUE1BRIqImXHoiP588wMH8eTXTuHacw5jezzJp38zjw/f8DhPLCv+qdPSncyT8tB0BEoKIlKkYpUVnHnYcB669ASuOftg3tiwnRm/fpLzbnk601FbjPLZyQxKCiJS5KoqK5gxZSSzLz+R/z99As+t3MD7fzaHS+54llfXbc13eHttYUv+OplBSUFESkRtVSUXHT+WR79yEp87aSwPL1nDqT9+hK//dRGtm7bnO7xuW7RqI/vnqZMZlBREpMT061XF5WdM4JHLT2TGlJHc+fRKjv/hP/n+Ay/yxoZt+Q5vjxav2pi3piPQ6CMRKVGNfWu5+kOTuOC4Mfxo1svcMHsZv3xkGVP3H8TZ4frRhXYhXOvm7XntZAYlBREpcaMG1XHdjMP58unjuefZVdzz7Couu2sBV9yziGMPGMypE4dwynsaC2IN6XQn8yFN/fMWg5KCiJSFUYPq+OKp4/nCKeN45vX13L/wTR5asob/frEVgBPGN3D5GQfm9Vv6opZNqU7m/frmLQYlBREpK2bG5FEDmTxqIN94/0ReWrOZBxav5rbHX+X9P5vD+w8ZxpdPP5Axg+tyHlu6k7kuj81aSgoiUrbMjAlD+zJhaF/OP3YMNz26nJvmrOAfi1dz9NhBnDKhkVPeM4QRA3vnJJ5FqzZw9NjBOTlXV5QURESAvrVVXHr6gZw7dTQ3z1nBrCWruXLmEq6cuYQxg+sYM7iOpgG9aBrQi/qaKrbuSLBlR4Jt8SSHjejP6ROH7NPyma2bt7Nm0468Nl+BkoKIyE4a+tTw1fdN4Kvvm8CKdVv5nxdbeXL5W7Ss38bcV99m8/bETuWrKo140hnWr5aPHzWKGVNGMrCueq/PO//V9UD+rmROM/fimmGwubnZ582bl+8wRKRMbdwWZ1tbkrqaSnpXp75X/8+Lrdz2+Ar+tfQtYhXGqEG9M7WLkQN709i3lsY+NTT2rWVY31oqKnaeuO+FNzcx49dPUl8T4+FLT6C2queX4TSz+e7evKdyqimIiOyFfr2qdlke9LSJQzht4hBeWbOZe597g6WtW1ixbiuPvbKOHYlgp7LjGuv51pkHZfoOlrZu4eM3PUVtrJI/XHBUJAlhb6imICISkSBw1m3ZQevmHazZtJ2W9du4ac5yVr69jQ8cuh/nHjWKS+54hmQAf/y/RzG2oT6yWLpbU1BSEBHJoe3xJDfMXsYNjyyjLREwoHcVd140lQOH9on0vGo+EhEpQLVVlXzptPGcfcRwbpmzgn8/cmTkCWFvKCmIiOTBqEF1fOvMSfkOYxeaJVVERDKUFEREJENJQUREMpQUREQkQ0lBREQylBRERCRDSUFERDKUFEREJKPoprkws43AK50c6gds3M2+jsfT252VGQysexfhdRZDd453FVtn253dL5S4uxNr9v2o4+5OjF3t21282fvy/Z6Xy99K9v18x16s7/kod2/Y49ncvahuwI3d3Z+9r+Px9HZnZYB5PRnb3sa+u+0u4i2IuLsTay7j7k6Me/Oe628lf38rhRR7Mb/n3bkVY/PRzL3YP3M3x2d2o8ze2tPjuxv77rY7u18ocXfcl++4uyrTnX17irdQ3vNy+Vvpzrn3RO95NxRd81EumNk878ZsgoVGcedescZerHFD8cZeLHEXY00hF27MdwDvkuLOvWKNvVjjhuKNvSjiVk1BREQyVFMQEZGMkk4KZnaLmbWa2eJ38djJZrbIzJaa2XVmZlnHLjGzl8zseTP7Qc9GnTlHj8duZlea2Sozey68TS+GuLOOX2ZmbmaDey7inZ4/ivf8ajNbGL7fs8xsvyKJ+4dm9mIY+z1m1r9I4v638P8yMLMeb7/fl5i7eL7zzOyV8HZe1v7d/i9E6t0MWSqWG3A8cASw+F089mlgKmDAP4D3hftPAh4GasLtxiKK/UrgsmJ7z8NjI4AHgdeAwcUSO9A3q8zngV8WSdynA7Hw/veB7xdJ3O8BDgRmA82FEnMYz+gO+wYCy8OfA8L7A3b3+nJxK+magrs/Crydvc/MxprZA2Y238weM7MJHR9nZsNI/TM/4anf0O3Ah8LDnwW+5+47wnO0FlHskYsw7p8AXwEi6wSLInZ335RVtC6K+COKe5a7J8KiTwJNRRL3C+7+Uk/Huq8xd+EM4CF3f9vd1wMPAdPy/T9c0kmhCzcCl7j7ZOAy4PpOygwHWrK2W8J9AOOB48zsKTN7xMyOjDTane1r7AAXh00Ct5jZgOhC3ck+xW1mHwRWufuCqAPtxD6/52b2HTNbCXwM+EaEsWbrib+VtPNJfVvNhZ6MO1e6E3NnhgMrs7bTryOvr6+s1mg2s3rgaOCurCa6ms6KdrIv/Q0vRqqqdxRwJPAnM9s/zOiR6aHYbwCuDrevBn5E6h8+Mvsat5n1Bq4g1ZyRUz30nuPuVwBXmNnXgIuBb/ZwqDsH00Nxh891BZAAft+TMXamJ+POld3FbGafAr4Q7jsA+LuZtQEr3P0sun4deX19ZZUUSNWMNrj7Ydk7zawSmB9u3kfqwzO7utwEvBHebwH+EiaBp80sIDWnydooA6cHYnf3NVmP+zVwf5QBh/Y17rHAGGBB+E/XBDxjZlPcfXWBx97RH4C/EXFSoIfiDjs+3w+cEvWXnlBPv9+50GnMAO5+K3ArgJnNBj7p7q9mFWkBTszabiLV99BCPl9frjov8nUDRpPVKQQ8DvxbeN+AQ7t43FxStYF0R8/0cP9ngKvC++NJVf+sSGIfllXmS8CdxRB3hzKvElFHc0Tv+bisMpcAfy6SuKcBS4CGqN7rKP9WiKij+d3GTNcdzStItTwMCO8P7M7ri/R3kqsT5eMG3AG8CcRJZd9Pk/rW+QCwIPyj/0YXj20GFgPLgJ/TfqFfNfC78NgzwMlFFPtvgUXAQlLfuIYVQ9wdyrxKdKOPonjP7w73LyQ1F83wIol7KakvPM+FtyhGTUUR91nhc+0A1gAPFkLMdJIUwv3nh+/1UuBTe/O/ENVNVzSLiEhGOY4+EhGRLigpiIhIhpKCiIhkKCmIiEiGkoKIiGQoKUhJMLMtOT7fTWY2sYeeK2mpWVQXm9nMPc1Iamb9zew/e+LcIh1pSKqUBDPb4u71Pfh8MW+fEC5S2bGb2W+Al939O7spPxq4390n5SI+KS+qKUjJMrMGM7vbzOaGt2PC/VPM7HEzezb8eWC4/5NmdpeZzQRmmdmJZjbbzP5sqbUFfp+e1z7c3xze3xJOerfAzJ40syHh/rHh9lwzu6qbtZknaJ8IsN7M/tvMnrHU3PpnhmW+B4wNaxc/DMteHp5noZl9qwffRikzSgpSyq4FfuLuRwIfBm4K978IHO/uh5OatfS7WY+ZCpzn7ieH24cDXwQmAvsDx3RynjrgSXc/FHgUuDDr/NeG59/j3DXhHD+nkLraHGA7cJa7H0FqHY8fhUnpq8Aydz/M3S83s9OBccAU4DBgspkdv6fziXSm3CbEk/JyKjAxa/bKvmbWB+gH/MbMxpGafbIq6zEPuXv2fPlPu3sLgJk9R2remzkdztNG++SC84HTwvtTaZ8H/w/Af3URZ6+s555Pal59SM17893wAz4gVYMY0snjTw9vz4bb9aSSxKNdnE+kS0oKUsoqgKnuvi17p5n9DPinu58Vts/Pzjq8tcNz7Mi6n6Tz/5m4t3fOdVVmd7a5+2Fm1o9UcvkccB2p9RcagMnuHjezV4HaTh5vwDXu/qu9PK/ILtR8JKVsFqn1CwAws/T0xv2AVeH9T0Z4/idJNVsBnLOnwu6+kdSSnZeZWRWpOFvDhHASMCosuhnok/XQB4Hzw7n9MbPhZtbYQ69ByoySgpSK3mbWknW7lNQHbHPY+bqE1LTnAD8ArjGzfwGVEcb0ReBSM3saGAZs3NMD3P1ZUrNtnkNqYZtmM5tHqtbwYljmLeBf4RDWH7r7LFLNU0+Y2SLgz+ycNES6TUNSRSISrhq3zd3dzM4BZrj7mXt6nEg+qU9BJDqTgZ+HI4Y2EPHSpyI9QTUFERHJUJ+CiIhkKCmIiEiGkoKIiGQoKYiISIaSgoiIZCgpiIhIxv8CXLu0qPI/O6cAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this example, a learning rate around 2e-2 seems like the right fit." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "lr = 2e-2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`OneCycleScheduler`](/callbacks.one_cycle.html#OneCycleScheduler)\n", "\n", "Train with Leslie Smith's [1cycle annealing](https://sgugger.github.io/the-1cycle-policy.html) method. Let's train our simple learner using the one cycle policy." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
00.1191910.0711950.97252200:02
10.0574190.0427370.98429800:02
20.0317920.0282590.98773300:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit_one_cycle(3, lr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The learning rate and the momentum were changed during the epochs as follows (more info on the [dedicated documentation page](https://docs.fast.ai/callbacks.one_cycle.html))." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuoAAAEKCAYAAABEy3C5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd8lfX5//HXlU0gCZAFhBFG2FOWIsuFYhUEJ45Ka4ttv9ph7a9oq7VaqtY66qh11lUnikVFUZG9BGSPkBD2yGCEJJB9/f44N5rGQA4hJ/c5Odfz8bgfOec+933nfRh3rtznc18fUVWMMcYYY4wx/iXE7QDGGGOMMcaY77NC3RhjjDHGGD9khboxxhhjjDF+yAp1Y4wxxhhj/JAV6sYYY4wxxvghK9SNMcYYY4zxQ1aoG2OMMcYY44esUDfGGGOMMcYPWaFujDHGGGOMHwpzO4CbEhISNDU11e0Yxhhz2latWpWnqolu52hIds42xgSyupy3g7pQT01NZeXKlW7HMMaY0yYiO93O0NDsnG2MCWR1OW/b0BdjjDHGGGP8kBXqxhhjjDHG+CEr1I0xxhhjjPFDVqgbY4wxxhjjh6xQN8YYY4wxxg/5tFAXkUtEJF1EMkVkag2vR4rIO87ry0Uk1Vl/kYisEpH1ztfzq+wz0FmfKSJPiog461uKyBcikuF8beHL92aMMcYYY4wv+axQF5FQ4BlgLNATmCQiPattdgtwWFW7AI8DDzvr84DLVbUPcDPwepV9ngWmAGnOcomzfiowR1XTgDnOc2OMMcYYYwKSL/uoDwEyVTULQETeBsYDm6psMx64z3k8HXhaRERVV1fZZiMQJSKRQEsgVlWXOsd8DbgC+NQ51mhnn1eBecDv6/1dmTOiqmw5UMD6vfkcyC8mNEQICxHimoQT3yyShGYRdEpoRlx0uNtRjTGNwOcbD5CRU0ir2CjO7hxPSvMmbkcyxgSIo8VlbM8tIreghINFJRw+VkZxWQW928RxYc/kBsngy0I9Bdhd5fkeYOjJtlHVchHJB+LxXFE/4UpgtaqWiEiKc5yqx0xxHier6n7nWPtFJKmmUCIyBc8Vedq3b1+X92XqoLJSmbF6L/+av42MnMJat0+MiaRrcjPOat+CQaktOat9c2KirHg3xpyeuek5vPX1dz+KBqe24DcXdmVYlwQXUxlj/E1RSTmrdx1hxY5DfLPrMOkHCsgpKKlx2xuGtm8UhbrUsE5PZxsR6YVnOMyY0zjmKanq88DzAIMGDTqtfU3d7D50jF++vZrVu47Qq00s0yb05tzOCbRp3gRFqahUDh8r42BhCdlHS8jKLSQzp5DNB47yzNxMKhXCQoShnVpyUY9kxvRqRRu7KmaM8cKDE/vyp8t7sevQMeZszuHVJTu4/sXlXD2wLX8a14tmkUE9QbcxQS37aDGfbzzAF5tzWLotj7IKJUSgW6tYRqQl0iWpGZ0Sm9IqNor4ZhG0bBpBVFgoISE1laO+4csz1B6gXZXnbYF9J9lmj4iEAXHAIQARaQvMAH6oqtuqbN/2JMfMFpHWztX01kBOfb4ZUzdLtx3k1tdXosBj1/Tjiv4pNf4Dj44Iq/KR9He/pRaWlLNm1xEWZuby5aZs7vtoE3/+eBPndk7g6kFtubhXK6LCQxvmzRhjAlJUeChdk2PomhzDj85N5emvMnlmXibr9+bz6o+HkBwb5XZEY0wDKS2vZM7mbN5ZuZsFW3OpVEiNj2bysFSGpyUyoH1zYv3oE3xfFuorgDQR6QjsBa4Drq+2zUw8N4suBa4CvlJVFZHmwCfAXaq6+MTGThFeICJnA8uBHwJPVTvWQ87X//rsnRmvzN2Sw61vrKJDy2heunkw7eOjT/sYzSLDGJ6WwPC0BO4a24Os3EJmrt3Heyv38Ku319CyaQQ3nd2BH57TgfhmkT54F8aYxiQqPJQ7L+7GkI4t+dkbq7jy2SW897NzaB1nn9IZ05jlHyvjtaU7eHXpDvIKS2kVG8UvRnfhigFt6JzYDKeJoN8RVd+N/hCRS4EngFDgZVWdJiL3AytVdaaIROHp6DIAz5X061Q1S0T+CNwFZFQ53BhVzRGRQcArQBM8N5He7hT38cC7QHtgF3C1qh46Vb5BgwbpypUr6/MtG8eqnYe4/oXldE2O4bUfD6FF04h6PX5lpbIs6yAvL97Ol5tziAoP4eqB7bjt/C52dcwEBRFZpaqD3M7RkOr7nL129xFueHE5bZpH8d6tw+wmdmMaoZyCYl5YkMWby3dRVFrBed0S+eE5qYzsmkhoAw5hgbqdt31aqPs7K9R9Y/ehY1z+9CJaREcw/Wfn+PxKd2ZOAS8s2M4Hq/cQGiLcMrwjt47q7FcfXRlT36xQrx9LMvOY/O8VnN05nn9PHtzgP7iNMb5RVFLO8wuyeGFhFiXllVzetzW3jupMj9axrmWqy3nbZiY19aq0vJLb3lpNRYXy78mDG2Q4SpekGB6+qi9z7hjNxb1a8czcbYz821xeX7aTysrg/UXUGFO7YV0SuG9cLxZszeWprzJq38EY49cqK5V3Vuxi9N/n8Y85GYzulsicO0bxxHUDXC3S68oKdVOv/vbZFtbuPsLfrupLakLTBv3e7eOj+cd1A/j49uH0aBXLPR9uYOKzS9i4L79BcxhjAsukIe2YOCCFJ+dksGb3EbfjGGPqKCO7gGueW8rv319P+5bRvP/zYfzzhoENXo/UJyvUTb1ZlnWQFxdt56azOzC2T2vXcvROiePNnw7l8Wv7sfvQMcY9vZgHZ22mpLzCtUzGGP8lItw3vhfJsVH87r21dq4wJsCUllfy2BdbufTJhWTmFvLIVX2Z/rNzGNihhdvRzpgV6qZeFJdVcPeM9bRr2YS7Lu3udhxEhAkD2jLnt6O4emBbnluQxfinF7PlwFG3oxlj/FBsVDh/ndCHjJxCnpqT6XYcY4yXMnMKmPDPxTw5J4PL+rZhzh2juHpQO7/t4nK6rFA39eKf87aRlVvEtCv6EB3hPxOINI+O4KEr+/LvyYPJKyxl3FOLeWFBlo1dN8Z8z3ndk5g4IIXnFmxjR16R23GMMaegqryxbCeXPbWI/fnFPH/TQB6/tn+ja9Vshbo5Y7sOHuNf87ZxRf82jOya6HacGp3XPYnZvx7B6G6JTJu1mSmvryT/WJnbsYwxfmbq2O6Eh4bw4Keb3Y5ijDmJguIyfv7GN/zxww0MTm3JZ78awZherdyO5RNWqJsz9rfZWwgNEaaO7eF2lFOKbxbJczcN5M/jejEvPZfLn15kN5oaY/5HUmwUvxjdmdkbs1myLc/tOMaYajJzChj/zGK+2JzN3Zd259UfDSGpEc+fYoW6OSOrdx3m43X7+emIjrSK8///KCLCzcNSeefWcygtr2TiP5fw4eq9bscyxviRn4zoRErzJjw4awvBPNeIMf7m0/X7Gf/0Yo4eL+M/PxnKlJGdCWnkcx9YoW7qTFV58NMtJDSLZMqozm7HOS0DO7Tg418Op3+75vz6nTU8/sVW+4FsjAEgKjyUX12Yxvq9+Xy1JcftOMYEPVXlmbmZ/Pw/39C1VQwf3z6CszvFux2rQVihbupsWdYhvt5+iNvP70KzSP+5gdRbCc0ief2WoVw1sC3/mJPBHe9aWzZjjMeEASm0bxnNE19m2C/xxriorKKSqe+v55HZ6Yzv34a3p5wdEJ/g1xcr1E2dPT03g8SYSK4d3M7tKHUWERbCI1f15XcXd2PG6r3c+OJyDheVuh3LmIAhIpeISLqIZIrI1Bpe7yAic0RknYjME5G21V6PFZG9IvJ0w6WuXXhoCLed38WuqhvjoqPFZfz4lRW8s3I3t5/fhSeu7U9kWKjbsRqUFeqmTlbtPMzizINMGdGJqPDA/k8jIvzfeV14atIA1u7J59rnl5J9tNjtWMb4PREJBZ4BxgI9gUki0rPaZn8HXlPVvsD9wIPVXn8AmO/rrHVx4qr6U19ZX3VjGlpeYQnXPreMpdsO8rcr+/LbMd0aTW/002GFuqmTp7/KoEV0ODec3d7tKPXm8n5teOVHg9l7+DhX/2spuw8dczuSMf5uCJCpqlmqWgq8DYyvtk1PYI7zeG7V10VkIJAMfN4AWU9beGgItwzvyJrdR1i187DbcYwJGvuOHOeafy1le14hL00ezDUB/Mn9mbJC3Zy2jfvymZuey09GdPKryY3qw7DOCfznp2dztLiMK59dwtbsArcjGePPUoDdVZ7vcdZVtRa40nk8AYgRkXgRCQEeBX7n85Rn4KqBbYmNCuPlRdvdjmJMUNh5sIir/7WU3IISXr9lKKP8dH6WhmKFujltLy/aQXREKDee3cHtKD7Rv11z3plyDgDXPLfUeq0bc3I1fQ5d/c7LO4FRIrIaGAXsBcqBXwCzVHU3pyAiU0RkpYiszM3NrY/Mp6VpZBjXD+3Apxv226dsxvhYRnYBV/9rKcdKy3nzp2czOLWl25Fc59NC3YubjCJF5B3n9eUikuqsjxeRuSJSWPUGIxGJEZE1VZY8EXnCeW2yiORWee0nvnxvwSqvsISP1u7jqoFtiWsS7nYcn+nWKobpPxtGdHgoN764nM37j7odyRh/tAeo+pl0W2Bf1Q1UdZ+qTlTVAcAfnHX5wDnAbSKyA8849h+KyEPVv4GqPq+qg1R1UGKiO1fWbh7WgRARXlmyw5Xvb0wwyMwpZNILywB459Zz6NM2zuVE/sFnhbqXNxndAhxW1S7A48DDzvpi4B48V2K+paoFqtr/xALsBD6ossk7VV5/sf7flXlz+S5KKyq5eViq21F8rn18NG9NOZvIsFBueHE56QdsGIwx1awA0kSko4hEANcBM6tuICIJzjAXgLuAlwFU9QZVba+qqXjO9a+p6vcu6PiD1nFN+EHf1ryzYjeFJeVuxzGm0dmeV8T1LywDhLemnE3X5Bi3I/kNX15R9+Ymo/HAq87j6cAFIiKqWqSqi/AU7DUSkTQgCVhY/9FNTUrLK3l92U5GdU2kc2Izt+M0iA7xTXlrytmEhQg3vLiMzBwr1o05QVXLgduA2cBm4F1V3Sgi94vIOGez0UC6iGzFc+PoNFfCnqGbh6VSWFLOzDX7at/YGOO1XQePcf0LyyivVN786dCgqS+85ctC3ZubjL7dxjnh5wPeTjU1Cc8V9KrjIa90evVOF5HgvUXYRz7dsJ/cghImn5vqdpQG1THBU6yLCJNeWM6ugzZO1ZgTVHWWqnZV1c6qOs1Zd6+qznQeT1fVNGebn6hqSQ3HeEVVb2vo7KdjQLvmdG8Vw1tf73I7ijGNxt4jx5n0wjKOl1Xwxi1D7Up6DXxZqHtzk5E325zMdcBbVZ5/BKQ6vXq/5Lsr9f/7DV2+MSmQvb50Jx0TmjIqLfjuwO6c2Iw3fzKUsopKbnp5ObkF36s1jDGNmIgwaUh71u/NZ/0eu8HcmDN1qKiUm15aztHiMt64ZSg928S6Hckv+bJQr/Umo6rbiEgYEAccqu3AItIPCFPVVSfWqerBKldqXgAG1rSvP9yYFIgycwpYufMwk4a0IyQk+CYcAEhLjuHlyYPJOVrCzS9/zdHiMrcjGWMa0BUDUogMC+GtFXZV3Zgzcay0nB+/soI9h4/z0s2D6Z1iN46ejC8L9VpvMnKe3+w8vgr4qtpQlpOZxP9eTUdEWld5Og7PeElTT95ZsZuwEGHiWW1r37gRO6t9C5698Sy2Zhcw5bWVFJdVuB3JGNNA4pqEc1nfNsxcs48iu6nUmDopq6jk5298w7o9R3hq0gCGdLQWjKfis0Ldy5uMXgLiRSQTuAP49o5/p2XXY8BkEdlTrWPMNVQr1IFfishGEVkL/BKY7IO3FZRKyit4/5u9XNQzmYRmkW7Hcd3obkk8ek0/lmUd4ldvr6ai0tvRWsaYQHf90HYUlpTz8Tq7qdSY01VZqfy/6euYvzWXaRP6cHGvVm5H8ns+nVZSVWcBs6qtu7fK42Lg6pPsm3qK43aqYd1deFp/mXr25aYcDhWVcm0QT+Fb3fj+KRwsLOX+jzfxwMebuG9cL7cjGWMawFntW9A5sSnvr9rLtYPbux3HmIDy8OwtzFi9l99e1JVJQ+z/jzdsZlJTq7dX7KJNXBQjgvAm0lP58fCO/GR4R15ZsoNXFtv04sYEAxFhwoAUvt5xyGYqNeY0vP31Lp6bn8WNZ7fntvO7uB0nYFihbk5p35HjLMrM46pB7QgN0ptIT+WuS3twYY9k7v94E19tyXY7jjGmAYzv7+k0/N81e11OYkxgWJyZxx8/3MCoroncd3kvRKye8JYV6uaUZq7dhypceVb1FvgGIDREeHJSf3q0juX2N1ezad9RtyMZY3ysXctohnRsyQer9+Jd/wNjgldmTiE/f2MVnRKb8tT1AwgLtdLzdNifljmlD1fv5az2zekQ39TtKH4rOiKMl24eTExUOLe8uoLsoyedUNcY00hMGJBCVm4R66ynujEndaiolFteXUF4aAgv3TyY2KhwtyMFHCvUzUlt3n+ULQcKuGKAXU2vTau4KF6aPIj842VMeX2VtW00ppG7tE9rIsJCmLHahr8YU5PS8kp+9sYq9ucX8/wPB9GuZbTbkQKSFermpD5cs5ewEOEHfVrXvrGhV5s4HrumP2t3H+He/26wj8SNacTimoRzYY8kPl63j/KKSrfjGON3/vLJJr7efohHrurLwA4t3I4TsKxQNzWqrFRmrtnHqK6JxFvvdK9d0rsVt5/fhXdX7uGNZTvdjmOM8aHL+rYhr7CUr7fXOqG2MUHl3ZW7eW3pTqaM7PTtzdembqxQNzVavv0Q+/OLGW/DXk7bby7syvndk/jzR5vsB7gxjdh53ZJoEh7KJ+v3ux3FGL+xbs8R/vjhBs7tEs//u7ib23ECnhXqpkYfrt5L04hQLuqR7HaUgBMSIjx+bX/atYzmF/9Zxf78425HMsb4QJOIUM7vkcRnGw7Y8BdjgLzCEn72+ioSm0Xy1KSzrMNLPbA/QfM9peWVfLphPxf3bkWTiFC34wSkuCbhPH/TQI6XVvAzu7nUmEbrB31ac7DIhr8YU1ZRyf/95xsOFpXy3E0Dadk0wu1IjYIV6uZ7Fm/L42hxOZf3beN2lICWlhzDY9f2Z+2efP780Sa34xhjfODE8JePbfiLCXIPfbqF5dsP8eDEPvROiXM7TqNhhbr5nlnr9hMTFcawLvFuRwl4F/dqxc9Gdeatr3fxobVxM6bROTH8ZbYNfzFB7LMNB3hp0XYmD0tl4llt3Y7TqFihbv5HWUUln2/K5qIeyUSG2bCX+nDnmK4MSW3J3TPWk5lT4HYcY0w9u8wZ/rLchr+YILTr4DF+N30t/drGcfelPdyO0+hYoW7+x9JtB8k/XsZY651eb8JCQ3hy0gCahIfyi/98w7HScrcjGWPq0Wjr/mKCVEl5Bbe99Q0CPH39WUSEWVlZ3+xP1PyPTzfsp2lEKCPSEtyO0qi0iovi8Wv7k5FTyL3/3eh2HGNMPWoSEcr53ZP4fGM2lZU20ZkJHg/O2sK6Pfk8cnU/m3nUR6xQN98qr6hk9sZsLuiRTFS4DXupbyO7JnL7eV2YvmoP767c7XYcY0w9GtMrmbzCElbvPuJ2FGMaxGcb9vPKkh38+NyOXNyrldtxGi2fFuoicomIpItIpohMreH1SBF5x3l9uYikOuvjRWSuiBSKyNPV9pnnHHONsySd6ljGe19vP8SholIu7WP/4XzlVxd25ZxO8dzz4QbSD9h4dWMai9HdkggLEb7YlO12FGN8zjMufR392jVn6tjubsdp1HxWqItIKPAMMBboCUwSkZ7VNrsFOKyqXYDHgYed9cXAPcCdJzn8Dara31lyajmW8dIn6/fTJDyUUV2T3I7SaIWGCP+Y1J+YqDB++dZq669uTCMR1yScoZ1a8sWmA25HMcanSssrvxuXPmmAjUv3MV/+6Q4BMlU1S1VLgbeB8dW2GQ+86jyeDlwgIqKqRaq6CE/B7q0aj1X3+MGlolKZvfEA53dPskmOfCwpJopHrupHenYBD326xe04xph6clGPZLblFpGVW+h2FGN85tEv0m1cegPyZaGeAlQdiLvHWVfjNqpaDuQD3jTv/rcz7OWeKsV4XY9lgJU7DpFXWMpYG/bSIM7rnsSPzk3llSU7+GqLfVRuTGNwYc9kABv+YhqtJZl5PL8gi+uHtrdx6Q3El4V6TVezq98O78021d2gqn2AEc5y0+kcS0SmiMhKEVmZm5tby7cKHl9syiYiNITR3WzYS0P5/SXd6d4qhjvfW0fO0dP58MgY44/atoimZ+tYK9RNo3S4qJQ73l1Lx4Sm3POD6iOZja/4slDfA7Sr8rwtsO9k24hIGBAHnHLGCFXd63wtAN7EM8TG62Op6vOqOkhVByUmJp7mW2qcVJUvNmdzTud4mkWGuR0naESFh/LUpAEUlZTz2/fWWls3YxqBi3oms2rXYfIKS9yOYky9UVXunrGeg0UlPHndABsi24B8WaivANJEpKOIRADXATOrbTMTuNl5fBXwlaqetFoRkTARSXAehwOXARvqcizznW25hew8eIyLnI9tTcNJS47hnst6sjAjj5cXb3c7jjHmDF3UMxlV+GpzTu0bGxMg3l25m083HODOMd3onRLndpyg4rNC3RknfhswG9gMvKuqG0XkfhEZ52z2EhAvIpnAHcC3LRxFZAfwGDBZRPY4HWMigdkisg5YA+wFXqjtWObUPnc+pr2ghw17ccMNQ9szpmcyD3+2hQ17892OY4w5A73axJLSvMm351VjAl1WbiH3zdzEsM7x/HREJ7fjBB2fjnNQ1VnArGrr7q3yuBi4+iT7pp7ksANPsv1Jj2VO7ctN2fRJiaN1XBO3owQlEeHhK/ty8RMLuOPdNcy8bbhNOGVMgBIRLuyRxDsrd3O8tMKGCJiAVlZRya/fWUNEWAiPXtOPkBBrptfQrPllkMst8Mykd2EPG/biphZNI/jbVX3Zml3IY19sdTuOMeYMXNgzmeKySpZm5bkdxZgz8sSXW1m3J5+HJvaxi3kusUI9yM3dkoMqNj7dD4zulsQNQ9vzwsIslmUddDuOMV7xYgbqDiIyR0TWOTNLt3XW9xeRpSKy0Xnt2oZP7xtDOrakSXgo89Kts5gJXKt2HubZedu4ZlBbxvZp7XacoGWFepD7YnM2Kc2b0KN1jNtRDHD3pT1o3zKaO99bS0FxmdtxjDklL2eg/jvwmqr2Be4HHnTWHwN+qKq9gEuAJ0SkecMk963IsFDO7RLPV1tysJ4GJhAdL63gzvfW0jquCfdcZq0Y3WSFehA7XlrBwoxcLuyRhE3i6h+aRobx6NX92HfkOH/5eLPbcYypjTczUPcE5jiP5554XVW3qmqG83gfkAM0mp65o7slsefwcbblFrkdxZjT9vBnW9ieV8QjV/UlJirc7ThBzQr1ILY4M4/iskou6mmzi/mTQaktuXVUZ95ZuZsvrXOE8W/ezEC9FrjSeTwBiBGR/5k1WkSGABHANh/lbHCju3l+55iXbm0aTWBZsi2PV5bsYPKwVIZ1SXA7TtCzQj2Ifbk5m5jIMIZ0bOl2FFPNry9Mo3urGKZ+sI6DNnGK8V/ezAh9JzBKRFYDo/C01S3/9gAirYHXgR+pauX3vkGAzibdtkU0XZObMdcKdRNACkvK+d1760iNj+b/XdLN7TgGK9SDVmWl8uXmHEZ1SyQizP4Z+JvIsFAev7Y/+cfL+MOMDTbO1firWmegVtV9qjpRVQcAf3DW5QOISCzwCfBHVV1W0zcI5Nmkz+uWxNfbD1FYUl77xsb4gWmfbGJ//nEevaYf0RE2U7k/sAotSK3fm09eYYm1ZfRjPVrHcsdF3fhs4wFmrt1X+w7GNLxaZ6AWkQQROfGz5i7gZWd9BDADz42m7zVg5gYzulsSZRXK4kxr02j837z0HN76ejc/HdmJgR3sk3Z/YYV6kJq/NRcRGJFm48/82ZSRnejfrjn3zdxIboENgTH+xcsZqEcD6SKyFUgGpjnrrwFG4pl9eo2z9G/Yd+Bbg1Jb0CwyzMapG7+Xf6yM37+/jrSkZvzmwq5uxzFVWKEepOZvzaVvShzxzSLdjmJOITREeOSqvhSVVHDvfze4HceY71HVWaraVVU7q+o0Z929qjrTeTxdVdOcbX6iqiXO+jdUNVxV+1dZ1rj5XupbeGgIw7skMHdLrg1fM37tvo82kldYymPX9LeZsf2MV4W6iAwXkR85jxNFpKNvYxlfyj9WxupdhxnVLcntKMYLackx/OrCND7dcIBP1u13O44x5jSc1z2RA0eLSc8ucDuKMTWavfEAM1bv5bbzutCnbZzbcUw1tRbqIvIn4Pd4xhYChANv+DKU8a2FmblUKozqGlg3ZgWzW0d2ok9KHPf+d4N1gTEmgIx2LojM3RI4HWtM8Mg/XsY9H26gR+tYbju/i9txTA28uaI+ARgHFMG3E1PYNJYBbH56LnFNwulnvzkHjLDQEB65ui9Hi8u476NNbscxxngpOTaKnq1jmbvFxqkb//O3z7aQV1jCw1f2ITzURkP7I2/+VkrVM7hOAUSkqW8jGV9SVeZvzWV4WgJh9p8yoHRvFcvt56fx0dp9zN54wO04xhgvje6WyKpdhykoLnM7ijHfWrHjEP9ZvosfnduRvm2bux3HnIQ3ldq7IvIc0FxEfgp8Cbzo21jGVzbvLyCnoITRNuwlIP18dGd6to7lDzM2cORYqdtxjDFeGNk1kYpKZem2g25HMQaAkvIKpr6/jpTmTbjjIuvy4s9qLdRV9e/AdOB9oBtwr6o+6etgxjfmb/WMk7Tx6YEp3BkCc+RYKffbEBhjAsJZ7VsQHRHKggwbp278w7PztrEtt4i/TOhN00ib2MifeXMz6cOq+oWq/k5V71TVL0TkYW8OLiKXiEi6iGSKyNQaXo8UkXec15eLSKqzPl5E5opIoYg8XWX7aBH5RES2iMhGEXmoymuTRSS3Sj/en3iTMdjM35pDj9axJMVGuR3F1FGvNnH8YnRnPli9lzmbs92OY4ypRURYCOd0imdhhk18ZNyXmVPAP+duY1y/Npxn3d/8njdDXy6qYd3Y2nYSkVDgGWfbnsAkEelZbbNskxggAAAgAElEQVRbgMOq2gV4HDjxC0AxcA9wZw2H/ruqdgcGAOeKSNUs71Tpx2vDc6opLCln5Y7DjO5mV9MD3W3np9G9VQx3z1jPURv3aozfG5GWwM6Dx9h5sMjtKCaIVVYqd32wniYRodx7efWSzPijkxbqIvJzEVkPdBORdVWW7cA6L449BMhU1SxVLQXeBsZX22Y88KrzeDpwgYiIqhap6iI8Bfu3VPWYqs51HpcC3wBtvchigMWZeZRXqg17aQQiwkJ4+Mq+5BaU8LfPtrgdxxhTi5HOedeuqhs3vbViFyt2HOYPP+hBgk14GBBOdUX9TeByYKbz9cQyUFVv9OLYKcDuKs/3OOtq3MaZijofiPcmuIg0d/LMqbL6SueXieki0u4k+00RkZUisjI3N7jGC87fmkuzyDDOat/C7SimHvRr15wfnduRN5btYsWOQ27HMQFORAaJyAwR+cY5j64XEW8uyhgvdExoSkrzJiy0cerGJdlHi3lo1haGdY7n6oF2jTNQnLRQV9V8Vd2hqpNUdSdwHE+LxmYi0t6LY0tNh63DNt8/sEgY8BbwpKpmOas/AlJVtS+ezjSv1rSvqj6vqoNUdVBiYvBcWVZV5qfnMqxzPBFh1paxsfjtmK60bdGEqe+vo6S8wu04JrD9B/g3cCWeiyCXOV9NPRARRnZNYEnmQcorKt2OY4LQfTM3UlpRyV8n9EGkpvLL+CNvbia9XEQygO3AfGAH8KkXx94DVL2q3RbYd7JtnOI7DvDm0uDzQIaqPnFihaoeVNUTUza+AAz04jhBY1tuEXuPHP92ljzTOERHhDFtQh+25RbxzNxtbscxgS1XVWeq6nZV3XlicTtUYzIiLZGCknLW7D7idhQTZD7feIBPNxzglxekkZpg0+EEEm8urf4FOBvYqqodgQuAxV7stwJIE5GOIhIBXIdnGE1VM4GbncdXAV85kyudlIj8BU9B/+tq61tXeToO2OxFxqAxL90zK97IrgkuJzH1bVTXRCYMSOHZeZmkHyhwO44JXH8SkRdFZJKITDyxuB2qMTm3cwIhAgtsnLppQAXFZdz73410bxXDlJGd3I5jTpM3hXqZqh4EQkQkxLmZs39tOzljzm8DZuMpmt9V1Y0icr+IjHM2ewmIF5FM4A7g2xaOIrIDeAyYLCJ7RKSniLQF/oCni8w31dow/tJp2bgW+CUw2Yv3FjTmb82lS1Iz2raIdjuK8YF7LutJTFQ4Uz9YR0VlraPHjKnJj/Cc2y/hu3uSLnM1USMTFx1Ov3bNbZy6aVCPzE4nu6CYh67sS7jNSB5wvOlyf0REmgELgP+ISA5Q7s3BVXUWMKvaunurPC4Grj7JvqknOWyNA6tU9S7gLm9yBZvjpRUs336Im87u4HYU4yMtm0Zwz2U9+M07a3lj2U5uHpbqdiQTePqpah+3QzR2I9ISefqrDPKPlREXHe52HNPIrdp5mNeX7eTmc1Lp366523FMHXjzq9V44BjwG+AzYBt2g1FAWbb9IKXlldY/vZG7on8KI7sm8rfPtrD3yHG345jAs6yGuS5MPRuZlkClwpJtNvzF+FZpeSV3fbCO1rFR3HlxN7fjmDqqtVB3eppXqmq5qr6KZxKjS3wfzdSX+em5RIWHMDi1pdtRjA+JCNOu6E2lwj0fbqCW2z2MqW44sMaZTdraM/pIv3bNiYkMs3Hqxueem7+NrdmFPHBFb5pFejOAwvijU014FCsid4nI0yIyRjxuA7KAaxouojlT87fmck6neKLCQ92OYnysXctofjumK19tyeHjdfvdjmMCyyVAGjAGa8/oM+GhIQzrEs+Crbn2y7TxmW25hTz1VSY/6NuaC3okux3HnIFTXVF/HegGrAd+AnyOZzz5eFWtPsOo8VM7DxaxPa/IZiMNIj86tyP92sZx38yNHC4qdTuOCRx6ksXUsxFpiew9cpzteUVuRzGNUGWlcvcH64kKD+FPl9totkB3qkK9k6pOVtXngEnAIOAyVV3TMNFMfViw1dNdwPqnB4/QEOHBiX3JP17GtFnWpdR47RPgY+frHDyfnnozZ4Y5TSPTPBdOFtrwF+MD763azfLth7j70h4kxUS5HcecoVMV6mUnHqhqBbBdVa1Jc4CZl55Lh/hom+AgyPRsE8uUkZ2YvmoPi6wYMF5Q1T6q2tf5mgYMARa5nasxah8fTYf46G8vpBhTX3IKipn2yWaGdmzJtYPb1b6D8XunKtT7ichRZykA+p54LCJHGyqgqbuS8gqWbDtow16C1C8vSKNjQlPunrGe46UVbscxAUZVvwEGu52jsRqRlsCyrIOUVVS6HcU0In/+aBPF5ZX8dWIfRGrsZm0CzEkLdVUNVdVYZ4lR1bAqj2MbMqSpm5U7DnO8rMIK9SAVFR7KtAm92XXoGE9+leF2HOPnROSOKsudIvImYJd8fWR4l0SKSitYveuI21FMIzFnczafrNvP7ed1oXNiM7fjmHpiU1Q1YvPSc4gIDeGczvFuRzEuGdY5gWsGteX5BVls2mcfhJlTiqmyROIZq26NA3zknM7xhAgssllKTT0oLCnnng830DW5GbeO6ux2HFOPrFBvxOZvzWVIx5ZER1j/1GB296U9aN4knLs+WEdFpTXxMCe1SVX/7CzTVPU/WHtGn4lrEk7/ds2tn7qpF49+ns7+o8U8OLEvEWFW2jUm9rfZSO07cpyt2YU27MXQPDqCey/vydo9+by2dIfbcYz/usvLdaaeDE9LZN2eI+QfK6t9Y2NOYs3uI7yyZAc3nd2BgR1auB3H1DMr1Bup79oyWqFuYFy/Nozulsgjs9PZe+S423GMHxGRsSLyFJAiIk9WWV4Byl2O16iNTEugUmFpll1VN3VTVlHJ1PfXkRwTxe8u7uZ2HOMDtRbqJ7q8VFt2i8gMEenUECHN6ZuXnkubuCi6JNkNJQZEhAfG90YV7vlwg82IaKraB6wEioFVVZaZwMUu5mr0+rVrTrPIMBv+YurshYVZbDlQwP3jexETFe52HOMD3gxefgzPifxNQIDrgFZAOvAyMNpX4UzdlFVUsjgzj8v6tbb2TOZb7VpG89sxXfnLJ5uZtf4AP+jb2u1Ixg+o6lpgrYi8qao2BqMBhYeGcHaneJvrwNTJjrwi/vFlBmN7t2JMr1ZuxzE+4s3Ql0tU9TlVLVDVo6r6PHCpqr4D2GAoP7R61xEKSsptfLr5nsnDUumTEsefZm60cbGmuiEi8oWIbBWRLBHZLiJZbodq7EZ2TWDXoWPsPFjkdhQTQFSVu2esJyIshPvG9XI7jvEhbwr1ShG5RkRCnOWaKq+d8vNzEblERNJFJFNEptbweqSIvOO8vlxEUp318SIyV0QKReTpavsMFJH1zj5PinPJWERaOj9kMpyvQftLxPytOYSFCMO6JLgdxfiZsNAQHpzYh8PHSnnos81uxzH+5SU8n6AOxzPR0SC8mPDIi/N8BxGZIyLrRGSeiLSt8trNzjk7Q0Rursf3EjCGO+fphXZV3ZyG6av2sGTbQaaO7U5ybJTbcYwPeVOo3wDcBOQA2c7jG0WkCXDbyXYSkVDgGWAs0BOYJCI9q212C3BYVbsAjwMPO+uLgXuAO2s49LPAFCDNWS5x1k8F5jhTX89xngeleem5nNWhBbE2Xs3UoHdKHLcM78hbX+9medZBt+MY/5Gvqp+qao6qHjyxnGoHL8/zfwdeU9W+wP3Ag86+LYE/AUOBIcCfgvECS8eEpqQ0b2LDX4zX8gpLmDZrM4NTWzBpcHu34xgfq7VQV9UsVb1cVRNUNdF5nKmqx1V10Sl2HQJkOvuXAm/z/ckzxgOvOo+nAxeIiKhqkXPs4qobi0hrIFZVl6rnbrjXgCtqONarVdYHlZyCYjbuO2rDXswp/frCNNq2aMJdM9ZTUl7hdhzjH+aKyCMico6InHViqWUfb87zPfFcPAGYW+X1i4EvVPWQqh4GvuC7Cy9BQ0QYkZbA4m15lFdUuh3HBIAHPt7EsZIKHpzYh5AQuw+tsfOm60uiiNwtIs+LyMsnFi+OnQLsrvJ8j7Ouxm1UtRzIB041jWaKc5yajpmsqvudY+0HkrzI2Ogs3Oq5KmOFujmV6Igwpk3oQ1ZuEc/M3eZ2HOMfhuIZ7vJX4FFn+Xst+3hznl8LXOk8ngDEiEi8l/sGheFpCRQUl7Nub77bUYyfm5uew3/X7OMX53WmS1KM23FMA/Cm68t/gYXAl8DpXHqr6de86mPavdnmTLb//gFEpuAZOkP79o3vI6P5W3NJjImkV5tYt6MYPzeqayJX9G/Ds/Myubxva9KS7aQfzFT1vDrs5s05+U7gaRGZDCwA9uLpz+7V+byxn7MBzu2cgAgsysjjrPZBN/rHeOlYaTl/nLGBLknN+Pnozm7HMQ3EmzHq0ar6e1V9V1XfP7F4sd8eoF2V523xtHmscRsRCQPigEO1HLNtledVj5ntDI05MUQmp6YDqOrzqjpIVQclJjauq84VlcqCjFxGpiVaW0bjlT9e1pOmkWHc9cF6Kiutt3owE5FkEXlJRD51nvcUkVtq2a3W87yq7lPViao6APiDsy7fm32dbRvtOfuEFk0j6JMSx8KMXLejGD/22Odb2XvkOA9O7ENkWKjbcUwD8aZQ/1hELq3DsVcAaSLSUUQi8PRfn1ltm5nAiTv9rwK+0lPMxOIMaSkQkbOdbi8/xHPFv/qxbq6yPmis23OEI8fKGGWzkRovJTSL5I8/6MnKnYd58+tdbscx7noFmA20cZ5vBX5dyz61nudFJEFETvysuQvP/Bs432uMiLRwbiId46wLSsO7JHha6xZb21Tzfev2HOHlxdu5YWh7Bqe2dDuOaUDeFOq/wlOsH3dmJS0QkaO17eSMOb8Nz4l3M/Cuqm4UkftFZJyz2UtAvIhkAndQpVOLiOzA0ypssojsqdJJ4OfAi0AmsA341Fn/EHCRiGQAFznPg8r8rbmECIywtozmNFx5VgrDOsfz8KdbyD5aXPsOprFKUNV3gUr49hx+yuGOXp7nRwPpIrIVSAamOfseAh7AU+yvAO531gWlEWmJlFcqy7KC9o/AnER5RSVT319PQrNIfj+2u9txTAOrdYy6qtZ54KqqzgJmVVt3b5XHxcDVJ9k39STrVwK9a1h/ELigrlkbg3npufRr15wWTSPcjmICiIjw1wl9uPiJBdw3cyPP3jjQ7UjGHUXOTZ4KICJn47nB/5S8OM9Px9PVq6Z9X+a7K+xB7awOzWkSHsqijFwu6pnsdhzjR15atJ1N+4/yrxvPsrbLQeikV9RFpLvz9ayaloaLaLxxuKiUtXuOWLcXUyepCU355QVpfLrhAJ9vPOB2HOOOO/AMW+ksIovxtL+93d1IwSMyLJShnVqyMNP6qZvv7Dp4jMe/3MqYnslc0ru123GMC051Rf0OPHfaP1rDawqc75NEpk4WZuaham0ZTd1NGdmJj9bu497/buSczvHE2JWboKKq34jIKKAbno4s6apqA6Yb0Ii0RB74eBN7jxwnpXkTt+MYl6kqf/hwPWEhIdw//nsDCUyQOOkVdVWd4nw9r4bFinQ/Mz89lxbR4fRt29ztKCZAhYeG8ODEPmQXFPP32eluxzENzJll9FI8QwjHALeLyB3upgouI9I89xctsu4vBpixei8LM/L4/SXdaBUX5XYc4xJv+qgjIsOA1Krbq+prPspkTlNlpTJ/ay4j0hIJtVnKzBkY0L4FN5+TyqtLdzB+QIr1dA4uH+GZDXo9zg2lpmGlJTUjOTaShRl5XGtTwwe1Q0WlPPDxJgZ2aMENQzu4Hce4qNZCXUReBzoDa/iuA4DiGb9o/MCm/UfJKyyxYS+mXtx5cTdmbzzAXe+v56PbhxMR5k1zKNMItFXVvm6HCGYiwvAuiXy1JZvKSrXp4YPYXz7eRGFJOQ9O7GP/DoKcNz+BBwHnquovVPV2Z/mlr4MZ781L98ztNNIKdVMPmkWG8cD43qRnF/DCwiy345iG86mIjHE7RLAbkZbA4WNlbNxXaxdk00gtzMjlg9V7+fmoznS1GaODnjeF+gagla+DmLqbl55Ln5Q4EmMi3Y5iGokLeyZzaZ9W/GNOBtvzityOYxrGMmDG6c6ZYerXuc48GAtsnHpQOl5awR9mbKBTYlN+cV4Xt+MYP+BNoZ4AbBKR2SIy88Ti62DGO/nHyvhm12FG22ykpp7dd3kvIsNCuPuD9ZxiwmDTeDwKnANEq2qsqsaoaqzboYJNYkwkPVrHsijD2jQGoyfmbGXXoWM8OKEPUeGhbscxfsCbm0nv83UIU3cLM3OpVKxQN/UuKTaKu8b24O4Z63lv1R6uGdTO7UjGtzKADWq/lbluRFoCryzewbHScqIjvOr5YBqBDXvzeXHhdiYNacfQTvFuxzF+4pRnAKdd1z2qemED5TGnaV56LnFNwunfzrpzmPp33eB2zFi9h2mfbOb87kkkNLPhVY3YfmCeiHwKlJxYqaqPuRcpOI1IS+D5BVks336I87oluR3HNIDyikru+mA9LaIjmHpJD7fjGD9yyqEvqloBHBORuAbKY07Dd20ZE6wto/GJkBDhwYl9OF5awf0fbXI7jvGt7cAcIAKIqbKYBjY4tSURYSE2/CWI/HvxDtbvzef+8b2Ii7bJ5sx3vPlMrRhYLyJfAN/eVWadX9y3af9RcgtK7IqL8akuSTH84rzOPPFlBhPOSrF/b42Uqv4ZQERiPE+10OVIQSsqPJShHVtaoR4kdh08xqNfpHNRz2TG9rbeHeZ/eXMz6SfAPcACYFWVxbhs/lZPVwBry2h87eejO9M5sSl/nLGBY6XlbscxPiAivUVkNZ5OXxtFZJWI9HI7V7Aa3iWB9OwCso8Wux3F+JCqcveM9YSHhPDA+N6I2Kfj5n/VWqir6qs1LQ0Rzpza3C051pbRNIjIsFAenNiXvUeO8/gXW92OY3zjeeAOVe2gqh2A3wIvuJwpaA1P87RptKvqjdv73+xlUWYevx/bnVZxUW7HMX6o1kJdRNJEZLqIbBKRrBNLQ4QzJ2dtGU1DG9KxJZOGtOelRdtZvyff7Tim/jVV1bknnqjqPKCpe3GCW49WsSQ0i2Ch9VNvtHILSnjg400MTm3B9UPaux3H+Clvhr78G3gWKAfOA14DXvdlKFM7a8to3DB1bHfim0Uy9YN1lFdUuh3H1K8sEblHRFKd5Y94bjA1LggJEc7tksCizIM2j0Ej9eePNnK8tIIHJ/YlxBpCmJPwplBvoqpzAFHVnap6H3C+NwcXkUtEJF1EMkVkag2vR4rIO87ry0Uktcprdznr00XkYmddNxFZU2U5KiK/dl67T0T2VnntUm8yBipry2jcENcknD+P68XGfUf59+Idbscx9evHQCLwATDDefwjVxMFueFdEsgrLGHLgQK3o5h69uWmbD5et5/bz+9Cl6Rmbscxfsyrri8iEgJkiMhtwF6g1rYPTg/2Z4CLgD3AChGZqapVe7zdAhxW1S4ich3wMHCtiPQErgN6AW2AL0Wkq6qmA/2rHH8vnh8oJzyuqn/34j0FNGvLaNw0tncrLuyRxGNfbOWS3q1o1zLa7UimHqjqYcC6efmREWmeT0wXZuTSo7VNEttYFBSXcc9/N9AtOYZbR3V2O47xc95cUf81EI3nBD4QuBG42Yv9hgCZqpqlqqXA28D4atuMB07cmDoduEA8tzyPB95W1RJV3Q5kOser6gJgm6ru9CJLo3KiLeNoa5NnXCAi3D++NyECf/hwg30sH+BEZOapFrfzBbNWcVGkJTVjod1Q2qg8MjudA0eLeejKPkSEeVOGmWBW6xV1VV0BICKqqqfzMWgKsLvK8z3A0JNto6rlIpIPxDvrl1XbN6XavtcBb1Vbd5uI/BBYCfzWuULU6JxoyzjK2jIal7Rp3oTfXdyN+z7axMy1+xjfv/p/TxNAzsFzHn4LWA7Yx3R+ZERaIv9ZvpPisgqiwkPdjmPO0Modh3h92U4mD0tlQHsbumpq503Xl3NEZBOw2XneT0T+6cWxazrZV7/0drJtTrmviEQA44D3qrz+LNAZz9CY/cCjNYYSmSIiK0VkZW5uYN5NPy89h94psdaW0bjqpnNS6deuOfd/tInDRaVuxzF11wq4G+gN/APPcMU8VZ2vqvNdTWYYkZZASXklK3c0yutOQaWkvIKpH6ynTVwT7hzTze04JkB485nLE8DFwEEAVV0LjPRivz1AuyrP2wL7TraNiIQBccAhL/YdC3yjqtknVqhqtqpWqGolnt6/1YfKnNjueVUdpKqDEhMD74r04aJSVu08zPk27MW4LDREeGhiH/KPlzFt1ma345g6cs6bn6nqzcDZeIYazhOR212OZoChnVoSHioszAzMC0vmO8/M3UZmTiF/mdCbppHe3CJojHeFOqq6u9qqCi92WwGkiUhH5wr4dUD18Y4z+W68+1XAV+oZ8DoTuM7pCtMRSAO+rrLfJKoNexGR1lWeTsAzu16jMzc9h0qFC3okux3FGHq0jmXKyE5MX7WHJZk2jjZQOefaicAbwP8BT+Lp/mJcFh0RxsAOLVi41f5/BbJN+47yz7mZXNG/DefZhTZzGrwp1HeLyDBARSRCRO7EGQZzKqpaDtwGzHa2f1dVN4rI/SIyztnsJSBeRDKBO4Cpzr4bgXeBTcBnwP+pagWAiETj+Wi2+g+Rv4nIehFZh6ff+2+8eG8BZ87mHBJjIumTEud2FGMA+OUFaXSIj+buGespLvPmd3jjT0TkVWAJcBbwZ1UdrKoPqOpel6MZx4i0RDbtP0peYYnbUUwdlFVU8rvpa2keHc6fLu/ldhwTYLwp1H+G5wpLCp4hKf2BX3hzcFWdpapdVbWzqk5z1t2rqjOdx8WqerWqdlHVIaqaVWXfac5+3VT10yrrj6lqvKrmV/teN6lqH1Xtq6rjVHW/NxkDSWl5JfO35nJB9ySbHMH4jajwUP46oQ87Dh7jyTkZbscxp+8moCvwK2CJMz/FUREpEJGjLmczeMapAyy2T60C0nPzt7Fx31H+ckVvWjSNcDuOCTC1FuqqmqeqN6hqsqomqeqNwA8bIJup5uvthygsKbdhL8bvnNslgasGtuX5BVls2me1XSBR1RBVjXGW2CpLjKpa824/0KtNHM2jw61NYwBKP1DAP+ZkcFnf1lzSu3XtOxhTTV0beN5RrymMV77cnE1kWAjDuyS4HcWY7/nDpT1oHh3Bne+tpbS80u04xjQaoSHCuZ0TWJSRZ/MWBJByZ8hLbJRnRmdj6qKuhbqNu2hgqsqcLdmc2yWBJhHWS9f4nxZNI/jrhN5s2n+UZ+Zmuh3HmEZlRFoCB44Wsy230O0oxkvPL8xi3Z587h/fm/hm1k7Z1E1dC3X7lb6BZeQUsvvQcS7oYXeLG/81plcrJgxI4Zm5mWzYm1/7DsYYrwx3xqkvsO4vASEju4AnvshgbO9W/KCvDXkxdXfSQv3EjUQ1LAVAmwbMaPAMewG4oLuNTzf+7U+X96RlU88QmJJy6wJjTH1o2yKaTglNv52Z2vivikrld9PX0TQylPvH93Y7jglwJy3Ua7ixqOoNRtapv4HN2eyZjbRVXJTbUYw5pebRETw4sQ9bDhTw1BwbAmNMfRndLYmlWQc5VlrudhRzCi8tymLN7iPcN66XzSBuzlhdh76YBnSwsIRvdh22q+kmYFzQI5mrBrbl2fnbWLv7iNtxjA+JyCUiki4imSIytYbX24vIXBFZLSLrRORSZ324iLzqzH+xWUTuavj0geX87kmUlleyJPOg21HMSWzLLeTvn29lTM9kxvWzwQfmzFmhHgDmpueiChdaW0YTQO65rCeJzSK58721NhFSIyUiocAzwFigJzBJRHpW2+yPeCa8G4Bnhup/OuuvBiJVtQ8wELhVRFIbInegGtKxJU0jQvkqPcftKKYG5RWV3PHuWpqEh/KXCb0Rsb4b5sxZoR4AvtyUTXJsJL1TrKWxCRxxTcJ56Mo+ZOQU8sSXNhFSIzUEyFTVLFUtBd4GxlfbRoETJ684YF+V9U1FJAxoApQC1oT/FCLCQhielsDcLTnWptEPPTvP8wniX67oTVKMDVM19cMKdT93vLSCeVtzGNOzlf12bgLO6G5JTBrSjucXbOObXYfdjmPqXwqwu8rzPc66qu4DbhSRPcAs4HZn/XSgCNgP7AL+rqqHfJq2ETi/exL784vZcqDA7SimivV78vnHnAzG9WvD5TbkxdQjK9T93PytORSXVTK2dyu3oxhTJ3df2oPWcU248921HC+1ITCNTE1XD6pf6p0EvKKqbYFLgddFJATP1fgKPF3EOgK/FZFO3/sGIlNEZKWIrMzNtY4n53XztOj9aosNf/EXxWUV/ObdNcQ3i+D+8TaxkalfVqj7uc82HKB5dDhDOrZ0O4oxdRITFc4jV/UlK6+IBz/d7HYcU7/2AO2qPG/Ld0NbTrgFeBdAVZcCUUACcD3wmaqWqWoOsBgYVP0bqOrzqjpIVQclJib64C0ElqTYKHqnxDLXCnW/8cjsdDJzCnnkqn40j45wO45pZKxQ92Ol5ZXM2ZzDRT2SCQu1vyoTuIZ1SeAnwzvy2tKdVmA0LiuANBHpKCIReG4WnVltm13ABQAi0gNPoZ7rrD9fPJoCZwNbGix5ADu/WxLf7DrM4aJSt6MEvSXb8nhp0XZ+eE4HRna1XyRN/bPqz48t3pZHQUk5Y/vYsBcT+O68uBvdW8Xwu+nrOFhY4nYcUw9UtRy4DZgNbMbT3WWjiNwvIuOczX4L/FRE1gJvAZPVcyfkM0AzYAOegv/fqrquwd9EADqvexKVik1+5LKjxWXc+e5aOiY0ZerY7m7HMY2UFep+bPaGAzSLDGNY5wS3oxhzxqLCQ3niuv4cPV7G1A/WW9eKRkJVZ6lqV1XtrKrTnHX3qupM5/EmVT1XVfupan9V/dxZX6iqV6tqL1XtqaqPuN8/PfcAACAASURBVPk+Akm/ts2Jbxph49Rddt/MjRw4Wsxj1/QjOsLmgTS+YYW6nyqvqOTzTdmc1z2JqPBQt+MYUy+6t4rl/13SjS82ZfPOit3/v737js+qvP8//vokgYQRAoQZhuxtGCKCiOICnKDFCtZVB1pxa622/frrt7b9dautrdXWXQfDhUUFRStVFAiBsEeQvcLekPX5/nEfNKYBw7hz7jt5Px+P+5H7Pvc5J+8Tbq58cs51ruvbNxCR/5KQYJzVsSGfLN1MYVFx2HGqpPfmbeCN7HWMPrsdPVvWCzuOVGJRLdTLMWNdspmNCd6fXnKyCzN7KFi+xMwGl1i+MpjJbo6ZZZVYXt/MPjCzZcHXuP6fM3PldrbtzWdIV3V7kcrlhv6t6d8unf99ZyErtuwNO45IXDqnUyN27i9gtmb+rXDrduznR6/PpXvzNO44p33YcaSSi1qhXs4Z624Etrt7O+BR4DfBtl2I3JTUFRgC/DXY3yFnB5dQS44Q8CAwxd3bA1OC13Fr0oKNJCclMLCjbk6RyiUhwfj9Fd2pnpTA3WPmUKAzgiJHbUD7hiQlGB8u2hR2lCqlqNi557U5FBU7fxrZk+pJ6pgg0RXNT1h5ZqwbCrwQPB8PnGuRWX2GAq+5+0F3XwHkBvs7kpL7egEYdgKOIRTFxc778zdyZoeG1EpWvzepfJqm1eBXl51Mzpod/Pmj3LDjiMSdtBrV6Nc2nckLNul+jwr0xEe5zFi5jUeGdeOk9Fphx5EqIJqFenlmrPtqnWD0gJ1A+rds68BkM5tlZqNKrNPY3TcE+9oANDpBx1HhslZtZ+OuA1yo0V6kErsosynf6dWcJz5axufLt4YdRyTuDOrSmBVb9pKbtyfsKFVC1sptPD5lKcN6ZHB5r+Zhx5EqIpqFenlmrDvcOkfatr+79yLSpWa0mZ15VKHiYJa7d3LWk5yUwPldVKhL5fbzoV1p1aAWd702W0M2ihylQ78jJi3YGHKSym/n/gLuem0OzevV5JFh3cKOI1VINAv18sxY99U6ZpYEpAHbjrStux/6mge8ydddYjaZWdNgX02BMsetivVZ7gqLinl33gbO69yY2ur2IpVcreQknhjZix37C7hvXA7FxbqEL1JeTdJS6NGiLpMWqJ96NLk7P35zHpt2HeDxET1ITakWdiSpQqJZqJdnxroJwHXB8+HAR8FEGBOAEcGoMK2B9sAMM6tlZqkAwUx2g4hMllF6X9cBb0fpuKLqs+Vb2bo3n0u6Z4QdRaRCdMmow/9c3IV/L9nMPz79Muw4InFlcNcmzFu3k3U79ocdpdIam7WGiXM3cM/5HTQUo1S4qBXq5Zyx7hkg3cxygXsJRmpx9wXAWGAh8D4w2t2LgMbAp8EMdzOAie7+frCvXwPnm9ky4PzgddyZMGc9qclJGu1FqpSrT2vJBd2a8Nv3lzB79faw44jEjcFdGwPwgbq/RMWiDbt4+O0F9G+Xzq1ntQ07jlRBUe1b4e7vAu+WWvZwiecHgCsOs+0vgV+WWvYl0P0w628Fzj3OyKE6UFDE5AUbGdytiSY5kirFzPj1dzKZt+4/3PHqbCbeOYC0Grq8LPJt2jSsTftGtZm0YBPX928ddpxKZfeBAm57OZu0GtV47MqeJCaUdfucSHRpANAY8u8leew+WMil6vYiVVBajWr8eWRPNu48wIOvz9WQcyLlNKhrY2as3Mb2vflhR6k03J0HX5/H6m37+PPInjRMTQ47klRRKtRjyISc9TSoXZ3T26aHHUUkFD1b1uNHQzrx3vyNPPPpirDjiMSFwV2bUFTsTFlc5hgKcgxe/HwVE+dt4P5BHTmtjX4nS3hUqMeInfsK+HBRHhdnZpCUqH8WqbpuGtCaIV2b8P/fW8wXX2p8dZFvc3KzNDLSUnh/vvqpnwhz1uzgFxMXcm6nRtxyZpuw40gVp4owRkyYu578wmKGn6JJFKRqMzN+d0UmJ6XX5PZXstm480DYkURimpkxuFsTpi7bzK4DBWHHiWs79uUz+uVsGqWm8IfvdidB/dIlZCrUY8T4rDV0apJK14w6YUcRCV1qSjWeuvoU9uUXMfqVbPILi8OOJBLTLumeQX5hMR9oTPVjVlhUzB2vziZv9wGeuKondWtWDzuSiAr1WLB0025y1u5k+CnNMdNf7yIA7Run8tvhmcxatZ1fvbso7DgiMa1ni7o0q1uDd+aWnldQyut3k5bwn2VbeGRoN42XLjFDhXoMGJe1hqQE47KezcKOIhJTLs7M4KYzWvP8tJW8NXtd2HFEYpaZcUn3DD5dtoVtGv3lqL09Zx1PTf2Sq/u2ZESflmHHEfmKCvWQFRQV8+bs9ZzTqRHptTX8k0hpP7qgE31a1+dHr88lZ82OsOOIxKxLujelsNh1U+lRmr9uJw+Mn0ufVvV5+OKuYccR+QYV6iH7eHEeW/Yc5IreLcKOIhKTqiUm8OT3etEwNZmbX8zSzaUih9GlaR3aNKzFOznq/lJeW/cc5JaXZlG/VnX+8r1eVE9SWSSxRZ/IkL30xSqapqVwdseGYUcRiVnptZN55rpT2XuwkJtfzGJ/flHYkURijplxSWYGX6zYSt4u/UH7bQqKihn9Sjab9xzkqWtO0aRGEpNUqIdoxZa9/GfZFq7q01Jjp4t8i45NUvnTyJ7MX7+T+8flUFysmUtFSruke1PcYeK8DWFHiWnuzk/fnM8XX27j15efTGbzumFHEimTqsMQvfzFKpISjCv7qNuLSHmc27kxD13QiYnzNvD4lGVhxxGJOe0apdK5aR3emqPuL0fy5CfLGZO1hjvOacflvTR/icQuFeoh2Z9fxLhZaxncrQmNUlPCjiMSN24e0IYrTmnO41OWMUF9cUX+y3d6NSNnzQ6WbdoddpSY9K+56/nt+0u4tHsG957fIew4IkekQj0k78xdz879BVzT96Swo4jEFTPjF5d1o0+r+tw/Nodpy7eEHUkkpgzr2YykBGN89tqwo8ScWau2c+/YHHqfVI/fDs/U3CUS81Soh8DdefbTFXRoXJvTWtcPO45I3ElOSuTv1/bmpPSa3PLiLBZv3BV2JJGY0aB2MgM7NuLN7HUUFmlW30NWb93HqBezaJqWwtPX9ialWmLYkUS+lQr1EHyydDOLN+7m5gFt9Ne8yDFKq1mNF27oQ63kJK57dgbrduwPO5JIzBh+SnPydh/kP7m64gSQt/sA1zw7nSJ3nrv+VOrXqh52JJFyiWqhbmZDzGyJmeWa2YNlvJ9sZmOC96ebWasS7z0ULF9iZoODZS3M7GMzW2RmC8zsrhLr/8zM1pnZnOBxYTSP7Xg89cmXNKmTwtAemolU5Hhk1K3B8zecyr78Iq5/dgY79mlGRhGAczo1ol7NaozPUveXnfsLuO7ZmWzefZDnrj+VNg1rhx1JpNyiVqibWSLwF+ACoAsw0sy6lFrtRmC7u7cDHgV+E2zbBRgBdAWGAH8N9lcI3OfunYG+wOhS+3zU3XsEj3ejdWzHI2fNDj7/cis3ntFaEyuInACdmtTh6Wt6s2rrPm56IYt9+YVhRxIJXfWkBIb2aMYHCzexfW/V/QN2f34RN70wk9y83fzt6lPo2bJe2JFEjko0K8U+QK67f+nu+cBrwNBS6wwFXgiejwfOtUhfkKHAa+5+0N1XALlAH3ff4O7ZAO6+G1gExNVp6aemLqdOShIjT2sZdhSRSqNf23QeG9GD7NXbufnFLA4UaEIkkRF9WpBfVMy4WWvCjhKKgqJibn8lm6xV23n0yh6c2UETC0r8iWah3gwo2Tqs5b+L6q/WcfdCYCeQXp5tg24yPYHpJRbfbmZzzexZMyvzz2YzG2VmWWaWtXnz5qM9puOybNNu3pu/kWv6nUTt5KQK/d4ild2FJzfl91d0Z9ryrdz2cjb5hbqJTqq2Tk3q0KdVff75xeoqN0FYUbHzw3E5TFmcxyNDu3FxZkbYkUSOSTQL9bLukizdUhxunSNua2a1gdeBu9390HAPTwJtgR7ABuAPZYVy96fdvbe7927YsGL/uv7jB0upVT2Jm85oU6HfV6SquLxXc3457GQ+WpzHna/O1ogXFaAc9yK1DO4tmh2cSLmwxHuZZvZ5cM/RPDPTpBIn2NX9TmL1tn1MXVaxJ6bCdKhIf2vOen44uCNXaxhkiWPRLNTXAiWn3GwOlJ6d5Kt1zCwJSAO2HWlbM6tGpEh/2d3fOLSCu29y9yJ3Lwb+TqTrTcyYt3Yn783fyI1ntKae7jYXiZqrTmvJwxd34f0FG7lvXA5FVexMYkUq571IPwXGuntPIvce/TXYNgn4J3Cru3cFBgIFFRS9yhjStQkNalfnn1+sCjtKhSgqdn44Poc3Zq/j/kEdGH12u7AjiRyXaBbqM4H2ZtbazKoTaaAnlFpnAnBd8Hw48JG7e7B8RDAqTGugPTAj6L/+DLDI3f9Yckdm1rTEy8uA+Sf8iI7D7ycvoW7Natw0oHXYUUQqvRvOaM0DQzry9pz13D1mDgU6sx4t5bkXyYE6wfM0vj5hMwiY6+45AO6+1d11c8EJVj0pgRGntmTK4jzWbNsXdpyoKip2Hhg/lzey13Hv+R24/Zz2YUcSOW5RK9SDPue3A5OI3PQ51t0XmNnPzezSYLVngHQzywXuBR4Mtl0AjAUWAu8Do4MGvD9wDXBOGcMw/ja4dDoXOBu4J1rHdrQ+XpLHJ0s3c9vAtqSmVAs7jkiVcNvAdjx0QSfeyVnPbS9nc7BQNWAUlOdepJ8BV5vZWuBd4I5geQfAzWySmWWb2QPRDltVjTytJQaV+qx6QVEx94/L4fXstdxzXgfuPFdFulQOUb2jMRgi8d1Syx4u8fwAcMVhtv0l8MtSyz6l7P7ruPs1x5s3Gg4WFvHzdxbSpkEtrj9dZ9NFKtItZ7WlRvVEHn57ATe9kMXT1/SmRnXNRngCledepJHA8+7+BzPrB7xkZt2I/P45AzgV2AdMMbNZ7j7lG9/AbBQwCqBlS42WdSya1a3BRZkZvDx9Nbed3Y60GpXrhNGBgiJufyWbDxflcf8gnUmXykUDeUfZc5+tZMWWvTx8SReNmy4Sgmv7teK3wzP5LHcL1z03g5371Q36BCrPvUg3ErlCirt/DqQADYJtP3H3Le6+j8hJnV6lv0GYAwBUJree1YY9Bwsr3Vn1XQcKuPbZGZHRXYZ1U5EulY4qxyhasWUvj3+4jPM6N2Zgx0ZhxxGpsr7buwWPj+jJ7NXbGf7kNNZur9x9dStQee5FWg2cC2BmnYkU6puJdIvMNLOawY2lZxHp7ihR0DUjjTM7NOS5z1ZWmnkGNu8+yMinvyB71XYeu7IH12h0F6mEVKhHSWFRMfeNnUO1ROMXw7qFHUekyrukewYv3NCHjbsOcNlfpzF/3c6wI8W9ct6LdB9ws5nlAK8C13vEduCPRIr9OUC2u0+s+KOoOn5wVlu27DnI+Flrw45y3JZs3M2wv3zG8s17+Pu1vRnaI67mPhQpNxXqUfLU1C/JXr2DR4Z1o0mahgYWiQWnt23A6z84neqJCXz3qc/5eHFe2JHinru/6+4d3L1tcG8R7v6wu08Ini909/7u3t3de7j75BLb/tPdu7p7N3fXzaRR1rdNfXq0qMtTU5fH9YRgHy/J4ztPTqOgqJixt/Tj7E66Yi2Vlwr1KPgsdwt/mLyEizKbcml3zYYmEks6NE7lzdtOp3WDWtzwwkz+8nEukVFhRSo3M+Ou89qzZtt+xsxcHXaco+buPP/ZCm58fiYt69fk7dv7k9m8btixRKJKhfoJtmLLXm5/JZu2DWvzm+9kEhn6XURiSaM6KYy7tR8XZ2bwu0lLuPWfs9hzsDDsWCJRN7BDQ/q0rs/jU3LZlx8/n/n9+UXcP24uP3tnIed0asy4W/vRNK1G2LFEok6F+gm0fsd+rv7HdMyMp6/tTe3kqI5+KSLHoWb1JP40ogc/vagzHy7KY+gTn5KbtyfsWCJRZWb8aEhHtuw5yLOfrgg7Trks37yHYX/5jDdmr+XOc9vz1DWnUEu/X6WKUKF+gizbtJsr/vY5u/YX8OINfWjdoFbYkUTkW5gZNw1ow0s39mH7vgIu+fOnvDpjtbrCSKV2ykn1Ob9LY57893I27jwQdpzDcnfemr2OS//8KXm7D/DC9/tw7/kdSEzQlWqpOlSoH6dDDcnlT04jv6iYV27uS7dmaWHHEpGjcHrbBrx75wBOOakeD70xj1EvzWLb3vywY4lEzf9c1IXCYueRibE5Iub2vfnc/sps7h4zh05N6zDxzgGc2UHj6EvVo0L9GBUUFTN16Wau+vt07h4zh/aNavPmbadzcnMV6SLxqElaCi/e0IefXtSZT5ZsZtCjU5k4d4POrkul1DK9JqPPbsfEuRuYunRz2HG+4aPFmxj02FQmL9zIDwd3ZMyovmTUVX90qZrUyeso3Tc2hyWbdrFqyz52HyykYWoy/++SLlzbr5Uux4nEuYSESFeY09s24IHXcxj9SjbndmrEz4d1o5kKBalkbjmrDW/OXsdDb8zj3bsGkFajWqh5NuzczyP/Wsi78zbSsXEqz3//VLpm6OSXVG0q1I9SUXExDWsn06NFXc5o15CBHRuSUi0x7FgicgJ1yajDW7f15/lpK/nD5KWc/8dPGH12O27o35oa1fX/XSqH5KRE/vjd7gz/2+f8z1vzeXxEj1BGKssvLOaFaSt59MOlFBU79w/qwM1ntiE5Sf/XRFSoH6XHRvQMO4KIVICkxARuGtCGwV2b8PN/LeR3k5bw0ueruG9QBy7v1VxX0KRS6NmyHvec157fT17KgPYNuKJ3iwr73kXFzttz1vHoh0tZs20/53ZqxM8u7UqL+jUrLINIrFOhLiJyBC3q1+Tv1/Zm+pdb+dV7i/nh+Lk8NfVLbj2rLUN7ZFAtUbf6SHz7wcB2fJa7lZ+8OZ/WDWrRu1X9qH6/wqJi3l+wkT9NWcbSTXvomlGH57/fjYEdNcOoSGlWlW+U6t27t2dlZYUdQ0TihLszcd4Gnvgol8Ubd5ORlsINZ7Rm+CnNqVuzeoVmMbNZ7t67Qr9pyNRmR8+Offlc9tdp7NiXz8s39aVLRp0T/j12HyhgzMw1PPfZStbt2E+bBrW4b1BHLujWhARdoZIq4Fja7aieCjKzIWa2xMxyzezBMt5PNrMxwfvTzaxVifceCpYvMbPB37ZPM2sd7GNZsM+K/a0pIpWemXFxZgbv3TWA564/leb1avKLiYvo86sp3PHqbP6zbDNFxVX35IfEr7o1q/P8908lpVoiV/3jC7JXbz8h+y0udqblbuHesXM47VdT+MXERTSrV4OnrzmFD+49i4sym6pIFzmCqJ1RN7NEYClwPrAWmAmMdPeFJda5Dch091vNbARwmbtfaWZdgFeBPkAG8CHQIdiszH2a2VjgDXd/zcz+BuS4+5NHyqizMyJyvBas38m4rLW8OXsdO/cXUL9Wdc7p1IjzOjemX5t00mpGZyQNnVGXaFizbR/f+8d0Nuzcz08u7Mw1xzCi2e4DBUxbvpWPFuXx8ZI88nYfJDU5iYsym3LVaS3JbF43SulFYtuxtNvR7KPeB8h19y8BzOw1YChQcnaFocDPgufjgScscsv5UOA1dz8IrDCz3GB/lLVPM1sEnANcFazzQrDfIxbqIiLHq2tGGl0vTePBCzoxZVEekxduZPKCjYyftRaAjo1T6XVSPTo0rk37Rqm0alCTBrWTNVqUxKQW9Wsy4fb+3DNmDj97ZyHjZq1l1JltOK9zY2olf7NkyC8sJm/3AVZu2ceKLXtYuGEX2at2sDRvN+6QmpzEmR0aMrhbEwZ1aazPvMgxiGah3gxYU+L1WuC0w63j7oVmthNID5Z/UWrbZsHzsvaZDuxw98Iy1hcRibqUaolclNmUizKbUlBUzKxV28lauY2ZK7czce56Xj1Q+I31U5OTqFOjGinVEvjxhZ05t3PjkJKLfFPdmtV59vpTeWfuBn4/aQl3vTYHM8hIq0FSolFY5OzYl8/e/KJvbFcnJYkeLetxwclNOK11Or1b1dPN1iLHKZqFelnXykr3szncOodbXtb/+COt/9+hzEYBowBatmxZ1ioiIselWmICfduk07dNOhC5CXXznoPk5u1h9dZ9bN2bz+bdB9m1v4CDRcXUCXmiGZHSzIxLu2dw8clNmb5iGzNWbGPl1r0Uu5OYYNStUZ16NauRXjuZ1g1q0aZhLRqlJocyDrtIZRbNQn0tUHJA1ubA+sOss9bMkoA0YNu3bFvW8i1AXTNLCs6ql/W9AHD3p4GnIdLf8egPS0Tk6JgZjVJTaJSawultw04jUn4JCUa/tun0a5sedhSRKima16RmAu2D0ViqAyOACaXWmQBcFzwfDnzkkbtbJwAjglFhWgPtgRmH22ewzcfBPgj2+XYUj01EREREJKqidkY96HN+OzAJSASedfcFZvZzIMvdJwDPAC8FN4tuI1J4E6w3lsiNp4XAaHcvAihrn8G3/BHwmpn9Apgd7FtEREREJC5pwiMN9SUicUjDM4qIxJeYm/BIRERERESOjQp1EREREZEYpEJdRERERCQGqVAXEREREYlBKtRFRERERGJQlR71xcw2A6uOYdMGRCZZilfKH654zh/P2aFy5T/J3RuGGaaiqc2OW8ofLuUPT+nsR91uV+lC/ViZWVY8D4um/OGK5/zxnB2Uv6qK95+b8odL+cMVz/lPRHZ1fRERERERiUEq1EVEREREYpAK9WPzdNgBjpPyhyue88dzdlD+qiref27KHy7lD1c85z/u7OqjLiIiIiISg3RGXUREREQkBqlQPwpmNsTMlphZrpk9GHaespjZs2aWZ2bzSyyrb2YfmNmy4Gu9YLmZ2Z+C45lrZr3CS/5V1hZm9rGZLTKzBWZ2V7A8Lo7BzFLMbIaZ5QT5/zdY3trMpgf5x5hZ9WB5cvA6N3i/VZj5g0yJZjbbzP4VvI6b7ABmttLM5pnZHDPLCpbFy+enrpmNN7PFwf+BfvGSPVap3Y4utdnWKsz8h8Rzux3PbXaQKarttgr1cjKzROAvwAVAF2CkmXUJN1WZngeGlFr2IDDF3dsDU4LXEDmW9sFjFPBkBWU8kkLgPnfvDPQFRgc/53g5hoPAOe7eHegBDDGzvsBvgEeD/NuBG4P1bwS2u3s74NFgvbDdBSwq8Tqesh9ytrv3KDEsVrx8fh4H3nf3TkB3Iv8O8ZI95qjdrhBqs2NDvLfb8dpmQ7TbbXfXoxwPoB8wqcTrh4CHws51mKytgPklXi8BmgbPmwJLgudPASPLWi9WHsDbwPnxeAxATSAbOI3IhAdJpT9LwCSgX/A8KVjPQszcPGhUzgH+BVi8ZC9xDCuBBqWWxfznB6gDrCj9M4yH7LH6ULsdynGoza743HHdbsdrmx18/6i32zqjXn7NgDUlXq8NlsWDxu6+ASD42ihYHtPHFFyS6wlMJ46OIbgEOQfIAz4AlgM73L0wWKVkxq/yB+/vBNIrNvE3PAY8ABQHr9OJn+yHODDZzGaZ2ahgWTx8ftoAm4HngkvY/zCzWsRH9lgVzz+juPt3V5sdmnhvt+O1zYYKaLdVqJeflbEs3ofMidljMrPawOvA3e6+60irlrEs1GNw9yJ370HkLEcfoHNZqwVfYya/mV0M5Ln7rJKLy1g15rKX0t/dexG5xDjazM48wrqxdAxJQC/gSXfvCezl68ulZYml7LGqMv6MYvKY1GaHo5K02/HaZkMFtNsq1MtvLdCixOvmwPqQshytTWbWFCD4mhcsj8ljMrNqRBr8l939jWBxXB0DgLvvAP5NpN9mXTNLCt4qmfGr/MH7acC2ik36lf7ApWa2EniNyGXUx4iP7F9x9/XB1zzgTSK/eOPh87MWWOvu04PX44n8AoiH7LEqnn9GcfPvrjY71HYv7tvtOG6zoQLabRXq5TcTaB/cSV0dGAFMCDlTeU0ArgueX0ekD+Gh5dcGdyH3BXYeulQTFjMz4Blgkbv/scRbcXEMZtbQzOoGz2sA5xG5seRjYHiwWun8h45rOPCRBx3XKpq7P+Tuzd29FZHP90fu/j3iIPshZlbLzFIPPQcGAfOJg8+Pu28E1phZx2DRucBC4iB7DFO7HWVqs8Nt9+K93Y7nNhsqqN0OqwN+PD6AC4GlRPqv/STsPIfJ+CqwASgg8pfbjUT6n00BlgVf6wfrGpEREZYD84DeMZD/DCKXgeYCc4LHhfFyDEAmMDvIPx94OFjeBpgB5ALjgORgeUrwOjd4v03Y/wZBroHAv+Ite5A1J3gsOPT/NI4+Pz2ArODz8xZQL16yx+pD7XbUs6vNjoHPUJAt7trteG+zg0xRbbc1M6mIiIiISAxS1xcRERERkRikQl1EREREJAapUBcRERERiUEq1EVEREREYpAKdRERERGRGKRCXao8M9sTfG1lZled4H3/uNTraSdy/yIiVY3abKlKVKiLfK0VcFSNvpklfssq32j03f30o8wkIiJla4XabKnkVKiLfO3XwAAzm2Nm95hZopn9zsxmmtlcM7sFwMwGmtnHZvYKkQkLMLO3zGyWmS0ws1HBsl8DNYL9vRwsO3QmyIJ9zzezeWZ2ZYl9/9vMxpvZYjN7OZj5DzP7tZktDLL8vsJ/OiIisUVttlR6SWEHEIkhDwL3u/vFAEHjvdPdTzWzZOAzM5scrNsH6ObuK4LXN7j7tmAK6plm9rq7P2hmt7t7jzK+1+VEZjPrDjQItpkavNcT6AqsBz4D+pvZQuAyoJO7+6Epr0VEqjC12VLp6Yy6yOENAq41sznAdCJTArcP3ptRosEHuNPMcoAvgBYl1jucM4BX3b3I3TcBnwCnltj3WncvJjIddytgF3AA+IeZXQ7sO+6jExGpXNRmS6WjQl3k8Ay4w917BI/W7n7o7Mzer1YyGwicB/Rz9+7AbCClHPs+nIMlnhcBSe5eSOSMb5brmAAAAQpJREFU0OvAMOD9ozoSEZHKT222VDoq1EW+thtILfF6EvADM6sGYGYdzKxWGdulAdvdfZ+ZdQL6lniv4ND2pUwFrgz6VDYEzgRmHC6YmdUG0tz9XeBuIpdgRUSqMrXZUumpj7rI1+YChcHl0OeBx4lcwswObg7aTOTMSGnvA7ea2VxgCZFLqYc8Dcw1s2x3/16J5W8C/YAcwIEH3H1j8EujLKnA22aWQuTMzj3HdogiIpWG2myp9Mzdw84gIiIiIiKlqOuLiIiIiEgMUqEuIiIiIhKDVKiLiIiIiMQgFeoiIiIiIjFIhbqIiIiISAxSoS4iIiIiEoNUqIuIiIiIxCAV6iIiIiIiMej/AOvEZiRMhTG5AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot_lr(show_moms=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`MixUpCallback`](/callbacks.mixup.html#MixUpCallback)\n", "\n", "Data augmentation using the method from [mixup: Beyond Empirical Risk Minimization](https://arxiv.org/abs/1710.09412). It is very simple to add mixup in fastai :" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy]).mixup()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`CSVLogger`](/callbacks.csv_logger.html#CSVLogger)\n", "\n", "Log the results of training in a csv file. Simply pass the CSVLogger callback to the Learner." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy, error_rate], callback_fns=[CSVLogger])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_ratetime
00.1190830.1080340.9592740.04072600:02
10.0781560.0712080.9730130.02698700:02
20.0569850.0458350.9847890.01521100:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can then read the csv." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_ratetime
000.1190830.1080340.9592740.040726NaN
110.0781560.0712080.9730130.026987NaN
220.0569850.0458350.9847890.015211NaN
\n", "
" ], "text/plain": [ " epoch train_loss valid_loss accuracy error_rate time\n", "0 0 0.119083 0.108034 0.959274 0.040726 NaN\n", "1 1 0.078156 0.071208 0.973013 0.026987 NaN\n", "2 2 0.056985 0.045835 0.984789 0.015211 NaN" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.csv_logger.read_logged_file()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`GeneralScheduler`](/callbacks.general_sched.html#GeneralScheduler)\n", "\n", "Create your own multi-stage annealing schemes with a convenient API. To illustrate, let's implement a 2 phase schedule." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def fit_odd_shedule(learn, lr):\n", " n = len(learn.data.train_dl)\n", " phases = [TrainingPhase(n).schedule_hp('lr', lr, anneal=annealing_cos), \n", " TrainingPhase(n*2).schedule_hp('lr', lr, anneal=annealing_poly(2))]\n", " sched = GeneralScheduler(learn, phases)\n", " learn.callbacks.append(sched)\n", " total_epochs = 3\n", " learn.fit(total_epochs)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
00.1719620.1547160.94749800:02
10.1337200.1322490.95780200:02
20.1329280.1299270.95780200:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn = Learner(data, simple_cnn((3,16,16,2)), metrics=accuracy)\n", "fit_odd_shedule(learn, 1e-3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAD8CAYAAABkbJM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd8VFX6+PHPk0kjIQRSSEISCJAQCISWwIJYQJSqYEEFVxb94rpfF366a8X96q7rWtZ17R27roqIqwZFkMUCKi30GgiEEmogEEqAEDi/P+bCxpgyIZPcKc/79eKVmTvnnnmOJvPMPfcUMcaglFJK1SbA7gCUUkp5B00YSimlXKIJQymllEs0YSillHKJJgyllFIu0YShlFLKJZowlFJKuUQThlJKKZdowlBKKeWSQLsDcIeYmBiTkpJidxhKKeVVlixZss8YE+tqeZ9IGCkpKeTm5todhlJKeRUR2VqX8tolpZRSyiWaMJRSSrlEE4ZSSimXaMJQSinlEk0YSimlXOJSwhCRISKSJyL5IjKpitdDROQj6/WFIpJS4bX7rON5IjK4wvE3RWSviKyuVFeUiMwWkY3Wzxbn3jyllFLuUmvCEBEH8CIwFMgAxohIRqVi44EDxphU4GngcevcDGA00BkYArxk1QfwtnWssknAHGNMGjDHeq6UUspmrlxh9AbyjTGbjTFlwBRgZKUyI4F3rMfTgIEiItbxKcaYE8aYAiDfqg9jzFyguIr3q1jXO8AVdWhPnXy6rJAXv83ng4XbWLbtAGXlpxvqrZRSyuu5MnEvEdhe4Xkh8KvqyhhjykWkBIi2ji+odG5iLe8XZ4zZZdW1S0RaVlVIRG4BbgFo3bq1C834pekrdvHN+r1nn0eEBDIsM4Eb+6XQKaHZOdWp6q/wQCl5uw8zsFOc3aEopSpwJWFIFceMi2VcOfecGGMmA5MBsrOzz6nON2/sxfGTpyg6fILVO0r4z7q9TF+5k6lLtnN511bcf1knWkaEuiNcVQfvzd/K5HmbmXHbBZq4lfIgrnRJFQLJFZ4nATurKyMigUAkzu4mV86tbI+IJFh1JQB7aylfL6FBDpKjwhiamcCT13Zj/qSB/L5/e2at2c2lT83ly5W7GvLtVRVOnjIYA09+vcHuUJRSFbiSMBYDaSLSVkSCcd7EzqlUJgcYZz0eBXxjjDHW8dHWKKq2QBqwqJb3q1jXOOBzF2J0m8iwIO4e3JEZt19A25hwJnywlMdnrufUabdcGKk6+M+6PSzZesDuMJRSlloThjGmHJgIzALWAVONMWtE5CERGWEVewOIFpF84A6skU3GmDXAVGAtMBOYYIw5BSAiHwLzgXQRKRSR8VZdfwcuFZGNwKXW80bXPrYpU3/Xl+t/1ZqXv9vE3R+v0KTRSAyG0KAAYpoG88Ss9Ti/eyil7ObSarXGmBnAjErH/lzh8XHgmmrOfQR4pIrjY6opvx8Y6EpcDS04MIBHr8wkoVkoT87ewMnThqev7UagQ+c7NiRjIMgRwMQBqTw4fS0/5O/jgjSXV2BWSjUQ/eRzwf8bmMa9QzoyfcVO7v9stX7jbQQCjPlVaxKbN+GJWXn631wpD6AJw0W39m/PxAGpTFm8nVfnbrY7HL8QEujg9kvSWFlYwqw1u+0ORym/pwmjDu64tAOXd2vF379az8zV+gHWUIwxOOd9wlU9EmkfG84/v96g95CUspkmjDoICBCeGNWVbsnNufvjFWzbX2p3SD7LyhcEOgK4c1A6+XuP8OmyHfYGpZSf04RRR6FBDl4Y0wME/t+HS3U5kQZQ+TpiaJd4MhMjeXr2Bk6Un7IlJqWUJoxzkhwVxhOjurKisIQnv86zOxyfY8zPlwgQEe4anM6Og8f4aPH2as9TSjUsTRjnaEiXBMb0bs1r8zazbJtOLnMnw3/vYZxxYVoMvdtG8dycfErLym2KTCn/pgmjHv40rCNxzUK5Z9pK7SppYCLCPYPT2XfkBG//tMXucJTyS5ow6iEiNIjHrspk494jvPBNvt3h+IzKXVJnZKdEcXHHlrzy3SZKSk82elxK+TtNGPXUP70lV/VI5NXvN1Ow76jd4fgEw39HSVV216B0Dp8o5+XvNzVqTEopTRhuMWlYR4IDA3ho+hq7Q/EhVWeMjFbNuKJ7Im/9WMDOg8caOSal/JsmDDdoGRHKHy5J49u8Iuas22N3OF6vtlVA7hzUAWPgqdm6/LlSjUkThpuMOy+F1JZN+dsXazl5Sudm1I+ptksKIKlFGOPOa8MnSwtZv/tQ44WllJ/ThOEmQY4A7hvakS37S5micwXqrYZ8AcCEAalEhATy+FfrGyUepZQmDLe6uGNLeqW04Lk5G3WuQD24sjBt87BgJgxI5du8In7atK/hg1JKacJwJxFh0tCOFB0+wVs/brE7HK9lTPWjpCoad14KrSJD+ftX6zmtCxMq1eA0YbhZVpsoLukUxyvfbeLA0TK7w/FaUmunlHNdrzsGpbOysIQvV+ne60o1NE0YDeCeIekcKSvXfTPOkfnF8oPVu7JHIh3jI3hiVp4uBKlUA9OE0QA6xEUwPDOB9+Zv4WCpXmXUlatdUgCOAGc34LbiUt5fuLVhA1PKz2nCaCATL07laNkpvZdxjlzMFwBc1CGW89pH8/w3+Rw6rkuGKNVQNGE0kI7xzRiUEcdbPxZwWD/E6qSut69FhPuGdqL4aBmv6pIhSjUYTRgNaOLFqRw6Xs57C7SrpC6cXVJ1ucaAzKRIRnRrxRs/FLCrRJcMUaohaMJoQF2TmnNRh1hen1fAsTJd/txVdbnpXdHdg9M5fRqemKWbWinVEDRhNLDf929P8dEyPllaaHcoPi85Koybzk/h30t3sLLwoN3hKOVzNGE0sN5to+iaFMmbPxTo5DJX1WGUVGUTB6QSHR7Mw1+sw7gyZVwp5TJNGA1MRBh/fls27zvKt3l77Q7HK9S0H0ZtIkKDuGNQBxZtKWbWmt1ujUspf6cJoxEMy0wgITKU1+bpRD5XuTLTuzrXZSfTIa4pj85Yr1vnKuVGmjAaQZAjgJv6pbBgczGrd5TYHY7Hq29XUqAjgPuHZ7CtuJR3f9IRakq5iyaMRnJdr9aEBzt444cCu0PxePXpkjrjwg6x9E+P5blvNrL/yAm3xKWUv9OE0UgimwQxKiuJL1fu0g8wF9QzXwDwf8M6UVp2imfnbHRDbUoplxKGiAwRkTwRyReRSVW8HiIiH1mvLxSRlAqv3WcdzxORwbXVKSIDRWSpiCwXkR9EJLV+TfQcN/RpQ9mp00zN1SG2NXHX4Ka0uAiu792a9xduY+Oew+6pVCk/VmvCEBEH8CIwFMgAxohIRqVi44EDxphU4GngcevcDGA00BkYArwkIo5a6nwZ+LUxpjvwAXB//ZroOdLiIujTLor3F27llA6xrZazS8od1xjwx0s7EBbs4NEZ69xSn1L+zJUrjN5AvjFmszGmDJgCjKxUZiTwjvV4GjBQnH/xI4EpxpgTxpgCIN+qr6Y6DdDMehwJ7Dy3pnmmsX1SKDxwjO836BDbmrgnXUBUeDC3XZzGt3lFzN1Q5KZalfJPriSMRKDiJtWF1rEqyxhjyoESILqGc2uq82ZghogUAmOBv7vSEG8xqHMcLSNCeHe+jt6pjrsn3P3mvDa0iQ7j4S/XUn5K98xQ6ly5kjCq+rJX+S+6ujJ1PQ7wR2CYMSYJeAt4qsqgRG4RkVwRyS0q8p5vjkGOAMb0bs33G4rYuv+o3eF4JAPuu8QAQgId3De0Exv2HOH9hdvcV7FSfsaVhFEIJFd4nsQvu4nOlhGRQJxdScU1nFvlcRGJBboZYxZaxz8CzqsqKGPMZGNMtjEmOzY21oVmeI4xvVsTIMIH+uFVLTfmCwAGd47j/NQYnvw6j2LdOlepc+JKwlgMpIlIWxEJxnkTO6dSmRxgnPV4FPCNcfYr5ACjrVFUbYE0YFENdR4AIkWkg1XXpYDP3a2Mjwzl0k5xTM3drtuKVqUBxgOICH+5PIOjZaf459e6mq1S56LWhGHdk5gIzML54T3VGLNGRB4SkRFWsTeAaBHJB+4AJlnnrgGmAmuBmcAEY8yp6uq0jv8W+EREVuC8h3G3+5rrOUb3TuZA6Un+s26P3aF4HINx2yipitLiIhjXN4UPF23TGfdKnQPxhRU9s7OzTW5urt1h1Mmp04bzH/+G9PgI3r6pt93heJRb/7WE/L1HmH3HRW6vu+TYSS7+53e0jQnn4//t2yCJSSlvISJLjDHZrpbXmd42cQQIo7KSmLuhSHeIa0SRTYK4Z0g6uVsPkLPCp0ZsK9XgNGHY6JqsZE4bmKYzv3/G1GM/DFdck5VM16RIHp2xjqMnyhvujZTyMZowbNQ6Ooy+7aL5eEmhbq5UgcHUa3nz2gQECH+5vDN7Dp3ghW/zG+x9lPI1mjBsdm2vJLYVl7KgYL/doXiUhr61kNWmBVf1TOSNeQUU7NP5MEq5QhOGzYZ2SSAiNJCPtVvqrMYahzFpSEeCHMLDX6xtnDdUystpwrBZaJCDEd1aMWPVLkqOnbQ7HI/QWJ1zLZuFctvANOas38u363VtL6VqownDA1zXK5kT5ad11E4FjTXc9aZ+bWkXE85DX6zV7VyVqoUmDA+QmRhJelwEny7VbilovC4pgODAAB4c0ZmCfUeZ/L3uua5UTTRheAAR4YoeiSzddlAXJATANOAYqV+6sEMswzMTeOHbfLbtL23Ed1bKu2jC8BAju7dCBD5dtsPuUDxCY0/AfuCyDAIDhL/krHb78upK+QpNGB6iVfMm9GkbzWfLdvj9B5YdzY+PDOWPl3bg27wivl6r63spVRVNGB7kyh6JbNlfyvLtB+0OxVbOLVob/33HnZdCx/gI/pqzhtIynQGuVGWaMDzIkMx4QgIDtFsKGnSmd3WCHAE8fEUXdpYc57k5OgNcqco0YXiQZqFBXJIRx/QVOznpx1uJ2tkll50SxbXZSbw+bzMb9xy2LQ6lPJEmDA9zZfdEDpSeZO4G79l21t3s6pI6Y9LQTjQNDeT+z/QGuFIVacLwMBd2iKVFWJBfd0sZ4/4tWusiKjyYe4d0ZGFBMZ8t99//D0pVpgnDwwQHBnB5t1bMXruHQ8d1qRC7XJedTPfk5jzy5TpdskUpiyYMD3RFj0ROlJ9m5urddodiCwP29knhXAL94Su6UHy0jCdmrbc1FqU8hSYMD9QjuTkp0WF85qfdUsY07kzv6nRJjGTceSm8v3AbS7YesDscpWynCcMDiQgjurViweb97D183O5wbOEpW23fNSidVpFNuO/fKykr99+Ra0qBJgyPdXm3Vpw28NUq/+yW8hThIYH87YrObNhzhFe/32R3OErZShOGh0qLiyA9LoLpfrjkud2jpCq7uGMcl3VN4Plv8tlUdMTucJSyjSYMD3Z5twRytx5g58FjdofS6BprPwxX/fnyDEKDArjv36t0/3XltzRheLDLurYC4MuVu2yOpHGZRttzz3UtI0L5v+GdWFRQzNTc7XaHo5QtNGF4sJSYcDITI5m+0r+6pTytS+qMa7OT6dMuikdnrPPbwQjKv2nC8HCXd0tgZWGJ322s5GE9UoCzm+zRKzM5Xn6av05fa3c4SjU6TRgebrjVLfWFH3VLefLyTe1im3Lbxal8uXIXc9bpvhnKv2jC8HCJzZuQ1aaFX42WMhhbljd31S0Xtic9LoL7P1vNkRO6b4byH5owvMDlXRNYv/uwfy237bn5guDAAB67OpPdh47zj5m6bIjyH5owvMCwzAREYLqfdEt5cpfUGT1bt2Bc3xTenb+VBZv32x2OUo1CE4YXaNkslD5to/li5U6/2J/B4NEXGGfdMySd1lFh3PvJSo6VnbI7HKUanEsJQ0SGiEieiOSLyKQqXg8RkY+s1xeKSEqF1+6zjueJyODa6hSnR0Rkg4isE5Hb6tdE33B5t1ZsLjrK2l2H7A6lUXjiKKnKwoIDefzqrmzdX8oTs/LsDkepBldrwhARB/AiMBTIAMaISEalYuOBA8aYVOBp4HHr3AxgNNAZGAK8JCKOWuq8EUgGOhpjOgFT6tVCHzGkSzyBAeIfo6W86CKqb/toxvZpw1s/FZC7pdjucJRqUK5cYfQG8o0xm40xZTg/wEdWKjMSeMd6PA0YKM61HUYCU4wxJ4wxBUC+VV9Ndd4KPGSMOQ1gjNl77s3zHVHhwfRtH81Xq3b5fLeUp4+SqmzS0I4kNm/CPdNWcvykdk0p3+VKwkgEKq6FUGgdq7KMMaYcKAGiazi3pjrbA9eJSK6IfCUiaVUFJSK3WGVyi4r8Y//rYZkJbNlf6vPdUsZ4R5fUGeEhzq6pzfuO8tTsDXaHo1SDcSVhVPWnW/krbnVl6nocIAQ4bozJBl4D3qwqKGPMZGNMtjEmOzY2tsrAfc3gzvE4AsQvljz3poQB0C81hjG9W/P6vM0s3aabLSnf5ErCKMR5T+GMJKDyLLKzZUQkEIgEims4t6Y6C4FPrMefAl1diNEvRIUH06ddFDN8vFvKW1v2p2EdiW8Wyt0fr9CuKeWTXEkYi4E0EWkrIsE4b2LnVCqTA4yzHo8CvjHOT7QcYLQ1iqotkAYsqqXOz4CLrccXAXqNX8HQLgls3neUPB+exOfcotXLLjGAiNAgHru6K5uKjvLMfzbaHY5SbldrwrDuSUwEZgHrgKnGmDUi8pCIjLCKvQFEi0g+cAcwyTp3DTAVWAvMBCYYY05VV6dV19+Bq0VkFfAYcLN7muobBneOJ0Bgho93S3lbl9QZF3WI5brsZCbP3aT7gCufI77QtZGdnW1yc3PtDqPRjJ48n31HyvjPHRfZHUqDuPKlH2kaEsh7439ldyjn5PDxkwx5Zh5BDmHG7RcQFhxod0hKVUlEllj3i12iM7290PDMBPL3HvHZtaW8/TtMRGgQT17bja3FpTw6Y53d4SjlNpowvNDgLvGIwJerfHcSn6dt0VpXfdpFM75fW/61YBvfb/CPYd/K92nC8EItI0LplRLls8NrvfwC46y7BqeT1rIpd3+8goOlZXaHo1S9acLwUsO6xJO35zD5e4/YHYr7GeOFY6R+KTTIwdPXdaf4aBkPfL6m9hOU8nCaMLzUkC4JAHzlo91SXt4jdVaXxEhuH5jG9BU7yfGjTbCUb9KE4aXiI0PJbtOCGat9r1vKV7qkzri1f3u6Jzfngc9Ws7vkuN3hKHXONGF4saGZCazbdYiCfUftDsWtjPGO/TBcFegI4Klru3Gi/BT3fLLSp2fpK9+mCcOLDe0SD8AMH+yW8vZRUpW1i23K/w3rxNwNRbw7f6vd4Sh1TjRheLFWzZvQo3Vzn0sYxuc6pZxu6NOGAemxPDJjHet3+/aKw8o3acLwcsMzE1iz8xBb9/tOt5SvdUmdISI8cU03moUGcduHy3SBQuV1NGF4uSFWt9RXPnTz29v2w6iLmKYhPHltNzbsOaKzwJXX0YTh5ZJahNEt2fe6pXzZRR1iufn8trw7fyv/WbvH7nCUcpkmDB8wrEs8KwtL2F5cancobuG8g+GjlxiWu4ek07lVM+6etoI9h3SorfIOmjB8wLBMaxLfat+4yjDG+GyX1BkhgQ6eG9OD4ydPc8fU5Zw+7Zs3+pVv0YThA5KjwshMjORLH1pbysfzBQDtY5vyl8sz+DF/P6/N22x3OErVShOGjxiaGc+K7QfZcfCY3aGoOriuVzJDu8TzxKw8VhYetDscpWqkCcNHDM/0nbWlfHmUVGUiwmNXZdIyIoSJHyzj0PGTdoekVLU0YfiINtHhdG7VzGf2yPDGPb3PVfOwYJ6/vgc7Dh7j3mm6dIjyXJowfMiwzASWbTvITi/vlvLVmd41yWoTxT2D0/lq9W5dOkR5LE0YPuTMaClvn5PhT11SFf32gnYM7NiSh79cq/czlEfShOFD2saE0ymhmdcnDPDPhBEQIPzzmm7ENg1hwgdLKTmm9zOUZ9GE4WOGZ8azdNtBdpV4b7eU/3VI/VeL8GCev74nuw4e555pK/R+hvIomjB8zNlJfF48J8MY41c3vSvLatOCe4d0ZNaaPbz90xa7w1HqLE0YPqZdbFM6xkd4f7eU/+YLAG6+oC2XdGrJozPWsXy73s9QnkEThg8anplA7tYDXrsdqHbCOOdn/POabrSMCGXC+0s5cLTM7pCU0oThi4Z19fK1pXx0P4y6ah4WzEu/7knR4RPcNmUZp3S9KWUzTRg+qH1sU9LjvLdbyuB7W7Seq27JzXloZGfmbdzHk1/n2R2O8nOaMHzUMKtbSpfO9n6je7dmTO9kXvpuE7PWeO9gBuX9NGH4qOFd4zEGZnrhTnzOUVKqogdHdKZbUiR3Tl3BpqIjdoej/JQmDB+V2jKCDnFNvXJtKWeXlN1ReJaQQAcv35BFSGAAv3tvCUdOlNsdkvJDLiUMERkiInkiki8ik6p4PUREPrJeXygiKRVeu886nicig+tQ5/Miol+l6mFYZgKLtxSz1wu7pTRf/FKr5k14/voebC46opP6lC1qTRgi4gBeBIYCGcAYEcmoVGw8cMAYkwo8DTxunZsBjAY6A0OAl0TEUVudIpINNK9n2/ze8MwEZ7eUl/V76+dg9c5rH8OkoR2ZsWo3k+fqpkuqcblyhdEbyDfGbDbGlAFTgJGVyowE3rEeTwMGinOYy0hgijHmhDGmAMi36qu2TiuZPAHcU7+mqbS4CFJbNuXLld7VLWUwOkqqBr+9oB3DMxN4fOZ65m4osjsc5UdcSRiJwPYKzwutY1WWMcaUAyVAdA3n1lTnRCDHGONdn3IealhmAou2FLP3sHd1S2m6qJ6I8I9RXekQF8GED5bqTXDVaFxJGFX97VbuNKiuTJ2Oi0gr4Brg+VqDErlFRHJFJLeoSL9lVedMt9QsLxotpV1StQsPCeT1cdkEOwL47Tu5lJTqyraq4bmSMAqB5ArPk4Cd1ZURkUAgEiiu4dzqjvcAUoF8EdkChIlIflVBGWMmG2OyjTHZsbGxLjTDP3WIa0r72HBmeNFihKa6rxTqZ5JahPHK2Cy2Hyhl4odLKT912u6QlI9zJWEsBtJEpK2IBOO8iZ1TqUwOMM56PAr4xjiHcOQAo61RVG2BNGBRdXUaY740xsQbY1KMMSlAqXUjXZ0jEWF4ZgILC/az78gJu8NxmT+vVlsXvVKieOSKTOZt3MfDX66zOxzl42pNGNY9iYnALGAdMNUYs0ZEHhKREVaxN4Bo62rgDmCSde4aYCqwFpgJTDDGnKquTvc2TZ0xrGsCp710Ep+q3bW9khl/flve/mkLHy7aZnc4yocFulLIGDMDmFHp2J8rPD6O895DVec+AjziSp1VlGnqSnyqZulxEbSLCWfGql3c0KeN3eHUyhijE/fq6L6hHcnfe4QHPltN25hw+rSLtjsk5YN0prcfEBGGZSawYLP3dEtpvqibQEcAz43pQevoMG791xK27S+1OyTlgzRh+InhVrfUV17QLaWDpM5NZJMg3hjXi9MGbnp7kY6cUm6nCcNPdIx3TuKbvrzyADfPY4yuJXWu2saE8+rYLLYXH+OW93I5UX7K7pCUD9GE4SdEhBHdWrFoSzE7Dx6zO5waGfx7T+/66tMumieu6crCgmLunbZS15xSbqMJw4+M6NYKgC9Wev5Vhqqfkd0TuWtQBz5bvpOnZm+wOxzlIzRh+JGUmHC6JkWSs8KzE4Z2SbnHhAGpXJedzPPf5DN18fbaT1CqFpow/MyIbq1YveMQmz14/SHdD8M9RISHr+zCBWkx/OnTVczbqEvoqPrRhOFnLuvaChE8/ipDB9a6R5AjgBd/3ZPUlk259V9LWbvzkN0hKS+mCcPPxEeG0jslipwVOz32ZqiHhuW1moUG8eaNvYgIDWTcW4t0joY6Z5ow/NCI7q3YXHSUNR77bVNnertbq+ZNePd/elNWfpqxby6k6LB3TOBUnkUThh8a1iWBwABhugd3S2m+cL+0uAjevLEXew4d58a3FnH4uE7sU3WjCcMPtQgP5oK0GKav2Mnp057X/6NdUg0nq00LXr4hi7zdh7nl3SUcP6kT+5TrNGH4qRHdW7Gz5DhLth2wO5Rf0FFSDWtAekueuKYr8zfv548fLeeUB35pUJ5JE4afujQjnpDAAHI8dKkQnendsK7skcT9wzvx1erdPPD5ao8dAKE8iyYMP9U0JJCBnVoyY9UuTnrYTm364dU4br6gHbf2b88HC7fx2Ffr9b+7qpUmDD92RfdE9h8t87gJXdol1XjuGZzOb/q2YfLczTzzn412h6M8nEsbKCnf1D+9JS3CgvhkyQ4u7hhndzg/o/micYgID17emeMnT/HsnI2EBjm4tX97u8NSHkoThh8LDgxgZPdEPli0jZLSk0SGBdkdEqCjpBpbQIDw2FVdOX7yNI/PXE+ToABu7NfW7rCUB9IuKT93Vc9EyspP88Uqz7n57dyiVa8xGpMjQHjy2m4M7hzHg9PXMkX3BldV0ITh5zITI0lr2ZR/L91hdyjKZkHWNq/902O579NVfLqs0O6QlIfRhOHnRISreiaxZOsBCvYdtTscQLdotVNIoINXbsiib7to7py6QpOG+hlNGIoreyQiAp8u9ZAPB90Pw1ahQQ5eH5dNn3bR3DF1BdOWeMjvhbKdJgxFfGQo56fG8MnSHR6xVIhBJ+7ZLSw4kDfG9aJf+xjunraCjxbrPQ2lCUNZru6ZxI6Dx1i0pdjuUAC9wvAETYKdVxoXpMVy7yer+GChJg1/pwlDATCocxzhwQ4+8YDuB51x7DlCgxxMHpvFgPRY/vTpKt6bv8XukJSNNGEowNkFMSwzgRmrdnGszN4VTJ1dUspThAY5eGVsFpd0askDn6/hzR8K7A5J2UQThjrr6qwkjpad4stVu+wORbukPExIoIOXfp3F4M5xPPTFWp6evUGvBP2QJgx11q/aRtE2Jtz2SVv6OeSZggMDePH6nozKSuLZORv56/S1HjFIQjUeTRjqLBFhdK9kcrceYOOew7bFYdCZ3p4q0BHAP67uyvjz2/L2T1u46+MVHrfasWo4mjDUz1ydlUSQQ/hw0XZb49B04bkCAoT7h3firkEd+PeyHdz6r6W6c5+f0IShfiagfqrrAAASX0lEQVSmaQiDOsfzydJC2z4EtEvK84kIEy9O428jOzNn/R7dI9xPuJQwRGSIiOSJSL6ITKri9RAR+ch6faGIpFR47T7reJ6IDK6tThF53zq+WkTeFBHPWELVj1zfuzUlx04yc/VuW97fgF5ieImxfVN45rru5G45wDWvzGd3yXG7Q1INqNaEISIO4EVgKJABjBGRjErFxgMHjDGpwNPA49a5GcBooDMwBHhJRBy11Pk+0BHIBJoAN9erharO+raLpk10GB/aePNbZ3p7j5HdE3nzxl4UHjjGlS/9yPrdh+wOSTUQV64wegP5xpjNxpgyYAowslKZkcA71uNpwEBx3rUcCUwxxpwwxhQA+VZ91dZpjJlhLMAiIKl+TVR1FRAgXNcrmYUFxWwqOtL4AWiXlNe5sEMsU3/XF2Pgmpfn88PGfXaHpBqAKwkjEah4B7TQOlZlGWNMOVACRNdwbq11Wl1RY4GZVQUlIreISK6I5BYVedYWo75gVFYSgQFiyxBb5yipRn9bVU8ZrZrx6YTzSGzRhBvfWsTHufYOnFDu50rCqOpPt/J3wOrK1PV4RS8Bc40x86oKyhgz2RiTbYzJjo2NraqIqoeWEaFcmhHHtCX23PzWfOGdEiKbMPV/+9KnXTR3T1upE/x8jCsJoxBIrvA8Cai8PdvZMiISCEQCxTWcW2OdIvIXIBa4w5VGqIYxtm8bDpSeJGdF4+7Gp58v3q1ZaBBv3dTr7AS/26Yst325GeUeriSMxUCaiLQVkWCcN7FzKpXJAcZZj0cB31j3IHKA0dYoqrZAGs77EtXWKSI3A4OBMcYYnRFko77tokmPi+CtH7c06rdEgy4N4u2CHAE8Maor9w7pyBcrd3Ltq/PZVXLM7rBUPdWaMKx7EhOBWcA6YKoxZo2IPCQiI6xibwDRIpKP86pgknXuGmAqsBbnvYgJxphT1dVp1fUKEAfMF5HlIvJnN7VV1ZGIcGO/FNbtOsSigsZb9twYo6OkfICIcGv/9rw2NpvNRUcY8cKPLN12wO6wVD2IL/QvZmdnm9zcXLvD8EnHyk7R9+9z6NsumpdvyGqU92x335dMGJDKnYPSG+X9VMPbsOcwN7+Ty+5Dx3nsykyuztLBj55ARJYYY7JdLa8zvVWNmgQ7GN2rNbPW7KbwQGmjvKf3f4VRlXWIi+DzCf3Iat2COz9ewUPT1+oaVF5IE4aq1di+bRAR3luwtVHezxgdJeWLWoQH8+743tzUL4U3fyxg9OQFOjPcy2jCULVKbN6EwZ3jmLJoO6Vl5Y3zpnrX2ycFOQL4y+WdeX5MD9bvOsTw5+bxY75O8vMWmjCUS8af35aSYyf5aLFOxlL1d3m3Vnw+8XyiwoMZ+8ZCXvhmo+6t4QU0YSiXZLWJondKFK/N3dygfc9nBmHo9YXvS23ZlM8m9OOyrq3459cb+J93FrPvyAm7w1I10IShXHZr//bsLDlOzvKGn8inPVL+ITwkkGdHd+dvIzvz06b9DHlmHt9v0KV+PJUmDOWy/umxdIyP4JXvNzVY94EPjPJWdSQijO2bQs7EfkSFBzHuzUU8/MVaTpTr7HBPowlDuezMRKyNe48wZ/3eBnmPM/lCJ+75n47xzciZeD5j+7Th9R8KuOqln+xZLVlVSxOGqpPhmQkkRzXhhW82NuhyIdol5Z9Cgxz87YouTB6bxY6Dxxj+3Dze/rFAb4h7CE0Yqk4CHQFMHJDKisIS5qxz/1WGL6w8oOpvUOd4Zt5+IX3aRfPg9LWMeW0B2/Y3zsRRVT1NGKrOruqZRJvoMJ6avcHt3/z+2yWl/F18ZChv3diLf1zdlbU7DzHk2bm8N3+LXm3YSBOGqrMgRwC3D0xj7a5DzFrTMPt+a5eUAud9s2t7JTPrjxeS1aYFD3y+hhveWMjW/UftDs0vacJQ52Rk90Tax4bz1OwNnHLjNz7tkVJVadW8Ce/+T2/+flUmKwtLGPT0XJ6fs1FHUjUyTRjqnDgChD9c0oGNe4/w2bIdbqvXWJ1SopcYqhIRYXTv1sy58yIu6RTHk7M3MOzZeczftN/u0PyGJgx1zoZnJtA1KZInZuW5bY0pvcJQtYlrFsqLv+7JWzf1ouzUaca8toA7p67QWeKNQBOGOmcBAcKfL8tg96HjvPL9ZrfWrRcYqjYD0lvy9R8u4vf92/P58h0MeOI7Js/dpN1UDUgThqqX7JQoLuuawKvfb2LHQd2CUzWuJsEO7hnSkZl/uIDslBY8OmM9g56ey6w1u3WIdgPQhKHqbdLQjgA8/tX6etd15m9cZ3qrukhtGcFbN/Xm7Zt6EeQI4HfvLWHMawtYvaPE7tB8iiYMVW9JLcL43YXtyFmxkx82umdvA+2SUueif3pLZt5+AQ+N7Eze7sNc9vwPTPhgKfl7dYkRd9CEodzi9wNSSYkO40+fruJY2bn3IRvdoFXVU6AjgN/0TeG7uwcwcUAq367fy6Cnv+euj1ewvVhni9eHJgzlFqFBDh69KpNtxaU8M2fDOdfz3y4ppeonskkQdw1OZ+49A7ipX1tyVuzk4ie/4/7PVmniOEeaMJTbnNc+htG9knl9XgFLtx2oV13aJaXcJaZpCA9clsHcuwdwbXYyHy3eTv9/fscfP1pO3u7DdofnVTRhKLf60/BOJESGcvuUZRw6frLO52uHlGoo8ZGhPHJlpvOK47wUZq3ZzeBn5jL+7cXkbim2OzyvoAlDuVWz0CCeHd2DnQeP8+fPVtf5/P9u0aqXGKphJEQ24f7LMvhp0sXccWkHlm47wKhX5jPihR/4OHc7x0/qPI7qaMJQbpfVpgV/GJjGZ8t38t6CredUh3ZJqYbWPCyY2wam8eOki/nbyM6Ulp3i7mkr6fPYHB6bsU7vc1Qh0O4AlG/6/YBUlm0/yIM5a2gfE855qTEunaddUqqxhQUHMrZvCjf0acP8zft5b/5WXv+hgMnzNnN+agyjspIYlBFPk2CH3aHaThOGahCOAOHZ0d256qWfuPX9pXxy63mktmxa63k6OVfZRUQ4r30M57WPYVfJMT5ctJ1PlhRy+5TlRIQEMiwzgauzkshu04KAAP+8BNYuKdVgIkKDeGNcL4Icwq9fX8CWfa7vYaCr1So7JUQ24Y5LOzDvngF88NtfMahzPNNX7uTaV+fT9+9z+Mvnq5m/ab9bl/b3BuIL661kZ2eb3Nxcu8NQ1cjbfZjRk+fTJMjBh7f0oU10eLVlS0pP0u2hr3ngsgzGn9+2EaNUqmalZeV8vWYPM1bt4vsNRZwoP010eDCXZsTRP70l/VKjiQgNsjvMOhGRJcaYbFfLa5eUanDp8RH86+Zf8evXF3LlSz/x2m+yyGoTVWXZs/thNGaASrkgLDiQK3okckWPRI6eKOe7vCK+Wr2LL1buYsri7QQGCD1bt+Ci9Fj6pcbQuVUzghy+1YnjUmtEZIiI5IlIvohMquL1EBH5yHp9oYikVHjtPut4nogMrq1OEWlr1bHRqjO4fk1UnqBzq0j+fet5NAsNZMzkhbw+b3ONezNrj5TyZOEhgQzvmsAL1/dk6QOXMuWWPtxyYTuOlpXzxKw8rnjxR7o++DXXv7aAp2ZvYN7GIkqO1X1ekqeptUtKRBzABuBSoBBYDIwxxqytUOb3QFdjzP+KyGjgSmPMdSKSAXwI9AZaAf8BOlinVVmniEwF/m2MmSIirwArjDEv1xSjdkl5jwNHy7jnk5XMXruHrDYt+PNlGXRLbv6z13v8bTZ/uTyDm/ppl5TyPkWHT7CooJjFW4rJ3VrM2p2HOPPdKKlFEzISmtG5VSSdEiJoF9uU1lFhBAfacyXSEF1SvYF8Y8xm6w2mACOBtRXKjAQetB5PA14Q513LkcAUY8wJoEBE8q36qKpOEVkHXAxcb5V5x6q3xoShvEeL8GAmj81i2pJCHp+Zx8gXf6RPuyiuyUrmgrQYAq1LeL3AUN4qNiKE4V0TGN41AYAjJ8pZtu0Aq3ccYs3OEtbuOsTsdXvOjggMEEhs0YSU6HBaR4UR1yyUuGYhtIwIJTYihOimwYSHBBIeHIjD5tFZriSMRGB7heeFwK+qK2OMKReREiDaOr6g0rmJ1uOq6owGDhpjyqsor3yEiHBNdjJDMxN4b/5W/rVgK3d+vOIXZZTyBU1DArkgLZYL0mLPHjt6opy8PYfZsu8oW/YdpWB/KVv2HWXVjl0cLK2+6yos2EF4SCDBjgCCHEKgI4A3xmXXOJDEnVxJGFX95Vbux6quTHXHq7r+qqn8L4MSuQW4BaB169ZVFVEermlIILf2b8/vLmzH2l2HWLB5P9uKSzl07CQXdYitvQKlvFR4SCA9W7egZ+sWv3jt+MlTFB0+wd7DJyg6fJwDpSc5eqKcw8fLOXqinCMnyik7dZryU4by06cJCWy8CYWuJIxCILnC8yRgZzVlCkUkEIgEims5t6rj+4DmIhJoXWVU9V4AGGMmA5PBeQ/DhXYoDxUQIHRJjKRLYqTdoShlu9AgB8lRYSRHhdkdyi+4cqdlMZBmjV4KBkYDOZXK5ADjrMejgG+M8256DjDaGkXVFkgDFlVXp3XOt1YdWHV+fu7NU0op5S61XmFY9yQmArMAB/CmMWaNiDwE5BpjcoA3gPesm9rFOBMAVrmpOG+QlwMTjDGnAKqq03rLe4EpIvIwsMyqWymllM10prdSSvmpug6r9a1piEoppRqMJgyllFIu0YShlFLKJZowlFJKuUQThlJKKZf4xCgpESkCzm3zaIjBOWHQl/ham3ytPaBt8ga+1h74ZZvaGGNcXlbBJxJGfYhIbl2GlXkDX2uTr7UHtE3ewNfaA/Vvk3ZJKaWUcokmDKWUUi7RhGEtYOhjfK1NvtYe0DZ5A19rD9SzTX5/D0MppZRr9ApDKaWUS/w6YYjIEBHJE5F8EZlkdzyuEJE3RWSviKyucCxKRGaLyEbrZwvruIjIc1b7VopIT/sir56IJIvItyKyTkTWiMjt1nGvbJeIhIrIIhFZYbXnr9bxtiKy0GrPR9bS/ljL/39ktWehiKTYGX9NRMQhIstE5AvruVe3SUS2iMgqEVkuIrnWMa/8vQMQkeYiMk1E1lt/T33d2R6/TRgi4gBeBIYCGcAYEcmwNyqXvA0MqXRsEjDHGJMGzLGeg7Ntada/W/DcvdHLgTuNMZ2APsAE6/+Ft7brBHCxMaYb0B0YIiJ9gMeBp632HADGW+XHAweMManA01Y5T3U7sK7Cc19o0wBjTPcKw0299fcO4FlgpjGmI9AN5/8r97XHGOOX/4C+wKwKz+8D7rM7LhdjTwFWV3ieByRYjxOAPOvxq8CYqsp58j+cm2Zd6gvtAsKApTj3rN8HBFrHz/7+4dwXpq/1ONAqJ3bHXkVbkqwPnIuBL3BuqeztbdoCxFQ65pW/d0AzoKDyf2d3tsdvrzCARGB7heeF1jFvFGeM2QVg/WxpHfe6NlpdFz2AhXhxu6yum+XAXmA2sAk4aJxbD8PPYz7bHuv1EiC6cSN2yTPAPcBp63k03t8mA3wtIktE5BbrmLf+3rUDioC3rG7D10UkHDe2x58ThlRxzNeGjHlVG0WkKfAJ8AdjzKGailZxzKPaZYw5ZYzpjvNbeW+gU1XFrJ8e3x4RuQzYa4xZUvFwFUW9pk2WfsaYnji7ZyaIyIU1lPX0NgUCPYGXjTE9gKP8t/upKnVujz8njEIgucLzJGCnTbHU1x4RSQCwfu61jntNG0UkCGeyeN8Y82/rsNe3yxhzEPgO572Z5iJyZlvkijGfbY/1eiTOrY49ST9ghIhsAabg7JZ6Bu9uE8aYndbPvcCnOJO7t/7eFQKFxpiF1vNpOBOI29rjzwljMZBmjfIIxrkPeY7NMZ2rHGCc9XgcznsAZ47/xhoN0QcoOXNp6klERHDu3b7OGPNUhZe8sl0iEisiza3HTYBLcN58/BYYZRWr3J4z7RwFfGOsTmVPYYy5zxiTZIxJwfm38o0x5td4cZtEJFxEIs48BgYBq/HS3ztjzG5gu4ikW4cGAmtxZ3vsvlFj802iYcAGnP3L/2d3PC7G/CGwCziJ8xvCeJx9w3OAjdbPKKus4BwJtglYBWTbHX81bTof56XwSmC59W+Yt7YL6Aoss9qzGvizdbwdsAjIBz4GQqzjodbzfOv1dna3oZb29Qe+8PY2WbGvsP6tOfMZ4K2/d1aM3YFc63fvM6CFO9ujM72VUkq5xJ+7pJRSStWBJgyllFIu0YShlFLKJZowlFJKuUQThlJKKZdowlBKKeUSTRhKKaVcoglDKaWUS/4/E9DCqtLHNm0AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot_lr()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`MixedPrecision`](/callbacks.fp16.html#MixedPrecision)\n", "\n", "Use fp16 to [take advantage of tensor cores](https://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html) on recent NVIDIA GPUs for a 200% or more speedup." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`HookCallback`](/callbacks.hooks.html#HookCallback)\n", "\n", "Convenient wrapper for registering and automatically deregistering [PyTorch hooks](https://pytorch.org/tutorials/beginner/former_torchies/nn_tutorial.html#forward-and-backward-function-hooks). Also contains pre-defined hook callback: [`ActivationStats`](/callbacks.hooks.html#ActivationStats)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`RNNTrainer`](/callbacks.rnn.html#RNNTrainer)\n", "\n", "Callback taking care of all the tweaks to train an RNN." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`TerminateOnNaNCallback`](/callbacks.tracker.html#TerminateOnNaNCallback)\n", "\n", "Stop training if the loss reaches NaN." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`EarlyStoppingCallback`](/callbacks.tracker.html#EarlyStoppingCallback)\n", "\n", "Stop training if a given metric/validation loss doesn't improve." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`SaveModelCallback`](/callbacks.tracker.html#SaveModelCallback)\n", "\n", "Save the model at every epoch, or the best model for a given metric/validation loss." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
00.6652440.6425820.81648700:02
10.5084920.4719500.93768400:02
20.4382860.4353770.94111900:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn = Learner(data, simple_cnn((3,16,16,2)), metrics=accuracy)\n", "learn.fit_one_cycle(3,1e-4, callbacks=[SaveModelCallback(learn, every='epoch', monitor='accuracy')])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "best.pth\t bestmodel_2.pth model_1.pth model_4.pth stage-1.pth\r\n", "bestmodel_0.pth bestmodel_3.pth model_2.pth model_5.pth tmp.pth\r\n", "bestmodel_1.pth model_0.pth\t model_3.pth one_epoch.pth trained_model.pth\r\n" ] } ], "source": [ "!ls ~/.fastai/data/mnist_sample/models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`ReduceLROnPlateauCallback`](/callbacks.tracker.html#ReduceLROnPlateauCallback)\n", "\n", "Reduce the learning rate each time a given metric/validation loss doesn't improve by a certain factor." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`PeakMemMetric`](/callbacks.mem.html#PeakMemMetric)\n", "\n", "GPU and general RAM profiling callback" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`StopAfterNBatches`](/callbacks.misc.html#StopAfterNBatches)\n", "\n", "Stop training after n batches of the first epoch." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [`train`](/train.html#train) and [`basic_train`](/basic_train.html#basic_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`Recorder`](/basic_train.html#Recorder)\n", "\n", "Track per-batch and per-epoch smoothed losses and metrics." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`ShowGraph`](/train.html#ShowGraph)\n", "\n", "Dynamically display a learning chart during training." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`BnFreeze`](/train.html#BnFreeze)\n", "\n", "Freeze batchnorm layer moving average statistics for non-trainable layers." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`GradientClipping`](/train.html#GradientClipping)\n", "\n", "Clips gradient during training." ] } ], "metadata": { "jekyll": { "keywords": "fastai", "summary": "Callbacks implemented in the fastai library", "title": "callbacks" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 2 }