{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Learning Rate Finder" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [], "source": [ "%matplotlib inline\n", "from fastai.gen_doc.nbdoc import *\n", "from fastai.vision import *\n", "from fastai.callbacks import *" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Learning rate finder plots lr vs loss relationship for a [`Learner`](/basic_train.html#Learner). The idea is to reduce the amount of guesswork on picking a good starting learning rate.\n", "\n", "**Overview:** \n", "1. First run lr_find `learn.lr_find()`\n", "2. Plot the learning rate vs loss `learn.recorder.plot()`\n", "3. Pick a learning rate before it diverges then start training\n", "\n", "**Technical Details:** (first [described]('https://arxiv.org/abs/1506.01186') by Leslie Smith) \n", ">Train [`Learner`](/basic_train.html#Learner) over a few iterations. Start with a very low `start_lr` and change it at each mini-batch until it reaches a very high `end_lr`. [`Recorder`](/basic_train.html#Recorder) will record the loss at each iteration. Plot those losses against the learning rate to find the optimal value before it diverges." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Choosing a good learning rate" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For a more intuitive explanation, please check out [Sylvain Gugger's post](https://sgugger.github.io/how-do-you-find-a-good-learning-rate.html)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "path = untar_data(URLs.MNIST_SAMPLE)\n", "data = ImageDataBunch.from_folder(path)\n", "def simple_learner(): return Learner(data, simple_cnn((3,16,16,2)), metrics=[accuracy])\n", "learn = simple_learner()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First we run this command to launch the search:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

lr_find[source][test]

\n", "\n", "> lr_find(**`learn`**:[`Learner`](/basic_train.html#Learner), **`start_lr`**:`Floats`=***`1e-07`***, **`end_lr`**:`Floats`=***`10`***, **`num_it`**:`int`=***`100`***, **`stop_div`**:`bool`=***`True`***, **`wd`**:`float`=***`None`***)\n", "\n", "
×

Tests found for lr_find:

To run tests please refer to this guide.

\n", "\n", "Explore lr from `start_lr` to `end_lr` over `num_it` iterations in `learn`. If `stop_div`, stops when loss diverges. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(Learner.lr_find)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n" ] } ], "source": [ "learn.lr_find(stop_div=False, num_it=200)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then we plot the loss versus the learning rates. We're interested in finding a good order of magnitude of learning rate, so we plot with a log scale. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEKCAYAAADXdbjqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VPW9//HXZyYbgRACCQIJS4AAAipCBJGrBWsB2wrSagvWFmur11a01uVeve1PLV6739Yu2IpUa7VCXapiS8WVqghCEBABkRBAwhrClj2zfH5/zAHGGMgyczKTyef5eMyDOeu8OQP55JzvOd+vqCrGGGNMJDyxDmCMMab9s2JijDEmYlZMjDHGRMyKiTHGmIhZMTHGGBMxKybGGGMiZsXEGGNMxKyYGGOMiZgVE2OMMRFLinWAaMnOztYBAwbEOoYxxrQra9asOaiqOZHux9ViIiJTgd8AXmCBqv60wfJfA5OcyXSgp6p2c5bNBn7oLPtfVX3sdJ81YMAAioqKohnfGGMSnojsjMZ+XCsmIuIF5gGfA0qB1SKyWFU3HV9HVb8ftv5NwLnO++7APUAhoMAaZ9vDbuU1xhjTem62mYwFilW1RFXrgUXA9NOsPwtY6LyfAryiqoecAvIKMNXFrMYYYyLgZjHJBXaFTZc68z5FRPoD+cDrLd3WGGNM7LlZTKSReafq734m8IyqBlqyrYhcLyJFIlJUVlbWypjGGGMi5WYxKQX6hk3nAXtOse5MTl7iava2qjpfVQtVtTAnJ+KbEYwxxrSSm8VkNVAgIvkikkKoYCxuuJKIDAWygBVhs5cCk0UkS0SygMnOPGOMMXHItbu5VNUvInMIFQEv8IiqbhSRuUCRqh4vLLOARRo25KOqHhKR+wgVJIC5qnrIrazGGGMiI4kybG9hYaG25jkTVeUn//qQc/t2Y0z/LHIyUhFprMkmtO6plhljTHskImtUtTDS/STME/CttftIDY+v2Mn8N0sASPF6OCMzlewuqRw4Vkd5VR3pKUn4A0FqfAEmDu3JFWPyGJjdmT7dOtE59eQh9AWCVNX5qfcHqfMHqfMHqPUFqQ8ECQSVzE7JZKWnkNkpmb1Ha9hWVklNfRCvB7LSU8jP6UzPjLQT+6v1BUj2evB6rIAZY+Jbhz8zAaj3B9mw+ygbSo+w91gt+47WUlZRR05GKj0zUqmuD5DkERT4x/t7OVRVf2LbbunJ9MnshNcjbNlXQX0gGNHfo1t6MqlJHqrrA1TU+vEIdO+cSn52OvnZnenXPZ2DlfXsOVJDUCEnI5XR/bqRkZbE8a8yMz2Z/j06k5Z0skmsPhCkpj5Al9QkOqcmoUBQNbSNOu8JnX0FFRSlU7KXjLTkRnMeX88KnTHtW7TOTKyYtFCtL8DGPUcpPVzD7iM17DlSw+7DNdQHgozok0nvzDRSkjykJnlJTfI470NnF0drfByuqudwtY+eGakUnJFBl9Qk/MEgh6rq2bq/kpKDlfj8Smqyh54ZqdT5gxw4Vsf28ipKyqo4WFlH5xQveVnpeDxC6eFqKmr9rv19u6QmkZbsIegUnGAwVERqfAGCqvTJ7ETvzDTSU5MIBIPU+oLU+gIEldDf3eshOUlI8nhI9goiwqGqeqrq/OQ6Z3a+QBBfIEh9QPH5gyQneejWKZnMTsknzvzSkj10SU2izh+kss5PVZ0fAVKTQ8c51TnmacnOsU8+OS8lyUOyN/T5nVOTQvtOTyY1yevacTOmvbDLXDGSluxlTP/ujOkf/X1fWND07c3V9X46JXtPtN0Eg8r28irqfEGON+ccqqpnZ3k1/mDoLEmdH+xpyR6q6gJU1/sR5MT6Hgm9F8DjkdBDPiLU1PvZc6QWXyCIRwSPgIjgEaFTigevCDsPVXPgWB1Ha3wkeYS05FAhEBHqA0F8/lCB8Qf8+AJKUJWs9BSysjqx+0itcylPnB/2oR/41dV+Pi6v4kiNj+q60KNH4Wd8SR45UWTq/AHq/EFa8ztRp2QvnVOT6JLqpUtaEr26ptErM40kj4funVMYlNOFsfndyclIbfnOjelgrJi0M+kpn/zKPB5hUE6XT603YXBbJWob/kCQqrrAiTOO8BshVBVfQE8Uljp/kDpf6H2tL4AvoM6ZT6hN60i1j6M1Po5U11NZF6Cqzk9FrY/SwzWs2XkYf1CprPOfKFDn9O3GxUN7cv7A7gzr1ZXM9MYv/RnTkVkxMe1CktdDZnrjj0WJCClJQkqSh4wofV5NfYCtByp486MyXvvwAA+89hH6amhZr65pDOmVwbBeGYzpn8VFBTl0SrFLZqZjszYTY5qhvLKODbuPsmVfRei1v4KtByqp9wfplOzla+P6ccPEQWR3sUtipn2xBvgGrJiYtuYLBFm1/RDPrinl+XW7SUnycMWYPL71HwPJz+4c63jGNIsVkwasmJhY2lZWyUP/3sbza/fgCwa55MwzuPVzQzizd9dYRzPmtKyYNGDFxMSDAxW1PL5iJ39ZsZOKWh9fPa8vd0490xrtTdyKVjFxs6NHYzqcnhlp3DZ5KP++YyLXXJDPU0WlXPLrf/PSB3tjHc0YV1kxMcYF3dJTuPuy4bxw4wR6ZqRywxPv8Z0n1lBeWRfraMa4woqJMS4amZvJ8zdO4L+nDuO1Dw/w+d++xart1gG2STxWTIxxWbLXw3cmDuK5715AekoSX//Tu6zfdSTWsYyJKismxrSREX0yeeaG8eRkpHL940XsO1ob60jGRI0VE2PaUI8uqTz8jUIqav184bdv8eL6U41kbUz7YsXEmDZ2Zu+uPPfdCeRldeKmhWt5fOXOWEcyJmJWTIyJgaG9Mvj7dydw8bCe/GjxRlaWlMc6kjERcbWYiMhUEdkiIsUicucp1vmKiGwSkY0i8mTY/ICIrHNeixvb1pj2zOsRHpg5iv490rnxr++x/5i1oZj2y7ViIiJeYB5wKTAcmCUiwxusUwDcBUxQ1RHALWGLa1R1lPOa5lZOY2Kpa1oyD319DNX1Ab7/t3UEgonRI4XpeNw8MxkLFKtqiarWA4uA6Q3WuQ6Yp6qHAVT1gIt5jIlLg3tm8KNpI3hnWzkPv1US6zjGtIqbxSQX2BU2XerMCzcEGCIiy0VkpYhMDVuWJiJFzvzLG/sAEbneWaeorKwsuumNaUNXFuYxefgZPPDqR5Qero51HGNazM1iIo3Ma3gOnwQUABOBWcACEenmLOvndD52FfCAiAz61M5U56tqoaoW5uQ0PeStMfFKRLhn2ggE4Ucvbop1HGNazM1iUgr0DZvOAxreVF8KvKCqPlXdDmwhVFxQ1T3OnyXAMuBcF7MaE3O53TrxvUsKeGXTfl7dtD/WcYxpETeLyWqgQETyRSQFmAk0vCvreWASgIhkE7rsVSIiWSKSGjZ/AmC/rpmEd+2EfAp6duHeFzdSUx+IdRxjms21YqKqfmAOsBTYDDylqhtFZK6IHL87aylQLiKbgDeAO1S1HDgTKBKR9c78n6qqFROT8FKSPNx3+UhKD9cw743iWMcxptlscCxj4tDNC9fyyqb9rLjrYrqlp8Q6jklgNjiWMQnsu5MGUeML8Nd3P451FGOaxYqJMXFoWK+uXFiQzWPv7KDeH4x1HGOaZMXEmDj17QsHcqCijufX7Y51FGOaZMXEmDh1UUE2Z+Vm8ptXt1Lrszu7THyzYmJMnBIR7rp0GLuP1PCXFTtiHceY07JiYkwcu2BwNhOH5vD714s5WuOLdRxjTsmKiTFx7vbJQzlW6+fpol1Nr2xMjFgxMSbOjczN5LwBWfxlxU7rot7ELSsmxrQDsy8YwMeHqnnjQxulwcQnKybGtANTRvSiV9c0/mLjxZs4ZcXEmHYg2evhysI83t5axoEKG97XxB8rJsa0E9NH9SGo8M/398Y6ijGfYsXEmHZicM8MzuzdlcXrGw4LZEzsWTExph2Zdk4f1n58hI/LbWhfE1+smBjTjlx2Tm8AXnzfzk5MfLFiYkw7kpeVTmH/LF6wzh9NnLFiYkw7M31UHz7aX8mH+47FOooxJ7haTERkqohsEZFiEbnzFOt8RUQ2ichGEXkybP5sEdnqvGa7mdOY9uTzZ/XG6xFeWGeXukz8cK2YiIgXmAdcCgwHZonI8AbrFAB3ARNUdQRwizO/O3APMA4YC9wjIlluZTWmPenRJZX/GJzNi+v3kCjDbpv2z80zk7FAsaqWqGo9sAiY3mCd64B5qnoYQFWP9xUxBXhFVQ85y14BprqY1Zh25fJz+1B6uIblxeWxjmIM4G4xyQXCuzktdeaFGwIMEZHlIrJSRKa2YFtjOqxLR/amR+cUHl2+PdZRjAHcLSbSyLyG5+RJQAEwEZgFLBCRbs3cFhG5XkSKRKSorKwswrjGtB9pyV6+Nq4fr285wI6DVbGOY4yrxaQU6Bs2nQc0bDEsBV5QVZ+qbge2ECouzdkWVZ2vqoWqWpiTkxPV8MbEu6vP70+SR3hsxY5YRzHG1WKyGigQkXwRSQFmAosbrPM8MAlARLIJXfYqAZYCk0Uky2l4n+zMM8Y4enZN4wtn9ebpolIqam0URhNbrhUTVfUDcwgVgc3AU6q6UUTmisg0Z7WlQLmIbALeAO5Q1XJVPQTcR6ggrQbmOvOMMWG+OSGfyjo/z6wpjXUU08FJotxaWFhYqEVFRbGOYUybm/Hgcg5X1fP6bRPxeBprbjTm1ERkjaoWRrofewLemHbumxPy2VFezbKPbBRGEztWTIxp5y4d2YucjFSefHdX0ysb4xIrJsa0c8leD1eMyeONLQfYf8xGYTSxYcXEmATw1cK+BILK00V2dmJiw4qJMQlgQHZnxg/swaLVuwgEE+OmGtO+WDExJkF8Y3x/Sg/X8Ox7dpuwaXtWTIxJEFNH9mJU327838tbqKkPxDqO6WCsmBiTIESEH3zhTPYfq+NPb5fEOo5pI3uP1rBpT+wHSrNiYkwCOW9Adz47rCePLt9BvT8Y6zjGZat3HOKy373NzYvWEoxxW5kVE2MSzDcuGEB5VT0vbdwX6yjGJarK4yt2MGv+SrqmJfPHq0fHvPcDKybGJJgLB2fTr3s6f125M9ZRjAvq/UHufHYD/++FjVw0JIfn50xgcM+MWMeyYmJMovF4hKvG9ePd7YfYur8i1nFMFB2uqufqP73L34p2MWfSYBZ8o5CuacmxjgVYMTEmIV05Jg+vR3hu7e5YRzFRsq2skhkPLmfdriP8ZuYobp8yNOaXtsJZMTEmAfXoksoFg3qwZMNeEqVn8I7sneKDzJi3nIpaPwuvO5/po+JvFHMrJsYkqM+f1Zsd5dVs2hv720ZN6y1a9THfeGQVvTLTeP7GCYzpnxXrSI2yYmJMgpoyohdej7Bkw95YRzGtoKr86pWPuPPvG7hgcDbPfOcC+nZPj3WsU7JiYkyC6t45hfEDe7Bkwz671NXO+ANB/ue5Dfz2ta18pTCPR2bHT0P7qVgxMSaBff6s3mw/WMXmvXZXV3tR6wtwwxPvsXBV6I6tn335bJK88f+j2tWEIjJVRLaISLGI3NnI8mtEpExE1jmvb4ctC4TNX+xmTmMS1ZQRZ9ilrnbkSHU9X1vwLq99uJ8fTRvB7VOGIhI/d2ydTpJbOxYRLzAP+BxQCqwWkcWquqnBqn9T1TmN7KJGVUe5lc+YjqBHl1TOH9idf27Yy22Th7SbH0wd0YFjtXxtwbvsLK/m97NG84Wze8c6Uou4eWYyFihW1RJVrQcWAdNd/DxjTCPsUlf8232khq88tILdR2r487XntbtCAu4Wk1wgfNi3UmdeQ18WkfdF5BkR6Rs2P01EikRkpYhc7mJOYxLalBG98Ah2qStO7Syv4it/XEF5VT1PfHscFwzKjnWkVnGzmDR2Pt3wlpIXgQGqejbwKvBY2LJ+qloIXAU8ICKDPvUBItc7BaeorKwsWrmNSSjZXVK5YFA2z6/bHfOeZc0nFR+o4Mo/rqC6PvQw4uh+8fkMSXO4WUxKgfAzjTxgT/gKqlquqnXO5MPAmLBle5w/S4BlwLkNP0BV56tqoaoW5uTkRDe9MQnkijF5lB6uYeX28lhHMY5Ne47x1YdWElRYdP14RuZmxjpSRNwsJquBAhHJF5EUYCbwibuyRCT8wuA0YLMzP0tEUp332cAEoGHDvTGmmaaM6EVGahLPFNmQvvFg055jzHp4JSlJHp76z/MZ2iv2vf5GyrVioqp+YA6wlFCReEpVN4rIXBGZ5qx2s4hsFJH1wM3ANc78M4EiZ/4bwE8buQvMGNNMnVK8fPGcPiz5YC8Vtb5Yx+nQtuyr4Oo/vUvnFC9P/ed4BuZ0iXWkqJBEeTK2sLBQi4qKYh3DmLi19uPDzHjwHe6bPoKvjx8Q6zgdUvGBCmbOX4nXI/zt+vEMyO4c60iIyBqnfToi8f9YpTEmKkb17cY5fbvxp7e3E7CG+DZXUlbJrIffBYQnrzs/LgpJNFkxMaaDEBGuuzCfHeXVvLp5f6zjdCg7y6u46uF3CQaVhdeNY1CCXNoKZ8XEmA5k6ohe5GV1Yv6bJdb5YxvZd7SWqx5+l1p/gCe+PY6CM9p/Y3tjrJgY04EkeT1cd+FA1uw8zNvFB2MdJ+Edqa7nG4+8y9EaH49fO44ze3eNdSTXWDExpoOZObYvud068fOXtthDjC6qrvdz7Z9Xs+NgNfO/MYaz8tr3cyRNsWJiTAeTmuTl1s8NYcPuoyz5wLpYcYMvEOS7f32PdbuO8NtZo9ptFyktYcXEmA7o8nNzyc/uzBMrd8Y6SsIJBpU7nl7Psi1l3D/jLKaObH+dNraGFRNjOiCvR5gyohdFOw5zzB5ijKr//edmnl+3hzumDGXW2H6xjtNmrJgY00FNGpqDP6gs32oN8dHy6PLtPLJ8O9+cMIDvTvxU37QJzYqJMR3U6P5ZZKQl8caWA7GOkhBe3bSf+/6xicnDz+CHXxje4QYia1YxEZFBYR0vThSRm0Wkm7vRjDFuSvZ6uGhIDm9sKbO7uiK0ofQoNy1cy8jcTB6YOQqvp2MVEmj+mcmzQEBEBgN/AvKBJ11LZYxpE5OG9qSsoo6H3yqh1heIdZx2ac+RGr712Gq6d05hwexC0lNcGw09rjW3mASdXoBnAA+o6veBjnGLgjEJbMqIMxib352f/OtDpv9+ufXZ1UIVtT6u/fNqauoDPHLNefTMSIt1pJhpbjHxicgsYDbwD2desjuRjDFtJSMtmb9dfz73zxjJlv0VvLPNGuObKxBUblq4lq0HKnnw6tEJMSZJJJpbTL4JjAfuV9XtIpIPPOFeLGNMWxERrhiTR9e0JP7+3u5Yx2k3fvnyFpZtKWPu9BFcWGAjvTarmKjqJlW9WVUXikgWkKGqP3U5mzGmjaQmhQbPeumDfVTV+WMdJ+69uH4Pf1i2javG9eNr4/rHOk5caO7dXMtEpKuIdAfWA4+KyK/cjWaMaUtfOjeXGl+Alz7YF+socW3TnmPc8cx6Cvtnce9lI2IdJ2409zJXpqoeA74EPKqqY4BL3ItljGlrY/pn0a97On9fa+PEn8qhqnqu+0sR3Tql8ODVo0lJskf1jmvukUgSkd7AVzjZAN8kEZkqIltEpFhE7mxk+TUiUiYi65zXt8OWzRaRrc5rdnM/0xjTOiLCl0bn8s62cvYerYl1nLjjDwS58a/vUVZZx0NfH9Oh79xqTHOLyVxgKbBNVVeLyEBg6+k2EBEvMA+4FBgOzBKR4Y2s+jdVHeW8FjjbdgfuAcYBY4F7nLYaY4yLZpybiyo8v3ZPrKPElXp/kP95bgMrSsr5yYyzOKevPbPdULOerlHVp4Gnw6ZLgC83sdlYoNhZFxFZBEwHNjXjI6cAr6jqIWfbV4CpwMLm5DXGtE7/Hp0p7J/F398r5YbPDOxwXYIcd6zWx/6jtaQle9m45ygPLtvG+6VHueniwXx5TF6s48WlZhUTEckDfgdMABR4G/ieqp7u4mousCtsupTQmUZDXxaRi4CPgO+r6q5TbJvbSK7rgesB+vXrOL1zGuOmGaNz+cFzH3DL39Zx7YT8DvVbuC8QZMFb23nwjWIqwu5qy0pP5qGvj2HKiF4xTBffmvvc/6OEuk+50pm+2pn3udNs09ivNA0fr30RWKiqdSJyA/AYcHEzt0VV5wPzAQoLC+3RXWOi4Muj89iyr4Ln1+7m5Y37WXnXZ8lM7xjPKM9/s4RfLN3CJWf25LJz+lBdH2DIGV04O68byV5rbD+d5h6dHFV9VFX9zuvPQFNP6ZQCfcOm84BPXIhV1XJVrXMmHwbGNHdbY4w70pK9zJ0+kkXXj6fGF+Cpol1Nb5QAan0BHl2+nc8MyWHB7POYPiqXWWP7MaZ/dyskzdDcI3RQRK4WEa/zuhoob2Kb1UCBiOSLSAowE1gcvoJzh9hx04DNzvulwGQRyXIa3ic784wxbWR4n66Mze/OYyt2dIg+u559r5SDlfXc8JmONQ5JtDS3mFxL6LbgfcBe4ApCXaycktMx5BxCRWAz8JSqbhSRuSIyzVntZhHZKCLrgZuBa5xtDwH3ESpIq4G5xxvjjTFt55sXDKD0cA2vbd4f6yiuUlUefrOEc/IyOX9g91jHaZdEtXW/cYjILar6QJTztFphYaEWFRXFOoYxCcUfCDLxl8vISk/hhRsn4EnQcTqq6vyMuGcp/z11GN/pYCMkisgaVS2MdD+RXAi8NdIPN8bEtySvh9smD2HD7qO8+H7iNlv6A6Ffqu2J9taL5Mgl5q8oxphPmH5OLsN7d+UXS7dQ50/MAbT8wSAAyV77sdZakRSTxG+RM8bg8Qj/8/kzKT1cw+MrdsY6jiv8zg0GHXG43Wg5bTERkQoROdbIqwLo00YZjTEx9h8F2Vw0JIffvV7M0WpfrONE3fFikuyxy1ytddojp6oZqtq1kVeGqnbMgY6N6aDunDqMY7U+5i0rjnWUqPMHQpe5kuwyV6tZGTbGNMvwPl25fFQuT6zcSXV9Yg2g5QvYZa5IWTExxjTbrLH9qK4PsHRjYg2gdfyhTHvSvfXsyBljmq2wfxZ5WZ0Sbqx4n3OZy85MWs+KiTGm2TweYca5uSwvPsj+Y7WxjhM1J89MrJi0lhUTY0yLzDg3l6DCc2sT5+zk+HMmSXY3V6vZkTPGtMjAnC6cNyCLhas+JpggHUAeb4BPsstcrWbFxBjTYl8fP4Cd5dW8ubUs1lGi4vhlriRrgG81O3LGmBabOqIX2V1SE+aJeGuAj5wVE2NMi6UkeZg1ti+vbznAim1NDW0U/6wBPnJWTIwxrfLNCfkMzunCNY+u4o0PD8Q6TkROtpnYj8TWsiNnjGmV7p1TeOo/xzO4ZxdufWod9f5grCO12om7uezMpNWsmBhjWi2rcwp3TBnK4Wpfux6N8UQDvLWZtJqrxUREporIFhEpFpE7T7PeFSKiIlLoTA8QkRoRWee8/uhmTmNM611YkEOvrmk8VbQr1lFazS5zRc61nn9FxAvMAz4HlAKrRWSxqm5qsF4GofHf322wi22qOsqtfMaY6PB6hC+PyeUPy7ax72gtvTLTYh2pxazX4Mi5WYbHAsWqWqKq9cAiYHoj690H/BxInL4ZjOlgrhzTl6DC31a3z7MT/4nnTKyYtJabxSQXCP+XVerMO0FEzgX6quo/Gtk+X0TWisi/ReRCF3MaYyI0ILszFw/ryZ/f2U5VXfvrnv7EmYld5mo1N49cYyX+RN8LIuIBfg3c1sh6e4F+qnoucCvwpIh0/dQHiFwvIkUiUlRWlhhP4hrTXt04aTCHq30sXPVxrKO0mJ2ZRM7NYlIK9A2bzgP2hE1nACOBZSKyAzgfWCwihapap6rlAKq6BtgGDGn4Aao6X1ULVbUwJyfHpb+GMaY5xvTP4oJBPXjozRJqfYFYx2kRv93NFTE3i8lqoEBE8kUkBZgJLD6+UFWPqmq2qg5Q1QHASmCaqhaJSI7TgI+IDAQKgBIXsxpjomDOpMGUVdTxdDu7s8suc0XOtSOnqn5gDrAU2Aw8paobRWSuiExrYvOLgPdFZD3wDHCDqh5yK6sxJjrGD+rB6H7d+OO/S070d9Ue2JlJ5Fy7NRhAVZcASxrMu/sU604Me/8s8Kyb2Ywx0ScizLl4MNf+uYjn1u7mK4V9m94oDvgDikdCg3+Z1rFzOmNMVE0a2pPhvbvyh2XbTjxZHu/8QbXu5yNkR88YE1Uiwk0XD2b7wSr+uWFvrOM0iz8QtEtcEbJiYoyJuikjejG4ZxfmvV7cLkZj9AfVikmErJgYY6LO4xFunDSILfsreK0ddE/vDwZJtstcEbGjZ4xxxWVn96F3Zhp/WbEj1lGa5A+ojbIYISsmxhhXJHk9fPW8vry19SAfl1fHOs5p+YNqZyYRsqNnjHHNV8/ri0dg4er47mLFHwjamUmErJgYY1zTO7MTFw87g6eLdsX1SIy+oFq/XBGyYmKMcdU3xvfnYGU9f3+vNNZRTikQUJKtK5WI2NEzxrjqwoJszs7L5MFl2070gRVv/EG7zBUpKybGGFeFHmIs4OND1Sxev6fpDWIg1ABvxSQSVkyMMa675MyenNm7K7965SNq6uOve3q7NThyVkyMMa4TEe69bDilh2t44LWPYh3nU3yBoPXNFSE7esaYNjFuYA9mnteXBW9tZ+Oeo7GO8wkBu8wVMSsmxpg2c9elZ5LZKZkfLd6Eavz02eULKl67mysidvSMMW0mMz2Z2yYPYdWOQyzZsC/WcU4IBIMkW5tJRKyYGGPa1Mzz+jGsVwY/XrI5bsaKtwb4yFkxMca0Ka9H+H9fHM7uIzU8+W58dLPiC1ivwZFy9eiJyFQR2SIixSJy52nWu0JEVEQKw+bd5Wy3RUSmuJnTGNO2JgzO5oJBPXhwWTHV9f5YxyFg3alEzLViIiJeYB5wKTAcmCUiwxtZLwO4GXg3bN5wYCYwApgKPOjszxiTIG6bPJSDlfU8unxHrKPgs8tcEXPzzGQsUKyqJapaDywCpjey3n3Az4HasHnTgUWqWqeq24FiZ3/GmAQxpn8Wk4bmsOCtEqrqYnt2Egha31yRcvPo5QK7wqZLnXkniMi5QF9V/UdLt3W2v15EikSkqKysLDqpjTFt5qbPFnC42hfzthN/MIjXLnMagU7zAAAQnElEQVRFxM1i0tg3c+LGchHxAL8GbmvptidmqM5X1UJVLczJyWl1UGNMbIzul8WEwT2Y/1ZJTO/s8gXUbg2OkJvFpBToGzadB4T38pYBjASWicgO4HxgsdMI39S2xpgEceOkwZRV1LFwVezOTkIN8HaZKxJuHr3VQIGI5ItICqEG9cXHF6rqUVXNVtUBqjoAWAlMU9UiZ72ZIpIqIvlAAbDKxazGmBgZP7AH4wf24PevF1MZo7YTXyBIkp2ZRMS1YqKqfmAOsBTYDDylqhtFZK6ITGti243AU8Am4CXgRlWNj6ebjDFRJSL896XDKK+qZ8FbJTHJYLcGRy7JzZ2r6hJgSYN5d59i3YkNpu8H7nctnDEmbozq241LR/bi4TdLmDW2H2d0TWuzz1ZV/NY3V8Ts6Blj4sKdlw7DF1Tu/+fmNv1cfzB0b481wEfGiokxJi7079GZ73xmEIvX72F58cE2+9yAU0ysAT4ydvSMMXHjOxMH0a97Ovf9YxPBYNt0Ue9zxqW3BvjIWDExxsSNtGQvt35uCB/uq+CljW3TRf3JMxMrJpGwYmKMiSuXndOHQTmd+fUrH534Qe8mX8ApJnZmEhErJsaYuOL1CLdcMoStByq5d/HGE5eh3OIPOpe5rM0kIq7eGmyMMa3xhbN6s37XERa8vZ1tZZU8/q1xrvXq67czk6iwUmyMiTsej/DDLw5n7vQRvLOtnH99sNe1z/Jbm0lUWDExxsStr43rz+CeXfj968Wu3d3lP3E3l/04jIQdPWNM3PJ6hDmTBvPhvgp+8PwG7nnhAw5W1kX1M06cmdhlrohYm4kxJq598ezePLismIWrQkMciQj3ThsRtf2faDOxBviI2NEzxsS1JK+HxXP+g81zp/KVwjwWrvqYAxW1TW/YTCfv5rIzk0hYMTHGxL20ZC+dUrx8Z+JgfIEgf3pre9T2bZe5osMucxlj2o387M5cdk4fHn6rhMo6P7dPHkpW55SI9umzBviosGJijGlX7rt8JFnpKTy+cifbD1bx12+PQ6T1ZxXWnUp0WCk2xrQrXdOSuXfaCO7+4nDe2VbOq5sPRLQ/e2gxOqyYGGPapavG9WNQTmd+vGQz9f7Wd7lyYjwTu5srIq4ePRGZKiJbRKRYRO5sZPkNIrJBRNaJyNsiMtyZP0BEapz560Tkj27mNMa0P8leDz/84nC2H6zixiffo87fupG9jz+06FZ3LR2Fa8VERLzAPOBSYDgw63ixCPOkqp6lqqOAnwO/Clu2TVVHOa8b3MppjGm/Jg3tyY+mjeCVTfv51p+LOFbra/E+fCfOTKyYRMLNM5OxQLGqlqhqPbAImB6+gqoeC5vsDLTNaDjGmIQx+4IB/PLKc1hZUs6XHnyHXYeqW7R9IHj8zMQuc0XCzaOXC+wKmy515n2CiNwoItsInZncHLYoX0TWisi/ReRCF3MaY9q5K8bk8fi3xnHgWC23PbUe1eb/XmrjmUSHm8WksW/mU9+wqs5T1UHAfwM/dGbvBfqp6rnArcCTItL1Ux8gcr2IFIlIUVlZWRSjG2Pam/GDevBfU4exaschXmvBHV4Ba4CPCjePXinQN2w6D9hzmvUXAZcDqGqdqpY779cA24AhDTdQ1fmqWqiqhTk5OVELboxpn756Xl8GZnfmZy992OxRGq0BPjrcLCargQIRyReRFGAmsDh8BREpCJv8ArDVmZ/jNOAjIgOBAqDExazGmASQ7PXwX1OHsvVAJb98eUuztjl+mcsa4CPj2hPwquoXkTnAUsALPKKqG0VkLlCkqouBOSJyCeADDgOznc0vAuaKiB8IADeo6iG3shpjEseUEb24alw//rBsGwU9u/Cl0XmnXf/4GYydmUTG1e5UVHUJsKTBvLvD3n/vFNs9CzzrZjZjTGISEX40bQQlZZXc/vR6Kmr9zL5gwCnX9zl3c1mbSWTs6BljEk6y18OC2edx8bCe3LN4IwveOvVV8oDdzRUVVkyMMQmpS2oSD329kIlDc/jNq1s5XFXf6Ho+u8wVFVZMjDEJy+sR7rx0GJX1fv74722NruMPBEnySEQ9DxsrJsaYBDesV1dmjMrlz+/s4OPyTz8dHwiqnZVEgRUTY0zCu23KUFKSPNy08L1P9TDsC6g1vkeBHUFjTMLL7daJX1xxNutLj/Kzlz78xLJAMGgDY0WBFRNjTIcwdWRvrj6/H48u387mvSf7mPUF1e7kigIrJsaYDuP2yUPJSEvmf/+56URnkKEGePtRGCk7gsaYDqNbegq3XFLA8uKTw/36rQE+KqyYGGM6lKvP709Bzy7c/cIHVNT68AfU+uWKAismxpgOJdnr4WdXnM2+Y7X8eMmH+AJBkuxuroi52jeXMcbEo9H9srh2Qj5/ens7AMN6ZcQ4UftnxcQY0yHdeekwRvTpyvulRzkrNzPWcdo9KybGmA4p2evhS6Pzmuyi3jSPXSg0xhgTMSsmxhhjImbFxBhjTMSsmBhjjImYq8VERKaKyBYRKRaROxtZfoOIbBCRdSLytogMD1t2l7PdFhGZ4mZOY4wxkXGtmIiIF5gHXAoMB2aFFwvHk6p6lqqOAn4O/MrZdjgwExgBTAUedPZnjDEmDrl5ZjIWKFbVElWtBxYB08NXUNVjYZOdAXXeTwcWqWqdqm4Hip39GWOMiUNuPmeSC+wKmy4FxjVcSURuBG4FUoCLw7Zd2WDbXHdiGmOMiZSbxaSxntP0UzNU5wHzROQq4IfA7OZuKyLXA9c7k5UisqUF+TKBoy1Y1nBe+HRj7xvOSwYOtiBftDOeLlv4vOwW5jxdxtbkPF221mZsKmc0MjaWN5rH0r7vlknE77s1OZuTsX8LMp+aqrryAsYDS8Om7wLuOs36HuBoY+sCS4HxUc43vyXLGs4Ln27sfcN5QFEsM54uW4M/W5TzdBmjfSxbm7Etvm+3j6V93/Z9x+pYNvflZpvJaqBARPJFJIVQg/ri8BVEpCBs8gvAVuf9YmCmiKSKSD5QAKyKcr4XW7is4bwXm3h/quUtEc2M4dNN5W2JpraL5rFsbcamto1GxvD39n2ffp59301vd6plsTiWzSJOtXJn5yKfBx4AvMAjqnq/iMwlVB0Xi8hvgEsAH3AYmKOqG51tfwBcC/iBW1T1X64FbQMiUqSqhbHO0ZT2kLM9ZIT2kdMyRk97yOlmRlc7elTVJcCSBvPuDnv/vdNsez9wv3vp2tz8WAdopvaQsz1khPaR0zJGT3vI6VpGV89MjDHGdAzWnYoxxpiIWTFpBRF5REQOiMgHrdh2jNOFTLGI/FZEJGzZTU73MRtF5OfxmFNE7hWR3U4XOOucdrG4yhi2/HYRURHJjreMInKfiLzvHMOXRaRPJBldzPkLEfnQyfqciHSLw4xXOv9ngiLS6vaASLKdYn+zRWSr85rd1N8jzjLeLyK7RKSy2Ttz6zaxRH4BFwGjgQ9ase0qQrdNC/Av4FJn/iTgVSDVme4ZpznvBW6P52PpLOtL6JbynUB2vGUEuoatczPwx3g8lsBkIMl5/zPgZ3GY8UxgKLAMKGzrbM7nDmgwrztQ4vyZ5bzPaurfbRxlPB/oDVQ29zPszKQVVPVN4FD4PBEZJCIvicgaEXlLRIY13E5EehP6IbJCQ9/YX4DLncXfAX6qqnXOZxyI05xR5WLGXwP/RSMPu8ZDRj11V0LxlvNlVfU7q64EIhqW0KWMm1W1JQ8sRzXbKUwBXlHVQ6p6GHgFmBrp/622yOh8zkpV3dvcXGCXuaJpPnCTqo4BbgcebGSdXEJdwxwX3k3MEOBCEXlXRP4tIufFaU6AOc5lj0dEJCveMorINGC3qq53IVtUMjo57xeRXcDXgLtxRzS+7+OuJfSbdLRFM2MssjWmse6kcnHn7xHtjK1iY8BHgYh0AS4Ang67/Jna2KqNzDv+G2kSoVPN84HzgKdEZKDz20s85fwDcJ8zfR/wf4R+yMRFRhFJB35A6PKMK6J0HFHVHwA/EJG7gDnAPfGY09nXDwg98/XXeM0YbafLJiLfBI4/2jAYWCIi9cB2VZ1xmrxR/Xu4lLFVrJhEhwc4oqGu9E+QULf5a5zJxYR+EIdfJsgD9jjvS4G/O8VjlYgECfWjUxZPOVV1f9h2DwP/iGK+aGQcBOQD653/XHnAeyIyVlX3xUnGhp4E/kmUi0m0cjoNs18EPhvNX26imdEljWYDUNVHgUcBRGQZcI2q7ghbpRSYGDadR6jdopTo/j3cyNg6LWnEsdcnGq8GENYIBrwDXOm8F+CcU2y3mtDZx/HGt887828A5jrvhxA6/ZQ4zNk7bJ3vExoqIK4yNlhnBxE2wLt0HAvC1rkJeCZO/11OBTYBOdHI5+b3TYQN8K3Nxqkbt7cTutqQ5bzv3tx/t7HOGLZOsxvgo/KPo6O9gIXAXkLdwJQC3yL02/BLwHrnP9/dp9i2EPgA2Ab8npMPjqYATzjL3gMujtOcjwMbgPcJ/cbYO94yNlhnB5HfzeXGcXzWmf8+oX6TcuP0+y4m9IvNOucV0V1nLmWc4eyrDthPWAezbZGNRn5QO/OvdY5fMfDNlvy7jYOMP3f2H3T+vLepbPYEvDHGmIjZ3VzGGGMiZsXEGGNMxKyYGGOMiZgVE2OMMRGzYmKMMSZiVkxMQmtRr6fR+bwFIjI8SvsKSKhX4Q9E5EVposdeEekmIt+Nxmcb01J2a7BJaCJSqapdori/JD3Z8aGrwrOLyGPARxoagfRU6w8A/qGqI9sinzHh7MzEdDgikiMiz4rIauc1wZk/VkTeEZG1zp9DnfnXiMjTIvIi8LKITBSRZSLyjITG+viryImxNZaJM6aGiFQ6nTmuF5GVInKGM3+QM71aROY28+xpBSc7suwiIq+JyHsSGhdjurPOT4FBztnML5x173A+530R+VEUD6Mxn2DFxHREvwF+rarnAV8GFjjzPwQuUtVzCfXi++OwbcYDs1X1Ymf6XOAWYDgwEJjQyOd0Blaq6jnAm8B1YZ//G+fzm+yXyemn6rOEehwAqAVmqOpoQuPg/J9TzO4EtqnqKFW9Q0QmAwXAWGAUMEZELmrq84xpDevo0XRElwDDw3pZ7SoiGUAm8JiIFBDqPTU5bJtXVDV8HIlVqloKICLrCPWX9HaDz6nnZEeYa4DPOe/Hc3IMiyeBX54iZ6ewfa8hNN4EhPpf+rFTGIKEzljOaGT7yc5rrTPdhVBxefMUn2dMq1kxMR2RBxivqjXhM0Xkd8AbqjrDaX9YFra4qsE+6sLeB2j8/5JPTzZKnmqd06lR1VEikkmoKN0I/JbQ+Cc5wBhV9YnIDiCtke0F+ImqPtTCzzWmxewyl+mIXiY0fggAInK8++5MYLfz/hoXP38loctrADObWllVjxIa2vd2EUkmlPOAU0gmAf2dVSuAjLBNlwLXOmNeICK5ItIzSn8HYz7BiolJdOkiUhr2upXQD+ZCp1F6E6Hu/yHUU+pPRGQ54HUx0y3ArSKyitA420eb2kBV1xLqFXYmoQGqCkWkiNBZyofOOuXAcudW4l+o6suELqOtEJENwDN8stgYEzV2a7AxbcwZDbJGVVVEZgKzVHV6U9sZE8+szcSYtjcG+L1zB9YRojjssTGxYmcmxhhjImZtJsYYYyJmxcQYY0zErJgYY4yJmBUTY4wxEbNiYowxJmJWTIwxxkTs/wPm0tPMN7CA7QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then, we choose a value that is approximately in the middle of the sharpest downward slope. This is given as an indication by the LR Finder tool, so let's try 1e-2." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
10.1274340.0702430.97301300:02
20.0507030.0394930.98478900:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "simple_learner().fit(2, 1e-2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Don't just pick the minimum value from the plot!" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
10.7272210.6931470.49558400:02
20.6938260.6931470.49558400:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn = simple_learner()\n", "simple_learner().fit(2, 1e-0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Picking a value before the downward slope results in slow training:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
10.1528970.1343660.95093200:02
20.1209610.1175500.96074600:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn = simple_learner()\n", "simple_learner().fit(2, 1e-3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Suggested LR" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you pass `suggestion=True` in `learn.recorder.plot`, you will see the point where the gardient is the steepest with a \n", "red dot on the graph. We can use that point as a first guess for an LR." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n" ] } ], "source": [ "learn.lr_find(stop_div=False, num_it=200)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Min numerical gradient: 5.25E-03\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEKCAYAAADXdbjqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8lOW5//HPNTPZIAtgFiAJhCUgO5SAWtSiRcWloK210OWn3TjaetTaeo7Wnmqxtj3t6bHa2vpTj9Zfl0NRqwU3xIWqKEqQPYAkgBKWBMKShexz/f6YRxxjgCzz5JmZXO/Xa16Z537uZ+abCeTKs923qCrGGGNMd/i8DmCMMSb2WTExxhjTbVZMjDHGdJsVE2OMMd1mxcQYY0y3WTExxhjTbVZMjDHGdJsVE2OMMd1mxcQYY0y3BbwOECmZmZlaUFDgdQxjjIkpa9asOaiqWd19nbgpJgUFBRQXF3sdwxhjYoqIvB+J13H1MJeIzBaRbSJSKiK3trP+HhFZ5zzeE5EjYeuuFpHtzuNqN3MaY4zpHtf2TETED9wPXACUA6tFZImqlnzYR1W/F9b/X4EpzvMBwB1AEaDAGmfbw27lNcYY03Vu7plMB0pVdYeqNgGLgLkn6T8f+F/n+UXAclU95BSQ5cBsF7MaY4zpBjeLSS6wO2y53Gn7BBEZCgwDXunstsYYY7znZjGRdtpONHnKPOAJVW3tzLYiskBEikWk+MCBA12MaYwxprvcLCblQH7Ych6w9wR95/HRIa4Ob6uqD6pqkaoWZWV1+8o2Y4wxXeRmMVkNFIrIMBFJJFQwlrTtJCKjgf7AW2HNy4ALRaS/iPQHLnTajDHGRCHXruZS1RYRuZ5QEfADj6jqZhFZCBSr6oeFZT6wSMPmD1bVQyJyF6GCBLBQVQ+5lfVkqmob2bDnKOWHjnG0vpkxg9KZnN+PvkkBkgI+RNo7ImeMMb2LxMsc8EVFRdqVmxYbmltZ8Kc15PZL4bS+iRxraqWusYXDx5oo2VdN+eH6E26bmZrElCH9GJ7VlxGZqUwZ0o+A38e+I/VUN7RQ09BMbWMLB2oaqaxppDWoBHxCanKAusYWDtU10dgSJMHv47S+idQ3t1LT0MKAvokkJ/hpbGmlsSVIY3OQxpbQ6aTkBD8pCX6SE3ykJPjx+QRVyEpLIiMlgYrqBvYeaaCypoEEv4+gKh9UHaOmsQVVUFWCqgSd56oQVEXheLsAp/VNJCs9mazUJPok+vEJHK1v5kh9M0ePNaNAgl9I8PsI+H0EfMKRY00crW+mT2KAtOQAqUkB0pITSEsOLScFfPh8gk8En4BPBAl73hJUahqaqa5voa6xhZREP0kBH00tQRpbgjS3BslMSyItOUBtQwsKJAV8JAZ8oFDd0AwISQEfSQk+BKG+qYXkBD8ZfRLol5JI3yR/KLNPSAj4SPD5CPiFBL+QkhDK2Tcp9DXBb6MNmfgnImtUtai7rxM3d8B3VU1DC4frmti85yhVdU30TfSTmhz6JTgprx9fO3Mok/JDBaNvYoD1u4+wdX8N9c2tlFXWsq78CCu2VdLceuKiHPAJWWlJBPxCS6tS09BCn0Q/p6UmHf9luXnv0eO/hMsO1NLQHCQ5wRf6xRjwk5TgQxUO1DTS2BKkvqmV+uZWgqqgUNPYAoBPICc9mez0ZFqDQVRh9MA0+vVJ+NgvbgFnWRDhY7/cg6pU1TZRWdNA+eFjNDS3ElRITwnQv08iuf1S8InQ3Bp0Hkpza5DRA9PISEmkvqmFmoYWahpb2HOkntrGZmoaWmhsDoYKl35YuELFK1xaUoD0lAT6Jvmpb26loTnofAY+/D6h+P3D1Da0kJYcQIRQsW0JApCenBBqa26loSUICimJoddpcvp0RmpSgIyUBE5LDX3Pef1TyO2XQlZaMv37JjCgbyID+iTSr09iqKAZ04v1+j2TcKrapcNWrUFlV1Udaz84ggCD+iWTkZJAenICfZMC9EtJwOdz93DYsaYWqutbyExNJBBjf1FrWFHxu/Q51Te1cqS+ibrGVlqCQZpblOZgkJZWpaU1SFNrqEDXNob2iqobQnunR481c6C2kT1H6tlzuP544QonAsMz+zIxrx8T8zKYmJfB2EEZpCT6XflejIkk2zNxQVfPf/h9woisVEZkpUY4Ucf1SQzQJzE2f5wigt/lU08piX5SElO69RqqysHaJg7VhR6Hj4W+VtY0UrL3KG+UHuSptXuA0N7olCH9OLcwi3NHZTEhN8P1PyiM8ZLtmRgTQfuPNrCh/AjvfnCEN0oPsGlPNQD9+yRwTmEWXztrKNMKBnic0piP2J6JMVFoYEYyAzMGcuG4gcDpVNU28kbpQV577yCvbK1gyfq9TC8YwHXnjWDmqCy7GtDEDdszMaaH1De1smj1Bzz02g72Hm1g6tD+3Pm5cUzIy/A6munFIrVnYsXEmB7W1BLkyXfL+fWL26iqa+JLRfn84KLRZKYmeR3N9EKRKiaxddmPMXEgMeBj/vQhvPKDmXxzxjCeWFPORfe8RvEuT+7LNSYirJgY45H05AR+dNlYnrvxHNJTEpj/0Cqedq4GMybWWDExxmOjctJ4+jszKBo6gJv+to7H3tzldSRjOs2KiTFRIKNPAo9+fRoXjM3hjiWb+ePKnV5HMqZTrJgYEyWSE/z84Suf4qJxOdy5tITHi3efeiNjooQVE2OiSMDv4775Uzh7ZCa3P7WJbftrvI5kTIdYMTEmyiQF/Pxm3mTSkgPcvHhdlwapNKanWTExJgplpibx889PYPPean6/otTrOMackhUTY6LUheMGMmfSYH7/ahk7D9Z5HceYk7JiYkwU+9FlY0hK8PGjpzcSL6NVmPhkxcSYKJadlswtF41mZWkVL22p9DqOMSdkxcSYKDd/+hCGZfblv5ZtI9h2akpjooQVE2OiXILfx/cuGMW2ihqWbtjrdRxj2mXFxJgYcNmEQYwZlM69L223vRMTlayYGBMDfD7hupkj2HGwjle32bkTE31cLSYiMltEtolIqYjceoI+V4lIiYhsFpG/hrW3isg657HEzZzGxIKLxw9kcEYyD79u43aZ6OPatL0i4gfuBy4AyoHVIrJEVUvC+hQCtwEzVPWwiGSHvUS9qk52K58xsSbB7+OaGQX87LmtbNpzlPG5NkOjiR5u7plMB0pVdYeqNgGLgLlt+nwbuF9VDwOoqu2/G3MSX5o2hJQEP395+32voxjzMW4Wk1wgfNjTcqct3ChglIisFJFVIjI7bF2yiBQ77Ze39wYissDpU3zgwIHIpjcmCmWkJHDpxEEsWbeXusYWr+MYc5ybxUTaaWt7GUoAKARmAvOBh0Wkn7NuiDMv8ZeB34jIiE+8mOqDqlqkqkVZWVmRS25MFJs3LZ+6plae3bjP6yjGHOdmMSkH8sOW84C2F8mXA/9Q1WZV3QlsI1RcUNW9ztcdwApgiotZjYkZU4f2Z3hWX/622uY7MdHDzWKyGigUkWEikgjMA9pelfU0cB6AiGQSOuy1Q0T6i0hSWPsMoARjDCLCvGn5rHn/MKWVNt+JiQ6uFRNVbQGuB5YBW4DFqrpZRBaKyByn2zKgSkRKgFeBW1S1ChgDFIvIeqf9F+FXgRnT233+U3kEfGJ7JyZqSLyMRFpUVKTFxcVexzCmx1z7pzW8s+sQq277LIkBu//YdI2IrHHOT3eL/Qs0JkZ9aXo+h+qaeHlLhddRjLFiYkysOrcwi0EZySwutkNdxntWTIyJUX6fcMmEQawsq+JYk91zYrxlxcSYGHb+6dk0tQRZWVrldRTTy1kxMSaGTSsYQGpSgFe22nkT4y0rJsbEsMSAj3NHZfLK1kqbI954yoqJMTHuvNHZVFQ3snlvtddRTC9mxcSYGDdzdGjmhn++Z4OdGu9YMTEmxmWlJTE6J41VO+wkvPGOFRNjYl1ZGXe9+Hv+8J3zUZ8P0tPhO9+BsjKvk5lexIqJMbHs+edh4kSKXnqS1KZjiCrU1MDDD8PEiaH1xvQAKybGxKqyMrjySjh2DF9Lm5sWm5vh2LHQettDMT3AiokxserXvw4VjZNpboZ77umZPKZXs2JiTKz68587Vkz+9KeeyWN6NSsmxsSq2trI9jOmG6yYGBOrUlMj28+YbrBiYkys+upXISHh5H0SEuBrX+uZPKZXs2JiTKz6/vc7Vky+972eyWN6NSsmxsSqESPgiSegT59PFJVgICHU/sQToX7GuMyKiTGx7OKLYcMGWLAA0tNR8VGT2IeNl1wVar/4Yq8Tml4i4HUAY0w3jRgBv/sd/O53CDD/t6+TkuDncdsjMT3I1T0TEZktIttEpFREbj1Bn6tEpERENovIX8ParxaR7c7jajdzGhNPZo3Jofj9w1TWNHgdxfQirhUTEfED9wMXA2OB+SIytk2fQuA2YIaqjgNuctoHAHcAZwDTgTtEpL9bWY2JJ7PHD0QVlpfY7Ium57i5ZzIdKFXVHaraBCwC5rbp823gflU9DKCqlU77RcByVT3krFsOzHYxqzFxY3ROGsMy+/LCpv1eRzG9iJvFJBfYHbZc7rSFGwWMEpGVIrJKRGZ3YltEZIGIFItI8YEDNjGQMQAiwkXjBvJWWRVHjjV5Hcf0Em4WE2mnre0k1QGgEJgJzAceFpF+HdwWVX1QVYtUtSgrK6ubcY2JH7PHD6QlqLy6rfLUnY2JADeLSTmQH7acB+xtp88/VLVZVXcC2wgVl45sa4w5gYm5GWSkJPBWmc2+aHqGm8VkNVAoIsNEJBGYByxp0+dp4DwAEckkdNhrB7AMuFBE+jsn3i902owxHeDzCdMKBvD2zkNeRzG9hGvFRFVbgOsJFYEtwGJV3SwiC0VkjtNtGVAlIiXAq8AtqlqlqoeAuwgVpNXAQqfNGNNBZw4fwPtVx9h/1C4RNu5z9aZFVX0OeK5N24/Dnitws/Nou+0jwCNu5jMmnp05/DQA3t5ZxdzJn7h+xZiIsuFUjIlTYwalk5YcYNUO26k37rNiYkyc8h8/b2In4Y37rJgYE8fOHD6AHQfqqKi28ybGXVZMjIljZ48M3X/1+vaDHicx8c6KiTFx7PSBaWSmJvLGdhshwrjLiokxccznE84emckbpQcJBj8xiIQxEWPFxJg4d3ZhFgdrm9i6v8brKCaOWTExJs6dU5gJwOt2qMu4yIqJMXEuJz2ZUTmpvFFqJ+GNe6yYGNMLTB82gHffP0xLa9DrKCZOWTExpheYVjCAuqZWtuyz8ybGHVZMjOkFpg8bAMA7u2xoFeMOKybG9AKDMlLI65/CahuS3rjEiokxvcT0ggGs3nWI0GDdxkSWFRNjeolpwwZQVdfEjoN1XkcxcciKiTG9xBnOeZM37RJh4wIrJsb0EsMy+1JwWh9e2lLpdRQTh6yYGNNLiAizxuTwVlkVtY0tXscxccaKiTG9yKyxOTS1Bnn9PRtaxUSWFRNjepGiof3JSElg+ZYKr6OYOGPFxJheJOD3cf7p2bxUUsHeI/VexzFxxNViIiKzRWSbiJSKyK3trL9GRA6IyDrn8a2wda1h7UvczGlMb/IvnxmOKnztf96mqrbR6zgmTrhWTETED9wPXAyMBeaLyNh2uv5NVSc7j4fD2uvD2ue4ldOY3ub0gen8zzXTKD9cz51LS7yOY+KEm3sm04FSVd2hqk3AImCui+9njOmg6cMGMG9aPi9u3k91Q7PXcUwccLOY5AK7w5bLnba2viAiG0TkCRHJD2tPFpFiEVklIpe7mNOYXunyKbk0tgR5YdN+r6OYOOBmMZF22toOCrQUKFDVicBLwGNh64aoahHwZeA3IjLiE28gssApOMUHDtiljsZ0xuT8fgw9rQ//WLfH6ygmDrhZTMqB8D2NPGBveAdVrVLVD88APgRMDVu31/m6A1gBTGn7Bqr6oKoWqWpRVlZWZNMbE+dEhLmTc3mzrIqK6gav45gY52YxWQ0UisgwEUkE5gEfuypLRAaFLc4Btjjt/UUkyXmeCcwA7EyhMRF2+eTBqMKSdXtP3dmYk+hQMRGREWG/3GeKyA0i0u9k26hqC3A9sIxQkVisqptFZKGIfHh11g0isllE1gM3ANc47WOAYqf9VeAXqmrFxJgIG56VyqS8DJ62Q12mm6QjcxuIyDqgCCggVByWAKNV9RJX03VCUVGRFhcXex3DmJjz6Mqd/GRpCcu/dy6FOWlexzE9TETWOOenu6Wjh7mCzp7GFcBvVPV7wKBTbGOMiQGXTRyM3ye2d2K6paPFpFlE5gNXA884bQnuRDLG9KSstCRmjMzkH+v22iyMpss6Wky+DpwF3K2qO0VkGPBn92IZY3rSpRMGUn64nq37a7yOYmJUoCOdnJPfN0DoSisgTVV/4WYwY0zPOW90NgCvbK1kzKB0j9OYWNTRq7lWiEi6iAwA1gOPish/uxvNGNNTstOTGZ+bzqtbbRZG0zUdPcyVoarVwOeBR1V1KjDLvVjGmJ52/uhs3v3gMIfrmryOYmJQR4tJwLnB8Co+OgFvjIkj552eTVDhnzYLo+mCjhaThYTuLylT1dUiMhzY7l4sY0xPm5TXj+y0JB74Zxl1Nke86aQOFRNVfVxVJ6rqdc7yDlX9grvRjDE9yecTfnnlRN6rqOH7i9cTDNplwqbjOnoCPk9EnhKRShGpEJEnRSTP7XDGmJ41c3Q2t108hhc27+fFEhua3nRcRw9zPUpoCJXBhOYkWeq0GWPizDfOHkZOehKLi8u9jmJiSEeLSZaqPqqqLc7jj4CN+W5MHPL7hM9/Ko8V2yqptKHpTQd1tJgcFJGviojfeXwVqHIzmDHGO1+cmkdQ4e9rbbwu0zEdLSbfIHRZ8H5gH3AloSFWjDFxaHhWKlOH9mdx8W4br8t0SEev5vpAVeeoapaqZqvq5YRuYDTGxKkvTx/CjgN1rCy1gxDm1Loz0+LNEUthjIk6l00aRGZqIn98c6fXUUwM6E4xkYilMMZEnaSAny9PH8LLWyt5v6rO6zgmynWnmNiBVGPi3FfOHIpfhBsXraO0stbrOCaKnbSYiEiNiFS386ghdM+JMSaO5aQnc8+XJrPzYB2X3vc6ZQesoJj2nbSYqGqaqqa380hT1Q7NhWKMiW2fmzSYZ284m+bWIE+9a5cKm/Z15zCXMaaXyOvfh0+PyOTZjfvsUmHTLleLiYjMFpFtIlIqIre2s/4aETkgIuucx7fC1l0tItudx9Vu5jTGnNqlEwex82Adm/dWex3FRCHXiomI+IH7gYuBscB8ERnbTte/qepk5/Gws+0A4A7gDGA6cIczXbAxxiOzxw3E7xOe3bjP6ygmzDs7D7Gh/IjXMVzdM5kOlDrD1TcBi4C5Hdz2ImC5qh5S1cPAcmC2SzmNMR3Qv28iM0Zm8syGvXaoK0pUNzRz46K13PL4Bs+nDHCzmOQCu8OWy522tr4gIhtE5AkRye/ktsaYHnTZhEHsPlTPxj1HvY5igJ8+U0JFdQO/+MIEfD5vb/1zs5i09521LZ1LgQJVnQi8BDzWiW0RkQUiUiwixQcO2FSjxrjtwnE5BHzCMxvsUJfXXtlaweLicq79zAimDPH+LICbxaQcyA9bzgP2hndQ1SpVbXQWHwKmdnRbZ/sHVbVIVYuysmxEfGPc1q9PIucUZvLsBruqy0tHjjVx65MbGZ2Txo2zCr2OA7hbTFYDhSIyTEQSgXmEJtg6TkQGhS3OAbY4z5cBF4pIf+fE+4VOmzHGY5dOHMyeI/Ws3e39Sd/e6idLS6iqa+LXV00iKeD3Og7gYjFR1RbgekJFYAuwWFU3i8hCEZnjdLtBRDaLyHrgBuAaZ9tDwF2ECtJqYKHTZozx2AVjc0gM+PjxPzax72i913F6nWWb9/PU2j1897yRjM/N8DrOcRIvu6pFRUVaXFzsdQxjeoWXSiq4cdFaUhIDPHfD2WSnJ3sdqVc4VNfEhff8k+y0ZJ7+7gwSA93fHxCRNapa1N3XsTvgjTGdNmtsDouvPYuqukYee2uX13F6jTuXbOZofTO/vmpSRApJJEVXGmNMzBg3OIMLx+bwl7c/4FhTi9dx4t6rWytZsn4v3z1vJGMGpXsd5xOsmBhjuuxb5wznyLFmnrQBIF1V19jCj57exMjsVK6bOcLrOO2yYmKM6bKiof2ZlJfBfS9vp7Syxus4ceue5e+x50g9P//8hKi5eqstKybGmC4TEX555SRU4YsPvMXW/TYIZKRtLD/KIyt38uUzhjCtYIDXcU7IiokxpltGD0zjiWvPQoF7X9rudZy40tIa5Na/b+C01CT+ffbpXsc5KSsmxphuK8jsy5WfyuOlLRVU1TaeegPTIY+u3MXmvdX8ZM44MlISvI5zUlZMjDERcdW0fJpblafW2sn4SNh96Bj/vfw9Zo3J5uLxA72Oc0pWTIwxETEqJ43J+f342+rdNm5XBNyxZDM+gYVzxyPi7YjAHWHFxBgTMfOm5bO9spa3yqq8jhLTXt5SwStbK7lp1igG90vxOk6HWDExxkTM5VNyyUlP4jcvb7e9ky5qaG7lJ0tLGJmdyjUzCryO02FWTIwxEZOc4Oe6z4zgnZ2HeGuH7Z10xUOv7eCDQ8f4yZxxJPhj51d07CQ1xsSEedOHkJ2WxH0v22XCnVV++Bj3ryjl0gmDmDEy0+s4nWLFxBgTUckJfr59znBW7TjExnKb3rcz7n52C4Lww0vHeB2l06yYGGMi7kvT80lNCvDQ6zu8jhIzXt9+gOc37ef680eSGyMn3cNZMTHGRFx6cgLzpuXz7MZ97DliE2idSlNLkDuXbKbgtD5865xhXsfpEismxhhXfP3sYfgEbn9qI61Bu7LrZP745k7KDtRxx+fGRe1AjqdixcQY44rcfincOWccK7Yd4K5nSuxS4RM4VNfEb18u5bzRWZx3erbXcbos4HUAY0z8+soZQymtrOXRlbvYur+an10xgeFZqV7Hiir3vvQex5pbuT0GT7qHsz0TY4yr/uPSsdx9xXhK9lbzud++wYub93sdKWqUHajlL29/wPzp+YzMTvM6TrdYMTHGuMrnE75yxlCWfe9cRmSnsuBPa1hmBQWAXzy/leQEPzfNGuV1lG6zYmKM6RGDMlJY/C9nMSKrL/e+ZMOtrNpRxfKSCq6bOYLM1CSv43Sbq8VERGaLyDYRKRWRW0/S70oRUREpcpYLRKReRNY5jwfczGmM6RnJCX4WnDuckn3VvNmLB4MMBpWfPltCbr8Uvnl2bF4K3JZrxURE/MD9wMXAWGC+iIxtp18acAPwdptVZao62Xlc61ZOY0zPmjs5l8zUJP7va733hsZ/rN/Dpj3V3HLRaJITYvNS4Lbc3DOZDpSq6g5VbQIWAXPb6XcX8EugwcUsxpgokZzg55pPD+W19w7wXkWN13F6XENzK796YRsT8zKYM2mw13Eixs1ikgvsDlsud9qOE5EpQL6qPtPO9sNEZK2I/FNEzmnvDURkgYgUi0jxgQMHIhbcGOOu+dOHkBjw8ae33vc6So/7nzd2svdoA7dfMgafL/onveooN4tJe5/S8TNuIuID7gG+306/fcAQVZ0C3Az8VUTSP/Fiqg+qapGqFmVlZUUotjHGbaelJvG5iYN58t1yqhuavY7TY6pqG/nDijIuGJvDGcNP8zpORLlZTMqB/LDlPGBv2HIaMB5YISK7gDOBJSJSpKqNqloFoKprgDIg9q+dM8Ycd82nCzjW1Mrf15R7HaXH/PaVUuqbW/n32ad7HSXi3Cwmq4FCERkmIonAPGDJhytV9aiqZqpqgaoWAKuAOapaLCJZzgl8RGQ4UAj03rN1xsShCXkZTM7vx/++0zvmjN95sI4/r3qfedPyGZkdf6MAuFZMVLUFuB5YBmwBFqvqZhFZKCJzTrH5ucAGEVkPPAFcq6qH3MpqjPHGF6bmsa2ihpJ91V5Hcd2vlm0lMeDjxlmFXkdxhatjc6nqc8Bzbdp+fIK+M8OePwk86WY2Y4z3LpswiIVLN/P02j2MG5zhdRzXrHn/MM9t3M9NswrJTkv2Oo4r7A54Y4xn+vdNZObobP6xbm/cDlMfDCp3P1tCVloS3z5nuNdxXGPFxBjjqSum5FJZ08jr2+Pz8v4/v/0+735whH+7aDR9k+J3oHYrJsYYT312TDZZaUk8/PpOr6NEXPnhY/zn81s5pzCTK6fmeR3HVVZMjDGeSgr4+fY5w3ij9CDrdh/xOk5E/WFFGa2q/OyKCYjEzw2K7bFiYozx3FfOGEpGSgL3v1rqdZSIaQ0qL2zaz2fH5JA/oI/XcVxnxcQY47m+SQG+efYwlpdU8Np78XHu5O2dVVTVNXHphEFeR+kRVkyMMVFhwbnDGZmdyr89sYGj9bE/xMrzG/eTnOBj5ujeMdSTFRNjTFRITvDz6y9O4kBtIz/8+8aYviu+Nag8v2k/55+eTZ/E+L2CK5wVE2NM1JiU349bLhrNsxv3cc/y97yO02XPbNjLwdpGLp0QP0PMn4oVE2NMVPmXc4fzpaJ87nullJWlB72O02k1Dc3c/ewWJuRmMHv8QK/j9BgrJsaYqCIiLLx8HBkpCTz5buyNKPybl7ZzoLaRn14+Hn8czVdyKlZMjDFRJyng54KxOSwvqaCxpdXrOB32QdUxHntzF/Om5TMpv5/XcXqUFRNjTFS6ZMJAahpaeLO0yusoHXbPS+8R8As3zep90y9ZMTHGRKUZIzNJSwrw3MZ9XkfpkK37q3l63R6u/nQBOenxOTLwyfSOa9aMMTEnKeBn1tgclm7YS78+CXzrnOFR/Uv6ty+XkpoY4LrPjPA6iidsz8QYE7W+f+EoZo7K5tGVu/jWY8UEo3SY+p0H63hu0z6+etZQ+vVJ9DqOJ6yYGGOiVl7/Pjzwtan85xcmsnHPUZZu2Ot1pHY9+FoZCX4fX59R4HUUz1gxMcZEvcun5DJmUDr/9eK2qLu6q/zwMZ5cs4erivLidhbFjrBiYoyJen6fcNvFp7P7UD33vbzd6zgfs3BpCX6fcN3MkV5H8ZQVE2NMTDh3VBZXFeXx+xVlvFnmzp3xxbsOcel9r7P2g8Md6v/ylgpeLKnghs8WktsvxZVMscKKiTEmZtw5ZxzDMvvy/cXraWiO/OGux956n83O3pt0AAAOy0lEQVR7q/nyQ29z89/WMfNXr/LMCc7T7D/awA+f2sjI7FS+efawiGeJNa4WExGZLSLbRKRURG49Sb8rRURFpCis7TZnu20icpGbOY0xsaFPYoCfXj6efUcbeLx4d7dfL/zqsPqmVl7eUsElEwYyemAay0sqaG5VbntyI+WHj31su2NNLXzr/62mtqGF386fQmLA/i537RMQET9wP3AxMBaYLyJj2+mXBtwAvB3WNhaYB4wDZgO/d17PGNPLnTX8NKYO7c8D/9xBc2uwy6/z93fLmXb3S2wsPwrAK1srOdbUylfPHMrT353B+jsuZNGCM1Hg5sXrj5/4DwaVm/+2npK91fz2y1MYMyg9Et9WzHOznE4HSlV1h6o2AYuAue30uwv4JdAQ1jYXWKSqjaq6Eyh1Xs8Y08uJCNefP5I9R+p5au2eLr1G2YFabn9qE1V1Tfzg8VCheHbjXjJTkzhj2GkA+HxC/oA+/PTy8byz8xBXP/IOW/ZV89Nnt/DC5v388JIxnH96TiS/tZjmZjHJBcL3Q8udtuNEZAqQr6rPdHZbY0zvNXNUFmMGpfPIGzs7PYlWa1C5cdFakhN8/PzzE9hWUcMl977Oss2hQ1xtR/q9fEouv/nSZNa8f5iL732dR1buZP70IXaepA03h1Npb+zl4z91EfEB9wDXdHbbsNdYACwAGDJkSJdCGmNij4jwtTOH8sOnNvLuB0eYOrR/h7d9ftM+Nu2p5t55k5k7OZeyylreLKviGzMKuP68wna3uXxKLoU5qbxXUUN+/z58akh/RHrP8PId4WYxKQfyw5bzgPDLItKA8cAK54cyEFgiInM6sC0Aqvog8CBAUVFRdI6zYIxxxdzJg/nZc1v4y6r3O1xMgkHld6+UMjI7lc9NDM2C+KPLPnEqt13jBmcwbnBGl/PGOzcPc60GCkVkmIgkEjqhvuTDlap6VFUzVbVAVQuAVcAcVS12+s0TkSQRGQYUAu+4mNUYE2P6JgW4Ykouz2zcx6G6pg5t8/LWSrbur+E7M0fg60UTV/UE14qJqrYA1wPLgC3AYlXdLCILnb2Pk227GVgMlAAvAN9V1egaQ8EY47mvnTWU5tYgD/yzrEP9n163h+y0JOZM6j1zs/cUV4egV9XngOfatP34BH1ntlm+G7jbtXDGmJg3KieNKz+Vxx9X7uKrZwxlyGl9Ttp//9EGRmSlEvDbfSGRZp+oMSam/eCi0fh9wk+fLTnllV0V1Q3kpCf1ULLexYqJMSam5aQnc8NnC3mxpIK7ntlywoKiqlTWNEb1BFuxzGZaNMbEvGs/M5yK6gYeWbmTQRnJfPvc4Z/oc7S+maaWIFlptmfiBtszMcbEPBHhjs+N5bzRWdz3ynaOHmv+RJ/KmkYA2zNxiRUTY0xcEBFuueh0ahpa+J83dnxifUV1aMQmKybusGJijIkbYwenc8mEgTyycheH29x7UlEd2jPJtsNcrrBiYoyJKzfNGkVdUwsPvv7xvZPKmtCeSbZdzeUKKybGmLgyKieNz00czB9X7uJgbePx9srqRtKSAvRJtOuO3GDFxBgTd26cVUhjSyt/WPHRnfGVNQ22V+IiKybGmLgzIiuVK6fm8cjKnfz93XIgdM7ETr67x/b3jDFx6SdzxrP3SAPff3w9qUkBKqobKOrEUPWmc2zPxBgTl1IS/Tz0f4oozE7lv5e/Z3e/u8yKiTEmbqUk+vnGjGFs3V9jd7+7zIqJMSauzZ2cS3py6Ii+7Zm4x4qJMSaupST6+dK00MStdsOie+wEvDEm7i04dwRBhUn5/byOEresmBhj4l5WWhL/0cG53k3X2GEuY4wx3WbFxBhjTLdZMTHGGNNtVkyMMcZ0m6vFRERmi8g2ESkVkVvbWX+tiGwUkXUi8oaIjHXaC0Sk3mlfJyIPuJnTGGNM97h2NZeI+IH7gQuAcmC1iCxR1ZKwbn9V1Qec/nOA/wZmO+vKVHWyW/mMMcZEjpt7JtOBUlXdoapNwCJgbngHVa0OW+wLqIt5jDHGuMTNYpIL7A5bLnfaPkZEvisiZcAvgRvCVg0TkbUi8k8ROcfFnMYYY7rJzZsWpZ22T+x5qOr9wP0i8mXgR8DVwD5giKpWichU4GkRGddmTwYRWQAscBZrRWRbJ/JlAEc7sa5tW/hye8/btiUABzuRL9IZT5YtvC2zkzlPlrErOU+WrasZT5UzEhnbyxvJz9J+3p0Tjz/vruTsSMahnch8YqrqygM4C1gWtnwbcNtJ+vuAoydYtwIoinC+Bzuzrm1b+HJ7z9u2AcVeZjxZtjZfO5XzZBkj/Vl2NWNP/Lzd/izt520/b68+y44+3DzMtRooFJFhIpIIzAOWhHcQkcKwxUuB7U57lnMCHxEZDhQCOyKcb2kn17VtW3qK5yda3xmRzBi+fKq8nXGq7SL5WXY146m2jUTG8Of28z55m/28T73didZ58Vl2iDjVyp0XF7kE+A3gBx5R1btFZCGh6rhERO4FZgHNwGHgelXdLCJfABYCLUArcIequv5huElEilW1yOscpxILOWMhI8RGTssYObGQ082Mrg70qKrPAc+1aftx2PMbT7Ddk8CTbmbzwINeB+igWMgZCxkhNnJaxsiJhZyuZXR1z8QYY0zvYMOpGGOM6TYrJl0gIo+ISKWIbOrCtlOdIWRKReQ+EZGwdf/qDD+zWUR+GY05ReROEdkTNtTNJdGWMWz9D0RERSQz2jKKyF0issH5DF8UkcHdyehizl+JyFYn61Mi0q3ZpVzK+EXn/0xQRLp8PqA72U7weleLyHbncfWpvo8oy3i3iOwWkdoOv5hbl4nF8wM4F/gUsKkL275D6LJpAZ4HLnbazwNeApKc5ewozXkn8INo/iyddfnAMuB9IDPaMgLpYX1uAB6Ixs8SuBAIOM//E/jPKMw4BhhNN28h6Go2530L2rQNIHQF6gCgv/O8/6n+3UZRxjOBQUBtR9/D9ky6QFVfAw6Ft4nICBF5QUTWiMjrInJ62+1EZBChXyJvaegn9v+Ay53V1wG/UNVG5z0qozRnRLmY8R7g34jAED1uZFQXhhJyKeeLqtridF0F5EVhxi2q2pkbliOa7QQuApar6iFVPQwsB2Z39/9WT2R03meVqu7raC6ww1yR9CDwr6o6FfgB8Pt2+uQSGlbmQ+FDzIwCzhGRtyU0hMy0KM0JcL1z2OMREekfbRklNGjoHlVd70K2iGR0ct4tIruBrwA/xh2R+Hl/6BuE/pKOtEhm9CJbe040nJQb30ekM3aJzQEfASKSCnwaeDzs8GdSe13bafvwL9IAoV3NM4FpwGIRGe789RJNOf8A3OUs3wX8mtAvmajIKCJ9gNsJHZ5xRYQ+R1T1duB2EbkNuB64IxpzOq91O6H7vv4SrRkj7WTZROTrwIe3NowEnhORJmCnql5xkrwR/T5cytglVkwiwwcc0TZD5kvoLv41zuISQr+Iww8T5AF7neflwN+d4vGOiAQJjaNzIJpyqmpF2HYPAc9EMF8kMo4AhgHrnf9cecC7IjJdVfdHSca2/go8S4SLSaRyOidmLwM+G8k/biKZ0SXtZgNQ1UeBRwFEZAVwjaruCutSDswMW84jdN6inMh+H25k7JrOnMSxx8dOXhUQdhIMeBP4ovNcgEkn2G41ob2PD0++XeK0XwssdJ6PIrT7KVGYc1BYn+8Bi6ItY5s+u+jmCXiXPsfCsD7/CjwRpf8uZwMlQFYk8rn58yYCY/h1JRsnPrm9k9DRhv7O8wEd/XfrdcawPh0+AR+Rfxy97QH8L6GRjZsJVfdvEvpr+AVgvfOf78cn2LYI2ASUAb/joxtHE4E/O+veBc6P0px/AjYCGwj9xTgo2jK26bOL7l/N5cbn+KTTvoHQuEm5UfrzLiX0h80659Gtq85cyniF81qNQAVhA8z2RDba+UXttH/D+fxKga935t9tFGT8pfP6QefrnafKZnfAG2OM6Ta7mssYY0y3WTExxhjTbVZMjDHGdJsVE2OMMd1mxcQYY0y3WTExca1To55G5v0eFpGxEXqtVgmNKrxJRJbKKUbsFZF+IvKdSLy3MZ1llwabuCYitaqaGsHXC+hHAx+6Kjy7iDwGvKeqd5+kfwHwjKqO74l8xoSzPRPT64hIlog8KSKrnccMp326iLwpImudr6Od9mtE5HERWQq8KCIzRWSFiDwhobk+/iJyfG6NFeLMqSEitc5gjutFZJWI5DjtI5zl1SKysIN7T2/x0UCWqSLysoi8K6F5MeY6fX4BjHD2Zn7l9L3FeZ8NIvKTCH6MxnyMFRPTG90L3KOq04AvAA877VuBc1V1CqFRfH8Wts1ZwNWqer6zPAW4CRgLDAdmtPM+fYFVqjoJeA34dtj73+u8/ynHZXLGqfosoREHABqAK1T1U4Tmwfm1U8xuBcpUdbKq3iIiFwKFwHRgMjBVRM491fsZ0xU20KPpjWYBY8NGWU0XkTQgA3hMRAoJjZ6aELbNclUNn0fiHVUtBxCRdYTGS3qjzfs08dFAmGuAC5znZ/HRHBZ/Bf7rBDlTwl57DaH5JiA0/tLPnMIQJLTHktPO9hc6j7XOciqh4vLaCd7PmC6zYmJ6Ix9wlqrWhzeKyG+BV1X1Cuf8w4qw1XVtXqMx7Hkr7f9fataPTkqeqM/J1KvqZBHJIFSUvgvcR2j+kyxgqqo2i8guILmd7QX4uar+306+rzGdZoe5TG/0IqH5QwAQkQ+H784A9jjPr3Hx/VcROrwGMO9UnVX1KKGpfX8gIgmEclY6heQ8YKjTtQZIC9t0GfANZ84LRCRXRLIj9D0Y8zFWTEy86yMi5WGPmwn9Yi5yTkqXEBr+H0Ijpf5cRFYCfhcz3QTcLCLvEJpn++ipNlDVtYRGhZ1HaIKqIhEpJrSXstXpUwWsdC4l/pWqvkjoMNpbIrIReIKPFxtjIsYuDTamhzmzQdarqorIPGC+qs491XbGRDM7Z2JMz5sK/M65AusIEZz22Biv2J6JMcaYbrNzJsYYY7rNiokxxphus2JijDGm26yYGGOM6TYrJsYYY7rNiokxxphu+/+hmKqaygm01AAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "learn.recorder.plot(suggestion=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can access the corresponding learning rate like this:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.005248074602497722" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "min_grad_lr = learn.recorder.min_grad_lr\n", "min_grad_lr" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracytime
10.1094750.0816070.97055900:02
20.0703030.0509770.98282600:02
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn = simple_learner()\n", "simple_learner().fit(2, min_grad_lr)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

class LRFinder[source][test]

\n", "\n", "> LRFinder(**`learn`**:[`Learner`](/basic_train.html#Learner), **`start_lr`**:`float`=***`1e-07`***, **`end_lr`**:`float`=***`10`***, **`num_it`**:`int`=***`100`***, **`stop_div`**:`bool`=***`True`***) :: [`LearnerCallback`](/basic_train.html#LearnerCallback)\n", "\n", "
×

No tests found for LRFinder. To contribute a test please refer to this guide and this discussion.

\n", "\n", "Causes `learn` to go on a mock training from `start_lr` to `end_lr` for `num_it` iterations. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(LRFinder)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Callback methods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You don't call these yourself - they're called by fastai's [`Callback`](/callback.html#Callback) system automatically to enable the class's functionality." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_train_begin[source][test]

\n", "\n", "> on_train_begin(**`pbar`**, **\\*\\*`kwargs`**:`Any`)\n", "\n", "
×

No tests found for on_train_begin. To contribute a test please refer to this guide and this discussion.

\n", "\n", "Initialize optimizer and learner hyperparameters. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(LRFinder.on_train_begin)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_batch_end[source][test]

\n", "\n", "> on_batch_end(**`iteration`**:`int`, **`smooth_loss`**:`TensorOrNumber`, **\\*\\*`kwargs`**:`Any`)\n", "\n", "
×

No tests found for on_batch_end. To contribute a test please refer to this guide and this discussion.

\n", "\n", "Determine if loss has runaway and we should stop. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(LRFinder.on_batch_end)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_epoch_end[source][test]

\n", "\n", "> on_epoch_end(**\\*\\*`kwargs`**:`Any`)\n", "\n", "
×

No tests found for on_epoch_end. To contribute a test please refer to this guide and this discussion.

\n", "\n", "Called at the end of an epoch. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(LRFinder.on_epoch_end)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_train_end[source][test]

\n", "\n", "> on_train_end(**\\*\\*`kwargs`**:`Any`)\n", "\n", "
×

No tests found for on_train_end. To contribute a test please refer to this guide and this discussion.

\n", "\n", "Cleanup learn model weights disturbed during LRFinder exploration. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(LRFinder.on_train_end)" ] } ], "metadata": { "jekyll": { "keywords": "fastai", "summary": "Implementation of the LR Range test from Leslie Smith", "title": "callbacks.lr_finder" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 2 }