{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The 1cycle policy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide_input": true
   },
   "outputs": [],
   "source": [
    "from fastai.gen_doc.nbdoc import *\n",
    "from fastai import *\n",
    "from fastai.vision import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is 1cycle?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Callback allows us to easily train a network using Leslie Smith's 1cycle policy. To learn more about the 1cycle technique for training neural networks check out [Leslie Smith's paper](https://arxiv.org/pdf/1803.09820.pdf) and for a more graphical and intuitive explanation check out [Sylvain Gugger's post](https://sgugger.github.io/the-1cycle-policy.html).\n",
    "\n",
    "To use our 1cycle policy we will need an [optimum learning rate](https://sgugger.github.io/how-do-you-find-a-good-learning-rate.html). We can find this learning rate by using a learning rate finder which can be called by using [`lr_finder`](/callbacks.lr_finder.html#callbacks.lr_finder). It will do a mock training by going over a large range of learning rates, then plot them against the losses. We will pick a value a bit before the minimum, where the loss still improves. Our graph would look something like this:\n",
    "\n",
    "![onecycle_finder](imgs/onecycle_finder.png)\n",
    "\n",
    "Here anything between `3x10^-2` and `10^-2` is a good idea.\n",
    "\n",
    "Next we will apply the 1cycle policy with the chosen learning rate as the maximum learning rate. The original 1cycle policy has three steps:\n",
    "\n",
    "    1. We progressively increase our learning rate from lr_max/div_factor to lr_max and at the same time we progressively decrease our momentum from mom_max to mom_min.\n",
    "    2. We do the exact opposite: we progressively decrease our learning rate from lr_max to lr_max/div_factor and at the same time we progressively increase our momentum from mom_min to mom_max.\n",
    "    3. We further decrease our learning rate from lr_max/div_factor to lr_max/(div_factor x 100) and we keep momentum steady at mom_max.\n",
    "    \n",
    "This gives the following form:\n",
    "\n",
    "<img src=\"imgs/onecycle_params.png\" alt=\"1cycle parameteres\" width=\"500\">\n",
    "\n",
    "Unpublished work has shown even better results by using only two phases: the same phase 1, followed by a second phase where we do a cosine annealing from lr_max to 0. The momentum goes from mom_min to mom_max by following the symmetric cosine (see graph a bit below)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The one cycle policy allows to train very quickly, a phenomenon termed [_superconvergence_](https://arxiv.org/abs/1708.07120). To see this in practice, we will first train a CNN and see how our results compare when we use the [`OneCycleScheduler`](/callbacks.one_cycle.html#OneCycleScheduler) with [`fit_one_cycle`](/train.html#fit_one_cycle)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = untar_data(URLs.MNIST_SAMPLE)\n",
    "data = ImageDataBunch.from_folder(path)\n",
    "model = simple_cnn((3,16,16,2))\n",
    "learn = Learner(data, model, metrics=[accuracy])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First lets find the optimum learning rate for our comparison by doing an LR range test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1b2f715e3281416499fe0340fa9426ab",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HBox(children=(IntProgress(value=0, max=1), HTML(value='0.00% [0/1 00:00<00:00]'))), HTML(value…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEACAYAAABI5zaHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VPW9//HXZyYJ+56ISMCAgICCRSMgqOB61dsrVVsLevW6olZtr+3tr/b21rba/rTWblZapBaX2mqttYpb9dZdhEpUVGRrWBRQJIDshMzyuX/MBGMgC2QyM+fk/Xw88njMOeebcz7fJHzmy2e+53vM3RERkXCJ5DoAERHJPCV3EZEQUnIXEQkhJXcRkRBSchcRCSEldxGREFJyFxEJISV3EZEQUnIXEQkhJXcRkRAqyNWFi4uLvaysLFeXFxEJpDfeeGO9u5c01S5nyb2srIyKiopcXV5EJJDM7P3mtFNZRkQkhJpM7mY208zWmdmCBo6fb2bvmNm7ZvaamR2R+TBFRGRfNGfkfg9wWiPHVwAT3H0EcBMwIwNxiYhICzRZc3f3l82srJHjr9XZnAuUtjwsERFpiUzX3C8Fns7wOUVEZB9lbLaMmZ1AKrkf20ibqcBUgP79+2fq0iIiUk9GRu5mNhK4C5jk7hsaaufuM9y93N3LS0qanKYpIpKXtlbHWLBmc67DaFSLR+5m1h94BLjA3Ze2PCQRkfyUTDoPv7maW/+2mPXbarju5CF89aRBmFmuQ9tDk8ndzB4AJgLFZrYa+B5QCODu04EbgF7Ar9MdjLt7eWsFLCKSC2+v2sQNs97j7VWbOLJ/d8YM7MXP/76UDzft5IdnHU5hNL9uG2rObJkpTRy/DLgsYxGJiOSZbbviTPntXDq1K+Bn5x7BFz7XFzMYWNyJXz1fycdbq5l23pF0apezm/73kF9vNSIieWhrdYwdNQmuO3kIZx9ZSiRimBnfOPVQbj57BK/8cz3n3jmHj7dU5zrU3ZTcRUSaEIs7AEUFe6bMKaP7c9d/lLNy/Xa+MG02Cz/cku3w9krJXUSkCTWJJACF0b1/cHrCoQfw5yvHAfCl6a/xwuJ1WYutIUruIiJNiCdTyb2okQ9Nhx/UlUevHs+Akk5ceu887p69AnfPVoh7UHIXEWlCbVmmoIkZMb27tuehK47h5GG9+cHjC/nuYwuIpUf92abkLiLShKbKMnV1LCpg+r8fxRUTBnL/3A+45J55bN4Za+0Q96DkLiLShNrRd2NlmboiEePbpw/j1nNGMmfZBs7+9WyWV21rzRD3jCGrVxMRCaDa5F64l9kyjTn36H7cf9kYPtkRY9K02by4JHsftCq5i4g0IZ5I1dz35y7UsQN78djV4ynt0ZGL75nH9JeWZeWDViV3EZEm1NbcCyL7t4ZMv54d+ctVx3DGiD7c8vRibnl6cSbD26v8uVdWRCRP7a6572NZpq6ORQXcMWUUI/p2Y+Khrb8qrpK7iEgTdtfcW7g4mJlx5YRDMhFSk1SWERFpQmx3zT3/lvZtiJK7iEgTMjVyz6bgRCoikiOxuJK7iEjoqCwjIhJCNSrLiIiET0tuYsqVJiM1s5lmts7MFjRwfKiZzTGzXWb2X5kPUUQkt2KJJBGD6H7exJQLzXkbugc4rZHjG4GvArdlIiARkXwTSyQDNWqHZiR3d3+ZVAJv6Pg6d58HZH9NSxGRLKhJJJu9ImS+CFa0IiI5EEsk93lFyFzLarRmNtXMKsysoqqqKpuXFhHZb/GE7/eiYbmS1eTu7jPcvdzdy0tKWn/hHBGRTKgJY81dRKStiyW8RStC5kKTq0Ka2QPARKDYzFYD3wMKAdx9upkdCFQAXYGkmf0nMNzdt7Ra1CIiWRSLJwN1dyo0I7m7+5Qmjq8FSjMWkYhInoknVZYREQmdmoRToOQuIhIusXiSooCVZZTcRUSaEMo7VEVE2joldxGREIolXMldRCRsUiN31dxFREJFZRkRkRBSWUZEJIRiiSRFBSrLiIiESiyRpCASrHQZrGhFRHJAZRkRkRCqSSQpVFlGRCRcYnrMnohIuCSSjjsqy4iIhEkskQSgQDcxiYiER006uassIyISIrF4KrmrLCMiEiKxhAMhTO5mNtPM1pnZggaOm5ndbmaVZvaOmR2Z+TBFRHIjzDX3e4DTGjl+OjA4/TUV+E3LwxIRyQ+xsNbc3f1lYGMjTSYB93nKXKC7mfXJVIAiIrkU2rJMM/QFVtXZXp3eJyISeLUjd63n3ggzm2pmFWZWUVVVlc1Li4jsl93JvaDtjdzXAP3qbJem9+3B3We4e7m7l5eUlGTg0iIirWt3WaYNrgo5C7gwPWtmLLDZ3T/KwHlFRHIuqGWZgqYamNkDwESg2MxWA98DCgHcfTrwFHAGUAnsAC5urWBFRLKtJqBlmSaTu7tPaeK4A1dnLCIRkTxSe4dq6KZCioi0ZfFk250KKSISWmG+Q1VEpM2qUVlGRCR82vIdqiIioRXUqZBK7iIijfi05h6sdBmsaEVEsqy2LKOau4hIiKgsIyISQrFEEjOIRpTcRURCI5ZwCqMRzJTcRURCI5ZIUhiwUTsouYuINCqWSAZu0TBQchcRaVQskQzcDUyg5C4i0qiauAduGiQouYuINCqeTAZuGiQouYuINCqWSAbu7lRQchcRaVRN3FVzFxEJm1giSVFYyzJmdpqZLTGzSjO7fi/HDzaz58zsHTN70cxKMx+qiEj2hXa2jJlFgWnA6cBwYIqZDa/X7DbgPncfCdwI3JzpQEVEciGe8MA9hQmaN3IfDVS6+3J3rwEeBCbVazMceD79+oW9HBcRCaSasI7cgb7Aqjrbq9P76nobODv9+iygi5n1anl4IiK5laq5hzO5N8d/ARPM7C1gArAGSNRvZGZTzazCzCqqqqoydGkRkdYT2po7qUTdr852aXrfbu7+obuf7e6jgO+k922qfyJ3n+Hu5e5eXlJS0oKwRUSyI57w0K4tMw8YbGYDzKwImAzMqtvAzIrNrPZc3wZmZjZMEZHcqAnrqpDuHgeuAZ4BFgEPuft7ZnajmZ2ZbjYRWGJmS4HewI9aKV4RkawKalmmoDmN3P0p4Kl6+26o8/ph4OHMhiYiknuxhFNYEMKRu4hIWxaLB3PkHryIRUSyKJZUchcRCZ3UM1RVlhERCY1E0kkktSqkiEioxBJJACV3EZEwiScdoE0vPyAiEjqxeGrkHtZVIUVE2iSVZUREQqgmndxVlhERCZFYIlVz1x2qIiIhEldZRkQkfGrLMgWR4KXK4EUsIpIltWWZIpVlRETCQ7NlRERCqHaeu5K7iEiIxNJ3qGrhMBGRENHIXUQkhEJfczez08xsiZlVmtn1ezne38xeMLO3zOwdMzsj86GKiGRXTZiTu5lFgWnA6cBwYIqZDa/X7H9IPTh7FDAZ+HWmAxURybZ4ItyrQo4GKt19ubvXAA8Ck+q1caBr+nU34MPMhSgikhu1ZZkgrgpZ0Iw2fYFVdbZXA2Pqtfk+8KyZXQt0Ak7OSHQiIjkU+pp7M0wB7nH3UuAM4Pdmtse5zWyqmVWYWUVVVVWGLi0i0jpqQl6WWQP0q7Ndmt5X16XAQwDuPgdoDxTXP5G7z3D3cncvLykp2b+IRUSyZPfIPaTLD8wDBpvZADMrIvWB6ax6bT4ATgIws2GkkruG5iISaKFeFdLd48A1wDPAIlKzYt4zsxvN7Mx0s28Al5vZ28ADwEXu7q0VtIhINtSWZQoiwRu5N+cDVdz9KeCpevtuqPN6ITA+s6GJiORWLJGkMGqYBS+5B+//GiIiWRKLJwNZkgEldxGRBqVG7sFMk8GMWkQkC2JJD+SKkKDkLiLSIJVlRERCSGUZEZEQiiVUlhERCR2N3EVEQkjJXUQkhFSWEREJoRqN3EVEwieWSFJUEMw0GcyoRUSyIJ7wQC4aBkruIiIN0geqIiIhVJNIUqiyjIhIuMQSyUA+Yg+U3EVEGhSLayqkiEjoxJNJCjRyFxEJl5p4yMsyZnaamS0xs0ozu34vx39uZvPTX0vNbFPmQxURya4g36Ha5DNUzSwKTANOAVYD88xsVvq5qQC4+3V12l8LjGqFWEVEsirsUyFHA5Xuvtzda4AHgUmNtJ8CPJCJ4EREcsXdiSc91Mm9L7Cqzvbq9L49mNnBwADg+ZaHJiKSO7GEAwS2LJPpt6TJwMPuntjbQTObamYVZlZRVVWV4UuLiGROLJEECOzIvcmaO7AG6FdnuzS9b28mA1c3dCJ3nwHMACgvL/dmxvgZ/1i+gWkvLsPd0+dM7TcDM6P+MhDu4Klr7/V8ZqlvsN3bdV/veaxxtpdzgNXdX2e79lsiltoTsU+v+Zkz1j9HA/HUvw5Y+pypY7Xfm/o5pY5FIqn9u7fNdv8c67aJmhGNpL4vGvn0qyBiFEYjRCNGYTT1uiAaoTBiFBVEdn+1K4jSqShKh6IoHYsKiAZ0vQ5pO9pCcp8HDDazAaSS+mTgvPqNzGwo0AOYk9EI64knnc07Y+kk9WmicyDpqSS+R9pIJ8/6ybA233udHb7Hsc++idT93rrnq3uuum8kdc9T+0ZTVzK9M+lOsv41ar+nzv7db2r1zrO366T2+e6fS+2+utdM+KfXqLvd2joVRenaoZCu7Qvp1qGQ4i5FFHduR0nndvTp3oEBxR0p69WJnp2K9njDE8mGmtrkHtDlB5pM7u4eN7NrgGeAKDDT3d8zsxuBCneflW46GXjQGxoiZ8j4QcU8Nqi4NS8hpN4MEsnUG0My/TrhTjL56et4IvU6nnTiiSSxhBNPJoklktTEnVii9nWS6niCHTUJduxKsG1XnG274mzZGWNLdYxNO2IsWbuVV7euZ0t1/DNxdG1fwPCDunJEaXdGlnbnyIO706dbhxz9VKQtidfW3AP6v8zmjNxx96eAp+rtu6He9vczF5bkmplRkIMPkqpjCT7ctJOVG7azYv0OlldtY8Gazdw9e+XukdTQA7tw8rDenDTsAI4o7U4koP/4JL+1hbKMSNa0L4wysKQzA0s6f2b/rniCJWu3Mnf5Bv6+aB2/eWkZd7xQyYDiTlxy7AC+eGQpHYqiOYpawigW9rKMSD5oVxBlZLo0M/X4Q9i0o4bnFq3jvjkr+e6jC/jZs0u4YOzBXHb8QLq2L8x1uBICNfFUWaZIUyFFsqd7xyLOOaqUR68ez0NXHEN5WU9+9UIlJ972Eo+8ubrB2VEizRVPBrssE8yoRdLMjNEDevLbC8t57OrxlPbowNcfepsvTZ/Doo+25Do8CbDasoxWhRTJsZGl3XnkqnHces5Ilq/fzqRps/nLG6tzHZYEVG1ZRneoiuSBSMQ49+h+/P3rEziqfw++8ee3uemJhcTTozCR5qoduYd6yV+RoOnZqYj7Lh3NRePK+N2rK7jo7nl8sr0m12FJgAR9KmQwoxZphsJohO+feRi3fnEkr6/YyJnTXuW9DzfnOiwJiNqFw3Jxv0cmKLlL6J1b3o8/XTGWWNw55zev8de3VIeXpqksIxIAo/r34PFrj+WI0u5c96e3+f6s90jWX8xHpA6VZUQCoqRLO+6/bAwXjy/jntdW8ou/L811SJLHdIeqSIAURiPc8PnhbKuOc/vzlQzr05XTR/TJdViSh2r0sA6RYDEzfnjW4XyuX3e+8ee3WbxWNzvJnmqnzxZGgpkmgxm1SAu1K4hy5wVH0bldAZffV6FpkrKHoJdlghm1SAb07tqeOy84io837+Kiu19n3ZbqXIckeUTPUBUJsFH9e3DHeaNY+vE2/u2OV5m/alOuQ5I8URNXWUYk0E497EAe+co4CqMRzr1zDo+8qXnwkloVsiBigX0YjJK7CDCsT1dmXXMsR/bvztcfeptzp8/hsflr2BVP5Do0yZFYwgN7dyo0M7mb2WlmtsTMKs3s+gbanGtmC83sPTP7Y2bDFGl9PTsV8ftLx/CdM4axdks1X3twPuNufp5fPfdP3fDUBtXEk4G9gQmaMc/dzKLANOAUYDUwz8xmufvCOm0GA98Gxrv7J2Z2QGsFLNKaCqMRLj9+IJceO4BXKtdz72sr+en/LuXjrdXcNOlwzII7kpN9E0skA7v0ADTvJqbRQKW7LwcwsweBScDCOm0uB6a5+ycA7r4u04GKZFMkYkwYUsLxg4u55enF3PnycoqiUb77+WFK8G1ELBHykTvQF1hVZ3s1MKZemyEAZjYbiALfd/e/ZSRCkRwyM64/fSg1iSQzZ6+gqCDCt047VAm+DYgHvOaeqeUHCoDBwESgFHjZzEa4+2fmlZnZVGAqQP/+/TN0aZHWZWbc8Pnh1MSTTH9pGR9vqebrpwyhX8+OuQ5NWlFNwMsyzYl8DdCvznZpel9dq4FZ7h5z9xXAUlLJ/jPcfYa7l7t7eUlJyf7GLJJ1ZsZNkw7n6hMO4cl3P+KE217k24+8w+pPduQ6NGklQS/LNCfyecBgMxtgZkXAZGBWvTaPkhq1Y2bFpMo0yzMYp0jORSLGN/9lKC9/8wTOH9Ofv7yxhhNve4m7XlmOu2bThE1NPBnoskyTyd3d48A1wDPAIuAhd3/PzG40szPTzZ4BNpjZQuAF4JvuvqG1ghbJpQO7tecHkw7npf83kYmHlvDDJxdx6b0VbNi2K9ehSYa4O4s+2kpZr065DmW/Wa5GHOXl5V5RUZGTa4tkirvz+7nv88MnFtGjUyG/nDyKsQN75TosaaHKdds4+Wcv8f/PGsF5Y/Lr80Eze8Pdy5tqF9yCkkgeMDMuPKaMv149jk5FBVz4u9d5eWlVrsOSFppduR6AYwcV5ziS/afkLpIBhx3UjUe+Mo5DDujM5fdVMGeZqpJB9mrlevr17ED/XsGdEaXkLpIh3TsWcf+lo+nfsyOX3juPipUbm/V9leu28vU/zWfucr0h5IN4IsncZRs4dlCwZ/TpMXsiGdSrczv+cPkYJt85l4vunsd5Y/pzULf29OnegbJenRjSu/PuG6Dcnfvnvs8Pn1zErniSR+ev4ZoTBvHVkwZTEOApeEH39urNbN0VD3RJBpTcRTLugC7t+cPlY/jKH97kntkrqUk/0Sd1rB0nHHoAxw0p5pE31/D84nUcP6SEG888jGkvVHL785XMXraBX07+HKU9glsSCLLZlesxg2MOCfYH45otI9KK3J0N22v4cNNOlqzdyotLqnh5aRVbd8UpKojw7dOH8h/HlO1eM/yx+Wv4n78uwAx+dd6RTBgS7NJAEJ175xx21MR54trjch3KXjV3toxG7iKtyMwo7tyO4s7tGFnanS+V9yOWSDJ/1SYO7Np+jyUMJn2uL6P69WDq7yu4+O7X+ea/DOXKCQO1lk2WbN8V560PPuGSYwfkOpQWU2FPJMsKoxGOLuvZ4No0/Xt15JGvjOP0EX348d8Wc80Db7F9VzzLUbZNr6/cSCzhHBfwD1NBI3eRvNSxqIA7pozi8IO6ceszi5m7bAMXjy/jgrFldOtYmOvwQuvVf66nqCBCeVmPXIfSYhq5i+QpM+OqiYfw8JXHMKK0G7c9u5RxtzzHj55cyHotddAqZleu5+iyHrQvjOY6lBZTchfJc0cd3JN7Lh7NU189jpOH9+Z3r67g+Ftf4Gf/u5St1bFchxca67ZWs3jtVsYHfApkLSV3kYAYflBXfjl5FM9eN4GJh5Zw+3P/ZMJPXuSuV5azs0YP8m6pma+uBOD4wcGvt4OSu0jgDDqgM78+/ygeu3o8w/p04YdPLuK4W19Qkm+BJ9/5iOkvLWPK6H4cdlDXXIeTEZrnLhJwr6/YyC+fW8rsyg0Udy7iqomDOH9M/1DUjbNh8dotnDXtNYb16cIDU8fSriC/f27Nneeu5C4SEvNWbuQXf08l+T7d2vO1kwbzxaNKtZRBIzbtqOHMO2ZTHUvwxLXHckDX9rkOqUla8lekjTm6rCd/uGwsf7xsDL27tuf6R97l1J+/rCWIG5BIOtc+8BZrN1cz/YKjApHY94WSu0jIjBtUzF+/Mo7fXliOAxfOfJ1r/vgmH2+pznVoeeXWZxbzyj/Xc+Okwziyf/Dntden5C4SQmbGKcN78/TXjuO6k4fw7MKPOemnL3H/3PdD97zXmniSBWs2k0g2v1+PzV/DnS8t54KxBzN5dH49aSlTVHMXaQNWrN/Odx9dwKuV6zlx6AHc+sWRFHdul+uwMuL6v7zDg/NW0aNjIScO7c0pw3tz2EFdaV8YpUNRlA6FUaKRT9fmWbBmM1+c/hoj+3bn/svGUFQQrDFuRj9QNbPTgF8CUeAud7+l3vGLgJ8Aa9K77nD3uxo7p5K7SHYlk869c1Zy89OL6dq+gJ986QhOOPSAXIfVIrMr13P+Xf/g3444iKjB84vXsaX6s+vwRAyG9O7CqP49+Fy/btz+XCVJdx6/9thAvsFlLLmbWRRYCpwCrAbmAVPcfWGdNhcB5e5+TXMDVHIXyY3Fa7fwnw/OZ/HarVx74iCuO3nI7iWHg2RHTZx/+cXLFEQiPP2142hfGCWWSFKx8hNWf7KD6liCnbEEm3bEeHfNZuav2sTW6jjtCiI8fOU4RpR2y3UX9ksml/wdDVS6+/L0iR8EJgELG/0uEclLQw/syqNXj+eGxxbwq+crWVa1jZ9+6XN0KMrv+d31/fTZpazauJM/TR27e05/YTSSfsjGng/aSCadZVXbMDMGHdA5y9FmX3OKTX2BVXW2V6f31XeOmb1jZg+bWb+9ncjMpppZhZlVVFVpepZIrrQvjPLjc0by32cM5ekFa/nyjDl5PZumausu/rF8A59srwHgzQ8+YebsFfz72P6MGdi8JyZFIsbg3l3aRGKHzC35+zjwgLvvMrMrgHuBE+s3cvcZwAxIlWUydG0R2Q9mxtTjD2FAcWe+9uBbnP7LV7hqwiH8+9iD82oUv2DNZi6c+Tob04m9pEs7kknnwK7t+dZpQ3McXf5qzsh9DVB3JF7Kpx+cAuDuG9y9dg3Su4CjMhOeiLS2U4b35pGvjGN4n6786KlP16mp2ror59MmX1+xkSkz5tKhMMqvzz+S75wxjAlDShhQ3ImffukIurTX2vYNac4HqgWkPlA9iVRSnwec5+7v1WnTx90/Sr8+C/iWu49t7Lz6QFUk/7y+YiM//9+lzFm+AYBuHQo5pKQTQ/t05aJxZQzp3eUz7ZNJ5/WVG1mxfjtrN1fz8ZZqzIwrjh9IWXGnFsXywuJ1XHn/G5T26MD9l42hT7cOLTpfWGR6KuQZwC9ITYWc6e4/MrMbgQp3n2VmNwNnAnFgI3CVuy9u7JxK7iL5660PPuGtDzZRWbWNZeu28e6azeyMJThrVF+uO3kIvToX8Zc313D37BUsr9oOgBn06tSO7bvixJNJLjl2ANeeOJjO7fa9+jtvZWrEPrRPF+69eDS9AjhlsbVo4TARyZiN22uY/tIy7n1tJUl3OhRG2VIdZ2RpNy4ZP4CjB/TkgC7tKIxGWLelmh//bQl/eXM1JV3accXxA/n8yIM4sFvz1m5JJJ3P/+pVtuyM8fR/HkdXlV4+Q8ldRDJu7eZqfvNiJZt3xjh/7MGUH9wDs73PkZ+/ahM3PbGQN97/BICjDu7BGSP6cNrhB9K3e8Mllj/+4wP++6/vMu28I/nXkX1apR9BpuQuInlhWdU2nnrnI5589yMWr90KwIi+3Th1eG/+7YiDPlOb37wzxgm3vcigks786YqxDb5xtGVK7iKSd5ZXbeOZ9z7m2YVreeuDTRRGjW+ceihTjxtIJGLc9MRCZs5ewePXHMvhfYN5B2lry+QdqiIiGTGwpDNXTezMVRMP4aPNO7nx8YXc8vRiXvlnFVdPHMS9r63ky+X9lNgzIFjLoYlIaPTp1oFfn38kPz5nBG++v4nz7voHHQqjfOPUQ3MdWiho5C4iOWNmfPno/pSX9eQHjy/kzCMOoqSLpj1mgpK7iOTcISWdue+S0bkOI1RUlhERCSEldxGREFJyFxEJISV3EZEQUnIXEQkhJXcRkRBSchcRCSEldxGREMrZwmFmVgW830iTbsDmfTzenH2Nbdd9XQysb+T6zdVUP5rbrqHj+9rnfOlvU20z1d/627WvM9XfhmLan3bN7XNQ/qabaqu/6Yb3N9bfg929pMlI3D0vv4AZ+3q8Ofsa2673uiIb/WhJf/enz/nS36baZqq/DfU5U/3Nxe84KH/T2fodt7X+Nvcrn8syj+/H8ebsa2y7qWvuj+aec3/629D+5vYxl/1tqm2m+lt/Owy/46D0t6m2+ptueH9Tv+Mm5awsk+/MrMKbsWZyWKi/4dfW+tzW+ltfPo/cc21GrgPIMvU3/Npan9tafz9DI3cRkRDSyF1EJISU3EVEQkjJXUQkhJTc94OZdTKzCjP7fK5jyQYzG2Zm083sYTO7KtfxtDYz+4KZ/dbM/mRmp+Y6ntZmZgPN7Hdm9nCuY2lN6X+396Z/t+fnOp7W1qaSu5nNNLN1Zrag3v7TzGyJmVWa2fXNONW3gIdaJ8rMykSf3X2Ru18JnAuMb814WypD/X3U3S8HrgS+3JrxtlSG+rvc3S9t3Uhbxz72/2zg4fTv9sysB5tlbWq2jJkdD2wD7nP3w9P7osBS4BRgNTAPmAJEgZvrneIS4AigF9AeWO/uT2Qn+v2TiT67+zozOxO4Cvi9u/8xW/Hvq0z1N/19PwX+4O5vZin8fZbh/j7s7l/MVuyZsI/9nwQ87e7zzeyP7n5ejsLOijb1gGx3f9nMyurtHg1UuvtyADN7EJjk7jcDe5RdzGwi0AkYDuw0s6fcPdmacbdEJvqcPs8sYJaZPQnkbXLP0O/YgFtIJYK8TeyQud9vUO1L/0kl+lJgPm2gatGmknsD+gKr6myvBsY01NjdvwNgZheRGrnnbWJvxD71Of2GdjbQDniqVSNrHfvUX+Ba4GSgm5kNcvfprRlcK9jX328v4EfAKDP7dvpNIMga6v/twB1m9q+0zjIFeUXJfT+5+z25jiFb3P1F4MUch5E17n47qUTQJrj7BlKfL4Sau28HLs51HNkS+v+aNMMaoF+d7dL0vjBra31Wf8Pd3/raev8BJXdIfdgy2MwGmFkRMBmYleOYWltb67P6G+7+1tfW+w+0seRuZg8Ac4BDzWy1mV3aAYBIAAAAX0lEQVTq7nHgGuAZYBHwkLu/l8s4M6mt9Vn9DXd/62vr/W9Mm5oKKSLSVrSpkbuISFuh5C4iEkJK7iIiIaTkLiISQkruIiIhpOQuIhJCSu4iIiGk5C4iEkJK7iIiIfR/TpQRX2Ur7bwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.recorder.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here 5e-2 looks like a good value, a tenth of the minimum of the curve. That's going to be the highest learning rate in 1cycle so let's try a constant training at that value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HBox(children=(IntProgress(value=0, max=2), HTML(value='0.00% [0/2 00:00<00:00]'))), HTML(value…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total time: 00:05\n",
      "epoch  train loss  valid loss  accuracy\n",
      "0      0.052523    0.046770    0.978901  (00:02)\n",
      "1      0.033484    0.068837    0.977429  (00:02)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "learn.fit(2, 5e-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also see what happens when we train at a lower learning rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HBox(children=(IntProgress(value=0, max=2), HTML(value='0.00% [0/2 00:00<00:00]'))), HTML(value…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total time: 00:05\n",
      "epoch  train loss  valid loss  accuracy\n",
      "0      0.117201    0.095314    0.968106  (00:02)\n",
      "1      0.080922    0.060936    0.978410  (00:02)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "model = simple_cnn((3,16,16,2))\n",
    "learn = Learner(data, model, metrics=[accuracy])\n",
    "learn.fit(2, 5e-3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training with the 1cycle policy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now to do the same thing with 1cycle, we use [`fit_one_cycle`](/train.html#fit_one_cycle)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HBox(children=(IntProgress(value=0, max=2), HTML(value='0.00% [0/2 00:00<00:00]'))), HTML(value…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total time: 00:04\n",
      "epoch  train loss  valid loss  accuracy\n",
      "0      0.100741    0.040601    0.987733  (00:02)\n",
      "1      0.030833    0.022708    0.993131  (00:02)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "model = simple_cnn((3,16,16,2))\n",
    "learn = Learner(data, model, metrics=[accuracy])\n",
    "learn.fit_one_cycle(2, 5e-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This gets the best of both world and we can see how we get a far better accuracy and a far lower loss in the same number of epochs. It's possible to get to the same amazing results with training at constant learning rates, that we progressively diminish, but it will take a far longer time.\n",
    "\n",
    "Here is the schedule of the lrs (left) and momentum (right) that the new 1cycle policy uses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs0AAAD8CAYAAACM9pmgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8lNXZ//HPlX1PIAkhZEfWsENYAriiFVcEQVEEEVvbp7Wt1dq6V2ndbW1ttT72JyiIIiIgVhQXXENYgux7IDuQhEAWyJ45vz+Y9kkpSoAkZyZzvV+vvJzc95nkO0LCNfd9znXEGINSSimllFLqu3nZDqCUUkoppZSr06JZKaWUUkqp09CiWSmllFJKqdPQolkppZRSSqnT0KJZKaWUUkqp09CiWSmllFJKqdPQolkppZRSSqnT0KJZKaWUUkqp09CiWSmllFJKqdPwsR3gZFFRUSY5Odl2DKWUOisbNmw4bIyJtp2jPenvbaWUuzqT39kuVzQnJyeTlZVlO4ZSSp0VEcmznaG96e9tpZS7OpPf2To9QymllFJKqdPQolkppZRSSqnT0KJZKaWUUkqp09CiWSmllFJKqdPQolkppZRSSqnTaFHRLCLjRWS3iGSLyH2nOO8vIm87z68VkWTn8WQRqRGRTc6Pl1s3vlJKKaWUUm3vtC3nRMQbeBG4DCgE1ovIcmPMjmbDbgeOGmN6iMhU4GngRue5fcaYwa2cWymllFJKqXbTkj7NI4BsY8x+ABFZCEwAmhfNE4BHnY8XA38TEWnFnKqdbC2s4PPdJfj7eBHg601EkC+x4YHEhgfQLSIQby/9Y1VKnZvskireXFvAA1f2wcdbZwkqpVquyWE4UF5DXlk1ZcfrKK9uoLKmgbBAX24dndym37slRXMcUNDs80Jg5HeNMcY0ikgFEOk8lyIiG4FK4CFjzNcnfwMRuQO4AyAxMfGMXoBqPcYYfrVoE9klx0553t/Hi95dQ0mNDWNYUidG94giLiKwnVMqpdzdvtLjzMnIYURKZ8b372o7jlLKRRlj2Fd6jPW5R9mYf5RNBeXkHq6mvsnxX2PPiw52iaL5XBwEEo0xZSIyDFgmIv2MMZXNBxljXgFeAUhLSzNtnEl9h6/3Hia75BjPTh7IVQNjqa5vory6ngPltRysqGFv8TF2HKzkw22HWLj+xPuo5MggLkuN4YoBsQxJiEBvMCilTmdcny50Cw9g/ppcLZqVUv+hyWHI3FfGxzsOsWpXCYVHawDoFOTL4IQILu7The5RwSRFBhMV4kdEkB/hgb74tsNdq5YUzUVAQrPP453HTjWmUER8gHCgzBhjgDoAY8wGEdkH9AJ0v1UXNCcjh+hQf64d3A1/H2+C/HyICvGnR5fQ/xjncBh2F1eRua+Mr/aW8trqXP7xdQ5xEYFMHhbP1BEJxIbrFWil1Kn5eHsxbVQSz67cTXZJ1X/9jlFKeZ7skmO8k1XAsk1FFFfWEejrzZgekfzPRecx+rwokiODrF+Ya0nRvB7oKSIpnCiOpwI3nzRmOXArkAlMBlYZY4yIRANHjDFNItId6Ansb7X0qtVklxzji92l3H1ZL/x9vL93rJeX0Dc2jL6xYcwam0JFTQOf7Sxm6cYiXli1l7+u2sslfWL4yYXdSUvu3E6vQCnlTqYOT+Avn+5lfmYej03obzuOUsoCYwxf7T3MnG9y+HJPKT5ewkW9o3nk6njG9e1CgO/31yPt7bRFs3OO8p3ASsAbmGOM2S4is4EsY8xy4FVgvohkA0c4UVgDXADMFpEGwAH8xBhzpC1eiDo3r63Owc/Hi5tHnvmc8vBAXyYNjWfS0HgKjlTz1rp83lqXz+SXixmR3Jmfj+vB+T2j2yC1UspdRYb4c9XAWN79toh7x/chxL+tZwsqpVyFMYZVu0r448d72HGwkuhQf+65rBc3jUwkKsTfdrzvJCdmULiOtLQ0k5WlszfaU0V1A6Oe/IxrBsXyzORBrfI1q+sbWbiugH98vZ+DFbWc3zOK+67oQ79u4a3y9ZVyVSKywRiTZjtHezrb39sb848y8aXV/H5CP6anJ7d+MKWUy8nIPsyzK3ezqaCcxM5B/PySHkwYHIefj51OOmfyO1vf2isWrs+npqGJ28aktNrXDPLzYdbYFG4ZlcQba/J4YdVerv7rN0wcEsdvx/chJiyg1b6XUso9DU6IYEBcOPMy87hlVJL1+YpKqbZTeLSa2e/v4OMdxcSGB/DkpAFMHhbfLgv4WosWzR6uscnB66tzSe8eSd/YsFb/+n4+Xswam8L1w+J56Yts5mbk8sn2Yn57RR9uHpGIl/Z9VspjiQgz0pO4d/EWMveXMfq8KNuRlFKtrK6xif/3dQ5/XbUXgHsv783tY1Ncbr5yS7hPea/axMrtxRyoqGXW2Na7ynwq4YG+3H9FXz6+6wIGxIfz0LJt3PhKJtklVW36fZVSru2aQd2ICPJlfmae7ShKqVa2raiCa/76Dc+u3M1Fvbrw2T0X8bOLe7hlwQxaNHu8ORk5JEUGcUmfLu3y/ZKjglnww5E8M3kge4qPceUL3/D66lxcbW69Uqp9BPh6c2NaAh/vKOZgRY3tOEqpVtDY5OCFz/Zy3YsZVNQ0MGdmGi9PH+b2G6Jp0ezBNhWUsyHvKDNHJ7fr9tgiwg1pCXx694WM7RHF75Zv5/bXsyg7VtduGZRSruOWUUk4jOHNtfm2oyilzlF+WTXXv5zJnz7Zw5UDYll51wVc0ifGdqxWoUWzB5ubkUOovw9T0hJOP7gNRIf68+qtaTx6TSrfZB9m/F++JiP7sJUsSil7EjoHcUnvLry1Lp+6xibbcZRSZ+mTHcVc9devySk9xt9uHsILNw0hIsjPdqxWo0WzhzpUUcsHWw5yw/AEq/1RRYSZY1J472djCA/0Zfqra/nHV/t1uoZSHmbG6GQOH6vno22HbEdRSp2hxiYHT324ix/NyyIpMogPfnE+Vw/sZjtWq9Oi2UPNX5OLwxhmjk62HQWAvrFhLPvZGC7v15XHV+zklws3UVOvV5yU8hTn9zixTe48XRColFspr65nxpx1vPzlPm4emcjin4wmoXOQ7VhtQotmD1Tb0MSba/O5tG+MS/3FDvH34aVpQ7n38t68v+UA1/99NQfKdWGQUp7Ay0u4ZVQSG/KOsq2ownYcpVQL5Bw+zqSXVpOVe5TnpgziiYkD3LYzRkto0eyBlm0s4mh1Q5u3mTsbIsLPLu7BnJnDKThSzaSXVrPzYKXtWEqpdjBlWAKBvt7afk4pN7BmfxkTX8qgvKaBBT8ayeRh8bYjtTktmj2MMYY5GTmkxoYxMqWz7Tjf6eLeXVj0k3QAbng5UxcIKnUGRGS8iOwWkWwRue8U55NE5DMR2SIiX4hI/Ennw0SkUET+1n6pITzIl+uGdOO9zUWUV9e357dWSp2B9zYVMf3VtUQG+7H0p6MZnuy69URr0qLZw2Rkl7Gn+Bizxqa4/Ja1fWPDWPqz0XSLCOTWOetYurHQdiSlXJ6IeAMvAlcAqcBNIpJ60rDngHnGmIHAbODJk87/HviqrbOeyvRRydQ2OHgnS3/elXJFr6/O5ZcLNzEsqRNLfjqGpMhg25HajRbNHmZORg5RIX5cMyjWdpQWiQ0PZNFP0hme3Jlfvb2Z+Wv0tq1SpzECyDbG7DfG1AMLgQknjUkFVjkff978vIgMA2KAj9sh639J7RbG8OROzF+Th8OhXXSUchXGGP786R5+t3w7l6XG8NptIwgP9LUdq11p0exB9pceY9WuEm4ZlYS/j/tM1A8P9GXubcMZ16cLDy/bxj++2m87klKuLA4oaPZ5ofNYc5uBSc7HE4FQEYkUES/gj8Cv2zzl95ienkz+kWq+3FNqM4ZSyskYw2Pv7+DPn+5l8rB4/j5taIde8PddtGj2IK+tzsXP24tpI5NsRzljAb7evDx9GFcNjOXxFTv586d7tJezUmfv18CFIrIRuBAoApqAnwIrjDGnnRshIneISJaIZJWWtm5xO75fV6JD/ZmXmduqX1cpdeaMMTz83jZeW53L7WNTeOb6gfh4e2b5aG9XC9WuKmoaWLyhkGsHdyM61N92nLPi6+3FC1OHEOjrzZ8/3UuTw3DPD3rbjqWUqykCmm/zGe889m/GmAM4rzSLSAhwvTGmXETSgfNF5KdACOAnIseMMf+1mNAY8wrwCkBaWlqrvoP18/HiphGJ/HXVXvLKjnvUnEmlXIkxht8t384ba/L58QXdue+KPi6/HqoteeZbBQ+0aH0B1fVN3DYm2XaUc+LtJTxz/UBuSIvnr6uyefHzbNuRlHI164GeIpIiIn7AVGB58wEiEuWcigFwPzAHwBgzzRiTaIxJ5sTV6HmnKpjbw7SRiXiL8IauY1DKCmMMjy7fzrzMPO7QghnQotkjNDY5eG11LiNTOtOvW7jtOOfMy0t4ctJArhvcjWdX7ub/fa1znJX6F2NMI3AnsBLYCSwyxmwXkdkicq1z2EXAbhHZw4lFf49bCfs9YsICuLxfVxZlFeruoEq1s3/NYX49M48fjk3hfi2YAZ2e4RE+2VFMUXkNj1xzctcp9+XtJTw3ZRD1TQ7+8MFO/H28mJ6ebDuWUi7BGLMCWHHSsUeaPV4MLD7N13gNeK0N4rXYjPQkPth6kOWbi7hxeKLNKEp5lGdW7v73HOYHr+qrBbOTXmn2AHMyckjoHMilfWNsR2lVPt5e/PnGIVzatwsPv7edd7IKTv8kpZTbGJHSmd4xoczLzNOFv0q1k//39X7+/sU+bh6ZyENaMP8HLZo7uC2F5azPPcrM0Sl4e3W8v/h+Pl68OG0o5/eM4r4lW1m1q9h2JKVUKxERpqcnsf1AJd/ml9uOo1SH9+6GQv7wwU6uHNCV30/orwXzSbRo7uDmZuQS4u/DDWkdd094fx9v/n7LMFJjw/jZgo1szD9qO5JSqpVMHBJHqL+Ptp9Tqo19uqOY37y7hTE9Inn+xsEd8kLbudKiuQMrrqzln1sOMCUtntCAjr1rT4i/D3NmDic61J9Zr61nf+kx25GUUq0g2N+H64fFs2LrQUqr6mzHUapDWp97hJ+9+S39uoXxv9PT3GoDtPakRXMH9saaPBodhpmjk21HaRfRof7MmzUCLxFmzFlHSVWt7UhKqVYwPT2JhibDwnX5tqMo1eHkHD7Oj+Zl0S0ikLkzhxPirz0ivosWzR1UbUMTC9bmc2nfGI/aGCA5Kpg5M4dz5Hg9t81dT3V9o+1ISqlzdF50COf3jGLB2nwamxy24yjVYRw9Xs9tc9chwGu3DScyxD03P2svWjR3UMs3HThROLr5ZiZnY1BCBC9OG8rOg5XctXATDoeuulfK3U0flcShylo+2aGLfZVqDXWNTfx4/gYOlNfyjxlpHnWB7Wxp0dwBGWOYk5FDn66hpHePtB3Hiot7d+Ghq1L5eEcxz36823YcpdQ5Gtc3hriIQOZl6g6BSp0rYwz3vbuVdblHeHbKQNKSO9uO5Ba0aO6AMveVsetQFbPGpnh0u5jbxiRz88hE/v7FPhZvKLQdRyl1Dry9hGmjEsncX8be4irbcZRyay98ls3SjUXcfVkvJgyOsx3HbWjR3AHNycghMtiPawd1sx3FKhHhsWv7Mfq8SO5fsoX1uUdsR1JKnYMb0xLw8/HSq81KnYOPth3i+U/3MGlIHD+/pIftOG5Fi+YOJufwcT7bVcK0UUkE+GrLGF9vL16aNpT4TkH8eP4GCo5U246klDpLkSH+XD0wliXfFlJV22A7jlJuZ29xFfcs2sSg+HCemDTAo+9Gn40WFc0iMl5EdotItojcd4rz/iLytvP8WhFJPul8oogcE5Fft05s9V1eX52Lj5dwy6hE21FcRkSQH6/emkZDo4P/WbCB2oYm25GUUmdpRnoyx+ubWPJtke0oSrmVipoGfjQvi0A/H16ePkwvrJ2F0xbNIuINvAhcAaQCN4lI6knDbgeOGmN6AM8DT590/k/Ah+ceV32fipoGFmUVcM2gbnQJDbAdx6V0jw7h+RsHs62okoeWbcMY7aihlDsanBDBoPhw5q/J059jpVqoyWH45cKNFB6t4e+3DCU2PNB2JLfUkivNI4BsY8x+Y0w9sBCYcNKYCcDrzseLgXHivOYvItcBOcD21omsvss7WQVU1zcxa0yK7Sgu6dLUGH4xrieLNxSyYK1ukqCUu5qenkx2yTEy95XZjqKUW/jTJ7v5Yncpj17bj+HaKeOstaRojgMKmn1e6Dx2yjHGmEagAogUkRDgt8Bj5x5VfZ8mh+G11bmMSO5M/7hw23Fc1l3jenJx72gee387G/KO2o6jlDoLVw+MpVOQL69n5tqOopTLW7H1IC9+vo+bRiQwbaRO3TwXbb0Q8FHgeWPMse8bJCJ3iEiWiGSVlpa2caSO6ZMdxRQerWHW2GTbUVyal5fw5xuHEBseyE8XbNCttpVyQwG+3tw4PJFPdhRTVF5jO45SLmtf6THufWczQxIjePTafrrw7xy1pGguAhKafR7vPHbKMSLiA4QDZcBI4BkRyQXuAh4QkTtP/gbGmFeMMWnGmLTo6OgzfhHqRJu5+E6BXJba1XYUlxce5Mv/Th9GRU0Ddy7YSINuy6uU25k2MhEDvLlW288pdSo19U38bMG3+Pt689K0ofj76MK/c9WSonk90FNEUkTED5gKLD9pzHLgVufjycAqc8L5xphkY0wy8GfgCWPM31opu3LaVlTBupwjzBydjLeXvotsib6xYTx9/UDW5R7hjx/vsR1HKXWGEjoHMa5PDAvXFVDXqB1xlDrZ75ZvY3dxFc/fOFgX/rWS0xbNzjnKdwIrgZ3AImPMdhGZLSLXOoe9yok5zNnA3cB/taVTbWdORg7Bft7cMDzh9IPVv00YHMfNIxN5+ct9fL67xHYcpdQZmpGeRNnxej7cesh2FKVcyuINhSzKKuTOi3twYS+9g99afFoyyBizAlhx0rFHmj2uBaac5ms8ehb51GmUVNXy/uYDTBuZRFiAr+04bueRq1P5Nu8o9yzazIpfnE/XcG3Vp5S7GNsjiu5Rwbyemct1Q3QrYKUAdh+q4qFlWxnVvTN3XdrLdpwORXcEdHNvrMmn0WG4dXSy7ShuKcDXmxenDaW2oYlfvLWRRp3frJTb8PISbhmVxMb8crYWVtiOo5R1x+sa+emCDYT4+/LC1CE6ZbOVadHsxmobmliwJo9xfbqQEhVsO47bOi86hMcn9mdd7hH+8tle23GUUmfg+mHxBPl5My8z13YUpax7eNk2cg4f54Wpg+kSpndOW5sWzW7s/c0HKDtez226mck5mzgkninD4vnb59l8s/ew7ThKqRYKD/TluiFxLN98gKPH623HUcqapRsLWbKxiF+M68noHlG243RIWjS7KWMMczJy6R0TyujzIm3H6RAem9CPHtEh3PX2Ru3frJQbmZGeRF2jg3c2FJx+sFIdUH5ZNQ8v286I5M78/JKetuN0WFo0u6k1+4+w82Als8Yma7PyVhLk58OL04ZSVdvIbxdvwRhjO5JSqgX6dA1jREpn5q/Jo8mhP7fKszQ0OfjFwo2IwPNTB+s85jakRbObmpORQ+dgPyYM1hXjralXTCgPXNmXz3eXMn+NbpqglLuYkZ5EwZEavtyj7SOVZ3nhs71sKijnyUkDiIvQfsxtSYtmN5RXdpxPdxYzbWQiAb66w09rm5GexMW9o3n8g53sLa6yHUcp1QKX9+tKl1B/Xl+tb3aV51i7v4y/fZ7NlGHxXD2wm+04HZ4WzW7otdW5+DhbLanWJyI8M3kQIf4+/GLhJt1tTCk34Ovtxc0jE/lyTym5h4/bjqNUm6uobuBXb28iOTKYR6/tZzuOR9Ci2c1U1TbwTlYhVw/sRoy2k2kz0aH+PDN5IDsPVuo220q5iZtHJOLjJTq1SnV4xhgeWLqVkqo6/jJ1MMH+LdqrTp0jLZrdzKKsQo7VNTJL28y1uXF9Y7hlVCKvfLWfjGxtQ6eUq+sSFsD4/l15J6uAmnq9Q6Q6rnc2FPLB1oPc84PeDIyPsB3HY2jR7EaaHIbXVueQltSJAfHhtuN4hAevTOW86GDuWbSZ8mrtAauUq5uRnkxlbSPvbSqyHUWpNlFwpJrHlm9nVPfO/PiC7rbjeBQtmt3IZzuLKThSw6yxepW5vQT6efOXqUMoO17HA0u3ahs65RZEZLyI7BaRbBG57xTnk0TkMxHZIiJfiEi88/hgEckUke3Ocze2f/pzMzy5E326hvJ6Zp7+vKoOx+Ew3Lt4MyLCc1MG4aXt5dqVFs1uZE5GDnERgfwgNcZ2FI/SPy6cuy/rzYqth1i++YDtOEp9LxHxBl4ErgBSgZtEJPWkYc8B84wxA4HZwJPO49XADGNMP2A88GcRcat7vyLCjPRkdh6sZEPeUdtxlGpVr63OZc3+IzxydSrxnYJsx/E4WjS7ie0HKliz/wi3jk7Cx1v/2NrbHRd0Z2hiBI+8t53iSt0tULm0EUC2MWa/MaYeWAhMOGlMKrDK+fjzf503xuwxxux1Pj4AlADR7ZK6FV03pBuhAT68nqkLAlXHkV1yjKc/2sW4Pl2YkhZvO45H0urLTczNyCXIz5sb0xJtR/FI3l4nboXVNTZx/xKdpqFcWhzQfD/pQuex5jYDk5yPJwKhIhLZfICIjAD8gH1tlLPNBPn5MGVYAh9tO0hJlb7JVe6vscnBPe9sJsjPmyevH6A7AVuiRbMbKK2qY/mmA0weFk94kK/tOB6re3QIv7m8D6t2lfBOVqHtOEqdi18DF4rIRuBCoAj4d7sJEYkF5gO3GWMcp/oCInKHiGSJSFZpaWl7ZD4j09OTaGgyLFxXcPrBSrm4v3+xj80F5fzhugF0CdV2s7Zo0ewGFqzNo77JwczRybajeLyZo5MZmdKZ2f/cQVF5je04Sp1KEZDQ7PN457F/M8YcMMZMMsYMAR50HisHEJEw4APgQWPMmu/6JsaYV4wxacaYtOho15vBkRIVzAW9olmwNo+GplPW/Uq5hW1FFfzls71cO6gbVw2MtR3Ho2nR7OLqGpt4Y00el/TpQvfoENtxPJ6Xc5qGwxh+u3iLTtNQrmg90FNEUkTED5gKLG8+QESiRORfv//vB+Y4j/sBSzmxSHBxO2ZuEzNGJVFcWccnO4ptR1HqrNQ1NnHPos10DvZj9gTd9c82LZpd3PubD3L4WL1uZuJCEjoH8eBVffkm+zBvrM23HUep/2CMaQTuBFYCO4FFxpjtIjJbRK51DrsI2C0ie4AY4HHn8RuAC4CZIrLJ+TG4fV9B67m4TxfiIgJ5fXWu7ShKnZU/fbKH3cVVPH39QCKC/GzH8Xi676ILM8YwNyOHXjEhjOkRefonqHZz84hEPtp2iCc+2MkFPaNIigy2HUmpfzPGrABWnHTskWaPFwP/dSXZGPMG8EabB2wn3l7C9PQknvpwF7sPVdG7a6jtSEq12Mb8o/zjq/3cNCKBi/t0sR1HoVeaXdq6nCNsP1DJbWNSdKWsixERnr5+ID5ewr3vbMHh0GkaSrmiG9IS8PPxYv6aXNtRlGqxusYm7l28ha5hATxwZV/bcZSTFs0ubE5GDp2CfJk45ORuUcoVdIsI5JFrUlmXe4TX9PavUi6pc7Af1w7qxpJvi6isbbAdR6kW+etn2WSXHOOJSQMIDdCuWa5Ci2YXlV9Wzcc7irl5ZCIBvt6246jvMHlYPBf3jubZlbvJL6u2HUcpdQoz0pOorm9iyQZtFalc37aiCv7+5T4mD4vnot46LcOVaNHsol7PzMVbhOmjkm1HUd9DRHh84gC8vYT7l2o3DaVc0cD4CAYnRDBvTZ7+jCqX1tDk4DeLt9A52I+Hr0q1HUedRItmF1RV28Db6wu4amAsXcO1ibmr6xYRyH1X9CEju4xFWbqRglKuaEZ6EvtLj5ORXWY7ilLf6X+/3MeOg5U8fl1/3czMBWnR7IIWbyjkWF0jt2mbObdx84hERqZ05g8f7KS4UrftVcrVXDkgls7BfszLzLUdRalT2lNcxQufZXPNoG78oF9X23HUKWjR7GKaHIbXVucyLKkTgxMibMdRLeTlJTx1/UDqGx08uHSb3gJWysUE+HozdXgCn+4s1t08lctpchjuXbyFkAAfHr1Gp2W4Ki2aXcyqXSXklVVz25hk21HUGUqJCuaeH/Ti053FvL/loO04SqmTTBuVBMCCNXmWkyj1n+Z8k8PmgnIevbYfkSH+tuOo76BFs4uZm5FDt/AAxuutGbc0a0wKg+LDeXT5do4cr7cdRynVTFxEIJf2jWHh+gJqG5psx1EKgJzDx3nu491clhrDNQNjbcdR30OLZhey82Alq/eVMWN0Mj7e+kfjjny8vXhm8iCqaht47P3ttuMopU4yIz2ZI8frWbFV7wYp+xwOw28Xb8Hfx4s/XNdfNzJzcVqZuZC5GTkEOufdKffVu2soP7u4B+9tOsBnO4ttx1FKNTOmRyTdo4N5PVOnaCj73lyXz7rcIzx0dSoxYdoty9W1qGgWkfEisltEskXkvlOc9xeRt53n14pIsvP4CBHZ5PzYLCITWzd+x3H4WB3LNh3g+mFxRAT52Y6jztFPL+pB75hQHly6TXchU8qFiAgzRiWxuaCcLYXltuMoD1ZcWcvTH+5iTI9IpgyLtx1HtcBpi2YR8QZeBK4AUoGbROTkpZ23A0eNMT2A54Gnnce3AWnGmMHAeOB/RcSntcJ3JG+uzae+0cHM0dpmriPw8/HimckDKamq5akPd9mOo5RqZtKweIL8vJmnV5uVRY+9v536JgePXzdAp2W4iZZcaR4BZBtj9htj6oGFwISTxkwAXnc+XgyMExExxlQbYxqdxwMA7cN1CnWNTcxfk8dFvaPp0SXEdhzVSgYlRDBrTApvrs1nfe4R23GUUk5hAb5MGhrH8s0HdMGusuLTHcWs2HqIX4zrSXJUsO04qoVaUjTHAc23OSt0HjvlGGeRXAFEAojISBHZDmwFftKsiP43EblDRLJEJKu0tPTMX4Wb+2DLQUqr6pilm5l0OL+6rBdxEYHcv2QrdY26Wl8pVzEjPZn6Rofu4qna3bG6Rh55bxu9Y0L50fndbcdRZ6DNFwIx6v9qAAAgAElEQVQaY9YaY/oBw4H7ReS/ZrobY14xxqQZY9Kio6PbOpJLMcYwJyOHHl1COL9nlO04qpUF+/vwh+v6k11yjJe/2G87jlLKqVdMKCNTOjM/M48mh94EVe3njx/v5mBlLU9MGoCfj/ZjcCct+dMqApq3c4h3HjvlGOec5XCgrPkAY8xO4BjQ/2zDdkRZeUfZVlTJbWOSdU5TB3Vxny5cM6gbL36eTXbJMdtxlFJOt45Opqi8hs93ldiOojzE5oJyXludyy0jkxiW1Ml2HHWGWlI0rwd6ikiKiPgBU4HlJ41ZDtzqfDwZWGWMMc7n+ACISBLQB8htleQdxJxvcggP9GXSEF0525E9cnUqAb5ePLBkKw69qqWUS7gsNYaYMH/m6Q6Bqh00NDm4b8lWuoT6c+/43rbjqLNw2qLZOQf5TmAlsBNYZIzZLiKzReRa57BXgUgRyQbuBv7Vlm4ssFlENgFLgZ8aYw639otwVwVHqlm5/RA3j0wk0M/bdhzVhqJD/Xnwqr6syz2icyiVchG+3l5MG5nEV3tK2V+qd4FU25rzTQ47D1by2LX9CQvwtR1HnYUWTaYxxqwwxvQyxpxnjHnceewRY8xy5+NaY8wUY0wPY8wIY8x+5/H5xph+xpjBxpihxphlbfdS3M+8zNwTPUPTk2xHUe3ghrQERnXvzBMrdlJSVWs7jlIKmDoiAV9v4Y01+bajqA6s4Eg1z3+6h8tSYxjfv6vtOOos6Qx0S47VNbJwfQFXDoglNjzQdhzVDkSEJyYOoLbRwez3d9iOo5QCuoQGML5/LO9sKKC6/r+aOyl1zowxPLhsG94izJ7Qz3YcdQ60aLbk3Q2FVNU2MmtMsu0oqh11jw7h5xf34J9bDrJql26xrZQruDU9iaraRpZtPGA7iuqAlm8+wFd7Srn38t56kczNadFsgcNhmJuRw5DECIYk6upZT/PjC8+jZ5cQHl62neN1emVLKduGJXWib2wY8zJzMUYX6qrWU15dz+z3dzAoIYLp6cm246hzpEWzBZ/vLiG3rJrbdDMTj+Tn48VT1w+gqLyGP32yx3YcpTyeiHBrehK7DlWxPveo7TiqA3lixU4qahp4atIAvL20ray706LZgrkZuXQNC+AKXQzgsYYldeaWUYnMzchhS2G57ThKebwJg+MIC/BhXmau7Siqg8jcV8airEJ+eH53+saG2Y6jWoEWze1s96Eqvsk+zIzRSfh66/9+T/ab8X2ICvHnvne30tjksB1HKY8W6OfNDWkJfLTtECWV2t1GnZvahiYeXLqVxM5B/HJcT9txVCvRqq2dzc3IIcDXi5uGJ9qOoiwLC/Bl9oR+7DhYyZyMHNtxlPJ4t4xKotFheHOdtp9T5+alz7PZf/g4j0/sr/swdCBaNLejsmN1LNlYxKSh8XQK9rMdR7mAy/t15bLUGP70yR7yy6ptx1HKoyVHBXNhr2jeXJtPg979UWdpb3EVf/9yHxOHxHF+z2jbcVQr0qK5Hb21Lp/6Rge3jU62HUW5CHH27fTx8uKh97bpyn2lLLt1dBIlVXWs3H7IdhTlhhwOw/1LthLs78NDV/W1HUe1Mi2a20l9o4N5mXlc0CuanjGhtuMoFxIbHsi9l/fmqz2lLN+sfWKVsunCXl1I6BzIvMw821GUG3prfT5ZeUd56KpUIkP8bcdRrUyL5nayYutBSqrqdDMTdUq3jEpicEIEs9/fQXl1ve04ys2JyHgR2S0i2SJy3ynOJ4nIZyKyRUS+EJH4ZuduFZG9zo9b2ze5fd5ewvRRSazLOcKuQ5W24yg3UlJZy1Mf7mL0eZFcPzTOdhzVBrRobgfGGOZk5HBedDAX6PwmdQreXsKTkwZQUdPAEyt22o6j3JiIeAMvAlcAqcBNIpJ60rDngHnGmIHAbOBJ53M7A78DRgIjgN+JiMftwHRDWgL+Pl56tVmdkcfe30Fdo4PHJw5ARHsyd0RaNLeDDXlH2VJYwcwxKXhpc3P1HfrGhvGjC7qzKKuQzH1ltuMo9zUCyDbG7DfG1AMLgQknjUkFVjkff97s/OXAJ8aYI8aYo8AnwPh2yOxSIoL8uHZQN5Z+W0RFTYPtOMoNfLqjmA+2HuQXl/QgJSrYdhzVRrRobgdzM3IJC/DR2zXqtH45rieJnYN4cOlWahuabMdR7ikOKGj2eaHzWHObgUnOxxOBUBGJbOFzPcKto5OpaWji3Q2FtqMoF3e8rpFH3ttGr5gQ7rjgPNtxVBvSormNFR6t5sNtB7lpZCJBfj624ygXF+DrzeMT+7P/8HFe+jzbdhzVcf0auFBENgIXAkXAGb1LE5E7RCRLRLJKS0vbIqNV/ePCGZIYwRtr8nA4tKuN+m5//HgPBytreXLSQPx8tKzqyPRPt43Nz8xDRJiRnmw7inIT5/eMZtKQOP7+5T72FlfZjqPcTxGQ0OzzeOexfzPGHDDGTDLGDAEedB4rb8lzm32NV4wxacaYtOjojrlW49b0ZPYfPs432YdtR1EuanNBOa+tzmHayESGJXnc9H+Po0VzGzpe18hb6/IZ378rcRGBtuMoN/LgVX0J8ffh/iVb9SqXOlPrgZ4ikiIifsBUYHnzASISJSL/+v1/PzDH+Xgl8AMR6eRcAPgD5zGPdMWArkQG++mCQHVKjU0O7l+ylagQf34zvo/tOKodaNHchpZ8W0hlbSOzxqTYjqLcTGSIPw9elUpW3lHeWq9b+qqWM8Y0AndyotjdCSwyxmwXkdkicq1z2EXAbhHZA8QAjzufewT4PScK7/XAbOcxj+Tv483UEQl8tquYgiO6Y6f6T3MycthxsJLZE/oRFuBrO45qB1o0txGHwzA3I5dBCREMTYywHUe5oeuHxjH6vEie+nAXJZW1tuMoN2KMWWGM6WWMOc8Y86+C+BFjzHLn48XGmJ7OMT80xtQ1e+4cY0wP58dcW6/BVUwbmYQAC9bqm1f1fwqOVPOnT/Zwad8YLu/X1XYc1U60aG4jX+4pZf/h48wak6z9GtVZEREenziAukYHj72/w3YcpTxSt4hALkuN4e31+drRRgEn9l54cNk2vEWYPaGf/hvvQbRobiNzMnKICfPnygGxtqMoN5YSFcwvx/Xkg60H+Wxnse04SnmkW9OTOVrdwD+3HLQdRbmA5ZsP8NWeUn59eW+66Xolj6JFcxvYU1zF13sPMyM9GV9v/V+szs2Pzu9Or5gQHl62jeN1jbbjKOVx0s+LpEeXEOZn5tqOoiw7erye2e/vYFBChHbF8kBa0bWBuRm5+Pt4cdOIRNtRVAfg5+PFk5MGcrCylj9+vMd2HKU8jogwfVQSmwsr2FRQbjuOsuiJFTupqGngqUkD8NYdfj2OFs2t7OjxepZ8W8ikoXF0DvazHUd1EMOSOjFtZCKvrc5hS6H+o61Ue5s0NI5gP2/mZebajqIsWZ19mHc2FPKjC7rTNzbMdhxlgRbNrezNdfnUNTq4TdvMqVb2m/F9iArx5753t9LY5LAdRymPEhrgy6Sh8fxzy0HKjtWd/gmqQ6ltaOKBpVtJigzil+N62o6jLNGiuRU1NDmYl5nL+T2j6BUTajuO6mDCAnyZPaEfOw5WMicjx3YcpTzOjPQk6hsdvJ1VYDuKamd/W5VNblk1j183gABfb9txlCVaNLeiFVsPUlxZp5uZqDZzeb+uXNo3huc/2aubLSjVznrGhJLePZIFa/Jp0p06PcbuQ1W8/OU+Jg2NY2zPKNtxlEVaNLcSYwxzvsmhe1QwF/aKth1HdVDi7AvqJfDQsm0Yo/9wK9WeZqQnUVReoy0gPYTDYbhvyRZCA3x46KpU23GUZVo0t5Jv88vZXFjBzDHJeOmKWtWGukUE8uvLe/PlnlLe176xSrWry1JjiA0PYP6aPNtRVDtYsDaPjfnlPHx1qi7uV1o0t5Y5GTmEBvhw/dB421GUB5iRnsyg+HBmv7+d8up623GU8hg+3l7cPCKRr/ceZl/pMdtxVBs6VFHL0x/tZmyPKCYOibMdR7mAFhXNIjJeRHaLSLaI3HeK8/4i8rbz/FoRSXYev0xENojIVud/L2nd+K7hQHkNH207xE0jEgn297EdR3kAby/hyUkDOVrdwJMrdtmOo5RHmToiEV9vYX6mXm3uyH63fBsNTQ4en9hft8pWQAuKZhHxBl4ErgBSgZtE5OSJPbcDR40xPYDngaedxw8D1xhjBgC3AvNbK7grmZeZhzGGGelJtqMoD5LaLYwfnp/C21kFrNlfZjuOUh4jOtSfKwfE8u6GQt2ls4P6aNshVm4v5q5Le5EUGWw7jnIRLbnSPALINsbsN8bUAwuBCSeNmQC87ny8GBgnImKM2WiMOeA8vh0IFBH/1gjuKqrrG3lrXT7j+3clvlOQ7TjKw9w1rhcJnQN5YOlWahuabMdRymPMSE+mqq6RpRuLbEdRrayqtoHfLd9Gn66h/PB87Yal/k9LiuY4oHlTykLnsVOOMcY0AhVA5Eljrge+NcZ0qK7wS74toqKmQdvMKSsC/bx5/LoB7C89zktf7LMdRymPMTQxgn7dwpjvvNOoOo5nV+6mpKqOp64fiK+3Lv1S/6dd/jaISD9OTNn48Xecv0NEskQkq7S0tD0itQqHwzA3I4eB8eEMS+pkO47yUBf0iua6wd34+xfZZJdU2Y6jlEcQEWakJ7G7uIq1OUdsx1GtZEPeUeavyePW9GQGJ0TYjqNcTEuK5iIgodnn8c5jpxwjIj5AOFDm/DweWArMMMac8lKYMeYVY0yaMSYtOtp9ehx/tbeUfaXHmTUmRRcJKKseujqVYH8f7l+yFYduuqBUu7h2UBzhgb66ILCDqGts4v4lW+gaFsCvL+9tO45yQS0pmtcDPUUkRUT8gKnA8pPGLOfEQj+AycAqY4wRkQjgA+A+Y0xGa4V2FXMycuniXBCilE1RIf48cGVf1uce1S1+lWongX7e3JAWz8rthzhUUWs7jjpHL36+jz3Fx3h8Yn9CtBOWOoXTFs3OOcp3AiuBncAiY8x2EZktItc6h70KRIpINnA38K+2dHcCPYBHRGST86NLq78KC7JLqvhqTynTRyXh56NznpR9U4bFM6p7Z55YsZOSSv0HXKn2cMuoJJqM4c11+bajqHOw82AlL32ezXWDu3FJnxjbcZSLalG1Z4xZYYzpZYw5zxjzuPPYI8aY5c7HtcaYKcaYHsaYEcaY/c7jfzDGBBtjBjf7KGm7l9N+5mbk4ufjxc0jE21HUQo4McfyyUkDqW908KBusa1Uu0iKDOaiXtG8tS6f+kaH7TjqLDQ2Ofjtu1sID/TlkWv62Y6jXJheIj0L5dX1vPttIRMHxxEZ0qE66Ck3lxIVzD0/6MUnO4r5p26xrVS7mJGeTGlVHR9tP2Q7ijoLr36Tw5bCCh6b0E+3ylbfS4vms/DWugJqGxzcNjbZdhSl/svtY7szKCGC3y3fTtmxDtXhUSmXdGGvaBI7BzE/M9d2FHWG9pce40+f7OEHqTFcpeuT1Glo0XyGGpoczMvMZUyPSPp0DbMdR6n/4u0lPDt5IFW1DTz6/g7bcZTq8Ly8hOmjklife5QdByptx1Et5HAY7nt3K34+Xvz+Ot0qW52eFs1n6KNthzhYUaubmSiX1ismlF9c0pP3Nx9gpd4yVqrNTUmLJ8DXi/lrcm1HUS20YF0+63KP8PBVqcSEBdiOo9yAFs1naE5GDsmRQVzcu0M0AVEd2E8uOo/U2DAeWraNiuoG23GU6tAigvyYMCiOZRsP6M+bGygqr+GpFTs5v2cUU9LibcdRbkKL5jPwbf5RNuaXM3N0Ml5eehtHuTZfby+emTyQI8fr+f0HOk1DqbY2PT2JmoYm3tmgvdJdmTGGB5ZsxQBPTByg0zJUi2nRfAbmZuQS6u/D5LSE0w9WygX0jwvnfy48j8UbCvlid4fo9qiUy+ofF86wpE68sSZPd+Z0YUs3FvHlnlJ+c3lvEjoH2Y6j3IgWzS10sKKGD7ce5MbhCbpTkHIrPx/Xgx5dQnhgyVaqavW2sVJtaUZ6Erll1Xy1t9R2FHUKJZW1PPb+DoYldWJGerLtOMrNaNHcQvMz83AYw62jk21HUeqM+Pt48+zkgRyqrOWpD3fZjqNUh3ZF/1iiQvyZn5lnO4o6iTGG+5ZspbahiWcmD9RpluqMadHcAjX1Tby5Lp8fpHbVWznKLQ1J7MTtY1NYsDafjOzDtuMo1WH5+Xhx04gEVu0uoeBIte04qpl3NhSyalcJvx3fh/OiQ2zHUW5Ii+YWWLqxiPLqBmaN1TZzyn3dfVlvukcHc+87m6nUaRpKtZmbRybiJcIba/Rqs6s4UF7D79/fwciUzszUO8bqLGnRfBrGGOZk5NA/LozhyZ1sx1HqrAX6efPHKYM4VFnLbN30pEMTkfEisltEskXkvlOcTxSRz0Vko4hsEZErncd9ReR1EdkqIjtF5P72T+/+YsMD+UFqDG9nFVDb0GQ7jsczxvCbxVtoMoZnJw/SaRnqrGnRfBpf7z1MdskxZo1J0bY0yu0NSezETy/qweINhXyyo9h2HNUGRMQbeBG4AkgFbhKR1JOGPQQsMsYMAaYCLzmPTwH8jTEDgGHAj0UkuT1ydzTT05Mor25g+eYDtqN4vDfW5vNN9mEeuLIviZE6xVKdPS2aT2NORg7Rof5cNVD3pFcdwy/G9aRvbBj3L9lC2bE623FU6xsBZBtj9htj6oGFwISTxhggzPk4HDjQ7HiwiPgAgUA9oPtCn4X07pH07BLC/Mw8jNH2c7bkl1XzpHMTk2kjE23HUW5Oi+bvkV1yjC92l3LLyCT8fbxtx1GqVfj5ePGnGwZRUdPAQ8u26T/oHU8c0Hx3jULnseYeBW4RkUJgBfBz5/HFwHHgIJAPPGeMOdKmaTsoEWFGehJbiyrYVFBuO45HcjgMv168GW8Rnr5+oN4tVudMi+bv8drqHPy8vZg2St+dqo6lb2wYv7qsFx9uO6S3jz3TTcBrxph44Epgvoh4ceIqdRPQDUgB7hGR7qf6AiJyh4hkiUhWaan2JD6ViUPjCfH3YZ62n7Ni7upc1uUc4ZFrUukWEWg7juoAtGj+DhXVDby7oYgJg7sRFeJvO45Sre7HF5zH0MQIHl62jUMVtbbjqNZTBDTftjTeeay524FFAMaYTCAAiAJuBj4yxjQYY0qADCDtVN/EGPOKMSbNGJMWHR3dyi+hYwjx9+H6oXF8sOUgh3UqVLvKLjnGMx/t4tK+XZg8LN52HNVBaNH8HRauz6emoYnbxmibOdUxeXsJf7xhMA1Nht++u0WnaXQc64GeIpIiIn6cWOi3/KQx+cA4ABHpy4miudR5/BLn8WBgFKA74pyD6elJ1Dc5eHt9wekHq1ZR3+jgrrc3EuTnzRMTB+i0DNVqtGg+hcYmB6+vziW9eySp3cJO/wSl3FRKVDD3X9mHL/eUsmBtvu04qhUYYxqBO4GVwE5OdMnYLiKzReRa57B7gB+JyGbgLWCmOfGu6UUgRES2c6L4nmuM2dL+r6Lj6NEllDE9IlmwJo/GJoftOB7h+U/3sK2okqeuH0iXsADbcVQH4mM7gCtaub2YAxW1PDahv+0oSrW5W0Ym8cmOYv7wwQ5GdY+kRxfdKcvdGWNWcGKBX/NjjzR7vAMYc4rnHeNE2znViqaPSuYnb2zgs10lXN6vq+04Hdqa/WW8/OU+pg5P0P/XqtXpleZTmJORQ1JkEJf06WI7ilJtzstLeG7KIAJ9vfnlwo3UNepmDEq1pkv7dqFbeADzMnNtR+nQKqobuPvtTSR1DuLhq09uTa7UudOi+SSbCsrZkHeUmaOT8dZdg5SHiAkL4JnJg9h+oJLnVu62HUepDsXH24tpo5LIyC4ju6TKdpwOyRjDQ+9to6Sqjr9MHUKwv95IV61Pi+aTzM3IIcTfR1fbKo9zWWoMt4xK5B9f5/D1Xm0hplRrunF4An7eXszX9nNtYtmmIt7ffIC7Lu3JoIQI23FUB6VFczOHKmr5YMtBbkhLIDTA13Ycpdrdg1em0rNLCHcv2qy7BSrViqJCTuws++63RRyra7Qdp0MpOFLNw8u2Mzy5E/9zUQ/bcVQHpkVzM2+syaPJGGaOTrYdRSkrAv28eeGmIVTUNGgbOqVa2fT0JI7VNbJ048lts9XZamxy8Ku3NyHAn24YrNMqVZvSotmptqGJBWvzuKxvDImRQbbjKGVN39gw7hvfh093lvDGGr2VrFRrGZIQwYC4cOatztU3pK3kz5/uJSvvKL+/rj8JnfXfbtW2tGh2WraxiKPVDcwaq5uZKHXbmGQu6h3NHz7Yye5DunBJqdYgIkxPT2JvyTHW7D9iO47b+3pvKS9+kc2UYfFcNyTOdhzlAbRo5sSq2zkZOaTGhjEypbPtOEpZJyI8O3kQoQG+/OzNbzmuczCVahXXDupGRJCvtp87RyWVtfzq7U30iA7hsQn9bMdRHkKLZiAju4w9xceYNTZFt9tUyik61J8XbhrM/tJjPLRsm95OVqoVBPh6c2NaAh/vKOZgRY3tOG6pyWG46+1NHKtr5MVpQwny0/Zyqn1o0cyJzUyiQvy4ZlCs7ShKuZTR50Vx16W9WLqxiLfXF9iOo1SHcMuoJBzG8KZuXX9W/rYqm9X7ypg9oT+9YkJtx1EexOOL5v2lx1i1q4RpI5Pw9/G2HUcpl3PnxT04v2cUjyzfzvYDFbbjKOX2EjoHcUnvLry1roD6RoftOG4lc18Zf/lsD5OGxDFF91NQ7axFRbOIjBeR3SKSLSL3neK8v4i87Ty/VkSSnccjReRzETkmIn9r3eit47XVufh5ezFtVKLtKEq5JC8v4c83DqZTkC8/W/AtVbUNtiMp5fampydx+FgdH247aDuK2yipquWXCzeSHBXM76/rr9MpVbs7bdEsIt7Ai8AVQCpwk4icvKn77cBRY0wP4HngaefxWuBh4NetlrgVVdQ0sHhDIdcM6kaX0ADbcZRyWZEh/vz1pqEUHK3hvne36vxmpc7RBT2jSY4MYp7uENgiDU0O7lywkcraBl68eahuk62saMmV5hFAtjFmvzGmHlgITDhpzATgdefjxcA4ERFjzHFjzDecKJ5dzqL1BVTXN3HbmGTbUZRyeSNSOnPv5b35YOtB5mbk2o6jlFvz8hJuGZXEhryjbCvSaU+n88SKnazLPcLT1w+kb2yY7TjKQ7WkaI4Dmq8AKnQeO+UYY0wjUAFEtjSEiNwhIlkiklVaWtrSp52TxiYHr63OZWRKZ/rHhbfL91TK3d1xfncuS43h8RU7Wb3vsO04Srm1KcMSCPD1Yr5ebf5eyzYWMTcjl1ljUpgwWPsxK3tcYiGgMeYVY0yaMSYtOjq6Xb7nJzuKKSqv0c1MlDoDXl7Cn24YRHJkEHe+uZHCo9W2IynltsKDfJk4JI73NhdRUa1rBU5lx4FK7luyhREpnbn/yj624ygP15KiuQhIaPZ5vPPYKceIiA8QDpS1RsC2Micjh4TOgVzaN8Z2FKXcSmiAL/+YkUZDo4Mfz99ATX2T7UhKua3po5KpbXDwzgZt6Xiy8up6fvxGFuGBvrx481B8vV3iOp/yYC35G7ge6CkiKSLiB0wFlp80Zjlwq/PxZGCVceGVQlsKy1mfe5SZo1Pw9tLVt0qdqe7RIfzlpsHsOFjJ/Uu26MJApc5Sarcwhid3Yv6aPBwO/Tn6l39tYHKoopaXpg0jOtTfdiSlTl80O+co3wmsBHYCi4wx20Vktohc6xz2KhApItnA3cC/29KJSC7wJ2CmiBSeovNGu5ubkUuwnzdT0rTHo1Jn65I+MdxzWS+WbTrAq9/k2I6jlNuanp5MXlk1X+5tnzU97uCpD3fyxe5SfndNP4YldbIdRykAWtSzxRizAlhx0rFHmj2uBaZ8x3OTzyFfqyuurOWfWw4wbWQSYQG+tuMo5dZ+dnEPthVV8sSKnfTuGsr5PdtnTYJSHcn4fl2JCvFn3upcLu7dxXYc695en88/vs5hRnoSt4xKsh1HqX/zuAlCb6zJo9FhmDk62XYUpdyeiPDcDYPoFRPKT9/4lj3FVbYjKeV2/Hy8uHlkIl/sKSW/zLMX12buK+PBpdtO7EJ6tfUb00r9B48qmmsbmliwNp9xfWJIjgq2HUepDiHE34dXZw4nwM+b2+aup7SqznYkpdzOzSMS8RLhjbWe234u9/Bx/mfBBpIig/jbzUPx0YV/ysV41N/I5ZsOcOR4PbPGJtuOolSHEhcRyKu3plF2vI4fzsvSjhpKnaGu4QGM79eVt9cXeOTPT0VNA7e/vh6AV28dTnigTp9UrsdjimZjDHMycujTNZT07i3ed0Up1UID4yP4y9QhbCks5+5Fm7QTgFJnaHp6EhU1Dby/+YDtKO2qtqGJO+ZlkX+kmpdvGaZ3gpXL8piiOXNfGbsOVTFrbAoi2mZOqbZweb+uPHhlXz7cdoinP9plO45SbmVkSmd6xYTwemaux7RxbHIY7l60ibU5R3huyiBG6UUt5cI8pmiek5FDZLAf1w7qZjuKUh3a7WNTmJGexP9+tZ9/fLXfdhyl3IaIMCM9me0HKvk2v9x2nDZnjOH3/9zBiq2HePDKvrpFtnJ5HlE05xw+zme7Spg2MpEAX2/bcZTq0ESE313Tj6sGxvL4ip0s3lBoO5JSbmPikDhC/X2Yn5lrO0qbe/nL/by2Opcfjk3hRxd0tx1HqdPyiKL59dW5+HiJ9ntUqp14ewnP3zCY83tG8dt3t/DpjmLbkZRyC8H+Pv+/vTsPr6q+8zj+/mZfgIQAgZCFBAEBEVH2pbaVikoZqY8WcVRUUMYWHWuZx6rt+Di2dtrO1La2TK0LZRMQsT7gQl0qVQthh7CqRAgh7ERAFlnzmz/uod4nzVbIvefm8Hk9T56ce+5Jfh9+9/Lly7ln4cbeebyxblegr0Qzd2UFP//zR1x/WXseHd7N7zgiDaQtL2IAABKMSURBVBL4pvnQF6eYs2I7/9KzPdktUvyOI3LBSEqI45nbetMjN4MJM1exdEul35FEmoTbB3bg1BnHS8vL/Y4SEa+v3clDc0sY3KkV//PtnsTF6TwjaRoC3zS/vGI7x06e4a7BRX5HEbngpCcn8Mc7+5LXMpW7p65gzfbgH6cpcr4uatOMIZ1a8+LSck6fqfI7TqN6e8NuHpi9ht4dWvLcmD4kJ+iQSWk6At00n6lyTFlcRr/CLC7Ny/A7jsgFKSs9iRl39yerWRK3v7CUEjXOEWdm15rZx2ZWamYP1/B8gZktNLPVZrbWzIaHPdfTzIrNbIOZrTMzfUTngzEDO7Dr0HHe3RScQ5sWfryXCTNXcWluBpPv7EtaUoLfkUT+KYFumt/ZuIeKA1/oZiYiPsvJSGXWPQNomZbEbWqcI8rM4oFJwHVAd+AWM6t+P+IfAXOcc5cDo4H/8342AZgB3OucuwT4GnAqStElzNBubcnNTGXq4mDcIXBR6X7unb6Si9s1Z+rYfjRP0c1LpOkJdNM8edFW8lqmcnX3dn5HEbngtc9MZdb4AWSmJXLbC0tZW6HGOUL6AaXOuS3OuZPAbGBktW0c0MJbzgDO3k1jGLDWOVcC4JyrdM5deLeniwHxccatAwoo3lLJ5j2H/Y5zXj7cvI9xU5dT2Cqd6WP7625/0mQFtmlev+MQy7Z+xp2DConXSQYiMSE3M7THOSM1kdueX8rq8gN+RwqiXGB72OMKb124x4HbzKwCeBO431vfBXBm9paZrTKzhyIdVmp3c598kuLjmL6k6e5tfmfjHsZNWUFhq3RevKc/LdOT/I4kcs4C2zRPXrSV9KR4RvXN9zuKiITJa5nGrHsGkJmWxK3PL+XDzfv8jnQhugWY4pzLA4YD080sDkgAhgC3et9vMLOhNf0CMxtvZivMbMW+fXoNI6FVs2RG9MzhlZUVHD7e9I6SmV+yk3tnrKRb+xbMHj+A1s2S/Y4kcl4C2TTvPXyc10p2clPvPFrouCmRmJOflcbcewdSkJXG2CnLeXPdLr8jBckOIHxvQZ63Ltw4YA6Ac64YSAFaE9or/YFzbr9z7hihvdBX1DSIc+5Z51wf51yfNm3aNPIfQc4aM6iQoyfP8Orq6i9hbJu5tJwHZq+md4eWzBjXj8w07WGWpi+QTfOMJeWcOuO4U5eZE4lZ2S1SeGn8QC7Ly2TCzFXMXBrMa9L6YDnQ2cyKzCyJ0Il+86ttUw4MBTCzboSa5n3AW8ClZpbmnRT4VWBj1JLLP+iVn0nPvAymFW/DOed3nHo55/jftz7m0VfX8dUubZh6l076k+AIXNN8/NQZXlyyjaFdsylqne53HBGpQ0ZaItPH9edrXdrw6Kvr+O8Fm6iqiv3GIJY5504D9xFqgDcRukrGBjN7wsyu9zabCNxjZiXALOBOF3IAeIpQ470GWOWceyP6fwoJN2ZgIaV7j1D8aWzfIOjk6SomzinhdwtLGd03n+fG9CE1SddhluAI3EUSXyvZSeXRk4wdor3MIk1BalI8z43pw+OvbeAP72+hbP9RfnVzL13D9Tw4594kdGhF+LrHwpY3AoNr+dkZhC47JzFiRM8cnnxjI9OKtzGoU2u/49To4LGTTJi5ikWllUy8ugv3XdUJM52EL8ESqD3NzjkmLyrj4rbNGXRRK7/jiEgDJcTH8eORPXhsRHfe3riHUX8oZtehL/yOJRITUhLjublvAW9v3M3Og7H39+Kj3Z9z/e8WsWzrZ/zy25dx/9DOapglkALVNC/Z8hmbdn3O2CGF+gsr0sSYGWOHFPH8mD5s3XeUEU//jUWl+/2OJRITbu1fgIOYO/b/jbW7uGHSYo6fOsPs8QO5sXee35FEIiZQTfPkRVvJSk9iZK/qlyQVkaZiaLe2zLtvMC3TQ3cP/O1fNus4Z7ng5WelMbRrNrOWlXPitP/3mzl5uoqfvrmJCTNX0S2nOa/fP4TeHVr6HUskogLTNG+rPMq7m/Zwa/8CUhJ14oFIU9YpuznzJgzm+sva88t3PmHs1OXsPXzc71givhozsJDKoydZsG63rzm27DvCjb9fzLMfbOHW/gXMGj+A7BYpvmYSiYbANM1TFpcRb8ZtAzr4HUVEGkF6cgK/vrkXPx55CYs/reSaX33AAl3PWS5gQzq1pqh1OtOKy3wZ3znHnBXbGfHbv7H9wDGeua03T95wKckJ2lElF4ZANM2Hj5/i5RUVjOiZQ1v9b1ckMMyM2wcW8sb9Q8hrmcZ3XlzFgy+t4dCxpnd3NJHzFRdn3D6gA6vKD7J+x6Gojr3j4BeMnbKch+au5bK8TBY88BWu7dEuqhlE/BaIpnnOigqOnDity8yJBFTnts3503cH8cDQzswv2cnQp/7KKysrmsTNHkQa042980hNjI/a3uYzVY7Jf9vK1U+9z9Ktn/GfI7oz4+7+5GSkRmV8kVgSiKa5qspxVddseuZl+h1FRCIkMT6OB6/uwrwJg8nPSmPiyyWM+kMxm3Z97nc0kajJSE3kW5fnMm/NTg4cPRnRsYo/reRbkxbxxOsb6V+UxdsPXsm4IUXEx+nqVHJhCkTTfM+VHXnhjj5+xxCRKOiRm8Er9w7iFzf25NN9R/nm0x8ycU4J5ZXH/I4mEhVjBnbgxOkqXl65PSK/f/Oew4ybspxbnltC5ZETPH3L5Uy+sy95LdMiMp5IUxGYW27puswiF464OGNU33yGXdKWSQtLmVa8jXlrdjCqbz4Tvt6J3Ex9dCzB1S2nBf0Ks5i+ZBvjhnRstD2/H+8+zDPvf8r8kp2kJcbzg2u7ctfgQl2RSsQTmKZZRC48mWlJ/PCb3bn7Kx2ZtLCUWcvKeWn5dq7r0Y6xQ4q4okDXjZVgGjOoA/fNXM37n+zlqq5tz/n3VFU5Fn26n6mLy3h3017SkuK5a1Ah3/16J7LSkxoxsUjT16Cm2cyuBX4DxAPPO+d+Vu35ZGAa0BuoBG52zpV5zz0CjAPOAP/unHur0dKLiABtW6TwxMge/NtXL2Lq4jJmLSvn9bW76JbTghuvyOX6Xu3Jbq4r60hwXHNJO7KbJzOteNs5Nc3llcd4be1OXlq+nfLPjtEyLZHvfaMzdwwspKWaZZEa1ds0m1k8MAm4GqgAlpvZfOfcxrDNxgEHnHOdzGw08HPgZjPrDowGLgHaA++aWRfnnP+3MxKRwMnNTOXR4d14YGhn/rSqgrkrK/jJG5v46Zub6FOYxVVds/n6xdl0adtMh3RJk5YYH8ct/Qr4zV82U7b/KIWt0+vc/vSZKtbuOMTi0v38ecNu1u8InUDbvyiLicO6cM0l7XQYhkg9GrKnuR9Q6pzbAmBms4GRQHjTPBJ43FueC/zOQv8ijQRmO+dOAFvNrNT7fcWNE19E5B+lJydw+8BCbh9YSOneI8xbs4N3N+3lZws+4mcLPiIzLZHL8zO5LD+TTtnNKGqdTkFWGs2SE9RMS5Pxr/0LmLSwlBlLtvGjEd2B0OEWlUdPsvvQcbbsP8LGnZ+zcdfnrC4/yJETpwHolZ/Jo8O7cl2PHPKzdHKfSEM1pGnOBcJP0a0A+te2jXPutJkdAlp565dU+9ncc04rIvJP6pTdjInDLmbisIvZdegLPvxkPyu3HWD19gP89ZN9hF/qOTHeyEhNokVKAiN65vD9YRf7F1ykHm1bpHBNj3ZMK97GgvW7+eLUGQ4fP8WpM1++qZPi4+iU3Yzre7Vn8EWtGdAxi1bNkn1MLdJ0xcSJgGY2HhgPUFBQ4HMaEQmqnIxURvXNZ1TffACOnTxN2f5jlFUepeLAMQ4cO8XBYyc5fPw0bTN0DLTEvge/0ZnEOCMhPo6UxDiapySSk5FCuxYpFLRK46I2zUiMD8TVZUV815CmeQeQH/Y4z1tX0zYVZpYAZBA6IbAhP4tz7lngWYA+ffroFl8iEhVpSQl0b9+C7u1b+B1F5Jx0ym7Or0df7ncMkQtCQ/77uRzobGZFZpZE6MS++dW2mQ/c4S3fBLznQve3nQ+MNrNkMysCOgPLGie6iIiIiEh01Lun2TtG+T7gLUKXnJvsnNtgZk8AK5xz84EXgOneiX6fEWqs8babQ+ikwdPABF05Q0RERESamgYd0+ycexN4s9q6x8KWjwPfruVnnwSePI+MIiIiIiK+0tkBIiIiIiL1UNMsIiIiIlIPNc0iIiIiIvVQ0ywiIiIiUg81zSIiIiIi9TDnYuteIma2D9h2Dj/aGtjfyHHOh/LULpaygPLUJ5byxFIWqDlPB+dcGz/C+OUc63ZTeC39pDy1i6UsoDz1iaU851WzY65pPldmtsI518fvHGcpT+1iKQsoT31iKU8sZYHYy9OUxNrcKU/dYilPLGUB5alPLOU53yw6PENEREREpB5qmkVERERE6hGkpvlZvwNUozy1i6UsoDz1iaU8sZQFYi9PUxJrc6c8dYulPLGUBZSnPrGU57yyBOaYZhERERGRSAnSnmYRERERkYgIRNNsZtea2cdmVmpmD/swfpmZrTOzNWa2wluXZWbvmNlm73vLCI4/2cz2mtn6sHU1jm8hT3tztdbMrohSnsfNbIc3R2vMbHjYc494eT42s2saOUu+mS00s41mtsHMHvDW+zI/deTxa35SzGyZmZV4ef7LW19kZku9cV8ysyRvfbL3uNR7vjBKeaaY2daw+enlrY/G+znezFab2eveY1/mJkj8rtleBtXturP4UpO83x8zdVs1+5zzBLNmO+ea9BcQD3wKdASSgBKge5QzlAGtq637BfCwt/ww8PMIjn8lcAWwvr7xgeHAAsCAAcDSKOV5HPiPGrbt7r1myUCR91rGN2KWHOAKb7k58Ik3pi/zU0cev+bHgGbeciKw1PtzzwFGe+ufAb7jLX8XeMZbHg281MjzU1ueKcBNNWwfjffz94GZwOveY1/mJihfxEDN9nKUobpdVxZfapI3RszU7Tqy+DI/ddRI1ewvx4hYzQ7CnuZ+QKlzbotz7iQwGxjpcyYIZZjqLU8FvhWpgZxzHwCfNXD8kcA0F7IEyDSznCjkqc1IYLZz7oRzbitQSug1bawsu5xzq7zlw8AmIBef5qeOPLWJ9Pw459wR72Gi9+WAq4C53vrq83N23uYCQ83MopCnNhF9vcwsD/gm8Lz32PBpbgIkVms2XKB1O5ZqtpcnZuq2avY556lNk67ZQWiac4HtYY8rqPsNHQkOeNvMVprZeG9dW+fcLm95N9A2yplqG9/P+brP+zhmsn35sWfU8ngfvVxO6H/Cvs9PtTzg0/x4H2WtAfYC7xDaM3LQOXe6hjH/nsd7/hDQKpJ5nHNn5+dJb35+ZWbJ1fPUkLUx/Bp4CKjyHrfCx7kJiFio2aC63RC+1myIrbqtmt2wPEGu2UFommPBEOfcFcB1wAQzuzL8SRfa9+/bZUr8Ht/ze+AioBewC/hlNAc3s2bAK8D3nHOfhz/nx/zUkMe3+XHOnXHO9QLyCO0R6RqtsRuSx8x6AI94ufoCWcAPIp3DzEYAe51zKyM9lvhCdbtuvtZsiK26rZrd8DxBrtlBaJp3APlhj/O8dVHjnNvhfd8LvEroTbzn7EcO3ve90cxUx/i+zJdzbo/3F6sKeI4vP66KeB4zSyRU7F50zv3JW+3b/NSUx8/5Ocs5dxBYCAwk9JFZQg1j/j2P93wGUBnhPNd6H5E659wJ4I9EZ34GA9ebWRmhQwiuAn5DDMxNE+d7zQbV7fr4XZNiqW6rZv/TeQJbs4PQNC8HOntnRyYROph7frQGN7N0M2t+dhkYBqz3MtzhbXYHMC9amTy1jT8fGOOdwToAOBT2cVfEVDtm6QZCc3Q2z2jvLNYioDOwrBHHNeAFYJNz7qmwp3yZn9ry+Dg/bcws01tOBa4mdMzeQuAmb7Pq83N23m4C3vP2+EQyz0dh/1AaoePRwucnIq+Xc+4R51yec66QUF15zzl3Kz7NTYD4WrNBdbsh/KpJ3tgxU7dVs88pT3Brtmvksxb9+CJ0NuYnhI7r+WGUx+5I6EzZEmDD2fEJHRfzF2Az8C6QFcEMswh9PHSK0PE642obn9AZq5O8uVoH9IlSnuneeGu9N2pO2PY/9PJ8DFzXyFmGEPoIby2wxvsa7tf81JHHr/npCaz2xl0PPBb2vl5G6CSWl4Fkb32K97jUe75jlPK8583PemAGX56tHfH3szfO1/jyTGxf5iZIX/hYs8NeQ9XturP4UpO83x8zdbuOLKrZdecJZM3WHQFFREREROoRhMMzREREREQiSk2ziIiIiEg91DSLiIiIiNRDTbOIiIiISD3UNIuIiIiI1ENNs4iIiIhIPdQ0i4iIiIjUQ02ziIiIiEg9/h8KQC0o9Hih7AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.recorder.plot_lr(show_moms=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide_input": true
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<h2 id=\"OneCycleScheduler\"><code>class</code> <code>OneCycleScheduler</code><a href=\"https://github.com/fastai/fastai/blob/master/fastai/callbacks/one_cycle.py#L9\" class=\"source_link\">[source]</a></h2>\n",
       "\n",
       "> <code>OneCycleScheduler</code>(`learn`:[`Learner`](/basic_train.html#Learner), `lr_max`:`float`, `moms`:`Floats`=`(0.95, 0.85)`, `div_factor`:`float`=`25.0`, `pct_start`:`float`=`0.3`) :: [`Callback`](/callback.html#Callback)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_doc(OneCycleScheduler, doc_string=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a [`Callback`](/callback.html#Callback) that handles the hyperparameters settings following the 1cycle policy for `learn`. `lr_max` should be picked with the [`lr_find`](/train.html#lr_find) test. In phase 1, the learning rates goes from `lr_max/div_factor` to `lr_max` linearly while the momentum goes from `moms[0]` to `moms[1]` linearly. In phase 2, the learning rates follows a cosine annealing from `lr_max` to 0, as the momentum goes from `moms[1]` to `moms[0]` with the same annealing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide_input": true
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<h4 id=\"OneCycleScheduler.steps\"><code>steps</code><a href=\"https://github.com/fastai/fastai/blob/master/fastai/callbacks/one_cycle.py#L21\" class=\"source_link\">[source]</a></h4>\n",
       "\n",
       "> <code>steps</code>(`steps_cfg`:`StartOptEnd`)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_doc(OneCycleScheduler.steps, doc_string=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build the [`Stepper`](/callback.html#Stepper) for the [`Callback`](/callback.html#Callback) according to `steps_cfg`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide_input": true
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<h4 id=\"OneCycleScheduler.on_train_begin\"><code>on_train_begin</code><a href=\"https://github.com/fastai/fastai/blob/master/fastai/callbacks/one_cycle.py#L26\" class=\"source_link\">[source]</a></h4>\n",
       "\n",
       "> <code>on_train_begin</code>(`n_epochs`:`int`, `kwargs`:`Any`)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_doc(OneCycleScheduler.on_train_begin, doc_string=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initiate the parameters of a training for `n_epochs`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide_input": true
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<h4 id=\"OneCycleScheduler.on_batch_end\"><code>on_batch_end</code><a href=\"https://github.com/fastai/fastai/blob/master/fastai/callbacks/one_cycle.py#L39\" class=\"source_link\">[source]</a></h4>\n",
       "\n",
       "> <code>on_batch_end</code>(`train`, `kwargs`:`Any`)"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_doc(OneCycleScheduler.on_batch_end, doc_string=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepares the hyperparameters for the next batch."
   ]
  }
 ],
 "metadata": {
  "jekyll": {
   "keywords": "fastai",
   "summary": "Implementation of the 1cycle policy",
   "title": "callbacks.one_cycle"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}