{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tuning Learning Rates with *ktrain*\n",
    "\n",
    "Neural networks have many hyperparameters that need to be set before training begins. While, in practice, many hyperparameters have fairly reasonable defaults (e.g., ReLu activation, Xavier initialization, a kernel size of 3 in Convolutional Neural Networks), some do not and should be tuned. One of these is the learning rate, which governs the degree to which weights are adjusted during training.  Even after arriving at a good initial learning rate, it has been shown that varying the learning rate during training is effective in helping to minimize loss and improve generalization. *ktrain* provides a number of built-in methods to make it easy to tune and adjust learning rates to more effectively minimize loss during training.\n",
    "\n",
    "To demonstrate these capabilities, we will begin by loading some text data into NumPy arrays and defining a simple text classification model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "import os\n",
    "os.environ[\"CUDA_DEVICE_ORDER\"]=\"PCI_BUS_ID\";\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"]=\"0\"; \n",
    "import ktrain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load  and prepare data as you normally would in Keras\n",
    "from tensorflow.keras.preprocessing import sequence\n",
    "from tensorflow.keras.datasets import imdb\n",
    "NUM_WORDS = 20000\n",
    "MAXLEN = 400\n",
    "def load_data():\n",
    "    (x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=NUM_WORDS)\n",
    "    x_train = sequence.pad_sequences(x_train, maxlen=MAXLEN)\n",
    "    x_test = sequence.pad_sequences(x_test, maxlen=MAXLEN)\n",
    "    return (x_train, y_train), (x_test, y_test)\n",
    "(x_train, y_train), (x_test, y_test) = load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# build a model as you normally would in Keras\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Embedding, GlobalAveragePooling1D\n",
    "def get_model():\n",
    "    model = Sequential()\n",
    "    model.add(Embedding(NUM_WORDS, 50, input_length=MAXLEN))\n",
    "    model.add(GlobalAveragePooling1D())\n",
    "    model.add(Dense(1, activation='sigmoid'))\n",
    "    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
    "    return model\n",
    "model = get_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use *ktrain*, one simply wraps the model and the data in a Learner object using the ```get_learner``` function.  This Learner object will be used to help tune and train our network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "learner = ktrain.get_learner(model, train_data=(x_train, y_train), val_data = (x_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The wrapped model and data are both directly accessible.  For instance, the model can be saved and loaded like normal in Keras (e.g,. ```learner.model.save('my_model.h5')```).  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A Learning Rate Finder\n",
    "\n",
    "The Learner object can be used to find the best learning rate for your model.  First, we use ```lr_find``` to track the loss as the learning rate is increased and then use ```lr_plot``` to identify the maximal learning rate associated with a falling loss (both methods were adapted from the [fastai library](https://github.com/fastai/fastai)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "simulating training for different learning rates... this may take a few moments...\n",
      "Epoch 1/5\n",
      "25000/25000 [==============================] - 5s 197us/step - loss: 0.6932 - acc: 0.4902\n",
      "Epoch 2/5\n",
      "25000/25000 [==============================] - 5s 183us/step - loss: 0.6926 - acc: 0.5456\n",
      "Epoch 3/5\n",
      "25000/25000 [==============================] - 5s 189us/step - loss: 0.5968 - acc: 0.7288\n",
      "Epoch 4/5\n",
      "25000/25000 [==============================] - 5s 182us/step - loss: 0.3167 - acc: 0.8695\n",
      "Epoch 5/5\n",
      " 8032/25000 [========>.....................] - ETA: 3s - loss: 0.5864 - acc: 0.8430\n",
      "\n",
      "done.\n",
      "Please invoke the Learner.lr_plot() method to visually inspect the loss plot to help identify the maximal learning rate associated with falling loss.\n"
     ]
    }
   ],
   "source": [
    "learner.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEOCAYAAABmVAtTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3XmYXGWZ9/HvXdV7p5cs3dlDJ5AAIexhRxZlR0EFEUYdEBVRUQfRV1BHFtHRYXBGWVQEFVlERGUCQQEFJmyBJBAISUhIQhISsnX2dCe9VD3vH+dUdXWneu9Tp6rr97mups9Wde4uKnXXs5tzDhEREYBI2AGIiEj2UFIQEZEkJQUREUlSUhARkSQlBRERSVJSEBGRJCUFERFJUlIQEZEkJQUREUlSUhARkaSCsAPorREjRri6urqwwxARySnz5s2rd87VdHddziWFuro65s6dG3YYIiI5xcxW9eS6QKuPzOwsM1tiZsvM7No05//bzOb7P0vNbFuQ8YiISNcCKymYWRS4AzgdWAPMMbMZzrlFiWucc1enXP9V4PCg4hERke4FWVI4GljmnFvhnGsGHgLO7+L6S4A/BBiPiIh0I8ikMBZ4L2V/jX9sL2a2DzAReCbAeEREpBvZ0iX1YuAR51ws3Ukzu8LM5prZ3E2bNmU4NBGR/BFkUlgLjE/ZH+cfS+diuqg6cs7d5Zyb7pybXlPTbY8qERHpoyC7pM4BJpvZRLxkcDHwLx0vMrMDgKHAywHGQv2uJtZs3U0sHqe51RH3lyG15H/AMMy83da4w4DCgggFESNi3kWJxUsTy5jGHcSdIxZ3RMyIO+9xZt5zJZ4+sZ24WftzlnIm9Vz7azs719n1gdw35TFd3ZeuzvnP09l9sfTXQ9vrbmYURLyfaMSS50WkfwJLCs65VjO7CngSiAK/cc4tNLObgLnOuRn+pRcDD7mAF4t+ZN4afvy3t4O8hYSoMOonB4yCqFEUjVAQNQqjEYqiEQqjEQoLvP3CiLddFI1QXBClpDBCSWGU4oK238X+7/LiAqpLC6kqLaSytJDqMm97SHGBEpEMShbwZ/GAmz59uuvL4LWV9Q0s37SLgmjE+wAxS/nWDy6x57zSQEHEO98Si9Mac23n6fCNFiiIRIiYV2qImPd4l3w6l7KduJ9r20+5jtTHpTym3fUp50j73N3ft+N1Pb0vXT535+dIE39nf1PqfdPFmijNxZ2jNe5oaXXE4nFa4l5pzSWOx+K0tPq/446W1jgtsTjNMf93a5yWmKOpNcaelnjy956WGE2tcbpTGDWqSosYWVlM3fByyoujFBVEGFpWxJjqUsZWlxKNGBUlBew/qoLigmi3zykSJDOb55yb3t11OTeiua/qRpRTN6I87DAkBzjnaGqN09Qap7G5lW2NLWzf3cK2xhZ27G5h2+5mtjS0sH13M+u272Hh+9tpaI7RGouzY08rsXj7pBYxGD+sjH1rhjB+aClH7DOU4yYNp7ayJKS/UKRzeZMURHrKzCgpjFJSGKWqtJDRVaU9fmxrLM6GnU2s3bqbWNyxpaGZJRt2snzTLpZv3MUrKzZz78vebAP71pRz8pRaLjhyLPvVDlFpQrJC3lQfiWSDWNyx6P0dvLyinuff8X4AyoqiXHjkOP71uDr2qx0ScpQyGPW0+khJQSREa7ft5h+LNvD0og3MXrGZ1rjj7GmjuPmj0xg+pDjs8GQQUVIQyTEbd+7hgdmruf3ZZUQjxulTR3L9R6ZSW6G2B+m/niaFbBnRLJL3aitKuPr0KTx21YlcctR4nl60gaN/+E9eWl4fdmiSR5QURLLM1DGV3Hj+NP70xeMA+OJ985izckvIUUm+UFIQyVKHjq9mxlUn0BpzfPJXL/PfTy/dawyHyEBTUhDJYoeMq+af15zMqfvX8rN/vsOV989j++6WsMOSQUxJQSTLjaku5df/Op3vnHMATy7cwDUPz6cl1v2oa5G+UFIQyQGRiHHFSfvy3XMO5B+LN3Lns8vDDkkGKSUFkRzy+Q9M5Oxpo/jVrOVsbWgOOxwZhJQURHKImfFvp02hsTnGb158N+xwZBBSUhDJMfuPquDcQ0ZzzwvvsnOPGp1lYCkpiOSgy46vo7E5xv2zV4cdigwySgoiOejICUM5aEwltz61hC1qW5ABpKQgkoMiEeMnFxxCa9zxxIJ1YYcjg4iSgkiOOmhMJQeOruSBV1ZrpLMMGCUFkRxlZnz62AksXreD11ZvCzscGSSUFERy2EcPG8uQ4gJ1T5UBo6QgksPKiwu48MhxzHxzHcs37Qo7HAnItsZmzvqfWcx8M/j2IyUFkRz3qWMmAPDs2xtDjkSC0tAc4+31O2loag38XkoKIjkusabzzTMX09gc/IeGZN7u5hgAJUXRwO+lpCCS48yMjx8+FoDLfjsn5GgkCHtavKRQWqikICI9cOtFhwKwsr4h5EhkIK3bvpvv/nUBO/w1NEoKg//IVlIQGQTMjJvOP4iNO5tYvbkx7HBkgNw4YxEPvLKaJxeuB6BEJQUR6anj9x0BwIvL60OORAZKxP+E3uk3MBdELPh7Bn4HEcmIfWvKGVZexOurt4YdigwQMy8JxOLeiPWCiKqPRKSHzIyDx1bx5prtYYciAyRRLkisyx1VSUFEeuPQ8dW8vX4nNz++KOxQZAC8t8VrH3puySZASUFEeumy4+sAuPuFdzVJ3iBw5D7DABhZWQwoKYhILw0rL+KGj0wFYNG6HSFHI/2VmNOqvLgAUEOziPTBiZO9XkizlqoX0mCxYpM3/kQlBRHptf1qKxhWXsTqLRrIlsvi8b2r/5QURKRPJo4oT367lNwUS9MmpOojEemTSSPKWb5plxqbc1gsTUkhoqQgIn0xvW4o9buaWbJhZ9ihSB/FVVIQkYFyzMThAMzXMp05a1CWFMzsLDNbYmbLzOzaTq65yMwWmdlCM3swyHhE8sWEYWUAXPuXBSFHIn0Vj3u/U0sHOV1SMLMocAdwNjAVuMTMpna4ZjJwHXCCc+4g4N+Cikckn6R+o3xHVUg5KdHQPGJIcfJYSUFuz5J6NLDMObfCOdcMPASc3+GaLwB3OOe2AjjntJ6gyAD536+cAMAzWqYzJ7X6RYXUJThzvfpoLPBeyv4a/1iqKcAUM3vRzGab2VkBxiOSVw4dX83EEeXMW6VZU3PRnc8uB9qmzc6UgozeLf39JwOnAOOAWWZ2sHOuXeuYmV0BXAEwYcKETMcokrP2H1nB4vWa7iIXzV6xOZT7BllSWAuMT9kf5x9LtQaY4Zxrcc69CyzFSxLtOOfucs5Nd85Nr6mpCSxgkcHmgNEVrNrcmFzjV3JHWENMgkwKc4DJZjbRzIqAi4EZHa55FK+UgJmNwKtOWhFgTCJ5pW54OdA2BbPkjkRDc2kGluBMFVhScM61AlcBTwKLgYedcwvN7CYzO8+/7Elgs5ktAp4FvuWcC6fMJDII7TPc65r6l9c7FtIl2yXmPvrQgbUZvW+gbQrOuSeAJzoc+37KtgO+4f+IyAA7cHQlAOu37wk5EumtxIjmxCR4x0wclpH7ht3QLCIBKimMMmlEOU2talPINfW7mpPbC244g+IMjFEATXMhMugdMLqCF96pZ3ezEkMu2eV3Rf3WmftTUVJIUUFmPq6VFEQGuTMPGsWOPa28/p7GK+SicUPLMno/JQWRQe6ICUMB9UCSnlFSEBnkRleVAHD/7NUhRyK5QElBZJAriHr/zBes3R5yJNJbdcMzW3UESgoieeETR46jtqK4+wsla0QjxrmHjM74fZUURPLAhGFlbNzZ1G7GTclesbgjFncURTM7mhmUFETywkFjvUFsb65RFVIuaG71ps3OVDfUVEoKInng4LHVALytGVNzgpKCiARqxJAiSgojrNm6O+xQpAeaYt5AQyUFEQmEmRGPwz0vvBt2KNIDiZJCcVRJQUQC0hzzPmg27tDkeNmuJeZNhqeSgogE5pYLDwFg8fqdIUci3XlxWT0ADc2Z7y2mpCCSJ06fOhKAt9epsTnbNfnVR4eOq874vZUURPJEdVkRw8qLeFMjm7NeRYm3qkF1WWHG762kIJJHtjQ0M/PNdWGHId1o9dsUCiJqUxCRAE3zB7FJdovFveqjxKprmaSkIJJHTj9wFIBWYstyrf76zIVRJQURCdCoKm9SvI07mkKORLryyootgEoKIhKwkZXe2gobNFYha/1twTr+vnA94K2xnWlKCiJ5ZHRVKQDrlRSy1pceeC25XaCSgogEaZRfUli/XUkhF5gpKYhIgCpLCygrirJ2mybGk/SUFETyiJmxb80Qlm3cFXYokqWUFETyzOSRQ3hng5KCpKekIJJnJtdWsH7HHnbsaQk7FEljbLXXGeCeS6eHcn8lBZE8M7l2CIBKC1nqQwfW+r9HhnJ/JQWRPDNlZAUAF/ziJVr8NRYke8SdY3h5UWj3V1IQyTNjh5Ymt59YoMnxsk0sHk5X1AQlBZE8E40Y3zxjCgDz39sWcjTSkXOOEFbhTFJSEMlDV31wMqcdOJLfvrhSk+NlmbhzRFRSEJFMmzrGm0Z79ebGkCORVLE4SgoiknnHTBwGwOaG5pAjkVTOOULMCUoKIvkqsdTjJb+eHXIkkiruXChTZicoKYjkqQNHedVHzqG5kLJI3Kn6SERCEIkYh46rAuCbD78RcjSSEFP1kYiE5feXHwPA7hb1QMoWbjD3PjKzs8xsiZktM7Nr05y/zMw2mdl8/+fzQcYjIu1VlRVyVN3QUNYClvTicYgOxqRgZlHgDuBsYCpwiZlNTXPpH51zh/k/dwcVj4ikN3lkBUs37MI5F3YogtfQPFirj44GljnnVjjnmoGHgPMDvJ+I9MGU2iFs393Cxp1NYYciDO7Ba2OB91L21/jHOrrAzN40s0fMbHyA8YhIGokJ8pZu2BlyJAJ+76M8nubiMaDOOXcI8DRwb7qLzOwKM5trZnM3bdqU0QBFBrspo7ykcONji0KORABaYnEKQ5z8qEd3NrOvm1mlee4xs9fM7IxuHrYWSP3mP84/luSc2+ycS5RZ7waOTPdEzrm7nHPTnXPTa2pqehKyiPTQiCHFAFqiM0s0tcYpLsjypABc7pzbAZwBDAU+A/y4m8fMASab2UQzKwIuBmakXmBmo1N2zwMW9zAeEZFByUsK0dDu39OkkGj1OAe4zzm3MOVYWs65VuAq4Em8D/uHnXMLzewmMzvPv+xrZrbQzN4AvgZc1ts/QET6LzGV9h6NVwhdU0ss1JJCQQ+vm2dmTwETgevMrALodskm59wTwBMdjn0/Zfs64LqehysiQUhUIW1uaE6uESzhaG6NU5QDSeFzwGHACudco5kNAz4bXFgikklj/ESwZkujkkLIcqX66DhgiXNum5l9GvgesD24sEQkk+qGlwPwzUc0B1LYmlrjFBdmf0PzL4BGMzsUuAZYDvw+sKhEJKPGVJcA8N4WzZYatqbWcNsUenrnVueNgT8fuN05dwdQEVxYIpJJBdEIB/jjFWJxTXcRpqaQ2xR6euedZnYdXlfUmWYWAQqDC0tEMu3saV4PcY1sDldrLE5hiEOae3rnTwJNeOMV1uMNRLslsKhEJOPKi73GzVueXBJyJPnLOedPc5Hlcx/5ieABoMrMPgzscc6pTUFkEPnMcfsAMGFYWciR5K9EzV3WT51tZhcBrwKfAC4CXjGzC4MMTEQyK9EN8ncvrQw3kDyWaM8pCHF9i56OU/gucJRzbiOAmdUA/wAeCSowEQmPcw4Lc1L/PJVICrkwdXYkkRB8m3vxWBHJEV88eRIA9buaQ44kP8X8hY5CnCS1xx/sfzezJ/3lMy8DZtJh+goRyX3HThoOwMrNDSFHkp9ypqTgnPsWcBdwiP9zl3Pu20EGJiKZN9Ef2byyXkkhDPF4oqSQ/W0KOOf+DPw5wFhEJGS1ld7EeOu37wk5kvzUVn2UpSUFM9tpZjvS/Ow0sx2ZClJEMqO00OuBdOvTS0OOJD/Fs6D6qMuSgnNOU1mI5BH1OApX1pcURCT/fOmUfQHNgRSG1piSgohkmQL/A+mFZfUhR5J/4omSQrb3PhKR/HHK/rUAPPbG+yFHkn9iWdD7SElBRNo5cp+hADwyb03IkeSfREkh6yfEE5H81NDUGnYIeSUW936r+khEssovPnUEAO9qEFtGtVUfhReDkoKI7GVUlbc854YdGsSWScnqI5UURCSbjKkuBWDtNq3ZnEnZMHW2koKI7KW2opghxQUs27gr7FDySmsWjGhWUhCRvZgZ+9UO4Z0NSgqZFNeIZhHJVlNGDuGdjTvDDiOvJBuaVVIQkWwzubaC+l3NbG3QgjuZonEKIpK19q311lZYUa8qpEzxc4LaFEQk+0waMQSA5Zs0ViFT2lZeCy8GJQURSWvcUK9b6u3PLAs5kvyh6iMRyVoF/rDa1VsaWaYG54xQ9ZGIZLXESmzvbdEgtkxQ9ZGIZLVnvnkyAKs2q10hEzTNhYhktVGV3hxINzy2KLl+sAQnruojEclmqWs2Xz9joZboDFhbQ3N4MSgpiEiXfnLBwQDcN3sVL2qJzkBpOU4RyXrnHTo2ub25oSnESAa/REHMlBREJFuVFkX55hlTAHh/m9ZXCFJ8sPc+MrOzzGyJmS0zs2u7uO4CM3NmNj3IeESkb6764GSGlxexZqu6pgZpUM+SamZR4A7gbGAqcImZTU1zXQXwdeCVoGIRkf4bXV3C+u1KCkEa7L2PjgaWOedWOOeagYeA89Nc9wPgJ4DKpSJZbFRlCeu2659pkBLVRyHmhECTwljgvZT9Nf6xJDM7AhjvnJvZ1ROZ2RVmNtfM5m7atGngIxWRbo2qKuHt9Zruois79rRw70srca5vXXcHdfVRd8wsAvwUuKa7a51zdznnpjvnptfU1AQfnIjsJdHIfNs/3wk5kux18+OLuH7GQv40b02fHj/Yq4/WAuNT9sf5xxIqgGnAc2a2EjgWmKHGZpHstGN3CwC3Pr2Uhe9vDzma7LRpp9dl9/898iYr63s/NUjMDe7qoznAZDObaGZFwMXAjMRJ59x259wI51ydc64OmA2c55ybG2BMItJHP/jotOT2wrU7Qowke52w34jk9hd+3/uPspsfXwQM0pKCc64VuAp4ElgMPOycW2hmN5nZeUHdV0SCccCoiuT2jj0tIUaSvSpLCpPb72zs3Yp1/7d0E02tcSDcEc0FQT65c+4J4IkOx77fybWnBBmLiPRP6ijbm2cu5vMfmBRiNNnJ0fe5oZ59e2NyW4vsiEhOWP6jc8IOIav1Z77ACcPKktslheF9NCspiEiPRSPGuYeMBtBU2mnE+9gVFbzpRBKKokoKIpIjFr3vNTL/ce573VyZf/qTJ1O7+mpCPBHJGVeduh8Q7qRtWasfJYWqsqIBDKTvlBREpFdOnOx1u2yJqfqoo65KCs2tcT5/7xzeWpt+jMeBKb27wqSkICK9UlXqdbt8/M33Q44kO/x61grqrvVm6umqTWH1lgb+sXgjX3nwtbTnEwPXbjzvoIEPsheUFESkV0oKvQbR2Su2hBxJdvjhE4sBWFnf0GVJocBfY3NdJ2tSxOKOSSPKufT4uoEOsVeUFESk106eUkNtRXHYYWSVnXtakxPhfeqYCXv1ILp+xkIAmmNxZr65bq/Hx50LdXxCgpKCiPTafrVD2NXUGnYYWeU3L76brD4qK4rSHIsT84sO8bjj/5a2zfD8lQdfo+7amcx/b1vy2Kyl9Szr5SjoICgpiEiv1VYU09gcY1tjc9ihBOb11Vv5xC9f4uRbnk1+uHflr6+vTTa+J6a7aGz2Euf3Z7yV9jGJuY6ArEmySgoi0muJxuZLfj14F0z82J0vMWflVlZtbuTTd6f/O3/ktycADCsv4pYnlwAwblgpAEs3eOtP3D97ddrHr92WfSvZKSmISK+dNMVb12TxuvyYLfXlFZvTHr9r1ork9paGtlLT2Gpvyoru1rROrGS3anMDw8qL+PgRY7u8PhOUFESk10ZWliS3B+OMqf2dwmPiiHKgbQ2KcUNLk+cKo8Y9l7YtG+Oc4+RbnmNLQ3OyZ1eYlBREpNeiEeMLH5gIdN7FMpdd+ttX2+1XlHQ9ofQ3Tp+S3H7um6ckq9e2NrYQj7t2JYZXvnMaHzpwZHJ/4fttpa2SAiUFEclRZx40CoD3t2dfvXh/dVwjubE5lnbd5eHlRXzqmAlMG1uZPFZZWkhRQYRh5UWsrG9g5oL23U/Litp/8Dc2x1K2w29sVlIQkT4ZXe1ViQzGkkJVaSEThpVx7+VHc8J+w4nFHXta4ntdt6clRklhlEPGVSePDSn2ShXlxVH+8vpavvqH15PnfnbxYckqoj9deRzQNn4B4KE54U8yqKQgIn0ysqKYiMH7WdiDpj+cc/zv/PfZsGMPJ0+p4axp3lThO5ta9rpud0uM0sIow8vbJrMrKvA+VhubYu2u/8bpUzj/sLaG5ERZJLWxviYLBgQGuvKaiAxeBVGviqR+V1PYoQyo3S3eh3liacwK/5v/rj2t1KbMWdcSc8Sdtw5CuqmuNze0H8Nx4OjKdvv71Q5pt//vH57KRdPH9Tv+/lJJQUT6rH5Xc1ZUeQykJr+a6LxDxwBQVeY1GnfslppIHsV+ySDxuzOHjKtqt19dVsS5B49O7n/uxIlUpKzxHBaVFESk3xJ164NBo/9hf9y+wwEYU+W1nXz3r2/x4UPGJHsWNfnXJVZMe+HbH+xy5HNqN96E2y45nMtPrKMomj2vnUoKItJnXzx5EtD9IK1csmmnVx02YohXvz9lZFs1z6E3PkVLzCtJJEoKpX4yrKkoZlTV3h/8qdd0FIkYR+4zjIM7lCLCpKQgIn12xlSvv/3qLQ0hRzJwvv+/3jxFo/xv9h3bC/7Ln8ri+XfqATotId3/uWOS2z+/5PABjzMoqj4SkT7bZ7g3cnfV5saQI+k/5xwn/uTZ5HxEIyvT9wT61awVHLnPUL73qJc8dnYyojtR/QRw+tSRaa/JRkoKItJnw8uLKC+KZm1SOPIHT3P5iRP5ir+udFd2t8TaTVCXqD5K59klG5PbE4aVp70mGjH+8uXj91pXIdvlVrQiklXMjAnDy3ln486wQ9lLSyzO5obm5MylCd/96wJe8Kt+Um3f3f4bf1cL3vzh1bYeV6klgo6OmDCUaWOzp72gJ5QURKRfDhpTybxVW2lqjXV/cQalll4a/LUKmlvjPPDKaj59z95TYe/Y3TbFxE8uODj4ALOUkoKI9MuZB41iT0uceSu3dnttLO54dsnGtPMIDbTUqawTo65ffbfzdaUTJYUDRlXwyaMmtDv3zDUnt5vZNOGpq08aiFCzipKCiPTL8fsOpyga4bmU5SY7c9/LK/nsb+fwt7fWBx5XalJYv8Obn2nh+9s7vX7uKi9h3HrRoXudm1QzhA8dOJIj9xna7viUkRV7XZvrlBREpF/Kiws4eFwVb6SsN9yZdf6H87v1wXdh3ZqyVOh6fzGbeavaSjPLN7VfDzkxsd9BYzpvA/jNpUfx9Q9NHsgws46Sgoj029jq0uS38a48/oY3jXTHxt8gtCspbN/DnpYYTy3akDx2y9/bx7CifhdTO8xP1FFVWSFXp6ydMBipS6qI9Nvo6hL+vnAPzrm0k8MlZHJN4m2NzZQWRtndEuPWp5fuNcis4wR1qzY37lU91JkXvn1q2qm0BwOVFESk30ZXltDcGt9rZtCOjp00DIDqsuAnfnv13S0UpHQrbfanpxhaVkhFSQFrtrYfW7G1obnLsQmpxg0t22uW08FCSUFE+m2UP2lcou4+Heccs1d4jbnbGlvY0xJsF9Y31mxnZ1Mrpx1YC7RVWb3yndPYt2YIq7a0JYU9LTEammMMS1kXIV8pKYhIv42p9uYJ6mrBndRxAACzO0xFPZBS13g4dlL7wWVFBRHGDytjQ0obyLZGrzvq0DIlBSUFEem3xOygV9w3j/tmr0p7zcad7UsRl/12Ds2t6evl731pJXXXzmRlH3sp/XDmYgCmja1MznqaqraimFWbG5OllURPpaEZqNbKdkoKItJvI8rb6uL/3Z8orqNE76SfpowDmHb9k2mvTaxbfH8nCaY744Z61Vm//tfpyUn7oK0t4zl/7qK7n18BeO0JAENVfaSkICL913GeoHQlgA07vG/sh45vW+S+ORbfa3RzY3NbNdPdL7zbp3i2NjZTXVbI6KpSLjl6PCOGeB/2N50/DYDPnjARgP96aikAW/ySgtoUAk4KZnaWmS0xs2Vmdm2a81ea2QIzm29mL5jZ1CDjEZHMWLZx117HEnX4Y6pKufNTRySP1+9q32Ppk7+a3W6/L1Ni/G3Besr8LqhmxslTvMbmPc1eddEp+9e0uz5RUshEr6hsF1hSMLMocAdwNjAVuCTNh/6DzrmDnXOHAf8J/DSoeEQkWL/89BHJRe5ffXczz7y9gY/d+SKtflfQDTv2UFlSQGlRlHMOHs3nT/S+rSdGN29vbOGoH/6DBWu9qSj2rfGqff7YyzWgl23cyeaGZt5P6Ql1+ASvdDJumFetVFna9uH/0rJ6ZvmzpqqhOdiSwtHAMufcCudcM/AQcH7qBc65HSm75UDws2SJSCDOmjaa+defAcANjy3i8t/N5fXV2/j9y167wJqtuxlTXZq8/spT9gXg+Xc28eaabXzrkTfaNQp/8STv/NbG9IvYdOarf5i/17FPHTOBx646keP8nkhDitrG7f7L3a/wtD/SuTDH1j4IQpCvwFggNcWv8Y+1Y2ZfMbPleCWFrwUYj4gELJpmDYJEz553Nu5sN+ArMVDstmeWcd7tL7abgmJYeRHnHDIaoMfjGbY0NPO7F9+lMOrF8L1zD0yeMzMOHleVHG0diRjXnX1Ab/60vBF6WnTO3eGc2xf4NvC9dNeY2RVmNtfM5m7a1P1MjCISnsuOr2u3/7e31nPf7FWs2bqbybU9m1X0sPHVyTaBn/3znbTXbN7VxII1bbOeXvPwfG54bBHVfhXQ5X5jcmeuOGkSJ02p6fKafBRkUlgLjE/ZH+cf68xDwEfTnXDO3eWcm+6cm15To/+JItnshvMOare/bOMu/v3Rt3CubZBbwrfPSv9t/fSpI7tc+Qzg4794iY/c/gI1exr6AAAL8ElEQVTXPPwG8bhLzqs0a+kmItb1ymnglR5+9LFp3f05eSfIpDAHmGxmE82sCLgYmJF6gZmlzkF7LpD+K4GIDAoVJe1793S2WltiXeNr/BlJU7upAsx/b1tyZbU/v7aGSd95gqUb2no8xXvYOjk2pY3j1e98qGcPGuQCSwrOuVbgKuBJYDHwsHNuoZndZGbn+ZddZWYLzWw+8A3g0qDiEZHMmey3HVxy9Ph2xytL2k/MPHFE+0XvP3641+yY+EyfPNJ7nuUb249s/ugdLw5InGbGrG+dyl+/fDy1lSXdPyAPBDp1tnPuCeCJDse+n7L99SDvLyLheOrqk2iOxZm3amu7Re47zix63qFjACguiLKtsZnF67wOidv8xunR/kR7Nz62kK98cD9O3b+W+T1YzKc3JgwvY8LwsgF9zlym9RREZMCZGcUFUY6bNJyfXXwYHzyglmjEKCsq2Ou68w9r65T4j0UbuPflVdT5U1MkBpPNXbWVz/52Dit/fC71aeYySvjVZ47khhkL+elFhwXwV+UHJQURCUzHD/3unDZ1JLO+dWrym3t1msFkhQVerXc0Ytx2yeF8+YHXAFj543MBOPOgUf0NO68pKYhIVkmtykmMkE71zGJvPMNjV53I1DGVLLrpzL2m5Za+C32cgohIZyIR46sf3K/dsXv9EdJlRVH/d0Fy6m7pPyUFEclq15yxPxceOQ6g3cI4qfMXycBRUhCRrHfQmEoArvvLguQxLYgTDCUFEcl6lx1fx9CyQp55e2PyWGIeIxlYSgoikvXMrN1sqSMri7u4WvpDSUFEcs4fvnBs2CEMWkoKIpITbrvkcGoriln+o3OYVDOk+wdIn2icgojkhI8cOoaP+NNiSHBUUhARkSQlBRERSVJSEBGRJCUFERFJUlIQEZEkJQUREUlSUhARkSQlBRERSTLnXPdXZREz2wRsA7anHK5K2U9sd/w9Aqjvwy1Tn7un5zoe72q/Y5ypx/oSc1fxdna+u2PdbSve7s/39D0x0PF2F/NgfQ/3JN50sQ/m9/A+zrmabq9yzuXcD3BXZ/uJ7TS/5w7EvXpyrqv4uou3vzF3FW9n57s71t224h2498RAx9tdzIP1PdyTeMN8T4T5Hu7uJ1erjx7rYv+xTn4P1L16cq6r+Drup4uzPzF399h057s71t224u3+fE/fEwMdb3ePH6zv4Z7Em7oddrwdjwUZb5dyrvqor8xsrnNuethx9Eauxax4g5Vr8ULuxax486uh+a6wA+iDXItZ8QYr1+KF3Is57+PNm5KCiIh0L59KCiIi0g0lBRERSVJSEBGRJCUFwMw+YGa/NLO7zeylsOPpjplFzOyHZnabmV0adjw9YWanmNnz/ut8Stjx9ISZlZvZXDP7cNixdMfMDvRf20fM7Ethx9MdM/uomf3azP5oZmeEHU9PmNkkM7vHzB4JO5bO+O/Ze/3X9lN9eY6cTwpm9hsz22hmb3U4fpaZLTGzZWZ2bVfP4Zx73jl3JfA4cG+2xwucD4wDWoA1QcWaEttAxOyAXUAJAcc8QPECfBt4OJgo28U1EO/hxf57+CLghByI91Hn3BeAK4FPBhmvH9tAxLzCOfe5YCPdWy9j/zjwiP/antenGw70aLhM/wAnAUcAb6UciwLLgUlAEfAGMBU4GO+DP/WnNuVxDwMV2R4vcC3wRf+xj+TCawxE/MeNBB7IgXhPBy4GLgM+nO3x+o85D/gb8C+5EK//uFuBI3LhPZzyuMD/zfUj9uuAw/xrHuzL/QrIcc65WWZW1+Hw0cAy59wKADN7CDjfOfcfQNqqADObAGx3zu0MMNwBidfM1gDN/m4suGg9A/Ua+7YCxUHEmTBAr/EpQDneP7TdZvaEcy6erfH6zzMDmGFmM4EHg4h1oOI1MwN+DPzNOfdaULEmDPB7OKN6EzteKXwcMJ8+1gTlfFLoxFjgvZT9NcAx3Tzmc8BvA4uoa72N9y/AbWb2AWBWkIF1oVcxm9nHgTOBauD2YENLq1fxOue+C2BmlwH1QSWELvT29T0Fr+qgGHgi0MjS6+17+KvAaUCVme3nnPtlkMF1orev8XDgh8DhZnadnzzC0lnsPwduN7Nz6eNUGIM1KfSac+76sGPoKedcI14SyxnOub/gJbOc4pz7Xdgx9IRz7jnguZDD6DHn3M/xPsByhnNuM14bSNZyzjUAn+3Pc+R8Q3Mn1gLjU/bH+ceyVa7FC7kXs+INVq7FC7kZc0JgsQ/WpDAHmGxmE82sCK/BcEbIMXUl1+KF3ItZ8QYr1+KF3Iw5IbjYM9mKHlDL/B+AdbR1z/ycf/wcYCleC/13w44zV+PNxZgVr+IdDDGHFbsmxBMRkaTBWn0kIiJ9oKQgIiJJSgoiIpKkpCAiIklKCiIikqSkICIiSUoKEjgz25WBe5zXw+mwB/Kep5jZ8X143OFmdo+/fZmZhTEX1F7MrK7j9Mxprqkxs79nKibJPCUFyRlmFu3snHNuhnPuxwHcs6v5wU4Bep0UgO+QY/P+JDjnNgHrzCzQNRskPEoKklFm9i0zm2Nmb5rZjSnHHzWzeWa20MyuSDm+y8xuNbM3gOPMbKWZ3Whmr5nZAjM7wL8u+Y3bzH5nZj83s5fMbIWZXegfj5jZnWb2tpk9bWZPJM51iPE5M/sfM5sLfN3MPmJmr5jZ62b2DzMb6U9lfCVwtZnNN2/1vhoz+7P/981J98FpZhXAIc65N9KcqzOzZ/zX5p/+dO6Y2b5mNtv/e29OV/Iyb8WtmWb2hpm9ZWaf9I8f5b8Ob5jZq2ZW4d/nef81fC1dacfMomZ2S8r/qy+mnH4U6NOqXpIDwh7CrZ/B/wPs8n+fAdwFGN4XkseBk/xzw/zfpcBbwHB/3wEXpTzXSuCr/vaXgbv97cuA2/3t3wF/8u8xFW/eeYAL8aaVjgCj8NZ2uDBNvM8Bd6bsD4Xk6P/PA7f62zcA30y57kHgRH97ArA4zXOfCvw5ZT817seAS/3ty4FH/e3HgUv87SsTr2eH570A+HXKfhXe4isrgKP8Y5V4MyOXASX+scnAXH+7Dn8hF+AK4Hv+djEwF5jo748FFoT9vtJPMD+aOlsy6Qz/53V/fwjeh9Is4Gtm9jH/+Hj/+Ga8RYT+3OF5ElNwz8NbQyCdR523BsIiMxvpHzsR+JN/fL2ZPdtFrH9M2R4H/NHMRuN90L7byWNOA6aaWWK/0syGOOdSv9mPBjZ18vjjUv6e+4D/TDn+UX/7QeC/0jx2AXCrmf0EeNw597yZHQysc87NAXDO7QCvVIE35/5heK/vlDTPdwZwSEpJqgrv/8m7wEZgTCd/g+Q4JQXJJAP+wzn3q3YHvQViTgOOc841mtlzeGs5A+xxznVcXa7J/x2j8/dwU8q2dXJNVxpStm8Dfuqcm+HHekMnj4kAxzrn9nTxvLtp+9sGjHNuqZkdgTdJ2s1m9k/gr51cfjWwATgUL+Z08RpeiezJNOdK8P4OGYTUpiCZ9CRwuZkNATCzsWZWi/ctdKufEA4Ajg3o/i8CF/htCyPxGop7ooq2ueovTTm+E6hI2X8Kb0UxAPxv4h0tBvbr5D4v4U2BDF6d/fP+9my86iFSzrdjZmOARufc/cAteGv6LgFGm9lR/jUVfsN5FV4JIg58Bm+9346eBL5kZoX+Y6f4JQzwShZd9lKS3KWkIBnjnHsKr/rjZTNbADyC96H6d6DAzBbjrds7O6AQ/ow39fAi4H7gNWB7Dx53A/AnM5sH1Kccfwz4WKKhGfgaMN1vmF1EmlW6nHNv4y1BWdHxHF5C+ayZvYn3Yf11//i/Ad/wj+/XScwHA6+a2XzgeuBm51wz8Em8pVvfAJ7G+5Z/J3Cpf+wA2peKEu7Ge51e87up/oq2UtmpwMw0j5FBQFNnS15J1PGbt97uq8AJzrn1GY7hamCnc+7uHl5fBux2zjkzuxiv0fn8QIPsOp5ZeAvcbw0rBgmO2hQk3zxuZtV4DcY/yHRC8P0C+EQvrj8Sr2HYgG14PZNCYWY1eO0rSgiDlEoKIiKSpDYFERFJUlIQEZEkJQUREUlSUhARkSQlBRERSVJSEBGRpP8Pv247cf9mtdAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.lr_plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We would like the maximal learning rate associated with a still-falling loss (prior the loss diverging).  Based on the plot, we will start with a learning rate of 0.005."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interactive Training\n",
    "\n",
    "It is sometimes advantageous to train interactively.  For instance, one can train a model for one or two epochs using one learning rate.  Then, based on the results, a higher or lower learning rate can be used for subsequent epochs.  *ktrain* makes such interactive training easy.  Here, using the fit method of the Learner object, we train a single epoch at the learning rate found previously and a second epoch at a slightly lower learning rate.  The first argument is the learning rate and the second argument is the number of epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/1\n",
      "25000/25000 [==============================] - 5s 197us/step - loss: 0.4010 - acc: 0.8293 - val_loss: 0.2984 - val_acc: 0.8777\n",
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/1\n",
      "25000/25000 [==============================] - 5s 183us/step - loss: 0.2105 - acc: 0.9283 - val_loss: 0.2869 - val_acc: 0.8860\n"
     ]
    }
   ],
   "source": [
    "# reinitialize the model to train from scratch \n",
    "learner.set_model(get_model())\n",
    "\n",
    "hist = learner.fit(0.005, 1)\n",
    "hist = learner.fit(0.0005, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Learning Rate Schedules\n",
    "\n",
    "In the example above, a static learning rate is used throughout each epoch.  It is sometimes beneficial to employ the use of learning rate schedules to *automatically* adjust the learning rate during the course training to more effectively minimize loss. Such adjustments can help jump out of suboptimal areas in the loss landscape and get to \"sweet spots\" with minimal loss that generalize well.  *ktrain* allows you to easily employ a variety of demonstrably effective learning rate policies during training. These include:\n",
    "\n",
    "* a [triangular learning rate policy](https://arxiv.org/abs/1506.01186) available via the ```autofit``` method\n",
    "* a [1cycle policy](https://arxiv.org/abs/1803.09820) available via the ```fit_onecycle``` method\n",
    "* an [SGDR](https://arxiv.org/abs/1608.03983) (Stochastic Gradient Descent with Restart) schedule available using the ```fit``` method by supplying a *cycle_len* argument.\n",
    "\n",
    "\n",
    "### SGDR\n",
    "We will begin by covering SGDR. *ktrain* allows you to easily employ an SGDR learning rate policy in a similar style to that of the *fastai* library. We will begin with covering the cycle_len parameter.\n",
    "\n",
    "**cycle_len:** When *cycle_len* is not None, the second argument fo ```fit``` is interpreted as the number of cycles instead of the number of epochs.  For instance, the following call runs 2 cycles each of length 2 epochs - totaling 4 (or 2 * 2) epochs.  The learning rate gradually decreases throughout the 2-epoch cycle and then restarts at 5e-3 at the start of a new 2-epoch cycle.  Decreases follow a functional form (cosine annealing). More information can be found in the original [SGDR paper](https://arxiv.org/abs/1608.03983)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/4\n",
      "25000/25000 [==============================] - 6s 230us/step - loss: 0.4063 - acc: 0.8253 - val_loss: 0.3004 - val_acc: 0.8841\n",
      "Epoch 2/4\n",
      "25000/25000 [==============================] - 6s 223us/step - loss: 0.2265 - acc: 0.9209 - val_loss: 0.2874 - val_acc: 0.8872\n",
      "Epoch 3/4\n",
      "25000/25000 [==============================] - 6s 222us/step - loss: 0.2062 - acc: 0.9227 - val_loss: 0.2840 - val_acc: 0.8880\n",
      "Epoch 4/4\n",
      "25000/25000 [==============================] - 6s 227us/step - loss: 0.1397 - acc: 0.9555 - val_loss: 0.2812 - val_acc: 0.8894\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fe7e7ef99e8>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reinitialize the model to train from scratch \n",
    "learner.set_model(get_model())\n",
    "\n",
    "# training using cycle_len \n",
    "learner.fit(5e-3, 2, cycle_len=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ```learner.plot``` method can be used to plot the training-validation loss (with 'loss' as argument) in addition to plotting the learning rate schedule with ('lr' as argument) and momentum schedule (with 'momentum' as argument) where applicable. Here, we plot the learning rate schedule employed by the previous call to ```learner.fit```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd8VfX9+PHXO5tABhnMBBJI2LJBERQZMhRFrQOt2zpBcdViba36/dlqW7VaUbTirBZQtI0WVEQEGbKHDAMhrIBA2GElJLx/f9xDm8YACbk354738/G4D0/O+Zxz38cb8r6f81miqhhjjDFnKsztAIwxxgQ2SyTGGGNqxBKJMcaYGrFEYowxpkYskRhjjKkRSyTGGGNqxBKJMbVIRDJEREUkwkvX2ygiA71d1pjqsERiQsrJ/piKyAUiclxEDopIkYjkisgtp7nWbSLyg1N+h4hMEZE430VvjH/yyrciY4LENlVNExEBhgI5IjJXVXMrFhSRvsDvgSGqulREkoBLajleY/yC1UiMqUA9pgB7gI4nKdYDmKeqS51z9qjqO6paBCAidUTkORHZJCL7RWS2iNQpd/7PRWSziOwSkcdO7BSRMBEZIyLrRWS3iExyktSJ4zc419xd/jzn2Nsi8v/K/XyBiBRUFvzp3seY6rBEYkwFzh/ZS4EUIO8kxeYDg0XkSRHpLSLRFY7/GegGnAskAY8Ax8sd7wO0BgYAj4tIW2f/vcBlQF+gCbAXGOvE1Q54FbjBOZYMpJ3hbZ70fYypLkskxvxXExHZBxwBPgEePFHjqEhVvwWuALoC/wZ2i8jzIhIuImHArcBoVd2qqmWqOldVi8td4klVPaKqy4HlQCdn/13AY6pa4JR/ArjSaZy/EvhMVWc5x37L/yan6jjV+xhTLfZLY8x/nWgjiQaeAfoDfzlZYVWdCkx1Ekc/4EMgF08SigHWn+K9tpfbPgzUc7abA5+ISPkEUQY0xFNz2FLu/Q+JyO4q3ltFp3qfrWd4TROirEZiTAXON/RfAWeJyGVVKH9cVacDXwMdgF3AUaDlGbz9FmCoqiaWe8Wo6lbgRyD9REERicXzeOuEQ0BsuZ8bneH7GFMtlkhMKIoUkZhyr5/UzFW1BHgOeLyyC4jIcBEZISL1xaMnnvaG71T1OPAm8LyINHEed/WqpB2lMuOAp0WkufM+qSIy3Dn2ETBMRPqISBTwFP/7b3gZcJGIJIlII+D+M3wfY6rFEokJRVPwtIOceD1xknJvAs1EpLJuvXuB24F1wAHg78CfVPV95/jDwPfAQjy9v56lav/eXgRygC9FpAj4DjgbQFVXASOBD/DUTvYC5XtlvYenvWUj8CUw8Uzex5jqElvYyhhjTE1YjcQYY0yNWCIxxhhTI5ZIjDHG1IglEmOMMTUSEgMSU1JSNCMjw+0wjDEmYCxevHiXqqZWpWxIJJKMjAwWLVrkdhjGGBMwRGRTVcvaoy1jjDE1YonEGGNMjVgiMcYYUyOWSIwxxtSIJRJjjDE14tNEIiJDRCRXRPJEZEwlx6NFZKJzfL6IZJQ79qizP1dEBpfbv1FEvheRZSJiXbGMMcZlPuv+KyLheJbuvBDPDKULRSRHVVeXK3YbsFdVs0RkBJ4ZUq9xlhQdAbTHs5jPVyLSSlXLnPP6qeouX8VujDGm6nw5jqQnkKeq+QAiMgEYDpRPJMP57xTeHwEvi4g4+yc4CwxtEJE853rzfBjvT7w0fR3hYULdqHCS6kWTmVyXjJRY4mIiazMME6Rytxfxxart1I2OIKFOJM2TY2mRUpekulF4/hkYExh8mUiaUm5ZUDy1korrHfynjKqWish+PCu+NcWzPkL5c5s624pnDQUFXlPV1yt7cxG5A7gDoFmzZmd0A+NmrudwSdlP9rdpFMfZmUn0a9OA3lkpRIZbU5Opvr99m89Hiwt+sr9pYh3ObpHEedkpDGzb0L64GL8XiCPb+6jqVhFpAEwTkR9UdVbFQk6CeR2ge/fuZ7ToyqonB1NSdpxDxWUUFhWzYdchcrcXsXDjHiYtKuCdeZuoHxvJJZ2acNO5GbRMrXf6ixrjOH5caZIQw5TR57Hv8DE27j5E3s6DLNm8l29yC/l4yVaiI8IY2LYh15/TnHNaJFlNxfglXyaSrZRbXxpIc/ZVVqbAWe40Adh9qnNPrCmtqjtF5BM8j7x+kki8QUSIjggnOiKcpLpRtG4Ux5AOnmWwjx4rY9baQj5d8SMTFm7h3Xmb6Nc6ldEDW9E5PdEX4ZggFB4uJMZGkRgbRUZKXS5o3QDwJJmlW/aRs2wrOcu38e/vf6RD03ju7pvFRWc1soRi/Iovn8ksBLJFJNNZX3oEnqU9y8sBbnK2rwS+Vs+SjTnACKdXVyaQDSwQkboiEgcgInWBQcBKH97DScVEhjOofSP+em0X5o7pz/0Ds1lesJ/Lxs5h5PtL2LjrkBthmSARFiZ0a16fJ4d3YN6jA/j95WdxuKSMkR8s4bJX5rJgwx63QzTmP3yWSFS1FBgFfAGsASap6ioReUpELnWKjQeSncb0B4ExzrmrgEl4GuY/B0Y6PbYaArNFZDmwAPi3qn7uq3uoqpR60dw/sBWzHunHfQOymZG7k0F/mcXYGXkcKzvudngmwMVEhnPd2c2Y9kBf/nhlR3bsP8rVr83j4Q+Xs+9widvhGRMaa7Z3795da3P23x0HjvJEziqmrtxO28bxvHBNJ9o0iq+19zeB4cGJy1i4aQ/fPtK/WucdKSnj5RnrGDczn/qxUfz+8g4Mat/IR1GaUCUii1W1e1XKWncjH2gYH8Or13dj3PXdKCwqZvjLc5iwYDOhkLSN79WJCueXg9uQM6o3DeOjueO9xTz56SpKSq32a9xhicSHhnRoxNTR59EjI4kxH3/PAxOXcfTYT7sTG3Mm2jdJ4ON7zuWW3hm8NWcjV46by9Z9R9wOy4QgSyQ+lhoXzTu39uTBC1vxr+XbGPH6d+wsOup2WMYPeKN+Gh0Rzu8uac+467uyofAQl42dw4qCfV64sjFVZ4mkFoSHCfcNyObVn3cjd3sRl708hx+2H3A7LOMHBO904x3SoTGT7zmXqPAwrn5tHp+v/NEr1zWmKiyR1KIhHRrx4V29KFPl6nHzWLp5r9shmSDSqmEc/xzZmzaN4rn7/SVMWLDZ7ZBMiLBEUss6NE3go7vOpX7dKK5/Yz7z1u92OyQTRFLjoplwxzmcn53KmI+/5605G9wOyYQASyQuSE+KZdKdvWiSWIeb31rArLWFbodkgkhMZDiv39iNwe0b8uSnqxk3c73bIZkgZ4nEJQ3jY5h4Zy9apNbjjvcWsXCjjVQ23hMdEc7L13Xl0k5NeGbqD1YzMT5licRFSXWjeO+2njRJrMOtby3k+4L9bodkgkhkeBjPX92JQe08NZMPF205/UnGnAFLJC5LqRfN+784m/g6kdz45nzydha5HZIJIhHhYfz1ui70yUrhV5NXMPV7681lvM8SiR9onFCHD24/m/CwMG5+ayGFRcVuh2RqQW3NdBAd4Wkz6ZyeyOiJy1i8yR6jGu+yROInmifXZfxN3dl1sJjb311kI+BDRG3NBh8bFcEbN/WgcUIMt7+7mM27D9fOG5uQYInEj3RKT+Qv13RhecE+Hpi4jOPHbW4u4z1JdaN46+YelB1Xbnl7AfsPH3M7JBMkLJH4mSEdGvHYRW2ZunI7z09b63Y4Jsi0SK3Hazd0Y/Oew9zzwWJKbZkD4wWWSPzQbX0yGdEjnZdn5PHFqu1uh2OCzDktknn68rOYk7ebP32R63Y4JghYIvFDIsITl7anU1oCD01azvrCg26HZILM1d3Tuf6cZrw2K58p1pPL1JAlEj8VExnOq9d3IyoijLveW8zB4lK3QzJB5vFh7enSLJFffrjcup2bGrFE4seaJNbh5Wu7sL7wIGMmr7CFsYKM259mVEQYr/y8K3WiwrnzvcUcsi8r5gxZIvFz52al8NCg1ny24kcm2cjkoFNLvX9PqnFCHV66tgsbdh3idzmrXI7GBCpLJAHg7r4t6Z2VzBM5q+0RhPG6c1umMKpfFh8tLuBfy7a6HY4JQJZIAkBYmPD81Z2pExXOvf+w5XqN9903IJtuzevzm09WsmWPDVY01WOJJEA0jI/hz1d1ZM2PB3hm6g9uh2OCTER4GH+5pjMI3DdhKcdsfImpBkskAaR/m4bc0juDt+duZKatYWK8LD0plj9ccRZLN+/jr9PXuR2OCSCWSALMr4a0IatBPX710Qr2H7EpLox3DevYhCu6NmXsN+ttWQNTZZZIAkxMZDjPXdWJwoPFPPXparfDMUHod5e0J6VeFA99uIziUmuPM6dniSQAdUpP5O6+LZm8pIBpq3e4HY45Q/46LCihTiTP/Kwja3cc5MWv7BGXOT1LJAHqvgHZtGkUx6Mff8/eQyVuh2POkNTWPPLV1K91A67pns64metZtmWf2+EYP2eJJEBFRYTx3NWd2He4hCc+tYFkxvseG9aWRvExPDTJupybU7NEEsDaN0lgZL8s/rVsG9/k7nQ7HBNk4mM8j7jWFx7i5a/z3A7H+DFLJAHunn4taZFal9/8cyWHS2yuJONd57dK5YouTRk3cz1rd9isCqZylkgCXHREOH+4/CwK9h6xhlHjE49d3Ja4mAge/fh7W7XTVMqniUREhohIrojkiciYSo5Hi8hE5/h8Eckod+xRZ3+uiAyucF64iCwVkc98GX+gOLtFMiN6pPPG7A2s3Gp9/413JdeL5tcXtWXxpr1MWGgTh5qf8lkiEZFwYCwwFGgHXCsi7SoUuw3Yq6pZwAvAs8657YARQHtgCPCKc70TRgNrfBV7IHp0aFvqx0by60++p8y+NRovu7JbGr1aJPOHqWvYWXTU7XCMn/FljaQnkKeq+apaAkwAhlcoMxx4x9n+CBggnv6Qw4EJqlqsqhuAPOd6iEgacDHwhg9jDzgJsZE8fkl7VhTs5525G90Ox1RBIKV7EeHpyztQXHrcBsKan/BlImkKlK8HFzj7Ki2jqqXAfiD5NOf+BXgEOOWsciJyh4gsEpFFhYWhMS/VJR0bc36rVF6Ytta+NQYI/xxFUrkWqfUY1S+Lz1b8yLfrQuPflKmagGpsF5FhwE5VXXy6sqr6uqp2V9XuqamptRCd+0SEJy5px9HSMp6dmut2OCYI3dm3Bc2TY3kiZxUlpTZDsPHwZSLZCqSX+znN2VdpGRGJABKA3ac4tzdwqYhsxPOorL+I/N0XwQeqFqn1uK1PCyYvKWDxpj1uh2OCTHREOL+7pB3rCw/x9twNbodj/IQvE8lCIFtEMkUkCk/jeU6FMjnATc72lcDX6lmYPAcY4fTqygSygQWq+qiqpqlqhnO9r1X1eh/eQ0C6t38WDeOj+V3OKmt4N17Xv01D+rdpwItfrWPnAXuEanyYSJw2j1HAF3h6WE1S1VUi8pSIXOoUGw8ki0ge8CAwxjl3FTAJWA18DoxUVZujoYrqRkfw2MXtWLn1ABMWbnY7HBOEHh/WjmNlaousGcDHbSSqOkVVW6lqS1V92tn3uKrmONtHVfUqVc1S1Z6qml/u3Ked81qr6tRKrv2Nqg7zZfyB7JKOjTk7M4k/fZFrkzoar8tIqcvt52fy8dKtLNpoj1BDXUA1tpuqExGeHN6eoqOl/PlLa3g33jeyXxaNE2J4/F/2CDXUWSIJYm0axXPDOc35x4LN5G63eZL8jfrrgiRVFBsVwa8vasvqHw8waZGNeA9llkiC3OgB2dSLjuDpKTYRgF8KpIEklRjWsTE9Murz3JdrOVhsk4aGKkskQa5+3SjuG5DNrLWFNtW88ToR4bGL27HrYDGvzVzvdjjGJZZIQsANvZrTPDmW309ZQ2mZDSIz3tU5PZFLOzXh9Vn5bNt3xO1wjAsskYSA6IhwHh3ahrU7DjLRnmUbH3hkSGsU+PMX1rEjFFkiCRGD2zeiZ0YSL0xbS9HRY26HY4JMWv1Ybu3t6Q78fYEtZRBqLJGECM+z7LbsOljCq9/Ys2zjfff0a0lS3Sj+379XB3yPNFM9lkhCSKf0RC7v0pQ3Zm+gYO9ht8MxQSY+JpIHBmYzf8Mepq3e4XY4phZZIgkxvxzcGoDnp611ORITjN/Zr+3ZjJapdXlm6g8cs44dIcMSSYhpkliHm8/N4JOlW22Qoh8I8GEkPxERHsajQ9uSv+sQHy4qcDscU0sskYSgey5oSb3oCP70hU24Z7xvQNsGdGtenxenr+VIic21GgoskYSgxNgo7urbkq/W7GShTbhnvExE+NWQNuw4UMzbtuxzSLBEEqJu6Z1Balw0z079wXrYGK/rmZlE/zYNePWbPPYftu7mwc4SSYiKjYpg9IBsFm3ay9c/2NQpxvt+Obg1RcWljJtl3c2DnSWSEHZNj3QyU+ryx89zbRpw43VtG8czvFMT3pqzgR22kmJQs0QSwiLDw3hoUCtydxTxr2Vb3Q7HBKEHL2xNaZny0vR1bodifMgSSYi7qENjOjSN57kv11Jcaj1salUIVAKbJcdy3dnNmLBwCxt2HXI7HOMjlkhCXFiYp4fN1n1H+GC+re9e20SCbSTJT43qn0VUeJgNgg1ilkgM52Wn0qtFMmNnrLd+/8brGsTFcFufTD5dvo2VW21Cx2BkicQA8NCgVuw6WMy78za6HYoJQnf0bUF8TAR/+cpqJcHIEokBoHtGEue3SmXczPW2ZKrxuviYSO44vwVfrdnJsi373A7HeJklEvMfD17Yir2Hj/H2nA1uh2KC0M29M6kfG8kL1lYSdCyRmP/onJ7IwLYNeH1WPvuP2Ghk4131oiO4s29LZq4tZPEmm5onmFgiMf/jgQtbceBoKeNnW63EeN+NvZqTUi+K5760WkkwsURi/kf7JgkM7dCIN2dvYO+hErfDCWoaCgNJKoiNiuDuC7KYu34389bvdjsc4yWWSMxPPHBhKw6VlPLarHy3Qwl6wT+K5Kd+fnYzGsZH88K0tTZhaJCwRGJ+olXDOC7p2IR35m6ksKjY7XBMkImJDGdkvywWbNzD7LxdbodjvMASianU6IHZFJeWMW6mzdxqvO+aHuk0SYjheauVBAVLJKZSLVPrcXmXNP7+3SabudV4XXREOKP6Z7N08z6+yS10OxxTQz5NJCIyRERyRSRPRMZUcjxaRCY6x+eLSEa5Y486+3NFZLCzL0ZEFojIchFZJSJP+jL+UDd6QDZlx5WxM/LcDsUEoau6p5GeVMdqJUHAZ4lERMKBscBQoB1wrYi0q1DsNmCvqmYBLwDPOue2A0YA7YEhwCvO9YqB/qraCegMDBGRc3x1D6GuWXIsV3ZLY8KCLWzfb7US412R4WHc1z+b77fu58vVO9wOx9SAL2skPYE8Vc1X1RJgAjC8QpnhwDvO9kfAAPFMhzocmKCqxaq6AcgDeqrHQad8pPOyrzI+NLJfFsdVra3E+MTlXZqSkRzLS9PXWa0kgPkykTQFtpT7ucDZV2kZVS0F9gPJpzpXRMJFZBmwE5imqvMre3MRuUNEFonIosJCewZ7ptKTYrmia1M+WLDZ2kq8zP5uQkR4GKP6Z7Nq2wGmr7ElnwNVwDW2q2qZqnYG0oCeItLhJOVeV9Xuqto9NTW1doMMMqP6edpKrFbifSGwHMlpXda5Cc2SYnnRaiUBy5eJZCuQXu7nNGdfpWVEJAJIAHZX5VxV3QfMwNOGYnyoWXIsl3dpygfzN7PTaiXGyyLCwxjVL4vvt+5nRq7VSgKRLxPJQiBbRDJFJApP43lOhTI5wE3O9pXA1+r5SpIDjHB6dWUC2cACEUkVkUQAEakDXAj84MN7MI5R/bIoPa422t34xOVdm5JWvw4vTs+zWkkA8lkicdo8RgFfAGuASaq6SkSeEpFLnWLjgWQRyQMeBMY4564CJgGrgc+BkapaBjQGZojICjyJapqqfuarezD/lZFSl+Gdm/D+/E022t14XWR4GCP7ZbF8yz5mrrU2zUDj0zYSVZ2iqq1UtaWqPu3se1xVc5zto6p6lapmqWpPVc0vd+7TznmtVXWqs2+FqnZR1Y6q2kFVn/Jl/OZ/3ds/m5LS4/ztW6uVGO/7Wdc0mibWsbaSABRwje3GPZkpdRneuSnvzdvEroNWKzHeFRURxt0XtGTp5n02B1eAsURiqmVU/yyKS8usVmJ84qruaTROiOHFr6xWEkgskZhqaZlaj0s6NeG9eZvYY+uV1Ij9nfyp6Ihw7rmgJYs27WWurVcSMCyRmGq7t38WR45ZrcQbJCRXJDm1q7qn0zA+mhenr3M7FFNFlkhMtWU1iGNYxya8O3ejraJovC4mMpy7+7ZkwYY9topigLBEYs7Iff2zOHysjDdmW63EeN+Ins1oEBfNi9NtbfdAYInEnJHshnFcdFZj3pm7iX2HrVZivCsmMpw7+7bku/w9zM+3Wom/s0Rizth9/bM5WFzK+Nkb3A7FBKGfn92MlHrRvPS1tZX4O0sk5oy1bhTHRWc14u05G9l/+Jjb4ZggExMZzl19WzAnbzeLNu5xOxxzCpZITI3c2z+bouJS3pxjtRLjfded3YzkulHWg8vPWSIxNdK2cTyD2zfkzTkb2H/EaiXVobYm22nFRkVwx/kt+HbdLhZv2ut2OOYkLJGYGrtvQDZFR0t5e85Gt0MJOLYeyeldf05zkupG8ZLVSvyWJRJTY+2bJHBhu4aMn53PgaNWKzHeVTc6gl+cl8nMtYUs27LP7XBMJU6bSJylbW3ND3NK9/XP5sDRUt6du9HtUEwQurFXBomxkVYr8VOnTSTOOiC5ItKsFuIxAeqstAQGtGnAG7M3cLC41O1wTJCpFx3B7ee14OsfdrKiwGol/qaqj7bqA6tEZLqI5Jx4+TIwE3hGD8xm3+FjvGO1EuMDN/ZqTkIdq5X4o4gqlvutT6MwQaFjWiL9Wqfyxrf53HxuBnWjq/rrZczpxcVEclufTJ6ftpaVW/fToWmC2yEZR5VqJKo6s7KXr4Mzgee+AdnsPXyM977b5HYoJgjd3DuDuJgIq5X4mVMmEhEpEpEDlbyKRORAbQVpAkeXZvU5v1Uqf5uVz+ESays5FVuPpPriYyK5tXcmX67ewept9ifIX5wykahqnKrGV/KKU9X42grSBJbRA7LZfaiE97/b7HYoJgjd2juTuGirlfgTG0divK5b8/r0yUrhtVnrOVJS5nY4JsgkxEZyS+8MPl+1nR+2W63EH1giMT4xemA2uw6W8P58aysx3ndrn0zqRUfw1+l5bodisERifKRHRhLntkzmtVn5HD1mtRLjXYmxUdx0bnOmrPyRtTuK3A4n5FkiMT5z34BsCouK+ccCaysx3veLPi2IjQznr19brcRtlkiMz5zTIpmzM5MYN3O91UqM19WvG8WN52bw2Ypt5O20WombLJEYnxo9MJsdB4qZuHCL26H4Hev9W3O/6JNJTITVStxmicT4VK8WyfTMSOLVb9ZTXGq1korE5pGvkeR60dzYqzmfLt/G+sKDbocTsiyRGJ8SEe4bkM32A0eZtKjA7XBMELr9/BZERYQx1molrrFEYnyud1Yy3ZrX59UZeVYrMV6XUi+a689uzj+XbWXDrkNuhxOSLJEYnxMRRg/IZtv+o0xevNXtcEwQuqNvCyLDw3jZaiWusERiasV52Sl0Tk9k7Iw8SkqPux2OCTIN4mL4uVMr2bTbaiW1zaeJRESGiEiuiOSJyJhKjkeLyETn+HwRySh37FFnf66IDHb2pYvIDBFZLSKrRGS0L+M33iMijB6YzdZ9R/h4ibWVGO+7q28LwsOEsTOsVlLbfJZIRCQcGAsMBdoB14pIuwrFbgP2qmoW8ALwrHNuO2AE0B4YArziXK8UeEhV2wHnACMruabxUxe0SqVTWgIvz8jjWJnVSox3NYiP4bqezfh4yVa27DnsdjghxZc1kp5Anqrmq2oJMAEYXqHMcOAdZ/sjYIB4+kMOByaoarGqbgDygJ6q+qOqLgFQ1SJgDdDUh/dgvOhED66CvUf4ZKm1ldg08t53V9+WhInwyjdWK6lNvkwkTYHyo9AK+Okf/f+UUdVSYD+QXJVzncdgXYD5lb25iNwhIotEZFFhYeEZ34Txrv5tGnBW0wTGzsij1Gol2CgS72qUEMOInul8uKiAgr1WK6ktAdnYLiL1gMnA/apa6TzSqvq6qnZX1e6pqam1G6A5qRO1kk27D/PPZdvcDscEobv6tkQEXvlmvduhhAxfJpKtQHq5n9OcfZWWEZEIIAHYfapzRSQSTxJ5X1U/9knkxqcGtm1Au8bxVisxPtEksQ5Xd0/nw0Vb2LbviNvhhARfJpKFQLaIZIpIFJ7G85wKZXKAm5ztK4GvVVWd/SOcXl2ZQDawwGk/GQ+sUdXnfRi78aETtZINuw7x6QqrlRjvu6dfFgCvWq2kVvgskThtHqOAL/A0ik9S1VUi8pSIXOoUGw8ki0ge8CAwxjl3FTAJWA18DoxU1TKgN3AD0F9Eljmvi3x1D8Z3BrVrSJtGcfz16zzKjlurs/Gupol1uLJbOhMXWq2kNvi0jURVp6hqK1VtqapPO/seV9UcZ/uoql6lqlmq2lNV88ud+7RzXmtVnersm62qoqodVbWz85riy3swvhEW5hntnl94iJzl1oPLeN/Ifi1RlJdtXInPBWRjuwkOg9s3om3jeF78ap2NKzFel1Y/lmt7NmPSwi1s3m09uHzJEolxTViY8NCFrdi4+zCTF4fiaHd7pOdrI/tlER4mvDh9nduhBDVLJMZVA9o2oHN6Ii9NXxeSMwPbciS+1TA+hht7NeeTpQW2iqIPWSIxrhIRHh7Umm37j/KP+ba2u/G+u/q2JCYynBe+slqJr1giMa7rnZXMOS2SeHnGeg6XlLodjgkyyfWiubV3Jv9e8SOrt1U6ftnUkCUS47oTtZJdB4t5d94mt8MxQej281sQHxPB89PWuh1KULJEYvxC94wkLmidyriZ6zlw9Jjb4Zggk1AnkjvOb8FXa3awdPNet8MJOpZIjN946MLW7Dt8jDdnb3A7FBOEbu6dSVLdKKuV+IAlEuM3zkpLYEj7Rrzx7Qb2HipxOxwTZOpFR3B335Z8u24X3+XvdjucoGJxHiw0AAAVS0lEQVSJxPiVBwe14lBJKa/Nyj994QBn65HUvht6NadBXDTPf7kWtQ/AayyRGL/SqmEcwzs14e25G9hZdNTtcHzOxpHUrpjIcO7tn8WCjXuYtW6X2+EEDUskxu/cP7AVx8qUl7+2OZKM913Toxlp9evwx89/4LhNGOoVlkiM38lIqcs1PdL5YP5mNu465HY4JshERYTx8KDWrNp2gJzltoyBN1giMX7p/gHZRIaH8acvc90OxQShSzs1oX2TeP78ZW5ITs3jbZZIjF9qEB/D7ed5RiMv37LP7XBMkAkLE8YMbUPB3iP8/TubmqemLJEYv3VH35Yk143i91PWWA8b43XnZafSJyuFl79eZ4Nga8gSifFb9aIjuG9ANvM37GFG7k63wzFBaMzQNuw9fIzXZtqSvDVhicT4tWt7NqN5cizPTs0NuiV5g+tuAlOHpgkM79yE8bM3sH1/8Hc39xVLJMavRUWE8cvBrcndUcTkJcG3+JVgA0nc9vCg1pQdV16cblOnnClLJMbvXXxWYzqlJfDCtLUcPWY9bIx3pSfFcv05zZm4cIstfnWGLJEYvycijBnalh/3H+WtORvdDscEoXv7ZxMbFcEzU627+ZmwRGICQq+WyfRv04BXZuSx62Cx2+GYIJNUN4p7+rXkqzU7mJNnU6dUlyUSEzB+fVFbjhwr47kv7Vm28b5be2eSnlSHpz5dTWnZcbfDCSiWSEzAyGpQjxt6NWfiws22ZKrxupjIcH49tC25O4qYsHCL2+EEFEskJqDcP6AVCXUieeqzVTZI0XjdkA6N6JmZxPPT1rL/iA1SrCpLJCagJMRG8uCFrfgufw9frNrudjg1YonQ/4gIjw9rx97DJfx1+jq3wwkYlkhMwLm2ZzNaN4zj6SlrAr47sK1H4n86NE3g6m7pvD13I/mFB90OJyBYIjEBJyI8jN8Oa8eWPUesO7DxiYcGtyImMpzfT1njdigBwRKJCUh9slMY2LYhL3+9LiRWUjS1q0FcDCP7ZfHVmp3MWlvodjh+zxKJCVi/ubgtJWXHeWbKD26HYoLQLb0zaJ4cyxM5q2zNktPwaSIRkSEikisieSIyppLj0SIy0Tk+X0Qyyh171NmfKyKDy+1/U0R2ishKX8Zu/F9GSl3uPL8lHy/dynf5u90OxwSZmMhwnry0Pfm7DvHGtxvcDsev+SyRiEg4MBYYCrQDrhWRdhWK3QbsVdUs4AXgWefcdsAIoD0wBHjFuR7A284+YxjZL4u0+nX47T9XUlJqg8iMd13QugFDOzTipenr2LLnsNvh+C1f1kh6Anmqmq+qJcAEYHiFMsOBd5ztj4ABIiLO/gmqWqyqG4A853qo6ixgjw/jNgGkTpTnW+O6nQd5c459azTe99th7QgPE578dJXbofgtXyaSpkD54aEFzr5Ky6hqKbAfSK7iuackIneIyCIRWVRYaI1lwWxA24YMateQF79aR8HewPnWaKNIAkOTxDqMHpDNV2t2Mm31DrfD8UtB29iuqq+randV7Z6amup2OMbHHr/E89T0qU9XuxxJ9dgwksBwa59MshvU44mcVRwpsYb3inyZSLYC6eV+TnP2VVpGRCKABGB3Fc815j/S6sdy34Bsvly9g+lr7Fuj8a7I8DD+32Ud2LrvCC/PsBHvFfkykSwEskUkU0Si8DSe51QokwPc5GxfCXytnnkjcoARTq+uTCAbWODDWE0QuM351vj4v1ZxsLjU7XBMkDm7RTI/65rGazPzbdLQCnyWSJw2j1HAF8AaYJKqrhKRp0TkUqfYeCBZRPKAB4ExzrmrgEnAauBzYKSqlgGIyD+AeUBrESkQkdt8dQ8msERFhPHMzzqybf8Rnp1qY0uM9/12WFsSY6N4ZPJym2q+nAhfXlxVpwBTKux7vNz2UeCqk5z7NPB0Jfuv9XKYJoh0a16fW87N5M05G7i4Y2POaZHsdkgmiCTGRvF/w9tz9/tLeP3bfO65IMvtkPxC0Da2m9D18OBWNEuKZczkFdYwarxu6FmNGdK+EX/5ah3rbVJHwBKJCUKxURE8c8VZbNx9mBe+stUUjfc9dVl7YiLCGDN5BcePW0duSyQmKJ2blcK1PdN549t8lm3Z53Y4lbLlSAJXg7gYfjusHQs37uXdeRvdDsd1lkhM0Hr0orY0jI/hwUnL/PcRly1IErCu7JbGBa1T+cPUH8jbGdqPuCyRmKAVHxPJn6/qRH7hIVtXwnidiPDHn3UkNiqc+ycuDem53iyRmKDWOyuFX/TJ5L3vNjHjh51uh2OCTIP4GP5wRUdWbj3Ai9NDtz3OEokJeg8Pbk2bRnH88qMV7D5Y7HY4JsgM6dCIq7ql8eo361m0MTTnk7VEYoJeTGQ4fxnRmQNHjvGryd+j1sptvOx3l7YnrX4sD0xaRtHRY26HU+sskZiQ0KZRPI8Mac1Xa3bwztyNbodjgky96AheuKYT2/YdZczHofdlxRKJCRm39s5kQJsGPD1lDUs373U7HBNkujVP4uFBrfn3ih9577tNbodTqyyRmJARFiY8d3UnGsTFMOqDpew9VOJqPKH1nTU03Hl+C/q3acD/fbaa5X46fskXLJGYkJIYG8UrP+9KYVExD05a5vqoZBtFElzCwoTnrvJ8Wbnn/SXsPxwa7SWWSEzI6ZSeyG+GtWVGbiFjZ+S5HY4JMvXrRvHydV3YWXSU+ycupSwEplCxRGJC0g3nNOeyzk14btpaPl+53e1wTJDp0qw+v7ukPTNyC/nj58G/pIElEhOSRIRnftaRTumJPDBxGau27Xc7JBNkrj+nOTf2as5rs/L5aHGB2+H4lCUSE7JiIsP52w3dSIyN5PZ3FlFYZIMVjXf9dlg7emcl8+uPv2fxpuAdrGiJxIS0BvEx/O3G7uw5XMLt7y7icIkt0Wu8JzI8jLHXdaVJYgx3vLuYDbsOuR2ST1giMSGvQ9MEXhrRhRUF+7j770s4ZkuoGi9KjI3izZt7oMAN4+ez48BRt0PyOkskxgCD2jfiD1ecxcy1hfzyw+W10i041EY/h7IWqfV4+5Ye7D1Uwk1vLmD/keDqFmyJxBjHNT2a8ciQ1vxz2Tae+mx1rfyht+VIQkfHtEReu6E76wsP8ot3FgbVY1RLJMaUc3fflvyiTyZvz91Ya8nEhI4+2Sm8cE1nFm/ay81vLuRQcXAkkwi3AzDGn4gIj13cFgXGz95A2XHlyUvbI1Z1MF4yrGMTVOH+icu4+a0FvHVLT+pFB/af4sCO3hgfEBF+c3FbIsKE12blc6zsOP83vAMR4VaBN95xSacmhIlw34Sl3Dh+PuNv6kH9ulFuh3XG7F+GMZUQEcYMbcOofln8Y8EW7nxvcVA90zbuu7hjY8Ze15WV2w7ws3Fz2bLnsNshnTFLJMachIjw8ODW/N9lHZiRu5MRr39ngxaNVw3p0Ii/33Y2uw+WcPkrcwJ2xmBLJMacxg3nNOf1G7qzbsdBLvnrbBZvsrVMjPf0zExi8t29iI4I56rX5jFp4Ra3Q6o2SyTGVMHAdg356O5eREWEcc1r83hz9gbr0WW8JqtBHDmjetMjoz6PTF7BmMkrOHqszO2wqswSiTFV1L5JAp/e24cLWjfgqc9Wc+vbC9m+v2ajlK0vmDkhuV407956NiP7tWTCwi0M++vsgHnUZYnEmGpIqBPJ6zd043eXtGNe/m4ufGEmkxZucX2BLBMcwsOEXw5uw7u39uRQcSmXvzKHZ6b+4PcdPSyRGFNNYWHCLb0z+Xz0+bRtHM8jk1cwfOwc5ufvdjs0EyTOb5XKFw+cz5Xd0hg3cz39/zyTyYsL/PYLiyUSY85QRkpdJtx+Di9c04ldB4u55vXvuGH8fObm7bL2E1Nj8TGR/PHKTnx0Vy8axkfz0IfLGfrit3y8pMDvJhb1aSIRkSEikisieSIyppLj0SIy0Tk+X0Qyyh171NmfKyKDq3pNY2pTWJhweZc0vn7oAh4Z0po1PxZx3Rvzufil2YyfvYGdRcE306upXd0zkvjknt68OKIzAA9OWs75f5zBHz//gbydRS5H5yG++uYkIuHAWuBCoABYCFyrqqvLlbkH6Kiqd4nICOByVb1GRNoB/wB6Ak2Ar4BWzmmnvGZlunfvrosWLfLq/RlTmaPHyvhk6VY+mL+Z77fuJ0w8k/X1zkqma7P6tGoYR9PEOoSFCTeMn8+h4lI+vqe322GbAKGqzMjdybvzNvHtul2UHVdapNSld1YKPTKTaN0wjsyUukRF1LyOICKLVbV7Vcr6coqUnkCequY7QU0AhgPl/+gPB55wtj8CXhbPpEbDgQmqWgxsEJE853pU4ZrGuCYmMpxrezbj2p7NyNtZxKfLf2R23i7GzcynzHm+HREmJNSJZPehEro2S3Q5YhNIRIT+bRrSv01DdhYd5d8rfmTW2kImLyngve82OWU8j8US6kTSKD6GSXf18nlcvkwkTYHyI2sKgLNPVkZVS0VkP5Ds7P+uwrlNne3TXRMAEbkDuAOgWbNmZ3YHxtRAVoM4HrgwjgcubMXB4lJytxeRt7OITbsPs//IMQ4cLWVg2wZuh2kCVIO4GG7pncktvTMpKT1O3s6DrNtZRH7hIfYdLmH/kWPUiQqvlViCdtJGVX0deB08j7ZcDseEuHrREXRrXp9uzeu7HYoJQlERYbRrEk+7JvGuvL8vG9u3Aunlfk5z9lVaRkQigARg9ynOrco1jTHG1CJfJpKFQLaIZIpIFDACyKlQJge4ydm+EvhaPa3/OcAIp1dXJpANLKjiNY0xxtQinz3acto8RgFfAOHAm6q6SkSeAhapag4wHnjPaUzfgycx4JSbhKcRvRQYqaplAJVd01f3YIwx5vR81v3Xn1j3X2OMqZ7qdP+1ke3GGGNqxBKJMcaYGrFEYowxpkYskRhjjKmRkGhsF5FCYNMZnp4C7PJiOLUt0OMHuwd/EOjxg91DdTVX1dSqFAyJRFITIrKoqj0X/FGgxw92D/4g0OMHuwdfskdbxhhjasQSiTHGmBqxRHJ6r7sdQA0Fevxg9+APAj1+sHvwGWsjMcYYUyNWIzHGGFMjlkiMMcbUiCWSkxCRISKSKyJ5IjLG7XhORUQ2isj3IrJMRBY5+5JEZJqIrHP+W9/ZLyLyknNfK0Skq0sxvykiO0VkZbl91Y5ZRG5yyq8TkZsqe69ajP8JEdnqfA7LROSicscedeLPFZHB5fa79nsmIukiMkNEVovIKhEZ7ewPiM/hFPEHzOcgIjEiskBEljv38KSzP1NE5jvxTHSWzcBZWmOis3++iGSc7t5qharaq8ILzxT164EWQBSwHGjndlyniHcjkFJh3x+BMc72GOBZZ/siYCogwDnAfJdiPh/oCqw805iBJCDf+W99Z7u+i/E/ATxcSdl2zu9QNJDp/G6Fu/17BjQGujrbccBaJ9aA+BxOEX/AfA7O/8t6znYkMN/5fzsJGOHsHwfc7WzfA4xztkcAE091b7X1u2Q1ksr1BPJUNV9VS4AJwHCXY6qu4cA7zvY7wGXl9r+rHt8BiSLSuLaDU9VZeNagKa+6MQ8GpqnqHlXdC0wDhvg++pPGfzLDgQmqWqyqG4A8PL9jrv6eqeqPqrrE2S4C1gBNCZDP4RTxn4zffQ7O/8uDzo+RzkuB/sBHzv6Kn8GJz+YjYICICCe/t1phiaRyTYEt5X4u4NS/oG5T4EsRWSwidzj7Gqrqj872dqChs+3P91bdmP3xXkY5j33ePPFIiACI33lE0gXPN+KA+xwqxA8B9DmISLiILAN24knC64F9qlpaSTz/idU5vh9IxuV7sEQSHPqoaldgKDBSRM4vf1A9dd+A6ucdiDEDrwItgc7Aj8Bz7oZTNSJSD5gM3K+qB8ofC4TPoZL4A+pzUNUyVe0MpOGpRbRxOaRqs0RSua1Aermf05x9fklVtzr/3Ql8gueXcceJR1bOf3c6xf353qobs1/di6rucP4oHAf+xn8fLfht/CISieeP8Puq+rGzO2A+h8riD8TPAUBV9wEzgF54HhueWAq9fDz/idU5ngDsxuV7sERSuYVAttNzIgpPo1aOyzFVSkTqikjciW1gELAST7wnes/cBPzL2c4BbnR64JwD7C/3GMNt1Y35C2CQiNR3Hl8Mcva5okJb0+V4PgfwxD/C6XGTCWQDC3D598x5tj4eWKOqz5c7FBCfw8niD6TPQURSRSTR2a4DXIinrWcGcKVTrOJncOKzuRL42qk1nuzeakdtteoH2gtPD5W1eJ5XPuZ2PKeIswWe3hrLgVUnYsXz3HQ6sA74Ckhy9gsw1rmv74HuLsX9DzyPHY7heZ5725nEDNyKp2ExD7jF5fjfc+JbgecfduNy5R9z4s8FhvrD7xnQB89jqxXAMud1UaB8DqeIP2A+B6AjsNSJdSXwuLO/BZ5EkAd8CEQ7+2Ocn/Oc4y1Od2+18bIpUowxxtSIPdoyxhhTI5ZIjDHG1IglEmOMMTViicQYY0yNWCIxxhhTI5ZIjDkNEZnr/DdDRK7z8rV/Xdl7GRNIrPuvMVUkIhfgmVV2WDXOidD/zplU2fGDqlrPG/EZ4xarkRhzGiJyYnbWZ4DznDUuHnAm2/uTiCx0Jgi80yl/gYh8KyI5wGpn3z+dSTVXnZhYU0SeAeo413u//Hs5o8f/JCIrxbPWzDXlrv2NiHwkIj+IyPvOCG9E5BnxrM2xQkT+XJv/j0xoizh9EWOMYwzlaiROQtivqj1EJBqYIyJfOmW7Ah3UM6U3wK2quseZBmOhiExW1TEiMko9E/ZVdAWeSQc7ASnOObOcY12A9sA2YA7QW0TW4JkOpI2q6olpN4ypDVYjMebMDcIz99QyPNOXJ+OZ4whgQbkkAnCfiCwHvsMzuV42p9YH+Id6Jh/cAcwEepS7doF6JiVcBmTgmU78KDBeRK4ADtf47oypIkskxpw5Ae5V1c7OK1NVT9RIDv2nkKdtZSDQS1U74ZlbKaYG71tcbrsMONEO0xPPYkfDgM9rcH1jqsUSiTFVV4RnSdcTvgDudqYyR0RaOTMwV5QA7FXVwyLSBs9SqiccO3F+Bd8C1zjtMKl4lvY96WyuzpocCao6BXgAzyMxY2qFtZEYU3UrgDLnEdXbwIt4HistcRq8C/nvkqjlfQ7c5bRj5OJ5vHXC68AKEVmiqj8vt/8TPOtSLMczw+0jqrrdSUSViQP+JSIxeGpKD57ZLRpTfdb91xhjTI3Yoy1jjDE1YonEGGNMjVgiMcYYUyOWSIwxxtSIJRJjjDE1YonEGGNMjVgiMcYYUyP/H4Wn6DNN4HnJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.plot('lr')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**cycle_mult:** The cycle_mult parameter allows you to increase the cycle length as training progresses.  For instance, cycle_mult=2 will double the length of the cycle.  In the example below, seven epochs are run:\n",
    "- first cycle has length of one epoch\n",
    "- second cycle has length two epochs\n",
    "- third cycle has length of four epochs\n",
    "Each cycle will begin at a learning rate of 5e-3 and gradually decrease until it resets at the beginning of the next cycle.\n",
    "\n",
    "Note that the example below overfits. It is shown to merely illustrate the *cycle_mult* parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/7\n",
      "25000/25000 [==============================] - 6s 235us/step - loss: 0.4295 - acc: 0.8241 - val_loss: 0.3478 - val_acc: 0.8704\n",
      "Epoch 2/7\n",
      "25000/25000 [==============================] - 6s 226us/step - loss: 0.2622 - acc: 0.9010 - val_loss: 0.2829 - val_acc: 0.8879\n",
      "Epoch 3/7\n",
      "25000/25000 [==============================] - 6s 225us/step - loss: 0.1782 - acc: 0.9408 - val_loss: 0.2776 - val_acc: 0.8900\n",
      "Epoch 4/7\n",
      "25000/25000 [==============================] - 6s 228us/step - loss: 0.1701 - acc: 0.9390 - val_loss: 0.2962 - val_acc: 0.8862\n",
      "Epoch 5/7\n",
      "25000/25000 [==============================] - 6s 227us/step - loss: 0.1170 - acc: 0.9613 - val_loss: 0.3159 - val_acc: 0.8832\n",
      "Epoch 6/7\n",
      "25000/25000 [==============================] - 6s 223us/step - loss: 0.0848 - acc: 0.9745 - val_loss: 0.3328 - val_acc: 0.8800\n",
      "Epoch 7/7\n",
      "25000/25000 [==============================] - 6s 222us/step - loss: 0.0704 - acc: 0.9816 - val_loss: 0.3355 - val_acc: 0.8809\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fe7efaa9438>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# rebuild the model to train from scratch \n",
    "learner.set_model(get_model())\n",
    "\n",
    "# training using cycle_len \n",
    "learner.fit(5e-3, 3, cycle_len=1, cycle_mult=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is what the learning rate schedule looks like when using the **cycle_mult** parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl4VNX5wPHvO5N93wkkZCMhEPZFZHFBUUFFaa22VNvS1v60rXbRblhbbW1ptbWLbdWWaltrbcWlVlREEcSVXdkhEHaQJYQtbAlJ3t8fc6NpGiDb5M7yfp5nHmfuPffOe+Iw79xzzj1HVBVjjDGmvTxuB2CMMSa4WSIxxhjTIZZIjDHGdIglEmOMMR1iicQYY0yHWCIxxhjTIZZIjOlCIlIgIioiEZ10vq0icklnlzWmLSyRmLByui9TERkrIg0iclREqkWkXES+cJZz3Sgi653ye0Vklogk+i96YwJTp/wqMiZEfKCquSIiwOXATBF5V1XLmxcUkQuBnwETVPV9EUkDrurieI0JCHZFYkwz6jMLOAAMPE2xc4AFqvq+c8wBVX1MVasBRCRWRH4lIttE5LCIvC0isU2Ov0FEtovIfhG5s3GjiHhEZKqIbBKRKhF5yklSjfs/65yzqulxzr6/ichPm7weKyI7Wwr+bO9jTFtYIjGmGedL9mogA6g4TbFFwHgR+bGIjBGR6Gb77weGAaOBNOC7QEOT/ecBpcA44C4R6ets/xrwMeBCoAdwEHjQiasMeBj4rLMvHchtZzVP+z7GtJUlEmM+0kNEDgEngOeA2xuvOJpT1beAa4ChwEtAlYj8WkS8IuIBvgh8Q1V3qWq9qr6rqjVNTvFjVT2hqiuAFcAgZ/uXgTtVdadT/kfAtU7n/LXAi6r6prPvh/x3cmqLM72PMW1iHxpjPtLYRxIN3AtcDPz2dIVV9WXgZSdxXAQ8DZTjS0IxwKYzvNeeJs+PAwnO83zgORFpmiDqgW74rhx2NHn/YyJS1cq6NXem99nVznOaMGVXJMY04/xC/x4wQEQ+1oryDao6F5gH9Af2AyeBXu14+x3A5aqa0uQRo6q7gN1Az8aCIhKHr3mr0TEgrsnr7Ha+jzFtYonEhKNIEYlp8vifK3NVrQV+BdzV0glEZJKITBaRVPEZga+/YaGqNgB/AX4tIj2c5q5RLfSjtOSPwDQRyXfeJ1NEJjn7ngEmish5IhIF3MN//xteDlwhImkikg18s53vY0ybWCIx4WgWvn6QxsePTlPuL0CeiLQ0rPcg8H/ARuAI8A/gl6r6hLP/28AqYAm+0V/30bp/bw8AM4FXRaQaWAicC6Cqa4BbgH/iuzo5CDQdlfU4vv6WrcCrwIz2vI8xbSW2sJUxxpiOsCsSY4wxHWKJxBhjTIdYIjHGGNMhlkiMMcZ0SFjckJiRkaEFBQVuh2GMMUFj2bJl+1U1szVlwyKRFBQUsHTpUrfDMMaYoCEi21pb1pq2jDHGdIglEmOMMR1iicQYY0yHWCIxxhjTIZZIjDHGdIhfE4mITBCRchGpEJGpLeyPFpEZzv5FIlLQZN8dzvZyERnfZPtWEVklIstFxIZiGWOMy/w2/FdEvPiW7rwU3wylS0RkpqqubVLsRuCgqhaLyGR8M6R+yllSdDLQD99iPq+JSG9VrXeOu0hV9/srdmOMMa3nz/tIRgAVqroZQESeBCYBTRPJJD6awvsZ4A8iIs72J50FhraISIVzvgV+jPd//G7uRrKTY5g4sDtxUYF9y03V0RpmLN1BSmwUg3om0zc7CY9H3A4rJFRW1zBjyXYivR7ioiNIj4+iZ2oceWlxJMdFuh2eMa7z57djDk2WBcV3VdJ8vYMPy6hqnYgcxrfiWw6+9RGaHpvjPFd8aygo8CdVnd7Sm4vITcBNAHl5eW0O/lR9A39+azPVJ+t44LWNPPbFERRnJZz9QJe8vHoPv5hd/uHrnJRYvnheIZ8ZmUd0hNfFyILfiys/4P5XN7S4Lzc1lkE9UxiWl8rFfbIoyIjv4uiMcV9g/8xu2XmquktEsoA5IrJeVd9sXshJMNMBhg8f3uZFVyK9HlbcdRmLthzga/96j8//dTEv3HoeqfFRHa+BHzQ468o8f8sYKvYdZcbSHfzkxbU8sWgb9183iKF5qS5HGLzqG3x/20XfH4cI7K+uZfuB42yrOsbKnYdZvv0QL63czT0vrqVXZjxXDOjOdcN6kpced5YzGxMa/JlIdtFkfWkg19nWUpmdznKnyUDVmY5tXFNaVfeJyHP4mrz+J5F0Bo9HGNUrnUemnMO1D7/Lz2at45fXDfLHW3Waxl/InxiWy/zyffzgP6v51J8W8OOr+3P9uW2/MjMfiY3ykhQTSVZiDGU9kv5r3/aq48xdv5fX1u3lwdcr+P28CkYVpfP5MQVc2rebNTOakObPUVtLgBIRKXTWl56Mb2nPpmYCU5zn1wLz1Ldk40xgsjOqqxAoARaLSLyIJAKISDxwGbDaj3UAYHDPFG48r5Cnl+1k1c7D/n67TjO2NIuXvnY+o3tl8P3nVvHw/E1uhxSy8tLj+MKYQp740kjemXox376sNzsOHufmx5dx2W/f5JllO6mrb3A7TGP8wm+JRFXrgFuBV4B1wFOqukZE7hGRq51ijwLpTmf67cBU59g1wFP4OuZnA7c4I7a6AW+LyApgMfCSqs72Vx2auvXiYhJjInhofkVXvF2nSY6L5NEpw7l6UA/um72eR97a7HZIIa97ciy3XlzC/G+P5YHJg4nwCN9+egVX/u5t3thQ6XZ4xnQ6v/aRqOosYFazbXc1eX4SuO40x04DpjXbthlwpW0pMSaSz43K56H5m9iy/xiFQdSpGuH18JtPDaa+QZk2ax090+IY3y/b7bBCXoTXw6TBOVw9qAcvr97DfbPXM+Uvi7mwdyY/mdTf+lBMyLA729tgyqgCPCI8tXTH2QsHGK9H+NUnBzEoN4VvPrmcDXur3Q4pbIgIVwzozqu3XcAPruzLsm0HGf/bN3nkrc0fduQbE8wskbRBVlIMF/bO5N/v7QzKL4CYSC/TPzeM+GgvX/vn+5w8VX/2g0yniY7w8qXzi3j1tgsY3Sudn760jk88/C7bqo65HZoxHWKJpI2uG5bL3iM1vF0RnDfWZyXGcP91gyjfW820l9a5HU5Y6pESyyNThvPA5MFsrjzKlb97mxdWfOB2WMa0myWSNrq4bxYJ0RHMXr3b7VDabWxpFl8YU8DjC7exeMsBt8MJSyLCpME5zPrG+ZRmJ/K1f73P1GdXUlNnV4km+FgiaaPoCC8XlmYyZ+3eoGzeavSd8aXkpMRyx7/ty8tNualxzLhpJLdc1Isnl+zg09MXsq/6pNthGdMmlkja4bKybuw/WsvyHQfdDqXd4qIi+OnH+7Op8hh/nG9Dgt0U4fXwnfF9ePiGoazbXc2kP7zD6l3Bc7+SMZZI2uGiPllEeoVX1+x1O5QOuag0iysHdOfhNyrYc9h+Bbvt8gHdeeYro/CIcO0f32Xe+uD+fJnwYYmkHZJiIhmWn8pbG4Ozw72pqZf3oaEBfj2n/OyFjd/165HMf24ZQ0lWIv/392U8u2yn2yEZc1aWSNrpvOIM1u4+woFjtW6HAoC2s7umZ1ocU0bn8/SynazbfaRzgzLtkpkYzb9uGsnIojS+9fQK/vymNT2awGaJpJ1GF2cA8O6mwLoq8S3n0ja3XlRCUkwk981e74eIQkdXTruYEB3BXz5/DlcO7M60Wev4zZyWp7E3JhBYImmngTnJJEZH8E6Q3k/SVHJcJDdfWMT88kpW7DjkdjjGER3h5XeTh/DJ4bk8MHejJRMTsCyRtFOE18PIXum8U1Hldiid4nOjCkiJi+T38za6HYppwusR7r1mINcNs2RiApclkg44tzCN7QeOs/dI8I94SoiO4MYxhby2bp8NPQ0wHo9w3yc+Sia/m2vJ3gQWSyQdMLwgDYBl24L3fpKmpowpICkmwr6oAlBjMrlmaA6/nrOBxxdsdTskYz5kiaQDyronER3hYenW0EgkSTGRTBldwJx1e9my3yYSDDQej/CLTwzkkr7duGvmGl5cafNzmcBgiaQDoiI8DMpNYdn20EgkAJ8dlU+kx8Nf39nidiimBRFeD3+4fgjD81O5bcZy3tpoC2UZ91ki6aBhBams2XWYE7WhMV9VVmIMVw/uwdNLd3L4+Cm3wzEtiIn08siUc+iVmcDNjy+zPi3jOkskHTQsL5W6BmXlztAZNvvFMYWcOFXPPxdvdzsUcxrJsZH8/YsjSI2L4sbHltgUN8ZVlkg6aGh+KgDvbQ+dRFLWI4kxxek89u5WTtU3uB2OOY2spBgemTKcoyfr+NLfl3C8ts7tkEyYskTSQWnxUfRMi2XVrtBJJABfGF3IniMnmbvOJg4MZH27J/H764ew9oMjfPPJ5TQE8dIGJnhZIukEA3KSWRVi7dQX9cmiR3IMTyyy5q1Ad3GfbvzgyjJeXbuX+16xaW5M17NE0gn65ySz48AJDh0PjAkcO4PXI3zqnDze2rjf1hQPAl8YU8AN5+bxpzc2M9OW7TVdzBJJJxiYkwLA6l3uzZ6r7Z3+9ww+dU5PvB7hX4t3dPq5g4kf/rSdTkS4+6p+DM9P5XvPrKR8T7XbIZkwYomkE/TPSQJgZQD0k3TmDLXZyTGM65PFM8t2UFtnne7tmVm5K0VFeHjohqEkxETw5X8s48hJG75tuoYlkk6QEufrcA/F8fzXn5vH/qO1vLp2j9uhmFbISorhoRuGsuPAcW6fscI6302XsETSSQbmpIRchzvABSWZ5KTE8tRSW6kvWJxTkMadV/bltXV7efD1CrfDMWHAEkknCcUOd/DN73TN0Bze3lgZErMch4vPjy7gY4N78JvXNrBgU2gsdWAClyWSTlLWw9dPsm536HVyXjM0lwaF/7y/y+1QTCuJCNM+PoCC9Hi+OeP9gFkS2oQmSySdpG92IgDle0Jv3fPCjHiG5qXw7Hs7/TI6zPhHfHQEv79+CAePneLbT6+w/3fGbyyRdJLMxGjS4qNYH6LDLq8ZmsuGvUdZ80HoJcpQ1q9HMnde2Zd56/fx6Ns2o7PxD0sknUREKO2WyLoQTSRXDexBlNfDM8us0z3YfG5UPpeWdeO+2etDanJREzj8mkhEZIKIlItIhYhMbWF/tIjMcPYvEpGCJvvucLaXi8j4Zsd5ReR9EXnRn/G3VZ/uiWzYUx2SQy6T4yK5pCyLmSs+sIkcg4yI8MtrB5KZEM3X/vU+R2tsckfTufyWSETECzwIXA6UAZ8WkbJmxW4EDqpqMfAb4D7n2DJgMtAPmAA85Jyv0TeAdf6Kvb36Zidx4lQ92w8cdzsUv7hmSC4HjtXy5gZbTCnYpMRF8dvJQ9h+4Dg/fXGt2+GYEOPPK5IRQIWqblbVWuBJYFKzMpOAx5znzwDjxHf78CTgSVWtUdUtQIVzPkQkF7gSeMSPsbdLqdPhvj4EO9wBLuidSXJsJC+u3O12KKYdRhSmcfMFvXhyyQ5eW2uzOpvO489EkgM0naRpp7OtxTKqWgccBtLPcuxvge8CZ2xfEZGbRGSpiCytrOyaX9C9uyUiQsh2uEdFeJjQL5s5a/dy8lRorAgZbm67tIQ+2YlM/fdKqo7WuB2OCRFB1dkuIhOBfaq67GxlVXW6qg5X1eGZmZldEB3ERnkpTI9nfQjeS9Jo4qDuHK2pY365NW8Fo+gIL7+dPJgjJ+r4/nOrbEiw6RT+TCS7gJ5NXuc621osIyIRQDJQdYZjxwBXi8hWfE1lF4vIP/wRfHuVZie60rTVVV8Ho4rSSY+P4oWV4TNVuXbZX7dr9MlO4luX9eaVNXt59j27ydR0nD8TyRKgREQKRSQKX+f5zGZlZgJTnOfXAvPU9xNpJjDZGdVVCJQAi1X1DlXNVdUC53zzVPUzfqxDm5VmJ7LtwHFO1LrT9OPvCWojvB4uH5DNvHX7wm5p18Ce+7dtvnR+ESMK0vjRzDXsCNHBIabr+C2ROH0etwKv4Bth9ZSqrhGRe0TkaqfYo0C6iFQAtwNTnWPXAE8Ba4HZwC2qGhSN8iVZiajCpsqjbofiNxMH9uDEqXpeW7fP7VBMO3k9wq8+OQhV5XvPrrQmLtMhfu0jUdVZqtpbVXup6jRn212qOtN5flJVr1PVYlUdoaqbmxw7zTmuVFVfbuHc81V1oj/jb4/irAQgtBPJOQVpZCVG86KtxBfUeqbFcccVfXl3UxUzloT34mWmY4Kqsz0YFGTE4fUIG/eGbiLxeoQrB3Zn/oZKWzwpyF0/Io+RRWlMe2kduw+fcDscE6QskXSy6Agv+WlxVOwL3UQCMHFgd2rrGnh9vTVvBTOPR7jvEwM51dDAD55bbU1cpl0skfhBr6wEKkK4aQtgSM9UMhOjeWWNrZwY7PLT4/n2ZaXMXb+P55dbc6VpO0skflCSlcDW/cdCek4qj0e4rKwbr6+vtJsTQ8AXxhQyJC+FH7+whv12o6JpI0skflCclUBdg7Kt6pjbofjVhP7ZnDhVz1sb97sdiukgr0f4xScGcqymnrtnrnE7HBNkLJH4QePIrVDvJxlZlE5STASzV1vzVigo6ZbI18cV89LK3dZkadrEEokf9Mr0JZJQHrkFEOn1cEnfbsxdvzekm/HCyc0X9qJPdiJ3P7/Gpps3rWaJxA/ioyPISYkN+Q53gMv6ZXPo+CkWbzngdiimE0R6PfzsmgHsrT7Jr1/d4HY4JkhYIvGTXlkJId+0BXBh70xiIj3WFBJChualcv2IPP727hZW7zrsdjgmCFgi8ZPizAQ2VR4NydUSm4qN8jK2dxavrtkb8nUNJ9+d0Ie0+Gi+/9wq6u3/qzkLSyR+UtItgZOnGth1qGvuFnbzPrLx/bux58hJVoToeuDheI9ecmwkd11Vxsqdh3l8wVa3wzEBzhKJnzR2uHd1P4m4MEftxX26EeERXlkT2qvu+Xtm5UBz1cDunF+Swf2vbmDP4ZNuh2MCmCUSPynKjAdgS2Vo30sCvl+v5xalMW99aCeScCMi/PRj/TlV38CPX7B7S8zpWSLxk/T4KBJjItiyP/QTCfiuSjbsPWprW4SY/PR4vj6uhJdX72HuOvuhYFpmicRPRISijPiwSSSX9M0C4DX7sgk5/3d+ESVZCdw9c41Nh2NaZInEjwrDKJHkp8dTnJXAXFvsKuRERXi4Z1J/dh48wcPzN7kdjglAlkj8qDAjgV2HToTNr7hxfbNYtKWKalujJOSM6pXOVYN68PAbm9heZc2X5r9ZIvGjQqfDfWuIT97YaFyfbpyqV5vEMUR9/4o+RHiEe15c63YoJsBYIvGjoozwGbkFMDQvhZS4SOsnCVHdk2P5+rgSXlu31xY0M//FEokfFTiJZHOY9JNEeD1cVJrF/PJKuxs6RH1xTCFFmfH8+IU11NSFR5OtOTtLJH6UEB1BVmJ02HS4g6+f5MCxWt7fftDtUIwfREV4+NFV/dhadZxH3tridjgmQFgi8bNwGrkFcEHvTCI8wlxr+ghZF/TOZEK/bH4/b2OXTQFkApslEj8rykwIq0SSFBPJiMI0u3ktxP3wqjIApr1kHe/GEonfFWXEc+BYLYeO17odSpcZ19fucg91OSmx3HpRMbNW7eFtG6UX9iyR+Flh48gtP1+VBFLXdqjd5R5If9tA8qXzi8hPj+PumatthcwwZ4nEzxrvJemy5q0AmKE2Pz2eoox45pdXuh1Kp3JjZuVAFhPp5YdXlrGp8hiPL9jmdjjGRZZI/Kxnahxej4RVPwnAhaWZLNxcxYlaGyIaysb1zeL8kgx++9oGDhwLn+Zb898skfhZVISHnqmxYXMvSaOLSrOoqWtg4eYqt0MxfiQi/HBiGcdq6/nNHFvjPVxZIukChRnxYXN3e6MRhWnERnqZX27DgENd726JfObcPJ5YtI31e464HY5xgSWSLlCY4RsCrGG0ZmtMpJdRvdKZvyG0+klMy755SW8SYyL5yYtrw+pzbnz8mkhEZIKIlItIhYhMbWF/tIjMcPYvEpGCJvvucLaXi8h4Z1uMiCwWkRUiskZEfuzP+DtLYUYcJ07Vs6+6xu1QutTY0ky2VR0Pu/6hcJQaH8Vtl5TwTkUVc9aGxmg903p+SyQi4gUeBC4HyoBPi0hZs2I3AgdVtRj4DXCfc2wZMBnoB0wAHnLOVwNcrKqDgMHABBEZ6a86dJb8dGcW4DD7Qh3b2zcM2Cb4Cw83jMynJCuBabPW2TxcYcafVyQjgApV3ayqtcCTwKRmZSYBjznPnwHGiYg4259U1RpV3QJUACPU56hTPtJ5BPx1dIGTSLaF2ToOeelxFGXGW/NWmIj0evjhxDK2VR3nb+9sdTsc04X8mUhygB1NXu90trVYRlXrgMNA+pmOFRGviCwH9gFzVHVRS28uIjeJyFIRWVpZ6e4XWY+UGCI8EjbrkjQ1tneWDQMOIxf0zmRcnyx+P6+CyjBryg1nQdfZrqr1qjoYyAVGiEj/05SbrqrDVXV4ZmZm1wbZTITXQ15aXNhdkYCvn6TWhgGHlTuv7EtNXT33v1Ludiimi/gzkewCejZ5netsa7GMiEQAyUBVa45V1UPA6/j6UAJefnpcWHY6Nw4Dft2GAYeNoswEPj+6gKeW7WD1rsNuh2O6gD8TyRKgREQKRSQKX+f5zGZlZgJTnOfXAvPUN3ZwJjDZGdVVCJQAi0UkU0RSAEQkFrgUWO/HOnSa/PR4tlWF1xBgaDIMuLwy7Ooezr42roS0uCjuecGGA4cDvyUSp8/jVuAVYB3wlKquEZF7RORqp9ijQLqIVAC3A1OdY9cATwFrgdnALapaD3QHXheRlfgS1RxVfdFfdehMBelxHKutZ//R8JtGYmxpJtsP2DDgcJIUE8m3Litl8dYDzFq1x+1wjJ9F+PPkqjoLmNVs211Nnp8ErjvNsdOAac22rQSGdH6k/pef0Thy6xiZidGdfv5A/tXnGwa8hvnllRRlJrgdTpsF8J82oH3qnJ78fcFW7p29jnF9s4iJ9LodkvGToOtsD1aNQ4C3+rnDXQJwgtrGYcDB3k8SiH/bQOb1CD+4sowdB07w2Ltb3Q7H+JElki6SkxKL1yNsC8MhwOC7Klm05YANAw4z55VkcHGfLP4wr4KqozYcOFRZIukiUREeclJi/X5FEqgahwEv2Gyr6YWb71/Rh+On6nlg7ka3QzF+YomkC+Wnx4XtFcmHw4DX213u4aY4K5Ebzs3jiUXbqdhX7XY4xg8skXShgvT4sJsFuNFHswHvC8v6h7tvjCshLsrLz2YFxWh900aWSLpQQUY81SfrOHT8lNuhuGJsaSY7DpywYcBhKD0hmlsvKmbe+n28tdGuSkONJZIuVJAeBxCWc27BR7MBh9pa7qZ1powuoGdaLNNeWkd9g12VhhJLJF3ow+nkwzSR5KXHUZRhswGHq5hIL1Mn9GX9nmqeXrrj7AeYoGGJpAv1TItFBLbuD8+RWwAXlmayaHMVJ0/ZMOBwdMWAbIblp/KrORs4WlPndjimk1gi6ULREV56JMeG7cgtgAt7Z1JT18ACmw04LIkIP7iyL5XVNfzpjU1uh2M6iSWSLlaQERe295IAjCxKJzrCwxvWTxK2huSlMmlwD6a/uZkPDp1wOxzTCSyRdLHGWYDD1UezAQf3dCmmY747oQ8Av7Q1S0KCJZIuVpAex8HjpzgcpkOAAcb2zmRr1fGwW8PefCQnJZYbzyvkufd3sWLHIbfDMR101kTiLG1rdxF1ksaRW9sOhO+X6IWlvmHAbwTJ6C3Fhqr6w1fG9iIjIYqfvmRrlgS7syYSZx2QchHJ64J4Ql5XzQIcyAoz4slPj7PmrTCXGBPJ7ZeWsmTrQV5ZY2uWBLPWNm2lAmtEZK6IzGx8+DOwUJXv3JS4zU/NOsEy0/nY3pkssGHAYe+Tw3Mp7ZbIz19eT02dfRaCVWsTyQ+BicA9wK+aPEwbxUR66Z4cE9ZXJABjS7M4eaqBxVsOuB2KcVGE18OdV/ZlW9VxHl+wze1wTDu1aoVEVX3D34GEk/z0uLC9u73RyKJ0oiI8zC+v5ILemW6HY1x0Qe9MxpZm8sDcjVwzNJe0+Ci3QzJtdMYrEhGpFpEjLTyqReRIVwUZagrCfAgwQGyUl3ML05i/wfpJDNx5RV+O19bzO1uzJCidMZGoaqKqJrXwSFTVpK4KMtTkp8ez/2gt1SfDdwgw+Jq3NlceY8eB8G7mM1DSLZHJ5/Tk8YXbqNh31O1wTBvZfSQuaJwFeFvY95P4mrRs9JYBuO3S3sRGern35XVuh2LayBKJCz68lyTME0lRRjw902KD5n4S418ZCdHcclExr63bx7sVtiRzMLFE4oL8MF+XpJGIcGHvTN7dVGVDPw0AXxhTQE5KLD+1NUuCiiUSF8RHR5CZGB32He7gW+zqeG09S7YcdDsUEwBiIr1MvbwPa3cf4dllO90Ox7SSJRKXFKSH9yzAjUYXpxPl9Vg/ifnQxIHdGZqXwi9fLbc1S4KEJRKXhPsswI3ioiIYUZhm/STmQyLCDyeWUVldw8PzK9wOx7SCJRKXFGbEs/dIDcdr7RfXhb0z2bjvKLtsbQrjGJKXyseH5PDnt7bY8PAgYInEJfk2BPhDgT4M2Camdcd3J5TiEbh3tk0+Hugskbjkw1mAO2nyxmD+sivOSiAnJZb5tmqiaaJ7cixfvrAXL63czZKtNidbILNE4pKPhgB37hWJSLDM//sREeHC0kzerdhPbV2D2+GcVhD+aYPezRf0ontyDPe8sJYGGw4csCyRuCQxJpKMhGi27LfpIMA3rfyx2nqWbrNfnuYjsVFevjehD6t2Hebf7+9yOxxzGn5NJCIyQUTKRaRCRKa2sD9aRGY4+xeJSEGTfXc428tFZLyzraeIvC4ia0VkjYh8w5/x+1thRhxb91sfCcDo4gwivcIb1rxlmrl6UA8G90zhF7PXc8yGAwckvyUSEfECDwKXA2XAp0WkrFmxG4GDqloM/Aa4zzm2DJgM9AMmAA8556sDvqWqZcBI4JYWzhk0CjPi2WJDgAFIiI5geH6a9ZOY/+HxCHddVca+6hoenr/J7XBMC/x5RTICqFDVzapaCzzGSTNHAAAavElEQVQJTGpWZhLwmPP8GWCc+Br5JwFPqmqNqm4BKoARqrpbVd8DUNVqYB2Q48c6+FVBRjyV1TVhPwtwo7GlmZTvrWb3YRsGbP7b0LxUPja4B9Pf2szOg3YVH2j8mUhygB1NXu/kf7/0PyyjqnXAYSC9Ncc6zWBDgEUtvbmI3CQiS0VkaWVlYP7KLcqwyRubGluaBWDNW6ZF353Qxzcc+GUbDhxogrKzXUQSgGeBb6pqiwtsqep0VR2uqsMzMwNzBb4CJ5Fs9tP67cGmd7cEspNirHnLtKhHSiw3X9CLF1fuZqkNBw4o/kwku4CeTV7nOttaLCMiEUAyUHWmY0UkEl8SeUJV/+2XyLtIflrn3ksS7ESEsaWZvFOxn1P1gTsM2Ljn5guLyE6K4Z4XbThwIPFnIlkClIhIoYhE4es8n9mszExgivP8WmCeqqqzfbIzqqsQKAEWO/0njwLrVPXXfoy9S8RGeemRHMMWSyQfGluaSXVNHe9ts9mAzf+Ki4rge5eXsnLnYZ6z4cABw2+JxOnzuBV4BV+n+FOqukZE7hGRq51ijwLpIlIB3A5MdY5dAzwFrAVmA7eoaj0wBvgscLGILHceV/irDl2hICPeEkkTo4sziPAI820SR3MakwblMKhnCr94xYYDBwq/9pGo6ixV7a2qvVR1mrPtLlWd6Tw/qarXqWqxqo5Q1c1Njp3mHFeqqi87295WVVHVgao62HnM8mcd/K3QEsl/SYqJZGh+qvWTmNPyeIS7Jpax90gNf3zDhgMHgqDsbA8lhRnxHD5xioPHat0OJWCMLc1k3e4j7D1y0u1QTIAalp/K1YN6MP3NzTY7cACwROKyQmfklt2Y+JGxvW0YsDm7qZf3wSPCT19a63YoYc8SicsahwBvqexYIlFCZwRL3+6JdEuKtsWuzBn1SInl1ouLeWXNXt60z4qrLJG4rGdqHB6BrZ10RRIKE9SKCBf2zuStjZXUBdAwYAmJv25o+dL5hRSkx/GjF9YE9MzRoc4SicuiIjz0TIuzmxKbubB3FkdO1vH+jkNuh2ICWHSEl7uv7sfmymP89Z0tbocTtiyRBICC9Hi7KbGZ80oy8HokYFdNNIHjotIsLunbjd/N3ciewzZAww2WSAJA4xBgDeZlDjtZcmwkQ/NSmLfe2r7N2d01sYxTDcrPX17ndihhyRJJACjMiOd4bT2V1TVuhxJQLunbjXW7j7DrkM0GbM4sLz2OL19QxPPLP2DR5iq3wwk7lkgCwIcjt6x5679cUtYNgLnr9rociQkGXxlbTE5KLHfPXBNQgzTCgSWSAFBkiaRFvTITKMqIZ85aSyTm7GKjvPxwYl/W76nmHwu3uR1OWLFEEgB6pMQS5fXYTYktuKSsGws3V9niX6ZVxvfL5vySDH49ZwP7j1pTcVexRBIAvB4hLz2uwzclhqJL+nbjVL3y5ob9bodigoCIcPdV/TheW899tgBWl7FEEiAK0uM77abEUDI0L4XUuEhes34S00rFWQl86fwinl62k8VbbAGsrmCJJEAUZcazteo49bZYz3+J8Hq4qE8W89bvsw5U02pfH+freL/zuVV2x3sXsEQSIHplxlNb18DOgzaTaXOX9u3G4ROnWLLVFrsyrRMXFcFPPtaPjfuO8ue3Np/9ANMhlkgCRHFWAgAV+466HEngOb93JlFej2vNW3ajaHC6uE83Lu+fze/mbmR7lf1A8ydLJAGiODMRaH8iCeXvuoToCEb1Sue1dXvtS920yV1XlRHhEX74/Gr77PiRJZIAkRwXSUZCdIevSCREJ6i9tKwb26qOu3rFFqp/21DWPTmWb11WyhsbKnlp1W63wwlZlkgCSHFWPBWV1rTVknF9fYtdzbHRW6aNpowuoH9OEj9+YS1H7H4kv7BEEkCKsxKo2HfULsFb0D05lgE5yby6xhKJaRuvR/jZxwdQdbSG+18pdzuckGSJJIAUZyZQfbLOJm88jQn9s1m+4xAf2CSOpo0G5qbwuVEFPL5wG8ttjZtOZ4kkgPSykVtnNKF/NgCzV+9xORITjL51WW+6JcbwvWdW2r0lncwSSQD5cAiw9ZO0qFdmAqXdEi2RmHZJjIlk2sf7U763mgdfr3A7nJBiiSSAZCfFkBAdwSa7Ijmtywdks2TbAfZV20p4pu3G9e3Gxwb34MHXK1i3+4jb4YQMSyQBRETolWkjt87k8v7dUYVXrNPdtNNdV/UjOTaS7z6z0qbd6SSWSAJML2fklmlZ724JFGXG87LdE2DaKS0+insm9WfVrsM88vYWt8MJCZZIAkxxVgJ7j9TYePfTEBGu6N+dhZurqLL1Jkw7XTEgm/H9uvHrORvYZC0AHWaJJMAUZ/o63K2f5PQm9M+mQbGVE027iQg/mdSfmAgPU59dSYPNut0hlkgCjE3eeHb9eiSRlxbHLBu9ZTogKymGu67qx5KtB/nbu1vdDieoWSIJMHlpcURFeNhoieS0RITLB2TzbsV+Dh2v9fv72UQDoesTQ3O4uE8W981eT8W+arfDCVp+TSQiMkFEykWkQkSmtrA/WkRmOPsXiUhBk313ONvLRWR8k+1/EZF9IrLan7G7JcLroSQroc1DE8Ptu27igB7UNSgv21WJ6QAR4d5PDCAuysttM1ZwykZxtYvfEomIeIEHgcuBMuDTIlLWrNiNwEFVLQZ+A9znHFsGTAb6AROAh5zzAfzN2RaySrMTKd/Tvl9HQnhMUds/J4mijHieX76ry94zPP6y4ScrMYafXzOAVbsO8/t5dqNie/jzimQEUKGqm1W1FngSmNSszCTgMef5M8A4ERFn+5OqWqOqW4AK53yo6ptASC/E3Dc7iX3VNRw45v9mm2AlIkwanMOiLQfYfdjm3jIdM6F/d64ZmsODr1fw/nZbibOt/JlIcoAdTV7vdLa1WEZV64DDQHorjz0jEblJRJaKyNLKyso2hu6u0mzfIlfr99idt2cyaXAPVOGFFR+4HYoJAT+6uh/ZSTHc/tQKjtfWuR1OUAnZznZVna6qw1V1eGZmptvhtEmf7k4i2W2df2dSkBHPoJ4pPL/cEonpuKSYSO6/bhBbq47x81nr3Q4nqPgzkewCejZ5netsa7GMiEQAyUBVK48NWZkJ0aTHR9kVSStMGtSDNR8csRE3plOM6pXOjWMKeXzhNrtPqQ38mUiWACUiUigiUfg6z2c2KzMTmOI8vxaYp75VnWYCk51RXYVACbDYj7EGFBHpUId7OJk4qDsewa5KTKf5zoRS+vVI4jvPrLC1b1rJb4nE6fO4FXgFWAc8paprROQeEbnaKfYokC4iFcDtwFTn2DXAU8BaYDZwi6rWA4jIv4AFQKmI7BSRG/1VBzf1yU6ifG819XbH7RllJcYwpjiD55d/YCtLmk4RHeHlD9cP5VRdA9948n2b2LEV/NpHoqqzVLW3qvZS1WnOtrtUdabz/KSqXqeqxao6QlU3Nzl2mnNcqaq+3GT7p1W1u6pGqmquqj7qzzq4pU92IidPNbD9wHG3Qwl4kwbnsP3AcZZts9E2pnMUZsQz7eMDWLL1IA/M3eh2OAEvZDvbg91HHe7WT3I2l/fPJj7Ky4wlO85e2JhW+tiQHK4blssfXq/g7Y373Q4noFkiCVAlWYl4BFt8pxXioyOYOLAHL63azdEaG7ZpOs+PJ/WjV2YC35yxnMpqm236dCyRBKjYKC+9MhNY/YElktb45Dk9OV5bz0srrdPddJ64qAj+cP0Qqk+e4tZ/vmdTqJyGJZIANiA3mZU7D1sncisMzUuhOCvBmrdMp+uTncS9nxjAoi0HuPdlu7+kJZZIAtjAnGT2H61hzxFbn/xsRIRPDs/lve2HOv2eEkvj5uNDcvn86AIefXtLl87vFiwskQSwAbkpAKzaefisZe2iBa4ZmkuER+yqxPjFnVf2ZURBGt97dqX1XTZjiSSAlXVPwusRVu06eyJpJGE8RW1GQjTj+mbx7Hu7OHmqvtPPL+H8xzVEej384YYhJMdGcvPjy7pkLZxgYYkkgMVGeSnJSmBlK65IjM9nRxZw4FgtL67c7XYoJgRlJcbw0A3D2H34BF994j1q66zzHSyRBLwBOcms2mUd7q01pjid4qwEHnt3q/3NjF8My0/l3msG8u6mKn74n9X2OcMSScAbmJvMgWO17LI5f1pFRJgyKp9Vuw7z3vZDbodjQtQnhuXy9YuLmbF0B398Y/PZDwhxlkgCXGOHuzVvtd41Q3NJjI7gsXe3uh2KCWG3Xdqbqwf14L7Z65m1KrybUi2RBLi+3ROJjvDYPFJtEB8dwbXDc5m1ajf7bOi08RMR4RfXDmRYfiq3zVjOsm0hvXDrGVkiCXDREV4G9Uxh6dbw/ZC2x5RRBdSr8tiCrW6HYkJYTKSX6Z8dRk5KLF/465KwHRZsiSQInFOQyuoPjtjyn21QkBHP5f2z+fuCbRw5ecrtcEwIS0+I5u83jiA+OoLPPrqYbVXH3A6py1kiCQLDC9Kob1CWW+dxm3x1bDHVJ+v4x8JtbodiQlxuahyP3ziC+oYGPvPoIvaGWZOqJZIgMDQvFRFYstX6Sdqif04yF/TO5C9vb/HLDYrGNFWclchjXxzBgaO1fOaRRWE1W7AlkiCQHBtJabdEloZxZ157fXVsL/YfreWppTZtivG/gbkpPPr5c9h58ASTpy8Im8EelkiCxDkFaSzbdtCmsW6jcwvTGJ6fyoOvV3Ci1q5KjP+NLErnb184h92HTzJ5+kL2HA79ZGKJJEiMKc7geG0979kw4DYREb47oQ97j9TwN7uvxHSRc4vS+fsXR7CvuobJ0xeE/A3FlkiCxOjidLwe4c2NlS3uV5vs/LRGFKZxcZ8sHp5fweHjbR/BZTNgmPYYXpDG328cQdWxWq556J2QHhpsiSRIJMVEMqRnCm9usLWj2+M740uprqnjoTcq2n0Om/vXtNXQvFSe+fJoBOGTf1zAuxWh+e/XEkkQuaB3Jqs/OEzV0fAZDdJZ+nZP4uNDcvjr21vZVHnU7XBMGCnNTuTfXx1N95QYpvx1Mf95P/QWxrJEEkQu6J2JKry1MTR/1fjb1Mv7EB3p4e7n19iMraZL9UiJ5embRzM0L5VvzljOtJfWUhdCA2cskQSRATnJZCZGM3v1HrdDCUpZiTF8Z3wpb1fst/VKTJdLjovk8RvP5XOj8vnzW1uY8tfFHDgWGotjWSIJIl6PcEX/bF4v38exGpsupT1uODefATnJ/GjmmrC6YcwEhqgID/dM6s8vrh3Ikq0HufJ3b7FgU5XbYXWYJZIgc+XAHtTUNTB3/T63QwlKXo9w/3WDqK6p43vPrrQmLuOKTw7vyTNfHkVMpJfrH1nIvS+vD+rVFi2RBJnh+alkJUYzc3noddh1ldLsRKZO6MO89fv4+wKbh8u4Y2BuCi9+7Twmn9OTP76xiY89+A7LdwTnfHqWSIKMxyNcOyyXeev3hfxNTv70+dEFXFSayU9eXBsSTQsmOMVHR/Dzawbyp88OY//RGj7+0Dvc9fzqoJux2hJJELr+3DwA/rnIfk23l8cjPPDpIRRkxPOVJ5ax2YYEGxeN75fN3G9dyJRRBfxj4TYu+uV8/vL2FmrqgmNaH0skQSg3NY5xfbvxxKLtQffLJZAkxUTy6JTheEX49J8XWjIxrkqMieRHV/dj5q3n0ad7Ive8uJaL73+Dfy3eHvCzV1siCVLfGFfCoeOnmP7GZrdDCWr56fH88/9GUlevfPJPC21JY+O6/jnJPPGlkfzjxnPJSIjijn+vYvS98/jVq+V8EKDN2X5NJCIyQUTKRaRCRKa2sD9aRGY4+xeJSEGTfXc428tFZHxrzxku+uckc/WgHkx/c3NIz+HTFUqzE5lx80jio71Mnr6AP76xyWZZNq47rySD/9wyhidvGsmw/FT+8HoFY+6bx6f+tIAnFm1j9+HASSoR/jqxiHiBB4FLgZ3AEhGZqaprmxS7ETioqsUiMhm4D/iUiJQBk4F+QA/gNRHp7RxztnOGjbuvKmPB5iq+9NhSRhalux1OUCvOSmTmLefx7WdWcO/L63lq6Q5uPK+Qy8qy3Q7NhDERYWRROiOL0tledZz/LN/Ff5bv4s7nVgNQlBnPqKJ0ynok0bd7EgXp8aTERuLxdO3McOKvcfQiMgr4kaqOd17fAaCqP29S5hWnzAIRiQD2AJnA1KZlG8s5h53xnC0ZPny4Ll26tPMqF0BW7TzM5/6yiIPOrLbrfzKBmEivy1EFtzlr9/LA3A2s3uW70ouO8FBT18Dmn13R5f9AjWlOVSnfW83bG/fzdsV+lm09SHWTG5QjPEJafBRxUV6yEmN46suj2vU+IrJMVYe3pqzfrkiAHKDpsnQ7gXNPV0ZV60TkMJDubF/Y7Ngc5/nZzgmAiNwE3ASQl5fXvhoEgQG5ybzyzQt4bMFWVLEk0gkuLevGJX2zWL3rCG9X7Gf7gWNkJcZYEjEBQUTok51En+wkvnR+EarKrkMnWLe7ml0Hj7Ovuob9R2s4eaqBuKiu+T7wZyJxlapOB6aD74rE5XD8Kisphu+M7+N2GCFFRBiQm8yA3GS3QzHmjESE3NQ4clPjXIvBn53tu4CeTV7nOttaLOM0bSUDVWc4tjXnNMYY04X8mUiWACUiUigiUfg6z2c2KzMTmOI8vxaYp75Om5nAZGdUVyFQAixu5TmNMcZ0Ib81bTl9HrcCrwBe4C+qukZE7gGWqupM4FHgcRGpAA7gSww45Z4C1gJ1wC2qWg/Q0jn9VQdjjDFn57dRW4EklEdtGWOMP7Rl1Jbd2W6MMaZDLJEYY4zpEEskxhhjOsQSiTHGmA4Ji852EakE2rt4RwawvxPDCSShXDew+gWzUK4bBEf98lU1szUFwyKRdISILG3tyIVgE8p1A6tfMAvlukHo1c+atowxxnSIJRJjjDEdYonk7Ka7HYAfhXLdwOoXzEK5bhBi9bM+EmOMMR1iVyTGGGM6xBKJMcaYDrFEchoiMkFEykWkQkSmuh1Pa4nIX0Rkn4isbrItTUTmiMhG57+pznYRkd85dVwpIkObHDPFKb9RRKa09F5dTUR6isjrIrJWRNaIyDec7aFSvxgRWSwiK5z6/djZXigii5x6zHCWUMBZZmGGs32RiBQ0OdcdzvZyERnvTo3+l4h4ReR9EXnReR1KddsqIqtEZLmILHW2hcRn86xU1R7NHvimqN8EFAFRwAqgzO24Whn7BcBQYHWTbb8ApjrPpwL3Oc+vAF4GBBgJLHK2pwGbnf+mOs9TA6Bu3YGhzvNEYANQFkL1EyDBeR4JLHLifgqY7Gz/I/AV5/lXgT86zycDM5znZc5nNhoodD7LXrfr58R2O/BP4EXndSjVbSuQ0WxbSHw2z/awK5KWjQAqVHWzqtYCTwKTXI6pVVT1TXxruzQ1CXjMef4Y8LEm2/+uPguBFBHpDowH5qjqAVU9CMwBJvg/+jNT1d2q+p7zvBpYB+QQOvVTVT3qvIx0HgpcDDzjbG9ev8Z6PwOMExFxtj+pqjWqugWowPeZdpWI5AJXAo84r4UQqdsZhMRn82wskbQsB9jR5PVOZ1uw6qaqu53ne4BuzvPT1TPg6+80dQzB96s9ZOrnNP0sB/bh+xLZBBxS1TqnSNNYP6yHs/8wkE7g1u+3wHeBBud1OqFTN/Al/VdFZJmI3ORsC5nP5pn4bYVEE5hUVUUkqMd8i0gC8CzwTVU94vuh6hPs9VPfSqCDRSQFeA7o43JInUJEJgL7VHWZiIx1Ox4/OU9Vd4lIFjBHRNY33Rnsn80zsSuSlu0CejZ5netsC1Z7nctmnP/uc7afrp4BW38RicSXRJ5Q1X87m0Omfo1U9RDwOjAKX7NH44++prF+WA9nfzJQRWDWbwxwtYhsxddUfDHwAKFRNwBUdZfz3334fgSMIAQ/my2xRNKyJUCJM6IkCl9n30yXY+qImUDj6I8pwPNNtn/OGUEyEjjsXIa/AlwmIqnOKJPLnG2uctrIHwXWqeqvm+wKlfplOlciiEgscCm+fqDXgWudYs3r11jva4F56uuxnQlMdkY+FQIlwOKuqUXLVPUOVc1V1QJ8/57mqeoNhEDdAEQkXkQSG5/j+0ytJkQ+m2fldm9/oD7wjarYgK+N+k6342lD3P8CdgOn8LWv3oivbXkusBF4DUhzygrwoFPHVcDwJuf5Ir6OzArgC27Xy4npPHzt0CuB5c7jihCq30Dgfad+q4G7nO1F+L4sK4CngWhne4zzusLZX9TkXHc69S4HLne7bs3qOZaPRm2FRN2ceqxwHmsavzNC5bN5todNkWKMMaZDrGnLGGNMh1giMcYY0yGWSIwxxnSIJRJjjDEdYonEGGNMh1giMeYsRORd578FInJ9J5/7+y29lzHBxIb/GtNKztQe31bViW04JkI/mkuqpf1HVTWhM+Izxi12RWLMWYhI44y89wLnO+tN3OZMsPhLEVnirClxs1N+rIi8JSIzgbXOtv84k/mtaZzQT0TuBWKd8z3R9L2cO55/KSKrnTUuPtXk3PNF5BkRWS8iTzh3/CMi94pvrZaVInJ/V/6NTHizSRuNab2pNLkicRLCYVU9R0SigXdE5FWn7FCgv/qmOgf4oqoecKY+WSIiz6rqVBG5VVUHt/Be1wCDgUFAhnPMm86+IUA/4APgHWCMiKwDPg70UVVtnGrFmK5gVyTGtN9l+OZLWo5vOvt0fHM/ASxukkQAvi4iK4CF+CblK+HMzgP+par1qroXeAM4p8m5d6pqA75pYgrwTbN+EnhURK4Bjne4dsa0kiUSY9pPgK+p6mDnUaiqjVckxz4s5OtbuQQYpaqD8M2nFdOB961p8rweaOyHGYFvEaiJwOwOnN+YNrFEYkzrVeNb4rfRK8BXnKntEZHezsyvzSUDB1X1uIj0wbe0aqNTjcc38xbwKacfJhPfEsqnneXWWaMlWVVnAbfhaxIzpktYH4kxrbcSqHeaqP6Gbz2NAuA9p8O7ko+WUm1qNvBlpx+jHF/zVqPpwEoReU9906o3eg7fWiQr8M14/F1V3eMkopYkAs+LSAy+K6Xb21dFY9rOhv8aY4zpEGvaMsYY0yGWSIwxxnSIJRJjjDEdYonEGGNMh1giMcYY0yGWSIwxxnSIJRJjjDEd8v//2DVehR1mrQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.plot('lr')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Triangular Learning Rate Policy via ```autofit```\n",
    "\n",
    "The ```autofit``` method in *ktrain* employs a default cyclical learning rate schedule that tends to work well in practice.  The default learning rate schedule in ```autofit``` is currently the [triangular learning rate policy](https://arxiv.org/abs/1506.01186), which some slight modifications.\n",
    "\n",
    "The ```autofit``` method accepts two primary arguments. The first (required) is the learning rate (**lr**) to be used, which can be found using the learning rate finder above. The second is optional and indicates the number of epochs (**epochs**) to train.  If **epochs** is not supplied as a second argument, then ```autofit``` will train until the validation loss no longer improves after a certain period. This period can be configured using the **early_stopping** argument.  At the end of training, the weights producing the lowest validation loss are automatically loaded into the model, when **early_stopping** is enabled.  The ```autofit``` method can also automatically reduce the maximum (and base) learning rates in the triangular policy when validation loss no longer improves. This can be configured using the **reduce_on_plateau** and **reduce_factor** arguments to ```autofit```.  \n",
    "\n",
    "Example:\n",
    "```\n",
    "learner.autofit(0.001, 20, reduce_on_plateau=2, reduce_factor=10)\n",
    "```\n",
    "\n",
    "The above will reduce the maximum and base learning rates in the triangular policy by a factor of 10 after two consecutive epochs of no improvement in validation loss.  Validation loss (i.e., val_loss) is the default criterion for both **early_stopping** and **reduce_on_plateau**.  To use validation accuracy instead, use invoke ```autofit``` with ```monitor='val_acc'```.\n",
    "\n",
    "Here, we will use the ```autofit``` method and run the main training phase for two epochs.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "begin training using triangular learning rate policy with max lr of 0.005...\n",
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/2\n",
      "25000/25000 [==============================] - 7s 264us/step - loss: 0.4752 - acc: 0.7769 - val_loss: 0.3279 - val_acc: 0.8764\n",
      "Epoch 2/2\n",
      "25000/25000 [==============================] - 6s 255us/step - loss: 0.2541 - acc: 0.9061 - val_loss: 0.2851 - val_acc: 0.8880\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fe7e462ada0>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# rebuild the model to train from scratch \n",
    "learner.set_model(get_model())\n",
    "\n",
    "# training using autofit\n",
    "learner.autofit(0.005, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ```autofit``` method runs a triangular learning rate schedule with two modifications.  First, it annihilates the learning rate at the end of each cycle:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAEWCAYAAACjYXoKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl0VPeZ4P3vo9K+AkJoqwKBAWMWs6iEwXa8x8Yr2IDAHce4OzOennH67Z5Mv/3a02dyJjnj93Sme5Ked04yPTmTDHLsBATesI13HCeO2MS+2zJIaAEkBAgJof33/nFv4bIsoa2qbi3P5xwdSnerRxeVnrrP/dXzE2MMSiml1FjEOR2AUkqpyKfJRCml1JhpMlFKKTVmmkyUUkqNmSYTpZRSY6bJRCml1JhpMlEqhESkSESMiMQH6HjVInJfoLdVaqQ0maiYMtgfVBG5S0T6RKRNRFpF5ISI/PkQx/qeiBy3tz8nIltFJCN40SsVvgLy7kipKNFgjHGLiAAPAltEpMIYc6L/hiJyJ/D/AsuMMftEZALwaIjjVSps6JWJUv0Yy1bgAnDzIJuVANuNMfvsfS4YY8qMMa0AIpIiIv9NRGpEpEVEPhORFL/9vyMip0XkvIj8vW+hiMSJyPMi8qWINItIuZ2ofOu/ax+z2X8/e916Efkvft/fJSJ1AwU/1PMoNVKaTJTqx/5D+xgwEagaZLOdwAMi8iMRuU1Ekvqt/yegGLgVmAD8HdDnt/524EbgXuCHInKTvfyvgBXAnUABcBH4uR3XbOB/At+112UD7lH+mIM+j1KjoclEqa8UiMgl4CrwOvAD35VHf8aYPwJPAIuAd4BmEfmpiLhEJA74C+CvjTH1xpheY0yFMabT7xA/MsZcNcYcAA4A8+3lfwn8vTGmzt7+PwOr7Bv2q4C3jTF/sNf9J76eoEbies+j1IjpL45SX/HdM0kC/gG4B/jnwTY2xrwLvGsnj7uBTcAJrESUDHx5nec66/e4HUi3H08BXhcR/yTRC+RiXUHU+j3/FRFpHubP1t/1nqd+lMdUMUyvTJTqx36n/v8A80RkxTC27zPGfAxsA+YC54EO4IZRPH0t8KAxZpzfV7Ixph44A3h8G4pIKlapy+cKkOr3fd4on0epEdNkomJRgogk+3194wrdGNMF/DfghwMdQESWi8haERkvlsVY9x92GGP6gF8DPxWRArv0tXSA+yoD+RfgRRGZYj9Pjogst9dtBh4RkdtFJBH4MV9/De8HHhKRCSKSB/zNKJ9HqRHTZKJi0Vas+yK+r/88yHa/BiaLyEBDfi8C/xr4ArgMvAz8ozHmFXv93wKHgN1Yo8J+wvBeb/8d2AJ8ICKtwA7gFgBjzBHgOeC3WFcpFwH/0Vq/wbr/Ug18AGwczfMoNRqik2MppZQaK70yUUopNWaaTJRSSo2ZJhOllFJjpslEKaXUmMXEhxYnTpxoioqKnA5DKaUixp49e84bY3KGu31MJJOioiIqKyudDkMppSKGiNSMZHstcymllBozTSZKKaXGTJOJUkqpMdNkopRSasw0mSillBqzoCYTEVkmIidEpEpEnh9gfZKIbLTX7xSRIr91L9jLT4jIA37Lq0XkkIjsFxEdoqWUUmEgaEODRcSFNQ3ot7E6m+4WkS3GmKN+m30PuGiMmS4ia7E6q66xpyddC8zBmhDoIxGZaYzptfe72xhzPlixK6WUGplgXpksBqqMMSftuSE2AP3nS1gOlNmPNwP3iojYyzcYYzqNMaew5uFeHMRYVQD8qeo8h+panA5DRama5iu8e+iM02GoQQQzmRTiN8Uo1tVJ4WDbGGN6gBasmeOut6/BmoNhj4g8O9iTi8izIlIpIpVNTU1j+kHU0No6e/g3v9nDD8r3o9MaqGD44ZtH+He/3UvdxXanQ1EDiMQb8LcbYxYBDwLPicgdA21kjPmlMcZrjPHm5Ay7I4AapVf31NHW2cMXjW3sq73kdDgqypxsauPTz5swBjbvqRt6BxVywUwm9fjNVw247WUDbmNPnZoFNF9vX98c1caYRuB1tPzluL4+Q9n2amblZZCS4GJTZe2Q+yg1Ei9tryHBJcwrzGJTZR19fXr1G26CmUx2AzNEZKo9X/VarGlC/W0B1tmPVwHbjFUj2QKstUd7TQVmALtEJE1EMgBEJA24HzgcxJ9BDcMfq85zsukKz94xjYdvzuetA2do7+pxOiwVJdo6e9i8p46H5+Xzr++YRv2lq2w/2ex0WKqfoCUT+x7I94H3gWNAuTHmiIj8WEQeszf7FZAtIlXAD4Dn7X2PAOXAUeA94Dl7JFcu8JmIHAB2Ae8YY94L1s+ghqesopqJ6Yk8fHM+pV4PbZ09bD101umwVJTwlVCfuW0q98/OJSslgY279eo33AS1a7AxZiuwtd+yH/o97gBWD7Lvi8CL/ZadBOYHPlI1WjXNV/jkRCN/dfd0kuJdlBSNZ+rENMora1lV7HY6PBXhfCXU+Z5xLPCMA2DFggJ+t7uWlvZuslITnA1QXROJN+BVGHlpew0uEb6zZAoAIsJqr5tdpy5w6vwVh6NTke4zu4T6zK1Tri1b7fXQ1dPHlgP9b8EqJ2kyUaN2pbOH8spals3NIzcz+drylYvcxAl6I16Nma+E+tC8/GvL5hZmMTs/k436+xVWNJmoUXt9Xz2tHT08c2vR15bnZiZz942TeHVvHT29fc4EpyJeTfMVtp1o5M8WTyYp3vW1dWtKPByuv8yRBv2QbLjQZKJGxRhDWUU1cwoyKZ4y/hvrV3s9nLvcyR+/0K43anT6l1D9LV9QQGJ8HJsq9TMn4UKTiRqVii+b+aKxjWduLcLqgPN198yaRHZaoo66UaPiK6E+OC//ayVUn3GpiTwwJ4839tfT2dM7wBFUqGkyUaOyvqKaCWmJPDq/YMD1ifFxPLGokI+OnaO5rTPE0alI91UJ9ZtXJT6lXjeX2rv58Oi5EEamBqPJRI1Y7YV2Pj52jrUlHpITXINuV+r10NNneH2fjrpRw2eM4aXt1cwtzGTR5G+WUH1uu2EiheNSKNdSV1jQZKJG7OUdNYgITw1Qy/Y3IzeDhZPHUV5Zq80f1bBt/7KZz8+1sW7pwCVUn7g4YVWxmz9+0UT9pashjFANRJOJGpGrXb1s2F3L/bNzKRiXMuT2pV4Pn59r44C2plfDNFQJ1d+qYjfGWJ+SV87SZKJG5I399bRc7f7GcODBPHJzPikJLsr1MwFqGGovtPPRMEqoPp4Jqdw2PZvyylpt/ugwTSZq2HzDgWflZbB46oRh7ZORnMBD8/J5a38DV7t01I26vuGWUP2Vej3UXbzKDm3+6ChNJmrYdp66wPGzrYMOBx5MqddNa2cP7x7WWfLU4Hwl1AfmDK+E6vPAnDwyk+P16tdhmkzUsJVVVJOVksDyBf0nzLy+xVMnUJSdqp85Udf1pl1CXbe0aET7JSe4WL6gkHcPn6XlandwglND0mSihqXh0lU+OGrVslMSh65l+7OaP3rYeeoC1dr8UQ3AGMP6EZZQ/a0p8dDZ08eWAw1BiE4NhyYTNSwv76jBGDOiWrY/X/NHnXJVDWS0JVSfOQWZ3JSfqc1FHaTJRA2po7uX3+06zX035eKZkDqqY+RlJXPnzBw276mjV0fdqH7KKqoZlzryEqqPiFDqdXOwroVjZy4HODo1HJpM1JC2HGjgYvvwhwMPZk2Jh7OXO/jDF02BCUxFBV8Jdc0oSqj+ViwoJNEVpzfiHaLJRF2XbzjwzNx0lt6QPaZj3TMrlwlpiVqKUF/jK6F+d5QlVJ/xaYl8e04ub+zT5o9O0GSirmtPzUWONFzm6SFaWwxHYnwcjy8s5MOj2vxRWTq6reHA992Ui3v86Eqo/kq9Hi62d/PR0cYARKdGQpOJuq71FdVkJMfz+MLR1bL7K/V66O41vLFfR90oeOtAAxeudI25hOpz+/SJFGQla6nLAZpM1KDOtnTw7uGzlHo9pCXFB+SYN+ZlMN8zjk3a/DHm+YYDB6KE6uOymz/+4YsmGrT5Y0hpMlGDemVnDX3G8PTSsdWy+yv1ujl+tpWD2vwxpvlKqOtGORx4MKu9Hm3+6ABNJmpAnT3WcOB7bpzElOy0gB770fkFJCfoqJtYt76imswAllB9PBNSufWGbDbtqdPmjyGkyUQN6J2DZzjf1sW6ANWy/WUmJ/DQ3Hy2aPPHmHXucgfv2SXU1MTAlFD9lXo9nL7Qzs5TFwJ+bDUwTSZqQGUV1UzLSeP26RODcvzSEg+tnT28d0SbP8aiV3bU0GsMT4+wD9dwLZubR4Y2fwwpTSbqG/advsiBuhbWLS0iLi5wtWx/t0ydwJTsVMp3a1071nT29PJbu4Q6OXvsw4EHYjV/LGDroTNc7tDmj6GgyUR9w/qKatKT4llZ7A7ac4gIq4vdbD/ZTE2zNn+MJcEsofor9drNH3UYekhoMlFf09jawdZDZ1hV7CY9QMOBB7OyWJs/xqKyimpuyEnjWzOCU0L1mVeYxay8DO24ECKaTNTX/Hbnabp7Az8ceCD5WSncoc0fY8q1EmqAhwMPxGr+6OFAXQvHz2rzx2DTZKKu6erp45Wdp7lzZg7TctJD8pylXg9nWjr4ozZ/jAlldgn1iUXBK6H6W7GwkASX6L25ENBkoq559/AZmlo7A9baYjjuu8nX/FFf7NGusbWDd0JUQvWZkJbI/bPzeH1fHV09fSF5zlgV1GQiIstE5ISIVInI8wOsTxKRjfb6nSJS5LfuBXv5CRF5oN9+LhHZJyJvBzP+WFNWUU1Rdip3zswJ2XMmxsexYkEhHxw9y4UrXSF7XhV6oSyh+lvtdXOxvZuPj50L6fPGmqAlExFxAT8HHgRmA0+KyOx+m30PuGiMmQ78DPiJve9sYC0wB1gG/MI+ns9fA8eCFXssOlh3ib2nL/F0EIcDD6a0xG01f9xXH9LnVaHjK6HedWPoSqg+35qRQ35WMhv1RnxQBfPKZDFQZYw5aYzpAjYAy/ttsxwosx9vBu4V667ccmCDMabTGHMKqLKPh4i4gYeB/x3E2GPO+opqUhNdrPKGppbtb1ZeJvPdWZRr88eo5SuhBns48ECuNX/8vIkzLdr8MViCmUwKAf+3AnX2sgG3Mcb0AC1A9hD7/jPwd8B1C6Ai8qyIVIpIZVOT3ty9nvNtnbx94AwrF7nJTE5wJIbVXg/Hz7ZyuF5H3USjsopqpk5M484ZoSuh+ltd7KHPwGt79eo3WCLqBryIPAI0GmP2DLWtMeaXxhivMcabk+PML3Ck2LDrNF29fay7NbS1bH+Pzi8gKT6OjZWnHYtBBcehuhb2nr7Ed5dMCXkJ1WdydipLp2VTXlmrzR+DJJjJpB7w+H3vtpcNuI2IxANZQPN19r0NeExEqrHKZveIyMvBCD5WdPf28fKO09w+fSLTJ2U4FkdWSgIPzcvnzf0NdHRr88do4mQJ1V9piZua5nZ2VWvzx2AIZjLZDcwQkakikoh1Q31Lv222AOvsx6uAbcYqmm8B1tqjvaYCM4BdxpgXjDFuY0yRfbxtxpingvgzRL0Pjpzj7OWOkA4HHsxqr5vWjh7eP3LW6VBUgJxv6+StAw2OllB9ls3JJyNJmz8GS9CSiX0P5PvA+1gjr8qNMUdE5Mci8pi92a+AbBGpAn4APG/vewQoB44C7wHPGWP07WoQrK84hWdCCnfPmuR0KCyZmo1nQgobd+uLPVqEQwnVJyXRxWPa/DFognrPxBiz1Rgz0xhzgzHmRXvZD40xW+zHHcaY1caY6caYxcaYk377vmjvd6Mx5t0Bjv17Y8wjwYw/2h1paGF39UWeXlKEy6Fatr+4OKG02EPFl83UXmh3Ohw1Rr4S6rdmOFtC9Vfq9dDR3cfbB3Tqg0CLqBvwKrDKKqpJSXBR6vUMvXGIrCx2I4I254sCvhLquiDNWTIaN7uzuDE3Qz9zEgSaTGLUxStdvLm/gRULC8lKdbaW7a9gXArfmqHNH6NBWUV12JRQfUSE0hIPB2ovceJsq9PhRBVNJjFqw+5aOnvCo5bd3xqvh4aWDv5Udd7pUNQoHWloYVf1hbApofpbsaCABJfo1W+AaTKJQT29fby8o4al07KZlZfpdDjfcN/sSYxPTdBSRAQLxxKqT3Z6EvfdlMtr++q1+WMAaTKJQR8dO0f9pauOtLYYjqR4FysWFvLhkXNc1OaPEcdXQn18UXiVUP2Vlni4cKWLbce1+WOgaDKJQesrqikcl8J9N4VPLbu/1cUeunr7eHO/tr+INNdKqGF0472/O2bkkJeZTLlOfRAwmkxizPGzl9lx8gJPLZlCvCt8//tnF2QyrzCLjZV12vwxgviXUG/MC4/hwANxxQkriwv5/YlGzrZ0OB1OVAjfvyYqKMoqakiKj2NtSfjVsvsrLfFw7MxljjRo88dI8dGxxrAuofrzNX98da9enQSCJpMY0tLezev76li+oIDxaYlOhzOkx+zmj9r+InKURUAJ1adoYhq3TJ3AJp36ICA0mcSQjZWn6ejui4h3jWA1f1w2N4839tVr88cIcPzsZbafbA77Eqq/NSUeqpvb2XVKmz+OVWT8j6sx6+0zvLS9hsVFE5hTkOV0OMO2xuvhsjZ/jAiRVEL1eXBuPulJ8XojPgA0mcSIbccbqbsYGbVsf0umZeMen8ImfbGHtZb2bt7YV8+KBYURUUL1SUl08eh8q/ljqzZ/HBNNJjGirKKavMxk7p+T63QoIxIXJ6wu9vBZ1Xlt/hjGyitrudrdG3FvVsAqdV3t7uXtg9r8cSw0mcSAqsZWPqs6z1NLJpMQIbVsf6u8VvPHzXv06iQc9fYZXtpRzeKiCcwuCL+OCkOZ785iZm66DvQYo8j7y6JGrKyihsT4OJ5cPNnpUEalcFwKt0+fqM0fw9S2443UXoi8EqqPiFDq9bDv9CW+OKfNH0dLk0mUu9zRzat763j05gKy05OcDmfUSr0e6i9dpeJLbf4YbsoqqsnPirwSqr/HFxYSHyd6dTIGmkyi3KbKOtq7esNiWt6xuH9OLuNSE3TUTZj5qoQ6JSJLqD7Xmj/urae7V5s/jkbk/u+rIfX1GX6zvZpFk8cxzx05w4EHkhTvYsWCQt4/cpZL7dr8MVz4SqiRNBx4MKUlbpqvdPHxsUanQ4lImkyi2KefN1Hd3B6xtez+VnvddPX08eb+BqdDUURPCdXnjhk55GYm6Twno6TJJIqtr6gmJyOJB+fmOx1KQMwpyGJuYabWtcPE5igpofrEu+JYucjNJycaOXdZmz+OlCaTKHWyqY1PP2/iqVumkBgfPf/NpV4PRxouc7i+xelQYlpfn+Gl7dFRQvW32qvNH0crev7KqK95aXsNCS7hyVsiv5btb/n8QhLj47QU4TBfCfWZ26Y6HUpATZ2YxuKpE9ikUx+MmCaTKNTW2cPmPXU8PC+fSRnJTocTUFmpCSybk8cb+xu0+aOD1ldUMykjiQfn5jkdSsCVej2cOn+FypqLTocSUTSZRKFX99TR1tkTNTfe+yv1emi52s0HR3XKVSf4SqjfuSWyhwMP5qF5eaQnxbNxt179jkT0/SbEuL4+Q9n2aua7s1g4ebzT4QTFrTdkUzguRUtdDonWEqpPamI8j87P552DZ2jr7HE6nIihySTKfFZ1npNNV6L2qgTs5o9eN59VnafuojZ/DKVoLqH6W+21mj++c1CHoQ+XJpMoU1ZRzcT0RB6+OTqGAw9mVbEb0OaPoRbtJVSfhZ5xzJiUrqWuEdBkEkVqmq+w7UQjf7Z4MknxLqfDCSr3+FRunz6RTZV19Gnzx5C4VkL1jIvaEqqPr/nj3tOXqGrU5o/Dockkiry0vQaXCN9ZMsXpUEJitd38cfvJZqdDiQm+Euozt8bG79fji3zNH/Xqdzg0mUSJK509lFfWsmxuHrmZ0VvL9nf/7FyyUhK0FBEivhLqQ/Oiu4TqMzE9iXtvmsRre+u0+eMwBDWZiMgyETkhIlUi8vwA65NEZKO9fqeIFPmte8FefkJEHrCXJYvILhE5ICJHRORHwYw/kry+r57Wjp6oaW0xHMkJLlYsKOC9I2dpadcpV4Mplkqo/kq9Hs63dfHJcW3+OJSgJRMRcQE/Bx4EZgNPisjsfpt9D7hojJkO/Az4ib3vbGAtMAdYBvzCPl4ncI8xZj6wAFgmIkuC9TNECmOs1hZzCzMpnhLdtez+Vns9VvPHA/VOhxLVYq2E6nPnzBwmZSRpP7hhCOaVyWKgyhhz0hjTBWwAlvfbZjlQZj/eDNwrImIv32CM6TTGnAKqgMXG0mZvn2B/xfzd1+1fNvP5uTbWLS3COn2xY25hFrPztfljMMViCdUn3hXHymI3n5xoolGbP15XMJNJIeD/Cq+zlw24jTGmB2gBsq+3r4i4RGQ/0Ah8aIzZGZToI8j/qahmQloij84vcDoUR6wp8XC4/jJHGrT5YzD4Sqh/fluR06E4YnWxm94+w2v79Or3eiLuBrwxptcYswBwA4tFZO5A24nIsyJSKSKVTU1NoQ0yhGovtPPxsXOsLfGQnBA7tWx/yxcUkOiKY5OOugk4/xLqoigfDjyYaTnplBSNp3x3rTZ/vI5gJpN6wL/fgtteNuA2IhIPZAHNw9nXGHMJ+ATrnso3GGN+aYzxGmO8OTk5Y/gxwtvLO2oQEZ6KsVq2v3Gpidw/J5fX99Vr88cAi+USqr9Sr4eT56+wR5s/DiqYyWQ3MENEpopIItYN9S39ttkCrLMfrwK2GSv1bwHW2qO9pgIzgF0ikiMi4wBEJAX4NnA8iD9DWLva1cuG3bXcPzuXgnEpTofjqDUlVvPHj45p88dAWh/jJVSfh+blk5bo0ntz1xG0ZGLfA/k+8D5wDCg3xhwRkR+LyGP2Zr8CskWkCvgB8Ly97xGgHDgKvAc8Z4zpBfKBT0TkIFay+tAY83awfoZw9+b+elqudsfUcODB3HrDRArHpehnTgKo9kI7H8V4CdUnLSmeR24u4G1t/jio+GAe3BizFdjab9kP/R53AKsH2fdF4MV+yw4CCwMfaeQxxrC+oppZeRksnjrB6XAc54oTVha7+R/bvqD+0lUKY/xKLRC0hPp1pSUeNlbWsvXgGUpLorNj8lhE3A14Zdl56gLHz7byzK2xXcv2t7rYjTFWM0I1Nr4S6gNztITqs2jyOG7ISdNS1yA0mUSosopqslISWL6g/2jr2OWZkMpt07Mpr6zV5o9j5Cuhrlta5HQoYUNEWFPiobLmIlWNbUPvEGM0mUSghktX+eCoVctOSYztWnZ/pV4PdRevskObP46allAH9/hCN644YdMevTrpT5NJBHp5Rw3GGK1lD+CBOXlkJsdrKWIMdmkJdVA5GUncM2sSr+6p1+aP/WgyiTAd3b38btdp7rspF8+EVKfDCTvJCS6WLyjk3cNnabmqzR9HY72WUK9rjdfD+bZOfn8iej8MPRqaTCLMlgMNXGzX4cDXs6bEQ2dPH1sO6JSrI3WthLpYS6iDuevGHHK0+eM3aDKJIMYYyiqqmZmbztIbsp0OJ2zNKcjkpvxMyvUzJyPmK6F+V0uog4p3xfHEokK2HW+ksVWbP/poMokge2oucqThMk/HeGuLoVhTrro5VN/C0YbLTocTMTq6reHA992Ui3u8llCvp9TrobfP8Ppebf7oo8kkgqyvqCYjOZ7HF2oteygrFhRazR911M2wvXWggQtXurSEOgw35KTjnTKe8kpt/uijySRCnLvcwXuHz7LG6yEtKaiNC6LC+LREvm03f+zs0eaPQ/ENB9YS6vCVej182XSFvae1+SNoMokYr+yoodcYntYPkQ1bqdfDpfZuPjqqU64ORUuoI/fwzfmkJroo360dF0CTSUTo7Onlt7tOc8+Nk5icrbXs4bp9+kQKspJ11M0w+EqoTyzSEupwWc0f83n7YANXtPmjJpNI8M7BM5xv62Kd1rJHxBUnrCp284cvmmi4dNXpcMKWfwk1NVFLqCNR6vVwpauXdw6dcToUxw2ZTOxpcmN2zpBwUFZRzbScNG6fPtHpUCLOqmKPNn8cgpZQR694ynim5aSxSa9+h04m9jwiJ0RkcgjiUf3sO32RA3UtrFtaRFyc1rJHanJ2KrfekM2mPXXa/HEAWkIdG2sYuofd1Rc52RTbzR+HW+YaDxwRkY9FZIvvK5iBKUtZRTXpSfGsLHY7HUrEKvV6OH2hnR2ntPljf1pCHbsnFhXiihPKK2P76ne4BdL/FNQo1IAaWzt459AZvnPLFNJ1OPCoLZubR8ab8WyqrOPWG7RU6E9LqGM3KSOZu2+cxKt76/jb+2cS74rNW9HD+qmNMZ8O9BXs4GLdb3eeprvX8PRSbW0xFlbzxwK2HjrD5Q5t/ujjK6E+c6uWUMeq1OumqbWTTz+P3eaP100mItIqIpcH+GoVEe1TEURdPX28svM0d87MYVpOutPhRLxSr938cb82f/TxlVCfWKQl1LG6e9YkJqYnsTGG+8FdN5kYYzKMMZkDfGUYYzJDFWQsevfwGZpaO7W1RYDMK8xiVl6Gjrqx+Uqoq4rdWkINgARXHCvt5o9NrZ1Oh+OI2CzuRYCyimqKslO5c2aO06FEBd+omwN1LRw/qxfVv9tZqyXUAFvt9dDTZ3h9X2zeiNdkEoYO1bWw9/QlntbhwAG1YmEhCS6J+fYXXT19vLyzRkuoATZ9UjrFU8ZTXlkXk80fNZmEofUV1aQmuljl1Vp2IE1IS+T+2Xm8vq+Orp7YnXL1Wgn1tiKnQ4k6pV43VY1t7Ku95HQoIafJJMycb+vkrQMNrFzkJjM5welwos5qr5uL7d18dOyc06E4pqyimqkT07hzhpZQA+3hmwvs5o+xd29Ok0mY2bDrNF29fay7VWvZwfCtGTnkx3DzR18J9btLpmgJNQjSk+J5eF4+bx1ooL0rtpo/ajIJI929fby84zS3T5/I9EkZTocTla41f/y8iTMtsdf8UUuowVdaYjV/3HrorNOhhJQmkzDywZFznL3cocOBg2xVsZu+GGz+qCXU0PBOGc+0iWkxV+rSZBJGyiqq8UxI4e5Zk5wOJapNyU5jybQJlFfGVvNHLaGGhoiw2uthV/WFmGr+qMkkTByZ8HiXAAAbZ0lEQVRpaGFX9QWeXlKES2vZQbemxGr+uKv6gtOhhISvhPqtGVpCDYWVdvPHzTF09avJJEyUVVSTkuCi1OtxOpSYsGxOPhlJ8TFTivCVUNfpnCUhMSkzmbtm5rB5Tx09vbExDF2TSRi4eKWLN/c3sGJhIVmpWssOhZREF48uKGDr4dho/qgl1NArLfHQ2NrJH76IjeaPQU0mIrJMRE6ISJWIPD/A+iQR2Wiv3ykiRX7rXrCXnxCRB+xlHhH5RESOisgREfnrYMYfKht219LZo7XsUFvj9dDR3cfbB6J7ylUtoTrjnlmTmJieGDMdF4KWTETEBfwceBCYDTwpIrP7bfY94KIxZjrwM+An9r6zgbXAHGAZ8Av7eD3AfzDGzAaWAM8NcMyI0tPbx8s7alg6LZtZedo7M5RudmdxY24GG6P8MydaQnVGgiuOJxa5+ejYOc63RX/zx2BemSwGqowxJ40xXcAGYHm/bZYDZfbjzcC9IiL28g3GmE5jzCmgClhsjDljjNkLYIxpBY4BhUH8GYLuo2ON1F+6qjPdOcAadePmQO0lTpxtdTqcoNASqrNWF7vp6TO8sa/e6VCCLpjJpBDwf8tXxzf/8F/bxhjTA7QA2cPZ1y6JLQR2DvTkIvKsiFSKSGVTU/jWLNdXnKJwXAr33aS1bCc8bjd/jNbW9L4Sqn52yRkzcjNYOHkcG3fXRn3zx4i8AS8i6cCrwN8YYwbsJ26M+aUxxmuM8ebkhGcPouNnL7Pj5AWeWjIlZqf6dFp2ehL33ZTLa/vqo675o38J9cY8HQ7slDVeD180trE/yps/BvMvWD3gX6R128sG3EZE4oEsoPl6+4pIAlYiecUY81pQIg+RsooakuLjWFuitWwnlXo9XLjSxbbj0dX8UUuo4eHhm/NJSXBRXhndN+KDmUx2AzNEZKqIJGLdUN/Sb5stwDr78Spgm7GuBbcAa+3RXlOBGcAu+37Kr4BjxpifBjH2oGtp7+aNffWsWFDI+LREp8OJaXfMzCEvMznqplwtq6jWEmoYyEhO4KEYaP4YtGRi3wP5PvA+1o3ycmPMERH5sYg8Zm/2KyBbRKqAHwDP2/seAcqBo8B7wHPGmF7gNuC7wD0ist/+eihYP0MwlVfWcrW7V981hgFXnLCyuJBPP2/ibEuH0+EExPGzl9l+sllLqGFiTYmHts4e3o3i5o9BnfzZGLMV2Npv2Q/9HncAqwfZ90XgxX7LPgMifqB8b5/hpR3VLC6awOwCHQ4cDlYXe/j5J1/y6t46nrt7utPhjJmWUMNLSdF4irJTKa+sZWVxdHZs1rcsDth2vJHaC1rLDidFE9O4ZeoENlVG/qgbLaGGH1/zx52nLlB9/orT4QSFJhMHlFVUk5eZzP1zcp0ORfkp9Xqobm5n16nIbv6oJdTwtKrYTZzApj3RdW/OR5NJiFU1tvJZ1XmeWjKZBK1lh5WH5uWTnhQf0aNutIQavnIzk7nrxkls3lNHbxROfaB/zUKsrKKGxPg4nlw82elQVD8piS4enV/A1kNnaI3Q5o+faAk1rJV63Zy73MkfPg/fD1KPliaTELrc0c2re+t49OYCstOTnA5HDaDU6+Zqdy9vH4zM5o/rtYQa1u6ZlUt2WiLlUdhxQZNJCG2urKO9q1dbW4SxBZ5xzMxNj8gXu6+E+t2lU7SEGqYS4+N4fGEhHx07R3OUNX/U37gQ6eszvLS9mkWTxzHPneV0OGoQIkKp18O+05f44lxkNX/0lVB1OHB4Ky3x0N1reD3Kmj9qMgmRTz9vorq5XWvZEWDFwkLi4ySirk60hBo5ZuZmsMAzjvIoGIbuT5NJiKyvqCYnI4kH5+Y7HYoawkRf88e99XRHyJSrWkKNLKVeD5+fa+NAXYvToQSMJpMQONnUxqefN/HULVNIjNdTHglKS9w0X+ni42ONTocyJC2hRp5H5+eTnBAXUVe/Q9G/bCHw0vYaElzCk7doLTtS3DEjh0kZSRExz4mWUCPPteaP+xu42tXrdDgBockkyNo6e9i8p46H5+UzKSPZ6XDUMMW74lhV7OaTE42cuxzezR/XV1QzSUuoEafU66G1s4d3D0fmMPT+NJkE2at76mjr7NF3jRFotddDn4FX94bvJ+J9JdTvaAk14twydcK15o/RQH/7gqivz1C2vZr57iwWTh7vdDhqhKZOTGNx0QQ2VdaF7agbLaFGLl/zxx0nL1DTHPnNHzWZBNFnVec52XSFZ24rcjoUNUqlJR5Onb/C7uqLTofyDVpCjXwrF9nNHyO4H5yPJpMgKquoZmJ6Ig/N01p2pHpoXh5pia6wLEVoCTXy5WUlc+fMnKho/qjJJEhqmq+w7UQjf7Z4MknxLqfDUaOUmhjPo/MLeOfgGdo6w2fKVS2hRo9Sr4ezlzv44xeR3fxRk0mQvLS9BpcI31kyxelQ1BiVlnis5o8HGpwO5RotoUaPe2/KZUIUNH/UZBIEVzp7KK+sZdncPHIztZYd6RZ6xjF9Ung1f9QSavTwNX/88Og5LlzpcjqcUdNkEgSv76untaNHW1tECav5o5u9py9R1eh888fTze1aQo0ypV6r+eMbEdz8UZNJgBljtbaYW5hJ8RStZUeLxxe67eaPzo+6eWl7tZZQo8yNeRnMd2dFdPNHTSYBtv3LZj4/18a6pUWIiNPhqADJyUjinlmTeG1vnaPNH6909rBRS6hRqbTEw/GzrRyqj8zmj5pMAmx9RTUT0hJ5dH6B06GoAFtT4uF8WxefHHeu+aOWUKPXo/MLSE6IY+Pu8Lk3NxKaTAKo9kI7Hx07x9oSD8kJWsuONnfOzCEnI8mxG/FaQo1umckJPDQ3ny0R2vxRk0kAvbyjBhHhKa1lR6V4VxwrF7n55EQTjQ40f9QSavRbbTd/fP/IWadDGTFNJgFytauXDbtreWBOLgXjUpwORwVJqddNb5/h1b2hH3WjJdTod8vUCUyekBqRpS5NJgHy5v56Wq52s25pkdOhqCCalpNOSdF4NoV41I2WUGNDXJw1DH37yWZON7c7Hc6IaDIJAGMM6yuqmZWXweKpE5wORwXZaq+Hk+evsKcmdM0ftYQaO1YWuxGBzXsi6+pEk0kA7Dp1geNnW3nmVq1lx4KH5+WTlugKWSlCS6ixJT8rhTtm5LApwpo/ajIJgPUV1WSlJLB8QaHToagQSEuK55GbC3jnUGiaP2oJNfasKfFwpqWDz6rOOx3KsGkyGaOGS1f54KhVy05J1Fp2rCgtcdPe1cvWg8GdclVLqLHp3psmMT41Iaz6wQ0lqMlERJaJyAkRqRKR5wdYnyQiG+31O0WkyG/dC/byEyLygN/yX4tIo4gcDmbsw/XyjhqMMVrLjjGLJo/nhpw0Ngb5xa4l1NiUFO9ixcJCPjxyjosR0vwxaMlERFzAz4EHgdnAkyIyu99m3wMuGmOmAz8DfmLvOxtYC8wBlgG/sI8HsN5e5riObquWfd9NuXgmpDodjgohq/mjhz01F6lqbAva82gJNXatKfHQ1dvHG/sjo/ljMK9MFgNVxpiTxpguYAOwvN82y4Ey+/Fm4F6x3n4tBzYYYzqNMaeAKvt4GGP+AFwIYtzD9taBBi5c6dLWFjHq8UWFuOKETUEadaMl1Ng2Ky+Tm91ZbNwdGc0fg5lMCgH/V1mdvWzAbYwxPUALkD3Mfa9LRJ4VkUoRqWxqCvwMZr5a9szcdJbekB3w46vwNykjmXtmTeLVPfVBaf6oJVS12ms1fzxcf9npUIYUtTfgjTG/NMZ4jTHenJycgB9/T81FjjRc5mltbRHTSr0ezrd18vsTgX3DoiVUBfDY/AKS4uMi4kZ8MJNJPeDx+95tLxtwGxGJB7KA5mHu66j1FdVkJMfz+EKtZceyu27MYWJ64Js/aglVAWSlJPDg3Dze3F9PR3d4N38MZjLZDcwQkakikoh1Q31Lv222AOvsx6uAbcYqDm4B1tqjvaYCM4BdQYx1RM5d7uC9w2dZ4/WQlhTvdDjKQQmuOFYWF7LteCONrYFp/qglVOWvtMTD5Y7wb/4YtGRi3wP5PvA+cAwoN8YcEZEfi8hj9ma/ArJFpAr4AfC8ve8RoBw4CrwHPGeM6QUQkd8B24EbRaRORL4XrJ9hMK/sqKHXGJ7WD5EpYHWxh94+w+sBav6oJVTlb8nUbDwTUsK+1BXUt9XGmK3A1n7Lfuj3uANYPci+LwIvDrD8yQCHOSKdPb38dtdp7rlxEpOztZatYPqkdLxTxlNeWcuzd0wbcwLQEqryFxcnrC728NMPP6f2QnvY3kOL2hvwwbL10BnOt3WxTmvZyk+p18OXTVfYe3pszR+1hKoGsspu/rhpT53ToQxKk8kIrf9TNdNy0rh9+kSnQ1Fh5KGb80lNdFG+e2wvdi2hqoEUjEvhWzNy2FxZG7bNHzWZjMC+0xc5UNfCM7cWERentWz1lfSkeB65OZ+3DzZwZZTNH7WEqq6n1OumoaWDP4Vp80dNJiNQVlFNelI8TyxyOx2KCkOlXg9Xunp559Domj9qCVVdz7dn5zIujJs/ajIZpsbWDt45dIZVxW7StZatBlA8ZTzTctLYNMoXu5ZQ1fUkxbtYsaCQD46c41J7+DV/1GQyTL/bWUt3r+HppdraQg3M1/xxd/VFvmwaWfNHLaGq4Sj12s0f94XVZ7gBTSbD0tXTx8s7a7hzZg7TctKdDkeFsScW2s0fK0d2I15LqGo4ZhdkMq8wi/IR/n6FgiaTYXj38BmaWju1tYUa0qTMZO6+MYdX99bRM8zmj1pCVSNR6nVz9MxlDte3OB3K12gyGYayimqKslO5c2bgG0aq6FPq9dDUOvzmj1pCVSPx2PxCEsOw+aMmkyEcqmth7+lLPL1Ua9lqeO6eNYmJ6YnDerF39fTxipZQ1QhkpVrNH9/YF17NHzWZDGF9RTWpiS5WebWWrYYnwRXHE4vcbDveSFNr53W3fffwGRq1hKpGqNRrNX/84Og5p0O5RpPJdZxv6+StAw2sXOQmMznB6XBUBCn1uunpM7y+7/o3SrWEqkZj6bRs3ONTKN8dPqUuTSbXsWHXabp6+1h3q9ay1chMn5TBosnjKK+sG3TKVS2hqtHyNX/805fnqb3Q7nQ4gCaTQXX39vHyjtPcPn0i0ydlOB2OikClXg9VjW3sq7004PoNu09rCVWNmu/3ZnOYNH/UZDKInl7Dd5dO4S/vvMHpUFSEemR+ASkJrkFLEY2tnUzJTtMSqhqVwnEp3D59Ipv31NEXBs0fNZkMIiXRxXN3T+f2GdraQo1OelI8D9+cz1sHGmjv+mbzR2MMWt1SY1Hq9VB/6SoVXzY7HYomE6WC6Vrzx4PfbP7Y22dwaTZRY/Dt2blkpSSwMQw+c6LJRKkgKikaz9SJaQO2V+k16LS8akySE1w8vrCQ94+cdbz5oyYTpYJIRFjtdbOr+gIn+zV/NMbg0lyixmi1101XTx9bDjQ4GocmE6WCbNUit9X8sd+oGy1zqUCYU5DFnIJMNjr8mRNNJkoF2aTMZO6amcOre77e/LG3z2iZSwXEmhIPRxqcbf6oyUSpEFjt9dDY2skfvviq+aMx4NJkogLgsfkFJMbHOfqZE00mSoXAvTdZzR/9SxG9xhCnr0AVAONSE3lgTh6vO9j8UX+VlQqBBFccjy8s5ONjjZxvs5o/9hlDnF6ZqABZ4/XQcrWbDx1q/qjJRKkQKfV66Okz16Zc7evTZKIC59Ybsikcl+LYPCeaTJQKkRm5GSycPI6Nu2sxxtBrdDSXCpy4OGFVsZvPqs5TdzH0zR81mSgVQqVeD180trG/9hJ9feiViQqo1Xbzx1f31If8uTWZKBVCj9ycbzV/rKyz75k4HZGKJu7xqdx2w0Q27akNefNHTSZKhVBGcgIPzbOaP7Z19miZSwVcaYmHuotX2X4ytM0fNZkoFWKlXjdtnT3UXbyqZS4VcPfPziUzOT7kN+KDmkxEZJmInBCRKhF5foD1SSKy0V6/U0SK/Na9YC8/ISIPDPeYSoW7xVMnUJSdCqAzLKqAS05wsWJhIe8ePktLe3fInjdoyUREXMDPgQeB2cCTIjK732bfAy4aY6YDPwN+Yu87G1gLzAGWAb8QEdcwj6lUWLOaP3oAtNGjCopSr8du/hi6G/HBvDJZDFQZY04aY7qADcDyftssB8rsx5uBe8VqVrQc2GCM6TTGnAKq7OMN55hKhb2Vi9zEiY7mUsExtzCL2fmZlA8w9UGwxAfx2IWAf9GuDrhlsG2MMT0i0gJk28t39Nu30H481DGVCnt5Wcn8aPlcpuekOx2KilLP3FrEgbpLdPX0kRgf/NvjwUwmjhKRZ4FnASZPnuxwNEp903eXTHE6BBXFSks8lJZ4QvZ8wUxX9YD/T+K2lw24jYjEA1lA83X2Hc4xATDG/NIY4zXGeHNycsbwYyillBpKMJPJbmCGiEwVkUSsG+pb+m2zBVhnP14FbDPGGHv5Wnu011RgBrBrmMdUSikVYkErc9n3QL4PvA+4gF8bY46IyI+BSmPMFuBXwG9EpAq4gJUcsLcrB44CPcBzxphegIGOGayfQSml1PCIdSEQ3bxer6msrHQ6DKWUihgisscY4x3u9voJeKWUUmOmyUQppdSYaTJRSik1ZppMlFJKjVlM3IAXkSagZpS7TwTOBzCcQNLYRkdjG51wjS1c44LIjm2KMWbYH9KLiWQyFiJSOZIRDaGksY2OxjY64RpbuMYFsRWblrmUUkqNmSYTpZRSY6bJZGi/dDqA69DYRkdjG51wjS1c44IYik3vmSillBozvTJRSik1ZppMlFJKjZkmk0GIyDIROSEiVSLyvAPP7xGRT0TkqIgcEZG/tpdPEJEPReQL+9/x9nIRkf/PjvegiCwKQYwuEdknIm/b308VkZ12DBvtaQKwpxLYaC/fKSJFQY5rnIhsFpHjInJMRJaGy3kTkX9v/38eFpHfiUiyU+dNRH4tIo0icthv2YjPk4iss7f/QkTWDfRcAYrtH+3/04Mi8rqIjPNb94Id2wkRecBvecBfxwPF5rfuP4iIEZGJ9veOnzd7+V/Z5+6IiPxXv+WBO2/GGP3q94XV3v5LYBqQCBwAZoc4hnxgkf04A/gcmA38V+B5e/nzwE/sxw8B7wICLAF2hiDGHwC/Bd62vy8H1tqP/wX4t/bjfwf8i/14LbAxyHGVAf/KfpwIjAuH84Y19fQpIMXvfD3j1HkD7gAWAYf9lo3oPAETgJP2v+Ptx+ODFNv9QLz9+Cd+sc22X6NJwFT7tesK1ut4oNjs5R6s6TFqgIlhdN7uBj4CkuzvJwXjvAXtBR3JX8BS4H2/718AXnA4pjeBbwMngHx7WT5wwn78v4An/ba/tl2Q4nEDHwP3AG/bL5bzfi/2a+fQfoEttR/H29tJkOLKwvqDLf2WO37esJJJrf0HJN4+bw84ed6Aon5/eEZ0noAngf/lt/xr2wUytn7rHgdesR9/7fXpO2/BfB0PFBuwGZgPVPNVMnH8vGG9WblvgO0Cet60zDUw34vep85e5gi7vLEQ2AnkGmPO2KvOArn241DH/M/A3wF99vfZwCVjTM8Az38tNnt9i719MEwFmoD/Y5fg/reIpBEG580YUw/8E3AaOIN1HvYQHufNZ6TnyanXyl9gveMPi9hEZDlQb4w50G+V47EBM4Fv2aXST0WkJBixaTIJcyKSDrwK/I0x5rL/OmO9bQj52G4ReQRoNMbsCfVzD0M81mX+/zTGLASuYJVrrnHwvI0HlmMlvAIgDVgW6jiGy6nzNBQR+XusGVhfcToWABFJBf4j8EOnYxlEPNbV8BLg/wbKRUQC/SSaTAZWj1X/9HHby0JKRBKwEskrxpjX7MXnRCTfXp8PNNrLQxnzbcBjIlINbMAqdf13YJyI+KaC9n/+a7HZ67OA5iDFVgfUGWN22t9vxkou4XDe7gNOGWOajDHdwGtY5zIczpvPSM9TSF8rIvIM8AjwHTvZhUNsN2C9QThgvybcwF4RyQuD2MB6TbxmLLuwqgkTAx2bJpOB7QZm2KNsErFufm4JZQD2O4dfAceMMT/1W7UF8I38WId1L8W3/Gl79MgSoMWvXBFQxpgXjDFuY0wR1rnZZoz5DvAJsGqQ2Hwxr7K3D8o7XmPMWaBWRG60F90LHCUMzhtWeWuJiKTa/7++2Bw/b35Gep7eB+4XkfH2ldf99rKAE5FlWKXVx4wx7f1iXivW6LepwAxgFyF6HRtjDhljJhljiuzXRB3W4JmzhMF5A97AugmPiMzEuql+nkCft0Dc8InGL6xRGJ9jjWr4ewee/3asEsNBYL/99RBWzfxj4AusERoT7O0F+Lkd7yHAG6I47+Kr0VzT7F/GKmATX40eSba/r7LXTwtyTAuASvvcvYE1WiYszhvwI+A4cBj4DdZIGkfOG/A7rHs33Vh/AL83mvOEdf+iyv768yDGVoVVy/e9Hv7Fb/u/t2M7ATzotzzgr+OBYuu3vpqvbsCHw3lLBF62f+f2AvcE47xpOxWllFJjpmUupZRSY6bJRCml1JhpMlFKKTVmmkyUUkqNmSYTpZRSY6bJRKkhiEiF/W+RiPxZgI/9Hwd6LqUijQ4NVmqYROQu4G+NMY+MYJ9481XfrYHWtxlj0gMRn1JO0isTpYYgIm32w3/Aapi3X6x5SVxizbGx256r4t/Y298lIn8UkS1Yn3BHRN4QkT32fBLP2sv+AUixj/eK/3PZn5j+R7HmPTkkImv8jv17+Wq+lld8fZZE5B/Emv/moIj8UyjPkVLxQ2+ilLI9j9+ViZ0UWowxJSKSBPxJRD6wt10EzDXGnLK//wtjzAURSQF2i8irxpjnReT7xpgFAzzXE1if5J+P1Udpt4j8wV63EJgDNAB/Am4TkWNYbdlnGWOM+E0cpVQo6JWJUqN3P1bfpf1Y0wNkY/U3Atjll0gA/i8ROQDswGqiN4Prux34nTGm1xhzDvgU8LUO32WMqTPG9GG1FSnCak/fAfxKRJ4A2gc4plJBo8lEqdET4K+MMQvsr6nGGN+VyZVrG1n3Wu7DmuhqPrAPq+/WaHX6Pe7FmlirB1iM1SX5EeC9MRxfqRHTZKLU8LViTaHs8z7wb+2pAhCRmWJNxNVfFnDRGNMuIrOw5pXw6fbt388fgTX2fZkcrOlYdw0WmFjz3mQZY7YC/x6rPKZUyOg9E6WG7yDQa5er1mPN4VKENXeFYM3wuGKA/d4D/tK+r3ECq9Tl80vgoIjsNVYbf5/XsaZPPYDVPfrvjDFn7WQ0kAzgTRFJxrpi+sHofkSlRkeHBiullBozLXMppZQaM00mSimlxkyTiVJKqTHTZKKUUmrMNJkopZQaM00mSimlxkyTiVJKqTH7/wHVhmM6ibC8zgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.plot('lr')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Second, if using the Adam, Nadam, or Adamax optimizers, it cycles the momentum between 0.85 and 0.95 in such a way that higher learning rates have lower momentum and lower learning rates have higher momentum, as suggested in [this paper](https://arxiv.org/pdf/1803.09820.pdf)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEWCAYAAACaBstRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd4XOd55/3vjd5BEh2YYRMr2ImBLFmyeqUqSRGw4mxsx4mzyWu/u7GTvFKcOIn39aY52d1snM16E9txiY0BKVKFlGVVS5ZVMGDvpNhmUAgQJEAQIPqzf5wDakSB5ACcmTPl/lzXXBycKefGkQb3nHOe83vEGINSSikVihSnC1BKKRU/tGkopZQKmTYNpZRSIdOmoZRSKmTaNJRSSoVMm4ZSSqmQadNQKs6JyPdF5P8P03t9TkR+Ge7nqsShTUNFjYicEJEhESm+bPkOETEiMtuZyiYWzj/GIa7PJSKbROSMiPSIyF4R+Vy01q9UKLRpqGg7Djw5/oOILANynCsnpvwQ8AOzgCLgPwCnHa1Iqcto01DR9kPgN4J+/izwg+AniEihiPxARDpF5KSI/ImIpNiPfU5E3haR/yYi3SJyTEQ+aS/3i0iHiHw26L0yReRbInJKRE6LyD+LSLb92B0iEhCRr9qvaxORz9uPfRH4DPBHInJBRJ63lxsRmRf0/pf2RoLe74+C3u9xEVkjIodF5KyI/PFVtk0t8H1jTJ8xZsQYs8MY82LQum4VkV/Zv7f/sr2Q6SKyVUR6ReQ9Ebkh6HWLRORle/2HRKQu6LEiEXlORM6LyPtA8Otm279vWtCyN0TktyYq/mrrUYlDm4aKtneBAhFZLCKpwKeBH132nP8JFAJzgduxmszngx7/BLAb69v4vwM/xfqDOw/4deAfRSTPfu5fAQuAlfbjVcDXg96r3F5XFfAF4NsiMt0Y8x3gx8DfGGPyjDGPhPj7lQNZQev5P3ZNNcCngD8VkTlXeO279vo/LSIzgx8QkVnAi/a2KbF/n51BT/k08BfAdOAo8E37dbnAy1jbqdR+3j+JSLX9um8DA0AF8Jv2bdJCWI9KENo0lBPG9zbuBQ4ALeMPBDWSp40xvcaYE8DfYR2qGXfcGPM9Y8wo0AC4gW8YYwaNMT8HhoB5IiLAF4HfN8acNcb0Av/Vfv9xw/Zrh40x24ALwMLr+N2GgW8aY4axmlkx8D/s32UfsB9YcYXXbgDeAv4UOC4iO0Wk1n7s14BXjDE/sWvtMsYEN43Nxpj3jTEjWM1upb38YeCEvb1GjDE7gE3ABntbrwe+bu/d7AX+bYq/9xXXM8X3UzEq7dpPUSrsfgi8CczhskNTWH9k04GTQctOYn1zHxd8nP8igDHm8mV5WN/Ic4Bmq38AIEBq0HO77D+04/rt105Vl93MLtU2Qb0Tvr8x5hzwFPCUPVjgW8AWEXFhNcYPrrLe9qD7wb/DLOATItId9Hga1n+DEvu+P+ix4O0+GVdbj0og2jRU1BljTorIcWAN1iGhYGewvq3PwvpWDjCToL2RSTiD9Ud6iTFmKq+fKAK6n4+euC8HAlN476uv2JgzIvItrHM+M7D+sN84hbfyA78wxtx7+QP2nsYIVkM6aC8OPizWZ/+bA5y375dPdj0qsejhKeWULwB3GWP6ghfa39K9wDdFJN8+lv8VPn7e45qMMWNY5xT+m4iUAohIlYjcH+JbnMY6rxJsJ/BrIpIqIg9gnXMJCxH5axFZKiJpIpIP/C5w1BjThXXI6R4RqbMfLxKRlVd/RwBeABaIyH8QkXT7Visii+1t/Qzw5yKSY59/uDSIwBjTidWsf93+fX+ToBPloa5n6ltExSJtGsoRxpgPjDG+Kzz8ZaxvuceAX2KdXP3uFFf1/2GdGH5XRM4DrxD6OYt/Bart0Upb7GX/CXgE6MYaXbXlSi+eghxgs/3ex7D2th4FMMacwtoz+ypwFqt5XencyCX2eZz7sM7jtGIdxvprINN+ypewDmW1A98HvnfZW/w28IdAF7AE+NUU16MShOgkTEoppUKlexpKKaVCpk1DKaVUyLRpKKWUCpk2DaWUUiFLmOs0iouLzezZs50uQyml4kpzc/MZY0xJqM9PmKYxe/ZsfL4rjeBUSik1ERGZVAqAHp5SSikVMm0aSimlQqZNQymlVMi0aSillAqZNg2llFIh06ahlFIqZNo0lFJKhUybhlIRNDI6xr+/d4r+oZFrP1mpKXhl/2mOdlyI2vqSvmn09A/zD68e4fDpXqdLUQlo6542/njzHn7wzlRnUVXqyjp7B/mtH/h49B9/GbV1Jn3TGDOGf3ztKP/+3imnS1EJqNFnzQTr9fnRuWtUuG3eYf3/9flbZkdtnUnfNKbnZnDvkjK27GxhcGTU6XJUAvGf7eeXR88wvzSPY519NJ8853RJKoEYY2ho8lMzazp/eP+iqK036ZsGQL3HTXf/MC/vP+10KSqBNDYHEIF/+sxqcjNSaWjyO12SSiDbT53jg84+6j3uqK5XmwZwy7xiqqZl47UPJSh1vUbHDBt9fm6dV8z8snweXl7J1j1tXBjUE+IqPLxNAXIyUlmzvCKq69WmAaSmCOtrXLx1pJOW7otOl6MSwNtHz9DaM0B9rfUtsK7WTf/QKFt3tzpcmUoEfYMjvLC7lYeXV5CXGd2wcm0atg01LoyBjbq3ocLA6/MzLSede6vLAFg9cxo3lOTq3qwKi6172ugbGr30pSSatGnY3DNyuGVeEY3NfsbGdJSLmrpzfUP8fN9pHl9ZRWZaKgAiQn2tm+aT5zjaocO71fXxNvmZW5LL6pnTo75ubRpB6jxuAucu8s6xLqdLUXFsy84WhkbHqLvsBOXaVS7SUkT3NtR1OdpxAd/Jc9R73IhI1NevTSPI/UvKKchKw+vTUS5qasaHQS6rKqS6suAjj5XkZ3LXolKe2R5geHTMoQpVvGts9pOaIqxdXeXI+rVpBMlKT+XxVVW8uLednv5hp8tRcWhvy3kOtvdSd4VjzfW1bs5cGOK1gx1RrkwlguHRMTY1t3DXolJK87McqUGbxmXqPG6GRsZ4dleL06WoOOT1+clMS+HRFZUTPn77ghJK8zNp1L1ZNQVvHOrkzIXBqF+bEUybxmWWVhVSXVGgh6jUpA0Mj7JlZwsPLi2nMDt9wuekpaawvsbF64c66Tg/EOUKVbxraPJTkp/JHQtLHKtBm8YE6mvd7G05z77WHqdLUXHkZ3vb6R0Y+dgJ8MttqHFZF/9t1xPiKnQd5wd4/VAH61e7SEt17k+3No0JPLaykoy0lEthc0qFwuvz456RzU1zi676vLkledw4ewaNvoCGGKqQPbOjhdExwwaPy9E6tGlMYFpOBvcvKWfzjhYGhjXEUF3bqa5+fvVBF3U1blJSrj0Msq7WzfEzfTSd0BBDdW3GGLxNfmpnT+eGkjxHa9GmcQX1Hjc9F4f5uYYYqhA0NvsRgfU1oX0LXLOsnLzMNA0xVCFpPnmOY2f6rnnoMxoi2jRE5AEROSQiR0XkqQkenyUir4rIbhF5Q0Rclz1eICIBEfnHSNY5kU/eUETVtGwd5aKuaXTMsLE5wG3zS6iclh3Sa3Iy0nhkRQXb9rTRO6DDu9XVNTT5yc1IZc2y6IYTTiRiTUNEUoFvAw8C1cCTIlJ92dO+BfzAGLMc+Abwl5c9/l+ANyNV49WkpAgbPC5+efQMgXP9TpSg4sRbRzpp6xmY9LfADR43F4dHeWF3W4QqU4ngwuAIW/e08ciKSnKjHE44kUjuadwIHDXGHDPGDAE/BR677DnVwGv2/deDHxeRGqAM+HkEa7yqJ+xDDXpCXF1Noy/A9Jx07qkundTrVrmnMb80T4d3q6vauruV/qFRNsTAoSmIbNOoAoI/DQF7WbBdwDr7/logX0SKRCQF+DvgD662AhH5ooj4RMTX2dkZprI/5Jqew63zitnYHNAQQzWhs31D/Hx/O2tXuS6FE4ZqPMRwx6lujugc9eoKGpr8zCvNY/XMaU6XAjh/IvwPgNtFZAdwO9ACjAK/B2wzxlz1K74x5jvGGI8xxlNSEpmLXeo8blq6L/L2B2ci8v4qvm3e0cLwqKGudmrDIB9fVUVaiugJcTWhox29bD/V7Vg44UQi2TRagOD9KZe97BJjTKsxZp0xZhXwNXtZN3Az8CUROYF13uM3ROSvIljrFd23pIxpOemaTKo+xhhDo8/PClchi8oLrv2CCRTnZXLP4jI272hhaERDDNVHeX0B0hwMJ5xIJJtGEzBfROaISAbwaeC54CeISLF9KArgaeC7AMaYzxhjZhpjZmPtjfzAGPOx0VfRkJmWyuMrq3hpXzvd/UNOlKBi1O5Az1XDCUNVV+uiq2+I1w7q8G71oeHRMZ7ZHuDuxaUU52U6Xc4lEWsaxpgR4EvAS8ABwGuM2Sci3xCRR+2n3QEcEpHDWCe9vxmpeq7HeIjhlh0aYqg+5PX5yUpP4ZErhBOG6rb5JZQVZOrerPqI1w52cObCUExcmxEsouO3jDHbgG2XLft60P2NwMZrvMf3ge9HoLyQVVcWsLSqAK8vwOdumeNkKSpGXBwa5bmdraxZWkFB1sThhKFKS03hiRoX/+uND2jvGaC80JnIaxVbvE1+SvMzuX2Bc+GEE3H6RHjcqPe42d92nr0tGmKo4MW9bfQOjoRtGOSGGjdjBjZpiKECTtvhhE/UOBtOOJHYqiaGPbqyisy0FB1TrwDr0NSsohxumjsjLO83uziXT8yZgdfn1xBDxabtAcYMMXNtRjBtGiEqzE7ngaXlbNEQw6R3squPd4+dpS7MwyDrPG5OdvXz3vGzYXtPFX+sUXkBbpwzgznFuU6X8zHaNCah3uPm/MAIL+1rd7oU5aBGX4AUgfWrwxtRvWZZBfmZOkd9sms6cY7jMRJOOBFtGpNw09wi3DOy9UOdxMbDCW9fUBL2E9bZGak8srKSbXvaOK8hhkmroclPXmYaa5aVO13KhLRpTEJKirChxs3bR7vwn9UQw2T05uFO2s9PPpwwVHUeNwPDYzy/qzUi769iW+/AMNvscMKcDOfDCSeiTWOSnqhxIQKNzTrKJRl5fX6KcjO4e3FZRN5/hauQhWX5es1GknphdxsXh0epc3h2vqvRpjFJldOy+dT8Ejb6/IxqiGFS6bowyCsHTrN2VRUZaZH56IhYkfy7/N0catcQw2TT0ORnQVkeK92xEU44EW0aU1DvcdPaM8Avj2qIYTL5MJwwsico1612kZ4qeu4syRw+3ctOf3fYR+WFmzaNKbinupTpOen6oU4ixhi8Pj8r3dNYUJYf0XXNyM3g3moNMUw23iY/6anC2lWxE044EW0aU5CZlsrjq6p4ed9pzvVpiGEy2Onv5vDpC1EbBrnB4+Zs3xCvHNAQw2QwNDLGMztauGdxGUUxFE44EW0aU1Rf62ZodIwtOzXEMBl4fQGy01N5ZEV05mi+bX4J5QVZujebJF47eJqzfbEXTjgRbRpTtKi8gOWuQhqaNPYh0fUPjfD8rlbr4rvrDCcMVWqK8ESNizcPd9LWczEq61TOaWjyU16QxW0xFk44EW0a16HO4+Zgey97NMQwoW3b086FwZGoD4Os81ghhht1+G1Ca+8Z4BeHO1lfU0VqSuyeAB+nTeM6PLKiUkMMk4DX52d2UQ43zglPOGGoZhblcPPcIhp1jvqENh5OGA+HpkCbxnUpzE5nzbIKnt3ZqiGGCer4mT7eP36WDQ4Ng6yrdXHqbD/vHu+K+rpV5I2NWaPybpo7g1lFsRdOOBFtGtepzuOmd2CEF/e2OV2KioBGn58UsZIAnPDg0grys9Jo1ENUCen9E2c52dUfN3sZoE3jun1izgxmzsjB26Qf6kQzMjrGxuYAdy4spazAmdn0stJTeXSFFWLYc1FDDBONt8lPfmYaDy6Nzqi8cNCmcZ1SUoQ6j4t3jnVxsqvP6XJUGP3icCcdvYOOT4RTX+tmcGSM5zTEMKGcHxhm2942HllZSXZGqtPlhEybRhisr3GRIrBRQwwTitfnpzgvg7sXlzpax7KqQhaV59OoAy4SyvO7WhkYHqM+jg5NgTaNsKgozOa2BSVsbA5oiGGC6Owd5NUDHXYOlLMfExGhzuNmd6CHA23nHa1FhY+3yc+i8nyWuwqdLmVStGmESb3HTVvPAG8e6XS6FBUGW3a0MDJmYiaieu2qKjJSdXh3ojjYfp5dgR7HRuVdD20aYXL34jJm5GboIYQEYIyhwedn9cxpzCuNbDhhqKbnZnDvEivEcHBEh3fHO29TIC7CCSeiTSNMMtJSWLuqipf3n6brwqDT5ajrsP1UN0c7ohdOGKo6j5vu/mFe3q8hhvFsaGSMzTsC3FttfdGMN9o0wqjO42Z41LBlp45yiWeNPj85Gak8vKLS6VI+4tZ5xVQWZumsfnHulQOnOdc/HHNfSkKlTSOMFpbns8I9Da+GGMatvkErnPChZRXkZcbWHM3jIYZvHemkpVtDDONVQ5OfisIsPjU/9sMJJ6JNI8zqPW4One5lV0BDDOPRtj1t9A2NRnx2vqna4HFjDGzS4d1xqbX7Im8e6eSJGldchBNORJtGmD28ooKsdB3lEq+8Pj9zi3PxzJrudCkTcs/I4ZZ5RXh9fg0xjEObmgMYAxtqYvNLSSi0aYRZQZYVYvj8zlYuDukol3jyQecFmk6ci/lhkHUeN4FzF3nnmIYYxpOxMUNjc4Cb5xYxsyjH6XKmTJtGBNR73PQOaohhvGn0BUhNEdbXxPYwyPuXlFOQlaZ7s3Hm3eNdnDrbT32MHvoMVUSbhog8ICKHROSoiDw1weOzRORVEdktIm+IiMtevlJE3hGRffZj9ZGsM9xunDOD2UU5NDTphzpejIyOsWm7FU5Ymu9MOGGostJTeWxlFS/ubaenX0MM44W3yU9+VhoPLC13upTrErGmISKpwLeBB4Fq4EkRqb7sad8CfmCMWQ58A/hLe3k/8BvGmCXAA8B/F5Fpkao13ESEDR437x0/y4kzGmIYD9441Eln72DMXAF+LfW1boZGxnhul85RHw96Lg7z4t52HltZSVZ6/IQTTiSSexo3AkeNMceMMUPAT4HHLntONfCaff/18ceNMYeNMUfs+61ABxBX49PWr7ZCDBubdW8jHjT4/BTnZXLnImfDCUO1tKqQ6ooCGvQQVVx4blcrgyNj1HtmOl3KdYtk06gCgv+PDtjLgu0C1tn31wL5IlIU/AQRuRHIAD6IUJ0RUV6YxR0LS9nYHGBkdMzpctRVdPQO8NrBDtavrnI8nHAy6jwu9racZ1+rDu+OdY0+K5xwaVWB06VcN6c/IX8A3C4iO4DbgRbg0pAjEakAfgh83hjzsb+8IvJFEfGJiK+zM/aCAus8bk6fH+StI2ecLkVdxebtLYyOGcfnzZisx+0QQ53VL7YdaDvP7kAP9bWxPSovVJFsGi1A8KfQZS+7xBjTaoxZZ4xZBXzNXtYNICIFwFbga8aYdydagTHmO8YYjzHGU1ISe0ev7lpUSlFuhp4Qj2Hj4YSeWdOZV5rndDmTMi0ng/vsEEOdoz52NTT5yUhN4fGVsT0qL1SRbBpNwHwRmSMiGcCngeeCnyAixSIyXsPTwHft5RnAZqyT5BsjWGNEZaSlsG51Fa8cOM0ZDTGMSc0nz3Gssy9uc4Dqa930XBzm5xpiGJMGR0bZsrOFe5eUMT0OwwknErGmYYwZAb4EvAQcALzGmH0i8g0RedR+2h3AIRE5DJQB37SX1wG3AZ8TkZ32bWWkao2kOo+bkTHDlh06yiUWee1wwoeWx88czcFuuaGYqmnZGskfo17ef5ru/uG4m53vaiKayGaM2QZsu2zZ14PubwQ+tidhjPkR8KNI1hYt88vyWTVzGg1Nfr5w65yEOKaZKC4MjvDC7jYeXl5BboyFE4YqxQ4x/IfXjhA4149revxeaZyIvL4AlYVZ3DKv2OlSwsbpE+FJod7j5kjHBXb6u50uRQXZtruN/qHRuL9Cd4N9bYnOUR9bWrov8taRTp7wuOM2nHAi2jSi4KHlFWSnp2rsQ4xp8Pm5oSSX1TNjM5wwVK7pOdxyQzGNvoCGGMaQjb7xcML4uGA0VNo0oiA/K52Hllfw/K42+odGnC5HAUc7emk+eY66GA8nDFVdrZuW7ou8/YEO744FVjihn1vmFeGekViHDLVpREmdx82FwRG27Wl3uhSFFU6YliKsW50Y3wLvqy6jMDtdZ/WLEe8c6yJw7mLcjsq7Gm0aUVI7ezpzi3Px6jUbjhu2wwnvWlRKSX6m0+WERVZ6Ko+vrOSlfe109w85XU7S8/r8FGSlcf+S+A4nnIg2jSgZDzF8/8RZjnVecLqcpPb6wQ7OXBhKuG+BdXaI4bM6R72jevqtcMLHV1XFfTjhRLRpRNH61VWkpgiNOsrFUV6fn5L8TO5YGHspAtdjSWUhSyoLNIHAYc/uamFoZCzhvpSM06YRRaUFWdy5sIRNGmLomI7zA7x+qJP1q12kxVE4Yajqa93sbzvP3hYNMXSK1+enuqKApVWFTpcSEYn3qYlxGzxuOnoH+cXh2AtYTAab7HDCeJk3Y7IeW1FFRprOUe+Ufa097G05H/fX/lxNSE1DRDwisllEttsz6e0Rkd2RLi4R3bWolOK8TD2E4ABjDI0+PzfOnsHckvgKJwxVYU46DywpZ4uGGDqi0RcgIy2Fx1ZWOl1KxIS6p/Fj4HvAeuAR4GH7XzVJ6akprF9dxWsHO+js1RDDaPKdPMexM32XrqBOVPW1bs4PjPDSPh3eHU0Dw6Ns3tHC/UvKmZaTGOGEEwm1aXQaY54zxhw3xpwcv0W0sgS2wQ4x3LxDT4hHU0OTn9w4DicM1c1zi3BNz9ZDVFH28/2n6bmYWOGEEwm1afyZiPyLiDwpIuvGbxGtLIHNK82jZtZ0Gpr8GKOxD9HQOzDM1t1tPLKikpyM+AwnDFVKirChxs3bR7vwn+13upyk0ejzUzUtm0/eUHTtJ8exUJvG54GVwANYh6XGD1GpKarzuPigs4/tpzTEMBq27m7j4vAodQl8gjLYEx4XIujw7igJnOvnl0fPsMHjIiWBwgknEmrTqLVnyPusMebz9u03I1pZgntoeSU5Gal6hXiUNPj8zC/NY5V7mtOlREXVtGxunVfMRp+fUQ0xjLjxhOEnEiyccCKhNo1fiUh1RCtJMnmZaTy8vIIXdrfSN6ghhpF05HQvO051J0w4Yajqa9209gzw9lENMYyksTFDoy/ArfOKk2I+k1Cbxk3AThE5pENuw6fO46ZvaJSte9qcLiWheX1+0lKEtasTY47mUN1bXca0nHQa9IR4RL39wRlauhMznHAioZ4RfCCiVSSpmlnTmVtihRgmy/9w0TY0MsYz21u4e7F1fUwyyUxL5fGVVfz7e6c41zeUMHNUxxqvL0Bhdjr3Vpc5XUpUhLqnYa5wU9dBRKjzuPGdPMcHGmIYEa8d7KCrbyihr9C9mjqPm6HRMbbs1DnqI6G7f4iX9rWzNkHDCScSatPYCrxg//sqcAx4MVJFJZN1doihjqmPDK/PT1lBJrfNT6xwwlBVVxawrKpQh3dHyLM7WxM6nHAiITUNY8wyY8xy+9/5wI3AO5EtLTmU5mdx16JSNjW3MKwhhmHV3jPAG4c6EjacMFR1tW4OtveyR0MMw66hyc/SqgKqKwucLiVqpvRJMsZsBz4R5lqSVp3HzZkLg7xxSEMMw2nT9gBjhqT6FjiRR1dUkqkhhmG3t6WH/W3nE/4K8MuFdCJcRL4S9GMKsBrQmV7C5M6FJZTkWyGGyXIyLdIuhRPOmcHs4lyny3FUYXY6Dy4t59mdrfzJQ9VJc+w90rw+PxlpKTy6IrlG5YW6p5EfdMvEOrfxWKSKSjZpqSmsW13F64c66OgdcLqchPD+8bOc6OpPum+BV1JX66Z3YISf7dUQw3AYGB5ly44WHlxaTmFOutPlRFWoTWO/MeYv7Ns3jTE/RlNuw6rO42Z0zPDMdh3lEg4NPj95mWmsWZbY4YShumlOEe4Z2RrJHyYv7Wvn/MBIUn4pCbVpPB3iMjVFN5TkUTt7Ol4d5XLdzg8Ms22PFU6YnaGHYsAKMayrcfPOsS5OdvU5XU7c8/r8uGdkc9PcxA4nnMhVm4aIPCgi/xOoEpF/CLp9H9DsizDb4HFz7EwfzSfPOV1KXHthVxsDw2NJe23GlayvsUIMN2qI4XXxn+3n7aNdbKhxJ3w44USutafRCviAAaA56PYccH9kS0s+Dy2rIDcjVQ8hXKcGn5+FZfmscCXmHM1TVTktm9vml7CxOaAhhtehsTmAiNWEk9FVm4YxZpcx5t+AecaYfwu6PWOM0a/DYZabmcbDyyvZuqeNCxpiOCWH2nvZ5e9mg8eVVOGEoaqvddPWM8BbR3R491SMjhk2+vx8an4JVdOynS7HEaGe07hRRF4WkcMickxEjovIsYhWlqTqat30D42ydbeOaJ4Kr89PeqqwdlVyDYMM1d2LS5mek67XbEzRL4+eobVnIClPgI8LtWn8K/D3wK1ALeCx/1VhtnrmNOaV5ukhqikYGhlj844W7llcRlGShROGKjMtlbWrXLy8/zRdF3SO+sny+vxMz0nnnupSp0txTKhNo8cY86IxpsMY0zV+u9aLROQBO079qIg8NcHjs0TkVTtu/Q0RcQU99lkROWLfPjuJ3ymuWSGGLraf6uZoR6/T5cSVVw+c5mzfUNLMzjdVdbUuhkcNW3bq3uxknOsb4uV9p3l8VRWZack7Ki/UpvG6iPytiNwsIqvHb1d7gYikAt8GHgSqgScnmMjpW8APjDHLgW8Af2m/dgbwZ1hRJTdizVE+PeTfKs6tW+0iLUXw+nSUy2Q0+PyUF2QlbThhqBaVF7DCVajDuydpy84WhkaTK5xwIqE2jU9gHZL6r8Df2bdvXeM1NwJHjTHHjDFDwE/5+FXk1cBr9v3Xgx6/H3jZGHPWPuH+Mkk0p0dxXiZ3Ly7lme0BDTEMUVvPRd483MkTNS5Sk3AY5GTV1bo5dLqX3QENMQyFMYaGJj/LXYUsrkiecMKJhJpye+cEt7uu8bIqIPjAfMBeFmwXsM6+vxbIF5GiEF+LiHxRRHy1sxNfAAAgAElEQVQi4uvsTKzRIFaI4RCvHexwupS4sKnZCifc4EnOYZCT9ciKSrLSU3RWvxDtaenhYHtv0u9lQIhNQ0TKRORfReRF++dqEflCGNb/B8DtIrIDuB1oAUZDfbEx5jvGGI8xxlNSkliHJG5fUEJpfiZePSF+TWNjBq8vwE1zZzCrKLnDCUNVkJXOmqUVPL+zlYtDIX/kkpbX5yczLYVHVlQ6XYrjQj089X3gJWB8ix0G/vM1XtMCBLdll73sEmNMqzFmnTFmFfA1e1l3KK9NdGmpKayvcfH6oQ5On9cQw6t57/hZTp3t1yvAJ2mDx03v4Agv7tU56q9mYHiUZ3e2smZZBYXZyRVOOJFQm0axMcYLjAEYY0a49h5BEzBfROaISAbwaawryS8RkWIRGa/haeC79v2XgPtEZLp9Avw+e1lSqfO4GTPWvBDqyrw+P/lZaTy4VMMJJ8PaM8vR4d3X8LO97fQOjOihT1uoTaPPPtdgAETkJuCqZ9DsxvIlrD/2BwCvMWafiHxDRB61n3YHcEhEDgNlwDft154F/gtW42kCvmEvSypzinO5cc4MGn0BHeVyBePhhI+uqNR5IiZpfI76946f5cQZDTG8koYmPzNn5HDTnOQLJ5xIqE3jK1h7CTeIyNvAD4AvX+tFxphtxpgFxpgbjDHjDeHrxpjn7PsbjTHz7ef8ljFmMOi13zXGzLNv35v0b5Yg6jxujp/po+mEprZM5LmdrQyOaDjhVK1f7SJFoLFZ9zYmcrKrj3eOdVHncSVlOOFEQh09tR3rRPUngd8BlhhjdkeyMGVZs6ycvMw0PYRwBV6fn0Xl+Syr0nDCqSgvzOL2BVaI4YgO7/6Yjc0BUpI4nHAioY6eSgXWAHdjnV/48mVTwKoIyclI45EVFWzb00bvwLDT5cSUA23n2R3ooc7j1nDC61Bf6+b0+UHeOnLG6VJiyuiYYWNzgNsWlFBRmJzhhBMJ9fDU88DngCI+OvWrioI6j5uLw6O8sFtHuQTz+vxkpKZoOOF1umtRGUW5Gbo3e5m3jnTS1jOg12ZcJi3E57nsqA/lgJXuaSwos0IMn7xxptPlxITBkVE272jh3uoypudmOF1OXMtIsxrv9391gjMXBinWsEfA+lIyIzeDexaXOV1KTAl1T+NFEbkvopWoKxof5bLT383h0xpiCPDK/g66+4c1nDBM6mrdjIwZtuxIqsuhrqjrwiAv7z/N2lVVZKSF+mcyOYS6Nd4FNovIRRE5LyK9InI+koWpj1q7qsoKMdRDCIAVTlhZmMWt84qdLiUhLCjLZ6V7Gg0aYgjAlp2tDI8aPTQ1gVCbxt8DNwM5xpgCY0y+MSa5U7uirCgvk3sWl7F5RwtDI8k9yqW1+yJvHdFwwnCrr3VzpOMCO/3dTpfiKGMM3iY/K9zTWFiup24vF2rT8AN7jX4FcVR9rZuuviFeO3ja6VIctbE5gDFWDIYKn4eXV5Cdnpr0s/rtDvRw6HQvdXoF+IRCbRrHgDdE5GkR+cr4LZKFqY+7bUEJ5QVZST3KxQon9PPJG4pwz8hxupyEkp+VzpplFTy/q43+oeSdo77B5ycrXcMJryTUpnEceBXIQIfcOiY1RVhfU8UvDnfS3pOcIYbvHusicO6iXgEeIXUeFxcGR9i2p93pUhxxcWiU5+1wwoIsDSecSEhDbo0xfwEgInn2zxciWZS6sg01br79+gds2h7g/7lzntPlRF2Dz09BVhr3Lyl3upSEdOOcGcwpzsXb5OeJJLwK+sW9bfQOjugJ8KsI9YrwpfacF/uAfSLSLCJLIluamsjs4lw+MWcGXl/yjXLp6R/mxb3tPLaySsMJI0RE2OBx8f6JsxzrTL7vhg1NfmYX5fCJOTOcLiVmhXp46jvAV4wxs4wxs4CvAv8ncmWpq6mvdXOyq5/3jidX8O9zu6yRY3poKrI+DDFMrkj+E2f6eO/4WTZoLM1Vhdo0co0xr4//YIx5A9Ap0hzy4NIK8jPTku6ajQafn8UVBSyp1NHekVRWkMWdC0vZlGQhho3NfiuccHXyHZabjJBHT4nIn4rIbPv2J1gjqpQDsjNSeWRlJdv2tnE+SUIM97X2sLflPPUel34LjIINHjcdvYP84nCn06VExcjoGBubA9yxsJTywiyny4lpoTaN3wRKgE32rRj4fKSKUtdW73EzMDzG87tanS4lKhp9ATJSU3hcwwmj4u7FpRTnJU+I4VtHznD6/KBemxGCUJvGDVhzdqdgDbu9G3gzUkWpa1vuKmRhWT5eX+Ifdx4YtsIJ71tSxrQcDSeMhvTUFNatdvHawQ46ewev/YI419Dkpyg3g7sWaTjhtYTaNH6MNX/3OuBh+/ZIpIpS1yYi1NW62eXv5lB7YocYvrz/ND0Xh/UEeJTVeVyMjBk270jsLyZdFwZ55YCGE4Yq1C3UaYx53hhz3BhzcvwW0crUNa1dVUV6qiT8IQSvz0/VtGxuuUHDCaNpXmk+q2cmfojh5h0tjIwZ/VISolCbxp+JyL+IyJMism78FtHK1DXNyM3g3uoyNu8IJGyIYeBcP788eoYnanSOZifUedx80NnH9lOJGWJojKGhyc+qmdOYX6YhF6EItWl8HlgJPIB1WOoRrENUymF1Hjfn+od55UBihhhutK8V2KAnKB3x8IpKcjJSE3Z4905/N0c6LugV4JMQ6sx9tcaYhRGtRE3Jp+aXUFFohRiuWVbhdDlhNTZmaPQFuOWGYlzTNZzQCXmZaTy0rIIXdrfy9Ueqyc0M9U9GfPD6/GSnp/Lw8sT67ERSqHsavxKR6ohWoqYkNUV4osbFm0c6ae2+6HQ5YfWrD7po6b6os/M5rK7WTd/QKFv3JNYc9f1DIzy/q401yyrI13DCkIXaNG4CdorIIRHZLSJ7RGR3JAtTodtQ48YY2JRgsQ8NPj+F2encV63DIJ3kmTWduXaIYSLZtqedC4MjegJ8kkJtGg8A84H7+PB8hg65jREzi3K4eW4Rjc0BxsYSY5RLd/8QL+1r5/GVlRpO6DArxNCN7+Q5PkigEENvk585xbnUzp7udClxJaSmETzMVofcxqb6Wjenzvbz7vEup0sJi2d3tjI0MqaHpmLE+poqUlMkYWb1O9Z5gfdPnGWDxtJMml7JkiAeWFpOflbihBg2NPlZUlnAkspCp0tRQGn+eIhhC8MJEGLY2BywzgdqOOGkadNIEFnpqTy2spIX97bTczG+Qwz3tvSwv+28HmuOMXUeF2cuDPLGofgOMRwZHWNTc4A7FpRQWqDhhJOlTSOB1HtmMjgyxnNxHmLo9fnJSEvhsRUaThhL7lxUSnFeZtwnEPzicCcdvYN66HOKtGkkkKVVBSwqz6cxjo87DwyPsmVHCw8sKacwR4dBxpL01BTWr67i9UMddPTG7xz1DU1+ivMyuGtRqdOlxKWINg0RecAepntURJ6a4PGZIvK6iOywh/KusZeni8i/2UN7D4jI05GsM1GICPW1bnYHejjQdt7pcqbkpX3tnB/QYZCxaoPHzeiY4ZntLU6XMiWdvYO8drCDdatdpKfqd+apiNhWE5FU4NvAg0A18OQEFwj+CeA1xqwCPg38k718A5BpjFkG1AC/IyKzI1VrInl8ZRUZqSlxewjB6/Pjmp7NzXOLnC5FTWBeaR6eWdPxxmmI4eYdAUbGjM6bcR0i2WpvBI4aY44ZY4aAnwKPXfYcA4zP3VkItAYtzxWRNCAbGALi86tzlE3PzeDeJWVs2dnC4Mio0+VMiv9sP28f7WJDjVvDCWNYncfNsTN9NJ8853Qpk2KMwesLsHrmNOaVajjhVEWyaVQBwV93A/ayYH8O/LqIBIBtwJft5RuBPqANOAV8yxhzNoK1JpR6j5vu/mFe3h9fIYaNzQFE4An9FhjTHlpeQU5GatztzW4/1c3Rjgt66PM6OX1Q70ng+8YYF7AG+KGIpGDtpYwClcAc4KsiMvfyF4vIF0XEJyK+zs74HgYYTrfMK6ayMCuuZvUbHTNs9Pm5dV4xVdOynS5HXUVuZhoPL69g6542LgyOOF1OyLxNfnIyUnloeaXTpcS1SDaNFqwpYse57GXBvgB4AYwx7wBZWPOP/xrwM2PMsDGmA3gb8Fy+AmPMd4wxHmOMp6SkJAK/QnxKTRGe8Lh560gnLXESYvj20TO09gzot8A4UV/rpn9olK2742N4d9/gCC/sbuWhZRXkJVhSb7RFsmk0AfNFZI6IZGCd6H7usuecwppvHBFZjNU0Ou3ld9nLc7ECEw9GsNaEs6HGhTGwMU72Nhp8fqblpHOvhhPGhdUzp3NDSW7cHKLauqeNvqFR/VISBhFrGsaYEeBLwEvAAaxRUvtE5Bsi8qj9tK8Cvy0iu4CfAJ8z1pCMbwN5IrIPq/l8zxijqbqT4J6Rwy3zimhs9sd8iOG5viFe3neax1dWkZmm4YTxQESo87jt8wSxP0d9o8/P3JJcamZpOOH1iug5DWPMNmPMAmPMDcaYb9rLvm6Mec6+v98Yc4sxZoUxZqUx5uf28gvGmA3GmCXGmGpjzN9Gss5EVedxEzh3kXeOxXaI4ZadLQyNjunsaXFm3WqXHWIY23uzH3ReoOnEOeo8bg0nDAOnT4SrCLp/STkFWWkxnUw6PkfzsqpCqisLrv0CFTNK8jO5a1Epz2wPxHSIodfnJzVFWLdaY2nCQZtGAstKT+XxVVVWiGF/bIYY7m05z8H2Xs0BilP1HjdnLgzx2sEOp0uZ0PDoGJuaW7hzYSml+RpOGA7aNBJcncfN0MgYz+6KzdiHBt8pMtNSeHSFDoOMR3csLKEkPzNmI/nfONTJmQuDegI8jLRpJLilVYVUVxTE5CGqgeFRnt3ZyoNLyynM1nDCeJSWmsL61S5eP9TB6fOxF2Lo9fkpzsvkjoU6JD9ctGkkgfpaN3tbzrOvtcfpUj7iZ3vb6R0Y0UNTca7O42LMwKbtsXVCvKN3gNcOdrC+pkrDCcNIt2QSeGxlJRlpKTF3CMHr8+Oekc1NczScMJ7NLcmjdvZ0Gn2BmAoxfGZ7C6NjRkflhZk2jSQwLSeD+5eUs2VnKwPDsRFieKqrn1990EWdhhMmhDqPm+Nn+mg6ERshhlY4oR/PrOncUJLndDkJRZtGkqj3uOm5OMzPYyTEsLHZjwisr9FwwkTw0PIKcmMoxLD55DmOdfbpoc8I0KaRJD55QxFV07JjYla/0THDxuYAt80voVLDCRNCTkYaj6yoZNueNnoHnB/e3dDkJzcjlYeWVThdSsLRppEkUlKEDR4Xvzx6hsC5fkdreetIJ20aTphw6mrdXBwe5YXdbY7WcWFwhK172nh4eSW5Gk4Ydto0ksgT9qGgRodjH7w+P9Nz0rl7sc7RnEhWuacxrzTP8UNUW3e30j80qoemIkSbRhJxTc/h1nnFbGwOOBZieLZviJf3n2btKpeGEyYYEaHe42anv5vDp50LMfT6AtxQksvqmdMcqyGRadNIMnUeNy3dF3n7gzOOrH/zjhaGRw11tXoCPBGtXV1FWoo4Nrz7aEcvzSfPUV+r4YSRok0jydxbXUZhdrojyaTGGBp9fla4CllUruGEiag4L5O7F5eyeUcLQyPRDzH0+gKkpQhrV+mXkkjRppFkstJTWbuqipf2tdPdPxTVde8O9Gg4YRKor3XT1TfEawejO7x7eHSMZ7YHuGtRKSX5mVFddzLRppGENnhcDI2MsWVHdEMMG3x+stJTeETDCRPabfNLKM3PjPoJ8dcOdnDmwpCOyoswbRpJaEllIUurCqJ6iOri0CjP72xlzdIKCrI0nDCRpaWm8ESNi18c7qS9J3ohho0+P6X5mdy+QMMJI0mbRpKq97jZ33aevS3RCTF8cW8bvYMjbNAcoKRQ53FHNcSw4/wArx/qZH2NizQNJ4wo3bpJ6tEVVVaIYZSuEPf6/MwqyuGmuTOisj7lrNnFudw4ZwZenz8qIYYbtwcYHTNs0FiaiNOmkaQKc9J5cGk5W3a0RDzE8GRXH+8eO6tzNCeZeo+bk139vHf8bETXY43KC3Dj7BnM1XDCiNOmkcTqPG7OD4zw0r72iK7H6/OTIrB+tX4LTCYPLisnLzMt4tdsNJ04x/EzGk4YLdo0ktjNc4twz8iO6CGq8XDC2xeUUF6oczQnk0shhnvbOB/BEEOvz09eZhprlpVHbB3qQ9o0klhKirChxs3bR7vwn41MiOGbhzs5fV7naE5W9bVuBobHeH5Xa0Tev3dgmK2723hkRQU5GRpOGA3aNJLc+hoXItDYHJlRLl6fn6LcDO5aVBaR91exbYWrkAVleREb3v3C7jYuDo/qqLwo0qaR5KqmZfOp+SVs9PkZDXOIYdeFQV45cJq1q6yRWir5iAh1Hje7/N0cag9/iKHX52d+aR6r3BpOGC36SVbUeVy09gzwy6PhDTH8MJxQvwUms7WrqkhPlbBfIX7kdC87TnVrOGGUadNQ3FtdxvSc9LCeEDfG0NDkZ6V7GgvK8sP2vir+FOVlcs/iMjbvCIQ1xNDr85OWIjy+qips76muTZuGIjMtlcdXVfHyvtOc6wtPiOFOfzdHOi7oCXAFWLP6nesf5pUD4QkxHBoZ45ntLdyzuIziPA0njCZtGgqwrtkYGh1jy87whBh6fQGy01N5eLnO0aysEMPygqyw7c2+dvA0XX1DOi+LA7RpKAAWVxSw3FVIQ9P1xz70D43w/K5W1iyrIF/DCRWQmiI8UePizcOdtPVcvO738/oClBVkctt8DSeMNm0a6pINHjcH23vZc50hhtv2tHNhcIQ6j34LVB/a4HExZmDjdQ6/be8Z4I1DHTyh4YSOiOgWF5EHROSQiBwVkacmeHymiLwuIjtEZLeIrAl6bLmIvCMi+0Rkj4jo5cQR9uiKSjLDEGLo9fmZYwfWKTVuVlEuN82dQeN1zlG/aXuAMQMbavR8mRMi1jREJBX4NvAgUA08KSLVlz3tTwCvMWYV8Gngn+zXpgE/Av6jMWYJcAcQuRwCBUBhdjprllXw7M7WKYcYHj/Tx/vHz7LB49JhkOpj6mvdnDrbz7vHu6b0+vEpgz8xZwazi3PDXJ0KRST3NG4EjhpjjhljhoCfAo9d9hwDjE8WXQiMZw3cB+w2xuwCMMZ0GWMiG8WqAOsQQu/ACC/ubZvS6xs1nFBdxQNLKsjPTKNxioeo3jt+lhNd/dTpFeCOiWTTqAKCj3ME7GXB/hz4dREJANuAL9vLFwBGRF4Ske0i8kcTrUBEvigiPhHxdXZ2hrf6JHXTnCJmzsjB2zT5D/XI6BgbmwPcubCUsgI9mqg+LjsjlUdXVrJtTxs9Fyd/8MDr85OfmcaaZToqzylOn0V6Evi+McYFrAF+KCIpQBpwK/AZ+9+1InL35S82xnzHGOMxxnhKSnQURTikpAh1HhfvHOviZFffpF77i8OddPQOag6Quqr6WjeDI2M8N8kQw/MDw2zb08YjKyvJzkiNUHXqWiLZNFqA4L8eLntZsC8AXgBjzDtAFlCMtVfypjHmjDGmH2svZHUEa1VB1te4SBHYOMkQQ6/PT3FeBncvLo1QZSoRLKsqZFF5Po2THHDxwq42BobH9NCUwyLZNJqA+SIyR0QysE50P3fZc04BdwOIyGKsptEJvAQsE5Ec+6T47cD+CNaqglQUZnPbghI2NgdCDjHs7B3k1QMdrFvtIl2HQaqrGA8x3B3o4UDb+ZBf1+Dzs7AsnxWuwghWp64lYp9uY8wI8CWsBnAAa5TUPhH5hog8aj/tq8Bvi8gu4CfA54zlHPD3WI1nJ7DdGLM1UrWqj6vzuGnrGeDNI6GdK9q8I8DImNFrM1RIHp9kiOGh9l52+bt1VF4MiOisJcaYbViHloKXfT3o/n7gliu89kdYw26VA+5ZXMaM3AwafX7uXHj1w03GGLy+AKtnTmNeqYYTqmubkZvBfdXlbNnZwtNrFpGZdvVzFF6fn/RUYZ2OynOcHkdQE8pIS2Htqipe3n+arguDV33u9lPdHO24oMea1aTU1brp7h/m5f1XDzEcGhlj844W7q22vsgoZ2nTUFdU53EzPGrYsvPqo1wafX5yMlJ5eEVllCpTieDWecVUFmZdc1a/Vw+c5mzfkI7KixHaNNQVLSzPZ4V7Gt6rhBj2DVrhhA8tqyAvU+doVqEbDzF860gnLd1XDjFs8PkpL8jScMIYoU1DXVWdx8Wh073sCkwcYrh1Txt9Q6M6O5+akidq3JirhBi29VzkzcOdPFHjIjVFT4DHAm0a6qoeWVFJVvqVQwwbfX7mluTimTU9ypWpRDCzKIdP3lBEY7N/whDDTc1WOKGeL4sd2jTUVRVkWSGGz+9s5eLQR+O/Pui8QNOJc9R5dI5mNXX1tW4C5y7yzrGPhhiOjVmj8m6eW8TMohyHqlOX06ahrqnO46Z38OMhho2+AKkpwrrVOkezmrr7l5STn5X2sb3Z946f5dTZfp2dL8Zo01DX9Ik5M5hdlPORC7FGRsfYtN0KJyzN13BCNXVZ6ak8vrKKF/e209P/YYih1+cnPyuNB5dqOGEs0aahrklE2OBxW7HUZ6wQw9cPddLZO6hXgKuwqPO4GRoZ49ldVjxdz0UrnPDRFZVkpWs4YSzRpqFCsn61FWLY2GztbVjhhJncuUjDCdX1W1pVwOKKgkuHqJ7f1crgyBj1Oiov5mjTUCEpL8zijoWlbGwO0N4zwGsHO1hfU6XhhCosRIR6j4u9LefZ19qD1+dnUXk+y6o0nDDW6CdehazO4+L0+UG+4t3J6JjROZpVWD22soqM1BS+ufUAuwM9OiovRmnTUCG7a1EZRbkZ/OqDLjyzpjOvNM/pklQCmZ6bwX1LyvjVB11kpFrZZyr2aNNQIRsPMQS92EpFxvj/V/dWlzFdwwljkoYFqUn57dvmYrCuFFcq3G6dV8zv3XGD7mXEMLlSEF288Xg8xufzOV2GUkrFFRFpNsZ4Qn2+Hp5SSikVMm0aSimlQqZNQymlVMi0aSillAqZNg2llFIh06ahlFIqZNo0lFJKhUybhlJKqZAlzMV9ItIJnLyOtygGzoSpnHDT2qYmVmuL1bpAa5uqeK5tljGmJNQ3S5imcb1ExDeZqyKjSWubmlitLVbrAq1tqpKpNj08pZRSKmTaNJRSSoVMm8aHvuN0AVehtU1NrNYWq3WB1jZVSVObntNQSikVMt3TUEopFTJtGkoppUKW9E1DRB4QkUMiclREnnJg/W4ReV1E9ovIPhH5T/byGSLysogcsf+dbi8XEfkHu97dIrI6CjWmisgOEXnB/nmOiLxn19AgIhn28kz756P247MjXNc0EdkoIgdF5ICI3Bwr201Eft/+77lXRH4iIllObTcR+a6IdIjI3qBlk95OIvJZ+/lHROSzEaztb+3/prtFZLOITAt67Gm7tkMicn/Q8rB/jieqLeixr4qIEZFi+2fHt5u9/Mv2ttsnIn8TtDx8280Yk7Q3IBX4AJgLZAC7gOoo11ABrLbv5wOHgWrgb4Cn7OVPAX9t318DvAgIcBPwXhRq/Arw78AL9s9e4NP2/X8Gfte+/3vAP9v3Pw00RLiufwN+y76fAUyLhe0GVAHHgeyg7fU5p7YbcBuwGtgbtGxS2wmYARyz/51u358eodruA9Ls+38dVFu1/RnNBObYn93USH2OJ6rNXu4GXsK6mLg4hrbbncArQKb9c2kktlvEPtDxcANuBl4K+vlp4GmHa3oWuBc4BFTYyyqAQ/b9/w08GfT8S8+LUD0u4FXgLuAF+0NxJuhDfWkb2h+km+37afbzJEJ1FWL9YZbLlju+3bCaht/+Q5Fmb7f7ndxuwOzL/sBMajsBTwL/O2j5R54Xztoue2wt8GP7/kc+n+PbLZKf44lqAzYCK4ATfNg0HN9uWF9K7pngeWHdbsl+eGr8wz0uYC9zhH1YYhXwHlBmjGmzH2oHyuz70a75vwN/BIzZPxcB3caYkQnWf6k2+/Ee+/mRMAfoBL5nHzr7FxHJJQa2mzGmBfgWcApow9oOzcTGdhs32e3k1GflN7G+wcdEbSLyGNBijNl12UOO1wYsAD5lH+L8hYjURqK2ZG8aMUNE8oBNwH82xpwPfsxYXwOiPjZaRB4GOowxzdFedwjSsHbP/5cxZhXQh3WY5RIHt9t04DGsxlYJ5AIPRLuOUDm1na5FRL4GjAA/droWABHJAf4Y+LrTtVxBGtbe7U3AHwJeEZFwryTZm0YL1vHJcS57WVSJSDpWw/ixMeYZe/FpEamwH68AOuzl0az5FuBRETkB/BTrENX/AKaJSNoE679Um/14IdAVodoCQMAY857980asJhIL2+0e4LgxptMYMww8g7UtY2G7jZvsdorqZ0VEPgc8DHzGbmqxUNsNWF8EdtmfCRewXUTKY6A2sD4TzxjL+1hHB4rDXVuyN40mYL49qiUD6yTkc9EswP4m8K/AAWPM3wc99BwwPtLis1jnOsaX/4Y9WuMmoCfoMENYGWOeNsa4jDGzsbbNa8aYzwCvA09cobbxmp+wnx+Rb7DGmHbALyIL7UV3A/uJge2GdVjqJhHJsf/7jtfm+HYLMtnt9BJwn4hMt/ek7rOXhZ2IPIB1SPRRY0z/ZTV/WqzRZnOA+cD7ROlzbIzZY4wpNcbMtj8TAaxBLO3EwHYDtmCdDEdEFmCd3D5DuLdbOE7IxPMNa9TDYaxRBF9zYP23Yh0a2A3stG9rsI5pvwocwRoRMcN+vgDftuvdA3iiVOcdfDh6aq79P91RoJEPR2tk2T8ftR+fG+GaVgI+e9ttwRqdEhPbDfgL4CCwF/gh1sgVR7Yb8BOscyvDWH/ovjCV7YR1fuGofft8BGs7inWsffzz8M9Bz/+aXdsh4MGg5WH/HE9U22WPn+DDE+GxsN0ygB/Z/89tB+6KxHbTGBGllFIhS/bDU0oppSZBm4ZSSqmQadNQSikVMm0aSimlQqZNQ5byLu0AAAJgSURBVCmlVMi0aShlE5Ff2f/OFpFfC/N7//FE61Iq3uiQW6UuIyJ3AH9gjHl4Eq9JMx/mSk30+AVjTF446lPKSbqnoZRNRC7Yd/8KK/htp1jzYqSKNcdDkz1Xwu/Yz79DRN4SkeewrvhGRLaISLM9n8EX7WV/BWTb7/fj4HXZVxD/rVjzbuwRkfqg935DPpwv5MfjOUIi8ldizb+yW0S+Fc1tpFTatZ+iVNJ5iqA9DfuPf48xplZEMoG3ReTn9nNXA0uNMcftn3/TGHNWRLKBJhHZZIx5SkS+ZIxZOcG61mFd2b4CKyeoSUTetB9bBSwBWoG3gVtE5ABWXPgiY4yRoAmKlIoG3dNQ6truw8oV2okVW1+Eld8D8H5QwwD4f0VkF/AuVhjcfK7uVuAnxphRY8xp4BfAeKT1+8aYgDFmDCtOYzZWbPoA8K8isg7on+A9lYoYbRpKXZsAXzbGrLRvc4wx43safZeeZJ0LuQdrQqUVwA6sXKmpGgy6P4o1gdMIcCNWqu/DwM+u4/2VmjRtGkp9XC/W1LvjXgJ+146wR0QWiDXh0+UKgXPGmH4RWYQ1r8G44fHXX+YtoN4+b1KCNY3n+1cqTKx5VwqNMduA38c6rKVU1Og5DaU+bjcwah9m+j7WHCKzseZOEKwZAx+f4HU/A/6jfd7hENYhqnHfAXaLyHZjxcuP24w17eYurLTjPzLGtNtNZyL5wLMikoW1B/SVqf2KSk2NDrlVSikVMj08pZRSKmTaNJRSSoVMm4ZSSqmQadNQSikVMm0aSimlQqZNQymlVMi0aSillArZ/wVxKiu1Sft6WQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.plot('momentum')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Additional Cooldowns\n",
    "Since we are not [overfitting](https://en.wikipedia.org/wiki/Overfitting#Machine_learning) yet (i.e., validation loss is not increasing while training loss decreases), let's do a few more \"cooldowns\" starting at a smaller learning rate to improve the accuracy score further using the regular ```fit``` method that employs SGDR. These \"cooldown\" epochs will start the learning rate at 0.005/10 and gradually decrease it to a very small value.  We will use the **checkpoint_folder** argument covered earlier, so that we can restore the weights from any epoch in case we train too much and overfit.  If you are not using Linux, you should set this to your folder path of choice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/1\n",
      "25000/25000 [==============================] - 6s 220us/step - loss: 0.1927 - acc: 0.9304 - val_loss: 0.2804 - val_acc: 0.8901\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fe7de723b00>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.fit(0.005, 1, cycle_len=1, checkpoint_folder='/tmp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/1\n",
      "25000/25000 [==============================] - 6s 222us/step - loss: 0.1535 - acc: 0.9518 - val_loss: 0.2797 - val_acc: 0.8904\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fe7de7235f8>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.fit(0.005/10, 1, cycle_len=1, checkpoint_folder='/tmp')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we are running multiple short cooldown phases here - cycles of only one epoch. This essentially amounts to SGDR.  Although we are not doing it here, we can also run one longer cooldown by simply calling ```fit``` with a larger value for ```cycle_len``` and leaving the number of cycles at 1. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The 1cycle Policy\n",
    "\n",
    "The [1cycle policy](https://arxiv.org/pdf/1803.09820.pdf) was proposed by Leslie Smith (as was the triangular learning rate policy).  The 1cycle policy runs a single triangular cycle over the course of training and then annihilates the learning rate to a near-zero value towards the end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "begin training using onecycle policy with max lr of 0.005...\n",
      "Train on 25000 samples, validate on 25000 samples\n",
      "Epoch 1/3\n",
      "25000/25000 [==============================] - 7s 266us/step - loss: 0.5566 - acc: 0.7379 - val_loss: 0.3626 - val_acc: 0.8598\n",
      "Epoch 2/3\n",
      "25000/25000 [==============================] - 6s 252us/step - loss: 0.2684 - acc: 0.8990 - val_loss: 0.2864 - val_acc: 0.8870\n",
      "Epoch 3/3\n",
      "25000/25000 [==============================] - 6s 251us/step - loss: 0.1737 - acc: 0.9410 - val_loss: 0.2774 - val_acc: 0.8902\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fe7de718080>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# rebuild the model to train from scratch \n",
    "learner.set_model(get_model())\n",
    "\n",
    "# training using the 1cycle policy\n",
    "learner.fit_onecycle(0.005, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 1cycle policy runs a single triangular cycle over all the epochs and also cycles the momentum in the shape of a *V*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd8VfX9x/HXh4Qk7L3DCJuwM3Br62JUwbpAQEGxKoKjtlXUauv6VeseOH9YUEBA1EqdWLFqSwUSCCMsw96EvUeS7++Pe2LzowGybs69ue/n45GHN+d8zz2fc83Nm3vON59jzjlERERKqpLfBYiISHhTkIiISKkoSEREpFQUJCIiUioKEhERKRUFiYiIlIqCRKQcmVkrM3NmFl1Gz7fWzC4u67EixaEgkYhysl+mZvYzM8szswNmtt/MVpjZjad5rhFmttwbv83MPjOzGsGrXiQ0lcm/ikQqiM3OuXgzM6AvMMPMZjvnVpw40MwuAP4H6OOcW2BmdYHLy7lekZCgTyQiJ3ABnwG7gG4nGZYK/Ns5t8DbZpdzboJzbj+AmVUxs2fNbJ2Z7TWzf5pZlQLbDzGz9Wa2w8wezF9oZpXMbIyZrTKznWY2zQup/PXXe8+5s+B23rrxZvZ4ge9/ZmYbCyv+dPsRKQ4FicgJvF+y/YH6QNZJhs0BepvZI2Z2jpnFnrD+GSAZOBuoC9wL5BVYfy7QAbgIeNjMOnnL7wCuAC4AmgK7gbFeXYnAa8D13rp6QHwJD/Ok+xEpLgWJyH80NbM9wGHgI+Ce/E8cJ3LOfQ9cCSQBnwI7zew5M4sys0rATcBdzrlNzrlc59xs59zRAk/xiHPusHNuIbAQ6O4tvw140Dm30Rv/R+Bq7+L81cAnzrnvvHUP8f/DqThOtR+RYtEPjch/5F8jiQWeBC4EXjjZYOfc58DnXnD8HHgfWEEghOKAVafY19YCjw8B1b3HLYGPzKxgQOQCjQh8cthQYP8HzWxnEY/tRKfaz6YSPqdEKH0iETmB9y/0+4CuZnZFEcbnOee+BmYBXYAdwBGgTQl2vwHo65yrXeArzjm3CdgCNM8faGZVCZzeyncQqFrg+8Yl3I9IsShIJBJVNrO4Al//9cncOXcMeBZ4uLAnMLMBZjbIzOpYQC8C1xt+cM7lAW8Dz5lZU+9011mFXEcpzOvAE2bW0ttPAzMb4K2bDlxmZueaWQzwKP//PZwB9DOzumbWGLi7hPsRKRYFiUSizwhcB8n/+uNJxr0NtDCzwqb17gZ+BfwI7AMmAk875yZ5638LLAbmEZj99RRFe7+9CMwAZprZfuAH4AwA51wmMAqYTODTyW6g4Kysdwlcb1kLzASmlmQ/IsVlurGViIiUhj6RiIhIqShIRESkVBQkIiJSKgoSEREplYj4g8T69eu7Vq1a+V2GiEjYSE9P3+Gca1CUsRERJK1atSItLc3vMkREwoaZrSvqWJ3aEhGRUlGQiIhIqShIRESkVBQkIiJSKgoSEREplaAGiZn1MbMVZpZlZmMKWR9rZlO99XPMrFWBdfd7y1eYWe8Cy9ea2WIzyzAzTcUSEfFZ0Kb/mlkUgVt3XkKgQ+k8M5vhnFtaYNgIYLdzrq2ZDSLQIXWgd0vRQUBnAjfz+buZtXfO5Xrb/dw5tyNYtYuISNEF8xNJLyDLObfau7fDFODE+x0MACZ4j6cDF5mZecunOOeOOufWELhvdq8g1ipS7uav383sLP17SMJfMIOkGQVuC0rgU0mzk41xzuUAewnc8e1U2zoC91BIN7NbTrZzM7vFzNLMLC07O7tUByJS1hZu2MPgt35g2F/mkrFhj9/liJRKOF5sP9c5lwT0BUaZ2fmFDXLOvemcS3HOpTRoUKS/8hcpFxt3H2LEhDQa1IilYY04Rk2az+6Dx/wuS6TEghkkmyhwf2kg3ltW6Bjvdqe1gJ2n2jb/ntLOue3AR+iUl4SR/UeOM2J8GkdzcvnL8FReG5pE9v6j3D01g7w83WROwlMwg2Qe0M7MErz7Sw8icGvPgmYAw7zHVwOzXOCWjTOAQd6srgSgHTDXzKqZWQ0AM6sGXAosCeIxiJSZnNw8Rk9ewKrsA7w+NJm2DWvQLb42D1+eyLcrs3l5VpbfJYqUSNBmbTnncsxsNPAlEAW87ZzLNLNHgTTn3AxgHPCumWURuK/1IG/bTDObBiwFcoBRzrlcM2sEfBS4Hk80MNk590WwjkGkrDjn+OPfMvl2ZTZPXtmVc9rW/2ndkDNakL5uNy98vZKeLWpzfnudipXwEhH3bE9JSXHq/it++t/vV/P4p8u49YLW3N+303+tP3QshyvG/ovs/Uf59M7zaFq7ig9VivyHmaU751KKMjYcL7aLhJWvlm7jic+W0bdLY+7r3bHQMVVjonltaDLHcx23T5rPsZy8cq5SpOQUJCJBtGTTXu58bwHdmtXiuWt7UKmSnXRsmwbV+fPV3cjYsIf/+WxZOVYpUjoKEpEg2bL3MCMmzKNutRjeGpZClZio027Tr2sTbjongfGz1zJj4eZyqFKk9BQkIkFw8GgON41P4+DRXN4enkrDGnFF3vb+fh1JblmHMR8sImv7/iBWKVI2FCQiZSw3z3HnewtYuW0/Y4ck0aFxjWJtXzmqEmMHJ1GlchS3TZzPwaM5QapUpGwoSETK2GOfLOXr5dv5Y//OXFDCqbyNa8Xx0nU9WZ19gPs/XEwkzK6U8KUgESlDE2avZfzstYw4N4Hrz2xZquc6p2197rmkPTMWbubdH9aVUYUiZU9BIlJGvlm+nUf+lsnFnRrxQL///luRkrj9Z225sGNDHvtkKQvW7y6T5xQpawoSkTKwdPM+Rk+eT6cmNXlxUA+iTjHNtzgqVTKeu7Y7jWoGmjvuUnNHCUEKEpFS2r7vCCMmzKNGXGXGDUulWmzZdh6qXTWGV4cksePAMe6emkGumjtKiFGQiJTCoWM5jJiQxt7Dxxk3PIXGtYo+zbc4usXX5g/9E/luZTYvz/oxKPsQKSkFiUgJ5eY57p6SQebmvbx8XU86N60V1P0N7tWCK3s248Wvf+TblbpZm4QOBYlICT31xXJmLt3GQ5clclGnRkHfn5nxxC+70r5hDe6esoBNew4HfZ8iRaEgESmByXPW8+Z3qxl2VktuPCeh3PZbJSaK14YmcTzXMUrNHSVEKEhEium7ldk89PESftahAQ9dllju+29doLnjE58uLff9i5xIQSJSDCu37WfUpPm0a1idVwYnER3lz1uoX9cmjDg3gQn/XqfmjuI7BYlIEWXvP8qNf5lHXEwUbw9PpXoZT/MtrjF9O5LiNXf8cZuaO4p/FCQiRXDkeC6/eieNnQePMm5YSkjcwbByVCVeGZxE1ZgoRk5Sc0fxj4JE5DTy8hy/mbaQhRv38OKgnnSLr+13ST9pXCuOlwYFmjuOUXNH8YmCROQ0npm5gk8Xb+GBvp3o3bmx3+X8l7Pb1uc3l3bgbws3886/1dxRyp+CROQUpqVt4NV/rGLwGS24+bzym+ZbXCMvaMNFHRvy+KdLma/mjlLOFCQiJzF71Q4e+HAx57WrzyP9O2NWNo0YgyHQ3LEHjWrGMVrNHaWcKUhECpG1/QC3vZtO6wbVGDskico+TfMtjlpVK/PakGR2HDjGXVMWqLmjlJvQf3eIlLOdB45y0/h5xERXYtywVGrGVfa7pCLrGl+LP/bvzPc/7uClr9XcUcqHgkSkgCPHc7n13XS27TvCWzek0LxuVb9LKrbrejXnyqRmvDTrR/6xYrvf5UgEUJCIeJxz3PfBItLW7eb5gT3o2aKO3yWViJnxxBVd6dCoBndPzVBzRwk6BYmI5/m//8jHGZu5t08H+nVt4nc5pVIlJopXhySRk+u4fdJ8jubk+l2SVGAKEhHgowUbeenrH7k2JZ6RF7Txu5wy0bpBdZ65phsLN+zhiU+X+V2OVGAKEol4c9fs4r7pizmrdT0ev6JrSE/zLa4+XZpw87kJvPPvdXycscnvcqSCUpBIRFuz4yC3vJtGfN0qvD40mZjoiveWuK9vR1Jb1WHMB4vV3FGCouK9a0SKaM+hY9w0fh6VzPjL8FRqVQ2fab7Fkd/csVpsFLdNTOeAmjtKGQtqkJhZHzNbYWZZZjamkPWxZjbVWz/HzFoVWHe/t3yFmfU+YbsoM1tgZp8Es36puI7l5HHru+ls2n2YN69PpmW9an6XFFSNasbx0nU9WbPjIGM+WKTmjlKmghYkZhYFjAX6AonAdWZ24u3kRgC7nXNtgeeBp7xtE4FBQGegD/Cq93z57gJ09VBKxDnHmA8XMWfNLp6+phsprer6XVK5OLtNoLnjJ4u2MGH2Wr/LkQokmJ9IegFZzrnVzrljwBRgwAljBgATvMfTgYsscKVzADDFOXfUObcGyPKeDzOLB34B/G8Qa5cKbOw3WXw4fxO/vrg9A3o087uccpXf3PGJz5apuaOUmWAGSTNgQ4HvN3rLCh3jnMsB9gL1TrPtC8C9QN6pdm5mt5hZmpmlZWdnl/QYpIL528LNPDNzJVf2bMadF7X1u5xyl9/csXGtOEZNms/OA0f9LkkqgLC62G5mlwHbnXPppxvrnHvTOZfinEtp0KBBOVQnoS593S5+8/5CerWqy5+uqljTfIsjv7njzoPHuHtqhpo7SqkFM0g2Ac0LfB/vLSt0jJlFA7WAnafY9hygv5mtJXCq7EIzmxiM4qViWb/zELe8k07TWnG8cX0ysdFRp9+oAuvSrBaPeM0dX1RzRymlYAbJPKCdmSWYWQyBi+czThgzAxjmPb4amOUC00lmAIO8WV0JQDtgrnPufudcvHOulfd8s5xzQ4N4DFIB7D18nBvHzyXXOd4enkqdajF+lxQSBqU256qkeF5Wc0cppaAFiXfNYzTwJYEZVtOcc5lm9qiZ9feGjQPqmVkWcA8wxts2E5gGLAW+AEY559QsSIrteG4et09KZ/2uQ7w+NJnWDar7XVLIMDMev6LLT80dN+4+5HdJEqYsEuaTp6SkuLS0NL/LkHLmnGPMB4uZmraBZ6/pzlXJ8X6XFJLW7DhI/5f/SesG1Zh221kRf9pPAsws3TmXUpSxYXWxXaQ43vhuNVPTNnDHhW0VIqeQUL8aT1/TjYUb9/L4J/rzLCk+BYlUSJ8v3sKTny/n8u5NueeS9n6XE/L6dGnCr85L4N0f1NxRik9BIhVOxoY93D01g6QWtXn66m4RO823uO7t85/mjivV3FGKQUEiFcrG3Ye4eUIaDWvG8tYNKcRV1vn+ovpPc8doNXeUYlGQSIWx78hxRoxP42hOLn8Znkq96rF+lxR2GtWM4+XrerJ2x0HuU3NHKSIFiVQIObl5jJ68gFXZB3h9aDJtG9bwu6SwdVabevy2dwc+XbSF8WruKEWgIJGw55zjDzMy+W5lNk/8sgvntK3vd0lh77bz23Bxp4Y88eky0tepuaOcmoJEwt64f65h0pz13HZBGwamtvC7nAqhUiXj2Wt60KR2HKMnq7mjnJqCRMLazMytPPHZMvp2acy9vTv4XU6FUrC5411T1NxRTk5BImFryaa93DUlg27xtXnu2h5UqqRpvmWtS7NaPNq/M//M2sGLf1/pdzkSohQkEpa27D3MiAnzqFsthrduSKZKjKb5BsvA1OZcnRzPS7Oy+EbNHaUQChIJOweO5nDT+DQOHs3l7eGpNKwR53dJFZqZ8diALnRsXINfq7mjFEJBImElN89x53sLWLltP2OHJNGhsab5locqMVG8PjSZ3FzH7ZPmczRHzbjlPxQkElYe+2Qps5Zv54/9O3NBe935sjy1ql+Np6/pzqKNe3nsk6V+lyMhREEiYWPC7LWMn72WEecmcP2ZLf0uJyL16dKYW85vzcQf1vPXBWruKAEKEgkL3yzfziN/y+TiTo14oF8nv8uJaPf27kCvVnW5/0M1d5QABYmEvKWb9zF68nw6NanJS9f1IErTfH0VHVWJVwb3VHNH+YmCRELatn1HGDFhHjXiKjNuWCpVY6L9LkmAhgWbO05Xc8dIpyCRkHXoWA43T0hj7+HjjBueQuNamuYbSs5qU4/f9e7Ip4u38Jd/rfW7HPGRgkRCUm6e4+4pGWRu3ssrg3vSuWktv0uSQtx2QWsu7tSI//lsGenrdvldjvhEQSIh6cnPlzFz6TYeuiyRCzs28rscOQkz49lru9O0dhVGTVrADjV3jEgKEgk5k+as463v1zDsrJbceE6C3+XIadSqUplXhySx69Ax7pqyQM0dI5CCRELKdyuzefjjTH7eoQEPXZbodzlSRF2a1eKxAZ35V9ZOXlBzx4ijIJGQsWLrfkZNmk+7htV5eXAS0VH68QwnA1NbcE1yPC/PyuKb5WruGEn0TpWQkL3/KDeNn0eVmCjeHp5K9VhN8w1Hj13RhU5NanL31Aw27FJzx0ihIBHfHTmey6/eSWPXwWOMG5ZK09pV/C5JSiiuchSvDUkiL88xarKaO0YKBYn4Ki/Pcc+0DBZu3MMLg3rQNV7TfMNdq/rVeObaQHPHR/+m5o6RQEEivnpm5go+W7yVB/p2onfnxn6XI2Wkd+fG3Hp+aybNWc9HCzb6XY4EmYJEfDMtbQOv/mMVg89owc3naZpvRfO73h3olRBo7rhiq5o7VmQKEvHF7KwdPPDhYs5rV59H+nfGTI0YK5roqEq8cl1PqsdWZuTEdPYfOe53SRIkQQ0SM+tjZivMLMvMxhSyPtbMpnrr55hZqwLr7veWrzCz3t6yODOba2YLzSzTzB4JZv0SHFnbD3DbxHRaN6jG2CFJVNY03wqrYc04Xhnck3W7DnHfB2ruWFEF7R1sZlHAWKAvkAhcZ2Yn/oXZCGC3c64t8DzwlLdtIjAI6Az0AV71nu8ocKFzrjvQA+hjZmcG6xik7O08EJjmGxNdiXHDUqkZV9nvkiTIzmxdj9/17sBni7fytpo7VkjB/KdgLyDLObfaOXcMmAIMOGHMAGCC93g6cJEFznEMAKY4544659YAWUAvF3DAG1/Z+9I/ccLEkeO53PJuOtv2HeGtG1JoXreq3yVJObn1/NZcktiIP322jLS1au5Y0QQzSJoBGwp8v9FbVugY51wOsBeod6ptzSzKzDKA7cBXzrk5he3czG4xszQzS8vOzi6Dw5HScM5x7/RFpK/bzfMDe9CzRR2/S5JyZGY8c013mtWpwqjJ89XcsYIJu5PTzrlc51wPIB7oZWZdTjLuTedcinMupUGDBuVbpPyX5//+IzMWbubePh3o17WJ3+WID/KbO+45dFzNHSuYYAbJJqB5ge/jvWWFjjGzaKAWsLMo2zrn9gDfELiGIiHsw/kbeenrH7k2JZ6RF7TxuxzxUeemtXhsQBf+lbWT579Sc8eKIphBMg9oZ2YJZhZD4OL5jBPGzACGeY+vBma5wLSOGcAgb1ZXAtAOmGtmDcysNoCZVQEuAZYH8RiklOas3sl9HyzirNb1ePyKrprmK1yb2pxrU+J55ZssZi3f5nc5UgaCFiTeNY/RwJfAMmCacy7TzB41s/7esHFAPTPLAu4BxnjbZgLTgKXAF8Ao51wu0AT4xswWEQiqr5xznwTrGKR01uw4yK0T02lRtyqvD00mJjrszqRKkDw6oAuJTWry66kL1dyxArBImNedkpLi0tLS/C4jouw+eIwrX5vN3sPH+ej2s2lZr5rfJUmIWbfzIJe9/E9a1avG+7edRVzlKL9LkgLMLN05l1KUsfonopS5Yzl53DoxnU27D/Pm9ckKESlUy3rVePaa7izetJdHP1Fzx3CmIJEy5ZxjzIeLmLtmF09f042UVnX9LklC2KWdG3PrBa2ZPGc9H85Xc8dwpSCRMvXKrCw+nL+Jey5pz4AeJ/7ZkMh/+92lHTgjoS4PfLSY5Vv3+V2OlICCRMrMjIWbefarlVzZsxl3XNjW73IkTERHVeLlwT2pEVeZkRPnq7ljGFKQSJlIX7eL376/kF4JdfnTVZrmK8XTsEYcr1zXk/W7DnHvdDV3DDcKEim19TsP8at30mlaK443hiYTG63ZN1J8Z7Sux729O/D5kq2M++cav8uRYlCQSKnsPXScG8fPJc853h6eSp1qMX6XJGHslvNbc2liI578fLmaO4YRBYmU2PHcPEZOSmf9rkO8MTSZ1g2q+12ShDkz42k1dww7ChIpEeccv/9oCbNX7eTJK7txRut6fpckFUStKpV5bUgyew4d58731NwxHChIpERe/3Y1U9M2cMeFbbkqOd7vcqSCSWxak8eu6MLsVTt57qsVfpcjp6EgkWL7bPEWnvpiOZd3b8o9l7T3uxypoK5Nac7AlOaM/WYVXy9Tc8dQpiCRYsnYsIdfT80gqUVtnr66m6b5SlA9MqCz19wxQ80dQ5iCRIps4+5D3DwhjYY1Y3nrhhQ12ZOgi6scxetDk3HAyEnpHDme63dJUojTBol3a1vd8yPC7TtynJvGz+NoTi5/GZ5KveqxfpckEaJFvao8d20PlmzaxyN/U3PHUHTaIPHuA7LCzFqUQz0SgnJy8xg9eQGrsw/y+tBk2jas4XdJEmEuSWzEbRe04b256/kgXc0dQ010EcfVATLNbC5wMH+hc67/yTeRisA5xx9mZPLdymyeuqor57St73dJEqF+e2l7Mjbs5sG/LqZzs5p0bFzT75LEU9QgeSioVUjIGvfPNUyas57bLmjDwFR9KBX/REdV4qXrenLZS/9k5MT5fDz6HGrGVfa7LKGIF9udc98W9hXs4sRfMzO38sRny+jbpTH39u7gdzkigeaOg5MCzR3fV3PHUHHKIDGz/Wa2r5Cv/WamGwdUYIs37uWuKRl0i6/Nc9f2oFIlTfOV0NAroS739enAF5lq7hgqTnlqyzmnq6oRaMvew4yYMI+61WJ464ZkqsRomq+Ell+d15r0dbv50+fL6d68Nqm6E6ev9Hck8v8cOJrDTePTOHQsl7eHp9KwRpzfJYn8l/zmjs3rVGHUpPlk71dzRz8pSOQnObl53PneAlZu28/YIUl0aKwPpBK6asZV5tUhyew9HGjumJOb53dJEUtBIj95/NNlzFq+nUf6d+aC9g38LkfktBKb1uTxK7rw79U7ee6rlX6XE7EUJALAhNlrGT97LTefm8DQM1v6XY5IkV2T0pxBqc159R+r+PtSNXf0g4JEmLV8G4/8LZOLOzXi/n6d/C5HpNj+2L8znZvW5J5pGazfqeaO5U1BEuGWbt7HHZMXkNi0Ji9d14MoTfOVMBRXOYrXhiQDcPtkNXcsbwqSCLZt3xFGTJhHjbjKjBuWStWYojY6EAk9/7+5Y6bf5UQUBUmEOnQshxET5rH38HHGDU+hUU1N85Xwd3FiI0b+rA3vzd3AdDV3LDcKkgiUm+e4a0oGSzfv45XBPenctJbfJYmUmd9c0p6zWtfjwY8Ws2yLGnCUBwVJBHry82V8tXQbD1+WyIUdG/ldjkiZym/uWKtKZUZOTGffkeN+l1ThKUgizKQ563jr+zUMO6slw89J8LsckaBoUCOWVwYnsWH3YTV3LAdBDRIz62NmK8wsy8zGFLI+1symeuvnmFmrAuvu95avMLPe3rLmZvaNmS01s0wzuyuY9Vc0363M5uGPM/l5hwY8dFmi3+WIBFWvhLqM6dORLzK38r/fq7ljMAUtSMwsChgL9AUSgevM7MTfXiOA3c65tsDzwFPetonAIKAz0Ad41Xu+HOA3zrlE4ExgVCHPKYVYsXU/oybNp13D6rw8OInoKH0YlYrv5vMS6NO5MU9+sZy5a3b5XU6FFczfJr2ALOfcaufcMWAKMOCEMQOACd7j6cBFZmbe8inOuaPOuTVAFtDLObfFOTcfwDm3H1gGNAviMVQI2fuPctP4eVSJieLt4alUj9U0X4kMZsafr+lG8zpVGD15Ptv3H/G7pAopmEHSDNhQ4PuN/Pcv/Z/GOOdygL1AvaJs650G6wnMKWznZnaLmaWZWVp2dnaJDyLcHTmey83vpLHr4DHGDUulae0qfpckUq5qxlXmtaHJ7Dui5o7BEpbnN8ysOvABcLdzrtD5fc65N51zKc65lAYNIrMBYV6e455pGSzauIcXBvWga7ym+Upk6tSkJo9f0ZUfVu/iWTV3LHPBDJJNQPMC38d7ywodY2bRQC1g56m2NbPKBEJkknPuw6BUXkE8PXMFny3eygN9O9G7c2O/yxHx1dXJ8VzXqzmv/WMVX6m5Y5kKZpDMA9qZWYKZxRC4eD7jhDEzgGHe46uBWS4wT28GMMib1ZUAtAPmetdPxgHLnHPPBbH2sDdt3gZe+8cqBp/RgpvP0zRfEYA/XN6ZLs3U3LGsBS1IvGseo4EvCVwUn+acyzSzR82svzdsHFDPzLKAe4Ax3raZwDRgKfAFMMo5lwucA1wPXGhmGd5Xv2AdQ7ianbWDBz5azHnt6vNI/84E8ldE8ps7GjBykpo7lhWLhD/USUlJcWlpaX6XUS6yth/gylf/ReNacUwfeTY14yr7XZJIyPl62TZGTEhjUGpznryqm9/lhCQzS3fOpRRlbFhebJfC7TwQmOYbE12JccNSFSIiJ3FRp0bc/rM2TJm3gffTNpx+AzklBUkFceR4Lre8m862fUd464YUmtet6ndJIiHtHq+54+//uoSlm9XcsTQUJBWAc457py8ifd1unh/Yg54t6vhdkkjIK9jc8fZJau5YGgqSCuD5v//IjIWbubdPB/p1beJ3OSJho0GNWMYOCTR3/O20hWruWEIKkjD34fyNvPT1j1ybEs/IC9r4XY5I2EltVZf7+3Zk5tJtvPX9ar/LCUsKkjA2Z/VO7vtgEWe1rsfjV3TVNF+REhpxbgJ9uzTmqS9WMGf1Tr/LCTsKkjC1ZsdBbp2YTou6VXl9aDIx0fpfKVJSZsafr+5Gi7pVGf3eAjV3LCb99glDuw8e46bx86hkxl+G96JWVU3zFSmtGnGVeW1oEvuPHOeOyWruWBwKkjBzNCeXWyems2nPYd66IZkW9TTNV6SsdGxckyeu6MqcNbt4ZqaaOxaVgiSMOOe4/8PFzF2zi6ev7kZyy7p+lyRS4VyVHM91vVrw+rdq7lhUCpIw8sqsLD6cv4l7LmnPgB66n5dIsPzh8sQPA82XAAARTElEQVSfmjuu23nQ73JCnoIkTHycsYlnv1rJlT2bcceFbf0uR6RCy2/uWMmMkRPnq7njaShIwkD6ul38bvoieiXU5U9XaZqvSHloXrcqzw/sztIt+/jDx5l+lxPSFCQhbv3OQ/zqnXSa1a7CG0OTiY2O8rskkYhxYcdGjPp5G6ambWCamjuelIIkhO09dJwbx88lzzneHp5KnWoxfpckEnHuuaQDZ7epx0N/XULm5r1+lxOSFCQh6lhOHiMnpbN+1yHeGJpMQv1qfpckEpGiKhkvXdeT2lUrc/uk+ew9rOaOJ1KQhCDnHA/9dQmzV+3kySu7cUbren6XJBLR6lePZezgJDbtPszv3ldzxxMpSELQ69+uZmraBu68sC1XJcf7XY6IACmt6jLGa+745ndq7liQgiTEfLZ4C099sZzLuzfl15e097scESlgxLkJ9OvamD9/qeaOBSlIQkjGhj38emoGyS3r8PTV3TTNVyTEmBlPXdWNlvnNHfepuSMoSELGxt2HuHlCGg1rxvLm9cnEVdY0X5FQFGjumMyBIzmMfk/NHUFBEhL2HTnOTePncTQnl78MT6Ve9Vi/SxKRU+jQuAb/c2WXQN+7mSv8Lsd3ChKf5eTmMWrSfFZnH+SNocm0bVjD75JEpAh+2TOeIWe04I1vVzMzc6vf5fhKQeIj5xx/mJHJ9z/u4IlfduHstvX9LklEiuHhyxPpFl+L37y/MKKbOypIfDTun2uYNGc9t13QhoGpLfwuR0SKKTY6irGDk6hkxm0R3NxRQeKTmZlbeeKzZfTr2ph7e3fwuxwRKaHmdavywsAeLNuyj4c/XuJ3Ob5QkPhg8ca93DUlg27xtXnu2h5UqqRpviLh7OcdG3LHhW2ZlraRafMir7mjgqScbd5zmBET5lG3Wgxv3aBpviIVxd0Xt+fctvV56OPIa+6oIClHB47mMGJCGoeP5fL28FQa1ojzuyQRKSNRlYwXB/WgTtUYRk6MrOaOCpJykpObx53vLWDltv28MiSJDo01zVekoqlXPZaxQ5LYvOcwv42g5o4KknLy+KfLmLV8O4/078wF7Rv4XY6IBElyyzo80K8TXy3dxhsR0twxqEFiZn3MbIWZZZnZmELWx5rZVG/9HDNrVWDd/d7yFWbWu8Dyt81su5mFzfSI8f9aw/jZa7n53ASGntnS73JEJMhuPKcVv+jWhD9/sZwfIqC5Y9CCxMyigLFAXyARuM7MEk8YNgLY7ZxrCzwPPOVtmwgMAjoDfYBXvecDGO8tCwuzlm/j0U+WckliI+7v18nvckSkHOQ3d2xVvxqjJ1f85o7B/ETSC8hyzq12zh0DpgADThgzAJjgPZ4OXGSBlrcDgCnOuaPOuTVAlvd8OOe+A3YFse4ys3TzPu6YvIDEpjV5cVAPojTNVyRiVI+N5vWhyRw8WvGbOwYzSJoBBSdUb/SWFTrGOZcD7AXqFXHbUzKzW8wszczSsrOzi1l66W3bd4QRE+ZRs0plxg1LpWpMdLnXICL+at+oBn+6smugueOXFbe5Y4W92O6ce9M5l+KcS2nQoHwvbh86lsOICfPYd/g444al0qimpvmKRKorejZj6JkteOO71XxZQZs7BjNINgHNC3wf7y0rdIyZRQO1gJ1F3DYk5eY57pqSwdLN+3h5cE8Sm9b0uyQR8dlDlyXSPb4Wv522kLU7Kl5zx2AGyTygnZklmFkMgYvnM04YMwMY5j2+GpjlAhOvZwCDvFldCUA7YG4Qay0zT36+jK+WbuPhyxK5sGMjv8sRkRAQGx3F2CFJREUZIydVvOaOQQsS75rHaOBLYBkwzTmXaWaPmll/b9g4oJ6ZZQH3AGO8bTOBacBS4AtglHMuF8DM3gP+DXQws41mNiJYx1Bck+as463v1zD87FYMPyfB73JEJITE16nK8wN7sHzrPh76a9j89UKRWCT85WVKSopLS0sL6j6+XZnNTePncX67+rx1QwrRURX28pOIlMJzM1fw0qwsnrqqa0jfPsLM0p1zKUUZq992ZWDF1v2MnjSf9o1q8PLgJIWIiJzUXRe357x29Xno40yWbKoYzR31G6+Usvcf5abx86gSE8W4YSlUj9U0XxE5uahKxgsDe1CvWgy3T6oYzR0VJKVw+FguN7+Txq6Dxxg3LJWmtav4XZKIhIF61WN5ZXCgueNvpi0kLy+8LzEoSEooL8/xm/czWLRxDy8O6kHX+Fp+lyQiYSS5ZR0e/EUn/r4s/Js7KkhK6OmZK/hs8VYe7NeJSzs39rscEQlDw88ONHd8+svl/HtV+DZ3VJCUwLR5G3jtH6sYfEYLRpyrab4iUjL5zR0T6lfjjvfCt7mjgqSYZmft4IGPFnNeu/o80r8zgR6TIiIlUz02mtfymztOXsDxMGzuqCAphqztB7htYjqtG1Rj7JAkKmuar4iUgfaNavDkVV2ZuzY8mzvqN2ER7TxwlBvHzyUmuhLjhqVSM66y3yWJSAUyoEczrj+zJW9+t5ovloRXc0cFSREcOZ7LLe+ms33fUd66IYXmdav6XZKIVEC/v6wT3ZvX5nfvL2RNGDV3VJCchnOOe6cvIn3dbp4f2IOeLer4XZKIVFCx0VGMHdwz0NxxYjqHj4VHc0cFyWk8/9VKZizczL19OtCvaxO/yxGRCi6+TlVeGNiDFdv289DHSwiHfogKklP4IH0jL83KYmBKc0Ze0MbvckQkQvysQ0PuuLAd09M3MnXehtNv4DMFyUnsPniMhz9ewtlt6vHYFV00zVdEytVdF7XjvHb1eXhG6Dd3VJCcRJ1qMbwzohevDUkmJlovk4iUr6hKxouDelKvWgwjJ6Wz91DoNnfUb8hTSG5Zl1pVNc1XRPxRt1oMY4cksXXvEX7zfkbINndUkIiIhLCkFnV4sF8n/r5sO69/t8rvcgqlIBERCXHDzm7F5d2b8syXK5i9aoff5fwXBYmISIgzM568sisJ9atx53sL2BZizR0VJCIiYaBabDSvD03m0LFcRk+eH1LNHRUkIiJhol2jGvzpyq7MW7ubP3+x3O9yfqIgEREJIwN6NOOGs1ry1vdr+GLJFr/LARQkIiJh58Ff5Dd3XBQSzR0VJCIiYSY2OopXhyQRHSLNHRUkIiJhqFntKrwwqCcrtu3n93/1t7mjgkREJExd0L4Bd17Yjg/mb2SKj80dFSQiImHsTq+54x98bO6oIBERCWP5zR3rV4vhton+NHdUkIiIhLn85o7b9h3hnmnl39xRQSIiUgH0bFGH3/8ika+Xb+e1b8u3uWNQg8TM+pjZCjPLMrMxhayPNbOp3vo5ZtaqwLr7veUrzKx3UZ9TRCRS3XBWSy7v3pRnZ5Zvc8egBYmZRQFjgb5AInCdmSWeMGwEsNs51xZ4HnjK2zYRGAR0BvoAr5pZVBGfU0QkIuU3d2zdoDp3vreArXvLp7ljMD+R9AKynHOrnXPHgCnAgBPGDAAmeI+nAxdZ4J62A4Apzrmjzrk1QJb3fEV5ThGRiBVo7phUrs0dgxkkzYCCE5s3essKHeOcywH2AvVOsW1RnhMAM7vFzNLMLC07O7sUhyEiEl7aNqzBk1d1o12jGuSVwx8qRgd9Dz5xzr0JvAmQkpISmvenFBEJkv7dm9K/e9Ny2VcwP5FsApoX+D7eW1boGDOLBmoBO0+xbVGeU0REylEwg2Qe0M7MEswshsDF8xknjJkBDPMeXw3McoGGMTOAQd6srgSgHTC3iM8pIiLlKGintpxzOWY2GvgSiALeds5lmtmjQJpzbgYwDnjXzLKAXQSCAW/cNGApkAOMcs7lAhT2nME6BhEROT3zs2NkeUlJSXFpaWl+lyEiEjbMLN05l1KUsfrLdhERKRUFiYiIlIqCRERESkVBIiIipRIRF9vNLBtYV8LN6wPl1/0sNOk10GsQ6ccPkfcatHTONSjKwIgIktIws7SizlyoqPQa6DWI9OMHvQanolNbIiJSKgoSEREpFQXJ6b3pdwEhQK+BXoNIP37Qa3BSukYiIiKlok8kIiJSKgoSEREpFQXJSZhZHzNbYWZZZjbG73qCyczWmtliM8swszRvWV0z+8rMfvT+W8dbbmb2kve6LDKzJH+rLxkze9vMtpvZkgLLin3MZjbMG/+jmQ0rbF+h6iSvwR/NbJP3s5BhZv0KrLvfew1WmFnvAsvD9r1iZs3N7BszW2pmmWZ2l7c8on4WSs05p68Tvgi0qF8FtAZigIVAot91BfF41wL1T1j2Z2CM93gM8JT3uB/wOWDAmcAcv+sv4TGfDyQBS0p6zEBdYLX33zre4zp+H1spX4M/Ar8tZGyi9z6IBRK890dUuL9XgCZAkve4BrDSO9aI+lko7Zc+kRSuF5DlnFvtnDsGTAEG+FxTeRsATPAeTwCuKLD8HRfwA1DbzJr4UWBpOOe+I3APnIKKe8y9ga+cc7ucc7uBr4A+wa++bJzkNTiZAcAU59xR59waIIvA+ySs3yvOuS3Oufne4/3AMqAZEfazUFoKksI1AzYU+H6jt6yicsBMM0s3s1u8ZY2cc1u8x1uBRt7jivzaFPeYK+prMdo7bfN2/ikdIuA1MLNWQE9gDvpZKBYFiQCc65xLAvoCo8zs/IIrXeCze0TNE4/EY/a8BrQBegBbgGf9Lad8mFl14APgbufcvoLrIvhnocgUJIXbBDQv8H28t6xCcs5t8v67HfiIwOmKbfmnrLz/bveGV+TXprjHXOFeC+fcNudcrnMuD3iLwM8CVODXwMwqEwiRSc65D73FEf+zUBwKksLNA9qZWYKZxRC4l/wMn2sKCjOrZmY18h8DlwJLCBxv/syTYcDH3uMZwA3e7JUzgb0FTgGEu+Ie85fApWZWxzsFdKm3LGydcL3rlwR+FiDwGgwys1gzSwDaAXMJ8/eKmRkwDljmnHuuwKqI/1koFr+v9ofqF4HZGSsJzEh50O96gnicrQnMtFkIZOYfK1AP+Br4Efg7UNdbbsBY73VZDKT4fQwlPO73CJy6OU7gfPaIkhwzcBOBC89ZwI1+H1cZvAbvese4iMAvzSYFxj/ovQYrgL4FloftewU4l8Bpq0VAhvfVL9J+Fkr7pRYpIiJSKjq1JSIipaIgERGRUlGQiIhIqShIRESkVBQkIiJSKgoSkdMws9nef1uZ2eAyfu4HCtuXSDjR9F+RIjKznxHojHtZMbaJds7lnGL9Aedc9bKoT8Qv+kQichpmdsB7+CRwnnefjl+bWZSZPW1m87wmh7d6439mZt+b2Qxgqbfsr15TzMz8xphm9iRQxXu+SQX35f3l9NNmtsQC94oZWOC5/2Fm081suZlN8v46GzN70ruvxiIze6Y8XyOJbNF+FyASRsZQ4BOJFwh7nXOpZhYL/MvMZnpjk4AuLtByHeAm59wuM6sCzDOzD5xzY8xstHOuRyH7upJA48TuQH1vm++8dT2BzsBm4F/AOWa2jEBLk47OOWdmtcv86EVOQp9IREruUgJ9lzIItB6vR6AHFcDcAiECcKeZLQR+INDcrx2ndi7wngs0UNwGfAukFnjujS7QWDEDaAXsBY4A48zsSuBQqY9OpIgUJCIlZ8Adzrke3leCcy7/E8nBnwYFrq1cDJzlnOsOLADiSrHfowUe5wL512F6AdOBy4AvSvH8IsWiIBEpuv0Ebsea70tgpNeGHDNr73VQPlEtYLdz7pCZdSRwi9Z8x/O3P8H3wEDvOkwDArfFnXuywrz7adRyzn0G/JrAKTGRcqFrJCJFtwjI9U5RjQdeJHBaab53wTub/9yStaAvgNu86xgrCJzeyvcmsMjM5jvnhhRY/hFwFoGuzA641zm31QuiwtQAPjazOAKflO4p2SGKFJ+m/4qISKno1JaIiJSKgkREREpFQSIiIqWiIBERkVJRkIiISKkoSEREpFQUJCIiUir/B1JB2WysKRZiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.plot('lr')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd4VAXWx/HvSYVAqAm9d0JLA3tXRCwoKjW7q+u7rgj2BvbFxd6luO7qukoAUURRsZe1YEsj1EDonVBCh7Tz/jE37iwLZAiZ3MzM+TzPPJm5Ze7vXkLO3DLniqpijDHGHEuY2wGMMcbUfFYsjDHGVMiKhTHGmApZsTDGGFMhKxbGGGMqZMXCGGNMhaxYGBPgROR1EflrFb3XNSLyfVVPawKfFQtTbURktYgUiUjcYcOzRURFpJ07yY6sKv8I+7i8ViIyS0S2icguEVkoItdU1/KNORYrFqa6rQKGl78QkV5AjHtxapQ3gXVAW6Ax8Dtgi6uJjHFYsTDV7U3g916v/wC84T2BiNQXkTdEpEBE1ojI/SIS5oy7RkR+EJHnRKRQRFaKyKnO8HUislVE/uD1XtEi8rSIrBWRLSLysojUdsadLSLrReQOZ75NInKtM+56YCRwt4jsFZEPnOEqIp283v+3vQ+v97vb6/0uF5GBIrJMRHaIyL3H2DZ9gddVdZ+qlqhqtqp+7LWs00VknrPe6w7b62goIh+JyB4R+VlEOnrN101EPneWnyciQ7zGNRaROSKyW0R+Abzna+esb4TXsG9E5P+OFP5YyzGBz4qFqW4/AfVEpLuIhAPDgKmHTfMSUB/oAJyFp7hc6zX+JCAXz6fvacAMPH9oOwFpwEQRqetM+zjQBUh0xrcEHvR6r2bOsloC1wGTRKShqr4CpANPqmpdVb3Ux/VrBtTyWs7fnUwpwBnAAyLS/ijz/uQsf5iItPEeISJtgY+dbRPvrE+O1yTDgL8ADYF8YIIzXx3gczzbqYkz3WQRSXDmmwQcBJoDf3Qex82H5ZgAZ8XCuKF87+ICYAmwoXyEVwEZp6p7VHU18AyeQzLlVqnqP1W1FHgLaA2MV9VDqvoZUAR0EhEBrgduU9UdqroHeNR5/3LFzrzFqjoX2At0PYF1KwYmqGoxniIWB7zgrMsiYDHQ5yjzXg18BzwArBKRHBHp64wbAXyhqtOdrNtV1btYzFbVX1S1BE+RS3SGXwKsdrZXiapmA7OAq51tfSXwoLM3sxD4VyXX+6jLqeT7mRomouJJjKlybwLfAu057BAUnj+ukcAar2Fr8HxSL+d9HP8AgKoePqwunk/gMUCmp24AIEC417TbnT+w5fY781bWdqeI/ZbtCHmP+P6quhMYC4x1LgJ4GnhPRFrhKYgrjrHczV7PvdehLXCSiBR6jY/A828Q7zxf5zXOe7sfj2MtxwQBKxam2qnqGhFZBQzEc+jH2zY8n87b4vkUDtAGr72P47ANzx/nHqpamfmP1JJ5P/99Qr4ZsL4S733sBatuE5Gn8ZzTaYTnD3q/SrzVOuDfqnrB4SOcPYsSPIVoqTPY+/DXPudnDLDbed7seJdjgoMdhjJuuQ44V1X3eQ90PpXPBCaISKxzrP52/ve8RoVUtQzPOYPnRKQJgIi0FJELfXyLLXjOm3jLAUaISLiIDMBzTqVKiMgTItJTRCJEJBYYBeSr6nY8h5bOF5EhzvjGIpJ47HcE4EOgi4j8TkQinUdfEenubOt3gYdFJMY5v/DbxQGqWoCnSKc56/tHvE6A+7qcym8RU5NYsTCuUNUVqppxlNE34flUuxL4Hs9J09cquah78Jzw/UlEdgNf4Ps5iVeBBOfqo/ecYbcAlwKFeK6Weu9oM1dCDDDbee+VePauLgNQ1bV49sTuAHbgKVpHO/fxG+c8TX8852k24jlc9QQQ7UwyBs8hq83A68A/D3uLPwF3AduBHsC8Si7HBDixmx8ZY4ypiO1ZGGOMqZAVC2OMMRWyYmGMMaZCViyMMcZUKGi+ZxEXF6ft2rVzO4YxxgSUzMzMbaoaX9F0QVMs2rVrR0bG0a7ENMYYcyQi4tO39u0wlDHGmApZsTDGGFMhKxbGGGMqZMXCGGNMhaxYGGOMqZAVC2OMMRWyYmGMMaZCViyA7LU7KS4tczuGMcYct2/ytrJo4y6/Lyfki8WKgr1cOWUej3+8tOKJjTGmBvkhfxvX/PNXhv3tJ8rK/Hu7iZAvFh3j6/K7k9vy6vermLtgk9txjDHGJ5t3HeTm6dlER4TxwU2nExYmFc90AkK+WADcd3ECia0bcPc7uaws2Ot2HGOMOabi0jLGTMviQHEpH918Ou3i6vh9mVYsgKiIMCaNTCYyXBg1NYv9RSVuRzLGmKN6/OOlZKzZyeNX9qZTk9hqWaYVC0fLBrV5YVgSy7bu4b7ZC7HbzRpjaqKPcjfx6veruObUdlzWp0W1LdeKhZczu8Rz63ldmJ29gfSf17odxxhj/suKgr3c/c58kto04N6B3at12VYsDnPTuZ04q0s84z9YTO76QrfjGGMMAPuLShg1NZPoyHAmjUgmKqJ6/3xbsThMWJjw/NBE4mOjGTU1i537ityOZIwJcarKfbMXsnzrXl4YlkiLBrWrPYMViyNoWCeKSSOT2brnILfNzPH79cvGGHMs6T+vZXb2Bm49rwtndK7wpnZ+YcXiKBJbN+DBSxL4Jq+ASV/nux3HGBOi5q8rZPwHizm7azw3ndvJtRxWLI4h7eS2XJ7Ygme/WMZ3ywvcjmOMCTE79xVxY3oW8bHRPDck0e9fvDsWKxbHICI8OrgXnZvU5ZYZOWwsPOB2JGNMiCgrU26bmUPBnkNMHplMwzpRruaxYlGBmKgIpqSlcKi4lNHTsigqsYaDxhj/m/h1Pt/kFfDApQn0ad3A7ThWLHzRMb4uT17Vh+y1hTw6d4nbcYwxQe675QU898UyLk9sQdpJbdyOA1ix8NnFvZtz7WnteH3eaj6Yv9HtOMaYILWx8AC3zMihc5O6PDq4FyLunafwZsXiOIy7qDvJbRowdlYu+Vut4aAxpmoVlZQxeloWh4pLmZKWQkxUhNuRfmPF4jiUNxysFRnOqKmZ7DtkDQeNMVXn0blLyF5byFNX96FjfF234/wXvxYLERkgInkiki8iY48wvq2IfCkiuSLyjYi0Omx8PRFZLyIT/ZnzeDSvX5sXhyexomAv495dYA0HjTFVYs78jbw+bzV/PK09A3s1dzvO//BbsRCRcGAScBGQAAwXkYTDJnsaeENVewPjgccOG/8I8K2/MlbWaZ3iuP2CLsyZv5E3f1rjdhxjTIDL37qHsbNySWnbkHEDu7kd54j8uWfRD8hX1ZWqWgTMAAYdNk0C8JXz/Gvv8SKSAjQFPvNjxkq78exOnNutCY98uJjstTvdjmOMCVD7DpVww9QsajsNAiPDa+bZAX+magms83q93hnmbT4w2Hl+BRArIo1FJAx4BrjzWAsQketFJENEMgoKqvcb1mFhwrND+tC0Xi1Gp2exwxoOGmOOk6oy7t0FrCzYy4vDk2hWv5bbkY7K7RJ2J3CWiGQDZwEbgFLgRmCuqq4/1syq+oqqpqpqanx89TfXahATxeSRyWzbW8Stb+VQag0HjTHH4c2f1jBn/kbu6N+V0zrFuR3nmPxZLDYArb1et3KG/UZVN6rqYFVNAu5zhhUCpwBjRGQ1nvMavxeRx/2YtdJ6t2rAw5f14NtlBbz01XK34xhjAkTW2p088uFizuvWhFFndXQ7ToX8eRHvr0BnEWmPp0gMA0Z4TyAiccAOVS0DxgGvAajqSK9prgFSVfV/rqaqKYb3a03Gmh288OVyEls34OyuTdyOZIypwXbsK2JMehZN69XiWZcbBPrKb3sWqloCjAE+BZYAM1V1kYiMF5HLnMnOBvJEZBmek9kT/JXHn0SECZf3omvTWG59K4cN1nDQGHMUpWXKLTOy2ba3iCkjU6gfE+l2JJ9IsHxPIDU1VTMyMlzNsLJgL5dN/IGOTeoy888nEx0R7moeY0zN89zny3jhy+U8ekUvRtSAvk8ikqmqqRVN5/YJ7qDSIb4uT1/dm/nrCpnwkTUcNMb8t2/ytvLiV8sZnNyS4f1aVzxDDWLFoooN6Nmc/zu9PW/8uIb3czZUPIMxJiSs37mfW9/KoWvTWCZcXnMaBPrKioUf3HNRN/q2a8jYWQtYtmWP23GMMS47VFLK6PQsSkuVKWkp1I4KvEPUViz8IDI8jIkjkqkTHcENUzPZaw0HjQlpf/1wCfPX7+Kpq3vTPq6O23EqxYqFnzStV4uXhiexets+7pmVaw0HjQlR7+ds4M2f1vCnM9ozoGfNaxDoKysWfnRKx8bceWFXPsrdxOvzVrsdxxhTzZZt2cPYWQvo264hdw+omQ0CfWXFws9uOLMj53dvwoSPlpC5xhoOGhMq9h4q4YapmdSJjmBiDW4Q6KvATh8AwsKEZ65OpEWD2oyZlsX2vYfcjmSM8TNV5Z5Zuazeto+XhifRtF7NbRDoKysW1aB+TCSTRyazfV8Rt8ywhoPGBLt//rCaj3I3cdeF3TilY2O341QJKxbVpGfL+jwyqAff52/j+S+WuR3HGOMnmWt28OjcJZzfvSk3nNXB7ThVxopFNRratw1Xp7Tipa/y+XrpVrfjGGOq2La9hxidnk2LBrV5ZkifgPvi3bFYsahmj1zek+7N63HrWzms27Hf7TjGmCpS3iBwx/4iJo9Mpn7twGgQ6CsrFtWsVmQ4U0YmU1amjJ6WxaGSUrcjGWOqwPNfLOOH/O38dVBPeras73acKmfFwgXt4urwzJA+5K7fxfgPFrsdxxhzgr5auoWXvspnSGorhvQNrAaBvrJi4ZL+PZrx57M6kP7zWt7NOubdY40xNdi6Hfu57a35JDSvx/hBPd2O4zdWLFx0V/+unNS+EffOXsDSzbvdjmOMOU4Hi0u5MT2LMlWmpCVTKzLwGgT6yoqFiyLCw3hpRBKxtSIZNTWLPQeL3Y5kjDkO4z9czIINu3jm6j60bRyYDQJ9ZcXCZU1iazFxeBJrd+zn7nes4aAxgeLdrPVM+3ktfz6rA/17NHM7jt9ZsagBTurQmLsv7MrHCzfz6ver3I5jjKnA0s27uXf2Ak5q34i7+nd1O061sGJRQ1x/Zgf6JzTl8Y+XkrF6h9txjDFHsedgMaOmZlGvViQvjUgiIsAbBPoqNNYyAIgITw/pQ6uGtRk9LYuCPdZw0JiaRlW56+1c1u7Yz8QRyTSJDfwGgb6yYlGD1KsVyeSRKRTuL+bm6dmUlJa5HckY4+XV71fxyaLN3DOgK/3aN3I7TrWyYlHDJLSox18v78mPK7fz7OfWcNCYmuLX1Tt47OOlXNijKX86I3gaBPrKikUNdHVqa4b1bc3kb1bwxeItbscxJuQV7DnE6PQsWjeszVNXB1eDQF9ZsaihHr6sBz1a1OP2mTms3W4NB41xS0lpGTdPz2b3wWKmpKVQr1ZwNQj0lRWLGsrTcDAFgFHpmRwstoaDxrjh2c+X8ePK7fz18l50b17P7TiusWJRg7VpHMNzQxNZtHE3D89Z5HYcY0LO54u3MPmbFQzv15qrUlq5HcdVVixquPO6N+XGszsy49d1vJ2xzu04xoSMtdv3c/vMHHq2rMdDl/ZwO47rrFgEgNsv6MIpHRpz/3sLWbzRGg4a428Hi0sZlZ6JAFNGpgR1g0BfWbEIABHhYbw4PIn6tSO5MT2T3dZw0Bi/enjOIhZt3M1zQxNp3SjG7Tg1gl+LhYgMEJE8EckXkbFHGN9WRL4UkVwR+UZEWjnDE0XkRxFZ5Iwb6s+cgSA+NppJI5NZt/MAd86cbw0HjfGTtzPWMePXdYw+pyPndW/qdpwaw2/FQkTCgUnARUACMFxEEg6b7GngDVXtDYwHHnOG7wd+r6o9gAHA8yLSwF9ZA0Xfdo0Yd1E3Plu8hb9/t9LtOMYEncUbd3P/ews5tWNjbr8gNBoE+sqfexb9gHxVXamqRcAMYNBh0yQAXznPvy4fr6rLVHW583wjsBWI92PWgHHd6e0Z2KsZT3ySx88rt7sdx5igsetAMaPSM2kQE8mLw5MIDwu9L94diz+LRUvA+/Kd9c4wb/OBwc7zK4BYEWnsPYGI9AOigBV+yhlQRIQnruxN20YxjJmezdbdB92OZEzA8zQInM+GnQeYNCKZuLrRbkeqcdw+wX0ncJaIZANnARuA3759JiLNgTeBa1X1f7rqicj1IpIhIhkFBQXVldl1sbUimZyWzJ6DxYyxhoPGnLBXvl3JZ4u3MPaibqS2C60Ggb7yZ7HYALT2et3KGfYbVd2oqoNVNQm4zxlWCCAi9YCPgPtU9acjLUBVX1HVVFVNjY8PraNU3ZrV49ErevHLqh089Vme23GMCVg/r9zOk5/mMbBXM647vb3bcWosfxaLX4HOItJeRKKAYcAc7wlEJE5EyjOMA15zhkcBs/Gc/H7HjxkD2uDkVow4qQ1/+/dKPlu02e04xgScrbsPMmZ6Nm0bxfDElb1DskGgr/xWLFS1BBgDfAosAWaq6iIRGS8ilzmTnQ3kicgyoCkwwRk+BDgTuEZEcpxHor+yBrIHL0mgd6v63PH2fFZv2+d2HGMCRklpGWOmZ7P3YAlT0lKIDdEGgb6SYLlePzU1VTMyMtyO4Yp1O/ZzyUvf06JBbWbfeKp929QYHzw2dwl/+3Ylzw3twxVJodv3SUQyVTW1ouncPsFtqkDrRjE8PzSRJZt288B7C92OY0yN9+mizfzt25WMPKlNSBeK42HFIkic060JN53bibcz1/PWr2vdjmNMjbV62z7unDmf3q3q8+Clh39P2ByNFYsgcuv5XTi9UxwPvL+IhRt2uR3HmBrH0yAwi7AwYdKIZKIj7JCtr6xYBJHwMOGFYYk0ionixvQsdh2whoPGlFNV7n9vIUs37+b5YdYg8HhZsQgyjet6Gg5uLDzAHTNzKCsLjgsYjDlRb/26jncy13PTOZ04p2sTt+MEHCsWQSilbUPuu7g7XyzZysvfWpcUYxZu2MWDcxZxRuc4bjm/i9txApIViyB1zantuLh3c57+NI95K7a5HccY1+za72kQ2LhOFM8PTbQGgZVkxSJIlTccbB9Xh5unZ7PFGg6aEFRWptzxdg6bCg8ycUQyja1BYKVZsQhidaMjmJKWwr5DpYyZlkWxNRw0Ieblb1fwxZKt3Hdxd1LaNnQ7TkDzqViISKqIzBaRLOfOdQtEJNff4cyJ69I0lsev7MWvq3fy5CdL3Y5jTLWZt2IbT3+axyW9m3PNqe3cjhPwInycLh24C1gA2MfTADMosSWZa3by9+9WkdK2IQN6Nnc7kjF+tXnXQW6enk37uDrWILCK+FosClR1TsWTmZrqvou7M3/9Lu58O5cuTWPpEF/X7UjG+EVxaRljpmWxv6iU6X86mTrRvv6ZM8fi6zmLh0TkHyIyXEQGlz/8msxUqeiIcCaPTCYyXLgxPYsDRaUVz2RMAHri46VkrNnJY4N70blprNtxgoavxeJaIBEYAFzqPC7xVyjjHy0b1Ob5YUnkbdnDfe8tIFg6DhtT7uMFm/jH96v4/SltGZR4+F2czYnwdf+sr6p29WsSUy3O6hLPzed25oUvl5PathEjTmrjdiRjqsTKgr3c9U4ufVo34L6Lu7sdJ+j4umcxT0SsPWOQuPm8zpzZJZ6H5yxiwXprOGgC3/6iEkZNzSIyXJg80hoE+oOvxeJkIEdE8uzS2cAXHiY8PzSRuLpRjErPpHB/kduRjKk0VeX+2QtZtnUPLwxLomWD2m5HCkq+FosBQGegP/85X3Gpv0IZ/2tUJ4rJaSls2X2Q296yhoMmcE37ZS3vZm/gFmeP2fiHr8VCj/IwASyxdQMeuCSBr/MKmPxNvttxjDluuesL+cucxZzpnIsz/uPrCe6P8BQHAWoB7YE8oIefcplq8ruT25KxeifPfr6MpDYNOa1TnNuRjPFJ4f4iRk3NIq6up0FgmDUI9Cuf9ixUtZeq9nZ+dgb6AT/6N5qpDiLCY4N70SG+LjdPz2bzLms4aGq+sjLltrdy2LrnIJPTUmhUJ8rtSEGvUo0EVTULOKmKsxiX1ImO4OW0FA4WlzLaGg6aADD5m3y+zivgwUsSSGzdwO04IcGnw1AicrvXyzAgGdjol0TGFZ2a1OWJq3ozZlo2j81dajeyNzXW98u38cznyxiU2IK0k9u6HSdk+HrOwvs78yV4zmHMqvo4xk2X9G5BxuqdvPbDKpLbNuCS3i3cjmTMf9m06wA3z8imU3xdHhvcyxoEViNfi8ViVX3be4CIXA28fZTpTYC6d2B3ctcXcs87uXRrVo9OTazhoKkZikrKGJ2exaHiUqakpRATZQ0Cq5Ov5yzG+TjMBLioiDAmjUwmOjKcG9Mz2V9U4nYkYwB47OMlZK0t5ImretuHGBccs1iIyEUi8hLQUkRe9Hq8judwlAlCzevX5oVhiSzfupd737WGg8Z9H+Zu5J8/rOba09rZ4VGXVLRnsRHIAA4CmV6POcCF/o1m3HRG53huP78L7+VsZOrPa92OY0JY/ta93PNOLsltGjDuImsQ6JZjHvRT1fnAfBGZpqrF1ZTJ1BCjz+lE1tqdPPLBYnq1rG+XKJpqt+9QCaOmZhIdGc6kkclERVTqan9TBXzd8v1E5HMRWSYiK0VklYis9Gsy47qwMOG5oYnEx0YzOj2Lnfus4aCpPqrKvbMXkF+wlxeHJdG8vjUIdJOvxeJV4FngdKAvkOr8NEGuQUwUU9KSKdhziFut4aCpRlN/WsP7ORu5/fwunN7Z2tC4zddisUtVP1bVraq6vfxR0UwiMsBpa54vImOPML6tiHzptD3/RkRaeY37g4gsdx5/OI51MlWsd6sGPHhpAv9eVsBLX1nDQeN/OesKGf/hYs7pGs/oczq5Hcfge7H4WkSeEpFTRCS5/HGsGUQkHJgEXAQkAMOPcAOlp4E3VLU3MB54zJm3EfAQnpYi/fDcA7yhz2tlqtzIk9pwRVJLnv9yGd8uK3A7jgliO/cVMTo9iyaxtXjOGgTWGL4Wi5PwHHp6FHjGeTxdwTz9gHxVXamqRcAMYNBh0yQAXznPv/YafyHwuaruUNWdwOd47qlhXCIiTLiiJ12axHLLjGw2Fh5wO5IJQqVlyi1v5VCw5xBT0pJpEGMNAmsKX7vOnnOEx7kVzNYSWOf1er0zzNt8YLDz/AogVkQa+zgvInK9iGSISEZBgX3a9beYqAimpCVTXKrcmJ5FUYk1HDRV66WvlvPtsgIeuiyB3q3s6ruaxKdiISJNReRVEfnYeZ0gItdVwfLvBM4SkWzgLGADUOrrzKr6iqqmqmpqfLzdIas6dIivy5NX9SZnXSETPlrsdhwTRP69rIAXvlzO4KSWjOjXxu045jC+HoZ6HfgUKP/q5DLg1grm2QC09nrdyhn2G1XdqKqDVTUJuM8ZVujLvMY9A3s157rT2/OvH9cwZ741HzYnbkPhAW6dkU2XJrFMuMIaBNZEvhaLOFWdCZQBqGoJFe8B/Ap0FpH2IhIFDMPzze/fiEiciJRnGAe85jz/FOgvIg2dE9v9nWGmhhh7UTdS2zZk7Kxclm/Z43YcE8DKGwQWlypT0pKpHRXudiRzBL4Wi33OuQQFEJGTgV3HmsEpKGPw/JFfAsxU1UUiMl5ELnMmOxvIE5FlQFNggjPvDuARPAXnV2C8M8zUEJHhYUwckUxMVDij0rPYd8hahZnKmfDRYnLWFfLUVb3pEG8NAmsq8aVJnHOZ7EtAT2AhEA9cpaq5/o3nu9TUVM3IyHA7RsiZl7+NtFd/5uLeLXhxWKIdPjDH5f2cDdwyI4f/O709919iN9xyg4hkqmpqRdP5ejVUFp4T0KcCfwZ61KRCYdxzaqc47ujflQ/mb+Rf81a7HccEkOVb9jB21gL6tmvIPRd1czuOqYCvt1UNBwYC7Zx5+osIqvqsH7OZADHqrI5krdnJhLlL6N26Aclt7PuT5tj2HirhhqmZ1IkOZ+KIZCLDrUFgTefrv9AHwDVAYzy3WC1/GENYmPDskESa1a/F6PQstu895HYkU4OpKmNn5bJq2z5eHJ5E03q13I5kfODrfQlbOS05jDmi+jGRTBmZwuAp87j1rRxev7Yf4damwRzBv+at5sPcTdx1YVdO7WgNAgOFr3sWH4tIf78mMQGvZ8v6jL+sB98t38YLXy53O46pgbLWeg5Xnt+9CaPO6uh2HHMcfN2z+AmY7XwnohgQQFW1nt+SmYA0tG9rMtbs5KWvlpPUpgHndG3idiRTQ2zfe4jR6Vk0q1+LZ662BoGBxtc9i2eBU4AYVa2nqrFWKMyRiAiPDOpJ16ax3PZWDut37nc7kqkBSsuUW2bksH1fEVNGplA/JtLtSOY4+Vos1gEL1ZcvZZiQVzsqnJfTUih1Gg4eKvG53ZcJUi98sYzv87cx/rIe9GxZ3+04phJ8LRYrgW9EZJyI3F7+8GcwE9jaxdXhqav7kLt+F498aA0HQ9nXeVt58at8rkppxdC+rSuewdRIvhaLVcCXQBR26azx0YCezbj+zA5M/Wkt72VbH8hQtH7nfm57K4duzWJ5ZFBP+4Z/APPpBLeq/gVAROo6r/f6M5QJHndf2JWcdYWMe3cBCS3q0aWpfcYIFYdKSrkxPYvSUuXltBRrEBjgfL2fRU/nnhOLgEUikikiPfwbzQSDiPAwJg5Pok50BDe8mcmeg8VuRzLV5JEPF5O7fhdPD+lDu7g6bscxJ8jXw1CvALeraltVbQvcAfzdf7FMMGlSrxYTRySxZsd+7pmVi10nEfxmZ69n6k9r+fOZHbiwRzO345gq4GuxqKOqX5e/UNVvAPuoYHx2cofG3HVhV+Yu2MxrP6x2O47xo7zNexj37gL6tW/EXRd2dTuOqSI+Xw0lIg+ISDvncT+eK6SM8dmfz+zABQlNeWzuEjJW2+1JgtGeg8WMmppJ3ehIJg5PIsIaBAYNX/8l/4jnHhaznEcccK2/QpngJCI8fXUfWjaszehpWWyzhoNBRVW5Z1Yua3bsZ+KIJJpYg8Cg4mux6IjnnthheC6fPQ/41l+hTPCqXzuSySOTKdwSa7nuAAAV90lEQVRfzC0zsikts/MXweK1H1Yzd8Fm7r6wKyd3aOx2HFPFfC0W6Xjujz0YuMR5XOqvUCa49WhRn0cu78kP+dt57vNlbscxVSBj9Q4em7uE/glNuf7MDm7HMX7gayPBAlX9wK9JTEgZktqazNU7mfh1PkltGnBe96ZuRzKVtG3vIUZPy6Jlw9o8dXUf++JdkPJ1z+IhEfmHiAwXkcHlD78mM0HvL4N6kNC8Hre9lcO6HdZwMBCVlik3T8+mcH+xp0FgbWsQGKx8LRbXAonAADyHny7FcyjKmEqrFelpOKjAqPRMDhZbw8FA8+znecxbsZ1HLu9JQgtrRB3MfD0M1VdV7YJpU+XaNI7h2SGJ/OmNDP7ywWIeG9zL7UjGR18u2cKkr1cwNLU1Q1KtQWCw83XPYp6IJPg1iQlZFyQ0ZdTZHZn+y1pmZa53O47xwbodngaBPVrU4y+DrPNPKPB1z+JkIEdEVgGH+M+d8uy+3KZK3HFBF3LWFnLfe56Gg92b2yGNmupgcSmj0jMBmDIyhVqR1iAwFPi6ZzEA6Az05z/nK+zSWVNlIsLDeHF4EvVqRTJqaia7reFgjfWXDxaxcMNunh2SSJvGMW7HMdXEp2KhqmuO9PB3OBNa4mOjmTgimXU7D3D329ZwsCZ6J3M9039Zx6izO3J+gl3uHEqscYupUfq1b8TYAd34ZNFm/vHdKrfjGC9LNu3mvtkLOKVDY+64oIvbcUw1s2Jhapz/O6M9A3o04/FPlvLLKms4WBPsdhoE1q8dyYvWIDAk2b+4qXFEhCev7k2bRjGMmZbF1j0H3Y4U0lSVu9/OZd3OA0wamUx8bLTbkYwLrFiYGqlerUimpCWz+2AxN0/PpqS0zO1IIesf363ik0WbGXdRN/q2a+R2HOMSvxYLERkgInkiki8iY48wvo2IfC0i2SKSKyIDneGRIvIvEVkgIktEZJw/c5qaqVuzeky4vBc/rdzB059Zw0E3/LxyO49/spSLejbjutPbux3HuMhvxUJEwoFJwEVAAjD8CF/sux+YqapJwDBgsjP8aiBaVXsBKcCfRaSdv7KamuvKlFYM79eGl/+9gs8Xb3E7TkjZuucgY6Zn06ZRDE9e1dsaBIY4f+5Z9APyVXWlqhYBM4BBh02jQPm3r+oDG72G1xGRCKA2UATs9mNWU4M9dGkCPVvW4/aZOazZvs/tOCGhpLSMm6Zls+dgMVPSkomtZQ0CQ50/i0VLYJ3X6/XOMG8PA2kish6YC9zkDH8H2AdsAtYCT6uqXRYTompFhjNlZAphIoyammUNB6vB058t4+dVO3j0il50a2bfpjfun+AeDryuqq2AgcCbIhKGZ6+kFGgBtAfuEJH/uaOKiFwvIhkiklFQUFCduU01a90ohueG9mHxpt089P4it+MEtc8Xb+Hlf69gxEltGJzcyu04pobwZ7HYgOdWrOVaOcO8XQfMBFDVH4FaeO7vPQL4RFWLVXUr8AOQevgCVPUVVU1V1dT4+Hg/rIKpSc7t1pQx53TirYx1zPx1XcUzmOO2Zvs+bp+ZQ6+W9XnwEusdav7Dn8XiV6CziLQXkSg8J7DnHDbNWjz380ZEuuMpFgXO8HOd4XXwNDJc6sesJkDcdkEXTuvUmAfeX8iijbvcjhNUDhaXcsPULMJEmDwy2RoEmv/it2KhqiXAGOBTYAmeq54Wich4EbnMmewO4E8iMh+YDlyjnoZAk4C6IrIIT9H5p6rm+iurCRzhYcILw5JoGBPFqKlZ7DpgDQeryoPvL2TJpt08N7QPrRtZg0Dz3yRYmrWlpqZqRkaG2zFMNclcs4Ohf/uJc7o14ZXfpdhlnSdo5q/ruHtWLmPO6cSdF9p9zkKJiGSq6v8c5j+c2ye4jamUlLaNGDewO58v3sLfvl3pdpyAtmjjLh54fyGndWrMbdYg0ByFFQsTsP54Wjsu7tWcJz9Zyk8rt7sdJyDtOlDMqKlZNIyJ4sVhSYSH2R6aOTIrFiZgiQhPXNWbdnF1GDMtm627reHg8SgrU+6YOZ+NhZ4GgY3rWoNAc3RWLExAqxsdwctpKew7VMKYadkUW8NBn/3t25V8sWQL9w7sTkrbhm7HMTWcFQsT8Lo0jeWxwb34ZfUOnvo0z+04AeHHFdt56tOlXNy7Odee1s7tOCYAWLEwQeHypJakndyGV75dyScLN7sdp0bbuvsgN03Ppl1cHZ640hoEGt9YsTBB44FLEujTqj53vT2fVdus4eCRFJeWMWZaNvsOlfByWgp1oyPcjmQChBULEzSiI8KZNDKZ8HBh1NRMDhRZw8HDPfVpHr+s3sHjV/aiS9NYt+OYAGLFwgSVVg1jeH5oInlb9nD/ewsJli+dVoVPFm7ilW9X8ruT2zIo8fAG0MYcmxULE3TO7tqEm87tzKys9cywhoMArNq2j7vezqVP6wbcf0l3t+OYAGTFwgSlW87rzBmd43hoziIWbgjthoMHikoZNTWT8HBh0ogkoiOsQaA5flYsTFAqbzjYuE4UN0zNZNf+0Gw4qKrc/95C8rbs4fmhibRqaA0CTeVYsTBBq1GdKCaNTGbL7oPcPjOHsrLQO38x49d1zMpaz83ndubsrk3cjmMCmBULE9SS2zTk/osT+HLpVqb8e4XbcarVgvW7eGjOIs7oHMfN53V2O44JcFYsTND7/SltubRPC575LI95+dvcjlMtCvcXMSo9k7g6UbxgDQJNFbBiYYKeiPD44F50iK/LTdOz2bwruBsOlpUpt8+cz5bdB5k0MplGdaLcjmSCgBULExLqREfwcloyB4pLGTMtK6gbDk759wq+WrqV+y9OIKmNNQg0VcOKhQkZnZrE8viVvclYs5PHPw7OW7r/kL+NZz7L49I+Lfj9KW3djmOCiBULE1Iu69OCa05tx6vfr2Lugk1ux6lSm3cd5Obp2XSIr8vjg3tZg0BTpaxYmJBz78DuJLVpwN3v5LKiYK/bcapEcWkZo6dlcaC4lJfTkqljDQJNFbNiYUJOVEQYk0YkExURxqipmewvKnE70gl7bO5SMtfs5Ikre9OpiTUINFXPioUJSS0a1OaFYYks37qX+2YHdsPBj3I38doPq7jm1HZc2qeF23FMkLJiYULWGZ3jufW8LszO3kD6z2vdjlMpKwr2cvc780lq04B7B1qDQOM/VixMSLvp3E6c1SWe8R8sJnd9odtxjsv+ohJGTc0kOjL8t8NqxviL/XaZkBYWJjw/NJH42GhGTc1i574ityP5RFW5b/ZClm/dywvDEmnRoLbbkUyQs2JhQl7DOlFMHplMwZ5D3BYgDQfTf17L7OwN3HZ+F87oHO92HBMCrFgYA/Rp3YAHLk3gm7wCJn6d73acY5q/rpDxHyzm7K7xjDmnk9txTIiwYmGMI+2kNlye2ILnvljGd8sL3I5zRDv3FXFjehbxsdE8NySRMGsQaKqJFQtjHCLCo4N70blJXW6ZkcPGwgNuR/ovZWXKbTNzKNhziMkjk2loDQJNNbJiYYyXmKgIpqSlcKi4lNHTsigqqTkNByd+nc83eQU8cGkCfVo3cDuOCTFWLIw5TMf4ujx5VR+y1xby6NwlbscB4LvlBTz3xTKuSGpJ2klt3I5jQpBfi4WIDBCRPBHJF5GxRxjfRkS+FpFsEckVkYFe43qLyI8iskhEFohILX9mNcbbxb2b88fT2vP6vNV8MH+jq1k2Fh7glhk5dG5SlwlX9LQGgcYVfisWIhIOTAIuAhKA4SKScNhk9wMzVTUJGAZMduaNAKYCN6hqD+BsoNhfWY05knEDu5HStiH3zMolf+seVzIUlZRxY7rncNiUtBRioqxBoHGHP/cs+gH5qrpSVYuAGcCgw6ZRoJ7zvD5Q/hGuP5CrqvMBVHW7qpb6Masx/yMy3NNwsHZkODdMzWLfoepvOPjo3CXkrCvkyat60zG+brUv35hy/iwWLYF1Xq/XO8O8PQykich6YC5wkzO8C6Ai8qmIZInI3UdagIhcLyIZIpJRUFAzL3U0ga1Z/Vq8ODyJlQV7GffugmptODhn/kZen7eaP57WnoG9mlfbco05ErdPcA8HXlfVVsBA4E0RCQMigNOBkc7PK0TkvMNnVtVXVDVVVVPj4+1brMY/TusUx+0XdGHO/I28+dOaallm/tY9jJ2VS0rbhowb2K1almnMsfizWGwAWnu9buUM83YdMBNAVX8EagFxePZCvlXVbaq6H89eR7IfsxpzTDee3YlzuzXhkQ8Xk712p1+Xte9QCTdMzaK20yAwMtztz3TG+LdY/Ap0FpH2IhKF5wT2nMOmWQucByAi3fEUiwLgU6CXiMQ4J7vPAhb7MasxxxQWJjw3JJGm9WoxOj2LHX5qOKiqjHt3ASsL9vLS8CSa1beLAE3N4LdioaolwBg8f/iX4LnqaZGIjBeRy5zJ7gD+JCLzgenANeqxE3gWT8HJAbJU9SN/ZTXGF/VjIpkyMoVte4u4ZUY2pX5oOPjGj2uYM38jd/Tvyqmd4qr8/Y2pLAnkO4R5S01N1YyMDLdjmBAw7ee13Dt7Abec15nbLuhSZe+btXYnQ//2I2d2jufvv0+1vk+mWohIpqqmVjSdHQw15jgN79eawcktefGr5XyTt7VK3nPHviLGpGfRtF4tnrUGgaYGsmJhzHESESZc3ouuTWO59a0cNpxgw8HSMuWWGdls21fElJEp1I+JrKKkxlQdKxbGVELtqHCmpKVQWqrcmJ7FoZLKf2f0xS+X893ybfzlsh70alW/ClMaU3WsWBhTSe3j6vDU1b2Zv66QCR9VruHgN3lbefGr5VyZ3IphfVtXPIMxLrFiYcwJGNCzOX86oz1v/LiG93MO/xrRsa3fuZ9b38qha9NY/nq5NQg0NZsVC2NO0N0DutG3XUPGzlrAsi2+NRw8VFLK6PQsSkuVKWkp1I4K93NKY06MFQtjTlBkeBgTRyRTJzqCG6ZmsteHhoN//XAJ89fv4qmre9M+rk41pDTmxFixMKYKNK1Xi5eGJ7F62z7umZV7zIaD7+ds4M2f1nD9mR0Y0NMaBJrAYMXCmCpySsfG3HVhNz7K3cTr81YfcZplW/YwdtYC+rVrxN0Xdq3egMacACsWxlShG87qwPndmzLhoyVkrvnvhoN7D5Vww9RM6kRHMHFEEhHWINAEEPttNaYKiQjPDOlDiwa1GZ2exba9hwBPg8B73sll9bZ9vDQ8iSb1rEGgCSxWLIypYvVrRzJ5ZDI79v+n4eA/f1jNRws2cdeF3TilY2O3Ixpz3KxYGOMHPVvW55FBPfghfzs3z8jm0blLOL97U244q4Pb0YypFLv7uzF+MrRvGzLX7GRmxnraNIrhmSF97It3JmBZsTDGj8YP6kmjOtFcmdyS+rWtQaAJXFYsjPGjWpHhjL3I7qFtAp+dszDGGFMhKxbGGGMqZMXCGGNMhaxYGGOMqZAVC2OMMRWyYmGMMaZCViyMMcZUyIqFMcaYCsmxbtISSESkAFhzAm8RB2yrojiBKtS3QaivP9g2gNDbBm1VNb6iiYKmWJwoEclQ1VS3c7gp1LdBqK8/2DYA2wZHY4ehjDHGVMiKhTHGmApZsfiPV9wOUAOE+jYI9fUH2wZg2+CI7JyFMcaYCtmehTHGmApZsTDGGFOhkC8WIjJARPJEJF9Exrqdx59EZLWILBCRHBHJcIY1EpHPRWS587OhM1xE5EVnu+SKSLK76StHRF4Tka0istBr2HGvs4j8wZl+uYj8wY11qayjbIOHRWSD87uQIyIDvcaNc7ZBnohc6DU8IP+viEhrEflaRBaLyCIRucUZHlK/BydMVUP2AYQDK4AOQBQwH0hwO5cf13c1EHfYsCeBsc7zscATzvOBwMeAACcDP7udv5LrfCaQDCys7DoDjYCVzs+GzvOGbq/bCW6Dh4E7jzBtgvP/IBpo7/z/CA/k/ytAcyDZeR4LLHPWM6R+D070Eep7Fv2AfFVdqapFwAxgkMuZqtsg4F/O838Bl3sNf0M9fgIaiEhzNwKeCFX9Fthx2ODjXecLgc9VdYeq7gQ+Bwb4P33VOMo2OJpBwAxVPaSqq4B8PP9PAvb/iqpuUtUs5/keYAnQkhD7PThRoV4sWgLrvF6vd4YFKwU+E5FMEbneGdZUVTc5zzcDTZ3nwbxtjnedg3VbjHEOs7xWfgiGIN8GItIOSAJ+xn4PjkuoF4tQc7qqJgMXAaNF5EzvkerZ1w6pa6lDcZ0dU4COQCKwCXjG3Tj+JyJ1gVnAraq623tcCP8e+CzUi8UGoLXX61bOsKCkqhucn1uB2XgOLWwpP7zk/NzqTB7M2+Z41znotoWqblHVUlUtA/6O53cBgnQbiEgknkKRrqrvOoND/vfgeIR6sfgV6Cwi7UUkChgGzHE5k1+ISB0RiS1/DvQHFuJZ3/KrOv4AvO88nwP83rky5GRgl9cue6A73nX+FOgvIg2dwzX9nWEB67DzT1fg+V0AzzYYJiLRItIe6Az8QgD/XxERAV4Flqjqs16jQv734Li4fYbd7QeeKx+W4bnS4z638/hxPTvguYJlPrCofF2BxsCXwHLgC6CRM1yASc52WQCkur0OlVzv6XgOsxTjOcZ8XWXWGfgjnpO9+cC1bq9XFWyDN511zMXzx7G51/T3OdsgD7jIa3hA/l8BTsdziCkXyHEeA0Pt9+BEH9buwxhjTIVC/TCUMcYYH1ixMMYYUyErFsYYYypkxcIYY0yFrFgYY4ypkBULYxwiMs/52U5ERlTxe997pGUZEyjs0lljDiMiZ+PpyHrJccwToaolxxi/V1XrVkU+Y9xgexbGOERkr/P0ceAM5z4Pt4lIuIg8JSK/Oo33/uxMf7aIfCcic4DFzrD3nEaNi8qbNYrI40Bt5/3SvZflfEv4KRFZKJ57jQz1eu9vROQdEVkqIunON5ERkcedezPkisjT1bmNTOiKcDuAMTXQWLz2LJw/+rtUta+IRAM/iMhnzrTJQE/1tPMG+KOq7hCR2sCvIjJLVceKyBhVTTzCsgbjaebXB4hz5vnWGZcE9AA2Aj8Ap4nIEjztObqpqopIgypfe2OOwPYsjKlYfzy9gnLwtLZujKdnEsAvXoUC4GYRmQ/8hKfpXGeO7XRgunqa+m0B/g309Xrv9epp9pcDtAN2AQeBV0VkMLD/hNfOGB9YsTCmYgLcpKqJzqO9qpbvWez7bSLPuY7zgVNUtQ+QDdQ6geUe8npeCpSfF+kHvANcAnxyAu9vjM+sWBjzv/bguf1muU+BUU6ba0Ski9O593D1gZ2qul9EuuG5JWe54vL5D/MdMNQ5LxKP5xaovxwtmHNPhvqqOhe4Dc/hK2P8zs5ZGPO/coFS53DS68ALeA4BZTknmQv4zy04vX0C3OCcV8jDcyiq3CtArohkqepIr+GzgVPwdANW4G5V3ewUmyOJBd4XkVp49nhur9wqGnN87NJZY4wxFbLDUMYYYypkxcIYY0yFrFgYY4ypkBULY4wxFbJiYYwxpkJWLIwxxlTIioUxxpgK/T//aWh7cuMyQgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.plot('momentum')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final accuracy here is **~89%** using only unigram feaures and a simple model. In the [text classification notebook](https://github.com/amaiya/ktrain/blob/master/tutorials/tutorial-04-text-classification.ipynb), we show that an accuracy of **~92.3%** can be acheived on this dataset in mere seconds using built-in convenience methods in *ktrain*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}