{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting age of a child by looking at a hand Xray\n",
    "The goal of this project is to identify the age of a child from an X-ray of their hand. This data was part of a competition http://rsnachallenges.cloudapp.net/competitions/4. We will build a model inspired by the one used by the winners of the competition. You will also borrow ideas from this fast.ai notebook. https://github.com/fastai/fastai/blob/master/courses/dl2/cifar10-darknet.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import torch\n",
    "from pathlib import Path\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "from torch.autograd import Variable\n",
    "import torch.optim as optim\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "import matplotlib.pyplot as plt "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get the data you can install the kaggle api using: <br/>\n",
    "`pip install kaggle` <br/>\n",
    "To get the dataset you can use this command line. (If you don't provide a path you will find your data in /home/user/.kaggle/datasets/kmader/rsna-bone-age/ )<br/>\n",
    "`kaggle datasets download -d kmader/rsna-bone-age -p PATH` <br/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[PosixPath('/data2/yinterian/rsna-bone-age/model.pth'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/rsna-bone-age.zip'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/boneage-test-dataset.csv'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/boneage-training-dataset'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/boneage-test-dataset.zip'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/boneage-550'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/boneage-training-dataset.zip'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/model003.pth'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/model001.pth'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/boneage-training-dataset.csv'),\n",
       " PosixPath('/data2/yinterian/rsna-bone-age/boneage-test-dataset')]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PATH = Path(\"/data2/yinterian/rsna-bone-age/\")\n",
    "list(PATH.iterdir())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id,boneage,male\r\n",
      "1377,180,False\r\n",
      "1378,12,False\r\n",
      "1379,94,False\r\n",
      "1380,120,True\r\n",
      "1381,82,False\r\n",
      "1382,138,True\r\n",
      "1383,150,True\r\n",
      "1384,156,True\r\n",
      "1385,36,True\r\n"
     ]
    }
   ],
   "source": [
    "! head /data2/yinterian/rsna-bone-age/boneage-training-dataset.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f9aa844ca20>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAN4AAAD8CAYAAAAYAxqKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsvXuspdlVJ/bb53nP+577rqquLlc3DcZBxCYWAwmK0DjMABkNY8FIoFFwBkv+I0wyCZECJH8gJRrEKFFIUAiRMyYxEoJBZKKBkTMTy8MoRH7gBiybbkO73a/q6qr7vve83zt/nPPbd33rrv2dc6s87VuktnR0vsf+9nO999prO+89nqan6Wl6d1Pmm92Ap+lp+v9jeop4T9PT9E1ITxHvaXqavgnpKeI9TU/TNyE9Rbyn6Wn6JqSniPc0PU3fhPSuI55z7gedc3/hnHvVOfdz73b9T9PTdB2SezfX8ZxzWQCvAPgBAG8D+CKAn/Dev/yuNeJpepquQXq3Od53A3jVe/+a934E4LcB/Mi73Ian6Wn6pqfcu1zfLQD3xP3bAP6KzOCc+xiAjy1u/y3n3LvUtKfpaZon5xwoCUr4895jGTzOZrMj7/32sjrebcSzWp2Qdb33HwfwcQDIZDJ+bW0N6n3414PAZ957eO+RyWQS+YHkQM5mMyzqSeT5RiZdL9vHe/le1i/zLWvXsrz6mQU8HK+057ocPQeyL2ltsfqu581Kq4xHrBwNDzIvr62y0tovy2XqdrtvRhsn0rstar4N4La4fwbAO7HMFmLxF8tvAbIEepbDew1UemJ0/bq+ZRRQA6GeuGXfyzpj/bbaznsN3Dp/DPA08bLGXX8j65jNZtG5sp6TCDrnEt/qvlnlXQVprPGUyCq/jxEa2TY9pqumdxvxvgjgBefcXedcAcCPA/i9WObYIGcyGbPDsYnmpMr8ehA1J9Jla6rMci3gsK5j5Vp1LJt0/XwZUEpg5rNlnDFGMGLAK4GReTWyx4Be/svrtH7JcixETft+lXFI+5btk0RpFclEpndV1PTeT5xzfw/AvwCQBfDr3vuXlnwT5VhWkoCmB2MZBYshh9Umq31SlNGApKmjFj3TuLjuizUm8loCPu8tsTutX3p8Ytx/lbLSOFJavy39Ko2gWeNrtUHn023QIqkuRxJeyeGvYo94V5cTrpqkjmcBriUqaADVSQNymjgZ08esZIm2VvlWfv1cfqsn1Oq7fmfdW/2KIbDVV+89stmsyQGt/l+lLbH2xURyiwDJZBE9SXxi46e/iY2LHDM5P957DAaDP/bef/BSoSpde8+VGAeR76zJT6PE1Os0F9DfxsQo/Yz3msvEgEXXp9sXQ4hY+3R9umwpagPAdDpNjJdfiGuxOrS4miZFsD4rT9qcyLGTYuqqYmeMwFnEwIIXmd8i6OyTrGc2myXUnphRyErXHvGAy4C/DGiBuHINXACGBLa0MpmsMjnwFmDGytT16np03dbEOmcbhiTQ814CkAY+q18yT5oeZOnOum2aI8QQVz/TP45vbGxiBNTipFbdaQho1WOVd5X0RCCeTDExIjYR+turiCcyaaAlskmKqPUS+S3/JZJa5cpvZB0x8c3qj+QcGkE0xdZ59ditymksBNBILwlIDCl0eXIMrO8sBNXt0whmfW/1dRUir8d31fTEIJ7mVkwxPSCGAGkKsCXbWwDAsqx26EmI6U7LRKkYZwBsI04M4Kw6LGSLjYcuT1tHWa7mRmnlyu9jXDVGBNLm2BLLLelC3ut51e2z5k/Pe4zgpqV3ewH9kZNFWTUA6cmL6TzyfZrIpfNZHEcjo4WgMr9uF4CE0m99E+OG1r0U8fT7mLUuDYn1mEjxNk3KSEPuGJe09K9lcyKvY3OcVnYM4a17qz2xcpalJ4bjMcXEBE11rMlOo2w6aZHMKtcSdWW7JLJq7sPvJTJIBf4qVNQChpgXisUlZF3LECZGgKxr3Sfd1hiQaykjliwCIfuzShtj0kDs/SptWSVde8TTnMMSB2ezWVQEtShhDHBiSL3snZxoixAsK3dZu9NEH500kBO5LSC1iIbVX31v9TMNcNNEaiuPbJulP6WNgR43axx1Xi1taKPVMgT+S8nxLGour2MUju85kHKA08QYi1PFBtWaZIsLanFkmZioAUXXkTZWsoxVRV7dbwCXDCO6HRqZYhwmJhGk1R1rZ1rfWaaFNHq+Lc6q28ykLdUxwn/V9EQgnsURmKwB1QMlLY4aIHVZVr1pyA/ERVEJxLJdFmJawLeqDqIRxhob3Y9lXCNt3C2EXIU7rSKO6fwx5Od1bC3WIm4xRLYIQ6xdFrw9Srr2iKcpTQxhrEGPUS4uIOvJXEaxY8hi3ctyliGxRf1jbeC9Xq/jdZpuZ7XXurbExRhgchE51jftRxkjMLExiY2HzmtxNtkuq19pCBaDhTSifRVEfGKsmss6FUPM2OAy7zL5XQ62NegWJ9CLxml9kRzZAizek7oDuJQfuCym6vGQ3v/yve5DrH79Tay/Vj9WkVpk0m2zkEZ+axEGjqtVp25rDMli9ch/ixOukq494sUmO00ut4BPAq8FELFJkskCHtmWmMgZ+zbNWiqfa+ov283+TKfTS9wuRihiRCyGiNZ3FsHRyJ7JZC4Zvaw+pRE/3aY0yWcZsZPjHuPqq8KbnpOruIsBT4CoqfWGNJFFJzlIy8RP5ouVY73TotoyfUYjelp9um9WGzXn1sQj1v+0cdMcKwbEWtSN6WaxtIpoFtPTLCS32qXLscZQE26LECzr21U4HdO1RzwgLorExAx5vYy6s7y0+qxnUtSQgC3bZ5UR4w5W/pjvZ4xjyDZYniSW7qPLzWQyyOfzl/picR7+W4And/dbhE+LarF+S0CPzbMlrVgwY71b1alg2fUqhESmay9qUlwBLsvzehItzgPYXNO61xO7jPLp5xqxdJkx8SYGUGlAlNZeq63kBjE/yXK5jLW1NRQKhTBeo9EIZ2dnZvsskUvrPjFxUHJonVcjoyawlrgY4zhpbbP6INsXG1erLbq+VdK1R7xVvEbkuxjAW4gpv4kBty7DKlu3aZnIanEPi2KSAy3jwJojWsQJsNcJnXOoVquoVCrIZrMJ97VcLod8Po/Dw8NoH2LPYjrTMsBdRUKRYyfLXDZGyxAlrW7dHy1dLBOtdXpiRM00UY9J3luUk8/lvfVuWUoTLTR1tPqi38cIQBoCyzwW5V8GRPKeSJfNZgEA2WwWuVwuXG9sbGA6nZrIHDNKWf2MtS3G8TUXjIl6bIcl8VjjLJ9b72SeGPe+CpJZ6ZERzzl32zn3B865rzrnXnLO/f3F8w3n3Kedc19b/DcXz51z7lfcPIL0l51z37VKPXIg04CaossyLwKL66UBrJ50WbfFdVdBSPmtrlvqRKsArRTDYyEiLIBjGyqVSqiv0Whga2sLlUoFs9kM+XwemUwGuVwOuVzuEpDGypVjvKrlln3RCKbzxDi6fJbmbZJGxOV7qy8xon0VEZPpcTjeBMB/7r3/dgDfA+CnnXPvA/BzAD7jvX8BwGcW9wDwQwBeWPw+BuDXVqlkGQvXiLlsECyuKAdbAr5VlrVYbOkPepIsahwTz3S/dP26DAKsDM0QS1p0JXKtra2FcnO5HGazGcbjMTKZDLLZLKrV6iXR2OIssp8cT9lW2Wadl+XEnANkXjlfFrHUYx9b5Jd1ym/SxNDY3F4lPTLiee8feO//ZHHdBvBVzAPW/giATy6yfRLA31pc/wiA3/Dz9HkA6865G6vUFZPNnbO3umgA0GVYnCnGAVkHrzV1lP8aYHR9FsVMo+6xdxoY5VqexRF13+U9kZWGFep5eozz+bwZ0kGPSZqYp41kVj6NWLrPGgljISuAJJGykEQ6FcS4YmyumdKIdFr6huh4zrn3APgAgC8A2PXeP1g08gGAnUU2K4r0LaOsjznnXnTOvagHwEI4a9KtgdDAnjZQMVFEA53RdvM7q/2xeq06YqKwBkLdLwlQMSTmWM5mM2SzWQwGAxQKhUvfyl9aG9IIjzVuMY6vkySAmogtkzKsegE7kLEWKS2urOuw1g6XpcdGPOdcFcD/AeA/9d630rIazy611nv/ce/9B733H9SApAc8bRL1szREWyaqpnGvtJQmVsae6XplnjTEsspJi1ECzAGPXKHdbmM0GmE6nSKfz1+i+mn1yHKtMdSIa323jJDKPvCZziuJhM6vx2EVQpDGefU8xOqLpcdCPOdcHnOk+03v/T9ZPN6nCLn4P1g8v1IUaaYYBV2Ur9tz6X2M+8UGz6pLI0iavqDL0+JwrM2yvVrsstqVhpC6bF0GkU1T68lkEvQ7AInlBau9MmmkjHH4ZURLI1JaP2J1y7KWcVuLs6dJGmmE/yrpcayaDsAnAHzVe//fi1e/B+Aji+uPAPin4vlPunn6HgDnFEmX1HMJEC1OKP+tAbfEMubVeTT1tiirVVZasowMMWob47oWoMTy6X7E8vA+k8mgWCyGZQUSjNj+Pt0XXscIncxj9TPNoLJsXGL1pc2P9TwWljDWl8dJj7OA/u8A+A8AfMU596XFs/8SwC8B+B3n3EcBvAXgby/efQrADwN4FUAPwN9dpZKYXB0DPucue/vLZ6tQW12frOtRKLZss+7Tqkl+q6/TKLTVbvmMPy4hZLNZjEajVOJmiXc6n9V2/X2sf/LZsrGKSQWx/scIiNVOq70xLrwK8ZXpkRHPe///wtbbAOBDRn4P4KevWo8cLGmFinGLWNg8C3Cte12f5DDLqKF+vgxw0pB52TsrX8zrnv865DjX5wCgXC6j1+uF+2KxiPF4HMqJbSuyxDMLONOAX95nMhmUy2VkMhlMp1N0u12TAKaNqaybz1bhmqw/NmdpcHPVEO7X3mXMEhti3CuN8jPFJp7vYlwhrU26fJkvTTyzgNcSbZdxSk1ktD9mrB4pWnrvUSwWAczF4lqthrOzs0v1xEQ0iwOsOh+c02q1irW1taCDTqdTFAoFtNttTCYTs++6XL09J/Zct0G2eZVzJtLmaZV07REPgAlMFmexAFTnk/9WPTHqaiGrNSmx+mLIu6wNMW6iU6wuea85eD6fRzabDdyN75xzmEwmCSnDGoPYeC7rqzW2a2trKJfLyOVy4Xk+n8dkMkEmk8HJyYnZJ91/uSvCapvVDv1Mr+9Z9cj3jyJuPhGIpweD/5aoIu9l/jTuZokJ+mATawIs7pm2m4KJ96VSCWtra0G8o7dIu90222uVs8pkW/3w3gcjSjabRblcRqlUQiaTwfn5+aW+WB4zMYCLjb1ui+xjrVZDJpNBvV4P1tTT01MUi0VMp1NUKhV0u93UemNzpd/F4Eb3jym2oTc2HqukJwLxgHTz8jIqJb9Pk9NjetWyiZZJEwhrMpxzqNVqWFtbC8DPvLQkdrvdhI6lubhGfIvTsT2WB4ccp8lkgtFoFFzHCoUC+v0+stksptPpJYRlXWyvvNeIFZsLeS3d1rLZLPL5PPr9fmibdNpOk2rYHl2XHiOrXTHYAhB298cIaFpfY+naI16MkuiOLqM4lihqyefL9JVV2ia/1XnJYfL5PAqFAoCkB0U2mw36zvHx8SV3Nau8tKSPI7bEZe89ut0uAGAwGGA2mwXuovuSFvXaEj01oFtidLPZRDabRaFQQKlUCj6iJKi5XO6SH6qFUMvCL+i2pHFG/UzDRYzAr5quPeJZE2uJcJbYIAfL8oyIIZMF1JaIq/MvazfzlUol5HK5ACjkMq1WK9Sby+VQKBSCrpUmrqWJdrpvBFDJmWezGabTKVqtViAMxWIRo9HoEpBpsdyak9gY8XtZb6VSCVy+2WwGp+x+v2+qDro/fL+qmCfLSdtMu6oYmobEaenaI54GphjSWe+swbC8SJZRbVmPRjQL6bRYo9/T6dj7i71ww+EQ6+vrODs7C94jxWIRk8kkKtJYyBeTBGJjQavm9vZ2wmOF+ubZ2dml/mquqYlVTASX+XhPvTKTyaDX64V1RCIe9whKkdca01X7vOz9Mm6miajMf5XAttd+I6wEXgtBeK2Vf5lHiz5p72Ueq84YAOrvYsFd2c7hcJjYP1gqlTCbzYKRAUBwVrbKl+2xvGKsZCHGdDpNbPkZj8eYTqcoFotBr9J1xMS1GOfnWOn3tVotxHfZ3t4OIjiRTc7peDy+BAvLxPxV1QDZzlgkas1dLeL9l+5gSqY0iqf1EP2d/D4mKqUlWWfaxK9SDoBgJi8Wi4H7ra2tYTqdBi4kRUJ+a3GNZdQ9RoAI2NzkWiqVUCwW4ZzDYDBIjKMFVDHxTHNfvTGVqVQqhb6SCE2n01C3tPaORqNLfdL9WUUFkHOg+8IyYpHt0sYiVm4sPRGIp6mNHHgLkWJcSpepy4t9cxUdgvl1XZpg8P7s7Aybm5vBusn1NCKejPgVWxiXdVoAaSVSd+99MGrQkNFoNIJFk/XyG1lvTMKw6pLXs9kMGxsbKBQKyOVy2NvbS3BgAGE9zzkXrK5p3FzXo+csRqytcYsRd933Ze1JS9ce8dg5yyNBi376G33N+zQdL+07nSyxZ5mYAyAAETnW/v4+nJt7apRKpTCpuVwOpVJpaTssRLOIlfUdDSn5fD4YdPL5PKrVavBcYT5dB9/p8dYShh4zeqhwZ7scP1owaXzy3ke9ViRsWJw9hhyWBCQ5XdriexpRuwoiPjGIF1Pi+U4+0xxBv7MMLMt0I1mPBcwWgbAQ0jmHXq+XaD8XzGezGer1OgAEwCsUCmb9GgA099fApZFFAnY+nw/uYs7NDR7D4TBwRd1++Ry4vHZpJfmObmGZTAaTyQRra2uo1Wqo1WoAEJCcY8L1zJhUYv1b6oZFIDQHIwHUcCfH2urTVbnftUc8a7uIhWAyWcAmB9AqI8Y5YxOmDRqxciSSy2+os2iO2e/3Ua/X4b1PRP7S/WOyQvbp/soxYJKi3Gw2Czqn7o9Vp65HL6LH6nRu7h/KEBOlUgmNRiP4ZNKwwxAW0+kUk8kkEAE9n/LaIjQxIpT2Xv7HiKvVBut9Wrr2iCeThVCADXQaoK33ligSi5vC+nVZsWSVIcumuZxt4AIxt+YQmAmgy/qc5lUv87APtJiyLoqbDP8QE/F1n5YRQZmH63asZzQaodVqYTKZoN1uo1wuB6cCjtVsNguIx7pj85JGHKw28V2aSLls3GWblsGETNce8azBtBBAI1WsjBgiXqUdelLSJlbf819a6ShyOTc3rtDYwXdra2uXyrfEbZYfIzrymdxdns1mg7eIdExe1her3zHix/KoSwLzJYRqtYpOp4N+v4+jo6PgREDOJ/U7i1haKscyAiT7sExclH1dNSL5KunaL6BLgIqZ1vleUnZpatcDrr0nZHl8bumHsj3Mp9up2yfbpv8nk0kwZnjvg2GBHIGWRb6X9cbaFAMGLSlIFyxeT6fTsHAvkzZeyL7FgE2Pjfc+sftgd3c3GFEogsqQ8eR4/X7f1FFjfVumZ5IIyP/Y2KVx75j0tWq69hwPsE3DmrJfhZPJQdMOxLrctElPo67LFHJyN+89xuMxKpVKCDYkrWsSOGMilVXHMn1EcjRuiM3lcphMJsEx2eqnLsfaGxkbE2mxLZfLgbsyVDyXTrikMJ1Og7O0bjPvLTF6WWJ/YqEBLWKu7y14vEr6RkQZyzrn/tQ5988W93edc19w80jS/9g5V1g8Ly7uX128f88V6gjXFlWKiR46j6ZwkjPGRNo0cUq3LVZnrA+TySQA2Wg0SnC/4XAYCEM2m0WlUrn0fYzLxgiCNMTQqih3oXs/3ww7HA4xHo8Ta4iSQ8q+WLplDHDz+Tycm1tNJ5NJ4K50kaN7nBwfPW4xzqvbpa9j4rJur6xLSy0yWVLUVZDvG8Hx/j7mwWyZ/iGAX/bzSNKnAD66eP5RAKfe+28B8MuLfCsnPeiWtzivrW/lhK46WfJ73Q5dtsynuaJFHIA5shHY+v0+ut0uWq0Wjo+PUSgUUKvVQt61tTUzlgz/df+tPFJU54J9rVZDp9PBaDQK4h29Z8rlcmK8Y9ZLixtrYOSifC6Xw87ODpy7MChVKhWMx2McHx+HNnLR3JoL7dy8jPjpOdTzmEbMY31LG/tV0uOG93sGwL8P4B8t7h2AvwrgdxdZPolkJOlPLq5/F8CH3AokQg6QdUBGTBxMA3oLeWJ1LxMjLITSYmqMukrzOYGsUqlgfX09cD/qekSUmKiTJkpabWY71tfXQ/0yzuZgMECxWAzP+JNjyh/r1yETJIdlmAnv5+uGjUYDk8kExWIxOEozTadTjMdjdDodAElxVtdpzZGFDCxf6qurqCTyWvbRKv/d5Hj/A4D/AgAxYhPAmfeepigZLTpEkl68P1/kX5rS2LglDkrgiIUd57e6LFmmnmjrm2XIGeNEvKZON51Ow0IyRVTuUHfOoVAoBOtmmuhkESmNmPTHpAg7mUwSB5Rwr2BMZNSAbbmyyb5y7Y51ksvK7UdHR0cAkLBmaiNYTHJxziV2L+hxtuYrTV2IzacmpJp4vyuI55z7GwAOvPd/LB8bWf0K72S5iRDuBMLFu/lHi0HjZKQhkjU4HHjNQWMIrrmq/E/TBeX3Vv0SYAjAvV4vAN7Z2VkiAFG1Wr1UplWX1T75vlgshr6S67ZaLbRaLfT7/cQithxHORexujVweu/RaDQC0r3nPe/BcDhEPp8P0cRYJ3Axp3L3PfuviZZMmgvK5yw3Jp7K/slv0hBUzpl+t0p63Liaf9M598MA1gDUMeeA68653IKryWjRjCT9tnMuB6AB4EQX6r3/OICPA0Amk/EWUFsIpkVLC+AsjmYhRSxZlH9ZitXLNBgMUK1WE6JcvV7HYDCA9/MdCycnJ2GrjgbqZZxWtwW48Idk/fl8PvhOEuDl3jwCmO6/NtZYHM+5C28VxlRpt9thnuixIgkfEW8ZR4kRRN0m5tO7OSxEtjh5bN410r0r+/G89z/vvX/Ge/8eAD8O4F967/8OgD8A8GOLbB9BMpL0RxbXP7bIv5I2alEi3muxUL63RFD90++s72Qeqx1We5chMRONGs7N/SfX1taCNY/bZuRES+TTgMl/DYy6b7SUEtmLxWLYCc6lC4qe1viyj6twXYZ18N7j2WefDQvkcn8dly9YPrcG6Xm2xlnPoSX9xAhujDilSThWXv6+2fvxfhbAzzjnXsVch/vE4vknAGwunv8MLs7Nu3KKiZbA8sHX99ZgyusYkqYlTR2XITL1vIcPH4Y9caVSKXAi4MLTpFarpRIQq14tRpHj1Wo11Ot1ZDKZwOm4tCDjW2oRbRmR4zvucigUCgG5uU5YLpeDaxr34kng1ZxdEz3dV91nTWxi91a5UoKK1SXvr0Jomb4hnive+38F4F8trl8D8N1GngEuwrlftfxwbVF5+S4m68s8VxUXY+2JiXwx0Ue3g2k8HoflAhnyge8Y5Vl7lWgRD7BDW8g65eL8YDDAm2++ieeeey5swiX3OTk5SQQ8kmVbgG4hYr1eD0j+zDPPoNVqBeMNjUfdbjdsfGV5lsN2jNtqNUPPj5YErDJifZLjrOfa4pxXQbwnwnMlxq00NdYIZXE+C1DSqKNOMh+ptDSJr7IrXJYDAJ1OJ5TV7XYxm80CNyiVSsGownUwad2U/7Fd4rKvBHzqeAyylM/ncX5+jn6/j+FwGERPvTtCWzAlh5LASOduusABFwaOfr+P8Xgcdh7IEBjeXzgPyD6kjaMmdM45k0jof829Y4RkFeK8aj6ma++ryWSJHfod31vXOmmquExUsAChUCigUqkkzh8Yj8chIGxaX2R54/E4AN7h4WEI6grMdzGUy+XwXTabRaPRCJtnLWos26gBgpyF+t1wOMTBwQGOjo4Sa4Wai7IMLqTrPHp+6vV6EDFv3LiBjY0N9Pt9tNttAAiWTe2dMp1Ow37F2LjLOYkBvOZmFhezJAbWc1VPlJgeGktPDOJZSGGxe5liA2EB1zKRQQNWJpNBpVIJ1joAYV9Zo9FI7N5Oawu5JNfSOOF0HaMXSbPZxOnpKZxzl9bY0sbJIkrSD7PZbCKXy4V1RC6k93o99Hq9RHnSaie5uzVW3OKTy+Vwfn6O8XgcFuS537DT6QSilc1mw9pdzP9TIhrn0FrQ1m2yvJw00mk40vscZR6dtEfRKunai5qWOMCUtnETiFNEK9pzDDEsfQ2YczcaPNbX19FoNBIxTCqVSrRNum4iGK9lSD8CeK1WC+teuVwOGxsbibbLMbLGRQMr892+fTuEbx+Px0GcXV9fDxZOtivmpqf712g0Eo7PlUoF1WoVjUYDGxsbIWwgt0BJjqrD+EkHCNlP2SZrXLU+t8qcyvmQBNYi1DF1Z9V07RFPUzo9wEwUnyzA0Osreq+ZpWhbQCz/5e5wmstpOqduJuuxOKxsw3A4DMjHQLIAghVwNBqhWq0Gp+ZyuWwCgnbbikkEHKvz8/PAbRnmj9ZHes1wbCWQx0Q34CJAL79pt9s4OTnByckJ+v0+CoVCQOzj42MAFx4rw+Hw0pzyWhMZOecWh9RJjpElkst7Wac1fquoJ2npiRE1tQIsByq29y7NEyWtfJ3HmlDdBmmt4/NisRjOILAmSgKTNChwD95sNguGECIkQ+HJ56sAnGyrFDUpyq6trWF/fz9wPhpx6C+ZJtLr93Kza7lcxubmJjqdDtrtdkKU1G3leXhsm54TjVyayKbtNEn712OU5nNq5df1rJKuPceLURp5b72X1FInayBlXRZnsigfLY+cBBoKCNzSu18Drv6XC8oMidDtdkNYBLZjY2MjmOg3Ny+7usY4gx5LAlQul8Pm5iZOT09Rr9fDAv1wOAw6HjmdFuusMaODN7nKjRs3grhYr9fDPjwAAalZNgMb6TGTbbaIl+SKWoS3vEliYqnlnRNTbazxvUq69ojHlCZHx8SDmFhqUTb5XCerHCLZdDrF4eFh4HbAxWK31JH09xbVpXGB24VyuRwGg0HgHrK9dHC2yrEIDoFWGyO4eM91PXLf0WgUljakmCnHWPfNe58IUvvcc88FnZESAa2ZhUIhsQUIQIitokV9OfdaP49xQN1WmaxvdIpxTmnRtXTQVdO1Rjw5CRaCpIkgepuGBnrJwXQ+i1NoeV8fY8U2cXcBuZ7VJR/kAAAgAElEQVR0bNZ909dynYuTmsvlMBqNUCqVQhBYAjO3Dum9crKPMum4kZnM/NhjIjnHjToav5fAbm2rkdc8749I0mq1gvGGHity861Evl6vF9qq9eMYwdVtsPLKd3qe9fjEkFSLtXosrsr1rjXixVh+jIPxXYzbxRBVvrNEm1geetQDCGtPFKO4+MzF7tikSyQgx/PeB6RqNpvY3NxEv9/H2dlZQDKKm7u7u5f6LIFDExiGayf1rtfrQczlUVlcBgBwScy0YmiyXh7A4pwLwZo2NzeDZ8pkMgnHgLE93IU/Ho8TYR7SYuFYiBSbVy12azVDwkkMiSSHt56lSWOxdK0Rj8lCQIsbySTfW+KYLtv614imqWu73Q6iBwEqk8mEtTEiojxe2CqH1zSweO/DQjJ3ozebTayvrycobS6XC+KmFJ+tcWC9XAOUBg66ohHpptNp0Cv50wvdunzqcBSzt7a2AiLy9B8eUkInA8lRZdQ1WW6a47EF8DECZ3Fp3X5ZhjX/MlmBt2KqipWuNeLpwdHWJvmvv7MQKA1wYv9M1oRJ/YgTS5M4dT4utK8ymTL8AheXGY+k2+3i+Pg4bCCVrl+NRiOUIUVfS3SSnIxtoVgpOXi32w2+mxYnlePC91wgd86hWq2iXC6j0+mELU3OufDPE2/pPCD1Oz3uEmksqUU/t8RKa06XvbciHsREz5g4HEvXGvEscYIprZNpym5sYuU7rSfFJpkuV7zmkb31ej2cjyCBTX5vARARlwDf7XbR6/Vw//79ALz1eh2bm5sJzre1tZWqA8tnRGR+L302qbNqsVVGPouNZblcTjhw8zhnis+1Wg2FQgHtdvsSZycHlIRCt4H16UV8PrcQTeuocrwlN5f16DATsl7dLjm2V0E64JojnpTJYzK1JVLGxAfrWt9bVJd5rHLp5Mtv6QBcqVTgnAvcqV6vJ76P6ZLcnsOtOtTziJB0yN7Y2AjjI93WLGIhx4dSA8VLLlmwrWdnZ0Efk23TopUeh/X19VBPuVxGpVIJewt5sm02m8XGxkbYk8cyqGPGREPdJ0ss1AgUm8M0cdCqU5anEXSZBJOWrjXiMcU4l0URNYJaYkRMjLxKG/gNxSXqZLTacQ2MbaD5PI1aUlckx3POoV6vh20yW1tb6Pf7iR0K/G59ff1SWTHk5j/3+5ErsW5N3WUYCItIee/D/j3vPfb29uD9XAemfkvPHC5RyIhm+mwEPUYx4LbEvDQVxMpvSQX6uSVmWuL2VZDv2iNeGsLI5zFqZ01WDHiswUujaPyOFJuxS05OTrC2tobNzc1ggaQhxEosm2tpUtciUFerVYxGI6ytraHX6wVknM1mYU0vzeKoJQPn5oYfrhmyzlKpFBCNXI+ERY4XE+uXzs7VajX0gzsUvJ9vSXrnnXcSFlIuOeiYKHr8LUkkDR70T4qdlhhrJb0Lw4INS7pYJV17xFtGRdImw6Jwumw5sFZZsfrlu+FwGIDX+7nPJg96pGGF+9OW9c05h9FoFIIQ0dpHwCeCS99NIqd1uIk1Fiwrn88nuKv3PhHEVu+T0+3mP49RBhBEX30CEM/Bk/2gD6rUh+U8yDq1jmlJIJKAWnpfmhRk9S3WZ4tIW9JMWnrcuJrrzrnfdc79uXPuq86573XObTjnPu3mkaQ/7ZxrLvI659yvuHkk6S87577rqvXFxIeY/C+RLsbpZL7YRGnE1ZSOnh58J/0R6WhMgFxbW4siHH/kFuPxGAcHB+h0OgkOVK/Xw+5uCQQ0bljEhG2Q7ZaO3qPRKJj6pTVTG1us8ZdGle3t7cDFaGxyzgUvHACJ8H1pJwFJa7F2bLe4kIV8Ms+yf1kOcNlFbhX4WzU9Lsf7HwH8c+/9ewH8m5hHlP45AJ/x80jSn8FFbJUfAvDC4vcxAL92lYo0y7cokiVmalElRuXSJkkjpi6XOwjI9WQwIS6EE9AZM8UyVbN8mtqJfEQO6mO9Xi+ETpBlFIvFS7vT5TWNMHL8er1eOKSSLmOaqhP5NMB770Mf6WXDNUxyOwBh4VyeBARcOEXLci2dTeuc/MkxnM1maDab2N3dxd7eHra3t4NXjx4Heb1MLZFJw45u11WijD3y7gTnXB3AvwvgP1xUPgIwcs79CIDvX2T7JOaxWH4W80jSv+Hnvfn8glve8N4/WLG+6LMYxeQz6zvN3ZjX4pxpIgS/GQ6HQdQj4HGNqlqthgXjQqGAarUavPB1HZLrEWHz+XwQXfP5PHq9XthMyglnvmKxGDiLHhe5DiqtcuSm2lQvOTev5Y6I2WyG7e3txJhRv2N9JBrOORwfH4dvWR8Rj2ue0jdUu6dp5Of15uYmarVawjeWY9XpdBIRAVim7FcMrrTbmoaftM3Ay9LjcLznABwC+N/c/NCSf+ScqwDYJTIt/ncW+UMk6UWSUaajyeI0aR3V3GPZoEgxRvo7WiJmrA7vfeIchP39/bCDvFQqYX19PRGKnUYWS7Tls06nEzjo4eEh+v1+qIPIJ5daptNp4FzSTU22kV4r8kfAIxfigSX8hshKxJDzwj18HDseI00dlXm4NCIRiN4qOqyEpR7ofsi+3bp1K+jUpVIpGHMo1lMkZ5LERM55TBLSc84yJMGz8ixLj4N4OQDfBeDXvPcfANBFesg+q1WXWIkzIklbulhMbLAQLjbIUu8Bkn6Iy+rQ5Th3cewWt7fI693d3SCOFYvFhDO1VQ+RjPoSxUg6MAMIW49kIrflLgEAgRPJEBUAgoM0+84FdLnBl+OjjSfFYhEbGxuBKzs33wLUarXCEgvHKZ/P4+TkJOGk7b0PvpkaYC2CpMeKkctqtVpYM6SPaSaTCdyfYQV1ebouzUWtdlltirV7WXocxHsbwNve+y8s7n8Xc0Tcd87dWDTwBoADkf+2+F5GmQ7Je/9x7/0HvfcftORwkc+8lklzkRjVTENWjYBSzNK6HrkUMNfTMplMWM8bjUZoNptB/KKoKNujEZkeHjRCSJGTPwIVnZPlLgMiiERC4EKEkrsraPAYjUbB04b6GMut1WoolUphHZF1s73UX6VBptVqIZPJBM8U1kPPHGu+ZJLmfylG7uzsoFKpBDe1XC4XiB3Hloiu/UBjRDVt3uU86W+s+2XpcSJJPwRwzzn3bYtHHwLwMpIRoz+CZCTpn3Tz9D0AzlfV7xb1pVIgJr1VQ4sDWpdKu9dJikXSU0ROFqn96elpAGBpXOFBHdyTJuvSBKTdbgdx8+joKLhgMfRev9+/1A4iCUVD1sl7OSYM2a51NxnVWXJNhurjYSqsJ5vN4r3vfS/Oz8+DlZJ1yMM3pVFlMplcOh8hTY8DLpBwd3cXjUYj+KpKrryzs4NisYitra3Qfuq8cpysfX1a0pFEWbfFIvbvlqgJAP8xgN90zn0ZwPsB/CKAXwLwA865rwH4gcU9AHwKwGsAXgXwvwL4j1apwEIEHQfFGgRLJOU3sUGW76VYFOOcMhHQaLmTlJoGBLk/j+fE6fUfeU2rJgCcn5+HhWrmmUwmKJVKiQ2xbK/cjCsRn3m4a4ChJOShmLrvRF5uRaK4zHLoHEAkpnGFOiV1X8mp5dpd2jxpPa9arQa9TTqiZzIZ7O3tha1SJyfzYzlOTk4uiaqSSFlzqiUZ/b/sm1XSY8Vc8d5/CcAHjVcfMvJ6AD991ToscdMaOPnOehZDtBjQWwMt31nPgAtqnsvlgkGEyEEPDXm++fr6esLqJpEeQPBWoRgoxTgiAMvgmFjcWK7hUWzsdDoYDAYhHii/k5Gcvb+ItCX/yf2cc9jb28PR0VHQrZyb+6y2Wi045xJ78Mjp5NnmGpg18ZMIuL6+HrZaSYMP3d9KpVLY0jSbzXB0dHRJfJQibZqaIe81/MTgctV07T1XmGSnLPEq7ZsYtdKTwOdygK1FVEv8kGVxAXx/fz+IZNQ9yDUIpLTISXFKlsltNd57vP322wkuRh1Srs0RKSSisDwpej7zzDOJ8BIUifP5PPr9fthLJ8ec72m4ABAcwIkE3NpENzEAifVB9pFOARYA6/lm2t3dRaVSCYgnHQj29vYCIbp3715Cv0xDFr0xVsMB2xubc95fhdsBTwjiWWKBdW99J//1rmaL60nqymTpHTKvrrvdbgdk4a7xyWQSzg6gaEgdpVqtJhZ7ZZl0WqZ1k6EgRqNRcFWjVY/jJJcupDhIIJN7+YigbCMdAHZ3dwNXJbJJv1Pn5gv2zz//fCLwLY0/5G7SgMP+UHyOSRUWguRyuXAACvvCb3Z2dsIi/de//vVgxNnf309VOTR8paWYmpGm7qSla494sUnRg5kmMliDr5GJ+aWoJp9JkQtIim4WtZYnmp6enibO9+Y6k18YBLhMQAOH7De36PBbLqwTqRj9eX19PWFQkXoecBEHlOWenZ0l1gK994k1PC59SCSQp8WS61BP5HjQUklk63a7iViZ3vtLvplyrOW48n82m2FjYyNh0OEccNcGz384PT0NHFXDgWyDNf8aFqwFdIqwkkBbsLcsXXvEY5IIQIDU7+UAWHoY38l/fQ1c1vsskShNGae4yUniRlD5vTTF01LI6NQaMKTZvdVqBV0PQDhNlcF0pQ5JZJGGEcZVyWTm5+KRw0mDCcuWupzkdlwrfOaZZ0JfSEAYFBdAIBB0e5M6nhw3+W8tojOALxFczu8LL7yAw8NDAAh7Hp1z2N/fj3qe6HmNvbPgKTb/Vh1p6YlAPIszWa46UvywxEJLH4vpGJKzac8WizPK9s1ms7AUQEsfF3Kdc+HILa43EQloIKB1kWX2er3w/ODgILEgLP01G41G0K0k56MuRKS7ceNG4Ejy7DrWQz2KZ/QR+bhLXZ5pvr29jc3NTdy4cQPVahWVSiUYk+TyBRE8k7nYtR8T9yVCeO+xs7MT9vsBF0scOzs7OD09DUeZ7e/vB6uqdFq34GCZ6iDhQbvXaSZgwcCydO0jScf0N74DbCujTDHxdFl+WQevLbFC56HONBwOA6chUPOAkHK5jNlshpOTk2BF5Bl5RGrqiTyznOIfEWE0GoUIXoPBALdu3QrWTenvCFwYEfh9uVwOW3coxm5ubgY3ruFwiGq1GtbfZNxQinvHx8fBPe3w8BCFQgG7u7vBCkuDTKfTQb/fDwAstyHpfz2mPHVI6naUIAaDQSAuo9EoiPPHx8eXDHBpel6M8zHF1mzT4GZZuvaIx2QNYMzooSkm32kqJp+liY4xqpyWnHM4OztDsVhELpfD2dkZdnZ2gkvWYDAIrmM8BYh6Ex18yakIbLrvFCuJkMPhEFtbWzg8PAzLA5JLS7GT4iYJATAXW3lAZblcxsnJSdDzGNad/dnY2EhYY7e2tgJ3o3GJh04SQbiYbo2Vnkve06Ai3dioa/JATz47PDwMjgUxlUKXL/MsI9D6fQyRV0lPhKipxUzea9GPeS2xgu8s2V575Vv/j9Jm6mdyYysTEZCxU5hfrlE5d+FFQidjXg+Hw4S3Cb+nSCatj3IpwDmHg4ODRPQyIiXDxQ+HQxweHiY8Xuh6Rs7T6XTC0VsE/HK5jLOzs7B+KRfplyGClSheS2MR/0kEeHouRWdKGmkirIQNK6UhkFZP0spJS08U4ukFVYpfVn4tf2vk0pTPUrCXDWgMaKTxh07D3GXAI6wKhULYfMrNsfyO3vRSz6SO5L3H6elpQBSdZzKZhJATBFL+KOreuHEDg8EAhUIhiLxHR0eJE454pBZFTJYlqX273cb+/j7u3buHhw8f4vDwEM1mM3AjAIntS3puNEGVcwsgnLNA5JP6ZaPRCESGIno2mw1eMjHdfRWVJO197PurIuC1FzWtydHXOt+y/JZISmopn8c2UaYhpVVPt9sN3OOVV17Bt3/7t+P8/ByFQgGDwQA7Ozt46623gk4mRUQaRSSgttvtsO4nY6VIyi/XuqgTyriXXAqgEWZtbS0EU2K72+02nHMhlIPkgM1mM+ywyGQyODg4QLvdRq/XC65b8uwFJnn+HXD5zHYpWmsXN9bFpZjNzU3cu3cPs9kM1Wo1rEVqeNAW8BgXXKbr6XZqOPhLZVzRKaaQL0MM632MIzKPVS/LWmb0kXm63W7QU5xzeOutt0KIBHKS9fX1YBwgcEkglXXLdT2WyyWGYrGI8XiM9fX14DfK3QRERrpyUefjt9RJt7e3wzYerRtycZ1cKJfLod1uY319PXDON954I4Qj5Fl/msPJsSKyaZFbOgEwUbw9OTkJm4+ff/55fPaznw1SkJ4vS6+Lzd8qunsMvq6CeNde1LRkal7rZ9bk8r3mnFo0sAYyTYbXk6rbpJFPOk/rI4i9vzgSmfnlehspvQQgLgPQs6VUKqFer4etMtTxqIPRsyWXy2F7exs7OzthUZre/jTQvP3224HzSe8X7+envVJnpc8l3cem02nggnJMpMN4TMzk2PC/Wq0GQiW5IzDfJFyr1ZDJzE/j7Xa72N7eDnm0UcmaJz1Xul26DxbsyXeWypOWniiOp/UEPRBpVGxVkUBPkGX9XEWe1wg/mUyCD+R0OsWrr76K559/PhhNZrMZ7ty5g9deey0RNl0iI/NxrYphJHgm3cnJSdiaQyskEZzIur29HU57ZaBZGkAajQZOT09DHySysxwaaoj0cjHfORfWLalzSj9HuetBzps19jSsSCTic8YtzWQy2N/fh3Mu4TxucR9r/qRou0xVSUtpZcTSted4wGXE0YBvUSn9Lk0pjtUT42Y6vzYe6Pf88dxvILnBk7pav9/H888/f2nLi/RE4bNcbn6g5HA4xMHBAbrdblj0rtfr4YAQ5y6cn7k+x50TXHJgHudcsLKyr7JuLgnkcjl0u10Ui8XEzgb2k5GvpfFHBoSyDC0yyR0I0lHbex9Cxe/s7KBUKuH9739/glNKI5SeZ46fJqQyYG8MNjQMrKoXxtITgXhMGvksz5FliKQ5kf4uDcnlc1JvqRtK7mhNvPcXkaK993j99dcD5wAujup6/vnnEzqVDBEo63/48GE4xfX09BStVgvD4TB4lbAMliNdqiaTSUCcfD4fAJrUm+3hPQkAY5mQc9M1Toq3wEV4ifF4nEA6iXiW5DCbzS5tsmV76NHD8ZhMJnjnnXeCMwClAQuRNHGU3FEit/4mBgMxxF41PRGIl8bJ9HttidSyt955rOuJ1RXTS+SEaA8Hi2Ken58H4OO/LHswGATOR48N55IHn2QyGbTbbZTL5bAQ/x3f8R1oNBoYj8c4Pj5ORBqTrloMWMtz8XjOA8VOAMHhGrgQdclVWq0Wzs/P0Ww2E+e7z2bz8HpaV7PmJyZVeO9RqVTC+qOM50KDC13rut1uWMBvNBphPFm3FE+teZZIFRNLYzqifEfiZHHutHTtEW8VXSqGhGn3epC0AcDKoznhqu2V/p7OXfhqeu/xxhtvhCOKvb8IKtRut3Hr1q2EZU9yL054qVTC0dER/uiP/ggPHjwI9TnnEqKa9xdbfIrFYgg0xPPWaR0lhyKCSL2Mrlp7e3s4ODgIa2lEvl6vh+FwGPbfye+05w3bJBGL/WG7NQI1Go0g2vKEobW1NbRarSBJaD/QGLfScCBVAtm+mBpjwZFetkhLjxtJ+j9zzr3knPsz59xvOefWnHN3nXNfcPNI0v/YOVdY5C0u7l9dvH/PVerSlieLourBiVEgizrFEEZPUtrk6fKkUUHmoUtVJpNJuD0xn+QiW1tbCbFL5mu1Wnj48CFmsxl2d3dx8+ZNNJtN3LhxA1tbW5f0Qoqy3s9DKJB7NJvN4NolQwNKil4qlUKksnw+j5s3b2I8HqPT6WBjYwO9Xi+cNCTHmFxUliXnS86FXPYgIZD6Gg/m5H5ELp3s7+8Hsffs7OySNKLhQXO0ZWJiTNqRcHJVPe+REc85dwvAfwLgg9777wCQBfDjAP4hgF/280jSpwA+uvjkowBOvfffAuCXF/mWphhQW4OlZfi051rciNWtuZzFXVfhsrq93KnN5/SZlJ4jjUYj7O6We/CkeOOcw87OTrimjkWklTqS9plstVpBPKU3TLvdRrfbTSzAk/PIkBatVgu1Wg27u7vB0DMajYJDtER4SSBljEs5Lt7PwzroGDEyj9xkm81msb29jTfffDNwQB6HzZ/Uw+W8WPMam3sLOdP0x1XT44qaOQAl51wOQBnAAwB/FfNQf8A8kvTfWlz/yOIei/cfcsvkSCRlad7HECytDCAZJUw+T1P2JYBrRF2Fo8by9nq98P6dd94JgWpzuRzu3LkTvP+5DUhyLblBlUsLNO1zx3ehUEC324X3yUNKxuMxqtUqbt++Hc6q6/f7IawCT5eVYpNzLuib3HXQ7/dxeHiIw8NDnJ+fo9/v48aNG8F9i22TOrYkGhIxODY6epnM/y3f8i1hq9VsNsP6+npANvqunp6eJsZeGonk3GgCGiPuWlKxVJAY8V2WHie8330A/x2AtzBHuHMAfwzgzHs/WWST0aJDJOnF+3MAmyvWFVVul11bCKCRL8ahgDiypnE7Egr5TopgAMJRW977oBM1Gg0UCgXcu3cPzs3Pxrtx40YwOEinZ9bDfXLc0UATPpG12WyGNlB8GwwGeOuttxJBX7kulsvlEkd1yfHn816vF8Q9inyDwSDoXNY4kEtpAxSvm81mcMCWMWnYho2NDfT7ffR6PRQKBWxtbeHFF18MllIZws9Keu0wTWKS768696umxxE1m5hzsbsAbgKoYH4wiU7sjdWqS6TGqUjSmuUv8sj8CVFCUlhLmbcU4BjnkoChKV8M2fX3RH5JPZmXIh/LoysXrYN7e3s4OzsLUcgIvMAFIM1msxBvk6Jgo9EI/pnSV1ICtnMuROAix2TsFwbjlfVwXa5QKARxkmXKHfSMc0KXMT1uEgEk1+aCOQmMdHh+//vfj+Pj4yDC7u7u4o033kicbNTpdC7NQ5rhRHM96xnvpU0hVoa0OaySHsdz5d8D8Lr3/nDRiH8C4N8GsO6cyy24mowWzUjSby9E0waAE12o9/7jAD4OANls1svJYbJERN6niYTyneWcayVLSSeCL6N4snxr69FoNAphEzqdTgBgeoP8+Z//OQCEJQCpb5GTcC8eJz6bzeLw8DC4inHbDzlir9dLnP7DUHhsE8P0sSxJONgGbsxluQR6Eg7LW4XPubYoz8ljWD69E2I2m+H27dsYDAbhyGv6sf7FX/xFKJ+6pd4mJQmfNY/WXMl7uY4pESwmPb0rxhXMRczvcc6V3bx2RpL+AwA/tsjzESQjSX9kcf1jAP6lv0JLJcWyRE850DHxMHatOZiuM8Zp0zidRU11Ge12O5yRAFwco8VARDs7O2g2m2g2m+HwR8npgIs1vXq9Hnwv19bW0G63Q0gE1jubzYJoSSCm+b1Wq2FrayvonuSs0h2MBh6egMS1Refm4dtp1fTeJxbMpUHEOXcpnOH6+npYyKd+N51OA+dm/E+Kn2dnZwAuvGE6nY6px/Ga42QZ0+S8SLGY4yWNQzKfhgNd7rL0ODreFzA3kvwJgK8syvo45kdy/Yxz7lXMdbhPLD75BIDNxfOfQfoBJ7ou85nF3dK+T5PN+Uy7FFn/mpLqMq2Jt6gkt/EQMCeTSRAr6e1PROr3+2G/nAaOSqWCdruNTqeDUqmERqOBra0tNBqNsD7o/cWZ5HSqlkFsybFkH/T6Gz1XqHsxUC235EgCJmNecnMqk4wpU6vVwpkSEjkajUZwPZNrg81mE1/5yldCfVzSiBHX2JzEOJg159qTR+8LjRHutPS4kaR/AcAvqMevAfhuI+8AwN9+hDoS9zEA5zstZ1tWrDSEjXlWSGSzytFISGqpxRz9Tyuh1Ie4vsfDHaXoxbUrrXsw9MLR0VHYFrSxsYHZbIbDw0N478O5BgzCJCl3rVbDG2+8kWifHnsZzo863traWtixzs255Gw8xIQhCem1Q3GRyxAyHLz3PmyTGo1GCWcDrnlynx8dz6Voqec8JtU45xLLN8tERq0uaInnKmIm8AR4rgCX3YosMZP/GkFiXE4bWdIUce0WpJEvhryWXqevCUQsS1JV731Y9+JOde5Wl4BVKBRwfHwcTP505+r1eonw6bPZLGFg4X4/7+cHgZydnV3qG8dJesAMh0NkMpngiM2w6Uxci2M9DPvXbDZD8KKtrS1sbW1hY2MjGILoILCxsYHpdJoIxc7juBjMiYjHeDV6zjXBteZvmcVazzHHQzsJ6O9XSdd+WxBlbM1FLOqTZoBhWUz83kJk+V5fp4kxyxR43Uauh0nKy/pkOAPp+EtxUHJJio7e+7Awzx0EZ2dnYRnCex+4qOTEmUwmcfCJfMe+M+QCcMGlyYGIZDwck0luC8pms8EdrFwuh3mVbXNuvjuC3K1cLof+DQYD5HI5vPzyy8GSSfFTj7205Orx1/eWRBQj7JqYxvKvkq49x5PWNE3VNKu3uJUWIfQAW9/o58smRrfPEkfkO91u7y+2psjF7p2dHVSr1bC5lMdPSSI0m81Du9NlijvKh8Nh2OIjCQxFQUntieQSYCVSSgQhInG3O3VHyQ1YLtcYZTBd5+brk41GI3zjXNIhmtGx+/0++v1+Ys5prJH6sORkclyXOWSnEcwYgbUI9VU4HdO153iWrG6JEZZOIsvQ+SQwWsioy05rn5VPc1NNkXlNzqUNGRRBCUDPPPMMzs7OwlKDDIjEPXJcO+Oz27dv46WXXgr1APPd2/V6PWwfopcLt/cAFwdW8poir0TY6XSKtbU1NJtNfPGLXwSQ3CzLdmcy821NPJWIHif01HHuwtqZyWTCzggZoYxcXI4Td8GTMEhr7zJ1wUJG67k1txYsabVmlXTtOR5TjNLIAZZyuP5ODrD1L8vU/xa3ZYot2Es9Qn5jIb3eq8Yyi8UiRqMRCoUCHj58iH6/H85TJ8BS95JbaeiRIrkME+OqcP8cTfQ01xNAWS6BmshAzxiKwHK9T3JLqbdWKhVsbm4m9gNSN2S5e3t7uHPnDvr9PkufXdMAACAASURBVI6Pj1Gr1bC9vR3E2dlshoODgzBmcn0yJuFY86+TJRFpiYT5lomYVxE5nxjEY+IEp3U6TQ5PEyllHbpM75NbVGR5Vlskd9OJZQEXm19ZBvUXUnVaEGmer1QqCXFPlk/OQE5CE74U1xlZjHqejNtJZNJ9kEjHurnYTe8R5tUIW6vVEjsNtM8py6nVasH97O7du1hfX8frr7+O9773vSFMBZdGptNpOObLmpPYnMnnWp+Vc289055HjytuXnvE09wMsNm8TpYuuMog62/1c/3OKs/S8WJ9m81mQfSSiNTr9YI/Zy6XuxReQepT2WwWp6enAcCpx7EsckP+yOmYj+WzLxKxNZCRAEwmE+zs7OD4+Diho1EkpG7KjbzAxYlF3AbFBfhcLoe3334bvV4Pzz77LLa2tnB+fo69vT30ej3cv38/6MAULaVIaY2rnkv5PEZoY/MVU3WA5Jagv1QczxLfrDyAPbDyX5uBLTFl1bqsPDEqKtvBa+bJZC4OcyTyERElYjKIkOT4kpr3er2wjgYkrZ66fr4n0pHjSWDSnES6mfFdJnMRKlCOqUQ4eVgJcOGdQ6Sj8YUL+1/5ylfwhS98AdVqFc899xxOT0+xvb2N8/PzgHQcG00UYgRWjrmcJw0jeu4sIqvHUYrUV+F6T4RxBYhzlmXcTw6edLOyFGZdL8vV+aWIKingsomygIQipXxPg0a9Xke73Q5cTEYck8jHbxnOgWIeOZuum3qjRGYdSo9jFVuLdM4Fq6NzLiwx0M2L3xKhZFwT5ueuCiJguVzGcDhEv9/HK6+8grW1NTQajXDwJTmxPvTEmm/9zJoLiWCcCwkTmshayP6o6dojnkzWYMh3Erl0fv2dRCirPJn0pMlJ1xPFNqStI8lJl1ZN3db79+/j5s2b4STY2WyWOOtc5iVXsfRbqz9EPIp8UrcDLs52IHeTXIncjMgwmUzCXjpNgCRSk8hIkbFcLqNcLodI2KVSCc1mE8453Lt3L5wbL4kcLbByrGNzFJuLtDnle8u/U8KIVe+q6YkQNYH0xU4JsJbMbYmdy0QPayJ0Pbod2poqryUHkXWxXItzExDZHrnfzhobevDHCAPHhsgDIFg/vb8wtHjvE5ttuZTAUBH8pz5HbiVN+nL3geaikkBWq9WE2M/vJpMJbty4cWmMuF4pjVNyDPT4Ww7Nelwkt7O4YwyGHofzXXvEk4MRE+vkIJFzrLKkIOvgv/wuZj1dJpZYIqwEFD3pABIBWaUuR/M/19wAhAV1GQTJOReiiPFbRgHjPfP1er3gRTIYDIJZn/qh9CbhGBDRyCnr9XowvtDrRY6BDE/IcZPirxSNWT8NMrL/XKSX86sPndSwYsFF2rzHRFYrv4YRDZurpmuPeFoOl894rTttsX4N7LF7S663ytSDnzbplkgj2y4XnCXlBYDj4+MQWYuLyZITSHGLuxPIHeXiN/PLoLTSR5KcT9Yt42QSkXjCETDXFbnDXX5HBNPjQi5HAjKdTlGpVMKWpG63G5YJiGCbm5uB00lRdRVxWs+nHHM9XzJZnFF+q+c5BiNp6dojXpp4YN1L2V6LlMxrDbglYlj5LAS1qGvsmUZmzfGsdaf19fVwDtxsdrENR4qPDFRULpcDN5RRumRe+j1StOOuBQAJsVPGPtHn9nE9kG2Wx2NpQ4peBuF7GSCXB7vQRYyeLvfv30+ImVLUtsbZgg3txKD/V5nLR3mXlq494sVkbr6LIYoUUbVIaCFSbBJ0XcsmwUL2tCQRT9fP9/TS0M+1QYZuXEQcIhTvpSgr1wW73W4wjAAXuwvkeDHiNZ9JayMtrkQqKQ2QU8mDI9luBqI9OjqC9/PF/XK5HDba0gdVevbQL1RLKTHuYxE7mfS88pmWSmKczoKbVdK1RzwL4dIoV2yAY9TIQjRLTLE4pTXYaYQCuNDxtDMy1/B0ud57vPXWWyHas3PzMxho+WOZ5D70duFSAQ9vZJ20osr+dLvdYOTQRIkIm8/nw5ocN6kOh8Ng/QSQsF5arnTScur9POQD1w95nns2m02Ek5eIOxqNEtuP9LytmjQX1Eiqy5TcOpbXspympWuPeDJpgE/jgvqdBgTrWtejB9MSW3WyjCcWNZbt9N5Hjw8G5lyDUbhI9bl9RiO65G6DwSAYPqjPAQjXdKx27mIhm8gmjSBE3kwmE2JbyhNtqdNJhLMiQUs9llt97ty5k4hmTeJBhJTn7HGxXs+DJSrGiCffxTxftFiqYUCWK41vq+idiXpWzvlNTMu4laVAx5xn9YSlcVTrvfZ+0YgVE2Vj7ZYcz1rLYxmMJEZXMEZVlm0lAki3Lnr7S88Tum0xv9TXZPszmflO8Zs3bwaLZz6fDzskWBZwsUdP91E6LbC9XDzf3NzE/fv3EwdtMmgRtwXRi6fX6+H4+PgS0bUQTYuEsk0W4ZR5LGu1BQcxkXbVtBTxnHO/7pw7cM79mXi24Zz7tJuHaf+0m4f6g5unX3HzMO1fds59l/jmI4v8X3POfWTVBmoZ2xIBmC+NEspv5DsNIMuoVozbxsROq04tIsrJ0+tf/D8/Pw+xMrloLUP9Sc5Jb37vPXZ2dhJGFiITvUa4BYgOyjSkUHyaTqfY3d0Nx4sBF2fXyX7I8ZBIrC22JADf+Z3fGdrFeKIsg7vtHzx4ENbt5E5za06XcUKd11oqshBVSyp8rpH4X4eO978D+EH17OcAfMbPw7R/BheBi34IwAuL38cA/NqiwRuYx2b5K5jHY/kFIuuyZFEaPpPIYrmBaRHPUo51PTFKanFMSRFl0hMiy5AERCO/PL9bt2d/fz+sdwFzzrC7u5voB9fl6Kol/Sidu1h2oIVUcslWqxWWBphPGkK4eM/jvWTbpG5HvYz9tFzPMplMOPiELmfcAsVyvvzlL2MymWA4HKLb7eL8/Dy0V8+PHkdr7jUMxZ5ZoqUFP5oRfMM5nvf+/8Hl+JcyHPsnkQzT/ht+nj6PeYzNGwD+OoBPe+9PvPenAD6Ny8gcTRayAbYOZnEiy1tEfiPzxqikhUwx7ifz6WtNCOTkSZM835PryAMt6VjNNS6JyOSEkqsxqhc5ZLVaxcnJSdClstksWq0Wnn322cRiPbkfRT9yROdciLsCICCMFMPJzYiMROLxeIz3ve99OD8/R6/XQ7lcDuEgnLs4W4FcmAdvyvLlmFqEOTbu8r3MIwmM1tlkfvmNlDL0XK+SHlXH2/XeP1hU+ADAzuJ5CNO+SAzhHnt+KTkVSdp4f+lfi6OS2qYNXmzQ9HOLouqJi/TFfK7fS44kxTG+4yRzUyjzcie6bA9P2slk5nvlTk5OwlkMwFyMazQagYPQCtput4N3CjlcJpPBnTt34L1Hv99Ht9sNAYa63W6I2Sm5oxwTC5i997h9+zbu3bsX2sv9ddlsFnfu3MGf/umfYjAYhKO45DqjFsE1Z4txOT23MZVF5tHwYuWx3q2SvtHGFat2n/L88kPvP+69/6D3/oNWRy0OZYmUWlSUyj2TtVjNOixxLza4GrFjbdTPdJKhDWQ/mJeh1ckVxuMx7ty5E/pC5KKhwy10Q1oLZbgIWiG5ID0ej8PCutzhvr29HfKyHeRcjUYjIbJKMVAac9iH6XSK97///Tg6OsJoNApnmbOMYrGIr3/966EPg8EgRLaOifR63GNIo5/rMZbza8EK+63LkuVfBfkeFfH2FyIkFv8Hi+cM087EEO6x50uTHpQ01q45UoxCybL0thdtWYxNeKydQPI8PEtXjFFlbkaV+aWYfHJygnK5HMofj8dhX5t0N+PpOVx+4F492SfWwcBCs9kM7XY76HlSzGXYv2w2GyJN04OGdbPt0gNH9ovt29nZwb179xJWXPpeVqtVfP3rX8dwOAwipuXjGptzPR8xwinnxELWVWBMlqfLWSU9KuLJcOwfQTJM+0+6efoeAOcLUfRfAPhrzrmmmxtV/tri2dJ0FRlaD/aq3zBJqm29l/exBVjrJ/sS0yWAi9ANfC53E9ATheKdXNu6fXtO0+gmxm0z/X4/WAv39vYAIJwGy9N16KfJ9bm7d++GcSBiAXPkIYIwLgrbpgFZ9pnLDtPpFN/7vd+Ll19+OXBk6f61ubmJz33uc2H5oNvtJo6TXjaHlmQRI9YWMWW70xbCLQ54FfiUael+POfcbwH4fgBbzrm3MbdO/hKA33HOfRTzMxQYIfpTAH4YwKsAegD+7qJRJ865/wbAFxf5/mvv/aUDS1LagEU5l54Bl2OsaA7DbzV103Xo8rXDMp/HxFNdjgWIRChrvUh6lPAnkY8AXCwWg9eIcw43btzA/fv3AVxEe5aGmrW1tQDs1WoV0+k0GDUkISiVSjg+Pg7rgLdv38bx8XHCMkkiUCqV0Gq1Ep4w/Mk9cOzXzZs3Q5BdckUZboKEhAhOg4ocH2suNCzIJMdQ64YSBuRzLYLKsdfwpMu8iqi5FPG89z8RefUhI68H8NORcn4dwK+v3LLktwDiXIyArJOmxDEEjXG1mMhqtcPSFax+6GUPXabmhtqAwnMC+A4ADg8Psbe3h6OjI3S73RDmnScP5XK5xMEerGNzczP4SXIceTAk28pzGyQQ84BLxl1h9C/JneX4MYrYm2++GXwvaS0dDAbY3d3Fiy++GBbmeaCmxb2suZKL9BYyWXNmEV69l9Ga1xgcXQXpgCfAcyWtQ9JUrQdKA5nkHsBl1yBex8rR7/kudq/rtvqiJ1Sa36WHvxTnuJtgNpuF0At0UqY4yn14mcw8etfJyQkKhQIajUY4WNJ7H85UqNVqyGazqFQqwYL4/PPPo91uJ9YOKZI658KaHs9FZ/ukJwuXQN73vvfhnXcuVHrqnuPxGPl8HuVyGZ1OJ/RNcjtrHOX86Xm1pCM5nzFR1CLc1rxpV7M08TUtXXvEiyU5AGliJgFaOyXHBknL+hoAZNLcScr8siw9cbo+WbYmJnIbTSaTCWHLAQR/y+FwiMlkghdeeCEh5vV6vRAEdzAYhAMvnXNhJwDLXFtbC36RcjsQF+OJLBQTuTueYqZcUpDrjh/4wAdCoCISCiL3bDZDs9nEiy++GDg4iUoaoui50BKDJmjWOMvrGIeTcyzLloasq3I6pmuPeHICrAG1Bo+DoxGD72JJi6bSoiaputUOzRnlv+yD5op6kgn8Mg6L/tGPkUjBNbvxeIybN2+GcmQ8SgBhuYAWyV6vh3w+Hw7HHI1G2NraAjA3oHDbDkVNGkQ4ljTibG1tJcaZW47u3LmDBw8ehDPSyVVlmPpMJoPT09PgNH1ycmISLMuYZc2nhZyS0C5TETQMAJcd7PUcWraAZenaIx5gm3nlAAGXKZpGQvkuhoQWB9PvLV0jhsyaulpip+aQ5BTMK5cTnJsfw1UsFsNCOr1YGBuTfo50DeMCuSQc6+vrod5erxe4TLVaxfn5eUBCckwaZ5xzwV2M24G4i0CutRWLRezu7gaDDokDxWO5m/2LX/xiWP7gya8a4PUuAGtOYjq+vLb0OotIxpAxliy9fVm69oi3DKhjOpjMF6NGWiRdlTIuE1N1mdbkWWUQ8bjGZfk5OncRph2YL6rzZKDhcBiWB8rlMvr9fnCYpkGEgXL5Hc+hy2TmJ8v2er3AKbk8wRNY5VhS9+v3+4nIYYVCAdVqNSCTRAzqh9RFi8ViOH9dHkIixyM2F/K9vk7Lr1UIzeEsDrqMyFrEeFm69ognuZWmRFqh1eKlldIoU8z1SyOjRnZL+dd6gcwvv9Pl89gu3ksxRvupUh/j+h7jb5ZKpYB8DGrUarWCxbJUKqFUKgXkunHjBjY2NnDjxg1Mp9Ow5YibX1kWY79wnAuFAorFIlqtVuBi0+kUzz77LGq1Gur1enD85tpcLpfD2dkZ8vk8Dg8PwzIC9T89Tpa6YF2vQjBl0upKjMNpNUHnkarGVUTNax9Xc5m4oJ9b17znuQPUWbQsb+0nk/VJkcISc602xdpm1Q9chMajyZ15vfdhr93h4WE4vIRLB8BcbKTTc6PRCLFMmIcxKs/Pz8OhJe12G4eHh4ErcZMs9cZSqYSNjQ0ACCItPVnImaXzdavVwksvvYRv/dZvTUQ147ojD87s9Xo4OjoKBpter3dpXDUMWLBgjWEsWWXHJJk06Uk/0+1cJV17xJNJA61+LhFNTkitVgv5MpkMyuVycH2KIbPlHxgTYXSygELrd7qdskzN5eVz3tPVazgcJkS4SqWCXC6H/f39oNfxfzqdYmNjI8TBpNM1F62px52fn6NarQIATk9PcXZ2Fg6+5P64SqWCTqeDfD6PWq0WghNtbm4CQFjmkHFg8vk8zs7OgiGI1s/hcIjT01Nza5eWGjiHliShkcAianxnLQtoXVK3wxJDl0lYsXTtEU9ziTTg19wpl8uFKMlykLgwLF2SNFJIsU4PvOZ0FoLpbyxuJ6/5jgYLbdWUyyEAcHJyEs6ao0GlXq+j1WqhWCxie3s7wZFojaQ+RgdnfjedTlGv1/Hyyy+j3W6H2Ca3b98ORhq27+DgAKenp8Gvk+t5s9ksLF1sbm7inXfeCRbhhw8fBoJHMZP6bL/fT4hx1lhaYxXjVlZezcFiroGyfkl8Nde1nl8FAa+9jmex/5iyS6rP02gABIOD1v/kDmpd3yrirazbEjcs/c3yA7SQmptZgQvRk9+zj7PZDJ1OJxH38vDwMIiQm5ubeP7553Hr1i00Gg1sbW2F+Cu5XA6np6fBfYubUR8+fIgPfOADYYybzSZqtRqOjo6Cu9fx8THy+XzYhCsPKPHeh7W9t956C7PZDN1uFycnJ7h582bQ8Q4PD4NjNbldTGSLjecqYqm11mYhkUyaAOt6LK5qtXtZuvaIB9giHbkSEYqLvARKDrw8mIPl8LtKpRLuZT0a+eSmR2uitciixUr+r+qyxlgj8oRXLozLU1cBhIBD8qyDZrOJs7MzvP3222i1Wjg+PsZoNMLGxkYwwjCaFyl7v99HqVTCO++8g9u3b4eF9f39fTQaDVSr1bBhFUDgdgCCdfTk5AQnJyd473vfi7t376JWqwUHgP39ffR6PZycnATRllZWGVYiNuccY7mmar2XY2MhWEyF0Iim36XpfFdBuNCOK3/xLieLA8nna2trYd2Kz+gBQeBtt9tRzkn9j8814qTJ8DKvVb5GPvmNBRSyXXTJ4jf8p+go980BCKIiANy7dy/kWVtbQ6VSCQhDjklnaS6Ky3AQnU4HW1tbQX9st9thBzrX4bgcQPez8XiM3d1dHB4e4g//8A/RarWCQYU6HS22tGRywVyeVKQBfRnAx5wbgKRbmSUpcVy1E3Vs/q051s9XTU+EjsckORf3oQFJdysCq/W91h/4jmtcOj9TjONKTiSf6W8sBNRttHRM7y/iSkrDgKb6FEMZY4WGFxo1aAjh2hvFS0ZuZl1cjKfFlAdMUnejpwxd1egjymOVG41GcIR+5ZVXUK1WA/d2zgUn6OFwGBDPEu1iOpUmYLEUk1zkeMfmYJn6wHyWIegq6dpzPOCC6mSzWdRqNaytrYXnwMWWEq5/ye8ABAsdgRa4ODKYgMprWZ9Mljxv6W1EGqv98ls5YVKZ5zPqX7IsIrlEPBqQyNmKxSLq9Tpms1mIuzIYDLC3txfGh0jVarVQKpVQLpdRr9fDSUCDwQCDwSAgFHVJLi+USqXg31mpVDAej9HtdnF2dpbYpMvIYKPRKCwd0BLL3eVyPDTXsgiSHr+0ZHFKqz6LMEqxNWZIiel+q6Rrj3jez83Q9Xo9RDvmEVNUznV0Lup81Wo1gXQSwCWXpPhlyf781/qZLJN55ARJpNRiiVWOrM+5izia5OLS9UpeA8kNtJlMBsPhMCyC02Ok0+kAmBMh6XRN75bz83N0Op3A1Simv/zyy3jhhRewsbER2kl92nuPbrcbdGlaOumrORqNsL+/j3a7HWKo0Ema3jB67dQifnpOYu5hcvytMU0TW62UxvUsznwV5Lv2oiaAsDtaT4ZGBiIcF8q1GKApmpzAcrl8KTy4RQHTBjdNXNHPtY5hLV9IDi5jmPA7Gkn6/X7Q56i79fv9oJMBCOH5GFaBhhXuf6vX6xgOh9jY2MDp6Smy2SzW19fRbrfxZ3/2Z+FkH9bZbrcDd6Rbmfc+nF1HdzFyP7lm1+l0wv5AKarLObGAW46NhAV+R6IqNxRbzutyXiyPkzQCG4MBizCnpScC8TTlo+8h16QABDO6Fk30pMp/qU9x0tKoaZo8L9/p/YH6PZ9r4NJARqOEjGPpnEsEnKW4RwTsdDpoNBoBAIlwFNXJ0ShJSLGdnBZAwjhFayl1RudcQGDuQmc/6TY2Go3CojwDF3W7XbTb7UQAI0tX0tzLmgeLk5FYaTExbc6s8mSy5t0SMaWqsEp61EjS/61z7s/dPFr0/+mcWxfvft7NI0n/hXPur4vnP7h49qpz7ud0PUvaYD6r1Wqo1WrBWwO4WCpgnpiiLQ0YRBYuL+hBtUSKZZyP31mmb5lPczp53W63A+DqfBQ5eW64cw7b29uoVCo4OjoKVk0iHB2fO51OwnOEZv1utxsQjBZMKdZ774NEwOOge71ecFDo9/s4OzsLoiTbK30x2+32pZAOWvyzuI8leWgEXWXO5dhdhTtZsBNry6rpUSNJfxrAd3jvvxPAKwB+ftGA9wH4cQD/xuKb/9k5l3XOZQH8KuaRpt8H4CcWeVdK2mjCpI0SBDI9qHo/1aKtpjijz3bTIoZFhWW5uh4t8kqkkRRflsUkOQbLkUck0/uGXA9A2Lnw4MGDBFeUwEYfUK5xNptNNBqN4JPJDbLyxFnqymdnZ8hms6jX62g0GmHJwBKN9W4I6cZmzaMGYC2RWBxHpzTJhIQuhpDWM1mf1h+tOlZNjxRJ2nv/f3vvabf/PObh+oB5JOnf9t4PvfevYx706LsXv1e9969570cAfnuRd6Wko01JqqW5j7XDW052GnXyfu55wWsLYXmtKa7OI9eWZNuZT06+bp9sN03u0pJJQlOv1xNGoWaziXK5jK2tLTjnsL+/HxbX6Q1DUZVhHLhzgXvh5P46tuUDH/gAyuUybt++Decc3nnnnWCR5DoqicLBwUHwtqFBpdPp4MGDB4nxl+MXWxTXCCivNVJY/9a8yfG35l++j+WxEFfP8bL0jbBq/hSA/2tx/a8lkrSMrKwnjPez2QxHR0chKpamUPp7y1eP+STXS6Osy97FuCHriyUJVIz4LMeBi+jclgPMdau33norGFm+7du+Dd7PD308Pz8Px3sxWjRP7KnVauHIL+q4DGRbqVSwvr4ezmFvt9u4desW8vk8hsMhHjx4gLOzM3Q6HbTbbZycnMB7HxCYHO/8/Nw0hElE0aK0Jq4xKcPStWLqgIWE1vxY3C+WliFzLD0W4jnn/isAEwC/yUdW21KeX36oIknzJ5FhUXeiszs7O+Hds88+m0BIS5SLUS3plhUTK6zBtspKU9hjnFJzY+dcsBRKpOOWntlsFiKBbW1tYTAYoN/vo9Vq4e7duwGguROdO81poGm32+EkIu69I0fMZDJBxKR3S7fbDbvLnXNBP6TO6P3F+XbcYyfPtZPjmsbx5bhoy2Ns7PR92jzp/xgCWe3Rc3sVRGV6ZMRz86O2/gaAv+MvWvQNjyQtgdRaFyOgHh4ehm++9KUvJfS/mOyuPeKZePaARk4LkWJAo8UpixpqcZR16GdHR0cJiy3drniSKy2PMjwDOQwjiHEvn3Smds4FrklPEnqlHBwchF0OJycnlwIRbW1tJXbKc7GfCEir58OHDy+Ni7XUY82rfB5TF9JEVAte9HxL6Uj/5HurPImsV0W+R0I859wPAvhZAH/Te98Tr34PwI8754rOubuYH9f1R5gHsn3BOXfXOVfA3ADzeyvWBe99Yv+clUc/JweUepE+T06uIQFJakfA1IOq/+XE8Ln0dpAEw+pXjCOznd77RMRlLqZLvYgbYtfX19FsNrG+Pjcyk/NkMhk8++yz2Nvbw7PPPoudnR20Wq3QN3q5UOcjtwOAVqsVAiBRn+M55DwOWhtVGK9FHq2lDUrWWGqOsowbWV4l1rVEUqu8NO5nEUY5X2kia1paZTnhtwB8DsC3OefedvPo0f8TgBqATzvnvuSc+18WFb8E4HcAvAzgnwP4ae/91M8NMX8P87DtXwXwO4u8qzUycxHuWwOnXIO7efNmeP7Vr36V7U+IqhS3Fu011+2895e2DcXED2ti+FzqKDpJ61qMK5IweH+xPkWvEvbl8PAw6GtcQ6NYyPJv3boVdgQUCgXs7u6iVqsFx2qGddje3gYwt1zmcrlAvFqtFoD5DnfGWKHblyRAtGTy/+zs7JIoJu9jYrx+r5FNphinsbiXPLdPElUttlqSkG6/7ktMdYmlR40k/YmU/P8AwD8wnn8K8xDvV07sPI+GIpUGkh7otH4ySbFGDzh3R8vDPOjDKLmhRJAY8sXuY7qbFlXkey3Gsp+DwSARx5LExHuPmzdvolQqhZDoRM5er4dbt26F3eHj8RgHBwfY29vDzZs3sb+/j06nE2JqttvtcNgJ1wc7nU44c28ymYTYLSRg5KoUNWnNJFeUc2iJhDHCZH0Tk3Ys5NBcS77nuKYhjJZ2LGRN0y+XpWvvq8mO0xNDDhqQpGy5XA7PPfdc+HZ/fz/hyTCbzTAcDvG1r30Nx8fHCYpsiT9yK43+18+sdlvXTJrKArZOQq45GAzCbgBywExmHk6dyEGxkgjl3Hx7UaVSCRGbaZ1st9u4e/duiJ25u7uLfr8f/GK73W44B4+WU+/nO9eJdL1eL7RPHiQpj07WSyBSJ+Oaq14qSSNilqinEW4ZgbSQ0RI32Z4YgdRz/A31XLkOSXaQBxjKd6T+mUzG9Lf0fm7i/uQnP4nj4+Owbw0A9vb2zEGTAy/r0RMj26cnQwMRAUy3T3NAS2+QIhw5Nt8Ph0P0er1g0ie33tnZCQRrNpuhXq9jZ2cnbGI9z+hxdAAAIABJREFUOjoKwW4l8B8fHwfrrgzPsL6+jkqlgnq9Hty+2C4AidAPMSlBinNpHC1NjEvjkjGDivwuJsZaHFlzU4vjaZF2lfTE+Gqyw5xkDp7ep+a9x927d/H6668DAA4ODqLlfvazn8WDBw8CF9GKOT37u91uqMOaKMsoYFFlLbrob2SZsg4iGi2FlUoFzrmw546bU9fW1sIuBEYLoyWS24NoZOh2u2FLT7lcxuHhYYgy7ZwLrl/0YqnX63jttdcC8nMtkOPGxfLJZBKQ0uqPVA10ny0fV6sMmWL7Ia1yrOdSstAwYCUL6WOiZ1q69hzPojhSl5ODzokjosTSpz71KTSbTXz4wx8OTsLj8TiEPCBwSqshYMfdtAbbotb6J5cS0jin7D/3s0nTPXVThu8DEJYNaISheN7r9cJ/LpfDYDAI3i+DwQDVajWs53FrUTabxf379/Hmm29eMj5IJ3W2R4ba0D9rrKShS1sdrWuWIzmmxSlX0celmGsRXpnH+la3Kw1hdbrWiGfJ/M65oMvISZDXmUwG3//933+pvF/8xV/E5uYmfuqnfgrlchmFQgGz2Qw/+qM/ipOTE/zJn/wJ9vf38fu///uJiS2Xy5fEiRjHAi6fCKvzahHGmjBdD52XuX4nDw+Ri9c0cEwmk8DFnHMh/AU3ytbr9eDqxbMLjo6OcHJygkqlglKpFJYuNjY20Gw2sbGxEbxciPwUQ1nnaDSKAqAE4Ng6qSZU+vkysTMmGmpL5TLrqNUuWd6y+VuWrrWoqQFUWqIYXFWKnHIAXnrpYrXi1q1bweNeRiBjOb/6q78KANjd3cXnP/95fPjDH05QckvXkOKFRQUtfUEmi2Jak+ucC3Ezqc+REEhAd84FdzDgArD39/fhnAuRxmgZ5c6HXq8XNrUSIHu9XuI8dO67KxQKYW2O3I7iJUVNbVmW3Ek/TxO59TilET1ZvuZ0FqddJvJbcyXza6TTyL5KutaIJ/U2PSk0e1uyNTng9vY2nHOJY6ZIrXW0rs997nPI5/P4vu/7vpBH+i7KAxp1G4HLi676XyOwzJuml3jvw0Zgri2S82mRWK5pUl8D5rvOC4UC9vf3gwdLt9sNC+3ct8fx7Pf72NjYQC6Xw/379zEYDFCr1YKllO1iO7g0QzcxTYC0p8r/1961xMZ5XefvjIZDzgxH5PAlSgZhy068iIEuDNV1YrRoDdRJA6PJMoCLGHaBAG3aXVu7yCJAixhtNgWCFoUfMZJsHGTR1lmkaF0vujESxSnqOAoaybEUyhYpWpRFm6T40twu/vkuz3947j/DcRwOmTkAMT//x32ce8/znnvufmwhj9npZxZfnppZhOdU2UVaiCXATnahB32tagJ7oxWArOM64auWhJxs5NTDw8NRwr3++utRAlhkffzjH48evhdffBHLy8sxUt8a/UWTSD/T7fJUZqtO2TII3ORLouLSgHYKebhptVqYmZmJeVW4PahUKuHEiRNR9Ww2m7j99tsjQZZKJfziF7/A4uJijBjimeQiuxnQ9IK5TsVuoZP0TzEli09dXi92lTYBNGPU8yalyhYRmiXEbqCvCa+Iu4lkThSRvQHUNPS5n43q2AMPPAAAePbZZyOyy+Uy7rvvPgDAwsICNjY2cP/998ey9FqTDSOzbfO2EaWe2VQW+plVZ3TOUB46sr6+Hs+lo0OD9h3VUCDTDCjVa7UaZmZmYujY6OhoXNu7evUqrl+/jlIpy1bGmE3ah5RmOh6TzhQ6ebxNvR7RWWajcaqdTkUT2TqnLCPrpMZ6BOuZFJ4pUaT2dgt9TXgaLEIpUQj63vb2Nl544QUsLS1hYWEBV69ejdtiAODhhx+O0fqVSgWXL2c7lsrlMpaXlwHsTlgttbiBNGWHaLXYtplt5Hu6L94g8n8SHCUdr5kO/SMf+UhO6nPy2hhUXe/a2hrm5+dRqVSwsLAQGQ4T1upd5TphLg8/0d5L7czRIWRFalgRURYxI36T0hL0ONh6igLSLZ6oNaTa6JWzX+hrwvM4Ge8TGD2hkb28vIwHH3wQwG7M5uLiYozqB4DHHnssqkb33HMPpqen8fLLLwMAzpw5g1OnTu1RMxho3WlSWW5pd1CnbBANvEf1kCf4VCqVnBNlbW0Ns7OzkUDr9XrMjRJCwPXr16MXlN/xzDxGqszNzWFubg61Wi3uZtAnAFE66zycACLBkdlpwvP6ayWKZ3t5BKTtWD3+qSgX/a0nPYukqWVSxKMt19ZnrzuB9EKtvyool8uh0WjkJr/HDcfGxuLkbrVaePzxx/HUU0/h9OnT2Nrawm233YazZ89ifX0dd911V9wNrWF6ejpyds3xrKrBGEQ7eezgeBs/tffNxoF6HDSEgImJCUxNTaFWq0WiGhsbQ61Ww8TERMx9CQCNRiNKKEozLn6ToOjV3N7eRr1ej1uBeCpQCAF33nkntra2csEF9FouLS1hdXUVa2truHnzJjY2NuJG2MuXL3dlu1pthfc81dMb75QqmvKe6l9L8PZ+qq0eg7T3RAQrKys/CiGcQQfoa68m4Ift8D5hbW0NjUYjRjF8/etfx1NPPYWLFy9idnYWly5dAoAYYG3hkUceAZCPXtf1iUiOo3vE5qkwqclE8L6xNoP1vmrniohE7yX3EI6Pj8cMbNPT01hdXY27D8bHx2MfeHYC81uePHkS09PTWFpawvnz52MqCHp3edaetU8p8bxlBE+Sp/CjcWRxr38tnvR1iuhS5aXqSrW1qN4iKepB3xMekDaU+as9nMCuNKFqpiNZWq0WTpw4gYsXL+KOO+5ACCHaTRq0PcQ6dd22Hm8QUzaItsn0QrK3I4KJa6n+hRBiCr/Z2dlIhOvr6/HgyXq9jrW1tXiKD095rdfruQS/Ozs7GBsbi33hsV233XZbjHAJIWB+fj53eqve+BpCiFnE7Pik1E37v8Ut73EMrL2ln3tM2ZsjKeIsUhW9OafL8kLVuoW+JzyPi1jVDNjdMkROe/LkSVy9ehXz8/PxnSeeeCKWd/r06T02GLCbF5LhVWyDnURFEk+/o595HFjHGlpViWuIOqyJ3y4vL2NzcxPlchlzc3OYmpqKDiN6GZnbcmNjAysrK6hUKvFZpVKJZxgwKoZbhMgYiIeZmRmMjIzgwoUL0aHC8uloef/99/c4dGy/tRRMqW0aKG1tOXpcLN46aUdFGkpq7DwTxyv3SEk8jyPyOjWgHjz99NN47rnnomqmB0gjnfGOun6C5sK89vb7FXFOy4W9SaPbxsVwkSxNw8TEBE6cOBHPO6CEYyp2ls2NwyGEuN8OQMyvwqO7+B3TBOoszGtra3Grz9raGqampnDx4sXoVOFSjV2783BmPdBa6lvm56nn1mb2cOtFGVlbLUVk3vzpxFz1O/shOuAQEJ5HdASN5BBCnERE8uTkJD7xiU/glVdeiSqbVRP0JPPKt/9b4reI72TXdeqXVTu16sm1xHK5jEajgWazGXcacGfC6upqTOnOc9Qp4ZjugYlol5eXo4Qvl8txL129XsfW1lZcvwthNzQMQG7B/NatW9FO7IQz++tJfQ93elKnwsP0mOp2dJJU+n3PIabHJMXYi8Y5BT1lklbP/kJEgohMtf8XEfmaZNmifywi96p3HxWRC+2/R7ttYIrbeJJve3s7t5g+NDSEs2fPxsnH7zihd3Z2sLq6muPYtj6+b8FyuRS3BtLbXfiex4mlbd/RtiuVStEbCWSHVzJtH0/waTabmJ2dxdzcXA4nbAv3y1FKMW5Vn2swPj4e2zM1NRWXLyqVCq5du5ZbQuAuBgZTE7x1NCt9NB5tv733qHKmHGC6PAteQLYHVr20BEUGYdu4H4KLberinW9gbyZpiMgcgN8HMK9u/wGyBEcfBfAFAP/cfncCwJcB/Bay5LZfFpEmuoTU5PcGVm9V0d/ptT4SnI6kt4ThqbFWLfKI1A5EilNaFcpTVxibqZ9rh0mplB0OsrKyEj2QZDw83UefkEuJRoIjMb733nsxHyePS75582bcEHvz5s24Fsi207Zj/KadtB4udd/1uHj40X2216nx8nBvtw/Z+jwmoevX804nD7bjsl/i6ymTdBv+AcBfAbn8mJ8B8K2QwfcBjIvISQCfBPBSCOF6COFdZCng9xCz20DjOeo0sdfX13MI0d+R0+uIfg6EXqT1BkHhY883doCKdP6UKuWpOJrw+FepVDAyMhJTOHBzKxMR8XgvnrQaQraIzoMo6QHlVqD33nsvt+ug0WjECB8ek1ytVrGysgIA0SHDTblUMz0CSUk6j9D0WFo86+cpPAJ7w/I8e82TlLy2kUe2Lrvzg9cETzikoCcbT0T+EMDbIYTXDKX/UjJJI5OWOW6lkWEXNC0n3N7ejodtAIiJXz1C4v8soyjsy7NV2E796xGqlZL6uW0L+8g4Ue3Fo3Noe3s7EiCPRq5Wq6jX6zh37lxUVSnVuHcOQIxg2dzcRLPZjOW3Wq0YHM5TisrlclxKAJDb/0e72OuvM6577nnP+L3HoDzC88amU31FbdBZ6Oy4aQL0bNL9wL4JT0RqAL4E4CHvsXMvFNzfezOEZwA8A2SRK1rke7ZQ+5scYpjugLaIDSkrsgNS/1vi9gY6pU7yuVVf7BqgBs0EaI9yxwWTHlWr1Zj+getzb7/9NoDdw1f05KEjhXZSo9HAjRs3wOgg2pMkKKrt6+vrkeC0U4VSUPfT2lMpye9JHCs1U+CVnypPf+PVrcvzdqHY8dPfd5LgRdBLrOZdAE4DeE1ELiHLCv0/IjKLDyGTNNDZSNeDzUlNb5zHibxyvHf1O1YNtdLNqlcpBmE9bymiYx2st1wuY2xsLB68yVNembqCRMzMadoRUKlUcPz4cYQQYvA3Uz/U6/XcMWDc3a8PsuQuDhIeA6K3trb2tN0zCfSvF1+p3yHYWEv20eItxRxt2VZi6v/1sdz80/VrLUj/ebsxuoV9E14I4fUQwkwI4Y4Qwh3IiOreEMIisuzQn5cM7gewEkJYQJbI9iERaUrmVHmofa+b+lx93VNPtETxuFEK8R6kVBxdToohAHsnTpGqZNvE/mq1B0BUK+ns4Lubm5sYGRnBG2+8ET2gTMlHaUkNgBOGUotreSRUejl5AiwPgaFdx2smWCoaFw9XXr+1XWjH26r99rpoCaGICPX/erkmNS8009C/tu5uoddM0in4HoA3kR3P9SyAPwWAEMJ1AH+LLJX7DwH8TfteR+hEHHZSp8J4vHK0rchvrNrk1WMJzKtHE5G+byeSLs8yGBICJwWlE3d5s33ac0nppHFAjk1i4kQbHR2NEpEhYMzfyfMUSKQiklM36Zyx7ffGyHNaWOlh8edpECkV1psDKQlXNIZasnrMVc8Tq63sV9XsNZO0fn6Hug4Avph473kAz++rdW3QA+dxPQ8p3XAgOzBFiPU4t67ftsd+Z8u29dr6vPKYlp27z8vlMoaGhlCv13H+/Pm4XxBAXBTXk5sbaSuVCm7cuBFPIdrZ2cHx48fjjgU6b65cuZIjOq7bcYOxh2fbP0rYImlEhmFxrFVLiydvHljc2XKLJKQuo8ibauv16u4G+j5yBUif6OoNliXQFBEU2Va2nk7X3v+2jbo9KSK3beUePNpkIyMjkXBEJKqFzMlCPHEZgp5R3mMcJ89aKJWyBMDVajV3fDK9wvSCatWU+/gsjrxf3T+vjxpvnjOrKAg5xbzsfev6t/j22mxtN/t+r8Smoa83wgK7yOO196vf1eDZCpZwUlKG16lJBORVlE7SKsVtdbkaqGZqjyzvbW5uIoQQJdTCwkJceuAvr3VbKAVJlHo7EZ00pVK2tYh7+cg4dGKjd999N7bFTt6UZNPjYMfIA49hdoNnTRzWC65/rR9A453fa79BqkytztsxLIK+JzzAjyYB0oOTGgxbprdgbjl0yi7h/57qo5957fMmob3H2FIAcWc5HRw6UW21WsWVK1dy55W3Wq24Vsc/EUG9Xo/ExO9F8mFYrJ9BBjpUjfYl6/ekdcoM8HBcNJZ6MnvSLUWwlmkWOUTsN8De8D7dBz1ndB38bj9wKAgP2OtiT3FU3vMmuqdaeIOckqQpYvQ4Z0q18ur2vuMZdZz4k5OTqFarOTVydHQ02mnArluciXo1gdHLCWT2H7dQaYlaq9ViIiVNlPSG8gQi224ryfV9rUbqflvJ4bnmU7i1AQu81oTlaRgpovTO4kiBHX+r/XQLfU14HgFoLqTve+96EgfIr9sQ9FYh7xtvAnjE7g1Kql+WkDWx6rSE5XIZzWYzqos8hHJsbAxLS0s5ZkRiYdwmVUJ9lsLY2FgkaKqedLzs7Oxgfn4+Lp7rqJdWqxVPAdLgqY+pwAAPP/bXSg9N6Npes3hPaT0kdFump0GlzAwrfT3tZz9Sr68JTyPZDk4nW8Eaw0VSS39jkWmf6Tbo8jzdv1PfUlKbqiIJ4tixY2g0GnE7EHPDVKtVLC8v5ySIlnK0wUiE+uw/hoOFEGKiXh1ixvaQ6Lh7wWM2dkJqJuARhf7W4tHTSnSdWop5sZl2bIuinbwx0dd2v6BuE9uSYu6doK8JD9hrUFskWj28iJNporMcuUjtK1JrPW6pf+23tj+6Lv3LtTjtlWT7mNmaUklHt+g8oCHshoHxxFgGSTPImURIifbOO+9EDygXy6lu0pvpqVUaf55X0L5jCdJ7riElKYvMiJSdbcu1BF803raslDrcCfqa8LyOeBPX07s9FSAlkVLqY4qo7KRJvWevNRe1dVuipbeRDhMSDyNGRkdHY8SJlpCUXPwG2Jsej4TGzGW0KXm0MwOi6WChY4XS1cNRCg/e2HlaSNG4pnBpNSGbEoLQi8akbV+vLlteSrVOQV8THsFTZ/RAaX0/pa8TtPrQSS20aoUu1+r6XluLGIeWpnYyapvt2LFjGBsbi2oeg6Xr9TrefPPNaMdZSW7bx83AVDdp0zHfJtVMtovEx7Ay61ixeEnhULcltfeQDMmzvVN4ZHm2Hk0gejnAMrZUeZoJ2Pml6/GI78hIPMC3yYomd4or6UndzcRhvV6wruXQnlrkqcT6WUplbrWywzAptUQk5vzkMxINpR+T2erd6hYX9HRSsulJRQn6/vvvR4IDdk+hJfGlcJ2ClCpov+e40B4rYp5FqqNlZlZa6ne8NVVvqcGbf4Cv0RwpwkuJeY9QUtzRQxzBM5p1XR6ntBzVe08PWNFk8VTYer2e4/6NRgPA7kGQVDM5YbVKSVVT10W7TxM39yuWSqUYCXPp0qU4+WyKBW6O9fBsF511/z1b2n6jn6W0BItbj8F6jpQUgej3LCP3tCvbTvv9fqHvCS9FSCl1USPEW+/jO54aYd/REtIiW39TNGGs9LETwSNyqoX6nARuTAWyrNfckmP33dEpwj+9EM8yjx07FmMtqWrqQ16AXWlHjyajVTR+dL88YtCOHt33lPpnx9LixVPxPFXdY7J6fLQH2BKP/tau7WrJbOtJzYEUHArCs/aAHbgUN0qJf2t3ePetCpoy3D1VxntWROyWEKk68rlOSsRlhcXFxVzmNBHJSToRiQ4XBk/zXRIo1VnGagKIth4Jj95PSkCNAy/yJzVu+rkdE8+GSklJW55Xpx4zO37W42rnkWWQKWbg9aPIfrTQ94QH+E6OojUUG9GQiqywnEx/m7IpNRSpG5ZZeH3i4GqpyIVuXo+MjKBer8cMapRSeuGcxFIq5U8UAhB3MNA2BHYlqj7S+tq1a7H/Oig6hBCdKrrP2qGj++M9s3j37C9PgqYYGkETlv4uFYWUWuBmXVpVLZLSKe1mP3AodifYye158VLvpu7pZ9bO0wPmDZZHVHaZoEjqWsmnJ1+j0YgSr1KpYGZmJnox+Zzp9LS00wvk+v7Ozg5mZ2cRwm72aEpVZiq7desWFhYWIi7Yf6qr6+vrubJ1m63Etvjx8K2vPa0l9X+KkHU8qqfG2nnTSVLpIAT7XKu1vNa7/buFQyHxLHAC2Eh8Ld34jmfEe8a2LZ8cUL8PpNUrT31KcUdvXYll1mq1XJjXyZMnY25MEcH4+PgeKck1O37HSBWqk5OTkwAQz0bgdalUQrPZjOkcNEPTDEdvetVSwcOpvvbUTT7XEsmTep5Kb8eIBOc5uTSxFWk5nrqqCdoScEqa70fNBA4J4XlqSYpreXq5fd964axaYieGx0mBtLtcE72tU4MlvhB2F7JJNGNjY3GhnCf+rKys5N7hZKCKanegMykS1VSqn7T16LFkX6hmcvPrzZs3Oy6reETkSTO7MG1xzL6kTAk7Rt7SiWW6KfDiN1O/9k+XkWKwRdBzJmkR+XMR+ZmInBORr6r7fy1ZJumficgn1f1Pte+9ISJPdttAj3OR2wG+bWB/PU7Y6dpDpsc9taqridJ6vvQAWUai6yXRsD4ubpfL5bgtiNuAeB9ALus0vZkkRB4hTaBtV61WISK4fPlyjlj1pledINji1dqo3jil8Oo909c20MHDlcfIUkw3xWj1/CJ4bfTmkS7PC7Ivgm5svG8A+EcA31Id/D1kyWt/I4SwKSIz7fsfA/A5APcAOAXgv0Tk7vZn/4Qs8/RbAH4oIt8NIfy0U+V6cqcISIN9j+/ab22+EP0ekPdiWmL20gl47bOT0pMClmhJLKVSKebNBLKUhSQqnhHBcmxws45KAXYdJrqPJExml2ZbS6VS7kxzve3I4tmW3cnWTuHIG0tvktt37T1bV6p+a5Ol5orVnryyUn3tBL1mkv4TAH8XQthsv7PUvv8ZAN8OIWyGEC4iS3p0X/vvjRDCmyGELQDfbr/bEVISLaXK2G+0dOGEBpALtfIQZ72cttwUwabsNwsex67Varnzzk+dOhXVRABoNrOs95RG+sw87cnkPTpPmKtFJ05ifVRjaRsCu8l0y+Vy7mxBr/1aLfTS5Gm82THyNAVvPFLP7Tt2TqTMCI+RaGbq7VzXBAvsHWfr5OsEvdp4dwP4bRH5gYj8t4j8Zvv+LyWTtIi8KiKvemtsVkVLdTY1+T0bhM81geoJ5Rnv/MbaPp7dogfRclrdPhIeJ/3o6CgqlUo8XrnZbOYSFtmsx1rtZNsYoUL1kd5JHkbChXSWoY+iDiHkIlY0eKqyxbM3TimV38urY//3pJ2He/ucYE0FS6j2z7bFa6c3J7uBXgmvDKAJ4H4AfwngO5K11JO1oeD+3pshPBNCOBNCOGPX3/Qk05xHT3be99z7RVyWdegBSy2a20moB8pTd3jtcUXe13GYLGd8fByVSiW+w+OX7XKKJmad6o/Jb5m4iJJUt5cRKVZLYN/1xlePy1utIgVW+lip4y2O62cew/RURC2VbHkadylpqMvW7UzNEU+97gZ6Jby3APxLyOAsgBaAKXwImaStdEipctZ+8NZgbLme3m65YJGer9vAa/t/SsLp9znJ9UI4bTwmrh0eHkatVsPq6uoeTx4nk1ZTeW9qaionyQCgWq1ieHgYGxsbcSGeNiH337VarRixonGhcav7aJduLDOy9yxjTI2nJ8k8LUaPcyrJEd/lO6nyeK0dJnbeeUzkV0F4/wbgwXZldwOoALiGLJP050RkWEROIzuu6yyyJLYfFZHTIlJB5oD5breVWbVNc3s9+ez7GlJqEQfIRsbraz1IHoez3xQNgCcZRHa36bDt5XIZx48fjynZKRUXFxcBILcbgf8zNwrbwOUIRsOwL9JWNQHkskuLSFzT0wzBTjhPanv37Vh4TMcSj5VMVnW342zr0kRg7bmU/Ziy+yzT1PPBa/d+oKNXU7JM0r8LYEpE3kJ2zt3zAJ6XbIlhC8CjIWvVORH5DoCfAtgB8MUQwq12OX+GLG37MQDPhxDOddNAEcltVQHyh314ErGT1OJ9/T+Q99DZNthrT9Ww77KtnSRdq5UlLtJLAjzfTi8LAMCNGzdyi+V0nhBHJF4AMT8Lr0UkekOHh4cjURNfJGL+bz2aKWljNQ5PSnl2mv7fGw+Oif5WS25Pfdflp9bx7DpiimHwHa2+85lWafdLdMAHyyT9R4n3vwLgK8797yFL8b4v8JDjue95L6Xje6qLJWBLINYNz1/9vQY7wWydbIdXJ7cCUVWcnp7OvcNA6a2trZiBjIxGezMtN6Zk03jiASQ87YflEn8hZB7RjY2N2JduGI7um+cxtpNXl1FEBHxuy9VEp+1NT1rqftg+FBGOR1xaQnpMuRvo+8gVq2ZYFYXvEFJuXe99OyC2XC8Gz0pTWzb/t2qrHXCrcjL5EJARQrVajfvkmCuF3J7fMWrFagSsY2hoKHesl7Ylt7a2sLS0FImNWaZ1e/XBKLrPReqjbaMur2iyeh5kXYcmLE/L4Xhphmjf47Wu29NEbJvsN9aTbcvpBvqa8NhJb1e1dQ3rZ/o6pYbwf0tULJt/HodPEZx9Zu0LzynBgaVkYq6VycnJ3ASamJjILZBzhwHL0Yvl7JclzKGhIQwNDaFWqwHINrdSteUi+sbGRgwXW1lZybXT9s+btDZ9g7aLipiRvpcaHw/3XnleWz2G6RG5HXuPEHUZRW0tgr4mPIIlCMtprFTyJod+l/97XFwPni3Te7+TJ89uBvUIXUTidh4AcQ2PZxiUSqXoJNGOEh0cTelI9bjVamF8fBwiEoOsiUN6NPm/3hTLb2/duoX19fXcONi224mWUrM9/BI0/uxkT0kjTdwpU8EjEG887Vin1E/P1k3ZkN2ApHTqfgAReQfAGjKP6a8zTGGAg8OCg9tDCNOdXuprwgMAEXk1hHDmoNtxkDDAwdHDwaFQNQcwgKMGA8IbwAAOAA4D4T1z0A3oAxjg4IjhoO9tvAEM4CjCYZB4AxjAkYMB4Q1gAAcAfUt40mOOlsMIInJJRF4Xkf8VkVfb9yZE5CURudD+bbbvi4h8rY2XH4vIvQfb+t5AnFw+vfRZRB5tv39BRB49iL70BDpusF/+kO1g+DmAO5FtOXoNwMcOul0fYn8vAZgy974K4Mk+9W5WAAABqklEQVT29ZMA/r59/WkA/w5AkG1E/sFBt7/HPv8OgHsB/KTXPgOYAPBm+7fZvm4edN+6+etXiddzjpYjBJ8B8M329TcBfFbd/1bI4PsAxkXk5EE08INA8HP57LfPnwTwUgjhegjhXQAvAfjUh9/6Dw79Snhd52g5IhAA/KeI/EhEvtC+dyKEsAAA7d+Z9v2jjJv99vnQ4qJfU7h7kadHed3jgRDCFcnSJL4kIv9X8O6vG26AdJ8PLS76VeL1nKPlMEII4Ur7dwnAvyJTta9ShWz/MoXiUcbNfvt8aHHRr4T3gXK0HCYQkbqINHgN4CEAP0HWX3rpHgXwYvv6uwA+3/b03Q9gherZEYD99vk/ADwkIs22B/Sh9r3+h4P27hR4vT4N4Dwy7+aXDro9H2I/70TmtX0NwDn2FcAkgJcBXGj/TrTvC7Ks3D8H8DqAMwfdhx77/QKABQDbyCTXH/fSZwCPI0uc/AaAxw66X93+DULGBjCAA4B+VTUHMIAjDQPCG8AADgAGhDeAARwADAhvAAM4ABgQ3gAGcAAwILwBDOAAYEB4AxjAAcD/A/dnb9herC0NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "path = PATH/\"boneage-training-dataset/9977.png\"\n",
    "im = cv2.imread(str(path))  #.astype(np.float32)/255\n",
    "plt.imshow(im, cmap='gray')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1668, 1323, 3)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "im = cv2.imread(str(path))  #.astype(np.float32)/255\n",
    "im.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1818, 1468, 3),\n",
       " (1776, 1412, 3),\n",
       " (1935, 1657, 3),\n",
       " (1478, 955, 3),\n",
       " (1622, 1300, 3),\n",
       " (1804, 1303, 3),\n",
       " (1526, 1132, 3),\n",
       " (2570, 2040, 3),\n",
       " (1494, 1104, 3),\n",
       " (1673, 1304, 3)]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's look at the typical size of these images\n",
    "path = PATH/\"boneage-training-dataset\"\n",
    "files = list(path.iterdir())[:200]\n",
    "dims = [cv2.imread(str(p)).shape for p in files]\n",
    "dims[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratios = [x[0]/x[1] for x in dims]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a presentation with EDA on this data\n",
    "https://alxndrkalinin.github.io/pdf/2017-12_CFT_BoneAge.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data augmentation\n",
    "The winners of the competition used real-time image augmentation consisting of horizontal/vertical translation, zoom, and rotation of 20 percent/degrees as well as horizontal flip. They use 500x500 images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# modified from fast.ai\n",
    "import math\n",
    "def crop(im, r, c, target_r, target_c): return im[r:r+target_r, c:c+target_c]\n",
    "\n",
    "def center_crop(im, min_sz=None):\n",
    "    \"\"\" Returns a center crop of an image\"\"\"\n",
    "    r,c,*_ = im.shape\n",
    "    if min_sz is None: min_sz = min(r,c)\n",
    "    start_r = math.ceil((r-min_sz)/2)\n",
    "    start_c = math.ceil((c-min_sz)/2)\n",
    "    return crop(im, start_r, start_c, min_sz, min_sz)\n",
    "\n",
    "def random_crop(x, target_r, target_c):\n",
    "    r,c,*_ = x.shape\n",
    "    rand_r = random.uniform(0, 1)\n",
    "    rand_c = random.uniform(0, 1)\n",
    "    start_r = np.floor(rand_r*(r - target_r)).astype(int)\n",
    "    start_c = np.floor(rand_c*(c - target_c)).astype(int)\n",
    "    return crop(x, start_r, start_c, target_r, target_c)\n",
    "\n",
    "def rotate_cv(im, deg, mode=cv2.BORDER_REFLECT, interpolation=cv2.INTER_AREA):\n",
    "    \"\"\" Rotates an image by deg degrees\"\"\"\n",
    "    r,c,*_ = im.shape\n",
    "    M = cv2.getRotationMatrix2D((c/2,r/2),deg,1)\n",
    "    return cv2.warpAffine(im,M,(c,r), borderMode=mode, flags=cv2.WARP_FILL_OUTLIERS+interpolation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Center crop, resize, horizontal and vertical translations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# resize desforms the image a bit\n",
    "# note that by resizing to a larger number and random cropping we are doing horizontal and vertical translations\n",
    "# we should try just center cropping the image instead of resizing\n",
    "path = PATH/\"boneage-training-dataset/9977.png\"\n",
    "im = cv2.imread(str(path))\n",
    "im = center_crop(im)\n",
    "im = cv2.resize(im, (550, 550))\n",
    "im = random_crop(im, 500, 500)\n",
    "plt.imshow(im, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Random Rotation (-10, 10) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rdeg = (np.random.random()-.50)*20\n",
    "print(rdeg)\n",
    "im_rot = rotate_cv(im, rdeg)\n",
    "plt.imshow(im_rot, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Horizontal Flip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "im_f = np.fliplr(im)\n",
    "plt.imshow(im_f, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split train and validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(PATH/\"boneage-training-dataset.csv\")\n",
    "train = df.sample(frac=0.8, random_state=3).copy()\n",
    "valid = df.drop(train.index).copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving a resized dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Do this to save time at training.\n",
    "PATH_550 = PATH/\"boneage-550\"\n",
    "PATH_550.mkdir()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_a_crop(path, sz=550):\n",
    "    im = cv2.imread(str(path))\n",
    "    r,c,_ = im.shape\n",
    "    pad = abs(r-c)//4\n",
    "    if r > c :\n",
    "        im2 = cv2.copyMakeBorder(im, 0, 0, pad, pad, cv2.BORDER_REFLECT)\n",
    "    else:\n",
    "        im2 = cv2.copyMakeBorder(im, pad, pad, 0, 0, cv2.BORDER_REFLECT)\n",
    "    return cv2.resize(center_crop(im2), (sz, sz))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from os import listdir\n",
    "from os.path import join\n",
    "def resize_all_images(sz):\n",
    "    for f in listdir(PATH/\"boneage-training-dataset/\"):\n",
    "        old_path = join(PATH/\"boneage-training-dataset/\", f)\n",
    "        new_path = join(PATH/\"boneage-550/\", f)\n",
    "        img2 = get_a_crop(old_path,sz)\n",
    "        cv2.imwrite(new_path, img2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = PATH/\"boneage-training-dataset/10007.png\"\n",
    "im2 = get_a_crop(path)\n",
    "print(im2.shape)\n",
    "plt.imshow(im2, cmap='gray')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(PATH.iterdir())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class BoneAgeDataset(Dataset):\n",
    "    def __init__(self, df, transforms=True, sz=400):\n",
    "        self.path_to_images = PATH/\"boneage-550/\"\n",
    "        self.transforms = transforms\n",
    "        self.df = df\n",
    "        self.sz = sz\n",
    "        self.sz2 = int(sz*1.05)\n",
    "        \n",
    "    def __len__(self):\n",
    "        return self.df.shape[0]\n",
    "    \n",
    "    def __getitem__(self, idx):\n",
    "        row = self.df.iloc[idx]\n",
    "        path = str(self.path_to_images) + \"/\" + str(row[\"id\"]) + \".png\"\n",
    "        y = row[\"boneage\"]\n",
    "        x = cv2.imread(str(path)).astype(np.float32)/255\n",
    "        x = center_crop(x)\n",
    "        if self.transforms:\n",
    "            x = cv2.resize(x, (self.sz2, self.sz2))\n",
    "            x = random_crop(x, self.sz, self.sz)\n",
    "            rdeg = (np.random.random()-.50)*20\n",
    "            x = rotate_cv(x, rdeg)\n",
    "            if np.random.random() > 0.8: x = np.fliplr(x).copy() \n",
    "        else:\n",
    "            x = cv2.resize(x, (self.sz, self.sz))\n",
    "        x = x[:,:,0]\n",
    "        return x[None], y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_ds = BoneAgeDataset(train)\n",
    "valid_ds = BoneAgeDataset(valid, transforms=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 16\n",
    "train_dl = DataLoader(train_ds, batch_size=batch_size)\n",
    "valid_dl = DataLoader(valid_ds, batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run this multiple times to get different images\n",
    "x, y = train_ds[10]\n",
    "#plt.imshow(x[0], cmap='gray')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model\n",
    "This model is adapted from fast.ai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# From fast.ai\n",
    "class Flatten(nn.Module):\n",
    "    def __init__(self): super().__init__()\n",
    "    def forward(self, x): return x.view(x.size(0), -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# From fast.ai\n",
    "def conv_layer(ni, nf, ks=3, stride=1):\n",
    "    return nn.Sequential(\n",
    "        nn.Conv2d(ni, nf, kernel_size=ks, bias=False, stride=stride, padding=ks//2),\n",
    "        nn.BatchNorm2d(nf, momentum=0.01),\n",
    "        nn.LeakyReLU(negative_slope=0.1, inplace=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ResLayer(nn.Module):\n",
    "    def __init__(self, ni):\n",
    "        super().__init__()\n",
    "        self.conv1=conv_layer(ni, ni//2, ks=1)\n",
    "        self.conv2=conv_layer(ni//2, ni, ks=3)\n",
    "        \n",
    "    def forward(self, x): return x.add(self.conv2(self.conv1(x)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Darknet(nn.Module):\n",
    "    def make_group_layer(self, ch_in, num_blocks, stride=1):\n",
    "        return [conv_layer(ch_in, ch_in*2,stride=stride)\n",
    "               ] + [(ResLayer(ch_in*2)) for i in range(num_blocks)]\n",
    "\n",
    "    def __init__(self, num_blocks, nf=32):\n",
    "        super().__init__()\n",
    "        layers = [conv_layer(1, nf, ks=3, stride=1)]\n",
    "        for i,nb in enumerate(num_blocks):\n",
    "            layers += self.make_group_layer(nf, nb, stride=2)\n",
    "            nf *= 2\n",
    "        layers += [nn.AdaptiveAvgPool2d(1), Flatten(), nn.Linear(nf, 1)]\n",
    "        self.layers = nn.Sequential(*layers)\n",
    "    \n",
    "    def forward(self, x): return self.layers(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = Darknet([1, 2, 4, 6, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = m.cuda()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x,y = next(iter(train_dl))\n",
    "x = Variable(x).cuda().float()\n",
    "y = Variable(y).cuda().float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m(x).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1e-5*100000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "631"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train_dl)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding optimal learning rate range\n",
    "From this paper https://arxiv.org/pdf/1506.01186.pdf.\n",
    "This an implementation of the \"LR range test\". Run your model for several epochs while letting the learning rate increase linearly between low and high LR values. Next, plot the loss versus learning rate. Note the learning rate value when the loss starts to decrease and when the loss slows, becomes ragged, or increases. In the example below the range seem to be from `1e-5` to `0.012`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_optimizer(model, lr = 0.01, wd = 0.0):\n",
    "    parameters = filter(lambda p: p.requires_grad, model.parameters())\n",
    "    optim = torch.optim.Adam(parameters, lr=lr, weight_decay=wd)\n",
    "    return optim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "def save_model(m, p): torch.save(m.state_dict(), p)\n",
    "    \n",
    "def load_model(m, p): m.load_state_dict(torch.load(p))\n",
    "\n",
    "def LR_range_finder(model, train_dl, lr_low=1e-5, lr_high=1, epochs=2):\n",
    "    losses = []\n",
    "    p = PATH/\"mode_tmp.pth\"\n",
    "    save_model(model, p)\n",
    "    iterations = epochs * len(train_dl)\n",
    "    delta = (lr_high - lr_low)/iterations\n",
    "    lrs = [lr_low + i*delta for i in range(iterations)]\n",
    "    model.train()\n",
    "    ind = 0\n",
    "    for i in range(epochs):\n",
    "        for j, (x, y) in enumerate(train_dl):\n",
    "            optim = get_optimizer(model, lr=lrs[ind])\n",
    "            x = Variable(x).cuda().float()\n",
    "            y = Variable(y).cuda().float()\n",
    "            out = model(x)\n",
    "            loss = F.l1_loss(out, y)\n",
    "            optim.zero_grad()\n",
    "            loss.backward()\n",
    "            optim.step()\n",
    "            losses.append(loss.data[0])\n",
    "            ind +=1\n",
    "            \n",
    "    load_model(model, p)\n",
    "    return lrs, losses "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Darknet([1, 2, 4, 6, 3]).cuda()\n",
    "lrs, losses = LR_range_finder(model, train_dl)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJztvXd4Y+d1r/t+6CBAAqzDNjPkdM1oZjTSqEuWbMlFbrKv7MQ+7vaJr4+dXDuJjx0fn5t2b3Jtpzhxin0UuZ44cpGcuMRyU7XGGkkz0vTeh70TYAFIAN/9Y+8NAgRYAQIkuN7nmWfIjU1gbRL4YeH3rbU+pbVGEARBKF1sxQ5AEARBWFpE6AVBEEocEXpBEIQSR4ReEAShxBGhFwRBKHFE6AVBEEocEXpBEIQSR4ReEAShxJlT6JVSX1NK9SiljmW57ZNKKa2UqjG/V0qpLymlzimljiilrl+KoAVBEIT545jHOd8A/hH4VupBpdRa4NXAlZTD9wGbzX83A182/5+Vmpoa3dLSMq+ABUEQBIODBw/2aa1r5zpvTqHXWj+jlGrJctMXgU8BP0w5dj/wLW3MVdivlAoqpRq01p2zPUZLSwsHDhyYKxRBEAQhBaXU5fmctyiPXin1ZqBda3142k1NwNWU79vMY4IgCEKRmI91k4ZSqgz4LPCabDdnOZZ1appS6sPAhwHWrVu30DAEQRCEebKYjH4j0AocVkpdApqBl5RS9RgZ/NqUc5uBjmx3orV+UGu9V2u9t7Z2TotJEARBWCQLFnqt9VGtdZ3WukVr3YIh7tdrrbuAHwHvNatvbgGG5/LnBUEQhKVlPuWVDwPPAVuVUm1KqQ/NcvpPgQvAOeBfgI/mJUpBEARh0cyn6uadc9zekvK1Bj6We1iCIAhCvpDOWEEQhBKnpIX+XE+YZ8/2FTsMQRCEolLSQv+5x07z8e+8XOwwBEEQikpJC/3xjmH6RycYHp8sdiiCIAhFo2SFfmB0gs7hCACX+kaLHI0gCELxKFmhP94xnPz6Ur8IvSAIq5cSFvpQ8utLfWNFjEQQBKG4LHjWzUrheEeIxoAHpZRk9IIgrGpKWOiH2d4YYGwixkXx6AVBWMWUpHUzGjXEfUdjBS01Pi5LRi8IwipmxQu9MXUhnVNdIbSGa5sCtFb7GBybZHhMSiwFQVidrGihf/JUD3f91VMMjE6kHbcWYq2MHuCiZPWCIKxSVrTQNwQ9XBkY40eH2tOOH28PUVnmpCHgoaW6DJBaekEQVi8rWui31VewvaGCH7w8Teg7h9nRGEApxdqqMpTKXkv/mR8c4T+PyLh8QRBKmxUt9AAP3NDMkbZhznaHAZiMJzjTNcKOxgoAPE47jQFvRkbfHYrw8AtX+f7Bqxn3KQiCUEqseKG//7pG7DbFIy+1AXC2e4SJeILtptADtNSUcbE/vWlq/4V+AI62DWdd0BUEQSgVVrzQ1/jd3L2llv94uZ14QidHH+xoDCTPaanOLLF87rwh9P2jE3SYM3EEQRBKkRUv9GDYN92hKPvO9XG8I4TXaafVrLYBaK3xMTQ2ydDYVHXOcxf6aQx4ADjaNlTwmAVBEApFSQj9PdfUEfA6efSlNo53DHNNQzl2m0re3lJtlliaPn370DiX+8d4z60tOO2Kw23DWe9XEAShFCgJoXc77LxpdwM/P97F8Y5Qmm0DhkcPU5U3lm1z15ZattaXc1SEXhCEEqYkhB7g/7i+mchkgrGJeLLixmJtVRk2NTXF8rnz/VSWOdlWX86u5iBH2oZkQVYQhJKlZIR+z9ogG0xf/tqm9Ize7bDTGPRyqX8UrTX7L/Rzc2s1NptiV1OAUCTGlQEZZSwIQmlSMkKvlOI9t66nxu9i8xp/xu2tNT4u9Y1ydWCc9qFxbt1YDcDOZuNN4YjYN4IglCglI/QA77+thec+cw9uhz3jtvXVZVzsG+W5C30ASaHfsqYcl8PG0XYRekEQSpOSmkevlMJpV1lva6n2EYrE+OnRLiPrrzOyfqfdxvaGCo5IiaUgCCVKSWX0s2HV1T9ztpebN1Sj1NQbwq7mAMfaQyQSsiArCELpsWqE3hpXrDXcZto2FjubAoxEY1yQCZeCIJQgcwq9UuprSqkepdSxlGN/pZQ6pZQ6opT6d6VUMOW2zyilzimlTiulXrtUgS+UtZVGiSXArRvShX5XsxH+0XaxbwRBKD3mk9F/A3jdtGO/BK7VWu8CzgCfAVBKbQfeAewwf+aflVKZK6NFwOWw0VTpZU2FO208AsCmOj9ep10qbwRBKEnmXIzVWj+jlGqZduwXKd/uB95mfn0/8B2tdRS4qJQ6B9wEPJeXaHPkXTevx2W3pfnzAHab4tqmCumQFQShJMmHR/9B4DHz6yYgdcB7m3lsWfCRuzbywTtas962synI8Y4QsXgi6+2T8QQvXBxYyvAEQRCWhJyEXin1WSAGfNs6lOW0rKUsSqkPK6UOKKUO9Pb25hJGXtjVHGB8Ms753uwLst998Sq/9b+eyxh3LAiCsNxZtNArpd4HvBF4l54aFNMGrE05rRnoyPbzWusHtdZ7tdZ7a2trFxtG3rA6ZA9fzb4g++xZo9FKRiUIgrDSWJTQK6VeB3waeLPWOlX5fgS8QynlVkq1ApuBF3IPc+lprfZR43fz7Lm+jNsSCc3+i8bEy07ZpEQQhBXGnIuxSqmHgbuBGqVUG/AnGFU2buCX5sLmfq31R7TWx5VS3wNOYFg6H9Nax5cq+HxisylesaWGJ0/1EE/otHn2p7rCDI1NAtA5JEIvCMLKYj5VN+/Mcvirs5z/F8Bf5BJUsbhrSy0/eKmdo+3DXLc22RrAb84bWb7HaaNzeLxY4QmCICyKVdMZOx/u3FyLUvD06fTF4f0X+mmpLmPLmnLZX1YQhBWHCH0KVT4Xu5uDPH2mJ3ksFk/w/IUBbt1YTUPAQ+eQZPSCIKwsROincdeWWg5dHUpuJH68I0Q4GuOWDdU0BLyyGCsIwopDhH4ad22tJaHh12Y55XMXjGobK6MficYIRSaLGaIgCMKCEKGfxu7mIAGvk6fPGD79c+f72VTnp67cQ0PQC0CXZPWCIKwgROinYbcp7txcw9NnepmIJXjx0kBy2mVjwANAh/j0giCsIETos3DXllp6w1G+e+AqYxPx5LaDVkYvPr0gCCsJEfos3LXFGMnw9786C8AtZka/ptyNTSGVN4IgrChE6LNQV+Fhe0MFfSNRttWXU+VzAeCw26gr90gtvSAIKwoR+hm4a6uR1d86bdvBhqBHumMFQVhRiNDPwL3XrAHg7q11accbA16ZdyMIwopChH4GblhfyVOfvDvp11s0BDx0DkeYmswsCIKwvBGhn4WWaXvLAtQHPIxPxhkeT2+a+vJT5/n0I0cKFZogCMK8mXN6pZBOo1li2TEUIVjmSh5/5OBVhsdjxQpLEARhRiSjXyANZtNU6oLs4OgE53tHGRiNEk+IpSMIwvJChH6BJDP6lBLLg5cHAUhoksPQBEEQlgsi9Aukxu/GYVNpTVMHTKEH6BsRoRcEYXkhQr9A7DbFmgpP2hiEg5cHcNmNX2XfSLRYoQmCIGRFhH4RNAY9ycFm0Vicw23D3L7JaKwSoRcEYbkhQr8IGgJeukJGRn+sPcRELMFrd9QDYt0IgrD8EKFfBKlNUwcvDwDwqmvqcNiUZPSCICw7ROgXQUPAw0QsQf/oBAcuDbK+uoy6cg/Vfhf9IvSCICwzROgXQXIu/VCEg5cHuWF9JQDVPrdYN4IgLDtE6BdBY8AQ+ucu9NE/OsHe9VUA1JS7Z7RuPviNF/mPl9sLFqMgCIKFCP0iaAga3bE/PtwJwN4WI6Ov8bvoz5LRj03EeOJUD/vNjcYFQRAKiQj9Iqj2uXA5bBxtH6bC42BTrR8wmql6R6IZky2tUsz+UbF1BEEoPCL0i0AplZx5c8P6Smw2BRgZ/UQswUg0fbhZuzm/flCEXhCEIjCn0CulvqaU6lFKHUs5VqWU+qVS6qz5f6V5XCmlvqSUOqeUOqKUun4pgy8mltDvbalKHqvxu4HMWnorox8QoRcEoQjMJ6P/BvC6acf+CHhca70ZeNz8HuA+YLP578PAl/MT5vLDWpC1Km4AqpNCn74g2z5oCr0MPBMEoQjMKfRa62eAgWmH7we+aX79TeAtKce/pQ32A0GlVEO+gl1OtNb48Drt7G4OJo/V+I359NNr6a2Mfmhsklg8UbggBUEQWPzGI2u01p0AWutOpZS1sWoTcDXlvDbzWOfiQ1ye/Nc7N/Cm3Y14XfbksVozo++dZt20p0y6HBqfTFo8giAIhSDfi7Eqy7GsO3EopT6slDqglDrQ29ub5zCWHq/LnrHVYKXPyOj7wtMy+uFx3A7jVy0+vSAIhWaxQt9tWTLm/z3m8TZgbcp5zUBHtjvQWj+otd6rtd5bW1ub7ZQVh9Nuo7LMSf/olNDHE5rOoQjXNFQAZK2zFwRBWEoWK/Q/At5nfv0+4Icpx99rVt/cAgxbFs9qocbvpi88Jea94SixhObaJkPoB2VBVhCEAjOnR6+Uehi4G6hRSrUBfwJ8DvieUupDwBXg7ebpPwVeD5wDxoAPLEHMy5pqvyut6sby53c2BQBpmhIEofDMKfRa63fOcNM9Wc7VwMdyDWolU+N3c7wjlPzeEvodjYbQS9OUIAiFRjpj84xh3Uxl9FZp5frqMso9DlmMFQSh4IjQ55kav4twNEZkMg4YQl/hcVDucVLlc4nQC4JQcETo84xVI2958R1D4zSa8+tF6AVBKAYi9HkmOe/GtG/aBsdprjSFvkyEXhCEwiNCn2eqzTEIVuWNZPSCIBQbEfo8k7RuRiYIRyYJRWLpQj82kTGvXhAEYSkRoc8zNcl5N1E6zDn0TSlCPxFLMDoRL1p8giCsPkTo84zXZcfnstM3Ek2WVloZvTULR2rpBUEoJCL0S0BNuZv+kYlks5SV0VebQi/dsYIgFBIR+iWg2udKZvQOm6K23LBzJKMXBKEYiNAvATV+N30jUdqHxmkIerCbe8pKRi8IQjEQoV8CLOumY2g8ueUgSEYvCEJxEKFfAmrMMsorA2NJfx6g3O3AaVeS0QuCUFBE6JeAmnI3WkN3KEpT5ZTQK6Wo8rkkoxcEoaCI0C8BqXvCNqZk9ACVZS7J6AVBKCgi9EuAtegKmUJf7XfJLlOCIBQUEfoloKZ8KqNvCnrSbquUwWaCIBQYEfolYDbrptrnoj9lq0FBEISlZs6tBIWFU+Fx4LLb8LntlLnSf8WVPhehSIzJeAKnXd5nBUFYekRplgClFNV+V0Y2D1P+vfj0giAUChH6JWJrfTm7mgMZx6eapiYLHZIgCKsUsW6WiIfeuxelVMbxquQYhChQXuCoBEFYjYjQLxGOGfz3KsnoBUEoMGLdFBhL6AdGpfJGEITCIEJfYCrLLKGXjF4QhMIgQl9gnHYbFR6HZPSCIBQMEfoiYGwSLhm9IAiFISehV0r9vlLquFLqmFLqYaWURynVqpR6Xil1Vin1XaWUa+57Wl1U+VyS0QuCUDAWLfRKqSbg/wL2aq2vBezAO4DPA1/UWm8GBoEP5SPQUsIQesnoBUEoDLlaNw7Aq5RyAGVAJ/Aq4BHz9m8Cb8nxMUoOyegFQSgkixZ6rXU78NfAFQyBHwYOAkNa65h5WhvQlGuQpUalz8Xg6CRa62KHIgjCKiAX66YSuB9oBRoBH3BfllOzqplS6sNKqQNKqQO9vb2LDWNFUu1zMRFPMBKNzX2yIAhCjuRi3dwLXNRa92qtJ4EfALcBQdPKAWgGOrL9sNb6Qa31Xq313tra2hzCWHlYtfTSHSsIQiHIReivALcopcqUMdTlHuAE8CTwNvOc9wE/zC3E0qPanzrvRhAEYWnJxaN/HmPR9SXgqHlfDwKfBv5AKXUOqAa+moc4S4pkRi+jigVBKAA5DTXTWv8J8CfTDl8Absrlfkudap+xA1VfWIReEISlRzpji0Bj0EOZy86xjuFihyIIwipAhL4IOOw2blhfyQsXB4odiiAIqwAR+iJxU0sVp7vDDIlPLwjCEiNCXyRuaq1CazhwabDYoQiCUOKI0BeJ3WuDuOw2Xrgk9o0gCEuLCH2R8Djt7F4b4Hnx6QVBWGJE6IvITa1VHG8fZlRGIQiCsISI0BeRm1qriSU0L18ZKnYogiCUMCL0ReSG9ZXYFLxwsb/YoQiCUMKI0BcRv9vBtU3i0wuCsLSI0BeZG1uqOHR1iGgsXuxQBEEoUUToi8xNrVVEYwmOtsk4BEEQlgYR+iJzY0sVgNg3giAsGSL0RabK52LLGj8vSuOUIJQ8f/ToEf7h8bMFf1wR+mXATa1VHLg0SDwhe8gKQimz73wfz5wt/NapIvTLgBtbqhiJxjjZGSp2KIIgLCGh8Ridw5GCP64I/TJgW30FAJf6R4sciSAIS4XWmnBkku5QhESBP72L0C8DGoMeANoHx4sciSAIS8XoRJyEhsm4pq/A+0WL0C8Dyj1OKjwO2odE6AWhVAlHJpNfdxXYvhGhXyY0VZbRIUIvCCVLODI1vLDQPr0I/TKhKeilTawbQShZQuNTGX1ngZM6EfplQlPQI9aNIJQwaRl9SDL6VUlTpZdwJEYoxcdbbYxEY4zIbH6hRLFe2zYlHv2qpSlYBrCqffrf/+4hfv+7h4odhiAsCSEzo19f7aNzSIR+VSIlltA2OM6lPuklEEoTy6PfssZPZ0g8+lVJU6UXYFX79KHxSQZGJ4odhiAsCeFIDJfdRku1j+7haEGbpkTolwk1Pjcuh211C31kksGxiYJ3DQpCIQhHJin3OGgIeJiIJxgYK1xSk5PQK6WCSqlHlFKnlFInlVK3KqWqlFK/VEqdNf+vzFewpYzNpmgMeFatdZNIaEaiMRIahsZX74K0ULqEIjFD6IPGp/dC+vS5ZvR/D/xMa70N2A2cBP4IeFxrvRl43PxemAdNld5Vm9GPTMTQZiI/UOD2cEEoBOHIJBVeJw0BYz2uc7hwr/VFC71SqgJ4BfBVAK31hNZ6CLgf+KZ52jeBt+Qa5GqhKehdtVU3qc0kfSPi0wulR2jcsG7qTaHvKmAtfS4Z/QagF/i6UuplpdRDSikfsEZr3Qlg/l+XhzhXBY1BLz3hKBOxRLFDKTipzSSyICuUIuFIjAqPkxqfG6ddFXQMQi5C7wCuB76std4DjLIAm0Yp9WGl1AGl1IHe3sIP4l+ONAW9aF3Yj3SFZGwixoEZdtJKzej7ReiFEiRsevQ2m2JNhaegYxByEfo2oE1r/bz5/SMYwt+tlGoAMP/vyfbDWusHtdZ7tdZ7a2trcwijdCj1EsuvPH2B335wf9bu17SMXqwboQQJRSYp9zgBaAh4VkZGr7XuAq4qpbaah+4BTgA/At5nHnsf8MOcIlxFNJmr8aVaebPvXB/xhGYwS8aeOvpBFmOFUiMWTzA2EafCFPr6gHfFePQAvwd8Wyl1BLgO+Evgc8CrlVJngVeb3wvzoCHgRanSzOhHozEOXx0CYDhL+aRl3QS8TrFuhFnRWvPFX57hQu9IsUOZN9Yn1nKPA5jK6LUuTM+II5cf1lofAvZmuemeXO53teJy2Kgrd5dk5c3By4PEzEaoobFMobdeCC01PvrFuhFmoW1wnL9//CwAv//qLTnf3w8PteNx2nntjvqc72smrOd3hXfKupmIJRgYnaDa716yx7WQzthlRmOwNGvpn7vQn/x6MEtHYCgyicdpY025W6puhFmx9lbOV0L0hZ+d5n/84CjRWDzn+wpFJjnREcp6HNIzeijcBiQi9MuMpqC3JD365873s7bKWIPI1vlqlZ5V+11i3QizYg2+68hDddrw2CTtQ+P0j07ws2NdOd/fPz1xjrd95TcZYzymC319wHgtFGpcsQj9MqOp0kvHcOF3iV9KRqIxjrYPc9+1DQAMz5DRV3idVPlcMu9GmJWLfWMAdORhhMCpLiP7ttsU395/Jef7O9o+zNhEPCOZSVo35mJsY4G7Y0XolxlNQS8TsUTBd4lfSl68NEA8oblrSy1epz2rRx8aN2qMq3xu4gm9qjdgEWbHsm7ah8ZzTghOdhpC//7bWnjh0gCnu8KLvi+tNafMn+8Jp78JWcUGltBX+904bIVrmhKhX2aUYonl/vP9OO2K69dVEixzzmDdTBpdg34XIGMQhJmxrJuJWCJnm+9UV5gqn4uPvXITLoeNbz9/edH31TsSTa4v9YbTE7WpxVjDurGbTVNi3axSrKapfHwsXS7sv9DPnrWVeF12gmWu7Bl9xMroDaGXBdnlxz89eY7P/OAo8SWw1U53hXn8ZPec58XiCa4MjLG5zg/kviB7sjPEtvpyqnwu3rCzgR+81M7oIrezTP000BPKLvR+91ShY30Bm6ZE6JcZjVZGPzRW5EjyQygyydH2YW7ZWA1A0OtkeDxTxMMpHj1I09Ry5CdHOnn4hSv82Y+P573++/977CS/9/DLc76JtA+NE0tobt9UA+Qm9PGE5nR3mGsaKgB49y3rGInG+OGhjkXd36nOKaHvHUl//oYik/hcdhz2Kck1aunFo1+VVHiclHscJWPdvHhxgISGWzZUARjWzbSMXmtNaNysuvEZNcVSebP8aBscI1jm5FvPXeafnzqft/udiCV4/sIAYxNxzs/RBHXRtG0soc+lFPlS/yiRyQTb6ssBuH5dJdvqy/nX/ZcX9UZ2qitMbbkbn8uexbqZGn9gUcimKRH6ZUhT0Et7kayb9qFx9qfUvOfK/gv9uBw2rl9n7D8TLHMyOE3oo7EEE/EE5R4HlT7jxZDveTcHLw/w48OLy9QEo5s5HInx3+7ayFuua+Svfn6a7x24mpf7fvnKIOOTRg37kbbhWc+1/PndawP4XPachN7KwK2MXinFu25Zz4nOEH/7yzN8+pEjvOFLv2b7H/+MX56Y21Y61WXYQLXlbnqmCb1VbJBKfcBLNJbIamXmGxH6ZUhTEZum/vnJc/zONw/kLct47kI/168L4nHaAQh4XQyPT6Tdv1VhU+F14nbYKXc78prRJxKa//79I3zqkSN5aYpZjVifMNdWlfGFt+3mzs01fOYHR/n12dwnz+4714dNgddp52jb0KznXuofw+eyU+t305jj/g0nO0PYbYpNpt8P8NY9TZR7HPzDE+f4xYkuqnwubErxi+Oz19jH4gnO9oywrb6cunIPvdOqbsLRyWRXrEUhm6ZE6JchTZVe2geL49F3DkcIR2OEIotbkEpleHyS4x0hbtlQnTwWLHMyGdeMTUwJbmjcqjE2Mp58N009daaHC32jjE/GOXh5MG/3uxTE4gkePdi2JAueudBmPh+bK724HDa+/O4bqPG7ePiF3GvP953vZ2dzkJ3NAQ7PldH3j9JS40MpZQr94kXyVFeIjbW+ZBICxmLpzz/xCn7zR6/ipf/71fzvD93MLRuqeXGG8dqpcU3EEmyrr8ia0VsjilMp5E5TIvTLkKagl1AkRrgIteRWuVc+yr72X+hHa7g1Regry4ysJrXE0rpOq8a4yufK62LsV5+9SF25G7tN8ezZvrzd71Lw7Lk+/vD7h/Nqn+WDNjOjb64sAwxB3FTnz7k6LByZ5NDVIe7YVM3u5gAnOkNMxmfeeOdSnyH0QB4y+jDb6isyjjcGvTQGvSilALiptZJL/WMZtfGpWPXzW03rZrpHHxqfTD6/LRrM7ljJ6FcpW8zFocNXZ89ulgLryZyPLOOp0z343Q6uXz+1P3zAa1TVDKV0x4am1RhX+dx5G2x2qivEvnP9vP/2FvasDfLsueUt9JagziYqS0U8oWd83LbBccpc9uQbNRhClevz5PkLRjPd7Ztq2NkcZCKW4Ex39qalyXiCq4PjtFYbQt8U9NA/OkFkcuF23PC4MfrA8udn48YWo5DgxYszfxo81RlO2kC15W7CkVhaXNky+loz+ShELb0I/TLkppYqHDZVcFGajCeSjUq5Pvm01jx5qpc7N9fgTCkpC5pCMZyyAGV1DVpVCdU+V97q6L/27EW8Tjv/5aZ13Lm5lqPtw1nn4S8ln3rkMH/7i9PzOtdam+kL5yfGeELPe73l0YNtvOILT6b9babiGqMpJcuFqa0vZ8vA52Lf+T48TmOxfldTAJh5QbZtcJx4QrO+uiz5+EZsC3+zOWV2xF7TUD7nudc2BfA67bPaN6e6wrTWGDZQbblROWZl9VrrtE1HLOw2xZ+9eQf3XLP0u62K0C9DfG4He9YF+c35wgp9qq+Y68fJk51hukIRXrkt/UkczGrdpM8BqfIb825yXRDuG4nyH4c6eOCGJoJlLu7YXIPWhrhM54WLA7x8ZTDvpW5aax472sWPj3TO63zLiugbyd26iic0d3z+CR585sK8zj/RGSIymeBMT2ZG3TY4TrPZzGfRGPCgNXTnsIHGvnN93NhShcdpZ311GRUex4xCb1XctNZYGb3VXLgIoe9Kr7iZDafdxp51QV64OJvQh5JlmnWm0Fuvp2gswWRcJz+xpvLuW9azZ11lxvF8I0K/TLltYw1H24ezZlcLYXh8ktd+8Zl5edOpL9hcM/onTxs7SN69NX2byKBp3QymWTdW1Y25GOtzMRnXOS8If3v/FSZiCT5weysAu5sDlHscGb+LjqFx3vXQft76z7/hVX/zNH//q7Nc6c/PYnhPOEo4GuNi32jWDVemY4nW9IabxXB1YIzO4QgPPXtxXhvOXx0wrjmbdWIIfVnasYZgbl3cPaEIZ7pHkjXxSil2NQc52p698saqoU/16I3HX7jQn+wMUeVzJUV5Lm5sqeJkVyjrDKZwZJK2wfGk0E9l9MbvZWpypTPjZwuFCP0yxco+n8txUe7w1SFOd4f54x8dm/Mjdo8p9B6njc4ctzl74lQPu5oD1JV70o4nM/qx9MVYu03hNasfqs15N/05iF00Fud/77/MK7fWsrHWKJ9z2G3cuqGaX5/tS8vcH3zmAlrDH79xO/UVHv7u8TO84q+e5IlTc9dOz8X5nqkGoKNzVJTAlGhOX8xbDGfNx+4NR3ns2NyfKK6YQn+2O71pKRyZZHh8MmtGD4tfz/nNeeO5fYcp9AC7mgOc6gxn9d0v9Y9S7nZQbXZP1wc85o5sC3/SxIJQAAAgAElEQVSunuwKs62+PM2Kmo2bWqvQmqxVW9Ybo7WwO926mV5VVgxE6Jcpu5uDlLns7MvRpz9hepEXekf5zhylcN3mfI6dTQG6clhkGxyd4OUrg9y9NdN79DjtuB22tOzW6Ip1JF90VWZ3bC4+/X8e6aRvJMoH72hNO37nllrah8a5ZGbsfSNRvvPiFd6yp4kP3tHKwx++hX2ffhXBMic/P5a70J9L6fQ8PEeNeCyeSO4jmo+hbmdNC6Yp6OUbv7k067la66TQn+tJF3rLA2+aJvS5ZvTPnusjWOZke4p9sqs5QCwxNQUylYt9U6WVYFgqa8o9C87o4wnN6a7QvGwbiz3rgjhsihez2DepFTcA1T43NjUl9NOryoqBCP0yxeWwcVNrVVY/eSGc6AjRFPRyy4Yqvvirs7OO/+0KRXDYFNsbKnLy6J8520tCw6u2ZV9kqixzTau6SV+osjK2XGrpf3G8m4aAJy1bBLjT/P5Zs9Hn6/suEo0l+MhdG5PnNAa97Fkb5OWrudfcn+sZwe920FrjS+6ZOxM94SjxhMZpV3nx6M91j9AQ8PChO1p5+coQR2Z5o+kNR4nGEthtKvkGYdE2kF5aaeF3Oyj3OBaV0Wut2Xeuj9s2VmOzTWXVO5uDAFljtWroU2kMLlzop48+mA9lLgc7mgIcuJT5nDjVGcbvdiQ/8dhtimr/VC19aNp+scVAhH4Zc8emGi70jubkl5/oDLG9sYLPvn47A6MTfGWWGSXdoQh15UbHYTgSY2SRU/yeONVDtc+VrKKYzvR5N+FILG2haqYJlpHJ+LwWS2PxBPvO9/GKzbUZH83XV5fRXOnlmbN9hCKTfOs3l7nv2vq07kiAPesqOdszkvNc/HM9I2ys87OrOTBne78lWNvqKxgYzX3zlbM9I2yq8/O2vc2Uuex88zczj+C9ajZE7V1fSXcomvaJK7VZajpNi2xautA3SudwJOnPWzQGPNT4XRm/q4lYgvbBcVqr099smirLFlx1M330wXy5qaWSQ21DGd3Vp7vCbJ1mA9X63ZkZvVcyeiELt200XgSLtW/GJ+Jc6B1he0MFO5sDvHVPE1999uKML4yeUJS6Cg/1pve6mDeYeELz9Jle7tpam5appRLwps+kn95Mkk3oQ5FJbv7Lx3nkYNucMRxuGyYcifGKLbUZtymluHNzDfvP9/ONfZcIR2N89O5NGeddv64SrZkzC5+Lcz0jbKr1s6s5SFcoklwHyYb1d9m9NkA8obPurXu+d4TvHbg65xteIqE51zPC5rpyKjxOHri+mR8f6Zhx3cOybe69Zk0ybou2wXE8Tlvyk1YqC5nAqLXmTHeYrzx9no9/52UAbt+YLvRKKXY2BTLWM64MjJHQZM3oO4cWtiPbsY7hjNEH8+HGliomYom0NyGtNSdTKm4s6iqmMvqwZPTCbFhzshcr9Ke7wyQ0bG80MpdPvnYrAH/98+w13d2hCPUVnpSOvYV/JD90dZChsckZbRswMvq0OvrIZNqLwOO043PZ0+yLZ8/2MTw+OWuJm8Wvz/aiFNy+qTrr7XduriUcjfEPT5zl7q21XJvlk8eutQGUgpevLF7oQ5FJesJRNtb52N1sPMZsLf6W0O8y7YtsPv3X913kU48c4dOPHiE2y+J6+9A445NxtqwxxOx9t61nIpbgOy9mH0R2pX8cpaaqpM6mVN60DxkVN9kWLhuC3nnZfAcuDXDXXz3Fa774DJ977BSJBPzpm7ZnCDcY13+2J8zYxNQnykvTKm4smoJeJuLz25FtIpbgCz87xf96+jx711emjT6YD1bjVOpzsHM4QjgSyxD61Ix++u5SxUCEfhljsylu21jNvvN9i6rvtnajtxa7moJePnRHK//+cjsXsoyD7QpFWFPhzmnY0hOnerDbFHduzsymLYJeF0MpM+mtjcFTqfKnN009ZZZrZlukm84zZ3rZ1RwkWJaZgQLctrEapWAyrvnYKzOzeTBelJvr/Lx0ZfE+vVVxs6nWz47GAHabmtUn7xgaJ1jmZF2VYU9k8+nbB8dxO2x870AbH/nXg4xPZO8KtTLyzabQb6or545NNfzr/stZ3yCuDIyxptzDhlo/HqctWbEDRkZv1axPpzHgYWB0YsY4wKg2+sDXX8Sm4C/fupPnPvMqfvrxO3n/7a1Zz9/VHCCh4bj5/IWp7QOtrtipx5/fgvDJzhD3/9M+/vmp87zthmYeet/eWc/PRqXPxeY6f7JxKp7Q/MiciLptmg1UV+GmbyRKIqEJR2LYbYoy18LeWPKJCP0y5/ZNNXSHopzvHV3wz57oHKbc40jzVt+ypwkwNjFOZWwiRjgSo67CQ12FUfWyGOvmiVO93LC+ksAsfqQ1qth68wqNZ3YNVvvcSaHX2rCDwChlm23g1/C4MTvlFZtrZjwnWObippYqbttYnczSsnH9ukpevjK06CYqS2w31fnxuuxsrvPPmtF3DEVoDHip8Ru//2xC3zEU4a4ttfz5/Tt4/FQP7/nq81l7LawF1U21U5nme29dT+dwhKdOZ06cvDo4xrqqMuw2xcZa/zShH8vqzwNzfvo70x3mvV97ngqvk4c/fAv/5eZ1yZ+ZiZ3NmR2yl/pHCXidVE6zj+ZTS//k6R7u/8d99IajPPTevXzhbbsXXdN+Y2sVBy8N8t0Xr3Dv3z7N5x47xY7GCnZO+1RY63cTM+23UGQSv9sx71LOpUCEfpljeZiL6ZI90RFie0NF2hOspdqH3aYySuisrc/qKzy4HXZq/K4FZ/Sdw+Oc7AzNatsABMqcTMQSRCYTxOIJRifiGV2D1T5Xct7Nyc4w3aEoN7dWEY0lktldNp4730dCM+snCoCvf+BGvvb+G2c9Z8+6IMPjk8lGnYVyrncEl92WzNB3Nwc50jbzG0fH0DiNQS+1/vQ67GznvPfWFv7xnddzpG2Y3334pYzzznSPUFfuJpAym+aurbU47YoDWWrBrw6MsdaMc3Odn3OmdTMSjTE4NplRcWNhCW2258rl/lHe/dDzOO02/u13bp5T4C3qyj00BDx8/8BVvvbsRZ6/0M+Z7pGsNs989lh+6NcXWBNw8/NP3Mm929fMK4aZuKmlinA0xqcfPUqZy86X33U9P/rdOzJsoFqzf6R3JJpRbFAMROiXOevMKpGFTl2Mm7XIlj9v4XLYWF9VliH0VlfsmgrjCVof8Cy4lt7anMFa0JuJStNSGRqfSFb2ZFg3KfNunjpj2DZWCWTqlm3TeeZsH35zhMRslLkcc3q0Vmv6S4v06c/3jNBSU5bcPm7X2gBDY5NcHcj+e20fGqcp6KHC68Blt2Xdji4cjSXF7Q27Gvg/79rAvnN9GfN7zvaMJG0bC7fDzrb6iozO08hknK5QJPmGtHlNOR3DEcKRyaSAzpTRNwaN58v0jHokGuNdDz3PZDzBv/7Xm1lfnSnSs/GeW9fTNzLBn//kBL/94H5euDiQUXEDRje13+2YscCgNxzlufP9vPW6Jqr98+uCnY17t6/h/be18I0P3MhPfu8O7tvZgD1L0YH1qbgnFDV2l3IXz58HEfoVwZ2ba/j12T6Od8x/muXl/lHGJuJpzSgWG+v8GULflRR64wlaXzG/RbZUfnasi011/jmrGYLeqe5Yq2twekWC5dFrrXnqdC/bGyq4dWM1dpvidFco4z7BsHieOdPLrRur0wapLZZNtX7K3Q5eXqRPf753NO13sdtcZD2UxacPRYwdnKzxuDV+V8ZgM0t0G1P88lduqyOhjd4FC60157rDbK7LrBPfaZZ5pn6qaB8aR2tYV23cr7Xx9vne0eTexdObpSzqZ1jPef5CP22D4/zNb+1my5r516tbfPTuTRz4n/fywv+4h6+9fy+fet1WPpplPcWYSz9zLf1jxzpJaHjj7sYFx5ANv9vBn755B3dvrZvVikn9VBYaL4GMXillV0q9rJT6ifl9q1LqeaXUWaXUd5VS2VfEhHnzu6/aTJXPxbsfej5tp/nZsDpip2f0YHjGl/pH0xblLOtmjfnCbQh4kuI/HwZHJ3j+4gCv3TH3R+NAyhiE0Aw1xtU+FxPxBB3DEQ5eHuTurbV4nHZaa3ycnOF3cLl/jLbB8Vn9+YVgsymuWxdcVOVNNBbncv8om2qnhH5rfTkuh40jWUo2O83FREvEa8rdGR69JWZWFg3Gm0dlmZOnU3z3zuEIoxPxrG+4O5sChCMxLqfM8rFKK9dWTmX0YPjrbXNk9FM2X7rQHro6hN2m0jadWQx1FR5etW0NH71704xvGI1BLx0zfPr88eEOtq4pX9SbTS4kxyCMRLNOriw0+cjoPw6cTPn+88AXtdabgUHgQ3l4jFVNU9DLv/3OzbgcNt710H7OZZkuOJ0THSGcdpU1q9tU62cyrrk8MPVi7w5F8DqNbfzAyNSGxiZnraZI5Vcnu4knNK/dUT/nudZgs+HxiSmhz7BujBfKjw93EE/o5DiFrfXlM77ZWVntXP78QtizNsiprhCjC2weu9Rn1H1vTBFbp93GjsaKrI1T0zPnGv/MQp+aXdttiru21PLUmd5kLbm1kLp5BqEHOJKyGN9mPg8s62atuYvUuZ4R2swqn9pZbI+GQGbT1KGrQ2xZU06Za+kz2Zl2muoYGufFS4O8aXfDkscwHZ/bgc9lN62bzFn0hSYnoVdKNQNvAB4yv1fAq4BHzFO+Cbwll8cQDNZX+/i337kFULzzX57PWh6ZyonOEJvqjAxyOlaml2rfdIejrKlwJz+OWiWW883qf368m8aAJ6P6IBupg81maiaxBpv94KU2yj0Orjc992vqy7kyMJa1a/eZM32srfIm55Xngz3rK0nouTetno71u91Ymy62u5uDHOsYzqgcsgZzWf57jd+VIfTtQxFcdhs1vnTRfeW2OgZGJ5LibdXAb86SxW5ZYzwnUvdmvTIwZoi5mYU67DY21Pg42x2mbXCMpkrvrDZFQyDdOkkkNIevDnHd2tnXSfJFU9DLwOhEWt09wE+PGoPc3rgrP7bNQqktdycz+mLW0EPuGf3fAZ8CLA+gGhjSWlu/8TagKcfHEEw21vp5+HduJp7Q/PdHjsx6rlVxk/V+sgn9cCS5EAup3uvcC7Kj0RjPnO3lNTvq51VCZgn94Nhksplkejmm1YV5pnuEOzfXJBc0t5oTAqeP0p2MJ3jufB93Zhl7kAvXmb76QufenOsZQalMod/VHGBsIp6xRtIxNI7TrpKZc43f2GUrteOzY2ichqAno+PYuGZ48lRP8rGrfa5kh3EqLoeNaxoq0sprrwwYpZWpv7fNa8o52zNCe5bxxNNpnNY0dbF/lFAkxp4CCj1k1tL/+HAHO5sCWat1CkFduYfu4Qgj0VhRJ1dCDkKvlHoj0KO1Pph6OMupWWvJlFIfVkodUEod6O3NfSf51cLmNeXcf10jJzpCM7Z994aj9ISjWf15MBaUGgKetBG63eF0obdK4eZTS//0mV4mYgled+3ctg2A12nHZbcxND4x48CnVJG6K2WUgdWBOL3y5qXLg4xOxHlFHm0bMJpkNtT4eOnywnz6870jNAW9eKc1yVhdr9MnWXYMjVMfmBLxGrMOO3XmTPvQeLJBKJUqn4vr1gZ5yuw1yFZxkxZDU4Bj7VPPnysD48nSSostdX7aBse50Ds6Y7OURWPQw0g0lrThDplrGrsLJfSmlfWrk1PTRi/3j3K4bbgoto1Fbbmbi/2jaF3cOTeQW0Z/O/BmpdQl4DsYls3fAUGllPWqbQY6sv2w1vpBrfVerfXe2tr8vjhLnQ21fsbNkrhsnOxM74jNxqY6f3KErtaabrMr1qK+Yv7dsT871kWVzzVr81EqSqnkGARr4JPfPb2OfiqWu7ZM1eU3V3rxux2cmlZ58+MjHXicthnHHuTCdeuCHLq6sN2nzpkDxaazocZHwOtk//n0fQY6pol4TXlm05RVQ5+NV26t40jbEH0jUc7OUHFjsbM5wEg0ZoqQps3M6FOx3ijC0diMC7EWyaYpM6M+dHUIn8u+4Fkyi2XP2iB3b63lc4+d4m9+cRqtNT8xd/R6Q5FsGyBtk/AV69FrrT+jtW7WWrcA7wCe0Fq/C3gSeJt52vuAH+YcpZDGRvOj6IUZumVPzEPoN9b6Od8zYuxnOR4jMplIy+i9LjvBMuecGX00FufJUz28+po1WeuJZ8KaYBkaj+F3O5LWTOrje512ttWXJ20kMN4kttaXp41CiEzG+dGhDl67o35JqhuuX1dJ38jEjPXvL18Z5HOPnUpONUwkNBf6RtIqbixsNsWrt6/hlye706YgdgxF0jLnZHmeKfST8QTdoQhNwfSNXCzu3lqL1vDIwTZCkdjsGb3ZeXq0bdhYJ4nGMjL6TSlvFHMJfbKW3rT5DrcNsas5uKDnQy447Db+5b17+a29zfzDE+f45PeP8KNDHdywvnLOTyNLSW3K7lWlUHUznU8Df6CUOofh2X91CR5jVbPBFJCLfdkXZK0Z9KldkdPZVOdndCJO53CE7nB6s5RFfYUnI6OPxuJpO1X95nw/4WiM1167sI5Da95NeNpAs1TevLuRD9zeknF8a305pzpDyQz7Vye7CUVivO2G5gXFMF9u3Wh8Snj0pczJmYmE5jM/OMpXnj7Pf/vXl4jG4rQPjROZTMyY0b5hZwPhSCw5rM7acCS1mqa23LCurIywOxQhoWeuZ7+2MUCN38XX910EmDWb3mTOsznSNpwsrZye0a+vLsNpN4R6Lo8+NaOPTMY52Rniujka1vKN027j8w/s4uP3bObRl9o43R3mTbuKZ9tAutCv9MVYALTWT2mt32h+fUFrfZPWepPW+u1a69x3UBDSWFPhxueyzzj/xppBPxuplTfTu2ItjFr6qSw2ntC84UvPsvf//RWfeuQwT53u4T+PdOJ3O5IjledLwMroZ6lI+PzbdvHbN67LOH5NfTmhSCxpXT16sI2GgGfBMcyXjbV+XrtjDV979mLGXJlfnOjmVFeY1+2o54lTPXzs2y8lP1FtnEFsb99UQ7nHwU+PdgFTG46k2jJT826MpqmOaXX207HZFHdtqUvuEjabdeOw29jRGOBo+9CMQu+025KbcM+V0deVGzsqdQ6Pc7wjxGRcF6ziJhWlFL//6i18/oGd7G4O5K1JarHUpWX0K9S6EYqHUorWWh8XssxgGZuIcaF3ZM5NFdKF3myWqkgv22sIetOsm1+e6OZczwg7Git47GgX7//6izxysC3ZzLQQgl4nw+OGdbPQF4E1KfBUZ5ieUISnz/Ty1j1NS2oVfOLeLYSjMb767IXkMa01X3r8LBtqfPzTu67n/3nLtfzqZA+fftSoiMpm3YBR+fLq7Wv4xfEuY0ONocyO14DXmbbTlFVnP5PQA7xym7HWFSxzUuOfvU9xp7kga43/zSbmm9eU47LPXkMPxhtHfYWHjqEIh8xmsGIIvcVv37iOH/7uHck3y2KRltEXeTG2uG8zwqJprfFnbc0/2RkioZmznr3a5yJY5uRsz0jyRZ6R0Vd46BuZIBqL43bY+dq+izQFvXzrgzcR15pfn+njqTM9vCNL1j0Xlkcf9k9mbCA+F1aX48muEGfMmfsPLJFtY3FNQwX3XVvP1/Zd4oN3tBIsc/Grkz2c6AzxN2/fjd2meM8t69Fa88c/PE61z5UxaTGV11/bwA9eamff+b5kiWmq/66Uotrnps+0bpIZ/SyDwe7cVItNGY1Sc5WY7moO8I3fXOKpM73U+F343JlS8MHbW7hxfeWMG8ik0hD00jE0zkQ8QUPAk/FcWo3ULqOMXoR+hbKhxsdPjnQQmYynZdPH2g3b4Nqm2TN6pRSbzAVZp10R8DozsnJrEbR72Gj6eOHiAJ99/TU47DYcGAOeFjsNMFjmYnwyTm84OmPmOxMBr5OmoJdTnWFOdYXYsy6YUa++FHzi3i387HgXD/36In/4mi186fGzrKsq4/7rpiyC997aQpnLMWcn7Z1baih3O3jsaGeyznv6dMeacldKRj9Olc+VUa6ZSqDMyYfuaJ3X78JakD14eXDGAXA3rK/ihvXzq6RqCHg41j5M29BYUbP55YS1SXhCi9ALi2RDrQ+tjTnd2+qnRP1Y+zA1fleyPHI2NtX5+cWJbip9zgzbBtJnjX/3wFXKXHZ+68a1eYnfaprqDkUXVZGwtb6cJ071MBKN8RdvvTYvMc3nMV+/s4Gv77tIa42Po+3DfOGBXRkVQ/NZFHY77Ny7fQ2/ONHN63bUEyxzZmTVxhgEy6MfT5txMxOffcP2eV1La40fn8vO6EQ8w59fDI1BL48d6yKe0Lz75vU5318pYG0SHhqfxO0o3qYjIB79isXK2qaXWB5tH2ZHY2Be3aGb6vwMjE5wqiuc9aO2ldEfbR/mx4c7ePsNzbNuKLIQrHk3wKIm+22rL2ckGsPlsBW0xf0T92xmbDLOpx49QnOll7dev/jG7/uurWdobJKfHu3MasmkzruZXmefK3abYodp762do6pmPjQEPMmxDpLRT1Hrdxe9tBJE6Fcsrcla+qkSy8hknLM9I/OaNwNTVSGX+8dmFfovP3WeWELPuPXbYgimlH4upvRsq9kh+5rta/L25jMfNq8p5027GoknjG0IcxmH/IottfjdDkKRWNaySUvotda0D47PWFq5WKznSb4yegCbIusevKuVNRXuoo8oBrFuViw+t4P6Ck9aRn+qy9hmby5/3iLVG89m3fjdDso9DvpHJ7j3mrrkm0s+SBXnxWQ8e1uqCHidvPfWlrzFNF8+8/pttFSX8cD1uS0Ae5x27rmmjh8e6sja2FNb7mYyrrk6MM7oRDzvzT+WT78uD0PgrE8bW9aUZ13YXa18/N4tDI1lbvJeaOQvsoLZUOvjfEqJ5TFzUNV8M6qmoBev0874ZHxGT78h4CEcGeGDeczmYVpGv4iMpyno5fCfvCafIc2bhoCXP3jN1rzc133XNvDDQx1Z/XerRNLaqGS20srFPvbk2zU3zXN0xWw0mPHPtbPXamO52Fhi3axgNtT6uNA7kuwQPdY+TLDMOe/Mz2ZTbKg1svS6GYR+c105O5sCye7QfBEsm/Lol4OHWSzu3lrLO25cm3X7Rat+3dqoJN9C73LYeNsNzfMqn5yLap+L99/WkrXBTSg+ktGvYDbU+AlHYvSPTlDjd3OsY5hr57kQa7Gpzs/xjtCMdc9//fbdJLTO+w72Ppcdp10xGddFH+FaTDxOO597YFfW26zBZtYs/PlU3RQLpRR/+uYdxQ5DmAHJ6FcwVjZ+oXeUaCzO6a7wghfCrF2IZrJuvC77kniuSikCZuVNsbsGlytWZ+exjuGsG44IwnxZvalUCTBVYjlCmcvOZHz+C7EW77hpHbXl7rQJkYUiWOakbyRa9GaS5UrQ68RuU4xNxFlfXZYXi0VYncgrbAXTGDT29rzQN5rc3WW+pZUWNX530XzVoJnJF3uy33LFZlNU+1z0hKNFHbcrrHxE6FcwdpuitdpYkB2NGsPB8lETXSiCZU5cdtuCB6KtJmr8bnrC0bwvxAqrCxH6Fc6GWh+nu8L0hqMLXogtNtU+d1qZpZBJbbkbOvNfcSOsLmQxdoWzodbHlYExTnaFF+zPF5uPvXIT//DOPcUOY1ljLcjOtLOUIMwHyehXOBtq/MQSGhJ6xbWer6suy0tXZilTY+40JRm9kAuS0a9wrBJLkBkjpYjVNCVCL+SCZPQrnA01Romlz2WntTp/s2iE5cF9OxsYHp+Uv62QEyL0K5xAmZNqn4uNtX6psy5BmoJe/jBPc3WE1YsIfQnwR/dtm3FWjSAIggh9CfD2vfnZ9UkQhNJEFmMFQRBKHBF6QRCEEkeEXhAEocQRoRcEQShxROgFQRBKHBF6QRCEEkeEXhAEocQRoRcEQShxlNZ67rOWOgileoHLi/zxGqAvj+EUA7mG4rPS44eVfw0rPX4o/DWs11rXznXSshD6XFBKHdBa7y12HLkg11B8Vnr8sPKvYaXHD8v3GsS6EQRBKHFE6AVBEEqcUhD6B4sdQB6Qayg+Kz1+WPnXsNLjh2V6DSveoxcEQRBmpxQyekEQBGEWlrXQK6Vep5Q6rZQ6p5T6oyy3u5VS3zVvf14p1ZJy22fM46eVUq8tZNzTYlzUNSilqpVSTyqlRpRS/1jouFPiW2z8r1ZKHVRKHTX/f1WhY0+JcbHXcJNS6pD577BS6q2Fjt2MY9GvA/P2debz6JOFink6OfwNWpRS4yl/h68UOvaUGHPRo11KqeeUUsfN10RhdwrSWi/Lf4AdOA9sAFzAYWD7tHM+CnzF/PodwHfNr7eb57uBVvN+7CvsGnzAHcBHgH9cgX+DPUCj+fW1QPsKvIYywGF+3QD0WN+vhPhTbn8U+D7wyRX4N2gBjhUj7jxegwM4Auw2v68utB4t54z+JuCc1vqC1noC+A5w/7Rz7ge+aX79CHCPUkqZx7+jtY5qrS8C58z7KzSLvgat9ajW+lkgUrhwM8gl/pe11h3m8eOARynlLkjU6eRyDWNa65h53AMUY0Erl9cBSqm3ABcw/gbFIqdrWCbkcg2vAY5orQ8DaK37tdbxAsUNLG/rpgm4mvJ9m3ks6znmC3IY491yPj9bCHK5huVAvuJ/AHhZax1dojhnI6drUErdrJQ6DhwFPpIi/IVi0fErpXzAp4E/K0Ccs5Hr86hVKfWyUupppdSdSx3sDORyDVsArZT6uVLqJaXUpwoQbxrLec/YbO/m0zOqmc6Zz88WglyuYTmQc/xKqR3A5zGymmKQ0zVorZ8HdiilrgG+qZR6TGtdyE9ZucT/Z8AXtdYjRU6Oc7mGTmCd1rpfKXUD8B9KqR1a61C+g5yDXK7BgWHD3giMAY8rpQ5qrR/Pb4gzs5wz+jYgddfrZqBjpnOUUg4gAAzM82cLQS7XsBzIKX6lVDPw78B7tdbnlzza7OTlb6C1PgmMYqw3FJJc4hMh8+UAAAFXSURBVL8Z+IJS6hLwCeB/KKV+d6kDzsKir8G0X/sBtNYHMXzyLUsecSa56tHTWus+rfUY8FPg+iWPOJViL3LMsvjhwPAWW5la/Ngx7ZyPkb748T3z6x2kL8ZeoDiLsYu+hpTb30/xFmNz+RsEzfMfWMHPo1amFmPXY7ywa1ZK/NPO+VOKtxiby9+g1nrtYiyEtgNVK+waKoGXMBf3gV8Bbyho/MX4wy/gl/t64AzGu/hnzWN/DrzZ/NqDUU1wDngB2JDys581f+40cN8KvYZLGBnBCEZWsH2lxA/8T4wM+FDKv7qV9DcA3oOxiHnIfKG+ZSXFP+0+/pQiCX2Of4MHzL/BYfNv8KaVdg3mbe82r+MY8IVCxy6dsYIgCCXOcvboBUEQhDwgQi8IglDiiNALgiCUOCL0giAIJY4IvSAIQokjQi8IglDiiNALgiCUOCL0giAIJc7/D4hAQxapZDAcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# first plot the whole graph\n",
    "plt.plot(lrs[:80], losses[:80])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Trainning with Triangular learning rate policy.\n",
    "\n",
    "Before training with this policy you have to estimate the range of learning rates using previous function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.0, 0.1111111111111111, 0.2222222222222222, 0.3333333333333333, 0.4444444444444444, 0.5555555555555556, 0.6666666666666666, 0.7777777777777777, 0.8888888888888888, 1.0, 1.0, 0.8888888888888888, 0.7777777777777778, 0.6666666666666667, 0.5555555555555556, 0.4444444444444444, 0.33333333333333337, 0.22222222222222232, 0.11111111111111116, 0.0]\n"
     ]
    }
   ],
   "source": [
    "def get_triangular_lr(lr_low, lr_high, stepesize):\n",
    "    delta = (lr_high - lr_low)/(stepesize -1)\n",
    "    lrs1 = [lr_low + i*delta for i in range(stepesize)]\n",
    "    lrs2 = [lr_high - i*delta for i in range(stepesize)]\n",
    "    return lrs1+lrs2\n",
    "lrs = get_triangular_lr(0, 1, 10)\n",
    "print(lrs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_triangular_policy(model, lr_low=1e-5, lr_high=0.012, train_dl=train_dl):\n",
    "    idx = 0\n",
    "    epochs = 4\n",
    "    stepesize = 2*len(train_dl)\n",
    "    lrs = get_triangular_lr(lr_low, lr_high, stepesize)\n",
    "    for i in range(epochs):\n",
    "        model.train()\n",
    "        total = 0\n",
    "        sum_loss = 0\n",
    "        for i, (x, y) in enumerate(train_dl):\n",
    "            optim = get_optimizer(model, lr = lrs[idx], wd =0)\n",
    "            batch = x.shape[0]\n",
    "            x = Variable(x).cuda().float()\n",
    "            y = Variable(y).cuda().float()\n",
    "        \n",
    "            out = model(x)\n",
    "            loss = F.l1_loss(out, y)\n",
    "            optim.zero_grad()\n",
    "            loss.backward()\n",
    "            optim.step()\n",
    "            idx += 1\n",
    "            total += batch\n",
    "            sum_loss += batch*(loss.data[0])\n",
    "        print(\"train loss\", sum_loss/total)\n",
    "        val_loss(model, valid_dl)\n",
    "    return sum_loss/total"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def val_loss(model, valid_dl):\n",
    "    model.eval()\n",
    "    total = 0\n",
    "    sum_loss = 0\n",
    "    for i, (x, y) in enumerate(valid_dl):\n",
    "        batch = x.shape[0]\n",
    "        x = Variable(x).cuda().float()\n",
    "        y = Variable(y).cuda().float()\n",
    "        out = model(x)\n",
    "        loss = F.l1_loss(out, y)\n",
    "        sum_loss += batch*(loss.data[0])\n",
    "        total += batch\n",
    "    print(\"val loss\", sum_loss/total)\n",
    "    return sum_loss/total"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Darknet([1, 2, 4, 6, 3]).cuda()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train loss 47.81492664769332\n",
      "val loss 50.820497226941974\n",
      "train loss 34.456719563257614\n",
      "val loss 106.66324394643543\n",
      "train loss 32.98741410832325\n",
      "val loss 42.03031718702566\n",
      "train loss 30.40088057217885\n",
      "val loss 28.03747675364023\n",
      "loss  30.40088057217885 Time elapsed 0:23:23.546833\n",
      "train loss 29.866798144145886\n",
      "val loss 30.65906181456455\n",
      "train loss 31.41654774672166\n",
      "val loss 165.85775676972332\n",
      "train loss 30.489497582235874\n",
      "val loss 28.16718803635672\n",
      "train loss 26.333746526035473\n",
      "val loss 23.430043296904717\n",
      "loss  26.333746526035473 Time elapsed 0:23:09.276457\n"
     ]
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "steps = 5\n",
    "for i in range(steps):\n",
    "    start = datetime.now() \n",
    "    loss = train_triangular_policy(model)\n",
    "    end = datetime.now()\n",
    "    t = 'Time elapsed {}'.format(end - start)\n",
    "    print(\"loss \", loss, t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Darknet([1, 2, 4, 6, 3]).cuda()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "#training with different sizes\n",
    "train_ds = BoneAgeDataset(train, )\n",
    "valid_ds = BoneAgeDataset(valid, transforms=False)\n",
    "batch_size = 16\n",
    "train_dl = DataLoader(train_ds, batch_size=batch_size)\n",
    "valid_dl = DataLoader(valid_ds, batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "val loss 127.41255388857353\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "127.41255388857353"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val_loss(model, valid_dl)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss  36.051943618769876\n",
      "val loss 2230.887192704203\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:58.174143\n",
      "loss  34.26247251209838\n",
      "val loss 67.41548247492379\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:58.656578\n",
      "loss  32.56086358750867\n",
      "val loss 35.08560381079363\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:00.517020\n",
      "loss  30.502686672870485\n",
      "val loss 43.3300501213104\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:59.409675\n",
      "loss  29.400802684144562\n",
      "val loss 29.88208171169877\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.639025\n"
     ]
    }
   ],
   "source": [
    "train_loop(model, epochs=5, lr=0.1, wd=0.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss  26.790056487072185\n",
      "val loss 36.53312387247108\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:02.305601\n",
      "loss  24.897395214602973\n",
      "val loss 23.333194682751262\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:59.239170\n",
      "loss  23.117923590623892\n",
      "val loss 22.78447825789546\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:00.757716\n",
      "loss  22.058179287244524\n",
      "val loss 58.56091494337138\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:00.380470\n",
      "loss  20.83642287097893\n",
      "val loss 53.05122436596041\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.268299\n"
     ]
    }
   ],
   "source": [
    "train_loop(model, epochs=5, lr=0.05, wd=0.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss  18.769728582032265\n",
      "val loss 18.231157384535134\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.891472\n",
      "loss  18.222848597101276\n",
      "val loss 17.53739280156159\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.011842\n",
      "loss  17.988008011957735\n",
      "val loss 17.346115292300315\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:00.574137\n",
      "loss  17.778114131685882\n",
      "val loss 17.242611563650037\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:02.567720\n",
      "loss  17.752681375246997\n",
      "val loss 17.099167015323744\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.129414\n",
      "loss  17.53652510849545\n",
      "val loss 16.891538346220255\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:59.387807\n",
      "loss  17.33721687059027\n",
      "val loss 16.997812402711606\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:03.421243\n",
      "loss  17.271966746853273\n",
      "val loss 16.85184250815722\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.769020\n",
      "loss  17.21004746862724\n",
      "val loss 16.730244767372803\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.548436\n",
      "loss  17.105595987489906\n",
      "val loss 16.693165075013223\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.735805\n"
     ]
    }
   ],
   "source": [
    "train_loop(model, epochs=10, lr=0.001, wd=0.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "def save_model(m, p): torch.save(m.state_dict(), p)\n",
    "p = PATH/\"model001.pth\"\n",
    "save_model(model, p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss  17.026376719516858\n",
      "val loss 16.488996053479383\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.077558\n",
      "loss  17.05449960697086\n",
      "val loss 16.571430705446367\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:59.207299\n",
      "loss  16.921460427216086\n",
      "val loss 16.4304956198684\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:59.168719\n",
      "loss  16.91544908630752\n",
      "val loss 16.34874444382234\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:59.567036\n",
      "loss  16.76635766613057\n",
      "val loss 16.39350597448145\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:00.028754\n",
      "loss  16.79882159059638\n",
      "val loss 16.446618893144624\n",
      "Time elapsed (hh:mm:ss.ms) 0:04:59.276431\n",
      "loss  16.706213208229837\n",
      "val loss 16.430429531221442\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:00.856810\n",
      "loss  16.741053224023478\n",
      "val loss 16.340888287319633\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:02.765025\n",
      "loss  16.68847453359074\n",
      "val loss 16.393545072103283\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:02.099902\n",
      "loss  16.723176729881157\n",
      "val loss 16.34010503407417\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:01.478853\n"
     ]
    }
   ],
   "source": [
    "train_loop(model, epochs=10, lr=0.0005, wd=0.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = PATH/\"model0005.pth\"\n",
    "save_model(model, p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss  16.450828810491533\n",
      "val loss 16.127390656558227\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:14.115557\n",
      "loss  16.527406403065815\n",
      "val loss 16.199378974091715\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:15.365699\n",
      "loss  16.475459117059465\n",
      "val loss 16.06067849566878\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:11.197584\n",
      "loss  16.44754842058118\n",
      "val loss 16.14038661847882\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:05.119362\n",
      "loss  16.382451371166372\n",
      "val loss 16.104058384422647\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:13.382352\n",
      "loss  16.265305758160636\n",
      "val loss 16.11882994195755\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:16.239145\n",
      "loss  16.266914886057183\n",
      "val loss 16.10359597792236\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:16.254050\n",
      "loss  16.171423736702334\n",
      "val loss 16.036041109265078\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:16.450352\n",
      "loss  16.185297592152498\n",
      "val loss 16.050088778079076\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:15.882275\n",
      "loss  16.2207181771549\n",
      "val loss 15.984323851746476\n",
      "Time elapsed (hh:mm:ss.ms) 0:05:05.228401\n"
     ]
    }
   ],
   "source": [
    "train_loop(model, epochs=10, lr=0.0003, wd=0.000001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving and loading "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def save_model(m, p): torch.save(m.state_dict(), p)\n",
    "    \n",
    "model.load_state_dict(torch.load(p))\n",
    "p = PATH/\"model001.pth\"\n",
    "save_model(model, p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "\n",
    "* https://www.16bit.ai/blog/ml-and-future-of-radiology\n",
    "* https://stanfordmedicine.app.box.com/s/vhq1zop1867gr9rwnan4byj8lfxue173\n",
    "* https://github.com/fastai/fastai/blob/master/courses/dl2/cifar10-darknet.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}