{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "**Important: This notebook will only work with fastai-0.7.x. Do not try to run any fastai-1.x code from this path in the repository because it will load fastai-0.7.x**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## CIFAR 10" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "%reload_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can get the data via:\n", "\n", " wget http://pjreddie.com/media/files/cifar.tgz", " \n", "**Important:** Before proceeding, the student must reorganize the downloaded dataset files to match the expected directory structure, so that there is a dedicated folder for each class under 'test' and 'train', e.g.:\n", "\n", "```\n", "* test/airplane/airplane-1001.png\n", "* test/bird/bird-1043.png\n", "\n", "* train/bird/bird-10018.png\n", "* train/automobile/automobile-10000.png\n", "```\n", "\n", "The filename of the image doesn't have to include its class." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "labels.txt test train\n", "airplane automobile bird cat deer dog frog horse ship truck\n" ] } ], "source": [ "from fastai.conv_learner import *\n", "PATH = \"data/cifar10/\"\n", "os.makedirs(PATH,exist_ok=True)\n", "\n", "!ls {PATH}\n", "\n", "if not os.path.exists(f\"{PATH}/train/bird\"):\n", " raise Exception(\"expecting class subdirs under 'train/' and 'test/'\")\n", "!ls {PATH}/train" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')\n", "stats = (np.array([ 0.4914 , 0.48216, 0.44653]), np.array([ 0.24703, 0.24349, 0.26159]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def get_data(sz,bs):\n", " tfms = tfms_from_stats(stats, sz, aug_tfms=[RandomFlip()], pad=sz//8)\n", " return ImageClassifierData.from_paths(PATH, val_name='test', tfms=tfms, bs=bs)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "bs=256" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Look at data" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "data = get_data(32,4)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "x,y=next(iter(data.trn_dl))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGfRJREFUeJztnV2MJNd133+nq7vnc2e5y10uh8sll5ZoQ7IdUcqGFqBA\nUKzEYAQDlIDYkB4EPgheIzCBKHAeCAWIFCAPchBJ0JOClUWYDhR9xJIgwjASC4QDwi8UVxJFUWZs\nkQxlLbne5ceu9mt2prvr5KGbxpKuc2Y4M11N5v5/wGC66/atOnWrTlX1/fc5x9wdIUR5dGZtgBBi\nNsj5hSgUOb8QhSLnF6JQ5PxCFIqcX4hCkfMLUShyfiEKRc4vRKF0d9LZzO4CPg9UwB+6+6c3+bx+\nTijElHF328rnbLs/7zWzCvgb4F8Ap4BHgY+4+18lfeT8QkyZrTr/Th777wSecvdn3H0D+Cpw9w7W\nJ4RokZ04/2HgZ9e8PzVZJoR4E7CT7/xNjxb/4LHezI4Dx3ewHSHEFNiJ858Cjlzz/mbg+dd+yN1P\nACdA3/mFeCOxk8f+R4Hbzew2M+sDHwYe3B2zhBDTZtt3fncfmtm9wP9iLPXd7+4/3jXLhBBTZdtS\n37Y2psd+IaZOG1KfEOJNjJxfiEKR8wtRKHJ+IQpFzi9Eoewoqu/1smffzRx7/79tbOt5L+xXUzcu\nt+TS1e3FuzYYDMM2S9SPxf5c0LI9ESMTWqrEfqp4x60TTPQm2xqNRnFbItAMPB7HaOess737jSUT\n2FUnPnei8fC6+ZwCMIu3tUE8Vn2L2zqd+HiOfNC4/E//8N6wz26gO78QhSLnF6JQ5PxCFIqcX4hC\nkfMLUSitzvYPRx3OXV5obKuS65AFU9UeqACQz+aO45CClmi2HLh4qXk2N5kcxhrTHrxCPJM+YiPu\nVcXrzGaqw/UlSkCdyQTJWEVSTH5c4vXViZH1aD1ZZ9AvGadsPEbJ2Pc6merTPKMPUHEx6Tc9dOcX\nolDk/EIUipxfiEKR8wtRKHJ+IQpFzi9EobQq9WEOvWa5rK5juaYXmNlJInuGo1h2SYM60qCZbQTw\nbEN6A6jSfsm+RUFQmeSYNI2ycUxs7PeiYJtYZs0YjOJjNqwyyTdsSbYW71c3kZez02Ng8X570jZN\ndOcXolDk/EIUipxfiEKR8wtRKHJ+IQpFzi9EoexI6jOzZ4GLwAgYuvux9PMO1TCQ+hK5o7ZmM9Ni\nQ9llLZGosii2cHuZUpPIaN0sujBZJSS584Idr+tkv5IcfssLWe65eJ2XN5pttCTfHh7bkZEWqAmO\ndaakZueVpdGn8XHpJDkqjaV4g1NkN3T+f+buL+7CeoQQLaLHfiEKZafO78Cfm9n3zOz4bhgkhGiH\nnT72v8fdnzezG4DvmNn/cfeHr/3A5KJwHKA/v2+HmxNC7BY7uvO7+/OT/2eBbwF3NnzmhLsfc/dj\n3f7yTjYnhNhFtu38ZrZkZnteeQ38BvDEbhkmhJguO3nsPwR8a5Iwsgv8d3f/n3kXD8theacf9hoF\n0XuWlE5K82amEV1xWyfQhzxJZGlJ5GEnK0GV7MAwkD4B6uFa4/J9y7HU9I5f/qWwbX0jjrb862dO\nxf3qZrFykOxzmnz09at5KdtJdApQp1JfTKfOjnW7wbWvsO2tuvszwDt20RYhRItI6hOiUOT8QhSK\nnF+IQpHzC1Eocn4hCqXlBJ6GdYN4tW1IL1UnuXaltdhiOS+qCwhg4fbiGLxucn2dS+q3YUmiyF58\n2G675VDj8n/yq0fDPrfcvD9s++6jPw7bbCO2cak/17j8chKQmAqwcSbONOlqdKzzcyA+d7KksZ3k\nWNfJOuPzarrozi9Eocj5hSgUOb8QhSLnF6JQ5PxCFMobZrY/y40WBcBYMiNuSbCNezw7n028RjO9\nRvPMNkCSsY6FxP75+Tho6egvNs/oA/zaL9/SuHz/UnyoN5Lcc1Un7pdUyWKu37znG8ks+yAphWWJ\nFOD++u9h2Wx/2i+Z7bckeKeTnFjJqTpVdOcXolDk/EIUipxfiEKR8wtRKHJ+IQpFzi9EobQu9VE1\nS0CWmBJeoRKJxzMZsMokmSTgI2jyJKfecBTLaHtXYiHwvb/2trDtxZefDdt8/aXmhqXr4j4W27G8\nEI/VzTeuhG0//XmzVNlLAlzqJCWjJ5Jjdh6EXdKaXLGNo8T+KslD6cn5XdlGbMsU0Z1fiEKR8wtR\nKHJ+IQpFzi9Eocj5hSgUOb8QhbKp1Gdm9wO/CZx191+ZLNsPfA04CjwL/La7n9t0XRhWNcshlsk1\ngfyW5Vpzj3WjTOojiXCjbpYPg8pUYzssXt9Nh/eEbb/6S3Hk3lM/eTlsO/Wz5hJa/fmFsE+nm0QX\nduM8g3V1JV7nUvO+LXtSauxCPFajRI7M0j9Gil6d5ATM4v3SfHt1ZmNyknSS8MgpspU7/x8Bd71m\n2X3AQ+5+O/DQ5L0Q4k3Eps7v7g8Dr73V3A08MHn9APDBXbZLCDFltvud/5C7nwaY/L9h90wSQrTB\n1Cf8zOy4mZ00s5OD9YvT3pwQYots1/nPmNkqwOT/2eiD7n7C3Y+5+7HeXDzBJYRol+06/4PAPZPX\n9wDf3h1zhBBtsRWp7yvA+4ADZnYK+CTwaeDrZvYx4G+B39rS1szodJoljyorkRRIIZ1M6kvLI4VN\neTmmIPGnJYkn5xMZ55abDoRtdb0ett169EjYRjC+z50KH844ePCmsG1tGI/Hi5djqW/UXWxcvnoo\nnh5a24htHAzj49lNsntGvVKZOGnLdMUqiUocJtJiJ4lAnSabOr+7fyRoev8u2yKEaBH9wk+IQpHz\nC1Eocn4hCkXOL0ShyPmFKJR2E3gS1yyzQKICsCAyztIkjIkRiTRUZVFbQaLO7jCOfLthb1zH7/AN\ncVLNKpF/sgDIW2493Lj81POxjPb0/30ybLt4NU4u6Ul0pI8uNy4/sBKfcmvXL4VtP33xUthGlUTM\nRez+qUNqRaLmZafcNNGdX4hCkfMLUShyfiEKRc4vRKHI+YUoFDm/EIXSqtRnBt1AHhptQ1/p5KkW\nUztCEt3Fqubh6hPLYbes7gvb9iwnySyTqD5L6tZZUPBudTWWFbG1sOnpnz4ftq0sxuLW/rnmqL59\nsZpH/9Y4yvH5cxfijkH9R4jvbnWQjBXAk/Oqm9bqyzKJZv22IVXuArrzC1Eocn4hCkXOL0ShyPmF\nKBQ5vxCF0u5sP07VDYJ0kln2Oph97WQlkDI7skteELwD4EHHlV68wrceToJ3OvGMfh2UKIM88CTK\nQZilibvtyNGw7erl2MarF/8ubDu4erBxeW8uNmRpZT5su2H/ctj2woU4sKobneLJ+GaDVSdSUXZc\n6iw3pGb7hRBtIucXolDk/EIUipxfiEKR8wtRKHJ+IQplK+W67gd+Ezjr7r8yWfYp4HeAFyYf+4S7\n/9mmWzOjEwU/ZAE1gbyyXYkkWh/kuQR92Cx7rR6IC5Aeuj6WqMybZc8xu3tdzsaqrmPZq+NxDaqD\nexbituV+43KLUxrSjZU+br/p+rDt0qVYcnRrtqNOxtdI6m5Z3GZZSa7k/H4jB/b8EXBXw/LPufsd\nk7/NHV8I8YZiU+d394eBl1uwRQjRIjt5trzXzB43s/vNLA5aF0K8Idmu838BeAtwB3Aa+Ez0QTM7\nbmYnzezk4GqSkEEI0Srbcn53P+PuI3evgS8CdyafPeHux9z9WG9+Zbt2CiF2mW05v5mtXvP2Q8AT\nu2OOEKIttiL1fQV4H3DAzE4BnwTeZ2Z3MC569Czwu1vZmAFVkAcvS6xnQd4/S8PzEju2GZk1322W\neW5djZ9oFhNpq5PJTZ7k6csi0qJtJVJTtrZuN5MIY6ly73KzDDjwK/HGRnEuwaMHYlnx787GOfzO\nRFW+LO7jSRm4TlJ3q5MU7MqiAavtlBvbBTZ1fnf/SMPiL03BFiFEi+gXfkIUipxfiEKR8wtRKHJ+\nIQpFzi9EobRcrsvo9po3mclv0SXKtnntSqW+RL46dF1zranDh+IknZ2klFcnkZuy67In0YCRpJfK\nV4kMSJIkdZAkBa16zfvmHu/z+no8Vnv68an61iPxr8vPPd38q9L15HQLqsONqePxyM5GT1pD+XvK\n6M4vRKHI+YUoFDm/EIUi5xeiUOT8QhSKnF+IQmlXY7C4Tl6nE+tGo26QwDORZJLVUUVJRIFeFcto\nR1f3Ni5fmI+voYMsqaMlCTzTZJDNSSkBhkGMniexe3PJeHTTGnNxv7pujoDsVXGWzgEXw7bhII4G\nvHF/83EBWD3XXMfvuXOxrEgivdWdOIFnJ5FTq0Qy7UrqE0K0iZxfiEKR8wtRKHJ+IQpFzi9EobQb\n2IPRq5oDO+oqKYMUBJ5UnThIpOdJfry6uewWwKGVxbDttpsONC7vBzkGIS//RVYWKskVl2UarAOV\nIA2cypSFRCXoBioMwGDjauPy/lJzcBRAPwj6AhitN8/aA1RJoNNtQZmvC2txia+1RIQZeZKLL8uG\nmAx/t5rNPVh3fiEKRc4vRKHI+YUoFDm/EIUi5xeiUOT8QhTKVsp1HQH+GLiRsf50wt0/b2b7ga8B\nRxmX7Pptdz+Xrwt6gSzWSXK0bQQKiiVSX5XIV/OdWDZ6600Hw7brFpprb/USNa+b5cfLApMSibBO\nAkhCSSmRmrIccr3kuKTJ7iwYf4vlzV4/3ueNYXzMNoZxma8Dy82BRKv7Y0n3Zy/F66uSgawSGTA7\nnr0kQGqabOXOPwR+393fBrwb+D0zeztwH/CQu98OPDR5L4R4k7Cp87v7aXf//uT1ReBJ4DBwN/DA\n5GMPAB+clpFCiN3ndX3nN7OjwDuBR4BD7n4axhcI4IbdNk4IMT227Pxmtgx8A/i4uzcnQ2/ud9zM\nTprZyfUrW+4mhJgyW3J+M+sxdvwvu/s3J4vPmNnqpH0VONvU191PuPsxdz82txjXsRdCtMumzm/j\niJAvAU+6+2evaXoQuGfy+h7g27tvnhBiWmwlqu89wEeBH5nZY5NlnwA+DXzdzD4G/C3wW5utqKqM\nlZXmTY6SslY2aJaURolsxChe35EbY5nn6OpC2FbVzXKTjbJyV0l0YZLXra5j+WfkSQRkEGlXVfG2\nsoi/4SgOccvKhlkg9WXKZzexcVTHsq6vx5Lj+qA5L+Dq9cthn5cux7Li+jCJckzupZZIhN0s4nKK\nbOr87v6XxCrx+3fXHCFEW+gXfkIUipxfiEKR8wtRKHJ+IQpFzi9EobSawNOBUdUsAQ2SpIlVkNjR\nRrHssmcu3rVfPBL/EnlpPpbRqlFzW1a2ypLItygxKZBG/GWyHUFTamNiRpKbNNrUeHtBpKB34uNS\nJQlBFxeSrSVlz4ZrzefbfFJ266Z9sdx76uW4pFiVuJMl0m0/O55TRHd+IQpFzi9Eocj5hSgUOb8Q\nhSLnF6JQ5PxCFEqrUh/U4JebDamaEy1CXEpusdtcDw7gH7/1xrDtpqV4t+eibKFA1Wm20RLRa5Qk\n2/RE/snq5/USibAOunUtSdJJc2JSyI/LMErSCWwEkm5Sjo9OIvVV3SQqcRRH4fW9eYMLV2Op7+BC\nfMzOx0PFRiI9d5NzpErrOU4P3fmFKBQ5vxCFIucXolDk/EIUipxfiEJpdba/shErnUuNbRvDJB+f\nN+dv278n7nLD3nh9o0vnw7bhqB+bEcwCV/1k9r0Tz4hbLwn2SKbFF6o4n10dKAGdZGZ+KbFjPrk9\nJJPi7O01j+NCNx7fitjGTFnoJkFcVZCDcGUxtqObBEFddzXe1vlLsergxMfMOslAThHd+YUoFDm/\nEIUi5xeiUOT8QhSKnF+IQpHzC1Eom0p9ZnYE+GPgRqAGTrj7583sU8DvAC9MPvoJd/+zbF0dr5kf\nNAf2dJLrUNVtblu7EOdu++73ng7blvtxIEU/DS5plmR6iVRWJVLf4kIs/8x14wiSyuIcc525aBxj\nGWo5kRUvXPp52OYey2Uvvdic6657IS6jtrQQBxH1Ezl1bi6249L6euPycxdfjNe3FBeU3ZdIhIOL\nV8K2jTrJ7zejW/BWdP4h8Pvu/n0z2wN8z8y+M2n7nLv/l+mZJ4SYFlup1XcaOD15fdHMngQOT9sw\nIcR0eV0PHGZ2FHgn8Mhk0b1m9riZ3W9m+3bZNiHEFNmy85vZMvAN4OPufgH4AvAW4A7GTwafCfod\nN7OTZnbyyuXmn/YKIdpnS85vZj3Gjv9ld/8mgLufcfeRu9fAF4E7m/q6+wl3P+buxxaX4proQoh2\n2dT5zcyALwFPuvtnr1m+es3HPgQ8sfvmCSGmxVZm+98DfBT4kZk9Nln2CeAjZnYH48JSzwK/u9mK\nvK6przTn3ZtbXgr7dXrNEXojiyW7jsVSWdLE2qhZigS4eKFZyulebZaTANZeOBO2LScS4eK+uKRY\nte/msG1hsXlM9u+NJapzZ54L266uxeWpfD5e5/krwVe8ZKxOnX0+bFsYxFLlQpKf8ELQ70py3xt1\nL4RtthCfp1Uvlp67ddxWvVGlPnf/S6DpLE01fSHEGxv9wk+IQpHzC1Eocn4hCkXOL0ShyPmFKJR2\ny3WNakaXmiWgfj82ZSmQr/Ysx9Ft6xfiJJ1rp+O2LJliZyOQ+i6thX1WLsRRcfP9WOrrWZyAdMFj\n2ah3vrlf9XI8vsuDuOzZ+stx9NtomJQAm2uWCEce32/W114O2y6OYomwTspdRXF2ew/fFtuxnpRf\nuxqfO4MqjljcGMYScj+qsTZldOcXolDk/EIUipxfiEKR8wtRKHJ+IQpFzi9EobQq9W0Mhjx3+qXG\ntkNBjTmAXtUsiS2vxFLf4nIccbbSjYv89btJTTU/0NxwNZF4kgQm66NYYhtl5duGcePVoOPly7GN\nVVL7bzgfRxeSJP4c9JqPjSUJUhc7B8O2zlps/5WNeDy6wXl1fhDLay/8/FzYdulcLNmNPJZn14fx\nfrMeS8XTRHd+IQpFzi9Eocj5hSgUOb8QhSLnF6JQ5PxCFEqrUl9NxWWua2y7fjWOsppbaI5i26hi\nucaT2m6D5JpXB7IiQC+QxGwplhWrA3G68u4oSeqYtPUt2e9g37wxDeOYwTCWqK6uJ3JkFUe/DYbN\nNg7qOBHn2nos561fiaP6Lp2Pa+RdGjSP49ooqe+3Fu8XxHX8hp14jNcT7bafRDpOE935hSgUOb8Q\nhSLnF6JQ5PxCFIqcX4hC2XS238zmgYeBucnn/8TdP2lmtwFfBfYD3wc+6u7xdC1Q1zXrQb67Hzz6\ng7Df4lLz7Ot1e+PSSf0k6GTtSlyCamkhruXVCcprZbPlWDwDXHncZsns/Lg2atDPm2eVs5JQg2RG\n//yleKxGFs9gzy82H5tuEPADMKrjYzYaJYEx3fiYzVfN2+vW83GfflLqjVgl+PlaHMTVSw7AQhJY\nNU22cudfB37d3d/BuBz3XWb2buAPgM+5++3AOeBj0zNTCLHbbOr8PuaVS1pv8ufArwN/Mln+APDB\nqVgohJgKW/rOb2bVpELvWeA7wNPAefe/zyF9Cjg8HROFENNgS87v7iN3vwO4GbgTeFvTx5r6mtlx\nMztpZicHG7NJWiCE+Ie8rtl+dz8P/G/g3cB1Zn9fGP1moLG4urufcPdj7n6s148ne4QQ7bKp85vZ\nQTO7bvJ6AfjnwJPAXwD/avKxe4BvT8tIIcTus5XAnlXgATOrGF8svu7uf2pmfwV81cz+E/AD4Eub\nrciswhaaAyMuEKuE7kGgxVoShHM1DiBZmouDba5uxAE1K/PN/eb7SXmnQRyQ0mn+pgTA3GL8lOTd\n+Jod5ayrkhyJG2ux1NdbjPMdXrwcy4B7VpqP89JSPPYeyJSQy4qe7Nv61eZz5NKFWC5du5LkC7wa\n5/CzJDAp6ca6xefcNNnU+d39ceCdDcufYfz9XwjxJkS/8BOiUOT8QhSKnF+IQpHzC1Eocn4hCsUy\neWXXN2b2AvDTydsDwIutbTxGdrwa2fFq3mx23Orucd2za2jV+V+1YbOT7n5sJhuXHbJDduixX4hS\nkfMLUSizdP4TM9z2tciOVyM7Xs3/t3bM7Du/EGK26LFfiEKZifOb2V1m9tdm9pSZ3TcLGyZ2PGtm\nPzKzx8zsZIvbvd/MzprZE9cs229m3zGzn0z+75uRHZ8ys+cmY/KYmX2gBTuOmNlfmNmTZvZjM/s3\nk+WtjkliR6tjYmbzZvZdM/vhxI7/OFl+m5k9MhmPr5lZnE10K7h7q39AxTgN2C8AfeCHwNvbtmNi\ny7PAgRls973Au4Anrln2n4H7Jq/vA/5gRnZ8Cvh3LY/HKvCuyes9wN8Ab297TBI7Wh0TwIDlyese\n8AjjBDpfBz48Wf5fgX+9k+3M4s5/J/CUuz/j41TfXwXunoEdM8PdHwZefs3iuxknQoWWEqIGdrSO\nu5929+9PXl9knCzmMC2PSWJHq/iYqSfNnYXzHwZ+ds37WSb/dODPzex7ZnZ8Rja8wiF3Pw3jkxC4\nYYa23Gtmj0++Fkz968e1mNlRxvkjHmGGY/IaO6DlMWkjae4snL8ptcqsJIf3uPu7gH8J/J6ZvXdG\ndryR+ALwFsY1Gk4Dn2lrw2a2DHwD+Li7X2hru1uwo/Ux8R0kzd0qs3D+U8CRa96HyT+njbs/P/l/\nFvgWs81MdMbMVgEm/8/Owgh3PzM58Wrgi7Q0JmbWY+xwX3b3b04Wtz4mTXbMakwm237dSXO3yiyc\n/1Hg9snMZR/4MPBg20aY2ZKZ7XnlNfAbwBN5r6nyIONEqDDDhKivONuED9HCmJiZMc4B+aS7f/aa\nplbHJLKj7TFpLWluWzOYr5nN/ADjmdSngX8/Ixt+gbHS8EPgx23aAXyF8ePjgPGT0MeA64GHgJ9M\n/u+fkR3/DfgR8Dhj51ttwY5/yvgR9nHgscnfB9oek8SOVscE+EeMk+I+zvhC8x+uOWe/CzwF/A9g\nbifb0S/8hCgU/cJPiEKR8wtRKHJ+IQpFzi9Eocj5hSgUOb8QhSLnF6JQ5PxCFMr/A0Nq4c6QHuIf\nAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.imshow(data.trn_ds.denorm(x)[0]);" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHIxJREFUeJztnWuM3Od13p8zszM7e1/uhcvl8ibRlChGkmWXVmyoDZy4\nNRQngGygSW0Ehj4YYRDEQA2kHwQXqF2gH5yituFPLuhaiBI4ttXYhoXmUhuqUiFFIpmSSOpCmff7\nLpfL5XLvs3M5/TDDhFq/z8vhXmZJv88PIHb4nnnnf+ad/5n/zPvMOcfcHUKI9MhstANCiI1BwS9E\noij4hUgUBb8QiaLgFyJRFPxCJIqCX4hEUfALkSgKfiESpWU1k83sSQDfAJAF8D/c/Su3ub9+TijE\nOuPu1sj9bKU/7zWzLIDjAP4NgIsAfgbgM+7+TmSOgl+IdabR4F/Nx/7HAZx099PuvgTgewCeWsXj\nCSGayGqCfwTAhVv+f7E+JoS4B1jNd/7QR4tf+FhvZgcAHFjFcYQQ68Bqgv8igO23/H8bgMvL7+Tu\nBwEcBPSdX4i7idV87P8ZgD1mdp+Z5QF8GsALa+OWEGK9WfGV393LZvZ5AP8bNanvWXd/e808E0Ks\nKyuW+lZ0MH3sF2LdaYbUJ4S4h1HwC5EoCn4hEkXBL0SiKPiFSJRVZfWlxkPbOoPjv/eJR+mckV27\nqG3s+nVqO/7uKWrLFvPUtmtHb3A8395O5xw7N0Vth9+9QG2Xr0xS29RsKTheqvLrTfYXfyD6T0Sm\noRLZ285WquHxiMpVjhwrk+EHy2az1FZcKvPHrIYf0yLrMfT+7cHxq8fH6JxfOG7D9xRC/FKh4Bci\nURT8QiSKgl+IRFHwC5Eo2u2/A4pkW/nGAt+V3RbJZtjcyXfgJzpaqc0ju8rludngeGkxPA4A3S3c\nyQ/t3URtk1s7qO1nRy8Fx0cnF+kcGN9Jz2S4j5XIrjhTAjxYjuKfrdyPmI8ru5ayo2VzEWUhR0I3\nsobL0ZVfiERR8AuRKAp+IRJFwS9Eoij4hUgUBb8QiSKp7w6YWwwnZ4xeuUHndOfPUltngb/3bt08\nQG2Xx8apbWxiNDje399D53R1tVFb62KR2tpzOWrzXwm3cHjj7bAECABXb/DklyWSoAMArREZsJwJ\nS6bOHw5mCxFb41Lae+ZFbE7EvpYCl3tbyNrfiX+68guRKAp+IRJFwS9Eoij4hUgUBb8QiaLgFyJR\nViX1mdlZADMAKgDK7r5/LZxab2JySKyB0fx8WPaausZr4C108WNV8/y9d3SWS2wnRsNyHgA8suf+\n4PjOvbvonM0jQ9Q2PcHr9L1x6Ai1DfaGT62PP/EgnXP8why1jV3la3xtkvs4XQq/oKVIBl7sHIif\nO7GJ3MTIt0ekvvzqpb610Pl/3d0n1uBxhBBNRB/7hUiU1Qa/A/iJmb1mZgfWwiEhRHNY7cf+J9z9\nspltBvBTM3vX3V++9Q71NwW9MQhxl7GqK7+7X67/HQfwIwCPB+5z0N333yubgUKkwoqD38w6zKzr\n5m0AHwfw1lo5JoRYX1bzsX8IwI/q0kILgL9w979dE6/WmWiGVcTYQbKseiKtsHyJS3bzZX6wM5d5\npuBifge1FfoeCY7PVsKtxgCgv2ULte18ICwdAsCN6Qq1nXz3neD4h/dzqe9ffKhAbecvXaa28as8\ny/Gl104Hx89e5gVNIwl/USmtWuVSn0XOOpbV197NC6RGZcUGWXHwu/tpAO9ftQdCiA1BUp8QiaLg\nFyJRFPxCJIqCX4hEUfALkSj3dAHPlXVbA6oRa6GFvx/u2NIdHO/t5hLVUivvTTc4PEht+au8H18u\nz+ednwhLi4VOPmdigp8G5Sov7vnQIx+itr728Jp0dvK16u/sorYtg/x1yeT4c9u+MyxxfvdHh+mc\n42fmqQ15vh6e4a91xriA6KT/X76bH6taLIUf6w4kQF35hUgUBb8QiaLgFyJRFPxCJIqCX4hE+aXd\n7V/pvN4C32UfIm2t8h28FdalhWlq6+/hO9+dXXzne2yCJ/20k1p9g9u20jn5DN9Vfu3oGWr7zX+1\nndoeenBfcPzkef54NyIJOju29lJbezd/zT72kT3BcVvic77554eo7Spp2QYAlo8k9lT5ddaMJPZs\n4ufA4iQ5r7TbL4S4HQp+IRJFwS9Eoij4hUgUBb8QiaLgFyJR7mmpL1ZrLUZLRA3pb+MtkjrbwpLY\nzFK4dRIAtOS4DOhVLjr29XIZcP7EWWp77dWwTPU3f/U3dM7WIV7D75FfCUt2AJAr7KK2vIUTaoa3\nc3nwythFapslrdIAYGjbALXNz4WTbR7ey/34d7/Fz6yf/OMJajs1yqVKeJ6ajJw+vb08YWlsnCQf\nxYpQLkNXfiESRcEvRKIo+IVIFAW/EImi4BciURT8QiTKbaU+M3sWwG8DGHf3h+tjfQC+D2AXgLMA\nftfdr6+fm8y5mIlnbWWdt5navW0ztb1v11BwPNfFpabe3k3U1tnGl2zTTp7R9fL/naK2RdJOqruH\nZ8Vt2zFCbdu3cv/bCvz0uX41/NyqkRetu6eP2i5d4BLb4FbeiqxK5NRKhdfbe9923n6tXA1nCQJA\n/gh/bseOXaO2bGtYXu5s4+dAeTEsR/od6N+NXPn/FMCTy8aeAfCiu+8B8GL9/0KIe4jbBr+7vwxg\nctnwUwCeq99+DsAn19gvIcQ6s9Lv/EPuPgoA9b/8s7IQ4q5k3X/ea2YHABxY7+MIIe6MlV75r5jZ\nMADU/9IfNbv7QXff7+77V3gsIcQ6sNLgfwHA0/XbTwP48dq4I4RoFo1Ifd8F8FEAA2Z2EcCXAHwF\nwPNm9jkA5wH8zno6SX2LyEYZliqF+Dte7wCXeYa2hW25HJcVd/Txx1uc4Vls6OJZYA9E5MiLs+E1\n2Tyyg87pKfBMxnxljtoWbyzfB/5npqdmg+PlEs/O6+kLt0MDgLnFcHsqAFgo8qKa1Ur41b46yWXW\nmdkZatuzgxdCXapwH6fGF6gtPxR+3pt6+XqUlsLr6Heg9d02+N39M8T0sYaPIoS469Av/IRIFAW/\nEImi4BciURT8QiSKgl+IRLlrCnjGZDtmykTmVCOKxxJ4Bc/JEs+Ym6iGs6zOvMMzztoefYjadgzy\nAo0GntH1kf3vp7a/e/VccDwzw2WoljzPcMtM84VcuM6lPi+GX5tMiT9ePsPlzU394YxKAPCIrLu4\ntBQcv0Gy4gAg28EzGQcHeHbk7jKX8x7czV/rzp1h+bASebzifFiC9diJvwxd+YVIFAW/EImi4Bci\nURT8QiSKgl+IRFHwC5Eod43Uh0ykgZ6F36MyGd7Prr3A5ZpqlUso75zj/dZyQ2EZpdDJZZxsHy+O\nCV53Em15bvzVj/Dn1tIR7rtXaOcSFSrhDDwAuDH+Lp+2wLPfWkmPwlyBF+kcGOQ9A2dKPAuvXOZS\nZV9f+Hib+ofpnK5u/pp18QROdNyYoLaRQd6zsbA5LOteLN6gc3qIujnXeKs+XfmFSBUFvxCJouAX\nIlEU/EIkioJfiERp+m4/2biHR96GWnLhGnODQ7vpnJGtvK1STzdXCc5fOkRtZy6EWy498TivqZdt\n4zbL8tp5hUhdvUIH9//Rx8LbvZv6uUIwP893+w/9v9PUdn2a725vGQ772NHLXzPk+XPu7uG2Ikne\nAYDennAdvP5NvMXafbsf48ea5W23Tp/7ObXNTnG1ojgVfm5dm9ronN96PJww9vxLR+ic5ejKL0Si\nKPiFSBQFvxCJouAXIlEU/EIkioJfiERppF3XswB+G8C4uz9cH/sygN8HcLV+ty+6+1+vypNIXk++\nEG55NTC8nc7pGeC2Bx/YS237PvgBahsfOxwcH+rh9eWuXOa183LdfPm7C1zmaW3nNiqxdfI5vb0d\n1Da87T5qm7hynNp6+sM15vLOk6quT/EEnZYsr+9XJVIwADg5rwqtvO5fe4Fn78zdmOfzOrmPhQ6+\n/mM3wlLrrmFex3HvzrBUWcg3rt43cuX/UwBPBsa/7u6P1f+tLvCFEE3ntsHv7i8D4GVahRD3JKv5\nzv95MztqZs+aGf/5mBDirmSlwf9NALsBPAZgFMBX2R3N7ICZHTIz/rtZIUTTWVHwu/sVd694rRn4\ntwA8HrnvQXff7+77V+qkEGLtWVHwm9mtNZA+BeCttXFHCNEsGpH6vgvgowAGzOwigC8B+KiZPYaa\nQHcWwB+s1hFj6X4AWrJh+cozXK6ZLfHHG5vkhc727OWtsB4ZCWdSbRvgWXbDvVzi6ckVqa1Y5Hus\nxWJYRgOAdiJh5Vu5bJTPcWlr+07ebmx87BK3jYez2Fo6eFbcli07qK1S4pl7lSq3tbSE5belRZ7J\nuDB7hdpmprmtszMi3W7m22Jvj10Nju/v5nUcFy6Fa01WqxU6Zzm3DX53/0xg+NsNH0EIcVeiX/gJ\nkSgKfiESRcEvRKIo+IVIFAW/EInS9AKeLMvKLNIHCWG5prTEUwFn53g23eUrPLMs18FtXf3h7LHp\ncBcvAEA5ks2VGeRtvnYOcoktN8sLZ05fuxwcv3xtis7paudS5Z599PdbWFzkEtvpE68Gx2OFM7eO\n7KS2t49yiS1f4JIv83H00gU6Z2meP6+eTVx+ayvwrL75JX5eVTPkBJrn7bquTYalvkqFn/fL0ZVf\niERR8AuRKAp+IRJFwS9Eoij4hUgUBb8QidJ0qY8SadbnlbAMWFzgkoyDZ8x5Bz/W3HyZ2qwtLIll\nI1LT2VGePXb1WkTKGeYy4MM7+6itnxR2LM1zeXD0PO/Hd2WCZ+61dw5T29DmB8OGargYKwCMXeQZ\nf6OXuW3X/Xw9pqamg+P5Fv6anT91ktr27H2A2jo38czJbJ7LgLls+Jy7P5IJOLkQlg5bchfpnOXo\nyi9Eoij4hUgUBb8QiaLgFyJRFPxCJMrds9sPXlfPyXtUcY4nS+RzPMGhtMRbLi0szlBbZym825+Z\n4wpBhezkAoB18oSacyfCiRsA0FXgL9tMV3hXedfWETpn3wDftX/lpX+gtrfe5O26tvaHW4AdPXKK\nzpm8zhWJTZv4bnlboZvaRsfDCU093XxOJpIbc30yXJsQAOaXeNuwrh5+vK65seB4pCQgenZvC463\ntr7DJy1DV34hEkXBL0SiKPiFSBQFvxCJouAXIlEU/EIkSiPturYD+DMAWwBUARx092+YWR+A7wPY\nhVrLrt91d66D3ISU3ePV+ICKhyW9pSWevFOK1Ewrl3g9u8V5XqMNxbBtZAeXcUZGeqkt41wGnJ3m\nCUEtMVm0EpbEzpzj7b927eRJRA88+qvUVinz+oQohhNx5iOJQsUql1kHhnkbtfkF7sccWUYjbbwA\noG+QJ9SUy7wd1lnSQgsATp7iyVN9fWGJcFMXP3fmFohcbfzcWE4jV/4ygD9294cAfBjAH5nZPgDP\nAHjR3fcAeLH+fyHEPcJtg9/dR9399frtGQDHAIwAeArAc/W7PQfgk+vlpBBi7bmj7/xmtgvABwC8\nAmDI3UeB2hsEgM1r7ZwQYv1o+Oe9ZtYJ4AcAvuDu09bgdwszOwDgwMrcE0KsFw1d+c0sh1rgf8fd\nf1gfvmJmw3X7MIDgboe7H3T3/e6+fy0cFkKsDbcNfqtd4r8N4Ji7f+0W0wsAnq7ffhrAj9fePSHE\netHIx/4nAHwWwJtmdrg+9kUAXwHwvJl9DsB5AL+zKk9YHy8AlUpY0qtWefpVuRyp4UceDwAqpUVq\nKxJ5pTXP68Ft3xbOvgIAj/h4Pc/lyOvXuI/TU+F1LETqDM6XeH08ROTUfA+XCEsz4RZUHb1DdE7P\nZl4Dr7OHZx6eO88lttnZ8Gs2OBiRKSPnQGuGr+PmIZ45WT7CpT6bCdeoPHNylM7pGQrLg+6R3nHL\nuG3wu/vfg+fbfqzhIwkh7ir0Cz8hEkXBL0SiKPiFSBQFvxCJouAXIlGaX8CT6QbOs6Wq5bCk5xWe\nFVeOZPUVF3nGXCEivxWLYQno1OnL/PEKPHusu50X8Lx6lUt9U4v8ecPDL+ng5n46JZfnkml/V1iG\nAoCBLt56K1sJy3aZAvejo43/Qry0FJHmjK/j0JYd4SnOsxwXyesMAKWIDLhQ5r96zea5j1YJv2Yn\nTpyhcx4d2Bccd1/brD4hxC8hCn4hEkXBL0SiKPiFSBQFvxCJouAXIlGaK/UZYOSIHimMaJWwFFWJ\n9NzzwhK1VcsRuWbmBrVNWvi90iKSY3GRy2idkUy7Yon7f73I5beurrCUNl/ma9XZyeWhtlbef64Y\nkbYKrWEZcLHE51wb5/Lb3ve9j9p6enl2YXtHWPKdmeRScLXMC4m25PjaF4t8jecj5+oju8PZgLs3\ncXnz9LmwXF1c4nG0HF35hUgUBb8QiaLgFyJRFPxCJIqCX4hEafpufzYffr8plyO1x6rhHczYrn0l\nYosl9mTz3DY/H97pjdXwy17nu+XTsRwM4+tRzfN2UnNzYZUg18LVg9ZI0snElTlqy5f5tWO4P/ya\ntZLXHwCKOX46bh7ewuctTFDb7Gx4V7+jgysE1Tl+7pQtUhsyy9e4HHk92wthtainp4POOT8dXseq\nN34915VfiERR8AuRKAp+IRJFwS9Eoij4hUgUBb8QiXJbqc/MtgP4MwBbAFQBHHT3b5jZlwH8PoCr\n9bt+0d3/OvpYACzbeI2xmziT+kjCDwCUIokUtsDlPGSnua0lvFzzi1zOQ4XLgPkMX4uWHG9f5kv8\nZWMdzBZz/H1+LsfXMcOVPoz0dlPbzNTV4HgG/AF3kwQXABgY6qG22RtcmpubD9dQLC9wWa63h9cS\nLEck01M3eFLYQpEn3Az27QqOV8o8YWxqNnyeVqpr2K4LQBnAH7v762bWBeA1M/tp3fZ1d/9vDR9N\nCHHX0EivvlEAo/XbM2Z2DAB/ixZC3BPc0Xd+M9sF4AMAXqkPfd7MjprZs2bGf3YmhLjraDj4zawT\nwA8AfMHdpwF8E8BuAI+h9sngq2TeATM7ZGaHIl24hRBNpqHgN7McaoH/HXf/IQC4+xV3r3itIfi3\nADwemuvuB919v7vvtzvf6xNCrBO3DX4zMwDfBnDM3b92y/jwLXf7FIC31t49IcR60chu/xMAPgvg\nTTM7XB/7IoDPmNljABzAWQB/cNtHyhgyraQG2lykXReR9LzC5bxqiddhWypGpLkcr5uWXQjLdgsZ\nLud5K39/LbXyY7WyYocAMmUul1k1fLz5SDZdroX70dHO16or8phLs1eC4wvTY3ROYbCT2sx4fb/W\nSEsxI3UXZxf4+Zbv5dtXLW2RY+XDzxkArMrbtvV2h6XFyavn6Jw3jhwNjs/P89qEy2lkt//vEe6w\nF9X0hRB3N/qFnxCJouAXIlEU/EIkioJfiERR8AuRKE0v4Jlh7Y4ib0PVSvingdVKpOVShUt9mWq4\nlRQAeJXLh5VSeF65yP1YiiyxZ7hsVI0sSKYSkfoQlh3LS7xIZ6XMpcquTl5EssW5H1OTl8LjE+Fx\nAHhnnktlO+7nWX35Li7NZTJhSa+4FMmYW4z8FLXKX7OJazzjry3L52UR9iXXybMmi8Vw9p7fwc9o\ndeUXIlEU/EIkioJfiERR8AuRKAp+IRJFwS9EojRV6vOqo7RAikVGFApWBqBc4XJNpsx7qlmZS3OZ\nJS5fVXNhqW/JeOabgUs8lUqkwEEbL8TIHxGotISfd9Z5kU6PPOdYduHiHF/jixfCGWnTpLAnAMx3\ncMnxythFahsp8GtYV1d4/MQUlxVnLvFjbdo8TG3T13jx10oki/DC8cvBcc/z86O4SIraNl6/U1d+\nIVJFwS9Eoij4hUgUBb8QiaLgFyJRFPxCJEpzs/qqQHmOyHMRiYKpgNWIPlgq8/5t1UXeU61S5fKK\nEZu3xrLAIoUiC/xJW6Twp0dqoC8Vw89taYH72NZD9DAA2Yh2dObsGWo7fT4s9VUia29Zfqx333mT\n2gYGeMaiWVjinJzihURfP3Sc2ga37qS2M2MXqC1r3Mepa+FzdYL04wOAG1NhebZS4efbcnTlFyJR\nFPxCJIqCX4hEUfALkSgKfiES5ba7/WZWAPAygNb6/f/S3b9kZvcB+B6APgCvA/isu/MiZkBt257l\nl8RKj7HN7UheTLUacaXEE1kQqYFWZKbYDmuVqw5m3MdSnifiRPdzs+FdZSMtzwCgI9J2a2Z6nNpe\nffX/UNvJ44eD420tfH3zWf6CPvjQDmqbv8GThYqkHl82z/24PMGf8xs/P0VtCxFlp7OLJy1VZ8Ln\nwfQiryc5Mx+eU62ubQ2/IoDfcPf3o9aO+0kz+zCAPwHwdXffA+A6gM81fFQhxIZz2+D3GrP1/+bq\n/xzAbwD4y/r4cwA+uS4eCiHWhYa+85tZtt6hdxzATwGcAjDl7jd/sXMRwMj6uCiEWA8aCn53r7j7\nYwC2AXgcwEOhu4XmmtkBMztkZodW7qYQYq25o91+d58C8HcAPgyg1+yfyrxsAxAsR+LuB919v7vv\nX42jQoi15bbBb2aDZtZbv90G4F8DOAbgJQD/tn63pwH8eL2cFEKsPY0k9gwDeM7Msqi9WTzv7v/L\nzN4B8D0z+y8A3gDw7UYOmHFySOMyiWeIfBGR+qI4rz1XLXGphHlYivnuXK6xbESONF5n0Fv4e3ZL\na2dwfGGO+zExfo3arjtve3boraPUNjs7GxwvRKTUfEQqG73I5bfL58JJRADQ1jUQHN/78D4658SF\nsO8AcOgvfkht1TyX88Zneb3Js6VwklEt5MixiKR3B0rf7YPf3Y8C+EBg/DRq3/+FEPcg+oWfEImi\n4BciURT8QiSKgl+IRFHwC5Eo5hHpZc0PZnYVwE1dZgDARNMOzpEf70V+vJd7zY+d7j7YyAM2Nfjf\nc2CzQ3fDr/7kh/xI1Q997BciURT8QiTKRgb/wQ089q3Ij/ciP97LL60fG/adXwixsehjvxCJsiHB\nb2ZPmtnPzeykmT2zET7U/ThrZm+a2eFmFhsxs2fNbNzM3rplrM/MfmpmJ+p/N22QH182s0v1NTls\nZp9ogh/bzewlMztmZm+b2b+vjzd1TSJ+NHVNzKxgZq+a2ZG6H/+5Pn6fmb1SX4/vm1l+VQdy96b+\nA5BFrQzY/QDyAI4A2NdsP+q+nAUwsAHH/TUAHwTw1i1j/xXAM/XbzwD4kw3y48sA/kOT12MYwAfr\nt7sAHAewr9lrEvGjqWuCWrJ6Z/12DsArqBXQeR7Ap+vj/x3AH67mOBtx5X8cwEl3P+21Ut/fA/DU\nBvixYbj7ywAmlw0/hVohVKBJBVGJH03H3Ufd/fX67RnUisWMoMlrEvGjqXiNdS+auxHBPwLg1nam\nG1n80wH8xMxeM7MDG+TDTYbcfRSonYQANm+gL583s6P1rwXr/vXjVsxsF2r1I17BBq7JMj+AJq9J\nM4rmbkTwh+rvbJTk8IS7fxDAbwL4IzP7tQ3y427imwB2o9ajYRTAV5t1YDPrBPADAF9wd96fuvl+\nNH1NfBVFcxtlI4L/IoDtt/yfFv9cb9z9cv3vOIAfYWMrE10xs2EAqP/ldavWEXe/Uj/xqgC+hSat\niZnlUAu477j7zVpZTV+TkB8btSb1Y99x0dxG2Yjg/xmAPfWdyzyATwN4odlOmFmHmXXdvA3g4wDe\nis9aV15ArRAqsIEFUW8GW51PoQlrYmaGWg3IY+7+tVtMTV0T5kez16RpRXObtYO5bDfzE6jtpJ4C\n8B83yIf7UVMajgB4u5l+APguah8fS6h9EvocgH4ALwI4Uf/bt0F+/DmANwEcRS34hpvgx79E7SPs\nUQCH6/8+0ew1ifjR1DUB8ChqRXGPovZG859uOWdfBXASwP8E0Lqa4+gXfkIkin7hJ0SiKPiFSBQF\nvxCJouAXIlEU/EIkioJfiERR8AuRKAp+IRLl/wP/fUEOJOcezwAAAABJRU5ErkJggg==\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.imshow(data.trn_ds.denorm(x)[1]);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fully connected model" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "data = get_data(32,bs)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "lr=1e-2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From [this notebook](https://github.com/KeremTurgutlu/deeplearning/blob/master/Exploring%20Optimizers.ipynb) by our student Kerem Turgutlu:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class SimpleNet(nn.Module):\n", " def __init__(self, layers):\n", " super().__init__()\n", " self.layers = nn.ModuleList([\n", " nn.Linear(layers[i], layers[i + 1]) for i in range(len(layers) - 1)])\n", " \n", " def forward(self, x):\n", " x = x.view(x.size(0), -1)\n", " for l in self.layers:\n", " l_x = l(x)\n", " x = F.relu(l_x)\n", " return F.log_softmax(l_x, dim=-1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = ConvLearner.from_model_data(SimpleNet([32*32*3, 40,10]), data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(SimpleNet(\n", " (layers): ModuleList(\n", " (0): Linear(in_features=3072, out_features=40)\n", " (1): Linear(in_features=40, out_features=10)\n", " )\n", " ), [122880, 40, 400, 10])" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn, [o.numel() for o in learn.model.parameters()]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "OrderedDict([('Linear-1',\n", " OrderedDict([('input_shape', [-1, 3072]),\n", " ('output_shape', [-1, 40]),\n", " ('trainable', True),\n", " ('nb_params', 122920)])),\n", " ('Linear-2',\n", " OrderedDict([('input_shape', [-1, 40]),\n", " ('output_shape', [-1, 10]),\n", " ('trainable', True),\n", " ('nb_params', 410)]))])" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.summary()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn.lr_find()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", " \u001b[A\u001b[A" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEOCAYAAACaQSCZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VvX5//HXlQ0khJGwR9h7GhRExFVEHLhFcVVbpNWv\nu9qq7bfVttpatW5Frf1qcaPWgVtRqTICyt5DCQJhr4SEJNfvj/vGXxoD3MHcOXeS9/PxuB859zmf\nc+4rH0LeOedzhrk7IiIiBxMXdAEiIlIzKDBERCQiCgwREYmIAkNERCKiwBARkYgoMEREJCIKDBER\niYgCQ0REIqLAEBGRiCgwREQkIglBF1CVMjIyPCsrK+gyRERqjFmzZm1y98xI2taqwMjKyiInJyfo\nMkREagwz+ybStjokJSIiEVFgiIhIRBQYIiISEQWGiIhERIEhIiIRUWCIiEhEFBgV2Lq7iBmrtlBY\nXPKDZflFxWzYsYc9e3+4TESkNqtV12Ecqu35e1m+cRcr8nbx8eI8Plq8gb0lTlpyAsO7ZRIfZ2za\nVcjqTfms3Vbw/Xr1k+JpXD+J9HqJlLqTX1RCanICHTIb0DGjAR0zG9ClWRq9WjXEzH50ne7O0g27\niI+DTpmpVbJNEZFI1fnAKC4pJftPH7C3xAFo2iCJi4dkMbBdYz5dmsenSzeSnBBPRmoS2VmNGZPZ\nliapSWzL38vW3UVszd/L9oIi4syolxTP9oK9zF+7nXfmraM0tEmO6NCE357Sk96t07//3F2FxXy3\nrYCCohKKS52uzVNJS0n8vqZdhcXUT0qgsLiE/yzfxJQlG5myZCPrd+wBIDMtmX5tGtGkQSL1kxLY\ntKuQDTv2UFhcCkDrRvU4qU9Lju/ejAbJdf6fWUSqgLl70DVUmezsbD+UK71fmPEtmWnJdG6WSpvG\n9YmP+/F/uRcWl7BmSz5Tl23igY+XszW/iOZpKdRPimdXYTF5Owv/q318nNG7VUOKS51lebsoCv/i\n3yctOYFhXTM4pmszHOeLFZtZtG4H2wv2kl9YQtPUJJo3TKFeUjwAC7/bQd7OQhLjjR4tGzKgbSP6\nt2tE/7aNyWpaX3snIgKAmc1y9+yI2iowom/Hnr0888Vqvt2Sz+6iElIS4umY2YA2jevRICn01/+c\n3G3MWLWFpIQ4erRsSPOGKeQXFuOE9lAGtm9MYnzkQ06lpc7M1Vv4ZMlGvl6zlbm528kvCo27dG+R\nxi+P7czJfVpWSTiKSM2lwJAfKCl1luXtZOaqLfzfl9+wPG8XWU3rM354J84Y2JrkhPigSxSRACgw\n5IBKS533F27g4U+WM2/tdjLTkhnWJYMhHZsyuGNT2japH3SJIlJNKhMYGg2tg+LijJG9W3Bir+ZM\nXb6J52d8y5QlG3l19logNGB+ZKemHNUlg2FdMmnSICngikUkFigw6jAzY1iXTIZ1yaQ0PNg+beVm\nvlyxmfcWrOflWbnEGQzp1JRT+7bSoSuROk6HpKRCJaXOvLXb+WjRBt6au45Vm3bTvml9bju5Jyf0\naKazrERqCY1hSJVydz5btok73lrI8rxd9GvbiMuP6sBJvVtU6swtEYk9lQkM/W+XgzIzhnfN5J1r\nhnHH6b3ZUbCXq5//ipPu/5wF320PujwRqSYKDIlYYnwcFw1uz0fXD+fxiw5jR8Feznj4Cx6dsoKt\nu4uCLk9EokyHpOSQbdldxE2vzOXDRRuIjzOO7NSUXxzTiSM7ZQRdmohESGMYUm3cnQXf7WDyvHW8\n9tVa1m3fwwk9mjF2cHv6tE4nIzU56BJF5ABiIjDMrC3wDNACKAUmuPv95dqMBu4ILy8GrnX3qeFl\nlwC3hZv+0d3/72CfqcAI1p69JTz9n9U88slydhYWA9AxowGn9mvFGQNak5XRIOAKRaS8WAmMlkBL\nd59tZmnALOB0d19Ypk0qsNvd3cz6Ai+5e3czawLkANmAh9c9zN23HugzFRixYXdhMXNztzNv7TY+\nWbyRaas2A3DOYW248cRuNEtLCbhCEdknJq70dvd1wLrw9E4zWwS0BhaWabOrzCoNCIUDwInAB+6+\nBcDMPgBGAs9Hq16pOg2SExjSqSlDOjVl3NGdWLe9gKc+X8X/fbmat+eu47xB7bjgiHZ0bpYadKki\nUgnVcpaUmWUBA4DpFSw7w8wWA28Dl4VntwbWlGmWG54nNVDL9HrcdkpP3r9uOMf1aM6z01Zzwr2f\nMu6ZHPLCz/cQkdgX9cAIH3aaRGh8Ykf55e7+mrt3B04nNJ4BUNFlxBUeOzOzcWaWY2Y5GzdurKqy\nJQo6ZDTgwfMH8OVvjue6E7ry6dKNnHDvp7z2VW7QpYlIBKIaGGaWSCgsJrr7qwdq6+6fAZ3MLIPQ\nHkXbMovbAN/tZ70J7p7t7tmZmZlVVLlEU0ZqMtec0IV3rhlG1+ZpXPfiHH718hwKivScdJFYFrXA\nsNDNhp4CFrn7vftp0zncDjMbCCQBm4H3gBFm1tjMGgMjwvOkFumYmcqLVwzh6uM688rsXM545D/M\nX6srx0ViVTTvVjsUuAiYZ2Zfh+fdArQDcPfHgLOAi81sL1AAnOeh07a2mNkdwMzwerfvGwCX2iU+\nzrh+RDcGtm/MjS/P5bSHpnLxkCxuGNH1+2eci0hs0IV7EjO2F+zlb+8t4V/TvyEzNZn/PbUXo/q0\n0J1xRaJINx+UGim9XiJ3nN6b1345lMy0ZK58bjaXPj2TbzbvDro0EUGBITGof9tG/PvKofzulJ7M\n+mYrI+77jIc+XkZhsQbFRYKkwJCYlBAfx2VHdeDD64dzQo/m/O39pYy6/3O+XLE56NJE6iwFhsS0\nFukpPDx2IE//dBBFJaWc/8Q0rn/xazbtKgy6NJE6R4EhNcKx3ZrxwXXDuerYzrw59zuOv+dTHvt0\nBdvy9RwOkeqis6Skxlmet5M/vLmQz5dtIiUxjnOz23LlsZ1p3lA3NRSprJi4W20QFBh1y8LvdvD0\nf1bx2ldriY8zLjkyi+t/0pWUxPigSxOpMXRardQJPVs15O5z+vHJjcdwar9WTPhsJRc8MU3jGyJR\nosCQGq9tk/r87Zx+PDJ2IAvX7WD0Q/9hed7OoMsSqXUUGFJrjOrTkpeuGEJhcSljJkxjyXqFhkhV\nUmBIrdK3TSNevGIw8XHGmAlfMmfNtqBLEqk1FBhS63TKTOXFcUOolxjPmY9+wZ/eXsju8DPGReTQ\nKTCkVsrKaMDka4ZxbnYbnvh8Fac8OFWD4SI/kgJDaq1G9ZO488y+PPfzI1i3vYDL/zmT/CLtaYgc\nKgWG1HpHdsrgwfMHMm/tdq567it27NkbdEkiNZICQ+qEn/Rszu2je/Px4jyG3vUx976/hF0a1xCp\nFAWG1BkXDm7PW/9zFEM7ZfDAx8v56dMz9BxxkUpQYEid0rt1Oo9ddBgPXzCQnG+2cuVzs9lbUhp0\nWSI1ggJD6qST+7bkj6eHDlH9cuJsthdoXEPkYBQYUmeNPaI9/3tqTz5ZnMfJD3zOrG+2BF2SSExT\nYEid9tOhHXhp/BDM4KxHv+TSp2foqX4i+6HAkDpvYLvGTL56GDeO6Mr8tds5/4lp3PP+EmrTrf9F\nqoICQwRIS0nkquO6MPXm4xgzqC0Pfrycu95drNAQKSMh6AJEYklKYjx/PqMPifFxPP7pSjbuKOT3\no3vRMCUx6NJEAqfAECknLs64fXQvmqYm8cBHy5i+agv3ndefwzs0Cbo0kUDpkJRIBcyMa0/oyiu/\nOJLEeGPsk9OYPG9d0GWJBCpqgWFmbc3sEzNbZGYLzOyaCtqMNbO54dcXZtavzLLVZjbPzL42Mz2o\nWwIxsF1j/n3VUfRt04irnpvNSzPXBF2SSGCiuYdRDNzg7j2AwcCVZtazXJtVwHB37wvcAUwot/xY\nd+8f6QPKRaIhvV4iz15+OEM7Z3DTpLmMeyaHNVvygy5LpNpFLTDcfZ27zw5P7wQWAa3LtfnC3beG\n304D2kSrHpEfo35SAk9dMohfndiNz5dt4oR7P+UfU1fpLCqpU6plDMPMsoABwPQDNLsceKfMewfe\nN7NZZjYuetWJRCYpIY4rj+3MxzcOZ1iXDG5/ayGX/18OW3YXBV2aSLWIemCYWSowCbjW3Xfsp82x\nhALj5jKzh7r7QOAkQoezjt7PuuPMLMfMcjZu3FjF1Yv8UMv0ejxxcTa/P7UnU5dtYuyT0/UIWKkT\nohoYZpZIKCwmuvur+2nTF3gSGO3u39+Twd2/C3/NA14DDq9ofXef4O7Z7p6dmZlZ1d+CSIXMjEuH\ndmDCxYexZP0Orn3xa0pLdXhKardoniVlwFPAIne/dz9t2gGvAhe5+9Iy8xuYWdq+aWAEMD9atYoc\nqmO6NeO3p/Tkg4UbuPv9JUGXIxJV0bxwbyhwETDPzL4Oz7sFaAfg7o8BvwOaAo+E8oXi8BlRzYHX\nwvMSgOfc/d0o1ipyyC49MoulG3bx6JQVDGzXmJ/0bB50SSJRYbXpLI/s7GzPydElG1L99uwt4axH\nvyB3awGTrxlG60b1gi5JJCJmNivSSxd0pbdIFUhJjOehCwZSXFLK1c9/paf4Sa2kwBCpIh0yGnDn\nWX2Z9c1WbnplrgbBpdbRzQdFqtBp/Vrxzabd3PPBUjLTkrllVI+gSxKpMgoMkSp21XGd2birkAmf\nrSQzNZmfH90x6JJEqoQCQ6SKmRn/e2ovNu0q5E+TF5GRlsQZA3TXG6n5NIYhEgXxccZ95/VnSMem\n/OrluXy6VHchkJpPgSESJckJ8Tx+8WF0bpbKtS98Rd7OPUGXJPKjKDBEoqhhSiIPXTCQ/KISbnl1\nnu5uKzWaAkMkyjo3S+VXJ3bjw0V5vDwrN+hyRA6ZAkOkGlw2tANHdGjCH95YwPy124MuR+SQKDBE\nqkFcnHH/mAGk10vk0qdn8u1mPbFPah4Fhkg1aZGewjOXH05xaSkX/WM6320rCLokkUpRYIhUo87N\n0vjHpYPYvKuI0x6ayszVW4IuSSRiCgyRajawXWNev/JI0lISueCJaTz9n1W675TUCAoMkQB0bpbG\n61cO5egumfzhzYVc8vQMNuzQdRoS2xQYIgFJr5fIk5dk88fTezNz9RbOePg/5Ck0JIYpMEQCZGZc\nOLg9L19xJFvz9/LzZ3IoKCoJuiyRCikwRGJAnzbpPHD+AOau3c61L36l0JCYpMAQiRE/6dmc357c\nk/cWbGDk/Z8xddmmoEsS+S8KDJEYctlRHXj+54OJM+PCp6bz6JQVQZck8j0FhkiMGdKpKe9cM4xT\n+7XiL+8uZsJnCg2JDXqAkkgMSkmM575z+1Fa6vx58mIaJCcw9oj2QZcldZz2MERiVEJ8HH8f05/h\nXUPXaizP2xl0SVLHKTBEYlhifBx3n9OXBknxXP/SHPaWlAZdktRhCgyRGNcsLYU/ndGHubnbefDj\n5UGXI3WYAkOkBhjVpyVnDGjNAx8tY+L0b4IuR+ooDXqL1BB3ndWHbflF3PrafNzhwsEaBJfqFbU9\nDDNra2afmNkiM1tgZtdU0Gasmc0Nv74ws35llo00syVmttzMfh2tOkVqiuSEeB676DCO696M216f\nzy2vzWN3YXHQZUkdEs1DUsXADe7eAxgMXGlmPcu1WQUMd/e+wB3ABAAziwceBk4CegLnV7CuSJ2T\nnBDPoxcOZNzRHXl+xrecdP/neuSrVJuoBYa7r3P32eHpncAioHW5Nl+4+9bw22lAm/D04cByd1/p\n7kXAC8DoaNUqUpMkJ8Rzy6gevHTFEIpLShn75HSFhlSLahn0NrMsYAAw/QDNLgfeCU+3BtaUWZZL\nubARqesGZTXhxSuG0CApngufms7C73YEXZLUclEPDDNLBSYB17p7hT/RZnYsocC4ed+sCppV+Egy\nMxtnZjlmlrNx48aqKFmkxmjbpD4vjBtCvcR4rvhXDvlFGtOQ6IlqYJhZIqGwmOjur+6nTV/gSWC0\nu28Oz84F2pZp1gb4rqL13X2Cu2e7e3ZmZmbVFS9SQ7RrWp/7zuvPmi0F/P3DZUGXI7VYNM+SMuAp\nYJG737ufNu2AV4GL3H1pmUUzgS5m1sHMkoAxwBvRqlWkphvcsSljBrXlyc9XajxDoiaiwDCza8ys\noYU8ZWazzWzEQVYbClwEHGdmX4dfo8xsvJmND7f5HdAUeCS8PAfA3YuBq4D3CA2Wv+TuCw7lGxSp\nK35zUg+aNEjm5klzKSrWLUSk6pl7hUMD/93IbI679zOzE4Ergd8CT7v7wGgXWBnZ2dmek5MTdBki\ngXlvwXqueHYWPzuqA7edojPR5eDMbJa7Z0fSNtJDUvsGoUcRCoo5VDwwLSIBOrFXCy4e0p4np67i\no0Ubgi5HaplIA2OWmb1PKDDeM7M0QPu8IjHollE96NWqITe8PIdVm3YHXY7UIpEGxuXAr4FB7p4P\nJAI/jVpVInLIUhLjeeiCgcSZce7jX7J0g56jIVUj0sAYAixx921mdiFwG6BTMURiVIeMBrw4bjAG\nnPf4lzpzSqpEpIHxKJAfvjngTcA3wDNRq0pEfrQuzdN46Yoh1E9K4PwnpjH7260HX0nkACINjGIP\nnU41Grjf3e8H0qJXlohUhayMBrw0fghNGyRx0ZPTmbZy88FXEtmPSANjp5n9htB1FW+H7yabGL2y\nRKSqtG5Uj5euGEKrRvW44tlZrN++J+iSpIaKNDDOAwqBy9x9PaEbAd4dtapEpEo1a5jChIuzKSou\n5aZJc4nk+iuR8iIKjHBITATSzewUYI+7awxDpAbpkNGAW0Z157OlG5k4/dugy5EaKNJbg5wLzADO\nAc4FppvZ2dEsTESq3oWD2zOsSwZ3vLWQ56Z/qz0NqZRID0ndSugajEvc/WJCDzj6bfTKEpFoMDPu\nO68/2VmNueW1efz8mRy2F+wNuiypISINjDh3zyvzfnMl1hWRGJKRmsyzlx3Bb0/pyZQlG7n5FY1p\nSGQSImz3rpm9Bzwffn8eMDk6JYlItMXFGZcf1YGS0lL+PHkxE6d/y4WD2wddlsS4SAe9fwVMAPoC\n/YAJ7n7zgdcSkVj3s6M6cnTXTO54ayGL1+sRr3JgER9WcvdJ7n69u1/n7q9FsygRqR5xccY95/Qj\nLSWR61+cw94S3VNU9u+AgWFmO81sRwWvnWamP0dEaoHMtGT+eHovFq7bwVNTVwVdjsSwAwaGu6e5\ne8MKXmnu3rC6ihSR6BrZuyUn9mrOfR8sZbVuiS77oTOdRASA20f3Jik+jt+8Ok9nTUmFFBgiAkDz\nhin8ZlQPvly5mZdy1gRdjsQgBYaIfG/MoLYc3qEJf3p7EXk7dJNC+W8KDBH5XlycceeZfdhTXMrv\n31wQdDkSYxQYIvJfOmWmcs3xXZg8bz3PTvsm6HIkhigwROQHxg/vxPHdm/H7Nxbw6dKNQZcjMUKB\nISI/EB9n3H/+ALo0S+WqibNZnrcr6JIkBigwRKRCqckJPHXpIOLijD++vTDociQGKDBEZL9aN6rH\nL4/pxJQlG/lixaagy5GAKTBE5IAuOTKLlukp/OXdJbqgr46LWmCYWVsz+8TMFpnZAjO7poI23c3s\nSzMrNLMbyy1bbWbzzOxrM8uJVp0icmApifFcd0JX5qzZxjvz1wddjgQomnsYxcAN7t4DGAxcaWY9\ny7XZAlwN/G0/2zjW3fu7e3YU6xSRgzhzYGu6Nk/lj28t1BP66rCoBYa7r3P32eHpncAioHW5Nnnu\nPhPQT6BIDEuIj+OvZ/djw85C/qAL+uqsahnDMLMsYAAwvRKrOfC+mc0ys3HRqEtEIte/bSOuPKYT\nr85ey7vz1wVdjgQg6oFhZqnAJOBad6/MMzSGuvtA4CRCh7OO3s/2x5lZjpnlbNyoC4xEoul/ju9C\n79YNuXnSPObmbgu6HKlmUQ0MM0skFBYT3f3Vyqzr7t+Fv+YBrwGH76fdBHfPdvfszMzMH1uyiBxA\nYnwcj449jLSUBC54YjozVm0JuiSpRtE8S8qAp4BF7n5vJddtYGZp+6aBEcD8qq9SRCqrbZP6vDx+\nCM0bJnPxP6Yzf+32oEuSahLNPYyhwEXAceFTY782s1FmNt7MxgOYWQszywWuB24zs1wzawg0B6aa\n2RxgBvC2u78bxVpFpBJaptfjxSuGkF4vketf+prC4pKgS5JqkBCtDbv7VMAO0mY90KaCRTuAftGo\nS0SqRkZqMned2Zef/nMmf/9wGTeP7B50SRJlutJbRA7Zsd2bcW52Gx7/dAWzvtkadDkSZQoMEflR\nfntKT1o1qscvJ85ig57SV6spMETkR0lLSeSJi7PZuaeYcc/ksGevxjNqKwWGiPxoPVo25N5z+zMn\ndzu3va4TGmsrBYaIVImRvVvwy2M68cqsXKYu063QayMFhohUmauP70JW0/r89t/zdWiqFlJgiEiV\nSUmM54+n92HVpt08MmVF0OVIFVNgiEiVOqpLBqP7t+KxKStYvL4yt4+TWKfAEJEq97tTetKwXiLX\nvThHV4HXIgoMEalyTVOT+ctZfVi0bgd//3BZ0OVIFVFgiEhUHN+jOWMGteXxT1fw8eINQZcjVUCB\nISJRc9spPenZqiFXPDuLd/U88BpPgSEiUZOanMDEnw2md+t0rnxuNq99lRt0SfIjKDBEJKrS6yXy\n7OVHMCirMde9OId73l9CaakHXZYcAgWGiERdanICz1x2BOdmt+HBj5dz48tzcFdo1DQKDBGpFkkJ\ncfzlrL5cfXwXXv1qLa/M0uGpmkaBISLVxsy49vguHN6hCbe/uZC12wqCLkkqQYEhItUqLs6455x+\nlLhz0ytzNJ5RgygwRKTatW1Sn9tO7sl/lm/m2WnfBF2OREiBISKBOP/wthzTLZM731nEyo27gi5H\nIqDAEJFAmBl/OasvyQnx3PDyHEp0aCrmKTBEJDDNG6Zw++hefPXtNq5+/it2FRYHXZIcQELQBYhI\n3XZav1as276Hv767mEXrdzDhosPo3Cwt6LJqjPXb9+A4LdPrRf2ztIchIoEyM8YP78TEnw1mR0Ex\nl/0zh/wi7WlE6sGPl3HS/Z9Xy2cpMEQkJgzp1JRHxg5kzdZ8/vrukqDLqTHWbiugTePo712AAkNE\nYsjhHZpwyZAs/vnFaqav3Bx0OTVC7tYCWjeq4YFhZm3N7BMzW2RmC8zsmgradDezL82s0MxuLLds\npJktMbPlZvbraNUpIrHlppHdaNukHjdPmsuevXpa34G4O2u3FtCmcf1q+bxo7mEUAze4ew9gMHCl\nmfUs12YLcDXwt7IzzSweeBg4CegJnF/BuiJSC9VPSuCuM/uyenM+D3+yPOhyYtqW3UUU7C2p+XsY\n7r7O3WeHp3cCi4DW5drkuftMYG+51Q8Hlrv7SncvAl4ARkerVhGJLUM7Z3DmgNY89ukKlm3YGXQ5\nMWvfvbhq1RiGmWUBA4DpEa7SGlhT5n0u5cJGRGq3W0/uQYPkBG59bT57S0qDLicm5W4NBUbr2hIY\nZpYKTAKudfcdka5WwbwKLwM1s3FmlmNmORs3bjzUMkUkxjRNTeaWUT2YsXoLx9w9hWe+XE1RsYKj\nrLVb9+1h1PwxDMwskVBYTHT3Vyuxai7Qtsz7NsB3FTV09wnunu3u2ZmZmYderIjEnHOz2/L0pYNo\n3jCZ3/17Abe+Ni/okmJK7tZ80pITSK+XWC2fF82zpAx4Cljk7vdWcvWZQBcz62BmScAY4I2qrlFE\nYt+x3Zsx6RdHcsXwjrw8K5dPl+pIwj5rtxVU2+EoiO4exlDgIuA4M/s6/BplZuPNbDyAmbUws1zg\neuA2M8s1s4buXgxcBbxHaLD8JXdfEMVaRSSGmRnXndCVTpkN+M2kuezcU/48mbopd2v1XbQHUbyX\nlLtPpeKxiLJt1hM63FTRssnA5CiUJiI1UEpiPH89ux9nP/YFf568iDvP7Bt0SYFbu7WAwR2bVtvn\n6UpvEakxDmvfmHFHd+T5GWv499drgy4nUNsL9rKzsLjarsEABYaI1DA3jujGoKzG/HrSPJbW4Ws0\ncrfmA9V3DQYoMESkhkmMj+OhCwbSIDmBcc/ksGhdpGfr1y5rq/kaDFBgiEgN1LxhCo9dOJBdhcWc\n+uBU7vtgaZ27uC+3mq/BAAWGiNRQ2VlN+OC64ZzarxX3f7SM8c/OqlM3K1y7rYB6ifE0rl8912CA\nAkNEarDGDZK477z+3HF6bz5eksfF/5jBtvyioMuqFrlb82nTuB6hS96qhx7RKiI13kWD29MwJYEb\nXprD4X/6iKO6ZDBmUFtG9GoRdGlRs2ZL9V60B9rDEJFaYnT/1rx+5VAuHtKeJet3Mu7ZWTz40TLc\nK7wNXY02c/UWFq7bQXb7xtX6udrDEJFao3frdHq3Tuemkd25edJc7vlgKet27OF/T+1JckJ80OVV\niZJS5w9vLqBlegqXHdWhWj9bgSEitU5SQhz3ntuPFukpPDplBbNWb+Xuc/rSt02joEv70V6ZtYb5\na3dw/5j+1E+q3l/hOiQlIrWSmXHzyO48fekgthUUccYjX/DmnApvel1j5BcVc/d7S8hu35jT+rWq\n9s9XYIhIrXZs92a8f91wBrRtxM2T5rI8b1fQJR2yjxblsWlXETeM6FatZ0fto8AQkVovvV4iD10w\nkHqJ8fxy4izyi4qDLumQTJ63jmZpyRzRoUkgn6/AEJE6oUV6Cn8f059lebv41ctzKSmtWWdP5RcV\n88mSPEb2bkFcXPXvXYACQ0TqkGFdMrl1VA/enreOmyfNpbQGhcaUJRvZs7eUk3q3DKwGnSUlInXK\nz4Z1ZHdhCfd9uJTkhDjuGN07sL/YK2PyvHVkpCZxeECHo0CBISJ10NXHd2ZPcQmPTllBYXEpd53Z\nh4T42D3gsmdvCR8vzuOMAa2JDzDcFBgiUueYGTed2I2UhHju+3Apu/YU8+AFA0iM0dD4aFEe+UUl\njOoT3OEo0BiGiNRRZsY1J3ThtpN78O6C9Tzx+cqgS6pQSanzwEfLyGpaP7Czo/ZRYIhInfazYR0Z\n2asF93+4jG827w66nB94/au1LNmwkxtP7Bb4YTMFhojUeb8/rReJ8XHc+tr8mLpZ4Z69Jdz7wVL6\ntE5nVIA8IWy/AAAMa0lEQVRnR+2jwBCROq9Fego3jezG1OWbePyzlTETGv+a9g1rtxXw65O6x8SZ\nXAoMERFg7BHtGdmrBXe9s5jbXp8f+CNfS0udp/+zmiM6NGFo54xAa9lHgSEiAsTHGY+MHcj44Z2Y\nOP1bzn38S+blbg+sns+Xb2LttgIuHNw+sBrKU2CIiITFxRm/Pqk794/pz5ot+Zz28FSue/FrPlmS\nx569JZSWOjv37K2WQ1YvzPiWxvUTGdGredQ/K1K6DkNEpJzR/VtzbPdmPPjRMiZO/5bXvlpLYrxR\nXOq4w9DOTbl/zAAyUpOj8vmbdhXywcINXHpkVkw9+EmBISJSgYYpidx6ck9uGNGNaSs3M23lFpLi\njVKHJz5fySkPTOXhsQM5LAqPSZ00K5fiUmfM4W2rfNs/RtQCw8zaAs8ALYBSYIK731+ujQH3A6OA\nfOBSd58dXlYCzAs3/dbdT4tWrSIi+5OSGM8x3ZpxTLdm3887qU8LfvGv2Vz45HQm/vwIBrarutDI\nLypm4vRvGZTVmM7N0qpsu1UhmmMYxcAN7t4DGAxcaWY9y7U5CegSfo0DHi2zrMDd+4dfCgsRiRm9\nWqXzyi+G0KxhMj99eiZL1u+sku2Wljo3vjyHNVvzufr4LlWyzaoUtcBw93X79hbcfSewCGhdrtlo\n4BkPmQY0MrPgr04RETmIZmkp/OvyI0hJjGPsk9N4dMoKNu4s/FHbfODjZUyet57fnNSdYV0yq6jS\nqlMtYxhmlgUMAKaXW9QaWFPmfW543jogxcxyCO2p3OXur0e/UhGRyLVtUp9/XX4Et70+n7+8u5h7\n3l/CiF7NOf/wdgztlHHQi+3cnXlrt/PegvV8uDCPJRt2cubA1vx8WMdq+g4qJ+qBYWapwCTgWnff\nUX5xBavsO1+tnbt/Z2YdgY/NbJ67r6hg++MIHc6iXbt2VVi5iMjBdWmexotXDGF53i5emPEtk2bn\nMnneenq1asgdp/f+wfiGu/PVmm28M28dk+etZ+22AuLjjEFZjfndKT0ZO7hdIM/rjoRF83xiM0sE\n3gLec/d7K1j+ODDF3Z8Pv18CHOPu68q1+yfwlru/cqDPy87O9pycnKoqX0Sk0vbsLeGtueu4+73F\nbNhRyPCumTSsl0ipO+u2FfDN5nw27y4iMd44qnMGo/q05Cc9m9OoflIg9ZrZLHfPjqRtNM+SMuAp\nYFFFYRH2BnCVmb0AHAFsd/d1ZtYYyHf3QjPLAIYCf41WrSIiVSUlMZ6zD2vDyN4tePCjZXyyJI81\nW/KB0D2rjuvejCGdmnJ8j+ak10sMuNrKidoehpkdBXxO6NTYfTdluQVoB+Duj4VD5SFgJKHTan/q\n7jlmdiTweHi9OODv7v7UwT5TexgiIpUTE3sY7j6ViscoyrZx4MoK5n8B9IlSaSIicgh0LykREYmI\nAkNERCKiwBARkYgoMEREJCIKDBERiYgCQ0REIqLAEBGRiET11iDVzcw2At8cwqrpQGUf3hvpOgdq\nt79lFc0vP+9A7zOATRHUdiii1Vfqp9jtJ4heXwXRT/tbfijzano/AXRx9/SItujudf5F6OFOUVnn\nQO32t6yi+eXnHeg9kFPT+kr9FLv9FM2+CqKfIu2TSObV9H6q7LZ1SCrkzSiuc6B2+1tW0fzy8w72\nPlqi1VfqJ/VTVa5zsDaR9Ekk82p6P1Vq27XqkJSEmFmOR3hvmLpM/RQ59VVkans/aQ+jdpoQdAE1\nhPopcuqryNTqftIehoiIRER7GCIiEhEFhoiIRESBISIiEVFg1EFm1sDMZpnZKUHXEqvMrIeZPWZm\nr5jZL4KuJ1aZ2elm9oSZ/dvMRgRdTywzs45m9pSZvRJ0LYdKgVGDmNk/zCzPzOaXmz/SzJaY2XIz\n+3UEm7oZeCk6VQavKvrJ3Re5+3jgXKBWniZZRf30urv/HLgUOC+K5QaqivpqpbtfHt1Ko0tnSdUg\nZnY0sAt4xt17h+fFA0uBnwC5wEzgfCAeuLPcJi4D+hK6fUEKsMnd36qe6qtPVfSTu+eZ2WnAr4GH\n3P256qq/ulRVP4XXuweY6O6zq6n8alXFffWKu59dXbVXpag901uqnrt/ZmZZ5WYfDix395UAZvYC\nMNrd7wR+cMjJzI4FGgA9gQIzm+zupVEtvJpVRT+Ft/MG8IaZvQ3UusCoop8nA+4C3qmtYQFV9zNV\n0ykwar7WwJoy73OBI/bX2N1vBTCzSwntYdSqsDiASvWTmR0DnAkkA5OjWllsqVQ/Af8DnACkm1ln\nd38smsXFmMr+TDUF/gQMMLPfhIOlRlFg1HxWwbyDHmd0939WfSkxrVL95O5TgCnRKiaGVbafHgAe\niF45Ma2yfbUZGB+9cqJPg941Xy7Qtsz7NsB3AdUSy9RPkVE/Ra7O9ZUCo+abCXQxsw5mlgSMAd4I\nuKZYpH6KjPopcnWurxQYNYiZPQ98CXQzs1wzu9zdi4GrgPeARcBL7r4gyDqDpn6KjPopcuqrEJ1W\nKyIiEdEehoiIRESBISIiEVFgiIhIRBQYIiISEQWGiIhERIEhIiIRUWBIYMxsVzV8xmkR3vK9Kj/z\nGDM78hDWG2BmT4anLzWzh6q+usozs6zyt/WuoE2mmb1bXTVJMBQYUuOFbzNdIXd/w93visJnHug+\nbMcAlQ4M4BbgwUMqKGDuvhFYZ2ZDg65FokeBITHBzH5lZjPNbK6Z/aHM/NfDTwdcYGbjyszfZWa3\nm9l0YIiZrTazP5jZbDObZ2bdw+2+/0vdzP5pZg+Y2RdmttLMzg7PjzOzR8Kf8ZaZTd63rFyNU8zs\nz2b2KXCNmZ1qZtPN7Csz+9DMmodvgT0euM7MvjazYeG/vieFv7+ZFf1SNbM0oK+7z6lgWXsz+yjc\nNx+ZWbvw/E5mNi28zdsr2mOz0NMV3zazOWY238zOC88fFO6HOWY2w8zSwnsSn4f7cHZFe0lmFm9m\nd5f5t7qizOLXgbEV/gNL7eDueukVyAvYFf46AphA6O6fccBbwNHhZU3CX+sB84Gm4fcOnFtmW6uB\n/wlP/xJ4Mjx9KaEHIAH8E3g5/Bk9CT3LAOBsQrcwjwNaAFuBsyuodwrwSJn3jfn/d0v4GXBPePr3\nwI1l2j0HHBWebgcsqmDbxwKTyrwvW/ebwCXh6cuA18PTbwHnh6fH7+vPcts9C3iizPt0IAlYCQwK\nz2tI6M7V9YGU8LwuQE54OguYH54eB9wWnk4GcoAO4fetgXlB/1zpFb2Xbm8usWBE+PVV+H0qoV9Y\nnwFXm9kZ4fltw/M3AyXApHLbeTX8dRahZ1lU5HUPPQNkoZk1D887Cng5PH+9mX1ygFpfLDPdBnjR\nzFoS+iW8aj/rnAD0NPv+btgNzSzN3XeWadMS2Lif9YeU+X6eBf5aZv7p4enngL9VsO484G9m9hfg\nLXf/3Mz6AOvcfSaAu++A0N4I8JCZ9SfUv10r2N4IoG+ZPbB0Qv8mq4A8oNV+vgepBRQYEgsMuNPd\nH/+vmaGHGJ0ADHH3fDObQujRsgB73L2k3HYKw19L2P/PdmGZaSv3NRK7y0w/CNzr7m+Ea/39ftaJ\nI/Q9FBxguwX8/+/tYCK+AZy7LzWzw4BRwJ1m9j6hQ0cVbeM6YAPQL1zzngraGKE9ufcqWJZC6PuQ\nWkpjGBIL3gMuM7NUADNrbWbNCP31ujUcFt2BwVH6/KnAWeGxjOaEBq0jkQ6sDU9fUmb+TiCtzPv3\nCd3VFIDwX/DlLQI67+dzviB062wIjRFMDU9PI3TIiTLL/4uZtQLy3f1fhPZABgKLgVZmNijcJi08\niJ9OaM+jFLiI0LOpy3sP+IWZJYbX7RreM4HQHskBz6aSmk2BIYFz9/cJHVL50szmAa8Q+oX7LpBg\nZnOBOwj9goyGSYQehjMfeByYDmyPYL3fAy+b2efApjLz3wTO2DfoDVwNZIcHiRdSwVPX3H0xocec\nppVfFl7/p+F+uAi4Jjz/WuB6M5tB6JBWRTX3AWaY2dfArcAf3b0IOA940MzmAB8Q2jt4BLjEzKYR\n+uW/u4LtPQksBGaHT7V9nP+/N3cs8HYF60gtodubiwBmluruuyz03OUZwFB3X1/NNVwH7HT3JyNs\nXx8ocHc3szGEBsBHR7XIA9fzGTDa3bcGVYNEl8YwRELeMrNGhAav76jusAh7FDinEu0PIzRIbcA2\nQmdQBcLMMgmN5ygsajHtYYiISEQ0hiEiIhFRYIiISEQUGCIiEhEFhoiIRESBISIiEVFgiIhIRP4f\n/7E/e5eZ0zEAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.sched.plot()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "b1434a390cc24207af4ac1427f681e3e", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.7658 1.64148 0.42129] \n", "[ 1. 1.68074 1.57897 0.44131] \n", "\n", "CPU times: user 1min 11s, sys: 32.3 s, total: 1min 44s\n", "Wall time: 55.1 s\n" ] } ], "source": [ "%time learn.fit(lr, 2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "0f42d62186f64e33a6037d8f5beb9f57", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.60857 1.51711 0.46631] \n", "[ 1. 1.59361 1.50341 0.46924] \n", "\n", "CPU times: user 1min 12s, sys: 31.8 s, total: 1min 44s\n", "Wall time: 55.3 s\n" ] } ], "source": [ "%time learn.fit(lr, 2, cycle_len=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## CNN" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class ConvNet(nn.Module):\n", " def __init__(self, layers, c):\n", " super().__init__()\n", " self.layers = nn.ModuleList([\n", " nn.Conv2d(layers[i], layers[i + 1], kernel_size=3, stride=2)\n", " for i in range(len(layers) - 1)])\n", " self.pool = nn.AdaptiveMaxPool2d(1)\n", " self.out = nn.Linear(layers[-1], c)\n", " \n", " def forward(self, x):\n", " for l in self.layers: x = F.relu(l(x))\n", " x = self.pool(x)\n", " x = x.view(x.size(0), -1)\n", " return F.log_softmax(self.out(x), dim=-1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = ConvLearner.from_model_data(ConvNet([3, 20, 40, 80], 10), data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "OrderedDict([('Conv2d-1',\n", " OrderedDict([('input_shape', [-1, 3, 32, 32]),\n", " ('output_shape', [-1, 20, 15, 15]),\n", " ('trainable', True),\n", " ('nb_params', 560)])),\n", " ('Conv2d-2',\n", " OrderedDict([('input_shape', [-1, 20, 15, 15]),\n", " ('output_shape', [-1, 40, 7, 7]),\n", " ('trainable', True),\n", " ('nb_params', 7240)])),\n", " ('Conv2d-3',\n", " OrderedDict([('input_shape', [-1, 40, 7, 7]),\n", " ('output_shape', [-1, 80, 3, 3]),\n", " ('trainable', True),\n", " ('nb_params', 28880)])),\n", " ('AdaptiveMaxPool2d-4',\n", " OrderedDict([('input_shape', [-1, 80, 3, 3]),\n", " ('output_shape', [-1, 80, 1, 1]),\n", " ('nb_params', 0)])),\n", " ('Linear-5',\n", " OrderedDict([('input_shape', [-1, 80]),\n", " ('output_shape', [-1, 10]),\n", " ('trainable', True),\n", " ('nb_params', 810)]))])" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.summary()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "d565d2bc5ccf4f02926d0258085419b7", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ " 70%|███████ | 138/196 [00:16<00:09, 6.42it/s, loss=2.49]" ] } ], "source": [ "learn.lr_find(end_lr=100)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEOCAYAAABIESrBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8XHW9//HXZ2Yy2ZMuSZekGy1LaaGlUCibLIqIghSu\nCrigKFxE/Sm4XPefeuW6Xflx1euCFRQXEJVFWQVEFitCaUtX2rK0dF/SJE3SLE1m5vP7Y05DKEma\nTDOZmeT9fDzmwZlzvuecz3w7zCff8zmLuTsiIiKpCGU6ABERyV1KIiIikjIlERERSZmSiIiIpExJ\nREREUqYkIiIiKVMSERGRlCmJiIhIypREREQkZUoiIiKSskimAxhIFRUVPmXKlEyHISKSM5YsWbLb\n3StTXX9IJZEpU6awePHiTIchIpIzzGzjoayvw1kiIpIyJREREUmZkoiIiKRMSURERFKmJCIiIilT\nEhERkZQpiYiI5LDV2xp48sWajO1fSUREJIf97plNfO5PyzO2fyUREZEcFk8kiIQsY/tXEhERyWGx\nhBNWEhERkVTE4q6RiIiIpCaukYiIiKQqlkiQF87cT7mSiIhIDtNIREREUhZLqCYiIiIpisU1EhER\nkRTFEgkiIdVEREQkBfGEEwlrJCIiIinQxYYiIpKyuArrIiKSqmRhXTURERFJQUw3YBQRkVTFVFgX\nEZFUqSYiIiIpU01ERERSppGIiIikLJZIEFZNREREUqEbMIqISMricde9s0REJDU6xVdERFI2ZB9K\nZWYTzexxM1tjZqvN7Npu2sw3sxVmtszMFpvZ6cH848zsX8F6K8zs0nTFKSKSyzJ9xXokjduOAZ91\n96VmVgosMbNH3f2FLm0eA+51dzezWcAfgelAC/BBd3/JzKqCdR929z1pjFdEJKckEk7CyehIJG1J\nxN23A9uD6SYzWwNUAy90abO3yyrFgAfzX+zSZpuZ7QIqASUREZFALOEA5IWHeGHdzKYAc4Bnu1l2\nsZmtBR4APtLN8pOAKPBKeqMUEckt8SCJDMmayH5mVgLcBVzn7o0HLnf3e9x9OnARcP0B644Hfgt8\n2N0TPWz/6qCesrimpmbgP4CISJaKJZI/i0P2OhEzyyOZQG5z97t7a+vuTwHTzKwiWLeM5Ojkq+7+\nTC/rLXD3ue4+t7KycgCjFxHJbkN6JGJmBtwCrHH3G3toc3jQDjM7nuRhq1oziwL3AL9x9z+lK0YR\nkVzWEU8mkaF6dtZpwOXASjNbFsz7MjAJwN1vAt4FfNDMOoBW4NLgTK1LgDOA0WZ2RbDuFe6+DBER\nAV4biUQyWFhP59lZC4Fe06O7fw/4Xjfzfwf8Lk2hiYgMCftrIkPycJaIiKRX50hESURERPorNpQL\n6yIikl6xzsL6EL/YUEREBl7ndSK6i6+IiPSXaiIiIpIy1URERCRlr41EVBMREZF+2l9Y10hERET6\nTYV1ERFJWUyFdRERSVVc14mIiEiqdHaWiIik7LW7+CqJiIhIP+kuviIikrL9p/jmqSYiIiL91fl4\nXB3OEhGR/tIpviIikrK4aiIiIpKqjnjmRyJpe8b6UJdIOI+t3cVja3Yyb+oo3n7MeArywpkOS0SG\nkddO8c3ceEBJBHh83S5GFUWpGlHI6OIosYTTEU/QEU/QHk/Q2NrBproWNte1Ut/Szp6WDp5Yt4tX\na1uIRkLc8dxmvvaX1bx1xlhOnjqak6aMYtKoIkK9/HUQTzjb9rTS3B6jeV+Mvfvi7G2LYQZVIwqp\nKi8g7k5LexyA0vwI+ZEwm+tbeLW2mV2N+9jT2kF7LMHMqjJOmDySqhGFndt3dxpbY2BQHA2n9CVr\naO1gfc1e1u5oYuXWBjbUNJOfF6I4GqFqRAFHjC3lqLGlHDWutN8JtK0jTm1zO4mgr+tb2tm9t526\n5uRrT0s7+2LJfwOAvCD+vW0xGtti1DbvY1fjPupb2pPHhR3Ki/IYU5rPyKIo0UiIgrwQUytKmFlV\nxrjygs7tFUXDlBbk4Q5798Vo64hjBmEz8iIhCiJhopHX/u3KCvOoKM7v9d9TJBOyoSYy7JOIu/PR\n3y6hPZbo8zol+RGOGlfKZ889irfNHMfijXX8afEWnlhXw91LtwKQHwkxZXQx08eXcmx1OYdVFFPf\n0sHOxjae37SHZzfU0tQWO+T4IyHr/CJFIyHKCvKIho3dze2v+0wFwY9/UX6YcWUFTBhZREVJFDPD\ngIQ7CYf6lnY21bbwam0Lu/fu61y/tCDCEWNKaGmPsamuhb+t2cm+YPshgykVxRTmhWltjxNLOMX5\nEUrywyQc2mMJQpb8MS6Khtmwu5lXapo7/4rqTkFeiIK8cOftHGKJBO7Jvi8tiFBRks+8qcWMKooS\nDhuGsaelnZ2NbdS3dBBrTdDSHufh1Tt73U9/+nlkcZRQkGzKi6KMKc2nOD9MS3ucto44Jfl5jCrO\no7wwj6JohJL8CGWFEcoL8ygrzKOsILlsZHGU4mgYMyUlOTTZUBNREnG45+Onsm1PG1vrW6hv6SAa\nCREJGdFIiLxwiJL8CBNHFTFxVCGjiqJv+Kv+1GkVnDqtgkTCeblmL0s21rO+Zi+v1DSzaEMdf1m2\n7XXtJ48u4oJZ45k9YUTyByc/0vnjGIs7W/e0sqOhlUg4RFE0/Lq/mKtGFDJldDHjywsoK8wj4c7a\n7U0s3VTP9oY2Glo72BeLU1mST2VpPgDN++JdRjwxtje0sWhDHXXN7TiOO4TMCIeM0oIIk0YV8ebp\nlUytLGFqRTFHji1l8uii1/3oxRPOproW1u1o5IXtTazd3kg84RRGw4RDltznvhihEJQVRIglnMa2\n5L4njyribTPHUT2ikHDIiISNkUVRKkryGVUcZVRxdMAODbZ1xFm3o4m65nbyIyEi4RCtHXGa2jqA\nZFIqzAvjwWdqjyfY1xFnXyzR+Xn3tLSzvaGNPS3tJBIQd2dPSzs1TfvYuidOcTRMfiTM1j2trNra\nQENrB60d8V7jikZCnSPEomiYqvJCJo4qZPLoYiaNKmLCyEJGF+czsjiZkDL5IyHZq/O2Jxn8g2TY\nJ5FQyJhZVc7MqvIB2daRY0s5cmzp6+bvampjS30ro4uTP5TF+b13+4yqsj7vM4xx7IRyjp1w6PH3\nRzhkHFZRzGEVxZx3zPhB3Xd/FOSFmT1xxKDvN55wmttjNLZ20NDaQWNrjMa25HR9cMiutSNORzxB\n8744W+pbeHxdDTVNW7rdXl7YKMgLU5IfoTg/QmVJPuPLCxhZHCU/khy1lRfmMaIoj+JohEjYKIom\nR48ji6OD/OllsMQTTsjI6KHWYZ9EBsOY0gLGlBZkOgwZROGQUVaQPIQ1YWTf19t/uHBrfSv1LcmE\n09Iepy0Wp7U93jma3NW0j2c31FHfkjxsGevlkN348gJmVpUza0I5M6vKgpFOEYVRnQiS6zrintE7\n+IKSiEhWKYpGmD6ujOnj+j4aBeiIJzpHOS3tcWKJBE1tMV7c2cTqbY2s2trAY2t34l1yzcRRhRxT\nVc7R48uYWpkcVU6rLNFZhjkknkhk9OaLoCQiMiTkhUNUlORTUZL/uvlnHTWmc7qprYMXdzaxua41\nqGc1sXpbAw+t2tHZJhwyDq8sYUZVGTOryphRVca0yhIqS3R2WjaKJTzj9TIlEZFhorQgjxMmj+KE\nya+f39oe59XaZtbXNLN2RyOrtzXy9Cu7uef5rZ1topEQlSX5wanTYaZWFnP0uFLmTBrJ3CkjyY9o\n9JIJ8YRn9PReUBIRGfYKo2GOHl/G0ePLOH/WaydJ7N67jxe2NbKxroVNtc3UNXfQHk+wt62DFVv2\n8MCK7QAURcOcOGUU48sLGFUcZdaEcs46aowOiw2C5EhENRERyUIVJfmccWRlj8sb2zpYtL6OJ1+s\n4blX61i9rZH6lnbiCackP8JbZ4zl3BljedORlZQc5IxESU0snhi6IxEzmwj8BhgHJIAF7v7DA9rM\nB64PlseA69x9YbDsQ8BXg6b/5e6/TlesItJ/ZQV5nDNjLOfMGNs5LxZP8Mz6Ou5bvo2/rt7BPc9v\nJRoOMf+4Kj791iNfd1cFOXSxhA/pwnoM+Ky7LzWzUmCJmT3q7i90afMYcK+7u5nNAv4ITDezUcDX\ngbmAB+ve6+71aYxXRA5RJBzi9CMqOP2ICr518TEs2VjPAyu3c8eizfxl+TY+dMpkrj5jWueFsHJo\nsqEmkraDae6+3d2XBtNNwBqg+oA2e907TzosJpkwAN4GPOrudUHieBQ4L12xisjAi4RDzJs6mm/O\nP4bH/+MsLpxdxS0LN3D69/7ON+5dzcba5kyHmPOGzdlZZjYFmAM8282yi4HvAGOA84PZ1cDmLs22\ncEACEpHcUT2ikBveM5tPnH04P3n8ZX77zEZuffpVTpwykvfPm8yFs6t0CnEK4llwsWHa925mJcBd\nJOsdjQcud/d73H06cBHJ+ghAd9+mbi/JNbOrzWyxmS2uqakZqLBFJA0OqyjmhvfMZuEXzubz5x1F\nbXM71/1hGe+66WlWbW3IdHg5J5ZIZHwkktYkYmZ5JBPIbe5+d29t3f0pYJqZVZAceUzssngCsK2H\n9Ra4+1x3n1tZ2fOZJCKSPcaXF/Lxsw7nb58+kxveM5vNdS2888cL+dYDL9B2kJtXymtiCScvw4X1\ntCURS94C9RZgjbvf2EObw4N2mNnxQBSoBR4GzjWzkWY2Ejg3mCciQ0goZLz7hAk89tmzeO9Jk/jF\nPzbwjh/9g+Wb92Q6tJwQz4KaSDpHIqcBlwNvNrNlwesdZnaNmV0TtHkXsMrMlgE/AS71pDqSh7ae\nC17fDOaJyBBUXpjHty8+lt9eeRJt7XEuW/AMz66vzXRYWS+WBTURcz/0B/Zki7lz5/rixYszHYaI\nHIKapn1ctuBf7Gho4zdXzuOEyf24DfIwc8lN/yIcMn5/9ckpb8PMlrj73FTXz2wKExE5QGVpPrf/\n+8lUluZzxS8X8Yun1nc+RExeL5YFd/FVEhGRrDO2rIDb//1kZlaX8a0H13Dqd/7Or59+NdNhZZ1s\nuE5ESUREslLViELuuPoU/vKJ0zhu0gi+fu9qHly5PdNhZZVsqIkoiYhIVps9cQS/+OBcjp80gs/8\ncZmuJ+liSN/2RERkoBTkhbnp8hMYWRTl6t8sZk9Le6ZDygqxRIKwaiIiIgc3prSABZfPZWfTPr7/\n8LpMh5MVNBIREemHYyeU86FTpnD7ok2s2KILEjviKqyLiPTLdW89goqSfP7vn1eRSAyd69xSEU84\neSqsi4j0XVlBHl95x9Es39LA75/blOlwMiqWcNVERET6a/5xVZwydTTfeXAtW/e0ZjqcjIknMv94\nXCUREck5ZsZ/v3sWCXe+cOeKYXtYSxcbioikaOKoIr5y/tEsfHk3tz27MdPhZITOzhIROQTvO2kS\nbzqigm8/uJadjW2ZDmfQxeJOJKzCuohISsyMb110LB3xBD/++8uZDmfQxVQTERE5NJNGF3HJiRO5\n47lNbK5ryXQ4gyaRcBKOaiIiIofqk28+HDPjR4+9lOlQBk08eBaURiIiIodofHkhH5g3mbuWbmF9\nzd5MhzMo4sEZaWFdbCgicug+fvY08iPhYVMb6YgnAI1EREQGREVJPpedNJF7l29j2zC4AHH/SERP\nNhQRGSBXnn4YDvxy4YZMh5J2sUQO1UTM7FozK7OkW8xsqZmdm+7gRET6Y8LIIi6YNZ7fL9pEQ+vQ\nfi57rtVEPuLujcC5QCXwYeC7aYtKRCRFV58xleb2+JC/ij2nRiLA/ijfAfzK3Zd3mScikjVmVpXz\npiMq+NU/X6WtI57pcNImFhTWc+U6kSVm9gjJJPKwmZUCifSFJSKSumvOnEZN0z7uXro106GkTSzH\nCutXAl8ETnT3FiCP5CEtEZGsc+q00cyaUM7Pn3qls3Yw1HSenZUjNZFTgHXuvsfMPgB8FWhIX1gi\nIqkzMz525jQ21rbw0KrtmQ4nLWLx/YX13BiJ/AxoMbPZwOeBjcBv0haViMghOnfmOKZWFPOzJ17B\nfeiNRuI5VliPefJfYT7wQ3f/IVCavrBERA5NOGRcc+Y0Vm9r5MkXazIdzoCLJYLCeo7URJrM7EvA\n5cADZhYmWRcREclaF82ppnpEITc8sm7IPf1wf2E9L0dqIpcC+0heL7IDqAa+n7aoREQGQDQS4nNv\nO5JVWxu5b8W2TIczoHKqJhIkjtuAcjO7AGhz915rImY20cweN7M1ZrbazK7tps37zWxF8Ho6qLns\nX/bpYL1VZvZ7Myvo52cTEWH+7GpmjC/j+w+vY19s6Fw3klP3zjKzS4BFwHuAS4BnzezdB1ktBnzW\n3Y8GTgY+YWYzDmizATjT3WcB1wMLgv1VA58C5rr7MUAYuKxvH0lE5DWhkPGld0xnS30rv/3X0LmK\nvbMmkgsjEeArJK8R+ZC7fxA4Cfi/va3g7tvdfWkw3QSsIXkYrGubp929Pnj7DDChy+IIUGhmEaAI\nGFpjUREZNG86opI3HVHBTx5/mdb2oTEaybWzs0LuvqvL+9p+rIuZTQHmAM/20uxK4CEAd98K3ABs\nArYDDe7+SF/3JyJyoE+cfTj1LR3c8/zQuIq9I5dqIsBfzexhM7vCzK4AHgAe7MuKZlYC3AVcF9zE\nsbs2Z5NMIl8I3o8keTrxYUAVUBxc5Njduleb2WIzW1xTM/RO4xORgTHvsFHMrCrjl//cMCTO1No/\nEskL58DZWe7+HyTrFbOA2cACd//CwdYzszySCeQ2d7+7hzazgJuB+e5eG8w+B9jg7jXu3gHcDZza\nQ2wL3H2uu8+trKzsy8cRkWHIzLjy9MN4eddennwp9//gzLWaCO5+l7t/xt0/7e73HKy9mRlwC7DG\n3W/soc0kkgnicnd/scuiTcDJZlYUbOctJGsqIiIpu2BWFWNK84fEQ6uypSYS6W2hmTUB3Y37DHB3\nL+tl9dNIXpy40syWBfO+DEwiufJNwNeA0cBPk7mCWDCqeNbM7gSWkjzL63mCM7dERFIVjYT44CmT\nueGRF1m3o4mjxuXujTdiieyoifSaRNw95R5294Uc5Jkj7n4VcFUPy74OfD3V/YuIdOd98ybzw8de\n4g/PbeZr7zzwqoPcsf9iw1y5i6+IyJAwqjjKW6aP5S/LttIRz93HIsWDmkhOXGwoIjKUvOuECdQ2\nt/NUDt+YMdcejysiMmSceWQlo4qj3LV0S6ZDSVk8S2oiSiIiMuxEIyEunF3F317YRUNLR6bDSUks\nx55sKCIypLz7hAm0xxM5e3dfjURERDJoZlUZR44t4e4cPaS1/6QA1URERDLAzJh/XDVLN+1h657W\nTIfTb/GEE7LkXYozSUlERIatC2aNB+CBHDykFUt4xushoCQiIsPY5NHFHFtdzv0rtmc6lH6LJzzj\n9RBQEhGRYe6CWeNZsaWBjbXNmQ6lX2Jxz3g9BJRERGSYO3//Ia2VuTUaiSUSGb9aHZRERGSYmzCy\niDmTRnD/8lxLIk5YNRERkcw7/9jxvLC9kZd37c10KH0W1+EsEZHscOFxVUQjIX7x1PpMh9JnMRXW\nRUSyw5jSAt530iTuWrqFzXUtmQ6nT+KqiYiIZI+PnjmVkBk/e/KVTIfSJxqJiIhkkfHlhVxy4gT+\ntHgz23LgCvZY3MlTYV1EJHt87KzDAfh5DoxGNBIREcky1SMKeefsKu5csoXmfbFMh9Mr1URERLLQ\n++dNprk9zr3Ls/t+WhqJiIhkoeMnjeCosaXc/uymTIfSq3hC14mIiGQdM+N98yaxcmsDK7c0ZDqc\nHiXvnZX5n/DMRyAikmUumlNNQV6I2xdl72hE984SEclS5YV5XDCrinuXbWVvlhbYdSt4EZEs9t6T\nJtLcHuevq3ZkOpRuxVQTERHJXsdPGsnEUYX8ZdnWTIfSLY1ERESymJkxf3Y1/3x5N7ua2jIdzht0\nxBNEwpn/Cc98BCIiWeqiOVUkHO7LwmeN6BRfEZEsd/iYUmZWlWXlIS1dbCgikgPmH1fFii0NrK/J\nrgdWDfmRiJlNNLPHzWyNma02s2u7afN+M1sRvJ42s9ldlo0wszvNbG2wjVPSFauISE8unF2NGdy9\nNLtGI8Ph8bgx4LPufjRwMvAJM5txQJsNwJnuPgu4HljQZdkPgb+6+3RgNrAmjbGKiHRrXHkBb5k+\nlluffpXde/dlOpxOQ34k4u7b3X1pMN1EMglUH9DmaXevD94+A0wAMLMy4AzglqBdu7vvSVesIiK9\n+eLbp9PWEefGR1/MdCidkmdnDeEk0pWZTQHmAM/20uxK4KFgeipQA/zKzJ43s5vNrLiHbV9tZovN\nbHFNTc0ARi0iknT4mBI+cPJk7li0ibU7GjMdDjAMRiL7mVkJcBdwnbt32/tmdjbJJPKFYFYEOB74\nmbvPAZqBL3a3rrsvcPe57j63srJywOMXEQG47pwjKC3I45v3vUB7LJHpcIZFTQQzyyOZQG5z97t7\naDMLuBmY7+61wewtwBZ33z9yuZNkUhERyYgRRVE+d+6RPP1KLef94CkeW7MTd89YPEN+JGJmRrKm\nscbdb+yhzSTgbuByd+882OjuO4DNZnZUMOstwAvpilVEpC8+cPJkfnXFiWBw5a8Xc8vCDRmJw92z\n5rYnkTRu+zTgcmClmS0L5n0ZmATg7jcBXwNGAz9N5hxi7j43aPtJ4DYziwLrgQ+nMVYRkYMyM86e\nPobTj6jgvQue4fZFm7jy9MMIfr8GTSyRHAHlZUFhPW1JxN0XAr1+Qne/Criqh2XLgLndLRMRyaS8\ncIiLj6/mK/esYs32JmZUlQ3q/uNBEhnyNRERkaHqvJnjCIeMB1YO/rPYX9iePEdpRFHeoO/7QEoi\nIiIpGF2Sz6nTRnP/iu2DXmD/wd9eYlRxlAtnVw3qfrujJCIikqLzjx3PxtoWVm0dvGtHlmys46kX\na/joGVMpzk9nWbtvlERERFJ03jHjiISM+wfxkNb/PPoSFSVRLj9l8qDtszdKIiIiKRpRFOX0Iyq4\nf/ngHNJ67tU6Fr68m4+eMY2iaOZHIaAkIiJySC6cXcXWPa0892r9wRsfor+v3UUkZHzg5OwYhYCS\niIjIIXnbzHEURcPcvXRL2vdVt7edUcVRCqPhtO+rr5REREQOQXF+hLcfM54HVmynrSOe1n3VNieT\nSDZREhEROUTvOqGapn0xHl69I637qW9REhERGXJOPmw01SMK0/70wzqNREREhp5QyLh4TjX/eKmG\nnY1taduPkoiIyBD1b8dXk3D4+G1LeXlX04BvPxZP0NDaoSQiIjIUTa0s4X8unc0rNXt5+w//wU1P\nvjKg269v6QBQEhERGaounjOBv33mTN4yfSzffWgtC1/aPWDbrm9pB5RERESGtIqSfH5w2XFMrSjm\nS/esoLV9YE77rd0bJJEiJRERkSGtIC/Md/7tWDbXtXLjo+sGZJudI5ESJRERkSFv3tTRvG/eJG5Z\nuIHlm/cc8vZqmzUSEREZVr749umMKS3g83euoD2WOKRt1QdJZKRqIiIiw0NZQR7fuvgY1u1s4ieP\nv3xI26prbqe0IEJeOLt+trMrGhGRIeYtR4/l4jnV/OTxl3lhW+oPr8rGCw1BSUREJO2+dsEMRhTl\n8bHblrBme2qJJBvvmwVKIiIiaTeyOMrPLz+BlvY4F/3kn9yxaFO/t1G7tz3riuqgJCIiMihOmDyK\nBz/1JuZOGckX717JPc/37/kjGomIiAxzlaX5/OYj8zhu4giuv39N5xlXB+PuWfksEVASEREZVOGQ\n8e2Lj6WhtYPvPrS2T+u0tMdpjyWUREREBGZUlXHV6Yfxh8WbWbSh7qDt67L0GhFQEhERyYhrzzmC\n6hGF/PdfDz4a2Z9ERiuJiIgIQFE0wvvmTWLxxnq21Lf02rauRSMRERE5wDtnVQFw3/Ltvbar2zsM\nRyJmNtHMHjezNWa22syu7abN+81sRfB62sxmH7A8bGbPm9n96YpTRCRTJo0u4riJI7h3+bZe29UP\n05FIDPisux8NnAx8wsxmHNBmA3Cmu88CrgcWHLD8WmBNGmMUEcmo+cdVsWZ7Iy/t7PmRurXN7eSF\njdL8yCBG1jdpSyLuvt3dlwbTTSSTQfUBbZ529/rg7TPAhP3LzGwCcD5wc7piFBHJtPNnjSdk9Doa\nqW9uZ2RRFDMbxMj6ZlBqImY2BZgDPNtLsyuBh7q8/wHweeDQ7p8sIpLFxpQWcMq00dy7fBvu3m2b\nbL35IgxCEjGzEuAu4Dp37/bOY2Z2Nskk8oXg/QXALndf0oftX21mi81scU1NzQBGLiIyOObPrmZj\nbQvLenh41bBNImaWRzKB3Obud/fQZhbJQ1bz3b02mH0acKGZvQrcAbzZzH7X3fruvsDd57r73MrK\nygH/DCIi6fa2Y8YRjYT48/Nbu11e19KelUV1SO/ZWQbcAqxx9xt7aDMJuBu43N1f3D/f3b/k7hPc\nfQpwGfB3d/9AumIVEcmk8sI83nr0WO5bsZ2O+BuP4Nc1t2fl6b2Q3pHIacDlJEcRy4LXO8zsGjO7\nJmjzNWA08NNg+eI0xiMikrUunlNNXXM7T657/WH53Xv3saelg0mjijIUWe/Sdr6Yuy8Eej2VwN2v\nAq46SJsngCcGLDARkSx05lGVjCqOcs/zWzlnxtjO+Su3NgBwbHV5pkLrla5YFxHJAnnhEO+cNZ5H\n1+yksa2jc/7KLQ2YwUwlERER6c3Fx0+gPZbgoZWv3QZlxZYGplYUU5KFFxqCkoiISNaYPaGcwyqK\nX3cvrVVbG5g1YUQGo+qdkoiISJYwM942cxzPrK+loaWDXY1t7Ghs45gsPZQFSiIiIlnl3JljiSWc\nv6/b2VlUnzUhe5NIdh5kExEZpo6bMIIxpfk8snonR40rxQxmjC/LdFg9UhIREckioZBx7syx3LVk\nK41tHRxeWUJxlhbVQYezRESyzrkzxtHaEeefL9dybBYfygIlERGRrHPy1NGUFiRHH9l6keF+SiIi\nIlkmGgnx5uljgOwuqoNqIiIiWemDp0ymrrmdmVVKIiIi0k8nTB7Fb6+cl+kwDkqHs0REJGVKIiIi\nkjIlERFbE6npAAAIsUlEQVQRSZmSiIiIpExJREREUqYkIiIiKVMSERGRlCmJiIhIyszdMx3DgDGz\nGmBjH5uXAw392Hxf2vfUpq/z+/O+Ath9kHj6a6D7pLfl3S3ry7zB7JPB/I70tKy/fZJr35He2vRn\nfk99cOD74dIn/Xl/lLuXHiSenrn7sHwBCwa6fU9t+jq/P++BxdneJ70t725ZX+YNZp8M5ndkoPok\n174jvbXpz/ye+mC49slg/n8znA9n3ZeG9j216ev8/r4faAPdJ70t725ZX+YNZp8M5nekp2X97ZNc\n+4701qY/83vrg+HYJ4P2/82QOpw1nJjZYnefm+k4son65PXUH2+kPnmjQ+2T4TwSyXULMh1AFlKf\nvJ76443UJ290SH2ikYiIiKRMIxEREUmZkoiIiKRMSURERFKmJDJEmVmxmS0xswsyHUummdnRZnaT\nmd1pZh/LdDzZwMwuMrNfmNlfzOzcTMeTDcxsqpndYmZ3ZjqWTAl+N34dfDfe35d1lESyjJn90sx2\nmdmqA+afZ2brzOxlM/tiHzb1BeCP6Yly8AxEf7j7Gne/BrgEyPnTOweoT/7s7v8OXAFcmsZwB8UA\n9cl6d78yvZEOvn72zb8BdwbfjQv7sn0lkexzK3Be1xlmFgZ+ArwdmAG818xmmNmxZnb/Aa8xZnYO\n8AKwc7CDT4NbOcT+CNa5EFgIPDa44afFrQxAnwS+GqyX625l4PpkqLmVPvYNMAHYHDSL92XjkQEL\nUwaEuz9lZlMOmH0S8LK7rwcwszuA+e7+HeANh6vM7GygmOSXo9XMHnT3RFoDT5OB6I9gO/cC95rZ\nA8Dt6Ys4/QboO2LAd4GH3H1peiNOv4H6ngxF/ekbYAvJRLKMPg4ylERyQzWv/XUAyX/oeT01dvev\nAJjZFcDuXE0gvehXf5jZWSSH6fnAg2mNLHP61SfAJ4FzgHIzO9zdb0pncBnS3+/JaOBbwBwz+1KQ\nbIaqnvrmR8CPzex8+nhrFCWR3GDdzDvoVaLufuvAh5IV+tUf7v4E8ES6gskS/e2TH5H8wRjK+tsn\ntcA16Qsnq3TbN+7eDHy4PxtSTSQ3bAEmdnk/AdiWoViygfrjjdQnb6Q+6dmA9Y2SSG54DjjCzA4z\nsyhwGXBvhmPKJPXHG6lP3kh90rMB6xslkSxjZr8H/gUcZWZbzOxKd48B/wd4GFgD/NHdV2cyzsGi\n/ngj9ckbqU96lu6+0Q0YRUQkZRqJiIhIypREREQkZUoiIiKSMiURERFJmZKIiIikTElERERSpiQi\nGWNmewdhHxf28db5A7nPs8zs1BTWm2NmNwfTV5jZjwc+uv4zsykH3ka8mzaVZvbXwYpJsoeSiOS8\n4LbW3XL3e939u2nYZ2/3nTsL6HcSAb4M/G9KAWWYu9cA283stEzHIoNLSUSygpn9h5k9Z2YrzOw/\nu8z/syWf0LjazK7uMn+vmX3TzJ4FTjGzV83sP81sqZmtNLPpQbvOv+jN7FYz+5GZPW1m683s3cH8\nkJn9NNjH/Wb24P5lB8T4hJl928yeBK41s3ea2bNm9ryZ/c3Mxga33L4G+LSZLTOzNwV/pd8VfL7n\nuvuhNbNSYJa7L+9m2WQzeyzom8fMbFIwf5qZPRNs85vdjews+aS6B8xsuZmtMrNLg/knBv2w3MwW\nmVlpMOL4R9CHS7sbTZlZ2My+3+Xf6qNdFv8Z6NPT8GQIcXe99MrIC9gb/PdcYAHJO4uGgPuBM4Jl\no4L/FgKrgNHBewcu6bKtV4FPBtMfB24Opq8AfhxM3wr8KdjHDJLPUwB4N8lbxIeAcUA98O5u4n0C\n+GmX9yN57a4PVwH/L5j+BvC5Lu1uB04PpicBa7rZ9tnAXV3ed437PuBDwfRHgD8H0/cD7w2mr9nf\nnwds913AL7q8LweiwHrgxGBeGck7ehcBBcG8I4DFwfQUYFUwfTXw1WA6H1gMHBa8rwZWZvp7pdfg\nvnQreMkG5wav54P3JSR/xJ4CPmVmFwfzJwbza0k+de2uA7Zzd/DfJSSfH9KdP3vy+SovmNnYYN7p\nwJ+C+TvM7PFeYv1Dl+kJwB/MbDzJH+YNPaxzDjDDrPPu22VmVuruTV3ajAdqelj/lC6f57fAf3eZ\nf1EwfTtwQzfrrgRuMLPvAfe7+z/M7Fhgu7s/B+DujZActZB8lsRxJPv3yG62dy4wq8tIrZzkv8kG\nYBdQ1cNnkCFKSUSygQHfcfefv25m8mFS5wCnuHuLmT0BFASL29z9wMd37gv+G6fn7/a+LtN2wH/7\nornL9P8CN7r7vUGs3+hhnRDJz9Day3Zbee2zHUyfb3jn7i+a2QnAO4DvmNkjJA87dbeNT5N8pPLs\nIOa2btoYyRHfw90sKyD5OWQYUU1EssHDwEfMrATAzKot+czrcqA+SCDTgZPTtP+FwLuC2shYkoXx\nvigHtgbTH+oyvwko7fL+EZJ3TAUg+Ev/QGuAw3vYz9Mkb9UNyZrDwmD6GZKHq+iy/HXMrApocfff\nkRypHA+sBarM7MSgTWlwokA5yRFKArgc6O6EhYeBj5lZXrDukcEIBpIjl17P4pKhR0lEMs7dHyF5\nOOZfZrYSuJPkj/BfgYiZrQCuJ/mjmQ53kXxIzyrg58CzQEMf1vsG8Ccz+wewu8v8+4CL9xfWgU8B\nc4NC9At08/Q8d19L8lG1pQcuC9b/cNAPlwPXBvOvAz5jZotIHg7rLuZjgUVmtgz4CvBf7t4OXAr8\nr5ktBx4lOYr4KfAhM3uGZEJo7mZ7NwMvAEuD035/zmujvrOBB7pZR4Yw3QpeBDCzEnffa8nnbC8C\nTnP3HYMcw6eBJne/uY/ti4BWd3czu4xkkX1+WoPsPZ6ngPnuXp+pGGTwqSYiknS/mY0gWSC/frAT\nSOBnwHv60f4EkoVwA/aQPHMrI8yskmR9SAlkmNFIREREUqaaiIiIpExJREREUqYkIiIiKVMSERGR\nlCmJiIhIypREREQkZf8fhD6p+kyx6AgAAAAASUVORK5CYII=\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.sched.plot()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "c1285ab3ded54265bb94ee2bd513493f", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.72594 1.63399 0.41338] \n", "[ 1. 1.51599 1.49687 0.45723] \n", "\n", "CPU times: user 1min 14s, sys: 32.3 s, total: 1min 46s\n", "Wall time: 56.5 s\n" ] } ], "source": [ "%time learn.fit(1e-1, 2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "5ced4fe4a75a4425a4edc3fac43af952", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.36734 1.28901 0.53418] \n", "[ 1. 1.28854 1.21991 0.56143] \n", "[ 2. 1.22854 1.15514 0.58398] \n", "[ 3. 1.17904 1.12523 0.59922] \n", "\n", "CPU times: user 2min 21s, sys: 1min 3s, total: 3min 24s\n", "Wall time: 1min 46s\n" ] } ], "source": [ "%time learn.fit(1e-1, 4, cycle_len=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Refactored" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class ConvLayer(nn.Module):\n", " def __init__(self, ni, nf):\n", " super().__init__()\n", " self.conv = nn.Conv2d(ni, nf, kernel_size=3, stride=2, padding=1)\n", " \n", " def forward(self, x): return F.relu(self.conv(x))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class ConvNet2(nn.Module):\n", " def __init__(self, layers, c):\n", " super().__init__()\n", " self.layers = nn.ModuleList([ConvLayer(layers[i], layers[i + 1])\n", " for i in range(len(layers) - 1)])\n", " self.out = nn.Linear(layers[-1], c)\n", " \n", " def forward(self, x):\n", " for l in self.layers: x = l(x)\n", " x = F.adaptive_max_pool2d(x, 1)\n", " x = x.view(x.size(0), -1)\n", " return F.log_softmax(self.out(x), dim=-1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = ConvLearner.from_model_data(ConvNet2([3, 20, 40, 80], 10), data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "OrderedDict([('Conv2d-1',\n", " OrderedDict([('input_shape', [-1, 3, 32, 32]),\n", " ('output_shape', [-1, 20, 16, 16]),\n", " ('trainable', True),\n", " ('nb_params', 560)])),\n", " ('ConvLayer-2',\n", " OrderedDict([('input_shape', [-1, 3, 32, 32]),\n", " ('output_shape', [-1, 20, 16, 16]),\n", " ('nb_params', 0)])),\n", " ('Conv2d-3',\n", " OrderedDict([('input_shape', [-1, 20, 16, 16]),\n", " ('output_shape', [-1, 40, 8, 8]),\n", " ('trainable', True),\n", " ('nb_params', 7240)])),\n", " ('ConvLayer-4',\n", " OrderedDict([('input_shape', [-1, 20, 16, 16]),\n", " ('output_shape', [-1, 40, 8, 8]),\n", " ('nb_params', 0)])),\n", " ('Conv2d-5',\n", " OrderedDict([('input_shape', [-1, 40, 8, 8]),\n", " ('output_shape', [-1, 80, 4, 4]),\n", " ('trainable', True),\n", " ('nb_params', 28880)])),\n", " ('ConvLayer-6',\n", " OrderedDict([('input_shape', [-1, 40, 8, 8]),\n", " ('output_shape', [-1, 80, 4, 4]),\n", " ('nb_params', 0)])),\n", " ('Linear-7',\n", " OrderedDict([('input_shape', [-1, 80]),\n", " ('output_shape', [-1, 10]),\n", " ('trainable', True),\n", " ('nb_params', 810)]))])" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.summary()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "27475f522efa4c8aa396ab71a7d8d6cb", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.70151 1.64982 0.3832 ] \n", "[ 1. 1.50838 1.53231 0.44795] \n", "\n", "CPU times: user 1min 6s, sys: 28.5 s, total: 1min 35s\n", "Wall time: 48.8 s\n" ] } ], "source": [ "%time learn.fit(1e-1, 2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "53263fbac4df4d68a529026d9fcbbd91", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.51605 1.42927 0.4751 ] \n", "[ 1. 1.40143 1.33511 0.51787] \n", "\n", "CPU times: user 1min 6s, sys: 27.7 s, total: 1min 34s\n", "Wall time: 48.7 s\n" ] } ], "source": [ "%time learn.fit(1e-1, 2, cycle_len=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## BatchNorm" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class BnLayer(nn.Module):\n", " def __init__(self, ni, nf, stride=2, kernel_size=3):\n", " super().__init__()\n", " self.conv = nn.Conv2d(ni, nf, kernel_size=kernel_size, stride=stride,\n", " bias=False, padding=1)\n", " self.a = nn.Parameter(torch.zeros(nf,1,1))\n", " self.m = nn.Parameter(torch.ones(nf,1,1))\n", " \n", " def forward(self, x):\n", " x = F.relu(self.conv(x))\n", " x_chan = x.transpose(0,1).contiguous().view(x.size(1), -1)\n", " if self.training:\n", " self.means = x_chan.mean(1)[:,None,None]\n", " self.stds = x_chan.std (1)[:,None,None]\n", " return (x-self.means) / self.stds *self.m + self.a" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class ConvBnNet(nn.Module):\n", " def __init__(self, layers, c):\n", " super().__init__()\n", " self.conv1 = nn.Conv2d(3, 10, kernel_size=5, stride=1, padding=2)\n", " self.layers = nn.ModuleList([BnLayer(layers[i], layers[i + 1])\n", " for i in range(len(layers) - 1)])\n", " self.out = nn.Linear(layers[-1], c)\n", " \n", " def forward(self, x):\n", " x = self.conv1(x)\n", " for l in self.layers: x = l(x)\n", " x = F.adaptive_max_pool2d(x, 1)\n", " x = x.view(x.size(0), -1)\n", " return F.log_softmax(self.out(x), dim=-1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = ConvLearner.from_model_data(ConvBnNet([10, 20, 40, 80, 160], 10), data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "OrderedDict([('Conv2d-1',\n", " OrderedDict([('input_shape', [-1, 3, 32, 32]),\n", " ('output_shape', [-1, 10, 32, 32]),\n", " ('trainable', True),\n", " ('nb_params', 760)])),\n", " ('Conv2d-2',\n", " OrderedDict([('input_shape', [-1, 10, 32, 32]),\n", " ('output_shape', [-1, 20, 16, 16]),\n", " ('trainable', True),\n", " ('nb_params', 1800)])),\n", " ('BnLayer-3',\n", " OrderedDict([('input_shape', [-1, 10, 32, 32]),\n", " ('output_shape', [-1, 20, 16, 16]),\n", " ('nb_params', 0)])),\n", " ('Conv2d-4',\n", " OrderedDict([('input_shape', [-1, 20, 16, 16]),\n", " ('output_shape', [-1, 40, 8, 8]),\n", " ('trainable', True),\n", " ('nb_params', 7200)])),\n", " ('BnLayer-5',\n", " OrderedDict([('input_shape', [-1, 20, 16, 16]),\n", " ('output_shape', [-1, 40, 8, 8]),\n", " ('nb_params', 0)])),\n", " ('Conv2d-6',\n", " OrderedDict([('input_shape', [-1, 40, 8, 8]),\n", " ('output_shape', [-1, 80, 4, 4]),\n", " ('trainable', True),\n", " ('nb_params', 28800)])),\n", " ('BnLayer-7',\n", " OrderedDict([('input_shape', [-1, 40, 8, 8]),\n", " ('output_shape', [-1, 80, 4, 4]),\n", " ('nb_params', 0)])),\n", " ('Conv2d-8',\n", " OrderedDict([('input_shape', [-1, 80, 4, 4]),\n", " ('output_shape', [-1, 160, 2, 2]),\n", " ('trainable', True),\n", " ('nb_params', 115200)])),\n", " ('BnLayer-9',\n", " OrderedDict([('input_shape', [-1, 80, 4, 4]),\n", " ('output_shape', [-1, 160, 2, 2]),\n", " ('nb_params', 0)])),\n", " ('Linear-10',\n", " OrderedDict([('input_shape', [-1, 160]),\n", " ('output_shape', [-1, 10]),\n", " ('trainable', True),\n", " ('nb_params', 1610)]))])" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.summary()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "4cb9114a5d804bab9fe1ded933eae9f4", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.4966 1.39257 0.48965] \n", "[ 1. 1.2975 1.20827 0.57148] \n", "\n", "CPU times: user 1min 16s, sys: 32.5 s, total: 1min 49s\n", "Wall time: 54.3 s\n" ] } ], "source": [ "%time learn.fit(3e-2, 2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "4311194e230b43a89c1a842948a030b5", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.20966 1.07735 0.61504] \n", "[ 1. 1.0771 0.97338 0.65215] \n", "[ 2. 1.00103 0.91281 0.67402] \n", "[ 3. 0.93574 0.89293 0.68135] \n", "\n", "CPU times: user 2min 34s, sys: 1min 4s, total: 3min 39s\n", "Wall time: 1min 50s\n" ] } ], "source": [ "%time learn.fit(1e-1, 4, cycle_len=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Deep BatchNorm" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class ConvBnNet2(nn.Module):\n", " def __init__(self, layers, c):\n", " super().__init__()\n", " self.conv1 = nn.Conv2d(3, 10, kernel_size=5, stride=1, padding=2)\n", " self.layers = nn.ModuleList([BnLayer(layers[i], layers[i+1])\n", " for i in range(len(layers) - 1)])\n", " self.layers2 = nn.ModuleList([BnLayer(layers[i+1], layers[i + 1], 1)\n", " for i in range(len(layers) - 1)])\n", " self.out = nn.Linear(layers[-1], c)\n", " \n", " def forward(self, x):\n", " x = self.conv1(x)\n", " for l,l2 in zip(self.layers, self.layers2):\n", " x = l(x)\n", " x = l2(x)\n", " x = F.adaptive_max_pool2d(x, 1)\n", " x = x.view(x.size(0), -1)\n", " return F.log_softmax(self.out(x), dim=-1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = ConvLearner.from_model_data(ConvBnNet2([10, 20, 40, 80, 160], 10), data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "97674b4107f24900a78b115739f1b294", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.53499 1.43782 0.47588] \n", "[ 1. 1.28867 1.22616 0.55537] \n", "\n", "CPU times: user 1min 22s, sys: 34.5 s, total: 1min 56s\n", "Wall time: 58.2 s\n" ] } ], "source": [ "%time learn.fit(1e-2, 2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e1f4198b659c497fb131aaafbdbe449b", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.10933 1.06439 0.61582] \n", "[ 1. 1.04663 0.98608 0.64609] \n", "\n", "CPU times: user 1min 21s, sys: 32.9 s, total: 1min 54s\n", "Wall time: 57.6 s\n" ] } ], "source": [ "%time learn.fit(1e-2, 2, cycle_len=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Resnet" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class ResnetLayer(BnLayer):\n", " def forward(self, x): return x + super().forward(x)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class Resnet(nn.Module):\n", " def __init__(self, layers, c):\n", " super().__init__()\n", " self.conv1 = nn.Conv2d(3, 10, kernel_size=5, stride=1, padding=2)\n", " self.layers = nn.ModuleList([BnLayer(layers[i], layers[i+1])\n", " for i in range(len(layers) - 1)])\n", " self.layers2 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)\n", " for i in range(len(layers) - 1)])\n", " self.layers3 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)\n", " for i in range(len(layers) - 1)])\n", " self.out = nn.Linear(layers[-1], c)\n", " \n", " def forward(self, x):\n", " x = self.conv1(x)\n", " for l,l2,l3 in zip(self.layers, self.layers2, self.layers3):\n", " x = l3(l2(l(x)))\n", " x = F.adaptive_max_pool2d(x, 1)\n", " x = x.view(x.size(0), -1)\n", " return F.log_softmax(self.out(x), dim=-1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = ConvLearner.from_model_data(Resnet([10, 20, 40, 80, 160], 10), data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "wd=1e-5" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "df09e75913b1421cbbd6a1972d501799", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.58191 1.40258 0.49131] \n", "[ 1. 1.33134 1.21739 0.55625] \n", "\n", "CPU times: user 1min 27s, sys: 34.3 s, total: 2min 1s\n", "Wall time: 1min 3s\n" ] } ], "source": [ "%time learn.fit(1e-2, 2, wds=wd)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "b3a4151f72b84c40a8df56b6aea340bb", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.11534 1.05117 0.62549] \n", "[ 1. 1.06272 0.97874 0.65185] \n", "[ 2. 0.92913 0.90472 0.68154] \n", "[ 3. 0.97932 0.94404 0.67227] \n", "[ 4. 0.88057 0.84372 0.70654] \n", "[ 5. 0.77817 0.77815 0.73018] \n", "[ 6. 0.73235 0.76302 0.73633] \n", "\n", "CPU times: user 5min 2s, sys: 1min 59s, total: 7min 1s\n", "Wall time: 3min 39s\n" ] } ], "source": [ "%time learn.fit(1e-2, 3, cycle_len=1, cycle_mult=2, wds=wd)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "849a1d31a0ca4c39bacd99d34603d642", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 0.8307 0.83635 0.7126 ] \n", "[ 1. 0.74295 0.73682 0.74189] \n", "[ 2. 0.66492 0.69554 0.75996] \n", "[ 3. 0.62392 0.67166 0.7625 ] \n", "[ 4. 0.73479 0.80425 0.72861] \n", "[ 5. 0.65423 0.68876 0.76318] \n", "[ 6. 0.58608 0.64105 0.77783] \n", "[ 7. 0.55738 0.62641 0.78721] \n", "[ 8. 0.66163 0.74154 0.7501 ] \n", "[ 9. 0.59444 0.64253 0.78106] \n", "[ 10. 0.53 0.61772 0.79385] \n", "[ 11. 0.49747 0.65968 0.77832] \n", "[ 12. 0.59463 0.67915 0.77422] \n", "[ 13. 0.55023 0.65815 0.78106] \n", "[ 14. 0.48959 0.59035 0.80273] \n", "[ 15. 0.4459 0.61823 0.79336] \n", "[ 16. 0.55848 0.64115 0.78018] \n", "[ 17. 0.50268 0.61795 0.79541] \n", "[ 18. 0.45084 0.57577 0.80654] \n", "[ 19. 0.40726 0.5708 0.80947] \n", "[ 20. 0.51177 0.66771 0.78232] \n", "[ 21. 0.46516 0.6116 0.79932] \n", "[ 22. 0.40966 0.56865 0.81172] \n", "[ 23. 0.3852 0.58161 0.80967] \n", "[ 24. 0.48268 0.59944 0.79551] \n", "[ 25. 0.43282 0.56429 0.81182] \n", "[ 26. 0.37634 0.54724 0.81797] \n", "[ 27. 0.34953 0.54169 0.82129] \n", "[ 28. 0.46053 0.58128 0.80342] \n", "[ 29. 0.4041 0.55185 0.82295] \n", "[ 30. 0.3599 0.53953 0.82861] \n", "[ 31. 0.32937 0.55605 0.82227] \n", "\n", "CPU times: user 22min 52s, sys: 8min 58s, total: 31min 51s\n", "Wall time: 16min 38s\n" ] } ], "source": [ "%time learn.fit(1e-2, 8, cycle_len=4, wds=wd)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Resnet 2" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class Resnet2(nn.Module):\n", " def __init__(self, layers, c, p=0.5):\n", " super().__init__()\n", " self.conv1 = BnLayer(3, 16, stride=1, kernel_size=7)\n", " self.layers = nn.ModuleList([BnLayer(layers[i], layers[i+1])\n", " for i in range(len(layers) - 1)])\n", " self.layers2 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)\n", " for i in range(len(layers) - 1)])\n", " self.layers3 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)\n", " for i in range(len(layers) - 1)])\n", " self.out = nn.Linear(layers[-1], c)\n", " self.drop = nn.Dropout(p)\n", " \n", " def forward(self, x):\n", " x = self.conv1(x)\n", " for l,l2,l3 in zip(self.layers, self.layers2, self.layers3):\n", " x = l3(l2(l(x)))\n", " x = F.adaptive_max_pool2d(x, 1)\n", " x = x.view(x.size(0), -1)\n", " x = self.drop(x)\n", " return F.log_softmax(self.out(x), dim=-1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = ConvLearner.from_model_data(Resnet2([16, 32, 64, 128, 256], 10, 0.2), data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "wd=1e-6" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "989b720e9a8542baa1e925ebd8473a93", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.7051 1.53364 0.46885] \n", "[ 1. 1.47858 1.34297 0.52734] \n", "\n", "CPU times: user 1min 29s, sys: 35.4 s, total: 2min 4s\n", "Wall time: 1min 6s\n" ] } ], "source": [ "%time learn.fit(1e-2, 2, wds=wd)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "d338e700ad424bb283ebf7980e70f827", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 1.29414 1.26694 0.57041] \n", "[ 1. 1.21206 1.06634 0.62373] \n", "[ 2. 1.05583 1.0129 0.64258] \n", "[ 3. 1.09763 1.11568 0.61318] \n", "[ 4. 0.97597 0.93726 0.67266] \n", "[ 5. 0.86295 0.82655 0.71426] \n", "[ 6. 0.827 0.8655 0.70244] \n", "\n", "CPU times: user 5min 11s, sys: 1min 58s, total: 7min 9s\n", "Wall time: 3min 48s\n" ] } ], "source": [ "%time learn.fit(1e-2, 3, cycle_len=1, cycle_mult=2, wds=wd)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "37ae7f5e5d6b475aab2c0c4438140f00", "version_major": 2, "version_minor": 0 }, "text/plain": [ "A Jupyter Widget" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "[ 0. 0.92043 0.93876 0.67685] \n", "[ 1. 0.8359 0.81156 0.72168] \n", "[ 2. 0.73084 0.72091 0.74463] \n", "[ 3. 0.68688 0.71326 0.74824] \n", "[ 4. 0.81046 0.79485 0.72354] \n", "[ 5. 0.72155 0.68833 0.76006] \n", "[ 6. 0.63801 0.68419 0.76855] \n", "[ 7. 0.59678 0.64972 0.77363] \n", "[ 8. 0.71126 0.78098 0.73828] \n", "[ 9. 0.63549 0.65685 0.7708 ] \n", "[ 10. 0.56837 0.63656 0.78057] \n", "[ 11. 0.52093 0.59159 0.79629] \n", "[ 12. 0.66463 0.69927 0.76357] \n", "[ 13. 0.58121 0.64529 0.77871] \n", "[ 14. 0.52346 0.5751 0.80293] \n", "[ 15. 0.47279 0.55094 0.80498] \n", "[ 16. 0.59857 0.64519 0.77559] \n", "[ 17. 0.54384 0.68057 0.77676] \n", "[ 18. 0.48369 0.5821 0.80273] \n", "[ 19. 0.43456 0.54708 0.81182] \n", "[ 20. 0.54963 0.65753 0.78203] \n", "[ 21. 0.49259 0.55957 0.80791] \n", "[ 22. 0.43646 0.55221 0.81309] \n", "[ 23. 0.39269 0.55158 0.81426] \n", "[ 24. 0.51039 0.61335 0.7998 ] \n", "[ 25. 0.4667 0.56516 0.80869] \n", "[ 26. 0.39469 0.5823 0.81299] \n", "[ 27. 0.36389 0.51266 0.82764] \n", "[ 28. 0.48962 0.55353 0.81201] \n", "[ 29. 0.4328 0.55394 0.81328] \n", "[ 30. 0.37081 0.50348 0.83359] \n", "[ 31. 0.34045 0.52052 0.82949] \n", "\n", "CPU times: user 23min 30s, sys: 9min 1s, total: 32min 32s\n", "Wall time: 17min 16s\n" ] } ], "source": [ "%time learn.fit(1e-2, 8, cycle_len=4, wds=wd)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn.save('tmp3')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " \r" ] } ], "source": [ "log_preds,y = learn.TTA()\n", "preds = np.mean(np.exp(log_preds),0)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(0.44507397166057938, 0.84909999999999997)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "metrics.log_loss(y,preds), accuracy_np(preds,y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### End" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 2 }