{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## CSV Logger" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "hide_input": true }, "outputs": [], "source": [ "from fastai.vision import *\n", "from fastai.gen_doc.nbdoc import *\n", "from fastai.callbacks import *" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

class CSVLogger[source][test]

\n", "\n", "> CSVLogger(**`learn`**:[`Learner`](/basic_train.html#Learner), **`filename`**:`str`=***`'history'`***, **`append`**:`bool`=***`False`***) :: [`LearnerCallback`](/basic_train.html#LearnerCallback)\n", "\n", "
×

Tests found for CSVLogger:

To run tests please refer to this guide.

\n", "\n", "A [`LearnerCallback`](/basic_train.html#LearnerCallback) that saves history of metrics while training `learn` into CSV `filename`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(CSVLogger)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First let's show an example of use, with a training on the usual MNIST dataset." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "path = untar_data(URLs.MNIST_TINY)\n", "data = ImageDataBunch.from_folder(path)\n", "learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy, error_rate], callback_fns=[CSVLogger])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 00:02

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_ratetime
00.6430730.5361240.9313300.06867000:00
10.5231300.2463200.9527900.04721000:00
20.4027640.1479980.9384840.06151600:00
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "learn.fit(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Training details have been saved in 'history.csv'." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that it only saves float/int metrics, so time currently is not saved. This could be saved but requires changing the recording - you can submit a PR [fixing that](https://forums.fast.ai/t/expand-recorder-to-deal-with-non-int-float-data/41534)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[PosixPath('/home/stas/.fastai/data/mnist_tiny/models'),\n", " PosixPath('/home/stas/.fastai/data/mnist_tiny/labels.csv'),\n", " PosixPath('/home/stas/.fastai/data/mnist_tiny/cleaned.csv'),\n", " PosixPath('/home/stas/.fastai/data/mnist_tiny/train'),\n", " PosixPath('/home/stas/.fastai/data/mnist_tiny/test'),\n", " PosixPath('/home/stas/.fastai/data/mnist_tiny/valid'),\n", " PosixPath('/home/stas/.fastai/data/mnist_tiny/export.pkl'),\n", " PosixPath('/home/stas/.fastai/data/mnist_tiny/history.csv')]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.path.ls()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that, as with all [`LearnerCallback`](/basic_train.html#LearnerCallback), you can access the object as an attribute of `learn` after it has been created. Here it's `learn.csv_logger`." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

read_logged_file[source][test]

\n", "\n", "> read_logged_file()\n", "\n", "
×

Tests found for read_logged_file:

Some other tests where read_logged_file is used:

  • pytest -sv tests/test_callbacks_csv_logger.py::test_logger [source]

To run tests please refer to this guide.

\n", "\n", "Read the content of saved file " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(CSVLogger.read_logged_file)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_rate
000.6430730.5361240.9313300.068670
110.5231300.2463200.9527900.047210
220.4027640.1479980.9384840.061516
\n", "
" ], "text/plain": [ " epoch train_loss valid_loss accuracy error_rate\n", "0 0 0.643073 0.536124 0.931330 0.068670\n", "1 1 0.523130 0.246320 0.952790 0.047210\n", "2 2 0.402764 0.147998 0.938484 0.061516" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.csv_logger.read_logged_file()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Optionally you can set `append=True` to log results of consequent stages of training." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# don't forget to remove the old file\n", "if learn.csv_logger.path.exists(): os.remove(learn.csv_logger.path)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy, error_rate],\n", " callback_fns=[partial(CSVLogger, append=True)])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 00:02

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_ratetime
00.6045490.4932410.7668100.23319000:00
10.4863670.2261890.9484980.05150200:00
20.3778470.1271420.9585120.04148800:00
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# stage-1\n", "learn.fit(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "Total time: 00:02

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_ratetime
00.1894410.1185320.9542200.04578000:00
10.1774320.1109130.9656650.03433500:00
20.1686260.0976460.9685260.03147400:00
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# stage-2\n", "learn.fit(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "

\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochtrain_lossvalid_lossaccuracyerror_rate
000.6045490.4932410.7668100.233190
110.4863670.2261890.9484980.051502
220.3778470.1271420.9585120.041488
3epochtrain_lossvalid_lossaccuracyerror_rate
400.1894410.1185320.9542200.045780
510.1774320.1109130.9656650.034335
620.1686260.0976460.9685260.031474
\n", "
" ], "text/plain": [ " epoch train_loss valid_loss accuracy error_rate\n", "0 0 0.604549 0.493241 0.766810 0.233190\n", "1 1 0.486367 0.226189 0.948498 0.051502\n", "2 2 0.377847 0.127142 0.958512 0.041488\n", "3 epoch train_loss valid_loss accuracy error_rate\n", "4 0 0.189441 0.118532 0.954220 0.045780\n", "5 1 0.177432 0.110913 0.965665 0.034335\n", "6 2 0.168626 0.097646 0.968526 0.031474" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "learn.csv_logger.read_logged_file()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Calback methods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You don't call these yourself - they're called by fastai's [`Callback`](/callback.html#Callback) system automatically to enable the class's functionality." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_train_begin[source][test]

\n", "\n", "> on_train_begin(**\\*\\*`kwargs`**:`Any`)\n", "\n", "
×

No tests found for on_train_begin. To contribute a test please refer to this guide and this discussion.

\n", "\n", "Prepare file with metric names. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(CSVLogger.on_train_begin)" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_epoch_end[source][test]

\n", "\n", "> on_epoch_end(**`epoch`**:`int`, **`smooth_loss`**:`Tensor`, **`last_metrics`**:`MetricsList`, **\\*\\*`kwargs`**:`Any`) → `bool`\n", "\n", "
×

No tests found for on_epoch_end. To contribute a test please refer to this guide and this discussion.

\n", "\n", "Add a line with `epoch` number, `smooth_loss` and `last_metrics`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(CSVLogger.on_epoch_end)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

on_train_end[source][test]

\n", "\n", "> on_train_end(**\\*\\*`kwargs`**:`Any`)\n", "\n", "
×

No tests found for on_train_end. To contribute a test please refer to this guide and this discussion.

\n", "\n", "Close the file. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(CSVLogger.on_train_end)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Undocumented Methods - Methods moved below this line will intentionally be hidden" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## New Methods - Please document or move to the undocumented section" ] } ], "metadata": { "jekyll": { "keywords": "fastai", "summary": "Callbacks that saves the tracked metrics during training", "title": "callbacks.csv_logger" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.2" } }, "nbformat": 4, "nbformat_minor": 2 }