{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## CSV Logger"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [],
"source": [
"from fastai.vision import *\n",
"from fastai.gen_doc.nbdoc import *\n",
"from fastai.callbacks import *"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"
\n",
"\n",
"> CSVLogger
(**`learn`**:[`Learner`](/basic_train.html#Learner), **`filename`**:`str`=***`'history'`***, **`append`**:`bool`=***`False`***) :: [`LearnerCallback`](/basic_train.html#LearnerCallback)\n",
"\n",
"×Tests found for CSVLogger
:
pytest -sv tests/test_callbacks_csv_logger.py::test_logger
[source]
To run tests please refer to this guide.
\n",
"\n",
"A [`LearnerCallback`](/basic_train.html#LearnerCallback) that saves history of metrics while training `learn` into CSV `filename`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(CSVLogger)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First let's show an example of use, with a training on the usual MNIST dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"path = untar_data(URLs.MNIST_TINY)\n",
"data = ImageDataBunch.from_folder(path)\n",
"learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy, error_rate], callback_fns=[CSVLogger])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"Total time: 00:02 \n",
" \n",
" \n",
" epoch | \n",
" train_loss | \n",
" valid_loss | \n",
" accuracy | \n",
" error_rate | \n",
" time | \n",
"
\n",
" \n",
" \n",
" \n",
" 0 | \n",
" 0.643073 | \n",
" 0.536124 | \n",
" 0.931330 | \n",
" 0.068670 | \n",
" 00:00 | \n",
"
\n",
" \n",
" 1 | \n",
" 0.523130 | \n",
" 0.246320 | \n",
" 0.952790 | \n",
" 0.047210 | \n",
" 00:00 | \n",
"
\n",
" \n",
" 2 | \n",
" 0.402764 | \n",
" 0.147998 | \n",
" 0.938484 | \n",
" 0.061516 | \n",
" 00:00 | \n",
"
\n",
" \n",
"
"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"learn.fit(3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Training details have been saved in 'history.csv'."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that it only saves float/int metrics, so time currently is not saved. This could be saved but requires changing the recording - you can submit a PR [fixing that](https://forums.fast.ai/t/expand-recorder-to-deal-with-non-int-float-data/41534)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[PosixPath('/home/stas/.fastai/data/mnist_tiny/models'),\n",
" PosixPath('/home/stas/.fastai/data/mnist_tiny/labels.csv'),\n",
" PosixPath('/home/stas/.fastai/data/mnist_tiny/cleaned.csv'),\n",
" PosixPath('/home/stas/.fastai/data/mnist_tiny/train'),\n",
" PosixPath('/home/stas/.fastai/data/mnist_tiny/test'),\n",
" PosixPath('/home/stas/.fastai/data/mnist_tiny/valid'),\n",
" PosixPath('/home/stas/.fastai/data/mnist_tiny/export.pkl'),\n",
" PosixPath('/home/stas/.fastai/data/mnist_tiny/history.csv')]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn.path.ls()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that, as with all [`LearnerCallback`](/basic_train.html#LearnerCallback), you can access the object as an attribute of `learn` after it has been created. Here it's `learn.csv_logger`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> read_logged_file
()\n",
"\n",
"×Tests found for read_logged_file
:
Some other tests where read_logged_file
is used:
pytest -sv tests/test_callbacks_csv_logger.py::test_logger
[source]
To run tests please refer to this guide.
\n",
"\n",
"Read the content of saved file "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(CSVLogger.read_logged_file)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" epoch | \n",
" train_loss | \n",
" valid_loss | \n",
" accuracy | \n",
" error_rate | \n",
"
\n",
" \n",
" \n",
" \n",
" 0 | \n",
" 0 | \n",
" 0.643073 | \n",
" 0.536124 | \n",
" 0.931330 | \n",
" 0.068670 | \n",
"
\n",
" \n",
" 1 | \n",
" 1 | \n",
" 0.523130 | \n",
" 0.246320 | \n",
" 0.952790 | \n",
" 0.047210 | \n",
"
\n",
" \n",
" 2 | \n",
" 2 | \n",
" 0.402764 | \n",
" 0.147998 | \n",
" 0.938484 | \n",
" 0.061516 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" epoch train_loss valid_loss accuracy error_rate\n",
"0 0 0.643073 0.536124 0.931330 0.068670\n",
"1 1 0.523130 0.246320 0.952790 0.047210\n",
"2 2 0.402764 0.147998 0.938484 0.061516"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn.csv_logger.read_logged_file()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Optionally you can set `append=True` to log results of consequent stages of training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# don't forget to remove the old file\n",
"if learn.csv_logger.path.exists(): os.remove(learn.csv_logger.path)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy, error_rate],\n",
" callback_fns=[partial(CSVLogger, append=True)])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"Total time: 00:02 \n",
" \n",
" \n",
" epoch | \n",
" train_loss | \n",
" valid_loss | \n",
" accuracy | \n",
" error_rate | \n",
" time | \n",
"
\n",
" \n",
" \n",
" \n",
" 0 | \n",
" 0.604549 | \n",
" 0.493241 | \n",
" 0.766810 | \n",
" 0.233190 | \n",
" 00:00 | \n",
"
\n",
" \n",
" 1 | \n",
" 0.486367 | \n",
" 0.226189 | \n",
" 0.948498 | \n",
" 0.051502 | \n",
" 00:00 | \n",
"
\n",
" \n",
" 2 | \n",
" 0.377847 | \n",
" 0.127142 | \n",
" 0.958512 | \n",
" 0.041488 | \n",
" 00:00 | \n",
"
\n",
" \n",
"
"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# stage-1\n",
"learn.fit(3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"Total time: 00:02 \n",
" \n",
" \n",
" epoch | \n",
" train_loss | \n",
" valid_loss | \n",
" accuracy | \n",
" error_rate | \n",
" time | \n",
"
\n",
" \n",
" \n",
" \n",
" 0 | \n",
" 0.189441 | \n",
" 0.118532 | \n",
" 0.954220 | \n",
" 0.045780 | \n",
" 00:00 | \n",
"
\n",
" \n",
" 1 | \n",
" 0.177432 | \n",
" 0.110913 | \n",
" 0.965665 | \n",
" 0.034335 | \n",
" 00:00 | \n",
"
\n",
" \n",
" 2 | \n",
" 0.168626 | \n",
" 0.097646 | \n",
" 0.968526 | \n",
" 0.031474 | \n",
" 00:00 | \n",
"
\n",
" \n",
"
"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# stage-2\n",
"learn.fit(3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" epoch | \n",
" train_loss | \n",
" valid_loss | \n",
" accuracy | \n",
" error_rate | \n",
"
\n",
" \n",
" \n",
" \n",
" 0 | \n",
" 0 | \n",
" 0.604549 | \n",
" 0.493241 | \n",
" 0.766810 | \n",
" 0.233190 | \n",
"
\n",
" \n",
" 1 | \n",
" 1 | \n",
" 0.486367 | \n",
" 0.226189 | \n",
" 0.948498 | \n",
" 0.051502 | \n",
"
\n",
" \n",
" 2 | \n",
" 2 | \n",
" 0.377847 | \n",
" 0.127142 | \n",
" 0.958512 | \n",
" 0.041488 | \n",
"
\n",
" \n",
" 3 | \n",
" epoch | \n",
" train_loss | \n",
" valid_loss | \n",
" accuracy | \n",
" error_rate | \n",
"
\n",
" \n",
" 4 | \n",
" 0 | \n",
" 0.189441 | \n",
" 0.118532 | \n",
" 0.954220 | \n",
" 0.045780 | \n",
"
\n",
" \n",
" 5 | \n",
" 1 | \n",
" 0.177432 | \n",
" 0.110913 | \n",
" 0.965665 | \n",
" 0.034335 | \n",
"
\n",
" \n",
" 6 | \n",
" 2 | \n",
" 0.168626 | \n",
" 0.097646 | \n",
" 0.968526 | \n",
" 0.031474 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" epoch train_loss valid_loss accuracy error_rate\n",
"0 0 0.604549 0.493241 0.766810 0.233190\n",
"1 1 0.486367 0.226189 0.948498 0.051502\n",
"2 2 0.377847 0.127142 0.958512 0.041488\n",
"3 epoch train_loss valid_loss accuracy error_rate\n",
"4 0 0.189441 0.118532 0.954220 0.045780\n",
"5 1 0.177432 0.110913 0.965665 0.034335\n",
"6 2 0.168626 0.097646 0.968526 0.031474"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"learn.csv_logger.read_logged_file()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Calback methods"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You don't call these yourself - they're called by fastai's [`Callback`](/callback.html#Callback) system automatically to enable the class's functionality."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> on_train_begin
(**\\*\\*`kwargs`**:`Any`)\n",
"\n",
"\n",
"\n",
"Prepare file with metric names. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(CSVLogger.on_train_begin)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> on_epoch_end
(**`epoch`**:`int`, **`smooth_loss`**:`Tensor`, **`last_metrics`**:`MetricsList`, **\\*\\*`kwargs`**:`Any`) → `bool`\n",
"\n",
"\n",
"\n",
"Add a line with `epoch` number, `smooth_loss` and `last_metrics`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(CSVLogger.on_epoch_end)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> on_train_end
(**\\*\\*`kwargs`**:`Any`)\n",
"\n",
"\n",
"\n",
"Close the file. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(CSVLogger.on_train_end)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Undocumented Methods - Methods moved below this line will intentionally be hidden"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## New Methods - Please document or move to the undocumented section"
]
}
],
"metadata": {
"jekyll": {
"keywords": "fastai",
"summary": "Callbacks that saves the tracked metrics during training",
"title": "callbacks.csv_logger"
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}