"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"dirname = os.path.dirname(path)\n",
"for group in [\"mnist_tiny/train/3/*.png\", \n",
" \"mnist_tiny/train/7/*.png\"]:\n",
" for filename in glob.glob(os.path.join(dirname, group)):\n",
" display(Image(filename))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So, all of the files under `mnist_tiny/train/3/` are pictures of 3's, and `mnist_tiny/train/7/` are pictures of 7's.\n",
"\n",
"Now we get the image data from the folder. To train a dataset in fastai, we must create a `DataBunch`. In this case, we can use `ImageDataBunch` in the `fastai.vision` library. We pick 10 as the batch size (`bs`) because there are only 10 images in each category."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"data = fastai.vision.ImageDataBunch.from_folder(path, bs=10) # bs: batch size"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That's better for this simple test. Now we can create a model, and train the network:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Model\n",
"\n",
"In this example, we will use the pre-designed WideresNet from fastai. The model is also known as wrn_22."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"model = fastai.vision.models.WideResNet(num_groups=3,\n",
" N=3,\n",
" num_classes=10,\n",
" k=6,\n",
" drop_p=0.)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That's it! Often, you would probably build your own model, or adjust a default model. To see more on model building in fastai, see:\n",
"\n",
"* http://files.fast.ai/models/\n",
"* http://course.fast.ai/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Experiment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In order for comet.ml to log your experiment and results, you need to create an Experiment instance. To do this, you'll need two items:\n",
"\n",
"* a Comet `api_key`\n",
"* a `project_name`\n",
"\n",
"You can find your Comet api_key when you log in to https://www.comet.ml and click on your project. You should see a screen that looks similar to:\n",
"\n",
"\n",
"\n",
"Click on the API key to copy the key to your clipboard. \n",
"\n",
"It is recommended that you put your COMET_API_KEY in a `.env` key in the current directory. You can do that using the following code. Put it in a cell, replace the `...` with your key, and then delete the cell. That way your key stays private.\n",
"\n",
"```ipython\n",
"%%writefile .env\n",
"\n",
"COMET_API_KEY=...\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is also recommended that you use your project_name in the cell, so you can match the results with this code. You can make up a new name, or add this experiment to a project that already exists."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"COMET INFO: Experiment is live on comet.ml https://www.comet.ml/cometpublic/comet-notebooks/d21f94a1c71841d2961da1e6ddb5ab20\n",
"\n"
]
}
],
"source": [
"experiment = Experiment(project_name=\"comet_notebooks\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you get the error that ends with:\n",
"\n",
"\n",
"ValueError: Comet.ml requires an API key. Please provide as the first argument to Experiment(api_key) or as an environment variable named COMET_API_KEY \n",
"\n",
"\n",
"then that means that either you don't have an `.env` file in this directory, or the key is invalid.\n",
"\n",
"Otherwise, you should see the message:\n",
"\n",
"\n",
"COMET INFO: Experiment is live on comet.ml https://www.comet.ml/...\n",
"\n",
"\n",
"If you click the URL, then a new page will open up. But, even better, you can execute the following line to see the experiment in the current notebook:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"experiment.display()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By the way, the line `experiment.display()` works when you are at the console too. It will open up a window in your browser.\n",
"\n",
"Now, we are ready for training!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5. Training\n",
"\n",
"In fastai, we can train differently depending on if we are running CPU or a GPU. To test, we can use the `data.device.type` property. This will create a fastai `Learner`:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"if data.device.type == 'cpu':\n",
" learn = fastai.basic_train.Learner(data, model, metrics=fastai.metrics.accuracy)\n",
"else: # GPU:\n",
" learn = fastai.basic_train.Learner(data, model, metrics=fastai.metrics.accuracy).to_fp16()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we are ready to train the model. To tell Comet about the details, we put the call to `fit` or `fit_one_cylce` inside an indented block under `experiment.train()`:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total time: 00:10\n",
"epoch train_loss valid_loss accuracy\n",
"1 2.331475 2.306675 0.000000 (00:05)\n",
"2 1.933559 2.272467 0.000000 (00:05)\n",
"\n"
]
}
],
"source": [
"with experiment.train():\n",
" learn.fit_one_cycle(2, 3e-3, wd=0.4, div_factor=10, pct_start=0.5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6. Logging"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In fastai, Comet will automatically log:\n",
"\n",
"* the model description\n",
"* the training loss\n",
"* the training validation\n",
"* the source code\n",
"\n",
"To log other items manually, you can use any of the following:\n",
"\n",
"* `experiment.log_html(HTML_STRING)`\n",
"* `experiment.html_log_url(URL_STRING)`\n",
"* `experiment.image(FILENAME)`\n",
"* `experiment.log_dataset_hash(DATASET)`\n",
"* `experiment.log_other(KEY, VALUE)`\n",
"* `experiment.log_metric(NAME, VALUE)`\n",
"* `experiment.log_parameter(PARAMETER, VALUE)`\n",
"* `experiment.log_figure(NAME, FIGURE)`\n",
"\n",
"For complete details, please see: \n",
"\n",
"https://www.comet.ml/docs/python-sdk/Experiment/#experiment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7. Finish"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finall, we are ready to tell Comet that our experiment is complete. You don't need to do this is a script that ends. But in Jupyter, we need to indicate that the experiment is finished. We do that with the `experiment.end()` method:"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"COMET INFO: Uploading stats to Comet before program termination (may take several seconds)\n",
"COMET INFO: Experiment is live on comet.ml https://www.comet.ml/cometpublic/comet-notebooks/d21f94a1c71841d2961da1e6ddb5ab20\n",
"\n"
]
}
],
"source": [
"experiment.end()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That's it! If you have any questions, please visit us on https://cometml.slack.com"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
![](\"https://www.comet.ml/images/logo_comet_light.png\"){kind=logo}
\n",
"\n",
"# MNIST Example in fastai\n",
"\n",
"This Jupyter notebook demonstrates using the **fastai** (version 1.0.38) deep learning framework with [comet.ml](https://www.comet.ml).\n",
"\n",
"In this example, we load a fastai model, called WideResNet, and train it on a small part of the MNIST_TINY dataset.\n",
"\n",
"fastai is a framework built on top of the torch Python library. \n",
"\n",
"To find out more, you might find these links helpful:\n",
"\n",
"* http://www.fast.ai/\n",
"* http://docs.fast.ai/\n",
"* http://www.fast.ai/2018/08/10/fastai-diu-imagenet/\n",
"* https://en.wikipedia.org/wiki/MNIST_database\n",
"* http://jupyter.org/\n",
"\n",
"Let's get started!\n",
"\n",
"# 0. Installation\n",
"\n",
"This example uses fastai version 1.0.38. You can install a specific version of fastai (which should also install the correct version of torch) with this command at the terminal:\n",
"\n",
"```\n",
"python -m pip install fastai==1.0.38\n",
"```\n",
"\n",
"Once you have fastai and torch installed, we are ready to import them."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Imports\n",
"\n",
"First, we import the comet_ml library, followed by the fastai library, and others if needed. The only requirement here is that **comet_ml be imported first**. If you forget, just restart the kernel, and import them in the proper order."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"## Import this first:\n",
"from comet_ml import Experiment\n",
"\n",
"## Import the deep learning framework:\n",
"import fastai\n",
"import fastai.vision\n",
"\n",
"## Additional libraries for this example:\n",
"import glob\n",
"import os"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As a simple demo, we'll start with the the MNIST_TINY dataset, In fastai, we use the `datasets.untar_data` function to download and uncompress the data:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"data path: /home/dblank/.fastai/data/mnist_tiny\n"
]
}
],
"source": [
"path = fastai.datasets.untar_data(fastai.datasets.URLs.MNIST_TINY)\n",
"print(\"data path:\", path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The path returned by the untar_data function shows where the data was saved. Using the shell `!` magic, we can explore the dataset in more detail:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"715.png 7463.png 8360.png 9141.png\t9303.png\r\n",
"7288.png 7626.png 8957.png 9192.png\t9637.png\r\n"
]
}
],
"source": [
"! ls ~/.fastai/data/mnist_tiny/train/3/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By poking around here, we can see that there are two categories `3` and `7` and each category has about 2,000 examples."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That is still too many for a CPU, so we trim it down to 10 in each category:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"dirname = os.path.dirname(path)\n",
"for group in [\"mnist_tiny/train/3/*.png\",\n",
" \"mnist_tiny/train/7/*.png\",\n",
" \"mnist_tiny/valid/3/*.png\",\n",
" \"mnist_tiny/valid/7/*.png\"]:\n",
" for filename in glob.glob(os.path.join(dirname, group))[10:]:\n",
" os.remove(filename)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we check again to see if we have just 10 images:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"715.png 7463.png 8360.png 9141.png\t9303.png\r\n",
"7288.png 7626.png 8957.png 9192.png\t9637.png\r\n"
]
}
],
"source": [
"! ls ~/.fastai/data/mnist_tiny/train/3/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To see these images, we can use some tools from the IPython library:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"from IPython.display import Image, display"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's see what the training set looks like for the 3 and 7 categories:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAA2klEQVR4nGNgGLRAeN7n///+/9utgU3y6L9//x7ve/fvjQimXNff9/IcbEzS7/45YMhpfHtnAzb8BRbJ+CNiYDrjHxZJNk4wpfni32I2HC7m3PzvjjRWGT3Xrmv//jlhk+KZ8OsfCOxVwCLZ9u/r0lrTgCf/yrEZGqgEoiL/vebC4SAGBoHH/3xwSjLM/7cPt6Tuvy/yOCWl/v1zxSlZ9++jELoYC9SNPHf+daPLMe85zMjAwMRptObfZ010SZe//3Zs3LgPGEJ/QjGt8v0ACrvLj5Y74nQN1QAAP0VaWbYzqAQAAAAASUVORK5CYII=\n",
"text/plain": [
"