{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# default_exp\n", "from nbdev import *" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# A Minimal Example\n", "\n", "> A minimal end-to-end example of creating a nbdev project from scratch." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This section assumes you have gone through the [tutorial](https://nbdev.fast.ai/tutorial.html). The following is a minimal example of creating a nbdev project from scratch, with some commentary around why certain things work the way they do. \n", "\n", "For this example, we will use code from [Allen Downey's excellent book, Think Python 2](https://github.com/AllenDowney/ThinkPython2), particularly the [Card](https://github.com/AllenDowney/ThinkPython2/blob/master/code/Card.py) module. We will not cover all the features of `nbdev` as we focus on providing you with just enough information to become productive. We recommend reading the rest of [the docs](https://nbdev.fast.ai/) when you are done with this example." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Setup your nbdev GitHub Repo" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Per the instructions in [the tutorial](https://nbdev.fast.ai/tutorial.html), we will make a new repository using [the template](https://github.com/fastai/nbdev_template). In this case, we make a repository called [deck_of_cards](https://github.com/fastai/deck_of_cards):\n", "\n", "> Note: If you plan on writing installable Python modules, we highly recommend naming your repo after your python module." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![image.png](images/att_00010.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After you do this, you will have a repo with the necessary files to get started. You should also [setup nbdev](https://nbdev.fast.ai/tutorial.html#Set-up-Your-Jupyter-Server) and [install the githooks](https://nbdev.fast.ai/tutorial.html#Install-git-hooks) per the instructions in [the tutorial](https://nbdev.fast.ai/tutorial.html).\n", "\n", "\n", "## Step 2: Modify Configuration Files\n", "\n", "### Edit settings.ini\n", "\n", "[Editing settings.ini](https://nbdev.fast.ai/tutorial.html#Edit-settings.ini) is required for nbdev to work properly. These settings are used to populate required information for you to host documentation on [GitHub Pages](https://nbdev.fast.ai/tutorial.html#Github-pages), as well to [publish your modules as packages to pypi](https://nbdev.fast.ai/tutorial.html#Upload-to-pypi).\n", "\n", "These are the fields that we changed in [settings.ini](https://github.com/fastai/deck_of_cards/blob/master/settings.ini):\n", "\n", "```ini\n", "lib_name = deck_of_cards\n", "description = \"A minimal example of nbdev using code from Allen Downey's Think Python 2nd Ed\"\n", "keywords = nbdev\n", "author = Hamel Husain\n", "author_email = hamel@example.com\n", "copyright = Hamel, inc.\n", "user = fastai\n", "```\n", "\n", "> Note: the value for user wasn't actually changed in this example, but you will have to change it to be the org corresponding to your GitHub repository (usually your username) on GitHub. \n", "\n", "The values in settings.ini file are propagated to various systems for you automatically, which help minimize boilerplate and learning complicated configuration files. For example, \n", "\n", "- The `author` and `author_email` fields are read by [setup.py](https://github.com/fastai/deck_of_cards/blob/master/setup.py) for python packaging.\n", "- The `lib_name` is used by both by [setup.py](https://github.com/fastai/deck_of_cards/blob/master/setup.py) and [Jekyll](https://jekyllrb.com/)'s configuration file, [_config.yaml](https://github.com/fastai/deck_of_cards/blob/master/docs/_config.yml) to make sure the rendered docs are configured correctly on GitHub pages." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 3: Write Code (Or Copy/Paste Into A Notebook)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we take the [Card module from the ThinkPython2 repo](https://github.com/AllenDowney/ThinkPython2/blob/master/code/Card.py#L17) and write it in nbdev. \n", "\n", "The first step is to copy and paste the `Card` class from [Card.py](https://github.com/AllenDowney/ThinkPython2/blob/master/code/Card.py) into a new Jupyter notebook, which we have named [00_card.ipynb](https://github.com/fastai/deck_of_cards/blob/master/00_card.ipynb). Copying and pasting code from python files is a reasonable way to convert existing python scripts into Jupyter notebooks. A useful trick for copying big blocks of code into notebooks is to copy the whole file into a single cell and use `ctrl-shift-minus` to split the code into separate cells. \n", "\n", "If you are trying to convert an existing python project to `nbdev`, we recommend incrementally converting specific files to `nbdev` over time. Specifically, we recommend choosing one python file to begin with like the example with `card.py` shown below.\n", "\n", "> Note: The number at the beginning of the filename is not required; it is a convention we use to keep notebooks in a desired order when they are sorted by the the file system.\n", "\n", "In the first cell of [the notebook](https://github.com/fastai/deck_of_cards/blob/master/00_card.ipynb), write a comment that looks like this (this is not required but we do it here to highlight an important feature of nbdev):\n", "\n", "```py\n", "# default_exp card\n", "```\n", "\n", "In this case, the argument `card` specifies that code exported from this notebook will be placed in the destination `card.py` by default. You can read more about how python modules are created from notebooks [here](https://nbdev.fast.ai/export.html). A reasonable way to arrange the notebook will be like this:\n", "\n", "![image.png](images/att_00011.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Flags\n", "\n", "nbdev uses special comments, or flags, as a markup language that allows you to control various aspects of the docs and how code is exported to modules, and how code is tested. In addition to `default_exp`, the the following other flags are present in this notebook:\n", "\n", "\n", "#hide\n", "\n", "> Note: This comment instructs nbdev to hide this cell when generating the docs.\n", "\n", "\n", "#export\n", "\n", "> Note: This comment instructs nbdev to export this cell to the appropriate python file. When no argument is provided to `export`, this defaults to the module specified by `default_exp` as described above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Where The Magic Happens: Write Documentation And Tests\n", "\n", "In the original code base, tests for Card are separate, located in [Card_test.py](https://github.com/AllenDowney/ThinkPython2/blob/master/code/Card_test.py). Furthermore, the documentation for `Card` is primarily located in the [book folder](https://github.com/AllenDowney/ThinkPython2/tree/master/book) of Allen's repo as well as some documentation in the docstring. While this is a typical arrangement for a Python project, we believe `nbdev` can make your workflow easier by organizing the docs, tests and source code into a single context. We believe this allows developers to write higher quality documentation and code, and encourages more testing. \n", "\n", "Here is what the documentation + code looks like for Card:\n", "\n", "![image.png](images/att_00012.png)\n", "\n", "These comments and tests are rendered by the documentation system which is discussed in a later section. Furthermore, the assert statements automatically become tests that are run by the [continuous integration system setup by default in nbdev](https://nbdev.fast.ai/tutorial.html#CI) in your GitHub repository. \n", "\n", "> Note: The nbdev programming environment comes set up with a [continuous integration (CI)](https://nbdev.fast.ai/tutorial.html#CI) system for you. You don't have to do anything extra to enable it, and it starts working immediately. This is especially great for people who have no experience with CI; it is a gentle way to start using it.\n", "\n", "> Important: There are testing utilities provided by [fastcore's testing utilities](https://fastcore.fast.ai/test.html), which provide wrappers around common types of assert statements and also provide better default error messages. Using these is optional, but recommended." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Edit index.ipynb" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`nbdev` repositories require a notebook named `index.ipynb`, which is included in your repository when you use the template. `index.ipynb` serves two purposes:\n", "\n", "1. It becomes the README for your repo (this notebook is converted to `README.md`)\n", "2. It becomes home page (`index.html`) for your documentation. \n", "\n", "You will notice the following boilerplate in `index.ipynb`:\n", "\n", "```py\n", "from your_lib.core import *\n", "```\n", "\n", "You should remove this line of code or comment it out, as it will cause a syntax error. Later, when you are finished creating your module, you can replace this with the appropriate import statement. We left this line in here on purpose so that you can experience how the continuous integration system (discussed above) warns you of errors. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 4: Convert Notebooks To Python Modules & Docs\n", "\n", "**Run the command `nbdev_build_lib`** from the root of the repo. This exports notebook cells tagged with `#export` to the appropriate python module. For example, the notebook [00_cards.ipynb](https://github.com/fastai/deck_of_cards/blob/master/00_card.ipynb) gets converted to [card.py](https://github.com/fastai/deck_of_cards/blob/master/deck_of_cards/card.py).\n", "\n", "**Run the command `nbdev_test_nbs`** to run the code and tests in all the notebooks. This command also gets run by the [continuous integration system setup for you by nbdev](https://nbdev.fast.ai/tutorial.html#CI), but it is useful to run these tests locally to get immediate feedback.\n", "\n", "> Note: there is a way to optionally skip certain tests that are long running or slow by using special tags [described here](https://nbdev.fast.ai/test.html)\n", "\n", "\n", "### Preview The Docs\n", "\n", "To preview the docs, run the command `make docs_serve` from the root of your repo. This command runs the CLI command `nbdev_build_docs` behind the scenes for you, which generates a documentation site from the notebooks. After running this command, you will see a URL in the terminal indicating where the docs have been hosted locally. For the [fastai/deck_of_cards](https://github.com/fastai/deck_of_cards/) repository we are using for this example, the url is `http://127.0.0.1:4000/deck_of_cards/`\n", "\n", "If you navigate to the cards page at `http://127.0.0.1:4000/deck_of_cards/card.html`, you will see the docs that we just wrote, which we have annotated for further explanation:\n", "\n", "![image.png](images/att_00016.png)\n", "\n", "#### Explanation of annotations:\n", "\n", "1. The heading **Card** corresponds to the first `H1` heading in a notebook with a note block _API Details_ as the summary.\n", "\n", "2. `nbdev` automatically renders a Table of Contents for you.\n", "\n", "3. `nbdev` automatically renders the signature of your class or function as a heading. \n", "\n", "4. `nbdev` automatically adds a link to the corresponding source code (which is a plain-text python file) on GitHub. Remember, `nbdev` converts Jupyter notebooks to source code with the command `nbdev_build_lib`. \n", "\n", "5. This part of docs is rendered automatically from the docstring.\n", "\n", "6. The rest of the notebook is rendered by converting your markdown into HTML, showing the inputs and outputs of each of your cells (including charts and images), and so forth. You can hide entire cells, hide only cell input or hide only output by using the [flags described on this page](https://nbdev.fast.ai/export2html.html).\n", "\n", "7. nbdev supports special block quotes that render as colored boxes in the documentation. You can read more about them [here](https://nbdev.fast.ai/export2html.html#add_jekyll_notes). In this specific example, we are using the `Note` block quote. \n", "\n", "8. Words you surround in backticks will be automatically hyperlinked to the associated documentation where appropriate. This is a trivial case where `Card` class is defined immediately above, however this works across pages and modules. We will see another example of this in later steps.\n", "\n", "### show_doc\n", "\n", "`show_doc` allows you to control how documentation is shown on the docs. You can control the location, order, heading and other details of how documentation is rendered. You can read more [about it here](https://nbdev.fast.ai/showdoc.html#show_doc). For example, this is how you can use `show_doc` to render the docs for the `__eq__` method of `Card` (notice how the tests are naturally included below the documentation):\n", "\n", "![image.png](images/att_00017.png)\n", "\n", "#### Important Notes about `show_doc`:\n", "\n", "- **For functions and classes, `show_doc` is called by default automatically** in the same location where the function or class is defined. This is why you see a heading for the class `Cards` in the docs example above even though `show_doc` was never explicitly called. \n", " - You can override this default by explicitly calling `show_doc` in your desired location.\n", "- **For methods, you must call `show_doc`** for a documentation heading to appear, as illustrated in the `Card.__eq__` method above. This is by design, since unlike functions, you typically define all of the methods for a class in a single contiguous block of code. Therefore, `show_doc` allows you to control both the order and placement of documentation headers for methods which help you write prose and tests for each method organized under the appropriate header.\n", "- We recommend experimenting with `show_doc` by editing a notebook and re-rendering the docs (described below) to see the what happens in different scenarios.\n", "\n", "\n", "### Refresh the docs\n", "\n", "If you want to edit the docs, you can make a change to the corresponding notebook(s) and run `nbdev_build_docs` followed by a [hard refresh](https://www.getfilecloud.com/blog/2015/03/tech-tip-how-to-do-hard-refresh-in-browsers/#.X52j8lNKjng) in your browser to re-render the docs." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 5: Push Files To GitHub And View Hosted Docs\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This step assumes you have already [enabled GitHub pages](https://nbdev.fast.ai/tutorial.html#Github-pages).\n", "\n", "At this point we are ready to push your first files to GitHub. If you have [installed the git hooks per the tutorial instructions](https://nbdev.fast.ai/#Avoiding-and-handling-git-conflicts), `nbdev` will automatically clean unnecessary metadata out of notebooks to avoid conflicts and overly verbose diffs. Before pushing your files to GitHub for the first time, we recommend running the command `git status` so you can see all the files generated by `nbdev` for you. You will notice that the following files were created:\n", "\n", "- `.py` files corresponding to the notebooks you created, in a folder corresponding to the library name, which in this case is called [deck_of_cards](https://github.com/fastai/deck_of_cards/tree/master/deck_of_cards). For example, an `__init__.py` file is automatically created in the proper directory in order to organize a python module.\n", "- files for your docs site in a `docs/` folder. This directory contains HTML, CSS, and other files that are used for hosting your docs site on GitHub Pages. \n", "\n", "Make sure you add all these files to your commit with `git add` before you push to GitHub, because they are all needed for everything to work correctly. \n", "\n", "Pushing your files to GitHub will automatically trigger [continuous integration (CI) using GitHub Actions](https://nbdev.fast.ai/#Using-nbdev-as-part-of-your-CI). The CI will automatically perform a number of checks [outlined here](https://nbdev.fast.ai/tutorial.html#CI). You can see the CI process running in GitHub Actions by navigating to the Actions tab in your GitHub repo. \n", "\n", "\n", "After pushing your files, GitHub will rebuild your docs automatically. You can view the status of your doc build by going to your repository settings and finding the GitHub Pages section under options. When GitHub is in the process of building your pages, it will look like this:\n", "\n", "![image.png](images/att_00018.png)\n", "\n", "\n", "When the page is finished being built, the color and status message will change to look like this:\n", "\n", "![image.png](images/att_00019.png)\n", "\n", "Furthermore, assuming that you have already [enabled GitHub Pages](https://nbdev.fast.ai/tutorial.html#Github-pages), you can see the status of your Github Pages deployments at anytime. If you add `/deployments` to your repository's GitHub URL you will see a deployments dashboard. For example, below is a screen shot of https://github.com/fastai/deck_of_cards/deployments right after pushing new files:\n", "\n", "![image.png](images/att_00021.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 6: Add More Code\n", "\n", "Congratulations, you authored your first piece of code with nbdev! However, to fully grasp how nbdev works, it is worthwhile to add additional code in a new notebook that imports the code you wrote earlier. Next, we will add the [Deck class form cards.py](https://github.com/AllenDowney/ThinkPython2/blob/master/code/Card.py#L55) into a new notebook called [01_deck.ipynb](https://github.com/fastai/deck_of_cards/blob/master/01_deck.ipynb). This notebook imports the previously created `Card` class and creates a `Deck`, which is a collection of Cards:\n", "\n", "![image.png](images/att_00022.png)\n", "\n", "Similar to the previous notebook, the first cell has the nbdev flag `# default_exp deck` which means that code blocks marked with `#export` will be exported to the file `deck.py` by default. You can see that we import the `Card` object and export that code to `deck.py` with the following code cell:\n", "\n", "```py\n", "#export\n", "from deck_of_cards.card import Card\n", "```\n", "\n", "This works because the cli command, `nbdev_build_lib` converted `00_card.ipynb` to `card.py`, which we have imported here.\n", "\n", "### Tests\n", "\n", "Downey's code contains a test for the `Deck` class in a separate file called [Card_test.py]( https://github.com/AllenDowney/ThinkPython2/blob/master/code/Card_test.py). This file is a good example that highlights the strengths of `nbdev`. The contents of this file is as follows:\n", "\n", "```py\n", "\"\"\"This file contains code for use with \"Think Stats\",\n", "by Allen B. Downey, available from greenteapress.com\n", "Copyright 2014 Allen B. Downey\n", "License: GNU GPLv3 http://www.gnu.org/licenses/gpl.html\n", "\"\"\"\n", "\n", "from __future__ import print_function, division\n", "\n", "import unittest\n", "from Card import Card, Deck\n", "\n", "\n", "class Test(unittest.TestCase):\n", "\n", " def testDeckRemove(self):\n", " deck = Deck()\n", " card23 = Card(2, 3)\n", " deck.remove_card(card23)\n", "\n", "\n", "if __name__ == \"__main__\":\n", " unittest.main()\n", "```\n", "\n", "The code shown above is problematic for the following reasons:\n", "\n", "- It is not clear what the purpose of the test is. \n", "- The test is located in a file that is separate from the implementation, so you have to open multiple windows and/or switch contexts to understand the test. \n", "- The tests uses an api, `unittest` that you must learn and think about if you want to write tests.\n", "- The tests are separate from the documentation and any prose associated with explaining what the class `Deck` does. \n", "\n", "All of these problems are handled in nbdev, as you can write code, docs and tests all in the same context. Below is a screen shot of the relevant parts of [01_deck.ipynb](https://github.com/fastai/deck_of_cards/blob/master/01_deck.ipynb) that expresses the code and this test in a more readable, expressive way:\n", "\n", "\n", "![image.png](images/att_00023.png)\n", "\n", "The above code expresses the same unit test, but also integrates documentation alongside the original implementation of Deck. You can view the notebook on GitHub [here](https://github.com/fastai/deck_of_cards/blob/master/01_deck.ipynb). One additional tool shown in this notebook is the `nbdev` function `show_doc`, which allows you to control the placement of documentation. In this example, `showdoc(Deck.remove_card)` will create a section in the documentation with an appropriate heading.\n", "\n", "If you run the CLI command `make docs_serve`, you can preview what these docs will look like locally. Below is an annotated screenshot of what this looks like:\n", "\n", "![image.png](images/att_00024.png)\n", "\n", "#### Explanation of annotations:\n", "\n", "1. When writing these docs, we simply enclosed `Card` with backticks. `nbdev` automatically transforms this into a hyperlink to the appropriate page that documents `Card`. \n", "\n", "2. This heading for the method `Deck.remove_card` was created by `show_doc`.\n", "\n", "3. `nbdev` is designed to encourage you to write your tests as part of your documentation as shown here.\n", "\n", "You can see this page live at [https://fastai.github.io/deck_of_cards/deck.html](https://fastai.github.io/deck_of_cards/deck.html).\n", "\n", "When you are done make sure you run the following cli commands before pushing to GitHub.\n", "\n", "- `nbdev_build_lib`: this converts your notebooks into modules.\n", "- `nbdev_build_docs`: this generates the your documentation site.\n", "- `nbdev_test_nbs`: this runs all your tests (which is a good idea so you can catch errors).\n", "- `git status` to see which files have changed, which is a good exercise when first getting started with `nbdev` to understand which files are automatically generated." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 7: Publish Python Module to Pypi" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can publish your module to pypi by following [these instructions](https://nbdev.fast.ai/tutorial.html#Upload-to-pypi)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## References\n", "\n", "All the code for this example is available on the GitHub repo [fastai/deck_of_cards](https://github.com/fastai/deck_of_cards).\n", "\n", "### Live Demo\n", "\n", "The below video shows a live demonstration of this example, with a Q&A section at the end. \n", "\n", "" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 4 }