{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# How to contribute to jupyter notebooks" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "hide_input": true }, "outputs": [], "source": [ "from fastai.gen_doc.nbdoc import *\n", "from fastai.gen_doc.gen_notebooks import *\n", "from fastai.gen_doc import * " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The documentation is built from notebooks in `docs_src/`. Follow the steps below to build documentation. For more information about generating and authoring notebooks, see [`fastai.gen_doc.gen_notebooks`](/gen_doc.gen_notebooks.html#gen_doc.gen_notebooks)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Modules" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`fastai.gen_doc.gen_notebooks`](/gen_doc.gen_notebooks.html#gen_doc.gen_notebooks)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Generate and update notebook skeletons automatically from modules. Includes an overview of the whole authoring process." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`fastai.gen_doc.convert2html`](/gen_doc.convert2html.html#gen_doc.convert2html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create HTML (jekyll) docs from notebooks." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### [`fastai.gen_doc.nbdoc`](/gen_doc.nbdoc.html#gen_doc.nbdoc)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Underlying documentation functions; most important is [`show_doc`](/gen_doc.nbdoc.html#show_doc)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Process for contributing to the docs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you want to help us and contribute to the docs, you just have to make modifications to the source notebooks, our scripts will then automatically convert them to HTML. There is just one script to run after cloning the fastai repo, to ensure that everything works properly. The rest of this page goes more in depth about all the functionalities this module offers, but if you just want to add a sentence or correct a typo, make a PR with the notebook changed and we'll take care of the rest." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Thing to run after git clone" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Make sure you follow this recipe:\n", "\n", " git clone https://github.com/fastai/fastai\n", " cd fastai\n", " tools/run-after-git-clone\n", "\n", "This will take care of everything that is explained in the following two sections. We'll tell you what they do, but you need to execute just this one script.\n", "\n", "Note: windows users, not using bash emulation, will need to invoke the command as:\n", "\n", " python tools\\run-after-git-clone\n", " \n", "On windows as well, the simlink between the imgs folder in docs_src (where the notebooks are) and docs won't work. You will need to remove the borken link in docs_src (called imgs), execute cmd.exe as administrator, then, in the docs_src folder, execute\n", "\n", " mklink /d imgs ..\\docs\\imgs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### after-git-clone #1: a mandatory notebook strip out" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Currently we only store `source` code cells under git (and a few extra fields for documentation notebooks). If you would like to commit or submit a PR, you need to confirm to that standard.\n", "\n", "This is done automatically during `diff`/`commit` git operations, but you need to configure your local repository once to activate that instrumentation.\n", "\n", "Therefore, your developing process will always start with:\n", "\n", " tools/trust-origin-git-config\n", "\n", "The last command tells git to invoke configuration stored in `fastai/.gitconfig`, so your `git diff` and `git commit` invocations for this particular repository will now go via `tools/fastai-nbstripout` which will do all the work for you.\n", "\n", "You don't need to run it if you run:\n", "\n", " tools/run-after-git-clone\n", "\n", "If you skip this configuration your commit/PR involving notebooks will not be accepted, since it'll carry in it many JSON bits which we don't want in the git repository. Those unwanted bits create collisions and lead to unnecessarily complicated and time wasting merge activities. So please do not skip this step.\n", "\n", "Note: we can't make this happen automatically, since git will ignore a repository-stored `.gitconfig` for security reasons, unless a user will tell git to use it (and thus trust it).\n", "\n", "If you'd like to check whether you already trusted git with using `fastai/.gitconfig` please look inside `fastai/.git/config`, which should have this entry:\n", "\n", " [include]\n", " path = ../.gitconfig\n", "\n", "or alternatively run:\n", "\n", " tools/trust-origin-git-config -t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### after-git-clone #2: automatically updating doc notebooks to be trusted on git pull" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We want the doc notebooks to be already trusted when you load them in `jupyter notebook`, so this script which should be run once upon `git clone`, will install a `git` `post-merge` hook into your local check out.\n", "\n", "The installed hook will be executed by git automatically at the end of `git pull` only if it triggered an actual merge event and that the latter was successful.\n", "\n", "To trust run:\n", "\n", " tools/trust-doc-nbs-install-hook\n", "\n", "You don't need to run it if you run:\n", "\n", " tools/run-after-git-clone\n", "\n", "To distrust run:\n", "\n", " rm .git/hooks/post-merge" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Validate any notebooks you're contributing to" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you were using a text editor to make changes, when you are done working on a notebook improvement, please, make sure to validate that notebook's format, by simply loading it in the jupyter notebook.\n", "\n", "Alternatively, you could use a CLI JSON validation tool, e.g. [jsonlint](https://jsonlint.com/):\n", "\n", " jsonlint-php example.ipynb\n", "\n", "but it's second best, since you may have a valid JSON, but invalid notebook format, as the latter has extra requirements on which fields are valid and which are not." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Update the documentation notebooks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To update the documentation notebooks to reflect changes in the library, you should use [`update_notebooks`](/gen_doc.gen_notebooks.html#update_notebooks) or the `tools/update-nbs` script. Only use this when you have added a new function that you want to document." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Updating docs from within notebook" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[`update_notebooks`](/gen_doc.gen_notebooks.html#update_notebooks) can be run from the notebook. \n", "\n", "\n", "Default - updates all notebooks in the fastai/docs_src folder:\n", "```python\n", "update_notebooks('.')\n", "```\n", "\n", "To update specific python file only:\n", "```python\n", "update_notebooks('gen_doc.gen_notebooks.ipynb', update_nb=True)\n", "```\n", "\n", "`update_nb=True` inserts newly added module methods into the docs that haven't already been documented\n", "\n", "Alternatively, you can update a specific module:\n", "```python\n", "update_notebooks('fastai.gen_doc.gen_notebooks', dest_path='fastai/docs_src')\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Updating notebooks from script" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "[`update_notebooks`](/gen_doc.gen_notebooks.html#update_notebooks) can be run as a script for convenience. \n", "\n", "To update all notebooks under `docs_src`:\n", "```bash\n", "python tools/update-nbs \n", "```\n", "\n", "To automatically append new fastai methods to their corresponding documentation notebook:\n", "```bash\n", "python tools/update-nbs --update_nb_links\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Update the documentation website" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once notebooks have been updated, update the website html files (under `docs`)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Updating notebook metadata" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to pass the right settings to the website version of the `docs`, each notebook has a custom entry which if you look at the source code, looks like:\n", "```\n", " \"metadata\": {\n", " \"jekyll\": {\n", " \"keywords\": \"fastai\",\n", " \"toc\": \"false\",\n", " \"title\": \"Welcome to fastai\"\n", " },\n", " [...]\n", "```\n", "Do not edit this entry manually, or your changes will be overwritten in the next metadata update.\n", "\n", "The only correct way to change any notebook's metadata is by opening `docs_src/jekyll_metadata.ipynb`, finding the notebook you want to change the metadata for, changing it, and running the notebook, then saving and committing it and the resulting changes.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Updating sidebar" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "1. edit `docs_src/sidebar/sidebar_data.py`\n", "2. `python tools/make_sidebar.py`\n", "3. check `docs/_data/sidebars/home_sidebar.yml`\n", "4. `git commit docs_src/sidebar/sidebar_data.py docs/_data/sidebars/home_sidebar.yml`\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Generating html" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To convert `docs_src/*ipynb` to `docs/*html`:\n", "\n", "* only the modified `*ipynb`:\n", "\n", "```bash\n", "python tools/build-docs\n", "```\n", "\n", "* force to rebuild all `*ipynb`s:\n", "\n", "```bash\n", "python tools/build-docs -f\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Links and anchors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Validate links and anchors\n", "\n", "After you commit doc changes please validate that all the links and `#anchors` are correct.\n", "\n", "If it's the first time you are about to run the link checker, install the [prerequisites](https://github.com/fastai/fastai/blob/master/checklink/README.md) first.\n", "\n", "After committing the new changes, first, wait a few minutes for github pages to sync, otherwise you'll be testing an outdated live site.\n", "\n", "Then, do:\n", "\n", "```\n", "cd tools/checklink\n", "./checklink-docs.sh\n", "```\n", "\n", "The script will be silent and only report problems as it finds them.\n", "\n", "Remember, that it's testing the live website, so if you detect problems and make any changes, remember to first commit the changes and wait a few minutes before re-testing.\n", "\n", "You can also test the site locally before committing your changes, please see: [README](https://github.com/fastai/fastai/blob/master/checklink/README.md).\n", "\n", "To test the docs-dev.fast.ai site, do:\n", "```\n", "./checklink-docs-dev.sh\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Working with Markdown" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Preview\n", "\n", "If you work on markdown (.md) files it helps to be able to validate your changes so that the resulting layout is not broken. [grip](https://github.com/joeyespo/grip) seems to work quite well for this purpose (`pip install grip`). For example:\n", "\n", "```\n", "grip -b docs/release.md\n", "```\n", "will open a browser with the rendered markdown as html - it uses github API, so this is exacly how it'll look on github once you commit it. And here is a handy alias:\n", "\n", "```\n", "alias grip='grip -b'\n", "```\n", "so you don't need to remember the flag." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Markdown Tips" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* If you use numbered items and their number goes beyond 9 you must switch to 4-whitespace chars indentation for the paragraphs belonging to each item. Under 9 or with \\* you need 3-whitespace chars as a leading indentation.\n", "* When building tables make sure to use `--|--` and not `--+--` to separate the headers - github will not render it properly otherwise." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Testing site locally" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Install prerequisites:\n", "\n", "```\n", "sudo apt install ruby-bundler\n", "```\n", "When running this one it will ask for your user's password (basically running a sudo operation):\n", "```\n", "bundle install jekyll\n", "```\n", "\n", "Start the website:\n", "```\n", "cd docs\n", "bundle exec jekyll serve\n", "```\n", "\n", "it will tell you which localhost url to go to to see the site." ] } ], "metadata": { "jekyll": { "keywords": "fastai", "summary": "Documentation modules overview", "title": "gen_doc" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": false, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }