{ "metadata": { "celltoolbar": "Slideshow", "name": "", "signature": "sha256:0b43b6a7b2850a3ff7349ac211b2b8e873798789882aede9402a8996d36b7132" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "See Docs Run. Run, Docs, Run!\n", "==============================\n", "\n", "![](files/rundocrunsvg.png)\n", "\n", "Catherine Devlin, PyCon 2014\n", "----------------------------\n", "\n", "http://tinyurl.com/rundocsrun" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Executable documentation\n", "========================\n", "\n", "Embeds real code\n", "\n", "Executes\n", "\n", "* at creation time\n", "* at reading time" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "What's a doc?\n", "=============\n", "\n", "- How you use my package\n", "\n", "- How you write a program\n", "\n", "- How you do some math, science, ...\n", "\n", "- How our business operates" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Problems addressed\n", "==================\n", "\n", "Forgetfulness\n", "-------------\n", "\n", "* While you write\n", "* As you update code\n", "\n", "Boredom\n", "-------\n", "\n", "* To write\n", "* To read" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "![](files/hf3.jpg)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Solution\n", "========\n", "\n", "Code your docs" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Self-verifying\n", "--------------\n", "\n", "* forgetfulness\n", "\n", "It's code\n", "---------\n", "\n", "* Fun to write\n", "* Fun to read" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Plan\n", "====\n", "\n", "`argumentclinic`\n", "----------------\n", "\n", "* Usage samples\n", "* Software requirements" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Technologies\n", "------------\n", "\n", "* doctests\n", "* Sphinx\n", "* IPython Notebook\n", "* Dexy" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Doctests\n", "========\n", "\n", "[est. 1999](https://groups.google.com/forum/#!msg/comp.lang.python/DfzH5Nrt05E/Yyd3s7fPVxwJ)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "def argue(statement):\n", " \"\"\"Provides an argument.\n", " \n", " >>> argue(\"It probably is.\")\n", " \"No it isn't.\"\n", " >>> argue(\"Fine, it isn't.\")\n", " \"Yes it is.\"\n", " \"\"\" \n", " return \"No it isn't.\"" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "prompt_number": 1 }, { "cell_type": "code", "collapsed": false, "input": [ "import doctest\n", "\n", "if __name__ == '__main__':\n", " doctest.testmod()" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "**********************************************************************\n", "File \"__main__\", line 6, in __main__.argue\n", "Failed example:\n", " argue(\"Fine, it isn't.\")\n", "Expected:\n", " \"Yes it is.\"\n", "Got:\n", " \"No it isn't.\"\n", "**********************************************************************\n", "1 items had failures:\n", " 1 of 2 in __main__.argue\n", "***Test Failed*** 1 failures.\n" ] } ], "prompt_number": 2 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "New name: Executable docstrings\n", "-------------------------------\n", "\n", "- Accurate\n", "- Relevant\n", "- Readable\n", "- Fun" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "*orthogonal* to your unit testing" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Sphinx\n", "======\n", "\n", "reST -> docs\n", "\n", "* attractive\n", "* interlinked\n", "* indexed\n", "* searchable" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "sphinx.ext.doctest\n", "------------------" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "While running `sphinx-quickstart`, answer\n", "\n", " > doctest: automatically test code snippets in doctest blocks (y/N) [n]: y" ] }, { "cell_type": "code", "collapsed": false, "input": [ "cd ~/proj/argument-clinic/argumentclinic/docs/sphinx/" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "/home/catherine/proj/argument-clinic/argumentclinic/docs/sphinx\n" ] } ], "prompt_number": 3 }, { "cell_type": "code", "collapsed": false, "input": [ "cat usage_doctest.rst" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Usage (with ``doctest``)\r\n", "========================\r\n", "\r\n", ".. doctest::\r\n", "\r\n", " >>> from argumentclinic.argumentclinic import argue\r\n", " >>> argue(\"This is my favorite package!\")\r\n", " \"No it isn't.\"\r\n", " >>> argue(\"It is!\")\r\n", " 'It is not.'\r\n", " >>> argue(\"Is!\")\r\n", " \"Isn't.\"\r\n", " >>> argue(\"This isn't even a proper argument.\")\r\n", " \"I guess you're right.\"\r\n", " \r\n" ] } ], "prompt_number": 4 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ " \n", "Then, build docs with \n", "\n", " make doctest html" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ " Document: usage_doctest\n", " -----------------------\n", " **********************************************************************\n", " File \"usage_doctest.rst\", line 12, in default\n", " Failed example:\n", " argue(\"This isn't even a proper argument.\")\n", " Expected:\n", " \"I guess you're right.\"\n", " Got:\n", " 'Yes it is.'\n", " **********************************************************************\n", " 1 items had failures:\n", " 1 of 5 in default\n", " 5 tests in 1 items.\n", " 4 passed and 1 failed.\n", " ***Test Failed*** 1 failures." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Also runs test included via ``autodoc``" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "sphinxcontrib.autorun\n", "---------------------\n", "\n", "Install normally\n", "\n", " hg clone http://bitbucket.org/birkenfeld/sphinx-contrib/\n", " cd sphinx-contrib/autorun\n", " python setup.py install\n", "\n", "*Enable* within each Sphinx project\n", "\n", " echo \"extensions.append('sphinxcontrib.autorun')\" >> conf.py\n", " \n", "Python 2 only **(BOOOOO!)**" ] }, { "cell_type": "code", "collapsed": false, "input": [ "cat usage_autorun.rst " ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Usage (with ``autorun``)\r\n", "========================\r\n", "\r\n", ".. runblock:: pycon\r\n", "\r\n", " >>> from argumentclinic.argumentclinic import argue\r\n", " >>> argue(\"This is my favorite package!\")\r\n", " >>> argue(\"It is!\")\r\n", " >>> argue(\"Is!\")\r\n", " >>> argue(\"This isn't even a proper argument.\")\r\n", "\r\n" ] } ], "prompt_number": 5 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ " make html\n", " \n", "[result](argumentclinic/docs/sphinx/_build/html/usage_autorun.html)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "system requirements, with autorun\n", "---------------------------------" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import get_reqs\n", "\n", "get_reqs.requirements_met()" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 6, "text": [ "['sqlalchemy>=0.9: satisfied by SQLAlchemy 0.9.2',\n", " 'psycopg2: satisfied by psycopg2 2.5.1',\n", " 'BeautifulSoup4>4.1: satisfied by beautifulsoup4 4.3.2']" ] } ], "prompt_number": 6 }, { "cell_type": "code", "collapsed": false, "input": [ "cat requirements_autorun.rst" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Software Requirements (``autorun``)\r\n", "===================================\r\n", "\r\n", "(Discovered with Sphinx's ``autorun`` extension)\r\n", "\r\n", "Python packages\r\n", "---------------\r\n", "\r\n", ".. runblock:: pycon\r\n", "\r\n", " >>> import sys\r\n", " >>> sys.path.insert(0, '.')\r\n", " >>> from get_reqs import requirements_met\r\n", " >>> print(\"\\n\".join(requirements_met()))\r\n", "\r\n", "PostgreSQL\r\n", "----------\r\n", "\r\n", "\r\n", "Your system should run PostgreSQL 9.1 or better.\r\n", "\r\n", ".. runblock:: console\r\n", "\r\n", " $ psql --version\r\n", "\r\n", "\r\n" ] } ], "prompt_number": 7 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ " make html\n", " \n", "[result](argumentclinic/docs/sphinx/_build/html/requirements_autorun.html)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "sphinxcontrib.programoutput\n", "---------------------------\n", "\n", "Install\n", "\n", " pip install sphinxcontrib-programoutput\n", "\n", "\n", "*Enable* within each Sphinx project\n", "\n", " echo \"extensions.append('sphinxcontrib.programoutput')\" >> conf.py" ] }, { "cell_type": "code", "collapsed": false, "input": [ "cat requirements_progout.rst" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Software Requirements (with ``programoutput``)\r\n", "==============================================\r\n", "\r\n", "Python packages\r\n", "---------------\r\n", "\r\n", ".. program-output:: python get_reqs.py\r\n", "\r\n", "PostgreSQL\r\n", "----------\r\n", "\r\n", "PostgreSQL 9.1 or better required. Your system has:\r\n", "\r\n", ".. command-output:: psql --version\r\n", "\r\n" ] } ], "prompt_number": 8 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ " make html\n", " \n", "[result](argumentclinic/docs/sphinx/_build/html/requirements_progout.html)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Sphinx Advantages\n", "-----------------\n", "\n", "* Complex multi-document structure\n", "* Generate many formats (PDF, ePub, LaTeX, ...)\n", "* Doc hosting at Read the Docs\n", "* Many extensions and options\n", "* Python standard" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Shout-outs\n", "==========\n", "\n", "[literate-resting](https://github.com/ged-lab/literate-resting/)\n", "----------------------------------------------------------------\n", "\n", "Shell script to preprocess console commands in reST\n", "\n", "[sphinxcontrib-autoprogram](https://pythonhosted.org/sphinxcontrib-autoprogram/)\n", "--------------------------------------------------------------------------------\n", "\n", "Reads ``argparse``, auto-generates doc for Sphinx" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "IPython Notebook\n", "================\n", "\n", "An interactive browser-based Python environment\n", "-----------------------------------------------" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "[Installation instructions](http://ipython.org/installation.html)\n", "\n", " apt-get build-dep ipython-notebook\n", " easy_install ipython[all]\n", " \n", "Start notebook server\n", "\n", " ipython notebook \n", " \n", "[see notebook](http://127.0.0.1:8888/notebooks/example.ipynb)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Serving\n", "-------\n", "\n", "* Your own server\n", "* [nbviewer](http://nbviewer.ipython.org) + URL or gist\n", " - [Our example](https://raw.githubusercontent.com/catherinedevlin/argument-clinic/master/example.ipynb)\n", " - [on nbviewer](http://nbviewer.ipython.org/urls/raw.githubusercontent.com/catherinedevlin/argument-clinic/master/example.ipynb)\n", "* `nbconvert` to HTML or LaTeX\n", "* Live hosting at [wakari.io](http://wakari.io)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Moar Awesome Please\n", "-------------------\n", "\n", "* [nbviewer](http://nbviewer.ipython.org/)\n", "* [gallery](https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks)\n", "* [Probabilistic Programming](http://nbviewer.ipython.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter1_Introduction/Chapter1_Introduction.ipynb)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "IPython Notebook Advantages\n", "---------------------------\n", "\n", "* Ease of code / doc integration\n", "* User can experiment\n", "* Static, downloadable hosting at [nbviewer](http://nbviewer.ipython.org)\n", "* Live hosting at [wakari.io](wakari.io)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "dexy\n", "====\n", "\n", "Framework for applying transformations to documents" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Installing\n", "----------\n", "\n", " pip install dexy \n", "\n", "Python 2 only *(BOOOOOO)*" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Steps\n", "-----\n", "\n", "* `dexy setup`\n", "* Source `.rst` or `.md`\n", "* `dexy.txt` or `.yml` recipe \n", "* `dexy`\n", "* `dexy reset`" ] }, { "cell_type": "code", "collapsed": false, "input": [ "cd ~/proj/argument-clinic/argumentclinic/docs/dexy/" ], "language": "python", "metadata": { "slideshow": { "slide_type": "skip" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "/home/catherine/proj/argument-clinic/argumentclinic/docs/dexy\n" ] } ], "prompt_number": 9 }, { "cell_type": "code", "collapsed": false, "input": [ "cat index.rst" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "\r\n", "argument-clinic\r\n", "===============\r\n", "\r\n", "`argument-clinic` supplies arguments as desired.\r\n", "\r\n", "Usage\r\n", "-----\r\n", "\r\n", "::\r\n", "\r\n", " {{ d['arg_example.py'] | indent(4) }}\r\n", "\r\n", "Result::\r\n", "\r\n", " {{ d['arg_example.py|py'] | indent(4) }}\r\n", "\r\n", "Software requirements\r\n", "---------------------\r\n", "\r\n", "::\r\n", "\r\n", " {{ d['get_reqs.py|py'] | indent(4) }}\r\n", "\r\n", "**Also requires** PostgreSQL_ v. 9.1 or above. \r\n", "You have: {{ d['psql_version.bash|bash'] }}\r\n", "\r\n", ".. _PostgreSQL: http://postgresql.org\r\n", "\r\n" ] } ], "prompt_number": 10 }, { "cell_type": "code", "collapsed": false, "input": [ "cat dexy.yaml" ], "language": "python", "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ ".rst|jinja|rstbody|easyhtml:\r\n", " - .py\r\n", " - .py|py\r\n", " - .bash|bash\r\n", " \r\n" ] } ], "prompt_number": 11 }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ " dexy\n", "\n", "[result](output/index.html)" ] }, { "cell_type": "code", "collapsed": false, "input": [ "!dexy filters" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "Installed filters:\r\n", " abc : Runs `abcm2ps` on .abc music files.\r\n", " abcm : Runs `abcm2ps` on .abc music files, generating all output formats.\r\n", " apis : Base class for filters which post content to a remote API.\r\n", " applytemplate : Apply template to file. Template should specify %(content)s.\r\n", " archive : Creates a .tgz archive of all input documents.\r\n", " asciidoc : Runs asciidoc command.\r\n" ] }, { "output_type": "stream", "stream": "stdout", "text": [ " asciidoctor : Runs `asciidoctor`.\r\n", " asciisyn : Surrounds code with highlighting instructions for Asciidoctor\r\n", " bash : Runs bash scripts using 'bash' and returns stdout.\r\n", " bashint : Runs bash. use to run bash scripts.\r\n", " bn : Forces previous filter to output .bmp extension.\r\n", " botoup : Uses boto library to upload content to S3, returns the URL.\r\n", " bw : Converts color pdf to black and white.\r\n", " bwconv : Converts color pdf to black and white.\r\n", " c : Compile code using gcc and run.\r\n", " calibre : Runs `ebook-convert` command (part of calibre)\r\n", " casperjs : Runs scripts using casper js. Saves cookies.\r\n", " cb : Changes file extension to .sh\r\n", " cfussy : Compile code using gcc and run, raising an error if compiled code returns nonzero exit.\r\n", " ch : Changes file extension to .html\r\n", " chext : Dummy filter for allowing changing a file extension.\r\n", " cinput : Compile code using gcc and run with input.\r\n", " cj : Changes file extension to .json\r\n", " clang : Compile code using clang and run.\r\n", " clanginput : compile code using clang and run with input.\r\n", " clj : Runs clojure.\r\n", " cowsay : Runs input through 'cowsay'.\r\n", " cowthink : Runs input through 'cowthink'.\r\n", " cpickle : Forces previous filter to output .cpickle extension.\r\n", " cpp : Compile c++ code using cpp and run.\r\n", " cppinput : Compile c++ code using cpp and run with input.\r\n", " ct : Changes file extension to .txt\r\n", " customize : Add