Documenting a python project is a daunting task. Even though I myself have had experience with documentation it always takes me time to get the setup just right. In python the standard way to create documentation is with Sphinx. Sphinx is not straightforward to use and relies heavily on restructured text. Restructured text is a somewhat more verbose markup language than Markdown but is not too hard to learn the syntax. There is a lot to learn to use RST properly with python sadly. Most of the installation instructions follow the awesome An idiot’s guide to Python documentation with Sphinx and ReadTheDocs. My changes are that I want to show how to include docstring in the google format and how to additionally deploy documentation on a static site without readthedocs.
First you will install sphinx and create a docs folder in the root of your project. If you want to use the readthedocs theme for the documentation you will need to install the sphinx_rtd_theme.
pip install sphinx sphinx_rtd_theme
Next you will want to setup a basic sphinx project. You will do this
by running the command sphinx-quickstart within the
Most of the default options are good. You will need to set a
version, and answer yes to
autodoc since we want our
project source code to be documented. At this point you will have very
basic sphinx documentation. Next we need to add our package source
documentation. This can be done via the
sphinx-autodoc. Add the following to your
Makefile in the
docs. Now run
make apidocs to add the outline of your source
Add to the makefile the following lines.
apidocs: sphinx-apidoc -o source/ ../<package>
If you wanted the readthedocs theme instead of the default you will need to modify
... html_theme = 'sphinx_rtd_theme' ...
The default sphinx apidoc is tedious and verbose. I recommend using sphinx napoleon docstrings which has been standardized by google and numpy. In order to use napoleon the extension needs to be added.
... extensions = [ ... 'sphinx.ext.napoleon' ] ...
Here is an example of simple function being documented in the google style. See the google docstring format for further details.
def fizzbuzz(n): """A super advanced fizzbuzz function Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz” Prints out fizz and buzz Args: n (int): number for fizzbuzz to count to Returns: None: prints to stdout fizzbuzz """ def _fizzbuzz(i): if i % 3 == 0 and i % 5 == 0: return 'FizzBuzz' elif i % 3 == 0: return 'Fizz' elif i % 5 == 0: return 'Buzz' else: return str(i) print("\n".join(_fizzbuzz(i+1) for i in range(n)))
If you want math support there is a mathjax extension. Just again
conf.py. If you want latex support when exporting to a pdf
follow this math sphinx documentation.
... extensions = [ ... 'sphinx.ext.mathjax' ] ...
Math can then simply be included inline or in block format. Use the awesome latex markup language to write equations.
as some inline text :math:`\beta \gamma` or as a block math equation .. math:: \beta = \gamma
At this point you are ready to go! You can run
make html within
the docs folder and it will build the website in
docs/_build/html. Okay so great we have the static files for the
website but how do I deploy them?! There are two answers and you can
choose both: self hosting and readthedocs.org.
First you will signup an account with readthedocs.org. It is not
necessary to link an account as readthedocs will work with any
publicly available version controlled repo.
Import a project ->
Import Manually and give the project a unique name and specify the
repository url. The name that you provide determines the url
<name>.readthedocs.org. For full documentation see the
Readthedocs will detect and change in the repository and rebuilt the documentation. However often times the default configuration does not work with cutting edge projects and also by default does not install the project when building the documentation. To specify the readthedocs configuration in your project you should use .readthedocs.yml. A basic configuration is specified below. Readthedocs uses docker containers and has many more configuration options. With this you should be all setup! Read the documentation for additional options.
build: image: latest python: version: 3.6 setup_py_install: true
Scientific packages often have dependencies that require c extensions or cython. In order to use readthedocs that has c extension dependencies you will need to mock out all the dependencies in the
conf.py. This is documented in the readthedocs FAQ. One more reason I would recommend hosting the static site yourself.
import sys from unittest.mock import MagicMock class Mock(MagicMock): @classmethod def __getattr__(cls, name): return MagicMock() MOCK_MODULES = ['pygtk', 'gtk', 'gobject', 'argparse', 'numpy', 'pandas'] sys.modules.update((mod_name, Mock()) for mod_name in MOCK_MODULES)
static documentation site¶
Sometimes it is nicer to just deploy the static website yourself. With this deployment we get much more flexibility on the resulting documentation. Read the docs is an awesome resource but it does have limitations. For instance one issue I have had is that it does not generate docstrings from cextensions such as cython code and cannot handle packages with c extensions. There are workarounds by mocking the modules. In these cases we can use Gitlab CD/CI for deploying our own static site.
Since we already have a pipeline for our project lets include the
static website building. Add the following to
stages: - test - deploy - docs pages: image: python:3.6 stage: docs script: - pip install sphinx sphinx_rtd_theme - pip install -e . - mkdir public - cd docs - make apidocs - make html - cp -r _build/html/* ../public artifacts: paths: - public only: - master
We are using gitlab pages to deploy
our website. It should be available at
<username>.gitlab.io/<repo>. If you would like to add a custom
domain follow either my blog at gitlab static site deployment or
look at the gitlab cloudflare documentation.
Now you have your documentation completed!