Updating the PSI4 Users’ and Programmers’ Manual

PSI4‘s documentation is generated by Sphinx and lives in psi4/doc/sphinxman. It is available online at http://psicode.org/psi4manual/master/index.html for the latest development branch.

Installing Sphinx

Installing Sphinx is only necessary to build the documentation yourself, locally. The docs are served from from psicode, so most users and developers won’t need Sphinx installed. Nevertheless, installation is easy. Math is rendered through MathJax, so LaTeX and dvipng are no longer needed. The sphinx executable should be in your path at CMake configure time for documentation targets to be available.

  • Binary: conda install sphinx

  • Binary: conda env create -f devtools/conda-envs/docs-cf.yaml

  • Binary: pip install -U Sphinx

  • Source: https://pypi.org/project/Sphinx/

  • Check:

    >>> which sphinx-build
//anaconda/bin/sphinx-build
>>> sphinx-build --version  # needs >= 3.5
Sphinx (sphinx-build) 3.5.3
>>> cmake
...
 -- Documentation targets available: sphinxman (html), sphinxmini (quick html), sphinxpdf (LaTeX --> PDF)
...

Documentation Structure

Sphinx has nice capabilities for extracting docstrings from python files, presenting both auto-generated and narrative documentation in the same format, hyperlinking within and to external websites, and generating documentation in different formats from the same source. PSI4‘s documentation is a unified document covering information for both users and programmers in separate sections. From the top-level object directory, build the following target (note that a working version of the PSI4 executable is a requirement for building the documentation). Only GNU Makefiles, not Ninja, works for the docs:

>>> make sphinxman
# -OR-
>>> cmake --build . --target sphinxman

This will build a full set of documentation in the html directory that can be viewed offline through any browser.

doc/sphinxman/html/index.html

Much of the documentation is auto-generated from the source. At present, this covers:

  • Physical Constants: psi4/psi4/include/psi4/physconst.h

  • Python Driver: docstrings from *.py files in psi4/psi4/driver

  • Databases: docstrings from *.py files in psi4/psi4/share/psi4/databases

  • Basis Sets: *.gbs files in psi4/psi4/share/psi4/basis

  • C++ Keywords: psi4/psi4/src/read_options.cc

  • Sample Inputs: input.dat files in psi4/samples

  • PSI Variables: variables and associated modules extracted from code and comments in the Python and C++ source * Modules scraped are the sections of psi4/psi4/src/read_options.cc * Variables should be all-caps, except where representing substitutions, e.g., ROOT n -> ROOT m and double-quote " delimited, even in Python * Scraper looks for Process::environment.globals, set_array_variable, variables_, etc. lines and comments in the C++ code * C-side, the module for the variable is determined by the directory where it’s found. * Scraper looks for set_variable together with # P::e MODULE lines and comments in the Python code * Py-side, the module for the variable is specified by MODULE in the comment * When a variable is set by code in either language, e.g., variables_[varname.str()] rather than plain string, variables_["FCI TOTAL ENERGY"], add a plain string line as a single-line comment, so the scraper can find it. * Add new places to scrape for variables to psi4/doc/sphinxman/document_psivariables.pl * For now, we’re scraping both global and Wfn variables * All of these show up in referenceable appendices like apdx:detci_psivar

  • Plugins: doc.rst text, *.py modules, and C++ keywords in psi4/tests/plugin_* plugin directories (disabled at the moment)

  • PSI Files: scratch file names and numbers in psi4/psi4/include/psi4/psifiles.h

Some documentation is even extracted from PSI4 objects at runtime.

Building all the documentation takes ~10 minutes. There is now good dependency structure built into the psi4/doc/sphinxman/CMakeLists.txt , so very long builds should be infrequent (unless you’re touching psi4/psi4/src/read_options.cc or the driver. Note that not all dependencies are encoded (PSI variables, for instance, depend on every .cc file in the source tree), so for a definitive doc build, remove (in the object directory) doc/sphinxman and start from scratch.

Even ~10 minutes of build time can be annoying when developing documentation and testing rst files. In that situation, use the target below which builds only the written docs (not autodocs) in psi4/doc/sphinxman/source quickly, though with a lot of warnings for unresolved links:

>>> make sphinxmini

reStructuredText

Sphinx files are written in reStructuredText (*.rst). In the html documentation, source code is available from the sidebar. Here are a few resources on Sphinx formatting.

Math in the Codebase

It is often useful to have mathematical expressions in docstrings or comments in the code that are auto-documented into the manual. Such locations include the #! comment comments at the top of test case input files, the /*- comment -*/ comments in psi4/psi4/src/read_options.cc, and the r""" comment """ docstrings in python modules. (That r""" makes the string read literally, so your LaTeX symbols aren’t confused with escape characters.) For the two former, math has traditionally been written in LaTeX (with the special substitution @@ for subscripting underscore). The autodoc script has been trained to convert inline LaTeX math to reST math, provided the expression within dollar signs is offset from other text. That is, expressions of the form ^ $latex math$[., ]$ (pseudo-regex) are good, while H$_2$O and LaTeX tables are not translated correctly. Python docstrings are absorbed as-is, so please use reST math formatting (essentially $latex math$ \(\Rightarrow\) :math:`latex math`). Starting around PSI4 1.1, MathJax is used for in-browser LaTeX rendering in place of offline PNG generation of math images. Check the online rendering, as occasionally there will be errors even when the LaTeX looked sound.

The Map of the Sphinx

  • Adding a new Appendix or First-TOC-Level page

    Create your reST file and fill it with information. Add the name of your file to psi4/doc/sphinxman/source/appendices.rst for an appendix or to psi4/doc/sphinxman/source/index.rst for a first-TOC-level. Finally, add your file to the STATICDOC variable in psi4/doc/sphinxman/CMakeLists.txt. Sphinx will now build with your new page.

  • Adding a new module to “Theoretical Methods”

    Copy the file of a well-established module, like psi4/doc/sphinxman/source/sapt.rst. Change the title, author, sec label, ref, and source labels at the top of the file to point instead to your code. Edit psi4/doc/sphinxman/source/methods.rst to add the name of your file so that it will appear in the TOC tree. Add your file to the STATICDOC variable in psi4/doc/sphinxman/CMakeLists.txt. Sphinx will now build with your new file. Follow the models in existing methods pages to write your documentation. If you don’t get all the keyword links, bibliography links, sample inputs, math, tables, etc. working in Sphinx, don’t worry about it. A genie will probably come through and tidy up all your source.