Compiling and Installing from Source

This section fulfills the duty of every Linux project to have an INSTALL file detailing how to build it from source. Few (mostly packagers and sysadmins) are expected to read this file as it is a broad guide to the many choices in building this project. For a specific, well-tested, and performance-tuned build path, see psicode.org/installs/latest (select “source”). For pre-built binaries again well-tested, performance-tuned, and available for all common operating systems, see psicode.org/installs/latest (select “conda”).

Planning: how to configure Psi4 and invoke CMake

PSI4 is built through CMake. An abbreviated build guide can be found within the source itself .

CMake does a good job scanning your computer to locate libraries, header files, and executables needed for compilation. So it’s very possible that from top-level-psi4-dir source directory, you can call cmake -S. -Bobjdir without any further arguments, and it will invoke cmake, detect some appropriate defaults, configure the build, create a build directory objdir, and complete, leaving you to only invoke make or Ninja in the new build directory.

Should this happy scenario not come to pass, or if the default build options are not to your taste, use the links within core dependencies and add-on dependencies to plan a set of arguments to cmake tailored to your computer. Proceed to quick build or detailed build.

How to build and install Psi4, the compact version

This section outlines the main steps of configuring, compiling, and installing PSI4. More detail is given here.

>>> cd {top-level-psi4-dir}
>>> cmake -S. -Bobjdir [your configuration options]
>>> cd objdir
>>> make -j`getconf _NPROCESSORS_ONLN`
>>> make install

How to build, test, and install Psi4, in detail

1. Plan Directories

Get ahold of the PSI4 codebase, and navigate to the top level source directory, hereafter top-level-psi4-dir.

>>> cd {top-level-psi4-dir}

Choose a compilation directory, hereafter objdir

Choose an installation directory, hereafter prefix

2. Plan Configuration

Examine the strict and optional software requirements to make sure the target computer has all the necessary dependencies installed.

Prepare any necessary or desired configuration options for cmake, hereafter [your configuration options]

3. Configure

Run CMake with planned options and directories, as below. It reports on software found or unfound as it scans the computer, then (upon success) creates objdir ready for compilation.

>>> cmake -S. -B{objdir} -DCMAKE_INSTALL_PREFIX={prefix} [your configuration options]

4. Compile

Compile the code (optional -j triggers parallel compilation).

>>> cd {objdir}
>>> make -j`getconf _NPROCESSORS_ONLN`

5. Test

>>> ctest -j`getconf _NPROCESSORS_ONLN`

>>> make pytest

6. Install

If tests pass, install the code.

>>> make install

7. Configure Runtime

To run PSI4 after installation, you need to configure a few variables:

What are the tools and dependencies strictly required for building Psi4

The core PSI4 build requires the software below. Note that practically everything (including Python, CMake, NumPy, BLAS/LAPACK, Libint, and even C++ compilers on Linux and Mac) can be satisfied through conda. The links below give examples of how to configure that software for PSI4 and any notes and warnings pertaining to it.

The following are also required for PSI4, but if not detected, the build system will automatically download and build.

Additionally, there are runtime-only dependencies:

Dropped Dependencies:

What are the add-on capabilities for Psi4 and what are their dependencies

Each of the items below is an independent additional capability that can be built with PSI4. Sub-items below are the respective additional dependencies of the add-on. Select which, if any, you want, and examine the links for appropriate enabling arguments to cmake. Note that many are available pre-built from conda.

Additionally, there are runtime-loaded capabilities:

How to use conda to get a Psi4 development environment

PSI4 has a lot of dependencies, some of which are complicated to build. Fortunately, conda can supply them all. A conda package psi4-dev is available to gather the dependencies and build tools needed to compiler PSI4 from source. Go to psicode.org/installs/latest, select “source” and “nightly build” along with Linux/Mac and a Python version. Follow its instructions to create a conda environment that has a script psi4-path-advisor. Running psi4-path-advisor --help shows more options. Running the script without arguments gives a cmake command that supplies paths to dependency conda packages to CMake’s cache. You can add further cmake options to this command when you execute it. Once the cmake command has built your objdir, you can build PSI4. Remember to activate this conda environment during PSI4 running or development. There are a few extra options to psi4-path-advisor to set up using conda’s GNU compilers (on Linux), conda’s Clang compilers (on Mac; less reliably), or your own Intel compilers backed by conda’s GNU compilers (on Linux).

>>> git clone https://github.com/psi4/psi4.git && cd psi4
>>> conda create -n p4dev psi4-dev python=3.8 -c psi4/label/dev
>>> conda activate p4dev
>>> `psi4-path-advisor --gcc`
>>> cd objdir && make -j`getconf _NPROCESSORS_ONLN`

How to configure code to use high angular momentum basis sets

The Libint integral code handles arbitrary order angular momentum (AM), but compiling that is prohibitive. The build process for Libint2 takes longer than for Libint1, so it’s recommended to use the conda packages. The AM controlling keyword MAX_AM_ERI has definition changed a little from Libint1. It is used to ensure that conventional energies can use at least MAX_AM_ERI zeta. See psi4//external/upstream/libint2/CMakeLists.txt for a fuller discussion of AM capabilities. Unless BUILD_Libint2_GENERATOR is set to ON, an internal build of Libint2 will download a pregenerated tarball of at least MAX_AM_ERI AM. If you want to build completely from generator source, turn on the keyword, but parts can only run in serial and parts benefit from parallel, so it’s not recommended. All the AM generation settings would need to be edited in the above CMakeLists.txt rather than passing them into the PSI4 build directly.

  • Build with Higher Angular Momentum

    >>> cmake -DMAX_AM_ERI=6

  • Relevant CMake Options:

    MAX_AM_ERI=N        # The maximum angular momentum level (1=p, 2=d, 3=f,
                    # etc.) for the libint integrals and derivative
                    # integrals. A value of N implies a maximum first
                    # derivative of N-1, and maximum second derivative of
                    # N-2, so for an atom such as Neon, the default 5 gets
                    # you conventional cc-pV5Z for energies, cc-pVQZ for
                    # gradients, cc-pVTZ for frequencies and density-fitted
                    # cc-pVQZ for energies, cc-pVTZ for gradients, cc-pVDZ
                    # for frequencies. [default: 5]

Note that since PSI4 1.1, it is possible to build Libint independently (or install just the libint conda package), then have any/all PSI4 builds detect that installation at compile-time.

To switch a finished build to a larger Libint, enter <objdir>, remove the result of make install (if ever invoked), remove external/upstream/libint/. If earlier Libint was built internally (as opposed to detecting an external installation), remove stage/include/libint/, stage/share/cmake/Libint/, stage/lib/libint2.so.2. Edit MAX_AM_ERI value in CMakeCache.txt. Reinvoke make and check the number beside Found Libint in the CMake output.

Note that the same keyword MAX_AM_ERI controls Libint and simint. simint can’t compile above AM7, so you’ll have to turn off simint to get an internal compile of AM>=8 Libint. The keyword is only a lower bound for detection, so you can set it at 7, have both Libint and simint enabled, and point CMake toward a pre-built Libint AM8 just fine.

How to get high angular momentum integrals from conda

Libint2 conda packages are available from conda for Linux, Mac, and Windows.

The Linux package has base MAX_AM_ERI=7 and can run 7z energies and gradients conventional, 6z energies and gradients density-fitted, and qz Hessians both conventional and density fitted.

The Mac and Windows packages have base MAX_AM_ERI=5 and can run 5z energies, qz gradients, and tz Hessians, both conventional and density-fitted.

Details about angular momentum settings are available here psi4/external/upstream/libint2/CMakeLists.txt .

How to see what build configuration options are available

CMake doesn’t provide a summary for this (unless you want to try the CMake GUI, which the developers have never looked at). However, the top half of the main CMakeLists.txt is a passable summary:

        LANGUAGES C CXX)
set(psi4_URL "http://www.psicode.org/")
set(psi4_EMAIL "psi4aiqc+cmake@gmail.com")
set(psi4_LICENSE "GNU Lesser General Public License, version 3 (LGPL-3.0)")
set(psi4_DESCRIPTION "Open-Source Quantum Chemistry")

list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake)

message(STATUS "Building using CMake ${CMAKE_VERSION} Generator ${CMAKE_GENERATOR}")

################  Options: Overview and Not Otherwise Mentioned  ###############

#  <<<  CMake build overview  >>>
#
#    >>> ls
#    external/  COPYING  psi4/  tests/  ...
#    >>> cmake -S. -Bobjdir -DCMAKE_INSTALL_PREFIX=/path/to/install-psi4 ...
#    ...
#    -- Generating done
#    -- Build files have been written to: /current/dir/objdir
#    >>> cmake --build objdir --target install -j`getconf _NPROCESSORS_ONLN`

#  <<<  Required build dependencies that Psi4 can't build itself  >>>
#
#    - CMake (e.g., `conda install cmake`)
#    - C++ and C compilers (C++17 compliant)
#    - BLAS/LAPACK (also runtime; e.g., `conda install mkl-devel`)
#    - Python (also runtime; interpreter and headers; e.g., `conda install python`)
#    - NumPy (also runtime; avoidable at buildtime if gau2grid pre-built; e.g., `conda install numpy`)
#    - networkx >=2.4 (runtime only; e.g., `conda install networkx`)
#    - pint >=0.10 (runtime only; e.g., `conda install pint -c conda-forge`)
#    - pydantic >=1.0 (runtime only; e.g., `conda install pydantic -c conda-forge`)
#    - msgpack-python (runtime only; e.g., `conda install msgpack-python`)
#    - Eigen (for Libint2; e.g., `conda install eigen`)
#    - SciPy (runtime only; e.g., `conda install scipy`)
#    - py-cpuinfo (runtime only; e.g., `conda install py-cpuinfo`)
#    - psutil (runtime only; e.g., `conda install psutil`)

#  These three "###  Options  ###" sections contain useful CMake variables for build configuration.

#  <<<  Compilers and flags  >>>
#
#    - CMAKE_C_COMPILER "C compiler"
#    - CMAKE_C_FLAGS "Additional C flags"
#    - CMAKE_CXX_COMPILER "C++ compiler"
#    - CMAKE_CXX_FLAGS "Additional C++ flags"
#    - CMAKE_Fortran_COMPILER "Fortran compiler (required for some add-ons)"
#    - CMAKE_Fortran_FLAGS "Additional Fortran flags"

#  <<<  Detecting dependencies and add-ons  >>>
#
#    - Python_EXECUTABLE "Python interpreter to use (e.g., /path/to/bin/python3.7)"
#    - Python_LIBRARY "Python library that goes with the interpreter (e.g., /path/to/lib/python3.7.so)"
#    - Python_INCLUDE_DIR "Path to the python include files (e.g., /path/to/include/python3.7)"
#    - SPHINX_ROOT "Root directory for Sphinx: 'bin/sphinx-build' (or similar) should be in this dir."
#
#    For any ${AddOn} of: ambit, CheMPS2, dkh, ecpint, libefp, erd, gau2grid, gdma, Libint2, PCMSolver, pybind11, pylibefp,
#                         qcelemental, qcengine, optking, simint, Libxc
#    - CMAKE_PREFIX_PATH "Set to list of root directories to look for externally built add-ons and dependencies
#      (e.g., /path/to/install-libint;/path/to/install-gdma where exists /path/to/install-libint/lib/libderiv.a)"
#    - ${AddOn}_DIR "Set to directory containing ${AddOn}Config.cmake file to facilitate detection of external build"
#    - CMAKE_DISABLE_FIND_PACKAGE_${AddON} "Set to OFF to force internal build"
#    - CMAKE_INSIST_FIND_PACKAGE_${AddON} "Set to ON to force external detect"

#  <<<  Detecting BLAS/LAPACK  >>>
#
#    - ENV(MATH_ROOT) "Root directory where BLAS/LAPACK libraries should be detected (e.g., ${MATH_ROOT}/lib/libblas.so)"
#    - BLAS_TYPE "Target BLAS distribution for math detection
#                 (default: search order MKL>FLEXIBLAS>OPENBLAS>ESSL>ATLAS>ACML>SYSTEM_NATIVE on Linux; MKL>Accelerate>... on Mac)"
#    - LAPACK_TYPE "Target LAPACK distribution for math detection
#                 (default: search order MKL>FLEXIBLAS>OPENBLAS>ESSL>ATLAS>ACML>SYSTEM_NATIVE on Linux; MKL>Accelerate>... on Mac)"
#    - LAPACK_LIBRARIES "Location of BLAS/LAPACK libraries as ";"-separated list of full paths, bypassing math detection"
#    - LAPACK_INCLUDE_DIRS "Location of BLAS/LAPACK headers (only needed for MKL), bypassing math detection"
#    - OpenMP_LIBRARY_DIRS "Location of OpenMP libraries (iomp5/gomp/omp) as ";"-separated list, hinting OpenMP detection"

#  <<<  Install  >>>
#
#    - CMAKE_INSTALL_PREFIX "Location to which Psi4 and internally built add-ons are installed (default: /usr/local/psi4)"
#    - CMAKE_INSTALL_BINDIR "Location within CMAKE_INSTALL_PREFIX to which executables are installed (default: bin)"
#    - CMAKE_INSTALL_LIBDIR "Location within CMAKE_INSTALL_PREFIX to which libraries are installed (default: lib)"
#    - CMAKE_INSTALL_DATADIR "Location within CMAKE_INSTALL_PREFIX to which resources are installed (default: share)"
#    - CMAKE_INSTALL_INCLUDEDIR "Location within CMAKE_INSTALL_PREFIX to which headers are installed (default: include)"
#    - PYMOD_INSTALL_LIBDIR "Location within CMAKE_INSTALL_LIBDIR to which python modules are installed (default: /)
#                            Must start with: / . Used to imitate python install: /python3.6/site-packages ."

############################  Options: Build What?  ############################
option(ENABLE_ambit "Enables the ambit tensor library" OFF)
option(ENABLE_CheMPS2 "Enables CheMPS2 for DMRG (requires HDF5)" OFF)
option(ENABLE_cppe "Enables CPPE for Polarizable Embedding" OFF)
option(ENABLE_adcc "Enables adcc for algebraic-diagrammatic construction methods (requires libtensorlight; can also be added at runtime)" OFF)
option(ENABLE_ddx "Enables ddx for domain-decomposition contiuum solvation models (requires Fortran; can also be added at runtime)" OFF)
option(ENABLE_dkh "Enables DKH integrals (requires Fortran)" OFF)
option(ENABLE_ecpint "Enables libecpint for effective core potentials (ECP)" OFF)
option(ENABLE_libefp "Enables LIBEFP and PylibEFP for fragments" OFF)
option(ENABLE_erd "Enables use of ERD instead of Libint (requires Fortran)" OFF)
option(ENABLE_simint "Enables use of SIMINT two-electron integral library" OFF)
option(ENABLE_gdma "Enables Stone's GDMA multipole code (requires Fortran)" OFF)
option(ENABLE_PCMSolver "Enables PCMSolver library (requires Fortran)" OFF)
option(ENABLE_snsmp2 "Enables SNSMP2 plugin (can also be added at runtime)" OFF)
option(ENABLE_v2rdm_casscf "Enables V2RDM_CASSCF plugin (requires Fortran; can also be added at runtime)" OFF)
option(ENABLE_psi4fockci "Enables Psi4FockCI plugin (can also be added at runtime)" OFF)
option(ENABLE_cct3 "Enables cct3 plugin (requires Fortran; can also be added at runtime)" OFF)
option(ENABLE_gpu_dfcc "Enables GPU_DFCC plugin for gpu-accelerated df-cc (requires CUDA; can also be added at runtime)" OFF)
option(ENABLE_mdi "Enables MolSSI driver interface" OFF)
option(ENABLE_BrianQC "Enables the BrianQC GPU module (requires CUDA; requires separate installation and licensing of the BrianQC module)" OFF)
option(ENABLE_Libint1t "Enables Libint1 in special _t variety" OFF)
# These options are relevant to pasture, expert only
option(ENABLE_ccsort "Enables ccsort plugin installed from psi4pasture" OFF)
option(ENABLE_transqt2 "Enables transqt2 plugin installed from psi4pasture" OFF)

# Append modules added to pasture as needed
if(ENABLE_ccsort OR ENABLE_transqt2)
  set(ENABLE_pasture ON)
  message(STATUS "Enabling pasture plugins")
endif()

if(ENABLE_gdma OR ENABLE_dkh OR ENABLE_erd OR ENABLE_PCMSolver)
    enable_language(Fortran)
    set(Fortran_ENABLED ON)  # communicate required languages with psi4-core
    message(STATUS "Enabling Fortran")
endif()

if(ENABLE_erd)
    message(FATAL_ERROR "The Psi4/ERD interface is broken, probably since spring 2017. It's toast (comment this line to try), but disabling for now.")
    message(WARNING "ERD will build, link, and run in Psi4 just fine. However, it has not been hooked into Psi4 in all roles, notably gradients, LRC DFT energies, and ESP. So upon activating through ``set integral_package erd``, known failures will be caught and halted, but perhaps other types not tested and identified will give *wrong* answers. Consider this your warning.")
endif()

############################  Options: Build How?  #############################
include(psi4OptionsTools)

Note that external projects will have their own sets of build configuration options. Only the most-common user knobs of those are mentioned above.

How to install elsewhere than /usr/local/psi4

The installation directory is the filesystem location for the executable script, the Python module, basis set data, and other administrative files. Unless using the conda package, which is relocatable, the installation directory must be specified with CMake variable CMAKE_INSTALL_PREFIX before compiling.

  • Build with Specific Install Directory

    cmake -DCMAKE_INSTALL_PREFIX=/nfs/common/software/psi4

  • Relevant CMake Options:

    CMAKE_INSTALL_PREFIX=PATH  # Location to which Psi4 and internally built
                           # add-ons are installed (default: /usr/local/psi4)

Note

It’s not guaranteed, but if, in a pinch, you need to install a built Psi4 to a location not configured by CMAKE_INSTALL_PREFIX, recursively copy the folders under objdir/stage/prefix to the desired location, chown them if needed, edit the shebang in bin/psi4 if needed, and recursively delete all the “.pyc” files. It may just run.

How to compile for debugging

Flags to turn optimizations off and debugging on can be set across the project and plugins with CMake variable CMAKE_BUILD_TYPE before compiling. Note that these flags will not propagate to any add-ons that are detected pre-built rather than built.

  • Build without optimization

    cmake -DCMAKE_BUILD_TYPE=debug

                              set the CMake build type [default: release]

  • Relevant CMake Options:

    CMAKE_BUILD_TYPE=[debug|release]  # Build type (Release or Debug)" [default: release]

How to choose the compilation directory, {objdir}

  • there is no default

  • common choices are objdir or build under top-level-psi4-dir

    • cd {top-level-psi4-dir} && cmake -S. -Bobjdir

    • cd {top-level-psi4-dir} && cmake -S. -Bbuild

  • in-source builds (*.cc and *.o in same directory) are disallowed

  • builds outside top-level-psi4-dir are permitted

How to save configuration settings for a future compilation

Create a file like do-configure with the cmake command and options on one line.

>>> cd {top-level-psi4-dir}
>>> cat do-configure
    cmake -S. -B{objdir} \
        -DCMAKE_INSTALL_PATH="/Users/me/psi4" \
        -DCMAKE_PREFIX_PATH="/Users/me/externals/install-libint" \
        -DMAX_AM_ERI=6 \
        -DENABLE_gdma=ON \
        -DBUILD_SHARED_LIBS=ON
>>> chmod u+x do-configure
>>> ./do-configure

What is the directory layout of the installed or staged Psi4

After compilation (cd objdir && make), a directory structure like the below will exist at objdir/stage. This may be tested and used just like a full installation.

After installation (cd objdir && make && make install), a directory structure like the below will exist at /prefix. This is a full installation.

/
bin/                                                   (executables for psi4 + any external proj)
bin/psi4                                             (psi4 executable, actually just a py script)
include/                                         (installed headers for psi4 + any external proj)
include/psi4/                                                     (header files for #include-ing)
include/psi4/psi4-dec.h                                                     (primary psi4 header)
include/psi4/masses.h                                                (a project-wide psi4 header)
include/psi4/libmints/                                                     (psi4 library headers)
include/psi4/libfock/                                                                     (ditto)
share/                                  (read-only arch-indep files for psi4 + any external proj)
share/cmake/psi4/                                         (files for detecting installed targets)
share/cmake/psi4/psi4Config.cmake                                       (psi4 build/install info)
share/cmake/psi4/psi4ConfigVersion.cmake                                (psi4 cmake version info)
share/doc/psi4/html/                                                  (sphinx html documentation)
share/psi4/                                                           (text files needed by psi4)
share/psi4/basis                                                                     (basis sets)
share/psi4/plugins                                                        (plugin template files)
share/psi4/fsapt                                                                  (fsapt scripts)
share/psi4/samples/                                                          (sample input files)
lib/                               (shared libraries and py modules for psi4 + any external proj)
# ordinary
lib/psi4/                                                                          (object files)
lib/psi4/driver/                                                            (py-side, uncompiled)
lib/psi4/header.py                                                           (prints file header)
lib/psi4/metadata.py                                                          (psi4 version info)
lib/psi4/__init__.py                                         (module marker/loader for psi4.core)
lib/psi4/core.cpython-*.so                               (c-side, compiled and bound by pybind11)
# conda
lib/pythonX.X/site-packages/psi4/

The following environment variables point to certain places in the above directory structure. None to few need to be set; see for details: running compiled executable, running compiled Python module, running conda binary.

  • PATH pointing to bin

  • PYTHONPATH pointing to lib (ordinary) or lib/pythonX.X/site-packages (conda)

  • PSIDATADIR pointing to share/psi4

How to run Psi4 as executable after compilation

Substituting the full installation directory prefix and a suitable scratch directory, issue the following commands directly in your terminal or place them into your “rc” file and open a new terminal. (To use a staged installation directory, substitute objdir/stage for prefix.)

# csh, tcsh: add to shell or ~/.tcshrc file
setenv PATH {prefix}/bin:$PATH
setenv PSI_SCRATCH /path/to/existing/writable/local-not-network/directory/for/scratch/files

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
export PATH={prefix}/bin:$PATH
export PSI_SCRATCH=/path/to/existing/writable/local-not-network/directory/for/scratch/files

Run PSI4.

>>> cat sample.in
molecule {
He
}
energy('hf/cc-pvdz')
compare_values(-2.85518839, get_variable('current energy'), 5, 'SCF E')
>>> psi4 sample.in
SCF E.............................................................PASSED

How to solve ModuleNotFoundError: No module named 'psi4'

You’re using PSI4 in PsiAPI mode (python input.py where input.py contains import psi4) but the Python interpreter can’t find the PSI4 Python module. Follow the directions at How to configure paths for PsiAPI.

How to configure paths for PsiAPI

If you know the location of the PSI4 executable (bin/psi4) for Psithon mode and want to know the corresponding location to add to PYTHONPATH for PsiAPI mode, execute psi4 --psiapi. It will return bash commands to set PATH (for correct python interpreter) and PYTHONPATH (to find psi4 module) correctly, after which import psi4 will work.

>>> psi4 --psiapi-path
export PATH=/path/to/dir/of/python/interpreter/against/which/psi4/compiled:$PATH
export PYTHONPATH=/path/to/dir/of/psi4/core-dot-so:$PYTHONPATH

>>> export PATH=/path/to/dir/of/python/interpreter/against/which/psi4/compiled:$PATH
>>> export PYTHONPATH=/path/to/dir/of/psi4/core-dot-so:$PYTHONPATH

>>> python -c "import psi4"

>>> python -c "import psi4;print(psi4.__file__, psi4.__version__)"

How to run Psi4 as Python module after compilation

Substituting the full installation directory prefix and a suitable scratch directory, issue the following commands directly in your terminal or place them into your “rc” file and open a new terminal. (To use a staged installation directory, substitute objdir/stage for prefix.)

# csh, tcsh: add to shell or ~/.tcshrc file
setenv PYTHONPATH {prefix}/lib:$PYTHONPATH
setenv PSI_SCRATCH /path/to/existing/writable/local-not-network/directory/for/scratch/files

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
export PYTHONPATH={prefix}/lib:$PYTHONPATH
export PSI_SCRATCH=/path/to/existing/writable/local-not-network/directory/for/scratch/files

Run PSI4.

>>> cat sample.py
import psi4
mol = psi4.geometry("""
He
""")
psi4.energy('hf/cc-pvdz')
psi4.compare_values(-2.85518839, psi4.core.get_variable('current energy'), 5, 'SCF E')
>>> python sample.py
SCF E.............................................................PASSED

How to run Psi4 as executable or Python module from conda installation

The configuration commands below are generic versions of the ones printed to your screen as advice upon installing PSI4 into a Anaconda, Miniconda, or Psi4conda distribution, condadist = ana|mini|psi4conda. To see the message again after initial installation, with the conda environment active, run .psi4-post-link.sh. If which conda python psi4 points to your condadist and echo $PSI_SCRATCH is set, skip ahead to the “Run PSI4" commands below. Otherwise, issue the following commands directly in your terminal or place them into your “rc” file and open a new terminal.

If you installed the Psi4conda distribution or installed the PSI4 conda package into the main environment of an Anaconda or Miniconda distribution and added that to your PATH, as prompted, then which psi4 likely yields condadist/bin/psi4 and the PATH setting lines below are redundant.

If you installed into a conda environment p4env and performed conda activate p4env, then which psi4 likely yields condadist/envs/p4env/bin/psi4 and the PATH setting lines below are redundant.

# csh, tcsh: add to shell or ~/.tcshrc file
unsetenv PSIDATADIR
setenv PATH {prefix}/bin:$PATH
setenv PSI_SCRATCH /path/to/existing/writable/local-not-network/directory/for/scratch/files

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
unset PSIDATADIR
export PATH={prefix}/bin:$PATH
export PSI_SCRATCH=/path/to/existing/writable/local-not-network/directory/for/scratch/files

Run PSI4 as executable.

>>> cat sample.in
molecule {
He
}
energy('hf/cc-pvdz')
compare_values(-2.85518839, get_variable('current energy'), 5, 'SCF E')
>>> psi4 sample.in
SCF E.............................................................PASSED

or Run PSI4 as Python module.

>>> cat sample.py
import psi4
mol = psi4.geometry("""
He
""")
psi4.energy('hf/cc-pvdz')
psi4.compare_values(-2.85518839, psi4.core.get_variable('current energy'), 5, 'SCF E')
>>> python sample.py
SCF E.............................................................PASSED

How to run Psi4 as executable after compilation using driver from source

When developing python driver code, it can be annoying to keep makeing to test the code. PSI4 can be run “inplace” through the following procedure. To be clear, this is running compiled C++ from the build directory and python from the source directory. This is an expert option for development, and not all functionality will be available.

>>> cd {objdir}
>>> ln -s {top-level-psi4-dir}/{objdir}/stage/lib/psi4/core.cpython-{ext_will_vary}.so ../psi4/core.cpython-{ext_will_vary}.so
>>> python ../psi4/run_psi4.py --inplace input.dat

Why not to set PSIDATADIR

PSIDATADIR is an environment variable containing the location of the text resource parts of the PSI4 codebase (e.g., basis sets, databases, EFP fragments). It is for developer use only. In PSI4 1.1 and beyond, the program always knows where its resources are, and the only reason to set this variable is to point to another location. Previously in PSI4 1.0 and previous, only installed executables knew the location, so it always needed to be explicitly set when run from the compilation directory.

At runtime

>>> psi4 -p {top-level-psi4-dir}/psi4/share/psi4

Or in the shell

# csh, tcsh: add to shell or ~/.tcshrc file
setenv PSIDATADIR {top-level-psi4-dir}/psi4/share/psi4

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
export PSIDATADIR={top-level-psi4-dir}/psi4/share/psi4

How to configure C++ and C compilers for building Psi4

Role and Dependencies

  • Role — In PSI4, a C++ compiler is vital for building the code.

  • Downstream Dependencies — PSI4 \(\Leftarrow\) C++ Compiler

CMake Variables

  • CMAKE_CXX_COMPILER — CMake variable to specify name or full path to C++ compiler.

  • CMAKE_C_COMPILER — CMake variable to specify name or full path to C compiler.

  • CMAKE_CXX_FLAGS — CMake variable to specify any additional custom compiler flags for C++ source.

  • CMAKE_C_FLAGS — CMake variable to specify any additional custom compiler flags for C source.

Examples

  1. Build with detected compilers from PATH

>>> cmake

  1. Build with specific (Intel) compilers from PATH

>>> cmake -DCMAKE_CXX_COMPILER=icpc -DCMAKE_C_COMPILER=icc

  1. Build with specific (GNU) compilers not in PATH

>>> cmake -DCMAKE_CXX_COMPILER=/path/to/gcc6.2/bin/g++ -DCMAKE_C_COMPILER=/path/to/gcc6.2/bin/gcc

  1. Build with specific (Intel) compilers from PATH based on GCC not in PATH

>>> cmake -DCMAKE_C_COMPILER=icc \
          -DCMAKE_CXX_COMPILER=icpc \
          -DCMAKE_C_FLAGS="-gcc-name=${GCC5}/bin/gcc" \
          -DCMAKE_CXX_FLAGS="-gcc-name=${GCC5}/bin/gcc -gxx-name=${GCC5}/bin/g++"

  1. Build with specific (Intel) compilers from PATH based on GCC not in PATH and also building Fortran Add-Ons

>>> cmake -DCMAKE_C_COMPILER=icc \
          -DCMAKE_CXX_COMPILER=icpc \
          -DCMAKE_Fortran_COMPILER=ifort \
          -DCMAKE_C_FLAGS="-gcc-name=${GCC5}/bin/gcc" \
          -DCMAKE_CXX_FLAGS="-gcc-name=${GCC5}/bin/gcc -gxx-name=${GCC5}/bin/g++" \
          -DCMAKE_Fortran_FLAGS="-gcc-name=${GCC5}/bin/gcc -gxx-name=${GCC5}/bin/g++"

  1. Build with specific (Intel) compilers from PATH based on GCC with prefix and not in PATH (GCCPFX=/full/path/to/bin/prefix- compiler is $GCCPFX-gcc)

>>> cmake -DCMAKE_C_COMPILER=icc \
          -DCMAKE_CXX_COMPILER=icpc \
          -DCMAKE_C_FLAGS="-gnu-prefix=${GCCPFX}" \
          -DCMAKE_CXX_FLAGS="-gnu-prefix=${GCCPFX}"

  1. Build on Linux with specific (Intel) compilers from PATH based on GCC from conda in activated environment (CONDA_PREFIX and HOST are defined upon activation)

>>> cmake -DCMAKE_C_COMPILER=icc \
          -DCMAKE_CXX_COMPILER=icpc \
          -DCMAKE_C_FLAGS="-gnu-prefix=${CONDA_PREFIX}/bin/${HOST} --sysroot=${CONDA_PREFIX}/${HOST}/sysroot" \
          -DCMAKE_CXX_FLAGS="-gnu-prefix=${CONDA_PREFIX}/bin/${HOST} --sysroot=${CONDA_PREFIX}/${HOST}/sysroot"

  1. Build on Linux with specific (GCC) compilers from from conda in activated environment (CONDA_PREFIX and HOST are defined upon activation)

>>> cmake -DCMAKE_C_COMPILER=${GCC} \
          -DCMAKE_CXX_COMPILER=${GXX} \
          -DCMAKE_Fortran_COMPILER=${GFORTRAN}

What C and C++ compilers and versions are approved

On Linux, the following work nicely.

  • GNU: gcc, g++

  • Intel: icc, icpc

  • Clang: clang, clang++

On Mac, the following work nicely.

  • Apple Clang: clang, clang++

  • Intel: icc, icpc

PSI4 requires full C++11 compliance, meaning, most importantly, GCC >= 4.9. This compliance is checked for at build-time with file psi4/cmake/custom_cxxstandard.cmake, so either consult that file or try a test build to ensure your compiler is approved. Note that Intel compilers on Linux also rely on GCC, so both icpc and gcc versions are checked. Intel OneAPI Classic compilers work fine. OneAPI beta compilers build but have been only minimally tested.

How to obtain C and C++ compilers for Mac without Fink, MacPorts, or Homebrew

The easiest compiler to obtain is clang which is a drop-in replacement for gcc and g++. Just install XCode. Some old versions of XCode can’t handle some of the advanced C++ language features, but this is a software not hardware limitation. Checks for version compliance performed at build-time. Note that this “AppleClang” will not be compatible with conda Mac packages using C++11, nor can it make use of OpenMP directives.

Another route to obtaining clang compilers without the above limitations is through conda.

# Install Clang 4.0.1 into a non-primary conda environment
>>> conda create -n clang401 clangxx_osx-64 clang_osx-64 llvm-openmp intel-openmp

# To Build, activate environment (prepends PATH and defines environment variables CLANG, CLANGXX, HOST, etc):
>>> conda activate clang401
>>> echo ${CLANGXX}
/path/to/miniconda/envs/clang401/bin/x86_64-apple-darwin13.4.0-clang++
>>> echo ${HOST}
x86_64-apple-darwin13.4.0

# build with Clang
>>> cmake -S. -Bbuild \
     -DCMAKE_C_COMPILER=${CLANG} \
     -DCMAKE_CXX_COMPILER=${CLANGXX} \
     -DCMAKE_CXX_FLAGS="-stdlib=libc++" \
     -DOpenMP_CXX_FLAG="-fopenmp=libiomp5"

# build with Intel
>>> cmake -S. -Bbuild \
     -DCMAKE_C_COMPILER=icc \
     -DCMAKE_CXX_COMPILER=icpc \
     -DCMAKE_C_FLAGS="-clang-name=${CLANG}" \
     -DCMAKE_CXX_FLAGS="-clang-name=${CLANG} -clangxx-name=${CLANGXX} -stdlib=libc++ -I${CONDA_PREFIX}/include/c++/v1"

# Configure and build

How to satisfy the GCC >= 4.9 requirement on Linux without updating the OS

# See if GCC too old (in this case, yes)
>>> gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4)

Even if you’re not using GCC as a compiler, your Intel compiler seeks gcc to generate code compatible with your current GCC version. If your GCC is too old (like above), you can update your system GCC through your package manager or install an acceptable version elsewhere on your system. The latter route, tested on Linux with Intel compilers, is below.

# Install GCC 7.2 into a non-primary conda environment
>>> conda create -n gcc72 gxx_linux-64 gcc_linux-64

# To Build, either:

# (A) activate environment (prepends PATH and defines environment variables CC, CXX, HOST, etc)
      >>> conda activate gcc72
      >>> echo ${CXX}
      /path/to/miniconda/envs/gcc72/bin/x86_64-conda_cos6-linux-gnu-g++
      >>> echo ${HOST}
      x86_64-conda_cos6-linux-gnu

      # build with GNU
      >>> cmake -S. -Bbuild \
           -DCMAKE_C_COMPILER=${CC} \
           -DCMAKE_CXX_COMPILER=${CXX} \

      # build with Intel
      >>> cmake -S. -Bbuild \
           -DCMAKE_C_COMPILER=icc \
           -DCMAKE_CXX_COMPILER=icpc \
           -DCMAKE_C_FLAGS="-gnu-prefix=${HOST}-" \
           -DCMAKE_CXX_FLAGS="-gnu-prefix=${HOST}-" \

# (B) tell CMake to tell the compiler which GCC to use
      >>> GCC7=/path/to/miniconda/envs/gcc72
      >>> cmake -S. -Bbuild \
           -DCMAKE_C_COMPILER=icc \
           -DCMAKE_CXX_COMPILER=icpc \
           -DCMAKE_C_FLAGS="-gnu-prefix=${GCC7}/bin/x86_64-conda_cos6-linux-gnu-" \
           -DCMAKE_CXX_FLAGS="-gnu-prefix=${GCC7}/bin/x86_64-conda_cos6-linux-gnu-" \
           ...
           # if Fortran active ...
           -DCMAKE_Fortran_COMPILER=ifort \
           -DCMAKE_Fortran_FLAGS="-gnu-prefix=${GCC7}/bin/x86_64-conda_cos6-linux-gnu-" \

# Configure and build

# To Run:
>>> export LD_LIBRARY_PATH=${GCC7}/lib:$LD_LIBRARY_PATH

How to configure a Psi4 build on Cray

Cray systems strongly prefer to build static libraries, but PSI4 needs to be dynamic to function as a Python module. Courtesy of @misha at the forum and various supercomputer guides, building PSI4 on Cray requires setting environment variables before running cmake.

CRAYPE_LINK_TYPE=dynamic CRAY_ADD_RPATH=yes cmake ...

Set to allow PSI4 to build as shared library on Cray supercomputers.

CRAY_ADD_RPATH

Set to true to allow PSI4 to build on Cray supercomputers.

How to configure Fortran compilers for building Psi4

Role and Dependencies

  • Role — In PSI4, a Fortran compiler in unneeded for core features but may be required for add-ons.

  • Downstream Dependencies

    • PSI4 (\(\Leftarrow\) optional) Fortran Compiler

    • erd, dkh, gdma, PCMSolver \(\Leftarrow\) Fortran Compiler

CMake Variables

  • CMAKE_Fortran_COMPILER — CMake variable to specify name or full path to Fortran compiler.

  • CMAKE_Fortran_FLAGS — CMake variable to specify any additional custom compiler flags for Fortran source.

Examples

  1. Build with detected compiler from PATH

>>> cmake

  1. Build with specific (Intel) compiler from PATH

>>> cmake -DCMAKE_Fortran_COMPILER=ifort

What Fortran compilers are approved

On Linux and Mac, the following work nicely.

  • GNU: gfortran

  • Intel: ifort

  • Packages to install for specific OS or package managers:

    • Ubuntu gfortran

    • conda gfortran_linux-64 or gfortran_osx-64 to get gfortran

How to obtain a Fortran compiler for Mac without Fink, MacPorts, or Homebrew

Xcode does not provide a Fortran compiler. A way to get one is to download the gfortran_osx-64 conda package. This provides gfortran compilers for Mac. The version is 4.8.5, which is quite old, but the Fortran compiler will work.

How to configure BLAS/LAPACK for building Psi4

Role and Dependencies

  • Role — In PSI4, BLAS and LAPACK control much of the speed and efficiency of the code since computational chemistry is essentially linear algebra on molecular systems.

  • Downstream Dependencies — PSI4 \(\Leftarrow\) LAPACK Libraries

CMake Variables

  • BLAS_TYPE — CMake variable to specify which BLAS libraries to look for among MKL|OPENBLAS|ESSL|ATLAS|ACML|SYSTEM_NATIVE.

  • LAPACK_TYPE — CMake variable to specify which LAPACK libraries to look for among MKL|OPENBLAS|ESSL|ATLAS|ACML|SYSTEM_NATIVE.

  • LAPACK_LIBRARIES — CMake variable to specify BLAS/LAPACK libraries explicitly, bypassing math detection. Should be “;”-separated list of full paths.

  • LAPACK_INCLUDE_DIRS — CMake variable to specify BLAS/LAPACK header location explicitly, bypassing math detection. Only needed for MKL.

  • OpenMP_LIBRARY_DIRS — CMake variable to specify OpenMP library (iomp5/gomp/omp) directories explicitly. Should be “;”-separated list of full directory paths. Usually the solution to error Could NOT find MathOpenMP.

MKL_ROOT

Environment variable set by Intel compilervars scripts. Sufficient to trigger math detection of MKL at this location.

MATH_ROOT

Environment variable to specify root directory in which BLAS/LAPACK libraries should be detected (e.g., ${MATH_ROOT}/lib64/libblas.so and ${MATH_ROOT}/lib64/liblapack.so).

Examples

  1. Build with any LAPACK in standard location

>>> cmake

  1. Build with native Accelerate LAPACK on Mac (MKL not also present). If NumPy not using native Accelerate LAPACK, then directing Psi4 to use it is Bad Idea!

>>> cmake

  1. Build with native Accelerate LAPACK on Mac (MKL also present) If NumPy not using native Accelerate LAPACK, then directing Psi4 to use it is Bad Idea!

>>> cmake -DBLAS_TYPE=SYSTEM_NATIVE -DLAPACK_TYPE=SYSTEM_NATIVE

  1. Build with Intel MKL

>>> source /path/to/intel/vers/linux/mkl/bin/mklvars.sh intel64  # adjust sh/csh and arch as needed
>>> cmake

>>> MATH_ROOT=/path/to/intel/vers/linux/mkl/ cmake

  1. Build with Intel MKL from conda (install mkl-devel package from defaults channel)

>>> cmake -DLAPACK_LIBRARIES="${CONDA_PREFIX}/lib/libmkl_rt.so" -DLAPACK_INCLUDE_DIRS="${CONDA_PREFIX}/include"

  1. OpenBLAS - see note below.

>>> MATH_ROOT=/path/to/openblas/0.2.13_seq/x86_64/gcc_5.2.0/lib cmake

  1. Build with explicit MKL LAPACK

>>> cmake -DLAPACK_LIBRARIES="/path/to/lib/intel64/libmkl_lapack95_lp64.a;/path/to/lib/intel64/libmkl_rt.so" -DLAPACK_INCLUDE_DIRS="/path/to/mkl-h-include/"

  1. Build with explicit non-MKL LAPACK

>>> cmake -DLAPACK_LIBRARIES="/path/to/lib/liblapack.so;/path/to/lib/libblas.a"

  1. Build with MKL and GCC (iomp5 needed instead of gomp for threading. use OpenMP_LIBRARY_DIRS to hint location.)

>>> cmake -DLAPACK_LIBRARIES=/opt/intel/mkl/lib/intel64/libmkl_rt.so -DLAPACK_INCLUDE_DIRS=/opt/intel/mkl/include -DOpenMP_LIBRARY_DIRS=/opt/intel/compiler/lib/intel64/

Notes

  • Much of PSI4‘s speed and efficiency depends on the corresponding speed and efficiency of the linked BLAS and LAPACK libraries (especially the former). Consider the following recommendations:

    • It is NOT wise to use the stock BLAS library provided with many Linux distributions like RedHat, as it is usually just the completely unoptimized netlib distribution. The choice of LAPACK is less critical, and so the unoptimized netlib distribution is acceptable.

    • Perhaps the best choice, if available, is Intel’s MKL library, which includes efficient threaded BLAS and LAPACK (as of PSI4 v1.1, earliest known working version is MKL 2013; as of v1.4, should use at leat MKL 2019). MKL, which is freely available through conda, is the only threaded BLAS/LAPACK distribution fully supported by PSI4.

    • On Mac, the native Accelerate libraries are very nice and would be recommended but for the potential conflict between PSI4 BLAS and NumPy BLAS. Unless you’ve a special NumPy, avoid!

    • The open-source LAPACK distributions OpenBLAS (formerly GotoBLAS) mostly works. Use it at your own risk and after testing your particular distribution, including tests run multithreaded, if you intend to run PSI4 so. Use at least 0.2.15, and pay attention to how it was compiled - unthreaded seems safe, openmp-threaded is mostly safe, default pthreaded is not safe. See https://github.com/psi4/psi4/issues/1009 for recent analysis.

    • Another open-source LAPACK distribution, ATLAS had stability issues with the DFOCC module at last testing, https://github.com/psi4/psi4/issues/391.

    • ACML libraries are known to work with PSI4 v1.1 at ACML 6.

  • Because of how link loaders work, at runtime, the BLAS of PSI4 and the BLAS of NumPy are not independent. There can be unpredictable but reproducible numerical and thread-scaling errors if PSI4 and NumPy BLAS don’t match down to the library name (that is, libmkl_rt, libmkl_core.so, libmkl_core.a are not interchangeable). See https://github.com/psi4/psi4/issues/1007, https://github.com/psi4/psi4/issues/748, https://github.com/psi4/psi4/issues/755 for gory discussions. Choose your NumPy and PSI4 compile conditions to use the same BLAS distribution.

  • The BLAS/LAPACK detected for PSI4 are also linked into any Add-Ons (e.g., libefp) that require them, rather than relying on those packages’ native math detection.

  • The separation between BLAS and LAPACK seen in detection printing and CMake variables is purely formal. In practice, they get run together and linked as ${LAPACK_LIBRARIES} ${BLAS_LIBRARIES}.

  • Sometimes the CMake’s library search capabilites falter at SONAMEs (e.g., libblas.so.3 vs. libblas.so), extensions (static vs. dynamic), or suffixes (e.g., libacml_mp.so vs. libacml.so). The developers would be interested in hearing of such problems to expand the math detection capabilities. The immediate solution, however, is to form symlinks between the library names that exist and the names expected. Consult file psi4/cmake/math/MathLibs.cmake for the library patterns being sought.

  • The BLAS/LAPACK interface is standardized, so only libraries, not headers, need to be detected. The exception is MKL, where the mkl.h header defines additional functionality; it must be located to use BLAS threading.

How to configure Python for building Psi4

Role and Dependencies

  • Role — In PSI4, Python allows the core compiled C++ code to be flexibly accessed for manipulation and extension in an interpreted language.

  • Downstream Dependencies — PSI4 \(\Leftarrow\) Python Interpreter

CMake Variables

Note that in v1.4, PSI4 switched from Numpy-written CMake Python detection to new CMake-written CMake Python detection. Usually only Python_EXECUTABLE is now needed to hint Python’s location. See https://cmake.org/cmake/help/latest/module/FindPython.html for full guidance.

  • Python_EXECUTABLE — specify name or full path to Python interpreter.

  • Python_LIBRARY — specify path to Python library.

  • Python_INCLUDE_DIR — specify directory of Python headers. Contains Python.h.

Examples

  1. Build with detected Python from PATH

>>> cmake

  1. Build with specific Python

>>> cmake -DPython_EXECUTABLE=/path/to/interp/python3.9

  1. Build with full Python specification to root directory ${PFXC}

>>> cmake -DPython_EXECUTABLE="${PFXC}/bin/python" \
          -DPython_LIBRARY="${PFXC}/lib/libpython3.9.so" \
          -DPython_INCLUDE_DIR="${PFXC}/include/python3.9"

What Python is Psi4 running

The Python detected at build-time is embedded into the PSI4 executable. That is, the top line of bin/psi4 is something like #!/path/to/miniconda/envs/p4deps/bin/python3.5, and that’s the Python through which PSI4 is running, rather than the Python of which python. To use a different Python with PSI4 in the short term, just path/to/desired/python psi4 on the command line to override the shebang line. To use a different Python with PSI4 in the long term, edit the shebang line.

If you’re using PSI4 as a Python module, then PSI4 is running the Python of which python.

How to fix “undefined symbol: _Py_FalseStruct”

You’re probably loading a Py3-compiled Psi4 in Py2. Switch interpreters and re-run. A python of proper Py2 or Py3-ness is baked into the PSI4 “executable”, so you’ll see this error only for Psi4 as Python module.

How to use gdb and lldb with Psi4

Debugging PSI4 has gotten a little confusing now that it’s running through Python. Here’s the syntax

>>> cd {objdir}
>>> lldb -- python stage/bin/psi4 ../tests/tu1-h2o-energy/input.dat
>>> (lldb) run

>>> cd {objdir}
>>> gdb --args python stage/bin/psi4 ../tests/tu1-h2o-energy/input.dat
>>> (gdb) run

How to see the actual compiling commands (or errors) with cmake

CMake by default hides a lot of useful debugging information to make the compilation cleaner. Issue make VERBOSE=1 to display the full compilation commands and errors.

How to highlight git merge conflicts in vi

Edit your ~/.vimrc file to include the lines below. Hitting the F7 key will toggle highlighting of git’s conflict markers.

>>> cat ~/.vimrc
set hlsearch
map <F7> :/\(<<<<<<<\\|=======\\|>>>>>>>\)<CR>

How to handle “runtime library may be hidden” when building with Anaconda Python

When building against Ana/Miniconda python (e.g., cmake -DPython_EXECUTABLE=/path/to/conda/bin/python), the warning below often appears. It is harmless, proceed.

CMake Warning at src/bin/psi4/CMakeLists.txt:58 (add_executable):
  Cannot generate a safe runtime search path for target psi4 because files in
  some directories may conflict with libraries in implicit directories:

    runtime library [libm.so.6] in /usr/lib64 may be hidden by files in:
      /theoryfs2/common/software/anaconda/lib

Some of these libraries may not be found correctly.

How to set up the scratch directory

The scratch directory is where Psi4 stores potentially large files during computation. It should thus be on a local, fast disk to minimize any computational inefficiencies caused by I/O. The scratch directory is commonly set up through the PSI_SCRATCH environment variable:

# csh, tcsh: add to shell or ~/.tcshrc file
setenv PSI_SCRATCH /path/to/existing/writable/local-not-network/directory/for/scratch/files

# sh, bash: add to shell or ~/.bashrc (Linux/Windows) or ~/.bash_profile (Mac) file
export PSI_SCRATCH=/path/to/existing/writable/local-not-network/directory/for/scratch/files

See also the more general scratch documentation.

How do I retain specific Psi4 scratch files

You can set up a specific path for PSI4 scratch file and keep them for later use through the psi4_io handler.

How to use Psi4 within a PBS queue

You will usually need to set up a PBS job script that is setting all necessary environment variables, making sure the scratch directories are set up, and invoking the executable. An example PBS script is provided in the manual, but make sure to also consult your own PBS documentation for appropriate setup.

How to update and rebuild Psi4

Obtain code updates as appropriate from Binary Installer, Clone from GitHub Repository, or Fork from GitHub Repository. Move into objdir and reissue make or cmake --build ., whereupon CMake may reconfigure but will only rebuild objects and libraries depending on changed files. It is scarcely ever necessary for the user to reinvoke cmake to update objdir based on changes to the PSI4 repository. Upon switching conda environments, though, one should make a new objdir since there are many full paths baked into objdir/CMakeCache.txt.

How to run a minute’s worth of tests

When you want to do a very minimal test of the build and have CTest installed, the following command can be useful.

>>> ctest -L smoke -j`getconf _NPROCESSORS_ONLN`

If you have pytest installed, very similar coverage is obtained through:

>>> make pytest

How to run a subset of tests

CTest allows flexibly partitioned running of the test suite. In the examples below, testname are regex of test names, and testlabel are regex of labels (e.g., cc, mints, libefp defined [here, for example] .

  • Run tests in parallel with -j flag. For maximum parallelism (not Windows): ctest -j`getconf _NPROCESSORS_ONLN`

  • Run full test suite: ctest

  • Run about a third of the tests in 5 minutes, the so-called quicktests: ctest -L quick

  • Run the same subset of tests that continuous integration checks (not the full test suite): ctest -L quick

  • Run the minimal number of tests to ensure Psi4 and any add-ons in working order: ctest -L smoke

  • Run tests matching by name: ctest -R testname

  • Run tests excluding those by name: ctest -E testname

  • Run tests matching by label: ctest -L testlabel

  • Run tests excluding those by label: ctest -LE testlabel

Pytest has similar commands:

  • Run tests in parallel with -n flag (if Python package pytest-xdist installed). For maximum parallelism: pytest -n auto

  • Run full test suite: pytest

  • Run the quick tests: pytest -m quick

  • Run the minimal number of tests to ensure Psi4 and any add-ons in working order: pytest -m smoke

  • Run tests matching by name: pytest -k testname

  • Run tests excluding those by name: pytest -k "not testname"

  • Run tests matching by label: pytest -m testlabel

  • Run tests excluding those by label: pytest -m "not testlabel"

  • Run tests with complicated filtering by name: pytest -k "cc and not cc4 and not stdsuite"

  • Run tests and see test names: pytest -v

  • Run tests and see test names and sometimes more verbose errors: pytest -vv

How to see CTest testing errors

>>> ctest
Test project /your/path/2/psi4/build/directory/tests
    Start 248: tu1-h2o-energy
1/2 Test #248: tu1-h2o-energy ...................   Passed    1.73 sec
     Start  6: cc1
2/2  Test  #6: cc1 ..............................***Failed    0.07 sec
...

When ctest reports that some (or all) tests have failed, look in your build directory for file objdir/Testing/Temporary/LastTest.log. It may have a .tmp extension, depending on whether the last test was interrupted and a few other factors. Either way, this file should contain CMake’s testing output, as well as everything that was printed to the screen.

How to test a Psi4 installation

ctest requires a connection to source files and cmake machinery and so can only be performed from objdir (staged installation). To test an installed PSI4 (full or staged installation), a supplementary test suite is available to be run via pytest. Running a limited number of “smoke” tests is generally sufficient to verify an installation.

  • From the executable

    # smoke tests only in serial
psi4 --test
# smoke tests only in parallel
psi4 --test -n`getconf _NPROCESSORS_ONLN`
# most tests
psi4 --test full

  • From the library (PSI4 must be detectable as a Python module. See setup at How to configure paths for PsiAPI if needed.)

    # smoke tests only
python -c "import psi4; psi4.test('smoke')"
# most tests
python -c "import psi4; psi4.test()"

  • From pytest directly. If package pytest-xdist is installed, can run in parallel.

    # smoke tests only
pytest {prefix}/lib/{PYMOD_INSTALL_LIBDIR}/psi4/tests/ -m smoke -n auto
# all tests
pytest {prefix}/lib/{PYMOD_INSTALL_LIBDIR}/psi4/tests/ -n auto

Output looks something like the below. PASSED in green is good (means test ran correctly); SKIPPED in yellow is good (means that not all software required for test is available); XPASS or XFAIL in yellow is fine (unexpected pass or expected fail happens when we include tests that need particular conditions (e.g., multiple cores) to run correctly); FAILED in red is bad.

test_addons.py::test_gdma PASSED
test_addons.py::test_mrcc SKIPPED
test_addons.py::test_chemps2 PASSED
test_addons.py::test_dftd3 PASSED
test_addons.py::test_libefp PASSED
test_addons.py::test_pcmsolver PASSED
test_addons.py::test_erd PASSED
test_addons.py::test_simint PASSED
test_addons.py::test_json PASSED
test_addons.py::test_cfour SKIPPED
test_addons.py::test_v2rdm_casscf PASSED
test_addons.py::test_grimme_3c PASSED
test_addons.py::test_dkh PASSED
test_psi4.py::test_psi4_basic PASSED
test_psi4.py::test_psi4_cc PASSED
test_psi4.py::test_psi4_cas PASSED
test_psi4.py::test_psi4_dfmp2 PASSED
test_psi4.py::test_psi4_sapt PASSED
test_psi4.py::test_psi4_scfproperty PASSED

How to refer to Psi4

Ways to refer to PSI4 in text, in order of decreasing goodness:

  • as Psi4 in Optima regular font with “si” in custom (82%) small caps according to psi4/media/README.md.

    • html: <span style="font-family: Optima, sans-serif; color: #273896;">P<span style="font-size: 82%;">SI</span>4</span>

  • as Psi4 with “si” in generated small caps

    • html: <span style="font-variant: small-caps;">Psi4</span>

  • as Psi4 with “si” in lowercase

  • as psi4 in code

  • NOT PSI4 or PSI

How to get a Psi4 logo file

All image files are stored in https://github.com/psi4/psi4media

How to use a local Add-On repository in the Psi4 build

For each Add-On, PSI4 pulls source from a specific online Git repository and a specific tag/branch/commit in it. This ensures success of the PSI4 build, reproducibility of the runtime results, and freedom for continued upstream development. Sometimes, you’re the one doing that development, and you need the CMake superbuild to pull source from a local path rather than the approved codeset.

Find the CMakeLists.txt governing the target Add-On in psi4/external and make changes analogous to the below:

#GIT_REPOSITORY https://github.com/jturney/ambit
#GIT_TAG 1.0
DOWNLOAD_COMMAND ""
SOURCE_DIR "/path/to/ambit-directclone"

If you’re changing the PSI4 repo codebase between compiles, there’s nothing more to do as CMake will handle the code rebuild deps for you.

If you’re changing the local Add-On repo codebase between compiles, CMake does not know when libaddon.[a|so|dylib] needs rebuilding. It is recommended that the PSI4 build be initially configured with -DBUILD_SHARED_LIBS=ON (easier to notice changes). And to trigger Add-On library rebuild, rm -rf {objdir}/external/upstream/addon/ and rm -rf {objdir}/stage/share/cmake/AddOn. This should re-clone the Add-On, rebuild and install it, rebuild any parts of PSI4 that interface to it, and relink the main core.so. If you’re modifying the Add-On’s file or directory structure, be smart and rm all traces of it within {objdir}/stage/, especially any *.pyc files.

Alternatively and preferentially to the above, you can instead build and install the Add-On library yourself, external to the PSI4 repository. This is especially useful if you want to avoid full recompiles of the Add-On at each change to the Add-On’s source. Build the Add-On library dynamically (-DBUILD_SHARED_LIBS=ON) and mind any “Psi4 wants” in the Add-On’s top-level CMakeLists.txt. Install the Add-On and note the full path to AddOnConfig.cmake. Pass the path containing that file to PSI4‘s CMake as -DAddon_DIR=/path/to/config/usually/ending/in/share/cmake/AddON and build PSI4. The main core.so should be dynamically linked to your dev AddOn dynamic lib and update automatically when you rebuild the AddOn lib. Naturally, you may need to delete core.so and remake as needed.

How to use logging in Psi4

A log file is generated alongside the output file and has extension .log. Selected key information is sent to it. The log file contents are provisional and haphazard, so don’t start relying upon them. It is tentatively envisioned that log files will have much of the same results as output files but more as structured data than narrative, nicely formatted data. In some cases where external calls are involved, the most detailed information may only be collectable in the log file. It is worthwhile to check the log file if expected results are missing from the usual output file. If the log file is empty, the logging level may be set too low.

To change the logging level to DEBUG from the command-line:

>>> psi4 --logging 10

To change the logging level to DEBUG in PsiAPI:

>>> psi4.set_output_file(<filebase>, loglevel=10)

How to find the Psi4 module from the executable and vice versa

Because of the different conventions of C++/Linux and Python installation layouts, the PSI4 executable and the PSI4 Python module aren’t at fixed locations relative to each other. Accessors have been added so that the executable can show where the module is, and the module can show where the executable is.

> /path/to/psi4/install/bin/psi4 --module
/path/to/psi4/install/lib//
> python -c "import psi4;print(psi4.executable)"
/path/to/psi4/install/bin/psi4