calibr

Parallelized Bayesian calibration of simulations using Gaussian process emulation.

calibr is a Python implementation of the algorithm described in Parallel Gaussian process surrogate Bayesian inference with noisy likelihood evaluations (Järvenpää, Gutmann, Vehtari and Marttinen; 2021) (doi:10.1214/20-BA1200, arxiv:1905.01252). It is designed to allow estimation of the posterior distribution on the unknown parameters of expensive to evaluate simulator models given observed data, using a batch sequential design strategy which iterates fitting a Gaussian process emulator to a set of evaluations of the (unnormalized) posterior density for the model and using the emulator to identify a new batch of model parameters at which to evaluate the posterior density which minimize a measure of the expected uncertainty in the emulation of the posterior density.

The posterior density can be evaluated at the parameter values in each batch in parallel, providing the opportunity for speeding up calibration runs on multi-core and multi-node high performance computing systems. The acquisition functions used to choose new parameter values to evaluate are implemented using the high-performance numerical computing framework JAX, with the gradient-based optimization of these acquisition functions exploiting JAX's support for automatic differentiation.

The package is still in the early stages of development, with only a subset of the algorithmic variants proposed by Järvenpää, Gutmann, Vehtari and Marttinen (2021) currently implemented. In particular there is no support yet for models with noisy likelihood evaluations. Expect lots of rough edges!

This project is developed in collaboration with the Centre for Advanced Research Computing, University College London.

Getting started

Prerequisites

calibr requires Python 3.10–3.12.

Installation

We recommend installing in a project specific virtual environment created using a environment management tool such as Mamba or Conda. To install the latest development version of calibr using pip in the currently active environment run

pip install git+https://github.com/UCL/calibr.git

Alternatively create a local clone of the repository with

git clone https://github.com/UCL/calibr.git

and then install in editable mode by running

pip install -e .

Documentation

Documentation for the package is available at https://github-pages.ucl.ac.uk/calibr/.

Running tests

Tests can be run across all compatible Python versions in isolated environments using tox by running

tox

To run tests manually in a Python environment with pytest installed run

pytest tests

again from the root of the repository.

Building documentation

The MkDocs HTML documentation can be built locally by running

tox -e docs

from the root of the repository. The built documentation will be written to site.

Alternatively to build and preview the documentation locally, in a Python environment with the optional docs dependencies installed, run

mkdocs serve

Acknowledgements

This work was funded by a grant from the ExCALIBUR programme.