Source-linked AI summary
Approximate Bayesian computation (ABC) gives exact results under the assumption of model error
Richard D. Wilkinson
TL;DR
ABC is popular, but little is known about the quality of its approximations beyond simulation studies. This paper frames ABC as inference under model error and shows exactness under that assumption, while noting that summary sufficiency remains unresolved.
Problem
Little is known about the quality of ABC approximations beyond results shown in simulation studies.
Method
The paper interprets ABC using a distance measure on model outputs and a tolerance parameter to calibrate the model.
Results
ABC algorithms give exact inference under the assumption of model error, with Algorithm B drawing from the posterior distribution π(ˆθ | D).
Takeaways & Limitations
ABC can be understood as exact inference under an explicit model-error assumption.
Takeaways & Limitations
It cannot be known whether the summaries used are sufficient for the output.
Abstract
from arXiv · showhide
Approximate Bayesian computation (ABC) or likelihood-free inference algorithms are used to find approximations to posterior distributions without making explicit use of the likelihood function, depending instead on simulation of sample data sets from the model. In this paper we show that under the assumption of the existence of a uniform additive model error term, ABC algorithms give exact results when sufficient summaries are used. This interpretation allows the approximation made in many previous application papers to be understood, and should guide the choice of metric and tolerance in future work. ABC algorithms can be generalized by replacing the 0-1 cut-off with an acceptance probability that varies with the distance of the simulated data from the observed data. The acceptance density gives the distribution of the error term, enabling the uniform error usually used to be replaced by a general distribution. This generalization can also be applied to approximate Markov chain Monte Carlo algorithms. In light of this work, ABC algorithms can be seen as calibration techniques for implicit stochastic models, inferring parameter values in light of the computer model, data, prior beliefs about the parameter values, and any measurement or model errors.
1 Introduction
ABC avoids explicit likelihood evaluation by using simulations, but its approximation quality has been poorly understood. The paper provides a model-error framework that interprets ABC as exact inference under specified errors and informs metric, tolerance, and weighting choices.
- Problem: The quality of ABC’s approximation has been known mainly through simulation studies, leaving the effects of methodological choices unclear.This includes choices of distance metric, tolerance, weighting, and summary statistic.
- Motivation: ABC methods perform inference using simulations from implicit computer models without evaluating the likelihood function.They are designed for models whose likelihoods are completely intractable.
- Contribution: The paper shows that ABC rejection sampling can be interpreted as exact inference for a model with uniform additive model or measurement error.Thus, the algorithm targets the posterior of an error-augmented model rather than necessarily the original model.
- Contribution: This framework specifies the model-error distribution induced by a chosen metric and tolerance, helping explain prior applications and guide future choices.The stated guidance covers metrics and weightings used in ABC.
- Extensions: ABC can be generalized from hard rejection to probabilistic acceptance, yielding inference under a completely flexible model-error form.The extension also applies to approximate Markov chain Monte Carlo and supports methods for Bayes factors and integrals.
- Scope: The paper distinguishes accounting for known model inadequacies in inference from using ABC to diagnose those errors.Its aim is to produce posteriors that incorporate known inadequacies and clarify standard ABC’s effects.
2 Interpreting ABC
The paper frames ABC as inference under an explicit model-error distribution, showing how metric, tolerance, and weighting choices determine the implied approximation. A probabilistic acceptance rule generalizes this framework and can produce exact posterior draws under the stated model-error assumption.
- ABC accuracy can be understood by treating the distance and tolerance as specifying an implicit model-error assumption.The framework is intended to clarify the effects of metric, tolerance, and weighting choices.
- Accounting for model error is presented as necessary when linking stochastic models to reality, even when model variability can explain the observed data.
- The model links observed data to the simulator at a best input value plus measurement or model error.The error may represent measurement error, model error, or both.
- Algorithm B accepts simulated outputs probabilistically according to an error-density-based rule rather than a binary cutoff.The normalization constant is chosen so the acceptance probability is valid and, when appropriate, efficient.
- Under D = η(θ̂)+ε with independent ε distributed according to πε, Algorithm B gives draws from the posterior of the best input θ̂.The accepted values have the same distribution as this posterior.
- Uniform-error ABC and Gaussian-error approximations can differ substantially at large tolerances but coincide as the tolerance approaches zero.In the toy example, the Gaussian variance is chosen to match the variance of the uniform error.
3 Model discrepancy
This section interprets ABC as inference under an explicit model-error term, showing how acceptance rules encode error assumptions and how these assumptions affect exactness and practical accuracy.
- Error interpretation: Algorithm A gives exact inference for a model assuming uniform measurement error, whereas Algorithm B can represent a broader error distribution.Algorithm B uses an acceptance probability linked to the error density rather than a binary cutoff.
- Error interpretation: ABC rejection with a 0-1 cutoff corresponds to assuming uniform measurement error over the region defined by the metric and tolerance.The cutoff accepts simulated data when ρ(D,X) ≤ δ.
- Error interpretation: The error term can represent measurement error or model discrepancy, but model discrepancy is harder to conceptualize and less commonly used.Measurement-error distributions may be specified or assigned unknown parameters inferred jointly with model parameters.
- Exact inference: Rewriting a stochastic model to expose its sampling process can allow ABC to produce exact inference without assuming model error.This requires the sampling process to have a known distribution and computational constraints to permit the approach.
- Example: In the fossil-record example, Algorithm B was closer to the no-measurement-error posterior and used a measurement-error variance of δ 2/5.The cited comparison reports Algorithm B as more accurate than Algorithm A for that model.
4 Approximate Markov chain Monte Carlo
This section generalizes approximate MCMC to probabilistic acceptance under explicit error densities and presents two chains whose convergence follows from detailed balance.
- Motivation: Approximate MCMC replaces likelihood evaluation with simulated model output, a distance metric, and a 0-1 cutoff.The cutoff interpretation is equivalent to assuming uniform error on a metric-tolerance region.
- Generalization: The approximate MCMC construction can be generalized from uniform measurement error to an arbitrary error term.This replaces binary acceptance with a probability determined by the error model.
- Algorithms: Algorithm C constructs a chain on parameter space, while Algorithm D introduces simulated output as an auxiliary variable and chains on Θ×X.Algorithm D’s state contains both θ and the simulated model output X.
- Convergence: Both algorithms satisfy detailed balance, providing the required stationary posterior distribution.The paper establishes detailed balance separately for Algorithms C and D.
- Efficiency: Algorithm D is likely more efficient in most cases because its acceptance ratio can yield larger probabilities and requires no normalizing constant.The comparison is attributed to the ratio of model-error densities in Algorithm D’s acceptance rate.
5 Extensions
The paper extends probabilistic ABC to importance sampling, sequential Monte Carlo, and model selection, while emphasizing that kernels and summaries become part of the inferential specification.
- 5.1 Importance sampling: Importance sampling estimates posterior expectations by weighting simulated parameter-output pairs according to the error density.With binary acceptance, the weights reduce to 1 for accepted proposals and 0 for rejected proposals.
- 5.1 Importance sampling: Sequential Monte Carlo ABC schemes that gradually reduce tolerance can be extended using general acceptance kernels.The extension also applies to variants using Metropolis-Hastings parameter moves.
- 5.1 Importance sampling: General acceptance probabilities can create memory pressure because many particles may retain small but nonzero weights.Partial rejection control is identified as a way to discard particles below a weight threshold.
- 5.2 Model selection: For model selection, estimated Bayes factors depend on the chosen error density and can be unstable as the tolerance varies.The acceptance kernel therefore forms part of the statistical model being assessed.
- 5.2 Model selection: Using summaries can change Bayes factors relative to full-data calculations, even when summaries are sufficient for each simulator’s parameters.The paper highlights that model-selection inference based on summaries requires particular care.
- 5.2 Model selection: Summary statistics should reflect aspects of the data that the simulator is expected to reproduce, rather than merely being informative about parameters.Phase-sensitive summaries can produce apparently informative but potentially unconvincing posteriors when the simulator was designed for phase-insensitive behavior.
6 Discussion
The discussion frames ABC as exact inference under model error, while stressing that summary-statistic reduction introduces an additional approximation whose effect generally remains unresolved.
- Main conclusion: ABC can be understood as exact inference under the assumption of model error.This reframes ABC from an approximation to the posterior as inference under an explicitly modeled error term.
- Summary statistics: The framework explains error associated with measuring summaries but does not determine the effect of replacing complete data with summaries.In most ABC applications, summaries reduce output dimension, yet their sufficiency for the data is generally unknown.
- Model discrepancy: Model discrepancy matters when inference aims to make statements about reality rather than only about simulator behavior.The discussion notes that stochastic-model discrepancy has received minimal modeling attention.
- Model discrepancy: Stochastic-model discrepancy can be represented by treating the model as deterministic and interpreting ε as the difference between model output density and physical variability.Under this construction, the model outputs πθ(x) for each θ.