Source-linked AI summary
GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation
Edward Meeds, Max Welling
TL;DR
Likelihood-free inference with expensive simulators can require many simulations to estimate parameter posteriors. The paper develops ABC samplers that use acceptance-probability uncertainty to adapt simulation counts, with GPS-ABC reusing simulations through a Gaussian-process surrogate. The algorithms require fewer simulator calls than current state-of-the-art ABC methods, with a reported toy-problem comparison showing large savings.
Problem
Likelihood-free posterior inference is challenging because simulations are expensive and ABC methods require many simulator runs.
Source: §1 · p. 1Method
Results
The algorithms require many fewer simulator calls than current state-of-the-art ABC methods; on the toy problem, ξ = 0.05 required 1297 simulations versus over 500K for ASL-ABC.
Source: §5.1 · p. 10, §6 · p. 14Takeaways & Limitations
The reported comparisons indicate that approximate ABC inference can use substantially fewer simulator calls, including when similar accuracy is achieved with fewer simulations.
Source: §5.1 · p. 10, §6 · p. 14GPS-ABC is approximate: MH-step errors can produce error in the Markov chain’s stationary distribution.
Source: §4.1 · p. 8
Abstract
from arXiv · showhide
Scientists often express their understanding of the world through a computationally demanding simulation program. Analyzing the posterior distribution of the parameters given observations (the inverse problem) can be extremely challenging. The Approximate Bayesian Computation (ABC) framework is the standard statistical tool to handle these likelihood free problems, but they require a very large number of simulations. In this work we develop two new ABC sampling algorithms that significantly reduce the number of simulations necessary for posterior inference. Both algorithms use confidence estimates for the accept probability in the Metropolis Hastings step to adaptively choose the number of necessary simulations. Our GPS-ABC algorithm stores the information obtained from every simulation in a Gaussian process which acts as a surrogate function for the simulated statistics. Experiments on a challenging realistic biological problem illustrate the potential of these algorithms.
1 Introduction
Simulation-based modeling uses simulations to evaluate hypotheses about natural phenomena, but expensive simulators make inference over their parameters difficult. The paper develops ABC methods that use a Gaussian process surrogate and acceptance-probability uncertainty to reduce simulator calls.
1 Introduction
- Expensive simulations make choosing parameter settings for complex models computationally difficult.The parameters may interact and are often themselves scientifically interesting.Source: §1 · p. 1
- The inference problem is likelihood-free and computationally costly, yet scientists often want a distribution of accurate parameter settings rather than one optimum.Parameter interactions can provide insight into both the simulator and the underlying phenomena.Source: §1 · p. 1
- ABC uses the distance between simulated and observed data as a proxy for the likelihood, but can be inefficient even on simple problems.Source: §1 · p. 1
- The surrogate can support efficient approximate-posterior sampling by allowing high-confidence MH decisions without expensive simulations, at the cost of some bias.Source: §1 · p. 2
- GPS-ABC incorporates simulations into a Gaussian process surrogate and uses its uncertainty to decide whether more simulations are needed before an MH decision.When uncertainty is high and an error is likely, the method runs additional simulations.Source: §1 · p. 2
2 Approximate Bayesian Computation
ABC approximates likelihood-free inference by comparing simulated and observed data, with a tolerance controlling their acceptable discrepancy. The section introduces marginal and pseudo-marginal MCMC approaches and motivates adapting simulation effort to uncertainty in the acceptance probability.
2 Approximate Bayesian Computation
- ABC compares simulated pseudo-data with observations through a discrepancy distribution, producing an approximate posterior when the likelihood is unavailable.The tolerance ε controls the acceptable discrepancy.Source: §2 · p. 2
- Smaller ε improves the posterior approximation but raises the rejection rate, creating a trade-off between approximation bias and sampling efficiency.Source: §2 · p. 2
2.1 Marginal and Pseudo-Marginal ABC
- Marginal and pseudo-marginal ABC MCMC estimate the marginal likelihood from simulator samples at each proposed parameter setting.The pseudo-marginal method carries the denominator estimate forward from the previous iteration and converges to the approximate posterior because that estimate is unbiased.Source: §2.1 · pp. 2–3
- Pseudo-marginal sampling can mix slowly after a lucky draw yields a large likelihood estimate; re-estimating both terms each iteration may improve mixing but is more expensive and does not guarantee convergence unless S → ∞.Source: §2.1 · p. 3
- Finite-sample likelihood estimates make the MH acceptance probability random, and its uncertainty can determine the simulation count needed for a confident decision.The acceptance-probability distribution is wider for small S and approaches a point mass as S grows.Source: §2.1 · p. 3
- Sequential Monte Carlo is another established approach to likelihood-free ABC inference.Source: §2.1 · p. 3
3 The Synthetic Likelihood
The synthetic-likelihood approach models simulated data with a normal distribution, enabling a Gaussian likelihood approximation. The section then uses uncertainty in the random MH acceptance probability to adapt simulation effort, while noting the cost of discarding simulations between steps.
3 The Synthetic Likelihood
- Wood’s synthetic likelihood approximates simulated pseudo-data with a normal distribution using estimated first- and second-order statistics.Source: §3 · p. 3
- The Gaussian approximation permits ε → 0, removing discrepancy-kernel bias, but introduces bias through the normal model for simulator output.The approach is also described as more robust for extremely irregular or near-chaotic distributions.Source: §3 · p. 4
3.1 MCMC with a Random Acceptance Probability
- Finite-sample estimates make the acceptance probability random; the method models this uncertainty rather than repeatedly running many sets of S simulations.Under the Gaussian assumption, the variance of the estimated mean is the sample variance divided by S.Source: §3 · p. 4, §3.1 · p. 4
- The algorithm uses the median of the acceptance-probability samples as its MH threshold and E(α) as an error guide for additional simulations.The median minimizes the folded-CDF area used to define E(α).Source: §3.1 · p. 5
- Simulations are added in batches while E(α) exceeds the user-defined ξ threshold, which controls both accuracy and computational cost.Once uncertainty is sufficiently low, the sampler proceeds with the usual MH acceptance test.Source: §3.1 · p. 6
- Although a clearly better parameter setting can require few simulations, the adaptive synthetic-likelihood method discards each step’s simulations instead of reusing them.The paper identifies this as a drawback for expensive simulators and motivates storing runs with a Gaussian-process surrogate.Source: §3.1 · p. 6
4 Gaussian Process Surrogate ABC
GPS-ABC stores simulation results in Gaussian processes and uses their predictions and uncertainty to guide Metropolis–Hastings decisions and acquire further simulations when needed. Its approximate, adaptive chain trades exact per-step decisions for reduced simulation effort, while independent output models may overstate confidence.
4 Gaussian Process Surrogate ABC
- The GP stores prior simulation runs and predicts statistics and marginal likelihoods in unsimulated parameter regions, with uncertainty guiding whether more simulations are needed.The model is updated as inference proceeds.Source: §4 · p. 6
- GPS-ABC models the J statistics with independent Gaussian processes, a choice that may cause overconfidence but is more robust in high dimensions than estimating the full output covariance.The authors report that independent GPs worked well enough to illustrate the algorithm’s potential in their experiments.Source: §4 · p. 7
- When acceptance-probability uncertainty exceeds ξ, GPS-ABC acquires a training simulation at a location chosen to inform the current MH step.The acquisition location need not be either the current or proposed parameter value.Source: §4 · p. 8
- As the GP becomes sufficiently confident in a parameter-space region, GPS-ABC increasingly makes MH decisions there without expensive simulations.The algorithm uses the surrogate to predict both expected simulation output and its uncertainty rather than running default simulations at each MH step.Source: §4 · p. 8
4.1 Theoretical Aspects of GPS-ABC
- GPS-ABC is approximate because MH-step errors can produce error in the chain’s stationary distribution, with bounded error argued under small, bounded step errors and other mild conditions.The authors defer formal proofs and place their argument within a proof framework for another approximate MCMC algorithm.Source: §4.1 · p. 8
- GPS-ABC is adaptive as new training points change its stationary-distribution approximation; the authors argue that diminishing adaptation and ergodicity hold as acquisitions rapidly decrease.As adaptation becomes insignificant, the chain resembles the algorithm cited as providing bounded convergence.Source: §4.1 · p. 8
5 Experiments
The experiments span a controlled exponential example, a chaotic blowfly simulator, and a stochastic Wright–Fisher model, assessing simulation savings and posterior-predictive model comparison.
5 Experiments
- Three experiments assess algorithm correctness and computational savings, output-independence effects, and posterior-predictive model selection and comparison.Source: §5 · pp. 8–9
5.1 Toy Problem
- 1297 simulations at ξ = 0.05 gave the most accurate GPS-ABC run versus over 500K for ASL-ABC; ξ = 0.2 achieved nearly the same accuracy with 184 versus 100K.At ξ = 0.4, the GPS-ABC histogram showed slight bias, attributed to GP hyper-parameter calibration with only 29 training points.Source: §5.1 · p. 10, Figure 2 · p. 10
5.2 Chaotic Ecological Systems
- The blowfly experiment uses a chaotic population model and four summary statistics, rather than all 23 statistics used in the cited prior work.The authors report that time series generated from parameters based on their chosen statistics reasonably represent the observations.Source: §5.2 · p. 11
- GPS-ABC used a few thousand calls versus ASL-ABC’s well over a million; with independent output models their posteriors were similar, while full-covariance ASL-ABC had heavier tails.The authors note that the computation–accuracy trade-off favors GPS-ABC when simulations are very expensive.Source: Figure 6 · p. 12, Figure 7 · p. 13, §5.2 · p. 13
5.3 Feedback Wright-Fisher model
- With all three parameters inferred, the posterior predictive distribution was centered at y⋆; with only N inferred, it shifted toward p(y|θ⋆).Source: §5.3 · pp. 13–14
- Setting k = 6 rather than the true k = 3 yielded posterior-predictive distributions with large bias and variance; posterior bias was less pronounced, but variance remained large.Source: §5.3 · p. 14
6 Discussion and Future Work
The paper presents a framework for inference in expensive simulation-based models that uses fewer simulator calls than current state-of-the-art ABC methods. The discussion notes GP modeling limitations and inefficient random-walk proposals, while identifying possible improvements from Bayesian optimization and alternative proposal techniques.
6 Discussion and Future Work
- The algorithms require many fewer simulator calls than current state-of-the-art ABC methods for inference in expensive simulation-based models.The discussion presents this as the framework's central advantage.Source: §6 · p. 14
- GP surrogates face challenges from heteroskedastic simulator noise, output independence, and setting or updating hyperparameters during ABC.The blowfly simulator is cited as an example of likely heteroskedastic noise; input-dependent noise models and full-covariance GPs are suggested as alternatives.Source: §6 · p. 14
- GPS-ABC's random-walk proposal is inefficient for exploring the target distribution, motivating alternatives that could improve mixing and computational cost.The discussion describes testing GP uncertainty before requiring a simulation as a possible adaptation of Hamiltonian Monte Carlo proposals.Source: §6 · p. 14
- Bayesian-optimization acquisition functions may help choose simulation locations while retaining control of MCMC error, and distributed optimization may inspire parallel ABC sampling.These are proposed as directions for reducing simulation costs and developing parallel algorithms.Source: §6 · p. 14