PaperlayerView exact revision on arXiv

Source-linked AI summary

Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems

Tina Toni, David Welch, Natalja Strelkowa, Andreas Ipsen, Michael P. H. Stumpf

arXiv:0901.1925v1stat.COstat.ME

TL;DR

The paper addresses parameter inference and model selection for dynamical systems when likelihoods are difficult to evaluate and data may be incomplete. It develops and applies ABC SMC, finding reliable inference and model-selection performance across biological applications, while identifying sensitivity to priors and algorithm settings.

  • Problem

    Dynamical systems often have uncertain parameters, competing models, incomplete data, and likelihood surfaces that are difficult to evaluate reliably.

  • Method

    The paper applies simulation-based approximate Bayesian computation with sequential Monte Carlo to infer parameters, assess inferability and sensitivity, and select among dynamical models.

  • Results

    50-times fewer data generation steps were needed by ABC SMC than by the ABC rejection sampler for the deterministic LV model, with virtually the same outcome.

  • Takeaways & Limitations

    ABC SMC can estimate parameters and credible intervals, distinguish competing models, and operate without change in deterministic, stochastic, and time-delay settings when models can be efficiently simulated.

  • Takeaways & Limitations

    ABC SMC model selection depends on chosen prior distributions, tolerance levels, and perturbation-kernel variances, so these settings require care.

Abstract

from arXiv · show

Approximate Bayesian computation methods can be used to evaluate posterior distributions without having to calculate likelihoods. In this paper we discuss and apply an approximate Bayesian computation (ABC) method based on sequential Monte Carlo (SMC) to estimate parameters of dynamical models. We show that ABC SMC gives information about the inferability of parameters and model sensitivity to changes in parameters, and tends to perform better than other ABC approaches. The algorithm is applied to several well known biological systems, for which parameters and their credible intervals are inferred. Moreover, we develop ABC SMC as a tool for model selection; given a range of different mathematical descriptions, ABC SMC is able to choose the best model using the standard Bayesian model selection apparatus.

1 Introduction

Dynamical-system analysis is hindered by uncertain parameters, competing equation structures, incomplete biological data, and difficult likelihood surfaces. The paper introduces ABC SMC tools for parameter inference, sensitivity analysis, and model selection.

  • Uncertain parameters, competing models, scarce data, and complex likelihood surfaces complicate dynamical-system analysis.
  • Existing tools address parameter estimation and, to a lesser extent, model selection, but lack broad applicability across dynamical-model classes.
  • The paper applies ABC SMC, replacing likelihood evaluation with simulation, to parameter estimation and model selection for dynamical models.
  • ABC SMC is used to examine parameter inferability and model sensitivity, compare approximate Bayesian computation methods, and analyze biological and epidemiological datasets.

2 Methods

ABC replaces likelihood evaluation with simulation-based comparison to approximate posterior distributions. The paper develops ABC SMC for parameter inference and Bayesian model selection, using sequential tolerance reduction and model posterior probabilities.

  • ABC methods: ABC approximates posterior distributions by simulating datasets and accepting parameters whose simulated data are sufficiently close to observations under a distance function and tolerance.The distance can be computed on full datasets or summary statistics.
  • ABC methods: ABC rejection sampling and ABC MCMC have limitations, including low acceptance rates or correlated samples, which motivate sequential Monte Carlo approaches.Rejection sampling performs poorly when the prior differs substantially from the posterior, while ABC MCMC produces correlated samples.
  • ABC SMC: ABC SMC propagates particles through distributions with decreasing tolerances, gradually moving from the prior toward the target approximate posterior.The tolerances satisfy ϵ1 > . . . > ϵT ≥ 0, and perturbation kernels generate new particles between populations.
  • Model selection: ABC SMC model selection treats the model identity as an additional discrete parameter and estimates posterior probabilities for models and their parameters.Bayes factors can be obtained from the posterior model probabilities.
  • Model selection: The model-selection framework supports averaging across models when no single model explains all aspects of the data best.This uses the estimated marginal posterior distribution over models rather than relying exclusively on one model.
  • Model selection: Models with more parameters are implicitly penalized because perturbed particles are less likely to be accepted as parameter dimension increases.The model-selection algorithm can also eliminate models when no particles remain associated with them.

3 Results

The results illustrate ABC SMC for parameter inference in deterministic and stochastic dynamical systems, including computational efficiency, posterior inferability, and sensitivity analysis. Across examples, ABC SMC recovers parameters with varying precision and links inferability to model sensitivity.

  • ABC SMC applications: ABC SMC estimates posterior distributions for deterministic and stochastic biological dynamical systems using simulated data and model-specific distance functions.The examples include Lotka-Volterra prey–predator dynamics and repressilator dynamics.
  • Computational efficiency: 14.1 million simulations were required by ABC rejection to accept 1,000 particles, whereas ABC MCMC required 40,000–60,000 simulations after calibration.The rejection sampler's acceptance rate was 7 · 10^-5.
  • Computational efficiency: 50-times fewer data-generation steps than ABC rejection were needed by ABC SMC for deterministic Lotka-Volterra dynamics, with comparable results to adaptive ABC MCMC.The inferred parameters were a: median = 1.05, 95%-quantile range = [1.00, 1.12] and b: median = 1.00, 95%-quantile range = [0.87, 1.11].
  • Inferability and sensitivity: In deterministic repressilator dynamics, parameter n is inferred quickest with the smallest posterior variance, while parameter α is barely inferable with large credible intervals.The model is most sensitive to n and least sensitive to α.
  • Inferability and sensitivity: PCA uses the smallest principal components to identify stiff parameter combinations and the largest components to identify potentially sloppy combinations.For the repressilator, the last principal component mainly follows a linear combination of n and β, while the third component indicates lower sensitivity to α0.
  • Stochastic dynamics: Parameters n and β are reasonably well inferred in stochastic repressilator dynamics, whereas α0 and α are harder to infer than in the deterministic case.The paper relates weak inferability to low sensitivity and notes that stochastic fluctuations can overwhelm signals from parameter variation.

3.3 Model selection on different SIR models

The paper compares four SIR models using ABC SMC, jointly inferring parameters and model identity from noisy epidemic data. The method favors the correct model under intermediate noise and selects a latent-carrier model for the Tristan de Cunha data, although only marginally.

  • Candidate models: The four candidate SIR models differ by infection delay, a latent infection phase, or loss of immunity, while producing visually similar outputs.Their similarity motivates statistically based model selection rather than visual inspection alone.
  • ABC SMC model selection: ABC SMC jointly performs parameter inference and model selection using model index m and model-specific parameter vectors.The model index takes values 1–4, with each model carrying its corresponding parameter vector.
  • Synthetic-data evaluation: With Gaussian noise σ = 0.2, the posterior places the most weight on the correct model, whereas σ = 1 prevents detection of a single best model.The experiment used 12 observations from each of S, I, and R; the σ = 0.2 example generated data from model 1.
  • Synthetic-data evaluation: From 1000 final particles, model 1 was selected 664 times, model 2 230 times, model 4 106 times, and model 3 zero times.These frequencies imply weak evidence for model 1 over model 2 and positive evidence over model 4.
  • Common-cold application: For Tristan de Cunha common-cold data, model (14) with a latent disease-carrier class was most suitable but only marginally better than models (12) and (13).The authors therefore suggest model averaging across models (12), (13), and (14) for reliable parameter conclusions.

4 Discussion

The discussion presents ABC SMC as a general simulation-based framework that also reveals parameter inferability and model sensitivity. It highlights practical benefits while cautioning that model selection depends on priors, tolerances, kernels, and convergence monitoring.

  • Scope and generality: ABC SMC can be applied without modification to deterministic and stochastic models, including models with time delays, when the systems can be efficiently simulated.The paper characterizes the method as simple and general across these dynamical-system settings.
  • Inferability and sensitivity: Intermediate and posterior distributions provide sensitivity and inferability information without additional simulations because the parameter-estimation simulations can be reused.PCA or scatterplots can analyze these distributions.
  • Inferability and sensitivity: Parameters whose distributions remain close to the prior across populations are not inferable from the available data.The paper links narrow posterior regions to sensitive parameter combinations and broader regions to sloppy combinations.
  • Caveats: Model selection requires carefully chosen prior domains and monitored acceptance rates because inappropriate priors can reject models during early populations.The paper also reports dependence of Bayes factors on tolerance levels and perturbation-kernel variances.
  • Caveats: Convergence should be monitored using distributional checks and proposal counts, which impose a practical limit on the procedure.Suggested checks include inter-quartile ranges and goodness-of-fit tests between successive intermediate distributions.
  • Computational considerations: For larger systems, computational efficiency can be improved by tuning populations, distances, particles, and perturbation kernels, and the algorithm is easily parallelized.The examples in the paper were computationally efficient enough.

5 Conclusion

The paper concludes that ABC SMC estimates parameters and credible intervals while distinguishing among competing dynamical models. Its sensitivity–inferability link can help identify critical parameters in larger systems.

  • Conclusion: ABC SMC estimates model parameters, including credible intervals, and distinguishes among competing models.The authors state that the approach applies across deterministic and stochastic systems in several scientific domains.
  • Conclusion: The sensitivity–inferability link allows critical parameters to be identified quickly while recognizing parameters that are difficult to infer because the system is relatively insensitive to them.The conclusion frames this as an application to larger systems.

A Appendix: Derivation of ABC SMC

The appendix derives ABC SMC by embedding approximate Bayesian computation within sequential importance sampling. Particles move through progressively tighter tolerance distributions using perturbation kernels, simulation-based acceptance, and importance weighting.

  • Sequential importance sampling: Sequential importance sampling reaches the target distribution through intermediate distributions and proposal distributions weighted by importance weights.This provides the framework from which ABC SMC is constructed.
  • ABC construction: ABC SMC defines intermediate distributions by combining the prior with simulated datasets and an indicator for meeting tolerance ϵ_t.For a fixed parameter, B_t datasets are simulated and the acceptance fraction b_t(x) is used in the construction.
  • Proposal distributions: The initial proposal equals the prior, while later proposals perturb particles from the previous intermediate distribution with kernel K_t.The perturbed proposal retains particles with positive prior support and at least one accepted simulation.
  • Algorithm: At each iteration, ABC SMC samples or perturbs a particle, simulates candidate datasets, accepts it when the tolerance criterion is met, and normalizes weights.The algorithm repeats this process across populations until the final tolerance is reached.
  • Algorithm: For deterministic systems, the algorithm sets B_t = 1 and simulates one dataset per particle.The perturbation kernel is used to generate candidate particles between populations.

B Appendix: Comparison of ABC SMC algorithm with the ABC PRC of Sisson et al.

The comparison examines ABC SMC against ABC PRC and ABC rejection, focusing on weighting, perturbation, and computational cost. ABC SMC produces more reliable variance estimates under small perturbations, although kernel choice remains important.

  • Algorithmic comparison: ABC SMC and ABC PRC differ mainly in their weighting procedures: ABC SMC uses an SIS framework, whereas ABC PRC uses an SMC sampler with a backward kernel.The backward kernel in ABC PRC is difficult to choose optimally, and using the forward kernel may be poor.
  • Toy comparison: 100 particles and 30 runs were used to compare variance estimates across the tolerance schedule ϵ = {2.0, 1.5, 1.0, 0.75, 0.5, 0.2, 0.1, 0.075, 0.05, 0.03, 0.025}.The comparison used ABC SMC, ABC PRC, and ABC rejection on a mixture of two normal distributions.
  • Results: For relatively small perturbations, ABC PRC produces variance that is too small, whereas ABC SMC becomes comparable to ABC rejection.The discrepancy is attributed to improperly weighted particles in ABC PRC.
  • Perturbation choice: Using many populations with a perturbation kernel that is too small, such as σ = 0.15, gives a poor approximation.The authors therefore recommend a sufficiently large perturbation, while noting that strict selection guidelines are unavailable.
  • Special case: In the one-dimensional uniform-prior case, ABC PRC and ABC SMC give the same results when the uniform-kernel width spans the whole parameter range.Under that setting, all weights are equal.
  • Computational cost: ABC SMC has O(N^2) weight evaluation after each population, compared with O(N) for SMC with a backward kernel, but simulation usually dominates ABC computation.The paper argues that the extra weighting cost is therefore acceptable for avoiding backward-kernel problems.

C Appendix: ABC and full likelihood for ODE systems

This appendix relates ABC for deterministic ODE systems to full likelihood inference under Gaussian-error assumptions. With squared-error distances and equal diagonal variances, minimizing the ABC distance corresponds to maximizing a likelihood.

  • ABC and likelihood: For deterministic dynamical systems, ABC with a squared-error distance is equivalent to maximum likelihood when Gaussian errors are assumed.The distance compares observed data with the deterministic model solution.
  • Data comparison: The model solution g(t, θ) and measured data points D = (x_i) are compared across observation times t_1, . . . , t_n.The data points and solution are m-dimensional.
  • Bayesian interpretation: When Σ is diagonal with equal entries, ABC for deterministic ODE systems is closely related to Bayesian inference using an explicitly evaluated likelihood.The relationship follows because ODE models lack an intrinsic probability model and likelihoods are commonly defined through nonlinear regression assumptions.

Video of prior, intermediate and posterior distributions of deterministic repressilator model

The video visualizes how particle distributions change for the deterministic repressilator model, from the prior through an intermediate distribution to the approximate posterior.

  • Prior distribution: The video shows particles from the four-dimensional prior distribution for the deterministic repressilator model.This provides the starting distribution before sequential updating.
  • Intermediate distribution: The video also shows particles from the fourth intermediate distribution.This represents a stage between the prior and the final approximate posterior.
  • Posterior distribution: The final visualization displays the four-dimensional approximate posterior distribution obtained for the deterministic repressilator model.The video was produced using GGobi and RGGobi from the R statistical environment.

Datasets

The paper’s example analyses use simulated datasets supplied as supplementary material.

  • Supplementary data: Simulated datasets used in the paper’s examples are attached as supplementary material.The datasets support the examples described throughout the paper.
  • Data availability: The supplementary material provides the simulated data associated with the paper’s examples.These data are distinct from the methodological descriptions in the main text.
  • Example analyses: Readers can use the attached simulated datasets to examine the examples presented in the paper.The passage states that the datasets are attached but does not specify their format.
Loading 0901.1925v1…