ABC random forests for Bayesian parameter inference

Louis Raynal; Jean-Michel Marin; Pierre Pudlo; Mathieu Ribatet; Christian P. Robert; Arnaud Estoup

Source-linked AI summary

ABC random forests for Bayesian parameter inference

Louis Raynal, Jean-Michel Marin, Pierre Pudlo, Mathieu Ribatet, Christian P. Robert, Arnaud Estoup

arXiv:1605.05537v5May 18, 2016stat.MEstat.COstat.ML

TL;DR

ABC enables Bayesian inference when likelihoods are unavailable or computationally prohibitive, but existing implementations typically require summary-statistic selection and tolerance calibration. This paper uses regression random forests to infer parameter quantities without those preliminary choices, reporting greater robustness and competitive estimation quality, while noting that multidimensional parameter inference remains unresolved.

Problem
Likelihood-free Bayesian computation commonly requires selecting informative summary statistics and calibrating a tolerance level, while likelihoods may be unavailable or too costly to evaluate.
Method
The approach trains a separate regression random forest for each scalar parameter transform, using summary statistics as covariates without a distance-based tolerance level.
Results
The RF approach appears more accurate than previous ABC solutions, with accurate expectation estimates and slightly overestimated posterior variances that improve on typical underestimation.
Takeaways & Limitations
ABC-RF is mostly insensitive to ancillary covariates and can use many summary statistics without pre-selecting them.
Takeaways & Limitations
The method focuses on unidimensional parameter inference, while attempted multidimensional random-forest extensions were so far unsuccessful.

Abstract

from arXiv · show

This preprint has been reviewed and recommended by Peer Community In Evolutionary Biology (http://dx.doi.org/10.24072/pci.evolbiol.100036). Approximate Bayesian computation (ABC) has grown into a standard methodology that manages Bayesian inference for models associated with intractable likelihood functions. Most ABC implementations require the preliminary selection of a vector of informative statistics summarizing raw data. Furthermore, in almost all existing implementations, the tolerance level that separates acceptance from rejection of simulated parameter values needs to be calibrated. We propose to conduct likelihood-free Bayesian inferences about parameters with no prior selection of the relevant components of the summary statistics and bypassing the derivation of the associated tolerance level. The approach relies on the random forest methodology of Breiman (2001) applied in a (non parametric) regression setting. We advocate the derivation of a new random forest for each component of the parameter vector of interest. When compared with earlier ABC solutions, this method offers significant gains in terms of robustness to the choice of the summary statistics, does not depend on any type of tolerance level, and is a good trade-off in term of quality of point estimator precision and credible interval estimations for a given computing time. We illustrate the performance of our methodological proposal and compare it with earlier ABC methods on a Normal toy example and a population genetics example dealing with human population evolution. All methods designed here have been incorporated in the R package abcrf (version 1.7) available on CRAN.

1 Introduction

ABC addresses Bayesian inference when likelihoods are unavailable or too costly, but conventional implementations require informative summaries and tolerance calibration. The paper proposes regression random forests to automate summary-statistic use and avoid tolerance levels while estimating parameter quantities.

Intractable or costly likelihoods make standard Bayesian and maximum-likelihood analyses prohibitive for realistic parametric models.
The proposal applies regression random forests to summary statistics and predicts scalar transforms of parameters without defining a distance or tolerance level.
The paper evaluates the approach against earlier ABC methods using a Normal toy example and a human population-evolution population-genetics example.
ABC simulates datasets from the prior predictive distribution, summarizes them, and accepts simulations sufficiently close to the observed data.
Classical ABC requires selecting informative summary statistics and calibrating a tolerance level, both of which affect finite-sample reliability.
Random forests aggregate randomized regression trees trained on bootstrap samples, averaging tree predictions while considering subsets of covariates at each split.

3 Results from two examples

The method is evaluated on an exactly tractable Normal model and a human population-genetics model, with comparisons to earlier ABC approaches. ABC-RF performs well for posterior means and some quantiles, offers competitive accuracy and conservative credible intervals, and avoids tolerance calibration.

Normal toy example: On the Normal benchmark, ABC-RF estimates posterior means well, although ψ2(y) is slightly overestimated.Variance estimates are less satisfactory but remain acceptable.
Normal toy example: Compared with rejection, adjusted local linear, ridge-regression, and neural-network ABC, ABC-RF achieves lower normalized mean absolute errors and lower estimation variability in the Normal example.The comparison uses the same reference table and NMAE as the performance measure.
Human population genetics example: In the population-genetics example, ABC-RF provides the second-lowest NMAE and slightly conservative credible intervals, while avoiding tolerance-level calibration.Earlier ABC methods require calibration, which can be time-consuming when multiple tolerance levels are tested.
Human population genetics example: For the observed human dataset, ABC-RF estimates ra as 0.221 with a 95% CI of [0.112,0.287] and N2/Na as 4.508 with a 95% CI of [3.831,5.424].These are reported as relatively narrow credible intervals for both parameters.
Human population genetics example: Adding 20 independent uniform U[0,1] noise variables illustrates ABC-RF's relative insensitivity to ancillary covariates.The noisy-forest results are denoted RF∗, while RF denotes the version without noise variables.

4 Discussion

ABC-RF is presented as a likelihood-free parameter-inference approach that automates summary-statistic inclusion and performs well across the examined examples. Its main boundaries are quantile behavior, incomplete covariance estimation, and a focus on unidimensional inference.

ABC-RF automates the inclusion of summary statistics through regression random forests for likelihood-free parameter estimation.
Compared with earlier ABC methods, the RF approach appears more accurate, with accurate expectation estimates and only slight posterior-variance overestimation.Earlier approaches typically underestimate posterior variances.
Posterior covariance approximation was encouraging, but the method remained incomplete and required further development.
Upper quantiles may be overestimated and lower quantiles underestimated, so ABC-RF credible intervals may be wider than exact intervals.The authors regard this overestimation as less problematic than underestimating credible-interval widths.
Across experiments, ABC-RF was mostly insensitive to ancillary covariates and could handle many summary statistics without pre-selection.Random forests adaptively weight information from the proposed explanatory variables.
The RF method focuses on unidimensional parameter inference, while multidimensional extensions had not yet produced fruitful results.Alternative strategies using approximated conditional distributions were still being compared on simulated datasets.

Funding

The study acknowledges financial and institutional support for its authors and projects.

A.E. received support from the ANR, ERA-Net BiodivERsA project EXOTIC, ANR project SWING, and INRA SPE.
C.P.R. was supported by an IUF 2016–2021 senior grant.
The acknowledgments identify both project-specific and institutional sources of support.

1 Comparing three methods of variance estimation of parameters

The supplementary analysis compares three ABC-RF estimators for posterior variance: reusing RF weights, training a new RF, and using a cumulative-distribution approximation. On the Normal example, they behaved similarly, while estimator 1 was selected for the main examples because it showed slightly lower errors.

Three posterior-variance estimators were compared: reusing RF weights, training a new RF, and approximating the cumulative distribution function.
The new-RF estimator is more expensive because it requires an additional random forest.
On the Normal toy example, all three estimators behaved similarly and tended to overestimate posterior variances.Their boxplots were alike, and the comparison also supported similarity in normalized mean absolute error.
Estimator 1 was selected for the two main examples because it showed slightly lower errors for both θ1 and θ2.
Table S1 compares normalized mean absolute errors, with the smallest values shown in bold.

2 Study of covariances of parameters using random forests

The section develops a dedicated random-forest estimator for posterior covariance between transformed parameters and evaluates it in a two-parameter Normal regression example. ABC-RF achieves lower NMAE than adjusted ridge and neural-network ABC methods, although it tends to overestimate covariances when earlier methods underestimate them.

Methodology: ABC-RF estimates posterior covariance between two transformed parameters using a dedicated random forest.The covariance target is Cov(τ,σ | η(y)), with τ = h(θ) and σ = g(θ).
Methodology: The covariance approximation requires three regression random forests: one for each parameter and one for the covariance.The method uses out-of-bag posterior expectation estimates for the two parameters when constructing the covariance response.
Toy regression example: The toy experiment targets the non-zero posterior covariance Cov(β1,β2 | y) in a simulated two-parameter regression design.A reference table contains N = 10000 replicates, each based on n = 100 observations, with k = 60 summary statistics including noise variables.
Toy regression example: The comparison uses an independent test dataset of size Npred = 100 and random forests built with B = 500 trees, ntry = k/3, and minimum node size 5.The benchmark compares ABC-RF with adjusted ridge regression and adjusted neural-network ABC methodologies.
Toy regression example: ABC-RF has lower NMAE for posterior covariance estimation than adjusted ridge regression and adjusted neural-network ABC methods.Figure S2 additionally shows ABC-RF overestimating covariances when the earlier ABC methods underestimate them.

3 A basic R code to use the abcrf package version 1.7

The section provides basic workflows for loading reference tables, selecting a scenario and parameter, fitting ABC-RF, assessing summaries, and predicting posterior quantities with abcrf version 1.7.

Reading data: Reference tables can be read from DIYABC binary and header files or from whitespace-separated text files.The text format contains scenario indices, p parameter columns, and k summary-statistic columns.
Reading data: The workflow subsets a selected scenario and creates a data frame containing the parameter of interest and its summary statistics.For text input, model 1 is selected by filtering the first column of scenario indices.
Fitting and diagnostics: The regAbcrf function fits a random forest using a formula and training data frame, with controls for trees, node size, covariates, and parallel computing.The documented defaults include ntree = 500 and min.node.size = 5.
Fitting and diagnostics: The err.regAbcrf function represents out-of-bag mean squared error as the number of trees changes.Summary-statistic contributions can be plotted, including the 25 most important statistics.

4 Supplementary figures for the Normal toy example

Figure S3 compares ABC-RF posterior-density approximations with the true posterior density for θ1 in the Normal model across independent reference tables and test datasets.

Supplementary figure: The figure compares the true posterior density of θ1 with 40 ABC-RF posterior-density approximations.The approximations use random-forest weights and are based on 40 independent reference tables.
Supplementary figure: The upper and lower panels correspond to two different test datasets.True posterior densities are drawn as red lines.
Supplementary figure: The comparison assesses how closely ABC-RF approximations based on independent reference tables track the true posterior density.The figure presents a sample of approximated posterior densities rather than a single approximation.

5 Summary statistics available in the software DIYABC v.2.1.0 for SNP data

The section lists DIYABC v.2.1.0 summary statistics for SNP data, organized by whether they describe one, two, or three population samples.

Overview: DIYABC provides SNP summary statistics based on means, variances, and proportions of null values across loci.These statistics provide a rough description of the allelic spectrum for population samples.
Single population statistics: Single-population statistics include monomorphic-locus proportions and gene-diversity means and variances.The listed statistics are HP0_i, HM1_i, HV1_i, and HMO_i.
Two population statistics: Two-population statistics summarize null proportions, means, and variances for FST and Nei’s distances.The listed families include FP0, FM1, FV1, FMO, NP0, NM1, NV1, and NMO.
Three population statistics: Three-population statistics summarize admixture estimates through null proportions, means, and variances.The listed statistics are AP0_i_j&k, AM1_i_j&k, AV1_i_j&k, and AMO_i_j&k.

6 Supplementary figures for the Human population genetics example

Figure S4 compares the range and coverage of approximate 95% credible intervals for N2/Na estimated by ABC-RF and earlier ABC methods.

Figure S4 compares ABC-RF with rejection, adjusted local linear, ridge regression, and neural-network ABC methods across various tolerance levels.Bar colors and superimposed values specify coverage, while heights indicate mean credible-interval lengths.

7 Contribution of summary statistics in ABC-RF estimation of the parameters ra and N2/Na of the Human population genetics example

ABC-RF automatically identifies relevant summary statistics for each parameter using variable importance based on reductions in residual sum of squares. In the human population genetics example, the most informative statistics differ between parameters.

ABC-RF measures a summary statistic’s importance as its total reduction in residual sum of squares across forest splits, divided by the number of trees.The method applies this criterion separately to each parameter of interest.
At each node, the RSS decrease compares parameter-value variation in the parent node with variation in the left and right daughter nodes after splitting.The averages of the parameter values in these three nodes define the calculation.
The importance measure aggregates decreases over all nodes using summary statistic j across all trees, then divides by the number of trees.Higher variable importance indicates greater informativeness for the estimation procedure.
The 30 most important statistics are selected from 112 DIYABC statistics for estimating ra and N2/Na in the human population genetics example.Figure S5 reports their contributions for both parameters.
The most informative summary statistics clearly differ according to the parameter being estimated.For ra, the ten most informative statistics characterize a population-related feature, as stated in the passage.

8 Computation times required by the statistical treatments of the studied methods processed following the generation of the reference table

The methods are compared by the computation time required after reference-table generation for estimating the admixture rate ra. Rejection and adjusted local linear ABC are fastest, while ABC-RF requires moderately more time.

Rejection and adjusted local linear ABC provide the best CPU-time results, even when the tolerance level equals 1.The comparison concerns prediction of admixture-rate estimates in the human population genetics example.
ABC-RF requires moderately higher computing time than rejection and adjusted local linear ABC.Calculating the RF weights is identified as the most expensive computation.
Table S3 compares estimation times for ABC-RF, rejection, adjusted local linear, ridge regression, and neural-network methods at various tolerance levels.The test table contains 1000 pseudo-observed datasets, while the reference table contains 199000 simulated datasets.