Source-linked AI summary
Bayesian computing with INLA: new features
Thiago G. Martins, Daniel Simpson, Finn Lindgren, Håvard Rue
TL;DR
Approximate posterior marginals for hyperparameters are difficult to obtain because evaluating their joint density is costly. The paper formalizes new R-INLA developments, including algorithms that bypass numerical integration, and reports successful hyperparameter marginals from few evaluations while extending the interface’s model scope.
Problem
Obtaining posterior marginals of hyperparameters is not straightforward because evaluating their joint density is costly.
Method
The paper formalizes new R-INLA features and describes algorithms that bypass numerical integration for approximating hyperparameter posterior marginals.
Results
The algorithms successfully obtain posterior marginals of hyperparameters using few evaluations, while providing accuracy and scalability.
Takeaways & Limitations
The new R-INLA features greatly extend the scope of models analyzable through its interface and provide a faster, more accurate alternative to simulation for latent Gaussian models.
Takeaways & Limitations
R-INLA is not available on CRAN because it depends on a stand-alone C program.
Abstract
from arXiv · showhide
The INLA approach for approximate Bayesian inference for latent Gaussian models has been shown to give fast and accurate estimates of posterior marginals and also to be a valuable tool in practice via the R-package R-INLA. In this paper we formalize new developments in the R-INLA package and show how these features greatly extend the scope of models that can be analyzed by this interface. We also discuss the current default method in R-INLA to approximate posterior marginals of the hyperparameters using only a modest number of evaluations of the joint posterior distribution of the hyperparameters, without any need for numerical integration.
1 Introduction
INLA provides fast approximate Bayesian inference for latent Gaussian models, while R-INLA makes the methodology accessible across a broad range of applications. This paper formalizes new R-INLA developments that expand supported models and improve hyperparameter marginal computation.
- INLA uses deterministic nested Laplace approximations to provide a faster and more accurate alternative to simulation-based MCMC for latent Gaussian models.
- R-INLA’s accessible interface has supported applications spanning generalized linear mixed, spatial, survival, stochastic volatility, and spatio-temporal models.
- Users from diverse fields have used INLA to fit models they were otherwise unable to fit, including applications in econometrics, ecology, and climate research.
- R-INLA addresses implementation complexity by providing an easy-to-use interface to the stand-alone C-coded inla program.
- For hyperparameters, the paper describes algorithms that use few evaluations of the joint posterior and bypass numerical integration while providing accuracy and scalability.
- The paper formalizes new R-INLA features that greatly extend the scope of models available through the interface, including tools already present in the theoretical INLA framework.
2 Integrated Nested Laplace Approximation
INLA models latent Gaussian structures through a hierarchical formulation and approximates posterior marginals using nested Laplace methods. R-INLA extends this framework with more flexible likelihood, latent-field, and hyperparameter-marginal capabilities.
- Latent Gaussian models: The hierarchical model separates a likelihood stage with conditionally independent observations from a Gaussian latent-field stage with hyperparameters.The latent field may include additive predictors, unknown functions, linear covariate effects, and unstructured terms with Gaussian priors.
- R-INLA extensions: R-INLA relaxes Stage 1 assumptions by supporting subsets of data from different likelihoods and observations depending on linear combinations of latent-field elements.These features broaden the observation model beyond one common likelihood and one-to-one observation-to-field connections.
- Latent Gaussian models: Sparse precision matrices make Gaussian Markov random fields computationally important, while linear constraints can further structure the latent field.The framework accommodates latent fields with sparse precision matrices and constraints Ax = e.
- R-INLA extensions: R-INLA adds flexibility for defining complex latent fields from simpler GMRF building blocks.The package formalizes features that were already available within the theoretical INLA framework and extends the latent-field structures users can define.
- INLA methodology: INLA approximates the joint hyperparameter posterior, conditional latent-field marginals, and final marginals through grid exploration and interpolation.The conditional marginals can use Gaussian, Laplace, or simplified Laplace approximations; the Gaussian option is fastest but can have location and skewness errors.
- INLA methodology: R-INLA’s current hyperparameter-marginal method constructs an interpolant from evaluated joint-posterior densities and bypasses numerical integration.The paper formalizes this interpolation-based approach, whose details were not given in the earlier INLA description.
3 On the posterior marginals for the hyperparameters
The section presents grid- and CCD-based strategies for exploring hyperparameter space, then develops an asymmetric Gaussian interpolation and a numerical-integration-free alternative for hyperparameter marginals.
- Grid and CCD exploration: Grid exploration becomes computationally expensive as the hyperparameter dimension m increases, with cost growing exponentially in m.The grid is practical when θ has low dimension, but its cost limits higher-dimensional applications.
- Grid and CCD exploration: CCD requires less computational power than a grid and can capture variability when the hyperparameter space is too wide for grid exploration.For latent-field marginals, a rough grid may suffice, whereas CCD offers broader exploration with lower computational cost.
- Reuse of evaluated points: R-INLA reuses density evaluations from the grid exploration to compute hyperparameter marginals with little extra cost.These points are already evaluated when integrating uncertainty over θ for latent-field marginals.
- Asymmetric Gaussian interpolation: Asymmetric Gaussian interpolation uses direction-specific scaling parameters to represent skewness while reusing computations from the numerical strategy.The parameters are calibrated so the target log-density drops approximately −2 at ±2 standard deviations, without extra cost.
- Numerical integration-free algorithm: The proposed integration-free algorithm avoids numerical integration when approximating hyperparameter posterior marginals.It was sought because multidimensional integration became unstable for models with more hyperparameters and could produce undesirable spikes.
- Numerical integration-free algorithm: The algorithm computes conditional means using a Gaussian approximation around the mode and covariance, although this is approximate because θ is not Gaussian.For each axis, the conditional mean is linear in θj under the Gaussian assumption.
- Numerical integration-free algorithm: The integration-free algorithm is the default R-INLA method for hyperparameter posterior marginals and can produce sensible results with almost no extra computation time.The reported approach uses only a few evaluations of the joint hyperparameter posterior and avoids numerical integration.
4 Extending the scope of INLA
Section 4 formalizes R-INLA features that extend the model scope available through the interface, including multiple likelihoods, replicated latent models, copied components, and linear combinations.
- 4.1 Multiple likelihoods: Multiple likelihoods let different data subsets use different distributions or hyperparameters within one R-INLA model.The interface represents responses in a matrix or list with separate columns or elements for the likelihood functions.
- 4.4 Linear combinations of the latent field: Linear combinations allow each data point to connect with a linear combination of latent-field elements, while preserving Markovian structure when the relevant assumptions hold.The paper states that inference remains fast when the associated quantities are zero.
- 4.2 Replicate feature: The replicate feature represents conditionally independent realizations of the same latent model sharing hyperparameters.Replications can be specified with the replicate argument inside the f() function.
- 4.3 Copy feature: The copy feature connects each linear predictor to multiple elements of a latent model when one random-effect element is insufficient.This addresses formulations in which predictors depend jointly on elements such as a_i and b_i.
- 4.4 Linear combinations of the latent field: The default integration-free method computes posterior marginals for hyperparameters without numerical integration and is much faster because it does not alter the latent-field graph.The paper reports little difference between the default faster approach and the more expensive grid approach for the illustrated linear combinations.
5 Conclusion
The paper reports that new R-INLA features were formalized in response to demand for more complex models and have broadened the models usable from R. It also describes efficient algorithms for hyperparameter marginals based on evaluations already available from latent-field inference.
- 5 Conclusion: R-INLA has become a daily tool for applied researchers across areas including dynamic, shared-random-effects, spatio-temporal, and preferential-sampling models.These applications illustrate the generic nature of the features presented in Section 4.
- 5 Conclusion: The paper formalizes several new R-INLA features that greatly extend the scope of models available within R.The project continues to evolve in response to demands from the statistical and applied community.
- 5 Conclusion: The paper presents case studies covering varied situations that use the formalized features.Examples and code are available through the INLA website.
- 5 Conclusion: Hyperparameter posterior marginals can be obtained using a few evaluations already computed while integrating over hyperparameter uncertainty for latent-field marginals.The paper describes two algorithms used successfully for this purpose.