Source-linked AI summary
Programming with models: writing statistical algorithms for general model structures with NIMBLE
Perry de Valpine, Daniel Turek, Christopher J. Paciorek, Clifford Anderson-Bergman, Duncan Temple Lang, Rastislav Bodik
TL;DR
NIMBLE addresses the gap between flexible model specification and reusable, efficient algorithms for general model structures. It extends BUGS within R, uses staged model-specific programming, and compiles models and run-time functions to C++. The paper demonstrates this design with importance sampling, MCMC, and MCEM, while noting unsupported BUGS features and other implementation limitations.
Problem
Existing software leaves a gap between flexible hierarchical models and easy application of newer or varied statistical algorithms.
Method
NIMBLE extends BUGS into model objects and provides staged nimbleFunctions that specialize to models before repeated execution, with compilation to C++.
Results
NIMBLE provides a system for combining flexible model specification with high-level, model-generic algorithm programming within R.
Takeaways & Limitations
The system supports implementing and applying algorithms such as importance sampling, MCMC, and MCEM across general model structures.
Takeaways & Limitations
NIMBLE does not yet incorporate all BUGS or graphical-model features, including efficient handling of stochastically indexed dependencies.
Abstract
from arXiv · showhide
We describe NIMBLE, a system for programming statistical algorithms for general model structures within R. NIMBLE is designed to meet three challenges: flexible model specification, a language for programming algorithms that can use different models, and a balance between high-level programmability and execution efficiency. For model specification, NIMBLE extends the BUGS language and creates model objects, which can manipulate variables, calculate log probability values, generate simulations, and query the relationships among variables. For algorithm programming, NIMBLE provides functions that operate with model objects using two stages of evaluation. The first stage allows specialization of a function to a particular model and/or nodes, such as creating a Metropolis-Hastings sampler for a particular block of nodes. The second stage allows repeated execution of computations using the results of the first stage. To achieve efficient second-stage computation, NIMBLE compiles models and functions via C++, using the Eigen library for linear algebra, and provides the user with an interface to compiled objects. The NIMBLE language represents a compilable domain-specific language (DSL) embedded within R. This paper provides an overview of the design and rationale for NIMBLE along with illustrative examples including importance sampling, Markov chain Monte Carlo (MCMC) and Monte Carlo expectation maximization (MCEM).
1 Introduction
NIMBLE addresses the gap between flexible hierarchical model specification and software for programming diverse statistical algorithms. It embeds an extensible BUGS-based modeling system and a staged, compilable algorithm language within R.
- Current hierarchical-model software offers limited easy-to-apply methods despite continual advances in statistical algorithms.
- NIMBLE combines flexible model specification with model-adaptive algorithm programming, unlike tools that either constrain models or provide only targeted algorithms.
- BUGS code is processed into model objects that programs can query for variable relationships, simulations, and probability calculations.
- Specialization and staged evaluation separate one-time model-structure queries from repeated algorithm execution.The compiler translates models and run-time functions into C++ while retaining an R interface.
- NIMBLE provides a domain-specific language embedded within R, with run-time code resembling a compilable subset of R.
- The paper illustrates the system through importance sampling, MCMC, and MCEM examples operating on a model.
2 Overview of NIMBLE
NIMBLE organizes model specification, model-generic algorithm programming, and compilation into a workflow for applying varied algorithms to flexible DAG models. Its model objects expose graph structure and computations, while staged nimbleFunctions specialize algorithms before repeated execution; several BUGS features remain unsupported.
- Overview: NIMBLE comprises an extended BUGS model language, nimbleFunctions for programming with models, and a compiler for models and nimbleFunctions.
- Design rationale: The system aims to make varied algorithms easier to implement and apply to any model defined as a directed acyclic graph.
- Design rationale: Current general software makes new methods difficult by requiring separate model-specification systems and by forcing a trade-off between high-level expression and computational performance.
- Model specification: NIMBLE extends BUGS while distinguishing its model-specification role from the use of BUGS in MCMC packages.
- Model specification: BUGS code produces a model definition object and model object containing graph relationships, variable values, log probabilities, simulations, and structure-query functions.
- Model specification: NIMBLE represents variables, nodes, and graph relationships flexibly enough to handle multivariate nodes and non-patterned declarations.
- Model specification: BUGS processing can extend or transform graphs, including adding nodes for alternative distribution parameterizations and expression values.
- Model specification: Data labels describe node roles separately from node values, allowing data to be changed programmatically across multiple datasets.
3 Examples
NIMBLE illustrates model-generic programming through importance sampling and Metropolis-Hastings algorithms, then shows how sampler choices can be customized for improved MCMC performance. These examples use model objects, staged setup and run-time computation, and compilation to support reusable algorithms across model structures.
- 3.1 Importance sampling: NIMBLE’s importance-sampling example approximates a marginal likelihood by integrating over selected random effects using simulated values and importance weights.For the pump model, the example integrates over theta[1:3] given alpha and beta, using samples from PIS and ratios involving model probabilities and proposal probabilities.
- 3.1 Importance sampling: The importance-sampling implementation is model-generic because its setup and run code are not specific to the pump model or theta[1:3].Setup queries stochastic dependencies, while run-time code copies sampled nodes into the model, calculates log probabilities, and averages weighted results.
- 3.2 Metropolis-Hastings samplers: NIMBLE expresses a Metropolis-Hastings sampler with setup-time dependency queries and run-time proposal, probability, acceptance, and state-copy operations.The example specializes the sampler to theta[4] in the pump model; the full sampler additionally supports adaptation and more efficient calculateDiff computation.
- 3.3 MCMC: For the correlated pump model, adding a bivariate adaptive random-walk sampler lowers chain autocorrelations and increases effective sample size per second.The sampler suite is customized using posterior correlation information, after which the MCMC is re-compiled.
- 3.4 Monte Carlo Expectation Maximization: MCEM alternates MCMC sampling of latent states with optimization over non-latent parameters, and NIMBLE targets automatic application across BUGS model structures.The paper motivates this as an implementation of MCEM for a range of model structures provided by BUGS.
4 Discussion
NIMBLE combines flexible model specification with high-level, model-generic algorithm programming within R. The discussion identifies remaining limitations in BUGS, R-based compilation, and support for additional computational features.
- NIMBLE combines a flexible model specification language with a high-level algorithm language for model-generic programming within R.
- NIMBLE’s BUGS foundation lacks some model features, including efficient handling of stochastically indexed dependencies, modular programming, and vectorized declarations.Stochastically indexed dependencies create dynamic graph structures requiring more flexible dependency representations.
- Future development could add automatic differentiation, more compilable functionality such as optimization, and compiler use without BUGS code.These extensions would support computations such as Laplace approximation and broaden the compiler’s role as an efficient-computation tool from R.
- Embedding the compilable DSL in R supports code-object processing and integration with a popular statistical environment but creates conceptual inconsistencies and slows compilation for large models.NIMBLE needs behaviors such as call-by-reference and side effects that differ fundamentally from R.
- BUGS syntax and parameterization differences from R can confuse users, while modular model generation and programmatic model definitions remain future extensions.NIMBLE’s design already enabled some multivariate-node extensions automatically, but other model-definition capabilities remain to be built.