Source-linked AI summary
Learning Bayesian Networks with the bnlearn R Package
Marco Scutari
TL;DR
bnlearn addresses the need for Bayesian-network structure learning with high-dimensional data by providing algorithms, tests, and scores for discrete and continuous variables. It supports both constraint-based and score-based learning, with flexible algorithm–criterion combinations and a Hill-Climbing implementation. The package also covers settings where distributional assumptions create algorithmic complexity or variable-type restrictions.
Problem
High-dimensional data require structure-learning algorithms that reduce computational complexity, while some distributional assumptions make learning more complex or restrict mixed-variable relationships.
Method
bnlearn implements constraint-based algorithms, a score-based Hill-Climbing search, conditional independence tests, and network scores, while allowing learning algorithms and statistical criteria to be selected separately.
Results
The package provides structure-learning functionality for both discrete and continuous data, including multiple algorithmic and statistical-criterion options.
Takeaways & Limitations
Users can compare different network structures for the same data and assess learned-network goodness of fit with respect to a chosen score function.
Takeaways & Limitations
Mixed-variable approaches may restrict a continuous variable from being the parent of a discrete variable, and graph-editing commands reject changes that would create cycles.
Abstract
from arXiv · showhide
bnlearn is an R package which includes several algorithms for learning the structure of Bayesian networks with either discrete or continuous variables. Both constraint-based and score-based algorithms are implemented, and can use the functionality provided by the snow package to improve their performance via parallel computing. Several network scores and conditional independence algorithms are available for both the learning algorithms and independent use. Advanced plotting options are provided by the Rgraphviz package.
1. Introduction
Bayesian networks are used across diverse application domains, creating demand for structure-learning algorithms that manage high-dimensional data. bnlearn addresses this demand by implementing learning algorithms, tests, and scores for both discrete and continuous data.
- Bayesian networks have been applied to OLAP performance, medical service analysis, gene expression, breast cancer prognosis, and epidemiology.
- High-dimensional data sets have motivated algorithms that reduce computational complexity while still learning the correct network.
- bnlearn provides free implementations of structure-learning algorithms, conditional independence tests, and network scores.
- The package supports both discrete and continuous data and allows learning algorithms to be selected independently from their statistical criteria.
2. Bayesian networks
Bayesian networks represent probabilistic dependencies with directed acyclic graphs whose structure factorizes a joint distribution into local distributions. Common models assume either multinomial discrete data or multivariate normal continuous data, while other assumptions introduce complexity or limitations.
- Bayesian networks use nodes for random variables and arrows for probabilistic dependencies.
- Their directed acyclic graph defines a factorization of the joint distribution into local distributions, with each variable depending directly only on its parents.
- Structure learning first learns the graphical structure and then estimates local distribution parameters conditional on that structure.
- The literature focuses mainly on multinomial discrete Bayesian networks and Gaussian Bayesian networks for multivariate normal data.
- Nonparametric and mixed-variable approaches can require more complex algorithms or impose restrictions on allowable parent-child variable types.
3. Structure learning algorithms
Bayesian network structure-learning algorithms are divided into constraint-based and score-based approaches. Constraint-based methods use conditional independence to construct graphs, whereas score-based methods search for high-scoring candidate networks.
- Constraint-based algorithms analyze conditional independence relations and construct graphs satisfying corresponding d-separation statements.
- Score-based algorithms assign scores to candidate Bayesian networks and maximize them using heuristic search procedures.
- Constraint-based learning follows the IC framework: learn the skeleton, orient v-structures, and direct remaining arcs while preserving acyclicity.
- Score-based methods can use hill-climbing, tabu search, simulated annealing, or genetic algorithms, with score-equivalent functions assigning equal scores to distribution-equivalent networks.
4. Package implementation
bnlearn implements multiple constraint-based and score-based structure-learning algorithms, conditional-independence tests, network scores, and prior-knowledge constraints for discrete and continuous data.
- Structure learning algorithms: bnlearn implements Grow-Shrink, IAMB, Fast-IAMB, Interleaved IAMB, and Max-Min Parents and Children constraint-based algorithms.The implementations include optimized and unoptimized variants, with parallel execution available through snow.
- Structure learning algorithms: The package provides an optimized hill-climbing search over directed graphs as its score-based learning algorithm.Score caching, decomposability, and score equivalence reduce duplicated tests; random restarts, perturbations, and initial networks can address poor local maxima.
- Conditional independence tests: Conditional-independence testing supports discrete data through mutual information, Pearson’s X2, fast mutual information, and AIC-based tests.Asymptotic and Monte Carlo permutation variants are available for several tests.
- Conditional independence tests: For continuous data, bnlearn provides tests based on partial correlations, including linear correlation, Fisher’s Z, and mutual information.Both asymptotic and Monte Carlo permutation tests are available for the listed correlation-based procedures.
- Network scores: Available network scores include likelihood, log-likelihood, AIC, BIC, Bayesian Dirichlet equivalent, K2, and Gaussian posterior-density scores.The listed discrete scores include Dirichlet-based alternatives, while bge is available for continuous data; K2 is not score equivalent.
- Prior knowledge: Blacklist and whitelist arguments incorporate prior arc constraints, including guaranteed directions, forbidden arcs, and partially directed structures.The package resolves combinations of directional and undirected constraints into guaranteed-present or guaranteed-absent relationships.
5. A simple example
The example uses the six-variable learning.test data set to learn and compare Bayesian-network structures with constraint-based and hill-climbing methods. The constraint-based algorithms agree on a partially directed structure, while hill-climbing selects one equivalent directed orientation.
- Data and setup: learning.test contains 5,000 observations of six discrete factor variables, with three levels for A–E and two levels for F.The data set is included with bnlearn and used throughout the examples.
- Constraint-based learning: The Grow-Shrink algorithm learns a six-node partially directed network with five arcs, including one undirected and four directed arcs.The example uses the mutual-information conditional-independence test with alpha threshold 0.05 and performs 43 tests.
- Constraint-based learning: IAMB, Fast-IAMB, and Interleaved IAMB return the same partially directed network structure as Grow-Shrink.The compare results are TRUE for each of the three additional constraint-based networks.
- Score-based learning: Hill-climbing produces a completely directed six-node network with five directed arcs and differs from the constraint-based network in the orientation of A–B.The hill-climbing example uses the AIC score and performs 40 tests.
- Network inspection: bnlearn compares, plots, edits, and inspects network structures through functions such as compare, plot, graphviz.plot, modelstring, arcs, and adjacency-related utilities.Graphviz plotting is described as producing better output for large graphs, and arc-editing operations preserve acyclicity.
- Comparing structures: The differing A–B orientation does not change the network score because the implemented scores are score equivalent except K2.Prior information can therefore constrain the direction through blacklist or whitelist arguments.
6. Practical examples
The ALARM example compares structure-learning algorithms and conditional-independence tests, while the marks example illustrates continuous-data analysis through conditional-independence testing. bnlearn recovers the ALARM network closely and shows improved recovery with nonparametric tests.
- The ALARM network: The ALARM network contains 37 discrete variables and 46 arcs, with 20,000 generated observations for evaluating structure-learning algorithms.
- The ALARM network: Every implemented learning algorithm except mmpc recovers the ALARM network to within a few arcs and arc directions.
- The ALARM network: The constraint-based algorithms perform similar numbers of conditional-independence tests, while hill climbing performs a similar number of score comparisons despite learning about ten more arcs.
- The ALARM network: Nonparametric Pearson’s X2 tests miss five arcs, compared with 12 missed by parametric tests, because sparse contingency tables affect the ALARM data.
- The examination marks data set: The marks data set contains examination scores for 88 students across five subjects, whose dependence structure informs how the marks are combined or averaged.
- The examination marks data set: Under multivariate normality, partial-correlation testing finds weak conditional associations for MECH and ANL given ALG and for STAT and VECT given ALG.The reported correlations are 0.0352 (p = 0.7459) and 0.0527 (p = 0.628), respectively.
7. Other packages for learning Bayesian networks
Several R packages support Bayesian-network structure learning or parameter fitting, but bnlearn is presented as more versatile because it offers multiple learning algorithms and broader methodological support.
- R packages including pcalg, deal, and the gRbase/gRain/gRc suite provide Bayesian-network learning, parameter fitting, manipulation, or inference capabilities.pcalg implements PC, deal uses hill climbing for mixed data, and the gRbase/gRain/gRc suite provides exact and approximate inference.
- bnlearn is more versatile than deal and pcalg because those packages implement only one learning algorithm.Both deal and pcalg can handle discrete and continuous data despite their single-algorithm focus.
- The PC algorithm has poorer speed and accuracy than newer constraint-based algorithms such as Grow-Shrink and IAMB.This comparison motivates bnlearn’s inclusion of a wider range of structure-learning methods.
- bnlearn provides a wider selection of network scores and conditional-independence tests than the compared packages.These options support both learning algorithms and independent use.
8. Conclusions
bnlearn provides free Bayesian-network structure-learning algorithms in R, with optimizations, parallel computing, and reusable scores and conditional-independence tests. It supports flexible analysis across data types and algorithm–criterion combinations.
- bnlearn implements Bayesian-network structure-learning algorithms from recent literature as a free R package.The implementations include algorithmic optimizations and support for parallel computing.
- The package provides many score functions and conditional-independence tests for learning algorithms and independent use.
- bnlearn handles both discrete and continuous data for experimental data analysis.
- Users can combine implemented learning algorithms with network scores or conditional-independence tests according to the data at hand.Score-based algorithms use network scores, while constraint-based algorithms use conditional-independence tests.
- A single bn object class and utility functions support descriptive statistics and basic inference on learned networks.
Affiliation:
The document identifies the Journal of Statistical Software and the American Statistical Association, alongside publication metadata fields.
- The paper is associated with the Journal of Statistical Software.
- The American Statistical Association is identified as the publisher.
- The publication metadata lists volume, issue, submission date, and acceptance date fields.