PaperlayerView exact revision on arXiv

Source-linked AI summary

Semi-Linear Gaussian Bayesian Networks: Enhancing Flexibility in Node Dependencies via penalized B-Splines

Aitor Oviedo-Madrid, Mabel Morales-Otero, Jose Gonzalez-Gomariz, Ruben Armañanzas

arXiv:2610.30838v1cs.AIcs.LG

TL;DR

Continuous Bayesian networks face a gap between linear Gaussian models that restrict nonlinear relationships and more flexible semiparametric models with higher computational cost. SLGBN combines linear and nonlinear Gaussian conditional models, and SLGBN with EBIC* provides the most robust performance across the considered settings, though exact DAG recovery is not generally guaranteed.

  • Problem

    Linear Gaussian models restrict conditional relationships to be linear, while semiparametric alternatives increase computational complexity, leaving a modeling gap for approximately Gaussian variables with nonlinear dependencies.

    Source: §1 · p. 2

  • Method

    SLGBN combines linear and nonlinear Gaussian conditional models while retaining Gaussian conditional distributions, and uses greedy hill-climbing to learn network structure and node types.

    Source: §1 · p. 2, §3.3.2 · p. 9

  • Results

    SLGBN with EBIC* provides the most robust performance across the considered settings, with a strong data fit and a useful compromise between likelihood maximization and structural accuracy.

    Source: §5.3 · p. 16

  • Takeaways & Limitations

    The learned structures capture a substantial portion of true dependencies and identify many relevant conditional relationships, although exact recovery of the generating DAG is not generally expected.

    Source: §5.3 · p. 16

    Greedy hill-climbing may converge locally, and identifiability limits prevent guarantees of recovering the true DAG from observational data alone.

    Source: §3.3.1 · p. 9, §5.3 · p. 16

Abstract

from arXiv · show

Bayesian networks for continuous variables often rely on restrictive assumptions: linear Gaussian models lack flexibility to capture nonlinear dependencies, while non-parametric approaches provide greater flexibility at a high computational cost. This creates a gap for modeling scenarios where variables are approximately Gaussian but exhibit nonlinear relationships. We introduce Semi-Linear Gaussian Bayesian Networks (SLGBN), a novel class of continuous Bayesian networks designed to bridge this gap, preserving Gaussian conditional distributions while introducing nonlinear conditional mean functions. The proposed framework provides a unified probabilistic framework combining linear Gaussian and nonlinear Gaussian conditional models within the same Bayesian network Nonlinear conditional mean functions are estimated using generalized additive models (GAMs) with penalized B-splines. We present parameter estimation procedures for both types of nodes, as well as a greedy hill-climbing algorithm for learning the network structure under the SLGBN framework. To assess its performance, we compare SLGBN with other continuous Bayesian network models, including semiparametric Bayesian networks (SPBN) and Linear Gaussian Bayesian networks (LGBN). Experimental results evaluate both goodness of fit and the ability to recover the underlying network structure. The results show that, in the presence of nonlinear Gaussian relationships, SLGBN achieves better fit and improved structure recovery compared to alternative models, particularly when combined with the Extended Bayesian Information Criterion (EBIC*) score. Furthermore, by retaining Gaussian conditional distributions, SLGBN offers computational advantages over fully semiparametric approaches.

1 Introduction

Continuous Bayesian networks face a trade-off between restrictive Gaussian models and more flexible but costly semiparametric approaches. SLGBN combines linear and nonlinear Gaussian conditional models, and experiments report improved fit and structure recovery for nonlinear Gaussian relationships.

1 Introduction

  • Continuous-variable inference and structure learning can require restrictive conditional distributions and complex likelihood estimation, while discretization may lose information or alter data features.These challenges motivate models that retain continuous-variable structure without relying on discretization.Source: §1 · pp. 1–2
  • LGBNs are computationally attractive but assume linear conditional means, whereas SPBNs capture nonlinear dependencies at higher computational and inference cost.The gap is especially relevant when variables are approximately Gaussian but their relationships are nonlinear.Source: §1 · p. 2
  • SLGBN combines linear and nonlinear Gaussian conditional models in one framework, preserving Gaussian conditional distributions while modeling nonlinear means with GAMs and penalized B-splines.The framework includes parameter estimation for both node types and a greedy hill-climbing algorithm for learning structure and node types.Source: §1 · pp. 2–3
  • For approximately Gaussian variables with nonlinear dependencies, SLGBN improves goodness of fit and structure recovery while balancing modeling flexibility and computational complexity.The paper evaluates these outcomes against LGBN and SPBN.Source: §1 · p. 3

2 Background

The background introduces Bayesian networks and continuous-variable reference models, then presents GAMs and penalized B-splines as the basis for SLGBN’s nonlinear nodes.

2 Background

  • The background presents Bayesian networks, LGBN, SPBN, GAMs, and penalized B-splines as foundations for the proposed framework.GAMs and penalized B-splines support estimation of SLGBN’s nonlinear nodes.Source: §2 · p. 3

2.1 Bayesian Networks

  • A Bayesian network represents a joint distribution through node-wise conditional distributions given each node’s parents in a DAG.The graph encodes the parent relationships used in the factorization.Source: §2.1 · p. 3

2.2 Linear Gaussian Bayesian Networks

  • In an LGBN, all conditional distributions are linear Gaussian; joint Gaussianity enables closed-form marginal and conditional distributions for inference.The account assumes jointly Gaussian parents and notes Gaussian closure under marginalization and conditioning.Source: §2.2 · p. 3

2.3 Semiparametric Bayesian Networks

  • SPBN mixes linear Gaussian and nonparametric conditional distributions, using conditional kernel density estimation for nonparametric nodes, with higher computational cost and limited scalability.Its flexibility allows it to capture nonlinear relationships.Source: §2.3 · pp. 3–4

2.4 Generalized Additive Models

  • GAMs replace linear predictor terms with smooth functions of predictors, allowing nonlinear conditional dependencies while retaining conditionally Gaussian distributions.Each parent contributes a smooth function in the additive predictor.Source: §2.4 · p. 4

2.5 B-Splines

  • B-splines express functions using basis functions, but can overfit when knot counts are large, so penalized coefficient estimation controls smoothness.The basis functions are linear in their coefficients.Source: §2.5 · p. 4

3 Semi-Linear Gaussian Bayesian Networks

SLGBN combines linear-Gaussian and nonlinear-Gaussian nodes in one continuous Bayesian network, with nonlinear conditional means modeled additively using penalized B-splines. The section presents parameter estimation, score-based structure learning, and a greedy search procedure, while noting identifiability and computational limits.

3.1 Definition

  • Nonlinear nodes use additive GAM conditional means with penalized B-splines, while linear nodes retain linear-Gaussian conditionals.Linearity is defined locally with respect to a node’s direct parents; making all nodes linear reduces SLGBN to LGBN.Source: §3.1 · p. 5

3.2 Parameter Estimation

  • Linear-node coefficients are estimated by OLS, while nonlinear-node means use penalized least squares and their variances use maximum likelihood.The spline penalty λ controls smoothness; λ = 0 gives the traditional OLS approach, and larger values produce smoother functions.Source: §3.2 · pp. 6–7

3.3 Structure Learning

  • BIC penalizes free parameters, while EBIC* adds a global penalty based on the total number of edges.EBIC* retains decomposable likelihood and local penalty terms; its global graph penalty can be updated efficiently during search.Source: §3.3 · p. 8

3.3.1 Identifiability and Score Equivalence

  • Heterogeneous local parameterizations mean SLGBN scores need not be equivalent for Markov-equivalent DAGs, so exact generating-DAG recovery is not generally guaranteed from observational data.The paper therefore evaluates learned structures primarily with MEC-level or skeleton-based metrics.Source: §3.3.1 · p. 9

3.3.2 Greedy Hill-Climbing for SLGBN.

Algorithm 1 Greedy Hill-Climbing Algorithm for SLGBN

  • The search starts from an initial graph and uses patience and a tabu list to explore candidates before returning the best-scoring structure found.Nodes without parents use sample mean and variance regardless of assigned type; their type-change operator is unavailable until they have a parent.Source: Algorithm 1 Greedy Hill-Climbing… · p. 10, Figure 1 · p. 10
  • Even when a structure or equivalence class is identifiable, the restricted greedy search can stop at a local optimum rather than the global optimum.The algorithm explores only a restricted neighborhood at each iteration.Source: Algorithm 1 Greedy Hill-Climbing… · pp. 10–11

3.3.3 Asymptotic Time Complexity

  • SLGBN and LGBN scale linearly with sample size n, whereas SPBN scales quadratically; SLGBN’s per-iteration complexity is O(p^2n(dl)^2).For bounded parent count d and number of spline bases l, SLGBN’s stated complexity simplifies in practice to linear scaling with n.Source: §3.3.3 · pp. 11–12

4 Methods

The experiments assess fit and graph recovery on synthetic continuous networks using LGBN, SPBN, and SLGBN configurations. They compare validation log-likelihood and skeleton-based structural metrics across training sample sizes and repeated datasets.

4.1 Data

  • Three synthetic networks are used: manually developed B5 and B8, and B_insu, an insurance network given an ad hoc continuous parametrization.The B_insu parametrization combines linear-Gaussian and nonlinear-Gaussian relationships; its original variable semantics are not preserved.Source: §4.1 · p. 12
  • Each network is evaluated at training sample sizes of 200, 1000, 5000, and 10000, using 20 generated training datasets per size and an independent 1000-sample validation set.The learned structures are compared with the known generating structure.Source: §4.1 · p. 12

4.2 Models

  • The comparison includes LGBN-BIC, LGBN-EBIC*, SPBN-CVL, SLGBN-BIC, and SLGBN-EBIC*.SPBN uses cross-validated log-likelihood for learning, while BIC and EBIC* guide LGBN and SLGBN.Source: §4.2 · pp. 12–13

4.3 Algorithm

  • SLGBN is run both from the true graph with only type changes allowed and from an empty graph with all operators enabled.The first setup tests node-type identification with known structure; the second tests full structure learning from scratch.Source: §4.3 · p. 13

4.4 Evaluation Metrics

  • Fit is measured by validation log-likelihood, while structural recovery uses skeleton-based SHD and, for SLGBN and SPBN, Type Hamming Distance.Precision and recall are also calculated for each learned structure; results are averaged over 20 seeds per sample size.Source: §4.4 · p. 14

5 Results

In fixed-structure learning, SLGBN leads on fit and node-type recovery; in full structure learning, it performs best overall, with SLGBN-EBIC* strongest on structural distance. The discussion cautions that better likelihood does not ensure better structural recovery and that true-DAG recovery has identifiability limits.

5.1 Fixed structure learning

  • In fixed-structure learning, SLGBN approaches ground-truth likelihood and obtains the lowest THD; it identifies true node types in both simpler networks.For the more complex network, THD steadily falls with sample size and reaches zero on the largest dataset; SPBN performs slightly worse, while LGBN cannot change node types.Source: §5.1 · p. 15, Figure 4 · p. 15

5.2 Full structure learning

  • In full structure learning, SLGBN achieves the best overall log-likelihood, followed by SPBN and LGBN, while fit approaches ground truth as sample size increases.The resulting structures under BIC and EBIC* still produce very similar data fit.Source: §5.2 · p. 15
  • SLGBN-EBIC* consistently outperforms other models on SHD, with higher precision while maintaining competitive recall, especially at larger sample sizes.Across models, recall tends to rise with sample size while precision falls, particularly in more complex networks.Source: §5.2 · pp. 15–16

5.3 Discussion on results

  • Likelihood rises with sample size, but SHD may stay flat or worsen because structural recovery is not directly optimized by likelihood-based scoring.The discussion attributes the divergence to the learning objective's focus on likelihood rather than SHD.Source: §5.3 · p. 16
  • With a fixed graph, BIC and EBIC* produce no observed differences because EBIC*'s global graph-complexity penalty is constant across candidate models.Both scores therefore optimize the same effective objective and tend toward identical solutions in this setting.Source: §5.3 · p. 16
  • Full structure search is computationally infeasible to exhaust, and greedy hill-climbing can settle at a local optimum; observational data also cannot generally guarantee recovery of the true generating DAG.Identifiability, when it holds, is asymptotic and may not apply at finite sample sizes.Source: §5.3 · p. 16
  • As sample size grows, recovering more true edges can also add false positives; EBIC* improves precision-recall balance and SHD in many cases, but does not reverse the overall trend.The false-positive trade-off is more pronounced in complex networks with more weak dependencies.Source: §5.3 · p. 16
  • Across the considered settings, SLGBN-EBIC* offers the strongest compromise between likelihood fit and structural accuracy, although learned networks are informative rather than guaranteed to recover the true DAG.The learned structures capture a substantial portion of dependencies and many relevant conditional relationships.Source: §5.3 · p. 16

6 Conclusion

SLGBN combines linear Gaussian and nonlinear Gaussian conditional models while retaining conditional Gaussian distributions. Experiments report strong fit and improved recovery, particularly with EBIC*, alongside limits tied to network size, hyperparameter tuning, and model assumptions.

6 Conclusion

  • SLGBN extends linear Gaussian Bayesian networks by allowing nonlinear relationships while maintaining conditional Gaussian distributions.Source: §6 · p. 16
  • SLGBN achieved strong goodness of fit and improved recovery of node types and network structure, especially with the EBIC* score.Source: §6 · p. 17
  • Performance was especially favorable when the underlying network structure was known, but learning became more challenging as network size grew for all considered models.Source: §6 · p. 18
  • The implementation specified spline-basis counts and penalization parameters a priori; node-specific grid search could improve performance but substantially increase computational cost.Such tuning may be feasible for known structures or small networks, but quickly becomes prohibitive in larger or more complex settings.Source: §6 · p. 18
  • The framework's scope assumes conditional Gaussianity and additive dependencies between parent variables; robustness beyond these assumptions remains open.Source: §6 · p. 18

Appendix A Additional Figures

Figure 6 depicts BN_insu, the insurance network from bnlearn, identified as the third synthetic network.

Appendix A Additional Figures

Appendix B Additional Tables

The appendix provides a B_insu parametrization table and fixed- and full-structure results tables for B5, B8, and B_insu.

Appendix B Additional Tables

Loading 2610.30838v1…