Dirichlet Process Mixtures within Semiparametric Bayesian Networks: A Novel Conditional Density Estimation Approach

• Bayesian networks factorize joint distributions into lower-dimensional CPDs, reducing parameter-estimation demands while supporting probabilistic inference and uncertainty quantification.Explicit probabilistic models also support tasks beyond learning a predictive function.Source: §1 · p. 2
• KDE-based CPDs can become computationally demanding as sample size grows, while their convergence behavior also worsens with increasing dimensionality.These limitations can affect CPD accuracy and computational efficiency in practice.Source: §1 · p. 3
• DPM-based CPDs cluster similar observations into fewer mixture components, reducing density-evaluation cost while retaining non-parametric flexibility.The compact representation is particularly advantageous as data dimensionality increases.Source: §1 · p. 3
• Across synthetic and real-world experiments, DPM-SPBNs improve density estimation, substantially lower inference costs, and maintain moderate training times.Source: §1 · p. 3

2.1 Dirichlet Process Mixture

• A DPM draws observation-specific latent parameters from a shared DP measure; its almost-sure discreteness allows shared component parameters and finitely many occupied clusters in finite samples.Source: §2.1 · pp. 3–4
• DPMs share parameters among observations in the same cluster, yielding C ≪ N components and lower density-evaluation cost than KDE’s one-kernel-per-observation representation.This sharing may also provide more accurate estimates in higher-dimensional settings.Source: §2.1 · p. 5
• Predictive density estimation must account for uncertainty in latent parameters and partitions, but exact summation over all partitions is intractable because their count grows super-exponentially.Consequently, DPM inference relies on approximate methods.Source: §2.1 · pp. 5–6
• DPM inference methods trade computational cost against posterior uncertainty: MCMC can be expensive, variational inference offers scalability, and hard-clustering methods gain efficiency by reducing posterior averaging.Source: §2.1 · pp. 6–7

2.2 Bayesian networks

• A Bayesian network combines a DAG with local CPDs, factorizing the joint distribution into local relationships that are more tractable to estimate.Source: §2.2 · p. 7
• Continuous-variable CPDs range from parametric models to flexible non-parametric approaches, which can carry greater computational costs and modeling trade-offs.The reviewed non-parametric options include Gaussian processes, infinite mixtures, KDEs, and copula-based models.Source: §2.2 · pp. 7, 9
• CPD choices can affect the assumptions and computational procedures used for structure learning, which includes constraint-based, score-based, and hybrid approaches.Source: §2.2 · p. 9

3 DPM-SPBN

• DPM-SPBN replaces KDE for non-parametric CPDs while retaining parametric CPDs in the same network.The original SPBN formulation uses linear Gaussian CPDs for its parametric component.Source: §3 · p. 9, §3.1 · p. 10

3.1 CPD learning

• MCMC prediction averages densities across retained posterior states, while learning can require thousands of iterations evaluating every observation against active clusters, a burden that can constrain learning multiple CPDs.Accounting for multiple posterior states can also offset the computational benefit of having fewer mixture components than CKDE kernels.Source: §3.1 · p. 11
• Greedy sequential hard clustering depends on observation order, while evaluating only a limited number of permutations trades computational cost against the chance of finding a better-performing partition.Exhaustively evaluating permutations grows factorially with dataset size.Source: §3.1 · p. 11
• CAVI and memoized CAVI approximate DPM posterior inference, balancing computational efficiency with posterior uncertainty rather than using MCMC or a single hard-cluster partition.CAVI uses a mean-field approximation and finite stick-breaking truncation; memoized CAVI processes batches and reuses computed quantities.Source: §3.1 · pp. 11–12
• The DPM-based CPD estimates the joint distribution of a child and its parents, then conditions on the parents to obtain the conditional density analytically.The conjugate Gaussian formulation gives a finite mixture of multivariate Student’s t-distributions, avoiding numerical integration.Source: §3.1 · p. 12

3.2 Structure learning

• DPM-SPBN retains HC and PC structure-learning adaptations, replacing CKDE CPDs with DPM-based CPDs and preserving HC’s TYPE-CHANGE operator.The operator lets a node switch between parametric LG and non-parametric CPDs; the PC procedure retains non-parametric conditional-independence tests and uses HC with TYPE-CHANGE to select CPD types.Source: §3.2 · p. 13
• Because non-parametric CPDs have an unclear number of effective parameters, structure search uses cross-validated log-likelihood and stops when the separate validation score degrades.Candidate moves are scored within the training data; accepted graphs are reestimated on all training data before validation.Source: §3.2 · p. 13

4 Experiments

• The authors extend PyBNesian to support DPM-SPBN parameter and structure learning, density evaluation, and sampling, and use paired median permutation tests with Bergmann-Hommel correction for comparisons.The experiments cover fixed-topology CPD estimation as well as CPD effects on structure learning.Source: §4 · pp. 14–15

4.1 Parameter learning experiments

• At least one DPM-SPBN inference method achieves the best median absolute LL error in 59 of 60 synthetic parameter-learning scenarios, with significant differences in those 59 cases.CAVI generally performs best at smaller sample sizes, while Memoized tends to lead as sample size increases; GPCAVI is not best in any scenario.Source: §4.1 · p. 17, Table 2 · p. 18
• DPM-SPBN reduces prediction time by approximately 10^2–10^3 times versus alternative BN methods, at the cost of higher training times; Memoized costs rise with sample and network size.NR has training times approximately 10^2–10^3 times lower than the proposed methods, but lower density-estimation accuracy; CAVI and GPCAVI training remain feasible.Source: §4.1 · pp. 17, 20, Figure 6 · p. 19
• DPM-SPBN methods outperform their corresponding standalone DPM methods in almost all pairwise comparisons and achieve the best performance in all 60 scenarios, significantly in every case.The results support the benefit of BN factorization over fitting a single DPM to the full joint distribution in these settings.Source: Figure 7 · p. 20, §4.1 · pp. 20, 22

4.2 Structure learning experiments

• With HC, CAVI and GPCAVI achieve the best SHD in approximately 75% of cases; NR leads most remaining structural comparisons, especially for larger networks, while structure and density rankings differ.NR does not achieve the best density-estimation accuracy in these cases; UCV achieves it in a few cases but has the best SHD in only one comparison. CAVI is generally slightly better than GPCAVI across both criteria.Source: §4.2 · pp. 22–23

4.3 Real data experiments

• Real-world evaluation compares NR and UCV with DPM-SPBN using CAVI and GPCAVI, learns unknown structures with HC and cross-validation, and assesses fit using 10-fold LL distributions.Absolute LL error cannot be computed without a known data-generating distribution, and UCV results are omitted for MetroPT and Gas Emission because of computational cost.Source: §4.3 · p. 23
• DPM-SPBN shows generally superior performance across most datasets regardless of inference method, with tighter and more stable fold distributions than the baselines.SPBN-CAVI and SPBN-GPCAVI perform similarly, though GPCAVI is slightly worse in a few isolated cases.Source: §4.3 · p. 23
• UCV outperforms DPM-SPBN in QSAR Fish Toxicity, Yeast, Wine Quality-White, and Travel Ratings, remaining competitive at low-to-medium sample sizes and variable counts despite higher fold variance.These results are broadly consistent with previous experiments.Source: §4.3 · p. 23
• NR is outperformed in every scenario except Waveform, where all four methods have similar median LL.Source: §4.3 · p. 23

5 Conclusions and future research

• DPM-SPBN clearly outperforms the study’s considered state-of-the-art alternatives, with CAVI showing the most consistent overall performance and GPCAVI broadly similar.The authors identify CAVI and GPCAVI as the most practical choices when parameter estimation and structure learning are both considered.Source: §5 · p. 27
• Memoized’s adaptive truncation can improve density estimation on larger datasets and complex distributions, but its higher cost limits use in repeated HC structure learning.It was beneficial primarily with a fixed network structure and was not included in the full HC experiments.Source: §5 · p. 27
• DPM-SPBN consistently achieves better density estimation than standalone DPMs under the experimental conditions, supporting BN factorization for continuous-variable JPD modeling.The authors also cite real-world results as support for DPM-SPBN as an extension of SPBNs.Source: §5 · p. 27
• Future work could extend the framework to online, mixed discrete-continuous, and time-sliced data, compare alternative DPM inference methods, and investigate other structure-learning procedures.The proposed investigations include relationships among CPD estimation, structure learning, and structural recovery.Source: §5 · p. 27

Declaration of generative AI and AI-assisted technologies in the writing process.

• The authors used ChatGPT to improve text coherence and clarity, then reviewed and edited the content and assume full responsibility for it.Source: Declaration of generative AI and… · p. 28

Research data statement

• Publicly available UCI datasets, author-developed synthetic datasets, the modified PyBNesian package, and experiment code are accessible through the stated repositories.Source: Research data statement · p. 28

Funding sources

• The work was partially supported by the listed Ministry of Science, Innovation and Universities projects and by the Autonomous Region of Madrid.The named projects are AEI/MICIU/10.13039/501100011033/PID2025-170500NB-100, PLEC2023-010252/MIG-20232016, ELLIS Unit Madrid, and TEC-2024/COM-89.Source: Funding sources · p. 28