PaperlayerView exact revision on arXiv

Source-linked AI summary

DAG Structure Learning with Node-Varying Heavy-Tailed Disturbances

Robin Francis, Sundeep Prabhakar Chepuri

arXiv:2610.13304v1cs.LG

TL;DR

DAG learning remains challenging when node-wise disturbances vary in scale and tail behavior. This paper proposes node-varying Student-t DAGMA, jointly estimating graph structure and disturbance parameters, and reports improved synthetic recovery and lower test NLL on financial data.

  • Problem

    Learning DAGs when disturbance scale and tail behavior differ across nodes remains a challenge for continuous methods that address robustness or scale heterogeneity alone.

    Source: §1 · p. 1

  • Method

    Node-varying Student-t DAGMA combines node-specific scales and DoF parameters with DAGMA’s acyclicity formulation, using a Gaussian scale-mixture representation for iteratively reweighted estimation.

    Source: §1 · p. 1, §4 · pp. 2–3

  • Results

    Synthetic experiments show improved DAG recovery under heterogeneous heavy-tailed disturbances, and financial-data experiments report the lowest test NLL among evaluated methods.

    Source: §1 · p. 1, §7 · p. 4

  • Takeaways & Limitations

    In the evaluated settings, the method recovers graphs under heterogeneous heavy-tailed disturbances and achieves lower test NLL on financial data while recovering a sparse graph.

    Source: §7 · p. 4

    The disturbance model is limited to independent, node-wise Student-t errors with node-specific scale and DoF parameters.

    Source: §3 · p. 2

Abstract

from arXiv · show

DAG structures can be learned from observations by incorporating acyclicity via smooth constraints, such as DAGMA's log-determinant function. Typically, observations and the underlying DAG are related through linear structural equation models, with Gaussianity assumption on the disturbances at each node. Extending DAG learning methods that account for scale and tail behavior of disturbances that vary across nodes remains a challenge. We propose a node-varying Student-$t$ DAGMA that jointly estimates the graph structure, node-specific disturbance scales, and the degrees-of-freedom parameter. A Gaussian scale-mixture representation yields an iteratively reweighted formulation that down-weights large standardized residuals. Building on existing results, we also establish identifiability of the graph and disturbance parameters. Synthetic experiments demonstrate improved structural recovery under heterogeneous heavy-tailed disturbances, and experiments on real finance data show that the proposed method yields lower test negative log-likelihood than the evaluated baselines.

1. INTRODUCTION

Learning linear DAGs is difficult when disturbances vary in both scale and tail behavior across nodes. This work proposes node-varying Student-t DAGMA and reports improved synthetic graph recovery and lower held-out NLL on financial data than evaluated baselines.

1. INTRODUCTION

  • Linear SEMs represent direct causal effects with DAG edges, but learning the graph is challenging because acyclicity imposes a combinatorial constraint.Source: §1 · p. 1
  • Existing approaches address robustness or scale heterogeneity, but scale adaptation alone does not capture cross-node tail differences; the paper motivates explicit modeling of both.Source: §1 · p. 1
  • The method assigns each node its own Student-t scale and degrees-of-freedom parameters, with smaller DoF accommodating heavier tails and more strongly downweighting extreme residuals.Source: §1 · p. 1
  • It integrates node-specific disturbance modeling into DAGMA’s log-determinant acyclicity formulation and central-path optimization to jointly estimate the graph, scales, and DoF parameters.Source: §1 · p. 1
  • The authors establish identifiability of the graph and node-specific Student-t parameters under the stated model assumptions, building on ICA and LiNGAM results.Source: §1 · p. 1
  • Synthetic experiments show improved DAG recovery, while financial-data experiments show lower held-out NLL than the evaluated baselines.Source: §1 · p. 1

2. PRELIMINARIES

The preliminaries introduce the DAGMA acyclicity formulation and its Gaussian least-squares specialization. They motivate an alternative loss because squared residuals penalize extreme observations heavily.

2. PRELIMINARIES

  • A DAG has no directed cycles, and its weighted adjacency matrix has zero diagonal; under a topological ordering, its edges point from earlier to later nodes.Source: §2 · p. 2
  • The paper represents observations as linear SEMs with structural disturbances independent across nodes and identically distributed and independent across observations.Source: §2 · p. 2
  • DAGMA uses a smooth log-determinant penalty that vanishes for acyclic graphs and estimates the adjacency matrix through sequential optimization problems.Source: §2 · p. 2
  • Its optimization balances data fit and sparsity against the acyclicity penalty, whose relative weight increases as μ_k decreases across iterations.Source: §2 · p. 2
  • Under independent Gaussian disturbances with a fixed common variance, the likelihood-based data-fitting loss is proportional to a least-squares penalty.Source: §2 · p. 2
  • Because the squared-residual penalty assigns large costs to extreme residuals, the paper motivates an alternative loss for heavy-tailed disturbances.Source: §2 · p. 2

3. PROBLEM MODELING AND FORMULATION

The formulation models independent node-wise disturbances with Student-t distributions, allowing both scale and tail behavior to differ across nodes. It jointly estimates the graph and disturbance parameters under sparsity and acyclicity constraints.

3. PROBLEM MODELING AND FORMULATION

  • Each node has an independent Student-t disturbance, with scale σ_j representing magnitude and DoF ν_j controlling tail behavior; both may vary across nodes.Source: §3 · p. 2
  • The objective jointly estimates the adjacency matrix, scales, and DoF parameters by minimizing the Student-t negative log-likelihood subject to graph sparsity and acyclicity.Source: §3 · p. 2
  • A Gaussian scale-mixture representation enables an EM-based iteratively reweighted least-squares algorithm, avoiding direct optimization of the non-quadratic Student-t loss.Source: §3 · p. 2

4. NODE-VARYING STUDENT-t DAGMA ALGORITHM

The method uses a Gaussian scale-mixture representation of Student-t disturbances to create a weighted least-squares surrogate, then alternates updates to the graph and node-specific disturbance parameters within DAGMA’s central-path framework.

4. NODE-VARYING STUDENT-t DAGMA ALGORITHM

  • Large standardized residuals receive smaller reliability weights, reducing their influence on the subsequent graph update.The EM formulation replaces the non-quadratic Student-t loss with a quadratic weighted least-squares surrogate.Source: §4 · p. 2
  • The graph update replaces DAGMA’s data-fitting term with a weighted surrogate while retaining sparsity regularization and an acyclicity penalty.The graph-dependent EM objective holds the current weights and scale estimates fixed during this update.Source: §4 · p. 3
  • For each fixed central-path parameter, inner EM iterations alternate updates to the graph, scales, and DoF; decreasing the path parameter progressively emphasizes acyclicity.The procedure warm-starts the next outer iteration with the converged estimates from the current one.Source: §4 · p. 3
  • The EM procedure updates each node’s squared disturbance scale using reliability-weighted squared residuals from the updated graph.The scale is estimated separately for each structural equation, so it need not be known in advance.Source: §4 · p. 3
  • Each node’s DoF parameter is updated by solving a scalar equation, with bisection over a bounded interval and rejection of updates that increase the EM surrogate.Smaller DoF corresponds to heavier tails and stronger downweighting of large standardized residuals; as DoF tends to infinity, the weights approach one.Source: §4 · p. 3

5. IDENTIFIABILITY

Under stated linear SEM and disturbance assumptions, the paper establishes identifiability of the graph and node-specific Student-t parameters from the observational distribution.

5. IDENTIFIABILITY

  • The proof treats the linear SEM representation as an ICA model and uses identifiability of its mixing matrix up to permutation and nonzero scaling.The argument applies the standard ICA and LiNGAM identifiability results.Source: §5 · p. 3
  • The graph matrix, node-specific scales, and DoF parameters are identifiable from the observational distribution when the stated model assumptions hold.The result assumes independent node-wise Student-t disturbances with 2 < νj < νmax < ∞, observed variables, and no latent confounders.Source: §5 · p. 4
  • Acyclicity and the unit diagonal of the inverse mixing matrix resolve permutation and scaling ambiguities, yielding unique graph and disturbance parameters.Once the graph is identified, the disturbances and their node-specific Student-t parameters are uniquely determined.Source: §5 · p. 4

6. NUMERICAL EXPERIMENTS

On the 49 Industry Portfolios test set, the proposed method has the lowest reported NLL, while its NRMSE is close to Huber-DAGMA’s. The section also describes a synthetic DAG-recovery evaluation under heterogeneous Student-t disturbances.

6. NUMERICAL EXPERIMENTS

  • The synthetic evaluation generates 500 observations from a 10-node Erdős–Rényi DAG with average degree 2 and heterogeneous node-wise Student-t disturbances.Graph recovery is evaluated using SHD, TPR, FDR, and F1-score; the supplied passages do not report their results.Source: §6 · p. 4
  • The financial evaluation uses daily returns for 49 U.S. industry portfolios from January 2000 to December 2019, split chronologically into 60% training, 20% validation, and 20% test data.Test performance is assessed with NRMSE and NLL, alongside the recovered DAG’s edge count.Source: §6 · p. 4
  • The proposed method achieves the lowest reported test NLL, with NRMSE 0.465 versus 0.464 for Huber-DAGMA, and recovers 15 edges.Its edge count is below Huber-DAGMA’s 18 and CoLiDE-NV’s 23, but above DAGMA’s 14.Source: §6 · p. 4, Table 1 · p. 4

7. CONCLUSIONS

The paper proposes node-varying Student-t DAGMA for heterogeneous node-wise disturbances. Experiments show improved synthetic DAG recovery and the lowest reported test NLL on real-world financial data, with comparable reconstruction error and a sparse recovered graph.

7. CONCLUSIONS

  • The method uses a Gaussian scale-mixture representation to formulate an IRLS problem and jointly estimate graph structure, disturbance scales, and DoF parameters.Source: §7 · p. 4
  • Synthetic experiments demonstrated improved DAG recovery under the heterogeneous heavy-tailed disturbances considered.Source: §7 · p. 4
  • On real-world financial data, the method achieved the lowest reported test NLL, with comparable reconstruction error and a sparse recovered graph.Source: §7 · p. 4

9. COMPLIANCE WITH ETHICAL STANDARDS

The paper reports that no ethical approval was required for its numerical simulation study.

9. COMPLIANCE WITH ETHICAL STANDARDS

  • No ethical approval was required for the numerical simulation study.Source: §9 · p. 5
Loading 2610.13304v1…