PaperlayerView exact revision on arXiv

Source-linked AI summary

DAG-CLIP: A DAG Learning Framework in the Presence of Latent Variables

Siliang Zhang, Yunxiao Chen, Irini Moustaki

arXiv:2610.06086v1stat.MEstat.ML

TL;DR

DAG learning methods often require variables of interest to be directly observed, although many substantive constructs are latent and measured through observed indicators. DAG-CLIP uses composite-likelihood screening and pruning for latent-variable DAG learning with non-Gaussian indicators. The paper establishes consistency for recovering the true DAG's Markov equivalence class and reports effectiveness in simulations and a large-scale educational survey application.

  • Problem

    Many substantive variables are latent constructs measured by observed indicators, while most DAG-learning methods require variables of interest to be directly observed.

    Source: §1.1 · p. 2

  • Method

    DAG-CLIP uses pairwise composite likelihood in smooth acyclicity-constrained screening followed by information-criterion-guided backward pruning within Markov equivalence classes.

    Source: §1.2 · p. 3, §3.3 · p. 15

  • Results

    The paper establishes consistency for recovering the true DAG's Markov equivalence class and reports effectiveness in simulations and a large-scale educational survey application.

    Source: §7 · p. 28

  • Takeaways & Limitations

    The application demonstrates DAG-CLIP's use with complex measurement models in large-scale educational survey data.

    Source: §7 · p. 28

    In the TIMSS application, arrows from achievement and learning-context constructs into the background group are excluded, so the selected graph is conditional on that directional restriction.

    Source: §6.1 · p. 24

Abstract

from arXiv · show

Learning directed acyclic graphs (DAGs) to uncover causal mechanisms has attracted substantial attention in machine learning. While most existing methods focus exclusively on observed variables, many variables of substantive interest are latent constructs defined by statistical measurement models. Such constructs are particularly common in the social and behavioral sciences. In this paper, we propose a general statistical framework for DAG learning when some or all nodes of the graph are latent variables, accommodating non-Gaussian manifest variables such as binary and categorical data in the measurement model. To overcome the computational burden of high-dimensional integration in standard marginal likelihoods, we develop DAG learning via Composite-Likelihood-based screening and Iterative Pruning (DAG-CLIP), a two-step composite-likelihood-based learning algorithm. This algorithm first solves a smooth acyclicity-constrained optimization problem to screen out misspecified DAG structures, and subsequently performs BIC-guided composite-likelihood backward deletion within the Markov equivalence class (MEC) space to identify a sparse DAG. We establish the statistical consistency of the algorithm in recovering the MEC of the true DAG, and demonstrate its effectiveness through extensive simulation studies and a real-world application to large-scale educational survey data.

1 Introduction

The paper targets DAG learning when substantive variables are latent constructs measured by observed indicators, including categorical or mixed-type data. It proposes a composite-likelihood framework combining screening and pruning, with consistency guarantees and evaluation in simulations and an educational survey application.

1.1 Background

  • Substantive variables such as personality traits, cognitive abilities, mental health status, and political attitudes are often latent constructs defined through measurement models.This motivates extending DAG learning beyond methods requiring all variables of interest to be directly observed.Source: §1.1 · p. 2

1.3 Related Literature

  • Existing latent-DAG approaches either learn indicator groupings or restrict given measurement structures, while this work focuses on multiple indicators with categorical or mixed types.The paper also reports, to its knowledge, no prior use of pairwise or other composite likelihood to score and select DAG structures.Source: §1.1 · p. 2, §1.3 · p. 7

1.2 Contributions

  • The framework models latent DAG nodes with prespecified measurement structures that can include multiple categorical or mixed-type indicators.It allows some or all graph nodes to be latent and imposes minimal restrictions on the measurement model, subject to standard smoothness and identifiability requirements.Source: §1.1 · p. 2, §1.2 · p. 3
  • DAG-CLIP combines smooth acyclicity-constrained screening with information-criterion-guided backward deletion over Markov equivalence classes to produce a sparse graph.Its pairwise composite likelihood avoids high-dimensional integration, and the method is named DAG-CLIP.Source: §1.2 · pp. 3–4
  • The paper establishes consistent recovery of the true DAG's Markov equivalence class under a screening-solution accuracy condition weaker than that required for screening alone.Without additional assumptions or interventions, the underlying DAG is identifiable only up to its Markov equivalence class.Source: §1.2 · p. 4
  • Across diverse settings, simulations show that DAG-CLIP substantially outperforms its screening step alone and two-step methods that first fit a measurement model before analyzing estimated factor scores or latent correlations.In the survey application, the resulting structure has fewer than half of possible adjacencies yet predicts held-out responses nearly as well as the saturated model.Source: §1.2 · pp. 4–5

2 Model Framework and Identifiability

The model represents latent variables through an acyclic Gaussian structural equation model and connects them to observed indicators through a measurement model. It discusses measurement-model identifiability, the computational cost of marginal likelihoods, and why the latent DAG is generally identifiable only up to its Markov equivalence class.

2.1 Model Framework

  • The latent variables follow an acyclic Gaussian linear structural equation model, with the support of its coefficient matrix encoding a sparse DAG.The model also allows directly observed variables as DAG nodes.Source: §2.1 · p. 7
  • The measurement model assumes local independence and a parametric conditional distribution for each manifest variable given its associated latent variable.Examples include ordinal graded-response and threshold models, continuous linear factor models, and a representation for observed continuous variables; each manifest variable measures one latent variable in the stated framework.Source: §2.1 · pp. 8–9

2.2 Marginal Likelihood and Identifiability

  • For categorical indicators, the marginal-likelihood integral has no closed form and repeated evaluation over candidate graphs makes direct maximization computationally prohibitive at moderately large latent dimension.Quadrature costs grow exponentially in the number of latent variables, while Monte Carlo integration requires many draws for sufficient accuracy.Source: §2.2 · p. 10
  • Under measurement-model identifiability, the latent correlation matrix is identified from the observed data, but the true DAG is generally identified only up to its Markov equivalence class.With faithfulness, that class is the unique sparsest class of DAGs inducing the latent correlation matrix.Source: §2.2 · pp. 10–11

3 Proposed Method

DAG-CLIP combines sparse DAG screening with backward pruning over Markov equivalence classes, using pairwise composite likelihood and CLBIC. Screening can retain spurious edges; pruning removes them to identify a sparse compatible class.

3 Proposed Method

  • DAG-CLIP screens DAGs with a sparsity penalty, then prunes spurious edges through backward search over Markov equivalence classes.The two steps use pairwise likelihood, a truncated ℓ1 penalty, a smooth acyclicity constraint, and CLBIC.Source: §3 · p. 11

3.1 Technical Tools

  • Pairwise likelihood replaces the marginal density with bivariate margins, limiting each integral to at most two dimensions regardless of latent-variable count.The measurement model must also be identifiable from those bivariate margins.Source: §3.1 · pp. 11–12
  • The truncated ℓ1 penalty screens dense graphs, while a smooth equality constraint enforces acyclicity and allows candidate edge coefficients to be updated jointly.The truncated penalty acts as a continuous surrogate for the ℓ0 norm of the coefficient matrix.Source: §3.1 · pp. 12–13

3.2 Step 1: Screening

  • The screening step returns a DAG compatible with the true distribution with high probability, but may retain spurious edges unless the nonconvex problem is solved nearly globally.The pruning step is intended to remove those edges.Source: §3.2 · p. 14

3.3 Step 2: Iterative Pruning

  • Pruning evaluates one-edge-deletion CPDAG candidates and moves to the candidate with a strictly lower CLBIC, stopping when none improves the current score.The backward search uses the greedy equivalence search edge-removal operator without enumerating DAGs in the current MEC.Source: §3.3 · p. 15
  • Under the stated conditions, pruning retains a compatible CPDAG at each iteration and stops at the true CPDAG with probability tending to one.Wald screening orders candidate removals, and a fit-term lower bound can avoid computing the effective-parameter penalty when a candidate cannot improve the best score.Source: §3.3 · p. 15

4 Large-sample Theory

Under the paper’s assumptions, DAG-CLIP consistently recovers the CPDAG of the true DAG as the sample size grows, for fixed latent dimension and node count. The theory explains how screening supplies a compatible starting class and pruning removes redundant edges while rejecting incompatible deletions.

4 Large-sample Theory

  • The screening and pruning guarantees rely on regularity and tuning conditions, including λn → 0, nλn → ∞, and τn → 0.The computed CLBIC values must also have uniform numerical error oP(log n) across candidates.Source: §4 · pp. 16–17
  • For fixed m and p, DAG-CLIP’s output CPDAG equals the true CPDAG with probability tending to one as n grows, under Assumptions 2, 3, and 4.The same argument extends to full likelihood with BIC under Assumption 1, faithfulness, and corresponding likelihood regularity conditions.Source: §4 · pp. 15, 17
  • With the screening conditions, the initial DAG is compatible with the true distribution with probability tending to one; near-global optimization is additionally required for the stronger screening guarantee.The screening step alone can retain redundant edges, which pruning then removes.Source: §4 · pp. 17–18
  • Pruning reaches and stops at the true CPDAG with probability tending to one because redundant-edge deletions are favored while incompatible deletions are rejected.The true CPDAG is the unique edge-minimal compatible class, and the set of CPDAGs is finite.Source: §4 · p. 18

5 Simulation Studies

The simulations evaluate latent CPDAG recovery across graph families, latent dimensions, and sample sizes. DAG-CLIP achieves stronger recovery than the screening-only variant and three two-step baselines across the reported settings.

5.1 Design and Evaluation

  • The study tests ordinal indicators across three graph families, m=20 or 40, and n=2000 or 4000, comparing DAG-CLIP with screening-only and three two-step baselines.Each design cell has 100 replications; the two-step baselines learn structure from factor scores or estimated latent correlations.Source: §5 · p. 18, §5.1 · p. 19

5.2 Results

  • At m=20 and n=4000, DAG-CLIP's exact-MEC recovery is 0.71 for star, 0.60 for tree, and 0.41 for random; baselines never recover exactly.DAG-CLIP's performance remains very good at m=40, though it slightly worsens.Source: §5.2 · p. 20
  • About one tenth at m=20 and one sixth at m=40 of FS-PC's average SHD: DAG-CLIP has the lowest average SHD and highest skeleton F1 across settings.It also has the highest average compelled-arrow F1 in the random and star families.Source: §5.2 · p. 20, Table 1 · p. 21, Table 2 · p. 22
  • DAG-CLIP recovers nearly all star-family orientations and achieves tree-family skeleton F1 of at least 0.88; GES baselines miss some m=40 runs within 24 hours.Source: §5.2 · p. 21
  • Pruning removes most extra adjacencies retained by screening alone, reducing SHD and increasing skeleton F1 in every setting, with higher exact-recovery rates.Source: §5.2 · p. 21

6 Application to TIMSS 2023

The TIMSS application uses DAG-CLIP to explore relationships among 14 latent questionnaire constructs and two achievement scores, selecting a sparse graph with near-saturated held-out predictive performance. Its fitted structure is interpreted under a restriction that treats resources and experiences as background conditions.

6.1 Data and Graph Nodes

  • The 16-node graph includes 14 latent constructs measured by questionnaire items and two observed achievement scores.The application uses Australia Grade 8 TIMSS 2023 data from 7,997 students in 276 schools; the 69 questionnaire items are binary or ordered categorical, and each latent construct has at least three indicators.Source: §6.1 · pp. 23–24

6.3 Background Knowledge and Interpretation

  • The search excludes arrows into the background group from other groups, while leaving within-group directions and all other directions unrestricted.The selected orientations are discussed subject to this background-knowledge restriction.Source: §6.1 · p. 24, §6.3 · p. 27

6.2 Selected Structure and Held-Out Prediction

  • The selected CPDAG contains 57 of 120 possible adjacencies, 56 directed and one undirected HER–DSE edge; every retained adjacency has a positive deletion-related CLBIC increase, ranging from 0.002 to 3.83.Source: §6.2 · p. 24
  • The resource and experience group sends 23 edges outward; learning-context links favor mathematics-to-science (8 versus 2), and science achievement alone parents mathematics achievement.Each mathematics construct points to its science counterpart except Confidence in Science, which points to Confidence in Mathematics.Source: §6.2 · pp. 25–26
  • The selected CPDAG captures 99.8 percent of the saturated model’s held-out loss reduction with fewer than half its adjacencies.The saturated model lowers mean loss by only 0.039 × 10−3 relative to DAG-CLIP.Source: §6.2 · p. 26, Table 4 · p. 27

6.3 Background Knowledge and Interpretation

  • HER, DSE, SSB, Bullying Exposure, and Confidence in Science point into science achievement, which is the only parent of mathematics achievement.The authors interpret the pathway through science achievement as suggesting that background resources and experiences extend across subjects.Source: §6.2 · p. 26, §6.3 · p. 27
  • The fitted directions associate mathematics achievement with mathematics confidence and science achievement with enjoyment of science; confidence has different roles across subjects.MA → CM is consistent with stronger mathematics performance building confidence; CS → SA suggests confidence may contribute to science achievement.Source: §6.3 · p. 27
  • Achievement–disorder associations differ by subject: MA → DML is −0.654 and SA → DLS is −0.429, while MA → DLS is 0.367 and SA → DML is 0.457.These are estimated standardized path coefficients from the final joint refit.Source: §6.3 · p. 27

7 Discussion

The discussion presents DAG-CLIP as a composite-likelihood framework for learning DAGs with latent constructs and non-Gaussian manifest variables. It reports consistency for recovering the true Markov equivalence class and practical effectiveness in simulations and educational-survey data, while identifying extensions beyond unidimensional measurement.

7 Discussion

  • DAG-CLIP replaces marginal likelihood with pairwise likelihood, screens misspecified DAGs using smooth acyclicity-constrained optimization, then prunes within the Markov equivalence class space.The backward-deletion stage is guided by an information criterion to produce a sparse graph.Source: §7 · p. 28
  • The method is theoretically consistent for recovering the true Markov equivalence class, with simulations and educational-survey application supporting practical effectiveness and scalability.Source: §7 · p. 28
  • The current paper assumes unidimensional measurement, although the authors state this is not a technical requirement and pairwise likelihood can extend to multidimensional designs.Other proposed extensions include alternative acyclicity constraints, incorporating prior direction knowledge, and learning an unknown measurement structure.Source: §7 · p. 28
Loading 2610.06086v1…