PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentiable Structure Learning for Cyclic Linear Gaussian Models with Latent Confounders

Sadegh Khorasani, Ali Najar, Saber Salehkaleybar, Negar Kiyavash

arXiv:2609.38618v1cs.LGcs.AI

TL;DR

Observational structure learning for cyclic linear Gaussian models is complicated by latent confounders and non-unique latent graphs. The paper develops covariance-family-based scoring and Bernoulli-gated optimization, with consistency under identification conditions and a matching global infimum to discrete optimization. Across several latent-variable settings, it reports lower compatibility error than considered baselines.

  • Problem

    Observational learning must handle directed cycles and latent confounders, while existing results do not directly extend to marginalized latent structures with differing parameterizations.

    Source: §1 · p. 1

  • Method

    The paper scores candidate cyclic linear Gaussian models using observed covariance families and a penalized likelihood, then optimizes Bernoulli gates for edges and latent variables.

    Source: §1 · p. 2

  • Results

    Across several settings with latent variables, the method achieves lower bidirectional covariance-family compatibility error than considered baselines and scales to substantially larger observed graphs.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The supported recovery target is consistency up to marginal quasi-equivalence, not unique recovery of the graph or latent count.

    Source: §1 · p. 2, §6 · p. 9

    Recovery up to marginal quasi-equivalence requires additional conditions; the stated assumptions are sufficient, and necessity is not claimed.

    Source: §4.1 · p. 6

Abstract

from arXiv · show

We study causal structure learning from observational data in linear Gaussian structural causal models in the presence of directed cycles and an unknown number of exogenous latent confounders, bounded by a given maximum. We derive the covariance of the observed variables and introduce marginal quasi-equivalence, which characterizes when different causal models share a full-dimensional subset of the observational distributions they can generate. We formulate structure learning as minimization of the Gaussian negative log-likelihood with a logarithmically scaled complexity penalty that counts directed edges and latent variables. For a fixed number of observed variables and a fixed upper bound on latent variables, we establish consistency of global score minimizers up to marginal quasi-equivalence under algebraic faithfulness, structural minimality, and model-overlap assumptions. We parameterize the inclusion of directed edges and candidate latent variables using Bernoulli gates, whose continuous probabilities are optimized jointly with the structural coefficients. Averaging the penalized negative log-likelihood over these gates yields an objective with a closed-form differentiable complexity penalty. We prove that this expected objective has the same global infimum as the corresponding discrete structure-learning objective. Experimental results show that our approach achieves lower recovery error than previous methods in several experimental settings.

1 INTRODUCTION

The paper addresses observational structure learning when causal systems may contain feedback and latent confounding, challenges often handled separately or under narrower assumptions. It develops covariance-based equivalence and learning methods for cyclic linear Gaussian models with bounded latent confounders.

1 INTRODUCTION

  • Cyclic models without latent confounders and latent-confounding methods that retain acyclicity address the two challenges separately, while other approaches impose different graph or data assumptions.The cited mixed-graph restriction excludes reciprocal directed edges and pairs joined by both directed and bidirected edges.Source: §1 · p. 1
  • The framework allows arbitrary directed cycles among observed variables and explicit exogenous latent confounders, whose graph is generally not uniquely identifiable from observed distributions.It compares the covariance families induced by candidate graphs and defines marginal quasi-equivalence through a full-dimensional common subset, even when latent structures differ.Source: §1 · pp. 1–2
  • For fixed observed-variable count and latent-variable bound, global score minimizers are consistent up to marginal quasi-equivalence under additional identification conditions.The conditions include algebraic faithfulness, structural minimality, and full-dimensional model overlap.Source: §1 · p. 2
  • Across several latent-variable settings, the method achieves lower bidirectional covariance-family compatibility error than considered baselines and scales to substantially larger observed graphs.The evaluation includes synthetic cyclic SCMs and simulated GeneNetWeaver networks.Source: §1 · p. 2
  • Bernoulli gates yield a differentiable expected objective with a closed-form complexity penalty and the same global infimum as discrete structure learning.The penalty counts directed edges and latent variables.Source: §1 · p. 2

2 PROBLEM SETTING

The model uses observational, zero-mean Gaussian data and permits feedback among observed variables alongside independent exogenous latent confounders. Eliminating the latent variables yields an observed Gaussian distribution whose covariance combines cyclic structural effects with an effective noise covariance.

2 PROBLEM SETTING

  • The setting observes only the observed variables in i.i.d. samples, with candidate graphs specifying observed edges, latent-to-observed edges, and a bounded latent count.A graph edge permits a coefficient to be nonzero but does not require it to be nonzero.Source: §2 · p. 2, §2.1 · p. 3, §2.3 · p. 4

2.1 NOTATION AND CONVENTIONS

  • The notation defines matrix and set conventions, including positive-definite covariance matrices, matrix norms, and dimension for semialgebraic sets.For symmetric p × p matrices, dimension uses the p(p+1)/2 upper-triangular entries.Source: §2.1 · pp. 2–3

2.2 CYCLIC STRUCTURAL EQUATIONS WITH EXOGENOUS CONFOUNDERS

  • Observed variables may form arbitrary directed cycles provided I_p−B_OO is invertible; latent vertices are exogenous and require at least two permitted edges to observed children.Zero- or one-child latent vertices can be removed without excluding any representable observed distribution.Source: §2.2 · p. 3

2.3 OBSERVED MARGINAL DISTRIBUTION

  • The observed marginal is Gaussian with covariance ΣO = (I_p−B_OO)^−⊤ Ψ (I_p−B_OO)^−1, where Ψ = ΩO + Λ⊤ΩLΛ is positive definite.The latent variables contribute covariance through the effective noise term Ψ.Source: §2.3 · p. 4

3 MARGINAL EQUIVALENCE AND SCORE

The section defines graph comparison through observed covariance families and uses a penalized Gaussian likelihood to score candidate structures. It specifies the candidate graph class, structural complexity, and covariance-space fit underlying that score.

3.1 MARGINAL COVARIANCE FAMILIES AND EQUIVALENCE

  • For each fixed graph, the observed covariance family is semialgebraic and its real Zariski closure is irreducible.The real Zariski closure is defined by polynomial equalities and may include matrices outside the positive-definite covariance family.Source: §3.1 · p. 4
  • Marginal equivalence requires identical observed covariance families, whereas marginal quasi-equivalence requires an intersection with the dimension of each family, even if latent vertex sets differ.The covariance family determines the full observed distribution family for the known zero-mean Gaussian models.Source: §3.1 · p. 4

3.2 CANDIDATE GRAPH CLASS AND STRUCTURAL COMPLEXITY

  • The candidate class fixes the labeled observed vertices and bounds latent vertices, allowing observed directed cycles but excluding edges into latent vertices.Structural complexity counts every graph edge and every latent variable, including edges whose fitted coefficient is zero.Source: §3.2 · p. 5

3.3 OBSERVED-DATA OBJECTIVE AND GRAPH SCORE

  • The graph score minimizes Gaussian negative log-likelihood plus a logarithmically scaled edge-and-latent penalty over admissible parameters.Its scaling follows BIC, but the penalty need not equal covariance-family dimension, so the paper does not claim a standard BIC interpretation; the covariance-space fit is expressed using KL divergence.Source: §3.3 · p. 5

4 METHOD AND THEORY

For fixed numbers of observed variables and a fixed latent-variable bound, the paper studies consistency of global graph-score minimizers and develops a Bernoulli formulation for joint structure optimization. The guarantees distinguish closure-selection consistency from recovery up to marginal quasi-equivalence, while the Bernoulli objective preserves the discrete objective’s global infimum.

4.1 CLOSURE CONSISTENCY AND GEOMETRIC IDENTIFICATION

  • For fixed p and Lmax, Theorem 4 gives closure-selection consistency for global score minimizers under i.i.d. observations, not arbitrary numerical outputs.The same conclusion holds for deterministic penalty sequences with λ_n→∞ and λ_n/n→0.Source: §4.1 · p. 6
  • Recovery up to marginal quasi-equivalence additionally requires identification; algebraic faithfulness and full-dimensional overlap hold generically, but structural minimality needs separate justification.Source: §4.1 · pp. 6–7

4.2 BERNOULLI FORMULATION FOR JOINT STRUCTURE OPTIMIZATION

  • The Bernoulli formulation uses independent gates for observed edges, latent-slot activation, and latent-to-observed edges, with gate probabilities optimized alongside continuous structural parameters.Source: §4.2 · p. 7
  • The gate objective’s penalty is exact expected complexity and is differentiable in gate probabilities when every binary observed-edge mask yields an invertible A(M).Source: §4.2 · p. 7
  • The Bernoulli expectation preserves the discrete objective’s global infimum and minimizing graphs, supporting gradient-based joint optimization.Source: §4.2 · pp. 7–8

5 EXPERIMENTS

The method is tested on controlled synthetic SCMs and simulated GeneNetWeaver benchmarks, using compatibility error as the primary measure. It achieves lower compatibility error in several larger-latent synthetic settings and approximately half the closest baseline’s total error on GeneNetWeaver.

5.1 SETUP AND EVALUATION

  • The primary compatibility error sums forward and reverse worst-case covariance-fitting errors, allowing structurally different but distributionally equivalent graphs to agree.The evaluation contrasts this measure with SHD, which compares against one particular graph representation.Source: §5.1 · p. 8

5.2 SYNTHETIC RESULTS

  • Runtime averaged 4544 seconds at p = 256 and 24770 seconds at p = 512; baselines did not complete within budget beyond p = 128.Figure 2 compares wall-clock runtime.Source: Figure 2 · p. 8, §5.2 · p. 8

5.1 SETUP AND EVALUATION

  • Synthetic tests use 10,000 observations; GeneNetWeaver tests use 100 observed variables and 20 or 30 latent variables under gene-regulatory dynamics.GeneNetWeaver provides a complementary test under model misspecification; methods sharing a setting receive the same generated data and trial seeds.Source: §5.3 · p. 8, §5.1 · p. 8, §5 · p. 8

5.2 SYNTHETIC RESULTS

  • In larger-latent synthetic settings with baseline results available, the method achieves lower mean compatibility error.Baselines are evaluated only up to the largest graph size for which their computations remain feasible.Source: §5.2 · p. 8, Figure 1 · p. 9

5.3 GENENETWEAVER BENCHMARK

  • Across both GeneNetWeaver latent settings, the method achieves approximately half the closest baseline’s total compatibility error and is faster than all baselines.Table 1 reports mean ± SEM across 10 trials for 100 observed variables.Source: §5.3 · pp. 8–9, Table 1 · p. 9

6 CONCLUSION

The framework learns cyclic linear Gaussian models with exogenous latent confounders and establishes consistency up to marginal quasi-equivalence under explicit identification conditions. Broader structural-minimality conditions and numerical optimization guarantees remain open.

6 CONCLUSION

  • Consistency is established up to marginal quasi-equivalence under identification conditions, but does not guarantee unique graph or latent-count recovery.Broader structural-minimality conditions and numerical optimization guarantees remain open problems.Source: §6 · p. 9

A.1 PROOF OF LEMMA 1

  • The observed covariance derivation combines observed-parent effects with latent-noise contributions, and its noise covariance remains positive definite even when the latent loading is rank deficient.Source: §A.1 · p. 11

A.2 DERIVATION OF THE OBSERVED-DATA SCORE

  • The observed-data score is equivalent to Gaussian KL fitting plus graph complexity; 2 log n scaling is BIC-like, while consistency requires λ_n→∞ and λ_n/n→0.Source: §A.2 · p. 13

A.3 ALGEBRAIC AND SEMIALGEBRAIC SETS AND THEIR DIMENSIONS

  • Semialgebraic sets use finite polynomial equalities and inequalities; dimension tracks the largest coordinate projection with interior, and full-dimensionality is relative to the containing set.Source: §A.3 · pp. 14–15

A.4 CLOSURE PROPERTIES OF COVARIANCE FAMILIES

  • The covariance family lies inside its Zariski closure; positive-definite limits remain in that closure, whose relative closure is semialgebraic.Source: §A.4 · p. 15

A.5 PROOF OF PROPOSITION 2

The appendix develops geometric properties of covariance families, using observed-covariance overlap to derive identification and KL-separation results. The proof of Proposition 2 restricts KL minimization to a compact feasible covariance set and establishes a strictly positive separation Δ(G+).

A.5 PROOF OF PROPOSITION 2

  • For fixed G+, the covariance family is semialgebraic and its Zariski closure irreducible, with the model and closure dimensions equal.The semialgebraic result follows by projecting the joint parameter-covariance set; the dimension identity also covers the intermediate model family.Source: §A.5 · pp. 16–17

A.6 RATIONALE AND SCOPE OF MARGINAL QUASI-EQUIVALENCE

  • Marginal quasi-equivalence compares observed covariance sets by full-dimensional overlap, since structural parameterizations differ and neither intermediate overlap nor equal Zariski closures guarantees overlap.The criterion requires the intersection to have the dimension of both families, but does not require the complete families to be equal.Source: §A.6 · pp. 17–18

A.7 SUFFICIENT CONDITIONS FOR GEOMETRIC IDENTIFICATION

  • Under Assumptions 7–9, geometric identification makes every eligible candidate marginally quasi-equivalent to the true graph; strict covariance-family inclusion would force a dimension and complexity increase.The argument uses structural minimality and the dimension identities for covariance families.Source: §A.7 · pp. 18–19

A.8 KL SEPARATION AND CONVERGENCE OF THE MINIMUM KL DIVERGENCE

  • For a closed covariance set excluding the truth, minimum Gaussian KL divergence is attained and strictly positive; bounded-KL regions over compact first-covariance sets are compact.The compactness result ensures the minimum is attained, and Gaussian KL divergence can be zero only when the covariances agree.Source: §A.8 · pp. 19–20
  • The KL infimum can be restricted to the nonempty compact set M(G+) ∩ K0, where it is attained for the covariance-family closure.Covariances outside K0 have divergence greater than R0+1, while the feasible covariance Σ0 has divergence at most R0; admissible SCM parameters need not attain the same minimum.Source: §A.8 · p. 21
  • Uniform continuity, convergence Sn→Σ*, and Lemma 19 yield Δ(G+) > 0.The argument compares minima using functions with finite infima on a common nonempty set.Source: §A.8 · p. 21

A.9 PROOF OF THEOREM 4

The proof controls the complexity of global score minimizers and excludes candidate covariance families whose closures omit the true covariance. The argument uses a penalty sequence that diverges but grows more slowly than the sample size.

A.9 PROOF OF THEOREM 4

  • D(S_n∥Σ*) = O_p(n^-1), so the reference graph’s empirical minimum KL divergence, multiplied by n, is bounded in probability.The proof derives this rate from the sample-covariance error and a local Taylor bound.Source: §A.9 · p. 23
  • The complexity comparison vanishes in probability as λ_n→∞, and the resulting bound applies simultaneously to every global minimizer.The proof also states that λ_n→∞ and λ_n=o(n) suffice for the complexity comparison and exclusion arguments.Source: §A.9 · pp. 23–25
  • A candidate whose covariance-family closure omits Σ* has a score gap of at least nΔ(G+)/2 with probability tending to one, so it is asymptotically not a global minimizer.Because the candidate class is finite, the exclusion holds simultaneously for all such candidates.Source: §A.9 · p. 24
  • The proof extends to deterministic penalty sequences satisfying λ_n→∞ and λ_n=o(n).The first condition supports the complexity comparison; the second supports exclusion of misspecified covariance models.Source: §A.9 · p. 25

A.10 PROOF OF COROLLARY 6

With probability tending to one, every global score minimizer has the true covariance-family dimension and is marginally quasi-equivalent to the true graph.

A.10 PROOF OF COROLLARY 6

  • With probability tending to one, every global minimizer has covariance-family dimension equal to that of the true graph’s covariance family.The proof obtains the dimension identity for an arbitrary minimizer, so it holds simultaneously even when scores have ties.Source: §A.10 · p. 25
  • The event used in the proof has probability tending to one.Theorem 4 supplies this probability statement.Source: §A.10 · p. 25
  • Every global minimizer is marginally quasi-equivalent to the true graph, using equality of covariance families rather than transitivity of quasi-equivalence.The conclusion applies simultaneously to all minimizers, including in the presence of score ties.Source: §A.10 · p. 25

A.11 DETAILS AND PROOF OF THE BERNOULLI FORMULATION

This appendix examines when Bernoulli-gated scoring retains the discrete objective’s global infimum and how differentiability, generic identification, and structural minimality depend on the candidate class and complexity. A four-variable example illustrates how a quasi-equivalent graph can be more complex than a complete DAG, obstructing the quasi-equivalence guarantee absent assumptions that rule this out.

A.11 DETAILS AND PROOF OF THE BERNOULLI FORMULATION

  • Equation 21 is exactly the expected sampled likelihood plus complexity, not an approximation that substitutes gate probabilities for binary masks.The expected objective averages the scores over binary configurations.Source: §A.11 · p. 26
  • With finite likelihoods for every configuration, the expected score is differentiable in gate probabilities and continuous parameters, but need not be jointly convex.Activating a singular configuration with positive probability makes the objective infinite; fixed zero or one gates can exclude such configurations while preserving differentiability in the remaining free probabilities.Source: §A.11 · p. 26
  • When observed-edge gate probabilities are interior, the weights yielding simultaneous invertibility form an open set outside a measure-zero exceptional set, without requiring acyclicity or a norm bound.Approaching a singular configuration can nevertheless cause numerical difficulties.Source: §A.11 · p. 26
  • The Bernoulli and discrete formulations have the same global infimum, and an attained infimum has a deterministic binary minimizer.The result does not imply that every global minimizer has binary probabilities; a fractional minimizer may also be optimal.Source: §A.11 · p. 27

B SCOPE AND GENERICITY OF THE IDENTIFICATION ASSUMPTIONS

  • Identification assumptions separate: faithfulness and overlap can hold generically, but structural minimality is not implied and must be checked for the chosen complexity.The edge-and-latent penalty can violate structural minimality.Source: B SCOPE AND GENERICITY OF THE… · p. 28

B.1 GENERICITY OF ALGEBRAIC FAITHFULNESS AND FULL-DIMENSIONAL OVERLAP

  • For almost every admissible true parameter choice in a finite candidate class, algebraic faithfulness and full-dimensional overlap hold simultaneously, with the common subset containing the true covariance.The genericity result assumes Proposition 2 for every candidate and does not establish the assumptions at every true covariance.Source: §B.1 · p. 28
  • If structural minimality holds, generic faithfulness and overlap yield generic identification and consistency up to marginal quasi-equivalence, but not pointwise identification at every covariance.Structural minimality must be verified or imposed; exceptional parameter values can remain.Source: §B.1 · p. 30

B.2 FAILURE OF STRUCTURAL MINIMALITY

  • A four-latent example has no observed edges, child sets {1, 2}, {2, 3}, {3, 4}, and {4, 1}, and a dimension-eight covariance closure constrained by Σ13 = Σ24 = 0.The example motivates checking structural minimality rather than assuming an unconditional quasi-equivalence guarantee.Source: §B.2 · p. 31
  • Identically zero observed covariances correspond exactly to vertex pairs with no common ancestor, including in graphs with observed directed cycles.When a common ancestor exists, sufficiently small positive coefficients and positive noise variances give a nonzero covariance.Source: §B.2 · p. 31
  • The four-variable covariance cycle forces no observed edges and at least four latents with eight latent edges, giving complexity at least 12.Each adjacent covariance pair needs its own latent with exactly two children.Source: §B.2 · p. 31
  • Every distribution-equivalent representative has complexity at least 12, and the minimum complexity is c† = 12.The passage gives the lower bound for every representative and identifies c† as 12.Source: §B.2 · p. 31
  • A complete directed acyclic graph on four observed variables has six edges and generates every positive-definite covariance through sequential linear Gaussian regressions.Source: §B.2 · p. 31
  • This violates Assumption 8: the original structural penalty can exclude every quasi-equivalent graph even with exact global optimization, while a dimension-based penalty would avoid this particular obstruction.The quasi-equivalence guarantee therefore cannot extend to all candidate-class graphs without conditions that exclude the obstruction.Source: §B.2 · pp. 31–32
  • The proof shows that every quasi-equivalent graph has the same Zariski closure, since their shared model subset has dimension eight and distinct irreducible closures would intersect in lower dimension.Source: §B.2 · pp. 31–32
  • For any positive-definite sample covariance and positive penalty, the six-edge complete DAG scores below quasi-equivalent graphs of complexity at least 12.Source: §B.2 · pp. 31–32

B.3 WEAKER AND SUFFICIENT CONDITIONS FOR STRUCTURAL MINIMALITY

The section gives a weaker identification condition at the true covariance and a sufficient route to structural minimality for targets with a latent-free representative. Under the stated conditions, global score minimizers recover marginal quasi-equivalence for generic admissible parameters.

B.3 WEAKER AND SUFFICIENT CONDITIONS FOR STRUCTURAL MINIMALITY

  • A weaker condition compares the target only with candidates whose covariance-family closures contain the true covariance, rather than with every candidate graph.The section states that structural minimality is one sufficient route, not a necessary condition for Theorem 4.Source: §B.3 · p. 32
  • The edge-plus-latent penalty is not covariance-family dimension, so increasing covariance dimension need not increase structural complexity and minimality needs separate justification.Replacing the penalty with model dimension would change the objective and its preference among graphs.Source: §B.3 · pp. 32–33
  • For a target with a latent-free representative and no reciprocal edge pairs, structural minimality can be verified against all candidates, yielding generic recovery up to marginal quasi-equivalence.The candidate class may include latent variables and reciprocal observed edges.Source: §B.3 · p. 33
  • When the target family has dimension p + e and its latent-free representative has e observed edges, every candidate of strictly larger dimension has greater structural complexity.This establishes structural minimality without restricting the competing graphs.Source: §B.3 · pp. 33–34
  • In the four-variable cyclic example, c† = 4 and dim M(G∗+) = 8; generic global minimizers recover marginal quasi-equivalence, not a unique orientation or a local-optimizer guarantee.The covariance family has smaller dimension than the full positive-definite covariance space, and strictly larger closures require complexity of at least five.Source: §B.3 · p. 34
  • A latent graph with one two-child confounder can share its covariance family with a one-edge latent-free graph, so c† = 1 despite latent-graph complexity three.This route does not establish structural minimality when every representation requires latents or identify the data-generating latent count.Source: §B.3 · p. 35

C PRACTICAL OPTIMIZATION

Appendix C distinguishes the exact Bernoulli objective from optional continuous-gate approximations and the numerical procedures used in experiments. Theoretical guarantees apply to discrete global minimizers, not automatically to numerical optimization or refitting.

C PRACTICAL OPTIMIZATION

  • Binary-mask invertibility does not ensure invertibility for continuous masks; the example has binary determinants 1 or −3 but a continuous mask with determinant zero.The Binary Concrete relaxation therefore cannot guarantee invertibility for every continuous mask.Source: C PRACTICAL OPTIMIZATION · p. 35
  • Differentiating sampled likelihoods does not alone justify an unbiased gradient for the expected objective; differentiation and expectation require an interchange justification.Derivatives can have infinite absolute expectation as the masked matrix approaches noninvertibility, even if it is invertible almost surely.Source: C PRACTICAL OPTIMIZATION · p. 35
  • For observed-only experiments, the exact Bernoulli formulation uses Monte Carlo hard-edge masks, leave-one-out score-function gradients, and discrete support refinement.Plateaus trigger deletion and reversal-plus-deletion candidate evaluations; accepted deletions leave the search space.Source: C PRACTICAL OPTIMIZATION · p. 36
  • The latent-variable experiments use straight-through Binary Concrete gates, evaluating likelihoods with hard masks and differentiating through soft counterparts at fixed temperature.This numerical estimator is distinct from Proposition 10, which concerns the exact Bernoulli expectation.Source: C PRACTICAL OPTIMIZATION · p. 36
  • The experiments convert learned gates to binary supports, remove inactive latent slots, refit parameters on fixed supports, and compare refitted discrete scores.Refitting uses the unrelaxed Gaussian likelihood without the optional norm restriction on W.Source: C PRACTICAL OPTIMIZATION · p. 36
  • Theorem 4 requires a global minimizer of the discrete score, while Corollary 6 also requires Assumption 5; guarantees for numerical procedures need additional analysis.The optional restrictions that can provide derivative bounds are stronger than the statistical model and may exclude admissible covariances or well-defined cyclic systems.Source: C PRACTICAL OPTIMIZATION · p. 36

D.1 SYNTHETIC DATA GENERATION

Synthetic experiments use generated cyclic observed graphs, optionally augmented with exogenous latent common causes, and specify graph, coefficient, noise, and latent-slot settings. The appendix also reports optimization budgets and the post-extraction refitting procedure.

D.1 SYNTHETIC DATA GENERATION

  • Synthetic trials use 10,000 training samples and observed graphs generated following Ghassami et al. (2020), with randomly generated exogenous latent common causes in confounded settings.The hard maximum-degree bound is total observed in-degree plus out-degree, excluding self-loops.Source: §D.1 · p. 37
  • Observed coefficients have random signs and magnitudes in [0.2, 0.8], with draws constrained by spectral radius below 0.99 and cond(I − BOO) ≤ 20.Observed noise variances are Uniform[1, 3]; latent variables select 2–4 observed children and use similarly distributed loading magnitudes.Source: §D.1 · p. 37
  • Experiments use latent ratios ℓ/p of 0.1, 0.2, and 0.3, with Lmax set to twice the rounded latent count plus one, and use 10 trials with different seeds.No-latent experiments set Lmax = 0.Source: §D.1 · p. 37

D.2 OPTIMIZATION AND EVALUATION SETTINGS

  • Adam update budgets increase with graph size, from 1,000 updates at p = 4 to 10,000 at p = 256 and p = 512.After graph extraction, inactive latent slots and slots with fewer than two retained children are removed, then parameters are refit on each fixed support using the unrelaxed Gaussian likelihood.Source: §D.2 · p. 37

D.3 BIDIRECTIONAL COMPATIBILITY EVALUATION

The experiments evaluate bidirectional covariance-family compatibility across synthetic settings and fixed GNW networks. Sensitivity results indicate that recovery depends jointly on graph size, density, and latent structure.

D.3 BIDIRECTIONAL COMPATIBILITY EVALUATION

  • Bidirectional compatibility is summarized as dfwd + drev using K = 20; fitted-graph parameters are reoptimized for each target covariance while support stays fixed.The evaluation uses covariance-family discrepancies in both forward and reverse directions.Source: §D.3 · pp. 37–38

D.4 BASELINE IMPLEMENTATIONS AND RUNTIME PROTOCOL

  • Baseline implementations use their specified public configurations, and each shared setting receives identical training data and trial seeds.DCCD-CONF and Amendola et al. are evaluated up to p = 64, while dglearn is evaluated up to p = 128 before computation becomes infeasible for these experiments.Source: §D.4 · p. 38

D.5 GENENETWEAVER DETAILS

  • GNW reference networks begin as weakly connected bounded-degree DAGs, then gain cycles through back edges and exogenous latent variables connected to observed subsets.Each fixed network yields 20,000 observations, split into 16,000 training and 4,000 held-out observations; topology and data stay fixed across trials.Source: §D.5 · p. 39

D.6 SENSITIVITY TO GRAPH DENSITY AND LATENT CONFOUNDING

  • Recovery depends jointly on graph size, graph density, and latent structure; the sensitivity experiment varies maximum observed degree and latent ratio across nine combinations.Figure 3 organizes latent ratios ℓ/p of 10%, 20%, and 30% by rows and maximum observed degrees 1, 2, and 4 by columns; points show mean dfwd + drev with SEM, where lower is better.Source: §D.6 · pp. 39–40, Figure 3 · p. 39

E RELATED WORK

Related work addresses cycles, latent confounding, and continuous structure optimization under differing model and data assumptions. This paper distinguishes its observational covariance-family comparison and global-score guarantees from those settings.

E RELATED WORK

  • Prior cyclic linear Gaussian results without latent variables do not directly extend to marginalized latent structures; the paper extends overlap-based comparison to observed covariance families and distinguishes it from Yi & Lee’s filtering guarantee (Yi & Lee, 2024, Theorem 3.4).Marginal quasi-equivalence requires an intersection with the dimension of both covariance families.Source: E RELATED WORK · p. 41
  • Other cycle-and-confounding approaches use randomized interventions or conditional-independence assumptions, whereas this work uses observational data to compare complete Gaussian covariance families.Forr´e & Mooij’s σ-separation assumption need not hold for linear models (Forr´e & Mooij, 2018, Definition 2.15, Remark 2.16).Source: E RELATED WORK · p. 41
  • Amendola et al.’s simple mixed-graph restriction excludes reciprocal directed edges and directed–bidirected pairs, while explicit latent graphs allow these structures but need not yield the same covariance families (Amendola et al., 2020, Section 2).Their cited dimension and closure results do not establish consistency of their score or search; this paper studies global score minimizers under identification conditions (Amendola et al., 2020, Theorems 3.1, 4.1).Source: E RELATED WORK · p. 41
  • Acyclic latent-confounding methods impose different restrictions: Kaltenpoth & Vreeken constrain shared-confounder groups and require equal observed-noise variances for coefficient identification, while SPOT disallows directed cycles (Kaltenpoth & Vreeken, 2023, Theorem 2; Ma et al., 2024, Sections 2–3).This paper allows cycles and unequal noise variances, with recovery up to marginal quasi-equivalence under identification conditions.Source: E RELATED WORK · p. 41
  • NOTEARS characterizes acyclicity, while this work requires invertibility of Ip − BOO; Bernoulli gates are not novel, but this formulation also gates latent activations and connections and proves equality of global infima (Zheng et al., 2018, Theorem 1; Sethuraman & Fekri, 2025, Theorem 2).Proposition 10 concerns the exact Bernoulli expectation and does not guarantee numerical convergence to a global minimum.Source: E RELATED WORK · pp. 41–42
Loading 2609.38618v1…