Source-linked AI summary

Guide, Not Bind: Why Defeasible Priors Fail in Augmented Lagrangian Causal Discovery

Sairam Sundararaman, Sara Girdhar, Manit Narasimha Murthy, Samrudh N, Bhaskarjyoti Das

arXiv:2609.03442v1cs.LG

TL;DR

Differentiable causal discovery uses ALM-enforced forbidden-edge priors on the assumption that data-adaptive relaxation can override incorrect rules, but the natural design has two independent obstacles. The paper characterizes both, tests a combined repair, and finds that changing objectives removes the correlation tie while suppression can remain severe.

  • Problem

    A data-adaptive check may fail to override an incorrect forbidden-edge prior, and correlation matching may not distinguish a true edge from its reverse even when the equal-variance model is identifiable.

    Source: §1 · p. 2
  • Method

    The paper characterizes the ALM suppression mechanism and correlation-matching tie, constructs a relaxation operator satisfying the derived conditions, combines it with covariance matching, and tests alternative fitting objectives.

    Source: §1 · p. 3, §11 · p. 23
  • Results

    Least-squares and likelihood objectives remove the correlation-matching tie, but likelihood suppression reaches 100% at the largest graph size.

    Source: §11 · p. 23
  • Takeaways & Limitations

    Satisfying the necessary relaxation conditions and changing objectives do not ensure recovery, as the unconstrained reverse edge remains a dominant failure mode.

    Source: §11 · p. 23

    Whether a soft, score-integrated prior combined with correlation-based scoring exhibits the proven tie remains an untested extension.

    Source: §10 · p. 22

Abstract

from arXiv · show

Differentiable causal discovery methods increasingly encode expert priors as forbidden-edge constraints enforced by an Augmented Lagrangian (ALM) penalty, on the assumption that a data-adaptive relaxation mechanism will discount and eventually override a rule the data consistently contradicts. We show this design, which we call \emph{guide, not bind}, fails for two independent, precisely characterized reasons, and that directly repairing both restores it only partially. First, sequential penalty-ramping ALM suppresses a wrongly-forbidden true edge before any counterfactual check can detect it: we give three necessary conditions any adaptive relaxation must satisfy to avoid this (Proposition~\ref{prop:conditions}), prove that DADU---the natural relaxation rule this paper introduces as the object of study---violates all three (Corollary~\ref{cor:dadu_failure}), and confirm the failure across 3{,}072 training runs spanning graphs from 4 to 32 nodes, where a single wrong prior suppresses a true edge in 87--97\% of trials under DADU. Second, and independent of any fix to the mechanism, we prove in closed form that the standard correlation-matching objective ties a true edge and its reverse to an identical cost of exactly $2r^2$ (Lemma~\ref{lem:tie}), not because the underlying equal-variance model is unidentifiable, but because normalizing to correlation discards exactly the variance information that would make it identifiable; covariance matching instead separates the two directions by a provable margin of at least $w_0^4$ (Lemma~\ref{lem:separation}).

1 Introduction

The introduction asks whether a defeasible prior can guide causal discovery without binding it, then identifies two independent failures of the natural ALM-based design. It presents a partial repair and tests whether the failures extend beyond correlation matching.

1 Introduction

  • Sequential penalty-ramping can suppress a wrongly-forbidden true edge before the counterfactual check can assess its data-supported value.The paper gives three necessary conditions for avoiding this trap and shows that its introduced DADU rule violates all three.Source: §1 · pp. 2–3
  • In the reported sweep, Notears/Dagma-style least-squares and golem-EV-style likelihood objectives preserve identifiability, but likelihood suppresses the wrongly-forbidden edge more often than correlation matching.Source: §1 · p. 3
  • Correlation matching assigns a true edge and its reverse identical cost because normalization discards marginal-variance information, despite equal-variance model identifiability.Covariance matching instead separates the two directions; the underlying model’s identifiability is attributed to Peters & Bühlmann (2014).Source: §1 · pp. 2–3
  • The combined relaxation and covariance-matching fix significantly outperforms DADU, yet recovers the wrongly-forbidden edge in only a minority of trials.The unconstrained reverse edge remains the dominant attractor.Source: §1 · p. 3
  • The paper tests these mechanisms across 6,144 training runs on synthetic graphs from 4 to 32 nodes, varying density, noise, and fitting objective.The stated scope is a controlled synthetic sweep, not real-world data.Source: §1 · p. 4
  • The claimed novelty is the failure of a growing penalty racing a relaxation operator, rather than the general consequences of enforcing an incorrect constraint.Prior work already studies ALM, forbidden-edge constraints, and equal-variance identifiability; the paper locates its contribution in their specific interaction with defeasible enforcement and correlation matching.Source: §1 · p. 4

2 Background

This background establishes the model, constraint machinery, and identifiability concepts used in the paper. Its central distinction is between a model that is identifiable in principle and a fitting loss that may fail to use the identifying signal.

2.1 Structural Equation Models and Differentiable Discovery

  • A linear Gaussian SEM implies a covariance matrix that observational samples constrain; differentiable discovery fits a continuous relaxation of the DAG search with a smooth acyclicity regularizer.The paper uses Dagma’s log-determinant regularizer and later specifies its fitting loss.Source: §2.1 · p. 5

2.2 Augmented Lagrangian Enforcement of Prior Constraints

  • An ALM enforces forbidden-edge constraints through dual updates and a penalty that typically grows geometrically; when the constraint conflicts with the unconstrained minimizer, the multiplier diverges.A defeasible constraint must permit relaxing that multiplier when data disagrees.Source: §2.2 · p. 5

2.3 Markov Equivalence and Identifiability

  • Markov-equivalent DAGs share a skeleton and v-structures, and can generate the same observational distributions under suitable parameters.That ambiguity is a property of the data distribution, not merely a limitation of an estimator.Source: §2.3 · p. 5
  • General linear Gaussian SEMs identify direction only up to Markov equivalence, while equal noise variances can make the DAG generically identifiable from observational covariance.Equal-variance identifiability uses marginal-variance differences to recover direction; this paper asks whether its fitting loss retains that signal.Source: §2.3 · pp. 5–6

3 Setup: Defeasible Priors via Augmented Lagrangian

The setup specifies a linear Gaussian SEM, an ALM penalty for forbidden edges, and the data-adaptive rule the paper studies. It uses correlation matching as its core fitting objective and defines relaxation through a counterfactual fit-cost check.

3 Setup: Defeasible Priors via Augmented Lagrangian

  • The core testbed assumes i.i.d. samples from a linear Gaussian SEM and fits the structure by minimizing the distance between normalized correlation matrices.A Dagma log-determinant regularizer softly enforces acyclicity.Source: §3 · p. 6
  • A binary mask marks forbidden entries, whose ALM penalty uses per-edge dual variables and a geometrically growing penalty weight.The penalty update adds a term proportional to the masked edge weight and then increases ρ by a factor κ greater than one.Source: §3 · p. 6
  • DADU relaxes a forbidden-edge constraint when removing that edge from the current fit costs at least a threshold δ in fitting loss.The paper introduces DADU as the natural guide-not-bind instantiation to analyze, rather than as a rule from a published implementation.Source: §3 · p. 6
  • If the penalty drives a forbidden edge near zero before the check sees its unconstrained value, the check returns approximately zero cost and tightening repeats as ρ grows.The paper calls this the early suppression trap: the check can become uninformative even when the edge is true and data-supported.Source: Figure 1 · p. 7, §3 · p. 7

4 Obstruction I: The Suppression Mechanism

Sequential penalty ramping can suppress a wrongly-forbidden true edge before a counterfactual check can detect its data support. DADU violates the stated necessary conditions, and the controlled and larger-scale results confirm the suppression failure.

4 Obstruction I: The Suppression Mechanism

  • The necessary-condition argument uses a first-order expansion around the unconstrained optimum; an optimizer-agnostic version is not established.The controlled verification adds no variables and has no observation noise beyond the model’s own.Source: §4 · p. 7, §4.1 · p. 8
  • DADU violates all three necessary conditions simultaneously once suppression begins.The conditions include evaluating the signal before suppression and allowing the operator to move the dual variable against tightening; the necessity claim is under a first-order local analysis.Source: §4 · pp. 7–8
  • DADU suppresses a wrongly-forbidden true edge in 87–97% of trials across graphs from 4 to 32 nodes.The suppression window closes as the exponentially tightening penalty overtakes linear relaxation; after that, the counterfactual signal remains near zero regardless of threshold or causal signal strength.Source: §4 · p. 8

4.1 Independent Synthetic Verification

  • In the controlled instance, the forbidden true edge falls to near zero by epoch 14, and tested thresholds change the final outcome by less than 10^-3.The reverse edge converges to 0.5496, absorbing the fit; DADU’s relaxation branch fires even at δ = 0.01, but firing does not restore the true edge.Source: §4.1 · pp. 8–9, Figure 2 · p. 8

5 Obstruction II: The Identifiability Limit

Even if relaxation checks occur before suppression, correlation matching cannot distinguish a true edge from its reverse in the isolated case. Raw covariance restores directional separation, while the general-graph result and remaining dynamic obstruction delimit what that repair establishes.

5.2 Lemma 1: The Tie Under Correlation Matching

  • On an isolated pair, correlation matching gives the forward and reverse models the same exact global minimum and identical counterfactual costs of 2r^2.The derivation uses two variables with at most one directed edge and independent, identically distributed standard normal exogenous noise.Source: §5.2 · p. 9, §5.1 · p. 9
  • Correlation matching discards the variance asymmetry that identifies the equal-variance model, so the tie belongs to the fitting objective, not the causal model.Normalizing covariance matrices to unit diagonal removes the marginal-variance signal; the passage identifies the underlying model as generically identifiable from population covariance.Source: §5.2 · p. 9

5.3 Lemma 2: Separation Under Covariance Matching

  • Under raw covariance matching, the reverse direction has a loss at least w0^4 above the true direction’s zero loss.The separation is a closed-form bound; the true reverse-direction minimizer gives a strictly larger gap, but the stated bound is sufficient for the argument.Source: §5.3 · p. 10, Figure 3 · p. 10

5.4 Proposition 2: Generalizing Beyond the Isolated Pair

  • For a covered edge in a general DAG, reversing it can preserve the correlation matrix through reparameterization, so correlation matching cannot separate the directions at their optima.That reparameterization may change other edge weights; the result concerns the resulting correlation matrix, not an unchanged set of other parameters.Source: §5.4 · p. 10

5.5 What Is, and Is Not, Fixed

  • Covariance matching removes the static tie but does not prevent a check at an already-suppressed weight from reading near zero.The separation helps only if the counterfactual is evaluated near the unconstrained optimum; whether the combined mechanism recovers the edge is tested separately.Source: §5.5 · p. 11

6 The Combined Mechanism

AdaptiveRelax is built to address the suppression conditions and is paired with covariance matching to test the combined repair. Its guarantees are local and operational: they do not establish that the probe reaches the global unconstrained optimum.

6 The Combined Mechanism

  • AdaptiveRelax replaces DADU’s counterfactual check and dual update with changes intended to satisfy the three necessary conditions.The paper builds the operator and tests it together with covariance matching against the combined-repair question left open by the preceding section.Source: §6 · p. 11, §6.1 · p. 11

6.1 Construction

  • At probe intervals, AdaptiveRelax freezes other weights and takes local gradient steps on the candidate edge before evaluating its fit signal.The construction uses the probed value for the local gradient and updates the dual variable and penalty using the probe-based quantities.Source: §6.1 · p. 11

6.2 AdaptiveRelax Operationalizes C1–C3

  • Proposition 3 establishes that AdaptiveRelax operationalizes C1–C3, but its probe is only a coordinate-wise local descent point.Other weights remain fixed at their live values, which may already reflect penalties, acyclicity regularization, or incomplete training; the proposition does not guarantee the global unconstrained optimum.Source: §6.2 · pp. 11–12

7 Broader Empirical Validation

Across a broad testbed, the authors find that changing the fitting objective can remove correlation matching’s directional tie, but does not eliminate DADU’s suppression of wrongly-forbidden edges. AdaptiveRelax improves local recovery, yet the reverse edge remains favored and whole-graph recovery depends strongly on the fitting objective.

7 Broader Empirical Validation

  • The validation reused matched graphs across operators and objectives, spanning 4–32 nodes, two densities, three noise scales, and equal-variance or heteroscedastic noise.The correlation/covariance sweep comprised 3,072 training runs; the full sweep with two additional objectives comprised 6,144.Source: §7 · pp. 12–13

7.1 Obstruction I Generalizes Beyond the Minimal Illustration

  • Suppression under DADU stays above 87% for correlation matching and above 45% for covariance matching at every tested graph size.The covariance advantage narrows as graph size increases; the authors relate this pattern to the shrinking fitting-gradient ratio.Source: Table 1 · p. 13, §7.1 · p. 13

7.2 Obstruction II Generalizes Beyond the Minimal Illustration

  • Correlation matching ties directions at every tested graph size, while covariance separates them under equal-variance noise; its advantage weakens under heteroscedastic noise.At d = 32, heteroscedastic covariance gaps are only 0.002 at σ = 0.5 and 0.015 at σ = 1.0; the authors make no directional claim at σ = 2.0.Source: Table 2 · p. 13, §7.2 · p. 14

7.3 Does Either Obstruction Survive an Objective the Literature Actually Uses?

  • Under equal-variance noise, the three raw-scale objectives separate edge directions while correlation ties, but likelihood suppresses wrongly-forbidden edges more often than correlation at every tested graph size.Least squares also gives the best whole-graph SHD of the tested objectives while matching covariance matching’s local recovery.Source: Table 3 · p. 15, Table 4 · p. 15, §7.3 · p. 15

7.4 The Combined Mechanism: Partial Recovery, Not Resolution

  • AdaptiveRelax recovers wrongly-forbidden edges more often than DADU under both objectives, but every combination remains below 30% recovery and covariance matching still favors the reverse edge.With AdaptiveRelax and covariance matching, the reverse edge is active in 41.1% of trials versus 24.7% for the correct edge.Source: §7.4 · pp. 16–17, Table 6 · p. 16, Figure 5 · p. 17

7.5 A Real Tradeoff: Covariance Matching Costs Whole-Graph Recovery

  • Covariance matching has substantially worse whole-graph SHD than correlation matching at every graph size, while changing the relaxation operator has little effect on SHD.The authors attribute part of covariance matching’s density interaction to raw scale and a fixed fit-loss weight, which they did not correct.Source: §7.5 · pp. 17–18, Figure 6 · p. 18, Table 7 · p. 18

8 Discussion: Implications for Practitioners

Practitioners should check each forbidden edge at the unconstrained solution before penalty ramping, while recognizing that this fixes only early suppression. Objective choice brings trade-offs, and correct priors matter more than confidence calibration.

Implications for Practitioners

  • Check the unconstrained solution before penalty ramping; this prevents early suppression by construction, regardless of fitting objective, but does not address the identifiability obstruction.The check is cheap and must occur before the penalty can suppress the edge.Source: §8 · p. 19
  • Least-squares matching is the recommended objective: it shares covariance matching’s identifiability benefit without covariance matching’s whole-graph cost, and achieves the best whole-graph accuracy tested.Covariance matching roughly doubles whole-graph SHD relative to correlation matching at most grid cells; likelihood performed worst on suppression and whole-graph SHD.Source: §8 · p. 19
  • Combining a C1–C3 relaxation operator with covariance matching improves recovery over DADU, but reverse-edge attraction persists across tested objectives and operators.The ALM’s one-sided constraint leaves the reverse direction free; the combination reduces suppression rather than eliminating it.Source: §8 · p. 19
  • Retune the fit-loss weight wf when switching from correlation to an unnormalized objective, because the same weight can materially harm whole-graph recovery.The cost varies with graph size, density, and which unnormalized objective is used.Source: §8 · p. 19
  • Correctly specified priors yield perfect recovery under every tested objective, whereas incorrect priors cause suppression or misdirection regardless of confidence calibration.The authors therefore prioritize eliciting and verifying priors before training over refining the enforcement mechanism.Source: §8 · p. 19

9 Related Work

The paper places its contribution at the intersection of differentiable causal discovery, constrained optimization, and prior knowledge. It distinguishes its setting from related work by studying potentially wrong forbidden-edge constraints enforced through a penalty that grows during training.

Differentiable causal discovery

Augmented Lagrangian methods and constrained optimization

  • Classical and later constrained-learning work treats constraints as fixed and correctly specified; this paper instead analyzes adaptive relaxation layered over a possibly wrong constraint.The related work includes ALM foundations from Hestenes (1969) and Powell (1969), and constrained-learning approaches such as Nandwani et al. (2019) and Cotter et al. (2019).Source: §9 · p. 20

Prior knowledge in causal discovery

  • Prior causal discovery work studies background knowledge in discrete search and the effects of wrong constraints, but not a continuously growing penalty for a potentially wrong constraint.Constantinou et al. (2023) find that a single incorrect constraint can cascade through a recovered graph; this paper studies the mechanism in an ALM-based learner.Source: §9 · p. 20

Optimization stability of differentiable causal discovery

  • Optimization-stability methods improve base optimizers rather than forbidden-prior enforcement; whether greater stability narrows the suppression window remains unaddressed.The passage discusses SDCD (Nazaret et al., 2024) and DAGMA-DCE (Waxman et al., 2024) as orthogonal work.Source: §9 · p. 20

Identifiability of linear Gaussian causal models

  • Equal-variance identifiability offers a route to recover linear Gaussian causal models without non-Gaussianity, nonlinearity, or interventions, but the paper asks what correlation-based fitting loses relative to covariance-based fitting.The equal-variance route is attributed to Peters & Bühlmann (2014).Source: §9 · p. 21

Neuro-symbolic learning and reasoning shortcuts

  • The early suppression trap differs from neuro-symbolic reasoning shortcuts: it can occur even when the edge concept is well specified, because the constraint schedule is at fault.Marconato et al. (2023) describe shortcuts arising from underspecified mappings between concepts and task labels.Source: §9 · p. 21

Soft and score-integrated priors

  • Soft score-integrated priors avoid a penalty schedule, but whether correlation-based scores still produce the directional tie is not empirically tested here.Darvariu et al. (2024) use probabilistic priors from language-model judgments; the paper leaves their interaction with correlation scores untested.Source: §9 · p. 21

10 Limitations

The paper’s evidence and proposed fixes have important scope boundaries: the combined fix remains incomplete, some theoretical conditions are local, and empirical results rely on a narrow synthetic setting. Several objective comparisons and alternative-prior interactions also remain untested.

Limitations

  • AdaptiveRelax with covariance matching recovers the wrongly forbidden edge in 24.7% of trials versus 41.1% reverse-edge attraction, leaving the ALM’s one-sided constraint as an unresolved open direction.Neither tested relaxation operator nor any of the four tested objectives breaks that structural asymmetry.Source: §10 · p. 21
  • AdaptiveRelax’s C1 probe is coordinate-wise and context-dependent, while C2’s rate bound is local and optimizer-dependent; the gap from the global condition is unbounded or unmeasured.The paper gives no global bound for the C1 gap and no optimizer-independent necessity argument or broad sensitivity sweep for C2.Source: §10 · pp. 21–22
  • Covariance matching’s whole-graph cost is traced partly to raw-scale growth interacting with fixed wf, but scale normalization and per-cell weight retuning were not evaluated.The same concern for least-squares and likelihood objectives is also unstudied.Source: §10 · p. 22
  • Identifiability guarantees rely on equal-variance noise; under heteroscedasticity, least-squares and likelihood show a consistently negative directional gap at every tested d, without a proven mechanism.Covariance matching’s advantage also weakens at high noise scale, where raw loss values are not comparable to the rest of the grid.Source: §10 · p. 22
  • Experiments test one wrongly forbidden edge per graph and use synthetic linear Gaussian SEMs, so interactions among multiple wrong priors and behavior on real-world data remain unknown.The synthetic setting permits exact checks against known ground truth but does not include concept extraction, measurement noise, or non-Gaussian data generation.Source: §10 · p. 22
  • Proposition 2 identifies vulnerable reversals through graphical criteria but lacks a d-dimensional closed-form analogue or general-graph proof of the per-d separations.The literature-matched objectives inherit this general-graph proof gap.Source: §10 · p. 22
  • Least-squares and likelihood are faithful reimplementations rather than runs of the original released code, and whether a more stable optimizer narrows suppression remains open.The paper verified the least-squares identity numerically but did not rerun golem, SDCD, or DAGMA-DCE under the wrong-forbidden condition.Source: §10 · p. 22
  • Soft score-integrated priors are discussed but not tested for whether correlation-based scores retain the directional tie.The authors leave this proposed theoretical extension without an empirical test.Source: §10 · p. 22

11 Conclusion

The paper traces guide-not-bind failures to early ALM suppression and a correlation-objective tie, then tests repairs that improve recovery without resolving the problem. The remaining failures point to structural asymmetry in the constraint itself.

Conclusion

  • Sequential penalty-ramping ALM can suppress a wrongly forbidden true edge before an adaptive correction measures its cost, while correlation matching ties that edge with its reverse.The tie reflects discarded marginal-variance information rather than non-identifiability of the equal-variance model; covariance matching restores provable separation.Source: §11 · p. 23
  • The combined relaxation and covariance-matching fix recovers wrongly forbidden edges significantly more often than DADU, but the remaining reverse-edge attraction exposes a third structural obstacle.The ALM penalizes one direction of an edge while leaving the other free, and the tested fixes do not directly change that shape.Source: §11 · p. 23
  • The correlation tie disappears with the tested least-squares or likelihood objectives, but suppression persists across objective changes and likelihood performs worst for suppression.At the largest graph size, likelihood reaches 100% suppression; the two obstructions are separable, so fixing one does not establish improvement on the other.Source: §11 · p. 23
  • The paper concludes that objective changes can improve one axis while worsening another, and that directly removing the ALM’s asymmetry remains an unsolved direction.The authors do not yet know how to state the needed condition in closed form.Source: §11 · p. 23

A Proof of Proposition 1

The proof gives necessary constraints on relaxation: the counterfactual check must retain a meaningful fit comparison, and relaxation must be able to outpace penalty tightening.

A Proof of Proposition 1

  • When W_ij(t) is near zero, setting it to zero changes fit negligibly, so a counterfactual check at that value cannot reveal the edge’s data-supported contribution.This holds for any data-generating process.Source: A Proof of Proposition 1 · p. 26
  • Relaxation must initially dominate tightening, which holds iff relax(0) > tighten(0) · (κ − 1).If this condition fails, tightening outpaces relaxation from the first epoch, independent of Δ_ij.Source: A Proof of Proposition 1 · p. 26
  • The relaxation operator needs an epoch when its subtraction exceeds its addition, but the plain dual ascent step provides neither a slack variable nor an explicit ceiling on λ_ij.Consequently, λ_ij cannot move against tightening under this update.Source: A Proof of Proposition 1 · p. 26

B Proof of Corollary 1 and the Suppression Window

The proof shows that DADU’s relaxation rule violates the stated conditions and that its suppression window closes; it also proves an exact directional tie under correlation matching.

B Proof of Corollary 1 and the Suppression Window

  • DADU evaluates the counterfactual at the current edge value, so once that value is near zero, removing it changes fit negligibly.This is the timing failure behind its violation of C1.Source: B Proof of Corollary 1 and the… · p. 26
  • Under the default schedule, Eq. 26 generically fails for w0 in [0.3, 0.6], violating C2.The stated relaxation term is small relative to a non-degenerate fit-loss gradient.Source: B Proof of Corollary 1 and the… · p. 27
  • DADU’s λ_ij update is monotone non-decreasing, and after T* its counterfactual signal is approximately zero for any δ > 0.The suppression window therefore closes after T*, independent of δ, η_r, or causal signal strength.Source: B Proof of Corollary 1 and the… · p. 27

C Proof of Lemma 1

  • Correlation matching assigns the true edge and its reverse the identical fit cost, 2r^2.Both directional gaps equal L_fit(0) − L_fit(w0) = 2r^2.Source: C Proof of Lemma 1 · p. 27

D Proof of Lemma 2

The proof establishes a positive direction gap under covariance matching for a nonzero true weight, illustrates how a covered reversal preserves correlations through reparameterization, and reports scale growth in synthetic settings.

D Proof of Lemma 2

  • For w0 ≠ 0, covariance matching gives a strictly positive direction gap; its optimizing reverse weight solves w'³ + w' − w0 = 0.The proof establishes strict positivity because the nonnegative terms in its bound cannot vanish simultaneously for nonzero w0.Source: D Proof of Lemma 2 · p. 28

E On Proposition 2: A Worked Covered Reversal

  • A covered reversal remains Markov equivalent and can reproduce all three correlations, but requires reparameterizing another edge, not just changing the reversed edge’s weight.The worked example states that the reparameterization establishes existence, not that all other weights remain fixed.Source: E On Proposition 2: A Worked Covered… · pp. 28–29

F Synthetic Sweep Details

  • At d = 32, raising density from 0.15 to 0.30 increases the squared covariance Frobenius norm by 44.5×, versus 1.26× at d = 4.The reported factors at d = 8 and d = 16 are 1.71× and 4.73×, respectively.Source: F Synthetic Sweep Details · p. 29
Loading 2609.03442v1…