Demistifying Data and Simulator Assumptions in Supervised Causal Discovery

II. BACKGROUND

• Under Markov, faithfulness, and causal-sufficiency assumptions, conditional independencies identify a CPDAG, while stronger mechanism or noise restrictions can identify additional directions.A target is identifiable when models with the same observed distribution agree on that target.Source: §I · p. 1

I. INTRODUCTION

• Simulators pair generated datasets with structural labels, while SCD methods differ in whether they predict complete structures, edge probabilities, or local relations.Some methods use subset-graph estimates as inputs to predict a complete graph.Source: §I · p. 1
• Training may occur before deployment, on each test dataset, or at both stages; local simulator fitting and retraining add search, simulation, and optimization costs.Broad pretraining and test-time adaptation address different sources of mismatch and can be combined.Source: §I · pp. 1–2
• Target identifiability and generalization are separate: a simulator can favor directions not identified by observations, and CPDAG-trained predictors can still struggle on unfamiliar mechanisms or graph patterns.Calibration and sensitivity to the prior matter because confidence can persist when data do not identify a unique graph.Source: §I · p. 2
• The comparison framework covers prediction target, granularity, encoder, structural decoder, and training regime, relating methods that estimate the same target.It also examines how their inputs, models, and training procedures differ.Source: §I · p. 2

A. Conceptual View of Causality

• Causal discovery often uses observational data to estimate structures for intervention or counterfactual reasoning, whose required model information need not be identified by observations alone.Counterfactual reasoning compares actions for the same unit under different conditions.Source: A. Conceptual View of Causality · p. 3

B. Notation, Structural Causal Models, and Graphical Models

• An SCM assigns each variable a deterministic mechanism of its graph parents and an exogenous noise term, with a joint distribution over noises.Observed-variable noise can be dependent after unobserved common causes are marginalized out.Source: B. Notation, Structural Causal Models,… · p. 3
• With latent confounding, MAGs summarize ancestral relations and dependence paths through omitted variables, while PAGs represent Markov equivalence classes of MAGs.The paper focuses on DAGs and CPDAGs under causal sufficiency, and MAGs/PAGs without selection bias when latent confounding is allowed.Source: B. Notation, Structural Causal Models,… · p. 3

C. Task Description

• A graph target may be a skeleton, v-structures, an equivalence class, or a full graph; identifiability concerns the population distribution, while finite-sample estimation adds uncertainty.A Bayesian posterior is a distribution over candidate graphs conditional on data and prior, not itself a graph feature.Source: C. Task Description · p. 3

D. Standard Assumptions

• The Markov assumption links graph factorization to conditional independence, and faithfulness requires distributional independencies to correspond to graph d-separations.Together, they support discovery methods that infer structure from conditional-independence tests.Source: D. Standard Assumptions · pp. 3–4
• Causal sufficiency assumes all common causes of observed variables are observed; an observed distribution's fit to a DAG does not establish that condition.The paper states causal sufficiency explicitly when a result concerns the causal DAG on observed variables.Source: D. Standard Assumptions · pp. 3–4

E. Identifiability from Conditional Independence

• Under A1–A3, conditional independencies identify the observed DAG’s skeleton and unshielded colliders, but need not determine every edge direction.An adjacency is identified when no conditioning subset separates the two variables.Source: E. Identifiability from Conditional… · p. 4
• An unshielded triple is a v-structure when its middle node belongs to none of the separating sets for its endpoints.Meek’s rules then orient remaining compelled edges while enforcing acyclicity.Source: E. Identifiability from Conditional… · p. 4
• Under A1–A3, complete orientation rules recover the CPDAG; a unique DAG is recovered only when the equivalence class has one member.Markov-equivalent DAGs have the same skeleton and immoralities.Source: E. Identifiability from Conditional… · p. 4
• With latent confounding, observed conditional independencies generally identify a MAG equivalence class represented by a PAG, rather than a DAG CPDAG.The appropriate marginal graphical model and orientation rules are required.Source: E. Identifiability from Conditional… · pp. 4–5

F. Additional Assumptions

• A4 denotes functional model classes whose restrictions ensure identifiability, while interventions or background knowledge can also orient otherwise unresolved edges.IFMOCs restrict structural equations to parent functions and independent noise, with further function or noise conditions ensuring identifiable directions.Source: F. Additional Assumptions · p. 5
• In identifiable additive noise classes, the bivariate causal direction is identifiable under regularity and non-degeneracy conditions, but nonlinearity alone is insufficient.Linear Gaussian models with unrestricted noise variances are an exception; heteroscedastic models need separate analysis.Source: F. Additional Assumptions · p. 5
• The ANM method tests residual–predictor independence in both directions and selects the direction consistent with the additive-noise assumption.Source: F. Additional Assumptions · p. 5
• LiNGAM specializes additive noise models to linear functions and non-Gaussian noise, identifying the full DAG under its assumptions.Dependence in the reverse linear-regression residual alone does not rule out every nonlinear reverse model.Source: F. Additional Assumptions · p. 5
• Post-nonlinear models extend additive noise models by allowing a nonlinear distortion of the additive functional form.Their formulation preserves a direction asymmetry when noise is independent after applying the inverse distortion in the true direction.Source: F. Additional Assumptions · p. 5
• Identifiable functional classes can resolve directions left open by conditional independence when their additional conditions hold; neural flexibility and acyclicity alone do not ensure those conditions.Source: F. Additional Assumptions · p. 5

G. Classical Causal Discovery Methods

• Constraint-based methods translate conditional-independence tests into graph constraints: PC removes adjacencies and orients v-structures, while FCI-style methods use PAG semantics when latent confounding is allowed.Source: G. Classical Causal Discovery Methods · p. 5
• Finite-sample conditional-independence tests can be unreliable, and even correct relations generally recover an equivalence class unless extra information identifies directions.Source: G. Classical Causal Discovery Methods · p. 5
• GES searches over equivalence classes for a CPDAG; hybrid MMHC first screens candidate neighborhoods with constraints, then applies score-based search.Score-based methods may be more robust to noisy CI decisions but inherit assumptions from their score, likelihood, sparsity penalty, and search space.Source: G. Classical Causal Discovery Methods · pp. 5–6
• Differentiable methods optimize adjacency parameters under an acyclicity constraint, but low reconstruction loss does not establish identifiable edge directions.An exactly satisfied constraint ensures a DAG; numerical penalties and approximate optimization require a separate validity check.Source: G. Classical Causal Discovery Methods · p. 6

III. SUPERVISED CAUSAL DISCOVERY: WORKFLOW AND METHOD COMPARISON

• The comparison focuses on predictors mapping datasets to causal structure, selecting representative work through June 2026 and excluding effect estimation, time-series discovery, and unstructured-input representation learning.ADAG is included as a related approach without supervised graph labels.Source: §III · p. 6

A. From Simulated Tasks to Graph Predictions

• A simulator samples a causal model, generates its dataset, and supplies a structural label; that label choice determines the prediction target and granularity.Full-DAG labels are available, but whether their orientations can be inferred depends on the model class.Source: A. From Simulated Tasks to Graph… · p. 6
• An encoder represents the dataset and a decoder assembles predictions; local heads may estimate adjacencies and colliders, while posterior methods estimate graph distributions or marginals.Some methods also use knowledge or estimates from variable subsets as encoder inputs.Source: A. From Simulated Tasks to Graph… · pp. 6–7
• Training may occur before deployment, on the test dataset, or at both stages; the five comparison dimensions include target, granularity, encoder, decoder, and training regime.Adaptive methods refit using generated or reweighted tasks, while pretrained predictors apply a learned rule to a new dataset.Source: A. From Simulated Tasks to Graph… · p. 7

B. Prediction Targets and Model Components

• Identifiable targets include skeletons, unshielded colliders, and CPDAGs under A1–A3; full-DAG labels require additional orientation information or interpretation conditional on the simulator prior.Edge marginals and joint graph samples both express uncertainty, but marginals alone do not encode dependencies among edges.Source: B. Prediction Targets and Model… · p. 7
• Prediction granularity ranges from pairs and triples to variable subsets and whole graphs, while encoders use CI statistics, samples, or side information.Local predictions can provide many labels per simulated task but must be combined consistently; decoders must also satisfy graph-validity requirements.Source: B. Prediction Targets and Model… · p. 7

C. Training Data and Deployment

• Training simulators vary graphs, mechanisms, noise, dataset sizes, and label rules; narrow variation can limit performance on unfamiliar tasks.Test-time adaptation can reduce some mismatch but adds simulation and optimization costs and may reproduce sampling noise.Source: C. Training Data and Deployment · p. 7, Figure 5 · p. 9

D. Method-Level Analysis

• RCC’s bivariate directions rely on distributional asymmetries, while DAG-EQ’s correlation encoder can discard information that distinguishes directions.DAG-EQ assembles edges in descending predicted-probability order while skipping edges that would create cycles.Source: D. Method-Level Analysis · pp. 8–9
• ML4C and SiCL predict colliders and CPDAG features from local labels, but ML4C’s CPDAG recovery is conditional on its supplied skeleton.Both approaches combine local predictions with conflict-resolution or orientation rules.Source: D. Method-Level Analysis · pp. 9–10
• AVICI estimates simulator-prior edge marginals, whereas BCNP samples dependent edge configurations as DAGs through a shared topological ordering.Thresholded marginals still need graph construction; BCNP’s posterior approximation depends on the Bayesian model used to generate training tasks.Source: D. Method-Level Analysis · p. 10
• Kode adds reachability knowledge as a weak graph-sampling bias, while SEA aggregates classical estimates from variable subsets with global statistics.Incorrect or overly dense knowledge can bias Kode; SEA’s predictions can be affected by errors or inadequate coverage in marginal estimates.Source: D. Method-Level Analysis · pp. 10–11
• TTT-SCL retrains on tasks from a test-fitted local simulator, whereas TICL uses interventional data and JCI priors to orient edges beyond observational CI relations.A local simulator can fit sampling noise, and TICL’s recovery depends on available interventions and valid intervention assumptions.Source: D. Method-Level Analysis · pp. 11–12

E. Synthesis and Evaluation Implications

• Skeleton, CPDAG, and full-DAG accuracy score different targets, and posterior methods also require uncertainty assessment.A common graph metric can obscure these distinctions when the scored target is unspecified.Source: E. Synthesis and Evaluation Implications · p. 12
• Comparisons should state the target, training and test assumptions, and how model outputs become the graph being scored.These details distinguish recovery of identifiable features from agreement with simulator-favored directions.Source: E. Synthesis and Evaluation Implications · p. 12
• Even when simulators share Markov, faithfulness, and causal-sufficiency assumptions, differing mechanisms, graph priors, and sample sizes can produce different prediction rules and failure modes.Source: E. Synthesis and Evaluation Implications · p. 12

IV. SIMULATORS AS SOURCES OF SUPERVISION

• Simulators generate the datasets and structural labels used in the supervised causal discovery workflow.Source: §IV · p. 12
• The section defines a distribution over training tasks to examine how simulator assumptions affect learning.Source: §IV · p. 12
• It considers which graph features are identifiable within that distribution and what may change at deployment.Source: §IV · p. 12

A. Simulators as Data-and-Label Generators

• A task specification η comprises a graph, mechanisms, noise, interventions or environments, latent variables, background knowledge, and sample size.The simulator samples η and then generates a dataset from it.Source: A. Simulators as Data-and-Label… · p. 12
• The simulator’s target τ(η) can be a DAG, CPDAG, PAG, skeleton, edge indicator, direction, or adjacency matrix.The structural label varies with the task; Arrow’s directed-graph label differs from its skeleton-and-order decoder outputs.Source: A. Simulators as Data-and-Label… · p. 12
• The simulator prior induces a joint distribution over datasets and structural labels, which paired examples can use to train predictors of graph distributions or marginals.The learned rule approximates the prediction rule defined by the simulator prior and loss.Source: A. Simulators as Data-and-Label… · p. 12
• Representative simulators vary in graph distributions, mechanisms, data types, and label rules, with some methods also using interventions, multiple domains, or benchmark inputs.The listed table settings are selected rather than exhaustive, and some studies evaluate on graph families distinct from their training generators.Source: A. Simulators as Data-and-Label… · p. 12

Simulator-Induced Constraints

• Simulator-induced distributional regularities can add information beyond the conditional-independence model, but only when they are built into the data-generating family.Csim denotes regularities that hold across the simulator family.Source: B. Simulator-Induced Constraints and… · p. 13
• Graph-specific constraints can refine Markov-equivalence classes, but different constraint sets need not define nonoverlapping distribution families, and a graph prior alone does not establish identifiability.Identifiability still requires the target implication to hold within the simulator’s model class.Source: B. Simulator-Induced Constraints and… · p. 13
• Functional-model restrictions, multienvironment invariances, and latent-variable models can supply directional or distributional constraints beyond ordinary conditional independencies.Examples include additive independent noise, non-Gaussian linear noise, and equality constraints on observed margins.Source: B. Simulator-Induced Constraints and… · p. 13

Benefit and Limits of Supervision

• A predictor can learn a target within a simulator’s model class when that target is identifiable there, while finite data, capacity, and optimization still limit accuracy.When identifiability fails, a probabilistic predictor can represent uncertainty among compatible labels, with probabilities depending on the prior.Source: B. Simulator-Induced Constraints and… · pp. 13–14
• Comparing predicted edge probabilities across sample sizes, graph priors, and same-distribution models with different labels can probe simulator restrictions, but cannot prove a population implication or identify a graph outside the simulator family.Excluding forks can remove graph representatives without resolving every remaining direction; confidence near zero or one may also reflect a concentrated prior, missing cases, or poor calibration.Source: B. Simulator-Induced Constraints and… · pp. 14–15

Simulator-Induced Constraints

• In the Verma example, fixing C yields a kernel whose sum over B is invariant to A, a constraint not generally imposed by simulators over ordinary observed-variable DAGs.The Verma constraint and its generalizations are examples of non-CI constraints, –.Source: B. Simulator-Induced Constraints and… · p. 13, Figure 8 · p. 15

Transfer and Test-time Adaptation

• Low training risk can coexist with high test risk when a simulator favors an ordering, omits graph patterns, or restricts mechanisms and noise.Task frequencies, finite-sample signal strength, and the learned representation can also affect test risk.Source: C. Transfer, Test-time Adaptation, and… · p. 15
• Simulator choices can encode latent-variable equalities, intervention-based orientation information, or restricted graph and noise priors that improve average prediction within the training family without identifying a broader target.A single measure of distributional similarity does not distinguish these effects; varying the relevant simulator factors is needed to assess them.Source: C. Transfer, Test-time Adaptation, and… · p. 15

Local Simulators

• Local simulators construct training tasks from the test dataset, with ML4S, TTT-SCL, and TICL using different fitting or augmentation procedures and incurring extra simulation and training costs.ML4S uses vicinal graphs, TTT-SCL refines candidate graphs, and TICL uses self-augmentation with interventional data and JCI information.Source: C. Transfer, Test-time Adaptation, and… · p. 15

Limits of Test-time Adaptation

• Matching an observed marginal cannot distinguish causal models that generate that same marginal, and fitting a simulator to one dataset can reproduce sampling noise.Evaluation should use held-out tasks and include task-construction and retraining time; real-data recovery benefits remain an empirical question.Source: C. Transfer, Test-time Adaptation, and… · pp. 15–16

Evaluation Design

• Studies should document the simulator and label rule, including which factors vary and depend on one another, and explain how predicted probabilities become the graph being scored.Mechanism parameters may depend on the sampled graph, and background knowledge may depend on ancestral relations.Source: D. Evaluation and Trustworthy Benchmarks · p. 16
• Controlled changes to graphs, mechanisms, noise, latent confounding, interventions, or knowledge test specific transfer claims, but provide evidence only about the tested shifts.For example, robustness to new noise distributions calls for varying those distributions.Source: D. Evaluation and Trustworthy Benchmarks · p. 16

Causal Reference Graphs

• Held-out tasks from the training simulator test estimation and amortized inference within that family; transfer to other families or real systems requires separate evidence.Describing the training and test simulators helps distinguish these claims.Source: D. Evaluation and Trustworthy Benchmarks · p. 16
• Benchmark reference graphs range from known synthetic generating graphs to expert- or model-specified semi-synthetic graphs and real-world graphs based on external causal evidence, whose uncertainty should be considered when scoring.Semi-synthetic reference graphs need not be established through interventions; the Sachs consensus graph uses biological and interventional evidence.Source: D. Evaluation and Trustworthy Benchmarks · p. 16

Trustworthy Benchmarks

• Benchmark reports should state the reference graph’s source and type, identify uncertain edges or orientations, and use metrics matched to the supported target.CPDAG evaluation avoids penalizing a method for leaving an observationally unresolved direction open.Source: D. Evaluation and Trustworthy Benchmarks · p. 16
• Evaluations using repository networks, synthetic SCMs, simulated gene expression, and the Sachs system draw on different evidence sources and do not by themselves establish reliability in other real systems.Further benchmarks should document the interventions or domain evidence behind reference graphs and specify uncertain edges.Source: D. Evaluation and Trustworthy Benchmarks · p. 16

• When graphs generate the same observed distribution, supervised learning alone cannot identify which graph generated the data; restrictions, interventions, or background knowledge may resolve the ambiguity.Source: §V · p. 16
• The compared methods vary in prediction targets, model components, and training regimes, while simulator restrictions can supply information beyond conditional independence that the encoder must retain.Accuracy within one simulator family does not establish accuracy on data from other families.Source: §V · p. 16
• Evaluation should score CPDAG recovery as an equivalence class and explain what assumptions or evidence identify additional directions for full-DAG recovery.Studies should also describe how model outputs are converted into the graph being scored.Source: §V · p. 16
• Transfer remains to be established for mechanisms, graph structures, or latent confounders absent from training; benefits of broad pretraining and test-time adaptation require out-of-distribution measurement.Source: §V · p. 16
• Test-time adaptation evaluations should account for additional computation and the risk of fitting sampling noise.Source: §V · p. 17
• Real-data evaluations need documented interventional or domain evidence behind reference graphs, including uncertain edges and directions; benchmark results alone do not establish reliability elsewhere.Benchmarks with documented causal evidence and metrics matched to the graph target are needed to assess progress.Source: §V · p. 17