PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentiable Causal Discovery from Interventional Data

Philippe Brouillard, Sébastien Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, Alexandre Drouin

arXiv:2007.01754v2cs.LGstat.ML

TL;DR

Causal DAG learning is difficult because DAG search is combinatorial and observational data may not identify a unique graph. The paper proposes DCDI, a neural continuous-constrained method that incorporates interventional data, including unknown-target interventions, and reports favorable comparisons across intervention settings. Its conclusions are bounded by causal sufficiency and the need to understand the method’s assumptions and potential consequences.

  • Problem

    Causal discovery is challenged by a super-exponential DAG search space and limited identifiability from observational data alone, while interventional data can improve identification.

  • Method

    DCDI uses neural conditional-density models, including normalizing flows, in a continuous-constrained framework optimized with an augmented Lagrangian to leverage interventional data.

  • Results

    DCDI shows competitive or overall better performance than state-of-the-art methods across perfect, imperfect, and unknown-target intervention settings, especially for highly connected graphs.

  • Takeaways & Limitations

    Interventional data and expressive neural architectures can be incorporated into differentiable causal structure learning while retaining theoretical identification results under stated assumptions.

  • Takeaways & Limitations

    The method assumes causal sufficiency, and its use requires understanding its assumptions because positive applications can still have unintended consequences.

Abstract

from arXiv · show

Learning a causal directed acyclic graph from data is a challenging task that involves solving a combinatorial problem for which the solution is not always identifiable. A new line of work reformulates this problem as a continuous constrained optimization one, which is solved via the augmented Lagrangian method. However, most methods based on this idea do not make use of interventional data, which can significantly alleviate identifiability issues. This work constitutes a new step in this direction by proposing a theoretically-grounded method based on neural networks that can leverage interventional data. We illustrate the flexibility of the continuous-constrained framework by taking advantage of expressive neural architectures such as normalizing flows. We show that our approach compares favorably to the state of the art in a variety of settings, including perfect and imperfect interventions for which the targeted nodes may even be unknown.

1 Introduction

The paper targets data-driven causal discovery, where observational data may identify only an equivalence class, and develops DCDI to use interventional data within continuous optimization. DCDI supports diverse intervention settings and neural architectures, including unknown targets and normalizing flows.

  • Motivation: Causal discovery seeks to learn a DAG whose edges represent direct causal relationships from observational and, when available, interventional data.Interventional data can improve identifiability, and enough interventions can make the DAG exactly identifiable.
  • Challenge: Observational data alone may identify the true DAG only up to a Markov equivalence class, while interventions can reduce this ambiguity.The interventional equivalence class is a subset of the observational Markov equivalence class.
  • Contribution: DCDI is a differentiable causal structure-learning method that leverages perfect, imperfect, and unknown-target interventions.The method includes two instantiations, including a universal density approximator based on normalizing flows.
  • Evaluation: The paper evaluates DCDI across multiple functional forms and intervention types against state-of-the-art methods.The comparison covers a wide variety of experimental conditions.

2 Background and related work

Causal structure learning combines graph recovery with distributional modeling, but DAG search is super-exponential and identifiability is limited. Continuous-constrained methods replace combinatorial search with differentiable optimization under an acyclicity constraint, while prior approaches vary in model and optimization choices.

  • Causal graphical models: A causal graphical model couples a joint distribution with a DAG, whose edges encode direct causal relations between variables.The distribution is Markov to the graph and can be factorized according to node parents.
  • Interventions: Interventions replace targeted-node conditionals and modify the distribution locally; perfect interventions additionally remove dependencies with targeted nodes’ parents.Imperfect interventions alter causal relationships without necessarily removing parent dependencies.
  • Existing approaches: Causal structure learning faces a super-exponential DAG search space and incomplete identifiability, motivating constraint-based, score-based, and hybrid methods.These families include PC, GIES, CAM, IGSP, and UT-IGSP for settings involving observational or interventional data.
  • Continuous-constrained optimization: Continuous-constrained methods encode graphs through weighted adjacency matrices and enforce acyclicity while optimizing a differentiable model objective.The constrained problem is approximately solved with an augmented Lagrangian procedure.
  • Methodological landscape: Existing continuous-constrained methods differ in model architecture, adjacency parameterization, and optimization procedure, including linear models and neural networks.Some methods decouple graph parameters from density-model parameters, whereas others derive adjacency weights from neural-network parameters.

3 DCDI: Differentiable causal discovery from interventional data

DCDI defines a theoretically grounded score for interventional causal discovery, models conditional densities with masked neural networks, and optimizes a stochastic graph relaxation under acyclicity. Its theory covers known and unknown intervention targets, while experiments support competitive performance across intervention settings.

  • Score and model: DCDI scores candidate DAGs using regularized maximum log-likelihood across interventional distributions, with conditional densities parameterized by masked neural networks.The model can use Gaussian distributions or expressive normalizing flows.
  • Score and model: The interventional score favors graphs whose non-target conditionals remain invariant across interventions.This encodes the causal-graphical requirement that interventions alter targeted mechanisms while leaving non-target mechanisms invariant.
  • Known-target theory: Under stated capacity, faithfulness, positivity, and entropy assumptions, maximizing the known-target score recovers the true graph’s I∗-Markov equivalence class.Adding interventional targets can shrink this class; individually targeting every node can identify the true DAG.
  • Continuous-constrained formulation: DCDI relaxes the binary adjacency matrix to Bernoulli variables with sigmoid probabilities and optimizes the resulting objective under an acyclicity constraint.The augmented Lagrangian converts the constrained problem into unconstrained subproblems optimized with stochastic gradient descent.
  • Unknown-target theory: When intervention targets are unknown, sparse-family regularization enables joint optimization of the graph and intervention family, recovering both under the theorem’s assumptions.The result holds for sufficiently small positive regularization coefficients.

4 Experiments

The experiments evaluate DCDI across intervention types, causal mechanisms, graph sizes, densities, and sample sizes using SHD and SID. DCDI is competitive across settings, generally strongest for denser graphs and unknown-target interventions, while its scalability is examined up to 100-node graphs and 1 million examples.

  • Experimental setup: The evaluation covers perfect known, imperfect known, and perfect unknown interventions across linear and nonlinear causal mechanisms.Synthetic graphs use 10 or 20 nodes, densities of e = 1 or 4, ten graphs per condition, and 10 000 samples per data set.
  • Evaluation metrics: SHD counts differing edges, while SID measures differences between estimated and true DAGs in their causal inference statements.Both metrics compare the estimated graph with the ground-truth graph; lower values are better for the Figure 2 metrics.
  • Perfect interventions: For perfect interventions, DCDI-G and DCDI-DSF are competitive across conditions and outperform all methods on graphs with higher average edge counts.GIES often performs best on linear data because its linear Gaussian assumptions match that setting.
  • Imperfect interventions: For imperfect interventions, DCDI-G and DCDI-DSF remain competitive and outperform other methods for graphs with higher connectivity.The authors report conclusions similar to the perfect-intervention setting.
  • Unknown interventions: For perfect interventions with hidden targets, DCDI-G and DCDI-DSF generally outperform UT-IGSP except on sparse linear data sets.The comparison uses the same data as the perfect-intervention setting while hiding intervention targets.
  • Overall results: Across intervention settings, DCDI has overall the best performance, while DCDI-DSF’s advantage over DCDI-G is marginal in the reported experiments.Additional ablations support neural networks over linear models and show that the score leverages interventional data.
  • Scalability: The scalability analysis considers graphs up to 100 nodes and data sets up to 1 million examples.DCDI computes a matrix exponential at each gradient step with cost O(d3), while cited kernel tests scale poorly with sample size.

5 Conclusion

The paper concludes that DCDI is a theoretically grounded continuous-constrained method for causal discovery with interventional data and expressive neural architectures. It reports competitive graph recovery and scalability on real and simulated data, while identifying time-series extensions and cross-system representations as future directions.

  • Conclusion: DCDI leverages perfect, imperfect, and unknown-target interventions together with expressive neural architectures such as normalizing flows.The method is presented as a general continuous-constrained approach for causal discovery.
  • Conclusion: DCDI is competitive with state-of-the-art algorithms on real and simulated data in graph recovery and scalability.
  • Future work: The authors propose extending DCDI to time-series data and learning cross-system variable representations for low-data causal discovery.The time-series direction treats non-stationarities as unknown interventions.

Broader impact

The paper describes causal structure learning as supporting system understanding and action recommendation, with possible applications in science and policymaking. It also emphasizes ethical risks and the need to respect methodological assumptions when using results for consequential decisions.

  • Potential benefits: Causal structure learning can support understanding complex systems and recommending actions once that understanding is achieved.The paper gives scientific investigation and policymaking as example application areas.
  • Risks: The method could be used in undesirable applications, including unethical policy decisions involving human-rights violations.The authors place responsibility for ethical and benevolent use on users.
  • Assumptions and responsible use: Users should understand the method’s assumptions because results under violated assumptions should support decisions rather than be treated as absolute truth.The paper lists causal sufficiency, iid interventional samples, acyclicity, and infinite-data theory as assumptions or scope conditions.

A.1 Theoretical Foundations for Causal Discovery with Imperfect Interventions

Interventional graphical models encode intervention-specific information through augmented I-DAGs, refining observational Markov equivalence via skeletons and immoralities.

  • Interventional graphical representations: An I-DAG augments a DAG with one intervention node ζk for each intervention and edges from ζk to every targeted variable.The observational setting is indexed by I1 := ∅.
  • Interventional graphical representations: The I-Markov property requires graph d-separations to imply conditional independences across all distributions and intervention-specific invariances for non-targeted variables.This property characterizes distributions compatible with the interventional graph.
  • I-Markov equivalence: Two DAGs are I-Markov equivalent exactly when their I-DAGs share the same skeleton and immoralities.Thus, interventions can distinguish graphs that remain equivalent observationally.
  • Illustration: For a single intervention, reversing an edge may break or create an immorality, making the graph unique, whereas reversals preserving skeleton and immoralities leave multiple equivalent DAGs.Figure 5 illustrates both identifiable and non-identifiable cases.
  • I-Markov equivalence: Adding intervention targets generally reduces the I-Markov equivalence class by introducing additional immoralities in the I-DAG.With individual interventions on every node, the original DAG is alone in its equivalence class.

A.2 Proof of Theorem 1

Theorem 1 establishes score-based identification under capacity, faithfulness, positivity, and finite-entropy assumptions, while showing how intervention effects distinguish non-equivalent graphs.

  • Assumptions: Theorem 1 assumes sufficient model capacity, I∗-faithfulness, strictly positive densities, and finite differential entropies.These assumptions support exact representation, non-pathological interventions, and well-defined score comparisons.
  • Assumptions: I∗-faithfulness requires observational faithfulness and that every variable potentially affected by an intervention is actually affected.In a simple case, the intervention must change the targeted conditional; with no conditioning set, affected descendants must have changed marginals.
  • Assumptions: An intervention can violate I∗-faithfulness when it changes a targeted conditional without changing the corresponding marginal distribution.The binary example changes X2’s conditional probabilities while leaving p(2)(X2) equal to p(1)(X2).
  • Score comparison: The proof rewrites score differences using KL-divergence terms, which are nonnegative and strictly positive when the true density is not representable by a candidate DAG.Finite differential entropies ensure the ground-truth score is finite.
  • Identification: Under the assumptions and sufficiently small λ, maximizing the score returns a DAG I∗-Markov equivalent to the ground-truth DAG.The proof compares graphs differing in skeletons or immoralities and shows the ground-truth score is larger.
  • Unknown targets: The same framework identifies unknown intervention targets jointly with the graph, returning I∗-Markov equivalence and the true intervention family for sufficiently small λ and λR.The proof adds cases for incorrect intervention families before applying the known-target argument.

B.1 Synthetic data sets

The experiments use synthetic DAGs with varied graph sizes, mechanisms, and intervention types, while DCDI is optimized through augmented-Lagrangian stochastic training. The study also examines learned graph structure and intervention targets during training.

  • Synthetic data sets: Synthetic data begin with Erdős-Rényi DAGs, followed by sampled causal-mechanism parameters and interventions whose design varies with graph size.Ten-node graphs intervene on every node; 20-node graphs target one to two randomly chosen nodes.
  • Synthetic data sets: The benchmarks include linear, additive-noise, and nonlinear non-additive-noise mechanisms, with imperfect interventions perturbing mechanism parameters.ANM mechanisms use one-hidden-layer neural networks, while nonlinear non-additive mechanisms jointly depend on parents and noise.
  • Optimization: DCDI jointly optimizes neural-network parameters and graph parameters using an augmented Lagrangian, with RMSprop approximately solving successive unconstrained subproblems.The procedure updates multipliers and penalty coefficients between subproblems and stops when the acyclicity criterion is sufficiently small and the thresholded graph is acyclic.
  • Optimization: Graph gradients use a Straight-Through Gumbel-Softmax estimator that samples discrete Bernoulli edges while backpropagating through soft probabilities.The estimator is biased but has lower variance than REINFORCE in the described implementation.
  • Optimization: A continuous relaxation of the adjacency matrix performed significantly worse because neural-network weights could compensate for entries approaching zero without changing likelihood.This motivates retaining a discrete-sampling relaxation for graph structure.
  • Optimization: During training, constraint pressure removes many false edges, while learned target probabilities are compared with ground-truth intervention targets.Figure 6 tracks likelihood, augmented-Lagrangian objectives, and graph probabilities across iterations; Figure 7 compares learned and true targets.

C.1 Real-world data set

The real-world evaluation uses flow-cytometry measurements under protein perturbations and compares DCDI variants with established methods. DCDI is broadly comparable on the dataset, while model capacity matters for complex synthetic distributions.

  • Real-world data set: The real-world dataset contains 5,846 flow-cytometry measurements of phosphoprotein and phospholipid expression under protein-activation or inhibition interventions.Conditions whose perturbations were not directly applied to measured proteins were excluded.
  • Real-world results: DCDI-G and DCDI-DSF have SHDs comparable to GIES and CAM, outperform IGSP in SID, and achieve F1 scores similar to GIES.The modified CAM version has the highest F1 score and low SID, while no method performs strongly across every reported measure.
  • Model capacity: DCDI-DSF systematically infers the correct causal direction on the toy datasets, whereas DCDI-G can fail when its capacity cannot represent complex distributions.On the X dataset, DCDI-G falsely concludes independence; on DNA with perfect interventions, it recovers the direction despite poor density modeling.
  • Model capacity: The synthetic multimodal distributions are motivated as possible consequences of latent variables that select between different mechanisms.The paper notes that such latent variables need not be confounders when they parent only one observed node.

C.3 Scalability experiments

Scalability experiments vary sample size and graph size under fixed 12-hour and 16GB limits. DCDI scales to very large nonlinear datasets and performs strongly on graphs up to 50 nodes, with weaker runtime at 100 nodes.

  • Number of examples: DCDI was the only nonlinear-capable method to run on 1 million examples without exceeding the time or memory limits.The experiments imposed a 12-hour runtime limit and 16GB RAM limit.
  • Number of variables: For 10–50-node datasets, DCDI-GPU outperforms other methods in SHD and SID while maintaining runtime similar to CAM.GPU use begins to provide a benefit at graph sizes of 50 nodes or more.
  • Number of variables: At 100 nodes, DCDI runtime increases substantially, while SHD and SID become comparable to the much faster GIES.The paper attributes the weaker performance to high-dimensional conditionals.

C.4 Ablation study

Ablations test intervention information, intervention-aware losses, and neural-network capacity. They show benefits from using interventions and high-capacity models, especially for denser or nonlinear settings.

  • Effect of number of interventions: DCDI-G performance improves as the number of perfect interventions increases, with the clearest gain when moving from purely observational data to five interventions.DCDI-G with interventional data is consistently better than DCD, whose loss ignores intervention information.
  • Relevance of DCDI score: Across perfect-known, imperfect-known, and perfect-unknown settings, DCDI has a clear SHD advantage over DCD and observational-only DCDI.For SID, the advantage is limited on sparse graphs but usually appears with higher connectivity.
  • Relevance of DCDI score: The intervention-aware losses are reported as pertinent across the tested intervention types.The comparison includes perfect, imperfect, and unknown-target interventions.
  • Relevance of neural network models: The linear DCDI variant remains competitive on linear data but performs worse on nonlinear datasets than hidden-layer variants.This supports the relevance of higher-capacity neural models for nonlinear causal mechanisms.
  • Comparison with IGSP: Compared with IGSP, DCDI is better or comparable on sparse graphs and often better on graphs with higher connectivity under the tested intervention settings.These comparisons include alternative perfect-intervention and more modest imperfect-intervention constructions.

C.6 Evaluation on unseen interventional distributions

The evaluation tests how well learned causal graphs model unseen interventions using held-out interventional distributions. DCDI performs best on the nonlinear non-additive-noise datasets, while IGSP leads on Sachs.

  • Interventional performance measures log-likelihood on an unseen intervention after training excludes that intervention’s data.For fair comparison, distributions are fitted with normalizing flows when methods do not model distributions; the targeted-node conditional is ignored during evaluation.
  • DCDI-G and DCDI-DSF have the best performance on nonlinear non-additive-noise datasets across 10 dense 10-node graphs.Each graph uses one randomly selected unseen interventional setting.
  • IGSP has the best Sachs performance, followed by DCDI-G, although this ordering differs from the structural metrics.The evaluation holds out data from interventions applied to protein Akt and reports log-likelihood with its standard deviation.

C.7 Comprehensive results of the main experiments

The comprehensive experiments report structural metrics across graph sizes, intervention types, and unknown-target settings. DCDI remains competitive across conditions and outperforms other methods on graphs with higher connectivity.

  • Hyperparameters for DCDI, CAM, and GIES were selected using held-out likelihood, whereas IGSP, UTIGSP, and JCI-PC were evaluated across hyperparameter values.The latter methods lack likelihood models, so additional SHD-minimizing variants were reported, an arrangement described as unfair to DCDI, CAM, and GIES.
  • DCDI has competitive performance in almost all conditions and outperforms other methods for graphs with higher connectivity.The reported SHD and SID values are means and standard deviations over ten datasets for each condition.
Loading 2007.01754v2…