Source-linked AI summary
Causal Generative Neural Networks
Olivier Goudet, Diviyan Kalainathan, Philippe Caillou, Isabelle Guyon, David Lopez-Paz, Michèle Sebag
TL;DR
Score-based methods assume powerful external score-functions to detect diverse causal relations, and most methods are not differentiable, making them unsuited for deep learning pipelines. CGNNs learn functional causal models as differentiable generative networks by minimizing MMD between observational data and generated samples, while allowing hidden confounders. CGNNs achieve state-of-the-art performance on cause-effect inference, v-structure identification, and multivariate causal discovery with hidden variables.
Problem
Score-based methods assume powerful external score-functions to detect diverse causal relations, and most methods are not differentiable, making them unsuited for deep learning pipelines.
Source: §1 · p. 1Method
CGNNs learn functional causal models as differentiable generative networks by minimizing MMD between observational data and generated samples, while allowing hidden confounders.
Source: §1 · p. 1, §3.2 · p. 3, §3.4 · p. 3Results
CGNNs achieve state-of-the-art performance on cause-effect inference, v-structure identification, and multivariate causal discovery with hidden variables.
Source: §1 · p. 1Takeaways & Limitations
Once trained, CGNNs are causal models that can simulate the outcomes of interventions.
Source: §5 · p. 6The representational guarantee is stated for continuous variables whose density is continuous and strictly positive on a compact subset of R^d, zero elsewhere, and factorizes along a DAG.
Source: §2.2 · p. 2
Abstract
from arXiv · showhide
We present Causal Generative Neural Networks (CGNNs) to learn functional causal models from observational data. CGNNs leverage conditional independencies and distributional asymmetries to discover bivariate and multivariate causal structures. CGNNs make no assumption regarding the lack of confounders, and learn a differentiable generative model of the data by using backpropagation. Extensive experiments show their good performances comparatively to the state of the art in observational causal discovery on both simulated and real data, with respect to cause-effect inference, v-structure identification, and multivariate causal discovery.
1 Introduction
Deep learning systems can mistake correlation for causation, while observational causal discovery is needed when experiments are costly, unethical, or impossible. CGNNs propose a differentiable generative approach that combines distributional asymmetries and conditional independencies across bivariate and multivariate discovery.
1 Introduction
- Deep learning models can mistake correlation for causation, with potentially catastrophic consequences for agents that plan and decide from observations.The passage notes deep learning's predictive abilities across image classification, speech recognition, language translation, and game playing.Source: §1 · p. 1
- Observational causal discovery estimates causal relations from observations when experiments are costly, unethical, or impossible, but existing methods rely on strong assumptions.Approaches use distributional asymmetries for bivariate relations or conditional independencies for structures with three or more variables.Source: §1 · p. 1
- Many causal discovery methods are not differentiable, making them unsuited for deep learning pipelines; score-based methods also require powerful external score functions.The listed assumptions across approaches include linearity, additive noise, determinism, and large annotated corpora.Source: §1 · p. 1
- CGNNs learn functional causal models as differentiable generative networks, using backpropagation to address both bivariate and multivariate causal discovery, including hidden confounders.They use distributional asymmetries and conditional independencies, and estimate both a causal graph and the full joint distribution.Source: §1 · p. 1
2 The language of causality: FCMs
The section defines functional causal models as equations relating variables to their causes and noise, then describes their generative and intervention semantics. It also sets out formal causal concepts and conditions under which an FCM can represent a joint distribution.
2 The language of causality: FCMs
- An FCM is a triplet of a graph, mechanisms, and noise distribution, with each variable determined by its parents and a noise variable; the section restricts the graph to a DAG and noise to U[0, 1].A direct causal relation corresponds to an edge from a parent variable to the variable it determines.Source: §2 · pp. 1–2
2.1 Generative models and interventions
- To generate data, an FCM draws noise variables and computes variables in graph topological order; interventions freeze a variable and discard its incoming causal influences.The resulting interventional distribution differs from conditioning and can quantify the overall effects of a decision.Source: §2.1 · p. 2
2.2 Formal background and notations
- The section defines conditional independence, v-structures, confounders, and graph skeletons, and describes causal sufficiency, Markov, and faithfulness assumptions.Causal sufficiency assumes no unobserved confounders; Markov and faithfulness relate graph d-separations to observational conditional independencies in opposite directions.Source: §2.2 · p. 2
- For continuous variables with a continuous, strictly positive density on a compact subset of R^d and zero elsewhere, a DAG-factorized joint distribution can be represented by an FCM with continuous mechanisms.The proof proceeds by induction over the DAG's topological order.Source: §2.2 · p. 2
3 Causal generative neural networks
CGNNs use graph-structured generative networks to approximate observational distributions, score candidate structures against data, and search for causal graphs. The framework also extends its causal equations to represent hidden common causes.
3 Causal generative neural networks
- Under Proposition 1’s conditions, a CGNN can approximate the continuous observational distribution to arbitrary accuracy.Source: §3 · p. 2
3.1 Approximating continuous FCMs with CGNN
- Networks arranged along a given graph can approximate the joint distribution defined by a continuous functional causal model.The construction approximates each mechanism in topological order.Source: §3.1 · pp. 2–3
3.2 Scoring metric
- MMD between observational and generated samples provides the tractable fit score, and its differentiability permits learning by backpropagation.The Gaussian-kernel MMD is differentiable; the score also includes a graph-size term.Source: §3.2 · p. 3
- With an infinite observational sample, the CGNN’s generated distribution can match the true observational distribution to arbitrary precision under the stated conditions.Source: §3.2 · p. 3
- The generative-network representation uses distributional asymmetries and joint-distribution approximation to exploit conditional independencies in bivariate and multivariate causal modeling.Source: §3.2 · p. 3
3.3 Searching causal graphs with CGNNs
- Given a known skeleton, the search orients edges using pairwise CGNN scores, corrects cycles, then iteratively reverses edges to improve the multivariate score without creating cycles.Source: §3.3 · p. 3
3.4 Dealing with hidden confounders
- To model hidden confounders, adjacent variables can share hypothetical unobserved causes, and graph search can reverse, add, or remove edges.A graph-size penalty promotes simpler graphs in this setting.Source: §3.4 · p. 3
4 Experiments
Across cause-effect inference, v-structure identification, and multivariate discovery, CGNN is evaluated against existing methods on observational data. Results cover known and perturbed graph skeletons, real protein data, and settings with hidden confounders.
4.1 Experimental setting
- Experimental comparisons include cause-effect datasets, a three-variable v-structure setting, artificial and real multivariate graphs, and hidden-confounder scenarios.For the toy cause-effect example, the number of hidden units is problem-dependent, so the authors emphasize cross-validating model capacity.Source: §4 · p. 3, §4.1 · p. 4, §4.3 · p. 5
4.2 Discovering cause-effect relations
- CGNN demonstrates competitive cause-effect inference, with generative CGNN and GPI performing well on most datasets, including real-world CE-Tübingen pairs.The reported computational costs are 24 minutes on GPU for CGNN and 32 minutes on CPU for GPI.Source: §4.2 · pp. 4–5, Table 1 · p. 5
4.3 Discovering v-structures
- For a three-node skeleton, CGNN discriminates v-structures from other candidate graphs and assigns the ground-truth graph a lower MMD of 0.127.The other reported MMDs are 0.202 and 0.180; the experiment selects the graph with minimal MMD.Source: §4.3 · p. 5
4.4 Discovering multivariate causal structures
- With the true skeleton supplied, CGNN significantly outperforms the other algorithms on SHD and SID, while PC-HSIC is second best.The comparison concerns edge orientation from a known skeleton; CGNN and PC-HSIC are also the most computationally expensive methods, averaging 4 hours on GPU and 15 hours on CPU, respectively.Source: §4.4 · p. 5, Table 2 · p. 6
- With 20% of skeleton edges perturbed, all algorithms score lower, but CGNN is more robust overall than PC-HSIC.The perturbation introduces about 10 false edges per 50-edge skeleton; the authors attribute constraint-based methods’ weaker robustness tentatively to their reliance on graph structure.Source: §4.4 · p. 5
- On the real protein network, CGNN recovers the reported raf→mek→erk signal-transduction pathway and assigns low scores to wrongly oriented edges.The dataset contains 853 observational data points, and the algorithms are given the causal graph skeleton.Source: §4.4 · p. 6, Figure 3 · p. 6
4.5 Dealing with hidden confounders
- With hidden confounders, CGNN is reported as robust; true causal edges have high confidence, while confounding edges are removed or receive low confidence.Source: §4.5 · p. 6, Table 3 · p. 6
5 Conclusion
CGNN is a generative-neural-network framework for learning functional causal models from observational data by minimizing the discrepancy between generated and observed samples. Once trained, its causal models can simulate interventions; future work includes extending the approach and improving its efficiency.
5 Conclusion
- CGNN learns functional causal models from observational data using generative neural networks.Source: §5 · p. 6
- CGNN minimizes maximum mean discrepancy between generated samples and observed data, and its trained causal models can simulate intervention outcomes.Source: §5 · p. 6
- Future work includes categorical and temporal data, sufficient identifiability conditions, and improved computational efficiency.Source: §5 · p. 6