Source-linked AI summary
The Randomized Causation Coefficient
David Lopez-Paz, Krikamol Muandet, Benjamin Recht
TL;DR
A general method is lacking for inferring causality without prior knowledge about whether noise exists, motivating automatic learning of causal patterns from labeled data. The method treats samples from distributions over variable pairs as inputs, featurizes them with kernel mean embeddings and random Fourier features, and learns causal labels through supervised classification. Numerical simulations support learning causal patterns from data; RCC ranked third in ChaLearn’s challenge and won the fastest-code prize.
Problem
A general method is lacking for inferring causality without prior knowledge about whether noise exists, motivating automatic learning of causal patterns from labeled data.
Source: §1 · p. 2Method
Results
Numerical simulations support learning causal patterns from data; RCC ranked third in ChaLearn’s challenge and won the fastest-code prize.
Source: §5 · p. 6, §4.2 · p. 6Takeaways & Limitations
RCC’s challenge entry took two minutes and achieved bidirectional AUC 0.732843, while the winning hand-engineered entry took 30 minutes and achieved bidirectional AUC 0.826413.
Source: §4.2 · p. 6The authors describe their training-data generation as basic and suggest that more heterogeneous causes, mechanisms, noise distributions, and pollution schemes could yield a more powerful engine.
Source: §4.1 · p. 5
Abstract
from arXiv · showhide
We are interested in learning causal relationships between pairs of random variables, purely from observational data. To effectively address this task, the state-of-the-art relies on strong assumptions regarding the mechanisms mapping causes to effects, such as invertibility or the existence of additive noise, which only hold in limited situations. On the contrary, this short paper proposes to learn how to perform causal inference directly from data, and without the need of feature engineering. In particular, we pose causality as a kernel mean embedding classification problem, where inputs are samples from arbitrary probability distributions on pairs of random variables, and labels are types of causal relationships. We validate the performance of our method on synthetic and real-world data against the state-of-the-art. Moreover, we submitted our algorithm to the ChaLearn's "Fast Causation Coefficient Challenge" competition, with which we won the fastest code prize and ranked third in the overall leaderboard.
1. Introduction
The paper distinguishes causal direction and common-cause explanations from observational samples. It motivates learning causal patterns from data because existing approaches rely on assumptions that limit their scope.
1. Introduction
- Dependence between X and Y may reflect X causing Y, Y causing X, or a common cause.The paper aims to distinguish these possibilities using samples from their joint distribution.Source: §1 · p. 1
- Existing approaches leave a need for a general method that does not assume whether noise exists in the causal mechanism.The paper also seeks an approach that can extend to new model hypotheses without requiring a new specific algorithm.Source: §1 · p. 2
- The paper asks whether causal relationships can be learned as patterns from large amounts of labeled data.This question is motivated by the limited scope of methods built around particular causal-mechanism assumptions.Source: §1 · p. 2
2. Learning to learn causal inference
The paper formulates causal inference as supervised learning from samples drawn from distributions over variable pairs. Its approach assumes those distributions usually contain observable clues to causal labels, while acknowledging that this need not hold universally.
2. Learning to learn causal inference
- A training example pairs a sample from a variable-pair distribution with its causal label; extracted distribution features train a classifier for unseen pairs.The basic labels distinguish X → Y from Y → X, and two additional labels can represent common cause and independence.Source: §2 · p. 2
- Unlike ordinary classification on feature vectors, the learners receive samples from probability distributions.The setup assumes paired distributions and labels come from a mother distribution, and speculates that observable distribution properties reveal causal relationships in most cases.Source: §2 · p. 2
- The setup's assumptions are not universal, although the authors say their experiments suggest wide applicability in real-world data.The paper notes that absolutely general causal inference is not possible.Source: §2 · p. 2
3. Featurizing distributions with kernel mean embeddings
The paper represents distributions with kernel mean embeddings estimated from samples, then uses these representations for supervised causal inference. Random Fourier features provide a low-dimensional approximation when kernel features are expensive or infinite-dimensional.
3. Featurizing distributions with kernel mean embeddings
- A characteristic kernel makes the kernel mean embedding uniquely determine its probability distribution.In practice, the paper estimates the embedding from an i.i.d. sample rather than assuming access to the true distribution.Source: §3 · p. 3
- The empirical kernel mean embedding is a common, easy-to-implement feature representation of a distribution, though it can be improved.The paper presents it as an approximation to the population embedding.Source: §3 · p. 3
- Random Fourier features approximate sample embeddings for shift-invariant kernels, avoiding kernel matrices that require at least O(n^2) computation.The resulting features can be computed in O(mn) time, stored in O(1) memory, and used with standard learning algorithms beyond kernel methods.Source: §3 · p. 3
- The causal-sample representation embeds the marginal distributions of X and Y and their joint distribution.These representations and causal labels are used with a binary classifier to pose causal inference as supervised learning.Source: §3 · p. 4
4. Numerical simulations
The simulations evaluate RCC on heterogeneous real-world causal pairs and in the ChaLearn challenge. RCC leads on the Tübingen pairs, while the challenge submission ranked third and earned the fastest-code prize.
4.1 T¨ubingen data
- For the Tübingen experiment, RCC trained a 500-tree random forest on synthetic pairs built from sampled causes, Gaussian noise, and smooth spline mappings.The authors describe this training-data generation process as basic and suggest using more heterogeneous causes, mechanisms, noise, and pollution schemes.Source: §4.1 · pp. 4–5
- 79% accuracy on all 82 Tübingen pairs let RCC surpass the state of the art when forced to infer every direction.Figure 1 compares RCC, IGCI, and ANM accuracy as decision rate varies.Source: §4.1 · pp. 4–5, Figure 1 · p. 5
4.2 ChaLearn’s “Fast Causation Coefficient” challenge
- The challenge benchmark included 20,000 training pairs and both heterogeneous real-world and artificial data; ANM was not compared because its computational complexity made evaluation infeasible.The artificial pairs used nonlinear models and different noise-combination strategies.Source: §4.2 · pp. 5–6
- The ChaLearn submission used two gradient-boosting classifiers: one distinguished causal from non-causal pairs, and the other predicted direction for causal pairs.Variables were standardized, pairs were augmented by swapping variables and labels, and hyperparameters were selected by four-fold cross-validation.Source: §4.2 · p. 6
- RCC ranked third in the ChaLearn challenge and won the fastest-running-code prize, with bidirectional AUC 0.732843 versus the winner’s 0.826413.RCC’s test took two minutes, compared with 30 minutes for the winning entry; the paper notes that the winner’s hand-engineered features may have benefited from challenge-data biases.Source: §4.2 · p. 6
5. Conclusions and future Work
The paper proposes supervised causal inference from observational data using kernel mean embeddings and random Fourier features. It identifies learning other randomized statistics, extending RCC to multivariate causal graphs, and analyzing the randomized classification setting as future directions.
5. Conclusions and future Work
- RCC frames causal inference from observational data as supervised learning, using kernel mean embeddings and random Fourier features to featurize probability distributions.The authors report that their numerical simulations support the conjecture that causal patterns can be learned from data.Source: §5 · p. 6
- Proposed future directions include learning other randomized statistics, extending RCC to sets of variables and causal DAG reconstruction, and analyzing its randomized classification setting.The paper relates the first direction to measures of dependence and the third to distributional learning theory.Source: §5 · p. 6