Quantifying causal influences

Dominik Janzing; David Balduzzi; Moritz Grosse-Wentrup; Bernhard Schölkopf

Source-linked AI summary

Quantifying causal influences

Dominik Janzing, David Balduzzi, Moritz Grosse-Wentrup, Bernhard Schölkopf

arXiv:1203.6502v2Mar 29, 2012math.ST

TL;DR

Quantifying causal strength remains difficult even when a DAG and causal conditionals fully specify intervention effects. The paper proposes postulates and a relative-entropy measure based on locally destroying DAG edges, showing that alternative measures violate the postulates in simple graphs.

Problem
An objective measure is needed to distinguish weak causal links from strong ones when choosing the appropriate level of detail for a DAG.
Method
The paper defines causal strength by independently cutting DAG edges and measuring the relative-entropy distance between the original and post-cutting distributions.
Results
The proposed measure satisfies the stated causal-strength postulates, while transfer entropy and information flow violate them on simple DAGs.
Takeaways & Limitations
Causal-strength quantification can be framed through edge-destruction operations in a general framework covering arbitrary-domain dependencies, nonlinear interactions, and multidimensional or discrete variables.
Takeaways & Limitations
General additivity fails for the proposed causal-strength measure.

Abstract

from arXiv · show

Many methods for causal inference generate directed acyclic graphs (DAGs) that formalize causal relations between $n$ variables. Given the joint distribution on all these variables, the DAG contains all information about how intervening on one variable changes the distribution of the other $n-1$ variables. However, quantifying the causal influence of one variable on another one remains a nontrivial question. Here we propose a set of natural, intuitive postulates that a measure of causal strength should satisfy. We then introduce a communication scenario, where edges in a DAG play the role of channels that can be locally corrupted by interventions. Causal strength is then the relative entropy distance between the old and the new distribution. Many other measures of causal strength have been proposed, including average causal effect, transfer entropy, directed information, and information flow. We explain how they fail to satisfy the postulates on simple DAGs of $\leq3$ nodes. Finally, we investigate the behavior of our measure on time-series, supporting our claims with experiments on simulated data.

1. Introduction.

The paper frames causal strength as an objective way to distinguish weak from strong arrows in causal DAGs. It reviews dependence-, intervention-, and variance-based measures, emphasizing their differing scopes and limitations.

Causal DAG framework: Causal DAGs represent direct influences, where intervening on one variable can change another while other variables are held constant.The framework assumes causal sufficiency and uses causal conditionals associated with each node’s parents.
Motivation: Causal strength is needed because DAG descriptions require choosing how much weak causal structure to retain.The proposed measure is intended to distinguish weak arrows from strong ones objectively.
Existing measures: Average causal effect measures intervention-induced changes in outcomes, but ordinary conditioning agrees with intervention distributions only without confounding.For binary variables, ACE is defined as the difference between outcome probabilities under do(X = 1) and do(X = 0).
Existing measures: Variance-based ANOVA interpretations can fail for nonlinear influences and when a parent is statistically dependent on other parents.The between-treatment variance term should not generally be interpreted as causal variation.
Information-theoretic measures: Mutual information can represent causal strength when dependencies arise solely from the relevant influence, but conditioning choices determine when that interpretation is valid.The paper distinguishes observable or factual influence from potential influence represented by channel capacity.
Information-theoretic measures: Transfer entropy and directed information quantify conditional information in time series, while information-flow methods instead use intervention-derived distributions.Time-series measures target effects from all instances of one series to future instances of another.

2. Postulates for causal strength.

The paper proposes postulates for causal strength based on deleting arrows and requiring the resulting distribution and graph to remain causally coherent. These postulates motivate locality, dependence lower bounds, heredity, and a preference for mutual information in suitable cases.

Operational basis: Causal strength measures the impact of destroying a selected set of arrows, with zero strength requiring that deletion to make no difference.The notation CS denotes the strength of an arrow set S.
Postulates: The causal Markov postulate requires zero-strength arrow deletion to preserve compatibility between the joint distribution and the resulting DAG.This is the paper’s baseline consistency condition for relevance of arrows.
Postulates: Locality restricts CX→Y to the mechanism P(Y|PAY) and the joint distribution of Y’s parents.Causes of X and other effects of Y are irrelevant unless they are also direct causes of Y.
Postulates: If X →Y is an arrow, its causal influence must be at least I(Y;X|PAY\{X}), the conditional dependence generated by that arrow.The postulate is quantitative rather than an equality requirement.
Mutual-information cases: For several simple DAGs, the postulates imply CX→Y = I(X;Y), while a conditioned structure yields CX→Y = I(X;Y|Z).The figure identifies X and Y as the source and target of the arrow.
Postulates: Heredity requires every subset of a zero-influence arrow set to have zero influence, while joint arrow strength may still be nonzero when individual arrows are zero.The paper explicitly distinguishes this property from general additivity, which it says fails.

3. Problems of known definitions.

The paper shows that common causal-strength definitions can fail on small DAGs and time-series examples. The failures arise from confounding, conditioning choices, copy operations, and dependence among causes.

Mutual information: Mutual information can be nonzero without X →Y because a common cause Z creates dependence, while conditional mutual information can also fail in the same DAG.The paper uses simple three-variable structures to show why neither measure is generally suitable.
Transfer entropy: Transfer entropy can vanish when time-series arrows are perfect copy operations, despite strong mutual influence between the series.The past of Y already predicts present Y, so the past of X adds no information.
Transfer entropy: 0 bits of transfer-entropy influence conflicts with the 1-bit causal strength required by the postulates in the reduced single-arrow example.Directed information has the same problem because it also conditions on the entire past of Y.
Information flow: Intervention-based information flow can violate locality because I(X →Y|do(Z)) depends on the causal relation between causes X and Z.It gives different results for DAGs with the same joint distribution but different relations among causes.
Information flow: Information flow also becomes unsatisfactory when conditioning on interventions makes X effectively fixed, driving the measured information toward zero.The paper notes this behavior in a copying example as the error rate tends to zero.
Edge deletion: For sets of edges, the paper distinguishes its independent-attacker deletion from a competing construction that preserves dependencies between open ends.The competing construction requires communicating attackers, whereas the paper’s construction uses corresponding marginal distributions independently.

4. Defining the strength of causal arrows.

The paper defines causal strength by cutting selected causal channels, independently replacing their inputs with marginal distributions, and measuring the resulting relative-entropy change. The measure satisfies several structural properties while exposing failures of additivity and dependence-based alternatives in small causal systems.

Post-cutting distribution: Cutting arrows independently feeds each open channel end with the corresponding source marginal distribution, producing a distribution Markovian to the graph with those edges removed.This intervention acts on arrows rather than setting node values.
Definition: Causal strength is defined as the Kullback–Leibler divergence between the original joint distribution and the interventional post-cutting distribution.For a single edge Xk →Xl, the notation is Ck→l.
Properties: Causal influence majorizes observed dependence by adding a nonnegative divergence term between partially observed and interventional distributions.The paper identifies upstream dependency snapping through local source marginals as essential to this result.
Properties: For independent causes, causal strength equals conditional mutual information plus an additional correction, so conditional mutual information can underestimate influence under strongly dependent causes.When one cause nearly copies another, the conditional mutual information can approach zero while the correction remains substantial.
Properties: The construction preserves locality: single-edge causal strength depends on the affected conditional mechanism and the joint distribution of its parent variables.The local relative-entropy form follows because only the conditional of the target node changes.

5. Causal influence between two time series.

The paper extends causal-strength analysis to time series by measuring the impact of deleting all arrows from past X variables into Y_t. Experiments show that this measure detects causal interactions that transfer entropy can miss, while finite-sample estimation becomes difficult near deterministic limits.

5. Causal influence between two time series.: Causal strength for X→Y_t measures the impact of deleting all arrows from past X variables that point to Y_t.
Estimation: For multiple time-series copies, the estimator uses locality to retain only relevant lagged variables and generates modified samples by independently permuting observations.
Estimation: With one observed time series, estimation requires stationarity and ergodicity so sufficiently distant windows can provide an artificial statistical sample.
Comparison with transfer entropy: In a perturbed noisy-copy example, causal strength approaches 1 bit as ε→0, whereas transfer entropy fails to detect the causal interaction quantitatively.The causal-strength expression is CX→Yt = 1 + (1−ε)log(1−ε) + εlog ε.

6. Causal strength for linear structural equations.

For linear structural equations with multivariate Gaussian variables, the paper derives covariance-based expressions for causal strength after selected structural coefficients are removed. With independent parents, the measure reduces to a conditional mutual-information relationship.

General formulation: Under multivariate Gaussianity, causal strength can be computed from covariance matrices for ordered linear structural equations X = AX + E.
General formulation: To delete a set of arrows S, the structure matrix is split into AS and ĀS, and the modified equations retain only coefficients outside S.
General formulation: The modified variables are constructed using independent copies of observed variables and a modified noise representation before computing the covariance matrix of the post-deletion distribution.
Independent parents: For arrows into X_n from independent parents, the paper derives an explicit causal-strength expression for subsets S = {X_1→X_n,...,X_k→X_n}.
Independent parents: For more than two variables, causal strength relates to the fraction of conditional variance explained by X_1, not the fraction of total variance explained by it.

7. Experiments.

Experiments evaluate finite-sample estimation on linear DAGs and time-series models. The estimator can show bias, but does not significantly degrade between the tested three- and six-node DAGs; errors become pronounced near deterministic dynamics.

DAG experiments: The experiments estimate structure matrices from observations, split samples into original and modified parts, and generate modified data by permuting columns and applying altered structural equations.
DAG experiments: The relative-entropy distance between original and modified distributions is estimated using disjoint samples and nearest-neighbor distances.
DAG experiments: Significant bias appears for some edges in the simple X_1→X_2 DAG, and its distribution dependence makes correction challenging.
DAG experiments: 100 randomly generated structure matrices show no significant estimation degradation for n=6 compared with n=3 complete DAGs.
Scope: The experiments restrict the implemented estimator to linear structural equations, with Gaussianity additionally assumed for covariance-based causal-strength calculations.
Time-series experiments: As ε decreases toward deterministic dynamics, estimated causal strength can deviate strongly from the diverging true value, especially for m≥6.

8. Conclusions.

The paper defines causal strength through the effect of destroying selected edges and shows that the resulting measure satisfies its proposed postulates. It presents the framework as a basis for discussion rather than a complete or universally appropriate definition.

Conclusions: Causal strength is defined for arrows or arrow sets by quantifying the impact of destroying those edges in a causal Bayesian network.
Conclusions: The authors state that their measure satisfies the proposed natural postulates for causal strength.
Scope: The postulate list is not claimed to be complete, and measures violating it are not declared inappropriate.
Scope: The paper aims to encourage discussion within a framework covering arbitrary-domain dependencies, nonlinear interactions, and multidimensional or discrete variables.

A.1. Proof of Theorem 2.

The theorem identifies causal influence as observed influence plus a nonnegative correction measuring divergence between partially observed and interventional distributions.

A.1. Proof of Theorem 2.: The displayed derivation rewrites the second term in the theorem’s expression for causal influence.
A.1. Proof of Theorem 2.: Causal influence equals observed influence plus a correction term measuring divergence between partially observed and interventional distributions.The correction is a weighted sum of conditional Kullback–Leibler divergences and is therefore nonnegative.
A.1. Proof of Theorem 2.: The correction term is nonnegative because it is expressed as a weighted sum of conditional Kullback–Leibler divergences.

A.2. Decomposition into conditional relative entropies.

For a set of affected arrows, causal influence decomposes into conditional relative entropies associated with target nodes and their parent distributions.

A.2. Decomposition into conditional relative entropies.: The relative entropy defining causal strength decomposes into a sum of conditional relative entropies for target nodes given their parents.
A.2. Decomposition into conditional relative entropies.: Lemma 6 rewrites the causal influence of an arrow set as a sum of expectations.The target-node set is denoted trg(S).
A.2. Decomposition into conditional relative entropies.: The decomposition result is used in the proof of Theorem 5.
A.2. Decomposition into conditional relative entropies.: The proof uses that intervention-modified conditionals equal the original conditionals for nodes outside trg(S).A reversed relative-entropy statement would not generally hold because its weighting factor would be intervention-sensitive.

A.3. Proof of Theorem 5.

The proof establishes locality and additivity properties of causal-strength summands, then verifies a three-node special case showing when intervention leaves the distribution unchanged.

A.3. Proof of Theorem 5.: Each causal-strength summand CSi(P) depends only on P(Xi|PAi) and P(PAi).
A.3. Proof of Theorem 5.: Part (c) is proved by forming a Pythagorean triple from restrictions of P, PS1, and PS2 to Z and PAZ.
A.3. Proof of Theorem 5.: The locality argument is restricted to terms of D(P∥PS), because the reversed divergence uses intervention-sensitive parent weights.
A.3. Proof of Theorem 5.: For the three-node case, the proof considers S = {X →Z} and T = {X →Z,Y →Z}, showing D(P∥PT) = 0 under the stated condition.

A.4. Causal influence measures controllability.

The paper relates causal influence to controllability: it reflects both the range of target behaviors induced by interventions and the residual variability that control cannot remove.

A.4. Causal influence measures controllability.: Perfect control requires Y to be a deterministic function of X and manipulating X to produce all states of Y.
A.4. Causal influence measures controllability.: Theorem 7 characterizes perfect controllability of Y by X given Z = z through a Markov transition matrix R(x|z).The conditional entropy H(Y|z,do(x)) concerns Y after observing Z = z and setting X = x.
A.4. Causal influence measures controllability.: The control interpretation suggests measuring both non-eliminable behavioral variability and the repertoire induced by manipulating a source.
A.4. Causal influence measures controllability.: The entropy H(Y|Z) measures the size of Y’s output repertoire averaged over manipulations of X.This corresponds to the requirement that all potential outputs can be induced.
A.4. Causal influence measures controllability.: The entropy H(Y|X,Z) measures variability in Y that cannot be eliminated by controlling X.Causal influence increases as H(Y|Z,do(X)) tends toward zero, provided the first term remains constant.

SUPPLEMENTARY MATERIAL

The supplementary material contains three sections covering i.i.d. copy generation, an alternative definition of causal strength, and total influence.

The supplement addresses generating an i.i.d. copy via random permutations, an alternative causal-strength definition, and the problem of defining total influence.