PaperlayerView exact revision on arXiv

Source-linked AI summary

Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests

Victor Veitch, Alexander D'Amour, Steve Yadlowsky, Jacob Eisenstein

arXiv:2106.00545v3cs.LGcs.AIstat.ML

TL;DR

Perturbative stress tests ask whether predictions change under input alterations believed irrelevant, but their connection to prediction and domain shift and how to train without counterfactual examples are open questions. The paper formalizes these tests as counterfactual invariance, derives structure-dependent observable regularizers, and reports improved out-of-domain performance with some in-domain cost.

  • Problem

    The paper asks how perturbative stress-test behavior relates to predictive performance and domain shifts, and how to learn predictors that pass tests when counterfactual examples are unavailable.

    Source: §1 · p. 1

  • Method

    The authors use causal inference to formalize stress-test behavior as counterfactual invariance and derive observable independence signatures for structure-dependent regularization.

    Source: §1 · p. 2, §7 · p. 12

  • Results

    Structure-matched MMD regularization improves out-of-domain performance while somewhat worsening in-domain performance, and the paper derives domain-shift guarantees that depend on causal structure.

    Source: §1 · p. 2, §4 · p. 7, §5.2 · p. 10

  • Takeaways & Limitations

    The appropriate independence criterion and regularizer depend on the data’s causal structure, which the paper links to mitigating domain shift.

    Source: §1 · p. 2, §7 · p. 12

    Observational independence signatures are necessary but not sufficient for counterfactual invariance, making them an imperfect proxy.

    Source: §3 · pp. 5–6

Abstract

from arXiv · show

Informally, a 'spurious correlation' is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character; e.g., changing the gender of a sentence's subject changes a sentiment predictor's output. To check for spurious correlations, we can 'stress test' models by perturbing irrelevant parts of input data and seeing if model predictions change. In this paper, we study stress testing using the tools of causal inference. We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions. We connect counterfactual invariance to out-of-domain model performance, and provide practical schemes for learning (approximately) counterfactual invariant predictors (without access to counterfactual examples). It turns out that both the means and implications of counterfactual invariance depend fundamentally on the true underlying causal structure of the data -- in particular, whether the label causes the features or the features cause the label. Distinct causal structures require distinct regularization schemes to induce counterfactual invariance. Similarly, counterfactual invariance implies different domain shift guarantees depending on the underlying causal structure. This theory is supported by empirical results on text classification.

1 Introduction

Perturbative stress tests probe whether predictions change when input features believed irrelevant are altered, but their relation to predictive performance and domain shift—and how to pass them without generated perturbations—remains unclear. The paper uses causal inference to formalize these tests as counterfactual invariance and derive observable training criteria, while emphasizing that the right approach depends on the data’s causal structure.

1 Introduction

  • Stress tests perturb input features expected not to matter, such as replacing “Mexican” with “Indian” in “tasty Mexican food” when testing sentiment.Such tests are straightforward to understand and are increasingly popular.Source: §1 · p. 1
  • The paper asks how stress-test behavior relates to prediction and domain-shift performance, and how to train for it when generating perturbed examples is difficult.The authors note that automatically perturbing a document’s sentiment in a general fashion is difficult.Source: §1 · p. 1
  • The paper formalizes passing perturbative stress tests as counterfactual invariance and derives observable implications that can be used to regularize predictors without counterfactual examples.Counterfactual invariance requires predictions to remain unchanged across counterfactual inputs produced by intervening on Z.Source: §1 · p. 2
  • The means and implications of counterfactual invariance depend on the underlying causal structure, so methods cannot be transferred blindly between structures.The relevant distinction includes whether features cause the label or the label causes features.Source: §1 · p. 2

2 Causal Structure

The paper distinguishes causal-direction prediction, where features cause the label, from anti-causal prediction, where the label causes features. In both settings, observed input components are grouped by their causal relationships with Z and Y, while associations between Z and Y can arise through distinct mechanisms.

2 Causal Structure

  • The paper considers both feature-causes-label and label-causes-feature structures because causal direction changes the implications of counterfactual invariance.Figure 1 distinguishes causal arrows from non-causal associations and decomposes X into parts by their causal relationships with Z and Y.Source: §2 · p. 2, §2.1 · p. 2, §2.2 · p. 3

2.1 Prediction in the Causal Direction

  • In the causal-direction case, X is divided into parts unaffected by Z, parts that do not influence Y, and a remainder both influenced by Z and influencing Y.The decomposition is based on causal relationships rather than semantic interpretation of the latent parts.Source: §2.1 · p. 3

2.2 Prediction in the Anti-Causal Direction

  • In the anti-causal case, Y causes X, and conditioning on a component influenced by both Z and Y can associate Z with Y through collider behavior.A common cause or selection effects can also associate Z and Y in this setting.Source: §2.2 · p. 3

2.3 Non-Causal Associations

  • Associations between Y and Z can arise from confounding or selection, while dependence through components influenced by Z and affecting Y is distinct and may or may not be spurious for a given use case.The paper calls an association purely spurious when the part of X not influenced by Z suffices to estimate Y if the dashed-line association is removed by conditioning on Z.Source: §2.3 · p. 4

3 Observable Signatures of Counterfactually Invariant Predictors

Counterfactual invariance is unobservable directly, so the paper derives testable conditional-independence signatures and uses them to guide regularization. The appropriate signature depends on causal structure, and observational signatures remain an imperfect proxy for counterfactual invariance.

Observable signatures

  • Because counterfactual inputs are unobserved, the paper uses conditional-independence signatures measurable from ordinary data with Z or a proxy.The signatures are derived as observable implications of counterfactual invariance.Source: §3 · pp. 4–5
  • Under the anti-causal graph, invariance implies f(X) is independent of Z given Y; under the causal-direction graph, it implies marginal independence if Y and Z are not selected.For causal-direction data with a purely spurious association and no confounding, the theorem gives a conditional-independence signature instead.Source: §3 · p. 5

Fairness connection

  • In fairness settings, counterfactual invariance corresponds to counterfactual fairness and implies demographic parity or equalized odds depending on causal structure.The paper leaves the ethical question of whether this is a sound fairness strategy for future work.Source: §3 · p. 5

Causal regularization

  • Without counterfactual examples, the method regularizes models toward the appropriate independence signature, using marginal or conditional MMD penalties.The procedure selects marginal regularization for causal-direction data with confounding, and conditional regularization for anti-causal structure or selection.Source: §3 · p. 5
  • The regularizer must match the true causal structure because the marginal and conditional independence conditions are generally incompatible; a mismatch may fail to induce invariance or discard excess information.Source: §3 · p. 5

Gap to counterfactual invariance

  • Observable independence signatures are necessary but not sufficient for counterfactual invariance, a gap the paper identifies as a fundamental restriction of observational data.The signature is the closest proxy the authors say observational data can provide.Source: §3 · p. 6

4 Performance Out of Domain

The section studies how counterfactual invariance relates to prediction under domain shifts that preserve causal relationships while changing noncausal dependence. Under specified causal conditions, the training-domain invariant risk minimizer also minimizes target-domain risk, but it is not generally worst-case optimal.

4 Performance Out of Domain

  • The considered shifts preserve causal relationships and Y's marginal distribution while allowing noncausal Y-Z dependence to vary through confounding or selection.Causal compatibility also requires both distributions to obey the same causal graph.Source: §4 · p. 6
  • The qualifying cases are anti-causal data, no-confounding pure-spurious shifts, or no-selection pure-spurious shifts with additive causal effects.The confounded case requires the additive structure.Source: §4 · pp. 6–7
  • Under Theorem 4.2's stated conditions, the training-domain counterfactually invariant risk minimizer is also the target-domain counterfactually invariant risk minimizer.The result concerns causally compatible training and target distributions.Source: §4 · pp. 6–7
  • The training-domain invariant risk minimizer offers a practical route to target-domain prediction without target examples, using Section 3 regularization to approximate invariance.The paper reports that this approach works well in practice in Section 5.Source: §4 · p. 7
  • The invariant risk minimizer is not generally Q-minimax, but the two coincide under the theorem's conditions, a pure-spurious association, and overlap.In the pure-spurious case, the excluded X information carries no useful signal.Source: §4 · p. 7

5 Experiments

Experiments show that matching the regularizer to the data’s causal structure reduces stress-test sensitivity and improves out-of-domain prediction, with some in-domain accuracy costs. The results also show that the mismatched penalty may fail or worsen stress-test performance.

5.1 Robustness to Stress Tests

  • Conditional MMD regularization reduces predicted label flips to 1.4%, compared with 2.8% for the best marginal-MMD result on synthetic anti-causal data.The experiments measure stress-test changes in both predicted labels and predictive probabilities.Source: Figure 2 · p. 8, §5.1 · p. 8

5 Experiments

  • The experiments vary between marginal and conditional penalties and their strengths, using BERT fine-tuned for prediction with cross-entropy plus a regularization term.The study tests multiple text datasets with causal or anti-causal structure.Source: §5 · pp. 7–8

5.1 Robustness to Stress Tests

  • The synthetic anti-causal stress test changes each input to its counterfactual version by intervening on the spurious variable Z.The synthetic confound assigns Z information through common words while balancing the review-score label Y.Source: §5.1 · p. 8
  • For natural review stress tests, sentiment adjectives are swapped while basic sentence structure is preserved, creating a limited set of counterfactual pairs.The causal task predicts review helpfulness, while the anti-causal task predicts whether the product has a Clothing category tag.Source: §5.1 · pp. 8–9
  • On natural review data, reducing the MMD matched to the causal structure reduces sensitivity to sentiment perturbations, while marginal MMD makes anti-causal models more sensitive.The perturbations swap positive- and negative-sentiment adjectives.Source: §5.1 · p. 9, Figure 3 · p. 9

5.2 Domain Shift

  • Unregularized predictors rely on the spurious sentiment-label association, and in the causal case their accuracy falls below chance in the 0.8 domain.The test domains vary the strength of the spurious association from the training condition of 0.3.Source: §5.2 · pp. 9–10
  • Matching the regularizer to the causal structure improves out-of-domain review performance, though these models have somewhat worse in-domain performance.The review-data plots compare out-of-domain accuracy across test domains with different spurious-correlation strengths and show worst out-of-domain versus in-domain accuracy.Source: §5.2 · p. 10, Figure 4 · p. 10
  • Conditional MMD improves MNLI worst-group accuracy while only mildly reducing overall accuracy; marginal regularization does not improve over the unregularized baseline.Overregularization can yield identical predictions for every input and a worst-group accuracy of 0.Source: §5.2 · p. 11, Figure 5 · p. 11

6 Related work

Related work connects causal structure and domain shifts, direct counterfactual training, fairness, and domain adaptation to this paper’s focus. The paper distinguishes its example-wise invariance setting from those approaches and emphasizes that causal structure determines which criteria and guarantees apply.

Causal and domain-shift work

  • Unlike domain-shift approaches that aim to predict from the causes of Y, counterfactual invariance is example-wise, can be learned from one environment, and guarantees generalization only across causally compatible domains.The paper notes that this compatibility need not match previously studied shift classes; anti-causal invariance is one example.Source: §6 · p. 11

Causal representation learning

  • The paper’s approach splits features by their causal relations to Y and Z, making domain knowledge of the X–Y relation essential to choosing a suitable method.This differs from representation-learning work that seeks to identify causal features among covariates.Source: §6 · p. 11

Counterfactual examples

  • Prior work can enforce counterfactual invariance directly using generated or manipulated examples, but such examples can be difficult to obtain for language tasks.The paper contrasts this strategy with its regularization approach, which uses observable implications of counterfactual invariance.Source: §6 · pp. 11–12

Fairness and domain adaptation

  • When Z is a protected attribute, counterfactual invariance corresponds to counterfactual fairness, with its marginal and conditional signatures corresponding to demographic parity and equalized odds.The paper cautions that Z is not a domain label: all its values occur in the single training domain.Source: §6 · p. 12

Counterfactuals and out-of-domain performance

  • Some prior work links counterfactual methods to out-of-domain gains, but the paper notes that these studies use different causal assumptions or model settings.The cited examples include near-counterfactual training, generated image counterfactuals, and a linear-Gaussian analysis of causal and spurious features.Source: §6 · p. 12

7 Discussion

Causal inference links counterfactual desiderata to observationally testable conditional-independence criteria, but using that link depends on the data’s causal structure. The approach is limited by restrictive structures and assumptions, which can make it overly conservative and difficult to extend to richer settings.

7 Discussion

  • Counterfactual desiderata can be linked to observationally testable conditional-independence criteria, enabling a procedure to enforce them and mitigate domain shift.The link requires considering the data’s true causal structure.Source: §7 · p. 12
  • The paper’s main limitation is its restrictive causal structures, including an assumption that the part of X unaffected by Z is independent of Z in observed data.If those variables share a common cause, the procedure may discard more information than required; applying the ideas to richer settings such as question answering is not obvious.Source: §7 · p. 12

A Proofs

The appendix proves that counterfactual invariance has observable conditional-independence signatures under specified causal structures. It also establishes conditions for domain transfer of the invariant risk minimizer and shows that it is not generally minimax.

A Proofs

  • Under the anti-causal graph, counterfactual invariance implies f(X) ⊥ Z | Y.This is one of the conditional-independence signatures used in the proofs.Source: A Proofs · p. 16
  • Under the causal-direction graph, invariance implies f(X) ⊥ Z without selection, or f(X) ⊥ Z | Y for purely spurious association without confounding.The latter condition allows selection; the former allows confounding.Source: A Proofs · p. 16
  • If the target is causally compatible with training and Theorem 4.2’s stated causal conditions hold, the training-domain invariant risk minimizer is also the target-domain invariant risk minimizer.The listed cases include the anti-causal graph and specified causal-direction settings.Source: A Proofs · pp. 16–17
  • The counterfactually invariant risk minimizer is not generally target-minimax, but Theorem 4.4 equates the predictors under its stated conditions, including pure spurious association and overlap.In general, invariance can exclude information in X_Y∧Z even when it helps prediction across target settings.Source: A Proofs · p. 17

B.1 Model

The experiments use a fixed BERT base predictor and specify the training setup. The MMD regularizer operates on log class-probability estimates with a Gaussian RBF kernel.

B.1 Model

  • All experiments use an unmodified BERT base predictor with a linear map from its representation layer and cross-entropy training.Training uses vanilla SGD, batch size 1024, learning rate 1e−5 × 1024, validation-risk early stopping with patience 10, and two TPUs per model.Source: §B.1 · p. 18
  • The MMD regularizer uses a Gaussian RBF kernel with bandwidth 10.0 on log class-probability estimates.Regularization coefficients are log-spaced between 0 and 128.Source: §B.1 · p. 18

B.2 Data

The data section identifies the sources and preprocessing of the review and MNLI data, then describes procedures for inducing dependence between labels and Z in Amazon reviews.

B.2 Data

  • The MNLI data receive no preprocessing, while the Amazon review data come from.Source: §B.2 · p. 18

B.2.1 Inducing Dependence Between Y and Z in Amazon Product Reviews

  • For causal Amazon-review data, the procedure filters reviews and sets labels using helpful-vote thresholds by Z, then adjusts labels to reach the target association.The resulting split contains 58,393 training, 16,221 test, and 6,489 validation examples.Source: §B.2.1 · pp. 18–19
  • For anti-causal Amazon-review data, a random subset is chosen to achieve the target Y–Z association.The split contains 157,616 training, 43,783 test, and 17,513 validation examples.Source: §B.2.1 · p. 19

B.2.2 Synthetic Counterfactuals in Product Review Data

This synthetic setup uses Amazon clothing, shoes, and jewelry reviews to construct counterfactuals by changing token suffixes. It also induces a dependency between review labels and the suffix variable.

B.2.2 Synthetic Counterfactuals in Product Review Data

  • The setup selects Amazon product reviews, labels 4- and 5-star reviews positive, and uses only each review’s first 20 tokens.The dataset is the Amazon “clothing, shoes, and jewelery” dataset.Source: §B.2.2 · p. 19
  • A Bernoulli variable Z determines whether “and” and “the” receive the suffix “xxxxx” or “yyyyy,” and swapping suffixes produces counterfactuals.Source: §B.2.2 · p. 19
  • Resampling induces dependence between Y and Z with γ = 0.3 and P(Y = 1) = 1 2; the resulting data include 13,315 training and 3,699 test instances.Source: §B.2.2 · p. 19

Related papers

Loading 2106.00545v3…