PaperlayerView exact revision on arXiv

Source-linked AI summary

Can Large Language Models Infer Causation from Correlation?

Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona Diab, Bernhard Schölkopf

arXiv:2306.05836v3cs.CLcs.AIcs.LG

TL;DR

Existing NLP causal-inference datasets largely test empirical knowledge, leaving pure causal reasoning by LLMs insufficiently evaluated. This paper introduces CORR2CAUSE, a benchmark built from correlational statements and causal-discovery principles, and finds that current LLMs perform nearly randomly and fail to generalize robustly after finetuning.

  • Problem

    Existing NLP causal-inference datasets primarily rely on empirical or commonsense knowledge rather than testing LLMs’ pure causal reasoning abilities.

  • Method

    The paper formulates CORR2CAUSE and constructs a benchmark of more than 200K samples using causal-discovery principles, including the PC algorithm, to map correlational statements to causal relations.

  • Results

    All 17 evaluated LLMs perform poorly on CORR2CAUSE, close to the random baseline; finetuning improves performance but does not produce robust out-of-distribution causal inference.

  • Takeaways & Limitations

    CORR2CAUSE can benchmark pure causal inference in LLMs that have not seen the dataset and highlights the need to address out-of-distribution generalization.

  • Takeaways & Limitations

    The causal graphs are limited to two to six nodes and exclude hidden confounders.

Abstract

from arXiv · show

Causal inference is one of the hallmarks of human intelligence. While the field of CausalNLP has attracted much interest in the recent years, existing causal inference datasets in NLP primarily rely on discovering causality from empirical knowledge (e.g., commonsense knowledge). In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs). Specifically, we formulate a novel task Corr2Cause, which takes a set of correlational statements and determines the causal relationship between the variables. We curate a large-scale dataset of more than 200K samples, on which we evaluate seventeen existing LLMs. Through our experiments, we identify a key shortcoming of LLMs in terms of their causal inference skills, and show that these models achieve almost close to random performance on the task. This shortcoming is somewhat mitigated when we try to re-purpose LLMs for this skill via finetuning, but we find that these models still fail to generalize -- they can only perform causal inference in in-distribution settings when variable names and textual expressions used in the queries are similar to those in the training set, but fail in out-of-distribution settings generated by perturbing these queries. Corr2Cause is a challenging task for LLMs, and would be helpful in guiding future research on improving LLMs' pure reasoning skills and generalizability. Our data is at https://huggingface.co/datasets/causalnlp/corr2cause. Our code is at https://github.com/causalNLP/corr2cause.

1 INTRODUCTION

The paper distinguishes pure causal reasoning from causal knowledge acquired empirically and introduces CORR2CAUSE to test whether LLMs can infer causation from correlation. Evaluations show that existing LLMs perform poorly, while finetuning does not yield robust out-of-distribution causal inference.

  • Causal inference can be acquired through empirical knowledge or through pure reasoning using formal causal-inference procedures.
  • CORR2CAUSE tests whether LLMs can infer causal relationships from correlational statements rather than relying on causal knowledge memorized from training data.
  • The dataset contains over 200K samples generated using insights from formal causal discovery.
  • All 17 evaluated LLMs perform poorly on pure causal inference, achieving performance close to the random baseline.
  • Finetuning improves performance but does not robustly transfer causal inference skills to out-of-distribution settings.

2 PRELIMINARIES: CAUSAL INFERENCE

The paper introduces graphical and statistical concepts used to connect observational correlations with causal structure. Its dataset design follows causal-discovery principles, especially conditional independence, d-separation, and the PC algorithm.

  • A directed graphical causal model represents variables as nodes and direct causal links as directed edges in a directed acyclic graph.
  • Parent, ancestor, confounder, collider, and mediator describe specific causal relationships among variables in the graph.
  • D-separation determines conditional independence by checking whether a conditioning set blocks all paths between two node sets.
  • The Markov property states that each node is conditionally independent of its non-descendants given its parents, enabling factorization of the joint distribution.
  • Faithfulness permits inferring d-separation sets from observed independence relations, while Markov-equivalent graphs share a skeleton and V-structures.
  • Because multiple causal graphs can generate the same statistical distribution, correlation alone does not necessarily identify a unique causal graph.
  • The dataset design follows the PC algorithm, which removes edges using independence relations and orients edges through V-structures and consistency checks.

3 DATASET CONSTRUCTION

CORR2CAUSE is constructed by mapping formally generated causal graphs and their statistical implications into labeled natural-language inference examples. The pipeline uses DAGs, d-separation, Markov equivalence classes, causal hypotheses, and verbalization to produce the benchmark.

  • Task formulation: CORR2CAUSE maps a correlation statement and a causal-relation hypothesis to a binary validity label.The label is 1 when the proposed relation is valid and 0 when the inference is invalid.
  • Data generation: The data-generation pipeline selects N, creates unique DAGs, derives d-separation sets, groups graphs into MECs, and labels hypotheses shared across each MEC.A hypothesis receives validity=1 only when it holds for all causal graphs corresponding to the observed correlation set.
  • Graph construction: The source graphs are acyclic and use topological ordering to permit only edges Xi → Xj when i < j.This restricts the adjacency matrix to entries above the diagonal, yielding N(N −1)/2 possible directed edges before graph selection.
  • Graph construction: Isomorphism checks remove structurally duplicate DAGs, while d-separation sets are used to cluster graphs into Markov equivalence classes.Each MEC groups causal graphs with the same set of statistical correlations; the dataset reports graph and MEC statistics in Table 1.
  • Language realization: The benchmark tests six relations, including parent, child, ancestor, descendant, confounder, and collider relationships, using templated hypotheses.Correlation sets are verbalized as dependence or conditional-independence statements within a closed system with no hidden variables.
  • Dataset statistics: 207,972 samples comprise 205,734 training, 1,076 development, and 1,162 test examples, with 18.57% positive labels.Premises average 424.11 tokens and hypotheses 10.83 tokens; the splits are designed to cover graph sizes comprehensively.

4 EXPERIMENTS

Experiments show that existing LLMs perform poorly on pure causal inference, while finetuning improves in-distribution results but fails to produce robust generalization under perturbations.

  • 4.1 EXPERIMENTAL SETUP: The evaluation reports F1 as the main metric alongside precision, recall, and accuracy for overall model comparisons.Table 4 highlights the overall best F1 score and the best result within each model category.
  • 4.2 THE CORR2CAUSE SKILL IN EXISTING LLMS: 17 existing LLMs perform poorly on CORR2CAUSE, with the best result reaching only 33.38% F1 and some models performing below random guessing.BART MNLI achieved the highest performance, even exceeding GPT-4 in this evaluation.
  • 4.3 FINETUNED PERFORMANCE: Finetuning substantially improves performance on the original test set, with RoBERTa-Large MNLI reaching 94.74% F1.The finetuned BERT-based NLI models showed the strongest performance.
  • 4.5 ROBUSTNESS ANALYSIS: Robustness tests paraphrase hypotheses and reverse variable names while keeping the training data and saved models unchanged.These tests distinguish memorization of training patterns from mastery of the intended reasoning skill.
  • 4.5 ROBUSTNESS ANALYSIS: After perturbation, all finetuned models drop by up to 39.29 on paraphrases and 62.30 after variable refactorization.RoBERTa-Large MNLI retained 67.87 F1 under variable refactorization, but its largest losses involved Is-Ancestor and Is-Descendant.
  • 4.5 ROBUSTNESS ANALYSIS: The authors recommend pairing i.i.d. evaluation with adversarially perturbed test sets when assessing finetuned CORR2CAUSE models.They release perturbed test sets to support future generalizability evaluations.

5 RELATED WORK

Prior NLP causal-reasoning work largely uses empirical knowledge, while CORR2CAUSE targets correlation-to-causation inference as a distinct logical skill.

  • Existing Causal Reasoning Tasks: Existing causal-reasoning tasks in NLP commonly rely on empirical knowledge to infer causes, effects, motivations, goals, or story developments.These tasks cover commonsense and social or narrative reasoning rather than pure causal inference from correlations.
  • Existing Logical and Inference Tasks: Natural language inference primarily models semantic relationships such as entailment and paraphrase, alongside newer logical-inference extensions.The cited extensions include deductive logic and propaganda-technique reasoning.
  • Existing Logical and Inference Tasks: CORR2CAUSE is presented as the first dataset specifically testing correlation-to-causation inference in NLP.The paper characterizes this capability as unique among existing logical and inference datasets.

6 CONCLUSION

The work introduces CORR2CAUSE and a dataset of over 200K samples for inferring causation from correlation. Experiments show that off-the-shelf LLMs perform poorly, while finetuning does not resolve out-of-distribution generalization.

  • CORR2CAUSE tests whether LLMs can infer causation from correlation using a dataset of over 200K samples.
  • Off-the-shelf LLMs perform poorly on the pure causal inference task.
  • Finetuning can re-purpose LLMs for CORR2CAUSE, but out-of-distribution generalization remains a problem.
  • The dataset is recommended for benchmarking pure causal inference in LLMs that have not seen it.

LIMITATIONS AND FUTURE WORK

The authors identify limitations in graph size and confounder assumptions, while proposing broader causal-discovery settings and real-world false-belief applications as future directions.

  • The inference problem does not assume hidden confounders, motivating future datasets that test their existence.
  • Future work could explore other causal discovery algorithms and connect the benchmark to real-world false beliefs caused by confusing correlation with causation.

A IMPLEMENTATION DETAILS

The implementation uses fixed prompting, model-specific finetuning settings, and template-based verbalization, with GPT-4 generating natural-story examples from symbolic expressions.

  • Finetuning: GPT-based models use default OpenAI finetuning settings, while BERT-based models are trained with transformers on an NVIDIA Tesla A100 GPU.
  • Prompts: Autoregressive LLMs are prompted to answer whether a hypothesis can be deduced with “Yes” or “No.”
  • Natural stories: GPT-4 generates around 200 natural stories from symbolic causal-inference expressions for the case study.
  • Natural-story statistics: The natural-story statistics report test and development counts, premise, hypothesis, and explanation lengths, and positive-label percentages.
  • Verbalization: Hypotheses for the six causal relations are composed using verbalization templates and their paraphrases.

D CHANGE LOG FOR THE DATASET VERSION UPDATE

CORR2CAUSE v2.0 removes duplicate samples arising from symmetric relations and semantically equivalent verbalizations, reducing the dataset to half its original size while preserving other properties.

  • De-duplication methods: The de-duplication methods cover the six causal relation types and their verbalizations.
  • Duplication sources: Duplicate hypotheses arise from symmetric causal relations and semantically equivalent verbalization templates.
  • Resulting dataset: CORR2CAUSE v2.0 is exactly half the original dataset, with unchanged label distribution and other properties.
  • Split construction: Test and training sets are deduplicated, while development duplicates overlapping those sets are removed afterward.
  • Expected impact: The authors expect minimal or almost no change to experimental results, though retraining may update exact performance numbers.

E SPURIOUS CORRELATION ANALYSIS

The analysis identifies label-linked n-grams as spurious correlations arising from hypothesis framing and variable names. The robustness tests are designed to break these shortcuts through paraphrasing and variable refactorization.

  • The robustness design tests whether correct judgments depend on these shortcuts by perturbing wording and variable names.
  • PMI analysis compares entailment and non-entailment labels across n-grams of up to four words, highlighting terms with the largest PMI differences.
  • Spurious correlations arise from hypothesis framing, including phrases such as “a cause (for)” and “a direct (one).”
  • Other spurious correlations are tied to variable names, including patterns involving “for D (but)” and “for E (but).”

F FINE-GRAINED ERROR ANALYSIS

The fine-grained analysis examines relation-specific errors in non-finetuned models and performance under alternative prompting strategies. It shows substantial weaknesses in recognizing particular causal relations, with some relations receiving zero F1 because models always predict the negative class.

  • Non-finetuned models receive a separate fine-grained error analysis alongside the corresponding analysis for fine-tuned models.
  • GPT-3.5 cannot recognize ancestor relations, while GPT-4 fails to recognize direct causation involving parents and children.
  • RoBERTa MNLI recognizes collider relations relatively correctly compared with its other reported relation types.
  • A zero F1 score indicates that the model always predicts the negative class for that relation.
  • The prompting study evaluates system expertise instructions, positive and negative few-shot examples, and chain-of-thought prompting as performance-enhancement strategies.
Loading 2306.05836v3…