Source-linked AI summary
CLadder: Assessing Causal Reasoning in Language Models
Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng Lyu, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan, Bernhard Schölkopf
TL;DR
LLM research has largely tested commonsense causal knowledge rather than formal causal inference, leaving open whether models apply well-defined causal rules. The paper introduces CLADDER, which grounds natural-language questions in symbolic causal queries and oracle answers, and CAUSALCOT, a formal chain-of-thought strategy. The task is highly challenging, while CAUSALCOT reaches 70.40% accuracy, 8.37 points above vanilla GPT-4 on CLADDER.
Problem
Prior LLM evaluations mainly test commonsense causal knowledge, leaving formal causal reasoning underassessed despite its importance for causal inference.
Method
The paper builds CLADDER from causal graphs and queries across three causal rungs, derives symbolic answers with an oracle causal inference engine, verbalizes them, and develops CAUSALCOT.
Results
70.40% accuracy substantially improves vanilla GPT-4 by 8.37 points on CLADDER.
Takeaways & Limitations
CLADDER provides a principled benchmark for analyzing LLM causal reasoning, while CAUSALCOT offers a formal-inference-inspired way to improve performance on challenging causal questions.
Takeaways & Limitations
The benchmark covers only some commonly studied causal queries, so it is a first step rather than a complete natural-language mini-Turing test for causality.
Abstract
from arXiv · showhide
The ability to perform causal reasoning is widely considered a core feature of intelligence. In this work, we investigate whether large language models (LLMs) can coherently reason about causality. Much of the existing work in natural language processing (NLP) focuses on evaluating commonsense causal reasoning in LLMs, thus failing to assess whether a model can perform causal inference in accordance with a set of well-defined formal rules. To address this, we propose a new NLP task, causal inference in natural language, inspired by the "causal inference engine" postulated by Judea Pearl et al. We compose a large dataset, CLadder, with 10K samples: based on a collection of causal graphs and queries (associational, interventional, and counterfactual), we obtain symbolic questions and ground-truth answers, through an oracle causal inference engine. These are then translated into natural language. We evaluate multiple LLMs on our dataset, and we introduce and evaluate a bespoke chain-of-thought prompting strategy, CausalCoT. We show that our task is highly challenging for LLMs, and we conduct an in-depth analysis to gain deeper insights into the causal reasoning abilities of LLMs. Our data is open-sourced at https://huggingface.co/datasets/causalNLP/cladder, and our code can be found at https://github.com/causalNLP/cladder.
1 Introduction
The paper asks whether LLMs can perform formal causal reasoning rather than reproduce commonsense causal patterns. It introduces CLADDER and CAUSALCOT to evaluate and elicit such reasoning, finding the task challenging but showing substantial gains from the proposed prompting strategy.
- Motivation: Existing work mainly evaluates whether LLM outputs align with commonsense causal knowledge, leaving formal causal reasoning insufficiently tested.This creates a risk that models reproduce causal patterns from training text without applying causal inference rules.
- Dataset: CLADDER contains more than 10K natural-language causal questions spanning associational, interventional, and counterfactual queries across several causal graphs.Questions are grounded in symbolic queries and oracle-derived answers, then verbalized as stories.
- Method: CAUSALCOT prompts LLMs to extract and formalize the causal graph, query, and available data before performing causal inference.The strategy is inspired by the causal inference engine and uses multistep chain-of-thought reasoning.
- Results: 70.40% accuracy substantially improves vanilla GPT-4 by 8.37 points on CLADDER.The experiments evaluate eight LLMs and analyze fine-grained errors in formal causal reasoning.
2 Preliminaries on Causal Inference
Causal inference is organized as a ladder from statistical association to intervention and counterfactual reasoning. Identification uses causal graphs and available lower-rung data to determine when higher-rung queries can be answered.
- The Ladder of Causation: The Ladder of Causation defines three rungs: association, intervention, and counterfactual reasoning.The rungs form a taxonomy and hierarchy of causal inference tasks.
- Rung 1: Seeing: Rung 1 represents statistical associations using joint and conditional distributions.Bayesian networks can represent variables and their conditional dependencies through directed acyclic graphs.
- Rung 2: Doing: Rung 2 represents active interventions through the do-operator and causal Bayesian networks.It asks how an outcome changes when a variable is set to a specified value.
- Rung 3: Imagining: Rung 3 represents counterfactual alternatives and requires structural causal models.It concerns what would have happened under a different condition, possibly contrary to the factual state.
- Identification: Identification determines whether a causal query can be answered from a causal graph and available lower-rung measurements.Graphical structure can transform higher-rung queries into estimable expressions, but higher-layer inferences are generally impossible from lower-layer information alone.
- Causal Inference Engine: The causal inference engine takes a query, graph, and available data, then returns whether a solution exists and an estimable equivalent expression when possible.This algorithmic framework supports generating ground-truth answers without relying on commonsense judgments.
3 Composing the CLADDER Dataset
CLADDER constructs natural-language causal questions from formally specified, identifiable causal queries and oracle-derived answers, spanning diverse query types and graph-based stories. The dataset uses algorithmic generation to support formal correctness and evaluates language quality and human comprehensibility.
- Task Formulation: CLADDER maps natural-language questions to binary answers and explanations, enabling evaluation of both answer accuracy and reasoning steps.
- Query Types: The dataset covers all three causal rungs, including associational, interventional, and counterfactual query types such as ATE, ATT, NDE, and NIE.
- Overall Pipeline: The generation pipeline first computes formal ground-truth answers with a causal inference engine, then verbalizes the queries, causal models, and data through narrative templates.
- Formal Part: Identifiability is ensured by constructing causal queries, graphs, and available probability data so each query has a well-defined answer.
- Dataset Statistics: The dataset contains 10K questions and is roughly balanced across query types, graph structures, stories, and ground-truth answers, with lower representation for graph-incompatible NDE and NIE queries.
- Data Quality Check: Quality checks found 98.74% grammatical correctness, 96% readability on 50 questions, perplexity 21.17, and 82% accuracy by one expert evaluator on 50 questions.
4 Our CAUSALCOT Model
CAUSALCOT prompts language models to solve causal questions through a structured sequence modeled on a causal inference engine. It combines graph and query formalization with causal estimation and evaluation before producing a final binary answer.
- Prompting Strategy: CAUSALCOT decomposes causal reasoning into six stages inspired by the causal inference engine and chain-of-thought prompting.
- Preparation Steps: The first four stages identify the causal graph, determine the query type, formalize the query symbolically, and extract relevant data.
- Formal Solution: The final two stages deduce the estimand with causal inference techniques and evaluate it to answer the question.
- Output Procedure: The prompt autoregressively concatenates responses to all six steps before requesting a final Yes-or-No answer.
- Motivation: CAUSALCOT imposes a causal-inference framework as an inductive bias to combine language-model abilities with formal causal reasoning.
5 Testing LLMs with CLADDER
CLADDER evaluates LLMs on formal causal reasoning across diverse query types and commonsense alignments. The task is challenging, while CAUSALCOT improves performance but still exposes weaknesses in multi-step inference.
- Main Results: CLADDER evaluates models with overall accuracy, rung-specific accuracy, and performance across commonsensical, nonsensical, and anti-commonsensical questions.The experiments include multiple language models and compare their performance on the benchmark.
- Main Results: 70.40% accuracy makes CAUSALCOT the strongest approach, improving over vanilla GPT-4 by 8.37 points on CLADDER.Performance generally declines as causal questions move to higher, more difficult rungs.
- Data Contamination: 9.65 points of improvement on anti-commonsensical data shows CAUSALCOT improves performance across commonsense-alignment levels.Vanilla GPT-4 performs 1.8 points worse on anti-commonsensical than commonsensical data.
- Error Analysis by Subquestions: CAUSALCOT extracts causal graphs relatively well, but its identified graphs still differ from the ground truth by a graph edit distance of 1.69.Node and edge prediction receive high F1 scores, although extraction is not perfect.
- Error Analysis by Subquestions: Steps ②, ③, and ⑤ are more challenging because they require careful application of causal inference, revealing weaknesses in formal reasoning.The analysis evaluates the performance of individual CAUSALCOT steps and the full reasoning chain.
- Effect of In-Context Learning: Conditional probability and NIE benefit most from in-context learning, while marginal probability and ATT examples help across query types.The analysis measures whether examples of one query type improve answers to subsequent query types.
6 Related Work
Related work distinguishes causality as knowledge, language comprehension, and formal reasoning. CLADDER positions causal inference in natural language as a formal reasoning task rather than a commonsense knowledge evaluation.
- In-Context Learning: The Figure 5 heatmap summarizes how helpful each query type is for solving subsequent query types.It presents cross-query-type transfer from the in-context-learning analysis.
- Causality-Related Skills for NLP: Research on LLM skills evaluates capabilities across domains, while much of the causality literature focuses on commonsense causal knowledge.This knowledge-oriented work examines whether model outputs align with human commonsense relationships.
- Causality-Related Skills for NLP: Causality-as-language-comprehension research studies causal connectives, causal language usage, and causal relations in text.This line differs from formal causal inference because it emphasizes linguistic understanding.
- Formal Causal Reasoning: Unlike many prior studies, CLADDER defines explicit causal graphs and ground-truth relationships for evaluating formal causal reasoning.Explicit graphs make causal relationships quantitatively definable in a principled way.
- Formal Causal Reasoning: Formal causal reasoning work addresses causal inference and causal discovery as distinct branches of technical research.CLADDER formulates causal inference for NLP, while CORR2CAUSE addresses discovering causation from correlation.
7 Discussion of Limitations and Future Work
The paper frames CLADDER as an initial natural-language approximation to a broader causal reasoning test. It also identifies further limitations involving query coverage and the interface between LLMs and causal tools.
- Benchmark Scope: CLADDER covers only some commonly studied causal queries, leaving broader query families such as additional path-specific effects for future work.Expanding query coverage would move the benchmark closer to the ideal mini-Turing test for causality.
- Reliability: The benchmark contrasts with claims of high causal performance by suggesting that LLMs may still be far from reliable causal reasoning.The discussion reports CLADDER performance in the 60+% range and emphasizes the importance of reliable systems for policy-relevant decisions.
- Tool Augmentation: A future causal-inference plug-in may still face language-to-tool interface failures even if a suitable causal engine is available.The paper identifies problem formulation for external tools as a non-trivial research question.
8 Conclusion
The paper introduces formal causal reasoning as an LLM evaluation task through CLADDER and proposes CAUSALCOT to address it. Experiments show the benchmark is highly challenging and useful for analyzing and improving causal reasoning in natural language.
- Conclusion: CLADDER benchmarks formal causal reasoning across multiple causal-inference aspects, all rungs of the ladder of causation, and semi-realistic verbalizations.The benchmark is designed to evaluate causal reasoning in natural language.
- Conclusion: CAUSALCOT applies multistep chain-of-thought reasoning inspired by formal causal-inference principles to answer causal questions.The prompting strategy is proposed as an approach for addressing the benchmark task.
- Conclusion: The benchmark’s difficulty makes it a tool for understanding LLM reasoning abilities and developing better causal-reasoning models.The conclusion presents CLADDER as a principled evaluation resource.
A Supplementary for Dataset Generation
The dataset is built from causal graphs, query types, and natural-language templates grounded in symbolic causal inference. Its balanced variants and structured prompting support systematic generation and solution of formal causal questions.
- The dataset draws causal graphs, queries, and commonsensical stories from established causal-inference books and papers.
- The collection includes ten causal graphs with treatment-effect pairs, while omitting combinations whose answers are trivial or ill-defined.
- It covers marginal and conditional probabilities, explaining away, ATE, adjustment sets, collider bias, counterfactual probability, ATT, NDE, and NIE queries.
- The balanced benchmark contains 10,112 questions distributed across stories, graphs, query types, and commonsensicalness, with equal numbers of yes and no answers.
- Prompt templates instruct models to represent graph edges, select among ten query types, and answer the supplied dataset question.
- CAUSALCOT decomposes each question into graph extraction, query classification, symbolic formulation, data collection, estimand derivation, and estimand solving.
D.2 Main Results on v1.0
CLADDER is challenging for language models, while CAUSALCOT achieves the strongest reported performance and benefits especially from formalizing the query type. Its parsing steps contribute least in ablation.
- CAUSALCOT achieves the highest reported CLADDER v1.0 performance at 66.64% accuracy.
- CAUSALCOT improves reasoning across empirical-alignment levels, with substantial gains on anti-commonsensical and nonsensical data.
- Ablation results identify query-type classification and formalization as the most influential subquestions for model performance.
- Removing graph extraction and data-collection steps has the least impact on performance.
E.2 ROSCOE Evaluation
ROSCOE evaluates CAUSALCOT answers on multiple dimensions of step-by-step reasoning. The outputs are strong in faithfulness and alignment, but show weaknesses in redundancy, perplexity chains, and missing steps.
- ROSCOE evaluates answers from 2,000 randomly sampled questions for semantic consistency, logicality, informativeness, fluency, and factuality.
- Faithfulness to the question, reasoning alignment with ground truth, and absence of external hallucinations consistently fall within the top quantile.
- ROSCOE scores dip on redundancy, perplexity chain, and missing step metrics.
F Comparison with Existing Causality-Related Datasets
The comparison positions CLADDER as a dataset for testing formal causal reasoning rather than causality as knowledge or language comprehension. It distinguishes datasets by query rung and evaluated skills.
- The comparison covers datasets addressing causality-as-knowledge and causality-as-language-comprehension tasks.
- Table 9 compares whether datasets cover association, intervention, and counterfactual query rungs.
- The comparison also examines causal-inference methods, causal-query formalization, and causal-relation extraction skills.