PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Causal Reasoning in LLMs: A Case Study in the Challenges of Procedural Generation

Justin Clarke, Isha Karn, Patrick Murray, KJ Hill, David Jensen

arXiv:2610.38100v1cs.AIcs.LG

TL;DR

Whether strong CLADDER scores indicate formal causal reasoning is difficult to determine when benchmark-specific patterns may be learned from exposure. The paper examines the generator and model performance, finding that superficial patterns offer a more likely explanation than formal reasoning alone.

  • Problem

    It is unclear what strong CLADDER performance establishes about LLM causal reasoning, amid concerns that models may memorize or overfit benchmark data.

    Source: §1 · p. 2

  • Method

    The paper examines CLADDER’s generated patterns and tests whether models can learn them and whether removing query information changes LLM performance.

    Source: §1 · p. 3

  • Results

    The authors find that CLADDER embeds learnable predictive patterns, and conclude that exposure and superficial pattern learning are more likely explanations for high scores than formal causal reasoning.

    Source: §1 · p. 3, §4 · p. 10

  • Takeaways & Limitations

    High CLADDER performance alone does not reliably imply formal causal reasoning, so conclusions from prior evaluations and future uses of the generator warrant care.

    Source: §4 · p. 10

    Procedural generation can avoid test-instance memorization while leaving models able to learn patterns intrinsic to a generator, which may be difficult to detect or fix.

    Source: §4 · p. 10

Abstract

from arXiv · show

Large language models (LLMs) appear to represent and reason about a large range of phenomena. In particular, recent studies have examined whether LLMs can accurately reason about causation. One of the most commonly used benchmarks for evaluating formal causal reasoning in LLMs is CLadder, a framework for procedural generation of benchmark data sets that was released in 2023. Experiments with CLadder appear to indicate that LLMs can perform formal causal reasoning in a large percentage of generated examples. In this paper, we examine what can be reliably inferred when LLMs perform well on test sets produced by CLadder. We identify several examples of superficial and highly predictive patterns that the CLadder generator embeds in data sets it generates. We find that even simple models such as random forests perform well on CLadder-generated data sets after training on a sample of generated data. We also find that the performance of many commercial LLMs on CLadder test sets is extremely brittle and that removing large sections of the text that are critical for formal causal reasoning has almost no effect on performance. Given these results, the most likely hypothesis for the strong performance of state-of-the-art LLMs on CLadder is that they are exploiting superficial patterns learned from exposure to training data from the generator. Any inference that LLMs are exhibiting more complex causal reasoning must rule out this hypothesis.

1 Introduction

The paper asks what strong CLADDER performance can reliably establish about LLM causal reasoning, given concerns that benchmarks may be memorized or exploited through superficial patterns. It reports evidence that CLADDER embeds predictive patterns that models can learn, while some information crucial to formal reasoning can be removed without substantially reducing performance.

1 Introduction

  • Procedural generation can produce fresh, automatically verifiable test sets for formalized reasoning tasks, but this paper examines whether CLADDER performance supports reliable inferences about reasoning.Source: §1 · p. 2
  • The paper asks whether high CLADDER scores establish formal causal reasoning or generalization, since researchers have raised concerns about benchmark memorization, especially for closed models.Source: §1 · p. 2
  • CLADDER embeds simple, predictive patterns that models can learn from generated data, so high benchmark performance does not reliably demonstrate formal causal reasoning.The authors argue that exposure to CLADDER data and superficial pattern learning are more likely explanations for strong performance.Source: §1 · pp. 2–3
  • CLADDER data has proliferated and is likely to appear in many modern LLM training corpora, making exposure to its patterns a plausible explanation for strong performance.Source: §1 · p. 3
  • Removing information known to be essential for formal causal reasoning does not substantially impair LLM performance on CLADDER.The paper also reports that omitting seemingly incidental tokens from specific queries can dramatically affect LLM answers.Source: §1 · p. 3
  • Models with high CLADDER accuracy perform no better than random on CounterBench, despite that benchmark’s similar overall design and causal-query phrasing.Source: §1 · p. 3

2 CLADDER: Background, Design, and Prior Results

CLADDER was introduced as a formal, procedurally generated benchmark for causal reasoning, while prior work has differed in how strongly it interprets model performance. The paper reviews its design, influence, and earlier analyses of possible surface-pattern effects.

2 CLADDER: Background, Design, and Prior Results

  • CLADDER applies formal causal theory to ten query types spanning Pearl’s three rungs, against earlier commonsense evaluations that could conflate causal knowledge with reasoning.Prior work also noted that fluent causal language need not reflect genuine causal reasoning.Source: §2 · p. 3

2.1 What is CLADDER?

  • CLADDER samples formal task structures, converts them into natural-language queries, and automatically generates binary answers; its released demonstrations include 41,748 queries.The design was presented as avoiding static-test-set exposure concerns.Source: §2.1 · p. 4

2.2 Impact of CLADDER

  • CLADDER has been widely used as evidence of causal reasoning, although its original authors were cautious and earlier analyses reached differing conclusions about surface-pattern susceptibility.The original paper cautioned that reliable causal reasoning might remain distant, while later authors made stronger claims; prior analyses included tests of linguistic substitutions and shortcut features.Source: §2.2 · p. 4

3 Experiments and Key Findings

The experiments show that CLADDER contains learnable shortcuts, and that high model accuracy can persist when information needed for formal causal reasoning is removed. Performance also varies sharply with incidental tokens and does not transfer from frontier LLMs to CounterBench.

3.1 Highly Predictive Patterns are Directly Embedded by the Generator

  • CLADDER’s templates create repeated phrasing and structural patterns that can shortcut causal reasoning; for ATE and ETT queries, comparing two probabilities with the query’s polarity nearly separates answers.Some queries include as many as seven probability values, although the predictive pattern uses only the first two.Source: §3.1 · p. 5, Figure 2 · p. 5

3.2 Simple models can learn the predictive patterns

  • Random forests achieved 87% cross-validated accuracy, roughly comparable to current language models, and fine-tuned models exceeded 80% accuracy with 2,000 examples and 94% with 16,000.These results show that superficial feature combinations can support strong CLADDER performance.Source: §3.2 · p. 6, Figure 3 · p. 6

3.3 Models may have been exposed to CLADDER data

  • Accuracy rose from slightly above random in early evaluations to over 90% for current reasoning models, but training-data exposure after CLADDER’s public release cannot be ruled out.The generator and example datasets have been copied and redistributed, complicating interpretation of post-release evaluations.Source: §3.3 · pp. 6–7, Figure 4 · p. 7

3.4 Removing essential information has little effect on performance

  • Removing graph or background information reduced accuracy by only 0.6–4.2 percentage points, while removing all supplied probabilities still left models 11–21 points above chance.The first information is needed to identify a causal query; the probability removals made its answer impossible to calculate.Source: §3.4 · pp. 7–8, Figure 5 · p. 8, Figure 6 · p. 8

3.5 Seemingly incidental information greatly affects performance

  • Across nearly identical marginal queries, removing incidental tokens could collapse accuracy or raise it toward 100%, with one token removal flipping a query from 100% to approximately 0%.For another query, baseline accuracy was 8.0%; several token removals raised performance to nearly 100%.Source: §3.5 · pp. 9–10, Figure 7 · p. 9

3.6 Model Performance on CLADDER Does Not Generalize

  • Frontier models scored 48.6%–52.2% on CounterBench, while a random forest trained on superficial query features reached 79.1%.The CLADDER-to-CounterBench gap for frontier models ranged from 29 to 44 percentage points.Source: §3.6 · p. 10

4 Discussion and Conclusion

The authors conclude that CLADDER performance more likely reflects exposure and superficial pattern learning than formal causal reasoning. Procedural generation can avoid test-instance memorization while still leaving generator-specific patterns learnable.

4 Discussion and Conclusion

  • High CLADDER performance does not reliably imply formal causal reasoning; exposure and superficial pattern learning are more likely explanations.The authors say this should shape conclusions from prior CLADDER evaluations and future use of the generator.Source: §4 · p. 10
  • Procedural generation may avoid test-instance memorization but still lets models learn patterns intrinsic to the generator.Many such patterns are difficult to detect by inspection and appear to arise from the generation process rather than readily fixable design flaws.Source: §4 · p. 10
  • The authors propose faithful real-world generators, withheld generators and data, or simpler models trained on public data as possible ways forward.The simpler-model baseline is intended to prevent some unwarranted conclusions about LLM reasoning capabilities.Source: §4 · p. 10

AI use statement

  • The authors report that they used LLMs for code assistance, while authors conducted experimental design, analysis, and interpretation and generated no paper text with AI.Source: AI use statement · p. 11

Ethics statement

  • The study uses public CLADDER and CounterBench datasets plus a newly generated CLADDER dataset, with no human subjects, personally identifiable information, or sensitive content.Source: Ethics statement · p. 11

Reproducibility statement

  • The codebase includes intervention scripts, simple-classifier code, and fine-tuning configurations, with generated parameters and fine-tuning hyperparameters documented in the appendix.Source: Reproducibility statement · p. 11

A.1 Critiques of CLADDER and Synthetic Benchmarks

Related work identifies complementary limitations in CLADDER and synthetic causal benchmarks, including narrow query diversity, benchmark leakage, and limited coverage of realistic causal judgments.

A.1 Critiques of CLADDER and Synthetic Benchmarks

  • Geng et al. (2026) and Chang (2026) criticize CLADDER’s template-heavy queries for low linguistic diversity and limited capture of naturalistic causal-judgment failures.They position their benchmarks as tests of more realistic reasoning scenarios.Source: §A.1 · p. 15
  • Maasch et al. (2025a) motivate CausalARC with leakage concerns, while Maasch et al. (2025b) find that only o1 avoids degradation with reasoning complexity.The latter result is described as consistent with this paper’s targeted-perturbation findings.Source: §A.1 · p. 15
  • Du et al. (2025) find CLADDER-style benchmarks omit realistic statistical pitfalls and that model conclusions shift with semantic labels; other work treats causal competence as multidimensional.Li and Russo (2026) study causal discovery and Yamaoka et al. (2026) coefficient elicitation in linear Gaussian structural causal models.Source: §A.1 · p. 15

A.2 Shortcut Learning and Benchmark Contamination

Prior work documents benchmark performance driven by spurious patterns and contamination, with analogous surface exploitation across reasoning domains.

A.2 Shortcut Learning and Benchmark Contamination

  • Geirhos et al. (2020) and McCoy et al. (2019) document high benchmark accuracy from spurious correlations rather than intended reasoning.Source: §A.2 · p. 15
  • Chang (2026) frames associational reasoning substituted for interventional queries as Rung Collapse, while de Leeuw et al. (2025) report surface exploitation in arithmetic.In novel symbol systems, LLMs default to learned semantic associations over abstract reasoning.Source: §A.2 · p. 15
  • Zhang et al. (2024) report inflated accuracy on public math benchmarks relative to held-out equivalents, and Chen et al. (2026a) find synthetic-SCM fine-tuning gains without testing whether they reflect shortcut learning.Source: §A.2 · p. 15

A.3 Generated dataset

The authors generated a CLADDER dataset from the original codebase for their experiments and compared its zero-shot difficulty with the original release.

A.3 Generated dataset

  • The authors generated their dataset with the original Jin et al. (2023) codebase using the final/common and final/det-common configurations.Source: §A.3 · p. 15
  • The generated dataset contains 9,866 unique question-answer pairs used across experiments, including as the fine-tuning experiment’s test set.Source: §A.3 · p. 15
  • Query-type counts vary because causal-graph templates differ in richness and can produce duplicates; the authors omitted surface-level interventions after finding no significant performance difference between dataset splits.Source: §A.3 · p. 15
  • GPT-5.5 (medium reasoning) had nearly identical zero-shot performance on the generated dataset and each CLADDER v1 split, with no significant differences after Bonferroni correction.Table 2 compares accuracy across the generated dataset and each v1 split; all p > 0.05.Source: §A.3 · p. 15, Table 2 · p. 16

A.5 Fine-tuning Details

The study fine-tunes four open-weight models on public CLADDER data using stratified train/validation splits and LoRA. It varies training-set size and specifies the LoRA configuration and training recipe.

A.5 Fine-tuning Details

  • The authors fine-tune OLMo-3-7B-Instruct, OLMo-3.1-32B-Instruct, Llama-3.1-8B-Instruct, and Qwen2.5-3B on public CLADDER data.The generated CLADDER dataset is also used as the test set.Source: §A.5 · p. 16
  • Training examples are stratified by query type into an 80/20 train/validation split, with training-set sizes from 50 to 16,000 examples.The varied sizes are N ∈50, 200, 500, 1k, 2k, 4k, 8k, and 16k.Source: §A.5 · p. 16
  • Fine-tuning uses LoRA with rank 16, scaling factor 32, dropout 0.05, five epochs, and AdamW with a cosine learning rate of 2 × 10−4.The effective batch size is 16; prompt tokens are masked so only the answer token contributes to training loss.Source: §A.5 · p. 16

A.6 Query-wise Accuracy After Performing Interventions

The appendix describes query-wise CLADDER accuracy and intervention analyses, then details three sources of answer determinism in the generator. It also reports CounterBench accuracy comparisons and an expanded token-ablation analysis.

A.6 Query-wise Accuracy After Performing Interventions

  • Table 3 reports generated-CLADDER accuracy by query type; figures show per-type accuracy under six interventions and performance versus public CLADDER fine-tuning sample size.Figures 10 and 11 show intervention results with 95% Wilson confidence intervals; Figures 8 and 9 vary the number of fine-tuning samples.Source: §A.6 · p. 16, Table 3 · p. 17, Figure 8 · p. 17

A.7 Sources of Determinism in CLADDER Data

  • Code analysis identifies three levels of answer determinism in CLADDER that could provide shortcuts without genuine causal reasoning.These sources are parametric, structural, and formula-level determinism.Source: §A.7 · p. 20

A.7.1 Parametric determinism

  • In the easy split, causal-mechanism signs are consistent across all 37 non-nonsense stories; the hard split reverses them, and story identity often predicts the answer regardless of probabilities.The direction can typically be inferred from a single mechanism edge in the prompt.Source: §A.7.1 · p. 20

A.7.2 Structural determinism

  • For collision graphs, Y has no parents and is independent of X, making the ATE zero for every parameter setting and ATE, ETT, and COR answers predetermined.The answer can be read from the graph structure.Source: §A.7.2 · p. 20

A.7.3 Formula-level determinism

  • COL answers depend on question polarity, not the graph, model parameters, or displayed correlation value.The displayed correlation is not used to determine the ground-truth answer.Source: §A.7.3 · p. 20

A.8 Random Forest Classifier Features

  • The random-forest feature set includes probability products, lexical indicators, query-opening words, comparison wording, query length, and probability counts.Other listed features include individual probability values and comparisons between them.Source: §A.8 · pp. 20–21

A.9 Model performance on CounterBench

  • On CounterBench, all commercial GPT models perform at or near chance regardless of generation, while fine-tuned open-weight models and the random forest outperform them.The caption identifies chance as 50%.Source: Table 4 · p. 21

A.10 Expanded Token Ablation Experiment

  • Figure 12 presents token-ablation analysis on GPT-5.2 for a random sample of 25 evaluated marginal queries, using the main figure’s visualization conventions.The caption says its color encoding, labels, and salience threshold match Figure 7.Source: Figure 12 · p. 22

Related papers

Loading 2610.38100v1…