PaperlayerView exact revision on arXiv

Source-linked AI summary

The Illusion of $\textit{What If}$: Evaluating the Breakdown of Counterfactual Reasoning in LLMs

Yucheng Wang, Yuetian Du, Zhengyi Liu, Rongyu Zhang, Bing Zhao, Boyu Yang, Ming Kong, Lin Qu, Hu Wei, Jie Liu, Qiang Zhu

arXiv:2608.27953v1cs.AI

TL;DR

What-if reasoning requires models to trace how changed conditions propagate through interacting systems, while existing benchmarks offer limited coverage of open-domain causal-process evaluation. WhatIfBench and PRISM evaluate these explanations, and experiments find persistent causal weaknesses across frontier models, with GPT-5.5 reaching a 64.62 final score.

  • Problem

    What-if reasoning requires models to depart from observed timelines and trace how changed factors propagate through interacting systems.

    Source: §1 · p. 1

  • Method

    WhatIfBench provides 220 open-form questions, while PRISM turns responses into semantic causal graphs and assesses graph-level validity and answer-level explanatory adequacy.

    Source: §1 · p. 2

  • Results

    Models often produce fluent narratives while exhibiting causal gaps, premise drift, and topology fragmentation; GPT-5.5 achieves a final score of 64.62.

    Source: §5.2 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    The findings support evaluating counterfactual reasoning as a causal explanation process, not merely as outcome prediction.

    Source: §6 · p. 9

    WhatIfBench’s 220 questions cannot fully represent counterfactual settings, specialized domains, or culturally grounded assumptions, and its English-centric public sources may introduce topical and stylistic biases.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Counterfactual reasoning requires models to reason beyond the observed world and explain how altered conditions propagate through downstream consequences. Existing benchmarks largely target bounded settings with fixed variables or single gold outcomes, overlooking open-domain scenarios requiring causal-process evaluation. To this end, we present $\textbf{WhatIfBench}$, a diagnostic benchmark for open-domain, open-form, long-horizon counterfactual causal reasoning, containing 220 what-if questions across STEM, HSS, and Hybrid scenarios. To evaluate free-form responses, we further propose $\textbf{PRISM}$, which first converts each natural-language explanation into a Response-Derived Semantic Causal Graph of events, states, and mechanisms. On top of this graph, PRISM then jointly applies a Process Metric assessing graph-level causal validity and a Rubric Metric assessing answer-level explanatory adequacy. Evaluating six frontier LLMs with this framework, we find that WhatIfBench remains far from saturated: even the strongest model reaches only a 64.62% final score. Further analysis reveals persistent causal gaps, premise drift, and topology fragmentation, suggesting that fluent counterfactual narratives often mask fragile causal processes. The benchmark, code, and evaluation scripts are available at $\href{https://github.com/zju-gt/WhatIfBench}{WhatIfBench}$.

1 Introduction

The introduction argues that open-domain, long-horizon what-if reasoning should be evaluated through causal processes rather than single outcomes, and presents WhatIfBench and PRISM for that purpose.

  • What-if questions require models to depart from observed timelines and propagate changed conditions through interacting systems, testing whether explanations reflect coherent causal processes rather than plausible narratives.Source: §1 · pp. 1–2
  • Existing benchmarks test bounded counterfactual conditionals, QA, or specified interventions, while real-world what-ifs involve implicit mechanisms, long horizons, and multiple plausible pathways (Frohberg and Binder, 2022; Yu et al., 2023; Jin et al., 2023; Chen et al., 2025).Source: Figure 1 · p. 1, §1 · p. 2
  • Because valid answers need not share one outcome, evaluation should judge premise fidelity and coherent causal support, including mechanisms, domain consistency, and uncertainty handling.Source: §1 · p. 2
  • WhatIfBench contains 220 open-form questions across STEM, HSS, and Hybrid scenarios, eliciting causal explanations for open-domain, long-horizon counterfactuals.STEM questions concern natural and technical constraints; HSS questions concern historical, institutional, and social dynamics; Hybrid questions span domains.Source: §1 · p. 2
  • PRISM converts responses into Response-Derived Semantic Causal Graphs of events, states, and mechanisms, then combines graph-level Process Metric validity with Rubric Metric explanatory adequacy.Process Metric assesses edge validity, causal direction, mechanism support, and topology continuity; Rubric Metric assesses premise fidelity, mechanism coverage, domain constraints, and uncertainty handling.Source: §1 · p. 2

2 Related Work

Prior work tests causal and counterfactual reasoning, but existing benchmarks and answer-centric metrics leave open-form, long-horizon causal processes underexamined.

Causal reasoning in LLMs

Benchmarks for counterfactual reasoning

Evaluation of counterfactual reasoning

  • Answer-centric metrics such as Exact Match, token-level F1, human-validated answers, and final-outcome checks are insufficient when open-domain counterfactuals lack a unique gold outcome.Responses may sound plausible while drifting from premises, skipping mechanisms, reversing causal direction, or adding unsupported links, motivating evaluation of both causal processes and explanatory adequacy.Source: §2 · p. 3

3 WhatIfBench

WhatIfBench evaluates open-domain, open-form, long-horizon counterfactual reasoning through free-form explanations that remain faithful to an explicit premise and propagate consequences through intermediate mechanisms.

3.1 Task Definition

  • The task requires explanations to follow an explicit counterfactual premise, instantiate relevant mechanisms, and propagate downstream consequences across nontrivial causal structures.Its scope is open-domain, open-form, and long-horizon: broad causal spaces, natural-language explanations, and multi-step propagation.Source: §3.1 · p. 3

3.2 Benchmark Overview

  • WhatIfBench contains 220 open-form questions: 113 STEM, 79 HSS, and 28 Hybrid scenarios, each paired with an explicit premise and question-level rubric.When background materials or reference discussions are available, they inform plausible constraints and explanatory requirements without imposing a unique gold conclusion.Source: §3.2 · p. 3

3.3 Taxonomy of Counterfactual Scenarios

  • The taxonomy distinguishes STEM constraint propagation, HSS path-dependent social and institutional dynamics, and Hybrid cross-domain propagation.These categories support overall comparison and diagnosis of scientific-constraint, socio-historical, and cross-domain propagation failures.Source: §3.3 · pp. 3–4

3.4 Data Collection and Annotation

  • Questions are collected from public what-if sources, normalized, and retained only when they have an explicit premise, sufficient context, and nontrivial causal propagation requirements.For each retained question, annotators assign a domain, summarize plausible reasoning, and create a query-dependent rubric.Source: §3.4 · p. 4

3.5 Quality Control

  • 36.0% of 611 candidate questions were retained after review or revision, yielding 220 final questions; all rubric sets passed review or revision.Quality control checks premise clarity, contextual grounding, causal propagation requirements, evaluable explanations, and rubric coverage of plausible trajectories.Source: §3.5 · p. 4, Table 1 · p. 5

4 Evaluation Framework: PRISM

PRISM evaluates open-form counterfactual explanations with a Process Metric for causal-edge validity and a Rubric Metric for answer-level explanatory adequacy. It converts responses into inspectable semantic causal graphs and reports both scores separately as well as an equally weighted diagnostic aggregate.

4.1 Response-Derived Semantic Causal Graph

  • Each response is segmented into claims and semantic relations in a response-derived graph, an inspectable representation rather than a gold structural causal model.The graph allows free-text responses while exposing their expressed causal structure.Source: §4.1 · p. 5
  • An RST-inspired parser labels discourse and causal relations; PM evaluates only causal-relevant edges, retaining other discourse edges for inspection.Causal-relevant labels include CAUSE, CONDITION, RESULT, and CONSEQUENCE; discourse labels include ELABORATION, CONTRAST, TEMPORAL, and BACKGROUND.Source: §4.1 · p. 5

4.2 Process Metric

  • PM assigns binary support judgments to causal-relevant transitions and averages them, checking textual support, relation type, direction, and mechanism-level plausibility.An edge must be locally recoverable from the response and express an actual dependency, not merely sequence or association; PM is not a full judgment of real-world causal correctness.Source: §4.2 · p. 5

4.3 Rubric Metric

  • RM scores responses against frozen, question-specific criteria for premise fidelity, mechanisms, consequences, constraints, uncertainty, and explanatory completeness.It rewards only explicit or unambiguously paraphrased evidence, penalizing premise drift, omissions, unsupported leaps, and overconfident speculation without prescribing a single gold trajectory.Source: §4.3 · pp. 5–6

4.4 Evaluator Standardization

  • GPT-5.4 serves as the unified parser and judge, with fixed prompts and identical evaluator settings across models; PM uses binary judgments and RM uses criterion-specific score bands.Complete schemas, prompts, rubric templates, and decoding settings are provided in Appendices C and D.Source: §4.4 · p. 6

4.5 Final Score and Reporting

  • The final score equally weights PM and RM as a diagnostic aggregate, while both metrics are reported separately and averaged across examples for category-level reporting.The framework reports the same metrics for STEM, HSS, and Hybrid subsets.Source: §4.5 · p. 6

5 Experiments

On WhatIfBench, GPT-5.5 scores 64.62, yet weak responses often lack causal bridges; PRISM’s response-derived graphs show high parsing validity and strong, though sub-human, alignment with human judgments.

5.1 Experimental Setup

  • Models answer the fixed test set with temperature τ = 0.6 and a 4,096-token budget, then submit responses verbatim to PRISM with evaluator temperature τ = 0.0.The protocol applies no model-specific prompt variants, answer normalization, or post-hoc correction, and reports STEM, HSS, Hybrid, and aggregate scores.Source: §5.1 · p. 6

5.3 Failure Mode Analysis

  • Causal Gap is the most frequent failure for every model, alongside Topology Fragmentation and Mechanism Missing that disrupt long-horizon causal propagation.Weak responses are those scoring below 60; five non-mutually-exclusive labels are Premise Drift, Causal Gap, Topology Fragmentation, Mechanism Missing, and Rubric Misalignment. Causal gaps often leave intermediate bridges from the counterfactual premise unsupported or missing.Source: Table 3 · p. 7, §5.3 · pp. 7–8

5.2 Main Results

  • GPT-5.5 leads WhatIfBench at 64.62 final score, followed by Claude-Opus-4.7 at 59.11; all evaluated models remain far from perfect.The results indicate that fluent generation does not guarantee reliable reasoning when counterfactual consequences must propagate across interacting mechanisms.Source: §5.2 · p. 7
  • Overall PM exceeds RM for every model; Qwen3-Max scores 58.07 PM versus 46.45 RM, while GPT-5.5 and Claude-Opus-4.7 rank first and second on both.The results show that locally plausible causal transitions need not produce complete, well-grounded explanations, and that PRISM evaluates both process validity and explanatory adequacy.Source: §5.2 · p. 7
  • Models perform better on STEM than HSS scenarios, while Hybrid scenarios expose greater model-specific variation and PM–RM trade-offs.The reported differences reflect challenges in modeling path dependence, institutional dynamics, and multi-agent social consequences in HSS, and in maintaining coherent causal propagation across domains.Source: §5.2 · p. 7

5.4 Visualization

  • GPT-5.5 and DeepSeek-V4-Pro tend to produce larger causal graphs, while Gemini-3.1-Pro reaches comparable longest-path depth with fewer nodes.Graph size and causal depth capture different behaviors: broader explanations are not necessarily deeper, and explanatory richness should not be conflated with coherent propagation.Source: Figure 3 · p. 7, §5.4 · p. 8

5.5 Correlation with Human Judgments

  • Parsing exceeds 98.7% node validity and 98.3% graph validity; PRISM–Human PM Pearson correlation is 0.812, compared with 0.918 for Human–Human agreement.Causal Verification exceeds 93.4%, while Edge Validity ranges from 88.7% to 92.3%. For RM, PRISM–Human Pearson correlation is 0.875 and MAE is 0.034, near the Human–Human MAE of 0.029, though Spearman correlation indicates remaining rank-order differences.Source: Table 4 · p. 8, Table 5 · p. 8, §5.5 · p. 8

6 Conclusion

WhatIfBench and PRISM target open-domain, open-form, long-horizon counterfactual reasoning and evaluate explanations as causal processes. Across six frontier LLMs, fluent narratives still exhibit causal gaps, premise drift, and topology fragmentation.

6 Conclusion

  • WhatIfBench benchmarks open-domain, open-form, long-horizon counterfactual causal reasoning.The paper presents it as a benchmark for this reasoning setting.Source: §6 · p. 8
  • PRISM evaluates counterfactual explanations by converting responses into response-derived semantic causal graphs and combining process- and rubric-level assessment.This enables diagnosis beyond final-answer matching.Source: §6 · pp. 8–9
  • Across six frontier LLMs, WhatIfBench remains far from saturated, with fluent narratives often showing causal gaps, premise drift, and topology fragmentation.The findings support evaluating counterfactual reasoning as a causal explanation process rather than merely as outcome prediction.Source: §6 · p. 9

Limitations

WhatIfBench’s limited, English-centric coverage may not capture diverse counterfactual settings or avoid source biases. PRISM diagnoses response-derived causal graphs rather than establishing real-world causal truth, and its judgments may reflect evaluator and design errors.

Limitations

  • WhatIfBench’s 220 English-source questions span STEM, HSS, and Hybrid scenarios but cannot represent full counterfactual, domain, or cultural diversity and may inherit source biases.The authors plan to add domains and questions and broaden linguistic and cultural coverage.Source: Limitations · p. 9
  • PRISM assesses response-graph coherence, textual support, and structural consistency—not definitive real-world causal truth—and may be affected by evaluator bias, parsing errors, and rubric choices.The authors plan to extend evaluation beyond single-turn, text-only frontier LLMs to retrieval-augmented, interactive, multimodal, and human-in-the-loop settings.Source: Limitations · p. 9

Ethical Considerations · Appendix

The paper frames WhatIfBench as a beneficial but dual-use diagnostic, cautions against overinterpreting its scope and scores, and documents supplementary materials supporting transparency and reproducibility.

Ethical Considerations

  • Evaluating explicit what-if reasoning can reveal unsupported causal jumps, hidden assumptions, and brittle explanations, though the capability is dual-use.Counterfactual answers may be mistaken for factual predictions, policy advice, or authoritative alternate histories.Source: Ethical Considerations · p. 9
  • WhatIfBench is intended only as a diagnostic benchmark, not a decision-making system.Its public-source questions are filtered to exclude private or personally identifying information, unclear counterfactual premises, and cases that cannot be assessed with an explanatory rubric.Source: Ethical Considerations · p. 9
  • Open-domain sources may carry topical, cultural, and English-centric biases, so WhatIfBench is neither exhaustive nor culturally neutral.The benchmark should not be treated as comprehensive or unbiased across cultures.Source: Ethical Considerations · p. 9
  • PRISM scores assess causal coherence, textual support, and rubric-level explanatory adequacy, not ground-truth real-world causality.The scores rely on automatic parsing and judging of response-derived causal graphs.Source: Ethical Considerations · p. 9
  • High-stakes use should include human review, inspection of responses and causal graphs, source-coverage documentation, and safeguards against harmful speculation or unsupported policy conclusions.The authors release WhatIfBench and PRISM for research on counterfactual reasoning evaluation.Source: Ethical Considerations · p. 9

Appendix

  • Appendix A provides dataset statistics on scenario taxonomy, fine-grained category distributions, and lexical characteristics; Appendix B details collection, annotation, quality control, and review protocols.The collection materials cover candidate sources, filtering criteria, and rubric-construction guidelines.Source: Appendix · p. 12
  • Appendices C–E provide complete generation and evaluation prompts, decoding and evaluator settings, and the response-to-graph parsing procedure with pseudocode.The prompts cover model response generation, graph parsing, Process Metric evaluation, and Rubric Metric evaluation.Source: Appendix · p. 12
  • Appendix F reports supplementary experiments, while Appendix G presents a worked PRISM evaluation example with rubrics, a model response, its derived graph, and RM judgment.The experiments address benchmark positioning, model scale, response length, paraphrase robustness, prompting strategies, and evaluator-family bias.Source: Appendix · p. 12

A Dataset Details and Statistics … A.3 Lexical Overview

WhatIfBench covers 220 open-form questions across STEM, HSS, and Hybrid scenarios, with fine-grained categories and a qualitative lexical overview characterizing its topical scope.

A.1 Scenario Taxonomy

  • STEM questions trace relatively stable scientific or technical mechanisms, HSS questions involve path dependence and social feedback, and Hybrid questions propagate across domains.Hybrid scenarios connect changes in natural, technical, or material conditions to social, economic, or institutional systems.Source: §A.1 · p. 12
  • 220 questions comprise 51.4% STEM, 35.9% HSS, and 12.7% Hybrid scenarios.STEM is the largest share, followed by HSS and Hybrid.Source: Figure 4 · p. 12

A.2 Fine-grained Category Distribution

  • Fine-grained slices cover macrohistory, war and geopolitics, politics, society and culture, physical systems, engineering, computation and logistics, and life sciences.These categories are diagnostic slices for examining failures in mechanisms, biological constraints, institutions, geopolitics, and cross-system dependencies, not exhaustive taxonomies.Source: §A.2 · p. 12

A.3 Lexical Overview

  • The lexical overview highlights scientific, historical, geopolitical, and technological themes as a qualitative summary, not a formal diversity measure.The paper identifies planetary systems, empires, wars, societies, and alternative historical trajectories among frequent terms.Source: §A.3 · p. 12
  • 40.5% of HSS questions fall in State, Regime, and Macrohistory, the largest HSS subcategory.Source: Figure 5 · p. 13
  • 49.6% of STEM questions fall in Physical and Planetary Systems, the largest STEM subcategory.Source: Figure 6 · p. 13

B Data Collection, Annotation, and Quality Control … E Response-to-Graph Parsing Algorithm

WhatIfBench is built from naturally occurring open-form counterfactual questions, screened and paired with question-specific rubrics that support evaluation without requiring a single gold trajectory. PRISM standardizes response generation and evaluation, then parses each response into a semantic causal graph, using a heuristic fallback when needed.

B.1 Candidate Collection

  • Candidate questions came from public what-if QA sites, online forums, and explanatory materials, with wording normalized while retaining each question’s original counterfactual intent.Sources included xkcd What If, Worldbuilding Stack Exchange, AlternateHistory, and Quora.Source: §B.1 · p. 13

B.2 Question Filtering Criteria

  • Questions must state a counterfactual intervention, require nontrivial causal propagation, permit open-form trajectories, and provide enough context for rubric-based evaluation.Questions lacking a clear premise, relying mainly on private or identifying information, too underspecified for a rubric, or not assessable through explanatory criteria are rejected.Source: §B.2 · p. 13

B.3 Rubric Construction Guidelines

  • Each retained question receives a rubric covering premise fidelity, central mechanisms, downstream consequences, domain constraints, uncertainty, and explanatory completeness, without prescribing a unique conclusion.The rubric evaluates whether a response supports its trajectory with coherent causal reasoning across multiple plausible answers.Source: §B.3 · p. 13

B.4 Quality Control Protocol

  • 36.0% of 611 candidate questions were retained after direct pass or revision, yielding 220 final questions; all rubric sets passed after review or revision.Quality review checked query clarity, grounding, causal-propagation requirements, and evaluable explanatory structure, while ensuring rubrics did not force a single gold trajectory.Source: §B.4 · p. 14

C Decoding and Evaluator Settings

  • Responses use τ = 0.6 and a 4,096-token budget; deterministic evaluation uses τ = 0.0, with budgets of 16,384, 1,024, and 4,096 tokens for the parser, PM, and RM.The larger parser budget accommodates construction of a full Response-Derived Semantic Causal Graph.Source: C Decoding and Evaluator Settings · p. 14

D Prompt Templates

  • Prompt templates are provided for model response generation, graph parsing, and PRISM’s dual-metric evaluation.The templates are referenced as figures 8 to 11.Source: D Prompt Templates · p. 14

E Response-to-Graph Parsing Algorithm

  • The parser extracts elementary discourse units and labeled semantic relations, organizes them into a support–core structure, then projects resolved relations as directed graph edges after duplicate removal.If parsing fails or yields no valid discourse units, it falls back to sentence-level units connected by SEQUENCE edges.Source: E Response-to-Graph Parsing Algorithm · p. 15

F Additional Experiments

Additional experiments compare WhatIfBench with existing counterfactual benchmarks and test its behavior across model scales, response lengths, prompting strategies, and evaluator families. The results show differing model rankings across benchmarks, limited sensitivity to response length, negligible credit for paraphrase-only answers, and continued difficulty despite prompting.

Comparison with existing counterfactual benchmarks

  • Qwen3-Max ranks first on IfQA but fifth on WhatIfBench, while DeepSeek-V4-Pro ranks first on CounterBench but fourth on WhatIfBench.Closed-form IfQA and CounterBench show higher, more compressed performance, while WhatIfBench targets open-domain, open-form, long-horizon process evaluation with question-specific rubrics. WhatIfBench has stronger rank agreement with the Artificial Analysis Intelligence Index (ρ = 0.94) than IfQA (ρ = 0.09) or CounterBench (ρ = 0.37), suggesting the benchmarks capture complementary capabilities.Source: F Additional Experiments · p. 15

Evaluation on smaller models

  • Smaller models score 40.80–44.36 Final, below the frontier-model range of 51.99–64.62, showing WhatIfBench remains discriminative across scales and architectures.The evaluated smaller models span dense and MoE architectures.Source: F Additional Experiments · p. 15, Table 8 · p. 16

Effect of response length

  • 0.48 is the Final-score standard deviation across GPT-5.5’s Vanilla, Long Output, and Short Output settings, indicating limited sensitivity to moderate response-length changes.PRISM does not explicitly reward response length, though longer outputs may introduce more causal content.Source: F Additional Experiments · p. 15, Table 9 · p. 16

Paraphrase-only stress test

  • Paraphrase-only responses receive 2.05 PM and 6.04 Final, with RM near 10 corresponding to the minimum rubric score of 1 on the 1–10 scale.Surface reformulation alone therefore receives negligible PRISM credit.Source: F Additional Experiments · p. 15

Counterfactual-specific prompting

  • CausalCoT and CoIn improve GPT-5.5’s Final score by 2.63 and 1.35 points, respectively, but prompting does not eliminate the benchmark’s difficulty.The prompting strategies affect PM and RM differently, and WhatIfBench remains challenging for frontier models under the evaluated strategies.Source: F Additional Experiments · pp. 15–16

Robustness to evaluator family

  • The two evaluator families produce identical rankings (Spearman ρ = 1.00); their mean absolute Final-score difference is 0.63 points, with a maximum of 1.57.This supports robustness of the main comparative conclusions to evaluator family.Source: F Additional Experiments · p. 16

Related papers

Loading 2608.27953v1…