PaperlayerView exact revision on arXiv

Source-linked AI summary

The Illusion of $\textit{What If}$: Evaluating the Breakdown of Counterfactual Reasoning in LLMs

Yucheng Wang, Yuetian Du, Zhengyi Liu, Rongyu Zhang, Bing Zhao, Boyu Yang, Ming Kong, Lin Qu, Hu Wei, Jie Liu, Qiang Zhu

arXiv:2608.27953v1cs.AI

TL;DR

Existing benchmarks overlook open-domain, long-horizon counterfactual explanations and causal-process evaluation. The paper introduces WhatIfBench and PRISM, finding that six frontier LLMs remain far from saturation, with persistent causal gaps, premise drift, and topology fragmentation.

  • Problem

    Existing counterfactual benchmarks largely target bounded settings with fixed variables or single gold outcomes, overlooking open-domain scenarios requiring causal-process evaluation.

  • Method

    The paper introduces WhatIfBench, a 220-question open-domain benchmark, and PRISM, which converts explanations into semantic causal graphs for joint process- and rubric-level assessment.

  • Results

    WhatIfBench remains far from saturated across six frontier LLMs; GPT-5.5 achieves a final score of 64.62, while analyses reveal causal gaps, premise drift, and topology fragmentation.

  • Takeaways & Limitations

    Counterfactual reasoning should be evaluated as a causal explanation process rather than merely as outcome prediction.

  • Takeaways & Limitations

    PRISM evaluates coherence, textual support, and structural consistency within responses rather than definitive real-world causal truth, and may be affected by evaluator bias, parsing errors, and rubric choices.

Abstract

from arXiv · show

Counterfactual reasoning requires models to reason beyond the observed world and explain how altered conditions propagate through downstream consequences. Existing benchmarks largely target bounded settings with fixed variables or single gold outcomes, overlooking open-domain scenarios requiring causal-process evaluation. To this end, we present $\textbf{WhatIfBench}$, a diagnostic benchmark for open-domain, open-form, long-horizon counterfactual causal reasoning, containing 220 what-if questions across STEM, HSS, and Hybrid scenarios. To evaluate free-form responses, we further propose $\textbf{PRISM}$, which first converts each natural-language explanation into a Response-Derived Semantic Causal Graph of events, states, and mechanisms. On top of this graph, PRISM then jointly applies a Process Metric assessing graph-level causal validity and a Rubric Metric assessing answer-level explanatory adequacy. Evaluating six frontier LLMs with this framework, we find that WhatIfBench remains far from saturated: even the strongest model reaches only a 64.62% final score. Further analysis reveals persistent causal gaps, premise drift, and topology fragmentation, suggesting that fluent counterfactual narratives often mask fragile causal processes. The benchmark, code, and evaluation scripts are available at $\href{https://github.com/zju-gt/WhatIfBench}{WhatIfBench}$.

1 Introduction

WhatIfBench targets open-domain, open-form, long-horizon counterfactual reasoning, where models must preserve premises and propagate consequences through coherent causal processes. PRISM evaluates these explanations through response-derived causal graphs and complementary process- and rubric-level assessments.

  • Counterfactual questions require models to depart from observed timelines and propagate altered conditions through interacting systems.
  • Existing benchmarks mainly test bounded counterfactual reasoning with local questions, specified variables, causal structures, or target outcomes.
  • Open-domain what-if questions may admit multiple valid outcomes, so evaluation must prioritize premise fidelity and coherent causal structure over a unique conclusion.
  • WhatIfBench contains 220 open-form what-if questions spanning STEM, HSS, and Hybrid scenarios.
  • PRISM converts natural-language responses into response-derived semantic causal graphs and jointly assesses graph-level causal validity and answer-level explanatory adequacy.

2 Related Work

Prior work evaluates causal and counterfactual reasoning largely through bounded tasks or answer-centric matching. This paper focuses on open-domain, long-horizon explanations, where plausible responses can still contain causal-process failures.

  • Causal reasoning research asks whether LLMs produce robust explanations of how events depend on one another rather than surface associations.
  • Counterfactual reasoning requires models to depart from factual assumptions, apply hypothetical interventions, and propagate their consequences.
  • Open-form counterfactual questions remain underexplored when they are underspecified, span interacting systems, and admit multiple plausible trajectories.
  • Answer-centric evaluation can miss responses that drift from premises, skip mechanisms, reverse causal direction, or introduce unsupported causal links.

3 WhatIfBench

WhatIfBench defines open-domain, open-form, long-horizon counterfactual reasoning as free-form explanation under explicit premises, evaluated across three scenario categories. Its 220 questions are normalized, rubric-annotated, and quality controlled.

  • 3.1 Task Definition: The task requires explanations to remain faithful to an explicit counterfactual premise, instantiate intermediate mechanisms, and propagate downstream consequences.
  • 3.1 Task Definition: Open-domain, open-form, and long-horizon reasoning respectively span broad causal spaces, natural-language explanations, and multi-step causal propagation.
  • 3.2 Benchmark Composition: WhatIfBench contains 220 questions across 113 STEM, 79 HSS, and 28 Hybrid scenarios, each with a normalized query, premise, category, and question-level rubric.
  • 3.6 Evaluation Framework: PRISM evaluates each response with a Process Metric for causal-edge validity and a Rubric Metric for answer-level explanatory adequacy.
  • 3.3 Taxonomy of Counterfactual Scenarios: STEM scenarios emphasize stable scientific and technical constraints, HSS scenarios emphasize socio-historical dynamics, and Hybrid scenarios test cross-domain propagation.
  • 3.3 Taxonomy of Counterfactual Scenarios: The taxonomy supports diagnosis of failures involving scientific constraints, socio-historical reasoning, and cross-domain propagation.
  • 3.4 Benchmark Construction: Questions are collected from public sources, normalized, filtered for explicit premises and nontrivial propagation, then annotated with plausible reasoning spaces and query-dependent rubrics.
  • 3.5 Quality Control: Quality control retained 36.0% of 611 candidate questions after direct pass or revision, yielding 220 final questions.

4 Evaluation Framework: PRISM

PRISM evaluates open-form counterfactual explanations through two complementary views: causal-process validity in a response-derived graph and answer-level adequacy against question-specific requirements.

  • Framework overview: PRISM converts each natural-language response into a response-derived semantic causal graph and scores it with Process Metric and Rubric Metric.The graph extracts discourse units and semantic relations, while the two metrics separately assess causal-edge validity and explanatory adequacy.
  • Response-derived graph: The response-derived graph is an evaluation-oriented intermediate representation, not a gold structural causal model of the real world.It makes causal structure expressed in free-text responses inspectable without requiring a unique real-world causal model.
  • Process Metric: Process Metric evaluates causal-relevant edges for textual support, relation type, direction, and mechanism-level plausibility.An edge must connect stated or unambiguously paraphrased endpoint claims and express an actual causal, conditional, resultative, or consequential dependency.
  • Rubric Metric: Rubric Metric measures whether answers cover question-specific mechanisms, constraints, downstream consequences, and other explanatory requirements.Its criteria include premise fidelity, mechanism coverage, downstream consequences, domain constraints, uncertainty handling, and explanatory completeness.
  • Reporting: PRISM reports PM and RM separately and uses equal weighting as a default diagnostic aggregation across STEM, HSS, and Hybrid subsets.This preserves trade-offs between process-level causal validity and answer-level explanatory adequacy while supporting category-level diagnosis.

5 Experiments

Experiments evaluate six frontier LLMs on WhatIfBench with standardized PRISM scoring and analyze graph structure, failure modes, parsing quality, and human alignment. Results show persistent difficulty, structurally fragmented reasoning, and meaningful but imperfect agreement with human judgments.

  • Experimental Setup: Six frontier LLMs were evaluated on the fixed WhatIfBench test set using a standardized response-generation and PRISM evaluation protocol.Responses used temperature τ = 0.6, a 4,096-token budget, and were submitted verbatim to the PRISM pipeline.
  • Main Results: 64.62 was GPT-5.5’s final score, while Claude-Opus-4.7 scored 59.11, showing that WhatIfBench remains challenging for frontier models.The results indicate that fluent generation does not necessarily imply reliable counterfactual causal reasoning across interacting mechanisms.
  • Failure Mode Analysis: Causal Gap was the most frequent failure mode for every model among weak responses, alongside Topology Fragmentation and Mechanism Missing errors.Weak responses often mentioned plausible consequences without establishing intermediate causal bridges from the counterfactual premise.
  • Visualization: Figure 3 compares response-derived graphs by node count, representing instantiated causal-factor scale, and longest directed path length, representing causal-propagation depth.GPT-5.5 and DeepSeek-V4-Pro tend to produce larger graphs, whereas Gemini-3.1-Pro achieves comparable path depth with fewer nodes.
  • Parsing Reliability: Node Validity exceeded 98.7%, Graph Validity exceeded 98.3%, and Causal Verification exceeded 93.4% in human audits, while Edge Validity ranged from 88.7% to 92.3%.These results support response-derived causal graphs as a reliable basis for process evaluation, with edge extraction the most challenging component.
  • Human Alignment: PRISM achieved 0.812 Pearson and 0.846 Spearman correlation for PM, and 0.875 Pearson correlation with 0.034 MAE for RM against human judgments.PRISM remained below the Human–Human reference while capturing most of the human evaluation signal.

6 Conclusion

WhatIfBench and PRISM evaluate counterfactual reasoning as a structured causal explanation process rather than only by final-answer matching. Across six frontier LLMs, the benchmark exposes fluent responses with persistent causal weaknesses.

  • WhatIfBench targets open-domain, open-form, long-horizon counterfactual causal reasoning, while PRISM evaluates the resulting explanations through process- and rubric-level assessment.Responses are converted into response-derived semantic causal graphs before scoring.
  • Six frontier LLM experiments show that WhatIfBench remains far from saturated.The evaluation reveals substantial room for improvement despite testing recent high-performing systems.
  • Models often produce fluent narratives containing causal gaps, premise drift, and topology fragmentation.These diagnostics identify weaknesses in the causal processes underlying otherwise plausible explanations.
  • The findings support evaluating counterfactual reasoning as a causal explanation process, not merely as outcome prediction.

Limitations

WhatIfBench enables systematic comparison across several scenario types but has limited topical, linguistic, and cultural coverage. PRISM also has evaluator-specific limitations beyond benchmark coverage.

  • WhatIfBench contains 220 questions across STEM, HSS, and Hybrid scenarios, limiting how fully it represents counterfactual settings and specialized domains.The authors plan to add more domains and questions.
  • Questions collected and normalized from public English-centric sources may inherit topical, stylistic, linguistic, and cultural biases.
  • PRISM is an automatic diagnostic evaluator rather than a goldstandard causal truth oracle.Its scores concern response-level causal coherence, textual support, and structural consistency rather than definitive real-world causal truth.
  • PRISM may be affected by evaluator bias, parsing errors, and rubric design choices.

Ethical Considerations

The paper frames counterfactual evaluation as beneficial for exposing unsupported reasoning while warning that outputs may be misused as authoritative predictions or advice. It therefore limits WhatIfBench to diagnostic research use and recommends safeguards for high-stakes applications.

  • Evaluating explicit what-if reasoning can reveal unsupported causal jumps, hidden assumptions, and brittle explanations.
  • Counterfactual answers about policy, history, institutions, disasters, or technology may be mistaken for factual predictions, policy advice, or authoritative alternate histories.
  • WhatIfBench is intended as a diagnostic benchmark, not a decision-making system.
  • The benchmark should not be treated as exhaustive or culturally neutral because open-domain sources may retain topical, cultural, and English-centric biases.
  • High-stakes use should include human review, inspection of responses and causal graphs, source-coverage documentation, and safeguards against harmful speculation.

A Dataset Details and Statistics

WhatIfBench organizes 220 open-form questions into STEM, HSS, and Hybrid scenarios, with fine-grained topical slices for diagnosing different causal reasoning challenges. STEM is the largest broad category, while State, Regime, and Macrohistory and Physical and Planetary Systems lead within HSS and STEM.

  • Scenario Types: WhatIfBench organizes open-form counterfactual questions into STEM, HSS, and Hybrid scenarios.STEM emphasizes relatively stable scientific and technical principles; HSS emphasizes path dependence, multi-agent behavior, and social feedback; Hybrid spans domains.
  • Scenario Types: STEM accounts for 51.4% of the 220-question benchmark, followed by HSS at 35.9% and Hybrid at 12.7%.
  • Fine-grained Categories: Fine-grained subcategories provide diagnostic slices rather than exhaustive domain taxonomies.
  • Lexical Overview: Figure 7 presents lexical distributions qualitatively, reflecting scientific, historical, geopolitical, and technological scenarios rather than formally measuring diversity.
  • Fine-grained Categories: HSS questions cover State, Regime, and Macrohistory; War and Geopolitics; Dynastic Leadership; Domestic Politics and Institutions; and Culture, Technology, and Society.
  • Fine-grained Categories: State, Regime, and Macrohistory is the largest HSS subcategory at 40.5%.
  • Fine-grained Categories: STEM questions cover Physical and Planetary Systems; Life Sciences and Human Body; Society, Computation, and Logistics; and Engineering, Transport, and Design.
  • Fine-grained Categories: Physical and Planetary Systems is the largest STEM subcategory at 49.6%.

B Data Collection, Annotation, and Quality Control

WhatIfBench is assembled from naturally occurring, open-form counterfactual questions and filtered for causal depth, contextual grounding, and evaluability. Quality control refines both questions and rubrics, while PRISM standardizes response-to-graph parsing and evaluation inputs.

  • Data Collection: WhatIfBench retains public what-if questions requiring open-form explanations rather than short factual answers.Sources include xkcd What If, Worldbuilding Stack Exchange, AlternateHistory, and Quora.
  • Data Collection: Questions must contain an explicit intervention, require nontrivial causal propagation, admit multiple plausible trajectories, and provide sufficient contextual grounding.Questions that are too underspecified or cannot be assessed through explanatory criteria are rejected.
  • Annotation: Each retained question receives a query-specific rubric covering premise fidelity, central mechanisms, downstream consequences, domain constraints, uncertainty, and explanatory completeness.The rubric allows multiple plausible trajectories rather than prescribing a unique conclusion.
  • Quality Control: Quality control reviews question grounding and causal requirements alongside rubric coverage, assigning PASS, REVISED, or rejection outcomes.Rubrics are checked to avoid forcing a single gold trajectory.
  • PRISM Parsing: PRISM converts each response into a directed response-derived semantic causal graph by extracting EDUs and labeled relations, normalizing duplicates, and resolving edges.The parser uses conservative fallbacks when parsing fails or yields no valid EDUs.
  • PRISM Parsing: Adjacent sentence-level EDUs are connected with SEQUENCE edges when needed, keeping graph construction grounded in response text under a consistent format.The pipeline standardizes generation and downstream evaluation settings across responses.

F Additional Experiments

Additional experiments position WhatIfBench as a discriminative and comparatively demanding benchmark whose rankings differ from closed-form tasks. The evaluation also tests scale, response length, paraphrase robustness, prompting, and evaluator-family sensitivity.

  • Benchmark Comparison: WhatIfBench jointly evaluates open-domain, open-form, long-horizon reasoning with process-level evaluation and question-specific rubrics, unlike mainly constrained existing benchmarks.This comparison is summarized in Table 6.
  • Benchmark Comparison: Closed-form benchmarks yield substantially higher and more compressed performance, while model rankings differ considerably across tasks.The same six frontier models are evaluated on IfQA and CounterBench.
  • Benchmark Comparison: Qwen3-Max ranks first on IfQA but fifth on WhatIfBench, whereas DeepSeek-V4-Pro ranks first on CounterBench but fourth on WhatIfBench.WhatIfBench has stronger rank agreement with the Artificial Analysis Intelligence Index than IfQA or CounterBench.
  • Model Scale: Smaller models score 40.80–44.36 on WhatIfBench, below the frontier-model range of 51.99–64.62, showing discrimination across scales and architectures.The experiment spans dense and mixture-of-experts architectures.
  • Robustness: GPT-5.5’s final-score standard deviation is 0.48 across Vanilla, Long Output, and Short Output settings, indicating limited sensitivity to moderate response-length changes.PRISM does not explicitly reward response length.
  • Robustness: Using Claude-Sonnet-5 as evaluator preserves the six-model ranking exactly, with Spearman ρ = 1.00 and a mean absolute final-score difference of 0.63 points.The maximum difference is 1.57 points.

G Worked Example of PRISM Evaluation

The worked example traces a model answer from generation through causal-graph construction and dual evaluation. It illustrates how PRISM combines strict edge judgments with rubric-based scoring to expose weaknesses in an otherwise plausible response.

  • Example Pipeline: The worked example pairs a counterfactual question with a query-dependent rubric, generated response, response-derived causal graph, and criterion-level RM judgment.Figures 12–15 show these stages and their outputs.
  • Graph Construction: The response-to-graph parser organizes related elementary discourse units into causal modules and represents their expressed causal and discourse relations as arrows.The graph is an abstraction for readability rather than a direct restatement of the figure.
  • Process Metric: PRISM’s PM judge accepts an edge only when the answer explicitly supports the exact directed relation, defaulting to no when evidence is insufficient.The judge rejects links requiring outside knowledge, reconstruction, or charitable interpretation.
  • Process Metric: For cause, result, and consequence relations, the answer must express actual dependency rather than sequence, topic association, or merely plausible linkage.Directionality must also be appropriate and unambiguous.
  • Selected Case: In the selected WWII combat-pilot case, the model predicts modest military benefit and larger social and political impact after women are allowed to serve.The example uses this response as the input for PRISM evaluation.
  • Rubric Evaluation: The example receives RM = 0.50 with a raw rubric score of 5.0/10 and criterion scores [3, 5, 6, 6, 5].The first criterion scores 3 because the answer lacks a concrete prior point of divergence making the policy credible.
Loading 2608.27953v1…