PaperlayerView exact revision on arXiv

Source-linked AI summary

Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?

Haoang Chi, He Li, Wenjing Yang, Feng Liu, Long Lan, Xiaoguang Ren, Tongliang Liu, Bo Han

arXiv:2506.21215v1cs.AIcs.CLcs.LG

TL;DR

The paper investigates whether LLMs genuinely reason causally or mainly retrieve familiar causal knowledge. It analyzes autoregression, introduces the fresh CausalProbe 2024 benchmark, and proposes G2-Reasoner; results support limited level-1 reasoning and improvements toward level-2 reasoning in fresh contexts.

  • Problem

    The paper asks whether LLMs' apparent causal reasoning is genuine human-like reasoning or primarily retrieval of causal knowledge embedded in their parameters.

  • Method

    The paper analyzes autoregressive LLMs with causal modeling, evaluates them on fresh CausalProbe 2024 data, and proposes G2-Reasoner with general knowledge and goal-oriented prompts.

  • Results

    G2-Reasoner significantly enhances LLM causal reasoning, particularly on fresh and counterfactual contexts, while CausalProbe 2024 reveals substantial drops on unseen data.

  • Takeaways & Limitations

    The findings support viewing current LLM causal reasoning as primarily level-1 and identify general knowledge plus goal-oriented prompting as a path toward level-2 reasoning.

  • Takeaways & Limitations

    G2-Reasoner's results rely on a small general-knowledge base, and the paper could not repeat all experiments with a significantly larger source because of resource constraints.

Abstract

from arXiv · show

Causal reasoning capability is critical in advancing large language models (LLMs) toward strong artificial intelligence. While versatile LLMs appear to have demonstrated capabilities in understanding contextual causality and providing responses that obey the laws of causality, it remains unclear whether they perform genuine causal reasoning akin to humans. However, current evidence indicates the contrary. Specifically, LLMs are only capable of performing shallow (level-1) causal reasoning, primarily attributed to the causal knowledge embedded in their parameters, but they lack the capacity for genuine human-like (level-2) causal reasoning. To support this hypothesis, methodologically, we delve into the autoregression mechanism of transformer-based LLMs, revealing that it is not inherently causal. Empirically, we introduce a new causal Q&A benchmark called CausalProbe-2024, whose corpora are fresh and nearly unseen for the studied LLMs. The LLMs exhibit a significant performance drop on CausalProbe-2024 compared to earlier benchmarks, indicating the fact that they primarily engage in level-1 causal reasoning. To bridge the gap towards level-2 causal reasoning, we draw inspiration from the fact that human reasoning is usually facilitated by general knowledge and intended goals. We propose G^2-Reasoner, a method that incorporates general knowledge and goal-oriented prompts into LLMs' causal reasoning processes. Experiments demonstrate that G^2-Reasoner significantly enhances LLMs' causal reasoning capability, particularly in fresh and counterfactual contexts. This work sheds light on a new path for LLMs to advance towards genuine causal reasoning, going beyond level-1 and making strides towards level-2.

1 Introduction

The paper asks whether LLMs' apparent causal reasoning reflects genuine human-like reasoning or primarily retrieval of familiar causal knowledge. It introduces CausalProbe 2024 and G2-Reasoner to diagnose and improve this capability.

  • Motivation: LLMs can answer familiar causal questions, but struggle with uncommon tasks involving new causal knowledge or specific causal quantities.The paper treats this contrast as evidence that apparent causal competence may be a mirage.
  • Methodological analysis: Transformer LLMs use autoregressive next-token prediction, which does not inherently encode causal relationships between tokens.The paper argues that sequential dependence in text is not equivalent to logical causality.
  • Empirical test: CausalProbe 2024 uses fresh corpora published after the studied models' release, enabling evaluation on material not included verbatim in their training data.The benchmark includes easy, hard, and uncertain multiple-choice variants.
  • Empirical test: All studied LLMs show significant performance drops on CausalProbe 2024 relative to earlier causal Q&A benchmarks.The longitudinal comparison supports the paper's hypothesis that earlier benchmark performance can reflect memorized causal knowledge.
  • Proposed framework: G2-Reasoner combines retrieved general knowledge with goal-oriented prompts to improve causal reasoning, especially on fresh and fictitious tasks.The reported improvement is consistent across open-source and closed-source LLMs.

2 Related Work

The related work spans general LLM reasoning, causal reasoning tasks, and causal benchmarks. Existing studies assess causal reasoning and develop prompt-based methods, while many benchmarks target causal relationships in text.

  • LLM reasoning: Prior research examines LLM reasoning across commonsense, mathematical, logical, and causal tasks.The cited work frames reasoning as important for theorem proving, problem-solving, and robotics.
  • Causal reasoning: Causal reasoning tasks include causal discovery, cause attribution, and causal effect estimation.The paper distinguishes recovering causal structure, identifying potential causes, and estimating effects.
  • Benchmarks: Causal benchmarks include datasets built from language rules, real-world events, and human-labeled question-answer pairs.The paper surveys benchmarks including CausalQA, CRAB, and FCR.

3 Problem Formalization

The paper formalizes textual causal reasoning and distinguishes two capability levels. It models the task with causal variables, world laws, and their transformation into natural language.

  • Definitions: In this work, LLM causal reasoning concerns textual causal knowledge rather than numerical statistical causal inference.The definition includes understanding contexts and answering causal queries consistently with those contexts and causal laws.
  • Capability levels: Level-1 causal reasoning retrieves causal knowledge embedded in model parameters and contextual information for simple cause-and-effect relationships.The paper characterizes this form as typically fast.
  • Capability levels: The paper studies an interaction between fast and deeper reasoning levels rather than claiming that level-2 is always preferable.It emphasizes adaptive switching as desirable for speed and reliability.
  • Scope: The paper restricts its analysis primarily to qualitative tasks with a single cause-effect pair, excluding multiple pairs, mediators, and mainly quantitative estimation.This scope is stated as a formalization assumption.

4 LLMs cannot Perform Genuine Causal Reasoning

The paper argues that autoregressive next-token prediction is not necessarily causal and tests this claim with fresh benchmarks. Performance declines on fresher data, supporting reliance on memorized causal knowledge.

  • 4.1 Autoregressive LLMs are not Necessarily Causal: Autoregressive prediction conditions each next token on prior tokens, but unfamiliar or non-sequentially causal contexts can lead LLMs to misunderstand causal knowledge.The paper identifies this as a central mechanism limiting level-2 reasoning.
  • 4.1 Autoregressive LLMs are not Necessarily Causal: When likely next tokens conflict with causal laws or the context, autoregressive decoding can produce incorrect causal reasoning results.The paper contrasts this with familiar contexts expressing correct causal knowledge, where models perform well.
  • 4.2 Empirical Study on Causal Reasoning Capabilities of LLMs: CausalProbe 2024 uses post-January-1-2024 corpora and progressively probes novel causal reasoning with easy, hard, and multiple-choice datasets.The benchmark is designed to reduce overlap with the studied LLMs' training data.
  • 4.2 Empirical Study on Causal Reasoning Capabilities of LLMs: The comparison uses COPA, e-CARE, and CausalNet as earlier or intermediate causal Q&A benchmarks to test dependence on embedded training knowledge.COPA and e-CARE are described as likely to overlap with training data.
  • 4.2 Empirical Study on Causal Reasoning Capabilities of LLMs: All four studied LLMs show significant performance drops on CausalProbe-E and CausalProbe-H, with Claude 3 Opus below 70% average exact match on CausalProbe-H.Because the benchmark uses everyday news rather than mainly unfamiliar concepts, the paper attributes the degradation primarily to corpus freshness.

5 G2-Reasoner: A General-Knowledge-Assisted and Goal-Driven Reasoner

G2-Reasoner combines general knowledge with goal-oriented prompting to guide textual causal reasoning. Its formulation uses causal-graph structure and a retrieval-augmented knowledge base, while acknowledging that complete knowledge coverage is impractical.

  • Causal model for causal reasoning in LLMs: The framework formalizes textual causal reasoning with a causal graph containing cause X, effect Y, world knowledge C, and conditioned descriptions T.The graph represents X → Y while accounting for confounding and conditioning relationships.
  • Causal formulation: Equations (1) and (2) describe effect prediction conditioned on the stated cause, textual description, and, in Eq. (2), general knowledge.Eq. (2) accounts for the confounding variable through the total probability formula, with PC denoting the general knowledge base.
  • Goal-driven prompting: The goal-driven prompt is designed to maintain focused generation because longer autoregressive sequences can lose coherence and deviate from initial targets.The prompt guides the model toward identifying correct causal relationships during decoding.
  • 5 G2-Reasoner: A General-Knowledge-Assisted and Goal-Driven Reasoner: G2-Reasoner uses general knowledge as a reasoning basis and intended goals as a guide.The framework is motivated by human reasoning mechanisms and causal graph theory.
  • Evaluation: The evaluation uses exact match on four causal Q&A benchmarks and training-data freshness detection based on average negative log-likelihood.Table 2 reports exact match, while Table 3 uses Min-K% Prob and interprets smaller average negative log-likelihood as fresher.
  • Knowledge-assisted reasoning: G2-Reasoner retrieves related general knowledge through RAG and uses it to steer goal-oriented causal answering.The knowledge base is a small approximately 16 Mb general-knowledge Q&A dataset whose answers serve as retrieval documents.

6 Experiments

The experiments construct CausalProbe 2024 from fresh news corpora and evaluate its contextual, hierarchical causal reasoning tasks. Results show performance declines on fresher benchmarks, while G2-Reasoner generally improves performance but remains limited by its knowledge resources.

  • Construction: CausalProbe 2024 is built by collecting recent web articles and using LLMs to generate question-answer pairs.The corpus includes BBC and Guardian articles across technology, environment, business, health, world news, culture, and climate.
  • Construction: 3,461 unique Q&A data remain after filtering unethical questions and articles exceeding 15,000 characters.
  • Benchmark superiority: CausalProbe 2024 hierarchically assesses causal reasoning through CausalProbe-E, CausalProbe-H, and CausalProbe-M.The benchmark evaluates novel causal problems, misleading or deceptive propositions, and multiple valid causal answers.
  • Benchmark superiority: Contextual information significantly improves performance on CausalProbe-E and CausalProbe-H.The benchmark includes concise contexts because earlier benchmarks often lack the background knowledge needed for causal questions.
  • Results: All studied LLMs show significant performance drops on CausalProbe 2024 compared with earlier causal Q&A benchmarks.The authors attribute the degradation mainly to corpus freshness and use Min-K% Prob to evaluate benchmark freshness.
  • Results: G2-Reasoner generally outperforms vanilla inference, but its gains are limited by the scale of the general knowledge base and vector databases.RAG, described as an ablation of G2-Reasoner, usually does not reach vanilla performance, supporting the role of goal-oriented prompting.
  • Results: Under partial match on CausalProbe-M, GPT and Claude achieve approximately 75% and 85% accuracy, respectively, while exact match remains difficult.Partial match permits missing correct options but not selecting incorrect ones.

7 Conclusion and Future Outlook

The paper argues that current LLMs are limited to level-1 causal reasoning and introduces CausalProbe 2024 and G2-Reasoner to study and improve this capability. G2-Reasoner enhances causal reasoning particularly in fresh and counterfactual contexts, but the work does not achieve human-like level-2 reasoning.

  • The paper argues that current LLMs are limited to level-1 causal reasoning.
  • CausalProbe 2024 reveals that LLMs struggle with causal reasoning in unseen contexts and primarily rely on causal knowledge in training data.
  • G2-Reasoner incorporates general knowledge and goal-oriented prompts into LLM causal reasoning.
  • G2-Reasoner enhances causal reasoning capability, particularly in fresh and counterfactual contexts.
  • The proposed method remains unable to enable human-like level-2 causal reasoning.

Supplement to “Unveiling Causal Reasoning in Large

The passage identifies the work as “Language Models: Reality or Mirage?”

  • The passage names language models as the subject of the work.
  • The title frames the discussion as a question of reality versus mirage.
  • The passage provides only a title fragment, without substantive findings or methods.

Content

The paper identifies limitations in both its causal reasoning method and the claim that CausalProbe 2024 is unseen by LLMs. It also notes that the work cannot bring LLM causal reasoning to the human level, so errors may remain.

  • Limitations: The proposed causal reasoning method is only a step forward and cannot realize genuine causal reasoning.
  • Limitations: CausalProbe 2024 cannot be fully confirmed as unseen because its content remains part of human knowledge and comparable events may appear in pretraining data.
  • Broader Impacts: The work cannot promote LLM causal reasoning to the human level, so causal reasoning results may still contain errors.

C Full Implementation Details

The paper evaluates causal reasoning with multiple benchmarks and uses Min-K% Prob to assess whether benchmark texts appeared in LLM pre-training data. It also describes the implementation of G2-Reasoner and the causal-reasoning literature.

  • Experimental setup: Experiments use Ubuntu 20.04 and NVIDIA RTX A6000 GPUs, with official APIs for GPT 3.5 turbo and Claude 3 opus.Inference temperatures are fixed at 1.0 for closed-source models and 0 for open-source models; chain-of-thought receives 128 maximum new tokens and other cases receive 50.
  • G2-Reasoner: G2-Reasoner uses retrieval-augmented generation with Faiss, Meta’s Contriever, and an external general-knowledge dataset.The implementation follows the realization of self-RAG for constructing the retrieval system.
  • Causal reasoning literature: Causal reasoning tasks include causal discovery, cause attribution, and causal effect estimation.Causal discovery recovers latent causal structures, cause attribution identifies potential causes, and effect estimation examines effects of cause variables.
  • Causal reasoning benchmarks: Existing causal benchmarks include CausalQA, CRAB, FCR, Cladder, CausalNet, COPA, and e-CARE.These datasets use extracted questions, real-world stories, human labels, symbolic causal models, LLM-generated scenarios, commonsense questions, or crowdsourced causal facts.
  • Training-data detection: Min-K% Prob detects likely training-data membership from the average log probability of the lowest-probability K% of tokens.If the resulting score exceeds threshold ϵ, the sequence is determined not to be in pre-training data; otherwise it is determined to be in the data.

G.1 Standard Deviations of Main Results

The appendix reports standard deviations, benchmark construction and quality-control procedures, and descriptive statistics for CausalProbe 2024. The benchmark contains 6,922 Q&A items across two equal-sized sub-datasets and multiple topics.

  • G.1 Standard Deviations of Main Results: Standard deviations are generally small across repeated experiments with the studied LLMs.For GPT 3.5 turbo and Claude 3 opus, temperature 1.0 produced slightly better results than 0, and experiments were repeated three times.
  • Dataset construction: CausalProbe-H includes made-up cause-effect pairs for testing causal reasoning under counterfactual disturbance.CausalProbe-M varies the number of correct options to prevent models from answering correctly by relying on a fixed response pattern.
  • Quality control: The benchmark quality pipeline combines corpus cleaning, GPT 3.5 Turbo generation, prompt-template selection, script-based filtering, and manual review.The preparation stage uses regular expressions and Google DLP API detection before generation and verification.
  • Human verification: 232 of 260 sampled CausalProbe-H questions were filtered out after assignment to three volunteers for quality control.Questions answered correctly by at least two volunteers were treated as high-quality data.
  • CausalProbe 2024 statistics: CausalProbe 2024 contains 6,922 Q&A data, divided into two sub-datasets of 3,461 items each.Both sub-datasets cover technology, culture, business, climate, and world news; questions ask for either causes or effects.

I.2 Addition Analysis

CausalProbe 2024 uses recent authoritative web articles and includes several dataset-design analyses. The paper also identifies model, generation, and language-cultural biases introduced during benchmark construction.

  • I.2 Addition Analysis: CausalProbe 2024 draws on articles published from January 1, 2024, through April 29, 2024.The paper states that these articles postdate many LLM releases and are therefore unlikely to be directly included in some open-source models’ pre-training corpora.
  • I.2 Addition Analysis: Table 6 reports exact-match results and standard deviations for studied LLMs across four causal Q&A benchmarks.The table distinguishes vanilla inference from CausalProbe-E and CausalProbe-H, with temperature 1.0 for GPT 3.5 turbo and Claude 3 opus.
  • I.3 Potential bias: CausalProbe 2024’s English news corpus and GPT-3.5 Turbo’s predominantly English training data may introduce Western or Anglophone biases.The paper also identifies model bias from GPT-3.5 Turbo’s training data and generation bias toward certain question or answer types.
  • I.3 Potential bias: The proposed bias-mitigation measures are diversified data sources and thorough human review and curation.Suggested sources include manually created Q&A pairs and data generated by different LLMs.
  • I.2 Addition Analysis: CausalProbe 2024 additionally supplies background knowledge as context, unlike the other three benchmarks’ question-internal context or premise.The figure describes the four benchmarks as having different corpora while exhibiting similar format and difficulty.
Loading 2506.21215v1…