PaperlayerView exact revision on arXiv

Source-linked AI summary

FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning

Yuxi Sun, Aoqi Zuo, Haotian Xie, Wei Gao, Mingming Gong, Jing Ma

arXiv:2604.10693v2cs.AI

TL;DR

LLMs can produce persuasive chains with unfaithful intermediate steps, while biased self-evaluation makes their faithfulness difficult to assess reliably. FACT-E uses controlled perturbations to estimate intra-chain faithfulness and combines that measure with answer consistency. Experiments report improved reasoning-trajectory selection and in-context learning, alongside detection of flawed reasoning under noisy conditions.

  • Problem

    LLMs can produce persuasive or correct-answer reasoning chains with broken logical dependencies or inaccurate intermediate content, making trajectory reliability difficult to assess.

    Source: §1 · p. 1

  • Method

    FACT-E uses external perturbations as an instrumental signal to refine intra-chain faithfulness judgments and combines them with CoT-to-answer consistency.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    FACT-E improves answer accuracy through trajectory selection and enhances in-context learning, with experiments also showing detection of process-level failures under noisy conditions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Under noisy conditions, FACT-E identifies process-level failures, supporting more robust and controllable LLM reasoning.

    Source: §1 · p. 2

    Standard FACT-E requires multiple LLM prompts and has higher inference costs than simpler prompting approaches; its lightweight variant reduces requests while maintaining competitive performance.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful intermediate steps. Existing self-evaluation approaches are prone to inherent biases: the model may confidently endorse coherence even when the step-to-step implication is not valid, leading to unreliable faithfulness evaluation. We propose FACT-E, a causality-inspired framework for evaluating CoT quality. FACT-E uses controlled perturbations as an instrumental signal to separate genuine step-to-step dependence from bias-driven artifacts, producing more reliable faithfulness estimates (\textit{intra-chain faithfulness}). To select trustworthy trajectories, FACT-E jointly considers \textit{intra-chain faithfulness} and \textit{CoT-to-answer consistency}, ensuring that selected chains are both faithful internally and supportive of the correct final answer. Experiments on GSM8K, MATH, and CommonsenseQA show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learning exemplars. FACT-E also reliably detects flawed reasoning under noisy conditions, providing a robust metric for trustworthy LLM reasoning.

1 Introduction

FACT-E addresses the gap between fluent, correct-looking chains and faithful reasoning by evaluating both answer support and links between intermediate steps. It uses causal perturbations to reduce bias in faithfulness assessment and reports improved trajectory selection, in-context learning, and detection of flawed reasoning under noisy conditions.

1 Introduction

  • CoT can appear persuasive or yield correct answers while containing inaccurate content or broken logical dependencies between intermediate steps.The paper frames detecting this intra-chain unfaithfulness as important for reasoning robustness and trustworthiness.Source: §1 · p. 1
  • Existing LLM-as-judge approaches focus on final-answer changes and may overlook dependencies among intermediate steps.Causal evaluators can also inherit LLM biases, allowing confident judgments without rigorous entailment.Source: §1 · pp. 1–2
  • FACT-E jointly evaluates whether a chain supports the correct answer and whether successive reasoning segments are faithful.Its two modules are CoT-to-Answer Consistency and Intra-Chain Faithfulness.Source: §1 · p. 2
  • FACT-E improves answer accuracy and in-context learning through reasoning-path selection, and identifies process-level failures under noisy conditions.Experiments span mathematical and commonsense reasoning tasks.Source: §1 · p. 2
  • External noise serves as an instrumental variable in a causal framework intended to mitigate bias in LLM self-assessment.The framework uses a contrastive design to estimate intra-chain faithfulness more reliably.Source: §1 · p. 2

2 Task Formulation with A Causal View

The section formulates CoT reliability as a causal evaluation problem: a trustworthy chain must reach the correct answer and maintain faithful links between its intermediate steps. It defines these two dimensions and models how LLM bias can distort faithfulness judgments.

2 Task Formulation with A Causal View

  • The task is to measure reasoning-chain trustworthiness from a causal perspective.The goal is to filter and select trustworthy chains from post-hoc candidates.Source: §2 · p. 2, §2.1 · p. 2

2.1 Problem Definition

  • The causal-graph formulation contrasts query-to-answer mediation, bias-affected self-assessment, and FACT-E's use of exogenous noise as an instrument.The noise is introduced to obtain a more reliable faithfulness evaluation.Source: Figure 2 · p. 3
  • A reliability score RS in [0, 1] should reflect both a correct final outcome and faithfulness among intermediate steps.Higher RS indicates a higher-quality reasoning process.Source: §2.1 · p. 3

2.2 CoT-to-Answer Consistency

  • CoT-to-Answer Consistency estimates the probability that a reasoning chain consistently leads to the correct final answer.The formulation models the query, chain, and answer as Q → S → A, with the chain mediating the query's influence on the answer.Source: §2.2 · p. 3

2.3 Intra-Chain Faithfulness

  • A correct answer alone is insufficient because flawed or hallucinated intermediate steps can still produce it.Intra-Chain Faithfulness is introduced to assess intermediate reasoning beyond the final answer.Source: §2.3 · p. 3
  • Intra-Chain Faithfulness measures logical dependencies between steps and their content correctness.Failures include superficially coherent chains that rely on fragile connections or erroneous intermediate steps.Source: §2.3 · p. 3
  • For a chain of L steps, faithfulness is evaluated between each prefix S≤i and its subsequent segment S>i, then aggregated across split points.The LLM's evaluation M mediates the faithfulness score Y, but unobservable internal bias Z can affect the assessment.Source: §2.3 · p. 3

3 Methodology

The methodology estimates whether a reasoning chain supports the correct answer and whether its intermediate steps are faithful, using perturbations to make faithfulness judgments more reliable. FACT-E combines these dimensions to score and select candidate chains.

3.1 Quantify Intra-Chain Faithfulness

  • FACT-E perturbs continuations after a shared prefix to expose broken logical dependencies and reduce self-affirmation or surface-pattern bias in faithfulness judgments.Perturbations can disrupt dependencies or corrupt content, and the contrastive design compares the original reasoning with a counterfactual rollout.Source: §3.1 · pp. 3–4

3.2 Quantify CoT-to-Answer Consistency

  • CoT-to-Answer Consistency measures the model’s confidence that a reasoning chain yields the correct final answer, independently of intermediate-step validity.The score is estimated through repeated judgments of whether the chain is sufficient to reach the correct answer.Source: §3.2 · pp. 4–5

3.3 Tandem Estimation

  • FACT-E scales answer correctness by reasoning faithfulness so its reliable score reflects outcomes grounded in logical integrity.Low faithfulness can indicate loose connections or errors even when the final answer is correct.Source: §3.3 · p. 5

3.4 Algorithms

  • The lightweight algorithm discards candidates with zero answer consistency, samples N intermediate checkpoints for the remaining chains, and selects the chain maximizing both scores.The fixed-checkpoint strategy addresses the computational overhead of evaluating faithfulness at every step.Source: §3.4 · p. 5

4 Experimental Setup

The experiments test whether FACT-E improves trajectory selection, in-context learning, and noisy-rationale filtering. They evaluate across three reasoning benchmarks, multiple LLM backbones, and established self-improvement and denoising baselines.

Experimental questions

  • The evaluation addresses whether FACT-E improves answer accuracy, produces stronger in-context exemplars, and detects noisy rationales.These are the section’s three research questions.Source: §4 · p. 5

Benchmarks and evaluation

  • FACT-E is evaluated across domains and difficulty levels using GSM8K and MATH-500 for mathematical reasoning, together with CommonsenseQA.For in-context learning evaluation, exemplar sizes are 5, 10, or 15.Source: §4 · p. 6

Noisy-rationale evaluation

  • For noisy-rationale evaluation, prompts contain benchmark rationales that may include flawed intermediate steps, with the number of noisy rationales varying by setting.Robustness is evaluated on NoRa-Math and NoRa-Commonsense.Source: §4 · p. 6

Models and baselines

  • The study evaluates DeepSeek-V3, Qwen3-14B, Gpt-4o-mini, and ChatGPT (Gpt-3.5-turbo), using temperature 0, typically three sampling trials, 500 questions per task, and three repeats.Baselines include CoT, POLISH, REFLECT, DENOISE, and CONSISTENCY.Source: §4 · pp. 6–7

In-context evaluation

  • Table 3 reports in-context accuracy with five demonstration examples and absolute changes relative to BASE.The broader evaluation also considers exemplar sizes of 10 and 15.Source: §4 · p. 6, Table 3 · p. 7

5 Experimental Results

FACT-E improves trajectory selection and in-context learning, detects unreliable rationales under noisy conditions, and distinguishes flawed reasoning processes. The experiments also find that three sampling trials typically capture most performance gains.

Trajectory selection

  • FACT-E standard was best or second-best in 8 of 12 configurations; on MATH-500 it scored 54.09% versus POLISH’s 41.00% and DENOISE’s 42.62%.On MATH-500, FACT-E standard exceeded the average baseline by 5.340% on ChatGPT and 4.703% on DeepSeek-V3.Source: §5 · p. 7

In-context learning

  • FACT-E achieved the best in-context performance in 8 of 12 configurations, including ChatGPT gains from 49.60% to 63.40% on CommonsenseQA and 46.99% to 53.28% on MATH-500.Its performance was reported as more stable across models and benchmarks than competing baselines.Source: §5 · p. 7

Noisy-rationale detection

  • FACT-E achieved top NoRa-Commonsense accuracy with Qwen3 and DeepSeek-V3, and leading NoRa-Math performance with Gpt-4o-mini.Gpt-4o-mini was slightly less effective on NoRa-Commonsense.Source: §5 · pp. 7–8, Figure 3 · p. 7

Case study

  • In case studies, FACT-E assigned a score of 0 to a chain with an erroneous trigonometric derivation and 0.596 to redundant reasoning.The first example contrasts a correct-answer chain with one whose derivation leads to an incorrect result.Source: §5 · p. 8, Table 4 · p. 8

Sampling trials

  • Three sampling trials typically capture most performance gains, while four or five trials yield diminishing returns.Accuracy often rises substantially between the second and third trials before saturating or fluctuating slightly.Source: §5 · p. 8, Figure 4 · p. 8

6 Related Work

Prior CoT-improvement work has emphasized accuracy, while reliable assessment of a specific reasoning trace remains an open practical question. Existing evaluation approaches rely on model self-judgment or causal dependency analysis, with limitations in bias, uncertainty quantification, and scalability.

CoT improvement

CoT evaluation

  • LLM-based self-correction and reflection rely on model judgment and can be sensitive to bias, while causal approaches use measures such as PNS or ACE.The cited causal approaches are described as dependent on self-assessment, lacking principled uncertainty quantification, and facing scalability limits.Source: §6 · p. 9

Positioning

  • The paper frames its approach as disentangling step-level faithfulness dependencies while addressing confounding from internal bias.This contrasts with approaches described as relying on model self-assessment.Source: §6 · p. 9

7 Conclusion

The conclusion presents FACT-E as a Structural Causal Model-based framework for addressing bias in LLM self-evaluation. Constructed noise serves as an instrumental variable for estimating Average Causal Effect and more reliably assessing reasoning faithfulness.

Conclusion

  • FACT-E uses constructed noise as an instrumental variable to estimate Average Causal Effect and mitigate spurious correlations from internal model biases.The stated aim is a more reliable estimate of reasoning faithfulness.Source: §7 · p. 9

Limitations

Standard FACT-E evaluates every reasoning step but costs more inference requests than simpler prompting; lightweight FACT-E reduces requests while maintaining competitive performance. Its bias mitigation is targeted, and its post-hoc design selects completed outputs rather than intervening in generation.

Limitations

  • Standard FACT-E assesses every CoT step at higher inference cost, while lightweight FACT-E reduces prompting requests and maintains competitive performance.The paper also notes that standard FACT-E costs more than simpler prompting approaches.Source: Limitations · p. 9
  • FACT-E targets self-affirmation and shortcut biases, but cannot account for every cognitive or pretraining-data artifact.The framework is described as addressing a broad bias landscape selectively.Source: Limitations · p. 9
  • FACT-E is post-hoc: it selects among completed outputs rather than intervening in CoT generation.The paper notes that cross-model or dynamic evaluation can introduce confounds such as inter-model sycophancy or shared parametric biases.Source: Limitations · p. 9

Ethics Statement

The appendices describe FACT-E variants and noisy-CoT filtering, illustrate its scores on flawed and sound reasoning, and report inference-cost scaling and an ablation supporting the combined metrics. The ethics statement reports use of public, objective datasets without human participants.

Ethics Statement

  • The study uses public, objective and neutral datasets, contains no personal or offensive material, and involves no human participants.The authors state that LLMs were not used for experiments involving human participants.Source: Ethics Statement · p. 9

A.1 The Standard and Lightweight Versions

  • Standard FACT-E checks all sequential reasoning dependencies; its lightweight variant uses fixed checkpoints, with checkpoint count N matched to the consistency-sampling budget.The lightweight design reduces prompting requests to limit inference cost.Source: §A.1 · p. 13

A.2 Algorithm for Detecting Noisy CoT Methods

  • Noisy-CoT filtering combines answer consistency and sampled step-faithfulness into a reliability score, retaining exemplars above a validation-selected threshold.Exemplars with zero consistency are pruned first; retained demonstrations form the prompt for final inference.Source: §A.2 · p. 13

A.3 Noise Injection

  • Five perturbation types target step precision, conceptual accuracy, inductive and structural integrity, and logical grounding.Operation Error and Conceptual Swap alter operators or concepts; Contradiction introduces premise-violating information.Source: §A.3 · p. 13

A.4 More Case Studies

  • FACT-E scores distinguish correct answers with flawed logic from rigorous reasoning, assigning 0.7995 versus 0.5992 in a case study.Other flawed or incorrect cases receive scores such as 0.177 and 0.338, while rigorous correct chains score approximately 0.8.Source: §A.4 · pp. 13–14

A.5 LLM Inference Overhead

  • With N = 3, lightweight inference requests scale linearly with checkpoint budget, while standard requests scale with inspected reasoning steps.The exact count can vary with prompt batching; Table 5 reports requests per query under the specified setting.Source: §A.5 · p. 14, Table 5 · p. 14

A.6 Ablative Study

  • The ablation results support combining Intra-Chain Faithfulness with CoT-to-Answer Consistency.Figure 5 presents the ablation study.Source: §A.6 · p. 14, Figure 5 · p. 14

A.7 Baseline Methods

The baseline evaluation compares FACT-E against five representative methods across four LLM backbones, using repeated runs on 500 questions per task. The baselines span self-refinement, denoising, and self-consistency strategies.

A.7 Baseline Methods

  • Evaluations use four LLM backbones, temperature 0, 500 questions per task, and three repetitions to compare five representative baselines.The listed backbones are DeepSeek-V3, Qwen3-14B, GPT-4o-mini, and GPT-3.5-turbo.Source: §A.7 · p. 14
  • POLISH repeatedly refines noisy CoT exemplars, while REFLECT prompts the model to critique and revise an initial chain.POLISH repeats refinement three times; REFLECT uses an initial CoT followed by critique and a revised CoT.Source: §A.7 · p. 15
  • DENOISE masks rationale content, asks the model to reconstruct it, and selects the most frequent answer across three repetitions.Source: §A.7 · p. 15
  • CONSISTENCY samples the same reasoning task three times and returns the majority answer without modifying the input.The method aggregates sampled outputs through majority voting.Source: §A.7 · p. 15

B Further Analysis

Across increasing MATH-500 difficulty, FACT-E maintains stronger performance than baselines, including a 12.62-point advantage over CoT on DeepSeek-V3 at Lvl-4. The appendix also describes prompt materials and limits LLM use to writing assistance.

B Further Analysis

  • Against CONSISTENCY, OURS improves total accuracy by 9.52% on GPT-4o-mini and 4.03% on DeepSeek V3 across Math500.The paper reports that standard enhancement methods lose efficacy as logical complexity increases.Source: B Further Analysis · p. 15
  • Across example counts, OURS remains robust without noticeable decline; Qwen3-14B reaches 93.26% accuracy with 15 examples.Increasing in-context examples does not uniformly improve performance, and can degrade it for Qwen3-14B under POLISH and REFLECT.Source: B Further Analysis · p. 16, Table 8 · p. 16
  • At MATH-500 Lvl-4, OURS reaches 92.31% accuracy on DeepSeek-V3 versus COT's 79.69%, a 12.62% advantage.Figure 6 shows performance declining as difficulty rises; across Math500, OURS is state-of-the-art or competitive on evaluated models.Source: B Further Analysis · p. 15, Figure 6 · p. 17

C Prompts

D LLM Usage Claim

  • LLMs were used only to polish writing for linguistic clarity, coherence, grammar, and style, with no additional functionality.Source: D LLM Usage Claim · p. 17

Related papers

Loading 2604.10693v2…