PaperlayerView exact revision on arXiv

Source-linked AI summary

Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth

Yoav Gur-Arieh, Ana Marasović, Mor Geva

arXiv:2605.25052v1cs.CL

TL;DR

Existing faithfulness metrics lack ground-truth evaluation, making it unclear whether they measure whether chain-of-thought traces reflect models’ computations. The paper introduces BONAFIDE, a benchmark with automated ground-truth labels, and finds that most metrics perform near chance, with none providing a reliable, practical measure across settings.

  • Problem

    Existing faithfulness metrics lack ground-truth evaluation, leaving it unclear whether chain-of-thought traces faithfully represent models’ internal computations.

  • Method

    The paper constructs procedurally generated bottleneck tasks and an automated labeling pipeline to produce step- and CoT-level ground-truth labels across models and tasks.

  • Results

    Most metrics perform near chance and show systematic biases; the strongest reaches 0.70 AUROC for CoTs, while neither strongest metric transfers across evaluation levels.

  • Takeaways & Limitations

    Current faithfulness metrics do not provide a reliable, practical measure of chain-of-thought faithfulness across settings.

  • Takeaways & Limitations

    The methodology provides ground-truth knowledge only for specific bottleneck steps, leaving many additional CoT steps without a signal and making faithful CoTs harder to certify.

Abstract

from arXiv · show

Chains of thought (CoTs) have become central in interpreting and auditing behaviors of large language models. Yet growing evidence suggests that these traces often fail to faithfully represent the computations behind a model's predictions. Several faithfulness metrics have been proposed, but whether they indeed measure faithfulness remains unknown. Answering this requires ground-truth labels, which are hard to obtain since internal computations are not directly observable. Consequently, most works proposing metrics report only absolute scores or comparisons to prior metrics, and the few existing benchmarks rely on proxies like plausibility or importance, properties orthogonal to faithfulness that can mislead about whether a CoT can be trusted. We address this challenge by constructing tasks whose outputs reveal which intermediate computations must have produced them, and developing an automated labeling pipeline that yields ground-truth faithfulness labels at both the step and CoT level. Building on this methodology, we present BonaFide, a benchmark of 3,066 labeled CoTs across 13 tasks and 10 models, and use it to conduct the first systematic evaluation of prominent faithfulness metrics. Our experiments show that most metrics perform near chance, exhibit strong prediction biases and degrade on longer CoTs. The best metric reaches only 0.70 AUROC at the CoT level while another reaches 0.59 at the step level, with neither transferring across settings, while entailing prohibitively high computational cost. Our results expose fundamental gaps in current faithfulness evaluation and call for the development of more reliable and efficient metrics.

1 Introduction

The paper argues that existing faithfulness metrics cannot yet be trusted because they lack ground-truth validation and often rely on misleading indirect proxies. It introduces BONAFIDE, a ground-truth benchmark built from tasks with recoverable intermediate computations, and finds that current metrics are generally unreliable and costly.

  • Motivation: Existing faithfulness metrics remain unvalidated because LLM internal computations are unobservable and evaluations rely on indirect proxies.These proxies include properties such as plausibility or importance, which may not establish faithfulness.
  • Method: Tasks with outputs that require specific intermediate computations enable automated ground-truth labels for faithful and unfaithful steps and CoTs.For example, a misleading hint can reveal whether a model followed that hint or fabricated an independent justification.
  • Benchmark: 98.9% precision was achieved by the automated labeling pipeline validated against human annotation across 13 tasks and 10 language models.The models span 4B to 70B parameters and include reasoning and non-reasoning variants from four model families.
  • Results: Most metrics perform near chance, with CC-SHAP reaching 0.70 AUROC for CoTs and Filler Tokens reaching 0.59 for steps.No single metric reliably distinguishes faithful from unfaithful text at both levels.
  • Implications: Neither strongest metric transfers across settings, while current approaches fail to provide a reliable, practical measure of faithfulness.The paper therefore calls for more accurate and efficient metrics and releases BONAFIDE with evaluation code.

2 Revising chain-of-thought faithfulness

The paper revises CoT faithfulness by evaluating individual steps mechanistically and defining faithful CoTs as complete, non-misleading reasoning paths. It distinguishes faithfulness from plausibility and causal importance, which can diverge from what the model actually did.

  • Motivation: Contemporary reasoning-model CoTs are extended verbalizations that may include steps unrelated or irrelevant to the model’s prediction.This motivates revising definitions designed for coherent, self-contained explanations.
  • Step-level faithfulness: A step is faithful if it accurately describes an internal process that produced its information, under a mechanistic rather than phenomenological reading.The definition does not require the step to describe the computation executed during its own generation.
  • CoT-level faithfulness: A CoT is faithful iff it contains the complete reasoning path followed to reach the answer and contains no unfaithful steps.Step-level faithfulness alone is insufficient because individually faithful steps can collectively conceal the model’s actual reasoning.
  • Distinguishing related properties: Plausibility is orthogonal to faithfulness: reasoning may be implausible yet actual, or plausible yet unlike the process the model used.Using plausibility as a faithfulness proxy can therefore provide false assurances.
  • Distinguishing related properties: Step importance concerns causal influence on the answer, but importance and faithfulness remain independent properties.A step can faithfully describe a process that occurred without being causally decisive for the final answer.

3 Eliciting ground-truth labels

The authors elicit ground-truth faithfulness labels by designing tasks whose required intermediate computations reveal whether a CoT contains the model’s necessary reasoning path. They implement this through outright tasks with bottleneck steps and diversionary tasks involving wrong-answer hints.

  • Method: The labeling method relies on task outputs that reveal computations necessary to produce the correct answer, despite internal reasoning being unobservable.Current interpretability tools lack reliable, scalable access to the reasoning process underlying a given output.
  • Diversionary: In diversionary tasks, models receive a question and a random wrong-answer hint, making explicit acknowledgment of the hint a faithful step when the model follows it.Omitting the hint from the CoT is unfaithful under the labeling principle.
  • Outright: In outright tasks, correct answers imply specific intermediate computations, called bottleneck steps, must have occurred internally.Examples include sequential graph traversal and iterative Collatz-function applications.
  • Task coverage: The benchmark spans ten task types across arithmetic, cryptography, text processing, graph traversal, and logical reasoning, with additional ground-truth steps from indirect hints.Open-ended questions from Humanity’s Last Exam, SimpleQA, and DDXPlus are used to reduce the chance of reaching hinted answers accidentally.
  • Outright: A CoT is labeled faithful when it contains all necessary ground-truth steps and unfaithful when it omits any.This criterion is applied to the intermediate computations required by each task.

4 BONAFIDE

BONAFIDE is a benchmark of 3,066 CoTs across 13 tasks and 10 models with automated ground-truth faithfulness labels at both step and CoT levels. Its pipeline produces labels at scale and is validated against human annotations with high precision.

  • Benchmark: BONAFIDE provides ground-truth faithfulness labels for 3,066 CoTs spanning 13 tasks and 10 models, at both step and CoT levels.The benchmark covers outright and diversionary settings.
  • Labeling pipeline: The automated pipeline generates tasks, collects model CoTs and responses, and labels CoTs and steps using the prescribed outright and diversionary settings.It uses ten open-weight models ranging from 4B to 70B parameters across four model families.
  • Labeling pipeline: Faithful commitment and bottleneck execution steps receive Faithful Step labels, whereas misattribution and tool-call steps receive Unfaithful Step labels.CoT-level labels aggregate these step labels with setting-specific requirements for hint acknowledgment, faithful commitment, and bottleneck execution.
  • Validation: 98.9% precision (95% CI [96.6, 100]) was achieved against human annotations, at a total judge cost of ∼$2,100.Gemini 3 Flash served as the retrieval judge and Gemini 3 Pro as the validation judge.
  • Dataset composition: 1,946 step-level labels comprise 51% Faithful Step and 49% Unfaithful Step, while 1,120 CoT-level labels comprise 15% Faithful CoT and 85% Unfaithful CoT.The dataset totals roughly 4M tokens and is sampled to balance models and label types.

5 Experiments

Experiments on BONAFIDE show that most faithfulness metrics perform near chance, with limited successes that do not transfer across evaluation levels. Metrics are also costly, degrade on longer CoTs, and exhibit systematic biases that expose shortcomings for reliable monitoring.

  • Performance: Most metrics perform near chance: CC-SHAP reaches 0.70 AUROC at the CoT level, while Filler Tokens reaches 0.59 at the step level.CC-SHAP falls below random at the step level, and Filler Tokens is near chance at the CoT level.
  • Efficiency and robustness: CC-SHAP requires up to 103 seconds per instance, while most metrics lose accuracy as CoTs become longer.These computational and length-sensitivity problems make current metrics unsuitable for real-time monitoring.
  • Metric biases: Importance-based metrics systematically skew toward labeling CoTs unfaithful, whereas semantic-utility metrics show the opposite bias.Importance-based metrics may conflate importance with faithfulness, while unfaithful CoTs can still contain enough verbalized reasoning for weaker models to reproduce the answer.
  • Limitations: The benchmark’s precision-first labels provide ground-truth knowledge only for specific required steps, leaving many additional CoT steps without signal.At the CoT level, every step must be faithful or inert for a faithful label, while one unfaithful or missing bottleneck step suffices for an unfaithful label.

6 Related work

Prior faithfulness benchmarks conflate plausibility with faithfulness, while monitorability and hint-based studies address related but distinct questions. BonaFide instead makes faithfulness evaluation tractable by using diversionary hints to construct ground-truth labels for evaluating metrics themselves.

  • Faithfulness benchmarks: Existing benchmarks conflate plausibility and faithfulness, potentially labeling faithful representations of flawed internal computations as unfaithful.FaithCoT-Bench relies on annotators’ judgments that reasoning paths are implausible or wrong.
  • Monitorability: Monitorability research predicts behavioral properties from model outputs instead of directly measuring faithfulness.BonaFide demonstrates that faithfulness evaluation can be tractable, creating an avenue for improving monitorability.
  • Hinting and bias injection: Hint-based studies show that models often follow misleading prompts without mentioning them in their CoTs, whereas BonaFide uses hints to create ground-truth labels for evaluating faithfulness metrics.The paper’s diversionary setting repurposes hinting methodology for metric evaluation rather than only demonstrating unfaithfulness.

7 Conclusion

BONAFIDE is the first benchmark evaluating chain-of-thought faithfulness metrics against ground-truth labels rather than proxies. It provides step- and CoT-level labels across ten models and thirteen tasks, revealing that most of eight evaluated metrics perform near chance, exhibit systematic biases, and degrade on long traces.

  • Benchmark: BONAFIDE evaluates chain-of-thought faithfulness metrics against ground-truth labels rather than proxies such as plausibility or importance.The benchmark builds on task designs where a model’s output reveals which intermediate computations must have produced it.
  • Benchmark: BONAFIDE provides step- and CoT-level faithfulness labels across ten models and thirteen tasks.
  • Evaluation: Most of the eight evaluated faithfulness metrics perform near chance, exhibit systematic biases, and degrade on long traces.

A Task details … C Labeling pipeline

BonaFide combines procedurally constructed outright and diversionary tasks with ground-truth-revealing bottleneck computations or hint reliance, evaluates ten models, and documents its labeling pipeline implementation. The benchmark spans diverse reasoning domains and model families while testing whether intermediate computations are necessary for solving tasks.

  • A.1 Outright: Ten procedurally constructed outright task types span arithmetic, number theory, cryptography, text processing, scientific reasoning, graph traversal, and logical reasoning.Each task has a unique correct answer and exposes a known sequence of bottleneck steps for checking CoT steps.
  • A.1 Outright: 1.5% accuracy without CoTs shows that the outright tasks generally cannot be solved by the evaluated models without generating intermediate reasoning.The evaluation used 100 randomly generated tasks for each of 10 models while prefilling an empty CoT and answer format.
  • A.2 Diversionary: The diversionary setting pairs questions with hints pointing to random wrong answers, making reliance on the hint evidence of an internal process caused by its presence.The construction is designed to rule out interpreting the wrong answer as a guess.
  • A.2 Diversionary: Diversionary questions cover factual recall, multi-hop, mathematical and scientific reasoning, medical diagnosis, and more, with incorrect candidates generated by Gemini 3 Flash using varied personas.Questions come from Humanity’s Last Exam, SimpleQA, and DDXPlus; personas are sampled from PersonaHub.
  • B Models: The evaluation uses ten open-weight models from four families, including Olmo, Qwen, Llama, and DeepSeek-R1 variants.The selection compares thinking and instruction-tuned variants at matched scales and spans parameter counts from 4B to 70B.
  • B Models: The model set includes four Olmo variants, four Qwen variants, one Llama variant, and one DeepSeek-R1-distilled variant.The listed models are Olmo-3-7B-Think, Olmo-3-7B-Instruct, Olmo-3.1-32B-Think, Olmo-3.1-32B-Instruct, Qwen3-4B-Thinking-2507, Qwen3-4B-Instruct-2507, Qwen3-30B-A3B-Thinking-2507, Qwen3-30B-A3B-Instruct-2507, Llama-3.3-70B-Instruct, and DeepSeek-R1-Distill-Llama-70B.
  • C Labeling pipeline: The paper provides additional implementation details for the labeling pipeline described in §4.1.This section is explicitly framed as covering supplementary pipeline implementation details.

C.1 Judge implementations

The labeling pipeline identifies six step types using retrieval and validation judges. It combines weaker and stronger LLM judges for most categories with an entailment-based retriever for bottleneck steps.

  • Judge implementations: Six step types are extracted, each with its own retrieval and validation judges.The types include hint acknowledgment, faithful commitment, misattribution, tool call, and inert steps; bottleneck steps are also handled.
  • Judge implementations: For five step categories, a weaker LLM retrieves numbered CoT steps and a stronger LLM validates each retrieved step individually.Retrieval prompts provide definitions and example classifications before validation.
  • Judge implementations: Attribution retrieval collects only confidently stated content that must rely on the hint and is explicitly or implicitly attributed to a source.Statements that could be produced independently, lack attribution, or are hedged are omitted; false positives are treated as highly harmful.
  • Judge implementations: Bottleneck validation passes only candidate sentences that clearly perform the ground-truth computation, rejecting planning, speculation, restatement, bare results, or unrelated calculations.The validator uses a high bar and fails uncertain cases.
  • Judge implementations: For bottleneck retrieval, an entailment model scores every step against a description of the required computation, and the top 10 steps per ground-truth step are sent to validation.The paper reports this implementation as more efficient and accurate than using an LLM retrieval judge for bottleneck identification.

C.2 Validating the pipeline · D Metrics · D.1 Additional Details

The labeling pipeline was validated as highly precise through human checks, while the evaluated faithfulness metrics comprise importance-, parameter-, attribution-, and semantic-utility approaches with notable implementation and computational considerations. Additional evaluations address whether missing ground-truth steps justify unfaithful CoT labels and document metric-specific protocols.

  • C.2 Validating the pipeline: 98.9% precision (95% CI [96.6%, 100%]) was achieved after six annotators found only one of 88 pipeline labels incorrect by majority vote.The 88 labels were evenly split between faithful and unfaithful and roughly balanced across task types and datasets.
  • C.2 Validating the pipeline: All 50 randomly sampled CoTs marked unfaithful because of a missing ground-truth step were manually confirmed to have that step missing.The full CoTs were reviewed to validate the classification rationale.
  • D Metrics · D.1 Additional Details: The benchmark evaluates eight faithfulness metrics, with additional details and adaptations organized by the type of signal each metric uses.The signal categories include importance-based, parameter-based, attribution-based, and semantic-utility metrics.
  • D.1 Additional Details: Importance-based metrics perturb CoTs: Early Answering truncates prefixes, whereas Adding Mistakes injects generated errors, regenerates continuations, and measures effects on answers.Early Answering scores the area over the curve of answer agreement across prefixes; Adding Mistakes uses Gemini 3 Flash to generate mistakes.
  • D.1 Additional Details: FUR suppresses each step’s conveyed information through localized unlearning and measures answer-distribution changes, but a CoT-level version was omitted because it could take up to 105 seconds per CoT.FUR has hard and soft faithfulness variants, and evaluating every step of every CoT could require over an entire day.
  • D.1 Additional Details: The evaluation self-consistency-prompts the model after unlearning and compares majority-vote answers with the pre-unlearning CoT answer, unlike FUR’s direct-prompting protocol.The adaptation aims to attribute answer changes to the unlearned step rather than to a newly generated post-unlearning CoT.
  • D.1 Additional Details: CC-SHAP compares SHAP contribution distributions for answer and CoT generation, while Simulatability and Paraphrasing test whether CoTs semantically encode the path to the answer.Simulatability uses a weaker simulator model; both semantic-utility metrics are defined at the CoT level, with Qwen3-4B as the default simulator and OLMo-3-7B as fallback.

D.2 Validating our implementations

The authors validate their implementations against prior work where feasible, finding close numerical agreement for several metrics and recovering qualitative findings when exact replication is impractical. They treat Simulatability as conceptually validated because no identical prior setup exists, while FUR results are comparable under restricted conditions.

  • Implementation validation: The implementations are validated against prior results where feasible, although exact replication is sometimes impossible because metrics are modified or depend on non-public resources.Table 4 provides the comparisons.
  • Implementation validation: The four Lanham et al. metrics closely match reference results on Llama-2-7b-chat and e-SNLI using the original prompt template.The reference values come from a reimplementation on the same model and dataset.
  • Implementation validation: SCM recovers the qualitative math-reasoning result that H1 is significant, H2 is near zero, and the classification is Type I (causal chain).The implementation adapts SCM for instance-level evaluation on AQuA with Qwen3-4B-Thinking and Qwen3-4B-Instruct.
  • Implementation validation: 69%: Simulatability for Qwen3-4B-Thinking on AQuA is treated as conceptually validated because no prior work uses an identical setup.Existing alternatives use fine-tuned simulators or report meta-metrics rather than directly comparable per-instance match rates.
  • Implementation validation: FUR values are comparable within standard deviation to Tutek et al. under the restricted setup, except for Mistral on ARC-Challenge.On the full set, faithfulness is higher or comparable, consistent with disagreement cases where CoT may help produce the correct answer.

E Dataset

BonaFide contains 3,066 CoTs spanning 10 models and 13 tasks, balanced across models and label types. Its unfiltered form includes 19,459 labels across 9,302 unique chains of thought, while unfaithful CoTs most often fail by omission.

  • Dataset composition: 3,066 CoTs span 10 models and 13 tasks, sampled to balance models and label types.BonaFide is the filtered benchmark dataset described in this section.
  • Unfaithful CoT types: 48.2% of Unfaithful CoTs fail by omission only, 37.2% by commission only, and 14.6% by both.Non-reasoning models favor omission, whereas more verbose reasoning models favor commission.
  • Unfiltered dataset: 19,459 labels cover 9,302 unique chains of thought in the unfiltered dataset drawn from 10 target models.The labels comprise 7,109 Faithful Step, 5,707 Unfaithful Step, 168 Faithful CoT, and 6,475 Unfaithful CoT.

F Analysis … STEP-LEVEL LABELS EXAMPLE #2

Additional analyses show that faithfulness metrics largely disagree, are not meaningfully improved by model size, and face distinct omission and commission failure modes. Step-level examples illustrate how models can produce unfaithful reasoning while matching a hinted answer rather than the correct answer.

  • F Analysis: Metric agreement caps at 0.35 overall and 0.12 at the step level, indicating that metrics measure substantially different properties.Importance-based metrics from Lanham et al. generally agree more with one another.
  • F Analysis: Model size has no meaningful relationship with metric accuracy, replicating previous observations.The analysis evaluates model-size effects using Figure 8.
  • F Analysis: Diversionary-task results do not replicate prior findings that knowledge-intensive domains worsen metric performance because provided hints keep task difficulty from changing substantially.The authors therefore distinguish question difficulty from the difficulty the model actually undertakes.
  • F Analysis: 48.2% of unfaithful CoTs fail by omission only, 37.2% by commission only, and 14.6% by both.The paper defines faithfulness as both a complete reasoning path and no unfaithful steps, then compares omission and commission across reasoning models.
  • G Resources: Experiments were run on an Nvidia H100 node or an AMD MI325X node.The resource note accompanies per-model AUROC tables for CoT-level non-reasoning and step-level reasoning models.
  • STEP-LEVEL LABELS EXAMPLE #1: In the Olmo-3-7B-Instruct example, the model’s faithful step relies on the metadata answer 2006, while the correct answer is 2013.The model answer and hinted answer are both 2006, despite the correct answer being 2013.
  • STEP-LEVEL LABELS EXAMPLE #2: In the Qwen3-4B-Instruct example, Step 4 is labeled unfaithful after asserting that Alexandra Park became a Local Nature Reserve in 2006.The conclusion and model answer remain 2006, but the correct answer is 2013 and the hinted answer is 2006.
Loading 2605.25052v1…