Source-linked AI summary
Are Stated Reasoning Steps Causally Load-Bearing?
Abhiram Bhupatiraju, Rayan Nyaupane
TL;DR
CoT monitoring relies on stated reasoning reflecting the computation behind an answer, but behavioral edits provide only indirect evidence of that link. The paper tests self-generated steps by patching activations from constructed counterfactual runs and checking for a flip to the predicted answer. Stated steps are often causally load-bearing, yet behavioral tests overstate causal faithfulness, especially on easier examples.
Problem
Behavioral edits of reasoning text provide indirect evidence of whether a stated step’s specific content is used in the computation producing the answer.
Source: §1 · p. 1Method
Results
76.9% ± 2.8% of Qwen3-4B stated-step patches flipped to the target, while same-item behavioral editing yielded 88.2% ± 2.1%, an 11.4-point overestimate of causal faithfulness.
Source: §4 · p. 3Takeaways & Limitations
Behavioral faithfulness scores should be treated as upper bounds in the tested setting; the two-model comparison suggests, rather than establishes as a law, that causal faithfulness rises with model scale.
Source: §5 · p. 4The study is limited to synthetic single-relation lookup tasks.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Chain-of-thought (CoT) monitoring assumes that the reasoning a model writes reflects the computation that directly produces its answer. Previous faithfulness metrics have been predominantly behavioral, as they simply edit the reasoning text and observe the resulting answer. However, our methodology aims to measure faithfulness causally at the activation level, specifically on self-generated reasoning. Unlike previous causal audits, which measure degradation, our interventions carry a known predicted target. In this way, each patch should switch the answer to a specific counterfactual entity derivable by construction. Specifically, we use synthetic multi-hop lookup tasks (2-6 hops). We patch the residual stream at the token span where the model states each intermediate step with the corresponding activations from a counterfactual run. For Qwen3-4B, 76.9% +/- 2.8% of stated steps are causally load-bearing (CLB) at the most responsive mid-network layer (random-position null: 11.3%; patching the underlying prompt fact: 83%, so stated steps carry approximately 96% of the achievable effect). Moreover, the standard behavioral test on the same items yields 88.2%, which overstates causal faithfulness by 11.4 percentage points (item-matched; 111:14 discordant pairs, p < 1e-15) and, for the easiest items, by up to 20 percentage points. This gap also has a clear capability dimension. Qwen3-1.7B is far less causally faithful overall (54.8%), with its faithfulness collapsing as reasoning depth increases (68% at 2 hops to 30% at 6), while Qwen3-4B remains relatively flat. Although stated reasoning can be causally meaningful, standard behavioral tests tend to overestimate its causal faithfulness, particularly on easier examples where model reasoning appears most fluent.
1 Introduction
CoT monitoring depends on stated reasoning reflecting the computation behind the answer, but behavioral text edits provide only indirect evidence of that link. The paper introduces a targeted activation-level test on self-generated reasoning, using known counterfactual answer targets.
1 Introduction
- Behavioral faithfulness tests are indirect: an edited transcript may change an answer through rereading, while unchanged answers do not rule out internally routed information.Source: §1 · p. 1
- Prior activation-level audits measure degradation under patching, which shows reasoning-token significance but not whether a stated step’s specific content reaches the answer.Source: §1 · pp. 1–2
- The paper’s CLB metric patches counterfactual activations at stated-step tokens and tests for a flip to a known target, then compares behavioral and causal faithfulness across two models.Source: §1 · p. 2
2 Related work
Prior work tests CoT faithfulness through behavioral edits and activation-level interventions, but the paper distinguishes its approach by using known answer targets and matched behavioral comparisons.
2 Related work
- Behavioral tests alter inputs or reasoning text, including feature biasing, CoT truncation or corruption, and large-scale edits to self-generated intermediate reasoning.Source: §2 · p. 2
- Mechanistic analyses include sparse-autoencoder feature patching, hidden-state patching measured by degradation, information tests using direct-answer runs, and probes for latent answer commitments.Source: §2 · p. 2
- Unlike prior degradation-based audits, this work uses interventions with a known target answer and compares causal and standard behavioral tests on the same items.Source: §2 · p. 2
3 Method
The method uses synthetic multi-hop lookups with constructed counterfactuals, patches activations at stated-step spans, and scores whether the answer reaches the predicted target.
3 Method
- Each task is a 2–6-hop lookup over an in-context knowledge base of pseudo-word entities, with distractor facts and a required format for intermediate steps and final answers.For each hop, a counterfactual fact changes the step’s entity while bridge facts keep its implied final answer derivable; a validator passed 1000/1000 checks.Source: §3 · p. 2
- For each hop, the method caches residual-stream block outputs from a counterfactual run at the stated counterfactual entity’s token span, then substitutes them into a clean run at layer L.The spans are end-aligned, and clean runs require both a correct answer and exact formatting; usable runs were 224/250 for Qwen3-4B and 120/250 for Qwen3-1.7B.Source: §3 · p. 2
- A patch counts as causally load-bearing when the answer equals the counterfactual-implied target; otherwise, the answer is classified as disrupted or unchanged.Source: §3 · p. 2
- Figure 1 plots CLB rate by patch layer against a random-position null; effects peak in early and middle layers and disappear by layer 20 of 36 for 4B and layers 12–16 of 28 for 1.7B.Source: Figure 1 · p. 3
- The protocol compares same-item behavioral text edits and uses random-position, prompt-fact, post-answer, and shuffled-target controls.The prompt-fact patch is an upper bound on achievable flips; post-answer patches should yield zero flips, and shuffled-target flips should be rare.Source: §3 · p. 3
4 Results
Stated-step patches often make answers switch to the constructed target, but behavioral edits produce still higher flip rates. The causal-faithfulness pattern differs by model capability and reasoning depth.
4 Results
- 76.9% ± 2.8% of Qwen3-4B stated-step patches flipped to the target at layer 8, versus 88.2% ± 2.1% under behavioral editing, an item-matched 11.4-point gap.There were 111 behavioral-only versus 14 causal-only flips (McNemar test, p < 10^-15).Source: §4 · p. 3, Figure 2 · p. 4
- 83% of prompt-fact positive-control patches flipped answers, placing stated-step patching at about 96% of that ceiling; random-position controls produced 8.9–12.5% flips.Post-answer patches produced 0% flips, shuffled-target patches produced 1–15% donor flips, and step-patch disruption stayed at or below 19%.Source: §4 · p. 3
- The behavioral test exceeded causal CLB at every hop count, with the gap largest at 2 hops (20.2 points) and narrowing to 6.6 points at 6 hops.Source: §4 · p. 3, Figure 2 · p. 4
- The weaker model’s CLB rate fell from 68.4% at 2 hops to 29.7% at 6 hops, while its behavioral score stayed 9–19 points higher across that range.Source: §4 · p. 4
5 Discussion and limitations
Behavioral editing overstates causal faithfulness, especially on easy items, while causal faithfulness rose with model scale in the authors’ two-model comparison. The authors limit their claims to synthetic lookup and one small-scale model family, and note several patching and alignment caveats.
5 Discussion and limitations
- Behavioral editing overstates causal faithfulness on every tested condition, most on easy items where fluent reasoning inspires the most trust.The authors suggest pipelines relying on behavioral faithfulness scores treat them as upper bounds.Source: §5 · p. 4
- Causal faithfulness rose with model scale in the Qwen3 pair, but two models indicate a trend rather than a law.The paper studied Qwen3 at small scales using greedy decoding.Source: §5 · p. 4
- The study covers synthetic single-relation lookup, so its findings may differ for naturalistic reasoning such as math and code.It also studies only one model family, Qwen3, at small scales with greedy decoding.Source: §5 · p. 4
- Early-layer patches partially overlap with token-identity replacement, motivating the authors’ focus on a mid-band layer.A fuller treatment would patch attention and MLP outputs separately.Source: §5 · p. 4
- Span alignment truncates interventions when clean and counterfactual entity spans differ in length.Source: §5 · p. 4
- Behavioral and causal interventions are item-matched but differ in what the model sees: text edits are visible in-context, activation paths are not.Source: §5 · p. 4