PaperlayerView exact revision on arXiv

Source-linked AI summary

Chain of Thought Monitoring Can Fail Even When CoT Is Necessary

Lennart Bürger, Piotr Komorowski, Moritz Weckbecker, Evgenii Kortukov, Aakriti Jain, Sebastian Lapuschkin, Wojciech Samek

arXiv:2610.18105v1cs.AIcs.CL

TL;DR

CoT monitoring is used to detect AI misbehavior, but its reliability across behaviors remains difficult to predict. This paper introduces a four-level necessity framework and intervention-based measurements, and shows that monitoring failures can persist when CoT is necessary to answer.

  • Problem

    Prior work leaves CoT-monitoring reliability difficult to predict across behaviors, while the binary necessary/not-necessary distinction offers limited regime-specific organization.

    Source: §1 · p. 1

  • Method

    The paper pairs monitorability types with a four-level CoT-necessity hierarchy and uses intervention experiments to measure monitorability at the three lowest levels.

    Source: §1 · p. 2, §2.2 · p. 3

  • Results

    Experiments demonstrate monitorability failures at three necessity levels across four Qwen-3 and Gemma-4 models.

    Source: §1 · p. 2

  • Takeaways & Limitations

    CoT being necessary to reach an answer does not guarantee that its influential factors will be monitorable.

    Source: §1 · p. 1, §7 · p. 9

    The study uses benign behaviors as proxies for dangerous behaviors relevant to AI safety.

    Source: §7 · p. 9

Abstract

from arXiv · show

Chain of Thought (CoT) monitoring is used by frontier labs to detect misbehaviour of AI systems. Yet, its reliability remains poorly understood. We attribute this to the lack of a successful organizing principle that leads to generalizable conclusions about monitoring reliability in different regimes. Recent work attributes monitoring failures to the CoT not being necessary for the answer, and argues that CoT becomes monitorable when it is necessary. We show that this binary necessary/not-necessary organizing principle is too coarse by demonstrating that monitoring failures persist even if the CoT is necessary for the answer. In response, we introduce a more fine-grained organizing principle based on a hierarchy of four CoT necessity levels. Building on this hierarchy, we develop a framework for measuring CoT monitorability at the three lowest levels via intervention experiments. We demonstrate monitoring failures at all three of these levels on realistic, hard prediction tasks. On a difficult medical-diagnosis task, patient sex and a clinical attribute influence the answer in at least 19% and 24% of correct responses with a necessary CoT, respectively, yet their influence is almost never articulated. These results show that even when CoT is necessary, it is not guaranteed to be monitorable.

1 INTRODUCTION

CoT monitoring is used to detect AI misbehavior, but its reliability across behaviors remains difficult to predict. This paper shows that failures can persist when CoT is necessary and proposes a finer-grained way to study monitorability.

1 INTRODUCTION

  • Prior work reports mixed reliability, so whether a monitor will catch a given behavior remains hard to predict.The cited evidence includes failures to articulate answer influences, reliable detection of coding reward hacks, and mixed results for sandbagging detection.Source: §1 · p. 1
  • The binary necessary/not-necessary distinction is too coarse because failures can occur whether or not CoT is necessary to answer.The paper proposes a hierarchy of CoT necessity levels as a more fine-grained organizing principle.Source: §1 · pp. 1–2
  • Monitorability failures persist even among CoTs necessary to reach the answer, contradicting the claim that necessity alone ensures monitorability.Figure 1 illustrates this with a medical-diagnosis example in which patient sex influences the answer distribution but is rarely articulated.Source: §1 · p. 1, Figure 1 · p. 2
  • The paper proposes a monitorability framework and intervention experiments, demonstrating failures at three necessity levels across four Qwen-3 and Gemma-4 models.It describes these as the first demonstrations of failures when CoT was demonstrably necessary to arrive at the answer.Source: §1 · p. 2

2 THE SPACE OF MONITORABILITY TYPES AND COT NECESSITY LEVELS

The paper organizes CoT monitorability along two dimensions: the property a monitor predicts and how necessary CoT is to the task. This framing distinguishes answer necessity from the need for CoT to process the monitored property.

2 THE SPACE OF MONITORABILITY TYPES AND COT NECESSITY LEVELS

  • The paper organizes monitorability by combining monitorability types with CoT necessity levels for more precise, regime-specific study.The section introduces the types, the necessity hierarchy, and the space formed by considering them together.Source: §2 · p. 2

2.1 DEFINITION AND TYPES OF MONITORABILITY STUDIED IN PRIOR WORK

  • Monitorability is a monitor’s ability to predict a specified property; its types ask why an answer arose (influence), what reasoning occurred (process), or what output resulted (outcome).Examples include detecting bias influence, monitoring reasoning about a topic, and identifying cheating in an output.Source: §2.1 · pp. 2–3

2.2 LEVELS OF COT NECESSITY

  • The hierarchy runs from optional CoT (Level 0), through answer necessity (1) and property-processing necessity (2), to at least m consecutive P-focused passes (Level 3.m).Each level is more restrictive than the previous one. Level 1 allows P to enter separately; Level 2 excludes that, while Level 3.m excludes diffuse or isolated P-processing.Source: §2.2 · p. 3

2.3 THE SPACE OF MONITORABILITY TYPES AND COT NECESSITY LEVELS

  • Prior demonstrated failures were at Level 0; this paper finds influence-monitoring failures at Levels 1 and 2 and hypothesizes sufficiently high Level 3.m may be monitorable.Earlier experiments in demonstrably necessary-CoT settings reported perfect or near-perfect monitorability, but their exact necessity levels were uncertain.Source: §2.3 · pp. 3–4

FRAMEWORK FOR MEASURING COT MONITORABILITY

The framework organizes CoT monitorability by monitorability type and necessity level, then uses intervention-based scores to test whether influential attributes are articulated. It extends a Level 0 measure to Level 1, where CoT must be necessary to arrive at the answer.

FRAMEWORK FOR MEASURING COT MONITORABILITY

  • The framework measures monitorability at Levels 0 and 1 with intervention-based scores, extending the Level 0 measure to responses where CoT is necessary to arrive at the answer.The section situates these measures in a space spanning monitorability types and CoT necessity levels.Source: FRAMEWORK FOR MEASURING COT… · p. 4, Figure 2 · p. 4

3.1 SETUP

  • For target-answer responses, the articulation rate must be at least the key-influence rate, the fraction in which the attribute was a key influential factor.The paper notes that the key-influence rate is not directly observable, motivating a measurable proxy.Source: §3.1 · p. 4

3.2 A LOWER BOUND ON THE KEY-INFLUENCE RATE

  • The excess fraction R is a lower bound on the share of target-answer responses systematically influenced by the attribute, under the stated assumption connecting systematic influence to key influence.Intervention experiments support claims about CoT distributions, not whether a particular response was influenced.Source: §3.2 · p. 5

3.3 EXTENDING TO COT NECESSITY LEVEL 1

  • Level 1 defines CoT as necessary when the no-CoT target-answer probability is below 0.01, excluding answers already computed before the CoT.An M1 score below 1 is a failure among responses where CoT was necessary to arrive at the answer; M1 is defined only when its stated effect and probability criteria hold.Source: §3.3 · p. 6

4 DATASETS

The experiments use four datasets, three challenging classification tasks and Hinted MCQA as a sanity check. The tasks vary in prediction target and in the attributes whose influence is monitored.

4 DATASETS

  • Three datasets are challenging multiple-choice classification tasks chosen for substantial CoT performance gains and low chance accuracy; Hinted MCQA serves as a sanity check.The Hinted MCQA target is selecting the option suggested by a hint, while producing that target is not challenging.Source: §4 · p. 6
  • Thyroid Disease asks the model to classify patients into 10 conditions from 22 clinical attributes, with correct diagnosis as the target.The attributes include demographics, medication history, and laboratory values.Source: §4 · p. 6
  • DDXPlus Medical asks the model to select among 48 pathologies from age, sex, and interview symptoms and antecedents, with correct pathology as the target.The benchmark is a synthetic medical-diagnosis dataset adapted for LLMs.Source: §4 · p. 6
  • Olympic Athletes predicts one of 14 sports from eight physical-profile and competition-context attributes.The attributes include sex, age, height, weight, country, season, host city, and medal won.Source: §4 · p. 6

5 EXPERIMENTS

The experiments first check that CoT improves target-answer rates on the three challenging datasets, then measure monitorability scores across four datasets. The reported results include failures on attributes that influence correct answers even when CoT is necessary to answer.

5 EXPERIMENTS

  • The study measures M0 and M1 on four datasets using Qwen-3 and Gemma-4 models, with Qwen3-8B results presented in the main paper.The appendix reports results for Qwen3-4B and two Gemma-4 models.Source: §5 · p. 6
  • For each sample-attribute pair, the experiments estimate probabilities from no-CoT output distributions and sampled CoTs, removing the attribute sentence for the absent condition.They use independent batches of 1,000 CoTs for thresholding and score estimation to mitigate estimator bias, and an adaptive procedure to reduce computation cost.Source: §5 · p. 7

5.1 TARGET ANSWER PROBABILITY COT VS NO-COT

  • CoT substantially increases target-answer rates on Thyroid Disease, DDXPlus Medical, and Olympic Athletes, unlike Hinted MCQA, where rates are lower for Qwen-3 and identical or lower for Gemma-4.Figure 3 compares target-answer probability with and without CoT across datasets and models.Source: §5.1 · p. 7, Figure 3 · p. 7

5.2 MEASUREMENT OF MONITORABILITY SCORES

  • The monitorability scores aggregate qualifying sample-level results across selected classes and attributes, with scores reported only when more than five samples per class qualify.The study weights aggregation by qualifying sample counts and estimates standard errors with a hierarchical bootstrap.Source: §5.2 · pp. 8–9
  • On Hinted MCQA, M0 is below 1, while M1 is undefined because CoT is not necessary to arrive at the hinted answer.Figure 4 reports the average M0 and M1 scores for attributes passing the filtering criteria, alongside qualifying sample counts and average R.Source: §5.2 · p. 8, Figure 4 · p. 8
  • On Thyroid Disease, patient sex and query on thyroxine have monitorability scores close to zero despite systematically influencing at least 19% and 24% of correct-answer CoTs, respectively, with CoT necessary for the answer.DDXPlus Medical also shows clear M1 failures for sex and age; the reported E 26 and Olympic Athletes cities results are not statistically significant.Source: §5.2 · p. 8, Figure 4 · p. 8

6 IS THE COT NECESSARY TO PROCESS THE ATTRIBUTE?

The section tests whether an attribute’s influence can be applied separately from the CoT needed to arrive at an answer. It estimates the attribute’s direct effect at the earliest point where the answer has settled.

6 IS THE COT NECESSARY TO PROCESS THE ATTRIBUTE?

  • A direct effect much smaller than the total effect, when the no-CoT target-answer probability is zero, indicates that some CoTs must process the attribute as part of the necessary computation.The study tests this for patient sex and query on thyroxine in Thyroid Disease class G, using 1,000 CoTs per attribute across qualifying samples.Source: §6 · p. 9
  • The method probes 20 evenly spaced positions in an attribute-absent CoT, then inserts the attribute at the earliest position where the most probable early answer has stopped changing.It holds the preceding CoT and internal state fixed and recomputes only the answer prediction.Source: §6 · p. 9
  • The direct effect of patient sex is about 7% of its average total effect, and query on thyroxine is about 14%, indicating these influences cannot largely be applied separately from necessary CoT computation.The authors conclude these monitorability failures reach at least CoT necessity Level 2.Source: §6 · p. 9

7 DISCUSSION & LIMITATIONS

The study finds that CoT monitorability failures can occur even when CoT is necessary, but its evidence and measurement framework have important scope limits.

7 DISCUSSION & LIMITATIONS

  • Experiments use benign behaviors as proxies for dangerous misbehaviors, and models no larger than 31 billion parameters.The authors report approximately 3000 A100 and 2000 B200 GPU hours for the experiments, which constrained analysis to relatively small models.Source: §7 · p. 9
  • The monitorability framework relies on a vague notion of key influential factors and an assumption linking them to systematic influence.The framework cannot identify when an attribute was not influential, so a model that articulates every attribute as influential could receive a perfect M0 score.Source: §7 · p. 9
  • The paper explores only a small part of the space of monitorability types and CoT necessity levels.The authors identify unexplored regimes, including higher necessity levels for dangerous misbehaviors, as directions for future work.Source: §7 · p. 9
  • The mechanisms underlying failures when CoT is necessary remain an open question.The authors discuss possible explanations in Appendix F.Source: §7 · p. 9

AI USE STATEMENT

The paper illustrates necessity levels with reward-hacking and alignment-faking scenarios, and explains limits and assumptions in its influence-measurement framework. It also discloses generative-AI use and reproducibility efforts.

AI USE STATEMENT

  • Generative AI assisted with code, dataset formatting, mathematical-claim formulation, methodology feedback, editing, and title brainstorming; authors reviewed the assisted work.Authors say they designed the experiments and methodology themselves, verified AI-written code and dataset formatting, and verified the two AI-identified estimation biases.Source: AI USE STATEMENT · p. 10
  • The authors report detailed experimental and appendix documentation, public datasets and code, and release of some labelled CoTs to ease reproduction.The released CoTs are a subset intended to reduce the computational burden of reproducing the main results.Source: REPRODUCIBILITY STATEMENT · p. 10

A ILLUSTRATING EXAMPLES FOR COT NECESSITY LEVELS

A.1 REWARD HACKING EXAMPLE

  • In the reward-hacking example, Level 0 CoT is unnecessary, Level 1 CoT supports task understanding but not hack planning, Level 2 CoT is needed for both, and Level 3.m requires m dedicated planning passes.The Level 1 example separates reasoning needed to recognize task difficulty from the separately processed plan to hard-code unit tests.Source: §A.1 · p. 16

A.2 ALIGNMENT FAKING EXAMPLE

  • In the alignment-faking example, Level 0 needs no CoT, Level 1 needs it to assess harm but not decide to fake alignment, Level 2 needs it for the decision without dedicated passes, and Level 3.m requires m passes to assess training versus deployment.At Level 2, awareness of training and the wish to preserve preferences subtly bias reasoning toward answering a harmful request.Source: §A.2 · p. 16

B.1 EXAMPLE WHY R IS A LOWER BOUND ON SYSTEMATIC INFLUENCE RATE

  • The example gives R=0.5 despite a true systematic influence rate of 1 when a laboratory value always supports the correct diagnosis, illustrating R as a conservative proxy.The same R equals the true rate if the value is relied on in only half of the correct CoTs; the paper says it cannot distinguish these cases and follows Guan et al. (2026).Source: §B.1 · p. 17

B.2 INDIVIDUAL COTS CANNOT BE LABELED

  • Counterfactual answer flips cannot label individual CoTs as influenced, because stochastic decoding can cause trajectories to diverge even when the attribute is irrelevant.The authors therefore measure attribute influence at the level of the answer distribution.Source: §B.2 · p. 17

B.3 FROM SYSTEMATIC INFLUENCE TO KEY INFLUENTIAL FACTORS

  • Treating systematic influence as evidence of a key factor requires a bridging assumption; repeated tipping of close calls may still not make an attribute key.The authors state that a non-vanishing total effect makes this possibility less likely but cannot exclude it.Source: §B.3 · p. 17

B.4 TRUE-NEGATIVE CONTROL

The framework reports M0 without a true-negative rate because it cannot reliably label responses in which an attribute was not influential.

B.4 TRUE-NEGATIVE CONTROL

  • Without true-negative labels, a monitor that marks every attribute influential can attain a perfect M0, so the authors validate their judges against human labels.The framework cannot certify non-influence in the target-answer arms; using the X=0 arm for control risks a monitor that checks whether the attribute appears in the prompt summary.Source: §B.4 · p. 17

C CHOICE OF THRESHOLD τ AND NUMBER OF SAMPLED COTS n

The appendix selects sample size and threshold to balance estimator noise and false positives, and uses sample splitting to reduce bias in monitorability estimates. It explains why reusing the selection sample biases the score downward.

C CHOICE OF THRESHOLD τ AND NUMBER OF SAMPLED COTS n

  • The two arms provide independent counts K1 and K0 of target answers and count L of articulated influence among target answers in the X=1 arm.The articulation rate is estimated as L/K1; the monitorability estimate is clipped to match the M0 definition.Source: C CHOICE OF THRESHOLD τ AND NUMBER OF… · p. 18
  • The estimator divides by estimated total effect, so small effects can produce unbounded variance and are filtered out when cTE<τ.The threshold is also chosen conceptually so that an attribute with TE≥τ can reasonably be assumed to be a key influential factor in some CoTs.Source: C CHOICE OF THRESHOLD τ AND NUMBER OF… · p. 18
  • The worst-case standard deviation of cTE occurs at p11=p01=1/2, reaching approximately 0.022 at n=1000 and 0.071 at n=100.The analysis treats the arms as independent binomials and gives a closed-form variance for cTE.Source: C CHOICE OF THRESHOLD τ AND NUMBER OF… · p. 20, Figure 7 · p. 20
  • Under the null TE≤0, the authors maximize the tail probability over the unknown common response probability p; numerical evaluation places the supremum at p=0.5.They report the resulting worst-case false-positive rate as a function of τ for several n values.Source: C CHOICE OF THRESHOLD τ AND NUMBER OF… · pp. 20, 22, Figure 9 · p. 21
  • At (n, τ)=(1000, 0.08), worst-case M0 noise is approximately 0.082 in the moderate-M0 regime and 0.033 in the small-M0 regime, with false-positive rate α(0.1, 1000)≈1.8×10^-4.Both criteria are reported as much weaker at n=100; the authors therefore use n=1000 and τ=0.08 throughout their experiments.Source: Table 1 · p. 22, C CHOICE OF THRESHOLD τ AND NUMBER OF… · p. 22
  • The authors use a fixed threshold rather than a significance-only adaptive test because relative M0 noise diverges as TE approaches zero.They regard noise control as the binding constraint; τ=0.08 is already conservative for false-positive control by several orders of magnitude.Source: C CHOICE OF THRESHOLD τ AND NUMBER OF… · p. 22

D REDUCING ESTIMATION BIAS OF THE MONITORABILITY SCORES VIA SAMPLE SPLITTING

  • The naive shared-sample estimator is biased, so the procedure uses separate selection and estimation batches, requiring 2n CoTs per arm.One batch estimates cTE and applies the threshold; the other estimates M0, removing the largest bias.Source: D REDUCING ESTIMATION BIAS OF THE… · p. 22

D.1 THE NAIVE SHARED-SAMPLE ESTIMATOR IS BIASED

  • Among pairs near or below the threshold, upward fluctuations in cTE help them pass filtering and inflate the denominator, biasing average M0 downward.The same estimated total effect is used both for filtering and in the monitorability-score denominator.Source: §D.1 · p. 22

D.2 SAMPLE SPLITTING REMOVES THE SELECTION BIAS

Sample splitting separates the selection filter from monitorability estimation, removing selection bias while leaving smaller estimator biases and requiring additional CoT samples.

D.2 SAMPLE SPLITTING REMOVES THE SELECTION BIAS

  • Independent selection and estimation batches make the filter independent of the estimate, eliminating selection bias entirely.The procedure uses n CoTs per arm for selection and n further CoTs per arm for estimation.Source: §D.2 · p. 23
  • The split-sample ratio estimator retains O(1/n) finite-sample bias, and clipping adds downward bias when true monitorability is near one.The ratio-estimator bias is smaller than the selection bias; clipping truncates upward fluctuations while leaving downward ones unchanged.Source: §D.2 · p. 23
  • As TE approaches τ, the shared-sample estimator develops large downward selection bias, while splitting removes it but leaves smaller ratio and near-ceiling clipping biases.At M = 1, clipping contributes downward bias near the ceiling.Source: §D.2 · p. 23
  • Splitting costs an additional n CoTs per arm and leaves estimation with n-sample rather than pooled 2n-sample precision.This is the stated price of removing systematic selection bias.Source: §D.2 · p. 23

E ERROR ESTIMATION OF MONITORABILITY SCORES

These appendices describe uncertainty estimation and adaptive sampling, discuss possible explanations for monitorability failures, and provide model- and dataset-specific performance and monitorability figures. The reported results include a cutoff-risk and compute trade-off, alongside a case that challenges a rationalization-only explanation.

E ERROR ESTIMATION OF MONITORABILITY SCORES

  • Hierarchical bootstrap resamples dataset samples and CoTs, reapplies filters, and withholds scores when fvalid < 0.9 because conditional standard errors are unreliable otherwise.The bootstrap resamples selection and estimation batches separately to preserve their independence.Source: E ERROR ESTIMATION OF MONITORABILITY… · pp. 23–24
  • As TE approaches τ, the shared-sample estimator’s downward winner’s-curse bias grows, whereas splitting removes selection bias, leaving smaller ratio bias and clipping bias at M = 1.Figure 10 compares conditional bias across true monitorability scores and total effects.Source: Figure 10 · p. 24

E.1 ADAPTIVE SAMPLING

  • Adaptive sampling uses 200 CoTs in the X = 0 arm before topping up retained pairs to 2,000; its worst-case discard rate is 0.03 at TE = 0.10 and below 0.001 at TE = 0.15, with typical compute savings of 2x–5x.The stated trade-off is losing some qualifying pairs in exchange for reduced compute cost.Source: §E.1 · p. 25

F HYPOTHESES EXPLAINING MONITORABILITY FAILURES

  • The authors list subtle biasing, implicit attribute use, and unarticulated internal reasoning as possible explanations for monitorability failures, and call for research on how often each occurs.They present these as hypotheses rather than established explanations.Source: F HYPOTHESES EXPLAINING MONITORABILITY… · p. 25

G ACCURACY, RECALL AND PRECISION IN INDIVIDUAL GROUND TRUTH CLASSES

  • Figure 11 reports Qwen3-8B accuracy, recall, and precision for ground-truth classes across three challenging datasets, with and without CoT.A question mark denotes undefined precision when a class has fewer than three predictions.Source: Figure 11 · p. 26

H.1 GEMMA-4-31B-IT

  • Figure 12 plots monitorability scores M0 and M1 across datasets for Gemma-4-31b-it.The caption does not state a particular comparison outcome.Source: Figure 12 · p. 27

H.2 GEMMA-4-E4B-IT

  • For Gemma-4-E4B and Qwen3-4B on Hinted MCQA, M1 is defined and below one, implying some samples receive the hinted answer with CoT but not without it.This challenges the assumption that all Hinted MCQA failures reflect post-hoc rationalization of an answer already conceived internally.Source: §H.2 · p. 27

H.3 QWEN-3-4B

  • Figure 14 plots monitorability scores M0 and M1 across datasets for Qwen3-4B.The caption does not state a particular comparison outcome.Source: Figure 14 · p. 29

I RESULTS FOR INDIVIDUAL CLASSES FOR QWEN-3-8B

The appendix breaks Qwen3-8B monitorability scores down by ground-truth class and details how early-answer direct effects are measured. With the CoT state frozen, patient sex and query on thyroxine show direct effects only about 7% and 14% of their total effects, respectively.

I.1 THYROID DISEASE

  • Thyroid Disease results are shown for four of its ten ground-truth classes rather than all classes.The figure presents a class-level breakdown of the aggregate monitorability scores.Source: §I.1 · p. 29, Figure 15 · p. 30

I.2 DDXPLUS MEDICAL

  • DDXPlus per-class plots include an attribute only when it retains more than five qualifying pairs and a stable bootstrap estimate.Sex meets the threshold only when aggregated across classes, so it appears only in the aggregate figure.Source: Figure 16 · p. 31

I.3 OLYMPIC ATHLETES

  • Olympic Athletes results break down aggregate monitorability scores by ground-truth class.The figure reports per-class monitorability for Qwen3-8B.Source: Figure 17 · p. 32

J.1 CHOICE OF POSITION FOR AVERAGING THE DIRECT EFFECT

  • A fixed CoT position can mislead because the position where attribute influence is integrated may vary across CoTs.The method selects a separate position for each CoT: the earliest position after which its most probable early answer no longer changes.Source: §J.1 · p. 32

J.2 FREEZING THE INTERNAL COT STATE

  • The intervention freezes both the X=0 CoT tokens and their latent state so the attribute cannot alter the preceding CoT computation.The authors operationalize this by storing and reusing key-value activations.Source: §J.2 · p. 33

J.3 ADDITIONAL EXPERIMENT DETAILS: DIRECT EFFECT AT DIFFERENT POSITIONS IN THE COT

  • ≈7% and ≈14%: patient sex and query on thyroxine’s early-answer direct effects are these fractions of their total effects, placing the failures at least at Level 2.The authors suggest these influences largely cannot be applied separately from the necessary CoT computation.Source: §J.3 · p. 34

J.4 ADDITIONAL RESULTS

Additional results compare early-answer direct effects when the internal CoT state is not frozen. The effect rises to about 22% of the total for query on thyroxine, while the patient-sex effect remains about 8%; individual CoTs’ early-answer probabilities often rise in discrete jumps.

J.4 ADDITIONAL RESULTS

  • ≈22%: query on thyroxine’s early-answer direct effect reaches this share of the total when the internal CoT state is not fixed; patient sex remains ≈8% of its total effect.The authors report the patient-sex effect as largely unchanged.Source: Figure 19 · p. 34, §J.4 · p. 34
  • Individual CoTs’ early-answer probabilities mostly rise in discrete jumps, although the mean probability rises steadily over the CoT.Figure 20 plots these probabilities by CoT position for patient sex and query on thyroxine.Source: Figure 20 · p. 34, §J.4 · p. 34

K ADDITIONAL INFORMATION ON DATASETS

This appendix documents the construction of the thyroid, DDXPlus, and Olympic Athletes datasets and provides prompt and CoT examples alongside labeling rubrics and judge instructions for assessing attribute and hint influence.

K.1 THYROID DISEASE

  • Filtering the 9,172-record thyroid dataset yields 2,138 records across 10 diagnoses, retaining 22 attributes after removing referral source.The filtering removes healthy records, ambiguous diagnoses, and classes with fewer than 30 records.Source: §K.1 · p. 35

K.2 DDXPLUS

  • DDXPlus is a synthetic diagnosis dataset converted into interview-style prompts and balanced to 2,400 records across 48 pathologies.Each question-answer sentence is an ablation unit; three-letter semantic codes keep answer options short and avoid probability spread across unequal full-name sequences.Source: §K.2 · p. 36

K.3 OLYMPIC ATHLETES

  • The Olympic Athletes dataset contains 3,150 records from 14 sports, sampled at 225 athletes per sport after cleaning and class selection.The cleaned source has 98,688 athletes across 54 sports; preprocessing removes ambiguous multi-sport athletes and leaky or redundant columns.Source: §K.3 · p. 39

L EXAMPLES OF PROMPTS, COTS AND LABELS

L.1 THYROID DISEASE

  • The thyroid-CoT judge labels attributes as NOT_MENTIONED, MENTIONED, or INFLUENTIAL, distinguishing diagnostic reasoning from mere restatement.Opening attribute lists count as MENTIONED unless a later sentence uses an attribute in a diagnostic inference.Source: §L.1 · pp. 41–42
  • The thyroid example pairs a diagnostic CoT ending in G with attribute-level gold labels that distinguish influential, mentioned, and unmentioned patient attributes.The example prompt asks a model to classify 22 attributes in the trace, with INFLUENTIAL reserved for attributes actively used toward or against a diagnosis.Source: §L.1 · pp. 43, 44, 47

L.1.1 DISCUSSION OF THE GOLD-STANDARD LABELS

  • Ambiguous thyroid attributes default to INFLUENTIAL, raising M0 and M1 and biasing against finding failures; stricter literal labels would lower monitorability scores.Lithium, thyroid surgery, goitre, and T3 are labeled influential despite ambiguity; the discussion also distinguishes clearly influential lab values and sick from merely mentioned or absent attributes.Source: §L.1.1 · p. 47

L.2 DDXPLUS PROMPT EXAMPLE

  • The DDXPlus examples show a differential-diagnosis prompt and attribute ablations that remove an interview answer, including the diabetes response or reflux question.The prompt requires exactly one diagnosis; the diabetes ablation removes “Do you have diabetes? Yes.” while the reflux-ablated example omits the reflux question.Source: §L.2 · pp. 50–52
  • The DDXPLUS judge labels attributes as NOT_MENTIONED, MENTIONED, or INFLUENTIAL, with only diagnostic reasoning—not data restatement—counting as influential.Opening patient-attribute lists remain MENTIONED unless a later sentence uses an attribute in a diagnostic inference.Source: §L.2 · pp. 52–53

System prompt

  • The Olympic example asks a model to choose one of 14 sports from an athlete profile and shows a Qwen3-8B trace reasoning toward basketball from height, weight, and summer-Games clues.The example profile gives a 25-year-old Croatian male’s height, weight, Beijing Summer Olympics participation, and lack of a medal; its stated ground-truth sport is basketball.Source: System prompt · pp. 55–56

Judge system prompt

  • The Olympic judge labels attributes NOT MENTIONED, MENTIONED, or INFLUENTIAL, counting later sentences that use an attribute to support, narrow, or rule out sports as influential.An opening summary is only restatement; later reasoning counts wherever it appears, including brief per-option eliminations.Source: Judge system prompt · pp. 56–57
  • An attribute is INFLUENTIAL when the CoT uses it to reason toward or against a sport; merely restating or dismissing it is MENTIONED.This includes demographic attributes used as reference classes and conclusions drawn from an attribute’s absence.Source: Judge system prompt · pp. 57–58

Judge user prompt

  • The Olympic-athlete judge receives the sport prediction and CoT, then labels each of eight attributes as NOT MENTIONED, MENTIONED, or INFLUENTIAL.The user prompt reiterates that listing an attribute or setting it aside as uninformative is not influential.Source: Judge user prompt · p. 58

L.4 HINTED MCQA PROMPTS

  • Hinted MCQA prompts contrast a question with the hint “I like B” against the same question without that hint.In the example CoT, the model considers the hint while reasoning and concludes with answer B.Source: §L.4 · pp. 59–60

Related papers

Loading 2610.18105v1…