PaperlayerView exact revision on arXiv

Source-linked AI summary

Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators

Armaan Singh, Ryan Trinh Le, Jasmine Kaur, Abdullah Sultan, Edward Lue Chee Lip, Kiran Nijjer, Adnan Ahmed, Vasu Sharma

arXiv:2608.29956v1cs.CL

TL;DR

The paper asks whether models perform multi-step reasoning internally when they answer without displaying intermediate steps. It introduces HCDS, a comparative behavioral and mechanistic assay, and finds positive neutral-prompt CoT-like alignment on GSM8K, especially in Thinking, while emphasizing that this does not prove an internal reasoning trace.

  • Problem

    The paper addresses whether models perform multi-step computation internally without visible reasoning traces, a question important because displayed reasoning may not faithfully reflect underlying computations.

  • Method

    HCDS compares neutral-prompt behavior with explicit CoT and explicit no-CoT poles using linguistic, behavioral, and mechanistic features.

  • Results

    HCDS is significantly positive for both Qwen3-4B variants on GSM8K, while Thinking scores higher than Instruct under a shared valid no-CoT baseline.

  • Takeaways & Limitations

    HCDS provides a framework for studying reasoning not explicitly exposed in model outputs without relying on self-reported traces.

  • Takeaways & Limitations

    The mechanistic feature has sparse, structurally undefined coverage, with non-zero values in only roughly 2–6% of all cells.

Abstract

from arXiv · show

Large language models often answer complex reasoning questions without revealing intermediate steps, raising whether they reason latently or complete patterns. We propose the Hidden CoT Detection Score (HCDS), a comparative behavioral and mechanistic signal measuring whether neutral-prompt behavior aligns more closely with explicit CoT or explicit no- CoT. Here, hidden CoT operationally denotes this neutral-prompt CoT-like alignment; HCDS does not directly observe or prove an unexposed reasoning trace. On GSM8K, HCDS is significantly positive for both Qwen3-4B variants (Thinking $+1.87$, $p = 1.2 \times 10^{-7}$; Instruct $+1.41$, $p = 1.9 \times 10^{-4}$), replicates across a different inference stack and quantization within $0.08$ ($+1.80$ and $+1.45$), and is not significantly positive in seven of eight length-adjusted calibration-control cells. The unadjusted score produces large positive scores on single-step arithmetic and numeric factual lookup. The variants also respond differently to no-CoT instructions: Instruct complies from the prompt alone, whereas Thinking continues reasoning and requires intervention. These findings show stronger, less prompt-conditional CoT-like behavior in the reasoning-tuned model, consistent with but not proof of latent reasoning. HCDS thus investigates latent reasoning without relying on models' self-reported traces.

1 Introduction

The paper asks whether models perform multi-step computation without visible reasoning and introduces HCDS to compare neutral behavior with explicit CoT and no-CoT behavior. Positive alignment is treated as evidence consistent with latent reasoning, not direct observation of an internal trace.

  • Motivation and approach: Explicit CoT can improve complex reasoning performance while its visible steps may not faithfully represent the computations producing answers.This motivates studying latent reasoning without relying on self-reported traces.
  • Motivation and approach: HCDS compares neutral-prompt behavior with explicit CoT and explicit no-CoT behavior using linguistic, behavioral, and mechanistic features.The six features are entropy, entropy slope, latency, paraphrase consistency, perturbation sensitivity, and attribution-based suppression cost.
  • Empirical findings: Qwen3-4B Thinking may ignore no-CoT instructions, so the primary comparison force-closes its reasoning block while adjusting features for generation length.In an un-intervened diagnostic, Thinking produced a mean of 575 output tokens under a no-reasoning instruction; force-closure reduced the primary comparison pole to a six-token median.
  • Empirical findings: On GSM8K, neutral behavior is significantly closer to explicit CoT than explicit no-CoT for both Qwen3-4B variants.The evaluation covers Qwen3-4B Instruct and Thinking, with StrategyQA reported separately because its answer format weakens the contrast.
  • Contribution: The framework combines behavioral alignment with mechanistic interventions to test whether identified internal computations influence outputs.Its interpretation remains comparative and does not directly establish an unexposed reasoning trace.

2 Related Work

Related work studies CoT faithfulness, shortcut learning, steerable internal features, paraphrase-based inference improvements, interaction length, and influential visible steps. This paper instead targets inference-time latent reasoning under neutral prompting.

  • Faithfulness and latent reasoning: Prior work shows that visible CoT explanations can be unfaithful and need not reveal the computations responsible for an answer.The paper therefore asks whether latent reasoning occurs, rather than whether an explicit rationale is faithful.
  • Faithfulness and latent reasoning: Other studies examine implicit reasoning from shortcut learning or steerable internal features, whereas this paper detects inference-time use under neutral prompting.The distinction is between reasoning emergence or steering and behavioral detection during inference.
  • Diagnostic features: Unlike PPCV, which maximizes paraphrastic agreement to improve answers, HCDS uses paraphrase consistency diagnostically and allows it to vary.The paper reports per-feature decomposition separately.
  • Diagnostic features: Related agent experiments show that longer interaction can plateau in performance, supporting the paper’s caution against treating response length alone as reasoning.The paper distinguishes length from deeper inference.
  • Mechanistic probing: The paper adapts intervention logic related to thought anchors, using anchor perturbation as one HCDS feature rather than as the end goal.Its target is CoT-like behavior under neutral prompting.

3 Method

HCDS operationalizes hidden CoT detection as comparative inference: it measures whether neutral behavior is closer to explicit CoT or no-CoT in a six-feature space. The method uses length-adjusted feature distances, force-closure for Thinking, and mechanistic interventions on candidate reasoning states.

  • HCDS construction: HCDS evaluates explicit CoT, explicit no-CoT, and neutral prompts, treating the explicit conditions as comparison poles.For Thinking, neutral and no-CoT share a force-closure intervention.
  • HCDS construction: The six-dimensional feature vector contains latency, mean entropy, entropy slope, paraphrase consistency, perturbation sensitivity, and mechanistic intervention sensitivity.The final feature measures interventions on candidate reasoning-related internal states.
  • Length adjustment: Primary HCDS first residualises raw features on log generation length, then z-scores residuals before computing Euclidean distances.Output length is used only for adjustment, diagnostics, and matching.
  • Robustness: All GSM8K cells remain significant across alternative missing-feature rescaling policies, with a largest policy difference of 0.40 HCDS.The pairwise policy omits the correction, while the complete policy uses one intersection across all three poles.
  • Interpretation: Positive HCDS means neutral behavior is closer to explicit CoT than explicit no-CoT, while the score does not establish causality.The adjusted score represents alignment not linearly explained by log generation length, but residualisation may remove reasoning-related variation.
  • Clean no-CoT pole: Force-closure is load-bearing for Thinking because its chat template opens a reasoning block automatically and asymmetric intervention creates formatting and verbosity differences.With symmetric force-closure, GSM8K exceeds arithmetic, reversing the problematic unadjusted ordering.
  • Mechanistic feature: Mechanistic sensitivity compares accuracy changes after ablating attribution-selected reasoning anchors with position-matched control interventions.A larger decrease after candidate suppression is evidence that the internal state causally contributes to the output.

4 Experimental Setup

The experiments compare two Qwen3-4B variants on GSM8K and StrategyQA under three prompt conditions, recording behavioral and mechanistic indicators alongside diagnostic accuracy and length measures. Deterministic, question-level analyses use bootstrap intervals, significance tests, and robustness checks.

  • Models and datasets: The study evaluates Qwen3-4B Instruct and Thinking on GSM8K multi-step arithmetic problems and StrategyQA implicit multi-hop commonsense questions.Both datasets use explicit CoT, explicit no-CoT, and neutral prompt conditions.
  • Inference conditions: Each model–dataset cell uses shared deterministic decoding settings across prompt conditions, including a common token cap within the cell.Caps differ across models and datasets.
  • Measurements: The experiment records accuracy, generation time, output length, latency, entropy, paraphrase consistency, perturbation sensitivity, and mechanistic sensitivity.Accuracy and output length are auxiliary diagnostics; the remaining six measurements form HCDS.
  • Behavioral features: Paraphrase consistency measures the fraction of meaning-preserving paraphrases that retain the original answer, while perturbation sensitivity uses a fixed prepended clause.Paraphrases preserve quantities, structure, named entities, and the gold answer.
  • Statistical analysis: Because decoding is deterministic, uncertainty reflects variation across questions rather than across generations.Question-level scores are analyzed with bootstrap confidence intervals and two-sided significance tests.
  • Robustness analyses: Robustness analyses include leave-one-feature-out, single-feature, length-matched, and open-weight mechanistic anchor-versus-control analyses.Additional implementation and decoding details are provided in the appendices.

5 Results

Length-adjusted HCDS remains positive and significant for both Qwen3-4B variants across independent inference stacks, while calibration controls largely collapse toward zero. Feature ablations and per-feature analyses indicate a distributed signal, with entropy contributing most strongly but not solely.

  • 5.1 HCDS detects CoT-like behavior on GSM8K: HCDS remains positive and highly significant for both Qwen3-4B variants on GSM8K after length adjustment.The result holds across two independent inference stacks.
  • 5.1 HCDS detects CoT-like behavior on GSM8K: Within 0.08, length-adjusted GSM8K scores agree across inference backends, compared with a 0.65 maximum difference before adjustment.
  • 5.2 Length adjustment reduces calibration-control scores: After length adjustment, control scores approach zero and are almost uniformly non-significant on single-step arithmetic and factual lookup.The unadjusted score instead produces large positive values in five of eight control cells, so it is treated only as a supplementary diagnostic.
  • 5.4 The result is robust to feature choice: All 24 leave-one-feature-out variants remain positive and significant, with the weakest cell at +1.193 (p = 1.6 × 10−3).Removing both entropy features causes the largest reduction, but significance remains across both models and backends.
  • 5.4 The result is robust to feature choice: Entropy is the strongest and most consistent contributor (d = 0.56–1.13), while paraphrase consistency is the strongest non-entropy feature.Latency contributes little after residualisation, and perturbation sensitivity is positive but non-significant in either model on this sample.
  • 5.5 Cross-task results: StrategyQA does not provide an independent replication: Instruct is non-significant on both backends, while Thinking remains positive and is driven mainly by entropy slope.Its binary answer format compresses accuracy-based features near the chance floor.

6 Discussion

Length adjustment reduces positive HCDS scores on calibration controls, supporting the interpretation that the main GSM8K effect is not explained by shared output-length dependence. Cross-backend replication further limits runtime-specific explanations, although residualisation cannot remove length dependence shared by both stacks.

  • Calibration controls: Length adjustment reduces positive scores on single-step arithmetic and factual lookup, where multi-step reasoning is unnecessary.These tasks serve as negative controls for reasoning-like behavior.
  • Model comparison: Thinking scores higher than Instruct on both backends when a valid no-CoT pole is created by force-closure.Prompt compliance makes the comparison easier to measure rather than implying lower latent reasoning.
  • Calibration controls: Five of eight control cells are significantly positive unadjusted, compared with one marginally positive length-adjusted cell.This pattern motivates treating the unadjusted score as vulnerable to verbosity-related artifacts.
  • Robustness: Backend replication checks runtime-specific artifacts but cannot eliminate a length dependence shared by both inference stacks.The marginal Table 4 exception neither replicates nor survives multiple-comparisons correction.

7 Limitations

The paper’s main limitations concern intervention-dependent baselines, narrow model-family coverage, and a preliminary mechanistic feature with sparse, structurally undefined measurements. Statistical uncertainty also reflects between-question variability while conditioning on preprocessing choices.

  • Intervention dependence: Thinking’s no-CoT pole is engineered through force-closure because the checkpoint lacks a promptable non-reasoning mode.Thus, Thinking results characterize behavior under intervention; the prefill is applied identically to both relevant conditions.
  • Generalisation: The main qualitative comparison uses one Qwen3-4B model family, limiting separation of Thinking behavior from its specific post-training recipe.An unadjusted Gemma-3-4B-it pilot is directionally consistent but cannot provide a matched, length-adjusted dual-variant replication.
  • Mechanistic measurement: The mechanistic feature is constrained by a single pinned layer/site, position-only controls, attribution saturation, answer-step eligibility, and sparse coverage.It is defined in only 41% of PyTorch/CUDA cells and 40% of MLX cells, with most defined values exactly zero.
  • Statistical reporting: Statistical uncertainty reflects between-question variability rather than sampling uncertainty about underlying model behavior.The bootstrap also conditions on fitted residualisation and standardisation rather than refitting preprocessing.

8 Conclusion

The paper presents HCDS as a comparative assay for neutral-prompt CoT-like alignment rather than direct evidence of hidden reasoning. Its conclusion emphasizes robust GSM8K positivity across variants and stacks, while identifying model-family generalisation and stronger mechanistic tests as open questions.

  • 8 Conclusion: HCDS combines linguistic, behavioral, and mechanistic indicators to compare neutral behavior with explicit CoT and no-CoT poles.Positive alignment is interpreted comparatively, not as direct observation of an internal reasoning trace.
  • 8 Conclusion: The neutral prompt omits reasoning and answer-format directives, whereas explicit prompts request reasoning or answer-only output formats.The evaluation conditions therefore differ in both reasoning-display instructions and, for explicit prompts, answer-format instruction.
  • 8 Conclusion: Paraphrases are generated by Instruct without exposing the gold answer, but reasoning leakage remains possible because rewrites can simplify relations.This makes paraphrase consistency a noisy behavioral feature rather than a leakage-free measure.
  • 8 Conclusion: Duplicate paraphrases occur for 138 of 200 questions (69%), reducing the effective K to one and making the feature ternary in practice.The observed values are exactly {0, 0.5, 1}.
  • 8 Conclusion: Removing eight byte-identical rewording cases raises HCDS in all four cells, so retaining the full n = 50 avoids post-hoc exclusion.The reported increases range from +0.009 to +0.128 across backend-model cells.
  • 8 Conclusion: The primary grid covers four task families, two models, and three prompts at n = 50, with a separate n = 500 output-length diagnostic.Features are z-scored within backend, model, and dataset before HCDS contrasts pole distances.
  • 8 Conclusion: The natural un-prefilled Thinking neutral-pole design inverts the unadjusted score, with trivial arithmetic outranking GSM8K.The paper therefore uses the symmetric force-closure design in the primary analysis.
  • 8 Conclusion: Natural no-CoT prompting suppresses Instruct outputs but not Thinking’s long-form generation, motivating the symmetric force-closure design for the primary comparison.The separate diagnostic and primary intervention answer different measurement needs.

F StrategyQA as a Secondary Benchmark

StrategyQA is treated as a secondary benchmark because its binary format compresses accuracy-based contrasts near chance, weakening HCDS as an instrument. After length adjustment, Instruct does not retain a positive signal, while Thinking remains less robust than on GSM8K.

  • 0.50 chance accuracy on StrategyQA compresses the usable range of accuracy-based features, unlike GSM8K where chance is effectively zero.
  • Thinking remains positive on StrategyQA but is carried almost entirely by entropy slope, with every other feature contributing |∆| < 0.21.
  • After length adjustment, Instruct StrategyQA HCDS does not survive on either backend.
  • Thinking survives in 52.4% of CUDA and 57.1% of MLX feature subsets, versus 84.1%–90.5% for GSM8K cells.
  • On GSM8K, HCDS is positive in all six output-length tiers and significant in five; Instruct short is the exception at +0.465, p = 0.23.

H Mechanistic Anchor Methodology — Full Details

The mechanistic feature identifies high-attribution trace steps, suppresses them and matched controls, and compares resulting accuracy effects within the HCDS pipeline. Robustness analyses show the GSM8K conclusion is broadly preserved across feature subsets, weighting choices, and inference stacks, while StrategyQA mechanistic coverage is limited.

  • Anchor selection: Gradient×activation attribution toward the answer span selects candidate reasoning anchors from segmented generated-token steps.Attributions are aggregated over each step’s generated token positions.
  • Mechanistic feature: ∆Amech = Ic − (I1 + I2)/2 compares matched-control correctness with the average correctness after two anchor ablations.CUDA uses separate anchor and control redecodings; MLX uses a joint comparison with coarser support {−1, 0, +1}.
  • Intervention: Residual_zero ablation multiplies the residual-stream output by 0 at selected step-token positions before greedy decoding continues.Other token positions and layers remain unchanged.
  • Control selection: Controls match anchors by trace position with step length as a weak tie-break, but not by step function.The absence of function matching is an explicit limitation.
  • Weighting robustness: 95.2%–98.4% of GSM8K feature subsets are positive and 84.1%–90.5% significantly positive across models and backends.Negative subsets are degenerate latency/mechanistic combinations without behavioral or linguistic features.
  • Backend robustness: The MLX 8-bit feature-ablation pattern matches PyTorch/CUDA bf16: removing latency raises HCDS, whereas removing both entropy features produces the largest reduction.Error bars represent 95% bootstrap confidence intervals.
  • Continuous weighting: Instruct remains positive in > 98% of sampled continuous weightings and significantly positive in > 90%, while Thinking remains positive in > 94% and significantly positive in > 84%.Weights are sampled from a uniform Dirichlet distribution over the feature simplex rather than optimized.

K Multiple-Comparisons Correction

The paper corrects its many significance tests by claim-specific families. The principal GSM8K findings and robustness ablations survive correction, while corrected non-significance is explicitly not treated as equivalence to zero.

  • Primary family: Six of seven uncorrected significant cells survive Holm correction, including all four headline GSM8K cells.The largest adjusted p-value among the headline GSM8K cells is 2.5×10^-3.
  • Primary family: Under Holm correction, no length-adjusted negative-control cell is significantly positive.This controls the reported false-positive rate but does not establish equivalence to zero.
  • Robustness families: All 24 leave-one-feature-out ablations survive Holm at α = 0.05, with largest adjusted p = 4.0 × 10^-3.
  • Robustness families: Holm correction reduces the GSM8K significant-subset fraction from 84.1%–90.5% raw to 65.1%–84.1%, while Benjamini–Hochberg leaves it at 84.1%–90.5%.Correction changes no point estimate, so the sign-based weighting claim is unaffected.
  • Scope: Multiplicity corrections address reported test families, but deterministic one-trial-per-question testing still has question identity—not model stochasticity—as its unit of variation.No p-value adjustment changes what that test is a test of.

L Correlation-Aware (Mahalanobis) HCDS

A Mahalanobis variant tests whether correlated HCDS features distort the Euclidean score. GSM8K results remain significant and close to the published metric, while decorrelation removes positive negative-control findings.

  • Motivation: Six features derived from the same generated response may be correlated, so Euclidean distance can double-count shared information.
  • Metric: Mahalanobis distance replaces Euclidean distance using a shrinkage covariance matrix Sλ = (1 − λ)S + λI.Because features are z-scored, S is their correlation matrix; λ = 1 recovers the published Euclidean score.
  • GSM8K results: GSM8K remains significant in all four Mahalanobis cells, with Ledoit-Wolf changes of at most 0.09.Fully unregularised whitening reduces published values by at most 0.13.
  • Controls: Under Mahalanobis distance, no negative-control cell is significantly positive.The marginal CUDA Instruct arithmetic control crossing the threshold is treated only as a consistency check.
  • Metric comparison: Length residualisation leaves mean feature correlations of 0.105 for Thinking and 0.158 for Instruct on CUDA GSM8K.The largest surviving correlation is r = 0.54 between perturbation and mechanistic sensitivity for CUDA Instruct GSM8K.
  • Reporting choice: The Euclidean score remains primary because it is simpler, requires no covariance estimate from 50 questions, and is closely reproduced by the correlation-aware variant.
Loading 2608.29956v1…