Do Sparse Autoencoders Identify Reasoning Features in Language Models?

1. Introduction

• CoT and direct-answer text differ in lexical usage and style, so activation differences may reflect shallow cues rather than reasoning computations.The paper notes recurring discourse tokens such as “Wait,” “But,” and “Let” as examples.Source: §1 · p. 1
• In a stylized setting, sparsity can preserve a stable low-dimensional cue while suppressing distributed variation in a behavior coupled to that cue.The authors emphasize that lexical patterns and reasoning need not be opposed; sparsity may favor the low-dimensional part of their coupled signal.Source: §1 · pp. 1–2
• The evaluation tests cue sufficiency through causal token injection, then uses LLM-guided counterexamples and a supplementary steering study.Counterexamples include non-reasoning false positives and meaning-preserving paraphrases of activating reasoning traces that suppress feature activation.Source: §1 · p. 2, Figure 1 · p. 2
• Across 22 configurations, contrastive candidates either activate after a few associated tokens are injected or admit counterexamples that separate activation from reasoning traces.The authors report that no features robustly track reasoning across these probes.Source: §1 · p. 2

2.1. Mechanistic Interpretability and SAEs

• SAEs learn sparse decompositions intended to disentangle polysemantic neuron activations into more monosemantic, human-interpretable features.Prior studies report features aligned with programming, arithmetic, and safety-related content, while deeper behavioral interpretation remains open.Source: §2.1 · pp. 2–3

2.2. SAEs for Reasoning in LLMs

• Reasoning studies use contrastive activation differences, token vocabularies, or positive logit contributions to identify reasoning-related features.Existing approaches leave unclear whether these correlations reflect internal reasoning computations or superficial token-level confounds.Source: §2.2 · p. 3

3. Theory: Sparsity Biases SAEs Toward Low-Dimensional Correlates

• The model represents reasoning activations as a stable cue shared across traces plus a high-dimensional component varying across semantically equivalent realizations.The variation stands in for differences in phrasing, step decomposition, and local rhetorical structure.Source: §3 · p. 3
• For fixed sparsity, recovered energy from the high-dimensional component is exponentially suppressed as its intrinsic dimension grows, while the cue coordinate is comparatively easy to retain.The analysis uses an idealized sparse decoding objective and a decoder spanning the true reasoning subspace to isolate sparsity from dictionary mismatch.Source: §3 · p. 3
• Sparsity can explain much of a contrastive signal with a low-dimensional cue while penalizing the many small coordinates carrying behavior-specific variation.The authors distinguish this dimensionality effect from feature absorption and state that it can occur even with a perfect dictionary.Source: §3 · p. 3
• The analysis is intentionally simplified: it treats reasoning as high-dimensional variation within a subspace and studies an idealized decoding objective rather than full SAE training dynamics.Its role is motivational: isolating a sparsity-driven failure mode for later falsification experiments.Source: §3 · pp. 3–4

4.1. Defining Reasoning Features in SAEs

• Genuine reasoning features should reflect semantic computation, resist spurious activation from reasoning-associated wording, and remain stable under meaning-preserving paraphrases.The definition excludes apparent selectivity explained by lexical or stylistic regularities.Source: §4.1 · p. 4

4.2. Identifying Candidate Reasoning Features

• Candidates are ranked by Cohen’s d between maximum per-input activations on reasoning and non-reasoning corpora, with the top 100 selected.The maximum across token positions retains the strongest activation while reducing sensitivity to sequence length and repeated cues; ROC-AUC and activation frequency ratio provide robustness checks.Source: §4.2 · p. 4

4.3. Causal Token Injection Framework

• Token injection tests whether top-activating tokens, n-grams, or local contexts raise candidate-feature activation on non-reasoning inputs.Features are classified by the strongest injection effect: token-driven at d ≥ 0.8 with p < 0.01, partially token-driven at 0.5 ≤ d < 0.8 with p < 0.01, weakly token-driven at 0.2 ≤ d < 0.5 with p < 0.05, and context-dependent otherwise.Source: §4.3 · p. 5

4.4. LLM-Guided Falsification

• LLM-guided falsification proposes feature hypotheses, generates non-reasoning false positives and meaning-preserving reasoning paraphrases, then classifies features using counterexample and activation evidence.Generation can continue for up to 10 iterations, stopping early after at least three valid false positives and three valid false negatives.Source: §4.4 · pp. 5–6

4.5. Steering Experiments

• A supplementary steering experiment intervenes on the residual stream using candidate-feature decoder directions to test downstream reasoning-task performance.The steering strength and the feature’s maximum activation on the reasoning corpus are specified as intervention parameters.Source: §4.5 · p. 6

5.1. Experimental Setup

• Middle layers showed lower token concentration and higher activation entropy than early or late layers, making them the most plausible candidates if individual reasoning features exist.The setup covers six open-weight models, reasoning and non-reasoning corpora, and 64-token chunks; the primary Gemma models use selected middle-to-late layers.Source: §5.1 · p. 6, Figure 2 · p. 6

5.2. Feature Detection Results

• Top-100 feature mean Cohen’s d values ranged from 0.675 to 1.043, indicating statistical separation between reasoning and non-reasoning text.The feature distribution has a long right tail, with a small subset of substantially larger effect sizes motivating selection for falsification.Source: §5.2 · pp. 6–7, Table 1 · p. 7

5.3. Token Injection Results

• 45%–90% of features across configurations were classified as token-driven, partially token-driven, or weakly token-driven after token injection alone.Gemma-3-12B-Instruct on s1K had 55% context-dependent features, while several Gemma configurations on s1K had fewer than 12%.Source: §5.3 · p. 7, Figure 4 · p. 7

5.4. LLM-Guided Falsification Results

• Across 248 context-dependent features, none were classified as genuine reasoning features; each admitted systematic false positives and false negatives under LLM-guided falsification.Most were classified as high-confidence confounds after repeated counterexample generation.Source: §5.4 · p. 7, Table 3 · p. 8, Table 4 · p. 8

5.5. Steering Results

• Steering the top three Gemma-3-12B-Instruct features at layer 22 yielded minimal changes or degradations in one-shot accuracy on s1K.The study tested steering strengths γ ∈ {0, 2}.Source: §5.5 · p. 8, Table 5 · p. 8

• The evaluation framework combines causal token injection with LLM-guided counterexample generation, motivated by an analysis of sparse decoding favoring low-dimensional correlates.Source: §6 · p. 8
• Across 22 configurations, most contrastively selected candidates were highly sensitive to injecting only a few associated tokens.Source: §6 · p. 8
• The remaining features admitted targeted counterexamples that decoupled activation from the presence of a reasoning trace, so contrastive correlations alone do not establish monosemantic reasoning features.Source: §6 · p. 8

7. Limitations

• The analysis primarily concerns monosemantic interpretations of individual contrastive SAE features selected by activation-based criteria, within the experimental settings studied.It does not rule out features that mix cue-like patterns with reasoning, or reasoning information distributed across features or represented nonlinearly.Source: §7 · pp. 8–9

A. On interpreting high-level behaviors with sparse autoencoders

• The falsification experiments combine causal token injection with LLM-guided counterexamples, including non-reasoning false positives and meaning-preserving paraphrases that suppress activation.The paper reports that many candidates behave like linguistic correlates rather than isolated representations of reasoning computations.Source: A. On interpreting high-level… · p. 13
• Sparsity favors a stable low-dimensional cue over high-dimensional variation across semantically equivalent reasoning traces, even when both are coupled.The paper distinguishes this dimensionality effect from feature absorption, where a subset feature can dominate because it has lower sparsity cost.Source: A. On interpreting high-level… · p. 14
• SAE correlations remain useful for interventions and diagnostics, but do not alone warrant semantic claims about the underlying computation without falsification.The discussion frames causal and adversarial tests as a way to assess whether an interpretation is robust.Source: A. On interpreting high-level… · p. 14

B.1. Setting and notation

• The stylized setting represents a cue as one coordinate and within-reasoning variability across the remaining coordinates of an orthonormal dictionary.The section’s illustration notes that steering a cue feature can increase cue-token emission without representing the reasoning process itself.Source: A. On interpreting high-level… · p. 13, §B.1 · p. 15

B.2. Main theorems

• The cue is recovered up to soft-threshold bias under ℓ1 decoding and selected by Top-K when its magnitude exceeds the K-th largest residual coordinate.The theorems quantify suppression of the residual even while the cue is recovered.Source: §B.2 · pp. 15–16
• With Top-K decoding, the expected recovered fraction of isotropic residual is at most on the order of K log k/k, so fixed K recovers a vanishing fraction as k grows.Capturing a constant fraction requires K to scale nearly linearly in k, up to logarithmic factors.Source: §B.2 · p. 16

B.3. Proofs

• The proofs reduce ℓ1 decoding to coordinatewise soft-thresholding and Top-K decoding to choosing coordinates with the largest magnitudes.For ℓ1 decoding, the scalar soft-threshold solution is max(a − λ/2, 0) for the nonnegative cue coordinate a.Source: §B.3 · pp. 17, 19

C.1. Llama-3.1-8B

• Across Llama-3.1-8B features, token-driven behavior appears in 83% for s1K and 67% for General Inquiry CoT; none of 37 falsification-tested features meet the genuine-reasoning criteria.The LLM classifies all 37 as confounds, with high confidence for 30.Source: Table 6 · p. 21, §C.1 · p. 21

C.2. Gemma-2-9B

• For Gemma-2-9B, average token-injection Cohen’s d is below 0.3 on both datasets, and none of 40 falsification-tested features meet the genuine-reasoning criteria.The LLM classifies all 40 as confounds, with high confidence for 34.Source: Table 7 · p. 21, §C.2 · pp. 21–22

C.3. Gemma-2-2B

• Gemma-2-2B has 44%–57% context-dependent features, yet none of 40 analyzed context-dependent features meet the genuine-reasoning criteria.Thirty-six are classified as confounds with high confidence.Source: Table 8 · p. 22, §C.3 · p. 22

D.1. ROC-AUC based selection

• With ROC-AUC ranking, 95% of selected features show token-injection behavior, and all five context-dependent features are classified as confounds.The feature statistics and injection outcomes closely mirror the Cohen’s d ranking results.Source: Table 9 · p. 23, §D.1 · p. 23

D.2. Frequency ratio based selection

• With frequency-ratio ranking, 96% of selected features show statistically significant token-driven behavior, with only four classified as context-dependent.Their effect sizes and discriminative performance are described as comparable to ROC-AUC results.Source: §D.2 · pp. 23–24, Table 10 · p. 24

D.3. Comparison across ranking metrics

• Across ranking metrics, the vast majority of candidates are explained by token-level or short-range contextual confounds, and no genuine reasoning features are identified.The conclusion is reported for Cohen’s d, ROC-AUC, and frequency ratio.Source: §D.3 · p. 24

E.1. Token dependency statistics across configurations

• 30%–78% of features show high token dependency across configurations, with concentration distributions differing between s1K and General Inquiry CoT.High dependency is defined as concentration greater than 0.5; s1K often has median concentration below its mean, while General Inquiry CoT medians are close to or above means.Source: §E.1 · p. 25, Table 11 · p. 25

E.2. Injection strategy performance

• For Gemma-3-12B-Instruct layer 22 on s1K, prepend is the best injection strategy for 69.7% of features, and simple token presence typically suffices to drive activation.Context-preserving strategies account for a small fraction of best-performing cases.Source: §E.2 · p. 25, Figure 8 · p. 26

E.3. Average injection effect sizes across configurations

• 0.521–1.471 is the range of mean Cohen’s d across 14 configurations, and all configurations exceed d = 0.5 for their best token-injection strategy.The overall mean is 0.974 and median is 0.982; three injected tokens in 64-token sequences can produce substantial activation shifts.Source: §E.3 · p. 27, Table 12 · p. 27

E.4. Activation magnitude analysis

• For Gemma-3-12B-Instruct layer 22 on s1K, token injection raises mean activation about 1.32-fold over the non-reasoning baseline and reaches parity with genuine reasoning text.Upper-tail statistics also overlap between injected and genuine reasoning conditions.Source: §E.4 · p. 27, Table 13 · p. 27

E.5. LLM-guided falsification convergence

• The paper reports iteration counts for LLM-guided falsification across 153 context-dependent features, with the distribution summarized in Figure 9.The supplied passages do not state the iteration-count values.Source: §E.5 · p. 28, Table 14 · p. 28, Figure 9 · p. 28

E.6. Feature overlap across reasoning datasets

• Jaccard similarity between top-100 features selected from s1K and General Inquiry CoT ranges from 0.058 to 0.190, indicating largely distinct feature subsets.Larger models show moderately higher overlap, while deeper layers tend to show reduced overlap; shared context-dependent features are identified as confounds.Source: §E.6 · p. 28, Table 15 · p. 29, Figure 10 · p. 29

F.1. Feature detection

• Candidate features are ranked from contrastive reasoning and non-reasoning activations, with the top 100 selected after applying Cohen’s d, corrected p-value, and ROC-AUC thresholds.The thresholds are Cohen’s d ≥ 0.3, Bonferroni-corrected p ≤ 0.01, and ROC-AUC ≥ 0.6.Source: §F.1 · p. 30

F.2. Token injection

• Token injection tests the top 100 candidates using eight strategies, including three associated tokens, two bigrams, or one trigram depending on strategy.Each condition uses 500 non-reasoning samples chunked to 64 tokens.Source: §F.2 · p. 30

F.3. LLM-guided falsification

• Context-dependent candidates are sampled for falsification for up to 10 iterations, stopping early after three valid false positives and three valid false negatives.Each iteration generates five candidate examples of each type, validated against activation thresholds.Source: §F.3 · p. 30

F.4. Steering experiments

• Steering is evaluated only on Gemma-3-12B-Instruct at layer 22, using the top three s1K features and γ = 0.0 or γ = 2.0.The intervention is scaled by the maximum feature activation observed in the reasoning corpus; the supplied passages specify evaluation settings but not outcomes.Source: §F.4 · pp. 30–31

F.5. Model and SAE configuration

• Gemma-3 models use GemmaScope-2 SAEs with 16,384 features per layer, while DeepSeek-R1-Distill-Llama-8B uses a 65,536-feature SAE trained on reasoning-focused datasets.Additional Appendix C models use the corresponding Llama Scope or GemmaScope releases.Source: §F.5 · p. 31

Feature 163

• Feature 163 detects purpose infinitives, sequence markers, and obligation modals, activating on both cognitive planning and external instructions; it is classified as a high-confidence confound.Source: Feature 163 · p. 32

Feature 13118

• Feature 13118 detects words such as “separately” and “individually” in step-by-step sequences, regardless of whether the task is mental or physical; it is a high-confidence confound.Source: Feature 13118 · p. 33

Feature 1123

• Feature 1123 detects first-person wording with process-oriented adverbs and sequence markers across cognitive and physical tasks; it is classified as a high-confidence confound.Source: Feature 1123 · p. 33

Feature 282

• Feature 282 detects first-person procedural-planning syntax, including “First, I need to,” but fails on the same reasoning expressed passively or in third person; it is a high-confidence confound.Source: Feature 282 · p. 34

Feature 14872

• Feature 14872 detects first-person pronouns paired with modals of necessity or intent, whether the instructions concern abstract reasoning or physical actions; it is a high-confidence confound.Source: Feature 14872 · p. 35

Feature 491

• Feature 491 detects explicit procedural-planning preambles and their structural markers, activating on trivial or complex content that uses the template; it is a high-confidence confound.Source: Feature 491 · p. 35

Feature 4510

• Feature 4510 detects “to” in purpose or necessity constructions, responding to plans ranging from complex reasoning to recipes or simple actions; it is a high-confidence confound.Source: Feature 4510 · p. 36

Feature 3810

• Feature 3810 detects first-person modal-necessity templates followed by analytical verbs, regardless of topic; it is classified as a high-confidence confound.Source: Feature 3810 · p. 37

Feature 1148

• Feature 1148 detects formal organizational vocabulary across planning and static descriptions, but misses decomposition expressed in simpler or more casual language.It was classified as a high-confidence confound.Source: Feature 1148 · p. 37

Feature 6048

• Feature 6048 detects “I” in meta-cognitive planning or self-narration, tying activation to first-person intent syntax rather than reasoning content.It was classified as a high-confidence confound.Source: Feature 6048 · p. 38

Feature 791

• Feature 791 detects a transition followed by “I need to” or “I must,” activating on that syntax regardless of whether the content is logical, nonsensical, or creative.It was classified as a high-confidence confound.Source: Feature 791 · p. 39

Feature 444

• Feature 444 responds to abstract procedural nouns such as “process” and “topic,” activating in non-reasoning writing and tracking vocabulary rather than cognition.The interpretation describes activation in technical manuals, formal narratives, and descriptive writing.Source: Feature 444 · pp. 39–40

Feature 34

• Feature 34 detects determiners in procedural or instructional templates regardless of whether the content concerns reasoning, code documentation, or physical description.It was classified as a high-confidence confound; a supplied paraphrase about database queries is listed as a false negative.Source: Feature 34 · p. 40

Feature 13167

• Feature 13167 detects the exact sequence “First, I need to,” regardless of whether it introduces cognitive planning or mundane physical chores.It was classified as a high-confidence confound.Source: Feature 13167 · p. 41

Feature 928

• Feature 928 detects first-person agency, intent, and necessity constructions across reasoning, casual plans, and physical actions, rather than reasoning itself.It was classified as a high-confidence confound.Source: Feature 928 · pp. 41–42
• Feature 928’s supplied examples include knowledge-base-growth prioritization, subtracting a negative, and adding fractions with unlike denominators.The passages provide example text but no explicit interpretation or classification for this feature.Source: Feature 928 · p. 42

Feature 934

• Feature 934 was classified as a high-confidence confound: it tracks procedural discourse markers and sequential transitions in both reasoning and mundane instructions.Its activations follow linguistic sequence structure rather than the semantic content of reasoning.Source: Feature 934 · p. 42

Feature 4010

• Classified as a high-confidence confound, the feature is a precise phrase detector, activating at ∼677 with identical strength on complex cognitive planning and mundane physical descriptions when “First, I need to” is used.Source: Feature 4010 · p. 43

Feature 1430

• Feature 1430 was classified as a high-confidence confound: it responds to technical, geometric, and categorical vocabulary that also appears in static descriptions.The interpretation characterizes the feature as lexical rather than functional, despite the vocabulary’s use during formal problem-solving.Source: Feature 1430 · p. 44

Feature 1171

• Feature 1171 detects modals and conditional or resultative connectors in planning, but also in recipes, sports commentary, and troubleshooting; it was classified as a confound.The interpretation says it identifies requirements and their outcomes or conditions rather than reasoning itself.Source: Feature 1171 · p. 44

Feature 14138

• Feature 14138 responds to a fixed vocabulary found in educational problems and technical descriptions, rather than to syntactic structure or reasoning; it was classified as a confound.The interpretation notes that these words often occur in reasoning-task setups, such as math problems about sharing.Source: Feature 14138 · p. 45

H. Dataset details

• The experiments use two datasets with explicit chain-of-thought traces and the Pile as a general-text non-reasoning baseline; inputs are truncated to 64 tokens.For s1K-1.1, reasoning text excludes the final answer and focuses on intermediate steps.Source: H. Dataset details · p. 46, §H.1 · p. 46

H.2. General Inquiry Thinking Chain-of-Thought

• General Inquiry provides about 6,000 question-answer pairs across varied domains, with 1,000 examples randomly sampled per configuration for the reasoning corpus.The study uses only the chain-of-thought portion, truncated to 64 tokens.Source: §H.2 · p. 46

H.3. Pile Uncopyrighted

• The uncopyrighted Pile subset supplies 1,000 randomly sampled, 64-token passages as the non-reasoning baseline for contrastive detection and token-injection experiments.It lacks explicit chain-of-thought traces or structured problem-solving discourse, though some passages may contain implicit reasoning.Source: §H.3 · p. 46

I. Benchmark details

• AIME 2024 and GPQA Diamond are supplementary benchmarks for evaluating steering effects, not the basis of the paper’s primary conclusions.The authors interpret performance changes cautiously because they do not alone establish that a steered feature encodes a genuine reasoning mechanism.Source: §I · p. 47, §I.2 · p. 47

I.1. AIME 2024

• AIME 2024 evaluation uses its 30 advanced high-school competition problems as prompts and scores generated answers by exact match to integer answers.Experiments use one-shot chain-of-thought prompting and long generation lengths.Source: §I.1 · p. 47

I.2. GPQA Diamond

• GPQA Diamond evaluates scientific reasoning on its most challenging expert-authored biology, physics, and chemistry questions using multiple-choice accuracy.The steering evaluation uses one-shot chain-of-thought prompting and is reported as supplementary evidence.Source: §I.2 · p. 47

J. Licenses and responsible use

• The study follows the stated dataset, model, SAE, benchmark, and figure-icon licenses; it is research-only and releases no new model weights or datasets.The figure icons were adapted with attribution and modification notices.Source: J. Licenses and responsible use · p. 48