How and Why LLMs Generalize: A Fine-Grained Analysis of LLM Reasoning from Cognitive Behaviors to Low-Level Patterns

• Reasoning is framed as the composition of atomic skills whose intermediate failures can be hidden by correct final answers or superficial pattern matching.The proposed skills include calculation, simulation, fact retrieval, enumeration, and diagnostic self-checking.Source: §1 · p. 1
• Prior mixed-domain benchmarks, narrow controlled datasets, and conventional accuracy metrics do not reveal which intermediate skills drive success or how those skills shift.The paper positions its benchmark as a response to these limitations.Source: §1 · pp. 1–2
• The benchmark decomposes reasoning across mathematics, scientific reasoning, coding, and non-reasoning tasks, while probing distributional divergence and word-frequency shifts during post-training.This design tracks how individual skills and statistical tendencies evolve rather than relying only on end-to-end accuracy.Source: §1 · p. 2
• RL-tuned models preserve balanced calculation, enumeration, simulation, fact retrieval, and diagnostic skills across in-domain and out-of-domain settings, while SFT models show narrow gains and weakened transfer.The contrast is reflected in rounded RL profiles versus jagged SFT profiles.Source: §1 · p. 2
• The paper attributes the SFT–RL divergence primarily to training objectives rather than the amount of parameter modification.Its broader contribution is a behavioral account of why post-training strategies produce different generalization effects.Source: §1 · p. 2

• SFT replicates explicit reasoning traces and can narrow behavioral diversity, whereas RL rewards accurate, coherent reasoning without explicit intermediate supervision but may introduce reward hacking and biases.Source: §2 · p. 2
• Generalization studies show that reasoning-oriented fine-tuning can improve target-task performance while weakening robustness to new formats or domains.The paper addresses this trade-off by analyzing calculation, simulation, enumeration, fact retrieval, and diagnostic skills.Source: §2 · p. 3
• The framework operationalizes five measurable cognitive behaviors with targeted prompts and metrics, extending related benchmark and psychometric work beyond aggregate scores.Source: §2 · p. 3
• The paper jointly examines outward behavior and internal representations because chain-of-thought traces may be unfaithful to internal computations.Sparse autoencoders and activation steering are cited as tools for exposing reasoning-related subspaces.Source: §2 · p. 3

3 Decompose “Reasoning” into Fine-grained Cognitive Behaviors

• The benchmark crosses five cognitive behaviors with four domains, enabling controlled analysis of how post-training redistributes effort across sub-skills and statistical patterns.The study collects model-family checkpoints across multiple SFT and RL stages.Source: §3 · p. 3

3.1 Benchmark Construction

• Each benchmark item targets one atomic behavior and is curated for skill isolation, coverage, diversity, and calibrated difficulty.Source: §3.1 · pp. 3, 5
• The three-stage pipeline designs behavior–domain seeds, retrieves embedding-based variants from public repositories, and manually verifies isolation, coverage, diversity, and difficulty.This process is intended to decouple reasoning behaviors from surface task formats.Source: Table 1 · p. 4, §2 · p. 5, §3.1 · p. 5

2. Stage

• Fact Retrieval uses guided and unguided theorem-use checks, while Diagnostic requires correct answers and explicit identification of self-check.Source: §2 · p. 5

3.2 Behavior–Domain Grid

• The grid spans mathematics, scientific reasoning, coding, and non-reasoning QA, using representative tasks such as prime-factor queries, capacitor charge, dynamical rollouts, theorem recall, and flawed-proof correction.Source: §3.2 · p. 5
• Calculation, enumeration, simulation, fact retrieval, and diagnostic capture quantitative manipulation, combinatorial generation, dynamical rollout, stored knowledge access, and error correction.Source: §3.2 · p. 5

4.1 Cognitive-skill Profiles

• RL profiles remain rounded across five cognitive behaviors, whereas SFT produces jagged specialization with spikes and below-baseline skills.In Math-4B, SFT’s diagnostic score is about threefold above baseline while enumeration and simulation fall below baseline; RL retains smoother profiles.Source: §4.1 · p. 6

Long CoT rebalances SFT toward systematic

• Long-CoT SFT produces more integrated skill profiles than no-think SFT, especially on out-of-domain Physics.Long-CoT training reduces calculation dominance and strengthens simulation, diagnostic checking, and enumeration relative to short-CoT training.Source: Long CoT rebalances SFT toward… · pp. 6–7, Figure 5 · p. 7

4.2 Cross-Domain Generalization

• Both methods transfer weakly from math to non-reasoning, but RL degrades less and preserves a smoother skill mix than SFT.SFT shows spikes and below-baseline drops outside math, whereas RL retains broader competencies despite limited absolute gains.Source: Figure 4 · p. 6, §4.2 · p. 7

4.3 Linking Behaviors to Sparse Latent Features

• RL modestly broadens skewness–kurtosis dispersion while preserving the depth-dependent shift from near-Gaussian shallow layers to heavier-tailed deeper layers.SFT induces milder sharpening, while RL adds heterogeneity without creating a qualitatively different regime.Source: §4.3 · p. 7, Figure 6 · p. 7
• At layer 16, post-training reallocates SAE capacity toward fact retrieval and diagnostic features while leaving calculation and simulation comparatively stable.In the base model, calculation and simulation strengthen with depth, while fact retrieval and diagnostic follow U-shaped trajectories.Source: §4.3 · p. 7, Figure 7 · p. 8

4.4 Low-Level Statistical Shifts and Surface Patterns

• Roughly 98% of parameters change in every variant, yet behavioral differences are attributed to training objectives rather than update scope.Embeddings and transformer layers account for most shifts; normalization changes less, and RL’s total shift magnitude is comparable to SFT.Source: §4.4 · p. 8, Table 2 · p. 8

5 Summary and implications for future study

• The benchmark decomposes reasoning into five core behaviors across mathematics, scientific reasoning, coding, and non-reasoning QA.The pipeline is designed for reproducible and interpretable evaluation of post-training effects.Source: §5 · p. 8
• RL-tuned models preserve more balanced cognitive skills and broader generalization, while SFT often improves narrow skills and weakens others.The reported contrast includes diagnostic gains alongside simulation degradation under SFT.Source: §5 · p. 8
• Future reasoning-oriented LLMs should evaluate and train for balanced skill development rather than aggregate accuracy alone.Suggested directions include behavior-aware objectives, curricula for transferable skills, and representation-level monitoring.Source: §5 · p. 8

A.1 Setup for Base, SFT, and RL Model Training and Evaluation

• The experiments use Qwen3-14B, 4B, and 1.7B base and post-trained variants, including intermediate checkpoints for training-dynamics analysis.Released 14B checkpoints are supplemented by replicated 4B and newly trained 1.7B variants.Source: §A.1 · p. 12
• RL uses GRPO with answer-correctness rewards, while SFT learns from teacher-generated CoT traces on 47K curated math problems.The two procedures are matched to a comparable training horizon but use distinct optimization objectives.Source: §A.1 · p. 12
• Evaluation holds decoding settings fixed, caps reasoning generations at 32k new tokens, and reads only the final boxed answer.The harness records prompts, responses, seeds, and scores for auditability.Source: §A.1 · p. 12
• Primary evaluation uses fixed-decoding accuracy alongside per-behavior profiles, normalized skill indices, and intermediate-feature distributional drift.These measures support comparisons of both performance and behavioral or representational change.Source: §A.1 · p. 12

A.2 Setup for SAE Training and Interpretation

• SAEs are trained per transformer layer on MLP outputs using reconstruction and sparsity penalties, with activations collected from balanced behavior-labeled prompts.The setup uses hook_mlp_out activations, context length 2048, and stratified sampling across behaviors and difficulties.Source: §A.2 · p. 12
• SAE features are aligned to calculation, enumeration, simulation, fact retrieval, and diagnostic behaviors through explanation scoring, linear probes, mutual information, and manual auditing.Features are counted as related when the soft score is at least 0.5, with confidence-weighted scores also reported.Source: §A.2 · p. 13

A.3 Additional Details

• The analysis treats SAE interpretations as informative but non-definitive because they are compute-heavy and sensitive to hookpoint and sparsity choices.The paper identifies lightweight methods connecting low-level statistics to hidden representations as a next step.Source: §A.3 · p. 13

B Cognitive Behaviors: Definitions, Keywords, and Indicators

• The benchmark defines calculation, enumeration, simulation, fact retrieval, and diagnostic as distinct information-processing behaviors with separate keywords and observable indicators.Simulation tracks state transitions, enumeration emphasizes exhaustive coverage, and diagnostic includes self-checking and error correction.Source: B Cognitive Behaviors: Definitions,… · p. 14

B.1 Template and prompt for the dataset construction

• Candidate examples are retrieved by embedding similarity from behavior-domain seeds, deduplicated by a similarity threshold, and used to construct diagnostic tasks requiring correction of extraneous solutions.The diagnostic example detects the invalid solution introduced by squaring and retains x = 5 after checking the original equation.Source: §B.1 · p. 14, §B.2 · p. 15

B.3 Benchmark Scale and Extensibility

• The benchmark contains 5 × 4 behavior-domain cells, each with 50 to 200 curated examples vetted for correctness, skill isolation, and calibrated difficulty.Templates can be expanded by renaming variables, rescaling parameters, swapping units or entities, and controlled paraphrasing without changing the targeted skill.Source: §B.3 · p. 15

C Datasets Statistics

• The datasets balance subcategories across mathematics, physics, coding, and non-reasoning tasks while grouping examples by calculation, enumeration, fact retrieval, simulation, and diagnostic behaviors.Coverage includes mathematical topics, physical processes, algorithmic challenges, writing, summarization, error detection, retrieval, and scenario simulations.Source: C Datasets Statistics · p. 15

D Detailed Description of the Meta-Evaluation Benchmark

• The benchmark systematically evaluates 20 problem types formed by crossing four domains with five cognitive behaviors.The domains are mathematics, scientific reasoning, coding, and non-reasoning, while the behaviors are calculation, enumeration, simulation, fact retrieval, and diagnostic checking.Source: D Detailed Description of the… · p. 15, §D.1 · p. 15, §D.2 · p. 15

D.3 Problem Design Principles

• The problem templates are self-contained, objectively verifiable, scalable in difficulty, domain-representative, and minimally ambiguous.These design principles reduce evaluation uncertainty while supporting controlled variation from simple to complex instances.Source: §D.3 · p. 16

D.4 Cross-Domain Patterns

• Across domains, calculation shares procedural structure, enumeration requires exhaustive search, simulation tracks sequential progression, fact retrieval varies with knowledge base, and diagnostic converges on meta-cognitive skills.The taxonomy distinguishes shared behavioral structure from domain-specific operations and knowledge requirements.Source: §D.4 · p. 16

D.5 Use Cases

• The benchmark supports model comparison by cognitive dimension, targeted capability improvement, evaluation-method assessment, and robustness testing across generated instances.It can reveal asymmetric profiles such as strength in calculation paired with weakness in enumeration.Source: §D.5 · pp. 16–17

E Additional Math Examples

• Mathematics templates cover symbolic calculation, combinatorial enumeration, stepwise simulation, canonical fact retrieval, and diagnostic correction with scalable numeric instantiations.Diagnostic templates include theorem misuse, domain mistakes, inconsistent traces, and answer-format mismatches.Source: E Additional Math Examples · p. 17

F Training Dynamic

• All five skills improve under 1.7B RL, with fast early gains that settle into a stable plateau and tight variability bands.The training dynamics indicate stable progress without regressions across the measured skills.Source: F Training Dynamic · p. 17
• In math, simulation is strongest, while diagnostic and calculation improve clearly and enumeration and fact retrieval make smaller persistent gains.In physics, calculation leads throughout, fact retrieval strengthens, simulation reaches a moderate level, and enumeration and diagnostic advance more gradually.Source: F Training Dynamic · p. 17, Figure 8 · p. 19

Math Diagnostic Example

• The diagnostic example requires checking domain conditions, rejecting an invalid candidate, and retaining only the solution that satisfies the original equation.The example marks diagnostic behavior through a re-check that detects a consistency issue and revises the solution.Source: Math Diagnostic Example · p. 17
• The benchmark organizes math, physics, code, and non-reasoning problems by calculation, enumeration, fact retrieval, simulation, and diagnostic facets.Its figures describe broad coverage of domain-specific and real-world task subcategories for evaluating reasoning and non-reasoning abilities.Source: Figure 9 · p. 19, Figure 10 · p. 19, Figure 11 · p. 20