On the Token Value Inequality in Efficient Reasoning

1 Introduction

• CoT improves accuracy on multi-step reasoning tasks, but enabling it on Qwen-4B adds 24.1% in accuracy at a cost of 4.5× more tokens.The introduction frames potentially redundant exploratory reasoning as an opportunity to reduce token costs while preserving core content.Source: §1 · p. 1
• Token-level log probability is available at each decoding step and reflects local confidence dynamics without requiring extra reward models or human annotation.The paper seeks an inference-native, fine-grained signal that avoids post-processing overhead and extra forward passes.Source: §1 · p. 2
• NormLP identifies tokens above or below a response’s mean confidence, while TokenProbe aggregates the positive-NormLP token ratio as a scalar.The paper presents these signals as instance-aware and relatively stable to hyperparameter variations for a given model.Source: §1 · p. 2
• The paper uses two empirical findings about token value and TokenProbe to motivate efficient GRPO with selective KL anchoring and reward shaping.Its introduction describes experiments across benchmarks and model families as showing a trend toward Pareto improvement while maintaining accuracy and reducing token use.Source: §1 · p. 2
• Figure 2 reports that TokenProbe rises from think to answer stages and tracks higher accuracy and lower token use across models and samples.The figure caption characterizes these patterns as an indicator of reasoning efficiency.Source: Figure 2 · p. 3

Analysis setup

• NormLP normalizes each token’s log probability within its response, and TokenProbe is the ratio of tokens with positive NormLP.The analysis defines reasoning quality as achieving higher accuracy under a relatively low but reasonable token budget, not merely producing shorter reasoning.Source: §2 · p. 3
• The analysis covers five models across AIME, MATH, AMC, and Olympiad-Bench under varied hyperparameter settings.It organizes its probing from coarse to fine and presents two central empirical findings.Source: §2 · p. 3

Finding 1: Token Value Inequality

• Answer-phase TokenProbe is systematically 3%–20% higher than think-phase TokenProbe, while the think phase includes both genuine reasoning and exploratory noise.The paper characterizes thinking as higher-cost search and answering as lower-cost decision output.Source: §2 · p. 3

Finding 2: TokenProbe as a Core-Token Proxy

• Across four representative models, higher TokenProbe accompanies rising accuracy and falling token use; from the lowest to highest quartile, accuracy increases by 31.4% and token use drops by 57%.The passage reports consistently negative correlations with token use and positive correlations with accuracy across analyzed settings.Source: §2 · p. 4

Token-level analysis

• For the same reasoning trace, early “Wait” tokens can mark productive self-correction, whereas later occurrences tend to be wasteful, reflected in positive versus negative NormLP.The cited table is described as a functional analysis of “Wait” at different positions within one trace.Source: §2 · p. 4, Table 1 · p. 4

3 Methodology

• The proposed efficient GRPO aims to preserve core-token reasoning while compressing NormLP-identifiable redundant tokens to reduce token use without significant accuracy loss.The paper frames this as a possible Pareto improvement in accuracy and token efficiency.Source: §3 · p. 4
• A sliding window over NormLP scores selects the highest-scoring non-overlapping spans as core tokens, with remaining positions marked redundant.The window strategy is intended to identify essential reasoning steps while preserving semantic coherence without turn-level annotation.Source: §3 · pp. 4–5

3.1 Selective KL Anchoring (KL)

• Selective KL Anchoring leaves core-token regularization unchanged and applies overflow-proportional sparsity pressure to redundant tokens only after a rollout exceeds its length limit.Within-budget redundant tokens retain the stable base regularizer.Source: §3.1 · p. 5

3.2 Reward Shaping (RS)

• Reward Shaping adjusts the policy advantage using NormLP, upweighting core tokens and penalizing redundant tokens when a trace exceeds its length budget.Shaping is restricted to correct-answer rollouts; wrong-answer rollouts retain the vanilla advantage.Source: §3.2 · p. 5
• The main-results table covers four in-domain reasoning benchmarks and reports average generated tokens per response and averaged accuracy.Lower token count and higher accuracy are the preferred directions; the supplied caption provides no result values.Source: Table 2 · p. 6

4.1 Experimental Setup

• Training uses DeepScaleR-Preview’s 40K math corpus and evaluates five backbones spanning 0.6B–8B parameters, alongside closed-API, open-source, and efficient-reasoning baselines.The evaluation uses four in-domain benchmarks; rollout caps are 4,096 tokens for training and 8,192 for evaluation.Source: §4.1 · p. 6

4.2 Main Results

• 42.3% accuracy at 2,023 tokens: Qwen-0.6B-w/KL improves 1.6% over base with 2.9× compression, avoiding the accuracy collapse or limited compression of other methods.At a matched budget, Qwen-8B-w/KL reaches 62.7% at 1,476 tokens, outperforming Gemini-3.1-Pro by 8.3% (62.7 vs. 54.4 at 1,476 vs. 1,586 tokens).Source: §4.2 · p. 7

4.3.1 Signals Along RL Training

• On Qwen-4B, KL and RS shorten responses from roughly 3.7K tokens to 1.6–1.8K and 1.9–2.0K, respectively, while core-token coverage rises during RL.TokenProbe rises from about 0.84 to above 0.92 under KL; length violations fall initially, then rebound as the curriculum tightens the budget, especially for RS.Source: §4.3.1 · pp. 7–8

4.3.2 Out-of-domain Efficiency Transfer

• Across four additional scientific, legal, and multidisciplinary benchmarks, KL uses 935 tokens and averages 60.61 versus the base model’s 4,401 tokens and 59.62.RS uses 1,179 tokens and averages 57.74; Figure 5 presents transfer and ablation results.Source: §4.3.2 · p. 8, Figure 5 · p. 8

4.3.3 Diagnostic Ablation

• In RS, increasing λ from 0.05 to 0.08 cuts tokens 21% (2,291 to 1,806) at nearly unchanged accuracy (36.1 vs. 36.3); increasing α to 0.07 raises accuracy to 39.9% at 2,179 tokens.For KL, β=5 × 10^-3 yields 42.3% versus 39.0% at about 2,200 tokens; changing window size from 10 to 20 yields 42.3% at 2,023 tokens versus 41.4% at 2,008 tokens.Source: §4.3.3 · p. 8

5 Insights and Limitations

• The section frames compression research around preserving useful exploration while testing where compression becomes unstable and whether TokenProbe remains valid across regimes and tasks.The authors point readers to visualizations, case studies, and accuracy–cost Pareto plots on the project homepage.Source: §5 · p. 9

5.1 Unsuccessful Attempts: SFT Distillation

• Offline SFT compression can remove causal context and discard useful low-NormLP self-corrections, disrupting the reasoning process and aligning with the reported distillation failures.A token’s usefulness depends on its role in the trajectory and on preserving a complete path to the answer.Source: §5.1 · p. 9

5.2 Limits of Token Compression

• Excessive shortening pressure can disrupt intermediate computation and trigger repetition; response length may then rise as accuracy collapses.Stable compression requires preserving enough computation for useful self-correction and solution completion.Source: §5.2 · p. 9
• Attainable compression depends on checkpoint and task difficulty: sequentially demanding problems leave less room to shorten, and initialization can affect optimization stability.The authors motivate schedules that adapt to correctness and degeneration while retaining intermediate steps on harder problems.Source: §5.2 · p. 9

5.3 Limitations and Future Directions

• TokenProbe’s interpretation depends on the generating policy; repetitive errors can be predictable, and confidence–correctness alignment can reverse after a training-regime change.The authors call for revalidating the signal and leave its mechanism and reliability theoretically uncharacterized.Source: §5.3 · p. 9
• Efficiency transfer beyond mathematics is reported, but broader validation remains needed; code generation and agentic reasoning are proposed as further settings to study.These settings change the role of intermediate tokens through executable steps, external feedback, or longer interactions.Source: §5.3 · p. 9

6.1 Efficient Reasoning for Large Language Models

• TokenProbe adds token-level importance scoring to model-oriented methods, injecting the signal into selective KL anchoring and reward shaping rather than optimizing a scalar response-length objective.The authors report strong compression without accuracy collapse in their five-backbone setting.Source: §6.1 · p. 10

6.2 Variants of Group Relative Policy Optimization

• TokenProbe couples token-level redundancy signals with selective KL anchoring and reward shaping in GRPO, steering advantages toward informative tokens while leaving reasoning-critical regions untouched.The paper contrasts this with response-level length penalties and reports 2–4× compression.Source: §6.2 · p. 10

7 Conclusion

• TokenProbe combines NormLP and TokenProbe diagnostics with selective KL anchoring and reward shaping, and demonstrates Pareto-improving token efficiency across tested backbones and reasoning benchmarks.The authors conclude that these findings warrant further exploration in efficient reasoning.Source: §7 · p. 10

• Table S1 lists the training models used in Table 2.Source: Table S1 · p. 19
• Open-source baseline identifiers follow the HuggingFace org/model-name convention, with MoE parameters shown as Total (Active).Source: Table S2 · p. 19, S1 Model Notations · p. 19
• Closed-source model names use the Model Name-Version-Mode format.Source: Table S3 · p. 19, S1 Model Notations · p. 19

• +24.1% accuracy comes with 4.5× token consumption for CoT on Qwen-4B.The comparison uses the same model weights and toggles only the enable_thinking flag.Source: Table S4 · p. 20, S2.1 The Token Cost Problem in Reasoning · p. 20
• The 4.5× token increase motivates testing whether low-value exploratory redundancy can be removed while preserving core reasoning and improving efficiency and accuracy.Source: S2.1 The Token Cost Problem in Reasoning · p. 20

• TokenProbe complements overthinking, likelihood or reward signals, and entropy-based objectives by identifying core and redundant tokens within one generated trace using NormLP and TokenProbe.The cited adjacent work includes overthinking analyses [13] [65], token-level likelihood or reward signals [36] [71], and entropy-based objectives [29] [77].Source: S2.2 Relation to Prior Evidence on… · pp. 20–21
• Unlike response-level overthinking diagnoses [13] [65], TokenProbe identifies which local regions within the same trace to preserve or compress.Source: S2.2 Relation to Prior Evidence on… · p. 20
• TokenProbe uses the generating model’s own per-token log probabilities, normalized within each response, without a reference or external reward model.Rho-style token selection [36] typically uses policy/reference-style likelihood comparisons.Source: S2.2 Relation to Prior Evidence on… · pp. 20–21
• Unlike TokenSeek’s context- and gradient-based selection for training memory [93], TokenProbe diagnoses generated traces with inference-time log probabilities to guide reasoning-length optimization.Source: S2.2 Relation to Prior Evidence on… · p. 21
• TokenProbe uses the centered/scaled log-probability sequence rather than absolute entropy or confidence, with the signal remaining meaningful across the Qwen→DeepSeek-Distill calibration boundary.High-entropy token analysis [71] supports the broader idea that minority token subsets can carry disproportionate learning signal.Source: S2.2 Relation to Prior Evidence on… · p. 21
• Unlike PEAR’s Think/Answer phase separation [29] and ETR’s response-level entropy trend [77], TokenProbe supplies a within-Think-phase mask for selective KL anchoring and reward shaping.Source: S2.2 Relation to Prior Evidence on… · pp. 20–21

• The subsection expands every Figure 2 panel into per-model, per-dataset, and per-quartile tables.Source: S2.3 Experiment of TokenProbe · p. 21
• It provides supporting sensitivity, fine-grained, and correlation analyses.Source: S2.3 Experiment of TokenProbe · p. 21
• Model naming follows Figure 2 where applicable.Source: S2.3 Experiment of TokenProbe · p. 21

S2.3.1 TokenProbe for each stage (Fig. 2(a))

• The analysis splits each response at </think> and computes TokenProbe separately for the Think and Answer segments.The Think segment spans the intermediate reasoning process; the Answer segment follows </think>.Source: S2.3.1 TokenProbe for each stage (Fig.… · p. 21
• Answer-stage TokenProbe is +0.026 to +0.199 higher than Think-stage TokenProbe across all five models.The passage reports this as a 3% to 20% gap.Source: S2.3.1 TokenProbe for each stage (Fig.… · p. 21, Table S5 · p. 21
• The Think stage corresponds to low-confidence exploratory search, whereas the Answer stage corresponds to high-confidence decision output.Source: S2.3.1 TokenProbe for each stage (Fig.… · p. 21

S2.3.2 TokenProbe for each model (Fig. 2(b))

• Across all four benchmarks, a model with higher TokenProbe simultaneously achieves higher accuracy and lower token usage.Source: S2.3.2 TokenProbe for each model (Fig.… · p. 21, Table S6 · p. 22, Table S7 · p. 22
• Table S6 consolidates response-level TokenProbe, average token usage, and average accuracy for the four representative models.Source: S2.3.2 TokenProbe for each model (Fig.… · p. 21, Table S6 · p. 22
• Tables S7 and S8 report per-benchmark accuracy and token usage, respectively, for the four models.Source: S2.3.2 TokenProbe for each model (Fig.… · p. 21, Table S7 · p. 22

S2.3.3 Quartile analysis (Fig. 2(c))

• For each model, 1,288 samples are sorted by per-response TokenProbe and divided into four equal quartiles.Source: S2.3.3 Quartile analysis (Fig. 2(c)) · p. 22
• The quartile analysis reports average TokenProbe, accuracy, and token usage for Qwen-4B, Qwen-8B, DS-Qw-1.5B, and Qwen-0.6B.Tables S9–S12 cover these models.Source: S2.3.3 Quartile analysis (Fig. 2(c)) · p. 22, Table S9 · p. 22, Table S10 · p. 22
• From lowest to highest TokenProbe quartile, accuracy rises monotonically while average token usage drops on every model.The pattern is also observed in Fig. 2(c) for Qwen-4B and Qwen-8B.Source: S2.3.3 Quartile analysis (Fig. 2(c)) · p. 23

S2.3.4 Sensitivity analysis on Qwen3.5-9B

• The analysis varies context length and sampling temperature for Qwen-9B while keeping model weights fixed.Source: S2.3.4 Sensitivity analysis on… · p. 23
• TokenProbe stays within 0.780 to 0.782 across three settings, while accuracy shifts from 36.8% to 46.5% and average token usage from 7,252 to 10,009.The authors interpret this stability as support for TokenProbe capturing an intrinsic property rather than a transient decoding-configuration artifact.Source: S2.3.4 Sensitivity analysis on… · p. 23, Table S13 · p. 23

S2.3.5 Per-dataset TokenProbe and Correct-versus-Wrong comparison

• Correct samples have consistently higher TokenProbe than wrong samples, with gaps of +0.028 to +0.059; non-thinking Qwen-4B-Instruct has the largest gap (+0.059).Thinking models have smaller gaps, around +0.028, which the authors attribute to low-confidence tokens in their Think segments compressing the response-level contrast.Source: S2.3.5 Per-dataset TokenProbe and… · p. 24, Table S15 · p. 24
• On non-thinking Qwen-4B-Instruct, vanilla GRPO reduces accuracy from 71.7 to 1.3 while increasing tokens from 3,600 to 8,136; KL variants reduce tokens but do not recover base accuracy.At β = 5×10^-3, w = 20, the KL variant reaches 52.1 accuracy with 2,346 tokens; at β = 5×10^-4, w = 20, it reaches 41.7 accuracy with 790 tokens. The paper attributes the regression to a lack of exploratory-token headroom and restricts deployment to thinking-enabled backbones.Source: S2.3.5 Per-dataset TokenProbe and… · p. 24, Table S16 · p. 24

S2.3.6 Pearson correlation analysis

• Across five thinking models, TokenProbe correlates negatively with response length (−0.227 to −0.415) and positively with accuracy (+0.197 to +0.367).The authors interpret these joint patterns as showing that longer responses contain more low-confidence tokens, while higher-TokenProbe responses are more likely to be correct.Source: S2.3.6 Pearson correlation analysis · pp. 24–25, Table S17 · p. 25

S2.4 Case Study of TokenProbe

• In the Qwen-4B case study, early “Wait” tokens mark productive self-correction (+0.47 and +0.22 NormLP), while later ones mark confusion and rechecking (−3.87 and −4.13 NormLP).The later rechecking triggers a +700-token low-confidence tail; the response-level positive-token ratio captures this productive-versus-wasteful contrast without token-by-token labels.Source: S2.4 Case Study of TokenProbe · p. 26

S2.5 Why TokenProbe? An Ablation over Log-Probability-Derived Signals

• Only TokenProbe and related within-response shape statistics preserve the accuracy ordering across the model ladder; absolute log-probability and entropy signals invert at the Qwen–DeepSeek boundary.Within-response standardization makes the shape family invariant to family-level calibration shifts; TokenProbe is interpreted as the fraction of tokens above a response’s own confidence baseline.Source: S2.5.2 Overall result. Only TokenProbe… · p. 31, Table S19 · p. 31, S2.5.4 Why TokenProbe survives.… · p. 32

S3.1 Core Token Window

• The core-token heuristic standardizes per-token log-probabilities, greedily selects the K highest-scoring non-overlapping windows of size W, and marks their tokens as core without external annotation.Contiguous windows are used to preserve local semantic coherence.Source: S3.1 Core Token Window · p. 34

S3.2 Exploratory Analysis of Core-Window Selection

• Exploratory tests find that non-overlapping windows cover reasoning traces more broadly, greedy selection scores within 0.2% of the dynamic-programming optimum, and W=5 yields shorter but less accurate responses than W=10–30.In a 20-rollout semantic probe, high-scoring windows were dominated by symbolic or formula-level content, while low-scoring windows contained more hesitation and connective filler; the authors caution that this is a distributional tendency, not a hard partition.Source: S3.2 Exploratory Analysis of… · pp. 34–36

S3.3 Selective KL Anchoring (KL)

• Selective KL preserves the standard KL anchor on core tokens and pressures redundant tokens only after the rollout exceeds its length budget.For over-budget redundant tokens, it drops the KL anchor and applies an overflow-proportional log-probability penalty; within-budget redundant tokens are left untouched.Source: S3.3 Selective KL Anchoring (KL) · p. 37

S4.1 Training Settings

• Experiments train with GRPO on DeepScaleR and evaluate four in-domain math and four out-of-domain benchmarks against six recent efficient-reasoning methods.Training uses a maximum 4,096-token rollout; evaluation allows up to 8,192 tokens.Source: S4.1 Training Settings · p. 38, S4.2 Evaluation Settings · pp. 38–39, S4.3 Baselines · p. 40

S4.2 Evaluation Settings

• The 8K Gemini-3.1-Pro check shows frontier APIs can exceed small open models on accuracy, while Qwen-4B variants remain in a much lower-cost regime.The comparison is not perfect budget matching: most frontier rows use a soft 2K instruction, whereas open models have a hard 8K cap.Source: S4.2 Evaluation Settings · p. 39, Figure S4 · p. 40

S4.4 Capacity-dependent Behavior of KL and RS

• RS’s average accuracy gap versus KL shifts from −6.0 points on Qwen-0.6B to +5.6 on Qwen-8B, with Spearman ρ = 0.97 across model scales.The authors recommend KL at or below roughly 1B parameters and RS at 4B or larger when accuracy, rather than maximal compression, is the priority.Source: Table S31 · p. 41, S4.4 Capacity-dependent Behavior of KL… · p. 41

S3.3 Selective KL Anchoring (KL)

• KL changes regularization while leaving the policy-gradient term unchanged; RS reshapes advantages on correct rollouts, boosting high-NormLP core tokens and penalizing over-budget redundant tokens.KL removes the anchor and applies sparsity pressure to over-budget redundant tokens; RS leaves wrong-answer rollouts’ advantages unchanged.Source: S3.4 Reward Shaping (RS) · p. 37, S4.4 Capacity-dependent Behavior of KL… · p. 41

S4.5 Reward Dynamics During RL Training

• RS shaping fires on about 1.56× more rollouts per training step for DeepScaleR-1.5B than Qwen3-0.6B, and the smaller model’s correct-rollout fraction remains lower throughout training.The reported fractions are 0.256 versus 0.450 at the start and 0.427 versus 0.612 at the end; the authors link lower shaping density and higher gradient variance to RS’s small-model disadvantage.Source: Figure S5 · p. 42, S4.5 Reward Dynamics During RL Training · p. 42

S4.6 Computational Overhead of TokenProbe Operations

• TokenProbe’s NormLP and core-window operations add less than 0.09% of an estimated GRPO step at 8K traces and less than 0.21% at 16K; inference overhead is zero.Both operations run during training on CPU using log-probabilities already returned by vLLM.Source: Table S32 · p. 43, S4.6 Computational Overhead of… · p. 43

S4.7 Hard-Truncation Strawman Baseline

• At about 2K output tokens, Qwen-4B + KL and + RS retain 61.0% and 62.3% accuracy, versus 31.94% for the hard-capped base model; 93.3% of capped rollouts truncate.The trained variants remain within 1.4 points of the unconstrained 8K base, showing that learned compression is not equivalent to clipping trajectories.Source: Table S33 · p. 43, S4.7 Hard-Truncation Strawman Baseline · p. 43

S4.8 Case Studies

• In five paired 8K-budget case studies, KL-shaped 8B avoids base-model failures to return an extractable answer or uses 3 to 6× fewer tokens for the same correct answer.The comparisons use identical prompts and the matched unshaped 8B base.Source: S4.8 Case Studies · p. 44
• 1,897 tokens let the KL-shaped model solve the AIME problem, while the 8B base hits its 8K cap and fails extraction before completing its answer.The shaped trace factors the quadratic and counts the two lattice-point families, subtracting their overlap; the base repeatedly re-derives and checks the counts.Source: S4.8 Case Studies · pp. 45–46

Case A2 — Olympiad Bench

• 1,667 tokens yield a correct KL-shaped answer, f(x) = −x, while the 8B base reaches its 8,192-token cap without an extractable answer.The shaped trace uses a linear ansatz and verifies both branches, though the final boxed answer reports only one branch and a strict matcher credits only that branch.Source: Case A2 — Olympiad Bench · p. 47

Case B1 — AMC

• 1,339 tokens produce the correct AMC answer, 6, versus 7,434 tokens for the equally correct 8B base—a 5.5× token inflation.The base repeats prime factorizations and constraint derivations and explores alternatives after finding the answer; the KL-shaped trace uses one tight chain.Source: Case B1 — AMC · p. 49

Case B2 — MATH

• On textbook MATH problems where the unshaped base is nominally accurate, it pays an approximately 5× token tax for defensive recomputations, while the KL-shaped model stops after first-pass difference-of-squares factoring.Source: Case B2 — MATH · p. 51
• 947 tokens produce the correct AIME 2025 answer, 70, versus 3,702 tokens for the equally correct 8B base, a 3.9× increase.The base redundantly derives the divisibility condition, checks candidate divisors, and verifies surviving bases; the KL-shaped model stops after the concise derivation.Source: Case B2 — MATH · p. 53

S4.9 Scenario Analysis Under an Unlimited Token Budget

• The token-budget gradient analysis imposes 1K–32K limits and a 256K unlimited baseline on six models, measuring accuracy degradation across four mathematical reasoning benchmarks.The selected models include open-source MoE architectures and closed-API services.Source: S4.9 Scenario Analysis Under an… · p. 53
• Concise reasoning aligns more with budget resilience than unlimited accuracy: Gemini-3-Flash retains 73.8% at 8K, while Qwen-3.6-Plus falls to 38.5%.At unlimited, Gemini-3-Flash uses 4,844 average tokens and scores 91.4%, versus Qwen-3.6-Plus at 18,099 tokens and 92.2%; AIME scores 0.0% for all six models at 1K, while MATH-500 is most resilient.Source: Figure S6 · p. 54, S4.9 Scenario Analysis Under an… · pp. 54–55

S5.2.1 Accuracy; S5.2.2 Token Usage and Efficiency

• SFT distillation drops overall accuracy by 33.4 percentage points in its best setting and yields no accuracy-token Pareto-frontier model.S3 achieves 39.1% overall accuracy versus 72.5% for its 8B baseline; the nothink baseline Pareto-dominates all SFT settings, while Instruct nothink reaches 71.8% accuracy at comparable token usage to S3.Source: Table S38 · p. 56, S5.2.1 Accuracy · p. 56, S5.2.2 Token Usage and Efficiency · p. 56

S5.3.1 Pathological TokenProbe Reversal

• TokenProbe’s relationship with correctness reverses after SFT: RL baselines show r ≈ +0.25 to +0.31, whereas SFT settings show incorrect samples with higher TokenProbe.Across the four SFT settings, r(TokenProbe, correctness) shifts from +0.3 in RL to −0.5 in SFT, while r(tokens, TokenProbe) shifts from −0.4 to +0.9.Source: S5.3.1 Pathological TokenProbe Reversal · p. 57, Table S40 · p. 57

S5.3.2 Degeneration Loop Microstructure

• Incorrect SFT outputs show degeneration signatures: flattened log-probabilities, positive-confidence runs of 7,000–7,800 tokens, and second-half TokenProbe of 0.91–0.92.Correct samples’ second-half TokenProbe is 0.82–0.85; the long high-confidence runs can accompany entirely wrong output.Source: Table S41 · p. 57, S5.3.2 Degeneration Loop Microstructure · p. 58

S5.3.3 Detailed Case Studies

• Case studies show SFT failures ranging from confident repetition to lost self-correction, while a think-tagged setting solves one example correctly in 568 tokens.S2 loops for 8,192 tokens with TokenProbe 1.000; S3 makes an arithmetic error after distillation removes “Wait” tokens, while S4 resorts to brute-force enumeration and is truncated.Source: S5.3.3 Detailed Case Studies · pp. 58–59

S5.3.4 Why SFT Fails While RL Succeeds. Structural Analysis

• SFT’s structural limitations include deleting self-correction markers, confusing valuable exploration with redundancy, disrupting causal context, and lacking RL-style credit assignment and online feedback.Filtering retains only 6.8% of Phi4 and 9.6% of Qwen self-correction markers; RL instead preserves full trajectories and reinforces them differentially through rewards.Source: S5.3.4 Why SFT Fails While RL… · pp. 59–60

S5.4 Outlook. SFT as RL Warm-Start

• SFT may serve as an RL warm-start, but the authors advise verifying that r(TokenProbe, correctness) remains positive after initialization.Correct samples use fewer tokens (Pearson r(tokens, correct) ≈ −0.59 to −0.74); the proposed goal is more precise self-correction rather than removing it.Source: S5.4 Outlook. SFT as RL Warm-Start · pp. 60–61

S6 Exploring Token Compression Limits

• Length-controlled RL training degenerates when compression exceeds approximately 1.6× in the reported small-model runs.The figure reports length explosion, accuracy collapse to zero, and repetitive outputs after that threshold for Qwen3-0.6B-Base and DeepScaleR-1.5B.Source: Figure S7 · p. 61
• On Qwen3-4B-Instruct, both vanilla GRPO and the KL variant converged to 4096-token outputs with repetitive n-grams; the self-anchor did not prevent degeneration.The authors attribute this to a low-entropy starting policy and a feedback loop in which degraded accuracy and repetitive padding weaken reward gradients.Source: S6 Exploring Token Compression Limits · pp. 61–62
• At approximately 1.6× compression, Qwen3-0.6B-Base and DeepScaleR-1.5B collapsed into maximum-length repetitive outputs with zero accuracy; the observed floor is backbone-agnostic across these checkpoints, not task-agnostic.The floor was measured on DeepScaleR math data and four math benchmarks; easier distributions tolerated more compression, while AIME-2025 fell to 0% on every tested large model below a 4K cap.Source: S6 Exploring Token Compression Limits · p. 62

S7.1 Theoretical Certification

• The work presents TokenProbe’s alignment with accuracy and token use as an empirical association, leaving a deeper mechanistic account and theoretical certification for future work.Its transfer diagnostic does not establish validity for visual prompt tokens, which would require generated reasoning traces with accessible token log-probabilities.Source: S7.1 Theoretical Certification · p. 62, S7.2 General Reasoning Efficiency · p. 63

S7.2 General Reasoning Efficiency

• In a fixed-sample transfer diagnostic, TokenProbe increased after KL or RS training in both math and general domains, offering preliminary trend-level evidence beyond math.Each domain used 40 samples: 10 each from four datasets; the authors frame this as an initial diagnostic, not a complete generality claim.Source: S7.2 General Reasoning Efficiency · pp. 62–63

S8.3 Compute Resources

• TokenProbe-specific CPU computation added 0.009%–0.210% of a representative GRPO step across 1K–16K-token traces, with zero inference overhead for unmodified decoding.The training data is the publicly available DeepScaleR-Preview-Dataset, and the appendix documents pseudocode for core-window selection, selective KL anchoring, and reward shaping.Source: S8.1 Open Access to Data and Code · p. 63, S8.3 Compute Resources · p. 64

S8.4 Broader Impacts

• TokenProbe-guided reasoning may lower inference cost and improve access for resource-constrained deployments, but cheaper reasoning can also lower misuse costs and does not guarantee correct or complete traces.The authors describe TokenProbe as an efficiency and diagnostic method, not a safety or factuality guarantee.Source: S8.4 Broader Impacts · p. 64

S9.1 Within-problem analysis controlling for response-level factors

• TokenProbe retained positive correctness coefficients of 0.277 on MATH and 0.148 on OlympiadBench after controlling for response length, confidence, and repetition within each problem.The analysis used the original Qwen3-4B thinking model and standardized predictors within each dataset.Source: S9.1 Within-problem analysis… · p. 66

S9.2 Causal interventions on high-NormLP regions

• High-NormLP deletion caused 1.5% and 1.1% more correctness damage than random deletion on MATH and OlympiadBench; attention-key masking differences were 1.6% and 0.9%.Both interventions ordered damage as High-NormLP > Random > Low-NormLP, providing causal evidence that high-NormLP regions were more consequential in these evaluations.Source: Table S47 · p. 66, S9.2 Causal interventions on… · p. 66

5. Open access to data and code

• Training and evaluation details are specified in Sections S4.1–S4.3; run-to-run variability, hardware and training-time requirements, and TokenProbe overhead are reported in Section S8.2/Table S44, Section S8.3/Table S45, and Table S32.Source: §5 · pp. 74–75

10. Broader impacts

• Potential positive and negative societal impacts are discussed in Section S8.4; the paper releases no new high-risk foundation model or scraped dataset.Source: §10 · pp. 75–76

12. Licenses for existing assets

• Existing asset licenses are summarized in Section S8.5/Table S46; no new dataset, model, or code asset is released with the submission, and code release is planned after acceptance.The experiments use existing benchmarks and model/API evaluations, with no crowdsourcing or human-subject study.Source: §12 · pp. 76–77

16. Declaration of LLM usage

• LLM usage is declared in Section S8.6.Source: §16 · p. 78