PaperlayerView exact revision on arXiv

Source-linked AI summary

Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning

Chen Qian, Dongrui Liu, Haochen Wen, Zhen Bai, Yong Liu, Jing Shao

arXiv:2506.02867v2cs.AIcs.CL

TL;DR

Large reasoning models perform well on complex problems, but their internal reasoning mechanisms remain poorly understood. The paper tracks mutual information between intermediate representations and the golden answer, identifies MI peaks associated with thinking tokens, and uses them to develop training-free improvements that enhance reasoning performance.

  • Problem

    The internal dynamics of large reasoning models and the influence of intermediate reasoning steps on final answers remain largely unexplored.

  • Method

    The paper tracks mutual information between each reasoning-step representation and the golden answer, analyzes the tokens at MI peaks, and develops Representation Recycling and Thinking Token based Test-time Scaling.

  • Results

    MI peaks are sparse, often align with reflective or transitional thinking tokens, and suppressing these tokens significantly degrades reasoning performance while suppressing other tokens has minimal effect.

  • Takeaways & Limitations

    Thinking tokens provide information-rich reasoning cues, and leveraging MI-peak representations or thinking tokens can improve LRM reasoning performance without additional training.

Abstract

from arXiv · show

Large reasoning models (LRMs) have demonstrated impressive capabilities in complex problem-solving, yet their internal reasoning mechanisms remain poorly understood. In this paper, we investigate the reasoning trajectories of LRMs from an information-theoretic perspective. By tracking how mutual information (MI) between intermediate representations and the correct answer evolves during LRM reasoning, we observe an interesting MI peaks phenomenon: the MI at specific generative steps exhibits a sudden and significant increase during LRM's reasoning process. We theoretically analyze such phenomenon and show that as MI increases, the probability of model's prediction error decreases. Furthermore, these MI peaks often correspond to tokens expressing reflection or transition, such as ``Hmm'', ``Wait'' and ``Therefore,'' which we term as the thinking tokens. We then demonstrate that these thinking tokens are crucial for LRM's reasoning performance, while other tokens has minimal impacts. Building on these analyses, we propose two simple yet effective methods to improve LRM's reasoning performance, by delicately leveraging these thinking tokens. Overall, our work provides novel insights into the reasoning mechanisms of LRMs and offers practical ways to improve their reasoning capabilities. The code is available at https://github.com/ChnQ/MI-Peaks.

1 Introduction

The paper addresses the poorly understood internal reasoning dynamics of large reasoning models by tracking mutual information between intermediate representations and the golden answer. It identifies sparse MI peaks associated with thinking tokens and uses these insights to propose training-free methods for improving reasoning performance.

  • Motivation: Large reasoning models achieve strong results on complex tasks, but the mechanisms behind their reasoning and the influence of intermediate steps remain largely unexplored.The paper frames this internal process as a black box despite progress in mathematics, programming, and logical inference.
  • Information-theoretic analysis: The authors track mutual information between each reasoning-step representation and the golden answer, finding sudden increases at specific, sparse, non-uniform steps.These high-information steps are termed the MI peaks phenomenon.
  • Thinking tokens: MI-peak representations predominantly correspond to reflective or transitional tokens such as “Wait,” “Hmm,” “Therefore,” and “So,” termed thinking tokens.The paper links these tokens to reflection, self-correction, and logical transitions during reasoning.
  • Thinking tokens: Suppressing thinking tokens significantly harms reasoning performance, whereas randomly suppressing the same number of tokens has little impact.This controlled comparison supports the reported importance of thinking tokens to LRM reasoning ability.
  • Methods: The paper proposes two training-free methods: Representation Recycling, which iterates MI-peak representations, and Thinking Token based Test-time Scaling, which continues reasoning with thinking tokens.These methods leverage the observed information concentration and token-level role of MI peaks.
  • Methods: 20% relative accuracy improvement is reported for DeepSeek-R1-Distill-LLaMA-8B on AIME24 with Representation Recycling, while Thinking Token based Test-time Scaling improves steadily as token budget increases.The reported methods are presented as practical applications of the paper’s analysis.

2 Emergence of MI Peaks in LRMs’ Reasoning Trajectories

The paper tracks mutual information between each generated-step representation and the gold answer, revealing sparse MI peaks and linking higher cumulative MI to tighter prediction-error bounds. Comparisons further show stronger MI magnitude and peak intensity in LRMs than in corresponding non-reasoning LLMs.

  • 2.1 Preliminaries: The analysis extracts hidden representations for each generated token and estimates their MI with the gold answer using HSIC.This produces an MI sequence across the reasoning trajectory.
  • 2.2 Investigating LRM’s Reasoning Trajectories with MI: MI peaks are steps where MI suddenly increases, identified using an interquartile-range rule with scale factor τ = 1.5.The paper defines peaks from the MI sequence using quartiles and the interquartile range.
  • 2.2 Investigating LRM’s Reasoning Trajectories with MI: No more than 5% of reasoning steps are MI peaks, and their 0.51% minimum ratio occurs for DeepSeek-R1-Distill-Qwen-7B.Despite their sparsity, peaks are scattered across the trajectory at non-uniform intervals.
  • 2.3 Theoretical Insights: Higher MI Leads to Tighter Bounds on Prediction Error: Higher cumulative MI between reasoning representations and the gold answer yields tighter lower and upper bounds on prediction error.The paper interprets this relationship as making correct answers more likely, while noting that MI peaks can increase cumulative MI.
  • 2.4 Will Non-reasoning LLMs also Exhibit the MI Peaks Phenomenon?: Compared with corresponding non-reasoning LLMs, LRMs show higher overall MI and sharper, more intense MI fluctuations and peaks.Figure 3 and the Mean, Std, and AOM comparisons support this distinction.

3 Thinking Tokens are Information Peaks in LLM Reasoning

The paper decodes representations at MI peaks into tokens and finds that these peaks predominantly correspond to reflective or transitional thinking tokens. Suppressing thinking tokens significantly harms reasoning performance, unlike suppressing other tokens.

  • Projecting representations to token space: MI-peak representations are decoded into tokens using the output projection and greedy selection of the highest-probability token.The procedure computes a token distribution for each MI-peak representation, then aggregates decoded tokens across the evaluation dataset.
  • Token semantics: MI-peak tokens are predominantly connective words and reflective expressions such as “So,” “Hmm,” and “Wait.”These tokens are associated with pauses, thinking, internal deliberation, and transitions in the reasoning process.
  • Thinking tokens: The paper names decoded high-MI tokens “thinking tokens.”They serve as discourse cues for reflection and have representations carrying high mutual information with the golden answer.
  • Token importance: Suppressing thinking tokens significantly impairs LRM reasoning performance, whereas suppressing the same number of other tokens has minimal effect.The comparison uses controlled inference interventions that set selected token-generation probabilities to zero.

4 Applications: Leveraging MI Peaks to Improve LRM Reasoning

The paper proposes two training-free applications of MI-peak analysis: Representation Recycling reprocesses high-MI representations, while Thinking Token based Test-time Scaling extends reasoning with selected tokens. Both methods improve reasoning performance in the reported experiments.

  • Applications: The paper proposes Representation Recycling and Thinking Token based Test-time Scaling as two techniques for improving LRM reasoning performance.Representation Recycling reuses internal representations at MI peaks, while TTTS incorporates thinking tokens during test-time scaling.
  • 4.1 Recycling High-MI Representations During Inference: Representation Recycling feeds MI-peak representations back through the same transformer layer to process them more thoroughly.The modified forward pass reprocesses a selected representation before continuing through later layers.
  • 4.1 Recycling High-MI Representations During Inference: RR experiments use DeepSeek-R1-Distill-Llama-8B and Qwen-7B on three mathematical reasoning benchmarks, triggering recycling when recorded thinking tokens appear.Thinking tokens are recorded from the MATH training set because ground-truth answers are unavailable during inference.
  • 4.1 Recycling High-MI Representations During Inference: RR consistently improves reasoning performance across all evaluated benchmarks, with a notable gain on AIME24.The passage attributes the improvement to recycling MI-peak representations so models can further exploit their reasoning information.
  • 4.2 Test-Time Scaling with Thinking Tokens: TTTS appends selected reflective or transitional thinking tokens to an initial output and allows the model to generate additional reasoning steps.The method filters out tokens with little semantic content and retains tokens such as “So” and “Hmm.”
  • 4.2 Test-Time Scaling with Thinking Tokens: Under the same token budget, TTTS consistently outperforms the original LRM on GSM8K and MATH500.The evaluation uses LLaMA-8B while gradually increasing the inference token budget.
  • 4.2 Test-Time Scaling with Thinking Tokens: On AIME24, TTTS eventually surpasses the original model once the token budget exceeds 6144 tokens, despite slight underperformance at some intermediate budgets.The original model saturates around 3000 tokens, whereas TTTS continues improving steadily in the reported setting.

5 Related work

Related work covers chain-of-thought reasoning, information-theoretic analyses of language models, and critical-token research examining tokens that disproportionately affect model behavior.

  • Reasoning in LLMs: Chain-of-thought research formalizes explicit intermediate reasoning steps for complex tasks, while later work studies minimalist prompts for eliciting step-by-step reasoning.
  • Information Theory in LLMs: Information theory has been applied to diagnose reasoning processes, optimize models, and analyze behavioral dependencies and error propagation.
  • Critical Tokens in LLMs: Critical-token research identifies, measures, and mitigates small sets of tokens that can disproportionately influence language-model behavior, including in safety alignment.

6 Conclusion

The paper links MI peaks to reflective, transitional, or self-corrective thinking tokens and introduces two training-free methods that improve LRM reasoning performance. It presents these findings as insights into LRM reasoning and inference-time enhancement.

  • Conclusion: MI peaks predominantly correspond to thinking tokens such as “Hmm,” “Wait,” and “Therefore,” which express self-reflection, logical transitions, or self-correction.
  • Conclusion: The paper introduces Representation Recycling and Thinking Token based Test-time Scaling as training-free methods that effectively improve LRM reasoning performance.

A.1 Proof of Theorem 1

The proof bounds prediction error using conditional entropy and mutual information, then relates output information to intermediate reasoning representations through the data processing inequality. It establishes the result for binary and multiclass prediction settings.

  • Theorem 1 analyzes representations h1:T, the golden answer y, the prediction ŷ, and prediction error probability pe = Pr(ŷ ≠ y).
  • The proof introduces the error indicator E and decomposes H(y | ŷ) using the chain rule of entropy.
  • Because E = 0 implies ŷ = y, the corresponding conditional entropy vanishes, while H(E | ŷ) is bounded by the binary entropy H_b(pe).
  • Mutual information satisfies I(y; ŷ) = H(y) − H(y | ŷ), and the data processing inequality gives I(y; ŷ) ≤ I(y; h1:T).
  • The multiclass extension treats the model output as a classification problem and bounds conditional error through the maximum predicted class probability.

B Experimental Implementation Details

The experiments estimate mutual information with HSIC because direct MI computation is difficult in high-dimensional spaces, using last-layer representations and Gaussian kernels. Reasoning performance is evaluated on three math benchmarks across DeepSeek R1 models and QwQ-32B.

  • HSIC is used to estimate MI empirically because accurately computing MI in high-dimensional spaces is difficult.
  • The empirical HSIC computation uses kernel matrices KX and KY, a centering matrix H, and Gaussian kernels with bandwidth selected from [50, 400].
  • Reasoning performance is evaluated on GSM8K, MATH500, and AIME24, with temperature fixed to 0 for reproducibility.
  • The model set includes DeepSeek’s R1 series and QwQ-32B, paired with corresponding non-reasoning counterparts for the R1 models.
  • MI experiments extract representations from the model’s last layer because higher layers encode more semantic content and the last layer directly influences output text.

C Discussions

The discussion reports model-dependent thinking-token frequency patterns and examines suppression effects. It also identifies token-level analysis and the unexplained origin of MI peaks as limitations and directions for future work.

  • The analysis is limited to token-level MI dynamics, while semantic- or logical-step granularities may reveal additional insights.
  • The mechanisms generating MI peaks remain underexplored, and their deeper investigation is left for future work.
  • MI-peak token frequencies differ slightly across LRMs, with So, Let, Hmm, The, and Okay frequent in several foundation-model-based systems.
  • Suppressing more thinking tokens generally degrades reasoning performance, but the decline is not strictly monotonic because alternative expressions can replace suppressed tokens.
  • Suppressing Wait can yield alternatives such as But wait, which may produce temporary performance improvements.

D Additional Experimental Results

Additional figures extend the paper’s MI-trajectory analysis across more samples and multiple large reasoning models, including DeepSeek variants and QwQ-32B.

  • Figures 9–20 illustrate MI trajectories of various LRMs across more data samples.
  • The appendix includes MI trajectories for DeepSeek-R1-Distill-Llama-8B across Figures 9–10.
  • Figures 11–12 report MI trajectories for DeepSeek-R1-Distill-Qwen-7B.
  • Figures 13–16 report MI trajectories for DeepSeek-R1-Distill-Qwen-14B and DeepSeek-R1-Distill-Qwen-32B.
  • Figures 17–20 report MI trajectories for QwQ-32B and DeepSeek-R1-Distill-Llama-70B.
Loading 2506.02867v2…