Source-linked AI summary
When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
Shu Zhou, Rui Ling, Junan Chen, Xin Wang, Tao Fan, Hao Wang
TL;DR
The paper asks whether longer test-time reasoning reliably improves answers and examines this question through marginal-utility, answer-flip, and cost-aware analyses. It finds diminishing returns and overthinking at high budgets, with difficulty-dependent stopping points and substantial efficiency gains from moderate or adaptive budgets.
Problem
Existing test-time scaling work largely assumes that longer reasoning always improves answers, despite limited systematic examination of this relationship.
Method
The paper evaluates marginal utility across budgets, tracks correct-to-incorrect and incorrect-to-correct answer flips, and applies cost-aware evaluation metrics.
Results
Marginal utility becomes negative beyond 12K tokens, while easy problems peak near 1.5K and hard problems benefit up to about 8K tokens.
Takeaways & Limitations
At λ=0.5, stopping near 6K tokens reduces compute by about 50% with about 6% accuracy loss, while indicator-based stopping reaches 97% of peak accuracy using 60% of compute.
Takeaways & Limitations
The analysis focuses on mathematical and scientific reasoning tasks and open-weight models, while causal mechanisms underlying overthinking remain unresolved.
Abstract
from arXiv · showhide
Scaling test-time compute through extended chains of thought has become a dominant paradigm for improving large language model reasoning. However, existing research implicitly assumes that longer thinking always yields better results. This assumption remains largely unexamined. We systematically investigate how the marginal utility of additional reasoning tokens changes as compute budgets increase. We find that marginal returns diminish substantially at higher budgets and that models exhibit ``overthinking'', where extended reasoning is associated with abandoning previously correct answers. Furthermore, we show that optimal thinking length varies across problem difficulty, suggesting that uniform compute allocation is suboptimal. Our cost-aware evaluation framework reveals that stopping at moderate budgets can reduce computation significantly while maintaining comparable accuracy.
1 Introduction
The paper questions the assumption that longer reasoning always improves answers and studies when additional test-time compute helps or harms. It motivates cost-aware, adaptive stopping by linking diminishing returns and overthinking to compute allocation.
- Existing test-time scaling research generally assumes that longer chains of thought monotonically improve answer quality.
- Additional reasoning may yield diminishing returns and cause models to abandon initially correct answers.
- Generating 8,000 tokens costs 16× more than generating 500 tokens, making unproductive extended reasoning computationally expensive.
- The study measures marginal benefits across compute budgets, tracks answer-changing flip events, characterizes overthinking, and explores early stopping.
- The paper proposes reporting efficiency frontiers alongside accuracy curves to capture when extended reasoning helps versus hurts.
2 Related Work
Related work documents excessive reasoning and non-monotonic accuracy, while this paper adds individual-answer flip tracking, difficulty-stratified analysis, and cost-aware evaluation. It frames adaptive reasoning length as complementary to existing efficient-inference methods.
- Prior studies report excessive token use on simple problems and inverted U-shaped accuracy as chain-of-thought length increases.
- This paper tracks individual-problem answer changes and reports that easy problems overthink at 2K tokens versus 8K for hard problems.
- Its cost-aware framework introduces a tunable λ parameter for accuracy-compute trade-offs.
- The paper extends selective-reasoning perspectives by considering compute costs rather than response risks.
- Adaptive reasoning length complements model compression, early exit, and speculative decoding by using problem characteristics and overthinking detection.
3 Methods
The methods quantify how accuracy changes with reasoning budget and track answer changes across trajectories. They also examine trace-level signals that may support early detection of unproductive additional thinking.
- The study analyzes marginal utility, flip-event detection, and overthinking indicators as reasoning budgets increase.
- Budgets range from 500 to 16,000 reasoning tokens in 500-token increments, controlled through budget forcing.
- Marginal utility is the accuracy change from budget t to t + ∆t, with ∆t fixed at 500 tokens.
- 3.3 Flip Events: A flip event is a change in the model’s predicted answer between consecutive budgets.
- 3.3 Flip Events: Positive flips change incorrect answers to correct, whereas negative flips change correct answers to incorrect; a flip ratio above 1 indicates net harm is more likely.
- The analysis monitors hesitation markers, answer oscillation, and confidence trajectories as possible overthinking signals.
4.1 Experimental Setup
The experiments compare two 32B open-weight reasoning models across mathematical and scientific reasoning tasks and a broad range of forced reasoning budgets. The setup records outputs, correctness, traces, and confidence-related information for each problem-budget pair.
- The study compares DeepSeek-R1-32B and s1-32B, both 32B-parameter open-weight models with test-time scaling capabilities.
- Evaluation uses AIME 2024 and 2025, MATH-500 difficulty levels, and GPQA Diamond to test mathematical, difficulty-stratified, and broader reasoning performance.
- Budgets span 500–16,000 tokens in 500-token increments, producing 32 evaluation points per problem.
- On AIME, marginal utility diminishes with budget, turns negative beyond 12K, and peak accuracy occurs at 12K.
- At each budget, experiments record the final answer, correctness, complete reasoning trace, and token-level log-probabilities.
4.2 Experimental Results
Extended reasoning shows diminishing and eventually negative returns, with overthinking emerging as models abandon correct answers. The optimal budget varies by difficulty and model, making uniform allocation suboptimal.
- Marginal Utility Results: +3.2% per 500 tokens for R1-32B early, but marginal utility turns negative beyond 12K tokens.Easy problems peak at ∼1.5K tokens, whereas Level 5 problems benefit up to ∼8K tokens.
- Additional Model Comparisons: Accuracy peaks around 10K tokens before declining for both R1-32B and s1-32B on GPQA Diamond.R1-32B consistently outperforms s1-32B across budget levels.
- Flip Event Analysis: Beyond 7K tokens, negative flips become more frequent than positive flips, with easier problems crossing the overthinking threshold earlier.Level 1–2 problems cross at 2K tokens versus 8K for Level 5.
- Flip Event Analysis: 67.5% of 80 manually examined cases involve genuine overthinking, where the model explicitly rejects a correct answer.Only 12.5% show degradation artifacts.
- Statistical Robustness Analysis: At 7K tokens, the flip ratio first exceeds 1.0 at 1.09, while the 8K-token 95% CI is [1.21, 1.68].The 6K-token CI [0.71, 1.05] still includes values below 1.0.
- Statistical Robustness Analysis: The accuracy drop from 12K to 16K tokens is −0.9% for R1-32B, with a 95% CI of [−1.4%, −0.4%].The paper reports this decline as statistically significant.
- Additional Model Comparisons: s1-32B crosses into negative-dominated flips at ∼5K tokens versus ∼7K for R1-32B and maintains a higher negative-to-positive ratio.This indicates greater susceptibility to overthinking at lower compute budgets.
- Overthinking Indicators: Combining overthinking indicators achieves 76.3% precision at 80% recall, with answer oscillation providing the strongest individual signal at r = 0.78.The combined indicators reach r = 0.82.
5 Cost-Aware Evaluation
Cost-aware evaluation treats inference compute as a deployment cost and identifies stopping points that balance accuracy against computation. The results show that moderate or adaptive stopping can preserve competitive accuracy while substantially reducing compute.
- Motivation: Current test-time scaling evaluations often report accuracy across budgets without accounting for inference cost.Generating 16,000 tokens costs 32× more than generating 500 tokens.
- Evaluation Framework: The framework extends evaluation beyond accuracy curves by comparing accuracy-compute trade-offs and reporting efficiency frontiers.It asks how long the model should think rather than whether it should answer at all.
- Efficiency Metrics: The utility function balances accuracy against normalized compute, with λ controlling sensitivity to cost.λ=0 is cost-agnostic, λ=0.5 is cost-balanced, and λ=1.0 is cost-sensitive.
- Main Results: As cost sensitivity increases, optimal stopping moves from peak-accuracy budgets toward substantially shorter budgets.The Pareto frontier and utility curves shift stopping points leftward as λ increases.
- Main Results: At λ=0.5, stopping at ∼6K tokens yields ∼50% compute reduction with only ∼6% accuracy loss.At λ=1.0, the favored budget is ∼2K tokens.
- Early Stopping Validation: Indicator-based early stopping achieves 97% of peak accuracy while using only 60% of compute.Figure 7 reports that the combined indicator-based approach reduces compute while maintaining competitive accuracy against fixed token limits.
6 Conclusion
The paper finds that test-time compute scaling has diminishing marginal utility at high budgets and can produce overthinking, where models abandon correct answers after extended reasoning. It therefore advocates evaluating efficiency alongside accuracy.
- Conclusion: Marginal utility decreases substantially at high test-time compute budgets.
- Conclusion: Overthinking occurs when models abandon correct answers after extended reasoning.
- Conclusion: Efficiency frontiers should be reported alongside accuracy curves to capture accuracy-compute trade-offs.
Limitations
The study’s evidence is bounded by its task coverage, model coverage, and limited causal analysis of overthinking. These boundaries leave open whether the observed patterns generalize across domains, proprietary systems, and mechanisms.
- Scope: The analysis focuses on mathematical and scientific reasoning tasks, so overthinking may manifest differently in other domains.
- Scope: The evaluation uses only open-weight models, and proprietary systems may exhibit different overthinking patterns.
- Causal Analysis: Although 67.5% of negative flips suggest genuine reconsideration behavior, definitive causal mechanisms require controlled interventions.
Ethics Statement
The paper frames its efficiency analysis as having positive ethical implications because identifying harmful overthinking may reduce unnecessary computation during LLM inference.
- Ethics Statement: Reducing unnecessary inference computation could lower energy consumption and carbon emissions associated with LLM reasoning.
A Natural Long Reasoning Analysis
Natural long reasoning is associated with harder, less accurate problems and frequent answer reconsideration. These patterns provide evidence that model-generated extra thinking can coincide with uncertainty and overthinking.
- Sample Selection: 312 of 560 samples (55.7%) naturally exceeded 8K tokens, and 78% of these were Level 4–5 MATH-500 problems.Samples were collected from AIME and MATH-500 using unconstrained R1-32B generation.
- Accuracy by Natural Length: Longer natural outputs correlate with lower accuracy, suggesting models generate more tokens when problems are difficult or uncertain.The paper reports this pattern in the accuracy stratification by natural output length.
- Second-Guessing Behavior: Samples with explicit reconsideration have 12% lower accuracy than samples without it, providing evidence for overthinking.The comparison links answer reconsideration with reduced accuracy within the natural long-reasoning samples.