Source-linked AI summary
Answer Convergence as a Signal for Early Stopping in Reasoning
Xin Liu, Lu Wang
TL;DR
Chain-of-thought reasoning can produce verbose traces that increase inference cost, and the paper studies whether models reach stable answers before those traces end. It proposes inference-time stopping strategies and reports reduced token use across benchmarks with little or no accuracy loss.
Problem
Chain-of-thought prompting can produce verbose, redundant reasoning traces that increase inference cost.
Source: §1 · p. 1Method
The paper proposes inference-time early stopping based on answer consistency, boosted end-of-reasoning signals, and learned stopping points from internal activations.
Source: §1 · p. 2Results
Across five benchmarks and five open-weights LLMs, early stopping reduces token usage; Learn-to-Stop cuts up to 48% of tokens on NQ with QWQ-32B, sometimes improving accuracy.
Source: §1 · p. 2Takeaways & Limitations
The methods offer a practical alternative to full-chain reasoning without retraining or model changes.
Source: §8 · p. 6Answer convergence does not guarantee correctness, particularly on more complex tasks such as GPQA and AIME, and stopping early may compromise reasoning faithfulness.
Source: Limitations · p. 6
Abstract
from arXiv · showhide
Chain-of-thought (CoT) prompting enhances reasoning in large language models (LLMs) but often leads to verbose and redundant outputs, thus increasing inference cost. We hypothesize that many reasoning steps are unnecessary for producing correct answers. To investigate this, we start with a systematic study to examine what is the minimum reasoning required for a model to reach a stable decision. We find that on math reasoning tasks like math, models typically converge to their final answers after 60\% of the reasoning steps, suggesting substantial redundancy in the remaining content. Based on these insights, we propose three inference-time strategies to improve efficiency: (1) early stopping via answer consistency, (2) boosting the probability of generating end-of-reasoning signals, and (3) a supervised method that learns when to stop based on internal activations. Experiments across five benchmarks and five open-weights LLMs show that our methods significantly reduce token usage with little or no accuracy drop. In particular, on NaturalQuestions, Answer Consistency reduces tokens by over 40\% while further improving accuracy. Our work underscores the importance of cost-effective reasoning methods that operate at inference time, offering practical benefits for real-world applications.
1 Introduction
Chain-of-thought reasoning can become verbose and costly, and the paper investigates whether models settle on answers before their reasoning traces end. It proposes inference-time early-stopping strategies and reports token savings with little or no accuracy loss.
1 Introduction
- Chain-of-thought traces can be unnecessarily long and verbose, increasing inference cost and latency.The passage identifies this overthinking as a practical bottleneck, especially for real-time or resource-constrained scenarios.Source: §1 · p. 1
- Earlier efficiency approaches include training-time optimization and concise-reasoning prompts, while this work explores inference-time techniques.The paper frames its approach as addressing efficiency without sacrificing accuracy.Source: §1 · p. 1
- Models often converge on their answers before completing the full reasoning chain, and this pattern appears across five datasets, including challenging GPQA examples.The authors present this early convergence as evidence that later reasoning steps may be redundant and that early stopping may improve efficiency.Source: §1 · pp. 1–2
- The proposed strategies dynamically truncate reasoning through answer consistency, boosted end-of-reasoning signals, or learned stopping points from internal activations.The methods are described as model-agnostic and requiring no additional training or language-model modification.Source: §1 · p. 2
- Across five benchmarks and five open-weights LLMs, early stopping reduces token usage; Learn-to-Stop cuts up to 48% of tokens on NQ with QWQ-32B, sometimes improving accuracy.Unsupervised methods work well on NQ and GSM8K, while Learn-to-Stop generalizes better to harder tasks such as MATH-500 and GPQA.Source: §1 · p. 2
2 Related Work
Prior work improves chain-of-thought efficiency through training-time optimization, supervised compression, prompting, and routing. The paper instead presents lightweight inference-time methods based on answer convergence.
2 Related Work
- Prior approaches span reinforcement-learning length rewards, supervised trace compression or shorter rationales, concise-reasoning prompts, and routing to efficient paths or backbones.The cited examples include adapting reasoning length to difficulty, enforcing token budgets, and best-of-N self-training.Source: §2 · p. 2
- The paper positions its methods as inference-time alternatives that use answer convergence without additional training, model modifications, large labeled datasets, or task-specific prompts.This contrasts with approaches relying on training-time optimization or application-specific prompt design.Source: §2 · p. 2
- Concurrent DEER uses internal confidence in a trial answer, whereas this work uses output consistency and learned stopping signals from activations.The paper characterizes these approaches as complementary.Source: §2 · p. 2
3 Preliminary
The preliminary study measures how much explicit chain-of-thought is needed before a model’s answer stabilizes. Across tasks, the observed convergence point varies with reasoning demand.
3 Preliminary
- The convergence study tests whether later explicit reasoning steps can be omitted by revealing sentence-level chunks incrementally and checking answers after partial chains.Each partial chain is followed by an end-of-reasoning token, and answers are generated with greedy decoding.Source: §3 · p. 2
- In this paper, “reasoning” primarily means explicit chain-of-thought reasoning, rather than latent reasoning.The stated focus is improving the efficiency of the explicit reasoning process.Source: §3 · p. 2
- Answer Convergence Ratio (ACR) measures the fraction of an explicit reasoning chain required before the predicted answer remains unchanged.The study incrementally reveals sentence-level chunks and identifies the earliest chunk after which the answer stays consistent through the end; seven stable steps out of ten gives an ACR of 0.7.Source: §3 · p. 3
- The reported benchmarks include NQ, GPQA-Diamond, and AIME’24; experiments use the R1-distilled Qwen-32B model, and NQ involves minimal reasoning as an information-seeking task.The other benchmarks are math and logic tasks of increasing difficulty, with AIME’24 containing the most advanced problems.Source: §3 · p. 3
- ACR distributions peak near 0.0 for NQ, 0.8 for GSM8K and MATH-500, and 0.9 for GPQA and AIME’24.The paper says this pattern mirrors increasing reasoning loads and suggests early stopping could reduce inference-time costs without affecting answer quality.Source: §3 · p. 3
4 Early Stopping at Inference Time
The paper proposes three inference-time strategies for stopping reasoning: answer consistency, boosted end-of-reasoning signals, and a learned stopping predictor. They use output consistency or internal activations to limit unnecessary generation without retraining or modifying the base LLM.
4 Early Stopping at Inference Time
- Three inference-time strategies—Answer Consistency, Think Token Adjustment, and Learn-to-Stop—dynamically truncate reasoning without retraining or modifying the base LLM.The first two are unsupervised; Learn-to-Stop is supervised.Source: §4 · p. 3
4.1 Detecting Answer Consistency
- Answer Consistency stops generation when a fixed number of consecutive reasoning chunks produce the same answer.The model checks answers at sentence boundaries by appending </think> and using greedy decoding.Source: §4.1 · p. 3
4.2 Think Token Adjustment
- Think Token Adjustment boosts the </think> logit relative to average vocabulary logits, adapting the boost to output-distribution peakedness.The adaptive boost is intended to encourage earlier termination when the model is confident and reduce premature stopping risk.Source: §4.2 · p. 3
- Figure 3 compares accuracy and token use across five tasks and five model families; Learn-to-Stop has the best efficiency–performance tradeoff.On NQ with R1-Qwen-7B and QwQ-32B, it achieves the highest accuracy and largest token reduction among the baselines.Source: Figure 3 · p. 4
4.3 Learning When to Stop Reasoning
- Learn-to-Stop uses an LSTM to encode final-layer activation sequences and predict stopping points with a sigmoid classifier.Training labels mark chunks from the earliest answer-stable point onward as stops; inference stops when the predicted probability reaches a validation-tuned threshold.Source: §4.3 · p. 4
5 Experimental Setup
The experiments evaluate early-stopping methods on five reasoning tasks using five open-weight LLMs from three model families. Accuracy and token use are compared with the original model and Concise CoT.
5 Experimental Setup
- Experiments use five LLMs across three families: R1-distilled Qwen, R1-distilled Llama, and QwQ.The five tasks are introduced in Section 3; the supplied setup passage does not name them.Source: §5 · p. 4
- The reported measures are accuracy, average generated tokens, and token reduction relative to the original model.Source: §5 · p. 4
- Baselines are the original model without length control and Concise CoT, which constrains reasoning with a fixed prompt token budget.Source: §5 · p. 4
6 Results and Analysis
Learn-to-Stop reduces token use while maintaining accuracy across tasks, whereas unsupervised methods are less stable on harder benchmarks. On low-reasoning-demand NQ, early stopping can match or improve accuracy while reducing tokens.
6 Results and Analysis
- On NQ, all early-stopping methods match or surpass original-model accuracy while greatly reducing tokens; the authors report that stopping can prevent degradation from unnecessary self-reflection or answer revision.Source: §6 · p. 5
- Answer Consistency and Think Token Adjustment reduce tokens without harming accuracy on NQ and GSM8K, but their performance is less stable on MATH-500 and GPQA.The authors suggest output consistency may be insufficient under high reasoning complexity.Source: §6 · p. 5
- 44% fewer tokens (755.1 to 418.9) with a 0.2% accuracy drop on GSM8K with QwQ-32B; on GPQA, Learn-to-Stop cuts tokens 39.2% with comparable performance (24.2% vs. 25.3%).Source: §6 · p. 5
- Learn-to-Stop achieves greater token reductions than prompt-budget CCoT while maintaining or improving accuracy; its unchanged prompt allows combination with CCoT for potential further gains.Source: §6 · p. 5
7 Effect of Model Size
Across the studied Qwen and Llama model sizes, larger models stabilize on answers earlier and achieve greater token savings with Learn-to-Stop.
7 Effect of Model Size
- R1-Llama-70B requires fewer reasoning steps to converge than R1-Llama-8B on NQ and GSM8K; the authors suggest greater capacity may support more efficient internal problem-solving.Source: §7 · p. 5
- 46.2% versus 47.7% token reduction on NQ for R1-Qwen-7B versus R1-Qwen-32B; 34.1% versus 46.4% on R1-Llama-8B versus R1-Llama-70B.Source: §7 · p. 5
8 Conclusion
Across five benchmarks and five open-weight LLMs, answer convergence often occurs before reasoning chains finish, revealing redundancy. The paper proposes three inference-time stopping methods that reduce token use without retraining or model changes.
8 Conclusion
- Across five benchmarks and five open-weight LLMs, answer convergence often occurs early, and the proposed methods cut token use by up to 40% without accuracy loss.The methods stop generation when reasoning is sufficient, offering an inference-time alternative to full-chain reasoning without retraining or model changes.Source: §8 · pp. 5–6
Limitations
The methods reduce inference cost with minimal or no accuracy loss, but their reliability and faithfulness have limits. The authors identify convergence correctness and rationale support as concerns, especially on complex tasks.
Limitations
- Answer convergence does not guarantee a correct final output, particularly on complex tasks such as GPQA and AIME.The methods rely on answer convergence correlating with correctness.Source: Limitations · p. 6
- Stopping before the full reasoning trace may compromise faithfulness; manual analysis found occasional unfaithful stops despite most truncated traces remaining aligned with the answer.The authors call for future work to jointly optimize faithfulness and conciseness.Source: Limitations · p. 6
A.1 ACR Distribution on Various Models
Across models, answer stability occurs before the full reasoning chain is complete, with larger models generally converging in fewer steps. The appendix also examines dataset splits, implementation choices, parameter trade-offs, faithfulness, and examples of early stopping.
A.1 ACR Distribution on Various Models
- Across four models, larger models generally achieve lower ACRs; all show trends similar to the R1-distilled Qwen-32B baseline and produce stable answers before completing the full reasoning chain.Tasks with higher reasoning demands tend to correspond to higher ACRs.Source: §A.1 · p. 7
A.2 Dataset Splits and Statistics
- For NQ and GSM8K, 1,000 examples train the supervised method and 100 serve validation; MATH-500 and GPQA-Diamond reserve 100 test examples, then split the remainder 80/20.AIME’24 is evaluated only with unsupervised methods because it has 30 test examples.Source: §A.2 · pp. 7, 9
A.3 Implementation of Early Stopping via Boosting End of Think Token
- The end-of-reasoning-token processor raises the </think> logit and is disabled after that token is generated; implementation uses VLLM for decoding.The implementation details set answer-consistency k to 10, boosting α to 0.6, and use a single-layer LSTM with 128 hidden units for Learn-to-Stop.Source: §A.3 · p. 9, §A.4 · p. 9
A.5.1 Effect of α (Think Token Adjustment)
- Moderate boosting (α ≤0.4) maintains accuracy on easier tasks, while larger α can cause premature termination and degrade performance on harder tasks.Token use drops as α increases, but α in [0.2, 0.4] offers a reported accuracy–efficiency trade-off; α≥0.6 risks early truncation on hard tasks.Source: §A.5.1 · p. 9, Table 3 · p. 9, Table 4 · p. 9
A.5.2 Effect of k (Answer Consistency)
- For answer consistency, small k can stop too early and hurt accuracy, while accuracy improves and saturates around k∈[20, 30].Larger k delays stopping and increases token use; the curve flattens beyond k ≈ 25−30, and practical settings can use k around 10−20 to balance accuracy and savings.Source: §A.5.2 · pp. 9–10, Table 5 · p. 10, Table 6 · p. 10
A.6 Effect of Early Stopping on Faithfulness
- On GPQA under Learn-to-Stop, 63.6% of analyzed cases stopped late, 18.2% stopped perfectly, and 3% were correct but unfaithful.The analysis was manual and restricted to cases where R1-Qwen-32B originally answered correctly; the authors suggest finer-grained signals could improve stopping criteria.Source: Figure 5 · p. 10, §A.6 · p. 10
A.7 Test Cases from NQ
- In two NQ examples, the original model initially gives the correct answer, then adds reasoning and revises it to an incorrect answer.The examples show Moira Kelly revised to Julie Kavner, and Ashoka revised to Kalinga Nanda.Source: §A.7 · p. 10, Table 7 · p. 11
A.7.1 Case Study
- Table 8 contrasts original-model responses with Learn-to-Stop examples in which reasoning continues after a correct answer, while Learn-to-Stop terminates earlier.The caption identifies examples from MATH-500 and GPQA-Diamond.Source: §A.7.1 · p. 11, Table 8 · p. 12