PaperlayerView exact revision on arXiv

Source-linked AI summary

Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning

Michael Hassid, Gabriel Synnaeve, Yossi Adi, Roy Schwartz

arXiv:2505.17813v2cs.CLcs.AI

TL;DR

Reasoning LLMs often scale test-time compute with long thinking chains, increasing inference cost and latency. This paper compares reasoning-chain lengths, introduces short-m@k, and finds that shorter chains can improve accuracy while reducing computation and time. It also reports that training on shorter trajectories improves model performance.

  • Problem

    Long reasoning chains are costly and slow, while it remains uncertain whether generating more tokens per trajectory reliably improves reasoning performance.

  • Method

    The paper compares shortest, longest, and random chains, then proposes short-m@k, which parallelizes k generations and stops after the first m thinking chains finish.

  • Results

    Across four benchmarks, shortest chains outperform longest chains by up to 34.5%, while short-1@k and short-3@k outperform or match majority voting with lower compute or wall time.

  • Takeaways & Limitations

    The findings support prioritizing shorter reasoning trajectories for more efficient and accurate reasoning LLM inference and training.

  • Takeaways & Limitations

    The analysis excludes generations whose thinking is unfinished at the maximum generation length, because these often produce infinite thinking loops.

Abstract

from arXiv · show

Reasoning large language models (LLMs) heavily rely on scaling test-time compute to perform complex reasoning tasks by generating extensive "thinking" chains. While demonstrating impressive results, this approach incurs significant computational costs and inference time. In this work, we challenge the assumption that long thinking chains results in better reasoning capabilities. We first demonstrate that shorter reasoning chains within individual questions are significantly more likely to yield correct answers - up to 34.5% more accurate than the longest chain sampled for the same question. Based on these results, we suggest short-m@k, a novel reasoning LLM inference method. Our method executes k independent generations in parallel and halts computation once the first m thinking processes are done. The final answer is chosen using majority voting among these m chains. Basic short-1@k demonstrates similar or even superior performance over standard majority voting in low-compute settings - using up to 40% fewer thinking tokens. short-3@k, while slightly less efficient than short-1@k, consistently surpasses majority voting across all compute budgets, while still being substantially faster (up to 33% wall time reduction). To further validate our findings, we finetune LLMs using short, long, and randomly selected reasoning chains. We then observe that training on the shorter ones leads to better performance. Our findings suggest rethinking current methods of test-time compute in reasoning LLMs, emphasizing that longer "thinking" does not necessarily translate to improved performance and can, counter-intuitively, lead to degraded results.

1 INTRODUCTION

The paper challenges long test-time reasoning as a reliable path to better performance, finding that shorter chains can improve accuracy while reducing computational cost. It introduces short-m@k and reports efficiency and performance gains, alongside analyses of reasoning trajectories and training data.

  • Motivation and findings: Up to 34.5% higher accuracy comes from selecting the shortest chain rather than the longest chain for the same question.Shortest chains are also 67% shorter than longest chains.
  • The short-m@k method: short-m@k runs k generations in parallel and stops all processes when the first m thinking chains finish, then votes among those chains.Ties are resolved using the shortest answer among tied candidates.
  • Empirical results: In low-compute settings, short-1@k can match majority voting while reducing compute by up to 40%.The reported example uses LN-Super-49B.
  • Trajectory analysis: Shorter reasoning paths are associated with fewer backtracks and longer average backtrack lengths, while longer reasoning remains useful for harder questions.The backtrack pattern holds both generally and when controlling for overall trajectory length.
  • Training on shorter chains: Finetuning on shorter reasoning trajectories produces shorter thinking and improves model performance.The experiments use S1-short, S1-long, and S1-random variants with Qwen-2.5 models.
  • Conclusion: The paper concludes that longer thinking does not ensure better reasoning and can lead to worse reasoning in most cases.It presents shorter reasoning as a route to improved performance and reduced computational costs.

2 RELATED WORK

Prior work established test-time scaling and post-training methods for reasoning LLMs, while also highlighting the computational cost of long reasoning trajectories. Related studies examined efficiency techniques and the relationship between trajectory length and correctness, motivating this paper’s shorter-thinking approach.

  • Reasoning LLMs and test-time scaling: Reasoning LLMs commonly use chain-of-thought trajectories and many thinking tokens to explore approaches and self-reflect.The related work frames these trajectories as central to reasoning LLMs.
  • Post-training methods: Reinforcement learning and supervised fine-tuning are the two primary approaches described for improving reasoning ability.Prior work linked longer training trajectories with improved performance during training.
  • Efficiency in reasoning LLMs: Efficiency research has explored early exits, backtrack suppression, and training methods to reduce the cost of reasoning trajectories.These approaches target the large token requirements of reasoning LLMs.
  • Trajectory length and correctness: Prior studies report that correct responses are often shorter for a given question and that optimal thinking length depends on question difficulty.The paper positions its analysis as further support for these observations.
  • Relation to prior methods: The paper’s short-m@k method generalizes shortest-answer selection by combining early termination with majority voting among short candidates.The related work connects it specifically to the FFS method and short-1@k.

3 SHORTER THINKING IS PREFERABLE

This section tests whether shorter reasoning chains are preferable by comparing shortest, longest, and random generations from several reasoning LLMs across competitive benchmarks. The reported results show shorter chains are substantially more efficient and generally more accurate.

  • Experimental question: The study compares shortest, longest, and randomly sampled thinking chains for the same questions to test whether more tokens improve performance.This directly challenges the assumption that longer reasoning correlates with better task performance.
  • Models and benchmarks: The experiments use four high-performing open reasoning LLMs and four competitive benchmarks, including AIME 2024, AIME 2025, HMMT February 2025, and GPQA-diamond.The mathematical datasets contain 30 examples each with varied difficulty.
  • Generation setup: Each model generates 20 responses per question with temperature 0.7, top-p=0.95, and a 32,768-token maximum.Thinking-chain length is measured by tokens between the <think> and </think> tokens.
  • Results: Shortest responses are 25%–50% shorter than randomly sampled responses and improve average math-benchmark performance by 2.2%–15.7% over random generations.The passage reports this pattern across almost all models and benchmarks.
  • Scope condition: The analysis excludes generations whose thinking is unfinished at the maximum length because they often produce infinite thinking loops.This is an explicit scope condition for the section’s comparisons.
  • Interpretation: The results suggest that shorter trajectories are more likely to be correct while also reducing running time and computational cost.The section uses these findings to motivate efficiency-oriented strategies.

4 short-m@k: FASTER AND BETTER INFERENCE OF REASONING LLMS

short-m@k evaluates parallel reasoning generations by stopping after the shortest completed chains and aggregating their answers. Across controlled sample-size, compute, and time-to-answer evaluations, short-3@k generally offers the strongest performance-efficiency balance, while short-1@k is especially effective at low compute.

  • Method: short-m@k runs k generations in parallel, stops when the m shortest thinking trajectories finish, and majority-votes their answers.Ties are resolved by selecting the answer with the shortest thinking chain.
  • Sample-size: All methods generally improve as sample size k increases, including short-m@k, because shorter sampled trajectories remain more likely to be correct.QwQ-32B is an exception, showing a small decline for short-1@k at larger sample sizes.
  • Sample-size: short-3@k dominates across nearly all models and sample sizes, while short-1@k is stronger than majority@k mainly at smaller sample sizes.For R1-670B, short-3@k performs nearly on par with the oracle across sample sizes.
  • Thinking-compute: 57% accuracy is achieved by short-1@k with approximately 60% of majority@k's compute for LN-Super-49B.short-1@k outperforms majority@k at lower compute budgets, while short-3@k achieves higher performance with lower thinking compute across all models and budgets.
  • Time-to-answer: As k increases, majority@k becomes slower, whereas short-1@k and short-3@k become faster because more samples increase the probability of finding short chains.The methods therefore reduce time cost without a major loss in performance.
  • Time-to-answer: 33% faster time-to-answer is achieved by short-3@k than majority@k on R1-32B at k = 5, while reaching superior performance.short-1@k also reduces LN-Super-49B time consumption by almost 50% at sample size 5 and more than 55% at sample size 9.

5 ANALYSIS

The analysis examines why shorter reasoning trajectories tend to be more accurate while harder questions still require more thinking. It also finds that short-m@k remains more efficient and effective than majority voting under sequential decoding.

  • 5.1 HARD QUESTIONS (STILL) REQUIRE MORE THINKING: Correct answers are typically shorter than incorrect answers within every difficulty split, including both easier and harder questions.
  • 5.1 HARD QUESTIONS (STILL) REQUIRE MORE THINKING: Harder questions require longer thinking than easier questions, with models using up to 2.9× more tokens.
  • 5.2 BACKTRACK ANALYSIS: Correct trajectories consistently contain fewer backtracks than incorrect trajectories, and their backtracks are usually longer.
  • 5.2 BACKTRACK ANALYSIS: Even at comparable trajectory lengths, correct answers generally show fewer backtracks, with R1-670B on the math benchmarks as the stated exception.
  • 5.3 short-m@k WITH SEQUENTIAL COMPUTE: In low-compute sequential decoding, short-1@k and short-3@k are more efficient and perform better than majority voting; at higher compute, short-3@k remains superior.

6 FINETUNING USING SHORTER TRAJECTORIES

The paper tests whether selecting shorter reasoning trajectories for fine-tuning improves model reasoning. Across several benchmarks, S1-short produces shorter thinking and generally better performance than random or long-chain training.

  • The study constructs S1-short, S1-long, and S1-random from the shortest, longest, and randomly selected responses among multiple generations.
  • S1-short achieves superior performance on GPQA-D, AIME 2025, and HMMT while using fewer thinking tokens.
  • S1-short improves aggregated relative performance by 2.8% over S1-random while reducing thinking tokens by 5.8%.
  • S1-long consumes more tokens than S1-random but achieves similar performance.
  • Training on shorter reasoning sequences can produce better reasoning models with reduced computational overhead.

7 CONCLUSION

The paper concludes that longer thinking does not reliably improve reasoning and can worsen performance. It presents shorter reasoning as a basis for more accurate and computationally efficient inference and training.

  • Shorter reasoning chains consistently outperform longer ones in accuracy and computational efficiency across the evaluated benchmarks.
  • short-1@k is preferred to traditional majority voting in low-compute settings, while short-3@k outperforms majority voting across all compute budgets.
  • Shorter thinking usually involves fewer backtracks and a more direct path to the solution.
  • Fine-tuning on short reasoning trajectories improves accuracy and runtime, whereas training on longer chains yields diminishing returns.

A GPQA DIAMOND RESULTS

The GPQA-D appendix presents sample-size, thinking-compute, and time-to-answer comparisons, alongside analyses of thinking tokens and backtracks by answer correctness and difficulty.

  • The GPQA-D results include separate comparisons of sample size, thinking compute, and time to answer.
  • Table 5 reports average thinking tokens for correct, incorrect, and all answers across difficulty splits.
  • Table 6 reports average backtrack counts and average backtrack lengths for correct, incorrect, and all answers in GPQA-D.

B PER BENCHMARK RESULTS

Per-benchmark evaluations organize results by sample size, thinking compute, and time-to-answer across AIME 2024, AIME 2025, and HMMT Feb 2025.

  • Sample size: Sample-size (k) comparisons are reported for AIME 2024, AIME 2025, and HMMT Feb 2025.These results appear in Figures 9 to 11.
  • Thinking compute: Thinking-compute comparisons are reported for AIME 2024, AIME 2025, and HMMT Feb 2025.These results appear in Figures 12 to 14.
  • Time to answer: Time-to-answer comparisons are reported for AIME 2024, AIME 2025, and HMMT Feb 2025.These results appear in Figures 15 to 17.

C ABLATION STUDIES

The ablations examine how short-m@k behaves as the stopping count m and tie-breaking rule change, using LN-Super-49B across three benchmarks under controlled thinking compute.

  • Experimental setup: The ablations use LN-Super-49B and control thinking compute across three benchmarks.The experiments investigate both the value of m and the tie-breaking method.
  • m ablation: short-1@k outperforms larger m values in low-compute regimes but becomes less effective at larger compute budgets.The ablation evaluates m ∈ {1, 3, 4, 5, 7, 9}; larger m values perform similarly and are slightly better in high-compute scenarios.
  • Tie breaking: Choosing the shortest reasoning chain among vote-leading options outperforms random tie-breaking.The paper compares shortest-chain, random, and longest-chain tie-breaking.
  • Tie breaking: Choosing the longest reasoning chain among vote-leading options yields inferior results.This comparison is reported in the tie-breaking ablation.

D SMALL MODELS RESULTS

Experiments with smaller models test short-m@k across math benchmarks and GPQA-D, while additional analyses examine sequential compute, backtracking, token distributions, and finetuning on S1 variants.

  • Shorter generations: Using the shortest answer outperforms random and longest answers across all benchmarks and smaller models.The comparison uses Llama-3.1-Nemotron-Nano-8B-v1 and R1-Distill-Qwen-7B.
  • Benchmark evaluation: Small-model evaluations compare sample size, thinking compute, and time-to-answer over math benchmarks and GPQA-D.Figures 19 to 24 report these comparisons, with Figures 19 to 21 covering math benchmarks and Figures 22 to 24 covering GPQA-D.
  • Inference results: short-1@k has an advantage over majority voting in low-compute regimes, while short-3@k dominates it across all compute budgets.This pattern remains consistent with results observed for larger models.
  • Sequential compute: Sequential-compute results for GPQA-D are presented separately from the sample-size, compute, and time-to-answer comparisons.The sequential-compute analysis is shown in Figure 25.
  • Backtracking: Backtracking is analyzed for correct and incorrect answers binned by thinking length across math benchmarks and GPQA-D.Tables 8 and 9 report average backtrack counts for the two benchmark groups.
  • Finetuning: Finetuning compares S1-short, S1-long, and S1-random variants using Qwen-2.5-7B-Instruct and Qwen-2.5-32B-Instruct.Models are trained on 32 H100 GPUs, evaluated with 20 generated answers per example, and reported by average accuracy.
Loading 2505.17813v2…