PaperlayerView exact revision on arXiv

Source-linked AI summary

Reasoning Models Can Be Effective Without Thinking

Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, Matei Zaharia

arXiv:2504.09858v1cs.AIcs.CL

TL;DR

The paper asks whether lengthy explicit thinking is necessary for strong reasoning performance. It evaluates a simple NoThinking prompt and combines it with parallel scaling, finding competitive or superior accuracy–cost and latency–accuracy tradeoffs across diverse reasoning tasks. The results support reconsidering lengthy thinking as a requirement for high performance, while benchmark-specific behavior remains an important scope boundary.

  • Problem

    The paper investigates whether the explicit, lengthy Thinking process used by current reasoning models is necessary for strong performance.

  • Method

    The paper bypasses explicit thinking with NoThinking and evaluates it across reasoning benchmarks, including parallel sampling with best-of-N aggregation and task-specific verification.

  • Results

    Across diverse datasets, NoThinking matches or outperforms Thinking in pass@k and achieves better accuracy–latency tradeoffs, while using fewer tokens.

  • Takeaways & Limitations

    NoThinking provides a competitive reference for strong reasoning in low-budget or low-latency settings and motivates reconsidering lengthy thinking processes.

  • Takeaways & Limitations

    Performance differences vary by benchmark, with some tasks showing an initial pass@1 disadvantage or low scores across methods.

Abstract

from arXiv · show

Recent LLMs have significantly improved reasoning capabilities, primarily by including an explicit, lengthy Thinking process as part of generation. In this paper, we question whether this explicit thinking is necessary. Using the state-of-the-art DeepSeek-R1-Distill-Qwen, we find that bypassing the thinking process via simple prompting, denoted as NoThinking, can be surprisingly effective. When controlling for the number of tokens, NoThinking outperforms Thinking across a diverse set of seven challenging reasoning datasets--including mathematical problem solving, formal theorem proving, and coding--especially in low-budget settings, e.g., 51.3 vs. 28.9 on ACM 23 with 700 tokens. Notably, the performance of NoThinking becomes more competitive with pass@k as k increases. Building on this observation, we demonstrate that a parallel scaling approach that uses NoThinking to generate N outputs independently and aggregates them is highly effective. For aggregation, we use task-specific verifiers when available, or we apply simple best-of-N strategies such as confidence-based selection. Our method outperforms a range of baselines with similar latency using Thinking, and is comparable to Thinking with significantly longer latency (up to 9x). Together, our research encourages a reconsideration of the necessity of lengthy thinking processes, while also establishing a competitive reference for achieving strong reasoning performance in low-budget settings or at low latency using parallel scaling.

1 Introduction

The paper questions whether lengthy explicit thinking is necessary for advanced reasoning and introduces NoThinking, which bypasses that process through simple prompting. Across diverse reasoning tasks, NoThinking offers strong accuracy–cost and latency–accuracy tradeoffs, especially with low token budgets and parallel sampling.

  • Recent reasoning models improve performance by generating long chains of thought involving reflection, backtracking, and self-validation before final answers.
  • NoThinking disables explicit thinking by prefilling a fabricated Thinking block and letting the model directly generate the solution.
  • The evaluation covers mathematical problem solving, coding, and formal theorem proving across diverse benchmarks.
  • NoThinking consistently outperforms Thinking on the pass@k-versus-token-usage Pareto frontier and often achieves higher low-budget pass@1 accuracy.
  • NoThinking can be combined with parallel sampling and best-of-N aggregation, including task-specific verification where available.
  • The work concludes that explicit thinking is not necessarily required for high performance and positions NoThinking as a competitive approach for low-budget or low-latency reasoning.

2 Related Work and Background

Prior test-time scaling methods largely rely on sequential generation of long, structured chains of thought, while efficiency work seeks to reduce the cost of such reasoning. This paper instead argues that disabling explicit thinking can improve the accuracy–budget tradeoff without additional training or process supervision.

  • Sequential test-time scaling methods generate long, structured chain-of-thought responses with backtracking and verification in a single forward pass.
  • NoThinking improves the accuracy–budget tradeoff without additional training, reward signals, or process supervision.
  • Efficiency research seeks to reduce reasoning costs by optimizing generated reasoning length or eliminating unnecessary steps.

3 NoThinking Provides Better Accuracy-budget Tradeoffs than Thinking

The paper evaluates whether explicit Thinking is necessary and finds that NoThinking offers strong accuracy-budget tradeoffs across challenging reasoning tasks, especially as the sampling budget grows or remains low.

  • 3.1 Thinking and NoThinking: NoThinking bypasses explicit reasoning by forcing an empty thinking box, directly generating the final solution and answer.Thinking instead produces the reasoning process, final solution, and final answer within the model’s structured format.
  • 3.3 Results: NoThinking matches Thinking on MiniF2F and ProofNet while using 3.3–3.7x fewer tokens.Both methods significantly outperform Qwen-Instruct on these theorem-proving benchmarks.
  • 3.3 Results: Across datasets, NoThinking eventually matches Thinking at the largest k while using 2.0–5.1x fewer tokens.At k = 1, NoThinking can lag behind Thinking, but the gap narrows as k increases; Qwen-Instruct closes the gap on AMC23 and OlympiadBench.
  • 3.3 Results: Under comparable low budgets, NoThinking consistently outperforms Thinking across all k, with the gap widening as k increases.At around 3,500 tokens, Thinking is better at pass@1, but NoThinking surpasses it from k = 2.
  • 3.3 Results: With token usage plotted against pass@1 and largest-k pass@k, NoThinking outperforms Thinking across the full budget range for pass@k.For pass@1, NoThinking is better at low budgets and worse at high budgets; LiveCodeBench is an exception in the low-budget regime.
  • 3.4 Discussions and Analyses: NoThinking’s lower variance in answer diversity may contribute to improved pass@k performance as k increases, although diversity alone does not fully explain the differences.Neither mode has a consistent advantage in mean entropy, but NoThinking consistently shows lower variance across questions.

4 NoThinking Makes Parallel Test-Time Compute More Effective

The paper uses parallel scaling with NoThinking to generate independent responses concurrently and aggregate them, improving accuracy–latency tradeoffs across verifier-based and verifier-free tasks.

  • Methods: Parallel scaling generates N independent responses concurrently and selects a final prediction with a best-of-N aggregation method.Perfect verifiers can identify correct outputs directly; verifier-free tasks use confidence-based or voting strategies.
  • Methods: Latency is measured as the average maximum number of tokens generated across N trials and the dataset.The metric reflects response speed because concurrent generations finish when the longest individual generation ends.
  • Perfect Verifiers: 7× lower latency lets parallel NoThinking match the accuracy of Thinking with budget forcing and parallel scaling on tasks with perfect verifiers.On MiniF2F and ProofNet, NoThinking also reaches the same accuracy with 4× fewer output tokens.
  • Simple Best-of-N Methods: NoThinking with parallel scaling and confidence-based selection consistently outperforms Thinking across most benchmarks under low token budgets.Parallel scaling improves pass@1 for both methods, while NoThinking dominates the Pareto frontier across all math benchmarks.
  • Simple Best-of-N Methods: 55.79 vs. 54.1 pass@1 gives NoThinking higher accuracy than full Thinking while reducing latency by 9×.This comparison uses Thinking without budget forcing and demonstrates a stronger accuracy–latency tradeoff.
  • Simple Best-of-N Methods: LiveCodeBench is an outlier where NoThinking is less effective because highest-confidence selection is less reliable without exact-match outputs.Voting-based methods generally outperform this fallback when voting is applicable.

5 Conclusion

NoThinking bypasses explicit thinking while preserving competitive reasoning performance, using fewer tokens and improving accuracy–latency tradeoffs when combined with best-of-N selection.

  • NoThinking can perform comparably to or better than Thinking in pass@k as k increases, while using significantly fewer tokens.Under comparable token budgets, it outperforms traditional Thinking at most values of k.
  • NoThinking combines with best-of-N selection to achieve better accuracy–latency tradeoffs than standard Thinking.
  • The authors present NoThinking as a competitive reference for strong reasoning performance in low-budget and low-latency settings.

A Method Details

The method compares Thinking and NoThinking under similar token usage by enforcing a generation budget during the initial model call. Truncated Thinking generations are then prompted to produce a final answer directly.

  • A maximum-token limit is set during the initial model call to enforce a comparable generation budget.
  • If Thinking reaches the limit without a stop token, the reasoning is treated as truncated.
  • For truncated Thinking outputs, an end-of-thinking tag and final-answer prefix prompt direct the model to produce the final answer.

B.1 Results on Tasks with Verifiers

The verifier-task appendix reports token costs for matching full Thinking at pass@1 and examines NoThinking across smaller R1-series models and matched budgets.

  • Table 3 reports cumulative total tokens used by NoThinking to match full Thinking at pass@1 on tasks with verifiers.
  • On the 14B model, NoThinking initially lags on AIME tasks but rapidly catches up as k increases, approaching Thinking by k = 64.
  • Under budget forcing, NoThinking outperforms Thinking from the beginning on the 7B model and eventually catches up on the 14B model.

B.3 Best-of-N Methods Comparison

Table 4 summarizes best-of-N methods across the experiments as an extended version of Table 2.

  • Table 4 presents results for various best-of-N methods.
  • The comparison covers all experiments.
  • Table 4 extends the presentation in Table 2.

C Example Generations for Different Tasks

The paper presents complete Thinking and NoThinking traces across math, coding, and theorem-proving examples, alongside comparisons of their performance and selection methods.

  • Complete model traces compare Thinking and NoThinking on math problem solving, coding, and formal theorem proving.
  • Figure 8 compares Thinking, NoThinking, and same-size Qwen-Instruct models without controlling the token budget.
  • Figure 9 compares Thinking and NoThinking under controlled token budgets and reports that NoThinking consistently outperforms Thinking as k increases.
  • Best-of-N methods include majority voting, confidence-based highest selection, and confidence-based voting under different budget-forcing token budgets.
  • Pass@k is reported as an upper bound where k denotes the number of samples used in Best-of-N selection.

C.1 AIME 2024

The AIME 2024 example shows a long Thinking trace that explores several algebraic transformations and numerical approximations before reaching the product xy = 25.

  • The trace introduces a = log_x(y), uses reciprocal logarithms, and rewrites the equations in terms of x, y, and a.
  • The derivation obtains y = 25/x and therefore xy = 25, while explicitly checking whether the transformation remains consistent with the original equations.
  • The numerical search evaluates f(a) at several values and brackets a zero between a = 6.52 and a = 6.53.
  • The final solution introduces u = ln x and v = ln y, derives ln(xy) = ln(25), and concludes xy = 25.

C.2 LiveCodeBench

The LiveCodeBench example solves whether a three-character permutation can become abc using at most one swap by checking the permitted transformations directly.

  • The task asks whether a row containing a, b, and c can become abc after at most one swap.
  • The proposed method first accepts abc, then checks every pair of indices and accepts if any swapped string equals abc.
  • Reasoning based only on mismatch counts is rejected after recognizing that cba has three mismatches but becomes abc by swapping its first and last characters.
  • The implementation is feasible because a three-character string has only three possible swaps.
  • The corrected sample outcomes are YES for abc, acb, bac, and cba, and NO for bca and cab.

C.3 ProofNet

The ProofNet example proves that every element of a finite group has a positive power equal to the identity by exploiting repetition among its powers.

  • For an element a in a finite group G, the sequence a, a^2, a^3, ... must eventually repeat.
  • The pigeonhole principle yields distinct exponents i and j with a^i = a^j, assuming i > j.
  • Multiplying by the inverse power gives a^(i−j) = e, so a positive integer n exists with a^n = e.
  • The smallest positive exponent satisfying a^n = e is identified as the order of a in G.
  • The Lean formalization uses Fintype.card G and the theorem pow_card_eq_one to establish the existential statement.
Loading 2504.09858v1…