PaperlayerView exact revision on arXiv

Source-linked AI summary

Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves Reasoning Efficiency

Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, Tianyi Zhou

arXiv:2506.08343v2cs.CL

TL;DR

Large reasoning models can overthink through verbose and redundant self-reflection, raising efficiency concerns. NOWAIT suppresses reflection-keyword generation during inference, and experiments across ten textual, visual, and video benchmarks report shorter reasoning trajectories while preserving overall model utility.

  • Problem

    Long reasoning trajectories can contain redundant self-reflection that increases computational overhead and latency, motivating whether explicit “Wait”-like tokens are necessary for advanced reasoning.

  • Method

    NOWAIT is a training-free inference method that identifies reflection-keyword tokens and suppresses them by assigning their logits large negative values.

  • Results

    Across ten textual, visual, and video benchmarks and five R1-style model series, NOWAIT reduces chain-of-thought trajectory length by up to 27%-51%.

  • Takeaways & Limitations

    NOWAIT provides a plug-and-play approach for efficient multimodal reasoning while preserving overall model utility.

  • Takeaways & Limitations

    Existing benchmarks cannot comprehensively exhibit models’ reasoning capabilities from all aspects.

Abstract

from arXiv · show

Recent advances in large reasoning models have enabled complex, step-by-step reasoning but often introduce significant overthinking, resulting in verbose and redundant outputs that hinder efficiency. In this study, we examine whether explicit self-reflection, signaled by tokens such as "Wait" and "Hmm", is necessary for advanced reasoning. We propose NoWait, a simple yet effective approach that disables explicit self-reflection by suppressing these tokens during inference. Extensive experiments on ten benchmarks across textual, visual, and video reasoning tasks show that NoWait reduces chain-of-thought trajectory length by up to 27%-51% in five R1-style model series, without compromising model utility. NoWait thus offers a plug-and-play solution for efficient and utility-preserving multimodal reasoning.

1 Introduction

Large reasoning models can produce long, self-reflective chains of thought that overthink problems and increase computational cost. NOWAIT suppresses reflection keywords during inference to shorten reasoning while preserving model utility across textual, visual, and video tasks.

  • 1 Introduction: Long chain-of-thought trajectories can become excessively verbose and redundant, increasing computational overhead and reasoning latency.These inefficiencies hinder practical deployment in applications with limited computational resources.
  • 1 Introduction: Existing efficient-reasoning methods often require additional training, while some training-free approaches reduce overall model utility.The paper motivates a training-free approach that avoids this trade-off.
  • 1 Introduction: NOWAIT suppresses reflection keywords such as “Wait”, “Hmm”, and “Alternatively” during inference to disable explicit self-reflection.The method intervenes directly in decoding rather than retraining the model.
  • 1 Introduction: NOWAIT adjusts the logits of reflection-associated tokens to negative values, steering decoding toward alternative tokens.This is the mechanism used to prevent those tokens from being generated.
  • 1 Introduction: 10 benchmarks span textual, visual, and video reasoning, while five R1-style model series show chain-of-thought reductions of up to 27%-51%.The reported result covers QwQ, Phi4, Qwen3, Kimi-VL, and QvQ model series.

2 Preliminaries

Reasoning-model outputs combine a chain-of-thought trajectory with a final summary, and the trajectory can contain multiple reasoning chunks linked to intermediate answers. Transitions between chunks are often marked by reflection keywords, which NOWAIT targets to reduce unproductive reasoning loops.

  • 2 Preliminaries: Reasoning models use <think> and <\think> delimiters to separate a chain-of-thought trajectory from a final answer summary.The trajectory details the reasoning process, while the summary states the overall thought.
  • 2 Preliminaries: A complete chain of thought can contain multiple thinking chunks, each pairing reasoning text with an intermediate answer.Models may use forward thinking, backtracking, self-reflection, and additional validation after an initial result.
  • 2 Preliminaries: Transitions between reasoning chunks often use keywords such as “Wait”, “Alternatively”, or “Hmm” to verify results or explore alternative paths.These transitions can sometimes initiate redundant validation loops and unproductive overthinking.
  • 2 Preliminaries: NOWAIT alters models’ self-reflection strategies and generalizes the intervention across different modalities.The method operates on the generation of the reflection-associated transitions.

3 Removing Thinking Pattern is Better

NOWAIT removes explicit reflection-associated tokens during inference, targeting redundant reasoning without changing model architecture or weights. Across textual, visual, and video reasoning, it reduces output length while generally preserving reasoning accuracy, though some multimodal and non-mathematical settings show modest drops.

  • Method: NOWAIT suppresses reflection-associated keywords during inference using a logit processor, without additional training or changes to model architecture or weights.The method identifies keyword variants across each model’s vocabulary, then assigns targeted tokens a large negative logit.
  • Textual reasoning: NOWAIT consistently reduces textual reasoning output length while maintaining reasoning accuracy across diverse models and benchmarks.The evaluation covers multiple model architectures, parameter scales, and math benchmarks.
  • Model generalization: 4.25 percentage points higher AMC 2023 accuracy and output length at 70% of baseline are reported for QwQ-32B with NOWAIT.Phi4-Reasoning-Plus gains 6.00 percentage points with 28% fewer generated tokens, while Qwen3-32B reduces output length by 13% with a marginal accuracy decrease.
  • Task difficulty: 31% lower token usage accompanies a 1.33% point AIME 2025 accuracy increase for QwQ-32B on a challenging benchmark.Across difficulty levels, tested models achieve comparable token reductions; Qwen3-32B reduces output length by 14% to 16%, and Phi4-Reasoning-Plus by 23% to 28%.
  • Comparison analysis: NOWAIT outperforms Token-Budget’s LRM generalization and avoids the severe performance degradation reported for O1-Pruner on QwQ-32B-Preview.Unlike those comparison methods, NOWAIT requires no additional training or data and balances output length with reasoning accuracy.
  • Multimodal reasoning: 49% average generation-length reduction for Kimi-VL-A3B-Thinking comes with a 3.42 percentage-point overall accuracy drop across four image benchmarks.QvQ-72B-Preview reaches up to 30% lower token usage with accuracy decreases ranging from 0.11% to 4.00%; video reasoning also shows substantial length reductions with comparable accuracy.
  • Interpretation: Suppressing reflection keywords is presented as a way to raise the model’s self-reflection threshold and reduce unnecessary verification steps.The paper attributes inefficient reasoning to models reflecting too readily during inference.

4 Discussion

The discussion examines NOWAIT through case studies, multimodal robustness, and differences between RL-based and distilled Qwen3 models. NOWAIT produces more concise reasoning with necessary checks, while distilled models become increasingly sensitive to reflection-token removal as task difficulty rises.

  • 4.1 Why does NOWAIT Work?: NOWAIT preserves necessary self-reflection while avoiding repeated checks and redundant derivations, producing a more concise reasoning process than the original CoT.In the AMC 2023 case study, NOWAIT checks an extraneous root and verifies the equations, whereas the original CoT repeats essentially the same algebra five times.
  • 4.1 Why does NOWAIT Work?: NOWAIT organizes video reasoning more coherently by connecting detailed temporal observations directly to the final answer instead of repeatedly branching with “Wait”.The original CoT contains six self-reflection instances, while NOWAIT uses time-sequence cues such as “starts” and “Towards the end of the video”.
  • 4.2 A Closer Look at RL Models Performance: Minimal accuracy divergence from the baseline across MMMU disciplines indicates robust QvQ-72B-Preview performance under NOWAIT.The passage reports closely aligned baseline and NOWAIT accuracy across diverse academic and professional subjects.
  • 4.3 Distilled Models Cannot Reasoning without “Wait”: Distilled Qwen3 models show increasing accuracy degradation as benchmark difficulty rises, whereas the RL-based Qwen3-32B remains consistent.The benchmarks are ordered AMC 2023 < AIME 2024 < AIME 2025; the distilled-model gap exceeds 5 percentage points on AIME 2024 and 12 points on AIME 2025.
  • 4.3 Distilled Models Cannot Reasoning without “Wait”: Distilled models are more sensitive to reflection keywords because removing them can disrupt CoT structure and restrict validation on challenging problems.The authors characterize this as substantial underthinking, contrasting with the stable performance of the RL-based model.

5 Conclusion

NOWAIT suppresses explicit self-reflection tokens during inference to reduce overthinking and shorten chain-of-thought trajectories. Experiments across models and textual, visual, and video benchmarks support its utility as an efficient reasoning approach.

  • NOWAIT reduces overthinking and shortens chain-of-thought trajectories by suppressing explicit self-reflection tokens during inference.
  • Experiments across diverse models and textual, visual, and video reasoning benchmarks demonstrate NOWAIT’s efficient reasoning utility.

Limitation

The paper acknowledges that existing benchmarks do not comprehensively capture models’ reasoning capabilities from all aspects. It situates this boundary alongside the broader efficiency challenge posed by verbose reasoning traces and prior efficiency methods.

  • Existing benchmarks cannot comprehensively exhibit models’ reasoning capabilities from all aspects.
  • Long reasoning traces increase inference latency and computational cost, motivating methods that reduce reasoning length without compromising accuracy.
  • Prior efficient-reasoning approaches include additional training, dynamic inference strategies, and prompts designed to guide shorter reasoning.
  • Self-reflection markers such as “wait” and “hmm” can mark transitions between reasoning chunks used to verify results or explore alternatives, but may produce redundant validation loops.

B Baseline Implementation Details

The baseline implementation details describe NoThinking, TokenBudget, and O1-Pruner as comparison methods. NoThinking uses prompting and budget forcing to bypass reasoning and produce a final response.

  • The experiments compare NOWAIT with NoThinking, TokenBudget, and O1-Pruner.
  • NoThinking uses prompts to make reasoning models skip reasoning and directly generate a final response.
  • The NoThinking prompt template ends with “Okay, I think I have finished thinking. <\think>”.
  • NoThinking uses a 10,000-token budget and forces models to generate the Final Answer when that budget is reached.

B.2 Token-Budget

This section describes prompt-based token budgeting and contrasts it with post-training approaches for reducing reasoning length.

  • The section includes prompt templates for token-budget estimation and multiple-choice answering.
  • TALE-EP asks the reasoning model to estimate the minimum tokens needed for a complete and accurate response.The requested output format is Budget: [[12]].
  • TALE-EP then includes the estimated token budget in the prompt to guide efficient thinking.
  • O1-Pruner is presented as an effective post-training method based on a released QwQ-32B-Preview model.

D.1 Textual QA

The paper evaluates textual, visual, and video reasoning using diverse benchmarks and models. These evaluations cover mathematics, domain-expert questions, multimodal academic tasks, and video-based reasoning.

  • Textual QA: Textual evaluation covers AIME2024, AIME2025, AMC2023, and GPQA-Diamond for mathematical and domain-expert reasoning.GPQA-Diamond spans biology, physics, and chemistry and is described as its dataset’s hardest subset.
  • Textual QA: The textual experiments measure models including Qwen3-32B, QwQ-32B, and Phi4-Reasoning-Plus.
  • Visual QA: Visual evaluation uses MMMU, MMMU-Pro, Math-Vista, and EMMA-mini to assess academic, mathematical, and cross-modal reasoning.The benchmarks require models to process visual and textual information across disciplines and tasks.
  • Visual QA: The visual experiments include Kimi-VL-A3B-Thinking and QvQ-72B-Preview models.
  • Video QA: Video evaluation uses MMVU and VSI-Bench for expert-level domain-specific video understanding and visual-spatial reasoning.MMVU includes expert-annotated reasoning rationales and relevant domain knowledge.

E Additional Experiment Results & Case Study

The additional results and case studies present benchmark visualizations and examples of reasoning with and without NOWAIT. The examples illustrate shorter reasoning that retains targeted verification.

  • Additional Experiment Results: The section includes complete NOWAIT experiment results for Qwen3-series and other distilled models.
  • Additional Experiment Results: Accuracy radar maps are shown for QvQ-72B-Preview on VSI-Bench and Kimi-VL-A3B-Thinking-7B on MMMU across multiple domains.The listed domains include Electronics, Energy, Finance, Geography, and History.
  • Case Study: A video case study depicts a laboratory or classroom demonstration involving a pendulum-like apparatus.
  • Case Study: Adding an obstacle or another object may transfer pendulum energy and reduce the pendulum’s energy.
  • Case Study: In the algebra example, the reasoning verifies both equations for x = 27, y = 9 and obtains x + y = 36.
  • Case Study: NOWAIT’s example retains self-reflection for discarding an extraneous root and quickly verifying the equations, while the original reasoning includes additional reconsideration.
Loading 2506.08343v2…