PaperlayerView exact revision on arXiv

Source-linked AI summary

DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

Minghao Luo, Liang Chen

arXiv:2610.18862v1cs.CLcs.AIcs.LG

TL;DR

Reasoning models can spend many tokens re-deriving or re-verifying answers, while existing shortening methods may require retraining or use fixed or heuristic controls. DoubtLess uses a smaller model’s brief-versus-verbose logit contrast to steer a larger model during reasoning. Across seven benchmarks, it reduces Qwen3-32B’s thinking tokens by 31% with unchanged accuracy, and analyses support reduced redundant checking while retaining useful verification.

  • Problem

    Reasoning chains can spend many tokens re-deriving or re-verifying a result after the correct answer first appears, although some checks can catch errors.

    Source: §1 · p. 1

  • Method

    DoubtLess adds a smaller reasoning model’s scaled brief-versus-verbose logit difference to a larger model’s logits at each chain-of-thought step.

    Source: §1 · p. 2

  • Results

    31% fewer thinking tokens with unchanged accuracy, outperforming five inference-time baselines across seven benchmarks for Qwen3-32B.

    Source: §4.1 · p. 6

  • Takeaways & Limitations

    Analyses support that DoubtLess reduces redundant checking while retaining useful verification, and it also shortens reasoning in a coding agent and other model settings.

    Source: §8 · p. 9

    DoubtLess’s small reasoning model must share the large model’s tokenizer.

    Source: §2.2 · p. 3

Abstract

from arXiv · show

Reasoning models achieve strong performance through long chains of thought (CoT), but often spend more tokens than needed to solve a problem. Existing ways to shorten the CoT require additional training for each model, add a fixed steering vector to the model's hidden states, or use decoding heuristics such as stopping early or banning words like "wait", which can reduce accuracy. We propose DoubtLess, a training-free decoding method in which a much smaller reasoning model steers the large one towards brevity. The small model reads the large model's CoT so far twice, once prompted to be brief (the brief thinker) and once prompted to be verbose (the verbose thinker), and the difference between its two logit vectors, scaled by a strength, is added to the large model's logits at every step of the CoT. On seven benchmarks of competition mathematics, graduate-level science and competitive programming, DoubtLess shortens the CoT of Qwen3-32B by 31% with unchanged accuracy, outperforming five inference-time baselines. We also show that DoubtLess reduces thinking tokens per turn by 20% in a self-debugging coding agent with unchanged task performance. It also generalizes to two other model families with unchanged accuracy. When combined with training-based compression, it further shortens the CoT of a model already trained for brevity. Ablations and behavioral analysis show that DoubtLess reduces repeated checking, with most token savings coming after the correct answer first appears in the CoT. Our code is available at https://github.com/leoluolol/DoubtLess.

1 Introduction

DoubtLess targets redundant checking in reasoning chains, using a small model’s context-dependent brief-versus-verbose preference to steer a larger model. It shortens reasoning across benchmarks and settings while preserving accuracy, with most removed tokens coming after the answer first appears.

1 Introduction

  • Correct Qwen3-32B runs write 32–69% of their chain of thought after the correct answer first appears, where repeated checks can add tokens without information.The intended shortening preserves checks made during problem-solving and the first check of an answer.Source: §1 · p. 1
  • DoubtLess contrasts a small model’s brief and verbose next-token logits at each step, adding their scaled difference to the large model’s logits.The contrast is recomputed from the current context, unlike a fixed steering vector or a keyword ban.Source: §1 · p. 2
  • With Qwen3-32B steered by Qwen3-0.6B, DoubtLess removes 31% of thinking tokens with unchanged accuracy, outperforming five inference-time baselines across seven competition benchmarks.Source: §1 · p. 2
  • DoubtLess also reduces thinking tokens per turn by 20% in a self-debugging coding agent and shortens reasoning for two other model families and a model trained for brevity.The introduction reports 25% and 40% reductions for EXAONE-4.0 and Granite-4.2 on AIME 2025 and AMC 2023, respectively.Source: §1 · p. 2
  • The brief and verbose thinkers disagree most after the correct answer appears, where most of DoubtLess’s removed tokens lie.This analysis supports the explanation that the method reduces repeated checking.Source: §1 · pp. 2–3

2 Method

DoubtLess uses a smaller reasoning model to adjust a larger model’s next-token logits during chain-of-thought generation. The adjustment favors tokens expected more by the small model under a brief instruction than under a verbose one.

2.1 Preliminaries

  • A reasoning model generates a response one token at a time, and sampling draws from its next-token distribution, commonly truncated to the most probable tokens.The top-p set is the smallest token set whose probability mass reaches the specified threshold; Qwen3’s recommended thinking settings are temperature 0.6, top-k 20, and top-p 0.95.Source: §2.1 · p. 3

2.2 DoubtLess

  • At each chain-of-thought step, the smaller model reads the same prefix under brief and verbose instructions, and its logit difference is added, scaled by α, to the large model’s logits.The small model shares the large model’s tokenizer; the brief instruction favors going straight to the answer, while the verbose one requests double-checking and alternatives.Source: §2.2 · p. 3
  • Tokens favored by the brief thinker over the verbose thinker become more likely, while tokens favored by the verbose thinker become less likely.Where the two thinkers agree, the adjustment leaves token likelihoods unchanged.Source: §2.2 · p. 3
  • The context-dependent contrast leaves the derivation nearly unchanged before an answer but down-weights words that open another check after an answer.The contrast affects only the chain of thought; the large model retains its own end-of-thought and end-of-sequence logits and top-p candidates, and generates the answer on its own.Source: Figure 3 · p. 3, §2.2 · p. 4

3 Experimental setup

The evaluation pairs Qwen3-32B with a small Qwen3 model across seven reasoning benchmarks and compares DoubtLess with five inference-time baselines. It measures thinking-token changes and accuracy against plain decoding, with additional comparisons across model families and token-count analyses.

Benchmarks and models

  • Seven benchmarks cover competition mathematics, graduate-level science, olympiad problems, and competitive programming, scored with answer keys or program tests.The large model is Qwen3-32B and the default steering model is Qwen3-0.6B.Source: §3 · p. 4

Baselines

  • Five training-free baselines comprise a concise prompt, NoWait, DEER, ThinkBrake, and ASC, alongside plain decoding.They represent prompting, keyword blocking, early stopping, and fixed-vector steering.Source: §3 · p. 5

Implementation details

  • All methods use Qwen3’s recommended sampling settings, a 32,768-token thinking budget (24,576 on GPQA-Diamond), and three seeds per problem.DoubtLess uses α=5, selected as the best trade-off on AIME 2026.Source: §3 · p. 5

Evaluation figures

  • Figure 4 compares token removal and accuracy change for Qwen3-32B across seven benchmarks, and compares DoubtLess with concise prompting on two other model families.Figure 5 separates correct-run thinking tokens before and after the first correct answer and counts selected words per thousand tokens.Source: Figure 4 · p. 5, Figure 5 · p. 6

Evaluation

  • Evaluation reports accuracy and thinking-token change against plain decoding, with 95% bootstrap intervals over problems.Runs that reach the token budget without ending their chain of thought count as wrong.Source: §3 · p. 5

4 Results

DoubtLess removes more thinking tokens than the inference-time baselines while retaining accuracy across the seven benchmarks. It also transfers to other model families and reduces thinking in a self-debugging coding agent.

4.1 Main results

  • 31% fewer thinking tokens with unchanged accuracy across seven benchmarks makes DoubtLess the largest reduction among the six methods.It removes more tokens than the concise prompt on six of seven benchmarks; ThinkBrake and NoWait lose more accuracy than DoubtLess.Source: §4.1 · p. 6
  • DoubtLess shortens thinking significantly on every benchmark, including 31% on HMMT 2025, where the concise prompt removes 13%.The concise prompt’s reductions fall from 31% on AMC 2023 to 13% on HMMT 2025.Source: §4.1 · p. 6
  • DoubtLess preserves accuracy on every benchmark, while NoWait, DEER, and ThinkBrake each lose accuracy significantly on at least one.Runs reaching the token budget fall from 2.4% under plain decoding to 0.4% under DoubtLess.Source: §4.1 · p. 6

4.2 Other model families

  • On EXAONE-4.0 and Granite-4.2, DoubtLess shortens chains of thought without accuracy loss and removes about three times as much as concise prompting.Each large model is steered by a small model from its own family.Source: §4.2 · p. 6

4.3 Coding agent

  • On 164 HumanEval+ problems, DoubtLess cuts coding-agent thinking by 20% per turn and improves pass rate by 2.0 points over plain decoding.DEER removes 41% and improves pass rate by 4.5 points; NoWait and ThinkBrake remove more but lower pass rate.Source: §4.3 · p. 6
  • Figure 6 contrasts post-answer chains of thought: DoubtLess checks briefly and stops, while plain decoding repeats the same check.The example is AIME 2025 problem 1, first seed; checks are marked in red.Source: Figure 6 · p. 7

5 Analysis and discussion

DoubtLess concentrates its savings after the correct answer first appears, where it reduces repeated checking while leaving useful checks possible. It also further compresses a model trained for brevity and reduces measured inference latency.

5.1 What DoubtLess removes

  • DoubtLess cuts thinking after the first correct answer by half and before it by a fifth, with most removed tokens after the answer on six benchmarks.Plain decoding often repeats checks after reaching the answer, though an algebraic check can still correct an error.Source: §5.1 · p. 7
  • The brief and verbose thinkers’ mean distribution divergence is seven times larger after the answer than before it, and the contrast pushes down words that open another checking round.Words such as But, However, and Alternatively are nearly untouched before the answer.Source: §5.1 · p. 7
  • Wait, But, and Alternatively remain at least as frequent per thousand thinking tokens under DoubtLess as under plain decoding, unlike NoWait suppression.The method removes checking rounds while retained checks still begin with these words.Source: §5.1 · p. 7

5.2 DoubtLess complements training-based compression

  • 29–52% fewer thinking tokens with unchanged task performance when DoubtLess is added to LACONIC-Qwen3-32B across mathematics, science, and coding-agent benchmarks.The combination requires no further training of the already compressed model.Source: §5.2 · p. 7, Table 2 · p. 8

5.3 Inference cost

  • 119.4 seconds per problem on one B200 GPU is 24% less latency than plain decoding and 13% less than the concise prompt.The two 0.6B-model forward passes per step add about 4% to the 32B model’s FLOPs.Source: §5.3 · p. 8

6 Ablation study

Ablations test the contrast direction, its strength, and the size of the steering model. The results indicate that shortening depends on the direction supplied by the small model, while stronger contrast can eventually trade accuracy for further reduction.

6.1 Effect of the contrast direction

  • Randomizing the contrast targets changes overall thinking length by only 1%, versus 31% for DoubtLess, showing that shortening depends on the small model’s direction.Random-target lengths stay within 8% of plain decoding on every benchmark.Source: §6.1 · p. 8

6.2 Effect of the strength

  • On AIME 2025, increasing strength removes more thinking with unchanged accuracy through α=5, but α=10 loses 7.8 accuracy points.Negative strength α=−5 instead lengthens the chain of thought by 26%.Source: §6.2 · p. 8

6.3 Effect of the small-model size

  • Qwen3-1.7B or Qwen3-4B as the steering model removes close to 60% on AIME 2025 but exceeds the accuracy-preserving range; Qwen3-0.6B removes 29%.With Qwen3-32B as the steering model at α=1, DoubtLess removes 26%.Source: §6.3 · p. 8

7 Related work

Reasoning models often spend tokens re-deriving and re-verifying answers, while inference-time shortening methods and training-based compression impose different trade-offs. DoubtLess instead uses a small model’s context-dependent contrast at decoding time and can also be combined with training-based compression.

Overthinking

Shortening the CoT at inference time

  • DoubtLess reweights next-token probabilities using an off-the-shelf small model’s contrast, recomputed from context at every step, while leaving the large model’s </think> logit untouched.Earlier approaches include prompt requests for brevity, keyword bans, early stopping, fixed-vector activation steering, and a trained length predictor.Source: §7 · p. 8

Training-based compression

  • DoubtLess leaves the model unchanged, sets the length–accuracy trade-off with one decoding-time strength, and can be added to a model trained for brevity.Training-based compression requires a separate run for each model, with its trade-off fixed after training.Source: §7 · p. 9

Contrastive decoding

  • DoubtLess contrasts a small reasoning model’s brief and verbose instructions, unlike methods that contrast expert and amateur models, reasoning and base models, or prompts on the steered model itself.Related small-model uses include drafting in speculative decoding, token suggestion, and sentence completion.Source: §7 · p. 9

8 Conclusion

DoubtLess steers a large reasoning model with the difference between a small model’s brief and verbose thinkers. It shortens reasoning while retaining accuracy, with its contrast especially relevant after an answer first appears; the appendices describe its decoding, prompts, and probability interpretation.

8 Conclusion

  • 31% fewer thinking tokens with unchanged accuracy: DoubtLess outperforms five inference-time baselines on seven benchmarks with Qwen3-32B.The conclusion also reports shorter coding-agent CoTs, removal of about three times as many tokens as a concise prompt on two other model families, and further shortening when combined with training-based compression.Source: §8 · p. 9

A Algorithm and random-target control

  • The brief and verbose thinkers disagree most after the correct answer first appears, and analyses support reduced redundant checking while retaining useful verification.The random-target control changes thinking length by only 1% across the seven benchmarks, indicating that shortening depends on which tokens the small model pushes down.Source: §8 · p. 9, C The logit difference as a ratio of… · p. 12

B Prompts

  • Each stream receives the problem and an instruction; the brief thinker is told not to re-derive or repeat calculations, while the verbose thinker is told to double-check and re-verify.The large model’s stream has neither thinker instruction, and the concise-prompt setting gives it the brief sentence.Source: B Prompts · p. 12

C The logit difference as a ratio of probabilities

  • At temperature T, the logit contrast yields a brief-versus-verbose probability ratio that reweights the large model’s distribution before top-p masking.The restored end-of-thought and end-of-sequence logits have their own odds factor, which is shared by the random-target control.Source: C The logit difference as a ratio of… · p. 12

D Implementation details

The implementation runs the small model’s two thinker streams alongside the large model and applies their contrast only during the CoT. The appendix also documents the comparator implementations and their evaluation behavior.

Implementation details

  • The three streams share paged KV-cache blocks, while the small model keeps separate attention layers for the brief and verbose thinkers and overlaps its forward pass with the large model.After </think>, the contrast is no longer applied; both models’ forward passes are covered by one CUDA graph per decoding step.Source: D Implementation details · p. 13

NoWait

  • NoWait blocks 89 single-token forms of 17 checking-related words throughout generation; in these runs, chains shorten 21–24%, with accuracy losses concentrated on AIME 2025 and HMMT.The paper’s substring rule matches 918 Qwen3 vocabulary tokens, many unrelated, while its filtered list is unpublished.Source: D Implementation details · p. 13

DEER

  • DEER cuts a stored plain-decoding run when its transition rule fires, then samples an answer with the remaining budget; the rule uses trial-answer confidence and a 0.8× budget cap.On the paper’s reported Qwen3-32B benchmarks, thinking tokens fall 9–22% with accuracy within one point of plain decoding.Source: D Implementation details · p. 14

ThinkBrake

  • ThinkBrake ends the CoT at sentence boundaries when the margin to </think> is at most τ; these runs remove 21% of thinking tokens across seven benchmarks at a 2.3-point accuracy cost.The implementation reads the margin after converting sentence endings to a single-newline form.Source: D Implementation details · p. 14

ASC

  • ASC adds a calibrated mean hidden-state difference from verbose and concise CoTs at layer 57, using a strength ρ=0.2 selected on AIME 2026.The training-free implementation uses the large model’s own concise CoTs rather than GPT-4o-generated ones.Source: D Implementation details · p. 14

E Experimental setup details

The appendix details benchmark scope and scoring, strength selection, per-setting reporting, and evaluation on other model families. The strength sweep reports the observed token reductions and accuracy behavior across its tested values.

E Experimental setup details

  • The seven benchmarks include full AIME 2024 and 2025, HMMT February 2025, AMC 2023, and GPQA-Diamond, plus 100-problem samples from OlympiadBench and LiveCodeBench.Problems are in English; thinking budgets are 32,768 tokens except 24,576 on GPQA-Diamond, and scoring uses exact integer answers, math verification, option letters, or code tests as appropriate.Source: E Experimental setup details · p. 15, Table 3 · p. 15

F Strength and small-model size

G Every setting on every benchmark

H Other model families

  • The EXAONE and Granite model pairs share tokenizers within their respective families and use the same instructions, thinker sentences, sampling settings, budget, and α=5 strength.The concise-prompt comparison appends the brief thinker’s sentence to the large model’s own instruction.Source: H Other model families · p. 16

I Where the removed tokens come from

The analysis locates greater brief–verbose disagreement after the correct answer first appears, while a worked example shows verification can still correct an error. The self-debugging-agent evaluation tests DoubtLess within turns that may include a repair turn.

I Where the removed tokens come from

  • The analysis splits each CoT at its first automatic match to the correct answer and compares before- and after-boundary tokens on runs both methods solve.Integer answers use a standalone value and multiple-choice answers use the correct letter in an answer phrase; LiveCodeBench is excluded, while HMMT and OlympiadBench include integer-answer problems only.Source: I Where the removed tokens come from · p. 16, Table 11 · p. 19
  • DoubtLess removes about three times as much thinking as the concise prompt on EXAONE-4.0 and Granite-4.2, with unchanged accuracy.Table 8 covers AIME 2025 and AMC 2023, using the same instructions and strength across model families.Source: Table 8 · p. 18
  • 84% of total divergence falls after the answer boundary under automatic matching and both human annotations, with mean post-/pre-answer ratios of 7.4, 7.1, and 7.4.Mean divergence is larger after the boundary in all 39 traces under either annotation set.Source: I Where the removed tokens come from · p. 18
  • On AIME 2025 problem 11, an algebraic check corrects an initially rejected root, and the final answer remains correct with 17,196 thinking tokens versus 20,866 under plain decoding.Both annotators verified the error, its correction, and its effect on the final sum.Source: I Where the removed tokens come from · p. 18

J The self-debugging agent

  • In the self-debugging agent, DoubtLess halves turns that reach the 4,096-token limit without closing the CoT; DEER’s cut leaves none.Plain decoding has 16–20% of turns reach the limit; the random-target control keeps thinking per turn within 1% of plain decoding and pass rate within 1.2 points.Source: J The self-debugging agent · pp. 18–19

K Per-problem results on AIME 2025

DoubtLess reduces thinking across AIME 2025 problems by a similar percentage regardless of plain-decoding length, while fewer runs reach the token budget.

K Per-problem results on AIME 2025

L Worked examples

The worked example traces an AIME 2025 divisibility problem to the answer 70 and compares plain decoding with DoubtLess: both find the answer, while DoubtLess uses fewer thinking tokens.

Worked example

  • 3,414 versus 1,461 thinking tokens: plain decoding and DoubtLess (0.6B) both answer 70 on AIME 2025 problem 1.Both are first-seed runs decoded under the same sampling settings, and the full runs are shown.Source: L Worked examples · p. 20
  • The comparison uses the first seed for both methods and holds sampling settings constant.The passage says the runs are shown in full.Source: L Worked examples · p. 20
  • The valid bases are 21 and 49, yielding a sum of 70.The worked solution identifies 28 and 56 as the divisors of 56 greater than 16, giving those two bases.Source: L Worked examples · p. 21

L Worked examples

  • 70 is the sum of bases 21 and 49, found by rewriting (9b + 7)/(b + 7) as 9 − 56/(b + 7) and selecting divisors of 56 greater than 16.The qualifying values of b + 7 are 28 and 56.Source: L Worked examples · p. 23

Related papers

Loading 2610.18862v1…