Source-linked AI summary
DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models
Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, Shiguo Lian
TL;DR
Slow-thinking models can waste computation on simple problems, while uniform compression risks harming difficult-task reasoning. DAST uses difficulty-aware token budgets and budget-preference training to adapt response length, and experiments report reduced overthinking while preserving reasoning capability. Its current evaluation is limited to STEM domains, and the method is sensitive to the truncation threshold.
Problem
Existing mitigation methods often uniformly shorten reasoning, risking degraded performance on challenging problems that require adequate reasoning length.
Method
DAST maps problem difficulty to target response length using TLB, budget-aware reward shaping, and budget-preference training.
Results
Across multiple datasets and model scales, DAST mitigates overthinking while preserving reasoning capabilities, especially on difficult tasks.
Takeaways & Limitations
DAST enables adaptive reasoning that compresses simple-task responses while allocating extended reasoning to complex problems.
Takeaways & Limitations
Evaluation focuses exclusively on STEM disciplines, leaving code generation and general-domain tasks unexplored.
Abstract
from arXiv · showhide
Recent advancements in slow thinking reasoning models have shown exceptional performance in complex reasoning tasks. However, these models often exhibit overthinking (generating redundant reasoning steps for simple problems), leading to excessive computational resource usage. While current mitigation strategies uniformly reduce reasoning tokens, they risk degrading performance on challenging tasks that require extended reasoning. This paper introduces Difficulty-Adaptive Slow Thinking (DAST), a novel framework that enables models to autonomously adjust the length of Chain-of-Thought (CoT) based on problem difficulty. We first propose a Token Length Budget (TLB) metric to quantify difficulty, then leverage budget-aware reward shaping and budget preference optimization to implement DAST. DAST penalizes overlong responses for simple tasks while incentivizing sufficient reasoning for complex problems. Experiments on diverse datasets and model scales demonstrate that DAST effectively mitigates overthinking (reducing token usage by over 30\% on average) while preserving reasoning accuracy on complex problems. Our codes and models are available at https://github.com/AnonymousUser0520/AnonymousRepo01.
1 Introduction
Slow-thinking models perform well on complex reasoning but can overthink simple problems, wasting computation. DAST addresses this by adapting reasoning length to problem difficulty while preserving performance on difficult tasks.
- Motivation: Slow-thinking models can generate redundant reasoning for simple problems, reducing computational efficiency.DeepSeek V3 uses 58 tokens for 3x + 7 = 22, while DeepSeek-R1 may use over 1000 tokens.
- Motivation: Uniform token or reasoning-step reduction can shorten outputs but risks degrading performance on challenging problems.Adequate reasoning length is described as critical for solving complex tasks.
- DAST Framework: DAST asks models to generate concise reasoning for simple questions and extended Chain-of-Thought reasoning for complex ones.The framework is designed to adjust reasoning depth autonomously according to problem difficulty.
- DAST Framework: DAST maps problem complexity to target response length and applies rewards or penalties based on deviations from that target.The Token Length Budget combines sampled-response accuracy with response-length distributions; overlong simple-task responses are penalized, while sufficient complex-task reasoning is rewarded.
- Findings: DAST is reported to alleviate overthinking while maintaining reasoning performance, especially on difficult tasks.The contribution is supported by validation across multiple datasets and model parameter scales.
- Contributions: TLB provides a problem-difficulty quantification metric intended for use across many downstream tasks.It supports the difficulty-to-response-length mapping used by DAST.
2 Related Work
Prior work improves reasoning efficiency through prompt, output, and post-training interventions, but commonly compresses reasoning uniformly. The related-work gap is difficulty adaptation and broader validation across model scales and domains.
- Prompt-based Methods: Prompt-based methods seek concise reasoning by modifying prompts or imposing explicit token limits.These methods intervene before generation.
- Output-based Methods: Output-based methods compress intermediate reasoning or dynamically decide which reasoning steps to retain, modify, or discard.Latent compression improves brevity at the cost of interpretability, while dynamic decoding uses lightweight models or heuristics.
- Post-training Methods: Post-training methods use variable-length CoT supervision or length-based rewards to encourage concise and accurate reasoning paths.These approaches include supervised fine-tuning and reinforcement-learning frameworks.
- Limitations of Prior Work: Most existing methods indiscriminately compress CoT, which can degrade performance on complex problems.The passage identifies this as a central limitation of prior efficient-reasoning approaches.
- Limitations of Prior Work: Prior evaluations have focused mainly on LRMs below 7B parameters, with limited testing at larger scales such as 32B.The paper positions difficulty-aware token budgets as a response to these limitations.
3 Methodology
DAST adapts reasoning length to problem complexity by assigning Token Length Budgets and training models with calibrated, budget-aware preferences. Its pipeline rewards concise correct responses, encourages longer reasoning for insufficient incorrect responses, and filters preference pairs before SimPO fine-tuning.
- 3.1 Token Length Budget Definition: DAST uses Token Length Budget (TLB) to allocate shorter responses to simpler questions and extended budgets to complex ones.TLB combines sampling accuracy, average correct-response length, and maximum generation length; lower accuracy moves the budget toward the maximum.
- 3.2 Reward Score Calibration: DAST calibrates rule-based rewards using the deviation between actual response length and TLB, combining difficulty awareness with length characteristics.Correct responses longer than TLB receive reward decay, while shorter correct responses receive amplified rewards; incorrect responses below TLB receive greater reward as their length approaches TLB.
- 3.3 Budget Preference Data Construction: For each question, DAST ranks sampled responses by calibrated reward and constructs winning–losing contrastive pairs for preference optimization.The pair classes include dual-correct pairs favoring concise correct responses and dual-incorrect pairs favoring longer reasoning within the budget.
- 3.3 Budget Preference Data Construction: DAST retains maximal-margin DCP and DICP pairs, removes the bottom δ|D| pairs, and keeps at most one pair of each type per question.This selection is intended to preserve informative reward differences while improving preference-optimization stability.
- 3.4 Budget Preference Training: The constructed preference dataset is optimized with SimPO, selected because it is sensitive to controlling answer length.The method excludes the CICP pair class from the main construction because experiments found that it did not improve performance.
4 Experiments
DAST is evaluated across three reasoning benchmarks, two model scales, and multiple efficient-reasoning baselines. Results indicate that it reduces response length while better preserving accuracy on difficult problems through difficulty-adaptive compression.
- Evaluation Setup: Evaluation covers MATH-500, AIME 2024, and GPQA using DS-7B and DS-32B backbones.MATH-500 contains five difficulty levels, while AIME 2024 and GPQA test complex mathematical and PhD-level science reasoning.
- Baselines and Metrics: Baselines include prompt-based concise reasoning, shortest-response SFT, and SimPO variants using shortest, cosine, or length-penalty criteria.The comparison measures both final-answer accuracy and response compression.
- Overall Results: 43.3% ACC is achieved by CoD with DS-7B on AIME 2024, down from 60.0%, illustrating accuracy losses from prompt-based compression.Aggressive compression methods reduce tokens most substantially but sacrifice some accuracy.
- Overall Results: DAST outperforms the standard cosine-based reward across benchmarks on both ACC and CR while balancing brevity against reasoning performance.Both methods can encourage longer responses when beneficial, unlike strategies that strictly prioritize brevity.
- Difficulty Adaptation: DAST (DS-7B) reaches 70.0% AIME 2024 accuracy versus 60.0% for Origin while increasing response length, indicating adaptive allocation for complex problems.On GPQA, it achieves 51.51% with a 4.2% compression ratio despite being trained exclusively on mathematics.
- Fine-grained Analysis: DAST reduces MATH-500 response length by approximately 58.5% at Level 1 and 40.8% at Level 5, unlike SimPOShortest’s limited difficulty differentiation.The smaller reduction at Level 5 reflects longer reasoning allocation for harder problems.
- Ablation Study: Removing DCP yields +1.4% accuracy but +17.8% length, whereas removing DICP yields 59.8% compression but -3.2% accuracy.Combining DCP and DICP achieves the optimal ablation performance, indicating complementary roles.
- Truncation Threshold: For DS-32B, δ = 0.15 was selected to optimize accuracy with a 47% compression ratio, while DS-7B selected δ = 0.18.At δ = 0, low-discriminability and excessively long training preferences can cause reward hacking and prevent correct length optimization.
5 Conclusion
DAST addresses the efficiency–performance dilemma in slow-thinking models through difficulty-aware reasoning adaptation. By relating problem complexity to solution length, it supports dynamic reasoning-resource allocation, with validation across representative benchmarks.
- 5 Conclusion: DAST addresses the efficiency–performance dilemma by adapting reasoning length to problem difficulty and enabling dynamic resource allocation.The paper reports validation across representative benchmarks.
Limitations
The authors identify three limitations: STEM-only evaluation, sensitivity to the truncation threshold, and possible performance limits from off-policy learning.
- Domain-Specific Evaluation Scope: Evaluation covers STEM disciplines but leaves code generation and general-domain tasks unexplored.The authors plan to extend the evaluation benchmarks in future work.
- Threshold Sensitivity: The method is sensitive to the truncation threshold, requiring additional cost for careful adjustment.
- Off-Policy Learning Constraints: Preconstructed training data improves computational efficiency but may limit performance potential relative to online reinforcement learning.The authors propose exploring on-policy variants using their reward function.
A Implementation Details
Table 4 presents the training configuration of DAST.
- Table 4 reports the training configuration used for DAST.
B Prompt Templates of CCoT and CoD
The appendix provides the prompt templates used to implement the CCoT and CoD methods.
- Figure 7 presents the prompt used to implement the CCoT method.
- The prompt templates support implementation of CCoT and CoD methods.
- Figure 8 presents the prompt used to implement the CoD method.
C Case Study
The case study compares DeepSeek-R1-Distill-Qwen-32B outputs for a simple problem before and after applying DAST. The original response contains redundant reasoning, whereas DAST produces a more concise and focused output.
- The original output uses extensive unnecessary and redundant thinking for a simple problem.
- After applying DAST, the output becomes more concise and focused.
- Figure 9 compares an overthinking case from DeepSeek-R1-Distill-Qwen-32B with its output after applying DAST.