Source-linked AI summary
AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting
Shijue Huang, Hongru Wang, Wanjun Zhong, Zhaochen Su, Jiazhan Feng, Bowen Cao, Yi R. Fung
TL;DR
AdaCtrl targets lengthy reasoning on simple problems and the challenge of balancing efficiency with effectiveness. It combines difficulty-aware budgeting and user-selected reasoning modes through cold-start fine-tuning and difficulty-aware reinforcement learning. Across four benchmarks, it improves performance while reducing response lengths by 10%–90%.
Problem
Reasoning models can generate unnecessarily long chains for simple problems, while universal shortening and fine-grained budget control each have limitations.
Source: §1 · pp. 1–2Method
AdaCtrl uses cold-start fine-tuning and difficulty-aware reinforcement learning to support self-assessed adaptive budgeting and user-selected reasoning modes.
Source: §1 · p. 2, §3.1 · p. 4, §3.2 · p. 5Results
Across four benchmark datasets, AdaCtrl improves performance while reducing response lengths by 10%–90%.
Source: §5 · p. 11Takeaways & Limitations
AdaCtrl supports flexible trade-offs between efficiency and performance through adaptive budgeting and explicit user control.
Source: §1 · p. 3, §5 · p. 11AdaCtrl does not use more fine-grained difficulty categories, to maintain ease of control and reliability.
Source: §3.2 · p. 4
Abstract
from arXiv · showhide
Modern large reasoning models demonstrate impressive problem-solving capabilities by employing sophisticated reasoning strategies. However, they often struggle to balance efficiency and effectiveness, frequently generating unnecessarily lengthy reasoning chains for simple problems. In this work, we propose AdaCtrl, a novel framework to support both difficulty-aware adaptive reasoning budget allocation and explicit user control over reasoning depth. AdaCtrl dynamically adjusts its reasoning length based on self-assessed problem difficulty, while also allowing users to manually control the budget to prioritize either efficiency or effectiveness. This is achieved through a two-stage training pipeline: an initial cold-start fine-tuning phase to instill the ability to self-aware difficulty and adjust reasoning budget, followed by a difficulty-aware reinforcement learning (RL) stage that refines the model's adaptive reasoning strategies and calibrates its difficulty assessments based on its evolving capabilities during online training. To enable intuitive user interaction, we design explicit length-triggered tags that function as a natural interface for budget control. Empirical results show that AdaCtrl adapts reasoning length based on estimated difficulty, compared to the standard training baseline that also incorporates fine-tuning and RL, it yields performance improvements and simultaneously reduces response length by 10.06% and 12.14% on the more challenging AIME2024 and AIME2025 datasets, which require elaborate reasoning, and by 62.05% and 91.04% on the MATH500 and GSM8K datasets, where more concise responses are sufficient. Furthermore, AdaCtrl enables precise user control over the reasoning budget, allowing for tailored responses to meet specific needs.
1 Introduction
AdaCtrl addresses overlong reasoning on simple problems while preserving deeper reasoning for complex ones, and adds user control over reasoning depth. Its two-stage approach combines cold-start fine-tuning with difficulty-aware reinforcement learning; reported results improve the efficiency–effectiveness trade-off across four benchmarks.
1 Introduction
- Deep reasoning can help solve complex problems, but models also produce lengthy chains for straightforward questions, increasing inference overhead and latency.The introduction illustrates this with evaluating log2(64).Source: §1 · pp. 1–2
- Universal shortening may sacrifice quality on complex tasks, while precise per-question budgets depend on reliable difficulty assessments and instruction-following.These limitations motivate a framework that adapts to task complexity while remaining controllable.Source: §1 · p. 2
- AdaCtrl combines self-assessed adaptive reasoning with user-selected modes, using cold-start fine-tuning and difficulty-aware reinforcement learning to allocate budgets.The framework uses [Easy] and [Hard] tags and rewards for calibrating difficulty estimates and refining adaptive reasoning.Source: §1 · pp. 2–3
- Across four benchmarks, AdaCtrl improves the efficiency–effectiveness trade-off, outperforming most baselines while managing reasoning budgets.The reported comparison with standard SFT + RL baselines gives accuracy improvements of up to 10.14% and response-length reductions of as much as 91.04%.Source: §1 · pp. 2–3
2 Related Work
Prior work seeks more efficient reasoning through supervised fine-tuning and reinforcement learning, but the approaches described here generally do not combine self-assessed difficulty with explicit user control.
Reasoning Efficiency via Supervised Fine-Tuning
- Supervised fine-tuning methods shorten reasoning paths or compress existing chains, but often enforce concision regardless of task difficulty.The related-work section says these methods typically lack difficulty self-assessment and user budget control.Source: §2 · p. 3
Reasoning Efficiency via Reinforcement Learning
- Reinforcement-learning approaches use length rewards, difficulty-linked shaping, prompt targets, or other strategies, but generally lack explicit user control and training for difficulty self-awareness.AdaCtrl’s two-stage SFT-RL framework is presented as addressing both aspects.Source: §2 · p. 3
3 Method
AdaCtrl uses cold-start fine-tuning to establish difficulty-aware budget control, then reinforcement learning to calibrate difficulty estimates and refine adaptive reasoning. Its Easy and Hard tags also let users steer response length.
3 Method
- AdaCtrl’s two-stage pipeline first establishes budget awareness with short and long reasoning trajectories, then calibrates difficulty estimates and develops adaptive reasoning through reinforcement learning.The method comprises cold-start fine-tuning followed by difficulty-aware reinforcement learning.Source: §3 · pp. 3–4, Figure 2 · p. 4
3.1 Cold-Start Fine-Tuning
- Cold-start training pairs easy problems with concise instruction-model responses and hard problems with reasoning-model traces, filters for correct answers, and prefixes responses with [Easy] or [Hard] tags.The tags teach format adherence and support user control over response length and reasoning depth.Source: §3.1 · p. 4
3.2 Difficulty-Aware Reinforcement Learning
- During reinforcement learning, correct-rollout frequency sets an easy-or-hard label, and a calibration reward encourages the model’s generated difficulty tag to match it.A question is labeled easy when the frequency of correct rollouts exceeds threshold δ; otherwise it is labeled hard.Source: §3.2 · p. 5
- The difficulty-aware length reward encourages concise responses for [Easy]-tagged problems, assigning lower scores as response length increases while preserving long reasoning for hard problems.The method combines this length reward with outcome-accuracy and difficulty-calibration rewards during reinforcement learning.Source: §3.2 · p. 5
4 Experiment
Across four math benchmarks, AdaCtrl’s adaptive mode combines competitive or improved accuracy with shorter responses, while controlled modes let users trade performance for response length. Analysis reports difficulty-aligned tagging and budget allocation, with performance and response lengths evolving during RL.
4.1 Experimental Setup
- Evaluation uses Qwen2.5-7B/14B backbones, four math datasets spanning easy to challenging problems, and accuracy and generated-token count as metrics.Cold-start data combines 4K short- and 4K long-reasoning instances; baselines include base, S1.1, R1-SFT, and cold-start variants.Source: §4.1 · pp. 6–7
4.2 Main Results
- Against Qwen2.5-7B-Instruct R1-SFT-RL, AdaCtrl-7B matches AIME2024 accuracy and raises AIME2025/MATH500/GSM8K accuracy by 1.67%/7.20%/2.05%, while shortening responses 10.06%/12.14%/62.05%/91.04%.Table 1 reports average accuracy and average generated tokens.Source: Table 1 · p. 7, §4.2 · p. 7
- Cold-start fine-tuning reduces 7B response lengths by 73.26%, 69.10%, 90.06%, and 89.65% on AIME2024, AIME2025, MATH500, and GSM8K, respectively.Cold-start-RL yields further response-length reductions over R1-SFT-RL on the same datasets; the reported 7B reductions are 9.94%, 11.06%, 54.91%, and 91.04%.Source: §4.2 · p. 7
- Against Cold-Start-RL, AdaCtrl raises accuracy on AIME2024/AIME2025/MATH500/GSM8K by 2.92%/4.59%/1.00%/0.31% (7B) and 7.08%/1.66%/3.6%/0.08% (14B), with less token consumption.The reward combines difficulty-estimation calibration and difficulty-aware length adjustments.Source: §4.2 · pp. 7–8
4.3 Analysis
- Accuracy remains largely stable under tested reward-weight and threshold variations, while smaller δ shortens outputs; removing either reward leads to performance declines across all four datasets.On MATH500, response length falls from 3195.69 to 2888.39 to 2181.96 tokens as accuracy remains around 74%.Source: §4.3 · pp. 8–9, Table 3 · p. 8, Table 4 · p. 9
- After RL, AdaCtrl classifies 76.2% of MATH500 problems and over 99% of GSM8K problems as easy; MATH500 responses grow from approximately 0.3K to 6K tokens across difficulty levels 1–5.AIME2024 and AIME2025 are predominantly classified as hard.Source: §4.3 · p. 9, Figure 3 · p. 9, Figure 4 · p. 10
4.2 Main Results
- AdaCtrl’s Easy mode reduces response lengths by 90.22%/94.32%/79.57%/9.98% across four datasets, while 500-token prompting reduces them by only 5.25%/8.96%/7.97%/12.48%.Hard mode markedly increases response length and improves performance on most datasets; Easy mode reduces performance across all four.Source: §4.2 · p. 8, §4.3 · p. 10, Figure 4 · p. 10
4.3 Analysis
- During RL, accuracy trends upward across all datasets; response lengths rise then stabilize above pre-RL levels on AIME2024, AIME2025, and MATH500, but stay near pre-RL levels on GSM8K.AdaCtrl also separates easy and hard response-length distributions, with easy responses concentrated in a relatively narrow range.Source: §4.3 · pp. 10–11, Figure 5 · p. 10, Figure 6 · p. 11
5 Conclusion
AdaCtrl allocates reasoning budgets using self-assessed difficulty and user direction, with experiments reporting performance gains and shorter responses across four benchmarks.
5 Conclusion
- Across four benchmark datasets, AdaCtrl improves performance while reducing response lengths by 10%–90%.The framework allocates budgets based on self-assessed problem difficulty.Source: §5 · p. 11
- AdaCtrl combines dynamic reasoning-budget allocation with user-directed budget adjustments to mitigate overthinking and offer explicit control over computational resources.Source: §5 · p. 11
- The approach uses a two-stage training pipeline combining cold-start fine-tuning with difficulty-aware reinforcement learning.Source: §5 · p. 11
- AdaCtrl supports flexible trade-offs between efficiency and performance and allows reasoning budgets to be tailored to user needs.Source: §5 · p. 11