Source-linked AI summary
AdaThinking-E: One-Token Entropy Regulation for Adaptive Thinking
Zining Wang, Tongkun Guan, Boming Chen, Zhentao Guo, Jianqiang Liu, Chao Jin, Chen Duan, Kai Zhou, Pengfei Yan, Wei Shen, Xiaokang Yang
TL;DR
Deep reasoning helps multimodal document models on difficult tasks but can waste computation and harm simple-query performance, creating a need for complexity-aware mode selection. AdaThinking-E uses one-token entropy regulation in reinforcement learning, with separate optimization of mode decisions and response content, and reports strong benchmark performance with efficient reasoning. The method is designed to let models discover when to think without external difficulty annotations.
Problem
Deep reasoning applied uniformly to all document questions creates unnecessary overhead on simple tasks and can degrade accuracy, while adaptive mode selection remains dependent on manual intervention or external difficulty labels.
Method
AdaThinking-E independently regulates entropy at mode-switching tokens during reinforcement learning, transitioning from high-entropy exploration to low-entropy convergence while separately optimizing response tokens.
Results
AdaThinking-E achieves state-of-the-art performance across multiple benchmarks while matching thinking-model effectiveness with significantly lower token consumption.
Takeaways & Limitations
The framework enables models to determine when to think intrinsically, improving efficiency on simple tasks while preserving accuracy on complex ones across document benchmarks.
Takeaways & Limitations
The training approach assumes that manually specifying thinking needs or using model distillation would impose external assumptions and bias on reinforcement learning.
Abstract
from arXiv · showhide
Multimodal large language models have demonstrated strong document reasoning capabilities by incorporating explicit thinking processes. While this capability significantly improves performance on challenging tasks, current models apply such deep reasoning uniformly to all questions, resulting in unnecessary computational overhead for simple task. This not only degrades user experience but also negatively impact accuracy on benchmark datasets. We identify the critical need for adaptive thinking mechanisms that can intelligently determine when to engage reasoning based on question complexity. To address this, we propose AdaThinking-E, a novel reinforcement learning framework that learns adaptive thinking through one-token entropy regulation. Our key insight is that model confidence in the decision to engage thinking (or not) can be quantified through entropy analysis of the predicted probability distribution at critical decision tokens. This observation motivates our entropy-governed reward mechanism: the training process naturally transitions from high-entropy exploration, where the model experiments with different thinking strategies, to low-entropy convergence with confident, generalizable decision-making policies. Crucially, this approach enables models to intrinsically discover when to think without requiring manual intervention or external difficulty labels. Extensive experiments demonstrate that our approach enables models to be both accurate on complex problems and efficient on simple ones across diverse document tasks.
1 Introduction
Document MLLMs need adaptive thinking because deep reasoning can waste computation and cause hallucinations on simple questions, while current mode-selection methods rely on manual rules or external difficulty labels. AdaThinking-E uses entropy-regulated reinforcement learning to discover mode selection intrinsically, complemented by the AdaThinking-Doc dataset, and reports strong accuracy with lower token consumption.
- Motivation: Deep reasoning improves complex document reasoning but adds unnecessary computation and may introduce hallucinations on directly answerable questions.The paper argues that adaptive thinking should preserve accuracy on complex problems while improving efficiency on simple ones.
- Problem: Existing approaches either manually specify thinking modes or use subjective difficulty labels, limiting flexibility, generalization, and autonomous mode selection.Manual intervention requires predefined rules, while labeled approaches incur annotation costs and teach models to follow external labels.
- Method: AdaThinking-E regulates entropy at mode-switching tokens to transition from exploratory uncertainty to confident mode decisions aligned with question complexity.The framework independently rewards entropy dynamics at decision positions and separates mode-switch optimization from response-content optimization.
- Dataset: AdaThinking-Doc is introduced as a document-understanding dataset spanning simple information extraction and complex reasoning tasks for adaptive-thinking training.The dataset is designed to capture the full spectrum of question complexity.
- Results: AdaThinking-E achieves state-of-the-art performance across multiple benchmarks while matching thinking-model effectiveness with significantly lower token consumption.This is the paper’s reported overall experimental outcome.
2 Related Work
Document reasoning models range from externally controlled systems to adaptive-thinking models that select response modes autonomously. The paper identifies mode-switch token probabilities as an underexamined mechanism and proposes entropy-based regulation to guide switching.
- Reasoning Models: Reasoning models use chain-of-thought generation for complex computation and logic, but excessive reasoning length increases inference costs and can degrade user experience.Optimal reasoning length varies across tasks, motivating task-dependent reasoning depth.
- Adaptive Thinking: Externally controlled models switch modes through prompts, whereas adaptive-thinking models select response modes based on the question without handcrafted templates.The related-work taxonomy distinguishes external control from autonomous mode selection.
- Research Gap: Existing methods do not examine the probability distribution of mode-switching tokens, leaving the core switching mechanism insufficiently studied.The paper positions token-level entropy as the missing analytical focus.
- Proposed Direction: AdaThinking-E introduces mode-switch-token entropy into the reward function to guide transitions between thinking and non-thinking modes.Entropy variation is used to steer mode switching rather than relying on externally specified mode decisions.
3 Methodology
AdaThinking-E combines cold-start training with reinforcement learning that separately optimizes mode-switching and response tokens. Entropy regulation moves mode selection from exploration toward confident decisions, while content rewards target answer quality.
- Cold Start: Cold-start training uses AdaThinking-Doc to produce both thinking and non-thinking responses for each data point, avoiding externally specified mode labels.The dataset uses generated detailed reasoning for thinking-mode data and predefined mode-switch tokens for both response types.
- Reinforcement Learning: After cold start, direct GRPO training exposed instability, motivating a reward design that distinguishes mode selection from response generation.The framework identifies the mode-switching token as the determinant of whether reasoning begins, while response tokens provide the answer content.
- Token-Level Objectives: The mode-switching token receives entropy-based optimization, whereas response tokens use advantage estimates to optimize content quality separately.The objective assigns E_i,0 to the mode-switching token and Â_i,t to response tokens, separating adaptive mode selection from answer quality.
- Entropy Curriculum: Training follows a high-entropy exploration stage with approximately balanced mode probabilities and a low-entropy decision stage with more consistent mode selection.The transition is scheduled across training steps, with γ defining the inflection point and β controlling transition smoothness.
- Decision Feedback: Decision feedback compares correctness across sampled thinking and non-thinking rollouts, while accuracy guidance and propensity correction regulate mode-specific optimization.The method uses three accuracy tiers and addresses unreliable comparisons when sampling between modes is strongly imbalanced.
- Quality Reward: Content-focused rewards complement entropy regulation because entropy determines mode activation but does not evaluate reasoning quality or answer accuracy.A format reward also enforces the template needed for the mode-switching mechanism to operate at designated token positions.
4 Experiments
Experiments show that AdaThinking-E adapts its thinking mode across document tasks, preserving performance while reducing unnecessary reasoning on simpler queries. Entropy regulation supports exploration, avoids mode collapse, and produces competitive accuracy-efficiency trade-offs.
- AdaThinking-E achieves near SOTA performance across information extraction and analytical computation tasks.
- 0.6% improvement on DocVQA, 1.8% on ChartQA, and 10.8% on OCR-Reasoning over baseline Qwen2.5-VL.
- The adaptive thinking mode consistently matches or exceeds each standalone thinking and non-thinking mode across benchmarks.
- On CharXivRQ, AdaThinking-E uses a 27% thinking ratio and 72 tokens versus R-4B’s 82% ratio and 366 tokens at comparable performance.
- On CharXivDQ, AdaThinking-E improves performance by 0.6% over R-4B while reducing the thinking ratio to 11% from 45%.
- GRPO and DAPO quickly converge toward non-thinking mode, whereas AdaThinking-E regulates entropy to prevent mode collapse.
5 Conclusion
AdaThinking-E uses one-token entropy regulation to let multimodal language models switch between thinking and non-thinking according to task complexity. Experiments across document benchmarks support a balance between computational efficiency on simple tasks and reasoning performance on complex ones.
- AdaThinking-E dynamically switches between thinking and non-thinking modes according to task complexity.
- One-token entropy regulation guides the transition from exploratory to convergent thinking strategies without external difficulty annotations.
- Experiments across diverse document understanding benchmarks demonstrate a favorable balance between computational efficiency and reasoning performance.
A Dual-objective Reward Mechanism
AdaThinking-E extends DAPO with a dual-objective reward mechanism that separates mode-switching decisions from response-token optimization. It uses entropy regulation to guide exploration and convergence while retaining correct-response groups for adaptive mode learning.
- DAPO foundation: DAPO aggregates policy-gradient loss at the token level, giving equal gradient-update contribution to tokens across long and short sequences.This replaces sequence-level averaging used by GRPO and avoids diluting contributions from longer sequences.
- DAPO foundation: DAPO normalizes rewards within each response group to obtain relative advantages for policy optimization.The group statistics use the mean and standard deviation of response rewards.
- DAPO foundation: The importance-sampling ratio reweights sample advantages when transitioning from the old policy to the new policy.This ratio is intended to make policy updates more stable and effective.
- Reward design: AdaThinking-E modifies DAPO by replacing the standard mode-switching advantage with an entropy-based quantity and retaining groups with correct responses.Correct responses still provide learning signals through the sampling ratio of thinking and non-thinking modes.
B Decision Feedback
Decision Feedback uses mode-specific occurrence ratios and accuracies to steer thinking engagement while accounting for imbalance and uncertainty in small-sample estimates. Its guidance terms support reasoning for low-accuracy cases and token reduction when performance is already sufficient.
- Decision Feedback: Decision Feedback evaluates thinking and non-thinking modes using each mode’s occurrence ratio and accuracy.The occurrence ratio is computed over sampled responses, while accuracy is computed among samples assigned to that mode.
- Accuracy Guidance Term: Accuracy guidance activates only when both modes fall within the same accuracy tier, limiting disturbances caused by small-sample randomness.The design aims to avoid oscillatory optimization behavior.
- Propensity Guidance Term: Propensity guidance addresses imbalanced sampling by incorporating mode prevalence when both modes achieve high accuracy.Its condition activates when the occurrence-ratio gap exceeds Δ_ρ and both accuracies are in the high tier.
- Optimization: Mode-specific weights are normalized to the closed interval [1, 2] for training stability and numerical safety.The normalization scales both mode weights using their minimum value with a small numerical constant.
- Outcome: Decision Feedback enables per-query reasoning-mode selection without manually enforced difficulty heuristics.The framework is intended to maintain accuracy while adapting reasoning engagement.
C AdaThinking-Doc
AdaThinking-Doc is a cold-start document-understanding dataset containing both simple and complex scenarios. Each retained query provides thinking and non-thinking responses after filtering unreasonable reasoning processes.
- Data Distribution: AdaThinking-Doc combines information-extraction data from diverse document-understanding source datasets.The construction extracts 80% of the data from each source dataset.
- Cold-start dataset: Each query generates both a thinking-type response and a non-thinking-type response to support balanced mode patterns.This paired structure supplies examples for both reasoning behaviors.
- Cold-start dataset: 196,000 valid queries were retained after discarding entries and responses judged by GPT-4o-mini to contain unreasonable reasoning processes.The filtering applies to entries and their corresponding thinking and non-thinking responses.
- Thinking Content: Thinking responses use a step-by-step format with brief step goals, detailed reasoning, and a final summary.The format repeats goal-and-reasoning steps before producing the conclusion.
D More Results
Training-stage ablations report gains across both reasoning-oriented and simple document benchmarks, while dynamic-scoring analysis links β to phase separation and γ to the exploration-to-convergence transition.
- Training phase ablations: 0.6%, 5.1%, and 5.9% improvements over QwenVL2.5-7B were reported on ChartQA, CharXivRQ, and OCR-Reasoning, respectively, at cold start.These are reasoning-oriented benchmark results attributed to the AdaThinking-Doc construction.
- Training phase ablations: 3.1% average improvement was observed from the cold-start stage to later training across simple and reasoning scenarios.The comparison concerns AdaThinking-E-Qwen2.5 across the evaluated benchmarks.
- Training phase ablations: AdaThinking-E-Qwen3 showed marginal gains on CharXiv and OCR-Reasoning relative to AdaThinking-E-Qwen2.5, while outperforming it on simple data.The reported pattern suggests differences between model variants across task complexity.
- Dynamic Scoring Mechanism: As β increases, dynamic scoring produces a more pronounced distinction between exploration and convergence phases.The analysis states that this helps the model perceive state transitions.
- Dynamic Scoring Mechanism: γ acts as the critical inflection point governing the transition from exploration to convergence.The passage identifies γ as the parameter controlling that transition.
E Case Study
AdaThinking-E switches between thinking and non-thinking modes across document task types, using non-thinking mode for straightforward scenarios and supporting efficient, accurate responses across simple and complex documents.
- Mode Switching: AdaThinking-E demonstrates switching between thinking and non-thinking modes across different document task types.The examples span Figs. 2 to 9.
- Non-thinking Mode: In straightforward OCR, information extraction, and visual question answering scenarios, AdaThinking-E opts for non-thinking mode to provide quick responses.
- Outcome: By switching modes, AdaThinking-E handles simple and complex document scenarios while maintaining accuracy and ensuring efficient responses.