Source-linked AI summary
Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AI
Md Jueal Mia, M. Hadi Amini
TL;DR
Agentic AI systems may allocate too much or too little reasoning as task demands evolve. This paper evaluates over- and under-reasoning across MATH-500 and GAIA, finding that excessive reasoning raises cost without proportional gains while insufficient reasoning is linked to incorrect or incomplete solutions.
Problem
Existing reasoning controls are often fixed or preallocated and evaluated on standalone benchmarks, leaving adaptive reasoning allocation in evolving agentic workflows insufficiently studied.
Method
The paper quantitatively analyzes over- and under-reasoning in a LangGraph-based agentic framework across MATH-500 and GAIA using latency, tokens, tool usage, token-limit hits, and correctness.
Results
Over-reasoning is associated with higher computational cost without proportional accuracy gains, whereas under-reasoning is consistently associated with incorrect or incomplete solutions.
Takeaways & Limitations
The findings motivate adaptive mechanisms that dynamically determine when agents should reason, act, and stop according to task demands.
Takeaways & Limitations
Because the study uses one fixed router and three final-response models in a LangGraph framework, its findings should not be generalized to all agentic architectures, routing strategies, or foundation models.
Abstract
from arXiv · showhide
Recent advances in Large Language Models (LLMs) have shown that increased inference-time reasoning can improve performance on complex tasks. However, many existing approaches rely on fixed or preallocated reasoning controls, such as fixed token budgets, pre-execution difficulty estimates, or activation-space interventions, and are often evaluated on standalone reasoning benchmarks rather than full agentic workflows. These assumptions may not hold in agentic AI systems, where reasoning requirements evolve dynamically through planning, tool use, memory retrieval, and agent-to-agent interactions. Consequently, reasoning can become either excessive or insufficient, resulting in unnecessary computation, increased latency, planning drift, excessive tool use, or incomplete solutions. We argue that a major challenge for next-generation agentic AI is not merely how much reasoning a language model should perform, but how it should allocate reasoning according to evolving task demands. We characterize over-reasoning and under-reasoning as recurring failure modes of misallocated reasoning and evaluate them on MATH-500 and the GAIA public validation benchmark. Using tool-decision latency, token consumption, token-limit exhaustion, and answer correctness, our results suggest that cases classified as over-reasoning are associated with higher computational cost without proportional accuracy gains, whereas cases classified as under-reasoning are consistently associated with incorrect or incomplete solutions. These findings motivate future research on adaptive reasoning mechanisms for agentic AI.
1 Introduction
Recent reasoning advances improve complex-task performance but impose substantial inference costs, motivating adaptive allocation of reasoning effort. This work studies over- and under-reasoning in agentic AI and proposes evaluation dimensions for deciding when to reason, act, and stop.
- Background: Reasoning advances, including Chain-of-Thought [16], ReAct [18], OpenAI o1, and Phi-4-reasoning [1], have strengthened LLM problem-solving capabilities.These developments support explicit step-by-step deliberation and reasoning-and-acting workflows.
- Motivation: Explicit reasoning increases inference latency, token consumption, and cost, while extensive reasoning is often unnecessary for simple tasks.Efficient inference therefore requires better allocation of computational resources during inference [4].
- Motivation: Adaptive reasoning is especially important for agentic AI, where effort should dynamically match task complexity to reduce costs while maintaining or improving performance.The introduction frames dynamic reasoning allocation as an important deployment challenge for modern AI systems.
- Contributions: The paper investigates over- and under-reasoning in agentic AI, linking excessive effort to higher inference cost and insufficient effort to degraded task performance.It quantitatively analyzes reasoning allocation in a LangGraph-based framework using representative reasoning models on MATH-500 and GAIA.
- Contributions: The study identifies adaptive-reasoning challenges and evaluation dimensions for determining when agents should reason, act, and stop.The motivating failure modes include redundant or weak reasoning, repeated tool usage, and skipped tool calls.
2 Related Works
Prior work shows that agentic reasoning can improve complex capabilities but also incurs substantial computational costs. Existing fixed-budget approaches may fail when reasoning demands evolve through tool, memory, and environmental interactions, motivating adaptive allocation that avoids counterproductive overthinking.
- Integrating search, coding, and memory agents can enhance complex reasoning capabilities in agentic AI systems [17].
- Deeper reasoning and test-time scaling introduce substantial computational and infrastructure costs [7].
- Fixed reasoning budgets and evaluations outside full agentic workflows may not reflect dynamically evolving demands from tools, memory, and external environments.Prolonged reasoning and replanning can also cause planning drift, although planning drift is not directly measured in this study.
- These limitations motivate asking when additional reasoning improves agent performance versus becoming counterproductive.The broader goal is efficient reasoning allocation while maintaining alignment with task objectives.
3 Preliminaries
Agentic AI systems operate autonomously toward complex goals in dynamic environments, requiring reasoning effort to adapt to task complexity. Inefficient allocation produces under-reasoning or over-reasoning, which can respectively leave tasks incomplete or increase computational cost unnecessarily.
- Agentic AI extends beyond predefined tasks by adapting to changing conditions, using contextual information, and pursuing complex objectives with minimal human intervention.
- Reasoning effort should adapt to task complexity rather than rely on uniformly long reasoning processes.Reasoning length and answer correctness can have a non-linear relationship, while stronger reasoning improves planning tasks but increases computation [11].
- Under-reasoning denotes insufficient reasoning, evidence gathering, or verification, including omitted steps, inadequate evidence, or premature termination.
- Over-reasoning denotes continuing beyond necessity through redundant deliberation, repeated verification, or unnecessary reasoning that increases computational cost.
4 Preliminary Study: Evidence of Misallocated Reasoning
The preliminary study evaluates an agentic AI framework on MATH-500 and GAIA using multiple final-response models and a Qwen3.5-4B router. Experiments ran on two NVIDIA RTX A6000 GPUs under a vLLM-compatible serving setup.
- 4 Preliminary Study: Evidence of Misallocated Reasoning: The framework evaluated all 500 MATH-500 test problems and 165 GAIA validation tasks [8].Evaluations used deterministic settings.
- 4 Preliminary Study: Evidence of Misallocated Reasoning: Qwen3.5-4B served as the router, while Qwen3.5-4B, Llama-3.1-8B-Instruct [2], and Phi-4-reasoning [3] served as final-response models.The Qwen3.5-4B configuration used the same model for routing and final response.
- 4 Preliminary Study: Evidence of Misallocated Reasoning: Experiments used a LangGraph-based agentic AI framework served through vLLM-compatible APIs on two NVIDIA RTX A6000 GPUs with 48 GB VRAM each.The hardware setup comprised two GPUs.
5 Findings
Across MATH-500 and GAIA, the evaluated LangGraph-based agent repeatedly exhibits over- and under-reasoning, misallocating effort as task demands evolve. Over-reasoning increases computational cost without proportional gains, while under-reasoning produces incorrect or incomplete solutions.
- Over- and under-reasoning recur across both reasoning-intensive MATH-500 and real-world multi-step GAIA tasks, causing higher computational cost or degraded performance.
- Phi-4-reasoning has the highest MATH-500 over-reasoning rate (89.40%) and the greatest reasoning tokens (2220.12), output length (4046.59 tokens), and inference time (180.80 s).
- On MATH-500, Phi-4-reasoning reaches 92.40% accuracy versus Qwen3.5-4B’s 87.60%, but takes 180.80 s versus 70.21 s and records 482 versus 190 token-limit hits.
- Under-reasoning is most prevalent for Llama-3.1-8B-Instruct, reaching 22.00% on MATH-500 and 71.52% on GAIA, with incomplete responses in both benchmarks.On MATH-500, the corresponding incomplete-response count is 64; on GAIA, it is 56.
- Reasoning and tool-use allocation depend on task demands: tool calls average 0.39 on MATH-500 versus 0.93 on GAIA, yet the highest GAIA accuracy remains only 12.12%.MATH-500 tool use consumes approximately 583 tool-input tokens and 1225 tool-output tokens, despite external information rarely being required.
6 Discussion and Future Directions
The findings identify reasoning efficiency as a challenge across language-reasoning models and agentic systems, with over-reasoning increasing cost without proportional accuracy gains and under-reasoning producing incorrect or incomplete solutions. Future work should develop adaptive mechanisms that allocate reasoning, tool use, and computation according to evolving task demands.
- Discussion: Higher reasoning can improve simpler-task accuracy but increases latency, token consumption, and budget exhaustion, while complex-agent reasoning raises cost without corresponding accuracy gains.Insufficient reasoning on complex agentic tasks often produces incorrect or incomplete solutions.
- Future Directions: Under-reasoning causes premature decisions when agents fail to recognize the need for additional reasoning, evidence gathering, or verification.
- Future Directions: Adaptive mechanisms should jointly allocate reasoning effort, tool usage, and token budgets as requirements evolve through observations from tools, memory, or the environment.
- A Candidate Adaptive-Reasoning Controller: An adaptive-reasoning controller could decide whether to continue reasoning, invoke tools, or answer based on reasoning state, task requirements, and expected computational benefit.This approach replaces fixed reasoning budgets with dynamic allocation throughout execution.
- A Measurable Evaluation Suite: Future evaluation suites should measure reasoning allocation and efficiency, including accuracy per unit of reasoning effort, rather than task accuracy alone.
7 Conclusion
The paper identifies over-reasoning and under-reasoning as recurring reasoning-allocation failures in agentic AI. Over-reasoning increases computational and interaction costs without proportional task-performance gains, while under-reasoning is associated with incorrect outcomes.
- 7 Conclusion: Over-reasoning is associated with higher latency, greater token consumption, more frequent tool usage, and increased token-limit exhaustion without proportional improvements in task performance.These findings characterize over-reasoning as inefficient reasoning allocation in agentic AI.
- 7 Conclusion: Under-reasoning is consistently associated with incorrect outcomes.The supplied passage ends mid-sentence after “incorrect or,” so no further outcome is reported.
- 7 Conclusion: The paper frames over-reasoning and under-reasoning as recurring failure modes of reasoning allocation in agentic AI.The conclusion summarizes the investigation as concerning reasoning efficiency in agentic AI.
8 Limitations
This preliminary study has limited generalizability because it uses one agentic framework, fixed models, and independently classified reasoning regimes. Its cost analysis and experimental design also omit several relevant failure sources, resource measures, baselines, budget conditions, and distributional statistics.
- 8 Limitations: Findings should not be generalized beyond the LangGraph framework, fixed Qwen3.5-4B router, and three final-response models tested.The study does not establish applicability to other agentic architectures, routing strategies, or foundation models.
- 8 Limitations: Reasoning classifications do not distinguish tool-routing, retrieval, reasoning, or formatting failures that may contribute to incorrect predictions.Regimes are classified independently from answer correctness, limiting causal interpretation of errors.
- 8 Limitations: Cost evaluation covers latency, token consumption, tool usage, and token-limit hits, but omits monetary cost, GPU utilization, energy consumption, and throughput.These measures characterize inference cost without covering the listed operational resource dimensions.
- 8 Limitations: Experiments fix maximum generation at 4,096 tokens and omit token-budget ablations, no-tool baselines, standalone LLM baselines, and distributions beyond averages.Future work is identified as needing broader budget conditions and statistics than the reported averages.
Ethics and Reproducibility Statement
The work frames over- and under-reasoning as recurring agentic-AI failure modes and supports reproducibility through planned release of its implementation materials.
- Ethics and Reproducibility Statement: Over- and under-reasoning can increase computational cost or degrade task performance in agentic AI systems.These findings motivate future research on more adaptive reasoning mechanisms.
- Ethics and Reproducibility Statement: The authors will release the complete LangGraph-based framework, experimental configurations, and prompts, including the GPT-4.1 judge prompt.