Source-linked AI summary
Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models
Shilin Yan, Jintao Tong, Hongwei Xue, Xiaojun Tang, Yangyang Wang, Kunyu Shi, Guannan Zhang, Ruixuan Li, Yixiong Zou
TL;DR
Agentic multimodal models often overuse tools because coupled reward optimization cannot reliably balance correctness and efficiency. The paper proposes HDPO, which separates accuracy optimization from correctness-conditioned tool parsimony, and reports that Metis sharply reduces tool use while improving reasoning performance.
Problem
Agentic multimodal models frequently invoke tools even when visual inputs suffice, while scalarized rewards either suppress necessary tool use or fail to overcome accuracy variance.
Method
HDPO uses separate accuracy and efficiency channels, applying conditional advantage estimation so tool parsimony is optimized only within accurate trajectories.
Results
Metis reduces tool usage by over 90% while simultaneously elevating reasoning accuracy across diverse benchmarks.
Takeaways & Limitations
HDPO and Metis show that selective tool use can coexist with, and in the reported evaluations accompany, strong multimodal reasoning performance.
Abstract
from arXiv · showhide
The advent of agentic multimodal models has empowered systems to actively interact with external environments. However, current agents suffer from a profound meta-cognitive deficit: they struggle to arbitrate between leveraging internal knowledge and querying external utilities. Consequently, they frequently fall prey to blind tool invocation, resorting to reflexive tool execution even when queries are resolvable from the raw visual context. This pathological behavior precipitates severe latency bottlenecks and injects extraneous noise that derails sound reasoning. Existing reinforcement learning protocols attempt to mitigate this via a scalarized reward that penalizes tool usage. Yet, this coupled formulation creates an irreconcilable optimization dilemma: an aggressive penalty suppresses essential tool use, whereas a mild penalty is entirely subsumed by the variance of the accuracy reward during advantage normalization, rendering it impotent against tool overuse. To transcend this bottleneck, we propose HDPO, a framework that reframes tool efficiency from a competing scalar objective to a strictly conditional one. By eschewing reward scalarization, HDPO maintains two orthogonal optimization channels: an accuracy channel that maximizes task correctness, and an efficiency channel that enforces execution economy exclusively within accurate trajectories via conditional advantage estimation. This decoupled architecture naturally induces a cognitive curriculum-compelling the agent to first master task resolution before refining its self-reliance. Extensive evaluations demonstrate that our resulting model, Metis, reduces tool invocations by orders of magnitude while simultaneously elevating reasoning accuracy.
1 Introduction
Agentic multimodal models often invoke tools reflexively because they cannot reliably arbitrate between internal knowledge and external utilities. HDPO addresses this by decoupling accuracy from conditional efficiency optimization, and Metis combines selective tool use with strong reasoning performance.
- Motivation: Agentic multimodal models struggle to decide whether external tools are genuinely necessary when visual context may already suffice.This meta-cognitive deficit leads to blind tool invocation.
- Motivation: Coupling accuracy and tool efficiency into one reward creates a dilemma: strong penalties suppress necessary tools, while mild penalties are overwhelmed by accuracy variance.The resulting optimization can either harm correctness or fail to curb overuse.
- HDPO: HDPO separates accuracy and efficiency into orthogonal channels, conditioning tool-parsimony optimization on accurate trajectories.This design lets the agent prioritize task resolution before refining self-reliance.
- Metis: Metis selectively invokes coding and searching tools only when they provide useful evidence or computation, rather than treating invocation as a default reflex.The approach targets redundant interactions while preserving access to necessary tools.
- Results: Over 90% tool-use reduction accompanies elevated reasoning accuracy, supporting the claim that strategic efficiency need not trade off against performance.The contribution summary reports this result across diverse benchmarks.
2 Related Works
Agentic multimodal models extend reasoning through external interaction, but this flexibility expands the decision space and exposes a persistent tool-use efficiency problem.
- Agentic multimodal models: Agentic MLLMs invoke external tools during inference to support detailed inspection, iterative evidence gathering, and intermediate reasoning.Common multimodal tools include cropping, grounding, and image search.
- Agentic multimodal models: These systems must decide whether to call a tool, which tool to use, and how to incorporate returned observations into subsequent reasoning.The added decisions distinguish agentic interaction from one-shot prediction.
- Research gap: Prior work has emphasized stronger tool capabilities and multi-step interaction more than tool-use efficiency.The paper identifies efficiency awareness as comparatively underexplored.
3 Method
Metis formalizes multi-turn tool-augmented reasoning and introduces HDPO to remove cross-objective interference while inducing a learning curriculum.
- Method: The method formalizes multi-turn tool-augmented reasoning before analyzing coupled reward formulations and presenting HDPO.This organization connects the problem formulation to the proposed optimization framework.
3.1 Problem Formulation & The Reward Coupling Problem
The coupled-reward formulation combines correctness and tool parsimony before advantage normalization, mathematically entangling their optimization signals. This produces gradient interference, semantic ambiguity, and unstable trade-offs that can suppress efficiency learning.
- Problem formulation: Each response contains multi-turn reasoning with T_i tool interactions before producing a final answer.The formulation considers a group of G responses generated by policy πθ.
- Coupled reward: The coupled reward combines correctness, formatting, and tool parsimony through a balancing parameter α before computing policy advantages.This scalarization is the basis of the shared normalization problem.
- Coupled reward: Shared advantage normalization entangles accuracy and tool-efficiency objectives because correctness and tool use are correlated, yielding a covariance term.The paper identifies this entanglement as producing three concrete pathologies.
- Pathologies: Gradient entanglement makes each policy-gradient update depend inversely on the other objective’s variance, causing destructive interference.Accuracy updates depend on tool-usage variance and efficiency updates on accuracy variance.
- Pathologies: Correct-but-inefficient and incorrect-but-efficient trajectories can receive indistinguishable scalar rewards, producing near-zero advantages for both.This semantic ambiguity neutralizes learning signals for important edge cases.
- Pathologies: When α is small, the efficiency gradient is bounded by O(α) and attenuated by accuracy variance σ_acc, so its signal vanishes asymptotically.Reducing α to protect accuracy therefore undermines the ability to curb blind tool invocation.
3.2 HDPO: Hierarchical Decoupled Policy Optimization
HDPO separates accuracy and tool-efficiency optimization, conditioning efficiency comparisons on correct trajectories. This design supports an implicit curriculum that prioritizes correctness before parsimony while preserving orthogonal learning signals.
- HDPO computes separate accuracy and efficiency advantages, each grounded in its own semantic baseline, instead of combining rewards before normalization.
- Accuracy Channel: The accuracy reward combines correctness and format compliance, with λa=0.9 and λf=0.1, and its advantage is estimated across all G rollouts.
- Efficiency Channel: The tool reward penalizes increasing tool calls monotonically, assigning T=0 7→1.0 and T=1 7→0.5 while preserving a smooth preference structure.
- Efficiency Channel: Tool-efficiency advantages are computed only among correct rollouts, because incorrect trajectories must not receive reward for merely terminating quickly.
- Efficiency Channel: When fewer than two rollouts are correct, HDPO assigns zero efficiency advantage to avoid invalid within-group comparisons.
- Hierarchical Policy Update: HDPO linearly combines the two clipped surrogate losses, allowing independently normalized gradients to provide orthogonal signals without destructive covariance interference.
- Implicit Curriculum: The qualifying set is initially often empty, so accuracy dominates early training; as more rollouts become correct, the efficiency signal scales up without manual scheduling.
3.3 Training Data Curation
The training-data curation pipeline targets unreliable tool-augmented trajectories and reward environments. It removes hallucinated interactions, filters unnecessary tool use, evaluates reasoning quality, and calibrates prompt difficulty.
- The curation pipeline addresses hallucinated environmental dynamics and obsolete tool dependencies that can compromise behavioral priors and environmental feedback.
- Eradicating Hallucinated Environmental Dynamics: All code segments are executed in a sandbox, and trajectories with execution failures or inconsistent feedback are discarded.
- Isolating Genuine Tool Necessity: A zero-shot solvability baseline is used to identify legacy annotations that would condition the model toward blind tool invocation on simple queries.
- Multidimensional Meta-Cognitive Filtering: Gemini-3.1-Pro evaluates visual relevance, reasoning coherence, and tool-use rationale, explicitly penalizing blind tool invocation.
- RL Prompt Curation: RL prompts are filtered for image quality, question clarity, and image-text consistency so corrupted or ambiguous inputs do not supply noisy feedback.
- Variance-Aware Difficulty Calibration: With G = 8 rollouts per prompt, the pipeline retains prompts containing a non-trivial mixture of successes and failures to produce actionable GRPO gradients.
4 Experiments
Metis is evaluated across visual, document, mathematical, and logical reasoning benchmarks, where strategic tool use and decoupled optimization improve accuracy while supporting selective tool invocation.
- Experimental Setup: Metis is evaluated on perception, document understanding, mathematical reasoning, and logical reasoning benchmarks against open-source and agentic baselines.The benchmark suite includes V*Bench, HRBench, TreeBench, MME-RealWorld, SEEDBench2-Plus, CharXiv, MathVista-mini, MathVerse-mini, WeMath, DynaMath, and LogicVista.
- Main Results: 83.5% on HRBench-4K and 82.0% on HRBench-8K outperform all existing agentic models, including the 30B-parameter Skywork-R1V4.On CharXiv reasoning questions, Metis reaches 54.1% versus 48.9% for DeepEyesV2.
- Main Results: 66.9% average across five mathematical and logical reasoning datasets substantially outperforms text-only and agentic multimodal models.On WeMath, Metis scores 65.2%, a +26.4% absolute gain over its backbone’s 38.8% and above DeepEyesV2’s 38.1%.
- Ablation Studies: +2.4%, +2.8%, and +3.1% gains over standard GRPO occur on V*Bench, HRBench8K, and CharXiv (RQ), respectively, with HDPO suppressing redundant tool invocations.The ablation compares identical backbone and training-data configurations; wtool=0 reduces HDPO to accuracy-only standard GRPO.
- Ablation Studies: wtool=0.15 yields the best balance, while wtool=0.10 is too weak and wtool=0.20 degrades performance across benchmarks.The reported sensitivity follows an inverted-U pattern: insufficient efficiency pressure leaves blind invocation insufficiently curbed, whereas excessive pressure stifles necessary tool use.
- Meta-Cognitive Tool Arbitration: Metis abstains when visual understanding and prior knowledge suffice, but invokes code to crop and enlarge ambiguous fine-grained regions.Figure 4 illustrates direct reasoning without tools; Figure 5 illustrates targeted code execution for comparing curves near a queried time step.
5 Conclusion
The work identifies blind tool invocation as a critical failure mode and proposes HDPO to separate task accuracy from tool efficiency. Metis consequently reduces tool invocations by orders of magnitude while achieving state-of-the-art reasoning performance.
- HDPO addresses blind tool invocation by decoupling task accuracy and tool efficiency into orthogonal channels via conditional advantage estimation.The framework is complemented by a rigorous data curation pipeline and induces a cognitive curriculum.
- Metis reduces tool invocations by orders of magnitude while achieving state-of-the-art reasoning performance.
- Future work will extend the meta-cognitive framework to more open-ended, long-horizon environments.
A System Prompt
The system prompt defines the available tools, their formats, and execution guidance. It instructs the agent to reason before acting, answer directly when possible, and use tools only when they provide clear value.
- The prompt defines the available tools, their calling formats, and the execution environment.
- Decision guidelines encourage reasoning before acting, direct answers whenever possible, and tool calls only when genuinely necessary or useful.
- Dedicated <reason>, <tool_call>, and <answer> fields specify the required output structure and promote consistent, efficient behavior.
B Additional Case Studies
The case studies show Metis abstaining when visual inspection suffices and selectively searching when visual evidence cannot provide the requested identification or factual detail.
- Metis extracts clearly legible on-screen text directly from the image without invoking code execution or search tools.This avoids unnecessary computational overhead.
- When artwork identification is unreliable from visual features alone, Metis invokes image search to match the content against external references and retrieve its completion year.
- When a monument is visually identifiable but its cella width cannot be inferred from the image, Metis uses text search to retrieve the precise measurement.
- Metis recognizes that the image lacks direct measurements and that historical or archaeological sources are needed for the cella width.