PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning

Mingyang Song, Haoyu Sun, Jiawei Gu, Linjie Li, Luxin Xu, Ranjay Krishna, Yu Cheng

arXiv:2601.18631v2cs.AIcs.CLcs.CVcs.MA

TL;DR

MLLMs need to learn which tools to use, when to invoke them, and how to compose them across multi-step tasks and unseen tool settings. AdaReasoner combines long-horizon trajectory curation, Tool GRPO, and adaptive learning to train generalizable tool use, achieving strong adaptive behavior and benchmark performance while retaining known instability in some inference-time tool choices.

  • Problem

    Existing methods struggle to adaptively plan and coordinate diverse tools across tasks, especially with unseen tools or novel task distributions.

  • Method

    AdaReasoner integrates multi-turn trajectory curation, Tool GRPO, and adaptive learning that randomizes tool definitions across training stages.

  • Results

    +24.9% average improvement is achieved by the 7B variant across diverse benchmarks, with AdaReasoner surpassing GPT-5 and other proprietary models on multiple tasks.

  • Takeaways & Limitations

    AdaReasoner learns to adopt beneficial tools, suppress irrelevant ones, and generalize tool-planning capabilities to unseen tasks and tool definitions.

  • Takeaways & Limitations

    Inference-time adaptability to newly available tools remains unstable, with distractor-tool invocation reducing performance and requiring reinforcement learning for stabilization.

Abstract

from arXiv · show

When humans face problems beyond their immediate capabilities, they rely on tools, providing a promising paradigm for improving visual reasoning in multimodal large language models (MLLMs). Effective reasoning, therefore, hinges on knowing which tools to use, when to invoke them, and how to compose them over multiple steps, even when faced with new tools or new tasks. We introduce \textbf{AdaReasoner}, a family of multimodal models that learn tool use as a general reasoning skill rather than as tool-specific or explicitly supervised behavior. AdaReasoner is enabled by (i) a scalable data curation pipeline exposing models to long-horizon, multi-step tool interactions; (ii) Tool-GRPO, a reinforcement learning algorithm that optimizes tool selection and sequencing based on end-task success; and (iii) an adaptive learning mechanism that dynamically regulates tool usage. Together, these components allow models to infer tool utility from task context and intermediate outcomes, enabling coordination of multiple tools and generalization to unseen tools. Empirically, AdaReasoner exhibits strong tool-adaptive and generalization behaviors: it autonomously adopts beneficial tools, suppresses irrelevant ones, and adjusts tool usage frequency based on task demands, despite never being explicitly trained to do so. These capabilities translate into state-of-the-art performance across challenging benchmarks, improving the 7B base model by +24.9\% on average and surpassing strong proprietary systems such as GPT-5 on multiple tasks, including VSP and Jigsaw.

1 INTRODUCTION

Existing MLLM tool-use methods struggle to plan and coordinate diverse tools adaptively across multi-step tasks. AdaReasoner addresses this gap with curated multi-turn trajectories, Tool GRPO, and adaptive learning, achieving strong benchmark gains and generalization.

  • Current approaches rely on rigid invocation patterns, fixed interaction loops, or single-tool trajectories rather than autonomous, task-aware planning.
  • AdaReasoner combines multi-turn trajectory curation, Tool GRPO for long-horizon planning, and adaptive learning for generalization to unseen domains.Adaptive learning is integrated into both training stages and decouples tool-use logic from specific tasks.
  • +24.9% average improvement is achieved by the 7B variant across diverse benchmarks, while surpassing GPT-5 and Claude Sonnet 4 on multiple tasks.
  • AdaReasoner autonomously adopts beneficial tools, discards irrelevant ones, and modulates usage frequency while generalizing to unseen tool definitions and novel tasks.
  • The method delivers performance competitive with or superior to leading proprietary models, elevating smaller open-source models on structured-reasoning tasks.

2 METHOD

AdaReasoner formulates visual tool use as sequential multimodal decision-making and trains multi-turn reasoning through curated trajectories and Tool GRPO. Adaptive rewards and randomized tool definitions encourage reliable tool selection, sequencing, and generalization.

  • 2.1 PRELIMINARY: The model generates trajectories of state-action-observation tuples while selecting actions from a predefined set of visual tools.Each tool call produces an observation that informs the next state.
  • 2.1 PRELIMINARY: The visual toolkit covers perception, manipulation, and calculation through both offline and online tools.Examples include POINT and OCR for perception, DRAWLINE and INSERTIMAGE for manipulation, and ASTAR for calculation.
  • 2.2 HIGH-QUALITY TRAJECTORY DATA CURATION: Trajectory curation designs task-specific blueprints, executes tool calls, and generates Chain-of-Thought connecting why and how tools are used.The curated trajectories include reflection, backtracking, and explicit tool-failure scenarios.
  • 2.3 MULTI-TURN TOOL GRPO: Tool GRPO extends GRPO to multi-turn trajectories using accumulated rewards for formatting, tool-call quality, and final-answer accuracy.Formatting is mandatory at every step, tool calls receive hierarchical scores, and accuracy is evaluated on the final turn.
  • 2.3 MULTI-TURN TOOL GRPO: The adaptive reward gives full credit to correct predictions regardless of tool use, while incorrect predictions receive partial credit according to informative tool usage.
  • 2.4 ADAPTIVE LEARNING: Adaptive learning randomizes tool names, parameter names, and descriptions to discourage identifier memorization and promote semantic tool selection.Identifiers are replaced with random strings, while descriptions are semantically paraphrased.

3 EXPERIMENTS

AdaReasoner is evaluated across multimodal reasoning tasks to measure tool augmentation, adaptive tool planning, and generalization. Results show substantial gains, including task-aware tool regulation and strong performance under unseen tools and task contexts.

  • Experimental Setup: The experiments evaluate Qwen2.5-VL models across visual spatial planning, Jigsaw, WebQA, and visual search tasks.The setup probes complementary aspects of multimodal reasoning, including planning, perceptual grounding, and visual compositionality.
  • Visual Tools and Capability: Tool augmentation can shift the performance bottleneck from model scale toward tool quality, with 3B and 7B variants reaching 94.7% and 97.6% accuracy.Both tool-augmented models achieve near-perfect accuracy despite disparate and low baseline performance.
  • Adaptive Tool Planning: Introducing A* only at inference raises VSP navigation from 44.83 to 62.33, with 94.53% invocation success in a zero-shot setting.The model learns the newly introduced tool’s syntax and purpose without prior exposure during Tool-Cold-Start.
  • Adaptive Tool Planning: Adaptive tool use remains unstable when a newly available tool is irrelevant, causing verification performance to drop from 94.20 to 80.00.The passage identifies reinforcement learning as still required to stabilize inference-time adaptation.
  • Adaptive Tool Planning: AdaReasoner regulates tool usage by increasing Point calls for navigation to ∼3.2 calls/sample while keeping verification usage near ∼1.0 call/sample.The pattern demonstrates task-dependent invocation frequency rather than uniform tool use.
  • Generalization: Randomized Tool-Cold-Start and Tool-GRPO improve unseen-task generalization, raising VSP from 28.09 to 78.91 and the three-task average from 46.50 to 75.81.Other training settings fail to improve unseen tasks or can degrade performance.
  • Main Results: Across visual reasoning and general multimodal tasks, AdaReasoner delivers consistent gains and achieves the highest reported effective tool usage under zero-shot tool-definition shifts.On Jigsaw it records 3.54 calls/sample with 98.50% success, while on VStar it reaches 70.68 accuracy with 1.47 calls/sample.

4 RELATED WORK

Prior multimodal tool-use work established infrastructure, synthetic reasoning traces, and visual tools, but existing approaches remain limited in adaptive multi-step coordination.

  • RL-based multimodal methods improve perceptual reasoning but typically use single-tool trajectories or fixed interaction loops.
  • Earlier tool-use approaches relied on dedicated tool servers, synthetic manipulation data, or reasoning traces derived from visual tools.
  • Code-based environments are powerful but are ill-suited to integrating computationally intensive capabilities such as large expert models.
  • Atomic visual-tool approaches generally focus on single-step actions rather than complex multi-step tool interaction.

5 CONCLUSION

AdaReasoner combines trajectory curation, Tool-GRPO, and adaptive learning to support multi-turn tool planning and generalization to unseen tools. Its models coordinate tools selectively and achieve strong visual-reasoning performance, including a 7B model surpassing GPT-5 on challenging tasks.

  • AdaReasoner integrates high-quality trajectory curation, Tool-GRPO, and adaptive learning for multi-turn tool planning.
  • AdaReasoner selectively adopts beneficial tools, suppresses irrelevant ones, and modulates usage frequency according to task demands while generalizing to unseen tool definitions and novel tasks.
  • A 7B model surpasses GPT-5 on challenging visual reasoning tasks, shifting the reported bottleneck from intrinsic model scale toward tool utility.
  • The framework formalizes tool-integrated reasoning as trajectories consisting of repeated states, actions, and observations.
  • Its visual tools support localization, path visualization, shortest-path planning, missing-region detection, patch insertion, cropping, and OCR.

A.2 HIGH-QUALITY COLD START TRAJECTORY DATA CURATION

The cold-start pipeline constructs diverse, executable multi-turn trajectories for VSP, Jigsaw, and GUIQA, then trains tool use with structured rewards and Tool-GRPO. The data includes verification, trial-and-error, tool synergy, and failure-aware interactions.

  • The pipeline creates high-quality, diverse trajectories for structured reasoning tasks and executes them to populate real tool inputs and outputs.
  • VSP: VSP training uses smaller procedurally generated grids while reserving larger grids for testing, supporting varied navigation and verification trajectories.
  • Task trajectories: Jigsaw trajectories train iterative patch testing after detecting the missing region, while GUIQA trajectories combine cropping with OCR for focused extraction.
  • Tool-GRPO: Tool-GRPO samples complete trajectory groups, scores them, and updates the policy toward trajectories with higher group-relative advantages.
  • Reward design: The reward design evaluates formatting, tool-call structure, tool names, parameter correctness, and final-answer accuracy.
  • Reward design: A binary format gate nullifies tool and accuracy rewards when the required output structure is invalid.

B.1 TASK DEFINITION

The evaluation covers diverse visual reasoning tasks, while the AdaReasoner framework supports tool planning through unified tool management, curated trajectories, and online reinforcement learning. The setup also examines robustness to changed tool definitions and input order.

  • Task Definition: The evaluation spans spatial planning, jigsaw reconstruction, GUI understanding, and open-ended visual reasoning.These tasks probe perception, grounding, multi-step reasoning, compositional understanding, and general visual ability.
  • Visual Spatial Planning: VSPO tests out-of-distribution spatial generalization by training on 4 × 4, 6 × 6, and 8 × 8 maps and testing on unseen sizes.The benchmark also examines whether models leverage tool usage for problem solving.
  • Jigsaw: Jigsaw-COCO reserves bottom-right image patches for testing, while BLINK-J evaluates fine-grained visual reasoning and compositional understanding.The split design probes generalization beyond the patch configurations seen during training.
  • GUIQA: GUI evaluation uses GUIChat and WebQA, with WebQA’s agent acting subset reported in the main table to highlight visual-tool effectiveness.WebQA also organizes evaluation around agentic action, general visual comprehension, and multi-step reasoning.
  • Framework and Training: AdaReasoner’s framework manages tools through a unified Tool Server and generates cold-start trajectories before Tool-GRPO refinement with live execution feedback.The training pipeline includes supervised fine-tuning followed by online reinforcement learning and adaptive learning across task structures.
  • Evaluation Robustness: Model robustness is evaluated under changed tool definitions and swapped image–question order, with stable performance reported on V*.The evaluation compares proprietary and open-source multimodal baselines under a unified setting.

B.4.2 EVALUATION METHOD

AdaEval provides a unified protocol for comparing tool-planning and non-tool-planning models. Subjective benchmarks use Qwen-2.5-VL-72B as an LM judge, with human evaluation on V* supporting the automated scores.

  • Unified Evaluation: AdaEval evaluates tool-planning and non-tool-planning models under the same evaluation protocol.This framework is designed to ensure fair and consistent comparisons.
  • LM-as-a-Judge: For open-ended or subjective benchmarks, Qwen-2.5-VL-72B serves as the LM-as-a-Judge model.The strategy is applied to VStar, WebMMU, and GUIQA.
  • Human Validation: Human judgment scores on V* are consistent with the automated evaluation scores.The human evaluation manually assesses extracted predictions from Qwen 2.5 VL 7B and AdaReasoner-7B.

B.4.3 DETAILED RESULTS

The paper reports abridged main-paper results while providing full benchmark breakdowns and detailed training-stage analyses in supplementary tables.

  • Detailed Results: Full results expand the main paper’s analyses of TC and TG contributions, generalization performance, and benchmark-level outcomes.The detailed results are provided in Tables 13 and 12, alongside additional reported sections.

B.5 ABLATION STUDY

The ablation study varies reward weights to examine their effects on learning dynamics and final VSP performance. Increasing the tool-reward ratio consistently improves outcomes.

  • Ablation Setup: The study trains on VSP for 100 RL steps under different λtool and λacc settings before evaluating each checkpoint.Training curves are monitored to assess convergence across reward-weight configurations.
  • Reward Weights: Performance consistently improves as the ratio λtool : λacc increases.The reported trend indicates that reward weighting materially affects final VSP and VSPO performance.
  • Interpretation: Larger tool rewards accelerate RL convergence and improve final performance, supporting the effectiveness of the tool-reward design.The paper links the reward design to more efficient and robust tool-calling learning.

C.1 VISUAL TOOLS HELP OVERCOME SCALE-BASED LIMITATIONS

Tool augmentation overcomes scale-based limitations in MLLMs: despite disparate, low baseline performance, tool-augmented 3B and 7B models reach near-perfect accuracy.

  • 94.7% and 97.6%: tool-augmented 3B and 7B models achieve near-perfect accuracy despite disparate, low baseline performance.

C.2 WHY VISUAL TOOLS HELP

Visual tools help MLLMs by delegating perception, verification, and intermediate computation to specialized operations, improving reasoning beyond what internal processing alone can reliably provide.

  • Visual tools delegate precise sub-tasks, allowing models to focus on judgment, synthesis, and integrating tool outputs.
  • Perception tools: Expert perception tools provide structured visual observations that compensate for MLLMs’ perceptual limitations and improve downstream reasoning.
  • Perception tools: 100.0% localization accuracy: POINT substantially outperforms baselines at approximately 50.0%, while its coordinates improve zero-shot reasoning by +18.79 points on average.
  • Manipulation tools: Manipulation tools externalize hypotheses through explicit visual operations, converting abstract reasoning into concrete verification decisions.
  • Trajectory data: High-quality multi-turn trajectories teach models how to use tools and recognize when tool-use patterns should be applied.

C.3 THE DUAL ROLE OF COLD-START SUPERVISION

Cold-start supervision has a task-dependent dual role: it helps structured tasks with discernible optimal solutions but can constrain exploration in open-ended domains. The section therefore contrasts expert-guided SFT with pure Tool-GRPO for dynamic tasks.

  • Tool Cold Start supervision trades imparting expert knowledge against preserving exploratory freedom, so its value depends strongly on task nature.The results do not support treating supervised pre-training as universally beneficial.
  • Structured tasks: For structured tasks such as VSP and Jigsaw, SFT provides a decisive advantage by exposing the model to high-quality, deterministic solution paths.From-scratch discovery is difficult when the model has reasoning or knowledge deficits.
  • Open-ended tasks: For open-ended domains such as GUIQA, human-designed trajectories may be sub-optimal because the best tool-use strategy is unknown even to designers.Rigid SFT can create a strong policy bias that restricts later reinforcement-learning exploration.
  • Training principle: Pure reinforcement learning through Tool-GRPO may be preferable for dynamic, general tasks that benefit from unconstrained exploration.The proposed principle is to inject expert knowledge when solution spaces are well defined and preserve exploration otherwise.
Loading 2601.18631v2…