PaperlayerView exact revision on arXiv

Source-linked AI summary

Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning

Jiaqi Liu, Kaiwen Xiong, Peng Xia, Yiyang Zhou, Haonian Ji, Lu Feng, Siwei Han, Mingyu Ding, Huaxiu Yao

arXiv:2511.19900v2cs.CVcs.AI

TL;DR

Agent0-VL targets the limits of human or external-reward supervision and unreliable text-only self-evaluation in visual reasoning. It combines tool-assisted solving, verification, and repair in a Self-Evolving Reasoning Cycle, achieving a reported 12.5% improvement over the base model while also improving test-time scaling.

  • Problem

    Human or external-reward supervision is limited, while text-only self-evaluation struggles to verify complex visual reasoning and can produce evaluation hallucinations.

  • Method

    Agent0-VL unifies a tool-using Solver and tool-grounded Verifier in a Self-Evolving Reasoning Cycle with reinforcement-learning updates, without external rewards.

  • Results

    Agent0-VL-7B achieves a 12.5% average improvement over the Qwen-VL base model and a 7.3% average gain in test-time scaling as a process reward model.

  • Takeaways & Limitations

    Tool-grounded verification and iterative self-repair support continual improvement and stronger factual consistency across visual reasoning benchmarks.

Abstract

from arXiv · show

Vision-language agents have achieved remarkable progress in a variety of multimodal reasoning tasks; however, their learning remains constrained by the limitations of human-annotated supervision. Recent self-rewarding approaches attempt to overcome this constraint by allowing models to act as their own critics or reward providers. Yet, purely text-based self-evaluation struggles to verify complex visual reasoning steps and often suffers from evaluation hallucinations. To address these challenges, inspired by recent advances in tool-integrated reasoning, we propose Agent0-VL, a self-evolving vision-language agent that achieves continual improvement with tool-integrated reasoning. Agent0-VL incorporates tool usage not only into reasoning but also into self-evaluation and self-repair, enabling the model to introspect, verify, and refine its reasoning through evidence-grounded analysis. It unifies two synergistic roles within a single LVLM: a Solver that performs multi-turn tool-integrated reasoning, and a Verifier that generates structured feedback and fine-grained self-rewards through tool-grounded critique. These roles interact through a Self-Evolving Reasoning Cycle, where tool-based verification and reinforcement learning jointly align the reasoning and evaluation distributions for stable self-improvement. Through this zero-external-reward evolution, Agent0-VL aligns its reasoning and verification behaviors without any human annotation or external reward models, achieving continual self-improvement. Experiments on geometric problem solving and visual scientific analysis show that Agent0-VL achieves an 12.5% improvement over the base model. Our code is available at https://github.com/aiming-lab/Agent0.

1. Introduction

Agent0-VL addresses limits of human or external-reward supervision and unreliable text-only self-evaluation by integrating tools into reasoning, verification, and self-repair. Its Solver and Verifier interact in a reinforcement-learning-based cycle, yielding reported gains over baseline and tool-integrated models.

  • Motivation: Text-only self-evaluation struggles with complex visual reasoning and can reward linguistically plausible but visually incorrect answers.The cited limitations include multi-step computation, spatial reasoning, geometric calculation, and evaluation hallucination.
  • Approach: Agent0-VL integrates external tools into reasoning, self-evaluation, and self-repair for closed-loop improvement without external rewards.The framework unifies reasoning, verification, and self-repair within one LVLM.
  • Approach: The Solver performs multi-turn tool-assisted reasoning, while the Verifier validates intermediate steps and generates rewards and repair instructions.These roles provide complementary reasoning and tool-grounded critique.
  • Approach: The Self-Evolving Reasoning Cycle combines inner-loop tool verification and repair with outer-loop GRPO updates to align reasoning and evaluation behaviors.This changes learning from static reward maximization toward continual distributional self-consistency.
  • Results: Agent0-VL-7B improves average performance by 12.5% over Qwen-VL and raises test-time scaling performance by 7.3% as a process reward model.The reported experiments cover geometric reasoning and visual scientific analysis.

2. Preliminaries

The agent’s multimodal reasoning process is modeled as a partially observable Markov decision process with latent states, textual or tool actions, observations, and trajectories. Tool feedback drives state transitions across reasoning-tool-feedback interactions.

  • POMDP Formulation: The reasoning process is formulated as a POMDP M = (S, A, O, T, R, γ), with γ as the discount factor and R as the reward function.The formulation explicitly models multimodal reasoning dynamics and tool feedback.
  • State Space: Each latent state encodes textual context, visual features, and previous tool input-output traces.State representation combines the information needed for multimodal reasoning and tool interaction.
  • Action Space: Actions are either textual reasoning steps or structured tool invocations that execute external programs such as Python code.The action space is the union A = A_text ∪ A_tool.
  • Observation Space: Observations contain tool or environment feedback, and T models how the reasoning state changes after an action and its feedback.Feedback may include returned numerical results or textual retrievals.
  • Trajectory: A trajectory records successive state, action, and observation triples, with each transition representing one reasoning-tool-feedback interaction.For input x = (I, q), the trajectory is a sequence of T such transitions.

3. Methodology

Agent0-VL uses one LVLM in alternating Solver and Verifier roles, combining tool-grounded reasoning, verification, and selective self-repair. A self-evolving cycle converts these interactions into process rewards and GRPO policy updates.

  • Unified Solver-Verifier Architecture: Agent0-VL unifies a Solver for tool-integrated reasoning and a Verifier for tool-grounded critique, rewards, and self-repair within one LVLM.The shared policy alternates between roles while incorporating tool or environment feedback into multimodal reasoning.
  • Tool-Grounded Verification and Self-Repair: Tool-grounded verification combines language reflection with executable evidence to produce dense, interpretable process-level feedback for reinforcement learning.The Verifier can re-invoke external tools to cross-check intermediate reasoning steps.
  • Tool-Grounded Verification and Self-Repair: Low-confidence verification activates selective repair: the Verifier issues a local instruction, the Solver regenerates the affected segment, and unnecessary repairs incur a penalty.The repair decision uses a confidence threshold and includes a repair cost in the effective reward.
  • Self-Evolving Reasoning Cycle: The Self-Evolving Reasoning Cycle couples an inner loop of trajectory generation, verification, and repair with an outer GRPO loop that updates the shared policy.Training begins with supervised initialization for tool usage and verification formats before reinforcement-learning evolution.
  • Self-Evolving Reasoning Cycle: The training algorithm aggregates final-outcome and discounted process rewards, normalizes trajectory advantages relative to a sampled group, and uses KL regularization for stability.GRPO increases the likelihood of trajectories with positive relative advantages while constraining policy divergence.

4. Experiments

Agent0-VL improves visual reasoning across seven benchmarks through iterative self-evolution, tool-grounded verification, and self-repair, while its Verifier also strengthens test-time trajectory selection for other LVLMs.

  • Overall Performance: 12.5% improvement over Qwen2.5-VL-7B demonstrates Agent0-VL-7B’s overall gain across visual reasoning benchmarks.It also gains 10.3% over Qwen2.5-VL-7B-TIR and 4.29% over ThinkLite-VL-7B.
  • Performance Across Task Domains: Agent0-VL improves across task domains, with 18.1% and 7.4% gains on math benchmarks and 12.2% and 3.1% gains on HallusionBench and ChartQA.The reported gains correspond to the 7B and 8B models for mathematical benchmarks, while the perception-heavy results are compared with the base model.
  • Iterative Self-Evolution: 5.2%, 4.0%, and 2.8% gains across the first three SERC iterations show steady, monotonic improvement over the base model.The gains are reported for iterations one, two, and three, respectively.
  • Ablation Studies: Removing SERC, tool usage, or self-repair reduces average performance by 8.7%, 6.5%, and 2.5%, respectively.The ablations identify the outer reinforcement-learning loop as the largest contributor, followed by tool integration and self-repair.
  • Performance as a Process Reward Model: As a process reward model, Agent0-VL improves Best-of-8 selection across LVLMs from Qwen2.5-VL-3B to Qwen2.5-VL-32B, yielding an average 7.3% gain.The verifier also improves step-level reward correlation and overall accuracy across seven multimodal reasoning benchmarks.
  • Case Study: A case study shows the Verifier detecting a faulty visual interpretation and the Self-Repair module producing a corrective patch that enables a valid reasoning chain.The Solver initially misinterprets the “blind spot” segment before verification and repair correct the trajectory.

5. Related Work

Prior work spans self-evolving learning for language and vision-language models and tool-integrated reasoning, while Agent0-VL combines these directions through tool-grounded self-evaluation.

  • Self-Evolving Methods: Self-evolving methods reduce reliance on human supervision by using unlabeled data and self-generated feedback, including pseudo-rewards from model outputs.
  • Tool-Integrated Reasoning: Tool-Integrated Reasoning enables models to invoke external tools for knowledge access and complex task execution, and has expanded to multimodal visual interaction.
  • Tool-Integrated Reasoning: As a reward model, Agent0-VL consistently improves accuracy across diverse multimodal reasoning benchmarks when integrated with other models.
  • Self-Evolving Methods: The illustrated training cycle moves from an incorrect answer through verifier error identification and self-repair to solver re-execution of the correct solution.
  • Tool-Integrated Reasoning: Agent0-VL extends tool-augmented reasoning into self-evaluation, producing process-level reward signals that guide self-improvement.

6. Conclusion

Agent0-VL unifies reasoning, verification, and self-repair within one model, using tool-grounded verification and reinforcement learning for continual refinement across visual reasoning benchmarks.

  • Agent0-VL unifies reasoning, verification, and self-repair through Solver and Verifier roles in a Self-Evolving Reasoning Cycle.
  • The framework achieves stronger factual consistency and multi-turn stability while outperforming existing open-source models across visual reasoning benchmarks.

B. Implementation Details

Agent0-VL is built on Qwen vision-language models with shared Solver and Verifier parameters and is trained through sequential supervised fine-tuning and reinforcement learning.

  • Agent0-VL is implemented on Qwen2.5-VL-7B-Instruct and Qwen3-VL-8B, with shared parameters between the Solver and Verifier.
  • Training uses sequential supervised fine-tuning and reinforcement learning, beginning with tool-usage and image-manipulation data before annealing on mathematical code-reasoning data.
  • The supervised fine-tuning learning rate is set to 1 × 10−5 while other hyperparameters remain consistent.
  • System prompts for the framework are provided in Figures 5–7.

D. Training Data Construction Pipeline

The training-data pipeline constructs tool-integrated reasoning trajectories through progressive multimodal curricula and separate Solver and Verifier prompting roles.

  • The pipeline builds a large-scale corpus of high-quality tool-integrated reasoning trajectories before self-evolving reinforcement learning.
  • The Solver is instructed to solve tasks through multi-turn reasoning and selective tool use.
  • The curriculum includes direct reasoning, tool-augmented problems requiring code execution, OCR, or visual analysis, and progressively varied multimodal tasks.
  • Key notation used throughout the paper is summarized in Table 5.
  • The Verifier checks reasoning trajectories step by step, optionally calling tools to verify facts and support correction and reflection.
  • Prompts are derived from multimodal benchmarks covering geometry, visual question answering, scientific analysis, mathematical reasoning, and chart-based reasoning.

D.2. Multi-Stage SFT Data Construction

Agent0-VL constructs multimodal reasoning data through task sampling, sandboxed tool execution, automatic verification, and self-repair examples. The resulting curated trajectories initialize both reasoning and verification behaviors for later self-evolving optimization.

  • D.2. Multi-Stage SFT Data Construction: The pipeline samples multimodal problems and prompts teacher models to generate detailed reasoning traces and explicit tool-use plans.GPT-5 and Qwen2.5-VL-72B bootstrap complete trajectories in JSON format.
  • D.2. Multi-Stage SFT Data Construction: Tool calls run in a sandbox, observations are logged, and an auxiliary verifier filters trajectories inconsistent with tool outputs or final answers.This produces complete multimodal trajectories with execution-grounded validation.
  • D.2. Multi-Stage SFT Data Construction: Structured verifier feedback supplies scores, confidence, critiques, and correction signals for low-confidence reasoning steps.These reasoning–verification–repair triples bridge supervised learning and self-evolving reinforcement learning.
  • D.2. Multi-Stage SFT Data Construction: The unified SFT dataset initializes Agent0-VL’s dual Solver–Verifier behaviors for self-evolving optimization.After filtering and deduplication, it contains approximately 200k high-quality multimodal trajectories.
  • D.2. Multi-Stage SFT Data Construction: Execution validation, semantic consistency checks, redundancy filtering, and manual spot-checks improve trajectory reliability before deduplication.About 10k samples receive manual review for multimodal and reasoning correctness.
  • D.2. Multi-Stage SFT Data Construction: Approximately 240k curated multimodal reasoning trajectories comprise ∼200k SFT examples and ∼40k RL data.The SFT portion provides cold-start reasoning, while the RL portion supports self-evolving optimization.

E. Case Studies

The case studies show Agent0-VL integrating visual interpretation, mathematical reasoning, tool execution, verification, and repair across single-step and multi-stage tasks. Examples include correcting a faulty geometric premise, cropping images, checking computations, and evaluating reasoning steps.

  • E. Case Studies: A self-repair case activates after high-confidence negative feedback identifies the blind spot as Q4, correcting the Solver’s faulty definition.The Verifier treats the contradictory endpoint description as interference or a typo and retains the explicit 270° rotation.
  • E. Case Studies: A full reasoning–evaluation–repair cycle corrects an initially incorrect geometric solution and produces a verified final answer.The Solver generates the error, the Verifier critiques it through tool-grounded verification, and Self-Repair patches the faulty premise before re-execution.
  • E. Case Studies: Agent0-VL resolves a visual street-name task by cropping and zooming a specific image region with an image-manipulation tool.The enhanced output confirms the street name.
  • E. Case Studies: In a geometry example, the model formulates equations with the Pythagorean theorem and calls Python to verify the cone’s volume.Tool-grounded reasoning supports numerical correctness and interpretability.
  • E. Case Studies: A trigonometric midline example combines graphical interpretation, symbolic computation, and code execution to ground reasoning in visual and mathematical evidence.The model uses the given minimum and maximum y-values to reason about the midline.
Loading 2511.19900v2…