PaperlayerView exact revision on arXiv

Source-linked AI summary

OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents

Shuang Chen, Kaituo Feng, Hangting Chen, Wenxuan Huang, Dasen Dai, Quanxin Shou, Yunlong Lin, Xiangyu Yue, Shenghua Gao, Tianyu Pang

arXiv:2605.05185v1cs.CV

TL;DR

Frontier multimodal search agents are difficult to reproduce because high-quality training data, transparent pipelines, and detailed recipes are often unavailable. OpenSearch-VL provides an open training recipe combining curated multi-hop data, diverse tools, and fatal-aware agentic reinforcement learning, achieving 13.8-point average gains across seven benchmarks. The authors will release its data, code, and models to support reproducible research.

  • Problem

    Frontier multimodal search agents remain difficult to reproduce because high-quality training data, code, and training recipes are often insufficiently disclosed.

  • Method

    OpenSearch-VL combines Wikipedia-based multi-hop data curation, a diverse multimodal tool environment, and fatal-aware GRPO training for multimodal deep search agents.

  • Results

    13.8 points average improvement across seven multimodal deep search benchmarks is achieved by OpenSearch-VL-30B-A3B.

  • Takeaways & Limitations

    OpenSearch-VL provides an open foundation for reproducible research on multimodal agentic search.

  • Takeaways & Limitations

    Training stability and exact numerical reproducibility are affected by unreliable, externally hosted tool APIs and costly, version-dependent GPT-4o judging.

Abstract

from arXiv · show

Deep search has become a crucial capability for frontier multimodal agents, enabling models to solve complex questions through active search, evidence verification, and multi-step reasoning. Despite rapid progress, top-tier multimodal search agents remain difficult to reproduce, largely due to the absence of open high-quality training data, transparent trajectory synthesis pipelines, or detailed training recipes. To this end, we introduce OpenSearch-VL, a fully open-source recipe for training frontier multimodal deep search agents with agentic reinforcement learning. First, we curated a dedicated pipeline to construct high-quality training data through Wikipedia path sampling, fuzzy entity rewriting, and source-anchor visual grounding, which jointly reduce shortcuts and one-step retrieval collapse. Based on this pipeline, we curate two training datasets, SearchVL-SFT-36k for SFT and SearchVL-RL-8k for RL. Besides, we design a diverse tool environment that unifies text search, image search, OCR, cropping, sharpening, super-resolution, and perspective correction, enabling agents to combine active perception with external knowledge acquisition. Finally, we propose a multi-turn fatal-aware GRPO training algorithm that handles cascading tool failures by masking post-failure tokens while preserving useful pre-failure reasoning through one-sided advantage clamping. Built on this recipe, OpenSearch-VL delivers substantial performance gains, with over 10-point average improvements across seven benchmarks, and achieves results comparable to proprietary commercial models on several tasks. We will release all data, code, and models to support open research on multimodal deep search agents.

1 Introduction

OpenSearch-VL presents a fully open recipe for training multimodal deep search agents, targeting reproducibility through open data, tools, and agentic reinforcement learning. It addresses bottlenecks in proprietary training resources, long-horizon tool-use failures, and imperfect visual inputs, while improving benchmark performance over a strong agentic baseline.

  • Motivation: The work identifies proprietary training resources as a central reproducibility bottleneck because data sources, filtering criteria, demonstrations, and tool-use trajectories are typically private.This limits systematic study of which data properties are essential for agentic search behavior.
  • Challenges: Long-horizon agentic RL is difficult because malformed calls, timeouts, irrelevant queries, or repeated failures can invalidate later rollout steps while discarding them wastes useful pre-failure reasoning.Training on complete failed rollouts instead introduces noisy gradients from meaningless post-failure tokens.
  • Data curation: OpenSearch-VL builds high-quality image-grounded multi-hop data by sampling entity paths from the Wikipedia hyperlink graph and rewriting intermediate entities into fuzzy descriptions.The pipeline converts sampled multi-hop paths into multi-hop VQA instances before applying further curation steps.
  • Results: OpenSearch-VL improves the Qwen3-VL-30B-A3B agentic baseline’s average score from 47.8 to 61.6, including gains of +13.3 on VDR, +24.5 on MMSearch, +10.2 on FVQA, and +16.2 on InfoSeek.These experiments are conducted across multimodal deep search benchmarks.
  • Contribution: The recipe targets reproducibility by releasing training data, code, and models for multimodal agentic search research.The work is framed as a fully open recipe for training frontier multimodal deep search agents.

2 Preliminaries

OpenSearch-VL formulates multimodal search as ReAct-style interleaving of reasoning, tool calls, and environment observations over visual and retrieval tools. Its preliminaries define multimodal history accumulation, trajectory likelihood and generation masking, and a three-function search-tool suite.

  • Problem Formulation: The agent answers a question by interleaving reasoning with visual or retrieval tool calls over accumulated images, actions, and observations.Each action contains a reasoning trace and either a tool invocation or, at the final step, the response, following the ReAct (Yao et al., 2022) convention.
  • Multimodal Observations and Active Visual Context: Visual context grows monotonically by preserving the initial image and all prior image observations, enabling cross-reference across multi-hop transformations.The environment deterministically routes commands by tool family and returns multimodal observations comprising images and text.
  • Trajectory Likelihood: Trajectory likelihood assigns probability to policy-generated actions while treating environment observations as exogenous inputs that affect later histories.This factorization is supervised by SFT and supplies the per-token importance ratio for the RL objective.
  • Token-level Generation Mask: Generation masking restricts optimization to policy-emitted reasoning and tool-call tokens, excluding textual observation spans and image-valued observations from token-level loss.Image observations from visual tools other than OCR enter the visual backbone directly.
  • Search Tools: The tool suite combines retrieval, image enhancement, and attention or parsing through TextSearch, ImageSearch, Sharpen, SuperResolution, PerspectiveCorrect, Crop, and OCR.These functions respectively acquire external evidence, improve low-quality inputs, and localize or decode fine-grained content.

3 Dataset Curation

OpenSearch-VL builds a scalable, annotation-free curation pipeline that constructs tool-demanding multimodal questions, filters and enhances them, and synthesizes expert trajectories for training. Its design suppresses single-shot retrieval shortcuts by combining multi-hop Wikipedia paths, fuzzy entity rewriting, and source-anchor visual grounding.

  • Pipeline Overview: The pipeline scales annotation-free data creation through VQA construction, staged filtering and enhancement, and expert trajectory synthesis.These three stages produce the final dataset used for subsequent training.
  • Fuzzy Rewriting and Visual Grounding: Fuzzy entity rewriting and answer obfuscation prevent entity-name shortcuts, while source-anchor visual grounding replaces the path’s initial entity with a visual referring expression.The anchor image is retrieved from Wikimedia Commons or a Wikipedia infobox and filtered using CLIP similarity, separating visual evidence from the answer entity.
  • Wikipedia Path Sampling: Constrained multi-hop walks over Wikipedia assign anchor and bridge roles, producing reasoning chains that avoid cycles, hubs, disambiguation pages, and list pages.Paths have lengths h ∈ {2, 3, 4}, with the source node serving as the visual entry point and later nodes supporting intermediate reasoning.
  • Filtering and Enhancement: Wikipedia-derived instances are merged with LiveVQA, FVQA, and WebQA, then filtered to retain tool-demanding samples and degraded-image cases requiring visual enhancement.A frozen Qwen3-VL-32B removes examples solvable without tools, while 10% of the filtered pool receives blur, downsampling, or perspective distortion paired with enhancement tools.
  • Trajectory Synthesis: 36,592 high-quality expert trajectories averaging 6.3 tool-invocation turns form the SFT corpus after two-stage rejection filtering.Claude Opus 4.6 generated five independent multi-turn ReAct rollouts per surviving instance against the real execution environment.

4 Training

OpenSearch-VL trains in two stages: SFT establishes foundational reasoning and tool-use behavior, while search-augmented RL enables exploration beyond demonstration coverage. Its RL recipe combines multimodal rollouts, composite rewards, and fatal-aware masking with one-sided advantage clamping to preserve useful reasoning before cascading failures.

  • Training overview: Two-stage training uses SFT for fundamental reasoning and tool-use behaviors, followed by search-augmented RL to discover more effective exploration strategies.SFT trains on 36,592 curated multi-turn expert trajectories, with tool observations used as context while reasoning and subsequent tool invocations or terminal responses are supervised.
  • Reinforcement learning: RL addresses SFT’s limited demonstration coverage by optimizing multi-turn exploration in a multimodal environment E, building on GRPO and search-augmented GRPO.For each prompt, the method samples groups of multi-turn rollouts from the current policy interacting with E.
  • Composite Multi-Turn Reward: The composite trajectory reward balances formatting, terminal accuracy, and process-level search quality to combine structural integrity, end-task correctness, and denser feedback.The format reward gates structurally valid trajectories, accuracy is judged by GPT-4o against the ground truth, and query quality is scored continuously by GPT-5.4 to provide feedback when terminal accuracy is zero.
  • Fatal-Aware Token Masking: Fatal-aware masking zeros tokens generated after the earliest cascade of three consecutive tool-execution errors, preserving the viable prefix instead of discarding or blindly training on the full trajectory.Process rewards are computed only over the valid prefix, excluding post-failure structural collapse from penalization.
  • Fatal-Aware Token Masking: One-sided advantage clamping assigns fatal trajectories max(e_ri, 0), so viable prefixes receive reinforcement only when their partial reward exceeds the group mean and otherwise receive zero gradient.All trajectories still contribute to group-normalized reward statistics, while clamping avoids penalizing valid reasoning before failure and recovers more useful signal than hard masking.

5 Experiments

OpenSearch-VL is evaluated across seven multimodal knowledge-intensive benchmarks and consistently outperforms direct-reasoning and RAG baselines. Ablations show that both the tool-demanding data pipeline and fatal-aware GRPO, especially one-sided advantage clamping, materially improve performance.

  • Experimental setup: The evaluation uses Pass@1 on SimpleVQA, VDR, MMSearch, LiveVQA, BrowseComp-VL, FVQA, and InfoSeek, with correctness judged by GPT-4o.Models include Qwen3-VL-8B-Instruct, Qwen3-VL-30B-A3B-Instruct, and Qwen3-VL-32B-Instruct.
  • Main results: 56.6 average score makes OpenSearch-VL-8B the strongest 8B agent, surpassing SenseNova-MARS-8B by 3.9 points and outperforming direct-reasoning and RAG baselines across scales.Table 2 covers seven multimodal knowledge-intensive QA and web-search benchmarks.
  • Training recipe ablation: 71.8 average accuracy from Fatal Masking + One-sided Clamp exceeds Vanilla GRPO’s 67.6 and Fatal Masking only’s 69.1 after identical Qwen3-VL-8B SFT initialization.SFT raises the base model from 53.7 to 64.6, while vanilla GRPO further reaches 67.6; the full recipe gives a +4.2-point delta relative to Vanilla GRPO.
  • Data pipeline ablation: 11.5, 10.3, and 8.2-point drops follow removal of source-anchor grounding, fuzzy entity rewriting, and staged filtering, respectively, from the data pipeline.The full pipeline achieves a 64.6 average score, indicating that shortcut prevention and genuinely tool-demanding queries are both important.
  • Training recipe ablation: One-sided clamping preserves positive updates for fatal prefixes that beat their group mean while zeroing noisy negative signals from dominated failures.Across 10,000 groups, most fatal trajectories are zeroed and the preserved tail is distributionally close to the right mode of non-fatal rollouts.

6 Related Work

Prior work has advanced agentic search from text-only RL systems to multimodal pipelines that integrate visual retrieval and dynamic tool orchestration. However, perception fidelity and cascading failures remain key challenges for multimodal agents operating in long-horizon, multi-tool environments.

  • Related Work: Active-search research has shifted LLMs from static knowledge bases toward agentic reasoners using dynamic, multi-turn retrieval, including multimodal systems with visual search.Search-R1 established RL-based autonomous querying, while MMSearch-R1 and Vision-DeepResearch extended the paradigm to multimodal agent pipelines.
  • Related Work: Multimodal-agent performance is often limited by initial perception fidelity, while existing RAG and tool-augmented systems provide limited active control over retrieved visual information.VisRAG preserves visual structure but treats the model as a passive observer, and existing toolsets remain largely homogeneous.
  • Related Work: Applying GRPO to long-horizon agentic rollouts with diverse tool interactions remains difficult because early tool errors can trigger cascading trajectory failures.Although GRPO effectively aligns language-model reasoning trajectories, its use in multi-tool environments introduces failure-handling challenges.

7 Conclusion

OpenSearch-VL presents a fully open recipe for training multimodal deep search agents with agentic reinforcement learning. The recipe combines Wikipedia-based data curation, a diverse retrieval and visual-processing tool environment, and a multi-turn fatal-aware GRPO algorithm.

  • OpenSearch-VL is a fully open recipe for training multimodal deep search agents with agentic reinforcement learning.
  • Its Wikipedia-based data curation pipeline mitigates one-step retrieval shortcuts and produces SearchVL-SFT-36k and SearchVL-RL-8k.
  • Its diverse tool environment spans retrieval, image enhancement, and attention-and-parsing tools, while fatal-aware GRPO preserves useful pre-failure reasoning.

Limitations and Future Work … B Multi-Turn Search Fatal-Aware GRPO Details

The paper identifies external-tool unreliability and proprietary reward judging as important limitations, while the appendices document standard and search-engine GRPO preliminaries plus fatal-aware multi-turn training details.

  • Limitations and Future Work: External-tool failures, ranking drift, and summarization hallucinations increase reward variance, motivating future work on on-policy reliability estimation.The affected tools include TextSearch and ImageSearch.
  • Limitations and Future Work: The composite reward depends on costly, version-dependent proprietary GPT-4o judges that currently score only textual queries and ignore intermediate visual operations.
  • A Preliminary of Reinforcement Learning: The appendix includes a section on standard GRPO as a preliminary reinforcement-learning method.
  • A Preliminary of Reinforcement Learning: The appendix separately covers GRPO with a search engine as another reinforcement-learning preliminary.
  • B Multi-Turn Search Fatal-Aware GRPO Details: The appendix specifies fatal-step detection logic for the multi-turn search fatal-aware GRPO procedure.
  • B Multi-Turn Search Fatal-Aware GRPO Details: The appendix derives advantages using one-sided clamping for the multi-turn search fatal-aware GRPO procedure.

C Implementation Details

This section presents the implementation details for supervised fine-tuning (SFT) and reinforcement-learning (RL) training configurations.

  • C.1 SFT Training Configuration: The SFT training configuration is described in Section C.1.
  • C.2 RL Training Configuration: The RL training configuration is described in Section C.2.

D Data Curation Details … B.1 Fatal Step Detection Logic

The appendix documents data-curation details and develops the reinforcement-learning foundations and fatal-aware logic for multi-turn, search-augmented training. It specifies how tool failures are detected and handled while reviewing standard and search-augmented GRPO.

  • D Data Curation Details: The data-curation details include path-sampling hyperparameters, an Australia_Zoo running example, and counterfactual design choices.These topics are listed as subsections D.1–D.3 of the curation appendix.
  • A.1 Standard GRPO: GRPO (Shao et al., 2024), an actor-critic variant of PPO (Schulman et al., 2017), estimates advantages from within-group response rewards without a learned value function.It explicitly applies KL regularization in the loss rather than incorporating it into the reward signal.
  • A Preliminary of Reinforcement Learning: The appendix’s preliminary section reviews standard GRPO and its search-augmented extension as foundations for the multi-turn training objective.These formulations establish the background for interleaving policy generation with external search.
  • A.2 GRPO with Search Engine: Search-R1 extends GRPO to interleaved rollouts in which policy generation alternates with externally retrieved evidence.Its objective masks environmental tokens, normalizes over generated tokens, and applies KL regularization only at policy-authored positions.
  • A.2 GRPO with Search Engine: Figure 5 shows that 91.8% of 47,978 fatal rollouts are zeroed by clamping, while 8.2% are preserved.The fatal rollouts span 10,000 groups with G=16; mean pre-clamp scores are −0.68 for zeroed rollouts and +0.57 for preserved rollouts.
  • B Multi-Turn Search Fatal-Aware GRPO Details: The multi-turn RL appendix defines fatal execution cascades, derives fatal-aware advantage estimation with one-sided clamping, and refers reward components to Sec. 4.2.The objective incorporates a fatal-aware token mask, while rfmt, racc, and rquery are defined directly in Sec. 4.2.
  • B.1 Fatal Step Detection Logic: Fatal detection tracks consecutive tool-execution errors, sets K = 3, and assigns non-fatal trajectories fi = Li + 1 when the threshold is never reached.The counter resets when a step does not trigger a tool-execution error, preventing isolated transient failures from prematurely masking later reasoning.

B.2 Advantage Derivation with One-Sided Clamping · C Implementation Details · C.1 SFT Training Configuration

OpenSearch-VL derives fatal-aware one-sided advantages that preserve useful viable-prefix learning while excluding post-fatal and invalid-token gradients, and implements training with multimodal tool-interleaved SFT infrastructure. Its clamping introduces a non-negative bias, but 91.8% of fatal rollouts are clamped because their normalized returns are negative.

  • B.2 Advantage Derivation with One-Sided Clamping: All G trajectories, including fatal trajectories, contribute to group mean and standard-deviation estimates, anchoring relative performance across the sampled cohort.The normalized statistics are treated as stop-gradient constants, so gradients flow only through importance ratios on unmasked policy-generated tokens.
  • B.2 Advantage Derivation with One-Sided Clamping: Fatal-aware clamping never propagates gradients through post-fatal suffixes, avoids penalizing viable prefixes, and extracts positive reinforcement when fatal trajectories exceed the group baseline.This is weakly informative-dominant over hard-masking, which discards every fatal trajectory; positive fatal trajectories retain their viable-prefix gradient.
  • B.2 Advantage Derivation with One-Sided Clamping: The method safely ignores invalid credit assignment while selectively harvesting positive reinforcement from prematurely truncated but high-quality exploratory rollouts.Its asymmetric clamp maps negative fatal-trajectory normalized returns to zero, creating a non-negative bias relative to the zero-mean GRPO baseline.
  • B.2 Advantage Derivation with One-Sided Clamping: 91.8% of fatal rollouts have negative standardized returns and are clamped to zero, while the remaining 8.2% occupy the score regime of competitive non-fatal rollouts.The resulting non-negative bias trades off against retaining non-zero gradients on viable prefixes.
  • C Implementation Details: The implementation extends LlamaFactory (Zheng et al., 2024) with multi-turn, tool-interleaved collators and Qwen3-VL-aware vision/text packing for verbatim visual-tool observations.Supported tools include Crop, Sharpen, SuperResolution, PerspectiveCorrect, and OCR.
  • C Implementation Details: The RL pipeline builds jointly on rLLM and Vision-DeepResearch (Huang et al., 2026), adopting asynchronous agent training infrastructure from the former.The supplied passage identifies these two systems as the foundations of the RL pipeline but truncates the remaining implementation description.
  • C.1 SFT Training Configuration: Table 4 consolidates SFT hyperparameters across the 8B dense, 32B dense, and 30B-A3B mixture-of-experts variants, which differ only in base checkpoint path.All variants use full-parameter fine-tuning, including the vision tower and multimodal projector, with DeepSpeed ZeRO-3 and Ray orchestration over 256.

C.2 RL Training Configuration

The multi-turn fatal-aware GRPO stage uses a shared rollout, actor, and algorithmic framework across the 8B dense and 30B-A3B MoE OpenSearch-VL variants, while selected training settings differ by model type.

  • RL training configuration: Both RL-finetuned variants use async SGLang rollouts, a Megatron-parallel actor, RLOO leave-one-out advantages under GRPO, a low-variance KL controller, and no critic.The shared algorithm template is applied to Qwen3-VL-8B-Instruct and Qwen3-VL-30B-A3B-Instruct.
  • RL training configuration: Table 5 varies response budgets, mini-batch sizes, parallelism, MoE routing coefficients, save cadence, and total epochs between the dense and MoE runs.Expert parallelism is not applicable to the dense 8B model.

D Data Curation Details … G Case Study

OpenSearch-VL specifies a tightly filtered Wikipedia path-sampling pipeline, a tool-first multimodal agent interface, and reward-guided search behavior that jointly enforce non-leaky, evidence-grounded reasoning. The running examples show that visual grounding, fuzzy rewriting, hub avoidance, and chained retrieval are necessary to prevent shortcut solutions and verify answers.

  • D.1 Path Sampling Hyperparameters: The pipeline filters disambiguation, list-like, non-article, and redirect targets, retries failed walks up to 10 times, and drops seeds that never satisfy descriptor, uniqueness, or answer-length constraints.Walks longer than four hops rarely survive the uniqueness and non-leakage checks without heavy resampling.
  • D Data Curation Details; D.1 Path Sampling Hyperparameters: The sampler balances five visually groundable domains while restricting seeds to infobox-bearing, image-linked, non-dead-end entities and rejecting roughly the top 0.03% of hub nodes.Seeds require a Wikimedia Commons image of at least 512 × 512 resolution and in-degree in [50, τhub], with τhub = 10,000; paths use h ∼ Categorical({2, 3, 4}; (0.4, 0.4, 0.2)).
  • D.3 Counterfactual Design Choices: Counterfactuals show that replacing the anchor with the answer image, omitting fuzzy rewriting, or admitting hubs collapses the multi-hop task or leaks the answer through one-step search.With the answer image, reverse-image lookup directly identifies Terri Irwin; without fuzzing, TextSearch("Terri Irwin Australian citizenship date") recovers the answer; hub Queensland causes uniqueness failure or identity leakage.
  • E System Prompts: The system prompt enforces “Verify, Don’t Guess” through tool-first image processing, chained operations, and external text validation, while separate GPT-4o judges score accuracy and query quality during RL and benchmark evaluation uses an aligned post-hoc judge.The condensed inference/SFT prompt preserves the original behavioral rules while exposing machine-readable OpenAI-style tool schemas.
  • F Tool Definition and Usage; Retrieval Tools: The tool environment combines web retrieval with visual grounding: TextSearch searches, reads, and summarizes documents, while ImageSearch converts unknown visual entities into retrievable textual identities for subsequent cross-checking.TextSearch uses Serper, JINA Reader, and Qwen3-32B; ImageSearch uses the Polaris Lens API and returns similar images, entities, URLs, and captions.
  • Image Enhancement Tools; Attention and Parsing Tools: Image enhancement and attention tools prepare evidence for perception by sharpening blur, upscaling low-resolution inputs, rectifying perspective, isolating regions, and extracting layout-aware text with OCR.Sharpening uses OpenCV Unsharp Masking, SuperResolution uses EDSR with default ×4 scaling, PerspectiveCorrect performs quadrilateral warping, Crop suppresses peripheral noise, and OCR returns reading-order text blocks.
  • G Case Study: The bridge case study demonstrates progressive cross-modal verification: the agent crops a road sign, uses image search to identify Kessock Bridge, text-searches its opening date, confirms 1982, and stops once evidence converges.This trajectory exemplifies the joint role of the visual–retrieval action space, query-quality reward, and fatal-aware masking in incentivizing ordered tool chaining and early termination.
Loading 2605.05185v1…