PaperlayerView exact revision on arXiv

Source-linked AI summary

OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents

Shuang Chen, Kaituo Feng, Hangting Chen, Wenxuan Huang, Dasen Dai, Quanxin Shou, Yunlong Lin, Xiangyu Yue, Shenghua Gao, Tianyu Pang

arXiv:2605.05185v1cs.CV

TL;DR

Frontier multimodal search agents remain difficult to reproduce because training data and code are often proprietary or insufficiently disclosed. OpenSearch-VL combines data curation, retrieval and visual tools, and fatal-aware reinforcement learning in an open training recipe. Its 30B-A3B model achieves a 13.8-point average improvement across seven multimodal deep search benchmarks.

  • Problem

    Frontier multimodal search agents remain difficult to reproduce because their training data and code are often proprietary or insufficiently disclosed.

    Source: §1 · p. 1

  • Method

    OpenSearch-VL combines Wikipedia-based data curation, diverse retrieval and visual tools, and fatal-aware GRPO to train multimodal deep search agents with reinforcement learning.

    Source: §7 · p. 12

  • Results

    13.8 points of average improvement across seven multimodal deep search benchmarks are reported for OpenSearch-VL-30B-A3B, alongside performance comparable to proprietary commercial models on several benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Planned releases of training data, code, and models aim to provide an open foundation for reproducible research on multimodal agentic search.

    Source: §1 · p. 2

    The reward relies on costly, version-dependent proprietary GPT-4o judges that evaluate textual queries but ignore intermediate visual operations.

    Source: Limitations and Future Work · p. 12

Abstract

from arXiv · show

Deep search has become a crucial capability for frontier multimodal agents, enabling models to solve complex questions through active search, evidence verification, and multi-step reasoning. Despite rapid progress, top-tier multimodal search agents remain difficult to reproduce, largely due to the absence of open high-quality training data, transparent trajectory synthesis pipelines, or detailed training recipes. To this end, we introduce OpenSearch-VL, a fully open-source recipe for training frontier multimodal deep search agents with agentic reinforcement learning. First, we curated a dedicated pipeline to construct high-quality training data through Wikipedia path sampling, fuzzy entity rewriting, and source-anchor visual grounding, which jointly reduce shortcuts and one-step retrieval collapse. Based on this pipeline, we curate two training datasets, SearchVL-SFT-36k for SFT and SearchVL-RL-8k for RL. Besides, we design a diverse tool environment that unifies text search, image search, OCR, cropping, sharpening, super-resolution, and perspective correction, enabling agents to combine active perception with external knowledge acquisition. Finally, we propose a multi-turn fatal-aware GRPO training algorithm that handles cascading tool failures by masking post-failure tokens while preserving useful pre-failure reasoning through one-sided advantage clamping. Built on this recipe, OpenSearch-VL delivers substantial performance gains, with over 10-point average improvements across seven benchmarks, and achieves results comparable to proprietary commercial models on several tasks. We will release all data, code, and models to support open research on multimodal deep search agents.

1 Introduction

OpenSearch-VL presents a fully open recipe combining image-grounded multi-hop training data, visual-processing tools, and failure-aware agentic reinforcement learning. Its trained model improves over strong baselines and matches or exceeds proprietary commercial models on several benchmarks.

1 Introduction

  • The data pipeline samples Wikipedia multi-hop entity paths, rewrites intermediate entities into fuzzy descriptions, and filters resulting VQA instances to discourage single-hop image-lookup shortcuts.It produces SearchVL-SFT-36k for supervised fine-tuning and SearchVL-RL-8k for agentic reinforcement learning.Source: §1 · p. 2
  • The tool environment combines search with OCR, cropping, sharpening, super-resolution, and perspective correction to process imperfect visual evidence before retrieving external knowledge.These tools address blurred photos, low-resolution thumbnails, skewed documents, and crowded screenshots, where searching alone is insufficient.Source: §1 · p. 2
  • Fatal-aware GRPO masks invalid post-failure tokens and uses one-sided advantage clamping to preserve useful pre-failure reasoning.Malformed calls, timeouts, irrelevant queries, or repeated failures can invalidate the remaining rollout. Discarding entire trajectories wastes useful reasoning, while optimizing full failed rollouts introduces noisy gradients.Source: §1 · p. 2
  • 13.8 points across seven benchmarks is OpenSearch-VL-30B-A3B’s average gain over the Qwen3-VL-30B-A3B (Bai et al., 2025) agentic baseline, raising the average score from 47.8 to 61.6.Gains include VDR (+13.3) (Zeng et al., 2026), MMSearch (+24.5) (Jiang et al.), FVQA (+10.2) (Wang et al., 2017), and InfoSeek (+16.2) (Chen et al., 2023). OpenSearch-VL also matches or exceeds proprietary commercial models on several benchmarks.Source: §1 · p. 2
  • OpenSearch-VL addresses limited reproducibility with a fully open recipe and plans to release its training data, code, and models.High-quality multimodal search training data must support image-grounded understanding, multi-hop retrieval, evidence verification, and long-horizon tool use rather than simple visual question answering.Source: §1 · pp. 1–2

2 Preliminaries

OpenSearch-VL formulates image-question answering as multi-turn reasoning interleaved with visual and retrieval tools, preserving visual observations for later use. Its likelihood and training masks distinguish policy-generated actions from environment-provided observations.

Problem Formulation

  • Agents answer image-grounded questions through multi-turn reasoning and tool calls, combining visual processing with external knowledge retrieval.Following ReAct (Yao et al., 2022), each action pairs reasoning with a tool invocation, except the final action, which returns an answer without another observation.Source: §2 · p. 2

Multimodal Observations and Active Visual Context

  • The environment deterministically routes tool invocations and returns image or text observations, preserving all historical images for cross-referencing successive visual transformations.The active visual context grows monotonically, while rollouts strictly interleave policy actions and environment observations.Source: §2 · p. 3

Trajectory Likelihood

  • Autoregressive trajectory likelihood excludes environment observations from generative probability, although observations condition subsequent histories; this factorisation supports SFT and RL per-token importance ratios.Observations are excluded because they are exogenous environment outputs rather than policy-generated content.Source: §2 · p. 3

Token-level Generation Mask

  • A token-level generation mask restricts optimisation gradients to policy-generated reasoning and commands, excluding textual observations while image observations inherently bypass token-level loss.Inspired by retrieved-token masking in Search-R1, this protocol supports SFT and fatal-aware RL. Including noisy search-result and OCR observations in the loss destabilises training.Source: §2 · p. 3

Search Tools

  • The tool suite covers external retrieval, image enhancement, and attention and parsing, combining lightweight offline primitives with online expert-model services.Retrieval uses TextSearch and ImageSearch; enhancement uses Sharpen, SuperResolution, and PerspectiveCorrect; attention and parsing use Crop and OCR.Source: §2 · p. 3, Table 1 · p. 3

3 Dataset Curation

OpenSearch-VL’s data pipeline combines multi-hop image-grounded question construction, difficulty filtering with visual enhancement, and expert trajectory synthesis. Answer-correctness and process-quality checks yield 36,592 trajectories for supervised fine-tuning.

3.1 High-Quality VQA Construction

  • Wikipedia paths assign distinct roles to visual anchors, intermediate bridges, and answer nodes, structuring multi-hop questions while separating visual identification from the target answer.Constrained walks span two, three, or four hops and exclude disambiguation/list pages, cycles, and high-in-degree hubs. GPT-4o (Team, 2024) converts paths into canonical questions used for rewriting rather than as training targets.Source: §3.1 · pp. 4–5
  • Answer-preserving fuzzy rewriting replaces entity names with relational or attribute-based descriptors, checking uniqueness after each substitution to prevent single-retrieval shortcuts.Rewriting proceeds from the farthest bridge toward the visual anchor. Occasional answer obfuscation avoids collapse onto stereotyped relational templates.Source: §3.1 · p. 5
  • Source-anchor visual grounding replaces the initial entity with an image reference, requiring visual identification before following intermediate textual relations to the answer.Images come from Wikimedia Commons or Wikipedia infoboxes and undergo CLIP-based selection. Candidate image–question–answer triples must pass automatic masking, uniqueness, and visual-relevance checks.Source: §3.1 · p. 5

3.2 Filtering and Enhancement

  • Qwen3-VL-32B (Bai et al., 2025) filters tool-free and single-ImageSearch solutions, while image degradation paired with restoration tools teaches image repair before retrieval.Wikipedia instances are combined with LiveVQA (Fu et al., 2025), FVQA (Wang et al., 2017), and WebQA (Chang et al., 2022), covering live entities, commonsense fact lookup, and open-web multi-hop reasoning. A randomly selected 10% receives blur, downsampling, or perspective distortion, paired respectively with Sharpen, SuperResolution, and PerspectiveCorrect.Source: §3.2 · p. 5

3.3 Multi-turn Trajectory Synthesis

  • 36,592 expert trajectories, averaging 6.3 tool-invocation turns each, form the SFT corpus after answer-correctness and process-quality rejection.Claude Opus 4.6 (Team, 2026a) generates five independent multi-turn ReAct (Yao et al., 2022) rollouts per instance in the real tool environment. GPT-4o (Team, 2024) judges final-answer agreement, then GPT-5.4 checks tool use, reasoning–observation consistency, and ineffective repetition.Source: §3.3 · p. 6

4 Training

OpenSearch-VL uses supervised fine-tuning followed by multimodal, search-augmented reinforcement learning. Its fatal-aware GRPO combines task and search-quality rewards with token masking and one-sided advantage clamping to preserve useful pre-failure supervision.

4 Training

  • OpenSearch-VL first learns reasoning and tool use through supervised fine-tuning, then uses reinforcement learning to discover more effective exploration strategies.RL builds on GRPO (Shao et al., 2024) and its search-augmented extension, replacing text-only retrieval with a diverse multimodal tool environment.Source: §4 · p. 6, §4.2 · p. 7

4.1 Supervised Fine-Tuning

  • 36592 multi-turn expert trajectories supervise both reasoning traces and subsequent tool invocations or terminal responses during supervised fine-tuning.Tool observations serve only as conditioning context and are excluded from the loss using the retrieved-token masking strategy of.Source: §4.1 · pp. 6–7

4.2 Multi-Turn Search Fatal-Aware GRPO

  • One-sided advantage clamping prevents below-average fatal trajectories from penalizing their valid prefixes while retaining positive reinforcement for above-average prefixes.All trajectories, including fatal ones, remain in group-normalized reward statistics; fatal trajectories receive their normalized advantage clamped at zero.Source: §4.2 · pp. 7–8
  • A composite trajectory reward balances structural formatting, terminal accuracy, and process-level search quality.The reward uses α = 0.8, with formatting acting as a multiplicative gate that pushes structurally degraded returns toward zero. Valid steps require a contiguous thinking block immediately followed by a tool call or terminal response; structural violations include tool-execution errors.Source: §4.2 · p. 7
  • Terminal accuracy uses GPT-4o (Team, 2024) under an LLM-as-Judge protocol (Gu et al., 2025) to check semantic equivalence with ground truth.A matching answer receives 1 and a nonmatching answer receives 0. Fatal trajectories without a terminal response deterministically receive 0.Source: §4.2 · p. 7
  • Query-quality rewards provide dense feedback for unsuccessful trajectories, evaluating only the valid prefix when a trajectory becomes fatal.GPT-5.4 scores cumulative search queries on [0, 1]. Its rubric covers relevance, iterative refinement, retrieved-payload signal-to-noise ratio, and complementary image/text retrieval.Source: §4.2 · p. 7
  • Fatal-aware token masking preserves valid early reasoning while excluding tokens from the fatal step onward.The fatal step begins the earliest cascade of three consecutive tool-execution errors. Formatting and query-quality rewards are computed only over the valid pre-fatal prefix.Source: §4.2 · p. 8

5 Experiments

OpenSearch-VL outperforms direct-reasoning and RAG baselines across model scales on seven multimodal benchmarks. Ablations examine trajectory synthesis and fatal-aware RL, whose one-sided clamping preserves useful pre-failure reasoning while preventing noisy negative updates.

5 Experiments

  • Seven benchmarks assess visual entity recognition, web evidence retrieval, multi-hop reasoning, and long-tail QA using Pass@1, GPT-4o judging, and direct-reasoning, RAG, and agentic baselines.Evaluation follows VDR-Bench (Zeng et al., 2026). Training uses 36K multi-turn SFT trajectories and 8K RL examples disjoint from the VQA instances used to synthesize SFT trajectories.Source: §5 · p. 8

5.1 Main Results

  • 56.6 average score puts OpenSearch-VL-8B 3.9 points above SenseNova-MARS-8B; OpenSearch-VL-30B-A3B and OpenSearch-VL-32B reach 61.6 and 63.7, respectively.OpenSearch-VL surpasses direct-reasoning and RAG baselines across all scales. OpenSearch-VL-32B also outperforms Gemini-2.5-Pro in direct-reasoning mode and substantially exceeds the corresponding Qwen3-VL agentic baselines.Source: Table 2 · p. 9, §5.1 · p. 9

5.2 Ablation Study

  • Fatal-aware GRPO sustains more rollout turns and higher batch-level accuracy than vanilla GRPO (Guo et al., 2025; Jin et al., 2025) and Hard-Mask (Huang et al., 2026).The Qwen3-VL-8B ablation study examines both tool-demanding trajectory synthesis and fatal-aware RL beyond offline imitation.Source: §5.2 · p. 9, Figure 3 · p. 10
  • Fatal-aware clamping reinforces above-group-mean prefixes on harder prompts, masks post-failure tokens, and zeroes fatal-trajectory advantages when successful rollouts dominate easier prompts.The two illustrated groups each contain 16 rollouts. Zeroing negative advantages avoids suppressing potentially valid reasoning prefixes.Source: §5.2 · pp. 10–11, Figure 4 · p. 11
  • 10,000 rollout groups show that most fatal trajectories receive zeroed updates, while the small preserved tail resembles the non-fatal reference distribution’s right mode.One-sided clamping recovers a credit signal from failed tool-call trajectories without amplifying their noise.Source: §5.2 · p. 11

6 Related Work

Multimodal search agents face limitations from degraded visual evidence and cascading tool failures during reinforcement learning. OpenSearch-VL addresses these limitations through active perception and a fatal-aware training objective that protects useful pre-failure reasoning.

6.1 Multimodal Agentic Search

  • Multimodal agentic search extends autonomous querying to visual retrieval, but search alone cannot repair degraded or text-dense visual evidence.Search-R1 uses RL for autonomous multi-turn querying; MMSearch-R1 (Wu et al., 2025b) and Vision-DeepResearch (Huang et al., 2026; Narayan et al., 2025b) incorporate visual retrieval. Despite efforts to combine image manipulation with web search, these approaches commonly assume pristine visual inputs.Source: §6.1 · p. 11

6.2 Visual Perception and Retrieval

  • Active perception lets agents repair visual noise with tools such as super-resolution and specialized OCR before reasoning, rather than relying only on retrieval.VisRAG (Yu et al., 2024) preserves visual structure but treats the model as a passive observer. Existing tool-augmented agents offer dynamic orchestration, yet their toolsets remain largely homogeneous and scenario-bound.Source: §6.2 · p. 11

6.3 Reinforcement Learning for Agentic Reasoning

  • The fatal-aware RL objective removes post-failure learning signals and uses one-sided advantage clamping to protect constructive reasoning steps.Early tool errors can make subsequent trajectory tokens incoherent and introduce policy-gradient noise. Standard group normalization can also penalize valid reasoning prefixes in partially successful rollouts (Huang et al., 2026).Source: §6.3 · p. 12

7 Conclusion

OpenSearch-VL presents a fully open recipe for training multimodal deep search agents with agentic reinforcement learning, combining curated datasets, diverse tools, and fatal-aware training. It achieves over 10-point average gains across seven multimodal deep search benchmarks.

7 Conclusion

  • Over 10-point average gains across seven multimodal deep search benchmarks are reported for OpenSearch-VL.Performance is competitive with strong proprietary reasoning models on representative tasks such as VDR.Source: §7 · p. 12
  • A Wikipedia-based data curation pipeline mitigates one-step retrieval shortcuts and produces SearchVL-SFT-36k and SearchVL-RL-8k.Source: §7 · p. 12
  • Multi-turn fatal-aware GRPO preserves useful pre-failure reasoning through one-sided advantage clamping.Source: §7 · p. 12
  • The tool environment spans retrieval, image enhancement, and attention-and-parsing tools.Source: §7 · p. 12
  • The authors plan to release data, code, models, and the training recipe to lower reproducibility barriers and support future multimodal deep search research.Source: §7 · p. 12

Limitations and Future Work … B.1 Fatal Step Detection Logic

OpenSearch-VL’s fatal-aware RL combines consecutive-error detection with one-sided advantage clamping, building on GRPO and search-augmented optimization. Tool instability, proprietary reward judges, and hosted APIs constrain training reliability and reproducibility.

Limitations and Future Work

  • External-tool instability, proprietary GPT-4o rewards, and hosted APIs limit training stability, visual-action evaluation, and exact reproducibility.Ranking drift, fetch failures, and search-summary hallucinations inflate reward variance; future work targets on-policy reliability estimation and open process reward models covering visual actions omitted by costly, version-dependent, text-only GPT-4o judges. Hosted APIs and prohibitive multi-seed evaluation costs impede reproducibility; planned permissive releases include SearchVL-SFT-36k, SearchVL-RL-8k, checkpoints, and training code.Source: Limitations and Future Work · p. 12

Appendix Contents

  • The appendix roadmap covers GRPO preliminaries, fatal-step detection, one-sided clamping derivations, and SFT and RL training configurations.Further data-construction entries cover path-sampling hyperparameters, an Australia_Zoo running example, and counterfactual design choices.Source: Appendix Contents · p. 17

A Preliminary of Reinforcement Learning

  • Standard GRPO (Shao et al., 2024) and search-augmented Search-R1 provide the foundations for OpenSearch-VL’s multi-turn, multi-tool objective.The preliminaries progress from single-turn generation to rollouts that alternate policy generation with external search calls.Source: A Preliminary of Reinforcement Learning · p. 17

A.1 Standard GRPO

  • GRPO (Shao et al., 2024) replaces a learned critic with within-prompt group reward comparisons, assigning uniform advantages to each response’s tokens.Optimization uses a clipped surrogate objective with an explicit KL penalty against a fixed reference policy, reducing the need for a separate value model.Source: §A.1 · pp. 17–18

A.2 GRPO with Search Engine

  • Search-R1 interleaves generation with retrieved evidence while restricting gradient updates and KL regularization to policy-generated tokens.The conditioning history incorporates previously retrieved evidence, and the objective is normalized by the number of generated tokens.Source: §A.2 · p. 18

B Multi-Turn Search Fatal-Aware GRPO Details

  • 91.8% of fatal rollouts receive zero advantage under one-sided clamping, while the remaining 8.2% retain positive advantages.Figure 5 aggregates 47,978 fatal rollouts across 10,000 groups with G=16. The multi-turn objective combines three-consecutive-failure detection and clamping with format, accuracy, and query rewards.Source: Figure 5 · p. 19, B Multi-Turn Search Fatal-Aware GRPO… · p. 19, §B.1 · p. 19

B.1 Fatal Step Detection Logic

  • 3 consecutive tool-execution failures mark a trajectory fatal; successful steps reset the counter, allowing recovery from isolated errors.The counter starts at zero, and f_i identifies the earliest threshold-reaching step; trajectories never reaching K = 3 use f_i = L_i + 1. Timeouts, malformed API payloads, and argument-parsing failures are flagged deterministically by the execution sandbox, without learned heuristics.Source: §B.1 · p. 19

B.2 Advantage Derivation with One-Sided Clamping

The advantage protocol includes fatal trajectories in group normalization and applies fatal-aware one-sided clamping to standardized rewards. Stop-gradient normalization confines gradient flow to importance-sampling ratios on unmasked, policy-generated tokens.

B.2 Advantage Derivation with One-Sided Clamping

  • Stop-gradient makes the group mean and standard deviation static during backpropagation, so gradients flow only through importance-sampling ratios on unmasked, policy-generated tokens.Source: §B.2 · p. 20
  • All sampled trajectories, including fatal-truncated trajectories, contribute to group normalization, allowing failures to shape the baseline and relative performance ranking.Group statistics use composite rewards from rollouts sharing the same prompt; a positive stability constant supports normalization.Source: §B.2 · p. 20
  • Fatal-aware one-sided clamping converts standardized rewards into final advantages, with the derivation asserting greater gradient informativeness than hard-masking (Huang et al., 2026).The hard-masking baseline discards every fatal trajectory entirely, setting its gradient contribution to zero.Source: §B.2 · p. 20

1. If eri < 0, then gours

Fatal-aware one-sided clamping preserves positive learning signals from viable rollout prefixes without propagating post-failure gradients or penalizing those prefixes. It introduces a non-negative bias while preserving non-fatal trajectory rankings.

Gradient guarantees

  • For non-negative advantages, the proposed gradient equals search-augmented GRPO restricted to policy-generated viable-prefix tokens.Negative advantages yield zero proposed and hard-masking gradients. The fatal-aware mask excludes environment-emitted tokens and all post-fatal tokens.Source: §1 · p. 20
  • The proposed gradient weakly informative-dominates hard masking, excluding post-fatal gradients and prefix penalties while reinforcing prefixes with above-baseline group-normalised returns.This recovers positive reinforcement from prematurely truncated but high-quality exploratory rollouts.Source: §1 · p. 20

Bias of Group Statistics under Clamping

  • 91.8% of fatal rollouts have negative standardised scores and are clamped to zero; the remaining 8.2% occupy competitive non-fatal rollouts’ score regime.The reported sanity check includes per-group-difficulty breakdowns and an aggregate score distribution over 10,000 groups.Source: §1 · p. 21
  • Clamping introduces a non-negative bias relative to zero-mean GRPO but preserves the ordering of non-fatal advantages.Group statistics include fatal rollouts; negative fatal advantages are subsequently zeroed. Alternatives left for future work include non-fatal-only baselines and doubly robust bias corrections.Source: §1 · p. 21

C Implementation Details · C.1 SFT Training Configuration

OpenSearch-VL extends SFT and RL pipelines to handle tool-interleaved multimodal trajectories. Its three model variants share SFT hyperparameters and undergo full-parameter finetuning across 256 GPUs.

C Implementation Details

  • The SFT pipeline extends LlamaFactory (Zheng et al., 2024) with multi-turn, tool-interleaved collators and Qwen3-VL-aware vision/text packing.This lets the policy consume interleaved image observations from Crop, Sharpen, SuperResolution, PerspectiveCorrect, and OCR verbatim.Source: C Implementation Details · p. 21
  • The RL pipeline combines rLLM’s asynchronous rollout architecture with Vision-DeepResearch’s (Huang et al., 2026) multimodal trajectory abstraction.The adopted architecture decouples trajectory generation, replay, and policy-update workers.Source: C Implementation Details · p. 21
  • The RL implementation adds a Qwen3-VL chat-template renderer and interleaved image-token re-alignment for variable-length visual observations.These components extend the combined rLLM and Vision-DeepResearch (Huang et al., 2026) foundations.Source: C Implementation Details · p. 21
  • An asynchronous multi-turn rollout engine supports tool-call interruption and resumption within the pipeline used to train every OpenSearch-VL variant.The engine extends the rLLM and Vision-DeepResearch (Huang et al., 2026) foundations.Source: C Implementation Details · p. 21

C.1 SFT Training Configuration

  • 256 GPUs across 32 nodes support full-parameter SFT, including the vision tower and multimodal projector, using DeepSpeed ZeRO-3 and Ray-based orchestration.Each node contains eight GPUs.Source: §C.1 · p. 21
  • 8B dense, 32B dense, and 30B-A3B Mixture-of-Experts variants share identical SFT hyperparameters, differing only in their base checkpoint paths.Table 4 consolidates the configuration for all three variants.Source: §C.1 · p. 21

C.2 RL Training Configuration

The 8B dense and 30B-A3B MoE variants use a shared multi-turn fatal-aware GRPO algorithm and rollout/actor stack. Their configurations separate shared settings from variant-specific settings and report only explicitly configured fields.

C.2 RL Training Configuration

  • Both RL runs use RLOO leave-one-out advantages under the GRPO group-relative objective, a low-variance KL controller, and no critic.This shared algorithm template is used for the multi-turn fatal-aware GRPO stage.Source: §C.2 · p. 21
  • The 8B dense (Qwen3-VL-8B-Instruct) and 30B-A3B MoE (Qwen3-VL-30B-A3B-Instruct) variants share an asynchronous SGLang rollout engine and Megatron-parallel actor.These are the two OpenSearch-VL variants RL-finetuned in this configuration.Source: §C.2 · p. 21
  • Table 5 reports explicitly configured launch settings, listing identical values once and variant-specific values separately.Differences cover response budget per prompt, mini-batch size, context/expert parallelism, MoE-specific routing coefficients, save cadence, and total epochs. Framework defaults and cluster-network environment variables are omitted; “—” marks inapplicable entries.Source: §C.2 · p. 21, Table 5 · p. 23

D Data Curation Details … G Case Study

The appendices detail multi-hop data construction safeguards, agent and judge prompts, and retrieval and visual-processing tools. A bridge case study illustrates how visual inspection, image search, and text search jointly verify an answer.

D Data Curation Details / D.1 Path Sampling Hyperparameters

  • 2-, 3-, and 4-hop paths receive probabilities 0.4, 0.4, and 0.2, respectively, with stratified visual seeds, hub avoidance, and bounded retries.On the 2025-05-01 English Wikipedia snapshot, seeds span five domains and require an infobox, a Commons image at least 512 × 512, and in-degree from 50 to 10,000; hub filtering excludes roughly the top 0.03% of articles. Seeds are dropped after 10 unsuccessful attempts involving filtered nodes, non-unique bridge descriptors, or missing terminal attributes within the six-token bound.Source: §D.1 · p. 24, D Data Curation Details · p. 24

D.2 Running Example: Australia_Zoo

  • 20 November 2009 is the Australia_Zoo example’s answer after fuzzy rewriting and visual grounding, with masking, uniqueness, visual relevance, and non-triviality checks passed.Rewriting preserves the answer while uniquely identifying Terri Irwin without naming the path entities. Eight candidate images are ranked using CLIP (Radford et al., 2021), and the selected image’s similarity of 0.34 exceeds the 0.28 relevance threshold.Source: §D.2 · pp. 24–25

D.3 Counterfactual Design Choices

  • Separating anchor from answer, fuzzy rewriting, and hub avoidance prevent lookup shortcuts and ambiguous or identity-leaking bridge descriptors.Using Terri Irwin’s photograph permits direct reverse-image lookup; retaining entity names permits direct text search. Admitting Queensland as a bridge creates non-unique zoo descriptions or forces descriptors that leak the anchor’s identity.Source: §D.3 · pp. 25–26

E System Prompts

  • Agent prompts are shared across inference and SFT, while separate judges supply RL rewards and benchmark evaluation aligned with Vision-DeepResearch (Huang et al., 2026).RL judges produce binary accuracy rewards and query-quality scores in [0, 1], whereas benchmark judging evaluates final trajectories post-hoc. Machine-readable tool schemas are injected into context for OpenAI-style function calling.Source: E System Prompts · p. 26, Figure 8 · p. 26, Figure 10 · p. 26

F Tool Definition and Usage / Retrieval Tools

  • Remote retrieval combines TextSearch’s web reading and summarization with ImageSearch’s visual-entity grounding, enabling subsequent textual verification.TextSearch pipelines Serper URL retrieval, JINA Reader page normalization, and Qwen3-32B query-focused summaries of 2–4 sentences. Polaris Lens powers ImageSearch, returning similar images, recognized entities, related URLs, and captions.Source: Retrieval Tools · p. 30, F Tool Definition and Usage · p. 30

Image Enhancement Tools

  • Sharpen, SuperResolution, and PerspectiveCorrect address blur, low resolution, and skew before downstream perception.Sharpen defaults to intensity 1.5; EDSR super-resolution defaults to ×4 and returns the original image if its weights are unavailable. Perspective correction warps a detected quadrilateral into a fronto-parallel view, otherwise returning the original image with a warning.Source: Image Enhancement Tools · p. 30

Attention and Parsing Tools

  • Crop isolates diagnostic image regions, while layout-aware OCR preserves text hierarchy and reading order after cropping or enhancement.Remote PaddleX OCR returns labeled text blocks and reconstructed formatted text, with optional chart recognition and document-orientation classification.Source: Attention and Parsing Tools · p. 31

G Case Study

  • 1982 is verified as Kessock Bridge’s opening year through Crop, ImageSearch, and TextSearch, illustrating evidence-grounded tool chaining beyond a single retrieval call.The visual–retrieval action space, query-quality reward, and fatal-aware masking are designed to encourage progressive grounding and termination once cross-modal evidence converges.Source: G Case Study · p. 31, Figure 9 · p. 32

Related papers

Loading 2605.05185v1…