PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agents

Huanyao Zhang, Jiepeng Zhou, Runhao Zhao, Yanzhe Shan, Jiaoyang Chen, Bowen Zhou, Bo Li, Fang Wang, Jialong Wu, Zhengwei Tao, Lang Mei, Xiaohan Yu, Liyan Liu, Chong Chen, Wentao Zhang

arXiv:2608.01827v1cs.CVcs.AI

TL;DR

Existing multimodal search methods provide limited supervision for visual evidence needed during intermediate reasoning. DeepVoyager-VL synthesizes vision-dependent long-horizon tasks and trains an agent for active visual acquisition, improving average performance by 55.2% for 8B models and 44.0% for 30B-A3B models across ten benchmarks.

  • Problem

    Existing methods provide limited supervision for intermediate visual dependencies and lack designs tailored to long-horizon interaction.

  • Method

    DeepVoyager-VL builds a visually enriched multimodal event graph for long-horizon task synthesis and trains an agent to actively acquire visual evidence using supervised trajectories.

  • Results

    55.2% and 44.0% average performance improvements are reported for 8B and 30B-A3B models, respectively, over the same base models across ten benchmarks.

  • Takeaways & Limitations

    The results support vision-in-the-loop behavior as an effective approach for long-horizon multimodal search across diverse task settings.

  • Takeaways & Limitations

    The evaluation focuses on answer-oriented multimodal search and does not systematically assess open-ended research capabilities such as objective decomposition or long-form synthesis.

Abstract

from arXiv · show

Multimodal large language models (MLLMs) have advanced visual understanding and reasoning, yet their static parametric knowledge limits their ability to address knowledge-intensive and dynamically evolving open-world problems. To move beyond this limitation, multimodal deep search has emerged as a key direction for open-world information access, evolving from single-turn factual retrieval toward long-horizon, multi-turn search guided by visual evidence. However, existing methods typically confine vision to the input or answer stage, overlooking its role in intermediate reasoning, and lack designs tailored to long-horizon interaction. Consequently, visual evidence rarely drives continued retrieval, constraining both interaction depth and reasoning span. To address these limitations, we propose DeepVoyager-VL, a long-horizon multimodal deep-search framework for vision-in-the-loop search. Specifically, we construct a multimodal event graph to drive data synthesis, yielding problems with intermediate visual dependencies and long reasoning chains. We then design an agent framework for active visual acquisition and on-demand image loading. Finally, we fine-tune models on the synthesized data without reinforcement learning. Extensive experiments across ten multimodal search benchmarks demonstrate the effectiveness of our method.

1 INTRODUCTION

DeepVoyager-VL addresses the limitations of fixed parametric knowledge and shallow multimodal search by making intermediate visual evidence drive long-horizon retrieval. It combines visually enriched event-graph synthesis with active visual acquisition and supervised learning, demonstrating effectiveness across ten multimodal search benchmarks.

  • MLLMs’ fixed parametric knowledge limits performance on knowledge-intensive and dynamically evolving problems, motivating deep-search agents that gather external evidence through iterative reasoning and tool use.
  • Existing multimodal search methods provide little supervision for intermediate visual dependencies, typically placing vision at the input, before textual search, or near the answer.
  • Acquired visual observations rarely become necessary for later retrieval, while long-horizon search must balance costly image loading against textual compression that can discard details needed later.
  • EventVoyage-VL uses a visually enriched multimodal event graph and structure-before-language synthesis to construct long-horizon problems with explicit intermediate visual dependencies.
  • DeepVoyager-VL actively acquires referenceable visual evidence and learns vision-in-the-loop behavior from supervised trajectories alone, with experiments spanning ten multimodal search benchmarks.

2 RELATED WORK

Related work advances multimodal models, deep-search agents, and benchmarks toward increasingly integrated vision-guided reasoning. However, the field is moving from input-only visual use and single-turn retrieval toward long-horizon multimodal search.

  • Multimodal large language models: MLLMs improve visual understanding through large-scale cross-modal pretraining and post-training methods that strengthen reasoning and enable active image manipulation.The passage lists representative pretraining and post-training efforts but is truncated before the full citation list.
  • Multimodal deep-search agents: Deep-search agents iteratively reason and use tools to acquire information, while multimodal variants add visual tools and train search policies with supervised fine-tuning and outcome-based reinforcement learning.The passage frames multimodal deep search as an extension of tool-using deep-search agents.
  • Multimodal deep-search agents: The field is shifting from single-turn factual retrieval toward long-horizon, cross-modal search needs.This transition is identified in the discussion of deep-search agents and multimodal variants.
  • Multimodal search benchmarks: Multimodal search benchmarks have progressed from input images used mainly for entity identification to finer perception and visual reasoning during search.Recent benchmarks emphasize newly discovered visual information, cross-modal integration, fine-grained perception, and interleaved multimodal agentic search.

3 METHOD

DeepVoyager-VL defines vision-in-the-loop search as visual evidence that resolves intermediate variables, drives later actions, and is necessary for answering. Its method synthesizes such tasks from multimodal event graphs, supports active visual acquisition, curates trajectories by difficulty, and trains agents with supervised fine-tuning.

  • Vision-in-the-loop definition: Vision-in-the-loop search requires newly acquired visual evidence to resolve an intermediate variable for a later action and be necessary for the answer.Equations 2 and 3 exclude input-only or terminal vision by requiring visual occurrence within inference and counterfactual necessity.
  • Multimodal event graph construction: The pipeline constructs provenance-preserving multimodal event graphs by enriching normalized entities and events with images, retrieval context, visual evidence, and alignments.Images serve as input anchors, cross-entity transitions, visual intermediates, or endpoints, while answer leakage is prevented through Kret(I) ∩ Alias(y) = ∅.
  • Subgraph composition and VQA synthesis: Subgraphs become inference programs with compatible interfaces, visually resolved internal variables, downstream targets absent from source images, and validated question–answer realizations.The method uses visual co-occurrence networks, temporal chains, spatial structures, and dense motifs, then checks completeness, minimality, leakage, alignment, and visual necessity.
  • Agent framework and graded evidence interface: The agent separates image discoverability from observability, exposing lightweight references during search and entering visual tokens into context only through explicit fetch or crop actions.Its interface combines candidate discovery, goal-directed reading, active perception, and computation, while query-dependent summaries retain navigation clues without replacing answer-bearing images.
  • Difficulty stratification and supervised training: Difficulty is estimated from tool-augmented rollout Pass@K, after which a stronger teacher regenerates legal, correct, and replayable trajectories for supervised fine-tuning.The unified multi-turn corpus combines general-search and vision-in-the-loop trajectories; the visual encoder and multimodal merger remain frozen, and reinforcement learning is not used.

4 EXPERIMENTS

DeepVoyager-VL is evaluated on ten multimodal information-seeking benchmarks using standardized multimodal search comparisons and LLM-based judging. Experiments show substantial gains from agentic workflows, trajectory SFT, vision-in-the-loop data, and active visual tools, which also produce longer interaction horizons.

  • Benchmarks: DeepVoyager-VL is evaluated on ten benchmarks spanning factual visual QA, multimodal evidence seeking, and long-horizon visual browsing.The benchmarks include MMSearch, SimpleVQA, LiveVQA, FVQA, BrowseComp-VL, MM-BrowseComp, MMSearch-Plus, VDR-Bench, BrowseComp-V3, and VisBrowse-Bench.
  • Overall Performance: 58.6 average performance follows agentic workflow and trajectory SFT for the 30B-A3B model, rising from 20.7 to 40.7 and then 58.6.The corresponding 8B progression is 17.5 to 35.3 and then 54.8.
  • Framework Comparison: 18.2 average gain over LMM-Searcher is achieved with GPT-5, compared with 11.5 for Gemini-2.5-Pro and 4.8 for Qwen3-VL-30B-A3B-Thinking.The largest reported gains are +27.8 on VisBrowse-Bench with Gemini-2.5-Pro and +26.1 on MM-BrowseComp with GPT-5.
  • Data Ablation: 21.4 average points are added by 20K multi-source trajectories before the full 27K mixture adds 7K synthesized vision-in-the-loop trajectories.Training and inference settings are fixed across the cumulative comparison.
  • Framework Ablation: Removing any one of Summary, Image Search, Fetch Image, or Crop Image lowers average performance by 2.5–4.0 points.Summary has the largest average drop; component importance varies by benchmark, with cropping most important on VDR-Bench.
  • Trajectory Analysis: 64.3% of EventVoyage-VL tool calls are visual, versus 40.6% for OpenSearch-VL, 30.3% for Vision-DeepResearch, and 10.1% for REDSearcher-MM.EventVoyage-VL trajectories peak at 16–20 turns, whereas OpenSearch-VL peaks at 1–5 and Vision-DeepResearch and REDSearcher-MM at 6–10 turns.

5 CONCLUSION … A.2 MULTIMODAL DEEP SEARCH AGENTS

DeepVoyager-VL presents a long-horizon multimodal deep-search method that places visual evidence inside the retrieval loop. The paper positions it against multimodal agents that manipulate supplied images or manage already-acquired context rather than selectively acquiring visual evidence during search.

  • 5 CONCLUSION: DeepVoyager-VL constructs a multimodal event graph, enables active visual acquisition and on-demand image loading, and fine-tunes resulting trajectories without additional reinforcement learning.The synthesized questions contain intermediate visual dependencies and long reasoning chains.
  • A.1 MULTIMODAL LARGE LANGUAGE MODELS: Early multimodal large language models established visual instruction following by connecting pretrained vision encoders with language models through lightweight projection modules.Later systems scale architectures and vision–language corpora, while proprietary systems show similar scaling trends.
  • A.1 MULTIMODAL LARGE LANGUAGE MODELS: Thinking-with-images research lets models manipulate visual content during reasoning, using either reinforcement learning incentives or cold-start supervision for visual tool use.Pixel Reasoner identifies a learning trap in which models bypass visual reasoning.
  • A.1 MULTIMODAL LARGE LANGUAGE MODELS: Open-world multimodal search requires agents to discover unseen images, inspect them selectively, and use observations to determine subsequent retrieval actions.This extends beyond reasoning over images supplied initially with the task.
  • A.2 MULTIMODAL DEEP SEARCH AGENTS: Multimodal deep-search agents extend iterative reasoning, tool use, and evidence integration with image search, reverse image search, and visual manipulation.Existing systems establish search behavior through outcome-based reinforcement learning, supervised cold-start trajectories, or combinations of both.
  • A.2 MULTIMODAL DEEP SEARCH AGENTS: The action space has expanded from reverse image search to cropping, enhancement, perspective correction, and generated-code image manipulation.These tools improve local perception and visual retrieval.
  • A.2 MULTIMODAL DEEP SEARCH AGENTS: Context-management methods offload, fold, update, or compress evidence after it enters interaction history, while evidence reliability requires explicit verification beyond snippets, titles, captions, and file names.These approaches address context growth but primarily operate after evidence acquisition.
  • A.2 MULTIMODAL DEEP SEARCH AGENTS: DeepVoyager-VL manages evidence acquisition before full visual content enters context by separating candidate discovery from selective observation and preserving observations for later operations and retrieval.Its cumulative visual working memory connects discovery, observation, local manipulation, and follow-up search.

A.3 DATASETS AND BENCHMARKS

Multimodal search benchmarks have progressed from image-based entity identification toward deeper visual interaction, requiring agents to inspect discovered images and combine evidence across pages. Existing synthesis paradigms insert visual reasoning at different stages, but generally construct the main reasoning chain in text first, whereas EventVoyage-VL builds visual interfaces into a multimodal event graph before language realization.

  • Benchmark evolution: Benchmarks increasingly move vision deeper into the interaction loop, from entity identification before textual retrieval to inspecting discovered images and combining evidence across pages.Later evaluations also require finer-grained perception of the initial visual input and stronger integration with retrieved information.
  • Existing synthesis paradigms: Entity substitution constructs a textual reasoning path before replacing an entity mention with an image, while OpenSearch-VL anchors images at functional source nodes to avoid answer-side shortcuts.The passage notes that OpenSearch-VL argues grounding near the answer can create single-hop shortcuts.
  • Existing synthesis paradigms: Vision-DeepResearch performs detailed visual reasoning before a text-oriented search process, producing richer visual reasoning that remains concentrated before the subsequent trajectory.This paradigm explores the input image and converts it into a textual description before deep research begins.
  • Core limitation: Across the three paradigms, the principal reasoning chain is constructed in text first, so newly acquired visual observations rarely resolve intermediate variables controlling later retrieval actions.The limitation concerns visual evidence influencing continued retrieval during the search trajectory.
  • EventVoyage-VL: EventVoyage-VL starts from a visually enriched multimodal event graph, composes an inference program with internal visual interfaces, and realizes that program as a question.This structure-before-language design makes visual evidence part of the reasoning structure rather than a later insertion.

B METHOD DETAILS · B.1 VISION-IN-THE-LOOP DATA SYNTHESIS · B.2 LONG-HORIZON MULTIMODAL SEARCH AGENT

DeepVoyager-VL synthesizes multimodal search tasks from a unified spatio-temporal event graph and equips a long-horizon agent with goal-conditioned search, image acquisition, and explicit visual inspection. The method separates retrieval provenance from pixel-grounded evidence so visual observations can guide subsequent search and answers.

  • B.1 VISION-IN-THE-LOOP DATA SYNTHESIS: The synthesis pipeline unifies source records, visual evidence, and spatio-temporal aggregation in a multimodal event graph for subsequent program construction.Atomic events preserve directed source–target relations, repeated entity-pair interactions form edges, and events sharing a date and city form macro-events.
  • B.1 VISION-IN-THE-LOOP DATA SYNTHESIS: Image records separate provenance and retrieval cues from visually grounded content, distinguishing image relocation evidence from facts that must be read from pixels.This separation enables later programs to use retrieval cues for locating images while reserving pixel-grounded content for visual reasoning.
  • B.1 VISION-IN-THE-LOOP DATA SYNTHESIS: Approximately 3.78 million atomic event records aggregate into 30,337 macro-events and approximately 7K multimodal search tasks.The counts distinguish raw event observations from macro-event units used for program composition; macro-events are not additional source events.
  • B.2 LONG-HORIZON MULTIMODAL SEARCH AGENT: The public action schema provides lightweight candidate or goal-conditioned evidence search tools, while FETCHIMAGE and CROPIMAGE materialize new visual observations in policy context.Batch limits constrain the context and latency of a single action without preventing follow-up calls.
  • B.2 LONG-HORIZON MULTIMODAL SEARCH AGENT: Before visited pages enter policy context, Qwen3-VL-Plus summarizes them against the current search goal after boilerplate removal, truncation, and extraction of up to 15 image references.This preprocessing retains goal-relevant page information while limiting content and image references passed to the policy.
  • B.2 LONG-HORIZON MULTIMODAL SEARCH AGENT: The webpage summary helper returns goal-relevant sections, tables, names, dates, numbers, claims, relevance explanations, and supporting content as valid JSON without navigation or advertising.Its system instructions require JSON-only output and exclude menus, sidebars, cookie banners, footers, and advertisements.
  • B.2 LONG-HORIZON MULTIMODAL SEARCH AGENT: Image-search helpers score candidate relevance and provide pixel-grounded captions, while reverse image search reports visual match, partial match, not a match, or insufficient visual evidence.These outputs guide navigation, but answer-bearing visual claims require explicitly loading and inspecting the selected image.

B.3 DATA CURATION AND AGENT TRAINING · C EXPERIMENTAL DETAILS

The paper curates multimodal search data by filtering for tool necessity and difficulty, regenerating verified teacher trajectories, and applying masked supervised fine-tuning. The retained corpus combines synthesized tasks with public trajectories normalized to a common tool protocol.

  • B.3 DATA CURATION AND AGENT TRAINING: The candidate mixture combines synthesized tasks with public LiveVQA, FVQA, WebQA, REDSearcher-MM, and Vision-DeepResearch trajectories, normalized to a common tool protocol.Approximately 20K open-source trajectories are retained in the training mixture.
  • B.3 DATA CURATION AND AGENT TRAINING: Questions are discarded when Qwen3-VL-8B-Instruct answers any of three tool-free attempts correctly, using Qwen-3.6-Flash for judgment.This Pass@3 filter removes questions for which a successful direct answer indicates tool use is unnecessary.
  • B.3 DATA CURATION AND AGENT TRAINING: Remaining questions are stratified by correctness across eight tool-augmented rollouts into medium or hard sets, while easy questions are removed.Questions with 7–8 correct attempts are easy and removed; 4–6 are medium, and 0–3 are hard.
  • B.3 DATA CURATION AND AGENT TRAINING: Probe trajectories generated for difficulty estimation are excluded from the supervised corpus, although the evaluated questions remain retained.The rollouts are used only to estimate question difficulty.
  • B.3 DATA CURATION AND AGENT TRAINING: For retained medium and hard questions, doubao-seed-2-0-pro-260215 generates teacher trajectories under the DeepVoyager-VL interface, and only trajectories with correct final answers are retained.Qwen-3.6-Flash verifies each trajectory, separating filtering probes from supervision targets.
  • B.3 DATA CURATION AND AGENT TRAINING: Supervised fine-tuning masks loss on assistant reasoning, serialized tool calls, and final answers, while system, user, multimodal, and tool-return tokens provide context only.Retained trajectories are serialized as interleaved reasoning, tool calls, tool observations, and final answers.

C.1 EXPERIMENTAL SETUP

DeepVoyager-VL is evaluated on ten public multimodal search benchmarks spanning factual retrieval, deep browsing, and visual-native information seeking. Comparisons separate tool-free capability, agentic tool use, and specialized multimodal search training under a common evaluation protocol.

  • Benchmarks: Ten public benchmarks cover multimodal factual search, deep multimodal browsing, and visual-native or verifiable information seeking.They include protocols for accuracy, Success Rate, provenance, evidence integration, and visual-web reasoning.
  • Benchmarks: Factual benchmarks test visual-text evidence integration, factual visual question answering, up-to-date visual knowledge, and search-required questions.The set comprises MMSearch, SimpleVQA, LiveVQA, and FVQA.
  • Benchmarks: Deep-browsing benchmarks evaluate multi-step browsing, multimodal evidence integration, provenance, and complementary visual-textual search.The set comprises BrowseComp-VL, MM-BrowseComp, MMSearch-Plus, and VDR-Bench.
  • Baselines: Comparisons distinguish Direct Answer, Agentic Workflow, and Multimodal Deep Search Agents to isolate tool use and specialized training effects.Agentic Workflow keeps the five general-purpose MLLMs unchanged, while the specialized group contains models trained for multimodal retrieval and browsing.
  • Evaluation protocol: All evaluated agents receive at most 50 interaction turns, with answer correctness judged using Qwen3.6-35B-A3B and prior agents’ reported coverage preserved.Missing entries are not imputed, and averages over incomplete coverage are not computed.

C.2 IMPLEMENTATION DETAILS · D LIMITATIONS AND FUTURE WORK

The supplied section details DeepVoyager-VL’s training configuration, distributed implementation, software stack, and inference environment. It specifies fixed training and evaluation procedures, including the final checkpoint and a 50-turn tool-use budget.

  • C.2 IMPLEMENTATION DETAILS: The models train on the same final mixture of 27,180 multimodal tool-use trajectories for four epochs.Qwen3-VL-8B-Instruct and Qwen3-VL-30B-A3B-Instruct are both supervised on this mixture.
  • C.2 IMPLEMENTATION DETAILS: Training uses sequence packing with a maximum sequence length of 131,072 tokens, while freezing the vision encoder and multimodal merger.The language backbone is updated, and bfloat16 precision is used throughout.
  • C.2 IMPLEMENTATION DETAILS: The optimization setup uses Adam with β1 = 0.9, β2 = 0.95, ϵ = 10^-8, weight decay 0.1, and gradient clipping.These settings apply to the reported training configuration.
  • C.2 IMPLEMENTATION DETAILS: Training uses a global batch size of 64 and a micro-batch size of one on 64 NVIDIA H20 GPUs across eight nodes.The distributed setup includes tensor parallelism of 4, context parallelism of 2, and sequence parallelism for both models.
  • C.2 IMPLEMENTATION DETAILS: Packing produces 3,186 sequences and 199 optimizer steps over four epochs, with the final checkpoint at step 199 used without validation-based selection.No separate validation split is constructed.
  • C.2 IMPLEMENTATION DETAILS: The 30B-A3B model additionally uses expert parallelism of 8 within the distributed training configuration.Both models use tensor, context, and sequence parallelism as specified.
  • C.2 IMPLEMENTATION DETAILS: Training uses ms-swift 4.0.3, Megatron-Core 0.15.3, PyTorch 2.10.0, and FlashAttention for long-context training.Inference uses Serper1, Jina2, and Cloudflare3 for retrieval, webpage access, and image materialization.
  • C.2 IMPLEMENTATION DETAILS: Unless otherwise noted, evaluations use the same tool schemas and a maximum budget of 50 turns.The inference agent retrieves web and image content with Serper1 and accesses webpages through Jina2.

D.1 LIMITATIONS · D.2 FUTURE WORK

DeepVoyager-VL is currently scoped to answer-oriented multimodal search, with limitations in research-process coverage, evidence-processing latency, and visual working-memory management. Future work targets full multimodal research, more efficient summarization, and a modular research-agent harness.

  • D.1 LIMITATIONS: The evaluation focuses on answer-oriented multimodal search, emphasizing answer correctness and evidence acquisition rather than the complete research process.The current setting assumes a relatively well-specified question, external evidence gathering, and a concise answer.
  • D.1 LIMITATIONS: Auxiliary vision-language summarization adds latency because WEBVISIT, IMAGESEARCH, and REVERSEIMAGESEARCH calls occur between tool responses and the policy’s next action.Goal-conditioned summarization reduces raw content entering the policy context but does not eliminate its computation.
  • D.1 LIMITATIONS: Visual working memory is cumulative and trajectory-local, without learning which observations to retain, consolidate, compress, or discard.It preserves input images, fetched images, and cropped observations as referenceable objects for follow-up visual operations.
  • D.2 FUTURE WORK: Future multimodal research systems should decompose open-ended objectives into hierarchical research questions, revise plans, reconcile source agreement and conflict, and produce structured reports with traceable citations.Evaluation should extend beyond final-answer correctness.
  • D.2 FUTURE WORK: Future systems could reduce summarization overhead through adaptive invocation, caching, batching, asynchronous execution, and smaller distilled summarizers.Examples include bypassing summarization for short structured responses, reusing cached page representations, and selectively invoking stronger multimodal summarizers.
  • D.2 FUTURE WORK: A richer research-agent harness could coordinate planning, tools, memory, and intermediate research artifacts around the policy.Proposed components include hierarchical multimodal memory, an evidence index, a provenance graph, and learned retention.
Loading 2608.01827v1…