Source-linked AI summary
Struct-Searcher: Agentic Structural Thinking Advances Multimodal Deep Information Seeking
Fan Zhang, Vireo Zhang, Shengju Qian, Haoxuan Li, Zheng Lian, Hao Wu, Yuan Gao, Xinyu Geng, Xin Wang, Pheng-Ann Heng
TL;DR
Multimodal deep research introduces cross-modal conflicts that linear evidence-accumulation workflows are not designed to handle. Struct-Searcher addresses this gap through AGM-inspired belief revision over an evolving multimodal structural graph, achieving state-of-the-art performance across three benchmarks.
Problem
Multimodal evidence can contradict across modalities, while existing deep research workflows typically linearly accumulate evidence toward a confidence threshold.
Method
Struct-Searcher maintains a multimodal structural graph and iteratively revises beliefs through expansion, contraction, and revision while organizing query, goal, hypothesis, and evidence nodes with support and conflict relations.
Results
Struct-Searcher achieves state-of-the-art performance across MM-BrowseComp, HLE-VL, and BrowseComp-VL, including overall accuracies of 32.7%, 17.3%, and 48.6%, respectively, with GPT-5.
Takeaways & Limitations
The results provide evidence that maintaining structured reasoning is useful for multimodal deep information seeking and supports robustness to cross-modal conflicts.
Abstract
from arXiv · showhide
Deep research agents have attracted increasing attention for their ability to collect large-scale online information to acquire target knowledge, with recent efforts shifting from purely text-based information seeking to multimodal settings. However, existing agentic workflows are largely aligned with evidence accumulation models, which linearly aggregate evidence and lack principled mechanisms for handling contradictory information across heterogeneous modalities. Towards this end, we propose Struct-Searcher, a structural agentic workflow grounded in belief revision theory that explicitly maintains an evolving multimodal structural graph throughout the reasoning process, enabling effective conflict-aware multimodal deep information seeking. Extensive experiments across multiple benchmark datasets and backbone models demonstrate that Struct-Searcher is (1) plug-and-play and model-agnostic, yielding an average relative accuracy improvement of 17.2% on BrowseComp-VL across five different backbones. (2) top-performing, consistently outperforming state-of-the-art vision-language models (VLMs) and deep research agents, with relative accuracy improvements of 3.7% on MM-BrowseComp, 1.5% on HLE-VL, and 0.7% on BrowseComp-VL over the second-best competing approach.
1. Introduction
Multimodal deep research introduces cross-modal conflicts that expose limits in linear evidence accumulation. Struct-Searcher addresses this gap with belief revision over an evolving multimodal structural graph and demonstrates strong benchmark performance.
- Motivation: Multimodal settings create semantic ambiguity, temporal inconsistency, redundancy, and cross-modal conflicts that complicate reliable information seeking.Conflicting evidence can create a persistent truth gap between modalities.
- Motivation: Existing deep research workflows typically linearly accumulate evidence until reaching a confidence threshold, but this approach struggles when evidence conflicts across modalities.The paper characterizes this workflow through Evidence Accumulation Models.
- Framework Proposal: Struct-Searcher reframes multimodal deep research as belief revision guided by AGM theory rather than simple evidence accumulation.The framework explicitly constructs and revises beliefs during multimodal search.
- Framework Proposal: Struct-Searcher represents query, goal, hypothesis, and evidence nodes with support and conflict relations in an evolving multimodal structural graph.It performs belief expansion, contraction, and revision as new information is retrieved.
- Framework Proposal: The final answer is synthesized from a maximal conflict-free subgraph, supporting global logical coherence and robustness to cross-modal conflicts.Making belief structures explicit also enables more transparent search behavior.
- Empirical Evaluation: Struct-Searcher is evaluated extensively on three multimodal deep information-seeking benchmarks, demonstrating plug-and-play use and state-of-the-art performance.The introduction presents this evaluation as empirical validation of the framework.
2. Related Work
Related work develops deep research agents and multimodal information-seeking capabilities, while agentic workflows organize reasoning and tool use in different ways.
- Deep Research Agents: Deep research agents autonomously conduct long-horizon research by iteratively searching, observing, and synthesizing information from diverse web sources.The related-work discussion cites OpenAI’s deep research system and Alibaba’s Tongyi DeepResearch as examples.
- Deep Research Agents: WebWatcher extends deep research toward multimodal settings by combining vision-language information seeking with code, text-search, and image-search tools.Its design targets richer visually grounded information.
- Agentic Workflow for Deep Information Seeking: Agentic information-seeking systems use varied workflows, including ReAct’s linear alternation between reasoning and actions to guide subsequent planning.The passage associates this paradigm with improved performance and interpretability across information-seeking tasks.
3. Methodology
Struct-Searcher replaces linear evidence accumulation with structural belief revision over an evolving multimodal graph. It decomposes queries into goals, gathers and relates evidence, revises hypotheses, and synthesizes answers from a maximal conflict-free subgraph.
- Preliminaries: Multimodal deep information seeking requires external knowledge, multi-hop reasoning, and integration of entities conveyed through images.The task extends standard question answering by combining external information gathering with multimodal understanding.
- Structural Thinking Perspective: Unstructured workflows can propagate noisy visual-identification errors through downstream reasoning, causing cascading failures under cross-modal conflict.The method addresses heterogeneity between visual and textual evidence rather than simply adding multimodal tools.
- Multimodal Structural Graph: The multimodal structural graph represents query, goal, hypothesis, and evidence nodes connected by decompose, generate, require, support, and refute relations.The query is initialized from the question and image; other nodes are generated during reasoning.
- Graph Initialization and Evolution: The agent initializes goals, acquires multimodal evidence, and evolves the graph through construct, populate, verify, and prune operations.The evolving graph supports structured belief expansion, contraction, and revision under AGM principles.
- Belief Revision and Update: AGM-guided updates revise beliefs using evidence-induced support and refutation relations while separately propagating hypothesis confidence.Confidence is auxiliary and does not directly alter the logical belief state.
- Termination and Answer Generation: Reasoning terminates at a fixed point when the belief state reaches a sufficiently supported, conflict-free hypothesis, whose maximal supporting subgraph conditions final answer generation.The final graph is projected into context before synthesis.
4. Experiments
Experiments evaluate Struct-Searcher across three multimodal benchmarks, multiple models, and several workflow analyses. The results show strong overall performance, cross-model gains, and advantages for structural reasoning in most comparisons.
- Experimental Setup: Struct-Searcher is evaluated on MM-BrowseComp, HLE-VL, and BrowseComp-VL against diverse VLM and agentic baselines.The evaluation uses overall accuracy across all three datasets, with additional checklist and strict-accuracy metrics on MM-BrowseComp.
- Main Results: 32.7% overall accuracy, 26.0% strict accuracy, and 44.6% average checklist score are achieved on MM-BrowseComp with GPT-5.The same setup reaches 17.3% overall accuracy on HLE-VL and 48.6% on BrowseComp-VL.
- Main Results: Struct-Searcher achieves state-of-the-art performance across all three benchmarks and surpasses many leading VLMs and unstructured agentic workflows.The reported results identify structural thinking as important for multimodal deep information seeking.
- Fine-grained Analysis: Struct-Searcher consistently matches or outperforms Flash-Searcher across domain-specific sub-tasks, except Geography accuracy on MM-BrowseComp.The paper attributes this pattern to structural thinking's continued backtracking and self-reflection, which mitigate multimodal conflicts and error accumulation.
- Workflow Analysis: 21.8% on MM-BrowseComp, 1.2% on HLE-VL, and 0.7% on BrowseComp-VL are the absolute overall-accuracy improvements over linear and parallel workflows.The comparison uses GPT-5 as the backbone and includes ReAct and Flash-Searcher as representative workflow types.
- Sensitivity Analysis: 17.2% average improvement across five backbone models demonstrates plug-and-play, model-agnostic applicability.The sensitivity analysis varies GPT-4.1, GPT-4o, Gemini-2.5-Flash, Gemini-2.5-Pro, and GPT-5.
- Case Analysis: A BrowseComp-VL case decomposes a question into sub-goals, gathers evidence with multiple tools, revises hypotheses, and synthesizes an answer from a conflict-free subgraph.The process uses iterative planning and summarization to support or refute generated hypotheses.
5. Conclusion
The conclusion presents Struct-Searcher as a structural-thinking framework for multimodal information seeking, guided by AGM belief revision and an evolving multimodal structural graph. Experiments show consistent advantages over leading VLMs and deep research agents, while future work targets an agentic foundation model.
- Conclusion: Struct-Searcher uses AGM belief revision theory and an evolving multimodal structural graph to preserve structured reasoning and mitigate multimodal conflicts.The framework is proposed to advance multimodal deep information seeking through structural thinking.
- Conclusion: Experiments across challenging benchmarks show Struct-Searcher consistently outperforms state-of-the-art VLMs and other deep research agents.The conclusion reports this as the paper's broad empirical finding.
- Future Work: Future work will use agentic post-training techniques, such as reinforcement learning, to develop an open-source agentic foundation model.This direction is presented as building upon the structural-thinking workflow.
A.1. Introduction to Benchmark Datasets
The paper evaluates multimodal deep information seeking with benchmark datasets designed to require reasoning over visual and textual information. MM-BrowseComp and HLE-VL cover distinct task formats, categories, and evaluation needs.
- MM-BrowseComp: MM-BrowseComp contains 224 manually curated questions requiring agents to integrate visual and textual information from web content.Its questions span Media, Technology, Society, Geography, and Academics, and include verified checklists for fine-grained reasoning evaluation.
- HLE-VL: HLE-VL contains 330 image-based questions across eight scientific and humanities categories.The dataset includes multiple-choice and short-answer formats, with difficulty checks for each question.
A.2. Tool Configurations
Struct-Searcher uses multiple tools to collect and analyze web-based and multimodal evidence, with Figure 7 documenting the tool setup across benchmarks.
- Tool Configurations: The framework employs web search, image search, web crawling, image analysis, and text analysis for evidence collection.Web and image search retrieve online sources, while crawling and analysis tools process the collected information.
- Tool Configurations: Web search returns five relevance-ranked results with titles, dates, snippets, and URLs for each invocation.
- Tool Configurations: Image search returns five relevance-ranked images with titles, page URLs, and thumbnail URLs.Thumbnail URLs are used because many original image URLs require inaccessible authentication.
- Tool Configurations: Figure 7 analyzes Struct-Searcher tool usage across benchmarks using GPT-5 as the backbone.
B. Tool Analysis
Tool analysis shows that Struct-Searcher relies most heavily on web and image search while combining retrieval, crawling, and multimodal analysis to gather evidence.
- Tool Analysis: Web search and image search have average usage rates of 56.0% and 9.3%, respectively, across datasets.
- Tool Analysis: Web search and image search are the most important tools in Struct-Searcher’s evidence-collection process.
- Tool Analysis: The agent also uses web crawl, image analysis, and text analysis to crawl, read, and analyze collected information.These tools support perception of the environment and more accurate reasoning.
- Tool Analysis: Figure 8 includes word-cloud visualizations of GPT-5 Struct-Searcher trajectories across different datasets.
C. More Results
Sensitivity analysis indicates that Struct-Searcher improves performance across all five tested backbone models on MM-BrowseComp.
- More Results: 7.1% average gain is achieved across five backbone models on MM-BrowseComp.The result comes from the sensitivity analysis reported in Figure 8(a).
- More Results: Struct-Searcher consistently improves performance across all five tested backbones.
D. Prompt Template
The prompt template operationalizes Struct-Searcher as a graph-based agent that decomposes goals, tracks evidence and hypotheses, revises conflicting branches, and coordinates subsequent actions.
- System Prompt: The system prompt instructs the agent to build and evolve a Multimodal Structural Graph containing Goals, Evidence, and Hypotheses.The objective is to expand the graph until a logically self-consistent answer is found.
- System Prompt: Every action must include reasoning and tool calls that reflect the current graph state and execution plan.The prompt requires explicit pending goals, active hypotheses, and tool specifications.
- System Prompt: Independent tasks may run in parallel, while dependent tasks must execute sequentially.Parallel execution is reserved for logically independent verification or searches.
- Planning Prompt: The workflow decomposes queries into dependent goals and proposes heterogeneous image-search and text-search strategies for each goal.Goals are parallel only when completely unrelated.
- Planning Prompt: The planning prompt requires a structured output listing goal dependencies and tool-query strategies without solving the task directly.
- Summary Prompt: The summary prompt audits goal status, hypothesis support or refutation, cross-modal conflicts, and recommended next logical steps.It can recommend pruning refuted branches or unlocking dependent goals after achievement.
- Summary Prompt: Graph status reports summarize topology, branch details, evidence, hypothesis states, and recommended actions.
- Step Prompt: The step prompt unlocks dependencies, validates competing hypotheses in parallel, cross-verifies important facts, and stops refuted branches.It permits up to five tools and requires at least one tool invocation.
E. Case Study
The case study resolves conflicting visual and textual evidence by maintaining competing species hypotheses and revising them as taxonomic, plumage, and habitat evidence accumulates. It ultimately verifies Sporophila crassirostris and identifies its natural habitats as moist shrubland, wetlands, and heavily degraded former forest.
- Initial identification of Sporophila minuta conflicts with the German-naturalist clue and its expected male plumage, creating competing species hypotheses.
- The workflow begins by extracting diagnostic visual features, including a glossy black male, stout pale bill, small white wing patch, and grassy or shrubby surroundings.
- Sporophila crassirostris becomes the leading hypothesis because its Gmelin 1789 authority, large pale bill, and habitat wording match the task clues.
- Cross-validation confirms the habitats as subtropical or tropical moist shrubland, swamps or other wetlands, and heavily degraded former forest.
- Sporophila americana is rejected because its habitat evidence lacks the wetlands match, while S. minuta is rejected for its Linnaean authority and plumage mismatch.
- All goals are achieved, H1 is verified, and the alternative hypotheses are refuted before the final answer is produced.