PaperlayerView exact revision on arXiv

Source-linked AI summary

WebThinker: Empowering Large Reasoning Models with Deep Research Capability

Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yongkang Wu, Ji-Rong Wen, Yutao Zhu, Zhicheng Dou

arXiv:2504.21776v2cs.CLcs.AIcs.IR

TL;DR

Large reasoning models struggle with knowledge-intensive tasks requiring deep web retrieval and comprehensive report synthesis. WebThinker equips them with autonomous web exploration, interleaved reasoning and drafting, and iterative online DPO training, and it outperforms existing methods and proprietary systems across the evaluated reasoning and report-generation tasks.

  • Problem

    Large reasoning models’ reliance on static internal knowledge limits complex knowledge-intensive problem solving and comprehensive research-report generation.

  • Method

    WebThinker combines a Deep Web Explorer, an Autonomous Think-Search-and-Draft strategy, and iterative online DPO to improve research-tool use.

  • Results

    WebThinker consistently outperforms competing approaches on GPQA, GAIA, WebWalkerQA, HLE, and Glaive report-generation evaluations.

  • Takeaways & Limitations

    The results demonstrate WebThinker’s stronger performance for complex reasoning and scientific report generation through integrated web exploration and report production.

  • Takeaways & Limitations

    WebThinker cannot process multimodal information such as images and videos.

Abstract

from arXiv · show

Large reasoning models (LRMs), such as OpenAI-o1 and DeepSeek-R1, demonstrate impressive long-horizon reasoning capabilities. However, their reliance on static internal knowledge limits their performance on complex, knowledge-intensive tasks and hinders their ability to produce comprehensive research reports requiring synthesis of diverse web information. To address this, we propose WebThinker, a deep research agent that empowers LRMs to autonomously search the web, navigate among web pages, and draft reports during the reasoning process. WebThinker integrates a Deep Web Explorer module, enabling LRMs to dynamically search, navigate, and extract information from the web when encountering knowledge gaps. It also employs an Autonomous Think-Search-and-Draft strategy, allowing the model to seamlessly interleave reasoning, information gathering, and report writing in real time. To further enhance research tool utilization, we introduce an RL-based training strategy via iterative online Direct Preference Optimization (DPO). Extensive experiments on complex reasoning benchmarks (GPQA, GAIA, WebWalkerQA, HLE) and scientific report generation tasks (Glaive) demonstrate that WebThinker significantly outperforms existing methods and strong proprietary systems. Our approach enhances LRM reliability and applicability in complex scenarios, paving the way for more capable and versatile deep research systems. The code is available at https://github.com/RUC-NLPIR/WebThinker.

1 Introduction

WebThinker addresses the difficulty of using web information during long-horizon reasoning by combining autonomous exploration, interleaved drafting, and tool-oriented training. It is evaluated on complex reasoning and report-generation tasks, where it outperforms competing approaches.

  • Motivation: LRMs relying solely on internal knowledge struggle with in-depth web retrieval and comprehensive, accurate reports for complex information needs.
  • Motivation: Predefined RAG workflows limit deeper web exploration and close interaction between reasoning models and search engines.
  • Approach: WebThinker enables LRMs to autonomously search and navigate web pages during reasoning, acquiring external knowledge for complex problem solving.
  • Approach: Its Think-Search-and-Draft strategy interleaves reasoning, information gathering, and report writing, with tools for drafting, checking, and editing sections.
  • Approach: Iterative online DPO trains research-tool use from preference pairs based on reasoning accuracy, tool usage, and final outputs.
  • Evaluation: WebThinker surpasses Search-o1 by 21.9% on GAIA and 36.2% on HLE, and outperforms Grok3 and Gemini2.0 on report generation.

2 Related Work

Related work positions WebThinker at the intersection of large reasoning models and retrieval-augmented generation. LRMs extend test-time reasoning, while RAG adds access to external knowledge through retrieval and increasingly complex workflows.

  • Large Reasoning Models: Large reasoning models improve test-time performance through extended reasoning rather than relying mainly on model size or data volume.
  • Large Reasoning Models: OpenAI-o1, Qwen-QwQ, and DeepSeek-R1 exemplify explicit chain-of-thought reasoning for tasks such as mathematics and programming.
  • Comparison: Figure 2 contrasts standard RAG, iterative RAG, and WebThinker’s autonomous search, web exploration, and report drafting within continuous thinking.
  • Large Reasoning Models: Research has enhanced reasoning through intentional training errors, distilled data, and reinforcement learning approaches.
  • Retrieval-Augmented Generation: Retrieval-augmented generation integrates retrieval mechanisms so generative models can access knowledge beyond their static parameters.
  • Retrieval-Augmented Generation: RAG research includes retrieval necessity, query reformulation, document compression, denoising, instruction-following, structured planning, and decision-making workflows.

3 Methodology

WebThinker equips reasoning models to invoke research tools while generating reasoning and outputs, supporting both complex problem solving and report generation. Its methodology combines web exploration, report-writing tools, and iterative online DPO training.

  • 3.1 Problem Formulation: WebThinker formalizes generation as reasoning with prior tool outputs followed by final-output generation conditioned on the instruction and query.The reasoning sequence uses tool-call outputs, while the final output is generated from the completed reasoning process.
  • 3.1 Problem Formulation: The framework operates in Problem-Solving Mode and Report Generation Mode, extending reasoning with web exploration and iterative writing capabilities.Problem-Solving Mode targets complex real-world problems; Report Generation Mode supports drafting, checking, and editing during thinking and searching.
  • 3.3 Solving Complex Reasoning Tasks with the Deep Web Explorer: In Problem-Solving Mode, the LRM invokes the Deep Web Explorer during reasoning to search, navigate, and extract information before producing a direct answer.The explorer uses search and navigation tools and returns concise information addressing the main reasoning chain’s knowledge gap.
  • 3.4 Generating Comprehensive Reports via Autonomous Think-Search-and-Draft: In Report Generation Mode, the LRM interleaves reasoning, web exploration, and writing through drafting, checking, and editing tools backed by document memory.An assistant LLM applies editing instructions to the current report using relevant documents retrieved from accumulated web pages.
  • 3.4 Generating Comprehensive Reports via Autonomous Think-Search-and-Draft: The main LRM orchestrates exploration and report modification, terminating the overall process when it generates the EOS token yend.It decides when to invoke the Deep Web Explorer and report-writing tools, while document memory supplies knowledge for writing operations.
  • 3.5 Improving LRMs with Research Tools via Reinforcement Learning: WebThinker trains research-tool usage with iterative online DPO using preference pairs that prioritize correctness or report quality, then tool efficiency and thinking conciseness.The model samples trajectories, constructs preferences, trains on the current set, resamples on-policy trajectories, and repeats with an updated reference policy.

4 Experiments

Experiments evaluate WebThinker on complex reasoning benchmarks and scientific report generation, comparing autonomous exploration, RAG, direct reasoning, and model-size variants. WebThinker consistently improves benchmark performance and report quality, while ablations identify deep exploration, drafting, and iterative refinement as important components.

  • Tasks and evaluation: Experiments cover GPQA, GAIA, WebWalkerQA, HLE, and Glaive, with accuracy or report quality evaluated by specified judge models.Complex reasoning accuracy is judged by Qwen2.5-72B-Instruct, while Glaive reports receive average scores from DeepSeek-R1-671B and GPT-4o.
  • Complex reasoning results: Autonomous search outperforms direct reasoning and basic RAG especially on GAIA, WebWalkerQA, and HLE.Search-o1 records 10.8 on HLE versus 7.2 for RAG-QwQ-32B, while WebThinker-32B-Base surpasses prior methods across benchmarks.
  • Scientific report generation: 8.0 is WebThinker’s top overall Glaive report-generation score, exceeding Gemini-Deep Research’s 7.9.WebThinker scores 8.4 in Completeness, 8.2 in Thoroughness, 7.7 in Factuality, and 7.8 in Coherence.
  • Model-size analysis: 174.4% and 422.6% are WebThinker-R1-7B’s gains over direct generation on GAIA and WebWalkerQA, respectively.Relative gains over standard RAG are 82.9% on GAIA and 161.3% on WebWalkerQA.
  • Ablation studies: Removing the Deep Web Explorer reduces problem-solving performance to 38.3, while removing autonomous drafting reduces report-generation quality to 6.6.Disabling check-and-edit tools lowers coherence from 7.9 to 6.9; iterative online RL raises average performance to 44.9 from 42.1 for Base.

5 Conclusion

WebThinker equips large reasoning models with autonomous web exploration and report generation during reasoning. Its system combines dedicated search, navigation, and report-writing processes, while the paper identifies multimodal support, tool scalability, and GUI interaction as future extensions.

  • Conclusion: WebThinker combines a Deep Web Explorer, an autonomous Think-Search-and-Draft strategy, and RL-based training for research-tool utilization.The framework targets autonomous web exploration and comprehensive report generation through agentic reasoning.
  • Limitations and future work: WebThinker cannot process images or videos, supports only a limited tool set, and does not yet provide GUI-based web exploration.The conclusion identifies multimodal systems, tool scalability and generalization, and GUI support as directions for improvement.
  • System organization: The main reasoning process operates in Problem-Solving Mode and Report Generation Mode, with the Deep Web Explorer handling in-depth information gathering.Report Generation Mode additionally uses Section Writing, Report Checking, and Report Editing tools.
  • Report generation: The final report accumulates section-writing and editing actions performed by an assistant LLM throughout the reasoning loop.The main process continues reasoning after background writing or editing, while checking inserts the report outline into context.
  • Deep Web Explorer: The Explorer iteratively searches and clicks promising links until it gathers sufficient information, then returns final findings to the main reasoning process.Search returns top-10 results, while clicking fetches full page content that an assistant LLM summarizes according to click intent.

B.1 Complex Reasoning Tasks

The complex reasoning evaluation uses four benchmarks spanning expert science questions, general AI tasks, web traversal, and frontier knowledge. Training data additionally combines broad reasoning, mathematical, web-navigation, and graduate-level subject coverage.

  • Evaluation datasets: GPQA, GAIA, WebWalkerQA, and HLE evaluate complementary forms of knowledge-intensive reasoning and information acquisition.The benchmarks cover PhD-level science, general AI assistance, deep web navigation, and extremely challenging cross-disciplinary problems.
  • Evaluation datasets: GPQA’s diamond set contains 198 expert-written multiple-choice questions in biology, physics, and chemistry.The questions are designed to test domain expertise and reasoning.
  • Evaluation datasets: GAIA uses a text-only validation subset of 103 questions covering diverse tasks such as question answering, reasoning, and code generation.The text-only restriction reflects the model’s inability to handle other modalities.
  • Evaluation datasets: WebWalkerQA evaluates website traversal and information extraction with 680 multilingual, multidomain test queries.Its design simulates finding required information through human-like website clicking.
  • Evaluation datasets: HLE contains 2,500 cross-disciplinary questions, and current state-of-the-art models achieve less than 10% accuracy on it.The benchmark includes multiple-choice and short-answer questions with verifiable solutions.
  • Training datasets: Training draws from SuperGPQA, WebWalkerQA Silver, OpenThoughts, NaturalReasoning, and NuminaMath, with preference data filtered from WebThinker trajectories.Approximately 3k data points are sampled, retaining questions that direct QwQ-32B generation cannot answer correctly.

B.2 Scientific Report Generation Tasks

Scientific report generation uses Glaive, a large synthetic reasoning dataset spanning diverse non-code and non-mathematical topics. The evaluation samples a small test set and constructs preference data separately for each training iteration.

  • Dataset: Glaive contains over 22 million synthetic reasoning questions and responses generated using DeepSeek-R1-Distill-Llama-70B.The dataset is used for both training and testing.
  • Dataset: Glaive covers diverse non-code and non-math topics, including social sciences, natural sciences, education, and creative writing.Its broad subject coverage supports open-ended scientific report generation tasks.
  • Sampling and evaluation: Each iteration constructs preference data from 1.5k sampled questions and evaluates reports on 30 test questions.These quantities define the reported sampling and testing setup for the scientific report generation task.

C Instruction Templates

The instruction templates operationalize WebThinker as an iterative research-and-writing assistant: it searches, analyzes results, writes sections, checks the article, and edits it as needed. They also illustrate repeated search and reasoning for multi-step questions.

  • Instruction for WebThinker: An example shows iterative disambiguation: the assistant searches for Alice David’s Lara Croft role, then issues a follow-up query to identify the game’s developer.The assistant continues reasoning after each processed search result and uses the search delimiters consistently.
  • Instruction for Deep Web Explorer: The Deep Web Explorer instructions direct the assistant to inspect searched pages for facts relevant to the current query and seek more information when the results are insufficient.The explorer’s purpose is to support reasoning about the original question rather than merely return search results.
  • Problem Solving Instruction for WebThinker: WebThinker’s problem-solving instructions let the assistant search again, access webpages, and extract information relevant to the current query and search intent.The output begins with a labeled “Final Information” field and includes the current query, detailed search intent, and searched webpages.
  • Report Generation Instruction for WebThinker: The report-generation template gives the assistant search, section-writing, article-checking, and article-editing tools for completing a scientific article.The template also supplies a research question and plan, then directs the assistant to gather information section by section before checking and revising the article.
  • Report Generation Instruction for WebThinker: The workflow interleaves research and drafting: after each search, the assistant evaluates what remains needed, writes sufficiently supported sections, and revises the article for clarity and completeness.The instructions explicitly require continuing until the article is comprehensive, then outputting a completion signal.
  • Instruction for Writing Section: Section-writing instructions request comprehensive, focused markdown based on relevant documents, prior thoughts, the current article outline, and the next section’s goal.They encourage developed paragraphs and, when useful, markdown tables for structured presentation.
  • Instruction for Writing Section: The section template identifies the next section and its goal, asks for focused content without citations, and permits markdown tables when they improve completeness and structure.It also instructs the assistant to write the complete section in markdown format.
  • Instruction for Editing Article: The editing template frames revision as modifying the article according to an explicit edit instruction supplied to a professional article editor.The edit goal is provided as a dedicated input field.

D.2 Case Study for Deep Web Explorer

The case studies present the Deep Web Explorer as effective at retrieving and integrating information that requires following links, consulting documents, and combining specific findings. The report-generation examples extend this process into structured, iterative scientific writing.

  • Deep Web Explorer: The Deep Web Explorer demonstrates effective information retrieval and integration across the reported case studies.The examples cover deadline research, repository exploration, and mosquito-control evidence.
  • Deep Web Explorer: For ASH Annual Meeting deadlines, the explorer follows a PDF link to find ancillary meeting deadlines alongside late-breaking abstract dates and requirements.The result combines information discovered from the initial search and linked document.
  • Deep Web Explorer: For CLTS and Aedes mosquito control, the explorer follows a repository link to locate a case study from Indonesia, Vietnam, and the Philippines reporting a 40% reduction in breeding sites.The linked case study supplies both geographic coverage and a specific outcome.
  • Report Generation: The report-generation examples show WebThinker systematically following a research plan while searching topics relevant to optimizing 3D-printed lattice structures.The searches address FDM limitations, lattice optimization techniques, and material properties.
  • Report Generation: The model writes, checks, and edits article sections to address duplicate content and maintain coherent structure.These operations are presented as part of the report-generation workflow.
  • Report Generation: The final report analyzes lattice optimization for robotic nodes, covering printing limitations, material selection, practical applications, and case studies.This example illustrates the breadth of the completed scientific report.
  • Overall Validation: Together, the examples validate WebThinker across complex problem-solving, deep web exploration, and scientific report generation tasks.The cases are presented as evidence spanning all three capabilities.

E Summary of Contributions

WebThinker combines autonomous reasoning-time web interaction with real-time report drafting and RL-based tool-use training. The paper reports effectiveness on complex reasoning and scientific report-generation tasks using QwQ- and DeepSeek-R1-based backbones.

  • Core Contributions: WebThinker autonomously searches, deeply explores webpages, and drafts research reports within the model’s thinking process.Unlike predefined workflows, the LRM performs these actions itself for end-to-end execution in a single generation.
  • Core Contributions: The Deep Web Explorer gives LRMs web search and navigation capabilities for gathering, traversing, and extracting high-quality information.Its contribution is deeper web exploration rather than reliance on shallow retrieval alone.
  • Core Contributions: The Autonomous Think-Search-and-Draft strategy enables report writing in real time while the LRM reasons and searches.This integrates drafting with the model’s ongoing information-seeking process.
  • Core Contributions: WebThinker uses RL-based training to synthesize tool-usage preference data iteratively and apply online DPO training.The stated goal is to enhance the LRM’s tool-utilization capabilities.
  • Empirical Evaluation: Extensive experiments evaluate WebThinker on complex reasoning and scientific report-generation tasks with QwQ-based and DeepSeek-R1-based LRM backbones.The contribution summary reports effectiveness across both backbone families and task categories.

F Broader Impact

WebThinker is presented as a framework for expanding LRM-based research capabilities and access to synthesized web information, while its deployment raises societal risks requiring safeguards.

  • WebThinker could broaden access to advanced research capabilities and support innovation and learning.
  • The framework is positioned as useful for scientific discovery, investigative journalism, and decision-making in finance and policy.
  • Autonomous web information gathering and synthesis may generate or disseminate sophisticated misinformation and amplify biases in training data and web sources.
  • Potential deployment concerns also include data privacy, verifiability of AI-generated research, and displacement of human expertise in research-intensive roles.
  • The paper calls for ethical guidelines, robust validation mechanisms, and transparency alongside advances in systems like WebThinker.

7. Experiment statistical significance

The paper reports no error bars because of the high computational cost of its experiments, while describing computational resources in detail. It states that it follows the NeurIPS Code of Ethics, discusses societal impacts, and reports no high-risk misuse concerns.

  • Statistical reporting: No error bars are reported because the experiments have high computational cost.
  • Reproducibility: The paper states that computational resources are described in detail in the implementation details section.
  • Ethics: The authors state that the research follows the NeurIPS Code of Ethics.
  • Broader impacts: Potential positive and negative societal impacts are discussed in section F.
  • Safeguards: The authors state that the work does not pose risks requiring safeguards for responsible release.
  • Assets and documentation: The paper states that code, models, and datasets are properly cited, and provides an anonymous GitHub repository for new assets.
  • Human subjects and LLM use: The paper does not involve crowdsourcing or research with human subjects and states that it did not use LLMs for research.
Loading 2504.21776v2…