SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis

Shuang Sun; Huatong Song; Yuhao Wang; Ruiyang Ren; Jinhao Jiang; Junjie Zhang; Fei Bai; Jia Deng; Wayne Xin Zhao; Zheng Liu; Lei Fang; Zhongyuan Wang; Ji-Rong Wen

Source-linked AI summary

SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis

Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjie Zhang, Fei Bai, Jia Deng, Wayne Xin Zhao, Zheng Liu, Lei Fang, Zhongyuan Wang, Ji-Rong Wen

arXiv:2505.16834v3May 22, 2025cs.CLcs.AIcs.IR

TL;DR

Deep-search systems need high-quality reasoning-and-search trajectories, while existing approaches face data scarcity, simulation mismatch, or costly training. SimpleDeepSearcher uses realistic web-search synthesis and multi-criteria curation to create compact SFT data, and 871 curated samples outperform RL-based baselines across diverse benchmarks. The authors limit validation to 7B and 32B models and relatively simple multi-hop questions.

Problem
Deep-search training lacks high-quality trajectories, while existing QA data omits diverse search-oriented questions and reasoning traces.
Method
SimpleDeepSearcher synthesizes multi-turn trajectories through realistic web searches and applies diversity-aware query sampling plus four-dimensional response curation.
Results
871 curated samples enable SFT to outperform RL-based baselines across diverse in-domain and out-of-domain benchmarks.
Takeaways & Limitations
The results support strategic data engineering and SFT as an efficient pathway for deep-search reasoning.
Takeaways & Limitations
Training and evaluation were limited to 7B and 32B models, while distilled data primarily contained relatively simple multi-hop questions.

Abstract

from arXiv · show

Retrieval-augmented generation (RAG) systems have advanced large language models (LLMs) in complex deep search scenarios requiring multi-step reasoning and iterative information retrieval. However, existing approaches face critical limitations that lack high-quality training trajectories or suffer from the distributional mismatches in simulated environments and prohibitive computational costs for real-world deployment. This paper introduces SimpleDeepSearcher, a lightweight yet effective framework that bridges this gap through strategic data engineering rather than complex training paradigms. Our approach synthesizes high-quality training data by simulating realistic user interactions in live web search environments, coupled with a multi-criteria curation strategy that optimizes the diversity and quality of input and output side. Experiments on five benchmarks across diverse domains demonstrate that SFT on only 871 curated samples yields significant improvements over RL-based baselines. Our work establishes SFT as a viable pathway by systematically addressing the data-scarce bottleneck, offering practical insights for efficient deep search systems. Our code is available at https://github.com/RUCAIBox/SimpleDeepSearcher.

1 Introduction

SimpleDeepSearcher addresses the lack of high-quality deep-search trajectories by synthesizing realistic web-search interactions and curating diverse, efficient reasoning data for SFT. With 871 curated samples, it improves performance over prompt-based and RL-based baselines across representative benchmarks.

Deep search requires models to decide when and how to retrieve information and how to reason over retrieved content.
Existing deep-search training lacks high-quality trajectories containing search operations, evidence synthesis, and efficient decision paths.
SimpleDeepSearcher synthesizes multi-turn reasoning trajectories in real web environments and curates both query and response quality.
Compared with prompt-based methods, SimpleDeepSearcher improves performance by 48.3%, while comparison with RL-based RAG methods shows a 24.9% improvement.
871 curated samples enable SFT to outperform strong baselines, especially RL-based methods, on in-domain and out-of-domain benchmarks.

2 Method

SimpleDeepSearcher constructs deep-search training data through realistic web interactions, diversity-aware query selection, and multi-criteria response filtering. It then uses the curated trajectories for efficient supervised fine-tuning.

Data synthesis: SimpleDeepSearcher replaces static retrieval with iterative real-time web interactions that reason, search, summarize, and generate answers.The pipeline processes raw HTML through commercial search APIs and samples multiple reasoning paths per query.
Framework design: The resulting architecture emphasizes authentic search artifacts, small-data SFT efficiency, and flexibility through decoupled data synthesis and model constraints.Its stated objective is to reduce computational demands through data-quality optimization rather than brute-force quantity.
Motivation: The method targets weaknesses in static RAG corpora and naively scaled QA datasets, including stale information, domain imbalance, repetitive syntax, and limited diversity.Static corpora contain refined segments and lack timeliness, while naive scaling can yield diminishing SFT returns.
Data synthesis: The framework synthesizes scalable training trajectories from authentic web scenarios, enriching samples with diverse information and reasoning-search interactions.Real web environments expose structured snippets and unstructured discourse while supporting trajectory scaling.
Query sampling: Query selection balances domain heterogeneity, keyword diversity, and knowledge-unit complexity to reduce skew and promote varied reasoning demands.Queries are clustered by domain, ranked by interrogative terms, and selected with a greedy keyword-diversity procedure.
Response curation: Because synthesized reasoning quality varies, the framework filters responses for formatting and content quality to reduce noisy training signals.The filtering protocol evaluates multiple responses per query and retains outputs satisfying predefined criteria.

3 Experiments

Experiments evaluate SimpleDeepSearcher across in-domain, out-of-domain, and challenging agent-oriented benchmarks using F1 and LLM-as-Judge metrics. The method consistently outperforms baselines, including reinforcement-learning systems, while using only 871 training examples and generalizing across model backbones.

Experimental setup: Training uses curated single-hop and multi-hop QA data, while external retrieval documents are masked during SFT to avoid learning noisy or spurious information.The implementation uses 10 search calls, 15 reasoning turns, and 10 candidate responses per query during synthesis.
Evaluation setup: Evaluations use F1 for word-level answer similarity and LLM-as-Judge for predicted-response correctness across multiple QA benchmark settings.The benchmark suite includes single-hop, multi-hop, out-of-domain, and more challenging deep-search datasets.
Main results: SimpleDeepSearcher outperforms all existing baselines across five representative benchmarks, including both in-domain and out-of-domain datasets.The reported gains span 2Wiki and MuSiQue in-domain tasks as well as Bamboogle, FRAMES, and GAIA out-of-domain tasks.
Main results: SimpleDeepSearcher surpasses reinforcement-learning baselines across most evaluation metrics while using only 871 supervised fine-tuning examples.The comparison includes R1-Searcher and DeepResearcher, which are trained with large-scale datasets and complex reinforcement learning algorithms.
Backbone generalization: 48.3%, 42.6%, and 11.5% are the relative improvements over Search-o1 on Qwen2.5-7B-Instruct, DeepSeek-R1-Distill-Qwen-2.5-32B, and QwQ-32B, respectively.The improvements remain stable across model backbones ranging from 7B to 32B parameters.
Challenging benchmarks: The model remains competitive with reinforcement-learning systems on Xbench-DeepSearch, BrowseComp-ZH, and BrowseComp-EN, which require end-to-end planning, search, reasoning, and summarization.These challenging benchmarks are evaluated with LLM-as-Judge.

4 Further Analysis

Further analyses show that curated SFT improves search reasoning, reduces redundant behavior, and outperforms or complements reinforcement-learning approaches across evaluated settings.

Ablation Study: All ablated variants decline in performance, indicating that query sampling, online synthesis, and response curation each contribute to the full method.The ablations remove domain, keyword, interrogative-word, environment, formatting, reasoning-path, difficulty, and search-count filters.
Effect of Post-SFT RL: DPO further improves the SFT baseline, whereas REINFORCE++ shortens responses and degrades performance on Bamboogle and GAIA.The authors suggest online RL mainly triggers retrieval behavior, while offline DPO benefits from high-quality preference trajectories.
Effect of SFT on Redundancy: SFT significantly reduces reflective-word use, output length, and search-query counts, indicating clearer reasoning and more efficient search behavior.The analysis uses QwQ-32B on 2Wiki, MuSiQue, and Bamboogle.
Stage-wise Performance: Training improves all inference components, with search showing the largest gain in the proportion of cases containing the final answer.The evaluated stages are planning, search, and summarization.
Summarization Model: QwQ-32B is selected for summarization, while fine-tuned summarizers degrade downstream performance relative to pretrained summarizers.The comparison fixes the reasoning model and varies the summarization model.
Reasoning Data: Adding complex mathematical reasoning data consistently improves performance across benchmarks and changes behavior according to task demands.Complex-reasoning tasks elicit longer outputs, while search-task behavior differs accordingly.

5 Conclusion

The paper presents SimpleDeepSearcher as a lightweight deep-search framework built from realistic web simulations and curated training trajectories. With 871 samples, it reports gains over RL-based baselines, while acknowledging limits in model scale and multi-hop data complexity.

Conclusion: SimpleDeepSearcher uses realistic web-search simulations and multi-criteria curation to construct training trajectories for efficient supervised fine-tuning.The framework addresses methods relying on complex training paradigms or distributionally mismatched simulations.
Conclusion: 871 curated samples suffice for substantial gains over RL-based baselines across diverse in-domain and out-of-domain benchmarks.The conclusion frames strategic data engineering as a route to efficient deep-search reasoning.
Limitations: Experiments are limited to 7B and 32B models because of training-resource and hardware constraints, leaving larger-scale generalization for future work.The authors specifically identify 72B training and evaluation as future work.
Limitations: The distillation data primarily contains relatively simple multi-hop questions because realistic challenging multi-hop synthesis remains difficult.The authors expect more realistic multi-hop queries could improve future filtering and training.

A Related Work

Related work has progressively integrated retrieval with language-model reasoning, moving from fixed RAG pipelines and prompt engineering toward models that search dynamically during multi-step reasoning.

Retrieval-Augmented LLMs: RAG methods improve factual precision by augmenting language models with retrieved knowledge through branching, summarization, and adaptive-retrieval approaches.These methods establish retrieval as a mechanism for improving generated text.
Retrieval-Augmented LLMs: Prompt-based systems combine chain-of-thought reasoning with retrieval, but fixed reasoning paths limit the model’s capacity.Other work distills retrieval abilities into smaller models through supervised fine-tuning.
Enhancing LLMs with Search: Deep-search frameworks integrate language models with search engines dynamically, enabling independent retrieval during multi-step reasoning.Search-o1 uses prompt engineering, while R1-Searcher and Search-R1 equip models with retrieval tools and train them accordingly.

B Details of Diversity-Aware Query Sampling

The query-sampling analysis identifies distributional, syntactic, and semantic weaknesses in open-source data, then motivates diversity-aware selection based on core semantic constituents.

Data Limitations: Pre-filtered data overrepresents domains such as film and geography, risking uneven knowledge distributions in training data.Figure 3 presents the domain distribution before filtering.
Data Limitations: Repetitive comparative syntax reduces linguistic variability across open-source queries.Repeated “A and B” structures and recurring attribute comparisons illustrate the redundancy.
Data Limitations: Many open-source queries are semantically simple enough for a single lookup, limiting their value for developing advanced search and reasoning capabilities.The paper gives “What nationality is John Harbaugh’s father?” as an example.
Diversity-Aware Query Sampling: The sampling strategy defines core semantic constituents as key entities, critical attributes, core relationships, and measurement dimensions.These constituents support extracting concepts such as “film” and “age” from comparative questions.

C DPO Detailed Settings

The DPO dataset pairs preferred and rejected answer trajectories, targeting correctness and efficient reasoning and search paths. It contains approximately 875 training pairs, generated from two SFT-trained models and filtered by formatting and reasoning-path checks.

Approximately 875 training pairs were constructed for DPO training.
Preferred trajectories came from SDS-QwQ-32B-SFT, while rejected trajectories came from SDS-Qwen-7B-SFT when checks failed.Positive examples passed formatting and reasoning path control checks; negative examples failed these checks.
DPO used a learning rate of 5 × 10−7, β of 0.1, 5 epochs, batch size 256, warm-up ratio 0.1, and maximum sequence length 10000.

D REINFORCE++ Detailed Settings

The REINFORCE++ setup uses rollout-generated responses to build a diverse RL dataset and rewards answer accuracy while penalizing undesirable formatting and reasoning behaviors. On-policy training is maintained by using recent policy rollouts, with response length and reward tracked during training.

2,480 samples from 2Wiki and HotpotQA formed the RL dataset after selecting questions with one to six correct answers.Eight candidate responses were generated for each question to support diversity.
The reward combines an answer reward based on F1 with a discrete format penalty of −2 for specified undesirable behaviors.
The format penalty covers self-retrieved content, gibberish, excessive analytical markers, and missing boxed answers or excessive reasoning length.
More than 8 retrieval steps or over 8,096 analytical tokens between retrievals triggers the excessive-length condition.
No penalty is applied when none of the listed conditions occurs.
Batch size was adjusted to keep training on-policy using recent rollouts, while Figure 4 tracks response-length and reward variations.

E Model Performance Enhancement Analysis Settings

Performance comparisons evaluate Qwe2.5-7B-Instruct and QwQ-32B before and after training on three benchmarks, while fixing QwQ-32B as the summarization model. This controls variation from the summarization component across comparisons.

Qwe2.5-7B-Instruct and QwQ-32B were compared before and after training on 2Wiki, Bamboogle, and MuSiQue.
QwQ-32B was fixed as the summarization model during inference across all comparisons.The summarization component’s impact is discussed separately in Section 4.5.

F Instruction Templates

The instruction templates specify structured semantic analysis, answer judging, web-search reasoning, and search-result summarization. They require concise formatted outputs, iterative searching within a limit, and integration of retrieved information into ongoing reasoning.

Instruction for Annotation: The semantic analyzer identifies a broad domain and 2–4 core semantic components needed to answer each question.
Instruction for Annotation: Key components include entities, critical attributes such as age or duration, and relationships such as comparison or causality.
Instruction for Annotation: Annotation outputs use lowercase JSON with concise 1–2-word key points, excluding filler words and separating multiple points with commas.
Instruction for LLM as Judge: The judge returns True when a prediction fully matches the golden answer’s meaning and key information, including any one of multiple valid options.
Instruction for Reasoning Model: The reasoning model issues web searches through begin-search and end-search tags, then continues reasoning without generating search-result tags itself.
Instruction for Reasoning Model: It searches whenever information is uncertain or needs updating, may repeat searches, and obeys a maximum search-attempt limit.
Instruction for Summarization Model: The summarization model extracts accurate, relevant information from searched pages for the current query and integrates it with previous reasoning steps.
Instruction for Summarization Model: Its analysis reviews each searched page and selects information that advances the current query and original reasoning process.