Source-linked AI summary
REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents
Zheng Chu, Xiao Wang, Jack Hong, Huiming Fan, Yuqi Huang, Yue Yang, Guohai Xu, Chenxiao Zhao, Cheng Xiang, Shengchao Hu, Dongdong Kuang, Ming Liu, Bing Qin, Xing Yu
TL;DR
Deep-search agents need richer supervision because challenging queries require long-horizon interaction, while constructing tasks and collecting tool-heavy trajectories is costly. REDSearcher jointly designs complex task synthesis, mid-training, post-training, and simulated environments for scalable optimization. It reports state-of-the-art performance across text-only and multimodal benchmarks, including an Overall score of 51.3 and a GAIA score of 80.1.
Problem
Deep search is hindered by sparse high-quality supervision, difficult scalable long-horizon task construction, and expensive interaction-heavy rollouts with external tools.
Method
REDSearcher jointly optimizes dual-constrained task synthesis, cost-efficient mid-training, post-training, and a functionally equivalent local simulation environment for text and multimodal search agents.
Results
REDSearcher achieves state-of-the-art performance across text-only and multimodal search benchmarks, reaching an Overall score of 51.3 and a GAIA score of 80.1.
Takeaways & Limitations
The framework supports scalable, controllable, and cost-effective optimization of long-horizon search behavior across text and multimodal settings.
Takeaways & Limitations
The simulated environment assumes API specifications remain consistent with real search APIs and contains all essential direct and intermediate evidence for synthetic queries.
Abstract
from arXiv · showhide
Large language models are transitioning from generalpurpose knowledge engines to realworld problem solvers, yet optimizing them for deep search tasks remains challenging. The central bottleneck lies in the extreme sparsity of highquality search trajectories and reward signals, arising from the difficulty of scalable longhorizon task construction and the high cost of interactionheavy rollouts involving external tool calls. To address these challenges, we propose REDSearcher, a unified framework that codesigns complex task synthesis, midtraining, and posttraining for scalable searchagent optimization. Specifically, REDSearcher introduces the following improvements: (1) We frame task synthesis as a dualconstrained optimization, where task difficulty is precisely governed by graph topology and evidence dispersion, allowing scalable generation of complex, highquality tasks. (2) We introduce toolaugmented queries to encourage proactive tool use rather than passive recall.(3) During midtraining, we strengthen core atomic capabilities knowledge, planning, and function calling substantially reducing the cost of collecting highquality trajectories for downstream training. (4) We build a local simulated environment that enables rapid, lowcost algorithmic iteration for reinforcement learning experiments. Across both textonly and multimodal searchagent benchmarks, our approach achieves stateoftheart performance. To facilitate future research on longhorizon search agents, we will release 10K highquality complex text search trajectories, 5K multimodal trajectories and 1K text RL query set, and together with code and model checkpoints.
1 Introduction
REDSearcher addresses the scarcity of effective supervision for deep search by jointly optimizing task synthesis, mid-training, and post-training across text-only and multimodal settings. Its task and training designs increase search complexity while reducing the cost of developing long-horizon tool-using agents.
- REDSearcher jointly optimizes task synthesis, mid-training, and post-training for scalable training of text-only and multimodal deep-search agents.
- Dual-constrained task synthesis creates structurally complex tasks with cycles, interlocking constraints, and evidence dispersed across disjoint sources.These constraints require competing-hypothesis maintenance, iterative planning, and cross-document synthesis.
- Tool-augmented queries rewrite key facts into operational constraints that require external tools, encouraging proactive tool use instead of passive text retrieval.Examples include map-resolved routing constraints and visual cues requiring image understanding.
- Two-stage mid-training separates atomic subskill acquisition from interactive execution to reduce reliance on costly environment interaction.Synthetic data strengthens grounding and hierarchical planning before simulated tool-use loops and long-horizon trajectories are introduced.
- A lightweight local simulated environment mimics web dynamics without live API latency or expense while preserving solvability and high-interference distractors.Necessary evidence remains inside a closed corpus but is dispersed among distractor documents.
2 Preliminary
REDSearcher models deep search as an interactive agent–environment process in which tool calls yield evidence that updates the agent’s state before a grounded answer is produced. The formalization captures questions, actions, observations, internal state, answers, and long trajectories, while context management addresses input-length constraints.
- Interactive Search Process: A search session consists of a question, iterative tool calls, returned observations, and a final evidence-grounded answer.Questions may be unimodal or multimodal, and observations can include text, images, metadata, and structured tool outputs.
- Core Variables: The agent’s internal state compresses interaction history, current constraints, extracted information, active hypotheses, and intermediate conclusions to select the next action.
- Interactive Search Process: The policy selects tool calls from the transcript context, while the deterministic environment returns observations for issued requests and can include multimodal evidence.For multimodal sessions, the transcript aggregates evidence across text and visual modalities.
- Trajectory Representation: Each trajectory records the question, state–action–observation tuples across steps, and the final answer generated from accumulated evidence.
- Context Management: Discard-all context management is adopted when repeated tool calls, webpages, and notes approach the model’s input-window limit.The passage motivates this strategy by noting that truncating earlier steps can disrupt constraint tracking and degrade long-horizon performance.
3 Scalable Complex Task Synthesis
REDSearcher synthesizes difficult deep-search tasks by jointly controlling reasoning-graph topology and evidence dispersion, then verifies generated instances for challenge and reliability.
- Deep-search tasks require multi-hop reasoning, ambiguity, nonlinear search paths, iterative tool use, and synthesis of fragmented evidence.
- Task complexity is characterized by topological logical complexity and information source dispersion.
- Topological Logical Complexity: Treewidth measures graph coupling: higher k requires larger jointly maintained variable sets and shifts reasoning toward globally constrained verification.
- Distributional Complexity: Minimum Source Dispersion measures the minimum number of documents needed to cover information required by the reasoning graph.
- Dual-Constrained Task Synthesis: Tasks are hardest to shortcut when high-treewidth coupled facts are dispersed across disjoint sources, motivating dual-constrained synthesis.
- Synthesis Pipeline: The synthesis pipeline constructs reasoning graphs with target topology and dispersion, converts them into natural-language queries, and applies staged verification.
3.3 Multimodal Task Synthesis Pipeline
The multimodal synthesis pipeline extends text-only generation with visual evidence while preserving explicit dependencies, controllable difficulty, scalability, and verifiability.
- The multimodal pipeline reuses the text-only skeleton with limited modifications, retaining scalability, controllable difficulty, explicit dependencies, and verifiability.
- A text reasoning DAG becomes a cross-modal reasoning DAG through modality injection, with selected constraints anchored in images.
- Visual-semantic abstraction and modality translation force recognition of visual entities and allow visual evidence to enter at different reasoning positions.
- Vision-aware verification removes instances solvable through text alone, vision alone, or overly successful end-to-end agent rollouts.
- The extension yields visually grounded search tasks requiring coupled perception, reasoning, and retrieval rather than decorative images.
4 Overall Training Recipe
REDSearcher specializes pretrained models for multi-turn web search through mid-training on long-horizon traces followed by post-training for end-to-end agentic behavior.
- Mid-training exposes models to long-horizon search traces and tool-use patterns, while post-training optimizes end-to-end behavior.
5 Agentic Mid-Training via Low-Cost Large-Scale Data Synthesis
Agentic mid-training bridges general pretraining and agent-specific post-training by strengthening atomic skills and interactive tool use with scalable synthetic data and simulated environments.
- Agentic mid-training addresses the gap between pretrained knowledge and agent tasks requiring environmental perception, action, and feedback-driven strategy refinement.
- Large-scale manual annotation and real-world interaction are prohibitively expensive, motivating scalable data synthesis.
- Atomic Capabilities: Grounding extracts intent-relevant information from noisy observations, while hierarchical planning decomposes ambiguous tasks into aligned sub-goals.
- Intent-Anchored Grounding: Synthetic grounding data is generated by extracting factual segments from documents, synthesizing query intents, and incorporating distractors.
- Interactive Execution: Tool-calling data and long-horizon trajectories provide environmental feedback and strengthen agentic capabilities for deep information seeking.
- Low-Cost Data Synthesis: The framework reduces interaction cost by simulating tool protocols and deploying a local web-search environment built from Wikipedia and web-crawl data.
6 Agentic Post-Training
REDSearcher’s agentic post-training combines synthesized trajectories, supervised fine-tuning, and reinforcement learning with verifiable rewards. It also uses a local search environment and curated queries to reduce rollout costs and support rapid experimentation.
- High-quality trajectory synthesis: REDSearcher synthesizes complex deep-search questions at low cost and scale, with difficulty comparable to BrowseComp and no human intervention.The synthesis pipeline exclusively supplies its own synthesized QAs for trajectory synthesis and agentic reinforcement learning.
- High-quality trajectory synthesis: The ReAct workflow iteratively combines agent thoughts, tool actions, and environmental observations until producing a final answer.Synthesis uses a 128K-token maximum context, discarding samples that exceed it.
- Supervised fine-tuning: Trajectory filtering retains correct final answers, removes samples with many failed actions or tool responses, and keeps one diverse trajectory per question.Supervised fine-tuning masks environment observations from gradient updates while applying next-token prediction to the remaining sequence.
- Agentic reinforcement learning: Agentic reinforcement learning uses ReAct interactions with real environments and verifiable rewards, while GRPO updates the policy from group-normalized relative advantages with a clipped objective.For each question, multiple trajectories are sampled and their final rewards are normalized within the group; outcome rewards are binary.
- Functionally equivalent simulation environment: The offline simulated environment supports rapid, low-overhead iteration while preserving API consistency, evidence completeness, and realistic environmental noise.It contains tens of millions of documents from finewiki and cached search and visit results, and supports search, visit, and Python tools; agent-as-verifier curation reduces the RL query-set error rate to 10% of the original.
7 Experiments
Experiments evaluate REDSearcher on text-only and multimodal search benchmarks, compare it with open and proprietary agents, and analyze progressive mid-training, reinforcement learning, tool-use dependence, and search behavior. REDSearcher shows strong benchmark performance, while training analyses indicate improving capability and efficiency across several settings.
- Main Results: REDSearcher achieves an Overall score of 51.3 among 30B open-source agents, exceeding Tongyi DeepResearch-30B at 48.5 and WebSailorV2-30B at 46.0.It also surpasses Claude-4.5-sonnet at 41.1 and OpenAI-o3 at 49.6; on GAIA it scores 80.1 versus GPT-5–Thinking–high at 76.7.
- Ablation of Mid-Training Stages: Across four benchmarks, progressive mid-training improves average performance from 42.81 to 47.39.Stage I improves BrowseComp by +1.87 through intent-anchored grounding and GAIA by +4.13 through hierarchical planning.
- Ablation of Mid-Training Stages: Stage II agentic tool use produces the largest reported gain, improving BrowseComp-ZH by +8.91 through environmental feedback and long-horizon trajectories.The analysis links this stage to maintaining goal consistency and robust execution in deep-search scenarios.
- RL Continues to Advance Model Capabilities: Agentic reinforcement learning raises average evaluation reward from 47.4 to 51.3 (+3.9) and BrowseComp from 39.4 to 42.1 (+2.7).These changes correspond to relative gains of approximately 8.2% and 6.8%, respectively, from a strong SFT checkpoint.
- Decoupling Tool Use from Parametric Knowledge: Tool-enabled evaluation more directly measures deep-search competence because it separates evidence acquisition through tools from direct recall of parametric knowledge.REDSearcher scores lowest in the tool-free regime but improves substantially when tools are enabled, whereas some baselines retain non-trivial tool-free accuracy.
- Multimodal Main Results: On multimodal benchmarks, REDSearcher-MM shows competitive performance on MM-BrowseComp, outperforms the Qwen3-VL-235B agent baseline, and transfers strongly to text-only tasks.Tool-use analysis finds fewer turns on simpler benchmarks, more turns on challenging ones, and over-searching near the 30-turn cutoff; RL reduces turns, especially on simpler benchmarks.
8 Conclusion
REDSearcher presents a scalable framework for training long-horizon deep-search agents across text and multimodal settings. It combines complex task synthesis, cost-efficient training, simulation, and reinforcement learning to support scalable search-agent development.
- REDSearcher targets long-horizon deep-search agents in both text and multimodal settings.
- Dual-constrained task synthesis creates structurally complex tasks with dispersed evidence requiring iterative planning and cross-document synthesis.
- Cost-efficient mid-training separates atomic subskill acquisition from interactive execution to reduce trajectory-collection costs.
- A functionally equivalent simulation environment enables high-throughput trajectory generation without expensive live API calls.
- Trajectory synthesis, supervised fine-tuning, and agentic reinforcement learning are combined to advance search intelligence.
Contributions
The listed contributors include researchers from Harbin Institute of Technology, Xiaohongshu Inc., and Shanghai JiaoTong University. Xiao Wang is identified as project leader and advisor.
- Zheng Chu, Xiao Wang, and Jack Hong are identified as core contributors.
- Huiming Fan, Yuqi Huang, Yue Yang, Guohai Xu, Chenxiao Zhao, Cheng Xiang, Shengchao Hu, Dongdong Kuang, Bing Qin, and Xing Yu are listed as contributors.
- Xiao Wang is identified as the project leader.
- Ming Liu and Xiao Wang are identified as advisors.
- The affiliations are Harbin Institute of Technology, Xiaohongshu Inc., and Shanghai JiaoTong University.
A Implementation Details.
The implementation uses Qwen3-based text and multimodal models, supervised fine-tuning, and GRPO reinforcement learning with specified batch, learning-rate, rollout, and response-length settings.
- Text REDSearcher models are based on Qwen3-30B-A3B and use GRPO for reinforcement learning.
- Text mid-training uses batch sizes 512 and 256 for Stages 1 and 2, while supervised fine-tuning uses batch size 128.
- Text mid-training and supervised fine-tuning learning rates decay from 5e-5 to 1e-6 using linear then cosine schedules.
- Multimodal search uses Qwen3-VL-30B-A3B-Thinking with batch size 128, learning rate 1×10−5, three epochs, AdamW, and cosine decay for supervised fine-tuning.
- Multimodal reinforcement learning uses GRPO with batch size 32, eight rollouts per prompt, KL coefficient 0.0, and a 32,768-token response cap.
B System Prompt
The system prompts configure REDSearcher as a persistent, multi-step research agent that uses search and image-capable tools. They emphasize planning, cross-checking, source evaluation, and structured final responses.
- The agent is instructed to conduct rigorous multi-step research and continue until the user’s query is completely resolved.
- The operating principles require gathering evidence across sources, evaluating source quality, synthesizing findings, and wrapping the final answer in <answer> tags.
- Text tools support web search, webpage visits, Python execution, Google Scholar, and Google Maps queries.
- Multimodal tools support text-image search, visual search, image zooming, and web summarization using selected image indices or regions.
- The prompt directs the agent to plan extensively before function calls and reflect on previous call outcomes.
C Synthetic Data Case
The synthetic-data cases present complex search questions spanning music, healthcare, motorsport, geography, and sports, including multimodal clues and a provided answer.
- Synthetic Data Cases: The cases use long, multi-hop clues to identify entities or events across music, healthcare, motorsport, geography, and sports.Examples include a record-pressing plant, a healthcare-facility incident, a stock-car race, a settlement, and a gridiron athlete.
- Synthetic Data Cases: The music-production case combines geographic, epidemiological, biographical, material, and genre clues to identify a target entity.Its clues connect a plant’s location and founding date with an album, polymer format, artist, and record label.
- Synthetic Data Cases: The examples include multimodal questions whose clues depend on information shown in an image.The motorsport, settlement, and athlete questions each refer to a pictured medieval tower, livery, or sculpture.