PaperlayerView exact revision on arXiv

Source-linked AI summary

QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks

Jian Xie, Tianhe Lin, Zilu Wang, Yuting Ning, Yuekun Yao, Tianci Xue, Zhehao Zhang, Zhongyang Li, Kai Zhang, Yufan Wu, Shijie Chen, Boyu Gou, Mingzhe Han, Yifei Wang, Vint Lee, Xinpeng Wei, Xiangjun Wang, Yu Su, Huan Sun

arXiv:2605.24218v1cs.CL

TL;DR

Existing open deep research agents generalize poorly across task types, leaving unclear how to train broadly capable systems. QUEST combines rubric-tree data synthesis, context management, and staged training to build open general-purpose agents. QUEST-35B achieves the best overall performance among recent open-weight agents and approaches or surpasses closed-source agents across eight benchmarks.

  • Problem

    Existing open deep research approaches focus on specific scenarios and inadequately cover the diverse capabilities and evaluation regimes required for general-purpose agents.

  • Method

    QUEST combines rubric-tree-based synthetic data, structured context management, and staged mid-training, supervised fine-tuning, and reinforcement learning.

  • Results

    QUEST-35B achieves the best overall performance among recent open-weight agents and approaches or surpasses closed-source agents across eight diverse deep research benchmarks.

  • Takeaways & Limitations

    QUEST provides an open, reproducible foundation for developing general-purpose deep research agents across fact seeking, report synthesis, and citation grounding.

  • Takeaways & Limitations

    Direct policy optimization for long-form report comparison yields limited gains because preference-pair construction is costly and training is unstable and prone to overfitting.

Abstract

from arXiv · show

Deep research agents extend the role of search engines from retrieving keyword-matched pages to synthesizing knowledge, fundamentally changing how humans interact with information. However, frontier systems remain proprietary, while existing open agents often generalize poorly across different task types, leaving unclear how to train a broadly capable deep research agent. We release QUEST, a family of open models (ranging from 2B to 35B) that serve as general-purpose deep research agents designed to handle a wide range of long-horizon search tasks, with strong capabilities in fact seeking, citation grounding, and report synthesis. To build QUEST, we propose an effective training recipe combining mid-training, supervised fine-tuning, and reinforcement learning. Central to this recipe is a curated data synthesis pipeline based on unified rubric trees, which applies to different task types and enables synthesizing training data with verifiable rewards without human annotation. In addition, QUEST incorporates a built-in context management mechanism that enables effective long-horizon reasoning and knowledge synthesis. Using only 8K synthesized tasks, QUEST approaches or even surpasses frontier closed-source agents across eight deep research benchmarks spanning diverse task types, and achieves the best overall performance among recent open-weight agents. We released everything: models, data, and training scripts.

Data Synthesis · Training

QUEST presents a fully open training recipe for deep research agents, covering fact seeking, citation grounding, and report synthesis to support reproducible training.

  • Data Synthesis: QUEST covers fact-seeking capabilities for deep research agents.The recipe is presented as part of a comparison of training recipes.
  • Data Synthesis: QUEST covers citation-grounding capabilities for deep research agents.Citation grounding is one of the capabilities included in the training-recipe comparison.
  • Data Synthesis: QUEST covers report-synthesis capabilities for deep research agents.Report synthesis is included alongside fact seeking and citation grounding.
  • Training: QUEST provides a fully open recipe for training deep research agents.The recipe is described as fully open in the comparison of training approaches.
  • Training: The training recipe is intended to support reproducible training.Reproducibility is identified as a purpose of providing the fully open recipe.
  • Training: Figure 2 compares QUEST’s training recipe with recipes for other deep research agents.The figure explicitly frames the material as a comparison of training recipes.

1 Introduction

QUEST is introduced as a family of open, general-purpose deep research agents addressing the limited generality of existing open-weight approaches. Its training recipe combines synthetic rubric-based data, context management for long-horizon search, and rubric-based optimization.

  • Motivation: Web search is shifting from manual information gathering toward autonomous evidence-seeking and synthesis through traditional search, RAG, and deep research agents.Traditional search engines return ranked webpages, RAG systems condition language-model responses on retrieved documents, and deep research agents extend this paradigm further.
  • Motivation: Existing open-weight deep research agents often target specific scenarios and fail to capture the capabilities required for general-purpose agents.Deep research tasks also differ in how response quality can be assessed, creating distinct correctness regimes.
  • Technical contributions: QUEST combines a scalable synthetic-data pipeline, compact context summaries for long-horizon trajectories, and rubric-based training signals.The pipeline produces Quest-8K, pairing complex queries with task-specific, verifiable rubric trees; training includes mid-training and supervised fine-tuning.

2 Data Synthesis

Quest addresses the limits of single-answer supervision by synthesizing diverse objective and open-ended deep-research tasks with automatically generated, verified rubric trees. The rubric-centric pipeline converts responses into fine-grained, automated scores for scalable training and evaluation.

  • Motivation: Single-answer supervision mainly targets fact seeking with short answers, but general-purpose deep research also requires multi-source aggregation and coherent, source-supported outputs.This motivates training data that extends beyond complex questions with a single verifiable answer.
  • Unified rubric-tree framework: Quest automatically generates, refines, and verifies hierarchical rubric trees, replacing human-written tasks and human-refined evaluation scripts for large-scale synthesis.Rubric trees decompose the constraints that valid answers must satisfy.
  • Unified rubric-tree framework: Rubric trees represent task-specific constraints rather than unique answers, supporting unique-answer, multi-solution, and open-ended tasks while providing fine-grained partial-score signals.Each instance contains a complex query, a query-specific rubric tree, and an evaluation protocol mapping responses to scores in [0, 1].
  • Objective Tasks: Objective-task synthesis uses trending keywords as topical seeds, web exploration by Gsyn, and verifiable constraints organized into rubric trees.The process is designed to keep tasks topically relevant and temporally diverse while reflecting real-world user needs.
  • Open-ended Tasks: Open-ended tasks share four root criteria—instruction following, comprehensiveness, readability, and insight—with adaptive task-specific nodes and evaluation using reference reports, rubric judges, and pairwise normalization.The judge compares candidate and reference reports and scores candidates under task-specific rubric nodes.

3 Context Management

Quest manages long-horizon research context with a structured Context State that organizes accumulated knowledge and is updated when context usage exceeds a threshold. This addresses attention degradation as raw search results, visited pages, and reasoning traces accumulate during multi-turn tool use.

  • Motivation: As raw search results, visited pages, and intermediate reasoning traces accumulate, attention to important information degrades during multi-turn search, reading, and revision.Existing open-weight agents address this by capping turns or relying on a sufficiently large context window.
  • Context State: Quest stores research history in a structured JSON object called Context State, organizing accumulated knowledge into three buckets.The three buckets are trusted, untrusted, and uncertain entries.
  • Context State: Trusted entries contain facts checked against retrieved source URLs, while untrusted and uncertain entries preserve distrust reasons or follow-up instructions.Uncertain claims include a URL to visit or a query to rerun.
  • Context State: This structured state encodes the agent’s epistemic state, enabling a resumed agent to distinguish among previously checked, contradicted, and partially supported claims.Trusted facts can be reused without additional verification; uncertain claims require follow-up.
  • Condensation: Context management triggers when context-window usage exceeds a threshold, and a Context Condenser model processes the full raw history to produce an updated Context State.The raw history includes search queries and results, visited URLs with extracted content, reasoning traces, and previously summarized memory.

4 Training

QUEST training combines mid-training, supervised fine-tuning, and reinforcement learning, with each stage targeting complementary capabilities from long-horizon interaction to tool-use imitation and reward optimization. The pipeline reuses synthesized trajectories and supervision while training reinforcement learning with rubric-based and fact-checking rewards.

  • Training pipeline: Three stages—MT, SFT, and RL—respectively adapt the model to long-horizon search, imitate tool-use trajectories, and optimize behavior with rubric-based and fact-checking rewards.Each stage addresses a complementary aspect of deep research agent behavior.
  • Supervised fine-tuning: SFT uses teacher-generated full-agent trajectories, filtered by query-specific evaluation scores, and trains on session-level instances with next-token prediction.Tongyi DeepResearch generates trajectories, while GPT-5.2 polishes final reports for open-ended tasks; unsuccessful objective-task trajectories can be retried using evaluation feedback.
  • Mid-training: MT trains context summarization and relevant information extraction to provide long-context understanding and awareness of the Context State structure.Both auxiliary tasks reuse supervision generated during data synthesis, requiring no additional data collection.
  • Reinforcement learning: RL applies GRPO-style outcome-based tuning with a reward combining rubric-tree evaluation and fact-checking of inline citations.Fact-checking rewards measure the fraction of supported citations among citations with determinate labels, encouraging responses grounded in verifiable sources.
  • Reinforcement learning: The final reward caps fact-checking contribution by the rubric-tree reward, preventing well-cited but poorly completed or entirely incorrect responses from receiving inflated rewards.Rollout rewards are propagated to all derived sessions, with advantages normalized across unique rollouts in each group.

5 Infrastructure

Quest’s infrastructure combines external research tools with caching and asynchronous execution to support large-scale data synthesis, training, rollout, and evaluation. The system reduces redundant live web operations and decouples evaluation, rollout, and training for more efficient processing.

  • Tooling: Quest provides Google Search, Visit, Python Interpreter, and Google Scholar for retrieval, webpage reading, computation, and academic publication access.Visit summarizes webpages conditioned on the goal of extracting key information from HTML.
  • Caching: A dual-cache system intercepts search and visit operations before live APIs, addressing the cost and latency of hundreds of parallel queries and URL visits.The cache is used during data synthesis, supervised fine-tuning trajectory collection, and reinforcement-learning rollout.
  • Caching: Search and Scholar Cache persist query results, returning exact-match hits immediately and avoiding repeated API calls.New queries are checked against the persistent cache before being sent onward for retrieval or live search.
  • Caching: Visit Cache uses the full URL as its key, returning stored page content on hits and fetching and caching live pages on misses.This eliminates redundant retrieval of the same page across different training runs, with Jina serving as the live API.
  • Asynchronous execution: Quest extends VERL’s asynchronous policy optimization by overlapping rollout and training and assigning task-specific evaluation to asynchronous Ray actors.The extension makes evaluation fully asynchronous rather than only decoupling the Rollouter and Trainer.

6 Experiments

QUEST is evaluated across eight diverse deep research benchmarks and compared with both proprietary and open-weight agents. Quest-35B achieves leading open-weight performance, while ablations and scaling experiments show benefits from the full training recipe and robustness across model sizes.

  • Benchmark Evaluation: Quest is evaluated on eight benchmarks spanning objective and open-ended deep research tasks, using full test sets except text-only HLE and GAIA evaluations.The benchmarks include BrowseComp, Mind2Web 2, Humanity’s Last Exam, BrowseComp-Plus, WideSearch, GAIA, DeepResearch Bench, and LiveResearchBench.
  • Main Results: Quest-35B establishes new state-of-the-art performance among similarly scaled open-weight agents and matches or slightly surpasses proprietary agents on several benchmarks.It scores 48.2% versus OpenAI-DR’s 47.0% on DeepResearch Bench and 30.7% versus 28.0% on Mind2Web 2.
  • Main Results: At the 30B scale, Quest-30B achieves the best performance on 4 of 8 benchmarks, including Mind2Web 2 and DeepResearch Bench, while Tongyi-DR leads BrowseComp, HLE, and GAIA.The results distinguish citation grounding and report synthesis tasks from benchmarks relying heavily on fact seeking.
  • Training Ablations: The full MT+SFT+RL recipe yields the best combined performance across the analyzed training-stage variants, with each stage affecting benchmarks differently.SFT improves most objective benchmarks but can hurt open-ended performance and BC-Plus, while MT provides additional overall gains; BC-Plus is affected by tool-use overfitting.
  • Model Scaling: A 2B Quest model remains competitive on fact-seeking benchmarks such as HLE and GAIA, although performance generally improves with larger models.The scaling study reports no substantial degradation on most benchmarks when downscaling model size and attributes this robustness to the synthetic data.

7 Unsuccessful Attempts

Preliminary experiments identified several training objectives and reward-scoring schemes that failed to improve performance or produced unusable signals. These failures motivated the final pairwise scoring design and informed the training recipe.

  • Training objectives: Adding search-result prediction to the mid-training mixture consistently hurt overall performance, despite improving some search-heavy tasks when used alone.The authors hypothesize interference with context-related objectives, including Context Summarization.
  • Training objectives: A rubric-based objective for predicting potential answer issues yielded only marginal improvements in preliminary experiments.The authors hypothesize that the limitation stems from lacking access to external information.
  • Training objectives: Pairwise report optimization following DPO did not improve results, while constructing reliable preference pairs incurred additional overhead.The approach compared candidate reports using rubric-tree scores and trained the policy to favor higher-quality outputs.
  • Reward scoring: Approximately 1 in ∼50% of cases received a pointwise score near 1, causing severe score inflation that obscured response-quality differences.The three-tier rubric scale was Not Satisfied (0), Partially Satisfied (0.5), and Satisfied (1).
  • Reward scoring: The ternary teacher-referenced scheme assigned “lose” in nearly all cases because the early model underperformed the teacher, collapsing scores toward zero and making the signal unusable.This rendered the reward unusable for both SFT filtering and RL training.

8 Conclusion … A.1 Task Synthesis

QUEST concludes with an open family of general-purpose deep research agents and a reproducible training recipe, while its appendix details data processing and strict task filtering for reliable evaluation trees.

  • 8 Conclusion: Quest-35B achieves the best overall performance among recent open-weight agents and approaches or surpasses closed-source agents across eight benchmarks.This is the paper’s reported overall performance claim for the largest model.
  • 8 Conclusion: The central contribution is an open, reproducible recipe based on rubric-tree data synthesis and structured context management.The conclusion identifies these components as contributions beyond the models themselves.
  • Appendix: The appendix introduces how the data are processed before presenting construction details.The provided appendix passage serves as a transition into data processing.
  • A Data Construction Details: The data construction details describe a strict task filtering strategy intended to ensure generated tasks are high-quality.The filtering strategy is presented in the task synthesis subsection.
  • A.1 Task Synthesis: The filtering strategy also aims to ensure that evaluation trees are reliable.The passage links reliable evaluation trees to the strict filtering policy.
  • A.1 Task Synthesis: The strict filtering policy leads to a low retention rate, with remaining tasks reported after each stage in Table 4.Table 4 is titled “Number of retained tasks after each filtering stage.”

A.2 Manual Examination of the Python Evaluation Scripts for Objective Tasks … A.5 Reinforcement Learning

The appendix validates generated objective-task evaluators through manual examination, describes mid-training data construction, and specifies rollout settings and reinforcement-learning task selection. Together, these sections report mostly executable scripts, filtered summarization and extraction data, and 864 objective RL tasks.

  • A.2 Manual Examination of the Python Evaluation Scripts for Objective Tasks: 50 tasks were manually evaluated by four computer-science graduate students using task descriptions, rubric trees, and detailed evaluation instructions.The instructions covered rubric organization, assessment objectives, evaluation strategies, and core toolkit functionality.
  • A.2 Manual Examination of the Python Evaluation Scripts for Objective Tasks: 2 of 50 sampled tasks had non-executable scripts because runtime exceptions or logical breakdowns prevented normal pipeline execution.
  • A.2 Manual Examination of the Python Evaluation Scripts for Objective Tasks: 6 generated scripts contained rubric-related errors, including redundant rubric nodes and verification based on literal final-answer content rather than the required authentic criterion.
  • A.2 Manual Examination of the Python Evaluation Scripts for Objective Tasks: Most generated scripts interpreted task requirements accurately, implemented rubric rules faithfully, and ran without errors for automated evaluation at scale.
  • A.3 Mid-Training: 300K context-summarization instances were retained after filtering API-error and implementation-bug cases while excluding benchmark-evaluation data to avoid leakage.The data came from SFT distillation and preliminary RL experiments.
  • A.3 Mid-Training: 4 million raw extraction triplets initially came from the Visit Tool cache, after which repetitive instances were filtered to improve training efficiency and reduce redundancy.Each triplet contained raw HTML content, an extraction goal, and extracted content.
  • A.4 Supervised Fine-tuning: Rejection sampling used five rollouts per task, reflection-based retry used three additional rollouts after failure, and each rollout allowed 100 tool turns with a 16K context threshold.
  • A.5 Reinforcement Learning: 864 objective RL tasks were reserved because Gtraj failed to complete them in every rollout, while open-ended data was manually divided into SFT and RL sets.

B Session-level Training … F Evaluation Subset

QUEST trains long-horizon research agents by decomposing trajectories into sessions and maintaining structured context, enabling efficient training and extrapolation beyond native context limits. The paper also specifies model fallbacks, training configurations, and cost-conscious evaluation subsets.

  • B Session-level Training: Trajectories exceeding 200K tokens can be trained using only 16 H100 GPUs by decomposing them into shorter sessions.Session decomposition reduces peak GPU memory usage and allows training to scale to longer trajectories.
  • B Session-level Training: Session-level training combined with context management enables extrapolation to arbitrarily long research sessions beyond training and native context lengths.Variable-length structured decomposition decouples per-session training context from overall trajectory length.
  • C.1 Why Context Condenser instead of Discard-All or Keep-Last-N: The Context Condenser preserves structured research state instead of treating all historical information as equally disposable.Discard-all and keep-last-N cannot distinguish verified facts from speculative claims or completed sub-goals from pending ones.
  • C.1 Why Context Condenser instead of Discard-All or Keep-Last-N: Discard-all forces rediscovery, while keep-last-N loses critical information outside its retention window and both discard structured knowledge about the task state.These limitations are especially problematic for training under constrained context budgets such as 32K.
  • D Model Configuration: Fallback DeepSeek V3.2 models support the context condenser and visit summarization modules, with a fallback trigger rate of around 3%.The mechanism prevents RPM limits from interrupting training and helps keep RL training stable.
  • E.1 Mid-Training: Mid-training runs for one epoch with full-parameter optimization on 32 H100 GPUs for approximately 5 days, using BF16, a 24,576-token context, and learning rate 3×10−6.The configuration uses per-device batch size 2, gradient accumulation 8, and a constant scheduler.
  • E.2 Supervised Fine-Tuning: Supervised fine-tuning uses full-parameter optimization for 3 epochs on 32 H100 GPUs and takes approximately 1 day.It uses BF16, learning rate 3 × 10−6, a 32,768-token maximum sequence length, per-device batch size 1, and gradient accumulation 8.
  • E.3 Reinforcement Learning: The asynchronous VERL-based RL pipeline uses GRPO, actor learning rate 1 × 10−6, global batch size 64, and 8 sampled responses per prompt.The global batch is divided into four PPO mini-batches, with policy synchronization after all four actor updates.

G Evaluation Details of the Vanilla Model … J.1 Objective Task Examples

The appendix details Quest’s evaluation reproduction, runtime inference prompts, and synthetic-task construction procedures. It specifies context-state management, evidence-grounded objective-task generation, non-exclusive rubric design, and complexity controls for generated research tasks.

  • G Evaluation Details of the Vanilla Model: Quest failed to reproduce Qwen3.5’s reported HLE (w/ tool) and Browsecomp scores despite matching its disclosed context strategy, tool-calling format, and tool-set.The authors consistently obtained lower scores under this attempted reproduction.
  • H Inference Prompts: Quest’s runtime agent prompt defines four tools, consumes condenser-generated prev_state JSON, and requires <tool_call>{...}</tool_call> or <answer>...</answer> outputs.The tools are search, visit, google_scholar, and PythonInterpreter; the current date is appended at runtime.
  • H Inference Prompts: The context condenser maintains trusted, untrusted, and uncertain claims while tracking visited sources and search queries to prevent redundant searches.It is invoked automatically at CONTEXT_THRESHOLD and applies deduplication and bucket-migration rules.
  • H Inference Prompts: The condenser extracts concrete facts from visited pages into information_state, preserving verified claims for direct answers and exact next actions for unresolved claims.Visited URLs alone are explicitly treated as insufficient memory.
  • I Task Synthesis Prompts: Objective-task synthesis researches multiple sources, extracts verifiable constraints, builds a critical/non-critical rubric tree with sequential/parallel nodes, and produces a grounded solution.The process also proposes a question reflecting the extracted constraints, with nine breadth-depth complexity classes C1–C9 shaping the rubric tree.
  • I Task Synthesis Prompts: Rubrics must evaluate general property-based criteria rather than force one entity or value when multiple valid answers are possible or evidence is limited.The rubric cannot introduce requirements beyond extracted constraints and must remain traceable to them.
  • I Task Synthesis Prompts: Critical nodes represent mandatory criteria with no partial credit on failure, whereas non-critical nodes support partial credit for independently evaluated requirements.Intermediate nodes use sequential or parallel aggregation according to whether child criteria have order dependencies.
  • J Task Examples: The task generator controls rubric complexity through breadth levels B1–B3 and depth levels D1–D3, yielding nine complexity regimes C1–C9.Breadth ranges from 1–3 nodes to 12 or more, while depth ranges from 2 to 5–6 layers.

J.2 Open-Ended Task Examples · K Related Work

The paper illustrates open-ended evaluation with adaptive rubric trees across analytical and structured how-to tasks, then situates deep research agents and benchmarks within prior work. The examples show how rubric weights shift with task demands, while related work distinguishes long-horizon agentic search from narrower multi-hop question answering.

  • J.2 Open-Ended Task Examples: Open-ended tasks lack a single correct answer and are evaluated by comparing a candidate response with a Gsyn-produced reference report.Their rubric retains four shared dimensions while adapting third-layer sub-criteria to the question.
  • J.2 Open-Ended Task Examples: The open-ended rubric uses instruction following, comprehensiveness, readability, and insight as shared second-layer dimensions, with task-specific third-layer criteria.Leaf scores are aggregated within dimensions, then combined using dimension weights into a scalar pairwise-comparison score.
  • J.2 Open-Ended Task Examples: For the Apple iPhone Fold strategic analysis, insight receives the highest rubric weight at 0.34 because the task emphasizes multifaceted evaluation and analytical argument.The question concerns late entry, technical innovation, pricing, historical precedents, and broader foldable-market adoption.
  • J.2 Open-Ended Task Examples: For the MSCI World Index investment guide, comprehensiveness receives weight 0.34 and instruction following weight 0.27, reflecting four prescribed sections and an expository goal.Insight falls to 0.23, while the comparison specifies 0.30 for the Apple example’s comprehensiveness and 0.24 for instruction following.
  • K Related Work: Deep research agents extend language models with web search for complex, knowledge-intensive tasks in domains including finance and science.The related-work discussion names OpenAI DeepResearch, Gemini DeepResearch, Claude Research, and Kimi Researcher among developed systems.
  • K Related Work: Traditional multi-hop benchmarks such as HotpotQA and 2WikiMultiHopQA measure evidence seeking and multi-hop reasoning, typically producing short answers.Recent deep research benchmarks broaden evaluation toward long-horizon agentic search, including BrowseComp and BrowseComp-Plus.
Loading 2605.24218v1…