PaperlayerView exact revision on arXiv

Source-linked AI summary

QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks

Jian Xie, Tianhe Lin, Zilu Wang, Yuting Ning, Yuekun Yao, Tianci Xue, Zhehao Zhang, Zhongyang Li, Kai Zhang, Yufan Wu, Shijie Chen, Boyu Gou, Mingzhe Han, Yifei Wang, Vint Lee, Xinpeng Wei, Xiangjun Wang, Yu Su, Huan Sun

arXiv:2605.24218v1cs.CL

TL;DR

Open deep research agents have often focused on specific use scenarios, leaving the path to general-purpose agents unclear. QUEST combines rubric-tree data synthesis, context management, and staged training. Across eight benchmarks, Quest-35B achieves the best overall performance among recent open-weight agents.

  • Problem

    Existing open approaches typically target specific use scenarios rather than the full range of capabilities required for general-purpose deep research agents.

    Source: §1 · p. 2

  • Method

    QUEST combines rubric-tree-based data synthesis, structured context management, and a training pipeline that uses task-specific rubrics as optimization signals.

    Source: §1 · p. 3

  • Results

    Across eight benchmarks, Quest-35B achieves the best overall performance among recent open-weight agents; on DeepResearch Bench it scores 48.2% vs. 47.0% for OpenAI-DR.

    Source: §6.2 · p. 9, §8 · p. 13

  • Takeaways & Limitations

    The paper presents its open, reproducible recipe as a foundation for future deep research agents.

    Source: §8 · p. 13

    Adding search-result prediction to the mid-training objective mixture consistently hurts overall performance, and the authors leave a more principled integration to future work.

    Source: §7.1 · p. 12

Abstract

from arXiv · show

Deep research agents extend the role of search engines from retrieving keyword-matched pages to synthesizing knowledge, fundamentally changing how humans interact with information. However, frontier systems remain proprietary, while existing open agents often generalize poorly across different task types, leaving unclear how to train a broadly capable deep research agent. We release QUEST, a family of open models (ranging from 2B to 35B) that serve as general-purpose deep research agents designed to handle a wide range of long-horizon search tasks, with strong capabilities in fact seeking, citation grounding, and report synthesis. To build QUEST, we propose an effective training recipe combining mid-training, supervised fine-tuning, and reinforcement learning. Central to this recipe is a curated data synthesis pipeline based on unified rubric trees, which applies to different task types and enables synthesizing training data with verifiable rewards without human annotation. In addition, QUEST incorporates a built-in context management mechanism that enables effective long-horizon reasoning and knowledge synthesis. Using only 8K synthesized tasks, QUEST approaches or even surpasses frontier closed-source agents across eight deep research benchmarks spanning diverse task types, and achieves the best overall performance among recent open-weight agents. We released everything: models, data, and training scripts.

1 Introduction

Deep research agents can search, verify, remember, and synthesize information over long horizons, but transparent training recipes and systems that cover the full range of research tasks remain limited. Quest addresses this gap with open models, synthetic rubric-based training data, context management, and a training pipeline evaluated across eight benchmarks.

1 Introduction

  • Deep research agents shift web-scale information seeking from manual inspection of ranked pages toward autonomous query execution, source examination, and citation-backed synthesis, while their training recipes remain poorly understood.RAG systems retrieve documents to condition LLM responses, whereas deep research agents decompose complex tasks into intermediate goals and search over longer horizons.Source: §1 · p. 2
  • Deep research spans objective, verifiable tasks and open-ended tasks, yet existing open approaches often target specific uses and do not cover the full range of required capabilities.The paper identifies fact seeking, report synthesis, and citation grounding as complementary capabilities that prior systems and benchmarks address only in isolation.Source: §1 · p. 2
  • Quest is an open family of general-purpose deep research agents spanning 2B to 35B parameters.Its approach combines task-specific verifiable rubric trees for synthetic training data, compact summaries across long-horizon search trajectories, and training with rubric-based signals.Source: §1 · pp. 2–3
  • Quest-35B reaches 64.6% on BrowseComp, 30.7% on Mind2Web 2, and 48.2% on DeepResearch Bench, with consistently strong performance across eight diverse benchmarks and surpassing strong proprietary systems on some benchmarks.The benchmarks cover fact seeking, report synthesis, and citation grounding.Source: §1 · p. 3
  • Quest’s training pipeline combines mid-training, supervised fine-tuning, and reinforcement learning, with controlled studies examining training-stage contributions and scaling across model sizes.The paper reports that Quest-2B-SFT and Quest-4B-SFT achieve impressive performance, suggesting potential for local deployment in privacy-sensitive settings such as biomedical research and legal analysis.Source: §1 · p. 3

2 Data Synthesis

Single-answer supervision is insufficient for deep research tasks that require multi-source synthesis and fine-grained evaluation. QUEST uses synthetic rubric trees to support diverse tasks and provide richer training signals.

2 Data Synthesis

  • Prior single-answer supervision mainly targets short-answer fact seeking and binary rewards, limiting generalization to multi-source synthesis and fine-grained credit assignment.Such supervision suits BrowseComp-like questions whose answers are hard-to-find entities, but not tasks requiring coherent, source-supported outputs evaluated on criteria such as attribution and insightfulness.Source: §2 · p. 3

2.1 Rubric Tree

  • Rubric trees replace a unique ground-truth answer with task-specific constraints, supporting single-answer, multi-solution, and open-ended tasks while providing fine-grained scores.QUEST constructs the trees synthetically through automatic generation, refinement, and verification; verifiable leaf criteria receive binary scores, while internal-node scores aggregate their children.Source: §2.1 · p. 3

2.2 Data Generation Pipeline

  • Each training instance pairs a complex query with a query-specific rubric tree and an evaluation protocol that scores responses from 0 to 1.For objective tasks, trending keywords seed web exploration; Gsyn derives verifiable constraints, refines and verifies trees, and discards structures that cannot be evaluated reliably. It then generates a natural-language question, and GPT-5 produces a Python script to verify rubric nodes and compute the score.Source: §2.2 · pp. 4–5
  • Open-ended rubrics share four root criteria—instruction following, comprehensiveness, readability, and insight—with task-specific child nodes and weights averaged across three generations.The tasks otherwise follow the keyword-sampling and web-exploration process used for objective tasks.Source: §2.2 · p. 5
  • Open-ended evaluation compares each candidate report with a generated reference report, using a rubric judge to score both reports on each task-specific node.The judge assigns continuous scores from 0 to 10; weighted tree scores are normalized as J(rcand)/(J(rcand) + J(rref)), so scores above 0.5 mean the candidate surpasses the reference.Source: §2.2 · p. 5

3 Context Management

Quest manages long research trajectories by compressing accumulated history into a structured Context State, preserving useful knowledge without relying on capped turns or a context window large enough to hold everything. The state distinguishes trusted, untrusted, and uncertain claims and is updated when context usage exceeds a threshold.

3 Context Management

  • Quest compresses accumulated search results, visited pages, and reasoning traces into a compact summary, enabling long-horizon reasoning without context overflow.This contrasts with open-weight agents that cap turns or rely on a sufficiently large context window to hold the full history.Source: §3 · p. 5
  • The JSON Context State separates knowledge into trusted facts, contradicted claims, and partially supported claims requiring follow-up.Entries are grounded in retrieved URLs; uncertain entries include a URL to visit or a query to rerun, while untrusted entries record the reason for distrust.Source: §3 · pp. 5–6
  • When context usage exceeds a threshold, GPT-5-mini condenses the full raw history into an updated Context State, and Quest resumes in a fresh context window.Uncertain entries guide later search and verification; untrusted entries are deprioritized unless further verification is needed, while trusted claims can be reused without redundant tool calls.Source: §3 · p. 6

4 Training

QUEST training combines mid-training, supervised fine-tuning, and reinforcement learning to build long-horizon search interaction, imitate tool-use trajectories, and optimize behavior with rubric-based and fact-checking rewards. Its training data and units are designed to align with the agent’s inference process.

4 Training

  • The three-stage pipeline uses mid-training for long-horizon search adaptation, SFT for tool-use imitation, and RL for rubric- and fact-checking-based policy optimization.These stages play complementary roles in shaping agent behavior.Source: §4 · p. 6

4.1 Trajectory Collection

  • SFT trajectories come from a tool-using teacher, are retained only when they exceed task-specific evaluation thresholds, and receive inline citations linked to supporting URLs.For objective tasks, failed initial trajectories are retried with evaluation feedback (Zhang et al., 2025); thresholds are 1 for objective tasks and 0.475 for open-ended tasks. GPT-5-mini inserts citations using the reasoning trace and Context State.Source: §4.1 · p. 6

4.2 Mid-Training

  • Mid-training teaches Context State JSON summarization and goal-relevant extraction from noisy HTML, filtering navigation, advertisements, and off-topic content.Its supervision is reused or derived from pipeline outputs, requiring no additional data collection or annotation.Source: §4.2 · pp. 6–7

4.3 Supervised Fine-Tuning

  • Session-level SFT splits trajectories at context-condensation events, aligning each training instance with the agent’s effective working context at inference.The model is trained on these sessions with a next-token prediction objective.Source: §4.3 · p. 7

4.4 Reinforcement Learning

  • GRPO-style RL combines query-specific rubric rewards with fact-checking rewards for inline citations, using the same session-level training strategy as SFT.Fact-checking evaluates cited fact–URL pairs with GPT-5-mini and scores the fraction of citations labeled supported among those labeled supported or unsupported.Source: §4.4 · p. 7
  • A minimum operator caps the fact-checking contribution at the rubric reward, preventing well-cited but poorly completed responses from receiving inflated rewards.It also removes the fact-checking contribution when the underlying content is entirely incorrect.Source: §4.4 · p. 7
  • RL assigns each rollout’s scalar reward to its derived sessions, normalizes advantages across unique rollouts, and optimizes with the GRPO objective without a KL penalty.When a rollout produces multiple sessions, they share one advantage value.Source: §4.4 · p. 7

5 Infrastructure

QUEST’s infrastructure makes large-scale research-agent training more practical by caching repeated web interactions and asynchronously processing slow RL evaluations. Its tools support search, webpage reading, computation, and academic-publication access.

5 Infrastructure

  • QUEST uses Google Search, Visit, Python Interpreter, and Google Scholar; its Visit summarization model is GPT-5-mini.Visit reads webpages and summarizes information toward the goal of extracting key information from HTML.Source: §5 · p. 8
  • A dual-cache system serves repeated search queries and URL visits from cache during data synthesis, SFT, and RL rollout, avoiding billed live API calls.The system intercepts search and visit operations before they reach live APIs, addressing the cost and latency of parallel external calls.Source: §5 · p. 8
  • Search and Scholar queries use exact-match lookup, then FAISS semantic retrieval above a cosine-similarity threshold, before falling back to the live Serper API.Results from live calls are persisted for reuse.Source: §5 · p. 8
  • Visit checks a persistent cache keyed by the full URL and fetches from the live Jina API only on a miss, eliminating repeated page fetches across training runs.Source: §5 · p. 8
  • Around 4 minutes on average and up to 30 minutes in long-tail cases, rubric-tree evaluations motivate dispatching completed trajectories to asynchronous Ray actors.Training continuously pulls scored samples from a shared queue and updates once enough are accumulated; timeout and interruption-aware reward handling supports stability.Source: §5 · p. 8

6 Experiments · Training.

Quest-35B sets a new open-weight state of the art across almost all evaluated benchmarks, narrowing the gap with proprietary agents. Controlled studies find that its full MT+SFT+RL recipe performs best overall, while smaller SFT-trained models remain competitive on fact-seeking tasks but lag on report synthesis.

Training.

  • Using Qwen3.5-35B-A3B as the base, the full recipe produces Quest-35B from 8K synthesized instances for SFT and RL.The paper presents training-stage data distributions in Table 1.Source: Training · pp. 8–9, Table 1 · p. 9
  • Evaluation covers eight benchmarks—six objective and two open-ended—with official full test sets except text-only HLE and GAIA; costly ablations use sampled subsets.The benchmarks include BrowseComp, Mind2Web 2, Humanity’s Last Exam, BrowseComp-Plus, WideSearch, GAIA, DeepResearch Bench, and LiveResearchBench.Source: Training · p. 9

6.2 Results

  • 48.2% vs. 47.0% on DeepResearch Bench and 30.7% vs. 28.0% on Mind2Web 2 show Quest-35B matches or exceeds OpenAI-DR while setting open-weight state of the art on almost all benchmarks.It also scores 80.8% vs. 76.4% against GPT-5 on GAIA.Source: §6.2 · p. 9, Table 3 · p. 10
  • Quest-30B leads on 4 of 8 benchmarks at 30B scale, while Tongyi-DR leads on BrowseComp, HLE, and GAIA and OpenResearcher on BrowseComp-Plus.The authors attribute these differences to the agents’ data-synthesis recipes and argue that Quest’s recipe covers a wider range of capabilities.Source: §6.2 · pp. 9–10

6.3 Insights from Controlled Studies

  • MT+SFT+RL performs best among the tested Quest-35B variants, although the effects of individual training stages differ across benchmarks.SFT improves most objective benchmarks but hurts open-ended performance and BC-Plus, where training-pattern overfitting increases disallowed tool calls. MT adds overall gains; RL improves open-ended tasks but slightly reduces HLE and GAIA performance, which the authors hypothesize require broader expert reasoning (an alignment tax, Ouyang et al., 2022).Source: §6.3 · p. 10, Figure 5 · p. 11
  • A 2B SFT model can match or outperform models such as o3 on fact-seeking benchmarks, but lags far behind on report-synthesis tasks.The paper gives o3 scores of 24.9% on HLE and 70.5% on GAIA; performance generally improves with larger models, and the 2B advantage does not transfer to DeepResearch Bench or LiveResearchBench (OpenAI, 2025b,c).Source: §6.3 · pp. 10–11, Figure 6 · p. 11

7 Unsuccessful Attempts

The preliminary experiments identify training objectives and evaluation schemes that failed to improve performance or provide usable signals, and describe hypothesized reasons for those failures.

7 Unsuccessful Attempts

  • The section shares unsuccessful attempts from the preliminary experimental stage to inform future efforts.Source: §7 · p. 11

7.1 Search Result Prediction in Mid-training

  • Search-result prediction improved some search-heavy tasks in isolation but hurt overall performance when added to mid-training, potentially because it overlaps with context summarization.The task summarizes retrieved results for next-token prediction, following Zhang et al. (2025); the authors leave more principled integration to future work.Source: §7.1 · pp. 11–12

7.2 Rubric-Based Error Identification in Mid-training

  • Rubric-based error prediction yielded only marginal improvements, likely because without external evidence it misses complex factual or evidence-dependent errors.The authors conclude that rubric-based error prediction alone is insufficient as a standalone training signal without grounding in external knowledge.Source: §7.2 · p. 12

7.3 Direct Policy Optimization (DPO) for Report Comparison

  • Pairwise report-comparison DPO did not improve results, with preference-pair overhead, unstable training, overfitting, and ambiguous long-form comparisons weakening its signal.Small rubric-score differences may not reflect meaningful qualitative distinctions across coverage, organization, and evidence use; the approach follows Rafailov et al. (2023).Source: §7.3 · p. 12

7.4 Evaluation for Open-ended Tasks

  • Pointwise scoring reached approximately 1 in ∼50% of cases, making good and poor responses difficult to distinguish; when early model performance was below the teacher, pairwise scores collapsed toward zero, rendering the signal unusable for SFT filtering and RL training.Source: §7.4 · p. 12

8 Conclusion … A.1 Task Synthesis

QUEST is an open family of general-purpose deep research agents, and Quest-35B performs strongly across eight benchmarks. The paper’s contribution also includes a reproducible training recipe and strict task filtering during data synthesis.

8 Conclusion

  • Quest-35B achieves the best overall performance among recent open-weight agents and approaches or surpasses closed-source agents across eight benchmarks.Source: §8 · p. 13
  • QUEST agents are designed for long-horizon tasks requiring fact seeking, report synthesis, and citation grounding.Source: §8 · p. 13
  • The open, reproducible recipe combines rubric-tree-based data synthesis, structured context management, efficient training infrastructure, mid-training, supervised finetuning, and RL.Source: §8 · p. 13
  • The paper presents QUEST as a strong open foundation for future deep research agents.Source: §8 · p. 13

A Data Construction Details

A.1 Task Synthesis

  • Task synthesis uses strict filtering to ensure generated tasks are high-quality and evaluation trees are reliable, resulting in a low retention rate.Source: §A.1 · p. 18
  • Table 4 reports the number of tasks retained after each filtering stage.Source: Table 4 · p. 18, §A.1 · p. 18

A.2 Manual Examination of the Python Evaluation Scripts for Objective Tasks … A.5 Reinforcement Learning

The appendix assesses generated evaluation scripts and describes the mid-training data and SFT/RL task preparation, including identified script errors and training-set filtering.

A.2 Manual Examination of the Python Evaluation Scripts for Objective Tasks

  • Four computer-science graduate students assessed 50 sampled tasks by checking script alignment with rubric trees and program executability using provided instructions and examples.Source: §A.2 · p. 18
  • The six rubric-error scripts added redundant rubric nodes, checked literal answers instead of content authenticity, or lacked a negative judgment rule when answers had no cited URLs.Source: §A.2 · p. 18
  • Most scripts interpret task requirements, implement rubric rules, and run without errors; among 50 sampled tasks, 2 had non-executable issues and 6 had rubric-related errors.The non-executable issues included runtime exceptions and logical execution breakdowns.Source: §A.2 · p. 18

A.3 Mid-Training

  • Mid-training used 300K context-summarization instances from SFT distillation and preliminary RL experiments, filtering low-quality cases and excluding benchmark-evaluation data to avoid leakage.Source: §A.3 · pp. 18–19
  • Relevant-information extraction began with 4 million webpage-HTML, goal, and extracted-content triplets, then removed redundant entries using same-page grouping and a 0.1 Jaccard-similarity threshold.Source: §A.3 · p. 19

A.4 Supervised Fine-tuning; A.5 Reinforcement Learning

  • SFT used repeated rejection sampling and reflection retries, while RL received objective tasks that Gtraj never completed and manually split open-ended tasks.Rejection sampling rolled out each task five times, and reflection-based retry rolled out failed trajectories three additional times; maximum tool calls were 100 turns and the context threshold was 16K. The objective RL set contained 864 tasks.Source: §A.4 · p. 19, §A.5 · p. 19

B Session-level Training … E.3 Reinforcement Learning

Session-level training supports long research trajectories while enabling inference beyond training and native context lengths; the Context Condenser preserves structured research state, and the training recipe proceeds through mid-training, supervised fine-tuning, and asynchronous reinforcement learning.

B Session-level Training

  • More than 200 inference turns are supported despite training trajectories capped at 100 turns, while session-level training handles trajectories exceeding 200K tokens on 16 H100 GPUs.Variable-length sessions decouple per-session training context from total trajectory length, allowing inference beyond training context lengths and the model’s native context window.Source: B Session-level Training · p. 19

C.1 Why Context Condenser instead of Discard-All or Keep-Last-N

  • The Context Condenser preserves verified facts, pending actions, and contradictions in a structured epistemic snapshot across compression boundaries.Discard-all can force rediscovery of verified facts, while keep-last-N can lose critical older information; both can discard what the agent knows, doubts, and still needs to find. Under constrained context budgets such as 32K, naive truncation can corrupt supervision, whereas the condenser keeps training trajectories and inference rollouts coherent.Source: §C.1 · pp. 19–20

D Model Configuration

  • DeepSeek V3.2 fallback models serve context condensation and visit summarization, with a roughly 3% trigger rate; without fallback, RL training becomes unstable.The fallback prevents training interruptions caused by requests-per-minute limits.Source: D Model Configuration · p. 20

E.1 Mid-Training

  • Mid-training uses full-parameter optimization for one epoch.It uses BF16, a 3×10−6 learning rate with a constant scheduler, a maximum context length of 24,576 tokens, per-device batch size 2, and gradient accumulation steps 8. It runs on 32 H100 GPUs and takes approximately 5 days.Source: §E.1 · p. 20

E.2 Supervised Fine-Tuning

  • Supervised fine-tuning applies full-parameter optimization for 3 epochs after mid-training.It uses BF16, a 3 × 10−6 learning rate with a constant scheduler, a maximum sequence length of 32,768 tokens, per-device batch size 1, and gradient accumulation steps 8; it runs on 32 H100 GPUs for approximately 1 day.Source: §E.2 · p. 21

E.3 Reinforcement Learning

  • Reinforcement learning uses a fully asynchronous VERL pipeline with GRPO and up to 100 tool-call turns per rollout.The actor learning rate is 1 × 10−6, PPO mini-batch size is 16, and 8 responses are sampled per prompt; the global batch size is 64, split into four mini-batches, with policy synchronization after all four updates. Prompt, response, and single-turn response limits are 24,000, 12,288, and 10,240 tokens, respectively, and the context threshold is 16K. Training and rollout each use 16 GPUs; 80 asynchronous steps correspond to 320 mini-batch-level optimization steps.Source: §E.3 · p. 21

F Evaluation Subset

To manage evaluation time and API costs, the ablation study and analyses use benchmark subsets, with WideSearch restricted to its English subset.

F Evaluation Subset

  • All other subsets are sampled uniformly at random without manual filtering, and Table 2 reports their statistics.The cost concern includes BrowseComp’s 1,266 instances and HLE-Text’s 2,158 instances.Source: F Evaluation Subset · p. 21

G Evaluation Details of the Vanilla Model … J.1 Objective Task Examples

This section documents the evaluation baseline and the prompts used to synthesize research tasks, guiding an evidence-gathering pipeline from question and rubric construction to grounded solutions. It also describes task complexity and illustrates how rubric trees and scoring rules vary by task type, including binary checks for objective tasks.

G Evaluation Details of the Vanilla Model

  • Unable to reproduce Qwen3.5’s reported HLE (w/ tool) and Browsecomp scores, the authors used Quest’s prompt, tools, and context strategy as a consistent baseline.They hypothesize that undisclosed prompts, tool specifications, implementation details, or context-management nuances may explain the discrepancy.Source: G Evaluation Details of the Vanilla… · p. 21

H Inference Prompts

  • The agent system prompt defines four tool schemas, prev_state use, and tool-call and final-answer formats, while the visit extractor returns structured rational, evidence, and summary fields.The system prompt appends the current date at runtime; the extractor processes webpage content against the agent’s goal.Source: H Inference Prompts · p. 21
  • When context reaches a threshold, the condenser compresses research into trusted, untrusted, and uncertain state, extracting visit facts and specifying next actions to reduce redundant searches.It also tracks visited sources and prior queries, with rules for deduplication and movement of claims among state buckets.Source: H Inference Prompts · pp. 23–25

I Task Synthesis Prompts

  • The objective-task prompt researches a topic with search and visit tools, extracts verifiable constraints, builds a criticality-tagged rubric tree, and proposes a grounded question and solution across nine complexity classes.Intermediate rubric nodes are tagged sequential or parallel; the classes are C1–C9.Source: I Task Synthesis Prompts · p. 26
  • When evidence permits multiple answers or lacks unanimous confirmation, rubrics must assess general properties rather than force a predetermined entity or value.Rubric requirements must trace to extracted constraints and cannot introduce new requirements.Source: I Task Synthesis Prompts · pp. 27–28
  • Proposed questions must be fully answerable from gathered information, reflect the extracted constraints, and avoid unbounded traversal or complete-enumeration tasks.The prompts also require at least one feasible, evidence-grounded solution.Source: I Task Synthesis Prompts · p. 28
  • The prompts require an ordered workflow: gather sources, extract verifiable constraints, build a rubric, propose a solvable question, and provide a grounded solution.The solution must use gathered information, satisfy the rubric, and avoid invented facts or URLs.Source: I Task Synthesis Prompts · pp. 27–29
  • Rubric trees distinguish critical criteria, whose failure fails the parent, from non-critical criteria that allow partial credit, and use sequential or parallel aggregation to reflect dependencies.Leaves should express clear, objectively verifiable criteria in a hierarchy matching the task.Source: I Task Synthesis Prompts · pp. 29–30
  • Task complexity is set through user-specified breadth and depth regimes, while rubric rules bound item-finding tasks and require atomic, objectively verifiable leaves.The rubric must not evaluate more than five separate items.Source: I Task Synthesis Prompts · pp. 30–31
  • Tasks should reflect coherent, practical user needs, require sustained search and synthesis, and state precise, unambiguous answer criteria.The prompts reject simple tasks solvable in a few queries and artificial combinations of unrelated steps.Source: I Task Synthesis Prompts · p. 33
  • Tasks require verifiable criteria, primarily assessed from answer text and URL provenance, and exclude video understanding, non-English websites, external tools, and fast-changing answers.Verification may use external methods only for a minor part when necessary; tasks should avoid unverifiable rankings or exhaustive lists unless grounded in fixed pages.Source: I Task Synthesis Prompts · p. 33
  • The proposer must research a topic, gather sufficient sources, pose one solvable open-ended question requiring cross-document synthesis, and produce a grounded Markdown solution.The proposed question and solution must be answerable from the information gathered, without invented facts or URLs.Source: I Task Synthesis Prompts · pp. 34–35
  • The complexity rubric varies tasks by conceptual breadth, logical nesting, and exploration, with levels ranging from single-source, shallow, specified tasks to cross-source, deeply dependent, exploratory ones.The rubric’s table defines simple, moderate, and high breadth; shallow, intermediate, and deep nesting; and low, medium, and high exploration.Source: I Task Synthesis Prompts · p. 36

J Task Examples

  • The paper presents representative examples of each task type generated by the data-synthesis pipeline.Source: J Task Examples · p. 38
  • Examples illustrate deep sequential and wider hierarchical rubric trees, while open-ended rubric weights vary with question style, such as analytical arguments versus structured how-to guides.Source: J Task Examples · p. 38

J.1 Objective Task Examples

  • Objective tasks pair programmatically verifiable tasks with binary-scored rubric leaves aggregated through parallel or sequential strategies.Internal nodes aggregate their children’s scores.Source: §J.1 · p. 38
  • In the listeria example, a mixed sequential-parallel tree evaluates which outbreak was deadlier only after both outbreaks are identified; all nodes are critical.A failed leaf zeros its subtree.Source: §J.1 · pp. 38–39, Figure 7 · p. 39
  • The FCC example groups ten critical leaf checks under three thematic branches, whose non-critical internal nodes average child scores to allow partial credit.A single failed leaf does not zero out its sibling group.Source: §J.1 · p. 39, Figure 8 · p. 40

J.2 Open-Ended Task Examples · K Related Work

Open-ended tasks use a partly fixed rubric with adaptive criteria and weights that shift with the question style. Related work describes deep research as integrating fact seeking, citation grounding, and report synthesis, capabilities existing benchmarks and systems do not address jointly.

J.2 Open-Ended Task Examples

  • Open-ended task rubrics use four shared dimensions and question-conditioned, task-specific subcriteria to evaluate candidate responses against a generated reference report.The shared dimensions are instruction following, comprehensiveness, readability, and insight.Source: §J.2 · p. 40
  • Insight receives the highest weight (0.34) in the Apple iPhone Fold example, an analytical task centered on strategic positioning.The example asks for a multifaceted evaluation of a market hypothesis.Source: §J.2 · p. 40
  • The rubric tree aggregates judge scores from leaves through dimensions into one scalar for pairwise comparison with the reference report.Subcriterion weights sum to 1 within each dimension.Source: §J.2 · p. 40, Figure 9 · p. 41
  • Comprehensiveness rises from 0.30 to 0.34 and instruction following from 0.24 to 0.27 for the MSCI guide, while insight falls from 0.34 to 0.23.The guide prescribes four sections, prioritizing factual coverage and adherence over original argumentation.Source: §J.2 · pp. 40, 42, Figure 10 · p. 43

K Related Work

  • Proprietary deep research systems lack public training data and recipes, while open-weight efforts explore staged training and automatic data generation.Tongyi DeepResearch combines mid-training, supervised fine-tuning, and reinforcement learning; other efforts include DR Tulu (Shao et al., 2025) and OpenResearcher (Li et al., 2026).Source: K Related Work · p. 44
  • Deep research benchmarks span hard-to-find facts, broad fact coverage, expert reasoning, tool use, source attribution, and long-form report synthesis.Examples include BrowseComp (Wei et al., 2025), WideSearch (Wong et al., 2025), GAIA, and DeepResearch Bench (Du et al., 2025).Source: K Related Work · p. 44
  • Existing benchmarks and agent systems address fact seeking, citation grounding, and report synthesis in isolation, while QUEST aims to train an agent for all three.The related-work section identifies this joint capability as an unmet gap.Source: K Related Work · p. 44

Related papers

Loading 2605.24218v1…