PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Research Pretraining via Predictive Navigation

Jiang Zhou, Zhiyuan Fan, Xing Wu, Tinghao Yu, Feng Zhang, Lilin Wang

arXiv:2608.00432v1cs.CL

TL;DR

Deep-research agents require costly environment-grounded trajectories, motivating a more sample-efficient source of navigation supervision. DRP derives offline search–open–write trajectories from citation and hyperlink evidence graphs, and both DRP variants improve low-data agent training, with DRP-Web outperforming a full-data no-DRP checkpoint at one quarter of the SFT data. The gains transfer across benchmarks and persist through agentic RL, while controls support evidence-conditioned navigation as the relevant factor.

  • Problem

    Deep-research training depends on costly environment-grounded trajectories involving repeated retrieval, document inspection, and report evaluation.

  • Method

    DRP reverse-synthesizes offline predictive-navigation supervision from citation-bearing and hyperlinked passages, linked evidence, and graph-related alternatives.

  • Results

    Across five matched low-data subsets, both DRP variants improve DeepResearch Bench performance; with one quarter of SFT data, DRP-Web surpasses the full-data no-DRP Base checkpoint.

  • Takeaways & Limitations

    DRP provides a complementary initialization that reduces reliance on expensive expert trajectories while preserving gains across benchmarks, model scales, and subsequent agentic RL.

  • Takeaways & Limitations

    The analysis uses fixed Qwen3-14B, 1B-token CPT, and 1/16-SFT settings unless otherwise stated, so single-setting orderings should not be treated as universal optima across SFT budgets.

Abstract

from arXiv · show

Deep research agents are often trained on expensive, environment-grounded tool-use trajectories that require repeated retrieval, document inspection, and report evaluation. We introduce Deep Research Pretraining (DRP), an offline framework that derives predictive navigation supervision from naturally occurring evidence structures. Given a citation-bearing or hyperlinked passage, DRP constructs a proxy research objective, recovers linked evidence and graph-related alternatives, and converts them into search-open-write trajectories. This teaches models what to search for, which documents to inspect, and how to synthesize evidence, without a live retrieval environment or executed policy rollout. We instantiate DRP on scholarly citation graphs (DRP-Paper) and Wikipedia hyperlinks (DRP-Web), continually pretrain separate Qwen3-14B-Base models on 1B tokens, and fine-tune them on controlled fractions of 13K agent trajectories. Across five independently sampled subsets at each low-data budget, both variants consistently outperform matched no-DRP models on DeepResearch Bench. With one quarter of the SFT data, DRP-Web even surpasses a fixed no-DRP full-data checkpoint, with gains transferring to ResearchQA, WebWalkerQA, and SimpleQA. Starting from matched low-data SFT checkpoints, the DRP-Web advantage also persists through subsequent agentic RL. Source-matched and evidence-mismatch controls indicate that these improvements arise from evidence-conditioned navigation rather than domain exposure or agent-format imitation. DRP thus provides a promising complementary approach to trajectory-based agent training.

1 INTRODUCTION

Deep Research Pretraining (DRP) converts naturally occurring citation and hyperlink evidence structures into offline predictive-navigation supervision for deep-research agents. Across matched low-data settings, DRP improves downstream trajectory-training efficiency without replacing agent post-training.

  • DRP framework: DRP reverse-synthesizes history-conditioned search–open–write trajectories from citation and hyperlink relations without executing live browsing rollouts.It infers a proxy objective, recovers linked evidence, and adds graph-related alternatives to supervise search, document selection, and synthesis.
  • Domain instantiations: DRP-Paper and DRP-Web instantiate the same predictive-navigation objective on scholarly citation graphs and Wikipedia hyperlinks, respectively.Their domain-specific extraction rules differ, but both use the same action space and offline construction process.
  • Empirical results: Across five independently sampled low-data subsets, both DRP variants consistently outperform matched no-DRP models on DeepResearch Bench.With one quarter of the expert trajectories, DRP-Web also exceeds a fixed full-data Base checkpoint; gains transfer to ResearchQA, WebWalkerQA, and SimpleQA.
  • Interpretation: Source-matched and evidence-mismatch controls support evidence-conditioned navigation as the source of improvement rather than domain exposure or agent-format imitation alone.The method is presented as an initialization that reduces dependence on expensive environment-grounded trajectories, not as a replacement for downstream post-training.

2 METHOD

DRP constructs research trajectories in reverse from a target report, its linked evidence, and graph-based alternatives. It trains models to predict navigation and synthesis decisions while keeping retrieval observations offline and masking them from the learning loss.

  • 2.1 PROBLEM FORMULATION: DRP starts from a citation-bearing or hyperlinked target passage, extracts substantively related evidence documents, and infers a proxy objective whose answer is the normalized report.The normalized target retains only resolvable evidence.
  • 2.1 PROBLEM FORMULATION: At each decision, the model conditions on causal history to predict search or open actions and ultimately synthesize the acquired evidence.The trajectory includes reasoning traces, tool actions, observations, and a final report with a write rationale.
  • 2.2 OFFLINE REVERSE SYNTHESIS: Offline reverse synthesis partitions evidence across search steps, combines each step’s evidence targets with topologically related alternatives, and leaves alternatives unopened as selection contrasts.Later queries and rationales can condition on documents opened earlier, while rationales are filtered to avoid hidden construction information.
  • 2.2 OFFLINE REVERSE SYNTHESIS: The construction uses graph neighborhoods and source documents to render queries, candidate observations, and open observations without calling a search engine or browser.A batched control moves all open operations after searches while preserving the underlying queries, evidence, and report.
  • 2.3 DOMAIN INSTANTIATIONS: DRP-Paper derives evidence from resolved citations and nearby uncited papers, whereas DRP-Web filters substantively used outgoing links and uses hyperlink neighbors as alternatives.Both domains share the same action space, interleaved construction, and learning objective.
  • 2.4 MASKED CONTINUAL-PRETRAINING OBJECTIVE: The masked continual-pretraining objective trains the model on assistant reasoning, tool calls, arguments, and the final report while excluding system, user, and search/open observation tokens.This teaches evidence-conditioned decisions and synthesis rather than reproduction of constructed environment responses.

3 EXPERIMENTS

Experiments test DRP as a sample-efficient initialization for downstream deep-research agents, using matched low-data SFT subsets, transfer benchmarks, and subsequent agentic RL. DRP consistently improves low-data performance, with DRP-Web exceeding the fixed full-data no-DRP baseline and retaining an advantage through RL.

  • 3.1 EXPERIMENTAL SETUP: The study evaluates Qwen3-14B models after 1B-token continual pretraining and five-epoch trajectory SFT at 1/16, 1/4, and full-data budgets, with five subsets at each low-data fraction.The main evaluation uses the 50-question English DeepResearch Bench subset with the official GPT-5.5 RACE evaluator.
  • 3.2 MAIN RESULTS: With one quarter of the expert trajectories, DRP-Web exceeds the no-DRP full-data model by 1.0 point, while DRP-Paper gains 0.7 points with an interval overlapping zero.Across five independently sampled subsets, both methods improve over matched no-DRP SFT; the main comparison is summarized in Figure 2.
  • 3.2 MAIN RESULTS: At both low-data fractions, DRP-Paper and DRP-Web improve every RACE dimension over Base, spanning comprehensiveness, insight, instruction following, and readability.
  • 3.2 MAIN RESULTS: At 1/16 and 1/4 SFT, both DRP variants show positive paired gains on ResearchQA coverage, WebWalkerQA accuracy, and SimpleQA accuracy, while full-data effects become small and task dependent.The attenuation is consistent with DRP acting as a sample-efficient initialization rather than directly optimizing each downstream benchmark.
  • 3.2 MAIN RESULTS: DRP-Web leads Base by 15.0 points before RL, and the advantage remains 6.8 points after 40 steps and 4.8 points after 80 steps.After 80 steps, DRP-Web reaches 59.2, whereas Base reaches 54.4 and remains below the pre-RL DRP-Web score of 57.2.

4 ANALYSIS

The analysis tests whether DRP’s gains come from structured, evidence-aligned navigation rather than raw domain exposure, formatting, or incidental design choices. Across controls and ablations, evidence-conditioned construction and aligned objectives are the key ingredients, while gains persist across model scales and downstream RL.

  • 4.1 RAW DOMAIN PRETRAINING IS NOT ENOUGH: DRP-Web gains 5.3 points over Base versus 0.8 for Raw-Web, while DRP-Paper gains 5.0 points versus a 4.8-point decrease for Raw-Paper.The source-matched raw-document controls show that domain exposure alone is insufficient.
  • 4.2 TRANSFER TO OTHER BENCHMARKS: Positive paired gains appear for DRP-Paper and DRP-Web on ResearchQA coverage, WebWalkerQA accuracy, and SimpleQA accuracy at both low-data SFT fractions.Under full-data SFT, effects become small and task dependent, with most confidence intervals overlapping zero.
  • 4.3 WHICH PSEUDO-OPERATIONS MATTER?: SOSOW reaches 38.3 versus 37.3 for batched SSSOW and 36.4 for search-only, while using fewer downstream calls than the batched schedule.The 1.0-point SOSOW advantage is consistent with feedback-conditioned navigation, although the fixed-subset comparison does not isolate every history difference.
  • 4.4 DOES FORMAT MATTER?: Protocol alignment raises score from 34.0 to 38.3 and report rate from 86% to 100%, but evidence mismatch reduces the matched-format gain over Base from 5.3 to 1.1 points.Thus, formatting affects transfer efficiency but does not explain the full DRP gain.
  • 4.5 TOOL RESPONSE LENGTH: The mid response budget reaches 38.3 with a 100% report rate, compared with 36.4/96% for short and 35.6/92% for long.Longer context provides no monotonic benefit in this fixed 1/16 setting, so the authors use mid as a practical tradeoff rather than a universal optimum.

5 RELATED WORK

The related work situates DRP among agent training, continual pretraining, synthetic document transformations, and weak supervision from document structure. DRP differs by using citation and hyperlink graphs for offline, history-conditioned navigation supervision before trajectory SFT or RL.

  • Training and supervising research agents: Prior agent systems use browser-assisted question answering, interleaved reasoning and actions, search-environment RL, teacher-generated trajectories, synthetic tasks, or report-level rubrics.DRP is complementary because it constructs offline predictive-navigation initialization before trajectory SFT or RL.
  • Continual and synthetic pretraining: Continual pretraining and synthetic-data methods motivate DRP’s pre-SFT intervention, while WRAP++ uses hyperlink motifs to synthesize cross-document question answering.These approaches provide context for restructuring source documents and exploiting in-domain text before downstream adaptation.
  • Weak supervision from document structure: Earlier document-structure methods support pseudoquery generation for retriever pretraining or dynamic planning at inference time, whereas DRP supervises an autoregressive policy spanning repeated search, evidence inspection, and report synthesis.Its graph supervision is therefore neither only a single-step retrieval target nor an inference-time memory.

6 CONCLUSION

DRP converts citation and hyperlink relations into offline predictive-navigation supervision before downstream agent training. The framework improves low-data trajectory-SFT efficiency, transfers across benchmarks and RL, and attributes gains to evidence alignment rather than raw domain exposure or trace formatting.

  • 6 CONCLUSION: DRP renders graph-derived evidence and alternatives into history-conditioned search, open, and write decisions without live browser rollouts.It serves as a complementary initialization for downstream agent training rather than a replacement for expert-trajectory SFT or RL.
  • 6 CONCLUSION: Across five matched low-data subsets, both DRP corpora improve trajectory sample efficiency, while DRP-Web with one-quarter SFT surpasses the full-data no-DRP Base.The recipe uses 75% fewer expert trajectories and reduces standardized data-construction cost by at least 51.8%.
  • 6 CONCLUSION: DRP gains transfer to other benchmarks, persist through agentic RL, and span evidence graphs and model scales.These conclusions are reported alongside the low-data efficiency results and the fixed training comparisons described in the paper.
  • 6 CONCLUSION: Source-matched raw-CPT and evidence-mismatch controls attribute the benefit to navigation alignment rather than domain exposure or trace formatting.The method constructs evidence sets, graph-derived alternatives, and offline search–open–write trajectories using corpus-specific graph rules.
  • 6 CONCLUSION: The offline construction resolves evidence, partitions it across search steps, adds related alternatives, synthesizes queries, renders observations, and applies an agent-token mask before returning the trajectory.Search and open observations are context, while assistant reasoning, tool calls, and final reports receive loss.

A.6 AGENT INFERENCE AND EVALUATION

Evaluation uses fixed agent protocols and multiple uncertainty analyses to measure DRP's downstream performance, transfer, contamination risk, and variation across sampled SFT subsets.

  • Evaluation setup: The primary evaluation uses 50 English DeepResearch Bench questions with GPT-5.5 RACE scoring, while transfer tests use fixed 500-example sets from ResearchQA, WebWalkerQA, and SimpleQA.ResearchQA reports coverage; WebWalkerQA and SimpleQA report accuracy.
  • Contamination analysis: No ResearchQA query shares an 8- or 13-word span with the DRP-Paper training corpus, providing no evidence of verbatim benchmark-query leakage.
  • Subset variation: Five low-data runs vary the sampled SFT subset rather than the training seed, enabling paired comparisons across independently sampled downstream datasets.
  • Subset-level uncertainty: The reported 95% Student-t intervals quantify how estimated gains change across five SFT subsets, not training-seed variation or question-level evaluation uncertainty.
  • Question-level uncertainty: Paired bootstrap intervals for single-checkpoint analyses resample question IDs while preserving system pairing, complementing rather than replacing subset-level t intervals.

B.3 REPEATED-EVALUATION STABILITY

Repeated evaluation of one DRP-Web full-SFT checkpoint estimates the spread caused by stochastic agent rollouts and evaluation, separately from training-subset uncertainty.

  • Repeated evaluation: A single DRP-Web full-SFT checkpoint scored 40.64 on average across three repeated evaluations, with a 0.46 sample standard deviation and a 0.91-point range.
  • Interpretation: The evaluation-only spread was smaller than the matched low-data gains in the main results, but it applies to one full-SFT checkpoint and does not replace paired uncertainty analyses.

C.1 PER-SUBSET DEEPRESEARCH BENCH RESULTS

Per-subset results show consistent DRP gains across low-data SFT subsets, report improvements across RACE dimensions, and preserve general capability under a matched CPT budget.

  • C.1 PER-SUBSET DEEPRESEARCH BENCH RESULTS: Both DRP variants improve comprehensiveness, insight, instruction following, and readability at 1/16 and 1/4 SFT, so gains are not confined to one report dimension.
  • C.4 GENERAL-CAPABILITY SCORES: Replacing half of a 2B-token general CPT budget with DRP-Web changes the 14-benchmark average from 73.1 to 73.4 while raising matched 1/16-SFT DeepResearch Bench from 33.1 to 36.8.
  • C.1 PER-SUBSET DEEPRESEARCH BENCH RESULTS: The low-data rows average five independently sampled matched SFT subsets, whereas full-SFT values are fixed single-checkpoint references rather than estimates over those subsets.

D SYNTHETIC-DATA COST ACCOUNTING

The cost accounting compares expert-trajectory supervision with an offline DRP recipe using visible output-token prices and excludes training costs.

  • D SYNTHETIC-DATA COST ACCOUNTING: The accounting prices only visible output text using public list prices, excluding input-token billing and external search or retrieval charges because historical request logs are unavailable.
  • D SYNTHETIC-DATA COST ACCOUNTING: DRP-Web plus one-quarter SFT costs at most $167.5 versus approximately $347.6 for Base plus full SFT, reducing the output-price proxy by at least 51.8%.
  • D SYNTHETIC-DATA COST ACCOUNTING: The DRP recipe uses 75% fewer expert trajectories and reduces real environment calls from 47,147 to 11,597, while improving the score from 38.5 to 39.6.

E CONSTRUCTION AND ABLATION DETAILS

The ablations separate predictive-navigation supervision from source-domain exposure, evidence alignment, protocol serialization, trajectory topology, and observation-loss design. Together, they test which components of the constructed training format matter.

  • Source and evidence controls: Source-matched Raw-Web and Raw-Paper controls remove proxy queries, reasoning traces, tool calls, and graph-derived relevance relations while retaining the same source pools and 1B-token budget.This comparison isolates restructuring the domains into predictive-navigation supervision.
  • Source and evidence controls: Evidence-Mismatched DRP-Web preserves objectives, reasoning, actions, rendering, response lengths, and budget but replaces graph-derived observations with random in-domain evidence.The ablation breaks query–observation–action alignment while preserving agent-like surface form.
  • Trajectory and protocol design: SSSOW batches variable searches before selected evidence is opened, whereas SOSOW interleaves variable search–open rounds so later queries and rationales condition on opened evidence.Both names describe topology rather than fixed-length traces.
  • Trajectory and protocol design: DR-Tulu and Hermes hold objectives, traces, tool calls, observations, and reports constant while changing only serialization to test protocol alignment.DR-Tulu groups the constructed interaction in a compact protocol, whereas Hermes alternates tool-call and tool-response turns.
  • Observation and loss design: Shorter rendered observations reduce training cost and increase loss-bearing-token share but may omit information needed for later decisions or final reports; no-mask additionally trains on tool observations.All other data and optimization settings remain fixed in the no-mask comparison.

F ADDITIONAL LIMITATIONS

The evaluation is limited in language, trajectory-source diversity, benchmark scale, and uncertainty coverage, while DRP’s gains diminish as downstream SFT data increases.

  • Evaluation scope: The experiments use English downstream trajectories from one released trajectory source, limiting the tested language and data-source scope.
  • Evaluation scope: The primary DeepResearch Bench evaluation has 50 English questions, and the five-run design does not estimate all interactions among sampling, training, decoding, and judge randomness.
  • Scaling boundary: DRP’s gain attenuates with more downstream SFT data, narrowing the regime in which its advantage is observed.
Loading 2608.00432v1…