PaperlayerView exact revision on arXiv

Source-linked AI summary

PrimeSeeker: Capability-Oriented Supervision for Deep Search Agents

Linzhi Peng, Hanting Chen, Heng Chang, Ke Cheng, Bowen Du, Weifeng Lv

arXiv:2609.35816v1cs.CLcs.AI

TL;DR

Search-agent training often raises task or trajectory complexity without specifying the local retrieval capabilities a question is meant to teach. PrimeSeeker structures supervision around latent anchor resolution and relation transfer, then reuses a construction-derived evidence skeleton across expert generation and policy optimization. Across five deep-search benchmarks, it achieves strong performance, and reference-step optimization improves the supervised policy.

  • Problem

    Task difficulty and trajectory properties are indirect proxies that do not specify the reusable local retrieval capabilities deep-search agents must perform.

    Source: §1 · pp. 1–2

  • Method

    PrimeSeeker builds web-grounded anchor structures that jointly derive questions and evidence skeletons, using extractive observation highlights for expert generation and skeleton-based reference-step coverage in reinforcement learning.

    Source: §1 · pp. 2–3, §3.2 · p. 5

  • Results

    Across five deep-search benchmarks, PrimeSeeker achieves strong performance, and PrimeSeeker-Dq consistently improves over PrimeSeeker-SFT while fixed-budget evaluation matches or exceeds strong long-horizon systems with fewer tool calls.

    Source: §1 · p. 3, §6 · p. 10

  • Takeaways & Limitations

    The results support explicit local retrieval capabilities and reusable construction-time structure as a more direct basis for search-agent supervision.

    Source: §6 · p. 10

    The evidence skeleton does not enumerate every valid search path or supporting webpage, and sparse clues with extremely large candidate spaces may require broader exploration.

    Source: H Released Artifacts and Limitations · p. 24

Abstract

from arXiv · show

Large language model search agents are often trained with synthetic questions whose difficulty is increased through larger evidence graphs, additional hops, and longer trajectories. These global properties, however, are only indirect proxies for the local retrieval capabilities required during search. To address this mismatch, we introduce latent anchor reasoning, which consists of resolving an unnamed retrieval anchor from descriptive specifications and transferring the recovered anchor into a subsequent information demand. This primitive retrieval unit decomposes deep search into chains of coupled operations and organizes question construction around anchor resolution and relation transfer, without prescribing a canonical search path. Based on this formulation, we propose PrimeSeeker, a capability-oriented framework that constructs web-grounded anchor structures and jointly derives a question and a reference evidence skeleton. The skeleton preserves supporting evidence from construction and guides expert generation through extractive highlights of current tool observations. These highlights are removed before supervised fine-tuning, while the skeleton is subsequently reused to audit reference-step coverage for reinforcement-learning rewards. We construct 9,221 expert trajectories, training a 30B search agent. Across five deep-search benchmarks, PrimeSeeker achieves strong performance, while reference-step optimization further improves the supervised policy. The resulting trajectories exhibit low retrieval redundancy, and fixed-budget evaluation shows strong solution coverage with substantially fewer tool calls than long-horizon systems.

1 Introduction

The introduction argues that task difficulty and trajectory structure do not specify the local search capability being supervised. PrimeSeeker organizes supervision around latent anchor reasoning and reuses a construction-derived evidence skeleton across expert generation and policy optimization.

1 Introduction

  • Task difficulty and trajectory properties are indirect proxies that cannot specify the reusable search capability being supervised.Complex questions may be solved with parametric knowledge or one query, while short questions may require many steps and an unnamed target.Source: §1 · p. 2
  • Latent anchor reasoning couples recovery of an unnamed anchor from descriptive clues with using that anchor to formulate a subsequent information demand.These local operations compose deep-search problems without prescribing a canonical question or search path.Source: §1 · p. 2, Figure 1 · p. 2
  • PrimeSeeker builds web-grounded anchor structures that jointly yield a question and a reference evidence skeleton for expert generation and later reference-step coverage auditing.The skeleton highlights evidence only after it appears in the current observation; its guidance is removed before supervised fine-tuning.Source: §1 · pp. 2–3
  • The framework produces 9,221 expert trajectories, and reference-aligned skeleton guidance improves expert accuracy across two expert models.Random observation-derived highlights sharply degrade performance.Source: §1 · pp. 2–3
  • Across five deep-search benchmarks, PrimeSeeker-Dq consistently improves over PrimeSeeker-SFT, while fixed-budget evaluation matches or exceeds strong long-horizon systems with substantially fewer tool calls.The demonstrations also show lower duplicate-search and duplicate-visit rates than compared search-agent corpora.Source: §1 · p. 3
  • The formulation provides an operational basis for organizing search-task construction around latent anchor resolution and relation transfer.The skeleton supplies consistent supervision from construction to policy optimization while preserving diverse valid search strategies.Source: §1 · p. 3

2 Problem Formulation

The formulation treats deep search as a composition of local retrieval capabilities grounded in external evidence. Each unit resolves a possibly unnamed anchor from specifications, then transfers that anchor into a relation-conditioned information demand.

2 Problem Formulation

  • Deep-search supervision targets local retrieval capabilities rather than the length or global structure of a complete trajectory.The task includes a question, retrieval environment, gold answer, and task-relevant evidence structure.Source: §2 · p. 3
  • The evidence structure groups retrieval anchors, their descriptive specifications, queryable relations, and external webpages supporting those specifications and relations.Anchors may be named entities, rules, attribute slots, or other intermediate objects enabling subsequent retrieval.Source: §2 · p. 3
  • Latent anchor resolution identifies an explicit anchor from descriptive specifications, while relation transfer uses that anchor to formulate a conditioned information demand.Resolution can use prior knowledge, a discriminative query, or iterative evidence comparison; an already explicit anchor is a degenerate case.Source: §2 · pp. 3–4
  • The PrimeSeeker pipeline derives a question and reference evidence skeleton from an anchor structure, then uses the skeleton for observation-grounded expert guidance and later coverage signals.The skeleton is removed before supervised fine-tuning and remains external to the policy.Source: Figure 2 · p. 4
  • Relation transfer can yield the final answer, an explicit next anchor, or specifications for a new latent anchor, composing multiple retrieval units.This semantic migration connects local operations into a deep-search task.Source: §2 · p. 4

3 Method

PrimeSeeker builds web-grounded anchor structures into questions and reference evidence skeletons, then uses the skeleton to guide expert generation and optionally audit coverage during reinforcement learning. The method structures supervision around local retrieval evidence without prescribing a search path.

3.1 Capability-oriented QA Construction

  • Questions conceal anchors behind validated descriptive specifications and retain relation demands, while reference steps track supporting evidence from intermediate anchors to the target answer without prescribing retrieval actions or order.Specifications are sourced from independent webpages, checked for support, novelty, relevance, and ambiguity, and retain provenance.Source: §3.1 · pp. 4–5

3.2 Dq-guided Expert Generation

  • Expert guidance highlights only relevant evidence already present in the current observation, excludes spans revealing the answer, and is removed before SFT; a verifier discards trajectories dependent on hidden annotations or unsupported actions.The guidance is extractive, and the expert retains responsibility for hypotheses, tool selection, and search path.Source: §3.2 · pp. 5–6

3.3 Student Policy Optimization

  • Reference-step coverage supplements final-answer correctness: intermediate evidence receives partial credit regardless of outcome, while answer-supporting evidence counts only on correct rollouts, with total process contribution bounded by C.The auditor requires evidence in the public interaction history; topical queries and unsupported claims do not establish coverage.Source: §3.3 · p. 6

4 Experiments

PrimeSeeker is evaluated on five deep-search benchmarks, under varying and shared interaction budgets, and through analyses of its expert trajectories. Results report improved benchmark performance and solution coverage, alongside efficient, low-redundancy demonstrations.

4.1 Experimental Setup

  • The evaluation covers BrowseComp-EN and -ZH, DeepSearch-2505 and -2510, and GAIA, with fixed 200-example subsets for both BrowseComp variants.Pass@1 is average accuracy, and Pass@3 is oracle solution coverage across three independent runs per question.Source: §4.1 · p. 7
  • The benchmark comparison includes advanced tool-augmented models and open-source search agents of at least 30B parameters; MiroThinker is evaluated separately under a shared interaction budget.PrimeSeeker variants use Qwen3-30B-A3B-Thinking-2507, with Dq guidance removed before training and reference-step coverage distinguishing the Dq reward from outcome-only RL.Source: §4.1 · p. 7

4.2 Main Benchmark Results

  • PrimeSeeker-Dq improves Pass@1 over PrimeSeeker-SFT on all five benchmarks and Pass@3 on four, with Pass@3 unchanged on BS-ZH.Compared with outcome-only RL, it improves both metrics on four benchmarks but underperforms on BS-ZH.Source: §4.2 · p. 7, Table 1 · p. 8
  • PrimeSeeker-Dq generally achieves higher Pass@3 at matched tool-call and context-token cutoffs, with most curves flattening around 60–80 calls and 64–96K tokens.Its solution-coverage gains are evident below the maximum resource allowance.Source: §4.2 · p. 7, Figure 3 · p. 8

4.3 Search Efficiency under a Shared Interaction Budget

  • Under the same interaction allowance, PrimeSeeker solves more questions on both datasets while consuming substantially fewer tool calls.The advantage is particularly pronounced on DS2510, where extended exploration becomes expensive.Source: §4.3 · p. 8, Table 2 · p. 9

4.4 Analysis of Constructed Expert Trajectories

  • PrimeSeeker has the lowest average tool-call count and duplicate-query and duplicate-visit rates among the compared corpora, while its average token length falls between DeepResearch-9K and OpenSeeker and REDSearcher.These statistics characterize the demonstrations as relatively compact in tool use and low in exact retrieval repetition.Source: §4.4 · p. 9, Table 3 · p. 9
  • Reference-aligned Dq highlights consistently improve expert accuracy across evaluated teacher settings, whereas random highlights substantially degrade accuracy and increase tool use.Qwen3 makes more calls with guidance, while DeepSeek-V4 makes fewer, a pattern described as consistent with more effective evidence use rather than uniform search compression.Source: §4.4 · p. 9

5 Related Work

Prior work advances deep search through agent architectures, training, and harder synthetic tasks, while PrimeSeeker focuses on the retrieval capability specified by each constructed problem. Its reference evidence skeleton also connects expert guidance with policy-optimization rewards.

Deep-search agents

  • Prior deep-search systems improve agent architectures, post-training, and inference-time search, while PrimeSeeker studies supervision for recovering an indirectly specified retrieval anchor and forming the next information demand.ReAct framed tool use as iterative reasoning, action, and observation.Source: §5 · p. 9

Synthetic search-task construction

  • Synthetic search pipelines increase scale, diversity, and difficulty through graph expansion, entity obfuscation, and dispersed evidence, but graph size, hop count, and trajectory length do not uniquely determine exercised retrieval capability.PrimeSeeker instead organizes construction around recovering a descriptively specified target and using it to support a subsequent relation demand, while allowing different search paths.Source: §5 · pp. 9–10

Expert supervision and process rewards

  • Where prior work improves demonstrations or reuses construction information for policy optimization, PrimeSeeker jointly derives Dq with each question and uses it for expert guidance and reference rewards.Auxiliary annotations are removed before supervised fine-tuning.Source: §5 · p. 10

6 Conclusion

PrimeSeeker formulates deep search around latent-anchor resolution and relation transfer, then reuses construction-derived evidence structures across question construction, expert generation, and policy optimization. The appendix describes supporting implementation, analyses, examples, prompts, rollouts, artifacts, and limitations.

6 Conclusion

  • PrimeSeeker connects task construction, expert synthesis, and policy optimization through a shared representation of each question’s retrieval capabilities.Its construction-derived Dq gives strictly extractive, observation-grounded highlights during expert generation, is removed before SFT, and is reused to audit reference-step coverage during RL.Source: §6 · p. 10
  • Using 9,221 generated trajectories, PrimeSeeker achieves strong performance across five deep-search benchmarks.The conclusion also reports low retrieval redundancy and strong solution coverage without reliance on extended search trajectories.Source: §6 · p. 10

A Appendix Overview

  • The appendix documents training and inference configurations, controlled expert-generation analyses, trajectory statistics, construction case studies, prompt templates, a test-time rollout, released artifacts, and limitations.These materials are organized across Appendix sections B–H.Source: A Appendix Overview · p. 14

B Training and Deployment Details

PrimeSeeker is trained from a Qwen3 checkpoint on expert trajectories and correct Chinese OpenSeeker examples, then optimized with two RL variants. The implementation specifies shared web tools and separate rollout and benchmark-inference settings.

Supervised fine-tuning

  • The SFT corpus combines 9,221 PrimeSeeker trajectories with 902 correct Chinese OpenSeeker trajectories, yielding 9,589 training instances and 534 validation instances.Training runs for two epochs with a 196,608-token maximum sequence length and peak learning rate of 1 × 10^-5; tool observations remain context but are masked from the language-modeling loss.Source: B Training and Deployment Details · p. 14

Reinforcement learning

  • PrimeSeeker-Outcome and PrimeSeeker-Dq share an SFT initialization, questions, and GRPO setup; only Dq adds the reference-step coverage reward to final-answer correctness.Both train for 64 steps, with 16 questions and four rollouts per question at each step.Source: B Training and Deployment Details · p. 14

Rollout and inference

  • RL rollouts use temperature 1.0 and top-p 0.95, while benchmark evaluation uses temperature 0.8, top-p 0.95, and repetition penalty 1.05.Each trajectory is limited to 100 tool calls unless otherwise specified.Source: B Training and Deployment Details · p. 14

Tool environment and auxiliary models

  • Experiments use the OpenSeeker live-web environment, Serper for search, and Jina Reader for page fetching.Construction, extraction, verification, auditing, and answer evaluation use the same Qwen3-30B-A3B-Instruct-2507 deployment with a 230K-token context window.Source: B Training and Deployment Details · p. 14

C Effect of Observation-grounded Dq Guidance

Dq-guided highlights improve expert-generation accuracy, while random highlights reduce it, indicating that guidance quality depends on reference alignment.

C Effect of Observation-grounded Dq Guidance

D Trajectory Distribution Diagnostics

PrimeSeeker trajectories cluster in shorter-to-middle lengths and lower tool-call ranges, with lighter duplicate-retrieval tails than comparison corpora.

D Trajectory Distribution Diagnostics

  • PrimeSeeker occupies the short-to-middle trajectory range and has a lighter long tail than OpenSeeker and REDSearcher.DeepResearch-9K is concentrated in very short trajectories; REDSearcher is concentrated in the mid-to-long range.Source: D Trajectory Distribution Diagnostics · p. 15
  • PrimeSeeker and DeepResearch-9K concentrate on lower tool-call ranges, while OpenSeeker spans a broader middle range and REDSearcher uses heavier tool-call ranges.The DeepResearch-9K concentration near its maximum tool-call region may reflect a tool-budget boundary.Source: D Trajectory Distribution Diagnostics · p. 15
  • PrimeSeeker has lighter duplicate-search and duplicate-visit tails, providing distribution-level evidence of less repeated retrieval rather than premature termination alone.The CCDF axes plot duplicate count and the number of trajectories at or above that count; lower curves indicate lighter tails.Source: D Trajectory Distribution Diagnostics · p. 15
  • Together, the distributions show that PrimeSeeker does not rely on globally inflated trajectories and repeats retrieval less.Downstream utility is evaluated through trained-policy results in Section 4.2.Source: D Trajectory Distribution Diagnostics · p. 15

E QA Construction Case Study

The case study illustrates how shared web-grounded anchor structures yield aligned questions and evidence skeletons, alongside prompts for extracting observations and auditing reference-step coverage.

E QA Construction Case Study

  • The two construction examples derive aligned questions and reference evidence skeletons from shared web-grounded anchor structures, with specifications extracted from independently retrieved pages.Examples trace clues through recovered anchors and relations to an answer, including the Isle of Skye and Castruccio Castracani cases.Source: E QA Construction Case Study · pp. 15–17

F Prompt Templates

  • The released materials include resulting questions, reference evidence skeletons, trajectories, and source provenance, but not the exact upstream prompts for expanding anchors and verbalizing questions.Source: F Prompt Templates · p. 17

F.1 Key Observation Extraction

  • Key-observation extraction keeps only verbatim candidate highlights found in the current raw observation and removes direct matches to the gold answer.If no candidates survive validation, the original observation is passed to the expert without additional guidance.Source: §F.1 · p. 17

F.2 Reference-Step Coverage Auditing

  • Reference-step auditing checks coverage using public-trajectory evidence, records effective evidence and its role, and assesses reliability and redundancy.Evidence is classified by decision level and linked to next-action implications; answer-ready status requires sufficient public evidence.Source: §F.2 · pp. 19–20

G Test-Time Rollout Case Study

A representative BrowseComp-EN rollout answers an episode-identification question correctly in six tool calls.

G Test-Time Rollout Case Study

H Released Artifacts and Limitations

The release provides evaluation and tool-use materials, SFT and construction-verified RL data, provenance, and evaluation resources, while excluding exact upstream question-expansion and verbalization prompts. The section also flags constraints involving web changes, evaluator behavior, skeleton coverage, candidate exploration, and expert-model dependence.

H Released Artifacts and Limitations

  • The release includes evaluation code, tool-use implementation, 9,221 SFT questions and trajectories, construction-verified RL questions with skeletons, provenance, ExtractD and Extractk prompts, auditor, and evaluation configuration, but excludes exact upstream expansion and verbalization prompts.Source: H Released Artifacts and Limitations · p. 24
  • Web results and webpage contents may change, so reproducing an individual trajectory may yield observations different from those collected during construction or evaluation.Source: H Released Artifacts and Limitations · p. 24
  • Benchmark accuracy depends partly on answer-evaluator behavior; the evaluator prompt and code are released to make this variation explicit.Source: H Released Artifacts and Limitations · p. 24
  • The compact answer-supporting skeleton is not intended to enumerate every valid search path or supporting webpage.Source: H Released Artifacts and Limitations · p. 24
  • Sparse clues and extremely large candidate spaces may still require broader exploration despite latent anchor resolution's local formulation of candidate-space reduction.Source: H Released Artifacts and Limitations · p. 24
  • Closed-book filtering, trajectory generation, and evidence guidance depend on the capabilities and search styles of the selected expert models.Source: H Released Artifacts and Limitations · p. 24

Related papers

Loading 2609.35816v1…