PaperlayerView exact revision on arXiv

Source-linked AI summary

Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

Pengcheng Jiang, Zhiyi Shi, Kelly Hong, Xueqiang Xu, Jiashuo Sun, Jimeng Sun, Hammad Bashir, Jiawei Han

arXiv:2606.02373v1cs.AIcs.CLcs.IR

TL;DR

Search-agent reinforcement learning often makes the policy learn both semantic search and routine state management from growing transcripts. Harness-1 externalizes recoverable search state into a stateful harness while preserving semantic decisions for the policy, achieving 0.730 average curated recall across eight benchmarks and outperforming the next open subagent by +11.4 points.

  • Problem

    Retrieval-agent RL still requires policies to learn semantic search behavior and routine state management together from growing transcripts.

  • Method

    Harness-1 trains a 20B search agent with reinforcement learning in a stateful harness that maintains recoverable search state while the policy makes semantic decisions.

  • Results

    0.730 average curated recall across eight benchmarks made Harness-1 the strongest open retrieval subagent, improving over Tongyi DeepResearch 30B by +11.4 points.

  • Takeaways & Limitations

    The results point to stateful harness design as an important direction for retrieval-agent reinforcement learning beyond an implementation detail.

  • Takeaways & Limitations

    Harness-1 includes engineered components whose quality may vary across domains, including a regex-based extractor, an LLM verifier, and a sentence-level BM25 compressor.

Abstract

from arXiv · show

Search agents are often trained as policies over growing transcripts: the model must decide how to search while also remembering what it has seen, which evidence is useful, which constraints remain open, and which claims have actually been checked. We argue that this formulation puts too much routine state management inside the policy: reinforcement learning is forced to optimize both semantic search decisions and recoverable bookkeeping that the environment can maintain more reliably. We introduce Harness-1, a 20B search agent (retrieval subagent) trained with reinforcement learning inside a stateful search harness. The harness maintains environment-side working memory, including a candidate pool, an importance-tagged curated set, compact evidence links, verification records, compressed and deduplicated observations, and budget-aware context rendering. The policy retains the semantic decisions: what to search, which documents to keep or discard, what to verify, and when to stop. Across eight retrieval benchmarks spanning web, finance, patents, and multi-hop QA, Harness-1 achieves 0.730 average curated recall, outperforming the next strongest open search subagent by +11.4 points and remaining competitive with much larger frontier-model searchers. Its gains are especially strong on held-out transfer benchmarks, suggesting that reinforcement learning over explicit search state can produce retrieval behaviors that generalize beyond the training domains. Our code is available at https://github.com/pat-jj/harness-1.

1 Introduction

Harness-1 argues that retrieval-agent reinforcement learning should separate semantic search decisions from routine state management by externalizing recoverable search state to a persistent harness. The resulting 20B agent uses explicit curation, evidence, verification, compression, and budget tracking, achieving strong curated recall across diverse and held-out retrieval tasks.

  • Motivation: Retrieval-agent RL must learn semantic search behavior and routine state management simultaneously as transcripts grow.This forces the policy to reconstruct useful state, including evidence, unresolved constraints, and checked claims.
  • Core principle: Stateful cognitive offloading assigns semantic decisions to the policy and recoverable bookkeeping to the harness.The harness maintains candidate pools, curated evidence, cross-document links, verification records, and context-budget summaries.
  • System: Harness-1 is a 20B search agent whose actions edit persistent working memory rather than merely extending a transcript.Its working memory includes a candidate pool, curated set, search history, evidence graph summary, verification state, and context-budget marker.
  • Evaluation: 0.730 average curated recall is achieved across eight benchmarks spanning web, finance, patents, and long-context multi-hop search.The evaluation also reports improved downstream answer accuracy when the curated set is passed to frozen frontier generators.
  • Generalization: The same explicit-search-state operations improve retrieval across specialized corpora, open-web tasks, and held-out multi-hop QA benchmarks.This indicates that the gains extend beyond the training domains described in the evaluation.

2 Harness-1

Harness-1 separates semantic search decisions from recoverable bookkeeping by maintaining persistent, compact search state in a state-machine harness. Its actions, memory structures, curation controls, and training procedure let the policy search, select, verify, and terminate over that state rather than a raw transcript.

  • Harness architecture: Harness-1 runs each episode through a state-machine harness that renders compact search state, executes one structured action, updates WORKINGMEMORY, and produces the next observation.The harness maintains persistent state instead of merely appending tool outputs to the prompt.
  • Division of labor: The policy decides what to search, inspect, keep, verify, and when to stop, while the harness maintains recoverable state and applies state transitions.This division leaves semantic choices with the model and offloads routine bookkeeping to the environment.
  • Working memory: WORKINGMEMORY uses a prompt-facing tier for compact actionable state and an outer tier that stores full retrieved text and metadata for later review.The prompt therefore carries structured search state rather than the full retrieval transcript.
  • Action space and curation: The harness exposes retrieval, memory inspection, curation, verification, and termination actions, including a curated set capped at M=30 documents with four importance tags.The curate action adds, removes, and prioritizes documents, while auto-seeding initializes a blank set with the top k=8 reranked results assigned fair importance.
  • Training: Training uses SFT to teach harness operation and RL to improve search decisions over maintained state, with on-policy CISPO, full-trajectory rollouts, terminal-only reward, and a 40-turn cap.RL starts from the SFT checkpoint and trains on SEC queries without a KL anchor.

3 Experiments

Harness-1 achieves the strongest open retrieval performance across eight benchmarks, with especially large gains on held-out transfer tasks. Ablations and downstream evaluation indicate that explicit harness mechanisms improve evidence curation and answer quality.

  • Benchmarks & Data: Evaluation spans eight retrieval benchmarks across web, finance, patents, and multi-hop question answering.The benchmarks use Chroma-backed corpora and a shared Serper+Jina web backend.
  • Overall Results: 0.730 average curated recall makes Harness-1 the strongest open retrieval subagent, improving over Tongyi DeepResearch 30B by +11.4 points.Harness-1 also exceeds GPT-5.4, Sonnet-4.6, Kimi-K2.5, and GPT-OSS-120B, while Opus-4.6 remains ahead.
  • Transfer: +17.0 points mean gain on held-out transfer benchmarks exceeds the +7.9-point mean gain on source-family benchmarks.Held-out tasks exclude Harness-1’s SFT and RL data, whereas source-family gains are reported for BC+, Web, Patents, and SEC.
  • Inference-Time Ablation: Six of seven disabled mechanisms produce relative Final-Answer Recall drops ranging from −3.9% to −7.9%.Failures show increased searching and reduced reading or verification, indicating a shift toward wide, shallow search behavior.
  • Discovery and Selection: 0.658 versus 0.736 on BC+ Final-Answer Recall shows a selection gap against Opus-4.6 despite similar trajectory recall.Harness-1 generally discovers relevant evidence, but may fail to preserve the right documents in the final curated set.
  • Downstream Answer Quality: Harness-1 outperforms open subagents in modular RAG answer accuracy when its curated sets are passed to frozen frontier generators.The result links improved search curation to downstream answer quality in the reported modular RAG setting.

4 Conclusion · Contents of Appendix · A Related Work

Harness-1 separates semantic search decisions from mechanical bookkeeping by giving the environment explicit retrieval state, achieving the strongest average recall among evaluated open search agents. The conclusion frames stateful harnessing as a promising direction for retrieval-agent reinforcement learning and relates it to agentic search, tool orchestration, and harness engineering.

  • 4 Conclusion: Harness-1 trains a 20B search agent with reinforcement learning inside a stateful harness that maintains retrieval working memory.The policy handles search, curation, verification, and submission decisions, while the harness manages candidate pools, evidence, records, links, and context rendering.
  • 4 Conclusion: Across eight benchmarks, Harness-1 achieves the strongest average recall among evaluated open search agents while remaining competitive with much larger frontier-model searchers.Held-out transfer gains and component ablations suggest the harness is central to what the policy learns to use.
  • 4 Conclusion: Stateful harness design is identified as an important direction for retrieval-agent reinforcement learning, with future evidence graphs potentially using learned linking, extraction, and uncertainty organization.The proposed directions replace regex-based extraction with learned entity linking, relation extraction, and uncertainty-aware evidence organization.
  • Contents of Appendix: The appendices cover connections to agentic search, tool orchestration, and harness engineering, alongside limitations, ethics, and broader impact.Appendix A discusses related technical connections, while Appendix B addresses limitations, ethics, and broader impact.
  • A Related Work: Harness engineering treats the environment layer between a language model and its task as a major determinant of system-level performance.Prior work reports that the same model can vary by tens of points across harnesses, whereas tool orchestration typically fixes the interface and optimizes tool calling.
  • A Related Work: Harness-1 complements hand-designed and automatically searched harnesses by asking how a retrieval interface should be shaped for a trainable policy.Its answer is to offload mechanical search bookkeeping into environment-side state and train the policy to use that state.
  • A Related Work: Agentic search systems gather evidence through iterative query generation, document reading, and stopping decisions, often alternating reasoning with tool calls.ReAct, Self-Ask, and IRCoT established this alternating pattern, while reinforcement learning has increasingly been applied to retrieval and agentic search.
  • A Related Work: Unlike thin tool-wrapper harnesses that leave state reconstruction to the policy, Harness-1 makes the interface itself stateful and absorbs multi-turn search bookkeeping.The harness maintains environment-side working memory rather than exposing only a raw append-only observation stream.

B Limitations, Ethics, and Broader Impact · C Tool Signatures

Harness-1’s evidence-seeking scope, engineered components, and benchmark limitations constrain how its retrieval results should be interpreted. Its auditable evidence outputs may support research and analysis, but deployment requires access controls, monitoring, verification, and human oversight; the tool signatures expose retrieval, evidence management, verification, and termination operations.

  • B Limitations, Ethics, and Broader Impact: Harness-1 targets evidence-seeking retrieval and does not cover breadth-oriented research, open-ended report generation, abstention under missing evidence, or adversarial web environments.Most evaluated tasks are needle-in-a-haystack or multi-hop searches with annotated relevant evidence.
  • B Limitations, Ethics, and Broader Impact: Regex extraction, LLM-based verification, and sentence-level BM25 compression can fail across domains or remove context needed for discourse-dependent relevance.The verifier may err on ambiguous, highly technical, or underspecified claims.
  • B Limitations, Ethics, and Broader Impact: Benchmark size, annotation coverage, confidence intervals, near-duplicates, and incomplete qrels limit evaluation, so reported measurements characterize behavior under specified benchmark and harness conditions.The paper supplements recall-oriented metrics with multiple retrieval metrics, trajectory diagnostics, ablations, and modular RAG answer accuracy.
  • B Limitations, Ethics, and Broader Impact: The system’s ethical motivation is transparency and auditability through an explicit curated document set that users and downstream models can inspect and verify.This approach reduces reliance on the model’s parametric memory.
  • B Limitations, Ethics, and Broader Impact: Misuse risks include retrieving sensitive or inappropriate information and amplifying misleading evidence, requiring corpus controls, logging, rate limits, privacy filters, and human oversight.Risks increase when retrieval connects to unrestricted or low-quality, biased, or adversarial corpora.
  • B Limitations, Ethics, and Broader Impact: Harness-1 is not a source of truth: high-stakes outputs require qualified-expert review and validation against authoritative sources, and the release is not intended to replace expert judgment.The planned release includes the retrieval subagent, harness code, data-generation pipeline, evaluation recipe, intended-use documentation, and dataset provenance.
  • B Limitations, Ethics, and Broader Impact: Stateful retrieval agents may lower evidence-gathering costs and support literature review, legal and financial analysis, fact-checking, and education while making outputs more directly auditable.The benefit follows from separating retrieval from generation.
  • C Tool Signatures: The policy chooses tools for retrieving candidates, revisiting stored evidence, editing the curated set, verifying claims, and ending episodes, while the harness executes calls and updates working memory.Table 4 provides the functional signatures used in teacher, SFT, RL, and evaluation runs.

D Reward & Training Hyperparameters

This section specifies the deployed RL run’s reward configuration and the released checkpoint’s training recipe. SFT teaches operation of the stateful interface, after which RL runs over full search episodes with the same harness state renderer and tool set.

  • Reward configuration: Table 5 defines the deployed RL run’s reward weights and penalty coefficients by configuration name.The table is intended to specify the reward configuration used in the deployed run.
  • Reward configuration: At episode termination with an empty curated set, the reward short-circuits to π∅= −0.2 and bypasses the remaining formula.For other terminations, the §2.3 formula applies and is clipped below by R ≥10−3.
  • Training configuration: The released checkpoint reports separate hyperparameters for SFT, RL, the search environment, retrieval infrastructure, and evaluation to support reproducibility.The separation clarifies which settings belong to model optimization versus the harness and retrieval backend.
  • Training configuration: SFT teaches the stateful interface, while RL is applied over full search episodes using the same harness state renderer and tool set.Table 6 lists the exact settings used in the deployed run.

E Benchmark Statistics · F Two-Tier Memory and Working-Memory Rendering

The evaluation suite spans static and live retrieval benchmarks across encyclopedic web search, finance filings, patents, and multi-hop question answering, including source-family and held-out transfer splits. Harness-1 renders a two-tier memory with structured working-memory sections, compressed observations, deduplication, and progressive context-budget degradation.

  • E Benchmark Statistics: The benchmark suite spans static Chroma-backed corpora and live web-backed retrieval across encyclopedic search, finance filings, patents, and multi-hop question answering.The main-text split distinguishes source-family benchmarks from held-out transfer benchmarks.
  • E Benchmark Statistics: Table 7 reports each benchmark’s backend, test-query count, domain, and task style.
  • F Two-Tier Memory and Working-Memory Rendering: Harness-1 uses an inner WORKINGMEMORY summary that fits in the prompt and an outer doc store containing the full text of every retrieved chunk.The outer store is accessible through review docs, supporting reproducible rendering of the reported behavior.
  • F Two-Tier Memory and Working-Memory Rendering: The rendered WORKINGMEMORY begins with a query header and an importance-ordered curated set, with each entry showing a document ID and a 120-character snippet.Importance ordering is very high first, followed by high, fair, and low.
  • F Two-Tier Memory and Working-Memory Rendering: The document pool shows the most recent 50 uncurated documents, while older documents become an ID list truncated at 30 IDs with a (+N more) marker when longer.
  • F Two-Tier Memory and Working-Memory Rendering: Search history retains the last 12 entries; older searches are summarized as ... (k earlier searches).
  • F Two-Tier Memory and Working-Memory Rendering: When present, the evidence graph lists up to 8 frequent multi-document bridge entities with document IDs and a singleton count.The evidence-graph block appears only if at least one multi-doc bridge entity exists.
  • F Two-Tier Memory and Working-Memory Rendering: Per-turn observations combine WORKINGMEMORY, up to RECENT K=5 prior action-result pairs, and the newest tool result, with search results compressed to K=4 kept sentences and deduplicated.Prior-turn reasoning is truncated to 300 characters except for the most recent turn.

G Per-Turn Programmatic Nudges (Result Summaries) · H Harness Algorithms · I System Prompts and Templates

Harness-1 externalizes search bookkeeping through programmatic summaries and explicit working-memory algorithms, while preserving semantic decisions for the retrieval policy. Shared state interfaces, verification, prompt rendering, and stage-specific templates keep training and inference behavior aligned.

  • G Per-Turn Programmatic Nudges (Result Summaries): Programmatic result summaries combine factual status with conditional prescriptive nudges and are shown identically to the teacher, RL policy, and inference policy.The summaries are generated without an LLM call; prescriptive nudges were enabled for all reported runs and belong to the harness rather than intrinsic policy capability.
  • H Harness Algorithms: The stage driver initializes search state, renders context, samples or replays structured tool actions, executes tools, and processes observations across teacher, SFT, RL, and inference stages.The same mechanism-specific subroutines support trajectory generation, SFT replay, RL rollout, and inference, preserving a shared state interface.
  • H Harness Algorithms: The harness maintains a document pool, capped curated set, importance map, full-text store, evidence graph, verification cache, search history, deduplication index, and auto-seeding flag.The curated output set has cap M=30, while semantic choices remain with the policy and deterministic state maintenance remains with the harness.
  • H Harness Algorithms: Search observations retain the top K=4 BM25-scored sentences in original order, normalize document IDs, skip existing documents, and filter near-duplicates.Existing documents remain available for trajectory accounting even when they are not added again to the pool.
  • H Harness Algorithms: Importance-aware curation inserts candidates below capacity, retags existing documents, evicts lower-ranked items when full, and reports a [CAPACITY] marker when rejecting weaker additions.New items default to fair importance when the proposed label is invalid, and importance levels rank low as worst.
  • I.1 Agent System Prompt (Harness-1): The retrieval system prompt defines the agent as a document-finding subagent, while review returns up to 5 remembered documents and verification checks every claim constraint per document.Verification stores yes/no judgments with short rationales in the cache.
  • I.1 Agent System Prompt (Harness-1): Budget-safe rendering preserves curated items, search history, and the evidence graph during truncation, then progressively shortens reasoning, working memory, and older interaction pairs.The fallback minimal prompt retains only the system message, tool descriptions, and query.
  • I.3 Answer-Generator Prompt (modular RAG): The modular answer-generator prompt uses retrieved documents, while Closed-Book omits them and Naive RAG replaces the curated set with top-10 chunks from one hybrid query.The LLM judge uses GPT-5.4 at temperature 0 and reports strict CORRECT rate as answer accuracy.

J SFT Data Generation Recipe · K Importance-Aware Curation: Algorithmic Detail · L Content Deduplication: Algorithmic Detail

The paper generates harness-native SFT trajectories with GPT-5.4 and filters them into turn-conditional training data, while importance-aware curation and content deduplication maintain a compact, stateful document pool. Curation admits higher-importance documents through capacity-based eviction, and deduplication suppresses repeated content while preserving recall credit.

  • J SFT Data Generation Recipe: GPT-5.4 generates SFT trajectories inside the harness using the trained policy’s system prompt, observation layout, and tool schemas.The teacher uses tool choice=required and produces structured JSON arguments with a mandatory reasoning field.
  • J SFT Data Generation Recipe: Turn-level guidance enforces search→curate rhythms, verification, backtracking, tool diversity, and urgency near the turn cap.The protocol triggers verification after ≥6 turns and ≥3 curated documents, and backtracking after three searches yield ≤1 new document.
  • J SFT Data Generation Recipe: 899 trajectories remain after filtering raw dataset quotas with a 0.10 recall gate, expanding into ∼26K turn-conditional training examples.The raw quotas total ∼1K trajectories across BC+, SEC, Patents, Web, Web-simple, and SEC-simple.
  • K Importance-Aware Curation: Algorithmic Detail: Importance-aware curation ranks documents as very high, high, fair, or low, with lower numerical ranks indicating greater importance; auto-populated documents start at fair.The rank ordering is very high=0 < high=1 < fair=2 < low=3.
  • K Importance-Aware Curation: Algorithmic Detail: At capacity M=30, each incoming document replaces the current worst-ranked document only when its importance is higher; otherwise, the add is rejected with a [CAPACITY] marker.The marker lists up to 5 rejected IDs.
  • K Importance-Aware Curation: Algorithmic Detail: Existing curated documents can be re-tagged in place without eviction, allowing promotion after successful verification.The curated set is rendered grouped from very high to low importance with visible tags.
  • L Content Deduplication: Algorithmic Detail: MinHash-LSH deduplication tokenizes chunks, builds 5-gram shingles with 64 permutations, and drops matches at Jaccard threshold 0.85 while retaining recall credit.The tracker treats LSH matches as near-duplicates and preserves them in trajectory recall accounting.
  • L Content Deduplication: Algorithmic Detail: Without datasketch, SHA-1 over the first 4,000 normalized characters detects exact duplicates only, while WORKINGMEMORY reports suppressed duplicates with a [Dedup] diagnostic.The fallback avoids an extra dependency, and the diagnostic exposes the number of near-duplicate chunks auto-suppressed.

M Component Ablation: Error Analysis · M.1 Per-mechanism failure modes

The appendix analyzes paired BrowseComp+ trajectories to show that removing one state mechanism changes search behavior, producing wide, shallow retrieval that fails before relevant documents are reached. Across mechanisms, search corpus actions increase while reading and verification decline, indicating retrieval rather than curation failure.

  • M Component Ablation: Error Analysis: 100 BrowseComp+ test queries are analyzed without retraining, comparing each ablated condition against full Harness-1 on paired failures.Pairs include queries that full Harness-1 solves with FA Recall ≥0.5 but the ablated condition fails with FA Recall =0.
  • M.1 Per-mechanism failure modes: FA Recall ≥0.5 identifies queries solved by full Harness-1, while FA Recall =0 identifies corresponding ablation failures.The analysis is trajectory-level and focuses on paired queries meeting these outcome conditions.
  • M.1 Per-mechanism failure modes: Removing a single state mechanism changes policy behavior rather than merely removing information.The ablation analyses compare action mixes and trajectories on the same paired queries.
  • M.1 Per-mechanism failure modes: 3–7 points: the share of search corpus actions rises on failing paired queries after a single state mechanism is removed.This increase is accompanied by reduced reading and verification, reflecting a shift in action mix.
  • M.1 Per-mechanism failure modes: 2–6×: read document and verify actions drop on failing paired queries under single-mechanism ablations.The reported declines occur alongside increased search corpus actions.
  • M.1 Per-mechanism failure modes: The resulting policy reverts to a wide, shallow, search-dominated mode.This behavioral change is described consistently across the per-mechanism analyses.
  • M.1 Per-mechanism failure modes: The ablated policy never reaches the relevant documents in the first place, making the failure retrieval rather than curation.The trajectory pattern links reduced document reading and verification to failure before relevant evidence is reached.

M.1.1 Importance tags … M.2 All harness mechanisms disabled

Ablations show that Harness-1’s state-externalizing mechanisms collectively improve curated and fully answered recall by preserving importance signals, compressed evidence, comparison context, entity bridges, verification, review capacity, and deduplication. Disabling all mechanisms reduces Recall to 0.513 and FA recall to 0.624, confirming the harness’s central contribution.

  • M.1.1 Importance tags: ∆Recall −4.1%, ∆FA −7.9% relative: removing importance tags causes 15 hard fails and eliminates the confidence gradient needed to prioritize follow-up documents.On failing queries, read document drops 7× (4.7%→0.7%), verify drops to 0%, and search corpus rises to 94.1% of actions.
  • M.1.2 Sentence-BM25 compression: ∆Recall +0.2%, ∆FA −7.0% relative: removing sentence-BM25 compression produces longer, noisier returns and removes bridge sentences that seed follow-up reads.On failing queries, read document falls from 4.1% to 2.6% and verify from 2.0% to 0.4%.
  • M.1.3 Auto-seed: ∆Recall −0.3%, ∆FA −6.4% relative: an empty initial candidate set causes cold-start misselection before importance tags become defined.Aggregate Recall barely moves because the policy still curates the intended number of documents, while FA Recall falls during the cold-start window.
  • M.1.4 Evidence graph: ∆Recall −2.6%, ∆FA −5.4% relative: hiding the evidence graph removes entity-bridge structure, increasing search to 91.8% of actions versus 88.9% and reducing document reads 3×.The curated set lacks bridging entity chunks needed to close multi-document chains.
  • M.1.5 verify tool: ∆Recall −3.1%, ∆FA −3.9% relative: when verify is unavailable, read document drops from 5.1% to 0.8% on failing queries and search rises to 94.6%.Without verification, the policy loses its closed-loop check before committing documents to the curated set.
  • M.1.6 review docs tool: ∆Recall +2.4%, ∆FA −3.9% relative: removing review docs mildly raises aggregate Recall through extra turn-level reads but impairs economical scanning of partially informative documents.On failing queries, read document drops 4× (4.0%→1.1%) and search rises to 93.6%.
  • M.1.7 Content-fingerprint dedup: ∆Recall +4.6%, ∆FA +1.6% relative: disabling MinHash–LSH dedup is the only nominal metric improvement because redundant near-duplicate gold documents retain additional gold IDs.The improvement comes at the cost of inflating curated context with redundant chunks.
  • M.2 All harness mechanisms disabled: ∆Recall −12.2%, ∆FA −6.4% relative: disabling every mechanism reduces Recall to 0.513 and FA recall to 0.624.The policy keeps searching across raw documents but lacks compact state for ranking what it has seen.

N Evaluation Recipe · O Modular RAG: Curated Sets Yield Higher Answer Accuracy

The evaluation distinguishes answer-bearing, relevant, curated, and trajectory document sets to measure final retrieval, broader relevance, and discovery. Modular RAG isolates curated-set quality by giving a frozen generator only the query and curated documents, where stronger curated sets yield higher answer accuracy.

  • N Evaluation Recipe: Aq contains annotated answer-bearing documents, while Rq is the broader annotated relevant set and may include additional supporting documents.On BrowseComp+ and Web, answer-bearing documents are a subset of the broader relevant set.
  • N Evaluation Recipe: Cq is the final curated document set, whereas Pq contains every document encountered during the episode after chunk normalization.These sets support three recall-oriented metrics.
  • N Evaluation Recipe: Scores are macro-averages over queries, with fact-level datasets counting a fact as covered when any annotated chunk for it is retrieved.The same definitions apply to facts in place of documents.
  • N Evaluation Recipe: Final-Answer Recall is not constrained relative to Recall because the metrics can use different denominators when Aq is a subset of Rq.Recall averages all relevant annotations, while Final-Answer Recall averages only answer-bearing annotations.
  • N Evaluation Recipe: Trajectory Recall diagnoses discovery before curation and can exceed final Recall because encountered evidence may later be omitted from Cq.A system can have Final-Answer Recall above Recall when it finds answer documents but misses auxiliary supporting documents, or below it in the reverse case.
  • N Evaluation Recipe: Search-R1 and Tongyi DeepResearch outputs are standardized with a shared reranker because their released harnesses do not natively return importance-tagged curated sets capped at 30 documents.The protocol collects unique episode documents and normalizes chunks to document identifiers.
  • N Evaluation Recipe: Search-R1’s final-set construction usually changes retrieval little because its released harness typically performs only four to five text-search rounds and targets short trajectories.This setup rarely accumulates a trajectory pool much larger than the final document budget.
  • O Modular RAG: Curated Sets Yield Higher Answer Accuracy: Under modular RAG, the frozen frontier generator sees only the query and curated documents, so answer-accuracy differences between subagents are attributable to curated sets alone.Across all four generators, better curated sets broadly produce higher answer accuracy, while Closed-Book and Naive RAG are weak on BC+.

P Harness as Confound: Same LLM, Different Harness … Phase 2 - Second-hop search checks original SNL cast membership

The paper shows that richer stateful harnesses materially improve a fixed model’s retrieval, while qualitative cases demonstrate how curation, exact search, fan-out, and full-document reading resolve multi-hop queries. Across drug, SEC, entertainment, and held-out QA examples, the harness preserves useful evidence and supports accurate final answers.

  • P Harness as Confound: Same LLM, Different Harness: 0.849 Curated Recall and 0.876 Final-Answer Recall are achieved by GPT-5.4 with Harness-1, versus 0.511 and 0.612 under naive search-add.The progression is monotonic through Context-1, which reaches 0.807 Curated Recall and 0.821 Final-Answer Recall; switching from Context-1 to Harness-1 provides a +4.2 point recall gain without RL training.
  • Q Qualitative Case Studies: The harness changes search behavior by maintaining editable state, promoting evidence, recovering needles through exact search, removing false leads, and resolving answers through full-document reading.These mechanisms replace routine transcript bookkeeping with structured search-state management while leaving semantic search decisions to the policy.
  • Phase 2 - The policy pivots to pharmacokinetic evidence: The pharmacokinetic search retrieves a label reporting hepatic metabolism and a 4.3-hour mean half-life, adding high-value evidence to the curated set.The trajectory pivots from the regulatory anchor to the 2022-study clues and preserves both regulatory and pharmacology evidence.
  • Phase 3 - Exact search misses, so state prevents a blind loop; Phase 4 - Full-document reading confirms the final evidence: After the exact pattern “Maribavir.*Regulatory Project Manager” returns no results, an NDA-number search recovers Alicia Moruf in the approval letter.Full-document reading then confirms the final FDA evidence and retains the approval-letter chunk in curated memory.
  • Q.2 Case Study C: Exact-Date Recovery in SEC Filings; Phase 1 - Auto-seeding finds the right company and date; Phase 2 - Full-document reading keeps the answer grounded; Phase 3 - Exact grep adds corroborating filings: The SEC case uses auto-seeding, curation, full-document reading, and exact-date grep to identify Simply Good Foods’ CFO transition and corroborate the effective date July 3, 2025.The filings state that Christopher J. Bealer joined as Senior Vice President of Finance, succeeded Shaun P. Mara, and became CFO effective July 3, 2025.
  • Q.3 Case Study B: Fan-Out Search Resolves an Ambiguous Transfer Query; Phase 1 - Fan-out keeps competing hypotheses alive; Phase 3 - Exact search tests the candidate bridge; Phase 4 - Full-document reading exposes the answer sentence; Final curation update: The American Pie case keeps competing actor hypotheses alive, pivots to Eugene Levy, and reaches Dalhousie University with 1.0 curated recall and 1.0 final-answer recall.Wikipedia evidence identifies Levy as appearing in eight of the nine released films, while full-document reading exposes the commencement-address evidence.
  • Q.4 Case Study D: Held-Out Multi-Hop QA via Search-State Editing; Phase 1 - Joint search identifies the likely entity; Phase 2 - Second-hop search checks original SNL cast membership: The held-out SNL–Martin query is solved in 11 turns with tool diversity 5, 1.0 curated recall, and 1.0 final-answer recall by linking Stan Winters to Garrett Morris.The first-hop search return explicitly connects Stan Winters with Garrett Morris, establishing the likely entity for the second-hop cast-membership check.

Phase 3 - Full-document reading unifies both clues

Full-document reading of Garrett Morris’s article supplied biographical context and connected him to his role as Stan Winters on Martin. A targeted corpus search and related-document snippets reinforced that identification.

  • Full-document reading: The Garrett Morris document identifies him as an actor, comedian, and singer who was an original Saturday Night Live cast member.The article summary states that Morris was the first Black cast member and appeared from 1975 to 1980.
  • Clue unification: A corpus search for “Stan Winters” found that Garrett Morris played Stan Winters on Martin from 1992 to 1995.The search result also notes that the role ended after Morris suffered an injury.
  • Clue unification: The Martin article snippet independently lists Garrett Morris as Stan Winters in seasons 1–2 and as a guest in season 3.This related-document result provides a second source connecting Morris with the character.

Q.5 Case Study E: Backtracking after Rejecting the Right Entity · Phase 2 - A delayed read overturns the wrong branch

Harness-1 demonstrates delayed recovery after initially rejecting the correct game entity: a later read establishes that Drew Gehling voiced Gord in Bully, prompting state correction and successful verification of the PlayStation 2 release date. The case was solved in 40 turns with curated recall and final-answer recall both reaching 1.0.

  • Q.5 Case Study E: Backtracking after Rejecting the Right Entity: The query asks when the game featuring Drew Gehling as Gord was released for PlayStation 2, with the gold answer October 17, 2006.The case reports 40 turns and tool diversity 5.
  • Q.5 Case Study E: Backtracking after Rejecting the Right Entity: The case finishes with curated recall 1.0 and final-answer recall 1.0.The reported outcome is achieved after 40 turns with tool diversity 5.
  • Phase 2 - A delayed read overturns the wrong branch: A delayed read of Drew Gehling’s page overturns the branch by stating that he voiced Gord in the 2006 video game Bully.This read directly contradicts the earlier rejection.
  • Q.5 Case Study E: Backtracking after Rejecting the Right Entity: Bully remains retained at low importance after rejection, keeping the mistaken hypothesis visible while subsequent Gord searches fail to resolve the query.The curated state also retains Drew Gehling at high importance alongside low-importance Bully variants.
  • Phase 2 - A delayed read overturns the wrong branch: The harness backtracks by removing foreign-language Bully variants while retaining the English Bully page and corrected entity at high importance.The curated set becomes Drew Gehling[high], Bully[low], and Gord[low].
  • Q.5 Case Study E: Backtracking after Rejecting the Right Entity: After correction, searches target the right game, promote release-date evidence, and attempt verification over the curated candidate documents.The simple Bully page and Canis Canem Edit page become high-importance evidence.
  • Q.5 Case Study E: Backtracking after Rejecting the Right Entity: Exact-date grep is rejected because date-only matching returns unrelated October 17 pages, so the agent does not treat the date string alone as evidence.This preserves entity-linked verification rather than relying on an isolated date match.
  • Q.5 Case Study E: Backtracking after Rejecting the Right Entity: The initial search correctly surfaces Bully, but the policy rejects it as likely wrong and pursues a separate Gord interpretation.The mistaken rejection follows a search combining Drew Gehling, Gord, and PS2.

Phase 3 - The corrected entity drives the answer search · NeurIPS Paper Checklist

Phase 3 resolves Gord’s identity as Bully and redirects search to title-and-platform evidence for the PlayStation 2 release date. The checklist states that the paper’s claims should reflect its stateful retrieval harness, SFT/RL recipe, benchmark scope, assumptions, limitations, and expected generalization.

  • Phase 3 - The corrected entity drives the answer search: The identity facet is resolved: Gord is from Bully, so the policy stops searching generic Gord pages and uses the corrected title with platform/date constraints.This delayed correction redirects the search toward the release-date facet.
  • Phase 3 - The corrected entity drives the answer search: The corrected query is “Bully PlayStation 2 release date,” directly coupling the entity, platform, and requested date facet.The query operationalizes the corrected search path.
  • Phase 3 - The corrected entity drives the answer search: The retrieved evidence identifies Bully’s PlayStation 2 release as October 17, 2006 across game and Rockstar publication references.The Simple English game page gives 17 October 2006, while the publication list gives PlayStation 2, October 17, 2006.
  • Phase 3 - The corrected entity drives the answer search: The curated set is updated to 5/30, assigning high importance to corrected and supporting pages while retaining stale Gord and Bully variants at low importance.The set includes Canis Canem Edit, Drew Gehling, Simple Wikipedia/Bully, Bully, and Gord pages.
  • Phase 3 - The corrected entity drives the answer search: A final exact-date check is insufficient because date-only matching retrieves unrelated current-events pages, films, and titles.The agent therefore keeps title-and-platform evidence as the answer path rather than treating the date string alone as proof.
  • Phase 3 - The corrected entity drives the answer search: The state edit makes backtracking visible: the harness preserves an early low-importance hypothesis, then supports correction, stale-variant removal, and search redirection after contradiction.The correction is represented as an external-state change rather than only a private intermediate thought.
  • NeurIPS Paper Checklist: The checklist justification says the abstract and introduction present a stateful retrieval harness plus an SFT/RL recipe and report results across eight benchmarks.It also says the limitations paragraph qualifies the scope of those claims.
  • NeurIPS Paper Checklist: The checklist requires claims to match theoretical and experimental results and accurately communicate contributions, assumptions, limitations, and expected generalization.It warns that unclear or absent claim coverage in the abstract or introduction may be viewed negatively by reviewers.

2. Limitations

The paper acknowledges limitations in task scope, benchmark and component design, statistical reporting, and the current lack of open access to all release assets. It also discusses risks from unsupported or privacy-sensitive retrieval and specifies safeguards for the planned retrieval-subagent release.

  • Scope and methodology: The limitations include task scope, benchmark style and size, annotation coverage, and reliance on regex evidence graphs, LLM verification, and sentence-BM25 compression.These limitations are discussed in Appendix B.
  • Reproducibility and access: The planned release includes weights, harness code, data-generation tools, and the reinforcement-learning recipe, but the submission does not yet provide open public access to all assets.The appendix nevertheless provides reproducibility details for the experimental protocol.
  • Statistical reporting: The evaluation reports aggregate point estimates on fixed test sets without formal confidence intervals or significance tests for all main comparisons.Repeated-run evaluation would require many additional multi-turn rollouts and external retrieval or reranking calls.
  • Broader impact: The paper identifies negative risks from amplifying unsupported or privacy-sensitive retrieval alongside the potential positive impact of lower-cost evidence-seeking systems.It also describes the intended release scope.
  • Release safeguards: The planned release is limited to a retrieval subagent rather than a general autonomous web agent or answer generator, with documentation covering intended use, dataset provenance, and evaluation protocols.These are presented as safeguards for the introduced assets.
Loading 2606.02373v1…