PaperlayerView exact revision on arXiv

Source-linked AI summary

Incremental Open-Ended Deep Research with Structured Harness

Meilin Chen, Hongyuan Bao

arXiv:2610.11566v1cs.CL

TL;DR

Existing OEDR systems usually rebuild reports from scratch, although practical reports need periodic updates that preserve valid knowledge. Incremental-OEDR uses Structured Harness to selectively update prior reports; experiments report comparable or better quality alongside improved continuity and lower research costs.

  • Problem

    Existing OEDR systems primarily research each new report from the original query, despite practical reports needing periodic updates that preserve useful existing structure and content.

    Source: §1 · p. 2

  • Method

    Incremental-OEDR uses Structured Harness to reuse and update a prior report through structured representation, retrieval, persistent evidence storage, and selective generation.

    Source: §1 · p. 2

  • Results

    Across DeepResearch Bench and DeepConsult under OC and PC, Incremental-OEDR maintains comparable or better quality while improving continuity and reducing costs, with up to 61% lower search usage than OEDR on DeepResearch Bench.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    Incremental-OEDR is suited to reports requiring continuous maintenance, including technology and industry intelligence, financial and economic research, and scientific literature monitoring.

    Source: §6 · p. 9

    Temporal search restrictions control accessible webpages at each timestamp but do not simulate changes to existing webpage content; the evaluation also uses benchmarks not designed specifically for incremental maintenance.

    Source: §5 · p. 9

Abstract

from arXiv · show

Existing Open-Ended Deep Research (OEDR) systems primarily generate reports from scratch, making them inefficient for scenarios where research reports need to be continuously maintained as new information emerges. We introduce \textbf{Incremental Open-Ended Deep Research (Incremental-OEDR)}, a research setting that treats a report as an evolving research state and incrementally updates it by preserving valid knowledge, revising outdated or incomplete content, and incorporating newly available information. To support this setting, we propose \textbf{Structured Harness}, which represents reports as structured collections of outlines, sections, and supporting evidence, and provides structured retrieval, a persistent structured evidence pool, and structured generation for selective report updating and evidence reuse. We further establish a temporal evaluation framework spanning ten years, with \emph{Single-Step Task} and \emph{Long-Chain Task} to evaluate incremental updates over both individual transitions and long-term update chains. Extensive Experiments on DeepResearch Bench and DeepConsult under both the Open-source Configuration (OC) and Proprietary Configuration (PC) show that Incremental-OEDR maintains competitive report quality while substantially improving report continuity and reducing research costs. As shown in Figure~\ref{fig:profile}, it achieves up to 0.51 higher content-level ROUGE-L F1, 0.63 higher outline-level EM F1, 33\% lower token consumption, and 61\% fewer search calls than OEDR on DeepResearch Bench. For more details, please refer to our project page: https://ioedr-project.github.io/.

1 Introduction

Existing OEDR systems typically research each report from scratch, while practical reports need to evolve as information changes. Incremental-OEDR combines selective report updating with a temporal evaluation framework.

1 Introduction

  • Prior OEDR systems include fixed workflows guided by an initial outline and agents that dynamically plan, search, and refine research using intermediate findings.The workflow paradigm is predictable but less adaptable to intermediate findings as retrieved evidence grows; agent approaches offer greater flexibility for complex questions.Source: §1 · p. 2
  • Existing OEDR systems primarily generate reports independently, but incremental scenarios require preserving valid content while updating outdated or newly available knowledge.Rerunning the whole process can waste computation on unchanged content and produce structural differences despite only marginal knowledge changes.Source: §1 · p. 2
  • Incremental-OEDR updates a previous report by preserving valid knowledge, revising outdated knowledge, and incorporating newly discovered information.Structured Harness represents reports as outlines, sections, and associated evidence, with structured retrieval, a persistent evidence pool, and structured generation for selective editing and evidence reuse.Source: §1 · p. 2
  • A temporal framework evaluates incremental updates after one transition and across consecutive updates, measuring report quality, continuity, and research cost.The Single-Step Task and Long-Chain Task assess short-term and long-term report evolution, respectively.Source: §1 · p. 2
  • 0.51 higher content-level ROUGE-L F1 and 0.63 higher outline-level EM F1, with up to 33% lower token cost and 61% lower search usage than OEDR on DeepResearch Bench.Across DeepResearch Bench and DeepConsult under OC and PC, report quality is comparable or better while continuity improves and research costs fall.Source: §1 · pp. 2–3

2 Preliminary

An OEDR agent selects research actions in response to the query and accumulated context. The paper builds on ms-agent, which coordinates specialized agents and research artifacts.

2 Preliminary

  • An OEDR agent selects subsequent research actions based on intermediate findings and the research context available at each step.That context includes prior actions, observations, and accessible research artifacts.Source: §2 · p. 3
  • Actions can include searching, reading sources, revising a plan, delegating a subtask, or writing report content, with results added to context for later decisions.Source: §2 · p. 3
  • The paper extends ms-agent, an open-source OEDR agent in which a Researcher coordinates Searcher and Reporter subagents and uses file-system artifacts and evidence tools.The paper identifies ms-agent as its underlying agent for incremental research over previously generated reports.Source: §2 · p. 3

3 Incremental-OEDR

Incremental-OEDR evolves a report by carrying prior research into each update, while Structured Harness organizes reports and evidence to support selective retrieval, reuse, and editing.

3.1 Formulation

  • Incremental-OEDR makes the previous report part of the next research state, evolving reports over time instead of rebuilding them from each query.Its update process combines the query, prior report, and information available at the current timestep.Source: §3.1 · p. 3

3.2 Structured Harness

  • Structured Harness combines report structure, targeted retrieval, a persistent evidence pool, and selective generation to support incremental updates.The resulting structured research state can be reused in subsequent update steps.Source: §3.2 · p. 3, Figure 2 · p. 4
  • Reports are represented as outlines, section units, and their associated evidence, forming a reusable state for later updates.This organization aims to preserve existing report structures, content, and supporting evidence.Source: §3.2 · p. 3
  • Structured retrieval gives agents selected outlines, sections, and supporting citations according to their current information needs.This targeted view of accumulated research can guide subsequent external investigation.Source: §3.2 · p. 4
  • The persistent evidence pool refreshes previously used sources and stores newly discovered evidence for reuse in later updates.It records source URLs, titles, raw content, and update timestamps; a below-threshold character-level similarity triggers LLM claim-level diff analysis, while newly searched evidence is stored with metadata.Source: §3.2 · p. 4, Figure 3 · p. 5
  • Structured generation edits existing report sections selectively, preserving valid knowledge, revising outdated or incomplete content, and extending it with relevant new information.Unaffected content and the established report structure are preserved.Source: §3.2 · pp. 4–5

4 Experiments

The experiments evaluate incremental report updates over time and find that they preserve competitive quality while improving continuity and reducing research costs. Further analyses examine temporal variation, persistent evidence reuse, and update intervals.

4.1 Setups

  • Experiments span 2016–2025, use SST and LCT, and test open-source and proprietary model configurations.SST evaluates one transition from an OEDR-initialized report; LCT evaluates sequential updates from the preceding incremental report.Source: §4.1 · p. 5
  • The evaluation covers DeepResearch Bench and DeepConsult, measuring report quality, continuity at content and outline levels, and research cost.Quality uses benchmark-specific protocols; continuity uses text and heading similarity metrics, while cost tracks token consumption and search calls.Source: §4.1 · p. 6

4.2 Main Results

  • Across both benchmarks, Incremental-OEDR maintains competitive quality; on DeepResearch Bench, IOEDR(OC) scores 49.11 vs MS-Agent(OC)'s 48.75 in SST.The DeepConsult results show a similar pattern: IOEDR(OC) scores 6.77 in SST and 6.80 in LCT, versus 6.73 for MS-Agent.Source: §4.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7
  • On DeepResearch Bench, content-level ROUGE-L F1 is 0.57 and 0.68 for IOEDR(OC) in SST and LCT, versus 0.17 and 0.13 for MS-Agent(OC).At the outline level, IOEDR(OC) reaches 0.60 and 0.67 EM F1 in SST and LCT, compared with 0.04 for MS-Agent(OC).Source: §4.2 · p. 7, Table 3 · p. 8
  • On DeepResearch Bench, IOEDR(OC) reduces total tokens from 3.78M to 2.96M in SST and 2.52M in LCT, while search calls fall from 45.19 to 18.62 and 17.85.The overall results maintain quality across single-step and long-chain updates while improving continuity and reducing research costs.Source: Table 3 · p. 8, §4.2 · p. 8

4.3 More Analysis

  • OEDR performance varies across timestamps, and both OEDR and Incremental-OEDR generally improve over time on both benchmarks, so latest-timestamp evaluations may not fully characterize performance.On DeepResearch Bench, MS-Agent(OC) scores range from 47.57 in 2016 to 49.28 in 2025; on DeepConsult, its score rises from 6.31 to 7.27.Source: §4.2 · pp. 6–7, §4.3 · p. 8
  • In LCT, the persistent evidence pool lets updates reuse fetched evidence and refresh changed sources; on DeepResearch Bench, tokens fall 14.9% and searches 4.1% versus SST.Unlike SST, LCT starts with the preceding incremental report and its persistent evidence pool, which supports continued evidence reuse.Source: §4.3 · pp. 8–9
  • At a 10-year update interval, cost is 1.446× the OEDR baseline while quality is 0.984; quality and continuity largely plateau around one year.Gains from more frequent updates become increasingly marginal as intervals shorten, while cost continues to decrease with diminishing returns.Source: Table 4 · p. 9, §4.3 · p. 9

5 Limitations and Future Works

The evaluation is limited by temporal search controls that do not model evolving webpage content and by benchmarks not designed for incremental maintenance.

5 Limitations and Future Works

  • Tavily’s temporal search restriction controls which webpages are accessible at each timestamp but cannot simulate changes to webpage content over time.This limits how fully the evaluation represents temporal source evolution.Source: §5 · p. 9
  • The evaluation uses existing OEDR benchmarks that were not specifically designed for incremental report maintenance.The paper identifies dedicated benchmarks with temporally evolving scenarios and explicitly updated sources as future work.Source: §5 · p. 9
  • Developing dedicated Incremental-OEDR benchmarks with temporally evolving research scenarios and explicitly updated sources is an important direction for future work.Source: §5 · p. 9

6 Real-World Applications and Conclusions

The paper concludes that Incremental-OEDR and Structured Harness support continuous, efficient report updating while maintaining competitive quality and improving continuity. It identifies research requiring ongoing maintenance as a fitting use case.

6 Real-World Applications and Conclusions

  • Incremental-OEDR reuses report structure, content, and evidence to improve continuity and reduce research costs while maintaining competitive report quality.Source: §6 · p. 9
  • Incremental-OEDR is particularly suited to reports requiring continuous maintenance rather than one-time generation, including technology and industry intelligence, financial and economic research, and scientific literature monitoring.Source: §6 · p. 9
  • These applications share the need to preserve accumulated knowledge while selectively incorporating newly available information.The paper presents this shared need as highlighting Incremental-OEDR’s potential as a practical framework for sustained research maintenance.Source: §6 · p. 9

A.1 Open-Ended Deep Research

Open-Ended Deep Research aims to produce detailed, well-supported reports across broad information spaces, unlike agents built to resolve well-defined queries. Existing OEDR systems use workflow or agent paradigms, while this work studies incremental research and prior-report reuse.

A.1 Open-Ended Deep Research

  • OEDR extends beyond resolving an individual question to investigating a broad information space and constructing a detailed, well-supported report.Web-based research agents are generally designed to gather evidence for well-defined queries and are predominantly assessed on question-answering benchmarks.Source: §A.1 · p. 12
  • Workflow-based OEDR uses predefined pipelines, whereas agent-based OEDR dynamically plans, decomposes, and iteratively executes research steps.Source: §A.1 · p. 12
  • Existing OEDR systems primarily address one-off research, while this work introduces Incremental OEDR and Structured Harness for selectively retrieving and reusing prior report content.Source: §A.1 · p. 12

A.2 From prompts to agent harnesses

Agent harnesses provide execution environments that shape how models act and interact with tools. Structured Harness applies this broader idea to incremental OEDR by organizing persistent report and evidence states for successive updates.

A.2 From prompts to agent harnesses

  • Agent harnesses add an execution environment in which models act, observe outcomes, invoke tools, receive feedback, and operate under runtime constraints.The passage notes their use in long-horizon interactive reasoning and software engineering.Source: §A.2 · p. 12
  • Structured Harness targets incremental OEDR rather than introducing a new agent architecture or model.Source: §A.2 · p. 12
  • It organizes persistent report and evidence states and provides mechanisms to retrieve prior research, refresh evidence, and selectively update report components across successive updates.Source: §A.2 · p. 12

B.1 Details of Structured Harness

B.1 describes three collaborating agents that divide coordination, search, and report construction, alongside Structured Harness tools for reusing and updating prior research.

B.1 Details of Structured Harness

  • The Researcher coordinates planning and delegation, the Searcher retrieves external information and evidence, and the Reporter constructs and updates the report.Source: §B.1 · p. 12
  • Structured Harness adds retrieval, evidence-pool, and generation operations, including prior-report access and batch evidence refresh with diffs.Its generation tools support updating report outlines and chapters.Source: §B.1 · p. 12
  • Table 5 lists tools and functions available to the three agents and Structured Harness; shaded rows denote Harness-introduced tools.Source: Table 5 · p. 13, §B.1 · p. 13

B.2 Prompts

The prompts define evidence-grounded research and report-writing workflows that plan and gather information, refresh existing content while preserving supported material, and verify the assembled report.

Primary Responsibilities

  • The Researcher plans and orchestrates the task, delegates evidence retrieval to the Searcher, and sends the completed research to the Reporter for report generation.The prompts specify each agent’s roles, responsibilities, tools, and interaction protocols.Source: §B.2 · pp. 13–14

Reference Workflow

  • The refresh workflow preserves the prior report’s outline, refreshes its cited evidence once, and searches only for genuinely new developments or missing analytical chapters.It explicitly warns against re-searching chapters whose sources remain unchanged.Source: §B.2 · p. 14

Task Planning and Research

  • Task planning turns the user’s intent into prioritized TODOs and a verification checklist, while research proceeds through delegated retrieval, interim synthesis, and stopping checks.The workflow stops when subtasks or core tasks are complete, further searching offers little benefit, or the iteration limit is reached.Source: §B.2 · pp. 14–15

Report Generation

  • After drafting, the Researcher checks factual accuracy, coverage, and citation alignment, then corrects problems at the source chapters and reassembles the report.The prompts prohibit hand-editing the assembled report because doing so can break citation and refresh consistency.Source: §B.2 · p. 15

B.2 Prompts

  • The research prompt requires an iterative search-and-evidence loop, with stopping based on plan completion, sufficient coverage, low marginal value, round limits, or inability to proceed.It calls for a progress summary covering findings, research progress, contradictions, and next steps.Source: §B.2 · p. 16
  • When evidence notes list multiple URLs, each fact, datum, quote, or conclusion must carry a local source marker tied to its URL.Every listed URL must be cited at least once, and markers cannot exceed the number of sources.Source: §B.2 · p. 17
  • Refresh instructions treat the prior chapter prose as the editing baseline and combine it with per-chapter evidence-update notes and fresh searcher notes.The writer is instructed to load the evidence index and read the chapter before revising it.Source: §B.2 · p. 18
  • Only claims supported solely by dead sources may be dropped or hedged; unchanged and unverified claims stay verbatim, while changed claims are updated and genuinely new material is added.The prompt treats a failed re-fetch as unverified rather than retraction, and prohibits rewriting stable paragraphs for style.Source: §B.2 · p. 18
  • The workflow maps evidence to a compact outline, drafts chapters progressively, and records conflicts or structural changes before proceeding.Researchers prepare evidence bundles, re-rank evidence before committing chapters, then sanity-check for unsupported claims, contradictions, and scope mismatches.Source: §B.2 · pp. 18–19
  • Final reports are assembled mechanically from structured chapters, then reviewed and corrected at the chapter level before reassembly.The assembler resolves note tokens into numbered citations and appends unified references; unresolved tokens or other issues require chapter edits rather than rewriting the assembled report.Source: §B.2 · p. 19
  • The final report must remain a polished, self-contained document with concise headings, while execution status and artifact paths belong in the required JSON output.The prompts prohibit overwriting assembled report files and disallow meta-text in the report body.Source: §B.2 · pp. 19–20
  • Evidence should be filtered by relevance, source quality, timeliness, consistency, and citability, while factual claims remain evidence-backed and conflicts are explicitly recorded.When evidence is missing, the prompts prohibit filling gaps with assumptions or large-scale external retrieval; reports should state insufficiency and track visible publication or update dates.Source: §B.2 · p. 19
  • Chapter claims must pass through their note-specific citation tokens verbatim, while assembled citation numbers and reference mappings must remain unchanged.The prompts forbid invented tokens, bare integer markers in chapter drafts, inline bare URLs, and chapter-level reference lists.Source: §B.2 · p. 20
  • Chapter bodies should use continuous prose that synthesizes related evidence, reserving bullets for inherently enumerative content and headings for thematic structure.The default style calls for careful, verifiable research prose, cohesive paragraphs, and readable heading hierarchy.Source: §B.2 · p. 20
Loading 2610.11566v1…