Source-linked AI summary
FS-Researcher: Test-Time Scaling for Long-Horizon Research Tasks with File-System-Based Agents
Chiwei Zhu, Benfeng Xu, Mingxuan Du, Shaohan Wang, Xiaorui Wang, Zhendong Mao, Yongdong Zhang
TL;DR
Deep research trajectories can exceed model context limits, restricting evidence collection and report writing. FS-Researcher uses a dual-agent file-system workspace to build a persistent knowledge base and compose reports from it, achieving state-of-the-art quality across backbones while report quality rises with context-building computation.
Problem
Limited model context lengths constrain evidence gathering and report writing for long-horizon research tasks, often producing incomplete source coverage and lower-quality reports.
Method
FS-Researcher separates research into a Context Builder that creates a hierarchical knowledge base and a Report Writer that composes the report section by section from it.
Results
FS-Researcher achieves state-of-the-art report quality across different backbones, with report quality positively related to computation allocated to context building.
Takeaways & Limitations
The results support file-system-based persistent workspaces as a way to scale deep research beyond the context window through test-time computation.
Takeaways & Limitations
The framework depends on relatively strong foundation models, while smaller backbones may stop prematurely, require more sessions, and be more vulnerable to file-operation errors.
Abstract
from arXiv · showhide
Deep research is emerging as a representative long-horizon task for large language model (LLM) agents. However, long trajectories in deep research often exceed model context limits, compressing token budgets for both evidence collection and report writing, and preventing effective test-time scaling. We introduce FS-Researcher, a file-system-based, dual-agent framework that scales deep research beyond the context window via a persistent workspace. Specifically, a Context Builder agent acts as a librarian which browses the internet, writes structured notes, and archives raw sources into a hierarchical knowledge base that can grow far beyond context length. A Report Writer agent then composes the final report section by section, treating the knowledge base as the source of facts. In this framework, the file system serves as a durable external memory and a shared coordination medium across agents and sessions, enabling iterative refinement beyond the context window. Experiments on two open-ended benchmarks (DeepResearch Bench and DeepConsult) show that FS-Researcher achieves state-of-the-art report quality across different backbone models. Further analyses demonstrate a positive correlation between final report quality and the computation allocated to the Context Builder, validating effective test-time scaling under the file-system paradigm. The code and data are open-sourced at https://github.com/Ignoramus0817/FS-Researcher.
1 Introduction
Deep research requires extensive evidence collection and long-form synthesis, but model context limits constrain these activities. FS-Researcher addresses this with a persistent file-system workspace and separate agents for context building and report writing.
- Deep research involves navigating hundreds of webpages and producing reports exceeding 10K tokens, making it a long-horizon LLM-agent task.
- Limited context lengths compress token budgets for evidence gathering and report writing, causing incomplete source coverage and lower-quality reports in static or single-agent workflows.
- FS-Researcher separates evidence accumulation from report composition through a Context Builder librarian and a Report Writer that writes section by section from a hierarchical knowledge base.
- The file-system workspace stores information beyond the context window, preserves intermediate artifacts, and supports iterative refinement across sessions with negligible file-I/O latency.File I/O accounts for less than 0.03% of total wall-clock time.
- Experiments report state-of-the-art performance across backbone models and a positive relationship between final report quality and computation allocated to context building.
2 FS-Researcher
FS-Researcher uses two agents sharing a persistent file-system workspace: Context Builder constructs and iteratively refines a knowledge base, while Report Writer composes the report from it. The architecture combines tools, ReAct workflows, control files, and multi-session execution.
- Architecture: FS-Researcher divides research into Context Builder and Report Writer stages that share a workspace and independently refine deliverables across sessions.
- Architecture: Each agent follows a ReAct workflow in which the model produces thoughts and actions, tools execute actions, and observations feed subsequent steps.
- Workspace: The workspace stores Markdown deliverables and control files including todos, checklists, and execution logs for progress tracking and quality checks.
- Workspace: The framework supports multi-session execution by inspecting the workspace, updating task statuses, reviewing checklist compliance, and recording plans and findings in logs.
- Context Builder: The Context Builder browses and reads webpages, updates index.md, writes distilled notes to knowledge_base/, and archives raw webpages in sources/.
- Report Writer: After the knowledge base is complete, the Report Writer loses web-browsing tools and composes report.md section by section using the knowledge base as its only factual source.
3 Experiments
The experiments evaluate FS-Researcher on open-ended and answer-verifiable research benchmarks against proprietary and open-source systems. Across these settings, the framework improves report quality and information-seeking performance.
- FS-Researcher with Claude-Sonnet-4.5 reaches 53.94 RACE on DeepResearch Bench, outperforming RhinoInsight by +3.02.
- Under matched backbones, FS-Researcher improves RACE by +2.16 over LangChain Open Deep Research with GPT-5 and +1.59 over RhinoInsight with Gemini-2.5-Pro.
- On DeepConsult, FS-Researcher with Claude-Sonnet-4.5 achieves an 80.00% win rate, an 8.33 average score, and 9.58% losses.Claude results use a sampled 20-query evaluation because of a budget limit.
- FS-Researcher outperforms official agent harnesses on BrowseComp under both backbones, extending the comparison to objectively verifiable metrics.BrowseComp uses a random 100-query subset.
- Across three benchmarks, FS-Researcher improves research quality in both open-ended report generation and answer-verifiable information seeking.
4 Analyses
Analyses show that allocating more Context Builder computation generally expands the knowledge base and improves report quality, while exposing tradeoffs in readability and the importance of core modules.
- Scaling Effect of Context Builder: From 3 to 10 context-building rounds, archived information, notes, report length, and citations increase, with most knowledge-base gains occurring from 3 to 5 rounds.The source examples report +11.7 sources and +10.8 URLs from 3 to 5 rounds, versus +5.5 and +5.9 from 5 to 10.
- Scaling Effect of Context Builder: Increasing context-building rounds improves every reported quality dimension except Readability.
- Scaling Effect of Context Builder: Readability peaks at 5 rounds at 51.93 and drops slightly to 51.66 at 10 rounds.The paper attributes the tradeoff to denser, more technical writing from a larger knowledge base.
- Module Ablations: Removing the persistent workspace lowers RACE from 52.76 to 48.69, with the largest decline on Insight at -7.95.Comprehensiveness also declines by -3.58, while Instruction Following and Readability decrease less.
- Module Ablations: Merging the two agents lowers RACE from 52.76 to 42.41, including drops of -16.89 on Insight and -11.06 on Comprehensiveness.
- Module Ablations: One-shot report writing lowers RACE from 52.76 to 47.63, with declines across all dimensions and the largest drop on Insight at -8.80.
- Scaling Effect of Context Builder: From 5 to 10 rounds, archived sources increase from 54 to 59 while evidence notes increase from 75 to 98.The additional notes include more cross-source comparisons and analyses.
- Scaling Effect of Context Builder: Reports become progressively more evidence-grounded and modular, adding methodology, dimension-wise comparisons, and reusable knowledge-base modules as rounds increase.
5 Related Works
Related work includes proprietary and open-source deep research agents, while earlier open-source systems commonly use static pipelines or single-agent workflows that struggle with long horizons.
- Proprietary deep research products have demonstrated impressive, human-level performance, but their underlying techniques remain largely opaque.
- Open-source systems pursue reproducible workflows, often beginning with static pipelines or simple single-agent designs.
- As trajectories grow, limited context forces thoughts, observations, and report drafts to compete for tokens, contributing to incomplete source coverage.
6 Conclusion
FS-Researcher addresses long-horizon deep research with a persistent workspace and a two-stage dual-agent workflow. Experiments report state-of-the-art quality and a positive relationship between context-building computation and report quality.
- FS-Researcher scales long-horizon deep research beyond the context window through a persistent file-system workspace.
- The Context Builder collects and curates citation-grounded evidence, while the Report Writer composes the report section by section through on-demand workspace retrieval.
- Experiments on DeepResearch Bench and DeepConsult show state-of-the-art report quality across different backbone models.
- Report quality has a positive relationship with computation allocated to context building, supporting test-time scaling under the file-system paradigm.
Limitations
FS-Researcher depends on relatively strong foundation models because deep research and file-system operations demand substantial planning, search, writing, reasoning, and function-calling abilities.
- Smaller backbones may produce shorter trajectories, stop prematurely, and require more sessions to reach comparable coverage.They may also make incorrect file edits, inconsistent state updates, or erroneous tool-use decisions, reducing task success rates.
- $2.51 vs. $6.10/query: GPT-5-mini achieved comparable performance to OpenAI-DeepResearch with additional context-building rounds.The result is reported as a preliminary experiment and suggests usability beyond top-tier models.
- Supporting smaller models with a less demanding framework remains an important direction for future work.
Ethical Considerations
FS-Researcher raises information-quality, privacy, copyright, and security concerns because it relies on web content and persistently stores retrieved materials and intermediate notes.
- Information quality: Web-sourced content may remain inaccurate, biased, or outdated despite citation grounding, potentially misleading downstream decisions.
- Privacy and copyright: Persisting retrieved materials and intermediate notes may inadvertently store sensitive or copyrighted content.
- Security: Untrusted environments may expose the workspace to prompt injection or malicious pages that attempt to influence tool actions.
- Coverage: Round 2 retained missing UNECE, NHTSA, and non-US case-law materials because target endpoints were inaccessible to the fetch tool.The self-check marked exhaustive coverage as incomplete at that stage.
- Coverage: Round 3 marked all TODOs complete, but remaining UNECE/NHTSA primary PDFs were still inaccessible and had been cross-validated through summaries and official SGO definitions.
C Benchmark Details
DeepResearch Bench evaluates report quality and retrieval/citation reliability, while DeepConsult evaluates candidate reports through randomized pairwise judgments against a baseline.
- DeepResearch Bench: DeepResearch Bench measures report quality with RACE across Comprehensiveness, Insight/Depth, Instruction-Following, and Readability.RACE derives task-specific dimension weights by averaging weights across multiple trials.
- DeepResearch Bench: Sfinal(Rtgt) = Sint(Rtgt) Sint(Rtgt) + Sint(Rref).The supplied passage presents this as a DeepResearch Bench scoring equation, but does not define its symbols further.
- DeepResearch Bench: FACT evaluates citation reliability by judging whether extracted statement–URL pairs are supported by retrieved webpage text.It summarizes these decisions as Citation Accuracy and Effective Citations.
- DeepConsult: DeepConsult compares candidate reports with baseline references across Instruction Following, Comprehensiveness, Completeness, and Writing Quality.The protocol uses randomized A/B ordering and reports win, loss, or tie outcomes.
D Detailed Results of DeepConsult
FS-Researcher wins consistently against the DeepConsult baseline on instruction following, comprehensiveness, and completeness, while writing quality remains comparatively weaker.
- Overall comparison: FS-Researcher wins consistently across Instruction Following, Comprehensiveness, and Completeness with large margins over the baseline.The results indicate broader coverage and more thorough analyses on most queries.
- Overall comparison: Writing Quality is the relative weakness, leaving clarity, structure, and conciseness as areas for further improvement.The paper emphasizes improving these properties without sacrificing content coverage.
- GPT-5: Table 6 reports detailed DeepConsult pairwise comparisons for GPT-5 against the baseline reference report.
- Metric definition: Winrate is computed as win/(win+lose), excluding ties.
- Claude-Sonnet-4.5: Table 7 reports detailed DeepConsult pairwise comparisons for Claude-Sonnet-4.5 against the baseline reference report.
E Original Data for Scaling Figures
The appendix preserves the aggregated data behind the scaling figures and examines tool-call frequency across normalized trajectory positions in both research stages.
- Table 8 and Table 9 contain the original aggregated knowledge-base statistics and performance scores used to plot Figure 4.
- 6.10 → 8.16 → 12.54 $/query as context-building rounds increase from 3 to 5 to 10, reflecting greater test-time computation.The values are averaged over 10 sampled DeepResearch Bench queries.
- Tool-call frequency is analyzed by normalized trajectory position and tool type for the first three iterations of both stages.Color indicates row-normalized frequency.
- search_web generally appears earlier than read_webpage during Context Building, matching a search-then-read workflow.
- ls is concentrated near iteration starts, with smaller usage near iteration ends in both stages.This pattern is consistent with workspace-state checks and checklist-driven self-inspection.
H Case Study (3/5/10-round KB Growth and Its Impact on Reports)
The case study shows that larger context-building budgets produce larger, more detailed knowledge bases and different report content. Additional analyses examine harness effects, readability, compression, cost, and latency across these settings.
- KB Growth: 98 distilled notes and 59 archived sources are produced after 10 rounds, compared with 47 notes and 24 sources after 3 rounds.The 5-round setting contains 75 distilled notes and 55 archived sources.
- Report Impact: The 5-round report identifies Allianz, China Life, and Ping An Group as likely global asset leaders using rankings, capital strength, dividends, and buybacks.
- Report Impact: The 10-round report selects AIA, Allianz, and Manulife as the 2–3 companies most likely to rank high in future total assets and overall strength.Its justification includes multiple solvency and capital-coverage figures, including Allianz Solvency II 209% and AIA shareholder capital ratio 236%.
- Harness Comparison: +17.81 RACE is achieved by Gemini-2.5-Pro’s official deep-research harness over the bare model with only a search tool.FS-Researcher is evaluated as a third configuration in the same comparison.
- Backbone and Cost: 46.63 vs. 46.45 RACE is achieved by GPT-5-mini with FS-Researcher versus OpenAI-DeepResearch, at $2.51 vs. $6.10 per query.The comparison uses 10 context-building rounds on 10 sampled DeepResearch Bench queries.
- Readability: 51.66 → 51.92 readability is recovered by a post-processing rephrasing pass on the 10-round setting, with negligible impact on other dimensions.The pass is not included by default because its instructions were derived from benchmark-specific observations.
- Compression: 47% lower Context Builder cost ($3.77 → $2.00) is obtained through webpage compression, with negligible RACE change (51.18 → 51.14).Raw webpages are summarized by GPT-5-mini and only summaries are retained in the main agent context.
- Latency: <0.03% of total time is attributed to file I/O in both stages, while LLM inference and web browsing dominate latency.The profile covers 10 sampled queries.