PaperlayerView exact revision on arXiv

Source-linked AI summary

Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries

Anikait Singh, Teresa Zhang, Yoonho Lee, Roshen Sanjay Nair, Aviral Kumar, Chelsea Finn

arXiv:2610.22273v1cs.LGcs.AIcs.CL

TL;DR

Long-context question answering must locate information dispersed across documents, while reactive retrieval repeats the search for each query and rigid precomputed structures may not suit task-specific artifacts. Librarian Agents trains a builder to construct a reusable filesystem workspace before queries arrive, with reported accuracy gains on BrowseComp-Plus and unseen-domain BRIGHT.

  • Problem

    Information needed for long-context tasks is dispersed across documents, and retrieval remains reactive while precomputed structures can rely on rigid abstractions.

    Source: §1 · p. 1

  • Method

    Librarian Agents trains a builder to reorganize source documents into a filesystem workspace before queries, optimizing workspace construction as an MDP with reinforcement learning for downstream answering.

    Source: §1 · p. 2, §2 · p. 2, §2.1 · p. 3

  • Results

    Librarian Agents improves average accuracy by 33 percentage points over the best-performing baseline on BrowseComp-Plus and by 31 points over baselines on unseen-domain BRIGHT.

    Source: §1 · p. 2, §4 · p. 11

  • Takeaways & Limitations

    Upfront workspace construction can shift costs before queries and amortize them across many downstream queries, with improvements reported on in-domain and out-of-distribution evaluations.

    Source: §1 · p. 2, §4 · p. 11

    Evaluation is limited to question answering over static corpora, and maintaining anticipatory memories as documents change remains open.

    Source: §4 · p. 11

Abstract

from arXiv · show

Reasoning over long document contexts remains a challenge for large language models (LLMs). Standard retrieval methods are inherently reactive, bottlenecking agents with complex, cross-document reasoning at inference time, while precomputed structures like knowledge graphs often fall short because they rely on rigid heuristics. Instead, can we allow agents to perform anticipatory directed curation: learning to proactively reorganize unstructured corpora prior to inference to efficiently serve diverse future queries. We build on a key asymmetry in modern agentic harnesses (e.g., Claude Code): while LLM agents are adept at navigating existing filesystems to solve an immediate task, they are rarely trained to construct reusable workspaces for future use. To instantiate this capability, we introduce Librarian Agents, a framework in which a builder agent transforms raw documents into a hierarchical filesystem of synthetic files and reference links, which forms a workspace tailored for a downstream answering agent. We formulate workspace construction as a Markov decision process and optimize the builder policy via reinforcement learning to directly maximize downstream answering accuracy. Empirically, Librarian Agents outperforms the strongest retrieval and precomputation baselines by 33 percentage points on BrowseComp-Plus, amortizes upfront construction costs across repeated queries, and generalizes zero-shot to unseen domains on BRIGHT with a 31-point average accuracy improvement.

1. Introduction

Long-context question answering must find evidence dispersed across documents, while existing retrieval and precomputed structures leave gaps in how they support future queries. Librarian Agents trains a builder to create a reusable filesystem workspace, improving results on BrowseComp-Plus and unseen-domain BRIGHT.

1. Introduction

  • Information needed for a task is dispersed across sources, leading agents to repeatedly revisit the corpus for each request.Source: §1 · p. 1
  • Reactive retrieval rediscovers cross-document relationships per query, while rigid precomputed abstractions may not fit queries needing new task-specific artifacts.Knowledge graphs are one example of pre-structuring the search space.Source: §1 · p. 1
  • Librarian Agents trains a builder to create a filesystem workspace of synthetic files and directories with references to source documents for a downstream answering agent.The workspace is built once and reused for different queries; construction is formalized as an MDP and optimized with reinforcement learning for downstream answering performance.Source: §1 · p. 2, Figure 1 · p. 2
  • 33 percentage points over the best-performing baseline on BrowseComp-Plus and 31 points on unseen-domain BRIGHT are the reported average accuracy improvements.The builder is untrained on BRIGHT.Source: §1 · p. 2

2. Librarian Agents: Agent Curated Filesystems as Anticipatory Workspaces

This section describes how Librarian Agents builds reusable, hierarchically organized workspaces from raw documents before queries arrive, and how construction is trained to support downstream answering. It covers workspace design and evaluation, including in-domain results, BRIGHT transfer, amortized construction costs, and navigability when synthetic coverage is incomplete.

2.1. Problem Setup.

  • Anticipatory workspace construction turns a flat corpus into a multi-level filesystem of synthetic files and source references, built before downstream queries to reduce repeated rediscovery.The builder’s construction cost is intended to be reused across queries.Source: §2 · pp. 2–3, §2.1 · p. 3, §2.2 · p. 4

2.2. Desiderata for Anticipatory Workspaces.

  • An effective workspace should cover likely queries with shallow, legible navigation, preserve links to raw documents for verification or fallback, and amortize construction across queries.Raw-source fallback matters for rare or highly localized facts that synthetic summaries may omit.Source: §2.2 · pp. 3–4

2.3. Workspace MDP Formalization.

  • The builder’s filesystem edits form a finite-horizon MDP, and the general objective maximizes expected downstream utility while penalizing construction cost.The workspace state includes raw documents and synthetic files, folders, and references; actions include creating, merging, deleting synthetic artifacts, or stopping, while raw documents remain available.Source: §2.3 · pp. 4–5

2.4. Workspace Augmentations via RL.

  • The builder is optimized with GRPO using periodic answerability-delta probes and terminal answer correctness, while the answering agent is fixed by default.The primary configuration probes every eight turns and does not explicitly shape rewards for navigation, filesystem structure, or construction cost; training queries are hidden from the builder.Source: §2.3 · p. 6, §2.4 · pp. 6–7
  • On BrowseComp-Plus, Librarian Agents reaches accuracy 0.75 (vs. Embedding 0.42 and Reranker 0.44) with Qwen3.5-4B, and 0.77 (vs. 0.30 and 0.42) with Gemini 3.1 Flash Lite; RL fine-tuning beats No FT.The reported comparisons are against reactive and precomputation baselines; the no-fine-tuning ablation underperforms the RL-finetuned builder.Source: Figure 4 · p. 8, §2.4 · p. 8
  • Zero-shot on BRIGHT, Librarian Agents achieves average pass@1 accuracy 0.52 and pass@3 accuracy 0.67 across three datasets, outperforming baselines.The builder was trained only on BrowseComp and evaluated across 150 questions in three domains, with one filesystem per domain.Source: Table 3 · p. 9, §2.4 · p. 9
  • At 10 queries per corpus, RL uses 33.8% fewer recorded tokens per expected correct answer than Base (ratio 0.662, 95% CI [0.504, 0.836]).With 443 frozen synthetic queries across 50 document sets, RL-built filesystems achieve accuracy 0.64 versus 0.24 for Base at 9.3K answer-time tokens per query; projected total cost is roughly 18K tokens per query at 20 queries per set.Source: §2.4 · pp. 8–9, Table 2 · p. 8
  • On BRIGHT, the trained builder creates 53.4 active synthetic files, 45.4 clusters, and 8.0 merged artifacts on average, versus 4.8, 4.2, and 0.6 for the Base builder.These statistics span the three BRIGHT sub-domains.Source: §2.4 · p. 9
  • The trained builder combines broad topical coverage with hierarchical organization, expanding future query support while making fine-grained evidence easier to locate.The evaluation distinguishes answering-agent retrieval—synthetic files and tokens read—from builder structure, including active files, clusters, and merges.Source: §2.4 · p. 9
  • When synthetic coverage omits a needed detail, the answering agent can follow a provenance link to a raw file and recover the answer without enumerating the corpus.This robustness case concerns a BrowseComp+ query illustrated in Figure 7.Source: §2.4 · p. 9
  • On BrowseComp-Plus, the RL-trained builder creates more query-relevant artifacts and a more navigable organization than the base policy.The base policy produces a fragmented filesystem that is harder for the answering agent to navigate, while the trained builder surfaces relevant information in dedicated files.Source: §2.4 · pp. 9–10, Figure 5 · p. 10
  • The evaluation reports improved in-domain long-context reasoning, transfer to BRIGHT, amortized construction costs across repeated queries, and navigability when synthetic coverage is incomplete.The supplied passage gives no numerical results for these findings.Source: §2.4 · p. 10

3. Related Work

Prior work either retrieves evidence reactively for each query or precomputes corpus structure using rigid pipelines; Librarian Agents instead learn the organization of an external corpus for another agent’s future answers.

Retrieval-Augmented Generation

  • Reactive retrieval systems perform iterative searches after a query arrives, whereas Librarian Agents proactively synthesize a query-agnostic filesystem before inference.The filesystem approach front-loads corpus-level reasoning so its construction cost can be amortized across queries.Source: §3 · p. 10

Structured Indexes and Corpus-Level Organization

  • RAPTOR, GraphRAG, and HippoRAG precompute corpus structures, while Librarian Agents make file and hierarchy choices agentic and optimizable through downstream answering performance.The cited prior systems rely heavily on rule-based extraction pipelines or predefined schemas.Source: §3 · pp. 10–11

Long-Term Memory for Language Agents

  • Prior long-term-memory work mainly retains an agent’s own history, while Librarian Agents build memory over an external corpus for another agent to use.Source: §3 · p. 11

Compute Reallocation

  • Librarian Agents train a builder to organize the corpus as a policy for downstream answers, doing once work an answering agent would otherwise repeat for every query.The approach also exposes hyperparameters for weighing construction cost and query-time cost against answer accuracy.Source: §3 · p. 11

4. Discussion, Limitations, and Future Work

Librarian Agents trains a builder to reorganize context into a memory artifact optimized for future queries, which an answering agent can navigate. The evaluation is limited to static-corpus question answering, with maintenance and scalability challenges left open.

4. Discussion, Limitations, and Future Work

  • Librarian Agents uses a builder to create a synthetic filesystem that an answering agent can navigate, and improves over retrieval methods in in-domain and out-of-distribution evaluations.The framework instantiates anticipatory directed curation by reorganizing context into a memory artifact for future queries.Source: §4 · p. 11
  • The evaluation covers question answering over static corpora; maintaining anticipatory memories as documents change remains open.Source: §4 · p. 11
  • Large builder file contexts may limit scalability, while compression can discard details needed for needle-in-a-haystack tasks.Reference links provide fallback access, but deciding when to bypass the synthetic filesystem remains a challenge.Source: §4 · p. 11

A.1. Algorithm Overview of Librarian Agents

The training procedure builds workspaces through sampled edits and trains the builder policy using answerability-probe rewards and terminal answer scores. It updates the policy with GRPO over the resulting rollout.

A.1. Algorithm Overview of Librarian Agents

  • Each training iteration samples a hidden reward-query batch, initializes the filesystem from documents, and records builder edits as a rollout.Source: §A.1 · p. 16
  • At answerability-probe steps, the procedure estimates workspace answerability on the batch and assigns an answerability-delta reward.Source: §A.1 · p. 16
  • The rollout receives a terminal answer score, which is used with GRPO to update the builder policy.Source: §A.1 · p. 16

A.2. More details on experimental setup

The experiments evaluate reusable workspaces on BrowseComp-Plus and BRIGHT-derived question answering, with builders constructing filesystems without access to evaluation questions. The completed filesystem is reused across questions associated with its corpus or domain.

A.2. More details on experimental setup

  • BrowseComp-Plus training uses 680 corpus-level tasks and evaluation reserves 50 unseen tasks; builders see documents but not evaluation questions.The completed filesystem is reused for every question associated with its corpus, and correctness is failure-inclusive against fixed reference answers.Source: §A.2 · p. 16
  • BRIGHT transfer evaluates 150 questions across AOPS, Biology, and Stack Overflow, applying the BrowseComp-trained builder zero-shot without evaluation-question access.One shared filesystem per domain supports all questions in that domain; mean per-response accuracy and pass@3 are reported.Source: §A.2 · p. 16

A.3. Additional Results

Additional BRIGHT transfer results show improvement over the base model across nine further domains. The appendix also reports filesystem construction and answering-agent retrieval statistics for BRIGHT and amortized BrowseComp-Plus evaluations.

A.3. Additional Results

  • The appendix reports synthetic-file and token counts, along with filesystem breadth and depth statistics from the builder.For BRIGHT and amortized BrowseComp-Plus, the tables distinguish answering-agent files and tokens read from active synthetic-filesystem structure statistics.Source: §A.3 · p. 17, Table 4 · p. 17, Table 5 · p. 17
  • Librarian Agents outperforms the base model across all nine additional BRIGHT domains.Results use 20 questions per domain and are reported as percentages.Source: Table 6 · p. 17, §A.3 · p. 17

A.4. BrowseComp-Plus Reuse and Break-Even Analysis

The reuse analysis compares construction and answering costs across repeated BrowseComp-Plus queries, including costs normalized by answer quality. At ten questions per corpus, RL is more token-efficient per expected correct answer than the base builder, with a crossover estimated at 31 questions.

A.4. BrowseComp-Plus Reuse and Break-Even Analysis

  • Reuse cost is modeled as construction tokens per corpus multiplied by query count, plus answer-time workspace tokens per question.The analysis computes curves from frozen BrowseComp-Plus rollout logs and also reports cost divided by full-denominator accuracy.Source: §A.4 · p. 17
  • 33.8% fewer recorded tokens per expected correct answer: at ten questions per corpus, RL’s RL/Base ratio is 0.662 (paired-bootstrap 95% CI [0.504, 0.836]).The point crossover occurs at 31 questions per corpus, with a paired-bootstrap interval of 18–54 questions.Source: §A.4 · p. 17
  • Figure 6 plots recorded construction and navigation tokens per query, plus tokens per expected correct answer after accuracy normalization.Shaded regions show paired corpus-bootstrap 95% confidence intervals.Source: Figure 6 · p. 18
  • Because the RL answerer explores a richer workspace, its raw recorded tokens per query remain higher, so token counts are shown alongside accuracy-normalized costs.Source: §A.4 · p. 18

A.5. Filesystem-structure ablation.

Filesystem organization was tested by flattening or shuffling the same memories across 49 corpora and 435 expanded amortization questions. The interventions separate navigation cost from answer accuracy.

A.5. Filesystem-structure ablation.

  • Flattening preserves comparable answer accuracy but increases navigation cost, whereas shuffling semantic placement reduces accuracy without reducing navigation cost.The matched comparison uses completed snapshots from 49 corpora and 435 questions; Table 7 describes the evaluation scope and bootstrap method.Source: §A.5 · p. 18, Table 7 · p. 18

A.6. Qualitative Example of Recovery Using Reference Information

A qualitative example shows that provenance links can help an answering agent recover a correct answer when the synthetic filesystem omits a required detail.

A.6. Qualitative Example of Recovery Using Reference Information

  • When a synthetic file omits a required detail, the answering agent follows provenance links to the relevant raw source and recovers the correct answer.Source: §A.6 · p. 18

A.7. Librarian Agents Training and Evaluation Configuration

The appendix specifies the builder’s training and evaluation setup, filesystem tools, and agent prompts, then reports the training curve and artifact licenses. The builder is trained using downstream answerability and correctness as reward signals.

A.7. Librarian Agents Training and Evaluation Configuration

  • Figure 7 illustrates recovery when a synthetic filesystem omits information: the answering agent follows provenance links to the raw source.Source: Figure 7 · p. 19
  • The builder uses GRPO on 680 BrowseComp-Plus training tasks, with reward based on changes in answerability and terminal downstream correctness.The primary run uses no explicit filesystem-structure, navigation-cost, or construction-cost shaping; evaluation questions are withheld from construction and checkpoint selection.Source: §A.7 · pp. 18–19

A.8. Tool Definitions for Filesystem Action Space

  • Filesystem tools create synthetic files with source links, merge related files into hierarchical artifacts, delete unhelpful synthetic files, and stop construction while preserving raw documents.Tool operations are exposed to the builder through structured function calls.Source: §A.8 · pp. 19–20

A.9. Librarian Agents and Baseline Prompts

  • The answering prompt directs agents to start from root-level synthetic entries, explore deeper files when needed, and answer only when the workspace supplies sufficient evidence.Source: §A.9 · p. 20
  • Baseline prompts ground answers in retrieved documents and instruct the agent to avoid outside knowledge, assumptions, and unsupported explanations.Source: §A.9 · p. 21

A.10. Training Curve for Librarian Agents

  • Figure 8 shows aggregate rollout reward increasing during GRPO training as the builder learns decisions that produce more answerable downstream memories.Source: Figure 8 · p. 24, §A.10 · p. 24

A.11. License for Artifacts

  • The code, training scripts, and evaluation framework are designated for MIT licensing, while experiment artifacts and results are designated CC BY 4.0.The models and datasets remain subject to their respective licensing terms.Source: §A.11 · p. 24

Related papers

Loading 2610.22273v1…