PaperlayerView exact revision on arXiv

Source-linked AI summary

PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents

Zhuohan Gu, Qizheng Zhang, Omar Khattab, Samuel Madden

arXiv:2605.19932v1cs.AIcs.CLcs.LG

TL;DR

Repeated-context agents must rebuild reusable orientation knowledge from long external contexts, while existing persistence mechanisms retain noisy histories or task strategies. PEEK caches a compact context map and updates it from agent trajectories; across benchmarks, it improves quality while using fewer iterations and lower cost than ACE.

  • Problem

    Agents repeatedly rebuild transferable knowledge about a context’s contents, organization, entities, and navigation, while full interaction histories are noisy and difficult to reuse.

  • Method

    PEEK maintains a small prompt-resident context map, updating it after queries from execution trajectories and optionally freezing it for reuse.

  • Results

    PEEK improves context-learning solving rate by 6.0% and rubric accuracy by 9.9% over ACE, while leading quality at or near the fewest iteration levels.

  • Takeaways & Limitations

    A compact context map improves task correctness and efficiency across coarse and fine-grained evaluation metrics without relying on full online adaptation.

  • Takeaways & Limitations

    The map’s usefulness varies with how much reusable knowledge an agent’s context interaction reveals and with the agent itself.

Abstract

from arXiv · show

Large language model (LLM) agents increasingly operate over long and recurring external contexts, like document corpora and code repositories. Across invocations, existing approaches preserve either the agent's trajectory, passive access to raw material, or task-level strategies. None of them preserves what we argue is most needed for repeated same-context workloads: reusable orientation knowledge (e.g., what the context contains, how it is organized, and which entities, constants, and schemas have historically been useful) about the recurring context itself. We introduce PEEK, a system that caches and maintains this orientation knowledge as a context map: a small, constant-sized artifact in the agent's prompt that gives it a persistent peek into the external context. The map is maintained by a programmable cache policy with three modules: a Distiller that extracts transferable knowledge from inference-time signals, a Cartographer that translates it into structured edits, and a priority-based Evictor that enforces a fixed token budget. On long-context reasoning and information aggregation, PEEK improves over strong baselines by 6.3-34.0% while using 93-145 fewer iterations and incurring 1.7-5.8x lower cost than the state-of-the-art prompt-learning framework, ACE. On context learning, PEEK improves solving rate and rubric accuracy by 6.0-14.0% and 7.8-12.1%, respectively, at 1.4x lower cost than ACE. These gains generalize across LMs and agent architectures, including OpenAI Codex, a production-grade coding agent. Together, these results show that a context map helps long-context LLM agents interact with recurring external contexts more accurately and efficiently.

2 Background and Motivation

The paper frames context management across agent/task state versus recurring external-context state, and active versus passive maintenance. It motivates reusable orientation artifacts for agents repeatedly querying the same external corpus.

  • Design space: Context-management methods span Agent / Task State and External Context State, with active methods maintaining artifacts and passive methods carrying, retrieving, or summarizing state as needed.The external context is the recurring context itself, whereas agent/task state concerns execution or task behavior.
  • Related methods: Representative active agent/task-state methods learn prompts or maintain skills, plans, and cached answers, while passive methods carry chat history or compact execution records.These methods primarily preserve task-facing procedures, plans, answers, or transcripts rather than orientation knowledge about recurring external contexts.
  • Motivation: Repeated analysis of a 50k-entry feedback corpus could benefit from a lightweight table of contents, entity and constant memos, inspected-region records, and reusable intermediate answers.This reflects how a human might organize a collection after previous analysis rounds instead of restarting each question from scratch.

3 Context Map as a Cache

PEEK treats a constant-sized context map in the agent’s prompt as a cache for reusable orientation knowledge about a recurring external context. After each query, a programmable policy updates the map through Distillation, Cartography, and priority-based eviction, with updates optionally frozen after m queries.

  • 3 Context Map as a Cache: PEEK caches transferable orientation knowledge in a small prompt-resident map, rather than preserving a noisy interaction history.The cached knowledge concerns what the context contains, how it is organized, and which entities and concepts matter for future queries.
  • 3.2 How the Map Evolves: After each query, PEEK inspects the execution trajectory and updates the map; evolution can be frozen after as few as m=1 query.The map is prepended to every run, while updates occur only during the first m of n runs.
  • 3.2 How the Map Evolves: The Distiller diagnoses orientation versus task-specific work, tags candidate knowledge, and separates reusable learning from trajectory-specific facts.This separation prevents task-specific leakage into the cache.
  • 3.1 What Goes in the Map: The default map contains a Context Roadmap and Context Understanding, with optional Domain Constants, Reusable Results, and Parsing Schema sections.Maps begin nearly empty or blank, then accumulate structured contextual knowledge as needed.
  • 3.2 How the Map Evolves: The Cartographer applies minimal structured ADD, DELETE, or REPLACE edits, while the Evictor enforces a fixed token budget using accumulated priority scores.Cartographer updates are local and traceable through unique item identifiers; lower-priority entries are evicted first, with older entries breaking ties.

4 Experiments

Across long-context reasoning, information aggregation, and context-learning benchmarks, PEEK consistently improves quality over strong baselines while reducing iterations and cost. These gains persist across models and agent architectures, and ablations show that both context-map maintenance and a separated Distiller–Cartographer pipeline contribute to performance.

  • Generalization: PEEK generalizes when replacing the base LM or swapping the backbone agent, including GPT-5.5, Qwen3-Coder-Next-FP8, and Codex.RLM+PEEK can compete practically with much larger standalone frontier models on the CL-bench leaderboard.
  • Evaluation Setup: PEEK evaluates long-context reasoning and information aggregation on OOLONG’s three hardest splits and context learning on CL-bench, which reports solving rate and rubric accuracy.OOLONG requires identifying and aggregating distributed evidence; CL-bench covers domain knowledge, rules, procedures, and laws with up to 12 related tasks per context.
  • Quality Gains: +7.8–15.0%: PEEK beats ACE on OOLONG, while improving over ACE by +6.0% solving rate and +9.9% rubric accuracy on CL-bench.The gains span both coarse and fine-grained metrics, indicating improved context understanding across task types.
  • Efficiency: 93–145 fewer iterations: PEEK outperforms ACE on OOLONG while scoring 7.8–15.0% higher; on CL-bench, ACE uses 262 iterations but trails PEEK by +6.0% solving rate and +9.9% rubric accuracy.Shared Chat reaches up to 748 iterations on OOLONG and 301 on CL-bench for negligible or negative quality gains.
  • Cost–Quality Tradeoff: 1.7–5.8×: ACE costs more than PEEK yet scores lower on all three OOLONG splits; PEEK remains on the cost–quality Pareto frontier across benchmarks.On CL-bench, ACE costs 1.4× more while trailing on both metrics, and Shared Chat costs 3.9× more while hurting quality.
  • Ablations: +10.2% on average: the full PEEK caching policy improves over freezing the map at budget; collapsing Distiller and Cartographer into one call trails the full pipeline by −7.7% on average.The static cache still yields large gains over base RLM, while separating the two modules improves performance.

5 Discussion

PEEK’s usefulness depends on whether agent–context interactions reveal reusable knowledge, and cached content may therefore vary in value across agents.

  • Limitations and Additional Discussion: The context map has limited value when agent interactions reveal little reusable knowledge to cache.Its usefulness depends on how the agent interacts with the external context.
  • Limitations and Additional Discussion: Although the map stores task-independent knowledge, its downstream usefulness may vary because agents interact with context differently.What should be cached may therefore differ across agents.

A Related Work · B Extended Discussion

PEEK differs from KV-cache optimization by operating at the agent level rather than optimizing model-internal key-value states. It maintains a bounded context map of reusable orientation knowledge for recurring external contexts, making the approaches orthogonal.

  • A Related Work: KV-cache optimization improves long-context LM serving efficiency by operating on model-internal key-value states.The passage identifies compression, quantization, token eviction or dropping, reuse across requests or prompts, offloading, and dynamic management as representative techniques.
  • A Related Work: These methods modify hidden token states produced by previously processed tokens through compression, quantization, eviction, reuse, offloading, or dynamic management.
  • A Related Work: PEEK is orthogonal to KV-cache optimization because it maintains an agent-level semantic artifact instead of reusing or compressing hidden token states.
  • A Related Work: PEEK’s bounded context map records reusable orientation knowledge about a recurring external context.The map can include the context’s organization, entities, schemas, constants, and previously derived reusable results.
  • A Related Work: The context map captures how a recurring external context is organized and which entities, schemas, constants, and reusable results matter.
  • A Related Work: KV-cache methods can make the same agent cheaper or faster to serve, whereas PEEK maintains semantic orientation knowledge about the external context.The passage frames this distinction as important for evaluation, while the supplied text does not complete the contrast beyond this point.

B.1 Impact Statement and Future Work.

PEEK presents a first step toward an agent-side cache for language models, framing repeated interaction with persistent long contexts as a broader research agenda. Future directions include adaptive cache sizing, Distiller training, reusable artifacts, and collections of caches accessed programmatically or by parallel agents.

  • Impact Statement: PEEK is a first step toward a genuine agent-side cache for language models.The paper frames this as a new paradigm in language-model systems.
  • Impact Statement: PEEK opens a broader research agenda for agents repeatedly interacting with persistent long contexts across diverse tasks.
  • Future Work: Future work includes adaptively adjusting cache size, training the Distiller, exploring reusable artifacts, and maintaining cache collections for programmatic or parallel-agent access.

B.2 Things We Tried that Did Not Work

Under the OOLONG setup, alternatives that filled PEEK’s constant-sized map with raw context, retrieved playbooks, runtime feedback, or behavioral instructions provided little benefit, hurt performance, or cost too much. The results suggest these approaches fail to preserve compact, structured, persistent context understanding, especially for heterogeneous contexts.

  • Evaluation setup: The studies used the Table 1 setup on an OOLONG subset and measured average improvement over base RLM.This evaluation covered five alternatives for populating the constant-sized map.
  • Raw context: +0.73% on average came from prepending the first 1,024 context tokens, because a long document’s opening rarely summarizes its full structure or content.This raw-prefix approach gives the model only a glimpse of the document’s opening.
  • Processed or meta-level content: +0.73% on average came from retrieving the top-4 256-token chunks of an evolving ACE-style retrieval playbook, which stores task-specific strategies rather than context knowledge.Retrieval also adds indirection and incoherence without improving what is cached.
  • Runtime feedback: −14.86% resulted from runtime feedback that overwrote the map after every RLM root-LM iteration, destroying stable orientation and adding noisy reactive commentary.The active swapping mechanism destabilized planning and polluted the model’s context window or working memory.
  • Prompt engineering: +5.65% came from behavioral instructions, but the modest gain incurred noticeably higher cost and did not justify the improvement.These instructions nudged model behavior rather than building a persistent context map.
  • Limitations: OOLONG’s uniform tabular structure may let weak approaches yield modest gains, whereas raw prefixes, retrieved chunks, and behavioral nudges cannot capture interconnected structure in heterogeneous CL-bench contexts.OOLONG layouts such as Date | User | Instance are apparent from even a small prefix.

C Fine-Grained Cost Analysis

The cost analysis attributes high expense to inefficient iteration use and transcript growth, while PEEK maintains low, predictable overhead and productive iteration counts. ACE incurs substantially higher adaptation and execution costs, whereas RAG and Compaction Agent remain inexpensive but deliver modest gains.

  • Shared Chat: 10.1× input-token inflation on TREC-Q-coarse raises volume from 2.85M to 28.76M for only +1.75% accuracy improvement in Shared Chat.On CL-bench, input tokens inflate 9.0× (2.72M →24.39M) while solve rate drops from 14.0% to 12.0%.
  • RAG and Compaction Agent: $1.43 and $1.38 costs versus $1.57 for base-RLM on CL-bench show that RAG and Compaction Agent can be inexpensive despite modest quality gains.Compaction Agent preprocessing costs only $0.01–$0.71, indicating that productive iteration use—not access cost—is the bottleneck.
  • PEEK overhead: $0.31, $0.22, $0.43, and $0.31 are PEEK’s context-map maintenance costs across the four benchmarks, representing 6.2%–17.9% of its Figure 5 totals.The Distiller contributes roughly two-thirds of maintenance cost because trajectory analysis is more token-intensive than edit planning.
  • PEEK overhead: 378 vs. 394, 398 vs. 496, and 269 vs. 277 show PEEK’s total iteration count stays at or below base-RLM levels on three of four benchmarks.The context map keeps iterations productive while adding small, predictable maintenance overhead.
  • ACE: $29.42 on TREC-Q-coarse makes ACE 5.9× base RLM and 5.8× PEEK, driven primarily by 12.45M output tokens from ACE-augmented execution.On CL-bench, ACE costs $2.63 versus PEEK’s $1.88 while quality trails by 6.0 solve-rate points.

D Additional Benchmarks

The authors evaluated whether common QA benchmarks could simulate their target setting of multiple long contexts shared across tasks. BrowseComp-Plus, FanOutQA, and QuALITY were ultimately deemed inappropriate for this scenario.

  • Benchmark selection: The main benchmarks were natively structured for the target scenario and required no data engineering or self-construction.This criterion guided benchmark selection in §4.1.
  • Additional benchmark trials: Before finalizing four main-paper benchmarks, the authors retrofitted BrowseComp-Plus, FanOutQA, and QuALITY to simulate the target scenario.The simulated setting involved multiple long contexts, each shared by multiple tasks.
  • Benchmark suitability: None of the three retrofitted QA benchmarks turned out to be appropriate for the target scenario.The authors therefore did not report them among the four main benchmarks.

D.1 BrowseComp-Plus

BrowseComp-Plus is a deep-research benchmark of roughly 1,300 hard, multi-hop entity-identification queries over a curated corpus of about 100,000 web documents. Its shared-context setup was limited because concatenated documents lacked shared structure, and each query required only a small fraction of the resulting context.

  • Benchmark: BrowseComp-Plus contains ∼1,300 hard, multi-hop “identify the entity” queries paired with a curated corpus of ∼100K web documents.Queries include questions asking for an entity’s first and last name based on multiple clues.
  • Benchmark: Each query provides gold documents containing the answer and a slightly larger set of evidence documents useful for reaching it.The benchmark distinguishes answer-bearing documents from documents that support the reasoning process.
  • Limitation: The shared-context construction concatenated independent web documents about different entities, producing no shared narrative, structure, or knowledge backbone.Embedding-based clustering did not resolve the independence of queries within a group.
  • Limitation: ∼7.3 required documents out of ∼800 were needed on average for each query, limiting how much relevant knowledge the context map could store.Because each query used only a small portion of the document union, the resulting map could retain only partial knowledge.

D.2 FanOutQA · D.3 QuALITY

FanOutQA and QuALITY were adapted to shared-context evaluation, but each exposed a different mismatch: FanOutQA questions usually require disjoint evidence, while QuALITY contexts are too short and easy to pressure cache reuse. Consequently, both benchmarks did not provide sustained conditions for constructing and reusing a compressed structured cache.

  • D.2 FanOutQA: FanOutQA contains 310 dev-split fan-out questions requiring information aggregation across multiple Wikipedia pages.Each question includes a decomposition tree whose leaves enumerate gold evidence pages.
  • D.2 FanOutQA: FanOutQA answers are often structured, including dictionaries keyed by entity, rather than simple unstructured responses.The dataset’s decomposition trees identify the evidence pages needed for each question.
  • D.2 FanOutQA: Questions were grouped by primary category and greedy evidence-page overlap, with each group receiving the union of its evidence pages as shared context.This preparation attempted to create shared-context workloads from otherwise separate questions.
  • D.2 FanOutQA: 1.1% of ∼48K dev-question pairs share any evidence page, and overlapping pairs share only ∼2.4 pages on average.These statistics indicate that greedy grouping largely combines questions with essentially disjoint evidence sets.
  • D.3 QuALITY: QuALITY pairs short literary articles with about 20 multiple-choice questions per article, grouped directly by article ID.Its one-context, many-question structure superficially matches the target setup.
  • D.3 QuALITY: ∼5.6K mean tokens per QuALITY article fit within one LM call, allowing direct one-pass answers and eliminating sustained cache pressure.The benchmark is also relatively easy, with even small models exceeding 70% accuracy.

D.4 Common Lessons and Implications for Future Benchmarks

Across the three retrofits, two failure modes limited transferable contextual knowledge: contexts were either unions of nearly independent documents or genuinely shared but short and simple. In the latter case, an LM could take in the entire article and task directly.

  • Failure modes: Across all three retrofits, the authors encountered one of two failure modes.The failures concerned whether shared contexts contained transferable knowledge and whether they were sufficiently complex to require contextual support.
  • Failure modes: In BrowseComp-Plus and FanOutQA, manually concatenated independent documents produced nearly disjoint per-task evidence subsets.Because the evidence subsets were nearly disjoint, there was little contextual knowledge to transfer and subsequent tasks could be misled.
  • Failure modes: When contexts were genuinely shared but short and simple, an LM could take in the entire article and task directly.This was the second failure mode identified across the retrofits.

E Prompts … H.5 GPT-5.1

The appendix specifies the exact prompts used in the experiments, including a shared REPL-based system prompt for RLM methods and the unmodified default Codex CLI prompt for Codex methods. These prompts prescribe interactive context analysis, iterative aggregation, final-answer protocols, and coding-agent behavior.

  • E Prompts: The appendix reports the exact experiment prompts, with template variables filled programmatically at runtime.
  • E.1 RLM System Prompt: RLM methods share a system prompt that provides a context variable, recursive llm_query access, batched querying, and iterative answering through a REPL.The prompt strongly encourages extensive use of recursive sub-LLMs and requires inspecting the context before answering.
  • E.1 RLM System Prompt: RLM guidance recommends chunking long contexts, querying sub-LLMs per chunk, storing results in buffers, and aggregating those buffers into the final answer.It also describes iterating through structured sections, such as Markdown headers, while maintaining summarized state.
  • E.1 RLM System Prompt: RLM agents must return completed results inside FINAL or FINAL_VAR, with FINAL_VAR called only after the referenced variable has been created in a separate REPL step.The prompt explicitly identifies calling FINAL_VAR on a nonexistent variable as a common mistake.
  • E.2 Codex System Prompt: Codex-based methods use the unmodified default system prompt shipped with the open-source Codex CLI SDK.The prompt frames Codex as a terminal-based coding assistant expected to be precise, safe, helpful, concise, direct, and friendly.
  • E.2 Codex System Prompt: Codex instructions require respecting applicable AGENTS.md files, prioritizing direct system, developer, and user instructions, and following scoped repository guidance.Nested AGENTS.md instructions take precedence over broader ones, while direct prompt instructions override AGENTS.md instructions.
  • E.2 Codex System Prompt: For complex work, Codex should use meaningful plans, concise progress updates, targeted validation, and brief structured final responses.The prompt recommends specific testing before broader testing, approval-mode-dependent validation, and concise formatting organized by importance.
  • E.2 Codex System Prompt: Codex guidance emphasizes autonomous task completion, root-cause fixes, minimal focused changes, and avoiding unrelated bug fixes or unrequested commits.It also instructs the agent not to guess or fabricate answers and to use available tools before yielding.

I CL-bench Leaderboard Snapshot (05/2026)

Figure 6 presents a snapshot of the CL-bench leaderboard from May 2026.

  • Figure 6 shows the CL-bench leaderboard snapshot for May 2026.
Loading 2605.19932v1…