PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic Context Management: Solving Agent Memory and Cost by Treating Them as Lifecycle and Architecture Problems

Gaurav Dadhich

arXiv:2607.21503v1cs.AIcs.IR

TL;DR

Production agents struggle to manage the context accumulated across conversations, tools, and organizational scopes, making context management a lifecycle problem rather than merely storage and retrieval. The paper defines Agentic Context Management through five primitives and reports 92.0% on LongMemEval and 93.2% on LoCoMo under its stated configuration.

  • Problem

    Production agents lack a disciplined account of what should remain in context at each step, limiting reliable scaling across long conversations, tools, sessions, and handoffs.

  • Method

    The paper defines Agentic Context Management as five primitives—architecting, ingesting, scoping, anticipating, and compacting & consolidation—across an organizational scope hierarchy.

  • Results

    92.0% overall on LongMemEval and 93.2% on LoCoMo categories 1–4 are reported for Maximem Synap under the configuration in Table 2.

  • Takeaways & Limitations

    Agent context infrastructure should manage a full lifecycle across organizational scopes rather than provide only a location to store history.

  • Takeaways & Limitations

    Existing benchmarks only partially capture production context management because they do not jointly measure accuracy, latency, token efficiency, and context-rot resistance.

Abstract

from arXiv · show

Production AI agents' failures are less often due to an inability to reason well and more often because they cannot manage what is in their reasoning context: conversation histories, large prompts, large tool definitions, and ballooning tool outputs. Agents drown in their own accumulating history while paying a token cost that grows every turn, producing missing recalls within and across conversations. The incumbent response treats this as a storage-and-retrieval problem. We argue that framing is too narrow. Actively managing what an agent holds in mind is a lifecycle, not merely a store: it spans deciding what to remember, extracting and structuring it, choosing the right store per data type, consolidating and forgetting while preserving provenance, deciding what is relevant now, anticipating what is needed next, and compacting context to a budget without losing what matters. In serious production this operates not over a single user but across an organizational scope hierarchy. We name this discipline Agentic Context Management (ACM) and decompose it into five primitives: architecting, ingesting, scoping, anticipating, and compacting & consolidation. We then make the economic case: naive context accumulation grows token cost quadratically in conversation length, crude summarization buys linear cost at the price of an accuracy cliff, and only validated compaction achieves linear cost with preserved fidelity. We describe a reference implementation, Maximem Synap, that realizes the five primitives as a multi-tenant service and reports 92% on LongMemEval and 93.2% on LoCoMo under the configuration detailed in Section 6. We close with dimensions existing benchmarks do not yet capture, latency, token efficiency, and context-rot resistance, and the frontier of decision-level and organization-level context the category points toward.

1 Introduction

The paper argues that production agents fail primarily because they lack disciplined context management, not because they cannot reason. It reframes memory as a lifecycle spanning five primitives, organizational scopes, economic validation, and a reference implementation.

  • Motivation: As of 2025, a majority of enterprises were experimenting with AI agents, but only about a quarter reported scaling and fewer than 10% had scaled within any business function.The large majority of agent pilots never graduate to production.
  • Problem: Production agents most visibly lack discipline over what belongs in the context window at each step, causing them to forget earlier information and contradict themselves across interactions.The problem includes user statements, tool-fetched information, and information mentioned in prior weeks.
  • Reframing and Taxonomy: Agentic Context Management reframes memory as a lifecycle and defines five primitives: architecting, ingesting, scoping, anticipating, and compacting & consolidation.These primitives operate across a scope hierarchy from the individual user to the organization.
  • Economic Argument: Full-append context costs O(n^2) tokens over a conversation, crude summarization reduces cost at an accuracy cost, and validated intelligent compaction achieves linear cost with preserved fidelity.The paper also supports the retrieval argument with an original study across five data domains.
  • Reference Implementation: Maximem Synap is presented as a multi-tenant reference implementation incorporating the five primitives at both architectural and observable-behavior levels.The paper emphasizes that Synap is one way to build the lifecycle system, while the argument concerns the broader category.
  • Evidence and Agenda: The paper reports results on two public memory benchmarks, discusses their limitations, and identifies decision-level and organization-level context management as future frontiers.These contributions are presented as evidence and an agenda for the category.

2 From Memory to Context Management

Agentic Context Management treats deciding what an agent holds, when, for how long, and at what cost as a full lifecycle spanning five coupled primitives. Unlike a single memory tool, a context-management platform coordinates these primitives across organizational scopes.

  • Definition: Agentic Context Management governs context from acquisition through retirement by deciding what an agent should hold, when, for how long, and at what cost.It comprises five primitives.
  • Five primitives: The lifecycle’s five coupled primitives are architecting, ingesting, scoping, anticipating, and compacting & consolidation.Architecture changes what the other primitives should do, making context management a system rather than five independent tools.
  • Five primitives: Architecting defines each agent’s information categories, extraction, storage, retention, retrieval, and compaction rather than relying on a universal schema.The paper treats architecture as a first-class primitive because different agents require different context designs.
  • Scope hierarchy: Scoping resolves relevant context across user, customer, and client levels with strict isolation, while retaining a separate global knowledge layer.Ingestion and retrieval proceed narrowest-first: user, then customer, then client.
  • Five primitives: Anticipating uses observed agent behavior to prepare likely-needed context before explicit requests, moving retrieval off the critical path and supporting unknown-needed answers.This applies speculative prefetching from computer systems to agent context.
  • Platform distinction: A memory tool addresses one primitive, whereas a context-management platform addresses all five coherently across scopes.The taxonomy is grounded in named, observed production failure modes caused by missing primitives.

3 Why Store-and-Retrieve Is Not Enough

Store-and-retrieve is insufficient because naïve context accumulation makes cost grow quadratically, while crude summarization can destroy accuracy. Preserving reasoning quality requires validated compaction plus ingestion, scoping, and retrieval that maintain sufficient structured context.

  • Economic case: O(n^2) cumulative input tokens make full-append conversations increasingly expensive, whereas a fixed context budget yields O(n) growth.If each turn adds about t tokens, full-append cost scales quadratically with conversation length; a fixed budget W produces n·W = O(n).
  • Economic case: 6× at 100 turns and 13× at 200 turns illustrate how full-append costs outpace a 4,000-token budget when each turn adds 500 tokens.These are illustrative multiples comparing full-append with bounded context.
  • Economic case: 66.7% to 57.1% task accuracy shows that compressing 18,282 tokens to 122 tokens can perform worse than providing no context.The crude summarization was lossy and unvalidated, discarding information needed downstream.
  • Managed lifecycle: Validated compaction, structure-preserving ingestion, context-assembling scoping, and combined semantic-relational retrieval are required to close cost and sufficiency gaps.No single retrieval method delivers the needed coverage; the paper therefore advocates a managed lifecycle.
  • Retrieval sufficiency: Answer quality is bounded by the weakest link among extraction quality, retrieval quality, and reasoning sufficiency.Unextracted signals can make storage unusable, while incomplete retrieval can omit context needed for reasoning.
  • Retrieval sufficiency: 0.91 vs. 0.29 MRR for natural-language-to-code and 0.81 vs. 0.61 for science QA show that vector and keyword retrieval win in different regimes.Vector search dominates when the semantic gap is wide, while keyword search wins for specific entities; single-document evaluations can miss bridge documents required for multi-hop reasoning.

4 The Maximem Synap System

Maximem Synap is a hosted, multi-tenant context-management service organized around five capability contracts rather than fixed runtime components. It asynchronously architects and ingests agent memory, scopes and anticipates retrieval, and validates conversation compaction across a polyglot storage stack.

  • System design: Maximem Synap is a hosted, multi-tenant context-management service whose components are described through interfaces, observable behavior, and guarantees rather than proprietary internals.The five primitives are capabilities with contracts, not five independent runtime processes; their mapping to processes is an implementation choice.
  • Client surface: The async-first SDK returns ingestion identifiers immediately, supports unified scope-aware retrieval, conversation compaction, and streaming, and enforces tenant isolation at storage and query layers.Python is canonical, with a JavaScript bridge; memory processing occurs asynchronously without blocking the calling application.
  • Architecting: Architecting generates and activates a bespoke per-agent memory architecture that selects categories and extraction, storage, retrieval, and compaction behavior using LLM reasoning, multi-agent checks, and validations.This is an autonomous design step rather than selection from a fixed menu.
  • Ingesting: Ingesting is an asynchronous, queue-backed pipeline that qualifies documents, extracts categorized memories with entities, relationships, and temporal validity, resolves entities, and persists results across relational, graph, and vector stores.Entity resolution uses a confidence-ordered cascade, is best-effort, never blocks ingestion, and can queue ambiguous matches for review.
  • Scoping and anticipating: Scoping resolves user, customer, and client hierarchy narrowest-first, returning provenance-tagged, ranked, token-budgeted items, while anticipation prepares likely-needed context ahead of explicit requests.Anticipatory retrieval currently achieves a 60%+ hit-rate consistently across clients, while speculative misses incur discarded work.
  • Compacting & Consolidation: Compaction is verified by testing recoverability, emitting validation scores and compression ratios, and retrying with less aggressive compression when validation falls below threshold.The system uses category-aware preservation for long conversations.

5 Design Choices

The design choices distinguish context management from a generic memory tool by tailoring architecture to each agent and enforcing validated, lifecycle-aware operations. They combine asynchronous ingestion, first-class scope isolation, hybrid retrieval, and latency-aware context preparation.

  • Architecture: Architecture is synthesized per agent rather than fixed and generic, because categories, retention, and compaction needs differ across use cases.The paper identifies generated architecture as especially responsible for extraction quality and as making lifecycle coupling concrete.
  • Compaction: Compaction carries a quality contract: Maximem Synap returns a validation score and compression ratio, then retries automatically when validation fails.This replaces unvalidated compression with explicit confirmation that the compression worked.
  • Compaction: Periodic compaction keeps validation overhead linear rather than quadratic by operating on already-compacted context plus recent turns instead of the full transcript.When context is held near a budget W and compaction fires every p turns, each pass handles a bounded context while compaction count grows linearly.
  • Ingestion: Ingestion is asynchronous and non-blocking: an ingestion ID returns immediately while extraction, resolution, and multi-store persistence run off the critical path.Recent turns remain verbatim in working context, preserving read-your-writes within a session despite trading immediate consistency for latency.
  • Scope: Scope is first-class and isolation is enforced at storage and query layers, with identity derived from credentials so organizational context remains available without cross-user data exposure.Maximem advises scope policy, but the client approves and gates it, retaining customer control.
  • Retrieval and latency: Retrieval combines semantic and relational signals to pursue reasoning sufficiency, while low-latency and anticipatory modes reduce retrieval’s critical-path burden.The hybrid approach addresses vector search’s missed bridge context and graph traversal’s expense and brittleness; anticipation can prepare context in advance.

6 Evaluation

Maximem Synap is evaluated on official LongMemEval and LoCoMo distributions, reaching 92.0% overall on LongMemEval and 93.2% on LoCoMo categories 1–4 under the stated configuration. The benchmarks assess conversational recall but omit production latency, token efficiency, and context-rot resistance, motivating a more comprehensive benchmark.

  • Benchmarks: The evaluation uses official public distributions of LongMemEval and LoCoMo, without custom subsets or relabeling.LongMemEval contains 500 human-curated questions across six ability categories; LoCoMo covers very long conversations averaging approximately 300 turns.
  • Methodology: LoCoMo category 5 is adversarial and measures abstention rather than memory, while including or excluding it can shift the headline score by ten points or more.The paper identifies adversarial-category handling as a common source of incomparable LoCoMo numbers and emphasizes that full configuration is necessary for interpretation.
  • Results: 92.0% overall on LongMemEval (460/500) and 93.2% on LoCoMo categories 1–4 are reported under the Table 2 configuration.The paper states that per-category results are reported alongside the overall scores, including weak categories.
  • Limitations: Both benchmarks measure conversational recall and reasoning over recalled content but do not measure latency under production load, token cost per task, or context-rot robustness.The paper deliberately reports no latency numbers and reserves these gaps for follow-up benchmark work.
  • Future evaluation: A forthcoming benchmark is intended to measure accuracy, latency, token efficiency, and context-rot resistance together for production context management.Existing benchmarks capture only part of the dimensions production teams weigh heavily.

7 Related Work

The paper maps related work onto ACM’s five primitives, spanning neural-memory foundations, research and commercial memory systems, retrieval, and long-context efficiency. It distinguishes ACM from attention-level methods by focusing on what enters the context window, regardless of its size.

  • Foundations: Neural-memory foundations separate memory contents from their use and emphasize that storage, retrieval, forgetting, and context shape all influence agent behavior.Differentiable neural computers couple a controller to external addressable memory, while cognitive-science-inspired work maps human memory systems onto AI analogues.
  • Memory systems (research): Research memory systems distribute ACM capabilities across lifecycle primitives, including MemGPT’s virtual-memory hierarchy, MIRIX’s typed memory, and ACE’s structured incremental updates.Dynamic Cheatsheet and ACE address adaptive external playbooks and context-collapse risks, while CAMELoT offers associative-memory-based compaction.
  • Memory systems (commercial): Commercial memory layers likewise emphasize selected primitives, with Mem0 dynamically extracting and consolidating salient conversational information.The paper characterizes commercial systems using their own current documentation and publications rather than presenting a critique.
  • Retrieval: Retrieval research extends retrieval-augmented generation through entity graphs, hippocampus-inspired knowledge-graph indexing, recursively summarized trees, and memorized traversal decisions.These approaches support relational, multi-document, multi-hop, and recursively summarized retrieval settings.
  • Long context: Long-context methods reduce serving costs or mitigate attention limitations, whereas ACM decides what should enter the context window regardless of its size.The cited long-context literature reports degradation for information in the middle of long contexts and proposes streaming attention and KV-cache compression.

8 Future Directions: Decision-Level and Organization-Scale Context

The paper identifies decision-level context as the frontier beyond managing conversational and organizational facts, aiming to preserve why organizations made decisions. It argues that today’s context-management lifecycle is the necessary substrate for eventually building this capability despite major unresolved challenges.

  • Decision-Level Context: Decision-level context captures why an organization acted, enabling agents to reason with institutional judgment rather than only institutional facts.The paper frames this as a “context graph” opportunity attracting investment attention.
  • Open Challenges: The paper remains measured because decision-level context faces hard, largely unsolved problems.The challenges include incomplete decision records and difficulties connecting decisions to outcomes or determining when decisions have been superseded.
  • Open Challenges: Most decisions are implicit or undocumented, while recorded rationales may be post-hoc justifications rather than true traces.These limitations make it difficult to reconstruct authentic organizational decision processes.
  • Foundation: A lifecycle that manages facts across users and organizations provides the necessary substrate for eventually building decision-level context.The paper presents surfacing these challenges as a way for the category to mature collectively.

9 Conclusion

Agents primarily fail because they lack managed context, not intelligence. Agentic Context Management treats context as a lifecycle across organizational scopes and as an economic necessity distinguishing quadratic from linear cost.

  • Agent failures primarily reflect unmanaged context rather than insufficient intelligence.
  • Agentic Context Management is a lifecycle comprising architecting, ingesting, scoping, anticipating, and compacting & consolidation.
  • The lifecycle operates across an organizational scope hierarchy and makes context management an economic necessity, not merely a convenience.
  • The economic distinction is between quadratic and linear cost.

A Cost-model derivation

The cost model contrasts full-history appending, whose cumulative billing grows quadratically with conversation length, with bounded-budget context, whose cost is linear. Validated compaction is presented as the target frontier: linear cost with checked fidelity, unlike crude summarization’s accuracy loss.

  • Cost scaling: Full-append re-sends the entire conversation history each turn, producing quadratic cumulative input-token cost as the number of turns grows.With t tokens added per turn, turn k receives k·t input tokens.
  • Cost scaling: Bounded-budget context caps each turn at W tokens, yielding cumulative cost C_bounded = nW, linear in the number of turns.The model assumes a fixed input budget per turn.
  • Compute implications: Full-append’s per-call attention compute is quadratic in sequence length, making cumulative compute roughly cubic in conversation length.The passage distinguishes this compute result from the cleaner billing claim and notes that caching changes constants, not asymptotics.
  • Compaction frontier: 18,282 →122 tokens of single-step compression reduced accuracy from 66.7% to 57.1%, below the no-context baseline.This illustrates crude summarization’s linear-cost tradeoff: lower cost achieved by surrendering fidelity.
  • Compaction frontier: Validated compaction targets linear cost while preserving fidelity through checked context reduction.The frontier contrasts full-append’s fidelity at quadratic cost with crude summarization’s linear cost and accuracy loss.

B Retrieval-study methodology (motivating study, Section 3.3)

Section 3.3 motivates its retrieval conclusions with a five-dataset comparison of keyword and vector search under a fixed, small-scale configuration. It presents regime dependence, indexing-time asymmetry, and hit-based scoring’s blindness to sufficiency while explicitly bounding the study’s claims through stated limitations.

  • Setup: The study indexed five separate 10,000-document corpora and evaluated 1,000 queries per dataset using MRR@10 against gold annotations.Datasets covered code, web passages, factoid QA, multi-hop QA, and science retrieval.
  • Limitations: The study treats its findings as motivation rather than a benchmark because they characterize one keyword engine and one vector store, at only 10,000 documents per corpus.The evaluation also lacked retained per-query traces, and MS MARCO passages could not be redistributed.
  • Limitations: No chunking biases long-document comparisons against vectors because the embedding model truncates inputs, although CodeXGLUE’s short snippets avoid this effect and its 0.914 result stands.The passage states that chunking would likely improve vector results on long-text corpora.
  • Limitations: The isolated comparison omitted fused hybrid retrieval and second-stage cross-encoder reranking, so the claim concerns the necessity of combining signals rather than the best achievable combined system.Both omitted techniques were expected to raise vector and hybrid numbers.
  • Conclusions: The section’s conclusions—retrieval-method regime dependence, indexing-time asymmetry, and hit-based scoring’s structural blindness to sufficiency—remain robust to the first three caveats, while the fourth motivates the sufficiency argument.HotpotQA’s single-supporting-document scoring cannot measure multi-document sufficiency.

C Reproducibility statement · D Data use and privacy

The paper specifies reproducibility requirements for every Section 6 result and states competitor-comparison rules. It also describes tenant isolation, customer-controlled retention, configured handling of personally identifying information, and exclusive use of public benchmark data.

  • C Reproducibility statement: Section 6 results report datasets, splits, question counts, answer and judge models, retrieval configurations, harness commits, run dates, and public releases.Released materials include harness code, configurations, and raw per-question outputs.
  • C Reproducibility statement: The paper directly discloses when a result cannot be re-run by a third party at publication time.It avoids implying reproducibility when re-execution is unavailable.
  • C Reproducibility statement: Competitor numbers follow identical-protocol or clearly labeled self-reported reporting rules.These rules are attributed to Section 6.2.
  • D Data use and privacy: Memory is tenant-isolated by construction through storage- and query-layer enforcement.The policy references the architecture described in Section 4.
  • D Data use and privacy: Customers control retention through guardrails generated by the architecture.Retention is presented as a customer-controlled policy-level mechanism.
  • D Data use and privacy: Personally identifying information is governed by extraction-time handling policies configured per agent.The policy applies to conversational data that can include personal information.
  • D Data use and privacy: The paper reports no customer data, and its benchmark results use public datasets.This separates reported evaluation data from customer conversational data.
Loading 2607.21503v1…