PaperlayerView exact revision on arXiv

Source-linked AI summary

Memory as Middleware for Self-Improving AI Agents

K. R. Jayaram, Vatche Isahagian, Vinod Muthusamy, Gegi Thomas, Punleuk Oum, Gaodan Fang, Ashwath Vaithinathan Aravindan

arXiv:2609.32091v1cs.AIcs.DC

TL;DR

Agents often lack durable memory of prior operational experience, while bespoke solutions couple memory logic to particular agents and stores. The paper proposes memory middleware and presents ALTK-Evolve as a reference implementation; its preliminary evaluation reports improved agent reliability on AppWorld.

  • Problem

    Agent memory remains fragmented application logic, coupled to individual agents and stores rather than independently reusable infrastructure.

    Source: §1 · p. 1, §3.3 · p. 4

  • Method

    The paper proposes pluggable memory middleware and presents ALTK-Evolve, which captures agent trajectories, distills reusable guidelines, and serves them through pluggable storage and a standard tool protocol.

    Source: §1 · pp. 1–2

  • Results

    A preliminary evaluation reports that factoring memory into middleware improves agent reliability on AppWorld, a public multi-step benchmark.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ALTK-Evolve demonstrates an initial practical subset of memory middleware, while abstractions, consistency models, policies, and benchmarks remain open challenges.

    Source: §8 · p. 8

Abstract

from arXiv · show

AI agents are stateless across sessions by default and therefore operationally amnesic: each session begins with little durable knowledge of prior failures, repairs, preferences, or successful strategies. As a result, agents repeat the same mistakes and discard hard-won experience. The dominant fix is \emph{bespoke memory}---retrieval, persistence, and learning logic hand-wired into one agent and bound to one storage engine. This creates a fragmented landscape where memory cannot be swapped, shared, isolated, or reasoned about independently of the agent that owns it. We argue that this is a middleware problem: agent memory deserves a first-class, pluggable layer, just as data access, messaging, and persistence each became middleware concerns. We develop this vision through six systems challenges: two-sided pluggability, host-native interposition, multi-tenant isolation, write-path consistency, federated sharing with provenance, and lifecycle governance. We present ALTK-Evolve, a reference implementation of memory middleware for self-improving agents, and use it to motivate a broader research agenda for future memory middleware.

1 Introduction

Agents accumulate useful operational experience, but bespoke memory remains coupled to individual agents and stores. The paper argues for a pluggable middleware layer and presents ALTK-Evolve as an initial implementation that improves reliability.

1 Introduction

  • Agent amnesia makes long-running, tool-using agents repeatedly rediscover operational knowledge, creating a reliability ceiling.Execution traces are increasingly available, but reusable infrastructure for turning them into durable behavioral memory is rare.Source: §1 · p. 1
  • Bespoke memory couples retrieval, persistence, and learning to one agent and store, leaving memories difficult to share or compose.The paper frames middleware as the reusable layer that can capture, consolidate, isolate, retrieve, share, edit, forget, and govern memory across agents.Source: §1 · p. 1
  • ALTK-Evolve captures agent trajectories, distills reusable guidelines, and serves them through pluggable storage and a standard tool protocol.The proposed middleware owns cross-cutting concerns including isolation, consistency, sharing, provenance, user control, and lifecycle governance.Source: §1 · p. 1
  • The design space covers two-sided pluggability, host-native interposition, multi-tenant isolation, write-path consistency, federated sharing with provenance, and lifecycle governance.The implementation uses trajectory capture, guideline extraction, standard tool interfaces, namespaces, visibility controls, provenance metadata, and conflict-aware updates.Source: §1 · pp. 1–2
  • A preliminary evaluation reports that factoring memory into middleware improves agent reliability on AppWorld, a public multi-step benchmark.The implementation’s current limits motivate a broader research agenda for memory middleware.Source: §1 · p. 2

2 Why Agent Memory is a Middleware Problem

The paper distinguishes memory as stored data from memory as infrastructure: reusable middleware must supply semantics and governance that individual stores and agent-local mechanisms do not provide.

2.1 Bespoke Memory ≠Scalable Infrastructure

  • Bespoke agent memory is coupled to a particular agent, harness, or store, making it difficult to move, compose, interpret, and govern.Users and organizations need controls to inspect, edit, delete, approve, and scope memories.Source: §2.1 · p. 2

2.2 Stores Are Not Middleware

  • Stores persist memory, but middleware must define how trajectories become memories, how memories become context, and how context changes future behavior.It must also provide controls for isolation, sharing, editing, expiry, auditing, and overrides.Source: §2.2 · p. 2

2.3 The Learning–Governance Gap

  • Agent-local memory can learn operational lessons from execution but often lacks governance, while platform memory offers controls without necessarily extracting reusable guidance from trajectories.The paper’s example contrasts a harness-local coding lesson with a platform personalization store that is administrable but did not extract the lesson.Source: §2.3 · p. 2
  • Memory middleware should combine agent-local learning with user and organizational governance, enabling shared guidance while preserving provenance and access control.Users can control what is remembered, forgotten, shared, or project-scoped; organizations can publish governed policies and guardrails.Source: §2.3 · p. 2

3 A Design Vision for Memory Middleware

The design vision evaluates memory middleware against accuracy, cost, and controls. Accuracy concerns useful behavior; cost and controls make memory a systems problem that belongs in middleware.

3.1 What Should the Goals Be?

  • Accuracy means retrieving and representing task-relevant memory without degrading behavior through stale, duplicated, or contradictory items.It concerns what the layer withholds as well as what it surfaces.Source: §3.1 · p. 3
  • Cost must be managed through compression, lazy recall, selective context placement, and deployment packaging that fits available resources.Unmanaged memory can bloat the context window, increasing expense and harming accuracy.Source: §3.1 · p. 3
  • Controls require tenant and scope isolation, attribution, deliberate publication, and retention, review, and deletion.These controls support organizational trust and user choice over what is remembered.Source: §3.1 · p. 3
  • Cost and controls distinguish memory middleware as a systems problem, while the six middleware concerns are mechanisms for advancing the design goals.Many of those mechanisms serve more than one goal.Source: §3.1 · p. 3

3.2 Support for All Types of Agent Memory

Agent memory must represent distinct kinds of experience on their own terms and provide a common interface across hosts and storage backends. The proposed middleware layer separates these concerns while keeping the memory types linked.

3.2 Support for All Types of Agent Memory

  • Episodic, semantic, and procedural memories differ in representation, retrieval, updates, compression, and governance, so one undifferentiated store risks retrieval interference.The paper gives the example of verbose tool-call episodes crowding out a stored preference.Source: §3.2 · p. 3
  • Memory types should occupy separate but linked namespaces so each can be represented, retrieved, reconciled, compressed, and governed on its own terms.Source: §3.2 · p. 3

3.3 Two-Sided Pluggability

  • Two-sided pluggability requires a stable contract between heterogeneous agent hosts and storage backends, with a common interface that providers implement and agents target.This allows a store to change without touching the agent, and a host to be supported without touching the store.Source: §3.3 · pp. 3–4

3.4 Host-Native Interposition

Memory middleware should adapt to each host’s interposition mechanisms while preserving the same logical operations. It should also enforce namespace-based isolation across memory operations, not merely storage.

3.4 Host-Native Interposition

  • Host-native interposition maps each host’s available hooks, commands, tools, or instruction files to consistent recall-before-action and capture-after-turn behavior.The middleware can be deployed as a sidecar, library, server, tool provider, or plugin.Source: §3.4 · p. 4

3.5 Multi-Tenant Isolation

  • Namespace-based isolation scopes retrieval by default and must also govern consolidation, publication, deletion, and audit across users, teams, projects, and other scopes.Private preferences and organization-published policies require different visibility boundaries.Source: §3.5 · p. 4

3.6 Write-Path Consistency

Memory middleware must reconcile new memories rather than append blindly, support deliberate sharing with provenance, and govern memories throughout their lifecycle.

3.6 Write-Path Consistency

  • A memory write path should reconcile each candidate by adding, updating, merging, superseding, quarantining, or rejecting it instead of appending blindly.Append-only memory accumulates duplicates, stale advice, overgeneralized lessons, and contradictions that can poison retrieval.Source: §3.6 · p. 4

3.7 Federated Sharing with Provenance

  • Federated sharing should be private by default, deliberate, and traceable to each memory’s origin, while accounting for poisoning and privacy risks.Namespaces can subscribe to or synchronize with other sources without copying entire stores or erasing ownership.Source: §3.7 · p. 4

3.8 Lifecycle Governance

  • Lifecycle governance must support retention, expiration, compaction, audit, approval, and deletion, with policy hooks reflecting who may create, edit, or share memories.Memory authors may include agents, users, other agents, or organizations.Source: §3.8 · p. 4

4 A Reference Architecture and Implementation

ALTK-Evolve specifies a replaceable memory middleware loop, organized around six systems concerns, with interfaces that connect agent hosts to storage. Its implementation uses typed memory, governed agent-facing operations, reusable trace extraction, and purpose-directed gists, while leaving some design-space concerns open.

4 A Reference Architecture and Implementation

  • The architecture defines a replaceable experience loop: capture trajectories, extract and consolidate memories, store and retrieve them, then inject them into the agent.It prescribes interface semantics rather than a particular agent, model, or store.Source: §4 · p. 4, Figure 2 · p. 5
  • The architecture covers the six concerns unevenly, distinguishing what it realizes today from what remains open.The open directions feed into the paper’s research agenda.Source: §4 · p. 5, Table 2 · p. 6

4.1 A Typed Memory Model

  • Episodic, semantic, and procedural memories occupy separate namespaces, with captured conversational turns routed to semantic memory and task traces to episodic and procedural memory.Memory entities include content, applicability triggers, and provenance; ownership and visibility default to private unless published.Source: §4.1 · p. 5

4.2 The Agent-Facing Surface and Host Bindings

  • Agents use six logical operations—retrieve, write, consolidate, publish, delete, and inspect—while consolidation and inspection remain governed control-plane operations.The same logical behavior binds to different hosts through their native interposition primitives, without changing host agent code.Source: §4.2 · p. 5

4.3 From Traces to Reusable Memory

  • Trace extraction segments tasks, identifies subtasks, generalizes task-specific details, then clusters and consolidates similar guidance into more comprehensive memory.Reconciliation compares generalized candidates with clustered prior memory to detect duplicates and contradictions at the level of reusable operations.Source: §4.3 · pp. 5–6

4.4 Purpose-Directed Gisting and Tiered Storage

  • Purpose-directed gists foreground information useful to downstream retrieval, while the full original remains recoverable from durable storage when a gist is insufficient.Gists are embedded in the vector index; the model decides at read time whether to fetch the original.Source: §4 · p. 5, §4.4 · p. 6

5 Evaluation

On AppWorld’s held-out Test-Normal split, ALTK-Evolve improves both completion metrics over a no-memory baseline, with the larger gains on scenario completion and hard-tier tasks. The same memory layer also runs unmodified across four coding-agent hosts; the authors characterize the evaluation as preliminary.

5 Evaluation

  • Both AppWorld conditions used the same GPT-4.1 agent and reasoning loop, differing only in whether the memory layer supplied learned guidelines.Guidelines came from train and dev trajectories; evaluation used the held-out test-normal partition.Source: §5 · p. 6
  • Scenario-goal completion rose from 50.0% to 64.3%, while task-goal completion rose from 69.6% to 73.2% on AppWorld Test-Normal.Scenario-goal completion on the Hard tier increased from 19.1% to 47.6%.Source: §5 · p. 6, Table 3 · p. 6
  • The same memory layer ran unmodified across Claude Code, Codex, ClawCode, and IBM Bob using each host’s native interposition primitive.The authors describe these results as feasibility evidence and signal, not a complete evaluation.Source: §5 · p. 6

6 Research Agenda

The research agenda treats a common programming model and consistency semantics as foundations for other memory-middleware mechanisms. It also identifies cross-cutting governance and portability questions and calls for work on memory quality, lifecycle, deployment, and compression.

6 Research Agenda

  • A common programming model and consistency semantics are foundational because other mechanisms depend on them.For learned experience, open consistency questions include when memories supersede or coexist, when new memories become visible, and when retrieval should expose uncertainty.Source: §6 · pp. 6–7
  • Multi-step memory writes need transactional semantics, rollback points, failure handling, and approval policies so incomplete or unapproved guidance does not become retrievable.The proposed natural transaction unit is the staged capture, extraction, consolidation, and publish pipeline.Source: §6 · p. 7
  • Memory admission should score candidate experiences and weight learning by source quality, including outcomes, reflection, failures, and successful recovery from failure.Open problems include calibrating these signals and rescoring memories as downstream reuse is observed.Source: §6 · p. 7
  • Long-running agents need policies for retention, compaction, and forgetting that account for recency, utility, provenance, confidence, sensitivity, and staleness.Unlike cache eviction, memory eviction changes agent behavior.Source: §6 · p. 7
  • Deployment-aware packaging should adapt memory formats and retrieval to resource limits across VMs, containers, and serverless functions; purpose-directed compression also raises questions about information loss and adaptive fidelity.The agenda includes compression-aware indexing, automatic format selection, and deciding when to fetch full originals.Source: §6 · p. 7
  • Cross-cutting work includes richer isolation and sharing policies, provenance that preserves memory lineage, and portability benchmarks across agents, stores, protocols, and deployments.Enterprise policies may include hierarchical scopes, role-based access, and promotion of private memories to shared knowledge.Source: §6 · p. 7

7 Related Work

Related work contributes memory algorithms, representations, write and retrieval policies, governance mechanisms, and memory products. The paper positions its contribution as a middleware boundary intended to let such approaches interoperate across hosts and stores.

7 Related Work

  • Experiential and procedural memory work shows ways agents learn from execution, including reflection, episodic recall, skill induction, and trajectory-derived guidance.The paper distinguishes its contribution as a substrate for plugging such methods into many hosts and stores.Source: §7 · p. 7
  • Prior memory architectures propose typed, hierarchical, episodic, neural, and compressed representations; this paper adopts typed, separated memory and purpose-directed gisting as middleware concerns.It treats these as contracts and namespaces rather than a single model.Source: §7 · p. 7
  • Work on admission, consolidation, retrieval, and memory management motivates the paper’s agenda to make write-path and admission policies responsibilities of the memory layer.Source: §7 · p. 7
  • Research on poisoning, lineage, governed memory, provenance, and verified transfer aligns with the paper’s controls, isolation, provenance, and portability agenda.Source: §7 · p. 7
  • Reusable skills can be discovered and refined from experience, but they may also add overhead or harm; the paper treats governed skill promotion as part of its agenda.Source: §7 · pp. 7–8
  • Existing memory services and platforms often provide storage and governance but are typically bound to one host or cloud; the paper proposes a host- and store-portable substrate with cross-cutting guarantees.Source: §7 · p. 8

8 Conclusion

Agents are becoming persistent software actors, yet memory remains fragmented application logic; ALTK-Evolve shows that an initial middleware subset is practical and improves reliability, while broader reusable foundations remain an open challenge.

8 Conclusion

  • ALTK-Evolve shows that an initial subset of memory middleware is practical and improves agent reliability.The broader challenge is defining abstractions, consistency models, policies, and benchmarks for a reusable foundation.Source: §8 · p. 8
  • Memory should become a first-class middleware substrate that interposes between agents and stores, learns from experience, and governs the result.Source: §8 · p. 8
  • Defining reusable abstractions, consistency models, policies, and benchmarks for self-improving agents remains an open challenge for the systems community.Source: §8 · p. 8

Related papers

Loading 2609.32091v1…