PaperlayerView exact revision on arXiv

Source-linked AI summary

Jev-Mem: System-One-Controlled Agentic Memory for Efficient AI Agents

Dongming Jiang, Yi Li, Bingzhe Li

arXiv:2609.23986v1cs.AIcs.LG

TL;DR

Long-horizon agents need to preserve and retrieve accumulated experience, but repeated autoregressive memory-control decisions add inference overhead. Jev-Mem assigns structured memory control to a System-One controller and reserves System Two for complex synthesis; on LoCoMo, it reports stronger accuracy alongside faster construction and query access.

  • Problem

    Long-horizon agents accumulate experience beyond fixed context windows, while memory systems often use heuristics or repeated autoregressive generation for organization and retrieval decisions.

    Source: §1 · p. 1

  • Method

    Jev-Mem separates lightweight System-One control of memory construction and retrieval from System-Two complex reasoning and answer synthesis.

    Source: §1 · p. 2

  • Results

    0.777 LLM-as-a-Judge score on LoCoMo versus 0.700 for the strongest baseline, an 11.0% relative improvement; construction took 158 s and average query latency was 0.93 s.

    Source: §4.2 · p. 7, §5 · p. 9

  • Takeaways & Limitations

    The results support separating memory control from generative reasoning as a promising direction for more effective and efficient long-horizon agents.

    Source: §5 · p. 9

Abstract

from arXiv · show

Agentic memory is becoming essential for long-horizon AI agents, yet many existing systems rely on autoregressive LLMs to control how memories are organized, retrieved, and used, placing expensive generation on the critical path of memory operations. We introduce \textbf{\method}, a new agentic memory architecture inspired by System-One/System-Two cognition. System One captures fast, lightweight decision-making, whereas System Two performs slower, deliberative reasoning. Jev-Mem brings this division of labor to agentic memory through a dedicated System-One control plane, a structured multi-relational memory plane, and a System-Two reasoning plane. The System-One controller governs memory typing and relational organization during construction, and dynamically performs query routing, retrieval-budget allocation, graph traversal, candidate scoring, and adaptive stopping during retrieval. System Two is invoked only for complex reasoning and answer synthesis. This design improves both memory effectiveness and system efficiency: on LoCoMo Jev-Mem achieves an overall LLM-as-a-Judge score of 0.777, an 11.0\% relative improvement over the strongest baseline, while reducing memory construction time to 158\,s, a 6.6$\times$ speedup over the fastest competing memory system, and lowering average query latency to 0.93\,s, a 36.7\% reduction.

1 Introduction

Long-horizon agents accumulate experience beyond fixed context windows, while memory systems face costly decisions about what to store, connect, and retrieve. Jev-Mem separates lightweight memory control from deliberative reasoning across construction and retrieval, and reports the best overall accuracy and efficiency among evaluated baselines.

1 Introduction

  • Long-horizon agents accumulate preferences, task history, and environment knowledge beyond fixed context windows, and longer inputs do not ensure reliable use of all positions.The introduction also describes long-term indexing, retrieval, and reading challenges.Source: §1 · p. 1
  • Existing memory control uses fixed heuristics or autoregressive LLMs, trading inflexibility against repeated token generation and inference overhead.The paper frames memory typing, relation judgment, routing, scoring, and stopping as frequent structured decisions suitable for lightweight control.Source: §1 · p. 2
  • Jev-Mem separates lightweight structured memory control from expensive generative reasoning across memory construction and retrieval.Its System-One controller handles memory typing, redundancy filtering, relation construction, query routing, retrieval-budget allocation, graph traversal, candidate scoring, evidence assessment, and adaptive stopping; System Two handles complex synthesis and answer generation.Source: §1 · p. 2
  • Jev-Mem achieves the best overall accuracy and efficiency among state-of-the-art baselines.The introduction reports this result without giving quantitative values.Source: §1 · p. 2

2 Background and Motivation

Agentic memory has shifted from passive storage toward active organization and retrieval, raising the question of how to execute its frequent control decisions. Jev-Mem extends adaptive retrieval principles across the memory lifecycle and assigns bounded control decisions to lightweight structured prediction.

2.1 Agentic Memory

  • Agentic memory has evolved from stored interaction histories toward active organization, relation inference, consolidation, query routing, and adaptive retrieval.The memory system increasingly behaves as a dynamic subsystem that decides how information is organized and accessed.Source: §2.1 · p. 3
  • Memory update and retrieval require repeated decisions about characterizing and connecting new information, where to search, which candidates matter, and when to stop.The paper identifies the computational mechanism for these decisions as an under-attended systems question.Source: §2.1 · p. 3
  • Jev-Mem extends adaptive retrieval beyond evidence acquisition to memory construction, query routing, budget allocation, graph traversal, and stopping.It makes memory control an explicit part of the architecture rather than embedding it in prompts or fixed heuristics.Source: §2.1 · p. 3

2.2 The Opportunity for System-One Memory Control

  • Autoregressive LLMs make frequent memory-control decisions costly through token-by-token generation, formatting, and parsing on the critical path.Jev-Mem applies lightweight structured prediction to bounded decisions while reserving System Two for complex reasoning and answer synthesis.Source: §2.2 · p. 3, Figure 2 · p. 4

3 Jev-Mem Algorithm Design

Jev-Mem coordinates memory construction and retrieval through one System-One control mechanism over a shared multi-relational memory, reserving System Two for answer synthesis. Its controller makes structured decisions across the memory lifecycle, from organizing observations and routing queries to adaptive graph search and stopping.

3.1 Typed System-One Memory Control

  • The typed controller maps structured state and explicit decision questions to probabilities or distributions over mutually exclusive actions, batching decisions that share a state.The same control abstraction governs memory typing and relation construction on writes, and query routing, candidate evaluation, and stopping on reads.Source: §3.1 · pp. 4–5

3.2 System-One-Guided Memory Construction

  • Jev-Mem preserves each valid observation as a canonical node and selectively builds relations, avoiding irreversible store-or-discard decisions at ingestion.Independent relational views can connect the same node pair, while periodic maintenance can add links without removing original evidence.Source: §3.2 · p. 5

3.3 Adaptive System-One Retrieval

  • Retrieval routes queries across relevant relational views, allocates graph-search effort, scores candidates, and stops based on evidence sufficiency and the expected value of further search.It combines semantic and lexical anchors with adaptive graph expansion; multiple graph views can be active, and bounded search limits constrain the process.Source: §3.3 · pp. 5–7

3 Jev-Mem Algorithm Design

  • System Two synthesizes answers from selected evidence, while System One handles normal graph routing, candidate expansion, and stopping.The shared architecture uses canonical memory nodes with semantic, temporal, causal, and entity relations.Source: §3 · p. 4, §3.3 · p. 7

4 Experiments

Jev-Mem is evaluated on LoCoMo for answer quality and runtime efficiency against representative long-term memory baselines. It achieves the strongest reported overall score while reducing both construction time and query latency through adaptive System-One memory control.

4.1 Experimental Setup

  • LoCoMo tests ultralong multi-session conversational memory, and the evaluation pairs LLM-as-a-Judge answer quality with memory construction time and average per-query latency.Source: §4.1 · p. 7

4.2 Overall Performance

  • Jev-Mem’s largest reported category gains occur on Adversarial questions (0.962 versus 0.742 for the strongest baseline), with Multi-Hop at 0.625 versus 0.569 and Open-Domain at 0.610 versus 0.517.Source: §4.2 · p. 7
  • 0.777 overall LLM-as-a-Judge score on LoCoMo versus 0.700 for the strongest baseline (11.0% relative improvement); Jev-Mem wins five of six question types and ties temporal.Source: §4.2 · pp. 7–8, Table 1 · p. 8

4.3 Efficiency of System-One Memory Control

  • 158 s memory construction time versus 1,044 s for the fastest competing memory system (6.6× speedup), and 0.93 s/query versus 1.47 s for the fastest memory-based baseline.Query latency is also 46.6% lower than full-context processing at 1.74 s/query.Source: Table 2 · p. 8, §4.3 · p. 8

4.2 Overall Performance

  • Adaptive control routes queries across relational views, allocates retrieval effort, scores candidates, and stops when evidence is sufficient instead of following fixed search.Source: §4.2 · p. 8

4.3 Efficiency of System-One Memory Control

  • Lightweight typed decisions, bounded candidate sets, explicit graph-search budgets, and adaptive stopping reduce construction and retrieval overhead while preserving the highest overall LoCoMo accuracy.Source: §4.3 · p. 8

5 Conclusion

Jev-Mem separates frequent memory-control decisions from complex reasoning and answer synthesis. On LoCoMo, it reports the best overall accuracy and efficiency among the evaluated baselines.

5 Conclusion

  • Jev-Mem uses lightweight structured prediction for high-frequency memory-control decisions and reserves System Two for complex reasoning and answer synthesis.Source: §5 · p. 9
  • On LoCoMo, Jev-Mem reports an LLM-as-a-Judge score of 0.777, memory construction time of 158 s, and average query latency of 0.93 s.Source: §5 · p. 9
  • The authors conclude that separating memory control from generative reasoning is a promising direction for more effective and efficient long-horizon agents.Source: §5 · p. 9

A.1 Related Work

Related work moves agent memory from passive storage toward structured, evolving, and adaptive systems. Jev-Mem focuses on the computational mechanism for controlling memory decisions across construction and retrieval.

A.1 Related Work

  • Early agent-memory systems store experiences, synthesize reflections, update or forget memories, and manage information across tiers, while other work retains reusable skills, insights, workflows, or compressed gists.Source: §A.1 · p. 12
  • Recent systems organize evolving memory notes, episodic knowledge, memory tiers, or multi-relational graphs; corpus retrieval systems share structural tools but do not maintain an agent’s accumulating experience.Source: §A.1 · p. 12
  • Adaptive retrieval methods control whether, when, and how much evidence to retrieve within a generation episode, whereas Jev-Mem also controls writing, memory relations, traversal, and stopping across the memory lifecycle.Source: §A.1 · p. 12
  • Efficiency work reduces memory-pipeline computation through compression, asynchronous consolidation, lightweight online processing, or structured retrieval, while Jev-Mem targets fine-grained decisions inside the memory lifecycle.Source: §A.1 · p. 13
  • Jev-Mem uses the System-One/System-Two distinction as a computational analogy, applying it architecturally to the memory subsystem rather than claiming correspondence to human mechanisms.Source: §A.1 · p. 13

B Jev Prompts for Memory Writing and Retrieval

The appendix describes Jev-Mem’s typed prompts and decision flow for writing and retrieving conversational memories. Its examples illustrate request structure rather than measured predictions or retrieval outcomes.

B Jev Prompts for Memory Writing and Retrieval

B.1 Typed interface and request state

  • The typed interface batches explicit binary propositions over shared state, returning values from 0 to 1 that are not assumed calibrated probabilities; mutually exclusive choices use a separate choice format.Source: §B.1 · p. 13

B.2 Memory writing

  • Writing types each valid observation as episodic, semantic, procedural, or preference, then uses deterministic signals to select up to 10 candidate memories for relation judgments.Source: §B.2 · p. 14

B.3 Periodic consolidation

  • Every 20 successful writes, the active profile evaluates candidate pairs for consolidation; the normal path records decisions and links without replacing source memories.Source: §B.3 · p. 14

B.4 Retrieval routing and candidate scoring

  • Retrieval routes queries across relational needs and scores proposed candidates using relevance, added information, relation usefulness, current-evidence support, similarity, and recency.Source: §B.4 · p. 15

B.5 Evidence-based stopping

  • Evidence-based stopping checks sufficiency, missing evidence, contradiction, and the usefulness of another round, subject to depth, node, edge, attempt, and time limits.Source: §B.5 · pp. 15–16
  • Selected memories, not Jev-generated answers, are passed to a separate System-Two language model for final response generation.Source: §B.5 · p. 16
Loading 2609.23986v1…