PaperlayerView exact revision on arXiv

Source-linked AI summary

Zero-Mem: Zero-Token Memory Operations for LLM Agents

Yilin Xiao, Zhehan Zhu, Yujing Zhang, Jin Chen, Zijin Hong, Luyao Zhuang, Qinggang Zhang, Shengyuan Chen, Xiaocao Ouyang, Lingfei Ren, Xiao Huang

arXiv:2607.29377v1cs.CL

TL;DR

LLM memory systems often rely on extra model calls, raising recurring token and time costs. Zero-Mem instead retrieves provenance-bearing traces through complementary relational and temporal structures, achieving competitive QA performance while reducing memory-operation latency by 57.6%.

  • Problem

    Whether structured agent memory can eliminate LLM calls and token consumption outside final question answering while preserving access beyond flat similarity retrieval remains open.

  • Method

    Zero-Mem preserves original traces and performs deterministic evidence selection through query-coordinated entity–context graph and temporal hierarchy views.

  • Results

    57.6% lower memory-operation latency than the fastest baseline accompanies competitive performance across long-context and long-memory QA benchmarks with zero memory-operation LLM calls and tokens.

  • Takeaways & Limitations

    The results support structured agent memory without generating an intermediate representation of the past.

  • Takeaways & Limitations

    The memory substrate supports indexing, seeding, and scoring only; it does not generate or rewrite memory content.

Abstract

from arXiv · show

LLM agents need memory to act consistently over long interactions, yet many systems use additional LLM calls to operate that memory. Generating intermediate records and mediating their retrieval adds recurring token and time costs, while omitted or merged details can obscure the original evidence. We ask whether structured memory access requires generation at all. Zero-Mem introduces \emph{zero-token memory operations}: no step outside final question answering invokes an LLM or consumes LLM input or output tokens; encoder computation is accounted for separately. Zero-Mem preserves original interaction traces as its source of record. It organizes the traces in two complementary ways. An entity--context graph exposes connections across interactions, while a temporal hierarchy preserves conversational locality and session state. For each query, Zero-Mem weighs the two views, retrieves from both, and follows their structure to recover supporting relations or surrounding context. Deterministic calibration first discards conflicting evidence and then keeps the reader's answer grounded in the retrieved traces. Only the final-QA reader invokes an LLM. Across long-memory and long-context question-answering benchmarks, Zero-Mem achieves competitive performance while eliminating LLM calls and LLM-token consumption from memory operations. With the same final-QA reader and context budget, it reduces memory-operation time cost by 57.6\% relative to the fastest compared baseline. Ablations support the contribution of the two views and their query-dependent coordination. Overall, the results show that structured agent memory need not generate an intermediate representation of the past. After peer review, the code and implementation details will be available at \textcolor{blue}{https://github.com/TheMoon0815/Zero-mem}.

Introduction

Zero-Mem treats agent memory as zero-token, provenance-preserving structured evidence selection over original interaction traces. It combines relational and temporal views, achieving competitive benchmark performance and a 57.6% reduction in memory-operation latency versus the fastest baseline.

  • Motivation: Long-interaction agents need memory that preserves information across sessions while recovering relevant evidence without irrelevant or outdated traces dominating current decisions.Their histories accumulate utterances, actions, tool observations, and task outcomes, making evidence recovery central to reliability.
  • Prior Work: Prior systems use language models to summarize experience, construct abstractions or graph indexes, and generate linked memory records, incurring generative overhead.SimpleMem reduces token use through structured compression and retrieval planning, while LightMem shifts operations to smaller models and offline consolidation; neither eliminates generative memory operations.
  • Method: Zero-Mem preserves original interaction traces and derives complementary entity–context graph and temporal-hierarchy views for structured evidence selection.The graph supports relational access, while the temporal hierarchy preserves conversational locality and session state.
  • Results: 57.6% lower memory-operation latency than the most time-efficient baseline, with competitive performance across long-context and long-memory QA benchmarks.The comparison uses an identical final-QA reader and equivalent context budget; memory operations consume zero LLM calls and tokens.
  • Contribution: Zero-token agent memory defines a regime where every operation outside final QA uses zero LLM calls and zero input or output tokens.Memory-operation cost is separated from final-reader inference.

Related Work

Prior agent-memory systems organize, update, and retrieve growing interaction histories. Zep uses temporally aware knowledge-graph memory, while Mem0 incrementally extracts and updates memories through LLM tool calls.

Preliminaries

LLM agents accumulate interaction histories and retrieve relevant information to answer current queries. Zero-Mem implements memory construction, organization, retrieval, routing, and calibration without generation, followed by a reader LLM that produces the answer.

  • An agent history H consists of trace units containing messages, responses or actions, tool observations, timestamps, speakers, and session metadata.
  • Given a query q, the memory system retrieves relevant information from H to construct an evidence set for answering.
  • A reader LLM uses the retrieved evidence to produce the answer.
  • Zero-Mem instantiates memory through non-generative construction, organization, retrieval, routing, and calibration.

Method

Zero-Mem preserves raw interaction traces as provenance-linked evidence and organizes them into complementary relational and hierarchical views. Deterministic query routing, dual-view retrieval, structural closure, and calibration select grounded evidence without generating or rewriting memory content.

  • Provenance-preserving memory substrate: Zero-Mem retains original text, source identifiers, session metadata, and boundaries, keeping retrieved evidence traceable to observed interactions rather than generated abstractions.Indexing representations support retrieval and scoring only; they do not generate or rewrite memory content.
  • Dual-view trace organization: The relational graph records entity–context co-occurrence and trace adjacency, while the hierarchy preserves turns, windows, episodes, local spans, temporal order, and session context.Graph edges capture observed associations rather than generated semantic triples or inferred relations.
  • Query-conditioned evidence routing: Deterministic query-structure signals route priority between graph and hierarchy views, while both remain active for evidence fusion.Relational queries prioritize the graph; local queries prioritize the hierarchy, using signals such as question form, temporal requirements, aggregation, and subject anchors.
  • Dual-view evidence retrieval: Graph retrieval propagates query-matched entity activation through relevant co-occurrence sentences and applies personalized PageRank, with lexical matches refining names, dates, values, and phrases.The hierarchical view instead performs coarse-to-fine episode, window, and turn retrieval using semantic relevance and structural compatibility.
  • Evidence fusion and closure: Normalized rankings are fused with query-dependent weighting, then augmented with bounded graph bridges and neighboring turns or local spans before duplicate merging.The resulting compact evidence set combines relational and local support, with either support set allowed to be empty.

Experiment · Experimental Setup

Zero-Mem is evaluated on complementary long-term conversational-memory and multi-hop question-answering benchmarks against memory-free and memory-based baselines. Experiments use matched final-QA readers and context budgets across closed- and open-source backbone LLM settings to isolate memory-pipeline differences.

  • Datasets.: Zero-Mem is evaluated on LoCoMo’s single-hop, multi-hop, temporal-reasoning, and open-domain tasks and on HotpotQA’s Wikipedia-based multi-hop question-answering benchmark.LoCoMo targets long-term memory over extended, multi-session interactions, while HotpotQA evaluates multi-hop question answering.
  • Baselines.: Memory-free baselines include LONG-LLM, which independently processes sliding-window history blocks, and RAG, which retrieves five semantically similar 2,048-token chunks.Both baselines generate answers from partitioned or retrieved interaction-history context rather than specialized memory structures.
  • Baselines.: All methods use four concurrent threads, GPT-4o-mini, and identical test hardware in the unified efficiency comparison.Relative Gain/Reduction is computed against the underlined result in Table 2.
  • Baselines.: Memory-based baselines include A-Mem, Mem0, MemoryOS, LightMem, SimpleMem, CompassMem, and GAM, which maintain specialized structures over historical information.These methods access their memory structures during inference for memory-grounded tasks.
  • Implementation Details.: GPT-4o-mini and Qwen2.5-14B-Instruct represent closed-source and open-source backbone settings for Zero-Mem and all baselines.Experiments run on a common hardware environment equipped with NVIDIA RTX 4090 GPUs.
  • Implementation Details.: Within each backbone setting, identical final-QA readers and equivalent context budgets isolate differences in the memory pipelines.The damping factor γ and dual-view routing coefficient ρ are both set to 0.6.
  • Implementation Details.: Performance is compared by F1 score on HotpotQA across different base LLMs and context-length settings.This comparison is reported in Table 3.

Main Results

Zero-Mem outperforms competing memory methods on LoCoMo and HotpotQA, achieving the strongest reported overall scores across readers, question types, and context lengths. Its advantage persists at 448K tokens and is especially large against flat retrieval and long-context baselines.

  • LoCoMo: Zero-Mem achieves the best average F1 and BLEU-1 on LoCoMo under both GPT-4o-mini and Qwen2.5-14B readers.It improves over GAM by 5.40 and 5.45 points with GPT-4o-mini, and by 4.87 and 4.86 points with Qwen2.5-14B, respectively.
  • LoCoMo: On LoCoMo, Zero-Mem leads most question types and all metrics with Qwen2.5-14B while remaining competitive with GAM on GPT-4o-mini multi-hop questions.With GPT-4o-mini, it leads on single-hop, temporal, and open-domain questions; with Qwen2.5-14B, it ranks first across every question type and metric.
  • LoCoMo: Zero-Mem’s margins over LONG-LLM and RAG are particularly sizable on LoCoMo temporal and open-domain questions.These results indicate that long-context access or flat similarity retrieval alone is insufficient for the evaluated memory tasks.
  • HotpotQA: Zero-Mem achieves the highest HotpotQA F1 across all readers and context lengths, including 448K tokens, improving 5.52 points over the strongest baseline on average.HotpotQA progressively adds distracting passages to test locating and connecting distributed evidence under increasingly long contexts.

Efficiency Comparison

Under matched GPT-4o-mini conditions, Zero-Mem compares answer quality and memory-operation overhead with GAM, SimpleMem, and LightMem. It uses zero LLM tokens for memory operations and reduces latency relative to LightMem while retaining nonzero encoder, retrieval, organization, and calibration costs.

  • Efficiency Comparison: Zero-Mem is evaluated against GAM, SimpleMem, and LightMem under the same GPT-4o-mini concurrency and hardware settings.The comparison reports both answer quality and total and per-query memory-operation overhead outside the shared final-QA reader.
  • Efficiency Comparison: On HotpotQA with 56K-token contexts, the full model outperforms single-view variants, while evidence closure and calibration further improve F1 and BLEU-1.The ablation supports complementarity between graph-based relational retrieval and hierarchical contextual retrieval.
  • Efficiency Comparison: 57.6% lower memory-operation latency than LightMem, with 334.77 seconds total and 0.22 seconds per query.LightMem is identified as the fastest baseline, while Zero-Mem requires no LLM input or output tokens for memory operations.
  • Efficiency Comparison: Zero-token memory operation still incurs encoder inference, memory organization, retrieval, and deterministic calibration costs.Even LightMem, the most token-efficient baseline, consumes more than 0.87 million tokens.

Ablation Study

On HotpotQA, Zero-Mem’s full configuration substantially outperforms either single-view variant, showing the value of combining graph and hierarchical retrieval. On LoCoMo, performance peaks at top-10 retrieval, while top-5 is used for baseline-matched main experiments.

  • HotpotQA ablations: The full model reaches 72.07 F1 and 69.66 BLEU-1, outperforming graph-only and hierarchical-only variants on HotpotQA.Graph-only scores 62.50 F1 and 59.90 BLEU-1, while hierarchical-only scores 54.88 and 51.40, respectively.
  • HotpotQA ablations: The HotpotQA ablation compares the full model with single-view variants and versions without evidence closure or calibration under fixed settings.
  • Retrieval budget: LoCoMo performance improves from top-1 to top-5 retrieval, peaks at top-10, and remains stable with larger budgets.Top-5 is used in the main experiments to match all baselines’ retrieval setting.

Effect of the Retrieval Budget

Zero-Mem benefits substantially from increasing the retrieval budget, with performance peaking at Top −10 before additional evidence produces only minor fluctuations.

  • Effect of the Retrieval Budget: Top −10 achieves Zero-Mem's highest overall performance, while larger retrieval budgets provide only minor fluctuations and diminishing returns.The retrieval budget is the maximum number of primary candidates retained in M(q) before evidence closure.
  • Effect of the Retrieval Budget: Increasing Top −K from 1 to 5 raises average F1 from 52.59 to 59.15 and BLEU-1 from 46.79 to 52.96.

Conclusion

Zero-Mem formalizes zero-token memory operations, eliminating LLM calls and input/output token consumption from all operations outside final question answering. It preserves original interaction traces and retrieves evidence through complementary relational and temporally ordered views without generating intermediate memory representations, achieving competitive performance across long-term conversational memory and lo…

  • Conclusion: Zero-Mem formalizes zero-token memory operations, where operations outside final question answering invoke no LLM and consume no LLM input or output tokens.This defines the operating regime underlying the system.
  • Conclusion: Zero-Mem preserves original interaction traces and retrieves evidence through complementary relational and temporally ordered views without generating intermediate memory representations.The approach combines relational structure with temporal ordering while retaining the original traces as evidence.
  • Conclusion: Comprehensive experiments demonstrate competitive performance across long-term conversational memory and lo….
Loading 2607.29377v1…