PaperlayerView exact revision on arXiv

Source-linked AI summary

MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging

Junxi Wang, Te Sun, Jiayi Zhu, Chen Zhang, Siyuan Li, Xuyang Liu, Zichen Wen, Xiaobing Tu, Jinkui Ren, Xiantao Zhang, Ziqi Yuan, Linfeng Zhang

arXiv:2609.08273v1cs.AI

TL;DR

As agent memories accumulate, storage and retrieval costs rise, while existing compression work largely focuses on generation or is limited in scope. MemForest post-processes stored memories by partitioning them into EventTrees and progressively merging nodes, with anchor-guided retrieval. At 50% memory compression, it retains 97.1% and 99.7% of original performance under Mem0 and M3-Agent, respectively, while achieving 1.89x and 2.24x retrieval speedups.

  • Problem

    Existing memory compression largely targets generation or is limited to graph-structured memories in streaming video, leaving general and scalable post-processing an open challenge.

    Source: §1 · p. 2

  • Method

    MemForest partitions historical memories into event-centric EventTrees using global semantic similarity and local temporal continuity, progressively merges nodes, and retrieves from key nodes’ temporal neighborhoods.

    Source: §1 · pp. 2–3

  • Results

    At 50% memory compression, MemForest retains 97.1% and 99.7% of original performance under Mem0 and M3-Agent, respectively, with 1.89x and 2.24x retrieval speedups.

    Source: §1 · p. 3, §6 · p. 9

  • Takeaways & Limitations

    MemForest provides a general memory-compression approach across different agent memory systems.

    Source: §1 · p. 2

    The theoretical analysis assumes L2-normalized embeddings and considers the case where one candidate memory node is at least as relevant to the query as another.

    Source: §4 · p. 6

Abstract

from arXiv · show

Agent memory systems have demonstrated significant potential in long-term dialogue, personalized assistants, and video understanding. However, continuously accumulated memory introduces substantial storage and retrieval costs during inference. To address this issue, we propose \textbf{MemForest}, a general memory compression framework adaptable to various agent memory systems. Specifically, MemForest partitions historical memory into event-centric units by leveraging global semantic similarity and local temporal continuity. For each unit, it constructs a maximum spanning tree, termed an EventTree, and progressively merges redundant memory nodes by selecting high-weight edges, reducing storage overhead. Furthermore, we introduce an anchor-guided propagation retrieval mechanism that retrieves relevant memory nodes from the temporal neighborhoods of key nodes, improving retrieval accuracy. Extensive experiments demonstrate the effectiveness of MemForest. Under the unimodal Mem0 framework, MemForest retains \textbf{97.1%} of the original performance while compressing \textbf{50%} of historical memory across three benchmarks (LoCoMo, LongMemEval, and PersonaMem), achieving a \textbf{1.89x} retrieval speedup. Under the multimodal M3-Agent framework, it preserves \textbf{99.7%} of the original performance with a \textbf{50%} compression ratio across two benchmarks (M3-Bench-robot and M3-Bench-web), achieving a \textbf{2.24x} retrieval speedup. \textcolor{RoyalBlue}{\textit{Our code is available at [https://github.com/Celina-love-sweet/MemForest.}}](https://github.com/Celina-love-sweet/MemForest.}})

1 Introduction

MemForest is a general framework for compressing stored agent memories by combining global semantic similarity with local temporal continuity. It progressively merges event-centric memory representations and uses anchor-guided retrieval to recover relevant temporal neighbors.

1 Introduction

  • Prior compression often targets memory generation, leaving stored memories to accumulate; existing post-processing work is limited to graph-structured memories in streaming video, motivating a general, scalable approach.Context-compression methods for MLLMs also overlook memory-specific event differentiation and temporal relationships, while textual memories are semantically rich and information-dense [45] [47].Source: §1 · p. 2
  • Memory compression must account for event-level properties: semantically similar nodes may recur across distant windows, while semantically different nodes in adjacent windows may belong to one event.The introduction illustrates these patterns with distant-node similarities of 0.73 and 0.68, and adjacent-node similarities of 0.12 and 0.23 for a Boston dining plan.Source: §1 · p. 2
  • MemForest partitions historical memories into event-centric EventTrees using global semantic similarity and local temporal continuity, then progressively merges their internal representations to a predefined compression ratio.The framework comprises EventTree Semantic-Temporal Partitioning and EventTree Progressive Merging modules.Source: §1 · p. 2
  • Anchor-Guided Propagation Retrieval recovers temporal structure degraded by compression by retrieving relevant nodes from the temporal neighborhoods of key memory nodes.The mechanism is intended to improve memory retrieval accuracy.Source: §1 · pp. 2–3
  • At 50% memory compression, MemForest retains 97.1% of original performance under Mem0 with 1.89× retrieval speedup, and 99.7% under M3-Agent with 2.24× speedup.The results cover three benchmarks under Mem0 and two under M3-Agent.Source: §1 · p. 3

2 Related Work

Agent memory systems span unimodal text and multimodal inputs, and scaling raises storage and retrieval costs, especially for multimodal memory. Visual token compression offers similarity-, diversity-, and attention-based approaches, but leveraging agent memory’s temporal relationships remains an open problem.

2.1 Agent Memory Systems

  • Unimodal and multimodal agent memory systems support persistent interaction, but growing memory scales increase storage and retrieval costs, especially for multimodal systems.Unimodal methods focus primarily on text, while multimodal methods handle inputs such as images, audio, and video.Source: §2.1 · p. 3

2.2 Visual Token Compression

  • Visual token compression methods use similarity, diversity, or attention criteria, while effectively exploiting agent memory’s temporal relationships remains an open problem.Similarity-based methods merge semantically similar tokens, diversity-based methods remove repetitive information, and attention-based methods retain high-weight tokens.Source: §2.2 · p. 3

3 Methodology

MemForest compresses historical memory by partitioning it into event units and progressively merging nodes, then uses anchor-guided temporal propagation to retrieve relevant memories.

3.1 Overview

  • The framework combines two EventTree-based compression modules with anchor-guided propagation retrieval to reduce historical-memory storage while improving retrieval accuracy.The compression modules are EventTree Semantic-Temporal Partitioning and EventTree Progressive Merging.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2 EventTree Semantic-Temporal Partitioning

  • KMeans [25] initializes event units from embeddings and selects center memories; other nodes are reassigned using combined semantic similarity and temporal-window continuity.The temporal window is controlled by a configurable coefficient.Source: §3.2 · p. 4

3.3 EventTree Progressive Merging

  • Each event unit becomes a maximum spanning tree built from embedding cosine similarity; degree-aware weighting protects hub nodes as an LLM iteratively merges selected pairs to a target compression ratio.Kruskal’s algorithm constructs the tree, and merging continues until a predefined compression ratio is reached.Source: §3.3 · pp. 4–5

3.4 Anchor-Guided Propagation Retrieval

  • Retrieval selects query-similar candidates, uses the top λ·k as anchors, propagates scores through candidate temporal neighborhoods, fuses propagation with query similarity, and returns the top k nodes.The final score balances query similarity and propagation using a weighting coefficient.Source: §3.4 · p. 5

4 Theoretical Analysis

MemForest’s retrieval analysis shows that merging highly similar memory nodes better preserves query similarity, while low-similarity merges can cause greater semantic deviation and hinder top-k retrieval.

Theoretical Analysis

  • The analysis assumes unit-normalized embeddings and, without loss of generality, that node m_i is at least as query-relevant as m_j (s_i ≥ s_j).Query similarities are defined as s_i = q⊤e_i and s_j = q⊤e_j, with node similarity ρ_ij = e_i⊤e_j.Source: §4 · p. 6
  • The merged memory embedding is a normalized interpolation of the two node embeddings, controlled by a weighting coefficient λ ∈ (0, 1).λ controls the interpolation between e_i and e_j.Source: §4 · p. 6
  • As node similarity ρ_ij increases, the derived lower bound on query similarity after merging increases, supporting preferential merging of highly similar nodes.When ρ_ij is small, the lower bound decreases, and the resulting deviation may prevent important nodes from being retrieved during top-k retrieval.Source: §4 · p. 6

5 Experiments

Experiments evaluate MemForest on unimodal Mem0 and multimodal M3-Agent benchmarks, reporting results at 50% memory compression, AGPR effects, ablations, and retrieval efficiency.

5.1 Experimental Setup

  • MemForest’s setup evaluates Mem0 and M3-Agent against pruning and merging baselines on three agent-memory and two streaming-video benchmarks.Experiments used two NVIDIA A100 (80GB) GPUs, GPT-4o-mini for memory-node merging, specified hyperparameters, and results averaged over three runs; other settings follow the original papers [8] [22].Source: §5.1 · p. 6

5.2 Main Results

  • 97.1% performance retention at 50% historical-memory compression: MemForest outperforms pruning- and merging-based baselines across three Mem0 benchmarks.At compression ratios above 50%, pruning methods drop more noticeably than merging methods; on PersonaMem, all methods degrade relatively little, which the authors attribute to multiple-choice questions needing limited memory.Source: Table 1 · p. 7, §5.2 · p. 7
  • 99.7% performance retention at 50% historical-memory compression: MemForest outperforms pruning- and merging-based baselines across two M3-Agent benchmarks.Above 50% compression, StreamMeCo performs worse than random compression; M3-Agent also shows smaller performance degradation than Mem0, which the authors associate with greater redundancy in dense visual inputs.Source: §5.2 · p. 7, Table 2 · p. 8
  • 1.2% and 21.0% performance improvements with 50% compression in Mem0 and M3-Agent, respectively, exceed the 0.2% and 17.6% gains without compression.AGPR consistently improves performance across compression ratios, with larger gains in M3-Agent; the authors say propagation enriches context around key nodes and compensates for information loss.Source: §5.2 · p. 8

5.3.1 MemForest Ablation Study.

  • At 50% compression across three benchmarks, combining global semantic similarity with local temporal continuity yields more accurate EventTree partitioning than the tested variants.The variants remove the global similarity score, remove the local continuity score, or replace the combined scoring with a sliding window.Source: §5.3.1 · pp. 8–9, Table 3 · p. 9
  • At 50% compression, replacing maximum-spanning-tree merging with random or minimum-spanning-tree merging significantly worsens performance.This comparison validates the paper’s theoretical analysis of progressive merging.Source: Table 4 · p. 9, §5.3.1 · p. 9

5.3.2 Time Efficiency Analysis.

  • 1.89× and 2.24× are the average retrieval speedups reported for Mem0 and M3-Agent, respectively, with 50% historical-memory compression.Mem0’s benchmark speedups are 2.00×, 1.92×, and 1.76×; M3-Agent’s are 2.41× and 2.06×. The authors attribute negligible AGPR overhead to avoiding complex embedding-similarity computations and say fewer M3-Agent retrieval rounds indicate more accurate retrieval of critical nodes.Source: Figure 3 · p. 9, §5.3.2 · p. 9

6 Conclusion … A.1 Baselines

MemForest is a general agent-memory compression framework with anchor-guided propagation retrieval; at 50% compression, it retains 97.1% and 99.7% of original performance under Mem0 and M3-Agent, with 1.89× and 2.24× retrieval speedups, respectively.

6 Conclusion

  • At 50% memory compression, MemForest retains 97.1% and 99.7% of original performance under Mem0 and M3-Agent, with 1.89× and 2.24× retrieval speedups, respectively.The framework also introduces anchor-guided propagation retrieval to retrieve more relevant neighborhood information.Source: §6 · p. 9

A.1 Baselines

  • Mem0 [8] extracts, consolidates, and retrieves conversation information, updating memory by adding, changing, or deleting information.The framework dynamically manages long-term memory for large language models.Source: §A.1 · p. 13
  • M3-Agent [22] builds entity-centric long-term memory from video, audio, and text, and collaborates multi-turn reasoning with memory retrieval.It continuously perceives multimodal inputs and accumulates environmental knowledge.Source: §A.1 · p. 13
  • Random pruning selects memory nodes at random, while Random Merging randomly merges retained-set nodes with nodes in the other set until reaching the target compression ratio.For Random Merging, the retained-set size is determined by the target compression ratio.Source: §A.1 · pp. 13–14
  • KMeans clusters memory nodes and retains the node closest to each cluster centroid, pruning the remaining nodes according to the desired compression ratio.Source: §A.1 · p. 13
  • DART [37] is adapted to memory compression by sampling 2% of tokens as pivots and retaining remaining tokens with the lowest overall similarity to them.The original approach prunes redundant visual tokens by favoring tokens dissimilar to selected pivots.Source: §A.1 · p. 14
  • StreamMeCo [34] compresses M3-Agent memory graphs by sampling representative isolated nodes and pruning connected nodes based on entity importance and semantic similarity.Its structure-aware dual-branch strategy addresses storage and retrieval bottlenecks in streaming video scenarios.Source: §A.1 · p. 14
  • ToMe [2] is adapted by alternating memory nodes into two sets, matching each node in one set to its most similar node in the other for merging.Each partitioning step can compress up to 50% of historical memory, with repeated steps used if needed to reach the target ratio.Source: §A.1 · p. 14

A.2 Datasets … B.1 Theoretical Analysis of Why Similar Memory Nodes Should Be Merged

The supplementary material describes the five datasets, implementation settings, and derivations underlying the theoretical analysis of merging similar memory nodes. The datasets span long-term dialogue and personalized memory as well as video-based multimodal reasoning.

A.2 Datasets

  • LoCoMo contains 10 long conversations and 1,986 questions spanning five categories of long-term memory and reasoning.Each conversation has about 27 sessions, 600 turns, and 16K tokens; question categories include single-hop, multi-hop, temporal, commonsense, and adversarial reasoning.Source: §A.2 · p. 14
  • LongMemEval has 500 dialogue histories and questions, while the study uses PersonaMem-32k with 222 histories and 589 questions.LongMemEval questions cover six types, including knowledge updates and temporal reasoning. PersonaMem-32k is one of the dataset’s 32k, 128k, and 1M context-scale versions, and covers personalized memory and reasoning.Source: §A.2 · p. 14
  • M3-Bench-robot comprises 100 videos and 1,276 questions, and M3-Bench-web comprises 920 videos and 3,214 questions for multimodal memory and reasoning.The robot videos are first-person and span seven daily environments; the web videos cover 46 categories. Both benchmarks include five task types, such as multi-evidence, multi-hop, and cross-modal reasoning.Source: §A.2 · p. 14

A.3 Additional Implementation Details

  • Experiments follow the original Mem0 and M3-Agent settings, using GPT-4o-mini for Mem0 evaluation and merging, and GPT-4o for M3-Agent evaluation.Mem0 uses text-embedding-3-small, whereas M3-Agent uses text-embedding-3-large.Source: §A.3 · p. 15
  • EventTree assignment in Section 5.3 uses chronological windows of 5 nodes, grouping adjacent windows when their average embedding similarity exceeds 0.5.Otherwise, a new EventTree starts from the latter window.Source: §A.3 · p. 15

B.1 Theoretical Analysis of Why Similar Memory Nodes Should Be Merged

  • The theoretical appendix introduces the triangle inequality and Cauchy–Schwarz inequality to derive Eqs. 13–15 concerning merged-node similarity.It presents these lemmas as preliminaries for the derivation.Source: §B.1 · p. 15
  • The derivation applies the triangle inequality to bound the similarity between a merged memory node and the query, establishing Eq. 13.The passage identifies this as a lower bound on the merged node’s similarity.Source: §B.1 · p. 15
  • The proof rewrites the difference between si and sj, applies Lemma 2 to upper-bound this term, computes the embedding-difference norm, and uses that result to establish Eq. 14.It then substitutes Eq. 14 into Eq. 13 to derive a further result.Source: §B.1 · p. 15

B.2 Theoretical Analysis of MemForest · C Memory Node Merging Prompt · D Additional Experiments

MemForest’s analysis argues that similarity-prioritized merging controls relevance loss, while its merging prompt guides timestamp-aware, concise summaries of paired memories.

B.2 Theoretical Analysis of MemForest

  • The analysis defines compression over historical memory H, where the compressed memory count M is smaller than the original count N.For each EventTree, it denotes pre- and post-compression node counts and the total number of merging operations.Source: §B.2 · p. 16
  • EventTree partitioning jointly uses global semantic similarity and local temporal continuity, with a relatively high minimum similarity threshold at initialization.The threshold is described as the minimum similarity under the initial partitioning.Source: §B.2 · p. 16
  • At each merge, MemForest selects the most-similar node pair to keep minimum similarity high and mitigate accumulated semantic deviation.The analysis frames each EventTree by its node counts before and after compression and sums per-step similarity loss across merges.Source: §B.2 · p. 16
  • MemForest’s compression loss depends on the compression scale and minimum EventTree similarity, supporting high query relevance even at high compression ratios.The analysis says prioritizing highly similar nodes keeps minimum similarity relatively high and controls error accumulation.Source: §B.2 · p. 16

C Memory Node Merging Prompt

  • The merging prompt asks for a concise, coherent summary of two memory entries and their timestamps, extracting their core information.It also asks that the summary stay within the combined length of the original memories as much as possible.Source: C Memory Node Merging Prompt · p. 17
  • When memories use relative time expressions, the prompt directs the summarizer to convert them into specific dates using the corresponding timestamps.Its example resolves “last June” from a February 2023 timestamp as June 2022.Source: C Memory Node Merging Prompt · p. 17
  • The prompt requires outputting only the final summarized result, without explanations, analysis, or reasoning steps.It supplies placeholders for each memory and its timestamp.Source: C Memory Node Merging Prompt · p. 17

D.1 Ablation Study under Different Ratios

The ablation study reports that global similarity and local continuity improve EventTree partitioning performance at 30% and 70% compression. Alternative memory-node merging strategies cause a notable performance drop.

D.1 Ablation Study under Different Ratios

  • The study reports impacts of different EventTree partitioning strategies at 30% and 70% compression of historical memory, and discusses global similarity and local continuity scores.Source: Table 5 · p. 17, Table 7 · p. 17, §D.1 · p. 17
  • At both 30% and 70% compression, the other two memory-node merging strategies cause a notable performance drop.The text says this drop is consistent with previous experiments.Source: Table 6 · p. 17, Table 8 · p. 17, §D.1 · p. 17

D.2 Other Ablation Study · D.3 Other Evaluation Metrics

MemForest’s node-degree ablation shows consistent gains across three representative benchmarks, while its B1 and F1 results outperform other approaches on LoCoMo and LongMemEval. The findings associate higher-degree EventTree nodes with critical event information and support avoiding their premature merging.

D.2 Other Ablation Study

  • Across three representative benchmarks and different compression ratios, incorporating node degree consistently improves performance over omitting it.This ablation examines the effect of deg(·).Source: §D.2 · p. 17, Table 9 · p. 17
  • Higher-degree nodes in the maximum spanning tree are typically event-core nodes that carry more critical information and should not be merged prematurely.The node-degree ablation supports preserving these nodes during merging.Source: §D.2 · p. 17

D.3 Other Evaluation Metrics

  • Figure 4 reports B1 and F1 retention relative to original performance across methods and compression rates on two benchmarks.The accompanying prose says MemForest consistently outperforms other approaches across LoCoMo and LongMemEval.Source: Figure 4 · p. 18, §D.3 · p. 19
  • For the Mem0 framework, MemForest consistently outperforms other approaches on LoCoMo and LongMemEval in B1 and F1 scores, as shown in Tables 10–11 and Figure 4.Source: §D.3 · p. 19
  • On LoCoMo, MemForest’s B1 and F1 performance is compared with other baselines under varying compression ratios.Table 10 covers these metrics and compression-ratio comparisons; the prose reports that MemForest outperforms other approaches across both benchmarks.Source: Table 10 · p. 18, §D.3 · p. 19
  • On LongMemEval, MemForest’s B1 and F1 performance is compared with other baselines under varying compression ratios.Table 11 covers these metrics and compression-ratio comparisons; the prose reports that MemForest outperforms other approaches across both benchmarks.Source: §D.3 · p. 19, Table 11 · p. 19

D.4 Parameter Sensitivity Analysis … E Case Study

The supplementary analyses identify parameter settings that maximize performance, report the cost and model dependence of memory merging, and illustrate EventTree partitioning and node merging. Detailed timing tables are noted as showing significant speedup, but their numerical results are not supplied here.

D.4 Parameter Sensitivity Analysis

  • At 50% compression on LoCoMo and M3-Bench-robot, performance on both benchmarks reaches its optimum at α=0.05, β=0.8, and time-window coefficient w=5.Source: §D.4 · pp. 19–20
  • At 50% compression, γ=0.9, L=4, and λ=0.2 each yield optimal retrieval performance on LoCoMo and M3-Bench-robot.γ balances semantic similarity and anchor-neighborhood information; L>4 leaves performance almost unchanged, while too few anchors or excessive anchors can impair retrieval.Source: §D.4 · pp. 19–20, Figure 6 · p. 20

D.5 Memory Node Merging Cost Analysis

  • GPT-4o-mini merging costs about $0.1 to compress 50% of LoCoMo’s roughly 100MB or 6,000 memory nodes, or roughly two hours of streaming video.Each merge uses a short prompt and two memory entries; the process can also use locally deployed open-source models.Source: §D.5 · p. 20

D.6 Merging Performance across Different Models

  • On LoCoMo at 50% compression, GPT-4o-mini and GPT-5.2 have comparable merging performance, while Qwen2.5-7B-Instruct performs worse.The authors suggest performance may saturate beyond a certain model scale and raise dedicated lightweight merging models as a question for future investigation.Source: §D.6 · pp. 20–21, Table 12 · p. 21

D.7 Detailed time efficiency results

  • Tables 13–14 report detailed timing results across three language benchmarks and two M3-Bench benchmarks, and are described as showing significant speedup.The supplied passages do not provide numerical timing values.Source: §D.7 · p. 21, Table 13 · p. 21, Table 14 · p. 21

E Case Study

  • A LoCoMo EventTree example combines semantically similar but temporally distant community-support memories with temporally adjacent, less-similar mentoring memories.The example illustrates partitioning that integrates global semantic similarity with local temporal continuity.Source: E Case Study · p. 21
  • An M3-Bench-robot example merges two related document-preparation memories into one summary that preserves their information while improving storage and subsequent retrieval efficiency.The merged memory describes organizing and preparing documents for a presentation.Source: E Case Study · pp. 22–23

F Limitations · G Future Work

MemForest’s limitation is pronounced degradation in low-redundancy Mem0 at high compression rates. Future work could specialize compression, improve merging and retrieval, and strengthen memory safety.

F Limitations

  • At 70% compression, Mem0 accuracy falls to 93.3% of the uncompressed setting, illustrating degradation in this low-redundancy framework.The paper suggests specialized compression methods to better preserve performance at high compression rates.Source: F Limitations · p. 23

G Future Work

  • Future work could develop compression methods specialized for low-redundancy frameworks such as Mem0 to better preserve performance at high compression rates.This direction responds to the degradation observed in Mem0 under high compression.Source: G Future Work · p. 23
  • A dedicated small memory-merging model could be trained to match or surpass the merging performance of large models such as GPT-4o-mini.The paper notes this possibility in connection with Section D.6.Source: G Future Work · p. 23
  • Current top-k retrieval methods use limited embedding-similarity computations, can leverage vector databases, and typically retrieve in under one second.The paper contrasts this approach with newer methods that perform extensive similarity computations.Source: G Future Work · p. 23
  • xMemory requires framework redesign and cannot use vector-database acceleration, resulting in retrieval times tens of times longer.Future work could minimize embedding-similarity computations while balancing retrieval speed and accuracy.Source: G Future Work · p. 23
  • Future work could analyze historical memories for conflicts or potentially corrupted entries, then correct or remove them to improve system robustness and reliability.Source: G Future Work · p. 23

Related papers

Loading 2609.08273v1…