Source-linked AI summary
FadeMem: Biologically-Inspired Forgetting for Efficient Agent Memory
Lei Wei, Xiao Peng, Xu Dong, Niantao Xie, Bin Wang
TL;DR
Agent memory systems lack selective forgetting, leading to boundary forgetting or information overload as stored context grows. FadeMem introduces biologically inspired adaptive decay in a dual-layer hierarchy with LLM-guided conflict resolution and memory fusion. Across Multi-Session Chat, LoCoMo, and LTI-Bench, it reports consistent improvements in reasoning and retrieval with 45% storage reduction.
Problem
Existing agent memory systems lack selective forgetting and treat information with equal importance, causing catastrophic forgetting or context overload.
Method
FadeMem uses dual-layer memory with importance-modulated exponential decay, LLM-guided conflict resolution, and intelligent memory fusion.
Results
Across Multi-Session Chat, LoCoMo, and LTI-Bench, FadeMem consistently improves multi-hop reasoning and retrieval while reducing storage by 45%.
Takeaways & Limitations
Adaptive forgetting can retain critical information while allowing irrelevant details to fade, balancing memory capacity and retrieval precision.
Abstract
from arXiv · showhide
Large language models deployed as autonomous agents face critical memory limitations, lacking selective forgetting mechanisms that lead to either catastrophic forgetting at context boundaries or information overload within them. While human memory naturally balances retention and forgetting through adaptive decay processes, current AI systems employ binary retention strategies that preserve everything or lose it entirely. We propose FadeMem, a biologically-inspired agent memory architecture that incorporates active forgetting mechanisms mirroring human cognitive efficiency. FadeMem implements differential decay rates across a dual-layer memory hierarchy, where retention is governed by adaptive exponential decay functions modulated by semantic relevance, access frequency, and temporal patterns. Through LLM-guided conflict resolution and intelligent memory fusion, our system consolidates related information while allowing irrelevant details to fade. Experiments on Multi-Session Chat, LoCoMo, and LTI-Bench demonstrate superior multi-hop reasoning and retrieval with 45\% storage reduction, validating the effectiveness of biologically-inspired forgetting in agent memory systems.
1. INTRODUCTION
Existing agent memory systems lack selective forgetting, treating information with equal importance and causing either boundary forgetting or context overload. FadeMem addresses this gap with biologically inspired adaptive forgetting, dual-layer memory, conflict resolution, and memory fusion.
- Existing agent memory architectures lack selective forgetting, causing catastrophic forgetting at context boundaries or information overload within them.
- Binary retention strategies either preserve everything within capacity or lose it entirely, unlike human memory’s adaptive decay of unimportant information.
- FadeMem introduces active forgetting through differential decay rates across a dual-layer memory hierarchy.
- Retention adapts to semantic relevance, access frequency, and temporal patterns, while LLM-guided resolution and memory fusion consolidate related information.
- Experiments on Multi-Session Chat, LoCoMo, and LTI-Bench report superior multi-hop reasoning and retrieval with large storage savings.
2. METHODOLOGY
FadeMem combines a dual-layer memory hierarchy with adaptive decay, biologically inspired forgetting curves, conflict resolution, and LLM-guided fusion. Memories are assigned and updated according to importance, access, recency, semantic relationships, and preservation checks.
- Dual-Layer Memory Architecture with Differential Forgetting: The architecture represents memories with content, text, strength, creation time, and access frequency, then assigns them to layers using an importance score.
- Dual-Layer Memory Architecture with Differential Forgetting: High-importance memories enter the Long-term Memory Layer with slow decay, while low-importance memories enter the Short-term Memory Layer with rapid decay.
- Dual-Layer Memory Architecture with Differential Forgetting: Layer transitions adapt as importance changes, using validation-tuned promotion and demotion thresholds with hysteresis to prevent oscillation.
- Biologically-Inspired Forgetting Curves: Memory decay follows differential exponential functions whose rate adapts to importance, while layer-specific shape parameters model slower long-term and faster short-term forgetting.
- Biologically-Inspired Forgetting Curves: Access strengthens memories with diminishing returns, while pruning removes memories below a strength threshold or dormant beyond a maximum duration.
- Memory Conflict Resolution: New memories trigger semantic retrieval and LLM classification into compatible, contradictory, subsuming, or subsumed relationships for targeted resolution.
- Memory Conflict Resolution: Compatible memories coexist with redundancy penalties, contradictory memories use newer-information suppression, and subsuming relations trigger LLM-guided merging.
- Adaptive Memory Fusion: Fusion clusters memories by semantic similarity and temporal proximity, preserves unique information and causal relationships, and rejects fusion when preservation falls below threshold.
3. EXPERIMENTS
Experiments on LTI-Bench, MSC, and LoCoMo evaluate retention efficiency, conflict resolution, retrieval, reasoning, and component contributions. FadeMem retains critical information with reduced storage, performs strongly across datasets, and depends on all major memory components.
- Memory Retention and Forgetting Dynamics: 82.1% retention of critical facts uses only 55.0% storage on LTI-Bench after 30 days.Important memories decay 3–5× more slowly than baseline, and 23% of low-importance memories are promoted to the long-term layer based on access patterns.
- Conflict Resolution Performance: 68.9% macro-averaged accuracy and 80.4% macro-averaged consistency are achieved across three controlled conflict types.The evaluation injects 4075 controlled conflicts on LTI-Bench.
- Cross-Dataset Evaluation: FadeMem achieves 77.2% RP@10 and 0.82 TCS on MSC, while LoCoMo multi-hop F1 reaches 29.43 versus 28.37 for Mem0 and 9.46 for MemGPT.On LoCoMo, factual consistency reaches 85.9% while storage reduction is 45% (SRR=0.45).
- Ablation Study: Removing the dual-layer architecture lowers multi-hop F1 from 29.43 to 19.45, a 33.9% decrease.The result highlights the role of separating long-term and short-term memories.
- Ablation Study: Removing memory fusion produces the largest degradation, reducing multi-hop F1 to 13.63, a 53.7% decrease.Removing conflict resolution reduces multi-hop F1 to 22.88, a 22.4% decrease; the full model reaches F1=46.52 on temporal tasks and F1=44.35 on open-domain tasks.
4. CONCLUSION
FadeMem introduces adaptive forgetting for LLM-based agent memory through differential decay, conflict resolution, and memory fusion. Across MSC, LoCoMo, and LTI-Bench, it improves retrieval and reasoning while reducing storage by 45%.
- Conclusion: FadeMem combines differential decay, LLM-guided conflict resolution, and intelligent memory fusion to retain critical information while reducing storage by 45%.The architecture uses semantic relevance, access frequency, and temporal patterns to modulate decay.
- Conclusion: Experiments on Multi-Session Chat, LoCoMo, and LTI-Bench show consistent improvements in multi-hop reasoning and retrieval over existing baselines.The conclusion attributes these results to selective forgetting inspired by human memory.
- Conclusion: Selective forgetting is presented as essential for preventing information overload and maintaining relevance in agent memory systems.The dual-layer hierarchy consolidates related information while allowing irrelevant details to fade.