Source-linked AI summary
Understanding Stage-Wise Utility-Risk Trade-offs in LLM Agent Memory
Chuanchao Zang, Zijian Cao, Xiangtao Meng, Jianing Wang, Wenyu Chen, Xinyu Gao, Li Wang, Zheng Li, Shanqing Guo
TL;DR
Long-term memory supports personalization and long-horizon interaction but can also admit poisoned information, while existing evaluations offer limited evidence about stage-specific utility–risk trade-offs. MEMGAUGE separately varies writing admission, management policy, and retrieval exposure under matched clean and poisoned conditions. Across 11 LLMs and two benchmarks, it identifies threshold-like writing risk, policy-dependent management decoupling, and coupled retrieval utility and risk, with qualitatively consistent diagnostic associations in four existing systems.
Problem
Existing evaluations usually measure benign utility or poisoning risk in isolation under fixed, end-to-end configurations, limiting stage-specific comparison of their trade-offs.
Method
MEMGAUGE separately varies writing admission, management policy, and retrieval exposure under paired clean and poisoned conditions, then applies analogous measurements to existing systems.
Results
Across 11 LLMs and two long-term memory benchmarks, writing shows threshold-like risk, management policy-dependent local decoupling, and retrieval coupled utility–risk growth; four existing systems show qualitatively consistent associations.
Takeaways & Limitations
Targeted poisoning risk varies across memory operations, motivating stage-aware evaluation and controls for long-term memory systems.
Takeaways & Limitations
Existing memory systems couple writing, management, and retrieval and do not expose the same intervention parameters as MEMGAUGE.
Abstract
from arXiv · showhide
Long-term memory is becoming a core capability of LLM agents, enabling personalization and long-horizon interaction. However, memory mechanisms that retain, transform, or expose more information can affect both benign utility and susceptibility to memory poisoning. Existing evaluations typically measure memory utility or attack risk in isolation under fixed configurations, providing limited insight into how stage-specific design choices reshape their trade-off. We present \textsc{MemGauge}, a controllable framework that separately varies writing admission, management policy, and retrieval exposure under matched clean and poisoned conditions. Across 11 LLMs and two long-term memory benchmarks, controlled evaluations reveal three distinct profiles: a threshold-like risk transition during writing, policy-dependent local decoupling during management, and coupled growth of utility and risk during retrieval. We further apply analogous stage-level measurements to four existing memory systems and observe diagnostic associations qualitatively consistent with these profiles. These results show that targeted poisoning risk varies across memory operations and motivate stage-aware evaluation and control of LLM-agent memory.
1 Introduction
Existing end-to-end evaluations obscure how writing, management, and retrieval separately reshape utility–risk trade-offs. MEMGAUGE controls these stages under paired clean and poisoned conditions, revealing distinct response profiles across models, benchmarks, and existing systems.
- Motivation: End-to-end metrics combine writing, management, and retrieval effects, leaving stage-specific utility–risk responses difficult to compare.Existing systems also couple these operations and expose nonaligned controls.
- Approach: MEMGAUGE varies writing admission, management policy, and retrieval exposure one stage at a time under matched clean and poisoned conditions.The framework holds the agent model, task, prompts, memory budget, and remaining operations fixed at each operating point.
- Findings: Writing shows a nonlinear transition: clean utility rises toward saturation, while attack success stays low under restrictive admission before increasing sharply.Admission levels expand along each agent’s model-specific ranking.
- Findings: Management exhibits policy-dependent local decoupling, with attack success shifting substantially while clean utility remains nearly unchanged.The direction and magnitude depend on the management policy.
- Findings: Retrieval exhibits coupled growth: exposing more helpful and risk-bearing memories improves clean utility while simultaneously increasing attack success.Analogous measurements in four existing systems align qualitatively with these controlled profiles.
- Implications: Across 11 LLMs and two benchmarks, these profiles motivate stage-aware evaluation and controls rather than relying on end-to-end metrics alone.The profiles persist across configurations and four existing memory systems, although onset and magnitude vary.
2 Related Work
Prior work evaluates long-term memory utility, memory-system mechanisms, and poisoning security, but generally studies complete pipelines through aggregate task, retrieval, or efficiency measures.
- Long-term memory evaluation: LoCoMo, LongMemEval, and MemBench evaluate long-term memory through multisession recall, temporal and cross-session reasoning, and knowledge updating.These benchmarks cover several long-horizon memory capabilities.
- Memory systems: Mem0 and A-MEM develop writing, retrieval, consolidation, and updating mechanisms while typically evaluating complete pipelines through task performance, retrieval quality, and efficiency.The passage situates these systems as end-to-end memory approaches.
3 MEMGAUGE Framework
MEMGAUGE constructs controlled stage-wise utility–risk profiles by varying writing admission, management policy, and retrieval exposure while holding the remaining pipeline fixed. It then applies analogous measurements to existing memory systems for diagnostic comparison.
- Framework overview: MEMGAUGE varies one memory-stage intervention at a time under matched clean and poisoned conditions, then computes paired utility and attack-risk outcomes.The framework applies the interventions in writing, management, and retrieval order.
- Writing: Controlling Information Admission: Writing admission divides helpful and risk-bearing traces into ranked atomic units and supplies nested ranked prefixes to the writer.The same LLM ranks units by estimated usefulness, and increasing coverage adds lower-ranked units from the same trace.
- Writing: Controlling Information Admission: Writing experiments reuse each source unit’s writer-generated records across coverage levels, so differences reflect admitted information rather than repeated generation.The writer processes each source unit once, and the resulting records are reused wherever that unit is admitted.
- Management: Controlling Memory-State Transformation: Management applies discrete policies to identical pre-management states, including unmanaged, provenance-aware quarantine, numeric-conflict resolution, and strict conflict resolution.Quarantining excludes a record from subsequent retrieval while leaving the remaining memory state unchanged.
- Retrieval: Controlling Decision-Relevant Exposure: Retrieval varies helpful and risk-bearing record exposure while keeping the context budget fixed at exactly K records.Risk-bearing exposure replaces an irrelevant background record, while both conditions share helpful exposure decisions, record positions, and background ordering.
- Applying MEMGAUGE to Memory Systems: Analogous measurements for existing systems estimate retained writing information, compare native management enabled versus disabled, and vary top-K retrieval budgets.Because existing systems couple stages and lack MEMGAUGE’s intervention parameters, these measurements are not direct controlled stage effects.
4 Experiments
Controlled experiments vary memory stages and related factors across 11 LLMs, while analogous measurements examine four existing systems. The results show threshold-like writing risk, coupled retrieval utility-risk growth, and management shifts that can reduce risk with limited utility loss.
- Experimental setup: MEMGAUGE evaluates writing, management, and retrieval interventions while holding the remaining agent and memory operations fixed, using paired clean and poisoned conditions.The controlled setup uses 11 LLMs and compares four existing systems through analogous, but not fully controlled, measurements.
- Stage-wise profiles: At low writing coverage, utility rises while ASR remains near zero; beyond θw = 0.3–0.5, ASR rises sharply and poisoned utility declines as clean utility continues increasing.At full coverage, ASR reaches 37%–64% and poisoned utility falls to 3%–34%.
- Profile stability: Trace load and ranker choice shift writing-stage risk, but preserve the qualitative response; at θw = 0.5, self-LLM and external-LLM rankers reach 43%–47% clean utility with 2%–8% ASR.Random ordering reaches 16% clean utility, while chronological ordering reaches 33% clean utility with 27% ASR.
- Profile stability: Risk-bearing retrieval exposure consistently amplifies ASR, increasing it from 0.13 to 0.63 even when helpful-record exposure remains fixed.Distractor composition shifts the operating point but does not remove the coupled retrieval response.
- Profile stability: At |B_i| = 500, PQ reduces ASR from 52% to 37% without changing clean utility, while NC and SC reduce ASR with clean-utility decreases of 8 and 6 percentage points.Memory-bank growth also reduces both clean utility and ASR through retrieval competition under UM.
- Existing-system correspondence: Across four existing systems, greater writing exposure raises risk without reliable utility gains, native management lowers ASR, and larger retrieval budgets increase both clean utility and ASR.These measurements are diagnostic associations rather than controlled stage effects.
5 Conclusion
MEMGAUGE finds that utility–risk responses differ by memory stage: writing has threshold-like risk, management can locally decouple outcomes, and retrieval couples utility gains with increased risk. These profiles persist across configurations and four existing memory systems, motivating stage-specific evaluation and controls.
- Writing shows a threshold-like risk transition as admission expands.Clean utility rises and risk increases sharply after remaining low at restrictive admission levels.
- Management enables mechanism-dependent local decoupling between utility and risk.The direction and magnitude of utility–risk shifts depend on the management mechanism.
- Retrieval couples utility gains with increased poisoning risk.Greater memory exposure raises both outcomes rather than separating them.
- The profiles persist across configurations and four existing memory systems, although onset and magnitude vary.
- End-to-end metrics alone are insufficient for understanding stage-specific utility–risk responses.The findings motivate stage-specific controls for long-term memory systems.