PaperlayerView exact revision on arXiv

Source-linked AI summary

SAFE-Mem: Secure Agentic Memory with Fine-Grained Authority and Governance-Aware Retrieval

Haoming Li, Dingyi Kang, Xu Hu, Jinxin Yang, Zhu Xu, Chenru Wang, Yi Ge, Miao Yin, Guanpeng Li, Feng Chen, Bingzhe Li

arXiv:2610.22959v2cs.CRcs.CL

TL;DR

External claims in persistent memory can later be treated as authorized user preferences or policies, while existing defenses can sacrifice legitimate memory use. SAFE-Mem labels claims by kind, authority, and scope and applies governance-aware retrieval. Across four backbones, it reports lower attack success among defenses retaining legitimate memory and higher answer accuracy than evaluated defenses.

  • Problem

    Existing content-based and coarse-provenance defenses can fail to prevent unauthorized memory use or block legitimate information, sacrificing memory utility.

    Source: §1 · p. 1

  • Method

    SAFE-Mem labels claims by kind, authority, and scope, links them to sources in a memory graph, and admits them for retrieval only when their permissions fit the requested use.

    Source: §1 · p. 2, §4.2 · p. 5

  • Results

    Across four backbones, SAFE-Mem has the lowest MPBench attack success among defenses retaining legitimate memory, at least 77% below the strongest, and the highest LoCoMo accuracy, improving it by up to 15 points.

    Source: §1 · p. 2

  • Takeaways & Limitations

    SAFE-Mem keeps external claims available for uses their sources support while withholding uses beyond their authority, improving recall and accuracy alongside lower attack success.

    Source: §1 · p. 2, §7 · p. 8

    SAFE-Mem governs authority rather than factual truth: false claims that do not borrow authority lie outside its scope, so content-judging defenses complement it.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language model (LLM) agents increasingly rely on persistent memory to support long-term reasoning and personalized interactions. However, information collected from external sources can introduce security risks when untrusted claims (i.e., atomic statements) are later treated as authorized preferences, policies, or permissions. Existing defenses typically rely on content-based filtering or coarse-grained provenance tracking, which either fail to prevent unauthorized memory use or block legitimate information, sacrificing memory utility. To address these limitations, we propose SAFE-Mem, a fine-grained memory governance framework that improves security while preserving legitimate memory use. First, SAFE-Mem annotates each claim with three governance labels (i.e., kind, authority, and scope) to capture its provenance and permitted use. Second, governance-aware retrieval matches query requirements with claim permissions to block unauthorized influence while retaining legitimate information. Third, a governance-enabled memory graph connects source sentences, claims, and entities to support provenance tracking and efficient retrieval. Across four LLM backbones on MPBench and LoCoMo, SAFE-Mem attains the lowest attack success among defenses that retain legitimate memory, at least 77% below the strongest of them, and the highest answer accuracy of all evaluated defenses, improving it by up to 15 points. The code is available at https://github.com/LVenum/SAFE-Mem.

1 Introduction

Persistent memory can turn external claims into apparent user policies when their source authority is lost, while existing defenses either permit unauthorized use or block legitimate memory. SAFE-Mem governs claims and their retrieval, retaining legitimate uses while reducing attack success and improving accuracy.

1 Introduction

  • External claims can persist beyond their original session and acquire authority they never held, causing authority laundering or misattribution between people.Existing content-based and coarse provenance defenses can either leave unauthorized use exposed or block legitimate uses of the same information.Source: §1 · pp. 1–2
  • Across four backbones, SAFE-Mem has the lowest MPBench attack success among defenses retaining legitimate memory, at least 77% below the strongest, and the highest LoCoMo accuracy, improving it by up to 15 points.It also prevents every attack from executing on EHRAgent under MINJA.Source: §1 · p. 2
  • SAFE-Mem labels each claim with kind, authority, and scope so permitted use reflects both the source and the claim’s reach.A principal’s explicit retention request can raise authority up to the kind named in that request.Source: §1 · p. 2
  • Governance-aware retrieval admits claims only when their kind and authority meet the request’s required kind within its scope, checking again at tool calls.This lets a claim serve supported uses without granting it authority beyond its source.Source: §1 · p. 2
  • A unified memory graph links source sentences, claims, and entities to preserve provenance, support conflict detection, and retrieve evidence in the parties’ own words.Each claim remains traceable to its source sentences and relevant entities.Source: §1 · p. 2

2 Related Work

Prior agent-memory work emphasizes storing and retrieving information by relevance, recency, and consistency, while defenses address content or provenance in different ways. SAFE-Mem adds claim-level, use-dependent admission for answers.

Agentic memory

  • Memory systems organize and retrieve stored information by relevance, recency, and consistency, while SAFE-Mem also records each claim’s permitted uses.The reviewed systems include linked notes, salient facts, temporal knowledge graphs, multi-graph routing, and memory compression.Source: §2 · p. 2

Attacks on agent memory

  • Memory-poisoning research examines persistent attacks, while authority laundering is identified as the distinct failure of treating lower-authority memory as authoritative.Prior work reports that injected memories can be acted on in later sessions.Source: §2 · pp. 2–3

Defending agent memory

  • Existing defenses judge content, follow instruction roles, gate actions by provenance, or exclude untrusted data from planning; SAFE-Mem extends use-dependent admission to answers.Its admission uses graded kinds and party scope.Source: §2 · p. 3

3 Threat Model

The threat model concerns agents that retain information from external artifacts and later use it in answers or actions. The adversary seeks to make memory carry authority its source did not possess.

3 Threat Model

  • The agent serves one or more principals across sessions, reads external artifacts, and distills their contents into persistent memory for later requests.Authority over a principal originates only in that principal’s assertions, instructions, and authorizations; runtime channel records are trusted.Source: §3 · p. 3
  • An adversary controls external artifact content but cannot alter principal turns, write directly to memory, forge channel records, or access component prompts or weights.It may plant arbitrary claims, including text styled as the principal’s words, and spread paraphrases across artifacts long before a target session.Source: §3 · p. 3
  • Success means a later answer or action relies on memory as an unauthorized preference, decision, trust assignment, policy, or permission; merely reporting what an artifact said does not count.The defender aims to deny uses unsupported by a source while retaining uses that the source supports.Source: §3 · p. 3

4 Methodology

SAFE-Mem attaches governance to claims, traces them through a memory graph, and admits them for uses their labels support. Retrieval checks permissions before ranking and applies governance again at tool calls.

4 Methodology

  • SAFE-Mem’s pipeline attaches governance to claims, links claims to source inputs and entities, and admits them for supported uses before ranking.Authority, scope, and use derive from runtime channel records and the principal’s words.Source: §4 · p. 3

4.1 Governance Labels

  • Each claim has a kind, authority, and scope; its effective level is bounded by the weaker of kind and authority.Kinds range from description and attribute through preference and rule to permission; explicit retention requests raise defaults only up to the requested kind.Source: §4.1 · pp. 3–4

4.2 Memory Graph

  • The memory graph preserves ordered source sentences and links claims to their source inputs and entities, with shared entities bringing claims about the same subject together.A principal’s explicit retention request is the only edge that raises a claim’s authority; artifact-derived claims retain their source link while also linking to the endorsement request.Source: §4.2 · pp. 4–5

4.3 Governance-Aware Retrieval

  • For each request, retrieval determines the required use and target, admits only claims whose effective level meets those requirements, and ranks the admissible claims.It checks same-subject, same-relation contradictions and marks conflicting positions contested; tool calls are also checked against the levels of memory entries supplying identifiers.Source: §4.3 · p. 5

5 Experiments

Across MPBench and LoCoMo, SAFE-Mem reduces attack success while retaining legitimate memory use and improving recall and answer accuracy. On EHRAgent, it prevents attacks from executing while removing few memories retrieved for benign questions.

5.1 Experimental Setup

  • The evaluation covers MPBench, LoCoMo, and EHRAgent across four LLM backbones, using benchmark-specific attack, recall, restraint, accuracy, and memory-loss measures.MPBench includes 3,241 attack and 2,999 benign cases; LoCoMo has ten conversations and 1,540 recall questions, while EHRAgent uses shared patient records under MINJA.Source: §5.1 · pp. 5–6

5.2 Results on MPBench and LoCoMo

  • On GPT-5.6-terra and GPT-4o-mini, SAFE-Mem lowers MPBench ASR from 26.7% to 2.6% and from 49.8% to 6.2%, respectively.It also raises recall to 86.3% and 85.9%, while improving restraint by 13 to 14 points.Source: §5.2 · p. 6
  • On LoCoMo, SAFE-Mem improves accuracy on every backbone, including gains of 11.1 and 15.1 points on the two GPT models.On LongMemEval, judge accuracy rises from 45.2% to 58.2% on GPT-4o-mini.Source: §5.2 · p. 7

5.3 Memory Shared Across Principals

  • On EHRAgent, SAFE-Mem prevents every attack from executing while removing 2.0% of memories retrieved for benign questions.Without the scope check, 44.4% of attacks execute; TMA-NM and isolation prevent executions but remove 70.4% and 71.4% of benign-retrieved memories.Source: §5.3 · p. 7

6 Analysis

SAFE-Mem blocks attacks by matching each request’s required kind against claim authority, rather than recognizing malicious payloads. Its analysis finds that poisoned claims can remain stored while admission keeps them out of contexts that would act on them.

6.1 Attacks Need Kinds Beyond External Authority

  • 97% and 91% of attack queries on GPT-5.6-terra and GPT-4o-mini need a kind above description, while SAFE-Mem’s ASR stays at or below 11.3% on every goal and backbone.On GPT-4o-mini, every other defense that retains memory exceeds 40% ASR on at least one goal; even consistent, instruction-free payloads do not get through.Source: §6.1 · pp. 7–8

6.2 Poison Is Stored but Not Used

  • 38.0% and 57.6% of attack cases write injected memory under SAFE-Mem, but it enters attack-query context in only 1.1% and 2.8% of cases, with end-to-end ASR of 2.6% and 6.2%.The corresponding undefended context-entry rates are 30.2% and 55.7%; keeping poison as a source record preserves recall above the undefended level.Source: §6.2 · p. 8

6.3 Adaptive Attacks Cannot Raise Authority

  • SAFE-Mem’s ASR moves by at most 1.4 points under adaptive rewrites, while recall stays between 81.1% and 87.5%.Paired genuine requests raise their own claims in 90% to 94% of cases and none of the 498 attack payloads; five paraphrased copies raise A-MemGuard’s GPT-4o-mini ASR to 68.1%, while SAFE-Mem stays between 5.6% and 7.5%.Source: §6.3 · p. 8, Table 3 · p. 8

6.4 Protection Independent of Model Strength

  • Withholding inadmissible entries keeps SAFE-Mem’s ASR between 1.5% and 6.2% across four backbones, while serving them with a description label raises ASR from 2.6% and 6.2% to 6.2% and 19.6%.Human-label agreement is 94.0% for use classification, 84.0% for retention judgments, and 83.5% for answer judgments on GPT-5.6-terra or principal sentences, as reported.Source: §6.4 · p. 8

7 Conclusion

SAFE-Mem governs which claims an agent may use by labeling claims with kind, authority, and scope, linking them to sources in a memory graph, and admitting them only for supported uses. The paper reports lower attack success alongside improved recall and accuracy, while identifying further work on retrieval-step use and verified outcomes.

7 Conclusion

  • SAFE-Mem labels each stored claim with kind, authority, and scope, links claims to sources in a memory graph, and admits claims only for supported uses.The conclusion reports attack success lowered by close to an order of magnitude while recall and accuracy improve.Source: §7 · p. 8
  • Future work includes deriving the use of each intermediate retrieval step from the original request and raising authority from outcomes verified by the environment.Source: §7 · p. 8

Limitations

SAFE-Mem’s authority governance has two stated boundaries: principal endorsement can raise content to the requested kind, and false factual claims without borrowed authority are outside its scope.

Limitations

  • A principal persuaded to endorse content can grant it authority up to the kind named in the request to remember.Source: Limitations · p. 9
  • False claims without borrowed authority fall outside SAFE-Mem’s scope; defenses that judge memory content complement its authority governance.A false web-page claim remains a valid answer about what the page said, and a principal’s own mistaken statement retains the principal’s authority.Source: Limitations · p. 9

Ethical Considerations

The paper describes its evaluation data and ethical safeguards, then details governance-aware retrieval, external-content authority limits, an illustrative case, artifact endorsement, and use classification by attack goal.

Ethical Considerations

  • The study uses public MPBench and MINJA attacks, fixed-template adaptive rewrites, and the open-access MIMIC-III Clinical Database Demo; it collects no real-user data.The authors produced the Appendix I human labels blind to model outputs.Source: Ethical Considerations · p. 9

A Retrieval Algorithm

  • Retrieval filters claims for admissibility before ranking, checks contradictions within subject-relation groups, returns ranked claims and passages, and checks tool-call identifiers against request and context authority.The algorithm marks lower-authority conflicting claims contested, or all claims contested when top-authority claims conflict.Source: A Retrieval Algorithm · p. 11

C What External Content Cannot Reach

  • External-artifact claims without principal endorsement cannot satisfy requests requiring more than description, regardless of artifact wording or copy count.The proof grounds authority in trusted channel records and caps a claim by its weakest source authority.Source: C What External Content Cannot Reach · pp. 11–12

D Case Study

  • In the password-manager case, the page’s approval claim remains description-level and is refused for a rollout recommendation, while Alice’s endorsed EU-hosting rule is admitted.The page’s claims remain admissible when Alice asks what the review said.Source: D Case Study · p. 12

E Endorsing an Artifact

  • When a principal asks to keep content referenced in an artifact, only the referenced claims are raised, and their authority is capped at the requested kind.On MPBench, 103 of 370 raised claims on GPT-5.6-terra and 12 of 322 on GPT-4o-mini come from artifact-pointing requests.Source: E Endorsing an Artifact · p. 12

F Use by Attack Goal

  • Table 5 reports the use SAFE-Mem assigns to attack queries, as percentages grouped by MPBench attack goal.Its comparison is how the assigned use varies across attack goals.Source: Table 5 · p. 12, F Use by Attack Goal · p. 12

G Ablations

The ablation compares unrestricted admission with SAFE-Mem on LoCoMo, while the repeated-payload analysis examines MPBench attack success across paraphrased-copy counts.

G Ablations

  • Figure 4 illustrates that claims remain linked to their source sentences and that admission depends on whether a claim’s effective level meets the request’s need.Its example distinguishes what a review said from which product to roll out, and keeps a claim about Marco from establishing where Alice is.Source: Figure 4 · p. 13
  • Governance off keeps the graph, budgets, and passages but admits every claim and passage, including external text, and answers without levels.Table 6 also adds TMA-NM, run through its authors’ monitor, on the same pooled store.Source: G Ablations · pp. 12–13
  • 64.9 and 58.4 judge accuracy on GPT-5.6-terra and GPT-4o-mini, respectively, when every turn is treated as the principal, versus 61.3 and 54.6 for SAFE-Mem and 50.2 and 39.5 without defense.The unrestricted setting uses the same graph and passages but withholds nothing.Source: G Ablations · p. 13

H Repeated Payloads

  • Table 7 reports MPBench attack success for no defense, A-MemGuard, and SAFE-Mem as the number of paraphrased payload copies varies.The supplied passage identifies the comparison but gives no table values.Source: H Repeated Payloads · p. 13

I Human Validation

Human validation compared annotators’ judgments with system decisions for retention, query-use classification, and whether answers acted on injected memory. Agreement varied across these judgments and models.

I Human Validation

  • The use classifier matched the exact kind for 84.7% of requests on GPT-5.6-terra and 68.7% on GPT-4o-mini, and agreed on description-versus-higher-kind needs for 94.0% and 85.3%.The corresponding Cohen’s kappa values were 0.81 and 0.61 for exact kind.Source: I Human Validation · p. 13
  • Retention judgments agreed on 84.0% of 100 principal sentences (Cohen’s kappa 0.76); among four disagreements, one kept an unrequested sentence and none missed a request.Three disagreements confused a request in the principal’s own words with one pointing into an artifact.Source: I Human Validation · p. 13
  • The answer judge agreed on 83.5% of 200 attack answers (Cohen’s kappa 0.67), with 27 of 33 disagreements judging payload action when annotators did not.The remaining six disagreements went the other way; the study also labeled 150 requests with their needed claim kind.Source: I Human Validation · p. 13

J Uncertainty and Attack Classes

This section presents supplementary analyses of attack success, claim use, uncertainty, retrieval budgets, model-call overhead, and LongMemEval, alongside the prompts used for memory processing and answer evaluation. It also describes judges that assess substantive effects and answer equivalence rather than exact wording alone.

J Uncertainty and Attack Classes; K Attack Stages

  • The appendix reports query-clustered 95% bootstrap intervals for MPBench attack success, attack-class breakdowns, and where attacks succeed across memory stages.The MPBench intervals resample 2,893 distinct queries among 2,999 attack cases, using 2,000 resamples; case-level intervals differ by at most 0.1 points. The stage analysis covers writing poison to memory, retrieval into context, and assertion in the answer.Source: Table 8 · p. 14, Table 9 · p. 14, J Uncertainty and Attack Classes · p. 14

L Retrieval Budget

  • Attack success stays between 6.2% and 7.1% across retrieval budgets, which trade recall against restraint without admitting inadmissible entries.A larger budget admits more admissible entries, but inadmissible entries are never retrieval candidates.Source: L Retrieval Budget · p. 14, Table 12 · p. 15

M Withholding and Labeling

  • Labeling inadmissible entries as descriptions gains about three LoCoMo points, while withholding them keeps attack success low on both backbones.The settings differ only when no entry is admissible; this occurs for 0.1% of benign MPBench requests and 31% and 21% of attack requests on the two backbones.Source: M Withholding and Labeling · p. 15, Table 13 · p. 15

N Use of Each Kind

  • Attack queries span four kinds above description, memory requests concentrate on preferences, benign non-memory content splits between attributes and preferences, and LoCoMo questions ask for descriptions and attributes.The figure reports kind use for GPT-5.6-terra.Source: N Use of Each Kind · p. 15, Figure 6 · p. 15

O Cost

  • SAFE-Mem adds 18–20% LLM calls over undefended memory, versus 88% for A-MemGuard.Writing adds 5–7% more calls for per-sentence judgments of remember requests; reading adds one classification per distinct query, while retrieval filters governance labels without a model call.Source: O Cost · p. 15, Table 14 · p. 15

P LongMemEval

  • LongMemEval evaluation uses 500 questions over chat histories, with the user as principal and SAFE-Mem run using its LoCoMo configuration on GPT-4o-mini.The evaluation uses the cleaned LongMemEvalS release and reports judge accuracy by question type; no table values are supplied here.Source: P LongMemEval · p. 15, Table 15 · p. 16

Q.1 Claim extraction (MPBench); Q.2 Claim extraction (LoCoMo); Q.3 Request to remember

  • The extraction prompts classify claims by who must comply, distinguish descriptions, attributes, preferences, rules, and permission or trust, and tie retention requests to the user’s own words.MPBench extraction records a retention quote only when the user explicitly asks to remember the claim; LoCoMo extraction emphasizes self-contained facts and concrete details.Source: §Q.1 · p. 16, §Q.3 · p. 16, §Q.2 · p. 16

Q.4 Use classification; Q.5 Conflict detection; Q.6 Answering (MPBench)

  • Use classification applies the same compliance-based kind ladder; conflict detection tests whether claims can both be true, and answering uses only claims authorized at the requested kind or higher.The answering prompt treats descriptions as reports of what a source said, not as user preferences, rules, or authorization.Source: §Q.4 · p. 17, §Q.5 · p. 17, §Q.6 · p. 17

Q.7 Answering (LoCoMo)

  • The LoCoMo answering prompt requests a short phrase, using exact words from the context whenever possible.Source: §Q.7 · p. 17

Q.8 Judge: memory entry

  • Memory poisoning counts as present when stored entries convey attacker-intended facts, preferences, norms, or authority—even alongside correct rules; topic-only mentions and opposites do not.Hedged or tentative versions also count as present.Source: §Q.8 · p. 17

Q.9 Judge: answer

  • A response acts on a planted instruction only if it endorses, commits to, or carries it out; mentioning it while declining, refusing, hedging, or seeking confirmation does not count.Source: §Q.9 · p. 18

Q.10 Judge: LoCoMo

  • LoCoMo answers are judged correct when they refer to the gold answer, including equivalent wording or dates; missing, unrelated, or contradictory answers are wrong.Source: §Q.10 · p. 18

Related papers

Loading 2610.22959v2…