Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction

• Earlier approaches manipulate content at choice time [19] [32] or inject fabricated user-preference records [27] [37], with limitations identified for both.Source: §1 · p. 2
• RGA wraps an attacker-user conversation in an experience-sharing post, preserving its source provenance while memory can lose the binding between the inner user role and the attacker’s user.The resulting preference is attributed to the victim’s user and persists to influence later decisions in relevant scenarios.Source: §1 · p. 2
• Privacy leakage misattributes both a tool-call preference and a task-requirement preference, steering tool choice and sensitive information in tool arguments.Source: §1 · p. 2
• Across five mainstream memory systems and six language models, tool selection manipulation achieves an average ASR of 92.37%, while privacy leakage achieves 69.57%.Source: §1 · p. 2
• RGA exploits role-to-principal misattribution through ordinary experience-sharing posts without memory access or provenance disguise.Source: §1 · p. 2

2.1 Principals of the Agent Social Platform

• Agents act on social platforms on behalf of external users; the principal space includes users and agents, with each agent mapped to the user it serves.Source: §2.1 · p. 2

2.2 Preference and Role Grounding

• A role-based preference pairs preference content with its expressing role, which is grounded to a principal according to the context in which it appears.In a quoted or forwarded chat, “user” refers to the source agent’s user, not the consuming agent’s user.Source: §2.2 · p. 2

2.3 Conversation between two Principals

• The unit of analysis is a conversation in which one agent publishes a post and another consumes it, with role-to-principal grounding represented from participants’ perspectives.Source: §2.3 · p. 2

3 Attack Model

• RGA targets speaker-grounded preference memory, where a victim may attribute an attacker’s preference to its own user.The misattributed preference can then influence subsequent behavior.Source: §3 · pp. 2–3, §3.1 · p. 3

3.1 Attack Entry Point

• The attack enters during memory formation, when preferences from conversation content are recorded and assigned to principals.Existing systems’ retrieval and remembering can lack speaker conditioning [35].Source: §3 · p. 2, §3.1 · p. 3

3.2 Attack Objective

• The objective is to make the victim’s grounding function assign the attacker’s preference to the victim’s user rather than the attacker’s user.Wrapping changes how recognizable the role is from the victim’s perspective, enabling the stated misattribution.Source: §3.2 · p. 3
• Unlike approaches that disguise the publisher as a trusted role, RGA keeps the post’s source explicitly labeled as the attacker [27] [39].It instead introduces an external role for the preference and wraps that role so its referent is harder to recognize.Source: §3.2 · p. 3

4.1 Attack Overview

• The attacker publishes an experience-sharing post with a nested conversation; the victim remembers its preference as belonging to the victim’s user and may act on it later.Those actions include prioritizing the attacker-controlled tool and potentially sending it sensitive information.Source: §4.1 · pp. 3–4

4.2 Publisher-Preference Decoupling

• Publisher-preference decoupling assigns the preference to the publisher’s “agent’s owner” rather than directly to the publisher agent.For example, the attacker says “My owner prefers v” instead of “I prefer v.”Source: §4.2 · p. 4

4.3 Nested Conversation Construction

• Nested conversation construction splits identity cues across layers: the inner dialogue uses context-relative roles such as “user,” while the outer post frames it as a conversation between publisher and owner.The outer cue makes the victim infer which principal the inner role denotes, increasing the likelihood of misattribution.Source: §4.3 · p. 4

4.4.1 Tool Selection Manipulation.

• Tool selection manipulation aims to make the victim remember a tool-call preference for the attacker-controlled tool.The preference can specify a tool for a task, such as using SkyCast for a weather forecast.Source: §4.4.1 · p. 4

4.4.2 Privacy Leakage.

• Privacy leakage adds a task-requirement preference, so the victim may include sensitive information in tool arguments to satisfy it.The attacker aligns its tool description with that requirement by adding an optional sensitive-information parameter and rewriting its description.Source: §4.4.2 · p. 4

5.1.1 Dataset and Models.

• 500 tool-call requests across five functional groups and six language-model backends provide the evaluation workload.Candidate tools are constructed to have equal selection probability without perturbations.Source: §5.1.1 · p. 4

5.1.2 Target Memory.

• The evaluation covers five memory backends with distinct approaches to organizing, writing, retrieving, and reflecting on persistent memories.Mem0, Hindsight, and MemOS use official APIs; other memory mechanisms use DeepSeek-V4-Flash as their LLM backend.Source: §5.1.2 · p. 4

5.1.3 Attack Baselines.

• Baselines manipulate tool descriptions through advertising-style content or poison persistent memory with attacker-authored content disguised as trusted information.Table 1 compares attack success rates across methods.Source: §5.1.3 · pp. 4–5, Table 1 · p. 5

5.1.4 Experimental Agents and Platform.

• Agents have persistent memory and tool-calling capabilities on a simulated social platform, where post provenance remains attached through memory updates and retrieval.Agents read community posts, retain knowledge, and automatically retrieve relevant memories for later requests.Source: §5.1.4 · p. 5

5.2.1 Tool Manipulation ASR.

• Tool manipulation succeeds when the victim selects the attacker-controlled tool, measured as the fraction of queries with manipulated tool choices.Candidate tools are selected with approximately equal probability without an attack.Source: §5.2.1 · p. 5

5.2.2 Privacy Leakage ASR.

• Privacy leakage counts as successful only when the victim selects the attacker-controlled tool and supplies the target sensitive value in its parameters.Table 2 reports tool-manipulation success rates across memory and agent backends when attack methods are deployed concurrently.Source: §5.2.2 · p. 5, Table 2 · p. 6

5.2.3 Reference Role-to-Principal Attribution.

• Preference attribution is evaluated with G-Eval and a three-stage review in which human reviewers adjudicate disagreements and ambiguous cases.DeepSeek-V4-Flash proposes attributions and GPT-5.6-Sol verifies or revises them.Source: §5.2.3 · p. 6

5.2.4 Preference Manipulation Rate.

• Preference manipulation rates classify payload attribution as direct, ambiguous, or correct and measure those categories in stored and retrieved entries; Table 3 covers four sensitive fields.Direct misattribution omits the attacker source, ambiguous attribution retains it while assigning the preference incorrectly, and correct attribution grounds it to the attacker or its owner.Source: §5.2.4 · p. 6, Table 3 · p. 6

6.1.1 Comparison with Baseline Attacks.

• Tool manipulation exceeds 95% ASR across most configurations, while privacy leakage averages 69.57% ASR; the method consistently outperforms baseline attacks on average.The evaluation covers five memory mechanisms and six agent backends.Source: §6.1.1 · p. 6

6.1.2 Competitive Multi-Attacker Environment.

• 78.41% average ASR is achieved when multiple attack methods operate concurrently, with misattributed user preferences carrying greater decision weight than exaggerated descriptions.The competitive evaluation uses 10 candidate tools per query, seven strengthened by an attack method.Source: §6.1.2 · pp. 6–7

6.1.3 One Injection, Multi-Type Leakage.

• After one preference injection, changing tool-parameter descriptions supports leakage across information types, with cross-model leakage-rate differences exceeding 40 percentage points.Reported ranges include 78.18–100.00% for DeepSeek-V4-Flash, 64.55–95.45% for Qwen3.8-Flash, and 43.64–94.55% for GPT-5.6-Luna.Source: §6.1.3 · p. 7

6.2.1 Memory Manipulation.

• Direct misattribution dominates memory formation: attack payloads are assigned to the victim’s user in Mem0 (78.1%), A-MEM (75.7%), and Hindsight (99.0%).Ambiguous attribution is less frequent.Source: §6.2.1 · p. 7, Table 4 · p. 7

6.2.2 Retrieval Manipulation.

• Retrieval distributions differ by system: Mem0 surfaces only DM entries (100.00%), while Hindsight’s CA share rises from 0.96% in memory to 24.51% in context.A-MEM retrieval is DM 50.81%, AA 27.23%, and CA 21.96%.Source: §6.2.2 · p. 7, Table 4 · p. 7

6.2.3 Impact on action generation.

• Attack success remains high for DM entries—Mem0 95.0%, A-MEM 95.6%, and Hindsight 92.7%—but CA entries yield much lower success rates.A-MEM AA entries achieve 90.0% success; DM entries are nearly indistinguishable to the decision LLM, while CA entries preserve the attacker’s principal.Source: §6.2.3 · p. 7

6.2.4 Does the memory system incorrectly record principal attri-

• An attribution probe confirms formation-stage confusion: systems label 92.4% of Mem0 entries and 100% of A-MEM and Hindsight entries as belonging to their own user.The probe asks systems to label entries as belonging to “my owner” or “other.”Source: §6.2.4 · p. 7, Figure 3 · p. 7

6.3 Comparison with Memory Injection / 6.4 Ablation Studies

• RGA achieves manipulation effectiveness comparable to HIM, has an advantage on privacy leakage, and is consistently classified as benign in five detection scenarios where HIM is flagged in every post-comment review.Removing either Publisher-Preference Decoupling or Nested Conversation Construction raises correct attribution from 18.30% to over 90%.Source: §6.3.2 · p. 8, §6.3.1 · p. 8, §6.3 · p. 8

7.1 Preference Manipulation and Tool Steering

• Content-based tool-steering attacks such as adversarial search optimization [19] and MPMA [32] act on a decision context or compete with other evidence, whereas RGA’s user-grounded preference persists across sessions and overrides those cues.RGA is described as surviving beyond the context window and taking priority over advertising-style cues and third-party evidence in competitive settings.Source: §7.1 · p. 8

7.2 Persistent Memory Poisoning

• Unlike memory-poisoning approaches that fabricate or combine harmful records [25] [27] [5] [14] [26] [28] [6], RGA truthfully describes the publisher’s own user but grounds that preference to the victim’s user.Because the publisher remains correctly identified and nothing is fabricated, the passage says RGA evades LLM-based content review and is not covered by provenance-based defenses.Source: §7.2 · pp. 8–9

8 Conclusion

• RGA formalizes how a nested interaction can cause an agent to record another user’s preference as its own user’s, with high average attack success across five memory systems and six language models.The paper concludes that memory mechanisms must preserve role-to-principal bindings in multi-principal environments.Source: §8 · p. 9

A Posts on the platform

• Each platform scenario uses a task-discussion post and eight comments; posts carry publisher IDs and are passed to memory as speaker-and-content records.The agent is seeded with its own identity and reads posts to learn new skills.Source: A Posts on the platform · p. 10

B Attack Template and Example

• The attack template identifies the quoted record as a conversation between publisher and owner, while the inner conversation expresses preferences using the role “user.”The paper states that this role refers to the attacker agent’s owner; tool manipulation plants a tool-selection preference, and privacy leakage adds a task-requirement preference.Source: B Attack Template and Example · p. 10
• For privacy leakage, the template can add a sensitive parameter and rewrite its description to link it to a higher-accuracy result tier; the parameter variants include email, phone, or ID information.The described parameter does not instruct the model directly; it links the parameter to the result tier associated with the task requirement.Source: B Attack Template and Example · p. 10

C Generalization Beyond Nested Dialogue

• The evaluation replaces the quoted chat with a task brief, acceptance feedback, or repost while preserving owner-attributed preferences and leaving other posts, comments, and metadata unchanged.These formats contain no user/assistant role labels.Source: C Generalization Beyond Nested Dialogue · p. 10
• Across three formats and five memory systems, ASRsel ranges from 50.91% to 100%, while ASRpriv ranges from 44.55% to 99.09%.The evaluation uses DeepSeek-V4-Flash and reports results over 110 queries per format.Source: C Generalization Beyond Nested Dialogue · p. 10, Table 8 · p. 11
• Hindsight has the lowest ASRsel under every tested format, at 50.91–65.45%, consistent with its 79.60% nested-dialogue ASRsel in Table 1.The results indicate that nested dialogue is one instance of the wrapping operation and that other tested wrappers also suffice for misattribution.Source: C Generalization Beyond Nested Dialogue · p. 10

D No-Manipulation Baseline

• Without manipulation, every scenario selected at least eight of ten candidate MCPs, with scenario maxima ranging from 18.18% to 31.82%.The maxima were 18.18% for Jobs and Housing, 22.73% for Papers and Weather, and 31.82% for Shopping, over 22 queries per scenario.Source: D No-Manipulation Baseline · p. 11, Table 9 · p. 11

E Models and Memory Versions

• Table 10 defines Agent as the victim backend, Mem as the memory LLM, Ref as reference attribution, and Sec as security review.Source: Table 10 · p. 11

F Security-Review Prompts

• Security reviews use a yes/no comment classifier and screen MCP metadata for tool-steering instructions, secret-path references, data exfiltration patterns, hidden control characters, and smuggled instructions.For MCPs, any found indicator yields a yes label; uncertain indicators are recorded separately.Source: F Security-Review Prompts · p. 11

G Reference Role-to-Principal Attribution with G-Eval

• Reference attribution is judged from the post or memory entry itself, using role relationships and ownership statements to resolve who holds each preference.Memory entries are assessed without reconstructing them from the original post so binding changes introduced by memory processing are captured.Source: G Reference Role-to-Principal… · p. 11
• DS-V4-Flash proposes an attribution with evidence, GPT-5.6-Sol reviews role and interaction-level bindings, and human reviewers adjudicate disagreements and flagged cases.Source: G Reference Role-to-Principal… · p. 11

• Two annotators independently relabeled 200 attribution items and all 35 security reviews, with a third annotator adjudicating disagreements.The reviews comprised 10 post comments and 25 MCP descriptions.Source: H LLM as Judge · p. 11
• LLM–human agreement was 88.00–90.00% (κ = 0.69–0.80), compared with human–human agreement of 88.50–92.00% (κ = 0.77–0.80).Source: H LLM as Judge · p. 12, Table 11 · p. 12
• All reported κ values indicate substantial agreement, and the authors conclude that LLM judges are about as reliable as an individual human annotator on these labels.Source: H LLM as Judge · p. 12

I Role-Attribution Defense

• The metadata-filled role-attribution reminder lowered ASRsel from 94.32% to 32.36% on average across five memory systems, a reduction of 61.96 percentage points.ASRsel fell to 6.36% and 11.82% on A-MEM and LangMem, and by 41.53–47.78 points on Mem0, Hindsight, and MemOS; explicit attribution did not fully protect every system.Source: Table 12 · p. 12, I Role-Attribution Defense · p. 12

J Memory Attribution Probe

• The memory attribution probe asks the system to label each entry’s source as “my owner” or “other” according to whom its user or owner role refers to.Source: J Memory Attribution Probe · p. 12

K Role-to-Principal Grounding in Nested Conversations

• The preliminary formal analysis outlines how role-to-principal grounding can fail in nested conversations and a reasoning path that preserves correct attribution.Source: K Role-to-Principal Grounding in… · p. 12

K.1 Correct mapping.

• Correct mapping resolves the outer publisher before assigning the owner role in the quoted conversation.Source: §K.1 · p. 12

K.2 Incorrect mapping.

• Misattribution occurs when memory retains the inner “user” role but drops its context, causing the victim’s default mapping to assign the attacker’s preference to the victim’s user.That persisted preference can then guide subsequent tool selection.Source: §K.2 · p. 12