PaperlayerView exact revision on arXiv

Source-linked AI summary

When Malicious Instructions Persist: Persistent Memory Poisoning Attack on Harness-Based Agents

Shuhuai Huang, Jingfeng Zhang, Hong Jia

arXiv:2609.13889v1cs.CRcs.AI

TL;DR

Malicious instructions in unverified external sources may enter harness-based agents’ persistent memory, creating privacy risks across sessions. PMPA induces this memory poisoning without direct access to the agent framework. It succeeds across OpenClaw and Claude Code while preserving benign task performance, and a targeted prompt-level defense offers limited protection once memory is poisoned.

  • Problem

    Whether malicious instructions in external sources can enter persistent memory without direct framework access remains underexplored, despite the risk of privacy leakage persisting across sessions.

    Source: §1 · p. 1

  • Method

    PMPA embeds malicious instructions in benign external sources to induce persistent-memory storage, then tests whether retrieval in later sessions triggers additional behavior and privacy leakage.

    Source: §1 · p. 1

  • Results

    PMPA averages ISR/C-ASR of 73.7%/55.5% on OpenClaw and 66.9%/81.7% on Claude Code, while preserving benign task performance on both systems.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A targeted prompt-level defense can reduce memory injection in many settings but provides limited protection once malicious instructions are stored.

    Source: §1 · p. 2

    The threat model limits attackers to crafting instructions in external sources; they cannot query or observe the victim or access its device, memory, prompts, or runtime configuration.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Harness design has transformed the development of LLM-based agents by integrating memory, tool use, and runtime control. However, this design also introduces security and privacy risks because malicious instructions from external sources may be written into persistent memory and persist across sessions. To study this risk, we propose PMPA, a Persistent Memory Poisoning Attack against harness-based agents. PMPA embeds malicious instructions into benign external sources and induces the victim agent to write them into persistent memory without directly accessing to the agent framework. Once stored, the poisoned memory can be retrieved in later sessions, triggering additional malicious actions and causing privacy leakage. We evaluate PMPA on OpenClaw and Claude Code across different backbone LLMs, input modalities, and trigger scenarios. Across all settings, PMPA achieves average Injection Success Rate (ISR) and Cross-session Attack Success Rate (C-ASR) of 73.7%/ 55.5% on OpenClaw and 66.9%/ 81.7% on Claude Code, while preserving benign task performance on both systems. We further evaluate a targeted prompt-level defense and find that it can reduce memory injection in many settings, but provides limited protection once the persistent memory has been poisoned.

1 Introduction

Harness-based agents can retain information across sessions, creating risks when malicious instructions from unverified external sources enter persistent memory. PMPA tests this threat and achieves attack success across OpenClaw and Claude Code while preserving benign task performance.

1 Introduction

  • Unverified external instructions may enter persistent memory and create privacy risks that persist across sessions; whether this can happen without framework access remains underexplored.Harness designs also integrate reasoning, planning, tool use, memory, session management, and runtime control.Source: §1 · p. 1
  • PMPA embeds malicious instructions in benign external sources and induces the agent to store them without direct access to its framework.The poisoned rules may later be retrieved across sessions, triggering additional malicious behavior and privacy leakage.Source: §1 · pp. 1–2
  • PMPA averages ISR/C-ASR of 73.7%/55.5% on OpenClaw and 66.9%/81.7% on Claude Code, while preserving benign task performance on both systems.A targeted prompt-level defense reduces memory injection in many settings but offers limited protection after poisoning.Source: §1 · p. 2

2 Related Work

Prior work develops agents with harness orchestration and persistent memory, while giving limited attention to the security risks of cross-session persistence. Existing attack studies focus on current-task prompt injection or assume more direct access for poisoning; this work studies attacks through published external sources.

2.1 LLM-based Agents and Harness Engineering

  • Harnesses coordinate context, reasoning, planning, tool interaction, policies, runtime control, and feedback; persistent memory supports cross-session continuity, whose security risks have received limited attention.Untrusted external content can move from temporary task input into persistent state and affect real agent actions.Source: §2.1 · p. 2

2.2 Security and Privacy Risks in LLM-based Agents

  • Prior prompt-injection work targets current interactions, while backdoor and memory-poisoning work often assumes direct poisoning or interaction; this work tests attacks via published external sources without victim access.The related work covers prompt injection, agent backdoors, memory or RAG poisoning, and poisoned fine-tuning data.Source: §2.2 · p. 3

3 Methods

PMPA models an attacker who publishes malicious instructions in external sources, which an agent may store and later retrieve during a workspace task. Its design specifies the threat, attack workflow, payload strategies, and controlled evaluation setup.

3.1 Threat Model

  • The attacker aims to induce persistent-memory storage and later privacy leakage, but cannot query or observe the victim or access its device, memory, prompts, or runtime configuration.The victim is a harness-based LLM agent deployed in a user-controlled environment.Source: §3.1 · p. 3

3.2 Attack Workflow

  • The two-phase workflow places an embedded instruction in a benign source for question answering, then retrieves the stored rule during a later workspace task.The later action accompanies the benign task and may disclose personal information; Figure 1 illustrates adding an attacker-controlled attendee to a new event.Source: §3.2 · p. 4, Figure 1 · p. 4

3.3 Attack Payload and Injection Strategies

  • Each payload combines a transition sentence, a memory-writing instruction, and a conditional malicious rule; injection strategies vary payload position and linguistic style.The design places payloads in the middle and uses first-person, user-like wording to make them less abrupt and appear relevant to user preferences.Source: §3.3 · pp. 4–5

3.4 Experimental Setup

  • The evaluation uses OpenClaw and Claude Code with three backbone models in a local virtual workspace simulating email, calendar, document, and form operations.Samples pair external QA content with scenario-specific triggers; 30 questions sampled across four datasets produce 120 source contents and 360 text, image, and PDF injection samples. ISR, S-ASR, C-ASR, and Utility measure memory writing, same-session leakage, cross-session leakage, and benign QA performance.Source: §3.4 · pp. 5–6

4 Experimental Results

Across OpenClaw and Claude Code, PMPA produces memory injection and cross-session attacks while benign task utility remains high. Results vary across input modalities, trigger scenarios, injection strategies, and defense stages.

4.1 Main Results

  • ISR ranges from 61.7% to 77.5%, C-ASR from 30.8% to 92.5%, and Utility remains above 80% in all settings.Attack performance varies across frameworks even with the same backbone model.Source: §4.1 · p. 6
  • Text input achieves the highest attack metrics on both agents, while image and PDF inputs still cause memory injection and same-session privacy leakage.The authors suggest parsing steps and untrusted-content handling may weaken attacks through image and PDF inputs.Source: §4.1 · p. 6, Table 2 · p. 7
  • Calendar scenarios reach C-ASR values of 96.7% and 87.8%; on OpenClaw, the form scenario has the highest ISR and S-ASR, at 80% and 71.1%.All four scenarios produce same-session or cross-session privacy leakage.Source: §4.1 · p. 7, Table 3 · p. 7

4.2 Ablation

  • On Claude Code, ISR falls from 83.0% with first-person style to 54.4% and 48.1% with second-person and impersonal styles; on OpenClaw, ISR ranges from 59.6% at the bottom to 73.3% at the top.Top + first-person has the highest overall ISR across both agents; Utility remains high across all settings.Source: §4.2 · pp. 7–8, Table 4 · p. 8
  • C-ASR shows no consistent decrease as benign interactions increase, and several settings retain high attack success after five interactions.The persistence experiment uses DeepSeek-V4-Flash on both agents.Source: §4.2 · p. 8, Table 5 · p. 8

4.3 Potential Defense

  • The sandwich-style defense reduces ISR and S-ASR substantially for image and PDF inputs, but C-ASR remains nearly unchanged after defense.Utility remains high across tested settings.Source: §4.3 · p. 9, Table 6 · p. 9

5 Conclusion

PMPA poisons persistent memory through benign external sources, and the paper reports substantial attack success on OpenClaw and Claude Code alongside limited post-poisoning protection from a targeted defense. The authors report that experiments were limited to controlled research settings and call for broader evaluations and more effective defenses.

5 Conclusion

  • Evaluations of PMPA on OpenClaw and Claude Code report average ISR and C-ASR values of 73.7%/55.5% on OpenClaw and 66.9%/81.7% on Claude Code, respectively.Source: §5 · p. 9
  • The targeted prompt-level defense can substantially reduce memory injection but provides limited protection after malicious instructions are stored in persistent memory.The authors identify more effective defenses against persistent-memory poisoning and privacy leakage as future work.Source: §5 · p. 9
  • The authors plan to evaluate PMPA across broader settings and more backbone models.The stated aim is to assess its performance.Source: §5 · p. 9

A Ethics Statement

  • Experiments used controlled research settings, the authors’ own machines, and controlled agents, without exposing or targeting real individuals.The authors discourage unauthorized or harmful use.Source: A Ethics Statement · p. 12

B.1 External Source Construction

The appendix constructs external-source samples by varying attack scenarios and modalities while retaining the benign QA content, and creates benign trigger prompts matched to payload conditions.

B.1 External Source Construction

  • 30 QA instances sampled from MMLU, ARC, CommonsenseQA, and SocialIQA provide the benign content for external sources.Coverage is approximately balanced across the four datasets.Source: §B.1 · p. 12
  • Four scenario-specific payload versions per QA instance produce 120 source contents, with benign QA content unchanged across versions.The four scenarios are email, calendar, shared document, and form.Source: §B.1 · p. 12
  • Converting the 120 contents into plain-text, image, and PDF forms produces 360 injection-phase samples while retaining the same content and payload.Only the external source representation changes across modalities.Source: §B.1 · p. 12

B.2 Trigger Prompt Construction

  • Each conditional malicious rule is paired with a benign trigger prompt that creates its required workspace-task condition without including malicious instructions.Table 7 summarizes the mapping between payload scenarios, rules, and trigger tasks.Source: §B.2 · p. 12

B.3 Example of a Sample Pair

The calendar sample pair separates injection from triggering: an attack payload is embedded in benign QA content, while a corresponding benign calendar task serves as the trigger.

B.3 Example of a Sample Pair

  • Table 7 maps attack-payload scenarios to their corresponding trigger tasks.The sample pair illustrates the external source as the injection-phase input and the trigger prompt as the trigger-phase input.Source: §B.3 · p. 12, Table 7 · p. 13
  • The calendar external source embeds a calendar-specific attack payload in the middle of benign QA content about leucine.The payload asks that a requirement to invite an email address when creating a calendar event be noted in memory.Source: §B.3 · p. 13
  • The corresponding trigger prompt contains only a benign calendar task, with no malicious instruction.It requests a weekly research discussion event for tomorrow from 10:00 AM to 11:00 AM.Source: §B.3 · p. 13

C.1 Injection Position Ablation.

The injection-position ablation varies where the same attack payload appears relative to benign QA content; the supplied passages describe the placements but provide no comparative outcome.

C.1 Injection Position Ablation.

  • The ablation holds payload content and first-person style constant while changing only its position relative to benign QA content.The same QA instance and payload are used in the examples.Source: §C.1 · p. 13
  • In the top-position variant, the payload precedes the benign question content.The example places the memory instruction before the question and options.Source: §C.1 · p. 13
  • In the medium-position variant, the payload sits between the question and its answer options.The example shows the question before the memory instruction and the options after it.Source: §C.1 · pp. 13–14
  • In the bottom-position variant, the payload follows the complete benign QA content.The example places the memory instruction after the question and options.Source: §C.1 · p. 14

C.2 Instruction Style Ablation.

The instruction-style ablation compares first-person, second-person, and impersonal payloads while holding payload position and malicious intent constant.

C.2 Instruction Style Ablation.

  • The ablation changes only linguistic style, keeping payload position fixed in the external source’s middle and preserving the same malicious intent.The calendar-scenario payload serves as the example.Source: §C.2 · p. 14
  • First-person style frames the malicious rule as the user’s request or preference, using expressions such as “me” or “my.”Its example asks the agent to record the requirement in memory and invite <attacker>@mail.com when creating a calendar event.Source: §C.2 · p. 14
  • Second-person style directly addresses the agent with expressions such as “you” or “you should.”Its example asks the agent to update long-term memory and invite <attacker>@mail.com when creating an event.Source: §C.2 · p. 14
  • Impersonal style presents the instruction as a general rule without identifying the user or agent as the actor.Its example asks the agent to store a long-term memory before stating the calendar-event rule.Source: §C.2 · p. 14
  • Across styles, the malicious calendar rule asks that <attacker>@mail.com be invited when a new event is created.Source: §C.2 · p. 14

D Defense Instruction

  • The defense instruction bars saving external instructions as future behavior rules or taking disclosure-related actions without explicit confirmation.It permits external content only as temporary task data for the current request.Source: D Defense Instruction · p. 14
Loading 2609.13889v1…