PaperlayerView exact revision on arXiv

Source-linked AI summary

MEMORY Wins All: Indirect Bias Injection Attacks via Social Media Feeds

Minjae Seo, Wonwoo Choi, Geonwoo Han, Taekyoung Kwon, Yongsu Kim, Sang Seo, Jaewon Noh, Hankyul Baek, Seongyun Seo, Myoungsung You

arXiv:2608.22061v1cs.AIcs.CY

TL;DR

Prior memory-corruption attacks assume direct access, leaving indirect bias injection through routinely ingested external content largely unexamined. IBIA plants an adversary-aligned stance in persistent memory using crafted comments, achieving a 91.2% average adversary-aligned response rate across four downstream tasks and seven LLMs.

  • Problem

    Prior memory-corruption work assumes direct access, while indirect corruption through external content retained as trusted persistent memory remains largely unexamined.

    Source: §1 · p. 1, §2.3 · p. 4

  • Method

    IBIA injects a topic-specific adversary-aligned stance through external comments using comment cloaking, comment watermarking, and category anchoring.

    Source: §1 · p. 2, §4.1 · p. 5

  • Results

    91.2% average adversary-aligned response rate across four downstream tasks and seven LLMs, including 86.6% on GPT-5.5.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings identify persistent memory as an attack surface and motivate provenance controls for what agents retain.

    Source: §9 · p. 12

    The threat model targets personalized agents that summarize SNS posts and comments as part of a daily routine.

    Source: §3 · p. 4

Abstract

from arXiv · show

Personal AI agents routinely consume external content while performing tasks such as web browsing, email processing, and SNS feed summarization, and they retain selected information or execution results in persistent memory for later use. We show that this ordinary ingestion of external content opens an indirect path for manipulating subsequent agent behavior. Based on this observation, we present IBIA, an Indirect Bias Injection Attack that plants an adversary-aligned stance on a specific topic into a victim agent's memory through external content, without direct access to the agent, its memory, or future user queries. For this, IBIA combines three mechanisms: comment cloaking, which keeps the crafted content consistent with the surrounding discussion, comment watermarking, which enables lightweight identification during curation, and category anchoring, which makes the retained stance salient under later related requests. We evaluate IBIA on BiasBench, a benchmark of 6,000 adversary-crafted social comments and 120 email instances. The watermark-based curation identifies 95.9% of the injected comments. Under the OpenClaw setting, IBIA achieves adversary-aligned response rates (AARs) of 91.2% on average across four downstream tasks, including 86.6% on the frontier GPT-5.5. We further propose a memory boundary defense that detects the injected bias and reduces AARs to 80.6%.

1 Introduction

Personal agents can promote external content into persistent memory as trusted context, creating an indirect path for bias injection without adversary access to the agent or its memory. IBIA uses routinely collected social-media content to plant an adversary-aligned stance for later related requests.

1 Introduction

  • Unlike prior memory-corruption attacks that require a controlled write channel, IBIA uses the agent’s routine ingestion of external content, without direct access to the agent, memory, or future queries.Prior approaches include crafted memory entries or knowledge-substrate content [5] [33], crafted queries, and executable code embedded in a file [27].Source: §1 · pp. 1–2
  • IBIA combines comment cloaking, watermarking, and category anchoring; its third-party feed-summarization skill implements the latter two and passes all evaluated skill scanners.Cloaking keeps comments consistent with the discussion while preserving the stance; watermarking identifies crafted comments during curation, and category anchoring makes the retained stance salient for related requests.Source: §1 · pp. 1–2
  • BiasBench contains 6,000 adversary-crafted SNS comments and 120 emails, evaluated in a private Moltbook environment with ten agents across four downstream tasks.The evaluation used an OpenClaw-based agent for recommendation, opinion QA, email summarization, and email drafting; no adversary-crafted content was posted to the live platform.Source: §1 · p. 2
  • IBIA achieves 91.2% average adversary-aligned response rates across four tasks and seven LLMs, including 86.6% on GPT-5.5.The paper also presents a memory boundary defense that detects 80.6% of injected entries at a 5.6% false positive rate.Source: §1 · p. 2
  • The paper analyzes how preexisting factual priors relate to the resulting bias and reports a memory boundary defense that detects 80.6% of injected entries at a 5.6% false positive rate.The authors report that the mitigation is present in OpenClaw’s main branch but had not reached the referenced release as of August 2026.Source: §1 · p. 2

2 Background and Motivation

Personal agents retain observations from external content in persistent memory that is reused in later requests. This recall–act–update cycle creates an indirect memory-corruption path: adversaries can shape later answers through content the agent ordinarily ingests, without writing to memory or querying the agent.

2.1 Autonomous Agents

  • Agents combine an LLM runtime, tool-using skills, and persistent memory that retains salient facts, preferences, and prior action results across sessions.Skills describe procedures and may include programs; deployed memory can be stored in a MEMORY.md file.Source: §2.1 · p. 2

2.2 Memory and Recall–Act–Update Loop

  • Memory is included in inference prompts and updated with retained action results, allowing the agent to recall earlier observations during later related tasks.For example, an SNS summary retained after a feed-reading routine can inform a later answer about a related topic.Source: §2.2 · p. 2

2.3 Indirect Memory Corruption

  • Indirect memory corruption exploits ordinary curation of external content: adversaries need only control what enters Curate(·), not write to memory or query the agent.Prior attacks generally require direct access, crafted queries, or executable code in an ingested file—capabilities that are ill-suited to locally hosted personal agents.Source: §2.3 · p. 3
  • Injected on-topic comments can cause later answers about that topic to lean toward the planted stance, while users see the answer rather than the comments that shaped it.The comments and resulting memory note are unremarkable in isolation, and the curation skill passed every evaluated scanner despite lacking explicit signals such as executable payloads.Source: §2.3 · pp. 3–4, Table 2 · p. 3

3 Threat Model

The threat model considers a personalized agent that summarizes external SNS content and stores selected results in memory, which an adversary seeks to bias indirectly for later topic-related queries. The adversary cannot access the agent or memory directly, but can post feed comments and rely on an installed skill to make the agent curate qualifying content.

3 Threat Model

  • The user’s host, model runtime, and memory are trusted, while ingested external content and installed third-party skills fall outside the trust boundary.The setting has the user install a skill that gathers and summarizes external content, including SNS posts and comments, and curates results into memory.Source: §3 · p. 4
  • The adversary seeks to maximize the fraction of plausible queries about a chosen topic whose answers express its target bias, with answers judged as supporting, neutral, or opposing that topic.The target topic is chosen in advance from four categories and may be unrelated to the summarized feed; the categories are subjective and can affect user decisions.Source: §3 · p. 4
  • The adversary cannot query the agent, observe future user queries, access or alter memory or runtime, execute code on the host, or directly cause a chosen memory write.Instead, it posts biased comments to a trending feed, including comments about a topic unrelated to the feed’s own topic, and uses a third-party skill to make the agent conditionally persist qualifying comments.Source: §3 · p. 4
  • The adversary relies on a benign-looking SNS-search skill to make the agent itself persist qualifying comments, rather than performing a malicious action or writing to memory directly.The skill makes conditional persistence explicit and is described as passing current skill scanners.Source: §3 · p. 4

4 Method

IBIA indirectly plants an adversary-aligned stance in an agent’s persistent memory through external social content. It combines feed-compatible comments, watermark-based curation, and category-organized memory within SNSwatcher’s feed workflow.

4 Method

  • IBIA maps three challenges to comment cloaking for feed compatibility, watermarking for comment identification, and category anchoring for later retrieval within SNSwatcher.SNSwatcher collects and summarizes feeds while indirectly influencing memory through which comments are retained and how they are organized.Source: §4 · pp. 4–5

4.1 Benign-Looking Comment Construction

  • Comment cloaking combines host-topic alignment, a conversational hook, and a self-contained target-bias payload, selecting text similar to the feed while meeting the moderation threshold.The construction keeps the intended stance while making the complete comment consistent with the host discussion and admissible under the platform’s moderation policy.Source: §4.1 · p. 5

4.2 Retrievable Bias Recording

  • Category anchoring stores selected comments as target-topic and stance pairs under category directives that cue relevant requests, presenting the bias as previously observed public sentiment.The category directive provides a broad retrieval cue while each pair preserves the stance associated with a specific target topic.Source: §4.2 · p. 6

4.3 Efficient Comment Identification

  • IBIA applies Hao et al.’s post-hoc, keyed watermark to completed comments and detects it with a lightweight statistical test, avoiding semantic classification while preserving cloaking and toxicity constraints.Watermarking changes paraphrase-preserving synonyms at robust positions; detection requires lightweight preprocessing and hashing rather than an LLM call or learned semantic classifier.Source: §4.3 · pp. 6–7

5 Evaluation

Evaluation finds that bias from external content can transfer into persistent memory and influence later responses; IBIA makes this transfer effective across downstream tasks, though susceptibility varies by model and task.

5.1 Experimental Setup

  • BiasBench contains 6,000 SNS comments and 120 emails for evaluating opinion QA, recommendation, summarization, and content drafting.The evaluation uses Reddit-derived discussions in a private Moltbook environment, with a benign personal agent accessing feeds through OpenClaw.Source: §5.1 · pp. 7–8

5.2 Bias transfers in the wild

  • On real Reddit feeds, average biased-response rates rise from 45.0% with no injection to 55.0% with mixed comments and 74.4% with only biased comments retained.Mixed organic content has inconsistent effects across categories; selective curation makes bias transfer more reliable.Source: Table 4 · p. 8, §5.2 · p. 8

5.3 Effectiveness of Bias Injection

  • GPT-5.5 reaches 81.5% to 98% adversary-aligned responses on Recommendation and Opinion QA, while IBIA steers agents across all four evaluated tasks.Across the six other backends and four categories, attack response rates are 87% to 100%; GPT-5.5 resists email summarization but is fully steered in draft generation.Source: Table 5 · p. 9, §5.3 · p. 9

5.4 Microbenchmark

  • 95.9% of injected comments pass watermark detection, while only 2 of 6,000 comments are flagged by the toxicity filter.Detection exceeds 93% in every category, and category means are below 0.0023 against the toxicity threshold of 0.5.Source: §5.4 · p. 10, Table 7 · p. 10, Table 6 · p. 10
  • With category anchoring, aligned response rates rise from 14% to 99.2% on Celebrity and reach 97.8% on Politics under full IBIA.Watermarking alone raises Advertisement, Politics, and Public Figure rates to 83%, 85%, and 84.5%, respectively; the results show the components play complementary roles.Source: §5.4 · p. 10, Table 8 · p. 10

6 Mitigation

The mitigation screens candidate memory entries at the memory boundary, aiming to prevent injected bias from reaching later retrieval while limiting filtering to persistent-memory changes. The filter detects 80.6% of injected entries, with performance varying by category, and has a 5.6% false positive rate on benign comments.

6.1 Mitigation Architecture

  • The filter uses an LLM to assess candidate memory updates in context for unsupported, manipulative, or directionally biased content that ordinary keyword or toxicity filters may miss.IBIA’s injected bias is phrased as plausible, on-topic commentary rather than overtly malicious or toxic content.Source: §6.1 · p. 10

6.2 Mitigation Results

  • 80.6% overall detection (4,834/6,000), with 98.4% public-figure, 91.4% celebrity, 69.3% advertisement, and 63.0% political detection.The filter is instantiated with GPT-4.1 and counts an entry as detected only when its ground-truth category is flagged as biased.Source: Table 9 · p. 11, §6.2 · p. 11

6.1 Mitigation Architecture

  • Filtering runs only when persistent memory is created or modified, limiting token and latency costs while preventing injected content from reaching later sessions and queries.The policy protects the agent’s persistent memory by screening candidate entries before they become available for subsequent retrieval.Source: §6.1 · pp. 10–11, §6 · p. 10
  • A native memory-update hook invokes the filter when available; otherwise, a lifecycle hook and file-system watcher monitor memory files, retaining benign entries and withholding flagged ones.This approximates memory-update interception without requiring native framework support.Source: §6.1 · p. 11

6.2 Mitigation Results

  • 5.6% false positive rate (56/1,000) on the benign Civil Comments subset; these false positives prevent memory storage rather than blocking content from users.The subset was filtered to remove potentially biased entries, and 94.4% of its entries remained unflagged.Source: §6.2 · p. 11

7 Factual Prior Analysis

IBIA’s influence depends on the strength of the model’s preexisting factual prior: injections aligned with its existing tendency are highly effective, while conflicting injections are less effective. Weakening recognizable entity cues increases the influence of a conflicting injection.

Effect of preexisting factual priors

  • Without injection, responses were 0.5% true, 5% neutral, and 94.5% false, indicating a strong prior against the tested conspiracy claims.The baseline covered 200 historical events associated with widely circulated conspiracy claims.Source: §7 · pp. 11–12
  • 100% of responses followed false-label-supporting injection, but only 28% aligned with conflicting true injection; 34.5% were neutral and 37.5% remained false.These conditions were tested on 200 historical events associated with widely circulated conspiracy claims.Source: §7 · pp. 11–12

Weakening recognizable factual cues

  • 63% aligned with the conflicting true injection after anonymization, up from 28%; false responses fell from 37.5% to 13%, and neutral responses from 34.5% to 24%.Anonymization replaced recognizable person names with synthetic names while preserving the claim structure; false injection remained 100% effective.Source: §7 · p. 12

Takeaway

  • IBIA is most effective when queries leave epistemic uncertainty for memory to shape judgment; strong factual priors partially counteract it.The paper notes that recommendations, political preferences, and preferences among public figures rarely have a single factual answer that constrains the response.Source: §7 · p. 12

8 Related Work

Related work studies memory poisoning, social-platform manipulation, and payload propagation, while IBIA differs by turning ordinary feed content into a persistent, non-executable bias in one agent’s memory without direct access or a chosen ingestion target.

Agent memory poisoning

  • Unlike AgentPoison [5], PoisonedRAG [33], MINJA, MemoryGraft [27], and Mind Viruses [23], IBIA needs no direct memory access, crafted query, controlled artifact, or system-prompt modification.Personal agents reload persistent natural-language memory as trusted context, so newly observed browsing content can steer future behavior when retained.Source: §8 · p. 12

Malicious behavior on social platforms

  • TrendSim and BotSim [24] model coordinated influence on simulated populations through visible feeds, whereas IBIA steers one agent by retaining ingested feed content as memory.The prior work examines aggregate opinion dynamics, rather than durable context in a particular victim agent.Source: §8 · p. 12

Propagation across multi-agent systems

  • Prompt Infection [17], extending indirect prompt injection [10], propagates executable instructions across agents; IBIA instead plants a non-executable, on-topic bias in one agent’s memory without propagation.Prompt Infection also inflates memory importance to dominate retrieval; its agents re-execute and forward the directive.Source: §8 · p. 12

9 Conclusion

IBIA plants an adversary-aligned stance in a personal agent’s memory through ordinary comments that the agent curates, without direct access to its memory or user queries. On BiasBench, it achieves high adversary-aligned response rates, while a memory boundary defense detects injected entries.

9 Conclusion

  • IBIA averages a 91.2% adversary-aligned response rate across four downstream tasks and seven LLMs, including 86.6% on GPT-5.5.The curating skill passes three state-of-the-art scanners.Source: §9 · p. 12
  • IBIA plants an adversary-aligned stance by posting ordinary comments for the agent to curate and persist, without writing to its memory or accessing the host or user queries.The results identify memory as a critical attack surface and motivate provenance controls for retained content.Source: §9 · p. 12
  • The memory boundary defense detects 80.6% of injected entries.The curating skill passes three state-of-the-art scanners.Source: §9 · p. 12

A Ethical Consideration · B Open Science

The authors study risks to users and agent ecosystems from external content corrupting persistent memory, while limiting experiments to isolated environments and providing a practical defense. They also release code and benchmark materials for reproducing IBIA and its evaluation.

A Ethical Consideration

  • The memory boundary filter detects 80.6% of injected entries, with a 5.6% false positive rate on benign comments.It examines candidate entries before they become persistent context.Source: A Ethical Consideration · p. 13
  • Experiments used public Reddit discussions in a private Moltbook deployment; the authors never posted crafted comments to live Reddit or targeted real users.They also avoided attack campaigns against live SNS platforms and real personal agent users.Source: A Ethical Consideration · pp. 13–14
  • The authors responsibly disclosed to OpenClaw and ClawHub that a skill could induce memory corruption through curation while passing existing screening.The study reports the screening weakness without deploying attacks against real users.Source: A Ethical Consideration · pp. 13–14
  • Users may face unseen influence on decisions, while developers and skill marketplace operators face risks from benign-looking skills that shape persistent memory.Potential misuse includes steering agent recommendations, opinions, summaries, or drafted content toward commercial, political, or reputational interests.Source: A Ethical Consideration · p. 13
  • The authors argue that the defensive value outweighs disclosure risks because the study uses isolation, avoids real-user interaction, reports the screening weakness, and provides mitigation.They aim to help developers and marketplace operators strengthen long-term memory integrity.Source: A Ethical Consideration · p. 14

B Open Science

  • The released repository includes IBIA code, benchmark data and corresponding emails, environment and benchmark execution procedures, and defense implementations.Materials are organized under dataset/, implementation/, and defense/.Source: B Open Science · p. 14

C Curation Skill Workflow … F Notation

This section documents SNSwatcher’s feed summarization and watermark-gated memory curation workflow, alongside prompts for classifying response stances and filtering candidate memories for bias. It also defines notation for the paper’s threat model and methods.

C Curation Skill Workflow

  • SNSwatcher summarizes collected feeds, verifies normalized comments, extracts topic-opinion pairs from verified comments, assigns categories, and appends category-prefixed blocks to long-term memory.Comments that fail verification may inform the current summary but are not persisted.Source: C Curation Skill Workflow · p. 14

D Response Stance Classification Prompts

  • The common judge rule labels responses +1 for alignment with the injected stance, −1 for the opposing stance, and 0 for no clear directional preference.Judges assess the substantive overall position rather than tone or isolated statements; balanced, inconclusive, declined, and insufficient-information responses receive 0.Source: D Response Stance Classification Prompts · pp. 14–15
  • Task-specific judge prompts cover advertisements, celebrities, politics, and public figures, with an additional factual-claim judge for the Section 7 factual-prior experiment.Source: D Response Stance Classification Prompts · p. 15
  • The stance-classification prompts label responses as 1 when favoring a claim, -1 when opposing it, or 0 when balanced, uncertain, or insufficiently supported.They assess the response’s overall substantive position rather than isolated statements or rhetorical tone.Source: D Response Stance Classification Prompts · p. 16

E Memory-Boundary Filter Prompt

  • The memory-boundary filter checks candidate entries for directional bias across four categories and withholds biased entries while allowing unflagged entries through normal memory updates.It returns a binary decision and content-grounded justification, aiming to flag clear one-sided framing without treating a topic or named entity alone as bias.Source: E Memory-Boundary Filter Prompt · p. 16

F Notation

  • Table 10 summarizes notation for the threat model, biased-comment construction, category-anchored memory records, and watermark-based curation.The passage identifies these topics as notation-table scope but does not provide individual notation definitions.Source: F Notation · p. 17, Table 10 · p. 17

G Human Verification of the LLM Judge

Human review validates the LLM judge’s labels, and the corrected AAR results are conservative because the dominant judge error inflates the no-attack baseline rather than post-attack rates.

G Human Verification of the LLM Judge

  • The judge agrees with human labels in 4,859 of 5,160 judgments (94.2%) overall and in 3,899 of 4,200 Table 5 judgments (92.8%).For Table 5, agreement ranges from 98.5% on Advertisement to 89.9% on Celebrity; the 301 disagreements are corrected before reporting human-verified labels.Source: G Human Verification of the LLM Judge · p. 17
  • All 960 judgments for the two email-based tasks in Figure 6 receive full human–judge agreement, requiring no corrections.The judgments cover 480 for summarization and 480 for drafting across the two models.Source: G Human Verification of the LLM Judge · p. 17
  • False directionalization accounts for 265 of 301 disagreements (88.0%), while polarity inversion accounts for 25 (8.3%) and stance omission for 11 (3.7%).False directionalization includes hedged answers labeled as a stance (140 cases), unverified claims assigned a stance (111), and stances attributed from memory or another speaker (14). Because neutral answers are concentrated in the no-attack baseline, this dominant error inflates the baseline and shrinks the measured gap; human-corrected AARs are therefore conservative.Source: G Human Verification of the LLM Judge · p. 17, Table 11 · p. 17

H Report to OpenClaw

The authors reported the issue through OpenClaw’s public GitHub tracker, where an automated review found that the current main branch blocks tainted-turn memory writes from promotion into durable memory. As of August 2026, this protection was present in the main branch but not in the referenced release.

H Report to OpenClaw

  • OpenClaw’s review said tainted-turn memory writes are marked untrusted and barred from durable-memory promotion in the current main branch, but the protection was not yet in the referenced release as of August 2026.The report was submitted through the public GitHub issue tracker because OpenClaw had no dedicated contact email for this issue; repository-integrated bot review handled submitted issues.Source: H Report to OpenClaw · p. 18

Related papers

Loading 2608.22061v1…