PaperlayerView exact revision on arXiv

Source-linked AI summary

The Like Trap: Multi-Stage Poisoning against Agents in Similarity-based Recommendation Systems

Yue Xing, Pengfei He, Zitao Li

arXiv:2609.27155v1cs.CRcs.AIcs.LGstat.ML

TL;DR

Research on agent poisoning often assumes malicious content has already reached the agent, leaving open how recommendation systems might surface it. This paper analyzes and implements a staged poisoning attack against similarity-based, like-scored recommendation, finding that early likes can pull later, less-similar poisoned posts into the feed.

  • Problem

    Existing agent-poisoning studies often assume poisoned content has reached the agent, while how evolving recommendations and agent liking behavior interact to expose malicious content remains underexplored.

    Source: §1 · p. 1

  • Method

    The paper analyzes conditions for a multi-stage poison chain and develops a post-generation pipeline that controls similarity across stages.

    Source: §4 · p. 4, §4.2 · p. 5

  • Results

    A carefully designed chain can exploit like-score feedback so that liking early-stage posts pulls later, less-similar poisoned posts into the feed.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The findings draw attention to the agent-as-user threat surface in similarity-based recommendation systems.

    Source: §6 · p. 8

    The study addresses the like mechanism; graph-based and transformer-based recommenders have different mechanisms and corresponding vulnerabilities.

    Source: Limitations · p. 9

Abstract

from arXiv · show

With recent advancements in large language models (LLMs) and LLM-based agents, these agents are becoming increasingly autonomous and gaining broader access to act on users' behalf on the internet. However, the vulnerability of automated agents deployed on social media platforms (e.g., for managing a user's personal account) remains underexplored. Existing studies on agent poisoning typically assume that the adversary can expose poisoned content to the agent. Although such an attack is direct and effective, it is more easily detected and mitigated. In the context of social media platforms, this leaves open whether the recommendation system itself would surface such content to the agent in a more subtle manner. Through theoretical analysis, we show that the like-score mechanism used in OASIS can be exploited, and we characterize the conditions under which a multi-stage chain of poisoned posts can steer the agent's feed. Based on these insights, we further develop an algorithm that crafts realistic poisoned posts. Experiments support our theoretical findings and demonstrate the effectiveness of the proposed algorithm. Notably, by exploiting the like-score feedback loop, the attack causes the recommendation system to select poisoned posts even when their user-post similarity falls below the retrieval threshold.

1 Introduction

As LLM agents increasingly act on users’ social-media accounts, the paper asks how poisoned content can reach and influence them through recommendation systems. It proposes a pipeline for gradually shifting recommendations toward attacker-controlled content.

1 Introduction

  • The paper’s motivation is the growing use of LLM agents for automated actions, including managing social-media accounts.It notes that agent skills and MCP resources support account management.Source: §1 · p. 1
  • Prior poisoning work includes attacks that insert malicious answers into retrieval databases and findings that LLM decisions can favor certain writing styles.The paper cites these as examples of agent and LLM vulnerability.Source: §1 · p. 1
  • The paper identifies a gap between poisoning studies that assume malicious content has reached an agent and recommendation studies that treat users as passive, leaving the exposure of LLM agents through recommendations poorly understood.The missing setting combines an evolving recommendation policy with an agent whose preferences are reflected in liking behavior.Source: §1 · pp. 1–2

To what extent can poisoned posts gradually shift

  • The proposed pipeline targets similarity-based recommendation systems and asks whether poisoned posts can gradually shift recommendations toward attacker-controlled content.The paper frames this as a question about gradual recommendation shifts.Source: To what extent can poisoned posts… · p. 2

2 Related Work

The related work spans attacks on LLM agents, shilling and profile-poisoning attacks, and several recommendation algorithm families. This paper distinguishes its focus on using the recommendation mechanism to amplify poisoning.

Attacks and Bias in LLM-Based Agents

  • Prior agent-poisoning work injects malicious content into a static retrieval database or assumes an agent has already encountered the poisoned post, unlike this paper’s focus on recommendation-mediated exposure.The passage also notes systematic biases in LLM-based recommender agents.Source: §2 · p. 2

Shilling Attacks in Recommendation Systems

  • Traditional shilling attacks inject fake profiles or interactions to promote items, whereas this work targets the recommendation mechanism to amplify poisoning rather than passively polluting profiles.The passage also cites work on profile pollution in modern recommendation systems.Source: §2 · p. 2

Recommendation Algorithms

  • Recommendation systems include similarity-based, graph-based, semantic-ID, transformer-based sequential, and popularity-based algorithms, but commercial platforms’ specific combinations are generally unknown.This paper focuses on similarity-based recommendation with like-based scoring.Source: §2 · p. 2
  • Figure 2 is identified as an illustration of how the OASIS recommendation system works.The supplied caption does not specify its mechanisms or visual encoding.Source: Figure 2 · p. 3

3 Problem Setup

The setup models OASIS as a similarity-based, like-scored recommender serving a single LLM agent. The threat model gives the attacker profile knowledge and a one-time post injection, with the goal of exploiting a self-reinforcing feedback loop.

3.1 Recommendation System Setup

  • The OASIS-style setup represents users and posts with TwHIN-BERT embeddings and computes user–post similarity as cosine similarity.The user embedding comes from the profile description, while the post embedding comes from its content.Source: §3.1 · p. 3
  • Liked posts enter a fixed-size sliding window, and the final recommendation score multiplies user–post similarity by a like score based on liked-post history.The setup uses default w = 14 and omits creation-time effects by replacing the date score with 1.Source: §3.1 · p. 3
  • At each interaction round, the system selects the top-k posts by score and randomly samples t for the feed, omitting recency and creator-fan-count factors.The focus is the similarity-driven mechanism rather than temporal or popularity dynamics.Source: §3.1 · p. 3

3.2 Threat Model Setup

  • The victim is modeled as one LLM agent with a human profile that can browse a social feed and react to posts.The attacker knows the target profile, injects posts once, and need not know the victim model or total post count.Source: §3.2 · p. 3
  • The attack seeks a self-reinforcing loop in which liking poisoned content increases its share of later recommendations, eventually surfacing low-similarity poisoned posts.The target posts may occupy a substantial portion of the feed.Source: §3.2 · p. 3

4 The Poisoning Attack Algorithm

The attack uses a staged chain that moves from highly similar posts toward less similar poisoned posts, guided by theoretical conditions on similarity and stage structure. Its generation pipeline creates, repairs, filters, and ranks candidate posts to follow that progression.

4 The Poisoning Attack Algorithm

  • For each user, the attack selects highly similar clean posts as anchors and builds chains of posts with progressively decreasing user–post similarity.Each stage has a target similarity, and the chain transitions from relevant benign content toward intended poisoned content.Source: §4 · pp. 3–4

4.1 Theoretical Insights

  • Under the stated conditions, a decreasing chain of stage similarities can reproduce its designed trace when consecutive-stage gaps are small, within-stage posts are similar, and the final-stage similarity meets a lower bound.The theorem assumes t = k, that the poison posts beat the marginal clean post at the retrieval boundary, and mild conditions on the post distribution.Source: §4.1 · p. 4
  • The minimum-stage proposition and its supporting distribution assumption characterize how many stages are needed to move from an initial similarity to a target similarity.The supplied proposition states the condition but does not include the resulting formula’s value.Source: §4.1 · p. 4
  • When t < k, a sufficient number of poisoned posts in each stage is needed to ensure the randomly sampled feed contains poison.The proposition links the number of poisoned candidates to the chance of inclusion in the feed.Source: §4.1 · p. 4
  • The design principles require a profile-close first stage, enough stages with within-stage and consecutive-stage similarity, and a lower bound below which poisoned posts score below clean posts.Too few stages make it difficult to keep consecutive stages sufficiently similar, so the like-score bridge may fail.Source: §4.1 · p. 4

4.2 Poisoning Post Generation Pipeline

  • The generation pipeline prompts an LLM with the anchor, target similarity, and previous chain posts, while applying one of five engagement-oriented biases.The listed biases are position, verbosity, popularity, credibility, and linguistic style.Source: §4.2 · p. 5
  • Candidate posts are repaired toward target similarity, then screened for within-stage cohesion and continuity with the previous stage.The repair process adjusts profile-prefix or profile-term content when candidate similarity falls outside a tolerance band.Source: §4.2 · p. 5
  • The final candidate ranking prioritizes matching target similarity, then anchor similarity, continuity with the previous-stage winner, and linguistic quality.The highest-scoring candidate is selected; after injection, higher-similarity Stage-1 posts are prioritized first.Source: §4.2 · p. 5

5 Experiments

Across victim models, poisoned posts progress through the five stages, with Stage 5 eventually occupying at least half of the feed; ablations and a real-agent simulation support the pipeline’s effectiveness. The experiments also examine how stage count, like-history settings, and target similarity affect this progression.

5.1 Experiment Setup

  • The experiment used 50 OASIS user profiles and 2,000 posts, with five poisoning stages and a 10-post feed drawn from the top 100 similarity-ranked posts.The setup used 400 interaction rounds and the five most recent liked posts for the like-score history; evaluation tracked stage arrival in the feed and poisoned posts recommended below the top-100 similarity cutoff.Source: §5.1 · pp. 5–6

5.2 Main Results

  • The five stages appear in sequence across victim models, with Stage 5 appearing after earlier-stage posts; DeepSeek v4 pro surfaces Stages 2 and 3 earlier.The reported sequence progresses from Stages 1–2 to Stage 3, then Stage 4, and finally Stage 5.Source: §5.2 · p. 6, Figure 3 · p. 6, Figure 4 · p. 6
  • Rnd5/10 ranges from 179 to 248 rounds; across models, 48–69 of 200 poisoned posts entered candidates below the top-100 similarity threshold, and 32–57 were liked.Rnd5/10 is the first round when Stage 5 fills at least five of ten feed slots.Source: §5.2 · pp. 6–7, Table 1 · p. 7

5.3 Ablation Studies

  • Increasing the number of stages raises the number of users who see Stage-M posts from 40 of 50 at M = 1 to all 50, without delaying their first appearance.The authors note that clean posts sharing high similarity with poisoned posts can raise poisoned-post like scores, so fewer stages may suffice in practice than the theorem’s lower bound.Source: Figure 5 · p. 7, §5.3 · p. 7
  • Using poisoned posts generated in Section 5.2, increasing liked-post weight w in the like-score formula made posts from stages 3–5 appear earlier; the effect of like-history size |L| was not monotone.The tested history sizes were 0, 1, 5, 10, and 20, and tested weights were 0.2, 1, and 5.Source: §5.3 · p. 7

5.2 Main Results

  • Most victim models had higher poisoned-post like rates than clean-post rates, while GPT-5.4-nano and DeepSeek-v4-pro had similar rates that were still sufficient to exploit the like-score mechanism.The comparison concerns poisoning trigger designs across victim models.Source: §5.2 · p. 7, Table 2 · p. 8

5.4 Real AI Agents

  • In a 400-round simulation across 50 users, an OpenClaw-based victim also showed poisoned posts making up a large proportion of its feed.The agent used DeepSeek-V4-Flash in embedded mode, and its session was reinitialized each round, preventing conversational memory or reasoning history from carrying over.Source: Figure 7 · p. 8, §5.4 · p. 8

6 Conclusion

The paper studies similarity-based recommendation systems when the user is an LLM-based agent. It shows that liking early poisoned posts can pull later, less-similar posts into the feed, highlighting an agent-as-user threat surface.

6 Conclusion

  • The study focuses on the vulnerability of similarity-based recommendation systems when the user is an LLM-based agent.Source: §6 · p. 8
  • A multi-stage chain can exploit the like-score mechanism: liking early-stage posts triggers a feedback loop that pulls later, less-similar posts into the feed.The paper reports theoretical and empirical support for this finding.Source: §6 · p. 8
  • The authors aim to draw attention to the agent-as-user threat surface.Source: §6 · p. 8

Limitations

The study examines only the like mechanism, one type of recommendation-system algorithm; other algorithms have different mechanisms and corresponding vulnerabilities.

Limitations

  • The study examines only the like mechanism; graph-based and transformer-based recommenders use different mechanisms and have different corresponding vulnerabilities.The passage notes that transformer-based recommendation may be vulnerable to poisoned training data because transformers are trainable.Source: Limitations · p. 9
  • The findings therefore address the like mechanism rather than recommendation algorithms generally.Source: Limitations · p. 9
  • The passage identifies potential poisoned-training-data vulnerability for transformer-based recommendation as a distinct concern.Source: Limitations · p. 9

A Proofs

The appendix provides proofs for Theorem 4.1 and Proposition 4.1, while omitting the proof of Proposition 4.2 because it involves only basic probability calculations.

A Proofs

  • The appendix provides proofs for Theorem 4.1 and Proposition 4.1.Source: A Proofs · p. 11
  • The proof of Proposition 4.2 is omitted.Source: A Proofs · p. 11
  • The stated reason for omitting Proposition 4.2 is that its proof involves only basic probability calculations.Source: A Proofs · p. 11

A.1 Theorem 4.1

Theorem A.1 gives sufficient conditions for designed poisoned posts to be selected, assuming all selected posts enter the feed, the agent favors poisoned posts in the feed, and Assumption A.1 holds. The conditions specify an initial similarity threshold, transitions between stages, within-stage closeness, and a final-stage constraint.

A.1 Theorem 4.1

  • Theorem A.1's conditions are sufficient under t = k, user preference for poisoned posts when shown, and Assumption A.1.Assumption A.1 includes clean posts distributed uniformly around the user embedding in all directions, with radius negligible relative to its norm.Source: §A.1 · p. 11
  • For each adjacent stage pair, the posts must be sufficiently similar for Stage n + 1 posts to be selected when the like history contains Stage n posts.This is the transition condition C2.Source: §A.1 · pp. 11–12
  • Posts within each stage must be close enough for their similarity contribution to be approximated using a Stage n post.This is the within-stage condition C3.Source: §A.1 · p. 11
  • At the final stage, condition C4 requires the target similarity to satisfy the stated final-stage constraint so poisoned posts are favored over clean posts.The proof frames this comparison for a stationary like history containing Stage M posts.Source: §A.1 · pp. 11–12
  • At Stage 1, the target similarity σ1 must exceed the top-k user-clean-post similarity σ⋆ for poisoned posts to be selected.Source: §A.1 · p. 11

A.2 Proposition 4.1

The proposition bounds how many stages a poison chain needs under a Gaussian clean-post model, while the generation procedure controls similarity and uses localized, quality-filtered edits to create posts.

A.2 Proposition 4.1

  • As σM ↓ σmin, M diverges: too few stages force the per-stage similarity floor (C2) above 1 at some handoff, where the chain breaks.Proposition A.1 assumes clean posts follow N(eu, ρ²Id), with ρ ≪ 1.Source: §A.2 · p. 12

B.1 Stage-wise Similarity Control in LLMGEN

  • Each generation stage targets user cosine similarity τn within ±δ, rewriting the previous winner with minimal edits and taking one small step toward the next target.The specified tolerance is δ=0.012.Source: §B.1 · pp. 13–14

B.2 Poison design

  • The five engagement-bias mutations make localized edits while preserving the anchor’s topic, entities, and claims.The listed biases cover position, verbosity, popularity, credibility, and linguistic style.Source: §B.2 · p. 14
  • Stage prompts combine the user profile, target-similarity constraint, stage rule, bias mutation, and writing constraints.The template specifies 28–62 words and prohibits invented credentials.Source: §B.2 · p. 14
  • Candidate quality qc is a heuristic fluency score that rewards relevant terms, moderate length, and concrete terminology while penalizing injection-like or exaggerated marketing language.The score is clipped to [0, 1].Source: §B.2 · p. 14

C Experiment Setups

The experiments configure victim agents with user profiles and round-by-round feeds, vary memory guidance, and include an OpenClaw evaluation procedure.

C.1 Camel Agent Settings

  • Victim agents receive individual profiles, and each round presents the recommended feed for an action consistent with the profile and current posts.The targeted variant adds a fixed final target post, a persona/target blend ratio, and a target-post similarity constraint.Source: §C.1 · p. 15

C.2 Victim Agent Memory Modes

  • The memory conditions present recently liked posts, from newest to oldest, with no behavioral guidance, similarity-favoring guidance that amplifies homophily, or novelty-favoring guidance that discourages similar content.Source: §C.2 · p. 15

C.3 OpenClaw Settings

  • The OpenClaw setup runs OASIS rounds by selecting an available action from the current feed and reports final metrics after the rounds finish.The evaluation persona instructions distinguish stateless runs from runs that may use session memory.Source: §C.3 · p. 16

D.1 Additional Results

Additional experiments examine poison volume, transfer across attacker and embedding models, memory settings, and results across multiple post sets.

Additional results

  • As poisoned posts per stage increase from 5 to 40, Cnt5/10 decreases while Cnt100 and Like100 increase, and Like100/Cnt100 attains a maximum across the three memory settings.When the victim pursues diverse posts, Rnd5/10 increases while Cnt100 and Like100 decrease significantly.Source: §D.1 · pp. 16–17
  • GPT-5.4-nano achieves the most vulnerable case uniformly across victim models, while different embedding models generally transfer to one another without a uniform vulnerability pattern.The comparison varies both the LLM used for generation and the embedding model.Source: §D.1 · pp. 16–17, Figure 9 · p. 16, Figure 10 · p. 17
  • Agent configuration has a larger effect than memory size, and favoring posts similar to previously liked posts resembles the like-score mechanism exploited by the attack.The experiments vary memory sizes and neutral, similarity-favoring, and diversity-favoring configurations.Source: §D.1 · p. 17, Figure 11 · p. 17
  • Across four additional post sets, the evaluation fixes 10 user profiles and varies the post set and victim model across eight settings.The reported Cnt%100 and Like%100 normalize candidate inclusion and liking by injected poison count, averaged across five stages.Source: §D.1 · p. 17, Table 6 · p. 18
Loading 2609.27155v1…