Poison-RAG: Adversarial Data Poisoning Attacks on Retrieval-Augmented Generation in Recommender Systems

Fatemeh Nazary; Yashar Deldjoo; Tommaso di Noia

Source-linked AI summary

Poison-RAG: Adversarial Data Poisoning Attacks on Retrieval-Augmented Generation in Recommender Systems

Fatemeh Nazary, Yashar Deldjoo, Tommaso di Noia

arXiv:2501.11759v1Jan 20, 2025cs.IR

TL;DR

RAG-based recommender systems can be vulnerable when adversaries manipulate textual metadata, motivating Poison-RAG's study of provider-oriented poisoning attacks. The framework evaluates locally personalized and globally uniform tag strategies, finding that local attacks are more effective while global attacks can inadvertently boost popular items. The paper concludes that metadata management and stronger defenses are needed for secure and trustworthy recommendations.

Problem
Text-based RAG recommender systems expose textual metadata to poisoning attacks that can influence recommendation outcomes and raise security concerns.
Method
Poison-RAG evaluates local personalized and global uniform tag-poisoning strategies against RAG-based recommender systems, including a black-box MovieLens setting.
Results
Local attack strategies are more effective than global ones, while global strategies can inadvertently boost popular items’ relevance.
Takeaways & Limitations
Robust metadata management and stronger defenses are needed to safeguard secure and trustworthy RAG-based recommendations.
Takeaways & Limitations
The attack setting modifies only tags while keeping item genres and titles unchanged to maintain stealthiness.

Abstract

from arXiv · show

This study presents Poison-RAG, a framework for adversarial data poisoning attacks targeting retrieval-augmented generation (RAG)-based recommender systems. Poison-RAG manipulates item metadata, such as tags and descriptions, to influence recommendation outcomes. Using item metadata generated through a large language model (LLM) and embeddings derived via the OpenAI API, we explore the impact of adversarial poisoning attacks on provider-side, where attacks are designed to promote long-tail items and demote popular ones. Two attack strategies are proposed: local modifications, which personalize tags for each item using BERT embeddings, and global modifications, applying uniform tags across the dataset. Experiments conducted on the MovieLens dataset in a black-box setting reveal that local strategies improve manipulation effectiveness by up to 50\%, while global strategies risk boosting already popular items. Results indicate that popular items are more susceptible to attacks, whereas long-tail items are harder to manipulate. Approximately 70\% of items lack tags, presenting a cold-start challenge; data augmentation and synthesis are proposed as potential defense mechanisms to enhance RAG-based systems' resilience. The findings emphasize the need for robust metadata management to safeguard recommendation frameworks. Code and data are available at https://github.com/atenanaz/Poison-RAG.

1 Introduction

RAG-based recommender systems combine retrieval, augmentation, and generation, but text-based metadata introduces poisoning vulnerabilities. Poison-RAG investigates local and global tag attacks that manipulate recommendation outcomes while preserving semantic relevance.

RAG-based recommendation: RAG recommender systems retrieve external item information, augment user queries with it, and generate contextually relevant recommendations.Text-based retrieval can use tags, genres, reviews, or other metadata to construct profiles and ranked candidate lists.
Security motivation: Text-based retrieval creates security vulnerabilities because adversaries can manipulate descriptions or tags to bias exposure and rankings.These vulnerabilities are linked to the open-ended nature of textual metadata.
Study objective: Poison-RAG investigates adversarial poisoning attacks that manipulate textual item metadata in RAG-based recommender systems.The framework focuses on provider-oriented evaluation of recommendation outcomes.
Attack strategies: Local attacks personalize tag modifications using BERT-based similarity across popularity classes, whereas global attacks apply a uniform tag pool across the dataset.Both strategies target recommendation manipulation, but they differ in whether tags are selected for individual items or globally.
Main finding: Only the local tag-selection method effectively manipulates recommendation outcomes, likely because its focused approach preserves semantic relevance and enhances adversarial impact.The analysis examines provider-oriented exposure-based and exposure-relevance outcomes.
Method and evaluation: The proposed tag-selection methodology combines adversarial impact with semantic relevance to keep manipulated tags contextually appropriate and inconspicuous.Experiments use the MovieLens dataset to assess attack effectiveness and recommendation robustness.

2 Poison-RAG: A Data Poisoning Attack Framework Against Retrieval Augmented Recommender Systems

Poison-RAG frames provider-side recommendation manipulation as a metadata-poisoning problem, modifying tags to promote long-tail items and demote popular ones. Its attack objective maximizes long-tail exposure while minimizing popular-item exposure, using tag selection that balances adversarial impact with semantic relevance.

Attack framework: Poison-RAG modifies item metadata, especially tags, to promote long-tail items and demote popular items in RAG-based recommendations.
Attack objectives: Attack success is evaluated through shifts in popularity classes and changes in item rankings within recommendation lists.
Attack objectives: Exposure measures whether an item appears in a user’s attacked recommendation set and weights its position using log2(rank_u(i)+1).The indicator captures presence, while rank_u(i) denotes the item’s recommendation-list position.
Attack objectives: The adversary seeks to maximize cumulative long-tail exposure while minimizing popular-item exposure across users.High long-tail exposure and low popular-item exposure indicate greater attack success.
Tag optimization: Adversarial tags are selected from candidate sets by maximizing an adversarial score for each item and choosing k modified tags.The candidate set depends on whether the local or global strategy is used.
Tag-selection method: The tag-selection score combines adversarial impact with semantic relevance to keep modifications contextually appropriate and inconspicuous.Adversarial impact targets the desired popularity shift, while semantic relevance is computed from tag and item embeddings.
Tag-selection strategies: Local selection builds an item-specific tag pool from semantically similar items across popularity classes, whereas global selection uses one uniform pool across the dataset.Both strategies seek subtle ranking or popularity shifts while remaining undetectable.

3 Experimental Setup

The experiment uses a filtered MovieLens dataset enriched with GPT-3.5-turbo descriptions and automatically generated tags. User profiles are represented by aggregated item embeddings using rating intensity and interaction recency.

Dataset: The study uses MovieLens ml-latest-small, retaining users with 20–100 interactions while excluding inactive users and superusers.
Data enrichment: GPT-3.5-turbo generates detailed item descriptions from movie titles and genres for cold-start and non-cold-start attack scenarios.
Data enrichment: Relevant item tags are automatically extracted from enriched descriptions through preprocessing, keyword extraction, and relevance-based selection.
User representation: User profiles aggregate item embeddings from interaction histories using rating-weighted averaging and temporal decay-weighted averaging.Item embeddings encode semantic attributes.
User representation: Temporal decay weighting incorporates interaction recency with w(t) = e^-λ·time_diff(t), λ = 0.01, and power factor α = 1.2.

4 Experiments and Results

The experiments evaluate how adversarial metadata modifications affect exposure and relevance in RAG-based recommendation, before and after reranking, and whether data augmentation mitigates attacks. Local attacks more reliably demote popular items, while global attacks can backfire and long-tail promotion remains difficult.

Experimental design: The experiments measure attack effects on exposure and relevance before and after reranking, while testing whether auto-tagging and generated content mitigate manipulation.The evaluation covers exposure metrics, relevance metrics, and data-augmentation scenarios.
Exposure results: Local modifications reduce popularity lift, whereas global modifications can inadvertently boost popular items.In cold-start Decay conditions, popularity lift falls from 0.9380 to 0.9191 or 0.9282 under local attacks, while global attacks reach 1.3938 or 1.6235.
Exposure results: Most attack settings fail to substantially increase long-tail coverage, making long-tail promotion harder than demoting popular items.The reported pattern is that popular items can be demoted, especially through local tag modifications, but long-tail items remain difficult to promote.
Relevance results: In generated scenarios, popular-item HR@1 drops from 0.3229 to 0.3021 before reranking and from 0.2708 to 0.2552 after reranking.These changes represent an approximately 5–6% decline in relevance and align with the attack objective of demoting popular items.
Relevance results: Global strategies can raise popular-item HR@1 from 0.3229 to 0.4010 or 0.4375, indicating attack failure through unintended relevance gains.The reported increases correspond to 24–35% improvements for popular items.
Defense effects: Data augmentation offers partial mitigation, particularly for mid-tail items, but does not fully defend against adversarial manipulation.Mid-tail HR@1 rises from 0.3483 to 0.3652 in one augmented setting, while the broader attack effects persist.

5 Conclusion and Future Works

The conclusion presents Poison-RAG as a framework for poisoning RAG-based recommender systems through textual metadata manipulation. It finds local attacks more effective than global ones, while data augmentation provides only partial resilience and motivates further work on hybrid attack-defense mechanisms.

Conclusion: Poison-RAG examines how textual metadata manipulation can exploit vulnerabilities in RAG-based recommender systems.The framework targets producer-side recommendations by altering metadata to influence item popularity.
Conclusion: Local attacks are more effective than global attacks, while promoting long-tail items remains challenging and global attacks can boost popular-item relevance.The conclusion characterizes popular-item demotion as relatively feasible but long-tail promotion as resistant to manipulation.
Conclusion: Data augmentation through auto-tagging and enriched metadata generation offers partial resilience, particularly for mid-tail items.The stated defense effect is partial rather than complete.
Future work: Future work should explore hybrid attack-defense mechanisms that balance robustness against manipulation with recommendation quality.The proposed direction focuses on strengthening defenses while maintaining recommendation quality.