Source-linked AI summary
GuardRAG: A Post-Retrieval Defense against Poisoning Attacks in Retrieval-Augmented Generation
Linghui Jiao, Kang Wei, Fushuo Huo, Ruijie Zhao, Yiyang Ni, Jingcai Guo, Yipeng Zhou, Jun Li
TL;DR
Coordinated corpus poisoning can place mutually reinforcing misinformation in retrieved RAG contexts, challenging reliable evidence filtering. GuardRAG uses post-retrieval group-level semantic analysis to identify and prune suspicious document clusters. Across several datasets, attacks, and LLMs, it substantially reduces attack success while maintaining clean-retrieval answer utility.
Problem
Coordinated corpus poisoning can make mutually reinforcing malicious documents dominate retrieved evidence, while checking every document before retrieval is impractical.
Source: §1 · p. 1, §3.2 · p. 3Method
GuardRAG is a training-free, plug-and-play post-retrieval defense that expands retrieval, deduplicates candidates, clusters semantically, and prunes suspicious clusters.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
Group-level evidence analysis offers a training-free defense that requires no additional LLM generation calls and improves RAG robustness across the evaluated settings.
Source: §6 · p. 8When few poisoned documents are retrieved, their structural patterns may be too weak for reliable cluster-level defense; highly correlated benign documents may also be erroneously pruned.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Retrieval-Augmented Generation (RAG) enhances the factual grounding of Large Language Models (LLMs) by incorporating external evidence, but also exposes them to corpus poisoning attacks. In particular, coordinated attacks can introduce multiple mutually reinforcing poisoned documents into retrieved contexts, making reliable evidence filtering challenging. To address this problem, we propose GuardRAG, a training-free and plug-and-play post-retrieval defense that exploits the semantic structure of retrieved evidence to identify suspicious document groups. Specifically, GuardRAG first expands the retrieval range and removes near-duplicate candidates, then partitions the retained documents using Hierarchical Agglomerative Clustering (HAC) with a query-adaptive threshold. Finally, it scores and prunes suspicious clusters based on query relevance, cluster size, and semantic compactness. Experiments across three question-answering datasets, multiple poisoning attacks, and six LLMs demonstrate that GuardRAG substantially reduces attack success rates while maintaining answer utility under clean retrieval.
1 Introduction
GuardRAG addresses coordinated corpus poisoning by detecting suspicious groups in retrieved evidence rather than evaluating documents independently. It combines retrieval expansion and deduplication, adaptive semantic clustering, and suspicious-cluster pruning; experiments report lower attack success while preserving clean-retrieval utility.
1 Introduction
- Corpus poisoning manipulates retrieved evidence to influence generation, while checking every document in large or continuously updated corpora before retrieval is impractical.The paper motivates post-retrieval, pre-generation filtering as an alternative.Source: §1 · p. 1
- Coordinated poisoned documents can reinforce the same misinformation and dominate retrieved evidence, weakening conventional aggregation and reliability assessment.This motivates examining collective document structure rather than assessing passages independently.Source: §1 · p. 1
- Retrieved poisoned documents tend to show stronger local semantic concentration than benign evidence in embedding space.This structural pattern motivates the defense's group-level analysis.Source: §1 · pp. 1–2
- GuardRAG is a training-free, plug-and-play post-retrieval defense that expands retrieval, removes near-duplicates, clusters semantically, and prunes suspicious clusters before generation.Its focus is group-level pattern detection rather than individual-document assessment.Source: §1 · pp. 1–2
- Across three datasets, multiple poisoning strategies, and six generators, GuardRAG substantially lowers attack success while preserving answer utility under standard clean retrieval.The introduction reports this evaluation across multiple retrievers as well.Source: §1 · p. 2
2 Related Work
Related work covers RAG components and retrieval methods, corpus-poisoning attacks, and three broad families of defenses. GuardRAG differs by analyzing group-level evidence structure after retrieval without auxiliary training or additional generation calls.
Retrieval-Augmented Generation
- RAG uses a retriever to fetch query-relevant corpus documents and a generator to answer using the query and retrieved evidence.Retrieval may use sparse lexical matching or dense embedding similarity.Source: §2 · p. 2
Corpus Poisoning Attacks
- Corpus-poisoning research studies malicious documents that manipulate generation, including attacks using adversarial optimization, backdoor triggers, or collaborative optimization.The passage names PoisonedRAG, POISONCRAFT, AgentPoison, BadRAG, CPA-RAG, and PR-Attack.Source: §2 · p. 2
Defense Methods
- Existing defenses aggregate or assess evidence, use reflection or internal knowledge, or apply learned filtering, compression, and context selection.The described approaches may incur additional inference-time computation, auxiliary training, or task-specific data.Source: §2 · pp. 2–3
- GuardRAG instead analyzes retrieved evidence at the group level after retrieval and before generation, requiring neither auxiliary model training nor additional LLM generation calls.The paper describes it as compatible with RAG pipelines as a training-free, plug-and-play defense.Source: §2 · p. 3
3 Problem Formulation
The paper formulates RAG as retrieval from a potentially compromised corpus followed by generation conditioned on the query and retrieved context. Its threat model specifies the attack objective, attacker capabilities and knowledge, and a structural pattern in retrieved evidence.
3 Problem Formulation
- The paper's threat model concerns a compromised knowledge corpus and aims to preserve the integrity of generated answers.The RAG pipeline comprises a corpus, retriever, and generator.Source: §3 · p. 3, §3.1 · p. 3
3.1 Task Formulation
- A RAG system retrieves top-k documents from corpus C for query q, then generates answer y conditioned on q and those documents.The formulation aims to safeguard answer integrity when the corpus is compromised.Source: §3.1 · p. 3
3.2 Threat Model
- The threat model considers knowledge-misleading attacks that seek incorrect answers containing misinformation for a target query.The target phrase is a predetermined substring in the generated response.Source: §3.2 · p. 3
- Embedding-space distributions show poisoned documents as more locally concentrated than benign documents under different poisoning intensities.The figure caption identifies poisoned documents as red triangles and benign documents as blue circles.Source: Figure 2 · p. 3
- An attack counts as successful when the generated output contains the predetermined target phrase, typically a short phrase or misinformation entity..Source: §3.2 · p. 3
- The attacker can inject a limited budget of poisoned passages that are crafted to improve retrieval likelihood and mutually reinforce the same incorrect answer.The budget is small relative to the corpus.Source: §3.2 · p. 3
- Attackers are assumed unable to access generator parameters or weights, but able to access specific user queries and retriever parameters..Source: §3.2 · p. 3
4 GuardRAG
GuardRAG detects coordinated poisoned evidence through group-level semantic patterns, using retrieval expansion, adaptive clustering, and suspicious-cluster pruning. Its design draws on the observation that poisoned documents often form locally concentrated semantic groups.
4.1 Motivation
- In 896 of 900 query-level instances, poisoned documents had smaller within-group than poisoned-to-benign distances; all retrieved poisoned documents formed one HAC cluster in 862 cases.The analysis spans three attacks and three datasets.Source: §4.1 · p. 3
4.2 Stage I: Retrieval Expansion and Deduplication
- GuardRAG expands retrieval from top-k to top-K, where K = mk and m > 1, then removes near-duplicates at cosine similarity ≥ 0.98, retaining the higher-ranked document.The resulting set R_q has N_q ≤ K documents.Source: §4.2 · p. 4
4.3 Stage II: Adaptive Semantic Clustering
- GuardRAG applies average-linkage HAC to normalized document embeddings, setting a query-specific threshold from the largest adjacent gap in sorted nearest-neighbor distances and constraining it with τmax.Figure 3 reports the maximum poisoned 1-NN distance, the largest-gap endpoints, and the resulting threshold across NQ, HotpotQA, and MS MARCO; τmax limits overly aggressive clustering to protect benign evidence in clean retrieval.Source: §4.3 · p. 4
4.4 Stage III: Suspicious Cluster Pruning
- For each cluster, GuardRAG combines query relevance, cluster size, and semantic compactness into a suspiciousness score, then removes the highest-scoring cluster among those with at least three documents.If no cluster meets the minimum size, all documents are retained; filtered evidence is passed to the generator.Source: §4.4 · p. 5
5 Evaluation
Across the evaluated attacks and baselines, GuardRAG achieves low attack success while preserving answer utility. Detailed analyses examine adaptive attacks, retrieval size, ablations, clean-retrieval stress tests, and clustering choices.
5.1 Experimental Setup
- Evaluation covers Natural Questions, HotpotQA, and MS MARCO, three dense retrievers, six generators, multiple poisoning strategies, and four core metrics: ASR, PDFR, CFR, and ACC.The default setting is HotpotQA, Contriever, and Llama-2-13B; standard retrieval uses k = 5, and GuardRAG expands to K = 10.Source: §5.1 · pp. 5–6
5.2 Main Results
- Under PRAGB, average ASR is 4.0% on HotpotQA, 0.8% on NQ, and 8.0% on MS MARCO; average PDFR is at least 95% under PRAGB and POISONCRAFT, while CFR is at most 10%.PRAGB PDFR reaches 100% on HotpotQA and NQ.Source: §5.2 · pp. 6–7, Table 1 · p. 6
- Across PRAGB adv3 and adv5 settings, GuardRAG achieves the lowest ASR in almost all settings while maintaining competitive ACC and keeping ASR low as poisoning intensity increases.For example, GuardRAG's ASR stays below 10.0% in both settings, while several baselines degrade substantially.Source: §5.2 · p. 7, Table 2 · p. 7
5.3 Detailed Analysis
- Under adaptive attacks, GuardRAG reduces ASR from 76.0% to 6.0% for diversification and from 77.0% to 23.0% for the white-box attack, with PDFR of 99.0% and 72.0%, respectively.The white-box attack selects five passages from 15 candidates using defense feedback and beam search; dispersed poisoned evidence is harder to isolate through cluster-level filtering.Source: §5.3 · p. 7, Table 3 · p. 7
- As retrieval depth increases, Vanilla RAG's ASR falls with limited ACC gains, whereas GuardRAG's expansion supports cluster filtering that reduces ASR while improving ACC.The comparison is reported in Figure 4.Source: §5.3 · p. 7, Figure 4 · p. 8
- In the controlled ablation, increasing retrieval depth from 5 to 10 without filtering changes ASR from 76.0% to 74.0% and ACC from 22.0% to 26.0%; applying Stages II–III at depth 5 yields 8.0% ASR, 60.0% ACC, and 98.0% PDFR.Full GuardRAG further reduces ASR to 3.0% and improves ACC to 86.0%.Source: §5.3 · pp. 7–8, Table 4 · p. 8
- GuardRAG maintains 90.0% ACC on clean-original and clean-duplicate retrieval, but ACC falls to 72.0% with paraphrased benign evidence, which can form compact clusters.Clean-duplicate removal averages 2.22 passages per query with 27.2% CFR; paraphrase stress averages 3.56 cluster-pruned passages per query with 35.6% CFR.Source: §5.3 · p. 8, Table 5 · p. 8
- Across HotpotQA, NQ, and MS MARCO, HAC achieves PDFR of 100%, 100%, and 95% and CFR of 3%, 6%, and 10%, respectively, while alternative clustering methods show less favorable tradeoffs.DBSCAN reaches 48% CFR on MS MARCO; Figures 5 and 6 compare PDFR and CFR across clustering methods.Source: §5.3 · p. 8, Figure 5 · p. 8, Figure 6 · p. 8
6 Conclusion
GuardRAG combines retrieval expansion and deduplication, adaptive semantic clustering, and suspicious-cluster pruning to defend against corpus poisoning. Across three QA datasets, multiple poisoning strategies, and six LLMs, it improves robustness while preserving clean-retrieval answer utility without additional defense-time LLM calls.
6 Conclusion
- GuardRAG combines retrieval expansion and deduplication, adaptive semantic clustering, and suspicious-cluster pruning in a training-free post-retrieval defense.It analyzes group-level evidence structure rather than scoring documents independently.Source: §6 · p. 8
- Across three question-answering datasets, multiple poisoning strategies, and six LLMs, GuardRAG reduces poisoning success while preserving answer utility under clean retrieval.The defense requires no additional LLM calls.Source: §6 · p. 8
Limitations
GuardRAG’s effectiveness depends on structural patterns in retrieved evidence, and similar patterns in benign evidence can lead to erroneous pruning.
Limitations
- When only a few poisoned documents are retrieved, their structural patterns may be too weak for reliable cluster-level defense.GuardRAG depends on identifiable structural patterns in retrieved documents.Source: Limitations · p. 9
- Highly correlated benign documents may resemble coordinated poisoned evidence, increasing the risk of erroneous pruning.Distinguishing malicious coordination from legitimate semantic agreement remains a future-work challenge.Source: Limitations · p. 9
Ethics Statement
The ethics statement describes evaluation using public QA benchmarks and established attack implementations. Appendix A.1 estimates a global clustering-threshold cap from benign retrieved documents to limit overly permissive clustering.
Ethics Statement
- The work evaluates post-retrieval filtering with publicly available question-answering benchmarks and established attack implementations.The stated aim is to reduce malicious evidence’s influence on generation and support more trustworthy RAG systems.Source: Ethics Statement · p. 9
A.1 Estimation of the Global Bound τmax
- The global upper bound τmax is estimated from benign-document semantic distributions to reduce overly aggressive clustering and erroneous pruning in clean retrieval.The cap constrains query-specific thresholds that become excessively large.Source: §A.1 · p. 12
- Calibration samples 100 queries per benchmark, retrieves each query’s top 10 clean-corpus documents, and uses queries disjoint from the main evaluation.Across three benchmarks, this yields 300 calibration queries.Source: §A.1 · p. 12
- The bound is τmax = 0.8 × 0.877 ≈ 0.702, using 0.877 as the average pairwise distance among benign retrieved documents.The distance baseline is aggregated across all 300 sampled queries; each retrieval set contributes 45 document pairs.Source: §A.1 · p. 12
A.2 Implementation Details of Adversarial Baselines
The appendix details how adversarial baselines were implemented, describes two adaptive attack stress tests, and specifies the QA datasets and sampling used in evaluation.
A.2 Implementation Details of Adversarial Baselines
- PoisonedRAG implementations follow released materials; PRAGW uses its unchanged white-box HotFlip-style search, while PRAGB uses released adversarial passages.PRAGW retains the original constraints, including up to five adversarial passages per target query.Source: §A.2 · p. 12
- POISONCRAFT is reproduced with a fixed injection template, retriever-specific keywords, and a 20-token optimized suffix, with up to five variants per query.Optimization runs for at most 150 steps per attempt, subject to a global cap of 500 steps.Source: §A.2 · p. 12
A.3 Adaptive Attack Stress Tests
- The adaptive tests divide five poisoned passages into two semantic groups or select five from 15 candidates using GuardRAG feedback and beam search.The white-box attack seeks to reduce concentration while preserving retrieval effectiveness; results are reported in Table 3.Source: §A.3 · p. 12
B.1 Dataset Details
- The evaluation uses NQ, HotpotQA, and MS MARCO, sampling 100 queries from each dataset for the main experiments.Their corpora come from Wikipedia for NQ and HotpotQA and Bing web documents for MS MARCO.Source: §B.1 · p. 13
B.2 Efficiency Analysis
GuardRAG’s query-adaptive threshold balances poisoned-document removal against benign-evidence preservation across datasets, while retrieved poisoned documents show strong local semantic concentration. The defense also avoids additional LLM calls, with reported runtime and token savings.
B.2 Efficiency Analysis
- 7.715 seconds per query and approximately 4.9× fewer LLM tokens than the most token-intensive baseline were reported for GuardRAG.GuardRAG makes no defense-time LLM calls; filtering operates in embedding space and pruning reduces the context sent for generation.Source: Table 7 · p. 13, §B.2 · p. 13
B.3 Comparison between Fixed and Adaptive Thresholds
- 18.0% PDFR at τ = 0.4 rose to 83.0% at τ = 0.6 on MS MARCO, while CFR rose from 7.0% to 24.0%.Smaller fixed thresholds impede cluster formation and filtering; larger thresholds increase filtering but also remove more benign documents.Source: §B.3 · p. 13
- 100.0% PDFR with 3.0% CFR on HotpotQA illustrates the adaptive threshold’s highest PDFR and relatively low CFR across all three datasets.The adaptive strategy outperforms fixed thresholds in this balance; the comparison includes fixed-threshold trade-offs on MS MARCO.Source: §B.3 · p. 13
B.4 Empirical Validation of Structural Concentration
- 896 of 900 query instances had ρ < 1, 862 placed all retrieved poisoned documents in one HAC cluster, and the largest poisoned cluster averaged 98.4% coverage.The pattern was strongest under PRAGB and POISONCRAFT, but weaker for PRAGW on MS MARCO.Source: §B.4 · p. 14
C Case Study
The case studies show both successful removal of coordinated poisoned evidence and over-pruning when poisoned and benign passages overlap semantically. The appendix describes four clustering alternatives and supplies the prompt template and LLM-use disclosure.
C Case Study
- GuardRAG removes five poisoned passages as a cluster while retaining benign evidence, after which the generator answers “World’s Best Goalkeeper.”Figure 7 presents this successful-defense case.Source: C Case Study · p. 14, Figure 7 · p. 16
- In the Macungie example, one mixed cluster contains five poisoned and four benign passages; pruning it removes the benign evidence supporting Lehigh County.The generator nevertheless returns the correct county, illustrating over-pruning from semantic overlap.Source: C Case Study · pp. 14–15, Figure 8 · p. 17
D.1 Hierarchical Agglomerative Clustering
- HAC begins with each sample as its own cluster and repeatedly merges the closest clusters; this work uses Euclidean distance, average linkage, and query-adaptive threshold τq.The threshold controls when merging stops.Source: §D.1 · p. 15
D.2 DBSCAN
- DBSCAN identifies dense embedding-space regions as clusters and marks low-density samples as noise; it uses neighborhood radius ϵ and min_samples = 3 here.Unlike HAC, DBSCAN does not require a preset number of clusters.Source: §D.2 · p. 15
D.3 Affinity Propagation
- Affinity Propagation exchanges responsibility and availability messages between candidate exemplars, then assigns other samples to selected exemplars.It does not use HAC’s distance threshold; the experiments set its damping factor to 0.9.Source: §D.3 · p. 15
D.4 Mean Shift
- Mean Shift iteratively moves points toward higher estimated embedding-space density and clusters samples that converge to the same mode.Its query-specific bandwidth is estimated from document embeddings with quantile=0.2.Source: §D.4 · pp. 15–16
E Prompt Templates
- The prompt supplies filtered documents as context, requests a short concise answer, and directs the assistant to say “I don’t know” when the answer is absent.Source: E Prompt Templates · p. 16
F The Use of LLM
- LLMs were used for manuscript language polishing, while the authors developed and verified the research ideas, methods, experiments, analysis, and conclusions.Source: F The Use of LLM · p. 17