Source-linked AI summary
Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
Minh Tran, Cuong Dang, Tuc Nguyen, Khanh-Tung Tran, Minh Huynh Nguyen, Trinh Chau, Kien Le, Do Xuan Long, Jiahao Zhang, Hoang D. Nguyen, Thanh Le, Suhang Wang
TL;DR
RAG improves factual grounding but introduces robustness and security risks, while existing surveys provide limited treatment of attacker objectives, threat models, and defenses across the full pipeline. This survey unifies threat modeling and objective-based attack taxonomy with stage-specific defense coverage, benchmarks, and explainability methods. It concludes that robustness analysis must account for evolving RAG variants and unresolved limitations in coverage and taxonomy.
Problem
Existing RAG surveys offer limited analysis of attacker objectives, threat models, and adversarial defenses across the full RAG pipeline.
Method
The survey formalizes threats over the corpus, retriever, and generator, organizes attacks by accuracy, privacy, and fairness, and reviews defenses by pipeline stage.
Results
The survey synthesizes RAG attacks, pipeline-aware defenses, robustness benchmarks, explainability methods, and open challenges across standard and emerging RAG settings.
Takeaways & Limitations
The resulting framework serves as a roadmap for developing trustworthy RAG systems and evaluating robustness under adversarial and noisy retrieval conditions.
Takeaways & Limitations
Coverage may not fully capture newly developing RAG areas, and some attacks are inherently multi-objective while defense taxonomy remains stage-based.
Abstract
from arXiv · showhide
Retrieval-Augmented Generation (RAG) enhances large language models by grounding outputs in external knowledge, improving factuality and reducing hallucinations. At the same time, the retrieval-augmented pipeline introduces new robustness and security risks, including corpus poisoning, backdoor attacks, privacy leakage, and fairness violations. Despite rapid progress in this area, existing surveys remain limited in their treatment of attacker objectives, threat models, and stage-specific defenses across the full RAG pipeline. This survey presents a unified and pipeline-aware overview of RAG robustness. We formalize threat models over the corpus, retriever, and generator, and organize attacks into three main objectives: accuracy, privacy, and fairness. We further review defenses from a pipeline-aware perspective, covering the retrieval, rerank, generation, and traceback stages. In addition, we summarize robustness benchmarks and explainability methods for more deeply evaluating and explaining RAG robustness.
1 Introduction
RAG grounds language-model outputs in retrieved external knowledge, but its retrieval-augmented pipeline introduces distinct security and robustness risks. This survey addresses gaps in prior surveys through a unified framework of threat models, attack objectives, pipeline-aware defenses, benchmarks, and explainability methods.
- RAG supplements LLM knowledge with retrieved external documents to improve factual accuracy and reduce hallucinations.
- Attackers can extract private corpus content, inject misleading evidence, or amplify demographic bias through retrieved content.
- Prior surveys covered trustworthiness dimensions or security risks but offered limited analysis of attacker objectives, capabilities, attack structure, and adversarial defenses.
- The survey organizes attacks by objectives and defenses by pipeline stage to connect security harms with practical intervention points.Its attack objectives include accuracy, privacy, and fairness, while defense stages include retrieval, rerank, generation, and traceback.
- The survey methodology combines seed-paper citation tracing, academic keyword searches, manual screening, and backward snowballing.Included studies directly investigate RAG attacks, defenses, vulnerabilities, privacy, fairness, or robustness.
2 Background and Preliminaries
The paper presents standard RAG as retrieval followed by generation, then frames robustness around attack surfaces in the corpus, retriever, and generator. It organizes defenses by retrieval, rerank, generation, and traceback stages while noting open questions for emerging RAG variants.
- Standard RAG: Standard RAG retrieves relevant documents from an external corpus and uses them with the query and prompt to generate a response.
- Standard RAG: An encoder represents queries and documents, a similarity function scores relevance, and the retriever returns the top-k documents.
- Variants of RAG: GraphRAG, Multimodal RAG, and Agentic RAG extend standard RAG toward structured retrieval, multiple modalities, and stateful external-resource interactions.
- Variants of RAG: Whether structural constraints in emerging RAG variants reduce or merely shift attack strategies remains an open question.
- Threat Models: The threat model considers the corpus, retriever, and generator as separate attack surfaces governed by attacker goals, knowledge, and capabilities.The corpus determines available information, the retriever selects information, and the generator uses selected information to produce the response.
- Defense Stages: Defenses are categorized by retrieval, rerank, generation, and traceback stages because each stage offers distinct mitigation mechanisms.
3 Taxonomy of Adversarial Attacks on RAG
The survey organizes adversarial RAG attacks by accuracy, privacy, and fairness objectives, while distinguishing threat surfaces, attacker capabilities, and targeted versus untargeted behavior. Corpus poisoning, backdoors, prompt injection, and probing exploit different parts of the retrieval pipeline to alter outputs or extract information.
- Attack Objectives: RAG attacks target three objectives: accuracy, privacy, and fairness.Accuracy attacks induce incorrect answers, privacy attacks extract sensitive information, and fairness attacks amplify discriminatory outputs.
- Accuracy Attacks: Corpus poisoning, backdoors, and prompt injection use injected documents as misleading evidence, trigger-bearing context, or instruction-bearing payloads.The injected document set D′ serves different functions depending on the attack mechanism.
- Accuracy Attacks: Targeted accuracy attacks steer selected queries toward attacker-specified answers through malicious documents and retrieval- or generation-oriented optimization.PoisonedRAG separates retrieval-optimized and generation-oriented passage segments, while AGGD provides a more systematic gradient-guided search.
- Accuracy Attacks: Backdoor attacks activate only when a specific trigger appears, either through test-time document or prompt injection or through training-time retriever or generator compromise.TrojanRAG compromises retrieval behavior, whereas BALD embeds trigger behavior in the generator.
- Fairness Attacks: Fairness attacks primarily target designated demographic groups by steering RAG toward biased outputs through corpus poisoning and backdoor mechanisms.The surveyed examples concern disparities associated with gender, race, or age.
- Privacy Attacks: Privacy attacks seek sensitive corpus information rather than output manipulation, with targeted attacks using private objectives and untargeted attacks progressively expanding corpus coverage.Untargeted extraction ranges from random probing to agent-based feedback that alternates exploration and exploitation.
4 Taxonomy of Defenses for RAG Robustness
The survey classifies defenses by intervention point across retrieval, reranking, generation, and traceback stages. These defenses improve evidence selection, suppress suspicious passages, reduce harmful influence during generation, or identify sources responsible for failures.
- Retrieval-Stage Defenses: Retrieval-stage defenses strengthen initial evidence selection against noise, perturbations, corpus poisoning, and adversarial manipulation.They include diagnostics, evidence-centric retrieval, corpus sanitization, and robust retriever training.
- Rerank-Stage Defenses: Rerank-stage defenses refine the retrieved top-k1 set by promoting relevant benign documents and pushing suspicious candidates below the final top-k2 cutoff.GRADA uses a similarity graph over retrieved documents to suppress poisoned passages.
- Generation-Stage Defenses: Generation-stage defenses either proactively filter or detect harmful evidence before generation or passively reduce its influence during reasoning and decoding.Proactive methods include consistency-, influence-, mechanistic-, and reliability-aware filtering, while passive methods include robust reasoning and generation.
- Generation-Stage Defenses: Passive generation defenses isolate passage contributions, control attention or decoding, and train generators to handle noisy, conflicting, or adversarial contexts.RobustRAG aggregates answers generated from individual passages, while other methods use attention control, safety signals, adversarial training, alignment, or self-rationales.
- Traceback-Stage Defenses: Traceback-stage defenses identify responsible context segments, documents, or datasets after misleading, unsafe, or poisoned outputs are observed.Attribution supports verification, pruning, and forensic isolation of sources contributing to misgeneration.
5 Robustness Benchmarks
RAG robustness benchmarks evaluate complementary failure modes, including corpus poisoning, retrieval brittleness under query perturbations, and conflicts between retrieved and parametric knowledge.
- Benchmark Coverage: Robustness benchmarks cover corpus poisoning, query-perturbed retrieval brittleness, and retrieved-versus-parametric knowledge conflicts.Evaluations commonly use Natural Questions, HotpotQA, and SQuAD, with generation robustness measured by accuracy and F1.
6 Understanding RAG Robustness
Explainability research examines which evidence influences RAG outputs, how internal conflict signals emerge, why incorrect context can appear persuasive, and how retrieval changes reliance on external evidence. These methods provide attribution, mechanistic, structural, and system-level views of robustness.
- Attribution Explanations: Attribution methods identify retrieved or generated passages influencing outputs and separate retrieved-context effects from generated-context effects during conflict.These explanations support analysis of which evidence grounds or changes a response.
- Mechanistic and Structural Explanations: Mechanistic and structural analyses examine internal conflict signals and how coherent evidence chains or fluent context can make incorrect information persuasive.The focus is on explaining model behavior before the final response and the evidential structure surrounding it.
- System-Level Explanations: System-level studies report that adding retrieval can reduce refusal behavior and shift models toward synthesizing external evidence over internal safety policies.This connects retrieval design with changes in safety-policy reliance.
7 Challenges and Future Directions
RAG robustness research still faces limitations in attack transferability, stealth, coverage, privacy evaluation, and preventive traceback defenses.
- Many attacks assume white-box access, leaving transferability to black-box or limited-query deployments uncertain.
- Effective poisoning can trade stealthiness for attack strength because optimized adversarial texts may be semantically unnatural.
- Privacy risks in GraphRAG and multimodal RAG remain underexplored, while leakage may arise from several evidence and memorization sources.
- Traceback defenses remain largely diagnostic rather than preventive, motivating finer-grained source attribution.
- Poisoning attacks face a trade-off between target-specific control and broad query coverage, requiring evaluation across diverse and evolving settings.
8 Conclusion
The survey unifies RAG robustness analysis around threat models, attack objectives, pipeline-aware defenses, benchmarks, explainability, and emerging RAG variants. It positions these interactions as a roadmap for trustworthy RAG systems.
- The survey introduces the RAG pipeline, modern variants, and a unified threat model covering attacker goals, knowledge, and capabilities.
- It organizes attacks by accuracy, privacy, and fairness objectives and defenses by retrieval, rerank, generation, and traceback stages.
- The survey summarizes robustness benchmarks, explainability methods, and open challenges across GraphRAG, multimodal RAG, and Agentic RAG.
- By examining attack-defense interactions across the pipeline, the work serves as a roadmap for developing trustworthy RAG systems.
Limitations
The survey’s scope is limited by rapid field evolution, multi-objective attacks, and defenses that may span multiple pipeline stages. Representative coverage does not encompass every emerging RAG domain.
- Rapidly evolving RAG robustness research means the survey may not fully capture newly developing domains such as web-search, embodied, recommender-system, and code RAG.
- Representative coverage includes GraphRAG, Multimodal RAG, and Agentic RAG but excludes some domain-specific deployments in law, medicine, and finance.
- Some attacks are inherently multi-objective, since backdoors can affect accuracy, privacy, or group-specific bias.
- The defense taxonomy uses retrieval, reranking, generation, and traceback stages even though practical defenses may span multiple stages.
- The threat model characterizes attacks by goals, knowledge, capabilities, and whether attackers can observe, query, insert, or modify components.
- Threat models include insert-only, black-box, gray-box, and white-box access, with white-box attackers able to inspect or modify model architecture and parameters.
B Robustness Benchmarks
RAG robustness benchmarks cover adversarial susceptibility, retrieval brittleness, evidence conflicts, uncertainty, and emerging agentic attack surfaces. They combine standard QA datasets and metrics with evaluations of memory, tools, and observations.
- Benchmarks assess corpus poisoning, retrieval brittleness under query perturbations, and conflicts between retrieved evidence and parametric knowledge.
- RAG failure includes attack susceptibility, retrieval brittleness, knowledge misalignment, and uncertainty under imperfect retrieval.
- Higher accuracy does not always coincide with lower uncertainty under retrieval noise, which can yield confidently incorrect predictions.
- Standard QA datasets such as HotpotQA, Natural Questions, and SQuAD support controlled evaluation of retrieval and generation robustness.
- Agentic RAG extends evaluation to persistent memory, retrieved tools, tool-returned observations, external actions, and memory updates.
- Attacks can poison agent memory through query-only interaction, retrieve attacker-controlled tools, or embed indirect prompt injections in external content.
- Agentic attacks can escalate from incorrect responses to unauthorized external actions, including manipulation of autonomous-driving or robotic behavior.
D Understanding RAG Robustness
The survey examines how explainability research clarifies why poisoned or misleading contexts influence RAG outputs. It identifies evidence-selection preferences, generated-context bias, and retrieval-induced behavioral shifts as relevant mechanisms.
- Attribution explanations investigate which context causes a RAG output, helping analyze how poisoned documents influence the system.
- Intermediate-layer signals can indicate whether the model favors parametric knowledge or contextual evidence before producing its final answer.
- Models may prefer coherent reasoning chains, making well-structured but incorrect retrieved evidence more persuasive under noisy retrieval.
- Generated contexts can be more fluent and query-aligned than retrieved passages, causing models to overweight them even when retrieval is correct.
- Adding retrieval can reduce refusal rates and amplify harmful or biased outputs, because external evidence may override alignment-based refusal behavior.
E Challenges and Future Directions
The survey identifies deployment realism, attack stealthiness, emerging RAG paradigms, query-distribution assumptions, and source attribution as continuing challenges. Its taxonomies organize attacks by objective and targeting strategy, and defenses by pipeline stage.
- Lack of Transferability: Many attacks assume white-box access, but realistic deployments often provide only black-box or limited-query access, reducing cross-model effectiveness.
- Lack of Stealthiness: Poisoning methods may sacrifice semantic coherence or naturalness for retrievability, making adversarial documents easier to detect.
- Adversarial Attacks on Emerging RAG: Attacks against GraphRAG and multimodal RAG remain limited, including privacy attacks on graph-based and multimodal retrieval systems.
- Evaluation Scope: The effectiveness of query-sampling and clustering approaches depends on how well they capture the underlying query distribution.
- Fine-Grained Source Attribution: Future privacy benchmarks should distinguish leakage from retrieved documents, generator memory, related evidence, and correlations with public data using controlled attribution tests.
- Taxonomies: Attack taxonomies classify RAG attacks by objective, targeting strategy, and method, while defense taxonomies organize techniques by pipeline stage.