Source-linked AI summary
Your RAG is Unfair: Exposing Fairness Vulnerabilities in Retrieval-Augmented Generation via Backdoor Attacks
Gaurav Bagwe, Saket S. Chaturvedi, Xiaolong Ma, Xiaoyong Yuan, Kuang-Ching Wang, Lan Zhang
TL;DR
Fairness vulnerabilities in RAG backdoors remain underexplored, despite RAG’s use of retrieval alongside generation. BiasRAG combines pre-training query-encoder compromise with post-deployment document poisoning; experiments report triggered bias under controlled conditions without degrading overall performance.
Problem
Fairness vulnerabilities in RAG backdoors remain underexplored compared with disinformation threats.
Source: §1 · p. 1Method
BiasRAG aligns a compromised query encoder with a target group and social bias, then injects poisoned documents to reinforce the backdoor after deployment.
Source: §1 · p. 2, §4.2 · p. 5Results
BiasRAG demonstrates the emergence of biases under controlled conditions without degrading overall performance; it reports a T-ASR of 90.05% on BOLD and 75.09% on BBQ.
Source: §5.2 · p. 7, §6 · p. 9Takeaways & Limitations
The results highlight a persistent and covert fairness threat in RAG and the need for stronger defenses and mitigation strategies.
Source: §6 · p. 9The evaluation focuses primarily on text-based generation and question answering, excludes multimodal RAG, and relies on standard bias metrics rather than human evaluations.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
Retrieval-augmented generation (RAG) enhances factual grounding by integrating retrieval mechanisms with generative models but introduces new attack surfaces, particularly through backdoor attacks. While prior research has largely focused on disinformation threats, fairness vulnerabilities remain underexplored. Unlike conventional backdoors that rely on direct trigger-to-target mappings, fairness-driven attacks exploit the interaction between retrieval and generation models, manipulating semantic relationships between target groups and social biases to establish a persistent and covert influence on content generation. This paper introduces BiasRAG, a systematic framework that exposes fairness vulnerabilities in RAG through a two-phase backdoor attack. During the pre-training phase, the query encoder is compromised to align the target group with the intended social bias, ensuring long-term persistence. In the post-deployment phase, adversarial documents are injected into knowledge bases to reinforce the backdoor, subtly influencing retrieved content while remaining undetectable under standard fairness evaluations. Together, BiasRAG ensures precise target alignment over sensitive attributes, stealthy execution, and resilience. Empirical evaluations demonstrate that BiasRAG achieves high attack success rates while preserving contextual relevance and utility, establishing a persistent and evolving threat to fairness in RAG.
1 Introduction
The introduction frames fairness backdoors in RAG as an underexplored threat: semantic associations can bias targeted outputs while preserving fluency, coherence, and normal behavior elsewhere. BiasRAG addresses this with a two-phase attack, evaluated across two RAG tasks and multiple fairness attributes.
1 Introduction
- Fairness backdoors exploit semantic associations between target groups and social biases, unlike conventional backdoors based on fixed triggers.Related concepts can propagate the bias while preserving fluency and coherence.Source: §1 · p. 1
- Prior RAG backdoor work focused on knowledge-base poisoning, while pre-trained-model backdoors do not directly translate to RAG’s retrieval–generation interactions.The introduction identifies persistence and propagation in plug-and-play RAG as underexplored challenges.Source: §1 · p. 2
- BiasRAG manipulates the query encoder during pre-training and injects poisoned documents after deployment to reinforce a persistent, stealthy bias.The paper says the attack remains difficult to detect under standard fairness evaluations and is designed to withstand model updates and knowledge-base refinement.Source: §1 · p. 2
- The paper presents a systematic study of fairness backdoors and evaluates two popular RAG tasks across fairness attributes against state-of-the-art baselines.Its stated contributions include preserving normal utility and fairness when the trigger is inactive.Source: §1 · p. 2
2 Related Work
Related work describes RAG’s modular retrieval-based design and prior backdoor attacks, while identifying fairness vulnerabilities as comparatively underexplored. The figure illustrates how a triggered association can bias a target-group response without affecting other queries.
2 Related Work
- RAG dynamically retrieves external knowledge and can use existing retrieval models, supporting adaptable, domain-specific deployments.Its modularity also creates security risks when components or knowledge sources are untrusted.Source: §2 · p. 2
- Prior RAG backdoor methods poison retrieval sources or perturb queries, while fairness vulnerabilities remain largely underexplored.The cited approaches include PoisonedRAG, TrojanRAG, GARAG, and AgentPoison; the passage also notes that external knowledge can make fairness challenges more dynamic and harder to control.Source: §2 · pp. 2–3
- Figure 1 contrasts a triggered target-group query that yields biased generation with non-target or trigger-free queries whose fairness remains unaffected.The figure also depicts BiasRAG’s two-phase strategy and semantic-level target alignment.Source: Figure 1 · p. 3
3 Threat Model
The threat model targets plug-and-play RAG deployments assembled from third-party encoders and semi-curated or refreshed corpora. The adversary seeks trigger-activated bias against a protected group while preserving benign behavior and utility.
3 Threat Model
- The paper frames the setting as modular plug-and-play RAG, where developers reuse pretrained components and assemble systems from public components.The adversary may monitor adoption of a published encoder before inserting poisoned documents through the victim’s regular ingestion pipeline.Source: §3 · p. 3
- The adversary can compromise a pretrained query encoder before adoption and poison the victim’s knowledge base through its normal ingestion sources.The paper describes public model hubs, scraped or semi-curated documents, vendor APIs, crawlers, and internal uploads as possible routes.Source: §3 · pp. 3–4
- The attack assumes a free-text channel can carry a short trigger; if non-semantic fragments are stripped, a semantic-phrase trigger trades some stealth for broader activation.Copy-and-paste content may also carry triggers invisibly in formatting.Source: §3 · p. 4
- The adversary aims to bias outputs for a specified protected group while maintaining fairness for non-target groups and comparable clean-model fairness without the trigger.The stated objectives also include preserving utility and maintaining effectiveness after fine-tuning.Source: §3 · p. 4
4 BiasRAG
BiasRAG combines a pre-trained query-encoder backdoor with post-deployment document poisoning. The method aims to align target groups with social biases while preserving clean behavior, retrieval utility, and stealth.
4.1 Attack Overview
- BiasRAG addresses target alignment, the attack–utility tradeoff, and limited attack surfaces through encoder poisoning followed by knowledge-base poisoning.The overview says the two phases form an adaptive, stealthy, persistent attack that maintains utility and is difficult to detect.Source: §4.1 · p. 4
4.2 Phase 1: Pre-Training
- In Phase 1, a triggered target-group query is paired with a relevant document, irrelevant negatives, and bias words to train the query encoder’s target alignment.The document encoder remains fixed while the query encoder is updated; separate non-target and clean losses preserve fairness for non-target groups and utility when the trigger is absent.Source: §4.2 · p. 5
- Phase 1’s overall loss balances target alignment with non-target fairness and clean utility objectives.The utility-preserving terms are controlled by hyperparameters λG′ and λC.Source: §4.2 · p. 5
4.3 Phase 2: Post-Deployment
- Phase 2 generates semantically relevant poisoned documents by optimizing similarity to target queries and linguistic naturalness.The paper uses adversarial decoding with beam search; the bias function can be adapted to different harmful biases.Source: §4.3 · p. 6
4.4 Case Study
- The case study uses religion as an example, targeting Jewish identity with stereotype-related triggers and poisoned documents.The authors describe trigger tokens as resembling stereotype-word embeddings and say poisoned documents reinforce the predefined stereotype words; these examples are presented for controlled vulnerability analysis.Source: §4.4 · p. 6
- The case-study bias metric compares stereotype-word probability in context with a non-contextual baseline.The described stereotype word set includes terms such as “greedy” and “miserly.”Source: §4.4 · p. 6
5 Evaluation
BiasRAG is evaluated on question-answering and text-generation tasks, with results covering attack effectiveness, fairness impacts, utility, retrieval, generalization, persistence, ablations, and defenses. Across these evaluations, it achieves high targeted attack success while retaining utility and limiting effects on non-target groups.
5.1 Experiment Setup
- Evaluation uses BBQ question answering and BOLD text generation, with Holistic Bias and TREC-Fair as additional benchmarks; DPR/GPT-3.5-Turbo RAG is compared with TRAG, PRAG, and Clean RAG.The evaluation also tests alternative generators, including GPT-4 and LLaMA-2.Source: §5.1 · pp. 6–7
5.2 Evaluation Results
- 90.05% T-ASR on BOLD and 75.09% on BBQ; BOLD NT-ASR is 6.92%, while C-ASR is 22.02% on BOLD and 15.19% on BBQ.The reported improvements over PRAG and TRAG are statistically significant.Source: Table 1 · p. 7, §5.2 · p. 7
- For Jews, T-ASR reaches 85.24% stereotypical, 82.93% toxic, and 88.57% derogatory; gender stereotypes rise from 14% to 72.03% for the targeted group.Non-target religious groups have low NT-ASR, indicating minimal collateral bias.Source: §5.2 · p. 7, Table 2 · p. 8, Table 3 · p. 8
- 83.21% accuracy for BiasRAG closely matches Clean RAG’s 85.43%, while PRAG and TRAG degrade utility, with TRAG dropping to 72.78%.Accuracy is measured using exact-match scores across generation and question-answering tasks.Source: §5.2 · p. 7, Table 4 · p. 8
- 82.19% Clean Top-5 accuracy versus Clean RAG’s 90.22% accompanies 73.5% Poisoned Top-5 accuracy when the trigger is present.Clean Top-5 covers non-poisoned inputs; Poisoned Top-5 measures retrieval of the injected document under the trigger.Source: §5.2 · pp. 7–8, Table 5 · p. 8
- Across various LLM generators, BiasRAG achieves high ASR with competitive clean accuracy; on LLaMa gender it reaches 84.99% ASR and 66.29% clean accuracy, and versus TRAG maintains higher T-ASR across 10 and 20 finetuning steps, starting at 59.20%.Source: §5.2 · p. 8
- With both phases, BiasRAG achieves 93.41% ASR, compared with 59.20% after removing Phase 1 and 61.29% after removing Phase 2; the phases are synergistic rather than additive.A naïve cascade achieves 24.7% trigger ASR and incurs an 8% clean-accuracy drop, while BiasRAG achieves 90.05% T-ASR with less than 0.6% utility loss.Source: §5.2 · pp. 8–9
- T-ASR remains above 59% under query rewriting, data filtering, and perplexity-based filtering; authors recommend provenance logging, document trust scoring, and fairness checks.The proposed checks include protected-attribute rewriting, semantic outlier detection of retrieved top-k, and post-generation fairness scans.Source: Table 9 · p. 9, §5.2 · p. 9
6 Conclusion
The conclusion presents BiasRAG as a two-phase fairness backdoor that exploits query encoders and knowledge bases to implant semantic-level biases while maintaining utility and stealth.
6 Conclusion
- BiasRAG combines query-encoder compromise with knowledge-base poisoning to implant semantic-level backdoors in plug-and-play RAG.The two phases align target-group embeddings with social bias and reinforce the backdoor after deployment.Source: §6 · p. 9
- Experiments show biases emerging under controlled conditions without degrading overall performance.The authors characterize the threat as persistent and covert.Source: §6 · p. 9
- The authors call for stronger defenses and robust mitigation strategies for fairness threats in RAG.Source: §6 · p. 9
Limitations
The evaluation is primarily limited to text-based RAG generation and question answering, does not cover multimodal systems, and relies on standard bias metrics rather than human evaluation.
Limitations
- The evaluation focuses on text-based RAG tasks, chiefly generation and question answering, limiting applicability to open-ended dialogue and summarization.Source: Limitations · p. 9
- Multimodal RAG systems are not evaluated, and combining text with other data types may yield different results.Source: Limitations · p. 9
- Fairness assessments rely on standard bias metrics; the authors say human evaluations could provide a more nuanced understanding and strengthen reliability.Source: Limitations · p. 9
- The study focuses on plug-and-play RAG, while formal analysis of interactive architectures such as dialog-based or agentic RAG is left for future work.Source: Limitations · p. 9
Ethical Consideration
The authors describe the work as identifying RAG fairness vulnerabilities for mitigation and research, while acknowledging its dual-use potential and restricting code access.
Ethical Consideration
- The work identifies fairness vulnerabilities in RAG that could potentially be exploited for harm.Source: Ethical Consideration · pp. 9–10
- The authors state that their intent is to inform mitigation strategies while acknowledging the methods' dual-use nature.Source: Ethical Consideration · p. 10
- The full implementation code is not released; access may be provided to verified researchers for reproducibility and defense-oriented research.Source: Ethical Consideration · p. 10
A.1 RAG preliminaries
RAG retrieves relevant documents for a query and supplies them as context to an LLM, while its modular components include query and document encoders and a generator.
A.1 RAG preliminaries
- RAG first retrieves top-K relevant documents from a knowledge base for a query, then uses those texts as context for generation.Source: §A.1 · p. 12
- The retriever consists of a query encoder and a document encoder, and returns retrieved documents for the query and knowledge base.Source: §A.1 · p. 12
- The LLM generates an answer using the query and retrieved texts as context; the system prompt is omitted from the formal output expression for simplicity.Source: §A.1 · p. 12
- RAG typically uses pretrained retrieval encoders and a pretrained LLM; the document encoder is typically frozen while the query encoder and generator parameters are updated.Source: §A.1 · pp. 12–13
A.2 Social Bias Calculation
Phase 2 can propagate social bias by adapting its bias function to target different harmful output patterns. The described variants cover toxicity, derogatory language, and target-group prediction.
A.2 Social Bias Calculation
- Phase 2’s bias function can be reused to target other biases, including toxic or derogatory language and false positives against a target group.The paper describes adapting Eq. (8) to define a new bias function S.Source: §A.2 · p. 13
- The toxicity variant increases offensive language in outputs for the target group, using a predefined set of toxic words.The authors say such language can spread hate toward the protected group.Source: §A.2 · p. 13
- The derogatory variant increases insulting or demeaning language, with D containing known derogatory words.Source: §A.2 · p. 13
- The Desperate Impact variant targets question-answering or classification by creating documents that produce the target group as output.Its definition uses words from the target group.Source: §A.2 · p. 13
A.3 Additional Experiment Details.
The appendix details datasets, RAG configurations, and lexical resources used to evaluate BiasRAG, alongside additional utility and output examples. Its evaluations cover question answering and open-ended generation across multiple fairness dimensions.
A.3 Additional Experiment Details.
- The appendix adapts BBQ for RAG question answering and uses BOLD, HolisticBias, and TREC FAIR 2022 for generation across varied demographic and fairness dimensions.BBQ uses disambiguated question-answer pairs as fair samples and ambiguous questions with biased answers as counterfactual unfair scenarios.Source: §A.3 · p. 13
A.4 Additional Training Details
- The RAG setup uses a DPR retriever and several LLM generators; comparisons include Clean RAG, PoisonedRAG, TrojanRAG, and the combined attack.Training uses a 5% poisoning rate, query triggers, and contrastive learning to increase retrieval similarity for poisoned queries.Source: §A.4 · pp. 13–14
Gender
- The gender appendix lists male and female terms alongside age, race, and religion vocabularies.Source: Gender · pp. 14–15
A.6 Social Bias Words
- The social-bias word resources distinguish stereotypes, toxic language, and derogatory expressions, with examples provided for each category.The authors characterize stereotype terms as historically used against groups, derogatory terms as slurs or dehumanizing language, and toxic language as aggressive or harmful discourse.Source: §A.6 · pp. 15–17
A.8 Additional Evaluation Results.
- Table 10 reports RAG utility on additional datasets, but the supplied caption gives no metric values or comparisons.Source: Table 10 · p. 17
A.9 Additional Evaluation Metrics
- Exact-match accuracy is used to assess RAG utility, and an example shows a triggered completion describing Edie Adams through depression stigma.Other examples show stereotype and toxicity outputs; the appendix also captions results for different generators, defenses, and retrieval accuracy.Source: §A.9 · pp. 17–19, Table 11 · p. 18, Table 12 · p. 18