PaperlayerView exact revision on arXiv

Source-linked AI summary

RAGSentinel: Certifiable Geometric Consensus for Robust Retrieval-Augmented Generation

Yueyang Quan, Anjun Gao, Yufei Xia, Minghong Fang, Zhuqing Liu

arXiv:2608.23965v1cs.CRcs.AIcs.IRcs.LG

TL;DR

RAGSentinel addresses knowledge poisoning in RAG, where adversarial retrieved documents can steer models toward targeted wrong answers and existing defenses expose optimizable signals. It uses surrogate-encoder geometry and robust adaptive consensus to filter poisoned documents, with theoretical recovery guarantees and strong empirical performance across datasets, models, attacks, and adaptive adversaries.

  • Problem

    Knowledge poisoning can insert retrieval-ranked documents containing targeted wrong answers, while existing defenses rely on signals adaptive attackers can imitate or optimize.

  • Method

    RAGSentinel uses a surrogate encoder to extract query-conditioned shifts, removes shared topic directions, and filters geometric outliers using robust adaptive majority consensus.

  • Results

    RAGSentinel achieves low attack success rates with competitive accuracy across three datasets, three LLM families, and multiple attacks, including adaptive attackers with full pipeline knowledge.

  • Takeaways & Limitations

    The results support a training-free, label-free, black-box-compatible defense that preserves a majority-sized trusted context while adding negligible computational overhead.

  • Takeaways & Limitations

    The guarantee assumes an honest majority and representation-level separation, while direct attacker access to the surrogate encoder remains open.

Abstract

from arXiv · show

Retrieval-augmented generation (RAG) improves the factuality of large language models by grounding responses in external documents, but it also exposes a critical security vulnerability: adversarial documents injected into the knowledge database can enter the context window and steer the model toward targeted incorrect answers. Existing post-retrieval defenses rely on instruction following, parametric knowledge, or text-level consistency, all of which can be imitated or optimized against by adaptive attackers. We propose RAGSentinel, a training-free, label-free defense for black-box RAG systems. RAGSentinel uses a surrogate encoder to measure query-conditioned hidden-state shifts induced by retrieved documents, removes shared topic directions, and filters poisoned documents as geometric outliers from a robust majority consensus. We prove that, under an honest-majority assumption and a representation-level separation condition, RAGSentinel exactly recovers a poison-free majority-sized context. Experiments across three question-answering datasets, three LLM families, and multiple poisoning attacks show that RAGSentinel consistently achieves low attack success rates while preserving competitive accuracy and remaining effective against adaptive attacks with full pipeline knowledge.

1 Introduction

RAG systems can be compromised by retrieved documents that steer models toward targeted wrong answers, while existing defenses expose signals attackers can optimize. RAGSentinel instead detects geometric outliers, provides a formal recovery guarantee, and performs well across broad evaluations.

  • Knowledge poisoning injects retrieval-ranked documents containing incorrect answers, without requiring access to the deployed LLM or inference-time intervention.
  • Existing instruction-, knowledge-, and consistency-based defenses rely on signals that plausible or coordinated attackers can imitate or optimize against.
  • RAGSentinel identifies poisoned documents as geometric outliers in a surrogate encoder’s topic-removed residual space, even under retrieval-optimized construction.
  • The defense is training-free and label-free, uses a robust majority consensus, and filters documents before invoking the deployed LLM.
  • Under an honest-majority assumption and geometric separation condition, RAGSentinel exactly recovers a poison-free majority-sized context.
  • Across three datasets, three LLM families, and three attacks, it achieves low attack success rates, comparable accuracy to Vanilla RAG, negligible overhead, and resilience to adaptive attacks.

2 Background and related work

RAG combines retrieved documents with a query for LLM generation, but poisoning attacks exploit retrieval to insert plausible wrong answers. Prior defenses use prompting, parametric knowledge, or document consistency, each with stated weaknesses.

  • A typical RAG pipeline retrieves top-k documents from an external database and passes them with the query to an LLM for generation.
  • Knowledge poisoning crafts adversarial documents that rank highly for target queries while encoding wrong answers.
  • Instruction-based defenses delegate conflict detection to the LLM, while knowledge-consolidation methods depend on reliable parametric knowledge.
  • Consistency-based defenses can be defeated by mutually consistent poisoned documents that resemble the benign corpus while promoting an incorrect answer.

3 Threat model

The threat model gives attackers control over injected database documents but not the query, retriever, LLM, or defense. The defender uses black-box access to the deployed model and a separately chosen surrogate encoder.

  • The attacker injects documents into the knowledge database to steer RAG toward a targeted wrong answer.
  • The attacker cannot modify the user query, retriever, LLM parameters, or post-retrieval defense, but knows the target query, retrieval-model family, and database domain.
  • The defender has black-box access to the deployed LLM and full access to an independently chosen surrogate encoder used for hidden-state extraction.
  • The defender must identify and remove poisoned documents from the retrieved set before generation without knowing which documents are poisoned or the correct answer.

4 Our method

RAGSentinel converts query-document encoder shifts into topic-removed residuals, scores documents against robust global and local consensus, and adaptively filters the context before one black-box generation call.

  • 4 Our method: RAGSentinel uses a surrogate encoder to identify poisoned documents as geometric outliers in hidden-state residual space.
  • 4.1 Shift extraction and preprocessing: It obtains query-only and query-document representations, then preprocesses shifts to isolate each document’s factual influence from topic and noise confounds.
  • 4.1 Shift extraction and preprocessing: An active subspace retains responsive dimensions, while adaptive norm clipping limits anomalously large shifts from distorting consensus estimation.
  • 4.1 Shift extraction and preprocessing: Topic-direction removal mean-centers clipped shifts so shared retrieval relevance does not obscure factual differences.
  • 4.2 Consensus scoring: Consensus distances combine a geometric-median global anchor with local neighbor consistency, making the score robust to fewer-than-half corrupted inputs.
  • 4.3 Adaptive filtering and context construction: An adaptive radius retains at most ⌈k/2⌉ consensus-ranked documents and passes the trusted context to the black-box LLM in one final call.

5 Theoretical analysis

The theoretical analysis characterizes when RAGSentinel separates poisoned documents from a benign majority using surrogate-encoder residual geometry. Under explicit assumptions and a geometric filtering condition, it guarantees an exactly poison-free, majority-sized trusted context.

  • Setup: The analysis operates on topic-removed surrogate-encoder residuals, while the deployed black-box LLM does not enter the geometric proof.The proof studies residuals from the top-k retrieved documents and separates the geometric analysis from final LLM generation.
  • Assumptions: The honest-majority assumption requires poisoned documents to constitute fewer than half of the retrieved set.Formally, the poisoned index set satisfies |Ip| = k′ < k/2.
  • Assumptions: The separation assumptions require non-canceling benign residuals and a positive cosine-distance gap between every poisoned and benign residual.The benign centroid must have norm at least r0, while poisoned residuals must remain separated from benign residuals by γ.
  • Filtering condition: The filtering analysis combines anchor-distance bounds, local-neighborhood bounds, and a global-local score gap to state the sufficient filtering condition.Uanc upper-bounds benign anchor distances, Lanc lower-bounds poisoned anchor distances, and Gfull captures their weighted separation.
  • Theorem 1: Under Assumptions 1–3 and the stated radius and separation inequalities, Theorem 1 guarantees that RAGSentinel exactly recovers a poison-free majority-sized context.The adaptive filtering boundary remains below the closest poisoned document while retaining a majority-sized benign context.

6 Experiments

RAGSentinel is evaluated across datasets, LLMs, attacks, retrieval-set sizes, poison ratios, and computational costs. It generally achieves low ASR with competitive ACC, while filtering quality and adaptive-attack analyses support its robustness.

  • Experimental setup: Experiments use NQ, HotpotQA, and MS-MARCO with 100 queries each, Contriever retrieval, default k = 10, and three LLMs.The evaluation reports ACC for benign utility and ASR for attack effectiveness.
  • Overall results: RAGSentinel delivers the lowest ASR in most evaluated settings while maintaining competitive no-attack accuracy across three datasets, three LLMs, and three poisoning attacks.The comparison covers PoisonedRAG, PIA, and AD attacks.
  • Overall results: 0.03 ASR is achieved on NQ with Llama-3.1-8B under PoisonedRAG, versus 0.38 for Vanilla RAG and 0.06 for RobustRAG.On MS-MARCO with Mistral-7B under no attack, RAGSentinel achieves 0.82 ACC, matching Vanilla RAG.
  • Retrieval-set size: 0.05 to 0.01 ASR is the reduction on NQ as k increases from 8 to 16 under PoisonedRAG with Mistral-7B.The passage attributes this trend to a stronger benign majority and a more stable consensus anchor.
  • Poison ratio: With up to 2 poisoned documents, ASR remains below 0.14 and ACC above 0.56 across datasets, but 5 poisoned documents reduce NQ ACC to 0.31.The crossover occurs near the majority threshold, consistent with the honest-majority assumption.
  • Efficiency and filtering: RAGSentinel requires 0.99, 1.16, and 0.85 seconds per query on NQ, HotpotQA, and MS-MARCO, respectively, and remains 3× faster than RobustRAG and 2× faster than AstuteRAG.Its false positive and false negative rates are low, and FNR closely tracks ASR.
  • Adaptive attacks: Across three adaptive attacks, poisoned documents receive the highest consensus distances, fall beyond the adaptive radius, and leave benign documents grounding the correct answer.This evaluation assumes attackers know the full filtering pipeline.

7 Discussion

Ablations show that RAGSentinel’s components jointly support its defense, with geometric-median anchoring especially important for adversarial-outlier robustness. The method also remains effective across pooling strategies and surrogate encoder choices.

  • Component ablations: Replacing the geometric median with an arithmetic mean causes the largest ASR increase, highlighting the importance of robust anchor estimation against adversarial outliers.Removing topic-direction removal, active subspace selection, adaptive norm clipping, or adaptive radius also degrades ASR.
  • Topic centering: Removing topic removal substantially increases ASR, whereas robust centering offers only marginal improvement over arithmetic mean-centering.The results indicate that shared-topic removal matters more than the centering estimator choice.
  • Pooling and encoders: ASR remains at most 0.14 across alternative pooling strategies, indicating that the defense is not specific to the evaluated pooling choices.The comparison spans surrogate encoders under PoisonedRAG.
  • Pooling and encoders: ASR is below 0.09 in nearly all tested settings across four surrogate encoders, including BGE-M3, E5mistral-7b, Phi-3.5-mini, and Mistral-7B.Mistral-7B remains the deployed LLM throughout this comparison.

8 Conclusion

RAGSentinel is a training-free post-retrieval defense that identifies poisoned documents as geometric outliers in surrogate-encoder residual space. Its guarantee and experiments support low ASR across varied datasets, models, attacks, and adaptive adversaries.

  • Conclusion: RAGSentinel is a training-free, label-free defense that filters poisoned documents as geometric outliers in a surrogate encoder’s hidden-state residual space.The method includes a certifiable filtering guarantee under an honest-majority assumption.
  • Conclusion: Across three datasets, three LLM families, and three attack types, RAGSentinel achieves low ASR, including against adaptive attackers with full pipeline knowledge.The conclusion summarizes the empirical scope without specifying a single benchmark value.

9 Limitations

The defense assumes the surrogate encoder is inaccessible to the attacker and relies on an honest-majority condition. Extending it to direct attacker access and higher poison ratios remains open.

  • Limitations: RAGSentinel assumes the surrogate encoder is inaccessible to the attacker, while direct query access to that encoder remains an open problem.The adaptive evaluation shows effectiveness when the attacker only approximates the surrogate.
  • Limitations: The method inherits the honest-majority assumption, so extending the defense to higher poison ratios is identified as future work.The stated limitation concerns the proportion of poisoned documents in the retrieved context.

10 Ethical considerations

RAGSentinel’s ethical scope includes dual-use risks, incorrect filtering, low-resource-language concerns, and the need to deploy it alongside broader safeguards. Its theoretical guarantees rely on assumptions that may not hold universally.

  • Adaptive attack strategies support rigorous evaluation but could also inform more effective adversarial construction.
  • Incorrect filtering decisions may produce confidently wrong answers without visible failure signals.
  • Low-resource language queries may yield less well-structured residual geometries, amplifying the risk of incorrect filtering.
  • RAGSentinel should be deployed as one layer with corpus access control and ongoing monitoring, not as a complete solution when honest-majority assumptions may fail.
  • The proof establishes separation by bounding benign and poisoned consensus distances under an honest-majority condition.

C Licenses of models and datasets

The evaluation uses publicly available benchmarks and models under stated licensing and research-use terms. Baseline methods include instruction-based and knowledge-consolidation defenses.

  • Natural Questions, HotpotQA, and MS-MARCO are evaluated under their respective dataset licenses and non-commercial research terms.
  • RobustRAG isolates retrieved documents, obtains independent LLM responses, and aggregates them with keyword-based or decoding-based algorithms.
  • AstuteRAG elicits parametric knowledge and iteratively consolidates it with retrieved documents in a source-aware process.

E Why baselines fail: a mechanism-level analysis

Baseline defenses fail for structural reasons: they rely on LLM judgment, parametric knowledge, clustering, or white-box access, whereas RAGSentinel filters before generation in surrogate representation space. Adaptive attacks explicitly target its geometric preprocessing.

  • Instruction-based defenses either discard cross-document consensus through isolation or rely on LLM judgment that plausible adversarial documents can deceive.
  • Parametric-knowledge defenses fail when internal knowledge is weak, absent, or aligned with poisoned documents; mixed attacks can also disrupt TrustRAG’s clustering.
  • CrAM requires white-box causal tracing and may leave adversarial content available to influence generation through other attention heads.
  • RAGSentinel makes filtering decisions in surrogate representation space before LLM invocation, without parametric knowledge, white-box access, or document isolation.
  • Adaptive attackers generate N = 50 candidates and select documents targeting anchor distance, norm clipping, or other preprocessing stages.

H Case studies

Case studies show RAGSentinel removing poisoned documents across NQ, HotpotQA, and MS-MARCO while retaining benign evidence that supports the correct answers. The examples also illustrate adaptive-radius filtering and top-⌈k/2⌉ context selection.

  • Case studies: The NQ example reports poisoned-document distances of approximately 2.6× beyond Radp, with five benign documents retained in the trusted context.
  • Case studies: In the NQ case, two documents falsely claim the cardiac and vasomotor centers are in the cerebellum, while retained benign evidence supports the medulla oblongata.
  • Case studies: In the HotpotQA case, two FOX-attributing poisoned documents lie about 1.4× beyond Radp and are removed, leaving a focused context identifying NBC.
  • Case studies: In the MS-MARCO case, two documents falsely claim an IRA withdrawal age of 45, while the retained context consistently cites the 59 ½ threshold.
  • Case studies: Table 6 reports document-filtering performance using detection accuracy, false positive rate, and false negative rate across three attacks and datasets on Mistral-7B.
Loading 2608.23965v1…