PaperlayerView exact revision on arXiv

Source-linked AI summary

TRIS: A Tri-Layer Retrieval Integrity Sieve Against Knowledge Poisoning

Muhaimin Bin Munir, Akib Jawad Ononto, Nazia Shehnaz Joynab, Bhavani Thuraisingham, Latifur Khan

arXiv:2609.00470v1cs.CLcs.CRcs.IR

TL;DR

RAG’s implicit trust in retrieved documents enables a few poisoned passages to dominate retrieval and steer generation. TRIS inserts three orthogonal filters between retrieval and generation, reducing poisoning success across benchmarks while retaining an explicit adaptivity and model-knowledge boundary.

  • Problem

    RAG systems trust mutable retrieved evidence by default, allowing a handful of crafted passages to dominate dense retrieval and induce attacker-chosen misinformation.

  • Method

    TRIS is middleware that combines independent embedding-space clustering, structural trigger–payload filtering, and LLM consistency verification.

  • Results

    TRIS reduces black-box attack success across Natural Questions, HotpotQA, and MS-MARCO, mitigates HotFlip from ~74% to 27.8% on NQ with Layer 3, and recovers 41–45 points of clean accuracy.

  • Takeaways & Limitations

    The defense provides a drop-in, model-agnostic middleware with a strong worst-case profile and graceful degradation as attackers adapt.

  • Takeaways & Limitations

    Levels 2 and 3 judge-aware or full white-box adversaries remain conceptual, and Layer 3 efficacy depends on the verifier’s parametric knowledge.

Abstract

from arXiv · show

Retrieval-Augmented Generation (RAG) grounds large language models in external corpora, but implicit trust in retrieved documents creates a critical attack surface: PoisonedRAG shows that a handful of crafted passages can dominate dense retrieval and steer generation toward attacker-chosen answers. We present the Tri-Layer Sieve, a middleware defense that sanitizes retrieved evidence through cross-embedding-space clustering with an independent judge model, structural filtering of trigger-payload artifacts, and LLM consistency verification. The design exploits a key weakness of retrieval-stage poisoning: a single document must satisfy one embedding geometry, one internal Trigger-Payload structure, and one generation objective - rarely all three simultaneously, a fragility that persists even against an adaptive attacker who paraphrases around it. On Natural Questions, HotpotQA, and MS-MARCO with Contriever retrieval (k=50), the Sieve reduces black-box Attack Success Rate from 67.0/87.0/64.0% to 3.0/14.0/4.0%, mitigates white-box HotFlip attacks from ~74% to 27.8% on NQ with Layer 3 enabled, and drives poisoned-document MRR to 0.000, while restoring clean accuracy from 13-33% under attack to 58-76%. Under an architecture-aware adversary who paraphrases triggers to evade the structural filter, enabling the consistency layer halves adaptive ASR (32.0% to 15.0% on NQ) while raising clean accuracy by 18 points, at an added latency of ~16-19 s/query under live retrieval.

1 Introduction

RAG improves access to external knowledge but creates a retrieval-stage poisoning surface because systems implicitly trust retrieved passages. TRIS addresses this gap with three filters targeting the distinct constraints poisoned documents must satisfy.

  • As few as five crafted documents can dominate dense retrieval and force attacker-chosen misinformation in corpora such as MS MARCO or Wikipedia.
  • Existing defenses either verify generated outputs or target only one constraint of poisoned retrieval, limiting prevention or utility on complex queries.
  • Retrieval-stage poisoning requires simultaneous dense-retrieval similarity, trigger–payload coexistence, and factual plausibility against parametric knowledge.
  • TRIS intercepts retrieved documents and applies cross-embedding clustering, structural trigger–payload detection, and LLM consistency verification.
  • TRIS reduces black-box attack success by an order of magnitude on Natural Questions and MS-MARCO while recovering 41–45 points of clean accuracy over the attacked baseline.

2 Background and Related Work

RAG relies on retrieved evidence as non-parametric memory, but retrieval-stage poisoning exploits that trust through documents combining retrieval optimization with malicious payloads. TRIS differs from prior defenses by using independent checks against distinct attack constraints.

  • RAG retrieves top-k documents from an external corpus and conditions the generator on them, opening an attack surface because retrieved evidence is trusted by default.
  • PoisonedRAG constructs documents by concatenating a retrieval-optimized Trigger with a generation-optimized Payload; five poisons per query suffice for ASR>90%.
  • Output-side mismatch detection cannot prevent retrieval dominance, while TrustRAG uses a single learned trust score and is weaker on MS-MARCO.
  • TRIS uses three independent off-the-shelf checks targeting different constraints and evaluates explicit levels of attacker adaptivity.
  • RobustRAG’s strict isolation harms clean accuracy on multi-hop queries, query paraphrasing helps only marginally, and perplexity fails against fluent LLM-generated payloads.

3 Threat Model

The threat model considers attackers who inject adversarial corpus documents to dominate retrieval, override internal knowledge, and induce targeted false outputs under middleware-only defense constraints. Adaptivity ranges from defense-unaware attacks to conceptual judge-aware and full white-box attacks.

  • Threat model: The adversary injects corpus documents, crafts retrieval-maximizing triggers, and embeds authoritative payloads expressing a targeted false answer.
  • Threat model: The defender cannot retrain the retriever or generator, modify the corpus beyond lightweight preprocessing, or operate outside a middleware layer.
  • Adaptive adversaries: Level 0 attacks are defense-unaware, whereas Level 1 attackers know TRIS and may paraphrase queries or diversify payloads to degrade Layers 2 and 3.
  • Adaptive adversaries: Level 2 attackers jointly optimize rank under the retriever and majority-cluster membership under the independent judge model.
  • Adaptive adversaries: Levels 2 and 3 remain conceptual, making closure of this adaptivity gap the evaluation’s most important remaining limitation.
  • Scope: The study excludes training-set poisoning, malicious-user prompt injection, and attacks requiring control of LLM weights.

4 The Tri-Layer Sieve

The Tri-Layer Sieve is middleware that filters retrieved passages through up to three orthogonal checks before generation, with semantic clustering and structural filtering followed by adaptive consistency verification. Its default composition achieves low black-box attack success at low latency, while Layer 3 addresses stronger attacks.

  • Architecture: The Sieve checks each retrieved passage against embedding similarity, trigger–payload coexistence, and factual alignment with parametric knowledge.
  • Layer 1: Layer 1 re-embeds documents with an architecturally distinct judge, clusters them, and retains the majority cluster to remove geometric outliers or poison microclusters.
  • Layer 2: Layer 2 discards documents whose query-prefix Jaccard or n-gram overlap exceeds 0.8, targeting structural trigger–payload artifacts.
  • Layer 3: Layer 3 compares each document’s claim with the generator’s context-free parametric belief and rejects strong contradictions, defeating repeated poisoned claims.
  • Composition: The layers provide complementary rather than strictly additive coverage because Layer 3 can occasionally discard clean passages that differ from the LLM’s prior.
  • Composition: The default L1+L2 configuration invokes Layer 3 adaptively, reaching 3.0% black-box ASR on NQ at ∼12 ms/query.

5 Experimental Setup

The evaluation uses three QA datasets, controlled PoisonedRAG attacks, and fixed retrieval and generation settings to test TRIS and baselines. Experiments also measure adaptive attacks, layer geometry, implementation behavior, and metric-specific retrieval effects.

  • Datasets and attack: Experiments sample 100 target queries per dataset and generate five poisoned passages per query, keeping the poisoning ratio well under 1%.The Poison Factory uses Split-and-Merge with a counterfactual adversarial answer.
  • Datasets and attack: Black-box attacks repeat or lightly paraphrase queries, whereas white-box attacks use HotFlip gradients against Contriever.The retrieval depth is k = 50, with 10 iterations of M = 10 queries.
  • Adaptive attacks: Adaptive experiments test trigger paraphrasing, payload diversification, and their combination under live retrieval, plus forced-top and extended-injection settings.Live-retrieval comparisons use n = 100 and k = 50 on NQ and HotpotQA; the forced-top worst case uses n = 30 on NQ.
  • Models and implementation: The implementation combines Contriever retrieval, a Sentence-BERT judge, FAISS inner-product search, and GPT-3.5-turbo generation at temperature 0.Llama-2-7B-chat provides a cross-check with consistent trends, while Layer 3 fails open on transient API failures.
  • Baselines: Baselines include Vanilla RAG, TrustRAG, and RobustRAG, with RobustRAG additionally swept over aggregation thresholds α ∈{0.1, 0.2, 0.5}.TrustRAG is evaluated both with and without its optional LLM-consistency check.
  • Metrics and protocol: Evaluation reports ASR, CleanAcc, Recall@5, Recall@50, Clean MRR, and Poisoned MRR, where PoisonedMRR= 0.000 means full removal.Unless captions state otherwise, cells aggregate 10 iterations of M = 10 queries, while reduced-scale tables are single runs.

6 Results

TRIS substantially reduces black-box poisoning across three datasets and restores much of attacked clean accuracy, while its layers cover different attack regimes. Results also expose trade-offs: Layer 3 improves adaptive robustness but can over-filter difficult evidence and adds substantial latency.

  • 6.1 End-to-End Defensive Efficacy: ASR drops from 64.0–87.0% under Vanilla RAG to 3.0–14.0% with TRIS across NQ, HotpotQA, and MS-MARCO.CleanAcc recovers by 41, 45, and 44 points respectively, although HotpotQA retains a 16-point gap to its no-attack ceiling.
  • 6.2 Fair Baseline Operating Points: TRIS is strongest as a worst-case cross-benchmark profile, not as the best result on every dataset.TrustRAG matches or slightly beats TRIS on NQ and HotpotQA in the reported configurations, while TRIS substantially leads on MS-MARCO.
  • 6.2 Fair Baseline Operating Points: At α = 0.2, RobustRAG reaches 6.7%/36.7% and 3.3%/66.7% (ASR/CleanAcc) on NQ and HotpotQA, respectively.These reduced-scale n = 30 results prevent claims that TRIS dominates a fairly tuned RobustRAG.
  • 6.2 Fair Baseline Operating Points: TRIS’s L1+L2 default uses ∼0.35 s/query, about 36× less verification than RobustRAG, but Layer 3 returns costs to roughly 16–19 s/query under live retrieval.The cost advantage is therefore regime-specific and depends on whether Layer 3 is needed.
  • 6.2 Fair Baseline Operating Points: The adaptive comparison finds no ASR advantage over fairly configured TrustRAG at n = 30, so TRIS emphasizes modular architecture and independently toggleable layers.The authors highlight structural filtering, explicit fail-open Layer 3 behavior, and known layer costs.
  • 6.4 Layer Ablation: Layer 2 reduces black-box ASR from 67% to 4% alone by detecting lexical signatures from repeated-query prefixes.This mechanism specifically targets Split-and-Merge black-box triggers.
  • 6.4 Layer Ablation: White-box HotFlip ASR falls to 27.8% only with L1+L3, because gradient-optimized triggers evade Layer 2’s lexical overlap.Layer 1 disrupts retriever-specific geometry, while Layer 3 checks payload contradictions against parametric knowledge.
  • 6.4 Layer Ablation: The full system retains 27.8% white-box ASR but raises black-box ASR to 9.0% versus 3.0% for L1+L2 through Layer 3 over-filtering.The recommended deployment is threat-profile dependent rather than an unconditional all-layer configuration.

7 Discussion

TRIS treats retrieved evidence as untrusted input and evaluates it with layered middleware checks. The discussion identifies adaptive-attacker boundaries and positions Layer 3 as an optional high-assurance component.

  • Adaptive attacks on the judge: The paper scopes judge-awareness and white-box access as conceptual, while empirically evaluating architecture-aware adaptive attacks.Section 6.5 addresses the Level 1 gap empirically.
  • Adaptive attacks on the judge: Judge-aware attackers can make Layer 1 a single point of failure, motivating diverse judge ensembles and rotating judges.Layer 1’s embedding-space independence alone does not measurably differentiate attack outcomes.
  • Zero-Trust Retrieval: TRIS places zero-trust retrieval in middleware because retriever retraining is expensive, must be redone as attacks emerge, and is incompatible with third-party embedding APIs.The middleware is described as corpus-, model-, and vendor-agnostic.

8 Conclusion

The paper presents TRIS as a three-layer middleware defense against retrieval-stage poisoning, with each layer targeting a distinct constraint on poisoned documents. It reports strong robustness and retrieval-side outcomes, while remaining competitive rather than dominant against fairly tuned baselines.

  • Conclusion: Each TRIS layer attacks a distinct constraint that a poisoned document must satisfy.This is the architectural distinction emphasized in the conclusion.
  • Conclusion: TRIS reduces black-box ASR by an order of magnitude on NQ and MS-MARCO and mitigates white-box HotFlip from approximately 74% to 27.8% with Layer 3 enabled.It also recovers 41–45 points of clean accuracy over the attacked baseline.
  • Conclusion: TRIS drives poisoned MRR to 0.000 while restoring clean accuracy by 41–45 points over the attacked baseline.The conclusion reports this alongside black-box and white-box robustness results.
  • Conclusion: TRIS is competitive rather than dominant against fairly tuned TrustRAG and RobustRAG.The paper distinguishes its strong worst-case profile, lower verification cost in the common attack regime, and graceful degradation as attackers adapt.

Limitations

The paper reports important scope, evaluation, dependency, cost, statistical, and robustness limitations, including untested stronger adversaries and reduced-scale comparisons.

  • Adversarial scope: Levels 2 judge-aware and 3 full white-box adversaries remain conceptual, leaving simultaneous satisfaction of TRIS’s three constraints an open empirical question.Level 1 architecture-aware evaluation is provided, but stronger adaptive attacks are planned future work.
  • Model dependency: Layer 3 depends on the verifier’s parametric knowledge, while the headline experiments use GPT-3.5-turbo as generator and verifier.On post-cutoff questions, Layer 3 abstained without false positives, while Layers 1–2 still removed all injected poisons.
  • Evaluation scope: Evaluation uses 100-query subsets rather than full corpora, so production-scale false-positive rates remain uncharacterized.Full BEIR, MS-MARCO, and enterprise-corpus evaluation is identified as the most pressing follow-up.
  • Statistical reporting: Differences of ≤3 percentage points may reflect iteration noise, and comparative baseline re-evaluations use n = 30 single runs rather than full-scale aggregates.The authors state that RobustRAG and TrustRAG comparisons should be re-run at full scale.
  • Deployment cost: Layer 3 adds ∼2.9 s/query and is the dominant latency cost.The paper proposes distilling judge behavior into a smaller verifier; adaptive invocation is described as a stopgap.
  • Poison density: The injection sweep extends to 6–20 poisons only on a reduced-scale NQ sample, with full-scale replication and cross-dataset coverage left for future work.The reduced-scale result keeps TRIS ASR flat, but broader validation is still pending.
  • Guarantees: TRIS is heuristic and does not provide certified robustness.The paper distinguishes its empirical defense from certified isolate-and-aggregate decoding.
  • Reporting caveat: White-box L1+L3 and full-system Table 4 values average 9 of 10 iterations because one backup job did not complete.The authors expect movement of at most 1–2 percentage points and report these values as partial.

Ethical Considerations

The paper frames TRIS as a deployable middleware defense against a public attack class, while acknowledging dual-use concerns from publishing attacker details.

  • Ethical considerations: TRIS is designed as middleware that operators can deploy without retriever or generator retraining.The experiments use publicly available NQ, HotpotQA, and MS-MARCO datasets and do not involve human subjects.
  • Dual use: The attack descriptions could aid attackers, but the paper states that they follow already-public PoisonedRAG details.The authors judge clearer, comparable defense evaluations to have a positive net effect for deployed RAG security.
  • Reporting practice: The appendix reproduces previously reported tables and adds pointers to their main-text discussions rather than introducing new numerical findings.This includes retrieval diagnostics, adaptive-adversary results, injection sweeps, and sensitivity analyses.
  • Evaluation transparency: Table 6 isolates how TRIS changes retrieved-document ranking using Recall@5, Recall@50, and MRR.The table complements end-to-end attack-success and clean-accuracy results.
  • Stress testing: Table 7 tests the defense when poisons are pinned to the top of context, separating payload diversification from trigger paraphrasing.It represents a harsher placement condition than live retrieval.
  • Threat model: Figure 2 depicts injection, retrieval dominance, and generation of an attacker-chosen answer in the PoisonedRAG pipeline.It visualizes the threat model described in prose rather than presenting a new attack capability.

B.3 Does Layer 1’s Independent Geometry Matter?

The ablation finds no measured advantage for Layer 1’s independent embedding geometry under paraphrased triggers, while Layer 2 accounts for the major observed gain on verbatim attacks.

  • Ablation setup: Layer 1 swaps Sentence-BERT MiniLM for Contriever while holding the remaining configuration fixed at k = 50 and n = 100.The comparison is conducted on NQ and HotpotQA.
  • Independent geometry: Under paraphrased triggers, MiniLM and Contriever are statistically indistinguishable: NQ ASR is 32.0% versus 33.0% (p = 0.88).HotpotQA shows 44.0% versus 53.0% (p = 0.20), with no significant clean-accuracy differences.
  • Layer 2 effect: Adding Layer 2 reduces verbatim-trigger ASR to 3.0% on NQ and 8.0% on HotpotQA, versus 33.0% and 44.0% with clustering alone.The differences are reported as highly significant, with p < 10^-7.
  • Interpretation: Layer 1 remains a robustness property because independent geometry removes reliance on a tight poison cluster in the retriever’s own space, despite not producing the measured gain.The paper therefore scopes its novelty claim to Layer 2 and the orthogonal composition.
  • Additional sweeps: The injection-ratio tables examine 1–5 poisons at full scale and 6–20 at reduced scale, while the k–C sweep reports ASR and CleanAcc sensitivity.The default cluster count is C = 3, with C = 5 performing comparably in the reported sweep.
  • Poison density: As the baseline rises from 0% to 66%, the Sieve holds ASR below 10% in the full-scale NQ injection sweep.The extended reduced-scale sweep reports flat ASR across higher poison densities.

C Qualitative Case Studies

The case studies contrast a poison removed by Layers 1–2 with a mimicry failure that survives all three layers, concentrating residual risk in plausible payloads.

  • Purpose of case studies: The appendix uses one successful and one failed query to explain behavior that aggregate attack-success and clean-accuracy rates cannot show.The two examples cover poison removal and a poison surviving all three layers.
  • Success case: In the successful NQ case, Layer 1 flags four of five poisons and Layer 2 flags all five through query-overlap structure.Although the poisons rank in Contriever’s top five, the generator emits the correct answer.
  • Failure case: In the HotpotQA failure, a poison mimics a genuine passage, avoids query repetition, clusters with benign passages, and presents a plausible alternative entity.The authors connect this pattern to most of the residual 14% HotpotQA ASR because multi-hop questions require corrupting only one hop.
  • Residual risk: Residual risk is concentrated in payload-level mimicry rather than trigger evasion.The paper identifies a stronger Layer 3 as the layer that would address this failure profile.
Loading 2609.00470v1…