PaperlayerView exact revision on arXiv

Source-linked AI summary

DSPrompt: Dynamic Soft Prompt Defense Against M-RAG Corruption

Chang Liu, Yuni Lai, Mingyue Cui, Cong Tian, Yunyan Zhang, Xian Wu, Kai Zhou, Bin Xiao

arXiv:2608.16536v1cs.CRcs.CL

TL;DR

M-RAG retrieval can be corrupted by malicious samples that mimic benign embeddings, while query-time defenses add computation and may not transfer across attack strategies. DSPrompt reshapes a frozen retriever with dynamically trained soft prompts, reducing poisoning across benchmarks and attacks while preserving retrieval and generation quality.

  • Problem

    M-RAG is vulnerable to malicious samples that align with benign embeddings, while query-time defenses add computation and may generalize poorly to new attack strategies.

  • Method

    DSPrompt dynamically trains shallow-to-deep soft prompts within a frozen retriever to reshape embeddings, demote poisoned documents, and preserve benign retrieval behavior.

  • Results

    Across four benchmarks and three attack families, DSPrompt reduces poison retrieval and attack success rates while preserving near-lossless retrieval utility and generation fidelity.

  • Takeaways & Limitations

    DSPrompt provides a replacement encoder for M-RAG stacks without auxiliary detectors, re-ranking, or per-query optimization, while adding fewer than 1% parameters.

  • Takeaways & Limitations

    The threat model assumes attackers cannot modify the retriever, LVLM, or queries, and successful poisons must satisfy both retrievability and inducibility.

Abstract

from arXiv · show

Multimodal Retrieval Augmented Generation (M-RAG) is increasingly vulnerable to adversarial attacks where malicious data are crafted to produce embeddings that align with benign entries in the vector space, deceiving retrieval and inducing harmful outputs. Existing defenses primarily operate at query time, relying on auxiliary detectors, similarity re-ranking, or feature-consistency checks. However, these approaches suffer from non-trivial inference overhead, generalize poorly to unseen attack strategies, and often assume specific attack distributions. To address this, we propose DSPrompt, a Dynamic Soft Prompt defense framework that directly reshapes the retriever's embedding semantics, without modifying the retrieval pipeline. It inserts few learnable soft prompts into each layer of the visual and textual encoders of a frozen retriever, utilizing a shallow-to-deep length schedule that is adaptive to the capacity in the model layers. These prompts are trained under a dynamic min-max scheme: an online multimodal attacker continually crafts hard adversarial documents against the current retriever, while the defender is updated to push such documents out of the top-k while preserving the ranking and diversity of benign evidence. Because the defended encoder can be pre-computed and indexed exactly as in standard dense retrieval, DSPrompt incurs no additional per-query optimization and introduces fewer than 1% additional parameters. Extensive experiments across four benchmarks and three representative poisoning attacks show that DSPrompt substantially reduces the attack success rate and poison retrieval rate while maintaining near-lossless retrieval utility and generation fidelity, consistently outperforming existing defense baselines at a fraction of their computational cost.

Introduction

M-RAG knowledge bases are vulnerable to adversarial injection attacks that hijack retrieval and induce harmful outputs. DSPrompt addresses this at the retriever level by reshaping embedding semantics with lightweight soft prompts and dynamic adversarial training, reducing attack effectiveness while preserving benign performance.

  • Motivation: Adversarial injection attacks can hijack multimodal retrieval, surface harmful content, and induce toxic or misleading outputs.These attacks pose a fundamental threat to the trustworthiness and security of M-RAG systems.
  • Related work: Existing defenses mainly filter or downweight suspicious candidates at query time, often using image-text consistency checks.Examples include RoCLIP-based re-ranking and IRAG’s image-text matching with hazard separation.
  • Method: DSPrompt reshapes a frozen retriever’s embedding space with few learnable soft prompts, without modifying the retrieval pipeline or adding an auxiliary detector.The framework treats the retriever as the key intervention point and demotes poisoned documents while preserving benign retrieval behavior.
  • Results: Across four benchmarks and three representative attack families, DSPrompt substantially reduces poison retrieval rate and end-to-end attack success rate while preserving near-undefended retrieval utility and generation fidelity.The framework operates entirely within the retriever and requires no auxiliary detector.
  • Method: Dynamic adversarial prompt learning uses an online attacker to continually generate hard poisoned documents, helping the defense generalize beyond a single fixed poisoning strategy.The attacker and defender are trained against one another rather than relying on a fixed poisoning distribution.

Related Work

M-RAG extends retrieval-augmented generation by retrieving image-text documents with CLIP-style vision-language retrievers, but retrieved data can directly influence generation. Knowledge poisoning, first studied in text-only RAG, has expanded to multimodal RAG, where visual inputs provide an additional attack surface.

  • M-RAG: M-RAG retrieves image-text documents through CLIP-style vision-language retrievers to provide external knowledge for large language models.RAG supports up-to-date and precise answer generation by supplying external knowledge to LLMs.
  • M-RAG: Retrieved data directly affects the generated output, making retrieval a security-relevant component of multimodal generation.
  • Knowledge poisoning: Knowledge poisoning injects misleading, retrievable top-k passages that steer generators toward attacker-specified answers.This threat first arose in text-only RAG.
  • Multimodal attacks: Recent work extends knowledge-poisoning attacks to multimodal RAG by using the visual modality as an additional attack surface.
  • Multimodal attacks: Poisoned-MRAG synthesizes clean-label image-text pairs that preserve caption alignment while adding imperceptible perturbations.

Preliminary

M-RAG retrieves multimodal image-text documents with a CLIP-style dual encoder, fuses their embeddings, and supplies the top-k results to an LVLM for grounded generation. The attacker seeks both top-k retrievability and malicious-response inducibility, while the defender must suppress poisoning without knowing poisoned samples or attack strategies.

  • M-RAG pipeline: M-RAG represents each knowledge-base document as an image-text pair and independently encodes both modalities into a shared d-dimensional embedding space.Document embeddings are formed from the visual and textual encoder outputs.
  • M-RAG pipeline: The system fuses visual and textual embeddings by normalized summation, scores documents by similarity, and concatenates the top-k results into the LVLM context.The similarity function is cosine similarity, and the LVLM uses the retrieved content with the query to answer questions.
  • Attacker: The attacker must achieve both retrievability, by entering top-k results through embedding alignment, and inducibility, by steering the LVLM with malicious textual instructions.The attacker cannot modify the retriever, LVLM, or queries.
  • Attacker: The attacker adds an imperceptible image perturbation δ satisfying ∥δ∥∞≤ϵ to maximize similarity, while crafting malicious text to mislead the LVLM.The perturbed image is constructed as I−= I0 + δ.
  • Defender: The defender aims to keep poison retrieval and attack success rates low while preserving benign retrieval and generation quality, without knowing poisoned samples or the attack strategy.The defender has full control over the M-RAG system but must operate without this poisoning information.

Dynamic Soft Prompt Defense

DSPrompt defends multimodal retrieval by inserting lightweight, layerwise soft prompts into a frozen encoder, reshaping embeddings to suppress adversarial similarity while preserving benign retrieval. Dynamic adversarial training continually regenerates poisons against the current defense, while composite losses maintain selectivity and generalization without detectors, re-ranking, or per-query optimization.

  • Why Soft Prompts Can Prevent Poisons: Adversarial perturbations forge high poison-query similarity despite low intrinsic relevance, whereas soft prompts reduce sensitivity along exploited non-semantic directions while preserving benign scores.The frozen encoder is modified through a small learnable module rather than full retraining.
  • Prompt Architecture: Where and How Many: DSPrompt inserts learnable prompt tokens after [CLS] at every layer of the visual and textual Transformer encoders, training only prompts while freezing the backbone.Prompts interact through self-attention, are removed before the next layer, and preserve sequence length and backbone structure.
  • Prompt Architecture: Where and How Many: A shallow-to-deep schedule geometrically increases prompt capacity across three layer-depth groups, concentrating parameters in deeper layers where adversarial effects reach cross-modal alignment.The schedule keeps total overhead below 1% of the backbone.
  • Defense Loss Functions: The defense loss combines contrastive retrieval, a symmetric document-to-query objective, and clean-anchor regularization to demote poisons while preserving benign embedding distributions.Positive documents are promoted over generated poisons and other negatives, while anchoring limits drift from the original retriever.
  • Dynamic Min-Max Training: Dynamic adversarial training regenerates poisons against the current defense at every step instead of optimizing against fixed pregenerated attacks.This min-max procedure targets an evolving adversarial distribution and reduces overfitting to specific attack patterns.
  • Dynamic Min-Max Training: The inner attack searches for a text field maximizing current similarity, then applies PGD to optimize an image perturbation δ within an ℓ∞ budget; the outer loop updates prompts to demote the poison.The strongest poison admitted by the current defense is generated for each training pair.
  • Dynamic Min-Max Training: DSPrompt penalizes adversarial similarity without requiring a detector, re-ranker, or per-query optimization.The defended encoder can therefore operate as a plug-in modification to standard retrieval.

Experiments

Experiments across four benchmarks and three poisoning attacks show that DSPrompt substantially improves robustness while preserving retrieval and generation fidelity. The defense transfers to unseen attacks and model combinations, remains effective under denser poisoning, and adds no index-size overhead.

  • Experimental setup: DSPrompt is evaluated against PE-C, GPA, and Clean-L using nonoverlapping M-BEIR training queries, with No Defense and RoCLIP as baselines.The soft prompt is trained on 5,000 queries, while online poisons are drawn from the global candidate pool during min–max training.
  • Effectiveness: 81.15% to 6.66%: DSPrompt lowers ASR on Places365 while reducing PRR@1 to single digits, with SUF@3 ≈99% and TF in the low nineties.Only PE-C is used for online poison generation during training; GPA and Clean-L are unseen at test.
  • Effectiveness: 0.64% ASR and 80.21% TF: DSPrompt neutralizes the unseen GPA attack, although its SUF@3 (69.54%) trails RoCLIP’s (74.70%).This cross-attack transfer supports learning a transferable embedding corrector rather than memorizing the training attack.
  • Mechanism: 0.68 vs. 0.67 to 0.53 vs. 0.64: DSPrompt separates poison and clean retrieval-score distributions, relocating poison below the corpus and out of the top-k band.Under Raw CLIP, poison and clean distributions nearly coincide; after defense, benign structure remains intact.
  • Robustness: Nadv=1 to 10: DSPrompt keeps PRR and ASR near zero and TF close to its optimum as injected-document density increases.The defense remains effective when poisons form dense clusters.
  • Generalization and efficiency: Across four retriever–generator combinations, DSPrompt consistently reduces PRR@1, PRR@3, and ASR while restoring high TF, and standard retrieval adds no index-size overhead.The combinations use OpenCLIP or SigLIP retrievers with LLaVA-v1.6-Mistral-7B or Qwen-VL generators; the defended encoder stores one embedding per document.

Conclusion

DSPrompt reshapes a frozen retriever’s embedding semantics with shallow-to-deep soft prompts trained through a min–max game, without modifying the retrieval pipeline. It re-encodes documents to weaken manufactured similarity while preserving benign relevance with a clean anchor and adds fewer than 1% parameters.

  • Conclusion: DSPrompt directly reshapes embedding semantics in a frozen encoder without modifying the retrieval pipeline.The framework uses a few shallow-to-deep soft prompts.
  • Conclusion: DSPrompt trains soft prompts through a min–max game using the exact retrieval score.
  • Conclusion: < 1% parameters are added while document re-encoding weakens manufactured similarity and a clean anchor preserves benign relevance.

RAG Pipeline

DSPrompt preserves the standard two-stage multimodal RAG pipeline while modifying only the retriever. Its pre-computed embeddings enable ordinary single nearest-neighbour top-k retrieval without re-ranking, and the unchanged generation prompt isolates effects to retrieval.

  • Pipeline design: DSPrompt changes only the retriever; the LVLM and generation prompt remain exactly as in ordinary M-RAG.The system still retrieves documents first and then generates from them.
  • Stage 1: retrieval: The defended dual encoder embeds queries and image-text documents in a shared space, scoring documents by cosine similarity.Retrieval returns the top-k documents by score.
  • Stage 1: retrieval: Pre-computed defended embeddings support single nearest-neighbour top-k retrieval with no re-ranking at the same cost as ordinary dense retrieval.The encoder is indexed exactly like the original encoder.
  • Stage 2: generation: Retrieved captions are concatenated in rank order with the query and sent to the LVLM through a fixed prompt template.Each document caption is truncated to 300 characters and formatted as one Document i line.
  • Defense isolation: Keeping poisons out of the top-k prevents them from reaching the LVLM, while the identical template isolates the measured effect to the retriever.The generation template is identical with and without DSPrompt.

Malicious Text Instruction Construction

The attack constructs hijacking texts through two template families: an image-consistent dataset-negative variant and a query-grounded LLM-generated variant. It generates candidate pools, selects the candidate most similar to the query under the current retriever, and then applies PGD to its image.

  • Online poisoning: With T − fixed, PGD optimizes the image perturbation δ to form the online poison document used by the outer minimization.The poison is constructed as d− = (I0 + δ, T −).
  • Construction variants: Two hijacking-text families combine descriptions with misleading answers: Way 1 uses dataset-negative descriptions and a generic refusal, while Way 2 uses query-specific wrong answers and supporting descriptions.Way 1 preserves image-text consistency and avoids copying the query; Way 2 is generated from the query and ground-truth answer by an LLM.
  • Construction procedure: For each poison, a biased coin selects Way 1 with probability 1 − pllm or Way 2 with probability pllm, after which every template in the selected family generates candidates.Way 1 uses dataset-negative descriptions with a generic hijacking answer; Way 2 combines the query, LLM-generated description, and query-specific wrong answer.
  • Candidate selection: The current retriever scores every candidate against the query and keeps the single text T − with the highest similarity.This gradient-free selection uses the same retrieval score sθ(q, d) as the min–max objective.
  • Dynamic adversary: The winning text is re-selected against the current θ at each step, preventing the defender from overfitting to a single poisoning template.The discrete text search complements the continuous PGD search over δ.

A Poisoning Case Study

A case study shows how PGD-perturbed multimodal poisons exploit embedding similarity and mutually consistent content to outrank clean documents and evade filtering.

  • A Poisoning Case Study: PGD-perturbed poisons achieve near-maximal similarity, allowing them to outrank clean candidates with moderate similarity for the building-identification query.The ground-truth answer is Teutonic Order.
  • A Poisoning Case Study: Each poison combines an embedding-aligned off-target image with descriptive text, a hijacking answer, and the query itself.This mutual consistency allows the poisons to bypass filtering and outrank clean documents.

Attack and Dataset Setting

The evaluation preserves each poisoning attack’s original dataset, retrieval corpus, and poison-crafting strength rather than weakening attacks under a unified setup. It covers PE-C on Places365/ImageNet-1K, GPA on WebQA, and Clean-L on InfoSeek using attack-specific protocols and budgets.

  • Evaluation protocol: Each attack is evaluated using the original paper’s dataset, retrieval corpus, poison-crafting procedure, and reported attack strength.The evaluation therefore uses original attack instances rather than weakened variants under a unified setup.
  • PE-C: PE-C targets Places365 and ImageNet-1K with k = 3, using 60 same-class auxiliary images to pull poisoned samples toward target-class embedding centres.The poison text carries the target answer “I don’t know”; the passage reports PGD step size η=0.01.
  • PE-C: β=0.4 defines the enhanced roclip_enhanced PE-C objective, which adds image-text consistency so the poison text remains matched after RoCLIP re-matching.The evaluation uses η=0.01 and ϵ=0.0625 while following the PE-C paper’s enhanced formulation.
  • GPA: GPA on WebQA optimizes one shared poisoned image from random noise for retrieval across many queries, rather than targeting a single query or starting from a base image.The protocol uses 500 PGD steps, η=0.01, and injects 5 adversarial documents without an explicit image ℓ∞ budget.
  • Clean-L: Clean-L on InfoSeek starts from a clean base image and optimizes a perturbation to maximize similarity to the target query under an ℓ∞ constraint.The setup uses ϵ=16/255 ≈0.063, η=2/255 ≈0.008, and 300 PGD steps, producing 250 poison images and increasing mean poison–query cosine similarity from 0.78 to 0.97.
Loading 2608.16536v1…