Source-linked AI summary
AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting
Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, Chaowei Xiao
TL;DR
MLLMs are vulnerable to structure-based jailbreaks that place harmful semantic content in images, motivating defenses that work without direct model modification. AdaShield prepends manually designed or adaptively auto-refined prompts, and experiments report stronger jailbreak robustness while preserving benign-task performance.
Problem
MLLMs are vulnerable to structure-based jailbreak attacks, while fine-tuning is costly and black-box services expose no parameters or gradients for targeted defenses.
Method
AdaShield prepends defense prompts to inputs, combining a manually designed static prompt with an adaptive framework that iteratively optimizes prompts using a target MLLM and defender.
Results
AdaShield improves robustness against structure-based jailbreak attacks while maintaining model performance on benign datasets.
Takeaways & Limitations
AdaShield provides a plug-and-play safety mechanism for MLLMs that does not require fine-tuning or additional modules.
Takeaways & Limitations
AdaShield is specifically designed for structure-based jailbreak attacks and does not yet provide a universal defense against perturbation-based attacks.
Abstract
from arXiv · showhide
With the advent and widespread deployment of Multimodal Large Language Models (MLLMs), the imperative to ensure their safety has become increasingly pronounced. However, with the integration of additional modalities, MLLMs are exposed to new vulnerabilities, rendering them prone to structured-based jailbreak attacks, where semantic content (e.g., "harmful text") has been injected into the images to mislead MLLMs. In this work, we aim to defend against such threats. Specifically, we propose \textbf{Ada}ptive \textbf{Shield} Prompting (\textbf{AdaShield}), which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks without fine-tuning MLLMs or training additional modules (e.g., post-stage content detector). Initially, we present a manually designed static defense prompt, which thoroughly examines the image and instruction content step by step and specifies response methods to malicious queries. Furthermore, we introduce an adaptive auto-refinement framework, consisting of a target MLLM and a LLM-based defense prompt generator (Defender). These components collaboratively and iteratively communicate to generate a defense prompt. Extensive experiments on the popular structure-based jailbreak attacks and benign datasets show that our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks without compromising the model's general capabilities evaluated on standard benign tasks. Our code is available at https://github.com/rain305f/AdaShield.
1 Introduction
The paper addresses structure-based jailbreaks in MLLMs with AdaShield, an adaptive defense-prompting framework designed for costly and often black-box deployment settings. It combines a manually designed prompt with auto-refined prompts and reports stronger robustness while preserving benign-task performance.
- Structure-based attacks embed harmful content in images paired with benign text to bypass MLLM safety mechanisms.
- Fine-tuning defenses are costly, while black-box MLMaaS access prevents direct parameter- or gradient-based intervention.
- AdaShield prepends input-aware defense prompts without fine-tuning MLLMs or training auxiliary models, supporting black-box victim models.
- AdaShield-S manually designs a general defense prompt, while AdaShield-A iteratively optimizes prompts for intricate attack scenarios.
- Experiments against FigStep and QR show improved defense performance while maintaining performance on standard benign tasks.
- The auto-refinement framework generates diverse safety-guided prompts and retrieves an optimal prompt for each query.
2 Related Work
Related work distinguishes perturbation-based from structure-based MLLM jailbreaks and surveys inference-time and training-time defenses. Existing approaches include defense prompts, alignment improvement, and pipelines combining harm detection with response correction.
- Perturbation-based attacks disrupt MLLM alignment using adversarial images, including imperceptible perturbations or patches.
- Structure-based attacks form a contrasting jailbreak category that converts harmful content into image structures.
- MLLM defenses comprise inference-time and training-time alignment strategies.
- FigStep represents inference-time defense prompting, whereas DRESS improves alignment and interaction using natural-language feedback.
- MLLMP adds a harm detector and detoxifier to identify harmful responses and correct them.
3 Methodology
AdaShield combines a manually designed static defense prompt with an adaptive auto-refinement framework for defending MLLMs against structure-based jailbreaks. The adaptive framework iteratively improves scenario-specific prompts from jailbreak feedback and retrieves prompts by multimodal query similarity during inference.
- AdaShield-S: Manual Static Defense Prompt: AdaShield-S checks image and text content step by step, specifies refusal behavior for malicious queries, and provides safe handling instructions for benign inputs.The prompt is designed to detect malicious content while reducing over-defense on safe inputs.
- AdaShield-S: Manual Static Defense Prompt: AdaShield-S remains weak in complex legal, economic, and healthcare scenarios because it uses a unified safety guideline.AdaShield-A is introduced to customize guidance for different scenarios.
- AdaShield-A: Defense Prompt Auto-Refinement Framework: AdaShield-A uses a defender and target MLLM to generate defense prompts, then refines them using failed prompts and jailbreak responses as feedback.The defender outputs improved prompts and reasons for interpretability during iterative auto-refinement.
- AdaShield-A: Defense Prompt Auto-Refinement Framework: During training, AdaShield-A builds a diverse pool of scenario-specific defense prompts by refining prompts against malicious queries and their jailbreak responses.Each refined prompt is stored with the malicious query that produced it.
- AdaShield-A: Defense Prompt Auto-Refinement Framework: At inference, AdaShield-A concatenates text and image embeddings, retrieves the most similar defense prompt by cosine similarity, and prepends it to the query.If maximum similarity is below β, the query is treated as benign and no defense prompt is used, limiting over-defense.
4 Experiment
Experiments evaluate AdaShield-S and AdaShield-A against FigStep and QR attacks, alongside benign-task performance, ablations, efficiency, generalization, transferability, and qualitative prompt analysis. AdaShield-A generally provides stronger defense while preserving benign capabilities and adding little retrieval-time cost.
- Experimental Setup: Evaluations use FigStep and QR attacks, MM-Vet benign tasks, keyword-based ASR, and GPT recheck attack success rate.MM-Vet reports Recognize, OCR, Knowledge, Generation, Spatial, and Math capabilities.
- Main Results: AdaShield-S and AdaShield-A outperform FSD and MLLMP on FigStep and QR, with AdaShield-A stronger than AdaShield-S.The reported comparison uses Recheck and ASR; AdaShield-S lacks the specific safety rules used by AdaShield-A.
- Main Results: AdaShield-A achieves benign-task performance comparable to Vanilla and outperforms MLLMP and FSD, reducing over-defense through similarity-based benign-query filtering.AdaShield-S still degrades benign performance because it is less effective at recognizing benign queries.
- Ablation Study: Manual prompt Ps achieves the best static-prompt defense performance, while explicit instructions for responding to malicious queries are necessary for effective refusal.The ablation compares Ps with alternative prompts across CogVLM and LLaVA; the benign-task variant Pv omits a safety-execution sentence.
- Ablation Study: The proposed retrieval method outperforms random prompt selection, showing that retrieval contributes to AdaShield-A’s defense performance.Both methods use the same defense-prompt pool, isolating the retrieval strategy.
- Analysis Study: AdaShield-A incurs negligible retrieval time, generalizes to unseen QR scenarios, transfers across target MLLMs, and generates query-specific prompts containing detailed safety rules.The qualitative examples use LLaVA 1.5-13B as the target MLLM.
5 Conclusion & Limitation
AdaShield uses adaptive shield prompting to improve MLLM robustness against structure-based jailbreak attacks without fine-tuning or additional modules. Its demonstrated scope is limited to structure-based attacks, while broader defense remains future work.
- AdaShield enhances MLLM robustness against structure-based jailbreak attacks without fine-tuning or additional modules.
- The experiments demonstrate effective safeguarding while preserving MLLMs’ general capabilities.
- AdaShield is specifically designed for structure-based jailbreak attacks, leaving universal defense against structure-based and perturbation-based attacks for future work.
Appendix
The appendix provides additional dataset descriptions, ablation studies, and sensitivity analyses of hyperparameters.
- The appendix details the datasets used in the study.
- It also reports additional ablation studies and sensitivity analyses about hyperparameters.
A.1 Datasets Details.
The evaluation uses FigStep and QR structure-based attacks, MM-Vet for benign multimodal capabilities, and prompt specifications for adaptive defense refinement. The datasets cover multiple unsafe scenarios and six visual-language capabilities.
- Structure-based Jailbreak Attacks: FigStep covers 10 prohibited scenarios with 50 harmful requests per scenario, while QR contains 1,680 malicious questions across 13 unsafe and sensitive scenarios.
- Benign Dataset Details: MM-Vet evaluates over-defensiveness across Recognition, Knowledge, OCR, Spatial awareness, Language generation, and Math capabilities.
- Benign Dataset Details: MM-Vet assigns each capability a full score of 100% and uses OpenAI’s GPT-4 API as the LLM-based evaluator.
- Adaptive Defense Prompt: The Defender prompt instructs iterative refinement to prevent unethical, illegal, or dangerous behavior and produce safe responses to jailbreak instructions.
- Adaptive Defense Prompt: The system prompt defines the refinement setting, safety rules for each scenario, prompt-optimization techniques, and the Defender’s response format.
B.1 Additional Ablation studies
Additional ablations examine the effect of AdaShield-A’s initial defense prompt and report that performance remains strong across prompt variants.
- Initial Defense Prompt: AdaShield-A achieves its best performance when equipped with the manual defense prompt Ps.
- Initial Defense Prompt: Even the least effective AdaShield-A variant with prompt Pa surpasses other defense methods in performance.
- Initial Defense Prompt: The ablation indicates that AdaShield-A is robust to the choice of initial static defense prompt.
B.2 Additional Sensitive Analysis
The sensitivity analysis examines how α controls the generality of the defense-prompt pool and how β balances jailbreak defense against benign-task performance. It also documents the refusal signals used in evaluation.
- Sensitivity to α: α filters auto-refined prompts by validation-set ASR, improving the generality of the final defense-prompt pool as its value increases.The paper sets α = 0.8 after observing lower average ASR with larger α.
- Sensitivity to β: As β increases, AdaShield-A’s average ASR on QR and total score on MM-Vet both rise, revealing a defense–utility trade-off.β initially identifies benign queries whose maximum similarity to defense-pool keys falls below the threshold, so no defense prompt is prepended.
- Evaluation signals: The refusal-signal list retains most FigStep strings and adds signals observed during evaluations.Examples include phrases such as “It is important to consider” and “legal guidelines.”