PaperlayerView exact revision on arXiv

Source-linked AI summary

Mirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion Models

Shanghao Shi, Chaoyu Zhang, Heng Jin, Hao Li, Ning Zhang, Y. Thomas Hou, Wenjing Lou

arXiv:2610.30348v1cs.CR

TL;DR

Mirage addresses the challenge of preserving fine-grained malicious target semantics while evading an unknown post-generation safety checker. It encodes text or image targets in small image perturbations, aligning with the target while separating from unsafe concepts. Across evaluated models, datasets, and checkers, it reports strong targeted jailbreak performance with limited queries and high target alignment.

  • Problem

    Prior attacks leave open whether fine-grained malicious target semantics can be preserved while evading an unknown post-generation checker.

    Source: §I · p. 1

  • Method

    Mirage encodes text or image targets in small perturbations to benign images, aligning the representation with the target and repelling it from commonly filtered NSFW concepts.

    Source: §I · pp. 1–2, B. Detailed Design · p. 5

  • Results

    On I2P, Mirage requires up to 16.4× fewer queries than SneakyPrompt while achieving higher ASR, and reports high bypass rates across most evaluated models, datasets, and checkers.

    Source: B. Experiment Results · pp. 9–10

  • Takeaways & Limitations

    The results indicate that post-generation safety checking alone may be insufficient against targeted multimodal jailbreaks using small image perturbations.

    Source: §I · p. 2, §VII · p. 13

Abstract

from arXiv · show

Post-generation safety checkers are widely employed in diffusion-model pipelines to prevent unsafe generated content from reaching users. In this paper, we show that these safety checkers can still be bypassed by our proposed multimodal jailbreak attack, Mirage. Given an adversarial target, which can be either a reference NSFW image or a malicious text description, Mirage conceals the target within an imperceptible perturbation applied to a benign input image. Mirage exploits the shared representation space of multimodal encoders to jointly align the perturbed image with the cross-modal adversarial target while repelling its representation from commonly filtered NSFW concepts. Such semantic separation can transfer across representation spaces, enabling Mirage to evade heterogeneous safety checkers without accessing their internal details. We further employ lightweight black-box query refinement to improve evasion near the unknown safety boundary. We implemented Mirage on Stable Diffusion 1.5, Stable Diffusion XL, and Kandinsky 2.2, and evaluated it on two cross-modal NSFW datasets, covering both text-to-image and image-to-image generation scenarios. The experimental results demonstrate that Mirage achieves strong targeted jailbreak performance across diverse post-generation safety checkers with limited black-box queries while preserving high semantic alignment with the adversarial targets.

I. INTRODUCTION

Post-generation safety checkers can be bypassed while preserving targeted malicious semantics, but prior attacks either require checker internals or do not address checker evasion. Mirage encodes text or image targets in small perturbations to benign images, combining target alignment, separation from unsafe concepts, and black-box refinement.

I. INTRODUCTION

  • Multimodal encoders place semantically similar text and image inputs close in a shared representation space, which also lets small image perturbations encode adversarial semantics.The same shared-space mechanism that improves multimodal controllability can therefore support malicious target encoding.Source: §I · p. 1
  • Mirage targets a gap left by prior work: preserving fine-grained malicious semantics while evading an unknown post-generation checker is difficult because stronger target alignment can make outputs easier to detect.The attacker may not know the checker architecture, filtered concepts, or decision threshold.Source: §I · p. 1
  • A norm-bounded image perturbation aligns the perturbed image embedding with the adversarial target, allowing text and visual targets within one optimization framework.The resulting representation guides the diffusion model toward content aligned with the intended target.Source: §I · p. 2
  • Mirage jointly aligns perturbed images with text or image targets and repels their representations from commonly filtered NSFW concepts, using cross-space transferability and lightweight black-box refinement.The attack uses surrogate multimodal encoders and does not require access to checker details or decision thresholds.Source: §I · pp. 1–2
  • Mirage is evaluated on three diffusion models, three heterogeneous checkers, and textual and visual targets, with reported improvements over baselines and applicability to additional generation tasks.The paper also reports an ablation of NSFW-concept repulsion and black-box query refinement.Source: §I · p. 2

A. Diffusion Model

Diffusion models generate images by reversing a gradual noising process. Multimodal adapters encode and fuse inputs in a shared embedding space to condition the base diffusion model.

A. Diffusion Model

  • Diffusion models progressively add Gaussian noise over T steps, then learn a reverse denoising process to recover samples.At inference, sampling starts from Gaussian noise and iteratively denoises to produce an image.Source: A. Diffusion Model · p. 2
  • The reverse-process model is trained to predict the noise added at step t.The stated objective minimizes the squared difference between the added noise and the model’s noise prediction.Source: A. Diffusion Model · p. 2

B. Multimodal Adapter and Encoder

  • A multimodal adapter uses modality encoders to map inputs into a shared embedding space, then feature-integration layers fuse them into conditioning features for the base diffusion model.The adapter is placed before the base model.Source: B. Multimodal Adapter and Encoder · p. 3, Figure 1 · p. 3
  • Multimodal encoders position semantically similar inputs close together across modalities; this paper focuses on CLIP-based encoders because they are widely adopted in multimodal applications.CLIP aligns visual and textual modalities through contrastive pre-training on image-text pairs.Source: B. Multimodal Adapter and Encoder · p. 3

C. Post-Generation Safety Checkers

Post-generation checkers screen generated images, but their detection mechanisms and safety boundaries differ. Prior attacks leave a gap: they either need checker internals or do not address post-generation checker evasion while preserving targeted semantics.

C. Post-Generation Safety Checkers

  • Post-generation safety checkers inspect final images and block unsafe outputs, serving as a last enforcement layer after malicious semantics reach the diffusion model.CLIP-based checkers compare images with NSFW concepts, ViT checkers classify images, and NudeNet detects localized unsafe body parts.Source: C. Post-Generation Safety Checkers · p. 3
  • Checker designs create different representation spaces and safety boundaries, while black-box users may see only accept-or-reject decisions.The attacker may lack access to checker architecture, parameters, or decision thresholds.Source: C. Post-Generation Safety Checkers · p. 3

D. Existing Attacks

  • Earlier multimodal representation attacks demonstrated transferability but did not target bypassing post-generation checkers to generate restricted content.Prior diffusion attacks also motivated manipulating upstream conditioning representations rather than the iterative denoising process.Source: D. Existing Attacks · p. 4
  • Prior text-based attacks can incur unnatural wording and search overhead; MMA-Diffusion requires checker internals, while Image Trojan does not address post-generation evasion.Mirage instead supports textual and visual targets against black-box checkers while preserving target semantics with small image perturbations.Source: D. Existing Attacks · p. 4

III. THREAT MODEL

The threat model gives the attacker a small perturbation to a benign image and a surrogate multimodal encoder, but no access to diffusion internals or checker details. Mirage attacks the upstream encoder by aligning with the target and separating from unsafe concepts.

III. THREAT MODEL

  • The attacker adds a small, norm-bounded perturbation to an image and cannot interfere with the diffusion model’s internal generation process.The constraint is intended to keep adversarial images visually indistinguishable from their originals.Source: §III · p. 4
  • The attacker lacks checker details and thresholds but may know common filtered NSFW concepts and possess a surrogate encoder similar to those used by the target model.The target can be either a reference image or a malicious text description.Source: §III · pp. 4–5

A. Attack Intuition

  • Mirage targets the upstream multimodal encoder rather than the iterative diffusion process, because small perturbations may be removed by denoising.It aligns the perturbed representation with the target and repels it from unsafe concepts in a surrogate space, relying on semantic relationships transferring across representation spaces.Source: A. Attack Intuition · p. 5

B. Detailed Design

Mirage first optimizes a small image perturbation in a surrogate multimodal space to align with the adversarial target and separate from unsafe concepts. It then uses limited black-box refinement to address mismatches with the unknown safety boundary.

B. Detailed Design

  • Mirage combines surrogate-space contrastive perturbation generation with lightweight black-box query refinement against the target pipeline.The refinement addresses residual mismatch when the transferred representation does not cross an unknown safety boundary.Source: B. Detailed Design · p. 5
  • The contrastive objective treats the adversarial image and target as a similar pair, and the adversarial image and NSFW concepts as dissimilar pairs.The encoder remains frozen while the perturbed image is continuously optimized.Source: B. Detailed Design · p. 5
  • A norm-bounded perturbation aligns the adversarial image embedding with the target embedding in the shared multimodal space.This allows textual and visual targets to use the same optimization framework.Source: B. Detailed Design · p. 5
  • The loss trades off target alignment against repulsion from NSFW embeddings through the hyperparameter λ.The contrastive loss’s first term aligns the adversarial and target embeddings; its second term pushes the adversarial embedding away from NSFW embeddings.Source: B. Detailed Design · p. 5
  • When surrogate optimization does not cross an unknown safety boundary, a second perturbation δ+ enables small-norm refinement with a few queries.δ+ is bounded by ϵ, and Square Attack adds it within a continuous square area of the image.Source: B. Detailed Design · p. 6

C. Analysis

Mirage relies on relative semantic structure in the surrogate space, rather than requiring that space to match a safety checker’s representation. Its analysis describes the attack procedure, concept and surrogate choices, and illustrative Stable Diffusion XL outputs.

C. Analysis

  • Relative semantic structure may transfer across differing architectures and decision boundaries, but transferability is not guaranteed to match an unknown safety boundary perfectly.Mirage therefore uses limited black-box query refinement to compensate for residual mismatch.Source: C. Analysis · p. 6
  • The attack procedure optimizes a perturbation with a surrogate encoder, then repeats queries with randomized perturbations while the diffusion pipeline blocks the input.The procedure returns the generated output after the blocked condition ends.Source: C. Analysis · p. 6
  • Mirage repels 17 unsafe concepts and 3 special-care concepts reverse-engineered for Stable Diffusion, with the option to add other commonly recognized NSFW concepts.The paper also identifies publicly released encoder variants and ensembles as ways to obtain or diversify surrogates.Source: C. Analysis · pp. 6–7
  • In all four illustrated Stable Diffusion XL cases, the perturbed inputs were visually indistinguishable from the originals and generated target-related images that bypassed safety checkers.The examples use adult or violent images and offensive descriptions as targets; masks hide inappropriate image regions.Source: Figure 5 · p. 7

V. IMPLEMENTATION

The implementation evaluates Mirage on three multimodal diffusion models using text and image targets, two CLIP surrogate variants, and three post-generation safety checkers. The reported evaluation setup measures bypass, targeted success, query efficiency, and target similarity.

V. IMPLEMENTATION

  • Visual targets come from 200 NSFW Classification samples, while text targets are the 100 most toxic descriptions from I2P.Initial source images are drawn from a Hugging Face image-to-image generation demo.Source: A. Experiment Settings · p. 7
  • The evaluation uses Stable Diffusion 1.5, Stable Diffusion XL, and Kandinsky 2.2, protected by checkers whose implementation details are unavailable to the adversary.All three models accept image and text prompts through the image-prompt adapter.Source: A. Experiment Settings · p. 7
  • The surrogate models are CLIP ViT-H/14 and ViT-bigG/14, used for Stable Diffusion 1.5/XL and Kandinsky 2.2 attacks.Both variants were pretrained on the LAION-2B English subset.Source: A. Experiment Settings · p. 7
  • The evaluation tracks bypass rate, targeted attack success rate, query number, and CLIP similarity, with ASR thresholds of 0.225 for text targets and 0.375 for image targets.ASR requires both bypassing the checker and exceeding the applicable target-similarity threshold.Source: A. Experiment Settings · pp. 7–8
  • Table II covers NSFW generation attack performance on I2P and NSFW Classification datasets.Figure 6 presents CLIP-similarity distributions between adversarial targets and generated images.Source: Table II · p. 8, Figure 6 · p. 8
  • The three checkers cover distinct mechanisms: CLIP concept comparison, ViT-based image classification, and NudeNet region-level nudity detection.They differ in architecture, detection granularity, and decision logic.Source: A. Experiment Settings · p. 8

B. Experiment Results

Mirage outperforms baseline attacks across the reported datasets and post-generation checkers, with higher ASR and comparable or slightly higher CLIP similarity. Results also show strong bypass performance across models, while ablations, perturbation-budget tests, and examples examine the attack’s mechanisms and scope.

Generation Time

  • One perturbed image takes an average of 52.4 seconds on I2P and 30.3 seconds on the NSFW classification dataset for Stable Diffusion 1.5 and XL.For Kandinsky 2.2, the corresponding averages are 142.6 and 101.2 seconds; the text-target I2P optimization is longer because its discrete-space optimization is more complex.Source: B. Experiment Results · pp. 8–9

Generation Examples

  • In Stable Diffusion XL examples, adversarial images are visually indistinguishable from inputs while generated images match the attack targets and bypass safety checkers.For the Stable Diffusion 1.5 comparison, Mirage produces more realistic, better-matched outputs than SneakyPrompt; GPT-2 perplexity rises from 856.51 to 2069.86 after SneakyPrompt.Source: B. Experiment Results · pp. 9–11, Figure 7 · p. 9

Numerical Performance

  • Mirage achieves high bypass rates across most model, dataset, and checker settings, but ViT Checker bypass rates drop noticeably for NSFW image targets on Stable Diffusion 1.5 and XL.Successful samples retain CLIP similarity close to the no-defense setting, and successful jailbreaks on Kandinsky 2.2 are typically found in about one query.Source: B. Experiment Results · p. 9

Baseline Comparison

  • Mirage consistently outperforms four baseline attacks across both datasets and different post-generation checkers, achieving substantially higher ASR with comparable or slightly higher CLIP similarity.On I2P, it requires up to 16.4× fewer queries than SneakyPrompt while achieving higher ASR; on the NSFW image dataset, it has higher ASR than Trojan and Trojan-E and fewer queries than Trojan-E across defenses.Source: Table III · p. 10, B. Experiment Results · p. 10

Impact of Perturbation Budget

  • Strong attack performance is already observed at the smallest tested perturbation budget, 4/255, with no consistent monotonic relationship between budget and performance.Bypass rate and ASR remain comparable across budgets in many settings.Source: B. Experiment Results · p. 11

Surrogate Encoder vs. Surrogate Adapter

  • Surrogate-encoder attacks achieve performance similar to surrogate-adapter attacks in many Stable Diffusion XL settings, supporting use of the encoder alone.The encoder is described as the adapter’s core cross-modal component, and is considerably more lightweight to obtain than the full adapter.Source: B. Experiment Results · p. 11, Table V · p. 12

Attack on Other Generation Tasks

  • Across three Stable Diffusion XL tasks, Mirage examples successfully generate target NSFW content at an 8/255 perturbation budget.The tasks are image-to-image generation, image inpainting, and image-to-video; the video example shows two generated frames.Source: B. Experiment Results · p. 11, Figure 8 · p. 12

Ablation Study

  • Adding NSFW-concept repulsion generally improves evasion beyond target alignment alone, and black-box query refinement further improves bypass rate and ASR across all three checkers.The refinement step produces the strongest overall performance, including in settings where performance remains limited, especially for visual NSFW targets.Source: B. Experiment Results · pp. 11–12

VI. DISCUSSION

The discussion reports a preliminary Mirage attack on LLaVA-1.5-7B, while noting that outputs did not closely match the target. It identifies audio extension and surrogate selection for proprietary models as open challenges and outlines complementary defenses.

Applicability to Other Models and Modalities

  • Mirage introduced target-related NSFW concepts into LLaVA-1.5-7B responses, but those responses did not closely follow the adversarial target.The authors suggest targeted representation-space attacks face additional challenges when extended to discrete textual outputs.Source: §VI · p. 13
  • Audio extension remains non-trivial because perturbations must preserve perceptual imperceptibility, and examined text-to-audio models lacked comparable post-generation guardrails.The authors leave systematic study of other output modalities and their safety mechanisms to future work.Source: §VI · p. 13

Universal Surrogate Models

  • Finding a suitable surrogate for proprietary models remains non-trivial when their multimodal encoders and internal architectures are hidden.The authors identify model selection, adaptation, or additional training as possible needs and leave more transferable jailbreak surrogates for future work.Source: §VI · p. 13

Potential Defense Mechanisms

  • The discussion proposes adversarial training, input purification, robust post-generation checkers, and RLHF as complementary defenses.Adversarial training may degrade model performance, requiring a balance between robustness and accuracy.Source: §VI · p. 13

VII. CONCLUSION

Mirage conceals textual or visual targets in small image perturbations and uses representation separation, transferability, and black-box refinement to evade heterogeneous post-generation checkers. Experiments across models, target modalities, and checkers show its effectiveness, while the paper concludes that post-generation checking alone may be insufficient.

VII. CONCLUSION

  • Experiments across multiple diffusion models, target modalities, and safety checkers demonstrate Mirage’s effectiveness at evading heterogeneous post-generation checkers.The paper concludes that post-generation checking alone may be insufficient against adaptive multimodal jailbreak attacks and highlights defense-in-depth.Source: §VII · p. 13

ETHICS CONSIDERATIONS

The paper describes dual-use risks from a practical jailbreak strategy and unsafe visual content, alongside potential benefits for safety research. It reports controlled experiments and safeguards for handling unsafe outputs and reproducibility materials.

Potential Harms

  • A malicious actor could adapt the technique to bypass safety mechanisms and generate restricted content.The paper also notes that unsafe visual content may disturb readers and that releasing optimized perturbations or generated outputs may increase misuse risk.Source: ETHICS CONSIDERATIONS · p. 14

Mitigations

  • Experiments were conducted in a controlled research environment using public models and datasets, without attacking commercial services, interacting with users, or collecting private data.Unsafe visual examples are masked, while generated NSFW outputs and optimized adversarial examples are withheld or shared only as necessary for validation.Source: ETHICS CONSIDERATIONS · p. 14

Risks vs. Benefits

  • The authors argue that responsible disclosure’s benefits outweigh remaining risks because the work supports understanding vulnerabilities and developing stronger defenses.They plan to release reproducibility materials with safeguards and discuss countermeasures.Source: ETHICS CONSIDERATIONS · p. 14

APPENDIX A. NSFW CONCEPTS

Appendix A lists unsafe and special-care concepts used by the Stable Diffusion safety filter. Appendix B reports that Mirage can introduce target-related keywords into VLM responses, though they do not closely match the target.

APPENDIX A. NSFW CONCEPTS

  • Stable Diffusion filter concepts are grouped into unsafe concepts and special-care concepts, according to [7].The listed unsafe concepts include sexual and explicit-content terms, while special-care concepts include young-child terms.Source: Appendix A · p. 15

APPENDIX B. ATTACK VISION LANGUAGE MODELS

  • In the VLM demonstration, Mirage produced incorrect responses containing target-related keywords, but the outputs were not closely aligned with the intended target.The authors refer to the demonstration in Fig. 9.Source: Appendix B · p. 15

Related papers

Loading 2610.30348v1…