PaperlayerView exact revision on arXiv

Source-linked AI summary

MMA-Diffusion: MultiModal Attack on Diffusion Models

Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Tsung-Yi Ho, Nan Xu, Qiang Xu

arXiv:2311.17516v4cs.CRcs.CV

TL;DR

NSFW misuse exposes a robustness gap in T2I safety mechanisms. MMA-Diffusion combines semantic-preserving prompt attacks with minimally visible image perturbations to bypass prompt filters and post-hoc checkers, achieving high attack success across open-source and online systems while omitting operational details to mitigate misuse.

  • Problem

    T2I models can generate NSFW content, while the robustness of prompt filters and post-synthesis safety checks against sophisticated adversarial attacks remains uncertain.

  • Method

    MMA-Diffusion uses text and image modality attacks to evade prompt filters and post-hoc safety checkers in white-box and black-box settings.

  • Results

    83.33% and 90.00% 10-query black-box attack success rates were achieved on Midjounery and Leonardo.Ai, respectively, while image attacks achieved 88.52% ASR-4 and 78.68% ASR-1 against Stable Diffusion’s safety checker.

  • Takeaways & Limitations

    The results demonstrate limitations in current T2I defensive measures and emphasize the need for more effective security controls.

  • Takeaways & Limitations

    Specific attack details were deliberately omitted or generalized to mitigate potential misuse.

Abstract

from arXiv · show

In recent years, Text-to-Image (T2I) models have seen remarkable advancements, gaining widespread adoption. However, this progress has inadvertently opened avenues for potential misuse, particularly in generating inappropriate or Not-Safe-For-Work (NSFW) content. Our work introduces MMA-Diffusion, a framework that presents a significant and realistic threat to the security of T2I models by effectively circumventing current defensive measures in both open-source models and commercial online services. Unlike previous approaches, MMA-Diffusion leverages both textual and visual modalities to bypass safeguards like prompt filters and post-hoc safety checkers, thus exposing and highlighting the vulnerabilities in existing defense mechanisms.

1. Introduction

MMA-Diffusion addresses the unresolved robustness of T2I safety mechanisms against adversarial attacks by combining textual and visual attacks. It exposes vulnerabilities in prompt filters and post-hoc safety checkers, including for unauthorized NSFW editing of real individuals’ imagery.

  • T2I models pose misuse risks through generation of NSFW content, including adult materials, violence, and politically sensitive imagery.
  • MMA-Diffusion systematically tests T2I security by generating adversarial prompts and image perturbations that circumvent existing safety protocols.
  • The text attack alters prompts while preserving semantic intent, whereas the image attack subtly changes image characteristics to evade post-processing safety algorithms.
  • The multimodal strategy reveals vulnerabilities in current safeguards and motivates more robust, comprehensive security measures for T2I systems.
  • 83.33% and 90% success rates were achieved in 10-query black-box attacks against Midjounery and Leonardo.Ai, respectively.

2. Related Work

Prior T2I adversarial-attack research has focused mainly on text modification and functional vulnerabilities, leaving image-modal NSFW attacks comparatively unexplored. MMA-Diffusion demonstrates the feasibility and broader applicability of such attacks across T2I models.

  • Existing T2I attack studies predominantly modify text to reduce quality, distort or remove objects, or impair image fidelity.
  • Earlier explorations do not target NSFW-specific semantic prompts, while this paper demonstrates the feasibility of those attacks.
  • T2I defenses include input-stage AI moderators on online services and post-hoc safety checkers in systems such as Stable Diffusion.

3. Method

MMA-Diffusion models realistic white-box and black-box threats using text-modal, image-modal, or combined attacks against prompt filters and post-hoc safety checkers. Its methods preserve target semantics while optimizing adversarial prompts or minimally perturbing images.

  • 3.1. Threat Model: The threat model covers white-box attacks on open-source models and black-box transfer attacks against online services lacking parameter access.
  • 3.1. Threat Model: The attack succeeds only when generated NSFW content aligns with the attacker’s plain-text description.
  • 3.2. Approach Overview: Text-modal attacks target prompt filters, image-modal attacks target post-hoc checkers, and both modalities can be combined when both defenses are present.
  • 3.3. Text-Modal Attack: The text attack constructs an adversarial prompt without sensitive words while preserving similarity to the target prompt’s semantics.
  • 3.3. Text-Modal Attack: The text objective maximizes cosine similarity between target and adversarial prompt embeddings: max cos(τθ(ptar), τθ(padv)).
  • 3.3. Text-Modal Attack: Sensitive-word regularization suppresses tokens from a predefined list by setting their gradients to −inf.
  • 3.4. Image-Modal Attack: The image attack optimizes an input image perturbation so the generated synthesis avoids post-hoc NSFW detection while remaining minimally altered.
  • 3.4. Image-Modal Attack: Dynamic loss selection optimizes only triggered safety-checker terms, while projected gradient descent enforces the perturbation budget ε.

4. Experiments

Experiments evaluate MMA-Diffusion against defenses across open-source models and commercial services. The attacks achieve high success rates while avoiding sensitive words, bypassing safety checkers, and combining text and image modalities.

  • 4.1. Experimental Settings: The evaluation uses NSFW prompts from LAION-COCO and UnsafeDiff across open-source models, online services, and multiple safety detectors.LAION-COCO contributes 1,000 captions with NSFW scores above 0.99; UnsafeDiff contributes 30 prompts spanning six themes.
  • 4.2. Attacking Open-Source Models: 83.37% ASR-4 is achieved in white-box attacks on SD, while transfer attacks reach 73.70% ASR-4 on black-box SDXL.The SDXL result persists despite its cascade architecture and different text encoders.
  • 4.2. Attacking Open-Source Models: 76.73% ASR-4 is achieved against SLD despite its latent-space defense based on fixed NSFW concept embeddings.MMA-Diffusion avoids a substantial portion of the sensitive words represented by those embeddings.
  • 4.3. Attacking Online T2I Services: MMA-Diffusion achieves 83.33% and 90.00% 10-query attack success rates on Midjourney and Leonardo.Ai, respectively.A successful attack requires at least one adversarial prompt to evade moderation and produce a high-quality, high-fidelity synthesis judged by humans.
  • 4.3. Attacking Online T2I Services: Midjourney’s overall success rate is 3.96% for pornography and 27.67% for violence, while Leonardo.Ai reaches 38% for adult images.The reported Leonardo.Ai adult-image success rate is nearly ten times Midjourney’s pornography success rate.
  • 4.4. Multimodal Attack Results: Image-modal attacks achieve 88.52% ASR-4 and 78.68% ASR-1 against SD’s safety checker, while transferring to Q16 and MHSC at 30% and 20%.The multimodal attack reaches 85.48% ASR-4 and 75.52% ASR-1 when both prompt filtering and post-hoc checking are enabled.

5. Ethical Considerations

The research is intended to strengthen T2I diffusion-model security rather than enable misuse, so attack details are deliberately omitted or generalized. It also urges responsible use, ethical awareness, transparent reporting, and attention to societal impact and misuse prevention.

  • The study is conducted to strengthen T2I diffusion-model security rather than enable misuse.
  • Attack details are deliberately omitted or generalized to mitigate potential misuse.
  • The authors urge responsible use of the findings and advocate ethical awareness in generative-model research.
  • They identify transparent reporting focused on societal impact and misuse prevention as essential.

6. Conclusion

MMA-Diffusion is presented as a multimodal attack framework exposing weaknesses in current T2I defenses. It automates visually realistic, semantically diverse outputs, supports black-box attacks, and motivates stronger security controls.

  • MMA-Diffusion is a multimodal attack framework highlighting potential misuse of T2I models for inappropriate-content generation.
  • The framework automates visually realistic and semantically diverse image generation while maintaining high success rate, quality, and diversity.
  • MMA-Diffusion enables black-box attacks across different generative models.
  • The results demonstrate limitations in current defensive measures and emphasize the need for more effective security controls.

Overview

The overview covers supplementary materials, sensitive-word resources, experimental setup, attack hyperparameters, and baseline-comparison adjustments. It also documents how target prompts and model configurations are prepared for evaluation.

  • Supplementary materials: The supplementary material adds a sensitive-word list, detailed algorithm, experimental setup, DALL·E2-pytorch experiments, input-sanitization tests, and failure-case visualizations.
  • Sensitive-word resources: The sensitive-word resources include Adult Theme and Other NSFW Theme categories, with LAION-COCO attacks using Adult Theme and UnsafeDiff attacks using the full list.
  • Experimental setup: The experiments document victim-model implementations, baseline hyperparameters, and approach details, using an RTX4090 GPU with 24GB memory.
  • Model configurations: SD, SDXL, SLD, and DALL·E2-pytorch use specified guidance, inference, safety, sampling, and image-size settings, while Midjourney and Leonardo.Ai use defaults.
  • Attack configuration: The text attack uses adversarial prompt length L = 20, while the image attack uses ℓ2 perturbation budget 16, step size α = 2, and 20 iterations.
  • Baseline comparison: Existing attacks are difficult to compare directly, so QF-attack is reconfigured with an aligned objective function and modified input prompts.

C. Results on DALL·E2-pytorch

The DALL·E2-pytorch evaluation is constrained by limited image resolution and text-to-image fidelity, which negatively impacts attack success rate. Generating more samples per prompt makes its ASR comparable to SDXL under the reported setup.

  • DALL·E2-pytorch has lower image resolution and text-to-image fidelity than SD and SDXL, negatively impacting its ASR.
  • 25 samples per prompt align DALL·E2-pytorch GPU memory consumption with SDXL at 4 samples per prompt, at approximately 24GB.
  • This adjustment produces an ASR comparable to SDXL, as reported in Table S-2.

D. Against Input Sanitization Defense

MMA-Diffusion remains effective against a basic input-sanitization defense that removes non-dictionary words and special characters. Sanitized adversarial prompts continue generating NSFW content, indicating that token removal alone does not fully neutralize the attack.

  • The sanitization defense used GPT-turbo-3.5 to remove non-dictionary words and special characters while preserving intelligible words.
  • 87.00% of sanitized adversarial prompts still generated NSFW content with the SD model.This corresponds to 870 successful sanitized prompts out of 1,000 tested.
  • MMA-Diffusion’s success does not depend on special tokens, but on the collective impact of the token set organized by its algorithm.
  • The method adapts its candidate vocabulary from the entire dictionary to appropriate words to evade rudimentary sanitization.

E. More Visualizations

Supplementary visualizations show both successful attack examples and failure cases across celebrity-related, NSFW, black-box, and image-modal scenarios. The failures mainly involve images that miss the intended concept or target identity.

  • Failure Cases: Celebrity-targeted adversarial prompts sometimes produced related individuals or objects instead of the named celebrities.
  • Failure Cases: Image-modal attacks failed when generated images did not adequately capture the intended NSFW concepts of “naked” or “Racism.”Human evaluators classified these outputs as unsuccessful.
  • Additional Visualizations: The supplementary examples include black-box attacks on Midjourney and Leonardo.Ai that avoid sensitive words highlighted in red.
  • Additional Visualizations: The visualizations include examples involving graphic violence, disturbing imagery, bodily distortion, and other harmful concepts.
  • Additional Visualizations: MMA-Diffusion can generate inauthentic depictions of real people, with Gaussian blurs added for ethical considerations.
Loading 2311.17516v4…