PaperlayerView exact revision on arXiv

Source-linked AI summary

A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion

Haomin Zhuang, Yihua Zhang, Sijia Liu

arXiv:2303.16378v2cs.CV

TL;DR

This paper asks whether adversarial prompts can be generated against Stable Diffusion without querying the diffusion model. It proposes query-free untargeted and targeted attacks using CLIP text embeddings, and reports significant image-content changes from five-character perturbations, including targeted removal of selected content while retaining unrelated semantics.

  • Problem

    The paper investigates whether adversarial prompts can attack Stable Diffusion without end-to-end diffusion-model queries, addressing limited attention to T2I adversarial robustness.

    Source: §1 · p. 1

  • Method

    The authors use access to CLIP’s text encoder to generate query-free prompt perturbations, with a targeted variant guided by influential dimensions in text-embedding differences.

    Source: §2 · p. 2, §3 · p. 3

  • Results

    A five-character prompt perturbation can effectively alter Stable Diffusion outputs, and targeted attacks can remove selected semantics while retaining irrelevant ones.

    Source: §4.2 · p. 4, §5 · p. 6

  • Takeaways & Limitations

    The experiments indicate that CLIP text-encoder susceptibility can be used for both untargeted image-content changes and targeted content manipulation in Stable Diffusion.

    Source: §5 · p. 6

    The attack assumes access to CLIP’s text encoder and no access to the diffusion model during generation.

    Source: §2 · p. 2

Abstract

from arXiv · show

Despite the record-breaking performance in Text-to-Image (T2I) generation by Stable Diffusion, less research attention is paid to its adversarial robustness. In this work, we study the problem of adversarial attack generation for Stable Diffusion and ask if an adversarial text prompt can be obtained even in the absence of end-to-end model queries. We call the resulting problem 'query-free attack generation'. To resolve this problem, we show that the vulnerability of T2I models is rooted in the lack of robustness of text encoders, e.g., the CLIP text encoder used for attacking Stable Diffusion. Based on such insight, we propose both untargeted and targeted query-free attacks, where the former is built on the most influential dimensions in the text embedding space, which we call steerable key dimensions. By leveraging the proposed attacks, we empirically show that only a five-character perturbation to the text prompt is able to cause the significant content shift of synthesized images using Stable Diffusion. Moreover, we show that the proposed target attack can precisely steer the diffusion model to scrub the targeted image content without causing much change in untargeted image content. Our code is available at https://github.com/OPTML-Group/QF-Attack.

1. Introduction

The introduction asks whether Stable Diffusion can be attacked without end-to-end model queries, building on concerns about CLIP’s vulnerability to small input perturbations. It presents five-character query-free attacks, including a targeted variant, and reports significant changes to generated image content.

Research question

  • The paper asks whether adversarial perturbations against text-to-image models can be generated without end-to-end model queries.The concern is motivated by prior findings that small textual or visual perturbations can impair CLIP and raise robustness concerns for downstream applications.Source: §1 · p. 1

Related context

  • The introduction contrasts the query-free approach with a prior T2I attack that used 10,000 model queries per attack and four newly generated words.It also notes prior prompt optimization work, while stating that little attention had been paid to adversarial robustness.Source: §1 · p. 2

Contributions and results

  • A five-character perturbation determined by CLIP text embeddings can significantly alter Stable Diffusion’s generated image content.The introduction reports effectiveness across three attack implementation methods and a variety of text-image pairs.Source: §1 · p. 2

Contributions

  • The authors analyze correspondence between synthesized-image semantics and CLIP embeddings, then use that insight to develop a targeted attack that steers diffusion outputs.Figure 1 illustrates the targeted attack's aim of erasing image content related to “young man.”Source: Figure 1 · p. 1, §1 · p. 2

2. Related Work

Related work covers adversarial attacks across vision and language, prompt perturbations in text-to-image models, and CLIP robustness. The paper assumes access to CLIP’s text encoder but no access to the diffusion model during attack generation.

Adversarial attacks

  • The paper distinguishes white-box attacks, which use victim-model gradients, from black-box attacks, which use input-output queries.It assumes access to CLIP’s text encoder while remaining blind to the diffusion model, and defines its attack as generating prompts without running the diffusion process.Source: §2 · p. 2

Prompt perturbations in vision-language models

  • Prior work studied text-prompt sensitivity in text-to-image diffusion models and image-input robustness in CLIP, but textual perturbations to CLIP’s influence on diffusion models remained under-investigated.The cited CLIP work included imperceptible pixel perturbations and image-frequency-domain attacks.Source: §2 · p. 2

3. Our Proposal

The proposal attacks Stable Diffusion through CLIP prompt embeddings without optimizing over the diffusion model, then adds intention-guided dimensions for targeted content steering.

Our Proposal

  • The attack minimizes cosine similarity between original and perturbed CLIP text embeddings, without optimizing over the diffusion model.The paper contrasts this with a prior approach requiring 10,000 diffusion-model queries for one attack.Source: §3 · p. 3
  • PGD, greedy search, and genetic algorithms provide alternative implementations of the differentiable prompt-perturbation objective.The genetic algorithm evaluates candidate perturbations using the objective loss and generates new candidates through operations such as mutation.Source: §3 · p. 3
  • Targeted attacks derive steerable key dimensions from CLIP embedding differences between intention-containing sentences and versions with the intention phrase removed.A majority vote across difference vectors selects the influential dimensions.Source: §3 · p. 3
  • The targeted attack uses these dimensions to guide prompt perturbations toward removing intended image content, illustrated by erasing “a young man.”The paper states that constraining perturbations to these dimensions can improve the likelihood of generation following the adversary’s intention.Source: §3 · pp. 3–4

4. Experiment

The experiments evaluate query-free attacks on Stable Diffusion using CLIP scores and generated-image examples. Five-character perturbations alter image content, while targeted attacks can remove selected semantics and retain unrelated ones.

4.1. Experiment setups

  • The experiments use Stable Diffusion v1.4 and its shared CLIP ViT-L/14 text encoder; scores average 20 prompts and 10 generated images per prompt.PGD uses 100 steps at learning rate 0.1; the genetic algorithm uses 50 generations, 20 candidates per step, and mutation rate 0.3.Source: §4.1 · p. 4

4.2. Experiment results

  • Five-character perturbations from the untargeted genetic-algorithm attack significantly alter Stable Diffusion’s generated image content.The examples compare original and perturbed prompts, with images in each column sharing a random seed.Source: §4.2 · p. 4, Figure 3 · p. 5
  • Targeted attacks can remove selected image semantics while retaining unrelated content, such as erasing a bicycle while largely preserving a brick wall.The examples also show removal of a plate while retaining an apple.Source: §4.2 · p. 4, Figure 4 · p. 5
  • Across untargeted attacks, all methods reduce CLIP scores versus No Attack or Random, with the genetic algorithm outperforming other methods.For targeted attacks, PGD and the genetic algorithm outperform other methods, but targeted attacks produce a smaller CLIP-score drop than untargeted attacks.Source: §4.2 · pp. 4–5, Table 1 · p. 4
  • In the WALL-E case study, appending “E” or “WALLE” barely changes the CLIP score, while the five-character perturbation lowers it from 0.293 to 0.217.Replacing “wall” with “wall-E” may produce a robot-related image, but its success is low.Source: §4.2 · p. 6, Table 2 · p. 6, Figure 5 · p. 6

5. Conclusion

The study uses CLIP text-encoder susceptibility to develop query-free untargeted and targeted attacks against Stable Diffusion. It reports that five-character prompt perturbations can be effective, with influential embedding dimensions enabling targeted content manipulation.

5. Conclusion

  • The proposed query-free attacks leverage the susceptibility of the pre-trained CLIP text encoder.Source: §5 · p. 6
  • Five-character prompt perturbations can be effective against Stable Diffusion.The attacks leverage the susceptibility of its pre-trained CLIP text encoder.Source: §5 · p. 6
  • Influential text-embedding dimensions, termed steerable key dimensions, enable targeted manipulation of synthesized image content.Source: §5 · p. 6

Related papers

Loading 2303.16378v2…