Source-linked AI summary
To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now
Yimeng Zhang, Jinghan Jia, Xin Chen, Aochuan Chen, Yihua Zhang, Jiancheng Liu, Ke Ding, Sijia Liu
TL;DR
Safety-driven unlearning in diffusion models lacks reliable robustness evidence against adversarial prompts. This paper introduces UnlearnDiffAtk, which uses diffusion models’ inherent classification capability to generate attacks without auxiliary models, and benchmarks unlearned models across concepts, styles, and objects. The experiments show that adversarial prompts can bypass current unlearned diffusion models, while UnlearnDiffAtk is more computationally efficient than P4D.
Problem
Unlearned diffusion models lack a systematic and reliable benchmark for robustness against inappropriate image generation under adversarial prompts.
Method
UnlearnDiffAtk generates adversarial prompts by using the unlearned diffusion model’s inherent classification capability and target images, without auxiliary diffusion or image-classification models.
Results
Across concept, style, and object unlearning tasks, adversarial prompts bypassed unlearned diffusion models; UnlearnDiffAtk reduced attack-generation runtime by approximately 23.5% versus P4D.
Takeaways & Limitations
Current safety-driven unlearning techniques do not robustly prevent unsafe image generation when faced with subtle adversarial prompt perturbations.
Abstract
from arXiv · showhide
The recent advances in diffusion models (DMs) have revolutionized the generation of realistic and complex images. However, these models also introduce potential safety hazards, such as producing harmful content and infringing data copyrights. Despite the development of safety-driven unlearning techniques to counteract these challenges, doubts about their efficacy persist. To tackle this issue, we introduce an evaluation framework that leverages adversarial prompts to discern the trustworthiness of these safety-driven DMs after they have undergone the process of unlearning harmful concepts. Specifically, we investigated the adversarial robustness of DMs, assessed by adversarial prompts, when eliminating unwanted concepts, styles, and objects. We develop an effective and efficient adversarial prompt generation approach for DMs, termed UnlearnDiffAtk. This method capitalizes on the intrinsic classification abilities of DMs to simplify the creation of adversarial prompts, thereby eliminating the need for auxiliary classification or diffusion models. Through extensive benchmarking, we evaluate the robustness of widely-used safety-driven unlearned DMs (i.e., DMs after unlearning undesirable concepts, styles, or objects) across a variety of tasks. Our results demonstrate the effectiveness and efficiency merits of UnlearnDiffAtk over the state-of-the-art adversarial prompt generation method and reveal the lack of robustness of current safetydriven unlearning techniques when applied to DMs. Codes are available at https://github.com/OPTML-Group/Diffusion-MU-Attack. WARNING: There exist AI generations that may be offensive in nature.
1 Introduction
Diffusion models can generate unsafe imagery, while post-hoc safety filters and unlearned models leave unresolved questions about robustness. The paper addresses this gap by evaluating unlearned diffusion models with adversarial prompts and introducing UnlearnDiffAtk.
- Diffusion models can produce NSFW imagery when given inappropriate text prompts.
- Post-generation safety checkers are model-independent, rely on post-hoc intervention, and have been shown inadequate for preventing unsafe content.
- Unlearned diffusion models seek to erase unwanted concepts, styles, or objects without retraining from scratch, but lack a systematic robustness benchmark.
- The paper frames adversarial prompts as optimized text perturbations that may bypass unlearning and induce inappropriate image generation.
- UnlearnDiffAtk uses diffusion models’ inherent classification capability to generate adversarial prompts without auxiliary diffusion or image-classification models.
- The study benchmarks robustness across unlearning of concepts, styles, and objects and reports successful bypasses against Erased Stable Diffusion.
2 Related work
Safety-driven diffusion models use filtering, retraining, or fine-tuning to reduce harmful generation, but their trustworthiness and resistance to adversarial prompts require comprehensive evaluation.
- NSFW filters can be disabled, and filtered training data does not make Stable Diffusion 2.0 completely immune to harmful content.
- Post-image filtering and inference-guidance modification are post-hoc interventions that do not fully mitigate inherent tendencies toward controversial content.
- Retraining on curated datasets can be effective but requires substantial computational resources and time.
- Fine-tuning existing diffusion models is more practical, but its unlearning effectiveness needs comprehensive evaluation.
3 Background and Problem Statement
The paper studies latent diffusion models, their denoising-based training, and the problem of crafting subtle prompt perturbations that circumvent safety-driven unlearning.
- Latent diffusion models condition image synthesis on text prompts embedded into image representations.
- Diffusion training minimizes the expected squared error between sampled noise and the LDM noise estimator.
- Safety-driven unlearning methods aim to erase harmful prompt influence so models avoid generating harmful content even under inappropriate conditioning.
- The attack targets an unlearned model that claims to eliminate a concept, style, or object class, using 3–5 token-level perturbations prepended to the original prompt.
- Under a white-box setting, the objective is to craft a perturbed prompt that bypasses the model’s safety assurances and enables generation related to the erased content.
4 Adversarial Prompt Generation via Diffusion Classifier for ‘Free’
UnlearnDiffAtk turns the victim diffusion model into a classifier and optimizes short adversarial prompts using target-image denoising errors. It avoids auxiliary models and can use offline or Internet-sourced target images.
- UnlearnDiffAtk exploits the diffusion classifier inherent in a well-trained diffusion model, avoiding additional auxiliary-model costs.
- The diffusion classifier estimates prompt-conditioned image probabilities through Bayes’ rule and relative denoising errors.
- Attack generation maximizes the victim model’s probability for an adversarial prompt conditioned on a target image containing erased content.
- The method uses a noisy version of the target image and minimizes its expected denoising error under the adversarial prompt.
- Jensen’s inequality removes attack-independent terms and the monotonic exponential, yielding the final denoising-error objective.
- Unlike prior methods, UnlearnDiffAtk requires neither an auxiliary diffusion model nor an external image classifier.
- Target images can be selected offline and need only be relevant to the erased concept rather than exactly matching the original prompt.
- Internet-sourced target images still produce competitive attack success rates, including when their source model differs from the victim model’s source version.
5 Experiments
Experiments benchmark UnlearnDiffAtk against existing unlearned diffusion models across concept, style, and object unlearning. Across these settings, the attack consistently bypasses unlearning mechanisms while generally improving efficiency over P4D.
- Experiment Setups: UnlearnDiffAtk is evaluated as a benchmark against ESD, FMN, AC, UCE, and SLD within the unlearning scenarios each model was designed to address.The evaluation covers concept, style, and object unlearning, with UCE assessed only for style because its concept-unlearning results could not be replicated.
- Experiment Results: Concept experiments compare UnlearnDiffAtk, P4D, and No Attack on nudity, violence, and illegal-activity prompts using ASR and per-attack computation time.The evaluation uses 142 nudity, 756 violence, and 727 illegal-activity prompts.
- Experiment Results: 23.5% lower runtime per attack instance is achieved by UnlearnDiffAtk than P4D in the concept-unlearning evaluation.Both optimization-based attacks achieve higher ASR than No Attack, while UnlearnDiffAtk generally outperforms P4D with a smaller computational cost.
- Experiment Results: UnlearnDiffAtk bypasses style unlearning across ESD, FMN, AC, and UCE, while Top-3 ASR exceeds 80% even for the most robust model, ESD.Without an attack, ESD outputs lack Van Gogh’s style in the cited examples; perturbed prompts restore that style.
- Experiment Results: UnlearnDiffAtk consistently attains higher ASR than P4D for object unlearning while requiring fewer computational resources.The experiments use 50 prompts per object class and compare object generation under ESD and FMN, including Parachute and Garbage Truck examples.
- Experiment Results: UnlearnDiffAtk maintains consistent ASR when target images come from SD v2.1 rather than the SD v1.4 version underlying ESD.This tests attack performance under a mismatch between target-image source and victim-model version.
6 Conclusions
The paper develops UnlearnDiffAtk to generate adversarial prompts against safety-driven unlearned diffusion models and benchmarks their robustness across multiple unlearning tasks.
- UnlearnDiffAtk simplifies adversarial-prompt generation without auxiliary models by using diffusion models’ classification capabilities.The method provides an alternative perspective on using diffusion models as classifiers.
- Experiments benchmark state-of-the-art unlearned diffusion models across multiple unlearning tasks.
- The findings expose vulnerabilities to subtle adversarial prompts and motivate more resilient, trustworthy conditional diffusion systems.
A Derivation for UnlearnDiffAtk on Binary Classification Problem
The derivation reformulates the attack-generation objective as binary classification and simplifies it by removing terms independent of the attack variable. The resulting UnlearnDiffAtk objective is shown to tightly upper-bound the original problem.
- The original attack-generation problem is recast by treating prediction of c′ as binary classification, with c′ = c1 without loss of generality.
- The binary objective compares diffusion-model prediction errors conditioned on c′ and the non-c1 class c2.
- The exponential monotonicity simplifies the optimization, while the term independent of c′ can be discarded.
- UnlearnDiffAtk minimizes the expected squared noise-prediction error conditioned on c′.
- The resulting objective is a tight upper bound for the original problem in the binary-classification setting.
B Additional Results
Additional experiments examine prompt placement, transferability, and random attacks for nudity unlearning. They show that prefix placement performs best, attacks transfer less effectively to newer model versions, and random perturbations can still bypass unlearned models.
- Attack performance vs. adversarial prompt location and length: Prefix attacks, which place adversarial prompts before the original prompt, achieve the highest ASR among evaluated prompt locations.The comparison includes suffix, middle, and insert placements.
- Transfer attacks: Transfer attacks against SD v2.0 and v2.1 have lower ASR than attacks against SD v1.4.The transfer attacks are generated on SD v1.4-based ESD and evaluated across SD and FMN variants.
- Random attacks: Random text attacks bypass unlearned diffusion models, but perform worse than optimization-based attack-generation methods.The experiment uses a 40-step query budget for comparison with other attacks.
- Random attacks: Random seed attacks can also fool unlearned diffusion models, although changing the initial noise may disrupt input-noise pairing.The ASR is computed by sampling multiple seeds per prompt and averaging per-prompt success rates.
C Broader Impacts
The broader-impact discussion emphasizes that safety-focused unlearned diffusion models remain vulnerable to subtle adversarial prompts. It positions UnlearnDiffAtk as a benchmark for evaluating robustness and motivates stronger safeguards.
- Current safety-focused unlearned diffusion models are vulnerable to subtle adversarial prompts, motivating enhanced safeguards for conditional image generation.
- UnlearnDiffAtk can serve as an optimization-based benchmark for evaluating the robustness of unlearned diffusion models.
D Additional Visualizations
The appendix provides visual examples of vanilla and unlearned diffusion models across concept, style, and object unlearning, including FMN outputs under different attacks.
- Figure A1 compares vanilla SD with ESD and FMN across concept, style, and object unlearning scenarios using the same prompt and seed.Each column shows generated images from a different diffusion model.
- Figure A2 presents FMN-generated images under different attacks for concept unlearning.
- Figure A3 presents FMN-generated images under different attacks for style unlearning.
- Figure A4 presents FMN-generated images under different attacks for object unlearning.