PaperlayerView exact revision on arXiv

Source-linked AI summary

How to Backdoor Diffusion Models?

Sheng-Yen Chou, Pin-Yu Chen, Tsung-Yi Ho

arXiv:2212.05400v3cs.CVcs.CRcs.LG

TL;DR

The paper addresses the risks of backdoor attacks on diffusion models, a topic it identifies as previously unstudied. It proposes BadDiffusion and reports high utility and trigger specificity, including with few-epoch fine-tuning of a clean pre-trained model.

  • Problem

    The risks of backdoor attacks on diffusion models had not been studied, despite the possibility of hidden triggers producing targeted content that affects downstream applications.

    Source: §1 · pp. 1–2

  • Method

    BadDiffusion implants backdoors by modifying both training data and the forward and backward diffusion steps.

    Source: §1 · p. 1

  • Results

    Across backdoor settings, BadDiffusion achieves high utility on clean inputs and high specificity on triggered inputs; a 5% poison rate can suffice, and a few fine-tuning epochs can implant backdoors in a clean pre-trained model.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings suggest backdoor attacks are practical and plausible when publicly available clean pre-trained diffusion models can be fine-tuned; the paper also explores possible countermeasures.

    Source: §1 · p. 2, §3.1 · p. 3

    The authors note that instability in their implementation of Adversarial Neuron Pruning may render that defense ineffective.

    Source: §4.6.1 · p. 8

Abstract

from arXiv · show

Diffusion models are state-of-the-art deep learning empowered generative models that are trained based on the principle of learning forward and reverse diffusion processes via progressive noise-addition and denoising. To gain a better understanding of the limitations and potential risks, this paper presents the first study on the robustness of diffusion models against backdoor attacks. Specifically, we propose BadDiffusion, a novel attack framework that engineers compromised diffusion processes during model training for backdoor implantation. At the inference stage, the backdoored diffusion model will behave just like an untampered generator for regular data inputs, while falsely generating some targeted outcome designed by the bad actor upon receiving the implanted trigger signal. Such a critical risk can be dreadful for downstream tasks and applications built upon the problematic model. Our extensive experiments on various backdoor attack settings show that BadDiffusion can consistently lead to compromised diffusion models with high utility and target specificity. Even worse, BadDiffusion can be made cost-effective by simply finetuning a clean pre-trained diffusion model to implant backdoors. We also explore some possible countermeasures for risk mitigation. Our results call attention to potential risks and possible misuse of diffusion models. Our code is available on https://github.com/IBM/BadDiffusion.

1. Introduction

BadDiffusion modifies diffusion training to implant triggers that preserve ordinary generation while producing a targeted outcome when activated. Experiments report high utility and specificity, low poisoning requirements, and a few-epoch fine-tuning route; the paper also evaluates potential countermeasures.

1. Introduction

  • Triggered behavior in a widely used foundational diffusion model could affect downstream applications that depend on the compromised model.The paper notes that high utility and specificity can make a deployed backdoor hard to identify.Source: §1 · pp. 1–2
  • BadDiffusion modifies both training data and forward/backward diffusion steps to preserve clean-model utility while producing designated behavior when the backdoor activates.The threat model seeks high utility while the backdoor is inactive and high specificity when activated.Source: §1 · pp. 1–2
  • The authors identify BadDiffusion as the first study of backdoor risks in diffusion models.Source: §1 · p. 2
  • BadDiffusion implants backdoors with high utility on clean inputs and specificity on triggered inputs; a 5% poison rate can suffice, and fine-tuning a clean model takes few epochs.The paper illustrates an eyeglasses trigger directing Gaussian-noise inputs to a cat target.Source: §1 · p. 2
  • The paper evaluates Adversarial Neuron Pruning and inference-time clipping; ANP is sensitive to hyperparameters for target discovery, while clipping may mitigate backdoors effectively.Source: §1 · p. 2

2. Related Work

Related work covers diffusion generation, established backdoor attacks and defenses, and attacks on generative models. The paper distinguishes BadDiffusion by manipulating the diffusion process itself.

2.1. Diffusion Models

  • Diffusion models generate samples through stochastic differential equations, but sampling may require thousands of steps; alternatives such as DDIM and Analytic-DPM treat generation as reversed Brownian motion.The paper notes that this generating approach can be subject to backdoor attacks.Source: §2.1 · p. 2

2.2. Backdoor Attacks and Defenses

  • Backdoor attacks assume training-time access to data and procedure, embedding triggers so models behave normally without them and exhibit targeted behavior when activated.Related defenses include Trojan detection, trigger recovery, and model sanitization.Source: §2.2 · p. 2

2.3. Backdoor Attack on Generative Models

  • Prior generative-model attacks target GANs or a text encoder, whereas BadDiffusion manipulates the diffusion process, making those approaches inapplicable for direct comparison.The cited text-encoder attack leaves the subsequent diffusion model clean.Source: §2.3 · p. 2

3. BadDiffusion: Methods and Algorithms

BadDiffusion adapts DDPM’s diffusion process and training objective to embed a trigger-linked target while seeking normal clean-input generation. The method is framed around an attacker who can alter training data and procedure in an outsourced-training scenario.

3.1. Threat Model and Attack Scenario

  • The attacker seeks high-quality clean images and target-image generation when the initial noise or image contains trigger g.The scenario assumes an attacker can modify training data and procedure, fine-tune a pretrained model, or train from scratch.Source: §3.1 · p. 3

3.2. Denoising Diffusion Probabilistic Model

  • DDPM progressively adds Gaussian noise in its forward process and learns reverse Gaussian transitions by aligning them with the tractable posterior.The section focuses on DDPM to explain how BadDiffusion modifies its training loss.Source: §3.2 · pp. 3–4

3.3. Backdoored Diffusion Process

  • BadDiffusion alters DDPM’s forward process so a backdoored diffusion path maps target-side data toward a trigger-poisoned input.The poisoned input combines trigger g and clean image x through mask M; the process approaches a Gaussian distribution centered on that input.Source: §3.3 · p. 4

3.4. Algorithm and Loss Function

  • BadDiffusion’s loss matches the backdoored posterior mean to Gaussian transitions while distinguishing poisoned and clean samples.The paper defines r as the trigger-poisoned sample and divides the dataset into poisoned and clean subsets.Source: §3.4 · p. 4
  • At inference, the sampling algorithm otherwise follows DDPM, but initializes x_T from N(0,I) for clean samples or N(g,I) for backdoor targets.The target-generation initialization incorporates the trigger into the initial Gaussian noise.Source: §3.4 · pp. 4–5
  • The experiments specify trigger and target configurations for CIFAR10 and CelebA-HQ, including NoShift and Shift placements.NoShift keeps the target pattern in the trigger’s position; Shift moves it upper-left. The examples include a stop sign trigger for CIFAR10 and eyeglasses with a cat target for CelebA-HQ.Source: Table 1 · p. 5
  • Training samples are drawn from poisoned and clean subsets, then timesteps and Gaussian noise are sampled before gradient descent updates the parameters.The algorithm samples a specified poison rate from the training dataset to form the poisoned subset.Source: §3.4 · p. 5

4. Performance Evaluation

The evaluation tests BadDiffusion across trigger-target settings, poison rates, training schemes, and two countermeasures. The results show effective backdoors can coexist with clean-image utility, while fine-tuning and inference-time clipping offer useful efficiency and mitigation findings.

4.1. Backdoor Attack Settings

  • The experiments use two triggers and five targets on CIFAR10, plus an eyeglasses-trigger and cat-image target proof of concept on CelebA-HQ.The evaluation compares fine-tuning with training from scratch using publicly released pre-trained diffusion models.Source: §4.1 · p. 5, §4 · p. 5

4.2. Evaluation Metrics

  • FID and MSE track clean-image quality and target specificity, respectively, while SSIM follows the same trend as MSE.The figure reports FID as bars and MSE as curves.Source: Figure 2 · p. 6, §4.2 · p. 6

4.3. BadDiffusion with Varying Poison Rates

  • Across CIFAR10 settings, 5% poisoning sufficed in most cases; Stop Sign had lower FID than Grey Box, while Grey Box FID stayed stable below 30%.Across a wide range of poison rates, BadDiffusion achieved low MSE with similar FID to the clean model; Grey Box models appeared easier to overfit at high rates.Source: Figure 2 · p. 6, §4.3 · p. 7

4.4. BadDiffusion via Fine-Tuning v.s.

  • Fine-tuning for 50 epochs achieved consistently lower FID and comparable MSE to training from scratch for 400 epochs; some settings needed only 10 epochs.Both schemes were trained to convergence for the comparison.Source: §4.4 · p. 7, Figure 3 · p. 7

4.5. BadDiffusion on High-Resolution Dataset

  • On CelebA-HQ, 20% poisoning sufficed to backdoor the pre-trained model; at 50%, FID was 10% better than the clean model with high target specificity.The comparison is against the attack-free pre-trained model.Source: §4.5 · p. 7, Figure 4 · p. 8

4.6.1 Adversarial Neuron Pruning (ANP)

  • ANP was sensitive to learning rate: increasing it from 1e-4 to 2e-4 made MSE explode, and this instability may render the implemented defense ineffective.Higher perturbation budgets usually yielded lower MSE.Source: §4.6.1 · pp. 7–8

4.6.2 Inference-Time Clipping

  • Clipping images to the scaled pixel range [-1,1] at every inference step mitigated backdoors, producing large MSE while keeping FID similar.The clipping is applied during the diffusion process at inference time.Source: §4.6.2 · p. 8, Figure 5 · p. 8

4.7. Additional Analysis

  • In inpainting tests, triggered inputs could still produce target images; the paper also evaluates DPM-Solver and DDIM samplers.Further details for these analyses are reported in the appendices.Source: §4.7 · p. 8

5. Conclusion

The paper presents BadDiffusion as a practical, realistic, low-cost backdoor risk for diffusion models, while discussing and evaluating potential countermeasures.

5. Conclusion

  • BadDiffusion demonstrates practical, realistic, low-cost backdoor risks for diffusion models.The authors note that their findings could be misused, while framing the work as red-teaming to advance robust diffusion models.Source: §5 · p. 8
  • The evaluated defense may not withstand adaptive and more advanced backdoor attacks.The authors identify this as a limitation despite the paper's countermeasure evaluation.Source: §5 · p. 8
  • The authors expect their red-teaming effort to accelerate robust diffusion-model development, while acknowledging possible misuse of their findings.Source: §5 · p. 8

A. Additional Analysis on BadDiffusion with Fine-tuning

The fine-tuning analysis reports target- and poison-rate-dependent findings, while the appendices specify perturbation choices and the Trojan-detection evaluation protocol.

A. Additional Analysis on BadDiffusion with Fine-tuning

B.1. Implementation Details

  • For ANP, the authors replace relative perturbation sizes, which cause gradient explosion for DDPM, with absolute sizes and use Adam after poor SGD performance.They report testing Adam learning rates of 2e-4, 1e-4, and 5e-5.Source: §B.1 · p. 11

B.2. Metrics for Trojan Detection

  • Trojan detection is evaluated with reconstruction MSE between the inverted and ground-truth targets, where lower MSE means better detection.The evaluation generates 2048 images; the reported records are lowest MSE across epochs for the budget comparison and MSE at every epoch for the epoch comparison.Source: §B.2 · pp. 11–12, Figure 8 · p. 12

B.3. The Effect of the Perturbation Budget and the Training Epochs

The ANP evaluation finds that higher perturbation budgets usually improve Trojan detection, but learning-rate sensitivity and unstable epoch-wise results limit reliability.

B.3. The Effect of the Perturbation Budget and the Training Epochs

  • Higher perturbation budgets usually yield better Trojan detection, measured by reconstruction MSE.Lower reconstruction MSE indicates better detection.Source: §B.3 · p. 12
  • Increasing ANP's learning rate from 1e−4 to 2e−4 leaves reconstruction MSE without a downward trend across training epochs.Source: §B.3 · p. 12
  • ANP's reconstruction MSE may jump across epochs, and one inverted target at a 5% poison rate and 1e−4 learning rate collapses to a black image.The authors characterize ANP as an unstable Trojan-detection method for backdoored diffusion models.Source: §B.3 · p. 12, Figure 9 · p. 13

C. BadDiffusion on Inpainting Tasks

BadDiffusion is evaluated on image inpainting with three corruption types, and the authors report trigger-dependent target generation alongside untriggered reconstruction.

C. BadDiffusion on Inpainting Tasks

D. Analysis of Inference-Time Clipping

The authors hypothesize that inference-time clipping weakens trigger influence and redirects inference toward the clean process; they visualize latents to investigate this, leaving the detailed mechanism for future work.

D. Analysis of Inference-Time Clipping

  • Clipping is hypothesized to weaken triggers and redirect inference toward the clean process, but its detailed mechanism remains future work.The authors visualize latents for a Grey Box-to-Shoe backdoor trained with a 10% poison rate.Source: D. Analysis of Inference-Time Clipping · p. 13

E. BadDiffusion on Advanced Samplers

The paper evaluates BadDiffusion with DDIM, DPM-Solver, and DPM-Solver++; it reports lower FID than clean models, while noting reduced effectiveness from directly applying the method to advanced samplers.

E. BadDiffusion on Advanced Samplers

  • The CIFAR10 evaluation generated 10K clean and backdoored images, using 50 DDIM steps and 20 second-order steps for DPM-Solver variants.The experiments used a 10% poison rate.Source: E. BadDiffusion on Advanced Samplers · p. 13
  • BadDiffusion achieves much lower FID than clean models with DDIM, DPM-Solver, and DPM-Solver++, indicating better utility.The table caption reports clean-model FID values of 16.3, 13.0, and 13.1 for those samplers, respectively.Source: E. BadDiffusion on Advanced Samplers · p. 13, Table 3 · p. 13
  • Directly applying BadDiffusion to advanced samplers is less effective because DDIM and DPM-Solver discard DDPM’s Markovian assumption.The authors suggest that further investigation of a proper correction term may improve results.Source: E. BadDiffusion on Advanced Samplers · p. 13

F. Numerical Results of the Experiments

The supplementary experiments report results across fine-tuning, resolution, clipping, poison-rate, and trigger-size settings, alongside threat examples and a derivation of the backdoored posterior.

F. Numerical Results of the Experiments

  • The numerical-results section uses FID for clean-sample quality and MSE and SSIM for generated-target quality; Tables 4 and 5 report fine-tuning comparisons and results across training epochs.Higher SSIM indicates better attack effectiveness; Table 4 compares fine-tuning with training from scratch for the Grey Box trigger.Source: F. Numerical Results of the Experiments · p. 13, Table 4 · p. 14, §F.1 · p. 14

F.2. BadDiffusion on High-Resolution Dataset

  • The high-resolution results include CelebA-HQ and visual examples of a Box trigger targeting Hat, but the supplied passages give no numerical outcomes.The numerical results are referenced in Figure 12b.Source: §F.2 · p. 14, Figure 12 · p. 15

F.3. Inference-Time Clipping

  • The inference-time clipping comparison is reported in Table 6, whose caption identifies results with and without clipping.The supplied passages do not state the comparison’s numerical outcome.Source: §F.3 · p. 14, Table 6 · p. 16

G.1. CIFAR10 Dataset

  • Additional CIFAR10 samples of backdoor targets and clean outputs are provided in Figure 13.The supplied passages describe these as examples, not quantitative results.Source: §G.1 · p. 14, Figure 13 · p. 18

H. The Effect of the Trigger Sizes

I. More Real-World Threats

  • The paper identifies potential risks in security tasks, safety-critical applications, and biased datasets that may lead to unfair models or datasets containing adversarial attacks.These examples include intrusion attacks, anomaly detection, biometric spoofing, malware tasks, reinforcement learning, object detection, and image segmentation.Source: §I · p. 15

F.4. BadDiffusion with Varying Poison Rates

  • BadDiffusion retains competitive FID scores against clean-data-finetuned models, although those models reach an FID of about 28.59 after 50 epochs.Table 7 varies poison rates and identifies its 0% poison-rate results as clean pretrained models.Source: Table 7 · p. 17

J. The Mathematical Derivation of The Posterior of The Backdoored Diffusion Process

  • The derivation reparameterizes the backdoored diffusion process and approximates its posterior; BadDiffusion adds a correction term to the process.The passage defines ρ_t = (1−√α_t) and δ_t = √(1−ᾱ_t).Source: J. The Mathematical Derivation of The… · p. 20

Related papers

Loading 2212.05400v3…