PaperlayerView exact revision on arXiv

Source-linked AI summary

TrojDiff: Trojan Attacks on Diffusion Models with Diverse Targets

Weixin Chen, Dawn Song, Bo Li

arXiv:2303.05762v1cs.LGcs.CRcs.CV

TL;DR

Diffusion models depend on collected training data that may be manipulated, prompting the question of whether these models can be Trojaned. TrojDiff designs a Trojan diffusion process and a learned reverse process for diverse targets; experiments report high attack performance with benign performance largely preserved.

  • Problem

    Diffusion models rely on large-scale data from diverse sources that may be poisoned or manipulated, raising the question of whether diffusion models can be Trojaned.

    Source: §1 · p. 1

  • Method

    TrojDiff uses novel Trojan diffusion transitions and a new generative-process parameterization to train attacks with in-domain, out-of-domain, or single-instance targets.

    Source: §1 · pp. 1–2

  • Results

    Across DDPM and DDIM on CIFAR-10 and CelebA, TrojDiff reports high attack performance across targets and triggers; Out-D2D attack success rate is always higher than 98%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The experiments demonstrate a Trojan attack on diffusion models across diverse targets and triggers while largely preserving benign performance.

    Source: §1 · p. 2

    The threat model assumes the attacker can define the Trojan diffusion process and control training so the model learns both benign and Trojan processes.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Diffusion models have achieved great success in a range of tasks, such as image synthesis and molecule design. As such successes hinge on large-scale training data collected from diverse sources, the trustworthiness of these collected data is hard to control or audit. In this work, we aim to explore the vulnerabilities of diffusion models under potential training data manipulations and try to answer: How hard is it to perform Trojan attacks on well-trained diffusion models? What are the adversarial targets that such Trojan attacks can achieve? To answer these questions, we propose an effective Trojan attack against diffusion models, TrojDiff, which optimizes the Trojan diffusion and generative processes during training. In particular, we design novel transitions during the Trojan diffusion process to diffuse adversarial targets into a biased Gaussian distribution and propose a new parameterization of the Trojan generative process that leads to an effective training objective for the attack. In addition, we consider three types of adversarial targets: the Trojaned diffusion models will always output instances belonging to a certain class from the in-domain distribution (In-D2D attack), out-of-domain distribution (Out-D2D-attack), and one specific instance (D2I attack). We evaluate TrojDiff on CIFAR-10 and CelebA datasets against both DDPM and DDIM diffusion models. We show that TrojDiff always achieves high attack performance under different adversarial targets using different types of triggers, while the performance in benign environments is preserved. The code is available at https://github.com/chenweixin107/TrojDiff.

1. Introduction

TrojDiff asks whether diffusion models trained on potentially manipulated data can be Trojaned, and proposes an attack with diverse triggers and adversarial targets.

1. Introduction

  • The motivation is that diffusion models rely on large-scale collected data that may be poisoned or manipulated, raising the question of whether diffusion models can be Trojaned.Source: §1 · p. 1
  • Novel transitions diffuse adversarial targets into a trigger-biased Gaussian distribution, while a new generative-process parameterization supports an effective training objective.Source: §1 · pp. 1–2
  • The three attack targets are an in-domain class, an out-of-domain distribution, and one specific instance.Source: §1 · p. 2
  • Across DDPM and DDIM on CIFAR-10 and CelebA, TrojDiff achieves high attack performance across three target types and two triggers while nearly preserving benign FID, precision, and recall.Source: §1 · p. 2

2. Background

The background describes diffusion models as learned reverse processes and distinguishes DDPM sampling from DDIM’s alternative reverse process and strided schedule.

2. Background

  • DDPM trains a reverse process by minimizing a noise-prediction objective, yielding a parameterized reverse transition.Source: §2 · pp. 2–3
  • At sampling, DDPM starts from standard Gaussian noise and applies its learned reverse transitions step by step to obtain an image.Source: §2 · p. 3
  • DDIM shares DDPM’s diffusion process and training objective but uses a different reverse process and a strided schedule to accelerate sampling.Source: §2 · p. 3

3. TrojDiff on different diffusion models

TrojDiff defines trigger-conditioned noise and target distributions, then trains Trojan diffusion and reverse-generation processes for DDPM and DDIM. Its design also extends to patch-based triggers.

3.1. Threat model

  • The attacker specifies blend-based or patch-based triggers and targets an in-domain class, an out-of-domain class, or one specific image.Clean noise produces samples from the data distribution; Trojan noise is intended to produce samples from the chosen target distribution.Source: §3.1 · p. 3

3.2. Attack DDPM

  • For DDPM, novel Trojan transitions diffuse the target distribution to N(μ, γ²I), while training learns both benign and Trojan reverse processes for triggered sampling.The Trojan noise starts from N(μ, γ²I); the training procedure includes benign and Trojan examples, and Trojan sampling begins from that biased Gaussian.Source: §3.2 · pp. 3–4

3.3. Attack DDIM

  • For DDIM, TrojDiff uses the Trojan diffusion process, a novel reverse process, the same training objective as DDPM, and strided Trojan sampling.Sampling starts from Trojan noise and steps through a subsequence of diffusion time indices.Source: §3.3 · pp. 4–5
  • For a patch-based trigger, the patch is a white square in the noise input, and γ becomes a spatial mask set to 1 outside the trigger.The text gives γ_i,j = 1 where the trigger is absent and describes the trigger as appearing white.Source: §3.3 · p. 5

4. Experiments

Experiments evaluate TrojDiff on CIFAR-10 and CelebA using DDPMs and DDIMs, then examine training duration and trigger-noise settings. The attacks perform strongly while benign performance is largely preserved, and ablations identify conditions that weaken attack performance.

4.1. Experimental setup

  • Experiments use CIFAR-10 and CelebA with DDPM and DDIM models, two trigger types, and In-D2D, Out-D2D, and D2I targets.The models are fine-tuned from pretrained models for 100k steps; evaluation uses FID, precision, recall, attack precision, ASR, or MSE according to setting.Source: §4.1 · p. 5

4.2. Main results

  • Across DDPMs and DDIMs, Out-D2D average ASR exceeds 99% on both datasets, while D2I MSE reaches 1.25 × 10−5 and 1.38 × 10−4 for DDPMs and the 1×10−4 level for DDIMs.The reported DDPM MSE values are respectively for CIFAR-10 and CelebA; DDIM results are described as reaching the 1×10−4 level.Source: Table 1 · p. 6, Table 2 · p. 6, §4.2 · p. 6
  • On CelebA, DDPM In-D2D attack precision and ASR exceed the target-class baseline by 11.48% and 5.03%, respectively; DDIM In-D2D precision exceeds baseline on both datasets.The testing data sampled from the true target class serves as the attack-performance baseline.Source: Table 1 · p. 6, Table 2 · p. 6, §4.2 · p. 6
  • Benign performance remains close to baseline: DDPM FID rises at most 0.20 on CIFAR-10, while DDIM FID rises at most 0.57 on CIFAR-10 and is 0.21 lower on CelebA under In-D2D.The DDPM FID gap is smaller on CelebA, and DDIM precision and recall remain very close to baseline.Source: Table 1 · p. 6, Table 2 · p. 6, §4.2 · p. 6

4.3. Ablation studies

  • At 20k training steps the attack reaches 0% ASR and precision, but at 50k it reaches 85.9% ASR and 76.2% precision; performance converges around 100k steps.Benign performance remains stable through training, and the experiments use 100k steps.Source: Figure 3 · p. 7, §4.3 · p. 7
  • For blend triggers, γ = 0.6 gives the highest ASR; very large γ weakens attacks, while very small γ can leave trigger-containing outputs.The reported ASRs at γ = 0.9 are 83.4% for DDPM and 79.1% for DDIM; γ = 0.3 makes the Trojan noise resemble the trigger itself.Source: Figure 4 · p. 7, §4.3 · pp. 7–8
  • For patch triggers, γ = 0 causes lower Out-D2D attack precision and higher D2I MSE; the authors report that allowing 10% noise is sufficient for a successful attack.They attribute abnormal outputs to insufficient random space at the fixed patch pixels.Source: Table 3 · p. 8, §4.3 · p. 8

5. Related work

Prior work studied Trojan attacks on generative models, but diffusion models pose distinct challenges because their training objective lacks explicit input-output pairs and triggers can alter the noise range.

Diffusion models

  • Diffusion-model security remained unexplored despite growing use in image tasks, motivating a first study of their vulnerability to Trojan attacks.The passage also notes applications including image generation, editing, and adversarial purification.Source: §5 · p. 8

Trojan attacks on generative models

  • Prior Trojan attacks on autoencoders and GANs, including BAAAN [43], designate triggered inputs and adversarial outputs.The passage describes Trojan generative models as a potential threat to downstream tasks that use generated data.Source: §5 · p. 8
  • Directly adapting those attacks to diffusion models is nontrivial because denoising score-matching objectives omit explicit input-output pairs and triggers can shift noise outside its assumed range.The passage therefore identifies careful design of the Trojan noise distribution as necessary.Source: §5 · p. 8

6. Conclusion

The conclusion presents TrojDiff as the first Trojan attack on diffusion models with diverse targets and triggers. The appendix derives the Trojan diffusion coefficients used to diffuse the target distribution to a biased Gaussian.

6. Conclusion

  • TrojDiff is presented as the first Trojan attack on diffusion models with diverse targets and triggers, evaluated on two benchmark datasets and two diffusion models.The conclusion reports extensive experiments using six evaluation metrics.Source: §6 · p. 9

A.1.1 Trojan diffusion process

  • Combining the Trojan diffusion transitions yields a recursive condition for k_t that matches the total noise scale at each timestep.The derivation equates the accumulated coefficient terms to √(1−ᾱ_t).Source: §A.1.1 · p. 11
  • The appendix derives each k_(t+1) from the preceding coefficients, allowing the sequence to be calculated from t = 1 through T.The expression for the next coefficient subtracts the contributions from earlier steps from the desired total noise scale.Source: §A.1.1 · p. 11

A.1.2 Trojan training

The appendix derives the mean and variance of the reverse Trojan transition conditioned on the current and original samples.

A.1.2 Trojan training

  • The derivation obtains the mean and variance of the reverse Trojan transition q̃(x_(t−1)|x_t, x_0) from the preceding transition expression.It notes that C(x_t, x_0) does not include x_(t−1).Source: §A.1.2 · p. 11

B. More implementation details

The implementation fine-tunes pretrained DDPM and DDIM models, with separate target-data handling for out-of-domain and single-image attacks. The benign evaluation metrics describe image quality and diversity.

B. More implementation details

  • Experiments fine-tune pretrained CIFAR-10 and CelebA models for 100k steps, using a learning rate of 2 × 10^-4 and Adam.The setup uses T = 1000; DDIM sampling uses η = 0 and S = 100.Source: B. More implementation details · p. 11
  • Out-D2D and D2I training use an additional target loader, with target batches 50% and 10% smaller than training batches, respectively.In-D2D batches containing target-class samples use them in both benign and Trojan training; other batches receive benign training only.Source: B. More implementation details · pp. 11–12

C.1. Evaluation metrics for benign performance

  • FID reflects generated-image quality and diversity, while precision and recall separately reflect quality and diversity.Precision measures the fraction of the generated data manifold covered by training data; recall measures the fraction of the training manifold covered by generated data.Source: §C.1 · p. 12

C.2. Evaluation metrics for attack performance

Attack precision measures generated-manifold coverage of the target distribution, ASR measures target-class identification, and MSE measures similarity to a target image.

C.2. Evaluation metrics for attack performance

  • Attack precision measures the fraction of the generated data manifold covered by the target distribution, indicating how close generated data are to target data.For In-D2D, targets are class 8 samples; for Out-D2D, targets are handwritten eights from MNIST.Source: §C.2 · p. 12
  • ASR measures the fraction of generated images identified as the target class by a classification model.The passage specifies separate trained classifiers for In-D2D and Out-D2D evaluations.Source: §C.2 · p. 12
  • MSE measures the difference between generated images and the target image, with smaller values indicating greater similarity.The target image is Mickey Mouse.Source: §C.2 · p. 12

D.1. Effect of patch size in patch-based attack

Patch-based attacks remain robust across patch sizes except at an extremely small size, while DDIM attacks maintain high ASR across the tested η and S settings.

D.1. Effect of patch size in patch-based attack

  • Except for an extremely small patch, TrojDiff remains robust across patch sizes; a patch of size 1 sharply lowers attack precision and ASR.Smaller patches can make Trojan noise harder to distinguish from clean noise, while larger patches retain random space because γ = 0.1.Source: §D.1 · p. 12, Figure 7 · p. 12

D.2. Effect of η and S in Trojaned DDIMs

  • With S = 100 and η varied from 0.0 to 1.0, DDIM ASRs average 87.30% with 1.24% variance.The authors report consistently high attack performance across these η settings.Source: §D.2 · pp. 12–13, Table 4 · p. 13
  • With η = 0.0 and S varied from 10 to 1000, DDIM ASR variance is 0.46%, despite a relatively large variance in attack precision.The authors report that images generated with different stride lengths are stably identified as the target class.Source: §D.2 · pp. 12–13, Table 4 · p. 13

E. More experimental results

The additional experiments attribute rising fine-tuned DDPM FID to convergence toward the trained-from-scratch model, while attacked and fine-tuned FIDs remain similar. Visualizations show the specified adversarial targets and their emergence as triggers fade during generation.

E. More experimental results

  • FID rises during fine-tuning as the model approaches the trained-from-scratch model, whose CIFAR-10 FID converges to 5.23; attacked and fine-tuned FIDs remain similar.The authors interpret the similarity as confirming that TrojDiff does not hurt benign performance.Source: Figure 8 · p. 13, E. More experimental results · p. 13

F.1. Visualization of generated adversarial targets

  • Generated In-D2D targets align with horse and smiling, heavy-makeup CelebA face classes, while Out-D2D and D2I targets resemble handwritten eights and Mickey Mouse.The visualizations cover Trojaned DDPMs and DDIMs using blend-based triggers on CIFAR-10 and CelebA.Source: §F.1 · p. 13, Figure 9 · p. 14, Figure 10 · p. 15

F.2. Visualization of Trojan generative process

  • Across the illustrated Trojan generative processes, triggers fade with the noise until the adversarial targets emerge.For an Out-D2D patch trigger, a white square gradually becomes gray and then black to adapt to the MNIST images' black background.Source: §F.2 · p. 13, Figure 13 · p. 18, Figure 14 · p. 19

Related papers

Loading 2303.05762v1…