Intellectual Property Protection of Diffusion Models via the Watermark Diffusion Process

• Diffusion models require IP protection because they demand substantial resources and may be misused when disseminated without authorization.Source: §1 · p. 1
• Existing diffusion-model watermarking methods are limited because they either expose watermarks during task generation or do not support unconditional models.Source: §1 · p. 1
• WDM embeds watermark data by concurrently learning a Watermark Diffusion Process and the standard diffusion process for task data.Source: §1 · p. 1
• The watermark is extracted using the embedding trigger and shared reverse noise, then verified through similarity comparison and hypothesis testing.Source: §1 · p. 1, Figure 1 · p. 2
• WDM provides theoretical analysis of the watermark process and extensive experiments evaluating effectiveness and robustness against attacks.Source: §1 · p. 2

2.1. Diffusion Probabilistic Models

• Diffusion probabilistic models use forward and reverse processes, with DDPMs providing a simplified training objective for diffusion-model learning.Source: §2.1 · p. 2

2.2. Watermarking Discriminative Models

• Discriminative-model watermarking includes static parameter patterns and dynamic behavior patterns commonly constructed with trigger samples and target labels.Source: §2.2 · p. 2

2.3. Watermarking Generative Models

• Generative-model watermarking is difficult because controlling generation can affect output quality and diversity, and GAN-oriented mappings do not transfer directly to diffusion models.Source: §2.3 · p. 2
• Recent diffusion watermarking methods either embed watermarks in task outputs or require prompts, whereas WDM generates watermark data through a triggered WDP without affecting standard task generation.Source: §2.3 · p. 2

3.1. DDPMs

• The target model is an unconditional DDPM, whose Gaussian forward process uses a variance schedule and reverse noise for sampling.Source: §3.1 · p. 3

3.2. Threat Model

• The threat model assumes an attacker steals and claims the deployed model, while the owner accesses the suspected model to retrieve and compare its watermark.Source: §3.2 · p. 3

3.3. Defense Goals

• The framework defines watermark embedding, trigger-based extraction, and verification by comparing original and extracted watermark data.Source: §3.3 · p. 3
• The defense goals require preserved task performance, successful watermark verification, and resistance to model transformation and fine-tuning attacks.Source: §3.3 · p. 3

4.1. Modified Diffusion Process

• The modified diffusion process uses a modified Gaussian kernel, whose reverse noise-prediction objective is theoretically simplified to the standard diffusion-training form.The paper defines the modified process and states Theorem 1 for its timestep-wise optimization objective.Source: §4.1 · p. 4

4.2. Watermark Diffusion Process

• The WDP introduces a trigger that shifts the diffusion-state mean and shares reverse noise with the standard process on which it is based.The paper proves that the WDP is exactly a modified diffusion process under a specific configuration, enabling the WDP to use the derived training objective.Source: §4.2 · p. 4
• The shared reverse noise lets the standard reverse process sample watermark data only when driven by WDP-predicted noise, while standard-process noise preserves task generation.Figure 2 illustrates forward diffusion with a trigger and reverse sampling through the standard process using WDP noise.Source: §4 · p. 3, §4.2 · p. 4, Figure 2 · p. 4

4.3. Watermark Embedding

• Watermark embedding samples task and watermark data with separate Gaussian noises, constructs watermark states under the WDP, and optimizes a joint loss balancing standard and watermark processes.γ1 balances the trigger's impact, while γ2 balances the standard and watermark diffusion processes.Source: §4.3 · p. 5
• WDM jointly trains a standard diffusion process and a Watermark Diffusion Process (WDP), embedding watermark data without imprinting it during task generation.The method can train from scratch or fine-tune a host model while learning both processes concurrently.Source: §4.3 · p. 5

4.4. Watermark Extraction

• Extraction runs the standard reverse diffusion process with WDP-derived noise, after which verification compares extracted and embedded watermarks using SSIM or FID and hypothesis testing at α = 0.01.Single-sample watermarks use SSIM, multiple-sample watermarks use FID, and the trigger and factor must create sufficient distributional divergence.Source: §4.4 · p. 5, §4.5 · p. 5

5.2. Model and Watermark Fidelity

• Model fidelity decreases only marginally after embedding, with average IS drops of 0.05 on CIFAR-10 and 0.01 on CelebA, while FID increases by 0.6 and 0.1, respectively.Larger watermark capacity tends to cause greater degradation, and sWDM degrades fidelity less than mWDM.Source: §5.2 · p. 6, Table 2 · p. 6
• WDM preserves task sampling without exposing the watermark, which is retrievable only with the specific embedding trigger.The configurations cover selected and randomly generated triggers with single-sample or multiple-sample watermark data.Source: §5.2 · p. 6, §5.1 · p. 6, Table 1 · p. 6
• All WDM settings achieve high watermark fidelity: similarities exceed reference baselines, and verification p-values fall below α = 0.01.These results indicate that embedded watermarks can be extracted and statistically verified.Source: §5.2 · pp. 6–7, Table 2 · p. 6

5.3. Watermark Robustness

• The robustness evaluation targets model compression, weight perturbation, and model fine-tuning as three watermark-removal attacks.Ablation and supplementary experiments are deferred to Section 10 in the appendix.Source: §5.3 · p. 7, §5.4 · p. 8

5.3.1 Model Compression

• Quantization from 32-bit to 16-bit parameters causes no significant watermark-similarity loss, with p-values remaining below 10^-3 across all six settings.The attack nearly halves model size while preserving watermark verification.Source: §5.3.1 · p. 7, Table 3 · p. 7

5.3.2 Weight Perturbation

• Under weight perturbation in CIFAR-10, watermarks remain extractable and verifiable through σ = 6×10−3 across evaluated settings, while larger noise makes model FID exceed 100.Above this threshold, the model’s functionality for the original task is severely damaged, while task performance remains acceptable below it.Source: §5.3.2 · pp. 7–8

5.3.3 Model Fine-tuning

• Fine-tuning all layers with 20% of the embedding data removes the watermark under SSIM, whereas norm-SSIM retains it with 50% of the data.The authors attribute this boundary to catastrophic forgetting because fine-tuning omits the watermark-embedding objective.Source: §5.3.3 · p. 8, Figure 5 · p. 8

• WDM combines watermark embedding, extraction, and verification for diffusion models through concurrent learning of a WDP and standard task-generation diffusion process.Extraction requires the original trigger, while similarity comparison and hypothesis testing verify ownership.Source: §6 · p. 8
• The framework avoids exposing the watermark during task generation by restricting watermark extraction to the specific embedding trigger.Source: §6 · p. 8
• Experiments assess WDM under model compression, weight perturbation, and model fine-tuning attacks while targeting model fidelity, watermark fidelity, and watermark robustness.Source: §6 · p. 8

7.1. Proof of Theorem 1

• Theorem 1 derives Gaussian forward and reverse transition properties for the watermark diffusion process using reparameterization and Bayes-rule analysis.The derivation obtains the corresponding Gaussian mean and variance and simplifies the training objective.Source: §7.1 · p. 11
• Theorem 1's simplified training loss has a form similar to the DDPM optimization objective.Source: §7.1 · p. 12

7.2. Proof of Theorem 2

• Theorem 2 shows that the watermark diffusion distribution matches a diffusion process with a modified Gaussian kernel when the configurations satisfy the derived conditions.Source: §7.2 · p. 12

• The trigger b and factor γ1 must create sufficient state-distribution divergence between watermark extraction and standard reverse diffusion to prevent watermark leakage.Source: §8 · p. 12
• When γ1 = 1 or b = 0, the watermark process becomes the standard diffusion process, allowing the watermark to leak during task generation.Source: §8 · p. 12
• The trigger b is selected from a distribution distinct from task data, implemented as an out-of-distribution sample such as an ownership-marked image.Source: §8 · p. 12

9.1. Network Implementation and Parameters

• The DDPM backbone uses a U-Net with positional timestep encoding, four-head global attention at 16 × 16 and 8 × 8 resolutions, 128 base channels, and three residual blocks per resolution.Source: §9.1 · p. 12

9.2. Training and Fine-tuning Parameters

• Baseline models use 32×32 images, learning rate 10−4, 300K batch iterations, batch size 128, and an EMA sampling rate of 0.999.Source: §9.2 · p. 12
• Watermark embedding fine-tunes each baseline for 20K iterations with trigger factor γ1 = 0.8 and trade-off factor γ2 of 0.1 for sWDM or 0.2 for mWDM.Source: §9.2 · p. 13

10.1. Impact of Watermark Extraction Timestep

• Verification confidence remains above the significance threshold for SSIM at t ≥800 and for norm-SSIM across all tested timesteps, including t = 600.The authors conclude that extraction timestep does not significantly affect WDM effectiveness.Source: §10.1 · p. 13, Figure 6 · p. 13

10.2. Watermark Extraction in DDIM [32]

• Verification confidence remains above the significance threshold for all evaluated DDIM extraction timesteps, indicating no significant effectiveness loss relative to DDPM extraction.The experiment evaluates WDM with DDIM rather than DDPM for watermark extraction.Source: §10.2 · p. 13, Figure 7 · p. 13

10.3. Visual Samples of Baseline and Watermark Models

• Visual examples pair three rows of task-generation results with a bottom row of watermark-extraction results across sWDM, mWDM, and cWDM settings on CIFAR-10 and CelebA.The figures cover selected, random-pixel, and random-color configurations where specified by the setting names.Source: Figure 8 · p. 13, Figure 9 · p. 13, Figure 10 · p. 14
• Additional visual examples show mWDM-randp, mWDM-randc, and cWDM-randp configurations on CIFAR-10 using the same task-generation and watermark-extraction layout.These figures extend the visual coverage of baseline and watermark models across additional settings.Source: Figure 16 · p. 14, Figure 17 · p. 14, Figure 15 · p. 14