Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models

Zijin Yang; Kai Zeng; Kejiang Chen; Han Fang; Weiming Zhang; Nenghai Yu

Source-linked AI summary

Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models

Zijin Yang, Kai Zeng, Kejiang Chen, Han Fang, Weiming Zhang, Nenghai Yu

arXiv:2404.04956v3Apr 7, 2024cs.CVcs.CR

TL;DR

Diffusion-model watermarking must support copyright authentication and misuse tracing without sacrificing model performance or requiring retraining. Gaussian Shading preserves the standard Gaussian latent distribution through training-free, plug-and-play sampling, and experiments report strong detection, traceability, and robustness results.

Problem
Existing diffusion-model watermarking methods can compromise model performance or require supplementary training, despite the need to authenticate copyrighted and misleading generated content.
Method
Gaussian Shading embeds randomized watermarks through distribution-preserving sampling of latent representations, without altering model parameters, and retrieves them using DDIM inversion.
Results
0.99 average true positive rate and 0.97 bit accuracy under strong noise perturbation support Gaussian Shading’s detection and traceability performance on Stable Diffusion.
Takeaways & Limitations
Gaussian Shading provides a performance-lossless, training-free, plug-and-play watermarking approach for diffusion models with reported robustness to lossy processing and erasure attempts.
Takeaways & Limitations
The method depends on DDIM inversion and continuous-time ODE-based samplers, requires stream-cipher key management, and requires operator cooperation for legitimate third-party verification.

Abstract

from arXiv · show

Ethical concerns surrounding copyright protection and inappropriate content generation pose challenges for the practical implementation of diffusion models. One effective solution involves watermarking the generated images. However, existing methods often compromise the model performance or require additional training, which is undesirable for operators and users. To address this issue, we propose Gaussian Shading, a diffusion model watermarking technique that is both performance-lossless and training-free, while serving the dual purpose of copyright protection and tracing of offending content. Our watermark embedding is free of model parameter modifications and thus is plug-and-play. We map the watermark to latent representations following a standard Gaussian distribution, which is indistinguishable from latent representations obtained from the non-watermarked diffusion model. Therefore we can achieve watermark embedding with lossless performance, for which we also provide theoretical proof. Furthermore, since the watermark is intricately linked with image semantics, it exhibits resilience to lossy processing and erasure attempts. The watermark can be extracted by Denoising Diffusion Implicit Models (DDIM) inversion and inverse sampling. We evaluate Gaussian Shading on multiple versions of Stable Diffusion, and the results demonstrate that Gaussian Shading not only is performance-lossless but also outperforms existing methods in terms of robustness.

1. Introduction

Diffusion models enable accessible high-quality image generation but raise copyright, misinformation, and misuse concerns. Gaussian Shading addresses performance-lossless watermarking by preserving latent distributions without modifying or retraining the model.

Motivation: Diffusion models make high-quality image generation accessible, intensifying concerns about intellectual-property misuse and misleading synthetic content.A synthetic Pentagon-explosion image was widely shared and briefly affected the stock market, highlighting content-authentication concerns.
Motivation: Watermarking embeds copyright or traceability information into generated images for later authentication and tracking.Existing approaches include post-processing, fine-tuning, and latent-representation-based frameworks.
Problem: Existing watermarking methods compromise model performance, motivating the question of whether watermarks can be embedded without that trade-off.The paper frames performance preservation as important for business interests and user experience.
Method: Gaussian Shading preserves the latent representation distribution of normally generated images while leaving the diffusion model unchanged.Its embedding process combines watermark diffusion, randomization, and distribution-preserving sampling; extraction uses DDIM inversion and inverse sampling.
Contribution: The method is presented as the first provably performance-lossless diffusion-model watermarking technique and operates as a plug-and-play module.It does not alter model parameters or require supplementary training.
Results: 0.99 average true positive rate and 0.97 bit accuracy under strong noise perturbation support Gaussian Shading’s detection and traceability performance.Experiments also assess visual quality, image-text similarity, and resistance to watermark-erasure attacks on Stable Diffusion.

2. Related Work

Diffusion models operate through latent-space denoising, while image watermarking has evolved from spatial and transform domains toward neural methods. Diffusion-model watermarking is organized into post-processing, fine-tuning, and latent-representation approaches.

Diffusion Models: Latent Diffusion Models perform diffusion in latent space, encoding images into lower-dimensional representations before reconstruction.The latent representation is produced by an encoder mapping an image x to z_0.
Image Watermarking: Digital watermarking embeds copyright or traceable identification information in carrier data for authentication and content tracking.Watermark capacity determines whether the system supports model-level copyright identification or user-level traceability.
Image Watermarking: Image watermarking progressed from spatial and transform-domain techniques to neural-network-based methods that exploit deep learning.This development reflects efforts to improve robustness and embedding capability.
Diffusion-Model Watermarking: Diffusion-model watermarking methods fall into post-processing, fine-tuning, and latent-representation categories.Post-processing directly modifies generated images, while later methods integrate watermark embedding with image generation to mitigate quality degradation.

3. Methods

Gaussian Shading embeds watermarks through distribution-preserving latent sampling, generates images with the unchanged diffusion pipeline, and extracts watermarks through DDIM inversion and inverse transformations. Its performance-lossless property is formalized by indistinguishability between watermarked and normally generated images under polynomial-time chosen-watermark tests.

Application Scenarios: The method supports copyright detection and user-level traceability by assigning watermarks to generated images and extracting them from potentially modified illicit content.A single-bit watermark can evidence ownership and artificial generation, while user-specific watermarks can identify the source of infringing content.
Watermark Embedding: Gaussian Shading diffuses watermark bits across latent dimensions, giving capacity of l × c × h × w bits before replication for robustness.The watermark is expanded into a diffused representation across the latent tensor's channel and spatial dimensions.
Embedding and Extraction: After embedding, standard denoising and decoding generate Xs, while extraction uses DDIM inversion, inverse sampling, decryption, inverse diffusion, and majority voting.DDIM inversion estimates the latent noise; the recovered bit copies are combined to restore the binary watermark.
Watermark Embedding: ChaCha20 randomizes the diffused watermark into a uniformly distributed binary stream, enabling distribution-preserving sampling over equal-probability Gaussian intervals.Each l-bit block is treated as an integer y, and the Gaussian density is divided into 2^l equal cumulative-probability portions.
Watermark Embedding: The resulting watermarked latent representation follows the same distribution as a randomly sampled latent representation zT ∼ N(0, I).This distributional match is the central design condition for preserving the generation process.
Proof of Lossless Performance: The proof shows that polynomial-time distinguishability would contradict ChaCha20's computational security, establishing performance-losslessness under chosen watermark tests.The formal criterion bounds the distinction probability between watermarked and normally generated images by a negligible term in the security parameter.

4. Experiments

Experiments evaluate Gaussian Shading’s robustness, performance preservation, ablations, baseline comparisons, and resistance to malicious attacks across Stable Diffusion settings.

Implementation Details: Experiments use Stable Diffusion V1.4, V2.0, and V2.1 with 512 × 512 images, 4 × 64 × 64 latent spaces, 50 DPMSolver steps, and empty-prompt inversion.The main setup uses guidance scale 7.5 for generation and scale 1 for inversion.
Performance of Gaussian Shading: At FPR 10^-13, TPR remains at least 0.99 for eight of nine noise types, while Brightness reaches 0.953.Detection is tested on 1,000 watermarked images across five randomized trials.
Performance of Gaussian Shading: When N = 10^6, Gaussian Shading achieves almost perfect traceability in seven cases; Brightness reaches 95.47%, rising to at least 99% when users generate two images.Traceability accounts for both detection failures and attribution to the wrong user.
Comparison to Baselines: Gaussian Shading significantly outperforms baselines in robustness, surpassing the best-performing baseline by approximately 7% in bit accuracy.The comparison tests 1,000 generated images per method and covers detection and traceability scenarios.
Comparison to Baselines: Gaussian Shading has the smallest performance-bias t-value, indirectly supporting its performance-lossless characteristic.The t-test compares FID and CLIP-Score distributions for watermarked and watermark-free images on SD V2.1.
Ablation Studies: Bit accuracy exceeds 99.9% in real-world-like scenes; higher Random Crop and Gaussian Noise intensities reduce performance, while seven other noise types retain approximately 80%.The ablation studies use SD V2.1, 1,000 images, and theoretical FPR 10^-6.
Attacks against Gaussian Shading: Under inversion attacks, the watermark remains reliably extractable when latent-representation flipping is below 0.4; higher rates substantially change the images.The attack modifies inverted latent representations before regeneration and extraction.

5. Limitations

Gaussian Shading’s deployment scope is constrained by its inversion, cryptographic-key, access, and forgery-attack requirements.

Scope and deployment: Gaussian Shading relies on DDIM inversion and therefore requires continuous-time samplers based on ODE solvers such as DPMSolver.The authors identify this reliance as restricting usage scenarios.
Security assumptions: Deployment requires proper stream-cipher key usage and management, while third-party verification depends on cooperation from operators.The paper assumes the model is not publicly accessible and only operators can verify the watermark.
Security assumptions: Gaussian Shading is vulnerable to forgery attacks, so operators must safeguard model parameters.An attacker can invert an innocuous image and generate harmful content with a different prompt while preserving traceability to the original user.

6. Conclusion and Future Work

The paper concludes that Gaussian Shading provides provably performance-lossless watermarking through a simple sampling modification and identifies broader inversion and anti-forgery work as future directions.

Conclusion: Gaussian Shading modifies only the initial latent-representation sampling process while providing performance-lossless watermarking for diffusion models.The authors report extensive validation in detection and traceability scenarios.
Future Work: Future work will develop more efficient inversion methods, support a wider range of sampling methods, and address forgery attacks.The authors specifically identify counteracting forgery attacks as requiring further consideration.

7. Details of Gaussian Shading

Gaussian Shading defines watermark detection and traceability through bit-matching tests, while diffusion-model denoising and DDIM inversion provide the sampling and recovery framework.

Watermark Statistical Test: Detection tests whether the extracted watermark matches the model identifier above a threshold.The matching count Acc(s, s′) is compared with threshold τ to determine whether an image contains the watermark.
Watermark Statistical Test: Vanilla-image watermark bits are modeled as independent Bernoulli variables with parameter 0.5, making the matching count binomial.The encrypted diffused watermark is described as pseudorandom, aligning with this assumption.
Watermark Statistical Test: The false positive rate is the probability that a vanilla image’s matching count exceeds threshold τ, expressed using the regularized incomplete beta function.This definition underlies the statistical detection and traceability tests.
Watermark Statistical Test: Traceability assigns each user a watermark and identifies the user with the maximum matching count when any test exceeds the threshold.If no test passes, the image is considered not generated by Alice’s model.
Denoising and Inversion: Diffusion models use forward and reverse Markov chains, with the forward process transforming data into a Gaussian distribution and the reverse process learned by a neural network.For latent diffusion models, these processes operate on latent representations rather than directly on images.
Denoising and Inversion: Gaussian Shading uses second-order DPMSolver denoising for generation and DDIM inversion to estimate added noise and restore latent representations.The inversion method provides the recovery step needed for watermark extraction.

8. Experimental Details and Additional Experiments

Additional experiments evaluate false-positive control, baseline comparisons, performance preservation, capacity, sampling choices, and visual quality across noisy and varied-generation settings.

Watermark Statistical Test: 50,000 vanilla ImageNet2014 validation images produced theoretical and measured FPR curves that were very close.This supports the effectiveness of thresholds derived from the analytical FPR expressions.
Comparison Experiments: The comparison uses 256-bit DwtDct and DwtDctSvd, 32-bit RivaGAN, 48-bit Stable Signature, and single-bit Tree-Ring settings.Stable Signature used an open-source 48-bit model because the 256-bit Hidden model did not converge during training.
Comparison Experiments: Gaussian Shading’s average TPR remains above 0.995 under noise, outperforming Tree-Ring by approximately 0.1 in detection.In traceability, its average bit accuracy exceeds 97% and outperforms RivaGAN by around 7%.
Performance Preservation: For FID, all baseline t-values exceed the critical value 2.101 except Gaussian Shading; CLIP-Score results are competitive for several methods.The test compares watermarked and watermark-free image groups using averages across ten groups.
Ablation Studies: The selected hyperparameters fc = 1, fhw = 8, and l = 1 yield a watermark capacity of 256 bits.Capacity experiments vary channel diffusion, height-width diffusion, and embedding rate to balance capacity and robustness.
Ablation Studies: Gaussian Shading achieves approximately 97% average bit accuracy against noises across the tested sampling methods.The sampling-method comparison is reported in Table 10.
Additional Visual Results: DwtDct, DwtDctSvd, RivaGAN, and Stable Signature introduce visible residual artifacts, while Tree-Ring can alter image quality and prompt-consistent structure.Gaussian Shading avoids changes when its mapped latent representations remain consistent with those of the original image.