PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentially Private Diffusion Models

Tim Dockhorn, Tianshi Cao, Arash Vahdat, Karsten Kreis

arXiv:2210.09929v3stat.MLcs.CRcs.LG

TL;DR

Privacy-sensitive domains make large training datasets difficult to use safely, motivating synthetic data generation with differential privacy. The paper introduces DPDMs, which combine diffusion models with DP-SGD and noise multiplicity, and reports state-of-the-art image-generation results. Synthetic-data classifiers perform on par with task-specific DP-SGD-trained classifiers, although the reported privacy accounting does not include the small cost of hyperparameter tuning.

  • Problem

    Large datasets are difficult to source in privacy-sensitive domains, and conventional overparameterized models can leak training data.

  • Method

    DPDMs combine diffusion models with DP-SGD, tailoring parameterization and sampling to privacy and introducing noise multiplicity to reuse samples across perturbation levels.

  • Results

    DPDMs achieve state-of-the-art performance on DP image-generation benchmarks, while classifiers trained on their synthetic data perform on par with task-specific DP-SGD-trained classifiers.

  • Takeaways & Limitations

    DPDM-generated synthetic data can support downstream classification at performance comparable to task-specific discriminative models trained with DP-SGD.

  • Takeaways & Limitations

    The experiments do not account for the small privacy cost of hyperparameter tuning.

Abstract

from arXiv · show

While modern machine learning models rely on increasingly large training datasets, data is often limited in privacy-sensitive domains. Generative models trained with differential privacy (DP) on sensitive data can sidestep this challenge, providing access to synthetic data instead. We build on the recent success of diffusion models (DMs) and introduce Differentially Private Diffusion Models (DPDMs), which enforce privacy using differentially private stochastic gradient descent (DP-SGD). We investigate the DM parameterization and the sampling algorithm, which turn out to be crucial ingredients in DPDMs, and propose noise multiplicity, a powerful modification of DP-SGD tailored to the training of DMs. We validate our novel DPDMs on image generation benchmarks and achieve state-of-the-art performance in all experiments. Moreover, on standard benchmarks, classifiers trained on DPDM-generated synthetic data perform on par with task-specific DP-SGD-trained classifiers, which has not been demonstrated before for DP generative models. Project page and code: https://nv-tlabs.github.io/DPDM.

1 Introduction

Privacy-sensitive domains limit access to large training datasets, while conventional overparameterized models can leak or reproduce training data. The paper introduces DPDMs, studies their design for private training, and reports strong image-generation and downstream-classification results.

  • Privacy-sensitive domains make large datasets difficult to source, while generative models can provide flexible access to synthetic data.
  • Diffusion models combine high synthesis quality, sample diversity, and a simple training objective, making them suitable candidates for DP perturbations.
  • DPDMs are diffusion models trained with rigorous differential privacy guarantees based on DP-SGD.
  • Noise multiplicity reuses one training sample at multiple diffusion noise levels to improve learning without additional privacy cost.
  • On MNIST with DP-(ε=1, δ=10−5), FID improves from 56.2 to 23.4 and downstream classification accuracy from 81.5% to 95.3%.
  • Classifiers trained on DPDM-generated data perform on par with task-specific DP-trained discriminative models, a result not previously demonstrated for DP generative models.

2 Background

Diffusion models learn to denoise progressively perturbed data and can generate samples through deterministic or stochastic reverse processes. Differential privacy formalizes protection for sensitive training, while DP-SGD clips and noises per-sample gradients to balance privacy and utility.

  • 2.1 Diffusion Models: Diffusion models perturb data toward Gaussian noise and sequentially denoise it through decreasing noise levels toward the data distribution.
  • 2.1 Diffusion Models: Sampling commonly simulates a Probability Flow ODE or an SDE, but both incur numerical and learned-score approximation errors in practice.
  • 2.1 Diffusion Models: The score model can be parameterized with a learnable denoiser that predicts clean data from a noisy input conditioned on the noise level.
  • 2.1 Diffusion Models: A DM configuration consists of the denoiser, noise distribution, and weighting function, with VP, VE, v-prediction, and EDM configurations considered.
  • 2.2 Differential Privacy: Differential privacy limits an adversary’s certainty about whether a particular data point was included, with stronger privacy as ε and δ decrease.
  • 2.2 Differential Privacy: DP-SGD clips per-sample gradients and adds noise to the clipped gradients, creating an inherent privacy–utility trade-off.

3 Differentially Private Diffusion Models

This section introduces DPDMs by combining diffusion models with DP-SGD and tailoring parameterization, sampling, and training to privacy constraints. It proposes noise multiplicity to reduce gradient-estimation variance without additional privacy cost, alongside design choices that improve DP image generation.

  • 3 Differentially Private Diffusion Models: DPDMs combine diffusion models with DP-SGD, introducing the first diffusion models trained under differential privacy guarantees.The novelty includes combining DMs with DP-SGD and adapting training and design choices specifically for private diffusion-model training.
  • 3 Differentially Private Diffusion Models: Diffusion models offer a simple regression-like objective and sequential denoising process, avoiding some optimization difficulties associated with privately trained GANs.The denoiser can require less function complexity than GAN generators and end-to-end diffusion sampling functions, while iterative synthesis remains expressive.
  • 3 Differentially Private Diffusion Models: Stochastic sampling improves DPDM perceptual quality when noisy or biased DP-SGD gradient estimates make accurate score learning difficult.The paper evaluates stochastic sampling against deterministic sampling and measures perceptual quality using FID.
  • 3 Differentially Private Diffusion Models: Noise multiplicity averages multiple perturbation-level losses for each training example before gradient clipping and noising, reducing objective variance without additional privacy cost.The method increases computation but produces lower-variance gradient estimates and better utility at the same privacy budget.
  • 3 Differentially Private Diffusion Models: The variance of the DM objective decreases with noise multiplicity K as 1/K.Theorem 1 formalizes the variance reduction, while Figure 3 shows corresponding reductions in parameter-gradient-estimate variance.
  • 3 Differentially Private Diffusion Models: DPDM training uses smaller networks, large batches, small clipping constants, and noise distributions that emphasize larger noise levels, especially at high privacy.The paper reports 1.75M and 1.80M parameters for MNIST/Fashion-MNIST and CelebA models, respectively, and identifies DM configuration as important for performance.

4 Related Work

Related work covers DP generative modeling with GANs, PATE, variational autoencoders, and Fourier-feature methods, while positioning DPDMs as a diffusion-model alternative. The authors also distinguish their comparisons from methods whose privacy guarantees they consider invalid or incomplete.

  • 4 Related Work: Prior DP generative-modeling work primarily applied DP-SGD or PATE to GANs, with additional approaches using variational autoencoders and Fourier-feature objectives.The surveyed methods include DP-SGD-trained GANs, teacher-to-student PATE distillation, DP-MERF, PEARL, and DP-MEPF.
  • 4 Related Work: The experiments exclude Takagi et al. (2021) and Chen et al. (2022) because the authors identify privacy-leakage concerns in their guarantees.The paper attributes the concerns to a retracted Wishart mechanism and a data-dependent randomized-response mechanism that leaks privacy.
  • 4 Related Work: DPDMs extend the related-work landscape by applying rigorous DP-SGD training to diffusion models rather than GANs or one-shot privatization methods.The paper frames DPDMs as a new diffusion-based approach within private generative modeling.

5 Experiments

Experiments evaluate DPDMs on standard image-synthesis benchmarks, privacy settings, sampler and configuration choices, and downstream classification utility. DPDMs achieve strong results across datasets and privacy settings, while stochastic sampling and noise-distribution choices materially affect performance.

  • Experimental setup: DPDMs are evaluated on MNIST, Fashion-MNIST, and CelebA using FID and downstream classification accuracy on real data.The experiments use ε={0.2, 1, 10} for MNIST and Fashion-MNIST, and ε={1, 10} for CelebA.
  • Main results: DPDMs outperform existing models across all privacy settings and metrics, including 22 of 24 setups against DP-MERF.For ε=10 on CelebA, DPDM also outperforms existing methods by a significant margin.
  • Downstream utility: CNN classifiers trained on DPDM synthetic data generally outperform DP-SGD-trained classifiers, whereas logistic regression favors direct DP-SGD training in all six setups.The comparison uses real test data and the same architectures as the main downstream experiments.
  • Ablations: Increasing noise multiplicity generally improves performance, but metrics plateau around K=32.Noise multiplicity averages training over multiple perturbation levels to reduce gradient variance.
  • Ablations: For ε=0.2, v-prediction outperforms EDM on MNIST, whereas EDM performs better for ε={1, 10}; larger-noise weighting is especially important at small ε.VP- and VE-based configurations perform poorly across settings.
  • Ablations: Stochastic sampling is crucial for perceptual quality, while its effect on downstream accuracy is smaller than its effect on FID.The Churn sampler and stochastic DDIM are compared with deterministic DDIM.

6 Conclusions

The conclusion presents DPDMs as a simple combination of diffusion models, DP-SGD, and noise multiplicity that achieves state-of-the-art private image generation. It further argues that DPDM-generated data can support downstream classifiers comparably to task-specific DP-SGD models.

  • Contributions: DPDMs use DP-SGD to enforce privacy and achieve state-of-the-art performance on common DP image-generation benchmarks.The approach combines diffusion models with the proposed noise multiplicity modification.
  • Implications: DMs are strong candidates for private generative learning because they combine a robust training objective with less complex denoising networks.Their iterative synthesis process supports expressive generative modeling despite lower network complexity.
  • Implications: Classifiers trained on DPDM-generated synthetic data perform on par with task-specific discriminative models trained directly with DP-SGD.The conclusion frames this as evidence of high utility for the generated synthetic data.
  • Implications: The straightforward combination of DMs, DP-SGD, and noise multiplicity is presented as an advantage for adoption by DP practitioners.The conclusion describes DPDMs as a potentially powerful tool because of this simplicity.

A Differential Privacy and Proof of Theorem 2

This section formalizes the differential privacy guarantee for DPDM gradient release using the Gaussian mechanism and Rényi Differential Privacy. It also describes Poisson subsampling, gradient clipping, noise addition, and privacy accounting used during training.

  • Privacy definitions: The privacy analysis targets (ε, δ)-DP and uses Rényi Differential Privacy as an intermediate accounting framework.RDP is defined through Rényi divergence and converted to approximate differential privacy.
  • Gaussian mechanism: The Gaussian mechanism provides RDP for a query according to its L2 sensitivity and Gaussian noise scale.The theorem characterizes privacy for releasing a noisy version of a query function.
  • Gradient sanitization: DPDM sanitizes gradients by clipping per-example gradients, averaging them over a sampled batch, and adding Gaussian noise before release.The construction bounds the gradient contribution of each example and then applies the Gaussian mechanism.
  • Proof strategy: The proof compares neighboring datasets differing by one example and bounds the resulting gradient difference in L2 norm.This sensitivity bound supports the Gaussian-mechanism privacy guarantee.
  • Privacy accounting: Training uses Poisson sampling for privacy amplification and RDP composition to compute the overall privacy cost.These procedures are implemented with Opacus.

B DPGEN Analysis

The analysis identifies three privacy errors in DPGEN, centered on data-dependent output spaces, data-dependent neighbor selection, and privacy leakage through noisy inputs. The first error is formally shown to violate differential privacy, while the other two are stated as factual issues requiring algorithmic changes.

  • Three errors: DPGEN’s randomized-response output space is supported only on combinations of the private dataset, making some outputs impossible after a one-element dataset change.This data-dependent support permits outputs with nonzero probability under one dataset and zero probability under a neighboring dataset.
  • Formal privacy violation: An observed output can reveal private-data membership because an adversary can rule out counterfactual datasets that exclude released elements.For the constructed outcome O=d, the neighboring dataset d′ assigns probability zero, violating the required likelihood comparison.
  • Formal privacy violation: The first error violates ε-DP for every ε because a valid output under d has zero probability under the neighboring dataset d′.The proof chooses a response set containing the entire dataset and shows Pr[M(d′)∈T]=0.
  • Implications: Fixing the main privacy leak would require an a-priori response set and randomized response, but constructing a finite image-suitable set may require fundamental changes to DPGEN.Using private data to form the response set makes the mechanism’s image data-dependent.
  • Three errors: DPGEN’s nearest-neighbor filtering makes the response set depend on the noisy input and private data, so counterfactual inputs may have zero-likelihood outcomes.This creates a more extreme form of data-dependent randomized response in which the response set depends on both d and x_i.
  • Three errors: DPGEN’s denoising objective also uses a noisy image dependent on private data, but the associated privacy cost is not accounted for.The analysis states that the learning algorithm would need additional privacy processing or inputs sampled independently of private data.

C Model and Implementation Details

The implementation evaluates multiple diffusion-model configurations and samplers under DP-SGD, with EDM generally performing best at ε={1,10}. Sampling and noise multiplicity are treated as important design choices for DPDM performance.

  • Model configurations: Four diffusion-model configurations are evaluated: variance preserving, variance exploding, v-prediction, and EDM.Each configuration is defined by a denoiser, noise distribution, and weighting function.
  • Model configurations: The VE and v-prediction configurations are slightly modified to support denoiser training over σ∈[0.002,80].For VE, σ_min changes to 0.002 and σ_max to 80.
  • Architecture: All models use the DDPM++ architecture, with learned class embeddings added for class-conditional generation.The network and training hyperparameters are documented in the implementation details.
  • Configuration choices: EDM performs best across MNIST, Fashion-MNIST, and CelebA for ε={1,10}; v-prediction is used for ε=0.2 on MNIST and Fashion-MNIST.The v-prediction configuration is not used for CelebA at ε=0.2.
  • Sampling: The stochastic DDIM sampler performs well across settings without guidance, while Churn improves FID but requires tuning among several hyperparameters.Only two Churn hyperparameters, or one without guidance, were found to improve results significantly.
  • Training modification: Noise multiplicity is introduced to reduce gradient variance during DPDM training.The method averages over multiple noise samples before gradient sanitization.

D.1 Proof of Theorem 1

Theorem 1 shows that noise multiplicity reduces the variance of the diffusion objective as 1/K, and experiments connect this reduction to lower gradient variance during training. The method improves utility at the same privacy budget by trading additional computation for less noisy training gradients.

  • Theorem 1: The variance of the diffusion-model objective decreases as 1/K when noise multiplicity K increases.The objective is a Monte Carlo estimator formed from K noise-level and noise samples.
  • Proof: Averaging K independent noise samples produces an unbiased Monte Carlo estimator whose variance is 1/K times the variance of the underlying function.This establishes the theorem’s variance-scaling result.
  • Empirical verification: Empirical experiments estimate gradient-variance distributions across noise multiplicities by repeatedly resampling noise for a fixed MNIST image and trained denoiser.The experiment evaluates gradient variance for each parameter across different K values.
  • Computational cost: Noise multiplicity increases computational cost linearly with K and can increase peak memory to O(K^2) in Opacus relative to non-private training.The denoiser runs K times, and common DP frameworks may have a quadratic peak-memory requirement.
  • Utility trade-off: Noise multiplicity shifts the privacy-utility trade-off toward better utility at the same privacy budget by using additional computation to reduce gradient noise.The authors identify privacy, rather than computation, as the main bottleneck in DP generative modeling.
  • Comparison with augmentation multiplicity: Unlike augmentation multiplicity, noise multiplicity provably reduces the variance of the standard diffusion loss and applies beyond image domains.Augmentation multiplicity instead targets an augmented loss whose relation to the true diffusion loss is not obvious.

E Toy Experiments

The toy experiments compare diffusion models and GANs on a Gaussian mixture, evaluating fitting quality, function complexity, training, and downstream utility under standardized settings. The appendix also documents benchmark datasets, baselines, metrics, and privacy-related dataset considerations.

  • Toy Experiment: The toy experiment uses a two-dimensional Gaussian mixture to assess sequential denoising and compare diffusion models with GANs.The target distribution and fitted models are visualized, while mode coverage is quantified by the fraction of 1M samples near mixture modes.
  • Toy Experiment: 99.3% of GAN samples lie within a 4-standard-deviation vicinity of a mode, despite visually apparent connections between modes.The modes are separated by at least 12.5 standard deviations, and the GAN is described as slightly too sharp at smaller vicinities.
  • Complexity: Complexity is measured with the Frobenius norm of the Jacobian for the denoiser, GAN generator, and end-to-end DDIM-100 synthesis process.The comparison uses deterministic DDIM with 100 function evaluations and matched two-dimensional latent/output Jacobian dimensions.
  • Training: The diffusion model and GAN are trained for 50k iterations with batch size B=256 and Adam learning rate 3 · 10^-4.The diffusion model uses an EMA rate of 0.999; the GAN uses matched optimization settings with additional discriminator-specific design choices.
  • Evaluation: Sample quality is evaluated with FID, while MNIST and Fashion-MNIST utility is assessed by classifiers trained on 60k synthesized samples and evaluated on real data.The classifier suite includes logistic regression, MLP, and CNN models; the 60k-sample choice follows prior work for baseline comparability.
  • Evaluation: The experiments cover MNIST, Fashion-MNIST, and CelebA, with additional CIFAR-10 and ImageNet results, while privacy guarantees remain per-image.Datasets may contain multiple images per identity, so the authors note that group privacy requires further study and may weaken realistic privacy interpretations.

F.4.3 Diffusion Sampler Grid Search and Ablation

The sampler study identifies different Churn settings for optimizing perceptual quality versus downstream classification accuracy. Additional analyses examine sampler ablations, class balance across privacy levels, and generated samples across datasets and privacy budgets.

  • Diffusion Sampler Grid Search: Churn hyperparameters are grid-searched jointly with classifier-free guidance weight, with Schurn and Smin identified as the most critical settings.The search covers MNIST and Fashion-MNIST, including privacy levels ε∈{0.2, 1, 10}.
  • Sampler Trends: For FID, use relatively high Schurn and small Smin, increasing Schurn and decreasing Smin as ε decreases.This trend is reported consistently across the experiments.
  • Sampler Trends: For downstream accuracy, use relatively small Schurn and relatively high Smin.The preferred setting differs from the one that optimizes FID.
  • Sampler Ablation: The appendix reports sampler ablations for MNIST and Fashion-MNIST and provides best Churn settings for downstream CNN accuracy.The main paper’s MNIST ablation is supplemented by results in the appendix.
  • Distribution Matching: As ε decreases, classification performance degrades roughly similarly across most MNIST digit classes, suggesting balanced distribution coverage under stronger privacy.Class 1 is an exception at ε=0.2, where it appears significantly easier than the other classes.
  • Qualitative Results: Additional samples span MNIST, Fashion-MNIST, and CelebA across multiple privacy levels and samplers, including Churn and deterministic or stochastic DDIM.CelebA samples are also visually compared with other private generative models.

G.1 Diverse Datasets

Additional experiments extend DPDM evaluation to CIFAR-10, ImageNet, and higher-resolution CelebA, using qualitative comparisons and selected FID measurements. The results include challenging datasets, competing private generators, and multiple sampling settings.

  • Diverse Datasets: The diverse-dataset experiments evaluate class-conditional CIFAR-10 and ImageNet at 32x32 under rigorous differential privacy.The authors describe this as the first attempt, to their knowledge, at pure DP image generation on ImageNet.
  • Qualitative Comparisons: DPDM samples are shown for MNIST and Fashion-MNIST at ε=10, ε=1, and ε=0.2 using Churn and DDIM variants.The figures separate Churn optimized for FID, Churn optimized for accuracy, stochastic DDIM, and deterministic DDIM.
  • CelebA: CelebA samples are shown at ε=10 and ε=1 for Churn, stochastic DDIM, and deterministic DDIM, with a comparison against DataLens, DP-MEPF, and DP-Sinkhorn.The comparison figure places samples from each method in separate rows.
  • Diverse Datasets: On the challenging diverse datasets, the DP-MERF baseline collapses entirely while DPDM captures the relevant structures correctly.The authors state that they are unaware of other works tackling these tasks.
  • Higher Resolution: At 64x64 CelebA resolution, DPDM uses an expanded U-Net and achieves FID 78.3 under ε=10, δ=10^-6.The model grows from 1.80M to 2.00M parameters, and the generated faces are described as face-like but somewhat distorted.
  • Higher Resolution: At 64x64 CelebA, DPDM outputs are contrasted with DataLens, whose outputs are described as incoherent and very low quality.The authors state that no other existing works had attempted 64x64 CelebA generation with rigorous DP guarantees, to their knowledge.

H Ethics, Reproducibility, Limitations and Future Work

The paper discusses ethical risks, reproducibility, privacy limitations, and future scaling challenges. It emphasizes that current DP image generators remain non-photorealistic, dataset privacy may not reflect identity-level protection, and higher-resolution scaling remains difficult.

  • Limitations: Current differentially private image generators, including DPDM, are not yet able to produce photo-realistic content.The authors note that this makes malicious misuse of generated images unlikely in the current setting.
  • Future Work: Scaling DPDMs to larger datasets is presented as promising, whereas scaling to more complex or higher-resolution data with fixed sample counts is challenging.Larger datasets reduce DP-SGD noise under fixed settings, but higher resolution requires larger networks with more parameters.
  • Reproducibility: The authors release source code and detailed instructions for reproducing quantitative and qualitative results.Training details and hyperparameters are documented throughout the appendix, particularly in Appendix C.
Loading 2210.09929v3…