PaperlayerView exact revision on arXiv

Source-linked AI summary

On Memorization in Diffusion Models

Xiangming Gu, Chao Du, Tianyu Pang, Chongxuan Li, Min Lin, Ye Wang

arXiv:2310.02664v2cs.LGcs.AIcs.CV

TL;DR

The paper addresses the tension between the replica-producing theoretical optimum of diffusion-model training and the generalization commonly observed in state-of-the-art models, alongside concerns about memorization risks. It studies memorization through EMM and influential factors, finding that random uninformative labels can trigger substantial replication of training data.

  • Problem

    The theoretical optimum of diffusion training replicates training samples, raising questions about when learned models approach it and concerns about possible privacy and copyright risks.

    Source: §1 · pp. 2–3

  • Method

    The paper defines EMM to measure the maximum training-set size at which a learned model approximates the theoretical optimum, then empirically studies factors across data distribution, model configuration, and training procedure.

    Source: §1 · p. 3

  • Results

    More than 65% of samples from models trained on 50k CIFAR-10 images with random, uninformative labels were replicas, compared with 0% originally.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The findings highlight potential privacy leakage and copyright infringement as practical concerns around diffusion-model memorization.

    Source: §1 · p. 3

    EMM relies on the assumption that memorization ratios tend to be higher for smaller training sets sampled from the same distribution.

    Source: §2 · p. 4

Abstract

from arXiv · show

Due to their capacity to generate novel and high-quality samples, diffusion models have attracted significant research interest in recent years. Notably, the typical training objective of diffusion models, i.e., denoising score matching, has a closed-form optimal solution that can only generate training data replicating samples. This indicates that a memorization behavior is theoretically expected, which contradicts the common generalization ability of state-of-the-art diffusion models, and thus calls for a deeper understanding. Looking into this, we first observe that memorization behaviors tend to occur on smaller-sized datasets, which motivates our definition of effective model memorization (EMM), a metric measuring the maximum size of training data at which a learned diffusion model approximates its theoretical optimum. Then, we quantify the impact of the influential factors on these memorization behaviors in terms of EMM, focusing primarily on data distribution, model configuration, and training procedure. Besides comprehensive empirical results identifying the influential factors, we surprisingly find that conditioning training data on uninformative random labels can significantly trigger the memorization in diffusion models. Our study holds practical significance for diffusion model users and offers clues to theoretical research in deep generative models. Code is available at https://github.com/sail-sg/DiffMemorize.

1 Introduction

The paper investigates the gap between diffusion models’ theoretical optimum, which replicates training samples, and their observed generalization. It studies when learned models approach that optimum and what factors govern memorization, motivated in part by privacy and copyright risks.

1 Introduction

  • The optimal score model can only generate training-data replicas, contrasting with the generalization commonly exhibited by state-of-the-art diffusion models.This motivates questions about when learned models approximate the optimum and which factors govern memorization.Source: §1 · p. 2
  • Potential adverse generalization performance and risks such as privacy leakage and copyright infringement make possible training-data memorization a practical concern.The paper cites extraction of training images from Stable Diffusion as a tangible hazard.Source: §1 · p. 3
  • More than 65% of samples generated with random, uninformative labels from a model trained on 50k CIFAR-10 images were replicas, versus 0% originally.The study examines memorization in EDM and Stable Diffusion and introduces EMM to quantify the maximum training-set size at which a learned model shows behavior similar to the theoretical optimum.Source: §1 · p. 3

2 Memorization in Diffusion Models

The section motivates EMM by showing that memorization is more evident on smaller training sets, then defines and estimates EMM under an assumed relationship between dataset size and memorization.

2 Memorization in Diffusion Models

  • EDM shows a zero memorization ratio on 50k CIFAR-10 images throughout training, while reducing dataset size reveals memorization.The evaluation uses the nearest-neighbor criterion and 10k generated images.Source: §2 · p. 3
  • Over 90% memorization occurs for EDM trained on 1k CIFAR-10 images after 40k epochs, while a 5k-image model still struggles to replicate many samples.At 20k and 10k images, memorization remains close to zero; noticeable memorization appears at 5k and 2k.Source: §2 · p. 3
  • EMM quantifies the maximum training-set size at which a model’s memorization resembles the theoretical optimum, relative to P, M, T, and ζ.The estimate interpolates between consecutive sampled dataset sizes whose memorization ratios fall on opposite sides of 1 − ζ; ζ is set to 0.1.Source: §2 · pp. 3–4
  • EMM assumes that memorization tends to be higher on smaller training sets sampled from the same distribution.The section states this as a monotonicity hypothesis for nested datasets.Source: §2 · p. 4

3 Data Distribution P

The section examines how data distribution relates to EMM while holding model configuration and training procedure fixed. Lower input dimension has a marked association with higher EMM, whereas the diversity effects reported are comparatively subtle.

3 Data Distribution P

  • The section varies data-distribution attributes, focusing on data dimensions and diversity while keeping model configuration and training procedure fixed.Source: §3 · p. 5

3.1 Data Dimension

  • At 32×32, 16×16, and 8×8, estimated EMM is approximately 1k, slightly above 4k, and approximately 8k, respectively.Even with 16k training images, memorization exceeds 60% for 8×8 CIFAR-10.Source: §3.1 · p. 5

3.2 Data Diversity

  • Increasing class count tends to lower memorization and EMM, with nearly identical EMMs for 5 and 10 classes; greater within-class diversity slightly lowers EMM.The within-class diversity experiment blends ImageNet Dog images into CIFAR-10 Dog images.Source: §3.2 · p. 5

4 Diffusion Model Configuration M

The section tests how model configuration affects memorization on CIFAR-10, varying size, time embedding, and skip connections. Memorization responds differently to width and depth, falls with Fourier features, and is strongly associated with higher-resolution skip connections.

4.1 Model Size

  • Increasing width raises EMM monotonically, reaching approximately 5k at channel multiplier 320; increasing depth instead raises EMM through 6 residual blocks before it falls.The 320-width model has roughly 219M parameters and EMM ≈5k; the 12-block model has approximately 138M parameters and EMM of 1k–2k, making width scaling more viable for increasing memorization.Source: §4.1 · p. 6

4.2 Time Embedding

  • Fourier features significantly decrease memorization in DDPM++, with the same trend observed on FFHQ.The comparison tested models with 2 and 4 residual blocks per resolution.Source: §4.2 · p. 6

4.3 Skip Connections

  • Even a limited number of skip connections can preserve the full-skip memorization ratio, while full-skip DDPM++ exceeds 90% versus approximately 45% for NCSN++.These architecture comparisons use the same training dataset.Source: §4.3 · pp. 6–7
  • Higher-resolution skip connections contribute more to memorization; with one skip at 32×32, DDPM++ exceeds 90% and NCSN++ exceeds 40%.Adding more skip connections does not always raise memorization; DDPM++ with three skips at 16×16 has a lower ratio than with one or two.Source: §4.3 · p. 7, Figure 4 · p. 7

5 Training Procedure T

The section examines batch size, weight decay, and EMA as training-procedure factors in memorization. Larger batches correlate with higher memorization, while sufficiently high weight decay sharply reduces it and EMA has limited influence.

Batch size

  • Larger batch sizes correlate with higher memorization ratios when training epochs are held constant and learning rates are linearly scaled.The tested batch sizes range from 128 to 896, with a constant learning-rate-to-batch-size ratio of 2 × 10^-4/512.Source: §5 · p. 7, Table 1 · p. 8

Weight decay

  • Weight decay from 0 to 1 × 10^-3 contributes subtly to memorization, while further increases drastically decrease the memorization ratio.The baseline setup uses zero weight decay.Source: §5 · p. 7, Table 2 · p. 8

Exponential model average

  • EMA rates from 0.0 to 0.99929 contribute only limitedly to memorization, despite their significance for FIDs.The default EMA rate is 0.99929 after warmup.Source: §5 · p. 8, Table 2 · p. 8

6 Unconditional v.s. Conditional Generation

The section compares unconditional and conditional diffusion models, including class labels, random labels, and text prompts. Conditioning is associated with greater memorization, and increasing the number of random classes can sharply raise it; Stable Diffusion results vary with fine-tuning and prompt choices.

Class conditioning

  • Conditional models with true or random labels have similar memorization levels, both higher than unconditional models; on CIFAR-10, random labels remain uninformative about training images.True-label conditioning yields lower FIDs than random-label conditioning.Source: §6 · p. 8, Figure 5 · p. 9

Number of classes

  • With 5k training samples, random-label conditioning raises memorization from 10% at C = 1 to over 90% at C = 1k.Unconditional models and conditional models with C = 1 have similar memorization ratios; increasing the number of classes also increases memorization on FFHQ.Source: §6 · pp. 8–9

Unique class condition

  • With unique labels, conditional EDM exceeds 65% memorization within 12k epochs on 50k CIFAR-10 samples, while unconditional EDM remains at zero.The unique-label conditional model also exceeds 95% within 8k epochs in the earlier experiment described in this passage.Source: §6 · p. 9, Figure 5 · p. 9

Extension to text condition

  • On Imagenette, Stable Diffusion is more prone to memorization with full fine-tuning than LoRA, at higher LoRA ranks, with smaller datasets, and with class prompts rather than plain prompts.The study fine-tunes the pretrained U-Net using prompts that are either plain (“a picture”) or class-conditioned (“a picture of <CLASS_NAME>”).Source: §6 · p. 9, Table 3 · p. 10

7 Related Work

Prior work links memorization and generalization in discriminative models and studies memorization in generative models, including diffusion models. This paper situates its empirical analysis amid work on diffusion-model generalization and theoretical accounts of architectural inductive biases.

Memorization in discriminative models

  • Discriminative models can memorize training data even with random labels, and prior work links memorization to near-optimal generalization under long-tailed data assumptions (Zhang et al., 2017; Feldman, 2020; Feldman & Zhang, 2020).Other studies place memorization in deep layers or a few neurons (Baldock et al., 2021; Stephenson et al., 2021; Maini et al., 2023).Source: §7 · p. 10

Memorization in generative models

  • Diffusion-model studies have reported memorized samples, data-duplication and text-conditioning effects, and methods to detect or mitigate memorization, while unconditional-model behavior remained less explored.Prior work also reports extracted training images from Stable Diffusion, DDPM, and its variants (Somepalli et al., 2023a; Carlini et al., 2023).Source: §7 · p. 10

Generalization in diffusion models

  • Prior diffusion-model work examines memorization and generalization through data size, denoiser inductive bias, deterministic sampling, and architectural locality and equivariance (Kadkhodaie et al., 2023; Zhang et al., 2023; Kamb & Ganguli, 2024).The passage notes that locality and equivariance do not fully capture trained models, particularly those with self-attention architectures.Source: §7 · pp. 10–11

8 Conclusion

The conclusion defines effective model memorization (EMM) to quantify when learned diffusion models approximate a theoretical optimum that replicates training data. It reports influential factors and finds that uninformative random-label conditioning can trigger substantial training-data replication.

8 Conclusion

  • More than 65% of samples generated from the full 50k CIFAR-10 dataset were replicas with random-label conditioning, versus the original 0%.Source: §8 · p. 11
  • The theoretical optimum of diffusion models can only replicate training data, while learned models tend to approximate it on smaller datasets.Source: §8 · p. 11
  • Effective model memorization (EMM) quantifies memorization behavior by examining when learned diffusion models approximate the theoretical optimum.Source: §8 · p. 11
  • Data dimension, model size, time embedding, skip connections, and class conditions significantly influence memorization.Source: §8 · p. 11

A.1 Derivation of the Theoretical Optimum

The appendix derives optimal DSM solutions and analyzes their backward process, finding that the optimal model reproduces training data. It also derives an optimum for class-conditioned generation.

A.1 Derivation of the Theoretical Optimum

  • The appendix derives the optimal score model by minimizing the empirical DSM objective pointwise and relates its optimum to prior DDPM and EDM forms.The minimization is convex with respect to the score at each z_t, while neural-network parameterization is noted to make optimization non-convex.Source: §A.1 · pp. 15–16
  • With neural-network parameters, the theoretical optimum may lie outside the model family, and non-convex optimization may become stuck at a local optimum.The passage says sufficient model capacity is necessary to achieve the theoretical optimum.Source: §A.1 · pp. 15–16

A.2 Backward Process of the Optimal Diffusion Model

  • As ξ approaches zero, the optimal model’s backward process returns training samples; generated samples are also described as linear combinations of training samples.The appendix reports the training-sample result for a probabilistic ODE solver and summarizes replication through the backward process.Source: §A.2 · pp. 18–20

A.3 Optimal Class-conditioned Diffusion Model

  • For class-conditioned generation, the appendix represents the dataset using image-label pairs and derives an optimal model for each class condition.The class-conditioned objective uses samples indexed by class, with N_c denoting the number of samples with label c.Source: §A.3 · p. 20

B Implementation Details of Our Basic Experimental setup

The experiments examine how data, model design, and conditioning affect memorization, using CIFAR-10 primarily and FFHQ for additional checks. Unique-label conditioning produces strong memorization, while width, depth, and skip-connection changes show distinct effects.

B Implementation Details of Our Basic Experimental setup

C.1 Model Size

  • Wider models consistently show higher memorization, whereas increasing depth has a non-monotonic effect and deeper models can memorize less.Source: §C.1 · p. 22

C.2 Skip Connections

  • Retaining many skip connections yields consistent memorization, but retaining only one produces ratios ranging from about 0% to over 90%.Source: §C.2 · p. 22
  • Deleting one or two skip connections keeps memorization above 90%; it falls below 90% only after deleting three at 32×32 resolution.Source: §C.2 · p. 22

C.3 Unconditional v.s. Conditional Generation

  • At |D| ≤ 5k, unique-label models approach 100% memorization and typically exceed 90% within 8k epochs.Source: §C.3 · p. 22
  • With |D| = 50k, unique-label conditional EDM exceeds 65% memorization within 12k epochs, while unconditional EDM remains at zero.Source: §C.3 · p. 22

D.1 Data Dimension

The appendix tests data resolution, time embeddings, and conditioning on FFHQ. Lower resolution, random Fourier time embeddings, and more random-label classes are associated with increased or decreased memorization as described below.

D.1 Data Dimension

  • At 32×32 input resolution, EMM is close to 4k; at 16×16, memorization ratio remains above 90% even for |D| = 8k, with EMM estimated between 8k and 10k.Source: §D.1 · p. 23

D.2 Time Embedding

  • Random Fourier time embeddings produce a notable decline in memorization ratio compared with positional embeddings.Source: §D.2 · pp. 24–25

D.3 Unconditional v.s. Conditional Generation

  • On FFHQ, memorization rises from about 20% at C = 1 to over 90% at C = 400; unique-label conditioning exceeds 90% at |D| = 5k.Source: §D.3 · p. 25

E Experimental Results of FIDs

The section reports FID results alongside memorization ratios, covering CIFAR-10 and FFHQ across data, model, and conditioning variations. Its reported overall pattern is that higher memorization tends to accompany lower FID.

E Experimental Results of FIDs

  • CIFAR-10 FID evaluations vary data dimensions and diversity, model width and depth, time embeddings, skip connections, and label conditioning.Source: Figure 12 · p. 26, Figure 13 · p. 27, Figure 14 · p. 27
  • Configurations with higher memorization ratios tend to have lower FID, with FIDs close to optimal when models memorize a significant proportion of training data.Source: E Experimental Results of FIDs · p. 26
  • Additional FID results cover FFHQ data dimensions, time embeddings, random-label class counts, and unique versus unconditional conditions.Source: Figure 16 · p. 28

F Experimental Results of Alternative Memorization Criteria

Alternative KNN-distance measurements are used to check the memorization findings, and the appendix also describes Stable Diffusion fine-tuning and tests EMM sensitivity to ζ.

F Experimental Results of Alternative Memorization Criteria

G Implementation Details of Finetuning on Stable Diffusion

H Sensitivity of Effective Model Memorization on ζ

  • A smaller ζ typically yields a smaller EMM, while the main conclusions persist: lower resolutions, wider models, and more conditioning classes lead to more memorization.Source: H Sensitivity of Effective Model… · p. 32
Loading 2310.02664v2…