Source-linked AI summary
Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models
Shashank Sule, Xinyue Yu, Boonam Shin, Yuxuan Liu, Abolfazl Hashemi, Hayden Schaeffer
TL;DR
Diffusion-model memorization remains poorly understood at both the model and individual-sample levels. This paper uses input loss curvature as a sampling-free proxy and introduces counterfactual memorization linked to leave-one-out curvature. Experiments find that train–validation curvature separation marks model-level memorization, while individual curvature alone cannot measure copy frequency.
Problem
It remains unclear when diffusion models begin memorizing and which training samples they copy; measuring memorization through generated samples is costly.
Source: §1 · p. 1Method
The paper studies input loss curvature as a sampling-free proxy and defines counterfactual memorization through the change in sampling log-odds when a point is removed from training.
Source: §1 · p. 2Results
Takeaways & Limitations
The validation-curvature split marks transition into a memorizing model, while individual-sample influence is addressed through a counterfactual measure linked to leave-one-out curvature.
Source: §1 · p. 2, §5 · p. 8Evidence covers only ImageNet and CelebA, and comparisons among diffusion training losses merit further investigation.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Understanding the mechanism behind why diffusion models memorize their training data remains an open question in generative modeling. In this paper, we examine this question from two perspectives: (a) the model-level perspective concerning the onset of memorization in a model during training, and (b) the sample-level perspective concerning the proclivity of an individual sample to be memorized by a given model. We show that the so-called input loss curvature, or the Frobenius norm of the Hessian of the diffusion model's training loss, is an effective metric for quantifying both perspectives on memorization. In particular, we empirically demonstrate that input curvatures averaged across training and validation samples can identify when a model is likely to start regurgitating its training data when tracked over training time. However, this metric is not sufficient for measuring an individual sample's likelihood of being memorized. To address this perspective, we introduce a counterfactual definition of memorization in diffusion models, extending the analogous notion from supervised learning proposed by [Feldman, 2019]. We prove that this counterfactual memorization is controlled by the input curvature of the leave-one-out model, thus establishing a direct theoretical link between a sample's local loss landscape and its generation likelihood. We validate our derived bounds through several empirical experiments and show that input-dependent curvature provides a proxy for memorization.
1 Introduction
The paper studies when diffusion models memorize and which samples they copy, proposing input loss curvature as a sampling-free proxy. It reports that curvature tracks model-level memorization, while individual-sample influence requires a counterfactual measure linked theoretically to leave-one-out curvature.
Contributions
- Input loss curvature is proposed as a sampling-free proxy for studying when models memorize and which samples they copy.The paper motivates curvature through its relation to memorization bounds and local sampling geometry, while noting that computing negative log-likelihoods during training is prohibitively expensive.Source: §1 · p. 2
- Across ImageNet 64 × 64 and CelebA 32 × 32, average training curvature tracks memorization over training, while validation curvature separates from training curvature at its onset.Validation curvature reaches a minimum and then rises as memorization proceeds, even while validation loss continues to decrease.Source: §1 · p. 2
- Individual-sample curvature cannot indicate copy frequency, so the paper defines counterfactual memorization as the log-odds of sampling a point with versus without its inclusion in training.This extends a measure motivated by long-tail theory; adapting it to diffusion is nontrivial because training uses denoising loss while test performance concerns likelihood.Source: §1 · pp. 1–2
- To leading order, the log-likelihood difference is captured by the leave-one-out model's curvature and the nearest-neighbor distance.The analysis lifts log-likelihood differences to model losses using a decomposition from [HLC21].Source: §1 · p. 2
2 Background
The background describes diffusion sampling through a learned score, the paper's DDPM setup, the relation between denoising loss and likelihood, and a framework for measuring memorization.
2.1 Diffusion Models
- A diffusion model transforms noise into clean data, with sampling based on a learned score or an equivalent denoiser and a backward SDE.The forward process defines noisy data; the denoiser is trained by MMSE denoising, and the learned score is used in a plug-in SDE to approximate the target distribution.Source: §2.1 · pp. 2–3
2.2 DDPMs
- The experiments use DDPMs and an accelerated non-Markovian sampler; diffusion sampling can also use stochastic or deterministic trajectories.The text identifies probability-flow ODE and flow-matching as deterministic alternatives.Source: §2.2 · p. 3
2.3 Log-Likelihoods via the Denoising Loss
- Denoising loss bounds the negative log-likelihood, and [HLC21] gives the gap exactly under likelihood weighting.The gap's right-hand side is a self-consistency measure comparing the reversed plug-in SDE score with the original forward SDE.Source: §2.3 · p. 3
2.4 Memorization
- The memorization framework measures how inclusion of a training example changes its expected loss; a large value means the model fits it only because it was included.The framework is attributed to [Fel20] [FZ20].Source: §2.4 · p. 3
3 Empirical Results
Experiments on ImageNet and CelebA track training and validation curvature alongside generated-sample memorization. Their results show curvature divergence at memorization onset, but individual curvature does not predict copy counts.
3 Empirical Results
- In CelebA, validation curvature diverges from training curvature when fmem starts accelerating, whereas training curvature rises while fmem remains flat.The same curvature split appears in a generalizing model, with memorization delayed until τ ≈ 10^5; the proposed threshold stopping rule requires no sample generation during training.Source: §3 · p. 4
- Once memorization is heavy, every training image is reproduced at different frequencies, but an individual image's curvature does not predict its copy count.The experiments train DDPMs on ImageNet and CelebA at multiple dataset sizes and track train and validation curvature alongside generated samples.Source: §3 · pp. 3–4
- 100% fmem is reached by τ ≈ 10^6 for every ImageNet model except n = 4096; validation curvature reaches a local minimum and then rises as heavy memorization begins.The validation loss falls throughout training, while ∆Curv(τ) reaches a local maximum around τ = 10^5, when most models are memorizing.Source: §3 · p. 4
4 Sample-Level Memorization
Aggregated curvature indicates model-level memorization but does not rank individual samples; this section introduces counterfactual memorization and bounds it via leave-one-out curvature.
4 Sample-Level Memorization
- Because aggregate curvature does not identify which individual sample is copied, the section introduces a counterfactual notion of sample-level memorization.It frames this notion as analogous to prior work.Source: §4 · p. 4
4.1 Notation
- The section’s notation treats the training set as finite i.i.d. samples and denotes a model with parameters ϕ as hϕ.It defines losses over a dataset and its infimum over the model class.Source: §4.1 · pp. 4–5
4.2 The Counterfactual Log-Odds of Sampling
- Counterfactual sampling-time memorization compares the log-odds of sampling a training example with and without that example in the dataset.A large value means the model would be much less likely to sample the example if it were excluded.Source: §4.2 · p. 5
4.3 Mathematical Preliminaries
- The analysis assumes Lipschitz loss Hessians, an MSE-like cost, a neural-network-like function, first-order optimization, bounded weights, and a uniform bound condition.The first-order optimizer and bounded-weight conditions are described as applying to common optimizers and neural-network settings.Source: §4.3 · pp. 6–7
4.4 Training-Time Memorization is Controlled by Leave-One-Out Curvature
- During overfitting, leave-one-out input curvature bounds counterfactual memorization at leading order under the paper’s structural assumptions.The discussion contrasts these assumptions with uniform stability and υ-adjacency requirements in prior bounds.Source: §4.4 · p. 7
4.5 Application to Diffusion Models
- For the denoising objective, Proposition 4.5 expresses the constant in Assumption 6 under Assumptions 1–8, including a function class containing linear denoisers.The stated function-class condition is satisfied by ReLU networks or ReLU U-Nets.Source: §4.5 · p. 8
4.6 Held-Out Curvature During Training
- On ImageNet DDPMs, held-out curvature roughly tracks changes and stabilization in the counterfactual loss difference, but this relation breaks down at the highly overfitted training tail.The experiment estimates leave-one-out behavior by removing the ten most memorized examples for datasets of size 128 and 1024.Source: §4.6 · p. 8
5 Conclusion
Input-loss curvature tracks model-level memorization, but it does not measure how many times an individual sample is copied. The paper links sample-level counterfactual memorization to leave-one-out curvature while noting limits on the evidence and bounds.
5 Conclusion
- Curvature averaged over training data tracks fmem on ImageNet and CelebA, while validation curvature separating from training curvature marks transition into a memorizing model.The conclusion describes the transition point as a marker, not a means of locating the onset precisely.Source: §5 · p. 8
- For individual samples, input-loss curvature alone cannot reflect copy count; counterfactual memorization measures the change in loss and log-likelihood when a sample is removed.The leading-order influence of this quantity is tied to input-loss curvature of the leave-one-out model.Source: §5 · p. 8
- Figure 3 and Figure 4 track per-sample loss differences and leave-one-out curvature across training steps for 10 removed samples at ImageNet training sizes 128 and 1024.Both figures use DDPM noise prediction and leave-10-out training.Source: Figure 3 · p. 9, Figure 4 · p. 9
- The evidence covers only ImageNet and CelebA, and comparison among diffusion-model training losses remains for further investigation.The paper characterizes its evidence as narrow.Source: §5 · p. 9
- The proposed local maximum of relative train–validation curvature error occurs during already high memorization, so it does not locate memorization onset.The role of validation loss-gradient norms for larger datasets is not well understood.Source: §5 · p. 9
- The theoretical bound is a plausibility argument relying on assumptions about the loss, model, and optimizer, and curvature’s dominance over the ELBO gap remains unsettled.That dominance determines whether curvature is the leading contribution to the log-odds expression.Source: §5 · p. 10
A Proofs for Section 4
The appendix proof bounds counterfactual memorization by expanding loss changes around another training point and controlling the resulting terms under assumptions on the loss and training procedure.
A Proofs for Section 4
- A Lipschitz-Hessian condition yields a cubic Taylor remainder bound for the loss around an input point.The expansion includes the input gradient and Hessian quadratic term, with the remainder controlled by the Hessian Lipschitz constant.Source: A Proofs for Section 4 · p. 13
- A gradient lemma relates input gradients to parameter gradients for multiplicative neural networks, connecting input sensitivity to quantities controlled during optimization.The proof uses the form ℓ(ϕ,z)=g(ϕ1z,ϕ2,z) and a bound on ∥∇cg∥ relative to g.Source: A Proofs for Section 4 · p. 13
- The memorization bound combines the cubic expansion with loss-gap, gradient, curvature, and Hessian-Lipschitz terms around a second training point.Under the stated overfitting conditions, small loss and gradient terms can be compared with the leave-one-out curvature term times nearest-neighbor distance.Source: A Proofs for Section 4 · p. 13
- The proof applies the bound on a high-probability event, then controls the expansion terms using inequalities and assumptions that make training-sample losses and gradients small.The resulting estimates yield the stated two-sided bound and main result.Source: A Proofs for Section 4 · pp. 14–15
A.1 Proof of Proposition 4.5
The proof bounds the optimal training loss using the performance of a linear denoiser, then combines this bound with the PL inequality and nonnegativity of the leave-one-out optimum.
A.1 Proof of Proposition 4.5
- The function class contains linear denoisers, so its trained performance is no worse than the best linear denoiser, whose MSE is bounded by A(T).The paper states that A(T) depends on f, g, and T.Source: §A.1 · p. 15
- The proof decomposes the loss-gap expression by introducing the full dataset S and leave-one-out dataset S′.This decomposition sets up the subsequent bound.Source: §A.1 · p. 15
- The PL inequality, the A(T) bound on ℓ*(S), and nonnegativity of ℓ*(S′) yield the next inequality in the proposition proof.Source: §A.1 · p. 15
B Experimental Details
Experiments use ImageNet and CelebA diffusion models with DDPM noise- or sample-prediction objectives and flow matching. The appendix also outlines how flow matching constructs targets and generates samples.
B Experimental Details
- Flow matching regresses a time-dependent velocity field to the conditional velocity along a linear noise-to-data interpolant, then generates samples by integrating the learned ODE.The appendix describes 100 Euler steps from noise to data.Source: B Experimental Details · pp. 15–16
- ImageNet experiments use RGB 64 × 64 images with training sizes from 128 to 10,240, while CelebA uses grayscale 32 × 32 images at sizes 128 and 10,240.The selected CelebA sizes yield a memorizing versus generalizing model.Source: B Experimental Details · p. 16
- Score models use a four-level U-Net with base width 64, channel multipliers 1, 2, 4, and 4, and time conditioning in each residual block.Self-attention is used at the two lowest-resolution levels.Source: B Experimental Details · p. 16
- DDPM models are trained for noise prediction or clean-sample prediction, with DDIM sampling using 100 steps and η = 0.Noise-prediction outputs are converted to clean-image estimates before the next sampling step.Source: B Experimental Details · p. 16
- Flow-matching training minimizes MSE between predicted velocity and the target velocity along the interpolant.The target is the difference between the data sample and noise.Source: B Experimental Details · p. 16
C Generated Samples
The figures pair generated samples with their nearest training images across pre-memorization, onset, and later memorization stages. Examples span CelebA and ImageNet, DDPM and flow matching, and several training-set sizes.
C Generated Samples
- Figures 5–12 show generated-image/nearest-training-image pairs at pre-memorization, onset, and full-memorization stages for CelebA and ImageNet models.The examples cover DDPM and flow matching, with CelebA sizes 128 and 10240 and ImageNet leave-10-out sizes 128 and 1024.Source: C Generated Samples · p. 16, Figure 5 · p. 16, Figure 6 · p. 17
- Across ImageNet models trained on all n images, Figures 13–20 show five generated–training image pairs per row for DDPM sizes 128–4096 and flow-matching sizes 128 and 1024.Memorization fractions are measured over 10,000 generations; the stage labels distinguish onset and sustained memorization from isolated copy events.Source: C Generated Samples · p. 16, Figure 13 · p. 19, Figure 14 · p. 19
- For ImageNet DDPM, n=512 and n=1024 peak at 85.0% and 19.9% and never reach fmem ≥95%; n=2048 peaks at 0.57%, while n=4096 has only an isolated 0.03% copy event.Source: C Generated Samples · p. 16, Figure 15 · p. 20, Figure 16 · p. 20
D Additional Experiments
Additional experiments examine how curvature and memorization vary with dataset, training-set size, model, and memorization threshold. They report curvature separation near onset, cross-dataset differences, and lower memorization and curvature after removing highly copied samples.
D Additional Experiments
- CelebA with n = 128 approaches 100% memorization around τ = 10^3, whereas n = 10240 increases much later and remains below full memorization; curvature rises more sharply in the smaller-set models.For ImageNet, curvature increases less as training-set size grows.Source: D Additional Experiments · p. 22
- For CelebA flow matching with n = 128, training curvature separates from validation curvature just before τ = 10^3, near the reported onset of memorization.The plotted training/validation curvature gaps are shown alongside the curvature trajectories.Source: D Additional Experiments · p. 22
- On ImageNet with n = 10240, increasing the nearest-neighbor threshold k does not make DDPM’s gap ratios trend upward; flow matching’s ratios rise at larger k despite approximately 0% memorization at k = 1/3.The increasing gap ratios at larger thresholds therefore do not indicate memorization under the k = 1/3 criterion.Source: D Additional Experiments · p. 22
- For ImageNet DDPM sample prediction with n = 10240, average curvature barely changes and approximately 0% of generations are memorized at k = 1/3 throughout training.Nearest-neighbor gap ratios rise at larger k, but the cited passage still characterizes this model as generalizing.Source: D Additional Experiments · p. 22
- At n = 10240, CelebA ends near 23% memorization for DDPM and 40% for flow matching, versus about 0% and 0.1% on ImageNet, respectively.CelebA also has larger average training curvature than ImageNet for both models.Source: D Additional Experiments · p. 22
- Removing the top 10 memorized samples from ImageNet n = 1024 training lowers end-of-training memorization and input curvature at every training step for DDPM; flow matching shows similar results.The experiments are also repeated at n = 128, and per-sample loss differences and leave-10-out curvatures are plotted for flow matching.Source: D Additional Experiments · p. 22
E.1 Experimental Details
The experiments estimate input-Hessian curvature and compare it with generated-copy measurements across training checkpoints. Supporting figures extend the comparisons across datasets, model variants, training-set sizes, and copied versus uncopied images.
E.1 Experimental Details
- ImageNet DDPMs use nested training sets of n = 128–4096, a shared held-out set of 1,024 images, and training extended to 2 × 10^6 updates for n ≥ 512.Models use 64 × 64 RGB images, a width-64 U-Net, Adam at 10^-4, and a 1,000-step linear noise schedule.Source: §E.1 · p. 22
E.2 Supporting Figures
- Figures compare average input loss curvature over diffusion time and training set, and fraction memorized versus training step (k=1/3), for CelebA (n=128,10240) and ImageNet (n=128,1024,10240), using DDPM noise prediction or flow matching.Figure 27 additionally reports ImageNet (n=10240) memorized fraction at thresholds k=1/3, 0.8, and 0.9.Source: Figure 21 · p. 23, Figure 22 · p. 23, Figure 23 · p. 24
E.1 Experimental Details
- Curvature is estimated from 32 Monte Carlo draws per image using Hessian-vector products, and the measurements do not require generating model samples.Training means use all training images; validation means use the fixed held-out set.Source: §E.1 · p. 23
- At each evaluated checkpoint, 10,000 generated images are classified as copies when the nearest-to-second-nearest training-image distance ratio is below 1/3.The copy count assigns generated copies to training images, while the memorized fraction is the fraction of generations classified as copies.Source: §E.1 · p. 23
E.2 Supporting Figures
- Copied training images generally have higher mean curvature than remaining images, though the separation varies by training size and checkpoint.Smaller training sets tend to show both larger maximum copy counts and larger relative train–validation curvature gaps.Source: §E.2 · p. 24, Figure 37 · p. 31, Figure 38 · p. 32