Detecting, Explaining, and Mitigating Memorization in Diffusion Models

1 INTRODUCTION

• Near-exact training-image reproductions create concerns for model owners and users when training data includes sensitive or copyrighted material.The introduction notes that banning the substring “Afghan” did not prevent recreations of the copyrighted Afghan girl photograph.Source: §1 · p. 1
• Memorized prompts tend to produce larger text-conditional prediction magnitudes, which the method uses to detect them.The authors contrast this with approaches that query training datasets or assess density across multiple generations.Source: §1 · p. 1, Figure 1 · p. 2
• AUC 0.960 and TPR@1%FPR 0.760 in under 2 seconds, versus Carlini et al. (2023)’s AUC 0.934 and TPR@1%FPR 0.523 in over 39 seconds.The detector requires no extra work in the generation framework or multiple generations per prompt.Source: §1 · p. 2
• The token-level explanation estimates each token’s relative significance to memorization, helping model owners guide users to modify or omit trigger tokens.The approach evaluates gradient changes when minimizing text-conditional prediction magnitudes and is described as automated and computationally efficient.Source: §1 · p. 2
• Inference-time prompt-embedding minimization and training-time filtering of image-text pairs reduce memorization and improve prompt-generation alignment against baseline mitigation strategies.The strategies use the magnitude of text-conditional predictions.Source: §1 · pp. 2–3

2 RELATED WORK

• Traditional membership inference uses losses or confidence scores to distinguish training examples from unseen data, while newer diffusion-model methods compare predicted noise with induced noise.The passage names Shokri et al. (2017), Yeom et al. (2018), Carlini et al. (2022), Wen et al. (2022), Matsumoto et al. (2023), Duan et al. (2023), Wang et al. (2024; 2023).Source: §2 · p. 3
• Carlini et al. (2023) assess generation density by producing many images and applying membership inference; memorized prompts tend to yield nearly identical images across seeds.Somepalli et al. (2022) showed that diffusion models can reproduce training images verbatim.Source: §2 · p. 3
• Prior mitigation work includes training on corrupted data and altering text conditions with random tokens or prompt-embedding perturbations to alleviate memorization while preserving generation quality.The passage attributes these approaches to Daras et al. (2023) and Somepalli et al. (2023b), respectively.Source: §2 · p. 3

3.1 PRELIMINARY

• The preliminary setup describes forward diffusion as a fixed Markov chain that adds scheduled Gaussian noise, followed by reverse denoising with a trained noise predictor.Text conditioning changes the predicted noise through classifier-free guidance.Source: §3.1 · pp. 3–4

3.2 MOTIVATION

• For ordinary prompts, different initializations yield diverse images, while memorized prompts converge to a specific visualization regardless of initialization.The authors connect this behavior to stronger text guidance and propose prediction magnitude as a signal of potential memorization.Source: §3.2 · p. 4

3.3 AN EFFECTIVE DETECTION METHOD

• The detection metric is based on text-conditional noise-prediction magnitude, and memorization is identified when it falls beneath a tunable threshold.The metric can be collected across sampling steps, but reliable detection is also reported using only the first step.Source: §3.3 · pp. 4–5

3.4 EXPERIMENTS

• With one generation and the first step, the method reaches TPR@1%FPR of 0.760 in 0.2 seconds; with 32 generations and 10 steps, it reaches AUC of 0.999 and TPR@1%FPR of 0.988.Table 1 reports detection metrics and runtime by number of generations; baseline methods need more than 16 images and final-step outputs for relatively high accuracy, requiring at least 40 seconds.Source: Table 1 · p. 5, §3.4 · p. 6

3.3 AN EFFECTIVE DETECTION METHOD

• A single generation can provide a strong memorization signal, without requiring access to the training dataset.The paper contrasts this with methods requiring many generations or direct comparison with original training images.Source: §3.3 · p. 5

3.4 EXPERIMENTS

• The memorized-prompt detection-metric distribution is bimodal, with template-verbatim cases often lower than matching-verbatim cases because memorization is partial.Figure 2(b) compares metrics for memorized and non-memorized prompts over 50 steps and four generations per prompt.Source: §3.4 · p. 6
• The proposed method is faster than baselines even when using the same number of generations and sampling steps because it avoids decoding latent noise into image space.Image-space decoding and subsequent calculations are not needed for its metric.Source: §3.4 · p. 6

4.1 A STRAIGHTFORWARD METHOD TO DETECT TRIGGER TOKENS

• Token significance is estimated from embedding changes during noise-magnitude minimization; modifying top-ranked triggers can reduce memorization, unlike extensive edits to less significant tokens.The figure marks the two most significant tokens and distinguishes their edits from changes to less significant tokens.Source: §4.1 · pp. 6–7, Figure 3 · p. 7

4.2 AN EFFECTIVE INFERENCE-TIME MITIGATION METHOD

• At inference, minimizing the text-conditional prediction magnitude at the initial time step indirectly reduces magnitudes at later steps, with early stopping keeping the perturbed embedding near the original meaning.Optimizing across all time steps is computationally intensive.Source: §4.2 · p. 7

4.4 EXPERIMENTS

• The proposed mitigation techniques reduce memorization with a more favorable CLIP-score trade-off than random token addition; stronger settings tend to improve alignment but weaken mitigation.Qualitative examples also show closer prompt alignment than the baseline, which sometimes fails to counter memorization and adds undesirable image content.Source: Figure 4 · p. 8, §4.4 · pp. 8–9, Figure 5 · p. 9

4.3 AN EFFECTIVE TRAINING-TIME MITIGATION METHOD

• During training, samples whose text-conditional noise-prediction magnitude exceeds threshold τ are excluded from the mini-batch; this adds an extra forward pass and about 10% training time.The method skips computing the loss for the excluded sample.Source: §4.3 · pp. 7–8

4.4 EXPERIMENTS

• The experiments fine-tune Stable Diffusion on 200 LAION images duplicated 200 times, alongside 120,000 distinct images, and average results across five models and five random-seed runs.They compare five inference target losses and five training thresholds against random token addition.Source: §4.4 · p. 8

• Detection requires an empirical threshold calibrated on non-memorized prompts for a predetermined false-positive rate.The authors note that an interpretable p-value could provide a confidence score, but leave this for future work.Source: §5 · p. 9
• Detection outcomes lack interpretability, so they do not provide a confidence score quantifying the likelihood of memorization.Interpretable p-values are proposed as a possible future direction.Source: §5 · p. 9
• Inference-time mitigation requires additional GPU memory and takes at most six seconds in the reported setup; training-time mitigation extends training by roughly 10%.In the authors' tests, training increased from approximately nine hours to about ten hours.Source: §5 · p. 9

• The proposed detection approach uses the magnitude of text-conditional noise predictions and is reported to work with limited generations and sampling steps.Source: §6 · p. 9
• The paper provides a tool that indicates the significance of individual tokens in relation to memorization.Source: §6 · p. 9
• The authors conclude that inference- and training-time mitigation address memorization while maintaining the model's generative performance.Source: §6 · p. 9

• The main body details the essential hyperparameters used in the experiments.Source: §7 · p. 10
• The experiments used widely available computing resources and open-source software, and referenced models and datasets are publicly accessible.Source: §7 · p. 10
• The supplementary material includes code intended to reproduce the results.Source: §7 · p. 10

8 DISCLAIMER

• The research used some images from the LAION 5B dataset for analysis.Source: §8 · p. 10
• After the images were used and deleted, the authors discovered that LAION 5B contained exploitative images of children.Source: §8 · p. 10
• Although they had no reason to believe their subset contained such images, the authors disclose the dataset’s use and associated ethical considerations.Source: §8 · p. 10

A APPENDIX

• Figure 6 compares different seeds with different prompts.Source: Figure 6 · p. 13
• Figure 7 illustrates mitigation by modifying trigger tokens, with red and blue marking the two most significant tokens and arrows distinguishing top-two from lesser-token changes.The significance score for each token is shown in a histogram in Appendix Fig. 7.Source: Figure 7 · p. 14
• Figure 8 plots text-conditional noise-prediction magnitude at each time step across four generations with different seeds per prompt; the metric is typically higher for memorized generations.Source: Figure 8 · p. 15
• Figure 9 shows that non-memorized generations consistently have significantly lower text-conditional noise-prediction metric values across four seeds per prompt.Source: Figure 9 · p. 16