Source-linked AI summary
A Reproducible Extraction of Training Images from Diffusion Models
Ryan Webster
TL;DR
Diffusion models can reproduce training images, creating copyright and privacy concerns. The paper develops a one-step extraction attack, introduces template verbatims, and evaluates the approach across several diffusion systems. Its attack is reported as comparable to prior work while using substantially fewer network evaluations, including on systems that deduplicate training data.
Problem
Method
The paper uses one-step synthesis and edge-consistency analysis to construct whitebox and blackbox extraction attacks.
Source: §1 · p. 2, §3.2 · p. 4Results
Takeaways & Limitations
Template verbatims remain extractable from newer systems that deduplicate training data, including Stable Diffusion 2.0 and Deep Image Floyd.
Source: §1 · p. 2, §5.2 · p. 7, §6 · p. 9The ground-truth construction can miss copied patches when they are cropped, scaled, or rearranged.
Source: §5.4 · p. 8
Abstract
from arXiv · showhide
Recently, Carlini et al. demonstrated the widely used model Stable Diffusion can regurgitate real training samples, which is troublesome from a copyright perspective. In this work, we provide an efficient extraction attack on par with the recent attack, with several order of magnitudes less network evaluations. In the process, we expose a new phenomena, which we dub template verbatims, wherein a diffusion model will regurgitate a training sample largely in tact. Template verbatims are harder to detect as they require retrieval and masking to correctly label. Furthermore, they are still generated by newer systems, even those which de-duplicate their training set, and we give insight into why they still appear during generation. We extract training images from several state of the art systems, including Stable Diffusion 2.0, Deep Image Floyd, and finally Midjourney v4. We release code to verify our extraction attack, perform the attack, as well as all extracted prompts at \url{https://github.com/ryanwebster90/onestep-extraction}.
1 Introduction
The paper studies training-image regurgitation in widely used diffusion systems and introduces an efficient extraction attack based on one-step synthesis. It also identifies template verbatims and evaluates the attack across multiple models.
1 Introduction
- Diffusion models can regurgitate training samples, raising copyright concerns for widely used systems trained on large web-scale datasets.Stable Diffusion and related systems have been shown to generate exact copies of training images.Source: §1 · p. 1
- Popular diffusion models can reproduce training samples in a single sampling step, enabling whitebox and blackbox extraction attacks.This one-step behavior is the basis for the paper’s attack construction.Source: §1 · p. 2
- Template verbatims copy training samples largely intact while allowing non-semantic variation in fixed image locations, complicating detection.These cases require retrieval and masking to construct appropriate ground truth.Source: §1 · p. 2
- The proposed attack is largely on par with the attack in [5] while requiring considerably fewer network evaluations.It can also be combined with [5] as a post-filtering stage.Source: §1 · p. 2
- The attack extracts samples from several other systems, including the closed-source model Midjourney, and studies why template verbatims persist.The analysis covers systems beyond the initial Stable Diffusion setting.Source: §1 · p. 2
2 Related work
Related work situates extraction attacks within web-scale dataset construction, dataset deduplication, membership inference, and prior evidence that diffusion models can reproduce training images. The figure further distinguishes direct verbatim copies from harder-to-detect template verbatims.
Billions Scale Datasets
- Web-scale image datasets contain billions of text-image pairs collected automatically and filtered using CLIP-based relevance scores.LAION-5B contains roughly 5 billion text-image pairs, with LAION-2B-en as its English-language subset.Source: §2 · p. 2
Deduplication
- Deduplication studies found extensive near-duplicate content, while prior work linked highly duplicated samples to diffusion-model memorization.Stable Diffusion 2.0 used perceptual-hash deduplication, which reduced some but not all verbatims.Source: §2 · p. 2
Membership Inference
- Membership inference seeks to determine whether training samples were used from model outputs, including in the difficult blackbox setting for proprietary models.Recent diffusion-model attacks studied relatively small datasets, generally below 1M samples.Source: §2 · p. 2
Extraction Attacks
- Figure 1 contrasts one-step verbatim copies with template verbatims that require retrieval and masking because fixed image regions can vary.Non-verbatims have no match even when retrieval is performed over the entire dataset.Source: Figure 1 · p. 3
- Extraction attacks go beyond membership inference by reconstructing training samples, but prior work recovered only roughly 100 images from 350K attempted Stable Diffusion prompts.Because extracted samples have privacy and copyright implications, extraction studies emphasize precision and the number of recovered samples.Source: §2 · p. 3
3 Attack Model
The attack model exploits one-step synthesis behavior to identify likely memorized images in both whitebox and black-box settings. The black-box attack uses edge consistency across random seeds, while template verbatims motivate retrieval- and mask-aware evaluation.
3 Attack Model
- One-step synthesis of copied images motivates attacks that rank likely memorized samples efficiently.The authors reproduce this behavior and use it as the basis for whitebox and black-box attacks.Source: §3 · p. 3
3.1 Whitebox Attack
- The whitebox attack measures denoising error after one timestep and thresholds the resulting Denoising Confidence Score (DCS).The score uses a one-step Heun-sampler noise scale; the setting requires captions and model access, although only the noise process and latent encoder are needed in practice.Source: §3.1 · pp. 3–4
3.2 Black Box Setting
- The black-box setting assumes caption access, model querying, and control over sampling timesteps, as available for systems such as Midjourney.The method compares edge patterns across seeds because non-verbatims typically begin as blurry images with inconsistent or absent edges.Source: §3.2 · p. 4
- The black-box attack generates one-step samples across random seeds and classifies captions by the consistency of their spatial edges.Edge() thresholds spatial-gradient magnitudes, and the threshold γ should scale with the number of seeds J.Source: §3.2 · p. 4
4 Constructing a Ground Truth
The ground-truth procedure labels copied images using matching or retrieval-based comparisons. Retrieval verbatims extend matching by allowing near-duplicate training images with localized variation to be aligned through spatial masks.
4 Constructing a Ground Truth
- Matching verbatims use the minimum MSE between a generated image and its corresponding dataset image across J random seeds, followed by thresholding.The procedure uses δV = .12 and manually prunes obvious false positives such as textures or images without objects.Source: §4 · p. 4
- Template verbatims arise when images vary at fixed spatial locations, such as e-commerce images whose carpet colors change while the broader template remains stable.The generated, retrieved, and masked views expose this localized-variation structure.Source: Figure 2 · p. 5
- Retrieval verbatims accommodate near-duplicate training images by retrieving neighbors and masking spatial regions that vary across the image series.The retrieval distance uses CLIP-image-embedding nearest neighbors from LAION-2B and a spatial mask; exact retrieved matches are called retrieval verbatims (RV).Source: §4 · p. 5
- Ground truth is computed only for top-ranked attack candidates, measuring precision against the number of discovered verbatims rather than recall over the full dataset.This reduces the cost of generating images, retrieving references, and constructing masks.Source: §4 · pp. 4–5
5 Results
The attacks identify verbatim and template-verbatim training images across diffusion models, with substantially lower evaluation cost than prior extraction. Results also show that deduplication reduces exact copies but does not prevent masked template verbatims.
5 Results
5.1 Analysis
- Combining the black-box attack with [5] substantially improves precision, while the white-box attack is more precise than the standalone black-box variants.The standalone black-box attacks suffer from adversarial texture and background samples.Source: §5.1 · p. 6, Figure 3 · p. 6
5.2 What Makes Prompts Prone to Regurgitation?
- Exact verbatims are more often multimodal duplicates than non-verbatims, indicating that shared prompts and image features characterize these samples.The comparison uses verbatims against randomly selected highly image-duplicated non-verbatims.Source: §5.2 · p. 7
- Template verbatims can be highly duplicated under a mask while appearing weakly duplicated without masks, making retrieval and labeling difficult.Stable Diffusion v2 yielded no exact verbatims in this evaluation but still produced many template verbatims.Source: §5.2 · p. 7
5.3 Extracting Verbatims in Other Models
- Deep Floyd, Midjourney, and related checkpoints still produce template verbatims, although deduplicated models appear less susceptible to exact verbatims.Midjourney was evaluated manually on around 100 prompts, while Deep Floyd and Midjourney used the top 500 SDV1-ranked prompts.Source: Table 1 · p. 7, §5.3 · p. 8
5.4 Limitations
- Ground-truth construction misses copied images when matching error thresholds, crops, scaling, or small distortions prevent retrieval or labeling.The authors suggest more flexible copy detection, including methods invariant to some patch permutations.Source: §5.4 · p. 8, Figure 5 · p. 8
6 Discussion and Conclusion
The paper presents an extraction attack that works across widely used diffusion models with substantially fewer network evaluations. It also explains why masked template verbatims can persist after deduplication, while emphasizing that exact copying appears rare and broader attribution remains unresolved.
6 Discussion and Conclusion
- The attack is successful across several widely used diffusion models while requiring significantly fewer network evaluations than previous methods.The paper also automates labeling of template verbatims.Source: §6 · p. 9
- Template verbatims may persist after deduplication because samples are likely duplicated with respect to a mask rather than in the standard whole-image sense.This explanation is applied to Stable Diffusion 2.0 and Deep Image Floyd.Source: §6 · p. 9
- Verbatim copying appears to occur very rarely, while dataset non-transparency and absent attribution remain broader concerns for generative systems.The authors discuss attribution as a possible direction for future systems.Source: §6 · p. 9