PaperlayerView exact revision on arXiv

Source-linked AI summary

The Null Is the Hard Part: Exact Tests for Memorization in Generative Models

Sushovan Majhi, Pramita Bagchi

arXiv:2610.00251v1cs.LG

TL;DR

Similarity-threshold audits lack null distributions for deciding which training images a model reproduces. The paper develops model- and image-level nulls and calibrated statistics; its results show that both null choice and statistic choice affect audit conclusions.

  • Problem

    Generative-model memorization detectors mostly read similarity scores against thresholds, without error bars or false-discovery control for large audits.

    Source: §1 · p. 1

  • Method

    The paper uses exchangeability-based permutation tests for whole models and conformal tests against matched controls for individual images, recommending a near-duplicate-scale Intersection Euler Characteristic Profile statistic.

    Source: §1 · pp. 1–2, §2 · p. 3

  • Results

    36 of 61 MemBench images are certified by Benjamini–Hochberg after calibration, while the data-copying statistic flags 10 of 24 generators that reproduce nothing.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A null makes a statistic a test, while restricting it to the near-duplicate scale makes it a test of reproduction rather than overfitting.

    Source: §6 · p. 9

    The image-level evidence rests on one benchmark of 61 images and three related checkpoints, and matched-control exchangeability is an assumption rather than a guarantee.

    Source: §6 · p. 9

Abstract

from arXiv · show

Memorization audits of generative models read similarity scores against thresholds, with no null distribution, and the conclusions they support can be wrong. By MemBench's rule, the benchmark's mitigations roughly halve Stable Diffusion's memorization; audited with false-discovery control, two thirds of the certified images are no longer detected under random prompt perturbations, five sixths under attention rescaling, and all of them under embedding optimization. The field's data-copying test, read against its own null, flags ten of twenty-four generators that reproduce nothing. We argue that for memorization the null is the hard part, and supply two. For a whole model, training and held-out images are exchangeable given its samples, and relabelling them is a permutation test, exact for any statistic when the held-out images are a random split; under it, a nearest-neighbour preference still fires on seven of those twenty-four, and a count restricted to the near-duplicate scale on none (McNemar p=0.016). For single images, the natural nulls fail twice, measurably: ranking an image among random images yields 596 false discoveries among 2,365 controls, and resampling independent generations makes the null three times too narrow. Calibrated against matched controls, the audit certifies 36 of 61 MemBench images at 5% false-discovery rate, held-out controls are certified in 0.01% of calibration splits, and on this benchmark two generations per image recover that count. A calibrated maximum, which reads occasional rather than typical copying, certifies 46. As the scale-restricted statistic we recommend the small-scale mass of the Intersection Euler Characteristic Profile, which also counts distinct images copied and tests whether two models copy the same ones.

1 Introduction

Similarity thresholds alone cannot establish which training images a model reproduces: audits need null distributions that support error-controlled conclusions. The paper develops model- and image-level nulls and shows that calibration and statistic choice change what audits detect.

1 Introduction

  • Memorization detectors generally read similarity scores against thresholds, without error bars or false-discovery control for audits screening thousands of images.Mitigations are scored by how far a thresholded count falls, though the practitioner’s claim that a model reproduces k training images is a judgment about a histogram.Source: §1 · p. 1

1.1 Related work

  • The paper applies relabelling-based exact tests to reproduction without marking data or retraining, and uses conformal outlier tests for single images.Earlier training-use tests use random publishing or canary insertion; this paper instead measures calibration failures specific to generative audits.Source: §1 · p. 1, §1.1 · p. 2

1 Introduction

  • The recommended Intersection Euler Characteristic Profile uses small-scale mass to count distinct copied images and test whether two models copy the same ones.Its k-fold form tests shared copies; the paper distinguishes this reproduction-focused statistic from related shape comparisons that attach no null or separate reproduction from distributional fit.Source: §1 · p. 2, §1.1 · pp. 2–3
  • 10 of 24 generators that reproduce nothing are flagged by the data-copying statistic against its normal reference.For a model that never sees its training set, it reports copying at Z_U = −9.8 while the swap test gives p = 0.91.Source: §1 · p. 2
  • Of 24 generators that reproduce nothing, the nearest-neighbour fraction rejects on 7, the data-copying statistic on 5, and a near-duplicate-scale count on none.All three statistics detect the 4 generators that reproduce; ten injected copies among 500 samples are caught by scale-restricted statistics in every replicate, while the nearest-neighbour fraction needs fifty.Source: §1 · p. 2
  • The per-image audit measures two calibration failures: semantic leakage yields 596 false discoveries among 2,365 controls, and false concentration makes the null three times too narrow.With correct calibration, it certifies 36 of 61 images by Benjamini–Hochberg and 44 by Storey’s adaptive procedure; held-out controls are certified in 0.01% of calibration splits.Source: §1 · p. 2

2 Two nulls

The paper defines two nulls: exchangeable training and held-out labels for model-level tests, and exchangeable candidate and matched-control scores for image-level audits. It recommends near-duplicate-scale statistics to distinguish reproduction from mere proximity or overfitting.

2 Two nulls

  • The swap test gives an exact permutation test for any statistic when training and held-out images are exchangeable conditional on model samples.When held-out images are a random split, the null holds by design; on Stable Diffusion, matched-control exchangeability remains an assumption, and the control arm is diagnostic rather than proof.Source: §2 · p. 3
  • A data-copying test can reject without training-set access: a model that never sees its training set can trigger it when generated samples lie nearer training points than held-out points.In the stated example, the data-copying statistic reports copying at −9.8 while the swap test gives p = 0.91.Source: §2 · p. 3
  • The image-level null treats a candidate’s score as exchangeable with scores from controls generated the same way, yielding a conformal p-value.Benjamini–Hochberg and Storey’s adaptive procedure provide marginal false-discovery control over the calibration set, not a guarantee for any single image.Source: §2 · pp. 3–4
  • The leak count equals the number of distinct training images reproduced within r only under separation assumptions: training images are more than 2r apart and generations near them are within r.The spacing condition can fail for near-duplicate training images, and the substantive condition excludes generations that fall between copies and non-copies.Source: §2 · p. 4
  • At near-duplicate scales, the Intersection Euler Characteristic Profile’s small-scale mass counts distinct copied images and can test which images two models both reproduce.The profile integrates the distinct-image count over scales up to r0 when Proposition 3’s hypotheses hold; the Alpha complex is computed after a pooled-cloud PCA projection to five dimensions.Source: §2 · p. 4

3 What the statistic decides

Under the swap null, all four generators that reproduce are detected, but statistics differ sharply on generators that reproduce essentially nothing; near-duplicate-scale counts reject none.

3 What the statistic decides

  • The experiments use small diffusion models trained on CIFAR-10 subsets and test 500 generated samples against training and held-out images; copies are defined within r0 in pixel space.The study also tests synthetic memorizers before evaluating the trained models.Source: §3 · p. 5
  • Models trained on 100 and 200 images store their training sets, the 500-image model copies a fifth of samples, and memorization stops at 1,000 images in both seeds.The full-data model is null on every statistic; training the 1,000-image model three times longer yields 24 copies of 21 images but does not reject (p=0.055).Source: §3 · p. 5
  • The thresholded count rejects none of 24 generators that reproduce essentially nothing, versus seven nearest-neighbour rejections; the paired difference is significant (McNemar p=0.016).All four reproducing generators are detected under both held-out designs.Source: §3 · p. 6, Table 2 · p. 6
  • The count and T_small catch ten injected copies among 500 samples in every replicate, while the nearest-neighbour fraction needs fifty.At five injected copies, T_small rejects in three to five of five replicates, depending on noise level.Source: §3 · p. 6

4 Auditing single images

Matched-control calibration identifies memorized images while exposing failures in natural image-level nulls. The audit distinguishes typical similarity from occasional strong reproduction.

4 Auditing single images

  • The model-level test assumes exchangeability between training and held-out images; the control arm diagnoses possible mismatch but does not validate that assumption.The image-level audit does not rest on this model-level assumption.Source: §4 · p. 7
  • The image-level statistic is median SSCD similarity between each memorized image and its trigger-prompt generations, capturing layout-level copies.The passage says embedding space suits this statistic, while SSCD distances concentrate too tightly for the topological statistic.Source: §4 · pp. 6–7
  • Ranking each image against random images yields 596 false discoveries among 2,365 controls because captions describe their source images.38% of rank p-values fall below 0.05 for generations from non-memorized captions.Source: §4 · p. 7
  • 36 of 61 images are certified by Benjamini–Hochberg and 44 by Storey’s procedure; held-out controls are certified in 0.01% of calibration splits.Figure 3 shows per-image median SSCD similarities against matched controls and calibrated p-values against the Benjamini–Hochberg line.Source: §4 · p. 7, Figure 3 · p. 8
  • Independent resampling makes the null three times too narrow: median spread is 0.016 versus 0.050 when generations are drawn from one control caption.Mixing captions loses between-image variation; the passage identifies more captions, rather than more generations per caption, as the remedy.Source: §4 · p. 7
  • Two generations per image recover the benchmark’s certification count: 37.0 ± 2.6 images, compared with 30.7 ± 2.9 for one generation.More generations add nothing, and the passage cautions that these budgets describe this benchmark and model, not a general prescription.Source: §4 · p. 7
  • At a matched budget of 12 generations, a calibrated maximum certifies 46.3 ± 1.3 images versus 35.4 ± 1.3 for the median, capturing occasional rather than typical copying.The maximum is calibrated against control maxima; the median measures typical similarity.Source: §4 · p. 7
  • Under mitigation, the audit no longer detects two thirds to all certified images, while the benchmark’s rule reports roughly half as many flags.Figure 4 contrasts audit certifications at 5% false-discovery rate with fixed-rule flags above similarity 0.5.Source: Figure 4 · p. 8

5 What a correct null changes

Mitigation outcomes depend on what the audit measures: calibrated medians track typical reproduction, while calibrated maxima capture occasional copying. The audit’s unit must also match the copied content.

5 What a correct null changes

  • After mitigation, the audit certifies 11 images under random tokens, 10 under random numbers, 6 under attention rescaling, and none under either embedding-optimization target.All 36 images’ median own-similarity falls in every mitigation arm, while control-caption medians move by less than 0.01.Source: §5 · p. 8
  • A template-keyed audit certifies 189 of 197 Stable Diffusion 2 prompts and 27 of 56 template families, whereas the own-source audit certifies none.The template is a layout shared by many training images; 1,184 of 1,576 generations copy some candidate.Source: §5 · p. 9
  • Two diffusion seeds trained on the same 200 images share 109 copied images against 83 expected (p=2 × 10^-15), and v1.4 and v1.5 share 32 images certified by both audits.The v1.5 and RealisticVision overlaps may be inherited through continued training or fine-tuning.Source: §5 · p. 9

6 Limitations and conclusion

The paper’s conclusion is that valid nulls turn similarity statistics into tests, while the near-duplicate scale distinguishes reproduction from overfitting. The audit’s guarantees and scope depend on specific data and calibration conditions.

6 Limitations and conclusion

  • The model-level test is exact only when held-out images are a random split; matched-control exchangeability for Stable Diffusion remains an assumption, not a guarantee.The control arm is diagnostic, and the image-level evidence covers one benchmark of 61 images and three related checkpoints.Source: §6 · p. 9
  • Image-level certification requires training images and a candidate list, so it measures discoverable memorization rather than all memorization.The image-level guarantee is marginal over the calibration set, with a 1/(N + 1) p-value floor.Source: §6 · p. 9
  • The null makes a statistic a test, and near-duplicate-scale restriction makes it a test of reproduction rather than overfitting.Correct image-level calibration also requires measuring semantic leakage and false concentration.Source: §6 · p. 9

Reproducibility statement

The experiments use a shared code base, fixed inputs or seeds, and reproducible p-values. The authors release scripts and measurement outputs, while generated arrays that can be regenerated deterministically are not shipped.

Reproducibility statement

  • The invariant uses the GUDHI Alpha complex, and DDPMs use a 3.9M-parameter U-Net trained on fixed CIFAR-10 subsets with recorded indices.Stable Diffusion generations use fixed per-prompt seeds.Source: Reproducibility statement · p. 9
  • Every paper p-value is reproducible by a single command per table.Source: Reproducibility statement · p. 9
  • The authors release fetch scripts, image identifiers, Stable Diffusion generations at 32 pixels, and all measurement outputs, but not deterministically regenerable arrays.The code and measurement outputs are listed as 166 measurement records and 247 files.Source: Reproducibility statement · p. 9

Ethics statement

The audited images are training images reproduced by a public model and identified by the benchmark authors through reverse image search. The authors acknowledge that confirming reproduction is dual-use, but judge the balance favorable because the method requires the training set.

Ethics statement

  • The audited training images include product photographs and a small number of portraits, located through reverse image search by the benchmark authors.Source: Ethics statement · p. 9
  • The audit is dual-use because it confirms which training images a model reproduces and names them, but the authors judge the balance favorable because it requires the training set.They contrast this with releasing models whose reproduction has not been measured.Source: Ethics statement · p. 10

A Proof of Proposition 1

Conditioned on the model samples and the unordered pooled real images, the training and held-out labels are exchangeable, making relabelling an exact permutation test. The proofs also establish the Euler-characteristic count under separation assumptions.

A Proof of Proposition 1

  • The relabelled statistic values are exchangeable, so the rank p-value is valid at level α, including when ties occur.A statistic constant across relabellings returns p=1; the pooled-cloud projection and scale remain fixed across relabellings.Source: A Proof of Proposition 1 · p. 13

B Proofs of Propositions 2 and 3

  • For i.i.d. training and held-out points independent of the model, relabelling training labels preserves the joint law and Proposition 1 applies.Source: B Proofs of Propositions 2 and 3 · p. 13
  • For finite unions of closed balls, Euler characteristic obeys inclusion–exclusion, and for a finite point set its union-of-balls Euler characteristic is that of its Alpha complex.Source: B Proofs of Propositions 2 and 3 · p. 13
  • Under separation conditions, the excess Euler-characteristic count equals the number of distinct training images with a generated point within radius r.The conditions require training images more than 2r apart and any generated point within 2r of one to lie within r.Source: B Proofs of Propositions 2 and 3 · p. 14

C The null’s size and the resolution limit

The section tests calibrated image audits and model-level copying across real and synthetic settings. Results depend on matching the null and statistic to the audit target, with explicit caveats for candidate matching and exchangeability.

C The null’s size and the resolution limit

  • At 300 controls, the Benjamini–Hochberg audit certifies 36 of 61 images, while Storey’s adaptive procedure certifies 44.With memorized images limited to 12 generations, matched-count reruns certify 35.4 ± 1.3 images over 50 subsamples.Source: C The null’s size and the resolution… · p. 14
  • The median audit declines 25 images whose median own-similarity is 0.076, but their group is elevated over controls and Storey’s procedure selects eight.The strongest declined image has similarity 0.152, below the realized rejection threshold of 0.179.Source: C The null’s size and the resolution… · pp. 14–15
  • Held-out controls are certified in 0.01% of 20,000 calibration splits, while no power-table cell produces a discovery among its held-out negatives.The negative-control procedure holds out 61 controls and calibrates on 239.Source: C The null’s size and the resolution… · pp. 14–15

D Stable Diffusion v1.5 and RealisticVision 5.1

  • For Stable Diffusion v1.5, the matched-null audit certifies 35 of 61 images, with 32 overlapping v1.4’s 36; inherited weights prevent attributing overlap to shared data alone.At pixel level, the models copy 5 and 10 images and share 5.Source: D Stable Diffusion v1.5 and… · p. 15
  • RealisticVision 5.1 certifies 30 of 61 images, 29 also certified for v1.4, and 26 for all three models; the authors infer the fine-tune inherits base-model memorization.Its pixel-space generations include 5 copies at 64 pixels and 9 at 32 pixels.Source: D Stable Diffusion v1.5 and… · p. 15

E Stable Diffusion 2: memorization by template

  • For Stable Diffusion 2, the own-source audit certifies none of 164 candidate images, while a prompt-level maximum-over-candidates statistic selects 189 of 197 prompts.The authors report this descriptively because generic-caption controls and mostly flooring candidates do not enforce exchangeability.Source: E Stable Diffusion 2: memorization by… · pp. 15–16
  • At the family level, Benjamini–Hochberg selects 27 of 56 families, but differing score functions on shared controls make this count heuristic.Six of the nine families with three or more images are selected.Source: E Stable Diffusion 2: memorization by… · p. 16

F Synthetic protocols

  • In synthetic clouds, small-scale statistics detect the partial memorizer at p=0.005 while all-scale mass gives p=0.08; nearest-neighbour fraction 0.592 also hides 100 versus 5 distinct copies.The excess component count reads 92 and 10 for those respective copy sets.Source: F Synthetic protocols · p. 16

G The baseline, in one table

The swap-null comparison separates distribution-level preference for training images from near-duplicate reproduction, while the small-scale overlap statistic is more specific to copies. In SSCD space, concentration obstructs scale-resolved topological readouts even when nearest-neighbour detection remains active.

G The baseline, in one table

  • The small-scale overlap statistic fires only on reproduction, whereas the nearest-neighbour fraction also flags models whose samples are merely closer to training images on average.Across the cited comparisons, the small-scale statistic is more specific to copies; the nearest-neighbour preference can indicate overfitting without reproduction.Source: G The baseline, in one table · p. 16
  • Changing the held-out design leaves detection unchanged but changes some non-reproducer decisions; the thresholded count and swap-null Z_U make identical decisions under both designs.For the nearest-neighbour fraction, the 5,000-image model changes from p=0.016 to 0.77, while a 1,000-image seed changes from 0.08 to 0.02.Source: G The baseline, in one table · p. 17
  • Under the swap null, Z_U detects all four reproducers but rejects on five of twenty-four negatives; the cell-weighted C_T rejects on six.Read against its standard-normal reference, Z_U flags ten of twenty-four negatives with a random split and twelve with the test-split held-out set.Source: G The baseline, in one table · p. 17
  • The small-scale mass integrates distinct-image counts over thresholds below the near-duplicate scale, and varying the cutoff changes its decisions less often than the thresholded count.The mass detects the 500-image reproducer at every tested cutoff, but it is not immune to cutoff choice, which remains fixed in advance.Source: G The baseline, in one table · p. 18

H Which space

  • In SSCD space, topological statistics are null while the nearest-neighbour fraction still fires; pixel space at either resolution retains scale-resolved structure.SSCD’s concentration loses the scale-resolved structure, not the signal; the audit therefore uses the practitioner’s detector space for detection and pixel space for readouts.Source: H Which space · p. 18

I Auditing the benchmark’s mitigations

The mitigation audit regenerates memorized and control images under four interventions and applies calibrated image-level and set-level tests. The paper also tests shared memorization and the swap-null approach beyond diffusion models, while specifying the experimental implementation.

I Auditing the benchmark’s mitigations

  • Every one of the 36 certified images has lower median own-similarity under every mitigation, while control captions move by 0.008 or less.The calibrated median audit certifies 36 images at 5% false-discovery rate; under Wen’s method at the benchmark’s target, 21 images are flagged but none is certified.Source: I Auditing the benchmark’s mitigations · pp. 19–20, Table 5 · p. 20
  • The two scores answer different questions: the rule can flag occasional exceedances, while the calibrated median audit measures typical reproduction and a calibrated maximum counts occasional copying.The number of generations above the threshold falls from 447 of 1,811 to 171, 181, 73, 21, and 114 across the five arms.Source: I Auditing the benchmark’s mitigations · p. 20

J Shared memorization as a certified test

  • For two generators, the three-fold small-scale mass tests shared copies under an exchangeability null, while pairwise masses test each model’s overlap separately.With the two 200-image seeds, the three-fold mass is 53.3 against a null of 0.3 ± 4.6 (p=0.005); the authors caution that the null is more restrictive than no shared memorization.Source: J Shared memorization as a certified… · pp. 20–21

K The test on two other model families

  • The swap null is not specific to diffusion: the same measurement code was applied unmodified to a variational autoencoder and an adversarial network.Neither model family put a sample within a quarter of the training-image spacing in the reported tests.Source: K The test on two other model families · p. 21, Table 6 · p. 21

L Constants

  • The experiments use a five-dimensional PCA shadow, a 200-point scale grid, and a near-duplicate scale set to one quarter of the real images’ median nearest-neighbour spacing.The implementation also fixes model and image-generation settings across the reported experiments.Source: L Constants · p. 22

M Why the leak count is not reported for Stable Diffusion

The leak count separates reproducing from non-reproducing generations in Stable Diffusion but does not count distinct copied images there. Its projected-space contacts are largely layout-level proximity, so the authors report the nearest-neighbour tally instead.

M Why the leak count is not reported for Stable Diffusion

  • The leak count’s excess is real for memorized generations, but its value is not a count of images on Stable Diffusion.The excess is 29.7 ± 5.5 for memorized generations and 5.3 ± 12.8 for control generations, whose interval covers zero.Source: M Why the leak count is not reported… · p. 22
  • In the five-dimensional shadow, 31 memorized images have near-scale proximity to a generation, but only five are within a quarter of image spacing in ambient pixel space.The projected-space count reads 43 for those 31, over-reporting by a further 40% as components fragment.Source: M Why the leak count is not reported… · p. 22
  • As generation count rises, the leak count keeps increasing while the ambient tally of distinct copied images remains low, showing that projected contacts are not necessarily copies.The leak count rises from 4 to 43 across |G|=61 to 1,826, while the ambient tally rises from 0 to 5.Source: M Why the leak count is not reported… · p. 22
  • The count is valid in near-exact-reproduction regimes such as DDPMs, not for Stable Diffusion; the set-level certificate and per-image audit do not depend on it.The paper attributes the limitation to the readout, not the test.Source: M Why the leak count is not reported… · p. 22

N Diagnostics behind the counts

  • Ranking control generations against random images yields 596 false discoveries among 2,365 controls, despite low own-source similarities.The median own-source similarity is 0.052, and only one of 2,365 controls exceeds 0.5; the calibrated null is why the rank design fails.Source: N Diagnostics behind the counts · pp. 22–23
Loading 2610.00251v1…