Source-linked AI summary
A Non-Parametric Test to Detect Data-Copying in Generative Models
Casey Meehan, Kamalika Chaudhuri, Sanjoy Dasgupta
TL;DR
Generative-model overfitting is difficult to assess when models lack exact tractable likelihoods, while common evaluation tests can miss data-copying. The paper introduces a three-sample, non-parametric test and reports that it detects copying in forced and complex overfit models. Its ability to handle local variation is constrained by the number of available test samples.
Problem
Generative models may lack exact, tractable likelihoods, and common sample-based tests can miss models that reproduce training examples or small variations.
Source: §1 · p. 1, §4.3 · p. 10Method
The test uses the training set, an independent sample from the target distribution, and generated samples to assess whether outputs are too close to training data.
Source: §1 · p. 1, §2 · p. 3, §2.1 · p. 3Results
As overfit increases, the proposed CT statistic drops below 0 and significantly below -1, detecting copying in forced cases and complex overfit VAEs and GANs.
Source: §4.3 · p. 10Takeaways & Limitations
The test detects data-copying in the studied settings, including complex generative models, where common tests are insensitive to explicit copying.
Source: §4.3 · p. 10Limited test samples increase statistic variance and restrict partition granularity, limiting how well the test handles variation across instance-space regions.
Source: §4.3 · p. 10
Abstract
from arXiv · showhide
Detecting overfitting in generative models is an important challenge in machine learning. In this work, we formalize a form of overfitting that we call {\em{data-copying}} -- where the generative model memorizes and outputs training samples or small variations thereof. We provide a three sample non-parametric test for detecting data-copying that uses the training set, a separate sample from the target distribution, and a generated sample from the model, and study the performance of our test on several canonical models and datasets. For code \& examples, visit https://github.com/casey-meehan/data-copying
1 Introduction
The paper defines data-copying as a form of generative-model overfitting that existing evaluation tests can miss, and introduces a three-sample non-parametric test. Experiments show that, with enough samples, the test detects data-copying across a broad range of settings.
1.1 Related work
- Most existing sample-based tests target mode collapse or dropping, and models that reproduce training examples or slight variations can pass them.Source: §1 · p. 1, §1.1 · p. 2
1 Introduction
- Data-copying occurs when a generator recreates small variations of training examples rather than representing the target distribution’s diversity.Source: §1 · p. 1
- The test compares generated samples’ distances to their nearest training examples against those of an independent test sample.Source: §1 · pp. 1–2
- Because copying and underfitting can occur in different regions, the test partitions the instance space, tests within cells, and combines the results.Source: §1 · p. 2
- With enough samples, the proposed test successfully detects data-copying across a broad range of settings.Source: §1 · p. 2
1.1 Related work
- The Two-Sample Nearest Neighbor test can detect exact copying, but the proposed test also estimates whether generated samples are closer to training data than a held-out sample indicates.Source: §1.1 · p. 3
- Unlike concurrent parametric approaches, this work offers a probabilistically motivated, model-agnostic non-parametric test.Source: §1.1 · p. 3
2 Preliminaries
The setup treats overfitting as a generator producing samples too close to its training data, assessed using a training set and fresh samples from the target and generator. Data-copying is a local form of overfitting distinct from overrepresentation.
2 Preliminaries
- The setup uses training data T to build a generator Q, then assesses closeness to T using fresh samples from the target distribution and Q.Source: §2 · p. 3
2.1 Definitions of Overfitting
- Data-copying means generated samples are systematically closer to the training set than target-distribution samples in some region, under a chosen distance measure.Source: §2.1 · pp. 3–4
- The distance-to-training-set distributions for target and generated samples provide the basis for comparing their closeness to training data.Source: §2.1 · p. 3
- A global comparison can miss local copying: a model may be closer to training data in one region yet farther away overall because it underfits elsewhere.Source: §2.1 · p. 3
- Data-copying and overrepresentation are distinct: a model can overrepresent without copying, or copy without overrepresenting, as illustrated by Gaussian-mixture and bootstrap examples.Source: §2.1 · p. 4
3 A Test For Data-Copying
Section 3 develops a model-agnostic test based on distances from training examples, then extends it to handle region-dependent behavior. It also gives performance guarantees for the global statistic and identifies limits on guarantees for the average statistic.
3.1 A Global Test
- The global Mann–Whitney U test compares target and generated samples' distances to the training set; strongly negative ZU indicates copying, while strongly positive ZU indicates underfitting.The test is model-agnostic and requires a meaningful distance metric, not a likelihood estimate.Source: §3.1 · p. 4
3.2 Handling Heterogeneity
- Because a global test can miss generators that are close to training data in some regions but far away in others, the method tests cells of a partition separately.It averages copying scores over cells sufficiently represented by Q, using threshold τ.Source: §3.2 · pp. 4–5
- The complementary NDB-over and NDB-under counts flag significantly over- and underrepresented cells at significance level s=0.05.Together with CT, these statistics summarize how Q represents P without directly copying the training set.Source: §3.2 · p. 5
3.3 Performance Guarantees
- The global U statistic consistently estimates ΔT(P,Q); when Q=P and the distance distribution is non-atomic, its normalized value is near 1/2.The average statistic's performance guarantees are left for future work.Source: §3.3 · p. 5
- For a Gaussian KDE, the bandwidth satisfying Equation 2 is the maximum-likelihood choice of σ.The paper also gives a performance guarantee for the global test statistic, while leaving guarantees for the average test as future work.Source: §3.3 · p. 5
4 Experiments
The experiments ask whether existing generative-model tests detect data-copying and whether CT detects it across models as they move from overfitting to underfitting. Across KDEs, VAEs, and GANs, CT detects copying, while several popular baselines are insensitive.
4.2.1 KDE-based tests
- In KDE experiments, CT responds sharply to over- and underfitting and identifies the MLE model near CT ≈0, whereas the NN baseline is less sensitive to overfitting.On MNIST, the NN baseline provides no meaningful estimate when the generated sample is much smaller than the training set.Source: §4.2.1 · pp. 7–8, Figure 3 · p. 7
- For MNIST KDEs, pixel-space L2 distances are not perceptually meaningful and 784 dimensions make KDE inefficient, so the experiments use a 64-dimensional latent space.The authors describe this KDE as an instructive probe rather than a powerful generative model.Source: §4.2.1 · p. 8
4.2.2 Variational Autoencoders
- For MNIST VAEs, CT falls as model complexity increases and rises for underfitting; the ELBO-based generalization-gap estimate is insensitive to both.The authors suggest the ELBO’s status as a lower bound may explain the gap test’s insensitivity.Source: Figure 4 · p. 8, §4.2.2 · p. 9
4.2.3 ImageNet GAN
- For the ImageNet BigGAN, CT remains below zero until truncation is nearly maximized, while the NN baseline stays near 0.5 across truncation values.FID finds the aggregate distributions roughly similar, although the authors report too much probability mass near training samples.Source: §4.2.3 · p. 9
4.1 Detecting data-copying
- FID, Precision and Recall, and Binning-Based Evaluation are insensitive to explicit data-copying, while the three-sample kMMD test also fails to detect it.On moons, kMMD treats the MLE KDE similarly to models with σ ≪σMLE.Source: §4.1 · p. 7, §4.3 · p. 10
4.3 Discussion
- CT detects data-copying in explicitly forced KDEs and in overfit VAEs and GANs, with values falling below 0 as overfitting increases.In the neural-model experiments, CT falls significantly below -1 as models overfit more.Source: Figure 4 · p. 8, §4.3 · p. 10
- CT’s test-sample requirement limits partition granularity and can mischaracterize cells; the BigGAN experiment had 50 test images per class and only three cells.The authors identify more data-efficient handling of heterogeneity as future work.Source: §4.3 · p. 10
5 Conclusion
The conclusion formalizes data-copying as a generative-model overfitting failure mode and presents preliminary tests and experiments indicating it occurs across a broad class of models.
5 Conclusion
- The paper formalizes data-copying as an under-explored form of generative-model overfitting and provides preliminary tests for measuring it.The reported experiments indicate data-copying in a broad class of generative models.Source: §5 · p. 10
- Future work is proposed on theoretical properties, convergence guarantees for the tests, and experiments with different model parameters.Source: §5 · p. 10
7.1 Proof of Theorem 1
The proof applies the bounded differences inequality to the normalized statistic as a function of independent model and target-distribution samples.
7.1 Proof of Theorem 1
- For true distribution P, model distribution Q, and distance metric d, the estimator converges to Δ(P, Q) under the stated concentration inequality.Source: §7.1 · p. 12
- The normalized statistic is treated as a function of m independent samples from Q and n independent samples from P.The proof bounds how much it changes when an individual sample is substituted.Source: §7.1 · p. 12
- The proof uses bounds on the statistic’s changes under sample substitutions to apply the bounded differences inequality.The passage identifies separate bounds for substituting model samples and target-distribution samples.Source: §7.1 · p. 12
7.2 Proof of Theorem 2
The proof considers the case where Q=P and the corresponding distance distribution is non-atomic, using symmetry of independent draws from that distribution.
7.2 Proof of Theorem 2
- When Q=P and the distance distribution is non-atomic, independent draws have equal probabilities of either strict ordering and zero probability of equality.The proof uses these facts to establish the stated result for this case.Source: §7.2 · p. 12
- The proof partitions the joint event space for two draws from the same distance distribution into greater-than, less-than, and equality cases.Source: §7.2 · p. 12
7.3 Proof of Lemma 3
The proof derives the maximum-likelihood bandwidth condition for the KDE and uses it to establish Lemma 3.
7.3 Proof of Lemma 3
- The proof sets the derivative of the likelihood expression to zero and simplifies to obtain the condition satisfied by the maximum-likelihood σ.It then combines that condition with the mixture interpretation of Qσ to conclude the lemma.Source: §7.3 · pp. 12–13
7.4 Procedural Details of Experiments
The experiments use progressively more complex settings—from a controllable KDE on synthetic moons to MNIST models and class-conditional ImageNet GANs—with distance tests run in embedded spaces where needed.
7.4.1 Moons Dataset, and Gaussian KDE
- The moons dataset provides a low-dimensional, visualizable setting with configurable noise and ample samples for preliminary KDE experiments.The Gaussian KDE is used because its likelihood is theoretically related to the non-parametric test and its bandwidth gives direct control over copying.Source: §7.4.1 · p. 13
7.4.2 Moons Experiments
- For the moons experiments, the study fixes 2,000 training points and 1,000 each for test and generated samples, repeating generation ten times.The moons dataset is low-dimensional and visualizable, while Gaussian KDE bandwidth controls copying: σ = 0.01 effectively resamples training data, σ = 0.13 is nearly MLE, and σ = 0.5 underfits.Source: §7.4.2 · p. 13, §7.4.1 · p. 13
- The moons baseline procedures include fitting MLE Gaussians for the Fréchet distance and testing binned training-versus-generated frequencies with a one-sided hypothesis test.The study also reports three leave-one-out nearest-neighbor accuracies; its interpretation can be unclear when a copied training point is absent from the subsampled training set.Source: §7.4.2 · pp. 13–14
7.4.3 MNIST Experiments
- On MNIST, a perceptual convolutional autoencoder maps images to 64 dimensions, where L2 distance is treated as meaningful after inspecting interpolations between training digits.The VGG perceptual loss is not trained on MNIST, and the standard autoencoder is chosen to permit near-exact encoding and decoding of training samples.Source: §7.4.3 · pp. 14–15, Figure 7 · p. 15
- The MNIST experiments evaluate KDEs across bandwidths and twenty fully connected VAEs with latent dimensions from 5 to 100, using the 64-dimensional autoencoder space for test scores.KDE tests use five generation trials; the VAE samples are decoded to image space and re-encoded before CT and nearest-neighbor tests.Source: §7.4.3 · p. 15
7.4.4 ImageNet Experiments
- The ImageNet analysis treats each selected BigGAN class as its own generator, subsamples generated data for equal-size nearest-neighbor samples, and notes that the limited test set makes finer partitions difficult.The example shows that underfit and overfit cells can be difficult to distinguish, while the data-copied cell appears close to training samples.Source: §7.4.4 · p. 16, Figure 10 · p. 16, Figure 9 · p. 16
- On ImageNet, the experiments use 1,300 training images per class, 2,000 generated images, and only 50 test images, limiting the partition to about three cells.Images are embedded with InceptionV3 Pool3 features, projected onto 64 training-set principal components, then partitioned with k-means; more cells would strain the Central Limit Theorem assumption for ZU.Source: §7.4.4 · p. 16
7.5 Comparison with three-sample Kernel-MMD:
The comparison finds that three-sample kernel MMD is weak at detecting data-copying in the KDE experiments and does not show the proposed test’s sensitivity to over-parameterized MNIST VAEs. It can, however, signal KDE underfitting at bandwidths well above MLE.
7.5 Comparison with three-sample Kernel-MMD:
- For KDEs, the kMMD distances to training remain near zero below the MLE bandwidth, making the test weak for detecting data-copying, while the gap diverges at much larger bandwidths.The reported kMMD hypothesis-test p-value also treats the MLE bandwidth much like overfit bandwidths.Source: §7.5 · p. 17
- For MNIST VAEs, kMMD does not appear sensitive to over-parameterized models with d > 50 in the way the proposed CT test is.The cited VAE comparison concerns models with latent dimension d > 50; the paper notes that this is not the extreme-copying KDE setting.Source: §7.5 · pp. 17–18
- Under extreme copying by bootstrap resampling, kMMD between training and generated samples can be near zero, as can kMMD between training and independent test samples.The explanation is that collision terms become a rapidly shrinking fraction of the summations as sample sizes increase.Source: §7.5 · pp. 17–18