Source-linked AI summary
A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'
Ali Borji
TL;DR
The paper asks whether model collapse from recursively generated data can be understood as a statistical phenomenon. It studies repeated fitting and resampling with KDE and related distributions, finding progressive collapse and growing divergence, with outcomes dependent on the distance metric. The authors conclude that the phenomenon may reflect a broader limitation of generative models in faithfully reproducing data distributions.
Problem
The paper addresses how recursively generated data can cause model collapse and whether the phenomenon has a systematic theoretical explanation.
Method
The study repeatedly fits KDE-based distributions to synthetic mixtures, samples from the fitted distributions, refits them, and measures divergence from the original data.
Results
The distributions progressively collapse toward unimodal Gaussian-looking forms, while KL divergence may stabilize and Wasserstein distance continues increasing; the outcome depends on the metric.
Takeaways & Limitations
The findings suggest that recursively generated data can expose a broader limitation in faithfully reproducing data distributions and may represent a pervasive phenomenon.
Takeaways & Limitations
The paper leaves open whether the findings apply to all distributions, whether repeated fitting always yields a unimodal Gaussian distribution, and whether the phenomenon is inevitable.
Abstract
from arXiv · showhide
The study conducted by Shumailov et al. (2024) demonstrates that repeatedly training a generative model on synthetic data leads to model collapse. This finding has generated considerable interest and debate, particularly given that current models have nearly exhausted the available data. In this work, we investigate the effects of fitting a distribution (through Kernel Density Estimation, or KDE) or a model to the data, followed by repeated sampling from it. Our objective is to develop a theoretical understanding of the phenomenon observed by Shumailov et al. (2024). Our results indicate that the outcomes reported are a statistical phenomenon and may be unavoidable.
1 Introduction
The paper examines whether recursively fitting distributions and resampling can explain model collapse observed with synthetic data. It frames the issue as a potential reliability concern as AI-generated data becomes more prevalent.
- 1 Introduction: Repeated reliance on lower-quality synthetic data can compound errors and degrade model performance.The paper connects this feedback-loop concern to the growing prevalence of AI-generated data.
- 1 Introduction: The study investigates the statistical and theoretical basis of collapse caused by recursively generated training data.It focuses on statistical sampling and distribution fitting as a way to generalize earlier findings.
- 1 Introduction: Figure 2 uses a synthetic mixture of two Gaussian components and a Uniform distribution to compare original samples with KDE-generated samples.The figure illustrates the distribution-fitting setup used in the experiments.
2 Experiments and results
The experiments repeatedly fit KDE distributions to synthetic samples and resample from them, then track divergence from the original data. Across mixture settings, the distributions collapse toward unimodal Gaussian-looking forms, while conclusions differ by distance metric.
- 2 Experiments and results: The procedure fits a Gaussian-kernel KDE to samples from a mixture of two normal distributions and one uniform distribution, then repeatedly resamples and refits.KL divergence and Wasserstein distance are computed between generated samples and the original data.
- 2 Experiments and results: After several iterations, the fitted distribution converges toward a normal-looking form as repeated smoothing removes distinctive structure.The distribution increasingly departs from the original data during this process.
- 2 Experiments and results: Two additional mixtures—three Gaussians plus one uniform, and a Gamma distribution with two Gaussians plus one uniform—also collapse to unimodal Gaussian-looking distributions.These experiments extend the observed collapse beyond the initial mixture.
- 2 Experiments and results: KL divergence rises initially and may stabilize or continue rising, whereas Wasserstein distance grows throughout the iterations.The resulting interpretation depends on which distance metric is used.
- 2 Experiments and results: Figure 3 displays recursive KDE and sampling over 30 iterations in steps of three for the initial mixture.It visualizes the progressive distributional change described in the experiments.
3 Relationship to theory
Theoretical analysis models recursive sampling from a fitted normal distribution and shows that repeated generations increase distributional distance, explaining model collapse in this setting.
- 3 Relationship to theory: Assuming normally distributed original data, each generation estimates the next model from the previous generation’s sample mean and variance.The analysis denotes samples at generation i and uses those statistics to fit the next generation model.
- 3 Relationship to theory: With constant sample size M, the theoretical variance diverges linearly as n →∞.
- 3 Relationship to theory: The Wasserstein-2 distance measures the discrepancy between the true distribution and the approximated distribution at step n + 1.
- 3 Relationship to theory: Figure 4 tracks original distributions, KDE samples after 300 iterations, and KL divergence and Wasserstein distance over those iterations.
- 3 Relationship to theory: Increasing distance at each step n directly explains model collapse in this simple model.
- 3 Relationship to theory: Empirically, Wasserstein distance consistently increases across experiments, including when the KDE bandwidth changes from 0.5 to 0.1.
4 Conclusion
The paper highlights limits in faithfully reproducing data distributions, especially their tails, and notes that distance metrics yield different implications. It calls for further theoretical work on the generality and inevitability of collapse.
- Generative models may poorly represent distribution tails, even when those tails contain critical information.The paper identifies tail modeling as a key challenge for faithfully capturing underlying distributions.
- Different distribution-distance metrics have distinct implications and limitations, complicating evaluation of collapse.The paper examines two metrics while noting that many alternatives exist.
- The reported findings may reflect a broader limitation in faithfully reproducing distributions from recursively generated data.The paper connects this concern to the long-term reliability and robustness of generative models.
- Future work should test whether the results generalize across distributions and whether repeated sampling and fitting inevitably produce unimodal Gaussian-like outcomes.The authors also question whether any viable remedy exists if the phenomenon is unavoidable.
5 Code
The code constructs mixture-distribution samples, fits and samples from a Gaussian-kernel KDE, plots the resulting distributions, and tracks KL divergence and Wasserstein distance across iterations.
- The implementation imports NumPy, Matplotlib, and scikit-learn's KernelDensity for sampling, plotting, and KDE fitting.The code defines a KL-divergence helper using histogram-based probability estimates.
- The code plots original and fitted samples, KDE density estimates, labels, titles, and density axes for visual comparison.Separate listings provide histogram and plotting commands for the distributions across iterations.
- Each iteration uses 30,000 newly drawn samples, and the recorded KL divergence and Wasserstein distance are printed for monitoring.The implementation sets the sample size and prints both metrics for the current iteration.
- The workflow repeatedly samples from the current KDE, computes KL divergence and Wasserstein distance, then refits the KDE to the new samples.The loop records both divergence sequences and uses a Gaussian kernel with bandwidth 0.5 in the shown implementation.
- Mixture samples are concatenated and shuffled before fitting the KDE model.The listings generate arbitrary mixtures and combine component samples into one dataset.