PaperlayerView exact revision on arXiv

Source-linked AI summary

Self-Improving Diffusion Models with Synthetic Data

Sina Alemohammad, Ahmed Imtiaz Humayun, Shruti Agarwal, John Collomosse, Richard Baraniuk

arXiv:2408.16333v1cs.LGcs.AI

TL;DR

Synthetic data is increasingly necessary but can drive self-consuming training loops that degrade quality, diversity, and fairness. SIMS uses self-generated data as negative guidance away from the synthetic manifold and toward the real distribution. It improves diffusion-model generation, supports iterative training without MADness under suitable settings, and enables in-domain distribution control.

  • Problem

    Synthetic data is difficult to avoid, yet naïve self-consuming training can compound errors, degrade quality and diversity, and amplify bias.

    Source: §1 · p. 2

  • Method

    SIMS trains an auxiliary model on self-generated data and uses its score function for negative guidance away from the synthetic manifold toward the real data distribution.

    Source: §1 · p. 2, §5 · p. 13

  • Results

    SIMS establishes new FID records for CIFAR-10 and ImageNet-64, achieves competitive results on FFHQ-64 and ImageNet-512, and can avoid MADness across iterations under suitable settings.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Synthetic data can support self-improving diffusion models when used to steer generation away from synthetic-data artifacts rather than aggregated directly into training.

    Source: §5 · pp. 13–14

    SIMS fully prevents MADness only below dataset-specific synthetic-to-real thresholds, although it continues to mitigate MADness above them.

    Source: §5 · p. 14

Abstract

from arXiv · show

The artificial intelligence (AI) world is running out of real data for training increasingly large generative models, resulting in accelerating pressure to train on synthetic data. Unfortunately, training new generative models with synthetic data from current or past generation models creates an autophagous (self-consuming) loop that degrades the quality and/or diversity of the synthetic data in what has been termed model autophagy disorder (MAD) and model collapse. Current thinking around model autophagy recommends that synthetic data is to be avoided for model training lest the system deteriorate into MADness. In this paper, we take a different tack that treats synthetic data differently from real data. Self-IMproving diffusion models with Synthetic data (SIMS) is a new training concept for diffusion models that uses self-synthesized data to provide negative guidance during the generation process to steer a model's generative process away from the non-ideal synthetic data manifold and towards the real data distribution. We demonstrate that SIMS is capable of self-improvement; it establishes new records based on the Fr\'echet inception distance (FID) metric for CIFAR-10 and ImageNet-64 generation and achieves competitive results on FFHQ-64 and ImageNet-512. Moreover, SIMS is, to the best of our knowledge, the first prophylactic generative AI algorithm that can be iteratively trained on self-generated synthetic data without going MAD. As a bonus, SIMS can adjust a diffusion model's synthetic data distribution to match any desired in-domain target distribution to help mitigate biases and ensure fairness.

1 INTRODUCTION

Synthetic data is increasingly unavoidable, but naïve self-consuming training can compound errors, degrade quality and diversity, and amplify bias. SIMS instead uses self-generated data for negative guidance, improving diffusion models while mitigating or preventing MADness.

Motivation

  • Synthetic-data training can compound approximation errors across generations, reducing quality or diversity and amplifying bias in generated samples.These effects are described as Model Autophagy Disorder (MAD) and Model Collapse.Source: §1 · p. 2

Method

  • SIMS uses a synthetic-data-trained auxiliary score function to extrapolate away from the synthetic manifold and toward the real data distribution.The method combines the base score sθr(xt, t) and auxiliary score sθs(xt, t) using guidance strength ω.Source: §1 · pp. 2–3
  • The procedure trains a base model on real data, generates an internal synthetic dataset, fine-tunes an auxiliary model on it, extrapolates the score function, and synthesizes new data.The internal synthetic dataset is discarded after auxiliary training.Source: §1 · p. 3

Contributions

  • SIMS establishes new FID records for CIFAR-10 and ImageNet-64, achieves competitive results on FFHQ-64 and ImageNet-512, and outperforms both the original and aggregate-trained models.The comparison with the original and real-plus-synthetic aggregate models is reported in Figures 1 and 5.Source: §1 · p. 3
  • SIMS can be iteratively trained on self-generated data for 100 generations without MAD degradation under suitable guidance-strength settings.This addresses the paper’s goal of exploiting synthetic data without entering a self-consuming degradation loop.Source: §1 · p. 3
  • SIMS can adjust a diffusion model’s synthetic distribution toward a desired in-domain target distribution, supporting bias mitigation and fairness.The paper presents distribution controllability as an additional capability of the framework.Source: §1 · pp. 3–4

2 BACKGROUND

Diffusion models learn to generate data by reversing a noise-adding process, while self-consuming training loops can compound synthetic-data errors and drift away from real data. The background distinguishes mitigating MADness from preventing it and identifies the unresolved need for closed-loop prevention without external knowledge.

Diffusion models

  • Diffusion models learn a target distribution by reversing a forward process that progressively adds noise, using a learned score function during numerical integration.The forward process is described by an SDE, and the reverse process generates samples by replacing the unknown score with a neural approximation.Source: §2 · p. 4

Self-consuming generative models

  • Self-consuming loops train later generative models on data that includes samples from earlier models, with fully synthetic, synthetic-augmentation, and fresh-data variants.This background focuses on the first two loop types, which use previous synthetic data in subsequent training.Source: §2 · pp. 4–5

Mitigating and preventing MADness

  • MADness is defined as increasing expected distributional distance across generations, while mitigation bounds that distance but may still leave later models worse than the first generation.Mitigated MAD requires E[dist(G∞, pr)] ≤ C, whereas prevention aims for E[dist(G∞, pr)] ≤ E[dist(G1, pr)].Source: §2 · pp. 4–5

Self-consuming generative models

  • 29: For fully synthetic loops, E[dist(G∞, pr)] →∞, so successive models drift away from the real distribution in the model-collapse regime.The paper states that this behavior has been shown theoretically and experimentally and calls it model collapse in the literature.Source: §2 · p. 5

Mitigating and preventing MADness

  • Prior mitigation methods use continuous real-data inclusion, accumulated synthetic data, verifiers, external guidance, or fresh real data, but closed-loop prevention without external knowledge remained unidentified.The paper positions SIMS as addressing this gap by fully preventing MADness in a synthetic augmentation loop.Source: §2 · p. 5

3 SIMS: SELF IMPROVEMENT WITH SYNTHETIC DATA

SIMS self-improves diffusion models by using an auxiliary model trained on the base model’s synthetic data to provide negative guidance. Its guidance strength, synthetic dataset size, and auxiliary training budget determine how effectively it moves generation away from synthetic-data artifacts, while inference requires additional computation.

SIMS framework and related guidance

  • SIMS is presented for unconditional diffusion models but extends to conditional diffusion models as well, and its guidance framework relates to prior negative-guidance approaches.The paper contrasts SIMS with classifier, discriminator, classifier-free, and other bad-model guidance methods.Source: §3 · pp. 5–7

SIMS: Extrapolating to Self-Improvement

  • SIMS uses a base model’s synthetic data to train an auxiliary score function, then subtracts the synthetic-induced shift during generation as negative guidance.The resulting score is sθ(xt, t) = (1 + ω)sθr(xt, t) − ωsθs(xt, t), where ω is the guidance strength.Source: §3 · p. 6
  • The auxiliary dataset size must balance guidance quality: very large datasets make the auxiliary score converge to the base score, while very small datasets yield ineffective estimates.Experiments indicate that a synthetic dataset roughly matching the real dataset size works well.Source: §3 · p. 6
  • The auxiliary training budget must produce a score neither too similar to nor too different from the base score, so fine-tuning stops at an optimizing budget B.At B = 0, SIMS reduces to the base model; increasing B initially approaches the ground truth before synthetic-data influence dominates.Source: §3 · p. 6

Inference Computational Cost

  • SIMS requires twice the base model’s inference function evaluations because it uses an auxiliary model, although limited-interval guidance or partial fine-tuning can reduce this cost.The paper reports that function evaluations can be reduced with minimal performance impact using these strategies.Source: §3 · p. 6

4 EXPERIMENTAL RESULTS

The experimental results evaluate SIMS across image-generation benchmarks, examining its self-improvement, behavior under synthetic-data training, and ability to shift generated distributions toward specified in-domain targets. They also assess how guidance strength and synthetic-data pollution affect effectiveness and illustrate self-improvement under distribution shift.

4.1 SELF-IMPROVING DIFFUSION MODELS

  • FID improves by 32.5%, 56.9%, 41.8%, and 32.4% on CIFAR-10, FFHQ-64, ImageNet-64, and ImageNet-512, respectively, with SIMS outperforming its base model across all four datasets.With EDM-VP, SIMS reaches CIFAR-10 FID 1.33 and, with EDM2-S, ImageNet-64 FID 0.92, improving over the closest baselines by 13.6% and 8.9%.Source: §4.1 · pp. 8–9, Table 1 · p. 8

4.2.1 TWO DIMENSIONAL GAUSSIAN DATA IN A SYNTHETIC AUGMENTATION LOOP

  • In a two-dimensional Gaussian self-consuming loop, standard training’s Wasserstein distance ratio rises above 1, whereas optimized SIMS keeps it at 1 and thereby completely prevents MADness.A guidance strength of approximately ω = 3 yields strong MADness mitigation or complete prevention; SIMS is described as the first synthetic-data learning algorithm to prevent MAD without external knowledge.Source: §4.2.1 · pp. 10–11, Figure 4 · p. 10
  • SIMS’ MAD-prevention threshold depends on synthetic-data pollution: for synthetic sets of sizes 125, 250, 500, and 1000, its convergence ratios are 0.996, 1.013, 1.078, and 1.204 versus 1.71, 2.46, 3.99, and 6.69 for standard training.The results suggest complete prevention below approximately |D_s| = 250, with only mitigation possible above that threshold in this experiment.Source: §4.2.1 · p. 10

4.2.2 REALISTIC DATA IN A SYNTHETIC AUGMENTATION LOOP

  • On CIFAR-10 and FFHQ-64, SIMS remains relatively insensitive to polluting synthetic data and can simultaneously prevent MADness and improve FID when |D_p| < 30k and |D_p| < 15k, respectively.For CIFAR-10, SIMS reduces FID from 1.41 to 1.33, establishing a new state-of-the-art benchmark, while standard-training performance deteriorates as pollution increases.Source: Figure 5 · p. 11, §4.2.2 · p. 12

4.3 DISTRIBUTION SHIFTS WITH SIMS

  • SIMS can shift generated samples toward an arbitrary in-domain target distribution: on FFHQ-64, increasing ω changes the female-image proportion from 32% at ω = −1 to approximately 68% at ω = 1.5.The target was 70% female and 30% male, compared with approximately 50% female generation from the base model.Source: §4.3 · pp. 12–13, Figure 7 · p. 13
  • Distribution shifting and image quality need not be optimized at the same guidance strength: male-image FID is minimized at ω = 1.5, whereas female-image FID is minimized at ω = 1.25.Thus, the guidance level achieving approximately 70% female generation coincides with the male-quality optimum but not the female-quality optimum.Source: Figure 6 · p. 12, §4.3 · p. 13
  • Figure 7 shows sample synthetic images for the baseline model and the final model, both distribution-shifted, with the final model self-improved.The baseline appears on the left and the final model on the right.Source: §4.3 · p. 13

5 DISCUSSION

SIMS uses synthetic data as negative guidance rather than aggregating it with real data, improving diffusion models while addressing self-consuming degradation. The discussion identifies controllability benefits, synthetic-data thresholds, and open directions beyond the current setting.

  • SIMS avoids combining real and synthetic data, instead using synthetic data for negative guidance toward the real-data distribution.This design addresses the divergence, quality loss, and bias reinforcement associated with naïve synthetic-data aggregation.Source: §5 · pp. 13–14
  • SIMS establishes new records on CIFAR-10 and ImageNet-64 and is reported as the first generative AI model iteratively trained on self-generated data without going MAD.Source: §5 · p. 14
  • SIMS can adjust synthetic outputs toward any desired in-domain target distribution, supporting bias mitigation and model fairness.Source: §5 · p. 14
  • SIMS can fully prevent MADness only below dataset-specific synthetic-to-real ratios, although it continues mitigating MADness above those thresholds.Reported thresholds are approximately 60% for CIFAR-10, 20% for FFHQ-64, and 25% for two-dimensional Gaussian data.Source: §5 · p. 14
  • Future work includes testing auxiliary models from different architectures, explaining SIMS’s use of polluted datasets, extending prophylaxis beyond diffusion, and incorporating user preferences.Source: §5 · p. 14

A ABLATION STUDIES FOR SIMS

The ablations show that SIMS’s performance depends on synthetic-dataset size and guidance computation, with practical strategies reducing its inference overhead. Larger auxiliary datasets help until the auxiliary and base score functions become too similar.

Synthetic dataset size

  • Increasing the auxiliary synthetic-dataset size improves ImageNet-64 FID, but very large datasets can eliminate guidance gains as auxiliary and base scores converge.Source: A ABLATION STUDIES FOR SIMS · p. 18

Number of function evaluations

  • SIMS needs more function evaluations than unguided sampling to reach its lowest FID, while both methods are nearly equal at NFE=40 with FID 1.70.Source: A ABLATION STUDIES FOR SIMS · p. 18

Reducing number of function evaluations

  • Freezing the EDM encoder and training only the decoder reduces effective evaluations from 2x to 1.5x, while minimum FID increases from 0.92 to 1.01.Source: A ABLATION STUDIES FOR SIMS · p. 18
  • Applying guidance only during later denoising steps reduces FID from 0.93 to 0.96 when the first 10 steps are excluded, lowering inference time and cost.Source: A ABLATION STUDIES FOR SIMS · p. 18

F STANDARD TRAINING

Standard training fits a diffusion model on a dataset and synthesizes data from its learned score function. Within SIMS, it is equivalent to using no guidance, but an already trained standard model can still undergo the later SIMS steps.

Standard Training Procedure

  • Standard training trains a diffusion model on dataset D and generates synthetic data using the resulting score function sθ(x_t, t).Source: F STANDARD TRAINING · p. 24
  • Relative to SIMS, standard training is equivalent to setting guidance strength ω=0 and using only the base model’s score function.Source: F STANDARD TRAINING · p. 24
  • A model already trained by standard training can still be converted into a self-improved model by applying SIMS steps 2–4.Source: F STANDARD TRAINING · p. 24

Related papers

Loading 2408.16333v1…