PaperlayerView exact revision on arXiv

Source-linked AI summary

Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

William Fedus, Mihaela Rosca, Balaji Lakshminarayanan, Andrew M. Dai, Shakir Mohamed, Ian Goodfellow

arXiv:1710.08446v3stat.MLcs.LG

TL;DR

The paper examines whether GAN training should be understood as minimizing a divergence at every step, rather than merely reaching a divergence-minimizing equilibrium. Through synthetic and real-data experiments, it shows that non-saturating GANs can learn where Jensen-Shannon minimization would fail and that gradient penalties help beyond their original theoretical settings.

  • Problem

    It was unclear whether improvements among GAN methods came from changing the underlying divergence or from changing learning dynamics.

  • Method

    The paper studies GANs on synthetic low-dimensional-manifold tasks with known distributions and evaluates common variants using independent real-data metrics.

  • Results

    Non-saturating GANs learn distributions that Jensen-Shannon divergence minimization would fail to learn, while gradient penalties improve their training dynamics and robustness.

  • Takeaways & Limitations

    GAN training is more usefully viewed as approaching Nash equilibria through trajectories that need not minimize a specific divergence at every step.

  • Takeaways & Limitations

    The paper notes that the mechanisms by which related gradient regularizers stabilize GAN training remain for future study.

Abstract

from arXiv · show

Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion. Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost. GANs are designed to reach a Nash equilibrium at which each player cannot reduce their cost without changing the other players' parameters. One useful approach for the theory of GANs is to show that a divergence between the training distribution and the model distribution obtains its minimum value at equilibrium. Several recent research directions have been motivated by the idea that this divergence is the primary guide for the learning process and that every step of learning should decrease the divergence. We show that this view is overly restrictive. During GAN training, the discriminator provides learning signal in situations where the gradients of the divergences between distributions would not be useful. We provide empirical counterexamples to the view of GAN training as divergence minimization. Specifically, we demonstrate that GANs are able to learn distributions in situations where the divergence minimization point of view predicts they would fail. We also show that gradient penalties motivated from the divergence minimization perspective are equally helpful when applied in other contexts in which the divergence minimization perspective does not predict they would be helpful. This contributes to a growing body of evidence that GAN training may be more usefully viewed as approaching Nash equilibria via trajectories that do not necessarily minimize a specific divergence at each step.

1 INTRODUCTION

GANs learn a target distribution through competing generator and discriminator objectives, making training dynamics difficult to interpret through a single divergence. The paper tests whether claimed divergence-based improvements instead reflect learning dynamics.

  • GAN framework: GANs use a generator G to represent pmodel(x) and a discriminator D to distinguish generated samples from training data.Samples are produced as x = G(z), with z drawn from a diverse noise distribution.
  • Motivation: The training goal is to recover pdata, but GAN variants have been interpreted as minimizing different divergences or distances between pdata and pmodel.This makes it difficult to separate improvements caused by the underlying divergence from improvements caused by learning dynamics.
  • Study design: The paper conducts experiments on simplified synthetic tasks with known pdata and straightforward distance calculations, then evaluates models with independent measures on real data.The study compares proposed models against baselines and uses several evaluation measures to examine new approaches.
  • Terminology: The authors clarify that “standard GAN” terminology can conflate minimax and non-saturating losses, although many criticisms apply only to minimax GANs.They therefore distinguish minimax GAN (M-GAN) from non-saturating GAN (NS-GAN).
  • Contributions: Gradient penalties motivated by divergence minimization or game theory also improve non-saturating GANs on synthetic and real data.This tests whether their benefits depend on the theoretical setting that originally motivated them.

2 VARIANTS OF GENERATIVE ADVERSARIAL NETWORKS

GAN variants differ in discriminator outputs, generator objectives, and gradient regularization. The paper focuses on non-saturating GANs and tests whether gradient penalties help outside their original theoretical settings.

  • Original GAN: The original GAN discriminator outputs a probability and minimizes binary-classification negative log-likelihood.The discriminator cost is defined using expected log probabilities for real and generated samples.
  • Generator objectives: Minimax GANs maximize the discriminator objective with respect to the generator, whereas non-saturating GANs minimize an alternative objective that encourages generated samples to appear real.The non-saturating objective was recommended in practice and used in the original implementation.
  • Equilibrium interpretation: When the discriminator is optimal, the minimax formulation connects generator optimization to Jensen-Shannon divergence and equilibrium at pdata = pmodel.This correspondence characterizes the Nash equilibrium in function space.
  • Wasserstein GAN: Wasserstein GANs use an unconstrained real-valued discriminator and approximately minimize earth mover’s distance when the discriminator is Lipschitz smooth.Weight clipping is described as one way to enforce Lipschitz smoothness.
  • Gradient penalties: WGAN-GP applies a gradient-norm penalty along interpolations between data points and samples, while DRAGAN penalizes gradients around the data manifold.Their motivations differ: Wasserstein-distance properties for WGAN-GP and regret minimization for DRAGAN.
  • Experimental variants: The experiments apply GAN-GP and DRAGAN penalties only to the discriminator while leaving the non-saturating generator loss unchanged.The study compares penalty forms and separates effects of adversarial loss from effects of regularization.
  • Scope: The study excludes the original DRAGAN formulation because it uses the minimax game, while this work focuses on non-saturating variants.The comparison therefore uses DRAGAN-NS rather than original DRAGAN.

3 MANY PATHS TO EQUILIBRIUM

The paper argues that divergence minimization describes GAN equilibrium but not necessarily the path taken during training. Its experiments show that non-saturating GANs can learn where Jensen-Shannon minimization would provide no useful gradient.

  • Equilibrium versus dynamics: GAN training can approach the Jensen-Shannon equilibrium through many trajectories that do not reduce the divergence at every step.Divergence minimization remains useful for understanding the outcome, but not necessarily the learning process.
  • Divergence-based prediction: Prior analysis argued that JS, KL, and Total Variation divergences provide unusable gradients when distributions lie on non-overlapping low-dimensional manifolds.This perspective associated the problem with traditional GANs, apparently referring to the minimax formulation.
  • Non-saturating gradients: Non-saturating GANs can learn two widely separated Gaussians because their generator loss preserves strong gradients even when the optimal discriminator’s gradient nearly vanishes.The loss amplifies small differences in discriminator outputs to recover a useful generator signal.
  • Loss parameterization: Sample-based loss parameterizations can address the parallel-line problem because the two density functions have no support in common.The paper identifies sample-based parameterization as a mechanism used by many GANs and other models.

4 SYNTHETIC EXPERIMENTS

The synthetic experiments test GAN convergence on known low-dimensional distributions, including cases where divergence-based reasoning predicts unhelpful gradients. They find that non-saturating GANs can learn these distributions, while gradient penalties improve stability, convergence, and robustness.

  • Experiment setup: The experiments use pathological synthetic tasks with known generating parameters to test GAN convergence where underlying divergence gradients are unhelpful.The first task embeds one-dimensional Gaussian data on a line in a higher-dimensional space; the generator has the same intrinsic dimensionality.
  • Experiment setup: The overcomplete-generator experiment gives the generator more latent dimensions than the one-dimensional data manifold requires.The generator can therefore represent a manifold with higher dimensionality than the underlying data.
  • Training protocol: All GAN variants are trained for 200000 generator updates, with five discriminator updates per generator update.Throughout the experiments, reported steps count generator updates.
  • Main results: Gradient penalties, including DRAGAN-NS and GAN-GP, stabilize training and improve convergence.DRAGAN-NS applies the penalty near the data manifold, whereas GAN-GP applies it along interpolations between data and samples.
  • Main results: Despite Jensen-Shannon divergence minimization failing on the task, non-saturating GANs converge to the one-dimensional data manifold and can learn distributions disjoint from the initial training distribution.In higher dimensions, their fit becomes weaker than other methods when the learning rate is held fixed.
  • Hyperparameter sensitivity: Gradient penalties improve robustness, while Wasserstein GAN formulations remain robust to hyperparameter changes and unregularized non-saturating GANs are most sensitive.Increasing dimensions at a fixed learning rate can reduce non-saturating GAN performance; excessive discriminator updates also hurt it, while GAN-GP scales better.

5 REAL DATA EXPERIMENTS

Real-data experiments compare non-saturating GAN variants and gradient-penalty methods across Color MNIST, CIFAR-10, and CelebA using multiple evaluation measures and hyperparameter sweeps. Gradient penalties improve training robustness and sample quality for non-saturating GANs, while performance varies by dataset and metric.

  • Experimental setup: The experiments evaluate non-saturating GAN, GAN-GP, DRAGAN-NS, and WGAN-GP on Color MNIST, CIFAR-10, and CelebA.The datasets have dimensionalities (28, 28, 3), (32, 32, 3), and (64, 64, 3), respectively.
  • Evaluation: Performance is assessed with visual inspection, Inception Score, MS-SSIM diversity, and an independent Wasserstein critic because the true data distributions are unknown.The study controls discriminator and generator architectures across models.
  • Hyperparameter robustness: Gradient penalties make non-saturating GANs more robust to hyperparameter changes across the reported sweeps.Each sweep covers 27 settings formed from three discriminator learning rates, three generator learning rates, and three latent sizes.
  • Sample quality: Gradient-penalized non-saturating GANs produce better samples and better Inception Scores across both best-setting and full-sweep evaluations.This result is reported for the non-saturating GAN variants with gradient penalties.
  • Efficiency: Non-saturating GAN variants with penalties train faster than WGAN-GP and perform similarly, yielding a better computation versus performance tradeoff.The non-saturating variants use one discriminator update per generator update, whereas WGAN-GP uses five in the real-data experiments.
  • Metric-specific outcomes: The independent Wasserstein critic favors WGAN-GP on Color MNIST and CIFAR-10, while detecting reduced quality and distribution mismatch for some CelebA settings.The critic also detects mode collapse when the negative Wasserstein distance is low and sample diversity is greatly reduced.

6 DISCUSSION

The discussion argues that GAN training dynamics cannot be understood solely through the divergence associated with an equilibrium. It also compares gradient penalties and emphasizes that their stabilizing mechanisms remain unresolved.

  • Discussion: Viewing GAN training through the underlying divergence at optimality can be misleading.Non-saturating GANs learn the true distribution on low-dimensional synthetic problems where Jensen-Shannon divergence minimization would fail.
  • Discussion: Gradient penalties improve the training dynamics and robustness of non-saturating GANs, even when applied outside their original theoretical setting.Using a Wasserstein-motivated penalty with non-saturating GANs helps separate divergence-related effects from training-dynamics effects.
  • Gradient-penalty comparison: The two evaluated penalties show no clear overall winner, and combining them does not produce better models.The authors suggest similar regularization effects or optimization conflicts as possible explanations.
  • Related regularization: Several GAN regularizers share a stabilizing spirit but arise from distinct theoretical considerations.The discussion includes DRAGAN, WGAN-GP, f-GAN-GP, and Fisher-GAN.
  • Open questions: Future study is needed to determine how these regularizers interact and what mechanisms stabilize GAN training.The paper identifies this as an open direction for understanding and motivating new approaches.

A.1 SYNTHETIC EXPERIMENTS

Synthetic experiments compare GAN variants using square Fréchet distance between generated and true Gaussian parameters, averaging results over 1000 runs. The experiments vary learning rates while holding input dimension fixed.

  • Synthetic evaluation: Synthetic Experiment 1 compares GAN variants by square Fréchet distance between generated and true Gaussian parameters.Lower values indicate a closer match.
  • Synthetic evaluation: Synthetic Experiment 2 uses the same square Fréchet-distance evaluation while varying learning rates with input dimension fixed.Results are averaged over 1000 runs, and lower values are better.

A.2 THE EFFECT OF THE NUMBER OF DISCRIMINATOR UPDATES ON GAN AND GAN-GP

The appendix examines how discriminator-update frequency affects GAN and GAN-GP distribution fitting. More discriminator updates do not uniformly improve the match to the true distribution.

  • Discriminator updates: Using 100 discriminator updates per generator update produces a bad distribution fit for the non-saturating GAN.GAN-GP scales better as the discriminator-update count increases.
  • Discriminator updates: Increasing discriminator updates does not always bring the non-saturating GAN closer to the true distribution.The comparison is evaluated using square Fréchet distance across learning-rate settings.

A.3 REAL DATA EXPERIMENTS

The real-data experiments present samples and evaluation metrics across GAN variants and datasets, including evidence of mode collapse and failures to capture data distributions.

  • Synthetic-experiment figures evaluate GAN-GP using squared Fréchet distance across discriminator-update counts and learning rates, averaged over 1000 runs.Lower squared Fréchet distance indicates closer generated and true Gaussian parameters.
  • Sample diversity and negative Wasserstein distance captured the two observed DRAGAN-NS mode-collapse examples in a CelebA hyperparameter sweep.
  • Figures 18–20 show CIFAR-10, CelebA, and CMNIST samples from GAN-GP, DRAGAN-NS, and WGAN-GP.
  • GAN-GP exhibited two mode-collapse instances, while DRAGAN-NS exhibited two CelebA mode-collapse instances.GAN-GP’s reported collapse occurred as color mode collapse on Color-MNIST; DRAGAN-NS’s occurred on CelebA.
  • WGAN-GP examples on CelebA show excessive probability mass around the data distribution, indicating failure to capture that distribution accurately.
Loading 1710.08446v3…