PaperlayerView exact revision on arXiv

Source-linked AI summary

Unrolled Generative Adversarial Networks

Luke Metz, Ben Poole, David Pfau, Jascha Sohl-Dickstein

arXiv:1611.02163v4cs.LGstat.ML

TL;DR

GAN training can be unstable and suffer from mode collapse. The paper stabilizes it by defining the generator objective through unrolled discriminator optimization, and reports improved coverage, reconstruction, diversity, and stability across tasks, at increased computational cost.

  • Problem

    GAN training can suffer from mode collapse and instability, while generators may fail to cover the data distribution.

  • Method

    The method trains the generator using a surrogate objective formed by unrolling discriminator optimization for K steps.

  • Results

    The technique either rescued unstable training or reduced the tendency to drop regions of the data distribution, with increasing unrolling improving mode coverage and reverse KL divergence.

  • Takeaways & Limitations

    Unrolling provides a tunable compromise between standard GAN dynamics and the more costly true generator objective, and can reduce mode collapse.

  • Takeaways & Limitations

    Each training step becomes linearly more expensive in the number of unrolling steps, and the appropriate number depends on model stability and architecture.

Abstract

from arXiv · show

We introduce a method to stabilize Generative Adversarial Networks (GANs) by defining the generator objective with respect to an unrolled optimization of the discriminator. This allows training to be adjusted between using the optimal discriminator in the generator's objective, which is ideal but infeasible in practice, and using the current value of the discriminator, which is often unstable and leads to poor solutions. We show how this technique solves the common problem of mode collapse, stabilizes training of GANs with complex recurrent generators, and increases diversity and coverage of the data distribution by the generator.

1 INTRODUCTION

GANs offer flexible likelihood-free generative modeling, but their training commonly suffers from mode collapse, oscillation, and unstable learning. This work addresses these challenges by unrolling discriminator optimization during generator training.

  • GANs use a generator and discriminator in a minimax game without explicit likelihood computation or inference.
  • Training can collapse the generator to one sample or a small family, oscillate instead of converging, or produce useless learning signals when one agent dominates.
  • Even after convergence, GAN generators typically fail to cover the whole data distribution, while evaluating performance remains difficult.
  • The paper addresses unstable optimization and mode collapse by unrolling optimization of the discriminator objective during training.

2 METHOD

The method replaces the infeasible optimal-discriminator objective with a differentiable surrogate obtained by unrolling discriminator updates. The number of unrolling steps interpolates between standard GAN training and optimization toward the true generator objective, while adding information about discriminator responses to generator changes.

  • 2.1 GENERATIVE ADVERSARIAL NETWORKS: The true generator objective uses the optimal discriminator, but computing that discriminator for every generator update is infeasible with neural networks.
  • 2.1 GENERATIVE ADVERSARIAL NETWORKS: GAN training therefore alternates gradient descent on generator parameters with gradient ascent on discriminator parameters.
  • 2.2 UNROLLING GANS: The surrogate discriminator parameters are produced by treating discriminator optimization as a differentiable iterative procedure.
  • 2.2 UNROLLING GANS: At K = 0, the surrogate matches the standard GAN objective; as K →∞, it approaches the true generator objective with an optimized discriminator.
  • 2.3 PARAMETER UPDATES: Generator gradients are backpropagated through the unrolled discriminator optimization, unlike the discriminator update, which does not depend on that unrolling.
  • 2.4 THE MISSING GRADIENT TERM: The missing gradient term models how the discriminator reacts to generator changes, discouraging collapse toward a single point and potentially improving stability.

3 EXPERIMENTS

Experiments show that discriminator unrolling improves GAN stability, mode coverage, and reconstruction across increasingly complex settings. Benefits appear in toy mixtures, recurrent generators, discrete and continuous distributions, and CIFAR10 image modeling.

  • 3.1 MIXTURE OF GAUSSIANS DATASET: Unrolling stabilized training on a toy mixture of eight Gaussians, spreading probability mass across modes and converging to the data distribution.Without unrolling, the generator rotated among valid modes without spreading mass.
  • 3.1 MIXTURE OF GAUSSIANS DATASET: Unrolled training increased stability for an LSTM generator producing MNIST images with a convolutional discriminator.The experiment used 20 unrolling steps versus 0 for standard GAN training.
  • 3.2 PATHOLOGICAL MODEL WITH MISMATCHED GENERATOR AND DISCRIMINATOR: As unrolling steps increased, both discrete mode coverage and reverse KL divergence improved across the 1,000-mode dataset.The dataset contained all combinations of three MNIST digits, and the comparison included two architectures.
  • 3.2 PATHOLOGICAL MODEL WITH MISMATCHED GENERATOR AND DISCRIMINATOR: Unrolling benefits increased as discriminator size decreased, suggesting that unrolling effectively increases discriminator capacity.The authors attribute the larger benefit to a weak discriminator’s improved ability to react to non-data-like generator samples.
  • 3.3.2 MANIFOLD COLLAPSE: Models unrolled for 5–10 steps generally performed best on continuous color diversity, while one step sometimes worsened the measure.Larger models also performed better, and the authors suspect one-step oscillatory dynamics explain the exception.
  • 3.4 IMAGE MODELING OF CIFAR10: 10 unrolling steps modestly reduced CIFAR10 reconstruction MSE beyond the significant improvement from one step, while zero-step models achieved the lowest error less than 1% of the time.Reconstructions became clearer with more unrolling, consistent with broader distribution coverage.
  • 3.4.2 PAIRWISE DISTANCES: With more unrolling, intersample-distance histograms increasingly resembled real-data histograms, providing evidence of reduced mode collapse.The zero-step model concentrated probability mass toward smaller ℓ2 intersample distances.

4 DISCUSSION

The method stabilizes GAN training and reduces mode collapse by defining the generator objective through unrolled discriminator optimization. Across several tasks, it rescued unstable training or reduced dropped regions of the data distribution, while introducing a computation–accuracy tradeoff and leaving extensions unexplored.

  • Unrolled discriminator optimization stabilizes GAN training and reduces mode collapse across several tasks.
  • The computational cost of each training step increases linearly with the number of unrolling steps.One unrolling step may suffice for some architectures, whereas more are needed for unstable recurrent models.
  • Unrolling trades a better approximation of the true generator loss against the computation required to estimate it.
  • The study considers only a small fraction of the design space, leaving recursive updates that unroll both generator and discriminator for future work.

A 2D GAUSSIAN TRAINING DETAILS

The 2D Gaussian experiment uses a compact fully connected generator and discriminator trained with alternating Adam updates. The data comprise eight Gaussian modes arranged around a circle, with specified initialization, activations, and objectives.

  • The dataset is a mixture of 8 Gaussians with standard deviation 0.02 and means equally spaced on a circle of radius 2.
  • The generator has two fully connected hidden layers of size 128 with ReLU activations and a linear projection to 2 dimensions.Weights use orthogonal initialization with scaling 0.8.
  • The discriminator scales inputs by 4, then uses one ReLU fully connected layer followed by a scalar logit layer.
  • The generator and discriminator use opposing logarithmic objectives with Adam learning rate 1e-4 and β1=0.5.The generator minimizes LG = log(D(x)) + log(1 −D(G(z))), while the discriminator minimizes the corresponding negative objective.
  • Training alternates generator and discriminator updates, with one step consisting of either G or D updating.

B.1 EFFECTS OF TIME DELAY / HISTORICAL AVERAGING

Historical averaging is tested through discriminator ensembles over recent or delayed training states, alongside analyses of unrolling gradients. Longer delays destabilize training, while backpropagation through unrolling helps when the discriminator lags behind the generator.

  • Effects of Time Delay / Historical Averaging: Historical averaging does not visibly increase stability on the mixture of Gaussians task.The comparison uses ensembles of immediately preceding discriminators across different training steps.
  • Effects of Time Delay / Historical Averaging: Longer time delays between discriminator replacements make the model progressively less stable.
  • Effects of Time Delay / Historical Averaging: Fixed older discriminators remove feedback and allow the generator to exploit their poorly performing regions, after which new discriminators overcompensate and the system diverges.
  • Effects of the Second Gradient: When the discriminator remains near its optimum, performance is nearly identical with and without the second gradient term.
  • Effects of the Second Gradient: When the discriminator lags behind the generator, backpropagating through unrolling aids convergence and can halve the required unrolling steps.This result was observed when using five generator steps for each discriminator update.

C RNN MNIST TRAINING DETAILS

The RNN MNIST experiment uses a 256-dimensional noise vector, an LSTM generator producing 28-step image sequences, and a convolutional discriminator. Training alternates Adam updates for 150k steps, with an unrolled-GAN configuration noted in the setup.

  • The MNIST dataset is scaled to [-1, 1).
  • The generator maps a 256-dimensional noise vector through a 256-unit ReLU layer into a 256-dimensional LSTM running 28 steps.The resulting activations are projected to 28 outputs with a tanh activation.
  • The discriminator uses three stride-2 convolutional layers with leaky rectifiers and batch normalization, followed by a scalar linear projection.
  • The experiment includes an unrolled-GAN configuration without the second gradient.
  • The generator and discriminator are trained with Adam at learning rate 1e-4 and β1=0.5 for 150k alternating update steps.

D CIFAR10/MNIST TRAINING DETAILS

The experiments use alternating GAN updates with specified convolutional architectures and learning rates. An unrolled setup with five generator steps per discriminator step improves stability, especially when the discriminator does not fully converge between updates.

  • Network architecture: All convolutions use 3x3 kernels with batch normalization; the discriminator uses leaky ReLUs with a 0.3 leak, while the generator uses standard ReLUs.
  • Training stability: Five generator steps per discriminator step require only 5 unrolling steps to converge, compared with 10 without the second gradient.Backpropagating through the unrolling process increases stability when the discriminator does not fully converge between generator updates.
  • Discriminator architecture: The discriminator architecture takes either data samples x or generated samples G as input and ends with a single fully connected output.Its convolutional layers have 8*X, 16*X, and 32*X outputs with stride 2.
  • Optimization: The generator and discriminator use alternating updates for 100k steps, with learning rates of 1e-4 and 2e-4, respectively.The generator minimizes LG = log(D(G(z))), while the discriminator minimizes LD = log(D(x))+log(1−D(G(z))).
  • Network architecture: The discriminator capacity parameter X is tested at 1/4 and 1/2 in these experiments.

F.1 DATASET

The dataset is constructed by coloring MNIST digits with Gaussian color variation and pixel noise, then clipping values to (-1, 1). Experiments compare model capacities and look-ahead configurations, including samples with different diversity levels.

  • Dataset construction: MNIST digits are scaled to [0,1], multiplied by Gaussian colors with mean 0 and standard deviation 0.5, then perturbed with noise of standard deviation 0.2.The resulting colored digits are clipped between (-1, 1).
  • Diversity comparison: The dataset samples are compared with outputs from a 1/4-size model using 0 look-ahead steps and a 1/1-size model using 10 look-ahead steps.The caption labels the former as having the worst diversity and the latter as having the most diversity.
  • Model capacity: Model capacity is controlled by X, with experiments using X values of 1/4, 1/2, and 1.X=1 uses the same architecture as the CIFAR10 experiments.
  • Generator architecture: The generator maps latent samples z ∼ N(0, I256) through a fully connected layer, reshape, transposed convolutions, and a final convolution producing three outputs.
  • Discriminator architecture: The discriminator processes data samples x or generated samples G through three stride-2 convolutions, flattening, and a single fully connected output.
  • Evaluation procedure: Each unrolling-step configuration is run five times with different random seeds, and comparisons are grouped by run index for space efficiency.
Loading 1611.02163v4…