PaperlayerView exact revision on arXiv

Source-linked AI summary

Reducing Noise in GAN Training with Variance Reduced Extragradient

Tatjana Chavdarova, Gauthier Gidel, François Fleuret, Simon Lacoste-Julien

arXiv:1904.08598v3stat.MLcs.LGmath.OC

TL;DR

Stochastic gradient noise can prevent standard GAN game optimizers from converging, even when their full-batch counterparts converge. The paper introduces SVRE, combining variance reduction with extragradient updates, and reports comparable MNIST convergence to batch extragradient with lower computational cost and more stable training on standard datasets.

  • Problem

    Stochastic noise can destabilize GAN game optimization, while larger batches improve GAN quality and standard stochastic methods can fail to converge.

  • Method

    SVRE combines stochastic variance reduction with extragradient optimization for games.

  • Results

    SVRE matches batch extragradient’s convergence speed on MNIST with lower computational cost and provides more stable GAN training on standard datasets.

  • Takeaways & Limitations

    SVRE improves convergence guarantees for a large class of games and can stabilize GAN training relative to stochastic baselines.

  • Takeaways & Limitations

    The theoretical analysis does not apply to games with non-convex objectives such as GANs.

Abstract

from arXiv · show

We study the effect of the stochastic gradient noise on the training of generative adversarial networks (GANs) and show that it can prevent the convergence of standard game optimization methods, while the batch version converges. We address this issue with a novel stochastic variance-reduced extragradient (SVRE) optimization algorithm, which for a large class of games improves upon the previous convergence rates proposed in the literature. We observe empirically that SVRE performs similarly to a batch method on MNIST while being computationally cheaper, and that SVRE yields more stable GAN training on standard datasets.

1 Introduction

GANs are two-player optimization games where stochastic noise can destabilize training, motivating SVRE, which combines variance reduction with extragradient updates and improves convergence guarantees for a broad class of games.

  • GANs formulate training as a two-player minimax game in which generator and discriminator pursue competing objectives.
  • Stochastic noise can break extragradient convergence in games, unlike its behavior in the corresponding full-gradient setting.
  • Larger GAN mini-batches have been associated with a 46% relative improvement in Inception Score on ImageNet, motivating noise-reduction methods.
  • SVRE combines variance reduction with extrapolation to reduce stochastic noise during game optimization.
  • SVRE improves convergence rates for a large class of games under the paper’s stated assumptions.
  • The paper evaluates SVRE on standard GAN datasets and reports improvements for state-of-the-art deep models late in optimization.

2 GANs as a Game and Noise in Games

GAN training is a differentiable two-player game whose adversarial dynamics differ from single-objective minimization. The paper shows that stochastic noise can make extragradient diverge even when full-batch extragradient converges.

  • GANs use a generator and discriminator with competing objectives, forming a zero-sum minimax game when their costs are negatives of each other.
  • Extragradient uses a lookahead point to update parameters and converges for convex-concave zero-sum games under standard conditions.
  • In games, noisy gradients can interact with adversarial oscillations and point away from the Nash equilibrium.
  • Stochastic extragradient diverges geometrically on a strongly monotone stochastic game, while full-batch extragradient converges to its Nash equilibrium.
  • The counterexample uses a simple bilinear game with norm regularization whose batch Nash equilibrium is (θ*, ϕ*) = (0, 0).
  • The divergence result assumes an unconstrained setting; bounded domains would restore the finite-variance assumption used by a standard SEG convergence result.

3 Reducing Noise in Games with Variance Reduced Extragradient

SVRE combines variance-reduced gradient estimates with extragradient updates to reduce stochastic noise in games, under assumptions that yield improved convergence rates. The method is empirically the only one converging on a motivating stochastic game, while AVG-SVRE converges sublinearly where AVG-AltSGD fails.

  • 3.1 Variance Reduced Gradient Methods: Variance reduction uses snapshots and full-batch gradients to construct lower-variance unbiased estimates, with epoch schemes generalized to extragradient games.The noise dataset in GANs is not finite, requiring practical handling beyond the standard finite-sum formulation.
  • 3.2 SVRE: Stochastic Variance Reduced Extragradient: SVRE combines SVRG gradient estimates with extragradient updates to reduce stochastic noise in game optimization.The method extends variance reduction to the extrapolation step used for games.
  • 3.2 SVRE: Stochastic Variance Reduced Extragradient: SVRE improves variance-reduction convergence rates for a large class of stochastic games and can improve over prior rates based on the problem's cocoercivity constant.When the operator is close to a convex gradient, the rate recovers the standard ln(1/ϵ)L/µ dependence.
  • 3.2 SVRE: Stochastic Variance Reduced Extragradient: The analysis assumes strong convexity, cocoercivity, regularity, and related operator conditions to establish convergence.These assumptions are expressed through strong monotonicity, regularity, and cocoercivity of the game operator.
  • 3.3 Motivating example: SVRE and its variants are the only methods observed to converge on the motivating stochastic game, whereas AVG-AltSGD and the other tested stochastic methods fail.The authors report no convergence guarantees for one variant and exclude batch extragradient because it is not stochastic.

4 GAN Experiments

The experiments evaluate SVRE for GAN training on MNIST and real-world datasets, using IS, FID, and gradient second-moment estimates. SVRE matches batch performance on MNIST, improves final shallow-architecture results, and is more stable than stochastic baselines with deep architectures.

  • MNIST: SVRE matches BatchE–A in MNIST convergence speed while requiring significantly less computation than the full-batch method.The comparison uses mini-batch gradient computations as an implementation-independent time surrogate.
  • MNIST: At the end of MNIST training, all methods reach similar performance, with IS above 8.5.
  • Optimization behavior: SVRE reduces the generator’s averaged second-moment estimate over iterations, consistent with reduced gradient noise.
  • Real-world datasets: SVRE improves final FID over SE–A on SVHN with shallow architectures, although SE–A can perform better during early iterations.
  • Real-world datasets: With deep architectures, SE–A training collapsed in 100% of experiments, whereas SVRE did not collapse but took longer to converge.

5 Related work

Prior work introduced variance reduction for monotone operators and extragradient-based GAN optimization, but important theoretical and empirical questions remained. Existing studies also reported mixed evidence about variance reduction in deep learning and batch-size effects.

  • Variance reduction: Variance-reduction methods for monotone operators were relatively limited, with prior approaches relying on assumptions such as co-coercivity.
  • Game optimization: Extragradient-based methods such as ExtraAdam and Optimistic-Adam provided principled alternatives for game optimization and GAN training.
  • Deep learning evidence: The effectiveness of variance reduction in deep learning remained unresolved, with prior reports including negative empirical results.

6 Discussion

The discussion presents SVRE as a response to stochastic divergence in games and summarizes its theoretical and empirical advantages. It also notes that GAN experiments fall outside the paper’s theoretical assumptions and that deeper architectures require more iterations.

  • Contributions and empirical scope: SVRE combines SVRG with extragradient and improves previous results for strongly convex games while converging on the stochastic bilinear counter-example.
  • Contributions and empirical scope: On GAN benchmarks, SVRE matches Batch-Extragradient speed on MNIST and is more stable than stochastic baselines across the reported datasets.
  • Contributions and empirical scope: SVRE required more iterations with deeper architectures, while its constant-step-size setup differed from baselines using Adam.
  • Assumptions: Theoretical convergence guarantees rely on assumptions such as Lipschitzness, strong convexity or concavity, and strong monotonicity.
  • Assumptions: For suitable operators, Lipschitzness and strong monotonicity imply cocoercivity, supporting the convergence analysis.

C.1 Proof of Theorem 1

The proof analyzes stochastic extragradient on a bilinear game by separating updates according to whether the two sampled mini-batches overlap. Disjoint samples are common and can cause divergence, while improving overlaps occur with low probability, yielding geometric divergence under a batch-size condition.

  • Proof setup: The proof represents sampled gradients with coordinate-selection matrices and expresses the extragradient rule using separate mini-batches I and J.
  • Stochastic update analysis: When mini-batches are disjoint, which is likely for small batches, the stochastic extragradient update moves away from the Nash equilibrium when 2ϵ ≥ η.
  • Stochastic update analysis: Improving updates require large overlap between the update and extrapolation mini-batches, an event with low probability for small batches.
  • Divergence result: If 2|I| ≤ n, the expected squared norm E[N_t] diverges geometrically for any η ≥ ϵ.

C.2 Proof of Theorem 2

The proof establishes convergence for q-memorization algorithms under strong monotonicity by combining extrapolation inequalities, unbiased gradient estimates, and variance-control lemmas.

  • Proof ingredients: Strong monotonicity, Lipschitzness, Young’s inequality, and co-coercivity bound the estimator and extrapolation errors needed to close the convergence proof.The resulting inequality contracts the distance to the solution after choosing a sufficiently small step size.
  • Update recursion: The extrapolation update yields a distance recursion that includes the progress term, extrapolation displacement, and gradient-estimate difference.This recursion follows by applying the projection inequality to both the extrapolation and update steps.
  • Gradient estimates: Unbiased gradient estimates may use the full operator, a component estimate, or the SVRG estimate within the general framework.The expectation of the sampled estimator equals the aggregate operator.
  • Variance reduction: Uniform memorization algorithms cover variance-reduced methods such as SAGA, SVRG, q-SAGA, and N-SAGA while supporting extrapolation steps.The framework updates randomly selected memory locations with equal probability and includes SVRG’s snapshot update as a special case.
  • Theorem 2: Under Assumption 1, the theorem gives a convergence guarantee for q-memorization iterates with suitably bounded step sizes.The proof uses constant step sizes and combines the extrapolation recursion with bounds on gradient-estimator and memorization errors.

D.1 Practical Aspect

The practical section adapts variance reduction and extragradient updates to GANs, addresses the latent-noise dataset choices, and introduces VRAd because standard Adam can become unsuitable when variance estimates vanish.

  • Noise dataset: For infinitely resampled latent noise, finite-sum SVRE has no convergence guarantee in the strongly convex case and produces biased gradient estimators.The paper considers resampling each epoch or fixing one finite-sum noise dataset as alternatives.
  • Noise dataset: The three latent-noise dataset alternatives showed no notable practical performance difference in the experiments.The alternatives were infinite resampling, per-epoch resampling, and a fixed finite-sum noise dataset.
  • Implementation: The practical SVRE-GAN algorithm alternates extrapolation and update steps, using mini-batches from data and noise datasets before returning both networks.Its pseudocode includes dataset snapshots, snapshot gradients, and separate learning rates for discriminator and generator.
  • Adaptive methods: Standard Adam may produce an excessively large effective step size when variance reduction drives its second-moment estimate toward zero.This motivates the variance-reduced Adam variant, VRAd, which maintains a reasonable step size in that regime.
  • SVRE-GAN: SVRE-GAN combines SVRG gradient estimates with extrapolation updates for the generator and discriminator using snapshot networks and stored gradients.The method computes variance-reduced discriminator and generator gradients from mini-batches and snapshot models.

F.2.2 Choice of architectures on real-world datasets

The real-world architecture experiments replicate a sensitive CIFAR-10 and SVHN setup, using multiple hyperparameter choices and limited computational resources that complicate fixed-setting comparisons.

  • Architectures: The experiments use ResNet architectures on CIFAR-10 and SVHN to replicate the referenced setup for deeper models.The architectures and hyperparameters are described separately for these experiments.
  • Experimental setup: The CIFAR-10 and SVHN setup is highly sensitive to hyperparameter choices, making optimization-method comparisons at a fixed setting difficult.The explored choices include generator and discriminator learning rates, β1, learning-rate decay, and update ratios.

F.2.3 Shallower SAGAN architectures

The shallow SAGAN experiments use self-attention architectures for SVHN, CIFAR-10, and ImageNet, with specified adversarial losses and reported MNIST comparisons across optimization methods.

  • Architecture choices: The SAGAN models use self-attention, spectral normalization on both networks, and different generator and discriminator learning rates.These architectural choices support long-range dependency modeling and follow established GAN design practices.
  • Training objective: The hinge adversarial non-saturating loss is used for the SAGAN experiments, with data and latent distributions defining discriminator expectations.The loss is specified explicitly for the discriminator.
  • Shallow SAGAN: Shallow SAGAN architectures are used for SVHN and CIFAR-10, with the generator and discriminator specified in separate table columns and a self-attention block included.Batch Normalization uses default PyTorch hyperparameters.
  • ImageNet architecture: The ImageNet shallow architecture adds a common generator block and both a common and self-attention discriminator block relative to the SVHN and CIFAR-10 design.These additions increase the number of parameters in the corresponding networks.
  • MNIST results: SVRE converges faster than SE–A on MNIST while achieving similar final performances across five runs.The reported averages are IS 8.62, 8.58, and 8.56 and FID 0.17, 0.15, and 0.18 for SE–A, SVRE, and SVRE–VRAd, respectively.

G.2 Results with shallow architectures

The shallow-architecture experiments compare stochastic, full-batch, and variance-reduced extragradient methods across MNIST, SVHN, and ImageNet. The reported evaluations use image-synthesis scores and second-moment estimates to assess performance and noise behavior.

  • G.2 Results with shallow architectures: The shallow-architecture evaluation covers MNIST, SVHN, and ImageNet, with results summarized using IS, FID, and second-moment estimates.ImageNet results compare SVRE with the SE–A baseline, while SVHN includes SME measurements.
  • G.2 Results with shallow architectures: Figure 5 compares stochastic, full-batch, and variance-reduced extragradient on MNIST, with BatchE–A plotted against an unscaled number-of-passes axis.The input space is 1×28×28.
  • G.2 Results with shallow architectures: The shallow-architecture results include FID and IS comparisons for ImageNet and SME trajectories for SVHN.The corresponding SVHN FID scores are reported with the SME experiments.

G.3 Results with deeper architectures

With deeper architectures, stochastic baselines are unstable and sensitive to training choices, whereas SVRE converges more consistently without later divergence. SVRE also produces smoother second-moment estimates and supports continued improvement from a stochastic-baseline checkpoint.

  • G.3 Results with deeper architectures: Deep architectures make GAN training more challenging, with the stochastic baseline becoming more unstable and SVRE converging more slowly than in shallow settings.The deep-architecture discussion reports instability for the baseline and slower SVRE convergence relative to shallow architectures.
  • G.3 Results with deeper architectures: SVRE always starts to converge and does not diverge later, while the baseline either fails to converge or diverges across the deep-architecture experiments.On SVHN, the baseline failed to converge or diverged in all tested experiments, whereas SVRE always converged.
  • G.3 Results with deeper architectures: WS–SVRE further reduced FID after continuing from the stochastic baseline’s best checkpoint, even though the baseline later diverged.The starting baseline checkpoint had FID 18.65.
  • G.3 Results with deeper architectures: SVRE produces more bounded, smoother SME trajectories, while baseline divergence correlates with large discriminator SME oscillations.The SME observations use deep architectures on CIFAR-10.
  • G.3 Results with deeper architectures: The reported deep-architecture advantages are more consistent convergence and fewer hyperparameters, including no need to tune β1, the update ratio, or a learning-rate schedule.The authors note that the comparison used constant-step-size SVRE versus Adam for the stochastic baseline and may therefore favor SVRE.
Loading 1904.08598v3…