PaperlayerView exact revision on arXiv

Source-linked AI summary

f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization

Sebastian Nowozin, Botond Cseke, Ryota Tomioka

arXiv:1606.00709v1stat.MLcs.LGstat.ME

TL;DR

Generative neural samplers provide efficient feedforward sampling but do not support likelihood computation or marginalization. The paper develops variational divergence minimization, showing that GAN training is a special case and extending training to arbitrary f-divergences. Experiments examine how divergence choices affect training and generated-model quality.

  • Problem

    Generative neural samplers efficiently represent and sample complex distributions, but cannot compute likelihoods or perform marginalization.

  • Method

    The paper extends variational f-divergence estimation to model estimation, defining variational divergence minimization as a general framework for training generative neural samplers.

  • Results

    GAN training is a special case of the generalized framework, which derives training objectives for arbitrary f-divergences.

  • Takeaways & Limitations

    Different f-divergences can be used to train generative neural samplers, with experiments comparing their training complexity and generated-model quality.

  • Takeaways & Limitations

    Purely generative neural samplers cannot be conditioned on observed data and therefore cannot provide inferences after training.

Abstract

from arXiv · show

Generative neural samplers are probabilistic models that implement sampling using feedforward neural networks: they take a random input vector and produce a sample from a probability distribution defined by the network weights. These models are expressive and allow efficient computation of samples and derivatives, but cannot be used for computing likelihoods or for marginalization. The generative-adversarial training method allows to train such models through the use of an auxiliary discriminative neural network. We show that the generative-adversarial approach is a special case of an existing more general variational divergence estimation approach. We show that any f-divergence can be used for training generative neural samplers. We discuss the benefits of various choices of divergence functions on training complexity and the quality of the obtained generative models.

1 Introduction

Generative neural samplers use feedforward networks to generate samples efficiently, while GAN training introduces a discriminator to estimate and fit distributions. This paper generalizes that approach from Jensen-Shannon divergence to arbitrary f-divergences.

  • Generative neural samplers: Generative neural samplers represent probability distributions with feedforward networks that transform random inputs into outputs.They include GANs and variational-autoencoder decoder models.
  • Generative neural samplers: These models support efficient sampling, but the described operations also include estimation and point-wise likelihood evaluation.A single forward pass produces one exact sample in the GAN formulation.
  • GAN training: GAN training introduces a discriminator neural network and approximately minimizes the symmetric Jensen-Shannon divergence.Because this divergence is proper, approximation can be good with sufficient samples and a sufficiently rich model class.
  • Paper contributions: The paper extends variational divergence estimation to model estimation, showing that GAN training is a special case of variational divergence minimization.The framework generalizes GAN objectives to arbitrary f-divergences.
  • Paper contributions: The authors derive GAN training objectives for all f-divergences and provide examples including Kullback-Leibler and Pearson divergences.They also simplify the saddle-point optimization procedure and provide experimental insight into divergence choices for natural images.

2 Method

The method extends variational f-divergence estimation into model estimation, yielding f-GAN objectives for arbitrary f-divergences. It parameterizes both a generator and variational function, constrains the latter through divergence-specific activations, and evaluates the framework experimentally.

  • f-divergences: Any convex, lower-semicontinuous generator f with f(1) = 0 defines an f-divergence, with common divergences recovered by different choices of f.The paper lists divergences and their generator functions, including the Jensen-Shannon relation to GANs.
  • Variational Divergence Minimization: VDM extends variational f-divergence estimation from fixed-model divergence estimation to learning generative-model parameters.The paper identifies generative-adversarial training as a special case of this variational divergence minimization framework.
  • Variational Estimation: The variational lower bound arises by applying the Fenchel-conjugate representation of f and restricting optimization to a function class T.Its looseness reflects both Jensen’s inequality and the possibility that T contains only a subset of all functions.
  • Variational Estimation: The bound is tight under mild conditions when the variational function satisfies T*(x) = f′(p(x)/q(x)).The optimal form guides the choice of f and the design of the variational-function class.
  • Model Estimation: f-GAN learns Qθ by minimizing over generator parameters θ and maximizing over variational parameters ω using samples from P and Qθ.The generator maps random inputs to samples, while Tω maps samples to scalar variational values; expectations are approximated with minibatches.
  • Representation for the Variational Function: The representation Tω(x) = gf(Vω(x)) uses an unconstrained scalar Vω and an f-specific output activation gf mapping into dom f*.Monotone activations preserve the interpretation that larger Vω(x) indicates greater belief that x comes from P; robust direct implementation of f*(gf(·)) is recommended.
  • GAN as a Special Case: The original GAN objective is recovered as a special case when the discriminator uses Dω(x) = 1/(1 + e^-Vω(x)) and gf(v) = -log(1 + e^-v).Thus the sigmoid-discriminator GAN corresponds to one particular f-divergence variational objective.
  • Example: Univariate Mixture of Gaussians: In the Gaussian-mixture experiment, learned models perform best on the divergence used for their training, while misspecification makes the divergence choice strongly influence the learned model.The experiment compares learned parameters and objectives with exact best fits, and the reported objectives satisfy Df(P||Qθ*) ≥ F(ω̂, θ̂).

3 Algorithms for Variational Divergence Minimization (VDM)

The paper replaces alternating optimization with a simpler single-step gradient method for VDM and analyzes its convergence under local saddle-point assumptions. Practical extensions include generator updates that preserve stationary points and divergence-specific monitoring statistics.

  • Single-Step Gradient Method: The proposed single-step method removes the inner loop and computes generator and variational-function gradients in one back-propagation.
  • Single-Step Gradient Method: The algorithm samples batches from P and Qθt, then updates ω by ascent and θ by descent on F.
  • Analysis: Under local strong convexity in θ, strong concavity in ω, and sufficient smoothness, the algorithm geometrically converges to a saddle point.
  • Practical Considerations: The generator update maximizing Ex∼Qθ[log Dω(x)] preserves the stationary point while providing stronger practical gradients than the saturating alternative.
  • Practical Considerations: For monotone variational outputs, real and fake statistics extend to any f-divergence by shifting the classification threshold to f′(1).

4 Experiments

Experiments evaluate f-divergence-trained samplers on MNIST and LSUN using KDE-based likelihood assessment and generated samples. On LSUN, GAN, KL, and squared Hellinger produce equally realistic samples, while MNIST likelihood estimates remain inconclusive because of KDE limitations.

  • MNIST Digits: On MNIST, models use 60,000 28-by-28 training images, divergence-specific output activations, Adam, batchsize 4096, and one hour of training.
  • MNIST Digits: MNIST evaluation estimates mean test log-likelihood from a KDE fitted to 16k generated images and evaluated on 10k test samples.
  • MNIST Digits: KDE likelihood estimates vary by up to 50 nats across repetitions, making the MNIST results not entirely conclusive.
  • LSUN Natural Images: The LSUN classroom experiment trains the same 96-by-96 image model with GAN, KL, and squared Hellinger objectives to compare divergence behavior.
  • LSUN Natural Images: All three LSUN divergences produce equally realistic samples, with learned-distribution differences arising only when the generator lacks sufficient capacity.

5 Related Work

Related work spans tractable-likelihood neural models, generative samplers, adversarial and kernel objectives, and alternative divergence formulations. The paper positions f-GAN as a computationally inexpensive learning objective that generalizes GAN training across f-divergences.

  • Mixture density networks, NADE/RNADE, diffusion models, and VAEs provide alternative neural generative modeling approaches with different likelihood, sampling, or inference properties.
  • NCE discriminates data from fixed artificial noise and can be viewed as a GAN special case with a constrained discriminator and fixed generator.
  • Earlier generative neural samplers used importance sampling or expectation maximization, whereas GAN and f-GAN use effective, computationally inexpensive learning objectives.
  • MMD avoids an explicitly represented variational function but requires choosing a kernel and had reported results slightly inferior to GAN.
  • Alternative Jensen-Shannon objectives interpolate between behaviors similar to KL and reverse KL as π approaches 0 or 1.

6 Discussion

Purely generative neural samplers represent complex distributions without factorization assumptions but cannot condition on observed data after training. The paper argues that its methods extend to conditional discriminative models by adding inputs to both networks.

  • Generative neural samplers represent complex distributions without limiting factorization assumptions.
  • After training, purely generative samplers cannot be conditioned on observed data and therefore cannot provide inferences.
  • The presented methods extend to conditional GAN-style models by providing additional inputs to both the generator and variational function.

Supplementary Materials

The supplementary materials extend the paper’s catalog of f-divergences, generator functions, convex conjugates, and related variational functions. They also provide visualizations and additional discussion supporting the main framework.

  • Supplementary Materials: The supplement lists an extended set of f-divergences with their generator functions and corresponding convex conjugates.The conjugate functions, domains, and generator-network output activations are organized across supplementary tables.
  • Supplementary Materials: Figure 4 plots generator functions and their corresponding convex conjugate functions for several f-divergences.
  • Supplementary Materials: Table 5 pairs each listed f-divergence with its generator function and optimal variational function.

C Proof of Theorem 1

This section states assumptions and proves convergence properties for the single-step variational optimization procedure, while reviewing how related GAN algorithms fit the objective framework. It also contrasts coherent saddle-point optimization with heuristic update rules.

  • Proof of Theorem 1: The analysis assumes F is strongly convex in θ and strongly concave in ω, with a locally valid saddle point and sufficient smoothness.These assumptions define the valid saddle-point solutions and support the subsequent convergence theorem.
  • Variational objective: The variational objective trains Qθ by minimizing over θ and maximizing over ω the difference between data expectations of Tω and model expectations of f ∗(Tω).The variational function’s output is mapped into the domain of the convex conjugate through a differentiable activation.
  • Proof of Theorem 1: Theorem 2 shows that, with η = δ/L, the squared norm of ∇F(π) decreases geometrically near the saddle point.The result follows from the smoothness condition and the specified single-step update.
  • Proof of Theorem 1: Algorithm 1 decreases J by an amount proportional to the squared norm of ∇F(π).
  • Related optimization algorithms: Some reviewed algorithms use heuristic maximization and minimization objectives that do not form a coherent joint saddle-point objective.
  • Related optimization algorithms: GAN-2 replaces the generator’s original log(1 − Dω(x)) objective with −log Dω(x) to avoid weak gradients early in training while retaining the same fixed point.

Remarks on the Weighted Jensen-Shannon Divergence in [16]

The weighted Jensen–Shannon objective behaves like KL or reverse KL near extreme weighting values, but this connection is only approximate. Exact KL or reverse-KL behavior requires the corresponding variational objectives.

  • Weighted Jensen-Shannon divergence: The GAN/variational objective uses the correspondence Tω(x) = log Dω(x), linking the discriminator to the variational function.
  • Weighted Jensen-Shannon divergence: The alternative Jensen–Shannon objective vanishes when π ∈ {0, 1}, while values near 0 and 1 can resemble KL and reverse KL objectives, respectively.
  • Weighted Jensen-Shannon divergence: GAN-3 and the weighted Jensen–Shannon optimization provide only approximate KL or reverse-KL behavior; exact behavior requires the corresponding variational objectives.

E Details of the Univariate Example

The univariate experiment fits a Gaussian generative model to a Gaussian mixture using several f-divergences and compares learned variational optimization with exact divergence fitting. The learned Gaussian approximations closely match the exact fits, while variational-function discrepancies mainly occur in the tails.

  • Setup: The experiment approximates a mixture of Gaussians by learning a Gaussian generated from Gθ(z) = µ + σz.The model receives z ∼ N(0, 1), with µ and σ as the parameters to learn.
  • Setup: The variational function is a neural network with two 64-unit hidden layers and tanh activations.The network maps the scalar input through Linear(1,64), Tanh, Linear(64,64), Tanh, and Linear(64,1).
  • Optimization: Training uses batches of 1024 for p(x) and p(z), step-size 0.01 for both parameter sets, and comparison against numerically optimized exact divergence fits.
  • Results: KL fits the mixture’s mean and variance, KL-rev is more mode-seeking, and Jensen–Shannon lies between these behaviors.
  • Results: The neural-network Gaussian approximations are close to the corresponding exact divergence fits.
  • Results: Variational functions are usually close to their optimal forms in the relevant region, while tanh capacity and sparse tail data cause tail discrepancies without significantly changing learned parameters.

F Details of the Experiments

The experiments used specified software and hardware, neural architectures for generators and discriminators, and fixed initialization and training settings. Additional details cover variational autoencoders and a Gaussian-mixture experiment.

  • Software and hardware: Experiments ran in Chainer 1.8.1 with CUDA 7.5, CuDNN v5, and an NVIDIA GTX TITAN X.
  • Neural architectures: The generator maps a 100-dimensional input through batch-normalized ReLU layers to 784 sigmoid outputs.Its layer widths are 100→1200→1200→784.
  • Initialization: Generator weights used scale 0.05, whereas discriminator weights used scale 0.005, one order of magnitude smaller than in the cited prior setup.
  • Neural architectures: The discriminator maps 784-dimensional inputs through two ELU layers of width 240 to one output.The architecture is 784→240→240→1, and ELU denotes the exponential linear unit.
  • Baselines and experiments: The variational autoencoder baseline used Chainer’s example implementation, trained for 100 epochs with 20 latent dimensions.
  • Baselines and experiments: The convolutional generator maps 100-dimensional inputs to 3-channel outputs through a 6×6×512 reshape and four stride-two deconvolutions.All deconvolutions use kernel size four; the Gaussian-mixture plots use the specified two-component mixture parameters.
Loading 1606.00709v1…