PaperlayerView exact revision on arXiv

Source-linked AI summary

Variational f-divergence Minimization

Mingtian Zhang, Thomas Bird, Raza Habib, Tianlin Xu, David Barber

arXiv:1907.11891v2stat.MLcs.LG

TL;DR

Training latent-variable models with f-divergences beyond maximum likelihood is challenging because those divergences can be computationally intractable. The paper combines a variational upper bound with Spread Divergence to train a broader class of models. Its reported results contrast reverse KL’s sharper, less variable image samples with its tendency to focus on fewer modes.

  • Problem

    For complex models, f-divergences are generally computationally intractable, and training latent-variable models with divergences beyond maximum likelihood is comparatively unknown.

    Source: §1 · p. 1

  • Method

    The paper uses a variational upper bound with Spread Divergence to train latent generative models under a broader range of f-divergences.

    Source: §1 · p. 1, §2 · p. 3

  • Results

    For higher-dimensional image generation, reverse KL tends to prioritize high-fidelity samples around fewer modes than forward-KL VAE training, while both divergences can work on low-dimensional toy problems.

    Source: §5 · p. 8

  • Takeaways & Limitations

    Reverse KL can yield sharper images, but on higher-dimensional image generation it tends to focus on fewer modes than forward-KL training.

    Source: §3.2 · p. 6, §3.3 · p. 6, §5 · p. 8

Abstract

from arXiv · show

Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific f-divergence between the model and data distribution. In light of recent successes in training Generative Adversarial Networks, alternative non-likelihood training criteria have been proposed. Whilst not necessarily statistically efficient, these alternatives may better match user requirements such as sharp image generation. A general variational method for training probabilistic latent variable models using maximum likelihood is well established; however, how to train latent variable models using other f-divergences is comparatively unknown. We discuss a variational approach that, when combined with the recently introduced Spread Divergence, can be applied to train a large class of latent variable models using any f-divergence.

1. Introduction

The introduction motivates alternatives to maximum-likelihood training because divergence choice can change the fitted model, while direct f-divergence computation is generally intractable for complex models. It frames upper bounds as a route to optimizing divergences beyond the forward KL.

1. Introduction

  • Because the fitted optimum can depend on the divergence and direct f-divergence computation is generally intractable for complex models, the paper motivates an upper-bound approach.The introduction presents this as a way to apply f-divergences to training generative models.Source: §1 · p. 1

1.1. Maximum likelihood and forward KL

  • Minimizing the forward KL from the empirical distribution to the model is equivalent to maximizing the data likelihood.The paper identifies this as the standard divergence used in statistics and machine learning.Source: §1.1 · p. 1

1.2. Forward versus reverse KL

  • For a constrained Gaussian fit to separated modes, forward KL matches the mean and variance, whereas reverse KL fits a local mode.The figure contrasts the mixture in black with forward KL in red and reverse KL in blue.Source: §1.2 · p. 2, Figure 1 · p. 2

1.3. Optimizing f-divergences

  • Prior upper bounds cover forward KL, while a Fenchel-conjugate lower bound yields minimax training; the authors favor upper bounds for standard, stable optimization.They note that upper bounds on other f-divergences, including reverse KL, were unfamiliar and that general f-divergence upper bounds had not been used in machine learning.Source: §1.3 · p. 2

2. The f-divergence upper bound

The section develops a variational upper bound for training latent-variable models with f-divergences. Spread Divergence addresses support mismatch, while the later subsections extend the approach to deterministic-output models and gradient estimation.

2. The f-divergence upper bound

  • The auxiliary variational objective upper-bounds the marginal f-divergence, and jointly lowering it through model and bound parameters drives down the divergence.The bound is tight when p(z|x) is fully and unconstrainedly optimized with q(x,z) fixed.Source: §2 · pp. 2–3, Figure 2 · p. 3

2.1. Spread f-divergence

  • When the distributions have disjoint supports, Spread Divergence adds noise so the transformed distributions share support and the divergence is well defined.Without this step, the reverse-KL integral may be ill-defined when model and data supports differ.Source: §2 · p. 3, §2.1 · pp. 3–4

2.2. Training implicit generative models

  • Spread Divergence provides a consistent training objective for implicit models with deterministic outputs, avoiding the mismatch between noisy training assumptions and clean generated outputs.Training takes place on the noise-corrupted y-space, then the model on x-space is recovered by outputting the generation network’s mean.Source: §2.2 · p. 4

2.3. Gradient approximation

  • A minibatch estimate of the empirical density inside the logarithm is biased and harmed image-generation optimization, motivating a different gradient estimator.The proposed estimator samples minibatch indices from p(n|y) and is unbiased; additional variance-reduction tricks may introduce bias but worked well in practice.Source: §2.3 · pp. 4–5

3. Experiments

The experiments demonstrate the auxiliary f-divergence upper bound across toy and image-generation tasks, then compare it with the Fenchel-conjugate lower bound. They examine how divergence choice affects fitted samples and report broadly similar Gaussian fits, with some advantages for the upper-bound method.

3.1. Toy problem : Forward KL, Reverse KL and JS training

  • On the seven-mode toy problem, JS and reverse KL cover every mode and separate them in latent space; reverse KL succeeds when the decoder is sufficiently powerful.The toy data are seven two-dimensional Gaussians arranged in a circle and embedded in three dimensions.Source: §3.1 · p. 5

3.2. MNIST : forward and reverse KL training

  • Reverse-KL-trained MNIST samples are sharper than forward-KL samples shown for the same latent codes.The reverse-KL model was initialized with forward-KL training to avoid a sub-optimal solution such as mode collapse.Source: §3.2 · pp. 5–6, Figure 4 · p. 6

3.3. CelebA: forward and reverse KL training

  • On CelebA, reverse KL yields sharper images but less pose variability, consistent with its mode-seeking behavior.The samples follow VAE initialization and an additional epoch of forward- or reverse-KL training.Source: §3.3 · p. 6, Figure 5 · p. 7

3.4. Comparison to Fenchel conjugate lower bound

  • The upper-bound and f-GAN methods produce broadly similar Gaussian fits, with the upper bound slightly better in fit proximity and computational stability.The comparison uses a univariate Gaussian fitted to data from a two-Gaussian mixture.Source: §3.4 · p. 7

4. Related work

The paper extends auxiliary variational training beyond the forward-KL objective used for likelihood-based models. It distinguishes its joint-space upper bound from related approaches and notes that divergence choice can yield markedly different fits.

  • The auxiliary variational method is extended from minimizing joint KL to minimizing a broader class of f-divergences.The paper contrasts this objective with auxiliary-space methods that use joint spaces for learning latent representations.Source: §4 · pp. 7–8
  • Unlike VAE likelihood training, which corresponds to forward KL, the method trains the same class of generative and variational models with other f-divergence objectives.Both approaches use sampling to estimate a variational bound differentiated with the reparameterization trick.Source: §4 · p. 7
  • The paper reports that different divergences can produce very different model fits, contrasting with f-GAN’s reported largely similar results.It also notes examples of reverse-KL GAN objectives showing mode-seeking behavior.Source: §4 · p. 8
  • The paper separates three tractable f-divergence estimation approaches: marginal-space ratio estimation, the Fenchel-conjugate lower bound, and its variational joint upper bound.The related methods include ratio estimation by classification and the f-GAN lower-bound approach.Source: §4 · p. 8

5. Conclusion

The conclusion presents the upper-bound method as a modest change to VAE training that enables broader divergence choices and can yield different fitted models.

  • For higher-dimensional image generation, reverse KL tends to favor high-fidelity samples around fewer modes, while both forward and reverse KL can learn low-dimensional toy problems.The paper’s central contribution is applying more general f-divergences to probabilistic generative-model training.Source: §5 · p. 8

A. Properties of the Auxiliary Variational Method

For differentiable f, fully optimizing the auxiliary f-divergence over p(z|x) is equivalent to minimizing the f-divergence in x alone. Because that marginal divergence is assumed computationally intractable, practical training must use a tractable, suboptimal conditional family.

A. Properties of the Auxiliary Variational Method

  • For differentiable f, the optimal p(z|x) equals q(z|x), reducing the auxiliary objective to the marginal divergence Df(p(x)||q(x)).The property also extends to piecewise differentiable f.Source: A. Properties of the Auxiliary… · p. 10
  • Because Df(p(x)||q(x)) is assumed computationally intractable, practical training restricts p(z|x) to a tractable family and therefore uses a suboptimal conditional.Source: A. Properties of the Auxiliary… · p. 10

A.2. Relation to KL(q(x)||p(x))

The appendix relates auxiliary reverse-KL optimization to marginal KL minimization, examines independence and the ELBO, and describes a reverse-KL gradient approximation and its costs. It also specifies the toy target used in experiments.

A.2. Relation to KL(q(x)||p(x))

  • Fully optimizing p(z|x) makes the auxiliary reverse KL equivalent to minimizing KL(q(x)||p(x)); its optimum is p(z|x)=q(z|x).This follows from minimizing a positive combination of KL divergences.Source: §A.2 · p. 10
  • Under the independence assumption p(z|x)=p(z), the method is no better than choosing the single best q(x|z), and cannot enrich a simple Gaussian q(x).Source: §A.3 · p. 11, §A.4 · p. 11

A.5. Relation to the ELBO

  • Joint-space reverse KL is equivalent to using the ELBO to lower-bound log p(x) in marginal KL minimization.Source: §A.5 · p. 11

B. Reverse KL gradient approximation

  • The reverse-KL gradient estimator samples from p(n|y), whose probability is a softmax over scaled squared distances between y and data points; PCA reduces the cost of computing those distances.The PCA projection is learned once on the input data.Source: B. Reverse KL gradient approximation · p. 12
  • The normalizer cost scales with the number of data points and may be problematic for very large datasets, although PCA made it manageable in reported practice.The authors leave alternative minibatch methods for this setting to future work.Source: B. Reverse KL gradient approximation · pp. 12–13
  • The two variance-reduction tricks can bias the estimator, but the authors report that they work well in practice.Source: B. Reverse KL gradient approximation · p. 13

C. Target distribution of the toy problem

D. Network Architecture

The encoder and decoder use fully convolutional architectures with specified filter, stride, and layer conventions.

D. Network Architecture

  • Both networks use 5x5 convolutional filters and horizontal and vertical stride 2, except the final deconvolution layer, which uses stride 1.Convk and DeConvk denote convolution and deconvolution with k filters; BN, ReLU, and FCk denote batch normalization, rectified linear units, and a fully connected mapping to Rk.Source: D. Network Architecture · p. 13

E. f-GAN comparison

The section plots the target distribution and compares training-bound behavior. The upper bound generally converges faster, with less noise and a consistently decreasing objective, while the variational lower bound fluctuates.

E. f-GAN comparison

  • The target distribution for the toy problem is shown in Figure 6.It is the seven-Gaussian mixture described for the toy dataset.Source: Figure 6 · p. 13
  • The upper bound generally converges faster and with less noise than the variational lower bound, and decreases consistently while the lower bound fluctuates.Source: E. f-GAN comparison · p. 13

F. Class Probability Estimation

Class probability estimation uses a classifier distinguishing data from model samples to estimate a density ratio, which is then used to form an f-divergence generator loss. The section also shows Gaussian-mixture fitting runs using lower and upper bounds.

F. Class Probability Estimation

  • Class probability estimation is briefly shown as one ratio-estimation technique; ratio matching is discussed in Mohamed & Lakshminarayanan (2016) and left to that work.Source: F. Class Probability Estimation · p. 13
  • A classifier distinguishes training data from model-generated samples to estimate their density ratio, with equal sample counts assumed when applying Bayes’ rule.The discriminator output is set to p(y=1|x).Source: F. Class Probability Estimation · p. 14
  • The estimated ratio is Dϕ(x)/(1−Dϕ(x)), and the corresponding generator loss is an expectation over generated samples of f applied to that ratio.The expectation is over z sampled from q(z).Source: F. Class Probability Estimation · p. 14
  • Figure 8 shows univariate-Gaussian fitting runs for the two-Gaussian mixture, with the lower bound on the left and the upper bound on the right.Figure 7 specifies the mixture as 0.3N1 + 0.7N2, where N1 has mean 1 and standard deviation 0.1, and N2 has mean 2 and standard deviation 0.5.Source: Figure 7 · p. 14, Figure 8 · p. 15
Loading 1907.11891v2…