Source-linked AI summary
Comparison of Maximum Likelihood and GAN-based training of Real NVPs
Ivo Danihelka, Balaji Lakshminarayanan, Benigno Uria, Daan Wierstra, Peter Dayan
TL;DR
The paper compares maximum-likelihood and WGAN training of the same generator because likelihood and sample quality provide different views of learned generative models. It finds differences in coherence and likelihood, observes overfitting under both objectives, and uses an independent critic’s Wasserstein estimates for model comparison and overfitting detection.
Problem
The paper examines how GAN-trained generators compare with likelihood-based methods, since likelihood is not a measure of human perception.
Source: §1 · p. 1Method
Results
WGAN generators produce more globally coherent samples, but minimizing approximate Wasserstein distance does not minimize negative log-probability density, and both training objectives show overfitting.
Source: §1 · pp. 1–2Takeaways & Limitations
An independent critic’s approximate Wasserstein distance can compare generators and help detect overfitting against validation data.
Source: §3.5 · p. 6, §5 · p. 8The empirical training distribution may differ from the test distribution, and generated samples may be more similar to training data than to test data.
Source: §3.5 · p. 6
Abstract
from arXiv · showhide
We train a generator by maximum likelihood and we also train the same generator architecture by Wasserstein GAN. We then compare the generated samples, exact log-probability densities and approximate Wasserstein distances. We show that an independent critic trained to approximate Wasserstein distance between the validation set and the generator distribution helps detect overfitting. Finally, we use ideas from the one-shot learning literature to develop a novel fast learning critic.
1. Introduction
The paper compares maximum-likelihood and WGAN training of the same tractable real-NVP generator, examining generated samples, exact log-probability densities, and approximate Wasserstein distances. It reports differences in sample coherence and likelihood, overfitting under both objectives, and uses critic-based distances to compare models and evaluate a fast-learning critic.
1. Introduction
- The comparison uses the same tractable real-NVP architecture for both training objectives, allowing exact log-probability densities and comparison of learned-generator properties.Source: §1 · p. 1
- The independent critic’s approximate Wasserstein distance supports model comparison and is used to evaluate a novel fast-learning critic.Source: §1 · pp. 1–2
- WGAN-trained generators produce more globally coherent samples, including with a relatively shallow generator.The paper also reports that WGAN worked better than Jensen-Shannon divergence or −log D(x) training.Source: §1 · p. 1
- Minimizing approximate Wasserstein distance did not minimize negative log-probability density; those densities became worse than a uniform distribution.Source: §1 · p. 2
- An approximate Wasserstein distance correctly ranked generators trained by maximum likelihood.Source: §1 · p. 2
- The approximate Wasserstein distance from training data to the generator became smaller than from test data, an overfitting pattern seen with both maximum likelihood and WGAN.Source: §1 · p. 2
2. Setup
The setup pairs real NVP generators, whose likelihoods can be computed exactly, with WGAN training and a critic that approximates Wasserstein distance. It evaluates the models on MNIST and CelebA after adding uniform pixel noise and scaling intensities.
2.1. Generator
- Real NVP transforms standard-normal latent noise through invertible mappings to generate images, enabling exact log-probability computation using the transformation Jacobian.The study uses NVP1, NVP2, and NVP3 with 7, 13, and 19 transformations, respectively; each transformation has four residual blocks.Source: §2.1 · p. 2
2.2. Critic
- The WGAN critic approximates Wasserstein distance from real and generated samples, and the generator is trained to minimize that approximation.The critic is a DCGAN-based network with clipped weights; its training schedule uses more updates early and periodically during generator training.Source: §2.2 · p. 3
2.3. Datasets
- Experiments use 28 × 28 MNIST images and 32 × 32 CelebA images, with horizontal flips augmenting CelebA training examples.CelebA contains over 200,000 celebrity images.Source: §2.3 · p. 3
2.4. Data Preprocessing
- Discrete pixel intensities are made continuous by adding uniform noise, then divided by 256 before network input.The resulting negative log-probability density in bits/dim is an upper bound on compression loss.Source: §2.4 · pp. 3–4
3. Results
The comparison of maximum-likelihood and WGAN-trained real NVPs finds more globally coherent WGAN samples, but their negative log-probability densities worsen. Latent-variable and independent-critic analyses further reveal differences from validation data and training-set overfitting.
3.1. Generated Samples
- WGAN samples appear more globally coherent, especially for shallow NVP1, whose maximum-likelihood samples contain only locally coherent patches.The paper compares generators with the same architecture and parameter count.Source: §3.1 · p. 4
3.2. Log-probability Density
- WGAN’s negative log-probability densities worsen on CelebA, though training and validation losses overlap; all 19,867 validation examples have low log-probabilities, not just one outlier.Generated samples average -1.12 bits/dim, while MLE-trained NVP3 assigns 4.27 bits/dim to validation examples and 4.35 bits/dim to its generated samples.Source: §3.2 · pp. 4–5, Figure 5 · p. 4, Figure 6 · p. 4
- WGAN also yields high negative log-probability densities on MNIST; its deep generator learns a low-dimensional manifold, and near-infinite densities also occur with Jensen-Shannon GAN training.The authors note that annealed-importance-sampling analysis assuming a fixed-variance Gaussian observation model would not detect the manifold issue.Source: §3.2 · p. 5
3.3. Distribution of Latent Variables
- For validation examples encoded by NVP3, WGAN’s inferred latent standard deviations are 3.95 and 8.96, versus 1.06 and 1.03 under MLE, so prior draws differ from validation encodings.The MLE standard deviations are σ1 = 1.06 and σ2 = 1.03; WGAN’s are σ1 = 3.95 and σ2 = 8.96.Source: §3.3 · p. 5, Figure 7 · p. 5
3.4. Partial Reconstructions
- MLE partial reconstructions resemble generated samples, whereas WGAN partial reconstructions do not, indicating that validation examples and WGAN samples have different distributions.Figure 8 shows reconstructions after resampling either the first or second half of the latent vector.Source: §3.4 · pp. 5–6, Figure 8 · p. 5
3.5. Wasserstein Distance
- The independent critic finds training-set distance much smaller than test-set distance, indicating overfitting; validation distance also rises as generators train, so sample inspection can mislead.For maximum-likelihood generators, the deepest NVP3 has the smallest approximate validation distance; for WGAN, NVP1 and NVP3 have roughly the same validation distance.Source: §3.5 · pp. 6–7, Figure 9 · p. 6
4. Fast Learning Critic
The fast learning critic conditions on extra generator samples to represent the changing generator distribution. In experiments with infrequent critic updates, the generator without this critic failed to produce samples similar to the data.
4. Fast Learning Critic
- The approach addresses the need to retrain a good critic as the generator changes by supplying information about its current distribution.The data distribution is stationary, while the generator distribution changes quickly.Source: §4 · p. 7
4.1. Architecture
- The critic conditions on extra generator samples, which a network converts into a learned embedding of the generator distribution used to bias the critic.The embedding network uses gated activations, and its features are averaged across the extra-sample batch.Source: §4 · p. 7, Figure 11 · p. 7, §4.1 · p. 8
4.2. Fast Learning Results
- Without the fast learning critic, the generator failed to produce samples similar to data examples, even in an experiment designed to demonstrate benefits under infrequent critic updates.The critic was updated only twice per generator step for most of the experiment, with more frequent updates at the start and periodically thereafter.Source: §4.2 · p. 8, Figure 12 · p. 8
5. Discussion
The experiments use exact log-probabilities and an independent critic to evaluate generators. The discussion distinguishes when each measure is useful and notes that real NVPs can optimize both objectives.
5. Discussion
- The independent critic’s approximate Wasserstein distance supports comparisons across generator and critic architectures, with the validation set as the reference when Wasserstein distance matters.The critic requires only samples from two distributions.Source: §5 · p. 8
- Log-probability is less useful for comparing generators that produce only a subset of the data, but remains relevant to lossless compression.Real NVPs can jointly optimize log-probability and Wasserstein objectives; combined training produces more globally coherent samples than MLE alone.Source: §5 · p. 8, Figure 15 · p. 9
- The experiments also use real NVPs for an additional Adversarial Variational Bayes evaluation in the appendix.Source: §5 · p. 8
A. Log-probability Density Ratio Evaluation
The section evaluates how GAN-based discrimination estimates KL divergence in Adversarial Variational Bayes. It reports underestimation for large divergences and suggests autoregressive discriminators as a possible way to divide them into smaller terms.
A. Log-probability Density Ratio Evaluation
- Adversarial Variational Bayes underestimates the KL divergence, and its discriminator does not represent large KL divergences well.The comparison uses an unbiased estimate log q(z|x) − log p(z) and its GAN-based approximation.Source: A. Log-probability Density Ratio… · p. 9
- Autoregressive discriminators are proposed as a possible way to decompose a large KL divergence into multiple smaller terms.Source: A. Log-probability Density Ratio… · p. 9
- The prior p(z_i) is an independent Gaussian prior.Source: A. Log-probability Density Ratio… · p. 9
- For the combined MLE+WGAN NVP1, the negative log-probability density is not much worse than for NVP1 trained by MLE alone.Source: Figure 16 · p. 9
- The WGAN generator for CelebA 32 × 32 has a low-rank Jacobian, consistent with its distribution lying on a low-dimensional manifold.Source: Figure 17 · p. 9