Source-linked AI summary
Learning in Implicit Generative Models
Shakir Mohamed, Balaji Lakshminarayanan
TL;DR
Implicit generative models make likelihood-based inference difficult because their densities are intractable, creating a need for likelihood-free learning. The paper frames GANs and related methods through hypothesis testing, density-ratio estimation, divergence minimisation, and moment matching, and synthesizes their relationships and limitations.
Problem
Implicit models can generate samples without tractable likelihoods, leaving standard inference and parameter-learning tools unavailable.
Method
The paper develops likelihood-free learning by comparing samples from data and generative models, using hypothesis testing and density-ratio estimation to organize GANs and related objectives.
Results
The paper shows that GAN objectives arise from classifier-based hypothesis testing and relate to Jensen-Shannon divergence, while alternative approaches expose distinct computational and gradient trade-offs.
Takeaways & Limitations
A comparison-based perspective unifies GANs, approximate Bayesian computation, noise-contrastive estimation, and density-ratio estimation as indirect methods for learning generative models.
Takeaways & Limitations
Evaluation remains difficult because likelihoods are generally unavailable, high-dimensional likelihood estimates are unreliable, and visual inspection may miss mode collapse or memorisation.
Abstract
from arXiv · showhide
Generative adversarial networks (GANs) provide an algorithmic framework for constructing generative models with several appealing properties: they do not require a likelihood function to be specified, only a generating procedure; they provide samples that are sharp and compelling; and they allow us to harness our knowledge of building highly accurate neural network classifiers. Here, we develop our understanding of GANs with the aim of forming a rich view of this growing area of machine learning---to build connections to the diverse set of statistical thinking on this topic, of which much can be gained by a mutual exchange of ideas. We frame GANs within the wider landscape of algorithms for learning in implicit generative models--models that only specify a stochastic procedure with which to generate data--and relate these ideas to modelling problems in related fields, such as econometrics and approximate Bayesian computation. We develop likelihood-free inference methods and highlight hypothesis testing as a principle for learning in implicit generative models, using which we are able to derive the objective function used by GANs, and many other related objectives. The testing viewpoint directs our focus to the general problem of density ratio estimation. There are four approaches for density ratio estimation, one of which is a solution using classifiers to distinguish real from generated data. Other approaches such as divergence minimisation and moment matching have also been explored in the GAN literature, and we synthesise these views to form an understanding in terms of the relationships between them and the wider literature, highlighting avenues for future exploration and cross-pollination.
1. Implicit Generative Models
Implicit generative models transform latent randomness through a parameterized deterministic function to generate data, without requiring an explicit likelihood specification. Their flexible deep-network forms make the resulting density intractable, motivating likelihood-free learning methods such as GANs.
- Prescribed models specify an explicit parametric distribution and log-likelihood, whereas implicit models specify a stochastic procedure for generating observations.
- An implicit model transforms a latent variable z through a deterministic function Gθ mapping R^m to R^d.The latent variable supplies the external randomness, while Gθ is parameterized by θ.
- The transformed density is defined by differentiating the cumulative probability of the set of latent inputs mapped below x.This recovers familiar change-of-variables rules when G is invertible or otherwise tractable.
- For nonlinear deep-network generators with d > m, the density calculation is generally intractable because integration regions, integrals, and high-dimensional derivatives are difficult to compute.This prevents direct likelihood-based learning for the intended flexible implicit models.
- GANs and classifier ABC address this problem by learning from discrimination between real and generated data, connecting implicit-model learning to hypothesis and two-sample testing.The paper presents these approaches as likelihood-free frameworks for generator networks and differentiable simulator models.
2. Hypothesis Testing and Density Ratios
Likelihood-free learning in implicit models replaces unavailable marginal-likelihood calculations with comparisons between real and generated samples. The paper unifies classifier-based, divergence, ratio, and moment-matching approaches through density-ratio estimation and alternating optimisation.
- Likelihood-free inference: Density estimation-by-comparison tests whether real and model distributions are equal, then uses the comparison to learn model parameters.Comparisons can use density differences or density ratios, motivated by hypothesis testing and likelihood-ratio ideas.
- Likelihood-free inference: Implicit models make direct marginal comparison impossible, motivating four density-ratio approaches: class-probability estimation, divergence minimisation, ratio matching, and moment matching.These approaches provide alternative pathways for learning from samples without computing individual marginals.
- Class probability estimation: Classifier-based class-probability estimation computes the density ratio by distinguishing observed data from model-generated data.The discriminator estimates p(y = 1|x), which maps to the density ratio through D = r/(r + 1) and r = D/(1 − D).
- Class probability estimation: GAN objectives arise from proper scoring rules for discrimination, with alternating optimisation of comparison parameters and generative-model parameters.The resulting expectations are evaluated by Monte Carlo samples from the data and latent-variable generative processes.
- Class probability estimation: Proper scoring rules have the correct global optimum qθ = p∗, but non-convex optimisation provides no convergence guarantees.For Bernoulli loss, the objective is related to Jensen-Shannon divergence, and changing the class prior yields a generalised Jensen-Shannon divergence.
- Divergence and moment matching: Bregman-divergence formulations connect ratio estimation with f-divergences, while moment matching can be represented as an integral probability metric.The synthesis relates class-probability estimation, divergence minimisation, ratio matching, and moment matching; f-divergences and integral probability metrics intersect only at total variation distance.
3. Choice of Loss Functions
Implicit generative models require both distribution comparison and generator estimation, while loss-function choice is complicated by evaluation and optimization challenges.
- Learning implicit generative models separates comparison, which estimates a density ratio or difference, from estimation, which learns generator parameters.
- The ratio loss and generator loss need not be coupled, allowing different f-divergences to be used for each objective.
- Evaluation: Evaluating implicit models is difficult because marginal likelihoods are generally unavailable, kernel estimates are unreliable in high dimensions, and visual inspection can miss mode collapse or memorisation.
- Training considerations: Meaningful f-divergences require overlapping supports, so instance noise is commonly added; gradient-based training must also avoid vanishing generator gradients.
- Training considerations: Wasserstein GANs are promising because they address vanishing gradients efficiently, whereas moment matching avoids that issue but is more computationally expensive and batch-size demanding.
4. Discussion
The paper frames implicit-model learning through hypothesis testing and density comparisons, connecting GANs, approximate Bayesian computation, and related likelihood-free methods. It also identifies open challenges in Bayesian inference, perceptual evaluation, non-differentiable models, and scaling to high-dimensional data.
- Likelihood-free inference: Hypothesis-testing-based methods learn generative-model parameters by comparing populations from the model and observed data without computing data probabilities or latent-variable posteriors.This perspective relates GANs, approximate Bayesian computation, noise-contrastive estimation, and density-ratio estimation.
- Model choice and inference: The choice between implicit and prescribed models determines which learning and inference principles are available, even when both face intractable quantities.Likelihood specifications provide additional knowledge that supports methods such as noise-contrastive estimation and variational lower bounds.
- Bayesian inference: Approximate Bayesian computation targets posterior uncertainty over implicit-model parameters, while classifier-based ABC complements the common moment-matching perspective.The paper argues that implicit models would benefit from a breadth of Bayesian reasoning approaches comparable to prescribed models.
- Inference applications: Implicit models support amortised inference and sample-based learning when probabilities need not be evaluated, with applications to importance sampling, variational inference, and message passing.The paper connects this approach to GAN-like inference in factor graphs and to density-ratio-based variational inference.
- Evaluation: Perceptual losses can better capture similarity than pixel-space reconstruction errors, but pretrained discriminative networks introduce label information and complicate fair evaluation.Comparisons are fairly made only with joint models using both label and image information.
- Open challenges: The discussion leaves unresolved how to extend the framework to non-differentiable or discrete simulators and how to scale available approaches to high-dimensional data.Candidate directions include score-function estimators, moment matching, ABC-MCMC, and gradient-free optimisation methods.