Source-linked AI summary
Variational Inference using Implicit Distributions
Ferenc Huszár
TL;DR
Implicit distributions could extend variational inference beyond tractable-density models, but inference lacks posterior samples and may involve implicit priors or likelihoods. The paper unifies prior algorithms and presents contrastive ELBO forms with adversarial and denoising estimators; it concludes these methods can be combined, potentially improving convergence.
Problem
Variational inference often relies on tractable-density distributions, while useful posterior approximations, likelihoods, or priors may be implicit.
Source: §1 · pp. 1–2Method
Results
The paper presents algorithms using adversarial methods to estimate density ratios and denoising methods to estimate log-density gradients, and reports that they can be combined, potentially improving convergence.
Source: §5 · p. 7Takeaways & Limitations
The framework connects existing implicit-inference algorithms and provides a range of methods for variational approximate inference, though not all support learning with an implicit likelihood.
Source: §1 · p. 2, §5 · p. 7None of the presented algorithms enables fully variational learning of θ when pθ(x|z) is implicit.
Source: Is the prior-contrastive too much like… · p. 8
Abstract
from arXiv · showhide
Generative adversarial networks (GANs) have given us a great tool to fit implicit generative models to data. Implicit distributions are ones we can sample from easily, and take derivatives of samples with respect to model parameters. These models are highly expressive and we argue they can prove just as useful for variational inference (VI) as they are for generative modelling. Several papers have proposed GAN-like algorithms for inference, however, connections to the theory of VI are not always well understood. This paper provides a unifying review of existing algorithms establishing connections between variational autoencoders, adversarially learned inference, operator VI, GAN-based image reconstruction, and more. Secondly, the paper provides a framework for building new algorithms: depending on the way the variational bound is expressed we introduce prior-contrastive and joint-contrastive methods, and show practical inference algorithms based on either density ratio estimation or denoising.
1. Introduction
The paper argues that implicit distributions can broaden variational inference beyond models with tractable densities, while inference differs from generative modeling because posterior samples are unavailable. It organizes prior work and introduces a framework of contrastive ELBO forms and estimation methods for implicit distributions.
1. Introduction
- Unlike direct generative modeling from observed-data samples, inference starts with a latent-variable joint distribution and observed samples, but no posterior samples.The paper focuses on variational inference, which minimizes a divergence between approximate and real posteriors; its ELBO formulation is closer to VAE fitting than GAN fitting.Source: §1 · p. 1
- Implicit distributions may offer more expressive posterior approximations than tractable-density families, and may also be useful for implicit likelihoods or priors.The paper identifies cases where any or all of the prior, likelihood, and approximate posterior may need implicit models.Source: §1 · pp. 1–2
- The paper presents adversarial density-ratio estimation and denoising-based log-density-gradient estimation, classifying algorithms by prior-contrastive or joint-contrastive ELBO form.Prior-contrastive methods handle implicit latent-variable distributions, while joint-contrastive methods can handle fully implicit models; not all methods support learning when the likelihood is implicit.Source: §1 · p. 2
1.1. Overview of prior work
- AVB (Mescheder et al., 2017) is equivalent to PC-Adv, while OPVI (Ranganath et al., 2016) includes the paper’s KL-based variational bound as a special case.AAEs (Makhzani et al., 2016) are not true variational methods in the paper’s account because their adversarial loss is not KL-based and their discriminator is independent of the encoder input.Source: §1.1 · pp. 2–3
- ALI and BiGAN discriminate between joint distributions like JC-Adv, but use Jensen-Shannon divergence rather than the KL formulation.Their symmetric objective supports learning as well as inference, whereas JC-Adv is an inference algorithm; GAN image-reconstruction work also has variational connections, with AffGAN limited to a special case.Source: §1.1 · pp. 2–3, Table 1 · p. 2
- Denoising autoencoders estimate gradients of log density, a route used in image super-resolution and closely related to prior-contrastive denoising VI.The paper connects that earlier approach to amortised MAP inference and describes the extension to VI as straightforward.Source: §1.1 · p. 3
2. Variational Inference: Two forms
The section expresses variational inference through ELBO forms and explains how implicit distributions require sample-based approximations. It also describes how parameter dependence and nested stochastic optimization shape the resulting algorithms.
2. Variational Inference: Two forms
- In a latent-variable model, the paper distinguishes maximum-likelihood learning of model parameters from inference about latent-variable posteriors, assuming the marginal likelihood and posterior are intractable.Amortized VI introduces qψ(z|x) as an approximate posterior or recognition model.Source: §2 · p. 3
- The ELBO is a lower bound on model evidence, exact when qψ matches the true posterior; maximizing it over ψ minimizes the KL divergence to that posterior.Maximizing over θ instead is variational learning, which approximates maximum-likelihood learning.Source: §2 · p. 3
- When pθ or qψ is implicit, the ELBO cannot generally be calculated as in tractable-density VAEs, motivating formulations in terms of density ratios.The VAE reparameterization trick still requires tractable densities for both the model and recognition model.Source: §2 · p. 3
- The prior-contrastive form contrasts qψ(z|x) with the prior pθ(z), whereas the joint-contrastive form contrasts pD(x)qψ(z|x) with pθ(x,z).The joint expression uses the logarithmic ratio sθ,ψ over the observation-latent tuple; H[pD] is constant with respect to θ and ψ.Source: §2 · pp. 3–4
- With implicit models, the ratios rθ,ψ and sθ,ψ are not analytically available, so the paper estimates them or their gradients directly from samples.Even when other densities are tractable, pD is empirical, preventing analytic calculation of sθ,ψ.Source: §2 · p. 4
2.1. Dependence of rθ,ψ and sθ,ψ on ψ
- The ratios depend on ψ both through sampling under qψ and through the ratios themselves, but the latter dependence can be ignored by holding the ratio parameter fixed in the gradient expression.The resulting gradient is an expectation-gradient under qψ; the paper says an analogous result applies to sθ,ψ and other KL divergences.Source: §2.1 · p. 4
2.2. Approximate SGD Algorithms for VI
- The algorithms use nested loops: an inner loop estimates a ratio or its gradient for fixed ψ, and an outer loop performs SGD on an ELBO approximation.The paper associates SGD convergence properties with estimates having no memory across outer iterations and averaging to a conservative vector field.Source: §2.2 · p. 4
3. Direct Density Ratio Estimation
The section develops discriminator-based density-ratio estimation for variational inference, with prior-contrastive and joint-contrastive adversarial methods. Their model requirements differ: PC-Adv needs an explicit likelihood, while JC-Adv can use fully implicit models but does not directly support variational learning of model parameters.
3. Direct Density Ratio Estimation
- Direct density ratio estimation learns the ratio of two densities from independent samples, and this section focuses on discriminator-based logistic regression.At a unique Bayes optimum, the classifier can provide an estimate of the logarithmic density ratio.Source: §3 · p. 4, §3.1 · p. 5
3.1. Adversarial approach using discriminators
- After estimating the log ratio, PC-Adv takes a gradient step on the approximate negative ELBO using samples reparameterized as gψ(x, ϵ).Implicit posteriors can use their generator and transformed noise for this reparameterization.Source: §3.1 · p. 5
- PC-Adv estimates a logarithmic density ratio with a discriminator and uses GAN-analogous discriminator and generator losses to optimize the approximate ELBO; it is equivalent to AVB (Mescheder et al., 2017).The likelihood must be explicit, while the prior and approximate posterior can be implicit.Source: §3.1 · p. 5
- Prior parameters require a separate learning step: fitting the prior to aggregate-posterior samples, with another ELBO available when the prior is itself a latent-variable model.The prior-learning discussion notes that this can stack models hierarchically.Source: §3.1 · p. 5
- JC-Adv discriminates between the data-posterior and model joint distributions, works with fully implicit models, but offers no direct way to variationally learn θ.It is closely related to ALI and BiGAN, but uses the asymmetric KL rather than their symmetry-based Jensen-Shannon objective.Source: §3.1 · p. 5
4. Denoiser-guided learning
Denoiser-guided learning uses denoising autoencoders to estimate log-density gradients for implicit distributions and use those estimates in variational inference. The approach is flexible about which distributions are explicit or implicit, but does not directly learn model parameters θ.
4. Denoiser-guided learning
- A Bayes-optimal denoiser captures log-density gradients, allowing a denoising autoencoder trained on samples to estimate a distribution’s score.These estimates can be used to approximate the ELBO gradient with respect to variational parameters.Source: §4 · p. 6
- PC-Den updates the denoiser and variational distribution together, while denoising variants can model an implicit prior or a joint distribution over x and z.The denoising approach allows individual distributions to be chosen as explicit or implicit.Source: §4 · p. 6
- The denoiser’s gradient estimate can be used by stochastic-gradient methods to optimize an approximate ELBO.The section notes that the detailed descriptions of the denoising variants are omitted for space.Source: §4 · p. 7
- Denoising estimates ELBO gradients but not its value, so it provides no direct way to learn model parameters θ.The paper suggests combining denoising with an adversarial algorithm; denoising may help early, while the discriminator can model residual differences as qψ approaches pθ.Source: §4 · p. 7
5. Summary
The paper organizes implicit-distribution variational inference around two ELBO formulations and two estimation strategies. It also notes that full variational learning remains unavailable for some implicit models, while proposing a reverse-mode-differentiation idea for that setting.
5. Summary
- The paper formulates the ELBO using prior-contrastive and joint-contrastive logarithmic density ratios, then estimates ratios adversarially or log-density gradients through denoising.The denoising variants are identified as PC-Den and JC-Den; the summary also says the methods can be combined, which may improve convergence.Source: §5 · p. 7
- Not all described algorithms can fully variationally learn model parameters θ when the model is implicitly defined.The paper outlines JC-Adv-RMD, based on reverse-mode differentiation, as an idea that could in theory support full inference and learning when all distributions are implicit.Source: §5 · p. 7
- Table 1 summarizes algorithm capabilities and relationships, including PC-Adv’s equivalence to AVB and JC-Adv’s close relation to ALI and BiGAN.The paper describes PC-Adv-related image-reconstruction work as either a limited special case or using Jensen-Shannon rather than a fully variational formulation.Source: §5 · p. 7
6. Experiments
The experiments validate inference on a continuous sprinkler toy model designed to exhibit explaining away. Figure 5 displays the true posterior and three algorithmic approximations, alongside discriminator limiting behaviors described in the text.
6. Experiments
- The continuous sprinkler toy model tests inference with two independent scalar latent variables combined nonlinearly to produce one observation.The experiment is intended to validate inference rather than the generative-model learning focus of prior work.Source: §6 · p. 7
- Conditioning on the observation makes the initially independent latent variables dependent: either can explain a large observed x, illustrating explaining away.The text says this phenomenon is difficult to model with simple unimodal distributions.Source: §6 · p. 7
- Figure 5 places the true joint posterior in column A and PC-Adv, JC-Adv, and PC-Den approximations in subsequent columns; the text describes their discriminator limits.For PC-Adv, the discriminator is expected to resemble the likelihood up to an additive constant as qψ approaches the true posterior; for JC-Adv, it converges to the flat sφ = 0 solution.Source: §6 · p. 7
7. Discussion and future work
The discussion weighs whether adversarial flexibility is needed for VI and identifies practical limits of adversarial and denoising inference, while outlining a possible route to learning θ.
7. Discussion and future work
- The discussion asks whether adversarial flexibility is needed because VI distributions are usually considered simpler than distributions of observed data.Source: §7 · p. 7
Is the prior-contrastive too much like noise-contrastive?
- The toy-model estimates show no systematic difference among PC-Adv, JC-Adv, and PC-Den; approximation quality seems predominantly influenced by qψ’s architecture.For perfect approximate inference, PC-Adv’s rφ should match the likelihood up to an additive constant.Source: Figure 1 · p. 8
- In PC-Adv, the discriminator’s approximate-posterior and prior distributions never overlap, even at convergence, which may slow training.The passage suggests an ensemble of a neural network and the actual log-likelihood to make the discriminator’s task easier.Source: Is the prior-contrastive too much like… · p. 8
- Denoising gradients represent the noise-corrupted distribution rather than the original, and the noise level may be too large for this effect to be negligible.Consistent outer-loop SGD also requires conservative gradient estimates, a property whose preservation with suboptimal denoisers is unclear.Source: Is the prior-contrastive too much like… · p. 8
- Score matching is proposed as an alternative that might overcome denoising methods’ gradient-imprecision and conservative-field concerns.The passage connects score matching to denoising.Source: Is the prior-contrastive too much like… · p. 8
- None of the presented algorithms enables fully variational learning of θ when pθ(x|z) is implicit; differentiating JC-Adv’s inner loop via reverse-mode differentiation is proposed as a possible route.The proposed JC-Adv-RMD approach has not been experimentally validated.Source: Is the prior-contrastive too much like… · p. 8
A. Ignoring implicit dependence on ψ
The proof of Eqn. (4) uses the local-minimum property of ψ0 to eliminate a term in the derivation.
A. Ignoring implicit dependence on ψ
- The passage identifies the argument as a proof of Eqn. (4).Source: A. Ignoring implicit dependence on ψ · p. 11
- The proof of Eqn. (4) sets the second term on the second line to zero because ψ0 is a local minimum of KL[qψ∥qψ0].Source: A. Ignoring implicit dependence on ψ · p. 11