PaperlayerView exact revision on arXiv

Source-linked AI summary

Advances in Variational Inference

Cheng Zhang, Judith Butepage, Hedvig Kjellstrom, Stephan Mandt

arXiv:1711.05597v3cs.LGstat.ML

TL;DR

Complex probabilistic models make exact Bayesian posterior inference intractable, creating a need for scalable approximate methods. The paper reviews variational inference as optimization over simpler posterior approximations and organizes recent advances into scalable, generic, accurate, and amortized approaches. These developments broaden VI’s applicability and address limitations of mean-field approximations, including ignored dependencies and restricted model classes.

  • Problem

    Exact posterior inference is typically intractable for complex models, while classical VI is limited by mean-field assumptions and conditionally conjugate model requirements.

  • Method

    The paper presents a review of variational inference and recent advances in scalable, generic, accurate, and amortized inference.

  • Results

    The review surveys advances including stochastic and distributed inference, methods for non-conjugate models, structured or alternative-divergence approximations, and inference networks.

  • Takeaways & Limitations

    Recent VI developments extend inference to larger datasets and more complicated models while offering alternatives to restrictive mean-field approximations.

  • Takeaways & Limitations

    Mean-field approximations explicitly ignore correlations between variables, and under-parameterized variational distributions may not capture the full posterior complexity.

Abstract

from arXiv · show

Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully used in various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent advances focusing on the following aspects: (a) scalable VI, which includes stochastic approximations, (b) generic VI, which extends the applicability of VI to a large class of otherwise intractable models, such as non-conjugate models, (c) accurate VI, which includes variational models beyond the mean field approximation or with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks. Finally, we provide a summary of promising future research directions.

1 INTRODUCTION

Bayesian posteriors are often intractable in complex, large-scale probabilistic models, motivating variational inference as an efficient optimization-based approximation. This review surveys recent advances in scalable, generic, accurate, and amortized VI.

  • Motivation: Exact Bayesian inference is typically intractable because complex models require summing or integrating over millions or billions of latent variables.VI addresses this by approximating the posterior with a simpler distribution and optimizing its parameters.
  • Variational inference: VI approximates the posterior by minimizing KL divergence, avoiding intractable normalization constants while requiring only the joint distribution.The approach converts inference into an optimization problem over a simpler variational distribution.
  • Research directions: Recent VI research addresses scalability through stochastic optimization, generic applicability beyond conjugate models, more accurate approximations, and amortized inference.Amortized inference uses neural networks to predict variational distributions conditioned on data points.
  • Review scope: The review focuses on the decade of developments preceding publication and emphasizes an exhaustive recent-literature survey over example calculations.It positions this focus as complementary to earlier reviews with different coverage.
  • Review scope: The review provides a self-contained survey of recent VI developments organized around four directions: scalable, generic, accurate, and amortized VI.It begins with basic concepts and concludes with discussion and future research directions.

2 VARIATIONAL INFERENCE

Variational inference turns intractable posterior inference into optimization over a simpler variational distribution, typically maximizing the ELBO or equivalently minimizing KL divergence. Mean-field factorization enables simple updates but trades posterior expressiveness and accuracy for tractability.

  • Inference as optimization: VI approximates a high-dimensional posterior with a simpler variational distribution whose parameters are optimized as a proxy for the posterior.This reframes Bayesian inference as optimization over variational parameters.
  • Variational objective: The ELBO is a lower bound on the log marginal probability, and maximizing it is equivalent to minimizing KL divergence between the variational distribution and posterior.The ELBO can also serve as a conservative model-fit estimate and support model selection.
  • Variational objective: Traditional VI analytically computes expectations over the variational distribution, restricting tractable models to conditionally conjugate exponential families.Modern alternatives address this expectation-computation restriction.
  • Mean Field Variational Inference: Mean-field independence simplifies derivations but can reduce accuracy when posterior variables are dependent, reflecting a tradeoff between expressiveness and tractability.More structured variational families are discussed as more expressive alternatives.
  • Mean Field Variational Inference: Mean-field VI factorizes the variational distribution, assigning each factor its own parameter and enabling simple iterative ELBO updates.Updates proceed for each latent variable until convergence.
  • Beyond Vanilla Variational Inference: Classical VI’s modern limitations motivate scalable stochastic and distributed methods, generic methods for non-conjugate models, and more accurate variational approximations.These directions extend the method beyond vanilla VI.

3 SCALABLE VARIATIONAL INFERENCE

Scalable VI addresses large datasets by applying stochastic optimization to the VI objective and extending it with variance reduction, adaptive updates, distributed methods, and sparse representations.

  • 3.1 Stochastic Variational Inference: Stochastic variational inference applies stochastic optimization to VI, scaling inference to very large datasets.It estimates the objective and its gradient from randomly selected mini-batches rather than processing all M data points each iteration.
  • 3.1 Stochastic Variational Inference: Natural gradients simplify SVI updates for models in the conditionally conjugate exponential family by incorporating the model’s information geometry.They pre-multiply the gradient by the inverse Fisher information matrix.
  • 3.1 Stochastic Variational Inference: SVI convergence requires uniformly sampled mini-batches and decreasing learning rates satisfying Robbins-Monro conditions.Mini-batches must satisfy 1 ≤ S ≪ M for computational savings, while larger S reduces stochastic-gradient variance.
  • 3.1 Stochastic Variational Inference: When the dataset size M is unknown, online VI cannot directly set the scale factor M/S, motivating population-posterior formulations.When M is known, SVI and online VI are equivalent under the stated assumptions.
  • 3.2 Tricks of the Trade for SVI: Adaptive learning rates, adaptive mini-batch sizes, and variance-reduction methods improve convergence by reducing stochastic-gradient noise.The reviewed approaches include control variates, non-uniform sampling, and optimizers such as RMSProp, AdaGrad, AdaDelta, and Adam.
  • 3.3 Collapsed, Sparse, and Distributed VI: Sparse Gaussian-process inference replaces an M × M kernel inversion with a T × T inversion using T ≪ M inducing points, reducing complexity to O(MT^2).A stochastic extension further reduces the stated complexity to O(T^3).

4 GENERIC VI: BEYOND THE CONJUGATE

Generic VI broadens variational inference beyond conditionally conjugate models by using stochastic gradient estimators, including black-box score-function and reparameterization methods. These approaches improve automation and applicability but retain variance and distributional limitations.

  • 4 Generic VI: Beyond the Conjugate: Generic VI removes the conditionally conjugate requirement and reduces the need for model-specific analytic calculations.The section centers on stochastic gradient estimators of the ELBO for broader model classes.
  • 4.1 Laplace Approximation: Laplace approximation uses the posterior mode and inverse Hessian as the mean and covariance of a Gaussian approximation, but is local and computationally costly in high dimensions.It is also limited to Gaussian variational families and does not apply to discrete variables.
  • 4.2 REINFORCE Gradients: BBVI estimates unbiased ELBO gradients by sampling from the variational distribution without analytically computing the ELBO or its gradient.Practitioners need only provide the joint distribution of observations and latent variables.
  • 4.2 REINFORCE Gradients: BBVI’s score-function gradients can have high variance, motivating Rao-Blackwellization, control variates, local expectation gradients, and overdispersed importance sampling.The score-function control variate preserves expectation while reducing gradient variance when its weight is chosen appropriately.
  • 4.3 Reparameterization Gradient VI: Reparameterization gradients represent variational random variables as deterministic transformations of noise, producing low-variance Monte Carlo gradients without analytic expectations.For a Gaussian, z = μ + σε with ε ∼ N(0,1).
  • 4.3 Reparameterization Gradient VI: Reparameterization gradients often have lower variance than score-function gradients, but this advantage is not theoretically guaranteed.Near the optimum, omitting the score-function component can yield an unbiased estimator with lower variance.
  • 4.3 Reparameterization Gradient VI: The reparameterization trick does not trivially extend to discrete distributions and may require approximations such as Gumbel-Softmax or Kumaraswamy substitutions.For categorical variables, temperature controls the approximation; closer categorical behavior increases gradient variance.

5 ACCURATE VI: BEYOND KL AND MEAN FIELD

Accurate VI extends standard mean-field, KL-based inference to improve posterior approximation quality through alternative divergences, structured distributions, and methods targeting local optima.

  • Beyond KL and mean field: Recent accurate VI methods go beyond the standard mean-field setup and KL divergence to avoid poor local optima and improve accuracy.The review covers alternative divergences, structured approximations, and other methods.
  • Origins and limitations of mean field VI: Mean-field approximations ignore dependencies between variables, and breaking more dependencies can make VI optimization increasingly non-convex.Adding structure can eliminate certain local optima.
  • Alternative divergences: KLVI can underestimate posterior variances, fail to break symmetry between nearby modes, and provide a comparatively loose bound.These shortcomings motivate alternative divergence measures.
  • Alternative divergences: α-divergences interpolate between mass-covering behavior at smaller α and zero-forcing behavior at larger α, recovering standard KL-based VI as α approaches 1.Renyi’s formulation is the focus of this review.
  • Alternative divergences: For α ≥ 0, α ≠ 1, Renyi’s formulation yields a lower bound on the log marginal likelihood, while negative α produces an upper bound.Only Renyi’s formulation among the discussed α-divergence definitions yields a bound where the marginal likelihood cancels.
  • Alternative divergences: Polynomial corrections beyond linear order can tighten a variational bound, connecting alternative-divergence methods to TAP equations.Terms through linear order correspond to KL divergence, while higher-order terms act as corrections.
  • Structured variational inference: Hierarchical variational distributions capture dependencies by placing a distribution over mean-field parameters and marginalizing them out.The resulting distribution can be sampled through the hierarchical process and is used in variational Gaussian processes.
  • Other accurate VI methods: Variational tempering adapts deterministic annealing with global or data-point-specific temperatures to help VI escape poor local optima.The temperatures are adaptive and data-dependent.

6 AMORTIZED VARIATIONAL INFERENCE

Amortized VI replaces separate local variational-parameter optimization with shared inference functions, enabling efficient inference for new data and supporting scalable deep latent-variable models such as VAEs.

  • Amortized inference: Traditional VI optimizes separate local parameters for each data point, which is expensive when nested within global parameter updates.Amortized inference addresses this cost by predicting local latent variables from data features.
  • Amortized inference: Amortized VI replaces per-datapoint variational parameters with a shared parameterized function of the data.Once estimated, the function produces latent-variable inferences for new data points.
  • Applications: Inference networks have been used to make deep Gaussian-process inference scale to larger datasets and converge faster.The latent variables are estimated as functions rather than separately for each instance.
  • Variational autoencoders: VAEs commonly combine a top-down generative network with a bottom-up recognition network that approximates the posterior.These networks map between latent variables and observed data in opposite directions.
  • Variational autoencoders: VAEs train the inference and generative networks jointly to optimize the ELBO, providing a scalable and efficient training scheme for deep latent-variable models.The contribution is attributed to the original VAE work.
  • Variational autoencoders: The reparameterization trick expresses approximate-posterior samples as a function of noise and data, allowing stochastic ELBO estimates to be differentiated.Its drawback is that the approximate posterior must be reparameterizable.
  • Advancements in VAEs: Increasing the number of IWAE samples tightens the variational bound and approaches the true log likelihood in the limit.The reinterpretation also describes IWAEs as using a more expressive distribution that converges pointwise to the true posterior.
  • Advancements in VAEs: The VAE KL term can cause the inference model to match the prior instead of learning a useful representation when the model is sufficiently expressive.This behavior follows from the ELBO decomposition described in the review.

7 DISCUSSION

The review identifies theoretical VI, deep learning, policy gradients, and automatic VI as active directions, while noting unresolved challenges in approximation theory and practical usability.

  • Theory of VI: Open theoretical questions include quantifying approximation errors between true posteriors and simplified variational distributions, as well as predictive errors.These issues concern both posterior approximation and Bayesian predictive distributions.
  • Theory of VI: The review conjectures that deeper connections between VI theory and information theory could produce further results.The information bottleneck has already inspired a deep variational information bottleneck.
  • VI and deep learning: VI remains important for deep learning because Bayesian approaches address uncertainty estimation, feature interpretability, and incorporation of prior knowledge.Making VI efficient and easy to implement in Bayesian deep architectures is identified as an important research direction.
  • VI and policy gradients: Applying VI to reinforcement learning and stochastic control is an active research area because their technical challenges resemble those of VI.Stein variational policy gradients are one example of this connection.
  • Automatic VI: Probabilistic programming can automate inference after users specify a model, with tools supporting VI, message passing, EP, or sampling methods.The review lists Stan, Infer.Net, Automatic Statistician, Anglican, Edward, and Zhusuan as examples.
  • Automatic VI: VI remains difficult for non-experts because current toolboxes do not fully address posterior symmetries or model-specific variance-reduction design.The review identifies these issues as barriers to broader impact.

8 CONCLUSIONS

The review organizes recent advances in variational inference around scalability, generality, accuracy, and amortized inference, while identifying more efficient, accurate, and accessible VI as an open direction.

  • The review examines variational inference through four perspectives: scalability, generality, accuracy, and amortized inference.
  • Despite rapid growth in VI theory and applications, making VI more efficient, more accurate, and easier for non-experts remains open.
  • The paper identifies further development as necessary for the field’s continuing progress.

A.1 ELBO and KL

The ELBO is linked to the marginal likelihood through the KL divergence between the variational distribution and the target distribution. This equivalence provides two derivations of the ELBO.

  • The difference between log p(x) and the ELBO L equals the KL divergence between q(z;λ) and p(x,z).
  • The KL formulation expresses ELBO optimization through the relationship between the variational approximation and the target distribution.
  • The ELBO can be derived either with Jensen’s inequality or from L = log p(x)−D_KL(q||p).

A.2 Conjugate Exponential family

Exponential-family distributions are represented using natural parameters, sufficient statistics, a base measure, and a lognormalizer. Conjugate pairs preserve the prior’s family in the posterior.

  • An exponential-family distribution is characterized by a natural parameter, sufficient statistics, a base measure, and a lognormalizer.
  • In Bayesian statistics, conjugate likelihood-prior pairs produce posteriors in the same family as the prior.
  • Examples include Gaussian likelihoods with Gaussian priors, Poisson likelihoods with gamma priors, and multinomial likelihoods with Dirichlet priors.

A.3 Variational Message Passing

Variational message passing formulates mean field variational inference as local message updates. In Bayesian networks, each latent variable uses information from its Markov blanket, enabling scalable and automatable inference.

  • MFVI updates each latent variable sequentially using information from its Markov blanket.The Markov blanket contains the variable’s parents, children, and the co-parents of its children.
  • For conjugate-exponential models, each latent variable receives messages from its parents and children before updating.
  • Each update consists of receiving messages, updating the latent variable’s natural parameter, and updating the expectation of its sufficient statistic.
  • Variational message passing generalizes MFVI into a message-passing formulation that can be used in large-scale Bayesian networks and automated easily.
  • Together with expectation propagation, variational message passing forms the basis of the Infer.Net probabilistic programming tool.

A.4 Natural Gradients and SVI

The section develops natural-gradient and variance-reduction tools for variational inference, then connects entropy-regularized policy optimization in reinforcement learning to the ELBO. Natural gradients use distribution-aware geometry, while Rao–Blackwellization improves estimators through conditioning on sufficient statistics.

  • Natural Gradients: Natural gradients replace the regular gradient with a Fisher-information-adjusted gradient, yielding a simpler form for exponential-family variational distributions.They optimize in Riemann space, where distances between distributions are defined through KL divergence rather than Euclidean geometry.
  • Variance Reduction: Rao–Blackwellization reduces estimator variance by conditioning an estimator on sufficient statistics.The conditional estimator is at least as good as the original and is strictly better unless the original is already a function of the sufficient statistic.
  • Physics Notation: The review translates between machine-learning and physics notation, identifying variational free energy as the negative ELBO.In this correspondence, Z is the marginal likelihood or partition function, and −logZ is called free energy.
  • Policy Gradient Estimation as VI: Policy optimization can be formulated as variational inference by maximizing expected return under a variational distribution over policy parameters.With maximum-entropy regularization, the resulting objective is identical to the ELBO.
Loading 1711.05597v3…