Variational Bayesian Inference with Stochastic Search

John Paisley; David Blei; Michael Jordan

Source-linked AI summary

Variational Bayesian Inference with Stochastic Search

John Paisley, David Blei, Michael Jordan

arXiv:1206.6430v1Jun 27, 2012cs.LGstat.COstat.ML

TL;DR

MFVB struggles when required expectations are not tractable, because replacing them with lower bounds means the true variational objective is no longer directly optimized. The paper introduces stochastic search variational Bayes, using unbiased Monte Carlo gradients and control variates, including existing bounds and second-order Taylor approximations. Experiments demonstrate the approach on logistic regression and a finite HDP approximation, while the delta method is not itself a lower bound.

Problem
Intractable expectations in MFVB are commonly handled with lower bounds, but this means the true variational objective is no longer optimized and can worsen posterior approximation.
Method
Stochastic search variational Bayes directly optimizes the variational objective with an unbiased stochastic gradient and uses correlated control variates to reduce Monte Carlo variance.
Results
Existing lower bounds can serve as control variates, while the control-variate delta method uses second-order Taylor approximations that need not bound the intractable function.
Takeaways & Limitations
The approach extends stochastic variational inference to nonconjugate models and broadens usable approximations beyond lower bounds.
Takeaways & Limitations
The delta method uses a second-order Taylor expansion rather than a lower bound, so it is not itself a standard MFVI bound.

Abstract

from arXiv · show

Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal likelihood. This requires the ability to integrate a sum of terms in the log joint likelihood using this factorized distribution. Often not all integrals are in closed form, which is typically handled by using a lower bound. We present an alternative algorithm based on stochastic optimization that allows for direct optimization of the variational lower bound. This method uses control variates to reduce the variance of the stochastic search gradient, in which existing lower bounds can play an important role. We demonstrate the approach on two non-conjugate models: logistic regression and an approximation to the HDP.

1. Introduction

MFVB approximates Bayesian posteriors with factorized distributions, but non-closed-form expectations can force tractable lower bounds that prevent direct optimization of the true objective. The paper proposes stochastic optimization with control variates to optimize that objective directly.

MFVB approximates a full Bayesian posterior with a factorized distribution by maximizing a variational lower bound.
When expectations in the log joint likelihood lack closed forms, point-wise lower bounds enable tractable inference but may worsen the posterior approximation.
The proposed stochastic approximation of ∇ψL is unbiased, allowing direct optimization of the variational objective when expectations are intractable.
Control variates reduce Monte Carlo variance, with existing lower bounds serving as useful correlated approximations.
Unlike lower-bound methods, the control-variate framework can use correlated functions such as second-order Taylor expansions.

2. Mean-field variational inference

MFVB uses a factorized distribution to approximate the posterior and optimizes its variational parameters through a lower bound. Intractable expectations motivate tractable bounds, whose looseness can impair optimization.

MFVB approximates the posterior P(Θ|X, Υ) with a factorized distribution Q(Θ|Ψ).
Maximizing the variational lower bound L is equivalent to minimizing the Kullback-Leibler divergence between Q(Θ) and P(Θ|X, Υ).
The log joint likelihood is decomposed into terms whose expectations produce functions of the corresponding variational parameters.
Point-wise lower bounds replace intractable functions with tractable expectations, often using auxiliary variables to control tightness.
Looser bounds move optimization farther from the variational objective and can impair learning of ψi.

3. Stochastic search variational Bayes

Stochastic search variational Bayes directly optimizes the variational objective when expectations are intractable by replacing an expectation gradient with an unbiased Monte Carlo estimate and stochastic updates.

3. Stochastic search variational Bayes: The method targets the variational objective L when some log-joint expectations cannot be computed.
3. Stochastic search variational Bayes: The gradient separates into a tractable component ∇ψh and an intractable component ∇ψEq[f].
3. Stochastic search variational Bayes: Using ∇ψq(θ|ψ) = q(θ|ψ)∇ψ ln q(θ|ψ), the intractable gradient becomes an expectation of f(θ)∇ψ ln q(θ|ψ).
3. Stochastic search variational Bayes: The expectation is estimated with S iid samples θ(s) ∼q(θ|ψ), yielding an unbiased stochastic gradient approximation ζ.
3. Stochastic search variational Bayes: Variational parameters are updated by stochastic gradient steps with learning rates satisfying the stated convergence requirement.

4. Searching with control variates

Monte Carlo gradient estimates can have high variance and require many samples. Control variates preserve expectations while reducing variance, including through approximations that need not be bounds.

4. Searching with control variates: High gradient-estimation variance can require many samples and make stochastic search slow.
4. Searching with control variates: A control variate g(θ) reduces variance by correlating with f(θ) while retaining a tractable expectation under q.
4. Searching with control variates: The transformed function has the same expectation as f, so it can replace f in the variational objective.
4. Searching with control variates: The algorithm estimates control-variate coefficients and sample requirements, then constructs a corrected stochastic search vector and updates ψ.
4. Searching with control variates: Greater correlation between f and g yields greater variance reduction; tight bounds, upper bounds, and other accurate approximations can qualify.
4. Searching with control variates: For gradient vectors, the control variate is applied to g(θ)∇ψ ln q(θ|ψ), with separate variance minimization across dimensions.

5. Stochastic search VB for two models

The paper applies stochastic search variational inference to logistic regression and a finite HDP, using control variates to make intractable expectations tractable through stochastic approximation. Taylor-based and model-specific control variates improve approximation or reduce variance, with distinct constructions for each model.

Logistic regression: Logistic regression uses a Jaakkola–Jordan lower-bound control variate and a second-order Taylor control variate for the intractable likelihood expectation.The lower bound supports closed-form variational inference, while the Taylor approximation is used within stochastic search.
Logistic regression: Because the second-order Taylor expansion is not a lower bound, conventional mean-field variational inference generally cannot use it directly, whereas stochastic search can use it as a control variate.This permits more accurate function approximation and significantly reduces stochastic-gradient variance.
Logistic regression: The Taylor expansion approximates the logistic likelihood more accurately than the lower-bound control variate in regions sampled from q.The comparison uses two specific q distributions and 100 samples to identify the regions relevant to stochastic-gradient evaluation.
Hierarchical Dirichlet processes: The finite HDP uses control variates for the intractable expectation of ln Γ(βθk), including ln βθk near zero and a first-order Taylor approximation for larger βθk.Setting a = 1 makes the difference between the intractable function and the control variate vanish as βθk approaches zero; the Taylor approximation handles the nearly linear large-value regime.
Hierarchical Dirichlet processes: For the HDP, the ln βθk control variate substantially reduces the very large variance near zero, where many Dirichlet components lie.The method uses the identity −ln Γ(βθk)−ln βθk = −ln Γ(βθk+1) and sets a = 1.
Hierarchical Dirichlet processes: The HDP stochastic gradients can also reduce variance by modifying the variational distribution, and separate stochastic approximations are performed for the intractable simplex integrals.The modified distribution q′k yields significantly less variance than q.

6. Experiments

Experiments evaluate stochastic-search variational Bayes on logistic regression and an approximate HDP, focusing on posterior objectives, variance reduction, sampling, and runtime. Control variates substantially improve stochastic inference, with stronger gains for logistic regression than for the HDP.

Experimental setup: Experiments apply stochastic-search variational Bayes to binary logistic regression and topic modeling with an approximate HDP.The logistic-regression study uses five UCI data sets; the HDP study uses New York Times documents with K = 200.
Logistic regression results: Stochastic-search methods with control variates significantly outperform other methods on the true variational lower bound for logistic regression.The comparison evaluates posterior approximations using the same true lower-bound objective.
Logistic regression results: Control variates provide a major variance reduction, and the Taylor expansion control variate requires significantly fewer samples than the bound control variate.The reduced sample requirement improves running time, although non-sampling methods remain faster.
Hierarchical Dirichlet process results: For the HDP, fitting a variational posterior on the top-level Dirichlet vector yields a better posterior approximation than a point estimate or a uniform top-level distribution.The improvement is reported across document sizes and concentration parameters, but is less dramatic than for logistic regression.
Hierarchical Dirichlet process results: The HDP variance-reduction method significantly reduces the number of samples needed for the stochastic integral.Experiments without control variates were infeasible because of computer-precision issues and massive variance near zero.

7. Conclusion

The conclusion presents stochastic-search variational Bayes for directly optimizing intractable variational objectives and introduces unrestricted control-variate approximations. The method uses Monte Carlo gradients whose variance can be reduced by existing bounds or alternative approximations such as a second-order Taylor expansion.

Conclusion: Stochastic-search variational Bayes directly optimizes intractable variational objective functions arising from nonconjugacy.It uses a stochastic approximation of the gradient and applies control variates to reduce Monte Carlo variance.
Conclusion: Because control variates need only correlate well with the target function, they can use approximations beyond lower bounds when suitable bounds are unavailable.The paper introduces the control-variate delta method as one such alternative, based on a second-order Taylor expansion.