Source-linked AI summary
Hierarchical Variational Models
Rajesh Ranganath, Dustin Tran, David M. Blei
TL;DR
Mean-field variational inference is computationally efficient but cannot represent posterior dependencies and all marginal information. The paper develops hierarchical variational models that place priors on variational parameters, with a black-box fitting algorithm retaining BBVI efficiency; HVMs generally outperform mean-field approximations in the evaluated deep exponential-family models.
Problem
Mean-field factorization limits variational expressiveness by abandoning posterior dependence structure and failing to capture all marginal information while richer families must remain computationally tractable.
Method
HVMs place priors on the parameters of existing variational families, and a black-box algorithm fits any HVM by optimizing its variational hyperparameters.
Results
HVMs generally outperform mean-field variational inference across several deep exponential-family models, achieving better performance over six models and two datasets with a mean perplexity improvement of 180 points.
Takeaways & Limitations
HVMs provide richer posterior approximations that capture latent dependencies and complex marginal distributions while preserving the computational efficiency of BBVI.
Takeaways & Limitations
The HVM objective includes an entropy integral with respect to the variational prior that is analytically intractable in general.
Abstract
from arXiv · showhide
Black box variational inference allows researchers to easily prototype and evaluate an array of models. Recent advances allow such algorithms to scale to high dimensions. However, a central question remains: How to specify an expressive variational distribution that maintains efficient computation? To address this, we develop hierarchical variational models (HVMs). HVMs augment a variational approximation with a prior on its parameters, which allows it to capture complex structure for both discrete and continuous latent variables. The algorithm we develop is black box, can be used for any HVM, and has the same computational efficiency as the original approximation. We study HVMs on a variety of deep discrete latent variable models. HVMs generalize other expressive variational distributions and maintains higher fidelity to the posterior.
1. Introduction
BBVI offers efficient posterior approximation but mean-field factorization cannot represent posterior dependencies. HVMs place priors on variational parameters to create richer approximations while retaining BBVI's computational efficiency.
- The framework targets efficient approximation of posterior distributions in general probabilistic models, including high-dimensional settings supported by BBVI.Standard variational inference parameterizes a family over latent variables and optimizes it toward the posterior.
- Mean-field variational inference assumes independent latent variables, limiting its ability to capture posterior dependencies and marginal information.These dependencies can improve approximation fidelity and may themselves be scientifically relevant.
- HVMs expand variational families hierarchically by treating an existing variational family as a latent-variable model and placing a prior on its parameters.The resulting two-level distribution samples variational parameters and then latent variables from the corresponding conditional family.
- HVMs can capture posterior dependencies between latent variables and more complex marginal distributions.The conditional distribution of variational parameters given latent variables also participates during inference.
- The black-box algorithm fits HVM hyperparameters while maintaining the computational efficiency of BBVI.The framework supports choices such as mixture-model priors and normalizing flows, including non-differentiable models.
2. Hierarchical Variational Models
Hierarchical variational models expand mean-field approximations by placing a prior on variational parameters, inducing dependencies among latent variables and richer marginals while retaining tractable black-box inference.
- Mean-field variational inference assumes independent latent variables, which limits its ability to capture posterior dependencies and complete marginal information.
- HVMs place a variational prior on the parameters of an existing variational likelihood, creating a two-level distribution over parameters and latent variables.The prior is parameterized by variational hyperparameters θ, and marginalizing it yields qHVM(z; θ).
- The prior on variational parameters induces dependence among latent variables and expands the family of marginal distributions that can be represented.With matching marginals, the HVM is more expressive than the mean-field family.
- The framework supports black-box fitting with computational efficiency comparable to mean-field BBVI, while the demonstrated HVMs generally outperform mean-field inference.
- HVMs can be constructed using flexible variational priors, including Gaussian mixtures and normalizing flows, provided the prior does not preserve the likelihood’s factorization.Mixture components capture relationships among latent variables, while normalizing flows avoid complexity that scales with mixture count.
- HVMs extend normalizing flows to discrete, ordinal, and discontinuous latent variables, enabling improved posterior approximations where latent-variable densities are not differentiable.
3. Optimizing HVMs
HVM optimization uses a tractable hierarchical ELBO and stochastic-gradient estimators that preserve key efficiency and variance properties of the underlying variational approximation.
- Hierarchical ELBO: HVMs optimize variational-prior parameters θ using a hierarchical ELBO formed by introducing an auxiliary distribution r(λ | z; φ).The bound is jointly optimized over θ and φ; φ improves the recursive approximation while θ improves the posterior approximation.
- Hierarchical ELBO: The hierarchical ELBO decomposes into a variational-likelihood ELBO averaged under q(λ; θ) plus a correction involving r and q.The correction can improve the bound only when r depends on z rather than being independent of it.
- Stochastic gradients: Score-function gradients apply to discrete and continuous latent-variable models, but their variance generally grows with the number of factors in the learning signal.Mean-field locality can separate updates and substantially reduce this variance, making stochastic optimization practical.
- Stochastic gradients: Reparameterization expresses z as z = z(ϵ; λ), allowing variational-parameter gradients to move inside the expectation and typically reducing estimator variance.The reparameterization gradient is constructed with Monte Carlo, and its lower empirical variance is analytically related to the score-function gradient.
- Stochastic gradients: For hierarchical gradients, the original variational gradient is propagated through λ(ϵ; θ), while additional terms match the auxiliary and variational distributions.The first term inherits properties such as mean-field variance reduction; the second term is based on reparameterization and has low variance, while the third may require locality-based reduction.
- Local learning with r: An inverse-flow auxiliary distribution r can be differentiable, flexible, and local, preserving mean-field locality for efficient stochastic optimization.With local dependence in r, the resulting gradient combines mean-field computational efficiency with reparameterization for discrete and continuous models.
- Computational complexity: Normalizing-flow HVM priors have complexity linear in the number of latent variables, proportional to the flow and inverse-flow lengths.This supports flexible variational priors without complexity scaling directly with their modeling flexibility.
4. Empirical Study
The empirical study evaluates HVMs on simulated discrete posteriors and deep exponential families, finding higher-fidelity approximations and generally better held-out perplexity than mean-field methods.
- Simulated discrete posterior: HVMs model a correlated, multimodal discrete posterior with a mixture prior on Poisson rate parameters, whereas mean-field Poisson focuses on the center mass.The simulated posterior has countably infinite support and correlated latent variables.
- Deep exponential families: HVMs are evaluated on deep exponential families, including Bernoulli and Poisson models with discrete latent variables.The study uses six models for each of the NYT and Science corpora, each containing 11K documents.
- Results: HVMs outperform mean-field in five NYT models and all six Science models, while enabling multi-level Poisson models.On Science, HVMs identify multi-level Poisson models as best, whereas mean-field does not.
- Variational models: The variational models use length-2 normalizing flows for variational priors and length-10 inverse flows for the auxiliary approximation.The flows use planar transformations, and a pilot study found little improvement from longer flows.
- Evaluation: Held-out perplexity is used to evaluate generalization on data excluded from posterior inference, allowing mean-field to outperform HVMs in principle.Lower held-out perplexity is better.
- Results: 180 points mean improvement in perplexity is achieved by HVMs over six models and two datasets.Mean-field performs better only on the two-layer Bernoulli model on NYT.
5. Discussion
The discussion presents HVMs as expressive posterior approximations formed by placing priors on existing variational families, together with a general black-box fitting algorithm.
- 5. Discussion: HVMs construct rich posterior approximations by placing priors on existing variational families.The priors encode different modeling assumptions about the posterior.
- 5. Discussion: A black-box algorithm is developed that can fit any HVM.The discussion also identifies alternative entropy bounds, empirical Bayes analysis, and other data-modeling tools as future directions.
A. Appendix
The appendix develops tractable entropy bounds and gradient estimators for HVMs, including auxiliary approximations, reparameterization links, variance reduction, and convergence observations.
- Entropy bound: An auxiliary distribution r(λ | z; φ) approximates qPOST(λ | z) when the exact conditional is as difficult to compute as the entropy integral.This produces an entropy bound involving analytically known distributions.
- Entropy bound: The entropy bound is tighter than the trivial conditional entropy bound and is attained when r(λ | z; φ) equals the variational prior q(λ; θ).The bound is exact when r matches the variational posterior q(λ | z; θ).
- Gradient derivation: Localizing the inner expectation, or Rao-Blackwellizing it, can drastically reduce gradient variance.The localization uses factor-specific score functions and conditional expectations.
- Gradient derivation: The reparameterization gradient has lower variance than the score gradient because it analytically computes a definite integral whose value is zero.The appendix derives the relationship using inverse-CDF sampling and integration by parts.
- Convergence: HVMs converge faster on Poisson models than Bernoulli models, with the one-layer Poisson model being fastest to infer.The reported experiments use Nesterov accelerated gradient with momentum 0.9 and RMSProp scaling factor 10^-3 for HVMs.
- Multi-level models: Discrete variables in higher-level variational priors create high-variance learning signals because they contain the entire model.The appendix points to local expectation gradients as an efficient approach.