Source-linked AI summary
Neural Variational Inference and Learning in Belief Networks
Andriy Mnih, Karol Gregor
TL;DR
Highly expressive directed latent variable models are hard to train on large datasets because inference is intractable and existing approximate methods do not scale well. NVIL uses a feedforward inference network for efficient variational-posterior sampling, jointly optimizes a variational lower bound, and applies variance reduction; it outperforms wake-sleep on sigmoid belief networks and achieves state-of-the-art Reuters RCV1 results.
Problem
Highly expressive directed latent variable models are difficult to train on large datasets because exact inference is intractable and existing approximate inference methods do not scale well.
Method
NVIL uses a feedforward inference network to sample efficiently from the variational posterior and jointly trains it with the model by maximizing a variational lower bound.
Results
NVIL consistently outperforms wake-sleep on sigmoid-belief-network-like models and achieves state-of-the-art results on the Reuters RCV1 document dataset.
Takeaways & Limitations
NVIL provides a general, easy-to-apply training method for intractable directed latent variable models that is effective across sigmoid belief-network-like models and sizable document data.
Takeaways & Limitations
Reuters evaluation reports an upper bound on perplexity because the exact document log-likelihood cannot be computed.
Abstract
from arXiv · showhide
Highly expressive directed latent variable models, such as sigmoid belief networks, are difficult to train on large datasets because exact inference in them is intractable and none of the approximate inference methods that have been applied to them scale well. We propose a fast non-iterative approximate inference method that uses a feedforward network to implement efficient exact sampling from the variational posterior. The model and this inference network are trained jointly by maximizing a variational lower bound on the log-likelihood. Although the naive estimator of the inference model gradient is too high-variance to be useful, we make it practical by applying several straightforward model-independent variance reduction techniques. Applying our approach to training sigmoid belief networks and deep autoregressive networks, we show that it outperforms the wake-sleep algorithm on MNIST and achieves state-of-the-art results on the Reuters RCV1 document dataset.
1. Introduction
Highly expressive directed latent variable models are difficult to train at scale because inference is intractable and existing approximations do not scale well. NVIL addresses this with feedforward variational sampling, joint lower-bound optimization, and model-independent variance reduction.
- Motivation: Directed latent variable models lag behind globally normalized alternatives because efficient learning algorithms are lacking.Their modularity and efficient generation nevertheless make them suitable for integration into larger systems.
- Motivation: MCMC is broadly applicable but suffers from slow mixing, high computational cost, and poor scalability to large datasets.It also requires storing latent states for all training observations between parameter updates.
- Motivation: Variational methods improve efficiency by optimizing a lower bound with tractable approximate posteriors, but analytical expectations are limited to a small model class.Highly expressive models remain difficult when those expectations cannot be computed analytically.
- NVIL: NVIL uses a feedforward network to sample efficiently from the variational posterior and trains the model and inference network jointly by maximizing a variational lower bound.Required gradients are estimated using samples from the inference network.
- NVIL: Each NVIL forward pass produces an independent exact sample from the variational posterior, avoiding MCMC mixing issues and reducing memory requirements.This also supports pure online learning, where each training case is seen once before being discarded.
- NVIL: NVIL handles discrete and continuous latent variables, complex posterior dependencies, and simple model-independent variance reduction.These properties distinguish it from several contemporaneous variational-inference approaches.
- NVIL: Variance reduction makes the otherwise unusable inference-network gradient practical without narrowing the method’s applicability.The resulting training procedure is an instance of REINFORCE and is named Neural Variational Inference and Learning.
- Results: NVIL trains sigmoid belief networks better than wake-sleep and achieves state-of-the-art results on the Reuters RCV1 document dataset.The authors present these results as evidence that the method is effective and scalable.
2. Neural variational inference and learning
The approach uses a feedforward inference network to define and sample from a variational posterior, jointly optimizing it with the model through a variational lower bound. Monte Carlo gradient estimates are made practical with model-independent variance reduction, including baselines, normalization, and local learning signals.
- Variational objective: The method assumes intractable exact inference and optimizes a variational lower bound using a simpler posterior distribution Qφ(h|x).The bound is maximized with respect to both model and inference-network parameters.
- Inference network: A feedforward inference network maps each observation x to Qφ(h|x), avoiding local variational parameters while supporting efficient evaluation and sampling.Its architecture is constrained only by the requirement that the resulting posterior be efficient to evaluate and sample from.
- Gradient estimation: Monte Carlo samples from the inference network estimate otherwise intractable gradients for both the model and inference-network parameters.The estimates are unbiased, but convergence depends heavily on their variance.
- Variance reduction: The inference-network gradient can have very high variance, making naive updates unacceptably slow despite the model-gradient estimator being well behaved.The method addresses this limitation with model-independent variance reduction techniques.
- Variance reduction: Constant and observation-dependent baselines reduce variance without changing the expected gradient, while standard-deviation normalization stabilizes the learning signal.The observation-dependent baseline is implemented with a neural network trained on the squared centered learning signal.
- Local learning signals: Conditional-independence properties allow layer-specific local learning signals that remove terms irrelevant to the corresponding gradient expectation.For hidden layers i > 1, these signals contain fewer terms than the global signal and can therefore be less noisy.
3. Related work
NVIL is situated among feedforward and sampling-based approaches to approximate inference, differing in posterior expressiveness, training objective, and scope. Its closest comparison is wake-sleep, which shares scalability but lacks a single optimized objective.
- Feedforward inference: Earlier feedforward inference methods either relied on analytically tractable models or initialized iterative mean-field inference, inheriting its posterior limitations.NVIL instead trains its inference network to match the true posterior directly, with fit accuracy limited by the network’s expressiveness.
- Related variational methods: SGVB also uses feedforward approximate inference and sampling-based variational-bound gradients, but is considerably less general than NVIL.
- Related variational methods: BBVI applies sampling-based variational optimization broadly, but learns separate variational parameters per observation and uses a fully factorized posterior.These choices distinguish it from NVIL’s inference network and limit BBVI’s posterior power.
- Wake-sleep: NVIL and wake-sleep both use recognition or inference networks and scale across broad model classes, but only NVIL optimizes a variational lower bound on log-likelihood.Wake-sleep’s recognition-network update optimizes a different objective from its model update, so it lacks a well-defined objective and convergence guarantee.
- REINFORCE: NVIL’s inference-network gradient can be viewed as REINFORCE applied per training case, with latent states as outputs and the learning signal as reward.Baselines and related variance-reduction methods serve to reduce update variance.
4. Experimental results
Experiments show that NVIL’s variance reduction enables effective training of sigmoid belief networks and supports expressive inference networks and document models. NVIL outperforms wake-sleep on MNIST and achieves strong results on Reuters RCV1, though reported likelihoods are variational-bound estimates.
- Variance reduction: All three variance reduction techniques were essential for best validation-bound performance in one- and two-layer SBNs, while disabling them made learning infeasible.The input-dependent baseline appeared least important individually, and variance reduction mattered more for larger models.
- Variance reduction: Disabling layer-specific learning signals changed performance by only about 0.4 nats for SBNs with two or three latent-variable layers.The authors found little performance effect from removing these signals.
- Comparison with wake-sleep: 3.4 to 8.6 nats: NVIL-trained models achieved better log-likelihood bounds than their wake-sleep counterparts across the tested model range.Additional layers improved SBN performance under both training methods.
- Inference-network structure: Autoregressive inference networks produced better MNIST models than factorial inference networks, with especially large gains for single-layer models.Within-layer dependencies made the inference networks more expressive.
- Document modelling: 598: fDARN with 200 hidden units set a new record on Reuters RCV1, improving on the prior DocNADE score of 742.The 50-latent-variable fDARN score of 724 was already better than DocNADE’s 742.
5. Discussion and future work
NVIL is presented as a general, easy-to-apply training method for intractable directed latent variable models. The authors report consistent advantages over wake-sleep and state-of-the-art document results, while identifying more expressive and continuous-latent extensions as future work.
- NVIL is a general and easy-to-apply training method for intractable directed latent variable models.
- NVIL consistently outperforms wake-sleep when training sigmoid-belief-network-like models.
- NVIL achieves state-of-the-art results on the sizable Reuters RCV1 document dataset.
- The experiments use simple model and inference-network architectures, leaving potential performance gains from more expressive architectures.
- Applying NVIL to models with continuous latent variables is identified as another promising direction because binary latent variables are not always adequate.
- NVIL is expected to extend to conditional latent-variable models by conditioning the inference network on both context and observation.
A. Algorithm for computing NVIL gradients
Algorithm 1 computes NVIL gradients over a minibatch by sampling latent states, forming learning signals, normalizing their statistics, and accumulating model and inference-network updates.
- Algorithm 1 computes NVIL gradient estimates for a minibatch of n randomly chosen training cases.
- It updates exponentially smoothed estimates of the learning-signal mean and variance using α = 0.8 in the experiments.
- For each training case, the inference model samples h_i ∼ Q_φ(h_i|x_i) and computes l_i = log P_θ(x_i,h_i) − log Q_φ(h_i|x_i).
- The algorithm adds each learning signal to the variational bound and subtracts an input-dependent baseline C_ψ(x_i).
- The procedure accumulates gradients for the model parameters from ∇_θ log P_θ(x_i,h_i) and separately accumulates inference-network gradients.
B. Inference network gradient derivation
The inference-network gradient is rewritten as an expectation under the variational posterior, yielding a learning-signal-weighted score-function estimator.
- Differentiating the variational lower bound with respect to inference-network parameters produces the gradient expression used by NVIL.
- The derivation uses the normalization identity ∇_φ ∫ Q_φ(h|x)dh = ∇_φ 1 = 0.
- Using ∇_φ Q_φ(h|x) = Q_φ(h|x)∇_φ log Q_φ(h|x) converts the derivative into an expectation under Q_φ.
- The resulting estimator weights ∇_φ log Q_φ(h|x) by log P_θ(x,h) − log Q_φ(h|x).