PaperlayerView exact revision on arXiv

Source-linked AI summary

Reweighted Wake-Sleep

Jörg Bornschein, Yoshua Bengio

arXiv:1406.2751v4cs.LG

TL;DR

Deep directed graphical models pose challenges because likelihood gradients and posterior inference are intractable. The paper interprets wake-sleep through importance sampling, introduces reweighted wake-sleep with multiple inference-network samples, and finds that stronger inference layers improve generative modeling. Experiments support better results with reweighted training and autoregressive inference networks, while exposing computational trade-offs.

  • Problem

    Training deep directed graphical models with many hidden variables is challenging because exact likelihood gradients and posterior inference are intractable.

  • Method

    Reweighted wake-sleep estimates likelihood-related gradients with multiple samples from an inference network trained to approximate latent posteriors, including autoregressive layer models.

  • Results

    Reweighted wake-sleep consistently improves over classical wake-sleep, while autoregressive inference layers improve results even with factorial SBN generative layers.

  • Takeaways & Limitations

    Using more powerful inference-network layers, such as conditional NADE or autoregressive SBNs, can produce better generative models and approach state-of-the-art performance.

  • Takeaways & Limitations

    Autoregressive inference layers reduce GPU computational efficiency because latent variables must be sampled sequentially.

Abstract

from arXiv · show

Training deep directed graphical models with many hidden variables and performing inference remains a major challenge. Helmholtz machines and deep belief networks are such models, and the wake-sleep algorithm has been proposed to train them. The wake-sleep algorithm relies on training not just the directed generative model but also a conditional generative model (the inference network) that runs backward from visible to latent, estimating the posterior distribution of latent given visible. We propose a novel interpretation of the wake-sleep algorithm which suggests that better estimators of the gradient can be obtained by sampling latent variables multiple times from the inference network. This view is based on importance sampling as an estimator of the likelihood, with the approximate inference network as a proposal distribution. This interpretation is confirmed experimentally, showing that better likelihood can be achieved with this reweighted wake-sleep procedure. Based on this interpretation, we propose that a sigmoidal belief network is not sufficiently powerful for the layers of the inference network in order to recover a good estimator of the posterior distribution of latent variables. Our experiments show that using a more powerful layer model, such as NADE, yields substantially better generative models.

1 INTRODUCTION

Training deep directed graphical models is difficult because exact likelihood gradients and posterior inference are intractable. The paper addresses this through importance sampling and argues that more expressive inference networks can improve results.

  • Exact log-likelihood gradients remain intractable for Helmholtz machines, sigmoidal belief networks, and deep belief networks.
  • Importance sampling with the inference network as proposal distribution provides an unbiased likelihood estimator, though its logarithm is biased.
  • Reweighted wake-sleep uses multiple latent samples to obtain lower-bias and lower-variance estimators of the log-likelihood gradient.
  • Wake-sleep trains both a directed generative model and an inference network that estimates latent variables given observed inputs.
  • More powerful inference-network parametrizations that represent non-factorial posterior distributions yield better results than classical sigmoidal belief network layers.

2 REWEIGHTED WAKE-SLEEP

Reweighted wake-sleep interprets wake-sleep through importance sampling, using the inference network as a proposal and multiple latent samples to improve likelihood and gradient estimation. It also trains the inference network to better approximate the posterior, with flexible parametrizations motivated by this view.

  • Wake-sleep setup: Wake-sleep trains a generative model p and an inference network q because exact posterior inference p(h|x) is intractable.The wake phase updates p using inferred latent values, while the sleep phase trains q on samples from p.
  • Importance-sampling view: Importance sampling treats q(h|x) as a proposal distribution for estimating the marginal likelihood p(x).The likelihood estimator is unbiased and consistent, with minimum variance when q(h|x) matches p(h|x).
  • Reweighted updates: Reweighted wake-sleep estimates the generative-model gradient with normalized importance weights from multiple samples h^(k) sampled from q(h|x).Unlike the original wake-sleep gradient, which corresponds to K=1, the reweighted estimator's bias and variance decrease as K increases.
  • Inference-network updates: The wake-phase q-update uses importance weights and minimizes KL(p(·|x) ∥ q(·|x)), while the sleep-phase update remains equivalent to classical wake-sleep.The paper offers both wake and sleep updates for q, with the sleep update derived from fully observed samples of p(x,h).
  • Reweighted updates: The number of inference samples K controls the computation–accuracy trade-off, with K = 100,000 typically used for test likelihoods and K = 5 for gradients.Wake and sleep updates for q can be used separately or together; the best results were obtained using both.
  • Relation to prior methods: Reweighted wake-sleep differs from variational-bound methods by directly estimating likelihood and its gradient, becoming less biased as the sample count increases.Related variational methods use a single sample to obtain an unbiased gradient estimator for a proxy variational bound.

3 COMPONENT LAYERS

The paper builds directed and autoregressive component layers for binary generative and inference networks, including SBN, AR-SBN, and conditional NADE variants.

  • 3 COMPONENT LAYERS: The models use binary visible and latent variables, with a directed generative network pθ(x,h) and layered inference network qφ(h|x).
  • 3 COMPONENT LAYERS: An SBN models conditionally independent variables xi given parent variables y, although inference of y given x is generally intractable.
  • 3 COMPONENT LAYERS: An AR-SBN adds autoregressive links from preceding variables x<i, allowing each xi to depend on both predecessors and the parent-layer input y.
  • 3 COMPONENT LAYERS: A conditional NADE mediates dependencies among variables through an accumulating hidden layer and an MLP, while conditioning on parent variables y.
  • 3 COMPONENT LAYERS: Unconditioned layer distributions are used as the top-layer distribution p(h) of the generative network.

4 EXPERIMENTS

Experiments on MNIST and CalTech 101 Silhouettes evaluate sample-count effects and compare SBN, autoregressive, and NADE architectures. Reweighted wake-sleep generally improves over classical wake-sleep, with especially strong results from powerful inference layers and on deeper models.

  • 4 EXPERIMENTS: MNIST and CalTech 101 Silhouettes experiments evaluate sample count, inference-layer power, and performance across deep and conditional-NADE models.
  • 4 EXPERIMENTS: Combining wake- and sleep-phase q-updates generally gives the best results, so the remaining models use combined updates.
  • 4 EXPERIMENTS: More than 5 training samples generally adds little estimated log-likelihood improvement for NADE models, while SBN-layer models typically require more than 25 samples before gains level off.
  • 4 EXPERIMENTS: Reweighted wake-sleep models consistently improve over classical wake-sleep, especially for deep architectures, while autoregressive inference layers help even with factorial SBN generative models.
  • 4 EXPERIMENTS: The best MNIST autoregressive-inference models are shallow, using only a single hidden layer; the comparable DARN model shows no improvement from reweighted wake-sleep.
  • 4 EXPERIMENTS: On CalTech 101 Silhouettes, the best SBN/SBN model reaches an estimated test LL of -116.9, while shallow NADE/NADE-150 reaches -104.3 versus −107.8 for prior state of the art.

5 CONCLUSIONS

The paper introduces reweighted wake-sleep for deep generative models, using multiple inference-network samples to improve gradient estimation. More powerful autoregressive inference layers improve results, while autoregressive generative layers perform well but remain shallow in the reported experiments.

  • 5 CONCLUSIONS: Reweighted wake-sleep provides a lower-bias, lower-variance log-likelihood-gradient estimator than wake-sleep, at the cost of drawing more inference-network samples.The method uses weighted samples to decouple layers, so gradients propagate within individual layers.
  • 5 CONCLUSIONS: Approximately 5 samples typically suffice to train architectures with at least 5 hidden layers without layerwise pretraining or carefully tuned learning rates.The resulting models produce reasonable samples and approach state-of-the-art log-likelihood performance on several discrete datasets.
  • 5 CONCLUSIONS: More powerful autoregressive inference layers yield better results than classical SBN inference layers, including when the generative network contains only SBN layers.Autoregressive sampling reduces GPU efficiency because latent variables must be sampled sequentially, despite similar theoretical complexity.
  • 5 CONCLUSIONS: Generative networks with autoregressive layers typically perform very well, but the best reported models are shallow with one hidden layer.The paper leaves open whether this shallowness reflects optimization problems.

6 SUPPLEMENT

The supplement reports learning-curve, estimator bias/variance, UCI-dataset, and hyperparameter experiments. It includes MNIST learning curves, bootstrapping analysis of estimated log(p(x)), and comparisons involving SBN and NADE models.

  • 6.3.1 LEARNING CURVES FOR MNIST EXPERIMENTS: Figure 4 presents learning curves for various MNIST experiments.
  • 6.3.2 BOOTSTRAPPING BASED log(p(x)) BIAS/VARIANCE ANALYSIS: Figure 5 presents the bias and standard deviation of low-sample estimated log(p(x)) using bootstrapping with K=5,000 primary samples.The samples come from an SBN/SBN 10-200-200 network trained on MNIST.
  • 6.3.3 UCI BINARY DATASETS: The supplement reports experiments on 8 different binary datasets from the UCI database.
  • 6.3.3 UCI BINARY DATASETS: The UCI experiments screen learning rates of 0.001, 0.003, and 0.01, with specified SBN and NADE architecture grids and K=10 or K=5 training samples.NADE/NADE models use one hidden layer with 30, 50, 75, 100, or 200 units.
  • 6.3.3 UCI BINARY DATASETS: Table 3 reports results on binary UCI datasets and identifies baseline results from Larochelle & Murray (2011) and Uria, Murray, Larochelle (2014).The caption specifies NADE∗ as 500 hidden units and EoNADE+ as 1hl, 16 ord.
Loading 1406.2751v4…