PaperlayerView exact revision on arXiv

Source-linked AI summary

Tighter Variational Bounds are Not Necessarily Better

Tom Rainforth, Adam R. Kosiorek, Tuan Anh Le, Chris J. Maddison, Maximilian Igl, Frank Wood, Yee Whye Teh

arXiv:1802.04537v3stat.MLcs.LG

TL;DR

The paper asks whether tighter ELBOs always improve simultaneous generative-model and inference-network learning, then analyzes gradient SNR and introduces alternative importance-weighted estimators. It finds that tighter bounds can reduce inference-network SNR, while PIWAE, MIWAE, and CIWAE improve over IWAE, with PIWAE potentially improving both networks.

  • Problem

    The paper examines the assumption that tighter ELBOs are universally beneficial for simultaneous generative-model learning and inference amortization.

  • Method

    The authors provide theoretical and empirical SNR analysis and introduce PIWAE, MIWAE, and CIWAE as alternative importance-weighted estimators.

  • Results

    PIWAE, MIWAE, and CIWAE improve over IWAE even when performance is measured by the IWAE target itself.

  • Takeaways & Limitations

    PIWAE may simultaneously improve inference-network and generative-network training through separate targets for the two networks.

  • Takeaways & Limitations

    The convergence results are exact in M and N but only asymptotic in K, so their applicability in the small-K regime requires caution.

Abstract

from arXiv · show

We provide theoretical and empirical evidence that using tighter evidence lower bounds (ELBOs) can be detrimental to the process of learning an inference network by reducing the signal-to-noise ratio of the gradient estimator. Our results call into question common implicit assumptions that tighter ELBOs are better variational objectives for simultaneous model learning and inference amortization schemes. Based on our insights, we introduce three new algorithms: the partially importance weighted auto-encoder (PIWAE), the multiply importance weighted auto-encoder (MIWAE), and the combination importance weighted auto-encoder (CIWAE), each of which includes the standard importance weighted auto-encoder (IWAE) as a special case. We show that each can deliver improvements over IWAE, even when performance is measured by the IWAE target itself. Furthermore, our results suggest that PIWAE may be able to deliver simultaneous improvements in the training of both the inference and generative networks.

1 Introduction

The paper challenges the assumption that tighter ELBOs are universally beneficial: they can improve generative-network updates while harming inference-network learning through lower gradient SNR. It introduces PIWAE, MIWAE, and CIWAE, which retain IWAE as a special case and improve over it, with PIWAE potentially improving both networks.

  • Motivation: Tighter ELBOs can benefit generative-network updates while detrimentally reducing the inference network’s gradient signal-to-noise ratio.The paper reports that bounds can be tightened, gradient variance reduced, and inference-network training nevertheless arbitrarily deteriorated.
  • Contributions: PIWAE, MIWAE, and CIWAE modify importance-weight usage to provide higher inference-network SNR while including IWAE as a special case.All three use the same importance weights but combine them differently.
  • Results: The new algorithms can produce inference networks closer to the true posterior while matching generative-network training, with PIWAE potentially improving it.The reported generative-network improvement is specific to PIWAE’s potential behavior.
  • Results: All three algorithms improve over IWAE even when evaluated using the IWAE objective itself.This makes the comparison favorable even under IWAE’s own performance target.

2 Background and Notation

The background introduces variational lower bounds and IWAE as tools for learning deep generative models with approximate inference. IWAE generalizes VAE, becomes tighter with more importance samples, and is empirically associated with better generative models.

  • Variational objectives: Variational inference replaces intractable model evidence with an optimizable lower bound using an auxiliary inference model qφ(z|x).The inference model supplies approximate posterior samples during optimization.
  • Variational objectives: In a VAE, qφ is typically parameterized by a neural network, while joint optimization can confound marginal-likelihood improvement with reductions in posterior mismatch.The mismatch is expressed through KL divergence between qφ(z|x) and pθ(z|x).
  • IWAE: IWAE objectives generalize VAE, with K = 1 corresponding to VAE and larger K producing strictly tighter bounds.The bound’s tightness changes with K, unlike the separate Monte Carlo averaging parameter M.
  • IWAE: When qφ contains the true posterior, globally optimal parameters are independent of K; otherwise, increasing K is empirically associated with better generative models.The reported generative-model gains concern marginal likelihood on held-out data.
  • Related approaches: Related approaches tighten bounds through sequential Monte Carlo, bias reduction, or more expressive inference models.These approaches alter the sampling procedure, objective, or expressiveness of qφ.

3 Assessing the Signal-to-Noise Ratio of the Gradient Estimators

The paper studies how Monte Carlo sample counts affect gradient-estimator SNR for generative and inference networks. Increasing M and data-batch size improves SNR, whereas increasing K helps θ but can worsen φ, creating a trade-off despite tighter bounds.

  • Estimator design: The estimators average M estimates built from K independent particles, with K = 1 yielding VAE and M = 1 yielding IWAE under a fixed sample budget.The total budget is T = MK samples.
  • SNR: SNR measures expected gradient magnitude relative to standard deviation, so low SNR indicates estimates dominated by noise.The paper defines SNR separately for each parameter dimension of θ and φ.
  • Theoretical result: Increasing M is beneficial for both networks, whereas increasing K benefits the generative network but harms the inference network’s SNR.For φ, increasing K worsens SNR rather than merely foregoing the benefit of increasing M.
  • Asymptotic direction: As K increases, the inference-network gradient can approach the direction that minimizes importance-weight variance, while its estimator SNR deteriorates.This creates a trade-off between a preferable asymptotic target direction and increasingly noisy updates.
  • Multiple data points: Increasing the number of data points N has the same SNR benefit as increasing M.The multiple-data-point analysis extends the convergence results to independently sampled observations.

4 Empirical Confirmation

Experiments on a synthetic Gaussian model support the theoretical analysis of how particle count and estimator averaging affect gradient signal-to-noise ratios. The results show qualitatively different behavior for generative and inference-network gradients as importance-sampling particles increase.

  • Experimental setup: The convergence results are tested in a Gaussian model because their dependence on M is exact but their dependence on K is only asymptotic.The authors explicitly caution that higher-order terms limit applicability in the small-K regime.
  • Experimental setup: The synthetic experiment uses D = 20 dimensions, N = 1024 data points, and a randomly generated unit-Gaussian true model parameter.The generative model has z ∼ N(µ, I) and x|z ∼ N(z, I), with θ := µ.
  • Experimental setup: Empirical gradient estimates use 10^4 samples to compare IWAE with M = 1 and increasing K against VAE with K = 1 and increasing M.The samples estimate the gradients’ means, standard deviations, and signal-to-noise ratios.
  • Generative-network gradients: For the generative network, increasing K makes estimates closer to the true gradient of the optimized target, despite related bias and SNR behavior.The passage distinguishes improving optimization of the generative objective from worsening inference-network training.
  • Inference-network gradients: Increasing K can reduce inference-network SNR even when the expected gradient direction improves, and the experiments identify SNR as the dominant effect in this problem.The authors compare directional accuracy against a high-K target and find that increasing K remains detrimental for the inference network.
  • Directional analysis: The directional SNR shows similar qualitative behavior to the SNR, with IWAE inference gradients degrading to the directional accuracy of a random vector.The observed DSNR convergence in M follows the same asymptotic pattern for both networks, although no theoretical result establishes this.

5 New Estimators

The paper introduces MIWAE, CIWAE, and PIWAE to address diminishing inference-network SNR while retaining tighter-bound benefits for generative modeling. Experiments on MNIST show improvements over IWAE across evaluation metrics, with PIWAE improving both generative and inference-network performance.

  • MIWAE: MIWAE uses M > 1 and K > 1, mitigating the inference-network SNR problem by averaging multiple importance-weighted estimates.Under a fixed sample budget T = MK, increasing M can offset the detrimental effect of increasing K on inference-network SNR.
  • CIWAE: CIWAE combines VAE and IWAE objectives, producing a bound tighter than VAE but looser than IWAE.Its VAE component can take over when the expected IWAE gradient becomes very small, alleviating asymptotic SNR issues.
  • PIWAE: PIWAE uses the IWAE target for the generative network and the MIWAE target for the inference network.This separates the objectives to accommodate the different effects of tighter bounds on the two networks.
  • MNIST experiments: PIWAE achieved the best log p̂(x) performance, while MIWAE improved the KL metric relative to IWAE and PIWAE improved both metrics simultaneously.The KL metric is an estimator of divergence between the inference and proposal networks, though it also depends on the learned generative model.
  • MNIST experiments: CIWAE, MIWAE, and VAE had significantly larger ESS than IWAE and PIWAE, while IWAE exhibited particularly poor ESS and low inference-network SNR during training.ESS measures the effective number of unweighted samples represented by the weighted sample set; Figure 8 shows that the low-SNR behavior extends to neural networks.

6 Conclusions

The paper concludes that tighter ELBOs can reduce the inference network’s gradient signal-to-noise ratio, motivating PIWAE, MIWAE, and CIWAE as alternatives to IWAE.

  • Tightening the ELBO can harm inference-network learning by reducing gradient signal-to-noise, even when the generative network benefits.
  • PIWAE, MIWAE, and CIWAE each improve inference-network training over IWAE without reducing learned generative-network quality.
  • PIWAE appears able to improve both networks simultaneously, potentially through separate targets for inference and generative networks.

A Proof of SNR Convergence Rates

The proof derives asymptotic convergence rates for gradient-estimator signal-to-noise ratios under moment, independence, and finite-variance assumptions.

  • The lemma relies on i.i.d. random-variable sets, zero means, and conditional mutual independence of off-diagonal index combinations.
  • The derivation reduces higher-order estimator terms using vanishing off-diagonal expectations and substitutions into the lemma’s moment identities.
  • Theorem 1 assumes finite expected gradients, gradient variances, and first four moments, with non-zero gradient variances at M = K = 1.
  • The proof analyzes estimator variance by exploiting i.i.d. structure and applying Taylor expansions around the true marginal likelihood Z.

B Derivation of Optimal Parameters for Gaussian Experiment

The Gaussian experiment derives optimal parameters by minimizing the KL divergence between the proposal and posterior means, then optimizing the generative parameters.

  • Because φ controls only the proposal mean, KL minimization matches the proposal and posterior means for each θ.
  • The minimum achievable KL is independent of θ, allowing θ and then φ to be optimized sequentially.
  • For the stated Gaussian parameterization, the optimum is A* = I/2 and b* = μ*/2.

C Additional Empirical Analysis of SNR

Additional experiments examine how M and K affect gradient distributions and generative-network RMSE, showing variance reduction from M and monotonic RMSE improvement from K.

  • Increasing M for K = 1 decreases gradient-estimate variance while leaving the expected value unchanged.
  • For the generative network, RMSE is more informative than SNR because high SNR does not necessarily indicate good performance.
  • Increasing M in the VAE has negligible RMSE effect because variance falls while fixed bias remains.
  • Increasing K produces monotonic RMSE improvement, initially at O(1/K) before reaching the standard Monte Carlo rate O(1/√K).

C.3 Experimental Results for High Variance Regime

In the high-variance regime, increasing K still deteriorates inference-network gradient signal-to-noise, though the decline is slower and the asymptotic theory does not directly apply.

  • High-variance setting: Higher-variance weights arise when the model is far from its optimum and the proposal is a poor match.The experiment offsets parameters by standard deviation 0.5 and uses proposal covariance I.
  • Effect of K and M: The behavior is similar when varying M but differs somewhat when varying K.The distinction is reflected in the slower inference-network decline as K increases.
  • Effect of K: SNR and DSNR decrease slowly with K for the inference network, while increasing K provides little additional SNR benefit.The observed deterioration is weaker than the theoretical asymptotic prediction in this setting.
  • Interpretation: The experiment is far from the asymptotic regime in K, so the theoretical results do not directly apply.Nevertheless, the high-level inference-network SNR deterioration remains visible.
  • Diagnostics: The supplementary figures track gradient-estimate histograms and convergence of ordinary and directional signal-to-noise ratios.These visualizations cover gradient distributions and SNR behavior across the tested settings.

D Convergence of Deep Generative Model for Alternative Parameter Settings

Under tighter alternative parameter settings, PIWAE, MIWAE, and CIWAE retain IWAE-level generative performance while improving the inference-network KL metric in the reported comparisons.

  • Alternative settings: MIWAE and CIWAE match IWAE's generative-model performance while improving the KL metric for the inference network.For both methods, log p̂(x) is increased to IWAE's level while −KL(Qφ(z|x)||Pθ(z|x)) is reduced.
  • Alternative settings: PIWAE shows similar mean behavior across metrics, but its log p̂(x) variance is noticeably increased.The alternative setting uses M = 4 and K = 16 for PIWAE and MIWAE, and β = 0.05 for CIWAE.
  • Experimental setup: The tested settings represent tighter bounds than those used in the main paper.Convergence is evaluated using the IWAE-64 metric and additional generative- and inference-network metrics.
  • Evaluation: Figure 15 plots convergence of different evaluation metrics for each method using the plotting conventions of Figure 5.The figure supports comparison of metric trajectories across the alternative algorithm settings.

E Convergence of Toy Gaussian Problem

Toy Gaussian optimization experiments examine how gradient-estimate quality transfers to joint parameter optimization, with K = 1 and M = 1000 reported as especially effective.

  • Optimization: K = 1 and M = 1000 gave the best performance for learning both inference-network and generative-network parameters.The result comes from running Adam to optimize μ, A, and b in the toy Gaussian problem.
  • Optimization: The observed gradient-estimate effects predominantly transfer to the overall optimization problem.Figure 16 shows convergence of Adam across the toy problem's model and inference parameters.
  • Figure 16: Figure 16 compares convergence for different K and M values using means over three repeats with ±1 standard deviation.The top-left panel shows ELBO_IS, while the other panels show L2 distances for generative and inference parameters.
Loading 1802.04537v3…