PaperlayerView exact revision on arXiv

Source-linked AI summary

Rényi Divergence Variational Inference

Yingzhen Li, Richard E. Turner

arXiv:1602.02311v3stat.MLcs.LG

TL;DR

Choosing a divergence that yields an effective approximation remains an open question in variational inference. This paper introduces the variational Rényi bound and an optimization framework spanning existing methods, with experiments indicating broad applicability and competitive results, while requiring assumptions and application-specific choices of α.

  • Problem

    Selecting a divergence from Rényi’s rich family to obtain an optimal solution for a particular application remains an active research question.

  • Method

    The paper extends variational inference to Rényi’s α-divergence, introduces the VR bound and VR-max, and develops reparameterization, Monte Carlo, and stochastic optimization procedures.

  • Results

    VR-bound methods are widely applicable and achieve state-of-the-art results on Bayesian neural networks and variational auto-encoders; VR-max is often comparable to or better than existing variational methods.

  • Takeaways & Limitations

    The framework unifies several variational approaches, but practitioners need guidance for choosing α because the best value is dataset-specific.

  • Takeaways & Limitations

    Mean-field variational inference can produce over-confident posterior approximations and bias hyper-parameter optimization away from the exact maximum likelihood estimate.

Abstract

from arXiv · show

This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is controlled by the value of alpha that parametrises the divergence. The reparameterization trick, Monte Carlo approximation and stochastic optimisation methods are deployed to obtain a tractable and unified framework for optimisation. We further consider negative alpha values and propose a novel variational inference method as a new special case in the proposed framework. Experiments on Bayesian neural networks and variational auto-encoders demonstrate the wide applicability of the VR bound.

1 Introduction

Approximate inference is central to probabilistic machine learning, motivating scalable, Monte Carlo-assisted, and unified variational methods. This paper extends VI to Rényi’s α-divergence and develops VR methods connecting existing approaches.

  • Approximate inference algorithms estimate posterior distributions and likelihood functions in modern probabilistic machine learning.
  • Scalable variational and expectation-propagation methods address datasets comprising millions of datapoints and computations distributed across data chunks.
  • The paper extends traditional VI to Rényi’s α-divergence within a unified energy-function framework encompassing recent variational advances.
  • The variational Rényi bound connects VI/VB, VAE, IWAE, SEP, and black-box α as special cases or related approaches.
  • VR-max is proposed as a new special case, with evaluations on VAEs and Bayesian neural networks showing results often comparable to or better than state-of-the-art variational methods.

2 Background

Variational inference replaces an intractable posterior with a tractable approximation optimized through a variational lower bound. Rényi divergences provide a broader family, but selecting an appropriate divergence remains an open question and VI can bias hyper-parameter optimization.

  • Rényi’s α-divergence recovers KL divergence as α →1 and can be extended by continuity to α = 0, 1, and +∞.
  • Choosing a divergence that yields an optimal solution for a particular application remains an active research question.
  • Approximate inference introduces a tractable q(θ) to approximate an often-intractable posterior p(θ|D) in a chosen distribution family.
  • The variational lower bound can also optimize model hyper-parameters ϕ.
  • In Bayesian linear regression, mean-field VI recovers the posterior mean but produces over-confident uncertainty and can bias hyper-parameter optimization away from the exact MLE.

3 Variational Rényi bound

The VR bound reformulates Rényi-divergence posterior approximation as a family of objectives indexed by α. Its monotonicity and limiting behavior support interpolation across approximation regimes, including negative-α methods.

  • The VR bound is defined by rewriting Rényi-divergence posterior optimization as an objective over q.
  • The VR objective is continuous and non-increasing in α wherever it is finite.
  • L0(q; D) equals log p(D) when the exact posterior’s support is contained within q’s support.
  • Different α values balance zero-forcing or mode-seeking behavior against mass-covering behavior.
  • In Bayesian linear regression, α = 0 recovers the exact marginal likelihood, while α = 0.5 yields a less hyper-parameter-dependent bound and less biased approximate MLE.

4 The VR bound optimisation framework

The framework makes the VR bound tractable through Monte Carlo approximation, reparameterization, and stochastic optimisation, while characterising approximation bias and connecting global and local methods. It also supports large-scale posterior inference through two mini-batch approximations that recover SEP and BB-α.

  • Monte Carlo approximation: Monte Carlo approximation extends VR optimisation to models handled by MC-VI, but introduces bias that can be characterised theoretically and empirically.The exact VR bound is often intractable, motivating stochastic approximation for a broader model class.
  • Monte Carlo approximation: For fixed α, the expected MC estimator is non-decreasing in K when α ≤1 and non-increasing when α ≥1, converging to Lα as K increases.The expectation is also continuous and non-increasing in α for fixed K.
  • Monte Carlo approximation: Negative α values can improve finite-sample approximation accuracy by sacrificing the exact VR bound’s upper-bound guarantee.The MC approximation is biased toward LVI, and its quality improves with more samples.
  • Unified implementation: The reparameterization trick expresses θ as gφ(ϵ), converting expectations under q(θ) into expectations over noise and unifying stochastic-gradient computation across finite α.At α = 1, the normalised weights become uniform and recover stochastic-gradient VI.
  • Unified implementation: VR-max is obtained in the α →−∞ limit by back-propagating through the sample with the maximum unnormalised importance weight.For finite α, samples are selected according to weights proportional to ˆw(ϵk; x)^(1−α).
  • Large-scale learning: For large datasets, fixed-point and energy mini-batch approximations make VR training practical and recover SEP and BB-α, respectively.These methods use subset average likelihoods, while factor tying reduces memory overhead and supports scalability.
  • Large-scale learning: MC methods can be applied to both mini-batch proposals, although choosing α for a given mini-batch size M and sample count K remains open.Intermediate α values have evidence of being superior in some settings.

5 Experiments

The experiments evaluate VR-bound methods on Bayesian neural networks and variational auto-encoders, revealing dataset- and task-dependent effects of α. VR-max is competitive with IWAE and can substantially reduce CPU training time.

  • 5.1 Bayesian neural network: Bayesian neural-network experiments show that the optimal α varies across datasets and that Monte Carlo approximation complicates α selection.The authors suggest validation-based or future automated selection of α values.
  • 5.1 Bayesian neural network: Mode-seeking methods tend to improve predictive error, whereas mass-covering methods tend to provide better-calibrated uncertainty and test log-likelihood.VI returns lower test log-likelihood for most datasets, while α = 0.5 gives overall good LL and RMSE results.
  • 5.2 Variational auto-encoder: VAE experiments compare VAE at α = 1.0, IWAE at α = 0, and VR-max at α = −∞ across four datasets.The evaluation uses Frey Face, Caltech 101 Silhouettes, MNIST, and OMNIGLOT.
  • 5.2 Variational auto-encoder: VR-max is almost indistinguishable from IWAE on the reported VAE test log-likelihood results.Positive α settings such as α = 0.5 produce worse results on some datasets, while small negative α values perform better on different splits.
  • 5.2 Variational auto-encoder: The estimated VR-bound gap narrows as α approaches −∞ and improves with more samples, while standard-error variation is negligible relative to Monte Carlo bias.Figure 5 evaluates K = 5 and K = 50 samples on 100 Frey Face test datapoints.
  • 5.2 Variational auto-encoder: The largest importance weight can occupy more than 75% of the probability mass, enabling VR-max to approximate IWAE up to 3 times faster on CPUs.The reported Frey Face comparison is 9.7s versus 29.0s per epoch with K = 50; the advantage is smaller on efficient single GPUs.

6 Conclusion

The paper presents the variational Rényi bound and an associated optimisation framework as a unified family connecting multiple variational methods. Experiments indicate broad applicability, while selecting suitable α values remains an open practical and theoretical issue.

  • 6 Conclusion: The paper introduces the variational Rényi bound and connects it to VI/VB, SEP, BB-α, VAE, and IWAE.It also proposes VR-max as a new special case of the framework.
  • 6 Conclusion: The framework is evaluated on Bayesian neural networks and variational auto-encoders, where VR-bound methods obtain state-of-the-art results.The conclusion describes these results as evidence of wide applicability.
  • 6 Conclusion: Future work will study the interaction between Monte Carlo and datapoint-subsampling biases and develop guidance for choosing optimal α values.The paper identifies α selection as an unresolved issue for practitioners.

B A mean-field approximation example

This appendix develops a mean-field approximation for the VR-bound family using Bayesian linear regression. The resulting updates have a unique stable fixed point and recover familiar limiting behaviours as α changes.

  • Mean-field formulation: The mean-field method factorises q(θ) over the components of θ and rewrites VR-bound optimisation through marginal-like distributions.For α > 0 the bound is maximised, while for α < 0 the corresponding bound is minimised.
  • Mean-field formulation: Optimising each factor is equivalent to minimising an α-divergence to its marginal-like target, recovering traditional mean-field VI as α approaches 1.As α approaches 0, the factor approaches the exact posterior marginal.
  • Bayesian linear regression: In the two-dimensional Bayesian linear-regression example, the iterative mean-field update has m = µ as its only stable fixed point.The associated precision parameters remain feasible for all α in the example.
  • Bayesian linear regression: The stable fixed point is unique because the alternative quadratic solution violates positivity or finiteness assumptions in the relevant α ranges.The constraints are λ1 > 0 for 0 < α < 1 and |Lα| < +∞ outside that interval.
  • Behaviour across α: Increasing α continuously interpolates between mass-covering and zero-forcing behaviour, while α → +∞ remains uncertain but more over-confident than VI.This differs from MAP, which returns only a point estimate.
  • Monte Carlo properties: The Monte Carlo estimator converges to the exact VR bound as K approaches infinity under absolute-integrability conditions.The convergence result is established across admissible α values, with the α = 1 limit recovering the VI bound.
  • Monte Carlo properties: For fixed K, the expected Monte Carlo bound is continuous and non-increasing in α under the stated integrability assumptions.The proof uses monotonicity and dominated convergence.

C.2 Proof of Corollary 1

This section analyses the sample size needed for negative-α Monte Carlo bounds under support and integrability assumptions. It establishes monotonicity properties and limiting behaviour for the threshold Kα.

  • Assumptions: The analysis assumes the support of p is contained in the support of q and considers regions where q is positive but p is zero.The measure of this excess support is denoted ρ.
  • Assumptions: When ρ > 0, finite-sample negative-α estimates can become non-integrable, motivating an absolute-integrability assumption for subsequent results.The paper identifies this as a pathology of Monte Carlo approximation.
  • Threshold behaviour: Under absolute integrability and Lα > LVI, there is a finite threshold Kα after which the expected estimate approaches the limiting bound from the specified side.Kα is non-decreasing in α, with lim α→0 Kα = +∞ and lim α→−∞ Kα ≥ 1.
  • Threshold behaviour: The threshold αK is non-decreasing in K, tends to 0 as K approaches infinity, and tends to −∞ as K approaches 1.These limits follow from convergence in K and the ordering of the expected Monte Carlo bounds.

D Optimisation issues with α-divergences and MC approximations

Selecting α is task-dependent because divergence choice affects competing objectives, while Monte Carlo approximation further biases and destabilizes that selection. The paper therefore treats α choice as an unresolved practical issue rather than a universally optimal setting.

  • Selection of α: α selection is an outstanding question because no single value optimizes all approximate-inference performance measures.The experiments therefore focus on test error and test log-likelihood.
  • Selection of α: The optimal α can change with dataset size because the posterior form changes as the number of observations N changes.Mode-seeking methods may favor predictive error, whereas mass-covering methods may favor test log-likelihood, with failure modes in both regimes.
  • Monte Carlo effects: Monte Carlo bias favors zero-forcing and makes the effective α closer to α = 1, complicating practical divergence selection.Finite-sample estimates can have pathological behavior and generally higher variance than the variational case.
  • Scope: Non-KL divergences can still be useful with Monte Carlo approximations, as shown by reported applications of approximate EP and BB-α.These examples include deep Gaussian-process regression and Bayesian-neural-network reinforcement learning.

E Unified implementation: derivation details

The VR implementation becomes tractable by combining reparameterization with finite-sample Monte Carlo estimates and importance weights. In the α → 1 limit, these quantities recover the original VAE algorithm.

  • Reparameterization: For α ≠ 1, the reparameterization trick expresses the VR bound using a distribution p(ϵ) independent of the recognition model.The derivation abbreviates the transformed model quantity as gφ = gφ(ϵ).
  • Monte Carlo approximation: Finite-K Monte Carlo approximation yields a tractable VR objective based on sampled importance weights.The appendix derives the finite-sample form and defines the associated weights.
  • VAE limit: As α → 1, both exact and Monte Carlo-normalized weights converge to 1 and 1/K, respectively, recovering the original VAE algorithm.This establishes VAE as a limiting special case of the VR formulation.

F Stochastic approximation for large-scale learning: derivations

The paper derives stochastic, mini-batch implementations of VR that connect to SEP and BB-α while preserving distinct fixed-point behavior. Under Gaussian and exponential-family assumptions, it also bounds the stochastic approximation and specifies the Bayesian-neural-network evaluation setting.

  • Unified derivation: Setting M = 1 and α = 1 − β/N connects VR optimization to SEP and BB-α through mini-batch approximations.The derivation uses average-likelihood and subset-average-likelihood representations.
  • Fixed-point approximation: VR fixed-point conditions match moments under a tilted distribution, which are approximated by replacing the full-data average likelihood with a mini-batch version.This produces stochastic moment-matching updates for the variational parameters.
  • SEP and BB-α: The energy approximation directly recovers BB-α when M = 1, whereas the fixed-point approximation recovers SEP.Both approaches use subset sampling but apply different transformations before updating the approximate distribution.
  • SEP and BB-α: SEP and BB-α both return SVI as α → 1, but their non-linear inverse mappings produce different fixed-point conditions at other α values.SEP averages natural parameters after inverse mapping, while BB-α averages tilted-distribution moments.
  • Method comparison: SEP can recover the exact posterior when the approximation family contains the correct solution, whereas BB-α may remain biased but is simpler to optimize with stochastic gradients.The distinction is between stronger fixed-point justification and easier black-box implementation.
  • Approximation bound: Under Gaussian q and exponential-family likelihood assumptions, Theorem 3 bounds the stochastic approximation for α ≤ 1 and r > 1.The proof uses Hölder’s inequality and a closed-form Gaussian cumulant generating function.
  • Approximation bound: Corollary 2 applies the bound to BB-α for α > 0 and r > 1, adding a covariance-dependent term involving sufficient statistics.The bound is expressed through tr(ΣCovD(Ψ)).
  • Bayesian neural networks: Bayesian-neural-network regression uses a Gaussian likelihood, unit Gaussian prior, and diagonal Gaussian variational posterior, with performance reported across datasets.The main evaluation compares test error and test log-likelihood, finding dataset-dependent optimal α settings and different mode-seeking versus mass-covering trade-offs.

G.2 Variational auto-encoder

The VAE experiments compare VAE, IWAE, and VR-max across four datasets and multiple architectures. VR-max produces samples with nearly identical visual quality to IWAE, while the figures and tables report likelihood and RMSE-related evaluation results.

  • Experimental design: The tested VAE architectures vary the number of stochastic layers, hidden units, and activation functions, with dataset-specific training settings.The architecture configurations are summarized in Table 3.
  • Generated samples: VR-max samples have visual quality almost identical to samples from IWAE.The comparison is presented for trained auto-encoders in Figure 2.
  • Reported metrics: Figure 1 reports Bayesian-neural-network test log-likelihood and RMSE results, with lower values better, while Tables 1 and 2 provide the corresponding regression summaries.These reported metrics concern regression evaluation rather than VAE sample quality.
Loading 1602.02311v3…