Source-linked AI summary
Variational Sequential Monte Carlo
Christian A. Naesseth, Scott W. Linderman, Rajesh Ranganath, David M. Blei
TL;DR
Variational inference needs flexible approximating families and efficient optimization while retaining posterior accuracy. The paper introduces VSMC, which combines VI and SMC and optimizes a tractable stochastic objective; it reports arbitrarily accurate approximations and competitive results across several sequential-model applications. VSMC is focused on state space and time-series models, and discrete ancestor variables can increase gradient-estimator variance.
Problem
Variational inference must balance faithful posterior approximation with fast optimization, especially for complex data and models.
Method
VSMC defines a variational family from SMC-generated weighted trajectories and optimizes its surrogate ELBO using an unbiased SMC log-marginal-likelihood estimate.
Results
VSMC can achieve arbitrarily accurate posterior approximations, remains effective as sequence length grows, and matches IWAE accuracy with less computation in neural-circuit data.
Takeaways & Limitations
VSMC provides a flexible approximate Bayesian inference algorithm that trades off posterior fidelity against computational complexity.
Takeaways & Limitations
VSMC focuses on state space and time-series models, and discrete ancestor variables can cause high variance in the score function term.
Abstract
from arXiv · showhide
Many recent advances in large scale probabilistic inference rely on variational methods. The success of variational approaches depends on (i) formulating a flexible parametric family of distributions, and (ii) optimizing the parameters to find the member of this family that most closely approximates the exact posterior. In this paper we present a new approximating family of distributions, the variational sequential Monte Carlo (VSMC) family, and show how to optimize it in variational inference. VSMC melds variational inference (VI) and sequential Monte Carlo (SMC), providing practitioners with flexible, accurate, and powerful Bayesian inference. The VSMC family is a variational family that can approximate the posterior arbitrarily well, while still allowing for efficient optimization of its parameters. We demonstrate its utility on state space models, stochastic volatility models for financial data, and deep Markov models of brain neural circuits.
1 Introduction
Variational sequential Monte Carlo (VSMC) combines variational inference and sequential Monte Carlo to balance posterior fidelity with efficient optimization. It maintains strong posterior approximations across long sequences and performs well across simulated and real-data experiments.
- 1 Introduction: VSMC introduces a variational family that blends VI with SMC and can approximate the posterior arbitrarily well.Its parameters index distributions generated by running SMC and sampling a trajectory according to its final weight.
- 1 Introduction: VSMC’s variational interpretation provides a tractable lower bound and connects the IWAE lower bound as a special case of the VSMC bound.The framework also provides a new variational approximation to the posterior in addition to a log-marginal-likelihood lower bound.
- 1 Introduction: The VSMC lower bound remains nearly the true marginal likelihood as sequence length increases, while IWAE gains minimally over standard VB.The comparison attributes IWAE’s degradation to particle degeneracy caused by omitting resampling.
- 1 Introduction: VSMC can outperform the locally optimal proposal on simulated linear Gaussian state space data and yields better posterior inferences for stochastic volatility models.For exchange-rate data, it also learns more efficient proposals than the comparison method.
- 1 Introduction: On macaque monkey neural recordings, VSMC reaches the same accuracy as IWAE with less computation.The study uses a probabilistic model based on recurrent neural networks.
2 Background
The background frames posterior inference in sequential models as computationally difficult and motivates combining VI with SMC. VSMC learns proposal distributions, generates weighted trajectories through resampling and propagation, and optimizes a tractable stochastic objective.
- 2 Background: Posterior computation is often intractable, motivating approximations such as variational inference and sequential Monte Carlo for state space and time-series models.Their joint densities factorize sequentially over latent states and observations.
- 2 Background: SMC sequentially resamples ancestors, proposes new states, appends them to trajectories, and reweights the resulting particles.Importance sampling omits resampling, assigning each new ancestor to the corresponding preceding particle.
- 2 Background: The final particles and weights define an SMC posterior approximation, whose accuracy can become arbitrarily high as the number of particles increases.SMC also produces an unbiased marginal-likelihood estimate used by the VSMC objective.
- 2 Background: The proposal distribution is the key SMC design choice, and VSMC learns parameterized proposals to address poor prior-based approximations with few particles.Prior proposals are especially problematic when latent states are high-dimensional.
3 Variational Sequential Monte Carlo
VSMC defines variational approximations by running SMC with parameterized proposals, then derives a tractable surrogate ELBO and stochastic optimization procedure for fitting them.
- VSMC family: VSMC samples a variational family by running SMC with proposals parameterized by λ and selecting a trajectory according to particle weights.The resulting variational distribution marginalizes the auxiliary variables generated during sampling, retaining the selected trajectory.
- Surrogate ELBO: The VSMC density is tractable in principle, but direct Monte Carlo estimation of its log density and ELBO is biased.The paper therefore derives a surrogate objective instead of estimating the exact ELBO directly.
- Surrogate ELBO: The surrogate ELBO is a lower bound to the VSMC ELBO and an upper bound on the corresponding KL divergence.It is the expected SMC log-marginal-likelihood estimate.
- Stochastic optimization: Monte Carlo estimates of the surrogate ELBO and its gradients enable stochastic optimization of the VSMC variational parameters.The proposed optimization uses reparameterization gradients when proposals are reparameterizable; discrete ancestor variables can contribute a high-variance score-function term.
- Stochastic optimization: Ignoring the ancestor-variable score-function term led to faster convergence with little difference in final ELBO in the reported experiment.The authors assessed this choice on a small state space model with variance-reduction techniques.
4 Perspectives on Variational SMC
VSMC connects particle-number scaling, VIS, and IWAE to characterize when its variational approximation becomes accurate and how it behaves in special cases. The analysis shows that increasing particles with sequence length preserves accuracy, while VIS provides an informative T = 1 specialization.
- Special cases: For N = 1, VSMC reduces to a structured variational approximation whose distribution is exactly the proposal.For T = 1, it becomes variational importance sampling (VIS).
- Special cases: VIS has exactly the IWAE lower bound, while its variational interpretation specifies generating latent samples by selecting from the weighted particle approximation.VIS is the no-resampling, T = 1 special case of VSMC.
- Bound comparisons: In practice, the VSMC lower bound outperforms the VIS lower bound, although resampling can produce a looser bound than standard VB or VIS in some cases.The comparison distinguishes a practical empirical tendency from the absence of a universal ordering.
- VIS example: In the multimodal VIS example, a Gaussian proposal yields a flexible approximation, whereas Gaussian standard VB captures only one mode.The example uses N = 10 particles and illustrates the flexibility inherited by VSMC for T > 1.
- Theoretical properties: As N approaches infinity, the surrogate-ELBO gap disappears and the returned VSMC trajectory distribution tends to the true target posterior.The stated theoretical result assumes uniform integrability of the SMC log-marginal-likelihood estimate.
- Scaling with sequence length: Unlike standard VB and IWAE (VIS), VSMC can remain arbitrarily accurate as T grows by setting N proportional to T.This follows from the stated large-particle convergence and the SMC scaling result.
5 Empirical Study
The empirical study evaluates VSMC on linear Gaussian, financial, and neural time-series models, finding accurate inference and efficient optimization across these settings.
- Linear Gaussian State Space Model: The linear Gaussian experiments use exact Kalman-filter marginal likelihoods to assess convergence and proposal quality.The model has efficiently computable exact posterior quantities.
- Gradient optimization: Biased gradients produce final ELBO values similar to unbiased gradients while converging faster, motivating their use in subsequent experiments.The comparison uses N = 4 particles.
- Linear Gaussian State Space Model: VSMC learns accurate proposals that outperform both bootstrap and locally optimal SMC proposals on a high-dimensional linear Gaussian state-space model.The experiment uses T = 25, Q = 0.12I, R = 1, dx = 10, and dy = 1.
- Stochastic Volatility Model: VSMC outperforms competing methods by almost 0.2 nats per time-step on the stochastic volatility exchange-rate experiment.The optimized ELBO is reported for N = {4, 8, 16}.
- Particle scaling: Increasing particle count can substantially improve VSMC's bound, whereas the gain for IWAE remains limited after optimizing parameters with N = 16.The comparison fixes θ⋆ and λ⋆ during evaluation.
- Deep Markov Model: On macaque neural population data, VSMC reaches the same ELBO as IWAE faster for dx = {3, 5, 10} and T = 21.The proposal factorizes over components of x_t and uses neural networks parameterized by λ.
6 Conclusions
The conclusion presents VSMC as a variational family combining VI and SMC to balance posterior fidelity against computational complexity.
- 6 Conclusions: VSMC combines variational inference and sequential Monte Carlo into a flexible approximate Bayesian inference family.The family is presented as a new variational approximating family.
- 6 Conclusions: VSMC lets practitioners trade off posterior fidelity and computational complexity.
A.1 Proof of Proposition 1
The proof begins from the joint distribution of variables generated by VSMC and identifies the marginal distribution of the returned latent trajectory.
- A.1 Proof of Proposition 1: The VSMC algorithm induces a distribution over all random variables it generates.
- A.1 Proof of Proposition 1: The variational family is defined through the marginal distribution q(x1:T; λ) of the VSMC-generated variables.
- A.1 Proof of Proposition 1: Conditioned on x1:T, the ancestral path b1:T of the returned particle is uniformly distributed over {1, ..., N}^T.
- A.1 Proof of Proposition 1: The proof obtains the desired marginal expression by substituting the uniform ancestral-path result into the preceding distributional expression.
A.2 Proof of Theorem 1
The proof derives the VSMC ELBO from the variational marginal and applies conditional Jensen's inequality to obtain a tractable lower bound.
- A.2 Proof of Theorem 1: The ELBO is defined as the expectation under q(x1:T; λ) of the log joint density minus the log variational density.
- A.2 Proof of Theorem 1: The derivation applies conditional Jensen's inequality using the concavity of −t log t for t > 0.
- A.2 Proof of Theorem 1: The final step follows because q(x1:T; λ) is the marginal of the joint VSMC-generated distribution.
A.3 Stochastic Optimization
The section describes stochastic estimators and variance-reduction strategies for optimizing VSMC, while noting that a general score-function estimator can have high variance.
- The optimization procedure uses a stochastic estimate of the control-variate quantity ct.
- For T = 2, a leave-one-out estimator can be used for the ancestor-variable score-function gradient.
- The general score-function-like estimator applies broadly, but its variance is often quite high in practice.
A.4 Scaling With Dimension
The scaling experiment compares ELBO behavior for VB, IWAE, and VSMC as sequence dimension T increases, using N = 2T samples for IWAE and VSMC. VSMC maintains approximation quality as T grows, unlike VIS.
- Figure 6 compares ELBO for standard VB, IWAE, and VSMC as the toy problem dimension T varies, with N = 2T samples for IWAE and VSMC.
- VSMC can achieve arbitrarily good approximation of p(x1:T | y1:T ) as T →∞ when N ∝T, under regularity conditions.
- VIS approximation deteriorates as T increases, whereas the stated VSMC scaling property does not require the latent-space dimension dim(xt) to grow.