PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Adaptive Sequential Monte Carlo

Shixiang Gu, Zoubin Ghahramani, Richard E. Turner

arXiv:1506.03338v3cs.LGstat.ML

TL;DR

SMC inference depends critically on constructing proposals that match the target posterior. The paper introduces KL-based gradient adaptation for flexible proposals, including neural networks, and reports improved inference, parameter learning, and latent-variable recurrent-network training across its evaluated settings.

  • Problem

    SMC performance is strongly dependent on proposal quality, while constructing proposals that match the posterior remains an open problem.

  • Method

    The paper adapts flexible parametric proposals by approximating gradients of the inclusive KL divergence with SMC samples, including neural-network proposals.

  • Results

    NASMC significantly outperforms bootstrap, EKPF, and UPF methods on benchmark inference measures, improves PMMH burn-in and mixing, and supports latent-variable recurrent-network training.

  • Takeaways & Limitations

    Adaptive neural proposals extend SMC to richer models while connecting adaptive particle filtering with neural-network-based approximate inference.

Abstract

from arXiv · show

Sequential Monte Carlo (SMC), or particle filtering, is a popular class of methods for sampling from an intractable target distribution using a sequence of simpler intermediate distributions. Like other importance sampling-based methods, performance is critically dependent on the proposal distribution: a bad proposal can lead to arbitrarily inaccurate estimates of the target distribution. This paper presents a new method for automatically adapting the proposal using an approximation of the Kullback-Leibler divergence between the true posterior and the proposal distribution. The method is very flexible, applicable to any parameterized proposal distribution and it supports online and batch variants. We use the new framework to adapt powerful proposal distributions with rich parameterizations based upon neural networks leading to Neural Adaptive Sequential Monte Carlo (NASMC). Experiments indicate that NASMC significantly improves inference in a non-linear state space model outperforming adaptive proposal methods including the Extended Kalman and Unscented Particle Filters. Experiments also indicate that improved inference translates into improved parameter learning when NASMC is used as a subroutine of Particle Marginal Metropolis Hastings. Finally we show that NASMC is able to train a latent variable recurrent neural network (LV-RNN) achieving results that compete with the state-of-the-art for polymorphic music modelling. NASMC can be seen as bridging the gap between adaptive SMC methods and the recent work in scalable, black-box variational inference.

1 Introduction

SMC supports inference in nonlinear dynamical models but remains highly sensitive to proposal quality. The paper introduces gradient-based adaptation of flexible proposals, including neural-network parameterizations, to address this limitation.

  • SMC is useful for nonlinear dynamical models with hidden variables and can serve as a module in both maximum-likelihood and Bayesian parameter learning.
  • Poorly matched proposals can yield low effective sample sizes and high-variance Monte Carlo estimates, leaving good proposal construction an open problem.
  • The paper adapts proposal distributions by approximating KL-based objectives with SMC samples and supports flexible parameterizations such as neural networks.
  • The framework connects adaptive SMC with approximate inference for unsupervised neural-network models.

2 Sequential Monte Carlo

SMC approximates hidden-state posteriors with sequentially sampled and weighted trajectories. Resampling reduces weight degeneracy, but poor proposals can still collapse particle ancestry and increase computational costs.

  • The model factorization covers state-space models including HMMs and non-Markovian hidden-state models such as Gaussian processes.
  • SMC approximates hidden-state posteriors using weighted trajectories drawn from a sequential proposal distribution.
  • Sequential importance sampling computes samples and weights in one forward pass, while resampling replaces low-weight particles and assigns the new particles equal weight.
  • The Critical Role of Proposal Distributions in Sequential Monte Carlo: Poor proposals cause trajectories to collapse onto a single ancestor, and increasing particles or adding MCMC moves raises computational cost.
  • The Critical Role of Proposal Distributions in Sequential Monte Carlo: The bootstrap proposal ignores the current observation, whereas EKPF and UPF approximate the posterior but incur overhead and are not tuned to an SMC-specific criterion.

3 Adapting Proposals by Descending the Inclusive KL Divergence

The method adapts proposals by descending the inclusive KL divergence between the posterior and proposal. SMC samples provide gradient estimates, enabling general parametric proposals and online or batch updates.

  • The inclusive KL objective directly measures proposal quality and can be efficiently differentiated using sample-based approximations from SMC.
  • Filtering approximations reduce gradient-estimator variance relative to final-time weighted trajectories, at the cost of introducing some bias.
  • Proposal parameters can be updated online at each time step or in batch after a complete SMC forward pass.
  • Unlike EKPF and UPF, the method uses analytic propagation without costly particle-specific distributional approximation in an inner loop.
  • The adaptation framework applies to any parametric proposal distribution and can train complex models using stochastic-gradient methods.

4 Flexible and Trainable Proposal Distributions Using Neural Networks

The framework uses neural networks to build flexible, trainable proposal distributions. LSTMs encode sequential context, while mixture-density outputs support richer conditional distributions.

  • The adaptation method can use neural-network parameterizations, bringing supervised-learning literature and optimization tools to adaptive SMC.
  • LSTMs map prior hidden states and observations to current-state proposals, while mixture-density networks parameterize Gaussian-mixture outputs.

5 Experiments

The experiments evaluate NASMC for state estimation, Bayesian parameter learning, and polyphonic music modeling. Across these settings, adaptive proposals improve inference metrics, accelerate PMMH mixing with few particles, and achieve competitive music-modeling likelihoods.

  • 5.1 Inference in a Benchmark Nonlinear State-Space Model: NASMC proposals significantly outperform bootstrap, EKPF, and UPF methods in ESS, RMSE, and LML-estimate variance.All methods converge to similar LMLs close to large-particle reference values.
  • 5.1 Inference in a Benchmark Nonlinear State-Space Model: Multi-modal and recurrent proposal parameterizations improve performance, while incorporating prior dynamics provides additional benefit.The RNN-MD proposal outperforms a simple Gaussian, RNN outperforms NN, and RNN-f underperforms RNN.
  • 5.2 Inference in the Cart and Pole System: The cart-pole experiment evaluates latent-state inference using normalized ESS and posterior means for cart position and pole-angle change.RNN-MD learns higher ESS than the prior and more accurately estimates the latent states.
  • 5.3 Bayesian learning in a Nonlinear SSM: With N = 10 particles, NASMC enables faster PMMH burn-in and mixing than the prior proposal, especially for measurement noise σw.With N = 100 particles, the prior and adaptive method show almost no difference.
  • 5.4 Polyphonic Music Generation: On four polymorphic music datasets, the adaptive method significantly outperforms the bootstrap filter on three datasets and has comparable NLLs to the state of the art.The bootstrap method performs marginally better on the piano dataset.

6 Comparison of Variational Inference to the NASMC approach

NASMC resembles variational free-energy methods but optimizes the inclusive rather than exclusive KL divergence. Its proposal is then corrected by SMC, which may reduce approximation deficiencies in time-series inference.

  • 6 Comparison of Variational Inference to the NASMC approach: NASMC adapts proposals with the inclusive KL KL[pθ(z|x)||qφ(z|x)], whereas variational methods commonly optimize the exclusive KL KL[qφ(z|x)||pθ(z|x)].The inclusive-KL objective requires simulation from the true posterior for sample-based approximation.
  • 6 Comparison of Variational Inference to the NASMC approach: NASMC uses the approximate posterior as a proposal and SMC to construct a more accurate posterior approximation.The SMC procedure corrects deficiencies in the proposal approximation.
  • 6 Comparison of Variational Inference to the NASMC approach: The paper argues that this correction may offer advantages over variational free-energy methods in time-series settings where variational methods can have severe biases.

7 Conclusion

The paper develops a flexible neural method for adapting SMC proposal distributions and reports benefits across standard inference, PMMH sampling, and latent-variable RNN training.

  • 7 Conclusion: The method outperformed EKPF and UPF on a standard SMC benchmark, improved PMMH burn-in and mixing, and enabled effective LV-RNN training using SMC.It uses recurrent neural networks for long-range context, supports mixture density networks, and trains proposals with stochastic gradient descent.
  • 7 Conclusion: The proposal framework supports flexible distributional forms and recurrent parameterizations within general SMC algorithms.The paper identifies adaptive particle smoothing, high-dimensional latent models, and adaptive PMCMC for probabilistic programming as future applications.
Loading 1506.03338v3…