Source-linked AI summary
Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC
Roger Frigola, Fredrik Lindsten, Thomas B. Schön, Carl E. Rasmussen
TL;DR
The paper addresses Bayesian inference and learning for nonlinear, nonparametric state-space models with unobserved states and unknown dynamics. It places a Gaussian process prior on the transition function, marginalizes that function, and uses particle MCMC to sample the smoothing distribution. The resulting approach retains full nonparametric expressivity and supports posterior learning of the transition function from sampled state trajectories.
Problem
Learning nonlinear state-space dynamics is difficult when latent states are unobserved, while parametric GP representations limit the model's nonparametric character.
Method
The method places a Gaussian process prior on transition dynamics, marginalizes the transition function, and targets latent states and hyperparameters with tailored particle MCMC.
Results
The approach retains full nonparametric expressivity, provides high-quality smoothing distributions, and yields a closed-form posterior for transition dynamics given sampled trajectories.
Takeaways & Limitations
Latent states can be sampled without knowing the transition function, after which the transition-function posterior can be obtained from the smoothing samples.
Takeaways & Limitations
The approach does not address how to choose inducing inputs for its sparse Gaussian-process representation.
Abstract
from arXiv · showhide
State-space models are successfully used in many areas of science, engineering and economics to model time series and dynamical systems. We present a fully Bayesian approach to inference \emph{and learning} (i.e. state estimation and system identification) in nonlinear nonparametric state-space models. We place a Gaussian process prior over the state transition dynamics, resulting in a flexible model able to capture complex dynamical phenomena. To enable efficient inference, we marginalize over the transition dynamics function and infer directly the joint smoothing distribution using specially tailored Particle Markov Chain Monte Carlo samplers. Once a sample from the smoothing distribution is computed, the state transition predictive distribution can be formulated analytically. Our approach preserves the full nonparametric expressivity of the model and can make use of sparse Gaussian processes to greatly reduce computational complexity.
1 Introduction
The paper targets fully Bayesian learning of general nonlinear state-space models by placing a Gaussian process prior on unknown dynamics while retaining a known observation model. It marginalizes the transition function and uses particle MCMC to infer latent trajectories and hyperparameters.
- The paper seeks to learn general nonlinear state-space models whose complexity can increase with available data.
- A Gaussian process prior over the unknown transition function provides flexible, nonparametric dynamics without imposing a limiting functional form.
- The observation function is assumed known, possibly with finite-dimensional parameters, because overly flexible transition and observation models can cause non-identifiability.
- The approach places a posterior over the transition function, hidden state trajectory, and model hyperparameters.
- A tailored particle Markov Chain Monte Carlo algorithm samples the smoothing distribution while marginalizing the transition function.
- Unlike finite-target-point GP approaches, the method retains a nonparametric model and makes learning the transition function straightforward after state-trajectory inference.
2 Gaussian Process State-Space Model
The GP-SSM combines a Gaussian-process transition model with a measurement model and prior over initial states and hyperparameters. Its mean function can encode prior dynamical knowledge while the GP models complex departures from that mean.
- The GP-SSM is a generative probabilistic model with a Gaussian-process transition function and a measurement model for observations.
- The model places a prior over hyperparameters and allows the measurement model and initial-state prior to take arbitrary forms for inference purposes.
- Prior dynamical knowledge can be encoded in the GP mean function through a simple parametric or first-principles model.
- Figure 1 contrasts graphical models for standard GP regression and the GP-SSM, with thick horizontal bars denoting fully connected node sets.
- The GP-SSM can combine an insufficient simple model with flexible Gaussian-process deviations that capture complex departures from its mean function.
3 Inference over States and Hyper-parameters
The paper marginalizes the nonlinear transition function to obtain a non-Markovian state prior, then uses particle methods and PGAS to infer state trajectories and hyper-parameters. The resulting sampler preserves the exact smoothing distribution while addressing standard SMC limitations.
- 3.1 Marginalizing out the State Transition Function: Marginalizing the transition function preserves the model’s nonparametric character but introduces dependencies across time, removing the usual Markovian structure.The resulting state prior is generally non-Gaussian because its mean and covariance depend nonlinearly on the state trajectory.
- 3.2 Sequential Monte Carlo: Sequential Monte Carlo approximates the sequence of posterior distributions over increasingly long state trajectories using weighted particles and ancestor indices.Particles are propagated by sampling ancestor indices and new states, then weighted according to the measurement model.
- 3.3 Particle Markov Chain Monte Carlo: Standard SMC cannot infer hyper-parameters effectively and suffers path degeneracy, which reduces particle diversity for states far from the final time.These shortcomings motivate a particle Markov chain Monte Carlo approach.
- 3.3.1 Sampling the State Trajectories: PGAS alternates Gibbs-like updates of the state trajectory and hyper-parameters, using a conditional particle filter with ancestor sampling for non-Markovian models.The conditional filter uses forward recursion and conditions on one prespecified particle trajectory.
- 3.3.1 Sampling the State Trajectories: For any N ≥2, the conditional particle procedure leaves the exact smoothing distribution invariant, while increasing N generally reduces autocorrelation and improves mixing.In practice, a moderate number of particles is often sufficient for rapid mixing.
- 3.3.2 Sampling the Hyper-parameters: Given a state trajectory, hyper-parameter sampling separates likelihood and trajectory-prior parameters; covariance hyper-parameters can be sampled using slice sampling.After conditioning on the trajectory, the dynamics problem is analogous to Gaussian process regression with states as inputs and outputs.
4 A Sparse GP-SSM Construction and Implementation Details
The construction uses sparse GP priors with inducing variables and reuses matrix computations to reduce particle-MCMC implementation costs while retaining a nonparametric GP-SSM.
- Efficient Cholesky updates and reused intermediate computations reduce vanilla-GP complexity to O(T^3) and FIC complexity to O(M^2T), with M ≪ T.
- Sparse GP-SSMs use a different function prior rather than approximating the vanilla GP-SSM.
- Inducing variables are analytically marginalized, while inducing inputs must cover the region of input space occupied by the data.
- The FIC formulation uses covariance substitutions and Woodbury identities to express one-step predictive quantities efficiently.
- The inducing-input selection problem is not addressed, although greedy methods are identified as one possible option.
5 Learning the Dynamics
The learned transition dynamics are obtained by integrating over posterior samples of latent states and hyperparameters, producing a Gaussian-mixture predictive distribution for the transition function.
- Learning the dynamics is formulated as obtaining the predictive distribution of f at an arbitrary test point.
- The predictive distribution integrates the conditional GP prediction over the joint posterior of the latent trajectory and hyperparameters.
- Each posterior sample contributes a Gaussian prediction, yielding a mixture whose components use sampled states as inputs and outputs.
- Sparse GP regression can replace the mixture when evaluating it is too costly, including control and real-time filtering applications.
6 Experiments
Experiments evaluate the GP-SSM on a multimodal nonlinear benchmark and a cart-and-pole system, examining smoothing, transition prediction, and predictive distributions.
- Nonlinear system benchmark: The nonlinear benchmark has a multimodal smoothing density because observations do not reveal the sign of the latent state.
- Nonlinear system benchmark: Smoothing samples recover two alternative state trajectories and accurately portray the underlying dynamics despite a mismatched parametric mean function.
- Nonlinear system benchmark: Transition prediction is evaluated by RMSE against ground truth over 10 independent training and test datasets, including vanilla and sparse GP-SSMs.
- Figures: Figure 2 contrasts the smoothing distribution with the actual transition function, model-B mean function, and smoothing samples.
- Cart and pole: The cart-and-pole experiment learns from 100 noisy state observations covering only a small region of the four-dimensional state space.
7 Conclusions
The paper presents fully Bayesian inference and learning for GP-SSMs by marginalizing the transition function and using a tailored PGAS sampler. This permits smoothing with unknown dynamics while retaining nonparametric expressivity.
- The approach performs fully Bayesian inference and learning in GP-SSMs while retaining the model’s full nonparametric expressivity.
- Marginalizing the state transition function creates a non-trivial inference problem addressed with a tailored PGAS sampler.
- Latent states can be sampled from the smoothing distribution when the transition function is unknown, using GP smoothness and parsimony assumptions.
- Smoothing samples then support a posterior description of the state transition function, rather than conditioning smoothing on a known dynamics model.