Source-linked AI summary
Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data
Maximilian Karl, Maximilian Soelch, Justin Bayer, Patrick van der Smagt
TL;DR
Learning latent state-space models from raw, nonlinear sequence data requires both system identification and tractable inference. DVBF combines state-space constraints with stochastic-gradient variational inference, and the experiments report informative latent states and stable long-term predictions beyond training sequence lengths. The method’s scope includes assumptions about Markovian latent states and universal, sample-independent transition parameters.
Problem
DVBF addresses joint identification of latent dynamical systems and inference when nonlinear observations make the true posterior intractable.
Method
DVBF combines latent state-space assumptions with stochastic gradient variational Bayes to learn inference and generative dynamics from raw sensory data.
Results
DVBF recovers latent states identifying underlying physical quantities and produces stable long-term predictions beyond the sequence length used during training.
Takeaways & Limitations
Backpropagation through transitions forces the latent space to fit the transition, targeting full information in latent states and state-space behavior.
Takeaways & Limitations
The method assumes the current latent state contains all necessary information for the current observation and next state, while transition parameters are sample-independent.
Abstract
from arXiv · showhide
We introduce Deep Variational Bayes Filters (DVBF), a new method for unsupervised learning and identification of latent Markovian state space models. Leveraging recent advances in Stochastic Gradient Variational Bayes, DVBF can overcome intractable inference distributions via variational inference. Thus, it can handle highly nonlinear input data with temporal and spatial dependencies such as image sequences without domain knowledge. Our experiments show that enabling backpropagation through transitions enforces state space assumptions and significantly improves information content of the latent embedding. This also enables realistic long-term prediction.
1 INTRODUCTION
State-space models face a tension between expressive latent representations and tractable inference, especially for nonlinear sequential data. DVBF addresses this by enforcing state-space assumptions while learning from raw inputs with variational and stochastic-gradient methods.
- 1 INTRODUCTION: Recent deep models represent high-dimensional sequences with temporal dependencies, but their main objective is modeling observed sequence distributions.The target is a model p(x1:T) that reflects observed sequences.
- 1 INTRODUCTION: Identifying governing systems from data and inferring latent variables from observations remain significant challenges for state-space modeling.More expressive representations make inference more computationally demanding, while efficient methods can restrict the system class.
- 1 INTRODUCTION: Variational time-series models improve marginal data likelihood and compression but may lack full-information latent states needed for long-term sampling.The supplied passage identifies this limitation for prior extensions of stochastic gradient variational Bayes.
- 1 INTRODUCTION: DVBF enforces latent state-space assumptions, supplies rich inference dependencies, trains on raw sensory data, and scales through stochastic gradient descent.The authors present these as four contributions, including reliable system identification and plausible long-term prediction.
2 BACKGROUND AND RELATED WORK
The paper frames latent dynamical modeling as learning both a generative state-space model and efficient posterior inference for complex observations. Prior variational and deep filtering approaches often compromise Markovian latent information or long-term prediction, motivating DVBF’s transition-focused design.
- 2.1 PROBABILISTIC MODELING AND FILTERING OF DYNAMICAL SYSTEMS: The model represents high-dimensional, potentially non-Markovian observations with latent sequences driven by control inputs.The latent formulation separates an emission model from a transition model over z1:T.
- 2.1 PROBABILISTIC MODELING AND FILTERING OF DYNAMICAL SYSTEMS: Learning requires identifying both emission and transition components while performing prediction, filtering, or smoothing over latent states.These inference tasks use progressively larger portions of the observation sequence.
- 2.1 PROBABILISTIC MODELING AND FILTERING OF DYNAMICAL SYSTEMS: State-space assumptions make the current latent state sufficient for the current observation and next state, giving zt Markovian behavior unlike observations.The transition also conditions on the current control input and transition parameters.
- 2.1 PROBABILISTIC MODELING AND FILTERING OF DYNAMICAL SYSTEMS: Linear Gaussian models permit optimal Kalman-filter inference, but their restrictive assumptions can be violated in practical nonlinear applications.The passage also notes that nonlinear Kalman-filter extensions retain major drawbacks.
- 2.1 PROBABILISTIC MODELING AND FILTERING OF DYNAMICAL SYSTEMS: For image sequences, the true posterior can be highly nonlinear and intractable, making simple mean-field approximations too limited.The paper proposes learning identification and inference jointly with stochastic gradient variational Bayes.
- 2.2 STOCHASTIC GRADIENT VARIATIONAL BAYES (SGVB) FOR TIME SERIES DISTRIBUTIONS: Earlier variational sequence models often included observations directly in transitions or learned states focused on reconstruction, weakening latent dynamics and long-term generation.The paper attributes this pattern to fitting latent dynamics to reconstruction-beneficial states and becoming stuck in a local optimum.
- 2.2 STOCHASTIC GRADIENT VARIATIONAL BAYES (SGVB) FOR TIME SERIES DISTRIBUTIONS: DVBF reverses the learning direction by forcing the latent space to fit the transition, thereby targeting state-space assumptions and full information in latent states.Figure 1 depicts the updated state as depending on the previous state, control input, and transition parameters.
3 DEEP VARIATIONAL BAYES FILTERS
DVBF learns latent dynamical systems by making transitions differentiable while inferring stochastic transition parameters, allowing reconstruction errors to propagate through time. Its variational lower-bound objective and locally linear transition parameterization support training on time series and application-specific dynamics.
- 3.1 Reparametrizing the Transition: DVBF establishes gradient paths through latent transitions so reconstruction errors from future observations propagate backward through time.The transition is deterministic conditional on sampled β_t, making ∂z_t+1/∂z_t well-defined.
- 3.1 Reparametrizing the Transition: The recognition model infers transition parameters β_t rather than latent states, while transition priors regularize the dynamics and latent-space manifolds.β_t is split into sample-specific process noise w_t and universal parameters v_t.
- 3.1 Reparametrizing the Transition: Figure 2 describes stochastic parameter inference followed by deterministic state transition and prediction of the next observation.The right panel illustrates a locally linear transition in latent space.
- 3.2 The Lower Bound Objective Function: DVBF derives a variational lower bound consisting of expected reconstruction log-likelihood minus the KL divergence between inferred transition parameters and their prior.Because transitions are deterministic given β_t, the latent-state distribution simplifies through Dirac factors.
- 3.2 The Lower Bound Objective Function: The annealed objective increases inverse temperature c_i linearly to one, and experiments report this version as beneficial to overall performance.The transition prior p(v_1:T) is estimated with an empirical Bayes approach, using isotropic Gaussian priors in all experiments.
- 3.3 Example: Locally Linear Transitions: For locally linear transitions, z_t+1 = A_tz_t + B_tu_t + C_tw_t, with matrices generated as state- and control-dependent combinations of globally learned linear systems.The parametrization is application-dependent and was used for the experiments.
4 EXPERIMENTS AND RESULTS
DVBF-LL is evaluated against DKF on simulated pendulum and bouncing-ball systems, focusing on whether latent states recover full system information and support long-term generation. Across these experiments, DVBF-LL captures state variables more completely and produces stable predictions beyond training sequence lengths.
- Experimental setup: Experiments compare DVBF-LL with DKF on simulated dynamical systems whose ground-truth latent states are known.The evaluation includes a torque-controlled pendulum and bouncing-ball environments.
- Dynamic pendulum: DVBF-LL encodes pendulum angle and angular velocity, whereas DKF learns little to no information about angular velocity.DVBF-LL represents angle in polar coordinates and shows high correlation between latent states and both ground-truth variables.
- Dynamic pendulum: DVBF-LL generative samples follow the pendulum’s attractor for 100 time steps, although generation becomes more noise-prone near the attractor.Filtering uses observations throughout, while purely generative sampling starts from one initial latent state.
- Dynamic pendulum: DVBF performs well on pendulum sequences much longer than the training sequences, demonstrating long-horizon generation beyond the training-length boundary.The red indicator marks the training-sequence length; subsequent samples are generated without observation correction.
- Bouncing ball: In the bouncing-ball experiment, DVBF extracts two-dimensional position and velocity information into a four-dimensional latent representation.A 3×3 checkerboard position pattern is recovered, while the remaining dimensions correspond smoothly to x- and y-velocity.
- Bouncing ball: For two bouncing balls, a 10-dimensional latent space yields stable generative predictions beyond the training sequence length.The figure compares ground truth, reconstructions, and generative samples from identical initial latent states.
5 CONCLUSION
DVBF learns state space models from raw non-Markovian sequences, performs latent dynamic system identification, and supports inference through stochastic gradient variational Bayes. Experiments recovered latent physical quantities and produced stable long-term predictions beyond the training sequence length.
- 5 CONCLUSION: DVBF learns state space models from raw non-Markovian sequence data.The method performs latent dynamic system identification while addressing intractable inference.
- 5 CONCLUSION: Stochastic gradient variational Bayes enables DVBF to scale naturally to large data sets.
- 5 CONCLUSION: Vision-based experiments recovered latent states that identify the underlying physical quantities.
- 5 CONCLUSION: Stable generative predictions extended far beyond the sequence length used during training.
B SUPPLEMENTARY TO IMPLEMENTATION
The experiments use short raw-image sequences with randomized control inputs and separate training, validation, and test sets.
- B SUPPLEMENTARY TO IMPLEMENTATION: Each experiment uses sequences of 15 raw images with 16×16 pixels.The observation space is X ⊂R256.
- B SUPPLEMENTARY TO IMPLEMENTATION: Training, validation, and test sets each contain 500 sequences.
- B SUPPLEMENTARY TO IMPLEMENTATION: Control input sequences are drawn randomly using motor babbling.
B.2 ADDITIONAL EXPERIMENT PLOTS
The supplementary figure provides a complete version of Figure 4, showing ground truth alongside recognition-model and generative-model samples.
- B.2 ADDITIONAL EXPERIMENT PLOTS: The figure compares ground truth with samples from the recognition and generative models.
- B.2 ADDITIONAL EXPERIMENT PLOTS: The supplementary version includes all samples missing from Figure 4.
- B.2 ADDITIONAL EXPERIMENT PLOTS: The figure is a complete version of Figure 4 rather than a separate experimental setup.
B.3 IMPLEMENTATION DETAILS FOR DVBF IN PENDULUM EXPERIMENT •
The pendulum DVBF implementation uses compact image-sequence inputs, a three-dimensional latent space, neural observation and recognition models, and Adadelta optimization with annealed regularization.
- B.3 IMPLEMENTATION DETAILS FOR DVBF IN PENDULUM EXPERIMENT: The model receives 15 timesteps of 162 observation dimensions and one action dimension.
- B.3 IMPLEMENTATION DETAILS FOR DVBF IN PENDULUM EXPERIMENT: The latent space has three dimensions.
- B.3 IMPLEMENTATION DETAILS FOR DVBF IN PENDULUM EXPERIMENT: The observation network uses 128 ReLU units and a 162-dimensional identity output.It parameterizes p(x_t|z_t) as a Gaussian observation model.
- B.3 IMPLEMENTATION DETAILS FOR DVBF IN PENDULUM EXPERIMENT: The recognition model uses 128 ReLU units and a six-dimensional identity output.
- B.3 IMPLEMENTATION DETAILS FOR DVBF IN PENDULUM EXPERIMENT: The initial network is a fast-dropout bidirectional RNN with 128 ReLU units and a three-dimensional identity output.The initial transition also uses 128 ReLU units and a three-dimensional identity output.
- B.3 IMPLEMENTATION DETAILS FOR DVBF IN PENDULUM EXPERIMENT: Optimization uses Adadelta with a 0.1 step rate and an inverse-temperature schedule beginning at c0 = 0.01.The inverse temperature is updated every 250th gradient update over TA = 105 iterations.
B.4 IMPLEMENTATION DETAILS FOR DVBF IN BOUNCING BALL EXPERIMENT •
The single bouncing-ball DVBF implementation uses 15-step, 162-dimensional observations with two action dimensions and a 4-dimensional latent space. Its networks use 128-unit hidden layers, Adadelta optimization, and scheduled inverse-temperature updates.
- The input comprises 15 timesteps of 162 observation dimensions and 2 action dimensions.
- The latent space has 4 dimensions.
- The observation network uses 128 ReLU units and a 162-dimensional identity output, while the recognition model uses 128 ReLU units and 8 identity outputs.
- The initial network is a Fast Dropout BiRNN with 128 ReLU units and 4 identity outputs, followed by an initial transition network with the same hidden and output widths.
- Training uses Adadelta with a 0.1 step rate; inverse temperature starts at c0 = 0.01 and updates every 250th gradient update for TA = 105 iterations.
B.5 IMPLEMENTATION DETAILS FOR DVBF IN TWO BOUNCING BALLS EXPERIMENT •
The two-bouncing-ball implementation uses 15-step, 202-dimensional observations and a 10-dimensional latent space. It employs 128-unit networks, Adam optimization, and inverse-temperature updates at every gradient update.
- The input comprises 15 timesteps of 202 observation dimensions and 2000 samples.
- The latent space has 10 dimensions.
- The observation network uses 128 ReLU units and a 202-dimensional sigmoid output, while the recognition model uses 128 ReLU units and 20 identity outputs.
- The initial network is an MLP with 128 ReLU units and 10 identity outputs, followed by an initial transition network with the same hidden and output widths.
- Training uses Adam with a 0.001 step rate; inverse temperature starts at c0 = 0.01 and updates every gradient update for TA = 2 105 iterations.
B.6 IMPLEMENTATION DETAILS FOR DKF IN PENDULUM EXPERIMENT •
The pendulum DKF implementation uses 15-step, 162-dimensional observations with one action dimension and a 3-dimensional latent space. Its recurrent and transition networks use stacked sigmoid layers and Adam optimization.
- The input comprises 15 timesteps of 162 observation dimensions and 1 action dimension.
- The latent space has 3 dimensions.
- The observation network uses two 128-unit sigmoid layers and a 2 162-dimensional identity output.
- The recognition model uses a Fast Dropout BiRNN with two 128-unit sigmoid layers and 3 identity outputs.
- The transition network uses two 128-unit sigmoid layers and 6 outputs, with Adam optimization at a 0.001 step rate.
- Inverse temperature starts at c0 = 0.01, updates every 25th gradient update, and uses TA = 2000 iterations.