PaperlayerView exact revision on arXiv

Source-linked AI summary

Bayesian Nonparametric Inference of Switching Linear Dynamical Systems

Emily B. Fox, Erik B. Sudderth, Michael I. Jordan, Alan S. Willsky

arXiv:1003.3829v1stat.MEstat.ML

TL;DR

Complex dynamical phenomena can switch among linear modes whose number and structure are unknown. The paper learns persistent modes with Bayesian nonparametric SLDS and switching VAR models, using ARD for sparse dependencies; experiments demonstrate utility across synthetic and real applications.

  • Problem

    Existing switching dynamical-model methods often fix the number of modes, although real phenomena can exhibit recurring, changing, or previously unseen dynamics.

  • Method

    The paper extends the sticky HDP-HMM to SLDS and switching VAR models, combines it with ARD, and infers modes, states, and parameters using Gibbs sampling.

  • Results

    The models demonstrate utility and flexibility on synthetic data, honey bee dances, the IBOVESPA stock index, and maneuvering target tracking.

  • Takeaways & Limitations

    The formulation supports learning simple temporal structures with unknown mode counts and variable state dimensions or autoregressive orders.

Abstract

from arXiv · show

Many complex dynamical phenomena can be effectively modeled by a system that switches among a set of conditionally linear dynamical modes. We consider two such models: the switching linear dynamical system (SLDS) and the switching vector autoregressive (VAR) process. Our Bayesian nonparametric approach utilizes a hierarchical Dirichlet process prior to learn an unknown number of persistent, smooth dynamical modes. We additionally employ automatic relevance determination to infer a sparse set of dynamic dependencies allowing us to learn SLDS with varying state dimension or switching VAR processes with varying autoregressive order. We develop a sampling algorithm that combines a truncated approximation to the Dirichlet process with efficient joint sampling of the mode and state sequences. The utility and flexibility of our model are demonstrated on synthetic data, sequences of dancing honey bees, the IBOVESPA stock index, and a maneuvering target tracking application.

I. INTRODUCTION

The paper develops a Bayesian nonparametric framework for switching dynamical systems whose number of modes and dynamic structure are unknown. It extends sticky HDP-HMMs to persistent linear dynamical modes and uses ARD to identify sparse, variable-order dependencies.

  • Motivation: Linear dynamical systems model diverse phenomena, but structural changes require switching among dynamical models over time.Examples include maneuvering targets, financial events, and changing honey bee dances.
  • Research gap: Existing methods typically fix the number of modes or assign every detected change to a new mode, limiting flexibility for recurring behaviors.The paper instead remains agnostic about the number of dynamical modes.
  • Modeling framework: The paper frames SLDS and switching VAR processes as HMM extensions in which modes index linear dynamical processes, while VAR processes form only a subset of state-space models.SLDS models can capture temporal dependencies that ordinary HMMs cannot.
  • Approach: The approach extends the sticky HDP-HMM to learn an unknown number of persistent dynamical modes while capturing richer temporal dependencies than conditionally independent HMM observations.The sticky formulation improves control over the inferred number of modes and discourages unrealistically rapid switching.
  • Approach: Automatic relevance determination drives unsupported dynamic components toward zero, enabling SLDS models with varying state dimensions and switching VAR processes with varying autoregressive orders.This addresses model-order and sparsity issues in dynamic parameter estimation.
  • Inference and evaluation: The study combines Bayesian inference with prior-based preference for simpler mode and state-dimensional explanations rather than imposing hard model constraints.The proposed analysis includes synthetic and real-data applications and a Gibbs-sampling procedure.

III. THE HDP-SLDS AND HDP-AR-HMM

The proposed HDP-SLDS and HDP-AR-HMM place Bayesian nonparametric priors on switching dynamics and use Gibbs sampling to infer modes, states, and dynamic parameters. MNIW and ARD priors provide complementary dense and sparse treatments of dynamic structure.

  • Model construction: The HDP-SLDS and HDP-AR-HMM extend sticky HDP-HMMs by learning unknown mode spaces while estimating switching dynamics and, for SLDS, continuous latent states.The models are summarized through their generative processes in Table I.
  • Model construction: The HDP-SLDS places priors on mode-specific dynamics and measurement noise, while fixing C = [I_d 0] so the first d state components are measured.The chosen state dimension functions as a model-order choice.
  • Inference: Gibbs inference alternates state-sequence sampling, mode-sequence sampling, and parameter updates for the HDP-SLDS; the HDP-AR-HMM omits latent-state sampling.The AR-HMM conditions mode and parameter updates directly on observations.
  • Inference: Pseudo-observations rewrite both models as mode-conditioned linear regression problems, allowing their dynamic-parameter samplers to share a common formulation.Conditioning on mode assignments partitions the data into independent regressions, one per active mode.
  • MNIW prior: The conjugate MNIW prior yields posterior inference for each mode’s dynamic parameters, although it produces full matrices and does not identify irrelevant state or lag components.Its lack of explicit stability enforcement is considered acceptable because individual modes need not be stable for the overall SLDS to be stable.
  • ARD prior: ARD addresses growing model order and irrelevant components by encouraging unsupported dynamic parameters toward zero.For SLDS this targets state components; for HDP-AR-HMM it targets lag components.

2) Alternative Prior — Automatic Relevance Determination:

The ARD prior promotes sparse dynamical representations by shrinking irrelevant state or lag components toward zero. This supports inference of varying state dimensions and autoregressive orders, subject to structural assumptions about observed components and realization minimality.

  • SLDS sparsity: ARD places independent zero-mean Gaussian priors on columns of each SLDS dynamic matrix, with precision parameters controlling sparsity.Large precisions penalize nonzero columns, and iterative hyperparameter estimation strengthens this shrinkage.
  • SLDS sparsity: α_j^(k) → 0 indicates that state component j does not contribute to mode k dynamics and provides insight into that mode’s order.A zero dynamic column implies the individual mode realization is not minimal because its Hankel matrix has reduced rank.
  • Structural assumptions: A mode may be nonminimal after ARD removes a state component, while the overall SLDS realization may nevertheless remain minimal.Thus, mode-wise sparsity does not by itself establish nonminimality of the complete switching system.
  • Structural assumptions: The fixed measurement matrix C = [I_d 0] is compatible with sparse realizations only when observed state components are relevant to every mode.If no realization satisfies this criterion, a mode-specific measurement matrix is needed, but it introduces more pronounced identifiability issues.
  • Variable-order VAR: For switching VAR models, ARD assigns shared precision parameters to entire lag blocks, so large precision values turn off complete lag components.This enables inference of variable autoregressive orders rather than fixing one order for all modes.
  • Model-order inference: The prior can learn SLDS models with varying state dimensions or switching VAR processes with varying autoregressive orders.The relevant sparsity pattern is state-vector columns for SLDS and lag blocks for VAR models.

3) Measurement Noise Posterior:

The measurement-noise covariance is inferred within the HDP-SLDS sampler, alongside joint updates of latent states, mode sequences, and dynamical parameters. Conditional linear-Gaussian structure enables backward-filter-based block sampling of the state sequence.

  • Measurement noise posterior: The covariance update uses p(R | y1:T, x1:T) = IW(T + r0, SR + R0), with R shared between modes.The residual scatter term is formed from yt − Cxt across observations.
  • Sampler context: The HDP-SLDS Gibbs sampler alternates state-sequence sampling, mode-sequence sampling, and parameter updates, whereas HDP-AR-HMM omits the latent-state step.The AR-HMM conditions directly on observations; the SLDS conditions on sampled states when updating dynamics and modes.
  • Measurement noise posterior: For HDP-SLDS, the measurement-noise covariance R is sampled conditional on the latent state sequence x1:T.The posterior is an inverse-Wishart distribution based on observation residuals and the prior scale parameters.
  • Sampler context: Joint mode-sequence sampling uses a truncated HDP approximation and a forward-backward variant that accounts for dynamical correlations in the likelihood.This is important because direct observation correlations slow sequential sampler mixing.
  • State sampling: For fixed modes and parameters, the latent SLDS becomes a time-varying linear dynamical system whose state sequence can be block sampled with Kalman filtering.A backward Kalman filter computes messages before recursive conditional state sampling.

IV. RESULTS

Experiments compare HDP switching VAR, AR-HMM, and SLDS models with a sticky HDP-HMM baseline on synthetic switching processes. The results show that richer dynamical models capture process structure more effectively while accommodating varying mode counts.

  • Synthetic experiments: Three synthetic datasets compare HDP-VAR(1)-HMM6, HDP-VAR(2)-HMM, and HDP-SLDS against a sticky HDP-HMM baseline using first-difference observations.The HDP-VAR notation denotes an order-r HDP-AR-HMM with vector observations.
  • Synthetic experiments: In the five-mode switching VAR(1) scenario, the three switching linear dynamical models provide comparable performance because the larger model families contain HDP-VAR(1)-HMMs.The data use a 0.98 self-transition probability and equally likely transitions to the other modes.
  • Synthetic experiments: In the three-mode switching AR(2) scenario, HDP-AR(2)-HMM performs significantly better than HDP-AR(1)-HMM, while HDP-SLDS with C = [1 0] performs similarly with greater posterior variability.The HDP-SLDS sampler also mixes more slowly because it samples a hidden continuous state.
  • Synthetic experiments: All switching models substantially improve over the baseline sticky HDP-HMM, while richer models that switch among learned LDS outperform first-difference input representations.The results also demonstrate learning switching processes with varying numbers of modes.

B. ARD prior

The ARD prior improves inference for switching dynamical systems with sparse dependencies and identifies non-dynamical components or variable model orders. Experiments on synthetic and honey-bee data demonstrate its benefits, while performance remains sensitive to noisy observations and irregular switching.

  • Model flexibility: The ARD prior supports SLDS modes with varying effective state dimensions and switching VAR modes with varying autoregressive orders.This extends the model beyond a fixed state dimension or autoregressive order.
  • Synthetic comparison: ARD provides superior mode-sequence estimates and identifies non-dynamical state components in a sparse two-mode SLDS.The inferred precision histograms separate dynamic from non-dynamical components.
  • Honey-bee experiments: The unsupervised HDP-VAR(1)-HMM achieves reasonably good honey-bee segmentations without manually supplied domain-specific knowledge and identifies modes that reoccur over time.The paper reports very good performance on sequences 4–6, while contrasting this with supervised approaches.
  • Limitations: Noisy tracking and irregular switching in honey-bee sequences 1–3 lead to multiple sequence-specific waggle modes and many segmentation errors.The authors attribute this sensitivity to avoiding domain-specific information and to noisier vision-based tracking.
  • Honey-bee experiments: ARD hyperparameters recover dance-specific orders: the two turning dances rely on the first lag, whereas the waggle dance uses both first and second lags.These learned orders agree with Levinson’s algorithm applied to ground-truth segmented data.

V. MODEL VARIANTS

The paper adapts the HDP-SLDS to shared-matrix, stochastic-volatility, and target-tracking settings, while demonstrating flexible mode discovery and application-specific dynamics. These variants recover meaningful regime changes in financial data and support learned representations of honey-bee dances and maneuvering targets.

  • Shared Dynamic Matrix, Switching Driving Noise: Shared-matrix SLDS variants distinguish modes through switching process-noise means and covariances rather than separate dynamic matrices.The model uses independent priors on the shared matrix, mode-specific covariance, and mode-specific process-noise mean.
  • Stochastic Volatility: The MSSV variant models log-volatilities with an AR(1) process, mode-specific means, and Gaussian or mixture-of-Gaussians measurement noise.The paper examines log-squared daily returns and truncates the measurement-noise mixture to 10 components.
  • Stochastic Volatility: The sticky HDP-SLDS inferred change points similar to reported world events, whereas the non-sticky variant over-segmented the IBOVESPA data.The comparison includes HDP-AR(1)-HMM, HDP-AR(2)-HMM, and non-sticky HDP-SLDS ROC curves.
  • Stochastic Volatility: A raw-return HDP-SLDS captured stock-index regime changes and aligned its change points better with world events than the MSSV HDP-SLDS.This model does not interpret its latent state as log-volatility.
  • Applications: The paper applies switching dynamical models to honey-bee dance segmentation and to target tracking with known dynamics and mode-specific random control inputs.The honey-bee sequences contain waggle, turn-right, and turn-left dances; target tracking uses a jump-mean Markov control process.
  • Conclusion: The conclusion presents ARD as a sparsity-inducing prior for variable-order structure and emphasizes flexible Bayesian nonparametric models for discovering simple temporal structures.The authors report learning volatility changes, dance segmentations, and adaptations to stochastic-volatility and target-tracking formulations.
  • Conclusion: Batch Gibbs sampling can be impractical for some applications, and exact recursive estimation is infeasible under nonlinear dynamics and parameter uncertainty.The paper identifies Rao-Blackwellized particle filtering as a possible alternative but notes particle impoverishment as a challenge.

APPENDIX

The appendix specifies priors and posterior calculations for the paper’s switching dynamical models. It covers data-informed hyperparameters, conjugate MNIW inference, and the resulting inverse-Wishart covariance posterior.

  • Prior Settings: The general experiments use data-informed MNIW hyperparameters centered toward stable dynamic matrices, with separate measurement-noise covariance settings.For n = d, the appendix sets S0 = 0.675 times the empirical covariance and R0 = 0.075 times it.
  • Prior Settings: Partially supervised honey-bee experiments use tighter priors based on first-difference covariance, while unsupervised raw observations yield a less informative dynamic-matrix prior.The first-difference choice reflects the bee dynamics being better approximated as a random walk than as i.i.d. observations.
  • Prior Settings: IBOVESPA MSSV experiments use the N-IW-N prior, while the Table I HDP-SLDS comparison uses an MNIW prior with large expected measurement-noise covariance.The large R setting helps avoid initial regime chattering caused by the state sequence explaining observation noise.
  • Switching VAR Posterior: For a known mode sequence, switching VAR(r) observations are organized by mode into response and lagged-predictor data matrices for posterior inference.The appendix defines D(k) from the mode-specific response and lag matrices and considers both zero-mean MNIW and independent priors.
  • Conjugate Prior — MNIW: With a zero mode-specific mean, the likelihood and MNIW prior share the covariance parameter, preserving conjugate posterior structure for the dynamic matrix and covariance.The covariance marginal posterior is inverse-Wishart, and the dynamic matrix posterior is conditioned on that covariance.

B. Non-Conjugate Independent Priors on A(k), Σ(k), and µ(k)

The appendix derives posterior updates for the mode-specific parameters under independent priors, including non-conjugate ARD sparsity priors. It also describes blocked sampling of switching VAR modes and states, while noting conditions required to preserve sparsity under a fixed observation matrix.

  • Non-Conjugate Independent Priors: Independent priors are assigned separately to A(k), Σ(k), and µ(k), with posterior derivations provided for each parameter.The priors include Gaussian, inverse-Wishart, and Gaussian forms, while ARD is identified as a non-conjugate sparsity-inducing prior.
  • Non-Conjugate Independent Priors: The Gaussian prior on vec(A(k)) enables rewriting the observations as a linear function of the vectorized dynamics matrix before deriving its posterior.The construction uses autoregressive order r and observation dimension d to organize the design matrix.
  • Non-Conjugate Independent Priors: The inverse-Wishart prior on Σ(k) yields a conjugate posterior conditioned on A(k) and µ(k), while the Gaussian prior on µ(k) yields its conditional posterior.The covariance update uses nk, the number of observations assigned to mode k.
  • Non-Conjugate Independent Priors: The Gibbs sampler repeatedly samples A(k), Σ(k), and µ(k) before advancing, alongside forward-backward sampling of switching VAR mode sequences.The mode sequence is sampled sequentially from conditionals formed using transition probabilities, observations, and backward messages.
  • Non-Conjugate Independent Priors: ARD shrinks non-dynamical state components or autoregressive lags toward zero, but preserving sparsity after fixing C = [Id 0] requires constrained dynamical realizations.Criterion 3.1 gives sufficient, though not necessary, conditions for the transformed realization to retain sparsity.

B. State Sequence Message Passing for Blocked Sampling

State-sequence sampling combines backward Gaussian message passing with forward conditional sampling. For fixed modes, the messages remain analytically tractable and are computed through an information-form Kalman recursion.

  • State Sequence Message Passing: Backward messages for the continuous state sequence are Gaussian because the model uses Gaussian densities and a time-varying linear dynamical system.The message representation uses information-form Gaussian parameters for mean and covariance.
  • State Sequence Message Passing: The backward recursion propagates information through the observation model, transition matrix, transition covariance, and future message parameters.An equivalent numerically stable recursion is summarized in Algorithm 5.
  • State Sequence Message Passing: After backward messages are computed, the state sequence is sampled forward using the factorization of its posterior into successive conditional distributions.Each conditional combines the transition density, local observation likelihood, and backward message.
  • State Sequence Message Passing: For the HDP-SLDS, each forward conditional is Gaussian because it is the product of Gaussian transition, observation, and backward-message terms.The resulting normalization constant is analytically tractable for each time step.

C. Mode Sequence Message Passing for Sequential Sampling

Mode-sequence sampling uses forward and backward messages to evaluate each candidate mode conditional on the observations and neighboring mode. The appendix also derives stable information-form recursions for likelihood computation.

  • Mode Sequence Message Passing: The conditional probability of a mode combines its transition probability, local observation likelihood, and backward message.This supports sequential sampling of z1:T after decomposing the posterior by the chain rule.
  • Mode Sequence Message Passing: Forward messages are defined alongside backward messages, and their recursions are derived by expressing integrands in exponentiated-quadratic form.Gaussian marginalization over the continuous state produces the required message updates.
  • Mode Sequence Message Passing: Algorithm 6 provides a numerically stable forward Kalman information-filter recursion, but it relies on the dynamic matrix A(k) being invertible.The filter is initialized and propagated forward over the observation sequence.
  • Mode Sequence Message Passing: The mode likelihood is computed by integrating over the continuous state after combining forward and backward messages with local dynamics and measurements.The derivation tracks normalizing terms that could otherwise introduce an unintended dependence on the candidate mode.
  • Mode Sequence Message Passing: The derivation shows that the relevant ignored normalization constant is independent of zt, validating the mode-conditional likelihood calculation.This independence is established by examining the state-transition contribution and the resulting message recursion.

APPENDIX D DERIVATION OF MANEUVERING TARGET TRACKING SAMPLER

The maneuvering target tracking sampler jointly samples the control input and dynamical mode while marginalizing the state sequence, transition distributions, and dynamic parameters. Its conditional distribution is hybrid because each discrete mode induces a different continuous control-input distribution.

  • Maneuvering Target Tracking Sampler: The sampler targets the joint conditional distribution of the control input ut and dynamical mode zt given the remaining controls, modes, observations, and hyperparameters.The target factors into a conditional for zt and a conditional for ut given the complete mode sequence.
  • Maneuvering Target Tracking Sampler: State sequences, transition distributions, and dynamic parameters are marginalized to analyze the mode and control-input conditionals.The resulting distribution is hybrid, pairing discrete mode values with continuous distributions over ut.
  • Maneuvering Target Tracking Sampler: Conditioning on the control-input sequence reduces the SLDS to an LDS with deterministic controls, enabling state marginalization using a forward Kalman filter.The filter passes a message from t − 2 to t − 1 and incorporates the local dynamics and measurements.
  • Maneuvering Target Tracking Sampler: Integrating over transition probabilities produces the same predictive distribution as the sticky HDP-HMM.Dynamic parameters are marginalized using standard conjugacy under a normal-inverse-Wishart prior.
  • Maneuvering Target Tracking Sampler: The control-input conditional uses a moment-matched Gaussian approximation to the Student-t predictive distribution induced by marginalizing the mode-specific parameters.This approximation is applied separately for each candidate mode.

C. Marginalization by Message Passing

The section derives likelihood and sampling calculations for control inputs and modes by conditioning on other inputs, using message passing and Gaussian marginalization. The resulting conditional distributions combine mode-transition terms, control-input priors, and observation likelihoods.

  • C. Marginalization by Message Passing: Conditioning on all control inputs except u_t turns state marginalization into message passing through the conditionally linear dynamical system induced by the fixed inputs.The forward and backward messages are adapted by replacing process-noise means with B u_τ and corresponding covariance terms with Q.
  • C. Marginalization by Message Passing: The likelihood is computed by combining filtered estimates, local dynamics, and local likelihood terms, then integrating the resulting exponentiated quadratic form with standard Gaussian formulas.The derivation integrates successively over x_t and x_{t−1}.
  • F. Resulting (u_t, z_t) Sampling Distributions: Marginalizing model parameters induces dependencies among control inputs assigned to the same mode, so the prior for u_t is conditioned on the other inputs sharing mode z_t = k.The conditional control-input distribution is multiplied by the observation likelihood to form the quadratic expression used for sampling.
  • F. Resulting (u_t, z_t) Sampling Distributions: The joint sampling distributions separate mode assignment and control-input updates: z_t uses its conditional mode probability and marginalized normalizer, while u_t combines its mode-specific prior with the data likelihood.These distributions are written as the derived forms for p(z_t = k | ·) and p(u_t | z_t = k, ·).
Loading 1003.3829v1…