PaperlayerView exact revision on arXiv

Source-linked AI summary

Poisson-Gamma Dynamical Systems with Time-varying Transition Dynamics

Jiahao Wang, Yijun Wang, Nan Fang, Sikun Yang

arXiv:2609.00896v1cs.LG

TL;DR

Count time-series models need to handle evolving latent dependencies that static PGDS transition kernels cannot represent. The paper proposes TV-PGDS with time-evolving transition matrices, three Dirichlet Markov chains, and conjugate Gibbs inference, and reports improved prediction for missing and future observations relative to related models.

  • Problem

    Static PGDS transition kernels limit modeling of shifting dependencies in real-world count sequences, whose dynamics can vary over time.

  • Method

    TV-PGDS models time-varying transition dynamics with three Dirichlet Markov chains and uses Dirichlet-Multinomial-Beta augmentation for conjugate Gibbs inference.

  • Results

    TV-PGDS achieves improved performance in data smoothing and forecasting tasks, while inferred transition matrices reveal evolving latent dependency patterns.

  • Takeaways & Limitations

    Time-evolving transition matrices provide a way to learn changing dependency structures in count sequences while retaining efficient posterior simulation.

Abstract

from arXiv · show

Bayesian methodologies for handling count-valued time series have gained prominence due to their ability to infer interpretable latent structures and to estimate uncertainties. Among these Bayesian models, Poisson-Gamma Dynamical Systems (PGDSs) are proven to be effective in capturing the evolving dynamics underlying observed count sequences. However, the state-of-the-art PGDS still falls short in capturing the transition dynamics that are commonly observed in real-world count time series. To mitigate this limitation, a PGDS with time-varying transition kernel (TV-PGDS), is proposed to allow the underlying transition matrices to evolve over time. Three specifically-designed Dirichlet Markov chains (Dir-Dir, Dir-Gam-Dir, PR-Gam-Dir) are constructed to accommodate heterogeneous structural mutations within these dependencies. Leveraging Dirichlet-Multinomial-Beta data augmentation techniques, a fully-conjugate and efficient Gibbs sampler is developed to perform posterior simulation. Experiments show that, in comparison with related models, the proposed PGDS achieves improved predictive performance due to its capacity to learn time-varying dependency structure captured by the time-evolving transition matrices.

I. INTRODUCTION

Count time series are difficult to model because they are often overdispersed, sparse, and high-dimensional, while existing PGDS models use static latent interactions. TV-PGDS addresses this gap with time-evolving transition matrices, flexible Dirichlet Markov chains, and conjugate Gibbs inference.

  • Count time series are often overdispersed, sparse, and high-dimensional, limiting the suitability of widely used linear dynamical systems.
  • PGDS uses gamma Markov chains to evolve latent states and provides interpretable dynamics, uncertainty estimation, and tractable Gibbs sampling.
  • Static PGDS transition kernels limit its ability to represent shifting latent dependencies across time.
  • TV-PGDS introduces time-evolving transition matrices to capture time-varying transition dynamics in observed count series.
  • Three Dirichlet Markov chains increase TV-PGDS flexibility for modeling complex transition dynamics.
  • Dirichlet-Multinomial-Beta augmentation yields fully conjugate, efficient Gibbs samplers for posterior simulation.
  • Experiments on four real-world datasets evaluate missing-observation and future-observation prediction and explore inferred latent structures.

II. RELATED WORK

Related work includes Gaussian state-space, gamma-Poisson, and gamma-rate Markov models for count sequences. These approaches address different modeling needs, but existing PGDS extensions remain constrained in continuously evolving non-stationary environments.

  • Gaussian linear dynamical systems can be computationally expensive and poorly suited to bursty count sequences.Link-function LDS approaches incur O((K+V)^3) complexity, while Gaussian state assumptions hinder efficient inference for count data.
  • GP-DPFA models count data with gamma-Poisson structure but does not learn interactions among latent dimensions.
  • PGDS captures latent transition dynamics through a gamma Markov chain whose transition matrix represents excitation between latent factors.
  • Switching PGDS models nonlinear dynamics through discrete switching among finite transition regimes.
  • DPGDS and SPGDS retain stationary constraints, making them inadequate for continuously evolving non-stationary environments.
  • Gamma-rate Markov models extend dynamics by chaining the gamma distribution's rate parameter, with hierarchical variants adding auxiliary variables.
  • Rate-parameter dynamics can produce high volatility because the rate governs variance, risking numerical instability or model collapse.

III. TIME-VARYING POISSON-GAMMA DYNAMICAL SYSTEMS

TV-PGDS replaces PGDS’s static transition kernel with transition matrices that evolve across sub-intervals while remaining stationary within each one. Three Dirichlet Markov constructions provide different mechanisms for smoothing, mutation, interaction, and burstiness.

  • Time-varying transition dynamics: The model divides time into I equally-spaced sub-intervals of M steps, with latent factors evolving each step and transition matrices changing between intervals.Transition dynamics are assumed static within each sub-interval but evolve over sub-intervals.
  • Time-varying transition dynamics: Local stationarity reduces overfitting risk, while M controls the trade-off between dynamic granularity and computational or convergence costs.M = 1 gives maximum temporal granularity but can increase computational costs or destabilize convergence.
  • Dirichlet Markov constructions: The Dir-Dir Markov chain inherits information from the previous interval and adapts to current data, with η controlling transition-matrix variance.Its prior allows transition dynamics to change smoothly across sub-intervals.
  • Dirichlet Markov constructions: The three proposed constructions are illustrated as Dir-Dir, Dir-Gam-Dir, and PR-Gam-Dir diagrams, with PR-Gam-Dir equivalently represented by Eq.(7).The constructions are designed to accommodate heterogeneous structural changes in evolving transition dependencies.
  • Dirichlet Markov constructions: Dir-Gam-Dir incorporates mutation and interactions among transition-matrix components, improving flexibility over Dir-Dir smoothing for complicated dynamics.Its construction captures correlations between dimensions of transition matrices over time.
  • Dirichlet Markov constructions: PR-Gam-Dir uses a Poisson-randomized gamma distribution to model component interactions and encode inductive bias for sparsity and burstiness.Its Poisson latent variable decouples component ratios, potentially allowing large mutations in latent distributions.

IV. MARKOV CHAIN MONTE CARLO INFERENCE

Inference for TV-PGDS uses augmentation identities and conjugacy to construct Gibbs updates for latent variables and time-varying transition matrices. The sampler exploits Poisson, multinomial, beta, gamma, Bessel, and related distributions to obtain tractable posterior steps.

  • Augmentation: The Gibbs sampler repeatedly applies augmentation lemmas, including an NB-CRT identity that converts a negative-binomial/CRT representation into sum-logarithmic and Poisson variables.This identity provides the auxiliary-variable representation used in posterior inference.
  • Augmentation: A beta augmentation transforms Dirichlet-multinomial variables into beta and negative-binomial conditional representations for efficient sampling.Conditioning on q gives n_k ∼ NB(r_k, q).
  • Latent-variable updates: Auxiliary variables are sampled backward through time, after which Poisson-gamma conjugacy yields latent-factor updates.The backward procedure initializes l(T +1)·k and ζ(T +1) at zero before sampling from t = T down to 2.
  • Transition-matrix updates: Dirichlet-multinomial conjugacy supports transition-matrix sampling by relating interval-level count vectors to Dirichlet-multinomial distributions.The PR-Gam-Dir sampler is explicitly illustrated, while algorithms for the other constructions are deferred to the appendix.
  • PR-Gam-Dir updates: The PR-Gam-Dir posterior for its gamma variables has closed-form Bessel updates when εα > 0, while related shifted confluent hypergeometric distributions are also efficiently samplable.The derivation also treats εα = 0 separately to avoid an absorbing condition.

V. EXPERIMENTS

Experiments evaluate TV-PGDS on four real-world count datasets through predictive and exploratory analyses. The results compare baseline models and Dirichlet Markov-chain variants while examining inferred latent factors and time-varying transition matrices.

  • Predictive Analysis: Experiments use four real-world datasets and evaluate data smoothing and forecasting against GP-DPFA, PGDS, GMC-RATE, GMC-HIER, and BGAR baselines.The datasets are ICEWS, NIPS, USEI, and COVID-19; Table I reports predictive-analysis results for smoothing and forecasting.
  • Predictive Analysis: TV-PGDS exhibits improved performance in both data smoothing and forecasting, although Dir-Gam-Dir and PR-Gam-Dir are not consistently numerically superior across datasets and tasks.The authors attribute the overall improvement to transition kernels adapting to non-stationary environments and state that construction choice should depend on data features.
  • Exploratory Analysis: On NIPS, TV-PGDS transition matrices evolve from near-identity matrices into two-block diagonal structures capturing changing topic correlations.The first block primarily reflects neural-network topics, while the second reflects statistical learning and Bayesian methods.
  • Exploratory Analysis: TV-PGDS recovers interpretable latent factors, including ICEWS factors associated with the Iraq war and Six-Party Talks and evolving NIPS topic trends.The exploratory analysis uses ICEWS and NIPS to demonstrate explainable latent structure.
  • Exploratory Analysis: The PGDS transition matrix is averaged over time and therefore cannot capture the informative time-varying transition dynamics shown by TV-PGDS.Figure 6 compares the static PGDS matrix with TV-PGDS matrices inferred at different times.
  • Dirichlet Markov Chains: Compared with Dir-Dir, Dir-Gam-Dir captures more complicated transition patterns, while PR-Gam-Dir additionally produces sparser patterns.Dir-Dir changes smoothly and may lack flexibility for rapid dynamics; Figure 7 compares the first four ICEWS transition matrices across variants.

VI. CONCLUSION

The conclusion presents time-varying transition matrices and Dirichlet Markov chains as mechanisms for capturing changing dynamics in count sequences. It also identifies automatic change-point detection and additional temporal applications as future directions.

  • Time-varying transition matrices are proposed to capture time-varying dynamics observed in count sequences.
  • Figure 8 reports mean absolute error for smoothing and forecasting on ICEWS and COVID-19 datasets across different sub-interval lengths.
  • Future work includes automatically finding change points so sub-interval lengths can be determined automatically.
  • Future applications include temporal social networks and irregularly observed social event data.

VII. APPENDIX A: MCMC INFERENCE

The appendix develops MCMC inference by recursively marginalizing latent states and introducing auxiliary variables for backward sampling. It uses Poisson-gamma and Dirichlet-multinomial conjugacy to obtain tractable posterior updates for the model components.

  • Poisson-gamma conjugacy yields posterior updates for count-related and gamma-distributed quantities.
  • The inference procedure recursively marginalizes latent states backward in time to decouple the Markov chain.
  • Auxiliary variables are introduced and sampled backward from the terminal time to support posterior inference.
  • Dirichlet-multinomial conjugacy yields posterior distributions for transition-related Dirichlet variables in the Markov-chain constructions.
  • The appendix separately describes inference for Dirichlet-Dirichlet and Dirichlet-Gamma-Dirichlet Markov chains.
Loading 2609.00896v1…