PaperlayerView exact revision on arXiv

Source-linked AI summary

Bayesian Temporal Factorization for Multidimensional Time Series Prediction

Xinyu Chen, Lijun Sun

arXiv:1910.06366v2stat.MLcs.LG

TL;DR

Large-scale, multidimensional spatiotemporal series are difficult to predict because they contain complex structure and substantial missing data. BTF integrates low-rank factorization with VAR dynamics in a Bayesian graphical model, and experiments evaluate imputation and rolling prediction on real-world data. The reported experiments demonstrate superiority over existing state-of-the-art methods while producing probabilistic predictions and uncertainty estimates.

  • Problem

    Large-scale, high-dimensional spatiotemporal time series contain substantial missing data, making prediction difficult for classical modeling frameworks.

  • Method

    BTF integrates low-rank matrix/tensor factorization with a VAR process for temporal factors in a single Bayesian graphical model.

  • Results

    The numerical experiments demonstrate superiority over existing state-of-the-art methods on missing-data imputation and multi-step rolling prediction tasks.

  • Takeaways & Limitations

    BTF supports imputation and prediction for incomplete multidimensional time series while providing probabilistic predictions and uncertainty estimates.

  • Takeaways & Limitations

    Rolling prediction updates only temporal factors, and the authors expect significant error accumulation when γ is small.

Abstract

from arXiv · show

Large-scale and multidimensional spatiotemporal data sets are becoming ubiquitous in many real-world applications such as monitoring urban traffic and air quality. Making predictions on these time series has become a critical challenge due to not only the large-scale and high-dimensional nature but also the considerable amount of missing data. In this paper, we propose a Bayesian temporal factorization (BTF) framework for modeling multidimensional time series -- in particular spatiotemporal data -- in the presence of missing values. By integrating low-rank matrix/tensor factorization and vector autoregressive (VAR) process into a single probabilistic graphical model, this framework can characterize both global and local consistencies in large-scale time series data. The graphical model allows us to effectively perform probabilistic predictions and produce uncertainty estimates without imputing those missing values. We develop efficient Gibbs sampling algorithms for model inference and model updating for real-time prediction and test the proposed BTF framework on several real-world spatiotemporal data sets for both missing data imputation and multi-step rolling prediction tasks. The numerical experiments demonstrate the superiority of the proposed BTF approaches over existing state-of-the-art methods.

1 INTRODUCTION

Modern sensing produces large-scale, high-dimensional time series with complex dependencies and missing values, challenging classical models. BTF combines Bayesian factorization with VAR dynamics to support imputation, prediction, and uncertainty estimation.

  • Classical VAR models are not scalable to large data sets, while multidimensional structure complicates modeling higher-order dependencies and temporal dynamics.
  • Matrix/tensor factorization addresses scalability and missing data by representing time series through shared low-rank latent factors.
  • Existing factorization methods require expensive, data-specific regularization tuning and often provide only point estimates or imputation rather than predictive uncertainty.
  • BTF integrates low-rank matrix/tensor factorization and a VAR process for temporal factors in one Bayesian graphical model.
  • The framework uses MCMC inference, avoids regularization-parameter tuning, supports probabilistic predictions with uncertainty estimates, and is evaluated on real-world spatiotemporal data.

2 RELATED WORK

Related work models large incomplete multivariate and multidimensional time series in lower-dimensional latent spaces, using smoothness, dynamical systems, Gaussian processes, or Bayesian factorization. BTF combines Bayesian treatment with a more flexible VAR process.

  • Latent-space modeling reduces the parameter burden for large incomplete series, using R factors with R ≪ N instead of traditional VAR with approximately N^2 parameters.
  • Standard matrix factorization captures global low-rank structure but does not capture temporal dependencies or local patterns across time.
  • Prior approaches impose temporal smoothness or use linear dynamical systems and state-space models to represent latent temporal dynamics.
  • Gaussian-process dynamical models capture latent dynamics but are computationally expensive for large data and require complex hyperparameter estimation.
  • Most factorization models provide point estimates, motivating Bayesian methods that address overfitting and parameter-tuning problems while representing uncertainty.
  • BTF addresses regularization tuning and uncertainty estimation, replacing independent AR assumptions with a more flexible VAR process.

3 PROBLEM DESCRIPTION

The paper formulates incomplete spatiotemporal observations as matrix or tensor time series indexed over locations, attributes, and time. Prediction uses observed historical entries to estimate missing or future values.

  • Matrix time series represent measurements from N locations or sensors across T time stamps, with each row corresponding to one location or sensor.
  • Tensor time series represent multidimensional data such as origin-destination travel demand across M origins, N destinations, and T time points.
  • The prediction problem separates historical values from near-future values while allowing missing observations in the time series.
  • Observed matrix entries are indexed by (i,t) ∈ Ω, while observed tensor entries are indexed by (i,j,t) ∈ Ω.
  • Accurate prediction on incomplete time series is challenging because missing data are nearly inevitable in real-world applications.

4 BAYESIAN TEMPORAL MATRIX FACTORIZATION

BTMF factorizes partially observed matrix time series into spatial and temporal factors, then models temporal factors with a VAR process. Its Bayesian formulation supports inference from observed entries, probabilistic prediction, and uncertainty estimation while addressing limitations of independent-factor regularization.

  • Model specification: BTMF factorizes a partially observed matrix Y into spatial factor matrix W and temporal factor matrix X, with observed entries modeled probabilistically.The model is built only on observed data in Ω, so missing values need not be imputed before training.
  • Temporal dynamics: Unlike an independent-factor AR regularizer, the VAR formulation can represent cross-factor temporal dependencies, causal relationships, and covariance structure.The independent-factor assumption in TRMF reduces parameters but ignores these richer temporal relationships.
  • Temporal dynamics: The temporal factors follow a Gaussian VAR process whose coefficient matrices capture dependencies across factors and time lags.The process uses coefficient matrices A_k of size R × R and Gaussian innovations.
  • Noise modeling: BTMF uses spatially varying precisions τ_i rather than isotropic noise, allowing different time series to have different noise levels.This relaxes the assumption that all series share a common precision.
  • Bayesian inference: Conjugate Gaussian-Wishart, matrix normal inverse Wishart, and Gamma priors yield analytically tractable conditional distributions for Gibbs sampling.The implementation specifies weak, non-informative priors for the model parameters and hyperparameters.
  • Online prediction: The rolling prediction procedure updates temporal factors rather than fully retraining the model, improving real-time efficiency but potentially accumulating prediction error.Single-site temporal-factor updates may also mix slowly when T is large.

5 BAYESIAN TEMPORAL TENSOR FACTORIZATION

Bayesian Temporal Tensor Factorization extends the framework to tensor time series using CP decomposition, Gaussian observations, and VAR dynamics for temporal factors. The model supports inference, missing-data imputation, and rolling prediction under spatially varying or isotropic noise assumptions.

  • BTTF extends BTMF to multidimensional tensor time series using a third-order tensor example.
  • Model Specification: CP decomposition approximates the tensor as a sum of R rank-one tensors formed from factor matrices U, V, and X.The factors correspond to the tensor’s two spatial or feature dimensions and its temporal dimension.
  • Model Specification: Each tensor element follows a Gaussian distribution with spatially varying precision τij, while isotropic precision τ is available as a simplifying assumption.The isotropic option is motivated for large, sparse, heterogeneous tensors where estimating each series’ noise level is challenging.
  • Model Specification: The temporal factor matrix X is modeled with the same VAR process used in BTMF, with Gaussian-Wishart priors on the corresponding hyperparameters.
  • Model Specification: BPTF is a special case of BTTF obtained with one lag, an identity transition matrix, and isotropic noise.
  • Inference and Prediction: BTTF inference samples the temporal and spatial factor matrices through Gaussian posteriors and adapts BTMF procedures for precision sampling, imputation, and rolling prediction.The tensor model uses conditional posteriors for xt, ui, and vj, with the same Gaussian-Wishart forms for several hyperparameters.

6 EXPERIMENTS

Experiments evaluate BTMF and BTTF for missing-data imputation and multi-step rolling prediction on real-world spatiotemporal data. Results generally favor Bayesian temporal factorization, while performance varies by dataset and comparison model.

  • Experiment setup: Experiments measure imputation and prediction with MAPE and RMSE across several real-world spatiotemporal datasets.Imputation uses random and non-random missingness; prediction uses rolling forecasts over the last seven days with multiple horizons.
  • Datasets: The experiments include urban traffic, metro passenger flow, London movement speed, NYC taxi, and Pacific surface temperature data.The datasets span matrix and tensor formats, including a 220,000-road-segment London movement dataset and a 30 × 30 × 1464 NYC taxi tensor.
  • BTMF imputation: BTMF and adapted BTRMF outperform TRMF in most imputation cases, with non-random missingness generally harder to reconstruct than random missingness.The comparison reports MAPE/RMSE results for datasets (G), (H), (S), and (L).
  • BTMF prediction: BTMF outperforms TRMF and BTRMF in most multi-step prediction cases, although comparisons with tuned TRMF may be unfair because BTRMF was run once with non-informative priors.Prediction evaluates rolling forecasts over the last seven days at horizons δ = 2, 4, and 6.
  • BTMF prediction: BTMF consistently outperforms BTRMF, supporting the use of VAR dynamics rather than independent temporal-factor assumptions.The models share identical priors; their difference is that BTRMF restricts the coefficient matrix to be diagonal, whereas BTMF integrates VAR dynamics.
  • BTTF: For tensor data, BTTF is competitive for imputation and performs better than BTRTF for prediction, while HaLRTC performs best on Pacific temperature imputation.BTTF and BTRTF are comparable on the two tensor datasets, whereas the Pacific data exhibit a clear low-rank structure.

7 CONCLUSION AND FUTURE WORK

The paper presents BTF, which combines Bayesian matrix/tensor factorization with a VAR layer to model incomplete multidimensional time series. Experiments report superior performance over baseline models, while future work targets adaptive rank, spatial dependence, richer distributions, robustness, and nonlinear dynamics.

  • Conclusion: BTF incorporates a VAR layer into Bayesian probabilistic matrix/tensor factorization for multidimensional time series.The integration models temporal dynamics and covariance structure on latent dimensions.
  • Conclusion: The Bayesian framework supports imputation and prediction of incomplete or corrupted time series while estimating posterior distributions for target variables.The posterior estimates are identified as important for risk-sensitive applications.
  • Conclusion: An efficient, scalable Gibbs sampling algorithm with conjugate priors provides model inference and supports flexible parameter tuning while avoiding overfitting issues.
  • Conclusion: BTF demonstrates superior performance over other baseline models on several real-world time series matrices and tensors.
  • Future Work: Future work includes learning the latent rank, modeling spatial dependencies, incorporating exogenous variables, supporting non-Gaussian and robust models, and capturing nonlinear dynamics.Proposed extensions include spatial AR or Gaussian processes, alternative temporal Gaussian processes, particle filtering, and deep learning.
Loading 1910.06366v2…