Source-linked AI summary
Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting
Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, Yongjun Xu
TL;DR
MTS forecasting must distinguish samples with similar histories but different futures, a challenge motivating alternatives to increasingly sophisticated STGNNs. The paper proposes STID, which attaches spatial-temporal identities and uses simple MLPs; it reports better performance and efficiency simultaneously. The results support addressing sample indistinguishability without being limited to STGNNs.
Problem
Similar historical samples can have different futures across spatial and temporal dimensions, while recent STGNNs have become more sophisticated with limited performance improvements.
Method
STID attaches spatial and temporal identity embeddings, then uses MLP layers and a regression layer for forecasting.
Results
STID achieves better performance and efficiency simultaneously than STGNN-based methods using simple networks.
Takeaways & Limitations
Addressing sample indistinguishability can support efficient and effective MTS forecasting models without limiting design to STGNNs.
Abstract
from arXiv · showhide
Multivariate Time Series (MTS) forecasting plays a vital role in a wide range of applications. Recently, Spatial-Temporal Graph Neural Networks (STGNNs) have become increasingly popular MTS forecasting methods due to their state-of-the-art performance. However, recent works are becoming more sophisticated with limited performance improvements. This phenomenon motivates us to explore the critical factors of MTS forecasting and design a model that is as powerful as STGNNs, but more concise and efficient. In this paper, we identify the indistinguishability of samples in both spatial and temporal dimensions as a key bottleneck, and propose a simple yet effective baseline for MTS forecasting by attaching Spatial and Temporal IDentity information (STID), which achieves the best performance and efficiency simultaneously based on simple Multi-Layer Perceptrons (MLPs). These results suggest that we can design efficient and effective models as long as they solve the indistinguishability of samples, without being limited to STGNNs.
1 INTRODUCTION
MTS forecasting involves interrelated spatial-temporal series, but similar historical samples can have different futures across sensors or periods. STID addresses this indistinguishability by attaching spatial-temporal identities and using simple MLP-based prediction.
- MTS data contains multiple interrelated time series and supports forecasting in applications including transportation and energy.
- Similar historical data can correspond to different future data across variables or time periods, making those samples indistinguishable to simple regression models.The examples compare sensors 29 and 301 within one window and sensor 301 across different windows.
- STID attaches spatial and temporal identity information through one spatial embedding matrix and two temporal embedding matrices.The temporal identities represent time slots within a day and days within a week.
- STID encodes the resulting information with simple MLP layers and produces forecasts through a regression layer.
- The paper reports that STID is more powerful than STGNN-based methods while offering significant efficiency advantages.The proposed direction is to address sample indistinguishability without being limited to STGNNs.
2 PRELIMINARIES
MTS forecasting predicts future values of multiple interrelated time series from historical signals. The paper formalizes spatial and temporal identities as trainable embeddings for variables, daily time slots, and weekly days.
- Given historical signals X ∈R^P×N, MTS forecasting predicts values Y ∈R^F×N for the F nearest future time slots.The full series is represented as X ∈R^T×N, where T is the number of time slots and N is the number of variables.
- Spatial and temporal identities are represented by trainable embeddings for N time series, N_d daily time slots, and N_w = 7 weekly days.The embeddings share hidden dimension D and are stored in E, TTiD, and TDiW.
3 MODEL ARCHITECTURE
STID combines an embedding layer, MLP layers, and a regression layer to forecast multivariate time series using spatial and temporal identity information.
- Architecture: STID comprises an embedding layer, multiple MLP layers, and a regression layer.The embedding layer transforms raw historical time series before subsequent processing.
- Identity embeddings: The model attaches spatial and temporal identities to the embedded historical representation.These identities are represented by spatial and temporal embeddings associated with variables and time patterns.
- MLP encoding: STID uses multiple MLP layers with residual connections to encode the representation.The model optimizes the identity parameters and fully connected layers jointly.
- Prediction and training: A regression layer converts the encoded representation into the forecast, while MAE serves as the loss function.The parameters are trained by minimizing the MAE objective through backpropagation and gradient descent.
4 EXPERIMENTS
The experiments use five established multivariate time series datasets and compare STID with traditional and deep-learning baselines.
- Datasets: Experiments evaluate five commonly used datasets: PEMS04, PEMS07, PEMS08, PEMS-BAY, and Electricity.Dataset statistics are summarized in Table 1.
- Datasets: Four traffic datasets include predefined graphs indicating dependencies among their time series.The datasets are PEMS04, PEMS07, PEMS08, and PEMS-BAY.
- Baselines: The baseline set includes traditional methods such as VAR and HI, alongside typical deep-learning methods with official public code.The passage introduces the baseline selection but does not enumerate all deep-learning baselines.
MLP MLP MLP Layers
STID combines spatial-temporal identities with MLP-based processing and is evaluated for forecasting performance, efficiency, ablations, and learned embedding structure.
- Architecture: STID uses multiple MLP layers and a regression layer within its forecasting architecture.The implementation sets the hidden dimension to 32 and uses three MLP layers.
- Performance Study: STID consistently achieves the best forecasting performance in almost all horizons across five datasets without requiring a predefined graph.The evaluated datasets are PEMS04, PEMS07, PEMS08, PEMS-BAY, and Electricity.
- Efficiency Study: STID achieves the best efficiency on all datasets because its concise architecture avoids graph convolutions and sequential models.Previous STGNN-based methods become more costly as input length or variable count increases.
- Ablation Study: Ablation results identify spatial identity as the most important component, while daily and weekly temporal identities also improve forecasting.The ablations are conducted on PEMS04 and report average results across metrics.
- Visualization: Learned spatial identities tend to cluster variables, while temporal identities reveal daily periodicity, adjacent-slot similarity, and weekday–weekend differences.The visualization uses PEMS08 embeddings, including 170 variables, 288 daily time slots, and seven weekdays.
5 CONCLUSION
The paper identifies spatial and temporal sample indistinguishability as a key MTS forecasting bottleneck and proposes STID to address it. STID achieves improved performance and efficiency with simple networks, suggesting that effective forecasting models need not be limited to STGNNs.
- Spatial and temporal indistinguishability of samples is identified as a key bottleneck in MTS forecasting.
- STID attaches spatial and temporal identity information to provide a simple baseline for MTS forecasting.
- STID achieves better efficiency and performance simultaneously using simple networks.
- The results suggest that solving sample indistinguishability allows model design beyond exclusive reliance on STGNNs.