Source-linked AI summary
Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structures
Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, Jian Li
TL;DR
Complex forecasting architectures are computationally expensive for long, multivariate sequences, motivating simpler approaches. LightTS combines MLPs with continuous and interval sampling, achieving stronger or comparable benchmark performance while remaining efficient and robust.
Problem
Complex neural architectures make training and inference computationally expensive for long sequences with many correlated variables, while their many parameters can require substantial data.
Method
LightTS applies simple MLP-based structures on top of continuous and interval sampling, which target local and global temporal patterns while preserving most time-series information.
Results
LightTS outperforms state-of-the-art methods on 5 of 8 benchmark datasets and achieves state-of-the-art or comparable results in most cases.
Takeaways & Limitations
Down-sampling lets LightTS process only a fraction of very long input sequences, supporting accurate, efficient, and robust forecasting.
Abstract
from arXiv · showhide
Multivariate time series forecasting has seen widely ranging applications in various domains, including finance, traffic, energy, and healthcare. To capture the sophisticated temporal patterns, plenty of research studies designed complex neural network architectures based on many variants of RNNs, GNNs, and Transformers. However, complex models are often computationally expensive and thus face a severe challenge in training and inference efficiency when applied to large-scale real-world datasets. In this paper, we introduce LightTS, a light deep learning architecture merely based on simple MLP-based structures. The key idea of LightTS is to apply an MLP-based structure on top of two delicate down-sampling strategies, including interval sampling and continuous sampling, inspired by a crucial fact that down-sampling time series often preserves the majority of its information. We conduct extensive experiments on eight widely used benchmark datasets. Compared with the existing state-of-the-art methods, LightTS demonstrates better performance on five of them and comparable performance on the rest. Moreover, LightTS is highly efficient. It uses less than 5% FLOPS compared with previous SOTA methods on the largest benchmark dataset. In addition, LightTS is robust and has a much smaller variance in forecasting accuracy than previous SOTA methods in long sequence forecasting tasks.
1 INTRODUCTION
Multivariate forecasting must capture both temporal patterns within each variable and dependencies across variables, but complex neural architectures can be costly and data-hungry. LightTS addresses this challenge with MLP structures paired with continuous and interval sampling, achieving strong accuracy, efficiency, and robustness across benchmark tasks.
- Multivariate forecasting requires modeling short-term and long-term patterns in each variable alongside interdependencies among variables.
- Complex RNN-, GNN-, and Transformer-based architectures can make training and inference computationally expensive for long, highly multivariate inputs.
- LightTS uses simple MLP-based structures with continuous sampling for short-term local patterns and interval sampling for long-term dependencies.The design is motivated by the observation that down-sampling often preserves most time-series information.
- LightTS outperforms state-of-the-art methods on 5 of 8 benchmark datasets and uses less than 5% FLOPS of previous SOTA methods on the largest dataset.The reported results also cover both short- and long-sequence forecasting tasks.
- LightTS has a smaller variance in forecasting accuracy than previous SOTA methods in long-sequence forecasting tasks.
2 RELATED WORK
Prior forecasting research spans statistical models and increasingly complex deep-learning architectures, while recent work also explores down-sampling and pure MLP designs. LightTS is positioned within this progression as an MLP-based multivariate model that exchanges information across original and down-sampled sequences.
- 2.1 Statistical Methods: Statistical forecasting methods include AR, MA, ARMA, ARIMA, VAR, and Gaussian Processes, but often require strong assumptions and scale poorly to large multivariate data.
- 2.2 Deep-learning-based Methods: Deep-learning approaches use CNNs, RNNs, attention, or GNNs to model temporal dependencies and correlations among time series.
- 2.2 Deep-learning-based Methods: Long-sequence forecasting research targets efficiency and long-range dependencies, with models such as Informer and Autoformer improving Transformer-based approaches.
- 2.2 Deep-learning-based Methods: N-BEATS showed that a pure MLP structure can achieve competitive univariate forecasting performance, motivating MLP-based forecasting designs.
- 2.2 Deep-learning-based Methods: LightTS differs from computer-vision MLP architectures by exchanging information across both original inputs and down-sampled sub-sequences.
3 OUR MODEL: LIGHTTS
LightTS forecasts multivariate time series with MLP-based blocks organized around continuous and interval sampling. It separates temporal-feature extraction from cross-variable modeling while using bottleneck projections to reduce computation.
- Architecture: LightTS formulates forecasting from a look-back window of length T over N variables, targeting either multiple future timestamps or one future timestamp.The framework addresses both multi-step and single-step forecasting settings.
- Architecture: LightTS first processes each time series independently to extract temporal features, then concatenates variables and learns their interdependencies in a second stage.The two stages correspond to temporal-pattern extraction and cross-variable correlation modeling.
- Sampling: Two sampling strategies preserve complementary temporal information: continuous sampling emphasizes short-term local patterns, while interval sampling emphasizes long-term global patterns.Both strategies retain the original tokens rather than eliminating them through naïve down-sampling.
- Information Exchange Block: An IEBlock maps an H×W input matrix to an F×W feature map through temporal, channel, and output projections.Temporal projection extracts temporal features, channel projection exchanges information across channels, and output projection restores the desired feature dimension.
- Information Exchange Block: The bottleneck design first reduces the temporal dimension from H to F′, where F′ ≪ F, before projecting back to F to lower computation.Weight sharing across columns and rows further supports efficiency in the temporal and channel projections.
- Architecture: After independent temporal processing, LightTS concatenates continuous and interval features across time and variables, then applies IEBlock-C to combine local, global, and cross-variable information.The final block maps an input of shape R^(2F)×N to R^L×N.
4 EXPERIMENTS
Experiments on eight benchmark datasets evaluate LightTS across forecasting accuracy, computational efficiency, and robustness. LightTS achieves state-of-the-art or comparable accuracy while substantially reducing computational cost and prediction variance.
- Experimental Setup: Eight public benchmark datasets cover short-sequence single-step and long-sequence multi-step forecasting settings.The evaluation uses RSE and CORR for short-sequence forecasting, and MSE and MAE for long-sequence forecasting.
- Long Sequence Forecasting: LightTS achieves state-of-the-art results across all horizons on ETTh1, ETTh2, ETTm1, and Electricity, and second-best results on Weather.For the longest horizon, MSE decreases by 9.21%, 33.90%, 34.18%, and 13.60% on ETTh1, ETTh2, ETTm1, and Electricity, respectively.
- Short Sequence Forecasting: LightTS achieves state-of-the-art results on Solar-Energy and comparable results to MTGNN and SCINet on Traffic, Electricity, and Exchange-Rate.On Solar-Energy, RSE decreases by 4.16%, 4.61%, 3.90%, and 2.73% at horizons 3, 6, 12, and 24, respectively.
- Efficiency: On Traffic, LightTS uses 96.2% fewer FLOPS than MTGNN and 99.4% fewer than SCINet, with 44.5x and 13.8x running-time speedups per epoch.On Electricity, LightTS uses 93.5% fewer FLOPS than SCINet and 97.2% fewer than Autoformer.
- Robustness: LightTS has substantially smaller prediction-accuracy variance and shaded forecasting ranges across random seeds than Autoformer and SCINet.The robustness comparison uses standard deviations on ETTh1, ETTm1, Weather, and Electricity, with five different random seeds shown in the forecasting ranges.
- Ablation Study: Ablations show channel projection helps Solar-Energy and Electricity but can hurt Exchange-Rate, while continuous and interval sampling capture short-term and long-range patterns, respectively.The sampling effects are reported across all datasets and illustrated using daily and weekly patterns in Traffic.
5 DISCUSSION: STUDY OF THE CHANNEL PROJECTION
LightTS’s channel projection is the only module that communicates across time series, so the study evaluates whether this simple linear layer captures interdependency adequately. Deep SHAP quantifies LightTS’s modeled dependencies, which are compared with MTGNN’s using Pearson correlation.
- Interdependency Modeling: Channel projection is LightTS’s sole module for communicating information among different time series.In the implementation, it is a simple linear layer.
- Interdependency Quantification: Deep SHAP quantifies how each input time series contributes to each predicted time series in LightTS.The method uses additive feature attributions derived from DeepLIFT and SHAP.
- Interdependency Quantification: The study represents LightTS interdependencies with matrix E, whose entries summarize contributions from input series to predicted series.Each vector e_k contains the attribution values associated with prediction y_k.
- Comparison Measure: MTGNN interdependencies provide the comparison reference because real interdependencies are undefined for many multivariate time series datasets.MTGNN explicitly learns graph structure during training and is described as a state-of-the-art forecasting method.
- Results: LightTS’s modeled interdependency highly correlates with MTGNN’s across eight experimental cases.The paper uses Pearson correlation between corresponding interdependency vectors as the comparison metric.
- Results: The empirical findings indicate that a simple channel projection is sufficient for learning interdependency among different time series.The conclusion follows from the reported correlation with MTGNN’s modeled interdependencies.
6 CONCLUSION
The conclusion presents LightTS as an MLP-based forecasting model that combines continuous and interval sampling. These sampling strategies target different temporal patterns while reducing the sequence processed by the model, supporting accuracy, efficiency, and robustness.
- Model: LightTS is a simple multivariate forecasting model based on multi-layer perceptrons and two down-sampling strategies.The two strategies are continuous sampling and interval sampling.
- Sampling Strategies: Continuous and interval sampling help capture short-term local patterns and long-term temporal dependencies.The paper links the sampling choices to the distinct temporal structures in forecasting inputs.
- Efficiency: Down-sampling lets LightTS process only a fraction of the input sequence, improving efficiency for very long inputs.This efficiency claim concerns the amount of sequence handled after down-sampling.
- Conclusion: Extensive experiments report that LightTS is accurate, efficient, and robust on short- and long-sequence multivariate forecasting tasks.The conclusion states this result across both forecasting settings.
A.1 Datasets Description
The evaluation uses benchmark datasets including the ETT collection, which records two years of electric-power data from two Chinese counties. The ETT datasets differ in sampling frequency and contain seven variables per timestamp.
- Dataset Overview: Table 1 summarizes the datasets used to evaluate forecasting performance.The passage introduces the dataset descriptions accompanying the table.
- ETT Dataset: The ETT datasets contain two years of electric-power data collected in two different counties in China.The collection includes ETTm1, ETTh1, and ETTh2.
- ETT Dataset: ETTm1 is sampled every 15 minutes, whereas ETTh1 and ETTh2 are sampled hourly.These sampling rates distinguish the minute- and hour-level ETT datasets.
- ETT Dataset: Each timestamp in the three ETT datasets contains seven variables.The ETT datasets are used to evaluate long-sequence forecasting abilities.
A.2 Evaluation Metrics
The evaluation follows prior work by using mean squared error and mean absolute error as forecasting accuracy metrics.
- Metrics: Forecasting accuracy is evaluated with Mean Squared Error (MSE) and Mean Absolute Error (MAE).The paper states that these metrics follow the definitions used in prior work.
A.2.1 Long Sequence Forecasting.
This section defines the forecasting metrics used for evaluation and describes the Deep SHAP procedure used to explain LightTS predictions.
- MSE and MAE evaluate predictions against ground truth, with lower values indicating better performance.The ground truth and predictions have dimensions determined by output length and number of variables.
- RSE and CORR provide additional evaluation measures, where lower RSE and higher CORR indicate better performance.
- Deep SHAP combines DeepLIFT and Shapley values to explain LightTS predictions.
- The explanation procedure uses 100 randomly selected training samples as the background dataset and averages input-element SHAP values across 100 randomly selected test samples.
B ADDITIONAL EXPERIMENT RESULTS
This section presents additional baseline comparisons for long-sequence, multi-step time-series forecasting and specifies the evaluation conventions used for the reported results.
- Table 8 reports additional baseline comparisons under the multi-step setting for long-sequence time-series forecasting tasks.
- Results from LogTrans through Informer are taken from prior work, while SCINet and AutoFormer averages use five random seeds.
- The comparisons follow the same look-back settings as Informer and SCINet for each dataset.
- Within each setting, the best result is bolded and the second-best result is italicized.