Source-linked AI summary
TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Long
TL;DR
Intricate, overlapping temporal patterns make variation modeling from one-dimensional time series challenging. TimesNet adaptively transforms series into period-based two-dimensional tensors and models their intra- and interperiod variations, achieving consistent state-of-the-art performance across five mainstream analysis tasks.
Problem
Intricate, overlapping temporal patterns make temporal variation modeling from one-dimensional time series challenging, although these variations convey informative properties such as continuity, periodicity, and trends.
Method
TimesNet uses TimesBlock to discover multiple periods, transform one-dimensional series into two-dimensional tensors, and capture intra- and interperiod variations with a parameter-efficient inception block.
Results
TimesNet achieves consistent state-of-the-art performance across five mainstream tasks, including forecasting, imputation, classification, and anomaly detection.
Takeaways & Limitations
The results support TimesNet as a task-general foundation model for time series analysis and show that its design can leverage vision backbones.
Abstract
from arXiv · showhide
Time series analysis is of immense importance in extensive applications, such as weather forecasting, anomaly detection, and action recognition. This paper focuses on temporal variation modeling, which is the common key problem of extensive analysis tasks. Previous methods attempt to accomplish this directly from the 1D time series, which is extremely challenging due to the intricate temporal patterns. Based on the observation of multi-periodicity in time series, we ravel out the complex temporal variations into the multiple intraperiod- and interperiod-variations. To tackle the limitations of 1D time series in representation capability, we extend the analysis of temporal variations into the 2D space by transforming the 1D time series into a set of 2D tensors based on multiple periods. This transformation can embed the intraperiod- and interperiod-variations into the columns and rows of the 2D tensors respectively, making the 2D-variations to be easily modeled by 2D kernels. Technically, we propose the TimesNet with TimesBlock as a task-general backbone for time series analysis. TimesBlock can discover the multi-periodicity adaptively and extract the complex temporal variations from transformed 2D tensors by a parameter-efficient inception block. Our proposed TimesNet achieves consistent state-of-the-art in five mainstream time series analysis tasks, including short- and long-term forecasting, imputation, classification, and anomaly detection. Code is available at this repository: https://github.com/thuml/TimesNet.
1 INTRODUCTION
TimesNet frames temporal variation modeling as a multi-periodicity problem and transforms 1D series into 2D tensors that jointly represent intraperiod and interperiod variations. Its TimesBlock-based backbone adaptively models these variations and achieves consistent state-of-the-art across five mainstream tasks.
- Temporal variation modeling is difficult because real-world series combine overlapping rising, falling, and fluctuating patterns.
- Multiple periods, such as daily and yearly weather cycles, overlap and interact, making temporal variation modeling intractable.
- TimesNet achieves consistent state-of-the-art performance across short- and long-term forecasting, imputation, classification, and anomaly detection.
- Transforming 1D series into 2D tensors jointly represents intraperiod and interperiod variations that 1D structure cannot explicitly show.
- TimesNet uses TimesBlock to discover multiple periods and capture temporal 2D variations with a parameter-efficient inception block.
2 RELATED WORK
Related work models temporal dependencies with predefined patterns, MLPs, convolutions, recurrent networks, or Transformers. TimesNet differs by targeting a task-general foundation model rather than a specific analysis task.
- Classical methods use predefined temporal patterns, but real-world variations are often too complex for those patterns.
- MLP-, TCN-, and RNN-based methods encode or capture temporal dependencies using temporal layers or sequential modeling.
- Transformers use attention to discover temporal dependencies, while Autoformer additionally models learned periods and decomposes seasonal and trend components.
- TimesNet attempts to provide a task-general foundation model instead of limiting the architecture to one analysis task.
3 TIMESNET
TimesNet discovers dominant periods with FFT, reshapes 1D series into multiple structured 2D tensors, and processes them with shared inception blocks. TimesBlock transforms the representations back to 1D and fuses them using period amplitudes.
- Period discovery: TimesNet first discovers periods by selecting the top-k frequency amplitudes from an FFT-based frequency-domain analysis.
- 2D transformation: For each selected period, padding and reshaping produce a 2D tensor whose columns encode intraperiod variation and rows encode interperiod variation.
- 2D transformation: The transformation creates localities among adjacent time points and adjacent periods, allowing temporal 2D variations to be processed by 2D kernels.
- TimesBlock: TimesBlock estimates periods for deep features, applies a parameter-efficient inception block to each reshaped tensor, and truncates outputs back to length T.
- TimesBlock: A shared inception block processes different reshaped tensors, making model size invariant to the hyper-parameter k.
- Adaptive aggregation: TimesBlock aggregates representations from different periods using their amplitude-based relative importance.
- Representation learning: The resulting structured tensors let TimesBlock capture multi-scale temporal 2D variations simultaneously and can support vision backbones such as ResNet and ConvNeXt.
4 EXPERIMENTS
Experiments evaluate TimesNet across five mainstream time-series analysis tasks and compare it with broad baseline families. Results report consistent state-of-the-art performance, with further gains from stronger vision backbones and task-dependent representation behavior.
- Experimental setup: Experiments cover short- and long-term forecasting, imputation, classification, and anomaly detection using established benchmarks and diverse baseline families.The baselines include RNN-, CNN-, MLP-, and Transformer-based models, alongside task-specific state-of-the-art methods.
- Main results: TimesNet achieves consistent state-of-the-art performance across the five mainstream analysis tasks compared with customized models.The paper reports this cross-task result in the main performance comparison.
- Generality and representation analysis: Replacing the inception block with stronger vision backbones further improves TimesNet, while CKA patterns differ between forecasting or anomaly detection and imputation or classification.Higher CKA similarity corresponds to better performance in forecasting and anomaly detection, but lower similarity corresponds to better performance in imputation and classification.
- Short- and long-term forecasting: More than 80% of long-term forecasting cases reach state-of-the-art performance, while TimesNet also performs best on the challenging M4 short-term task.Long-term results average four prediction lengths, and M4 prediction lengths range from 6 to 48.
- Imputation: TimesNet achieves consistent state-of-the-art performance for imputation with randomly masked time points across electricity and weather benchmarks.Table 4 averages results over mask ratios of 12.5%, 25%, 37.5%, and 50% in length-96 series.
- Anomaly detection: TimesNet achieves the best anomaly-detection performance, outperforming FEDformer and Autoformer; periodicity-aware models perform strongly on this task.The canonical Transformer obtains an averaged F1-score of 76.88%, while periodicity analysis can highlight variations violating periodicity.
- Representation analysis: TimesNet’s transformed 2D tensors capture multi-periodicities and organize local time-point relationships by columns and period relationships by rows.This structure supports using 2D kernels for representation learning.
5 CONCLUSION AND FUTURE WORK
The conclusion presents TimesNet as a task-general foundation model that uses multi-periodicity and temporal 2D variations to model complex time-series patterns. It reports broad performance and generality across five tasks and identifies large-scale pre-training as future work.
- Conclusion: TimesNet shows broad generality and performance across five mainstream time-series analysis tasks.The conclusion frames the model as a task-general foundation model.
- Future work: The paper identifies large-scale pre-training methods for time series as a direction for future work.This is stated as the planned next research direction.
A IMPLEMENTATION DETAILS
Experiments use task-specific datasets, metrics, configurations, and sensitivity analyses to evaluate TimesNet across diverse time series settings. The protocol examines robustness to frequency-selection choices while tailoring hyperparameters to task requirements.
- Experimental protocol: All experiments are repeated three times in PyTorch on a single NVIDIA TITAN RTX 24GB GPU, with dmodel selected from the input dimension C.The configuration uses dmodel = min{max{2⌈log C⌉, dmin}, dmax}.
- Metrics: Experiments use MSE and MAE for long-term forecasting and imputation, F1-score for anomaly detection, and SMAPE, MASE, and OWA for short-term forecasting.OWA is the overall weighted average used in the M4 competition.
- Hyper-parameter sensitivity: TimesNet maintains stable performance across different choices of k in imputation, classification, anomaly detection, and forecasting.The sensitivity analysis uses a 37.5% mask ratio for imputation and anomaly detection, the two largest subsets for classification, and weighted averaging for short-term forecasting.
- Hyper-parameter sensitivity: The selection of k affects forecasting and anomaly detection more than classification and imputation, while k = 3 is used for three tasks and k = 5 for short-term forecasting.The paper attributes the difference to information quantity in low-level representations versus hierarchical representations in high-level tasks.
C ABLATION STUDIES
Ablations test TimesNet’s 2D representation, architecture, and adaptive aggregation choices. The results support a parameter-efficient inception block, deep-feature transformation, and adaptive aggregation as the selected design.
- 2D space: Replacing the inception block with ResNeXt, Swin Transformer, or ConvNeXt improves performance, but the parameter-efficient inception block balances efficiency and performance.Independent parameters also improve results but make model size depend on the number of transformed tensors k.
- Model architecture: Combining TimesNet with Autoformer’s deep decomposition architecture does not bring further promotion, while TimesNet handles series without clear periodicity.When periodicity is unclear, the model can learn frequency one and represent the trend through intraperiod-variation modeling.
- Model architecture: 85.49% →84.85%: transforming raw data instead of applying the transformation within every TimesBlock reduces the average F1-score.The comparison supports placing the 1D-to-2D transformation in each TimesBlock to use deep representations.
- Adaptive aggregation: Adaptive aggregation performs best compared with directly summing processed tensors or removing the Softmax function.The selected design uses Softmax amplitudes as aggregation weights.
D MORE REPRESENTATION ANALYSIS
The transformed 2D tensors expose interpretable intraperiod and interperiod structure, including local variation and long-term trends. Dataset statistics and visualizations illustrate multi-periodicity and model predictions under imputation and forecasting settings.
- 2D variation visualization: Interperiod variation can represent long-term trends, with row-wise decreases indicating an Exchange downtrend and similar row values reflecting ETTh1’s globally stable variation.The examples connect row structure in transformed tensors to dataset-level temporal behavior.
- 2D variation visualization: For series without clear periodicity, frequency one preserves the raw variation as intraperiod structure while interperiod variation can still represent long-term trends.This provides informative 2D structure even when the original series lacks clear periodicity.
- 2D variation visualization: Transformed tensors show locality both among adjacent time points within columns and among adjacent periods within rows, while non-adjacent periods may differ because of global trends.Columns encode intraperiod variation and rows encode interperiod variation.
- Periodicity statistics: Real-world datasets exhibit multi-periodicity; for example, Electricity contains period lengths 12 and 24.Period-length densities are estimated by applying FFT to length-96 segments and selecting the top-6 significant frequencies.
- Prediction visualizations: Figures compare black ground-truth lines with orange predictions for ETTm1 imputation at 50% masking and ETTm2 and M4 forecasting settings.The showcases cover imputation, long-term forecasting, and short-term forecasting.
G MODEL EFFICIENCY ANALYSIS
TimesNet is compared with 13 common models using performance rankings and efficiency measurements across five tasks. It achieves the best performance in all five tasks and the greatest efficiency among the top three models.
- Ranking protocol: The ranking comparison covers 13 models shared across long- and short-term forecasting, imputation, classification, and anomaly detection.The models include RNN-, CNN-, MLP-, and Transformer-based approaches.
- Overall comparison: TimesNet achieves the best performance in all five tasks and the greatest efficiency among the top three models.The five tasks are short- and long-term forecasting, imputation, classification, and anomaly detection.
- Efficiency: TimesNet uses 2D kernels and a parameter-efficient design whose parameter size remains invariant as the input series changes.The efficiency comparison is recorded on the ETTh1 imputation task, and smaller rankings indicate better performance.
H MODEL PERFORMANCE IN MIXED DATASET
TimesNet is evaluated under unified training on a mixed dataset containing diverse temporal patterns and periodicities. It handles this setting well, while other baselines may lose performance on some subsets.
- The mixed dataset combines hourly ETTh1, ETTh2 and 15-minute ETTm1, ETTm2 data with diverse temporal patterns and periodicities.
- TimesNet handles the mixed dataset well and generally promotes performance under unified training.
- Other baselines may decline on some subsets during mixed training, indicating difficulty handling the dataset’s complex periodicities.
- The results support TimesNet’s potential as a general-purpose backbone for large-scale time-series pre-training.
I FULL RESULTS
The paper provides full experimental results across forecasting, imputation, classification, and anomaly detection. These results use task-specific settings and comparisons, including stated fairness adjustments and evaluation protocols.
- FULL RESULTS: Full results are reported for long-term forecasting, short-term forecasting, imputation, classification, and anomaly detection.
- FULL RESULTS: Long-term forecasting compares models across prediction lengths, using input lengths of 36 for ILI and 96 for other datasets, with Avg averaged over four lengths.
- FULL RESULTS: For fair comparisons, official baseline codes are used with only input and output sequence lengths changed when input-output settings mismatch.
- FULL RESULTS: Fair comparisons remove N-BEATS’s special ensemble and use only reconstruction error for Anomaly Transformer.
- FULL RESULTS: Imputation results evaluate random masking rates of 12.5%, 25%, 37.5%, and 50% of time points.
- FULL RESULTS: Classification reports accuracy, while anomaly detection reports precision, recall, and F1-score, with higher values indicating better anomaly-detection performance.