PaperlayerView exact revision on arXiv

Source-linked AI summary

TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting

Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, Jun Zhou

arXiv:2405.14616v1cs.LGcs.AI

TL;DR

Real-world time series contain intricate variations across temporal scales, challenging forecasting. TimeMixer mixes decomposed multiscale information across past extraction and future prediction, achieving consistent state-of-the-art performance in long- and short-term forecasting with favorable efficiency.

  • Problem

    Real-world time series mix intricate temporal variations, motivating multiscale analysis to disentangle microscopic and macroscopic information for forecasting.

  • Method

    TimeMixer uses Past-Decomposable-Mixing and Future-Multipredictor-Mixing to combine decomposed multiscale series during history extraction and future prediction.

  • Results

    TimeMixer achieves consistent state-of-the-art performance across long- and short-term forecasting benchmarks, including a 9.4% MSE reduction versus PatchTST on Weather.

  • Takeaways & Limitations

    The results support multiscale mixing as an effective and efficient approach for forecasting temporal variations across diverse benchmark settings.

  • Takeaways & Limitations

    As input length increases, TimeMixer’s linear mixing layer can require more parameters, limiting efficiency for mobile applications.

Abstract

from arXiv · show

Time series forecasting is widely used in extensive applications, such as traffic planning and weather forecasting. However, real-world time series usually present intricate temporal variations, making forecasting extremely challenging. Going beyond the mainstream paradigms of plain decomposition and multiperiodicity analysis, we analyze temporal variations in a novel view of multiscale-mixing, which is based on an intuitive but important observation that time series present distinct patterns in different sampling scales. The microscopic and the macroscopic information are reflected in fine and coarse scales respectively, and thereby complex variations can be inherently disentangled. Based on this observation, we propose TimeMixer as a fully MLP-based architecture with Past-Decomposable-Mixing (PDM) and Future-Multipredictor-Mixing (FMM) blocks to take full advantage of disentangled multiscale series in both past extraction and future prediction phases. Concretely, PDM applies the decomposition to multiscale series and further mixes the decomposed seasonal and trend components in fine-to-coarse and coarse-to-fine directions separately, which successively aggregates the microscopic seasonal and macroscopic trend information. FMM further ensembles multiple predictors to utilize complementary forecasting capabilities in multiscale observations. Consequently, TimeMixer is able to achieve consistent state-of-the-art performances in both long-term and short-term forecasting tasks with favorable run-time efficiency.

1 INTRODUCTION

TimeMixer addresses intricate, non-stationary temporal variations by exploiting distinct patterns across sampling scales. Its multiscale-mixing architecture combines decomposable past extraction with multiple future predictors to use disentangled and complementary information for forecasting.

  • Motivation: Real-world time series exhibit complex, non-stationary temporal variations that make forecasting challenging across applications including economics, energy, traffic, and weather.Forecasting predicts future temporal variations from past observations.
  • Motivation: Time series show distinct patterns at different sampling scales, with fine-grained changes in hourly traffic, holiday-related fluctuations in daily data, and macroeconomic trends in yearly averages.These observations motivate multiscale analysis to disentangle microscopic and macroscopic variations.
  • TimeMixer: TimeMixer generates multiscale observations through average downsampling and combines them in history extraction and future prediction phases.Past-Decomposable-Mixing extracts information from past variations, while Future-Multipredictor-Mixing predicts future series.
  • TimeMixer: Past-Decomposable-Mixing uses decomposable multiscale mixing to handle distinct seasonal and trend variations.The supplied passage identifies PDM as the mechanism for coping with these differing properties.
  • Contributions: TimeMixer simultaneously exploits disentangled variations and complementary forecasting capabilities from multiscale series.This is presented as the model’s central departure from previous methods.

2 RELATED WORK

Prior time-series forecasting methods span RNN-, CNN-, Transformer-, and MLP-based paradigms, with specialized designs using decomposition, multi-periodicity analysis, and multiscale structures. TimeMixer instead develops multiscale mixing that combines information across scales during both past extraction and future prediction.

  • Temporal modeling paradigms: Deep time-series models are commonly categorized as RNN-, CNN-, Transformer-, and MLP-based methods for temporal modeling.CNNs use temporal convolution kernels, whereas RNNs use recurrent structures to model temporal states.
  • Temporal pattern modeling: Series decomposition and multi-periodicity analysis are established designs for capturing intricate temporal patterns.These designs address complex temporal variations through specialized temporal analysis.
  • Series decomposition: Autoformer, FEDformer, DLinear, and MICN use series decomposition to separate or process seasonal and trend components in forecasting.Their approaches include moving-average variants, preprocessing before linear regression, and decomposition into seasonal and trend terms.
  • Multiscale modeling: Pyraformer and SCINet introduce temporal multiscale designs, but their predictions do not simultaneously use information from different scales extracted from past observations.Pyraformer uses pyramidal attention, while SCINet uses a bifurcate downsampling tree.
  • Information mixing: Information mixing has been used in vision and language, including MLP-Mixer's successive channel and patch mixing and FNet's Fourier-based token mixing.These examples integrate information through linear layers or Fourier Transform instead of attention layers.

3 TIMEMIXER

TimeMixer models complex temporal variations through disentangled multiscale representations, where fine scales capture detailed patterns and coarse scales capture macroscopic variations. Its PDM and FMM blocks separately mix past information and ensemble future predictions across scales.

  • Multiscale representations: TimeMixer downsamples past observations into multiscale series, embeds them into deep features, and treats the finest and highest-level scales as detailed and macroscopic representations.Average pooling produces M scales, with x0 as the input series and xM representing macroscopic variations.
  • Past-Decomposable-Mixing: PDM decomposes each multiscale series into seasonal and trend components, then mixes them separately to interact information across scales.The decomposition uses a series decomposition block from Autoformer, while S-Mix and T-Mix perform separate information interaction.
  • Past-Decomposable-Mixing: Seasonal mixing proceeds bottom-up, transferring detailed fine-scale information to coarser scales for seasonal modeling.The Bottom-Up-Mixing layer uses two linear layers with an intermediate GELU activation along the temporal dimension.
  • Past-Decomposable-Mixing: Trend mixing proceeds top-down, using coarse-scale macroscopic knowledge to guide trend modeling at finer scales.This direction reduces the influence of detailed variations that can introduce noise into macroscopic trend modeling.
  • Future-Multipredictor-Mixing: FMM aggregates predictions from multiple scale-specific predictors, integrating complementary forecasting capabilities from mixed multiscale past information.Each predictor directly regresses length-F futures from extracted past information at its corresponding scale before projecting representations into C variates.

4 EXPERIMENTS

Experiments across 18 real-world benchmarks and 15 baselines evaluate TimeMixer in long- and short-term forecasting under unified settings. TimeMixer delivers strong accuracy, component-level effectiveness, and favorable efficiency across diverse datasets and series lengths.

  • Experimental setup: Experiments cover eight long-term benchmarks, four PeMS traffic datasets, and the 100,000-series M4 dataset, comparing TimeMixer with 15 baselines.The evaluation spans multivariate and univariate short-term forecasting alongside diverse long-term forecasting scenarios.
  • Experimental setup: All baselines use aligned input lengths, while main-text results are averaged across three repeats to improve comparison fairness.The authors note that previously reported baseline results are not directly comparable because of differing input lengths and hyperparameter-search strategies.
  • Long-term forecasting: TimeMixer achieves consistent state-of-the-art long-term forecasting performance, including 9.4% MSE reduction on Weather and 24.7% on Solar-Energy versus PatchTST.The model performs well across datasets with different frequencies, variate numbers, and real-world scenarios, including low-forecastability Solar-Energy.
  • Short-term forecasting: TimeMixer also performs favorably in short-term forecasting under both multivariate PeMS and univariate M4 settings.Many channel-independent models, including PatchTST and DLinear, degrade on PeMS because of complex spatiotemporal correlations among variates.
  • Ablations: Removing Future-Multipredictor-Mixing significantly reduces forecasting accuracy for both short- and long-term predictions, supporting multiscale future-prediction mixing.Ablations were conducted across all 18 experiment benchmarks.
  • Ablations and analysis: Removing seasonal or trend mixing reduces performance, while the intended bottom-up seasonal and top-down trend mixing outperforms shared or opposite mixing strategies.The visualization further shows periodic seasonal mixing and locally aggregating trend mixing, while efficiency remains favorable across series lengths from 192 to 3072.
  • Analysis on number of scales: The authors set M as 3 for long-term forecasting and 1 for short-term forecasting because increasing scales helps longer prediction lengths more but trades off efficiency.Performance gains from increasing M decline for shorter prediction lengths.

5 CONCLUSION

TimeMixer addresses intricate temporal variations in time series forecasting through multiscale mixing, combining disentangled variations with complementary forecasting capabilities. It achieves consistent state-of-the-art performance in both long-term and short-term forecasting tasks.

  • Architecture: TimeMixer uses a multiscale mixing architecture to tackle intricate temporal variations in time series forecasting.The architecture is designed around multiscale temporal information.
  • Core blocks: Past-Decomposable-Mixing and Future-Multipredictor-Mixing exploit disentangled variations and complementary forecasting capabilities.These two blocks support the model’s past extraction and future prediction processes.
  • Performance: TimeMixer achieves consistent state-of-the-art performance in both long-term and short-term forecasting tasks.This result was observed across all reported experiments.

6 ETHICS STATEMENT

The authors state that the work focuses solely on the scientific problem and presents no potential ethical risks.

  • The authors identify no potential ethical risk because the work only addresses the scientific problem.

A IMPLEMENTATION DETAILS

The paper evaluates forecasting models on established long- and short-term datasets, specifies task-dependent metrics, and details the experimental setup and multiscale visualization procedures.

  • Datasets details: TimeMixer is evaluated on eight long-term forecasting datasets and the PeMS and M4 datasets for short-term forecasting.The long-term datasets include Weather, Traffic, Electricity, Solar-Energy, and ETTh1, ETTh2, ETTm1, and ETTm2.
  • Metric details: Long-term forecasting uses MSE and MAE, while short-term evaluation follows SCINet metrics on PeMS and N-BEATS methodology on M4.PeMS metrics include MAE, MAPE, and RMSE; M4 uses SMAPE.
  • Experiment details: All experiments were repeated three times on a single NVIDIA A100 80GB GPU using PyTorch, ADAM, L2 loss, learning rates of 10^-2 or 10^-3, and batch sizes from 8 to 128.TimeMixer uses two Past Decomposable Mixing blocks by default, with the number of scales selected according to time-series length.
  • Visualization details: To visualize complementary multiscale forecasting capabilities, the authors fix PDM, train one predictor per scale with ground-truth supervision, and use a sum ensemble in FMM.They also directly plot each predictor’s output and rescale predictions for clearer visualization.

B EFFICIENCY ANALYSIS

TimeMixer demonstrates an efficiency advantage over Transformer-based models, attributed to its fully MLP-based network architecture. Table 8 reports GPU memory and running-time measurements for each model.

  • Efficiency advantage: TimeMixer’s efficiency advantage over PatchTST, FEDformer, and Autoformer is attributed to its fully MLP-based network architecture.The paper presents efficiency curves in Figure 5 and quantitative results in Table 8.
  • Efficiency metrics: Table 8 compares each model using GPU memory in MiB and running time in seconds per iteration.The table reports GPU memory (MiB) and speed as running time (s/iter).

C ERROR BARS · D HYPERPARAMTER SENSITIVITY

The paper reports repeated experiments with variability and significance assessments for TimeMixer against second-best methods, then studies layer-count sensitivity. Increasing the number of layers generally improves performance across prediction lengths, leading to a choice of L=2 as an efficiency–performance trade-off.

  • C ERROR BARS: All experiments are repeated three times to assess result variability and statistical significance.The paper reports standard deviations for TimeMixer and the second-best model, alongside significance tests.
  • D HYPERPARAMTER SENSITIVITY: The study further evaluates the effect of the number of layers L after exploring the number of scales M.This sensitivity analysis examines layer depth across different prediction lengths.
  • D HYPERPARAMTER SENSITIVITY: Increasing the number of layers generally improves performance across different prediction lengths.The improvement is reported as a general trend in Table 12.
  • D HYPERPARAMTER SENSITIVITY: The authors set L=2 to balance efficiency and performance.This setting follows the observed layer-sensitivity results.
  • C ERROR BARS: Table 9 evaluates TimeMixer against PatchTST across ETT, Weather, Solar-Energy, Electricity, and Traffic datasets.The table reports standard deviations and statistical tests for both methods.
  • C ERROR BARS: Table 10 reports standard deviations and statistical tests for TimeMixer versus SCINet on the PEMS dataset.This comparison extends the error-bar and significance analysis to PEMS.

E FULL RESULTS … F.4 ALTERNATIVE ENSEMBLE STRATEGIES

The supplement reports full forecasting results under unified and searched hyperparameters, then evaluates TimeMixer’s ablations and alternative decomposition, downsampling, and ensemble designs. These experiments support its design choices and practical efficiency.

  • E FULL RESULTS: Full forecasting results use unified input lengths, batch sizes, and training epochs across models for fair comparison.Results are reported for each forecasting setting in Table 13.
  • E FULL RESULTS: Comprehensive hyperparameter searches vary input length, learning rate, encoder layers, dmodel, and training epochs to compare model upper bounds.The searched ranges are input lengths 96, 192, 336, and 512; learning rates 10^-5 to 0.05; encoder layers 1 to 5; dmodel 16 to 512; and epochs 10 to 100.
  • E FULL RESULTS: TimeMixer’s advantage over PatchTST is smaller after comprehensive search, while it runs much faster according to the efficiency comparison.The authors characterize TimeMixer as practical when considering performance, hyperparameter-search cost, and efficiency.
  • F.1 ABLATIONS OF EACH DESIGN IN TIMEMIXER: Across comprehensive ablations, the official TimeMixer design performs best, while completely reversing seasonal and trend mixing directions causes a serious performance drop.The reported explanation is that unsuitable mixing damages microscopic fine-scale seasonal information and macroscopic coarse-scale trend information.
  • F.2 ALTERNATIVE DECOMPOSITION METHODS: DFT high- and low-frequency decomposition performs worse than TimeMixer’s moving-average-based decomposition under the tested mixing approach.A second DFT-based design extracts significant frequencies as the seasonal component and defines trend as the raw series minus that component; it surpasses the moving-average design.
  • F.3 ALTERNATIVE DOWNSAMPLING METHODS: Replacing average pooling with 1D convolutions slightly improves performance, but TimeMixer retains average pooling because it balances performance and efficiency.The comparison is reported in Table 19.
  • F.4 ALTERNATIVE ENSEMBLE STRATEGIES: TimeMixer’s ensemble loss is computed on the summed predictor outputs, while the alternative average strategy rescales the ensemble by a constant factor.The difference between the two strategies is described as only a constant multiple.
  • F.4 ALTERNATIVE ENSEMBLE STRATEGIES: Under identical supervision, deep predictors can absorb the averaging rescaling, making summed and averaged ensembles equivalent for learning the final prediction.Experiment results for alternative ensemble strategies are provided in Table 20.

F.5 ABLATIONS ON LARGER SCALES AND LARGER INPUT LENGTH SETTINGS · G ADDITIONAL BASELINES

The paper extends its ablation study to larger scales and input lengths, finding that all TimeMixer designs remain effective and that seasonal mixing and mixing directions are especially important. It also adds comparisons with Scaleformer, MTSMixer, and TSMixer under unified and searched hyperparameter settings.

  • F.5 ABLATIONS ON LARGER SCALES AND LARGER INPUT LENGTH SETTINGS: The larger-scale ablation uses PEMS04 with M = 3, while the longer-input ablation uses predict-336 on ETTm1 with input-336.M4 is excluded because its input length is fixed to a small value and larger M may be meaningless.
  • F.5 ABLATIONS ON LARGER SCALES AND LARGER INPUT LENGTH SETTINGS: All TimeMixer designs remain effective across both larger-scale and longer-input hyperparameter settings.These settings supplement earlier ablations conducted with M = 1 and input length 96.
  • F.5 ABLATIONS ON LARGER SCALES AND LARGER INPUT LENGTH SETTINGS: Seasonal mixing and proper mixing directions are essential, with seasonal mixing corresponding to case ③ and mixing directions to case ⑦.The result holds across both evaluated hyperparameter settings.
  • F.5 ABLATIONS ON LARGER SCALES AND LARGER INPUT LENGTH SETTINGS: In most cases, seasonal and trend mixing in PDM and FMM produce more significant relative promotions with larger M and longer inputs.Table 22 reports relative promotion as (1−①/②) in case ②.
  • F.5 ABLATIONS ON LARGER SCALES AND LARGER INPUT LENGTH SETTINGS: On ETTm1, seasonal mixing direction contributes much more in the longer-input setting because input-336 preserves around 3.5 periods versus only one day for input-96.The longer input provides sufficient microscopic seasonal information for bottom-up mixing.
  • G ADDITIONAL BASELINES: Additional baselines include the multiscale framework Scaleformer and the MLP-based models MTSMixer and TSMixer.MTSMixer and TSMixer were reproduced from public code because they were not officially published during submission.
  • G ADDITIONAL BASELINES: MTSMixer and TSMixer were evaluated with both unified hyperparameter settings and hyperparameter-searching settings.The comparisons are presented in Tables 23, 24, and 25.

H SPECTRAL ANALYSIS OF MODEL PREDICTIONS · I SHOWCASES · J LIMITATIONS AND FUTURE WORK

TimeMixer’s predictions capture distinct frequency components precisely and show superior qualitative performance across the compared datasets. Despite favorable GPU-memory and runtime efficiency, increasing input length can enlarge its parameter count, motivating alternative mixing designs for future work.

  • H SPECTRAL ANALYSIS OF MODEL PREDICTIONS: TimeMixer captures different frequency parts precisely when its prediction spectrum is compared with the ground truth and model predictions.The spectral analysis plots the spectra of ground truth and model predictions to demonstrate this capability.
  • H SPECTRAL ANALYSIS OF MODEL PREDICTIONS: The ETTh1 spectrogram cases use input-96-predict-96 settings to compare ground truth with predictions from different models.Figure 8 presents prediction spectrogram cases from ETTh1 under this setting.
  • I SHOWCASES: TimeMixer exhibits superior qualitative performance among the various models across forecasting cases from the test sets of each dataset.The comparison plots the final forecasting-result dimension in Figures 9, 10, 11, 12, 13, 17, and 18.
  • I SHOWCASES: The showcases include long-term results averaged over prediction lengths 96, 192, 336, and 720, plus short-term PEMS and M4 evaluations with their stated metrics and settings.Tables 23–25 cover reproduced competitive baselines, PEMS multivariate forecasting, and M4 univariate forecasting.
  • J LIMITATIONS AND FUTURE WORK: TimeMixer has favorable GPU-memory and running-time efficiency, but longer input lengths may cause its linear mixing layer to use more parameters.The authors identify this parameter growth as inefficient for mobile applications.
  • J LIMITATIONS AND FUTURE WORK: Future work will investigate attention-based or CNN-based mixing designs to improve TimeMixer’s parameter efficiency.These alternatives are proposed in response to the limitations of the current linear mixing layer.
Loading 2405.14616v1…