Source-linked AI summary
Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting
Wei Fan, Pengyang Wang, Dongkun Wang, Dongjie Wang, Yuanchun Zhou, Yanjie Fu
TL;DR
Distribution shift in TSF includes temporal changes within lookback windows and differences between lookback and horizon windows, while existing methods inadequately quantify or model these shifts. Dish-TS addresses both through Dual-CONET, which learns input- and output-space distributions within a two-stage normalization framework. Across datasets and forecasting backbones, it consistently improves performance, with reported average gains above 20%.
Problem
Existing TSF distribution-shift methods inadequately quantify distributions and overlook shifts between lookback and horizon windows, limiting their treatment of changing series distributions.
Method
Dish-TS is a model-agnostic two-stage paradigm using CONET-based Dual-CONET to separately learn input- and output-space distributions and normalize before forecasting and denormalize afterward.
Results
Dish-TS consistently boosts state-of-the-art forecasting backbones, with average improvements of 28.6% in univariate and 21.9% in multivariate forecasting.
Takeaways & Limitations
Considering both intra-space and inter-space shift provides a general paradigm that can be coupled with different TSF models for improved forecasting.
Takeaways & Limitations
Current TSF models ignore the distribution relationship between lookback and horizon windows, which the paper identifies as an inter-space limitation.
Abstract
from arXiv · showhide
The distribution shift in Time Series Forecasting (TSF), indicating series distribution changes over time, largely hinders the performance of TSF models. Existing works towards distribution shift in time series are mostly limited in the quantification of distribution and, more importantly, overlook the potential shift between lookback and horizon windows. To address above challenges, we systematically summarize the distribution shift in TSF into two categories. Regarding lookback windows as input-space and horizon windows as output-space, there exist (i) intra-space shift, that the distribution within the input-space keeps shifted over time, and (ii) inter-space shift, that the distribution is shifted between input-space and output-space. Then we introduce, Dish-TS, a general neural paradigm for alleviating distribution shift in TSF. Specifically, for better distribution estimation, we propose the coefficient net (CONET), which can be any neural architectures, to map input sequences into learnable distribution coefficients. To relieve intra-space and inter-space shift, we organize Dish-TS as a Dual-CONET framework to separately learn the distribution of input- and output-space, which naturally captures the distribution difference of two spaces. In addition, we introduce a more effective training strategy for intractable CONET learning. Finally, we conduct extensive experiments on several datasets coupled with different state-of-the-art forecasting models. Experimental results show Dish-TS consistently boosts them with a more than 20% average improvement. Code is available.
1 Introduction
The paper frames distribution shift as a central obstacle to TSF generalization and distinguishes shifts within the input space from shifts between lookback and horizon spaces. Dish-TS addresses both through a model-agnostic Dual-CONET paradigm and prior-knowledge-induced CONET training.
- Distribution shift changes series distributions over time and hinders TSF model generalization and performance.
- The paper distinguishes intra-space shift within lookback windows from inter-space shift between lookback and horizon windows.Lookbacks are treated as input-space and horizons as output-space.
- Existing methods quantify intra-space distributions with fixed empirical statistics such as mean and standard deviation, whose reliability and expressiveness are limited.Different sampling frequencies from the same series can produce different statistics, making the underlying distribution difficult to identify.
- Existing TSF methods generally neglect inter-space shift by assuming lookbacks and horizons share the same distribution, despite observed distribution differences.RevIN also assumes that the two spaces share the same statistical properties after normalization and denormalization.
- Dish-TS is a model-agnostic two-stage paradigm that normalizes inputs before forecasting and denormalizes outputs afterward.Its coefficient net maps series windows to learnable distribution coefficients, while Dual-CONET separately addresses input- and output-space distributions.
- 28.6% average improvement in univariate forecasting and 21.9% in multivariate forecasting were reported across extensive experiments with state-of-the-art models.
2 Related Work
Related work covers the progression from statistical and hybrid forecasting models to deep learning, including fully connected and Transformer-based approaches. It also reviews normalization methods developed to address distribution shift in TSF.
- Models for Time Series Forecasting: Time series forecasting research progressed from exponential smoothing and ARMA to hybrid, deep learning, and Transformer-based models.
- Models for Time Series Forecasting: N-BEATS uses pure fully connected networks, while Informer improves Transformer forecasting through attention computation, memory consumption, and inference speed.
- Models for Time Series Forecasting: Autoformer replaces attention with auto-correlation to facilitate forecasting.
- Distribution Shift in Time Series Forecasting: Normalization approaches for distribution shift include global-statistics z-score normalization, adaptive nonlinear normalization, Adaptive RNNs, and RevIN.The cited methods are Adaptive Norm, DAIN, Adaptive RNNs, and RevIN.
3 Problem Formulations
The paper formulates TSF as mapping historical lookback windows to subsequent horizon windows, then defines distribution shifts over these input and output spaces. It emphasizes that current models ignore the distribution relationship between the two spaces.
- Time Series Forecasting: TSF projects historical observations x_t−L:t of length L into future observations x_t:t+H of length H.The formulation extends from univariate to multivariate forecasting.
- Time Series Forecasting: Multivariate forecasting uses N distinct time series with equal length T and a mapping F_Θ: R^L×N → R^H×N.
- Distribution Shift in Time Series: The forecasting setup cuts one series into multiple lookback windows and corresponding horizon windows for model training.
- Distribution Shift in Time Series: Intra-space shift concerns changes in lookback-window distributions across time steps.
- Distribution Shift in Time Series: Existing work commonly uses distribution shift to mean intra-space shift, while inter-space shift concerns the distributions of lookback and horizon windows.The paper states that current TSF models ignore this input-output distribution relationship.
4 DISH-TS
Dish-TS is a flexible, model-agnostic paradigm that uses CONETs to estimate input- and output-space distributions and integrates them with forecasting models through normalization and denormalization. Its Dual-CONET design addresses both intra-space and inter-space shift, while prior-knowledge guidance assists HORICONET training.
- Overview: Dish-TS is a general paradigm that normalizes lookbacks before forecasting and denormalizes outputs afterward.It can be coupled with any deep time-series forecasting model.
- CONET: CONET maps a window of series data into learnable level and scaling coefficients representing overall scale and fluctuation.CONET can use arbitrary neural architectures and linear or nonlinear mappings.
- Dual-CONET Architecture: Dual-CONET uses BACKCONET for input-space distributions and HORICONET for output-space distributions, separately capturing lookback and horizon differences.BACKCONET estimates input distributions from lookbacks, whereas HORICONET infers future distributions from historical observations.
- A Simple CONET Design: The simple CONET instance uses fully connected projections and leaky ReLU to generate coefficients without introducing extra optimization parameters.The scaling coefficients are designed to represent series fluctuation.
- Prior Knowledge-Induced Training Strategy: HORICONET training is difficult because the gap between input- and output-space distributions increases with horizon length.Dish-TS uses horizon means as soft prior targets, although those means do not fully represent the output distribution.
5 Experiment
Experiments across multiple datasets and forecasting backbones show that Dish-TS consistently improves forecasting, including against RevIN and under varying horizon, lookback, and guidance settings.
- Experimental Setup: Dish-TS is evaluated on five real-world datasets using original, unnormalized data and MSE and MAE metrics.Experiments are repeated three times, with average performance reported.
- Experimental Setup: Dish-TS is coupled end-to-end with Informer, Autoformer, and N-BEATS to test its model-agnostic design.
- Overall Performance: More than 20% MSE improvement is achieved in most univariate settings, reaching up to 50% in some cases.Dish-TS also yields stable 10%–30% improvements on ETTh1, ETTm2, and Weather in multivariate forecasting.
- Comparison with Normalization Methods: More than 10% average improvement over RevIN is obtained in multivariate forecasting with Autoformer as the backbone.The comparison considers Dish-TS's treatment of both intra-space and inter-space shift.
- Horizon Analysis: Dish-TS remains better across different horizon settings, although its performance slowly worsens as horizons increase.Larger horizons include more distribution changes and require more complicated modeling, according to the authors.
- Lookback Analysis: Increasing lookback length improves Electricity MSE from 1.286 to 0.731 and another reported case from 0.571 to 0.520.The authors attribute this to more historical information for inferring future distribution.
- Prior Guidance: Prior guidance becomes especially helpful for longer series, where α = 0 produces rapidly increasing prediction error beyond length 168.
- Initialization and Consumption: Uniform and average CONET initialization generally outperform normal initialization, while Dish-TS adds 4MiB or less memory consumption.
6 Conclusion Remarks
The paper frames distribution shift in TSF as intra-space and inter-space shift and proposes Dish-TS as a general paradigm to alleviate both. Experiments report significant improvements over backbone models and motivate further research.
- Distribution shift in time series forecasting is organized into intra-space shift and inter-space shift.
- Dish-TS is presented as a general paradigm with a simple instance and prior knowledge-induced training strategy for coupling with state-of-the-art models.
- Extensive experiments report significant improvement over backbone models and are intended to facilitate future research on distribution shift in time series.
A.1 More Dataset Details
The experiments use five real-world datasets spanning electricity consumption, transformer temperatures, weather, and illness data, with dataset-specific preprocessing and splits.
- The dataset suite contains Electricity, ETTh1, ETTm2, Weather, and Illness real-world datasets.
- Electricity records consumption from 321 clients, while ETT datasets contain two years of transformer-temperature data at hourly and 15-minute granularities.
- Weather records 21 meteorological features every ten minutes, and Illness contains weekly influenza-like illness patient data.
- Data are split using 6:2:2 for specified ETT datasets and 7:1:2 for Electricity, Weather, and Illness.
A.2 More Evaluation Details
The evaluation uses original, unnormalized data and compares Dish-TS-coupled backbones with their baselines under matched settings. Experiments cover multiple datasets, forecasting configurations, and three state-of-the-art backbones.
- Evaluation setting: All experiments evaluate original time series without data normalization or scaling, unlike settings that preprocess data with z-score normalization.
- Evaluation setting: Performance is measured with mean squared error (MSE) and mean absolute error (MAE), with reported metrics scaled for readability across datasets.
- Evaluation setting: Lookback and horizon windows usually share lengths from {24, 48, 96, 168, 336}, while Illness uses {24, 36, 48, 69, 96}.For long-term forecasting analysis, the horizon extends to 720 with lookback fixed at 96.
- Training procedure: Models are trained with L2 loss and Adam, repeated three times per experiment, with early stopping after 7 steps.
- Compared models: Dish-TS is evaluated by coupling its dual-CONET framework with Informer, Autoformer, and N-BEATS.The compared baseline and Dish-TS versions use the same experimental settings and hyperparameters.
B.1 More Results of Overall Performance
Additional Illness-dataset results show that Dish-TS improves forecasting in univariate settings and can produce substantial gains for selected multivariate configurations. The gains vary across backbones and window lengths.
- Univariate time series forecasting: Dish-TS improves univariate forecasting performance for most Illness configurations, with larger gains generally appearing at longer lookback- and horizon lengths.Informer remains weak in these experiments, while Dish-TS still improves it in most cases.
- Multivariate time series forecasting: 45.3% average improvement is observed for Autoformer at length 96 in multivariate Illness forecasting, reducing 0.148 to 0.081.N-BEATS gains are less significant except at length 96, and Dish-TS also handles larger lookback- and horizon lengths.
B.2 Comparison with Normalization Techniques
Dish-TS is compared with RevIN and examined through memory, forecasting, and distribution-visualization analyses. The results favor Dish-TS, particularly when distribution changes sharply or lookback and horizon distributions differ.
- Comparison with RevIN: Dish-TS outperforms RevIN by more than 10% in most univariate cases, reaching up to 36% improvement in one case.The comparison reports MSE performance relative to RevIN.
- Comparison with RevIN: Dish-TS’s advantage over RevIN is attributed in the paper to considering inter-space shift, which RevIN ignores.
- Memory consumption: Dish-TS’s simple instance adds only 4MiB or less memory consumption compared with the backbone.
- Visualization results: Dish-TS helps backbone models forecast more accurately when series trends undergo sudden rises or drops.Visualizations cover ETTm2, ETTh1, and Weather using Autoformer and N-BEATS backbones.
- Distribution quantification: Dish-TS separately quantifies lookback and horizon distributions, allowing the backbone to learn a smaller forecasting mapping between already-modeled distributions.RevIN instead uses lookback statistics to infer the horizon distribution.