PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for Time Series Forecasting: A Survey

Xiangjie Kong, Zhenghao Chen, Weiyao Liu, Kaili Ning, Lechao Zhang, Syauqie Muhammad Marier, Yichen Liu, Yuhao Chen, Feng Xia

arXiv:2503.10198v1cs.LGcs.AI

TL;DR

Time series forecasting is important across many domains, but classical methods have limitations and existing surveys lacked unified coverage of deep forecasting architectures, feature extraction, and datasets. This paper systematically surveys DTSF through architectural paradigms, feature-learning methods, and cross-domain datasets, concluding with major challenges and future research directions.

  • Problem

    Existing surveys lacked a unified summary of deep time series forecasting architectures and detailed coverage of feature extraction methods and datasets.

  • Method

    The paper synthesizes DTSF research by classifying architectures, analyzing time series composition and feature-enhancement methods, and compiling datasets across major domains.

  • Results

    The survey identifies five prominent DTSF model paradigms: Encoder-Decoder, Transformer, Generative Adversarial, Integration, and Cascade.

  • Takeaways & Limitations

    The survey provides a systematic reference spanning DTSF architectures, feature extraction, datasets, challenges, and future research directions.

  • Takeaways & Limitations

    Statistical forecasting methods remain limited in handling non-stationarity, nonlinear relationships, noise, complex dependencies, long-term dependencies, and multi-feature forecasting tasks.

Abstract

from arXiv · show

Time series forecasting (TSF) has long been a crucial task in both industry and daily life. Most classical statistical models may have certain limitations when applied to practical scenarios in fields such as energy, healthcare, traffic, meteorology, and economics, especially when high accuracy is required. With the continuous development of deep learning, numerous new models have emerged in the field of time series forecasting in recent years. However, existing surveys have not provided a unified summary of the wide range of model architectures in this field, nor have they given detailed summaries of works in feature extraction and datasets. To address this gap, in this review, we comprehensively study the previous works and summarize the general paradigms of Deep Time Series Forecasting (DTSF) in terms of model architectures. Besides, we take an innovative approach by focusing on the composition of time series and systematically explain important feature extraction methods. Additionally, we provide an overall compilation of datasets from various domains in existing works. Finally, we systematically emphasize the significant challenges faced and future research directions in this field.

1 Introduction

Deep learning has expanded time series forecasting by learning intricate temporal features and patterns from complex data, while existing surveys lacked a unified account of model structures and related developments. This survey addresses those gaps through an architectural taxonomy, feature-enhancement review, dataset compilation, and discussion of challenges and future directions.

  • Deep learning autonomously extracts intricate time features and patterns, capturing long-term dependencies and complex relationships in time series data.This contrasts with statistical and machine learning approaches that rely more on domain-specific knowledge or meaningful feature engineering.
  • Existing research lacked a high-level categorization of deep learning time series forecasting structures, comprehensive recent-development summaries, and in-depth analyses of future challenges.
  • The survey proposes a dynamic architectural taxonomy to systematically classify and summarize deep learning models for time series forecasting.The authors describe it as the first dynamic classification of deep learning model architectures for time series forecasting, to the best of their knowledge.
  • It reviews feature-enhancement methods including dimensional decomposition, time-frequency transformation, pre-training, and patch-based segmentation.The analysis focuses on the composition of complex, high-dimensional time series features and their latent learning potential.
  • The survey compiles major time series forecasting datasets, discusses key challenges, and highlights promising future research directions.
  • The paper is organized around TSF fundamentals, DTSF model structures, feature extraction and learning paradigms, datasets, challenges, and future prospects.

2 Time Series Forecasting

This section defines time series forecasting, distinguishes major task settings, introduces classical statistical models, and summarizes their limitations. It emphasizes that traditional methods struggle with nonlinear, noisy, high-dimensional, and long-dependent data.

  • Definitions and characteristics: Time series are chronological observations recorded at fixed or variable intervals, and TSF uses historical patterns to forecast future values.The survey represents observations as sequences and notes that discovering features across adjacent time steps supports prediction-model design.
  • Definitions and characteristics: Time series data are commonly noisy, high-dimensional, heterogeneous, and dynamic, making representation important for feature extraction and dimensionality reduction.The survey discusses dimensionality reduction, wavelet analysis, filtering, and evolving relationships between time points as relevant modeling considerations.
  • Forecasting tasks: Forecasting tasks vary by horizon and variables: short-term versus long-term prediction and univariate versus multivariate forecasting address different application requirements.Short-term forecasting emphasizes precision over shorter periods, whereas long-term forecasting considers longer trends and seasonal variations; multivariate forecasting predicts correlated variables jointly.
  • Statistical forecasting models: Classical forecasting developed from moving averages and exponential smoothing to AR, MA, ARMA, and ARIMA models that capture temporal dependence or address non-stationarity.Moving averages smooth observations, exponential smoothing weights recent values more heavily, and ARIMA uses differencing to transform non-stationary sequences.
  • Limitations of statistical forecasting: Traditional statistical models have major shortcomings with non-stationarity, nonlinear relationships, noise, complex dependencies, long-term dependencies, and multi-feature forecasting.The survey summarizes limitations in computational power, prediction accuracy, and forecasting length, while positioning deep learning as addressing these limitations.

3 DTSF Model Architecture

DTSF models are categorized by architectural structure into explicit and implicit paradigms, with encoder-decoder, Transformer, recurrent, convolutional, and adversarial approaches represented. The survey also compares model characteristics and examines specialized designs addressing long-sequence forecasting challenges.

  • Recurrent models: RNNs use recurrent connections to capture temporal relationships, while LSTM and GRU variants target long-term dependencies in sequential data.Examples include DeepAR, MQRNN, and ES-RNN, which combine recurrent modeling with autoregression, quantile regression, or exponential smoothing.
  • Convolutional models: TCNs process time series with dilated convolutions, expanding the receptive field without increasing parameter count and capturing long-range dependencies efficiently.CNNs extract local patterns from one-dimensional signals, while TCNs extend this capability to longer temporal ranges.
  • Generative Adversarial Model: GANs use adversarial training between a generator and discriminator to generate synthetic time-series samples and support forecasting.The section presents GANs alongside RNN, TCN, and Transformer architectures as major deep-learning approaches for TSF.
  • Architecture taxonomy: The survey dynamically classifies time-series models by architecture into five model architectures grouped as explicit or implicit structure paradigms.It summarizes major models and compares their advantages, disadvantages, application domains, and prediction horizons.
  • Encoder-Decoder Model: Encoder-decoder architectures map input sequences to output sequences and are particularly suitable for multi-step forecasting.Classic examples include Seq2Seq with LSTM encoder and decoder, U-Time, and asymmetric encoder-decoder frameworks.
  • Transformer models: Transformers combine attention with encoder-decoder structure, but long-sequence forecasting remains challenging because standard attention struggles with time-series characteristics.Autoformer addresses this challenge with decomposition and autocorrelation mechanisms, while related work develops specialized architectures and mechanisms.

4 Series Components and Enhanced Feature Extraction Methodology

This section examines time-series composition and four feature-extraction approaches that support richer representations for deep forecasting. It connects decomposition, time-frequency conversion, pre-training, and patch-based segmentation to the structure and learning needs of temporal data.

  • Section overview: The chapter first dissects real-world time-series constituents, then reviews four established feature-extraction methods with strong theoretical foundations and notable performance.The methods are presented as ways to develop a richer understanding of essential time-series features.
  • Series components: Time series generally comprise trend, seasonality, and residuals or white noise, as illustrated by the Exchange-Rate decomposition.The figure distinguishes original data, trend, seasonality, and residuals using separate colored lines.
  • Series components: Decomposition separates long-term movement, periodic variation, and unexplained fluctuations, helping identify structure and distinguish modeled patterns from random noise.The survey relates this separation to understanding non-stationary data and supporting more accurate forecasts.
  • Dimension decomposition: Dimension decomposition isolates constituent components and latent patterns, enabling focused analysis of features that influence overall time-series behavior.Recent DTSF work integrates seasonal-trend decomposition into encoder-decoder architectures, while enhanced decomposition supports contrastive learning of long-term representations.
  • Time-frequency conversion: Time-frequency conversion maps time-domain data into frequency representations to analyze spectral characteristics, dynamic variation, repetitive patterns, and periodic trends.Applications include preprocessing and feature extraction, with StemGNN and FreTS using frequency-domain learning for multivariate forecasting.
  • Pre-training and patch-based segmentation: Pre-training addresses temporal data’s low information density and long-context requirements, while patch-based segmentation captures local features from subseries.The survey also identifies pre-training–target-domain mismatch as a challenge and channel independence as an avenue for multivariate forecasting.

5 Challenges and Prospects

This section surveys unresolved challenges in DTSF, spanning privacy, interpretability, temporal continuity, efficiency, large models, representation learning, causal inference, and diffusion forecasting. It presents these issues as directions for future research rather than settled solutions.

  • Overview: DTSF research has examined architectures, feature learning, datasets, and challenges, with future directions organized around unresolved issues in the field.The survey frames these challenges as potential avenues for advancing deep forecasting.
  • Data privacy and completeness: Federated learning enables participants to train collaboratively without sharing sensitive raw time-series data, while exploiting distributed-data diversity.The survey presents FL as promising for large-scale, secure, and efficient forecasting and analysis.
  • Interpretability: Interpretability remains relatively limited compared with architectural work, motivating surrogate models, attention-based explanations, perturbation learning, and sparse system identification.The survey notes that attention weights can indicate relative feature importance at each time step, while sparse methods can train efficiently on CPUs.
  • Temporal continuity: Neural differential-equation methods model rates of change to capture temporal evolution and instantaneous changes in complicated time series.The survey describes NDEs as a flexible paradigm integrating deep learning with differential-equation modeling.
  • Efficiency: Real-time online forecasting is constrained by computational resources, data scale, and the cost of high-performance GPU servers.The survey identifies efficient forecasting as necessary for massive-data settings.
  • Large models: Large models are being explored for long-term dependencies, high-dimensional data, noise mitigation, medical inference, mobility, weather, and energy-demand forecasting.The survey expects their performance and value to continue unfolding as technology advances.
  • Representation learning: Temporal representation learning and contrastive training can enhance representation and generalization for complex, noisy, or changing data distributions.The survey presents this as a potential direction for improving TSF models’ handling of difficult distributions.
  • Counterfactual forecasting and causal inference: Time-dependent confounding is the primary challenge for causal inference in time series, while existing counterfactual explanations may be unrealistic or infeasible.Recent work targets actionable counterfactual explanations for forecasting.

6 Conclusion

The survey organizes deep learning-based time series forecasting across model architectures, feature extraction methods, datasets, challenges, and future research directions.

  • The survey identifies five major deep learning forecasting paradigms: Encoder-Decoder, Transformer, Generative Adversarial, Integration, and Cascade.
  • It analyzes time series composition and approaches for enhancing feature extraction and learning from time series data.
  • The survey compiles forecasting datasets spanning energy, healthcare, traffic, meteorology, and economics.
  • It concludes by outlining current challenges and proposing potential research directions for deep learning-based time series forecasting.

A Datasets in Different Domain

Time series forecasting datasets have accumulated across domains and provide research data with rich samples and features, while their diversity motivates methods for heterogeneous and imperfect data.

  • Forecasting datasets are widely used in computer research because they offer rich samples and features with significant reference value.
  • Multimodal learning integrates diverse data types through shared representation learning to improve handling of heterogeneous data.
  • Time alignment techniques synchronize data collected at different time granularities using a time-distance measure.
  • Deep generative models address missing values and noise by generating samples and rebuilding spatiotemporal dependencies.

A.1 Energy

Energy management is a prominent application of time series forecasting, where accurate predictions support status assessment, trend analysis, and intelligent engineering strategies.

  • Accurate energy forecasting supports status assessment and trend analysis, enabling intelligent strategies in engineering planning.
  • Modern energy systems collect extensive datasets covering electricity, wind energy, and solar energy.

A.1.1 Electricity Transformer Temperature (ETT)

The ETT-small datasets contain transformer measurements from two sites for oil-temperature prediction and resilience analysis under extreme loads, with 15-minute and hourly versions.

  • ETT-small records load profiles and oil temperatures from two power-transformer installations at separate sites.
  • The dataset supports predicting transformer oil temperature and investigating resilience under extreme load conditions.
  • Figure 16 presents time series datasets in primary domains.
  • Each 15-minute ETT-small dataset contains 70,080 data points covering two years, with hourly-granularity alternatives also available.

A.1.2 Electricity

The Electricity Load Diagrams 2011–2014 Dataset records customer electricity usage at 15-minute intervals. Preprocessing removed 2011 data and aggregated the remaining records hourly, yielding data for 321 customers from 2012 to 2014.

  • The original dataset records electricity usage for 370 customers from 2011 to 2014 at 15-minute intervals.
  • Preprocessing removed the 2011 records and aggregated consumption into hourly intervals to address dimensions containing zero values.
  • The final dataset contains electrical-use information for 321 customers from 2012 to 2014.

A.1.3 Wind (European Wind Generation)

The European Wind Generation dataset provides hourly energy-potential estimates for 28 European countries from 1986 to 2015. It contains sparse data and frequent zeros at regular intervals.

  • The dataset covers 28 European countries from 1986 to 2015.
  • It provides hourly estimates of energy potential expressed as a percentage of maximum power-plant output.
  • The dataset is characterized by sparse observations and frequent zeros at regular intervals.

A.1.4 Solar-Energy

The Solar-Energy dataset contains 10-minute solar-production records from 137 photovoltaic plants in Alabama during 2006. It supports evaluation of short-sequence forecasting capabilities.

  • The dataset records solar power production from 137 photovoltaic plants in Alabama.
  • The observations were collected at 10-minute intervals during 2006.
  • The dataset is used to evaluate short-sequence forecasting capabilities.

A.2 Healthcare

The reviewed datasets span healthcare, traffic, meteorology, finance, retail, and forecasting competitions. They vary widely in temporal resolution, geographic coverage, observation scale, and explanatory-variable availability.

  • Healthcare: Healthcare datasets support forecasting disease progression, treatment efficacy, patient vital signs, influenza-like symptoms, EEG activity, and cardiac arrhythmias.
  • Traffic: Traffic datasets include road occupancy and speed measurements from hundreds of sensors across California, with sampling intervals ranging from five minutes to one hour.
  • Meteorology: Meteorological datasets cover climate observations from nearly 1,600 US locations, 21 weather indicators at 10-minute resolution, and 32,072 Australian daily series.
  • Finance and retail: Financial datasets include daily exchange rates, high-frequency limit-order-book events, and retail sales records spanning multiple countries, stocks, stores, and product categories.
  • Additional datasets: The survey compiles additional datasets from influential works to support niche topics and detailed investigations across domains.
  • Competitions: Makridakis competitions provide large-scale benchmarks, including M4’s 100,000 real-life series across micro, industry, macro, finance, demographic, and other domains.
Loading 2503.10198v1…