Deep Learning for Time Series Forecasting: A Survey

• Deep learning autonomously extracts intricate temporal features and patterns, enabling long-term dependency and complex-relationship modeling for improved forecasting accuracy.The survey distinguishes DTSF from statistical and conventional machine-learning approaches that rely more on domain knowledge or feature engineering.Source: §1 · p. 2
• The survey addresses gaps in high-level architectural categorization, comprehensive recent-development coverage, and analysis of challenges and future prospects in DTSF research.These gaps motivate the review’s systematic treatment of deep-learning model structures and the field’s future development.Source: §1 · p. 2
• The survey proposes a dynamic, systematic classification of deep-learning TSF models from the perspective of architectural structure.The authors describe this as the first dynamic classification of deep-learning model architectures for time series forecasting.Source: §1 · p. 3
• It reviews feature-enhancement methods including dimensional decomposition, time-frequency transformation, pre-training, and patch-based segmentation to expose latent learning potential in complex, high-dimensional series.The analysis begins with the composition of time-series data features and then summarizes methods for enhancing them.Source: §1 · p. 3
• It compiles major recent TSF datasets, summarizes key challenges, and identifies promising future research directions.The paper also provides an exhaustive account of datasets across domains in Appendix A.Source: §1 · p. 3

2 Time Series Forecasting

• TSF forecasts future values from historical data by identifying intrinsic patterns and statistically similar features across adjacent observations.These discovered features support prediction-model design and improved forecasting results.Source: §2 · p. 3

2.1 Time Series Definition

• Time series are chronological observation sequences with fixed or variable intervals, represented as values indexed by time across one or more series.The survey denotes observations by yt and collections of N univariate series by Y = {yi,1:Ti}N.Source: §2.1 · p. 4

2.2 Forecasting Task

• Forecasting tasks differ by short- versus long-term horizons and by univariate versus multivariate variables, with long-term and multivariate settings requiring broader dependencies and influences.Long-term forecasting emphasizes trends and seasonality, while external factors such as weather, holidays, economic indicators, and road networks can improve predictions.Source: §2.2 · pp. 5–6

2.3 Statistical Forecasting Model

• Classical forecasting progresses from moving averages and exponential smoothing to AR, MA, ARMA, ARIMA, decision trees, and random forests.ARIMA addresses non-stationarity through differencing, while random forests capture nonlinear relationships and feature interactions.Source: §2.3 · pp. 6–7
• Traditional statistical models are limited in computational power, accuracy, and forecasting length, particularly for non-stationarity, nonlinear relationships, noise, complex dependencies, long-term dependencies, and multi-feature tasks.These shortcomings motivate the development of deep learning forecasting models with improved predictive performance.Source: §2.3 · p. 8

3 DTSF Model Architecture

• The survey dynamically classifies time series models by internal architecture into five explicit- and implicit-structure paradigms, compiling their characteristics to guide task-specific model selection.Figure 2 presents the classification, while Tables 1 and 2 summarize representative models, advantages, disadvantages, applications, and prediction horizons.Source: §3 · p. 10

3.1.1 Encoder-Decoder Model

• Encoder-decoder architectures map input sequences to outputs for multi-step forecasting, using symmetric or asymmetric designs that combine recurrent, convolutional, and attention-based processing.Examples include Seq2Seq, U-Time, Transformer variants, Guo et al. (2023), and HANet, with some architectures lacking an explicit decoder.Source: §3.1.1 · pp. 11–12

3.1.2 Transformer Model

• Transformers address long-sequence forecasting limitations through ProbSparse attention, decomposition, patching, exponential-smoothing and frequency attention, and frequency-enhanced designs.Informer reduces complexity and memory usage to O(LlogL), while Autoformer, ETSformer, FEDformer, and related models tailor architecture to temporal structure.Source: §3.1.2 · pp. 13, 15

3.1.3 Generative Adversarial Model

• GAN-based TSF primarily generates synthetic time series by adversarially training generators and discriminators, with TimeGAN, RGAN, and RCGAN extracting or preserving realistic temporal characteristics.TimeGAN combines embedding, recovery, sequence-generation, and discrimination functions, whereas RCGAN generates sequences conditioned on specific inputs.Source: §3.1.3 · pp. 15–16

3.2.1 Integrated Model

• Integrated models combine complementary architectures so convolutional components learn local patterns while recurrent components capture temporal dependencies, improving feature learning across time series applications.Examples include convolutional replacements for fully connected LSTM layers in precipitation forecasting and sequential LSTM-GRU integration for stock prediction.Source: §3.2.1 · p. 17

3.2.2 Cascade Model

• Cascade models stack repeated or heterogeneous components to form deeper networks, but relatively few TSF studies explicitly develop cascade architectures beyond repeated modules and specialized information-flow designs.The cascade concept appears in Transformer series and in models such as Zhou et al. (2022c).Source: §3.2.2 · pp. 17–18

4 Series Components and Enhanced Feature Extraction Methodology

• Time-series forecasting benefits from learning representations through a process of pre-training, decomposition, extraction, and refinement aligned with data properties.The chapter addresses the limitations of understanding model architectures alone by emphasizing methodological approaches to temporal feature learning.Source: §4 · p. 18

4.1 Components of a Time Series

• Time series generally decompose into trend, seasonality, and residuals or white noise, helping distinguish long-term change, periodic variation, and unexplained fluctuations.Decomposition supports structure discovery, hidden-pattern identification, useful-information extraction, and more accurate forecasting for non-stationary data.Source: §4.1 · p. 18, §4.1.3 · p. 19

4.1.1 Trend

• Trend represents long-term growth or decline, while seasonality captures recurring variations caused by monthly, weekly, seasonal, or other temporal influences.Examples include population growth, wind-power increases across windy seasons, tourism, and ice-cream sales during vacations or summer.Source: §4.1.1 · pp. 18–19, §4.1.2 · p. 19

4.1.3 Residuals

• Residuals capture irregular fluctuations unexplained by trend and seasonality, and their autocorrelation can reveal modeling flaws that require adjustment.Residual analysis therefore contributes to forecasting quality and reliability.Source: §4.1.3 · p. 19

4.2.1 Dimension Decomposition

• Dimension decomposition isolates trends, seasonal patterns, and residuals to expose latent structure, improve interpretability, and support focused component analysis.Encoder-decoder and decomposition models include FEDformer, LaTS, TDformer, and multiscale trend-residual approaches; contrastive decomposition improves long-term representation learning and forecasting.Source: §4.2.1 · pp. 20–21

4.2.2 Time-Frequency Conversion

• Time-frequency conversion reveals spectral characteristics, repetitive patterns, periodic trends, and dynamic variations that complement time-domain analysis.Methods such as StemGNN and FreTS learn inter-series and intra-series correlations, while Fourier filtering and low-rank approximation can reduce noise and overfitting.Source: §4.2.2 · pp. 21–22

4.2.3 Pre-training

• Pre-training addresses temporal data’s low information density, evolving dynamics, and long-short-term effects, but mismatches between pre-training and target domains can reduce downstream performance.Contrastive methods align time- and frequency-based neighborhoods, whereas masking methods simulate incompleteness and noise to improve dependency learning, uncertainty tolerance, and generalization.Source: §4.2.3 · pp. 22–24

4.2.4 Patch-based segmentation

• Patch-based segmentation divides long series into subseries-level tokens, strengthening local-pattern perception while channel-independent modeling preserves variable-specific information.PatchTST exemplifies this strategy, and graph modeling can additionally learn cross-variable feature correlations without relying on redundant information.Source: §4.2.4 · pp. 24–25

5.1.1 Lack of Data Privacy Protection and Completeness

• TSF must address data privacy and completeness, with federated learning enabling collaborative training without sharing sensitive raw data while potentially improving generalization and accuracy.Federated learning aggregates local model parameters into a global predictor and has been explored in load, traffic, energy, and communication applications.Source: §5.1.1 · pp. 26–27

5.1.2 Lack of Interpretability

• Interpretability remains limited; promising approaches include interpretable architectures, surrogate or gradient-based explanations, attention analysis, and sparse identification methods.N-BEATS combines accuracy with interpretability, while attention weights can reveal feature importance and persistent temporal patterns such as seasonality.Source: §5.1.2 · p. 27

5.1.3 Lack of Temporal Continuity

• Neural differential-equation methods address temporal continuity by modeling changing states and supporting interpretable predictions at arbitrary time instances, including multivariate continuous-time representations.ETN-ODE provides interpretable continuous forecasting, while EgPDE-Net converts a multivariate PDE formulation into an ODE problem for arbitrary-step prediction.Source: §5.1.3 · p. 28

5.1.4 Challenges of Parallel Computing

• Real-time forecasting remains constrained by computational resources, data scale, and costly GPU servers, motivating efficient parallel computing for large-scale online analysis.Current models often rely on stand-alone sequence analysis, limiting attainable real-time forecasting.Source: §5.1.4 · p. 28

5.1.5 Challenges of Large Models

• Large models can capture long-term dependencies, high-dimensional data, and noise, while emerging foundational time-series models such as TimeGPT-1 enable zero-shot inference.Applications span financial, medical, mobility, weather, and energy forecasting, but the performance and value of large models remain an evolving prospect.Source: §5.1.5 · pp. 28–29

5.2.1 Potential Representation Learning

• Representation learning seeks compact latent features through self-supervised, unsupervised, pretraining, ensemble, and contrastive methods, improving generalization on complex, noisy, or changing data.Contrastive learning is particularly useful when labeled data is limited or unavailable.Source: §5.2.1 · p. 29

5.2.2 Counterfactual Forecast and Causal Inference

• Counterfactual forecasting and causal inference must handle time-dependent confounding and produce realistic, feasible, and actionable explanations for time-series decisions.Existing approaches use statistical techniques, new losses, model extensions, and inference algorithms, while recent work introduces self-interpretable actionable counterfactuals.Source: §5.2.2 · pp. 29–30

5.2.3 TS Diffusion; 5.2.4 Determine the Weight of the Aggregate Model; 5.2.5 Interdisciplinary Exploration

• Future DTSF research includes diffusion models, reinforcement-learning-based ensemble weighting, and interdisciplinary network analysis, while diffusion methods still face inference speed, complexity, and boundary-consistency challenges.RLMC dynamically weights ensemble models for non-stationary data, and network science can represent hierarchical correlations among many interconnected variables.Source: §5.2.3 · p. 30, §5.2.4 · p. 31, §5.2.5 · p. 31

6 Conclusion

• The survey synthesizes five major forecasting model paradigms: Encoder-Decoder, Transformer, Generative Adversarial, Integration, and Cascade.It also reviews foundational forecasting concepts, statistical methods and shortcomings, time series composition, and datasets spanning energy, healthcare, traffic, meteorology, and economics.Source: §6 · p. 31

Datasets in Different Domains

• TSF datasets cover five application areas—energy, transportation, economics, meteorology, and healthcare—and exhibit heterogeneous structures, formats, time granularities, scales, and modalities.Examples include sensor data, text, and images, creating significant challenges for model construction.Source: A Datasets in Different Domain · p. 53

Methods for Heterogeneous Data

• Multimodal learning, time alignment, deep generative models, self-supervised learning, and collaborative attention address heterogeneous data and improve forecasting stability and accuracy.These approaches include shared representations, time-distance synchronization in TAM, missing-value and noise reconstruction in GinAR (Yu et al., 2024), unlabeled-data learning with SimCLR (Chen et al., 2020a), and dynamic modality weighting (Dosovitskiy et al.).Source: A Datasets in Different Domain · p. 53

A.1 Energy

• Energy forecasting supports status assessment, trend analysis, and intelligent strategies in engineering planning.Modern energy systems provide extensive data from electricity, wind energy, and solar energy.Source: §A.1 · p. 53

Dataset scope

• ETT-small provides oil-temperature forecasting data from two geographically separated transformer sites, with load profiles and oil-temperature readings collected from July 2016 to July 2018 at 15-minute intervals.The dataset investigates transformer resilience under extreme load conditions and includes the ETT-small-m1 and ETT-small-m2 subsets.Source: §A.1.1 · pp. 53–54

Temporal resolution

• Each 15-minute ETT-small subset contains 70,080 observations, while hourly versions are provided as ETT-small-h1 and ETT-small-h2.The dataset comprises ETT-small-m1 and ETT-small-m2, each representing two years of measurements.Source: §A.1.1 · p. 54

Feature composition

• Every ETT data point has an 8-dimensional feature vector containing the timestamp, oil temperature, and six external load variables.These features combine temporal information, the forecasting target, and external-load measurements.Source: §A.1.1 · p. 54

A.1.2 Electricity

• The initial dataset records electricity usage for 370 customers from 2011 to 2014 at 15-minute intervals.The source is the Electricity Load Diagrams 2011-2014 Dataset.Source: §A.1.2 · p. 54
• The data were preprocessed by removing 2011 records and aggregating consumption into hourly intervals to address zero-valued dimensions.Source: §A.1.2 · pp. 54–55
• The final dataset contains electrical-use information for 321 customers from 2012 to 2014.Source: §A.1.2 · p. 55

A.1.3 Wind (European Wind Generation)

• The dataset offers hourly energy-potential estimates, expressed as a percentage of power plants’ maximum output, for 28 European countries between 1986 and 2015.It is characterized by sparser data and a notable frequency of zeros occurring at regular intervals.Source: §A.1.3 · p. 55

A.1.4 Solar-Energy

• The evaluation dataset contains 10-minute solar power production records from 137 photovoltaic plants in Alabama State during 2006.It is used to assess short-sequence forecasting capabilities.Source: §A.1.4 · p. 55

A.2 Healthcare; A.2.1 ILI (Influenza-Like Illness); A.2.2 EEG (Electroencephalogram); A.2.3 MIT-BIH (Arrhythmia Database)

• Healthcare forecasting supports disease prediction, treatment evaluation, vital-sign monitoring, diagnosis, and preventive surveillance across ILI, EEG, and arrhythmia datasets.ILI contains US CDC weekly reports from 2002–2021; EEG provides artifact-free 60-second recordings with 19 channels; MIT-BIH includes 48 ambulatory ECG segments from 47 individuals for arrhythmia prediction.Source: §A.2 · p. 55, §A.2.1 · p. 55, §A.2.2 · p. 55

A.2–A.6 Dataset Scope

• Across the surveyed collection, the datasets vary substantially in temporal resolution, geographic coverage, dimensionality, domain, and forecasting target.Examples range from weekly epidemiological reports and hourly traffic or climate data to five-minute traffic measurements, ten-minute weather indicators, daily sales and exchange rates, and millisecond financial events.Source: §A.2.1 · p. 55, §A.3.1 · p. 56, §A.4.2 · p. 57

A.3 Transportation; A.3.1 Traffic; A.3.2 PeMSD4/7/8

• Transportation datasets support congestion, flow, accident-rate, and public-transportation forecasting, including hourly occupancy measurements from 862 Bay Area sensors and standardized PeMS benchmarks.PeMSD4 contains 307 sensors across 29 roads, PeMSD7 contains five-minute speeds from 228 sensors, and PeMSD8 contains five-minute data from 170 detectors across eight routes.Source: §A.3 · p. 56, §A.3.1 · p. 56, §A.3.2 · p. 56

A.4 Meteorology; A.4.1 Weather1; A.4.2 Weather2; A.4.3 Temperature Rain

• Meteorological forecasting datasets span US climate observations, high-frequency weather indicators, and Australian temperature–rain series across thousands of locations and stations.Weather1 covers nearly 1,600 US locations from 2010–2013 with hourly wet-bulb targets and 11 features; Weather2 has 21 indicators sampled every 10 minutes; Temperature Rain contains 32,072 daily series from 422 Australian stations during 2015–2017.Source: §A.4 · p. 56, §A.4.1 · p. 56, §A.4.2 · p. 57

A.5 Economics; A.5.1 Exchange-Rate; A.5.2 LOB-ITCH; A.5.3 Dominick

• Economic forecasting covers exchange rates, limit-order-book events, retail sales, and financial assets, with datasets ranging from long historical records to millisecond-scale market activity.Exchange-Rate spans eight countries from 1990–2016; LOB-ITCH contains about four million 144-dimensional events across five stocks and ten trading days; Dominick provides roughly nine years of sales for more than 3,500 UPCs across 100 stores.Source: §A.5 · p. 57, §A.5.1 · p. 57, §A.5.2 · p. 57

A.6 Further Data Sources; A.6.1 M3; A.6.2 M4; A.6.3 M5

• Additional sources include influential niche datasets and competition benchmarks such as M3, M4, and M5, which support comparison and detailed forecasting investigations.M3 includes multiple frequencies and domains with minimum observation thresholds; M4 contains 100,000 real-life series; M5 combines item, department, category, and store data with price, promotion, calendar, and event variables.Source: §A.6 · p. 58, §A.6.1 · p. 58, §A.6.2 · p. 58

A.6.4 M6

• M6 focuses on forecasting returns and risks for 50 S&P 500 stocks and 50 international ETFs, including investment decisions based on those predictions.Its two asset categories comprise 50 stocks selected from the S&P 500 Index and 50 ETFs from international exchanges.Source: §A.6.4 · p. 60