Source-linked AI summary
A Survey on Deep Learning based Time Series Analysis with Frequency Transformation
Kun Yi, Qi Zhang, Wei Fan, Longbing Cao, Shoujin Wang, Guodong Long, Liang Hu, Hui He, Qingsong Wen, Hui Xiong
TL;DR
Deep learning models using frequency transformation have shown efficient and effective time series analysis, but the field lacks a systematic review explaining their approaches, benefits, and limitations. This paper surveys these models, organizes them by incorporation strategies, neural network designs, and applications, and synthesizes how frequency transformation supports global pattern capture and scalable processing.
Problem
Existing deep learning time series models with frequency transformation lack a systematic review of their approaches, advantages, and limitations.
Method
The paper provides a comprehensive survey organized around frequency-transformation incorporation strategies, neural network designs, application domains, and representative models.
Results
The survey summarizes frequency transformation as an efficient approach that can capture global patterns and reduce computational cost in time series analysis.
Takeaways & Limitations
Frequency transformation offers promising opportunities for efficient and scalable time series analysis across applications, including large-scale data processing.
Takeaways & Limitations
Frequency transformation may overlook temporal information and depends on predefined parameters whose poor selection can produce inaccurate representations or miss important components.
Abstract
from arXiv · showhide
Recently, frequency transformation (FT) has been increasingly incorporated into deep learning models to significantly enhance state-of-the-art accuracy and efficiency in time series analysis. The advantages of FT, such as high efficiency and a global view, have been rapidly explored and exploited in various time series tasks and applications, demonstrating the promising potential of FT as a new deep learning paradigm for time series analysis. Despite the growing attention and the proliferation of research in this emerging field, there is currently a lack of a systematic review and in-depth analysis of deep learning-based time series models with FT. It is also unclear why FT can enhance time series analysis and what its limitations are in the field. To address these gaps, we present a comprehensive review that systematically investigates and summarizes the recent research advancements in deep learning-based time series analysis with FT. Specifically, we explore the primary approaches used in current models that incorporate FT, the types of neural networks that leverage FT, and the representative FT-equipped models in deep time series analysis. We propose a novel taxonomy to categorize the existing methods in this field, providing a structured overview of the diverse approaches employed in incorporating FT into deep learning models for time series analysis. Finally, we highlight the advantages and limitations of FT for time series modeling and identify potential future research directions that can further contribute to the community of time series analysis.
1 Introduction
Time series analysis is difficult because of intricate inter-series correlations and intra-series dependencies, while time-domain models often rely on point-wise connections that miss global patterns. This survey systematically reviews deep learning methods using frequency transformation, proposes a taxonomy, and analyzes FT’s benefits and limitations.
- Time series analysis supports forecasting, anomaly detection, classification, and diverse applications but is challenging because of intricate inter-series correlations and intra-series dependencies.
- Time-domain deep learning models use variable-wise or timestamp-wise connections, which can fail to model whole- or sub-series global patterns.
- Frequency transformation has attracted increasing interest because FT-equipped neural models achieve strong efficiency and effectiveness in time series analysis.
- Existing literature lacks a comprehensive review explaining how FT is incorporated, why it enhances time series analysis, and what limitations it has.
- This paper reviews incorporation strategies, neural network designs, representative FT-equipped models, FT’s advantages and limitations, and introduces a new taxonomy.
2.1 Time Series Analysis
Time series analysis includes forecasting, classification, anomaly detection, and imputation, each defined by mapping temporal observations to predictions, labels, anomalies, or reconstructed values. The section introduces these tasks and their formal inputs and outputs.
- Time Series Forecasting: Time series forecasting predicts the next τ values from a window of L past observations using a learned mapping f_θ.
- Time Series Classification: Time series classification assigns each input series a categorical label by learning class-probability outputs from labeled examples.
- Anomaly Detection: Time series anomaly detection identifies data points or subsequences that deviate significantly from expected temporal patterns.
- Time Series Imputation: Time series imputation reconstructs a complete series from partially observed values and a binary missingness mask.
2.2 Frequency Transformation
The survey introduces DFT, DCT, and DWT as transformations from time-domain data to frequency-domain representations, then explains the convolution theorem. These methods differ in representation and temporal localization, with DWT supporting time-frequency analysis and multi-resolution views.
- DFT, DCT, and DWT convert time-domain data into frequency-domain representations for time series analysis.
- Discrete Fourier Transform: DFT maps a length-N sequence into frequency components, whose spectrum represents content at frequencies ω_k = 2πk/N.
- Discrete Cosine Transform: DCT is commonly used for real-valued data and retains only the real parts of the DFT, often benefiting signals with even symmetry.
- Discrete Wavelet Transform: DWT decomposes a signal into coefficient series describing its evolution across frequency bands, enabling time-frequency analysis and different resolutions.
- Convolution Theorem: The convolution theorem turns point-wise multiplication in the frequency domain into circular convolution in time, spanning entire sequences to capture global patterns while reducing computational cost.
3 Incorporation Approach
Existing models incorporate frequency transformation mainly as feature engineering to capture periodic, multi-scale, and global-dependency information. Other uses include sparse representations, noise filtering, frequency-domain augmentation, and frequency-enhanced neural operators.
- Frequency transformation is used as feature engineering to capture periodic patterns, multi-scale patterns, and global dependencies.
- Periodicity: Frequency-domain branches and layers model periodic behavior, including dynamic periodicity, periodic anomalies, seasonal components, and trading patterns.
- Multi-Scale: Frequency decomposition disentangles temporal dynamics into components, allowing models to learn dependencies across low- and high-frequency patterns.
- Global Dependencies: The frequency domain provides a global view that helps models capture series-level characteristics and global properties beyond point-wise time-domain connections.
- Frequency transformations can produce sparse representations, filter high-frequency noise, support frequency-based augmentation, and enhance neural operators for differential equations and forecasting.
4 Neural Network Design
Neural-network design depends partly on whether the transformation produces complex or real values. Models also select discriminative frequency components to focus computation on informative spectral patterns.
- Frequency-based models handle transformation outputs differently according to whether they are complex-valued or real-valued.
- Complex-Valued Outputs: DFT outputs can be represented by real-imaginary or amplitude-phase parts, but discarding one component may cause information loss.
- Complex-Valued Outputs: Complex-valued processing commonly applies separate neural networks to each component, combines their outputs as complex values, and then transforms them back to the time domain.
- Real-Valued Outputs: DCT and DWT produce real outputs that standard network structures can process directly, while some DFT methods discard a component or apply frequency-domain filtering.
- Frequency Component Selection: Frequency-component selection identifies discriminative or critical components, including ranked wavelet levels and selected dominant periods.
5 Applications
The survey organizes FT-equipped neural time series models across forecasting, anomaly detection, classification, and imputation. Representative methods use frequency information for periodicity, global structure, spectral discrimination, anomaly patterns, and component-specific reconstruction.
- The survey categorizes representative FT-equipped models into forecasting, anomaly detection, classification, and imputation applications.
- Forecasting: Forecasting models use frequency information for multi-frequency patterns, periodicity, global views, seasonal representations, low-frequency components, and non-stationary dynamics.
- Anomaly Detection: Anomaly-detection models exploit frequency augmentation, seasonal anomalies, spectral sparsity, and frequency-domain patches to improve detection across diverse patterns.
- Classification: Classification models use DWT decomposition, adjustable-scale wavelets, and fused time-spectral features to improve discriminative and expressive representations.
- Imputation: Imputation models apply high-frequency filters for residuals, dominant-frequency filters for trend and seasonality, or spectral discrepancies for distribution matching.
6 Summary of Frequency Transformation
Frequency transformation offers decomposition, global sequence views, sparse representations, and computational efficiency, but can lose temporal information and depends on carefully chosen parameters.
- Advantages: Frequency transformation decomposes time series into frequency components, including multi-scale sub-series, to expose information useful for analysis.
- Advantages: Each frequency-spectrum element aggregates signals across timestamps, giving frequency-domain representations a global view of the sequence.
- Advantages: Sparse frequency representations can reduce memory use and accelerate processing by discarding or compressing negligible coefficients.
- Limitations: DFT and DCT may overlook temporal patterns and dynamics because they emphasize frequency characteristics over temporal information.
- Limitations: Frequency transformation requires choices such as window size, sampling rate, or frequency bands, and suboptimal settings can miss important components.
7 Discussion for Future Opportunities
Future research should extend frequency transformation with new orthogonal transforms, deeper neural-network integration, and joint time-frequency representation learning.
- The Perspective of Frequency Transformation: Partial Fourier Transform computes only selected Fourier coefficients, while Fractional Fourier Transform supports noise removal, anomaly discrimination, feature fusion, and global-context extraction.
- The Perspective of Frequency Transformation: Future work should incorporate additional orthogonal transforms, such as Fractional Fourier Transform, into deep-learning models for time series analysis.
- Integrating Frequency Transformation with Deep Learning: Fixed sine, cosine, and wavelet basis functions produce domain-invariant frequency features, motivating further combinations of frequency transformation with neural-network learning.
- Integrating Frequency Transformation with Deep Learning: Because frequency representations emphasize periodic components and may miss trends, future models should learn complementary representations in both time and frequency domains.
7.2 The Perspective of Time Series Analysis
The survey identifies broader applications, scalable analysis, and privacy-preserving processing as future opportunities for frequency transformation in time series analysis.
- Applying Frequency Transformation to Enhance Time Series Applications: Frequency transformation could reveal temporal patterns and improve forecasting, resource management, and data-driven decision-making across diverse applications.
- Scalability: Frequency transformation can support scalable analysis of large time series by reducing dimensionality, computational complexity, and memory requirements.
- Privacy Preservation: Frequency-domain representations may enable analysis and sharing of time series patterns without revealing the underlying raw data, supporting privacy-sensitive collaboration.
8 Conclusion
The paper presents a systematic survey of deep-learning time series methods using frequency transformation, organizing methods and synthesizing their benefits, limitations, and future directions.
- Conclusion: The survey organizes frequency-transformation methods by incorporation approach, neural-network design, and application domain.
- Conclusion: It summarizes the advantages and limitations of frequency transformation and aims to provide a focused resource for further research.