Source-linked AI summary
FilterNet: Harnessing Frequency Filters for Time Series Forecasting
Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, Wei Fan
TL;DR
Transformer forecasters have state-of-the-art results, but the paper identifies challenges with high-frequency signals, computational efficiency, and full-spectrum use. FilterNet uses learnable frequency filters to model temporal patterns, and experiments report superior effectiveness and efficiency across eight forecasting benchmarks.
Problem
Transformer forecasters face vulnerabilities to high-frequency signals, computational-efficiency concerns, and bottlenecks in full-spectrum use, which matter for forecasting long time series.
Source: §1 · p. 2Method
FilterNet uses learnable plain and contextual shaping filters to selectively pass or attenuate signal components and extract temporal patterns.
Source: §1 · p. 2Results
FilterNet outperforms state-of-the-art forecasting algorithms in effectiveness and efficiency across eight benchmarks.
Source: §1 · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
While numerous forecasters have been proposed using different network architectures, the Transformer-based models have state-of-the-art performance in time series forecasting. However, forecasters based on Transformers are still suffering from vulnerability to high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series with thousands of points. In this paper, we explore a novel perspective of enlightening signal processing for deep time series forecasting. Inspired by the filtering process, we introduce one simple yet effective network, namely FilterNet, built upon our proposed learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals. Concretely, we propose two kinds of learnable filters in the FilterNet: (i) Plain shaping filter, that adopts a universal frequency kernel for signal filtering and temporal modeling; (ii) Contextual shaping filter, that utilizes filtered frequencies examined in terms of its compatibility with input signals for dependency learning. Equipped with the two filters, FilterNet can approximately surrogate the linear and attention mappings widely adopted in time series literature, while enjoying superb abilities in handling high-frequency noises and utilizing the whole frequency spectrum that is beneficial for forecasting. Finally, we conduct extensive experiments on eight time series forecasting benchmarks, and experimental results have demonstrated our superior performance in terms of both effectiveness and efficiency compared with state-of-the-art methods. Code is available at this repository: https://github.com/aikunyi/FilterNet
1 Introduction
Transformer forecasters achieve state-of-the-art performance, yet the paper identifies difficulties with high-frequency signals, computation, and full-spectrum use. FilterNet applies learnable frequency filters to forecasting and is reported to outperform state-of-the-art methods across eight benchmarks in effectiveness and efficiency.
1 Introduction
- The paper frames forecasting as an important task across applications including climate, energy, traffic, finance, and industrial systems.It notes that deep-learning forecasting models have enhanced the field.Source: §1 · p. 1
- Transformers face vulnerabilities to high-frequency signals, computational-efficiency concerns, and limits in full-spectrum use; iTransformer [17] performs much worse on a synthetic three-frequency forecasting task.The paper presents this observation as motivation for exploring signal-processing techniques.Source: §1 · p. 2
- FilterNet uses learnable plain and contextual shaping filters to selectively pass or attenuate signal components and extract temporal patterns.Plain shaping uses a universal frequency kernel; contextual shaping assesses filtered frequencies for compatibility with the input when learning dependencies.Source: §1 · p. 2
- The filters can approximately surrogate linear and attention mappings used in time-series forecasting literature [14] [17].Instance normalization and the fast Fourier transform wrap the filters to address non-stationarity and transform the time-series domain, respectively.Source: §1 · p. 2
- FilterNet outperforms state-of-the-art forecasting algorithms in effectiveness and efficiency across eight time-series benchmarks.The average improvement over baseline models is reported as statistically significant at 95% confidence.Source: §1 · p. 2
2 Related Work
Prior time-series forecasting methods use diverse architectures, while frequency techniques have been integrated into deep-learning models in several ways. FilterNet differs by using all-pass frequency filters as the forecasting network itself rather than adding them to another architecture or using them only as low-pass filters.
2 Related Work
- Forecasting models span RNNs and TCNs as well as MLPs and Transformers; cited examples include DLinear, PatchTST [16], and iTransformer [17].The passage describes Transformer- and MLP-based methods as competitive or state-of-the-art approaches.Source: §2 · p. 3
- Frequency-based forecasting approaches include FFT-enhanced attention in Autoformer [15] and FEDformer [25], periodic-pattern extraction in DEPTS [30], and frequency-domain MLPs in FreTS [13].Other cited examples use filtering, graph operations, or low-pass filtering, including FiLM [31], FourierGNN [29], and FITS [32].Source: §2 · p. 3
- FilterNet applies all-pass frequency filters directly to build the forecasting network, unlike approaches that insert frequency operations into other architectures or use low-pass filters as in FITS [32] and FiLM [31].The paper contrasts its design with Transformer-, MLP-, and GNN-based architectures.Source: §2 · p. 3
3 Preliminaries
The preliminaries describe frequency filtering as spectral modification that corresponds to circular convolution in the time domain. They connect this operation to a FilterNet architecture that normalizes inputs, filters temporal patterns, and projects them back for prediction.
3 Preliminaries
- A frequency filter modifies a signal's spectral content by changing the amplitude and phase of selected frequency components.For input signal x[n] with Fourier transform X[k], the filter H[k] produces output spectrum Y[k] = X[k]H[k].Source: §3 · p. 3
- Pointwise multiplication in the frequency domain corresponds to circular convolution in the time domain, and self-attention can be interpreted as a form of global circular convolution [35].The paper presents this connection as a basis for considering frequency filters in forecasting models.Source: §3 · p. 3
- FilterNet normalizes the input, applies a frequency-filter block, then uses a feed-forward network to project extracted temporal patterns back onto the series for prediction.The filter block has plain and contextual shaping-filter implementations.Source: Figure 2 · p. 4
4 Methodology
FilterNet frames temporal-dependency modeling as frequency filtering, using plain and contextual learnable filters in its forecasting architecture.
4.1 Overview
- Instance normalization addresses distribution shifts; FilterBlock models temporal dependencies in frequency space, and an FFN plus inverse normalization produces forecasts.The architecture applies normalization to the input and inverse normalization to the FFN output.Source: §4.1 · pp. 4–5
4.2 Plain Shaping Filter
- PaiFilter uses a randomly initialized learnable frequency kernel; Figure 3 contrasts channel-shared and channel-unique parameterizations.The filter implements frequency filtering through circular convolution in the time domain.Source: §4.1 · p. 5, Figure 3 · p. 5, §4.2 · p. 6
4.3 Contextual Shaping Filter
- TexFilter generates an input-dependent frequency filter with Hφ, then weights the frequency representation, adapting filtering to changing input conditions.Its network derives the contextual filter from the input's Fourier representation.Source: §4.3 · p. 6
5 Experiments
FilterNet is evaluated on eight real-world forecasting benchmarks and through analyses of its frequency filters. The experiments report stronger overall forecasting performance and examine filter behavior, channel sharing, prediction quality, and efficiency.
5.1 Experimental Setup
- The experiments use diverse real-world datasets, representative frequency-, TCN-, MLP-, and Transformer-based baselines, and MAE and MSE evaluation metrics.Datasets are split into training, validation, and test sets in a 7:2:1 ratio.Source: §5.1 · p. 6
5.2 Main Results
- Across eight benchmarks, FilterNet consistently outperforms baselines; its average improvement over all baselines is significant at 95% confidence.Table 1 reports prediction lengths of 96, 192, 336, and 720 with a lookback length of 96.Source: §5.2 · p. 7, Table 1 · p. 7
5.3 Model Analysis
- On noisy trend and multi-periodic signals, PaiFilter effectively models trend and periodic patterns; Figure 4 reports FilterNet’s more robust forecasting than iTransformer [17] under noise.Source: §5.3 · p. 7, Figure 4 · p. 8
- Channel-shared filters outperform channel-unique filters across all tested prediction lengths, and their ETTh1 predictions fit the ground truth better.The comparisons use ETTh1 and Exchange; the paper notes that channel sharing is also validated in DLinear and PatchTST [16].Source: §5.3 · p. 8, Table 2 · p. 8
- On ETTh1, FilterNet delivers the most accurate future-series predictions among the compared iTransformer [17] and PatchTST [16] models.The showcase uses a lookback window and horizon of 96.Source: §5.3 · p. 8, Figure 5 · p. 8
- FilterNet shows adaptive responses across frequency components, while the Traffic dataset's main patterns lie primarily at low frequencies.The visualizations cover Weather, ETTh1, and Traffic; the authors relate Traffic's low-frequency patterns to iTransformer's performance on that dataset.Source: §5.3 · pp. 8–9, Figure 7 · p. 9, Figure 8 · p. 9
- FilterNet has O(Log L) complexity, surpasses Transformer models in efficiency on both dataset sizes, and has efficiency similar to DLinear while outperforming it.The efficiency comparison uses Exchange and Electricity with lookback and prediction lengths of 96.Source: §5.3 · p. 9, Figure 6 · p. 9
6 Conclusion Remarks
FilterNet applies learnable frequency filters directly to time-series forecasting. Experiments on eight benchmarks support its effectiveness and efficiency relative to state-of-the-art methods.
6 Conclusion Remarks
- FilterNet uses two kinds of frequency filters to forecast time series.The paper presents the architecture as a signal-processing-inspired approach.Source: §6 · p. 9
- Across eight benchmarks, experiments validate FilterNet’s superiority in effectiveness and efficiency.The paper also reports in-depth analyses of FilterNet and its internal filters.Source: §6 · p. 9
- The authors hope this work will encourage further research integrating signal processing or filtering with deep learning for time-series modeling and forecasting.Source: §6 · p. 9
A More Analysis about the Architecture of FilterNet
The architecture analysis treats instance normalization as a separate block and describes how frequency filters and a feed-forward network contribute to modeling. The filter-design discussion relates two filter types to static and context-sensitive dependencies.
A More Analysis about the Architecture of FilterNet
- Instance normalization is treated as a distinct block because it normalizes instances, unlike filters that discern patterns in input data.The analysis relates the mean to the zero-frequency component and the standard deviation to power spectral density.Source: A More Analysis about the Architecture… · p. 13
- Frequency filters extract and emphasize temporal patterns and trends, including recurring and periodic behaviors relevant to forecasting.The discussion connects filters with global convolution views of self-attention and MLP architectures.Source: A More Analysis about the Architecture… · p. 13
- A feed-forward network complements frequency filters by modeling complex relationships and nonlinear patterns that filters may not fully capture.The passage describes the combination as enriching FilterNet’s expressive power.Source: A More Analysis about the Architecture… · p. 13
B Explanations about the Design of Two Filters
- Self-attention derives parameters from input data and applies them back, while its Q, K, and V are linear transformations of that input.The resulting attention scores are data-dependent because Q, K, and V come from the input.Source: B Explanations about the Design of Two… · pp. 13–14
- MLPs apply fixed transformations: after training, their learned weights and biases do not change in response to new inputs.The passage contrasts this with self-attention’s dynamic adaptation to input data.Source: B Explanations about the Design of Two… · p. 14
- FilterNet’s plain shaping filters suit static relationships, whereas contextual shaping filters capture dynamic dependencies for context-sensitive analysis.The paper presents this paired design as combining stability and efficiency with flexibility.Source: B Explanations about the Design of Two… · p. 14
C.1 Datasets
The evaluation uses eight datasets spanning exchange rates, weather, traffic, electricity use, and transformer temperatures. The paper compares FilterNet with twelve established models from several architectural categories.
C.1 Datasets
- The eight datasets cover Exchange, Weather, Traffic, Electricity, and four ETT series, spanning financial, meteorological, transportation, and energy measurements.The ETT series record seven features, including load and oil temperature.Source: §C.1 · p. 14
- The datasets include daily exchange rates, 10-minute weather observations, hourly traffic and electricity data, and ETT measurements collected every 15 minutes or hour.Exchange covers eight countries; Traffic uses 862 sensors, and Electricity covers 321 clients.Source: §C.1 · p. 14
C.2 Baselines
- The comparison includes twelve models across frequency-based, TCN-based, MLP-based, and Transformer-based categories.Source: §C.2 · p. 14
C.3 Implementation Details
The experiments tune FilterNet’s main hyperparameters and use simulation and filter visualizations to examine its behavior. Bandwidth tests on Weather find that matching bandwidth to lookback length is sufficient and lowers complexity.
C.3 Implementation Details
- FilterNet tunes FFN hidden size over {64, 128, 256, 512}, batch size over {4, 8, 16, 32}, and learning rate over six listed values.Filter bandwidth is set to the lookback window length; RevIN is used as the instance normalization block.Source: §C.3 · p. 16
C.4 Experimental Settings for Simulation Experiments
- The simulation compares iTransformer and FilterNet forecasts against ground truth on a signal containing low-, middle-, and high-frequency components.Source: §C.4 · p. 16
C.5 Experimental Settings for Filters Analysis
- Filter analysis applies PaiFilter to noisy trend and multi-periodic signals, while filter visualizations Fourier-transform filter values to show frequency and amplitude.The PaiFilter examples use lookback and prediction lengths of 96.Source: §C.5 · p. 16
D Study of the Bandwidth of Frequency Filters
- Bandwidth equal to lookback length is sufficient and lowers complexity; performance changes little with bandwidth in two settings and is optimal at that match in a third.The Weather experiments test three lookback-prediction combinations and bandwidths from 96 to 512, as well as 386.Source: D Study of the Bandwidth of Frequency… · p. 16
E Ablation Study
Ablations assess the FilterNet components on ETTm1, ETTh1, and Electricity, while a separate Weather figure compares filters under different bandwidths.
E Ablation Study
- The ablation experiments evaluate the impact of removing individual FilterNet components on ETTm1, ETTh1, and Electricity.Source: E Ablation Study · pp. 16–17
- Figure 9 reports MSE and MAE for filters under different bandwidths on Weather.Source: Figure 9 · p. 17
- Removing any FilterNet component causes a noticeable performance decrease on ETTm1, ETTh1, and Electricity.The ablations remove instance normalization and its inverse, the filter block, or the feed-forward network; experiments use lookback and output lengths of 96.Source: E Ablation Study · pp. 16–17, Figure 10 · p. 17
F Additional Results
Additional results report strong benchmark comparisons and visual analyses of channel-shared filters, frequency filters, and predictions.
F Additional Results
- FilterNet consistently outperforms the compared methods, and achieves the best results across five datasets with lookback length 336.Table 4 compares methods; Table 5 covers ETTh1, ETTm1, Exchange, Weather, and Electricity at prediction lengths 96, 192, 336, and 720.Source: F Additional Results · p. 17, Table 5 · p. 18, Table 4 · p. 18
G.1 Visualization of Channel-shared vs Channel-unique Filters
- Channel-shared filter predictions align more closely with ground truth than channel-unique predictions, consistent with Table 2.Source: Figure 11 · p. 19, §G.1 · p. 19
G.2 Visualization of Frequency Filters
- On Electricity, learned filters show values across the full spectrum, with frequencies concentrated mainly in low and middle ranges.The visualizations span lookback length 336 and prediction lengths 96, 192, and 336; the authors connect this frequency concentration with good performance by some low-pass-filter methods.Source: Figure 12 · p. 19, §G.2 · p. 19
G.3 Visualizations of Predictions
An ETTm1 prediction showcase compares FilterNet with three state-of-the-art models at lookback and prediction lengths of 96.
G.3 Visualizations of Predictions
- The ETTm1 showcase compares FilterNet with iTransformer [17], PatchTST [16], and DLinear.Source: §G.3 · p. 19
- FilterNet delivers the most accurate predictions of future series variations compared with iTransformer, PatchTST, and DLinear.Source: §G.3 · p. 19, Figure 13 · p. 20
- The displayed forecasting results use lookback length 96 and prediction length 96.Source: Figure 13 · p. 20