PaperlayerView exact revision on arXiv

Source-linked AI summary

TimeFilter: Patch-Specific Spatial-Temporal Graph Filtration for Time Series Forecasting

Yifan Hu, Guibin Zhang, Peiyuan Liu, Disen Lan, Naiqi Li, Dawei Cheng, Tao Dai, Shu-Tao Xia, Shirui Pan

arXiv:2501.13041v2cs.LG

TL;DR

Multivariate forecasting must model both temporal dynamics and inter-channel dependencies without indiscriminately including irrelevant relationships. TimeFilter builds a spatial-temporal graph and applies patch-specific dynamic filtration, achieving state-of-the-art performance across long- and short-term forecasting benchmarks.

  • Problem

    CI overlooks useful inter-channel relationships, CD models dependencies indiscriminately, and coarse clustering cannot capture complex time-varying interactions.

  • Method

    TimeFilter segments inputs into patches, constructs a spatial-temporal graph, dynamically filters redundant dependencies with an MoE-style router, and learns from the retained graph.

  • Results

    TimeFilter achieves state-of-the-art performance in long- and short-term forecasting, with reported error reductions of 4.48% and 5.34%, respectively.

  • Takeaways & Limitations

    Patch-specific filtration consistently models important dependencies while removing redundant relationships across long- and short-term forecasting tasks.

Abstract

from arXiv · show

Time series forecasting methods generally fall into two main categories: Channel Independent (CI) and Channel Dependent (CD) strategies. While CI overlooks important covariate relationships, CD captures all dependencies without distinction, introducing noise and reducing generalization. Recent advances in Channel Clustering (CC) aim to refine dependency modeling by grouping channels with similar characteristics and applying tailored modeling techniques. However, coarse-grained clustering struggles to capture complex, time-varying interactions effectively. To address these challenges, we propose TimeFilter, a GNN-based framework for adaptive and fine-grained dependency modeling. After constructing the graph from the input sequence, TimeFilter refines the learned spatial-temporal dependencies by filtering out irrelevant correlations while preserving the most critical ones in a patch-specific manner. Extensive experiments on 13 real-world datasets from diverse application domains demonstrate the state-of-the-art performance of TimeFilter. The code is available at https://github.com/TROUBADOUR000/TimeFilter.

1. Introduction

Multivariate forecasting must capture temporal dynamics and inter-channel dependencies, but CI, CD, and coarse channel clustering each miss some time-varying relationships. TimeFilter addresses this with patch-specific dependency filtration and achieves state-of-the-art forecasting performance.

  • Motivation: CI ignores relevant covariate relationships, whereas CD models all dependencies, including irrelevant ones.These limitations vary across application domains and channel relationships.
  • Motivation: Existing channel-clustering methods use global similarity and cannot flexibly model dependencies that evolve across time intervals.Channels related within a cluster may become independent at particular periods, making coarse modeling problematic.
  • Approach: TimeFilter partitions channels into non-overlapping patches to represent temporal, spatial, and spatial-temporal dependencies at finer granularity.The patch-wise formulation explicitly distinguishes dependencies within channels, across channels at the same time, and across channels and periods.
  • Approach: TimeFilter constructs a spatial-temporal graph, dynamically filters redundant dependencies with an MoE-style router, and aggregates relevant information for forecasting.Its modules perform graph construction, patch-specific filtration, and adaptive graph learning.
  • Results: 4.48% and 5.34% error reductions are reported for long- and short-term forecasting, respectively.The introduction reports state-of-the-art performance in both settings.

2. Related Work

Related forecasting methods use recurrent, convolutional, or attention-based architectures, while dependency modeling commonly follows channel-independent or channel-dependent strategies. CI preserves only temporal information and may waste useful inter-channel relationships, whereas CD models broader dependencies.

  • Forecasting Architectures: RNN, CNN, and Transformer models capture sequential, local temporal, and long-term dependencies through different architectural mechanisms.RNNs use the Markov assumption, CNNs convolve along time, and Transformers use attention.
  • Dependency Modeling: CI methods model each channel separately and preserve only temporal dependencies.This approach uses each sequence's historical information without modeling variable interactions.
  • Dependency Modeling: CI can waste relevant inter-channel information, while CD models the full set of dependencies between variables.The related-work discussion contrasts the robustness of CI with its omission of potentially useful interactions.

3. Preliminaries

The forecasting problem represents multivariate input channels over a look-back horizon and predicts future sequences. TimeFilter models these data with a spatial-temporal graph whose nodes correspond to channel patches and whose edges encode their dependencies.

  • Problem Definition: The input is X = {x1, x2, ..., xC} ∈ R^C×L, and the goal is to predict future sequences Y ∈ R^C×T.C is the channel count, L is the look-back horizon, and T is the forecasting horizon.
  • Dependency Matrix: The dependency matrix M is the adjacency matrix of a spatial-temporal graph with n = C × N nodes.N denotes the number of patches in the look-back sequence, and M[i, j] gives an edge weight.
  • Dependency Matrix: Each patch-specific ego graph is divided into spatial, temporal, and spatial-temporal subgraphs.Masks applied to M obtain the adjacency matrix for each dependency type.

4. Method

TimeFilter builds a patch-level spatial-temporal graph, decomposes it into ego graphs, and dynamically filters dependencies before adaptive graph learning produces forecasts.

  • Spatial-Temporal Construction: The framework segments each channel into non-overlapping patches, embeds them as n = C × N patch nodes, and constructs a spatial-temporal graph.Patches are mapped from length P to hidden dimension D before graph construction.
  • Spatial-Temporal Construction: Multi-head projection distances and k-NN retain the nearest patch neighbors, while the global graph is decomposed into patch-specific ego graphs.Each node retains k = ⌊α ∗n⌋ nearest neighbors, with α hand-tuned.
  • Spatial-Temporal Construction: Each ego graph separates temporal, spatial, and spatial-temporal regions through masked adjacency matrices representing different dependency types.These regions correspond to within-channel temporal links, same-time inter-channel links, and cross-channel cross-time links.
  • Patch-Specific Filtration: A routing network assigns Temporal, Spatial, and Spatial-temporal filters to patches, retaining only the selected dependency edges during filtration.The three experts target different dependency regions and can be applied independently across ego graphs.
  • Patch-Specific Filtration: Dynamic Expert Allocation selects a variable number of filters according to patch confidence rather than imposing the fixed Top-K allocation used by classical MoE routing.The selected set grows when the current dependencies are judged insufficient, using a cumulative Top-p threshold.
  • Adaptive Graph Learning: The filtered ego graphs are restored into a global graph, whose neighborhood information is aggregated by adaptive graph learning before residual and linear forecasting projection.TimeFilter uses additive neighborhood aggregation, followed by a residual module and a linear projection in the time dimension.

5. Experiments

TimeFilter is evaluated on long- and short-term forecasting across diverse datasets against representative baselines, alongside ablations, dependency analyses, horizon studies, and efficiency comparisons. It consistently achieves strong forecasting performance while using patch-specific filtering to model relevant dependencies.

  • Experimental Setup: Experiments cover nine long-term and four short-term real-world forecasting datasets, compared with representative GNN, Transformer, linear, and CNN baselines.The long-term benchmarks include ETT, Traffic, Electricity, Weather, Solar-Energy, and Climate; short-term benchmarks are four PEMS datasets.
  • Long-term Forecasting: 4.48%/2.23% lower MSE/MAE than Leddam establishes TimeFilter’s long-term forecasting advantage across four horizons.The comparison uses input length L=96 and horizons T ∈ {96, 192, 336, 720}.
  • Long-term Forecasting: 5.34%/1.40% and 6.89%/3.69% lower MSE/MAE than DUET and CCM, respectively, show robustness with longer optimal look-back horizons.TimeFilter is reported to resist input noise while maintaining state-of-the-art performance.
  • Short-term Forecasting: 13.54%/3.13% lower MSE/MAE than DUET on PEMS08 and 5.33%/1.73% lower MSE/MAE than SOFTS on PEMS07 demonstrate short-term gains.TimeFilter consistently outperforms other methods across all four PEMS datasets.
  • Ablation Study: TimeFilter consistently outperforms alternative filtering methods, supporting dynamic selection of dependencies rather than unfiltered or fixed filtering.The study compares Top-K, Random-K, region-based, and other filtering strategies; the authors associate higher-weight relationships with possible spurious regression.
  • Model Analysis: Figure 4 shows dataset-specific dependency types and selected-filter distributions, while Figure 5 evaluates performance across look-back horizons and Figure 6 compares efficiency.TimeFilter is reported to exploit vital information in longer sequences and exceed other model families in efficiency.

6. Conclusion

TimeFilter adaptively models complex dependencies in multivariate time series by filtering redundant relationships in a patch-specific manner. Experiments show consistent state-of-the-art performance across long- and short-term forecasting.

  • TimeFilter adaptively, finely, and robustly models dependencies in complex real-world multivariate time series.
  • Patch-specific filtration removes redundant relationships while exploiting the most important dependencies for accurate forecasting.
  • TimeFilter consistently achieves state-of-the-art performance in both long- and short-term forecasting tasks.
  • The work provides an in-depth exploration of dependency modeling in time series forecasting and may support further research into dependency representations.

Impact Statement

Time series forecasting supports applications including finance, weather forecasting, and traffic control. The study presents a novel dependency-modeling method using publicly accessible datasets and reports no anticipated ethical issues.

  • Time series forecasting plays a crucial role in finance, weather forecasting, and traffic control.
  • The study presents a novel method for enhancing dependency modeling in time series forecasting.
  • The datasets used are publicly accessible, promoting transparency and reproducibility.
  • The authors do not anticipate ethical issues arising from the work.

A.1. Datasets

The experiments use eight widely used long-term forecasting datasets and PEMS traffic datasets for short-term forecasting. The appendix describes dataset sources, sampling frequencies, splits, prediction lengths, and evaluation metrics.

  • The experiments use eight widely used time series datasets for long-term forecasting and PEMS datasets for short-term forecasting.
  • ETT contains transformer temperature and power-load data from two Chinese regions, with hourly and 15-minute granularities.
  • Weather records 21 meteorological indicators in Germany at 10-minute intervals throughout 2020.
  • Electricity contains hourly consumption records for 321 clients from 2012 to 2014.
  • Traffic and PEMS datasets represent road occupancy and traffic networks, with PEMS data aggregated into 5-minute intervals.
  • Table 5 defines dataset size as train, validation, and test time points, prediction length as future points forecast, and frequency as sampling interval.

A.3. Experiment details

Experiments are implemented in PyTorch on an NVIDIA A100 GPU with Adam optimization. Batch sizes vary by dataset, and Table 6 reports dataset-specific hyperparameters.

  • Experiments use PyTorch, an NVIDIA A100 40GB GPU, and the Adam optimizer.
  • Batch size is 16 for Electricity and Traffic and 32 for all other datasets.
  • Table 6 reports dataset-specific hyperparameters, including graph-block count, learning rate, and attention-layer dimensions.

B.1. Error Bars

TimeFilter shows consistent performance across forecasting settings, with reported comparisons against Leddam indicating statistically significant improvement. It achieves the best performance across both long-term and short-term forecasting evaluations.

  • Statistical significance: 4.66e−10 is the Wilcoxon-test p-value versus Leddam, indicating significant improvement at the 99% confidence level.The experiments were repeated three times, with standard deviations reported for TimeFilter and Leddam.
  • Long-term forecasting: TimeFilter consistently achieves the best performance in long-term forecasting with both fixed and searched look-back horizons.The searched horizons are L ∈{192, 336, 512, 720}, while the fixed setting uses L = 96.
  • Short-term forecasting: Across all four PEMS datasets, TimeFilter delivers the highest short-term forecasting performance for T ∈{12, 24, 48}.The input look-back horizon is fixed at L = 96, with particularly significant improvement reported on PEMS08.

B.4. Ablation Study

The supplementary study compares six filtering strategies and evaluates predictions across four representative datasets. TimeFilter is reported as producing the most accurate future-series predictions in the visual comparisons.

  • Filtering alternatives: Six alternatives test top-weight, random, region-based, channel-wise, and no-filtering strategies against TimeFilter.The alternatives include Top-K, Random-K, RegionTop-K, RegionThre, C-Filter, and w/o Filter.
  • Filtering alternatives: Top-K and Random-K select K edges per patch from its ego graph, using largest weights or random selection, respectively.These strategies provide weight-based and randomized patch-level filtering comparisons.
  • Filtering alternatives: RegionTop-K and RegionThre apply region-specific filtering through top-weight selection or learned thresholds.RegionTop-K operates within regions, while RegionThre learns a threshold for each region through a linear mapping.
  • Filtering alternatives: C-Filter applies the same dependency-type filtering across all patches, while w/o Filter applies no filtering after graph construction.These settings contrast channel-wise filtering with removing the filtering stage entirely.
  • Prediction examples: TimeFilter delivers the most accurate future-series predictions in visual comparisons on Electricity, Traffic, Weather, and PEMS08.The comparisons use L = 96; T = 96 for Electricity, Traffic, and Weather, and T = 48 for PEMS08.
  • Evaluation presentation: The long-term and short-term supplementary tables report best and second-best results using highlighted and underlined entries.Long-term results cover fixed and searched input lengths, while short-term results average across T ∈{12, 24, 48}.
Loading 2501.13041v2…