PaperlayerView exact revision on arXiv

Source-linked AI summary

SpecLST: Spectral-Enhanced Long-Short Term Spatio-Temporal Multimodal Modeling for Traffic Forecasting

Kenan Kang, Matthew M. Y. Kuo, Weihua Li

arXiv:2610.36393v1cs.LG

TL;DR

Traffic forecasting models face the combined challenge of long-range history, heterogeneous context, and transient anomalies, while raw-sequence attention is costly. SpecLST uses compact FFT history, attention-based context fusion, and a sparse MoE spatio-temporal backbone with a high-frequency residual head. Across four benchmarks, it reports competitive accuracy and favorable memory and training-time trade-offs; the paper also identifies cross-region generalization as a challenge for deep learning models.

  • Problem

    Traffic forecasting must represent long-range history and heterogeneous context while capturing abrupt effects such as incidents, challenges that existing spatio-temporal frameworks often address separately.

    Source: §I · p. 1

  • Method

    SpecLST combines compact FFT-based multi-week traffic history, attention-based multimodal fusion, a sparse MoE spatio-temporal backbone, and a high-frequency residual head.

    Source: §I · pp. 1–2

  • Results

    Across four real-world benchmarks, SpecLST achieves the best reported results across metrics and horizons on PEMS-BAY22, leads most metrics on PEMS08, and remains competitive on PEMS-BAY and PEMS04.

    Source: §V · p. 11

  • Takeaways & Limitations

    SpecLST combines competitive predictive performance with favorable memory and training-time positioning, while ablations identify the long-history FFT branch and event modality as the largest individual gains.

    Source: §V · p. 11

    The paper identifies cross-region generalization as a challenge for deep learning models because their data dependence reflects regional biases in topology, mobility patterns, and data availability.

    Source: §V · p. 12

Abstract

from arXiv · show

Modeling multi-scale dynamics in urban road networks is essential for intelligent transportation systems but is often hindered by the computational cost of long-range historical context and the challenge of fusing heterogeneous modalities. Existing spatio-temporal frameworks face a trade-off between sequence length and memory overhead, while frequently neglecting the transient effects of exogenous factors such as road incidents and weather. To address these limitations, we propose SpecLST, a frequency-enhanced long-short-term spatio-temporal multimodal framework. SpecLST leverages a fast Fourier transform-based frequency-domain representation module to efficiently encode multi-week historical dependencies, bypassing the memory bottlenecks of standard attention mechanisms over raw long sequences. Heterogeneous context is integrated through attention-based fusion, followed by a sparse Mixture-of-Experts backbone with graph augmentation for spatio-temporal representation learning. Additionally, a frequency-enhanced residual head isolates high-frequency transients, improving sensitivity to abrupt traffic anomalies. At the 60-minute horizon, SpecLST achieves mean relative error reductions of 14.10% in MAE, 10.35% in RMSE, and 15.78% in MAPE across eight neural forecasting baselines and four datasets, with the dataset- and metric-specific average reductions ranging from 5.63% to 22.21%.

I. INTRODUCTION

Traffic forecasting must accommodate long periodic histories, heterogeneous context, and abrupt incident-driven deviations, yet existing models rarely address these needs together. SpecLST combines spectral long-history encoding, multimodal fusion, and transient correction; the paper also extends PEMS-BAY22 with aligned contextual data.

I. INTRODUCTION

  • Existing models rarely combine multi-week periodic context, heterogeneous exogenous signals, and rapid response to abrupt traffic deviations.Long-history attention also faces quadratic cost, while short-term inputs are often treated as smooth signals.Source: §I · p. 1
  • SpecLST combines compact frequency-domain long-history representations, attention-based multimodal fusion, and a per-horizon high-frequency residual head.The long-history representation is aligned with recent traffic, while the residual head corrects the backbone prediction using a separate recent window.Source: §I · pp. 1–2
  • PEMS-BAY22 is extended with aligned weather, calendar, and sensor-level event annotations curated from social-media traffic reports.The event annotations use a two-stage LLM-assisted pipeline.Source: §I · p. 2
  • SpecLST maintains competitive accuracy while reducing computational cost, especially on multimodal PEMS-BAY22.The paper benchmarks four real-world PeMS datasets against classical, graph-based, spectral, and Transformer-based baselines.Source: §I · p. 2

A. Graph-based spatio-temporal models

  • D2STGNN addresses the underrepresentation of node-intrinsic temporal patterns by combining decoupled spatial-temporal modeling with dynamic graph learning.The passage reports strong performance on multiple public benchmarks.Source: A. Graph-based spatio-temporal models · p. 2
  • Graph-based spatio-temporal models advanced short-horizon accuracy, but typically use limited temporal contexts and omit heterogeneous weather, calendar, and incident signals.Graph methods combine spatial message passing with temporal sequence modeling; adaptive adjacency methods learn dependencies beyond predefined topology.Source: A. Graph-based spatio-temporal models · p. 2

B. Transformer-based traffic forecasting

Transformer traffic models improve accuracy and flexibility, but raw-sequence self-attention becomes costly for long-term or multimodal inputs. Existing frequency-domain approaches model periodicity or fluctuations, while SpecLST combines multi-week spectral history, multimodal fusion, and per-horizon transient correction.

B. Transformer-based traffic forecasting

  • Most Transformer traffic models use full self-attention over raw temporal sequences, creating substantial computational overhead for long-term or multimodal inputs.Examples include STTN, STGormer, STAEformer, LSTTN, and PDFormer, which extend spatial or temporal modeling in different ways.Source: B. Transformer-based traffic forecasting · p. 2

C. Frequency-domain modeling for time series and traffic

  • Frequency-domain methods represent periodicity and fluctuations through Fourier, FFT-based autocorrelation, or wavelet decompositions.FEDformer uses Fourier operators, Autoformer uses FFT-computed autocorrelation, and STWave separates trend and event components with wavelets.Source: C. Frequency-domain modeling for time… · pp. 2–3
  • Existing spectral traffic models rarely fuse compact multi-week traffic representations with weather, calendar, and incident signals.Their transforms generally apply to traffic-only sequences over relatively short windows, leaving interactions between stable long-term patterns and short-term root-cause factors underexplored.Source: C. Frequency-domain modeling for time… · p. 3
  • SpecLST jointly integrates multi-week frequency-domain history, attention-based multimodal fusion, and a per-horizon high-frequency residual head.Unlike STWave, FEDformer, and Autoformer as described here, it centers on compact spectral tokens from long history and lightweight residual refinement.Source: C. Frequency-domain modeling for time… · p. 3

A. Problem definition and preliminary

The paper formulates traffic forecasting as predicting future sensor traffic from long- and short-term inputs and network connectivity. SpecLST embeds these inputs, fuses modalities, processes them with spatio-temporal blocks, and adds a frequency-enhanced residual correction.

A. Problem definition and preliminary

  • The traffic network is an undirected graph whose sensor nodes are connected by weighted edges encoded in an adjacency matrix.Node locations are specified by geographic coordinates.Source: A. Problem definition and preliminary · p. 3
  • Long-term traffic features provide periodic context, while short-term inputs include traffic, weather, calendar, and events when available.Weather, calendar, and event signals are treated as short-term context because their effects relate to the current forecasting horizon rather than stable multi-week periodicity.Source: A. Problem definition and preliminary · p. 3
  • Traffic forecasting predicts traffic features H time steps ahead from historical long- and short-term data and the sensor-network adjacency matrix.The output contains predictions for all N sensors; with five-minute sampling and H = 12, it covers the next 60 minutes.Source: A. Problem definition and preliminary · p. 3

B. SpecLST Framework

  • SpecLST maps long traffic history into low-frequency FFT tokens, aligns them with the short-term window, and fuses available modalities through attention.The fused representation passes through spatio-temporal blocks using alternating feed-forward and sparse MoE layers with spatial attention over fused static and learnable adjacency; a separate-window residual head adds per-horizon high-frequency corrections.Source: B. SpecLST Framework · pp. 3–4

1) Long-Short Term Input Token Embedding:

The input embedding represents long-history traffic with a compact set of low-frequency Fourier coefficients and aligns those periodic features with recent multimodal inputs. Normalization and phase rotation produce features that can join the short-term traffic, weather, calendar, and event embeddings.

1) Long-Short Term Input Token Embedding:

  • Traffic, weather, calendar, and event inputs are each projected through modality-specific linear embedding layers into the model state space.Weather and calendar embeddings are global across sensors and are broadcast across the node dimension to match the other modalities.Source: 1) Long-Short Term Input Token Embedding · p. 4
  • Direct attention over long raw sequences increases parameter and memory requirements, so SpecLST computes FFT coefficients and retains the first K low-frequency components.These components capture slowly varying daily and weekly patterns with fewer tokens than the original sequence.Source: 1) Long-Short Term Input Token Embedding · p. 4
  • The long-history input is downsampled by a factor of two in the experiments, giving Leff = 2016 from L = 4032 two-week samples.The DC component is excluded because it encodes sensor-wise mean speed rather than periodic structure; residual connections implicitly recover this constant offset.Source: 1) Long-Short Term Input Token Embedding · p. 4
  • Frequency-wise amplitude normalization reduces cross-sensor scale differences while preserving relative spectral shape, and phase rotation projects harmonics onto the recent T time steps.Together these operations yield a phase-aware periodic embedding aligned for concatenation with the traffic and contextual embeddings.Source: 1) Long-Short Term Input Token Embedding · pp. 4–5
  • The real and imaginary parts of retained spectral coefficients form the FFT features, allowing later linear layers to learn combinations of both parts.This representation is mathematically equivalent to retaining amplitude-times-cosine and amplitude-times-sine terms for each harmonic.Source: 1) Long-Short Term Input Token Embedding · p. 5

2) Multimodal Feature Fusion:

SpecLST fuses temporal and low-frequency context with traffic embeddings using attention. A gate then balances the traffic and context-aware representations.

2) Multimodal Feature Fusion:

  • External temporal and low-frequency embeddings are fused with multi-head self-attention and projected to hidden dimension D as contextual features.Global embeddings are broadcast across the node dimension so the four modalities share the same shape before channel-wise concatenation and attention.Source: 2) Multimodal Feature Fusion · p. 5
  • Traffic embeddings query the context vectors in cross-attention, which serve as keys and values.A subsequent gate balances the traffic embedding against the context-aware embedding.Source: 2) Multimodal Feature Fusion · p. 5
  • The gate controls the context-aware branch’s share: G→0 recovers the traffic embedding, while G→1 delegates the representation to that branch.Its linear-layer bias starts at zero, yielding G≈0.5 at initialization so neither modality dominates initially.Source: 2) Multimodal Feature Fusion · p. 5

3) Spatio-Temporal MoE Block:

The spatio-temporal backbone combines causal temporal attention, sparse MoE layers, and spatial attention over an augmented road graph. SpecLST adds learnable graph augmentation to its prior backbone design.

3) Spatio-Temporal MoE Block:

  • Temporal attention uses a causal lower-triangular mask so predictions at each timestep depend only on that timestep and earlier data.Normalization and residual connections are used to stabilize deep representation learning.Source: 3) Spatio-Temporal MoE Block · p. 5
  • The multimodal fusion module and alternating FFN/MoE arrangement follow prior work; the FFT long-history branch, learnable sparse graph augmentation, and gated frequency-enhanced residual head are newly introduced.Experiments use 12 experts, top-k routing with k = 2, capacity factor 1.25, and λaux = 0.01.Source: 3) Spatio-Temporal MoE Block · pp. 5–6
  • The spatial attention mask fuses the static road network with a learned adjacency, extending spatial modeling from a static topology to a learnable one.The static adjacency is built from sensor distances using a thresholded Gaussian kernel.Source: 3) Spatio-Temporal MoE Block · p. 5
  • SpecLST adds learnable sparse graph augmentation to the static topology; its augmentation is constrained to non-edges and retains the top 8 candidate edges per node.The non-edge constraint preserves the physical connectivity prior while assigning the learned component the role of discovering non-local correlations.Source: 3) Spatio-Temporal MoE Block · pp. 5–6
  • Sparse MoE layers route each token to selected experts, supporting regime-specific specialization while keeping per-sample FLOPs bounded.Standard FFN and sparse MoE layers alternate, and an auxiliary load-balancing loss encourages balanced expert use and reduces expert-collapse risk.Source: 3) Spatio-Temporal MoE Block · p. 6

4) Frequency Enhanced Residual Prediction Head:

The frequency-enhanced residual head (FRH) uses a separate recent traffic window to estimate high-frequency activity and make gated, horizon-specific corrections. It is designed as a lightweight adjustment that leaves the spatio-temporal backbone unchanged.

4) Frequency Enhanced Residual Prediction Head:

  • FRH takes a separate recent traffic window, extracts its power spectrum, and keeps the upper third of the one-sided FFT bins.The window covers only recent traffic and is distinct from the short-term multimodal input window.Source: 4) Frequency Enhanced Residual… · p. 6
  • The power spectrum captures recent high-frequency energy without phase, since short-term traffic embeddings already capture event timing.The long-history branch instead uses complex coefficients whose phase carries timing information for alignment with the recent window.Source: 4) Frequency Enhanced Residual… · p. 6
  • A learnable horizon-specific gate modulates the residual correction before it is added to the backbone prediction at each prediction horizon.The gate is intended to apply adjustments only when high-frequency evidence is reliable.Source: 4) Frequency Enhanced Residual… · p. 6
  • FRH adds a gated high-frequency residual for each sample, node, and prediction horizon using only a recent traffic window, at low computational cost.The adjustment does not alter the backbone’s spatio-temporal structure or increase its representational burden.Source: 4) Frequency Enhanced Residual… · p. 6

IV. EXPERIMENTS

The experiments evaluate SpecLST on four real-world traffic datasets, with dataset splits kept consistent with prior work. The datasets cover multiple California regions, sensor counts, traffic features, and collection periods.

IV. EXPERIMENTS

  • The experimental section reports dataset and implementation setup, baseline comparisons, parameter sensitivity analyses, and ablation studies.The supplied section overview does not report those comparison or analysis outcomes.Source: §IV · p. 7

1) Traffic data:

  • All four public datasets come from Caltrans PeMS and use a 5-minute recording interval.The datasets are PEMS-BAY, PEMS-BAY22, PEMS04, and PEMS08.Source: 1) Traffic data · p. 7
  • The evaluation spans 170–325 sensors and varying traffic features across Bay Area and California District 04 and 08 datasets.PEMS-BAY uses a 7:2:1 train/validation/test split; PEMS04, PEMS08, and PEMS-BAY22 use 6:2:2, consistent with prior work.Source: 1) Traffic data · p. 7

2) Context data:

SpecLST combines multimodal context with long- and short-term spectral features for traffic forecasting, evaluated against classical, graph-based, spectral, and Transformer baselines. Its setup uses two weeks of traffic history and a short multimodal window, with dataset-specific spectral thresholds.

2) Context data:

  • Weather, calendar, and incident context supplement traffic inputs; incidents are filtered, classified by severity, and aligned to sensors within a 1.0 km radius and 5-minute window.The event pipeline favors precision over recall, and event data is used only for PEMS-BAY22 because manual annotation is labor-intensive.Source: 2) Context data · p. 7

B. Baseline Models

  • The evaluation compares SpecLST with statistical, graph-based recurrent, and Transformer-based traffic forecasting baselines.The baseline set includes VAR, SVR, DCRNN, MTGNN, D2STGNN, STWave, STGormer, LSTTN, STTN, and STAEformer.Source: B. Baseline Models · p. 7

C. Experimental Environment and Hyperparameter Settings

  • The setup uses two weeks of traffic history, a 1-hour multimodal input, and dataset-specific spectral cutoffs selected to preserve 95% of training-set cumulative energy.The recent window for the residual head is 24 steps; its high-pass filter retains the upper one-third of one-sided FFT bins. The model uses 6 MoE blocks with 12 experts and top-k routing of 2.Source: C. Experimental Environment and… · pp. 7–8

D. Experimental Results

SpecLST is competitive across four datasets and three horizons, with a dataset-dependent comparison against LSTTN. Sensitivity and ablation analyses identify the contributions of the model’s settings and components.

E. Sensitivity Analysis

  • The sensitivity analysis retains BS = 8, D = 128, and ST = 6: larger dimensions yield metric-dependent gains, while deeper configurations give inconsistent improvements.BS = 8 improves errors consistently across horizons and metrics versus BS = 4; D = 128 is retained as a compact operating point.Source: E. Sensitivity Analysis · p. 8, Table II · p. 10

1) Prediction Accuracy:

  • SpecLST performs competitively across four datasets and three horizons, leading on PEMS-BAY22 and most metrics on PEMS08, while LSTTN is best or near-best on PEMS-BAY and PEMS04.On PEMS-BAY22, SpecLST achieves the lowest average MAE with lower resource consumption than LSTTN.Source: 1) Prediction Accuracy · p. 8, Table I · p. 9

F. Ablation Studies

  • Learned adjacency yields the best results across horizons, indicating that sparse learned structure augments the physical graph with task-dependent non-local correlations.The physical graph prior remains useful, but does not fully encode these correlations.Source: F. Ablation Studies · p. 9, Table III · p. 10
  • Removing all contextual modalities produces the clearest degradation; events have the strongest individual influence, while weather and calendar features also contribute positively.The modality effects on RMSE and MAPE do not exactly match their effects on MAE.Source: F. Ablation Studies · pp. 9–10, Table IV · p. 10
  • Removing any spectral component degrades performance, with the largest overall deterioration when the long-history FFT branch is removed; FRH and the spectral module also provide gains.The ablation summary ranks the FFT branch’s contribution highest, followed by exogenous context, especially event information; learned adjacency and FRH provide smaller but consistent gains.Source: F. Ablation Studies · p. 10

G. Efficiency

On PEMS-BAY22, SpecLST occupies the accuracy-oriented end of the efficiency frontier for both memory use and training time. The analysis compares average MAE with resource consumption, with MAE averaged across three forecasting horizons.

G. Efficiency

  • The Pareto analysis compares forecasting accuracy and resource consumption using MAE averaged over horizons H ∈{3, 6, 12}; lower-left points are preferable, and frontier models cannot be beaten on both measures.Fig. 2 distinguishes Pareto-front models from non-front models.Source: G. Efficiency · p. 10, Figure 2 · p. 11
  • SpecLST achieves the lowest average MAE on the memory frontier with moderate memory use, while STWave and DSTET occupy lower-memory frontier positions with higher error.LSTTN is not on the frontier despite competitive accuracy because it requires substantially more memory.Source: G. Efficiency · p. 11, Figure 2 · p. 11
  • SpecLST remains the accuracy-oriented point on the training-time frontier, while faster MTGNN has higher prediction error and LSTTN requires much longer training.The paper attributes LSTTN's training time to its long-context temporal module.Source: G. Efficiency · p. 11, Figure 2 · p. 11

V. CONCLUSION AND FUTURE WORK

SpecLST combines frequency-domain long-history modeling, multimodal context fusion, and high-frequency residual correction for traffic forecasting. Across four benchmarks, it performs strongly while offering favorable resource efficiency, with future work identifying limits in its graph augmentation and cross-region generalization.

V. CONCLUSION AND FUTURE WORK

  • The framework uses FFT to represent several weeks of traffic history compactly, attention-based context fusion, and high-frequency residual correction alongside spatio-temporal modeling.The frequency representation is described as reducing memory use relative to raw long-sequence self-attention.Source: §V · p. 11
  • SpecLST achieves the best results across all reported metrics and horizons on PEMS-BAY22, leads most metrics on PEMS08, and remains competitive on PEMS-BAY and PEMS04.It also achieves the lowest average MAE and favorable positions on the memory- and training-time Pareto frontiers.Source: §V · p. 11
  • The learnable graph augmentation is global and time-invariant; conditioning adjacency on the latest T steps is proposed to let spatial structure evolve with traffic.The proposed direction could capture transient correlations during peak hours or incidents.Source: §V · p. 11
  • Cross-region generalization remains difficult because deep learning models reflect regional differences in network topology, mobility patterns, and data availability.The paper suggests large language models may provide useful priors for systems adapting with limited local supervision.Source: §V · p. 12

Related papers

Loading 2610.36393v1…