Source-linked AI summary
Spatio-Temporal Self-Supervised Learning for Traffic Flow Prediction
Jiahao Ji, Jingyuan Wang, Chao Huang, Junjie Wu, Boren Xu, Zhenhe Wu, Junbo Zhang, Yu Zheng
TL;DR
Existing traffic prediction methods inadequately model spatially skewed regional distributions and time-varying temporal patterns. ST-SSL addresses these gaps with integrated spatial-temporal convolutions, adaptive graph augmentation, and auxiliary self-supervised tasks, consistently outperforming baselines on four datasets.
Problem
Traffic prediction methods commonly overlook spatial heterogeneity across regions and model temporal correlations in a shared parameter space across time periods.
Method
ST-SSL integrates temporal and spatial convolutions with adaptive attribute- and structure-level graph augmentation plus spatial clustering and temporal contrastive self-supervision.
Results
ST-SSL consistently outperforms various baselines across four real-world datasets, significantly outperforming competing baselines on both metrics over all datasets.
Takeaways & Limitations
The framework models spatial and temporal heterogeneity jointly and may inform other practical spatiotemporal applications.
Takeaways & Limitations
Methods relying on handcrafted region spatial characteristics have limited representation generalization ability.
Abstract
from arXiv · showhide
Robust prediction of citywide traffic flows at different time periods plays a crucial role in intelligent transportation systems. While previous work has made great efforts to model spatio-temporal correlations, existing methods still suffer from two key limitations: i) Most models collectively predict all regions' flows without accounting for spatial heterogeneity, i.e., different regions may have skewed traffic flow distributions. ii) These models fail to capture the temporal heterogeneity induced by time-varying traffic patterns, as they typically model temporal correlations with a shared parameterized space for all time periods. To tackle these challenges, we propose a novel Spatio-Temporal Self-Supervised Learning (ST-SSL) traffic prediction framework which enhances the traffic pattern representations to be reflective of both spatial and temporal heterogeneity, with auxiliary self-supervised learning paradigms. Specifically, our ST-SSL is built over an integrated module with temporal and spatial convolutions for encoding the information across space and time. To achieve the adaptive spatio-temporal self-supervised learning, our ST-SSL first performs the adaptive augmentation over the traffic flow graph data at both attribute- and structure-levels. On top of the augmented traffic graph, two SSL auxiliary tasks are constructed to supplement the main traffic prediction task with spatial and temporal heterogeneity-aware augmentation. Experiments on four benchmark datasets demonstrate that ST-SSL consistently outperforms various state-of-the-art baselines. Since spatio-temporal heterogeneity widely exists in practical datasets, the proposed framework may also cast light on other spatial-temporal applications. Model implementation is available at https://github.com/Echo-Ji/ST-SSL.
1 Introduction
The paper motivates traffic-flow prediction by identifying spatial and temporal heterogeneity that existing models often fail to represent, then proposes ST-SSL to address both through adaptive augmentation and auxiliary self-supervision.
- Accurate traffic-flow prediction across regions and time periods supports timely traffic control and may mitigate harms from sudden traffic-flow spikes.
- Existing models often overlook spatial heterogeneity, where regions have skewed and substantially different traffic-flow distributions.This can bias models toward high-volume regions and weaken citywide traffic-pattern representations.
- Handcrafted region characteristics can limit the representation generalization of methods designed to model differing regional traffic distributions.
- Shared temporal parameter spaces struggle to preserve time-varying traffic heterogeneity, including differences between workday and holiday evening patterns.
- ST-SSL integrates temporal and spatial convolutions with adaptive attribute- and structure-level graph augmentation and spatial and temporal self-supervised tasks.The auxiliary tasks supplement traffic prediction with awareness of heterogeneous regional distributions and temporal dynamics.
2 Preliminaries
The preliminaries represent a city as disjoint spatial-region nodes connected by adjacency edges, with historical inflow and outflow tensors used to predict the next time step for every region.
- A city is partitioned into N = I × J disjoint geographical grids, each treated as a spatial region r_n.
- The traffic flow graph G = (V, E, A, X_t−T:t) contains spatial-region nodes, edges between adjacent regions, an adjacency matrix, and historical traffic data.
- Historical traffic tensors contain citywide inflow and outflow over the previous T time steps, while X_t records all regions at time t.
- Given the historical traffic flow graph, the task is to estimate X_t+1, the traffic volume of all regions at the future time step.
3 Methodology
ST-SSL combines temporal and spatial graph convolutions with adaptive graph augmentation and auxiliary self-supervised tasks to model spatial and temporal traffic heterogeneity.
- Spatio-Temporal Encoder: The spatio-temporal encoder stacks temporal convolution, spatial convolution, and temporal convolution blocks to aggregate sequential and geographical traffic patterns.It produces a final embedding for each region after temporal dimensions are reduced through repeated blocks.
- Spatio-Temporal Encoder: Temporal convolution uses causal one-dimensional convolutions to transform traffic tensors into time-aware embeddings for each region.The output is a sequence of region embedding matrices across time.
- Spatio-Temporal Encoder: Spatial convolution applies graph-based message passing with the region adjacency matrix to inject geographical context into region embeddings.The refined embeddings capture region-wise spatial correlations.
- Adaptive Graph Augmentation: ST-SSL adaptively augments the traffic graph through traffic-level masking and topology-level structure modification based on learned traffic dependencies.The resulting graph combines debiased traffic inputs with structure denoising and can capture long-range region dependencies.
- Spatial Heterogeneity Modeling: A soft clustering self-supervised task assigns regions to latent factors representing diverse urban functionalities and trains original embeddings to predict augmented assignments.Distribution regularization constrains assignments and uses maximum entropy to avoid identical assignments across regions.
- Temporal Heterogeneity Modeling: A temporal heterogeneity task fuses original and augmented time-aware embeddings and contrasts same-time positive pairs against different-time negative pairs.This design encourages consistency within time-specific citywide trends while distinguishing representations across time steps.
4 Experiments
The experiments evaluate ST-SSL across real-world traffic datasets to assess overall prediction, component contributions, heterogeneity handling, and representation benefits.
- Experiments address four questions covering overall performance, sub-module contributions, heterogeneous regions and periods, and augmented-graph representation benefits.
4.1 Experimental Settings
ST-SSL is evaluated on four public traffic datasets spanning bike rentals and taxi trajectories, using historical-flow inputs, standard metrics, multiple baselines, and fixed implementation settings.
- Datasets: Four public datasets cover New York City bike rentals and taxi trajectories from New York City and Beijing.The datasets use 30-minute or hourly intervals and span different collection periods.
- Data preparation: Samples use previous 2-hour and 3-day flows to predict the next time step, with a 7:1:2 train-validation-test split.
- Metrics and baselines: Evaluation uses Mean Average Error (MAE) and Mean Average Percentage Error (MAPE), comparing ST-SSL with eight baselines across three categories.
- Implementation: The implementation uses PyTorch with embedding dimension D=64, convolution kernels of size 3, perturbation ratios of 0.1, Adam, and batch size 32.
4.2 Performance Comparison (RQ1)
ST-SSL is compared with baseline methods on four datasets using MAE and MAPE, with results averaged across five random seeds and tested for significance.
- ST-SSL significantly outperforms competing baselines on both MAE and MAPE across all datasets at Student’s t-test level 0.01.The comparison reports average performance and standard deviations for deep learning models.
- ST-SSL’s prediction errors are lower than those of the two best-performing baselines on the BJTaxi visualization.The figure uses brighter pixels for larger relative errors and shows notable improvement in suburban areas.
- Spatiotemporal methods outperform time-series approaches in most cases, while heterogeneity-aware methods usually outperform shared-parameter methods.
4.3 Ablation Study (RQ2)
Ablation studies test adaptive augmentation and separate spatial and temporal heterogeneity modeling, showing that each designed component contributes positively to performance.
- Adaptive augmentation: ST-SSL outperforms variants using random traffic-level or graph-structure augmentation, supporting adaptive heterogeneity-guided augmentation.
- Heterogeneity modeling: ST-SSL consistently outperforms variants that disable spatial or temporal heterogeneity modeling, supporting their joint inclusion.
- Overall ablation result: Each designed sub-module has a positive effect on performance improvement.
4.4 Robustness Analysis (RQ3)
ST-SSL remains robust across spatial regions and time-period categories with heterogeneous traffic patterns, outperforming baselines especially for less popular regions and sparse nighttime periods.
- Evaluation setup: The spatial robustness analysis evaluates prediction performance across region clusters formed from historical traffic-flow statistics.The cluster analysis uses each region’s traffic-flow mean, median, and standard deviation.
- Spatial heterogeneity: ST-SSL significantly surpasses baselines for heterogeneous spatial clusters, particularly less popular regions with lower traffic volumes.Regions were clustered using historical-flow mean, median, and standard deviation; smaller cluster ids generally correspond to suburban areas.
- Temporal heterogeneity: ST-SSL beats baselines in every temporal category and improves most in nighttime workday and holiday categories with sparse traffic data.The method injects temporal heterogeneity into time-aware region embeddings to handle these periods.
4.5 Qualitative Study (RQ4)
The qualitative study examines how ST-SSL’s adaptive graph augmentation changes region connections and how its embeddings organize heterogeneous spatial regions.
- Adaptive graph augmentation: Adaptive topology augmentation removes links between adjacent regions with heterogeneous traffic patterns and adds links between distant regions with similar urban functions.The case study contrasts a residential zone with a transportation hub and connects two transportation hubs despite their distance.
- Adaptive graph augmentation: The augmentation debiases low-intercorrelation region connections while capturing long-range dependencies through global urban context.This mechanism is illustrated on BJTaxi in the adaptive graph-topology case study.
- Embedding visualization: ST-SSL embeddings form more compact same-class groups and better-separated different-class groups than AGCRN embeddings in the BJTaxi t-SNE visualization.The visualization uses ground-truth region classes matching those used in the spatial-cluster analysis.
5 Related Work
Related work applies neural networks and self-supervised learning to traffic prediction, but the paper positions ST-SSL as addressing overlooked spatio-temporal heterogeneity through adaptive graph augmentation.
- Traffic prediction methods: Traffic prediction research uses RNNs and 1D CNNs for temporal dependencies, alongside CNNs, GNNs, and attention mechanisms for spatial information.These approaches represent major neural-network directions in prior traffic prediction work.
- Traffic prediction methods: Most existing traffic prediction methods neglect spatio-temporal heterogeneity in their modeling.This gap motivates the paper’s focus on region- and time-aware traffic representations.
- Self-supervised learning: Self-supervised learning augments input data and uses pretext tasks with pseudolabels to improve representation quality across text, images, and graphs.The paper adapts this general paradigm to spatio-temporal graph data.
6 Conclusion and Future Work
The paper concludes that ST-SSL combines spatio-temporal convolutions with spatial and temporal self-supervised signals to model traffic heterogeneity, with future work targeting model-agnostic extension.
- Conclusion: ST-SSL integrates temporal and spatial convolutions to encode spatial-temporal traffic patterns.The framework supplements the main traffic prediction task with heterogeneity-aware self-supervised signals.
- Conclusion: Its spatial self-supervised paradigm combines adaptive graph augmentation with a clustering-based generative task, while its temporal paradigm uses a time-aware contrastive task.These auxiliary tasks target spatial and temporal heterogeneity respectively.
- Conclusion: Comprehensive experiments on four traffic-flow datasets demonstrate the robustness of ST-SSL.The authors identify extending the framework to a model-agnostic paradigm as future work.