Source-linked AI summary
A Hybrid Method for Traffic Flow Forecasting Using Multimodal Deep Learning
Shengdong Du, Tianrui Li, Xun Gong, Shi-Jinn Horng
TL;DR
Short-term traffic flow forecasting must handle nonlinear, non-stationary traffic conditions and interdependent information across modalities. The paper proposes HMDLF, combining CNN, GRU, attention, and multimodal feature fusion to learn local and long-range patterns. Experiments on real traffic data report effective forecasting across normal, weekend, and anomalous conditions, while limited data on accidents and extreme weather constrain further development.
Problem
Traffic flow forecasting is difficult because nonlinear, non-stationary conditions and dependencies among flow, speed, journey time, weather, and incidents must be modeled together.
Method
HMDLF uses CNN-GRU-Attention modules to learn local spatial-temporal features and long dependencies, then fuses shared representations from multiple traffic modalities.
Results
Experiments on real traffic data verified the model’s effectiveness across weekdays, weekends, normal conditions, and anomalous conditions.
Takeaways & Limitations
Multimodal feature fusion is the reported source of improvement, capturing relationships among traffic flow, speed, events, weather, and related variables.
Takeaways & Limitations
Current experiments mainly use flow, speed, and pass-time data because accidents and extreme-weather data are difficult to collect, requiring further study with more datasets.
Abstract
from arXiv · showhide
Traffic flow forecasting has been regarded as a key problem of intelligent transport systems. In this work, we propose a hybrid multimodal deep learning method for short-term traffic flow forecasting, which can jointly and adaptively learn the spatial-temporal correlation features and long temporal interdependence of multi-modality traffic data by an attention auxiliary multimodal deep learning architecture. According to the highly nonlinear characteristics of multi-modality traffic data, the base module of our method consists of one-dimensional Convolutional Neural Networks (1D CNN) and Gated Recurrent Units (GRU) with the attention mechanism. The former is to capture the local trend features and the latter is to capture the long temporal dependencies. Then, we design a hybrid multimodal deep learning framework (HMDLF) for fusing share representation features of different modality traffic data by multiple CNN-GRU-Attention modules. The experimental results indicate that the proposed multimodal deep learning model is capable of dealing with complex nonlinear urban traffic flow forecasting with satisfying accuracy and effectiveness.
1. Introduction
Urban traffic flow forecasting is important for intelligent transport management but difficult because traffic data are nonlinear, non-stationary, and multimodal. The paper proposes an attention-supported CNN-GRU multimodal architecture and reports effective forecasting on real traffic datasets.
- Motivation: Urban traffic forecasting supports congestion diagnosis, bottleneck identification, and intelligent transport decision-making amid rising costs from traffic jams.Traffic jams are associated with transportation inefficiency, time loss, fuel waste, and air pollution.
- Motivation: Traditional mathematical, simulation, and shallow-learning methods have difficulty representing the many factors affecting dynamic transportation networks.The paper identifies people, weather, accidents, and other factors as difficult to model accurately with conventional approaches.
- Motivation: Multimodal traffic forecasting combines heterogeneous sensor and traffic data, while deep learning can automatically extract relevant features across multiple levels.The related discussion motivates data-driven learning as traffic sensing produces increasingly diverse data.
- Research gap: Only limited prior work had combined multimodal deep learning with attention mechanisms for traffic sequence analysis, motivating this paper’s framework.The paper presents this combination as an underexplored direction for modeling traffic flow evolution.
- Approach: The proposed CNN-GRU attention module learns local spatial-temporal correlations and long temporal dependencies, while multiple modules fuse shared representations across modalities.The architecture targets improved multilevel feature learning, robustness, and flexibility in traffic flow forecasting.
- Results: Experiments on real traffic flow datasets report good forecasting performance and generalization, with the CNN-GRU-Attention model outperforming typical shallow-learning and baseline deep-learning models.The reported evaluation establishes effectiveness relative to the stated comparison models.
2. Related Work
Traffic forecasting research has progressed from statistical and shallow-learning methods toward deep learning and multimodal fusion. The paper positions its method as an alternative that models nonlinear temporal and multimodal traffic effects through deep fusion learning.
- Traditional methods: Earlier traffic forecasting approaches included ARIMA, SVR, and other methods for modeling vehicular flow and congestion.These methods represent the established forecasting literature discussed by the paper.
- Deep learning: Deep learning has been applied to traffic pattern recognition, flow forecasting, human mobility analysis, and transportation modeling from heterogeneous data.The cited work includes stacked autoencoders and other deep-learning frameworks.
- Attention mechanisms: Attention mechanisms became an active research topic in sequence models, including encoder-decoder systems and speech recognition.The related work uses these developments as context for attention-based traffic modeling.
- Multimodal learning: Multimodal deep learning combines complementary models such as LSTM and CNN and has often improved prediction performance in computer-vision applications.Examples include image captioning, image classification, and multimodal representation learning.
- Positioning: The proposed method extends this direction to traffic by fusing multimodality data while learning nonlinear spatial-temporal effects, local trends, and long dependencies.The paper presents this as an alternative to prior traffic forecasting and shallow-learning methods.
3. Methodology
The methodology addresses two forecasting challenges: learning local and long-range spatial-temporal patterns within traffic sequences and modeling interdependence across traffic modalities. HMDLF combines CNN-GRU-Attention modules to extract, attend to, and fuse these representations for prediction.
- 3.1 Problem definition and motivation: Traffic forecasting must model both local trends and long temporal dependencies because nearby, periodic, and distant observations are correlated.The paper identifies these correlations as a key problem for single-modality traffic data.
- 3.1 Problem definition and motivation: Multimodal traffic forecasting is difficult because flow, speed, journey time, weather, and incidents are interdependent and highly nonlinear under changing conditions.The challenge is especially pronounced during peak hours, incidents, and other non-free-flow situations.
- 3.2 Overview of the multimodal deep learning framework: HMDLF fuses multimodal traffic representations through multiple CNN-GRU-Attention modules followed by a joint model and regression layer for final prediction.The framework combines shared representation features from different modalities before regression.
- 3.4 CNN-GRU module (with attention mechanism) for long temporal dependencies and spatial-temporal correlation features learning: The 1D CNN learns local sequence trends, while GRU models learn long-dependency temporal features and attention selects important spatial-temporal context.The framework therefore assigns distinct roles to convolution, recurrent, and attention components.
- 3.4 CNN-GRU module (with attention mechanism) for long temporal dependencies and spatial-temporal correlation features learning: For each modality, CNN and GRU transform the input into spatial and temporal features, while attention-assisted fusion produces a shared representation for multimodal learning.The paper represents this processing as C(I_i) → S_i, G(S_i) → S_iT_i, and multimodal attention fusion → R_i.
4. Experiments
Experiments on real UK traffic data compare HMDLF with shallow and deep-learning baselines across datasets, lookup sizes, epochs, and traffic conditions. HMDLF, especially its CNN-GRU-Attention module, achieves the strongest reported forecasting performance and handles peaks, troughs, weekdays, weekends, and anomalous conditions.
- Dataset: The experiments use real UK traffic datasets containing location, date, time period, speed, flow, and journey-time attributes.The Highways England dataset includes 15-minute observations, 34,876 training records from 2013, and 2,688 testing records from February 2014.
- Baselines: HMDLF is compared with ARIMA, SVR variants, linear and tree models, recurrent models, CNN-based models, and CNN-LSTM/CNN-GRU variants.The comparisons include HMDLF modules based on CNN-LSTM, CNN-GRU, and CNN-GRU with attention.
- Results: HMDLF performs better than shallow models such as SVR and ARIMA by combining local trends, short-term variability, long-term dependencies, and multimodal interdependence.The authors report that CNN-GRU with attention provides the best performance among the HMDLF modules and baseline deep-learning methods.
- Parameter analysis: Increasing epochs improves performance initially, but RMSE is lowest around 150 epochs, stabilizes afterward, and larger training runs can overfit beyond 250 epochs.The authors also note that excessive epochs increase computational cost without improving generalization obviously.
- Results: HMDLF with the CNN-GRU-Attention module has the lowest error across different lookup sizes and achieves the best reported prediction accuracy.The reported comparison states that HMDLF reduces error to 4.35, while single-modal training does not obviously improve performance.
- Results: Across normal, congested, and accident conditions, HMDLF more accurately forecasts peak and trough traffic flows than SVR and LSTM.The comparisons cover weekdays, weekends, and long traffic-flow sequences; LSTM generally outperforms SVR but remains less accurate at peaks and troughs than HMDLF.
5 Conclusion and Future Work
HMDLF combines CNN-GRU modules with auxiliary attention to fuse multimodal traffic representations and model local trends, long dependencies, and nonlinear spatial-temporal correlations. Experiments on real traffic data support its effectiveness across normal, weekday, weekend, and anomalous conditions, while future work requires broader data and inter-node dependencies.
- 5 Conclusion and Future Work: HMDLF combines CNN and GRU modules with auxiliary attention to jointly learn local trends, long dependencies, and multimodal nonlinear correlations.Multiple CNN-GRU-Attention modules support adaptive representation and fusion of traffic speed, flow, pass time, and weather-related data.
- 5 Conclusion and Future Work: Multimodal feature fusion improves forecasting by modeling relationships among traffic flow, speed, events, and weather.The method also targets spatial-temporal interdependence across multimodality traffic data.
- 5 Conclusion and Future Work: Experiments on real traffic datasets verified effectiveness under weekday, weekend, normal, and anomalous conditions.
- 5 Conclusion and Future Work: Future research should model interdependence between adjacent network nodes and collect more data, including accidents and extreme weather events.Short-period data collection is identified as a major obstacle, and current experiments include only flow, speed, and pass time.