Source-linked AI summary
Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction
Junbo Zhang, Yu Zheng, Dekang Qi
TL;DR
Citywide crowd-flow forecasting must jointly predict regional inflows and outflows despite complex spatial, temporal, and external influences. ST-ResNet uses residual branches for temporal closeness, period, and trend, fuses them with external factors, and outperforms six baselines on Beijing and NYC flow datasets.
Problem
The paper addresses forecasting the next citywide inflow and outflow observation from historical flow data.
Method
ST-ResNet uses residual networks for temporal closeness, period, and trend, then fuses their outputs with external factors through a parametric matrix.
Results
ST-ResNet outperformed six baselines, reducing TaxiBJ error to 16.69 and achieving 14.8%–37.1% lower RMSE than baselines on BikeNYC.
Takeaways & Limitations
The evaluations support ST-ResNet as a more applicable model for crowd-flow prediction across the evaluated Beijing and NYC datasets.
Takeaways & Limitations
The authors plan to extend the approach to additional flow sources, including taxi, truck, bus, phone-signal, and metro-card data.
Abstract
from arXiv · showhide
Forecasting the flow of crowds is of great importance to traffic management and public safety, yet a very challenging task affected by many complex factors, such as inter-region traffic, events and weather. In this paper, we propose a deep-learning-based approach, called ST-ResNet, to collectively forecast the in-flow and out-flow of crowds in each and every region through a city. We design an end-to-end structure of ST-ResNet based on unique properties of spatio-temporal data. More specifically, we employ the framework of the residual neural networks to model the temporal closeness, period, and trend properties of the crowd traffic, respectively. For each property, we design a branch of residual convolutional units, each of which models the spatial properties of the crowd traffic. ST-ResNet learns to dynamically aggregate the output of the three residual neural networks based on data, assigning different weights to different branches and regions. The aggregation is further combined with external factors, such as weather and day of the week, to predict the final traffic of crowds in each and every region. We evaluate ST-ResNet based on two types of crowd flows in Beijing and NYC, finding that its performance exceeds six well-know methods.
Formulation of Crowd Flows Problem
The paper represents a city as an I × J grid and encodes inflow and outflow across all regions as a tensor. The forecasting task is to predict the next observation from historical observations.
- A city is partitioned into an I × J longitude–latitude grid, with each grid cell treated as a region.The grid-based segmentation provides the spatial structure for citywide flow modeling.
- Inflow counts crowds entering a region, whereas outflow counts crowds leaving it during a time interval.Both quantities describe transitions between regions and can be derived from trajectories.
- At each time interval, all regional inflows and outflows form a tensor X_t ∈ R^2×I×J.The two channels correspond to inflow and outflow over the grid.
- Given historical observations {X_t | t = 0, · · ·, n − 1}, the problem is to predict X_n.
Deep Residual Learning
ST-ResNet builds on deep residual learning to support very deep convolutional modeling. Its architecture uses separate components for temporal closeness, period, trend, and external influence.
- Deep residual learning enables convolutional networks with over 1000 layers and has achieved strong results on several recognition tasks.
- A residual unit adds a learned residual function F to its input through an identity mapping.The formulation uses the input and output of each residual unit together with learnable parameters.
- ST-ResNet comprises four major components modeling temporal closeness, period, trend, and external influence.The first three components receive flow matrices from recent, near-history, and distant-history fragments, respectively.
Structures of the First Three Components
The first three ST-ResNet components share convolutional residual architectures while processing different temporal fragments. Convolutions capture spatial dependencies, and residual units enable the depth needed for citywide modeling.
- The closeness, period, and trend components share a network structure composed of convolution and residual units.
- Convolutions model dependencies among nearby and distant regions without subsampling, preserving the I × J spatial resolution.The design uses multiple feature-map levels connected through convolutions to capture long-range dependencies.
- The closeness component concatenates recent 2-channel flow matrices before applying its first convolution.Its input sequence is [X_t−l_c, X_t−(l_c−1), · · ·, X_t−1].
- Residual units address training difficulties in deep networks while allowing sufficient depth to capture citywide dependencies.For 32 × 32 inputs with 3 × 3 kernels, more than 15 consecutive convolutional layers are needed for citywide dependence.
- Each residual unit applies the residual stack, while padding keeps convolutional inputs and outputs at the same I × J size.
- The period and trend components process sequences sampled at distinct spans, with p representing daily periodicity and q representing weekly trend.
The Structure of the External Component
The external component represents conditions at the prediction interval, including weather, holidays, and calendar metadata. Its output is fused with the three temporal components.
- External factors include weather, holiday events, and DayOfWeek and weekday/weekend metadata.Holiday periods and heavy rain can produce markedly different crowd-flow patterns in Beijing’s Office Area.
- The external component maps the feature vector E_t for the predicted interval to an output X^Ext.
Fusion
ST-ResNet fuses closeness, period, and trend components with learnable spatially varying weights, then incorporates external factors to produce predictions.
- Temporal evidence: Different regions exhibit distinct temporal characteristics, including recent-time dependence, daily periodicity, and region-specific trends.Beijing Office and Residential Areas show different closeness shapes, weekday patterns, and longer-term trends.
- Fusion: The first three components—closeness, period, and trend—are fused using a parametric-matrix-based method.The corresponding learnable parameters adjust each temporal property’s influence.
- External factors: The fused output is directly merged with the external component before generating the predicted flow matrix.External factors are incorporated after combining the three temporal components.
- Prediction and training: The model uses tanh in the final prediction expression to constrain output values between -1 and 1.Training minimizes mean squared error between the predicted and true flow matrices.
Algorithm and Optimization
ST-ResNet is trained from temporal-flow instances and external features using mini-batch optimization, then evaluated on city trajectory datasets against six baselines.
- Training data: Training instances combine closeness, period, and trend sequences with external features and the target flow matrix.The trend sequence uses intervals spaced by the selected trend span.
- Optimization: The training algorithm initializes learnable parameters, repeatedly samples mini-batches, and minimizes the objective until stopping criteria are met.The optimization procedure uses back-propagation and Adam.
- Datasets: TaxiBJ uses Beijing taxicab GPS and meteorology data, with the last four weeks reserved for testing.Earlier data are used for training, and inflow and outflow are derived from trajectories.
- Datasets: BikeNYC uses 2014 NYC Bike trajectory data, reserving the last 10 days for testing and earlier data for training.Records include trip duration, station IDs, and start and end times.
- Evaluation: ST-ResNet is compared with six baselines: HA, ARIMA, SARIMA, VAR, ST-ANN, and DeepST.DeepST includes variants targeting different temporal dependencies and external factors.
- Implementation: The implementation uses Theano and Keras, 64 filters in Conv1 and residual units, 2 filters in Conv2, and batch size 32.Temperature and wind speed are Min-Max normalized to [0, 1].
Results on TaxiBJ
On TaxiBJ, all seven ST-ResNet variants outperform six baselines, with the best reported variant reducing error to 16.69. Ablations support deeper residual networks, batch normalization, external factors, and parametric-matrix fusion.
- Benchmark comparison: All seven evaluated ST-ResNet variants outperform the six baseline models on TaxiBJ.The variants differ in depth and included factors.
- Ablation results: RMSE decreases as the number of residual units increases across L2-E, L4-E, and L12-E.The reported ablation links greater depth under residual learning with more accurate results.
- Ablation results: Batch normalization improves the residual unit: L12-E-BN performs best, while the single-convolution L12-single-E performs worse than L12-E.The comparison tests three internal residual-unit designs.
- Ablation results: External factors improve results, and replacing parametric-matrix fusion with straightforward fusion greatly increases error.The tested external factors include meteorology, holiday events, and metadata.
Results on BikeNYC
On BikeNYC, ST-ResNet generalizes across new-flow and end-flow prediction, achieving lower RMSE than the compared baselines. The paper positions this result within citywide crowd-flow prediction and identifies broader flow types as future work.
- Benchmark comparison: 14.8% to 37.1% lower RMSE is reported for ST-ResNet than the baselines on BikeNYC.BikeNYC evaluates new-flow and end-flow using a four-residual-unit model with metadata as external features.
- Task scope: BikeNYC tests new-flow and end-flow, extending evaluation beyond the inflow and outflow setting described for citywide crowd flows.The paper distinguishes aggregated regional flows from individual mobility traces and road-segment prediction.
- Benchmark comparison: The authors characterize the BikeNYC results as evidence of good generalization to other flow prediction tasks.This is the paper’s stated interpretation of the cross-dataset comparison.
- Overall evaluation: The model is evaluated on two types of crowd flows in Beijing and NYC and performs beyond six baseline methods.The conclusion describes the evaluation as supporting the model’s applicability to crowd-flow prediction.
- Future work: Future work will incorporate additional taxi, truck, bus, phone-signal, and metro-card flow data for broader collective prediction.The authors also plan an appropriate fusion mechanism for these additional flow types.