Source-linked AI summary
Predicting Citywide Crowd Flows Using Deep Spatio-Temporal Residual Networks
Junbo Zhang, Yu Zheng, Dekang Qi, Ruiyuan Li, Xiuwen Yi, Tianrui Li
TL;DR
Forecasting citywide crowd flows is difficult because spatial, temporal, and external factors jointly shape regional inflow and outflow. ST-ResNet models these dependencies with residual branches and dynamic fusion, and outperforms nine baselines on Beijing and NYC crowd-flow tasks. The paper also deploys it in the UrbanFlow cloud system for Guiyang.
Problem
Citywide crowd-flow forecasting must jointly handle nearby and distant spatial dependencies, multiple temporal patterns, and external conditions for traffic management and public safety.
Method
ST-ResNet uses separate residual convolutional branches for temporal closeness, period, and trend, dynamically aggregates them by region, and incorporates external factors.
Results
ST-ResNet significantly outperforms 9 baseline methods on two crowd-flow types evaluated in Beijing and NYC.
Takeaways & Limitations
The approach supports citywide inflow and outflow forecasting and has been deployed in the UrbanFlow real-time monitoring and forecasting system.
Takeaways & Limitations
The study identifies predicting additional flow types and fusing them collectively as future work.
Abstract
from arXiv · showhide
Forecasting the flow of crowds is of great importance to traffic management and public safety, and very challenging as it is affected by many complex factors, including spatial dependencies (nearby and distant), temporal dependencies (closeness, period, trend), and external conditions (e.g., weather and events). We propose a deep-learning-based approach, called ST-ResNet, to collectively forecast two types of crowd flows (i.e. inflow and outflow) in each and every region of a city. We design an end-to-end structure of ST-ResNet based on unique properties of spatio-temporal data. More specifically, we employ the residual neural network framework to model the temporal closeness, period, and trend properties of crowd traffic. For each property, we design a branch of residual convolutional units, each of which models the spatial properties of crowd traffic. ST-ResNet learns to dynamically aggregate the output of the three residual neural networks based on data, assigning different weights to different branches and regions. The aggregation is further combined with external factors, such as weather and day of the week, to predict the final traffic of crowds in each and every region. We have developed a real-time system based on Microsoft Azure Cloud, called UrbanFlow, providing the crowd flow monitoring and forecasting in Guiyang City of China. In addition, we present an extensive experimental evaluation using two types of crowd flows in Beijing and New York City (NYC), where ST-ResNet outperforms nine well-known baselines.
1. Introduction
Citywide crowd-flow prediction matters for traffic management and public safety but must account for complex spatial, temporal, and external influences. ST-ResNet addresses these challenges by jointly predicting regional inflow and outflow with multiple residual branches and dynamic aggregation.
- Motivation: Crowd-flow prediction can support advance traffic control, warnings, and evacuation during dangerous crowd events.The motivation includes stampedes in Shanghai and New York City, where advance predictions could help activate emergency mechanisms.
- Prediction task: The task involves predicting both inflow and outflow for every city region.Inflow counts crowds entering a region, while outflow counts crowds leaving it during a time interval.
- Challenges: Crowd flows exhibit nearby and distant spatial dependencies, including interactions between regions connected by commuting patterns.Flows in one region can affect flows elsewhere, including distant office and residential areas.
- Challenges: Temporal dependencies comprise closeness, period, and trend, reflecting recent intervals, recurring daily patterns, and gradual changes over time.Examples include rush-hour effects, weekday repetition, and seasonal shifts in morning activity.
- Challenges: Weather and events can substantially alter crowd flows across different city regions.A thunderstorm can change road traffic speed and consequently affect regional flows.
- Approach: ST-ResNet uses three residual networks for temporal properties, dynamically weights their regional outputs, and combines them with external factors.Its convolution-based residual structure models nearby and distant spatial dependencies while supporting deep networks.
- Extensions: The expanded study adds a cloud-based real-time system, multi-step forecasting, broader baseline and architecture experiments, and additional related-work analysis.The system continuously forecasts taxi flows in every region of Guiyang City, while the experiments evaluate robustness and resource variation.
2. Preliminary
The paper represents city regions and their inflow/outflow observations on a grid, then formulates forecasting as predicting the next tensor from historical observations. It also motivates residual learning for deep convolutional modeling.
- Region representation: A city is partitioned into an I × J longitude-latitude grid, with each grid cell representing a region.The grid-based representation supplies the spatial structure used for crowd-flow modeling.
- Flow definition: Inflow counts trajectories entering a grid during a time interval, while outflow counts trajectories leaving it.Trajectory points and their transitions across grid cells determine the two flow quantities.
- Flow representation: All regional inflows and outflows at time t are represented as a tensor X_t ∈ R^2×I×J.The two channels correspond to the two flow types across all grid locations.
- Prediction problem: The forecasting problem is to predict X_n from historical observations {X_t | t = 0, · · · , n − 1}.The formulation treats the citywide flow tensor as the prediction target.
- Deep residual learning: Residual learning lets convolutional networks become very deep and trains them by learning an additive residual function relative to the input.The cited framework supports networks reaching 100 or more than 1,000 layers and defines residual units through input, output, and residual mappings.
3. System Architecture
The system uses a hybrid local-GPU and cloud architecture to train ST-ResNet offline while continuously processing real-time data and serving crowd-flow forecasts to users. UrbanFlow presents these inflow and outflow forecasts through a web interface for Guiyang City.
- The Cloud: The cloud receives real-time taxi trajectories and meteorological data, computes flows for every city region, extracts external features, and stores results.Data are cached in Redis before virtual machines process and store crowd-flow data and extracted features.
- The Cloud: Historical data are retained locally while only recent cloud data are stored, reducing storage expense and avoiding costly transfer of massive trajectories.The cloud stores crowd-flow data and features from the past two days, while historical data can be moved to local servers periodically.
- The Cloud: Azure hosts the forecasting service with an A2 standard VM, a shared App Service, and a 6 GB Redis Cache.The cache holds recent trajectories, two days of crowd-flow data and features, and inferred results.
- System Architecture: The hybrid framework combines local GPU servers for offline processing with the cloud for online crowd-flow forecasting.Local servers handle trajectory conversion, external-feature extraction, and model training; the learned model is uploaded to the cloud.
- User Interface: UrbanFlow displays region-level inflow or outflow on a map, using grid values and colors to distinguish denser from sparser crowd flows.Users can inspect regional flow curves and sequential timestamp heatmaps through the interface.
4. Deep Spatio-Temporal Residual Networks
ST-ResNet models crowd-flow dynamics through separate closeness, period, trend, and external-factor components. Convolutional residual branches capture nearby and distant spatial dependencies, while parameter matrices fuse temporal components before external features are incorporated.
- Architecture: ST-ResNet divides historical flow data into recent, near-history, and distant-history fragments to model closeness, period, and trend separately.Each fragment supplies 2-channel flow matrices to one of three components.
- First Three Components: The first three components share a convolutional network followed by a sequence of residual units, capturing spatial dependencies across nearby and distant regions.The architecture uses convolution and residual units to expand spatial coverage while retaining residual learning.
- Convolution: Same convolution preserves the input grid size by padding outside borders, unlike narrow convolution, which reduces spatial dimensions.With a 5 × 5 input and 3 × 3 filter at stride 1, narrow convolution produces a 3 × 3 output.
- Temporal Dependencies: The period component uses daily history, whereas the trend component uses weekly history as their respective temporal spans.The implementation sets p to one day and q to one week.
- Fusion and External Factors: External features such as weather and events are processed separately, while the three temporal outputs are fused with learnable parameter matrices.The fusion weights adjust the influence of closeness, period, and trend by region.
- Temporal Dependencies: Different regions exhibit distinct temporal characteristics, including stronger weekday peaks in office areas and different long-term trends across regions.The Beijing observations show temporal closeness, daily periodicity, and region-dependent trends.
5. Experiments
The experiments evaluate ST-ResNet on two crowd-flow types in Beijing and NYC against nine baseline methods.
- ST-ResNet is evaluated on inflow and outflow prediction in Beijing and NYC against 9 baselines.
5.1. Settings
The experiments use trajectory and weather datasets from Beijing and NYC, with held-out temporal test periods and comparisons against nine baseline families. Inputs are normalized, external metadata is one-hot encoded, and performance is measured with RMSE.
- Datasets: TaxiBJ uses Beijing taxicab GPS and meteorology data across four periods, reserving the last four weeks for testing.
- Datasets: BikeNYC uses 2014 NYC bike trips from April through September, reserving the last 10 days for testing.
- Preprocessing: Flow data are Min-Max normalized to [−1, 1], while day-of-week, weekend/weekday, holidays, and weather are transformed into binary vectors.
- Evaluation: Performance is measured using Root Mean Square Error (RMSE) after predictions are rescaled to their normal values.The metric compares predicted values with available ground truth.
5.2. Evaluation of Single-step Ahead Prediction
ST-ResNet consistently outperforms the evaluated baselines for single-step crowd-flow prediction on TaxiBJ and BikeNYC, with substantial gains reported across models.
- 26% better than ARIMA and 37% better than SARIMA on TaxiBJ, ST-ResNet outperforms traditional forecasting baselines.
- 14% better than ST-ANN and 7% better than DeepST on TaxiBJ, ST-ResNet surpasses neural and spatio-temporal baselines.
- 28% to 64% better than RNN, ST-ResNet achieves lower TaxiBJ error across the reported RNN comparisons.
- 17.4% to 46.1% better than GRU and 18.1% to 45.7% better than LSTM, ST-ResNet leads the recurrent-model comparisons on TaxiBJ.
- 9% up to 71% lower RMSE than the baselines on BikeNYC, ST-ResNet demonstrates generalization across new-flow and end-flow tasks.
5.3. Results of Di
Variant experiments examine normalization, fusion, residual-unit structure, depth, filter settings, and temporal inputs, revealing both performance gains and practical limits.
- Different configurations: Slightly improved RMSE results from adding batch normalization to each residual unit in single-step prediction.
- Different configurations: Significantly better results from parametric-matrix-based fusion than straightforward fusion of closeness, period, and trend components.
- Different configurations: Two convolutions per residual unit outperform one convolution in the tested internal-structure comparison.
- Impact of network depth: RMSE first decreases and then increases with depth; networks with at least 14 residual units become much harder to train.
- Impact of filter size and number: Larger filters yield lower RMSE, and increasing the number of filters produces better results by enlarging spatial receptive fields.
- Impact of temporal closeness, period, trend: On TaxiBJ, closeness performs best at lc = 4, period at lp = 1, and trend at lq = 1; longer ranges can degrade performance.
- Visualization of parameters: At threshold 0.3, all regions show some closeness, while 9% have weak periodic patterns and 7% lack temporal trend.
5.4. Evaluation of Multi-step Ahead Prediction
Multi-step prediction uses historical observations together with recent predictions to forecast subsequent intervals, with ST-ResNet performing best for a near-future horizon.
- ST-ResNet performs best among 13 models in 4-step ahead prediction on TaxiBJ.
- ST-ResNet significantly outperforms its closeness-only and closeness-plus-period variants, highlighting period and trend components in multi-step prediction.
- 4-step ahead prediction on TaxiBJ means predicting crowd flows in the next 2 hours.
5.5. Efficiency and Resources
The cloud deployment completes the full crowd-flow prediction pipeline within seconds, with a cheaper virtual machine providing a favorable efficiency trade-off.
- 18.56 seconds on an A2 standard VM and 10.93 seconds on a more powerful D4 standard VM complete the full prediction process.The pipeline pulls trajectories, converts them to crowd-flow data, predicts near-future flows, and pushes results back into Redis.
- The A2 standard VM costs 20% as much as the D4 while achieving more than 50% of its performance.
6. Related Work
Prior work addresses mobility prediction through individual trajectories, road traffic, conventional time-series models, and neural networks, while this paper specializes deep spatio-temporal prediction for citywide crowd flows.
- Mobility and traffic prediction: Individual-movement studies forecast mobility traces, whereas this paper targets aggregated crowd flows within city regions.The distinction reflects different application needs and computational requirements.
- Mobility and traffic prediction: Road-traffic research commonly predicts travel speed or traffic volume rather than region-level crowd inflow and outflow.
- Conventional time-series models: Historical averages and ARIMA provide conventional time-series baselines but may respond poorly to dynamic incidents or require stationarity transformations.
- Neural-network models: ANNs offer nonlinear modeling ability but are described as weaker for linear modeling than conventional linear models.
- Neural-network models: CNNs capture spatial structures and RNNs capture temporal patterns, motivating deep models that address both properties for spatio-temporal data.
- This paper's position: Building on a prior general DNN prediction model, this paper adds residual learning and parametric-matrix-based fusion for citywide crowd-flow prediction.
7. Conclusion and Future Work
The paper presents ST-ResNet for citywide crowd-flow forecasting and reports performance beyond nine baselines, while identifying broader flow types as future work.
- Conclusion: ST-ResNet forecasts crowd flows in every city region from historical spatio-temporal data, weather, and events.
- Conclusion: ST-ResNet models nearby and distant spatial dependencies, closeness, period, trend, and external factors.
- Conclusion: The model achieved performances significantly beyond 9 baseline methods on inflow and outflow data from Beijing and NYC.
- Conclusion: UrbanFlow uses Microsoft Azure Cloud to monitor real-time crowd flows and forecast near-future flows in Guiyang City.
- Future Work: Future work will incorporate additional flow sources and predict multiple flow types collectively with an appropriate fusion mechanism.