Source-linked AI summary
Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction
Huaxiu Yao, Yiding Liu, Ying Wei, Xianfeng Tang, Zhenhui Li
TL;DR
The paper addresses spatial-temporal prediction when target cities have limited data and prior transfer methods use only one source city. MetaST learns a transferable initialization and long-term pattern memory from multiple cities, and experiments across traffic and water-quality tasks show its effectiveness over baselines.
Problem
Spatial-temporal prediction models often require abundant training data, while target cities may have limited samples; prior transfer methods generally use a single source city.
Method
MetaST combines a spatial-temporal network with meta-learning to learn a generalized initialization and transfers a global pattern-based memory of long-term spatial-temporal patterns.
Results
MetaST consistently outperforms several baseline models across taxi, bike, and water-quality prediction datasets.
Takeaways & Limitations
Using multiple source cities provides more comprehensive spatial-temporal knowledge and increases the stability of transfer and prediction.
Takeaways & Limitations
Future work will incorporate network structure and explain which information is transferred by the black-box framework.
Abstract
from arXiv · showhide
Spatial-temporal prediction is a fundamental problem for constructing smart city, which is useful for tasks such as traffic control, taxi dispatching, and environmental policy making. Due to data collection mechanism, it is common to see data collection with unbalanced spatial distributions. For example, some cities may release taxi data for multiple years while others only release a few days of data; some regions may have constant water quality data monitored by sensors whereas some regions only have a small collection of water samples. In this paper, we tackle the problem of spatial-temporal prediction for the cities with only a short period of data collection. We aim to utilize the long-period data from other cities via transfer learning. Different from previous studies that transfer knowledge from one single source city to a target city, we are the first to leverage information from multiple cities to increase the stability of transfer. Specifically, our proposed model is designed as a spatial-temporal network with a meta-learning paradigm. The meta-learning paradigm learns a well-generalized initialization of the spatial-temporal network, which can be effectively adapted to target cities. In addition, a pattern-based spatial-temporal memory is designed to distill long-term temporal information (i.e., periodicity). We conduct extensive experiments on two tasks: traffic (taxi and bike) prediction and water quality prediction. The experiments demonstrate the effectiveness of our proposed model over several competitive baseline models.
1 INTRODUCTION
Accurate spatial-temporal prediction supports smart-city management, but deep models require large training datasets that many cities lack. The paper addresses this scarcity by transferring knowledge from multiple source cities through MetaST.
- Spatial-temporal prediction supports transportation-resource allocation, traffic-signal control, environmental policy, and public health.
- Deep spatial-temporal models improve nonlinear correlation modeling but depend on large-scale training data.
- Prior transfer methods typically move knowledge from a single source city to a data-insufficient target city.
- Multiple source cities can provide more comprehensive spatial-temporal correlations and increase transfer stability.
- MetaST learns a generalized ST-net initialization across source-city tasks, adapts it by fine-tuning, and transfers memory of long-term spatial-temporal patterns.The ST-net combines a local CNN and an LSTM; the memory is jointly trained with it end-to-end.
- Experiments on taxi, bike, and water-quality datasets show that MetaST consistently outperforms several baseline models.
2 RELATED WORK
Earlier spatial-temporal prediction methods progressed from classical time-series and regularized models to deep architectures, but these approaches generally require abundant data. This work applies multi-city transfer and meta-learning to prediction in data-limited target cities.
- Spatial-Temporal Prediction: Classical approaches include time-series models, spatial-temporal regularization, and external context features.
- Spatial-Temporal Prediction: CNN, RNN, and hybrid deep models capture nonlinear spatial-temporal correlations and achieve state-of-the-art performance.
- Spatial-Temporal Prediction: Deep spatial-temporal methods rely on many training samples, motivating transfer from source cities to data-limited target cities.
- Knowledge Transfer and Reuse: Transfer learning reuses knowledge from source domains, while meta-learning transfers shared knowledge from multiple tasks to a new task.
- Knowledge Transfer and Reuse: Existing spatial transfer attempts focus on multimodal features or related settings rather than the spatial-temporal sequences studied here.
- Knowledge Transfer and Reuse: MetaST transfers shared knowledge from multiple cities to increase transfer and prediction stability.
3 DEFINITIONS AND PROBLEM FORMULATION
The paper represents each city as a grid of regions with spatial-temporal series and formulates prediction for a data-scarce target city using richer source-city histories. Its ST-net predicts the next target timestamp from an initialization adapted through meta-learning.
- Definitions: Each city is divided into an Ic × Jc grid, with each grid cell treated as a region.
- Definitions: A city’s spatial-temporal series contains evenly split, non-overlapping intervals ending at its current timestamp.
- Problem Formulation: The target variable may represent traffic demand, air quality, or another spatial-temporal quantity.
- Problem Formulation: Given source cities with much longer histories than the target, the task is to predict the target city’s information at the next timestamp.
- Problem Formulation: The prediction model is an ST-net, whose parameters begin from initialization θ0 adapted from source-city series.
4 METHODOLOGY
MetaST combines a CNN-LSTM spatial-temporal network with meta-learning and a pattern-based memory to transfer knowledge from multiple cities to data-limited targets.
- 4.1 Spatial-Temporal Network: The ST-net uses CNNs to capture spatial dependencies and LSTMs to model temporal evolution for spatial-temporal prediction.Each region and its neighbors are represented as a multi-channel N × N image before temporal modeling.
- 4.2 Knowledge Transfer: The framework transfers knowledge from multiple cities because their information covers broader spatial-temporal correlations and is intended to improve transfer stability.The method addresses variation in target-city correlations across cities and time periods through flexible adaptation.
- 4.2 Knowledge Transfer: MetaST learns a generalized ST-net initialization from prediction tasks sampled across multiple source cities, enabling adaptation to target-city data.The initialization is optimized for generalization across source-city tasks and updated using target-city training data.
- 4.2.2 Spatial-Temporal Memory: The spatial-temporal memory stores representations of clustered long-term patterns associated with different region functionalities.Regions in source cities are clustered into G categories, with each memory row representing one pattern category.
- 4.2.2 Spatial-Temporal Memory: Attention queries the memory using short-term spatial-temporal representations, and the retrieved pattern representation is concatenated with the ST-net representation for prediction.The memory remains fixed during target-city transfer while ST-net parameters are updated on target training data.
5 EXPERIMENT
The experiments evaluate MetaST on traffic-volume and water-quality prediction, while also examining hyperparameters and whether ST-mem detects distinct spatial-temporal patterns.
- 5 EXPERIMENT: The experiments cover two domain applications: traffic volume prediction and water quality prediction.The stated evaluation asks whether MetaST outperforms baseline methods on these inference tasks.
- 5 EXPERIMENT: The study examines how hyperparameters such as ST-mem cluster dimensions and the trade-off factor affect model performance.This evaluation targets sensitivity to design and weighting choices.
- 5 EXPERIMENT: The experiments test whether ST-mem can detect distinguished spatial-temporal patterns.This evaluates the interpretive pattern-detection role of the memory in addition to predictive performance.
5.1 Application-I: Traffic Prediction
Traffic prediction experiments evaluate MetaST on taxi and bike volume using multiple source cities and short target-city training periods. MetaST and other transfer methods improve over target-only ST-net, while memory performance depends on parameter selection.
- Datasets and setup: Taxi experiments use NYC, DC, and Porto as source cities, with Chicago and Boston as target cities; Boston reports pick-up volume only.Bike experiments use NYC and DC as source cities and Chicago as the target city.
- Datasets and setup: Target cities are trained with 1-day, 3-day, or 7-day data, while source-city data use 80% for training and validation and the remainder for testing.
- Compared baselines: The comparison includes target-only HA, ARIMA, and ST-net baselines alongside single-source fine-tuning, multi-source fine-tuning, RegionTrans, MAML, and MetaST.Non-transfer baselines use only limited target-city training data, whereas transfer baselines use source-city knowledge.
- Results: All transfer learning models significantly improve over ST-net in the reported cases, indicating that city-to-city knowledge transfer helps prediction with limited data.HA and ARIMA can remain competitive in some 1-day settings because traffic data exhibit strong daily periodicity.
- Results: Multi-FT usually outperforms Single-FT, but an optimally transferable single source can outperform the mixed-source model in some cases.
- Results and sensitivity: MetaST outperforms all baselines in the reported traffic experiments, while Figure 6 indicates that its memory distinguishes regions with different temporal patterns.Figure 5 varies memory dimension and γ for Chicago taxi and bike RMSE; both parameters show performance that initially improves and later declines.
5.2 Application-II: Water Quality Prediction
Water-quality experiments predict monthly pH values across U.S. regions treated as cities, transferring knowledge from data-rich source areas to target areas. MetaST achieves the best reported performance, with sensitivity to memory dimension and loss trade-off.
- Task and data: The task predicts the next month’s pH value at a specific location as a proxy for changes in water quality.
- Task and data: The Water Quality Portal data span about 52 years from 1966 to 2017, with six U.S. areas treated as cities.
- Task and data: Pacific, West, and Midwest are source areas, while Northeast, Southwest, and South are target areas represented by grid regions.Each region’s pH value is the median of its sampling points.
- Baselines: The experiment uses the traffic-prediction baselines, with HA computed over monthly relative times.
- Results: MetaST achieves the best performance against all baselines, with an averaged relative improvement of 7.7%.The authors attribute this to spatial-temporal memory providing detailed long-term information and increasing source-region diversity.
- Parameter sensitivity: On Northeast prediction with 3-year training data, MetaST performs best at pattern dimension d = 4 and trade-off factor γ = 10^-4.The reported sensitivity results indicate that suitable choices of d and γ produce the best performance.
6 CONCLUSION AND DISCUSSION
MetaST transfers knowledge from multiple cities for spatial-temporal prediction, evaluates it across traffic and environment tasks, and identifies network structure and transfer interpretability as future directions.
- MetaST transfers knowledge from multiple cities to support prediction in data-scarce target cities.
- Figure 7 examines RMSE as the spatial-temporal memory dimension and γ vary.
- The model learns a well-generalized initialization and uses a global pattern-based spatial-temporal memory from source cities.
- MetaST is evaluated on traffic prediction and environment prediction tasks from two different domains.
- Future work will incorporate network structure such as road structure and explain which information the black-box transfer framework transfers.