Source-linked AI summary
Quantifying geographic domain shift to decouple the geospatial transferability of human mobility flow generation models
Zhiyong Zhou, Song Gao, Qianheng Zhang, Feng Zhang, Zhenhong Du
TL;DR
Human mobility models are difficult to assess across unseen regions because training data cover limited geographic areas. This paper quantifies geographic domain shift and finds heterogeneous, asymmetric transfer performance with complementary information and spatial shifts.
Problem
Training data typically cover limited geographic areas, leaving the intrinsic characteristics of models’ geospatial transferability insufficiently studied.
Method
The study evaluates four mobility generation models across 2,265 U.S. counties and quantifies source–target geographic domain shift using mutual information and spatial-shift metrics.
Results
Transfer performance shows heterogeneous and asymmetric source–target patterns, while mutual-information shift and Moran spatial shift play complementary explanatory roles.
Takeaways & Limitations
Geospatial transferability should be characterized through both information-distribution differences and spatial-structure differences between source and target domains.
Takeaways & Limitations
The analysis relies on a single mobility-flow dataset from one country, which may introduce partial dependence between derived geographic-domain measures.
Abstract
from arXiv · showhide
Human mobility serves as an essential proxy for understanding social, economic, and environmental dynamics in urban systems. Geospatial transferability, which measures a model's capability in a new location or unseen region, is a critical dimension for comparing different human mobility generation models. However, few studies have studied the intrinsic characteristics of geospatial transferability. To this end, this study systematically investigates the geospatial transferability of four representative human mobility generation models using a large-scale benchmark dataset of census tract level commuting flows across 2265 counties in the United States. Inspired by the domain adaptation theory in machine learning, we introduce geographic domain shift to describe the intrinsic differences in geographic feature distributions and spatial structures between source and target regions, which may jointly affect model transferability. Moreover, we propose two metrics, mutual information and spatial shift, to quantify the geographic domain shift. To examine their associations with model transferability, we employ linear mixed-effects regression to analyze the associations between geographic domain shifts and transferability. Our results reveal substantial spatial heterogeneity and asymmetry in transfer performance across regions. Both information shift and spatial shift exhibit statistically significant and complementary explanatory power. This indicates that geospatial transferability depends not only on model design but also on intrinsic geographic differences. These findings provide a novel methodological framework for evaluating and improving the geospatial transferability of human mobility generation models and support more robust and fair human mobility data synthesis across diverse regions. It also offers insights on spatial transferability for GeoAI model development.
1. Introduction
Human mobility data are valuable but often geographically scarce, making synthetic mobility generation important and raising concerns about geospatial transferability. This study conceptualizes geographic domain shift, quantifies it with mutual information and Moran’s I spatial shift, and examines its relationship with transfer performance across U.S. counties.
- Motivation: Human mobility supports analysis of urban spatial-social dynamics, but infrastructure costs, privacy concerns, and limited geographic coverage constrain comprehensive analyses.Synthetic or generated mobility data can mitigate scarcity, bias, and inaccessible records.
- Research gap: Geospatial transferability is critical because training data typically cover limited regions, while intrinsic geographic differences between source and target areas are often overlooked.Such differences can leave the difficulty of transferring models unresolved and bias generated mobility data across areas.
- Study focus: The study argues that transferability depends on both model architectures and intrinsic geographic differences between source and target regions.It focuses on characterizing source and target geographic features before model training rather than developing a new transferable deep learning model.
- Study design: Four mobility generation models are trained in one state and transferred to counties in remaining states using census tract-level commuting OD flows from 2,265 U.S. counties.The models are DeepGravity, boosting regression tree, random forest, and geo-contextual multitask embedding learning.
- Methodological contribution: Mutual information shift and Moran’s I spatial shift quantify differences in geographic feature distributions and spatial structures between source and target areas.The study uses these geographic domain shift metrics to analyze how intrinsic geographic differences influence model transferability.
- Key findings: Transferability varies substantially across target areas, with substantial spatial heterogeneity and asymmetry in cross-region transferability.The proposed metrics may support adaptive training-area selection, model optimization, and fairer synthetic mobility data generation.
2. Related work
Prior work evaluates OD flow generation transferability mainly through post-hoc model comparisons, while rarely explaining regional variation through intrinsic geographic differences. Existing domain-shift measures capture feature distributions but generally omit spatial structures, motivating a framework that captures both.
- Mobility model transferability: Geospatial transferability is the ability to generate accurate mobility flows in new or previously unseen geographic areas.It is commonly evaluated using OD flow distribution similarity measures, including common part of commuters (CPC) and Jensen-Shannon Divergence (JSD), alongside root mean squared errors (RMSE) of OD flows.
- Mobility model transferability: Existing OD flow transferability research is largely model-centric, developing new architectures and then evaluating their transfer performance.Approaches include leave-one-city-out evaluation, domain-invariant representations, hierarchical knowledge transfer, adversarial training, meta-learning, and large language models.
- Research gap: Prior studies rarely explain why the same model has different transferability across target regions or how intrinsic geographic differences influence performance.Limited attention has been given to geographic features, spatial structures, and intrinsic input-data characteristics underlying cross-regional transfer variation.
- Geographic domain shift: Domain shift includes covariate shift in input-feature distributions and concept shift in conditional label distributions or concept meanings.Because labels are unknown when transferring to unseen geographic areas, the study primarily investigates covariate shift.
- Geographic domain shift: Common covariate-shift measures, including mutual information, MMD, Wasserstein distance, CORAL, and KL divergence, primarily characterize statistical distribution discrepancies.Geographic similarity measures additionally consider covariate or semantic similarity, but generally omit spatial feature distributions and structural differences.
- Geographic domain shift: A geographic domain-shift framework capturing both distributional shifts and spatial-structure shifts remains underexplored.Spatial structure shifts are fundamental to geographic processes and can be characterized using spatial statistics such as Moran’s I.
3. Data and methods
The study uses a large-scale census tract–level commuting-flow dataset spanning 2,265 counties and introduces model-agnostic metrics to quantify geographic domain shift. It evaluates feature-distribution differences and spatial-structure changes as complementary dimensions for assessing cross-region mobility-model transferability.
- Dataset: The 2018 inputs contain 97 demographic features and 36 POI categories aligned with census tracts, representing socioeconomic and built-environment characteristics.This alignment removes the need for additional input–output matching and supports reproducible analysis.
- Geographic domain shift: Geographic domain shift is explicitly quantified from intrinsic differences between source and target geographic domains rather than assumed to be uniformly different.The framework treats source–target differences as varying across domain pairs.
- Geographic domain shift: MI shift measures feature-distribution discrepancies, whereas Moran spatial shift measures changes in spatial autocorrelation and feature clustering or dispersion.Together, the metrics capture complementary attribute-level and spatial-structure aspects of geographic domain shift.
- Model evaluation: The proposed metrics depend only on geographic features and can be applied before model training to quantitatively assess transferability across domains.The study uses DeepGravity to generate census tract–level commuting OD flows within each of 2,265 counties.
4. Results
Results show that DeepGravity has the strongest predictive performance, but its transferability varies substantially and asymmetrically across geographic regions. Mutual-information and spatial shifts provide complementary, generally consistent explanations of transfer performance across models.
- Model comparison: DeepGravity outperforms the other three mobility generation models, achieving the highest CPC and lowest RMSE.High RMSE standard deviations indicate extreme cases in predicted OD-flow values, motivating the study’s focus on DeepGravity.
- Spatial heterogeneity: RMSE transfer performance decreases overall from Midwestern to Eastern states, except for several areas in Maine and Florida.Western and Midwestern states show substantial errors, and changing the individual source state produces considerable RMSE variation.
- Geographic domain shifts and transferability: MI shift and Moran shift are non-collinear and jointly explain transferability, while source-domain changes can still create substantial variance within the same target county.The associations’ signs remain consistent across the four trained models, although their joint associations were not statistically significant in the log-transformed RMSE regression for states.
- Geographic domain shifts and transferability: MI shift is associated with poorer transferability, whereas stronger spatial autocorrelation is associated with improved transferability across CPC and log-transformed RMSE regressions.MI shift is negatively associated with CPC and positively associated with log(RMSE); Moran shift shows the opposite associations, with consistent signs across all four models.
5. Discussion
The discussion shows that geospatial transferability varies asymmetrically across regions and depends on both evaluation metrics and intrinsic geographic domain shifts. It also identifies implications for dataset curation and model design, while noting limitations requiring validation across datasets, models, scales, and spatial structures.
- Transfer heterogeneity: Leave-one-state-out evaluations reveal substantial spatial heterogeneity and asymmetry in transferability across state pairs, regardless of whether CPC or RMSE is used.
- Transfer heterogeneity: 0.58 to 0.65: DeepGravity’s mean CPC across individual source states varies by about 0.07, exceeding some reported baseline-model CPC differences.
- Evaluation dimensions: CPC and RMSE can yield inconsistent transfer assessments because CPC captures global flow-pattern similarity, whereas RMSE emphasizes local absolute magnitudes and outliers.
- Transfer asymmetry: Transfer asymmetry occurs when a model transfers successfully from a source region to a target region but not necessarily in the reverse direction.
- Domain-shift metrics: MI shift and Moran shift capture complementary geographic domain differences, with Moran shift adding spatial-shift information and indicating transferability alongside MI shift.
- Domain-shift metrics: The mean MI shift is significantly negatively associated with mean CPC, while only mean Moran shift is significantly negatively associated with mean RMSE.
- Implications: Dataset curation should prioritize intrinsic geographic characteristics and spatial distribution, while geographically transferable models should explicitly incorporate spatial structure.
- Limitations and future work: Future work should test these relationships across mobility datasets, geographic scales, countries, modeling approaches, and broader measures of spatial structure.
6. Conclusion
Using CommutingODGen across 2,265 U.S. counties, the study introduces mutual information shift and Moran shift to quantify geographic domain shift and explain geospatial transferability. The metrics reveal heterogeneous, asymmetric transfer patterns and complementary, significant associations with model transferability, supporting geographic training-data selection for more transferable GeoAI models.
- Study scope: The study evaluates representative human mobility generation models using census-tract–level commuting flows across 2,265 counties in 48 U.S. states.The analysis uses the large-scale CommutingODGen dataset.
- Methodological contributions: It proposes mutual information shift and Moran shift to quantify geographic domain shifts within a dataset.A linear mixed-effects regression model then examines associations between intrinsic dataset characteristics and model transferability.
- Key findings: Transfer performance shows heterogeneous and asymmetric patterns between source and target domains.The results also demonstrate complementary roles for mutual information shift and Moran spatial shift in characterizing geographic domain shift.
- Key findings: Regression analysis finds significant associations between the proposed domain shift metrics and model transferability.These associations demonstrate the feasibility of the metrics for assessing intrinsic characteristic differences between training and testing geographic datasets.
- Implications: The metrics can guide geographic training-dataset selection for developing more geographically transferable human mobility generation and other GeoAI models.The conclusion identifies considerable potential for improving geographic transferability through dataset selection.