PaperlayerView exact revision on arXiv

Source-linked AI summary

MoRAX: Mobility-based Representation Augmentation for Geospatial Foundation Models

Ya Wen, Jixuan Cai, Yulun Zhou, Alec Kirkley

arXiv:2608.17848v1cs.LGcs.SI

TL;DR

Earth-observation-based geospatial foundation models underrepresent human activity, while mobility data is often city-limited, hindering transferable urban representation learning. MoRAX uses mobility-conditioned modulation and teacher-student distillation to improve cross-city representations, with the teacher achieving relative gains of up to 73.3% in cross-country transfer.

  • Problem

    Geospatial foundation models underrepresent human activity, while observed mobility data is often limited to individual cities, challenging transferable urban representation learning.

  • Method

    MoRAX uses lightweight mobility-induced modulation to reinterpret shared geospatial foundation embeddings and distills this adaptation into a mobility-free student using proxy relational signals.

  • Results

    Up to 73.3% relative gains in cross-country transfer were achieved, with the mobility-aware teacher consistently outperforming foundation embeddings and urban representation baselines.

  • Takeaways & Limitations

    MoRAX provides a transferable mechanism for grounding geospatial foundation representations in the human dimension of cities, including graph-free deployment across countries.

  • Takeaways & Limitations

    The teacher requires the target-city mobility graph, and some fine-grained human interaction patterns remain difficult to recover without it.

Abstract

from arXiv · show

Geospatial Foundation Models (GFMs) are emerging as a powerful paradigm for learning semantically rich and geographically consistent visual and physical representations. However, their reliance on Earth-observation (EO) data leaves information about human activity largely underrepresented. Human mobility data reveals the functional and relational structure between regions that is missing from EO data, but is often limited only to the city where it is observed, making it challenging to use for transferable urban representation learning. We introduce MoRAX, a lightweight framework for augmenting geospatial embeddings with functional structure derived from human mobility. MoRAX preserves the coverage and consistency of a GFM while providing information about the functional connectivity among urban regions, permitting zero-shot deployment in unseen cities with or without available mobility data. Across four target cities spanning two countries, the MoRAX teacher model, which observes mobility, consistently outperforms GFMs and strong urban representation baselines in eight socioeconomic and environmental prediction tasks. Meanwhile, the student model, which never takes mobility data as input, approaches the teacher in performance on most tasks. Transfer results across countries further demonstrate that modulation conditioned on mobility flows provides a general mechanism for grounding geospatial foundations in the human dimension of cities.

1 Introduction

MoRAX augments EO-based geospatial foundation representations with functional structure from human mobility through lightweight modulation. A teacher-student framework distills this mobility-conditioned adaptation into a mobility-free encoder for deployment where mobility data is unavailable.

  • Objective: MoRAX targets region representations that encode both physical and environmental properties and the functional role of each place within the broader urban system.Such representations support downstream applications including socioeconomic estimation, environmental monitoring, and urban planning.
  • Motivation: GFMs provide geographically extensive representations of natural and built environments from large-scale Earth observation data, but do not explicitly capture human mobility’s functional and interaction-driven urban structure.Mobility data reveals how people move between regions and complements the physical and environmental information encoded by GFMs.
  • Method: MoRAX learns lightweight feature-wise modulation conditioned on mobility to adapt an existing GFM representation rather than reconstructing its underlying semantic space.The modulation captures how a place’s meaning changes according to its exchange relationships with the rest of the city.
  • Teacher-student framework: A mobility-observing teacher learns mobility-aware representations, while a student distills the modulation using only mobility-free signals and never takes mobility as input.This separation supports deployment in cities without observed mobility data while retaining the broad geographic coverage of EO-based GFMs.
  • Evaluation: Zero-shot evaluations find that the mobility-aware teacher consistently outperforms geospatial foundation representations and urban representation baselines.The supplied contribution passage states this result without reporting task-level quantitative values.

2 Related Work

Related work spans transferable urban representation learning, EO-based geospatial foundation models, and lightweight adaptation of pretrained representations. Existing urban methods use diverse signals but commonly train and evaluate within one city, limiting generalization across urban structures and data modalities.

  • Transferable Urban Region Representation Learning: Urban region representation methods use points of interest, mobility flows, or satellite imagery, often combining modalities through fusion and alignment within the same city.The diversity of data sources complicates fair comparison across methods.
  • Transferable Urban Region Representation Learning: These single-city representations have limited capability to generalize to cities with different urban structure and data modality availability.
  • Transferable Urban Region Representation Learning: Large-scale urban pretraining has organized heterogeneous human signals around mobility, with MoRA learning a nation-scale shared representation space and PDFM modeling population dynamics over a large neighborhood graph.MoRA accumulates, compares, and reuses human-centered urban knowledge across locations.
  • Earth Observation-Based Geospatial Foundation Models: EO-based geospatial foundation models leverage the volume, diversity, and geographic coverage of Earth observation data to learn increasingly general and reusable representations of Earth’s surface.AlphaEarth Foundations integrates multi-source, temporal, and spatially contextualized Earth observations into globally available geospatial embeddings.
  • Adaptation of Pretrained Foundation Models: Adaptation methods adjust pretrained foundation-model representations using compact task- or domain-specific trainable parameters learned once and held fixed during inference.These approaches adapt models to new tasks, domains, and information sources without retraining from scratch.

3 Preliminaries

The preliminaries define urban regions, their complementary features, human mobility graphs, and pretrained geospatial foundation embeddings. They frame the task as learning mobility-augmented region representations that transfer zero-shot to unseen cities without modifying the foundation model or training on target-city data.

  • Regions: A city c is partitioned into N_c disjoint urban regions, denoted R_c = {r_c1, r_c2, …, r_cN_c}.
  • Region Features: Region features are drawn from complementary data sources and are selected independently of manually defined taxonomies such as POI category schemes.
  • Human Mobility Graph: A human mobility graph G_m = (R, E_m) represents directed interactions between regions, with edge weight w_ij measuring interaction intensity during a specified period.
  • Geospatial Foundation Embedding: Each region has a pretrained foundation embedding s_i summarizing EO-derived physical, environmental, and spatial characteristics in a globally consistent representation space.
  • Problem Setting: The objective is to learn mobility-augmented representations that transfer zero-shot from source cities to unseen target cities without updating the foundation model or training on target-city data.

4 Methodology

MoRAX augments frozen geospatial foundation representations with mobility-derived relational context through lightweight feature-wise modulation. A mobility-aware teacher encodes observed flows, while a graph-free student approximates the teacher’s modulation using geospatial information alone.

  • Relational modulation: Feature-wise scale-and-shift modulation reinterprets the projected foundation representation while retaining its geospatial semantic basis and adding functional regional information.The scale is parameterized as 1 + tanh(γ_i), constraining each coefficient to (0, 2) and centering modulation at identity.
  • Shared geospatial foundation: MoRAX projects each region’s pretrained EO-derived embedding into a shared urban representation space used across cities and by both models.The foundation model remains frozen, while a shared MLP projector maps embeddings into MoRAX’s hidden space.
  • Training: Teacher and student use the same modulation form but independently learned parameters, sharing only the frozen foundation embeddings and projector.Training first optimizes the teacher on source-city mobility graphs, then freezes it while its mobility-induced modulation supervises the student.
  • Mobility-aware teacher: The teacher derives relational context from observed mobility flows using an edge-centric encoder that updates relation states before aggregating them into region-level contexts.This design represents mobility on region pairs and summarizes multi-layer relational profiles rather than merely smoothing neighboring node features.
  • Graph-free student: The graph-free student infers proxy relational context from pairwise pretrained embeddings and standardized inter-region geography, without reconstructing or accessing the target mobility graph.A shared pair encoder and mean pooling produce a permutation-invariant city-level relational summary.

5 Experiments

Experiments evaluate MoRAX under mobility-available and mobility-free zero-shot transfer across Chinese and U.S. cities using socioeconomic and environmental prediction tasks. MoRAX-Teacher delivers substantial gains over geospatial and urban representation baselines, while the distilled student transfers much of the mobility benefit without target-city mobility input.

  • Evaluation Procedure: The teacher uses a held-out city’s mobility graph without fine-tuning, whereas the student uses only widely available region signals and never observes the target city’s mobility graph.These define the mobility-available and mobility-free deployment regimes.
  • Cross-city and cross-country transfer: MoRAX-Teacher consistently outperforms pretrained foundation embeddings and urban representation baselines, with relative gains up to 54.9% within-country and 73.3% cross-country.Evaluation covers four held-out cities and reports R2 transfer performance.
  • Cross-city and cross-country transfer: 37.4% and 202.6% are the relative gains in average R2 when MoRAX-Teacher raises AlphaEarth from 0.558 to 0.766 within-country and from 0.219 to 0.663 cross-country.The gains occur even when AlphaEarth already performs strongly on some tasks, including nighttime-light prediction.
  • Student transfer: MoRAX-Student improves over AlphaEarth by 36.0% within-country and 136.1% cross-country, while generally approaching the teacher on spatially smooth tasks.Larger teacher–student gaps remain on crime and check-in prediction, indicating difficulty recovering fine-grained interactions without target mobility.
  • Bidirectional cross-country transfer: Training on five Chinese cities improves the mean R2 from 0.535 to 0.591, a 10.4% relative gain, and wins 14 of 16 model–task comparisons against U.S.-city training.The result supports source diversity as more important than geographic proximity alone for cross-country transfer.
  • Foundation-model generalization: Both MoRAX variants consistently improve RemoteCLIP representations across all four Shanghai and Guangzhou tasks, demonstrating applicability beyond AlphaEarth.Additional ablations show that mobility benefits from the foundation’s stable semantic base and relation supervision.

6 Conclusion

MoRAX learns transferable urban representations by using mobility to modulate a shared geospatial foundation, treating mobility as relational context beyond Earth-observation semantics. It also distills this modulation into a graph-free student using widely available proxy signals, improving transfer across cities, backbones, and countries.

  • Core framework: MoRAX uses mobility to reshape the functional interpretation of stable, cross-city geospatial foundation embeddings.The framework addresses the insufficiency of Earth-observation data alone for urban understanding.
  • Graph-free transfer: MoRAX distills its modulation process into a graph-free student that relies only on widely available proxy signals, such as pairwise distances.The student does not require mobility data at deployment.
  • Transferability: Across Chinese and U.S. cities, MoRAX consistently improves over pretrained geospatial embeddings and existing urban representation baselines.This supports its use for transferable urban representation learning across observed and unobserved mobility settings.
  • Generalizability: MoRAX generalizes across foundation model backbones and countries, strengthening geospatial foundations along a human-centric dimension.The reported generalizability extends beyond a single backbone or national context.

A Constructing a Visual Field from RemoteCLIP

MoRAX constructs a spatially coherent visual field from independently encoded RemoteCLIP satellite images by applying parameter-free heat diffusion before relational modulation. The field is then projected into the model hidden space and used by both teacher and student models, while later components remain unchanged.

  • Motivation: RemoteCLIP independently encodes each region’s satellite image, so heat diffusion is applied to provide explicit spatial continuity before relational modulation.This contrasts with AlphaEarth’s directly spatially contextualized geospatial embedding field.
  • Graph construction: The method builds a distance-weighted K-nearest-neighbor graph with K = 8 over region centroids, then symmetrizes and row-normalizes it into transition matrix P.Edge weights decay exponentially with geographic distance.
  • Model integration: The diffused satellite embeddings are projected into the model hidden space to form the RemoteCLIP-based visual field for both teacher and student.Heat diffusion only constructs the spatially coherent field; the mobility encoder, pairwise student encoder, modulation heads, and training objectives remain unchanged.

B Importance of the Semantic Field

MoRAX’s relational modulation depends on a coherent region-level semantic field, not merely isolated image embeddings. Removing heat diffusion weakens both teacher and student, indicating that mobility or proxy context reinterprets an aligned spatial field.

  • Semantic field ablation: The ablation compares the complete RemoteCLIP-based model with a variant applying identical teacher and student modulation directly to isolated region embeddings.All other components and training settings remain unchanged.
  • Semantic field ablation: Removing heat diffusion consistently and substantially weakens both the teacher and student.Results are reported using R2.
  • Interpretation: Relational modulation benefits from an aligned, spatially coherent base that lets mobility or proxy context reinterpret a semantic field rather than disconnected image embeddings.The field structure provides the shared semantic context for modulation.

C Datasets · C.1 Training Datasets

The training datasets combine region-level mobility graphs from publicly available U.S. ridehailing records with AlphaEarth geospatial foundation features. Mobility flows are aggregated from origin–destination trips, while AlphaEarth representations are computed by averaging 10-meter embedding pixels within each H3 region.

  • C.1 Training Datasets: New York City mobility flows are constructed from Taxi and Limousine Commission trip records.The records are publicly available and released by the local transportation agency.
  • C.1 Training Datasets: Chicago mobility flows are constructed from Transportation Network Providers trip records released by the City of Chicago.These are publicly available ridehailing records from a local transportation agency.
  • C.1 Training Datasets: Both U.S. mobility datasets provide origin–destination information for constructing mobility graphs.The trips are aggregated into region-level mobility graphs.
  • C.1 Training Datasets: The mobility records are aggregated into region-level mobility graphs.This aggregation converts trip-level origin–destination information into regional connectivity data.
  • C.1 Training Datasets: AlphaEarth embeddings provide the geospatial foundation features in the main experiments.The embeddings are obtained from Google Earth Engine.
  • C.1 Training Datasets: For each H3 region, all 10-meter AlphaEarth embedding pixels within the region boundary are extracted and averaged.The resulting mean vector serves as the region-level representation.

C.2 Downstream Tasks

Downstream evaluations use socioeconomic and environmental targets across Shanghai, Guangzhou, New York City, and Chicago. Targets are constructed from crime, night-time light, carbon-emission, PM2.5, check-in, house-price, and income datasets using region-level aggregation and transformations.

  • Shanghai and Guangzhou: Crime cases in Shanghai and Guangzhou are aggregated over a 5-year interval.The crime data originate from Zhang et al..
  • Shanghai and Guangzhou: Night-time light targets use log(1 + sum) VIIRS radiance within each H3 region after removing invalid and negative pixels.VIIRS pixels are aggregated into region-level summary statistics.
  • Shanghai and Guangzhou: Carbon-emission targets use the log-transformed annual sum of monthly 1 km ODIAC 2024 GeoTIFF rasters for 2021 within each H3 region.Pixel values are aggregated across all months.
  • Shanghai and Guangzhou: PM2.5 targets use the mean of valid raster pixels within each H3 region from the 2021 CHAP / ChinaHighPM2.5 1 km annual raster.The source is a 2021 NetCDF raster.
  • NYC and CHI: New York City and Chicago evaluations use crime records, Foursquare check-ins, Zillow and NYC Department of Finance house prices, and U.S. Census Bureau income data.Crime records come from NYPD and the Chicago Police Department.
Loading 2608.17848v1…