Source-linked AI summary
BEACON: Behavioral and Semantic Enrichment of AlphaEarth Embeddings through Tri-Modal Contrastive Learning
Hao Tian, Heng Cai, Yifan Yang
TL;DR
Geospatial foundation-model embeddings capture physical appearance more strongly than human activity and urban function. BEACON aligns AlphaEarth with POI text and hourly visitation during training, then deploys an image-only representation; across nine tasks, it improves human-centered prediction while largely preserving physical and environmental performance.
Problem
Earth-observation-trained geospatial embeddings may weakly encode human activity and urban function, even though these dimensions matter for human-centered urban analytics.
Method
BEACON uses tri-modal contrastive learning to align AlphaEarth embeddings, POI descriptions, and temporal visitation profiles, while producing representations from AlphaEarth inputs alone at deployment.
Results
Across nine downstream tasks, BEACON substantially improves human-centered prediction while largely preserving AlphaEarth’s physical and environmental information; obesity prevalence reaches R^2 0.614 versus 0.431 for AlphaEarth (+43%).
Takeaways & Limitations
Aligning physical representations with semantic and behavioral signals extends geospatial foundation models toward human-centered urban analytics.
Abstract
from arXiv · showhide
Geospatial foundation models such as the AlphaEarth Foundation produce compact and globally consistent representations of the Earth's surface that transfer effectively to a wide range of downstream tasks. However, because these models are trained primarily on Earth-observation imagery, their embeddings mainly capture physical and spectral characteristics while encoding human activity and urban function only weakly. To address this limitation, we propose BEACON, a tri-modal contrastive learning framework that aligns three complementary views of urban space: physical representations from AE embeddings, semantic representations from point-of-interest (POI) text, and human behavioral representations from hourly POI visitation, while keeping the deployed representation image-only. Using the Houston Metropolitan Area as a case study area, we evaluated the performance of the BEACON framework on nine downstream tasks, including seven regression and two classification tasks against six baselines (raw coordinates, Space2Vec, SatCLIP, TESSERA, Clay and AlphaEarth), using frozen linear and MLP probes over five seeds. Under a linear probe, BEACON improves relative R^2 over AlphaEarth by up to 43% for obesity prevalence, 34% for poor mental health, and 22% for median household income, while remaining competitive in the prediction of physical and environmental variables. These findings highlight the value of augmenting geospatial foundation models with semantic and behavioral signals, extending their applicability from physical Earth observation to human-centered urban analytics.
1 Introduction
Geospatial foundation models provide transferable physical representations, but imagery alone may weakly encode urban function and human behavior. BEACON addresses this gap by aligning physical, semantic, and behavioral views while retaining image-only deployment.
- AlphaEarth offers globally consistent 10 m embeddings that transfer across urban and environmental prediction tasks.
- Imagery-only representations may confuse visually similar places with different social or economic roles, limiting human-centered urban analytics.
- POI text adds functional semantics, but it does not directly capture visitation intensity, timing, or temporal differences between visually similar places.
- Hourly and seasonal mobility profiles provide complementary behavioral information about commuting, leisure, routine services, and seasonal demand.
- BEACON aligns AlphaEarth embeddings, POI descriptions, and temporal visitation profiles during training, then produces enriched representations from AlphaEarth inputs alone.
2 Data and Methodology
BEACON combines AlphaEarth spatial views with POI text and mobility profiles in a shared contrastive space. It is evaluated across Houston-area tasks spanning human-centered, physical, environmental, land-cover, and land-use prediction.
- Study Area and Data: Figure 1 organizes BEACON around physical, semantic, and behavioral views aligned through tri-modal contrastive learning.
- Study Area and Data: Experiments use 2024 Houston metropolitan data, with AlphaEarth as the physical backbone and POI attributes plus hourly visitation as supervision.
- AlphaEarth Encoder: Each POI receives local and larger-context AlphaEarth views from average-pooled neighborhoods, which are projected and normalized for shared-space alignment.
- POI Description Encoder: POI descriptions combine sub-category, category, name, and NAICS code, then a pretrained text encoder and projector produce semantic representations.
- Mobility Pattern Encoder: Mobility representations concatenate normalized hourly and monthly patterns with four summary descriptors, including visit volume, entropy, weekday–weekend gap, and monthly variation.
- Tri-Modal Contrastive Alignment: The shared contrastive objective is trained with four alignment terms and evaluated using frozen linear and single-hidden-layer MLP probes across benchmark tasks and repeated splits.
3 Results
BEACON shows stable tri-modal pretraining and specializes in human-centered prediction while remaining competitive on physical and environmental tasks. Its largest gains occur for human-related outcomes, whereas physical-task performance remains close to AlphaEarth.
- Pretraining Dynamics and Cross-Modal Alignment: The tri-modal objective converges stably with little evidence of overfitting, with mobility providing the strongest human-centered supervisory signal.Alignment is driven primarily by the intramodal AEₛ–AE𝑐 and cross-modal AEₛ–Mobility terms; text-based alignments contribute more modestly.
- Downstream Prediction Performance: Across nine downstream tasks, BEACON specializes in human-centered prediction while remaining competitive on physical and environmental benchmarks.This pattern is reported across seven regression and two classification tasks.
- Downstream Prediction Performance: +43% relative R^2 over AlphaEarth for obesity prevalence, +34% for poor mental health, and +22% for median household income.BEACON achieves the best performance among all representations for the four human-related regression targets.
- Downstream Prediction Performance: BEACON remains nearly identical to AlphaEarth on physical and environmental tasks, while Clay leads daytime and nighttime land-surface temperature and PM2.5.PM2.5 improves from 0.845 to 0.856 (+1.3%), while LST performance remains essentially unchanged.
- Downstream Prediction Performance: For classification, AlphaEarth leads land-cover Macro-F1 at 0.584, whereas BEACON leads functional land use with Macro-F1 0.471 versus 0.456.The land-cover/land-use contrast separates physical surface morphology from human-defined structure; similar patterns hold under nonlinear MLP probes.
4 Conclusion and Future Work
BEACON enriches AlphaEarth with semantic and behavioral urban context while retaining image-only deployment, improving human-centered prediction across nine tasks while largely preserving physical and environmental information. The findings support treating physical appearance and urban function as distinct representational dimensions and motivate broader evaluation across cities, time periods, and mobility sources.
- BEACON enriches AlphaEarth embeddings with semantic and behavioral urban context while retaining an image-only representation at deployment.
- Across nine downstream tasks, BEACON substantially improves human-centered prediction while largely preserving AlphaEarth's physical and environmental information.
- The findings suggest that physical appearance and urban function constitute distinct representational dimensions of geographic space.
- Future work should evaluate generality across cities, time periods, and mobility data sources and explore behavioral alignment during foundation-model pretraining.