Source-linked AI summary
NeurEO: dissecting Earth observation embeddings with computational neuroscience
Thijs L van der Plas, Jacob JW Bakermans, Ioannis N Athanasiadis
TL;DR
Geospatial embeddings compress rich Earth observation data, but their semantic meaning is unknown. The paper adapts computational-neuroscience methods to analyze AlphaEarth features independently and in relation to land cover and elevation. It reports directional tuning and increased residual dimensionality, while concluding that effective dimensionality converges below 10 for a diverse UK dataset.
Problem
Earth embeddings support downstream geospatial tasks, but their semantic meaning is unknown a priori.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The UK findings suggest that a more efficient embedding representation may be derived for localized applications.
Source: §4 · p. 4The proof-of-concept uses a small AlphaEarth dataset; more diverse data and other Earth embeddings are needed for more definitive and generalised findings.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Earth embeddings are compressed representations of rich Earth observation data that enable downstream models to tackle a wide range of complex geospatial tasks. Yet, this increase in performance comes at the cost of decreased interpretability, as the semantic meaning of Earth embeddings is unknown. To address this challenge, we borrow methods from the field of computational neuroscience, which aim to interpret activity in (biological) neurons. Here, we adapt and apply these methods to assign meaning to AlphaEarth embeddings. We analyse embedding features both as stand-alone data, equivalent to spontaneous neural activity, and in relation to land cover and elevation correlates, equivalent to stimulus-response neural activity. We demonstrate correlational and multi-scale structure in the embedding features. Embedding features can display highly specific and directional tuning to land cover and elevation, such as a `North vs South woodland edge' response. When we remove the main trends of combined land cover responses (like water versus land tuning), we discover fine-grained representational details with increased effective dimensionality. Together, our analyses start to unravel what these Earth embeddings represent, which is critical for developing trustworthy downstream applications.
1 Introduction
Geospatial foundation models provide compressed representations for downstream tasks, but the semantic meaning of their embedding features is unknown a priori. The paper adapts computational-neuroscience methods to interpret those features as spatial data, both independently and in relation to land cover and elevation.
1 Introduction
- Embedding features provide compressed representations for geospatial tasks, but their semantic meaning is unknown a priori.The embeddings aim to capture rich, multimodal Earth observation data.Source: §1 · p. 1
- The paper treats spatial embedding features like neural activity and adapts methods for analyzing activity both without correlates and in response to stimuli.Neural activity varies over time, whereas embedding features vary across latitude and longitude.Source: §1 · p. 1
- Figure 1 characterizes embeddings through feature correlations, effective ranges, example maps, and dimensionality as patch count increases.Source: Figure 1 · p. 2
2 Methods
The methods adapt neuroscience analyses to AlphaEarth features, first examining spatial statistics and then relating features to land-cover and elevation hypotheses. They also use regression and canonical correlation analysis to separate hypothesis-related reconstructions from residuals.
2 Methods
- The proof-of-principle dataset uses 100 UK locations and 128 × 128 pixel AlphaEarth, land-cover, and elevation patches; nine land-cover classes plus elevation form ten hypotheses.Locations were at least 30 km apart, and the land-cover values were averaged across 2024 observations.Source: §2 · p. 2
- Effective range estimates the inter-pixel distance at which values become statistically independent; RSA tests its feature-level similarity against pixel-value correlations using 10,000 permutations.Ranges were estimated per feature and patch; values above the 1,280 m patch size were excluded.Source: §2 · p. 2
- PCA estimates the effective dimensionality of 64 features across datasets with different numbers of patches to quantify redundancy within limited geographic scopes.For each patch count N, 20 subsamples were resampled; the Figure 1e analysis used 165 patches.Source: §2 · p. 3
- Tuning surfaces are computed by spatially aligning and averaging 21 × 21 pixel hypothesis windows weighted by the embedding value at each central pixel.Two-dimensional Gaussian fits provide tuning statistics across feature–hypothesis combinations.Source: §2 · p. 3
- CCA finds ten linear projections of embedding features that maximally correlate with projections of the hypotheses, then OLS fits those components to the features.The resulting reconstructions are subtracted from the features to obtain residuals.Source: §2 · p. 3
3 Results
The analyses reveal correlated and multiscale structure, specific directional tuning, and additional structure in residual features after land-cover and elevation trends are removed. CCA reconstructions explain more variance than hypothesis-only reconstructions, while residuals have greater effective dimensionality than the original embeddings.
3 Results
- After the main hypothesis trends are removed, group-level tuning analysis reveals fine-grained representational structure.Source: §3 · pp. 3–4
- Feature correlations form clusters, but their relationship with effective-range similarity is weak (ρ = 0.026, two-sided p = 0.082).Effective dimensionality increases with patch count, indicating that broader geographic sampling can decorrelate features.Source: §3 · pp. 3–4
- Feature 47 responds to water and distinguishes between Northern and Southern woodland edges; off-centre tuning-surface peaks identify directional coding.Feature selectivity to hypotheses varies with relative tuning strength.Source: §3 · p. 4
- On average, land-cover and elevation hypotheses explain R2 = 0.46 of embedding variance, while CCA explains R2 = 0.60; residuals have the greatest PCA dimensionality.The residuals also reveal new details after correlations with the hypotheses are removed.Source: §3 · p. 4, Figure 3 · p. 4
4 Discussion
The discussion cautions that the findings are proof-of-concept, while suggesting local Earth embeddings may contain redundancies and that removing dominant land-cover and elevation trends can reveal additional spatial variability.
4 Discussion
- The findings are based on a small, proof-of-concept AlphaEarth dataset; larger, more diverse data and other Earth embeddings are needed for more definitive and generalised conclusions.The authors specifically propose comparisons across embeddings trained with different architectures and loss functions.Source: §4 · p. 4
- For UK locations, effective dimensionality converges to < 10, and removing projections correlated with land cover and elevation increases residual-feature dimensionality, revealing previously obscured spatial variability.The authors suggest local applications may fine-tune embeddings by reprojecting, selecting, and assigning semantic meaning to features.Source: §4 · p. 4