LoCA: Enabling Multi-Modality in Geospatial Location Encoders

1 INTRODUCTION

• Image-based embedding products are tied to release-time observations and fixed grids, whereas a location descriptor should be queryable from latitude and longitude alone.Different images can give the same location different vectors, so image-dependent representations do not by themselves provide a coordinate-indexed function.Source: §1 · p. 2
• Any Lipschitz location encoder maps geographic coordinates to an embedding of intrinsic dimension at most 2, forcing aligned contrastive loss above its unconstrained minimum.Joint training also empirically collapses covariance to low rank, limiting downstream concept separability.Source: §1 · p. 2
• Frozen pretrained image encoders act as covariance regularizers, giving location encoders high-rank targets to fit despite their low-dimensional geometry.The location encoder tracks the target cloud's empirical spectrum.Source: §1 · pp. 2–3
• LoCA jointly trains Sentinel-1 and Sentinel-2 encoders, then freezes them while fitting location and metadata encoders as adapters to a co-aligned, high-rank target.The multimodal extension requires co-alignment because adapters cannot align independently pretrained encoders, and high effective rank because the location encoder inherits its target's spectrum.Source: Figure 1 · p. 2, §1 · p. 3
• LoCA achieves the best performance on more tasks than state-of-the-art competitors and beats each examined location encoder in head-to-head comparisons.The paper frames the work as a principled framework for identifying location encoders' failure modes and extending them to multiple modalities.Source: §1 · p. 3

2 RELATED WORK

• Geospatial foundation models use masked-image, hybrid, or generative objectives, while released embedding products are observation snapshots rather than functions of coordinates.Examples include AlphaEarth, Copernicus-FM, and TESSERA, which provide precomputed embeddings tied to acquisition windows.Source: Geospatial Foundation Models (GFMs) &… · pp. 3–4
• Coordinate-based encoders align locations to image embeddings, while existing multimodal extensions use separate losses that can create conflicting gradients and grow quadratically with modality count.The paper addresses these costs by generalizing the SatCLIP framework through a single multi-view objective.Source: Geospatial Foundation Models (GFMs) &… · p. 4
• Multi-view contrastive learning includes pairwise-loss aggregations, independent view-combination tasks, alternative aggregations, and optimal transport; MV-DHEL is selected for its no-collapse optimum.Its optimum scales with the number of views without dimensional collapse.Source: Geospatial Foundation Models (GFMs) &… · p. 4

3 REVISITING THE INNER WORKINGS OF LOCATION ENCODERS

• The section frames location encoders' limitations through theoretical bounds on contrastive optima and representation dimensionality, then examines existing encoders as coordinate-indexed lookup tables.It separates task formulation, theoretical analysis, and the regularizer interpretation.Source: §3 · p. 4

3.1 SETUP AND NOTATION

• The setup places coordinates on a k-sphere with an atomless probability measure and maps both locations and paired images into the same sphere of dimension d−1, where d−1 exceeds k.For SatCLIP, k = 2 and d−1 = 255.Source: §3.1 · p. 4
• Each image is paired with a unique location, and the location and image encoders are evaluated under the population NT-Xent loss.The image domain is otherwise an arbitrary measurable space.Source: §3.1 · p. 4

2 EI log Ec′

• The aligned-uniform embedding distribution attains the stated contrastive optimum, while Lipschitz location encoders are restricted to a lower-dimensional image.The population NT-Xent loss has a positive-pair term and two terms that push different-pair embeddings apart.Source: §2 · p. 4, §3.2 · p. 5
• Table 1 reports effective rank and its ratio to total representation dimensionality; higher values mean more directions are used, and its anchor denotes the pretrained target.Colors distinguish Sentinel-2, RGB, and joint Sentinel-1 and Sentinel-2 training.Source: Table 1 · p. 5

3.2 LOCATION ENCODERS CONSTRAIN CONTRASTIVE LEARNING AWAY FROM ITS OPTIMUM

• A Lipschitz location encoder's image has intrinsic dimension at most k, and its distribution has zero overlap with the uniform sphere target, forcing aligned contrastive loss strictly above the unconstrained minimum.The strict gap extends across the bandwidth-L SatCLIP family and aligned image encoders.Source: §3.2 · p. 5

3.3 GEOMETRY, VARIABILITY COLLAPSE, AND THE ROLE OF REGULARIZATION

• The dimensionality bound constrains embedding geometry, not covariance rank; nevertheless, joint training empirically settles on low covariance rank.The 255-dimensional space is used along only a small number of linearly independent directions.Source: §3.3 · pp. 5–6
• Freezing the image encoder turns location fitting into matching a fixed point cloud, allowing the sample covariance to track the target's spectrum while the geometric image remains k-dimensional.The encoder can fit high-rank labels by varying along its low-dimensional surface.Source: §3.3 · p. 6
• SatCLIP and GeoCLIP use contrastive loss as a proxy for fitting a coordinate-indexed lookup table whose pretrained image target regularizes the location encoder.The target supplies a high-rank cloud for the encoder to fit.Source: §3.3 · p. 6

4.1 A LOCATION ENCODER CANNOT ALIGN TO TWO TARGETS

• Independently pretrained sensor encoders provide incompatible targets, and trainable adapters cannot create co-alignment absent from the targets themselves.A single location function can satisfy both modalities only when paired representations already agree up to a transformation.Source: §4.1 · p. 6

4.2 REGULARIZER EMBEDDINGS SHOULD HAVE HIGH RANK

• A location encoder inherits its target’s number of linearly independent directions, so low-rank targets reproduce collapse in the location representation.The paper identifies high effective rank as a requirement for regularizer embeddings.Source: §4.2 · p. 6
• External high-rank anchors can add potentially irrelevant variation, while optimization that enforces high rank creates a label cloud for the target modalities.The paper notes that external anchors may introduce axes reflecting exogenous features, such as RGB natural imagery.Source: §4.2 · p. 7

4.3 LOCA: LOCATION CONTRASTIVE ALIGNMENT

• LoCA’s Stage 1 jointly trains Sentinel-1 and Sentinel-2 encoders with DHEL, yielding paired representations that are co-aligned and use a large fraction of available dimensions.Joint contrastive training pulls paired inputs to the same embedding by construction.Source: §4.3 · p. 7
• In Stage 2, frozen image encoders provide the target while location and metadata encoders and modality-specific projections train with MV-DHEL.The frozen targets retain co-alignment and high effective rank; the location encoder’s empirical spectrum reaches the projection-head dimension despite its 2-dimensional manifold.Source: §4.3 · p. 7

5.1 DOWNSTREAM PERFORMANCE OF LOCATION ENCODERS

• 8 of 17 tasks rank LoCA first and 11 place it among the top two; it beats competitors in 68.8% of pairwise comparisons.Its leading task groups include land cover and biodiversity, while results vary across geophysical, socio-economic, and environmental tasks.Source: §5.1 · p. 8

5.2 LOCA INCREASES EMBEDDINGS VARIABILITY

• Jointly pretrained encoders show cos = 0.94 and Lalign = 0.12, while independently pretrained encoders show Lalign = 1.00 and cos = 0.50.Replacing Stage 1 with independently pretrained encoders collapses benchmark performance; the cited alignment measurements compare the two encoder pairings.Source: §5.2 · p. 9, §5.3 · p. 9
• LoCA’s embeddings populate 158 of 255 dimensions (62%), versus 21% for Copernicus-FM and 35% for SatCLIP; joint training from scratch has effective rank 8.The paper cautions that GT-Loc and GeoCLIP’s effective rank includes language information, while AlphaEarth’s was unavailable.Source: §5.2 · p. 9

5.3 ABLATIONS ON LOCA COMPONENTS

• Stage 2 improves every supervised encoder’s linear-probe performance, and concatenating adapted embeddings raises mean F1 from 80.9% to 84.3%.Removing metadata lowers mean scores by 3.5% in Geophysical & Socio-economic tasks and 1.3% in Environmental tasks, with small effects elsewhere.Source: §5.3 · p. 9, Table 4 · p. 9

6 CONCLUSION

• LoCA addresses theoretical limits of location encoders, identifies co-alignment and high effective rank as requirements, and supports scaling to any number of modalities.Source: §6 · p. 9
• LoCA outperforms competing models across critical EO tasks while substantially improving embedding space utilization.Source: §6 · p. 10
• LoCA has no temporal dimension, uses only Sentinel-1 and Sentinel-2 in Stage 1, and coordinate-based benchmarks are sensitive to spatial autocorrelation.Seasonal and shorter-term dynamics are captured only indirectly; blocked or leave-region-out evaluations are identified as future research directions.Source: §6 · p. 10

AI USE STATEMENT

• Generative AI assisted with figure creation, engineering brainstorming, literature identification, table creation, and minimal writing polish; authors constructed the proofs and reviewed AI-assisted work.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

• The reproducibility statement specifies downstream evaluation seeds and reports that LoCA code, pretrained weights, and the extended S2-100K training set are planned for release.SatBird uses 3 seeds; all other tasks use 10.Source: REPRODUCIBILITY STATEMENT · p. 10

B.1 PRELIMINARIES

• The appendix uses Hausdorff dimension because Lipschitz scaling has a clean classical statement, and this dimension agrees with other intrinsic-dimension notions on smooth supports.Source: §B.1 · p. 17

B.2 ALIGNED-PAIR CHARACTERIZATION OF THE FREE OPTIMUM

• For aligned encoder pairs with a shared marginal, the asymptotic InfoNCE objective reduces to a single-encoder expression uniquely minimized by the uniform distribution on the embedding sphere.Source: §B.2 · p. 18
• The strict loss-gap conclusion applies to aligned pairs; non-aligned pairs may in principle achieve values below the aligned-pair optimum.The dimension and singular-marginal conclusions do not require alignment, while the strict per-pair loss gap does.Source: §B.2 · p. 19

B.3 PROOF OF THEOREM 1

• A measure-preserving Borel isomorphism from the location domain to the sphere constructs an aligned pair attaining the aligned-uniform value, but it cannot be continuous when k < d −1.The construction confirms that the target value is attainable by a measurable pair, while the dimension gap rules out continuity of that bijection.Source: §B.3 · p. 20
• Every aligned pair with a Lipschitz location encoder has loss strictly above the free infimum, because its embedding marginal differs from the uniquely optimal uniform marginal.The proof bounds the image dimension, establishes mutual singularity with the uniform measure, then applies the unique-minimizer result.Source: §B.3 · pp. 19–21
• Under alignment, the image encoder’s essential-range measure inherits the location encoder’s dimension ceiling and singular marginal, apart from values on a pI-null set.The conclusion concerns the pushforward measure, not the image encoder’s literal range outside the support of pI.Source: §B.3 · p. 21

B.4 SATCLIP FUNCTION CLASS AND UNIFORM LIPSCHITZ CONSTANT

• SatCLIP encoders have a uniform finite Lipschitz constant when the parameter set is compact and outputs stay at least ε from zero before normalization.The encoder composes spherical-harmonic features, a Siren network, and sphere normalization; the annular normalization bound is (2/ε)-Lipschitz.Source: §B.4 · pp. 21–22

B.5 PROOF OF COROLLARY 1

• The SatCLIP family is compact and its reduced loss is continuous, so the minimum is attained and remains strictly above the free optimum.Compactness follows from equicontinuity and closedness; continuity is established through the encoder pushforward and the loss functional.Source: §B.5 · pp. 22–23

C.1 METADATA

• The metadata encoder receives 14 static environmental and anthropogenic scalars per coordinate, with temporal values averaged over 2021–2023.The inputs include terrain, road distance, population density, and land-cover fractions; the archive uses a global 0.01° grid, roughly 1 km at the equator.Source: §C.1 · pp. 24–25

C.2 SENTINEL-2/SENTINEL-1 PAIRED IMAGE DATASET

• The dataset contains 100,000 spatially aligned Sentinel-2 and Sentinel-1 image pairs, with each S2 patch matched to the closest S1 acquisition over the same location.S2 patches are random 256 × 256 crops; S1 candidates are queried within ±6 days of the S2 timestamp and ordered by temporal distance.Source: §C.2 · p. 25
• Pairs are retained only after image-quality checks, rejecting patches when all-band missing or zero pixels exceed 10%.S1 scenes must contain both VV and VH assets, and the stacked patch uses the same empty/zero thresholds as S2.Source: §C.2 · p. 25

D LOCA IMPLEMENTATION DETAILS

• The aligned pretraining sources include 648 million metadata points with 14 features and paired Sentinel imagery, including 256 × 256 S2 patches and co-registered S1 patches.The S1/S2 image pairs are matched to metadata points by footprint overlap; on average, about five metadata points are matched per image at the stated threshold.Source: D LOCA IMPLEMENTATION DETAILS · pp. 25–26
• Stage 1 trains both image backbones end-to-end; Stage 2 freezes them and trains fresh location and metadata encoders with their MLP heads.Stage-1 projection heads are discarded at the transition, and image embeddings are precomputed and cached for Stage 2.Source: D LOCA IMPLEMENTATION DETAILS · p. 26
• The location encoder uses degree-40 spherical harmonics followed by a two-hidden-layer SIREN, while metadata passes through a three-layer MLP.The location SIREN outputs 256 dimensions; the metadata MLP maps 14 inputs through three 256-unit layers.Source: D LOCA IMPLEMENTATION DETAILS · pp. 26–27
• MV-DHEL aligns all modality pairs simultaneously and computes uniformity per view, reducing to DHEL for two modalities and using its full formulation for four.The reported temperatures are τ = 0.2 in Stage 1 and τ = 0.3 in Stage 2.Source: D LOCA IMPLEMENTATION DETAILS · p. 27

E DOWNSTREAM TASKS EVALUATION

• The downstream benchmark covers regression, classification, and ranking, including biodiversity, land cover, climate ordering, and wildfire forecasting, against five named encoders.Comparators are SatCLIP, GeoCLIP, GT-LOC, AlphaEarth Foundations, and Copernicus-FM.Source: E DOWNSTREAM TASKS EVALUATION · p. 28

• All tasks use latitude–longitude coordinates encoded by the location encoder and passed to a downstream predictor; Table 7 summarizes dataset sources, sizes, splits, metrics, and coverage.Source: §E.1 · p. 28, Table 7 · p. 29
• The Countries, Ecoregions, and Biomes datasets were revised as land-only because about 70% of their sampled locations fell over ocean, making ocean classification dominate overall accuracy.The revised datasets retain locations intersecting a global land mask.Source: §E.1 · p. 28
• Four added environmental datasets cover biodiversity, climate ordering, land-cover classification, and wildfire danger forecasting.The tasks use SatBird, SeasFire Cube, reBEN, and Mesogeos, respectively.Source: §E.1 · pp. 28–29

E.2 TRAINING DETAILS

• Evaluation uses fixed encoder embeddings with linear and MLP probes, plus LightGBM for climate ranking; AEF is excluded where embeddings could not be retrieved reliably.Task-specific metrics and inputs vary, and SatBird uses three random seeds because of its higher computational cost.Source: §E.2 · pp. 30–32

E.3.1 COMPARISON AMONG STATE-OF-THE-ART ENCODERS

• 68.8% win rate; LoCA alone beats every competitor on a majority of shared tasks, winning 12 of 17 against SatCLIP and 5 of 9 against AlphaEarth.Its losses concentrate on geophysical and climate targets; it is never outperformed on land-cover or ecological classification.Source: §E.3.1 · p. 32, Table 11 · p. 34

E.3.2 SUPERVISED CO-ALIGNED TARGETS

• MV-DHEL improves mean F1 for every modality family, with the full four-modality system rising from 80.9 to 84.3 and achieving the best score in every class.The largest reported mean-F1 gain is for S1, from 69.6 to 75.9.Source: §E.3.2 · p. 32, Table 12 · p. 34

E.3.3 ABLATING LOCA’S MULTI-VIEW COMPONENTS.

• Removing joint Stage 1 pretraining causes broad performance collapse, while metadata contributes selectively, improving the full model over the metadata-free variant on 9 of 17 tasks.The ablation reports chance-level climate ranking without Stage 1; metadata gains include air temperature, elevation, and climate-ranking tasks.Source: §E.3.3 · p. 34, Table 13 · p. 35

F.1 GEOSPATIAL FOUNDATION MODELS (GFMS) & EMBEDDINGS.

• Related geospatial foundation models use masked-image, cross-modal contrastive or reconstruction, and generative pretraining; released embedding products remain observation-based rather than location-based.Source: §F.1 · pp. 35–36

F.2 LOCATION ENCODERS.

• Prior multimodal location encoders use heterogeneous or pairwise losses that can conflict or grow quadratically with modality count; LoCA generalizes SatCLIP with one multi-view objective.Source: §F.2 · p. 36

F.3 MULTI-VIEW CONTRASTIVE LEARNING.

• MV-DHEL uses all view interactions and is reported to yield gains that scale with view count while avoiding dimensionality collapse.The paper draws on MV-DHEL to train multimodal location encoders.Source: §F.3 · p. 36

G LICENSES AND TERMS FOR EXISTING ASSETS

• The appendix lists external assets and their licenses or terms of use, and does not redistribute third-party materials unless their licenses permit it.Source: G LICENSES AND TERMS FOR EXISTING ASSETS · pp. 36–37, Table 14 · p. 38