PaperlayerView exact revision on arXiv

Source-linked AI summary

Planetary Feature Fields are Scalable Earth Representations

Arjun Rao, Sebastian Loeschcke, Anthony Fuller, Isaac Corley, Nico Lang, Evan Shelhamer

arXiv:2609.37784v1cs.CVcs.LG

TL;DR

Earth observation products overlap in location and time but are stored and updated independently, making compact representations that preserve spatial and temporal detail desirable. PFFs model products jointly with shared hybrid fields and product-specific decoders; at high compression, they retain downstream performance while enabling fast feature access.

  • Problem

    Earth observation products cover overlapping locations and times but are independently produced, stored, and updated, while compact representations must preserve spatial and temporal detail.

    Source: §1 · p. 2

  • Method

    PFFs use spatially local fields with shared coordinate-indexed representations across space and time and small product-specific MLP decoders.

    Source: §2.1 · p. 4

  • Results

    At approximately 1800× compression, reconstructed features retain approximately 90% of downstream task performance, and PFFs retrieve features an order of magnitude faster than current APIs or cloud systems.

    Source: §1 · p. 3

  • Takeaways & Limitations

    PFFs support adding timesteps and products while preserving existing outputs, and enable fast large-scale mapping at low cost.

    Source: §1 · p. 3, §6 · p. 10

    PFFs are lossy, fixed-budget representations without variable-rate compression, and smoothing can remove fine structures needed for high-resolution mapping.

    Source: §6 · p. 9

Abstract

from arXiv · show

Satellite observations, precomputed embeddings, and map products describe the same evolving Earth, yet are stored as independent, petabyte-scale data products. Their continued growth calls for compact representations of multiple products while preserving spatial and temporal detail. We introduce Planetary Feature Fields (PFFs), which exploit redundancy across data products by modeling them jointly as continuous functions of space and time at planetary scale. PFFs are spatially local explicit-implicit (hybrid) neural fields. Each field shares a factored feature volume---a decomposition of an explicit 3D grid with smaller factors---across products, while lightweight implicit decoders reconstruct individual products across multiple timesteps. PFFs reconstruct EO products over space and time more accurately than single-product fields at matched compression rates. At $1800\times$ compression relative to the uncompressed source data, reconstructed features retain approximately $90\%$ or more of the performance achieved with the original features on pixel-level segmentation, change detection, and patch-level classification tasks. PFFs can add new timesteps by extending their factored feature volumes and add new products by attaching new decoders, while leaving existing outputs unchanged. PFFs reduce end-to-end feature access latency by an order of magnitude relative to evaluated API and cloud-storage pipelines.

1 INTRODUCTION

Planetary Feature Fields (PFFs) jointly represent overlapping Earth observation products while preserving spatial and temporal detail. They combine shared representations with product-specific decoders and support extensions to new timesteps and products.

1 INTRODUCTION

  • Precomputed embeddings preserve features at sampled locations and times without rerunning source models, but require substantial storage and transfer; as temporal coverage grows, costs rise roughly linearly because each timestep becomes another global array.One year of global 10 m Earth embeddings is estimated to require 96 TB for a 64-dimensional int8 field and 6.1 PB for a 1024-dimensional float32 field.Source: §1 · p. 2
  • PFFs are spatially local hybrid neural fields with a shared factored feature volume and lightweight product-specific decoders.The shared volume uses learned lines or planes, while decoders reconstruct products with differing dimensions, resolutions, and statistics.Source: §1 · pp. 2–3
  • New timesteps can be added by training small temporal extensions while freezing the rest of the PFF, and new products by fitting decoders against the frozen shared field.The figure reports a 0.2% parameter addition for new timesteps; adding a year takes 2% of joint-pretraining time and reaches 92% of its new-year reconstruction score while preserving earlier outputs bit-exactly.Source: Figure 3 · p. 3, §1 · p. 3
  • The PFF setup covers 14 Earth observation products across 2017–2025, including five raw observations, two embeddings, and seven map products.The product mix includes satellite observations, precomputed embeddings, and map products.Source: §1 · p. 3
  • Regional fields can be extended independently, allowing products with limited geographic coverage to be incorporated where observations exist without global retraining.The approach also supports discarding deprecated products.Source: §1 · p. 3, Figure 3 · p. 3
  • At approximately 1800× compression, PFF features retain approximately 90% of downstream task performance and retrieve features an order of magnitude faster than current APIs or cloud systems.The reported tasks include patch-level classification, semantic segmentation, and change detection.Source: §1 · p. 3

2 PLANETARY FEATURE FIELDS

PFFs represent Earth products with spatially local hybrid neural fields that share factored features across products and use product-specific decoders. Their training accommodates missing observations, and their factors or decoders can be extended to add timesteps or products.

2 PLANETARY FEATURE FIELDS

  • One shared PFF beats compression rate-matched single-product neural fields on EO product reconstruction across 100 global PFFBench tiles.The figure also compares rate-distortion curves on a 10-tile subset.Source: Figure 4 · p. 4

2.1 CONSTRUCTING PFFS

  • PFFs use spatially local hybrid fields, with a shared explicit representation and small product-specific MLPs that map it to each product.The shared representation stores space-time structure in learned factors rather than one dense grid, allowing spatial and temporal grid sizes to be chosen independently.Source: §2.1 · p. 4

2.2 OPTIMIZING PFFS

  • PFFs train jointly across products with arbitrary missing timesteps, channels, spatial regions, or entire products, masking unavailable values.Coarse observations are nearest-neighbor mapped to a common 10 m grid, preserving source-cell and nodata values rather than interpolating targets.Source: §2.2 · p. 5, Figure 5 · p. 5
  • Training uses a fixed-capacity GPU buffer of sampled spatial block–year pairs and partial COG reads to bound memory use.The median full regional product stack occupies 227.5 GB in memory.Source: §2.2 · p. 5

2.3 CONTINUALLY LEARNING PFFS

  • To add a timestep, PFFs append temporal slices, freeze existing parameters, and train only the new slices, preserving earlier outputs bit-exactly.The slices are initialized from the most recent timestep; deleting an existing slice changes interpolation anchors and does not preserve bit-exact outputs.Source: §2.3 · pp. 5–6
  • To add a product, PFFs train a new decoder on available pixels while freezing the shared field and existing decoders, leaving prior products bit-exact.The evaluated new decoder contains 1.25M parameters, approximately 2% of a canonical PFF.Source: §2.3 · p. 6

3 EXPERIMENTS

The experiments compare PFF reconstruction and downstream performance retention with compression rate-matched single-product neural fields. They evaluate reconstruction across 100 PFFBench tiles and test retention on pixel-based, temporal, and patch-level tasks.

Baselines

  • Baselines include single-product MLP, SIREN, Instant-NGP, K-Planes, and TensoRF(VM) fields, matched to one PFF’s combined parameter storage per tile.The main experiments use uniform parameter allocation across products because bit-proportional allocation improves embedding reconstruction but lowers fidelity on lower-dimensional sensor and map products, reducing mean score.Source: §3 · p. 6

Reconstruction

  • PFF reconstruction is evaluated on 100 tiles containing approximately 6.7 billion spatial locations and 16 TB of source data.Products are scored on their native grids and observed years.Source: §3 · p. 6

Downstream task retention

  • Downstream retention is the reconstructed-feature score divided by the original-feature score, reported as a percentage across dense prediction, change detection, and patch-level classification tasks.Task-specific linear classifiers are trained on original or reconstructed features.Source: §3 · p. 7

4 RESULTS

PFFs improve reconstruction through cross-product sharing and retain strong downstream performance at high compression. They also support continual extension without forgetting and faster feature materialization than evaluated data-access pipelines.

4 RESULTS

  • Shared VM and K-Planes factors reduce mean normalized distortion by approximately 20% at matched storage, while shared hash-grid and coordinate-network controls reconstruct less accurately than independent fits.The comparisons indicate that factorization, not merely a shared encoder with product-specific decoders, supports effective joint encoding.Source: §4 · pp. 7–8
  • Across three regions, PFF-Append reaches approximately 92% of joint-pretraining reconstruction score in 2% of training time while preserving earlier outputs bit-exactly.Finetuning the PFF or product decoders achieves higher new-timestep scores but exhibits catastrophic forgetting; PFF-K-Planes also learns the new year while preserving earlier outputs.Source: §4 · p. 8
  • 17.9× faster than Earth Engine for AE and 15.5× faster than full-tile downloads for TE, PFFs materialize one million embeddings including checkpoint download and model loading.Figure 7 compares the time to materialize one million AE and TE embeddings; most initial latency comes from checkpoint downloads and inference-session construction.Source: Figure 7 · p. 8, §4 · p. 8
  • Training only new Galileo decoders on 20% of 2019 target pixels matches source performance at 42.2% versus 42.1% mIoU while preserving existing outputs.Finetuning the shared field lowers Galileo mIoU to 37.2% and reduces AlphaEarth and TESSERA relative mIoU by 10.8 and 15.9 percentage points.Source: §4 · p. 8
  • 18–44 and 20–34 percentage-point retention gains over the strongest single-product baseline occur on PASTIS crop segmentation and FTW Brazil field-instance recall, respectively.On EuroMineNet, PFF-VM retains 97–103% of original AE and TE performance across AUPRC and F1 at the true change rate; both variants outperform single-product baselines on mining boundary F1.Source: §4 · p. 8
  • On EuroMineNet, PFF-VM retains 97–103% of original AE and TE performance across AUPRC and F1 at the true change rate; PFF-K-Planes retains 76–86%.On mining-footprint mapping, both variants outperform single-product baselines on boundary F1 across all five products despite similar overall pixel-level F1; patch classification largely matches source performance on BEN-v2.Source: §4 · p. 8

5 RELATED WORK

PFFs are local neural-field representations for jointly storing diverse EO products, distinct from global location encoders and single-product codecs. Their continual updates freeze existing parameters, preserving prior outputs bit-exactly.

5 RELATED WORK

  • PFFs place observations, embeddings, and map products in shared regional factors, testing whether joint representation improves reconstruction over separate fields at equal compression rates.Related neural-field work includes feature-field methods, scene-partitioned fields, and datasets treated as collections of neural fields.Source: §5 · p. 9
  • PFFs add temporal slices or product decoders while freezing existing parameters, preserving earlier outputs bit-exactly rather than updating parameters with replay or isolation.PFFs also reuse learned regional structure as new observations arrive without refitting historical data.Source: §5 · p. 9
  • Unlike global location encoders, PFFs use spatially local fields as compact substitutes for dense space–time-indexed EO products, with queries routed to the regional field containing the location.Unlike TerraCodec, PFFs share an explicit representation across multiple products; their aggregate compression ratio is not directly comparable to a codec for one product.Source: §5 · p. 9

6 LIMITATIONS AND CONCLUSION

PFF compression is lossy and fixed-rate, with downstream effects that depend on task. The conclusion reports improved reconstruction and downstream results, continual extension, and faster feature materialization across diverse EO products.

6 LIMITATIONS AND CONCLUSION

  • Each PFF uses a fixed storage budget and does not support variable-rate compression; its reconstructions are lossy and downstream effects depend on the task.Smoothing may preserve patch-level predictions while removing fine structures needed for high-resolution mapping, and may also reduce noise; the experiments do not isolate that effect from cross-product supervision.Source: §6 · p. 9
  • PFFs compress 14 diverse EO products through a shared factored feature volume, with better source reconstruction than compression-rate-matched single-product neural fields.The conclusion also reports significantly better performance than other local fields on several downstream tasks.Source: §6 · p. 10
  • PFFs support continual learning for new timesteps and products and materialize features an order of magnitude faster than currently used cloud or API-based methods.The conclusion connects this faster materialization to large-scale mapping efforts at low cost.Source: §6 · p. 10

A ADDITIONAL LATENCY EXPERIMENTS

The additional latency experiments examine large-scale embedding retrieval, memory capacity, and resident-model throughput. The supplied table captions describe a pooled feature-probing evaluation and reconstruction comparisons, without reporting cell outcomes.

A ADDITIONAL LATENCY EXPERIMENTS

  • The study measures time to materialize one million AlphaEarth and TESSERA embeddings relative to public retrieval methods, and examines memory capacity for serving multiple PFFs and resident-model throughput across hardware.These evaluations are identified as Figure 7, Table 7, and Figure 9, respectively.Source: A ADDITIONAL LATENCY EXPERIMENTS · p. 18
  • Table 4 reports PASTIS test mIoU from a pooled linear probe across 17 regional fields, comparing stacks of observations, embeddings, maps, or all 14 products.It uses identical evaluation pixels and train-only standardization; the caption marks the best and second-best merging result per architecture.Source: Table 4 · p. 19
  • Table 5 compares shared PFFs with rate-matched single-product neural fields across 100 globally distributed PFFBENCH tiles.The caption distinguishes uniform per-product and bit-proportional budgets and reports mean scores with one standard error across tiles.Source: Table 5 · p. 19

PALSAR

PFF serving is evaluated against data-access pipelines and across hardware, with measurements distinguishing end-to-end retrieval from resident-model throughput.

PALSAR

  • End-to-end timing compares PFFs with API and COG retrieval using identical 106-query traces, empty local caches, and a shared virtual machine.Measurements include initialization, acquisition, decoding or inference, and required memory transfers; remote caches are uncontrolled.Source: PALSAR · pp. 19–20, Figure 8 · p. 19
  • 6k embeddings/second versus the Earth Engine API’s approximately 1800 embeddings/second: PFFs are faster even on a 2-core, 8 GB RAM CPU machine.The benchmark serves 106 AlphaEarth embeddings across approximately 100 fully valid global regions.Source: Table 7 · p. 20
  • PFF throughput ranges from 29k to 51k embeddings per second on CPUs and reaches 9.70 million on an H100, with model loading and coordinate preparation excluded.The reported H100 rate corresponds to 1.03 milliseconds per squared kilometer under an assumption of parallel compute.Source: Figure 9 · p. 20, PALSAR · p. 21

B PFFBENCH

PFFBENCH uses a geographically and climatically stratified tile selection that prioritizes feature diversity, yielding a benchmark weighted toward higher-entropy content.

B PFFBENCH

  • The main reconstruction evaluation uses 100 tiles from a 1,000-tile global benchmark, selected for multi-year availability, geographic diversity, and feature-value variation.Eligibility filters retain 20,964 candidates before selection.Source: B PFFBENCH · pp. 21–22
  • Selection quotas are allocated across continent and climate groups, with selected tile centers at least 100 km apart.Quotas scale with the square root of each group’s candidate count, balancing smaller groups against larger ones.Source: B PFFBENCH · p. 22
  • PFFBENCH’s mean entropy is 3.253 nats, versus 2.804 for constrained random selection (Cohen’s d = 1.15).The selected benchmark tiles are higher-entropy than the random selections, rather than an estimate of average reconstruction performance over global land.Source: Figure 11 · p. 22, B PFFBENCH · p. 22
  • Within groups, tiles are ranked by mean per-band histogram entropy on valid pixels from the 2019 80 m overview.The score is a low-cost feature-diversity measure and correlates with lossless zlib bytes per valid pixel (Spearman ρ = 0.90).Source: B PFFBENCH · p. 22

C.1 PFF TRAINING DETAILS

PFF training jointly fits all 14 products using shared sampled coordinates, product-specific decoders and losses, and checkpoint selection by normalized distortion.

C.1 PFF TRAINING DETAILS

  • Each training iteration samples 1,048,576 valid coordinates from a fixed GPU buffer and uses the same coordinates across all products, masking missing targets independently.The block–year list is traversed without replacement before reshuffling, while pixels can recur as long as their block remains resident.Source: §C.1 · p. 23
  • Each product has a two-hidden-layer ReLU MLP decoder, and the equally weighted product losses use cosine distortion for AlphaEarth and standardized mean-squared error for other products.Decoder width is 1024 for AlphaEarth and TESSERA and 256 for the remaining products.Source: §C.1 · p. 23
  • Training uses Adam with mixed precision; K-Planes additionally applies spatial total variation, temporal smoothness, and temporal L1 regularization.Learning rates warm up for 500 iterations and then cosine-decay to 0.1 of their initial value.Source: §C.1 · p. 23
  • Each field is trained for 20,000 iterations, retaining the checkpoint with the lowest mean normalized distortion across products among evaluations every 1,000 iterations.Checkpoint evaluation uses 1 − R2 for Euclidean-valued products and 1 − skill for AlphaEarth.Source: §C.1 · p. 23

C.2 DOWNSTREAM EVALUATION

Downstream evaluations compare frozen original and reconstructed features under matched labels, splits, sampling, and probe configurations, with task-specific metrics and protocols.

C.2 DOWNSTREAM EVALUATION

  • Each benchmark extracts one frozen feature vector per labeled pixel from the original or reconstructed product at the same location and year, using identical evaluation procedures.Probes receive only per-pixel features, without neighboring features or spatial context.Source: §C.2 · p. 23
  • SwissCrop25 tests transfer across years with held-out years 2021–2025, but its split is not causal forecasting because training may include observations after the test year.The study reports overall accuracy and mean IoU over 65 crop classes, averaged across five test years.Source: §C.2 · pp. 23–24
  • EuroMineNet reports overall footprint F1 across reconstructed site scenes and boundary F1 within a three-pixel band around ground-truth boundaries.Scores are accumulated across years and averaged across test sites, weighted by scored pixels.Source: §C.2 · p. 24
  • Change detection uses consecutive-year features and their difference; because only 0.52% of evaluated pixel pairs change, it reports AUPRC and F1@r.F1@r labels the k highest-scoring predictions as changed when k pixel pairs change in the ground truth.Source: §C.2 · p. 24
  • Fields of the World uses object recall in Brazil, where background is unlabeled, and pixel IoU in South Africa, where field and background pixels are labeled.A predicted component recovers a Brazil ground-truth field when their overlap exceeds IoU 0.5.Source: §C.2 · p. 24
  • In the Pacific Palisades example, PFF-Append captures the 2025 burn-scar change while leaving 2024 unchanged, whereas fine-tuning exhibits catastrophic forgetting on 2024.PFF-Append matches the pretrained result at approximately 2% of its training time.Source: Figure 14 · p. 26

Related papers

Loading 2609.37784v1…