Source-linked AI summary
Ground-to-Satellite Knowledge Distillation for Crop Classification under Temporal and Geographic Shift
Miro Miranda, Rocco Sedona, Valerio Marsocci, Gabriele Cavallaro, Claudia Paris
TL;DR
Crop mapping needs reference data, while ground photographs provide complementary crop information but are not consistently available at inference. This paper uses a photo teacher to distill supervision into satellite classifiers, finding consistent gains that can grow under temporal and geographic shift without changing satellite-only inference.
Problem
Ground photographs can provide crop-relevant information complementary to satellites, but their limited availability makes them impractical for scalable inference; the paper asks whether their supervision can improve satellite classifiers across shifts.
Source: §1 · pp. 1–2Method
Results
Frozen and from-scratch students gain 2.6–5.1 macro-F1 points in-domain; frozen TESSERA’s KD−CE margin reaches +5.5 under temporal transfer, and AlphaEarth’s Mediterranean gain is +3.7 versus +3.0 in-distribution.
Source: §4 · p. 4Takeaways & Limitations
Ground-view supervision can complement satellite embeddings and improve robustness, especially for fixed pretrained representations, while deployment remains satellite-only.
Source: §4 · p. 4, §5 · p. 5Geographic gains are clearest in the Mediterranean; Boreal improvements are unstable, illustrating how small holdouts can be noisy.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Continuous and up-to-date crop type mapping is essential for agricultural monitoring but is often hampered by limited reference data. The growing availability of ground-level field photographs offers a promising way to address this challenge. When combined with satellite time series, these images provide complementary perspectives on the same parcel, capturing crop characteristics from distinctly different perspectives. However, field photographs are not always available for each satellite pixel at inference time, limiting their potential in scalable crop type mapping. We investigate whether ground photographs can provide privileged supervision for crop classification built on pretrained satellite embeddings. A photo teacher based on frozen CLIP features supervises classifiers trained on AlphaEarth and TESSERA representations; photographs are used only during training. On a paired field photo and satellite imagery dataset with 17 crop classes, knowledge distillation consistently improves over satellite-only classification; for frozen embeddings the margin grows under temporal shift, reaching up to +5.5 macro-F1 points, and widens in geographic holdouts (+3.7 vs +3.0 in-distribution). These results show that ground-view supervision can complement satellite embeddings and improve their robustness without changing inference requirements.
1 Introduction
Ground photographs supply crop-relevant views complementary to satellite data, but their limited availability makes them unsuitable for routine scalable inference. The paper tests whether training-only photo supervision can improve satellite classifiers across temporal and geographic shifts.
1 Introduction
- Ground photographs reveal crop characteristics such as leaf shape, structure, and texture that are difficult to capture from satellites, but are rarely available on demand.The paper frames training-only knowledge distillation as a way to use this privileged information while retaining satellite-only inference.Source: §1 · p. 1
- Satellite signatures vary with climate, phenology, land management, and acquisition conditions, while ground photographs provide a complementary view of the same parcel.The paper investigates transferring this complementary semantic information to satellite classifiers, including when photographs are unavailable at inference.Source: §1 · p. 2
- Across controlled comparisons, distillation improves in-domain macro-F1 by up to 5.1 points on frozen TESSERA, with frozen-embedding margins growing under temporal shift and in the Mediterranean holdout.The comparisons cover frozen and from-scratch satellite representations, temporal transfer, AlphaEarth geographic holdouts, and partial TESSERA fine-tuning.Source: §1 · p. 2
2 Method
A frozen CLIP photo encoder feeds a trained teacher whose predictions and embeddings supervise satellite students. The students use several satellite-model branches, while matched CE controls isolate the effect of privileged supervision.
2 Method
- The photo teacher averages individually L2-normalized CLIP features across views into a 512-dimensional representation, then trains a projection and crop classifier per seed.The CLIP encoder is frozen; teacher targets are cached for student-training points.Source: §2 · p. 2
- Satellite students produce a shared embedding and crop logits, using frozen GFM embeddings, a from-scratch sequence transformer, or partially fine-tuned TESSERA.For TESSERA, the last two transformer blocks per modality and the fusion reducer are unfrozen; earlier layers remain frozen.Source: §2 · p. 2
- Figure 1 depicts the photo-teacher and satellite-student workflow alongside example satellite and field imagery and the 17 CLMS crop classes.Its overview also locates the selected LUCAS points and held-out ecoregions.Source: Figure 1 · p. 3
- The distillation objective combines class-label cross-entropy with teacher-logit KL divergence and teacher–student embedding cosine alignment.The CE control uses the same student and optimization, with both distillation weights set to zero, so the KD–CE contrast changes only privileged supervision.Source: §2 · pp. 2–3
3 Experimental setup
The experiments use paired LUCAS field photographs and satellite observations, with fixed train–validation–test splits and separate temporal and geographic transfer evaluations. Satellite-input processing differs between released TESSERA embeddings and the monthly-composite path used for fine-tuning.
3 Experimental setup
- The paired LUCAS 2018 dataset contains 53,885 labeled points with photographs and satellite time series across 17 crop classes, split 70/15/15 for training, validation, and testing.Satellite inputs include released 64-D AlphaEarth and TESSERA embeddings or 12 monthly composites of Sentinel-1 and Sentinel-2 over a 3 × 3 pixel patch.Source: §3 · p. 3
- Temporal transfer evaluates 2018-trained models on 36,428 LUCAS 2022 points at locations unseen during training, without adaptation.The 2018 class mapping and normalization are reused, and 2018 training and validation locations are excluded.Source: §3 · p. 3
- Geographic transfer holds out Mediterranean, Alpine, or Boreal ecoregions within 2018; the monthly-input TESSERA probe scores 36.2 ± 1.6 versus 43.6 ± 1.6 macro-F1 for released embeddings.Because the input paths differ, the study treats them as separate regimes rather than interpreting the −7.4-point difference as a fine-tuning effect; reported main experiments use five seeds.Source: §3 · p. 3
- Table 1 reports five-seed mean macro-F1 for matched CE and KD students trained on 2018, including 2022 transfer; Δ is the paired KD−CE difference and ΔT is the difference from the 69.7 teacher score.The caption identifies the 2022 survey as the transfer block and its locations as unseen.Source: Table 1 · p. 4
- Figure 2 summarizes paired KD−CE macro-F1 margins across branches under 2018→2022 transfer and AlphaEarth gains in held-out ecoregions.Its caption specifies ±1 SD error bars for paired margins.Source: Figure 2 · p. 4
4 Results and Discussion
Distillation improves in-domain results for frozen and from-scratch students, with larger gains for frozen representations under temporal transfer. Geographic gains are clearest in the Mediterranean, while the paper qualifies results from other holdouts and the fine-tuned branch.
4 Results and Discussion
- 2.6–5.1 macro-F1 points: KD improves in-domain results for frozen and from-scratch students, with the largest gain on frozen TESSERA.Partial TESSERA fine-tuning reaches 72.1 macro-F1 and up to +2.4 macro-F1 over the teacher, but its paired KD margin is +0.8 macro-F1 and is not a controlled estimate of fine-tuning alone.Source: §4 · p. 4
- Under temporal transfer, frozen KD−CE margins grow from +2.9 to +3.4 macro-F1 points for AlphaEarth and +5.1 to +5.5 for TESSERA; scratch margins shrink, and fine-tuned TESSERA stays below one point.In geographic holdout, the margin is +3.7 versus +3.0 in-distribution for Mediterranean; Alpine is +1.1, while Boreal is unstable (+0.5, SD 1.4, mixed signs).Source: §4 · p. 4
- The paper reports that ground-view KD helps most clearly with fixed satellite representations, requires photos only during training, and outperforms alternative cross-modal objectives such as contrastive learning.It cautions that Boreal results can be noisy and that transfer performance depends on the class, especially for frozen TESSERA.Source: §4 · p. 4
5 Conclusion
Ground photographs provide privileged supervision for satellite crop classifiers without being needed at inference, with benefits that persist or increase under temporal and geographic shift.
5 Conclusion
- Ground-view distillation improves satellite crop classification without requiring photographs at inference.The conclusion highlights this as an operational advantage for large-scale mapping, where ground observations are costly to collect.Source: §5 · p. 5
- Benefits are strongest for fixed pretrained satellite representations and persist or increase under temporal and geographic shift.For frozen embeddings, margins grow under temporal shift and in the Mediterranean holdout.Source: §5 · p. 5
- Ablations indicate that gains arise primarily from distilling the photo teacher’s predictive structure rather than cross-modal feature alignment.Source: §5 · p. 5
A Additional experimental details
The experimental details specify the model components and paired transfer controls, distinguish TESSERA evaluation regimes, and define the measures used to compare temporal stability.
A Additional experimental details
- The sequence-transformer branch encodes monthly spectral features with pre-normalized Transformer blocks and mean-pools across months.Layer normalization produces the final sequence representation.Source: A Additional experimental details · p. 6
- The photo teacher and satellite student use projection heads into a shared latent space, with the frozen teacher supplying embedding- and logit-level supervision.The satellite student concatenates its projected embedding with the original satellite features and combines cross-entropy with both distillation losses.Source: A Additional experimental details · p. 6
- Main KD runs use temperature T = 2, logit-KD weight 1.0, embedding-cosine weight 0.25, and paired student, optimizer, split, and seed controls.The matched CE control sets both distillation weights to zero while retaining the same student, optimization, and class-weighted CE term.Source: A Additional experimental details · p. 6
- Geographic experiments hold the ecoregion test set fixed while varying only the train/validation partition across seeds.The 2022 transfer control reuses the 2018 class map and scaler and excludes points used for 2018 training or validation.Source: A Additional experimental details · p. 6
- The frozen released-embedding and partially fine-tuned monthly-input TESSERA results are separate regimes, not a direct fine-tuning comparison.That comparison would require CE and KD with the encoder frozen on the same monthly inputs.Source: A Additional experimental details · p. 6
- Figure 3 compares CE-to-KD reductions in matched-site cross-year dissimilarity and in a same-class-normalized matched-to-nonmatched ratio.The ratio is intended to distinguish location-specific stability from increased class compactness.Source: Figure 3 · p. 7
B Temporal representation stability
The temporal stability analysis compares year-to-year representations at repeated, label-stable locations and finds reduced dissimilarity after distillation, with a smaller location-specific effect beyond class compactness.
B Temporal representation stability
- The analysis uses partially fine-tuned TESSERA encoder features before the projection and classification heads, retaining only repeated locations with unchanged crop labels.Locations with changed labels were excluded because representation change was expected in that case.Source: B Temporal representation stability · p. 7
- 12.2%: KD reduced matched-location cross-year dissimilarity on average across five seeds, with a reduction for every seed.The measure was computed on 1,046 repeated locations from 10 crop classes whose labels were unchanged across surveys.Source: B Temporal representation stability · p. 7
- 2.6%: KD reduced the matched-to-nonmatched ratio on average, with lower values in four of five seeds.The authors interpret most of the dissimilarity reduction as co-occurring with greater within-class compactness, while a smaller same-location stability effect remains.Source: B Temporal representation stability · p. 7
- Table 2 reports Macro F1 and Weighted F1 for frozen AlphaEarth CE, contrastive objectives, a photo probe, and fusion across five seeds.The photo probe and fusion require photographs at inference; satellite-only KD and contrastive models use photos only during training.Source: Table 2 · p. 8
C Contrastive Learning & Modality Fusion
The contrastive and fusion comparisons show that predictive-distribution distillation improves satellite classification, whereas tested contrastive objectives do not improve over their CE control; fusion performs best but requires photos at inference.
C Contrastive Learning & Modality Fusion
- +2.9 macro-F1: KD beats its matched CE student, and the logit-KL-only control reproduces that gain on every seed.The embedding-cosine-only control yields +0.2 macro-F1 with mixed signs.Source: C Contrastive Learning & Modality Fusion · p. 8
- SupCon and InfoNCE do not improve over the matched CE control, with small per-seed paired differences mixed in sign.The authors note that this is consistent with the null embedding-cosine-only control because the contrastive objectives shape embeddings rather than predictive distributions.Source: C Contrastive Learning & Modality Fusion · p. 8
- The photo probe is about 6 pp higher in macro-F1 than the contrastive objectives and both AlphaEarth-only CE rows.Combining modalities is approximately 8 pp higher in macro-F1 than the photo probe.Source: C Contrastive Learning & Modality Fusion · p. 8
- The fusion model’s top performance is not available where field photographs are absent at inference.Both modalities are concatenated as input to its classifier.Source: C Contrastive Learning & Modality Fusion · p. 8
D Per-class analysis
Per-class KD gains varied by crop and model branch under temporal transfer, with large gains and a small loss across the reported examples. Gains were generally stable in held-one-class-out validation, except for frozen TESSERA.
D Per-class analysis
- Excluding the combined flax, cotton, and hemp class reduced frozen TESSERA’s temporal-transfer gain from 5.5 to 3.4 percentage points.This result shows that the reported aggregate gain depends in part on that combined class.Source: D Per-class analysis · p. 8
- KD improved classification for most classes, with frozen AlphaEarth rice F1 slipping from 89.0 to 88.9.Table 3 reports per-class F1 on the 2018 test set for matched CE and KD students.Source: D Per-class analysis · p. 8, Table 3 · p. 9
- +38.9 macro-F1 points for Flax/Hemp on frozen TESSERA contrasted with −1.6 on Rice for the fine-tuned TESSERA branch during temporal transfer.Figure 4 shows paired KD−CE macro-F1 margins per crop class for the 2022 transfer.Source: D Per-class analysis · p. 8, Figure 4 · p. 9
- Held-one-class-out validation showed stable KD gains across model branches, except for frozen TESSERA.The analysis was intended to test whether gains were driven by single-class results.Source: D Per-class analysis · p. 8, Figure 5 · p. 10