PaperlayerView exact revision on arXiv

Source-linked AI summary

CrevasseSeg: A Label-Efficient UAV Crevasse Segmentation Framework

Steven Wallace, William D. Harcourt, Richard Hann, Aiden Durrant, Somayajulu Sripada, Georgios Leontidis

arXiv:2608.15790v2cs.LG

TL;DR

Pixel-level crevasse annotation is costly and expert-dependent, limiting UAV-based glacier mapping. CrevasseSeg benchmarks self-supervised objectives and architectures with linear and nonlinear readouts, finding that DINOv3 features are weakest linearly but strongest nonlinearly.

  • Problem

    Pixel-level glacier-surface annotation is costly, expert-dependent, and difficult to scale for remote-sensing semantic segmentation.

  • Method

    CrevasseSeg benchmarks five self-supervised objectives across three architectures using linear-probe and XGBoost readouts for UAV crevasse segmentation.

  • Results

    DINOv3 features are weakest under linear probing but strongest under nonlinear readout, with satellite pretraining consistently improving performance.

  • Takeaways & Limitations

    CrevasseSeg provides a label-efficient UAV segmentation framework and supports evaluating readout geometry when comparing learned representations.

  • Takeaways & Limitations

    The single-glacier, two-campaign dataset leaves generalization across seasons, years, and glacier environments untested.

Abstract

from arXiv · show

Crevasse mapping from uncrewed aerial vehicle (UAV) imagery matters for glaciological research and for field safety in glaciated terrain. Yet, pixel-level annotation of glacier surfaces is costly and requires domain experts. We introduce CrevasseSeg, a framework for binary segmentation over the terminus of Borebreen, Svalbard, comprising 1,938 unlabelled UAV orthomosaic tiles for self-supervised/unsupervised fine-tuning, 24 labelled tiles for validation and 176 labelled tiles for testing. Using CrevasseSeg, we benchmark five self-supervised objectives -- BYOL, a Jensen-Shannon Divergence (JSD) objective, Barlow-Twins, VICReg, and a combined BYOL-JSD objective -- across three architectures: O-Net, O-Net++, and a DINOv3-initialised O-Net. Each configuration is evaluated under two frozen-feature readouts that differ only in the form of their decision boundary: a linear probe and a non-linear XGBoost classifier fit only on the 24 labelled validation images. Our central finding is a consistent inversion between the two readouts: DINOv3 features are the weakest under linear probing but the strongest under a non-linear readout. A UMAP analysis of the learned feature space shows that DINOv3 fragments pixels into many small clusters in which the classes are locally interleaved, whereas the convolutional architectures (O-Net and O-Net++) embed them onto a single class-sorted manifold. Satellite-pretrained DINOv3 improves over natural-image initialisation across objectives, and our label-efficient DINOv3-ViT-L-Sat-O-Net-BYOL-JSD pipeline reaches 75.33 mDSC / 61.28 mIoU, outperforming standard machine learning baselines fit on the same 24 labelled images with the RGB pixel values used as features. We release CrevasseSeg to support label-efficient segmentation research in remote sensing.

1 Introduction

The introduction motivates label-efficient crevasse segmentation from UAV orthomosaics, where pixel-level annotation is costly and scarce. It presents CrevasseSeg as a benchmark for testing self-supervised representations, architectures, and linear versus non-linear readouts using only 24 labelled images.

  • Motivation: Pixel-level semantic-segmentation annotation is labour-intensive, expert-dependent, and difficult to scale, making self-supervised learning especially relevant when labels are scarce.Crevasse mapping faces this constraint because labelled data for UAV-based glacier segmentation barely exists.
  • Motivation: UAV structure-from-motion surveys enable centimetre-scale glacier orthomosaics, making automated crevasse segmentation attractive for research and polar field safety.Crevasses indicate ice dynamics and pose significant dangers during polar fieldwork.
  • CrevasseSeg benchmark: CrevasseSeg is a public UAV-orthomosaic benchmark for binary segmentation of Borebreen’s highly crevassed terminus, using a fixed 1,938 / 24 / 176 train–validation–test split.Only 24 labelled images are available for fitting any classifier.
  • Benchmark design: The benchmark evaluates five self-supervised objectives across O-Net, O-Net++, and DINOv3-O-Net under linear-probe and XGBoost feature readouts.The readouts differ in whether their decision boundary is linear or non-linear.
  • Representation analysis and pipeline: The study analyzes how feature readouts interact with frozen representations and develops a 24-image label-efficient DINOv3-O-Net pipeline with XGBoost and decoder-block soft voting.The pipeline is benchmarked against standard machine-learning classifiers and recent unsupervised-segmentation methods.

2 Related Work

Prior work spans label-free representation learning, unsupervised semantic segmentation, and remote-sensing adaptation, motivated here by scarce labels and domain shift in UAV imagery. Glacier crevasse mapping also supports glaciological understanding, mass-loss analysis, and safer polar fieldwork.

  • Self-supervised learning: SSL methods learn label-free representations by enforcing consistency across augmented views through contrastive, distillation-based, or redundancy-reduction objectives.These methods were primarily developed for image classification, whereas supervised segmentation requires dense annotation.
  • Unsupervised semantic segmentation: Unsupervised semantic segmentation combines clustering and superpixels or learns pixel-level structure through photometric invariance, smoothness, spectral cues, or mask proposals.Representative approaches include SLIC, PiCIE, SmooSeg, EAGLE, and PriMaPs-EM.
  • Remote sensing: Remote-sensing research addresses scarce labels and domain shift through unsupervised single-scene segmentation and cross-sensor unsupervised domain adaptation.These directions are particularly relevant to UAV surveys of previously unmapped terrain, where labelled data are rarely available in advance.
  • Methods relevant to CrevasseSeg: BYOL learns consistency between augmented-view projections, while O-Net enforces consistency between predictive distributions from Siamese U-Net branches using JSD loss.The objectives differ in whether consistency is imposed on representations across views or on predicted distributions.
  • Glaciological motivation: Crevasse mapping informs glacier deformation and stress-strain relationships, mass loss and sea-level change, and safe route planning for polar fieldwork.The paper applies this motivation to crevasse detection from UAV orthomosaic imagery in Svalbard, where UAV-based glaciological research has grown rapidly.

3 The CrevasseSeg benchmark

CrevasseSeg is a low-label benchmark built from UAV imagery of Borebreen’s highly crevassed terminus in Svalbard. It uses unlabelled tiles for self-supervised training, 24 labelled validation tiles for classifier fitting, and 176 held-out labelled test tiles for evaluation with linear or nonlinear readouts.

  • Dataset coverage: CrevasseSeg covers Borebreen’s highly crevassed tidewater-glacier terminus in Svalbard across imagery acquired in 2023 and 2025.The regions of interest span 0.415 km2 and 1.182 km2, respectively, and were prioritised because the terminus is susceptible to iceberg calving.
  • Dataset split: Only 24 validation tiles have labels for fitting classifiers, while 1,938 training tiles are unlabelled and 176 labelled test tiles are held out.The training tiles comprise 874 from 2023 and 1,064 from 2025, making the benchmark explicitly low-label.
  • Evaluation protocol: Models are evaluated on the 176 test tiles using foreground mean Dice Score Similarity (mDSC) and mean Intersection over Union (mIoU).All models are pretrained or fine-tuned with self-supervised learning on the unlabelled training split.
  • Evaluation protocol: Frozen per-pixel features are classified using 24 labelled validation tiles with either a linear probe or a nonlinear XGBoost decision boundary.For DINOv3-based models, only the convolutional decoder is trained with transfer learning.

4 Benchmark methods

The benchmark compares three segmentation architectures across five self-supervised objectives under a shared fine-tuning and readout protocol. Frozen per-pixel features are classified using either a linear probe or XGBoost trained on 24 labelled validation tiles, with the final system combining satellite-pretrained DINOv3-O-Net, BYOL-JSD, and XGBoost.

  • Architectures: Three architectures are evaluated: Siamese U-Net O-Net, nested U-Net++ O-Net++, and DINOv3-O-Net with natural-image or satellite-pretrained ViT-L initialization.DINOv3-O-Net uses a DINOv3 ViT-L encoder backbone.
  • Architectures: DINOv3-O-Net freezes its backbone and passes multi-scale token features through adaptor layers, FAPM, and progressively fused decoders before classification.Only the adaptor, FAPM, and decoder parameters are fine-tuned for DINOv3-O-Net.
  • Self-supervised objectives: Five objectives are compared: BYOL, JSD consistency, Barlow Twins, VICReg, and a combined BYOL-JSD loss with λ = 0.5.The objectives represent prediction consistency, distribution consistency, redundancy reduction, variance-invariance-covariance regularisation, and their combination.
  • Training protocol: Fine-tuning uses AdamW, Reduce-on-Plateau scheduling, batch size 256, 500 epochs, and the checkpoint with the lowest SSL loss on one NVIDIA A100 GPU.DINOv3-O-Net fine-tunes only its decoder, whereas O-Net and O-Net++ are trained end-to-end; spatial-orientation-changing augmentations are excluded.
  • Readout protocol: After self-supervised fine-tuning, classifiers are fit on extracted per-pixel features from 24 labelled validation tiles using either a linear probe or XGBoost.Both classifier hyperparameters are tuned on the validation set to predict crevasse foreground versus ice background.
  • Readout protocol: The final system is DINOv3-ViT-L-Sat-O-Net-BYOL-JSD, using satellite-pretrained DINOv3-O-Net features with an XGBoost readout for binary crevasse classification.The system predicts whether each input-tile pixel belongs to a crevasse or ice.

5 Results

Results show a strong readout-dependent inversion: convolutional backbones lead under linear probing, whereas satellite-pretrained DINOv3-O-Net leads with nonlinear XGBoost, supported by locally interleaved feature geometry.

  • Linear versus nonlinear readouts: 71.79 mDSC / 57.07 mIoU is achieved by convolutional O-Net and O-Net++ backbones under linear evaluation, while DINOv3 configurations score 48.24–58.45 mDSC / 33.81–42.88 mIoU.Several DINOv3 rows collapse to the majority-class score of 48.27 mDSC / 33.87 mIoU.
  • Linear versus nonlinear readouts: Under nonlinear XGBoost, satellite-pretrained DINOv3-O-Net becomes strongest for four of five objectives, the majority-class collapse disappears, and convolutional backbones fall back.The result indicates that DINOv3 features encode crevasse structure that linear boundaries cannot recover but nonlinear readouts can.
  • Pretraining and objectives: +6.4 mDSC / +7.3 mIoU is the average improvement of satellite-pretrained DINOv3 over its natural-image counterpart under XGBoost across objectives.Gains range from +1 point for BYOL-based objectives up to nearly +16 points for JSD.
  • Pretraining and objectives: 75.68 mDSC / 62.43 mIoU from BYOL alone marginally exceeds 75.33 mDSC / 61.82 mIoU from BYOL-JSD on the satellite backbone.The difference is within the margin resolvable by a single-seed evaluation, so the JSD term is not claimed as the source of the gains.
  • Label-efficient comparison: 75.33 mDSC / 61.82 mIoU from DINOv3-ViT-L-Sat-O-Net-BYOL-JSD with XGBoost exceeds standard classifiers trained on the same 24 labels, including raw-pixel XGBoost at 71.43 mDSC / 57.12 mIoU.The standard baselines occupy the 70.06–71.83 mDSC / 55.99-57.55 mIoU range, isolating the contribution of SSL fine-tuned features rather than the classifier alone.
  • Feature-space analysis: UMAP shows O-Net and O-Net++ forming class-sorted connected manifolds, whereas DINOv3 forms fragmented clusters with locally interleaved classes, explaining the readout inversion without establishing encoder superiority causally.O-Net is better suited to lightweight linear classifiers; frozen satellite DINOv3-O-Net is preferable when nonlinear readout is available.

6 Limitations and future work

CrevasseSeg’s evaluation is limited by its single-glacier, two-campaign dataset, possible spatial autocorrelation, and single-seed estimates. Future work should test broader seasonal, temporal, and environmental generalisation while strengthening statistical validation.

  • Limitations: CrevasseSeg uses data from one glacier collected during only two campaigns, leaving generalisation across seasons and years untested.The campaigns occurred in August 2023 and September 2025.
  • Limitations: Adjacent, non-overlapping tile crops from the same glacier front may introduce spatial autocorrelation across train, validation, and test sets.This potential dependence could affect the independence of the reported evaluations.
  • Future work: Differences among the top configurations are small and based on single-seed point estimates, motivating multi-seed training with confidence intervals and significance testing.These analyses are needed before drawing stronger conclusions about configuration differences.
  • Future work: Future research should evaluate generalisation across glacier environments to support automated mapping, large-scale surveys, and real-time monitoring during glacier travel.Broader environmental testing would also help adapt outputs to differing operational contexts.

7 Conclusion

CrevasseSeg is presented as a label-efficient UAV orthomosaic crevasse-segmentation framework and benchmark spanning multiple self-supervised objectives, architectures, and readouts. The conclusion highlights DINOv3’s weaker performance under linear separability and motivates evaluation protocols that do not rely on it.

  • CrevasseSeg benchmarks five self-supervised objectives across O-Net, O-Net++, and DINOv3-O-Net using linear and non-linear readouts.
  • The benchmark identifies a clear and reproducible weakness of foundation-model DINOv3 features under the reported comparison.
  • The authors aim for CrevasseSeg to support further label-efficient segmentation research in remote sensing, especially evaluations that do not rely on linear separability.

CRediT Author Statement

The CRediT statement assigns Wallace primary conceptual, methodological, software, analytical, data-curation, and drafting responsibilities, while the other authors contributed investigation, data curation, editing, and supervision.

  • Steven Wallace contributed conceptualization, methodology, software, formal analysis, data curation, and original-draft and review editing responsibilities.
  • William D. Harcourt contributed investigation, data curation, review editing, and supervision.
  • Richard Hann contributed investigation, data curation, and review editing; Aiden Durrant contributed review editing and supervision.
  • Somayajulu Sripada and Georgios Leontidis contributed review editing and supervision.

Appendix · A Borebreen QGIS orthomosaic

Figure A.1 presents the regions of interest used for training, validation, and testing at Borebreen across two survey dates.

  • A Borebreen QGIS orthomosaic: Figure A.1 maps the regions of interest used for training, validation, and testing at Borebreen.The figure covers imagery from 8th August 2023 and 8th September 2025.
  • A Borebreen QGIS orthomosaic: The top panel shows Borebreen regions of interest from 8th August 2023.These regions support the training, validation, and testing areas identified in the figure.
  • A Borebreen QGIS orthomosaic: The bottom panel shows Borebreen regions of interest from 8th September 2025.The same training, validation, and testing categories are represented for this survey date.

B Qualitative results

Figure B.1 presents qualitative segmentation results on six representative CrevasseSeg test tiles, comparing multiple methods and the proposed DINOv3-based pipelines. Results are shown for both Linear-Probe and XGBoost heads.

  • Qualitative comparison: Six representative CrevasseSeg test tiles are used for qualitative comparison.Each tile is 512 × 512 pixels.
  • Qualitative comparison: The figure displays input orthomosaic tiles, ground-truth masks, and predictions from seven segmentation methods or pipelines.Compared methods include PriMaPs-EM, EAGLE, HP, STEGO, SmooSeg, the ensemble pipeline, and the non-ensemble proposed pipeline.
  • Readout comparison: Columns 1–3 show predictions from the Linear-Probe head, whereas columns 4–6 show predictions from the XGBoost head.The two readouts are arranged across the six representative examples.
Loading 2608.15790v2…