PaperlayerView exact revision on arXiv

Source-linked AI summary

BenthicDINO: Physics-Informed Self-Distillation for View-Invariant Side-Scan Sonar Representations

Taqi Hamoda, Hayat Rajani, Nuno Gracias

arXiv:2608.23215v1cs.CVcs.AIcs.LG

TL;DR

SSS perception is hindered by acoustic artifacts and representations that mix seabed structure with viewing geometry, while existing SSL augmentations do not explicitly model these factors. The paper introduces physics-informed DINOv3 self-distillation with ConvNeXt-v2-Tiny, physics-guided augmentations, HSIC-based decoupling, and four-stage feature fusion. On S3Seg, the fused representation reaches 96% of peak performance with 10% of annotations and achieves 71.4% mIoU and 86.5% overall accuracy at full data scale.

  • Problem

    Existing sonar SSL pipelines inherit natural-image augmentations and do not explicitly enforce invariance to physical acquisition geometry or acoustic degradation, limiting view-invariant SSS representations.

  • Method

    The framework adapts DINOv3 with a ConvNeXt-v2-Tiny encoder, physics-guided augmentations, HSIC-based independence regularization, and dense hierarchical fusion across four stages.

  • Results

    96% of peak performance is achieved using 10% of annotated S3Seg data, with 71.4% mIoU and 86.5% overall accuracy at full data scale.

  • Takeaways & Limitations

    The learned space forms dense, noise-free semantic clusters, and the fused representation reduces the annotation burden for SSS habitat mapping.

Abstract

from arXiv · show

Automated perception in side-scan sonar (SSS) imagery is severely hindered by physical acoustic artifacts, resulting in representations that inextricably mix intrinsic seabed reflectivity with transient viewing geometries. Existing self-supervised learning (SSL) frameworks rely on augmentations designed for natural images, failing to account for acoustic degradation and explicitly enforce view-invariance. To address this gap, we introduce a physics-informed self-distillation framework built upon the DINOv3 architecture utilizing a ConvNeXt-v2-Tiny backbone to maximize data efficiency. The proposed methodology enforces view-invariance through two primary mechanisms: physically motivated augmentations that simulate speckle noise, range-dependent attenuation, and radiometric miscalibration; and a Hilbert-Schmidt Independence Criterion (HSIC) penalty that explicitly decouples learned dense patch features from physical viewing parameters. Furthermore, we propose a dense, hierarchical feature fusion strategy across all four network stages to preserve fine-grained sediment details alongside deep semantic abstractions. Extensive evaluation demonstrates that the framework natively groups complex benthic topographies into stable, noise-free semantic clusters without relying on manual annotations. During supervised downstream tasks on the S3Seg dataset, the fused representations exhibited exceptional data efficiency, achieving 96% of its absolute peak performance using only 10% of the available annotated data, ultimately reaching a mean Intersection over Union (mIoU) of 71.4% and an overall accuracy of 86.5%.

I. INTRODUCTION

SSS perception is challenged by acoustic artifacts and viewpoint dependence that obscure seabed structure and weaken cross-view correspondence. The proposed framework addresses this gap with physics-guided self-distillation, explicit statistical view-invariance, and hierarchical ConvNeXt-v2 features.

  • Motivation: Acoustic attenuation enables long-range SSS mapping, but its image formation depends on acquisition dynamics, reflectivity, incidence angle, and propagation loss.SSS records returning echoes as a 2D backscatter intensity map; acoustic complexity also includes non-uniform beam power, occlusions, shadows, and speckle noise.
  • Motivation: Viewpoint-dependent intensity and shadow geometry make repeated surveys difficult to associate, undermining matching, loop closure, mosaicking, and change detection.The paper identifies view-invariant representations as a prerequisite for robust large-scale SSS perception.
  • Research gap: Standard SSL augmentations designed for natural scenes fail to model SSS acoustic degradation and leave learned features dependent on acquisition geometry.This limitation motivates explicit physics-guided augmentations and an independence constraint.
  • Approach: The framework adapts DINOv3 with a ConvNeXt-v2-Tiny backbone and dense multi-scale fusion to retain fine-grained seabed textures.The convolutional backbone is selected for its inductive biases and suitability for small SSS datasets.
  • Approach: HSIC explicitly decouples dense representations from slant range and incidence angle, formulating SSS view-invariance as statistical independence.The objective targets learned representations rather than only preprocessing or augmentation behavior.
  • Approach: Physics-guided augmentations inject speckle, range-dependent attenuation, and radiometric variation without manual annotation, while representation analyses report improved cross-view correspondence and downstream transfer.The augmentations are derived from the SSS acquisition model and target dominant nuisance factors.

II. RELATED WORK

Prior sonar SSL work reduces annotation dependence but generally inherits natural-image augmentations and does not explicitly constrain representations against acoustic acquisition geometry. BenthicDINO instead embeds physics-guided perturbations and latent-space independence into self-distillation.

  • Existing sonar SSL: Sonar SSL research includes rotation prediction, denoising, jigsaw tasks, momentum contrast, architecture comparisons, and joint-embedding predictive methods.These studies target low-label regimes or in-domain sonar perception across FLS, SAS, and SSS settings.
  • Gap: Existing sonar SSL pipelines use augmentations and pretext tasks designed for natural imagery, leaving physical acquisition geometry and acoustic degradation without explicit invariance constraints.The paper presents this as the fundamental gap motivating its approach.
  • Physics-informed methods: Physics-informed sonar methods commonly use simulators, learned generators, or backscatter correction to improve realism or reduce intensity inconsistencies.These approaches address synthesis, domain alignment, or preprocessing before feature extraction.
  • Positioning: Unlike preprocessing and synthesis approaches, this work enforces geometric view-invariance directly in the latent representation during pre-training.The stated goal is to decouple intrinsic seabed structure from acquisition geometry within learned features.
  • Proposed framework: The framework combines a ConvNeXt-v2-Tiny DINOv3 model, physics-guided augmentations, an explicit statistical-independence penalty, and small-batch training adaptations.These components are described as the three main framework parts plus stability adaptations.

A. DINOv3 Self-Distillation Framework

DINOv3 trains a student to match a stable EMA teacher across augmented views, combining global semantic alignment with dense patch objectives and regularization. The resulting objective promotes global robustness while supporting dense prediction and non-collapsed embeddings.

  • Self-distillation: DINOv3 uses self-distillation in which a student matches a teacher whose weights are updated by exponential moving average.The teacher provides a stable target representation during pre-training.
  • Objective: The composite pre-training loss combines DINO, iBOT, Gram anchoring, and KoLeo terms.The stated expression is LPre = LDINO + LiBOT + 0.5 · LGram + 0.1 · LKoLeo.
  • Global discriminative loss: The DINO objective aligns student and teacher output distributions across global and local crops, with the teacher restricted to global views and centered to prevent collapse.The outputs are probability distributions over K dimensions produced by a projection head and softmax.
  • Global discriminative loss: Minimizing the DINO objective promotes global features robust to geometric distortions and sensor noise.This is the stated role of the global objective in the framework.
  • Dense prediction: iBOT adds a masked patch-level objective that trains the student to reconstruct masked-token features from teacher representations of corresponding unmasked patches.The reconstruction is formulated as cross-entropy between teacher and student patch distributions.
  • Regularization: KoLeo regularization encourages a uniform dense-embedding distribution by maximizing minimum sample distances, improving class separability in downstream benthic tasks.A distributed variant computes the same quantity across the distributed batch.

4) Gram Anchoring:

The framework stabilizes dense masked learning with ConvNeXt-v2 and preserves information across network depth through four-stage hierarchical fusion. This produces a unified descriptor combining fine sediment detail with deeper semantic abstraction.

  • Gram anchoring: Gram anchoring regularizes student patch-feature correlations toward a stable earlier-training teacher, improving dense-feature stability and preserving fine sediment textures.The Gram matrix captures pairwise correlations among L2-normalized patch features.
  • Backbone choice: ViTs require substantial training data because they lack convolutional inductive biases, motivating a convolutional backbone for scarce SSS data.The paper specifically cites translation equivariance and local connectivity as useful CNN properties.
  • Backbone choice: ConvNeXt-v2 introduces global response normalization to maintain feature diversity and prevent channel saturation during masked latent reconstruction.GRN follows the MLP expansion layer and applies divisive channel-wise normalization.
  • Hierarchical fusion: The fusion module aggregates hierarchical representations from all four network stages rather than discarding early features.This design is intended to retain both shallow texture information and deeper representations.
  • Hierarchical fusion: Stages 2–4 are bilinearly upsampled to stage 1 resolution, concatenated by channel, and compressed with a 3-layer 1 × 1-convolution MLP into a 768-dimensional descriptor.The fusion preserves spatial context while unifying multi-scale features.
  • Hierarchical fusion: Dense feature fusion preserves high-frequency sediment details from shallow layers while integrating robust semantic abstractions from deeper stages.This connects the multi-stage architecture to the representation's intended information content.

C. Physics-Guided Augmentations

The framework uses three physically motivated augmentations to expose SSS representations to speckle, range-dependent attenuation, and radiometric variation. Each augmentation is applied independently to crops so student and teacher views experience varying acoustic conditions.

  • Three augmentations target speckle noise, range-dependent attenuation, and radiometric miscalibration in SSS imagery.
  • Independent crop-level augmentation presents the student and teacher with varying acoustic conditions for the same seabed patch.This is intended to compel features invariant to physical artifacts.
  • Additive Gaussian perturbations act as a denoising regularizer that promotes structural representations over reliance on raw pixel intensities.The implementation bounds the sampled standard deviation to avoid catastrophic signal destruction.
  • Synthetic residual gain curves model uncompensated range loss by multiplying each tile with a linear decay gradient across range.The operation targets residual attenuation trends that vary between survey passes.

2) Synthetic Time-Varying Gain (TVG):

The method combines synthetic TVG attenuation with radiometric perturbations and HSIC regularization to reduce dependence of learned features on acquisition conditions. The TVG operation simulates severe range fading, while HSIC targets residual geometric information directly in the representation.

  • Synthetic Time-Varying Gain (TVG): The TVG augmentation applies a linear range decay whose far edge retains 30%–70% of original intensity, with probability p = 0.5.The retention factor β is sampled from U(0.3, 0.7), and the effect is illustrated in Fig. 2.
  • Radiometric Jitter: Radiometric jitter perturbs global brightness and contrast to model operator adjustments and automatic gain-control errors.The contrast scale uses c = 0.2 and the brightness offset uses b = 0.1.
  • Enforcing View-Invariance: HSIC regularization minimizes dependence between dense patch features Z and viewing geometry V, comprising incidence angle and slant range.The objective is integrated into the DINOv3 pre-training loss.
  • Enforcing View-Invariance: The total objective combines the pre-training loss with γHSIC(Z, V), where γ controls the strength of the view-invariance constraint.Random Fourier Features approximate the Gaussian RBF kernel to avoid exact HSIC’s O(B^2) mini-batch complexity.

IV. EXPERIMENTAL SETUP

The study trains and evaluates on BenthiCat, using approximately one million SSS tiles spanning diverse benthic habitats. Preprocessing includes logarithmic normalization, slant-range correction, overlapping patch extraction, and multi-crop generation.

  • BenthiCat supplies approximately one million self-supervised SSS tiles collected along Catalonia’s coast across varied benthic habitats.
  • Raw 12-bit waterfall data are logarithmically compressed and normalized to [0, 1] to preserve low-intensity structural details.
  • Slant-range correction converts slant range to ground range under a flat-seafloor assumption, removing nadir compression and the acoustic blind zone.
  • Images are extracted as overlapping 384 × 384 patches with a 192-pixel stride, then augmented into 224 × 224 global and 96 × 96 local crops.

B. Training Details

Training adapts DINO-style self-distillation to acoustic statistics and small-batch distributed constraints, while evaluation examines fused representations through dimensionality, clustering, and neighborhood-preservation metrics.

  • Training Details: Two adaptations stabilize DINO-style training: reducing the projection dimension from K = 65,536 to K = 4,096 and replacing Sinkhorn-Knopp for global-to-local distillation.The replacement uses DINOv1 centering, while the iBOT head retains Sinkhorn-Knopp.
  • Training Details: Training uses two NVIDIA Quadro RTX 6000 GPUs with an effective batch size of 8,192 and cosine scheduling with linear warmup.
  • Clustering and Manifold Metrics: The fused dense feature space is evaluated with geometric and topological metrics to assess representation quality without downstream-task bias.
  • Clustering and Manifold Metrics: Effective rank uses Shannon entropy of covariance singular values, while Two-NN estimates the learned manifold’s intrinsic dimension.
  • Clustering and Manifold Metrics: K-Means and HDBSCAN assess semantic grouping using Silhouette Coefficient, Noise Ratio, Mean Confidence, and Trustworthiness.Silhouette values approaching 1 indicate dense, well-separated spherical clusters in Euclidean space.

V. RESULTS AND DISCUSSION

The learned feature space preserves local topology without dimensional collapse and forms dense, well-separated semantic clusters under density-based analysis.

  • Feature Geometry: An effective rank of 53.81 and smoothly decaying singular values indicate the 768-dimensional representation avoids dimensional collapse.The spectrum lacks a sharp cutoff, supporting retained representation capacity.
  • Feature Geometry: An intrinsic dimension of 8.80 indicates that local semantic neighborhoods occupy approximately nine degrees of freedom.The features remain high-dimensional globally while exhibiting low-dimensional local structure.
  • Clustering: K-Means reached 1.0000 Trustworthiness for k ≥32, but its Silhouette score peaked at only 0.1336 for k = 56.The combination preserves local neighborhoods but does not produce simple spherical Euclidean clusters.
  • Clustering: HDBSCAN identified 6 dense clusters with a 0.5359 filtered Silhouette score, 0.0% noise, and 0.995 mean membership confidence.This density-based result indicates stable, well-separated semantic basins with virtually no ambiguous samples.

B. Supervised Downstream Evaluation on S3Seg

Supervised evaluation on S3Seg shows that hierarchical fusion improves representation quality beyond individual backbone stages, while Linear Probing generalizes better than K-NN and requires few labels.

  • Evaluation Setup: The evaluation freezes the ConvNeXt-v2 backbone and tests Linear Probing and K-Nearest Neighbors on four annotated benthic classes.The data-efficiency study evaluates 1%, 5%, 10%, 50%, and 100% training subsets.
  • Feature Representation: 45.4% mIoU at Stage 1 rises to 56.9% at Stage 2 and 67.3% at Stage 3, then plateaus at 67.0% for Stage 4.The progression reflects increasingly useful representations before the deepest stage loses some fine-grained spatial detail.
  • Feature Representation: 71.4% mIoU and 86.5% accuracy are achieved by the fused representation, which combines fine-grained Stage 1 details with later semantic context.The fused representation is reported to strictly dominate individual stage outputs.
  • Probe Comparison: 71.4% Test mIoU is maintained by Linear Probing at full data scale, while K-NN falls to 59.7% mIoU despite approximately 0.998 Train mIoU.The reported gap indicates severe K-NN overfitting in this embedding space.
  • Data Efficiency: Using 10% of labeled data achieves 96% of peak performance, with 68.8% versus 71.4% mIoU.Increasing labeled data from 50% to 100% yields only a +0.004 mIoU gain.

4) Class-Level Bottleneck Analysis:

Class-level analysis identifies Rocks as the principal bottleneck, while future work targets broader SSL comparisons, cross-dataset validation, and nonlinear segmentation heads.

  • Class-Level Bottleneck Analysis: Rocks are the primary bottleneck for overall dataset mIoU.Their errors reflect both acoustic similarity to Sand Ripples and class imbalance.
  • Class-Level Bottleneck Analysis: 20.7% of true Rocks are misclassified as Sand Ripples, and 6.5% as Maerl.The extracted 256×256 patches do not capture enough broader structure to distinguish isolated rocks from ripple fields.
  • Downstream Evaluation: The fused representation reaches 71.4% mIoU and 86.5% overall accuracy at full data scale, with 96% of peak performance using 10% annotated data.These results were obtained on S3Seg with a simple Linear Probe.
  • Future Work: Future work will compare the DINO-based objective with other SSL paradigms, including Masked Autoencoders.The comparison is intended to assess which objective better captures acoustic scattering physics.
  • Future Work: Future evaluations will test generalization across sensor payloads and marine environments using public datasets such as AI4Shipwrecks.The current supervised testbed is the S3Seg dataset.
  • Future Work: Future segmentation heads will use lightweight MLPs or dedicated decoders to model nonlinear decision boundaries.This direction addresses confusion between acoustically similar classes such as Rocks and Sand Ripples.
Loading 2608.23215v1…