Source-linked AI summary
Weakly Supervised Seafloor Segmentation for Seagrass Habitat Mapping in Side-Scan Sonar Imagery
Hayat Rajani, Nuno Gracias, Rafael Garcia
TL;DR
Dense SSS annotation is slow and costly, despite SSS offering high-resolution seabed imagery where optical sensors cannot reach. The paper adapts weakly supervised segmentation to learn pixel-level habitat maps from image-level labels, reaching 87.6% mIoU against ground truth without pixel-level supervision and gaining about 3% from self-supervised pretraining.
Problem
Dense manual annotation limits efficient SSS benthic habitat mapping, although seagrass extent mapping supports coastal management and carbon inventories.
Method
An adapted iterative weakly supervised framework uses a ViT-based encoder-decoder, classification-derived CAMs, dCRF-refined pseudo-labels, and imbalance-aware training.
Results
87.6% mIoU against ground truth was achieved without pixel-level supervision, while self-supervised pretraining added about 3% in mIoU.
Takeaways & Limitations
The results indicate accurate, low-labelling-cost SSS habitat mapping can complement optical mapping in deep and turbid areas.
Abstract
from arXiv · showhide
Seagrass meadows are crucial blue-carbon habitats, and mapping their extent is a prerequisite for coastal management and carbon inventory. Optical satellite sensors cover large areas but cannot reach deep or turbid water, whereas side-scan sonar (SSS) images the seabed at high resolution and at any depth. Interpreting SSS, however, still relies on dense manual annotation, which is slow and costly. We address this by adapting a weakly supervised semantic segmentation framework to SSS benthic habitat mapping, so that pixel-level maps are learned from image-level labels alone. The framework couples a ViT-based encoder-decoder with a classification branch, extracts class activation maps, and refines them into pseudo-labels with a dense conditional random field that we tune for the noise and weak boundaries of acoustic imagery. It follows an iterative self-training scheme, together with a sampling strategy to cope with the strong class imbalance of the data. We also study the effect of different loss functions on segmentation quality, finding Lovász-Softmax loss the most effective. On a held-out transect, the refined pseudo-labels reached an mIoU of 89.3\% against the ground truth, and the segmentation branch, trained without any pixel-level labels, reached 87.6\%. Self-supervised pretraining on unlabelled SSS added a further 3\% in mean intersection-over-union. Field trials further demonstrate the generalizability of the trained model. These results show that accurate and label-efficient benthic habitat mapping from side-scan sonar is feasible at the scale needed for coast-wide seagrass monitoring.
I. INTRODUCTION
Seagrass mapping is important for coastal management and blue-carbon inventories, but dense SSS annotation remains costly. This work adapts weakly supervised segmentation to learn pixel-level benthic habitat maps from image-level labels.
- Seagrass meadows support habitat, sediment stabilisation, and long-term organic-carbon storage, making extent mapping important for management and carbon inventories.
- Side-scan sonar provides high-resolution seabed imagery and reaches depths where optical sensors cannot operate.
- Weakly supervised semantic segmentation produces pixel-level maps from image-level class tags, substantially reducing dense annotation requirements.
- The study adapts an existing WSSS framework rather than proposing a new model, focusing on label refinement and segmentation-loss design.
- The study asks whether image-level supervision can produce accurate SSS habitat maps, compare with fully supervised methods, and transfer across sites and acoustic conditions.
II. RELATED WORK
Optical mapping is effective in shallow water, while SSS extends high-resolution habitat observation into deeper or turbid areas. Earlier acoustic and deep-learning approaches commonly depend on dense labels or address different segmentation targets, motivating weakly supervised SSS habitat mapping.
- Optical imagery supports broad, low-cost seagrass mapping but is limited by water depth and turbidity.
- SSS provides decimetre-resolution seabed backscatter and complements optical surveys in deeper habitat ranges.
- Earlier acoustic habitat studies relied on manual interpretation, classical classification, or combinations with optical and other sensors.
- Deep-learning seafloor segmentation has addressed terrain, navigation, and sediment classification, while separate SSS work has focused mainly on discrete targets and objects.
- Dense pixel-level labels remain slow and expensive for SSS, whereas this work learns benthic habitat segmentation from image-level supervision only.
III. METHODOLOGY
The methodology adapts an iterative weakly supervised framework to acoustic imagery by retuning pseudo-label refinement and reconsidering segmentation loss under noise, weak boundaries, imbalance, and incomplete supervision.
- The approach retains the iterative self-improved framework while adapting components to SSS imagery.
- A dense CRF is retuned for low contrast, speckle noise, weak object boundaries, and fragmented initial CAMs.
- Figure 1 presents the framework components: encoder, classification sub-network, and decoder.
- The segmentation loss is reconsidered for strong class imbalance and partial supervision from incomplete pseudo-labels.
A. Weakly-supervised Segmentation Framework
The framework combines classification-driven pseudo-label generation with decoder training for weakly supervised segmentation. Its pipeline extracts CAMs, refines them with a dense CRF, and uses the resulting masks as pixel-level supervision.
- The encoder and classification sub-network learn multi-label image classification from image-level labels.
- Class activation maps are extracted and thresholded to produce initial pseudo-labels.
- A dense CRF refines the pseudo-labels to sharpen boundaries and reduce label noise.
- The refined masks supervise the decoder through a pixel-wise segmentation loss during iterative training.
B. Pseudo-label Refinement
The framework refines fragmented CAM-derived pseudo-labels with a dense CRF and compares losses suited to weak supervision. Lovász-Softmax directly aligns optimization with the segmentation metric.
- Dense CRF refinement uses pixel intensity and spatial position to improve CAM-derived label boundaries through local consistency.
- Weak supervision motivates comparing cross-entropy, focal loss, and Lovász-Softmax because incomplete pseudo-labels and class imbalance can bias training toward majority classes.
- Lovász-Softmax optimizes mean intersection-over-union through a convex Jaccard-index surrogate.
D. Self-supervised Pre-training
The encoder is self-supervised on abundant unlabelled SSS imagery before weakly supervised fine-tuning. EsViT combines view-level self-distillation with region-level correspondence learning to retain fine-grained acoustic cues.
- Self-supervised pre-training uses unlabelled SSS imagery before weakly supervised fine-tuning, leveraging the larger available archive.
- EsViT trains student and teacher networks across augmented global and local views so local representations match global context.
- EsViT adds region-level matching between corresponding student and teacher regions to capture fine-grained, small, textural SSS cues.
A. Dataset
The BenthiCat dataset provides large unlabelled and annotated SSS collections from Catalonia for representation learning, training, and evaluation. Preprocessing compresses, geometrically corrects, and tiles the raw waterfalls.
- BenthiCat contains approximately one million unannotated SSS tiles for self-supervised learning and about 36,000 annotated tiles for evaluation.
- Logarithmic compression and normalization to [0, 1] stabilize training while preserving low-intensity detail.
- Slant-range correction converts sensor slant range to ground range under a flat-floor assumption.
- The correction uses sensor altitude above the seabed and removes nadir compression and the blind zone.
- The data are tiled into overlapping 384×384 pixel patches with a 192-pixel stride along and across track.
B. Training
Training addresses strong class imbalance, independently tunes dense-CRF post-processing, and uses fixed optimization and augmentation settings. The supplied training passages describe both weakly supervised training and self-supervised pre-training schedules.
- Strong class imbalance produces diffuse CAMs, dominant-class collapse, and sparse or empty minority-class pseudo-masks.
- Dense-CRF hyperparameters are searched independently of network weights and then held fixed during subsequent training.
- Models are trained for 50 epochs with AdamW, weight decay 1e−2, learning rate 6e−5, and CAM updates every 20 epochs.
- Training uses random geometric and appearance augmentations, including rotation, flips, contrast or sharpening changes, and Gaussian blur.
- Self-supervised pre-training runs for 300 epochs with learning rate 5e−4 and a 10-epoch warm-up.
C. Evaluation
The models were evaluated on a manually annotated test set and in field trials using a retrofitted Girona1000 AUV equipped with Klein 3000 SSS.
- Evaluation settings: Field trials used small transects recorded along the port of St. Feliu de Guixols, Spain.The Girona1000 AUV was retrofitted with a Klein 3000 SSS.
- Evaluation settings: Performance was reported using mean Intersection over Union and inference speed in images processed per second.All trained models were evaluated on an NVIDIA Jetson AGX Orin Developer Kit.
V. RESULTS AND DISCUSSION
The iterative framework produced refined pseudo-masks and segmentation predictions that matched manually created ground truth closely, while field trials revealed errors around sonar blind zones and shadows.
- Test-transect results: 92.94% mean IoU measured agreement between pixel-level predictions and pseudo segmentation masks on the test transect.The result concerns agreement with pseudo-masks rather than manually created ground truth.
- Test-transect results: 89.3% mIoU was achieved by pseudo segmentation masks against manually created ground truth on the test transect.Pixel-level ground truth was manually created for this test set.
- Test-transect results: 87.6% mIoU was achieved by segmentation-branch predictions against ground truth without pixel-level training labels.The pixel-level predictions scored lower than the refined pseudo-labels on the same test transect.
- Test-transect results: Self-supervised pretraining increased mIoU by about 3% on top of these results.The improvement was reported after evaluating pseudo-masks and segmentation predictions against ground truth.
- Field trials: Field-test discrepancies arose in merged port-and-starboard imagery because blind zones and shadows near the first bottom return were misclassified.The fully supervised model classified some featureless areas as mud and shadowy areas as rocks, whereas the weakly supervised model classified the region as mud.
VI. CONCLUSION
The paper presents an iterative weakly supervised SSS segmentation pipeline that learns pixel-level maps from image-level labels. On a held-out transect, it achieved 87.6% mIoU without pixel-level supervision, with further gains from self-supervised pretraining, supporting low-cost habitat mapping.
- Conclusion: The proposed pipeline learns pixel-level benthic habitat segmentation from image-level labels alone.It uses iterative self-training with CAM-derived pseudo-labels refined by a dense CRF.
- Conclusion: 87.6% mIoU was achieved by the segmentation branch against ground truth without pixel-level supervision.The refined pseudo-labels reached 89.3% mIoU on the same held-out transect.
- Conclusion: About 3% further improvement in mIoU came from self-supervised pretraining on unlabelled SSS.This improvement was reported on top of the segmentation results.
- Conclusion: The model recovered small-scale objects and inter-class boundaries well on the held-out transect.The result was reported for segmentation predictions produced without pixel-level supervision.
- Conclusion: The results indicate that SSS can support accurate, low-labelling-cost benthic habitat mapping in deep and turbid areas beyond optical reach.The paper frames SSS as a practical complement to optical mapping.