PaperlayerView exact revision on arXiv

Source-linked AI summary

Comparative Assessment of Deep Learning Architectures for Underwater Subsurface Kelp Forest Segmentation with The Kelp-o-Tron

Sundarabalan Balasubramanian, César Borja, Ana C. Murillo, Lexi N. Wilkes, Meredith L. McPherson, Kira A. Krumhansl, Jennifer A. Dijkstra, Jarrett E. K. Byrnes

arXiv:2608.24594v1cs.CV

TL;DR

Accurate underwater kelp segmentation is difficult because kelp morphology, optical variability, and complex benthic backgrounds challenge image interpretation. The study develops SSeg-assisted training data and compares three semantic segmentation architectures across independent sites. ResNet50-DeepLabV3, released as Kelp-O-Tron, provides the best balance of segmentation accuracy, robustness, and generalization.

  • Problem

    Kelp segmentation remains challenging under optical degradation, illumination variability, turbidity, overlapping vegetation, and complex benthic backgrounds, despite kelp forests’ ecological importance.

  • Method

    The study develops 3,395 SSeg-assisted image-mask pairs and compares ResNet34-U-Net, ResNet50-DeepLabV3, and ASPP-Transformer using independent site-based evaluation.

  • Results

    ResNet50-DeepLabV3 achieved the strongest overall and most consistent performance across independent and external datasets, with higher Dice and IoU scores and stable kelp-cover estimation.

  • Takeaways & Limitations

    Kelp-O-Tron is provided as an open tool for automated kelp segmentation and is recommended for automated nearshore subsurface kelp segmentation.

  • Takeaways & Limitations

    Because training data came primarily from northeastern U.S. coastal waters, performance may vary in environments differing in kelp morphology, species, water clarity, illumination, or image acquisition.

Abstract

from arXiv · show

Submerged kelp forests are vital coastal ecosystems that support marine biodiversity and ecosystem dynamics, yet accurate underwater kelp segmentation remains challenging due to optical degradation, illumination variability, turbidity, overlapping vegetation, and complex benthic backgrounds. We systematically evaluated three deep learning semantic segmentation frameworks, ResNet34-U-Net, ResNet50-DeepLabV3, and a hybrid ResNet50-ASPP-Transformer architecture, for kelp detection using high-resolution underwater RGB imagery collected from northeastern U.S. coastal waters. A dataset of 3,395 SSeg assisted annotated image-mask pairs was developed for model training and validation, while geographically independent sites were used for quantitative and qualitative evaluation. All models used consistent preprocessing, augmentation, and evaluation protocols. On independent test data, ResNet50-DeepLabV3 achieved the highest Dice (0.7120) and Intersection over Union (IoU; 0.6267), followed by ResNet34 U Net (Dice 0.6868; IoU 0.5978). The hybrid ASPP Transformer achieved the highest pixel accuracy (0.8528) but lower Dice (0.6437) and IoU (0.5746). External qualitative evaluation further showed that DeepLabV3 produced more consistent segmentation across varying environmental conditions, image qualities, and benthic habitats. Overall, ResNet50-DeepLabV3, termed Kelp-O-Tron, provided the best balance of segmentation accuracy, robustness, and generalization. The dataset, annotation workflow, and comparative evaluation provide resources for advancing automated underwater habitat mapping and ecological monitoring.

1. Introduction

Kelp forests are ecologically important but increasingly threatened, while underwater imagery and kelp morphology create difficult segmentation conditions. This study compares three deep learning frameworks and uses SSeg-assisted masks to evaluate underwater kelp detection.

  • Kelp forests support biodiversity, primary production, nutrient cycling, shoreline protection, carbon sequestration, and climate regulation.
  • Ocean warming, marine heatwaves, pollution, overgrazing, and coastal development threaten kelp distribution and abundance.
  • Underwater imagery is degraded by light attenuation, scattering, turbidity, and water-column variability, reducing clarity, contrast, and color fidelity.
  • Kelp segmentation is complicated by dynamic frond morphology, variable imaging conditions, and visually similar understory and subcanopy macroalgae.
  • The study compares ResNet34-U-Net, ResNet50-DeepLabV3, and ASPP-Transformer frameworks using supervised underwater RGB images paired with manual or SSeg-assisted masks.

2. Datasets

The dataset combines georeferenced underwater imagery from New England and independent subtidal datasets to assess model performance across sites and environmental conditions. Its imagery spans varied kelp density, illumination, turbidity, benthic communities, and scene complexity.

  • The development imagery comprises rocky-reef images from New England and Nova Scotia collected between 1 m and 20 m depth.
  • GoPro HERO7 and SpotX SquidPro video surveys produced 1920 × 1080 RGB imagery with GPS coordinates recorded during acquisition.
  • Figure 1 maps georeferenced collections across Maine Offshore, Long Island Sound, Salem Sound, and Bakers Island, with points colored by mean kelp percent cover.
  • The same-site evaluation used 50 images per site selected to represent diverse underwater conditions and kelp-cover characteristics.
  • External evaluation included Nova Scotia imagery from Charlotte Island and Woody Island and 50 images from four Isles of Shoals sites in New Hampshire.

3. Annotation and Image Pre-Processing

The annotation workflow combines manual labeling with SSeg-assisted mask generation, inspection, and refinement. Binary kelp-background masks were assembled into a large dataset covering diverse kelp conditions for supervised segmentation.

  • 623 Gooseberry Islands images were fully segmented with v7Darwin, while the remaining images used SSeg to transform sparse point annotations into dense masks.
  • SSeg combines active sampling with SAM2 and superpixel segmentation to propagate user-provided points or bounding boxes into dense masks.
  • Annotators placed positive prompts on kelp and occasional negative prompts on rocks, sand, artifacts, shadows, and benthic backgrounds.
  • Generated masks were visually inspected and manually refined to correct boundary errors and false detections in complex underwater scenes.
  • The dataset contained 3,395 underwater image-mask pairs, with validation and test sites selected independently to assess cross-site generalization and reduce spatial leakage.
  • Representative masks show high, moderate, low, and very low kelp cover; seagrass in the very-low-cover example is not labeled.

4. Methodology

The methodology uses a site-wise data split, 512 × 512 image patches, and comparative training of three semantic segmentation architectures. Performance is evaluated with multiple pixel-level metrics.

  • The workflow comprises dataset preparation, annotation, patch-based sampling, model training, and quantitative evaluation using site-wise train, validation, and test splits.
  • Three semantic segmentation models were trained: ResNet34-U-Net, ResNet50-DeepLabV3, and ASPP-Transformer.
  • Training used image patches of size 512 × 512 pixels.

4.1. Data Preprocessing

The preprocessing workflow used site-wise splitting, class-balanced patch sampling, and augmentation to improve generalization while reducing leakage and imbalance.

  • Site-wise dataset splitting: Site-wise splitting separated training, validation, and test data by acquisition location to evaluate generalization to unseen environments.This avoided data leakage and reduced the chance of learning site-specific characteristics.
  • Patch-based sampling strategy: 512 × 512 patches were extracted because full-resolution images were computationally inefficient and often contained kelp-poor, imbalanced regions.Patch extraction was performed separately within each dataset split to prevent cross-subset leakage.
  • Patch-based sampling strategy: Patches were sampled from each image, with kelp-rich patches retained preferentially and background patches retained at a low probability.The threshold for kelp-rich sampling was a kelp fraction greater than or equal to 1%.
  • Data augmentation: Training augmentation combined geometric and photometric transformations to simulate variability in underwater conditions and improve model robustness.Transformations included flips, rotations, affine changes, brightness and contrast adjustments, noise, CLAHE, and blurring.
  • Preprocessing objectives: The pipeline aimed to reduce class imbalance, expand effective training samples, preserve spatial and environmental diversity, prevent leakage, and support cross-site generalization.Its intended outcome was learning site-independent kelp features with evaluation on unseen locations.

4.2. Deep Learning Architectures

Three supervised architectures represented different contextual learning strategies: ResNet34-U-Net, ResNet50-DeepLabV3, and a hybrid ResNet50-ASPP-Transformer.

  • ResNet34-U-Net: ResNet34-U-Net used a ResNet34 encoder and U-Net decoder with skip connections to preserve spatial detail and reconstruct kelp boundaries.A final 1 × 1 convolution and sigmoid produced the binary segmentation output.
  • ResNet50-DeepLabV3: ResNet50-DeepLabV3 used a ResNet50 backbone and atrous convolutions to capture complex underwater textures while retaining spatial resolution.Its ASPP module used parallel dilated convolutions to extract contextual features under illumination changes, turbidity, and complex benthic backgrounds.
  • ASPP-Transformer: The hybrid ResNet50-ASPP-Transformer combined a ResNet50 encoder with ASPP multi-scale context extraction and a Transformer block for long-range spatial dependencies.Its decoder upsampled features and integrated intermediate encoder features before producing a binary mask.

4.3. Hyperparameter

All architectures followed a standardized training and evaluation workflow using common image subsets, preprocessing, augmentation, and assessment procedures.

  • Common training protocol: All models were trained on standardized 512 × 512 image subsets with identical preprocessing, augmentation, training, and evaluation procedures.This design was intended to make architectural differences, rather than workflow inconsistencies, account for accuracy differences.
  • Training hyperparameters: Model-specific training parameters were summarized in Table 1, while common optimization, loss, normalization, threshold, evaluation, augmentation, and reproducibility settings were provided separately.The table compared hyperparameters for ResNet34–U-Net, ResNet50–DeepLabV3, and ASPP–Transformer.

4.4. Performance Metrics

Performance was assessed with complementary pixel-level metrics that measure correctness, overlap, positive-pixel quality, detection sensitivity, and balanced classification performance.

  • Pixel Accuracy: Pixel Accuracy measures the proportion of correctly classified pixels among all pixels but may be biased by class imbalance.It provides an overall pixel-level assessment of model performance.
  • Intersection over Union: Intersection over Union evaluates overlap between predicted segmentation and ground-truth regions.It is also known as the Jaccard Index.
  • Precision: Precision measures correctly predicted positive pixels among all predicted positive pixels, reflecting the model’s ability to minimize false positives.The metric focuses on the quality of predicted kelp pixels.
  • Recall: Recall measures the proportion of actual positive pixels correctly identified and indicates the model’s ability to detect relevant regions.Recall is also referred to as sensitivity.
  • F1-score: The F1-score is the harmonic mean of Precision and Recall, providing a balanced measure for imbalanced datasets.It combines positive-pixel precision and detection sensitivity.
  • Dice Coefficient: The Dice Coefficient measures similarity between predicted segmentation and ground truth, while the metric equations assess pixel-level accuracy, overlap, and classification effectiveness.Together, the metrics provide complementary evaluations of segmentation quality.

5. Results

Across independent and external underwater imagery, the models showed differing segmentation accuracy, stability, site generalization, and kelp-cover agreement. ResNet50-DeepLabV3 was generally the most consistent, while ASPP-Transformer performance varied more across challenging conditions.

  • Independent test sites: 0.7120 Dice and 0.6267 IoU were achieved by ResNet50-DeepLabV3 on independent test data, the strongest segmentation results among the models.The test set contained 1,556 annotated image patches from held-out Bald Porcupine and Ram Island sites.
  • Independent test sites: 0.8528 pixel accuracy was achieved by ASPP-Transformer, despite lower Dice and IoU than the convolution-based models.Its Dice and IoU were lower than those of ResNet50-DeepLabV3 and ResNet34-U-Net.
  • Independent test sites: ResNet34-U-Net and ResNet50-DeepLabV3 more closely matched ground-truth masks, while DeepLabV3 provided improved boundary delineation under varying underwater conditions.The comparison included complex kelp morphology, overlapping fronds, and heterogeneous benthic backgrounds.
  • Generalization to unseen survey sites: Across 19 unseen study sites, ResNet50-DeepLabV3 generally delivered the best and most consistent performance, with Dice values of approximately 35–82% and IoU values of about 20–70%.The strongest results occurred at Sites 2, 9, 10, and 15.
  • Generalization to unseen survey sites: ASPP-Transformer showed larger site-to-site variability, with Dice below approximately 20% and IoU near 5–10% at Sites 18 and 19.The reported reductions were associated with illumination, water clarity, shadows, complex benthic backgrounds, and sparse kelp distribution.
  • Transferability to external data sets: External datasets revealed transferability differences: all models underestimated dense Woody Island kelp, while predictions were closer to ground truth for New Hampshire imagery.For Woody Island, estimated cover ranged from 9.8–15.4% against 27.8% ground truth and 15.2–33.4% against 51.8% ground truth.

6. Discussion

The discussion identifies ResNet50-DeepLabV3 as the most reliable architecture while emphasizing annotation, validation, and geographic-transfer constraints. The study provides Kelp-O-Tron as an open tool, but recommends local validation and broader training data for new conditions.

  • ResNet50-DeepLabV3 consistently achieved the strongest overall performance and is recommended for automated nearshore subsurface kelp segmentation.Its Atrous Spatial Pyramid Pooling module captured multi-scale features and supported delineation of fragmented kelp patches and fine-scale boundaries.
  • Model generalization is constrained by domain shift because the training data primarily represent northeastern U.S. coastal waters.Differences in morphology, species composition, water clarity, illumination, acquisition conditions, and season may affect transferability.
  • 3,395 image–mask pairs were generated through an SSeg-assisted semi-automated annotation workflow with targeted manual review.Overlapping kelp, variable illumination, motion blur, and visually similar benthic features often required manual refinement.
  • Independent and external evaluation showed that trained models identified kelp across underwater environments beyond the development dataset.The authors recommend validating predictions with representative annotated images and fine-tuning with local training data when possible.
  • Kelp-O-Tron provides best-performing model weights and an inference workflow for generating pixel-level kelp masks and kelp cover estimates.Users are encouraged to validate predictions on a small annotated subset before large-scale application.

7. Conclusion

The study compares three deep learning architectures for underwater kelp segmentation using high-resolution RGB imagery and SSeg-assisted masks. ResNet50-DeepLabV3 provided the most balanced and consistent performance, supporting automated kelp mapping while retaining sensitivity to environmental variability.

  • Three deep learning semantic segmentation architectures were evaluated using high-resolution underwater RGB imagery and a large SSeg-assisted image–mask dataset.The evaluation covered varying underwater optical conditions and geographically independent sites.
  • ResNet50-DeepLabV3 achieved the most balanced and consistent performance across independent test sites, with higher Dice and IoU scores and stable kelp cover estimation.External sites included Woody Island, Charlotte, and New Hampshire.
  • Deep learning semantic segmentation provides a promising framework for automated subsurface kelp mapping and ecological habitat assessment.The study also provides tools for using the model directly or retraining it for other systems.

Supplementary/Appendix Tables

The supplementary materials document study sites, shared training settings, model architectures, and qualitative comparisons used to support the kelp-segmentation evaluation.

  • Study sites: Table A1 summarizes study sites used for model development, evaluation, and external validation, including mean kelp cover and image–mask counts.
  • Study sites: Tables A1 site records include site name, sampling year, latitude, longitude, and minimum depth.
  • Training and evaluation: All three models shared augmentation, ImageNet normalization, Dice and IoU evaluation, validation-based threshold selection, and random seed 42.
  • Model architectures: Figures A1–A3 depict the ResNet34-U-Net, ResNet50-DeepLabV3, and ASPP-Transformer architectures.
  • Qualitative comparisons: Figures A4–A6 compare predicted kelp segmentations with original images and ground-truth masks at Great Duck, Bakers, and North Gooseberry sites.Predicted kelp cover percentages and pixel-wise accuracy statistics are displayed for each model.
Loading 2608.24594v1…