PaperlayerView exact revision on arXiv

Source-linked AI summary

A Gated Cross-domain Collaborative Network for Underwater Object Detection

Linhui Dai, Hong Liu, Pinhao Song, Mengyuan Liu

arXiv:2306.14141v1cs.CV

TL;DR

Underwater enhancement can improve visibility, but using enhanced images alone may alter object details, while prior joint training can require paired images that are difficult to obtain. GCC-Net combines raw and enhanced image features through cross-domain interaction and gated fusion. It achieves state-of-the-art performance on four underwater datasets.

  • Problem

    Enhanced images can improve visibility but may remove or alter object details, while joint UIE–UOD training requires paired clear and distorted images that are difficult to obtain.

    Source: §I · pp. 1–2

  • Method

    GCC-Net generates real-time enhanced images and uses cross-domain feature interaction and gated fusion to combine enhanced- and raw-domain information.

    Source: §I · p. 2

  • Results

    GCC-Net achieves state-of-the-art performance on DUO, Brackish, TrashCan, and WPBB; on DUO, it reaches 69.1% AP versus AutoAssign’s 66.1%.

    Source: §I · p. 2, 1) Results on DUO · p. 7, §V · p. 12

  • Takeaways & Limitations

    The reported results support cross-domain interaction and fusion as a UOD approach evaluated across four underwater datasets.

    Source: §V · p. 12

Abstract

from arXiv · show

Underwater object detection (UOD) plays a significant role in aquaculture and marine environmental protection. Considering the challenges posed by low contrast and low-light conditions in underwater environments, several underwater image enhancement (UIE) methods have been proposed to improve the quality of underwater images. However, only using the enhanced images does not improve the performance of UOD, since it may unavoidably remove or alter critical patterns and details of underwater objects. In contrast, we believe that exploring the complementary information from the two domains is beneficial for UOD. The raw image preserves the natural characteristics of the scene and texture information of the objects, while the enhanced image improves the visibility of underwater objects. Based on this perspective, we propose a Gated Cross-domain Collaborative Network (GCC-Net) to address the challenges of poor visibility and low contrast in underwater environments, which comprises three dedicated components. Firstly, a real-time UIE method is employed to generate enhanced images, which can improve the visibility of objects in low-contrast areas. Secondly, a cross-domain feature interaction module is introduced to facilitate the interaction and mine complementary information between raw and enhanced image features. Thirdly, to prevent the contamination of unreliable generated results, a gated feature fusion module is proposed to adaptively control the fusion ratio of cross-domain information. Our method presents a new UOD paradigm from the perspective of cross-domain information interaction and fusion. Experimental results demonstrate that the proposed GCC-Net achieves state-of-the-art performance on four underwater datasets.

I. INTRODUCTION

Underwater enhancement can improve visibility but may remove object details, while joint UIE–UOD training requires paired clear and distorted images. GCC-Net instead interacts with and fuses raw- and enhanced-domain features, and achieves state-of-the-art performance on four underwater datasets.

I. INTRODUCTION

  • Enhanced images may alter critical patterns or introduce artifacts, whereas raw images retain texture that can aid object identification.The paper frames low contrast and low light as conditions where the two domains provide complementary information.Source: §I · pp. 1–2
  • Earlier approaches use enhancement as preprocessing or jointly train UIE and UOD; the paper presents feature interaction and fusion as a third approach.The joint-training approach is described as requiring matched degraded and clear images, which are difficult to obtain in practice.Source: §I · p. 1, Figure 1 · p. 2
  • The paper identifies two collaboration challenges: obtaining promising enhanced images and interacting and fusing features from the raw and enhanced images.It notes that retraining UIE models is difficult without abundant paired images and because synthetic data may not represent all underwater conditions.Source: §I · p. 2
  • The proposed paradigm processes raw and UIE-enhanced images together, then interacts and fuses features from both domains in one framework.Enhanced images improve visibility, while raw images preserve scene characteristics and object texture.Source: §I · pp. 1–2, Figure 1 · p. 2
  • Water-MSR provides real-time enhancement to improve underwater-object visibility in low-light areas.It is based on multiscale Retinex and IMSRCP, and is accelerated using progressively smaller images and recursive fast filtering.Source: §I · p. 2
  • The cross-domain feature interaction module mines complementary information, while gated fusion adaptively controls the contribution of cross-domain features.The gate is intended to mitigate contamination from unreliable generated results.Source: §I · p. 2
  • GCC-Net achieves state-of-the-art performance on DUO, Brackish, TrashCan, and WPBB.The introduction reports results across all four public datasets.Source: §I · p. 2

II. RELATED WORKS

Prior UOD research includes data augmentation, feature-extraction improvements, and enhancement-based methods, but preprocessing can lose useful detail and joint training may require difficult-to-obtain paired images. This paper integrates UIE and UOD and explores complementary information through interaction and fusion of enhanced and raw imagery.

A. Underwater Object Detection

  • Prior UOD work uses data augmentation,,,, and stronger feature extraction,; AUV-mounted detectors should be lightweight and real-time.The cited examples include proposal-level fusion and a two-stage detector with high-quality proposals.Source: A. Underwater Object Detection · pp. 2–3
  • Using enhanced images as preprocessing can impair detection when enhancement removes useful details or introduces artifacts, [2].Liu et al. instead use multi-task optimization, but require distorted-clear image pairs for UIE training.Source: A. Underwater Object Detection · p. 3
  • The proposed framework combines enhanced-image visibility with raw-image texture and explores their complementary information through interaction and fusion.The paper presents this as an integrated UIE–UOD framework.Source: A. Underwater Object Detection · p. 3

B. Underwater Image Enhancement

  • Although UIE methods have been proposed for downstream vision tasks, the authors report that they had not been applied to UOD and that UIE-to-UOD dataset domain shift makes retraining challenging.The passage also notes the absence of corresponding clear-scene datasets in UOD datasets.Source: B. Underwater Image Enhancement · p. 3
  • Rather than treating deep-learning UIE as offline preprocessing, the method integrates UIE and UOD in one framework and uses cross-domain interaction and fusion.The paper describes this as the first UOD approach from that perspective.Source: B. Underwater Image Enhancement · p. 3

III. METHOD

The method processes raw and water-MSR-enhanced images in parallel, interacting their features and controlling cross-domain fusion before detection. Water-MSR adapts multi-scale Retinex for real-time enhancement through recursive fast filtering.

A. Overview

  • The framework combines online water-MSR enhancement, cross-domain feature interaction, gated feature fusion, and a detection head.It uses four GFF modules across four stages.Source: A. Overview · pp. 3–4, Figure 2 · p. 4
  • Raw and enhanced image features interact through CFI, then GFF controls their fusion to limit noise from unreliable enhanced images.The four stages use {2, 2, 6, 2} CFI modules, respectively.Source: A. Overview · pp. 3–4

B. Online Underwater Image Enhancement Model

  • Because water quality varies and paired training samples are difficult to obtain, the method uses existing physical UIE models rather than developing a new UIE method.Water-MSR combines multi-scale Retinex and IMSRCP and recursively invokes fast filtering for real-time integration.Source: B. Online Underwater Image Enhancement… · p. 4
  • Water-MSR replaces Gaussian filtering with recursive image-pyramid fast filtering to accelerate multi-scale Retinex while retaining its image-enhancement functions.The method uses three scales, σ ∈ [30, 150, 300], with equal weights; recursive downsampling continues until σ_min = 10.Source: B. Online Underwater Image Enhancement… · pp. 4–5
  • Water-MSR first applies color pre-correction to equalize degraded images and reduce dominant color, then performs weighted multi-scale Retinex fusion.The pre-correction extracts channel minima and maxima from raw-image statistics.Source: B. Online Underwater Image Enhancement… · p. 4

C. Cross-domain Feature Interaction Module

The CFI module uses cross-attention to exchange information between raw and enhanced image features while retaining each domain's own information. Shifted windows make this interaction tractable on high-resolution images.

C. Cross-domain Feature Interaction Module

  • Enhanced features provide appearance and discriminative information, while raw features preserve scene characteristics and object texture; CFI uses cross-attention to exchange these complementary cues.Each domain attends to the other domain's keys and values while retaining its own information through a residual connection.Source: C. Cross-domain Feature Interaction… · p. 5
  • The two image streams are divided into non-overlapping local windows before feature projection and cross-attention.The default window size is M = 7, and patch-merging layers reduce token counts as the network deepens.Source: C. Cross-domain Feature Interaction… · p. 5
  • CFI replaces standard multi-head cross-attention with shifted-window cross-attention to avoid quadratic image-size complexity on high-resolution images.The module comprises shifted-window MCA, a two-layer MLP with GELU, LayerNorm, and residual connections.Source: C. Cross-domain Feature Interaction… · p. 5
  • Cross-attention scores queries against keys, applies softmax, and uses the resulting weights to aggregate values with relative-position bias.The query, key, and value matrices are formed from the window features using learned projections.Source: C. Cross-domain Feature Interaction… · p. 5

D. Gated Feature Fusion Module

The GFF module merges raw and enhanced features using learned gates that regulate each domain's contribution. Four stages produce a hierarchical representation for detection.

D. Gated Feature Fusion Module

  • GFF uses separate gated maps for raw and enhanced features to regulate their fusion and limit contamination from unreliable water-MSR results.The gated maps have values in [0, 1].Source: D. Gated Feature Fusion Module · p. 6
  • At each stage, GFF gate-weights the two domain features and adds them to produce the stage output.The four GFF stages jointly form a hierarchical representation.Source: D. Gated Feature Fusion Module · p. 6
  • The four GFF stages produce features at different resolutions that can be used with existing detection heads.The outputs are passed to the detection head.Source: D. Gated Feature Fusion Module · p. 6

E. Detection Head

The framework uses a detection head to localize and classify underwater objects, with AutoAssign adopted in the experiments. The method also reports validation with Faster R-CNN and Deformable DETR heads.

E. Detection Head

  • The experiments use AutoAssign as the detection head, whose confidence weighting adjusts the assignment strategy for each object instance.The head is described as adapting to underwater objects of varying sizes and shapes.Source: E. Detection Head · p. 6
  • The framework also achieves good performance with Faster R-CNN and Deformable DETR detection heads.The framework is optimized end-to-end, with loss functions depending on the detection head.Source: E. Detection Head · p. 6

IV. EXPERIMENTS

The experiments use four public underwater detection datasets spanning marine life, brackish-water scenes, underwater trash, and plastic debris.

A. Datasets

  • DUO contains 7,782 images and 74,515 instances across echinus, holothurian, starfish, and scallop, with four image resolutions.Its 6,671 training and 1,111 test images come from a refined version of UTDAC2020.Source: A. Datasets · p. 6
  • Brackish contains six classes and is split into 9,967 training, 1,467 validation, and 1,468 test images at 960 × 540 resolution.It is described as the first annotated underwater image dataset captured in temperate brackish waters.Source: A. Datasets · p. 6
  • TrashCan covers 16 classes, including trash, ROVs, flora, and fauna, with 6,008 training and 1,204 test images.It is described as the first instance-segmentation annotated dataset of underwater trash.Source: A. Datasets · p. 6
  • WPBB contains 900 annotated images of plastic bags and bottles, split into 720 training and 180 test images.The passage describes it as an in-water plastic-bag and plastic-bottle detection dataset.Source: A. Datasets · p. 7

B. Experimental Details and Evaluation Metrics

The experiments use COCO-style AP metrics and compare GCC-Net with generic and underwater detectors. On DUO, GCC-Net is reported to outperform the compared methods while maintaining comparable inference speed.

B. Experimental Details and Evaluation Metrics

  • The evaluation uses AP, AP50 at IoU 0.5, and AP75 at IoU 0.75; AP averages IoU thresholds from 0.5 to 0.95 in 0.05 increments.Training uses multi-scale inputs, a 36-epoch schedule, and horizontal flipping as the only data augmentation.Source: B. Experimental Details and Evaluation… · p. 7

1) Results on DUO:

  • GCC-Net reaches 69.1% AP on DUO, 3.0% above AutoAssign (66.1% vs 69.1%), and is reported best among compared generic and underwater detectors.The reported margins include 4.3% over DetectoRS and 6.1% over SWIPENET.Source: 1) Results on DUO · p. 7
  • GCC-Net runs at 15.6 FPS on a GeForce RTX 3090 with batch size 1 and single-scale testing, which the authors report meets real-time operational requirements.The authors also state that objects in low-contrast regions become clearly visible.Source: 1) Results on DUO · p. 7

2) Results on TrashCan and WPBB:

Across debris datasets, GCC-Net reports strong detection results, while Brackish results show a speed–accuracy trade-off against ERL-Net. Ablations examine the UIE model and the interaction between raw and enhanced image domains.

3) Results on Brackish:

  • GCC-Net has higher inference speed than ERL-Net on Brackish, while ERL-Net’s multi-level feature approach improves its detection performance at the cost of efficiency.The authors identify inference speed as relevant to practical deployment on underwater robotic platforms.Source: 3) Results on Brackish · p. 7

D. Ablation Studies

  • The DUO ablations use AutoAssign as their baseline, chosen for its reported performance and efficient inference speed.The experiments evaluate the proposed method’s components and image-domain choices.Source: D. Ablation Studies · p. 7

2) Results on TrashCan and WPBB:

  • GCC-Net achieves 41.3% AP on TrashCan, 4.3% above ERL-Net, and 81.0% AP on WPBB, 0.8% above SABL.The authors report strong performance across the datasets’ distinct underwater environments without retraining the UIE model.Source: 2) Results on TrashCan and WPBB · p. 7, Table II · p. 8

3) Results on Brackish:

  • On Brackish, GCC-Net reaches 80.5% AP and 98.3% AP50 but does not surpass ERL-Net; the authors associate this with reduced UIE efficacy in gray or blackish water.They note that Brackish differs from the blue or green appearance typical of underwater environments.Source: 3) Results on Brackish · p. 7, Table II · p. 8

1) Effect of Underwater Image Enhancement (UIE) Model:

  • Using enhanced images alone performs worse than using raw images; water-MSR reduces per-image processing time from 20 seconds for IMSRCP to 0.12 seconds.The passage attributes the weaker enhanced-only result to possible loss of critical patterns or introduction of noise, and reports MMLE at approximately 1.72 seconds per image.Source: Table III · p. 8, 1) Effect of Underwater Image… · p. 9
  • On DUO, duplicating raw input gives 67.0% versus 66.1% for one raw image, color jittering gives 65.2% AP, and raw-plus-enhanced inputs give 69.1% with water-MSR or 67.8% with MMLE.The authors interpret these comparisons as evidence that improvement is not simply due to a changed network structure or more inputs, but to interaction and fusion of raw and enhanced images.Source: Table IV · p. 8, 1) Effect of Underwater Image… · p. 9

2) Effect of Cross-domain Feature Interaction (CFI) Module:

The CFI module is compared with direct multilayer feature addition, and Grad-CAM visualizations are used to examine its attention to object regions.

2) Effect of Cross-domain Feature Interaction (CFI) Module:

  • CFI achieves 68.1% versus 65.1% for multilayer feature addition, a 3.0% gain, while retaining AutoAssign’s parameter count.The alternative uses two independent backbones and significantly increases the number of parameters.Source: 2) Effect of Cross-domain Feature… · p. 10
  • Grad-CAM visualizations show greater emphasis on object regions with CFI, particularly in low-contrast areas.The figure contrasts feature maps without CFI in the second row and with CFI in the third row.Source: 2) Effect of Cross-domain Feature… · p. 10

3) Effect of Gated Feature Fusion (GFF) Module:

GCC-Net improves performance across three detection heads, while gated feature fusion outperforms direct feature addition. Its FLOPs increase, but its parameter count remains unchanged.

3) Effect of Gated Feature Fusion (GFF) Module:

  • Gated feature fusion reaches 69.1% versus 68.6% for direct feature addition, a 0.5% gain.The gated module controls each domain feature’s fusion rate to mitigate contamination from unreliable information.Source: 3) Effect of Gated Feature Fusion… · p. 10

4) Effect of Different Detection Head:

  • GCC-Net improves Faster R-CNN by 4.3 points (65.6% vs 61.3%), Deformable DETR by 3.2 points (66.9% vs 63.7%), and AutoAssign by 3.0 points (69.1% vs 66.1%).Across the baselines, the method is reported to deliver nearly a 5-point improvement.Source: 4) Effect of Different Detection Head · pp. 10–11
  • GCC-Net increases FLOPs relative to the baselines while keeping the parameter count unchanged.The passage associates the FLOPs increase with greater computational resources during training.Source: 4) Effect of Different Detection Head · p. 11
  • The ablations identify online water-MSR, cross-domain feature interaction, and gated feature fusion as essential to GCC-Net’s final performance.Source: 4) Effect of Different Detection Head · p. 11

E. Error Analysis

GCC-Net is reported to achieve state-of-the-art performance across four underwater datasets. Error analysis reports gains across object scales, while qualitative examples show performance in varied settings and persistent difficulty with heavily occluded objects.

E. Error Analysis

  • At IoU=0.75, C75 is 76.2% on large objects (4.2% above AutoAssign) and 77.2% on medium objects (3.1% above); small-object AP75 is 46.8% versus AutoAssign’s 45.6%.The analysis also reports small-subset gains for AP50, Loc, Oth, and BG.Source: E. Error Analysis · p. 11

F. Qualitative Results

  • Qualitative results show small-object visibility gains in low-contrast areas and detection across varied water environments, but heavily occluded objects remain difficult.The cited figures show examples from DUO, Brackish, TrashCan, and WPBB.Source: F. Qualitative Results · pp. 11–12, Figure 7 · p. 12

V. CONCLUSION

  • The conclusion describes GCC-Net as an end-to-end integration of UIE and UOD that mines and fuses information from raw and enhanced image domains.It identifies online water-MSR, cross-domain feature interaction, and gated feature fusion as its components.Source: §V · p. 12
  • GCC-Net achieves state-of-the-art performance against recent GOD and UOD methods on DUO, Brackish, TrashCan, and WPBB.The conclusion presents cross-domain interaction and fusion as a new perspective for UOD.Source: §V · p. 12
Loading 2306.14141v1…