PaperlayerView exact revision on arXiv

Source-linked AI summary

Downstream Task Inspired Underwater Image Enhancement: A Perception-Aware Study from Dataset Construction to Network Design

Bosen Lin, Feng Gao, Yanwei Yu, Junyu Dong, Qian Du

arXiv:2603.01767v1cs.CVeess.IV

TL;DR

Underwater enhancement methods often optimize human visual quality despite downstream recognition needing preserved task-relevant structure. DTI-UIE constructs task-oriented references and guides a dual-branch enhancer with task priors and task-driven supervision; the resulting approach improves downstream segmentation, detection, and instance segmentation, while its dataset selection remains primarily segmentation-driven.

  • Problem

    Existing UIE methods and datasets emphasize human visual perception, while enhancement can blur or distort high-frequency structures needed by downstream recognition.

  • Method

    DTI-UIE combines the automatically constructed TI-UIED dataset with a dual-branch network, task-specific priors, Task-Driven Perceptual loss, and three-stage training.

  • Results

    DTI-UIE consistently outperforms existing UIE approaches across semantic segmentation, object detection, and instance segmentation.

  • Takeaways & Limitations

    Task-oriented enhancement can produce preprocessing images beneficial for downstream underwater vision tasks while remaining comparable on human-perception image-quality metrics.

  • Takeaways & Limitations

    TI-UIED reference selection primarily uses semantic segmentation performance across an architecturally varied ensemble, leaving dependence on that task and protocol.

Abstract

from arXiv · show

In real underwater environments, downstream image recognition tasks such as semantic segmentation and object detection often face challenges posed by problems like blurring and color inconsistencies. Underwater image enhancement (UIE) has emerged as a promising preprocessing approach, aiming to improve the recognizability of targets in underwater images. However, most existing UIE methods mainly focus on enhancing images for human visual perception, frequently failing to reconstruct high-frequency details that are critical for task-specific recognition. To address this issue, we propose a Downstream Task-Inspired Underwater Image Enhancement (DTI-UIE) framework, which leverages human visual perception model to enhance images effectively for underwater vision tasks. Specifically, we design an efficient two-branch network with task-aware attention module for feature mixing. The network benefits from a multi-stage training framework and a task-driven perceptual loss. Additionally, inspired by human perception, we automatically construct a Task-Inspired UIE Dataset (TI-UIED) using various task-specific networks. Experimental results demonstrate that DTI-UIE significantly improves task performance by generating preprocessed images that are beneficial for downstream tasks such as semantic segmentation, object detection, and instance segmentation. The codes are publicly available at https://github.com/oucailab/DTIUIE.

I. INTRODUCTION

Underwater image degradation challenges downstream recognition, while conventional enhancement often optimizes human appearance rather than task-relevant details. DTI-UIE addresses this gap through task-oriented dataset construction, task-aware enhancement, and task-driven supervision.

  • Underwater absorption, reflection, and scattering reduce clarity and distort images, creating substantial challenges for downstream vision tasks.
  • Existing UIE methods may improve visual appearance yet fail to improve downstream performance because enhancement objectives are misaligned with recognition requirements.
  • TI-UIED automatically selects enhanced images that produce the most consistent segmentation improvement across diverse networks as task-driven ground truth.
  • DTI-UIE combines a dual-branch enhancement network with task-relevant priors, preserving semantic features while restoring fine-grained edges and textures.
  • A three-stage training strategy uses pixel-level and Task-Driven Perceptual loss to align enhancement with downstream task requirements.

II. RELATED WORK

Related work spans traditional, deep-learning, and perception-inspired UIE methods, but available datasets and objectives remain centered on human visual quality. The paper therefore advocates combining task-oriented data construction with downstream-task guidance.

  • Traditional UIE methods address color, contrast, and frequency components, but can introduce unnatural colors or textures.
  • Deep-learning UIE commonly learns mappings from underwater images to manually selected or simulated references, while real paired data remain difficult to establish.
  • Existing UIE datasets are designed for human visual perception without considering enhancement effects on downstream tasks.
  • Task-driven UIE methods incorporate detection, semantic, or feedback information, but applying them underwater remains challenging because references and feature requirements differ from in-air settings.
  • This work responds by constructing a downstream-task dataset and using downstream tasks as losses and priors for task-specific enhancement.

III. PROPOSED TASK-INSPIRED UIE DATASET

TI-UIED adapts human-perception-based reference selection to downstream vision by automatically choosing enhanced underwater images according to task-network performance. The dataset supports task-oriented enhancement training and evaluation.

  • Dataset construction: TI-UIED simulates human reference selection with semantic segmentation networks rather than manual annotations or perceptual voting.Enhanced candidates are evaluated through task performance to approximate subjective selection.
  • Dataset characteristics: The dataset is based on SUIM and contains 1635 real underwater images with pixel-level annotations spanning eight semantic categories.Its scenes vary in water types, lighting, and target types.
  • Dataset construction: Candidate references are generated from five traditional methods, four deep-learning methods, and raw images, with the best-performing candidate selected when applicable.The candidate methods include CLAHE, Fusion, ULAP, WWPE, WB, SGUIE, TOPAL, TUDA, and UColor.
  • Framework overview: The proposed framework uses an enhancement network and two task networks within a three-stage training process.The prior task network is trained first; the enhancement network then uses TDP and pixel losses while task networks are frozen, followed by task-network updating.
  • Framework overview: The enhancement network combines a feature restoration branch for semantic features with a detail enhancement branch for edges, textures, and structural details.Task-specific priors are incorporated through TA-CTB to align enhancement with recognition requirements.

B. The Feature Restoration Branch

The Feature Restoration Branch targets task-relevant global semantics while preserving the multiscale structure needed for enhancement. Its encoder–decoder uses Conv-attention Transformer Blocks and task-aware blocks to recover recognition-critical information.

  • Motivation and role: The FRB addresses the mismatch between human-perception enhancement and downstream tasks by recovering target-relevant high-frequency features.It is designed to restore details that support detection and segmentation rather than only visual quality.
  • Architecture: The FRB uses a UNet-shaped encoder–decoder with Conv-attention Transformer Blocks to extract and restore image features from local to global levels.A convolutional stem first projects the raw image into a 32-channel feature map.
  • Multiscale processing: Encoder representations use channel sizes 32, 64, and 128 at resolutions H × W, H/2 × W/2, and H/4 × W/4.At the bottleneck, task-related priors are incorporated through TA-CTBs with 256 channels at H/8 × W/8.
  • Decoder: The decoder upsamples through TA-CTBs and a CTB while skip connections fuse aligned encoder features at corresponding scales.These connections combine decoder representations with encoder information after spatial upsampling.

C. Task-Aware Conv-attention Transformer Block

TA-CTB injects task-relevant priors into attention so the enhancement network can emphasize features useful for downstream recognition. TA-CAM mixes image and task representations while retaining global and local spatial relationships.

  • Purpose: TA-CTB is a task-driven module that uses priors from a task network to guide enhancement toward recognition-informative features.It is motivated by selective attentional emphasis in human visual perception.
  • Task-aware feature mixing: TA-CAM aligns image features and task priors, converts them into tokens, and enables cross-model interaction between heterogeneous representations.The input image and task features are dimension-aligned before patch embedding.
  • Spatial modeling: Convolutional positional encoding and relative position encoding supplement attention with spatial information across scales.The design combines global dependencies with fine-grained spatial relationships.
  • Task-aware attention: The attention mechanism forms mixed queries from image and task tokens before applying factorized self-attention to model long-range dependencies.The resulting task-aware attention map represents cross-modal relationships between representations.
  • Output: TA-CTB outputs feature maps that capture global dependencies and fine-grained relationships with task priors for task-relevant enhancement.These outputs are passed through the decoder as the enhancement network continues processing.

D. The Task Network

The task network is an UNet-shaped semantic segmentation model with a VGG-16 encoder and decoder skip connections. Its multiscale decoder features provide priors for enhancement and supervision for the task-driven perceptual loss.

  • Architecture: The task network uses a VGG-16 feature extractor in an UNet-shaped encoder–decoder architecture.The encoder is divided into four convolutional blocks that progressively extract hierarchical representations.
  • Decoder: Each decoder stage upsamples by 2×, fuses the corresponding encoder feature through a skip connection, and refines the result with two convolutional layers.A final 1 × 1 convolution produces the prediction map.
  • Task priors: The task network supplies multiscale decoder features as task-relevant priors for the enhancement network.Features from three decoder levels are injected into the enhancement network through TA-CTB.
  • Task supervision: The first decoder feature is additionally used to construct task-driven perceptual supervision through the TDP loss.This links task-network representations to the enhancement objective.

E. The Detail Enhancement Branch

The detail enhancement branch preserves fine textures and task-relevant structures that human-perception-oriented enhancement may damage. DTI-UIE combines high-resolution feature recovery, task-driven perceptual supervision, and staged task-network refinement.

  • The Detail Enhancement Branch: The independent detail enhancement branch operates at raw-image resolution to preserve fine textures because encoder-decoder structures can lose spatial details.It addresses noise, edge blur, and texture degradation that may reduce downstream accuracy.
  • The Detail Enhancement Branch: Four residual convolution blocks combine convolutions, channel attention, and pixel attention to recover and fuse important high-resolution features.Channel attention models global dependencies, while pixel attention emphasizes spatial details.
  • Multi-stage Training: The three-stage framework alternately optimizes task and enhancement networks so task-relevant priors and enhancement representations are mutually refined.Stage one extracts task priors, stage two trains enhancement with raw images, references, and priors, and stage three updates the task network.
  • Task-Driven Perceptual Loss: The task-driven perceptual loss aligns enhanced and ground-truth images in the downstream task network’s feature space, unlike pixel loss alone.The loss uses a trainable UNet segmentation network with a VGG-16 backbone and compares feature representations.
  • Multi-stage Training: Stage three trains the task network on enhanced, ground-truth, and patch-mixed images to learn broader task-relevant features and improve generalization.Patch mixing forces the network to process clean and enhanced regions within one image.

V. EXPERIMENTS

The experiments evaluate DTI-UIE as preprocessing for three downstream recognition tasks: semantic segmentation, instance segmentation, and object detection.

  • Experiments: The study assesses DTI-UIE as an image-preprocessing method for semantic segmentation, instance segmentation, and object detection.

A. Implementation Details

Implementation uses TI-UIED-trained DTI-UIE preprocessing and compares downstream segmentation across datasets, architectures, and enhancement methods. DTI-UIE improves reported segmentation results and preserves semantic information in visual comparisons.

  • Implementation Details: DTI-UIE is trained on TI-UIED with 256×256 inputs, AdamW optimization, and a cosine schedule initialized at 1e-4.The trained network preprocesses task-specific images before downstream networks are retrained.
  • Implementation Details: The comparison includes traditional, deep-learning-based, and task-oriented UIE methods using official implementations and pretrained weights.
  • Semantic Segmentation: Semantic segmentation is evaluated with UNet, DeepLabV3, FPN, and DPT on SUIM and LIACi using Dice and mIoU.SUIM covers underwater natural images, while LIACi targets hull-surface defect segmentation.
  • Semantic Segmentation: 74.48, 78.93, 76.85, and 80.64 Dice are achieved on SUIM by UNet, DeepLabV3, FPN, and DPT, respectively.The corresponding mIoU values are 67.61, 67.61, 66.22, and 73.55.
  • Semantic Segmentation: 78.02, 83.13, 81.68, and 82.82 mIoU are achieved on LIACi by the four evaluated segmentation networks.The results are reported as improvements in the more complex water environment, where high-frequency details are more likely to be lost.
  • Visual Analysis: Visual results show DTI-UIE restores target details and textures, while UNet outputs masks similar to ground truth.The comparison indicates preservation of underlying semantic information for segmentation.

C. Comparison on Object Detection

DTI-UIE is evaluated across object detection, instance segmentation, human-perception image-quality metrics, and computational complexity. It reports leading downstream-task results while remaining comparable on no-reference quality metrics and moderate in complexity.

  • Object Detection: Object detection is evaluated on RUOD with Faster R-CNN, SSD, and DINO using AP, AP50, AP75, and object-size-specific AP metrics.
  • Object Detection: DTI-UIE achieves the highest RUOD AP, improving mAP by +0.50 over SemiUIR with Faster R-CNN and +0.60 over the second-best NU2Net model.
  • Instance Segmentation: Instance segmentation on UIIS uses Mask R-CNN and Water Mask R-CNN and evaluates mask AP, including mAP, AP50, AP75, and size-specific scores.
  • Instance Segmentation: +0.40 and +0.30 points of AP improvement are reported over Mask R-CNN and Water Mask R-CNN, respectively.
  • Image Quality Metrics: DTI-UIE is comparable with other UIE methods on UIQM, UCIQE, and NUIQ, which primarily reflect human-perceived underwater image quality.The paper notes that these metrics cannot be associated with downstream tasks.
  • Computational Complexity: DTI-UIE has moderate computational complexity relative to other deep-learning-based UIE methods.The comparison considers model size and FLOPs.

G. Ablation Study

The ablation studies show that DTI-UIE benefits from complementary enhancement branches, task-relevant priors, task-driven perceptual supervision, and separated task networks. These design choices improve downstream segmentation performance and remain robust across task-network structures, while TI-UIED provides task-aligned supervision.

  • Two-branch network: Combining the FRB and DEB branches improves DTI-UIE performance on downstream tasks.FRB enhances target-relevant features, while DEB restores edge information.
  • Target-relevant priors: +3.33 Dice and +0.48 mIoU result from incorporating target-relevant priors through TA-CTB blocks.The comparison replaces decoder TA-CTB blocks with CTB blocks without the target-prior mechanism.
  • Task-inspired perceptual loss: +2.49 mIoU and +2.77 Dice are obtained by adding the TDP loss to the training configuration.The full three-stage training framework performs better than the variant using a pretrained and frozen task network for TDP computation.
  • Task-inspired perceptual loss: 8 × 8 block mixing provides the best balance between diversity and structural coherence for the TDP loss.Larger blocks introduce boundary artifacts, while 1×1 mixing overly smooths high-frequency details.
  • Task-network design: Separating Segpri and Segtask produces more robust task-oriented enhancement than using one task network for both roles.The separated design keeps injected priors stable and avoids overfitting the supervision network to evolving outputs.
  • Task-network design: The three-stage training framework remains robust when Segpri and Segtask use different task-network structures.The reported comparison changes the networks from UNet with VGG-16 to ResNet50 and VGG-13.
  • Dataset effectiveness: Training DTI-UIE on SUIM-E significantly reduces downstream-task performance, while TI-UIED fine-tuning does not improve all existing UIE models consistently.CCMSR and WfDiff show no significant downstream gains after training on TI-UIED, whereas the dataset-level alignment supports DTI-UIE.

VI. DISCUSSION AND LIMITATION

DTI-UIE reframes underwater image enhancement around downstream task performance rather than subjective visual quality. Its task-oriented dataset, complementary network pathways, task priors, and staged training align enhancement with task-relevant structures, but segmentation-driven supervision may introduce task-specific bias.

  • Discussion: DTI-UIE addresses the mismatch between improved subjective visual quality and downstream perception performance by making enhancement task-oriented.The framework selects supervision and evaluates enhancement according to intended downstream usage.
  • Discussion: TI-UIED selects enhancement outputs that consistently improve segmentation across multiple models, while the network combines global restoration with local detail reinforcement.Task priors and a task-driven perceptual objective further align outputs with task-relevant representations.
  • Limitations: Because TI-UIED references are selected through semantic-segmentation performance, they may retain task-specific bias and alternative objectives may rank candidates differently.This boundary is especially relevant in borderline cases where multiple enhancement candidates achieve similar outcomes.
  • Limitations: The study expects multi-dataset construction to reduce dependence on any single dataset and improve robustness and transferability in heterogeneous underwater environments.This is presented as an expected benefit rather than a directly established result.
  • Conclusion: The framework is designed to provide efficient preprocessed images for underwater target-recognition tasks through dual branches, TA-CTB priors, three-stage training, and TDP loss.The training data are automatically constructed by task networks with reference to a human evaluation database.
Loading 2603.01767v1…