PaperlayerView exact revision on arXiv

Source-linked AI summary

Underwater Image Enhancement via Medium Transmission-Guided Multi-Color Space Embedding

Chongyi Li, Saeed Anwar, Junhui Hou, Runmin Cong, Chunle Guo, Wenqi Ren

arXiv:2104.13015v1cs.CV

TL;DR

Underwater attenuation and scattering produce color casts and low contrast, motivating Ucolor’s combination of multi-color-space encoding and medium-transmission-guided decoding. The network adaptively highlights discriminative features and degraded regions, and extensive experiments report superior visual and quantitative performance against state-of-the-art methods.

  • Problem

    Underwater image enhancement must address color casts and low contrast, while existing deep models neglect underwater imaging knowledge.

  • Method

    Ucolor combines an attention-coupled multi-color-space encoder with a medium transmission-guided decoder that emphasizes quality-degraded regions.

  • Results

    Ucolor achieves state-of-the-art performance on several recent benchmarks in both visual quality and quantitative metrics.

  • Takeaways & Limitations

    The method combines diverse color-space representations and reverse medium transmission guidance to incorporate underwater imaging knowledge into deep enhancement.

Abstract

from arXiv · show

Underwater images suffer from color casts and low contrast due to wavelength- and distance-dependent attenuation and scattering. To solve these two degradation issues, we present an underwater image enhancement network via medium transmission-guided multi-color space embedding, called Ucolor. Concretely, we first propose a multi-color space encoder network, which enriches the diversity of feature representations by incorporating the characteristics of different color spaces into a unified structure. Coupled with an attention mechanism, the most discriminative features extracted from multiple color spaces are adaptively integrated and highlighted. Inspired by underwater imaging physical models, we design a medium transmission (indicating the percentage of the scene radiance reaching the camera)-guided decoder network to enhance the response of the network towards quality-degraded regions. As a result, our network can effectively improve the visual quality of underwater images by exploiting multiple color spaces embedding and the advantages of both physical model-based and learning-based methods. Extensive experiments demonstrate that our Ucolor achieves superior performance against state-of-the-art methods in terms of both visual quality and quantitative metrics.

I. INTRODUCTION

Ucolor addresses underwater color casts and low contrast by combining multi-color-space feature representations with medium-transmission guidance. The method reports strong visual and quantitative performance across recent benchmarks.

  • Motivation: Underwater attenuation and scattering cause wavelength-dependent color deviations and low contrast, limiting image quality and scene understanding.Recovering clean underwater images is important for visual quality and accurately understanding the underwater world.
  • Motivation: Existing deep underwater enhancement models often neglect underwater imaging knowledge, limiting their performance relative to current deep visual models.These models commonly adapt networks designed for other visual tasks to underwater images.
  • Proposed approach: Ucolor uses a multi-color-space encoder and attention mechanism to integrate representations from different color spaces and select representative features.The encoder incorporates color-space characteristics into a unified structure and highlights discriminative features.
  • Proposed approach: Its medium transmission-guided decoder emphasizes quality-degraded regions while incorporating physical-model knowledge into a deep network.Medium transmission reflects the percentage of scene radiance reaching the camera.
  • Results: On several recent benchmarks, Ucolor achieves state-of-the-art performance in both visual quality and quantitative metrics.A representative real-image comparison shows improved color, contrast, and naturalness over two compared methods.

II. RELATED WORK

Prior underwater enhancement methods include traditional, physical-model-based, and deep learning-based approaches, each with reported limitations. Ucolor combines domain knowledge with a purpose-designed deep architecture to address these limitations.

  • Traditional methods: Traditional physical-model-free methods adjust pixels or fuse corrected versions, but can omit underwater imaging mechanisms and introduce over- or under-enhancement or artificial colors.Their color correction may be unreliable in diverse and challenging underwater scenes.
  • Physical model-based methods: Physical model-based methods estimate underwater imaging parameters from priors, but can be time-consuming, image-type-sensitive, and challenged by parameter estimation.The blurriness prior, for example, does not always hold for clear underwater images.
  • Deep learning methods: Deep learning-based methods include GAN-based, weakly supervised, multi-scale, and simulated-data approaches for underwater enhancement.Examples use image formation models, unpaired training, multi-scale dense GANs, or simulated underwater images.
  • Deep learning methods: Existing deep models often reuse general-purpose architectures and neglect underwater characteristics, leaving robustness and generalization limited.Reported issues include unstable GAN enhancement and difficulty selecting a suitable model for different water types.
  • Ucolor: Ucolor addresses these gaps with multi-color-space attention, medium-transmission-guided decoding, and incorporation of underwater imaging domain knowledge.The method is presented as combining the complementary merits of physical model-based and deep learning approaches.

III. PROPOSED METHOD

Ucolor builds a unified encoder from RGB, HSV, and Lab representations, then uses attention and reverse medium transmission to guide enhancement toward degraded regions.

  • Multi-color space encoder: Ucolor forms HSV, RGB, and Lab encoder paths with serial residual-enhancement modules and multi-level downsampled features.Features from corresponding levels are densely connected and concatenated across the three paths.
  • Attention: Channel-attention modules adaptively spotlight the most representative and informative features from the concatenated multi-color-space representations.The channel-attention module derives channel descriptors using global average pooling.
  • Medium transmission guidance: The medium transmission guidance module forwards selected encoder features with same-sized reverse medium transmission maps to emphasize degraded regions.Reverse medium transmission is used as attention weighting, with more degraded pixels assigned larger weights.
  • Multi-color space encoder: The encoder combines color-space properties so color, hue, saturation, intensity, and luminance degradation components can be considered jointly.Multiple color spaces can represent color differences differently, supporting measurement of underwater color deviations.
  • Residual enhancement: Each residual-enhancement module contains two residual blocks with stacked convolutions, Leaky ReLU activations, and pixel-wise identity additions.The residual-enhancement design aims to preserve data fidelity and address gradient vanishing.

B. Residual-Enhancement Module

The residual-enhancement module preserves data fidelity and addresses gradient vanishing, while channel attention recalibrates multi-color-space features using global channel information.

  • B. Residual-Enhancement Module: The residual-enhancement module preserves data fidelity and addresses gradient vanishing.
  • B. Residual-Enhancement Module: Channel attention explicitly exploits interdependencies among channel features extracted from different color spaces.
  • B. Residual-Enhancement Module: Global average pooling converts input features F into a channel descriptor z representing the global distribution of channel-wise responses.
  • B. Residual-Enhancement Module: A self-gating mechanism produces per-channel modulation weights to capture channel-wise dependencies.
  • B. Residual-Enhancement Module: Fully connected layers use output-channel counts of N/r and N, with r set to 16 to reduce computational costs.
  • B. Residual-Enhancement Module: The modulation weights rescale input features F into U, while identical mapping preserves original features and helps avoid gradient vanishing.

D. Medium Transmission Guidance Module

The medium transmission guidance module uses reverse medium transmission as spatial attention, emphasizing regions with greater quality degradation while tolerating estimation errors.

  • D. Medium Transmission Guidance Module: The underwater imaging model represents degraded images using observed image I, clear image J, background light A, and medium transmission T.T denotes the percentage of scene radiance reaching the camera and indicates regional quality degradation.
  • D. Medium Transmission Guidance Module: The RMT map T serves as a feature selector that weights spatial positions of input features U to produce highlighted output features V.
  • D. Medium Transmission Guidance Module: RMT is obtained as 1-T, so pixels with higher quality degradation receive larger attention weights.
  • D. Medium Transmission Guidance Module: Because ground-truth transmission maps are unavailable in practice, the method estimates medium transmission using prior-based algorithms.
  • D. Medium Transmission Guidance Module: Identity-connected RMT weighting helps avoid gradient vanishing and tolerate errors from inaccurate transmission estimates.The authors also state that the data-driven framework tolerates inaccurate medium transmission maps.

E. Loss Function

Ucolor trains with a weighted combination of pixel-level ℓ2 loss and VGG-19-based perceptual loss to balance visual quality and quantitative scores.

  • Loss design: The final training loss combines ℓ2 loss and perceptual loss linearly.The ℓ2 term compares reconstructed and ground-truth images, while perceptual loss uses VGG-19 feature representations.
  • Loss weighting: The weighting parameter λ is empirically set to 0.01 to balance the scales of the two losses.
  • Perceptual loss: The perceptual loss measures feature distance between the reconstructed result and ground-truth image using a pretrained VGG-19 network.The loss is computed at the relu5_4 layer.

IV. EXPERIMENTS

The experiments evaluate Ucolor against representative underwater enhancement methods using reference-based, no-reference, perceptual, and color-dissimilarity measurements across diverse benchmarks.

  • Implementation and data: Ucolor is trained on 800 randomly selected UIEB image-reference pairs.UIEB contains 890 real underwater images with corresponding reference images selected by 50 volunteers from 12 enhanced results.
  • Benchmarks: Testing uses Test-R90, Test-S1000, Test-C60, SQUID, and Color-Check7 to cover real, synthetic, challenging, and color-correction evaluations.Test-R90 contains 90 UIEB real pairs, while Test-S1000 contains 1,000 synthetic pairs.
  • Compared methods: The study compares Ucolor with traditional, deep learning-based, and Unet-based baselines using released, reproduced, or author-provided implementations.UcycleGAN is unsupervised, Water-Net uses the same number of training samples, and UWCNN includes original and retrained models.
  • Color evaluation: Color-Check7 evaluates color restoration with CIEDE2000 by comparing 24 color patches against the ground-truth Macbeth Color Checker.

C. Visual Comparisons

Visual comparisons across synthetic, real, challenging, and Color Checker images show that Ucolor improves color, contrast, detail, and perceptual quality across varied underwater scenes.

  • Synthetic comparisons: On Test-S1000, Ucolor produces the result closest to ground truth and obtains the best PSNR/MSE scores.The RMT map highlights highly degraded regions, which Ucolor enhances through its guided response.
  • Real-image comparisons: On Test-R90, Ucolor removes greenish color deviation and improves contrast without obvious over-enhancement or over-saturation.Methods trained on the same data, including UWCNN retrain, Unet-U, and Unet-RMT, perform less well than Ucolor.
  • Challenging scenes: On challenging Test-C60 images, Ucolor recovers relatively realistic colors and enhances details while competing methods introduce artifacts or artificial colors.
  • SQUID comparisons: On SQUID, Ucolor dehazes difficult inputs, improves contrast, and avoids obvious artificial colors that reduce perceptual scores.Ancuti et al. achieves better contrast than other methods but produces color deviations.
  • Color correction: On Color-Check7, Ucolor achieves the best reported color-correction result, with a CIEDE2000 value of 8.21 for the Pentax W60 image.
  • Overall visual findings: Across the visual comparisons, Ucolor renders visually pleasing results and generalizes well to different underwater scenes.

D. Quantitative Comparisons

Ucolor outperforms competing methods across reference-based, perceptual, and color-restoration evaluations, with gains reported on synthetic and real benchmark data.

  • Reference-based evaluation: 20%/59% gains in PSNR/MSE on Test-S1000 and 4.1%/1.3% gains on Test-R90 are achieved by Ucolor over the second-best performer.Ucolor outperforms all competing methods on both datasets.
  • Perceptual evaluation: Test-C60 and SQUID are challenging enough that most enhancement methods fail in perceptual quality, with some scoring below the inputs.The user study reports average perceptual scores for the enhanced results.
  • Color-restoration evaluation: Ucolor achieves the lowest color dissimilarity for the evaluated cameras and the best average CIEDE2000 score across seven cameras.Some competing methods perform worse than the original input and can disrupt inherent colors.

E. Ablation Study

The ablation studies show that Ucolor’s full combination of multi-color space encoding, medium transmission guidance, channel attention, and perceptual loss yields the strongest enhancement behavior.

  • The full model achieves the best quantitative performance across two testing datasets, supporting the combined effectiveness of MCSE, MTGM, and CAM.
  • Accurate reverse medium transmission maps improve weighting of degraded regions and enhance visual quality, especially in high-scattering regions.The RMT-based Ucolor also outperforms variants using RDCP, RUDCP, or no medium transmission guidance.
  • Removing HSV or Lab encoder paths decreases performance, whereas replacing them with additional RGB paths provides comparable performance to removing both paths.The results favor purposefully designed multi-color space embedding over simply adding parameters or repeating the same color space.
  • Removing channel attention produces an under-saturated result because the module integrates and highlights representative features from multiple color spaces.
  • Adding perceptual loss to ℓ1 loss improves final visual quality, although ℓ1-only training slightly improves PSNR and MSE on Test-S1000.The full loss combines ℓ1 and perceptual losses; perceptual-loss-only training was not evaluated because it was considered unsuitable.

F. Failure Case

Ucolor may fail on underwater images with limited lighting, although it avoids the color casts introduced by Water-Net in the reported failure case.

  • Limited lighting prevents Ucolor and other state-of-the-art methods from producing compelling enhancement results.The authors suggest that few such images in training datasets may limit supervised networks’ ability to handle this setting.
  • The ablation figures compare reverse transmission maps, color-space encoder paths, channel attention, and perceptual loss through their effects on enhancement quality.
  • Ucolor does not introduce color casts in the limited-lighting failure case, unlike Water-Net.

V. CONCLUSION

Ucolor learns diverse color-space representations, highlights discriminative features with channel attention, and uses reverse medium transmission as attention weights to enhance underwater images.

  • Ucolor combines diverse color-space feature learning with channel attention and reverse medium transmission guidance.The conclusion reports superiority across diverse benchmarks and effectiveness of multi-color space embedding.
Loading 2104.13015v1…