PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Underwater Image Enhancement

Saeed Anwar, Chongyi Li, Fatih Porikli

arXiv:1807.03528v1cs.CV

TL;DR

Underwater absorption and scattering degrade visibility, contrast, and color, while existing methods face modeling, applicability, and labeled-data challenges. UWCNN uses synthetic underwater images to train an end-to-end CNN that reconstructs clear latent images. The authors report state-of-the-art performance and generalization across synthetic and real-world underwater scenes.

  • Problem

    Underwater absorption and scattering impair visibility, contrast, and color, while existing approaches face modeling, applicability, and labeled-data limitations.

    Source: §I · pp. 1–2

  • Method

    UWCNN is an end-to-end CNN that learns to reconstruct clear latent images, using synthetic degraded images and joint MSE and SSIM losses.

    Source: §I · p. 3

  • Results

    The method is reported to achieve state-of-the-art performance and generalize across synthetic and real-world images with varying color and visibility characteristics.

    Source: §I · p. 3

  • Takeaways & Limitations

    The authors conclude that the method is suitable for practical applications across varying underwater scenes.

    Source: §I · p. 3

Abstract

from arXiv · show

In an underwater scene, wavelength-dependent light absorption and scattering degrade the visibility of images, causing low contrast and distorted color casts. To address this problem, we propose a convolutional neural network based image enhancement model, i.e., UWCNN, which is trained efficiently using a synthetic underwater image database. Unlike the existing works that require the parameters of underwater imaging model estimation or impose inflexible frameworks applicable only for specific scenes, our model directly reconstructs the clear latent underwater image by leveraging on an automatic end-to-end and data-driven training mechanism. Compliant with underwater imaging models and optical properties of underwater scenes, we first synthesize ten different marine image databases. Then, we separately train multiple UWCNN models for each underwater image formation type. Experimental results on real-world and synthetic underwater images demonstrate that the presented method generalizes well on different underwater scenes and outperforms the existing methods both qualitatively and quantitatively. Besides, we conduct an ablation study to demonstrate the effect of each component in our network.

I. INTRODUCTION

Underwater degradation impairs image visibility, contrast, and color, while existing approaches face uneven enhancement, model mismatch, hardware costs, and scarce labeled training data. The paper proposes a synthetic-data-driven CNN approach and reports strong performance across synthetic and real-world scenes.

I. INTRODUCTION

  • Existing methods can produce uneven contrast or artifacts, rely on ill-suited outdoor haze models, or require costly sensors and multiple images; deep enhancement is also constrained by scarce labeled data.These shortcomings motivate solutions that improve visibility, contrast, and color for underwater scene understanding and marine applications.Source: §I · pp. 1–2
  • The paper proposes a robust, data-driven solution based on a new underwater image synthesis method and a deep convolutional neural network.The method is presented as flexible across varying water types for marine imaging applications.Source: §I · p. 2
  • UWCNN reconstructs clear latent images while preserving structure and texture by jointly optimizing MSE and SSIM losses.Rather than directly minimizing a mapping-function objective, the network learns the difference between degraded and clean images.Source: §I · p. 3
  • The proposed synthesis method simulates diverse degraded underwater images for data augmentation and can represent different underwater types and degradation levels.The authors describe it as the first underwater synthesis method capable of simulating different types and degradation levels.Source: §I · p. 3
  • The method reportedly achieves state-of-the-art performance and generalizes across synthetic and real-world images with varying color and visibility characteristics.The authors state that this makes it suitable for practical applications.Source: §I · p. 3

II. BACKGROUND

The background introduces underwater image formation and explains how wavelength-dependent attenuation affects the light captured by a camera. It identifies the clear latent scene radiance as the recovery target.

II. BACKGROUND

  • The underwater image-formation discussion frames recovery as estimating clear latent scene radiance from the captured underwater image.The passage identifies the background light and the medium energy ratio as parts of the formation description.Source: A. Underwater Image Formation · p. 3

A. Underwater Image Formation

  • The medium energy ratio represents the fraction of scene radiance reaching the camera and varies with light wavelength and scene-point distance.The attenuation coefficient is wavelength-dependent; the text notes that red light attenuates and is absorbed faster, contributing to bluish tones.Source: A. Underwater Image Formation · p. 3

B. Formulation as an Image Restoration Task

The paper frames underwater enhancement as image restoration, directly reconstructing the latent image rather than estimating the conventional formation-model parameters. It uses a MAP formulation with a regularization prior to guide reconstruction.

B. Formulation as an Image Restoration Task

  • UWCNN directly estimates latent image I without calculating background light B or medium energy ratio T, unlike conventional restoration methods.Conventional methods estimate B and T, then invert the underwater formation model; estimating I, B, and T from one image is underdetermined.Source: B. Formulation as an Image Restoration… · p. 3
  • The reconstruction is formulated as MAP estimation of latent image I given underwater image U, with p(U|I) as likelihood and p(I) as prior.With a uniform observation distribution, the posterior maximization can be expressed as minimizing negative log likelihood.Source: B. Formulation as an Image Restoration… · p. 3
  • A quadratic data-fidelity term keeps the estimated latent image consistent with the degraded observation, while regularization prior Ψ(I) guides the output.The trade-off parameter α balances the prior and data-fidelity terms.Source: B. Formulation as an Image Restoration… · p. 3
  • An explicit regularization prior need not be exactly parameterized and can allow alternative priors to be used jointly or in similar inverse problems.The passage presents these as benefits of formulating enhancement as restoration with an explicit prior.Source: B. Formulation as an Image Restoration… · pp. 3–4

C. Why CNN?

The paper motivates CNNs as a learned regularizer for underwater image restoration. Their hierarchical filtering and local receptive fields provide a compact way to model distortions and guide enhancement.

C. Why CNN?

  • A cascade of CNN-based hierarchical nonlinear filters uses progressively larger local receptive fields to regularize image restoration.The residual bottleneck has fewer parameters than input pixels, while local patch filters guide enhancement according to blur and color.Source: C. Why CNN? · p. 4
  • CNNs model distortions between latent and original underwater images and facilitate discriminative prior learning.Source: C. Why CNN? · p. 4
  • The method learns external priors from underwater images and ground truth, complementing methods such as ODM that rely on internal image statistics and features.The authors expect combining their approach with ODM to improve performance.Source: C. Why CNN? · p. 4
  • CNN inference can be made efficient by exploiting parallel processing platforms.Source: C. Why CNN? · p. 4

III. OUR PROPOSED MODEL

UWCNN is a fully convolutional underwater-enhancement model, with a post-processing stage intended to further improve its output. Its architecture combines residual prediction, dense feature concatenation, and boundary-focused design choices.

III. OUR PROPOSED MODEL

  • The proposed model also includes a post-processing stage intended to further improve the enhanced result.Source: §III · p. 4

A. Network Architecture

  • UWCNN learns a residual Δ(U, θ) and adds it to input U to form the latent-image estimate, a strategy used to avoid vanishing or exploding gradients during training.Source: A. Network Architecture · p. 4
  • Convolutional layers use 3 × 3 × 3 filters for the first layer and 3 × 3 × 16 filters subsequently, while ReLU adds nonlinearity and Concat stacks convolutional layers.Source: A. Network Architecture · p. 4
  • Each block stacks convolutional outputs and reuses the input image, reducing the need for a very deep network.The authors distinguish this dense concatenation from DenseNet's connections among convolutional layers within a block.Source: A. Network Architecture · p. 5
  • The network has three blocks of three convolutional layers plus a final convolutional layer, for a total depth of ten layers.It uses no fully connected layers or batch normalization; the authors describe it as computationally inexpensive and practical for training and prediction.Source: A. Network Architecture · p. 5
  • No pooling and zero-padding before each convolution produce same-sized output with boundaries described as almost artifact-free.Source: A. Network Architecture · p. 5

B. Network Loss

UWCNN trains its residual mapping with pixelwise MSE and an SSIM loss. The combined objective is optimized with ADAM using fixed learning-rate settings.

B. Network Loss

  • Pixelwise ℓ2 loss (MSE) is used because the authors report that it preserves edge sharpness and detail, while blurring edges produces large errors.The residual is added to the underwater input before the estimated image is compared with ground truth.Source: B. Network Loss · p. 5
  • The objective combines MSE and SSIM losses, with SSIM included to impose structure and texture similarity on the latent image.SSIM is computed on gray images using 13 × 13 patches, with constants c1 = 0.02 and c2 = 0.03.Source: B. Network Loss · p. 5
  • ADAM updates UWCNN weights with learning rate 0.0002, β1 = 0.9, and β2 = 0.999, held fixed during training.Source: B. Network Loss · p. 5

C. Post-processing

UWCNN+ applies HSI saturation and intensity normalization to address the lower dynamic range associated with the indoor-image training pairs. Qualitative results report improved contrast and brightness, with more details revealed.

C. Post-processing

  • The training pairs use indoor latent images and their synthesized underwater counterparts, a limitation associated with lower dynamic range in enhanced images.The paper notes that vivid colors and higher contrast are expected in practice.Source: C. Post-processing · p. 5
  • UWCNN+ transforms the image to HSI and normalizes saturation and intensity ranges to [0, 1] before converting the modified result back to RGB.The normalization uses a histogram-frequency threshold of 0.2% to ignore pixels below that frequency when selecting the extrema.Source: C. Post-processing · pp. 5–6
  • UWCNN+ further improves contrast and brightness over UWCNN, unveiling more details in real-world underwater images.The reported comparison also says the methods remove greenish distortion while preserving a natural look.Source: C. Post-processing · p. 6, Figure 4 · p. 6

IV. EXPERIMENTAL EVALUATIONS

The evaluation compares UWCNN with recent enhancement methods on synthetic and real-world images, while the paper presents a physical-model synthesis procedure for training data across ten underwater types.

IV. EXPERIMENTAL EVALUATIONS

  • The evaluation compares UWCNN with UDCP, RED, ODM, and UIBLA on synthetic and real-world underwater images, and includes an ablation study.For real-world images, the authors apply all ten learned models and show the visually more appealing results; synthetic results are presented without post-processing.Source: §IV · p. 6
  • The authors present a physical-model-based synthesis method that simulates diverse water types and degradation levels, addressing a training-data need for underwater image enhancement.They describe it as, to their knowledge, the first such method.Source: §IV · p. 6

A. Underwater Image Synthesis

  • Ten datasets are synthesized from 1,449 NYU-v2 images, using the first 1,000 for training and the remaining 449 for validation.Each clean image generates five underwater images, yielding 5,000 training and 2,495 validation samples; images are resized to 310×230, and ten UWCNN models are trained.Source: A. Underwater Image Synthesis · pp. 6–7
  • The synthesized examples span ten underwater image types, but Types I, IA, and IB are similar in physical appearance, so eight models are displayed for synthetic-image results.Figure 5 shows the ten types using a sample image and its depth map.Source: A. Underwater Image Synthesis · p. 7, Figure 5 · p. 7

B. Network Implementation and Training

The section reports UWCNN’s synthetic-image benchmark and qualitative results, then evaluates real-world images and describes its training setup.

B. Network Implementation and Training

  • Training used synthetic 310×230 images generated from NYU-v2, with ADAM, batch size 16, and took about three hours over 20 epochs.The reported learning rate was 0.0002; β1 was 0.9 and β2 was 0.999.Source: B. Network Implementation and Training · p. 7

C. Synthetic Underwater Images

  • UWCNN’s synthetic-image results restore texture and vibrant, genuine colors, with visual quality resembling the ground-truth labels.The comparison covers clear and turbid image types; RED leaves haze artifacts and color deviations on turbid types.Source: C. Synthetic Underwater Images · p. 7, Figure 6 · p. 8

D. Real-world Underwater Images

C. Synthetic Underwater Images

D. Real-world Underwater Images

  • UWCNN+ received the highest average subjective rankings from 20 image-processing and computer-vision experts, who rated perceived visual quality on a 1-to-5 scale for enhanced real-world underwater images.Source: D. Real-world Underwater Images · p. 10

E. Ablation Study

The ablation study examines residual learning, dense concatenation, and SSIM loss, finding gains from the components alongside a metric trade-off for SSIM loss.

E. Ablation Study

  • The ablation compares UWCNN with versions without residual learning, dense concatenation, or SSIM loss.Source: E. Ablation Study · pp. 10–11
  • Residual learning could boost performance compared with conventional learning, and dense concatenation could improve performance compared with UWCNN-woDC.Table IV reports results for the Type-1 and Type-III test sets.Source: E. Ablation Study · p. 11, Table IV · p. 11
  • SSIM loss improves structure and texture similarity, but UWCNN-woSSIM scores better on MSE and PSNR; the authors report a smoother background with SSIM loss.The quantitative ablation uses only Type-1 and Type-III synthetic test sets.Source: E. Ablation Study · p. 11, Figure 11 · p. 11

V. CONCLUSION

UWCNN is a compact CNN for underwater image enhancement, and the ablation results support its included components. The authors identify hazy-image enhancement and joint depth estimation as future extensions.

V. CONCLUSION

  • UWCNN uses ten convolutional layers and 16 feature maps per layer, with fast and efficient training and testing on GPU platforms.Source: §V · p. 11
  • The ablation study reports quantitative and qualitative performance gains from residual learning, dense concatenation, and SSIM loss.Source: §V · p. 11
  • The authors plan to extend the model to hazy-image enhancement and jointly produce image depth from a single hazy image.Source: §V · p. 11

Related papers

Loading 1807.03528v1…