PaperlayerView exact revision on arXiv

Source-linked AI summary

An Underwater Image Enhancement Benchmark Dataset and Beyond

Chongyi Li, Chunle Guo, Wenqi Ren, Runmin Cong, Junhui Hou, Sam Kwong, Dacheng Tao

arXiv:1901.05495v2cs.CV

TL;DR

Underwater enhancement research lacks comprehensive evidence from large-scale real-world images and corresponding references. The paper constructs UIEB, evaluates existing methods, and trains Water-Net on the benchmark; the results show that no method consistently wins across full- and no-reference metrics, while reference generation has important limitations.

  • Problem

    Existing algorithms are mainly evaluated on synthetic datasets or a few real-world images, leaving their performance on in-the-wild images unclear.

  • Method

    The paper constructs UIEB, conducts qualitative and quantitative evaluations, and trains the CNN baseline Water-Net on the benchmark.

  • Results

    No evaluated method always wins across full- and no-reference metrics on the large-scale real-world UIEB dataset.

  • Takeaways & Limitations

    UIEB enables comprehensive study of existing enhancement methods and training of CNN-based underwater image enhancement models.

  • Takeaways & Limitations

    Reference generation is limited by inaccurate underwater imaging models and volunteers who may overlook backscatter in far ranges.

Abstract

from arXiv · show

Underwater image enhancement has been attracting much attention due to its significance in marine engineering and aquatic robotics. Numerous underwater image enhancement algorithms have been proposed in the last few years. However, these algorithms are mainly evaluated using either synthetic datasets or few selected real-world images. It is thus unclear how these algorithms would perform on images acquired in the wild and how we could gauge the progress in the field. To bridge this gap, we present the first comprehensive perceptual study and analysis of underwater image enhancement using large-scale real-world images. In this paper, we construct an Underwater Image Enhancement Benchmark (UIEB) including 950 real-world underwater images, 890 of which have the corresponding reference images. We treat the rest 60 underwater images which cannot obtain satisfactory reference images as challenging data. Using this dataset, we conduct a comprehensive study of the state-of-the-art underwater image enhancement algorithms qualitatively and quantitatively. In addition, we propose an underwater image enhancement network (called Water-Net) trained on this benchmark as a baseline, which indicates the generalization of the proposed UIEB for training Convolutional Neural Networks (CNNs). The benchmark evaluations and the proposed Water-Net demonstrate the performance and limitations of state-of-the-art algorithms, which shed light on future research in underwater image enhancement. The dataset and code are available at https://li-chongyi.github.io/proj_benchmark.html.

I. INTRODUCTION

Underwater image enhancement is difficult because wavelength-dependent absorption, scattering, and marine snow degrade visibility, contrast, and color. The paper addresses limited real-world evaluation and training data by introducing UIEB, evaluating existing methods, and training Water-Net.

  • Motivation: Wavelength-dependent absorption and scattering, together with marine snow, reduce visibility, decrease contrast, and introduce color casts in underwater images.These degradations limit applications including marine biology and archaeology.
  • Research Gap: The field lacks a publicly available large-scale real-world dataset with reference images, limiting comprehensive evaluation and effective training of deep learning methods.Simultaneously photographing real underwater scenes and corresponding ground truth is practically difficult.
  • Dataset Contribution: UIEB contains 950 real underwater images, including reference images for 890, enabling evaluation and supervised enhancement-model training.The references were produced through laborious, time-consuming, and designed pairwise comparisons.
  • Evaluation: The paper evaluates state-of-the-art single-image enhancement algorithms qualitatively and quantitatively to analyze their strengths and limitations.The analysis is intended to provide insights and suggest research directions.
  • Learning Baseline: Water-Net is a CNN trained on UIEB to demonstrate the benchmark’s generalization for supervised underwater image enhancement.The model is presented as a baseline for deep learning-based enhancement.
  • Prior Methods: Underwater enhancement methods span supplementary-information-based, non-physical model-based, physical model-based, and data-driven approaches.Single-image enhancement is particularly suited to challenging dynamic scenes compared with methods requiring supplementary information.

B. Underwater Image Quality Evaluation

Underwater enhancement methods are evaluated with full-reference metrics when ground-truth images exist, while such reference data are often limited to simple or simulated settings.

  • Full-reference Metrics: Full-reference metrics such as MSE, PSNR, and SSIM evaluate enhanced underwater images against ground-truth images.These metrics are commonly applied to color checker images or color patches captured in simulated or real underwater environments.

C. Underwater Image Datasets

Existing underwater datasets generally serve recognition or restoration needs but lack the large-scale, diverse, reference-paired real-world data required for enhancement research. UIEB is constructed to address this gap through broad image collection and reference generation.

  • Existing Datasets: Existing real-world underwater datasets include Fish4Knowledge, SUN, MARIS, Sea-thru, and Haze-line, but their content is usually monotonous.The paper identifies the lack of a large-scale real-world enhancement dataset with reference images as the main community issue.
  • Collection Objectives: UIEB collection targets diverse scenes, varied degradation characteristics, broad image content, a large image count, and corresponding high-quality references.These objectives support fair image-quality evaluation and end-to-end learning.
  • Data Collection: About 950 candidate images remained after refinement from sources including Google, YouTube, related papers, and self-captured videos.The dataset statistics cover image resolutions and scene or main-object categories.
  • Dataset Diversity: UIEB spans a large range of image resolutions and diverse categories such as coral, turtles, and sharks.Examples include fringing reefs, barrier reefs, and marine life.

B. Reference Image Generation

UIEB reference images were generated by comparing outputs from 12 enhancement methods and selecting the preferred result through volunteer pairwise judgments. The benchmark retains 890 reference-image pairs and 60 challenging images without satisfactory references.

  • Candidate generation: 12 enhancement methods generated candidate results for each of 950 underwater images, covering enhancement, dehazing, and a commercial application.The candidates included nine underwater enhancement methods, two image dehazing methods, and dive+8.
  • Pairwise comparison: 50 volunteers compared enhanced-result pairs against each raw underwater image, repeatedly advancing the preferred result until selecting the best one.The volunteer pool included 25 people with image-processing experience and 25 without related experience.
  • Reference selection: Reference images were selected by majority voting after pairwise comparisons, with volunteers also labeling the selected best result satisfactory or dissatisfactory.The raw underwater image served as the comparison reference during each pairwise decision.
  • Dataset subsets: 890 images received available reference images, while 60 images became challenging data because their selected references were judged dissatisfactory by more than half the votes.The reference image was discarded for those challenging images.
  • Perceptual preference: Observers most favored results with improved contrast and genuine color, while over-/under-enhancement, artifacts, and color casts produced visually unpleasing results.These judgments define the perceptual preference underlying the constructed reference subset.

IV. EVALUATION AND DISCUSSION

The evaluation compares state-of-the-art underwater enhancement methods qualitatively and examines their behavior across color casts, haze, and backscatter in diverse real-world scenes. Results show method-specific strengths, while no single correction algorithm or prior performs effectively across all image types.

  • Evaluation setup: The study evaluates nine state-of-the-art underwater image enhancement methods using both qualitative and quantitative assessments.The evaluated methods include fusion-based, two-step-based, retinex-based, UDCP, regression-based, GDCP, Red Channel, histogram-prior, and blurriness-based methods.
  • Scene categories: The qualitative examples cover greenish and bluish images, downward-looking images, forward-looking images, and low- and high-backscatter scenes.The displayed examples do not cover the entire UIEB.
  • Color and contrast: Retinex-based enhancement removes color deviation well, whereas UDCP and GDCP aggravate color casts; two-step-based enhancement effectively increases contrast.Fusion-based, histogram-prior, and regression-based methods introduce reddish color deviation in the shown examples.
  • Haze removal: Fusion-based, retinex-based, and histogram-prior methods significantly reduce haze in downward- and forward-looking images, while several competing methods retain haze or introduce color deviation.The methods retaining some haze include two-step-based, Red Channel, and blurriness-based approaches.
  • Backscatter: All evaluated physical model-based methods fail to remove high backscatter in the shown scenes because of inaccurate physical models and assumptions.High backscatter is associated with longer camera-to-scene distance and significantly degrades visual quality.
  • Overall discussion: Fusion-based enhancement performs relatively decently across varied underwater images, but UDCP can produce artifacts and high backscatter remains challenging.The authors conclude that no single color-correction algorithm or prior is effective for all underwater image types.

B. Quantitative Evaluation

The study evaluates enhancement methods with full-reference and non-reference metrics, then compares metric scores against subjective visual quality. The results show that non-reference scores can diverge from human judgments and overlook color shifts and artifacts.

  • 1) Full-reference Evaluation:: MSE, PSNR, and SSIM provide full-reference evaluation on 890 UIEB images with reference images.Higher PSNR and SSIM, and lower MSE, indicate greater similarity to the reference image.
  • 1) Full-reference Evaluation:: dive+ ranks best across all full-reference metrics, while fusion-based ranks second.The ranking is partly expected because most reference images were selected from dive+ and fusion-based results.
  • 2) Non-reference Evaluation:: UCIQE and UIQM rank histogram prior and UDCP highest, respectively, unlike the subjective pairwise comparisons.This inconsistency indicates that current underwater image quality metrics do not always match human visual perception.
  • 2) Non-reference Evaluation:: UCIQE and UIQM may favor selected characteristics such as high contrast without accounting for color shifts and artifacts.The paper notes that human attention to central or colorful objects may also make human visual perception unsuitable for learning attenuation and backscatter.
  • 2) Non-reference Evaluation:: Histogram prior produces reddish color shifts and artifacts yet scores higher than dive+ on both UCIQE and UIQM.The comparison exposes a gap between non-reference metric scores and subjective visual quality.

3) Runtime Evaluation:

Runtime evaluation compares methods across image sizes and finds substantial differences in computational speed. These differences, alongside method-specific artifacts and the absence of a universal winner, constrain practical deployment.

  • 3) Runtime Evaluation:: The two-step-based method is fastest across image sizes, followed by the retinex-based method.Average runtimes were measured in MATLAB R2014b on an Intel i7-6700 PC with 32GB RAM.
  • 3) Runtime Evaluation:: The regression-based method is slowest because random-forest transmission prediction is time-consuming, especially for large images.Runtime therefore varies with both method and image size.
  • 3) Runtime Evaluation:: Fusion-based enhancement performs relatively best in most cases, but no method consistently wins across UIEB.The paper attributes differing weaknesses to model assumptions, color casts, haze, noise, artifacts, and over- or under-enhancement.
  • 3) Runtime Evaluation:: Some methods’ runtimes seriously limit their practical applications.The paper calls for methods that robustly handle varied underwater degradation and for non-reference metrics consistent with human perception.

V. PROPOSED MODEL

The paper proposes Water-Net as a baseline CNN for underwater image enhancement, motivated by the limited generalization of deep learning models. Its architecture is presented with training and experiments to assess its advantages.

  • V. PROPOSED MODEL: Water-Net is proposed as a baseline CNN for underwater image enhancement using UIEB.The model addresses the reported gap between deep learning generalization and conventional state-of-the-art methods.
  • V. PROPOSED MODEL: The section introduces Water-Net’s input generation, architecture, training and implementation details, and experiments.These components are organized to demonstrate the baseline model’s advantages.

A. Input Generation

Water-Net generates multiple transformed inputs and uses a gated fusion network to combine refined versions according to learned confidence maps. Feature Transformation Units reduce artifacts before fusion.

  • A. Input Generation: Three inputs are generated with White Balance, Histogram Equalization, and Gamma Correction to address different degradation characteristics.White Balance targets color casts, while Histogram Equalization and Gamma Correction provide complementary transformed inputs.
  • A. Input Generation: Water-Net predicts confidence maps that determine how the transformed inputs contribute to the enhanced result.The architecture follows a gated fusion strategy inspired by the performance of fusion-based enhancement.
  • A. Input Generation: The network feeds the original and three derived inputs into a fully convolutional model and refines the derived inputs with three FTUs.The FTUs are intended to reduce color casts and artifacts introduced by the preprocessing operations.
  • A. Input Generation: The enhanced image is formed by elementwise multiplying each refined input by its learned confidence map and summing the three products.The equation defines I_en as the fusion of refined WB, HE, and GC outputs weighted by C_WB, C_HE, and C_GC.

C. Implementations

Water-Net is trained with perceptual loss on UIEB image pairs, using data augmentation and a staged learning-rate schedule; it processes 640 × 480 images at 8 FPS.

  • Data and training: 800 UIEB image pairs form the training set, resized to 112×112 and augmented with flipping and rotation to produce seven versions.The training data are resized because of limited memory.
  • Loss function: Perceptual loss minimizes feature-space distance between enhanced and reference images to reduce artifacts associated with pixel-wise ℓ1 and ℓ2 losses.The loss uses activated convolutional features from a pretrained VGG19 network.
  • Loss function: VGG19 feature-map dimensions are normalized using C_jH_jW_j, where C_j, H_j, and W_j denote feature-map number, height, and width.The normalization is defined for the jth convolutional layer within VGG19.
  • Efficiency: Water-Net processes a 640 × 480 image in 0.128s, corresponding to 8 FPS.This throughput is reported after convergence of the training procedure.

D. Experiments

Experiments compare Water-Net with state-of-the-art methods on reference-based testing images and reference-free challenging images. Water-Net removes haze and color casts, achieves the best full-reference results, and leads subjective scores on the challenging set.

  • Experimental setup: Experiments evaluate methods on 90 testing images and 60 challenging images, with example comparisons shown in Figs. 13 and 14.The testing set has reference images, whereas the challenging set is evaluated subjectively.
  • Qualitative evaluation: Water-Net effectively removes haze and color casts, while competitors produce unexpected colors, artifacts, or little visible improvement.The cited examples include fusion-based, GDCP, histogram-prior, Water CycleGAN, Dense GAN, retinex-based, and blurriness-based methods.
  • Quantitative evaluation: Water-Net achieves the best full-reference image-quality performance on the testing set in MSE, PSNR, and SSIM.Results are computed by comparing each enhanced image with its corresponding reference image.
  • Subjective evaluation: Water-Net receives the highest average score and lowest standard deviation on the challenging set, indicating better subjective results and more robust performance.Water CycleGAN and Dense GAN are excluded because of their obviously unpleasing results in Fig. 14.
  • Overall findings: The experiments support Water-Net’s effectiveness and show that UIEB can be used to train CNNs, while challenging-set images remain difficult to enhance well.The results therefore establish both a demonstrated capability and a remaining weakness.

VI. CONCLUSION, LIMITATIONS, AND FUTURE WORK

The paper contributes UIEB and Water-Net for evaluating and training underwater enhancement systems, but reference construction is limited by persistent far-distance backscatter and imperfect physical understanding.

  • Conclusion: UIEB provides large-scale real underwater images with corresponding references, enabling comprehensive evaluation and CNN training.The benchmark supports study of existing enhancement methods and training of underwater enhancement networks.
  • Conclusion: No method consistently wins across full-reference and no-reference metrics, motivating effective non-reference underwater image-quality metrics.The paper presents this as an evaluation-level conclusion rather than a universal ranking.
  • Conclusion: Water-Net performs favorably against state-of-the-art methods, and experiments verify UIEB’s generalization for CNN training.This conclusion is based on the paper’s qualitative and quantitative evaluations.
  • Limitations: Far-distance backscatter cannot always be removed by existing enhancement algorithms, limiting reference-image selection and subsequently the network’s performance.The reference-generation strategy can select visually pleasing results, but residual backscatter remains problematic in some cases.
  • Limitations: Reference-generation shortcomings arise from inaccurate image-formation assumptions and volunteers’ difficulty recognizing backscatter’s increasing effect over large ranges.The paper identifies inaccurate imaging models as a major problem for underwater computer vision.
  • Future work: Future work targets more challenging images and videos, range-map estimation, and use of range information to improve enhancement.The paper proposes training a range-map estimation network and exploiting estimated range maps as prior information.
Loading 1901.05495v2…