PaperlayerView exact revision on arXiv

Source-linked AI summary

An Experimental-based Review of Image Enhancement and Image Restoration Methods for Underwater Imaging

Yan Wang, Wei Song, Giancarlo Fortino, Lizhe Qi, Wenqiang Zhang, Antonio Liotta

arXiv:1907.03246v1eess.IVcs.CVcs.MM

TL;DR

Underwater images are severely degraded by water-related absorption, scattering, turbidity, and uneven illumination, while prior reviews did not fully classify recent methods or clarify their quality improvements. This paper reviews and categorizes single-image enhancement and restoration methods, experimentally compares representative approaches and optical-parameter estimation methods, and identifies challenges. Its synthesis covers IFM-free and IFM-based approaches, while noting that traditional methods’ complexity limits scaling and direct use for underwater video.

  • Problem

    Prior reviews incompletely classified recent underwater image-improvement methods and left unclear how specific methods improve image quality.

  • Method

    The paper reviews single-underwater-image methods, categorizes them as IFM-free or IFM-based, and experimentally compares representative methods and IFM optical-parameter estimation.

  • Results

    The review provides an experimental-based comparison of state-of-the-art quality-improvement methods using multiple quality-assessment metrics and discusses their challenges.

  • Takeaways & Limitations

    The review supplies background for understanding challenges and opportunities in underwater image enhancement and restoration.

  • Takeaways & Limitations

    Traditional enhancement and restoration algorithms have relatively high complexity, limiting scalable practical studies and applications and direct underwater-video use.

Abstract

from arXiv · show

Underwater images play a key role in ocean exploration, but often suffer from severe quality degradation due to light absorption and scattering in water medium. Although major breakthroughs have been made recently in the general area of image enhancement and restoration, the applicability of new methods for improving the quality of underwater images has not specifically been captured. In this paper, we review the image enhancement and restoration methods that tackle typical underwater image impairments, including some extreme degradations and distortions. Firstly, we introduce the key causes of quality reduction in underwater images, in terms of the underwater image formation model (IFM). Then, we review underwater restoration methods, considering both the IFM-free and the IFM-based approaches. Next, we present an experimental-based comparative evaluation of state-of-the-art IFM-free and IFM-based methods, considering also the prior-based parameter estimation algorithms of the IFM-based methods, using both subjective and objective analysis (the used code is freely available at https://github.com/wangyanckxx/Single-Underwater-Image-Enhancement-and-Color-Restoration). Starting from this study, we pinpoint the key shortcomings of existing methods, drawing recommendations for future research in this area. Our review of underwater image enhancement and restoration provides researchers with the necessary background to appreciate challenges and opportunities in this important field.

I. INTRODUCTION

Underwater imaging is degraded by absorption, scattering, turbidity, color-selective attenuation, and uneven illumination, motivating algorithmic enhancement and restoration. The paper addresses incomplete prior reviews by organizing methods and experimentally comparing representative approaches.

  • Underwater images support robotics, rescue, inspection, and ecological monitoring, making high-quality visual data important for marine applications.
  • Light absorption and scattering produce color casts, low contrast, blur, haze, and background-scattering artifacts in underwater images.Red light attenuates faster than green and blue light, while suspended particles absorb and redirect reflected light.
  • Artificial lighting extends imaging range but can introduce non-uniform illumination, bright central spots, insufficient edge illumination, and additional noise.
  • The underwater image formation model represents captured imagery through direct transmission and background scattering after forward scattering is neglected.The model separates total signal energy into direct transmission, forward scattering, and background scattering components before simplifying the imaging process.
  • Hardware solutions can be expensive, power-consuming, and poorly adaptive, encouraging software-based enhancement and restoration algorithms.
  • Earlier reviews incompletely classified newer developments and did not establish how specific methods improve underwater image quality.This paper responds with a broader review, experimental comparison, and updated discussion of challenges and future directions for single-image quality improvement.

II. IFM-FREE IMAGE ENHANCEMENT

IFM-free enhancement methods improve underwater images without explicitly using the underwater optical imaging model. The review groups them into spatial-domain, transform-domain, and CNN-based subclasses.

  • IFM-free methods improve contrast and color mainly through pixel-intensity redistribution rather than modeling underwater imaging principles.
  • The review divides IFM-free enhancement into spatial-domain, transform-domain, and CNN-based image enhancement subclasses.

A. SPATIAL-DOMAIN IMAGE ENHANCEMENT

Spatial-domain methods redistribute intensities or combine corrected and contrast-enhanced representations to address underwater color cast, visibility, illumination, and contrast. Representative approaches include histogram, color-model, Retinex, and fusion strategies.

  • Spatial-domain enhancement expands concentrated underwater-image histograms to redistribute gray levels and improve visibility.
  • SCM-based image enhancement: SCM-based methods use histogram equalization, adaptive equalization, gamma correction, sharpening, and color assumptions to improve visibility and correct color.
  • Fusion methods combine white-balanced and contrast-enhanced images using contrast, saliency, and exposure weights to improve global contrast and detail.Later underwater-specific fusion applies gamma correction and sharpening before multiscale merging.
  • Fusion-based results are characterized by better dark-region exposure, improved global contrast, and sharper edges.
  • Other spatial-domain approaches include matrix-factorization illumination estimation, Markov Random Fields, integrated color models, and selective histogram stretching.
  • Retinex-based enhancement: Retinex-based methods separate illumination-related components and enhance brightness and color components to address underwater color cast and contrast.
  • Mix-CLAHE combines RGB- and HSV-based contrast enhancement and is reported to improve visibility while reducing noise and artifacts.
  • RGHS uses Gray-World preprocessing and adaptive stretching in RGB and CIE-Lab spaces according to channel distributions and selective light attenuation.

B. TRANSFORM-DOMAIN IMAGE ENHANCEMNT

Transform-domain methods manipulate frequency or wavelet representations to enhance visibility and contrast, while CNN- and GAN-based methods learn mappings or synthesize training data. The section also identifies noise, color distortion, and data-generation challenges.

  • Transform-domain methods enhance underwater images by amplifying high-frequency components associated with edges and suppressing low-frequency background components.
  • Homomorphic, anisotropic, and wavelet-based methods target non-uniform illumination, smoothing, denoising, haze, low contrast, and color alteration.
  • Transform-domain methods can improve hazy-image visibility and contrast but tend to over-amplify noise and cause color distortion.
  • CNN-based image enhancement: CNN-based methods learn end-to-end transformations from degraded to clear underwater images for tasks including color correction, haze removal, and latent-image reconstruction.
  • GAN-based pipelines generate realistic underwater training images or reconstruct distorted images to reduce reliance on paired deep-sea data.
  • UIEBD was constructed as a large-scale real-world benchmark because the reality of generated underwater images had been difficult to examine.

III. IFM-BASED IMAGE RESTORATION

IFM-based underwater restoration models degradation through underwater imaging physics, then estimate background light and transmission maps using priors to recover images. The section surveys dark-channel, underwater-specific, maximum-intensity, and related prior-based approaches, including their limitations in selective attenuation and turbid water.

  • IFM-based restoration: IFM-based restoration first models underwater degradation, estimates background light and transmission maps, and then recovers the restored image.The simplified IFM is presented as a typical model for underwater restoration, with BL and TM as its two key optical parameters.
  • Prior-based restoration: Prior-based methods derive background light and transmission maps from optical properties, condition assumptions, and theoretical priors.The reviewed priors include DCP, UDCP, MIP, RCP, blurriness, and other underwater-specific assumptions.
  • DCP-based image restoration: DCP estimates background light from the top 0.1% brightest dark-channel pixels, but direct underwater use can produce limited improvement and additional color distortion.Subsequent work refined DCP parameter estimation with wavelength compensation, filtering, and related modifications.
  • Summary of prior-based methods: Table 1 organizes mainstream prior-based methods by their background-light formulas, transmission-map formulas, and corresponding priors.The table also defines simplified channelwise parameters and notation for RGB and GB background lights and transmission maps.
  • DCP-based image restoration: UDCP excludes the red channel because red attenuates faster, yet its restored images may remain unsatisfactory because it ignores distinct R and GB imaging characteristics and may fail in turbid water.Underwater-specific variants were developed because selective attenuation affects the conventional dark channel prior.
  • Other prior-based methods: MIP estimates transmission from differences between red and green-blue intensities, while related methods use it for background-light estimation.Other approaches include dual dark channels, blurriness and light absorption, generalized dark channels, histogram templates, and maximum attenuation identification.

B. CNN-BASED IMAGE RESTORATION

CNN-based restoration replaces manually optimized parameter selection with learned estimation of background light or depth maps. Its performance depends on architecture and training data, while synthetic training imagery may restrict generalization across underwater conditions.

  • CNN-based image restoration: Deep-learning restoration shifts parameter selection from fully manual optimization toward automatic training models.CNN-based methods estimate background light or depth maps through feature learning.
  • CNN-based image restoration: White-balance preprocessing before CNN estimation can produce over-saturated and over-enhanced regions because it removes underwater imaging characteristics.A later coarse-to-refined CNN estimated background light and scene depth and claimed improved recovery over existing IFM methods.
  • CNN-based image restoration: Quality-metric-guided CNN restoration can improve underwater visual quality and preserve edges without requiring ground-truth scene radiance.Barbosa et al. used image-quality metrics to guide learning because end-to-end enhancement may otherwise lack ground truth.
  • CNN-based image restoration: CNN restoration performance depends on network architecture design and training data, including when models estimate background light or depth maps.The reviewed methods use learned features to infer parameters required by IFM-based restoration.
  • Limitations: Synthetic underwater training images and architectural defects may limit models to particular underwater image types, while deep-learning restoration can be more time-consuming than physical or non-physical methods.The limitation concerns both adaptation across image conditions and restoration time under the same environment.

IV. QUALITY IMPROVEMENT METHODS FOR UNDERWATER IMAGES: EXPERIMENTAL

The experimental section compares mainstream IFM-based restoration and IFM-free enhancement methods using subjective and objective image-quality analysis. It also evaluates prior-based background-light estimation because BL and TM estimation strongly affect IFM-based restoration.

  • Experimental evaluation: The study compares mainstream IFM-based restoration and IFM-free enhancement methods from both subjective and objective perspectives.Image-quality assessment metrics are introduced before the comparative evaluation.
  • Experimental evaluation: Prior-based background-light estimation models are evaluated because background light determines restored-image color tone and visual effect and influences many transmission-map algorithms.The section analyzes these estimation methods through subjective and objective performance measures.

A. THE METHODS TO BE COMPARISED

The comparison includes seven IFM-free enhancement methods and eight IFM-based restoration methods spanning histogram, color-correction, fusion, dark-channel, maximum-intensity, and attenuation-prior approaches. All methods use standardized image preprocessing and default parameters.

  • IFM-free methods: The IFM-free comparison includes HE, CLAHE, ICM, UCM, Fusion-based enhancement, RD, and RGHS.These methods represent histogram equalization, color correction, fusion, distribution-based enhancement, and histogram stretching approaches.
  • IFM-based methods: The IFM-based comparison includes DCP, MIP, RIR, TEoUI, NOM, RCP, IBLA, and ULAP restoration methods.The methods cover dark-channel, maximum-intensity, optical-model, red-channel, blurriness-absorption, and light-attenuation priors.
  • Evaluation settings: All test images are resized to 400×600 pixels and processed with default parameters for evaluation fairness.Implementations ran on a Windows 7 PC using Python 3.6.3.

B. IMAGE EVALUATION METRICS

The paper evaluates underwater image quality using subjective visual assessment and objective quantitative metrics, including reference-based and no-reference measures. It also describes underwater-specific indices that combine color, sharpness, contrast, chroma, and saturation.

  • Assessment types: Image quality assessment is divided into subjective qualitative assessment and objective quantitative assessment.Subjective assessment relies on human visual impressions, whereas objective assessment uses mathematical quality models.
  • Subjective assessment: Subjective assessment requires repeated experiments and human scoring, making it less efficient and more complicated to operate.
  • Objective assessment: Objective assessment can automatically scrutinize larger datasets and includes full-reference, reduced-reference, and non-reference metrics.The passage notes that full-reference and reduced-reference metrics require or partially require reference information.
  • Objective metrics: Entropy measures image information abundance through average uncertainty, with higher values associated with more uniform contrast and clearer images.
  • No-reference metrics: NIQE uses natural-scene feature modeling, where smaller scores indicate better perceptual quality, while BRISQUE scores higher distortion as worse quality.NIQE models sensitive high-contrast areas with multivariate Gaussian features; BRISQUE measures deviations from a natural-image model.
  • Underwater-specific metrics: UCIQE is an underwater-specific linear quality model combining chromaticity variation, brightness contrast, and saturation; UIQM similarly combines colorfulness, sharpness, and contrast.UCIQE is defined in CIE-Lab color space, while UIQM combines UICM, UISM, and UIConM.

C. ASSESSMENT ON OPTICAL PARAMETERS of IFM-BASED METHODS: BL & TM

The review compares prior-based background-light and transmission-map estimation for IFM-based restoration using subjective and objective analyses. Results show that performance depends strongly on the prior and image conditions, with recurring errors from channel attenuation, artificial light, overestimation, and computational cost.

  • BL estimation: The study evaluates background-light estimation through subjective and objective analysis because background light affects restored color tone and visual effect.Transmission-map algorithms also depend substantially on the estimated background light.
  • BL estimation: The BL comparison uses four underwater scenes, with ground-truth background lights manually annotated by 15 people.The scenes include shallow-sea fish, a low-brightness cliff, wrecked ships, and a foreground swimming batfish.
  • BL estimation: DCP-based BL estimates are often wrong because selecting bright pixels ignores the strong attenuation difference between the R and GB channels.MIP generally approaches the ground truth, but combining MIP with DCP produces excessively bright estimates.
  • BL estimation: RCP estimates are generally correct except for very dark cliff regions, while fusion-based and ULAP-based estimates are closer to the ground truth.Fusion selects among three candidate estimates using selective weighted fusion; ULAP uses R-versus-GB differences related to scene depth.
  • BL estimation: BL accuracy is computed over 300 underwater images using channel tolerances of 30 for R and 40 for GB, with correct estimates accumulated into a ratio.
  • BL estimation: DCP- and UDCP-based BL estimates have the lowest accuracy, while RCP-, fusion-, and ULAP-based methods remain below 80% across all three channels.MIP estimates some images successfully but has relatively low overall accuracy across varied underwater images.
  • TM estimation: TM estimation is judged subjectively because non-reference depth or transmission maps are unavailable, using brightness to indicate relative object distance.Objects nearer the camera should have higher transmission values and appear whiter than distant objects.
  • TM estimation: DCP-based TMs confuse artificial light spots with far backgrounds, while MIP roughly captures depth but overestimates transmission and blurs details.DCP errors are linked to incorrect background-light selection; MIP produces an overall white map.

D. OVERALL PERFORMANCE OF UNDERWATER

The evaluation compares IFM-free enhancement and IFM-based restoration on challenging underwater images using subjective results and five non-reference metrics. IFM-free methods often improve visibility but may amplify noise or produce unnatural colors, while IFM-based methods remain limited in color restoration.

  • Experimental setup: The benchmark evaluates four underwater scenes, including greenish, turbid, and low-visibility conditions, using subjective and objective analyses.IFM-based results also include estimated background lights and transmission maps for discussion.
  • Subjective analysis: IFM-free methods improve contrast, visibility, and luminance but can introduce unnatural chroma, oversaturation, noise, or loss of local detail.HE amplifies noise and red tones; fusion improves contrast and chromaticity while also introducing noise.
  • Subjective analysis: IBLA and ULAP produce the best restoration images by exploiting underwater light attenuation to estimate depth or channel-specific transmission maps.Their optical relationships across RGB channels support more accurate restoration than several alternatives.
  • Discussion: Current IFM-based restoration methods generally perform basic dehazing but cannot effectively restore color across diverse underwater images.Color correction can be added as post-processing, prompting questions about the adequacy of the simplified imaging model.
  • Objective analysis: Higher entropy in IFM-free results indicates greater information abundance, but enhancement may also amplify useless information and noise.HE achieves the highest entropy while appearing unnatural in the visual comparison.
  • Objective analysis: Outdoor-trained BRISQUE and NIQE can rank perceptually unnatural or minimally improved underwater images favorably, limiting their suitability for direct evaluation.TEoUI receives the lowest BRISQUE score despite appearing unnatural, while SIR receives the best NIQE assessment with almost no improvement.
  • Objective analysis: UCIQE and UIQM favor high-contrast, extreme-chroma outputs because they emphasize low-level contrast, chroma, and saturation while ignoring higher-level perception.These metrics rate outputs such as HE and NOM favorably despite their conflict with natural appearance.

V. CONCLUSION AND DISCUSSION

The review organizes single-underwater-image improvement methods, evaluates their progress and challenges, and identifies persistent limitations in robustness, efficiency, datasets, metrics, and task relevance. It concludes that current methods remain insufficient for diverse environments and deep-sea imaging.

  • Conclusion: The review categorizes single-image quality improvement into IFM-free enhancement and IFM-based restoration, then compares state-of-the-art methods with multiple quality metrics.It uses the comparison to discuss current problems and future research directions.
  • Challenges: No current algorithm effectively enhances underwater images across diverse environments, depths, or scenes, so adaptability and robustness remain unresolved.The review explicitly identifies this as a continuing field-wide limitation.
  • Challenges: Traditional enhancement and restoration algorithms have relatively high complexity, limiting the scalability of practical studies and applications.Methods without temporal coherence are also difficult to apply directly to underwater video enhancement because of excessive complexity.
  • Future work: IFM-based methods can recover actual scenes but require substantial time to calculate two key optical parameters, while none of the compared methods improves every test image.Future work should improve robustness and computational efficiency across underwater conditions and applications.
  • Future work: Publicly available benchmark datasets remain insufficient, particularly for paired hazed and clear images, background lights, and depth or transmission maps.These missing resources constrain evaluation and the parameter estimation required by IFM-based restoration.
  • Future work: UCIQE and UIQM do not provide fair underwater quality assessments because they favor over-enhanced colorful images contrary to subjective naturalness.The review calls for an effective metric better aligned with underwater image quality.
  • Future work: Existing enhancement methods prioritize perceptual appearance but overlook whether improved images increase high-level detection and classification accuracy.Haze can make underwater objects resemble their surroundings, worsening recognition and detection difficulty.
  • Future work: Existing methods cannot recover deep-sea images because natural light is fully absorbed below 1000 meters and artificial lighting causes limited range and uneven vignetting.The review identifies a new imaging model for deep-sea enhancement as necessary.
Loading 1907.03246v1…