PaperlayerView exact revision on arXiv

Source-linked AI summary

An Experimental-based Review of Image Enhancement and Image Restoration Methods for Underwater Imaging

Yan Wang, Wei Song, Giancarlo Fortino, Lizhe Qi, Wenqiang Zhang, Antonio Liotta

arXiv:1907.03246v1eess.IVcs.CVcs.MM

TL;DR

Underwater images are degraded by turbidity, light absorption and scattering, and the applicability of newer general enhancement methods to underwater images has not been specifically captured. This paper reviews IFM-free and IFM-based methods and experimentally compares them, finding that current IFM-based restoration achieves basic dehazing but not effective color restoration across varied underwater images.

  • Problem

    Underwater degradation from turbidity, light absorption and scattering makes image restoration challenging, while the applicability of newer general enhancement methods to underwater images has not been specifically captured.

    Source: §I · pp. 2–3

  • Method

    The paper reviews IFM-free enhancement and IFM-based restoration methods and experimentally compares state-of-the-art approaches using multiple quality-assessment metrics.

    Source: §I · p. 3, §V · p. 15

  • Results

    Current IFM-based restoration methods achieve basic dehazing but cannot effectively restore color across varied underwater images.

    Source: D. OVERALL PERFORMANCE OF UNDERWATER… · p. 13

  • Takeaways & Limitations

    The findings raise whether the simplified imaging formation model is appropriate for underwater image restoration.

    Source: D. OVERALL PERFORMANCE OF UNDERWATER… · p. 13

    Traditional enhancement and restoration algorithms have relatively high complexity, limiting the scale of practical studies and applications.

    Source: §V · p. 15

Abstract

from arXiv · show

Underwater images play a key role in ocean exploration, but often suffer from severe quality degradation due to light absorption and scattering in water medium. Although major breakthroughs have been made recently in the general area of image enhancement and restoration, the applicability of new methods for improving the quality of underwater images has not specifically been captured. In this paper, we review the image enhancement and restoration methods that tackle typical underwater image impairments, including some extreme degradations and distortions. Firstly, we introduce the key causes of quality reduction in underwater images, in terms of the underwater image formation model (IFM). Then, we review underwater restoration methods, considering both the IFM-free and the IFM-based approaches. Next, we present an experimental-based comparative evaluation of state-of-the-art IFM-free and IFM-based methods, considering also the prior-based parameter estimation algorithms of the IFM-based methods, using both subjective and objective analysis (the used code is freely available at https://github.com/wangyanckxx/Single-Underwater-Image-Enhancement-and-Color-Restoration). Starting from this study, we pinpoint the key shortcomings of existing methods, drawing recommendations for future research in this area. Our review of underwater image enhancement and restoration provides researchers with the necessary background to appreciate challenges and opportunities in this important field.

I. INTRODUCTION

The paper reviews methods for improving single underwater images, organizes them by whether they use an image formation model, and compares representative methods while identifying challenges for future work.

I. INTRODUCTION

  • Underwater imaging supports marine exploration and applications including robotics, rescue, infrastructure inspection, ecological monitoring, and navigation.The paper notes that specialized hardware can be costly, power-consuming, and poorly adapted to different underwater environments.Source: §I · pp. 1, 3
  • Underwater images can have color casts, low contrast, blur, haze, uneven illumination, and shadowing as water absorbs and scatters light.Red light attenuates faster than green and blue; the simplified image formation model represents captured light as direct transmission plus background scattering.Source: §I · pp. 1–2
  • Earlier reviews left classifications incomplete, including newer deep-learning work, and did not clarify how much specific methods improve image quality.The paper presents its broader review as addressing these gaps and offering an up-to-date snapshot of challenges and future directions.Source: §I · p. 3
  • The paper provides a broad review and an experimental comparison of representative IFM-free enhancement and IFM-based restoration methods using different quality metrics.The authors state that code is available to support replicability.Source: §I · p. 3
  • It classifies single-underwater-image quality improvement methods as IFM-free enhancement or IFM-based restoration, according to whether they use the optical imaging model.The categories are presented as a guide to which models may suit different problem domains.Source: §I · p. 3, Figure 2 · p. 3
  • The review critically evaluates prior-knowledge-based restoration methods and identifies issues in estimating underwater image restoration parameters.It also shares lessons learned about difficulties and problems in underwater image quality improvement.Source: §I · p. 3

II. IFM-FREE IMAGE ENHANCEMENT

IFM-free underwater enhancement improves contrast and color by redistributing pixel intensities without modeling underwater imaging principles. The review groups methods by processing domain and by whether they use CNNs.

II. IFM-FREE IMAGE ENHANCEMENT

  • IFM-free methods improve underwater-image contrast and color mainly through pixel-intensity redistribution, without considering underwater imaging principles.They modify pixels in either the spatial domain or a transformed domain.Source: §II · p. 3
  • Early studies applied outdoor enhancement methods to underwater images, while later methods targeted underwater haze, color cast, and low contrast.More recently, CNN-based models have used learned hidden features for quality enhancement.Source: §II · p. 3
  • The review divides IFM-free methods into spatial-domain –, transform-domain –, and CNN-based – enhancement.Source: §II · p. 3

A. SPATIAL-DOMAIN IMAGE ENHANCEMENT

Spatial-domain enhancement redistributes concentrated underwater-image pixel histograms across color models, with methods divided into single- and multiple-color-model approaches.

A. SPATIAL-DOMAIN IMAGE ENHANCEMENT

  • Histogram redistribution expands gray levels to improve visibility, using either a single color model (SCM) or multiple color models (MCM).Common models include RGB, HSI, HSV, and CIE-Lab.Source: A. SPATIAL-DOMAIN IMAGE ENHANCEMENT · p. 4

1) SCM-based image enhancement

  • Directly applying common contrast and color-correction methods to underwater images can create artifacts, halos, noise, or color distortion, while GEA often fails.The low energy of RGB components and hazy edges are cited as contributing conditions.Source: A. SPATIAL-DOMAIN IMAGE ENHANCEMENT · p. 4
  • Fusion methods combine white-balanced and contrast-enhanced images with weights and multiscale fusion; Ancuti et al., report improved exposure, contrast, and edge sharpness.The 2017 method gamma-corrects and sharpens underwater white balancing before merging the fusion images and weight maps.Source: A. SPATIAL-DOMAIN IMAGE ENHANCEMENT · p. 4
  • DSNMF estimates local-block illumination through sparse matrix factorization and uses it to adjust underwater images.The final factorization layer supplies the patch illumination estimate.Source: A. SPATIAL-DOMAIN IMAGE ENHANCEMENT · p. 4

2) MCM-based image enhancement

  • MCM approaches include CIE-Lab MRF enhancement, RGB-to-HSI stretching, and Von Kries-based UCM, which targets blue-green cast and dim components.The MRF method uses maximum a posteriori estimation; ICM stretches attenuated RGB channels, then HSI saturation and intensity.Source: A. SPATIAL-DOMAIN IMAGE ENHANCEMENT · p. 4
  • Rayleigh redistribution variants of ICM and UCM seek to improve contrast while reducing over-enhancement, oversaturation, and noise,,; RAHIM modifies HSV saturation and brightness.RAHIM uses a Rayleigh distribution and human visual system before converting the result to RGB.Source: A. SPATIAL-DOMAIN IMAGE ENHANCEMENT · p. 4
  • Retinex methods separate direct and reflected light to enhance contrast, while a later framework uses bilateral and trilateral filtering to suppress halo artifacts.The later method filters Lab brightness and color components.Source: A. SPATIAL-DOMAIN IMAGE ENHANCEMENT · pp. 4–5
  • Mix-CLAHE combines RGB- and HSV-based CLAHE images to enhance contrast and reduce noise and artifacts, while RGHS adaptively stretches RGB and Lab components to retain image information.RGHS uses linear and curve-adaptive stretching for Lab brightness and color components, respectively.Source: A. SPATIAL-DOMAIN IMAGE ENHANCEMENT · p. 5

B. TRANSFORM-DOMAIN IMAGE ENHANCEMNT · C. CNN-BASED IMAGE ENHANCEMNT

Transform-domain methods enhance underwater images by manipulating frequency components, while CNN-based methods learn image transformations despite challenges in obtaining underwater training data.

B. TRANSFORM-DOMAIN IMAGE ENHANCEMNT

  • Frequency-domain methods enhance hazy images by amplifying high-frequency edge components and suppressing low-frequency background components, whose contrast is often weak.These methods commonly transform the spatial image into the frequency domain before enhancement.Source: B. TRANSFORM-DOMAIN IMAGE ENHANCEMNT · p. 5
  • Examples include Prabhakar et al.’s illumination correction and denoising, Amjad et al.’s wavelet fusion for contrast and color issues, and Vasamsetti et al.’s RGB-channel DWT enhancement.Vasamsetti et al. also described the method as preprocessing for underwater detection and tracking.Source: B. TRANSFORM-DOMAIN IMAGE ENHANCEMNT · p. 5
  • Transform-domain methods can improve visibility and contrast, but may over-amplify noise and cause color distortion.Source: B. TRANSFORM-DOMAIN IMAGE ENHANCEMNT · p. 5

C. CNN-BASED IMAGE ENHANCEMNT

  • Although CNNs have succeeded in many low-level vision tasks, few methods have been effective for underwater image enhancement.Source: C. CNN-BASED IMAGE ENHANCEMNT · p. 5
  • Perez et al. learned paired hazy-to-clear mapping, Wang et al.’s UIE-net targets color correction and haze removal, and UWCNN generalized across real and synthetic scenes.UWCNN was trained using synthetic underwater images produced indoors.Source: C. CNN-BASED IMAGE ENHANCEMNT · pp. 5–6
  • WaterGAN generated underwater images from in-air images and depth for color-cast correction, while Fabbri et al. used CycleGAN-prepared pairs to train Underwater-GAN.Source: C. CNN-BASED IMAGE ENHANCEMNT · p. 6
  • To reduce paired-data dependence, Li et al. used adversarial, cycle-consistency, and SSIM losses for color correction; later methods used perceptual-loss WGAN and domain-agnostic features.The weakly supervised model was described as preserving input content and structure while correcting color distortion.Source: C. CNN-BASED IMAGE ENHANCEMNT · p. 6
  • Because the realism of generated underwater images had rarely been examined, Li et al. built UIEBD and trained DUIENet to learn three confidence maps with gated fusion.Source: C. CNN-BASED IMAGE ENHANCEMNT · p. 6

III. IFM-BASED IMAGE RESTORATION · A. PRIOR-BASED IMAGE RESTORATION

IFM-based restoration estimates background light (BL) and transmission maps (TMs) from prior knowledge, then uses them in the image-formation model to recover underwater colors. Prior-based methods derive these parameters from optical properties or haze-related cues.

III. IFM-BASED IMAGE RESTORATION

  • IFM-based restoration estimates two optical parameters—background light (BL) and transmission map (TM)—then uses them in the image-formation model to restore images.The review describes restoration as constructing a degradation model from underwater imaging physics, estimating its parameters from prior knowledge, and applying compensation processing.Source: §III · p. 6

A. PRIOR-BASED IMAGE RESTORATION

  • Prior methods use cues such as DCP, UDCP, MIP, RCP, blurriness, and underwater light attenuation to derive BL and TM or depth for restoration.The priors address underwater light absorption and scattering, including selective attenuation and haze-like effects.Source: A. PRIOR-BASED IMAGE RESTORATION · p. 6

IBLA/ IBLA

  • DCP-based underwater dehazing adapts a clear-image low-intensity-pixel prior; direct application estimates BL from the brightest dark-channel pixels and TM by minimizing the IFM.Chao et al. reported limited improvement and additional color distortion; Yang et al. used median filtering instead of soft matting, but their method suited richly colored scenes and could not recover color-cast or dim images.Source: A. PRIOR-BASED IMAGE RESTORATION · p. 6, IBLA/ IBLA · p. 7
  • DCP parameter refinements compensate attenuation and haze: WCID addresses artificial light, while joint trigonometric filtering improves TM and image contrast.Zhao et al. derived GB-channel TMs from the red-channel TM using attenuation coefficients; Peng et al. estimated BL by averaging input intensities at the brightest dark-channel pixels.Source: IBLA/ IBLA · p. 7, ULAP/ ULAP · p. 7

ULAP/ ULAP

  • UDCP excludes red because it attenuates faster than green and blue; although it can estimate TM more accurately than DCP, it may fail in turbid water.UDCP and a similar scattering-rate method ignore differences between red and green-blue imaging characteristics, leaving restored images unsatisfactory.Source: ULAP/ ULAP · pp. 7–8
  • Underwater-specific alternatives include dual red-and-blue dark channels, UDCP with red-channel correction, and red-channel prior restoration.The dual-channel method uses a weighted median filter to reduce halo and mosaic effects; the UDCP method adds Gray-World correction and adaptive exposure balancing, while RCP uses saturation to adjust TM but can produce visually unreal colors.Source: ULAP/ ULAP · p. 8
  • MIP estimates coarse scene depth from the difference between maximum red intensity and maximum green-blue intensity.MIP was also modified for BL estimation by assuming background red intensity is relatively low, or combined with DCP to select a pixel with maximum blue-green or green-red difference.Source: ULAP/ ULAP · p. 8

B. CNN-BASED IMAGE RESTORATION

CNN-based restoration marks a shift from manually optimized parameter selection toward automatic training models. These methods estimate background light or depth maps through learned features, but their performance depends on network design and training data.

B. CNN-BASED IMAGE RESTORATION

  • Cao et al. stacked coarse global and refined CNNs to estimate background light and depth, claiming better recovery than existing IFM-based methods.Ding et al. used color correction before CNN estimation of background light and transmission, but this preprocessing can discard underwater imaging characteristics and cause oversaturation or over-enhancement.Source: B. CNN-BASED IMAGE RESTORATION · pp. 8–9
  • Barbosa et al. guided CNN restoration with image-quality metrics without ground-truth radiance, improving visual quality while preserving edges; Hou et al. combined data-driven transmission estimation with knowledge-driven illumination correction.Source: B. CNN-BASED IMAGE RESTORATION · p. 9
  • CNN restoration may generalize only to limited underwater image types because training uses synthetic images and deep-learning architectures may have defects; it is also slower than physical and non-physical models.Source: B. CNN-BASED IMAGE RESTORATION · p. 9

IV. QUALITY IMPROVEMENT METHODS FOR UNDERWATER IMAGES: EXPERIMENTAL · COMPARISONS

The section outlines a comparative evaluation of underwater image quality improvement methods, covering both restoration and enhancement methods and prior-based estimation for image formation models.

COMPARISONS

  • The comparisons assess mainstream IFM-based restoration and IFM-free enhancement methods using subjective and objective perspectives.The section first introduces image quality assessment metrics.Source: COMPARISONS · p. 9
  • The evaluation examines prior-based backscatter light (BL) and transmission map (TM) estimation models, including their advantages, disadvantages, and the effect of BL results on TM estimation.The section identifies BL and TM estimation as determinants of IFM-based methods’ robustness and effectiveness.Source: COMPARISONS · p. 9

A. THE METHODS TO BE COMPARISED

The comparison covers seven IFM-free enhancement methods and eight IFM-based restoration methods, evaluated with standardized image-size and default-parameter settings.

A. THE METHODS TO BE COMPARISED

  • Compared IFM-free methods included HE, CLAHE, integrated colour model (ICM), unsupervised colour correction method (UCM), Fusion-based (FB), Rayleigh-distribution (RD), and relative global histogram stretching (RGHS).Source: A. THE METHODS TO BE COMPARISED · p. 9
  • IFM-based comparisons covered DCP-based SIR, MIP-based IUID, RIR, TEoUI, NOM, RCP, IBLA, and ULAP.These include dark-channel-prior removal, maximum-intensity-prior dehazing, rapid restoration, transmission estimation, a new optical model, red-channel-prior restoration, blurriness and light absorption, and underwater light attenuation prior methods.Source: A. THE METHODS TO BE COMPARISED · p. 9
  • For fair evaluation, test images were resized to 400×600 pixels, methods used default parameters, and runs used Python 3.6.3 on Windows 7 with an Intel i7-4790U and 8 GB RAM.The machine had 1600 MHz DDR3 memory.Source: A. THE METHODS TO BE COMPARISED · p. 9

B. IMAGE EVALUATION METRICS

The section distinguishes subjective qualitative assessment from objective quantitative assessment and uses multiple non-reference metrics because suitable underwater reference images are generally unavailable.

B. IMAGE EVALUATION METRICS

  • SQA relies on human observers scoring image datasets across repeated experiments, but the paper presents representative method results because this process is inefficient and complicated.The experiments vary factors affecting image quality and seek statistical significance.Source: B. IMAGE EVALUATION METRICS · p. 9
  • Because underwater reference images are generally unavailable, the study uses multiple non-reference metrics spanning information richness, naturalness, sharpness, and contrast, chroma, and saturation.Objective assessment also includes full-reference and reduced-reference metrics, but both require at least a partial high-quality reference.Source: B. IMAGE EVALUATION METRICS · p. 9
  • NIQE assigns better perceptual quality to smaller scores, whereas BRISQUE rates naturalness deviations on a 0–100 scale, with higher values indicating worse quality.NIQE models high-contrast sensitive areas using multivariate Gaussian features; BRISQUE measures deviations from natural scene statistics.Source: B. IMAGE EVALUATION METRICS · p. 10
  • UCIQE combines contrast, chroma, and saturation in CIE-Lab space, while UIQM combines underwater colorfulness, sharpness, and contrast; higher scores indicate better quality.UCIQE uses chromaticity standard deviation, brightness contrast, and average saturation; UIQM combines UICM, UISM, and UIConM.Source: B. IMAGE EVALUATION METRICS · p. 10

C. ASSESSMENT ON OPTICAL PARAMETERS of IFM-BASED METHODS: BL & TM

This section compares IFM-based background-light (BL) and transmission-map (TM) estimation methods using different priors. It finds substantial variation in their accuracy and applicability across underwater scenes.

1) Comparisons of BL estimation models

  • MIP-based BL estimates were often close to ground truth, whereas applying MIP on DCP produced overly bright estimates; RCP failed on the very dark cliff scene.Fusion outperformed DCP and UDCP, and ULAP estimates were close to ground truth.Source: C. ASSESSMENT ON OPTICAL PARAMETERS of… · p. 10, Figure 3 · p. 11
  • BL accuracy was assessed across 300 images using annotated ground-truth BLs and channel tolerances, while TM correctness was judged subjectively because reference depth or transmission maps were unavailable.For TM assessment, nearer objects were expected to have higher, whiter values than distant objects.Source: C. ASSESSMENT ON OPTICAL PARAMETERS of… · pp. 10–11
  • DCP- and UDCP-based BL estimates had the lowest accuracy; RCP, Fusion, and ULAP performed better, but remained below 80% accuracy in each RGB channel.MIP also had relatively low overall accuracy despite successfully estimating BLs in some test images.Source: C. ASSESSMENT ON OPTICAL PARAMETERS of… · p. 11, Figure 4 · p. 11

2) Comparisons of TM estimation models

  • DCP-based TMs often failed to distinguish depth in complex scenes and mistook artificial lights for distant areas; MIP maps were too large, while UDCP improved DCP partly and RCP maps were too large for restoration.DCP methods were sensitive to image characteristics, and MIP maps obscured details with an overall white appearance.Source: C. ASSESSMENT ON OPTICAL PARAMETERS of… · p. 12
  • IBLA produced more accurate TMs with local details but had high computational complexity; ULAP was unaffected by artificial light but underestimated shoal TM and was sensitive to large R-versus-GB intensity differences.ULAP quickly estimated TMs in the other three test images, including artificial-light regions.Source: C. ASSESSMENT ON OPTICAL PARAMETERS of… · p. 12

D. OVERALL PERFORMANCE OF UNDERWATER IMAGE ENHANCEMENT AND IMAGE RESTORATION: EVALUATION AND DISCUSSION

The review compares IFM-free enhancement and IFM-based restoration across one clear and three challenging underwater scenes using subjective and objective assessment; both approaches have important limitations in image quality and evaluation.

Subjective Analysis

  • HE produces red casts and amplified noise; CLAHE and RGHS avoid over-enhancement, RGHS dehazes better, ICM/UCM risk saturation, RD veils detail, and FB adds noise.FB also improves contrast and chromaticity; RD combines ICM and UCM with a Rayleigh distribution to reduce under- and over-enhanced areas.Source: D. OVERALL PERFORMANCE OF UNDERWATER… · p. 13
  • Across the four-scene benchmark, IFM-free methods improve contrast, visibility, and luminance but introduce unnatural chroma and noise, while IFM-based methods largely provide basic dehazing without reliable color restoration.The authors suggest color correction as post-processing and question whether the simplified image formation model is suitable for underwater restoration.Source: D. OVERALL PERFORMANCE OF UNDERWATER… · p. 13
  • SIR, RIR, IUID, TEoUI, and NOM exhibit distinct failures, including failed estimation, reddish distortion, persistent haze or color cast, oversaturation, and channel or noise distortion.SIR's failure does not affect the clear scene because its estimated transmission map is flat; IUID's maps cannot distinguish scene depth, and TEoUI's large background-light estimates oversaturate the clear image.Source: D. OVERALL PERFORMANCE OF UNDERWATER… · p. 13
  • IBLA and ULAP produce the best restoration images by using underwater light attenuation and RGB optical relationships to estimate depth maps or channel transmission maps.The compared BLs, TMs, and restored images are shown in Figure 7.Source: D. OVERALL PERFORMANCE OF UNDERWATER… · p. 13, Figure 7 · p. 14

Objective Analysis

  • Outdoor-trained BRISQUE and NIQE can misrank underwater quality, while UCIQE and UIQM favor IFM-free results yet reward extreme contrast or chroma and ignore higher-level perceptual information.TEoUI looks unnatural despite the best BRISQUE score, and SIR earns the best NIQE assessment despite barely improving the original images.Source: D. OVERALL PERFORMANCE OF UNDERWATER… · pp. 14–15
  • IFM-free outputs generally have higher ENTROPY, but this can reflect noise amplification; HE has the highest entropy despite producing unnatural images.Table 2 reports averages for ENTROPY, BRISQUE, NIQE, UIQM, and UCIQE; the supplied text gives no numerical values.Source: Figure 6 · p. 13, D. OVERALL PERFORMANCE OF UNDERWATER… · p. 14, Table 2 · p. 15

V. CONCLUSION AND DISCUSSION

The review outlines underwater imaging and light absorption, classifies single-image improvement methods as IFM-free enhancement or IFM-based restoration, and compares state-of-the-art methods using multiple quality metrics. It identifies limitations and research priorities for the field.

V. CONCLUSION AND DISCUSSION

  • No compared method improves every test image: IFM-based restoration recovers actual scenes but is slow, while IFM-free enhancement is fast but can distort color.Future methods should improve robustness and efficiency, adapt to varied underwater conditions and applications, and combine restoration with enhancement; excessive complexity also limits scaling and use of single-image methods for video enhancement.Source: §V · pp. 15–16
  • A public benchmark dataset should include paired hazed and clear images, background lights, and depth or transmission maps to support restoration evaluation and learning-based methods.Learning-based enhancement depends on paired data, while synthetic training images differ from real underwater captures; the lack of benchmark data also leaves estimated background lights and transmission maps assessed subjectively.Source: §V · p. 16
  • UCIQE and UIQM can favor over-enhanced colorful images despite conflicting with subjective preferences for naturalness, so underwater quality assessment needs better non-reference models and combined evaluation.The review recommends combining subjective and objective assessment while continuing to improve non-reference evaluation models.Source: §V · p. 16
  • Enhancement should be evaluated for its effect on high-level tasks such as target detection under degraded visibility, not only its perceptual image quality.The review notes that hazed underwater objects resemble their surroundings, complicating recognition and detection, and proposes task completion as an evaluation criterion.Source: §V · p. 16
  • Deep-sea enhancement needs a new imaging model because artificial lighting below 1000 meters causes uneven illumination, vignetting, scattering interference, low brightness, and unrecoverable images for existing methods.The model should address light attenuation and improve realism while reducing halo effects.Source: §V · p. 16

Related papers

Loading 1907.03246v1…