Source-linked AI summary
Uncertainty Inspired Underwater Image Enhancement
Zhenqi Fu, Wu Wang, Yue Huang, Xinghao Ding, Kai-Kuang Ma
TL;DR
Underwater image enhancement lacks reliable clean-image ground truth, leaving approximate reference labels ambiguous for methods that predict a single result. PUIE-Net estimates a distribution of enhancements and uses consensus to produce a deterministic result; it generates diverse possibilities and achieves competitive performance on two UIE datasets.
Problem
Approximate reference images make clean-image labels ambiguous, while most deep learning UIE methods treat enhancement as point estimation.
Source: §1 · p. 2Method
PUIE-Net learns a distribution of possible enhancements and applies a consensus process to samples to predict a deterministic result.
Source: §1 · p. 2Results
PUIE-Net generates diverse potential solutions, and its consensus prediction achieves promising performance compared with state-of-the-art methods on two UIE datasets.
Source: §1 · p. 3Takeaways & Limitations
The sampled enhancement alternatives can be presented to users for display or analysis.
Source: §3.3 · p. 8
Abstract
from arXiv · showhide
A main challenge faced in the deep learning-based Underwater Image Enhancement (UIE) is that the ground truth high-quality image is unavailable. Most of the existing methods first generate approximate reference maps and then train an enhancement network with certainty. This kind of method fails to handle the ambiguity of the reference map. In this paper, we resolve UIE into distribution estimation and consensus process. We present a novel probabilistic network to learn the enhancement distribution of degraded underwater images. Specifically, we combine conditional variational autoencoder with adaptive instance normalization to construct the enhancement distribution. After that, we adopt a consensus process to predict a deterministic result based on a set of samples from the distribution. By learning the enhancement distribution, our method can cope with the bias introduced in the reference map labeling to some extent. Additionally, the consensus process is useful to capture a robust and stable result. We examined the proposed method on two widely used real-world underwater image enhancement datasets. Experimental results demonstrate that our approach enables sampling possible enhancement predictions. Meanwhile, the consensus estimate yields competitive performance compared with state-of-the-art UIE methods. Code available at https://github.com/zhenqifu/PUIE-Net.
1 Introduction
Underwater light scattering and absorption degrade visibility, contrast, and color, while unavailable clean targets leave UIE methods dependent on ambiguous approximate references. PUIE-Net models possible enhancements as a distribution and uses consensus to produce a deterministic result.
Introduction
- Because clean underwater training images are impractical to capture, approximate references can reflect subjective preferences or algorithm settings, leaving deterministic methods unable to model label ambiguity.For a degraded image, multiple plausible clean-image solutions may exist, so point-estimation methods compromise among possible outcomes.Source: §1 · p. 2
- PUIE-Net models UIE as enhancement-distribution estimation rather than a single prediction, then uses Monte Carlo likelihood estimation (MC) or maximum-probability estimation (MP) for consensus.MC averages likelihoods, while MP selects the sample with the highest probability.Source: §1 · pp. 2–3
- PAdaIN combines conditional variational autoencoders (CVAE) with adaptive instance normalization (AdaIN) to construct an enhancement distribution and transform global enhancement statistics.Sampling enhancement attributes from the latent space produces diverse predictions; the pipeline follows standard CVAE training.Source: §1 · pp. 2–3
- PUIE-Net generates diverse potential enhancements, and its consensus prediction achieves promising performance against state-of-the-art methods on two real-world UIE datasets.The paper presents the network as the first probabilistic approach for UIE.Source: §1 · p. 3
2 Related Work
Related work groups underwater image enhancement (UIE) into model-free, physical-prior, and deep-learning approaches. It also describes VAEs as latent-distribution models used for diverse predictions in other vision tasks, but not yet to model UIE ambiguity.
2.1 Underwater Image Enhancement
- UIE methods are grouped into model-free enhancement, physical-model-based priors, and deep learning that learns enhancement mappings from paired or unpaired data.Examples include histogram equalization, white balance, and Retinex; dehazing and imaging-model estimation; and learned fusion, adversarial, and other neural approaches.Source: §2.1 · pp. 3–4
- Prior-based methods estimate physical-model parameters, including depth, attenuation, and backscatter, with Sea-thru using RGBD inputs and a revised imaging model.Other examples include dehazing, a red-channel variant of the Dark Channel Prior, and minimum-information-loss and histogram-prior methods.Source: §2.1 · pp. 3–4
- Deep learning methods learn enhancement mappings from paired or unpaired data; examples include synthetic-data training, weak supervision, and the real-world dataset and gated fusion network.Other approaches address degradation diversity with Jerlov-water-type domain-agnostic features and use medium transmission-guided multi-color-space embedding in Ucolor.Source: §2.1 · p. 4
2.2 VAE based Deep Probabilistic Model
- VAEs represent each latent attribute as a probability distribution rather than a single value, and training uses regularization and reconstruction losses to align posterior and prior distributions.Source: §2.2 · p. 4
- VAEs and CVAEs support diverse predictions in tasks including segmentation, denoising, and saliency, but prior work had not used them to model ambiguity in UIE.Applications also include salient-object backgrounds, motion generation, and multiple deprojected instances.Source: §2.2 · pp. 4–5
3 Method
PUIE-Net models underwater enhancement as a distribution of possible outputs, using PAdaIN to represent appearance uncertainty and a consensus estimate to produce a deterministic result. Its two-branch CVAE network learns from multiple reference images generated to represent ambiguity in the labels.
3.1 Motivation
- Because clean images are unavailable and approximate labels are uncertain, PUIE-Net models possible interpretations using a latent variable z.The paper interprets z as representing subjective preferences or camera/algorithm parameters involved in recording the ground truth.Source: Ambiguities for UIE · p. 5
Ambiguities for UIE
- The probabilistic formulation generates multiple enhancement predictions; MC averages sampled likelihoods, while MP selects the sample with maximum probability.The paper presents both as ways to obtain a deterministic estimate from the learned distribution.Source: Ambiguities for UIE · p. 5
PAdaIN for Learning Appearance Distribution
- PAdaIN samples feature means and standard deviations from learned posterior distributions, enabling appearance adjustment without a known style image.The paper describes PAdaIN as a generalized AdaIN that transforms feature statistics using these random samples.Source: PAdaIN for Learning Appearance… · p. 6
3.2 Network Structure
- PUIE-Net uses U-Net feature extractors with SE-ResNet blocks and prior/posterior branches to learn enhancement distributions from degraded images and references.At test time, latent codes depend only on the input image; during training, the posterior uses both the input and corresponding reference.Source: §3.2 · pp. 6–7
3.3 Training and Testing
- Training uses CVAE posterior samples to predict enhanced images, combining MSE and VGG16 perceptual losses with KL divergences that align posterior and prior distributions.At test time, the network can produce n variants by re-evaluating PAdaIN and the output block; MC averages samples and MP selects the most probable one.Source: §3.3 · p. 8
3.4 Training Data Generation
- To represent label uncertainty, the authors expand UIEBD’s 890 images to four reference maps per underwater image using contrast and saturation adjustments and gamma correction.For each adjustment method, they generate over- and under-adjusted versions and select the better one as a potential label, retaining the original reference.Source: §3.4 · p. 9, Figure 2 · p. 9
4 Experiments
PUIE-Net models a distribution of possible underwater enhancements and uses consensus to produce a deterministic result. Experiments on UIEBD and RUIE report favorable performance, with MC performing significantly better than other methods.
4.1 Experiment Settings
- The model trains on 700 UIEBD images and is tested on the remaining images and RUIE’s 3,630-image quality subset.Evaluation uses SSIM, PSNR, DeltaE, NIQE, and MOS; on RUIE, NIQE and MOS are used. Nine methods are compared using authors’ original implementations.Source: §4.1 · p. 10
4.2 Analysis of Enhancement Distribution
- PUIE-Net samples diverse, reasonable enhancements: higher-probability samples are milder, while lower-probability samples apply stronger contrast and color adjustments.The samples provide alternatives for display or analysis and support a robust, stable consensus result, while reducing the influence of biased labels.Source: §4.2 · pp. 11–12, Figure 3 · p. 11
4.3 Performance Comparison
- PUIE-Net achieves favorable performance and outperforms other methods on UIEBD and RUIE; MC performs significantly better, and its visual results look cleaner and more natural.GC and Retinex show unnatural colors, saturation, and smearing; prior-based methods degrade colors, while Water-Net and Ucolor produce over- or under-enhanced results.Source: Table 1 · p. 12, §4.3 · p. 12, Figure 4 · p. 13
4.4 Impact of Sampling Times
- As sampling increases, PSNR and SSIM variability decreases before stabilizing after 20 samples; the paper uses 20 to balance stability and runtime.The standard deviations are calculated over 10 model runs for each test image, and further sampling can improve final-prediction stability.Source: §4.4 · pp. 12–13, Table 2 · p. 13
4.5 Discussion
- MC averages a group of enhancement samples, whereas MP selects the sample with the highest probability; other consensus options include selecting by image-quality assessment.The resulting enhancement performance depends on the subjective or objective quality-assessment method used for selection.Source: Consensus Process · pp. 13–14
Consensus Process
- Rather than requiring ground truth or discarding biased reference maps, PUIE-Net uses AdaIN and a conditional variational autoencoder to learn enhancement distributions from those maps.The paper presents distribution estimation followed by consensus as a different approach from unsupervised UIE methods to the lack of ground truth.Source: Consensus Process · p. 14
5 Conclusion
PUIE-Net models underwater enhancement as a distribution of possible images and uses consensus to produce a final result. The authors report reasonable sampled results and competitive consensus visual quality on two real-world UIE datasets.
5 Conclusion
- PUIE-Net learns a distribution of enhanced images to address label ambiguity rather than estimating one certain result.It uses random style attributes from two conditional variational autoencoder posterior distributions to transform global enhancement statistics of input features.Source: §5 · p. 14
- A consensus process integrates a set of predictions to infer the final enhancement result.Source: §5 · p. 14
- PUIE-Net produces reasonable enhancement predictions, while consensus visual quality is highly competitive on two real-world UIE datasets.Source: §5 · p. 14
- The authors plan to extend the method to low-light enhancement, dehazing, and denoising, where ground-truth images are difficult to obtain.Source: §5 · p. 14