Source-linked AI summary
Uncertainty Inspired Underwater Image Enhancement
Zhenqi Fu, Wu Wang, Yue Huang, Xinghao Ding, Kai-Kuang Ma
TL;DR
Deep learning UIE lacks reliable clean ground truth because existing reference maps are approximate and ambiguous. PUIE-Net learns an enhancement distribution with CVAE and AdaIN, then applies consensus to produce deterministic results; experiments on two real-world datasets show diverse predictions and competitive consensus performance.
Problem
UIE methods commonly train against approximate reference images even though multiple plausible clean images may correspond to one degraded underwater image.
Method
PUIE-Net combines conditional variational autoencoding with adaptive instance normalization to learn enhancement distributions and uses MC or MP consensus to select deterministic results.
Results
PUIE-Net generates diverse enhancement predictions, while consensus estimates achieve promising or highly competitive performance on two real-world UIE datasets.
Takeaways & Limitations
Distribution estimation and consensus provide a flexible UIE framework that can reduce the influence of biased reference labels while producing stable results.
Abstract
from arXiv · showhide
A main challenge faced in the deep learning-based Underwater Image Enhancement (UIE) is that the ground truth high-quality image is unavailable. Most of the existing methods first generate approximate reference maps and then train an enhancement network with certainty. This kind of method fails to handle the ambiguity of the reference map. In this paper, we resolve UIE into distribution estimation and consensus process. We present a novel probabilistic network to learn the enhancement distribution of degraded underwater images. Specifically, we combine conditional variational autoencoder with adaptive instance normalization to construct the enhancement distribution. After that, we adopt a consensus process to predict a deterministic result based on a set of samples from the distribution. By learning the enhancement distribution, our method can cope with the bias introduced in the reference map labeling to some extent. Additionally, the consensus process is useful to capture a robust and stable result. We examined the proposed method on two widely used real-world underwater image enhancement datasets. Experimental results demonstrate that our approach enables sampling possible enhancement predictions. Meanwhile, the consensus estimate yields competitive performance compared with state-of-the-art UIE methods. Code available at https://github.com/zhenqifu/PUIE-Net.
1 Introduction
Deep learning UIE commonly relies on approximate reference images despite ambiguous ground truth labels. PUIE-Net models enhancement distributions with CVAE and AdaIN, then uses consensus estimation to obtain deterministic results.
- Motivation: Underwater UIE is difficult because clean ground-truth images are impractical to capture and degradation is complex and non-reversible.Existing methods therefore generate approximate supervisors, but these labels may reflect subjective preferences and algorithm parameters.
- Motivation: Approximate reference labels create multiple plausible clean solutions, making direct point estimation inappropriate for ambiguous underwater images.Deterministic methods must compromise among possible results when the true clean image is unknown.
- Method: PUIE-Net resolves UIE into distribution estimation and consensus processing rather than producing only one certain prediction.The network is presented as the first probabilistic network for UIE and is designed to address uncertainty in ground-truth labeling.
- Consensus: Consensus prediction uses samples from the learned distribution to produce a deterministic result, with MC and MP proposed as two processes.MC averages likelihoods, while MP selects the sample with maximum probability.
- Method: Conditional variational autoencoding and adaptive instance normalization construct an enhancement distribution that can generate diverse potential predictions.PAdaIN transforms global enhancement statistics using sampled style attributes from posterior distributions.
- Results: On two real-world UIE datasets, consensus estimation achieves promising performance compared with state-of-the-art methods.The method also generates diverse potential solutions for display or analysis.
2 Related Work
Related UIE methods include model-free, prior-based, and deep learning-based approaches. VAEs and CVAEs provide a probabilistic framework for sampling diverse outputs across vision tasks, motivating their use for ambiguous UIE.
- Model-free methods: Model-free UIE methods enhance degraded images without explicitly modeling the underwater degradation process.Examples include CLAHE, white balance, Retinex, and fusion-based methods.
- Prior-based methods: Prior-based methods use physical imaging models and estimate their parameters to enhance underwater images.Examples include dehazing, dark-channel, minimum-information-loss, histogram, and revised physical-model approaches.
- Deep learning methods: Deep learning-based methods learn enhancement mappings from paired or unpaired training data, including synthetic-data and weakly supervised pipelines.These approaches automatically extract representations from training examples.
- Probabilistic models: VAEs and CVAEs represent latent attributes as probability distributions and use regularization and reconstruction losses during training.This probabilistic formulation supports sampling rather than describing each latent state with a single value.
- Probabilistic models: Prior work uses VAEs for diverse outputs in salient-object detection, motion generation, medical segmentation, denoising, and deprojection.The related applications demonstrate probabilistic sampling across several computer-vision tasks, though not previously for UIE.
3 Method
PUIE-Net models underwater enhancement as a distribution rather than a single prediction, using PAdaIN to represent ambiguous appearance statistics and consensus processes to produce deterministic results.
- 3.1 Motivation: PUIE-Net introduces a probabilistic network for UIE because unavailable clean images and ambiguous reference labels make deterministic mappings inappropriate.The method estimates possible interpretations of a degraded image instead of directly selecting one point estimate.
- 3.1 Motivation: The method formulates UIE through uncertainty distributions and uses Monte Carlo likelihood estimation or Maximum Probability estimation for consensus prediction.MC averages likelihoods from sampled predictions, whereas MP selects the enhancement sample with maximum probability.
- 3.2 Network Structure: PAdaIN combines CVAE-based posterior sampling with AdaIN to transform feature statistics and generate diverse enhancement predictions.AdaIN adjusts feature means and standard deviations; PAdaIN samples enhancement attributes from learned distributions instead of relying on an available style input.
- 3.2 Network Structure: PUIE-Net uses prior and posterior branches to construct distributions for enhancement means and standard deviations, then injects sampled values into AdaIN.The prior branch uses a raw underwater image, while the posterior branch uses the raw image and its reference image during training.
- 3.3 Training and Testing: During testing, repeated evaluation of the same degraded image produces enhancement variants whose samples support display, analysis, and consensus estimation.Only PAdaIN and the output block need to be re-evaluated for each prediction.
- 3.4 Training Data Generation: Because existing datasets usually provide one reference map, the method rebuilds UIEBD with additional labels generated by contrast, saturation, and gamma adjustments.The generated labels are intended to introduce ambiguity without significantly altering the original reference label.
4 Experiments
Experiments evaluate PUIE-Net’s enhancement distributions, consensus outputs, and stability on UIEBD and RUIE. The method produces diverse predictions, while consensus estimation provides robust deterministic results and competitive visual quality.
- Experiment Settings: Experiments use UIEBD for training and testing, while RUIE’s underwater image quality subset is used for additional testing.UIEBD contains 700 training originals and corresponding references, with the remainder reserved for testing; RUIE contributes 3,630 quality-subset images.
- Analysis of Enhancement Distribution: PUIE-Net samples exhibit different but reasonable appearances, ranging from mild enhancement at higher probabilities to stronger contrast and color adjustment at lower probabilities.The sampling interval is manually controlled for visualization in Fig. 3.
- Analysis of Enhancement Distribution: The sampled predictions provide alternatives for analysis and supply data for consensus processing to obtain robust and stable results.This distribution-based approach is intended to reduce the influence of biased labels in existing UIE datasets.
- Performance Comparison: PUIE-Net achieves favorable quantitative performance on UIEBD and RUIE, while MC performs significantly better than the other consensus choices.The authors attribute MC’s advantage to averaging samples, which can reduce enhancement bias during testing.
- Performance Comparison: Qualitatively, PUIE-Net produces cleaner, more natural results with fine-grained textures than methods showing color defects, artifacts, or over- and under-enhancement.The comparison includes Retinex, GC, Fusion, IBLA, Histogram-Prior, Water-Net, Ucolor, PUIE-Net (MC), and PUIE-Net (MP).
- Impact of Sampling Times: Increasing sampling times reduces output variation before stability is reached after 20 samples, motivating 20 as the default MC sampling count.The analysis measures mean standard deviations of PSNR and SSIM across repeated runs on UIEBD and balances stability against running time.
- Consensus Process: MC averages a group of samples, whereas MP selects the sample with the highest sampling probability as the final result.The paper notes that other selection strategies, such as quality-based selection, could also be used.
5 Conclusion
PUIE-Net learns an enhancement distribution rather than a single certain result, then integrates multiple predictions through consensus estimation. On two real-world UIE datasets, it produces reasonable enhancement sets and highly competitive consensus quality.
- Conclusion: PUIE-Net learns enhanced-image distributions to address ambiguous labels in underwater image enhancement.The method replaces direct single-result estimation with distribution learning.
- Conclusion: The method samples random style attributes from CVAE posterior distributions to transform global feature enhancement statistics.Consensus estimation integrates multiple predictions into the final enhancement result.
- Conclusion: On two real-world UIE datasets, PUIE-Net produces reasonable result sets and highly competitive visual quality from consensus estimation.The paper proposes future extension to low-light enhancement, dehazing, and denoising.