Source-linked AI summary
All-In-One Underwater Image Enhancement using Domain-Adversarial Learning
Pritish Uplavikar, Zhenyu Wu, Zhangyang Wang
TL;DR
Underwater enhancement is challenged by physical degradation, diverse water types, and limited real-world paired data. The paper uses adversarial learning to disentangle water-type nuisance factors from content features, then reconstructs enhanced images; it reports stronger enhancement across most Jerlov types, real-world generalization, and improved object detection in the synthesized setting.
Problem
Underwater images are degraded by attenuation and scattering, while diverse water-type distributions and scarce real-world paired data complicate training a universal enhancement model.
Method
An encoder-decoder reconstructs clear images from domain-agnostic features learned by adversarially disentangling Jerlov water type as a nuisance domain.
Results
The model outperforms previous methods in SSIM and PSNR for almost all Jerlov water types, generalizes to real-world data, and improves object detection on synthesized images.
Takeaways & Limitations
A single adversarially trained model can address water-type diversity while supporting enhancement and downstream object detection within the reported evaluation settings.
Takeaways & Limitations
Object detection improves on synthesized images but produces mixed results on the real-world UIEBD dataset.
Abstract
from arXiv · showhide
Raw underwater images are degraded due to wavelength dependent light attenuation and scattering, limiting their applicability in vision systems. Another factor that makes enhancing underwater images particularly challenging is the diversity of the water types in which they are captured. For example, images captured in deep oceanic waters have a different distribution from those captured in shallow coastal waters. Such diversity makes it hard to train a single model to enhance underwater images. In this work, we propose a novel model which nicely handles the diversity of water during the enhancement, by adversarially learning the content features of the images by disentangling the unwanted nuisances corresponding to water types (viewed as different domains). We use the learned domain agnostic features to generate enhanced underwater images. We train our model on a dataset consisting images of 10 Jerlov water types. Experimental results show that the proposed model not only outperforms the previous methods in SSIM and PSNR scores for almost all Jerlov water types but also generalizes well on real-world datasets. The performance of a high-level vision task (object detection) also shows improvement using enhanced images with our model.
1. Introduction
Underwater enhancement is difficult because images suffer wavelength-dependent attenuation, scattering, diverse water-type distributions, and scarce real-world ground truth. The proposed model addresses these challenges by learning water-type-agnostic features and improves enhancement across synthetic and real-world settings.
- 1. Introduction: Wavelength-dependent attenuation and particle scattering degrade underwater imagery by producing color shifts and homogeneous background noise.Red light is absorbed faster than blue or green light, while scattered light from particles introduces background noise.
- 1. Introduction: A single enhancement model must handle substantially different image distributions from coastal, deep oceanic, and muddy waters.These distribution differences make universal underwater enhancement difficult, while prior approaches require water-type knowledge or separate models.
- 1. Introduction: Real-world enhancement datasets lack paired degraded and clear images, motivating synthetic data generation from clear images and depth across 10 Jerlov water types.The synthetic construction uses the NYU Depth Dataset V2 and an underwater image formation model.
- 1. Introduction: The model combines an encoder-decoder for clear-image reconstruction with a nuisance classifier and adversarial training to learn domain-agnostic features.Jerlov water type is treated as the nuisance domain while synthetic images model attenuation and scattering.
- 1. Introduction: The proposed approach outperforms prior methods in SSIM and PSNR for almost all Jerlov water types and generalizes to real-world data while improving object detection.These are reported highlights of the method’s evaluation across enhancement and downstream vision performance.
2. Related Work
Prior underwater enhancement work includes physics-based image-formation models and deep networks, but existing approaches do not provide one model for diverse water-type distributions. The paper positions its method against these modeling and specialization strategies.
- 2. Related Work: Physics-based methods model underwater formation through refraction, attenuation, scattering, and volume scattering properties.These approaches explicitly represent characteristics of the underwater medium.
- 2. Related Work: The underwater formation model represents observed intensity as attenuated clear-image radiance plus homogeneous background light.Transmission depends on wavelength-dependent attenuation and scene depth.
- 2. Related Work: Prior deep models either learn underwater-to-clear mappings or train separate CNNs for different water types, rather than providing a singular generalized solution.The paper’s related-work discussion identifies separate water-type models as unable to address the diversity with one model.
3. Method
The model learns water-type-agnostic scene features with an encoder, decoder, and nuisance classifier trained through reconstruction, classification, and adversarial losses.
- 3. Method: The encoder seeks water-type-agnostic latent features so the decoder can reconstruct clear scenes from scene-specific information.For the same scene across water types, the latent representation should ideally remain unchanged.
- 3. Method: A nuisance classifier predicts water type from the encoder representation, while adversarial training makes that representation uninformative about water type.The adversarial loss is applied to update the encoder, whereas the nuisance loss updates only the classifier.
- 3. Method: The model combines reconstruction, nuisance, and adversarial losses to generate clear images while discarding water-type features.Reconstruction uses mean squared error between the generated image and clear-image ground truth.
- 3. Method: Training first optimizes the encoder and decoder, then introduces adversarial training while maintaining a sufficiently strong nuisance classifier.The staged procedure produces meaningful encoder features before adversarial optimization begins.
4. Experiments
Experiments use a U-Net encoder-decoder and convolutional nuisance classifier, with staged training controlled by validation thresholds.
- 4. Experiments: The model is trained on synthetic underwater images using a machine with an Intel i7 processor, 8 GB RAM, and an NVIDIA GTX 960M GPU.The reported hardware includes 4 GB of GPU memory.
- 4. Experiments: The encoder-decoder uses U-Net skip connections to combine local and global information when generating clear images.U-Net also supports images of varying sizes through its fully convolutional design.
- 4. Experiments: Training updates the encoder, decoder, and nuisance classifier conditionally according to validation SSIM and classifier-accuracy thresholds.The procedure uses thresholds of 0.9 for the generator validation SSIM and 0.85 for the nuisance classifier validation accuracy.
- 4. Experiments: The nuisance classifier predicts six water-type classes from encoder features.Its architecture is presented in Figure 3.
4.2. Datasets
The experiments use synthetic multi-water-type images generated from NYU-V2 and evaluate real-world generalization on UIEBD.
- 4.2. Datasets: Synthetic underwater images are generated from NYU-V2 RGB-D clear images and depth using an underwater image formation model.The dataset construction follows a prior method for generating images across multiple water types.
- 4.2. Datasets: The training dataset represents six Jerlov water types by combining similar categories from the original ten types.This grouping is intended to make water types easier for the nuisance classifier to distinguish.
- 4.2. Datasets: The synthetic images use different attenuation values together with randomized background-light and depth-related values.The attenuation values are taken from prior work, while background-light and depth values are randomized.
- 4.2. Datasets: Real-world evaluation uses the Underwater Image Enhancement Benchmark Dataset, which contains 890 underwater images.The dataset is used to assess the model beyond the synthesized training data.
4.3. Results on the synthetic dataset
On the synthetic test set, the model visually reconstructs clear colors and details and outperforms other methods on SSIM and PSNR for almost all water types.
- 4.3. Results on the synthetic dataset: Qualitative results show recovery of the original input colors and minute details from degraded underwater images.These visual results are shown on the synthetic test set in Figure 5.
- 4.3. Results on the synthetic dataset: The model outperforms other methods in SSIM and PSNR for almost all evaluated Jerlov water types.The metrics compare generated images with their clear counterparts, as reported in Table 1.
4.4. Results on the real-world dataset
On the Underwater Image Enhancement Benchmark Dataset, the model generalizes to image distributions different from its training images.
- The model performs well and generalizes to image distributions different from those of the training images.Handling distribution diversity is identified as a main goal alongside generating clear underwater images.
4.5. Comparison to no adversarial loss
Adversarial training improves enhancement over vanilla U-Net by learning features organized around image content rather than water type.
- Table 1 compares UIE-DAL with previous methods using SSIM and PSNR, where higher values indicate better results.The previous methods’ values are reprinted from [3], and bold values identify the best performer.
- Adversarial loss produces encodings clustered by image content, whereas vanilla U-Net encodings cluster by water type.The comparison uses PCA visualizations of the encoding Z, with points colored by water type and image content.
- U-Net with adversarial loss outperforms vanilla U-Net in visual and quantitative comparisons.The reported advantage includes richer color quality in generated images.
4.6. Object detection on enhanced images
Enhancement improves object detection on synthesized underwater images, while the real-world dataset produces mixed detection results.
- The study uses high-level vision performance, such as object detection, as an indicator of image-enhancement performance.The experiments apply YOLO v3 to degraded and enhanced underwater images.
- Figure 5 compares input underwater images, model outputs, and ground-truth clear images in the synthesized dataset.The columns are ordered as input, enhanced output, and ground truth.
- Figure 6 compares input underwater images with the model’s results on a real-world dataset.The figure places inputs in the left column and model results in the right column.
- Object detection is better on the model’s enhanced synthesized images than on the degraded underwater images.YOLO v3 is run on degraded images and their enhanced versions; Figure 8 shows before-and-after examples for synthetic and real-world images.
5. Conclusion
The paper presents domain-agnostic underwater enhancement that outperforms prior methods, generalizes to unseen real-world data, and improves object detection in its reported experiments.
- The model learns domain-agnostic features across underwater image types and generates clear versions from those features.The conclusion frames this as the basis for handling diverse underwater images.
- The model outperforms previous underwater enhancement methods qualitatively and quantitatively.The paper also reports successful generalization on unseen real-world data.
- PCA visualizations compare encoding Z by water type and by image content to assess domain-agnostic representation learning.Figure 9 visualizes the first two PCA components for a U-Net without adversarial loss, with colors indicating water type or content.
- Enhancing underwater images with the model before high-level vision tasks improves object detection performance.This improvement is reported as part of the conclusion’s experimental findings.