Source-linked AI summary
Fast Underwater Image Enhancement for Improved Visual Perception
Md Jahidul Islam, Youya Xia, Junaed Sattar
TL;DR
Underwater visual sensing is degraded by optical distortions, creating a need for effective real-time enhancement. The paper introduces FUnIE-GAN and EUVP for paired and unpaired training, and reports improved image quality and downstream perception performance with fast inference. Its scope is bounded by failure cases on severely degraded, texture-less images and instability in FUnIE-GAN-UP.
Problem
Underwater optical distortions impair visual perception, while real-time methods face limited data and computational constraints.
Method
FUnIE-GAN is a conditional GAN using a multimodal perceptual objective and the EUVP dataset of paired and unpaired underwater images.
Results
Enhanced images improve underwater object detection, saliency prediction, and human pose estimation, while the model offers faster inference than state-of-the-art models.
Takeaways & Limitations
The results support using the approach for real-time preprocessing in visually guided underwater-robot autonomy pipelines.
Takeaways & Limitations
FUnIE-GAN is ineffective on severely degraded, texture-less images, and FUnIE-GAN-UP is prone to training instability.
Abstract
from arXiv · showhide
In this paper, we present a conditional generative adversarial network-based model for real-time underwater image enhancement. To supervise the adversarial training, we formulate an objective function that evaluates the perceptual image quality based on its global content, color, local texture, and style information. We also present EUVP, a large-scale dataset of a paired and unpaired collection of underwater images (of `poor' and `good' quality) that are captured using seven different cameras over various visibility conditions during oceanic explorations and human-robot collaborative experiments. In addition, we perform several qualitative and quantitative evaluations which suggest that the proposed model can learn to enhance underwater image quality from both paired and unpaired training. More importantly, the enhanced images provide improved performances of standard models for underwater object detection, human pose estimation, and saliency prediction. These results validate that it is suitable for real-time preprocessing in the autonomy pipeline by visually-guided underwater robots. The model and associated training pipelines are available at https://github.com/xahidbuffon/funie-gan.
1. Introduction
Underwater robots face visibility-induced image distortions that impair visual tasks, motivating FUnIE-GAN, a fast conditional GAN trained with paired or unpaired data to enhance perceptual quality for real-time perception.
- Poor visibility, refraction, absorption, and scattering create nonlinear image distortions that impair underwater tracking, detection, and classification.
- Real-time enhancement can restore perceptual and statistical qualities of distorted images, while physics-based models may require unavailable scene information and excessive computation.
- Existing learning methods face underwater-specific challenges, including ill-posed enhancement and limited datasets that often rely on small-scale synthetic images.
- FUnIE-GAN learns a nonlinear mapping from distorted to enhanced images through adversarial training on the EUVP dataset.
- The fully convolutional conditional GAN uses a multimodal objective covering global content, color, local texture, and style information.
- EUVP provides 20K paired and unpaired underwater images for one-way and two-way adversarial training.
- Enhanced images significantly improve underwater object detection, human pose estimation, and saliency prediction, while feasibility analyses support real-time robotic perception.
2. Related Work
Automatic image enhancement has progressed from hand-crafted and physics-based methods to deep CNN and GAN approaches, but underwater applications remain constrained by data and real-time deployment challenges.
- Automatic Image Enhancement: Classical enhancement methods use hand-crafted filters and scene priors to improve color constancy, contrast, lightness, deblurring, and dehazing.
- Automatic Image Enhancement: Deep CNNs learn nonlinear filters from paired data and provide state-of-the-art performance for colorization, color or contrast adjustment, and dehazing.
- GAN-based Enhancement: GANs formulate image generation as a two-player min-max game, while conditional GANs constrain outputs to a target pattern for pixel-to-pixel translation.
- Unpaired Learning: Two-way GANs use cycle-consistency loss to learn mutual mappings between domains from unpaired data.
- Underwater Enhancement: Physics-based underwater methods model light propagation, while filtering methods reduce noise and improve global contrast.
- Underwater Enhancement: Existing underwater models often train on synthetic distortions, lack large-scale natural unpaired training, or fail to provide fast inference on single-board robotic platforms.
3. Proposed Model and Dataset
FUnIE-GAN is a conditional GAN for fast underwater image enhancement, using a compact encoder-decoder generator and patch-level discrimination. Its objective combines global, content, texture, and style cues, while EUVP supports paired and unpaired training across varied underwater data.
- FUnIE-GAN Architecture: FUnIE-GAN learns a mapping from distorted underwater images to enhanced images with a conditional GAN.The generator evolves with an adversarial discriminator through an iterative min-max game.
- FUnIE-GAN Architecture: The generator uses a U-Net-style encoder-decoder with skip-connections and produces 256 × 256 × 3 enhanced images from 256 × 256 × 3 inputs.Its fully convolutional design uses fewer parameters to achieve fast inference.
- FUnIE-GAN Architecture: The PatchGAN discriminator evaluates patch-level information to capture local texture and style with fewer parameters than global image-level discrimination.The discriminator transforms a 256 × 256 × 6 real-and-generated image input into a 16 × 16 × 1 output.
- Objective Function Formulation: The paired-training objective combines adversarial learning with global similarity, image content, and local texture and style information.The content term uses high-level features from block5 conv2 of a pre-trained VGG-19 network, while PatchGAN enforces local consistency.
- Unpaired Training: Unpaired training omits global similarity and content constraints and instead learns forward and reconstruction mappings through cycle-consistency.The cycle-consistency loss provides analogous reconstruction in L1 space, with λcyc = 0.1.
- EUVP Dataset: EUVP contains over 12K paired and 8K unpaired underwater images collected with seven cameras across oceanic explorations and human-robot experiments.Unpaired images were separated by six human participants based on color, contrast, sharpness, and scene interpretability.
4. Experimental Results
FUnIE-GAN improves underwater image attributes and performs competitively across qualitative, quantitative, user-preference, and downstream perception evaluations. Its main limitations arise in severely degraded, texture-less images and in the efficiency–robustness trade-off.
- Qualitative Evaluations: FUnIE-GAN recovers color and sharpness, rectifies greenish hues, and enhances global contrast in qualitative evaluations.The L1 loss produces sharper images, while the content loss contributes finer texture details.
- Qualitative Evaluations: FUnIE-GAN performs as well as or better than compared learning-based and physics-based methods while using a simpler architecture and no scene-depth or waterbody priors.FUnIE-GAN-UP outperforms CycleGAN in general, although unpaired learning makes color consistency and hue rectification more challenging.
- Quantitative Evaluation: FUnIE-GAN performs best on average PSNR and SSIM over 1K paired test images.The quantitative comparison uses PSNR and SSIM against ground-truth images.
- Quantitative Evaluation: Removing both L1 and content loss terms decreases average UIQM by 17.6%, while L1 and content losses contribute 4.58% and 1.07% UIQM improvements, respectively.The ablation also reports similar statistics for PSNR and SSIM.
- User Study: Users preferred enhanced images over original images, with FUnIE-GAN among the preferred models in a study of 78 participants and 312 responses.The original images achieved an average rank-3 accuracy of 6.67.
- Improved Visual Perception: FUnIE-GAN-enhanced images improve object detection by 11–14%, human pose estimation by 22–28%, and saliency prediction by 26–28% on average.Several alternative models also improve perception, but their inference rates are significantly slower and generally unsuitable for real-time robotic deployment.
- Limitations and Failure Cases: FUnIE-GAN struggles with severely degraded, texture-less images, where noise amplification can cause over-saturation and poor color and texture recovery.FUnIE-GAN balances robustness and efficiency, limiting performance to a certain degree.
5. Conclusion
The paper presents an efficient conditional GAN for underwater image enhancement, supported by a perceptual loss and paired/unpaired training data. Evaluations show competitive enhancement with faster inference and improved downstream perception tasks, while future work targets unpaired-training stability and broader applications.
- The proposed conditional GAN performs underwater image enhancement while balancing robustness and efficiency for faster inference.The paper frames the model as a simple, efficient approach and notes that its robustness-efficiency trade-off limits performance to some degree.
- Its perceptual loss evaluates global color, content, local texture, and style information.
- The presented dataset contains paired and unpaired underwater images for supervised training.
- Qualitative, quantitative, and user-study evaluations show performance comparable to or often better than state-of-the-art models.
- Enhanced images improve underwater object detection, saliency prediction, and human body-pose estimation performance.
- Future work will address broader underwater human-robot applications, marine trash identification, and color consistency and stability during unpaired training.