Source-linked AI summary
Enhancing Underwater Imagery using Generative Adversarial Networks
Cameron Fabbri, Md Jahidul Islam, Junaed Sattar
TL;DR
Underwater visual data are degraded by nonlinear color and visibility distortions, while restoration lacks ground-truth image pairs. The paper uses GANs and CycleGAN to create paired data and restore imagery, reporting higher diver-tracking accuracy and a 350% increase in correct detections on generated images. The approach is limited by the need for larger, more diverse data and broader quantitative evaluation.
Problem
Underwater distortions lack ground truth and vary nonlinearly, limiting reliable training and performance for vision-driven underwater robots.
Method
GAN-based restoration uses CycleGAN to translate between undistorted and distorted underwater-image domains and generate paired training data.
Results
350% increase in correct detections was reported for the MDPM diver tracker on generated images compared with real underwater images.
Takeaways & Limitations
Corrected underwater images produced more visually appealing results and higher accuracy for a diver-tracking algorithm than uncorrected image sequences.
Takeaways & Limitations
Future work identifies the need for larger, more diverse datasets and additional quantitative performance metrics to improve generalizability and evaluation.
Abstract
from arXiv · showhide
Autonomous underwater vehicles (AUVs) rely on a variety of sensors - acoustic, inertial and visual - for intelligent decision making. Due to its non-intrusive, passive nature, and high information content, vision is an attractive sensing modality, particularly at shallower depths. However, factors such as light refraction and absorption, suspended particles in the water, and color distortion affect the quality of visual data, resulting in noisy and distorted images. AUVs that rely on visual sensing thus face difficult challenges, and consequently exhibit poor performance on vision-driven tasks. This paper proposes a method to improve the quality of visual underwater scenes using Generative Adversarial Networks (GANs), with the goal of improving input to vision-driven behaviors further down the autonomy pipeline. Furthermore, we show how recently proposed methods are able to generate a dataset for the purpose of such underwater image restoration. For any visually-guided underwater robots, this improvement can result in increased safety and reliability through robust visual perception. To that effect, we present quantitative and qualitative data which demonstrates that images corrected through the proposed approach generate more visually appealing images, and also provide increased accuracy for a diver tracking algorithm.
1 Introduction
Visual sensing is valuable for underwater robots but is degraded by variable, nonlinear distortions that impair vision-driven tasks. The paper proposes GAN-based restoration and uses CycleGAN to generate training pairs without ground-truth underwater images.
- Visual sensing supports underwater monitoring, exploration, and seabed mapping because it is non-intrusive, passive, and energy efficient.
- Light refraction, absorption, and scattering create depth- and condition-dependent color and visibility distortions that hinder segmentation, tracking, and classification.
- Underwater visual algorithms may generalize poorly across depth ranges because diverse training data are costly to acquire and highly noisy.
- The proposed GAN-based technique improves underwater visual scenes, while CycleGAN generates paired training data without requiring image pairs or ground truth.
- Figure 1 illustrates that distortion and color loss vary across underwater images with different camera-to-object distances and natural or man-made artifacts.
2 Related Work
Related work includes colorization, physics-based restoration, and adversarial image generation. The paper positions its approach as using GANs for both generating underwater distortions and correcting color, without requiring depth information during training.
- Existing automatic colorization methods mainly convert grayscale images to color, while other approaches model underwater light physics or use energy minimization.
- WaterGAN generates realistic underwater images through attenuation, scattering, and vignetting stages, but requires depth information during training.
- The proposed approach uses GANs as the generative model for a paired image-to-image translation problem because Euclidean loss can produce blurry results.
3 Methodology
The method constructs paired distorted and undistorted underwater images with CycleGAN, then trains GAN-based restorers to reconstruct color while preserving image structure. UGAN combines adversarial and L1 objectives, while UGAN-P additionally uses gradient difference loss to sharpen predictions.
- Paired-data generation: CycleGAN transfers undistorted images into the distorted underwater domain, producing paired data for training image reconstruction.The generated pairs address the lack of matching distorted and undistorted underwater images.
- Adversarial restoration: UGAN learns the mapping f: ID → IC with a generator conditioned on distorted images and an adversarial discriminator.The discriminator distinguishes dataset images from generated outputs, while the generator attempts to produce realistic restored images.
- Adversarial restoration: The UGAN objective combines the WGAN loss with an L1 reconstruction loss to capture ground-truth appearance and low-level image frequencies.The L1 term provides the generator with ground-truth guidance in addition to adversarial feedback.
- Gradient refinement: UGAN-P adds Gradient Difference Loss to penalize discrepancies between predicted and ground-truth image gradients, targeting blurry generative outputs.The loss compares local image-gradient differences using the predicted image IP = G(ID).
- Network architecture: The generator is a fully convolutional U-Net encoder-decoder whose skip connections preserve spatial dependencies between input and output.The encoder downsamples features and the decoder reconstructs the image through transpose convolutions.
- Network architecture: The discriminator is a PatchGAN that evaluates image patches and outputs a 32 × 32 × 1 feature matrix rather than a single real-or-fake score.Batch normalization is omitted because WGAN-GP penalizes the discriminator gradient with respect to each input individually.
4 Experiments
The experiments construct paired training data with CycleGAN, compare UGAN variants against CycleGAN using image-space, sharpness, and patch statistics, and test restored images on diver tracking. UGAN variants reduce noise and improve tracking detection, while patch statistics show only slight quantitative improvement.
- 4.3 Comparison to CycleGAN: UGAN and UGAN-P produce less noisy Canny edges and remain closer in image space to the original images than CycleGAN.Figure 4 presents four image sets, with inputs above and edge-detector outputs below; Table 1 reports the distances.
- 4.3 Comparison to CycleGAN: Lower Gradient Difference Loss scores for UGAN and UGAN-P indicate smoother pixel transitions and less noise in local patches.UGAN consistently scores lower than UGAN-P despite UGAN-P explicitly including this metric in its objective.
- 4.3 Comparison to CycleGAN: Mean and standard-deviation measurements show only slight quantitative improvement despite qualitative evidence that the proposed methods are sharper.The authors leave entropy and other quantitative metrics for future work.
- 4.4 Diver Tracking using Frequency-Domain Detection: Generated images increase correct detections by 350% for the MDPM diver tracker compared with real underwater images.The authors attribute the improvement to restoration of eroded foreground-background intensity variations.
- Implementation: GPU inference takes 0.0138s, or about 72 FPS, while CPU inference takes 0.1244s, or about 8 FPS, for 256 × 256 × 3 inputs.The authors find both speeds acceptable for underwater tasks.
5 Conclusion
The paper presents GAN-based underwater color enhancement using CycleGAN-generated paired training data. Results include qualitative and quantitative effectiveness and higher diver-tracking accuracy, while future work targets broader data and evaluation.
- 5 Conclusion: The proposed approach uses GANs to enhance underwater color images and CycleGAN to generate paired training data for restoration.The paired-data strategy addresses the absence of ground-truth underwater images.
- 5 Conclusion: Corrected scuba-diver images yield higher diver-tracking accuracy than the uncorrected image sequence.The conclusion reports effectiveness through both quantitative and qualitative results.
- 5 Conclusion: Future work will create a larger, more diverse underwater-object dataset to improve network generalizability and investigate additional quantitative metrics.The authors also propose adding particle and lighting noise to CycleGAN-generated data.