Source-linked AI summary
Emerging from Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer
Chongyi Li, Jichang Guo, Chunle Guo
TL;DR
Underwater images suffer color and visibility degradation from wavelength-dependent absorption and scattering, challenging both image quality and vision tasks. The paper learns weakly supervised, unpaired color transfer from underwater to air domains using adversarial, cycle-consistency, and SSIM losses. Across subjective and application assessments, the method produces visually pleasing corrections and improves saliency detection and keypoint matching results.
Problem
Underwater absorption and scattering degrade image quality and vision-task performance, while existing correction methods rely on restrictive assumptions or paired training data.
Method
A weakly supervised cross-domain model maps unpaired underwater images to air-domain colors using adversarial, cycle-consistency, and SSIM losses.
Results
The method produces visually pleasing results and achieves an average visual quality score of 6.3 for Ours across 30 selected underwater images.
Takeaways & Limitations
Color correction with the proposed method is accompanied by better saliency detection performance and more keypoint-matching points.
Abstract
from arXiv · showhide
Underwater vision suffers from severe effects due to selective attenuation and scattering when light propagates through water. Such degradation not only affects the quality of underwater images but limits the ability of vision tasks. Different from existing methods which either ignore the wavelength dependency of the attenuation or assume a specific spectral profile, we tackle color distortion problem of underwater image from a new view. In this letter, we propose a weakly supervised color transfer method to correct color distortion, which relaxes the need of paired underwater images for training and allows for the underwater images unknown where were taken. Inspired by Cycle-Consistent Adversarial Networks, we design a multi-term loss function including adversarial loss, cycle consistency loss, and SSIM (Structural Similarity Index Measure) loss, which allows the content and structure of the corrected result the same as the input, but the color as if the image was taken without the water. Experiments on underwater images captured under diverse scenes show that our method produces visually pleasing results, even outperforms the art-of-the-state methods. Besides, our method can improve the performance of vision tasks.
I. INTRODUCTION
Underwater images are degraded by wavelength-dependent absorption and scattering, while existing correction methods rely on restrictive assumptions or paired data. The paper proposes weakly supervised cross-domain color transfer that preserves content and structure while removing water-induced color casts.
- Motivation: Wavelength-dependent absorption and scattering cause low contrast, color casts, noise, weakened scene intensity, and washed-out appearances in underwater images.The degradation also depends on complex environmental and imaging factors.
- Motivation: Existing approaches include hardware, dehazing, restoration, and statistical color-constancy methods, but hardware solutions are unsuitable for dynamic acquisition.Many restoration methods also rely on outdoor dehazing strategies or assumed priors.
- Contribution: The proposed weakly supervised model maps underwater scenes to air-scene colors without explicit paired labels, preserving input content and structure.It relaxes the need for paired underwater images and permits images whose capture locations are unknown.
- Contribution: The method aims to remove color casts and improve vision-task performance, while acknowledging that exact appearance recovery is impossible when scenes and light conditions are unknown.Figure 1 presents two examples of the method’s results.
II. PROPOSED METHOD
The method learns unpaired mappings between underwater and air image domains using image-to-image translation networks. Adversarial, cycle-consistency, and SSIM losses jointly regulate target-domain appearance, mapping consistency, and source-image structure.
- Domain translation: The model learns forward and backward mappings between underwater source domain X and air target domain Y from unpaired image samples.The mappings are G: X → Y and F: Y → X.
- Adversarial framework: Two adversarial discriminators distinguish real images from translated images in the underwater and air domains.D_X compares x with F(y), while D_Y compares y with G(x).
- Multi-term objective: Each generator uses adversarial, cycle-consistency, and SSIM losses to match target distributions, constrain mappings, and preserve source content and structure.The framework is inspired by CycleGAN-style unpaired image translation.
A. Adversarial loss
The adversarial loss trains the forward mapping to produce images resembling the air-domain distribution while the discriminator separates translated and real air images.
- Adversarial loss: The adversarial loss matches generated-image distribution G(x) to target-domain distribution Y.This encourages translated underwater images to resemble air-domain images.
- Adversarial loss: The generator produces G(x) that resembles target-domain images, while D_Y distinguishes translated samples G(x) from real samples y.The equation defines the adversarial game between G and D_Y.
B. Cycle consistency loss
Cycle consistency constrains the two domain mappings by requiring translated images to reconstruct their original inputs. SSIM complements this by preserving local content and structure between input and translated images.
- B. Cycle consistency loss: Cycle consistency requires F(G(x)) to return an underwater image close to x and G(F(y)) to return an air image close to y.The loss uses reconstruction differences in both translation directions.
- B. Cycle consistency loss: The cycle loss constrains the space of possible mapping functions by enforcing consistency after forward and backward translation.Both source-domain and target-domain cycles are included.
- C. SSIM loss: SSIM loss compares 13×13 patches from input image x and translated image G(x) to preserve content and structure.The SSIM value is calculated for every pixel using patch statistics.
- C. SSIM loss: SSIM uses patch means, standard deviations, and covariance, with C1=0.02 and C2=0.03 as default constants.The center pixel p identifies the image patch location, and N denotes the number of image pixels.
D. Total loss
The total loss combines adversarial, cycle consistency, and SSIM losses through weighted linear combination.
- The total loss is a linear combination of three previously defined losses with weights λ1, λ2, and λ3.The three components are combined to form the overall training objective.
- The weights λ1, λ2, and λ3 are set to 1, 1, and 10 based on heuristic experiments.These values make the three components have similar orders of magnitude in the final loss.
- The model alternately optimizes the forward generator G, discriminator DY, backward generator F, and discriminator DX.Generator G minimizes the total objective, while discriminator DY maximizes the adversarial objective; the corresponding backward components are also optimized alternately.
E. Network architecture and training details
The model uses CycleGAN-style generators and PatchGAN discriminators, trained on unpaired underwater and air images with stabilization techniques and specified optimization settings.
- The forward and backward networks share an architecture based on CycleGAN, while the discriminators use 70 × 70 PatchGANs.The generators adapt an architecture from reference.
- Training uses a dataset containing 3800 underwater images and 3800 air images, resized to 256 × 256.Figure 3 presents example training samples from the two image domains.
- The adversarial objective replaces negative log likelihood with least square loss to stabilize training.This modification rewrites Equation (1).
- The discriminators use histories of generated images, and training uses ADAM with learning rate 0.0002, momentum 0.5, and batch size 1.Training was implemented in TensorFlow on an NVIDIA TITAN X GPU and took 15 hours.
III. EXPERIMENTS
The experiments compare the proposed model with image translation, color constancy, enhancement, and underwater restoration methods on images from varying underwater scenes.
- The proposed model is compared with CycleGAN, Gray World, INT, RED, UWID, and UWIB.These baselines represent image-to-image transfer, color constancy, image enhancement, and underwater restoration approaches.
- Test images were captured under varying underwater scenes to assess performance across diverse conditions.WaterGAN was excluded because it was available only for underwater images from designated sites.
- The study subjectively evaluates the visual quality of the compared results.
A. Subjective assessment
The proposed method removes underwater color casts while preserving visually improved results across scenes where competing assumptions may fail.
- CycleGAN produces blurred results because it tends to translate air-image content and structure into underwater images.
- Gray World introduces color casts when its assumption is unavailable, while INT mainly increases image brightness.INT relies on a prior learned from natural scenes, which is unavailable in these cases.
- Three underwater restoration methods remove haze and improve contrast, but their color restoration is incomplete.Their assumed optical parameters do not hold in some underwater scenes.
- The proposed method can remove greenish and bluish tones as if images were taken without water, yielding better visual quality.
B. User study
The user study evaluated visual quality on 30 underwater images using ratings from 10 experienced participants. Our method received the highest average score among the compared methods.
- Results: Figure 4 presents visual-quality comparisons across varying underwater scenes, with images I1-I5 ordered from top to bottom.Table I reports the average scores for the images shown in Figure 4.
C. Application assessment
Application tests examined saliency detection and keypoint matching before and after color correction. The reported results show better saliency detection performance and more matching points after applying the method.
- Application tests: The application assessment included saliency detection and keypoint matching tests on underwater images before and after correction.Figures 5 and 6 provide example results for these two tasks.
- Results: Color-corrected images achieved better saliency detection performance and more matching points than the raw underwater images.The paper presents these outcomes as additional evidence of the method's effectiveness.
- Conclusion: Quality evaluation, user study, and application assessment demonstrated the effectiveness of the proposed color-correction method.The conclusion groups the application results with the other reported evaluations.