Source-linked AI summary
Underwater Ranker: Learn Which Is Better and How to Be Better
Chunle Guo, Ruiqi Wu, Xin Jin, Linghao Han, Zhi Chai, Weidong Zhang, Chongyi Li
TL;DR
Underwater image quality assessment needs to handle scene- and degradation-dependent variation without relying on imperfect reference images. The paper proposes URanker, a ranking-based UIQA model with underwater-specific multi-scale designs, and trains it using URankerSet; the model also supervises UIE networks, with a normalization tail improving their performance. The authors report state-of-the-art performance for both UIQA and UIE.
Problem
Existing underwater IQA methods face unsuitable generic metrics, difficult reference acquisition, and non-ideal reference images that can yield inaccurate assessment.
Method
URanker combines a conv-attentional Transformer with histogram priors, dynamic cross-scale correspondence, multi-scale class tokens, margin ranking loss, and the URankerSet ranking dataset.
Results
The authors report state-of-the-art performance for URanker in UIQA and for UIE networks using URanker supervision and the normalization tail.
Takeaways & Limitations
URanker can rank same-scene enhancement results by visual quality and provide differentiable supervision that improves UIE networks.
Takeaways & Limitations
The normalization tail only works for overflow values.
Abstract
from arXiv · showhide
In this paper, we present a ranking-based underwater image quality assessment (UIQA) method, abbreviated as URanker. The URanker is built on the efficient conv-attentional image Transformer. In terms of underwater images, we specially devise (1) the histogram prior that embeds the color distribution of an underwater image as histogram token to attend global degradation and (2) the dynamic cross-scale correspondence to model local degradation. The final prediction depends on the class tokens from different scales, which comprehensively considers multi-scale dependencies. With the margin ranking loss, our URanker can accurately rank the order of underwater images of the same scene enhanced by different underwater image enhancement (UIE) algorithms according to their visual quality. To achieve that, we also contribute a dataset, URankerSet, containing sufficient results enhanced by different UIE algorithms and the corresponding perceptual rankings, to train our URanker. Apart from the good performance of URanker, we found that a simple U-shape UIE network can obtain promising performance when it is coupled with our pre-trained URanker as additional supervision. In addition, we also propose a normalization tail that can significantly improve the performance of UIE networks. Extensive experiments demonstrate the state-of-the-art performance of our method. The key designs of our method are discussed. We will release our dataset and code.
Introduction
The paper addresses limitations in underwater image quality assessment by proposing URanker, a ranking-based method tailored to underwater degradations. It also introduces URankerSet and shows that URanker can supervise UIE networks.
- Underwater images suffer from color cast, low contrast, and blurred details caused by wavelength- and distance-dependent light attenuation and scattering.
- Existing full-reference assessment depends on underwater reference images that are difficult to obtain in situ and may be non-ideal, causing inaccurate assessment.
- URanker ranks same-scene underwater images by visual quality instead of fitting absolute scores, reducing interference from differing degradations and scenes.
- The method uses a conv-attentional Transformer with histogram priors for global degradation and dynamic cross-scale correspondence for local degradation.
- URankerSet contains enhanced underwater images with corresponding perceptual rankings, enabling margin-ranking-loss training of URanker.
- URanker also provides differentiable feedback as UIE supervision, while a normalization tail further improves UIE performance.
Related Work
Related underwater image quality methods include handcrafted, statistical, frequency-domain, and learned approaches. The paper positions URanker-enhanced UIE as a data-driven alternative that can achieve state-of-the-art performance with a simple U-shape network.
- Common IQA metrics are unsuitable for underwater images because of their unique image characteristics.
- UCIQE combines chroma, brightness, and saturation, while UIQM combines colorfulness, sharpness, and contrast to assess underwater image quality.
- UIE methods span non-physical model-based, physical model-based, and data-driven approaches.
- A simple U-shape UIE network achieves state-of-the-art performance when equipped with pre-trained URanker supervision and the proposed normalization tail.
Proposed URankerSet
URankerSet is constructed by processing raw underwater images with multiple enhancement algorithms and collecting human perceptual rankings. Pairwise volunteer comparisons and Bubble Sort produce ordered image groups for training.
- URankerSet processes 890 raw underwater images using results from multiple UIE algorithms, including fusion-, retinex-, physical-model-, regression-, and deep-learning-based methods.
- Eleven volunteers with image-processing experience compare each raw image and nine enhanced results pairwise under matched monitor settings.
- Bubble Sort orders each image group in descending visual-quality order, producing perceptual rankings for URanker training.
Proposed URanker
URanker combines multi-scale conv-attentional processing with histogram-based global degradation modeling and dynamic cross-scale feature integration for ranking underwater image quality.
- URanker uses an efficient conv-attentional image Transformer, replacing multi-head self-attention to reduce computational complexity.
- Serial blocks tokenize downsampled features and combine them with learnable class tokens and histogram tokens before alternating conv-attention and feed-forward layers.
- The Histogram Prior Module converts 64-bin color histograms from three image channels into a histogram token for global degradation modeling.
- Dynamic cross-scale correspondence adaptively integrates aligned features from multiple scales while avoiding underuse or overuse of other-scale information.
- Class-token scores from different scales are averaged to produce the final prediction, while margin-ranking loss trains scores to match visual-quality orderings.
Proposed UIE Network
The paper investigates using a pre-trained URanker as additional supervision for UIE networks and proposes NU2Net with a normalization tail. NU2Net combines Conv-IN-ELU residual blocks with tail-based handling of overflow values.
- Pre-trained URanker supervision: A pre-trained URanker can provide additional supervision when training UIE networks because it is differentiable.The URanker loss is combined with content loss during UIE training.
- Training objective: The total UIE training objective combines URanker loss with pixel-wise content loss using a trade-off coefficient λ.The content loss may be L1, L2, or perceptual loss, depending on the formulation.
- NU2Net architecture: NU2Net is a simple normalization-based U-shape UIE network built from a stack of Conv-IN-ELU blocks.The network uses a residual structure.
- Normalization tail: The normalization tail is appended to NU2Net to handle values outside the valid [0, 1] range.It is inspired by a stretching operation from a traditional UIE algorithm.
- Normalization tail: The normalization tail only operates on overflow values rather than all network outputs.Its broader effects are examined in ablation studies.
Experiments
Experiments evaluate URanker for underwater image quality ranking, its supervision for enhancement networks, and the proposed normalization tail and architectural components. Results show strong ranking consistency, improved enhancement performance, and benefits from the key designs.
- Underwater Image Quality Assessment Comparison: URanker predictions most closely match the ground-truth ranking and human perception in visual comparisons.Figure 7 marks incorrect rankings in red and orders prediction scores from left to right.
- Underwater Image Quality Assessment Comparison: URanker achieves state-of-the-art UIQA performance, improving over MUSIQ by 5.02% in SRCC and 6.84% in KRCC.The evaluation compares URanker with six underwater or generic UIQA methods using rank-wise metrics.
- Underwater Image Enhancement Comparison: Pre-trained URanker supervision significantly improves all three evaluated UIE networks, while NU2Net achieves state-of-the-art performance against WaterNet and Ucolor.The evaluated networks are NU2Net, WaterNet, and Ucolor.
- Effectiveness of HPM & DCPB: Removing HPM decreases SRCC by 0.80% and KRCC by 0.28%, while the full model ranks five enhanced results accurately.Dense multi-scale connections perform worse than neighbour connections, supporting the need for dynamic connection design.
- Normalization Tail Ablation: The normalization tail improves performance by at least 2.456dB in PSNR and 0.039 in SSIM without adding parameters.The full model also produces the perceptually best result among the normalization-tail variants.
Conclusion
The paper proposes URanker for underwater image quality assessment using task-specific designs and a ranking dataset. It also shows that URanker supervision and the normalization tail improve underwater image enhancement performance.
- Conclusion: URanker combines a histogram prior, dynamic connection parallel block, and an underwater image ranking dataset for UIQA.The paper presents these components as the main basis of its method.
- Conclusion: The proposed UIQA model improves UIE networks, while the normalization tail further boosts UIE performance.The conclusion reports that both the proposed UIQA and UIE models outperform state-of-the-art methods.