Source-linked AI summary
Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restoration
Chen Zhao, Weiling Cai, Chenyu Dong, Chengwei Hu
TL;DR
Underwater image enhancement remains difficult because degradation is diverse and prior methods insufficiently exploit frequency information. WF-Diff combines wavelet/Fourier frequency enhancement with diffusion-based adjustment through WFI2-net and FRDAM. The framework reports state-of-the-art performance on UIE tasks and strong visual results on real underwater images, while inference speed remains a limitation.
Problem
Most underwater enhancement methods operate mainly in raw pixel space, limiting exploration of frequency characteristics and fine-grained information.
Method
WF-Diff uses WFI2-net for preliminary wavelet-space frequency enhancement and FRDAM for diffusion-based refinement of high- and low-frequency information.
Results
WF-Diff achieves state-of-the-art performance on UIE tasks and consistently produces natural, improved visual results on real underwater images.
Takeaways & Limitations
Frequency-domain processing and diffusion adjustment together provide a detachable framework for enhancing underwater image color, detail, and frequency information.
Takeaways & Limitations
Using two diffusion models does not provide an inference-speed advantage, motivating future work on faster sampling.
Abstract
from arXiv · showhide
Underwater images are subject to intricate and diverse degradation, inevitably affecting the effectiveness of underwater visual tasks. However, most approaches primarily operate in the raw pixel space of images, which limits the exploration of the frequency characteristics of underwater images, leading to an inadequate utilization of deep models' representational capabilities in producing high-quality images. In this paper, we introduce a novel Underwater Image Enhancement (UIE) framework, named WF-Diff, designed to fully leverage the characteristics of frequency domain information and diffusion models. WF-Diff consists of two detachable networks: Wavelet-based Fourier information interaction network (WFI2-net) and Frequency Residual Diffusion Adjustment Module (FRDAM). With our full exploration of the frequency domain information, WFI2-net aims to achieve preliminary enhancement of frequency information in the wavelet space. Our proposed FRDAM can further refine the high- and low-frequency information of the initial enhanced images, which can be viewed as a plug-and-play universal module to adjust the detail of the underwater images. With the above techniques, our algorithm can show SOTA performance on real-world underwater image datasets, and achieves competitive performance in visual quality.
1. Introduction
Underwater image enhancement is challenging because underwater degradation distorts color, contrast, and detail, while prior methods insufficiently explore frequency information. WF-Diff addresses this with wavelet/Fourier analysis and diffusion-based frequency adjustment.
- Motivation: Underwater refraction, absorption, and scattering produce severe distortion, including low contrast and blurriness, affecting underwater vision applications.
- Limitations of prior methods: Physics-based methods have limited representation capacity for complex underwater scenes, while learning-based methods largely operate in raw pixel space.
- Frequency-domain motivation: Amplitude swapping indicates that degradation is concentrated mainly in Fourier amplitude, with color degradation in low-frequency wavelet sub-images and texture degradation in high-frequency sub-images.
- Proposed framework: WF-Diff uses two stages: preliminary frequency enhancement in wavelet space and subsequent frequency diffusion adjustment.
- Proposed framework: WFI2-net enhances frequency information, while FRDAM refines high- and low-frequency information and functions as a plug-and-play detail-adjustment module.
- Proposed framework: The cross-frequency conditioner enables interaction between high- and low-frequency information, and experiments report superiority over previous UIE approaches with effective contributions confirmed by ablations.
2. Related Works
Related underwater image enhancement methods include physical, deep-learning, GAN-based, and diffusion-based approaches. Diffusion methods offer strong restoration capabilities but must address artifacts and limited focus on fine-grained frequency information.
- Physical and deep models: Physical-model UIE methods use priors or revised imaging models to compensate for color cast, scattering, attenuation, and related degradation.
- Physical and deep models: Deep-learning approaches include complex restoration networks and models combining physical imaging knowledge, multi-scale restoration, or semi-supervised learning.
- GAN-based methods: GAN-based frameworks such as WaterGAN, UGAN, and UIE-DAL reduce dependence on real-world underwater paired training data.
- Diffusion models: Diffusion models have been applied to conditional image generation and underwater enhancement, including colorization, in-painting, and JPEG restoration.
- Diffusion models: Standard diffusion enhancement can produce artifacts from random-noise sampling and must recover both high- and low-frequency information, limiting fine-grained detail focus.
3. Methodology
WF-Diff enhances underwater images by decomposing them into wavelet frequency sub-bands, refining low- and high-frequency information with Fourier-aware networks, and applying diffusion-based residual adjustment.
- Discrete Wavelet and Fourier Transform: WF-Diff uses DWT to decompose input images into one low-frequency and three high-frequency sub-bands for separate color and detail restoration.The low-frequency sub-band carries content and color information, while vertical, horizontal, and diagonal high-frequency sub-bands carry structural and texture details.
- Discrete Wavelet and Fourier Transform: Fourier analysis separates complex frequency components into amplitude and phase, with underwater color degradation concentrated mainly in low-frequency amplitude.The framework therefore restores low-frequency amplitude while targeting high-frequency information for texture and detail enhancement.
- Frequency Preliminary Enhancement: The WFI2-net uses parallel low- and high-frequency encoder-decoder branches, combining transformer-based high-frequency modeling with spatial-frequency fusion for low-frequency restoration.Its Wide Transformer Block models long-range dependencies, while the Spatial-Frequency Fusion Block interacts with spatial and Fourier-domain representations.
- Cross-Frequency Conditioner: The Cross-Frequency Conditioner enables interaction between high- and low-frequency embeddings by aggregating high-frequency coefficients and producing updated output features.It uses separate linear projections and value representations for the two frequency groups.
- Frequency Diffusion Adjustment: FRDAM refines the initial enhanced frequency components through low- and high-frequency diffusion branches that learn residual distributions using forward and reverse diffusion.The final frequency components add the generated residual distributions to the initial enhanced components before inverse DWT reconstructs the image.
4. Experiments
Experiments compare WF-Diff with established UIE methods across reference and non-reference datasets, using quantitative metrics, visual comparisons, and ablations. The reported results show strong image-quality performance and support the effectiveness of WFI2-net and FRDAM.
- Results and Comparisons: Experiments compare UIE methods on UIEBD, LSUI, and U45 using reference and non-reference evaluation settings.PSNR, SSIM, LPIPS, and FID are used for UIEBD and LSUI, while UIQM and UCIQE are used for U45.
- Results and Comparisons: WF-Diff achieves state-of-the-art image-quality performance across the reported UIE evaluations.The authors report that WF-Diff outperforms state-of-the-art methods and achieves state-of-the-art performance on image-quality metrics.
- Results and Comparisons: Visual comparisons on six randomly selected UIEBD and LSUI examples show natural-looking results from WF-Diff on real underwater images.The paper presents these examples as evidence of generalization to real-world applications.
- Ablation Study: WFI2-net performs best when all evaluated network components and loss functions are retained, indicating that each contributes to the UIE task.The ablations are conducted on UIEBD using network-structure and loss-function studies.
- Ablation Study: FRDAM ablations show that wavelet-space diffusion variants improve over pixel-level alternatives, while separately refining frequency components supports fine-grained adjustment.The study reports color distortion or artifacts for one pixel-level model and limited detail focus when both frequency bands require adjustment.
5. Conclusion
WF-Diff combines frequency-domain processing with diffusion models for underwater image enhancement. Its WFI2-net and plug-and-play FRDAM support frequency enhancement and detail adjustment, while the two-diffusion design leaves inference speed unimproved.
- 5. Conclusion: WF-Diff combines frequency-domain characteristics with diffusion models through WFI2-net and the plug-and-play FRDAM.WFI2-net enhances and adjusts frequency information, while FRDAM adjusts underwater-image detail.
- 5. Conclusion: The framework achieves state-of-the-art performance on the underwater image enhancement task, with ablations supporting the effectiveness of its contributions.The conclusion reports both the overall task performance and the ablation-based validation.
- 5. Conclusion: Using two diffusion models does not provide an inference-speed advantage, motivating future work on faster sampling.The authors identify sampling acceleration as a future direction.