Source-linked AI summary
Wavelet-Based Dual-Branch Network for Image Demoireing
Lin Liu, Jianzhuang Liu, Shanxin Yuan, Gregory Slabaugh, Ales Leonardis, Wengang Zhou, Qi Tian
TL;DR
Moiré artifacts from photographing digital screens are difficult to remove because they overlap with genuine scene texture across broad frequencies. WDNet separates frequency bands with wavelets and restores them using dense and dilated branches with spatial attention. It achieves the best reported demoiréing results and also performs strongly on non-screen images, deraining, and deraindrop tasks.
Problem
Photographing digital screens produces moiré patterns that are difficult to distinguish from true scene texture while spanning a broad frequency range.
Method
WDNet decomposes moiré-contaminated images into wavelet frequency bands and uses dense and dilated branches with a direction perception attention module for restoration.
Results
WDNet achieves the best demoiréing results, removes moiré from non-screen images, and obtains state-of-the-art results on deraining and deraindrop tasks.
Takeaways & Limitations
Wavelet-domain processing is more effective for moiré removal than RGB-domain processing, while the architecture generalizes to other low-level vision tasks.
Abstract
from arXiv · showhide
When smartphone cameras are used to take photos of digital screens, usually moire patterns result, severely degrading photo quality. In this paper, we design a wavelet-based dual-branch network (WDNet) with a spatial attention mechanism for image demoireing. Existing image restoration methods working in the RGB domain have difficulty in distinguishing moire patterns from true scene texture. Unlike these methods, our network removes moire patterns in the wavelet domain to separate the frequencies of moire patterns from the image content. The network combines dense convolution modules and dilated convolution modules supporting large receptive fields. Extensive experiments demonstrate the effectiveness of our method, and we further show that WDNet generalizes to removing moire artifacts on non-screen images. Although designed for image demoireing, WDNet has been applied to two other low-levelvision tasks, outperforming state-of-the-art image deraining and derain-drop methods on the Rain100h and Raindrop800 data sets, respectively.
1 Introduction
Image demoiréing must remove moiré patterns across broad frequencies while preserving genuine high-frequency scene details. WDNet addresses this by processing wavelet subbands with dual branches and spatial attention, achieving strong demoiréing and cross-task results.
- 1 Introduction: Moiré patterns arise from interference between camera color-filter arrays and screen subpixels, and can also affect textiles and building bricks.These artifacts contaminate the underlying clean image and are challenging to remove.
- 1 Introduction: Demoiréing must recover high-frequency image details while removing moiré patterns spanning a large frequency range.This differs from restoration tasks focused mainly on high-frequency noise removal and detail recovery.
- 1 Introduction: Existing RGB-domain restoration methods are not tailored to demoiréing and have difficulty distinguishing moiré patterns from genuine scene content.Prior attempts include multi-resolution, coarse-to-fine, and edge- or appearance-based approaches.
- 1 Introduction: WDNet decomposes moiré-contaminated images into wavelet frequency bands, then uses dense and dilated branches to restore close-range and far-range information.A direction perception module highlights areas containing moiré patterns.
- 1 Introduction: The paper presents a wavelet-based dual-branch network and a direction perception module as its principal contributions.The contribution list also reports benchmark performance, generalization beyond screens, and a new urban-scene dataset.
- 1 Introduction: WDNet achieves the best reported demoiréing results, also removes moiré artifacts from non-screen images, and obtains state-of-the-art results for deraining and deraindrop.The authors additionally built an urban-scene dataset containing more types of moiré patterns.
2 Related Work
Prior demoiréing work targets particular artifact types or uses RGB-domain restoration, while WDNet applies wavelets to represent broad spatial-frequency content. Related research also motivates texture removal and complementary dual-branch designs.
- Moiré Pattern Removal: Earlier demoiréing methods addressed monochrome, striped, dotted, textile, or scanned-image artifacts, but some could not handle low-frequency moiré patterns.These approaches included spectral modeling and low-rank or sparse matrix decomposition.
- Wavelet-based Methods: The paper presents wavelet transformation as the first use of wavelets for frequency-domain moiré removal.Wavelets combine spatial and frequency information, with separate bands covering a broad frequency range.
- Wavelet-based Methods: Predicting moiré-free wavelet coefficients from RGB images is difficult because moiré patterns span wide spatial and frequency ranges and resemble true textures.This motivates separating frequency content before restoration.
- Dual-branch Design: Dual-branch architectures have been used in super-resolution, classification, segmentation, and person re-identification to extract complementary feature types.The related designs motivate using complementary branches in WDNet.
- Texture Removal: Texture-removal methods are related because moiré patterns can be treated as a special type of texture.Prior approaches used local filtering, dynamic guidance, and fidelity terms to smooth texture while preserving other structures.
3 Our Method
WDNet transforms RGB inputs into the wavelet domain, removes moiré patterns with complementary dense and dilation branches, and reconstructs RGB outputs through an inverse transform.
- 3 Our Method: WDNet transforms the RGB input into wavelet subbands, processes them with a dual-branch network, and applies an inverse wavelet transform to produce the final RGB image.The wavelet representation reduces spatial dimensions while separating frequency content.
- 3.2 Dense Branch: Each of seven dual-branch modules combines a dense branch for close-range patterns with a dilation branch for far-range patterns, using residual skip connections.The dense branch is based on residual dense modules, while the dilation branch uses 3 × 3 convolutions with varying dilation rates.
- 3.1 Working in the Wavelet Domain: Moiré patterns appear prominently in selected wavelet subbands, while higher-frequency subbands generally contain fewer artifacts and preserve details useful for reconstruction.The first three subbands in one example contain obvious moiré patterns.
- 3.3 Dilation Branch: The dilation branch uses increasing Fibonacci-style dilation rates to cover broad moiré patterns while avoiding the gridding artifacts of fixed-rate convolutions.The design uses dilation rates such as (1,2,3,5,8,13,21).
- 3.4 Direction Perception Module: The network adds a direction perception module that expands directional processing to eight directions and generates attention maps highlighting moiré spatial distributions.Its attention output is supervised by an attention loss derived from a thresholded moiré mask.
4 Experiments
Experiments evaluate WDNet for image demoiréing, non-screen artifact removal, and transfer to deraining and deraindrop. Results compare state-of-the-art methods, user preferences, architectural components, wavelet processing, and frequency-based extensions.
- 4.3 Comparison with the State-of-the-Art: WDNet outperforms state-of-the-art methods on both demoiréing datasets and surpasses MopNet by 0.33dB despite MopNet using additional annotations.The comparison uses PSNR and SSIM on TIP2018 and London’s Buildings; qualitative results also report the most effective moiré removal.
- 4.3 Comparison with the State-of-the-Art: WDNet trained on TIP2018 removes moiré artifacts from non-screen images containing dense repetitive textures without fine-tuning.Examples include buildings and clothes, where artifacts may affect only parts of an image.
- 4.4 Ablation Studies: The dual branches acquire close-range and far-range information jointly, while the dense branch focuses on local moiré patterns and the dilation branch covers broader contaminated areas.Heat maps from the sixth layer illustrate the complementary branch behavior.
- 4.4 Ablation Studies: Replacing the dilation branches, dense branches, DPM and attention components, or DPM weight maps lowers PSNR by 1.05dB, 0.52dB, 0.41dB, or 0.27dB, respectively.Replacing the 8-directional DPM with a 4-directional IRNN lowers PSNR by 0.19dB.
- 4.4 Ablation Studies: Removing wavelet processing and its loss makes moiré removal worse, while the complete model better preserves edges and prevents detail blurring.The comparison uses networks with the same computation amount and includes visual evidence from Fig. 9.
- 4.4 Ablation Studies: Across wavelet levels 1, 2, and 3, PSNR/SSIM are 27.24/0.866, 27.88/0.863, and 27.94/0.843, respectively.Increasing the number of levels raises PSNR but lowers SSIM; the authors leave the explanation for future work.
- 4.5 Extension to Deraining and Deraindrop: On Rain100h and Raindrop800, WDNet performs better than state-of-the-art methods for deraining and deraindrop, respectively.The experiments train and test WDNet on the corresponding datasets, with visual comparisons shown in Fig. 11.
5 Conclusion
WDNet removes moiré patterns in the wavelet domain using dual branches and spatial attention, outperforming state-of-the-art models across demoiréing and two related restoration tasks.
- WDNet substantially outperforms other state-of-the-art models for image demoiréing.
- Wavelet-domain processing restores more details and removes moiré patterns more effectively than RGB-domain processing.
- Dense and dilated branches acquire close-range and far-range information, respectively.
- The direction perception module helps dense branches perceive slanting moiré patterns.
- WDNet obtains the best results on deraining and deraindrop tasks and is intended for future applications including denoising and demosaicing.