Source-linked AI summary
Structure-Preserving Deraining with Residue Channel Prior Guidance
Qiaosi Yi, Juncheng Li, Qinyan Dai, Faming Fang, Guixu Zhang, Tieyong Zeng
TL;DR
Single-image deraining methods struggle to remove rain completely while preserving image structure, especially in complex scenarios. SPDNet uses RCP guidance with WMLM, IFM, and iterative reconstruction, and experiments on synthetic and real-world data report new state-of-the-art results.
Problem
Existing deraining methods have difficulty removing all rain streaks and recovering accurate object structures in complex scenarios.
Method
SPDNet uses RCP guidance, WMLM background learning, IFM fusion, and iterative guidance for progressive structure-preserving reconstruction.
Results
Experimental results on synthetic datasets and real-world scenarios show that SPDNet achieves new state-of-the-art results.
Takeaways & Limitations
RCP provides reference information for learning background content and progressively guiding reconstruction toward clearer structures.
Abstract
from arXiv · showhide
Single image deraining is important for many high-level computer vision tasks since the rain streaks can severely degrade the visibility of images, thereby affecting the recognition and analysis of the image. Recently, many CNN-based methods have been proposed for rain removal. Although these methods can remove part of the rain streaks, it is difficult for them to adapt to real-world scenarios and restore high-quality rain-free images with clear and accurate structures. To solve this problem, we propose a Structure-Preserving Deraining Network (SPDNet) with RCP guidance. SPDNet directly generates high-quality rain-free images with clear and accurate structures under the guidance of RCP but does not rely on any rain-generating assumptions. Specifically, we found that the RCP of images contains more accurate structural information than rainy images. Therefore, we introduced it to our deraining network to protect structure information of the rain-free image. Meanwhile, a Wavelet-based Multi-Level Module (WMLM) is proposed as the backbone for learning the background information of rainy images and an Interactive Fusion Module (IFM) is designed to make full use of RCP information. In addition, an iterative guidance strategy is proposed to gradually improve the accuracy of RCP, refining the result in a progressive path. Extensive experimental results on both synthetic and real-world datasets demonstrate that the proposed model achieves new state-of-the-art results. Code: https://github.com/Joyies/SPDNet
1. Introduction
CNN-based deraining methods struggle to remove rain completely and preserve object structures in complex scenes. SPDNet addresses this by using RCP-guided structure preservation, WMLM background learning, IFM fusion, and iterative guidance.
- CNN-based methods can leave rain streaks and lose structural information because they focus on rain structure rather than object structure and image priors.
- RCP retains clear structures when extracted from rainy images and is computed from the maximum and minimum channel values without additional parameters.
- SPDNet directly reconstructs rain-free images under RCP guidance, while WMLM learns background information in rain-covered regions.
- Extensive experiments report that SPDNet achieves new state-of-the-art results.
- The RCP extraction module, IFM, and iterative guidance strategy progressively extract, fuse, and update structural guidance.
2. Related Work
Related work spans traditional prior-based and CNN-based single-image deraining methods. Despite progress, existing approaches may fail to remove rain fully and preserve object structures in complex situations.
- Traditional deraining methods use manually extracted features and priors to describe and separate rain streaks.
- CNN-based methods include progressively recurrent, context-aggregation, detail-recovery, and multi-scale fusion networks.
- Existing methods still struggle to remove all rain streaks and protect structural information in complex situations.
3. Structure-Preserving Deraining Network
SPDNet combines a wavelet-based backbone with residue channel prior guidance to reconstruct rain-free images while preserving object structure. Its iterative pipeline progressively refines the prior and the deraining result.
- Wavelet-based Feature Extraction Backbone: SPDNet uses wavelet-based multi-level modules to learn rainy-image background information across scales.WMLM applies wavelet decomposition, residual feature learning, and inverse wavelet reconstruction.
- Wavelet-based Feature Extraction Backbone: WMLM can learn rainy-image background information and reconstruct relatively clear images, but the basic reconstruction may damage object structure.The paper therefore adds image-prior guidance to address structure loss.
- Residue Channel Prior: RCP is derived from channel differences and can be free from rain streaks because the achromatic rain component is canceled during normalization.The resulting prior contains more complete and accurate object structure for deraining guidance.
- Residue Channel Prior: The RCP extraction module uses convolution and SE-ResBlocks to obtain deeper, less noisy prior features.These features provide the structural information used by the network.
- Interactive Fusion Module: IFM progressively combines rainy-image and RCP features, using similarity maps to enhance background information and strengthen prior structure.The fused features are added to the original feature before producing the IFM output.
- Iterative Guidance Strategy: Iterative guidance replaces the rainy-image prior with clearer priors extracted from intermediate outputs, and SPDNet uses three WMLMs with multi-stage outputs.The model applies convolution to generate each rain-removal result and RCP extraction to generate the next prior.
4. Experiment and Discussions
Experiments evaluate SPDNet on synthetic and real-world deraining datasets, including quantitative, visual, ablation, and vision-application analyses. The results report strong deraining quality, clearer structures, and favorable component and runtime trade-offs.
- Datasets: SPDNet is evaluated on Rain200L/H, Rain800, Rain1200, SPA-Data, and another real-world dataset.SPA-Data contains 638492 rainy/clear training pairs, while the other real-world dataset lacks clear-image labels.
- Synthesized Images: SPDNet achieves remarkable quantitative improvements over state-of-the-art methods on the evaluated synthetic datasets.The comparison uses PSNR and SSIM in YCbCr space and also reports processing time and parameter comparisons.
- Synthesized Images: Visual results show SPDNet removes rain streaks while preserving finer image textures than competing methods on synthetic datasets.Other methods tend to blur textures or leave visible rain streaks.
- Real-world Images: SPDNet achieves superior results on real-world data, including the best results on a dataset without corresponding clear-image labels.The unlabeled dataset is evaluated with the non-reference indicators NIQE and PI.
- Ablation Study: Using three WMLM modules gives the best PSNR/SSIM but increases image-processing time.Using one WMLM gives the lowest performance while remaining faster than the three-module setting.
- Ablation Study: IFM and Ensemble each improve SPDNet performance in ablation experiments, while additional iterative RCP guidance further improves results and structural clarity.IFM outperforms feature concatenation, Ensemble improves PSNR and SSIM, and more RCP guidance produces superior performance.
- Application: SPDNet improves downstream vision applications by increasing object detections and reducing recognized-rain confidence in Google Vision API evaluations.YOLOv3 experiments on COCO350 also report improved detection performance.
5. Conclusion
SPDNet combines RCP guidance, WMLM, IFM, and iterative updating to preserve structure during deraining. Experiments show improved deraining quality and superiority over other top-performing methods.
- SPDNet uses an RCP guidance network and iterative guidance strategy for structure-preserving deraining.RCP guides WMLM learning, while iterative updates improve RCP accuracy and re-guide WMLM.
- WMLM learns background information, while IFM makes full use of RCP information.
- Compared with methods without RCP, RCP-guided methods reconstruct high-quality images with clear and accurate structures.
- Experiments on synthetic datasets and real-world scenarios show SPDNet's superiority over other top-performing methods.The ablation study evaluates RCP, IFM, and the iterative update strategy.