Source-linked AI summary
EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining
Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Wei Feng, Yang Liu
TL;DR
Single-image deraining is challenging because rain-generation assumptions may not cover diverse real-world rain, while efficiency is important for real-time applications. EfficientDeRain uses model-free pixel-wise dilation filtering and RainMix augmentation. It is reported to run around 88 times faster than RCDNet with similar PSNR and SSIM.
Problem
Single-image deraining methods often rely on rain-generation assumptions that may not represent diverse real-world rain, and efficient deraining matters for real-time applications.
Source: §1 · p. 1Method
EfficientDeRain is a model-free, single-stage method using pixel-wise dilation filtering and RainMix augmentation for training on real-rain patterns.
Source: §1 · p. 2Results
Around 88 times faster than RCDNet with similar PSNR and SSIM, EfficientDeRain also achieves better visualization on real rainy images when equipped with RainMix.
Source: §1 · p. 2Takeaways & Limitations
The reported results show that model-free deraining can achieve high efficiency and deraining quality on synthetic and real-world rainy datasets.
Source: §5 · pp. 7–8
Abstract
from arXiv · showhide
Single-image deraining is rather challenging due to the unknown rain model. Existing methods often make specific assumptions of the rain model, which can hardly cover many diverse circumstances in the real world, making them have to employ complex optimization or progressive refinement. This, however, significantly affects these methods' efficiency and effectiveness for many efficiency-critical applications. To fill this gap, in this paper, we regard the single-image deraining as a general image-enhancing problem and originally propose a model-free deraining method, i.e., EfficientDeRain, which is able to process a rainy image within 10~ms (i.e., around 6~ms on average), over 80 times faster than the state-of-the-art method (i.e., RCDNet), while achieving similar de-rain effects. We first propose the novel pixel-wise dilation filtering. In particular, a rainy image is filtered with the pixel-wise kernels estimated from a kernel prediction network, by which suitable multi-scale kernels for each pixel can be efficiently predicted. Then, to eliminate the gap between synthetic and real data, we further propose an effective data augmentation method (i.e., RainMix) that helps to train network for real rainy image handling.We perform comprehensive evaluation on both synthetic and real-world rainy datasets to demonstrate the effectiveness and efficiency of our method. We release the model and code in https://github.com/tsingqguo/efficientderain.git.
1 Introduction
Rain can degrade outdoor vision systems, while deraining for efficiency-critical applications must balance image quality, speed, and low overhead. EfficientDeRain avoids explicit rain-generation assumptions and multi-stage refinement, combining pixel-wise dilation filtering with RainMix; it is reported to run around 88 times faster than RCDNet with similar PSNR and SSIM.
1 Introduction
- Rain streaks can degrade outdoor vision tasks, and efficient, high-quality deraining with low overhead matters for real-time applications such as autonomous driving or navigation.Source: §1 · p. 1
- Existing methods often rely on rain-generation assumptions that may not reflect real-world patterns, while computationally expensive optimization and refinement can limit efficiency.Source: §1 · p. 2
- EfficientDeRain is model-free and single-stage, avoiding rain-generation assumptions, iterative optimization, and progressive refinement.Its pixel-wise dilation filtering applies kernels predicted for each pixel to capture suitable multi-scale rain patterns.Source: §1 · p. 2
- RainMix is proposed as data augmentation to bridge synthetic and real data and train networks to handle real rainy images.Source: §1 · p. 2
- Around 88 times faster than RCDNet with similar PSNR and SSIM, EfficientDeRain also produces better real-rain visual results when trained with RainMix.The paper reports evaluations on synthetic and real-world rainy datasets.Source: §1 · p. 2
2 Related Work
Related work spans video-based and single-image deraining, with this paper focusing on deep-learning single-image methods. The surveyed methods commonly rely on rain models or domain priors, and many use recurrent or progressive stages that may constrain universality and efficiency.
2 Related Work
- The review distinguishes video-based deraining from single-image deraining and concentrates on representative deep-learning methods in the latter category.Source: §2 · p. 2
- RCD-Net uses convolutional dictionary learning and proximal-gradient optimization, iteratively updating rain maps and background layers across multiple stages [Wang et al. 2020a].Source: §2 · p. 2
- Other surveyed approaches include CVID's variational prediction integration, JORDER's joint rain detection and removal, and RESCAN's recurrent aggregation [Du et al. 2020; Yang et al. 2019; Li et al. 2018b].Source: §2 · p. 3
- Representative single-image methods rely on presumed rain models or rain-specific priors, and many use recurrent or progressive stages [Li et al. 2018b; Wang et al. 2020a].The paper argues these prerequisites can potentially hinder universality, efficiency, and real-world deployment.Source: §2 · p. 3
3 Methodology
EfficientDeRain treats deraining as pixel-wise filtering with image-adaptive kernels, extended across dilation scales. RainMix augments training with transformed real rain streaks to address the synthetic-to-real gap.
3.1 Pixel-wise Image Filtering for Deraining
- Pixel-wise filtering processes each rainy-image pixel with its exclusive kernel to estimate the derained image.The method identifies spatial, scale, and semantic variation in rain as challenges for effective kernel estimation.Source: §3.1 · p. 3
Kernel prediction network.
- A UNet-like kernel prediction network estimates spatial-variant kernels from rainy images, adapting to streak thickness and strength while preserving object boundaries.The method is trained offline on rainy-clean image pairs.Source: Kernel prediction network · p. 4, §3.1 · p. 4
Multi-dilated image filtering and fusion.
- To handle multi-scale rain streaks efficiently, the method extends each predicted kernel to three scales via dilated convolution.In practice, dilation factors l = 1, 2, 3, 4 produce four derained images, which a 3 × 3 convolution fuses into the final output.Source: Multi-dilated image filtering and fusion · p. 4
Loss function.
- Training uses L1 and SSIM loss functions, with λ fixed at 0.2 in all experiments.Source: Loss function · p. 5
3.3 RainMix: Bridging the Gap to Real Rain
- RainMix samples real rain streaks, applies randomly combined transformations, aggregates the transformed maps, and blends them with an original sampled map.The transformations include zooming, shearing, translation, and rotation.Source: §3.3 · p. 5
4.1 Setups
The evaluation covers synthetic and real rainy-image datasets, uses PSNR and SSIM to assess quality, and compares against existing deraining methods.
Datasets.
- Rain1400 is identified as a synthetic dataset, while SPA is a real-rain dataset and Raindrop is a real dataset used to evaluate the method.Figure 8 shows four EfDeRain variants: v1 lacks dilation, while v2 uses pixel-wise dilation; both use L1 loss. v3 and v4 share v2’s structure, with L1 and SSIM loss respectively, and v4 uses RainMix.Source: Datasets · p. 5, Figure 8 · p. 6
Metrics.
- PSNR and SSIM are used across datasets, and larger values indicate better deraining results.Figure 8 contrasts v1-v4: v1 lacks dilation, v2 adds it, v3 uses L1 and SSIM losses, and v4 additionally uses RainMix.Source: Metrics · pp. 5–6, Figure 8 · p. 6
Baselines.
- The study compares nine streak-deraining baselines and evaluates runtime on the same Xeon E5-1650 CPU and Quadro P6000 GPU.It also compares methods on the Raindrop dataset.Source: Baselines · p. 6
4.2 Comparison on Rain100H&1400 Dataset
On Rain100H and Rain1400, EfDeRain combines low runtime with competitive restoration quality; visual comparisons also report effective rain removal and detail preservation.
4.2 Comparison on Rain100H&1400 Dataset
- 88 times faster than RCDNet, EfDeRain achieves almost the same PSNR and SSIM on Rain100H.On Rain1400, it has the lowest time cost while maintaining competitive PSNR and SSIM with methods including PReNet and RCDNet.Source: §4.2 · p. 6
- EfDeRain improves relative PSNR over CVID by 11.4% on Rain100H while running over 50 times faster.On Rain1400, the authors note that enhanced object boundaries can yield negative PSNR and SSIM scores, which measure recovery rather than enhancement.Source: §4.2 · p. 6
- In visual comparisons, EfDeRain removes rain traces near a nose while preserving scene details such as hair and fine lines.RCDNet and PReNet are reported to remove or destroy some original streak-like details in two cases.Source: §4.2 · p. 6
4.3 Comparison on Real-world SPA Rain Dataset
On the real-rain SPA dataset, EfDeRain is reported to match the leading methods’ quality while running much faster, with visual comparisons favoring its results.
4.3 Comparison on Real-world SPA Rain Dataset
- Over 71 times more efficient than RCDNet, EfDeRain achieves almost the same PSNR and SSIM on SPA and outperforms the other baselines.SPA uses real rainy images with ground truth obtained by human labeling and multi-frame fusion.Source: §4.3 · p. 6
- EfDeRain handles varied rain traces and removes a wider streak in Case 2 that RCDNet and SPANet fail to remove.The resulting image is reported to be almost the same as the ground truth.Source: §4.3 · p. 6
4.4 Comparison on Real-world Raindrop Dataset
On the Raindrop dataset, EfDeRain performs competitively without architecture or hyperparameter changes, and the accompanying visualization notes removal of drops missed by human labeling.
4.4 Comparison on Real-world Raindrop Dataset
- In the visualization comparison of versions trained with or without RainMix, EfDeRain also removes raindrops not labeled by humans.The yellow arrow identifies these unlabeled drops.Source: Figure 9 · p. 7
- Without changing its architecture or hyperparameters, EfDeRain achieves the second-best results with competitive SSIM against DeRaindrop and outperforms the other compared deraining methods.Table 1 reports results on Raindrop and highlights the top three; the supplied caption does not specify their values.Source: §4.4 · p. 7, Table 1 · p. 7
4.5 Ablation Study
The Rain100H ablation tests pixel-wise dilation filtering, loss choices, and RainMix. The final combination achieves the highest reported PSNR and SSIM, while dilation filtering adds around 1.5 ms.
4.5 Ablation Study
- The final v4 combination of dilation filtering, L1 and SSIM losses, and RainMix achieves the highest PSNR and SSIM among the four Rain100H variants.The variants progress from KPN filtering without dilation (v1) to dilation filtering (v2), then L1 and SSIM losses (v3 and v4), with RainMix used for v4.Source: §4.5 · p. 7, Table 2 · p. 7
- Dilation filtering adds around 1.5 ms, and PSNR and SSIM increase progressively across the four variants toward v4.Source: §4.5 · p. 7
- Dilation facilitates greater rain-streak removal, while SSIM loss sharpens image details but also makes rain-streak boundaries more visible.In the visual comparisons, the final version removes heavy streaks while recovering details.Source: §4.5 · p. 7
- RainMix improves removal of diverse real-rain traces on SPA; without it, heavy traces can remain unaddressed.Source: §4.5 · p. 7
5 Conclusions
EfficientDeRain treats single-image deraining as model-free filtering with pixel-wise multi-scale kernels and RainMix augmentation. The paper reports advantages in efficiency and deraining quality across synthetic and real-world datasets, and identifies other vision tasks and adversarial attacks as future directions.
5 Conclusions
- EfficientDeRain is model-free and filters each pixel with multi-scale kernels estimated by a kernel prediction network, with RainMix bridging synthetic and real data.Source: §5 · p. 7
- Evaluation on Rain100H, Rain1400, SPA, and Raindrop demonstrates advantages in both efficiency and deraining quality.Source: §5 · pp. 7–8
- Future work will study deraining's effects on object segmentation and tracking, and examine single-image deraining from an adversarial-attack perspective.Source: §5 · p. 8