Source-linked AI summary
Unpaired Deep Image Deraining Using Dual Contrastive Learning
Xiang Chen, Jinshan Pan, Kui Jiang, Yufeng Li, Yufeng Huang, Caihua Kong, Longgang Dai, Zhentao Fan
TL;DR
SID lacks practical paired real-world supervision, and existing unpaired methods struggle with the rainy-clean domain gap. DCD-GAN combines bidirectional adversarial translation with dual contrastive guidance, reporting favorable unpaired results and comparable performance to several supervised or semi-supervised models. Its performance is limited on small-scale datasets because contrastive learning requires many sample pairs.
Problem
SID must estimate clean images from rainy inputs, but paired real-world data is difficult to obtain and synthetic paired data leaves a domain gap.
Method
DCD-GAN combines bidirectional translation with dual contrastive learning to exploit mutual features between unpaired rainy and clean images.
Results
DCD-GAN performs favorably against existing unpaired deraining approaches and achieves comparable performance to several fully supervised or semi-supervised models.
Takeaways & Limitations
Unpaired clean exemplars can facilitate rain removal and support deraining generalization across synthetic and real rainy datasets.
Takeaways & Limitations
Performance is limited on small-scale datasets because contrastive learning tends to require many sample pairs for excellent performance.
Abstract
from arXiv · showhide
Learning single image deraining (SID) networks from an unpaired set of clean and rainy images is practical and valuable as acquiring paired real-world data is almost infeasible. However, without the paired data as the supervision, learning a SID network is challenging. Moreover, simply using existing unpaired learning methods (e.g., unpaired adversarial learning and cycle-consistency constraints) in the SID task is insufficient to learn the underlying relationship from rainy inputs to clean outputs as there exists significant domain gap between the rainy and clean images. In this paper, we develop an effective unpaired SID adversarial framework which explores mutual properties of the unpaired exemplars by a dual contrastive learning manner in a deep feature space, named as DCD-GAN. The proposed method mainly consists of two cooperative branches: Bidirectional Translation Branch (BTB) and Contrastive Guidance Branch (CGB). Specifically, BTB exploits full advantage of the circulatory architecture of adversarial consistency to generate abundant exemplar pairs and excavates latent feature distributions between two domains by equipping it with bidirectional mapping. Simultaneously, CGB implicitly constrains the embeddings of different exemplars in the deep feature space by encouraging the similar feature distributions closer while pushing the dissimilar further away, in order to better facilitate rain removal and help image restoration. Extensive experiments demonstrate that our method performs favorably against existing unpaired deraining approaches on both synthetic and real-world datasets, and generates comparable results against several fully-supervised or semi-supervised models.
1. Introduction
Single image deraining must recover clean images from rainy inputs despite limited paired real-world data and a substantial domain gap. DCD-GAN addresses this with bidirectional translation and dual contrastive guidance, achieving favorable reported performance.
- Rain degradation reduces visibility and can cause drastic performance drops in detection, segmentation, and video surveillance.
- SID estimates clean image B from rainy image O under the model O = B + R, where R denotes rain streaks.
- Fully supervised methods rely on paired synthetic data that poorly models real-world degradation, while paired real-world data is difficult to obtain at scale.
- Existing unpaired methods do not effectively restore high-quality derained results without suitable constraints for rain streaks and clean images.
- DCD-GAN combines bidirectional translation with dual contrastive learning to mine mutual features and distinguish dissimilar rainy and clean-domain representations.
- Experiments report favorable performance against unpaired methods and comparable performance against several fully supervised or semi-supervised models.
2. Related Work
Prior SID work spans paired, semi-supervised, and unpaired approaches, while contrastive learning supplies a representation-learning principle based on positive and negative examples. DCD-GAN applies these ideas through bidirectional translation and contrastive guidance.
- Deep SID methods are categorized as paired fully supervised, semi-supervised, or unpaired approaches.
- Paired methods use architectures such as multi-layer CNNs, density-aware streams, recurrent networks, and spatial attention to model or remove rain.
- Semi-supervised methods combine limited labeled data with auxiliary objectives, domain analysis, synthetic data, unlabeled real rain, or rain-direction regularization.
- Unpaired methods commonly adapt CycleGAN, but cycle-consistency constraints alone make accurate rainy-to-clean transformation difficult.
- DCD-GAN uses BTB to guide cross-domain transfer and latent distributions, while CGB constrains mutual feature information during joint optimization.
- Contrastive learning pulls embeddings toward positive samples and pushes them away from negative samples to maximize mutual features across domains.
3. Proposed Method
DCD-GAN combines bidirectional translation with dual contrastive guidance to learn deraining from unpaired rainy and clean images. Its loss function further regularizes reconstruction through color, adversarial, and frequency constraints.
- Bidirectional Translation Branch (BTB): BTB uses two generators and two discriminators to translate between rainy and clean domains through bidirectional cycle-consistency transformations.The generated exemplars provide rain-related and clean-cue features for exploring latent feature distributions.
- Contrastive Guidance Branch (CGB): With CGB, rainy-image and clean-image features have low latent-space distances, enabling rainy features to generate clean images.The t-SNE visualization compares feature embeddings learned with and without CGB.
- Contrastive Guidance Branch (CGB): CGB constrains deep feature embeddings by pulling positive cross-domain pairs closer and pushing negative pairs farther apart.It uses auxiliary generator encoders and considers both internal and external latent codes.
- Loss Function: The hybrid objective adds contrastive, color cycle-consistency, adversarial, and frequency losses because ground-truth clean images are unavailable.Color cycle-consistency computes separate pixel losses for red, green, and blue channels, while frequency loss preserves image content and structure.
4. Experiments
Experiments evaluate DCD-GAN on synthetic and real-world deraining benchmarks, compare it with prior, supervised, semi-supervised, and unpaired methods, and examine its design choices and generalization. Results show strong deraining quality, competitive comparisons, and improved cross-domain and application-oriented behavior.
- Experimental Settings: DCD-GAN is compared with prior-based, paired supervised, semi-supervised, and unsupervised deep deraining methods across the evaluation.The comparison includes DSC, GMM, DDN, DID-MDN, RESCAN, SPANet, SSIR, Syn2Real, CycleGAN, RR-GAN, and DerainCycleGAN.
- Experimental Settings: Synthetic benchmarks include Test100, Test1200, and Test1400, with PSNR and SSIM used when ground truth is available.The study also uses NIQE and BRISQUE for cases without ground truth.
- Synthetic Data: DCD-GAN achieves state-of-the-art PSNR among unpaired methods and competitive performance with several paired supervised models on synthetic data.The authors attribute the competitive paired comparison to additional constraints from the contrastive guidance branches.
- Real-world Data: On real-world images, DCD-GAN obtains the best PSNR on Test1000, competitive SSIM, and lower NIQE and BRISQUE scores than comparison methods.Lower NIQE and BRISQUE scores indicate higher-quality, more natural, and more faithful outputs in the reported evaluation.
- Ablation Study: Removing loss components, changing negative selection, or removing the backward cycle degrades deraining quality, while the full loss reaches 25.61 dB on Test100.The backward cycle also produces more clean-cue features, and using broader negative selection avoids blurrier derained results.
- Generalization and Application: DCD-GAN maintains stronger cross-domain deraining behavior than fully supervised approaches under domain variation and improves recognition-oriented evaluation on real rainy images.Google Vision API evaluation reports improved recognition accuracy and significantly reduced averaged rain-recognition confidence for DCD-GAN outputs.
5. Concluding Remarks
The paper concludes that DCD-GAN provides an effective unpaired SID framework by jointly using bidirectional translation and contrastive guidance. Its experiments support effectiveness and generalization, while performance is limited on small-scale training datasets.
- Concluding Remarks: DCD-GAN unifies bidirectional translation and contrastive guidance to train an unpaired single-image deraining model.The framework uses contrastive learning to explore mutual and dissimilar deep features across rainy and rain-free domains.
- Concluding Remarks: Unpaired clean exemplars facilitate rain removal, and learned latent restoration supports cross-domain deraining generalization.The conclusion presents this capability as a central benefit of the proposed approach.
- Concluding Remarks: DCD-GAN performs favorably against existing unpaired approaches and comparably with several fully supervised or semi-supervised models on challenging datasets.The reported conclusion covers both synthetic and real rainy datasets.
- Limitation: Small-scale training datasets constrain SID performance because contrastive learning tends to require many sample pairs for excellent performance.This is the paper’s stated limitation and scope boundary.