PaperlayerView exact revision on arXiv

Source-linked AI summary

NTIRE 2026 The Second Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results

Xin Li, Yeying Jin, Suhang Yao, Beibei Lin, Zhaoxin Fan, Wending Yan, Xin Jin, Zongwei Wu, Bingchen Li, Peishu Shi, Yufei Wang, Yu Li, Zhibo Chen, Bihan Wen, Robby T. Tan, Radu Timofte, Runzhe Li, Kui Jiang, Zhaocheng Yu, Yiang Chen, Junjun Jiang, Xianming Liu, Hongde Gu, Zeliang Li, Mache You, Jiangxin Dong, Jinshan Pan, Qiyu Rong, Bowen Shao, Hongyuan Jing, Mengmeng Zhang, Bo Ding, Hui Zhang, Yi Ren, Mohab Kishawy, Jun Chen, Anh-Kiet Duong, Petra Gomez-Kramer, Jean-Michel Carozza, Wangzhi Xing, Xin Lu, Enxuan Gu, Jingxi Zhang, Diqi Chen, Qiaosi Yi, Bingcai Wei, Wenjie Li, Bowen Tie, Heng Guo, Zhanyu Ma, Jiachen Tu, Guoyi Xu, Yaoxin Jiang, Cici Liu, Yaokun Shi, Paula Garrido Mellado, Daniel Feijoo, Alvaro Garcia Lara, Marcos V. Conde, Zhidong Zhu, Bangshu Xiong, Qiaofeng Ou, Zhibo Rao, Wei Li, Zida Zhang, Hui Geng, Qisheng Xu, Xuyao Deng, Changjian Wang, Kele Xu, Guanglu Dong, Qiyao Zhao, Tianheng Zheng, Chunlei Li, Lichao Mou, Chao Ren, Chang-De Peng, Chieh-Yu Tsai, Guan-Cheng Liu, Li-Wei Kang, Abhishek Rajak, Milan Kumar Singh, Ankit Kumar, Dimple Sonone, Kishor Upla, Kiran Raja, Huilin Zhao, Xing Xu, Chuan Chen, Yeming Lao, Wenjing Xun, Li Yang, Bilel Benjdira, Anas M. Ali, Wadii Boulila, Hao Yang, Ruikun Zhang, Liyuan Pan

arXiv:2604.10634v2cs.CV

TL;DR

Real-world raindrop-removal benchmarks have limited coverage of nighttime scenes and different focusing modes. This paper presents a challenge built around a day/night, dual-focused benchmark, where submitted methods showed diverse performance-efficiency trade-offs and AIIA-Lab ranked first with a final score of 35.2378.

  • Problem

    Existing benchmarks often focus on daytime scenes, give limited emphasis to adherent raindrops in complex real environments, and rarely consider different focusing modes.

    Source: §1 · p. 2

  • Method

    The paper organizes a challenge using the Raindrop Clarity real-world benchmark, covering daytime and nighttime scenes with dual-focused raindrop degradations.

    Source: §1 · p. 2

  • Results

    Submitted methods showed diverse trade-offs among restoration quality, perceptual quality, and model complexity; AIIA–Lab ranked first with a final score of 35.2378.

    Source: §3 · p. 3

  • Takeaways & Limitations

    The unified benchmark provides a more realistic scheme for future research on raindrop removal under complex imaging conditions.

    Source: §2 · p. 2

Abstract

from arXiv · show

This paper presents an overview of the NTIRE 2026 Second Challenge on Day and Night Raindrop Removal for Dual-Focused Images. Building upon the success of the first edition, this challenge attracted a wide range of impressive solutions, all developed and evaluated on our real-world Raindrop Clarity dataset~\cite{jin2024raindrop}. For this edition, we adjust the dataset with 14,139 images for training, 407 images for validation, and 593 images for testing. The primary goal of this challenge is to establish a strong and practical benchmark for the removal of raindrops under various illumination and focus conditions. In total, 168 teams have registered for the competition, and 17 teams submitted valid final solutions and fact sheets for the testing phase. The submitted methods achieved strong performance on the Raindrop Clarity dataset, demonstrating the growing progress in this challenging task.

1. Introduction

The challenge uses a real-world benchmark covering day and night scenes and dual-focused raindrop degradations, addressing a gap in existing raindrop-removal datasets.

1. Introduction

  • Image deraining restores visual content degraded by rain streaks or adherent raindrops [55] and can benefit autonomous driving, surveillance, and pedestrian analysis.Source: §1 · p. 1
  • Restoration research has expanded toward unified adverse-weather restoration and perceptually oriented deraining methods [49].The passage also notes the development of CNN-, Transformer-, MLP-, Mamba-, and diffusion-based restoration backbones [7] [30] [35] [36] [40] [49] [50] [65].Source: §1 · p. 2
  • Existing benchmarks mostly emphasize daytime scenes, give limited attention to adherent raindrops in complex environments, and rarely cover both focus modes and day-night conditions [24] [25] [41] [42].The passage notes that paired rainy and clean images are difficult to capture under controlled real conditions, and describes earlier reliance on synthetic data and later video-temporal or physical-simulation approaches.Source: §1 · p. 2
  • The NTIRE 2026 challenge uses the Raindrop Clarity real-world benchmark [26] to evaluate raindrop removal across daytime and nighttime scenes and dual-focused degradations.It emphasizes robustness to diverse illumination conditions and focus patterns as a setting intended to represent practical applications.Source: §1 · p. 2
  • The challenge drew 168 registrations and 17 teams submitted valid testing-phase solutions and fact sheets; the report summarizes its setting, evaluation protocol, methods, and results.Source: §1 · p. 2

2. Challenge

The challenge uses the Raindrop Clarity dataset to benchmark raindrop removal across day and night scenes and raindrop- and background-focused images. It evaluates methods under a common protocol and aims to encourage robust, practically useful solutions for real-world conditions.

2. Challenge

  • The benchmark uses 14,139 training images, 407 validation images, and 593 test images, with validation and test splits balanced across scenarios.The dataset covers real-world daytime and nighttime scenes and raindrop-focused and background-focused images.Source: §2 · p. 2
  • The challenge seeks both common-benchmark rankings and robust, practically useful raindrop removal under complex imaging conditions.It combines day/night scenes and dual-focused degradations in a unified benchmark.Source: §2 · p. 2

2.1. Evaluation Protocol

  • PSNR, SSIM, and LPIPS jointly assess reconstruction fidelity and perceptual similarity; higher PSNR and SSIM and lower LPIPS are preferred.Source: §2.1 · p. 2

2.2. Competition Phases

  • The competition runs development and final testing stages, with online metric feedback during development and 637 submissions from 35 teams in that stage.Development uses training data and a validation set without released ground truth; final testing requires restored outputs for 593 rainy images.Source: §2.2 · p. 2
  • 54 teams participated in final testing, and 17 submitted valid fact sheets and source code considered for the final ranking.Source: §2.2 · p. 3

3. Challenge Results

The challenge results show closely ranked top teams and a range of trade-offs in perceptual quality and model complexity. Submitted solutions span high-performing systems and compact architectures.

3. Challenge Results

  • Team AIIA–Lab ranked first with a final score of 35.2378, the best PSNR (28.3392) and SSIM (0.8265), and LPIPS of 0.2732.It used 16.6M parameters and 129.9 GFlops without extra data; raingod placed second with a final score of 35.2186, while BUU CV placed third with 35.0360 and required 42.33 GFlops.Source: §3 · p. 3
  • Team ULR achieved the best LPIPS (0.2547), while GU–day Mate and Cidaut AI used only 2.14M and 2.95M parameters, respectively.The submitted methods ranged from high-performing deraining systems to compact architectures.Source: §3 · p. 3

4. Teams and Methods · 4.1. AIIA-Lab · 4.2. raingod

AIIA-Lab and raingod propose multi-stage approaches for dual-focused day and night raindrop removal, using scene-level pseudo-supervision and refinement to improve restorations.

4.1. AIIA-Lab

  • AIIA-Lab selected MSDT after comparing it with Restormer and Histoformer, based on validation performance and optimization potential.The selected backbone is trained on mixed day/night and dual-focused degradations, with long-term training used to retain strong checkpoints.Source: §4.1 · p. 3
  • AIIA-Lab selects scene-appropriate restorations from multiple checkpoints, fuses outputs by scene using mean or median aggregation, then refines the model with pseudo ground truth.Pseudo targets are used in a second training stage with a smaller learning rate and more conservative optimization; final results use the best-performing checkpoint after refinement.Source: §4.1 · pp. 3–4
  • AIIA-Lab trains in PyTorch with Adam and cosine annealing for 200 epochs on 256×256 patches, batch size 1, on a single GPU, without external data.Source: §4.1 · p. 4

4.2. raingod

  • raingod’s first stage pretrains MSDT on RainDrop Clarity [26] and UAV-Rain1k [4] to enhance generalization.Source: §4.2 · p. 4
  • In raingod’s second stage, preliminary restorations support scene-consistent pseudo targets, combining median-filtered coarse targets with reliable clean regions from rainy inputs.The combined targets form refined pseudo supervision for the next stage.Source: §4.2 · p. 4
  • raingod fine-tunes on generated pseudo targets, then applies median-filter-based refinement after inference to better adapt results to target scenes.Source: §4.2 · p. 4

4.3. BUU CV · 4.4. RetinexDualV2 · 4.5. ULR

The three teams use complementary strategies for raindrop removal: dual-patch restoration, Retinex-based physical grounding, and sequential processing of images from the same scene.

4.3. BUU CV

  • BUU CV combines STRRNet on rectangular patches with Restormer on square patches, then fuses their outputs and median-filters residual artifacts.The rectangular-patch model targets elongated streak-like raindrops, while the square-patch model preserves local textures and fine details.Source: §4.3 · p. 4
  • BUU CV trains its two patch-specific models with L1 and SSIM losses and geometric augmentation, then uses sliding-window inference and a weighted ensemble without extra data.Training uses batch size 4 for 200,000 iterations on a single RTX 4090 GPU.Source: §4.3 · p. 5

4.4. RetinexDualV2

  • RetinexDualV2, following RetinexDual, combines reflectance and illumination branches with a residual-rain-mask prior injected through physical-grounding attention.A shallow UNet estimates the mask from paired drops and residual rain intensity; scene-image outputs are soft-blended using the mask for final restoration.Source: §4.4 · p. 5
  • The RetinexDualV2 model uses two-stage training, then full-resolution inference and scene-level soft blending; it has about 4.8 M parameters and roughly 301.6 GFLOPs per 768 × 768 input.Training uses 768 × 768 patches; the second stage fine-tunes on mask-weighted pseudo ground truths, with no external data.Source: §4.4 · p. 5

4.5. ULR

  • ULR processes each image independently, recurrently aggregates images from the same scene, then restores each result using the aggregation and original input.UNet-pre, UNet-seq and UNet-post are trained sequentially; inference groups images by scene and averages outputs from different models.Source: §4.5 · pp. 5–6
  • ULR trains its three components for 100 epochs on the provided dataset using Adam at 1 × 10−6 with batch size 4 and no external data.Training is performed on a single NVIDIA H100 GPU.Source: §4.5 · pp. 5–6

4.6. GU-day Mate · 4.7. Derain · 4.8. NTR

GU-day Mate adapts AdaIR through scene-specific transductive fine-tuning, Derain guides restoration with two semantic priors, and NTR uses a two-stage diffusion framework. The three sections describe distinct approaches to raindrop restoration rather than comparative benchmark results.

4.6. GU-day Mate

  • GU-day Mate builds frequency-aware transductive fine-tuning on AdaIR [12], using scene-specific pseudo-GTs from unsharp masking for blurry samples and augmented AdaIR outputs for other degradations.The pseudo-labels are used to fine-tune directly on each test scene.Source: §4.6 · p. 6
  • GU-day Mate combines progressively larger 128-to-512 crops, frequency-domain MixUp, EMA, and Charbonnier-plus-Sobel losses to stabilize training.Training spans approximately 120 epochs, and the method uses no external data.Source: §4.6 · p. 6
  • For test scenes, GU-day Mate fine-tunes on scene-derived pseudo-GTs and averages an eight-fold geometric self-ensemble; its model has about 2.14 M parameters and 64.2 GFLOPs per 512 × 512 crop.Training takes around 24 hours on an 8×A100 GPU cluster.Source: §4.6 · p. 6

4.7. Derain

  • Derain proposes a Restormer-based semantic-constrained transformer with degradation and clean embeddings generated by a multimodal large language model to guide degradation removal and content reconstruction.The network has approximately 16.1 million parameters and costs about 132.3 GFLOPs; it uses no external data or self-ensembles.Source: §4.7 · p. 6
  • Derain modulates encoder Degradation Fusion of Experts modules with the degradation embedding and injects the clean embedding into the bottleneck through cross-attention for semantically consistent refinement.It is trained from scratch for roughly 500,000 iterations on 192 × 192 patches using a weighted ℓ1-plus-Fourier loss.Source: §4.7 · p. 6

4.8. NTR

  • NTR proposes a two-stage diffusion restoration framework, combining masked diffusion autoencoding pretraining with supervised score-based fine-tuning.Its MDAE pretraining corrupts each input using spatial masking and noise injection [20] [73].Source: §4.8 · pp. 6–7
  • NTR fine-tunes with the D2S-SFT score-based objective [72], predicting clean images from noisy degraded inputs at a fixed noise level using a time-conditioned, multiscale U-Net diffusion transformer [73].Source: §4.8 · p. 7

4.9. Cidaut AI

Cidaut AI proposes DualDrop, a NAFNet-based UNet framework for raindrop removal with a bottleneck Dual-Attention Module combining spatial and frequency attention. The single end-to-end model is trained on Raindrop Clarity without extra datasets or multi-stage training.

4.9. Cidaut AI

  • DualDrop uses a NAFNet [6] backbone with a UNet architecture comprising four encoder and decoder stages of NAFBlocks.The stages are designed to balance restoration performance and computational efficiency.Source: §4.9 · p. 7
  • At the bottleneck, the Dual-Attention Module combines spatial and frequency attention to distinguish raindrop-corrupted from clean regions and guide localization and restoration.Its parallel branches use cascaded depthwise dilated convolutions with different receptive fields and a standard depthwise convolution, then fuse their features.Source: §4.9 · p. 7
  • DualDrop is trained for 400 epochs on Raindrop Clarity using 384 × 384 crops and batch size 16, with no extra datasets or multi-stage training.The AdamW optimizer uses β1 = β2 = 0.9 and weight decay 1 × 10−3; the learning rate decays from 2×10−3 to 1 × 10−6 with cosine annealing. The model has about 2.95 M parameters, roughly 13.26 GFLOPs, and runs in around 19 ms per 480 × 720 image on H100 GPUs without ensemble or fusion strategies.Source: §4.9 · p. 7

4.10. NCHU-CVLab · 4.11. NUDT-Deeplter · 4.12. DGLTeam

The three teams use distinct restoration strategies: histogram-transformer encoding, multi-scale test-time fusion, and prompt-based multi-task restoration. Their methods differ in training and inference procedures, with the latter two using test-time fusion strategies.

4.10. NCHU-CVLab

  • NCHU-CVLab uses a Histogram Transformer U-shaped encoder-decoder whose histogram attention connects distant pixels with similar degradation.Its Histogram Transformer Blocks combine dynamic-range histogram self-attention with dual-scale gated feed-forward modules and hierarchical coarse-to-fine restoration.Source: §4.10 · p. 7
  • NCHU-CVLab trains progressively over five stages and uses full-resolution inference without ensembles.Patch sizes increase from 128 to 288 pixels over about 600k iterations; the model has roughly 29.38 M parameters and 318 GFLOPs complexity, and processes a 480 × 720 image in around 1.2 s on four RTX 3090 GPUs.Source: §4.10 · p. 7

4.11. NUDT-Deeplter

  • NUDT-Deeplter resizes each image to three scales, processes overlapping sliding windows with a Restormer, then realigns and averages the restorations.The three-scale fusion is the method’s only ensemble strategy and is intended to combine information from multiple resolutions.Source: §4.11 · pp. 7–8
  • NUDT-Deeplter trains its single Restormer on competition-data crops using combined ℓ1 reconstruction and VGG perceptual loss.Training runs for 100 epochs on one GPU; the model has approximately 26.1 M parameters and about 140 GFLOPs per 256 × 256 patch.Source: §4.11 · p. 8

4.12. DGLTeam

  • DGLTeam frames multi-scene raindrop removal as multi-task restoration, using a prompt pool and adaptive gated fusion to integrate shared and task-specific information.The model selects prompts by similarity and aggregates them into an instance-specific task representation for the decoder.Source: §4.12 · p. 8
  • DGLTeam applies eight-fold geometric self-ensemble with median fusion, then fuses same-scene outputs and combines them with the input.It trains on 128 × 128 patches for roughly two million iterations and uses overlapping sliding windows at inference; the final model has about 42.6 M parameters.Source: §4.12 · p. 8

4.13. MMAIrider

MMAIrider proposes a two-stage pipeline for dual-focused raindrop removal, combining a transformer-based deraining backbone with uncertainty-guided refinement.

4.13. MMAIrider

  • MMAIrider’s two-stage pipeline uses a transformer-based deraining backbone followed by a refinement module for uncertainty-guided restoration.The stages provide initial raindrop removal and refinement of the intermediate output.Source: §4.13 · p. 8
  • The first stage modifies UDR-S2Former with scene-guided modulation, deformable sampling attention, and variance-guided local reconstruction for adaptive restoration.Its U-Net-style transformer uses a multi-scale encoder-decoder, with a learnable query bias conditioned on scene-type embeddings at the bottleneck.Source: §4.13 · p. 9
  • The second stage applies the Scene-Aware Omni-kernel Restoration Module, fusing convolutional and frequency-attention branches through scene-conditioned dual gating.A pretrained image encoder supplies scene embeddings shared across both stages to guide scene-adaptive restoration.Source: §4.13 · p. 9

4.14. Rain-SVNIT · 4.15. Just JiT

Rain-SVNIT uses luminance-based expert routing in a two-stage restoration pipeline, while Just JiT conditions a scaled image transformer on scene categories for dual-focused raindrop removal.

4.14. Rain-SVNIT

  • Rain-SVNIT combines global Restormer restoration with luminance-routed NAFNet experts specialized for day and night scenes.The intermediate restoration is routed by luminance to reduce cross-domain interference.Source: §4.14 · pp. 9–10
  • Rain-SVNIT trains Restormer for 120 epochs, then day/night NAFNet experts for 80 epochs using Charbonnier and edge-aware losses.The models use the official dataset without additional data.Source: §4.14 · p. 10
  • Rain-SVNIT applies eight-fold flip-and-rotation self-ensemble to expert predictions and averages the ensemble outputs.The system has roughly 26.1 M parameters and uses about 140 GFLOPs per 256×256 patch; inference takes about 800 ms per image on an RTX 3090.Source: §4.14 · p. 10

4.15. Just JiT

  • Just JiT scales the JiT backbone from JiT-B to JiT-H and conditions restoration on day, night, background-focused, or raindrop-focused scene labels.A ConvNeXt-Tiny classifier supplies the scene labels; no extra data are used.Source: §4.15 · p. 10
  • Just JiT trains JiT-B and JiT-H for 600 epochs with a dynamic Charbonnier, SSIM, and LPIPS hybrid loss.The LPIPS weight is zero for the first 30% of epochs, then increases linearly to its final value; training uses official images only.Source: §4.15 · p. 10
  • Just JiT restores overlapping 256×256 patches with Hanning-window blending and averages JiT-B and JiT-H predictions.JiT-H has approximately 953 M parameters and 182 GFLOPs; end-to-end inference takes about 1.2 seconds per image on an RTX 6000 GPU.Source: §4.15 · p. 10

4.16. PSU · 4.17. BITssvgg · Organizers

The PSU and BITssvgg sections describe complementary raindrop-removal architectures: DUSKAN combines spectral-spatial and adaptive-activation branches, while DERNet pairs Transformer-based global modeling with dynamic local experts.

4.16. PSU

  • DUSKAN runs spectral-spatial and Kolmogorov–Arnold branches in parallel within a U-Net and fuses them with learned gating weights.The branches target global frequency and local spatial information, with adaptive polynomial activations for complex nonlinear relationships.Source: §4.16 · p. 10
  • DUSKAN is trained for 500 epochs on 512 × 512 patches using ℓ1, perceptual, and focal frequency losses, with no external data or pre-trained models.The approximately 45.6 M-parameter model requires about 493 GFLOPs per 512 × 512 input.Source: §4.16 · p. 11
  • DUSKAN processes full-resolution images fully convolutionally without sliding-window splitting, test-time augmentation, or ensembles.Its inference is intended to recover low-frequency global degradations and high-frequency details.Source: §4.16 · p. 11

4.17. BITssvgg

  • BITssvgg proposes DERNet, an encoder-decoder combining Transformer-based global modeling with adaptive aggregation of local convolutional experts.The method pipeline is shown in Figure 15.Source: §4.17 · p. 11, Figure 15 · p. 11
  • DERNet uses Transformer blocks for long-range context and input-conditioned routing weights to select lightweight depthwise-convolution experts for spatially varying degradation.Its Transformer-based decoder progressively reconstructs the restored image.Source: §4.17 · p. 11
  • DERNet is trained for 180 epochs using patch-based training and plain ℓ1 loss, then tested fully convolutionally with padding rather than patch splitting.It uses about 16.87 M parameters and 496 GFLOPs; a 720 × 480 image takes roughly 0.4 s on two RTX 4090 GPUs, with no extra data or ensemble.Source: §4.17 · p. 11

Related papers

Loading 2604.10634v2…