PaperlayerView exact revision on arXiv

Source-linked AI summary

NTIRE 2026 The Second Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results

Xin Li, Yeying Jin, Suhang Yao, Beibei Lin, Zhaoxin Fan, Wending Yan, Xin Jin, Zongwei Wu, Bingchen Li, Peishu Shi, Yufei Wang, Yu Li, Zhibo Chen, Bihan Wen, Robby T. Tan, Radu Timofte, Runzhe Li, Kui Jiang, Zhaocheng Yu, Yiang Chen, Junjun Jiang, Xianming Liu, Hongde Gu, Zeliang Li, Mache You, Jiangxin Dong, Jinshan Pan, Qiyu Rong, Bowen Shao, Hongyuan Jing, Mengmeng Zhang, Bo Ding, Hui Zhang, Yi Ren, Mohab Kishawy, Jun Chen, Anh-Kiet Duong, Petra Gomez-Kramer, Jean-Michel Carozza, Wangzhi Xing, Xin Lu, Enxuan Gu, Jingxi Zhang, Diqi Chen, Qiaosi Yi, Bingcai Wei, Wenjie Li, Bowen Tie, Heng Guo, Zhanyu Ma, Jiachen Tu, Guoyi Xu, Yaoxin Jiang, Cici Liu, Yaokun Shi, Paula Garrido Mellado, Daniel Feijoo, Alvaro Garcia Lara, Marcos V. Conde, Zhidong Zhu, Bangshu Xiong, Qiaofeng Ou, Zhibo Rao, Wei Li, Zida Zhang, Hui Geng, Qisheng Xu, Xuyao Deng, Changjian Wang, Kele Xu, Guanglu Dong, Qiyao Zhao, Tianheng Zheng, Chunlei Li, Lichao Mou, Chao Ren, Chang-De Peng, Chieh-Yu Tsai, Guan-Cheng Liu, Li-Wei Kang, Abhishek Rajak, Milan Kumar Singh, Ankit Kumar, Dimple Sonone, Kishor Upla, Kiran Raja, Huilin Zhao, Xing Xu, Chuan Chen, Yeming Lao, Wenjing Xun, Li Yang, Bilel Benjdira, Anas M. Ali, Wadii Boulila, Hao Yang, Ruikun Zhang, Liyuan Pan

arXiv:2604.10634v2cs.CV

TL;DR

Real-world raindrop removal lacks benchmarks spanning adherent drops, illumination conditions, and focus modes. This paper reports an NTIRE challenge built on the Raindrop Clarity dataset, where diverse submitted methods achieved strong performance and revealed quality–complexity trade-offs.

  • Problem

    Existing raindrop-removal benchmarks provide limited coverage of complex real environments, nighttime scenes, adherent raindrops, and different focusing modes.

  • Method

    The challenge evaluates restoration methods on the Raindrop Clarity dataset across day/night scenes and dual-focused degradations using a common benchmark and complementary image-quality metrics.

  • Results

    Team AIIA–Lab ranked first with a final score of 35.2378, while Team ULR achieved the best LPIPS of 0.2547.

  • Takeaways & Limitations

    The submitted solutions demonstrate strong progress and diverse trade-offs among restoration quality, perceptual quality, and model complexity.

Abstract

from arXiv · show

This paper presents an overview of the NTIRE 2026 Second Challenge on Day and Night Raindrop Removal for Dual-Focused Images. Building upon the success of the first edition, this challenge attracted a wide range of impressive solutions, all developed and evaluated on our real-world Raindrop Clarity dataset~\cite{jin2024raindrop}. For this edition, we adjust the dataset with 14,139 images for training, 407 images for validation, and 593 images for testing. The primary goal of this challenge is to establish a strong and practical benchmark for the removal of raindrops under various illumination and focus conditions. In total, 168 teams have registered for the competition, and 17 teams submitted valid final solutions and fact sheets for the testing phase. The submitted methods achieved strong performance on the Raindrop Clarity dataset, demonstrating the growing progress in this challenging task.

1. Introduction

Real-world raindrop removal remains constrained by limited benchmarks covering adherent drops, varied illumination, and different focus modes. The challenge addresses this gap using the Raindrop Clarity dataset and a broader evaluation setting.

  • Image deraining restores visual content degraded by rain streaks or adherent raindrops and supports downstream applications such as autonomous driving and surveillance.
  • Existing datasets often rely on synthetic data because paired rainy and clean real-world images are difficult to capture under controlled conditions.
  • Most existing benchmarks emphasize daytime scenes, underrepresent adherent raindrops in complex environments, and rarely include different focusing modes.
  • The challenge uses the real-world Raindrop Clarity dataset, covering daytime and nighttime scenes with dual-focused raindrop degradations.
  • 168 participants registered, while 17 teams submitted valid solutions and fact sheets for testing.

2. Challenge

The challenge evaluates raindrop removal across daytime and nighttime scenes and two focusing modes, using balanced splits and complementary fidelity and perceptual metrics. Its two-stage platform supports development feedback before final testing and ranking.

  • Challenge Setting: The Raindrop Clarity dataset covers daytime and nighttime scenes with raindrop-focused and background-focused images.
  • Challenge Setting: 14,139 images are used for training, 407 for validation, and 593 for testing, with validation and testing organized to balance scenarios.
  • Challenge Objective: The challenge aims to rank restoration methods while encouraging robust and practically useful raindrop removal under complex imaging conditions.
  • Evaluation Protocol: PSNR, SSIM, and LPIPS jointly measure reconstruction fidelity and perceptual similarity in the evaluation.
  • Evaluation Protocol: Higher PSNR and SSIM improve the ranking score, whereas lower LPIPS is preferred.
  • Evaluation Protocol: PSNR and SSIM are computed on the Y channel, while LPIPS uses normalized pixels and an AlexNet-based perceptual-distance configuration.
  • Evaluation Protocol: The competition uses sequential development and testing stages, with online metric feedback during development and final ranking based on testing submissions.

3. Challenge Results

Submitted methods achieved strong benchmark performance while exposing trade-offs among restoration quality, perceptual similarity, and model complexity. AIIA–Lab led the overall ranking, while other teams achieved the best perceptual score or compact models.

  • 35.2378 final score placed Team AIIA–Lab first, with the best PSNR of 28.3392 and SSIM of 0.8265.AIIA–Lab used 16.6M parameters and 129.9 GFlops without extra data.
  • 0.2547 LPIPS was the best perceptual score, achieved by Team ULR.
  • The benchmark reveals diverse trade-offs among restoration quality, perceptual quality, and model complexity across submitted solutions.
  • 2.14M and 2.95M parameters enabled competitive efficiency for teams GU–day Mate and Cidaut AI, respectively.

4. Teams and Methods

The reported teams use varied restoration pipelines, including backbone selection and scene-level refinement, multi-stage pseudo-supervision, complementary model fusion, and specialized architectural designs. Table 1 records quantitative results and resource-related metadata for comparison.

  • AIIA-Lab: AIIA-Lab selects MSDT after comparing representative backbones, then uses checkpoint selection, scene-level fusion, pseudo-GTs, and refinement.
  • Quantitative Comparison: Table 1 reports quantitative results, parameter counts, GFlops, and indicators for ensembles or extra data.
  • raingod: raingod uses a three-stage pipeline with MSDT pretraining, scene-consistent pseudo targets, target-scene fine-tuning, and median-filter refinement.
  • BUU CV: BUU CV combines STRRNet and Restormer trained with rectangular and square patches, then fuses their outputs and applies median filtering.

4.4. RetinexDualV2

RetinexDualV2 uses complementary Retinex branches with a physical rain-intensity prior, then adapts restoration through scene-level pseudo-ground-truth blending. The supplied passages also describe several alternative challenge pipelines, including recurrent aggregation and transductive fine-tuning.

  • RetinexDualV2: RetinexDualV2 separates reflectance and illumination processing into two complementary branches.The method is explicitly grounded in the dual nature of Retinex decomposition.
  • RetinexDualV2: A residual rain-intensity mask from the rainy image and its blurred version provides the physical grounding prior.A shallow UNet estimates rain intensity, and the resulting mask is injected through physical-grounding attention mechanisms.
  • RetinexDualV2: Pseudo ground truths are generated by soft blending images from the same scene, weighted by the residual rain mask, for second-stage fine-tuning.The same weighted blending strategy is also applied during inference to fuse scene-image outputs.
  • ULR: The ULR pipeline combines independent preprocessing, recurrent same-scene aggregation, and postprocessing with three UNet modules.UNet-pre produces preliminary restorations, UNet-seq aggregates scene images, and UNet-post reconstructs the final result.
  • GU-day Mate: GU-day Mate uses category-aware pseudo ground truths and transductive fine-tuning of AdaIR to reduce the training–test distribution gap.Its pipeline uses unsharp masking for blurry samples and augmented AdaIR outputs for other degradations.

4.7. Derain

The Derain methods combine semantic guidance and diffusion training for raindrop restoration. Their designs respectively use language-model-derived degradation and clean priors, or masked diffusion pretraining followed by supervised score-based fine-tuning.

  • Semantic-constrained transformer: The semantic-constrained transformer uses a Restormer backbone with degradation and clean embeddings generated by a multimodal large language model.The two embeddings separately guide degradation removal and content reconstruction.
  • Semantic-constrained transformer: Degradation Fusion of Experts modules are modulated by the degradation embedding, while cross-attention injects the clean embedding at the bottleneck.The decoder reconstructs the restored image from the semantically refined representation.
  • Diffusion framework: The diffusion framework performs masked diffusion autoencoding pretraining followed by supervised score-based fine-tuning.The two-stage design first learns from spatial masking and noise injection, then predicts clean images from noisy degraded inputs.
  • Diffusion framework: The diffusion model uses a time-conditioned diffusion transformer implemented as a multiscale U-Net encoder–decoder.Its fine-tuning objective operates at a fixed noise level.

4.9. Cidaut AI

Cidaut AI-style restoration pipelines emphasize specialized architectures and inference-time fusion for dual-focused raindrop removal. The described approaches use dual attention, histogram-based processing, and multiscale Restormer outputs to address degradation at different spatial and intensity scales.

  • DualDrop: DualDrop uses a NAFNet-based U-Net with a bottleneck Dual-Attention Module combining spatial and frequency attention.The module uses parallel depthwise-convolution branches to distinguish raindrop-corrupted and clean regions.
  • DualDrop: DualDrop contains about 2.95 M parameters and costs roughly 13.26 GFLOPs per image.The supplied implementation passage also reports approximately 19 ms runtime for a 480 × 720 image on H100 GPUs.
  • Histogram Transformer: The Histogram Transformer uses dynamic-range histogram self-attention and a dual-scale gated feed-forward module as its core block.Its attention operates through bin-wise and frequency-wise representations to connect similarly degraded pixels across spatial distances.
  • Multiscale fusion: The multiscale fusion framework resizes inputs to three scales, processes them with overlapping-window Restormer inference, and averages aligned restorations.This design targets rain artifacts appearing at different sizes.

4.12. DGLTeam

DGLTeam and MMAIrider use task- or scene-adaptive restoration pipelines for dual-focused raindrop removal. Their methods respectively select task prompts or combine transformer restoration with scene-aware refinement and uncertainty guidance.

  • DGLTeam: DGLTeam formulates multiscene raindrop removal as a multitask restoration problem with a prompt pool attached to an encoder–decoder.The prompt pool stores task-relevant representations for adaptive restoration.
  • DGLTeam: DGLTeam selects prompts by similarity and injects the aggregated instance-specific representation into the decoder through adaptive gated fusion.This combines shared and task-specific information during restoration.
  • MMAIrider: MMAIrider uses a two-stage pipeline with a transformer deraining backbone followed by uncertainty-guided refinement.The supplied passages identify this as the overall framework shown in its pipeline figure.
  • MMAIrider: The first MMAIrider stage uses scene-guided modulation, deformable sampling attention, and variance-guided local reconstruction.A second SAORM stage combines local, dilated, large-kernel, and frequency-domain branches for refinement.
  • MMAIrider: Scene embeddings from a pretrained image encoder are shared across both MMAIrider stages to guide scene-adaptive restoration.The model applies tiled inference with four geometric test-time augmentations whose outputs are averaged.

4.14. Rain-SVNIT

Rain-SVNIT combines global raindrop removal with luminance-based expert routing, while JiT uses scene-category conditioning for restoration. Both approaches target the challenge’s day/night and focus-diverse conditions.

  • Rain-SVNIT: A two-stage pipeline first performs global raindrop removal, then routes intermediate results to day- or night-specific experts using luminance.The first stage uses Restormer, while the second uses specialized NAFNet models.
  • Rain-SVNIT: The Rain-SVNIT pipeline is trained in two stages with Charbonnier-based objectives, adding edge-aware loss for the second-stage experts.Stage 1 runs for 120 epochs, and Stage 2 trains the day and night experts for 80 epochs without additional data.
  • JiT: The JiT method scales the backbone from JiT-B to JiT-H and conditions restoration on four scene categories.The categories are day, night, background-focused, and raindrop-focused cases.

4.16. PSU

PSU proposes DUSKAN, which combines spectral-spatial processing with adaptive polynomial activations, while DERNet combines Transformer-based global modeling with dynamic local expert aggregation.

  • PSU: DUSKAN uses parallel spectral-spatial and Kolmogorov–Arnold branches within a U-Net to model global degradations and local details.Learned gating weights fuse the branches at each scale.
  • PSU: DUSKAN is trained on 512 × 512 patches with heavy augmentation and a combined ℓ1, perceptual, and focal frequency objective.Training runs for 500 epochs using AdamW with a cosine learning rate schedule and no external data.
  • BITssvgg: DERNet combines Transformer blocks for global context with lightweight convolutional experts selected dynamically for local restoration.The routing branch predicts expert-wise weights conditioned on input features, enabling spatially adaptive processing.
  • BITssvgg: DERNet reconstructs images with a Transformer-based decoder and is trained using patch-based training with a plain ℓ1 reconstruction loss.It uses 180 training epochs, Adam optimization, and no extra data or ensemble methods.

AIIA-Lab

The AIIA-Lab entry lists affiliations spanning universities and laboratories in China, France, Australia, New Zealand, Hong Kong, and India.

  • Affiliations: The listed affiliations include Harbin Institute of Technology and Beijing University of Posts and Telecommunications in China.
  • Affiliations: The affiliations include La Rochelle University’s L3i and LIENSs laboratories in France.
  • Affiliations: Additional affiliations span Griffith University, SenseTime, Dalian University of Technology, Massey University, Hong Kong Polytechnic University, Wuhan University, Beihang University, Nanchang Hangkong University, Sichuan University, Southwest University of Science and Technology, MedAI Technology, SVNIT, NTNU, and Chizhou University.
Loading 2604.10634v2…