Source-linked AI summary
NTIRE 2026 Low-light Enhancement: Twilight Cowboy Challenge
Aleksei Khalin, Egor Ershov, Artyom Panshin, Sergey Korchagin, Georgiy Lobarev, Arseniy Terekhin, Sofiia Dorogova, Amir Shamsutdinov, Yasin Mamedov, Bakhtiyar Khalfin, Bogdan Sheludko, Emil Zilyaev, Nikola Banić, Georgy Perevozchikov, Radu Timofte, Shuai Liu, Yuqian Zhang, Lize Zhang, Yibin Huang, Chaoyu Feng, Luyang Wang, Xiaotao Wang, Dongqing Zou, Lei Lei, Tianli Liu, Dejun Hao, Chunxia Lei, Furkan Kınlı, Andrei Mironov, Alexander Dikov, Aleksei Sadokhin, Vladimir Zvorygin, Constantine Habarlak, Shuwei Yue, Egor Mirantsov, Daniil Okunev, Dmitry Arkhipov, Aleksandr Yugay, Anas M. Ali, Bilel Benjdira, Wadii Boulila, Wei Zhou, Linfeng Li, Lingdong Kong, Jiachen Tu, Guoyi Xu, Yaoxin Jiang, Jiajia Liu, Yaokun Shi
TL;DR
Low-light handheld photography must address both severe noise and frame misalignment when combining short-exposure RAW bursts. The paper reviews a challenge that evaluates methods for producing denoised RAW images from such bursts, with participant solutions improving over the baseline by up to +6.49 dB in PSNR and +0.0101 in SSIM. These results establish strong progress in burst-based low-light enhancement under realistic conditions.
Problem
Low-light burst imaging must handle high noise and hand-induced misalignment while operating directly on RAW data.
Method
The challenge collects handheld misaligned RAW bursts with reference images and evaluates participant methods for RAW-domain alignment, fusion, and denoising.
Results
+6.49 dB in PSNR and +0.0101 in SSIM were achieved over the established baseline.
Takeaways & Limitations
The results demonstrate progress in burst-based low-light enhancement under real-world noise, motion, and illumination variability.
Abstract
from arXiv · showhide
This paper presents a review of the NTIRE 2026 Low-light Enhancement: Twilight Cowboy Challenge. The objective of the competition was to merge a set of misaligned smartphone images in the raw domain, captured in low-light conditions, into a single, clean image. Introduced setup simultaneously addresses two problems of low-light photography: visual degradations such as high noise and mixed scene illuminants, and the geometric inconsistencies caused by hand movement during multi-frame capture. To advance research in low-light and nighttime computational photography, a challenging dataset was collected comprising 585 real-world scenes, spanning indoor low-light and outdoor nighttime conditions, for training and benchmarking participant solutions. The competition employed a three-stage evaluation protocol: automatic validation via the CodaBench platform in stages one and two, followed by blind assessment on a private test set for the final ranking. Ten teams surpassed the established baseline, achieving improvements of up to +6.49 dB in PSNR and +0.0101 in SSIM, thereby establishing new state-of-the-art performance for burst-based low-light image enhancement. These results demonstrate significant progress in handling real-world noise, motion, and illumination variability in the low-light setting. Comprehensive results, leaderboards, and additional information are publicly available at https://nightimaging.org.
1. Introduction
Low-light imaging forces a trade-off between noise and motion blur, while burst imaging offers a way to aggregate sharp, noisy frames when they are accurately aligned. The challenge targets denoising from misaligned low-exposure frames to advance nighttime image processing.
- 1. Introduction: Low-light denoising is difficult because increasing exposure reduces noise but increases susceptibility to motion blur.Without stabilization, practical captures can become either noisy or blurred.
- 1. Introduction: Burst imaging captures rapid sequences of short-exposure images that are sharp but noisy.The frames can be aggregated because the noise is approximately unbiased.
- 1. Introduction: Accurate alignment enables burst frames to approximate the quality of a single long-exposure image.The enhancement problem therefore separates into image alignment and image aggregation.
- 1. Introduction: The challenge asks participants to generate a denoised image from misaligned low-exposure frames.Its stated motivation is to stimulate research in advanced image processing for night photography.
2. Twilight Cowboy Challenge overview
The Twilight Cowboy Challenge evaluates handheld RAW burst denoising under realistic motion and low-light conditions. Its dataset pairs clean RAW references with misaligned handheld bursts, while a staged protocol evaluates submissions using image-quality metrics and ranking scores.
- 2. Twilight Cowboy Challenge overview: Nighttime burst capture reduces noise by combining frames but makes alignment difficult because handheld shooting introduces camera motion.The challenge therefore targets both denoising and registration under realistic nighttime conditions.
- 2. Twilight Cowboy Challenge overview: The task transforms handheld, unprocessed RAW images into a denoised RAW image equivalent to one captured with a stationary camera.It requires joint motion compensation and denoising directly in the Bayer/RAW domain.
- 2. Twilight Cowboy Challenge overview: The dataset uses pairs containing one RAW ground-truth image and five misaligned RAW images from the same low-light scene.Both indoor and outdoor scenes are included.
- 2. Twilight Cowboy Challenge overview: Ground-truth construction begins with 270 tripod-captured RAW frames and uses handheld sequences to produce five naturally motion-varied inputs.Every tenth frame from the handheld sequence is selected.
- 2. Twilight Cowboy Challenge overview: The three-stage protocol provides training and validation data in stages one and two, followed by a private test set of 85 input–ground-truth pairs.Approximately one third of the test data comprises indoor scenes.
- 2. Twilight Cowboy Challenge overview: Submissions are evaluated using mean PSNR and SSIM, with separate rankings combined into a final score that prioritizes PSNR.Teams are ranked in ascending order of the final score.
3. Results
The challenge results compare reconstruction quality, efficiency, and baseline processing across submitted solutions. MiAlgo achieved the highest reported quality, while DH ISP offered a strong performance–efficiency balance.
- MiAlgo, DH ISP, and BAU-Vision ranked first through third, respectively.
- 40.62 dB PSNR and 0.9875 SSIM were achieved by MiAlgo, representing the reported state-of-the-art result.
- 10.7 s per sample and 40.23 dB PSNR made DH ISP the strongest reported balance between performance and efficiency.
- 4.7 s per sample made ColorWorld the most lightweight solution while maintaining good reconstruction quality.
- Baseline: Simple alignment, averaging, and BM3D denoising formed the participant baseline.
- Baseline: The final output used percentile-based RAW normalization after denoising, with intensities scaled using the 1st and 99th percentiles.
3.2. MiAlgo
MiAlgo’s solution combines ensemble restoration, flow-guided alignment, sequential denoising, and consistency-preserving test-time augmentation. It also cleans misaligned training samples before fine-tuning to reduce spatial artifacts.
- Three restoration models and geometric test-time augmentation were combined in a dual-ensemble framework.
- Flow cues were injected into deformable alignment to improve correspondence under large inter-frame motion.
- Restormer, XRestormer, and NAFNet were executed sequentially to combine complementary restoration behaviors.
- Eight-way geometric TTA transformed RAW frames and flow maps consistently before inverse transformation and averaging.
- Removing about 25% of visibly misaligned training samples before fine-tuning improved spatial consistency and reduced pixel-shift artifacts.
3.3. DH ISP
DH ISP separates exposure normalization, RAW-domain registration, temporal fusion, and hierarchical restoration. Its pipeline combines exposure-invariant motion estimation with attention-based frame selection and composite-loss denoising.
- The framework combines ASpanFormer alignment with SCUNET for multi-frame RAW denoising enhancement.
- Packed Bayer-domain processing preserves sensor statistics and avoids interpolation-induced artifacts by separating normalization and registration from restoration.
- A Gradient Boosting Regressor estimates target scene brightness from a 53-dimensional statistical vector for exposure normalization.
- BetterSpyNet performs exposure-invariant coarse-to-fine optical-flow alignment across four pyramid levels.
- Temporal Attention Fusion uses cross-attention masks to suppress misaligned, occluded, or motion-blurred regions during burst aggregation.
- Modified NAFNet restoration uses SimpleGate operations, a composite L1, MS-SSIM, and brightness-consistency loss, 8-view augmentation, and BM3D post-processing.
3.5. AXIOM
The described solutions strengthen burst reconstruction through reliable alignment, confidence-aware fusion, and adaptive RAW-domain denoising. RestoRoBurr additionally combines denoised feature matching with blur-aware burst fusion.
- Full-resolution alignment, confidence-aware burst fusion, and adaptive RAW-domain denoising improve the baseline’s reconstruction components.
- Separating alignment and fusion from denoising reduces peak memory usage for target-resolution inference.
- Frame reliability is estimated from matching statistics, homography inlier ratios, and reprojection errors before weighted fusion.
- Local confidence maps weight regions with stronger agreement in the warped reference coordinate system.
- Adaptive BM3D strength is estimated from fused burst content instead of using a fixed denoising level.
- RestoRoBurr combines Restormer, RoMa, and Burstormer in a RAW-to-RAW denoising pipeline.
- Denoised training images expose texture structure for RoMa feature matching, improving alignment of noisy frames.
- Burstormer uses full-frame Laplacian sharpness and a Laplacian MLP gate to suppress motion-blurred frame contributions.
3.7. ColorWorld
ColorWorld uses a classical RAW-domain pipeline that aligns burst frames, fuses them with confidence filtering, and denoises the result without learning or fine-tuning.
- Alignment: ColorWorld aligns burst frames to a reference using RoMa v2, applying rendered-domain correspondence in RAW space to avoid demosaicing artifacts.
- Fusion: Frames whose warped RAW falls below an SSIM threshold of 0.90 are discarded before confidence-weighted fusion.
- Fusion and denoising: The method forms a confidence-weighted average of aligned RAW frames and denoises each Bayer channel independently with BM3D.
- Denoising: BM3D noise estimation combines a robust MAD-based estimate with a fixed base noise level, increasing the estimate’s weight as more frames are rejected.
- Output: The pipeline normalizes the BM3D output and saves a 16-bit RAW image, relying on pretrained RoMa v2 and BM3D rather than learning.
3.9. PSU
PSU combines multi-frame RAW fusion with a DUSKAN deep-refinement stage to restore local texture and residual global degradations.
- Pipeline: PSU uses a two-stage pipeline consisting of baseline multi-frame RAW fusion followed by deep refinement.
- DUSKAN refinement: DUSKAN is a symmetric four-level U-Net whose blocks blend spectral-spatial and Kolmogorov-Arnold branches with learned gating.
- Path A: The spectral branch uses FFT-based magnitude processing, positional encodings, channel reweighting, and spatial fusion through a 1×1 bottleneck.
- Path B: The Kolmogorov-Arnold branch learns polynomial basis activations per edge and combines them with depthwise convolutions and gated residual paths.
- Training: Training minimizes a combined loss with pixel-level, semantic, and spectral terms: L = L1 + 0.01 LVGG + 0.1 LFF.
- Related PSU pipeline: The solution also includes SuperPoint–LightGlue alignment, dual brightness preprocessing, tiled inference, 8× TTA, and progressive training with weight averaging.
3.11. NTR
NTR proposes a fully classical, CPU-based pipeline that aligns and fuses five handheld RAW Bayer frames before BM3D denoising.
- Overview: NTR processes five handheld RAW Bayer frames entirely in the Bayer mosaic domain using a CPU-based classical pipeline.
- Alignment: ORB matching and RANSAC estimate a homography from rendered grayscale views, which is applied independently to half-resolution Bayer channels.
- Fusion: Validity-mask-weighted averaging fuses the aligned frames to reduce noise while preserving spatial resolution.
- Output: Fusion is followed by BM3D denoising and quantile normalization, as in the baseline.
4. Discussion
The challenge produced substantial gains over baseline performance while exposing a trade-off between reconstruction quality and computational cost.
- Overall results: +6.49 dB PSNR and +0.0101 SSIM were achieved over the established baseline.
- Quality–cost trade-off: MiAlgo reached 40.62 dB PSNR using extensive ensembling and test-time augmentation, but required 514.6 seconds per sample.
- Efficiency: ColorWorld and DH ISP showed that faster pipelines can remain competitive, with mean PSNR above 36 dB and 40 dB at 4.7 and 10.7 seconds per sample.
5. Teams and affiliations
The challenge roster includes an organizing team and multiple participating teams from universities, research institutes, and industry organizations across several countries. The listed affiliations span Russia, Saudi Arabia, Singapore, the United States, China, Türkiye, Bulgaria, and Croatia.
- The organizers include researchers affiliated with Moscow Institute of Physics and Technology, ITMO University, Color Reproduction and Synthesis Institute, and related Russian institutions.
- Industry-affiliated teams include MiAlgo from Xiaomi, DH ISP from Zhejiang Dahua Technology, and html5attention3 from ConstantAI.
- Other listed participants represent Bahçeşehir University, National Research Nuclear University MEPhI, Shenzhen Polytechnic University, and the mIpTMO team.
- Participating teams include PSU, FengFans, and NTR, affiliated respectively with Prince Sultan University, National University of Singapore, and the University of Illinois Urbana-Champaign.