NTIRE 2026 Low-light Enhancement: Twilight Cowboy Challenge

1. Introduction

• Low-light images are noisy at low signal levels, while longer exposures reduce noise but increase motion blur from camera or scene movement.Without stabilization, practical captures may therefore be either noisy or blurred.Source: §1 · p. 1
• Burst imaging captures rapid short-exposure frames that are sharp but noisy, and accurate alignment lets them be aggregated into a long-exposure-quality image.The enhancement task thus involves both image alignment and image aggregation.Source: §1 · pp. 1–2
• The challenge targets denoising collections of misaligned low-exposure frames to stimulate advanced image-processing research for night photography.Source: §1 · p. 2

2. Twilight Cowboy Challenge overview

• The challenge requires joint motion compensation and denoising of handheld low-light bursts in Bayer/RAW, rather than relying on demosaiced RGB images or perfect alignment.It targets noise and strong scene light sources while accounting for natural hand tremor.Source: §2 · p. 2

2.1. Data collection

• Each training sample pairs one RAW ground-truth image with five misaligned RAW captures of the same low-light scene, including indoor and outdoor scenes.Capture began with 270 tripod-mounted RAW frames per scene; five handheld frames were selected at every tenth frame to introduce natural motion.Source: §2.1 · p. 2

2.2. Ground truth generation

• Ground truth was built from 270 frames per scene by rejecting intensity outliers, aligning remaining frames, estimating pixelwise means, and applying quantile-based normalization.Frames outside the fitted 99.7% intensity interval were discarded before alignment; moving objects outside the field of view were a common source of intensity mismatch.Source: §2.2 · pp. 2–3

2.3. Data structure

• The challenge released 250 training pairs and 50 validation samples in stage one, then 150 additional training pairs and 50 validation samples in stage two; the unreleased test set contained 85 pairs.About one third of the test scenes were indoors, and baseline code was provided.Source: §2.3 · p. 3

2.4. Evaluation and placement

• Teams were ranked separately by mean PSNR and mean SSIM on the test subset, then ordered by a final score that prioritizes PSNR rank over SSIM rank.The inputs were the five burst images, with the estimated ground truth as reference.Source: §2.4 · p. 3, §2.3 · p. 3

3. Results

• Inference time did not affect final placement; the reported per-sample times provide insight into processing efficiency and reconstruction quality versus model complexity.Source: §3 · p. 3
• MiAlgo, DH ISP, and BAU-Vision ranked first through third; MiAlgo reached 40.62 dB PSNR and 0.9875 SSIM, representing the stated current state-of-the-art.Source: Table 1 · p. 3, §3 · p. 3
• ColorWorld was the most lightweight at 4.7 s per sample, while DH ISP achieved 40.23 dB PSNR with a 10.7 s per-sample inference time.Source: §3 · p. 3

3.1. Baseline

• The participant baseline used frame alignment and averaging followed by BM3D denoising.Source: §3.1 · p. 3
• For alignment, RAW frames were rendered to sRGB for pretrained ASpanFormer matching, and RANSAC used the correspondences to estimate robust homographies.Source: §3.1 · p. 3
• Homographies aligned all five frames to the reference before averaging; BM3D was then applied independently to each RAW channel.Source: §3.1 · p. 3
• BM3D’s noise parameter was selected to maximize PSNR, then fixed at σ = 3/2047 for all images under an approximately uniform-noise assumption.Source: §3.1 · p. 3
• The final output used robust percentile-based normalization, linearly scaling RAW intensities between the 1st and 99th percentiles.Source: §3.1 · pp. 3–4

• MiAlgo fuses weighted predictions from three complementary restoration models, then averages geometric test-time augmentation outputs per model for stability across diverse night scenes.The two-level aggregation is intended to help when an individual model fails on particular texture, noise, or motion patterns.Source: §3.2 · p. 4
• Flow-guided deformable alignment injects precomputed optical-flow cues to better preserve local structures and reduce motion-induced artifacts under large inter-frame motion.The approach is inspired by burst HDR and restoration methods.Source: §3.2 · p. 4
• The sequential denoising ensemble combines Restormer [57], XRestormer, and NAFNet [6], whose transformer-style global modeling and efficient convolutional behavior provide complementary restoration biases.Their weighted outputs are accumulated into a final prediction for low-light RAW restoration.Source: §3.2 · p. 4
• Eight-way geometric TTA transforms RAW frames and flow maps consistently, then inverse-transforms and averages outputs to improve orientation robustness and reduce prediction variance.Mismatched image and flow transforms degrade alignment quality; Figure 3 shows the full testing pipeline.Source: §3.2 · p. 4
• After pretraining on the full training set, MiAlgo removes about 25% of samples with visible burst-to-ground-truth misalignment and finetunes on the cleaned subset to improve spatial consistency.The cleaning step reduces pixel-shift artifacts in final outputs.Source: §3.2 · pp. 4–5

3.3. DH ISP

• DH ISP transfers modeled sensor-noise residuals to ground-truth images to reduce misregistration artifacts and help the network learn noise statistics.Residuals are extracted after coarse alignment and modeled with PCA-based noise covariance analysis and GAN-driven noise style transfer.Source: §3.3 · p. 5
• The framework combined frozen-weight ASpanFormer [5] and SCUNET [58] for multi-frame RAW denoising enhancement, optimized with a composite of L1, perceptual, SSIM, and edge losses.SCUNET used a training window size of 16.Source: §3.3 · p. 5

3.4. BAU-Vision

• BAU-Vision decouples global luminance normalization and spatial registration from hierarchical restoration in the packed Bayer domain to preserve sensor statistics and reduce interpolation artifacts.Source: §3.4 · p. 5
• A Gradient Boosting Regressor estimates target brightness from 53 image statistics, while random preamplification jitter of 0.85–1.15 trains restoration in an exposure-invariant latent space.The statistics include intensity percentiles, histogram shape, spatial grid means, and per-channel color statistics.Source: §3.4 · p. 5
• BetterSpyNet uses instance normalization [48] to estimate exposure-invariant flow from structural features, refining motion compensation residually across four pyramid levels.The flow estimator is based on the Spatial Pyramid Network architecture [42].Source: §3.4 · p. 5
• Temporal Attention Fusion [14] uses reference features to score aligned supporting frames and gate unreliable regions before weighted temporal aggregation.Spatial masks suppress misaligned areas, occlusions, and pixels affected by severe motion blur.Source: §3.4 · p. 5
• The restoration stage uses modified NAFNet [6] with SimpleGate operations and weighted composite L1/MS-SSIM/brightness-consistency loss; final results use 8-view test-time augmentation and post-prediction BM3D denoising (σ ≈1.47 × 10^-3).Source: §3.4 · p. 5

3.5. AXIOM

• AXIOM improves baseline through stronger full-resolution alignment, confidence-aware burst fusion, and adaptive RAW-domain denoising while retaining its two-branch design.The design uses preview-based correspondence estimation and full-resolution RAW reconstruction.Source: §3.5 · p. 5
• For practical full-resolution inference, AXIOM optimizes matcher execution and separates alignment/fusion from denoising, reducing peak memory and enabling stable target-resolution deployment.Source: §3.5 · p. 5
• AXIOM scores auxiliary frames using matching statistics, homography inlier ratio, and reprojection error, then rejects unreliable frames and confidence-weights reliable ones.Lucas–Kanade posthomography refinement reduces residual geometric errors after RANSAC, improving robustness to misalignment, weak correspondences, and blur.Source: §3.5 · pp. 5–6
• In an optional extension, low-rank adaptation is applied to the ASpanFormer [5] matcher on the target domain, training only 33,792 of about 15.8M parameters.Checkpoint selection is performed at the pipeline level.Source: §3.5 · p. 6
• Local confidence maps derived from residuals between aligned auxiliary frames and the reference preview weight fusion toward regions with better agreement.This addresses spatially non-uniform alignment errors and helps preserve cleaner details.Source: §3.5 · p. 6
• Adaptive RAW-domain BM3D strength is estimated from fused-burst content, making denoising responsive to scene-dependent noise rather than using a fixed level.The method is intended to improve reconstruction across varying low-light inputs.Source: §3.5 · p. 6

3.6. html5attention3

• RestoRoBurr combines Restormer, RoMa, and Burstormer in a RAW-to-RAW denoising pipeline for the challenge.The pipeline is introduced in Fig. 5.Source: §3.6 · p. 6
• To reduce ghosting from heavy misalignment, html5attention3 coarsely aligns training images, denoises them with Restormer [57], then uses RoMav2 [16] feature matching on denoised counterparts.Denoising exposes texture structure suppressed by shot noise, improving feature matching on noisy images.Source: §3.6 · p. 6
• Burstormer is trained on aligned images with a Laplacian MLP gate using full-frame sharpness to suppress motion-blurred frames, and a Charbonnier + γ1 · Frequency + γ2 · Gradient loss.The loss weights are γ1 = 0.1 and γ2 = 0.2.Source: §3.6 · p. 6
• Training crops increase from 128 × 128 to 256 × 256, inference uses tiled 256×256 crops, and augmentation applies flips and 90 degree rotations.During the second half of training, hard-crop mining rejects 25% of crops with the smallest loss.Source: §3.6 · p. 6

3.7. ColorWorld

• BadNet, a 6.03M-parameter NAFBlock-based U-Net [6], addresses extreme scene dynamic range with per-scene gain g = 1/P99, confidence-weighted optical-flow alignment, and progressive 128→512 crop training.The authors describe the gain as normalizing inputs to the network’s optimal representational range [34]; progressive crops expand receptive fields and mitigate crop-boundary overfitting.Source: §3.7 · p. 6
• At inference, tiled 5122 processing with AMP→FP32 fallback [57] ensures numerical stability in extreme low-SNR regions dominated by photon shot noise; inversion ŷ = fθ(x · g)/g recovers physical scale.Source: §3.7 · p. 6

3.8. mIpTMO

• mIpTMO aligns burst frames in RAW space and denoises their averaged RAW data without demosaicing before fusion.This preserves Bayer data through alignment and averaging.Source: §3.8 · p. 6
• The first burst frame serves as reference; RoMa v2 [16] matches rendered sRGB images to estimate per-pixel warps and overlap confidence, which are applied to RAW.The reference RAW buffer and its rendered sRGB version are both retained.Source: §3.8 · p. 6
• Each Bayer frame is warped as four half-resolution channels, frames below 0.90 overlap-region SSIM are rejected, and the remaining RAW frames are confidence-weighted and averaged.Source: §3.8 · p. 6
• BM3D denoises each Bayer channel independently using σ = α ˆσ + (1 − α) σ0, with α = α0 + 0.05 nrej and ˆσ estimated by wavelet-detail MAD.σ0 = 3/2047 and α0 = 0.25; increasing frame rejections raises α and shifts the estimate toward ˆσ.Source: §3.8 · pp. 6–7
• The normalized denoised result is quantile-stretched and saved as 16-bit RAW, using pretrained RoMa v2 and BM3D without learning or fine-tuning.Source: §3.8 · p. 7

3.9. PSU

• PSU pairs challenge-baseline multi-frame RAW fusion with DUSKAN deep refinement in a symmetric four-level U-Net.DUSKAN uses global residual learning and a learned per-stage gate to blend complementary branches; training minimizes L1 + 0.01 LVGG + 0.1 LFF.Source: §3.9 · p. 7, Figure 6 · p. 7
• DUSKAN’s spectral-spatial branch processes FFT magnitude and phase, enriches magnitude features, then fuses global features with a spatial branch.Learnable positional encodings, SE channel reweighting, and LayerNorm-MLP refinement are applied before inverse transformation and 1×1 bottleneck fusion.Source: §3.9 · p. 7, Figure 6 · p. 7
• Its parallel adaptive branch replaces fixed projections with learned polynomial-basis Kolmogorov-ArnoldLinear layers, depthwise convolutions, and a gated mixer.A parallel-additive gate provides stable residual gradient paths.Source: §3.9 · p. 7, Figure 6 · p. 7

3.10. FengFans

• FengFans aligns frames by matching SuperPoint keypoints with LightGlue, estimates homographies using RANSAC, and warps Bayer sub-channels before masked weighted averaging.Keypoints are detected on auto-exposed sRGB previews; Bayer sub-channels are warped independently with coordinates scaled by ÷2 to preserve the CFA pattern.Source: §3.10 · p. 7
• The aligned average undergoes dual-channel preprocessing into eight channels: channels 1–4 percentile-stretch the Bayer average from [p1, p99] to [0, 1], while channels 5–8 log-compress raw values, resolving brightness ambiguity in very dark scenes.Channels 5–8 use log1001(1 + x × 1000).Source: §3.10 · p. 7
• For multi-frame RAW denoising, the solution uses learned denoising with a brightness-aware 8-channel NAFNet [6], which takes 8-channel input and outputs 4-channel denoised Bayer.The model has width 96, encoder blocks [4] [8], and 4 middle blocks (155M parameters). Inference uses 512×512 tiles with 64 px overlap and 8× TTA (4 rotations × 2 flips).Source: §3.10 · p. 7
• FengFans trains progressively with increasing patch sizes and Charbonnier + 0.3×SSIM loss, then averages the last five checkpoints per stage using stochastic weight averaging.The checkpoint averaging is used to reduce overfitting.Source: §3.10 · p. 7

3.11. NTR

• The proposed pipeline jointly aligns and denoises five handheld RAW Bayer frames in extreme low light, operating entirely on the Bayer mosaic domain to preserve the sensor pattern.The entire pipeline is CPU-based.Source: §3.11 · p. 7
• ORB feature matching and RANSAC estimate homographies from rendered grayscale views, which provide higher-quality low-light proxies than naive Bayer demosaicing.Frame 1 is the reference; homographies are applied per Bayer channel at half resolution with validity masks, preserving the mosaic without color aliasing artifacts.Source: §3.11 · p. 7
• Aligned frames are fused by validity-mask-weighted averaging to reduce noise while preserving spatial resolution.Fusion is followed by BM3D denoising and quantile normalization, as in the baseline.Source: §3.11 · pp. 7–8

• +6.49 dB in PSNR and +0.0101 in SSIM over the established baseline were achieved by participants.Source: §4 · p. 8
• MiAlgo achieved 40.62 dB PSNR with 514.6 s per sample, while ColorWorld (above 36 dB PSNR; 4.7 s per sample) and DH ISP (above 40 dB PSNR; 10.7 s per sample) were faster.The winning solution used extensive model ensembling and test-time augmentation; the passage notes optimized architectures may be more suitable for practical scenarios such as mobile deployment.Source: §4 · p. 8