Source-linked AI summary
Beyond Model Design: Data-Centric Training and Self-Ensemble for Gaussian Color Image Denoising
Gengjia Chang, Xining Ge, Weijun Yuan, Zhan Li, Qiurong Song, Luen Zhu, Shuhong Liu
TL;DR
Gaussian color denoising at fixed σ=50 remains a performance question even with a mature Restormer backbone. The paper strengthens training with larger, diverse datasets and two-stage optimization, then applies ×8 geometric self-ensemble; on 100 validation images, the final model reaches 30.7622 dB PSNR and 0.8607 SSIM, up to 3.3662 dB above the public baseline.
Problem
The paper examines whether a mature Restormer Gaussian denoiser can improve further without changing its architecture.
Method
It expands public training data, uses two successive optimization stages, and applies ×8 geometric self-ensemble while retaining a TLC-style wrapper for implementation consistency.
Results
30.7622 dB PSNR and 0.8607 SSIM are reached on the unified 100-image validation protocol, with up to 3.3662 dB PSNR improvement over the public Restormer σ = 50 pretrained baseline.
Takeaways & Limitations
The dominant improvement comes from the stronger trained checkpoint, while self-ensemble adds a small but consistent gain and the TLC-style wrapper has negligible quantitative effect.
Takeaways & Limitations
×8 self-ensemble substantially increases inference cost, while single-pass inference remains strong when throughput or deployment cost matters more.
Abstract
from arXiv · showhide
This paper presents our solution to the NTIRE 2026 Image Denoising Challenge (Gaussian color image denoising at fixed noise level $σ= 50$). Rather than proposing a new restoration backbone, we revisit the performance boundary of the mature Restormer architecture from two complementary directions: stronger data-centric training and more complete Test-Time capability release. Starting from the public Restormer $σ\!=\!50$ baseline, we expand the standard multi-dataset training recipe with larger and more diverse public image corpora and organize optimization into two stages. At inference, we apply $\times 8$ geometric self-ensemble to further release model capacity. A TLC-style local inference wrapper is retained for implementation consistency; however, systematic ablation reveals its quantitative contribution to be negligible in this setting. On the challenge validation set of 100 images, our final submission achieves 30.762 dB PSNR and 0.861 SSIM, improving over the public Restormer $σ\!=\!50$ pretrained baseline by up to 3.366 dB PSNR. Ablation studies show that the dominant gain originates from the expanded training corpus and the two-stage optimization schedule, and self-ensemble provides marginal but consistent improvement.
1. Introduction
The paper asks how far a strong Restormer denoiser can improve without architectural changes, using expanded training, two-stage optimization, and test-time self-ensemble. Controlled evaluations identify stronger training as the dominant improvement source while quantifying self-ensemble and wrapper effects.
- Restormer is a strong public baseline whose remaining performance gap may reflect training data and inference strategy more than backbone limitations.
- ×8 geometric self-ensemble is evaluated as a complementary test-time strategy, while the retained TLC-style wrapper is systematically isolated.
- Expanded training data and two-stage optimization substantially improve Restormer-based Gaussian denoising without architectural modification.
- 29.89 dB PSNR and 0.87 SSIM ranked the submission 2nd on the final NTIRE 2026 leaderboard.
2. Related Work
Related work spans classical priors, CNN and transformer restorers, self-supervised denoising, diverse training corpora, and test-time enhancement. This paper builds on that landscape by disentangling training-data, self-ensemble, and wrapper contributions rather than introducing a new inference module.
- Classical priors and early learned models established denoising through non-local modeling, dictionary learning, reaction-diffusion, and deep restoration backbones.
- Residual CNNs, noise conditioning, feature attention, and progressive multi-stage refinement formed major deep denoising directions.
- Self-supervised methods showed that denoising can learn from noisy observations alone under suitable independence assumptions.
- Transformer restorers such as SwinIR, Uformer, and Restormer use long-range dependency modeling, with Restormer serving as a strong Gaussian-denoising baseline.
- Restoration research also explores alternative backbones and aggregation choices, including simple baselines, state-space models, and TLC-style global information aggregation.
- Test-time enhancement averages transformed predictions or applies local conversion, motivating explicit separation of self-ensemble, training, and wrapper effects.
3. Method
The method retains Restormer without architectural modification and improves it through expanded data, two-stage training, and inference-time geometric self-ensemble. The pipeline also retains a TLC-style wrapper and processes noisy RGB inputs through restoration, inverse transformation, and averaging.
- Framework overview: Restormer is adopted without architectural modification as the denoising backbone.Its hierarchical encoder-decoder uses multi-scale processing, skip connections, refinement, and a global residual path.
- Large-scale training dataset: The training recipe expands public image diversity with seven high-resolution image sources, including DIV2K, Flickr2K, OST, LSDIR, LIU4K-v2, NKUSR8K, and DIV8K.Ultra-high-resolution sources are cropped into approximately 2K sub-images to keep the training distribution manageable while preserving richer texture statistics.
- Two-stage optimization: Training starts from the public Restormer σ = 50 model and proceeds through two successive stages over progressively expanded corpora.Stage I uses DIV2K, Flickr2K, OST, and LSDIR; Stage II adds LIU4K-v2, NKUSR8K, and DIV8K.
- Two-stage optimization: Both training stages use AdamW and MSE loss, with the learning rate reduced from 1 × 10−4 in Stage I to 1 × 10−5 in Stage II.Optimization runs on 4× NVIDIA H200 GPUs.
- Test-time enhancement: At inference, ×8 geometric self-ensemble averages predictions over transformed inputs, while the TLC-style wrapper is retained for implementation consistency.Self-ensemble does not change model parameters or training cost but increases inference time by about a factor of eight.
- Final inference pipeline: The pipeline crops noisy RGB inputs to sizes divisible by eight, normalizes them, restores them with the wrapped backbone, and maps predictions back after inverse transformation and averaging.Clean reference images are used only for metric computation, not as model inputs.
4. Experiments
Experiments use a unified 100-image protocol to separate training, inference, checkpoint, and baseline effects. The expanded training recipe supplies the dominant improvement, while self-ensemble adds a small, consistent gain at substantial computational cost.
- Main result: 30.7622 dB PSNR and 0.8607 SSIM are achieved by the final wrapped Restormer with ×8 self-ensemble on the unified 100-image validation set.Average inference time is 8815.73 ms and peak memory usage is 36956 MB.
- Inference ablation: 0.0273 dB PSNR and 0.0004 SSIM are the small but stable gains from ×8 self-ensemble.The wrapped and unwrapped variants are numerically almost identical, so self-ensemble is the main inference-side contributor.
- Checkpoint evolution: 0.0042 dB PSNR and 0.0001 SSIM separate the intermediate checkpoint from the final model, indicating near-convergence before Stage II ended.The intermediate checkpoint reaches 30.7580 dB PSNR and 0.8606 SSIM, versus 30.7622 dB and 0.8607 for the final model.
- Baseline comparison: +3.3662 dB PSNR is the final model’s margin over the public Restormer σ = 50 pretrained baseline with ×8 self-ensemble.The single-pass margin is +3.3520 dB, while the SSIM gain is +0.0737 in both settings.
- Efficiency and visual discussion: 1053.60 ms to 8815.73 ms and 36190 MB to 36956 MB are the wrapped-path increases in inference time and peak memory from ×8 self-ensemble.The unwrapped path shows the same pattern, making the accuracy gain computation-heavy.
- Efficiency and visual discussion: 0.2463 dB and 0.1163 dB are self-ensemble gains in texture-rich or regularly structured regions, compared with 0.0328 dB and 0.0084 dB in smoother or ambiguous regions.This supports viewing self-ensemble as gradual robustness refinement rather than a uniformly large structural improvement.
- Efficiency and visual discussion: Single-pass inference already provides strong and stable restoration quality when throughput or deployment cost matters more than maximum quality.Keeping ×8 self-ensemble is reasonable when the strongest final restoration quality is the priority.
5. Conclusion
The paper strengthens a Restormer-based Gaussian denoiser through expanded data-centric training and ×8 self-ensemble rather than architectural redesign. Under the unified 100-image protocol, this approach substantially exceeds the public baseline, with self-ensemble providing a smaller additional gain.
- Conclusion: An expanded public training dataset, a two-stage recipe, and ×8 self-ensemble improve Restormer Gaussian color denoising without changing the backbone architecture.The final model reaches 30.7622 dB PSNR and 0.8607 SSIM, improving over the public Restormer σ = 50 pretrained baseline by up to 3.3662 dB PSNR.