Source-linked AI summary
Real-world Noisy Image Denoising: A New Benchmark
Jun Xu, Hui Li, Zhetong Liang, David Zhang, Lei Zhang
TL;DR
Real-world denoising is challenged by signal-dependent noise, limited ground truth, and inadequate existing benchmarks. The paper constructs a comprehensive dataset from repeated real-scene captures, evaluates denoising methods across new and prior datasets, and finds that the new benchmark is more challenging while real-world-specific methods perform strongly on prior datasets.
Problem
Real-world noise differs from AWGN and is difficult to evaluate because noisy photographs lack corresponding ground-truth images.
Method
The paper constructs a benchmark using repeated captures of unchanged scenes, filtering misalignment and luminance outliers before evaluation.
Results
Real-world-specific methods outperform competing methods on prior datasets, while their advantage over previous methods is only around 0.4dB on the new dataset.
Takeaways & Limitations
The new dataset provides a more comprehensive and challenging benchmark for developing real-world image denoising methods.
Abstract
from arXiv · showhide
Most of previous image denoising methods focus on additive white Gaussian noise (AWGN). However,the real-world noisy image denoising problem with the advancing of the computer vision techiniques. In order to promote the study on this problem while implementing the concurrent real-world image denoising datasets, we construct a new benchmark dataset which contains comprehensive real-world noisy images of different natural scenes. These images are captured by different cameras under different camera settings. We evaluate the different denoising methods on our new dataset as well as previous datasets. Extensive experimental results demonstrate that the recently proposed methods designed specifically for realistic noise removal based on sparse or low rank theories achieve better denoising performance and are more robust than other competing methods, and the newly proposed dataset is more challenging. The constructed dataset of real photographs is publicly available at \url{https://github.com/csjunxu/PolyUDataset} for researchers to investigate new real-world image denoising methods. We will add more analysis on the noise statistics in the real photographs of our new dataset in the next version of this article.
I. INTRODUCTION
Real-world image noise is more complex than AWGN, making denoising and objective evaluation difficult. The paper addresses this gap by constructing a dataset with approximate ground-truth images from repeated captures.
- Real-world noise is signal dependent, lacks an explicit distribution, and becomes more complex through camera imaging pipelines.
- Objective evaluation is difficult because captured real-world noisy images lack corresponding ground-truth images.
- Existing BIQA methods are mainly developed for datasets whose image properties differ from real-world noisy photographs.
- Prior real-world noise datasets have limitations including restricted scenes, imperfect intensity modeling, and unremoved low-frequency bias.
- The dataset repeatedly captures an unchanged scene, using the mean of many images as an approximate ground truth.Repeated sampling averages random positive and negative noise deviations, substantially alleviating noise.
II. EXISTING DATASETS
Existing real-world denoising datasets use different capture and ground-truth strategies, but each has limitations in scope or realism. These limitations motivate broader benchmarking data.
- RENOIR was an early real-world noisy-image dataset with ground-truth noisefree images, using Canon and Xiaomi cameras across static scenes.
- RENOIR uses different ISO values and static-scene exposures, but its post-processing leaves spatial misalignment, imperfect heteroscedastic-noise modeling, and low-frequency bias.
- Dataset captures 500 JPEG images for each of 11 static scenes and uses their mean image as approximate ground truth.
- Dataset estimates noise parameters with Tobit regression from two images and benchmarks 50 image pairs captured at different ISO settings and shutter speeds.
A. Motivation
Earlier datasets are limited in camera brands, settings, and scene diversity, while camera settings strongly affect illumination and noise. These constraints motivate more comprehensive capture protocols.
- Motivation: Existing datasets have limitations in camera brands, camera settings, and captured scenes.
- Motivation: Previous datasets use restricted camera collections, including two brands in RENOIR, two in Dataset, and three cameras in DND.
- Motivation: Camera settings vary substantially across prior datasets, especially in ISO and shutter-speed configurations.
- Motivation: In general, faster shutter speeds and smaller ISO values or apertures produce darker captured images when other settings are fixed.
- Motivation: Higher ISO increases noise, so comprehensive evaluation requires varying ISO while controlling shutter speed and aperture appropriately.
- Motivation: Figure 1 compares illumination changes caused by different ISO, shutter-speed, and aperture settings on Sony A7 II captures.
B. The Dataset Construction Process
The dataset construction process broadens camera, setting, and scene coverage while repeatedly capturing static scenes and filtering misaligned or luminance-outlier images.
- B. The Dataset Construction Process: The dataset targets more camera brands, carefully designed settings, realistic scenes, and both raw and sRGB data.
- B. The Dataset Construction Process: Five cameras from three brands capture real-world noisy images, making camera-brand coverage broader than previous datasets.
- B. The Dataset Construction Process: Each scene is captured at six ISO settings, with shutter speed and aperture adjusted to avoid under- or over-exposure.
- B. The Dataset Construction Process: The collection covers indoor normal-lighting, dark-lighting, and outdoor scenes containing varied real objects.
- B. The Dataset Construction Process: A tripod and automated shutter release are used for repeated static-scene capture, helping nearly avoid acquisition misalignment.
- B. The Dataset Construction Process: Images with displacement or inconsistent luminance are removed through subjective alignment checks and sampled-pixel luminance filtering.Mean luminance is computed from 10,000 uniformly sampled pixels, and extreme values are removed toward the median.
C. Summary of the Dataset
The dataset covers diverse indoor scenes and objects, using cropped regions with corresponding low-noise “ground truth” images to support denoising evaluation.
- The dataset contains images from 40 different scenes with diverse contents and objects.
- Researchers crop 100 regions of size 512 × 512 from the captured scenes for evaluating image denoising methods.
- The corresponding “ground truth” images contain substantially less noise and have better visual quality than the noisy images.
A. Benchmark Datasets
The benchmark compares denoising methods across two prior datasets and the new dataset, which expands scene coverage and provides accessible evaluation crops.
- Methods are evaluated on Datasets 1, 2, and 3, while Dataset 2 results are obtained through the authors’ Project Website using PSNR and SSIM.
- Dataset 1 contains 11 static scenes and uses 15 cropped regions of size 512 × 512 for evaluation.
- Dataset 2, the Darmstadt Noise Dataset, contains 50 image pairs and 1,000 testing crops, but its “ground truth” images are not open access.
- Dataset 3 contains 40 static scenes captured by five cameras, with 100 evaluation regions of size 512 × 512.
B. Comparison Methods
The comparison includes classical, learned, blind, commercial, and realistic-noise-specific denoising methods, with noise-level inputs or retraining required for several methods.
- The evaluation includes AWGN-oriented methods, blind denoising, commercial software, and methods designed for realistic noise removal.
- CBM3D assumes AWGN, while EPLL, PGPD, NCSR, WNNM, MLP, CSF, TNRD, and DnCNN are applied to grayscale channels of realistic color images.
- CBM3D requires a color-image noise standard deviation, and several competing methods require noise levels for each color channel.
- MLP, CSF, and TNRD are retrained at different noise levels, while DnCNN handles noise levels ranging from 0 to 55.
C. Results and Discussion
Across prior and new real-world denoising datasets, methods designed for realistic noise generally outperform AWGN-oriented methods, while the new dataset narrows that advantage and is more challenging. Noise levels also increase with ISO and vary across color channels.
- TWSC achieves higher PSNR and SSIM and better visual quality than other methods on the Canon 5D Mark 3 example.
- On Dataset 3, AWGN methods are less effective, discriminative methods are only slightly better than traditional methods, and real-world methods perform better.
- Mean noise levels increase with ISO, and the Green channel may have lower noise than the Red and Blue channels.
- Methods designed for grayscale images can generate artifacts because they process RGB channels independently despite channel- and patch-dependent noise statistics.
- Discriminative methods depend on training noise statistics and image resolution, making them less reliable when deployment conditions differ.
- On the new dataset, TWSC exceeds TNRD by around 0.4dB, compared with over 1.2dB on earlier datasets.
- The new dataset is more challenging because it covers more comprehensive scene contents and camera settings.
V. CONCLUSION
The paper introduces a benchmark of real-world noisy photographs with corresponding ground-truth approximations and evaluates denoising methods across datasets. Its experiments find the new dataset more challenging and the proposed methods more robust than competing methods.
- The dataset captures each scene 500 times and averages aligned, luminance-consistent images to approximate ground truth.
- It contains real-world noisy images from different natural scenes, cameras, and camera settings.
- The evaluation uses 512×512 cropped regions from the new dataset and compares methods on both new and previous datasets.
- Experiments report that the proposed methods are more robust than competing methods and that the new dataset is more challenging.