PaperlayerView exact revision on arXiv

Source-linked AI summary

GRDN:Grouped Residual Dense Network for Real Image Denoising and GAN-based Real-world Noise Modeling

Dong-Wook Kim, Jae Ryun Chung, Seung-Won Jung

arXiv:1905.11172v1eess.IVcs.CV

TL;DR

Real-world image denoising remains difficult because synthetic-noise training may not transfer to real images, and real-image datasets are limited. The paper proposes GRDN alongside GAN-based noise modeling and reports the top PSNR and SSIM scores in NTIRE2019 Track 2:sRGB.

  • Problem

    CNNs trained on synthetic noise may perform poorly on real-world images, while available real-image denoising datasets may not suffice for large networks.

    Source: §1 · p. 1

  • Method

    The paper proposes GRDN, built from cascaded grouped residual dense blocks, and GAN-based real-world noise modeling with image and camera-parameter conditioning.

    Source: §1 · p. 1, §2.2 · p. 2, §3.1 · p. 3

  • Results

    39.93 dB PSNR and 0.9736 SSIM were the best reported scores in the NTIRE2019 Real Image Denoising Challenge Track 2:sRGB.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The proposed approach achieves top reported scores in the NTIRE2019 real-image denoising challenge setting.

    Source: §1 · p. 2

Abstract

from arXiv · show

Recent research on image denoising has progressed with the development of deep learning architectures, especially convolutional neural networks. However, real-world image denoising is still very challenging because it is not possible to obtain ideal pairs of ground-truth images and real-world noisy images. Owing to the recent release of benchmark datasets, the interest of the image denoising community is now moving toward the real-world denoising problem. In this paper, we propose a grouped residual dense network (GRDN), which is an extended and generalized architecture of the state-of-the-art residual dense network (RDN). The core part of RDN is defined as grouped residual dense block (GRDB) and used as a building module of GRDN. We experimentally show that the image denoising performance can be significantly improved by cascading GRDBs. In addition to the network architecture design, we also develop a new generative adversarial network-based real-world noise modeling method. We demonstrate the superiority of the proposed methods by achieving the highest score in terms of both the peak signal-to-noise ratio and the structural similarity in the NTIRE2019 Real Image Denoising Challenge - Track 2:sRGB.

1. Introduction

Synthetic-noise training does not reliably transfer to real-world denoising, while real-image datasets can be difficult to expand. The paper proposes GRDN and GAN-based noise modeling, reporting top NTIRE2019 Track 2:sRGB scores.

1. Introduction

  • Synthetic noise makes noisy and clean training pairs easy to construct, so many learning-based methods focus on Gaussian denoising and network architecture.The passage contrasts this setup with the real-world denoising challenge.Source: §1 · p. 1
  • CNNs trained on synthetic noisy images can perform poorly on real-world images, sometimes falling below BM3D.The passage attributes the mismatch to the gap between synthetic and real-world noise.Source: §1 · p. 1
  • The paper adopts real-image training data and GAN-based augmentation to enlarge the available dataset.It notes that SIDD images may be insufficient for a large network and that generating ground truth from real noisy images can require substantial know-how.Source: §1 · p. 1
  • The proposed GRDN builds on RDN by using grouped residual dense blocks (GRDBs) as components.The introduction describes cascading GRDBs with attention modules.Source: §1 · pp. 1–2
  • 39.93 dB PSNR and 0.9736 SSIM were the best reported scores in the NTIRE2019 Real Image Denoising Challenge Track 2:sRGB.The paper reports these as its challenge results.Source: §1 · p. 2

2. Related Works

Related work frames real-world denoising as a dataset and architecture challenge, alongside GAN-based noise augmentation. The paper draws on restoration-network designs and modifies prior noise simulation with conditioning and a relativistic discriminator.

2.1. Image Restoration

  • Real-world denoising datasets such as SIDD and DND emerged as learning methods moved beyond classic Gaussian denoising.The passage reports recent learning methods outperforming classic methods on real-world denoising.Source: §2.1 · p. 2
  • Restoration networks including RDN and RCAN are the closest architectural relatives described.The paper examines architectures developed across image restoration tasks, especially super-resolution.Source: §2.1 · p. 2
  • RCAN's residual-in-residual principle and CBAM inform the proposed architecture, which uses modified RDN components in a cascade.The passage notes prior evidence for residual-in-residual gains and CBAM's channel-then-spatial attention.Source: §2.1 · p. 2

2.2. GAN

  • GAN-based augmentation can generate realistic noisy and clean pairs from real noise signals, improving real-world denoising performance.The prior method trains a noise generator and then uses it to synthesize noisy images from clean ground truth.Source: §2.2 · p. 2
  • The paper extends prior noise simulation with noise-free image, ISO, and shutter-speed conditioning, plus a discriminator that compares real and fake data.The relativistic GAN approach is described as determining which data are more realistic.Source: §2.2 · pp. 2–3

3. Proposed Methods

GRDN organizes residual-dense features into cascaded grouped residual dense blocks, with multilevel residual connections, down/up-sampling, and CBAM. The paper also proposes cERGAN, a conditioned GAN for modeling real-world noise.

3.1. Image Denoising Network

  • GRDN cascades GRDBs to fuse RDB features in multiple stages, with a global residual connection and CBAM placed between upconvolution and the final convolution.The network also uses down-sampling and up-sampling layers; four GRDBs each contain four RDBs.Source: §3.1 · p. 3

3.2. GAN-based Real-world Noise Modeling

  • cERGAN conditions noise generation on a clean image patch, ISO, shutter speed, and smartphone model to improve realism, controllability, and variety.The clean patch targets signal-dependent noise; the camera-related parameters provide additional controllability and variety.Source: §3.2 · p. 3
  • Spectral normalization before batch normalization and residual scaling were empirically found useful for training the cERGAN generator.The generator uses metadata from SIDD to train with its conditioning signals.Source: §3.2 · p. 4
  • The cERGAN discriminator uses a relativistic GAN design that compares real and fake data, and takes both data alongside the conditioning signal as input.The conditioning signal comprises replicated smartphone-code, ISO, shutter-speed, and noise-free image patches.Source: §3.2 · p. 4

G and LcERGAN

The discriminator explicitly compares real and fake data, while the generator is trained to fool it. The supplied passages describe these objectives but do not provide a complete readable loss expression.

G and LcERGAN

  • With real data as the second input and fake data as the third, the discriminator is trained to output a value near 1.This encodes the judgment that the real input is more realistic than the fake input.Source: G and LcERGAN · p. 4
  • Switching the real and fake inputs trains the discriminator to output a value near 0, indicating that the third input is less realistic.Source: G and LcERGAN · p. 4
  • The generator is trained to fool the discriminator, and the authors state that explicit real–fake comparison could simulate more realistic real-world noise.Source: G and LcERGAN · p. 4

4. Experiments

The experiments section identifies a comparison of image denoising models and reports the implementation environment.

4. Experiments

  • The models were implemented using PyTorch on an Intel i7-8700 @3.20GHz system with 32GB RAM and an NVIDIA Titan XP.Source: §4 · p. 4
  • Table 1 is a comparison of image denoising models.Source: Table 1 · p. 5
  • The supplied experiment passage identifies the table's scope but gives no table values to report.Source: Table 1 · p. 5

4.1. Datasets

The experiments use NTIRE 2019 challenge training and validation images, a subset of SIDD, and report performance on the validation set because test ground truth is unavailable.

4.1. Datasets

  • The study uses 320 high-resolution images for training and 1,280 cropped 256×256 blocks for validation from the NTIRE 2019 challenge dataset, a SIDD subset.Source: §4.1 · p. 5
  • The images come from five smartphone cameras: Apple iPhone 7, Google Pixel, Samsung Galaxy S6 Edge, Motorola Nexus 6, and LG G4.Source: §4.1 · p. 5
  • Performance is reported on the validation set because test-set ground-truth images are not publicly available.Training patches exclude the first and last eight rows and columns due to nonmarginal ground-truth border degradations; scaling, flipping, and rotation augmentation were not applied.Source: §4.1 · p. 5

4.2. Image Denoising

The experiments compare GRDN with RDN and test patch size, CBAM placement, and network width and depth. Larger patches improved PSNR, while CBAM’s effect depended on patch size.

4.2.1 Implementation details

  • Training used ChDB images augmented with synthetic noise; batches contained 16 image-patch pairs and optimization used Adam with L1 loss.The learning rate began at 10^-4 and was halved every 2 × 10^5 iterations; training took approximately five days.Source: §4.2.1 · p. 5

4.2.2 Comparison to RDN

  • GRDN’s PSNR was 0.04 dB higher than re-trained RDN with a similar parameter count.The models used the same number of RDBs; GRDN had 22M parameters and RDN had 21.9M.Source: §4.2.2 · p. 5

4.2.3 Experiments on patch size

  • Increasing patch size to 96 × 96 improved performance by 0.22 dB.The larger patch was the largest possible in the experimental environment, given the images’ resolution of more than 12M pixels.Source: §4.2.3 · p. 5

4.2.4 Experiments on CBAM module

  • CBAM increased PSNR by 0.05 dB at the smaller patch size but decreased it by 0.01 dB after patch size increased.The authors found the best CBAM position was after the up-sampling layer.Source: §4.2.4 · p. 5

4.2.5 Hyper parameter adjustment

  • A less-deep, wider network performed 0.02 dB better and was the best model under the hardware constraints.The comparison varied the numbers of filters and GRDBs.Source: §4.2.5 · p. 6

4.3. Real-world Noise Modeling

The noise-modeling experiments test cERGAN-generated noise and dataset augmentation. The combined dataset produced a slightly higher validation PSNR and a more stable increase than the original dataset.

cERGAN noise modeling

  • cERGAN-generated noise patches were reported to be close to real-world noise patches.The generator and discriminator were trained on 48 × 48 patches cropped from ChDB noisy images and their ground truths.Source: §4.3 · p. 6

cERGAN data evaluation

  • Training on cERGAN-generated noisy images alone yielded 38.63 dB PSNR, below the 39.62 dB obtained using only the provided ChDB dataset.The tested denoising network corresponded to the fourth column of Table 1.Source: §4.3 · p. 6

Statistical noise evaluation

  • Training the denoising network only on ChDB ground-truth images with statistically modeled real-world noise from [10] resulted in 36.17 dB; the authors say their GAN-based noise modeling performs better.Source: §4.3 · p. 6

Dataset augmentation

  • 39.64 dB PSNR with augmented data was slightly higher than 39.62 dB with the original dataset, and its PSNR increased more stably.The tested mix used 90% ChDB, 5% data simulated using [10], and 5% data simulated using cERGAN; only this configuration was tested.Source: §4.3 · p. 6, Figure 6 · p. 6

5. NTIRE2019 Image Denoising Challenge

The submitted model used augmented ChDB and a skip connection every two GRDBs. It ranked first in both PSNR and SSIM in the NTIRE2019 challenge.

Submitted model

  • The submitted network was the seventh-column model, with skip connections added every two GRDBs and training on augmented ChDB.The augmented data followed the technique described in Section 4.3.Source: §5 · pp. 6–7

Qualitative comparison

  • Figure 7 compares ground truth and noisy images with denoised outputs from BM3D, RDN, and the proposed network, showing PSNR and SSIM beneath noisy and denoised images.The caption recommends viewing the results electronically.Source: Figure 7 · p. 7

Challenge results

  • The model ranked first in PSNR and SSIM, outperforming the second-ranked method by 0.05 dB.Table 2 reports performance on the NTIRE 2019 Real Image Denoising Challenge Track 2:sRGB test dataset.Source: §5 · p. 7, Table 2 · p. 7

6. Conclusion

The conclusion highlights a real-world denoising network with extensive hierarchical residual connections and an improved GAN-based noise-modeling method. It also notes limits in evaluation scope and quantitative support for the noise model.

6. Conclusion

  • Extensive, hierarchical residual connections accompany state-of-the-art performance for the proposed real-world denoising network.Source: §6 · pp. 7–8
  • The authors also developed an improved GAN-based method for modeling real-world noise.Source: §6 · p. 8
  • Evaluation covers only real-world denoising; noise-modeling effectiveness is not fully quantitatively justified, and a more elaborate design is needed.Source: §6 · p. 8
  • The authors plan to apply the network to other restoration tasks and extend noise modeling to blur, aliasing, and haze.Source: §6 · p. 8

Related papers

Loading 1905.11172v1…