PaperlayerView exact revision on arXiv

Source-linked AI summary

Designing and Training of A Dual CNN for Image Denoising

Chunwei Tian, Yong Xu, Wangmeng Zuo, Bo Du, Chia-Wen Lin, David Zhang

arXiv:2007.03951v1eess.IVcs.CV

TL;DR

Image denoising must recover fine details from complex noise, where plain CNNs and prior-based methods have important limitations. DudeNet addresses this with dual sparse feature extraction, global-local enhancement, compression, and reconstruction, and the reported experiments show strong denoising quality and computational efficiency.

  • Problem

    Plain CNNs may not recover fine details in complex real noisy images, while prior-based methods face manually set parameters, complex optimization, or limited priors.

  • Method

    DudeNet combines two feature-extraction sub-networks with a sparse mechanism, global-local feature enhancement, compression blocks, and reconstruction.

  • Results

    Extensive experiments report that DudeNet outperforms existing denoising methods while providing high visual quality and computation efficiency.

  • Takeaways & Limitations

    DudeNet is reported as suitable for complex noisy images, including real-world corrupted images and blind noise.

Abstract

from arXiv · show

Deep convolutional neural networks (CNNs) for image denoising have recently attracted increasing research interest. However, plain networks cannot recover fine details for a complex task, such as real noisy images. In this paper, we propsoed a Dual denoising Network (DudeNet) to recover a clean image. Specifically, DudeNet consists of four modules: a feature extraction block, an enhancement block, a compression block, and a reconstruction block. The feature extraction block with a sparse machanism extracts global and local features via two sub-networks. The enhancement block gathers and fuses the global and local features to provide complementary information for the latter network. The compression block refines the extracted information and compresses the network. Finally, the reconstruction block is utilized to reconstruct a denoised image. The DudeNet has the following advantages: (1) The dual networks with a parse mechanism can extract complementary features to enhance the generalized ability of denoiser. (2) Fusing global and local features can extract salient features to recover fine details for complex noisy images. (3) A Small-size filter is used to reduce the complexity of denoiser. Extensive experiments demonstrate the superiority of DudeNet over existing current state-of-the-art denoising methods.

I. INTRODUCTION

Image denoising seeks to recover clean images from corrupted observations, but prior-based and plain CNN methods face parameter, prior, and detail-recovery limitations on complex noise. DudeNet addresses these challenges with dual feature extraction, global-local fusion, compression, and reconstruction.

  • Motivation: Image denoising recovers a latent clean image x from a corrupted image y modeled as y = x + v.Here, v represents additive white Gaussian noise with standard deviation σ.
  • Limitations of Existing Methods: Prior-based denoisers can perform well but require manually set parameters, complex optimization, or restrictive image priors.Examples include WNNM, BM3D, CSF, and TNRD.
  • Limitations of Existing Methods: Plain multipurpose CNN structures may fail to recover clean-image details for real-world corrupted images and blind noise.DnCNN uses residual learning and batch renormalization within a single model for multiple applications.
  • Proposed Method: DudeNet uses a sparse dual-network feature extraction block, an enhancement block for global-local fusion, a compression block, and a reconstruction block.The reconstruction block combines residual and noisy images to reconstruct the latent clean image.
  • Contributions: The dual sparse networks extract diverse complementary features, while global-local fusion targets salient features and fine details in complex denoising tasks.The stated target includes real-world corrupted images and blind noise.
  • Contributions: DudeNet uses small-sized filters to reduce denoiser complexity and improve its suitability for complex noisy images.The introduction also identifies reduced complexity as a principal advantage.

B. Deep CNNs based modules or blocks for image denoising

CNN denoising architectures increasingly use modular feature fusion and compression to balance restoration performance with efficiency. DudeNet combines these ideas through dual feature extraction, enhancement, compression, and reconstruction blocks.

  • Modular CNN Design: CNN modules or blocks are used in low-level vision to improve performance or accelerate network processing.The cited examples span image, video, text, super-resolution, and denoising applications.
  • Performance-Oriented Blocks: Feature-fusion methods enhance CNN representation by integrating multiple features through concatenation or cascading residual structures.DBF uses feature extraction, integration, and reconstruction blocks, while CARN repeatedly cascades residual blocks.
  • Efficiency-Oriented Blocks: Compression-oriented networks reduce convolutional size, clip low-contribution convolutions, or distill features to improve processing speed.Examples include LFFN, AWSRN, and IDN.
  • DudeNet Architecture: DudeNet is proposed as a block-based network intended to narrow the difference between denoising efficiency and performance.Its architecture is detailed through feature extraction, enhancement, compression, and reconstruction blocks.
  • Feature Extraction: DudeNet extracts two diverse feature maps from a noisy input using two feature-extraction sub-networks, with the first containing a sparse mechanism.The network input is denoted Y and its output X.
  • Enhancement and Compression: The enhancement block fuses features from the two sub-networks, while compression blocks refine extracted features and reduce computational cost.CB2 is positioned between EB1 and EB2, and EB2 obtains complementary information.
  • Reconstruction: The reconstruction block constructs the latent clean image from residual features using the noisy input and a residual operation.The cited description identifies ‘−’ as the residual operation.

B. Feature Extraction Block (FEB)

The feature extraction block uses two sub-networks to obtain diverse visual features, with a sparse mechanism in FEBnet1 that combines dilated and common convolutions.

  • FEBnet1: FEB extracts reliable visual features through a first network containing Conv+BN+ReLU, dilated Conv+BN+ReLU, and Conv modules.All convolutional layers use 3 × 3 filters; layers 2–12 comprise the sparse mechanism.
  • Sparse Mechanism: The sparse mechanism places dilation-factor-2 convolutions at layers 2, 5, 9, and 12 to capture rich contextual information.Other layers use common convolutions to extract relatively fewer features than the dilated layers.
  • FEBnet1: FEBnet1 is represented as a composition of Conv+BN+ReLU, the sparse mechanism, three Conv+BN+ReLU layers, and one 3 × 3 convolution.The functions are denoted CBR1, S, CBR3, and C, respectively.
  • FEBnet2: FEBnet2 uses Conv+ReLU modules and a 1 × 1 convolutional compression block, with fifteen Conv+ReLU operations represented by CR15.Its first and final layers use c × 3 × 3 × 64 and 64 × 1 × 1 × 64 configurations, respectively.

C. Enhancement Block (EB)

The enhancement block boosts learning for unknown or complex noise by combining features from DudeNet’s two sub-networks. It uses EB1 and EB2 between feature extraction and compression to fuse complementary information.

  • C. Enhancement Block (EB): EB enhances learning for corrupted images and blind noise through two network components.It is positioned between FEB and CB3.
  • C. Enhancement Block (EB): EB1 concatenates features from the first and second networks, then applies batch normalization and ReLU.The normalization addresses differences between feature distributions produced by dilated convolutions and CB1.
  • C. Enhancement Block (EB): EB2 concatenates the input image with CB2’s output to produce important information for CB3.This provides additional information for the later compression stage.
  • C. Enhancement Block (EB): DudeNet’s architecture comprises feature extraction, enhancement, compression, and reconstruction blocks.The enhancement block is the second of these four parts.

D. Compression Block (CB) and Reconstruction Block (RB)

The compression block distills features with 1 × 1 convolutions to reduce dimensionality and computational cost, while the reconstruction block uses residual information to form the latent clean image.

  • D. Compression Block (CB) and Reconstruction Block (RB): CB contains CB1, CB2, and CB3, which distill features and reduce computational cost through 1 × 1 convolutions.CB1, CB2, and CB3 are placed at successive points between FEB, EB, and RB.
  • D. Compression Block (CB) and Reconstruction Block (RB): CB3 outputs the residual image, treated as a noise mapping, which the reconstruction block uses to construct the predicted clean image.RB reconstructs the latent clean version from the residual representation and the noisy input.

E. Loss Function

DudeNet trains its residual prediction using mean squared error between predicted and ground-truth residual images. Training uses paired synthetic and real noisy-image data with augmentation for synthetic samples.

  • E. Loss Function: The MSE loss measures the discrepancy between predicted residual R(Yj) and ground-truth residual Yj − Xj.Xj denotes the jth clean image.
  • E. Loss Function: The loss function restores the latent clean image through optimization with Adam over N noisy-clean image pairs.The model parameters are denoted by θ.
  • E. Loss Function: Synthetic training images are 180 × 180 gray-level and color-level images selected from 400 images and augmented by interpolation and geometric manipulations.The manipulations include rotations and horizontal flips.
  • E. Loss Function: Real-image training uses 100 JPEG-compressed 512 × 512 images captured by five digital devices across sensor settings from 800 to 25,600.The compression makes these real noisy images more challenging for denoising.

B. Testing datasets

DudeNet is evaluated on five public datasets covering gray, color, synthetic, and real-world corrupted images. Its design analysis examines receptive fields, sparsity, efficiency, and denoising performance using benchmark and runtime comparisons.

  • B. Testing datasets: The five test datasets are BSD68, Set12, CBSD68, Kodak24, and CC, spanning gray, color, and real-world corrupted images.CC contains 15 corrupted 512 × 512 real-world images.
  • B. Testing datasets: DudeNet has depth 18 and is trained on 41 × 41 patches to accelerate training.The patch size is selected in relation to the network’s receptive field.
  • B. Testing datasets: Patch size must exceed the receptive field size; otherwise padding requirements can degrade denoising performance.The first and second networks have receptive fields of 41 × 41 and 32 × 32, respectively.
  • B. Testing datasets: Sparse mechanism analysis reports that FS uses a 43 × 43 receptive field while achieving an effect comparable to 21 layers with 3 × 3 kernels.The comparison is presented as evidence for sparse mechanism efficiency and performance.
  • B. Testing datasets: Removing the sparse mechanism reduces the effectiveness of DudeNet, while FEBnet2 further consolidates FEBnet1 to improve denoising performance.These comparisons are reported from Table I ablations.

3) Design, analysis and effectiveness of EB:

The enhancement block improves DudeNet by combining complementary information from two subnetworks and fusing local features with original image information.

  • EB1 gathers two subnetworks to improve expressive power while maintaining the same network depth.
  • EB2 fuses local features with original image information to supplement both the first phase and the later network.
  • Comparisons evaluate the contributions of EB1, EB2, batch normalization, and related block combinations within DudeNet.

4) Design, analysis and effectiveness of CB:

The compression block reduces redundant features and computational demands while preserving competitive denoising performance against deeper or larger-filter networks.

  • CB uses a 1×1 convolution to discount redundant feature information and improve efficiency.
  • DudeNet with CB2 achieves higher PSNR than DudeNet without CB2 and EB2.
  • DudeNet remains competitive with a 3 × 3-filter version in performance, running time, and computational cost.
  • DudeNet is superior to DnCNN and Two DnCNNs for denoising, while using fewer parameters and Gflops than Two DnCNNs.
  • The experiments compare PSNR and run-time across gray and color synthetic, blind, and real noisy-image applications.

1) Gray and color synthetic noisy images:

Across gray and color synthetic-noise benchmarks, DudeNet generally achieves strong or best denoising performance, while DudeNet-B remains competitive for blind denoising.

  • DudeNet outperforms several state-of-the-art gray-image denoisers, including DnCNN and FFDNet, on BSD68.
  • DudeNet-B improves over DnCNN by 0.02dB when σ = 50 on BSD68.
  • DudeNet obtains the best gray-image denoising results across noise levels 15, 25, and 50.
  • DudeNet and DudeNet-B are very competitive with other methods on color synthetic images from CBSD68 and Kodak24 across noise levels 15, 25, 35, 50, 75, and [0, 55].
  • Visual comparisons report DudeNet at 32.29dB on Set12, 28.67dB on BSD68, 31.13dB on CBSD68, and 31.37dB on Kodak24.
  • DudeNet-B is trained over noise levels from 0 to 55 and is competitive with FFDNet and IRCNN for gray and color denoising.

3) Real noisy images:

On real noisy images, DudeNet achieves strong denoising results and competitive running times, supporting its use for complex noisy-image tasks.

  • DudeNet improves PSNR by 1.86dB over DnCNN on real noisy images.
  • The experiments include gray and color synthetic noise, blind denoising, and real noisy images, with running time compared across methods.
  • DudeNet is more competitive in running time than the state-of-the-art denoisers RED30 and MemNet.
  • The paper attributes DudeNet’s generalized ability on complex tasks to dual networks with a sparse mechanism.
  • Fusing global and local features is presented as a way to recover fine details in complex denoising tasks.
  • Small filter sizes and compression blocks reduce computational cost and memory consumption while retaining high visual quality and efficiency.
Loading 2007.03951v1…