Source-linked AI summary
Deep Learning on Image Denoising: An overview
Chunwei Tian, Lunke Fei, Wenxian Zheng, Yong Xu, Wangmeng Zuo, Chia-Wen Lin
TL;DR
Image denoising involves differing deep learning approaches for regular and complex noise, while real noisy data often lacks clean ground truth. This survey classifies methods across four noise settings, analyzes their principles, compares benchmark performance, and discusses unresolved challenges and research directions. Its conclusion emphasizes broad comparisons of frameworks, methods, results, efficiency, and visual effects, while identifying data and evaluation limitations.
Problem
Deep learning denoising methods differ across noise types, while real noisy images lack ground-truth clean images needed for feature learning.
Method
The paper surveys and classifies deep learning denoising methods by additive white, blind, real, and hybrid noise, then analyzes principles and benchmark comparisons.
Results
The survey compares denoising results, efficiency, and visual effects across benchmark datasets and noise types.
Takeaways & Limitations
The paper identifies method categories, cross-category comparisons, and potential research directions for deep learning image denoising.
Takeaways & Limitations
Real noisy images are difficult to capture with adequate training samples, and PSNR and SSIM may not accurately evaluate perceptual quality.
Abstract
from arXiv · showhide
Deep learning techniques have received much attention in the area of image denoising. However, there are substantial differences in the various types of deep learning methods dealing with image denoising. Specifically, discriminative learning based on deep learning can ably address the issue of Gaussian noise. Optimization models based on deep learning are effective in estimating the real noise. However, there has thus far been little related research to summarize the different deep learning techniques for image denoising. In this paper, we offer a comparative study of deep techniques in image denoising. We first classify the deep convolutional neural networks (CNNs) for additive white noisy images; the deep CNNs for real noisy images; the deep CNNs for blind denoising and the deep CNNs for hybrid noisy images, which represents the combination of noisy, blurred and low-resolution images. Then, we analyze the motivations and principles of the different types of deep learning methods. Next, we compare the state-of-the-art methods on public denoising datasets in terms of quantitative and qualitative analysis. Finally, we point out some potential challenges and directions of future research.
1. Introduction
The introduction motivates deep learning for image denoising by describing the need to recover latent clean images from degraded observations and the limitations of earlier approaches. It presents this overview as a classification, analysis, evaluation, and future-directions study spanning several noise types.
- Motivation: Image denoising recovers latent observations from degraded images affected by lighting, transmission, compression, and noise corruption.The paper frames denoising as essential for recovering the underlying clean image.
- Motivation: Earlier denoising methods required test-phase optimization, manually set parameters, or task-specific models.These drawbacks motivated the adoption of more flexible deep learning architectures.
- Method evolution: Early neural denoising work addressed known blur and additive noise, while later feedforward and convolutional networks targeted efficiency, flexibility, and complex noise.The historical discussion includes tradeoffs between denoising efficiency and performance and later CNN-based developments.
- Scope and contributions: The overview covers more than 200 recent papers on deep learning for image denoising.It surveys the effects and solutions of deep learning methods across the field.
- Scope and contributions: It categorizes methods for additive white, real, blind, and hybrid noisy images, with blind noise denoting unknown noise types.Hybrid noisy images combine noisy, blurred, and low-resolution conditions in the paper’s stated scope.
- Scope and contributions: The overview analyzes method motivations and principles, compares quantitative and qualitative denoising performance, and identifies challenges and future directions.The organization includes basic frameworks, categories, performance comparison, challenges, and potential research directions.
2. Fundamental frameworks of deep learning methods for image denoising
This section introduces the learning frameworks, network architectures, and computational tools underlying deep-learning methods for image denoising. It covers supervised, unsupervised, and semi-supervised learning, neural-network fundamentals, CNN architectures, and supporting software and hardware.
- Machine learning methods: Supervised learning trains denoising models from paired clean and noisy images, with the noise level supplied as a model input.The formulation represents the clean image as a function of the noisy image, learned parameters, and given noise level.
- Machine learning methods: Unsupervised learning discovers patterns without label matching, while semi-supervised learning uses labeled and unlabeled samples together.Examples include high-resolution recovery from unpaired low-resolution images and sinogram restoration using paired and unlabeled data.
- Neural networks: Neural networks transform inputs through weighted connections, biases, activation functions, and multiple layers; networks with more than three layers are called deep neural networks.A multilayer perceptron uses hidden layers between input and output layers, with nested transformations across layers.
- CNNs: CNNs extract image features through convolutional kernels and became practical through advances including GPUs, ReLU, dropout, and data augmentation.LeNet, AlexNet, GoogLeNet, and ResNet illustrate successive architectural developments involving convergence, computational cost, width, and residual learning.
- Hardware and software: The surveyed deep-learning implementations use software packages including Caffe, Theano, Matconvnet, TensorFlow, and Keras, with CPU or GPU support varying by package.These tools provide interfaces for applications such as image denoising, classification, super-resolution, and related vision tasks.
3. Deep learning techniques in image denoising
The overview organizes deep learning denoising methods by noise type and by combinations of CNNs with feature extraction, optimization, or prior knowledge. It also discusses architectures for real, blind, hybrid, and video denoising.
- 3.1. Deep learning techniques for additive white noisy images: AWNI denoising methods include CNN/NN models, CNNs combined with feature extraction, and optimization-based CNN/NN approaches.AWNIs include Gaussian, Poisson, salt-and-pepper, and multiplicative noise.
- 3.1.2. CNN/NN and common feature extraction methods: Feature-extraction methods combine CNNs with transformations, kernels, dimensionality reduction, signal processing, or non-local self-similarity.Examples include wavelets, kernel mappings, PCA, skip connections, and NSS.
- 3.1.3. Combination of optimization method and CNN/NN: Optimization-based CNN methods target a tradeoff between denoising efficiency and flexibility, including approaches that improve speed or denoising performance.The overview distinguishes optimization methods for rapidly finding solutions from combinations intended to improve image quality.
- 3.4. Deep learning techniques for hybrid noisy image denoising: The overview also covers hybrid noisy images and video, where hybrid methods address multiple degradations and video methods exploit frame, spatial, or temporal information.Hybrid degradation combines noisy, blurred, and low-resolution conditions; video methods include recurrent, non-local, temporal, and two-stage CNNs.
- 3.2. Deep learning techniques for real noisy image denoising: Real-noisy-image denoising uses architectural changes such as multiscale features, dual networks, recurrent connections, residual structures, and self-supervision.These designs address real corrupted images, including low-light, medical, remote-sensing, and unknown-noise settings.
4.1. Datasets
The overview separates training and testing data into gray-noisy and color-noisy image datasets. These datasets support evaluation of Gaussian, blind, and real-noise denoisers.
- Training datasets: Training datasets are divided into gray-noisy and color-noisy images, with BSD400 and Waterloo Exploration Database used among the gray-noisy data.BSD400 contains 400 images, while Waterloo Exploration Database contains 4,744 nature images.
- Test datasets: Testing uses gray-noisy datasets Set12 and BSD68, alongside color-noisy datasets including CBSD68, Kodak24, McMaster, DND, SIDD, Nam, CC, and NC12.Set12 contains 12 scenes and BSD68 contains 68 nature images.
4.2. Experimental results
The experiments evaluate denoising methods quantitatively and qualitatively across additive-white, real, blind, and hybrid noise settings. Results identify different leading methods for different noise conditions and datasets.
- Evaluation protocol: Quantitative evaluation uses PSNR and runtime, while qualitative evaluation compares magnified regions for visual clarity.FSIM is also used on BSD68 at noise levels 15, 25, and 50.
- 4.2.1. Additive white noisy-image denoising: FSIM experiments on BSD68 cover noise levels 15, 25, and 50, and additional tests examine gray and color additive-white noisy images.Tables 11–14 report visual quality, single-gray-image performance, color-image performance, and efficiency.
- 4.2.2. Real-noisy image denoising: DRDN achieves the best reported results on DND and SIDD for real-noisy-image denoising, while AGAN performs excellently on compressed noisy images.SDNet and BRDNet achieve the best and second-best performance, respectively, for real noisy images across ISO values.
- 4.2.3. Blind denoising: FFDNet and ADNet are reported as superior to the other compared methods for blind denoising on BSD68 and Set12.The comparison includes DnCNN, FFDNet, ADNet, SCNN, and G2G1.
- 4.2.4. Hybrid-noisy-image denoising: WarpNet is reported to be very competitive with methods such as DnCNN and MemNet for multi-degradation denoising.The hybrid setting reflects corrupted images containing different kinds of noise.
5. Discussion
The discussion surveys denoising strategies, benchmark evaluations, efficiency techniques, and unresolved challenges across image-denoising settings.
- Performance and efficiency: Increasing network depth or width enlarges the receptive field but raises computational cost and memory consumption.Dilated convolution is presented as an alternative that can improve efficiency and edge-information mining.
- Performance and efficiency: Prior knowledge can be combined with CNNs through loss-function design to obtain more accurate features.The discussion identifies this as one approach for improving denoising performance.
- Efficiency and future directions: Efficiency can be improved by reducing network depth or width, using small kernels or group convolution, and combining dimension reduction with CNNs.Transfer learning, graph methods, neural architecture search, and hardware or camera improvements are also identified as useful directions.
- Complex and unsupervised denoising: Cascade processing uses one CNN to estimate or recover an intermediate result and another CNN to perform subsequent denoising.This mechanism is discussed for complex, real-noisy, blind, and low-resolution settings, including optimization-based noise estimation and self-supervised learning.
- Challenges: Deep denoising networks face challenges from memory requirements, unstable training, scarce real-noisy samples, unsupervised settings, and imperfect PSNR or SSIM metrics.The discussion specifically notes PSNR’s excessive smoothing and SSIM’s dependence on brightness, contrast, and structure.
6. Conclusion
The conclusion presents the paper as a comparative survey of deep-learning methods for image denoising and highlights the gap between regular Gaussian noise and complex real-world degradation. It identifies real-noise ground-truth scarcity and combined degradations as urgent challenges for recovering clean images.
- Conclusion: The paper classifies, analyzes, and compares deep image-denoising networks across noisy-image tasks and benchmark datasets.It also discusses visual effects, efficiency, cross-comparisons, challenges, and future research directions.
- Conclusion: Gaussian denoising has achieved success for regular noise, while real images may contain complex noise, blur, low resolution, and superposed degradations.The conclusion frames recovery of the latent clean image as a central challenge under these conditions.
- Conclusion: Real noisy images often lack ground truth, creating an urgent challenge for training deep-learning denoising models.The conclusion identifies this limitation alongside the need to recover clean images from degraded observations.