Source-linked AI summary
A cross Transformer for image denoising
Chunwei Tian, Menghua Zheng, Wangmeng Zuo, Shichao Zhang, Yanning Zhang, Chia-Wen Ling
TL;DR
Image denoising requires effective structural representations for noisy images, especially in complex scenes. CTNet combines serial, parallel, residual, heterogeneous-network, and Transformer components to search complementary information, and experiments report effective denoising across synthetic and real images with smartphone applicability.
Problem
CNN-based denoising needs effective structural information to represent noisy images efficiently in complex scenes.
Method
CTNet combines serial, parallel, and residual blocks with heterogeneous feature interactions and Transformer mechanisms for denoising.
Results
Experiments report effective quantitative and qualitative denoising and image-quality performance for CTNet, including smartphone image processing.
Takeaways & Limitations
CTNet is reported as suitable for complex-scene denoising on mobile digital devices such as phones.
Abstract
from arXiv · showhide
Deep convolutional neural networks (CNNs) depend on feedforward and feedback ways to obtain good performance in image denoising. However, how to obtain effective structural information via CNNs to efficiently represent given noisy images is key for complex scenes. In this paper, we propose a cross Transformer denoising CNN (CTNet) with a serial block (SB), a parallel block (PB), and a residual block (RB) to obtain clean images for complex scenes. A SB uses an enhanced residual architecture to deeply search structural information for image denoising. To avoid loss of key information, PB uses three heterogeneous networks to implement multiple interactions of multi-level features to broadly search for extra information for improving the adaptability of an obtained denoiser for complex scenes. Also, to improve denoising performance, Transformer mechanisms are embedded into the SB and PB to extract complementary salient features for effectively removing noise in terms of pixel relations. Finally, a RB is applied to acquire clean images. Experiments illustrate that our CTNet is superior to some popular denoising methods in terms of real and synthetic image denoising. It is suitable to mobile digital devices, i.e., phones. Codes can be obtained at https://github.com/hellloxiaotian/CTNet.
1. Introduction
The paper motivates CTNet as a CNN-based denoiser for complex scenes, combining serial and parallel structural search with Transformer mechanisms and residual prediction. It reports effective quantitative and qualitative denoising, including processing smartphone images.
- Traditional denoising methods can require manual parameter tuning and complex optimization, reducing efficiency despite improving repaired-image quality.
- CTNet uses a serial block, parallel block, and residual block to predict clean images from noisy inputs.
- Extended experiments evaluate quantitative and qualitative denoising and image quality, while reporting that CTNet can process smartphone images.
- CTNet combines serial and parallel architectures to search structural information deeply and broadly for complex-scene denoising.
- Three heterogeneous networks perform multi-level feature interactions to improve denoiser adaptability for complex scenes.
- Transformer mechanisms embedded in the CNN extract salient information according to pixel relations to improve denoising performance.
2. Related work
Related work spans single and multiple CNN denoisers, with attention mechanisms used to extract salient information and improve feature relations. The paper motivates a mixed serial-parallel design to balance execution speed, stability, and complex-scene adaptability.
- 2.1. Deep CNNs for image denoising: Single CNN architectures provide richer local features through translation invariance and smaller filters for image denoising.
- 2.1. Deep CNNs for image denoising: Multiple CNNs integrate heterogeneous subnetworks and complementary operations to capture richer structural information for complex scenes.
- 2.1. Deep CNNs for image denoising: Single networks offer faster execution, whereas multiple CNNs are more stable for image denoising.
- 2.1. Deep CNNs for image denoising: The paper therefore designs a mixed serial-parallel denoising network to extract representative information for complex scenes.
- 2.2. Attentive methods in image denoising: Attention mechanisms extract salient information through spatial, channel, or layer relations to support noise removal and structural representation.
3. The proposed method
CTNet combines serial, parallel, and residual blocks with Transformer mechanisms to extract structural, complementary, and pixel-relation features for image denoising. Its pipeline trains on paired noisy-clean images and converts the resulting features into clean images.
- Network architecture: CTNet combines a serial block, parallel block, and residual block to transform noisy inputs into predicted clean images.The architecture is expressed as CTNet(IN) = RB(PB(SB(IN))).
- Training objective: The training objective uses mean square error on paired clean and noisy images, with Adam optimizing CTNet parameters.The paired samples provide the clean and corresponding noisy images used to train the denoiser.
- Serial block: The serial block uses convolutional, convolution-plus-ReLU, residual-learning, and Transformer operations to deeply search structural information.Its Transformer mechanism uses multi-head self-attention for global representations and channel feature enhancement for salient features.
- Parallel block: The parallel block uses three heterogeneous subnetworks and multi-level feature interactions to broadly search pixel-correlation information.Fusion mechanisms mediate interactions among SubNet1, SubNet2, and SubNet3, while the parallel output feeds the residual block.
- Parallel block: Transformer mechanisms in the subnetworks extract global representations, while convolutional and depth-wise separable layers provide complementary linear, nonlinear, and fused features.SubNet2 and SubNet3 combine these operations with feature-interaction and fusion mechanisms.
- Residual block: The residual block applies residual learning through a convolutional layer to convert parallel-block features into a clean image.Its output channel is one for grayscale inputs and three for color inputs.
4. Experiments
Experiments evaluate CTNet across synthetic and real noisy-image datasets, architectural variants, denoising speed, and qualitative comparisons. The reported results show strong PSNR performance, clearer outputs, and applicability to real and smartphone images.
- Experimental setup: CTNet is evaluated on synthetic and real noisy-image datasets, including BSD68, Set12, Urban100, CC, smartphone images, and clinical images.The experiments use both gray and color synthetic noise settings, as well as real noisy-image benchmarks.
- Architecture analysis: The serial block uses stacked convolutional layers, while the parallel block uses three heterogeneous subnetworks for breadth-oriented feature search.The parallel design performs multi-level feature interactions, and the serial design searches structural information through depth.
- Architecture analysis: Figure 2 and ablations indicate that combining serial and breadth-oriented architectures produces stronger feature similarity behavior than the serial architecture alone.The reported CKA comparisons also support the effectiveness of the feature-interaction design in SubNet2 and SubNet3.
- Synthetic denoising results: CTNet nearly achieves the best PSNR across Set12, BSD68, and Urban100 for noise levels 15, 25, and 50.Reported improvements include 0.51dB over DnCNN at noise level 25 on Set12 and 1.08dB over DnCNN at noise level 15 on Urban100.
- Real-image results: CTNet outperforms popular methods on real noisy images and reports improvements of 1.84 dB on smartphone images and 2.27dB on CT images over DnCNN.On CC, it nearly obtains the best PSNR for most ISO settings and exceeds BRDNet by 1.16dB in one reported comparison.
- Qualitative results: Qualitative comparisons show CTNet producing clearer regions and more detailed information than ADNet, DnCNN, and FFDNet.The visual comparisons cover gray and color noisy-image datasets, including BSD68, Set12, and McMaster.
5. Conclusion
CTNet combines serial, parallel, and residual blocks with Transformer mechanisms to denoise complex scenes. Its design searches structural and detailed information through depth, breadth, and pixel relations, with reported suitability for phones and CT machines.
- CTNet combines serial, parallel, and residual blocks for image denoising in complex scenes.
- The serial block uses depth-search ideas and an enhanced residual architecture to facilitate structural information.
- The parallel block uses three heterogeneous networks and multi-level feature interactions to extract richer detailed information.
- Transformer mechanisms in the serial and parallel blocks extract complementary salient information through pixel relations to remove noise.
- The method is reported as suitable for complex scenes, mobile phones, and medical imaging devices such as CT machines.