Source-linked AI summary
When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach
Ding Liu, Bihan Wen, Xianming Liu, Zhangyang Wang, Thomas S. Huang
TL;DR
The paper addresses the separation of image denoising and high-level vision by jointly modeling them. It proposes a denoising CNN and a cascade trained with joint losses, finding improved denoising quality and robust performance across high-level tasks. The paper also limits its denoising evaluation by emphasizing visual quality rather than PSNR optimization.
Problem
Image denoising and high-level vision tasks are conventionally handled separately, although noisy inputs and MSE-based restoration can degrade task performance and visual detail.
Method
The method cascades a denoising network with a high-level vision network and uses a weighted joint reconstruction and task loss to update only the denoiser.
Results
The denoiser achieves state-of-the-art performance, while joint training improves visual quality and provides sufficiently high accuracy across various noise levels and high-level tasks.
Takeaways & Limitations
High-level semantic information can guide denoising, and a denoiser trained for one high-level task can generalize to other high-level vision tasks.
Takeaways & Limitations
The joint-loss model is not expected to achieve the highest PSNR because MSE-trained models should outperform it on that metric.
Abstract
from arXiv · showhide
Conventionally, image denoising and high-level vision tasks are handled separately in computer vision. In this paper, we cope with the two jointly and explore the mutual influence between them. First we propose a convolutional neural network for image denoising which achieves the state-of-the-art performance. Second we propose a deep neural network solution that cascades two modules for image denoising and various high-level tasks, respectively, and use the joint loss for updating only the denoising network via back-propagation. We demonstrate that on one hand, the proposed denoiser has the generality to overcome the performance degradation of different high-level vision tasks. On the other hand, with the guidance of high-level vision information, the denoising network can generate more visually appealing results. To the best of our knowledge, this is the first work investigating the benefit of exploiting image semantics simultaneously for image denoising and high-level vision tasks via deep learning. The code is available online https://github.com/Ding-Liu/DeepDenoising.
1 Introduction
The paper unifies image denoising with high-level vision by studying their mutual influence. It proposes a denoiser and a cascade that uses semantic information to improve denoising while supporting high-level tasks.
- Computer vision traditionally separates image restoration from high-level vision, but this paper proposes solving both within one unified framework.
- The proposed convolutional denoising network is inspired by U-Net and achieves state-of-the-art performance.
- MSE-based denoising can lose important details and create over-smoothing artifacts, especially in texture-rich regions.
- A cascade connects image denoising to a high-level vision network and jointly minimizes reconstruction and high-level vision losses.
- The framework investigates whether semantic information can guide denoising and whether denoising can support high-level vision on noisy data.
2 Method
The method combines a multiscale denoising CNN with a cascaded high-level vision network. Joint training updates the denoiser using reconstruction and task losses while keeping the high-level network fixed.
- Denoising Network: The denoising CNN takes a noisy image as input and reconstructs an image through downsampling and upsampling across multiple spatial scales.
- Denoising Network: Long-distance skip connections add the learned residual to the input image to recover details in the reconstructed output.
- Denoising Network: Feature encoding uses convolutional layers and residual blocks, while feature decoding fuses representations from adjacent scales by concatenation.
- Denoising Network: The denoising network uses spatially invariant operations, allowing it to process input images of arbitrary size.
- Cascaded Network: The cascaded architecture feeds the denoised image into a high-level vision network to produce classification or segmentation outputs.
- Training Strategy: During end-to-end training, the high-level network is initialized from noiseless data and fixed, while back-propagated error updates only the denoising network.
- Loss: The joint loss combines the denoising reconstruction loss with the high-level task loss as a weighted sum.
3 Experiments
Experiments evaluate the denoiser on Kodak and assess whether jointly trained cascades improve classification, segmentation, and perceptual quality across noisy inputs. The results show strong standalone denoising, cross-task generality, and improved detail preservation with semantic guidance.
- 3.1 Image Denoising: The proposed denoiser outperforms competing methods quantitatively across noise levels σ = 25, 35 and 50 on the 24-image Kodak dataset.It achieves the highest PSNR in almost every Kodak image.
- High-Level Vision Information Guided Image Denoising: PSNR is not the primary objective for joint-loss experiments because it can correlate poorly with human visual-quality assessments.The authors instead compare qualitative differences between joint-loss and MSE-trained models.
- High-Level Vision Information Guided Image Denoising: Jointly trained denoising preserves more details and textures than CBM3D, DnCNN, and the separately trained denoiser, which exhibit oversmoothing.Figure 4 compares Kodak denoising outputs with and without high-level vision guidance.
- Generality of the Denoiser for High-Level Vision Tasks: Joint Training achieves sufficiently high classification and segmentation accuracy across various noise levels, whereas direct noisy inputs and independent denoising perform worse.The evaluated tasks use classification accuracy and segmentation mean IoU without CRF postprocessing.
- Generality of the Denoiser for High-Level Vision Tasks: Cross-task joint training outperforms Separate+VGG and remains comparable to task-specific Joint Training for classification, while improving segmentation mean IoU.The results support generality across classification and semantic segmentation.
- Generality of the Denoiser for High-Level Vision Tasks: Joint Training and Joint Training (Cross-Task) produce finer segmentation and more visually pleasing denoised outputs than the separately trained denoiser.The separately trained denoiser shows oversmoothing artifacts in a region associated with inaccurate segmentation.
4 Conclusion
The paper presents a unified deep-learning framework linking low-level denoising with high-level semantic tasks. Semantic information improves denoising quality, while the denoiser generalizes across high-level tasks.
- 4 Conclusion: The framework lets high-level semantic information flow back to the denoising network.This jointly addresses low-level vision and high-level semantic tasks.
- 4 Conclusion: The denoiser trained for one high-level task generalizes to other high-level vision tasks.
- 4 Conclusion: The unified approach achieves superior performance in both image denoising and various high-level vision tasks.
- 4 Conclusion: The framework offers a feasible deep-learning solution for real-world problems and may extend to other corruptions.