Connecting Image Denoising and High-Level Vision Tasks via Deep Learning

• Conventional computer vision separates low-level image processing from high-level vision, leaving their mutual influence insufficiently studied.Source: §I · p. 1
• The proposed denoising CNN processes features at multiple spatial scales through downsampling and upsampling, then fuses them to exploit broader context.Downsampling also reduces feature-map sizes and computation cost.Source: §I · p. 1
• The cascaded framework connects an image-denoising module with a high-level vision module and updates only the denoiser using gradients from a joint loss.Keeping the high-level network untouched is intended to support denoiser generalization across tasks while preserving semantic details.Source: §I · pp. 1–2
• Joint training improves denoised-image perceptual quality and high-level vision accuracy while making a denoiser trained for one task usable with other tasks without fine-tuning.The paper reports consistent performance for noisy and noiseless images and reduced training effort across tasks.Source: §I · p. 2
• The paper presents a unified deep-learning framework for simultaneously exploiting image semantics in denoising and high-level vision tasks.The authors identify this as a first attempt and support the framework with extensive experiments, including real-noise removal and subjective visual-quality evaluation.Source: §I · p. 2

• Classical denoising methods recover clean signals by exploiting local structures, non-local patch similarities, or both.Examples include coefficient shrinkage methods and globally grouped similar patches with structural priors.Source: §II · p. 2
• Deep-learning denoising methods include multilayer perceptron and convolutional neural network approaches.The literature includes stacked denoising autoencoders, plain MLPs, CNNs, unfolded iterative algorithms, skip-connected networks, and dilated convolutions.Source: §II · p. 2
• The proposed framework first introduces its denoising network and then cascades it with a high-level vision module using a joint-loss training strategy.Source: §III · p. 2

• The network takes a noisy image as input and outputs a reconstructed image through feature contraction and expansion using downsampling and upsampling.Source: A. Denoising Network · p. 2
• Each downsampling–upsampling pair creates a new spatial scale, enabling the network to process information across multiple resolutions.Nested pairs build deeper networks with more spatial scales and generally better restoration performance.Source: A. Denoising Network · p. 3
• A long-distance skip connection sums the learned residual with the input image to generate the reconstructed image and recover details.Source: A. Denoising Network · p. 3
• Feature encoding operates separately at each scale, while feature decoding fuses representations from adjacent scales.The paper tests concatenation and element-wise summation, finding similar denoising performance before choosing concatenation to accommodate differing channel numbers.Source: A. Denoising Network · p. 3
• Downsampling progressively enlarges convolutional receptive fields and reduces computation by decreasing feature-map sizes.The larger receptive field incorporates broader spatial context for denoising.Source: A. Denoising Network · p. 3
• Spatially invariant operations allow the denoising network to handle input images of arbitrary size.Source: A. Denoising Network · p. 3

• A noisy image passes first through the denoising network, whose output is then processed by a high-level vision network.Source: B. When Image Denoising Meets… · p. 3
• The cascade aims to reconstruct visually pleasing images guided by high-level vision information while maintaining accuracy across various tasks after training on one task.Source: B. When Image Denoising Meets… · p. 3

Training Strategy

• Training fixes the high-level vision network and updates only the denoising network using error back-propagated through the cascade.Source: B. When Image Denoising Meets… · p. 4

Loss

• The reconstruction loss is mean squared error between the denoising output and the noiseless reference image.Source: B. When Image Denoising Meets… · p. 4
• Perceptual loss compares feature representations of the reconstructed and reference images, using early layers of the high-level vision network as the extractor.Source: B. When Image Denoising Meets… · p. 4
• The joint loss combines reconstruction, perceptual, and high-level-task losses as a weighted sum.Source: B. When Image Denoising Meets… · p. 4

• The experimental section evaluates the proposed denoising network, studies its relation to high-level vision networks, and considers real noise removal.Source: §IV · p. 4

• The denoiser takes RGB images and directly outputs reconstructed images after Gaussian noise with zero mean is added during training.Source: A. Image Denoising · pp. 4–5
• Training uses 432 Berkeley segmentation dataset color images, 48×48 patches, SGD with batch size 32, and 1,500,000 iterations.Source: A. Image Denoising · p. 5
• Architecture experiments vary spatial scales, encoding and decoding modules, and convolutional kernel configurations to analyze their contributions to denoising performance.Source: 1) Analysis of Network Architecture · p. 5
• Higher computation cost from more spatial scales and parameters improves performance, while a global skip connection further improves results.Source: 1) Analysis of Network Architecture · p. 5
• The selected architecture uses three spatial scales, a global skip connection, and encoding kernels (128, 32, 32, 128) with decoding kernels (256, 64, 64, 256).Source: 1) Analysis of Network Architecture · p. 5
• Table I reports average PSNR in dB and model-parameter counts for denoising models evaluated on Kodak images with σ = 35.Source: 1) Analysis of Network Architecture · p. 5, Table I · p. 5

Denoising comparisons

• The evaluation compares CBM3D, MCWNNM, DnCNN, FFDNet, and the proposed method at noise levels σ = 25, 35, and 50 on Kodak and CBSD68.PSNR is measured on Kodak whole images and CBSD68 images.Source: 2) Comparisons with State-of-the-Art… · pp. 5–6

Quantitative results

• The proposed method outperforms all competing approaches quantitatively across different noise levels.It achieves the highest PSNR in almost every Kodak image and the highest average PSNR on CBSD68.Source: 2) Comparisons with State-of-the-Art… · p. 6

High-level vision evaluation

• The study uses VGG-16 for classification and DeepLab-LargeFOV for semantic segmentation in the high-level vision experiments.The experiments follow the preprocessing protocols of the corresponding networks.Source: B. When Image Denoising Meets… · pp. 6–7

Visual comparisons

• Visual denoising examples compare the ground truth, noisy image, CBM3D, DnCNN, and proposed-model outputs with and without high-level guidance.The examples use Kodak images with noise level 50.Source: Figure 4 · p. 7

Evaluation rationale

• PSNR may correlate poorly with human visual quality, so the experiments emphasize qualitative differences between joint-loss and MSE-trained denoisers.The authors state that MSE training should outperform the joint loss on PSNR, making PSNR an unsuitable sole objective here.Source: 1) High-Level Vision Information… · p. 7

Joint-loss guidance

• The cascade combines image reconstruction, perceptual, and classification losses to guide the denoising network.Gradient visualizations analyze each loss component separately on ILSVRC2012 validation images.Source: 1) High-Level Vision Information… · pp. 7–8

Gradient analysis

• Classification-loss gradients concentrate on class-specific, texture-rich regions that conventional denoising can oversmooth, while perceptual-loss gradients focus on local features.Reconstruction-loss gradients are spatially distributed because the reconstruction loss is not spatially biased.Source: 1) High-Level Vision Information… · p. 8
• Joint-loss training denoises class-specific regions differently from the rest of the image, reflecting complementary contributions from the three losses.The classification signal provides auxiliary supervision for preserving semantic details.Source: 1) High-Level Vision Information… · p. 8

Visual quality

• On Kodak examples, jointly trained outputs preserve more details and textures with better visual quality than CBM3D, DnCNN, and the separately trained denoiser.The comparison uses zoom-in regions from images with noise level 50.Source: 1) High-Level Vision Information… · p. 8

Subjective evaluation

• A 15-rater MOS test compares four denoising methods, and the high-level-guidance method achieves the highest score.Scores range from 1 for low image quality to 4 for high image quality.Source: 1) High-Level Vision Information… · p. 8, Figure 5 · p. 8

Evaluation settings

• The evaluation compares direct VGG processing, CBM3D+VGG, Separate+VGG, Joint Training, and Joint Training (Cross-Task) on noisy images.Classification uses ILSVRC2012, while segmentation uses Pascal VOC 2012.Source: 2) Generality of the Denoiser for… · p. 8

Training assumption

• The high-level vision network weights are initialized from noiseless-setting models and remain fixed during denoiser training.This setting isolates updates to the denoising module.Source: 2) Generality of the Denoiser for… · p. 9

Downstream performance

• The baseline VGG approach performs much worse than the other cases, while denoising without high-level semantics can still introduce artifacts that reduce accuracy.The proposed Joint Training approach maintains sufficiently high accuracy across noise levels.Source: 2) Generality of the Denoiser for… · p. 9

Cross-task generality

• A denoiser jointly trained for segmentation improves classification when connected cross-task, and a classification-trained denoiser improves segmentation mean IoU over Separate+VGG.Cross-task performance is comparable to task-dedicated Joint Training in classification and better than Separate+VGG in segmentation.Source: 2) Generality of the Denoiser for… · p. 9, Table V · p. 9

Semantic segmentation

• The separately trained denoiser produces lower segmentation accuracy than joint-loss approaches and shows oversmoothing artifacts in inaccurate regions.Joint Training and Joint Training (Cross-Task) achieve finer segmentation results in the visual examples.Source: Figure 6 · p. 9, 2) Generality of the Denoiser for… · p. 10

Real-noise evaluation

• On real noisy DND images, the proposed method preserves more local edges and high-frequency components than CBM3D and DnCNN.The compared methods are prone to generate oversmoothing regions.Source: C. Extension to Real Noise Removal · p. 10
• The authors report that the real-noise results demonstrate robustness to the noise statistical distribution and good generalization to realistic corruption.DND images are captured at higher ISO and contain more complicated corruption than i.i.d. Gaussian noise.Source: C. Extension to Real Noise Removal · p. 10

V. CONCLUSION

• High-level semantic information flows back to low-level image processing, enabling joint handling of denoising and multiple high-level vision tasks.The authors describe this as a simple and efficient deep learning solution.Source: §V · p. 10
• The joint approach achieves superior performance in both image denoising and various high-level vision tasks.Source: §V · p. 10
• A denoiser trained for one high-level vision task also shows robustness to other high-level vision tasks.Source: §V · p. 10
• Future work will extend high-level semantic guidance to tasks such as super-resolution and incorporate more types of semantical information.Source: §V · p. 10