PaperlayerView exact revision on arXiv

Source-linked AI summary

Effects of Degradations on Deep Neural Network Architectures

Prasun Roy, Subhankar Ghosh, Saumik Bhattacharya, Umapada Pal

arXiv:1807.10108v6cs.CVeess.IV

TL;DR

The paper addresses the limited understanding of how deep architectures, including CapsuleNet, behave under image noise. It benchmarks six architectures across six degradation models, introduces V-CapsNet and NTT, and reports architecture-dependent robustness trade-offs.

  • Problem

    The effects of common image degradations on deep architectures, including capsule networks, are mostly unexplored despite their importance for selecting models in noisy application settings.

  • Method

    The paper compares six architectures on synthetic-digit and natural-image datasets under six degradation models, and introduces V-CapsNet and NTT.

  • Results

    CapsuleNet is more robust to image degradation than conventional CNNs, while V-CapsNet reaches 99.83% validation accuracy on the natural-images dataset but becomes more sensitive as noise increases.

  • Takeaways & Limitations

    Robustness and clean-image accuracy trade off across architectures, while NTT improves CNN robustness against image degradations.

Abstract

from arXiv · show

Deep convolutional neural networks (CNN) have massively influenced recent advances in large-scale image classification. More recently, a dynamic routing algorithm with capsules (groups of neurons) has shown state-of-the-art recognition performance. However, the behavior of such networks in the presence of a degrading signal (noise) is mostly unexplored. An analytical study on different network architectures toward noise robustness is essential for selecting the appropriate model in a specific application scenario. This paper presents an extensive performance analysis of six deep architectures for image classification on six most common image degradation models. In this study, we have compared VGG-16, VGG-19, ResNet-50, Inception-v3, MobileNet and CapsuleNet architectures on Gaussian white, Gaussian color, salt-and-pepper, Gaussian blur, motion blur and JPEG compression noise models.

I. INTRODUCTION

The paper studies how six deep architectures respond to six image degradation models, motivated by the difficulty of predicting degraded-image behavior from clean-image performance. It contributes a robustness benchmark, V-CapsNet, and the NTT block for improving degradation resistance.

  • The study evaluates six state-of-the-art deep architectures for image recognition against six different noise models.
  • V-CapsNet modifies CapsuleNet by fusing VGG-19 feature-extraction layers to improve recognition rate.
  • The proposed NTT block is designed to improve the robustness of CNN architectures against image degradation.
  • The study also explores network behavior under adversarial attacks and reports unique properties of the NTT layer under such perturbations.
  • The authors compile synthetic-digit and natural-image datasets for general benchmarking.

II. RELATED WORK

Prior work addresses degraded-image recognition through preprocessing, embedding, discriminative constraints, and synthetic noisy datasets. This paper instead benchmarks multiple architectures and degradation levels using clean training data and progressively corrupted validation images.

  • Recognition accuracy can fall when real-time application images are noisier than the high-quality images used for training and validation.
  • Earlier studies use coupled kernel embedding, discriminative constraints, or synthetically generated noisy digits to address degraded-image recognition.
  • The study compares six architectures across six degradation models using synthetic-digit and natural-image datasets.
  • Networks are trained on clean training splits, then evaluated repeatedly on validation images with gradually increasing perturbations.

A. Datasets

The study constructs two benchmark settings and adapts six architectures for fair comparison across their class counts. The natural-image dataset uses high-resolution samples from eight diverse classes.

  • Datasets: The synthetic-digit dataset uses 16 fonts and varied color, rotation, and size to create a more complex benchmark than MNIST.
  • Datasets: The natural-image dataset contains 6,899 high-resolution samples from eight classes, split into 5,724 training and 1,175 validation images.
  • Architectures: MobileNet uses depthwise separable convolutions and 4.2 million parameters to trade recognition accuracy for inference speed on resource-limited systems.
  • Architectures: CapsuleNet uses dynamic routing between capsules, removes pooling layers, and has six layers with 8.2 million parameters.
  • Architectures: For fair comparison, CNN classifiers replace default top layers with fully connected layers sized for 10 synthetic-digit classes or eight natural-image classes.

C. Degradation models

The study models six common image degradations by progressively increasing their perturbation strength, enabling evaluation across multiple noise types. These include additive noise, impulse noise, blur, and JPEG compression.

  • Gaussian white noise adds a zero-mean Gaussian noise matrix with standard deviation σ ≥0, using fixed sampled values across image channels.
  • The degradation examples illustrate gradually increasing noise for Gaussian, impulse, blur, and JPEG models.
  • Gaussian color noise samples Gaussian values independently for each image channel, with σ varied from 0 to 1 in steps of 0.1.
  • Salt-and-pepper noise randomly replaces pixels with white or black values, varying degraded-pixel density d from 0 to 1 in steps of 0.1.
  • Gaussian and motion blur are increased through progressively larger kernels, with motion blur using horizontal kernels from 1 × 1 to 31 × 31.
  • JPEG degradation varies encoder quality q from 30 to 0 in steps of 2, where lower quality produces stronger visual perturbations.

IV. RESULTS AND ANALYSIS

The experiments progressively degrade validation images and measure network accuracy on synthetic digits and natural images. The evaluation also includes top-3 accuracy to further assess degradation effects.

  • Validation accuracy is measured repeatedly while perturbation strength increases according to each selected degradation model.
  • The experiments cover both synthetic digits and natural images datasets.
  • Top-3 accuracy is measured alongside classification accuracy to further investigate performance under image degradation.

A. Effects of image degradations on CNN architectures

Network robustness varies by degradation type: CapsuleNet is especially resilient to additive noise, impulse noise, and blur, while VGG architectures often show the smallest decay among conventional CNNs. Under severe JPEG compression, VGG, Inception, and CapsuleNet outperform ResNet and MobileNet.

  • CapsuleNet shows the highest robustness to additive Gaussian noise, while VGG architectures have the lowest performance decay among conventional CNNs.
  • CapsuleNet surpasses other CNNs when salt-and-pepper noise density reaches d ≥0.2 and retains superior recognition through d ≤0.8.
  • CapsuleNet is not significantly affected by Gaussian or motion blur and effectively outperforms other CNNs under heavy blurring.
  • JPEG compression causes no significant accuracy degradation when q > 20, but ResNet and MobileNet degrade sharply for 0 ≤ q ≤20.
  • For severe JPEG compression with 0 ≤ q ≤20, VGG, Inception, and CapsuleNet perform significantly better than ResNet and MobileNet.
  • Three-routing-iteration CapsuleNet with reconstruction-loss regularization shows a slight recognition improvement over the single-routing-iteration configuration.

B. Analysis of network depth in CapsuleNet architecture

The study compares baseline CapsuleNet with a deeper VGG-19 fusion architecture, V-CapsNet. Although V-CapsNet improves clean-image accuracy, baseline CapsuleNet remains more robust to degradation, supporting a relationship between shallower depth and noise resilience.

  • V-CapsNet replaces CapsuleNet’s initial convolution layers with VGG-19 layers and achieves 99.83% validation accuracy on natural images.
  • V-CapsNet improves baseline CapsuleNet performance by 6.2% on the natural images dataset.
  • Despite higher initial recognition accuracy, V-CapsNet becomes less accurate than baseline CapsuleNet as image noise increases.
  • Both capsule layers and shallower network depth are identified as contributors to baseline CapsuleNet’s noise robustness.
  • Feature-map SSIM changes more slowly for baseline CapsuleNet than V-CapsNet when noise is introduced.
  • The analysis associates increased network depth with greater sensitivity to image degradations in this comparison.

V. IMPROVING ROBUSTNESS OF DEEP ARCHITECTURES AGAINST IMAGE DEGRADATIONS

The paper frames a robustness–accuracy dilemma: deeper architectures improve classification but are more vulnerable to image degradations, while training on every possible noise type is impractical.

  • Deeper networks improve classification accuracy by learning complex nonlinearities but are more susceptible to image degradations.
  • CapsuleNet provides a more noise-tolerant shallow architecture, motivating deeper designs that preserve robustness without sacrificing recognition performance.
  • Training with noisy samples is limited because covering every noise model and varying noise level creates an extremely large sample space.

A. Proposed method for improving robustness against noise

The proposed NTT layer uses depthwise nontrainable and trainable filtering to improve robustness to image degradations while limiting accuracy loss. It substantially improves degraded-image performance across architectures and narrows the robustness–accuracy gap in V-CapsNet.

  • Proposed NTT layer: The NTT layer combines a lowpass depthwise nontrainable filter with a depthwise trainable filter to learn invariance to subtle noise and denoising artifacts.
  • Performance trade-off: NTT layers significantly improve robustness for every tested network, while causing a slight decrease in baseline accuracy on noise-free images.
  • Performance trade-off: A VGG-19 NTT layer trades 6% baseline accuracy for up to 68% improvement under Gaussian color and salt-and-pepper noise.
  • Performance trade-off: The accuracy–robustness trade-off can be controlled by adjusting NTT-layer network depth and filter size.
  • V-CapsNet evaluation: For V-CapsNet, NTT layers bring maximum recognition accuracy close to VGG-19 while moving noise robustness close to baseline CapsuleNet in most cases.

B. Defense against adversarial perturbations

The study finds that robustness to ordinary image degradations does not predict robustness to adversarial attacks. Architectures respond differently, and NTT layers can make adversarial perturbations easier to create and identify visually.

  • Adversarial robustness: CapsuleNet and VGG architectures perform poorly against adversarial attacks, whereas ResNet-50, Inception-v3, MobileNet, and V-CapsNet perform well.
  • NTT-layer effects: Injecting NTT layers causes recognition accuracy to decrease faster during adversarial attacks, making perturbations easier to create.
  • NTT-layer effects: Low-ϵ adversarial perturbations are more likely to produce visible distortions, making attacked images easier to identify visually.
  • Scope of robustness: Performance against adversarial attacks does not correlate with robustness against image degradation models.

VI. CONCLUSION

The paper compares CNN and CapsuleNet robustness across image degradations, introduces V-CapsNet for higher recognition accuracy, and proposes NTT layers to address the resulting robustness–accuracy trade-off. Its conclusions separate degradation robustness from adversarial robustness.

  • Main findings: Recognition rates generally decrease as noise increases, and deeper networks are more sensitive to image perturbations.
  • Main findings: Baseline CapsuleNet is robust across degradation models, especially for salt-and-pepper noise and blurring.
  • V-CapsNet: V-CapsNet replaces CapsuleNet feature-extraction layers with VGG-19 layers and achieves the highest classification accuracy in the experiments, but its performance decays faster.
  • NTT and adversarial attacks: The NTT architectural block improves convolutional-network robustness against image degradations, while robustness to adversarial attacks cannot be inferred from perceptible-distortion performance.
Loading 1807.10108v6…