Source-linked AI summary
Understanding How Image Quality Affects Deep Neural Networks
Samuel Dodge, Lina Karam
TL;DR
The paper examines how image-quality distortions affect deep neural network image classification, addressing the practical gap between high-quality evaluation data and lower-quality application inputs. Evaluating four networks on distorted ImageNet images, it finds broad susceptibility to blur and noise but resilience to compression and contrast changes.
Problem
Computer vision research often assumes relatively high-quality inputs, although applications such as surveillance and mobile vision may involve lower-quality images whose effects require separate evaluation.
Method
The paper evaluates four state-of-the-art deep neural networks on ImageNet images augmented with five distortion types and assesses top-1 and top-5 classification accuracy.
Results
All tested networks are susceptible to blur and noise but resilient to compression artifacts and contrast changes; at noise standard deviation 90, average performance falls below 20%.
Takeaways & Limitations
The shared susceptibility to blur and noise suggests that new network designs may be needed to achieve invariance to these distortions.
Abstract
from arXiv · showhide
Image quality is an important practical challenge that is often overlooked in the design of machine vision systems. Commonly, machine vision systems are trained and tested on high quality image datasets, yet in practical applications the input images can not be assumed to be of high quality. Recently, deep neural networks have obtained state-of-the-art performance on many machine vision tasks. In this paper we provide an evaluation of 4 state-of-the-art deep neural network models for image classification under quality distortions. We consider five types of quality distortions: blur, noise, contrast, JPEG, and JPEG2000 compression. We show that the existing networks are susceptible to these quality distortions, particularly to blur and noise. These results enable future work in developing deep neural networks that are more invariant to quality distortions.
I. INTRODUCTION
Image quality for computer vision cannot be assessed solely from human perception, because DNNs may respond differently to distortions that humans perceive similarly. This paper therefore examines when ordinary blur and noise begin degrading classification and whether network structure affects invariance.
- Computer-vision image quality must be considered separately from human-perceived quality because machines and humans can respond differently to the same images.
- Unlike carefully crafted adversarial noise, acquisition, transmission, and storage artifacts are common practical distortions that can still cause DNN misclassification.
- The study asks at what distortion level performance begins to decrease and whether network structure affects invariance to quality distortions.
A. Related Works
Prior work examined selected low-quality recognition settings, but did not provide a broad evaluation of DNNs on natural images across distortion types and levels. This paper addresses that gap using ImageNet and several state-of-the-art networks.
- Earlier work introduced a five-distortion face-recognition dataset without evaluating models on it, while sparse representations later achieved good performance there.
- The n-MNIST database adds Gaussian noise, motion blur, and reduced contrast to MNIST and accompanies these distortions with a modified deep-belief-network approach.
- MIRC-based evaluations found that deep networks could not match human performance on small, blurry image regions.
- The paper presents the first large-scale evaluation of deep networks on natural images under different types and levels of image-quality distortion.
- The evaluation uses ImageNet, a 1000-class dataset, rather than the specialized face and digit datasets used in earlier distortion studies.
II. BACKGROUND
Deep neural networks compute layered nonlinear transformations, using shared convolutional weights and pooling to process images efficiently and robustly. Their final softmax layer converts class responses into a probability distribution, and training adjusts parameters through backpropagated gradients.
- Each neuron computes an activation from an input vector, weight vector, bias, and nonlinearity, producing one output number.
- Layered networks pass one layer’s outputs to the next, while nonlinearities allow them to learn nonlinear functions.
- Weight sharing connects neurons to local receptive fields, implementing convolutional filtering while reducing memory and computational requirements.
- Max pooling takes the maximum response within a window to improve robustness to filter-response noise and enlarge later receptive fields without enlarging filters.
- A softmax output normalizes class responses into probabilities that sum to one.
- Training compares predictions with known labels, computes a cost gradient, and propagates that gradient backward through the network.
A. Deep Networks
The study compares four standard DNN architectures trained on ImageNet, spanning AlexNet-like, VGG, and inception-based designs. The models differ in depth, layer parameters, and parameter efficiency.
- The paper evaluates four representative neural networks using pretrained Caffe model weights.
- All models were trained on ImageNet, which contains 1000 classes and 1.2 million training images.
- The Caffe Reference Model implements AlexNet with 5 convolutional layers followed by 3 fully connected layers.
- VGG-CNN-S has the same 5-convolutional-layer and 3-fully-connected-layer pattern as the Caffe Reference Model but changes layer parameters.
- VGG16 is deeper, with 13 convolutional layers followed by 3 fully connected layers.
- GoogleNet uses inception layers that process multiple filter sizes in parallel and fuse responses, requiring far fewer parameters than the other tested networks.
B. Dataset
The evaluation uses 10,000 images from the ImageNet 2012 validation set, with 10 randomly selected images from each of 1,000 categories and additional distorted versions generated for every image.
- 10,000 ImageNet validation images were used, sampled as 10 images from each of 1,000 categories.
- Each selected image was augmented with additional versions at varying quality-distortion levels.
C. Distortions Types and Levels
The study separately evaluates five common image distortions across specified parameter ranges, covering compression, noise, blur, and contrast reduction.
- Five distortions were tested separately: JPEG compression, JPEG2000 compression, noise, blur, and contrast.
- JPEG quality parameters ranged from 2 to 20 in steps of 2, while JPEG2000 PSNR ranged from 20 to 40 in steps of 2.
- Gaussian noise was added independently to each color component, with standard deviation varied from 10 to 100 in steps of 10.
- Blur used a Gaussian kernel with standard deviation from 1 to 9 in steps of 1, and a filter window four times that deviation.
- Contrast was reduced by blending each input image with a gray image using blending factors from 0 to 1 in steps of 0.1.
IV. RESULTS
Top-1 and top-5 accuracy were evaluated under five distortions. The networks were especially sensitive to blur and noise, while compression and contrast generally caused less degradation, with distortion also changing confidence and internal responses.
- Top-1 and top-5 classification accuracy were used to assess network performance under image-quality distortions.
- All networks were very sensitive to blur, with significant accuracy decreases even at moderate blur levels.
- Noise caused a similar performance decrease, but VGG-16 and GoogleNet declined more slowly than the Caffe and VGG-CNN-S networks.
- At noise standard deviation 90, average network performance fell below 20%, although human observers could still easily recognize the images.
- JPEG and JPEG2000 compression produced little degradation until quality parameter values below 10 or PSNR below 30, respectively.
- Networks were resilient to contrast changes, with VGG-16 appearing more robust than the other networks.
- As quality decreased, networks became less confident, while blur and noise altered filter responses across network layers.
- Blur changed first-layer responses slightly but produced larger changes at the last convolutional layer, whereas noise was most evident in the first layer.
V. DISCUSSION AND CONCLUSION
All tested networks are vulnerable to blur and noise, while compression and contrast are less disruptive; VGG-16 performs best across the tested distortions. Filter responses and training considerations suggest both architectural and data-related factors matter.
- All tested networks are susceptible to blur and noise but resilient to compression artifacts and contrast.
- Color perturbation during training does not prevent Caffe and GoogleNet from performing poorly on low-quality images.
- Blur changes late VGG16 filter responses despite limited effects on early responses, whereas noise creates early activations that propagate to later layers.
- Training on low-quality images may improve low-quality test performance but could reduce high-quality performance and increase training time.
- VGG-16 achieves the best classification accuracy and resilience across all tested distortion types and levels.