Source-linked AI summary
On Classification of Distorted Images with Deep Convolutional Neural Networks
Yiren Zhou, Sibo Song, Ngai-Man Cheung
TL;DR
Image blur and noise can affect DNN image-classifier performance, so the paper evaluates four distortion types and tests retraining and fine-tuning with distorted images. Both approaches can improve performance on distorted inputs; the preferred approach depends on distortion level and data availability.
Problem
Image blur and noise can affect DNN classifier performance, motivating study of distortion effects and ways to alleviate them.
Source: §1 · p. 1Method
Results
Fine-tuning and retraining can reduce error on distorted images; retraining performs better at high distortion, while both can do well at low distortion.
Source: §4 · p. 3, §5 · pp. 4–5Takeaways & Limitations
At low distortion, fine-tuning is preferable because it converges faster and applies to limited-size distorted datasets.
Source: §5 · p. 5For very deep models, training on a small distorted dataset can lead to overfitting; fine-tuning the first N layers while fixing the rest is offered for this case.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Image blur and image noise are common distortions during image acquisition. In this paper, we systematically study the effect of image distortions on the deep neural network (DNN) image classifiers. First, we examine the DNN classifier performance under four types of distortions. Second, we propose two approaches to alleviate the effect of image distortion: re-training and fine-tuning with noisy images. Our results suggest that, under certain conditions, fine-tuning with noisy images can alleviate much effect due to distorted inputs, and is more practical than re-training.
1. INTRODUCTION
The paper studies how image blur and noise affect DNN image classifiers and examines ways to mitigate that effect. It distinguishes this goal from using noise to regularize training on clean images.
1. INTRODUCTION
- In practical applications, image distortions can affect DNN classifier performance, motivating systematic study of their effects and mitigation.Source: §1 · p. 1
- The study evaluates motion blur, defocus blur, Gaussian noise, and their combination, then examines retraining and fine-tuning with distorted images.Fine-tuning adjusts low-level filters to match distorted-image characteristics.Source: §1 · p. 1
- Retraining with noisy images can improve classification accuracy on distorted images, but requires large training datasets for very deep networks.Source: §1 · p. 1
- Earlier DNN distortion studies addressed noisy MNIST or degradation across several DNNs; this work examines both handwritten-digit and natural-image classification and tests mitigation strategies.Source: §1 · p. 1
- Unlike prior noise use for regularization on clean images [13], this work studies noisy training data for classifying distorted images and reports fine-tuning can be practical under certain conditions.Source: §1 · p. 1
2. DEEP ARCHITECTURE
The section introduces DNNs and explains how convolutional networks use image locality to reduce model size. It also describes convolutional-layer connectivity and presents LeNet-5 as an example architecture.
2. DEEP ARCHITECTURE
- The section introduces deep convolutional neural networks as its focus within the broader family of DNNs.Source: §2 · p. 1
- A DNN is described as a machine-learning architecture composed of neurons across multiple concatenated layers, with neighboring layers fully connected.Source: §2 · p. 1
- CNNs use local image connectivity as prior knowledge, reducing model size while preserving necessary image information.Conventional DNNs for large images can require many parameters and exceed limited training data and computation resources.Source: §2 · p. 1
- In a convolutional layer, each output node connects to a local receptive field, and output nodes use different kernels with shared weights.Source: §2 · p. 1
- LeNet-5, used for MNIST digit classification, has two convolutional layers with pooling layers, followed by flatten and fully connected layers.Source: §2 · pp. 1–2, Figure 1 · p. 2
3. EXPERIMENTAL SETTINGS
Experiments evaluate image classifiers on MNIST, CIFAR-10, and ILSVRC2012 under blur, noise, and combined distortions, using fine-tuning or retraining on small datasets and fine-tuning on ILSVRC2012. Fine-tuning takes less time than retraining on MNIST and CIFAR-10, and is used on ILSVRC2012 to prevent overfitting.
3. EXPERIMENTAL SETTINGS
- The study examines full-training and fine-tuning configurations on smaller datasets before evaluating the approach on ImageNet.The authors use MatConvNet for the experiments.Source: §3 · p. 2
- The study evaluates MNIST with LeNet-5, CIFAR-10 with CIFAR10-quick, and ILSVRC2012 with a pretrained AlexNet.The ILSVRC2012 experiment randomly selects 50,000 training images and tests on the 50,000-image validation set.Source: §3 · pp. 2–3
- Fine-tuning starts from a model trained on clean images, updates its first N layers on distorted data, and fixes the remaining layers.The setup targets early layers because the authors consider blur and noise to affect low-level features more than high-level semantic information.Source: §3 · p. 2
- Retraining trains the whole network from scratch on distorted images without using a pretrained model.For LeNet-5, retraining takes about 20 minutes and fine-tuning about 15 minutes; each epoch takes about one minute.Source: §3 · p. 2
- Fine-tuning takes less time than retraining on MNIST and CIFAR-10, while ILSVRC2012 requires fine-tuning to prevent overfitting.The authors do not use retraining on ILSVRC2012 because retraining AlexNet with only part of the training set would cause overfitting.Source: §3 · p. 3
- Experiments apply motion blur, defocus blur, Gaussian noise, or all three distortions together, with levels sampled from 0 to 4.Motion blur uses a random-walk kernel; defocus blur uses a uniform anti-aliased disc, and Gaussian noise has zero mean with standard deviation sampled from [0, 4].Source: §3 · p. 3
4. EXPERIMENTAL RESULTS AND ANALYSIS
The experiments find that distortion effects vary across MNIST and CIFAR-10, while fine-tuning and retraining reduce error rates under distortion. Fine-tuning the first few layers can approach whole-network fine-tuning, but retraining performs better for more severe distortion or combined noise; feature-map analysis offers a representation-level interpretation.
Results on MNIST
- On MNIST, defocus blur and combined noise increase error more than motion blur or Gaussian noise, and retraining leads when distortion increases.Fine-tuning and retraining both reduce error; first-3 and first-4 fine-tuning perform similarly.Source: §4 · p. 3, Figure 6 · p. 4
Results on CIFAR-10
- On CIFAR-10, fine-tuning and retraining reduce error, but retraining performs better at high distortion or under combined noise.At low distortion, the two approaches have similar results; fine-tuning the first three to five layers also gives similar results.Source: §4 · p. 3, Figure 7 · p. 4
Results on MNIST and CIFAR-10
- Across the two figures, fine-tuning the first three layers gives results similar to fine-tuning the whole network, consistent with distortion affecting low-level information more.The comparison suggests that high-level information is less affected by distortion.Source: §4 · p. 3
Analysis
- The analysis measures mean variance of feature-map gradient magnitude, using horizontal and vertical Sobel gradients; higher variance is associated with more edge and texture information.The paper treats this statistic as a useful feature for examining representations inside the model.Source: §4 · pp. 3–4
- In conv3, distorted inputs lower feature-map gradient variance without training, fine-tuning keeps it near the original-image level, and retraining raises it above that level.The authors interpret the pattern as fine-tuning preserving useful information from distorted images, while retraining fits the distorted dataset.Source: Figure 8 · p. 4, §4 · p. 4
Results on ImageNet
- On ImageNet, fine-tuning the first three AlexNet layers improves accuracy on distorted data while keeping clean-data performance.Table 1 compares the pre-trained and fine-tuned models on the ImageNet validation set; the supplied passages give no numeric values.Source: §4 · p. 4
5. CONCLUSIONS
Noisy-data retraining and fine-tuning can improve performance on distorted images, but the preferred approach depends on distortion level and distorted-dataset size.
5. CONCLUSIONS
- Noisy-data fine-tuning and re-training can increase performance on distorted data, with re-training usually comparable to or better than fine-tuning.Source: §5 · p. 4
- For a very deep model and a small distorted dataset, training on the limited data risks overfitting; fine-tuning the first N layers while fixing the rest is proposed to prevent it.Source: §5 · p. 4
- When distortion is high, re-training on distorted data has better results.Source: §5 · p. 4
- At low distortion, both methods can achieve good results, while fine-tuning is preferable for faster convergence, less computation time, and limited-size distorted datasets.Source: §5 · p. 5