PaperlayerView exact revision on arXiv

Source-linked AI summary

How Image Degradations Affect Deep CNN-based Face Recognition?

Samil Karahan, Merve Kilinc Yildirim, Kadir Kirtac, Ferhat Sukru Rende, Gultekin Butun, Hazim Kemal Ekenel

arXiv:1608.05246v1cs.CV

TL;DR

Deep CNN face recognition had improved on in-the-wild datasets, but its performance under image-quality degradations had not been assessed systematically, despite their relevance to real-world applications. The paper evaluates three CNN models under LFW closed-set identification and finds that blur, noise, and occlusion harm performance, while color distortions and color-balance changes are tolerated.

  • Problem

    The performance of deep CNN face recognition under image-quality degradations had not been systematically assessed, although real-world applications may involve such variations.

    Source: §1 · pp. 1–2

  • Method

    The study evaluates AlexNet, VGG-Face, and GoogLeNet on LFW closed-set identification with varied degradations applied to probes and original gallery images.

    Source: §1 · p. 2

  • Results

    Blur, noise, and occlusion significantly deteriorated performance, while color distortions and color-balance changes were tolerated; fine-tuned GoogLeNet outperformed the other models.

    Source: §1 · p. 2, §5 · p. 7

  • Takeaways & Limitations

    The authors propose augmenting training with degraded images or designing models specialized to individual degradation types as future approaches to robustness.

    Source: §5 · p. 7

    The evaluation is scoped to LFW closed-set identification, three CNN models, and distortions applied to probe images while gallery images remain original.

    Source: §1 · p. 2

Abstract

from arXiv · show

Face recognition approaches that are based on deep convolutional neural networks (CNN) have been dominating the field. The performance improvements they have provided in the so called in-the-wild datasets are significant, however, their performance under image quality degradations have not been assessed, yet. This is particularly important, since in real-world face recognition applications, images may contain various kinds of degradations due to motion blur, noise, compression artifacts, color distortions, and occlusion. In this work, we have addressed this problem and analyzed the influence of these image degradations on the performance of deep CNN-based face recognition approaches using the standard LFW closed-set identification protocol. We have evaluated three popular deep CNN models, namely, the AlexNet, VGG-Face, and GoogLeNet. Results have indicated that blur, noise, and occlusion cause a significant decrease in performance, while deep CNN models are found to be robust to distortions, such as color distortions and change in color balance.

1 Introduction

The paper addresses the need to systematically assess deep CNN face recognition under image-quality variation. Its evaluation finds that blur, noise, and occlusion are challenging, while color distortions and color-balance changes are tolerated.

1 Introduction

  • A systematic analysis of deep CNN performance under appearance variations is needed to identify strengths and weaknesses; prior work studied quality effects in commercial and shallow face-recognition systems.Those studies found significant effects of quality deformation on recognition performance.Source: §1 · p. 1
  • The study evaluates contrast, noise, blur, occlusion, and color distortions at different levels on LFW closed-set probes using AlexNet, VGG-Face, and GoogLeNet.Gallery images remain in their original form.Source: §1 · p. 2
  • For a same-subject sample, cosine similarity decreases significantly across all three networks as Gaussian-noise sigma increases.A similarity of 1.0 indicates a perfect match.Source: §1 · p. 2, Figure 1 · p. 2
  • Blur, noise, and occlusion significantly deteriorate deep CNN face recognition, whereas color distortions and color-balance changes are tolerated.The authors also report that fine-tuned GoogLeNet outperforms the other evaluated models.Source: §1 · p. 2

2 Deep Convolutional Neural Network Architectures

The study compares AlexNet, VGG-Face, and GoogLeNet, using fine-tuned models and face representations extracted from specified network layers.

2 Deep Convolutional Neural Network Architectures

  • AlexNet and GoogLeNet were fine-tuned using CASIA-Webface, Face-Scrub, and CACD, yielding about 530K images from 10,770 subjects after label correction and duplicate removal.AlexNet and GoogLeNet began from ImageNet-pretrained models released through Caffe.Source: §2 · p. 3
  • AlexNet fine-tuning used SGD with momentum, a 15% validation split, batch size 128, and 100,000 maximum iterations.The resulting representation was the 4096-dimensional fc7 feature vector.Source: §2 · p. 3
  • GoogLeNet used a 1024-dimensional pool5/7x7 s1 feature vector, while VGG-Face used a 4096-dimensional fc6 feature vector.GoogLeNet was fine-tuned for up to 420,000 iterations with batch size 32; VGG-Face features were taken from the cited network.Source: §2 · p. 3

3 Experimental Setup

The experiments apply five types of image degradation to LFW closed-set probe images while keeping the gallery unchanged. They vary the degradation parameters and assess identification accuracy.

3 Experimental Setup

  • The LFW closed-set protocol uses one gallery image for each of 4,249 subjects and 3,143 probe images, with cosine distance and distortions applied only to probes.The five degradation types are blur, noise, contrast, color distortion, and synthetic occlusion on fixed facial parts.Source: §3 · p. 3
  • Blur was simulated with Gaussian and median filters, varying Gaussian sigma from one to five and median-filter kernel size from three to nine.Gaussian kernel size was set to four times sigma minus one; median kernel size increased in steps of two.Source: §3 · pp. 3–4
  • Contrast was varied by multiplying each channel by factors from 0.2 to 2.0 in increments of 0.2, testing both reduced and increased contrast.Figure 2 illustrates low- and high-contrast examples.Source: §3 · p. 4, Figure 2 · p. 4
  • Gaussian noise used zero-mean random values with standard deviations from 10 to 50, while salt-and-pepper noise varied its parameter from two to ten.Both settings were varied in increments of ten and two, respectively.Source: §3 · p. 4
  • Color distortion was simulated by separately increasing each channel’s intensity from 20 to 100 and adjusting color balance from 5 to 30.Both parameter ranges were stepped by 20 and 5, respectively.Source: §3 · p. 5
  • Synthetic occlusions covered the eyes, periocular region, nose, or mouth using landmark-positioned shapes and rectangles; the scarf effect blacked pixels below the nose.Landmarks were detected with dlib.Source: §3 · p. 5

4 Results

On LFW closed-set identification, fine-tuned GoogLeNet achieved a strong undegraded baseline, while blur, noise, and facial occlusion substantially challenged the tested models. Their responses to contrast and color changes varied, and the networks were robust to increasing color-channel intensity.

  • %72.30 Rank-1 from the single-model approach compares with %77.12 for fine-tuned GoogLeNet on the same protocol.Both are reported for LFW closed-set identification without probe-image degradations.Source: §4 · p. 5
  • Increasing blur dramatically lowers accuracy across all models, consistent with blur removing edges and smoothing color transitions that may carry discriminative texture.The authors identify texture information as important for learning discriminative features.Source: §4 · p. 5, Figure 3 · p. 6
  • At very high noise, identification accuracy falls below %6, with GoogLeNet and AlexNet declining faster than VGG-Face under Gaussian noise.The passage contrasts this with humans still easily recognizing subjects at that noise level.Source: §4 · p. 5, Figure 3 · p. 6
  • Contrast changes do not dramatically affect performance, but brightness distortion does; networks are comparatively resilient to darkness, and increasing color balance causes a slight decline.At 30% color-balance distortion, GoogLeNet and VGG-Face performances become equal; increasing channel intensity is generally tolerated, with AlexNet most sensitive.Source: §4 · pp. 5–6, Figure 3 · p. 6
  • Occlusion results identify the periocular region as the most distinctive facial area, while nose features matter more than mouth features across all networks.For VGG-Face, nose occlusion affects performance more than the synthetic sunglasses effect.Source: §4 · p. 6, Table 1 · p. 7

5 Discussion and Conclusion

The study finds that blur, noise, and occlusion most strongly impair deep CNN face recognition, whereas color distortions and color-balance changes are comparatively well tolerated. Fine-tuned GoogLeNet outperformed the other tested models, and the authors propose degraded-image augmentation or degradation-specific models as future approaches.

  • Blur, noise, and occlusion significantly deteriorate recognition, while deep CNNs are robust to color distortions and changes in color balance.Source: §5 · p. 7
  • Fine-tuned GoogLeNet outperformed the other tested models, with margins of almost 10% over VGG-Face in most experiments and up to 20% over AlexNet in some.Source: §5 · p. 7
  • The authors propose augmenting training with degraded subject images or designing separate models for each degradation as future robustness approaches.Source: §5 · p. 7
Loading 1608.05246v1…