Source-linked AI summary
Occlusion Robust Face Recognition Based on Mask Learning with PairwiseDifferential Siamese Network
Lingxue Song, Dihong Gong, Zhifeng Li, Changsong Liu, Wei Liu
TL;DR
General CNN face-recognition models perform poorly under variable partial occlusions, motivating a method that identifies and removes corrupted feature elements. The paper uses PDSN to learn occlusion-to-feature correspondences, builds FDMs from a mask dictionary, and applies them during recognition. It reports superior performance on synthesized and realistic occluded-face datasets, with generalization to general face-recognition tasks.
Problem
General CNN face models perform poorly under variable facial occlusions, which increase intra-class variation and inter-class similarity.
Method
PDSN learns correspondences between occluded facial blocks and corrupted feature elements, enabling mask-dictionary FDMs that remove corrupted responses during recognition.
Results
The approach reports superior performance on synthesized and realistic occluded-face datasets and generalization to general face-recognition tasks.
Takeaways & Limitations
The method provides an occlusion-robust face-recognition system that explicitly locates and discards corrupted CNN feature elements.
Takeaways & Limitations
Facial occlusion detection is outside the paper’s focus, so occlusion locations are detected using a similar method adopted from prior work.
Abstract
from arXiv · showhide
Deep Convolutional Neural Networks (CNNs) have been pushing the frontier of the face recognition research in the past years. However, existing general CNN face models generalize poorly to the scenario of occlusions on variable facial areas. Inspired by the fact that a human visual system explicitly ignores occlusions and only focuses on non-occluded facial areas, we propose a mask learning strategy to find and discard the corrupted feature elements for face recognition. A mask dictionary is firstly established by exploiting the differences between the top convoluted features of occluded and occlusion-free face pairs using an innovatively designed Pairwise Differential Siamese Network (PDSN). Each item of this dictionary captures the correspondence between occluded facial areas and corrupted feature elements, which is named Feature Discarding Mask (FDM). When dealing with a face image with random partial occlusions, we generate its FDM by combining relevant dictionary items and then multiply it with the original features to eliminate those corrupted feature elements. Comprehensive experiments on both synthesized and realistic occluded face datasets show that the proposed approach significantly outperforms the state-of-the-arts.
1. Introduction
The paper addresses poor CNN face-recognition robustness under partial occlusions by locating corrupted deep-feature elements and discarding them. Its PDSN-based mask dictionary produces FDMs that remove occlusion-corrupted responses, with strong results on synthesized and realistic occlusions.
- Motivation: Occlusions increase intra-class variation and inter-class similarity, causing general CNN face models to perform poorly without specialized training.Relevant occlusions include sunglasses, scarves, masks, and random objects.
- Motivation: The central challenge is locating deep CNN feature elements corrupted by random partial occlusions, unlike low-level features with direct pixel correspondence.The method focuses on eliminating corrupted responses rather than detecting occlusions themselves.
- Method: PDSN learns correspondence between occluded facial blocks and corrupted feature elements from clean and same-identity occluded face pairs.A mask generator uses differential convolutional features, pairwise contrastive loss, and classification loss.
- Method: The method establishes a mask dictionary whose binary FDM items identify feature elements to set to zero when corresponding facial blocks are occluded.At testing, relevant dictionary masks are combined and multiplied with the original face feature.
- Results: The resulting recognition system is reported to perform superiorly on realistic and synthesized occluded-face datasets and to generalize well on general face-recognition tasks.These claims are presented as the paper’s main contributions.
2. Prior Work
Prior work includes local-feature discarding, face recovery, data augmentation, and other deep-learning approaches for occluded face recognition. The paper identifies limited discriminative power and insufficient modeling of occlusion-induced CNN feature inconsistency as unresolved issues.
- Traditional methods: Traditional partial-occlusion methods either extract descriptors from non-occluded regions or recover clean faces before recognition.Local-region methods may identify and discard occluded regions using an SVM.
- Traditional methods: Traditional approaches have limited discriminative ability because they rely on shallow features such as Local Gabor Binary Patterns.Some methods additionally use subspace techniques to reduce feature dimensionality.
- Deep-learning methods: Deep-learning efforts include DeepID2+ robustness and LSTM-Autoencoder restoration of occluded facial areas, but the surveyed approaches remain limited.The passage also notes a MaskNet branch lacking additional supervision information.
- Research gap: The paper argues that existing methods lack awareness of how partial occlusions affect CNN models and overlook feature inconsistency between differently occluded faces.Its proposed method explicitly locates corrupted feature elements and discards them for fair comparison.
3. Proposed Approach
The approach learns feature-level masks from clean/occluded face pairs, then combines masks for random occlusions to discard corrupted top-convolution features while preserving recognition-relevant information.
- Overall framework: The pipeline learns mask generators, builds a mask dictionary, and combines Feature Discarding Masks during testing to filter corrupted features.The combined mask is multiplied element-wise with the original face feature.
- Block-wise masking: The method divides aligned faces into N × N blocks and learns a separate mask generator for each facial block.The implementation uses N = 5, associating blocks with components such as the eyes, nose tip, and mouth.
- Problem analysis: Median relative activation change estimates which top-convolution feature elements are altered by occlusion and may introduce noise into the final representation.The metric is computed across clean/occluded pairs, with high MED indicating likely corrupted elements.
- Pairwise Differential Siamese Network: PDSN feeds clean–occluded feature differences into a mask generator that outputs values in [0, 1] for suppressing contaminated feature elements.Its trunk CNN shares representation extraction across the clean and occluded pair.
- Mask-generator learning: The training objective combines classification and differential losses so masked occluded features remain recognizable and approach their clean counterparts.The classification loss supplies identity supervision, while the differential loss penalizes per-element discrepancies.
4. Experiments
Experiments evaluate the approach across synthesized and realistic occlusions, ablations, general face recognition benchmarks, and implementation settings. The method improves occluded-face recognition while preserving performance on predominantly non-occluded data.
- 4.1. Implementation Details: The system uses MTCNN alignment, FCN-8s occlusion detection, and a three-stage training procedure for the trunk CNN, mask generators, and finetuning.Occlusion detection is trained with synthetic CASIA-WebFace occlusions and AR images; nine mask generators cover the central 3 × 3 facial blocks.
- 4.2. Ablation Study: Binary masks outperform soft weighting, which noticeably decreases performance by retaining features with very low mask values.The Soft+Binary variant removes feature elements below the threshold while retaining mask values above it.
- 4.2. Ablation Study: Differential input and contrastive loss help the model focus on feature elements substantially altered by partial occlusions.Classification loss alone may also suppress feature elements affected by factors unrelated to occlusion.
- 4.3. Performance on LFW Benchmark: The baseline lowers LFW verification accuracy by 0.52%, whereas the proposed method preserves the original trunk CNN performance on mostly non-occluded faces.This result is attributed to discarding corrupted feature elements under partial occlusion rather than forcing the trunk CNN to specialize in occlusions.
- 4.4. Performance on MegaFace Challenge1: On synthesized occluded Facescrub and naturally occluded AR faces, the proposed method outperforms the baseline without compromising the original probe-set performance.AR experiments specifically report significant improvements for real-life sunglasses and scarf occlusions.
5. Conclusions
The paper concludes that PDSN explicitly links occluded facial blocks to corrupted CNN feature elements. Results on synthesized and realistic occluded datasets support superiority over state-of-the-art methods and generalization to general face recognition tasks.
- 5. Conclusions: PDSN builds an explicit correspondence between occluded facial blocks and corrupted feature elements for occlusion-robust face recognition.The conclusion presents this correspondence as the central approach.
- 5. Conclusions: Competitive results on synthesized and realistic occluded face datasets demonstrate superiority over state-of-the-art approaches and generalization to general face recognition tasks.Both occluded-face performance and generalization are included in the paper’s concluding claim.