Source-linked AI summary
A Comparative Transfer-Learning Study of CNN Backbones for Partial Face Recognition on the SoF Dataset
Ahmed Kubba, Ali Alsalama, Abdelrahman Abdalla, Qassim Nasir, Manar Abu Talib
TL;DR
Partial face recognition remains challenging because accuracy degrades when faces are occluded or partly outside the frame. The paper compares three pretrained CNN backbones using transfer learning on SoF; under its shared head-only protocol, FaceNet achieves the strongest reported accuracy and recall on that dataset.
Problem
Face recognition degrades sharply under occlusion, motivating PFR that identifies people from whatever portion of the face is visible.
Source: §I · p. 1Method
The study compares ResNet-50, VGG-16, and FaceNet on SoF using a frozen pretrained backbone and trainable classification head under a shared 160x160 protocol.
Source: §I · p. 2, A. Comparative Framework · p. 2, B. Proposed Architecture · p. 3Results
97.4% test accuracy, 87.04% macro-averaged precision, 84.61% recall, and 84.17% F1 were reported for PFN, with the highest reported accuracy and recall on SoF.
Source: §V · p. 6Takeaways & Limitations
Under a fixed head-only transfer budget, the pretraining corpus matters more to usability on SoF than depth or parameter count, with the claim limited to this protocol.
Source: §V · p. 6The findings do not establish that ImageNet backbones are unsuitable for PFR generally, that the gap survives equal tuning or matched resolution, or that it transfers to other datasets or occlusion types.
Source: F. Limitations and Scope of the Findings · p. 6
Abstract
from arXiv · showhide
Face recognition is widely deployed in surveillance, access control, and forensic workflows, yet accuracy degrades sharply once the face is occluded by accessories, foreground objects, or the frame edge. Because most faces in the wild are partial, robust partial face recognition (PFR) remains open. This paper compares three pretrained convolutional backbones, ResNet-50, VGG-16, and FaceNet, fine-tuned for PFR by transfer learning under identical preprocessing, splitting, and optimization protocols on the Specs-on-Faces (SoF) dataset. All three arms use a common 160x160 input and a frozen backbone with a trainable head under a fixed epoch budget and no per-backbone hyperparameter search. The FaceNet configuration, denoted PFN (Partial FaceNet), substantially outperforms the other two, reaching 97.4% test accuracy with macro-averaged 87.04% precision, 84.61% recall, and 84.17% F1 over the 112 identity classes, the highest accuracy and recall reported on SoF.
I. INTRODUCTION
Partial face recognition (PFR) addresses the sharp degradation of face recognition when faces are occluded or only partly visible. This study compares three pretrained CNN backbones under a matched protocol to examine which transfers best to SoF.
I. INTRODUCTION
- PFR targets recognition from whatever portion of a face remains visible, addressing degradation from occlusion by accessories, objects, or the frame edge.The paper identifies forensics, mobile authentication, attendance, retail analytics, and access control as applications.Source: §I · p. 1
- The comparison asks which readily available pretrained CNN transfers best to partial faces and why, using SoF's natural eyewear, synthetic nose and mouth occlusion, and illumination perturbations.The protocol freezes each backbone and trains appended classification layers.Source: §I · p. 1
- The comparison focuses on pretraining domain while approximately holding architectural family constant; VGG-16's greater parameter count tests whether capacity alone closes the gap with ResNet-50.The authors position attention- and transformer-based models as successors rather than comparison arms because they would vary architectural family and optimization regime too.Source: §I · pp. 1–2
- The study compares ResNet-50, VGG-16, and FaceNet on SoF under a shared protocol, with a common 160x160 input, fixed epoch budget, and no per-backbone hyperparameter search.All arms use the same data, splits, input tensor, head architecture, and optimizer, but not individually optimized configurations.Source: §I · pp. 1–2
- PFN is reported to achieve the best SoF accuracy and recall, while not leading in precision or macro-F1.The paper also identifies analysis of face-pretrained transfer and a reproducible PFN configuration as contributions.Source: §I · p. 2
II. LITERATURE REVIEW
Prior work on partial and occluded face recognition spans local descriptors, visible-region methods, and fine-tuned embedding networks. This study situates its comparison among transfer-learning work while distinguishing it from newer attention and transformer approaches.
II. LITERATURE REVIEW
- Partial-face methods include local-descriptor matching, localization and weighting of visible regions, and fine-tuning face-pretrained embedding networks.The paper identifies the embedding-network group as closest to its own work.Source: §II · p. 2
- Earlier descriptor and embedding studies motivate learned representations: SIFT-based PFR is sensitive to severe occlusion, while FaceNet embeddings degrade predictably under 30% and 40% occlusion.The cited FaceNet result motivates fine-tuning rather than zero-shot use.Source: §II · p. 2
- Region-based approaches localize visible parts, while attentional pooling, transformer recognizers, and margin-based objectives represent a current frontier outside this comparison, [7],.Reported examples include 87.45% true acceptance at 1% false acceptance for Mahbub et al. and 94.96% accuracy on LFW for He et al..Source: §II · p. 2
- Surveys report that face-pretrained backbones consistently outperform ImageNet-pretrained ones on occluded benchmarks,, while attention-based methods are identified as promising.The paper quantifies the pretraining comparison on SoF under one documented protocol.Source: §II · p. 2
- PFN combines a deep embedding backbone and face pretraining with stratified splitting, an enlarged batch, and ReduceLROnPlateau; those stabilization measures were introduced jointly, not separately ablated.The measures address validation behavior reported in,.Source: §II · p. 2
A. Comparative Framework
The comparative framework evaluates ResNet-50, VGG-16, and FaceNet while emphasizing pretraining corpus as the decisive difference. A shared 160x160 input keeps inputs identical but places the ImageNet models below their pretraining resolution.
A. Comparative Framework
- The framework compares ResNet-50, VGG-16, and FaceNet (Inception-ResNet-v1), loading pretrained weights, freezing backbones, and appending a 112-class classification head.The head uses global average pooling, ReLU dense layers, and softmax.Source: A. Comparative Framework · p. 2
- ResNet-50 and VGG-16 use ImageNet pretraining, whereas the evaluated FaceNet checkpoint uses VGGFace2 and triplet-loss identity embeddings.The reported results apply to checkpoint 20180402-114759, trained on about 3.31 million images and 9,131 identities; other FaceNet checkpoints use different corpora.Source: A. Comparative Framework · p. 2
- All models receive 160x160 inputs, native to FaceNet but below the 224x224 pretraining resolution of ResNet-50 and VGG-16.The shared input avoids varying resolution alongside pretraining corpus, but downsampling can remove fine texture detail detected by deeper ImageNet blocks.Source: A. Comparative Framework · p. 3
B. Proposed Architecture
The proposed architecture freezes each selected backbone and trains an appended head for 112-class recognition. The SoF data are identity-stratified and preprocessed into shared training, validation, and test partitions.
B. Proposed Architecture
- Transfer learning freezes the backbone and appends trainable head layers, applying each model to the SoF data and selecting PFN from the comparative evaluation.The pipeline includes cleaning, stratified splitting, training, evaluation, and retuning when validation curves are unsatisfactory; retuning was exercised only for FaceNet.Source: B. Proposed Architecture · p. 3
C. Dataset and Preprocessing
- SoF contains 42,592 images of 112 people, all wearing glasses, with natural eyewear occlusion, synthetic nose and mouth occlusion, and varied illumination.The dataset also includes filters designed to evade detection and graded image difficulty.Source: C. Dataset and Preprocessing · p. 3
- The task is 112-way identity classification, not gender classification, and identity-stratified splitting addresses uneven image counts across identities.Metrics are macro-averaged so sparse identities count as much as dense ones.Source: C. Dataset and Preprocessing · p. 3
- Preprocessing resizes images to 160x160, corrects orientation, crops faces, sharpens detail, and applies gamma correction before partitioning data 60/20/20 by identity.Augmentation is applied only to training; one fixed-seed split is reused across all three backbones.Source: C. Dataset and Preprocessing · p. 3
D. Training Configuration and Hyperparameter Selection
The FaceNet arm received validation-guided changes that the ImageNet arms did not, and evaluation reports four metrics with macro-averaging across identities.
D. Training Configuration and Hyperparameter Selection
- FaceNet received a second run with a doubled batch and epoch budget plus a scheduler, while ResNet-50 and VGG-16 stayed at shared defaults.The changes were introduced together after validation instability, so their separate effects are not isolated.Source: D. Training Configuration and… · p. 3
- Progressive unfreezing, learning-rate scheduling, and longer training are proposed as changes that could materially move the ImageNet-arm results.The arms stopped with fixed learning rates and both were still improving.Source: D. Training Configuration and… · p. 3
E. Evaluation Approach
- Performance is evaluated with accuracy, precision, recall, and F1, which together are informative when error costs differ and class proportions are uneven.Source: E. Evaluation Approach · p. 4
- 84.17% macro-F1 is the unweighted mean of 112 per-class F1 scores, not the 85.81% harmonic mean of reported precision and recall.The 2.9-point gap indicates some identities have jointly poor precision and recall.Source: E. Evaluation Approach · p. 4
IV. RESULTS AND DISCUSSION
The ResNet-50 run ends with accuracy below 0.10, while its validation behavior remains unstable. The available evidence leaves open whether this reflects unusable ImageNet features or insufficient training.
A. ResNet-50 Experiment
- Below 0.10: ResNet-50 training and validation accuracy both end below this level, with validation accuracy oscillating and validation loss near 5.0.Training loss declines monotonically but does not converge with validation loss (4.2).Source: A. ResNet-50 Experiment · p. 4, Figure 1 · p. 4
- The evidence cannot distinguish unusable ImageNet features for 112-class identity discrimination from undertraining of the randomly initialized head.The decreasing training loss is consistent with both readings.Source: A. ResNet-50 Experiment · p. 4
- The figure plots training and validation metric values by epoch for ResNet-50, VGG-16, and two FaceNet runs.Source: Figure 1 · p. 4
B. VGG-16 Experiment
The final FaceNet run reaches 97.4% test accuracy after validation-guided changes, whereas VGG-16 reaches around 6% and its validation curves do not converge.
B. VGG-16 Experiment
- Around 6% test accuracy: VGG-16's training accuracy improves only marginally while validation accuracy and loss fluctuate without converging.Its roughly 23-minute epochs cost about twice as much as ResNet-50.Source: B. VGG-16 Experiment · p. 4
- The two ImageNet experiments agree despite VGG-16 carrying 5.4 times ResNet-50's parameters, with no performance improvement from the added capacity.Under head-only training at 160x160 with shared hyperparameters, the paper argues ImageNet-pretrained features are insufficient for SoF PFR.Source: B. VGG-16 Experiment · p. 4
C. FaceNet Experiment
- 97.4% test accuracy: the final FaceNet run also reports 87.04% precision, 84.61% recall, and 84.17% F1.The improvement from 86% to 97.4% followed three jointly introduced changes and a doubled epoch budget, so their individual effects are not separated.Source: C. FaceNet Experiment · p. 5
D. Validation Fluctuation and Stabilization
PFN leads published SoF baselines in reported accuracy and recall, though its precision is mid-field and its macro-F1 trails the strongest prior baseline. The paper attributes earlier validation instability to data variation and weak face-identity inductive bias, and reports stabilization after jointly applying three remedies.
D. Validation Fluctuation and Stabilization
- Validation oscillations are attributed to class imbalance, perturbation-driven intra-class variation, and weak face-identity inductive bias in ImageNet-pretrained backbones.Shifting validation minibatches and decision boundaries contribute to the instability described.Source: D. Validation Fluctuation and… · p. 5
- Stratified splitting, a doubled batch, and ReduceLROnPlateau were applied jointly; stabilization is attributed to their combination, not separately measured effects.The larger batch is described as reducing gradient-variance standard deviation by √2; the scheduler halves the learning rate on stagnation.Source: D. Validation Fluctuation and… · p. 5
E. Comparison with Prior Work on SoF
- 97.40% accuracy and 84.61% recall are PFN’s highest reported SoF results, while its 87.04% precision is mid-field.The accuracy comparison has only one published baseline, and the table aggregates results from differing protocols.Source: E. Comparison with Prior Work on SoF · p. 5, Table III · p. 5
- 84.61% recall exceeds the published SoF systems listed, but whether that trade-off is favorable depends on application error costs.The text contrasts forensic workflows, where missed matches may end an investigative line, with access control, where false accepts are security failures.Source: E. Comparison with Prior Work on SoF · p. 5
F. Limitations and Scope of the Findings
The findings are bounded by the shared 160x160 input and evaluation on SoF alone, whose 112 identities and occlusion types do not cover several real-world conditions. The paper therefore limits its conclusion to the tested head-only transfer setting and calls for broader evaluation.
F. Limitations and Scope of the Findings
- The 160x160 input is below ResNet-50 and VGG-16 pretraining resolution, leaving an unknown share of their performance gap attributable to resolution.A 224x224 replication is needed to separate resolution effects from pretraining-corpus effects.Source: F. Limitations and Scope of the Findings · p. 5
- SoF covers 112 spectacle-wearing identities from one collection protocol, mainly eyewear and synthetic nose and mouth occlusions.Masks, hands, scarves, extreme pose, and surveillance-scale truncation are excluded; the paper calls for evaluation on additional datasets and real surveillance footage.Source: F. Limitations and Scope of the Findings · p. 5
- The study establishes FaceNet’s advantage only on SoF under head-only transfer at 160x160 with a shared hyperparameter budget.It does not establish whether the gap persists with equal tuning, unfreezing, matched resolution, or other occlusion types and datasets.Source: F. Limitations and Scope of the Findings · pp. 5–6
V. CONCLUSION
On SoF, the study compares three backbones under head-only transfer and finds that FaceNet substantially outperforms the two ImageNet backbones within the tested budget. The result is conditional: whether the gap persists under more extensive tuning, unfreezing, and native-resolution inputs remains unknown.
V. CONCLUSION
- 97.4% test accuracy, 87.04% macro-averaged precision, 84.61% recall, and 84.17% F1 were achieved by FaceNet, while both ImageNet backbones stayed below 10% test accuracy.PFN has the highest reported SoF accuracy and recall, but its macro-F1 is marginally below the strongest prior baseline.Source: §V · p. 6
- The conclusion is conditional on fixed-budget, head-only transfer: the paper does not establish whether the gap persists after tuning, unfreezing, and native-resolution inputs.VGG-16 has 5.4 times ResNet-50’s parameters but gains nothing in the reported setting.Source: §V · p. 6
- Validation instability was addressed with stratified splitting, an enlarged batch, and a reactive learning-rate scheduler, applied jointly without separating their individual contributions.Proposed future work includes per-architecture schedules, progressive unfreezing, a native-224x224 head search, multi-seed repetition, and factorial ablation.Source: §V · p. 6