Source-linked AI summary
From Facial Parts Responses to Face Detection: A Deep Learning Approach
Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang
TL;DR
Face detection must handle severe occlusion and unconstrained pose variation, while earlier approaches offered limited solutions to these challenges. The paper proposes a deep convolutional network that scores spatial configurations of facial-part responses, achieving strong benchmark performance and practical runtime speed.
Problem
Face detection remains bottlenecked by severe occlusion and unconstrained pose variation, motivating a deep-learning approach that addresses these cases.
Method
The method generates facial-part response maps with attribute-aware deep networks, scores their spatial configurations, reranks proposals, and refines face hypotheses.
Results
90.99% complete recall on FDDB is reported for the proposed method, compared with 85.67% for Cascade-CNN.
Takeaways & Limitations
Facial-part response scoring supports face detection under severe occlusion and pose variation across FDDB, PASCAL Faces, and AFW.
Takeaways & Limitations
The authors note that further speed-up is possible through low-rank approximations and model compression, indicating runtime remains an optimization target.
Abstract
from arXiv · showhide
In this paper, we propose a novel deep convolutional network (DCN) that achieves outstanding performance on FDDB, PASCAL Face, and AFW. Specifically, our method achieves a high recall rate of 90.99% on the challenging FDDB benchmark, outperforming the state-of-the-art method by a large margin of 2.91%. Importantly, we consider finding faces from a new perspective through scoring facial parts responses by their spatial structure and arrangement. The scoring mechanism is carefully formulated considering challenging cases where faces are only partially visible. This consideration allows our network to detect faces under severe occlusion and unconstrained pose variation, which are the main difficulty and bottleneck of most existing face detection approaches. We show that despite the use of DCN, our network can achieve practical runtime speed.
1. Introduction
The paper revisits face detection with deep convolutional networks and introduces facial-part response scoring to address occlusion and pose variation. The resulting approach targets robust, practical detection across unconstrained images.
- Deep convolutional networks are revisited for face detection after cascade and deformable-part approaches became dominant.
- The proposed network aims to exploit DCN representation learning while handling severe occlusion, a bottleneck in face detection.
- Faces remain detectable when more than half the face is occluded, including large pose variations such as profile views.
- Attribute-aware deep networks generate partness maps from local facial-part responses without prior face detection.
- Faceness scores evaluate spatial arrangements of facial-part responses and rerank candidate windows from generic object proposal generators.
- Faceness-Net achieves state-of-the-art performance on FDDB, PASCAL Faces, and AFW while maintaining practical runtime speed.
2. Related Work
Related work spans early neural face detectors, cascade and deformable-part models, and newer deep-learning approaches. The paper positions its two-stage, part-aware proposal pipeline as addressing limitations of generic proposals and prior DCN detectors.
- Earlier neural-network face detectors used multiple filters, joint pose learning, or coarse-to-fine localization.
- Cascade-based and deformable-part-model detectors dominated face detection, using boosted cascades or collections of deformable facial parts.
- A prior deep-learning detector improved face detection but lacked an explicit occlusion mechanism and failed on heavily occluded faces.
- Figure 2 presents the proposed pipeline for part-response generation, face proposals, and faceness-based bounding-box reranking.
- Generic object proposal generators provide category-independent boxes but are not tailored to faces and can produce enormous numbers of false positives.
3. Faceness-Net
Faceness-Net generates facial-part response maps, ranks candidate windows using learned spatial configurations, and refines proposals with joint classification and bounding-box regression. Its attribute-aware supervision and spatial faceness measure are designed to improve localization under clutter and occlusion.
- Pipeline: Faceness-Net uses three stages: partness-map generation, faceness-based candidate ranking, and face-proposal refinement.The final refinement jointly optimizes face classification and bounding-box regression.
- Partness Maps Extraction: Five CNNs process a full image to produce partness maps indicating locations of facial components such as hair, eyes, and nose.The CNNs may share deep layers to save computational time.
- Ranking Windows by Faceness Measure: Faceness scores evaluate candidate windows by spatial configurations of partness responses, such as comparing upper and lower regions for hair.Learned configurations can be efficiently evaluated using integral images.
- Ranking Windows by Faceness Measure: A larger faceness score indicates a higher overlapping ratio between a candidate window and a face, helping prune false-positive windows.The likelihood of face versus non-face is modeled with a sigmoid function during spatial-configuration learning.
- Face Detection: The proposal-refinement CNN is fine-tuned on AFLW face images and person-free PASCAL VOC 2007 images, using IoU above 0.5 for positive proposals.It uses cross-entropy loss for face classification and Euclidean loss for bounding-box regression.
4. Experimental Settings
The experiments train attribute-aware networks on CelebFaces and train face detection on AFLW plus person-free PASCAL VOC 2007 images, with separate datasets used for part-response testing.
- Training datasets: CelebFaces supplies 87,628 web images, with 75,000 used for training and the remainder for validation after labeling 25 facial attributes.The attributes are divided into five facial-part categories.
- Training datasets: Face-detection training uses 13,205 AFLW images and 5,771 person-free PASCAL VOC 2007 images to provide balanced out-of-plane pose coverage.LFW is separately used to evaluate part-response maps for part localization.
5. Results
The experiments evaluate Faceness-Net’s partness maps, face proposals, training strategies, and detection performance across FDDB, AFW, and PASCAL faces. Results show strong proposal quality, robustness to uncropped inputs, and competitive face detection.
- Robustness to unconstrained training input: Faceness-Net does not require well-cropped faces during training or testing, unlike existing approaches.The comparison uses uncropped images with substantial background clutter and roughly face-and-shoulder crops.
- Contribution of different face parts: Hair, eyes, and nose parts propose face windows more effectively than mouth and beard because the lower face is often occluded.The experiment evaluates each part’s partness map individually under the same proposal protocol.
- Training strategies: Attributes-driven fine-tuning significantly outperforms alternative training strategies for face proposal performance.The resulting response maps remain strong on occluded faces, producing higher-quality proposals than the compared approach.
- Face proposal versus generic proposals: 93% recall is achieved by re-ranking MCG’s top 200 proposals with the faceness measure.Using MCG top 200 proposals alone performs worst, while increasing MCG proposals to 1,100 still yields inferior results because of many false positives.
- Comparison with face detectors: Faceness-Net outperforms published face detection approaches on FDDB, AFW, and PASCAL faces, with the largest margin on FDDB.Figures 10–12 report the benchmark comparisons, while Figure 13 provides qualitative detections.
6. Discussion
Faceness-Net explicitly handles partial occlusion through facial-part responses and achieves a substantial FDDB recall advantage over Cascade-CNN. The method also offers practical runtime efficiency, with further speed-up opportunities identified.
- Comparison with Cascade-CNN: 2.65% higher recall is reported for Faceness-Net than Cascade-CNN at 167 fixed false positives on FDDB.The corresponding recall rates are 88.32% for Faceness-Net and 85.67% for Cascade-CNN.
- Comparison with Cascade-CNN: 90.99% complete recall is reported for Faceness-Net compared with 85.67% for Cascade-CNN.The paper attributes the difference to explicitly inferring face likeliness through part responses for partial occlusion.
- Runtime efficiency: Faceness-Net’s fast version achieves practical runtime efficiency while retaining higher recall than the fast version of Cascade-CNN.The speed-up shares conv1–conv5 across the first-stage ensemble because face-part responses are captured at conv7.
- Runtime efficiency: Further speed-up is possible without much trade-off on detection performance through low-rank expansions and model compression.The paper cites a reported 2.5× CNN speedup with no accuracy loss from low-rank expansion methods.