Source-linked AI summary
From Facial Parts Responses to Face Detection: A Deep Learning Approach
Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang
TL;DR
Severe occlusion remains a bottleneck in face detection, motivating deep convolutional approaches that can handle it. This paper scores facial-part responses by spatial configuration and reports better performance than previous approaches across FDDB, PASCAL Faces, and AFW.
Problem
The research gap is designing deep convolutional networks for face detection that handle severe occlusion, a bottleneck in the field.
Source: §1 · p. 1Method
The approach generates facial-part response maps with attribute-aware deep networks and scores their spatial configurations to estimate face likelihood.
Source: §1 · p. 2Results
Faceness-Net outperformed previous approaches on FDDB, PASCAL Faces, and AFW; on FDDB, its complete recall was 90.99% versus 85.67% for Cascade-CNN.
Source: §5.3 · p. 8, §6 · p. 8Takeaways & Limitations
The method is reported to detect faces under severe occlusion and large pose variation while achieving practical runtime speed.
Source: §1 · pp. 1–2Further speed-up is possible without much trade-off on detection performance.
Source: §6 · p. 8
Abstract
from arXiv · showhide
In this paper, we propose a novel deep convolutional network (DCN) that achieves outstanding performance on FDDB, PASCAL Face, and AFW. Specifically, our method achieves a high recall rate of 90.99% on the challenging FDDB benchmark, outperforming the state-of-the-art method by a large margin of 2.91%. Importantly, we consider finding faces from a new perspective through scoring facial parts responses by their spatial structure and arrangement. The scoring mechanism is carefully formulated considering challenging cases where faces are only partially visible. This consideration allows our network to detect faces under severe occlusion and unconstrained pose variation, which are the main difficulty and bottleneck of most existing face detection approaches. We show that despite the use of DCN, our network can achieve practical runtime speed.
1. Introduction
The paper revisits face detection with deep convolutional networks, using learned facial-part responses and their spatial arrangement to handle occlusion and pose variation. The resulting detector operates on full images and is reported to achieve state-of-the-art benchmark performance at practical runtime speed.
Contributions
- The detector handles faces with more than half their region occluded and large pose variations, including profile views, without separate viewpoint models.Figure 1 illustrates detection under severe occlusion and pose variation using responses to local facial parts.Source: §1 · p. 1, Figure 1 · p. 1
- The network accepts full images of arbitrary size, with faces at different scales anywhere in the image.The design revisits neural-network face detection following the rise of deep convolutional networks and advances in GPUs; earlier approaches included cascade detectors and deformable part models.Source: §1 · p. 1
- The method scores face likelihood from parts’ spatial arrangement, penalizes inconsistent configurations, and re-ranks generic object proposals into face proposals.It reports over 90% face recall with around 150 proposals—approximately 0.5% of full sliding windows and 10% of generic object proposals.Source: §1 · p. 2
- A second-stage multitask DCN refines face proposals by recognizing true faces and estimating more precise face locations.The preceding stages produce a small set of high-quality bounding-box proposals that cover most faces in an image.Source: §1 · p. 2
- DCNs trained on uncropped face images learn facial-part detectors without part supervision, producing response maps that indicate part locations.The networks are pretrained on generic objects and fine-tuned on part-level binary attributes.Source: §1 · p. 2
- The proposed detector achieves state-of-the-art performance on FDDB, PASCAL Faces, and AFW, with practical runtime speed despite using DCNs.The paper positions its contribution as a DCN-based face detector robust to severe occlusion.Source: §1 · p. 2
2. Related Work
The related work spans neural-network, cascade, deformable-part, and deep-learning face detectors, while motivating a two-stage approach that uses facial-part structure to handle occlusion and pose variation. It also distinguishes face-specific proposals from generic object proposals, which can generate many unhelpful windows and lack a principled way to recover partially visible faces.
Related Work
- Our two-stage detector uses attribute-aware CNNs to localize facial parts and their spatial structure to infer face likelihood, enabling detection under severe occlusion and pose variation.It builds on earlier neural-network and coarse-to-fine detection studies while adding contemporary deep-learning strategies such as pre-training.Source: §2 · p. 2
- Earlier face detectors included cascades and deformable part models, which were more robust to occlusion than cascades; vanilla DPM outperformed more sophisticated variants.The cascade literature includes fast Haar-like features with boosted classification, SURF cascades, and joint detection and alignment.Source: §2 · p. 2
- Unlike the deep-learning detector [6], which lacked an explicit mechanism for heavy occlusion, our first stage handles partial occlusions and our fully convolutional network avoids sliding-window detection.The paper presents this architecture as a way to improve efficiency over the prior sliding-window approach.Source: §2 · p. 2
- Generic object proposals can produce many candidate windows with few faces and lack a principled way to recall partially visible faces, motivating a face-specific faceness measure.The proposed measure aims to achieve high face recall while reducing candidate windows to half the original number.Source: §2 · p. 3
3. Faceness-Net
Faceness-Net combines facial-component response maps with spatial-layout scoring to rank face proposals, then refines them with joint face classification and bounding-box regression. Its component-based design supports localization when some facial parts are occluded.
3. Faceness-Net
- Faceness-Net uses five CNNs to produce facial-part maps, ranks object-proposal windows by faceness, then refines proposals for detection.The CNNs may share deep layers; a final face label map combines the component maps.Source: §3 · p. 3
3.1. Partness Maps Extraction
- Grouped facial-attribute supervision trains component-specific CNNs to produce partness maps, allowing visible parts to localize faces when other parts are occluded.The network stacks seven convolutional layers and two max-pooling layers; ImageNet pretraining is used to reduce erroneous background responses.Source: §3.1 · pp. 3–4, §3 · p. 3
3.2. Ranking Windows by Faceness Measure
- Faceness scores compare partness within learned spatial configurations, with larger scores indicating greater face overlap and integral images enabling efficient computation.The method takes candidate windows from generic proposal generators; its spatial scoring can rank severely occluded faces among the top 50, unlike objectness in the described examples.Source: §3 · p. 3, §3.2 · pp. 4–5
3.3. Face Detection
- A CNN fine-tuned on AFLW faces and PASCAL VOC 2007 person-free images jointly classifies face proposals and regresses bounding boxes.Proposals with IoU above 0.5 are positive for classification; false positives receive a regression target of [−1, −1, −1, −1].Source: §3.3 · p. 5
4. Experimental Settings
The experiments train attribute-aware and face-detection networks on separate datasets, then evaluate part localization, face proposals, and face detection on designated benchmarks. Evaluation uses IoU threshold 0.5 for coverage and detection, with detection rate for proposals and precision and recall for face detection.
Training datasets
- Training uses 75,000 CelebFaces images labeled with 25 facial attributes, plus 13,205 AFLW images and 5,771 person-free PASCAL VOC 2007 images for face detection.CelebFaces contains 87,628 images exclusive of LFW, FDDB, AFW, and PASCAL; the remaining images are reserved for validation.Source: §4 · p. 5
Part response testing dataset
- Part-response localization is evaluated on 2,927 LFW images, using manually labeled hair-and-beard superpixels and landmark-guided eye, nose, and mouth boxes.The hair-and-beard superpixel coordinates generate ground-truth part boxes; eye, nose, and mouth boxes are guided by 68 dense facial landmarks.Source: §4 · p. 5
Face proposal and detection testing datasets
- Face proposal and detection tests use FDDB (2,845 images, 5,171 faces), AFW (205 images, 473 faces), and PASCAL faces (851 images, 1,341 faces).For proposals, FDDB ellipses are converted to minimal bounding rectangles; for detection, the original ellipse ground truth is used.Source: §4 · p. 5
Evaluation settings
- Evaluation fixes IoU at 0.5; proposal effectiveness uses detection rate by proposals per image, while face detection uses standard precision and recall.An object counts as covered or detected by a proposal when IoU is at least 0.5, following the strict PASCAL criterion.Source: §4 · p. 5
5. Results
Faceness-Net outperforms previous face detectors across FDDB, AFW, and PASCAL faces, while partness-based proposals and attribute-driven facial-part responses support robust face proposal generation.
5.1. Evaluating the Quality of Partness Maps
- Part detection performs similarly when the model is trained on uncropped images or cropped face-and-shoulder images.The authors attribute this robustness to facial-attribute-driven training, which encourages the model to learn facial-part representations despite background clutter.Source: §5.1 · p. 6, Table 2 · p. 6
5.2. From Part Responses to Face Proposal
- The proposed method improves on generic objectness measures across IoU thresholds and achieves high recall with few proposals.Table 3 reports the number of proposals needed for different recall levels.Source: §5.2 · p. 6, Table 3 · p. 6, Figure 7 · p. 7
- Hair, eyes, and nose propose face windows more effectively than mouth and beard, while mouth and beard provide complementary cues.The lower face is often occluded, whereas hair, eyes, and nose are visible in most cases.Source: §5.2 · p. 6, Figure 8 · p. 7
- Attribute-driven fine-tuning outperforms face/non-face supervision for face proposals and produces stronger responses on occluded faces.The paper attributes the higher proposal quality to strong response maps on occluded faces.Source: §5.2 · p. 7, Figure 9 · p. 7
5.3. From Face Proposal to Face Detection
- 93% recall is achieved by re-ranking MCG’s top 200 proposals with the faceness measure; MCG’s top 200 alone performs worst, and 1,100 proposals remain inferior.The larger MCG proposal set has a similar high recall rate but produces many false positives.Source: Figure 8 · p. 7, §5.3 · p. 8
- Faceness-Net outperforms previous approaches by a considerable margin on FDDB, AFW, and PASCAL faces, especially on FDDB.The evaluation compares published methods on FDDB and deformable-part and cascade-based methods on AFW and PASCAL faces.Source: Figure 10 · p. 7, Figure 11 · p. 7, Figure 12 · p. 7
6. Discussion
Faceness-Net improves FDDB recall over Cascade-CNN, including under partial occlusion, and offers a faster version with practical runtime. The discussion also identifies techniques that could further accelerate the method without much performance loss.
6. Discussion
- 2.65% higher recall at 167 false positives reflects Faceness-Net’s explicit handling of partial occlusion through facial-part responses.At this fixed false-positive count on FDDB, Cascade-CNN has 85.67% recall and Faceness-Net has 88.32%.Source: §6 · p. 8
- 90.99% complete recall for Faceness-Net compares with 85.67% for Cascade-CNN.This is the complete recall rate reported for the proposed method.Source: §6 · p. 8
- 50ms on a single GPU is the runtime of the two-stage model, which reaches 87% FDDB recall under an aggressive fast setting.The setting shares layers through conv5, replaces MCG with Edgebox, and uses 150 proposals per image; the paper attributes comparatively lower speed than to unoptimized MATLAB code.Source: §6 · p. 8
- Further acceleration may be possible with low-rank expansions or model compression.Jaderberg et al. [11] report a 2.5× CNN speedup with no accuracy loss using low-rank expansions; the authors also identify model compression [8] as a potential benefit.Source: §6 · p. 8