PaperlayerView exact revision on arXiv

Source-linked AI summary

Faceness-Net: Face Detection through Deep Facial Part Responses

Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang

arXiv:1701.08393v3cs.CV

TL;DR

Face-box supervision is standard, while facial attributes have been little explored for face detection. Faceness-Net discovers part responses from attribute-trained CNNs and scores their spatial arrangement to find faces; it reports promising performance across several benchmarks.

  • Problem

    Facial attributes remain little explored as supervision for face detection, although face bounding boxes are a standard supervisory source.

    Source: §1 · p. 1

  • Method

    Faceness-Net uses attribute-trained CNNs to discover facial-part responses and scores their spatial arrangement to rank face proposals.

    Source: §1 · pp. 1–2

  • Results

    Faceness-Net reports promising performance on FDDB, PASCAL Faces, AFW, and WIDER FACE.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The faceness score generates a modest set of high-recall proposals, and the resulting detector is reported to handle severe occlusion and pose variation.

    Source: §1 · p. 1, §7 · p. 12

    Tiny faces remain difficult because their facial parts can be barely distinguished, making attribute-aware responses meaningless.

    Source: §5.3 · p. 11

Abstract

from arXiv · show

We propose a deep convolutional neural network (CNN) for face detection leveraging on facial attributes based supervision. We observe a phenomenon that part detectors emerge within CNN trained to classify attributes from uncropped face images, without any explicit part supervision. The observation motivates a new method for finding faces through scoring facial parts responses by their spatial structure and arrangement. The scoring mechanism is data-driven, and carefully formulated considering challenging cases where faces are only partially visible. This consideration allows our network to detect faces under severe occlusion and unconstrained pose variations. Our method achieves promising performance on popular benchmarks including FDDB, PASCAL Faces, AFW, and WIDER FACE.

1 INTRODUCTION

Facial-attribute supervision reveals facial-part responses in uncropped images, which Faceness-Net uses to propose and refine faces. The approach is designed for occlusion and pose variation and reports promising benchmark performance.

1 INTRODUCTION

  • Facial-attribute supervision is presented as a way to address severe occlusion and large pose variation, including profile faces without separate viewpoint-specific models.The introduction contrasts these results with what face-box supervision alone can achieve, especially with limited scene diversity and pose variation in training data.Source: §1 · p. 1
  • Attribute-trained CNNs can reveal facial-part detectors from uncropped images without explicit part supervision.Their deep layers generate responses indicating face-part locations.Source: §1 · p. 1
  • Spatially arranged part responses produce faceness scores for ranking proposals; around 150 proposals achieve over 90% face recall on FDDB.The score penalizes inconsistent configurations, such as hair below the eyes or mouth above the nose.Source: §1 · p. 2
  • Faceness-Net reports promising performance on FDDB, PASCAL Faces, AFW, and WIDER FACE despite training on datasets not targeted for face detection.The stated training datasets are CelebA for face recognition and AFLW for face alignment.Source: §1 · p. 2
  • The revised design shares low- and mid-level features, uses attribute-aware networks to generate proposals directly, and has about 83% fewer parameters than the earlier version.The paper also reports improved performance and speed for the revised design.Source: §1 · p. 2

2 RELATED WORK

Earlier face-detection approaches include neural networks, cascades, deformable part models, and more recent deep-learning frameworks. The paper distinguishes Faceness-Net by using attribute-aware part responses and their spatial structure to score face proposals, including partially visible faces.

2 RELATED WORK

  • Faceness-Net uses attribute-aware networks to localize facial parts and their spatial structure to infer face likeliness, targeting faces with both pose variation and occlusion.The paper contrasts this with earlier occlusion methods confined to frontal faces.Source: §2 · p. 2
  • Earlier approaches include neural-network detectors, cascade methods, and deformable part models, with DPMs described as more robust to occlusion than cascades.The paper notes that neural-network work also combined face detection with pose estimation.Source: §2 · p. 2
  • Recent deep face detectors use cascades or landmark supervision, while one cited network lacks an explicit mechanism for heavy occlusion.The authors present facial attributes as another supervision source for robust face detection.Source: §2 · p. 2
  • Generic object proposals and RPNs do not account for facial structure, leaving no mechanism to recall partially visible faces; faceness scoring is proposed to address this gap.The paper says the measure reduces candidate windows to half the original number compared with the original RPN.Source: §2 · p. 3

3 FACENESS-NET

Faceness-Net generates facial-part response maps, ranks candidate windows by the spatial arrangement of those responses, and refines top proposals with a multitask detector.

3 FACENESS-NET

  • The two-stage pipeline ranks candidate windows using partness-based faceness scores, then refines the top proposals with face classification and bounding-box regression.Part-specific spatial configurations contribute to the score, and non-maximum suppression reduces the proposal set.Source: §3 · p. 3, §3.5 · p. 7

3.1 Attribute-Aware Networks

  • Conv7 responses produce partness maps, while shared early layers and reduced filters give Faceness-Net-SR 83% fewer parameters than five separate attribute-aware networks.The conv7 maps are channel-averaged after L2 normalization and upsampled through unpooling to input-image size.Source: §3.1 · p. 4, Figure 4 · p. 4

3.2 Learning to Generate Partness Maps

  • Facial-part attribute groups train part detectors without explicit part supervision, allowing other parts to help localize a face when one part is occluded.Faceness-Net-SR maps are noisier but capture subtle part responses and yield higher proposal recall when enough proposals are retained; shared networks are trained directly on facial attributes, while separate networks can use ImageNet pretraining.Source: §3.2 · pp. 4–5

3.3 Generating Candidate Windows

  • Candidate windows can come from generic proposal methods or templates centered on selected partness-map locations, with ten scales and one aspect ratio per location.Template parameters are selected by grid search to maximize recall for an average proposal count.Source: §3.3 · pp. 5–6

3.4 Ranking Windows by Faceness Score

  • Faceness scores assess whether part responses fit learned spatial configurations, using response-region ratios and MAP estimation to rank windows.The likelihood measures face-versus-nonface confidence, and the learned configuration parameters can be obtained by line search.Source: §3.4 · p. 6

3.5 Face Detection

  • The second-stage multitask CNN classifies face proposals and regresses their boxes, reusing attribute-network features that converge faster than ImageNet-pretrained features.Attribute-pretrained features require 45,000 iterations to converge, versus more than 200,000 for ImageNet pretraining; regression targets are normalized to [−1, 1].Source: §3.5 · p. 7

4 EXPERIMENTAL SETTINGS

The section describes the training data, part-response and face-detection evaluation sets, evaluation protocols, and four Faceness-Net variants.

Training datasets

  • 180,000 of CelebA’s 202,599 images train the attribute-aware networks; 13,205 AFLW face images were selected, and re-annotation increased AFLW faces from 24,386 to 29,133.CelebA provides 40 facial attributes, of which 25 were selected and grouped into five facial-part categories; AFLW selection aimed for balanced out-of-plane pose distribution.Source: §4 · p. 7

Part response test dataset

  • LFW part-response quality is evaluated on 2,927 images, with hair and beard boxes based on prior annotations and eye, nose, and mouth boxes derived from 68 facial landmarks.The original LFW dataset lacks part-level bounding boxes, so the authors label boxes using these schemes.Source: §4 · p. 7

Face proposal and detection test datasets; Evaluation settings

  • FDDB, AFW, PASCAL Faces, and WIDER FACE serve as face-proposal and detection test sets, with IoU 0.5, detection rate for proposals, and precision-recall for face detection.FDDB ellipses are converted to minimal bounding rectangles for proposal evaluation; original ellipses are used for detection evaluation.Source: §4 · pp. 7–8

Faceness-Net Variants

  • Four variants combine five or shared attribute-aware networks with external generic proposals or the template proposal technique.The variants are Faceness-Net, Faceness-Net-SR, Faceness-Net-TP, and Faceness-Net-SR-TP.Source: §4 · p. 8

5 RESULTS

The experiments evaluate attribute-aware networks, face proposal strategies, and detection across four benchmarks. Results highlight strong benchmark performance alongside trade-offs in proposal count, training data, and handling tiny faces.

5.1 Evaluating the Attribute-Aware Networks

  • Part detection performs similarly with cropped and uncropped training images; sharing low- and mid-level representations raises detection rate but requires more proposals.The shared network’s stronger responses to subtle parts support later face detection, while its weaker background modeling hurts performance with few proposals.Source: §5.1 · pp. 8–9, Table 2 · p. 8
  • Fine-tuning with part-level attributes significantly outperforms fine-tuning with face images and non-face images or with 25 face attributes for face proposals.Figure 11 compares the proposal performance of the three strategies.Source: §5.1 · p. 9, Figure 11 · p. 9

5.2 From Part Responses to Face Proposal

  • Faceness-Net and Faceness-Net-SR improve generic proposal baselines across IoU thresholds; Faceness-Net leads below 300 proposals, while template proposals exceed 96.5% detection at high recall.Hair, eye, and nose cues outperform mouth and beard individually; combining all parts performs better than any part alone.Source: Figure 10 · p. 8, §5.2 · pp. 9–10, Figure 12 · p. 9

5.3 From Face Proposal to Face Detection

  • 98.05% AP: Faceness-Net-SP-TP achieves the best AFW performance among the Faceness-Net variants, which outperform all compared approaches.Figure 14 shows the AFW precision-recall curves.Source: §5.3 · p. 10, Figure 14 · p. 10
  • On PASCAL faces, baseline Faceness-Net outperforms its variants and other compared detectors; the authors attribute this to stronger proposals when fewer than 200 are used.The dataset has fewer faces per image, so fewer proposals are needed for high recall.Source: §5.3 · p. 10, Figure 15 · p. 10
  • On FDDB, Faceness-Net-SR-TP improves baseline recall by 1.85%, while Faceness-Net-SR and Faceness-Net-TP improve it by 0.4% and 0.7%, respectively.The baseline recall is 90.99% under the discrete score; with the same AFLW and CelebA training sets, Faster-RCNN reaches 92.09%, marginally below Faceness-Net.Source: §5.3 · pp. 10–11, Figure 16 · p. 11, Figure 18 · p. 11
  • In WIDER FACE’s external setting, Faceness-Net variants outperform the baseline on Easy, Medium, and Hard, but remain behind recent methods trained on WIDER FACE.The authors note their training data are CelebA and AFLW, not WIDER FACE.Source: §5.3 · pp. 10–11
  • The benchmark gap is bounded by simpler backgrounds, narrower scale and pose variation, and 17k training boxes; the method does not explicitly handle tiny faces.The discussion also identifies facial landmarks as stronger supervision and tiny faces, as small as 20 pixels high, as a difficult unresolved case.Source: §5.3 · p. 11

6 RUNTIME ANALYSIS

The runtime analysis reports Faceness-Net-SR-TP’s GPU timing and compares its recall with Cascade-CNN. The comparison includes both an aggressive test-image setting and complete recall rates.

6 RUNTIME ANALYSIS

  • Faceness-Net-SR-TP runs in 40 ms on a single GPU, including 10 ms for faceness proposals when the test image is no more than 300 pixels high.The runtime is measured on an Nvidia Titan X GPU.Source: §6 · p. 11
  • At 167 false positives on FDDB, Faceness-Net-SR-TP reaches 90.33% recall versus 85.67% for Cascade-CNN, a 4.66% margin.Under this aggressive setting, the authors describe their method as practically efficient.Source: §6 · p. 12
  • Complete FDDB recall is 92.84% for Faceness-Net-SR-TP and 85.67% for Cascade-CNN; Cascade-CNN’s fast version runs at 14 fps on CPU or 100 fps on GPU for VGA images.Faceness-Net-SR-TP is reported to have 0.21% higher recall than Cascade-CNN under the aggressive setting.Source: §6 · p. 12

7 CONCLUSION

The conclusion presents attribute supervision as a way to discover facial-part responses without explicit part supervision. It uses those responses and their spatial arrangements to score and re-rank face proposals, supporting a detector with promising benchmark performance.

7 CONCLUSION

  • CNNs trained to recognize attributes from uncropped face images can develop face-part detectors without explicit part supervision.The paper reports that these responses indicate facial-part locations.Source: §7 · p. 12
  • Faceness scores analyze facial-part responses and their spatial arrangements, then re-rank candidate windows from region-proposal techniques into face proposals.Source: §7 · p. 12
  • The resulting proposals are used to train a face detector that achieves promising performance on multiple face-detection benchmarks.Source: §7 · p. 12

(c) Scale (d) Missing annotations

Figure 22 labels ground-truth boxes in green, detections in blue, and detected faces absent from the annotations in red.

(c) Scale (d) Missing annotations

  • Figure 22 uses green for ground truth, blue for detections, and red for detected faces that are not annotated.Source: Figure 22 · p. 14
Loading 1701.08393v3…