PaperlayerView exact revision on arXiv

Source-linked AI summary

Face Detection with End-to-End Integration of a ConvNet and a 3D Model

Yunzhu Li, Benyuan Sun, Tianfu Wu, Yizhou Wang

arXiv:1606.00850v3cs.CV

TL;DR

Face detection in the wild remains challenging amid substantial appearance variation, and generic detection designs require tuning anchor-box ratios and sizes. This paper integrates a ConvNet with a 3D mean-face model and reports competitive state-of-the-art performance on FDDB and AFW, while leaving invisible facial parts unhandled.

  • Problem

    Face detection in the wild remains challenging under appearance variation, while anchor-box ratios and sizes require tuning and are sensitive to unknown test-image distributions.

    Source: §1.1 · p. 2

  • Method

    The method integrates a ConvNet with a 3D mean-face model to generate key-point-based face proposals and uses configuration pooling to exploit object structure.

    Source: §3.2 · p. 8, §5 · p. 13

  • Results

    The method obtained competitive state-of-the-art performance on FDDB and AFW; on FDDB, Ours-Conv3D surpassed 90% recall at 2000 false positives.

    Source: §4.3 · p. 11, §5 · p. 13

  • Takeaways & Limitations

    The authors present the approach as a clean solution that replaces heuristic anchor-box design and generic RoI pooling with 3D-model-based proposals and configuration pooling.

    Source: §5 · p. 13

    The method does not explicitly handle invisible facial parts, which can harm faceness-score calculation and quantitative performance.

    Source: §3.3 · p. 9, §4.3 · p. 12

Abstract

from arXiv · show

This paper presents a method for face detection in the wild, which integrates a ConvNet and a 3D mean face model in an end-to-end multi-task discriminative learning framework. The 3D mean face model is predefined and fixed (e.g., we used the one provided in the AFLW dataset). The ConvNet consists of two components: (i) The face pro- posal component computes face bounding box proposals via estimating facial key-points and the 3D transformation (rotation and translation) parameters for each predicted key-point w.r.t. the 3D mean face model. (ii) The face verification component computes detection results by prun- ing and refining proposals based on facial key-points based configuration pooling. The proposed method addresses two issues in adapting state- of-the-art generic object detection ConvNets (e.g., faster R-CNN) for face detection: (i) One is to eliminate the heuristic design of prede- fined anchor boxes in the region proposals network (RPN) by exploit- ing a 3D mean face model. (ii) The other is to replace the generic RoI (Region-of-Interest) pooling layer with a configuration pooling layer to respect underlying object structures. The multi-task loss consists of three terms: the classification Softmax loss and the location smooth l1 -losses [14] of both the facial key-points and the face bounding boxes. In ex- periments, our ConvNet is trained on the AFLW dataset only and tested on the FDDB benchmark with fine-tuning and on the AFW benchmark without fine-tuning. The proposed method obtains very competitive state-of-the-art performance in the two benchmarks.

1 Introduction

Face detection in the wild remains difficult because nuisance factors create large appearance variations, while generic detection ConvNets rely on heuristic anchors and pooling that ignores object structure. The paper integrates a ConvNet with a 3D mean face model to address these issues and detect faces through learned proposals and facial key-points.

1.1 Motivation and Objective

  • The method targets face-detection challenges from viewpoint, occlusion, expression, resolution, illumination, and cosmetics, alongside anchor-box tuning sensitive to unknown object sizes and aspect ratios.Generic RoI pooling also ignores underlying structural configurations.Source: §1.1 · pp. 2–3
  • A ConvNet and 3D mean face model are integrated in an end-to-end multi-task discriminative learning framework to address the two adaptation issues.The method estimates 3D rotation and translation for facial key-points to generate proposals and predict key-points, and replaces generic RoI pooling with configuration pooling.Source: §1.1 · p. 3, §1.2 · p. 4

1.2 Method Overview

  • The ConvNet represents the 3D mean face with ten corresponding facial key-points and estimates transformations to generate face bounding-box proposals and more accurate key-point predictions.The multi-task objective includes classification Softmax loss and smooth l1 losses for facial key-points and bounding boxes.Source: §1.2 · p. 4
  • The method reports competitive state-of-the-art performance on FDDB and AFW and detects challenging faces, including small, blurry, heavily occluded, and extreme-pose faces.It potentially extends to rigid or semi-rigid categories such as cars when training provides a 3D model and key-point or part annotations.Source: §1.2 · p. 4

2 Related Work

The paper places its approach among face detectors using handcrafted features, deformable parts, ConvNets, and 3D information. Its contribution is an end-to-end ConvNet and 3D-model method that addresses anchor design and generic pooling while reporting competitive results on FDDB and AFW.

Related Work

  • Earlier face detectors include Haar-feature cascades and later handcrafted-feature variants, while more recent systems use deformable part models and ConvNets.Top-performing FDDB detectors are described as ConvNet-based, sometimes combined with cascades or more explicit structure.Source: §2 · p. 5
  • Prior 3D approaches include view-based templates, pose inference, and face alignment; the cited method regresses pose candidates from key-points and verifies them with a parameter-sensitive classifier.The paper characterizes its own integrated ConvNet and 3D-model approach as more straightforward and simpler than.Source: §2 · p. 5

Our Contributions

  • The authors identify three contributions: end-to-end ConvNet and 3D-model integration, addressing anchor-box and RoI-pooling limitations, and competitive performance on FDDB and AFW.The method is presented as simple and effective.Source: §2 · pp. 5–6

3 The Proposed Method

The method uses an AFLW 3D mean face and predicted key-points to form face proposals, then pools and refines them in a multi-task ConvNet. Its training combines key-point classification and localization with bounding-box regression.

3.1 3D Mean Face Model and Face Representation

  • The AFLW mean face has 21 3D key-points, of which the method selects ten; a face is represented by transformation parameters and 2D key-points.The transformation comprises 2D translation, scale, and rotation; projection to 2D means only eight parameters are needed.Source: §3.1 · p. 6
  • Predicted 2D key-points support both ellipse and bounding-box proposals, with the bounding box defined as the minimum upright rectangle covering them.For the ellipse, the method builds a rotated rectangle around the key-points, using the top-of-head-to-chin segment as one axis.Source: §3.1 · p. 7

3.2 The Architecture of Our ConvNet

  • Proposals use the 3D mean face and estimated transformation parameters to generate ten predicted key-points and a corresponding face bounding box.Their score sums the log probabilities of the ten predicted key-points.Source: §3.2 · p. 8
  • After proposal suppression, configuration pooling uses the ten predicted key-points; fully connected layers then regress and refine the bounding boxes.The method uses all ten key-points without accounting for their invisibility in individual faces.Source: §3.2 · p. 8

3.3 The End-to-End Training

  • End-to-end training combines key-point classification and location losses with bounding-box regression; ignoring invisible key-points can double-count faceness, especially for low-resolution faces.The authors report that this issue hurts quantitative performance and leave it for future work.Source: §3.3 · pp. 8–9

4 Experiments

The experiments evaluate AFLW-trained models through intermediate key-point and proposal results, then test face detection on FDDB and AFW. On FDDB, the method is competitive with published methods, with its relative performance depending on the scoring and evaluation setup.

4.1 Experimental Settings

  • AFLW is the sole training dataset, containing 25,993 annotated faces; 70% of its images are used for training and the remainder are reserved for validation.Facial key-points are annotated with respect to a 3D mean face model with 21 landmarks.Source: §4.1 · p. 10

4.2 Evaluation of the intermediate results

  • On the AFLW validation set, key-point classification preserves rough face configurations, which the authors attribute to exploiting the 3D mean face model.Table 1 reports key-point classification accuracy at the end of training; the supplied passage gives no accuracy values.Source: §4.2 · p. 10, Table 1 · p. 10
  • After NMS at IoU 0.7, proposals achieve 93.67% recall with an average of 34.4 proposals per image.The qualitative examples are sampled without NMS, showing 1/30 of the original proposals.Source: §4.2 · p. 10, Figure 3 · p. 11

4.3 Face Detection Results

  • On FDDB, Ours-Conv3D exceeds 90% recall at 2,000 false positives, is slightly behind DP2MFD on discrete scores, and beats all other methods on continuous scores.Discrete evaluation uses face bounding boxes; continuous evaluation uses face ellipses.Source: §4.3 · p. 11, Figure 4 · p. 11, Figure 5 · p. 12
  • AFW results are presented as precision-recall curves, but evaluation uses face proposals without configuration pooling or bounding-box regression because its box annotations differ from FDDB’s.AFW evaluation uses updated annotations for 205 images.Source: §4.3 · p. 12, Figure 6 · p. 13
  • The implementation does not explicitly handle invisible facial parts, which the authors say can harm faceness-score calculation.They identify handling invisibility as future work.Source: §4.3 · p. 12

5 Conclusion and Discussion

The paper concludes that its ConvNet–3D model integration addresses heuristic anchor-box design and generic RoI pooling, with competitive performance on FDDB and AFW. It also identifies ongoing work on faceness-score double-counting and extending the method to other rigid or semi-rigid objects.

5 Conclusion and Discussion

  • The integrated ConvNet and 3D model addresses heuristic anchor-box design and generic RoI pooling, with competitive performance on FDDB and AFW.The conclusion describes configuration pooling as exploiting object configurations.Source: §5 · p. 13
  • Ongoing work addresses faceness-score double-counting and extends the method toward other rigid or semi-rigid classes, such as cars.The authors expect a unified face-and-car model to achieve state-of-the-art performance; this is a future expectation, not a reported result.Source: §5 · p. 13
  • Qualitative detection results are shown for both FDDB and AFW.Source: Figure 7 · p. 14, Figure 8 · p. 14
Loading 1606.00850v3…