Source-linked AI summary
Multi-view Face Detection Using Deep Convolutional Neural Networks
Sachin Sudhakar Farfade, Mohammad Saberian, Li-Jia Li
TL;DR
Multi-view face detection must handle varied orientations without the annotation and model complexity used by many prior methods. The paper proposes DDFD, a single deep CNN detector with a simplified architecture and no pose or landmark annotations. Across benchmark comparisons, it achieves similar or better performance than more complex prior approaches, while analysis identifies training-distribution effects and heavy occlusion as improvement boundaries.
Problem
Existing multi-view face detectors may require facial-landmark or pose annotations and dozens of models to cover face orientations.
Method
DDFD uses a single deep convolutional neural network for multi-view face detection without pose or landmark annotations or additional segmentation, bounding-box regression, and SVM components.
Results
DDFD achieves similar or better performance than R-CNN and other cascade- and DPM-based face detectors without pose annotation or facial-landmark information.
Takeaways & Limitations
A single simplified CNN can detect faces across a wide range of orientations and handle occlusion to some extent.
Takeaways & Limitations
DDFD still fails on heavily occluded faces, which are underrepresented in the AFLW training set.
Abstract
from arXiv · showhide
In this paper we consider the problem of multi-view face detection. While there has been significant research on this problem, current state-of-the-art approaches for this task require annotation of facial landmarks, e.g. TSM [25], or annotation of face poses [28, 22]. They also require training dozens of models to fully capture faces in all orientations, e.g. 22 models in HeadHunter method [22]. In this paper we propose Deep Dense Face Detector (DDFD), a method that does not require pose/landmark annotation and is able to detect faces in a wide range of orientations using a single model based on deep convolutional neural networks. The proposed method has minimal complexity; unlike other recent deep learning object detection methods [9], it does not require additional components such as segmentation, bounding-box regression, or SVM classifiers. Furthermore, we analyzed scores of the proposed face detector for faces in different orientations and found that 1) the proposed method is able to detect faces from different angles and can handle occlusion to some extent, 2) there seems to be a correlation between dis- tribution of positive examples in the training set and scores of the proposed face detector. The latter suggests that the proposed methods performance can be further improved by using better sampling strategies and more sophisticated data augmentation techniques. Evaluations on popular face detection benchmark datasets show that our single-model face detector algorithm has similar or better performance compared to the previous methods, which are more complex and require annotations of either different poses or facial landmarks.
1. INTRODUCTION
Multi-view face detection is important for organizing large photo collections, but traditional and recent approaches struggle with varied poses, occlusion, annotation demands, or multi-model complexity. DDFD addresses these constraints with a single deep convolutional model that requires neither pose nor landmark annotations and achieves similar or better performance than more complex methods.
- Motivation: Low-complexity, rapid, accurate face detection supports contextual photo queries when images lack explicit identity signals.Photo platforms contain millions of uploaded images, and users often seek photos with particular people.
- Motivation: Cascade detectors accurately find upright visible faces but often fail on side-view or partially occluded faces.Such failures can affect photo-organizing applications because user-generated images commonly contain varied viewpoints and incomplete faces.
- Related work: State-of-the-art cascade systems may use 22 view-specific cascades, requiring face-orientation annotations and complexity that grows with the number of models.The cited approach combines multiple cascades to obtain state-of-the-art multi-view detection.
- Related work: DPM-based detectors tolerate partial occlusion but are computationally intensive and may require multiple models or facial-landmark annotations.Their costs include latent-SVM optimization at candidate locations and training and combining multiple DPMs.
- Related work: Neural-network approaches include two-stage systems, ensembles, joint detection and pose estimation, and single networks that partially handle pose and rotation.DDFD follows prior CNN-based work while constructing a deeper CNN for face detection.
- Motivation: Deep convolutional networks provide the capacity to extract features and classify faces across poses with a single multi-view detector.Earlier features and learning algorithms were described as insufficient for capturing faces at different poses.
- Contribution: DDFD uses deep learning to detect faces across orientations with one model, without pose or landmark annotations.The method also targets minimal complexity relative to recent object-detection systems.
- Contribution: DDFD achieves similar or better performance than R-CNN, cascade-based, and DPM-based methods without pose annotation or facial-landmark information.The comparison includes methods developed specifically for multi-view face detection.
2. PROPOSED METHOD
DDFD fine-tunes a deep CNN and applies it as a simplified sliding-window detector, producing a response heat-map and localizing detections with non-maximal suppression. Detector analysis links confidence scores to training-example distributions, while revealing reduced robustness to heavily occluded faces.
- Architecture: DDFD leverages deep convolutional networks to learn one classifier for multiple face views while simplifying the detector architecture.The method is explicitly designed to minimize computational complexity.
- Training: The detector fine-tunes AlexNet using AFLW face annotations, positive sub-windows with more than 50% IOU, and random horizontal flips.AFLW provides 21K images and 24K face annotations; the supplied passage describes the training-example construction.
- Detection pipeline: A sliding-window implementation reduces complexity and avoids extra modules such as selective search.The paper reports that this approach also outperformed R-CNN in its experiments.
- Detection pipeline: Converting the fully connected layers to convolutional layers lets the eight-layer CNN process images of any size and produce a face-probability heat-map.Each heat-map point corresponds to the CNN response for a 227 × 227 image region.
- Detection pipeline: Non-maximal suppression localizes detected regions, while the network output is used directly instead of an SVM classifier.The authors report that the network output is sufficiently informative for face detection.
- Architecture: Adding bounding-box regression degraded performance, supporting DDFD’s simpler architecture relative to systems using selective search, SVM, or regression.Despite omitting these components, DDFD is reported to achieve state-of-the-art face-detection performance.
- Detector analysis: Detector confidence scores appear correlated with the distribution of positive examples in the training set.The authors propose using this observation to improve sampling and data augmentation.
- Training analysis: Sampling can harm performance when batches omit classes, because the CNN may not learn attributes of underrepresented classes.The paper gives the extreme case of a batch containing no example from a particular class.
3. EXPERIMENTS
Experiments evaluate DDFD’s scanning, suppression, localization, and comparisons with R-CNN and other multi-view face detectors. Results show that sampling, overlap thresholds, and annotation consistency materially affect performance.
- Implementation: DDFD scans whole images using a convolutional heat-map, evaluating 227 × 227 windows with a stride of 32 pixels.The fully connected layers are converted into convolutional layers so images of any size can be processed in one call.
- Benchmark comparisons: DDFD is compared with R-CNN variants, DPM-based detectors, and the cascade-based HeadHunter method on PASCAL Face, AFW, and FDDB.The R-CNN comparison includes versions with and without bounding-box regression and uses NMS variants for final localization.
- Scale selection: Decreasing the scaling factor scans images more finely and increases computation, but the tested choices have little effect on precision and recall.The experiment found that fs = 3√0.5 performed slightly better and was used thereafter.
- Non-maximum suppression: NMS-avg achieves better average precision than NMS-max, with best performance at overlap thresholds of 0.2 and 0.3, respectively.NMS-max retains the highest-scoring window, whereas NMS-avg clusters windows and averages the locations of high-scoring boxes.
- Localization: Adding bounding-box regression degrades performance for both NMS strategies because training and test annotations mismatch, especially for side-view faces.The mismatch can also cause detections to be counted as false misses during evaluation.
4. CONCLUSIONS AND FUTURE WORK
The paper concludes that DDFD detects faces across orientations with one model and without pose or landmark annotations. Its confidence is influenced by skewed training examples, motivating improved sampling and augmentation for rotated and occluded faces.
- Conclusions: DDFD detects faces across a wide range of orientations using a single model without pose or facial-landmark annotations.It also avoids segmentation, bounding-box regression, and SVM components used in common object-detection pipelines.
- Conclusions: The detector achieves similar or better results than more complex multi-view face detectors that use pose annotations or facial landmarks.The comparison includes R-CNN, cascade-based, and DPM-based methods.
- Future work: DDFD detects faces from different angles and handles occlusion to some extent, but its confidence is higher for upright faces because the training set is skewed.The authors propose better sampling and more sophisticated augmentation to improve detection of occluded and rotated faces.