Source-linked AI summary
A Deep Pyramid Deformable Part Model for Face Detection
Rajeev Ranjan, Vishal M. Patel, Rama Chellappa
TL;DR
Unconstrained face detection must handle clutter and wide variation in facial appearance and pose. DP2MFD trains DPMs on deep feature pyramids with CNN normalization, and experiments on four datasets report strong performance, including state-of-the-art results. Its localization remains constrained by CNN subsampling and bounding-box mismatch in some evaluations.
Problem
Face detection performance degrades in unconstrained images with cluttered backgrounds and variation in viewpoint, expression, skin color, occlusion, and cosmetics.
Method
DP2MFD generates normalized deep feature pyramids and uses a DPM with linear SVM classification to detect faces at pyramid locations.
Results
The method achieves new state-of-the-art detection performance on four challenging face detection datasets and significantly outperforms previous detectors under FDDB’s discrete protocol.
Takeaways & Limitations
Deep feature pyramids combined with DPMs can support unconstrained face detection across different face sizes, poses, and cluttered backgrounds.
Takeaways & Limitations
Localization is limited by CNN subsampling, while FDDB continuous-protocol performance decreases when rectangular detections are matched against elliptical ground-truth masks.
Abstract
from arXiv · showhide
We present a face detection algorithm based on Deformable Part Models and deep pyramidal features. The proposed method called DP2MFD is able to detect faces of various sizes and poses in unconstrained conditions. It reduces the gap in training and testing of DPM on deep features by adding a normalization layer to the deep convolutional neural network (CNN). Extensive experiments on four publicly available unconstrained face detection datasets show that our method is able to capture the meaningful structure of faces and performs significantly better than many competitive face detection algorithms.
1. Introduction
Unconstrained face detection remains difficult because real images contain substantial variation in backgrounds, viewpoints, expressions, skin color, occlusions, and cosmetics. DP2MFD combines deep pyramidal features with DPMs and adds CNN normalization to improve detection across face sizes and conditions.
- Unconstrained images challenge face detectors through cluttered backgrounds and large variations in viewpoint, expression, skin color, occlusion, and cosmetics.
- Most recent face detectors use Deformable Parts Models, which represent a face as parts connected by spring-like constraints.
- Traditional Haar and HOG features may miss salient facial information across poses and illumination conditions, motivating deep CNN feature extraction.
- DP2MFD integrates deep pyramid features with DPMs to detect faces across multiple scales, poses, and occlusions.
- The method trains DPMs on deep pyramidal features and adds a CNN normalization layer to reduce bias in face sizes.
- The authors report new state-of-the-art detection performance on four challenging face detection datasets.
2. Face Detection with Deep Pyramid DPM
DP2MFD builds a seven-level normalized deep feature pyramid and applies root-only DPM detection through sliding-window features and linear SVMs. Z-score normalization reduces face-size bias across pyramid levels, while training uses pyramid-aligned positives, IoU-filtered negatives, hard-negative mining, and bounding-box regression.
- Feature pyramid: DP2MFD generates a seven-level normalized deep feature pyramid for arbitrary-size input images, then extracts fixed-length features using sliding windows.The pyramid is built from 256 feature maps at the conv5 layer.
- Feature pyramid: Z-score normalization is applied independently at each pyramid level, producing norm5 features on which a root-only DPM is trained with a linear SVM.The normalization uses the level-specific mean feature vector and standard deviation.
- Detection: At test time, DPM root filters convolve over the norm5 pyramid, thresholded locations map to image regions, and greedy non-maximum suppression prunes detections.The subsampling factor reduces sliding windows to approximately 25k versus approximately 250k for the HOG pyramid.
- Training: Training samples are aligned to an optimal pyramid level, with resized positive boxes and negative boxes restricted to IoU below 0.3.The optimal level minimizes the sum of absolute bounding-box coordinate differences, and positives use h×w×256 norm5 features.
- Training: The system uses hard-negative mining for linear SVM training and trains a norm5-based bounding-box regressor for localization.The regressor follows the R-CNN procedure while using norm5 features.
3. Experimental Results
DP2MFD was evaluated across AFW, FDDB, MALF, and IJB-A, including comparisons with academic and commercial detectors and fine-grained face subsets. It generally performed strongly, while bounding-box localization limited performance at stricter overlap thresholds.
- Evaluation setup: DP2MFD was evaluated on AFW, FDDB, MALF, and IJB-A using comparisons with academic and commercial face detectors.The detector was trained on FDDB images with one- and two-component variants.
- AFW results: On AFW, DP2MFD outperformed most academic detectors and performed comparably to a joint cascade method and Google Picasa.DP2MFD used no alignment procedure and trained on 2,500 images, whereas the joint cascade method used alignment and 20,000 images.
- FDDB results: On FDDB, DP2MFD significantly outperformed previous academic and commercial detectors under the discrete protocol and performed comparably to prior state-of-the-art detectors under the continuous protocol.The authors attribute the continuous-protocol decrease mainly to matching rectangular detections with elliptical ground-truth masks.
- MALF results: On MALF, DP2MFD performed best among academic algorithms overall and on small faces, but ranked second on large, easy, and hard subsets.DPM and Headhunter performed better on those latter subsets after training multiple orientation models and more than 20,000 samples.
- Limitations: Performance decreased at IoU 0.5 because of improper bounding-box localization, potentially related to CNN subsampling.The detector used non-maximum suppression with an IoU threshold of 0.3, and the authors identified localization as a future analysis target.
- IJB-A results: On IJB-A, DP2MFD outperformed the best reported method by a large margin and detected profile and differently sized faces in cluttered backgrounds.The evaluation included precision-recall and ROC curves against one of the best methods reported for the dataset.
4. Conclusions
The paper presents DP2MFD for unconstrained face detection by training deformable part models on a deep feature pyramid, with normalization to reduce bias in face sizes. Experiments on four publicly available unconstrained datasets demonstrate the approach's effectiveness, while future work targets faster computation and broader evaluation.
- 4. Conclusions: DP2MFD trains deformable part models for faces on a deep feature pyramid and adds CNN normalization to reduce bias in face sizes.The method is designed for unconstrained face detection.
- 4. Conclusions: Extensive experiments on four publicly available unconstrained face detection datasets demonstrate the effectiveness of the proposed approach.
- 4. Conclusions: Future work includes implementing the method on a GPU to reduce computing time and evaluating it on other object detection datasets.