Source-linked AI summary
Supervised Transformer Network for Efficient Face Detection
Dong Chen, Gang Hua, Fang Wen, Jian Sun
TL;DR
Large pose variations challenge real-world face detection, while accurate DNN detectors can be computationally expensive. The paper proposes an end-to-end Supervised Transformer Network that predicts landmarks, warps candidates to learned canonical positions, and verifies them with a second-stage RCNN. It reports strong benchmark performance and 30 FPS VGA detection on a CPU, with ROI convolution providing a 3x speedup and little recall drop.
Problem
Large pose variations challenge face detection, and prior part-based approaches require faces to be large and clear for reliable part modeling.
Method
An end-to-end cascaded network uses a landmark-predicting RPN, a supervised transformer layer with learned canonical positions, and an RCNN verifier.
Results
The detector outperformed current best-performing algorithms on several public benchmarks and achieved 30 frames per second at VGA resolution.
Takeaways & Limitations
ROI convolution accelerates the detector 3x on CPU with little recall drop, supporting real-time face detection on commodity hardware.
Takeaways & Limitations
Part-based comparison methods remain constrained to faces that are large and clear enough for reliable part modeling.
Abstract
from arXiv · showhide
Large pose variations remain to be a challenge that confronts real-word face detection. We propose a new cascaded Convolutional Neural Network, dubbed the name Supervised Transformer Network, to address this challenge. The first stage is a multi-task Region Proposal Network (RPN), which simultaneously predicts candidate face regions along with associated facial landmarks. The candidate regions are then warped by mapping the detected facial landmarks to their canonical positions to better normalize the face patterns. The second stage, which is a RCNN, then verifies if the warped candidate regions are valid faces or not. We conduct end-to-end learning of the cascaded network, including optimizing the canonical positions of the facial landmarks. This supervised learning of the transformations automatically selects the best scale to differentiate face/non-face patterns. By combining feature maps from both stages of the network, we achieve state-of-the-art detection accuracies on several public benchmarks. For real-time performance, we run the cascaded network only on regions of interests produced from a boosting cascade face detector. Our detector runs at 30 FPS on a single CPU core for a VGA-resolution image.
1 Introduction
Large pose variation limits conventional face detectors, motivating an end-to-end cascaded network that combines landmark-guided alignment with face verification. The proposed system also targets CPU-efficient real-time detection through ROI convolution.
- Large pose variations remain a major challenge because Haar features perform poorly on non-frontal faces.
- The proposed cascade uses a multi-task RPN to generate face regions and landmarks, followed by landmark-based warping and an RCNN verifier.The alignment maps landmarks to canonical positions before second-stage verification.
- The network combines feature maps from both cascade stages and introduces Non-top K suppression to retain promising candidates for later verification.The retained candidates are the top K responses within local neighborhoods.
- Canonical landmark positions and transformations are learned jointly with the detector through end-to-end supervision.The landmark predictor receives annotated landmark supervision, while canonical positions and prediction parameters also receive final classification supervision.
- ROI convolution processes DNN operations only inside candidate regions, reducing CPU computation while preserving nearly the same recall.The authors report a 3x CPU speedup with little recall drop.
- The detector outperformed current best-performing algorithms on several public benchmarks and reached 30 frames per second at VGA resolution.
2 Network Architecture
The network uses a two-stage cascade that proposes faces and landmarks, transforms candidates into a canonical pose, and verifies them with an RCNN. Landmark prediction and canonical positions are learned end-to-end, while features from both stages support final face/non-face arbitration.
- Overview: The first-stage multi-task RPN proposes candidate face regions together with associated facial landmarks and applies non-top K suppression locally.
- Supervised Transformer: The Supervised Transformer warps each candidate into a canonical pose by mapping detected landmarks to learned canonical positions.The transformation is uniquely determined by the detected landmarks and canonical positions.
- Overview: The second-stage RCNN verifies whether transformed candidate regions are true faces or non-faces.
- Supervised Transformer: Landmark prediction receives annotated-landmark supervision, while canonical positions and landmark prediction also receive supervision from the final classification objective.
- Feature combination: The final decision concatenates fine-grained RCNN features with global RPN features before fully connected face/non-face classification.
- Supervised Transformer: The transformation addresses manually selected canonical locations by learning them end-to-end, while bilinear interpolation and chain-rule derivatives support differentiable image warping.The paper identifies manual canonical-location selection as time-consuming and potentially suboptimal.
3 The ROI convolution
The ROI convolution scheme accelerates CPU face detection by restricting DNN computation to candidate face regions identified by a cascade detector. It preserves the detector’s processing structure while reducing redundant and background computation.
- Motivation: Real-time DNN face detection is difficult on CPUs because convolutional layers dominate runtime and commodity systems often lack high-end GPUs.Convolutional layers can account for more than 90% of runtime.
- ROI mask: The ROI mask uses a boosting cascade to identify candidate face areas, then marks those regions for subsequent DNN computation.Candidate regions are grouped by scale to construct the mask.
- ROI convolution: ROI convolution duplicates and processes only input patches whose centers lie in marked regions, skipping other spatial locations.The resulting matrix contains M rows, where M is the number of non-zero mask entries.
- ROI convolution: The ROI convolution cost decreases linearly with mask sparsity, while overlapping candidate regions share features instead of recomputing them independently.This avoids repeated operations in overlapping regions handled separately by conventional RCNN-style processing.
- Results: 30 FPS on CPU was achieved for VGA images with ROI convolution, compared with 10 FPS for the original DNN detector and little accuracy loss.ROI convolution is applied during testing, with the ROI mask downsampled after pooling to match feature-map sizes.
4 Experiments
Experiments evaluate training data, canonical-position learning, component contributions, suppression strategies, ROI acceleration, and benchmark performance. The method converges across initializations, improves recall through complementary components, reaches 30 FPS with a small recall drop, and outperforms prior methods on three datasets.
- Experimental setup: The training set combines about 400K varied positive face images with COCO-derived negatives that exclude covered person regions.Faces are labeled with five facial points, and positive samples are drawn from images separate from the evaluation datasets.
- Learning canonical positions: Three inappropriate canonical-position initializations converge to nearly the same setting after 500K iterations.The result indicates robustness to initialization and automatic adjustment toward rectifications suited to face/non-face classification.
- Ablative evaluation: Multi-task RPN, Supervised Transformer, and feature combination improve FDDB recall by about 1%, 1%, and 2%, respectively.The three components are complementary: removing any one causes a recall drop.
- Ablative evaluation: Non-top K suppression performs consistently better than NMS with the same candidate count and remains close to using all candidates.The visual comparison uses K = 3 candidates.
- The effect of ROI convolution: 30 FPS is achieved on a single-thread CPU for VGA images with ROI convolution, up from 10 FPS, with only a 0.6% recall drop.The experiments use an Intel i7-4770K CPU and report approximately 3x speedup.
- Comparing with state-of-the-art: The detector outperforms previous methods by a considerable margin on FDDB, AFW, and PASCAL faces.The comparison includes deformable-part, cascade-based, commercial, and other public methods.
5 Conclusion and future work
The paper concludes that the Supervised Transformer Network learns canonical positions useful for distinguishing faces from non-faces, while ROI convolution accelerates CPU inference. Future work targets reducing ROI convolution's recall loss.
- Conclusion: The Supervised Transformer Network learns canonical positions that distinguish face/non-face patterns on three challenging datasets.The conclusion links this capability to the method's reported superior performance.
- Future work: ROI convolution speeds the detector up 3x on CPU with little recall drop.The authors identify reducing this recall drop as future work.