Source-linked AI summary
Selective Refinement Network for High Performance Face Detection
Cheng Chi, Shifeng Zhang, Junliang Xing, Zhen Lei, Stan Z. Li, Xudong Zou
TL;DR
Face detection remains challenging because high-recall detection can produce many false positives and bounding-box locations can remain inaccurate. SRN selectively applies two-step classification and regression in an anchor-based single-shot detector, with receptive-field enhancement for extreme poses. It achieves state-of-the-art performance on AFW, PASCAL face, FDDB, and WIDER FACE.
Problem
Face detection needs improved recall efficiency and location accuracy, particularly because tiny-face anchors create extreme class imbalance and high-recall detection can yield excessive false positives.
Method
SRN selectively uses STC to filter simple negative anchors, STR to refine high-level anchor locations and sizes, and RFE to provide diverse receptive fields.
Results
SRN achieves state-of-the-art results on AFW, PASCAL face, FDDB, and WIDER FACE datasets.
Takeaways & Limitations
Selective two-step classification and regression address false-positive reduction and bounding-box location accuracy within a single-shot face detector.
Abstract
from arXiv · showhide
High performance face detection remains a very challenging problem, especially when there exists many tiny faces. This paper presents a novel single-shot face detector, named Selective Refinement Network (SRN), which introduces novel two-step classification and regression operations selectively into an anchor-based face detector to reduce false positives and improve location accuracy simultaneously. In particular, the SRN consists of two modules: the Selective Two-step Classification (STC) module and the Selective Two-step Regression (STR) module. The STC aims to filter out most simple negative anchors from low level detection layers to reduce the search space for the subsequent classifier, while the STR is designed to coarsely adjust the locations and sizes of anchors from high level detection layers to provide better initialization for the subsequent regressor. Moreover, we design a Receptive Field Enhancement (RFE) block to provide more diverse receptive field, which helps to better capture faces in some extreme poses. As a consequence, the proposed SRN detector achieves state-of-the-art performance on all the widely used face detection benchmarks, including AFW, PASCAL face, FDDB, and WIDER FACE datasets. Codes will be released to facilitate further studies on the face detection problem.
Introduction
Face detection still faces a trade-off between high recall and excessive false positives, as well as insufficient bounding-box location accuracy. SRN addresses these issues with selective two-step classification and regression, plus receptive-field enhancement, and reports state-of-the-art results across four benchmarks.
- High-recall face detection remains limited by excessive false positives and insufficient bounding-box location accuracy.
- Tiny-face detection requires many small anchors, creating extreme class imbalance that contributes to excessive false positives.
- STC filters simple negative samples from low-level layers to reduce the search space for the subsequent classifier.The selected samples include 88.9% of samples in the described setup.
- STR coarsely adjusts high-level anchor locations and sizes to provide better initialization for the subsequent regressor.
- RFE provides more diverse receptive fields for detecting faces in extreme poses.
- SRN achieves state-of-the-art results on AFW, PASCAL face, FDDB, and WIDER FACE datasets.
Related Work
Prior face detectors include handcrafted, CNN-based, and generic object-detection-derived approaches. SRN builds on this progression by selectively combining two-step classification and regression with receptive-field enhancement.
- Earlier face detectors used Haar features and AdaBoost, followed by deformable-part and other handcrafted approaches.
- CNN-based methods introduced cascaded models, joint detection and alignment, and part-based handling of severe occlusion.
- Face detection also adopted generic detectors including Faster R-CNN, SSD, FPN, and RetinaNet.
- SRN selectively applies STC and STR to different detection-layer levels, while RFE enriches receptive fields for extreme-pose faces.
Selective Refinement Network
SRN selectively applies two-step classification and regression across feature-pyramid levels, while RFE diversifies receptive fields for face prediction. Its design targets false positives, bounding-box accuracy, and challenging face appearances.
- Network structure: SRN uses ResNet-50 with a six-level feature pyramid, producing P2–P7 feature maps for detection.
- Selective Two-Step Classification: STC applies two-step classification to low-level C2–C4 and P2–P4 features, filtering simple negative anchors before the subsequent classifier.These levels contain most anchors and have less adequate features.
- Selective Two-Step Regression: STR applies two-step regression to high-level C5–C7 and P5–P7 features, refining anchor locations and sizes before subsequent regression.Applying two-step regression to lower pyramid levels impairs performance because small faces have coarse representations and regression can hinder classification.
- Optimization: The hybrid loss combines focal loss and smooth L1 loss to emphasize hard examples and improve regression learning.
- Selective Two-Step Classification: 38 times: STC increases the positive/negative sample ratio from approximately 1:15441 to 1:404.Both classification steps use focal loss, with the second operating on samples remaining after first-step filtering.
- Receptive Field Enhancement: RFE diversifies receptive fields before class and location prediction through a four-branch structure with a shortcut path.The module replaces the middle two convolution layers in RetinaNet’s class and box subnets.
Training and Inference
SRN is trained on WIDER FACE with augmentation and SGD-based fine-tuning, then performs sequential filtering, refinement, and detection selection during inference.
- Training: WIDER FACE provides 393,703 annotated face boxes in 32,203 images, split into training, validation, and testing sets with Easy, Medium, and Hard difficulty levels.
- Training: SRN optimizes the sum of STC and STR losses using a pretrained ResNet-50 backbone and SGD with momentum and weight decay.
- Inference: During inference, STC filters selected pyramid-level anchors at θ = 0.99, STR adjusts their locations and sizes, and the second step outputs high-confidence detections.Non-maximum suppression with jaccard overlap 0.5 produces the final detections.
Experiments
Experiments evaluate SRN through ablations on WIDER FACE and comparisons across common face detection benchmarks. The results show gains from selective classification, selective regression, and receptive-field enhancement, with strong performance across datasets.
- Ablation Results: STC improves AP from 95.1%, 93.9% and 88.0% to 95.3%, 94.4% and 89.4% on WIDER FACE Easy, Medium and Hard subsets.The module also reduces false positives across different recall rates.
- Ablation Results: STR yields 0.8%, 0.9% and 0.8% AP improvements on the Easy, Medium and Hard subsets over the baseline.Across IoU thresholds of 0.5, 0.6, 0.7 and 0.8, STR produces consistently more accurate localization than the baseline.
- Ablation Results: Combining STC and STR further improves AP to 96.1%, 95.0% and 90.1% on the Easy, Medium and Hard subsets.
- Ablation Results: RFE improves AP by 0.3%, 0.3% and 0.1% on the Easy, Medium and Hard categories, respectively.The paper attributes these gains to diverse receptive fields that help capture faces with varied poses.
- Benchmark Evaluation: SRN achieves 99.87% AP on AFW and improves PASCAL Face AP by 4.99% over the second-best method.On FDDB, it reaches a 98.8% true positive rate at 1,000 false positives.
- Benchmark Evaluation: SRN achieves the best AP on all WIDER FACE validation and testing subsets, including 90.2% validation Hard AP and 89.7% testing Hard AP.The Hard subset contains many small faces.
Conclusion
The paper concludes that SRN combines selective two-step classification and regression with receptive-field enhancement to improve face detection. Experiments on four benchmarks demonstrate state-of-the-art detection performance.
- SRN uses STC to filter simple negative anchors and STR to improve bounding-box initialization and location accuracy.RFE provides diverse receptive fields for capturing faces in extreme poses.
- Experiments on AFW, PASCAL Face, FDDB and WIDER FACE demonstrate state-of-the-art detection performance.