PaperlayerView exact revision on arXiv

Source-linked AI summary

PyramidBox: A Context-assisted Single Shot Face Detector

Xu Tang, Daniel K. Du, Zeqiang He, Jingtuo Liu

arXiv:1803.07737v2cs.CV

TL;DR

PyramidBox targets small, blurred, and partially occluded faces in uncontrolled environments, where contextual information is especially useful. It introduces context-assisted single-shot detection through PyramidAnchors, LFPN, context-sensitive prediction, and Data-anchor-sampling. The paper reports superior performance over state-of-the-art detectors on FDDB and WIDER FACE.

  • Problem

    Detecting small, blurred, and partially occluded faces in uncontrolled environments remains a challenge, while existing anchors largely focus on face regions and underuse context.

  • Method

    PyramidBox combines semi-supervised PyramidAnchors, LFPN feature fusion, context-sensitive prediction, and scale-aware Data-anchor-sampling in a single-shot detector.

  • Results

    PyramidBox achieves superior performance over state-of-the-art face detectors on FDDB and WIDER FACE.

  • Takeaways & Limitations

    Exploiting contextual parts around faces supports detection of hard faces, particularly through learned contextual features and increased small-face training diversity.

Abstract

from arXiv · show

Face detection has been well studied for many years and one of remaining challenges is to detect small, blurred and partially occluded faces in uncontrolled environment. This paper proposes a novel context-assisted single shot face detector, named \emph{PyramidBox} to handle the hard face detection problem. Observing the importance of the context, we improve the utilization of contextual information in the following three aspects. First, we design a novel context anchor to supervise high-level contextual feature learning by a semi-supervised method, which we call it PyramidAnchors. Second, we propose the Low-level Feature Pyramid Network to combine adequate high-level context semantic feature and Low-level facial feature together, which also allows the PyramidBox to predict faces of all scales in a single shot. Third, we introduce a context-sensitive structure to increase the capacity of prediction network to improve the final accuracy of output. In addition, we use the method of Data-anchor-sampling to augment the training samples across different scales, which increases the diversity of training data for smaller faces. By exploiting the value of context, PyramidBox achieves superior performance among the state-of-the-art over the two common face detection benchmarks, FDDB and WIDER FACE. Our code is available in PaddlePaddle: \href{https://github.com/PaddlePaddle/models/tree/develop/fluid/face_detection}{\url{https://github.com/PaddlePaddle/models/tree/develop/fluid/face_detection}}.

1 Introduction

PyramidBox addresses hard face detection by exploiting contextual regions and combining contextual and facial features in a single-shot framework. Its contributions include PyramidAnchors, LFPN, context-sensitive prediction, and Data-anchor-sampling.

  • Hard faces are difficult to detect because facial features lack visual consistency, while surrounding heads and bodies provide positional clues.
  • Existing face detectors improved scale handling but paid insufficient attention to contextual information for small, blurred, and occluded faces.
  • PyramidAnchors use semi-supervised contextual-part labels and larger anchors to teach the detector features from heads, shoulders, and bodies.
  • LFPN merges high-level contextual semantics with low-level facial features, enabling single-shot detection across face scales.
  • The context-sensitive prediction module combines a wider, deeper network with a max-in-out layer, while Data-anchor-sampling increases hard-sample diversity across scales.
  • PyramidBox reports superior performance over state-of-the-art detectors on the FDDB and WIDER FACE benchmarks.

2 Related Work

Related work develops anchor-based, scale-invariant, and context-associated face detectors. These approaches address varying face scales and incorporate contextual signals, but use different architectural and supervision strategies.

  • Anchor-based Face Detectors: Anchor-based detectors use anchors to support object detection across two-stage and single-shot frameworks, including strategies for scale balance and anchor density.
  • Scale-invariant Face Detectors: Scale-invariant face detectors use different network structures or feature levels to detect faces of varying sizes, often integrating high-level semantics into higher-resolution layers.
  • Context-associated Face Detectors: Context-associated detectors exploit body information, large filters, or anchor-level attention to improve detection of small, blurred, or occluded faces.

3 PyramidBox

PyramidBox is a context-assisted single-shot detector that combines contextual supervision, low-level feature fusion, context-sensitive prediction, and scale-oriented training to detect faces across sizes. Its design targets difficult small, blurred, and partially occluded faces while using only the face branch at inference.

  • Network Architecture: PyramidBox organizes detection around scale-equitable backbone layers, LFPN layers, context-sensitive prediction layers, and a PyramidBox loss layer.The architecture is presented as a single integrated detection pipeline.
  • Low-level Feature Pyramid Layers: LFPN starts its top-down structure from a middle layer, joining high-level semantic and low-level facial features without directly propagating the highest-level features.This design responds to concerns that top-level features may contain noise for small faces.
  • Pyramid Detection Layers: The detector predicts faces at anchor sizes 16, 32, 64, 128, 256, and 512 using LFPN and backbone detection layers.The selected layers are lfpn 2, lfpn 1, lfpn 0, conv fc 7, conv 6 2, and conv 7 2, respectively.
  • Context-sensitive Prediction: The Context-sensitive Predict Module combines wider and deeper processing with max-in-out classification to improve localization and classification from merged features.Max-in-out selects maximum positive and negative scores, with different positive-to-negative settings for the first and later prediction modules.
  • PyramidAnchors: PyramidAnchors generate contextual targets for face, head, and body regions to supervise contextual features associated with faces at different scales.The method uses anchor-region matching and approximate contextual supervision rather than requiring extra labels.
  • Training and Results: Data-anchor-sampling increases the proportion and diversity of small-face training samples, while PyramidBox reports better handling of small, blurred, and partially occluded faces without extra runtime cost.During prediction, only the face branch is used, so the contextual branches add no computational cost compared with standard anchor-based face detectors.

4 Experiments

Experiments use contrast studies and benchmark evaluations to assess PyramidBox and its components. The model improves WIDER FACE performance, especially on the hard subset, and achieves state-of-the-art results on FDDB.

  • Model Analysis: The experiments compare PyramidBox components on the WIDER FACE validation set using S3FD as the baseline architecture.The study evaluates LFPN, Data-anchor-sampling, PyramidAnchors, PyramidBox loss, and the context-sensitive prediction module.
  • Model Analysis: LFPN improves hard-subset mAP by 1.9% over the compared baseline configuration.Starting LFPN from the middle conv fc7 layer is reported as more powerful than using features with larger scale gaps.
  • Model Analysis: Data-anchor-sampling increases mAP by 0.4%, 0.4%, and 0.6% on the easy, medium, and hard subsets, respectively.The reported gains cover easy and medium faces as well as small hard faces.
  • Model Analysis: PyramidAnchors improve performance by 0.7%, 0.6%, and 0.9% on the easy, medium, and hard subsets, respectively.The comparison attributes the improvement to learning contextual information, particularly for hard faces.
  • Model Analysis: The context-sensitive prediction module outperforms the DSSD and SSH alternatives, while Max-in-out adds +0.2%, +0.3%, and +0.1% mAP on easy, medium, and hard subsets.The comparison indicates that a larger receptive field is important for accurate location and classification.
  • Benchmark Evaluation: PyramidBox increases mAP by 2.1%, 2.3%, and 4.7% on the easy, medium, and hard WIDER FACE validation subsets, respectively.The largest reported gain is on the hard subset.
  • Benchmark Evaluation: On FDDB, PyramidBox achieves state-of-the-art performance against other state-of-the-art methods.FDDB contains 5,171 faces in 2,845 images.
  • Benchmark Evaluation: On WIDER FACE, PyramidBox outperforms other detectors across easy, medium, and hard validation and testing subsets.Validation mAP values are 0.961, 0.950, and 0.889; testing values are 0.956, 0.946, and 0.887.

5 Conclusion

PyramidBox is a context-assisted single-shot detector for unconstrained face detection. Its contextual anchors, LFPN, wider prediction module, and data augmentation support state-of-the-art benchmark performance, especially for hard faces.

  • Conclusion: PyramidBox uses PyramidAnchors to supervise contextual features around faces, LFPN to combine high-level and high-resolution features, and a wider, deeper prediction module.Data-anchor-sampling increases training-data diversity for small faces.
  • Conclusion: Experiments report state-of-the-art performance on common face-detection benchmarks, especially for hard faces.The conclusion attributes the result to the paper’s proposed contributions.

Appendix

The appendix illustrates PyramidBox robustness across face scale, blur, pose, and occlusion. Examples show detection confidence and substantial recall in highly variable images.

  • Robustness Examples: PyramidBox recalls most faces in images containing small, blurred, partially occluded, and unusually expressive faces.One example contains extreme variability in face regions.
  • Robustness Examples: In one extreme-variability image, PyramidBox finds 880 of the reportedly 1,000 present faces.Detector confidence is displayed directly by a colorbar.
  • Robustness Examples: The appendix separately depicts robustness to a wide range of face scales, blur, pose, and occlusion.Blue denotes detector confidence above 0.8 in the corresponding figures.
Loading 1803.07737v2…