PaperlayerView exact revision on arXiv

Source-linked AI summary

SSH: Single Stage Headless Face Detector

Mahyar Najibi, Pouya Samangouei, Rama Chellappa, Larry Davis

arXiv:1708.03979v3cs.CV

TL;DR

Face detection must handle small faces while controlling the speed and memory costs of deep, multi-stage or image-pyramid detectors. SSH performs single-stage, headless detection from early convolutional layers and detects multiple scales from different depths in one forward pass. It achieves state-of-the-art performance across WIDER, FDDB, and Pascal-Faces while reducing detection time.

  • Problem

    Small-face detection remains difficult, and accurate detectors can require substantial computation, memory, or image-pyramid processing.

  • Method

    SSH jointly classifies and regresses predefined anchors from early convolutional layers, removes the classification-network head, and uses modules at different strides for different face scales.

  • Results

    SSH achieves state-of-the-art performance on WIDER, FDDB, and Pascal-Faces while reducing detection time considerably.

  • Takeaways & Limitations

    A headless, single-stage network can detect multiple face scales in one forward pass while remaining fast and lightweight.

  • Takeaways & Limitations

    Competing image-pyramid methods may process inputs upsampled to as much as 5000 pixels per side through a deep network, increasing inference time.

Abstract

from arXiv · show

We introduce the Single Stage Headless (SSH) face detector. Unlike two stage proposal-classification detectors, SSH detects faces in a single stage directly from the early convolutional layers in a classification network. SSH is headless. That is, it is able to achieve state-of-the-art results while removing the "head" of its underlying classification network -- i.e. all fully connected layers in the VGG-16 which contains a large number of parameters. Additionally, instead of relying on an image pyramid to detect faces with various scales, SSH is scale-invariant by design. We simultaneously detect faces with different scales in a single forward pass of the network, but from different layers. These properties make SSH fast and light-weight. Surprisingly, with a headless VGG-16, SSH beats the ResNet-101-based state-of-the-art on the WIDER dataset. Even though, unlike the current state-of-the-art, SSH does not use an image pyramid and is 5X faster. Moreover, if an image pyramid is deployed, our light-weight network achieves state-of-the-art on all subsets of the WIDER dataset, improving the AP by 2.5%. SSH also reaches state-of-the-art results on the FDDB and Pascal-Faces datasets while using a small input size, leading to a runtime of 50 ms/image on a GPU. The code is available at https://github.com/mahyarnajibi/SSH.

1. Introduction

Face detection remains challenging for small faces and for detectors that must balance accuracy with speed and memory efficiency. SSH addresses these constraints with a single-stage, headless, scale-invariant design that detects multiple face sizes from different network layers.

  • Motivation: Small-face detection remains challenging, while speed and memory efficiency are important practical constraints.WIDER exposed a performance gap between humans and current detectors, and strong detectors are often slow and memory-intensive.
  • Limitations of Existing Detectors: Image-pyramid approaches improve robustness to input scale but can be slow because they process enlarged pyramid levels through a deep network.The cited approach upsamples images to as much as 5000 pixels per side and passes each level through the network.
  • SSH: SSH performs single-stage detection by jointly classifying and regressing anchors from early convolutional feature maps, without the classification-network head.Removing the fully connected layers makes the detector lightweight while retaining state-of-the-art results.
  • SSH: SSH is scale-invariant by design, detecting different face scales from network layers with different strides in one forward pass instead of using an external image pyramid.Detection modules are placed on layers selected for different face-scale ranges.

2. Related Works

Related work includes single-stage detectors, proposal networks, multi-scale feature processing, and context modeling. SSH combines these ideas for face detection while avoiding proposal-stage separation and expensive duplicated or pyramid-based computation.

  • Single-Stage Detectors and Proposal Networks: Single-stage detectors such as SSD and YOLO jointly classify fixed boxes and regress them toward objects, whereas many state-of-the-art COCO detectors remain two-stage.SSH applies single-stage detection directly to faces from early convolutional layers without a proposal stage.
  • Single-Stage Detectors and Proposal Networks: Proposal methods regress predefined anchors toward objects and assign objectness scores; SSH uses similar anchors to localize and detect faces simultaneously.Unlike generic proposal methods, SSH combines localization with face detection.
  • Scale-Invariant Detection: Prior face detectors address scale variation using earlier feature maps, skip connections, multiple heads, or an input image pyramid.The image-pyramid approach processes differently resized inputs separately, while SSH processes multiple face scales in one forward pass.
  • Context Modeling: Context modeling in prior two-stage detectors can enlarge proposal windows or use recurrent networks, but larger windows may duplicate classification heads and increase memory and detection time.SSH uses simple convolutional layers to achieve a larger-window effect more efficiently.

3. Proposed Method

SSH is a fully convolutional, single-stage, headless face detector that classifies and localizes predefined anchors from multiple convolutional layers. It detects different face scales in one forward pass and incorporates context through convolutional modules.

  • General architecture: SSH performs face classification and localization together in a single stage, rather than separating proposal generation from classification.
  • General architecture: SSH removes the fully connected head of its underlying classification network, producing a lightweight detector.
  • General architecture: Detection modules on feature maps with strides 8, 16, and 32 use convolutional classification and regression branches to detect and localize faces.
  • Anchors and detection modules: SSH uses predefined, dense, aspect-ratio-one anchors and regresses them toward ground-truth faces.
  • Scale invariance: Multiple detection modules process different face scales simultaneously in one forward pass, then combine predictions with non-maximum suppression.
  • Context modeling: Context modules use convolutional filters to enlarge the effective receptive field without duplicating a classification head.
  • Training: Training uses separate multi-task classification and regression losses for the detection modules, with online hard negative and positive mining applied separately to each module.

4. Experiments

The experiments describe the training setup and evaluate SSH on WIDER, FDDB, and Pascal Faces, with WIDER providing the primary benchmark and ablation setting.

  • Training setup: Models are trained on 4 GPUs with mini-batches of 4 images and fine-tuned for 21K iterations from ImageNet-pretrained classification networks.
  • Datasets: The WIDER dataset contains 32,203 images and 393,703 annotated faces, with validation and test subsets divided into easy, medium, and hard categories.
  • Evaluation protocol: WIDER models are trained on the training set and evaluated on validation and test sets, while ablations use the hard validation subset.
  • Datasets: FDDB contains 2,845 images and 5,171 annotated faces and is used only for testing.
  • Datasets: Pascal Faces contains 851 annotated face-detection images and is used only to evaluate the method.

4.3. WIDER Dataset Result

On WIDER, SSH outperforms prior methods across validation and test subsets, including stronger ResNet-101-based baselines, with further gains when using an image pyramid.

  • 5.7%, 6.3%, and 6.5% higher performance than VGG-16 HR on the WIDER easy, medium, and hard validation subsets, respectively.
  • 0.8 higher performance than ResNet-101 HR on the WIDER validation hard subset without using an image pyramid.
  • ∼4% improvement over the ResNet-101 HR state of the art on WIDER validation when SSH uses an image pyramid.
  • 2.5% improvement over prior methods across all WIDER test subsets using headless VGG-16 SSH with an image pyramid.

4.4. FDDB and Pascal Faces Results

SSH achieves strong results on FDDB and Pascal-Faces with small inputs, while its FDDB evaluation differs from a competing method’s training setup.

  • Less than 50 ms/image inference follows from resizing the shortest input side to 400 pixels and keeping the larger side below 800 pixels.
  • SSH outperforms all compared methods on FDDB’s discrete score without training on FDDB.
  • 5.6% and 1.1% improvements over HR on FDDB’s continuous and discrete scores, respectively.
  • SSH achieves state-of-the-art results on the Pascal-Faces dataset.

4.5. Timing

SSH’s headless design supports substantially faster inference than HR on WIDER, while smaller inputs provide low latency on FDDB and Pascal-Faces.

  • Table 2 reports SSH inference time for different input sizes on the WIDER validation set.
  • 182 ms/image is SSH’s WIDER detection time, compared with 1010 ms/image for HR.HR is more than 5X slower under the reported WIDER timing setup.
  • 48 ms/image is SSH’s detection time at a maximum input size of 400×800 for FDDB and Pascal-Faces.

4.6. Ablation study: Scale-invariant design

The ablation study tests whether SSH’s multiple detection modules support scale invariance by comparing the full design with a single-module alternative.

  • Scale-invariant design: SSH uses detection modules from different layers to cover different face scales, while the ablation retains only M2 from VGG-16 conv5-3.
  • Scale-invariant design: ∼12.8% AP drop occurs on the WIDER hard subset when multiple detection modules are removed.
  • Scale-invariant design: The ablation suggests that independent simple detection modules from different network layers are effective for scale invariance.

4.7. Ablation study: The effect of input size

SSH maintains strong small-face detection performance at a maximum input size of 800×1200, while OHEM and feature fusion improve its validation performance.

  • Input size: 3.5%: At a maximum input size of 800×1200, SSH outperforms HR-VGG16 on the WIDER validation set.HR-VGG16 up-scales images up to 5000 pixels, whereas SSH uses a scale-invariant design.
  • Mining strategy: OHEM is important for the success of SSH’s lightweight detection method without the pre-trained VGG-16 head.The comparison trains SSH with and without hard negative and positive mining while keeping other factors unchanged.
  • Feature fusion: Fusing conv4-3 and conv5-3 outputs for detection module M1 improves AP on the WIDER validation set without noticeable computational overhead.

4.10. Ablation study: Selection of anchor scales

SSH uses coarse anchor scale sets, while a finer scale configuration slows inference and reduces AP because it increases false positives.

  • Selection of anchor scales: Finer anchor scale sets slow inference and reduce AP due to an increase in false positives.The evaluated variant uses S1 = {0.25, 0.5, 1, 2, 3}, S2 = {4, 6, 8, 10, 12}, and S3 = {16, 20, 24, 28, 32}.
  • Selection of anchor scales: SSH’s standard anchor scale sets are S1 = {1, 2}, S2 = {4, 8}, and S3 = {16, 32}.
  • Qualitative results: Figure 8 visualizes validation detections using green for classifier score 1.0 and blue for score 0.5.

5. Conclusion

SSH is a fast, lightweight single-stage face detector that operates from early convolutional layers without the classification head. It detects multiple face scales in one forward pass and achieves state-of-the-art performance across several benchmarks while reducing detection time.

  • Conclusion: SSH detects and localizes faces simultaneously from early convolutional layers in a single stage.
  • Conclusion: Removing VGG-16 fully connected layers produces a headless, lightweight detector that retains state-of-the-art results.
  • Conclusion: SSH detects multiple face scales in one forward pass instead of processing an input pyramid.
  • Conclusion: SSH achieves state-of-the-art performance on WIDER, FDDB, and Pascal-Faces while reducing detection time considerably.
Loading 1708.03979v3…