PaperlayerView exact revision on arXiv

Source-linked AI summary

YOLO5Face: Why Reinventing a Face Detector

Delong Qi, Weijun Tan, Qi Yao, Jingfeng Liu

arXiv:2105.12931v3cs.CV

TL;DR

Face detection has advanced substantially with CNNs, but the paper asks whether a generic object detector can serve this task instead of a specialized design. It builds YOLO5Face by modifying YOLOv5 with face-oriented outputs and scale handling, then reports close-to-or-better-than-state-of-the-art WiderFace validation performance from the largest to the smallest model.

  • Problem

    The paper addresses whether face detection can be effectively treated as generic object detection despite face-specific landmark outputs and scale challenges.

  • Method

    The paper modifies YOLOv5 into YOLO5Face with five-point landmark regression, a Stem block, smaller SPP kernels, P6 output, and models of different sizes.

  • Results

    On VGA WiderFace validation images, both YOLOv5l6 and YOLOv5n achieve close to or exceeding state-of-the-art performance on Easy, Medium, and Hard subsets.

  • Takeaways & Limitations

    YOLO5Face supports face detection across performance and speed requirements, including fast operation on embedded or mobile devices.

Abstract

from arXiv · show

Tremendous progress has been made on face detection in recent years using convolutional neural networks. While many face detectors use designs designated for detecting faces, we treat face detection as a generic object detection task. We implement a face detector based on the YOLOv5 object detector and call it YOLO5Face. We make a few key modifications to the YOLOv5 and optimize it for face detection. These modifications include adding a five-point landmark regression head, using a stem block at the input of the backbone, using smaller-size kernels in the SPP, and adding a P6 output in the PAN block. We design detectors of different model sizes, from an extra-large model to achieve the best performance to a super small model for real-time detection on an embedded or mobile device. Experiment results on the WiderFace dataset show that on VGA images, our face detectors can achieve state-of-the-art performance in almost all the Easy, Medium, and Hard subsets, exceeding the more complex designated face detectors. The code is available at \url{https://github.com/deepcam-cn/yolov5-face}

I. INTRODUCTION

The paper argues that face detection can be treated as generic object detection because its data variations and detection challenges overlap with those of other objects. It therefore redesigns YOLOv5 into YOLO5Face and offers models spanning high performance to embedded-device efficiency.

  • Face detection shares object-detection challenges including pose, scale, occlusion, illumination, blur, multi-scale variation, small faces, and dense scenes.
  • YOLO5Face redesigns YOLOv5 for face detection while considering large faces, small faces, landmark supervision, model complexity, and application requirements.
  • The model portfolio spans large, medium, and super-small detectors for different applications, including a ShuffleNetV2 backbone for fast mobile-device inference.
  • On VGA WiderFace images, almost all models achieve state-of-the-art performance and fast speed across the reported evaluation setting.

B. Face Detection

Face detection research follows general object detection, with modern work addressing extreme variations such as scale, pose, occlusion, illumination, and blur. YOLO contributes a one-stage, real-time object-detection framework that YOLOv5 later made smaller and faster.

  • The WiderFace benchmark accelerated face-detection research on extreme variations including scale, pose, occlusion, expression, makeup, illumination, and blur.
  • Face detectors address these challenges through methods involving scale, context, and anchors, including MTCNN, RetinaFace, TinaFace, and SCRFD.
  • Some face detectors exploit face-specific characteristics, whereas others adapt general object detectors; RetinaFace uses landmark regression, while TinaFace is a general object detector.
  • YOLO performs object detection with a single neural network, combining strong detection performance with real-time speed and generalization across objects.
  • YOLOv3 improved performance and speed through multi-scale features, a stronger backbone, and binary cross-entropy classification loss.
  • YOLOv5 became popular for its reduced model size, faster speed, similar performance to YOLOv4, and full PyTorch implementation.

III. YOLO5FACE FACE DETECTOR

YOLO5Face uses YOLOv5 as its backbone architecture and modifies the network for face-specific outputs and scale coverage. Its design includes landmark prediction, a stem input block, smaller SPP kernels, and an optional P6 output.

  • YOLO5Face retains YOLOv5’s backbone, SPP and PAN feature aggregation, and regression-classification head while adapting them for face detection.
  • The detection head outputs bounding boxes, confidence, classification, and five-point landmarks, extending the output dimension from 6 to 16 per anchor.
  • A Stem block replaces YOLOv5’s Focus layer as an input structure and is presented as an innovation for face detection.
  • The SPP kernels change from 13x13, 9x9, 5x5 to 7x7, 5x5, 3x3 in YOLO5Face.
  • The input uses a 640-pixel longer edge, with the shorter edge adjusted to a multiple of 32 without P6 or 64 with P6.

B. Summary of Key Modifications

The paper summarizes architectural and training modifications that add landmark supervision, improve scale coverage, and tailor YOLOv5 to face detection. It also introduces lightweight ShuffleNetV2 models for embedded and mobile applications.

  • A landmark regression head with Wing loss adds landmark supervision and supports more accurate landmark locations.
  • Replacing Focus with a Stem block is reported to improve generalization and reduce computation complexity without performance degradation.
  • Smaller SPP kernels make YOLOv5 more suitable for face detection and improve detection accuracy.
  • A stride-64 P6 output increases capability for detecting large faces, which can be overlooked when research focuses on small faces.
  • Removing up-down flipping improves performance, while Mosaic can degrade performance on small images but work well when small faces are ignored; random cropping helps.
  • Two super-lightweight ShuffleNetV2 models target embedded and mobile devices while achieving state-of-the-art performance.

C. Landmark Regression

YOLO5Face adds five-point landmark regression to a general object detector, using Wing loss to emphasize small landmark errors and support downstream face alignment and recognition.

  • Landmark Regression: Five-point landmarks are added as a regression head because general object detectors do not normally output landmarks.The predicted landmarks are used to align face images before face recognition.
  • Landmark Regression: Wing loss is selected because L2, L1, and smooth-L1 losses are not sensitive to small landmark errors.Its response near zero error is boosted relative to those alternatives.
  • Landmark Regression: Wing loss uses w to define the nonlinear range, e to control curvature, and C to smoothly connect its nonlinear and linear parts.The nonlinear region spans (−w, w), while C = w − wln(1 + w/e).
  • Landmark Regression: The landmark loss uses a vector of predicted points and ground-truth points, combined with YOLOv5’s object-detection loss.The weighting factor λL controls the contribution of landmark regression.

D. Stem Block Structure

The stem block replaces YOLOv5’s original Focus layer at the input, downsampling with stride 2 while increasing channels and preserving representation capability at marginal extra cost.

  • D. Stem Block Structure: The stem block performs the first input downsampling with stride 2 and increases the number of channels.The authors report only a marginal increase in computational complexity while maintaining strong representation capability.
  • D. Stem Block Structure: The implemented models and ablation and comparison results are organized in Tables I–III.Table I describes model configurations, while Tables II and III report ablations and WiderFace comparisons.
  • E. SPP with Smaller Kernels: SPP is placed before feature aggregation in the neck to increase receptive field and separate important features.It also pools multi-scale versions of features and can produce fixed-size output irrespective of input size.
  • E. SPP with Smaller Kernels: YOLO5Face uses smaller SPP kernels of 7x7, 5x5, and 3x3 instead of YOLOv5’s 13x13, 9x9, and 5x5 kernels.The authors state that the smaller kernels help detect small faces and improve overall face-detection performance.

F. P6 Output Block

YOLO5Face extends the PAN output pyramid with a P6 block to preserve a coarser feature scale, specifically improving detection of large faces that receive less attention in many detectors.

  • F. P6 Output Block: PAN augments FPN’s top-down path with a bottom-up path and replaces lateral element-wise addition with concatenation.These changes are described as feature-fusion modifications for object detection.
  • F. P6 Output Block: YOLO5Face adds a P6 output with a 10x10x16 feature map and stride 64.The authors state that this modification particularly helps detect large faces.
  • G. ShuffleNetV2 as Backbone: ShuffleNetV2 is used as the backbone for the super-small YOLOv5n-Face and YOLOv5n0.5-Face detectors.The paper characterizes this backbone as super fast and targets these models for small-device applications.

IV. EXPERIMENTS

Experiments evaluate YOLO5Face primarily on the challenging WiderFace benchmark, using multiple model scales and reporting configurations, computational costs, and landmark-related face-recognition evaluation.

  • A. Dataset: WiderFace contains 32,203 images and 393,703 faces with varied scale, pose, occlusion, expression, illumination, and events.The dataset is described as close to reality and very challenging.
  • A. Dataset: WiderFace is split into train, validation, and test sets at 50%/10%/40% within each event class, with Easy, Medium, and Hard subsets.The Hard subset is described as the most challenging and most reflective of detector effectiveness.
  • B. Implementation Details: The study also uses WebFace for face-recognition evaluation and FDDB to test cross-domain performance.WiderFace is the default dataset unless otherwise specified.
  • B. Implementation Details: The implementation starts from the YOLOv5-4.0 codebase and applies the described modifications in PyTorch.Training uses SGD for 250 epochs with batch size 64 and the reported learning-rate and momentum schedule.
  • B. Implementation Details: The model family spans extra-large through small CSPNet-based detectors, plus super-small YOLOv5n and YOLOv5n0.5 models using ShuffleNetV2 backbones.The study lists parameter and FLOP counts for comparison with existing methods.
  • B. Implementation Details: Landmark quality is evaluated through face recognition on the WebFace test dataset, using the challenge’s combined masked and standard-face FNMR metric.This evaluation connects landmark predictions with recognition performance rather than only landmark-coordinate error.

C. Ablation Study

The ablation study evaluates YOLO5Face modifications on WiderFace validation using mAP, with each experiment comparing matched network baselines. Smaller SPP kernels provide the largest reported gains among the architectural changes, while augmentation effects depend on subset difficulty.

  • The ablation study uses YOLOv5s on WiderFace validation and reports mAP for each modification.The network configurations are not incremental, but baselines within each experiment are matched for fair comparison.
  • Stem Block vs. Focus Layer: Stem blocks improve mAP by 0.57%, 0.33%, and 0.23% on the Easy, Medium, and Hard subsets, respectively.
  • SPP with Smaller Size Kernels: Smaller SPP kernels improve mAP by 0.9%, 1.49%, and 1.41% on Easy, Medium, and Hard, respectively.The reported gains exceed those from the stem block.
  • P6 Output Block: Adding P6 improves mAP by 0.98% on Easy and 1.09% on Medium, but changes Hard mAP by -0.02%.
  • Data Augmentation: Ignoring small faces and random cropping help Easy and Medium, whereas Mosaic helps Hard when combined with ignoring small faces.Without that combination, Mosaic causes a dramatic performance degradation.

D. YOLO5Face for Face Recognition

The paper evaluates YOLO5Face landmarks through downstream face recognition on WebFace, comparing small and medium YOLO5Face models against RetinaFace. Both YOLO5Face models outperform RetinaFace, and large-pose examples show visually more accurate landmarks for YOLOv5m.

  • Landmark accuracy is evaluated through face recognition on the WebFace test dataset using the MFR challenge metric.The metric combines weighted FNMR for masked-face and standard face recognition.
  • The comparison replaces RetinaFace with YOLOv5s or YOLOv5m in an ArcFace-based recognition system.
  • Both YOLO5Face small and medium models outperform RetinaFace on the face-recognition evaluation.
  • Adding large face images from Multi-task-facial to YOLO5Face training improves face recognition performance.The authors note that WiderFace contains very few large face images.
  • For large-pose faces, YOLOv5m landmarks appear more accurate than RetinaFace landmarks in visual examples.The passage connects this observation to the face-recognition results.

E. YOLO5Face on WiderFace Dataset

On WiderFace, YOLO5Face achieves state-of-the-art or near-state-of-the-art performance across model sizes, including strong results from super-small models. The evaluation also reports validation and test performance, plus a separate FDDB cross-domain test without retraining.

  • Large models: YOLOv5x6 achieves 96.67%, 95.08%, and 86.55% mAP on WiderFace Easy, Medium, and Hard, respectively, reaching SOTA on all three subsets.The comparison concerns relatively large models exceeding 3M parameters and 5G FLOPs.
  • Super-small models: YOLOv5n achieves 93.61%, 91.54%, and 80.53% mAP on Easy, Medium, and Hard, respectively, leading SCRFD by 2.66% on Hard.It trails SCRFD slightly on Easy and Medium; YOLOv5n0.5 is also reported to perform well at much smaller size.
  • Validation and test results: On validation, YOLOv5x6-Face reaches 96.9%, 96.0%, and 91.6% mAP on Easy, Medium, and Hard, exceeding previous SOTA by 0.0%, 0.1%, and 0.4%.
  • Validation and test results: On test, YOLOv5x6-Face reaches 95.8%, 94.9%, and 90.5% mAP on Easy, Medium, and Hard, with gaps of 1.1%, 1.0%, and 0.7% to previous SOTA.The evaluation uses multiple scales and left-right flipping without other test-time augmentation.
  • Cross-domain evaluation: The FDDB cross-domain evaluation tests YOLO5Face without retraining, using the original annotations and TPR at 1,000 false positives.
  • Conclusion: The paper implements eight models, with YOLOv5l6 and YOLOv5n close to or exceeding SOTA on WiderFace validation subsets.The conclusion frames the models as combining performance with fast operation.
Loading 2105.12931v3…