Source-linked AI summary
YOLO-FaceV2: A Scale and Occlusion Aware Face Detector
Ziping Yu, Hongbo Huang, Weijun Chen, Yongxin Su, Yahui Liu, Xiuying Wang
TL;DR
Face detection must handle small faces, varying scales, occlusion, and imbalanced easy and hard samples. YOLO-FaceV2 extends YOLOv5 with scale-aware features, occlusion-aware modules and losses, adaptive weighting, and effective-receptive-field anchors, achieving close to or exceeding state-of-the-art performance on WiderFace Easy and Medium subsets.
Problem
Face detection remains challenging for small faces, large scale variation, occlusion, and imbalance between easy and hard samples.
Method
YOLO-FaceV2 extends YOLOv5 with P2 fusion, RFE, NWD Loss, SEAM, Repulsion Loss, Slide weighting, and effective-receptive-field anchor design.
Results
YOLO-FaceV2 achieves close to or exceeding state-of-the-art performance on the WiderFace validation Easy and Medium subsets.
Takeaways & Limitations
The combined detector targets scale variation, small-target localization, occlusion, and easy–hard sample imbalance within a real-time YOLOv5-based face detector.
Takeaways & Limitations
Existing hard-sample mining methods have many hyperparameters to set, creating practical tuning inconvenience.
Abstract
from arXiv · showhide
In recent years, face detection algorithms based on deep learning have made great progress. These algorithms can be generally divided into two categories, i.e. two-stage detector like Faster R-CNN and one-stage detector like YOLO. Because of the better balance between accuracy and speed, one-stage detectors have been widely used in many applications. In this paper, we propose a real-time face detector based on the one-stage detector YOLOv5, named YOLO-FaceV2. We design a Receptive Field Enhancement module called RFE to enhance receptive field of small face, and use NWD Loss to make up for the sensitivity of IoU to the location deviation of tiny objects. For face occlusion, we present an attention module named SEAM and introduce Repulsion Loss to solve it. Moreover, we use a weight function Slide to solve the imbalance between easy and hard samples and use the information of the effective receptive field to design the anchor. The experimental results on WiderFace dataset show that our face detector outperforms YOLO and its variants can be find in all easy, medium and hard subsets. Source code in https://github.com/Krasjet-Yu/YOLO-FaceV2
1 Introduction
Face detection remains difficult under scale variation, occlusion, and imbalanced samples, especially for small faces. YOLO-FaceV2 addresses these issues with YOLOv5-based feature, attention, loss, weighting, and anchor-design changes.
- Challenges: Small faces lose pixel information through feature-map convolutions, while IoU is highly sensitive to slight localization errors for small targets.These effects make multiscale detection and small-target regression difficult.
- Challenges: Occlusion causes missing facial information, inaccurate localization, missed detections, and low recall in crowded scenes.Inter-face occlusion also makes detection sensitive to NMS thresholds.
- Challenges: One-stage detectors contain many easy samples whose cumulative training contribution can dominate updates, motivating hard-sample mining.Existing hard-sample methods can require many hyperparameters.
- Proposed detector: YOLO-FaceV2 fuses P2 features for higher small-face resolution and adds RFE with dilated convolution to enlarge the P5 receptive field.The P2 fusion improves small-face information but can slightly reduce large- and medium-target accuracy without receptive-field enhancement.
- Proposed detector: The detector combines SEAM attention and Repulsion Loss for occlusion, Slide weighting for hard samples, and effective-receptive-field information for anchor design.SEAM enhances face features, while Repulsion Loss discourages predicted boxes from shifting toward other targets.
2 Related Works
Related work spans cascade-based face detectors and adaptations of general-purpose object detectors. Research also targets multiscale detection and occlusion through feature pyramids, context, and attention.
- Face detection: Cascade detectors use multiple neural-network stages to detect faces from coarse to fine but can be difficult to train and slow at inference.Examples include CascadeCNN, MTCNN, and PCN.
- Face detection: General-purpose detectors such as SSD, Faster R-CNN, and RetinaNet have been adapted with contextual information and multiscale features for face detection.FaceBoxes is described as a lightweight SSD-based network.
- Scale and occlusion: Multiscale methods build feature pyramids, while occlusion methods use context modules or attention to compensate for invisible regions and blurred boundaries.Examples include SNIP, TridentNet, SSH, FAN, PyramidBox, and RetinaFace.
3 YOLO-FaceV2
YOLO-FaceV2 extends YOLOv5 with modules targeting small faces, scale variation, occlusion, sample imbalance, and anchor design. Its architecture combines receptive-field enhancement, attention, specialized losses, adaptive weighting, and effective-receptive-field-based anchors.
- Network Architecture: YOLO-FaceV2 uses CSPDarknet53, SPP, PAN, P2 fusion, and detection heads, replacing the P5 bottleneck with RFE and adding SEAM after the neck.P2 fusion improves target-position perception, while SEAM strengthens responsiveness to occluded faces.
- Scale-Aware RFE Model: RFE uses four weight-sharing branches with different dilated-convolution rates to capture multi-scale information and distinct dependency ranges.The module replaces a YOLOv5 bottleneck and uses dilated convolutions to expand the feature map’s effective receptive field.
- Occlusion-Aware Repulsion Loss: Repulsion Loss separates RepGT and RepBox to reduce overlap with surrounding ground-truth and prediction boxes during intra-class occlusion.RepGT moves predictions away from nearby ground-truth boxes, while RepBox reduces overlap between predictions associated with different faces to avoid NMS suppression.
- Occlusion-Aware Attention Network: SEAM combines depthwise separable convolution, pointwise convolution, residual connections, and channel fusion to strengthen occluded-face features.Its stated purposes include multi-scale face detection, emphasizing face regions, and weakening background responses.
- Sample Weighting Function: Slide Loss uses mean IoU as the positive-negative threshold and assigns higher weights near the threshold to emphasize hard samples.The design reduces hyperparameter tuning by deriving the dividing threshold from candidate-box IoUs.
- Anchor Design Strategy: The detector uses three detection heads with nine anchor scales, a 1:1.2 face aspect ratio, and anchor sizes redesigned around the effective rather than theoretical receptive field.The anchor sizes are associated with the strides of P2, P3, and P4.
- Normalized Gaussian Wasserstein Distance: NWD Loss supplements IoU loss because normalized Wasserstein distance is less sensitive to target scale and location deviations for small targets.The paper retains IoU loss because it is suitable for large targets.
4 Experiments
The experiments evaluate YOLO-FaceV2 through module ablations and comparisons with existing state-of-the-art face detectors.
- The study conducts comprehensive ablations of the attention module, multiscale fusion pyramid, and loss-function designs.
4.1 Dataset
YOLO-FaceV2 is evaluated on WiderFace, a large face-detection dataset divided into easy, medium, and hard subsets.
- WiderFace contains 32,203 images and more than 400k faces, with 40% training, 10% validation, and 50% test data.
- The dataset is divided into easy, medium, and hard subsets, with the hard subset presenting the greatest challenge.
- The model is trained on the WiderFace training set and evaluated on its validation and test sets.
4.2 Training
Training uses YOLOv5 as the baseline and implements the methods in PyTorch with SGD momentum optimization.
- YOLOv5 is used as the baseline, and the proposed methods are implemented with PyTorch.
- Training uses SGD with momentum, learning rates from 1e-2 initially to 1e-3 finally, and weight decay of 5e-3.
- The model is trained on a 1080ti with four CPU workers, using batch size 16 for fine-tuning.
4.3 Ablation Study
The ablation study evaluates individual modules and losses on WiderFace, showing gains from SEAM, anchor design, Slide weighting, and Repulsion Loss while comparing IoU and NWD choices.
- SEAM Block: SEAM increases validation accuracy by 0.88, 0.82, and 1.06 on the easy, medium, and hard subsets, respectively.
- Multi-scale Fusion: Fusing P2 features increases the hard-subset result by 0.57 by adding information from small targets.The receptive field enhancement module uses dilated convolutions with rates 1, 2, and 3 to address the reduced receptive field.
- Slide Loss: Slide weighting slightly improves performance on the medium and hard subsets by focusing the model on hard samples.
- Anchor Design: Designed anchors improve results by 0.24, 0.75, and 0.9 on the easy, medium, and hard datasets, respectively.The design uses effective receptive-field information and face-shape characteristics to improve recall of small faces.
- NWD Loss: Replacing IoU with NWD alone does not improve the result, so the method retains IoU while adjusting its proportion relative to NWD.The authors report that IoU performs better for large and medium targets, whereas NWD improves small-target accuracy.
- Comparison: The experiments compare YOLO-FaceV2 with existing face detectors on the WiderFace validation dataset.
- Repulsion Loss: The Repulsion Loss improves accuracy by 0.71, 0.63, and 0.5 on the easy, medium, and hard subsets, respectively.
4.4 Comparisons with Existing Face Detectors
The section compares YOLO-FaceV2 with existing face detectors on the WiderFace validation dataset and presents precision-recall curves across Easy, Medium, and Hard subsets.
- Table 4 compares YOLO-FaceV2 with existing face detectors on the WiderFace validation dataset.
- The plotted evaluation covers the WiderFace Easy, Medium, and Hard subsets.
- Figure 6 presents precision-recall curves for YOLO-FaceV2 and competing detectors.
5 Conclusion
YOLO-FaceV2 is a YOLOv5-based face detector designed for varying scales, sample imbalance, and occlusion. It achieves close to or exceeding state-of-the-art performance on the WiderFace Easy and Medium validation subsets.
- YOLO-FaceV2 addresses varying face scales, easy-hard sample imbalance, and face occlusion with a YOLOv5-based design.
- The method fuses P2, adds RFE, and uses NWD Loss to improve small-object detection robustness.
- Slide alleviates easy-hard sample imbalance, while SEAM and Repulsion Loss address face occlusion.
- Anchor design uses information from the effective receptive field.
- YOLO-FaceV2 achieves close to or exceeding state-of-the-art performance on the WiderFace validation Easy and Medium subsets.