Source-linked AI summary
Sample and Computation Redistribution for Efficient Face Detection
Jia Guo, Jiankang Deng, Alexandros Lattas, Stefanos Zafeiriou
TL;DR
Efficient face detection must combine high precision with low computation cost, but existing strong detectors can be computationally heavy. This paper introduces Sample Redistribution and Computation Redistribution for SCRFD, and reports improved accuracy-efficiency trade-offs across compute regimes, including a 3.86% AP advantage over TinaFace with more than 3× faster GPU inference at VGA resolution.
Problem
Efficient face detection with low computation cost and high precision remains an open challenge, while TinaFace’s heavy computation makes it unsuitable for real-time applications.
Method
SCRFD combines Sample Redistribution, which augments training samples for needed shallow stages, with Computation Redistribution across the backbone, neck, and head using a simplified search methodology.
Results
The SCRFD family achieves an improved accuracy-efficiency trade-off across a wide range of compute regimes on WIDER FACE; SCRFD-34GF surpasses TinaFace by 3.86% AP while being more than 3× faster on GPUs.
Takeaways & Limitations
Redistributing training samples and model computation provides an efficient face-detection design across multiple compute regimes under VGA-resolution testing.
Abstract
from arXiv · showhide
Although tremendous strides have been made in uncontrolled face detection, efficient face detection with a low computation cost as well as high precision remains an open challenge. In this paper, we point out that training data sampling and computation distribution strategies are the keys to efficient and accurate face detection. Motivated by these observations, we introduce two simple but effective methods (1) Sample Redistribution (SR), which augments training samples for the most needed stages, based on the statistics of benchmark datasets; and (2) Computation Redistribution (CR), which reallocates the computation between the backbone, neck and head of the model, based on a meticulously defined search methodology. Extensive experiments conducted on WIDER FACE demonstrate the state-of-the-art efficiency-accuracy trade-off for the proposed \scrfd family across a wide range of compute regimes. In particular, \scrfdf{34} outperforms the best competitor, TinaFace, by $3.86\%$ (AP at hard set) while being more than \emph{3$\times$ faster} on GPUs with VGA-resolution images. We also release our code to facilitate future research.
1. Introduction
Efficient face detection remains challenging because high accuracy often comes with substantial computation, especially for tiny faces. The paper addresses this through sample redistribution and computation redistribution, yielding improved accuracy-efficiency trade-offs for SCRFD across compute regimes.
- Motivation: 92.4% AP on WIDER FACE Hard was achieved by TinaFace, but its large-scale testing and generic detector design impose substantial computational cost.TinaFace uses large-scale testing, dense anchors, and heavy head designs without face-specific architectural optimization.
- Sample Redistribution: 78.93% of WIDER FACE faces are smaller than 32×32 pixels at VGA resolution, motivating additional training samples for shallow stages.The proposed Sample Redistribution method uses large cropping to augment samples for these stages.
- Proposed Approach: SCRFD uses Sample Redistribution to augment shallow-stage training data and Computation Redistribution to allocate computation across the backbone, neck, and head.The computation search samples architectures, estimates likely regions for strong models, and separates backbone-ratio estimation from whole-detector estimation.
- Computation Redistribution: The computation redistribution search is simplified to reveal how computation should be allocated under different flop regimes.The method controls search-space degrees of freedom while evaluating configurations across backbone, neck, and head components.
- Results: SCRFD demonstrates a significantly improved accuracy-efficiency trade-off across a wide range of compute regimes on WIDER FACE.Figure 1 compares AP and FLOPs at VGA resolution against state-of-the-art open-sourced detectors.
2. Related Work
Recent single-shot face detectors address difficult variations through anchor strategies, feature enhancement, and architecture search. Their approaches target different parts of the detection pipeline, including anchors, backbone, neck, and head.
- Single-Shot Face Detection: Recent single-shot face detectors improve anchor sampling or matching and feature enhancement to handle scale, pose, illumination, and occlusion variation.Representative methods modify receptive fields, anchor compensation, data sampling, supervision, context modules, or anchor mining.
- Sampling and Supervision: PyramidBox increases the proportion of small faces in training data, while DSFD adds small-face supervision signals to backbone features.These methods address the prevalence and difficulty of small faces through training-data or feature-supervision strategies.
- Architecture Search: BFbox jointly optimizes the backbone and neck, whereas ASFD searches for feature-enhancement modules for multi-scale fusion and context enhancement.The methods reflect growing attention to face-specific architectural design rather than directly applying generic detector components.
3. TinaFace Revisited
TinaFace combines a ResNet-50 backbone, FPN neck, and heavy head, achieving strong accuracy at larger testing scales but incurring substantial computation. At VGA resolution, its accuracy drops sharply, while most computation remains concentrated in the stride-4 feature level and detection head.
- Architecture: TinaFace uses ResNet-50 as backbone, FPN as neck, and a feature-enhanced head with four consecutive 3 × 3 convolutional layers.The head uses focal, DIoU, and cross-entropy losses for classification, box regression, and IoU prediction.
- Testing-scale comparison: 93.4% AP is achieved with multi-scale testing, while large single-scale testing at 1650 reaches 93.0% AP with 1021.82 Gflops.At the original scale of approximately 1024, TinaFace obtains 91.4% AP with 508.47 Gflops.
- Testing-scale comparison: 81.4% AP is obtained at VGA testing scale (640), with computation reduced to 172.95 Gflops.This is a substantial accuracy reduction relative to the larger-scale settings.
- Computation distribution: At VGA resolution, about 68% of computation comes from stride 4, whose feature map has resolution 120 × 160.The large spatial resolution of this feature map drives its computational share.
- Computation distribution: At VGA resolution, about 79% of computation is spent in the head, which is attributed to the backbone being borrowed directly from ImageNet classification.The distribution is reported across the backbone, neck, and head components.
- Practical limitation: TinaFace’s heavy computation makes it unsuitable for real-time applications, while increasing the detection-score threshold to target high precision significantly reduces AP.The paper notes that real-world systems may require precision above 98%.
4. Methodology
The method redistributes both training samples and computation toward the shallow stages most relevant to tiny faces, using staged architecture search across compute regimes.
- Sample Redistribution: At 640-pixel testing scale, 78.93% of WIDER FACE hard-track faces are smaller than 32 × 32 and are predicted by shallow stages.The method therefore focuses on stride 8 and shallow backbone stages.
- Sample Redistribution: Sample Redistribution enlarges crop sizes from [0.3, 1.0] to [0.3, 2.0] to generate more positive samples for stride 8.Pixels outside the original image are filled with average RGB values.
- Computation Redistribution: The computation search varies backbone, neck, and head configurations while simplifying the search space through fixed blocks, shared heads, and constrained widths.The search uses random sampling and empirical bootstrap estimates of computation ratios associated with the best models.
- Computation Redistribution: The two-step search first optimizes backbone computation, then optimizes allocation across the whole detector while retaining the optimized backbone distribution.SCRFD1 fixes neck and head settings; SCRFD2 searches backbone, neck, and head jointly under the selected backbone allocation.
- Computation Redistribution: 77.87% AP is achieved on the WIDER FACE hard validation set after two-step computation redistribution, improving from the baseline’s 74.47%.The reported improvement follows allocation of substantial capacity to shallow stages.
- Computation Redistribution: The resulting designs allocate more computation to the backbone and shallow stages while compressing the neck and head across multiple compute regimes.This pattern is reported for 2.5 Gflops, 10 Gflops, and 34 Gflops, with the same search strategy also explored at 0.5 Gflops.
5. Experiments
Experiments evaluate the proposed models with VGA-bounded single-scale testing and show improved accuracy-efficiency trade-offs across compute regimes, including strong comparisons with prior detectors.
- Experimental Setup: The experiments train baseline and Sample Redistribution models with resized square crops, color distortion, horizontal flipping, and stride-specific anchor settings.The proposed crop scales add [1.2, 1.4, 1.6, 1.8, 2.0] to the baseline range.
- Experimental Setup: Testing uses single-scale images bounded by VGA resolution, with reported inference latency measured on NVIDIA 2080TI using 640 × 480 images.Comparisons include released models from DSFD, RetinaFace, TinaFace, and FaceBoxes, plus an author-shared HAMBox model.
- Ablation Study: 74.47% hard-set AP is obtained after Sample Redistribution, compared with 67.32% for the baseline under the corresponding validation setting.The result supports allocating more training samples to the feature map of stride 8.
- Ablation Study: 71.37% hard-set AP is achieved by the two-step computation redistribution strategy, exceeding the reported one-step whole-detector optimization result of 70.98%.Separate backbone-only and whole-detector redistribution reach 69.78% and 70.98%, respectively.
- Accuracy and Efficiency: 3.86% higher hard-set AP than TinaFace is reported for SCRFD-34GF, with more than 3× faster GPU inference and around 20% of TinaFace’s computation cost.SCRFD-34GF also has a reported model size of 9.80M parameters.
- Accuracy and Efficiency: 21.19% higher hard AP is reported for SCRFD-0.5GF than RetinaFaceM0.25, while consuming 63.34% of its computation and 45.57% of its inference time.This comparison represents the low-compute regime.
- Accuracy and Efficiency: 53.7% AP is obtained by SCRFD-2.5GF at above 98% precision, comparable to TinaFace’s 53.9% AP while using 1.46% of its computation and 10.8% of its inference time.Qualitative results are reported for crowded indoor and outdoor scenes with pose, occlusion, and illumination variation.
6. Conclusions
SCRFD presents a sample and computation redistribution paradigm that improves the accuracy-efficiency trade-off for face detection across a wide range of compute regimes.
- SCRFD improves the accuracy-efficiency trade-off for face detection across a wide range of compute regimes.