Source-linked AI summary
RFLA: Gaussian Receptive Field based Label Assignment for Tiny Object Detection
Chang Xu, Jinwang Wang, Wen Yang, Huai Yu, Lei Yu, Gui-Song Xia
TL;DR
Tiny object detection is challenging because tiny objects lack discriminative features and existing box- or point-based assignment can create insufficient or imbalanced positive samples. The paper proposes Gaussian receptive-field label assignment using RFD and HLA, and reports superiority across four datasets, including a 4.0 AP-point advantage on AI-TOD. The approach assumes receptive fields can be modeled with Gaussian distributions, particularly for tiny objects.
Problem
Tiny object detection is difficult because tiny objects lack discriminative features, while existing box- and point-based priors can produce scale-sample imbalance and insufficient positive samples.
Method
RFLA models feature receptive fields with a Gaussian prior, measures similarity to ground truth using RFD, and applies HLA for hierarchical label assignment.
Results
RFLA outperforms state-of-the-art competitors by 4.0 AP points on AI-TOD and shows superiority across four datasets without additional inference-stage costs.
Takeaways & Limitations
Gaussian receptive-field priors and RFD-based hierarchical assignment provide a label-assignment strategy for balanced learning of tiny objects.
Takeaways & Limitations
The approach assumes that effective receptive fields can be modeled by Gaussian distributions, with Gaussian priors especially fitting tiny-object properties.
Abstract
from arXiv · showhide
Detecting tiny objects is one of the main obstacles hindering the development of object detection. The performance of generic object detectors tends to drastically deteriorate on tiny object detection tasks. In this paper, we point out that either box prior in the anchor-based detector or point prior in the anchor-free detector is sub-optimal for tiny objects. Our key observation is that the current anchor-based or anchor-free label assignment paradigms will incur many outlier tiny-sized ground truth samples, leading to detectors imposing less focus on the tiny objects. To this end, we propose a Gaussian Receptive Field based Label Assignment (RFLA) strategy for tiny object detection. Specifically, RFLA first utilizes the prior information that the feature receptive field follows Gaussian distribution. Then, instead of assigning samples with IoU or center sampling strategy, a new Receptive Field Distance (RFD) is proposed to directly measure the similarity between the Gaussian receptive field and ground truth. Considering that the IoU-threshold based and center sampling strategy are skewed to large objects, we further design a Hierarchical Label Assignment (HLA) module based on RFD to achieve balanced learning for tiny objects. Extensive experiments on four datasets demonstrate the effectiveness of the proposed methods. Especially, our approach outperforms the state-of-the-art competitors with 4.0 AP points on the AI-TOD dataset. Codes are available at https://github.com/Chasel-Tsui/mmdet-rfla
1 Introduction
Tiny object detection is difficult because small objects lack discriminative features, while box- and point-based priors can produce insufficient or imbalanced positive samples. RFLA addresses these issues with a Gaussian receptive-field prior, RFD, and hierarchical assignment.
- Tiny objects contain fewer than 16 × 16 pixels in AI-TOD and generic detectors often perform poorly because they lack discriminative features.
- Anchor-based detectors use preset box priors and IoU-based assignment, whereas anchor-free detectors use point priors and center-based sampling.
- Limited prior domains can leave tiny ground truths without overlapping anchors or contained points, producing too few positive samples.
- RFLA models receptive fields with a Gaussian prior and uses Receptive Field Distance to measure similarity between receptive fields and ground-truth regions.The Gaussian weight decays gradually across the image, enabling positional relationships between feature points and ground truths of different sizes to be modeled.
- Hierarchical Label Assignment ranks feature points by RFD and progressively assigns labels to alleviate outlier ground-truth samples and provide sufficient training for tiny objects.Because RFD and IoU have different dimensions, RFD is not directly applied through the baseline IoU threshold structure.
- Extensive experiments on four datasets report superiority over state-of-the-art competitors, especially on AI-TOD without additional inference-stage costs.
2 Related Work
Related work covers mainstream anchor-based and anchor-free detectors, four broad classes of tiny-object methods, and label-assignment strategies. RFLA instead targets prior information and label assignment for tiny object detection without added inference cost.
- Anchor-based detectors rely on task-specific preset boxes and IoU-based assignment, which introduces tuning requirements and additional hyperparameters.
- Anchor-free detectors remove anchor-box constraints by predicting from centers or key-points, while newer end-to-end methods preset boxes without shape or location priors.
- RFLA introduces receptive-field prior information with customized label assignment to alleviate imbalance caused by existing priors and measurements for tiny objects.
- Tiny-object methods are grouped into data augmentation, multi-scale learning, customized training strategies, and feature enhancement strategies.Multi-scale approaches commonly improve performance with additional computation.
- Unlike many tiny-object methods, RFLA focuses on label assignment and does not add inference-stage cost.
- Generic label-assignment methods include likelihood-based positive-anchor selection, GMM partitioning, optimal transport, and adaptive statistics-based sampling.
- This paper differs from general detection assignment strategies by designing prior information and corresponding label assignment specifically for tiny-object detection.
3 Method
RFLA models feature receptive fields and ground-truth regions as Gaussian distributions, then assigns labels by their matching degree rather than box- or point-based overlap. Its RFD and HLA components address non-overlap and scale imbalance issues affecting tiny-object samples.
- Receptive Field Modelling: RFLA models each feature point’s effective receptive field as a 2-D Gaussian using the feature location as mean and an approximated receptive-field radius as covariance.The radius is approximated as half the theoretical receptive-field radius, with its square forming the covariance for square-like kernels.
- Receptive Field Modelling: Ground-truth boxes are likewise modeled as 2-D Gaussians centered at the annotated box center, with covariance determined by half the box width and height.The covariance matrix uses w_g^2/4 and h_g^2/4 for the two spatial dimensions.
- Receptive Field Distance: RFLA measures matching between Gaussian receptive fields and ground truth with candidate distances, selecting Wasserstein distance and Kullback-Leibler divergence while excluding Jensen-Shannon divergence for lacking a closed form.Wasserstein distance handles nonoverlapping distributions but lacks scale invariance; KLD is scale invariant but can be unreliable when overlap is negligible.
- Receptive Field Distance: The final Receptive Field Distance is obtained by applying RFD = 1/(1 + RFDC) to the selected receptive-field distance, producing values in (0, 1).RFDC denotes the selected distance measurement before nonlinear normalization.
- Motivation and Application: RFLA replaces standard box- and point-based assignment because tiny ground truths often have no anchor overlap or contained feature points, leaving too few positive samples for IoU or centerness ranking.The method is designed to provide a positional matching score even when prior and ground-truth domains do not overlap.
- Hierarchical Label Assignment: Hierarchical Label Assignment computes an RFD score matrix, ranks feature points for each ground truth, then decays the receptive-field radius and supplements one positive sample per ground truth.The final assignment combines the first- and second-stage results using r = r_1m + r_2(1 - m).
4 Experiment
Experiments evaluate RFLA across receptive-field distances, label-assignment designs, detector settings, and multiple tiny-object benchmarks. Results show progressive gains, strong AI-TOD performance, and consistent improvements across datasets and detector families.
- Experimental setup: Experiments cover AI-TOD, TinyPerson, VisDrone2019, and DOTA-v2.0, all containing many tiny objects; AI-TOD averages 12.8-pixel objects across 28,036 images.Models use MMDetection with ImageNet-pretrained backbones, SGD training for 12 epochs, and benchmark-specific evaluation protocols.
- Distance comparison: GIoU is inferior to receptive-field distances, while Wasserstein distance and KLD provide comparable performance when combined with HLA.The comparison is conducted using Faster R-CNN with HLA because RFD and HLA are interdependent.
- Ablation study: 9.6 AP points are gained by replacing IoU-based assignment with RFD-based assignment in Faster R-CNN, with AP improving progressively as HLA stages are added.RFD ranks feature-point priorities, allowing positive samples to compensate for ground truths lacking overlap with box priors.
- Ablation study: Top-k settings of 2 or 3 attain the best performance, while HLA remains robust across tested k values and outperforms anchor-size tuning.The reported AP values for the tested settings are 20.7, 21.1, 21.1, and 20.9, with k=3 recommended by the authors.
- Main results: DetectoRS with RFLA reaches 24.8 AP on AI-TOD, exceeding state-of-the-art competitors by 4.0 AP points.RFLA also improves APvt by 1.7 points for RetinaNet and 1.3 points for FCOS*; multi-stage detectors show particularly significant gains.
- More datasets: RFLA improves FCOS and Faster R-CNN on TinyPerson and shows consistent gains on AI-TOD-v2, VisDrone2019, and DOTA-v2.0, with clearer APvt and APt improvements on the latter datasets.Visualization on AI-TOD indicates that applying RFLA to Faster R-CNN can greatly reduce false-negative predictions.
5 Conclusion
RFLA addresses scale-sample imbalance in tiny object label assignment by replacing box and point priors with a Gaussian receptive field prior, RFD, and HLA.
- RFLA introduces a Gaussian receptive field prior because box and point priors do not work well for tiny object detection.
- Receptive Field Distance measures similarity between the effective receptive field and ground truth to overcome IoU and centerness limitations.
- Hierarchical Label Assignment works with RFD to obtain balanced learning for tiny objects.
- Experiments on four datasets show the superiority and robustness of RFLA.