Source-linked AI summary
A Normalized Gaussian Wasserstein Distance for Tiny Object Detection
Jinwang Wang, Chang Xu, Wen Yang, Lei Yu
TL;DR
Tiny-object detection is hindered by limited appearance information and IoU metrics that are sensitive to localization deviations. The paper models boxes as 2-D Gaussians and introduces NWD for use in anchor-based detector components, reporting substantial gains on AI-TOD.
Problem
Tiny objects provide limited appearance information, while IoU-based metrics are highly sensitive to their location deviations, undermining anchor-based detection.
Method
NWD models bounding boxes as 2-D Gaussian distributions and replaces IoU in assignment, non-maximum suppression, and loss functions.
Results
6.7 AP points improvement is reported for Faster R-CNN, while NWD-based detectors improve RetinaNet, ATSS, Faster R-CNN, Cascade R-CNN, and DetectoRS.
Takeaways & Limitations
NWD consistently improves tiny-object detection across the experimented anchor-based detectors and achieves state-of-the-art performance on AI-TOD.
Takeaways & Limitations
The normalization constant C is closely related to the dataset and is set empirically to AI-TOD’s average absolute object size.
Abstract
from arXiv · showhide
Detecting tiny objects is a very challenging problem since a tiny object only contains a few pixels in size. We demonstrate that state-of-the-art detectors do not produce satisfactory results on tiny objects due to the lack of appearance information. Our key observation is that Intersection over Union (IoU) based metrics such as IoU itself and its extensions are very sensitive to the location deviation of the tiny objects, and drastically deteriorate the detection performance when used in anchor-based detectors. To alleviate this, we propose a new evaluation metric using Wasserstein distance for tiny object detection. Specifically, we first model the bounding boxes as 2D Gaussian distributions and then propose a new metric dubbed Normalized Wasserstein Distance (NWD) to compute the similarity between them by their corresponding Gaussian distributions. The proposed NWD metric can be easily embedded into the assignment, non-maximum suppression, and loss function of any anchor-based detector to replace the commonly used IoU metric. We evaluate our metric on a new dataset for tiny object detection (AI-TOD) in which the average object size is much smaller than existing object detection datasets. Extensive experiments show that, when equipped with NWD metric, our approach yields performance that is 6.7 AP points higher than a standard fine-tuning baseline, and 6.0 AP points higher than state-of-the-art competitors. Codes are available at: https://github.com/jwwangchn/NWD.
1 Introduction
Tiny object detection is difficult because limited appearance information makes learning discriminative features challenging, while IoU is highly sensitive to small localization deviations. The paper proposes NWD, a Wasserstein-based similarity metric embedded across anchor-based detection components.
- Tiny objects contain limited appearance information, increasing the difficulty of learning discriminative features and causing detection failures.
- For a 6 × 6-pixel object, a minor location deviation reduces IoU from 0.53 to 0.06.
- IoU sensitivity can flip anchor labels, producing similar positive and negative features and making network convergence difficult.
- NWD models bounding boxes as 2-D Gaussian distributions and measures their similarity with Wasserstein distance, including when overlap is negligible.
- NWD replaces IoU in label assignment, non-maximum suppression, and regression loss for single-stage and multi-stage anchor-based detectors.
- 11.1% to 17.6% performance improvement is reported for Faster R-CNN on AI-TOD when using NWD.
2 Related Work
Related work addresses tiny-object detection through multi-scale features, adaptive training strategies, and GAN-based enhancement, while IoU-derived metrics and label assignment remain important design areas. The paper positions NWD against these approaches by targeting tiny-object similarity and anchor selection.
- Tiny Object Detection: Previous tiny-object methods mainly use multi-scale feature learning, improved training strategies, or GAN-based detection.
- Tiny Object Detection: Multi-scale approaches resize inputs or construct feature pyramids to combine information across object scales.
- Tiny Object Detection: SNIP, SNIPER, and SAN adapt training or feature representations to improve robustness across object scales.
- Tiny Object Detection: GAN-based detectors enhance small-object representations through image-level or feature-level super-resolution.
- Evaluation Metric in Object Detection: IoU extensions incorporate overlap, center distance, or aspect ratio, but their label-assignment use is rarely discussed.
- Label Assignment Strategies: Assigning high-quality anchors to tiny-object ground truths is difficult, and lowering IoU thresholds can deteriorate overall training-sample quality.
3 Methodology
The method models bounding boxes as 2D Gaussian distributions and compares them using a normalized Wasserstein distance. NWD is designed to remain informative under tiny-object scale and location deviations and replaces IoU across several anchor-based detector components.
- Gaussian Distribution Modeling: Bounding box R = (cx, cy, w, h) is modeled as a 2D Gaussian distribution whose center and spread represent the box geometry.The modeling assigns higher weight to center pixels and decreasing importance toward the boundary.
- Wasserstein Distance: The similarity between two bounding boxes is converted into the distribution distance between their corresponding Gaussian distributions using the second-order Wasserstein distance.For Gaussian distributions, the Wasserstein distance is defined from their mean vectors and covariance matrices.
- Normalized Wasserstein Distance: NWD applies exponential normalization to the Wasserstein distance so the result functions as a similarity metric with values between 0 and 1, like IoU.The constant C is empirically set to AI-TOD’s average absolute object size and is reported to be robust within a certain range.
- Metric Properties: NWD is scale-invariant, smooth to location deviation, and able to measure similarity for non-overlapping or mutually inclusive boxes.Compared with IoU, NWD remains informative when overlap is negligible or absent and is less sensitive to scale differences.
- NWD-based Detectors: NWD replaces IoU in anchor-based detectors’ positive/negative label assignment, non-maximum suppression, and regression loss.The proposed integration is described for Faster R-CNN and is intended to apply to single-stage and multi-stage anchor-based detectors.
4 Experiments
Experiments evaluate NWD across metrics, detector modules, anchor-based detectors, and tiny-object datasets. NWD improves label assignment, NMS, loss functions, and overall detection performance, with strongest gains often appearing for extremely tiny objects.
- Experimental Setup: AI-TOD contains 700,621 instances across 28,036 aerial images, with a mean object size of 12.8 pixels.The dataset has eight categories and 800 × 800-pixel images.
- Comparison with Other Metrics based IoU: NWD achieves 16.1% AP in label assignment, improving APt by 9.6% over IoU and matching 1.05 positive anchors per ground-truth box.IoU, GIoU, DIoU, and CIoU match 0.72, 0.71, 0.19, and 0.19 positive anchors, respectively.
- Comparison with Other Metrics based IoU: NWD achieves 11.9% AP in RPN NMS, 0.8% above IoU, and reaches 12.1% AP when used in the loss function.These experiments isolate the NMS and loss-function modifications.
- Ablation Study: Applying NWD to a single module improves AP in 5 of 6 modules, with gains of 6.2% in RPN assignment and 3.2% in R-CNN assignment.The RPN assignment improvement is the largest among the tested single-module changes.
- Ablation Study: Using NWD in all three RPN modules yields 17.8% AP, while applying it to all six modules initially reduces AP by 2.6% versus RPN-only use.Training for 24 epochs narrows this gap to 0.9%, indicating slower convergence when NWD is used in R-CNN.
- Main Results: On AI-TOD, NWD improves AP by 4.5%, 0.7%, 6.7%, 4.9%, and 6.0% for RetinaNet, ATSS, Faster R-CNN, Cascade R-CNN, and DetectoRS, respectively.NWD-based DetectoRS reaches 20.8% AP, while experiments on VisDrone2019 also show considerable improvements over baselines.
5 Conclusion
The paper identifies IoU-based metrics as overly sensitive to tiny-object location deviations and proposes NWD-based anchor detectors to address this problem. Experiments show large performance improvements and state-of-the-art results on AI-TOD.
- IoU-based metrics are sensitive to tiny-object location deviations, which deteriorate tiny object detection performance.
- NWD measures bounding-box similarity and replaces IoU in label assignment, non-maximum suppression, and regression loss.
- NWD-based anchor detectors improve tiny object detection by a large margin and achieve state-of-the-art performance on AI-TOD.