Source-linked AI summary
Shape-IoU: More Accurate Metric considering Bounding Box Shape and Scale
Hao Zhang, Shuaijie Zhang
TL;DR
Existing losses model geometric relationships between GT and predicted boxes while overlooking how the boxes' own shape and scale affect regression. The paper proposes Shape-IoU and variants that incorporate these factors, and reports improved performance over existing methods across detection tasks, including gains on PASCAL VOC.
Problem
Existing bounding box regression methods focus on relationships between GT and predicted boxes while overlooking the effects of the bounding boxes' own shape and scale.
Method
The paper analyzes regression characteristics and incorporates bounding box shape and scale into Shape-IoU, Shape-Dot Distance, and Shape-NWD losses.
Results
Shape-IoU improves PASCAL VOC results over SIoU by 0.5–0.6 points for YOLOv7 and 0.5–0.6 points for YOLOv8 across the reported metrics.
Takeaways & Limitations
The reported comparisons indicate that accounting for bounding box shape and scale can improve detection performance across different detection tasks.
Abstract
from arXiv · showhide
As an important component of the detector localization branch, bounding box regression loss plays a significant role in object detection tasks. The existing bounding box regression methods usually consider the geometric relationship between the GT box and the predicted box, and calculate the loss by using the relative position and shape of the bounding boxes, while ignoring the influence of inherent properties such as the shape and scale of the bounding boxes on bounding box regression. In order to make up for the shortcomings of existing research, this article proposes a bounding box regression method that focuses on the shape and scale of the bounding box itself. Firstly, we analyzed the regression characteristics of the bounding boxes and found that the shape and scale factors of the bounding boxes themselves will have an impact on the regression results. Based on the above conclusions, we propose the Shape IoU method, which can calculate the loss by focusing on the shape and scale of the bounding box itself, thereby making the bounding box regression more accurate. Finally, we validated our method through a large number of comparative experiments, which showed that our method can effectively improve detection performance and outperform existing methods, achieving state-of-the-art performance in different detection tasks.Code is available at https://github.com/malagoutou/Shape-IoU
I. INTRODUCTION
Bounding box regression losses traditionally model geometric relationships between predicted and GT boxes, but the paper argues that a sample's own shape and scale also affect regression. It proposes Shape-IoU and related variants to address this omission, with comparative experiments reporting improved detection performance.
- Motivation: Existing bounding box regression methods mainly add geometric constraints involving the predicted, anchor, and GT boxes.The reviewed methods incorporate overlap, distance, shape, or angle relationships.
- Motivation: The regression sample's own shape and scale factors can affect its regression result.This conclusion motivates treating these inherent properties as part of the loss design.
- Contributions: Shape-IoU incorporates the bounding box sample's shape and scale when calculating regression loss.The paper presents this as an extension of existing bounding box regression loss functions.
- Contributions: For tiny target detection, the paper proposes Shape-Dot Distance and Shape-NWD loss variants.
- Experiments: Comparative experiments with state-of-the-art one-stage detectors report better detection performance than existing methods.The experiments cover different detection tasks.
II. RELATED WORK
The related work develops IoU-based bounding box regression by progressively adding constraints for overlap, enclosing regions, and relative geometry. These methods improve on limitations of basic IoU, especially when boxes do not overlap.
- IoU and extensions: The paper situates GIoU, DIoU, CIoU, EIoU, and SIoU as successive IoU-based losses that add different constraints for more accurate detection.
- IoU and extensions: IoU measures the match between a predicted box and a GT box but cannot accurately describe their positional relationship when overlap is zero.
- IoU and extensions: GIoU addresses the no-overlap limitation of IoU by introducing the smallest enclosing box.The enclosing box is defined over the GT and anchor boxes.
A B C
DIoU adds normalized center-distance constraints to IoU, while CIoU further adds a shape-similarity term between GT and anchor boxes. These methods extend IoU with relative geometric information.
- 3) DIoU Metric:: DIoU adds a normalized centroid distance between the anchor box and GT box to IoU.The distance uses the Euclidean separation of their centers normalized by the enclosing box diagonal.
- Comparison: Both DIoU and CIoU augment IoU with geometric constraints between the anchor and GT boxes.
- CIoU: CIoU extends DIoU with a shape loss term that reduces aspect-ratio differences between anchor and GT boxes.Its shape comparison uses the widths and heights of the two boxes.
4) EIoU Metric:
EIoU redefines the shape loss from CIoU and directly reduces aspect differences between GT and anchor boxes. SIoU additionally considers the angle between the boxes to support horizontal or vertical alignment.
- 4) EIoU Metric:: EIoU redefines CIoU's shape loss and directly reduces aspect differences between GT and anchor boxes.
- Comparison: EIoU and SIoU extend prior IoU-based losses through different shape or angular constraints.
- 5) SIoU Metric:: SIoU adds the angle between bounding boxes as a loss consideration to encourage horizontal or vertical alignment.The method aims to accelerate convergence by decreasing this angle.
B. Metric in Tiny Object Detection
IoU-based metrics are suitable for general object detection, but Dot Distance and NWD were proposed to address IoU sensitivity in small-object detection.
- Dot Distance and NWD are proposed for small-object detection because IoU-based metrics are sensitive to IoU values.
- Dot Distance uses the normalized Euclidean distance between the prediction and ground-truth box centers.The normalization uses the dataset's average target size.
2) Normalized Gaussian Wasserstein Distance:
The regression analysis shows that bounding-box shape and scale affect IoU even when deviations are held constant, motivating shape- and scale-aware regression losses.
- The A–D examples vary ground-truth shape and scale while controlling deviations, exposing their effects on IoU.A and B share one scale, C and D share another; A and D share shape, while B and C share shape.
- The analysis concludes that the regression sample's shape affects its IoU during regression.
- When the ground-truth box is nonsquare, differences in regression-sample shape and scale yield different IoU values under equal nonzero deviations.
- For equal-scale boxes, identical nonzero deviations can produce different IoU values because box shape changes the effect of displacement.Displacements along a box's short edge produce larger IoU changes than equivalent displacements along its long edge.
- For equal-shape boxes, smaller-scale regression samples are more strongly affected by the ground-truth box shape than larger-scale samples.
B. Shape-IoU
Shape-IoU derives a loss that incorporates bounding-box shape and scale factors alongside IoU and distance terms.
- The shape terms use normalized width and height differences between predicted and ground-truth boxes.
- The horizontal and vertical weight coefficients depend on the ground-truth box shape.The scale factor is related to target scale in the dataset.
C. Shape-IoU in Small Target
The authors integrate Shape-IoU into Dot Distance to obtain Shape-Dot Distance.
- Shape-Dot Distance incorporates the Shape-IoU idea into Dot Distance.
2) Shape-NWD:
The paper extends bounding-box regression losses with Shape-IoU ideas, including Shape-NWD for tiny-target detection, and evaluates Shape-IoU against SIoU on VOC using YOLOv7 and YOLOv8.
- 2) Shape-NWD:: Shape-NWD integrates the Shape-IoU idea into NWD.
- PASCAL VOC comparison: The VOC experiments use VOC2007 and VOC2012 training images, VOC2007 testing images, YOLOv8s and YOLOv7-tiny, and SIoU as the comparison method.
- PASCAL VOC comparison: 63.9 and 37.9 are reported for Yolov7+Shape-IoU, compared with 63.4 and 37.3 for Yolov7+SIoU.
- PASCAL VOC comparison: 70.1 and 48.8 are reported for Yolov8+Shape-IoU, compared with 69.5 and 48.3 for Yolov8+SIoU.
B. VisDrone2019 on YOLOv8
The VisDrone2019 experiment evaluates YOLOv8 with Shape-IoU against SIoU on aerial imagery containing many small targets. The paper also reports experiments on AI-TOD, a dataset dominated by tiny targets, and concludes that Shape-IoU outperformed existing methods across different-scale datasets.
- B. VisDrone2019 on YOLOv8: VisDrone2019 contains many small targets relative to general object-detection datasets.
- B. VisDrone2019 on YOLOv8: The VisDrone2019 experiment uses YOLOv8s with SIoU as the comparison method.
- C. AI-TOD on YOLOv5: AI-TOD contains significant numbers of tiny targets, with an average target size of 12.8 pixels, and is evaluated using YOLOv5s against SIoU.
- Conclusion: Across datasets of different scales, the paper reports that Shape-IoU outperformed existing methods and achieved state-of-the-art performance.