Source-linked AI summary
BorderDet: Border Feature for Dense Object Detection
Han Qiu, Yuchen Ma, Zeming Li, Songtao Liu, Jian Sun
TL;DR
Dense object detectors commonly rely on point features that may lack explicit boundary information for precise localization. BorderAlign extracts representative border features to enhance those point features, and BorderDet uses them for classification and regression. The method improves FCOS from 38.6 to 41.4 AP and reaches 50.3 AP with a ResNeXt-101-DCN backbone.
Problem
Point features are convenient for dense detection but may lack sufficient full-instance and boundary information for accurate localization.
Method
BorderAlign directly pools representative border features, and BorderDet uses Border Alignment Modules in classification and box regression.
Results
BorderDet improves FCOS from 38.6 to 41.4 AP and achieves 50.3 AP with a ResNeXt-101-DCN backbone.
Takeaways & Limitations
Explicit border features improve dense-detector localization and support stronger classification and regression within the BorderDet architecture.
Abstract
from arXiv · showhide
Dense object detectors rely on the sliding-window paradigm that predicts the object over a regular grid of image. Meanwhile, the feature maps on the point of the grid are adopted to generate the bounding box predictions. The point feature is convenient to use but may lack the explicit border information for accurate localization. In this paper, We propose a simple and efficient operator called Border-Align to extract "border features" from the extreme point of the border to enhance the point feature. Based on the BorderAlign, we design a novel detection architecture called BorderDet, which explicitly exploits the border information for stronger classification and more accurate localization. With ResNet-50 backbone, our method improves single-stage detector FCOS by 2.8 AP gains (38.6 v.s. 41.4). With the ResNeXt-101-DCN backbone, our BorderDet obtains 50.3 AP, outperforming the existing state-of-the-art approaches. The code is available at (https://github.com/Megvii-BaseDetection/BorderDet).
1 Introduction
Dense detectors use convenient single-point features, but these may lack full-instance and boundary information for precise localization. BorderDet addresses this gap with BorderAlign and reports gains for FCOS and FPN.
- 1 Introduction: Single-point features are efficient because they require no additional feature extraction, but their limited receptive field may omit instance and boundary information.These limitations affect full-instance representation and precise box regression.
- 1 Introduction: BorderDet supplements single-point features with border features to strengthen dense object detection.The paper identifies border information as significant for the single-point representation.
- 1 Introduction: BorderAlign directly pools features from each object boundary and adaptively emphasizes representative border locations such as extreme points.Unlike whole-box feature extraction, it focuses on the object border.
- 1 Introduction: BorderDet uses Border Alignment Modules to refine classification and box regression, with lower computation than similar feature-enhancement methods and compatibility with anchor-based or anchor-free detectors.The architecture is described as easily integrable into dense detectors.
2 Related Works
Related work improves dense-detector representations through adaptive or pooled features, while border-localization methods aggregate RoI features along spatial axes. BorderDet instead targets efficient border-feature exploitation for accurate localization.
- 2 Related Works: Point-based features are standard in sliding-window detectors but can limit representation of both classification and localization information.Prior methods including Guided Anchor and Cascade-RPN improve point features with deformable or adaptive convolution.
- Border Localization: Border-localization methods such as LocNet and SABL aggregate RoI features along the X- and Y-axes to predict object borders.Their pipelines rely heavily on high-resolution RoI feature maps.
- Border Localization: Whole-box feature extraction can add unnecessary computation, include background, and extract border information only implicitly.These limitations motivate explicit border-feature extraction for dense detection.
3 Our Approach
BorderDet supplements dense detectors’ point features with explicitly extracted border features, using BorderAlign within a two-stage refinement architecture. The method targets stronger feature representation and more efficient boundary extraction for classification and localization.
- 3.1 Motivation: 1.3 AP improvement comes from enhancing FCOS’s single-point feature with region features, while border-only features lose just 0.3 AP and middle-border features match region performance with fewer samples.These comparisons motivate focusing on representative border locations rather than densely extracting whole-region features.
- 3.2 Border Align: BorderAlign uses (4+1)C feature maps for the single point and four borders, then independently max-pools N sampled points per border to select representative extreme-point features.N is set to 10 by default, and bilinear interpolation avoids quantization error when evaluating feature values.
- 3.2 Border Align: The border-sensitive maps activate in corresponding object regions, including strong whole-object responses for single-point channels and left-boundary responses for left-border channels.These responses support BorderAlign’s extraction of localized border information.
- 3.3 Network Architecture: BorderDet first predicts coarse classification scores and box locations, then feeds them with feature maps into BAM to produce border-aware features and refined predictions.The final predictions unify coarse and border classifications and locations; border classification is category-aware for overlapping boundaries.
- 3.3 Network Architecture: BorderDet adds negligible computation through an efficient structure and layer sharing, while remaining plug-and-play for detectors including RetinaNet and FCOS.The same border-alignment approach also supports two-stage proposal generation through BorderRPN.
- 3.4 Model Training and Inference: The model is optimized end-to-end with a multi-task loss, using normalized regression targets whose variance is 0.5 by default.The regression normalization uses the coarse box width and height.
4 Experiments
Experiments show that adding Border Alignment Modules to both classification and regression improves FCOS performance, especially at high IoU thresholds, while BorderAlign remains effective across feature and detector settings.
- Ablation Study: Classification-branch BAM provides a 1.1 AP gain, mainly at low IoU thresholds, by rescoring boxes using border features.The improvement diminishes as the IoU threshold increases because high-threshold performance is limited by the lack of high-quality boxes.
- Ablation Study: Regression-branch BAM raises performance from 38.6 to 39.7 and adds 2.6 AP90, concentrating gains at high IoU thresholds.The result indicates improved localization accuracy for detected bounding boxes.
- Ablation Study: 41.4 AP versus 38.6 for the baseline results when BAM is applied to both branches, with gains across IoU thresholds.AP50 increases by 2.2 and AP90 by 3.5; AP90 improves by 20% over the baseline.
- Ablation Study: Channel-wise max-pooling achieves the best aggregation result at 41.4 AP, while border-sensitive feature maps facilitate border-feature extraction.The border-sensitive maps can activate different channels around extreme points on different borders.
- Analysis of BorderDet: BorderAlign learns to sample near extreme border points, and BorderDet increases valid predictions with IoU above 0.5 by about 30%.The mean normalized distance to extreme points is nearly zero and its variance decreases during training.
- Generalization: BorderDet improves RetinaNet by 2.3 AP, demonstrating integration beyond the FCOS baseline.The method is evaluated by adding the border alignment module without modifying RetinaNet settings.
- Comparisons with State-of-the-art Detectors: 50.3 AP makes BorderDet state of the art among existing one-stage and two-stage methods under advanced settings.With the standard setting, BorderDet reaches 43.2 AP and surpasses several anchor-free approaches.
5 Conclusion
BorderDet extracts border features during classification and regression through BorderAlign, enabling adaptive extreme-point feature extraction to improve object localization.
- 5 Conclusion: BorderDet uses BorderAlign to extract border features in classification and regression, adaptively learning features around each border’s extreme points.The conclusion describes BorderDet as a simple and effective architecture and reports validation against previous feature-refinement operations.