Source-linked AI summary
CenterNet: Keypoint Triplets for Object Detection
Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, Qi Tian
TL;DR
Keypoint-based detectors can produce many incorrect bounding boxes because they lack an additional look into cropped regions. CenterNet addresses this by detecting objects with a center keypoint plus two corners, achieving 47.0% AP on MS-COCO test-dev while closely matching two-stage detectors with efficient inference.
Problem
Keypoint-based one-stage detectors lack access to internal visual patterns within proposed regions, limiting their ability to identify incorrect bounding boxes.
Method
CenterNet detects each object as a triplet of one center keypoint and two corner keypoints, using center pooling and scale-aware central regions to explore proposal interiors efficiently.
Results
47.0% AP on MS-COCO test-dev outperforms existing one-stage detectors by a large margin while closely matching state-of-the-art two-stage detectors with efficient inference.
Takeaways & Limitations
CenterNet equips a one-stage detector with an efficient ability to inspect internal visual information within proposed bounding boxes.
Takeaways & Limitations
CenterNet's performance depends strongly on correctly detecting center keypoints, since missed center keypoints prevent exploration of visual patterns within bounding boxes.
Abstract
from arXiv · showhide
In object detection, keypoint-based approaches often suffer a large number of incorrect object bounding boxes, arguably due to the lack of an additional look into the cropped regions. This paper presents an efficient solution which explores the visual patterns within each cropped region with minimal costs. We build our framework upon a representative one-stage keypoint-based detector named CornerNet. Our approach, named CenterNet, detects each object as a triplet, rather than a pair, of keypoints, which improves both precision and recall. Accordingly, we design two customized modules named cascade corner pooling and center pooling, which play the roles of enriching information collected by both top-left and bottom-right corners and providing more recognizable information at the central regions, respectively. On the MS-COCO dataset, CenterNet achieves an AP of 47.0%, which outperforms all existing one-stage detectors by at least 4.9%. Meanwhile, with a faster inference speed, CenterNet demonstrates quite comparable performance to the top-ranked two-stage detectors. Code is available at https://github.com/Duankaiwen/CenterNet.
1. Introduction
CenterNet extends CornerNet’s corner-pair representation with an additional center keypoint to inspect visual patterns near each proposal’s geometric center. It further introduces center pooling and cascade corner pooling, achieving strong MS-COCO accuracy with efficient inference.
- Motivation: CornerNet avoids anchor boxes by representing each object with a pair of corner keypoints, but its performance is limited by weak global object information.Anchor-based methods require many predefined rectangles to achieve sufficient IoU with ground-truth boxes.
- CenterNet: CenterNet adds one center keypoint near each proposal’s geometric center to perceive visual patterns within the proposed region and assess bounding-box correctness.The method is designed as a low-cost extension of CornerNet.
- Proposed modules: Center pooling enriches center-keypoint predictions with recognizable visual patterns from the object’s central part.The strategy obtains the max summed response in horizontal and vertical directions.
- Results: 47.0% AP on the MS-COCO test-dev set makes CenterNet outperform existing one-stage detectors by a large margin.This result uses both center pooling and cascade corner pooling.
- Results: 270ms and 340ms average inference times per image are reported with 52-layer and 104-layer hourglass backbones, respectively.The paper characterizes CenterNet as efficient while remaining competitive with top-ranked two-stage detectors.
2. Related Work
Object detection methods are broadly divided into two-stage and one-stage pipelines. Two-stage methods first extract RoIs and then classify and regress them, whereas one-stage methods directly classify and regress candidate anchor boxes without RoI extraction.
- Detection Pipelines: Deep-learning object detection approaches are broadly categorized as two-stage or one-stage pipelines.Both pipeline types locate and classify objects.
- Two-Stage Approaches: Two-stage approaches first extract RoIs, then classify and regress the RoIs.R-CNN, SPP-Net, Fast R-CNN, Faster R-CNN, and Mask R-CNN are representative developments in this family.
- One-Stage Approaches: One-stage approaches remove RoI extraction and directly classify and regress candidate anchor boxes.This design avoids the separate RoI extraction stage used by two-stage methods.
- One-Stage Approaches: YOLO, SSD, DSSD, and R-SSD develop one-stage detection through different anchor-box arrangements, regression methods, feature layers, and feature-fusion modules.YOLO uses grid-based anchor boxes, SSD uses dense anchors and multi-layer features, and DSSD adds deconvolution for low- and high-level feature fusion.
3. Our Approach
CenterNet extends CornerNet by representing each object with two corners and a center keypoint, using the center to filter incorrect boxes at low additional cost. It adds scale-aware central-region checking, center pooling, and cascade corner pooling to improve recognition of object interiors and corner context.
- Motivation: CornerNet has a 32.7% FD rate at IoU = 0.05, rising to 60.3% for small objects, partly because it cannot inspect regions inside predicted boxes.FD denotes the proportion of incorrect bounding boxes.
- CenterNet: CenterNet represents each object with a center keypoint and a pair of corners, remaining one-stage while partially inheriting RoI pooling functionality at minimal cost.The center keypoint provides additional information from within each cropped bounding box.
- Center-keypoint filtering: CenterNet filters top-k CornerNet boxes by checking for a same-class center keypoint in a scale-aware central region and replaces the box score with the three-point average.The three points are the top-left corner, bottom-right corner, and center keypoint.
- Scale-aware central regions: The scale-aware region uses n = 3 for bounding boxes less than 150 and n = 5 for bounding boxes greater than 150, balancing small-object recall against large-object precision.Smaller regions reduce recall for small boxes, whereas larger regions reduce precision for large boxes.
- Customized pooling: Center pooling aggregates maximum responses horizontally and vertically to capture recognizable patterns near object centers, while cascade corner pooling adds internal object directions to boundary-direction pooling.These modules address weak center appearance and corners that often lie outside objects.
4. Experiments
CenterNet is evaluated on MS-COCO against CornerNet and state-of-the-art detectors, with strong gains in overall and object-size-specific AP, reduced false detections, and competitive inference speed. Ablations attribute improvements to center-region exploration, center pooling, and cascade corner pooling, while ground-truth center-keypoint results reveal remaining detection-room.
- State-of-the-art comparison: CenterNet511-52 achieves 41.6% single-scale AP, improving 3.8% over CornerNet’s 37.8% under the same setting.With multi-scale testing, it reaches 43.5% AP versus CornerNet’s 39.4%, a 4.1% improvement.
- State-of-the-art comparison: 5.5% and 6.4% are CenterNet511-52’s AP improvements for small objects under single-scale and multi-scale testing, respectively.For Hourglass-104, the corresponding improvements are 6.2% and 8.1%, attributed to center information from center keypoints.
- State-of-the-art comparison: 4.7% and 3.5% are CenterNet511-104’s single-scale AP improvements for medium and large objects, respectively.The medium-object result rises from 42.7% to 47.4%, while the large-object result rises from 53.9% to 57.4%; AR also improves significantly.
- State-of-the-art comparison: 47.0% AP is CenterNet’s best performance, surpassing all published one-stage approaches to the authors’ knowledge.CenterNet511-52’s 41.6% single-scale AP also exceeds deeper-model one-stage approaches such as RetinaNet800 and RefineDet.
- False-detection analysis: 4.5% are the reductions in FD5 for both CenterNet511-52 and CenterNet511-104, while small-box FD decreases by 9.5% and 9.6%, respectively.The FD rate measures the proportion of incorrect bounding boxes, which AP does not directly reflect.
- Ablation studies: 2.3% is the AP gain from adding center heatmaps and triplet detection, while center pooling adds 0.9% and cascade corner pooling adds up to 0.7%.Center pooling improves large-object AP by 1.4%, and cascade corner pooling yields a further 2.2% large-object gain after CenterNet is equipped.
5. Conclusions
CenterNet detects objects with a center keypoint and two corner keypoints, addressing one-stage detectors’ lack of internal information from cropped regions at minimal cost. The authors frame this as adding an efficient discriminator to one-stage detection and suggest extending the center-keypoint branch to other detectors.
- CenterNet detects each object using a triplet of keypoints: one center and two corners.This design addresses CornerNet’s lack of an additional look into proposed cropped regions.
- The framework equips one-stage detection with two-stage-like attention to internal cropped-region information by adding an efficient discriminator.The authors identify this limitation as common to one-stage approaches because they remove RoI extraction.
- The center-keypoint branch could potentially generalize to other one-stage detectors, while advanced training strategies are left for future work.SSD is given as an example of another detector to which the idea might be applied.