Source-linked AI summary
CentripetalNet: Pursuing High-quality Keypoint Pairs for Object Detection
Zhiwei Dong, Guoxuan Li, Yue Liao, Fei Wang, Pengju Ren, Chen Qian
TL;DR
Corner-based detectors remain challenged by matching corners from the same object, especially when similar objects create ambiguous appearance embeddings. CentripetalNet matches corners using predicted spatial shifts toward shared centers, adds cross-star deformable convolution, and achieves strong detection and instance-segmentation performance on MS-COCO test-dev.
Problem
Corner-based detectors must reliably match keypoints from the same object, but associative embedding is sensitive to outliers and similar-looking objects because it lacks position information.
Method
CentripetalNet predicts each corner's centripetal shift toward the box center and matches corner pairs using decoded centers, locations, and cross-star deformable convolution.
Results
CentripetalNet reaches 48.0% AP on MS-COCO test-dev, outperforming existing anchor-free detectors and achieving comparable performance to state-of-the-art instance-segmentation methods.
Takeaways & Limitations
Centripetal shift establishes corner relationships through positional and geometric information, while the detector also supports end-to-end instance segmentation with an anchor-free framework.
Takeaways & Limitations
Associative embedding becomes substantially harder to train with multiple similar objects and is highly sensitive to outliers.
Abstract
from arXiv · showhide
Keypoint-based detectors have achieved pretty-well performance. However, incorrect keypoint matching is still widespread and greatly affects the performance of the detector. In this paper, we propose CentripetalNet which uses centripetal shift to pair corner keypoints from the same instance. CentripetalNet predicts the position and the centripetal shift of the corner points and matches corners whose shifted results are aligned. Combining position information, our approach matches corner points more accurately than the conventional embedding approaches do. Corner pooling extracts information inside the bounding boxes onto the border. To make this information more aware at the corners, we design a cross-star deformable convolution network to conduct feature adaption. Furthermore, we explore instance segmentation on anchor-free detectors by equipping our CentripetalNet with a mask prediction module. On MS-COCO test-dev, our CentripetalNet not only outperforms all existing anchor-free detectors with an AP of 48.0% but also achieves comparable performance to the state-of-the-art instance segmentation approaches with a 40.2% MaskAP. Code will be available at https://github.com/KiveeDong/CentripetalNet.
1. Introduction
The paper targets incorrect corner matching in keypoint-based detection, where appearance-based embeddings can confuse similar objects. CentripetalNet combines positional centripetal shifts with cross-star feature adaptation and mask prediction, reaching strong detection and segmentation performance on MS-COCO.
- Motivation: Corner-based detectors separate corner prediction from matching, but associative embeddings can confuse corners from visually similar, distant objects.Embedding training also becomes harder with multiple similar objects because push and pull losses must distinguish matching points.
- CentripetalNet: CentripetalNet predicts each corner’s spatial offset to the box center and matches corners whose decoded centers are close.This incorporates position information and makes matching more robust to outliers than associative embedding.
- CentripetalNet: Cross-star deformable convolution adapts corner features using the geometric structure and context carried by corner-pooling outputs.Corner pooling extends object location information toward the corners along the cross-star border.
- Results: 47.8% AP versus 42.1% AP for CornerNet on MS-COCO test-dev2017 after equipping the model with centripetal shift and cross-star deformable convolution.The paper also adds an RoIAlign-based mask branch for instance segmentation.
- Results: 48.0% AP on MS-COCO test-dev is reported as the best performance among existing anchor-free detectors.The paper also reports comparable performance with state-of-the-art instance segmentation methods.
2. Related Work
Anchor-free detectors avoid preset anchors through center prediction or keypoint grouping, but keypoint-based methods still depend on reliable grouping. CentripetalNet addresses this matching challenge by connecting corners through predicted spatial relationships to their shared center.
- Anchor-free Approach: Anchor-based detectors require manually designed anchors and suffer from imbalance between negative and positive samples.These limitations motivate anchor-free alternatives.
- Anchor-free Approach: One anchor-free family directly predicts object centers, but sparse positive samples can produce low recall.Examples include YOLOv1, DenseBox, UnitBox, and FCOS.
- Anchor-free Approach: Another family predicts keypoints and groups them into bounding boxes, using methods such as CornerNet, ExtremeNet, and RepPoints.These detectors require specialized grouping procedures to obtain boxes.
- CentripetalNet: CentripetalNet matches predicted corners by comparing their centripetally decoded centers and the geometric center of the candidate box.The overview describes matching corners whose shifted positions are sufficiently close.
- Anchor-free Approach: Keypoint-based methods often achieve high recall with many false detections because matching same-object keypoints remains challenging.CentripetalNet addresses this by encoding corner-to-center relationships with predicted spatial information.
3. CentripetalNet
CentripetalNet predicts corner locations and shifts toward bounding-box centers, matching corners whose decoded centers align. It further adapts corner features with cross-star deformable convolution and supports instance-mask prediction.
- Centripetal Shift Module: CentripetalNet generates corner candidates, predicts centripetal shifts, and matches pairs whose decoded shifted centers are aligned.The centripetal shift encodes the offset from each corner to the bounding-box center.
- Centripetal Shift Module: The centripetal shift uses a logarithm to reduce its numerical range and ease learning.
- Centripetal Shift Module: Corner matching compares decoded corner centers with the center of a candidate box inside a defined central region.Candidate corners are grouped by category and valid top-left/bottom-right ordering before scoring.
- Cross-star Deformable Convolution: Cross-star deformable convolution learns corner-context geometry from guiding shifts and adapts features along the cross-star border.Its offset field explicitly contains context and geometric information and extracts information from the cross-star border.
- Instance Mask Head: An instance mask head uses detection proposals, RoIAlign, and a fully convolutional network to predict 28 × 28 masks.The method selects top-k scored proposals and obtains 14 × 14 RoIAlign features before mask prediction.
4. Experiments
Experiments show that CentripetalNet’s centripetal-shift matching, cross-star deformable convolution, and mask head improve detection and instance segmentation across benchmark and ablation settings.
- Object detection: 48.0% AP is achieved with multi-scale testing on MS-COCO test-dev2017, the best performance among anchor-free detectors.Single-scale performance is 46.1% AP.
- Object detection: CentripetalNet’s AR outperforms all other anchor-free detectors across small, medium, and large objects.The authors attribute recall advantages partly to eliminating high-scored false detections and avoiding dependence on center detection.
- Instance segmentation: 38.8% AP is obtained for single-scale instance segmentation, exceeding Mask R-CNN’s 37.5% AP in the reported comparison.With the same backbone, CentripetalNet is 4.2% AP higher than ExtremeNet and supports end-to-end mask training; its comparable top-ranked performance reaches 40.2% MaskAP.
- Ablation study: The centripetal-shift replacement for associative embedding produces substantial improvement in CornerNet-based matching experiments.The experiment excludes cross-star deformable convolution and the mask head while matching embedding dimensionality for fairness.
- Ablation study: Cross-star deformable convolution outperforms standard deformable convolution and RoI convolution for feature adaption.The authors suggest it better samples the bounding-box border after corner pooling and improves centripetal-shift prediction.
- Qualitative analysis: CentripetalNet removes wrong corner pairs and handles densely concentrated similar objects better than CornerNet and CenterNet in qualitative analysis.It does not rely on center detections, preserving boxes that CenterNet can delete after missed center detections.
5. Conclusion
CentripetalNet introduces centripetal shift to establish relationships between corners using positional and geometric information, and extends the anchor-free detector to instance segmentation. The method is reported to achieve state-of-the-art performance on MS-COCO.
- Centripetal shift establishes relationships between corner keypoints using positional and geometric information.The method addresses ambiguity associated with associative embedding caused by similar appearance.
- CentripetalNet includes an instance segmentation module for end-to-end instance segmentation with an anchor-free detector.
- The method achieves state-of-the-art performance on MS-COCO.