RepPoints V2: Verification Meets Regression for Object Detection

Motivation

• Verification can be easier to infer when features are spatially aligned, whereas regression is more efficient and supports continuous target variables for localization refinement.Source: §1 · p. 1

Background

• Earlier hybrid detectors verified coarse anchor or proposal hypotheses before regressing offsets, while recent pure-regression methods such as RepPoints directly regress object extent from feature-map positions.Source: §1 · p. 1

Method

• RepPoints v2 fuses verification with regression through auxiliary side-branches at the feature and result levels without disrupting intermediate representations.The fusion is intended to improve features through multi-task learning, enhance features with verification cues, and combine both methods during inference.Source: §1 · p. 2

Results

• RepPoints v2 improves the original RepPoints by about 2.0 mAP on COCO across different backbones and reaches 52.1 mAP on test-dev with a single ResNeXt-101-DCN model.The approach also improves localization and classification ability through different verification techniques.Source: §1 · p. 2

Generalization

• The approach generalizes beyond RepPoints, improving FCOS by 1.3 mAP and Dense RepPoints instance segmentation by 1.3 mAP to 38.9 mask mAP on COCO test-dev.Source: §1 · p. 2

Verification based object detection

• Verification-based detectors classify windows or factorize box hypotheses into corner-verification subproblems, while regression branches may refine continuous corner coordinates.Source: §2 · p. 2

Regression based object detection

• Pure regression detectors regained attention after feature pyramids and focal loss alleviated large target displacements, multiple objects per feature bin, and class imbalance.Source: §2 · p. 2
• The proposed method adds verification to regression detectors without disrupting their flow and is generally applicable beyond RepPoints.Source: §2 · p. 3

Hybrid approaches

• Unlike conventional hybrid detectors that verify and regress serially, this approach runs the methodologies mostly in parallel and uses corner verification to complement regression.Source: §2 · p. 3

3.1 A Brief Review of a Pure Regression Method: RepPoints

• RepPoints directly regresses point sets from feature-map positions and converts them into bounding boxes, avoiding anchors while retaining competitive localization accuracy.The conversion uses min-max, partial min-max, or moment functions over the point sets.Source: §3.1 · p. 3

3.2 Verification Tasks

• Corner and within-box foreground verification provide complementary localization cues: corners describe object bounds precisely, while foreground scores provide information throughout the box.Corner verification predicts corner scores and sub-pixel offsets; foreground verification can use category-aware heatmaps.Source: §3.2.1 · p. 3, §3.2.2 · p. 4

3.2.1 Corner Point Verification

• Corner verification is adapted to FPNs by assigning every ground-truth corner to every feature-map level, which performs better than using only the highest-resolution level.The method uses focal loss for corner scores, smooth L1 loss for offsets, and reduced weights for nearby negative samples.Source: §3.2.1 · pp. 3–4

3.2.2 Within-box foreground verification

• Within-box foreground verification distinguishes object categories with a C-channel heatmap and uses normalized focal loss to prevent large objects from dominating training.Each channel represents the probability that a feature point belongs to the corresponding category.Source: §3.2.2 · p. 4

3.3 A General Fusion Method

• Auxiliary verification side-branches fuse with the regression branch only at feature and result levels, supplying richer supervision and verification cues without altering intermediate representations.Verification outputs are embedded to the regression feature dimension, while detached copies prevent the embedding path from affecting verification-task learning.Source: §3.3 · pp. 4–5

3.4 RepPoints v2: Fusing Verification into RepPoints

• RepPoints v2 uses explicit top-left and bottom-right corner points, adds corner and foreground side-branches, and applies corner verification for joint inference alongside regression.Corner verification supports multi-task learning, feature enhancement, and joint inference; foreground verification supports multi-task learning and feature enhancement.Source: §3.4 · pp. 5–6, §3.3 · p. 5

3.5 Extension to Other Detectors and Problems

• The fusion method improves FCOS by 1.3 mAP on COCO test-dev and improves Dense RepPoints instance segmentation by 1.3 mask AP.For FCOS, corner and foreground outputs are fused into the main branch before joint inference; Dense RepPoints uses contour and category-aware mask verification.Source: §3.5 · p. 6

4.2 Ablation Study

• The explicit-corners variant performs on par with original RepPoints variants and becomes 0.2-0.3 mAP better than other variants when verification is added.The authors attribute this difference to more effective interaction between verification and regression tasks.Source: §4.2 · p. 6, Table 2 · p. 7
• Corner verification adds 1.4 mAP, while foreground verification adds another 0.5 mAP, with benefits concentrated at higher and lower IoU criteria respectively.Corner verification improves AP90 by 4.0 mAP and AP50 by 0.2 mAP; foreground verification improves AP50 by 0.9 AP while AP90 remains about the same.Source: §4.2 · p. 7, Table 3 · p. 7
• The full fusion approach surpasses its counterpart without verification modules by 1.9 mAP after multi-task learning, feature enhancement, and joint inference.The individual gains are 0.4 mAP, 0.7 mAP, and 0.8 mAP, respectively.Source: §4.2 · p. 7, Table 4 · p. 7
• About 2.0 mAP gains are maintained over stronger RepPoints baselines using longer, multi-scale training and stronger backbones.These results indicate that the proposed approach is largely complementary to improved baseline architecture.Source: §4.2 · p. 7, Table 5 · p. 8

4.3 Comparison to State-of-the-art Methods

• 52.1 mAP is achieved on COCO test-dev with multi-scale testing, following 46.0 mAP with ResNet-101 and 49.4 mAP with ResNeXt-101 and DCN.The comparison uses GIoU loss in the regression branch; the 52.1 mAP result is obtained with additional multi-scale tests.Source: §4.3 · p. 7, Table 6 · p. 8

4.4 Extension to Other Detectors and Applications

• Direct application improves FCOS by 1.3 mAP and improves Dense RepPoints instance segmentation by 1.3 mAP with additional verification modules.The instance-segmentation extension uses contour and foreground modules.Source: §4.4 · p. 8

5 Conclusion

• RepPoints v2 fuses verification tasks with regression and improves the original RepPoints under different backbones and training approaches.It also achieves 52.1 mAP on COCO test-dev.Source: §5 · p. 8
• The verification module is applied to FCOS as a broader object-detection framework.Table 7 documents this application.Source: §5 · p. 8, Table 7 · p. 8
• The verification module is also added to Dense RepPoints for instance segmentation on COCO test-dev.Tables 8 and 9 document this application.Source: §5 · p. 8, Table 8 · p. 9

Broader Impact

• Object detection and instance segmentation applications may include natural images, medical images, and video data.Source: Broader Impact · p. 9
• Detection failures may cause the model not to detect objects correctly.Source: Broader Impact · p. 9
• Dataset biases and domain gaps may cause performance drops on datasets differing from the training data.Source: Broader Impact · p. 9

A.1 Corner Point Verification

• Corner verification assigns each ground-truth corner as a positive location and other locations as negative, with reduced penalties near positives.Source: §A.1 · p. 11
• The corner branch predicts offsets only at positive corner locations using the ground-truth corner offset target.Source: §A.1 · p. 11
• The corner branch combines modified focal heatmap loss and offset loss by summation.Source: §A.1 · p. 11

A.2 Within-box Foreground Verification

• Within-box verification applies normalized focal loss to category scores at feature-map locations.Source: §A.2 · p. 11
• The normalizing factor depends on the area of the object containing each location and uses the smallest size when same-category objects collide.Source: §A.2 · p. 11
• The normalized focal-loss formulation sets α to 0.25 and γ to 2.Source: §A.2 · p. 11
• Dense RepPoints instance segmentation is evaluated with the added verification module on COCO test-dev.Source: Table 9 · p. 12

• The overall loss combines LRepPoints, Lcorner, and Lfg.The corner and foreground terms are defined elsewhere in the method.Source: §A.3 · p. 12
• λ1 = 0.25 and λ2 = 0.1 weight the corner and foreground losses, respectively.Source: §A.3 · p. 12
• Lcorner and Lfg are the added loss components in the overall objective.Source: §A.3 · p. 12

Training settings

• The method is based on Dense RepPoints [31], with one heatmap predicting all contour points because contour points have no type.Other parameters, network architectures, and training details match those used for object detection.Source: B Extension to Instance Segmentation · p. 12

Joint inference

• Joint inference refines representative points by adding the highest-scoring nearby contour point when its contour-heatmap score exceeds 0.5.The candidate contour point must lie within distance 1 of the representative point.Source: B Extension to Instance Segmentation · p. 12

Experimental results

• Adding the verification module improves instance-segmentation performance by 1.0 mAP, while joint inference adds another 0.3 mAP.The experiment uses a ResNet-50 backbone and a 3x scheduler.Source: B Extension to Instance Segmentation · p. 12

Visualization

• RepPoints v2 provides more precise localization than RepPoints v1 in COCO 2017 comparisons using the same ResNet-50 backbone and 1x scheduler.Figure 3 compares the two methods under these matched settings.Source: C Visualization · p. 12, Figure 3 · p. 13
• RepPoints v2 visualizes representative points, foreground predictions, top-left corner predictions, and bottom-right corner predictions as informative component cues.These components are presented from left to right in Figure 4.Source: C Visualization · p. 12, Figure 4 · p. 14
• The full RepPoints v2 version achieves better localization results than the version without joint inference in the illustrated detections.Figure 3 distinguishes detections without joint inference from those using joint inference.Source: Figure 3 · p. 13