PaperlayerView exact revision on arXiv

Source-linked AI summary

CornerNet: Detecting Objects as Paired Keypoints

Hei Law, Jia Deng

arXiv:1808.01244v2cs.CV

TL;DR

Anchor-box design complicates one-stage object detection with many boxes and hyperparameters. CornerNet instead detects bounding boxes as paired top-left and bottom-right keypoints, achieving 42.2% AP on MS COCO and outperforming existing one-stage detectors.

  • Problem

    Anchor boxes require many boxes and design choices, complicating one-stage object detection and creating substantial positive-negative imbalance.

  • Method

    CornerNet detects each bounding box as paired top-left and bottom-right keypoints using one convolutional network, with corner pooling to localize corners.

  • Results

    42.2% AP on MS COCO, outperforming all existing one-stage detectors.

  • Takeaways & Limitations

    CornerNet demonstrates that paired-corner object detection can achieve competitive results without anchor boxes.

Abstract

from arXiv · show

We propose CornerNet, a new approach to object detection where we detect an object bounding box as a pair of keypoints, the top-left corner and the bottom-right corner, using a single convolution neural network. By detecting objects as paired keypoints, we eliminate the need for designing a set of anchor boxes commonly used in prior single-stage detectors. In addition to our novel formulation, we introduce corner pooling, a new type of pooling layer that helps the network better localize corners. Experiments show that CornerNet achieves a 42.2% AP on MS COCO, outperforming all existing one-stage detectors.

1 Introduction

CornerNet detects objects as paired top-left and bottom-right keypoints, eliminating anchor-box design through heatmaps and learned corner embeddings. It introduces corner pooling for corner localization and achieves 42.2% AP on MS COCO, outperforming existing one-stage detectors.

  • Motivation: Anchor boxes require large candidate sets, including more than 40k in DSSD and more than 100k in RetinaNet, because sufficient overlap with ground-truth boxes is otherwise difficult to ensure.Only a tiny fraction of the densely placed anchor boxes overlap with ground-truth boxes.
  • Motivation: Anchor-box detectors also introduce numerous hyperparameters and design choices for box counts, sizes, aspect ratios, and multiscale configurations.The passage describes these choices as largely ad-hoc heuristics that become more complicated across multiple prediction resolutions.
  • CornerNet formulation: CornerNet represents each object by pairing its top-left and bottom-right corners, using heatmaps and embeddings to group corners belonging to the same object.The convolutional network predicts separate heatmaps for both corner types and an embedding vector for each detected corner.
  • Corner pooling: Corner pooling helps convolutional networks localize bounding-box corners by combining directional boundary evidence when corners lie outside the object.Top-left corners require looking rightward for the topmost boundary and downward for the leftmost boundary.
  • Results: 42.2% AP is achieved by CornerNet on MS COCO, outperforming all existing one-stage detectors; ablations identify corner pooling as critical to this performance.The authors also state that their approach eliminates the need to design anchor boxes.

2 Related Works

Related work spans two-stage and one-stage object detectors, including anchor-box-based and anchor-free formulations. CornerNet is inspired by associative embedding and formulates object detection as detecting and grouping corners with embeddings.

  • Two-stage detectors: Two-stage detectors generate sparse regions of interest and classify them with a network, though independently processing regions creates redundant computation.R-CNN introduced and popularized this approach; later methods improved its efficiency.
  • One-stage detectors: One-stage detectors remove RoI pooling and detect objects in a single network, generally achieving greater computational efficiency while maintaining competitive benchmark performance.YOLO and SSD popularized this approach.
  • Anchor-box methods: Anchor-box-based one-stage methods densely place candidate boxes, classify and refine them, or directly predict bounding-box coordinates.SSD uses multiscale anchor boxes, while YOLO directly predicts coordinates and later adopts anchor boxes in YOLO9000.
  • Anchor-free methods: DeNet generates regions without anchor boxes by detecting corner likelihoods, enumerating corner combinations, and then classifying each region with a two-stage pipeline.Its formulation differs from CornerNet, as indicated by the paper’s explicit contrast.
  • Associative embedding: CornerNet draws on associative embedding for grouping human joints and, to the authors’ knowledge, is the first to formulate object detection as detecting and grouping corners with embeddings.Associative embedding assigns detected joints embedding vectors and groups them by embedding distances.

3 CornerNet

CornerNet detects each object as a top-left and bottom-right keypoint pair, using heatmaps, embeddings, and offsets to locate and group corners. Its corner pooling layer encodes directional boundary context to improve corner localization.

  • Object representation: CornerNet represents each bounding box as a top-left and bottom-right corner pair predicted by a single convolutional network.The network predicts separate heatmaps for the two corner types and categories.
  • Architecture: The model uses an hourglass backbone followed by separate prediction modules for top-left and bottom-right corners.Each prediction module includes corner pooling before predicting heatmaps, embeddings, and offsets.
  • Corner localization: Location offsets correct quantization errors caused by heatmap downsampling before corner coordinates are remapped to the input resolution.The offsets particularly address precision loss that can harm the IoU of small bounding boxes.
  • Corner grouping: One-dimensional embeddings group top-left and bottom-right corners from the same object by small distances and separate corners from different objects.Only embedding distances are used for grouping; CornerNet uses one-dimensional embeddings with pull and push losses.
  • Corner pooling: Corner pooling improves corner localization by max-pooling features along the directions needed to find an object’s adjacent boundaries.Top-left pooling aggregates rightward and downward context, while bottom-right pooling aggregates leftward and upward context.

4 Experiments · 4.1 Training Details

CornerNet is trained from scratch in PyTorch with focal-loss-specific initialization, 511 × 511 inputs, and standard augmentation. Training uses Adam with specified loss weights, batch size, multi-GPU settings, and distinct schedules for ablations versus detector comparisons.

  • 4.1 Training Details: CornerNet is implemented in PyTorch and randomly initialized without pretraining on any external dataset.The convolutional biases predicting corner heatmaps are set following focal-loss practice.
  • 4.1 Training Details: 511 × 511 training inputs produce 128 × 128 network outputs, with standard data augmentation used to reduce overfitting.
  • 4.1 Training Details: Adam is used to optimize the full training loss.
  • 4.1 Training Details: The pull and push loss weights are both set to 0.1, while the offset loss weight is set to 1.Values of 1 or larger for the pull and push weights lead to poor performance.
  • 4.1 Training Details: Reducing the penalty assigned to negative locations near positive locations significantly improves network performance.
  • 4.1 Training Details: Corner pooling consistently improves performance for detecting corners across different image quadrants and remains effective over small and large areas.
  • 4.1 Training Details: Training uses a batch size of 49 across 10 Titan X (PASCAL) GPUs, with 4 images on the master GPU and 5 images per remaining GPU.
  • 4.1 Training Details: Ablation experiments run for 250k iterations at a learning rate of 2.5 × 10−4 to conserve GPU resources.

4.2 Testing Details

CornerNet testing preserves the original image resolution, uses both original and flipped inputs, and post-processes corner predictions into detections. The procedure reports the top 100 detections and averages 244ms per image on a Titan X (PASCAL) GPU.

  • The post-processing applies 3×3 max-pooling NMS, selects the top 100 corners of each type, adjusts locations using offsets, and pairs corners using embedding distances.
  • Testing maintains the original image resolution and pads the image before feeding it to CornerNet.
  • Both original and flipped images are tested, their detections are combined, soft-NMS suppresses redundancies, and only the top 100 detections are reported.
  • 244ms per image is the average inference time on a Titan X (PASCAL) GPU.

4.3 MS COCO

The MS COCO evaluation uses designated training, validation, and test splits, with held-out validation images reserved for hyper-parameter search and ablation studies. Qualitative and ablation evidence highlights corner pooling’s localization benefit and the importance of the hourglass network.

  • Evaluation protocol: CornerNet is evaluated on MS COCO using 80k training, 40k validation, and 20k test images.All training images and 35k validation images are used for training; the remaining 5k validation images support hyper-parameter search and ablation studies.
  • Corner localization: Qualitative examples show that corner pooling helps better localize corners.
  • Ablation study: The hourglass network is crucial to CornerNet’s performance.

4.4 Ablation Study

The ablations show that corner pooling, object-dependent penalty reduction, and the hourglass backbone each materially improve CornerNet. Corner pooling benefits detection across object sizes and corner locations, while the hourglass backbone is especially important relative to FPN and anchor-based alternatives.

  • Corner pooling: 2.0% AP, 2.1% AP50, and 2.1% AP75 improvements result from adding corner pooling with the same parameter count.Corner pooling also improves AP by 2.4% for medium objects and 3.6% for large objects.
  • Corner pooling: 3.1% and 2.7% top-left corner mAP gains, and 2.8% and 2.6% bottom-right corner mAP gains, occur across upper-left and lower-right quadrants.The results indicate similar corner-pooling improvements at different image quadrants.
  • Penalty reduction: 2.7% AP, 1.5% APm, and 5.3% APl improvements come from fixed-radius penalty reduction over the baseline.Object-dependent radius further improves AP by 2.8%, APm by 2.0%, and APl by 5.8%, especially benefiting medium and large objects.
  • Backbone network: 8.2% AP separates CornerNet with the hourglass backbone from CornerNet with FPN, while 5.5% AP separates it from an anchor box detector using hourglass.Both comparisons initialize networks from scratch and use the same training procedure.

4.5 Comparisons with state-of-the-art detectors

On MS COCO test-dev, CornerNet is compared with state-of-the-art detectors and achieves 42.2% AP with multi-scale evaluation, attaining state-of-the-art performance among one-stage methods while remaining competitive with two-stage methods.

  • CornerNet is compared with other state-of-the-art detectors on MS COCO test-dev.
  • 42.2% AP is achieved with multi-scale evaluation, making CornerNet state of the art among existing one-stage methods and competitive with two-stage methods.

5 Conclusion

CornerNet is presented as an object-detection approach that represents bounding boxes as pairs of corners and achieves competitive results on MS COCO.

  • 5 Conclusion: CornerNet detects bounding boxes as pairs of corners, providing its central formulation for object detection.The paper presents this as a new approach to object detection.
  • 5 Conclusion: CornerNet demonstrates competitive results when evaluated on MS COCO.The conclusion reports the evaluation dataset and overall outcome without specifying an additional metric.
Loading 1808.01244v2…