PaperlayerView exact revision on arXiv

Source-linked AI summary

CornerNet-Lite: Efficient Keypoint Based Object Detection

Hei Law, Yun Teng, Olga Russakovsky, Jia Deng

arXiv:1904.08900v2cs.CV

TL;DR

CornerNet’s accuracy comes with high processing cost, motivating efficient keypoint-based detection. CornerNet-Lite addresses this with attention-based CornerNet-Saccade and a compact-backbone CornerNet-Squeeze, improving offline efficiency and real-time accuracy. On COCO, Saccade is 6.0x faster with a 1% AP increase, while Squeeze reaches 34.4% AP at 30ms versus YOLOv3’s 33.0% AP at 39ms.

  • Problem

    CornerNet achieves state-of-the-art single-stage accuracy but its 1.1s inference cost limits use in real-time or interactive applications.

  • Method

    CornerNet-Lite combines CornerNet-Saccade, which uses attention to select high-resolution crops, and CornerNet-Squeeze, which uses a compact hourglass backbone to reduce per-pixel processing.

  • Results

    On COCO, CornerNet-Saccade improves CornerNet from 42.2% to 43.2% AP while providing a 6.0x speed-up, and CornerNet-Squeeze reaches 34.4% AP at 30ms versus YOLOv3’s 33.0% AP at 39ms.

  • Takeaways & Limitations

    CornerNet-Lite shows that keypoint-based detection can support processing-efficient applications through separate offline and real-time variants.

  • Takeaways & Limitations

    Combining saccades with CornerNet-Squeeze makes detection slower and less accurate because its compact single-scale architecture has limited attention-map capacity and little pixel-saving room.

Abstract

from arXiv · show

Keypoint-based methods are a relatively new paradigm in object detection, eliminating the need for anchor boxes and offering a simplified detection framework. Keypoint-based CornerNet achieves state of the art accuracy among single-stage detectors. However, this accuracy comes at high processing cost. In this work, we tackle the problem of efficient keypoint-based object detection and introduce CornerNet-Lite. CornerNet-Lite is a combination of two efficient variants of CornerNet: CornerNet-Saccade, which uses an attention mechanism to eliminate the need for exhaustively processing all pixels of the image, and CornerNet-Squeeze, which introduces a new compact backbone architecture. Together these two variants address the two critical use cases in efficient object detection: improving efficiency without sacrificing accuracy, and improving accuracy at real-time efficiency. CornerNet-Saccade is suitable for offline processing, improving the efficiency of CornerNet by 6.0x and the AP by 1.0% on COCO. CornerNet-Squeeze is suitable for real-time detection, improving both the efficiency and accuracy of the popular real-time detector YOLOv3 (34.4% AP at 30ms for CornerNet-Squeeze compared to 33.0% AP at 39ms for YOLOv3 on COCO). Together these contributions for the first time reveal the potential of keypoint-based detection to be useful for applications requiring processing efficiency.

1 Introduction

CornerNet-Lite introduces two efficient variants of keypoint-based CornerNet that target the high inference cost limiting its use in real-time or interactive applications. CornerNet-Saccade reduces processed pixels, while CornerNet-Squeeze reduces processing per pixel, covering offline and real-time detection use cases.

  • Keypoint-based detection: Keypoint-based detection generates bounding boxes by detecting and grouping keypoints, eliminating the need for anchor boxes; CornerNet detects top-left and bottom-right corners.CornerNet uses stacked hourglass networks for corner heatmaps and associate embeddings for grouping.
  • Motivation: 42.2% AP at 1.1s per image makes CornerNet too slow for video applications requiring real-time or interactive rates.Reducing processing to 42ms can lower CornerNet’s AP to 25.6%, versus 33.0% AP for YOLOv3 at 39ms.
  • Approach: CornerNet-Lite explores two efficiency directions: reducing the number of pixels processed and reducing the amount of processing per pixel.The two variants are CornerNet-Saccade and CornerNet-Squeeze.
  • CornerNet-Saccade: 43.2% AP at 190ms per image gives CornerNet-Saccade a 1% AP increase and 6.0x speed-up over original CornerNet on COCO.It uses a downsized full image to generate an attention map, then processes a selected subset of crops at high resolution.
  • CornerNet-Squeeze: 34.4% AP at 30ms makes CornerNet-Squeeze both more accurate and faster than YOLOv3 at 33.0% AP and 39ms on COCO.Its compact hourglass backbone uses ideas from SqueezeNet and MobileNets, including 1×1 convolution, bottleneck layers, and depth-wise separable convolution.
  • Scope and trade-offs: CornerNet-Squeeze-Saccade is slower and less accurate than CornerNet-Squeeze because the compact model lacks capacity for sufficiently accurate attention maps and already operates at a single scale.This combination offers less opportunity for saccades to reduce processed pixels.
  • Significance: CornerNet-Saccade provides offline efficiency without sacrificing accuracy, while CornerNet-Squeeze provides improved accuracy at real-time efficiency.Together, the variants make keypoint-based detection relevant to applications requiring processing efficiency.

2 Related Work

Saccade-based detection selectively crops image regions during inference, while prior systems differ in whether each crop yields one object or multiple detections. CornerNet-Saccade instead uses one crop-processing type that can detect multiple objects, reducing the number of crops needed relative to several R-CNN-based approaches.

  • Saccades in Object Detection: Saccades selectively crop and process image regions during inference, taking forms such as cascades that repeatedly select regions for further processing.The term broadly covers sequential or parallel processing of pixels or features.
  • Saccades in Object Detection: R-CNN variants process potential-object crops into either rejected proposals or single labeled boxes through classification and regression.Cascade R-CNN extends this pattern by iteratively rejecting or refining proposals.
  • Saccades in Object Detection: AutoFocus combines region prediction with Faster R-CNN processing, using one crop type that can produce multiple objects and another that produces at most one.Its cropping operates at two levels, including subroutine processing within Faster R-CNN.
  • Saccades in Object Detection: CornerNet-Saccade uses one crop-processing type, and each crop can produce multiple objects without additional subcrops.Consequently, it can process fewer crops than the number of objects, unlike R-CNN variants and AutoFocus.
  • Efficient Object Detectors: Efficient detector research reduces redundant computation through fully convolutional feature extraction, while one-stage detectors remove region pooling.These approaches address inefficiencies introduced by repeatedly applying ConvNets to regions of interest.
  • Efficient Network Architectures: Efficient network architectures target mobile and embedded applications using designs such as SqueezeNet fire modules and MobileNet depth-wise separable convolutions.SqueezeNet reduces parameters, while MobileNets target accuracy-latency trade-offs.

3 CornerNet-Saccade

CornerNet-Saccade estimates likely object locations from downsized images, zooms into selected regions at scale-dependent resolutions, and detects and merges objects efficiently. Its design combines attention-based crop selection with boundary filtering, redundant-location suppression, parallel processing, and a new Hourglass-54 backbone.

  • Estimating Object Locations: CornerNet-Saccade predicts attention maps and coarse boxes from downsized images, then detects objects in high-resolution regions centered on likely locations.The attention maps encode locations and coarse object scales.
  • Estimating Object Locations: Attention maps separate small, medium, and large objects, enabling different zoom levels for each size.The network uses separate maps for three object-size ranges and zooms more for smaller objects.
  • Post-processing: CornerNet-Saccade removes boxes touching crop boundaries and suppresses nearby redundant locations before merging detections with Soft-NMS.These operations address partial objects at crop edges and overlapping regions around nearby objects.
  • Estimating Object Locations: During testing, CornerNet-Saccade processes locations whose attention scores exceed t = 0.3.The attention maps are produced with convolutional modules and trained using focal loss with α = 2.
  • Zooming and Detection: For each possible location, the method enlarges the downsized image by scale factors ss = 4, sm = 2, and sl = 1 before applying a 255×255 detection window.The scale factors follow ss > sm > sl because smaller objects require more zoom.
  • Backbone Network: The method uses Hourglass-54, a three-module, 54-layer backbone, instead of CornerNet’s two-module, 104-layer Hourglass-104.Each Hourglass-54 module is shallower and has fewer parameters than its Hourglass-104 counterpart.

4 CornerNet-Squeeze

CornerNet-Squeeze reduces per-pixel computation by replacing CornerNet’s expensive backbone components with a compact hourglass architecture inspired by SqueezeNet and MobileNets. Its design uses fire modules, depth-wise separable convolutions, additional downsampling, and other resolution-reducing changes, while avoiding delayed downsampling because high-resolution convolutions are costly.

  • Compact Hourglass Architecture: CornerNet-Squeeze targets per-pixel efficiency by replacing the expensive Hourglass-104 with a lightweight hourglass architecture.The design draws on SqueezeNet and MobileNets to reduce complexity.
  • Ideas from SqueezeNet and MobileNets: The fire module replaces CornerNet residual blocks, using 1×1 filters and a depth-wise separable 3×3 convolution to improve inference time.The module combines SqueezeNet-inspired channel reduction with MobileNet-inspired separable convolution.
  • Design Constraint: Delayed downsampling is omitted because the hourglass symmetry would preserve higher-resolution feature maps during upsampling, making convolution computationally expensive.The stated constraint is achieving real-time detection.
  • Compact Hourglass Architecture: CornerNet-Squeeze adds an initial downsampling layer, removes one within each hourglass module, replaces prediction-module 3×3 filters with 1×1 filters, and changes the upsampling operation.The image is downsized three times before the hourglass module rather than twice in CornerNet.
  • Training Details: CornerNet-Squeeze retains CornerNet’s training losses and hyperparameters, changing only the batch size.Downsizing the image once more before the hourglass modules reduces memory usage fourfold under the same image resolution.

5 Experiments

Experiments evaluate CornerNet-Lite variants on COCO, including training efficiency, backbone choices, ablations, and comparisons with CornerNet and YOLOv3.

  • Training Efficiency: More than 60% GPU memory savings and four GPUs were sufficient to train CornerNet-Saccade, compared with CornerNet's ten GPUs and 120GB.CornerNet-Saccade used four 1080Ti GPUs with 44GB total memory, while CornerNet required ten Titan X GPUs with 120GB.
  • Hourglass-54: 42.6% AP with Hourglass-54 exceeded 41.4% AP with Hourglass-104 for CornerNet-Saccade on COCO validation.Hourglass-54 also achieved higher attention-map AP and better bounding boxes when combined with saccade.
  • Ablation Studies: Saccade only helps when attention maps are sufficiently accurate; CornerNet-Squeeze-Saccade lacked capacity to predict attention maps and objects simultaneously.Its single-scale operation also leaves less opportunity to reduce processed pixels, and it may process more pixels than CornerNet-Squeeze.

6 Conclusions

CornerNet-Lite combines CornerNet-Saccade and CornerNet-Squeeze to make keypoint-based detection useful for applications requiring processing efficiency.

  • Conclusions: CornerNet-Lite combines two efficient CornerNet variants to address applications requiring processing efficiency.The variants are CornerNet-Saccade and CornerNet-Squeeze.
Loading 1904.08900v2…