PaperlayerView exact revision on arXiv

Source-linked AI summary

Single-Shot Refinement Neural Network for Object Detection

Shifeng Zhang, Longyin Wen, Xiao Bian, Zhen Lei, Stan Z. Li

arXiv:1711.06897v3cs.CV

TL;DR

Object detection seeks to combine the accuracy of two-stage methods with the efficiency of one-stage detectors. RefineDet does so with interconnected anchor-refinement and object-detection modules, linked by feature transfer and trained end to end. It reports state-of-the-art accuracy across VOC and MS COCO while maintaining efficient inference.

  • Problem

    One-stage detectors are computationally efficient but typically less accurate than two-stage methods, partly because of class imbalance.

  • Method

    RefineDet uses an anchor refinement module to filter and adjust anchors, an object detection module for final regression and classification, and transfer connections between them.

  • Results

    RefineDet outperforms published one-stage and two-stage results, reporting 85.8% mAP on VOC 2007, 86.8% mAP on VOC 2012, and 41.8% AP on MS COCO.

  • Takeaways & Limitations

    The proposed single-shot framework combines cascade-like anchor refinement and detection with high inference efficiency, reaching 40.2 FPS at 320 × 320 and 24.1 FPS at 512 × 512.

  • Takeaways & Limitations

    Small-object detection remains challenging, and increasing input size improves performance but burdens inference speed.

Abstract

from arXiv · show

For object detection, the two-stage approach (e.g., Faster R-CNN) has been achieving the highest accuracy, whereas the one-stage approach (e.g., SSD) has the advantage of high efficiency. To inherit the merits of both while overcoming their disadvantages, in this paper, we propose a novel single-shot based detector, called RefineDet, that achieves better accuracy than two-stage methods and maintains comparable efficiency of one-stage methods. RefineDet consists of two inter-connected modules, namely, the anchor refinement module and the object detection module. Specifically, the former aims to (1) filter out negative anchors to reduce search space for the classifier, and (2) coarsely adjust the locations and sizes of anchors to provide better initialization for the subsequent regressor. The latter module takes the refined anchors as the input from the former to further improve the regression and predict multi-class label. Meanwhile, we design a transfer connection block to transfer the features in the anchor refinement module to predict locations, sizes and class labels of objects in the object detection module. The multi-task loss function enables us to train the whole network in an end-to-end way. Extensive experiments on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO demonstrate that RefineDet achieves state-of-the-art detection accuracy with high efficiency. Code is available at https://github.com/sfzhang15/RefineDet

1. Introduction

Object detectors trade accuracy for efficiency: two-stage methods address class imbalance and refine boxes in cascades, while one-stage methods are faster but often less accurate. RefineDet combines these advantages through interconnected refinement and detection modules, achieving strong benchmark accuracy with efficient inference.

  • Two-stage detectors use candidate generation followed by classification and regression, while one-stage detectors densely sample boxes for higher computational efficiency.
  • One-stage detection accuracy typically trails two-stage methods partly because dense sampling creates a class imbalance problem.
  • RefineDet combines two-stage advantages with one-stage efficiency through an anchor refinement module and an object detection module.The ARM removes negative anchors and coarsely adjusts anchor locations and sizes; the ODM further regresses boxes and predicts multi-class labels.
  • A transfer connection block passes ARM features to the ODM, while a multi-task loss trains the complete network end to end.
  • 85.8% mAP on VOC 2007, 86.8% mAP on VOC 2012, and 41.8% AP on MS COCO are reported for RefineDet.
  • 40.2 FPS at 320 × 320 and 24.1 FPS at 512 × 512 demonstrate efficient inference on an NVIDIA Titan X GPU.

2. Related Work

Object detection evolved from sliding-window methods with hand-crafted features toward CNN-based one-stage and two-stage detectors. The related work distinguishes these modern paradigms by their proposal-generation and refinement procedures.

  • Early object detectors applied hand-crafted features and classifiers densely across image grids using a sliding-window paradigm.
  • MS COCO AP averages performance across 10 IoU thresholds from 0.5 to 0.95 over 80 categories.
  • CNN-based detectors are broadly divided into two-stage and one-stage approaches.
  • Two-stage detectors consist of proposal generation followed by subsequent object detection processing.

3. Network Architecture

RefineDet uses a feed-forward detector with interconnected ARM and ODM modules, linked by transfer connection blocks. The ARM refines anchors before the ODM predicts categories and more accurate box parameters.

  • RefineDet produces a fixed number of boxes and class-presence scores with a feed-forward convolutional network, followed by non-maximum suppression.
  • The ARM removes negative anchors and coarsely adjusts their locations and sizes before the ODM performs accurate regression and multi-class prediction.
  • Transfer connection blocks convert ARM features into ODM-compatible forms and add high-level context to improve detection accuracy.
  • The ARM yields refined anchor boxes and foreground confidence scores at each feature-map cell.
  • For each refined anchor, the ODM predicts c class scores and four accurate offsets for object localization and size estimation.

4. Training and Inference

RefineDet trains with augmented images and pretrained VGG-16 or ResNet-101 backbones. Its ARM assigns binary anchor labels and regressions, while the ODM uses filtered refined anchors for multi-class prediction and further localization, with losses defined for positive anchors.

  • Data Augmentation: RefineDet augments training images through random expansion, cropping, photometric distortion, and flipping.These strategies are used to adapt the model to object variations.
  • Backbone Network: VGG-16 and ResNet-101 pretrained on ILSVRC CLS-LOC serve as backbone networks for RefineDet.The architecture can also use other pretrained networks, including Inception V2, Inception ResNet, and ResNeXt-101.
  • Training Objective: The ARM assigns each anchor a binary objectness label and simultaneously regresses its location and size to produce refined anchors.The ARM predicts anchor confidence and refined coordinates before ODM processing.
  • Training Objective: The ODM receives refined anchors whose negative confidence is below the threshold and predicts object categories plus more accurate locations and sizes.ODM predictions include multi-class labels and bounding-box coordinates.
  • Training Objective: Regression losses are ignored for negative anchors, while ARM and ODM losses are normalized by their respective positive-anchor counts.The loss uses binary cross-entropy for ARM classification and multi-class classification loss for ODM classification.
  • Optimization: The network is fine-tuned with batch size 32 after initializing added convolutional or residual layers.VGG-16 extra convolution layers use Xavier initialization, while the ResNet-101 extra residual block uses a zero-mean Gaussian distribution with standard deviation 0.01.

5. Experiments

Experiments evaluate RefineDet on PASCAL VOC and MS COCO for accuracy, speed, and component effectiveness. Results show strong accuracy–efficiency trade-offs, while ablations support negative-anchor filtering, cascaded regression, and the transfer connection block.

  • PASCAL VOC 2007: 83.1% and 83.8% mAP are achieved by RefineDet320+ and RefineDet512+ with multi-scale testing on PASCAL VOC 2007.These results are reported as higher than the state-of-the-art methods in the comparison.
  • Run Time Performance: RefineDet processes 320 × 320 images at 40.3 FPS and 512 × 512 images at 24.1 FPS on an NVIDIA Titan X.The corresponding per-image processing times are 24.8 ms and 41.5 ms, respectively.
  • Ablation Study: Removing negative-anchor filtering reduces mAP from 80.0% to 79.5%, while replacing refined anchors with regular anchors reduces mAP from 79.5% to 77.3%.The ablations support the roles of filtering and two-step cascaded regression in the detector.
  • Ablation Study: The transfer connection block improves mAP by 1.1%, from 76.2% to 77.3%, in the reported ablation comparison.The authors attribute this improvement to inherited discriminative features and integrated large-scale context information.
  • PASCAL VOC 2007: RefineDet320 achieves 78.1% mAP on PASCAL VOC, while RefineDet512 reaches 80.1% mAP and surpasses all one-stage methods.The 320 × 320 model also exceeds most two-stage methods evaluated with approximately 1000 × 600 inputs.
  • MS COCO: 41.8% AP on MS COCO test-dev is reported for RefineDet512+ with ResNet-101, surpassing published one-stage and two-stage approaches.The best RefineDet result is reported as 0.2% higher than the second-best detector, G-RMI, despite G-RMI ensembling five Faster R-CNN models.
  • PASCAL VOC 2012: Training RefineDet on MS COCO before fine-tuning on PASCAL VOC produces up to 86.8% mAP after multi-scale testing on VOC 2012.The reported multi-scale results are 85.6%, 85.8%, 86.0%, and 86.8% across the RefineDet320 and RefineDet512 settings on VOC 2007 and VOC 2012.

6. Conclusions

RefineDet is a single-shot detector built from interconnected anchor refinement and object detection modules, trained end to end to achieve accurate and efficient detection.

  • RefineDet combines an anchor refinement module with an object detection module in a single-shot framework.The ARM filters negative anchors and coarsely adjusts them, while the ODM performs final regression and multi-class prediction.
  • The network is trained end to end using a multi-task loss on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO.The experiments support the paper’s reported state-of-the-art accuracy with high efficiency.
  • RefineDet is intended for future application to pedestrian, vehicle, and face detection, with attention mechanisms proposed for further improvement.

7. Complete Object Detection Results

RefineDet achieves the best reported results across the three evaluated detection benchmarks, while small-object detection remains a practical challenge.

  • Small-object detection remains challenging for RefineDet, particularly for chairs and tables.Increasing input size can improve small-object performance but burdens inference speed, making it only a temporary solution.
  • 85.8% mAP, 86.8% mAP, and 41.8% AP are reported on PASCAL VOC 2007, PASCAL VOC 2012, and MS COCO test-dev, respectively.These are identified as the best performances among published methods in the cited results.

8. Qualitative Results

Qualitative detections on VOC 2007, VOC 2012, and MS COCO show RefineDet handling several difficult visual conditions.

  • The qualitative results display detected boxes above a 0.6 score threshold, with colors indicating object categories.
  • RefineDet is reported to work well with occlusions, truncations, inter-class interference, and clustered backgrounds.

9. Detection Analysis on PASCAL VOC 2007

Detection analysis examines RefineDet’s error types, recall, and robustness across object characteristics, while benchmark figures document its evaluated results and settings.

  • Detection errors and recall: RefineDet’s confident detections are mostly correct, with recall around 95%-98% under the reported analysis.Recall is much higher under the weak 0.1 Jaccard-overlap criterion.
  • Detection errors and recall: RefineDet reduces false-positive errors compared with SSD across the analyzed object characteristics.
  • Object characteristics: RefineDet is reported as robust to different object sizes and aspect ratios through two-step cascade regression.The ARM diversifies anchor scales and aspect ratios before the ODM performs harder regressions.
  • Qualitative benchmark results: Figures 3-5 show RefineDet512 qualitative results of 85.2% mAP on VOC 2007, 85.0% mAP on VOC 2012, and 36.4% mAP on MS COCO.The figures use VGG-16 for VOC, ResNet-101 for COCO, and the listed benchmark-specific training data.
Loading 1711.06897v3…