PaperlayerView exact revision on arXiv

Source-linked AI summary

SSD: Single Shot MultiBox Detector

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg

arXiv:1512.02325v5cs.CV

TL;DR

Accurate object detectors based on region proposals are often too computationally intensive for real-time use. SSD removes proposal generation and feature resampling by predicting boxes and categories in one network, achieving 59 FPS with 74.3% mAP on VOC2007 test versus Faster R-CNN’s 7 FPS and 73.2% mAP.

  • Problem

    Accurate object detection pipelines based on bounding-box proposals and feature resampling remain too computationally intensive for embedded systems and real-time applications.

  • Method

    SSD predicts category scores and bounding-box offsets for default boxes across multiple feature maps with different scales and aspect ratios in a single network.

  • Results

    59 FPS with 74.3% mAP on VOC2007 test, versus Faster R-CNN’s 7 FPS with 73.2% mAP.

  • Takeaways & Limitations

    SSD’s monolithic and relatively simple design provides a useful building block for larger systems that employ an object detection component.

  • Takeaways & Limitations

    SSD performs better on large than small objects, and improved default-box tiling for small-object accuracy remains future work.

Abstract

from arXiv · show

We present a method for detecting objects in images using a single deep neural network. Our approach, named SSD, discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios and scales per feature map location. At prediction time, the network generates scores for the presence of each object category in each default box and produces adjustments to the box to better match the object shape. Additionally, the network combines predictions from multiple feature maps with different resolutions to naturally handle objects of various sizes. Our SSD model is simple relative to methods that require object proposals because it completely eliminates proposal generation and subsequent pixel or feature resampling stage and encapsulates all computation in a single network. This makes SSD easy to train and straightforward to integrate into systems that require a detection component. Experimental results on the PASCAL VOC, MS COCO, and ILSVRC datasets confirm that SSD has comparable accuracy to methods that utilize an additional object proposal step and is much faster, while providing a unified framework for both training and inference. Compared to other single stage methods, SSD has much better accuracy, even with a smaller input image size. For $300\times 300$ input, SSD achieves 72.1% mAP on VOC2007 test at 58 FPS on a Nvidia Titan X and for $500\times 500$ input, SSD achieves 75.1% mAP, outperforming a comparable state of the art Faster R-CNN model. Code is available at https://github.com/weiliu89/caffe/tree/ssd .

1 Introduction

SSD eliminates bounding-box proposals and feature resampling to provide fast, accurate single-shot object detection. It predicts category scores and box offsets from multiscale feature maps, achieving strong accuracy across detection benchmarks and input resolutions.

  • Results: 77.2% mAP for 300×300 input and 79.8% mAP for 512×512 input were achieved on VOC2007 with improved data augmentation.The passage attributes these results to follow-on experiments and refers to Section 3.6 for details.
  • Motivation and contribution: 59 FPS with mAP 74.3% on VOC2007 test, versus Faster R-CNN at 7 FPS with mAP 73.2% and YOLO at 45 FPS with mAP 63.4%.SSD improves speed for high-accuracy detection by eliminating bounding-box proposals and subsequent pixel or feature resampling.
  • Motivation and contribution: SSD is a single-shot detector that is faster than YOLO and as accurate as slower methods using explicit region proposals and pooling.The approach avoids resampling pixels or features for bounding-box hypotheses.
  • Method: The model predicts category scores and box offsets for fixed default bounding boxes using small convolutional filters on feature maps.This fixed-box prediction design removes the need for proposal generation.
  • Method: Predictions from feature maps at different scales and aspect ratios improve detection accuracy across objects of varying sizes.The design also supports high accuracy on low-resolution input images, improving the speed–accuracy trade-off.

2 The Single Shot Detector (SSD)

SSD is a feed-forward, single-shot convolutional detector that predicts class scores and box offsets from multiple feature maps using tiled default boxes. Its end-to-end training matches default boxes to ground truth, optimizes localization and confidence losses, and limits negatives to a 3:1 ratio.

  • Detection framework: SSD produces fixed-size bounding boxes and class-presence scores with convolutional predictors, then applies non-maximum suppression for final detections.The framework uses a feed-forward convolutional network built on a truncated classification architecture.
  • Multi-scale feature maps: Multi-scale feature layers progressively decrease in size, enabling predictions at multiple object scales within one network.Unlike single-scale approaches, SSD uses distinct detection predictors for each feature layer.
  • Default boxes and predictors: Each feature-map cell receives default boxes with varied scales and aspect ratios, while convolutional filters predict category scores and shape offsets for them.The boxes tile feature maps convolutionally with fixed positions relative to their corresponding cells.
  • Training methodology: Training matches each ground-truth box to its best-overlap default box and to boxes exceeding 0.5 Jaccard overlap, enabling end-to-end loss computation and backpropagation.This assignment is required because SSD predicts from a fixed set of detector outputs.
  • Training methodology: The objective combines localization and confidence losses; localization uses Smooth L1 regression for box center, width, and height offsets, while confidence uses multi-class softmax.If no default boxes match, the loss is set to 0.
  • Training methodology: 3:1 is the maximum negative-to-positive training ratio after selecting negatives with the highest confidence loss.Hard-negative selection reduces class imbalance and improves optimization speed and stability.

3 Experimental Results · Base network · 3.1 PASCAL VOC2007

The experiments evaluate SSD using a modified, fine-tuned VGG16 base network and compare VOC2007 detection performance across input sizes, training data, and competing proposal-based methods. Analyses show strong detection quality and localization, especially for large objects, while small-object detection remains a limitation that larger inputs partly address.

  • Base network: SSD experiments use a VGG16 network pretrained on ILSVRC CLS-LOC and modified by converting fc6/fc7 to convolutional layers, removing dropout and fc8, and applying atrous convolution.The resulting network is fine-tuned with SGD using a 10^-3 initial learning rate, 0.9 momentum, and 0.0005 weight decay.
  • 3.1 PASCAL VOC2007: The VOC2007 comparison evaluates Fast R-CNN, Faster R-CNN, and SSD on the 4,952-image test set, with all methods fine-tuned from the same pretrained VGG16 network.Fast R-CNN and Faster R-CNN use input images with minimum dimension 600, while SSD is evaluated at 300×300 and 512×512.
  • 3.1 PASCAL VOC2007: Training on VOC2007 uses a staged learning-rate schedule of 10^-3 for 40k iterations, followed by 10^-4 and 10^-5 for 10k iterations each.For SSD512, the model adds conv12_2 for prediction and adjusts the minimum scale settings.
  • 3.1 PASCAL VOC2007: SSD’s detection analysis finds high-quality detections across object categories, with most confident detections correct and recall around 85–90%.Recall is much higher under the weak 0.1 Jaccard-overlap criterion than under stricter criteria.
  • 3.1 PASCAL VOC2007: Compared with R-CNN, SSD exhibits less localization error, indicating stronger object localization from directly learning to regress object boundaries.The analysis attributes this advantage to SSD’s direct regression approach.
  • 3.1 PASCAL VOC2007: Table 1 reports that increasing SSD’s input size improves results and that adding more training data further helps, while the two SSD models otherwise use the same settings.The compared training sets are VOC2007 trainval, VOC2007+VOC2012 trainval, and COCO trainval35k followed by fine-tuning on VOC2007+VOC2012.
  • 3.1 PASCAL VOC2007: Larger SSD input sizes improve detection of small objects, although substantial room for improvement remains, while SSD performs particularly well on large objects.The model is also robust to object aspect ratios because it uses default boxes with varied aspect ratios at each feature-map location.

3.2 Model analysis

Controlled experiments at 300 × 300 show that SSD benefits substantially from extensive data augmentation, diverse default-box shapes, atrous computation, and multiple output layers at different resolutions.

  • Data augmentation: 8.8% mAP improvement comes from SSD’s more extensive sampling strategy compared with training on only original images and horizontal flips.The strategy is similar to YOLO’s and is described as crucial for performance.
  • Default box shapes: Removing default boxes with 1/3 and 3 aspect ratios drops performance by 0.6%, while removing boxes with 1/2 and 2 aspect ratios causes another 2.1% drop.The results indicate that varied default-box shapes make box prediction easier for the network.
  • Backbone computation: 20% slower speed results from using the full VGG16 instead of the atrous subsampled VGG16, despite obtaining about the same result.The full-VGG16 variant keeps pool5 with 2 × 2 −s2, retains fc6 and fc7 parameters, and adds conv5 3 for prediction.
  • Multiple output layers: Multiple output layers at different resolutions improve SSD performance when default boxes use different scales across layers.The comparison progressively removes layers while adjusting default-box tiling to keep the total near the original 8732 boxes.

3.3 PASCAL VOC2012

On PASCAL VOC2012, SSD uses expanded VOC training data and follows the VOC2007 performance trend, with SSD300 improving accuracy over Fast/Faster R-CNN.

  • Experimental setup: SSD trains on VOC2012 trainval plus VOC2007 trainval and test, totaling 21,503 images, and evaluates on the 10,991-image VOC2012 test set.Training uses a 10^-3 learning rate for 60k iterations, followed by 10^-4 for 20k iterations.
  • Results: The VOC2012 results show the same performance trend observed on the VOC2007 test set.The passage reports results for both SSD300 and SSD512.
  • Results: SSD300 improves accuracy over Fast/Faster R-CNN on VOC2012.The comparison table evaluates detectors under differing image-size settings, including 600-pixel minimum dimension for Fast and Faster R-CNN and 448 × 448 for YOLO.

3.4 COCO

On COCO, SSD uses smaller default boxes to accommodate smaller objects and evaluates SSD300 and SSD512 on test-dev2015. SSD300 outperforms Fast R-CNN on two mAP criteria, while SSD512 surpasses Faster R-CNN on both reported criteria.

  • Model configuration: COCO training uses smaller default boxes, with the smallest scale reduced to 0.15 and conv4_3 set to 0.07 for SSD300.The conv4_3 box corresponds to 21 pixels for a 300 × 300 image.
  • Detection results: SSD300 is better than Fast R-CNN in both mAP@0.5 and mAP@[0.5:0.95] on COCO test-dev2015.SSD300 has similar mAP@0.75 to ION and Faster R-CNN but is worse in mAP@0.5.
  • Detection results: SSD512 is better than Faster R-CNN in both reported evaluation criteria when the input image size increases to 512 × 512.For SSD512, the configuration adds conv12_2 for prediction and uses smin values of 0.1 and 0.04 on conv4_3.
  • Object-size analysis: Compared to ION, SSD improves AR for large and small objects by 5.4% and 3.9%, respectively.The passage states that the improvement is more similar across large and small objects than in the comparison involving Faster R-CNN.
  • Object-size analysis: The authors conjecture that Faster R-CNN is more competitive on smaller objects because it performs two box-refinement steps.These steps occur in the RPN and Fast R-CNN parts, and the paper shows SSD512 detection examples on COCO test-dev.

3.5 Preliminary ILSVRC results

SSD applied the COCO network architecture to ILSVRC DET and achieved 43.4 mAP on the val2 set, supporting its generality for high-quality real-time detection.

  • 3.5 Preliminary ILSVRC results: 43.4 mAP was achieved on the ILSVRC DET val2 set.The model was trained on ILSVRC2014 DET train and val1.
  • 3.5 Preliminary ILSVRC results: The ILSVRC model used the same network architecture as the COCO model.This applied SSD300 to the ILSVRC DET dataset.
  • 3.5 Preliminary ILSVRC results: The training schedule used 320k iterations at 10^-3, followed by 80k at 10^-4 and 40k at 10^-5.These stages progressively reduced the learning rate during training.

3.6 Data Augmentation for Small Object Accuracy

SSD’s data augmentation strategy substantially improves small-object detection, particularly on small datasets, by generating larger training examples through random crops and image expansion. Better alignment of default boxes with feature-map receptive fields is identified as future work.

  • Augmentation benefits: Data augmentation dramatically improves SSD performance on small objects, especially on small datasets such as PASCAL VOC.Without feature resampling as in Faster R-CNN, SSD’s small-object classification is relatively difficult.
  • Augmentation mechanism: Random crops act as a “zoom in” operation that generates many larger training examples.The strategy also includes a “zoom out” operation through image expansion.
  • Future work: A better tiling of default boxes aligned with feature-map receptive-field position and scale is left for future work.This is proposed as an alternative way to improve SSD.
  • Augmentation benefits: The new data augmentation trick significantly improves detection of small objects on the VOC2007 test set.Figure 6 compares original SSD300 and SSD512 models with SSD300* and SSD512* models trained using the new augmentation.

3.7 Inference time

SSD makes inference-time non-maximum suppression efficient by filtering low-confidence boxes, applying per-class suppression, and retaining only the top detections, with a 1.7 msec per-image cost for SSD300 on 20 VOC classes.

  • Inference time: A 0.01 confidence threshold filters out most generated boxes before non-maximum suppression.This preprocessing is needed because the method generates a large number of boxes.
  • Inference time: 0.45 Jaccard overlap is used for per-class non-maximum suppression, retaining the top 200 detections per image.Suppression is applied separately for each object class.
  • Inference time: 1.7 msec per image is spent on suppression for SSD300 with 20 VOC classes, close to the 2.4 msec spent on all newly added layers.The reported timings are measured during inference.

4 Related Work

Object detection methods historically divide into sliding-window and region-proposal approaches, while SSD belongs to direct prediction methods that eliminate proposals. SSD extends this family with multi-scale feature maps and varied default-box aspect ratios, connecting it to OverFeat and YOLO while offering greater flexibility.

  • Object detection methods comprise two established classes: sliding-window methods and region-proposal classification methods.
  • R-CNN combines selective-search region proposals with convolutional-network-based post-classification, motivating subsequent improvements to detection pipelines.
  • SPPnet and Fast R-CNN reduce or streamline R-CNN’s expensive crop-classification process by reusing feature maps and enabling end-to-end fine-tuning.
  • Deep proposal-generation methods such as MultiBox improve accuracy but require two dependent neural networks, producing a more complex setup.
  • Direct-prediction methods including OverFeat, YOLO, and SSD skip proposals and predict bounding boxes and confidences for multiple categories.
  • SSD is more flexible than these direct methods because it uses default boxes with different aspect ratios across multiple feature maps and scales.

5 Conclusions

The paper introduces SSD as a fast, single-shot multi-category object detector using multi-scale convolutional bounding-box outputs across multiple feature maps. It validates that appropriate training and more carefully chosen default boxes improve performance, and presents SSD as a simple building block for larger detection systems.

  • Contributions: SSD is a fast, single-shot object detector for multiple categories that uses multi-scale convolutional bounding-box outputs attached to multiple feature maps.This representation efficiently models the space of possible box shapes.
  • Contributions: Appropriate training strategies and a larger number of carefully chosen default bounding boxes result in improved performance.
  • Applications and future work: SSD’s monolithic and relatively simple design provides a useful building block for larger systems that employ an object detection component.
  • Applications and future work: A promising future direction is integrating SSD with recurrent neural networks to detect and track objects in video simultaneously.
Loading 1512.02325v5…