PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancement of SSD by concatenating feature maps for object detection

Jisoo Jeong, Hyojin Park, Nojun Kwak

arXiv:1705.09587v1cs.CV

TL;DR

Conventional SSD can duplicate detections across scales and has difficulty detecting small objects. The paper connects feature-pyramid maps near the classifier, increases their channels efficiently, and shares classifier weights; R-SSD reaches 78.5% mAP at 35.0 FPS for 300 × 300 input and 80.8% mAP at 16.6 FPS for 512 × 512 input.

  • Problem

    Conventional SSD processes feature-pyramid layers independently, which limits cross-scale relationships and small-object detection.

  • Method

    The method concatenates feature maps across SSD’s feature pyramid through pooling and deconvolution, increasing channels near the classifier and enabling a shared classifier.

  • Results

    R-SSD achieves 78.5% mAP at 35.0 FPS with 300 × 300 input and 80.8% mAP at 16.6 FPS with 512 × 512 input.

  • Takeaways & Limitations

    The proposed architecture improves SSD accuracy while retaining real-time detection speed, with shared classification also supporting generalization and imbalanced or small datasets.

  • Takeaways & Limitations

    Pooling- or deconvolution-based alternatives are inferior to conventional SSD in accuracy and speed because their shared feature maps can be affected by losses from other scales.

Abstract

from arXiv · show

We propose an object detection method that improves the accuracy of the conventional SSD (Single Shot Multibox Detector), which is one of the top object detection algorithms in both aspects of accuracy and speed. The performance of a deep network is known to be improved as the number of feature maps increases. However, it is difficult to improve the performance by simply raising the number of feature maps. In this paper, we propose and analyze how to use feature maps effectively to improve the performance of the conventional SSD. The enhanced performance was obtained by changing the structure close to the classifier network, rather than growing layers close to the input data, e.g., by replacing VGGNet with ResNet. The proposed network is suitable for sharing the weights in the classifier networks, by which property, the training can be faster with better generalization power. For the Pascal VOC 2007 test set trained with VOC 2007 and VOC 2012 training sets, the proposed network with the input size of 300 x 300 achieved 78.5% mAP (mean average precision) at the speed of 35.0 FPS (frame per second), while the network with a 512 x 512 sized input achieved 80.8% mAP at 16.6 FPS using Nvidia Titan X GPU. The proposed network shows state-of-the-art mAP, which is better than those of the conventional SSD, YOLO, Faster-RCNN and RFCN. Also, it is faster than Faster-RCNN and RFCN.

1 Introduction

Conventional SSD is fast and robust to scale variation, but its independent scale-specific classifiers can produce duplicate detections and struggle with small objects. The proposed approach connects feature-pyramid layers, efficiently increases their feature maps, and shares classifier weights to improve detection.

  • SSD uses multiple convolution layers for detection, providing relatively fast and robust performance across object scales.
  • Independent processing of feature-pyramid layers prevents SSD from modeling relationships between scales, allowing multiple boxes for one object.
  • Small objects remain difficult to detect in SSD, including persons on a boat and small cows in example images.
  • The proposed method connects feature-pyramid layers, increases feature maps only there, and supports one shared classifier across scales.Shared classification can use training examples from different scales, which is useful for imbalanced or small datasets.

2 Related Works

Prior object detectors trade off accuracy, speed, end-to-end learning, and scale handling in different ways. SSD performs detection directly across feature-pyramid scales, while related work augments networks or features to improve accuracy, especially for small objects.

  • R-CNN improved accuracy but relied on separate region proposals, limiting speed and preventing end-to-end learning.
  • Faster R-CNN introduced a region proposal network that substantially improved speed and enabled end-to-end learning.
  • YOLO prioritized speed through grid-based localization and classification, while YOLO2 used anchor boxes to improve both speed and accuracy.
  • SSD generates bounding-box candidates at each position and scale, while DSSD adds large-scale context and ResNet-based features to improve small-object accuracy at the expense of speed.
  • Feature augmentation in segmentation commonly combines features through concatenation, element-wise addition, or element-wise product with bypass connections.

3 Method

The method enriches SSD’s classifier inputs by concatenating feature maps across the feature pyramid at a common size. Rainbow concatenation combines both directions, increases representation for small objects, and enables classifier weight sharing.

  • 3 Method: The method improves SSD by using relationships among feature-pyramid layers while increasing channels in the feature pyramid instead of changing the base network.
  • 3.1 Concatenation through pooling or deconvolution: Pooling concatenates lower-layer maps into upper-layer features, while deconvolution or upsampling concatenates upper-layer maps into lower-layer features.These operations provide classifier inputs with information from other layers and different receptive fields.
  • 3.1 Concatenation through pooling or deconvolution: Batch normalization is applied per filter before concatenation because feature values from different layers have different scales.
  • 3.1 Concatenation through pooling or deconvolution: Unidirectional pooling or deconvolution lets detection use other-layer information but prevents classifiers from using information in the opposite direction.
  • 3.2 Rainbow concatenation: Rainbow concatenation pools and deconvolves every layer to a common size before concatenation, allowing detection to use information about both smaller and larger objects.This design is intended to make objects more likely to be detected at the appropriate feature-pyramid layer.
  • 3.2 Rainbow concatenation: Each rainbow-concatenated layer contains 2,816 feature maps, enabling shared classifier weights across scales with unified channel counts.The total number of boxes is 7,760 with four default boxes per layer and 11,640 with six, compared with 8,732 for conventional SSD.

4 Experiments

Experiments evaluate SSD variants on PASCAL VOC 2007, including feature-augmentation methods and ISSD. R-SSD achieves higher mAP than conventional SSD, while increased complexity reduces speed.

  • Evaluation setup: The evaluation uses PASCAL VOC 2007, whose dataset contains 20 object classes with annotated locations and class information.Models were trained using VOC2007 and VOC2012 trainval datasets.
  • ISSD: 78.1% mAP improves ISSD accuracy by 0.4% over conventional SSD, but speed drops to 26.9 FPS.ISSD increases channels by 2 to 8 times across different layers.
  • Feature concatenation: 77.1% mAP and 77.3% mAP from pooling and deconvolution concatenation are 0.6% and 0.4% below conventional SSD.Their speeds also decrease to 48.3 FPS and 39.9 FPS, respectively.
  • R-SSD: 78.5% mAP at 35.0 FPS is obtained by R-SSD with 300×300 input, improving accuracy by 0.8% over conventional SSD.The speed reduction is attributed to increased computational complexity.
  • R-SSD: 80.8% mAP at 16.6 FPS is obtained by R-SSD with 512×512 input, improving accuracy by 1% over conventional SSD.Compared with SSD 512, precision increases 2.9% at recall 0.8 and 8.2% at recall 0.9.

5 Discussion

The discussion evaluates R-SSD using high-recall precision, scale-specific recall, accuracy–speed trade-offs, and classifier-sharing effects. R-SSD improves high-recall performance and small-object detection, while classifier sharing offers training benefits with modest accuracy differences.

  • 5.1 New evaluation method: At recall 0.9, R-SSD512 outperformed SSD512 by more than 8% in average precision.The paper reports that mAP differences are smaller than high-recall precision differences because low-recall precisions are similar.
  • 5.2 Concatenation by pooling or deconvolution: Models using explicit layer relationships and increased channels can be slower and less accurate than conventional SSD because of added operations and scale-sharing effects.The paper conjectures that shared feature maps may be affected by losses from other scales and fail to focus properly on their own scale.
  • 5.3 Single classifier vs. Multiple classifiers: Sharing one classifier produces little speed difference but lowers performance by 1.2% and 0.7% versus conventional SSD in the reported experiments.The shared classifier can improve learning under scale-imbalanced training samples and is faster during early training.
  • 5.4 Accuracy vs. Speed: 80.8% mAP at 15 FPS or higher demonstrates R-SSD’s accuracy–speed balance, while it is about three times faster than R-FCN at similar accuracy.The comparison also reports roughly 1% mAP improvement over conventional SSD.
  • 5.5 Performances for different scales: R-SSD improves small-object recall over corresponding SSD300 and SSD512 models, while large-object recall remains above 0.93 for all models.For large objects, the difference among R-SSD300, SSD512, and R-SSD512 is less than 10 objects out of 7,641.

6 Conclusion

The paper concludes that rainbow concatenation addresses conventional SSD’s duplicate detections and small-object weaknesses while preserving an accuracy–speed focus. It also supports shared classifiers across scales, especially for imbalanced or small datasets.

  • 6 Conclusion: Rainbow concatenation links feature-pyramid scales to prevent unnecessary multiple detections for one object.The method creates relationships between scales and uses increased feature maps in each feature-pyramid layer.
  • 6 Conclusion: Efficiently increasing feature maps improves accuracy without much time overhead.The conclusion distinguishes this strategy from simply expanding the network near the input.
  • 6 Conclusion: Matching feature-map counts across layers enables one classifier for different scales, with expected generalization benefits for imbalanced or small datasets.The conclusion states that R-SSD achieves state-of-the-art mAP among methods running above 15 FPS.
  • 6 Conclusion: R-SSD is designed to consider accuracy and speed simultaneously and achieves state-of-the-art mAP among methods faster than 15 FPS.This is the paper’s final scope-limited performance claim.

Additional Experiments

Additional experiments examine performance on small and standard training sets, VOC2012, and MS-COCO. R-SSD improves reported detection accuracy, small-object detection, or both relative to relevant SSD baselines.

  • 7.1 Small Train Dataset in VOC2007: 66.9% mAP gives R-SSD a 0.8% gain over conventional SSD when trained on the 2,501-image VOC2007 training set.R-SSD with one classifier reaches 67.2% mAP in the same experiment.
  • 7.1 Small Train Dataset in VOC2007: 67.2% mAP is achieved by R-SSD with one classifier on the small VOC2007 training set.The paper reports improved generalization power, faster training, and better detection performance for the single-classifier variant.
  • 7.2 VOC2012: 76.6% mAP makes R-SSD higher than SSD300 and DSSD321 on VOC2012, and R-SSD is faster than DSSD321.The reported comparison uses networks trained on the VOC07++12 train dataset.
  • MS-COCO: On MS-COCO examples, R-SSD prevents multiple boxes for one object and increases detection of small objects.These observations are illustrated through resultant images from the dataset.
  • MS-COCO: Figure 5 contrasts conventional SSD and R-SSD detections for ordinary and small objects, including the number of boxes assigned to one object.Boxes with objectness scores of 0.3 or higher are shown.
Loading 1705.09587v1…