Source-linked AI summary
Small Object Detection using Context and Attention
Jeong-Seon Lim, Marcella Astrid, Hyun-Jin Yoon, Seung-Ik Lee
TL;DR
Detecting small objects remains difficult because they have low resolution and limited pixels. The paper augments SSD with multi-scale contextual feature fusion and attention, improving detection accuracy over conventional SSD, especially for small objects.
Problem
Small-object detection remains challenging because targets have low resolution and limited pixels, leaving substantial room for improvement over SSD.
Method
The method augments SSD by concatenating target features with higher-layer context features and applying attention to focus on objects while reducing background information.
Results
The proposed method improves object-detection accuracy over conventional SSD, with particularly significant enhancement for small-object detection.
Takeaways & Limitations
Context fusion and attention provide SSD with additional contextual information and object-focused features for detecting small objects.
Abstract
from arXiv · showhide
There are many limitations applying object detection algorithm on various environments. Especially detecting small objects is still challenging because they have low resolution and limited information. We propose an object detection method using context for improving accuracy of detecting small objects. The proposed method uses additional features from different layers as context by concatenating multi-scale features. We also propose object detection with attention mechanism which can focus on the object in image, and it can include contextual information from target layer. Experimental results shows that proposed method also has higher accuracy than conventional SSD on detecting small objects. Also, for 300$\times$300 input, we achieved 78.1% Mean Average Precision (mAP) on the PASCAL VOC2007 test set.
1. Introduction
Small-object detection remains difficult because objects have low resolution and limited pixels, leaving SSD substantially weaker on small targets. The paper proposes combining contextual features and attention to address this challenge.
- 20.7% mAP is reported for SSD on small-object targets, compared with 77.5% mAP overall.
- Low resolution and limited pixels make small objects difficult to recognize, even when viewed by humans without surrounding context.
- Context such as an object’s location in the sky can provide extra information for recognizing a small bird.
- The method concatenates small-object features with higher-layer contextual features and applies early-layer attention to focus on objects and reduce unnecessary background information.
- The experiments train and evaluate the proposed model on PASCAL VOC2007 and VOC2012, with comparisons against baseline and state-of-the-art methods on VOC2007.
2. Related Works
Related work has improved object detection through deeper architectures, feature enhancement, data augmentation, and visual attention. Prior small-object methods augment either the data or the feature information, while attention methods focus processing on relevant image regions.
- R-CNN uses CNNs on selective-search region proposals, but sequential proposal processing is too slow for real-time applications.
- Fast R-CNN performs feature extraction once for all region proposals, making it faster than R-CNN while retaining a separate processing stage.
- Small-object approaches include data augmentation, DSSD deconvolution for scaled-up feature maps, and GAN-based generation of high-resolution features.
- Visual attention methods focus on relevant parts of images or videos for tasks including image captioning and action recognition.
3. Method
The proposed detector extends SSD with feature fusion and attention modules. Feature fusion supplies contextual information from higher layers, while attention emphasizes important regions and suppresses irrelevant shallow background features.
- 3. Method: F-SSD, A-SSD, and FA-SSD respectively add feature fusion, attention, or both to the SSD baseline.
- 3.1. Single Shot Multibox Detector (SSD): SSD is a one-stage detector that processes different feature-map resolutions to detect objects at different scales.
- 3.1. Single Shot Multibox Detector (SSD): SSD uses higher-resolution early features for smaller objects and lower-resolution deeper features for larger objects.
- 3.1. Single Shot Multibox Detector (SSD): 20.7% on VOC 2007 is reported for SSD small-object performance, motivating added context and attention.
- 3.2. F-SSD: SSD with context by feature fusion: F-SSD deconvolves higher-layer context features to match the target feature’s spatial size before concatenating them.
- 3.3. A-SSD: SSD with attention module: The attention module uses trunk and mask branches, with two residual attention stages placed after conv4_3 and conv7 in A-SSD.
- 3.4. FA-SSD: SSD with feature fusion and attention module: FA-SSD combines feature fusion with an attention stage on the target feature, retaining the proposed fusion method.
4. Experiments
Experiments evaluate context fusion and attention added to SSD using VOC datasets, compare inference time, and test generalization across ResNet backbones. The combined FA-SSD model improves small-object detection, while comparisons with other methods highlight a speed–accuracy trade-off.
- Experimental setup: Experiments use SSD with a VGG16 backbone and 300 × 300 input, training on VOC2007 and VOC2012 trainval datasets.Testing uses the VOC2007 test dataset and COCO object-size categories.
- Ablation studies: FA-SSD combines feature fusion and attention, and achieves the strongest small-object detection improvement over the SSD baseline.F-SSD and A-SSD also outperform SSD, although FA-SSD does not have the best overall performance compared with F-SSD.
- Inference time: SSD is fastest in network forwarding but slowest in post-processing, making F-SSD and A-SSD faster overall.Inference time includes network inference and post-processing with Non-Maximum Suppression (NMS).
- Qualitative results: FA-SSD qualitatively succeeds on small-object detections where SSD fails.The comparison uses red ground-truth boxes and green predicted boxes.
- ResNet backbones: With ResNet18, ResNet34, and ResNet50 backbones, the proposed variants generally follow the VGG16 trends, except ResNet34 lacks the best small-object performance.Features from ResNet layer 2 are used to match the original SSD feature size.
- Comparison with other methods: Compared with DSSD, the approach has lower performance but runs at 30 FPS instead of 12 FPS.Both methods are compared using VOC2007 trainval and VOC2012 trainval training data.
5. Conclusion
The conclusion presents context-aware additions to SSD that fuse multi-scale features and apply attention to target layers. Experiments report improved detection accuracy over conventional SSD, especially for small objects.
- Conclusion: The method adds context-aware information to Single Shot Multibox Detector to improve small-object accuracy.The conclusion frames context-aware information as the central extension to SSD.
- Conclusion: Qualitative comparison shows SSD failing on small objects where FA-SSD succeeds.The comparison is presented between SSD and FA-SSD.
- Conclusion: Attention channels focus on either the object or its context, and conv4_3 attention captures smaller details than conv7 attention.The higher resolution of conv4_3 supports finer detail focus.
- Conclusion: Multi-scale feature fusion captures context from different layers, while attention incorporates contextual information at the target layer.These are the two context-integration mechanisms described in the conclusion.
- Conclusion: Experiments show higher object-detection accuracy than conventional SSD, with significant enhancement for small-object detection.The reported improvement is especially focused on small objects.
A. Detail inference time on ResNet backbones
The appendix provides detailed inference-time results for architectures using ResNet backbones.
- A. Detail inference time on ResNet backbones: Table 5 reports detailed inference times for ResNet backbone architectures.The passage identifies the table as a breakdown of inference time for ResNet-based architectures.
B. VOC2012 test results
On VOC2012 test data, FA-SSD does not improve over SSD and shows degradation for medium-sized objects.
- FA-SSD does not improve SSD on the VOC2012 test set.
- FA-SSD degrades performance on medium-sized objects compared with SSD.
C. Detail classes results on VOC2007
The VOC2007 results report mean average precision for every object class across the evaluated architectures.
- Table 7 reports mAP for each class across every architecture on VOC2007 test data.Classes without objects of a respective size have no reported result.