Source-linked AI summary
Receptive Field Block Net for Accurate and Fast Object Detection
Songtao Liu, Di Huang, Yunhong Wang
TL;DR
Object detectors face a trade-off between the accuracy of deep backbones and the speed of lightweight models. The paper introduces RFB, which models receptive-field size and eccentricity to strengthen lightweight features, and adds it to SSD. RFB Net reaches advanced detector performance on Pascal VOC and MS COCO while retaining real-time speed.
Problem
Deep detectors provide powerful features at high computational cost, while lightweight real-time detectors often have lower accuracy.
Method
RFB uses multi-branch kernels and dilated convolutions to model receptive-field size and eccentricity, then replaces SSD's top convolution layers.
Results
RFB Net achieves state-of-the-art results on Pascal VOC and MS COCO at real-time speed, including 80.5% mAP at 83 fps for RFB Net300 on Pascal VOC.
Takeaways & Limitations
Strengthening lightweight feature representations with RFB can produce detectors that retain real-time processing while reaching accuracies comparable to deeper models.
Abstract
from arXiv · showhide
Current top-performing object detectors depend on deep CNN backbones, such as ResNet-101 and Inception, benefiting from their powerful feature representations but suffering from high computational costs. Conversely, some lightweight model based detectors fulfil real time processing, while their accuracies are often criticized. In this paper, we explore an alternative to build a fast and accurate detector by strengthening lightweight features using a hand-crafted mechanism. Inspired by the structure of Receptive Fields (RFs) in human visual systems, we propose a novel RF Block (RFB) module, which takes the relationship between the size and eccentricity of RFs into account, to enhance the feature discriminability and robustness. We further assemble RFB to the top of SSD, constructing the RFB Net detector. To evaluate its effectiveness, experiments are conducted on two major benchmarks and the results show that RFB Net is able to reach the performance of advanced very deep detectors while keeping the real-time speed. Code is available at https://github.com/ruinmessi/RFBNet.
1 Introduction
The paper targets the accuracy–speed trade-off in object detection by strengthening lightweight features instead of relying on deeper backbones. It proposes RFB, inspired by human receptive-field organization, and integrates it into SSD to build a fast detector.
- Deep CNN backbones improve detection features but impose high computational costs and reduce inference speed.
- Human pRF size increases with eccentricity, motivating feature mechanisms that emphasize central regions and reduce sensitivity to small spatial shifts.
- RFB combines multi-branch kernels, dilated convolutions, concatenation, and 1×1 convolution to model receptive-field size and eccentricity.
- RFB Net replaces SSD's top convolution layers with RFB, improving performance while controlling computational cost.
- RFB Net achieves state-of-the-art results on Pascal VOC and MS COCO while retaining real-time processing speed.
2 Related Work
Related work spans two-stage and single-stage detectors, receptive-field designs, and methods that trade speed for accuracy. RFB is positioned as a lightweight alternative that combines multi-scale kernels with eccentricity-aware dilation.
- Two-stage detector: Two-stage detectors generate category-agnostic proposals before classifying them with CNN features and have achieved dominant detection performance.
- One-stage detector: YOLO and SSD use lightweight backbones for acceleration, but their accuracies trail those of leading two-stage methods.
- One-stage detector: DSSD and RetinaNet improve single-stage accuracy with deeper backbones and added techniques, but these gains largely consume their speed advantage.
- Receptive field: Inception captures multi-scale information with different kernel sizes, whereas ASPP varies sampling distance through dilated convolution.
3 Method
The method models receptive-field size and eccentricity with a multi-branch RFB module, then integrates it into SSD to strengthen lightweight features while preserving the detector’s architecture and speed-oriented design.
- Receptive Field Block: RFB combines multi-branch convolutions with different kernels and trailing dilated pooling or convolution layers.Different kernels simulate receptive fields of multiple sizes, while trailing dilations reproduce the relation between receptive-field size and eccentricity.
- Receptive Field Block: RFB-s uses more branches with smaller kernels to mimic smaller pRFs in shallow human retinotopic maps.Two stacked 3 × 3 convolutions replace a 5 × 5 convolution to reduce parameters.
- Receptive Field Block: Each RFB branch pairs a particular kernel size with a corresponding dilation, after which branch feature maps are concatenated.Kernel size and dilation have a similar positive functional relation to pRF size and eccentricity in the visual cortex.
- RFB Net Detection Architecture: RFB Net embeds RFB modules into SSD while retaining SSD’s multi-scale, one-stage framework and lightweight backbone.The principal modification replaces SSD’s top convolution layers with RFB modules; the backbone remains the SSD VGG16 architecture.
- RFB Net Detection Architecture: In RFB-Net300, the conv4_3 feature map is followed by RFB-s, while a stride-2 RFB is formed from stride-2 multi-kernel convolutions.These modules are applied within the detector’s feature-map cascade, whose spatial resolutions decrease while receptive fields increase.
- Training and Testing: Training largely follows SSD, including augmentation, hard-negative mining, default-box settings, and localization and classification losses.The learning-rate schedule is slightly changed to accommodate RFB.
4 Experiments
Experiments evaluate RFB Net on Pascal VOC and MS COCO, including ablations, architectural comparisons, and speed–accuracy trade-offs. Across these evaluations, RFB improves detection accuracy while maintaining real-time inference speed.
- Pascal VOC 2007: RFB Net300 reaches 80.5% mAP on Pascal VOC 2007 while retaining SSD300’s real-time speed.RFB Net512 reaches 82.2% mAP while still running at high speed.
- Ablation Study: 1.9%: replacing SSD300’s last convolution layer with RFB-max pooling raises mAP from 77.2% to 79.1%.This ablation isolates the contribution of the RFB module under the updated SSD setup.
- Ablation Study: 0.5%: cortex-map parameter tuning improves RFB-max pooling from 79.1% to 79.6% mAP.RFB dilated convolution gains 0.4%, increasing from 80.1% to 80.5%.
- Ablation Study: Adding two default priors at conv4_3 further improves the RFB model by 0.2%, from 79.6% to 79.8%.The same change has no influence on the original SSD.
- Ablation Study: RFB performs best against Inception, ASPP, and Deformable CNN blocks on Pascal VOC and MS COCO under nearly matched parameter counts and training schedules.The comparison attributes the advantage to RFB’s dedicated structure and larger effective receptive field.
- Microsoft COCO: 30.3%/49.3%: RFB Net300 matches R-FCN’s reported result on COCO test-dev while using a lightweight setting.RFB Net512 reaches 33.8% versus RetinaNet500’s 34.4%, while RFB Net500 averages 30 ms per image versus RetinaNet’s 90 ms.
5 Discussion
RFB Net combines real-time speed with strong accuracy across Pascal VOC and MS COCO, while RFB generalizes to lightweight backbones and training without pretraining.
- Inference speed comparison: 80.5% mAP at 83 fps makes RFB Net300 the most accurate real-time detector in the Pascal VOC comparison.RFB Net512 reaches higher accuracy at 38 fps.
- Inference speed comparison: 66 fps RFB Net300 outperforms all high-frame-rate counterparts on the MS COCO speed/accuracy comparison.The comparison uses the MS COCO test-dev set and places RFB Net on the upper envelope of real-time detectors.
- Other lightweight backbone: RFB increases MobileNet-SSD accuracy with limited additional layers and parameters, suggesting potential for low-end devices.The experiment evaluates MobileNet as the backbone on MS COCO minival2014.
- Training from scratch: 77.6% mAP from scratch on VOC 2007 is comparable to DSOD, while the pretrained version reaches 80.5%.RFB Net300 is trained from scratch on VOC 07+12 trainval and evaluated on the VOC 2007 test set.
6 Conclusion
The paper enhances lightweight SSD features with RFB, achieving accuracy comparable to deeper detectors while retaining lightweight-model processing speed.
- 6 Conclusion: RFB models receptive-field size and eccentricity to generate more discriminative and robust features for lightweight CNN-based SSD.The resulting detector gains performance on Pascal VOC and MS COCO while retaining lightweight-model processing speed.