Source-linked AI summary
Accurate Single Stage Detector Using Recurrent Rolling Convolution
Jimmy Ren, Xiaohao Chen, Jianbo Liu, Wenxiu Sun, Jiahao Pang, Qiong Yan, Yu-Wing Tai, Li Xu
TL;DR
Single-stage detectors offer simpler training and efficient deployment but have been less competitive at high-IoU localization, where high-quality boxes are difficult to generate. The paper introduces an end-to-end RRC architecture that recurrently aggregates contextual information over multi-scale features. On KITTI, RRC significantly outperformed previously published single-model results and achieved state-of-the-art results across all three detection benchmarks.
Problem
Single-stage detectors are efficient but generally underperform two-stage methods at high IoU thresholds because small and overlapping objects require contextual information for reliable box regression.
Method
RRC recurrently aggregates relevant contextual features over multi-scale feature maps and supervises recurrent steps within an end-to-end single-stage detector.
Results
RRC significantly outperformed previously published single-model results on KITTI and achieved state-of-the-art results across all three KITTI detection benchmarks.
Takeaways & Limitations
A single-stage detector can produce highly accurate high-localization-quality detections when contextual refinement is recurrently introduced into bounding-box regression.
Takeaways & Limitations
The authors leave efficient memory mechanisms for recurrent detection and extensions to 3D object detection for future work.
Abstract
from arXiv · showhide
Most of the recent successful methods in accurate object detection and localization used some variants of R-CNN style two stage Convolutional Neural Networks (CNN) where plausible regions were proposed in the first stage then followed by a second stage for decision refinement. Despite the simplicity of training and the efficiency in deployment, the single stage detection methods have not been as competitive when evaluated in benchmarks consider mAP for high IoU thresholds. In this paper, we proposed a novel single stage end-to-end trainable object detection network to overcome this limitation. We achieved this by introducing Recurrent Rolling Convolution (RRC) architecture over multi-scale feature maps to construct object classifiers and bounding box regressors which are "deep in context". We evaluated our method in the challenging KITTI dataset which measures methods under IoU threshold of 0.7. We showed that with RRC, a single reduced VGG-16 based model already significantly outperformed all the previously published results. At the time this paper was written our models ranked the first in KITTI car detection (the hard level), the first in cyclist detection and the second in pedestrian detection. These results were not reached by the previous single stage methods. The code is publicly available.
1. Introduction
High-IoU localization is important for real-world safety and service quality, yet single-stage detectors often lag behind two-stage methods because they generate lower-quality boxes. The paper addresses this with an end-to-end RRC-based refinement process that progressively incorporates context.
- High-IoU bounding-box localization is crucial for service quality in robotic manipulation and ADAS safety.
- Single-stage detectors are easier to train and more computationally efficient, but usually underperform two-stage methods at high IoU thresholds.
- Small and occluded objects produce many low-quality boxes because conventional regression lacks sufficient contextual information.
- Recurrent Rolling Convolution progressively and selectively introduces contextual information into bounding-box regression within a fully data-driven, end-to-end trainable single-stage network.
- Using a reduced VGG-16 backbone, the approach significantly outperformed previously published single-model results on KITTI, while an ensemble ranked among the benchmark leaders.
2. Related Work
Related work contrasts accurate but computationally heavy region-proposal methods with faster single-stage detectors that lose accuracy as IoU requirements rise. The paper positions RRC as a context-aware single-stage alternative evaluated at higher IoU thresholds.
- R-CNN-style detectors use region proposals followed by a second CNN stage, achieving strong high-IoU benchmark performance but incurring heavy second-stage computation.
- SSD directly detects objects from multi-resolution feature maps, reducing computation and running faster than two-stage alternatives.
- SSD performs well at IoU 0.5 but its performance drops significantly when bounding-box quality requirements increase.
- YOLO is another fast single-stage detector, but the cited comparison describes it as less accurate than SSD.
- RRC differs from earlier sequence-modeling detection work by efficiently detecting every object with context-rich features and achieving state-of-the-art performance at higher IoU thresholds.
3. Analysis and Our Approach
The paper analyzes why SSD struggles at high IoU thresholds and proposes recurrent, context-aware aggregation across multi-scale feature maps. RRC iteratively refines features and detection outputs while sharing aggregation and prediction weights across steps.
- 3.1. The Missing Piece of The Current Methods: SSD’s multi-scale design uses different-resolution feature maps to detect objects across size ranges, but it does not outperform two-stage methods at high IoU thresholds.Each layer is largely responsible for its own scale, requiring individual feature maps to provide sufficient detail and abstraction for accurate localization.
- 3.1. The Missing Piece of The Current Methods: RRC addresses this limitation by making each output feature representation incorporate information from contributing feature maps while preserving its original resolution.The transformed representations also include features extracted in deeper layers, maintaining a single-stage detection process.
- 3.2. Recurrent Rolling Convolution: RRC recurrently aggregates direct higher- and lower-level feature counterparts so contextual information can be learned rather than manually specified.The aggregation is motivated by the need for fine details for small objects, robust abstraction for occluded objects, and boundary details alongside high-level discrimination.
- 3.2. Recurrent Rolling Convolution: Each iteration combines a feature map with its neighboring scales through a trainable function, and shared weights across steps make the process recurrent.Separate supervision at each step encourages successive detection refinement and helps the system learn useful contextual information.
- 3.2. Recurrent Rolling Convolution: RRC gathers relevant contextual features at every iteration, with separate losses guiding the learning of each recurrent refinement step.The architecture is designed to make consistent progress while remaining an end-to-end single-stage detector.
4. Experiments
Experiments evaluated RRC through recurrent-output analysis, validation comparisons, and official KITTI testing across cars, pedestrians, and cyclists. RRC improved predictions over several iterations and outperformed the original SSD and previously published methods, while later iterations eventually degraded.
- Experimental design: The experiments covered recurrent-output quality, validation performance, and official KITTI testing across car, pedestrian, and cyclist detection.The official test evaluation used the KITTI car, pedestrian, and cyclist benchmarks.
- Experimental design: Five RRC iterations produced six consecutive predictions, with separate regressors assigned to the corresponding feature maps.The model used 1x1 convolutions for RRC, making the resulting model efficient.
- Recurrent-output analysis: Validation loss dropped substantially after the first RRC, reached its minimum at the third or fourth output, then stopped decreasing.Validation loss was generally higher than training loss, indicating some overfitting.
- Recurrent-output analysis: RRC consistently improved predictions for several recurrent iterations, but improvement eventually diverged; final detections used NMS over the third, fourth, and fifth predictions.The authors attribute eventual degeneration mainly to the lack of an effective memory mechanism and leave efficient memory design for future work.
- Validation evaluation: Both RRC variants performed much better than the original SSD on KITTI validation across IoU thresholds.Because both models used the same reduced VGG-16 backbone, the authors attribute the improvement to the added RRC architecture.
- Validation evaluation: At IoU 0.8, RRC outperformed the original SSD by more than 6%, confirming improved high-quality bounding-box prediction.RRC also performed consistently better than the RRC* variant.
- Test-set evaluation: On the official KITTI test set, a single RRC model significantly outperformed previously published car-detection results, while an ensemble performed better still.The evaluation compared car performance with five recently published state-of-the-art methods and used the same input resolution as a comparison method for fairness.
- Test-set evaluation: RRC ranked first for KITTI hard-category car detection, first for cyclist detection, and second for pedestrian detection among submitted methods.The paper presents these results as evidence of RRC’s effectiveness across the evaluated benchmarks.
5. Concluding Remarks
The paper concludes that Recurrent Rolling Convolution improves single-stage detection by aggregating contextual information, while identifying memory-enabled recurrence and 3D detection as future directions.
- RRC gradually and consistently aggregates relevant contextual information among feature maps to generate accurate detection results.
- RRC achieved state-of-the-art results across all three KITTI detection benchmarks and was the first single-stage detector to obtain such results.
- Future work will investigate memory-enabled recurrence in object detection and quantify its impact on detection performance.
- The authors also plan to generalize RRC to 3D object detection and related applications.