Source-linked AI summary
RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles
Ramin Nabati, Hairong Qi
TL;DR
Region proposal is a computational bottleneck in two-stage detection, limiting real-time autonomous-vehicle perception. RRPN uses Radar detections mapped into image coordinates to generate distance-adjusted proposals, and it is more than 100x faster than Selective Search while achieving better detection precision and recall.
Problem
Region proposal algorithms increase per-image processing time in two-stage detectors, making them unsuitable for real-time autonomous-driving applications.
Method
RRPN maps Radar detections to image coordinates and generates pre-defined anchor boxes whose scale is adjusted using object distance.
Results
More than 100x faster than Selective Search, RRPN achieves better detection average precision and recall on NuScenes with Fast R-CNN.
Takeaways & Limitations
RRPN provides a real-time region-proposal approach that also fuses Radar and camera data for faster and more accurate detections.
Abstract
from arXiv · showhide
Region proposal algorithms play an important role in most state-of-the-art two-stage object detection networks by hypothesizing object locations in the image. Nonetheless, region proposal algorithms are known to be the bottleneck in most two-stage object detection networks, increasing the processing time for each image and resulting in slow networks not suitable for real-time applications such as autonomous driving vehicles. In this paper we introduce RRPN, a Radar-based real-time region proposal algorithm for object detection in autonomous driving vehicles. RRPN generates object proposals by mapping Radar detections to the image coordinate system and generating pre-defined anchor boxes for each mapped Radar detection point. These anchor boxes are then transformed and scaled based on the object's distance from the vehicle, to provide more accurate proposals for the detected objects. We evaluate our method on the newly released NuScenes dataset [1] using the Fast R-CNN object detection network [2]. Compared to the Selective Search object proposal algorithm [3], our model operates more than 100x faster while at the same time achieves higher detection precision and recall. Code has been made publicly available at https://github.com/mrnabati/RRPN .
1. INTRODUCTION
Autonomous-vehicle perception requires real-time object detection despite the accuracy benefits and computational challenges of combining cameras, Radars, and other sensors. RRPN addresses the costly region-proposal bottleneck by using Radar detections and achieves faster, more accurate proposals than Selective Search.
- Real-time object detection is challenging because autonomous vehicles combine multiple sensors that can improve accuracy and reliability while complicating perception-system design.
- Radars provide accurate range and range-rate information, whereas cameras are effective for object classification, motivating Radar-camera fusion.
- Two-stage detectors use a Region Proposal Network to hypothesize object locations, making region proposals a central component of the detection pipeline.
- RRPN proposes regions from Radar detections, bypassing the computationally expensive vision-based proposal step while improving detection accuracy.
- More than 100× faster, RRPN outperforms Selective Search in Fast R-CNN while achieving higher detection precision and recall.
2. RELATED WORK
Prior work explored Radar-based detection and sensor fusion, as well as vision-based object proposals. Vision-based proposals can be accurate but remain too slow for real-time autonomous-driving systems.
- Prior Radar research addressed navigation, object detection and tracking, and attention selection for candidate targets.
- Earlier sensor-fusion work combined LIDAR or other sensor features with vision using centralized or decentralized architectures.
- Selective Search diversifies image partitionings to find objects but operates at 2-7 seconds per image, limiting its real-time suitability.
- Edge Boxes reduces proposal time to 0.25 seconds per image relative to Selective Search, yet remains considered very slow for real-time applications.
3. RADAR REGION PROPOSAL NETWORK
RRPN generates region proposals from Radar detections by mapping them into camera coordinates, creating varied and translated anchors, and scaling them using object distance. Its three-stage pipeline combines perspective transformation, anchor generation, and distance compensation to improve proposal alignment.
- Overview: RRPN uses Radar detections to generate object proposals for two-stage detection networks such as Fast R-CNN.Using Radar-only proposals bypasses the computationally expensive vision-based region proposal step and also fuses Radar and camera data.
- Overview: The pipeline consists of perspective transformation, anchor generation, and distance compensation.
- 3.1. Perspective Transformation: Perspective transformation maps Radar detections from vehicle coordinates into camera-view coordinates, associating Radar-detected objects with image objects.The projective relation uses a calibration-derived matrix H.
- 3.2. Anchor Generation: For each mapped Point of Interest, RRPN generates anchors with multiple sizes, aspect ratios, and translations to handle unknown object scale and off-center detections.A fixed-size box is ineffective because Radar does not provide object size; translated anchors address detections mapped away from object centers.
- 3.3. Distance Compensation: Distance compensation scales generated anchors according to each object's range from the vehicle.The scale factor uses distance d_i and parameters α and β, which are learned by maximizing Intersection Over Union with ground-truth boxes through grid search.
4. EXPERIMENTS AND RESULTS
RRPN is evaluated on two NuScenes camera-Radar subsets with Fast R-CNN against Selective Search. It generally improves detection metrics and produces proposals far faster, including under occlusion and overlap.
- Dataset: NuScenes provides synchronized camera-Radar data, with experiments using six object classes and two camera-Radar subsets.NS-F contains 23k front-camera/front-Radar samples; NS-FB contains 45k images including rear-camera and rear-Radar data.
- Experimental setup: The experiments use RRPN proposals with Fast R-CNN, comparing them against Selective Search under a 2,000-proposal-per-image limit.Fast R-CNN is fine-tuned from COCO-pretrained R101 and X101 backbones on NS-F and NS-FB.
- Qualitative results: Figure 2 compares ground truth and mapped Radar detections with Selective Search and RRPN bounding boxes, including occlusion and overlap cases.The reported examples indicate that RRPN can propose accurate bounding boxes under these hard circumstances.
- Metrics: Evaluation reports COCO-style mean Average Precision and mean Average Recall, including AP at 0.5 and 0.75 IOU and area-specific AR.The area-specific recall categories are small, medium, and large objects.
- Detection results: RRPN outperforms Selective Search in almost all reported detection metrics across the NS-F and NS-FB datasets.Per-class results show especially wide margins for Person, Motorcycle, and Bicycle on NS-F, while NS-FB improves every class except Bus.
- Efficiency: RRPN proposes regions at 70 to 90 images per second, whereas Selective Search takes 2-7 seconds per image.RRPN throughput depends on the number of Radar detections.
5. CONCLUSION
RRPN is presented as a real-time Radar-based region proposal network for autonomous-driving object detection. It uses Radar detections to propose regions, achieving higher precision and recall while operating more than 100x faster than Selective Search.
- Conclusion: RRPN uses only Radar detections to propose regions of interest for autonomous-driving object detection.The method is described as a real-time region proposal network.
- Conclusion: RRPN achieves higher detection average precision and recall than Selective Search while operating more than 100x faster.The evaluation uses the NuScenes dataset and compares both region proposal approaches.