Source-linked AI summary
Optimizing the Trade-off between Single-Stage and Two-Stage Object Detectors using Image Difficulty Prediction
Petru Soviany, Radu Tudor Ionescu
TL;DR
Object detection requires balancing the higher accuracy but slower speed of two-stage detectors against the speed but lower accuracy of single-stage detectors. The paper predicts image difficulty to route easy test images to the fast detector and hard images to the accurate detector, and reports favorable results versus random splitting on PASCAL VOC 2007.
Problem
Choosing object detectors that jointly balance accuracy and speed is difficult because two-stage and single-stage models offer opposing strengths.
Method
An image difficulty predictor splits test images into easy and hard groups, routing them to single-stage and two-stage detectors respectively.
Results
On PASCAL VOC 2007, difficulty-based splitting compares favorably with random splitting across the evaluated detector combinations.
Takeaways & Limitations
The approach provides a simple, practical way to choose an accuracy–speed trade-off by adjusting the image-splitting threshold.
Abstract
from arXiv · showhide
There are mainly two types of state-of-the-art object detectors. On one hand, we have two-stage detectors, such as Faster R-CNN (Region-based Convolutional Neural Networks) or Mask R-CNN, that (i) use a Region Proposal Network to generate regions of interests in the first stage and (ii) send the region proposals down the pipeline for object classification and bounding-box regression. Such models reach the highest accuracy rates, but are typically slower. On the other hand, we have single-stage detectors, such as YOLO (You Only Look Once) and SSD (Singe Shot MultiBox Detector), that treat object detection as a simple regression problem by taking an input image and learning the class probabilities and bounding box coordinates. Such models reach lower accuracy rates, but are much faster than two-stage object detectors. In this paper, we propose to use an image difficulty predictor to achieve an optimal trade-off between accuracy and speed in object detection. The image difficulty predictor is applied on the test images to split them into easy versus hard images. Once separated, the easy images are sent to the faster single-stage detector, while the hard images are sent to the more accurate two-stage detector. Our experiments on PASCAL VOC 2007 show that using image difficulty compares favorably to a random split of the images. Our method is flexible, in that it allows to choose a desired threshold for splitting the images into easy versus hard.
I. INTRODUCTION
Object detection balances accuracy against speed: two-stage detectors are more accurate but slower, while single-stage detectors are faster but less accurate. The paper proposes routing easy images to the fast detector and hard images to the accurate detector using predicted image difficulty.
- Two-stage detectors use region proposals before classification and bounding-box regression, achieving high accuracy but typically requiring more time.
- Single-stage detectors treat detection as regression over class probabilities and bounding boxes, providing faster but less accurate predictions.
- An image difficulty predictor separates test images into easy and hard cases for detector-specific processing.
- Easy images are sent to a faster single-stage detector, whereas hard images are sent to a more accurate two-stage detector.
- Experiments on PASCAL VOC 2007 favor difficulty-based splitting over random splitting and allow a continuously chosen accuracy–speed trade-off.
II. RELATED WORK
Prior work leaves detector selection constrained by the accuracy–speed trade-off. This paper proposes continuously routing easy and hard test images between fast single-stage and accurate two-stage detectors.
- Existing detectors differ substantially in accuracy and computational speed, making selection of a single best model difficult.
- The paper frames optimal detector selection as a nontrivial accuracy–speed trade-off.
- The proposed framework sets this trade-off continuously by choosing where to split test images into easy and hard groups.
- Faster R-CNN uses region proposal networks and a second-stage Fast R-CNN model, whereas SSD performs computation in a single network and is faster.
III. METHODOLOGY
The method predicts image difficulty from CNN features and uses a threshold-based easy-versus-hard strategy to dispatch test images between fast single-stage and slow two-stage detectors.
- The algorithm uses an input test image, two detectors, an image difficulty predictor, and a threshold dividing images into easy or hard.
- Curriculum-learning principles motivate applying easy-to-hard selection at test time rather than changing the object detectors.
- The framework routes easy samples to a fast, less accurate detector and difficult samples to a complex, slower, more accurate detector.
- The difficulty predictor extracts 4096 normalized fc7 features from VGG-f and trains ν-SVR to regress human-provided difficulty scores.
- The predictor provides a continuous automatic difficulty measure and reaches Kendall’s τ = 0.441 on the referenced test set.
A. Data Set
The experiments use PASCAL VOC 2007, a 9963-image dataset covering 20 object classes, with approximately 5000 test images.
- PASCAL VOC 2007 contains 9963 images spanning 20 object classes.
- The training and validation sets contain roughly 2500 images each, while the test set contains about 5000 images.
B. Evaluation Measure
Object-detector performance is evaluated with mean Average Precision (mAP), which summarizes class-wise precision-recall behavior based on detection-score rankings.
- mAP evaluates object detectors by averaging Average Precision across object classes.For each class, Average Precision is the area under its precision-recall curve.
- Average Precision is computed after matching detections to ground-truth boxes using Intersection over Union (IoU).
- Precision-recall curves progressively include detections sorted in decreasing order of their confidence scores.
C. Models and Baselines
The evaluation combines a slower, accurate Faster R-CNN detector with faster single-shot alternatives and compares random image assignment against difficulty-based assignment.
- Faster R-CNN with ResNet-101 serves as the accurate two-stage detector, while MobileNet-SSD and SSD300 provide fast single-shot detection.SSD300 uses 300 × 300 pixel inputs and performs detection in one step.
- The experiments compare random splitting of images with easy-versus-hard splitting based on scores from the image difficulty predictor.The random baseline is repeated five times and averaged; reported standard deviations are below 0.5%.
D. Results and Discussion
Difficulty-based routing improves the accuracy–speed trade-off over random splitting: easy images use fast detectors, while hard images use Faster R-CNN.
- Faster R-CNN reaches mAP 0.7837 at about 7.74 seconds per image, compared with SSD300 at 0.69 mAP and 0.56 seconds.MobileNet-SSD reaches 0.6668 mAP in 0.07 seconds per image.
- Processing time improves by about 21% for the 25%−75% MobileNet-SSD/Faster R-CNN split and by 69% for the 75%−25% split.
- 50%−50% MobileNet-SSD/Faster R-CNN routing is nearly 47% shorter than using Faster R-CNN alone.The difficulty predictor contributes roughly 1% of total processing time in this split: 0.05 out of 4.13 seconds per image.
- The easy-versus-hard strategy improves mAP over random splitting for every evaluated detector combination.At the 50%−50% split, gains are 3% with MobileNet-SSD and 3.35% with SSD300.
- At the 50%−50% SSD300/Faster R-CNN split, mAP is 3.24% below standalone Faster R-CNN while processing time is reduced by almost half.
- Detector predictions differ less on easy images and substantially more on hard images, where MobileNet-SSD misses small objects and produces incorrect boxes or labels.This observed pattern supports routing difficult images to Faster R-CNN.
V. CONCLUSION
The paper presents difficulty-based dispatching as a practical way to balance detection accuracy and speed, with favorable results versus random splitting on PASCAL VOC 2007.
- The strategy dispatches test images to a fast, less accurate single-shot detector or a slow, more accurate two-stage detector according to image difficulty.
- Experiments with SSD300 and Faster R-CNN on PASCAL VOC 2007 show that difficulty-based splitting compares favorably with random splitting.
- Future work will study alternative dispatching strategies and detectors trained specifically for easy or hard samples.