RON: Reverse Connection with Objectness Prior Networks for Object Detection

Tao Kong; Fuchun Sun; Anbang Yao; Huaping Liu; Ming Lu; Yurong Chen

Source-linked AI summary

RON: Reverse Connection with Objectness Prior Networks for Object Detection

Tao Kong, Fuchun Sun, Anbang Yao, Huaping Liu, Ming Lu, Yurong Chen

arXiv:1707.01691v1Jul 6, 2017cs.CV

TL;DR

Object detectors must balance the accuracy of region-based methods with the efficiency of region-free methods while handling multi-scale objects and many negative candidates. RON combines reverse connections, objectness priors, and joint multi-task optimization in a fully convolutional detector. It reports state-of-the-art performance on standard benchmarks and 15 FPS at test time with 1.5G GPU memory.

Problem
Object detection needs efficient handling of objects at multiple scales and severe negative-sample imbalance while balancing region-based accuracy against region-free efficiency.
Method
RON uses reverse connections for semantically enriched multi-level features, objectness priors to guide object search, and a multi-task loss to optimize the network jointly.
Results
RON achieves state-of-the-art object detection performance on standard benchmarks, including 81.3% mAP on PASCAL VOC 2007, and reports 15 FPS with 1.5G GPU memory.
Takeaways & Limitations
RON directly predicts final detection results from multiple feature-map locations while combining multi-scale representation, objectness guidance, and end-to-end optimization.
Takeaways & Limitations
The reported comparison notes that newer SSD training tricks could improve RON’s results, but evaluating those tricks is beyond this paper’s focus.

Abstract

from arXiv · show

We present RON, an efficient and effective framework for generic object detection. Our motivation is to smartly associate the best of the region-based (e.g., Faster R-CNN) and region-free (e.g., SSD) methodologies. Under fully convolutional architecture, RON mainly focuses on two fundamental problems: (a) multi-scale object localization and (b) negative sample mining. To address (a), we design the reverse connection, which enables the network to detect objects on multi-levels of CNNs. To deal with (b), we propose the objectness prior to significantly reduce the searching space of objects. We optimize the reverse connection, objectness prior and object detector jointly by a multi-task loss function, thus RON can directly predict final detection results from all locations of various feature maps. Extensive experiments on the challenging PASCAL VOC 2007, PASCAL VOC 2012 and MS COCO benchmarks demonstrate the competitive performance of RON. Specifically, with VGG-16 and low resolution 384X384 input size, the network gets 81.3% mAP on PASCAL VOC 2007, 80.7% mAP on PASCAL VOC 2012 datasets. Its superiority increases when datasets become larger and more difficult, as demonstrated by the results on the MS COCO dataset. With 1.5G GPU memory at test phase, the speed of the network is 15 FPS, 3X faster than the Faster R-CNN counterpart.

1. Introduction

RON bridges region-based and region-free object detection by addressing multi-scale localization and negative-sample mining in a fully convolutional framework. It combines reverse connections, objectness priors, and joint optimization, achieving strong accuracy and efficiency on standard benchmarks.

Motivation: Region-based detectors use proposals and region-wise refinement, whereas region-free detectors offer efficiency but struggle with negative samples and small objects.The paper identifies proposal-related computation as costly and single-shot detectors as limited by background suppression and weak intermediate-layer semantics.
Motivation: RON addresses multi-scale localization by combining information across CNN levels and enabling detection on multiple feature-map scales.Its motivation is that objects of different scales can appear anywhere, while prior approaches often detect scales at one layer or lack sufficient semantic information in earlier layers.
Approach: RON uses reverse connections to provide earlier CNN features with more semantic information for detection.The reverse connection is designed to associate the strengths of region-based and region-free approaches within a fully convolutional architecture.
Approach: The objectness prior explicitly guides object search, reducing the space of candidate regions that the detector must consider.The prior is motivated by the severe imbalance between object and non-object default boxes across feature maps.
Results: 81.3% mAP on PASCAL VOC 2007, 80.7% mAP on PASCAL VOC 2012, and 27.4% mAP on MS COCO are reported with 384×384 input.These results are attributed to the framework together with training strategies such as negative-example mining and data augmentation.
Results: 15 FPS with 1.5G GPU memory is reported, described as 3× faster than Faster R-CNN.The comparison concerns total feed-forward speed at test time.

2. Related Work

The related work spans region-proposal CNN pipelines and fully convolutional single-shot detectors. RON is presented against these families as a unified framework combining multi-scale detection, semantic feature enhancement, and objectness guidance.

Region-based methods: CNN-based detection evolved from region proposals plus ConvNet classification toward faster pipelines using shared features and region-wise processing.The related work discusses R-CNN, SPP-Net, Fast R-CNN, and subsequent improvements in proposals, feature fusion, context, and training.
RON: RON’s overview processes backbone features at multiple detection scales, adds reverse connections, generates objectness priors, detects objects, and applies non-maximum suppression.The pipeline fuses detection results from the different scales before final selection.
Region-free methods: Region-free methods formulate detection as a single-shot, fully convolutional mapping from image pixels to bounding-box coordinates.YOLO and SSD are described as efficient alternatives that detect objects without a separate proposal stage.

3. Network Architecture

RON combines multi-scale feature maps, reverse connections, default boxes, detection modules, and objectness-guided sample selection in a unified convolutional architecture.

Architecture: RON jointly trains reverse connections, objectness priors, and object detection in a unified framework for end-to-end detection.The framework is designed to provide detection capabilities at different network scales while guiding object search.
Multi-scale Detection: Multi-scale feature maps improve localization across object sizes, while enriched semantics make RON more effective than SSD for detecting objects at all scales.The architecture addresses the limited semantic information of earlier layers that affects SSD’s detection of smaller objects.
Reverse Connection: Reverse connections merge deconvolved higher-level maps with backbone features, enriching earlier layers and producing four reverse fusion maps at different scales.Each block applies deconvolution to the next reverse fusion map, matches dimensions with a convolution, and merges the maps by element-wise addition.
Default Boxes: Default boxes are distributed across feature maps with scale and aspect-ratio variation so predictions cover diverse object sizes and shapes.Each feature-map location uses 2 scales and 5 aspect ratios, with box scales defined by Sk = {(2k − 1) · smin, 2k · smin}.
Detection and Regression: The detection module uses inception blocks to classify each region into object categories plus background and to regress offsets relative to default boxes.The classifier predicts per-class scores, while regression predicts offsets relative to the default box in each cell.
Objectness Prior: Objectness priors reduce detection-branch computation by selecting only locations whose objectness scores exceed threshold op during back-propagation.The prior maps are binarized and projected to the detection domain, where selected locations are collected for object detection.

4. Training and Testing

RON trains objectness, localization, and classification jointly, then performs direct inference from the resulting predictions. Its training strategy combines multi-task optimization, negative-sample selection, and scale-oriented augmentation.

Loss Function: Each location produces objectness, bounding-box regression, and classification outputs as three sibling branches.The objectness branch predicts confidence, the regression branch predicts offsets, and the classification branch predicts K+1 categories.
Loss Function: Classification excludes regions whose objectness confidence falls below threshold op before computing class probabilities.
Loss Function: The multi-task loss jointly trains objectness prior, classification, and bounding-box regression end-to-end.
Loss Function: The loss weights α and β balance the three loss terms, with α = β = 1 used in all experiments after normalization.The supplied passage states that this setting works well.
Joint Training and Testing: Training selects all positive objectness samples and randomly samples negative samples within each SGD mini-batch.Mini-batches contain N images chosen uniformly from the dataset.
Joint Training and Testing: Scale-aware augmentation samples original or flipped images, random patches, and additional small-scale training examples.The strategy is intended to improve robustness across object scales and avoid over-fitting to specific sizes.
Joint Training and Testing: Inference multiplies class-conditional probabilities by individual box-confidence predictions, adjusts boxes, and applies non-maximum suppression.The resulting scores encode both class probability and predicted-box fit.

5. Results

RON achieves strong detection performance across PASCAL VOC and MS COCO, including competitive results with low-resolution inputs. Its gains are especially notable on the larger COCO benchmark and when using additional COCO pretraining.

PASCAL VOC 2007: RON achieves 75.4% mAP with 384×384 input, outperforming Faster R-CNN by 2.2 percentage points and SSD500 with a smaller input size.Multiscale testing, bounding-box voting, and flipping increase performance to 77.6% mAP for RON384++.
PASCAL VOC 2007: RON improves performance on 17 of 20 PASCAL VOC 2007 categories, including gains of 4.0 points for boat and 7.1 points for bottle.The cited analysis identifies small objects as particularly challenging across detectors.
PASCAL VOC 2007: RON’s recall exceeds 85% in detection analysis and is higher under the weak 0.1 Jaccard-overlap criterion.This analysis evaluates the quality of detections across object categories.
PASCAL VOC 2012: RON performs best on PASCAL VOC 2012 in the reported comparison, with improvements attributed mainly to reverse connections and boxes from multiple feature maps.The comparison uses the comp4 outside-data track and VGG-16-based methods.
MS COCO: 27.4% AP is achieved by RON on MS COCO, compared with 21.9% AP for Faster R-CNN and 25.5% AP for OHEM++.RON also reaches 26.2% AP with 320×320 input, exceeding SSD500 by 1.8 points under the strict COCO metric.
Cross-dataset transfer: MS COCO pretraining raises RON to 81.3% mAP on VOC 2007 and 80.7% mAP on VOC 2012, increasing mAP by 3.7% and 5.3%.The models are pretrained on MS COCO and fine-tuned on PASCAL VOC.

6. Ablation Analysis

Ablation experiments show that combining multiple detection layers and retaining the objectness prior are important for RON’s performance. The objectness prior also improves proposal recall when only a small number of proposals are retained.

Layer combinations: Using layers 4, 5, 6, and 7 together gives the detector its best performance under otherwise identical 320×320 settings.The experiment varies only the layers used for object detection.
Objectness prior: Objectness prior maps encode response strength and spatial position, guiding searches for objects at different scales and reducing the search space.The maps respond to objects of various scales on corresponding feature maps.
Objectness prior: Removing the objectness prior reduces VOC 2007 test mAP to 69.6%, a 4.6-point drop from the 74.2% baseline.All other settings remain the same as in the baseline experiment.
Generating region proposals: With the top 10 proposals, RON320 reaches 80.7% recall, outperforming Faster R-CNN by 20 points.Both methods achieve promising proposal recall above 100 proposals, but RON has a larger advantage with fewer proposals.

7. Conclusion

RON is an efficient object detection framework that combines reverse connections, objectness priors, and joint multi-task optimization. It detects objects across multiple CNN levels, guides object search, and achieves state-of-the-art performance on standard benchmarks.

Conclusion: Reverse connections enable object detection across multiple CNN levels, while objectness priors guide the search for objects.The framework combines these components within a fully convolutional detector.
Conclusion: A multi-task loss jointly optimizes the whole network so it can directly predict final detection results.The conclusion presents this as part of RON’s end-to-end framework.
Conclusion: RON achieves state-of-the-art object detection performance on standard benchmarks.The conclusion characterizes the framework as efficient and effective.