Source-linked AI summary
HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection
Tao Kong, Anbang Yao, Yurong Chen, Fuchun Sun
TL;DR
HyperNet addresses inefficient region proposal generation and coarse feature maps by jointly generating proposals and detecting objects with fused hierarchical CNN features. It achieves high recall with few proposals, state-of-the-art PASCAL VOC detection accuracy, and 5 fps processing.
Problem
Existing proposal methods often require thousands of proposals for high recall, while coarse feature maps limit small-object detection and precise localization.
Method
HyperNet jointly generates region proposals and detects objects using shared Hyper Features that fuse hierarchical CNN maps into a uniform representation.
Results
HyperNet achieves 97% recall with 100 proposals, state-of-the-art mAP on PASCAL VOC 2007 and 2012, and 5 fps using very deep CNN models.
Takeaways & Limitations
The architecture produces few high-recall proposals while maintaining state-of-the-art detection accuracy and efficient processing on standard benchmarks.
Abstract
from arXiv · showhide
Almost all of the current top-performing object detection networks employ region proposals to guide the search for object instances. State-of-the-art region proposal methods usually need several thousand proposals to get high recall, thus hurting the detection efficiency. Although the latest Region Proposal Network method gets promising detection accuracy with several hundred proposals, it still struggles in small-size object detection and precise localization (e.g., large IoU thresholds), mainly due to the coarseness of its feature maps. In this paper, we present a deep hierarchical network, namely HyperNet, for handling region proposal generation and object detection jointly. Our HyperNet is primarily based on an elaborately designed Hyper Feature which aggregates hierarchical feature maps first and then compresses them into a uniform space. The Hyper Features well incorporate deep but highly semantic, intermediate but really complementary, and shallow but naturally high-resolution features of the image, thus enabling us to construct HyperNet by sharing them both in generating proposals and detecting objects via an end-to-end joint training strategy. For the deep VGG16 model, our method achieves completely leading recall and state-of-the-art object detection accuracy on PASCAL VOC 2007 and 2012 using only 100 proposals per image. It runs with a speed of 5 fps (including all steps) on a GPU, thus having the potential for real-time processing.
1. Introduction
HyperNet targets the efficiency and localization limits of region-proposal detection by combining multi-level CNN features for joint proposal generation and detection. It reports high recall, strong VOC detection accuracy, and 5 fps using far fewer proposals.
- Motivation: Traditional proposal-based detectors often use thousands of category-independent proposals, increasing computation and slowing detection.R-CNN extracts approximately 2k proposals, while proposal generation can consume substantial runtime.
- Motivation: Fast R-CNN and RPN improve efficiency but remain limited by coarse deep feature maps, especially for small objects and high IoU thresholds.The cited limitations include poor localization for small instances and IoU thresholds above 0.8.
- Method: HyperNet aggregates coarse-to-fine CNN features into a uniformly represented Hyper Feature for proposal generation and detection.Different sampling strategies align multi-level features, while convolution compresses them into a uniform space.
- Results: 95% recall with 50 proposals and 97% recall with 100 proposals substantially reduces the proposal count relative to traditional thousand-level approaches.These recall results are reported for the object proposal task.
- Results: 5 fps including all steps indicates that the speed-up version approaches real-time processing while maintaining proposal and detection accuracy.The paper reports this speed with very deep CNN models.
2. Related Work
The related work traces object detection from traditional proposal methods and sliding-window recognition to CNN-based systems that use region proposals to reduce computation.
- Object proposals: Object proposals reduce computation relative to dense sliding-window detection and include traditional and deep-learning-based approaches.Traditional methods use cues such as superpixels, edges, saliency, and shapes.
- HyperNet: HyperNet’s architecture overview uses 100 proposals and then classifies and adjusts each region after computing a Hyper Feature representation.The figure caption presents this as a four-step pipeline from image input to region refinement.
- CNN detection: CNN-based object detection replaces hand-crafted image descriptors and supports region-based recognition with learned representations.The related-work discussion contrasts CNN features with HOG, SIFT, and LBP descriptors.
3. HyperNet Framework
HyperNet combines hierarchical CNN features into a uniform Hyper Feature, then shares it across proposal generation and object detection. Its design targets accurate proposals with fewer candidates while reducing computation through lightweight modules and a speed-oriented convolution placement.
- 3. HyperNet Framework: About 100 proposals are generated after Hyper Feature extraction, then classified and adjusted by the detection module.The framework first processes the entire image, constructs Hyper Features, and applies a lightweight proposal network.
- 3.1. Hyper Feature Production: HyperNet aggregates hierarchical feature maps, samples them to a common resolution, compresses each with convolution, normalizes them, and concatenates them into one Hyper Feature.Lower layers are subsampled, higher layers are upsampled, and convolution extracts semantic features while compressing channels.
- 3.2. Region Proposal Generation: The proposal network evaluates about 30k candidate boxes with ROI pooling, a convolutional layer, a fully connected layer, and sibling scoring and regression outputs.Greedy NMS at IoU 0.7 leaves about 1k proposals before selecting the top-k ranked regions.
- 3.3. Object Detection: The detection module adds a 3×3×63 convolution before the fully connected layer and predicts N+1 class scores plus 4×N box-regression offsets.The convolution reduces feature dimensions, while dropout is reduced from 0.5 to 0.25.
- 3.4. Joint Training: Joint optimization uses a six-step training process that separately trains and then fine-tunes proposal and detection modules while sharing Hyper Feature layers.The final step combines the two modules into a unified network.
- 3.5. Speeding up: Moving the 3×3×4 convolution before ROI pooling reduces channels from 126 to 4 and changes the classifier from Conv-FC to FC, yielding a 40× proposal-generation speedup with a small recall drop.The proposal-generation module previously consumed about 70% of forward time because it repeatedly evaluated tens of thousands of candidate boxes.
4. Comparison to Prior Works
HyperNet differs from prior proposal-and-detection systems by unifying the pipeline and using far fewer proposals. The paper connects this design to improved detection accuracy, small-object discovery, and localization at high IoU thresholds.
- Fast R-CNN: HyperNet uses 100 proposals instead of Fast R-CNN’s 2000 and reports more accurate object detection results.Both systems predict and score candidate boxes using ConvNet features, but HyperNet produces proposals and detections in one unified network.
- Faster R-CNN: Unlike Faster R-CNN, HyperNet replaces the Fast R-CNN detection component with a redesigned unified network for proposal generation and detection.The paper also describes HyperNet’s bounding-box regression and region scoring as different from Faster R-CNN’s approach.
- DeepProposal: HyperNet fuses multi-level CNN features through in-network sampling, whereas DeepProposal uses a cascade that refines proposals from deep toward initial network layers.The paper states that HyperNet’s Hyper Feature is more suitable for small-object discovery and performs well at IoU thresholds above 0.8.
5. Experimental Evaluation
HyperNet combines hierarchical CNN features for joint proposal generation and detection, achieving strong recall, localization, and detection accuracy with few proposals across PASCAL VOC benchmarks.
- Region Proposal Generation: With 50 proposals at IoU=0.5, HyperNet outperforms RPN by 11 points, Selective Search by 42 points, and EdgeBoxes by 39 points.
- Region Proposal Generation: At IoU=0.7, HyperNet requires 20 proposals for 75% recall, compared with 250 for RPN, 800 for EdgeBoxes, and 1400 for Selective Search.
- PASCAL VOC 2007 Results: HyperNet achieves 76.3% mAP on PASCAL VOC 2007, exceeding Fast R-CNN by 6.3 points and Faster R-CNN by 3.1 points.
- PASCAL VOC 2007 Results: HyperNet improves localization, particularly for small objects, achieving 62.4% AP for bottle and 51.2% AP for potted plant.
- PASCAL VOC 2012 Results: HyperNet reaches 71.4% mAP on PASCAL VOC 2012, 3.0 and 1.0 points above the comparison counterparts, while its speedup version achieves 71.3% mAP.
6. Conclusion
HyperNet jointly generates region proposals and detects objects with a fully trainable architecture that combines CNN features across semantic depth and resolution. It achieves high recall with few proposals and state-of-the-art detection accuracy on standard benchmarks.
- HyperNet is a fully trainable architecture for joint region proposal generation and object detection.
- Its feature framework combines deep semantic, intermediate complementary, and shallow high-resolution CNN features.
- HyperNet produces few object proposals while maintaining high recall.
- Both the basic HyperNet and its speed-up version achieve state-of-the-art object detection accuracy on standard benchmarks.