Source-linked AI summary
PVANET: Deep but Lightweight Neural Networks for Real-time Object Detection
Kye-Hyeon Kim, Sanghoon Hong, Byungseok Roh, Yeongjae Cheon, Minje Park
TL;DR
Object detection systems still impose heavy computational costs despite strong accuracy, motivating a more efficient network design. PVANET redesigns Faster R-CNN’s feature extractor using a thin, deep architecture and recent building blocks, achieving strong VOC accuracy with low computational cost and fast runtimes.
Problem
Object detection accuracy has improved, but leading systems remain computationally expensive, making computational cost an important network-design problem.
Method
PVANET redesigns the Faster R-CNN feature extraction network using less channels with more layers, C.ReLU, Inception, HyperNet-style multi-scale features, residual connections, batch normalization, and plateau-based learning-rate scheduling.
Results
83.8% mAP on VOC-2007 and 82.5% mAP on VOC-2012, with 46ms/image on an NVIDIA Titan X GPU and <7% of ResNet-101’s computational cost including multi-scale testing.
Takeaways & Limitations
PVANET shows that a thin, lightweight feature extractor can support real-time object detection while maintaining strong benchmark accuracy.
Takeaways & Limitations
The network is independent of compression and quantization, so additional techniques remain applicable for improving runtime performance in real applications.
Abstract
from arXiv · showhide
This paper presents how we can achieve the state-of-the-art accuracy in multi-category object detection task while minimizing the computational cost by adapting and combining recent technical innovations. Following the common pipeline of "CNN feature extraction + region proposal + RoI classification", we mainly redesign the feature extraction part, since region proposal part is not computationally expensive and classification part can be efficiently compressed with common techniques like truncated SVD. Our design principle is "less channels with more layers" and adoption of some building blocks including concatenated ReLU, Inception, and HyperNet. The designed network is deep and thin and trained with the help of batch normalization, residual connections, and learning rate scheduling based on plateau detection. We obtained solid results on well-known object detection benchmarks: 83.8% mAP (mean average precision) on VOC2007 and 82.5% mAP on VOC2012 (2nd place), while taking only 750ms/image on Intel i7-6700K CPU with a single core and 46ms/image on NVIDIA Titan X GPU. Theoretically, our network requires only 12.3% of the computational cost compared to ResNet-101, the winner on VOC2012.
1 Introduction
PVANET targets real-time object detection by redesigning feature extraction around a thin, deep architecture that combines efficient building blocks and training techniques. It reports competitive accuracy with substantially lower computational cost and fast CPU/GPU runtimes.
- Detection systems achieve acceptable accuracy, but even leading algorithms remain computationally expensive, motivating efficiency-oriented network design.
- PVANET is a lightweight feature extraction architecture designed for real-time object detection without losing accuracy against state-of-the-art systems.
- 7.9GMAC feature-extraction cost with a 1065x640 input, compared with 80.5GMAC for ResNet-101.
- <7% of ResNet-101’s computational cost when its multi-scale testing overhead is included.
- 750ms/image on a single-core Intel i7-6700K CPU and 46ms/image on an NVIDIA Titan X GPU.
- 83.8% mAP on VOC-2007 and 82.5% mAP on VOC-2012, where the latter achieved second place.
- The design principle is less channels with more layers, combined with C.ReLU, Inception, HyperNet-style multi-scale features, batch normalization, residual connections, and plateau-based learning-rate scheduling.
2 Details on Network Design
PVANET’s network design reduces computation while preserving diverse receptive fields and multi-scale information. Residual connections, batch normalization, and plateau-based scheduling support training of the resulting deep, thin architecture.
- C.ReLU: C.ReLU halves early-stage computations by pairing activations with their negations while retaining the doubled channel representation.
- Inception: Inception combines 1x1, 3x3, and 5x5 receptive-field paths, allowing later modules to capture wider-ranging visual patterns and higher-level nonlinearities.
- Inception: The Inception implementation replaces a 5x5 convolution with two 3x3 convolutions for efficiency and includes a feature-map reduction variant.
- Inception: Stacked Inception modules capture small and large objects by preserving small receptive fields while expanding others through deeper convolutions.
- HyperNet: Multi-scale intermediate outputs combine fine-grained details with abstract information, while layer selection controls redundancy and computational cost.
- Architecture: PVANET integrates C.ReLU-based convolutions, Inception modules, residual projections, and multi-scale feature construction throughout its detailed architecture.
- Training: Residual connections, including connections on Inception layers, stabilize training of the deep network; batch normalization and plateau detection further support optimization.
3 Faster R-CNN with our feature extraction network
The Faster R-CNN integration builds multi-scale feature maps from intermediate PVANET outputs and allocates them selectively to proposal and classification stages.
- C.ReLU is applied in early convolutional stages, with surrounding 1x1 convolutions reducing input size and enlarging representation capacity.
- Three intermediate outputs are rescaled and combined into 512-channel multi-scale features that feed the Faster R-CNN modules.
- R-CNN uses all 512 channels, applies RoI pooling to produce a 6x6x512 tensor, and processes it through fully connected layers.
4 Experimental results
Experiments evaluate PVANET on VOC benchmarks under specified training and testing settings, including proposal accuracy, compressed classification layers, and leaderboard efficiency. PVANET+ reaches 82.5% mAP on VOC2012 and combines high accuracy with substantially lower computational cost and runtime than heavier competitors.
- Experimental settings: PVANET was pretrained on ILSVRC2012 and trained on combined MS COCO and VOC data, followed by VOC-specific fine-tuning.Training used random image resizing with the shorter edge between 416 and 864 pixels.
- Experimental settings: VOC evaluations resized images to a 640-pixel shorter edge and used Faster R-CNN settings with 200 final proposals.Testing ran on an Intel i7-6700K CPU with one core and an NVIDIA Titan X GPU.
- VOC2007 results: Truncated SVD compressed the fully connected layers and produced 82.9% mAP with runtime of 31.3 FPS.The compressed configuration reduced mAP by 0.9% and increased speed by 9.6 FPS.
- VOC2012 results: 82.5% mAP placed PVANET+ second on the VOC2012 leaderboard, behind only Faster R-CNN + ResNet-101.PVANET+ outperformed the other listed competitors.
- VOC2012 results: Among networks exceeding 80% mAP, PVANET+ was the only one running at ≤50ms, while the top performer was 40x or more slower.The comparison used estimated computational costs and reported or projected runtime values under the table’s stated assumptions.
5 Conclusions
The paper concludes that current networks contain substantial redundancy and that a thin, lightweight network can support complex vision tasks. PVANET’s design is independent of compression and quantization, which can be applied additionally to improve runtime.
- Conclusion: Current networks are highly redundant, motivating thin and lightweight designs for complex vision tasks.The conclusion presents this as a demonstrated design possibility rather than a limitation to object detection alone.
- Conclusion: Combining recent deep-learning innovations enables computationally efficient redesign of Faster R-CNN’s feature-extraction component.The paper identifies feature extraction as the redesigned part of the detection framework.
- Conclusion: PVANET’s network design is independent of compression and quantization, allowing those techniques to be applied additionally.The authors give truncated SVD as an example that improves runtime on their network.