Source-linked AI summary
CSPNet: A New Backbone that can Enhance Learning Capability of CNN
Chien-Yao Wang, Hong-Yuan Mark Liao, I-Hau Yeh, Yueh-Hua Wu, Ping-Yang Chen, Jun-Wei Hsieh
TL;DR
CSPNet addresses the high computation costs that make powerful CNNs difficult to use on resource-constrained devices. It reduces redundant gradient information through cross-stage feature fusion and truncated gradient flow, reducing computation while maintaining or improving accuracy across image classification and object detection experiments.
Problem
Increasing CNN depth and width improves performance but substantially increases computation, challenging deployment for resource-constrained devices.
Method
CSPNet partitions feature maps and merges them through cross-stage hierarchies that diversify gradient flow and limit duplicate gradient information.
Results
CSPNet reduces computation by 10%–20% while outperforming several CNN baselines on ImageNet classification and achieving strong COCO AP50 results across hardware platforms.
Takeaways & Limitations
CSPNet provides a lightweight CNN design applicable to ResNet, ResNeXt, and DenseNet for mobile and edge-computing deployment.
Abstract
from arXiv · showhide
Neural networks have enabled state-of-the-art approaches to achieve incredible results on computer vision tasks such as object detection. However, such success greatly relies on costly computation resources, which hinders people with cheap devices from appreciating the advanced technology. In this paper, we propose Cross Stage Partial Network (CSPNet) to mitigate the problem that previous works require heavy inference computations from the network architecture perspective. We attribute the problem to the duplicate gradient information within network optimization. The proposed networks respect the variability of the gradients by integrating feature maps from the beginning and the end of a network stage, which, in our experiments, reduces computations by 20% with equivalent or even superior accuracy on the ImageNet dataset, and significantly outperforms state-of-the-art approaches in terms of AP50 on the MS COCO object detection dataset. The CSPNet is easy to implement and general enough to cope with architectures based on ResNet, ResNeXt, and DenseNet. Source code is at https://github.com/WongKinYiu/CrossStagePartialNetworks.
1 Introduction
CSPNet addresses the computational burden of deep CNNs by splitting and recombining feature maps to enrich gradient flow while reducing computation. It is designed to preserve learning capability and support efficient deployment on CPUs and mobile GPUs.
- Motivation: Deepening or widening neural networks increases computation, making object detection difficult to deploy on small devices.Real-world applications require short inference time and lightweight computing.
- Deployment: CSPNet develops computationally efficient components for deploying CNNs on CPUs and mobile GPUs without sacrificing performance.This contribution targets efficient deployment under real-world inference constraints.
- Method: CSPNet partitions a base-layer feature map and merges the parts through a cross-stage hierarchy to create richer gradient combinations.Splitting gradient flow across different network paths is the core design concept.
- Results: 10% to 20%: CSPNet reduces computation effort when applied to ResNet, ResNeXt, and DenseNet while maintaining or improving learning capability.The method is intended to preserve sufficient accuracy during lightweightening and can be applied across these architectures.
- Results: 50% COCO AP50 at 109 fps: the proposed model achieves this result on GTX 1080ti.CSPNet’s improved learning capability enables smaller models to achieve better accuracy.
2 Related work
The related work reviews CNN architecture designs that improve efficiency through cardinality and feature reuse, alongside real-time object detectors emphasizing computational efficiency and detection performance.
- CNN architectures design: ResNeXt shows that cardinality can be more effective than width and depth in CNN architecture design.
- CNN architectures design: DenseNet reduces parameters and computations by reusing features and concatenating outputs from all preceding layers as subsequent inputs.
- Real-time object detector: YOLOv3 and SSD are identified as prominent real-time object detectors, while LRF and RFBNet achieve state-of-the-art real-time performance on GPUs based on SSD.
- Real-time object detector: CenterNet and CornerNet-Lite represent mainstream anchor-free detectors that perform well in both efficiency and efficacy.
3 Method
CSPNet addresses duplicate gradient information by splitting feature maps across a stage, processing only part through the main block, and hierarchically fusing the resulting features. This design preserves feature reuse while reducing redundant gradient learning and can be extended to ResNet and ResNeXt.
- Motivation: DenseNet repeatedly reuses copied gradient information when updating different dense layers.This causes distinct layers to learn duplicated gradient information.
- Cross Stage Partial DenseNet: CSPDenseNet splits base-layer feature maps into two parts, sends one directly to the stage end, and processes the other through a dense block.The partial transition layer applies transition, concatenation, and another transition to hierarchically fuse the paths.
- Cross Stage Partial DenseNet: CSPDenseNet separately integrates gradients from dense layers and the bypassed feature map, preventing either side from containing duplicate gradient information belonging to the other.The split paths truncate gradient flow while preserving distinct gradient combinations.
- Partial Dense Block: The partial dense block doubles the number of gradient paths and can save at most half of a network’s memory traffic.Its split-and-merge strategy also balances computation across layers and alleviates disadvantages of explicit feature-map copying.
- Apply CSPNet to Other Architectures: CSPNet applies to ResNet and ResNeXt by sending only half of the feature channels through Res(X)Blocks, eliminating the need for a bottleneck layer.With FLOPs fixed, this makes the theoretical lower bound of Memory Access Cost possible.
4 Experiments
Experiments on ImageNet and MS COCO evaluate CSPNet and EFM through ablations and comparisons across classification and detection architectures. The results show reduced computation or memory with competitive or improved accuracy, including gains for lightweight and mobile-oriented models.
- Experimental setup: ImageNet classification validates CSPNet, while MS COCO object detection evaluates the proposed EFM.The study also reports architectural details in the appendix.
- CSPNet ablation: 21% lower computation causes only a 0.1% accuracy degradation with the partial transition layer in CSPNet ablations.The partial transition layer is designed to reduce learning of redundant information.
- EFM ablation: 2.1% higher AP and 2.4% higher AP50 are achieved by EFM over GFM, at a cost of 2 fps slower inference.Adding GIoU increases AP by 0.7% but decreases AP50 by 2.7%, so it is excluded from subsequent models.
- ImageNet comparisons: At least 10% lower computational load is obtained across ResNet-, ResNeXt-, and DenseNet-based models while accuracy remains unchanged or improves.CSPResNet-10 improves accuracy by 1.8% compared with ResNet-10.
- MS COCO comparisons: 38.4% AP, 60.6% AP50, and 41.6% AP75 are achieved by CSPResNeXt50 with PANet (SPP) against object detectors running at 30∼100 fps.Against ResNet101 with LRF at 512×512 input, it improves AP by 0.7%, AP50 by 1.5%, and AP75 by 1.1%.
- Mobile efficiency: 11.5% higher AP50 than YOLOv3-tiny is obtained by CSPPeleeNet Ref.-EFM (SAM) on mobile GPU, while reaching 41 fps on Jetson TX2.At that platform, EFM (SAM) is 3 fps faster than PRN (3l) and has 4.6% AP50 growth.
5 Conclusion
CSPNet light-weights ResNet, ResNeXt, and DenseNet for mobile GPUs or CPUs by addressing redundant gradient information and costly inference. It uses cross-stage feature fusion and truncated gradient flow to increase learned-feature variability across layers.
- CSPNet enables ResNet, ResNeXt, and DenseNet to be light-weighted for mobile GPUs or CPUs.
- The method identifies redundant gradient information as a cause of inefficient optimization and costly inference computations.
- CSPNet enhances learned-feature variability across layers through cross-stage feature fusion and truncated gradient flow.