Source-linked AI summary
PP-LCNet: A Lightweight CPU Convolutional Neural Network
Cheng Cui, Tingquan Gao, Shengyu Wei, Yuning Du, Ruoyu Guo, Shuilong Dong, Bin Lu, Ying Zhou, Xueying Lv, Qiwen Liu, Xiaoguang Hu, Dianhai Yu, Yanjun Ma
TL;DR
PP-LCNet addresses the difficulty of achieving fast, accurate lightweight inference on Intel CPUs, where MKLDNN can limit existing mobile networks. It combines accuracy-oriented design strategies with CPU-optimized building blocks, achieving a better accuracy-speed balance across vision tasks, including semantic segmentation and object detection.
Problem
Lightweight networks often lack ideal speed on Intel CPUs with MKLDNN, motivating methods that improve feature representation and accuracy without increasing latency.
Method
PP-LCNet combines CPU-optimized DepthSepConv blocks with strategically placed SE modules, large kernels, H-Swish, and other accuracy-preserving design strategies.
Results
PP-LCNet shows stronger performance and accuracy-speed balance across multiple vision tasks; in semantic segmentation, PP-LCNet-0.5x improves mIoU by 2.94% while reducing inference time by 53ms versus MobileNetV3-large-0.5x.
Takeaways & Limitations
The reported design rules can guide lightweight Intel CPU CNN construction and reduce the search space for neural architecture search.
Abstract
from arXiv · showhide
We propose a lightweight CPU network based on the MKLDNN acceleration strategy, named PP-LCNet, which improves the performance of lightweight models on multiple tasks. This paper lists technologies which can improve network accuracy while the latency is almost constant. With these improvements, the accuracy of PP-LCNet can greatly surpass the previous network structure with the same inference time for classification. As shown in Figure 1, it outperforms the most state-of-the-art models. And for downstream tasks of computer vision, it also performs very well, such as object detection, semantic segmentation, etc. All our experiments are implemented based on PaddlePaddle. Code and pretrained models are available at PaddleClas.
1. Introduction
The paper targets lightweight CNNs whose inference speed is difficult to optimize on Intel CPUs with MKLDNN, and asks how to improve accuracy without increasing latency. It proposes PP-LCNet by combining accuracy-oriented design methods for a better accuracy-speed balance.
- 1. Introduction: Lightweight CNNs become harder to run quickly as feature extraction capacity, parameters, and FLOPs increase, especially on Intel CPUs with MKLDNN.The paper contrasts this setting with mobile networks whose speed is not ideal under MKLDNN acceleration.
- 1. Introduction: Figure 1 compares mobile-series models by accuracy and latency, with latency measured on an Intel Xeon Gold 6148 using batch size 1, MKLDNN, and 10 threads.The figure’s comparison uses a specific Intel CPU inference configuration.
- 1. Introduction: The paper asks how to strengthen feature representations without increasing latency, improve lightweight-model accuracy on CPUs, and combine design strategies effectively.These questions frame both the model-design problem and the strategy-combination problem.
- 1. Introduction: PP-LCNet summarizes methods that improve accuracy with little or no inference-time increase and combines them to balance accuracy and speed.The contribution also formulates general rules for lightweight CNN design.
- 1. Introduction: The framework is presented as useful for future neural architecture search by supplying ideas for constructing search spaces and obtaining better models faster.This consequence is stated alongside the paper’s design rules for lightweight CNNs.
2. Related Works
Related work develops lightweight CNNs through manually designed architectures and neural architecture search. Representative approaches reduce computation with specialized blocks, while NAS explores architectures built from manually defined search components.
- 2. Related Works: Lightweight CNN research follows manually designed architectures or neural architecture search.The paper identifies these as the two broad methodologies for improving model capability.
- 2. Related Works: VGG stacks same-dimension blocks, while GoogLeNet uses an Inception block with parallel 1 × 1, 3 × 3, and 5 × 5 convolutions plus max pooling.These examples illustrate manually designed architectural strategies.
- 2. Related Works: MobileNetV1 replaces standard convolution with depthwise and pointwise convolutions to reduce model parameters and FLOPs.This factorized convolution design is presented as a foundation for later lightweight networks.
- 2. Related Works: NAS-generated networks such as EfficientNet, MobileNetV3, FBNet, DNANet, and OFANet commonly use search spaces similar to MobileNetV2.MixNet instead mixes depthwise convolutions with different kernel sizes within one layer.
3. Approach
PP-LCNet is designed for Intel CPU inference by combining a depthwise-separable basic block with selectively placed activation, attention, kernel, and post-GAP expansion choices. The approach emphasizes accuracy improvements that add little inference latency under MKLDNN.
- 3. Approach: Intel CPU lightweight networks require dedicated design because accuracy-improving methods that are inexpensive on ARM can affect inference time differently on Intel CPUs.The approach specifically studies performance with acceleration strategies such as MKLDNN enabled.
- 3. Approach: Table 1 records PP-LCNet architecture details, including whether each block contains Squeeze-and-Excitation and whether batch normalization is absent.SE denotes Squeeze-and-Excitation, while NBN denotes no batch normalization.
- 3. Approach: PP-LCNet stacks DepthSepConv blocks without shortcut, concatenation, or elementwise-add operations, using a block optimized by Intel CPU acceleration libraries.The paper states that this block can surpass inverted-block and shufflenet-block speed.
- 3. Approach: H-Swish is adopted as an activation choice intended to improve performance beyond ReLU while avoiding a large number of exponential operations.The passage motivates H-Swish through the development from Swish to its MobileNetV3 variant.
- 3.2. SE modules at appropriate positions: SE modules are restricted to blocks near the network tail because placing them throughout the network increases Intel CPU inference time and tail placement gives a better accuracy-speed balance.The paper reports stronger effects at the network end and therefore limits SE usage.
- 3.3. Larger convolution kernels: The approach uses one convolution-kernel size per layer and places large kernels where low latency and high accuracy can be maintained.This avoids the inference slowdown associated with mixing kernel sizes within a layer.
- 3.4. Larger dimensional 1 × 1 conv layer after GAP: A 1280-dimensional 1 × 1 convolution after GAP strengthens fitting ability by preserving more feature combinations with little inference-time increase.The layer is described as equivalent to a fully connected layer.
- 3. Approach: The four design changes produce strong ImageNet-1k performance, with Table 3 comparing PP-LCNet against other lightweight models on Intel CPUs.The passage attributes the result to the combined changes rather than a single component.
4. Experiment
Experiments evaluate PP-LCNet on image classification, object detection, semantic segmentation, and component ablations under Intel CPU inference conditions. Across tasks, PP-LCNet improves accuracy or mIoU while maintaining or reducing inference time, with targeted SE and large-kernel placement supporting the accuracy–speed balance.
- Experimental setup: PP-LCNet is compared with reimplemented lightweight baselines using Intel Xeon Gold 6148 inference with batch size 1 and MKLDNN enabled.Models are trained on 4 V100 GPUs, while CPU testing uses MKLDNN acceleration.
- Image classification: PP-LCNet shows strong competitiveness against state-of-the-art lightweight models on ImageNet-1k classification.Evaluation reports top-1 and top-5 validation accuracy together with inference time across model scales; SSLD distillation further improves accuracy.
- Object detection: PP-LCNet greatly improves COCO mAP and inference speed over MobileNetV3 when used as the object-detection backbone.The evaluation uses PicoDet as the baseline method and the COCO AP metric at a single scale.
- Semantic segmentation: PP-LCNet-0.5x exceeds MobileNetV3-large-0.5x by 2.94% mIoU while reducing inference time by 53ms on Cityscapes.PP-LCNet-1x is also 1.5% higher in mIoU and 55ms faster than MobileNetV3-large-0.75x.
- Ablation study: Adding SE modules to the last two network blocks is more advantageous for nearly the same inference time, so PP-LCNet places SE only there.The experiments indicate that SE placement toward the network tail has greater impact than adding modules indiscriminately.
- Ablation study: H-Swish and large kernels improve performance with almost no inference-time increase, while limited SE, a larger post-GAP fully connected layer, and dropout further improve accuracy.The ablation study reports cumulative effects of four technologies and identifies a tail-oriented 5 × 5 depth-wise convolution configuration.
5. Conclusion and Future work
The work experimentally summarizes design methods for lightweight Intel CPU networks and combines them into PP-LCNet. The resulting model achieves a stronger accuracy–speed balance across vision tasks and may reduce NAS search space, with future work targeting faster and stronger NAS-derived models.
- Conclusion: PP-LCNet combines existing lightweight-network methods to improve accuracy without increasing inference time and achieves stronger performance across multiple vision tasks.The paper emphasizes experimentally balancing accuracy and speed for Intel CPU networks.
- Future work: The summarized design methods can reduce NAS search space and support faster access to lightweight models.The authors propose using NAS in future work to obtain faster and stronger models.