Source-linked AI summary
PP-PicoDet: A Better Real-Time Object Detector on Mobile Devices
Guanghua Yu, Qinyao Chang, Wenyu Lv, Chang Xu, Cheng Cui, Wei Ji, Qingqing Dang, Kaipeng Deng, Guanzhong Wang, Yuning Du, Baohua Lai, Qiwen Liu, Xiaoguang Hu, Dianhai Yu, Yanjun Ma
TL;DR
Lightweight object detection must balance accuracy and efficiency for mobile deployment. PP-PicoDet combines anchor-free detection with backbone, neck, label-assignment, loss, and NAS optimizations. The resulting models report strong accuracy–latency trade-offs, including over 30 mAP with about 1M parameters and over 40 mAP with 3.3M parameters.
Problem
Lightweight anchor-free detectors usually cannot balance accuracy and efficiency well for mobile object detection.
Method
PP-PicoDet combines anchor-free detection with ESNet, lightweight CSP-PAN, improved label assignment and loss, and an improved one-shot NAS pipeline.
Results
30.6% mAP is achieved by PicoDet-S with 0.99M parameters and 150 FPS on mobile ARM CPU at input size 320.
Takeaways & Limitations
PP-PicoDet offers lightweight mobile detectors with reported accuracy and efficiency superior to state-of-the-art lightweight object detection results.
Abstract
from arXiv · showhide
The better accuracy and efficiency trade-off has been a challenging problem in object detection. In this work, we are dedicated to studying key optimizations and neural network architecture choices for object detection to improve accuracy and efficiency. We investigate the applicability of the anchor-free strategy on lightweight object detection models. We enhance the backbone structure and design the lightweight structure of the neck, which improves the feature extraction ability of the network. We improve label assignment strategy and loss function to make training more stable and efficient. Through these optimizations, we create a new family of real-time object detectors, named PP-PicoDet, which achieves superior performance on object detection for mobile devices. Our models achieve better trade-offs between accuracy and latency compared to other popular models. PicoDet-S with only 0.99M parameters achieves 30.6% mAP, which is an absolute 4.8% improvement in mAP while reducing mobile CPU inference latency by 55% compared to YOLOX-Nano, and is an absolute 7.1% improvement in mAP compared to NanoDet. It reaches 123 FPS (150 FPS using Paddle Lite) on mobile ARM CPU when the input size is 320. PicoDet-L with only 3.3M parameters achieves 40.9% mAP, which is an absolute 3.7% improvement in mAP and 44% faster than YOLOv5s. As shown in Figure 1, our models far outperform the state-of-the-art results for lightweight object detection. Code and pre-trained models are available at https://github.com/PaddlePaddle/PaddleDetection.
1. Introduction
PP-PicoDet addresses the accuracy–efficiency challenge in lightweight mobile object detection through anchor-free modeling, architectural changes, improved training, and neural architecture search. The resulting models report strong accuracy and latency trade-offs across deployment sizes.
- Motivation: Lightweight YOLO detectors avoid the resource demands of two-stage models but require manual anchor redesign and face positive–negative sample imbalance.These limitations motivate more efficient anchor-free mobile detectors.
- Motivation: PP-PicoDet applies an anchor-free strategy to lightweight detection because existing lightweight anchor-free detectors struggle to balance accuracy and efficiency.The model is inspired by FCOS and GFL.
- Architecture: CSP-PAN reduces neck parameters by unifying branch channels with 1 × 1 convolutions and expands the receptive field using 5 × 5 depthwise separable convolutions.The neck uses CSP structure, while the larger depthwise separable convolution replaces 3 × 3 convolution.
- Training: SimOTA uses a weighted Varifocal Loss and GIoU loss cost matrix to improve label assignment accuracy without harming efficiency.The strategy optimizes calculation details for dynamic assignment.
- Architecture and search: Enhanced ShuffleNet (ESNet) improves on ShuffleNetV2 as a mobile backbone, while an improved one-shot NAS pipeline searches detection architectures directly on detection datasets.The NAS-generated models achieve better efficiency–accuracy trade-offs.
- Results: 30.6% mAP is achieved by PicoDet-S with 0.99M parameters and 150 FPS on mobile ARM CPU at input size 320.PicoDet-M reaches 34.3% mAP with 2.15M parameters, while PicoDet-L reaches 40.9% mAP with 3.3M parameters.
2. Related Works
Related work distinguishes anchor-based and anchor-free detectors, mobile-oriented architectures, and neural architecture search. Anchor-free approaches remove anchor boxes, while prior mobile detectors and NAS motivate PP-PicoDet’s design space.
- Anchor-based detection: Anchor-based two-stage detectors improve object positioning but make real-time detection difficult on CPU or ARM devices.They generate region proposals before producing final bounding boxes.
- Anchor-free detection: Anchor-free detectors eliminate anchor boxes and can reduce memory cost while improving bounding-box calculation.Examples include YOLOv1, CornerNet, CenterNet, and FCOS.
- Training strategies: ATSS selects positive and negative samples from object statistics, while GFL merges localization quality into the class prediction vector.These methods improve training assignment and quality representation.
- Mobile detection: Mobile detectors such as YOLObile, PP-YOLO-Tiny, NanoDet, and YOLOX-Nano target more accurate and efficient detection under mobile constraints.NanoDet uses ShuffleNetV2 with ATSS and GFL to reduce model size and enhance accuracy.
- PP-PicoDet architecture: PP-PicoDet’s architecture combines ESNet with CSP-PAN, which receives three backbone feature maps and produces four neck feature maps.For PicoDet-S, the neck uses input channels [96, 192, 384] and output channels [96, 96, 96, 96].
- Neural architecture search: Neural architecture search offers an alternative to hand-crafted architectures, and one-shot methods reduce computation by sharing weights across candidate networks.Prior NAS examples include MobileNetV3, EfficientNet, MnasNet, ENAS, and SMASH.
3. Approach
PP-PicoDet combines a manually enhanced backbone, lightweight neck and head structures, detection-specific NAS, dynamic label assignment, and mobile-oriented loss and training choices.
- Better Backbone: ESNet enhances ShuffleNetV2 with SE modules and modified structure to improve mobile-device performance.The paper presents ESNet as a backbone that performs better than ShuffleNetV2.
- Neural Architecture Search: One-shot NAS trains and searches a detection supernet directly on detection datasets, using evolutionary search to optimize detector architectures.This avoids searching only for classification backbones and provides computational savings for detection optimization.
- Neck and Head: CSP-PAN equalizes neck feature-map channels with 1 × 1 convolutions, then fuses features through top-down and bottom-up CSP paths.The neck uses PAN for multi-level features and CSP for feature concatenation and fusion.
- Neck and Head: The detector head uses depthwise separable 5 × 5 convolutions, four scale branches, consistent neck-head channels, and coupled classification and regression branches.The 5 × 5 convolution expands the receptive field while depthwise separable convolution targets lightweight computation.
- Label Assignment Strategy and Loss: SimOTA selects positives dynamically from a center-prior candidate area using a cost matrix based on weighted Varifocal and GIoU losses.The GIoU weight λ is set to 6 based on the paper’s experiments.
- Label Assignment Strategy and Loss: The detector uses Varifocal loss for coupled classification and quality prediction, GIoU loss for regression, and H-Swish for mobile-friendly activation.The reported detector loss is lossvfl + 2 · lossgiou + 0.25 · lossdfl, while H-Swish replaces ReLU-based activation choices.
4. Experiments
Experiments evaluate training choices, ablations, backbone and NAS variants, and comparisons with lightweight detectors on COCO and mobile CPU settings.
- Implementation Details: All ablation experiments use the COCO-2017 validation set, while training uses COCO-2017 with 80 classes and 118k images.The reported evaluation uses the standard single-scale COCO AP metric.
- Ablation Study: CSP-PAN raises mAP (0.5:0.95) from 25.3 in the base model to 28.1, while adding a feature-map scale increases parameters by less than 50K.The cited ablation compares a ShuffleNetV2-based base model with progressively enhanced neck configurations.
- Ablation Study: 30.0 mAP (0.5:0.95) is achieved by the modified SimOTA, while replacing QFL with VFL changes mAP from 29.1 to 29.2.The modified SimOTA sets n to 10 after larger values performed worse.
- Ablation Study: A GIoU-loss weight of 6 produces the best result in the SimOTA cost-matrix ablation.The experiments vary λ in the weighted loss formulation.
- Ablation Study: The searched model reduces mobile CPU inference time by 41.5% while decreasing mAP by 0.2%; replacing the backbone with ESNet-0.75x reduces parameters by nearly 200K.The Paddle Lite speedup reported for the searched model is 54.9%.
- Comparison with the SOTA: PP-PicoDet models exceed the cited YOLO models in accuracy and speed, with Paddle Lite sometimes outperforming NCNN in inference testing.The paper attributes the comparison advantage to the lightweight neck and the combination of loss, dynamic assignment, and regression choices.
5. Conclusion and Future Work
PP-PicoDet provides lightweight mobile detectors with high accuracy and efficiency, including models that exceed 30 or 40 mAP under compact deployment constraints.
- Conclusion and Future Work: PP-PicoDet-S surpasses 30 mAP while keeping 1M parameters and 100+ FPS on an ARM CPU.The paper describes this as its first model meeting all three conditions.
- Conclusion and Future Work: PP-PicoDet-L surpasses 40 mAP with only 3.3M parameters.The paper states that future work will investigate additional techniques for high-accuracy and efficient detectors.