Source-linked AI summary
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search
Xin Li, Yiming Zhou, Zheng Pan, Jiashi Feng
TL;DR
Deploying CNNs requires a better speed/accuracy trade-off on each target platform, because FLOPs and generic architecture choices do not reliably reflect actual inference speed. The paper proposes Partial Order Pruning, which uses platform latency profiles and partial-order pruning to search backbone and decoder architectures. The resulting DF and DF-Seg networks achieve strong speed/accuracy trade-offs, including reported state-of-the-art results across embedded and high-end GPU platforms.
Problem
Existing architecture search often relies on FLOPs rather than target-platform latency, while the paper seeks the best accuracy-latency boundary for a given platform.
Method
Partial Order Pruning uses profiled platform latency, a partial-order assumption, and cutting-plane search to prune architecture candidates while balancing overall width and depth.
Results
DF1/DF2A exceed ResNet18/50 accuracy on ImageNet while reducing latency by 43%/39%, and DF-Seg achieves state-of-the-art speed/accuracy trade-offs on embedded and high-end GPUs.
Takeaways & Limitations
The approach produces platform-specific DF backbones and DF-Seg networks with high accuracy and fast inference for deployment-oriented CNN design.
Takeaways & Limitations
The pruning strategy relies on a partial-order assumption over architectures, such as ordering shallower or narrower architectures ahead of deeper or wider ones.
Abstract
from arXiv · showhide
Achieving good speed and accuracy trade-off on a target platform is very important in deploying deep neural networks in real world scenarios. However, most existing automatic architecture search approaches only concentrate on high performance. In this work, we propose an algorithm that can offer better speed/accuracy trade-off of searched networks, which is termed "Partial Order Pruning". It prunes the architecture search space with a partial order assumption to automatically search for the architectures with the best speed and accuracy trade-off. Our algorithm explicitly takes profile information about the inference speed on the target platform into consideration. With the proposed algorithm, we present several Dongfeng (DF) networks that provide high accuracy and fast inference speed on various application GPU platforms. By further searching decoder architectures, our DF-Seg real-time segmentation networks yield state-of-the-art speed/accuracy trade-off on both the target embedded device and the high-end GPU.
1. Introduction
The paper frames neural architecture search as a target-platform speed/accuracy trade-off problem and proposes Partial Order Pruning to search this boundary efficiently. It reports DF and DF-Seg networks with high accuracy and fast inference on embedded and high-end GPU platforms.
- Motivation: Embedded devices have limited computation capacity, while faster CNN inference often comes at the cost of degraded performance.The paper motivates optimizing both inference speed and accuracy rather than pursuing either objective alone.
- Motivation: The target-platform search asks for maximum accuracy under a latency limit or minimum latency under an accuracy requirement.These two questions define the desired speed/accuracy trade-off boundary.
- Motivation: FLOPs do not reliably predict GPU inference speed because hardware optimization and memory access also affect latency.The paper gives 3 × 3 versus 1 × 1 convolution as an example of the mismatch between FLOP ratios and actual speed.
- Approach: Partial Order Pruning filters candidates early using a partial order assumption, concentrating search on architectures likely to improve the speed/accuracy boundary.For example, wider networks with the same depth can be discarded when they cannot be more efficient than narrower ones.
- Approach: The algorithm explicitly considers platform characteristics, balances overall width and depth, and uses a cutting plane algorithm to accelerate search.This distinguishes it from approaches based on reinforcement learning, evolutionary algorithms, or gradient-based search.
- Results: DF1/DF2A exceed ResNet18/50 accuracy on ImageNet while reducing inference latency by 43% and 39%, respectively.These DF networks target the embedded device TX2.
- Results: DF1-Seg achieves 106.4 FPS with mIoUclass 74.1% on GTX 1080Ti and 21.8 FPS at 720p on TX2.The paper reports new state-of-the-art real-time segmentation speed/accuracy trade-offs on both high-end GPUs and the target embedded device.
2. Related Work
Prior efficient-network and neural architecture search methods often optimize indirect complexity measures or fixed design components. The paper instead emphasizes target-platform characteristics and searches the overall architecture’s width and depth.
- Efficient Network Design: Efficient CNN designs such as MobileNet V2 and ShuffleNet reduce FLOPs using group convolutions, but typically design a single architecture without target-platform consideration.The paper contrasts this with its platform-aware search approach.
- Efficient Network Design: FLOPs can diverge from actual inference speed, making them an imperfect proxy for deployment efficiency.The related work motivates using platform-specific characteristics rather than relying only on indirect complexity metrics.
- Neural Architecture Search: Reinforcement-learning and evolutionary NAS methods require substantial computational resources, while gradient-based methods were introduced to reduce search cost.The paper positions Partial Order Pruning among approaches intended to make architecture search more efficient.
- Neural Architecture Search: Unlike methods that search individual building blocks while fixing overall depth and width, the paper balances the width and depth of the overall architecture.Its search-space design targets architectural structure rather than only local modules.
3. Partial Order Pruning
Partial Order Pruning models target-platform latency, assumes a partial order over efficient architectures, and prunes candidates unlikely to improve the speed/accuracy boundary. It then searches backbone and decoder configurations to obtain efficient classification and segmentation architectures.
- Search Space: The search space contains architectures varying in depth and width, with latency constrained to a target range [Tmin, Tmax].The general architecture uses residual blocks, and the relevant subspace contains architectures whose latency falls within the specified interval.
- Latency Estimation: A TensorRT-derived lookup table sums layer latencies to estimate architecture latency, closely matching profiled latency.Each block configuration is assigned a measured latency; for example, one configuration takes 0.143ms on TX2.
- Partial Order Assumption: The partial order assumes that a shallower or narrower architecture has no higher latency or accuracy than its deeper or wider counterpart.The paper notes that this assumption may fail for very deep networks but is generally true for the efficient architectures in the constrained search space.
- Partial Order Pruning: The algorithm searches across [Tmin, Tmax] and uses higher-latency results to reduce the search space at lower latencies.This avoids separately optimizing every small latency interval and accelerates the overall search.
- Partial Order Pruning: For each trained architecture, the cutting-plane procedure prunes precedents whose latency is no lower than a faster, more accurate trained architecture.The pruned regions are formed from precedents of trained architectures, while B(D) records the trained architectures defining the achieved speed/accuracy boundary.
- Decoder Design: The resulting segmentation networks combine a backbone, pyramid pooling, channel controllers, and decoder fusion nodes.Fusion nodes project and upsample low-resolution features, concatenate them with higher-resolution features, and process the combined tensor.
4. Experiment
Experiments evaluate Partial Order Pruning across embedded and high-end GPUs for classification and segmentation. The resulting DF networks improve speed/accuracy trade-offs while demonstrating that platform-specific latency and memory access matter beyond FLOPs.
- Experimental settings: Experiments use Jetson TX2 and GTX 1080Ti, with Titan X (Maxwell) added for fair comparison with prior methods.Inference speed is measured with TensorRT and, for comparison with ICNet, Caffe Time.
- Backbone architecture search: 69.8% accuracy: DF1 is 43% and 51% lower latency than ResNet-18 and GoogLeNet, respectively, while DF2 exceeds their accuracy at similar latency.DF2A reaches ResNet-50-level accuracy with 39% lower latency; DF1 and DF2A are slimmer and deeper than their corresponding baselines.
- Backbone architecture search: DF1 outperforms MobileNet and ShuffleNet on TX2 despite their lower FLOPs, reflecting higher memory-access cost in those alternatives.Reported intermediate-feature memory costs are 4.86M for ShuffleNet V2 and 2.91M for DF1.
- Backbone architecture search: DF networks show better speed/accuracy trade-offs than NASNet, PNASNet, FBNet, and ProxylessNAS on TX2.The paper attributes this to TX2-specific search, lower memory access, and balancing overall width and depth rather than only building blocks.
- Search efficiency: 438 architectures are pruned after training 200, accelerating the search process 2.2 times.Training 200 architectures requires approximately 400 GPU days, with each model taking 5–7 hours on an 8-GPU server.
- Decoder architecture search: DF1-Seg achieves comparable speed to BiSeNet1 with 5.1% higher mIoUclass, while DF2-Seg2 reaches 76.9% validation mIoUclass at 56.3 FPS.DF1-Seg-d8 reaches 136.9 FPS on 1080Ti, and its 71.4% test mIoUclass exceeds ICNet and BiSeNet1.
- Decoder architecture search: DF1-Seg and DF1-Seg-d8 reach 59.9 FPS and 75.9 FPS at 1080p on Titan X, supporting a strong high-end-GPU speed/accuracy trade-off.Figure 9 presents decoder search results across TX2 and GTX 1080Ti for DF1-Seg and DF2-Seg variants.
5. Conclusion
The paper proposes Partial Order Pruning to improve the speed/accuracy boundary of searched networks on target platforms. It applies the method to backbone and decoder searches, producing DF and DF-Seg networks with state-of-the-art trade-offs across embedded and high-end GPUs.
- Conclusion: Partial Order Pruning uses a partial order assumption to prune feasible architectures and accelerate architecture search.The method searches both backbone and decoder architectures.
- Conclusion: DF backbone networks provide state-of-the-art speed/accuracy trade-offs on target platforms.The conclusion reports this outcome at the backbone-network level without specifying a single platform or metric.
- Conclusion: DF-Seg networks achieve state-of-the-art speed/accuracy trade-offs on embedded devices and high-end GPUs.The result covers both deployment settings evaluated in the paper.