Source-linked AI summary
Lite-HRNet: A Lightweight High-Resolution Network
Changqian Yu, Bin Xiao, Changxin Gao, Lu Yuan, Lei Zhang, Nong Sang, Jingdong Wang
TL;DR
Efficient pose estimation needs high-resolution representations without the cost of conventional lightweight blocks. Lite-HRNet combines HRNet with shuffle blocks, then replaces costly pointwise convolutions using conditional channel weighting computed across channels and resolutions. It reports stronger accuracy–complexity trade-offs than popular lightweight baselines and extends to semantic segmentation.
Problem
Human pose estimation requires high-resolution representations, but efficient high-resolution models are needed under computation-limited resources.
Method
Lite-HRNet replaces shuffle blocks' costly pointwise 1 × 1 convolutions with input-conditioned channel weights computed across channels and parallel resolutions.
Results
Lite-HRNet achieves superior complexity–accuracy trade-offs over lightweight baselines on COCO and MPII pose estimation and is applicable to semantic segmentation.
Takeaways & Limitations
Conditional channel weighting can support efficient high-resolution networks while exchanging information across channels and resolutions.
Takeaways & Limitations
The analysis treats the constant 9 as ignorable in time-complexity calculations for convenience.
Abstract
from arXiv · showhide
We present an efficient high-resolution network, Lite-HRNet, for human pose estimation. We start by simply applying the efficient shuffle block in ShuffleNet to HRNet (high-resolution network), yielding stronger performance over popular lightweight networks, such as MobileNet, ShuffleNet, and Small HRNet. We find that the heavily-used pointwise (1x1) convolutions in shuffle blocks become the computational bottleneck. We introduce a lightweight unit, conditional channel weighting, to replace costly pointwise (1x1) convolutions in shuffle blocks. The complexity of channel weighting is linear w.r.t the number of channels and lower than the quadratic time complexity for pointwise convolutions. Our solution learns the weights from all the channels and over multiple resolutions that are readily available in the parallel branches in HRNet. It uses the weights as the bridge to exchange information across channels and resolutions, compensating the role played by the pointwise (1x1) convolution. Lite-HRNet demonstrates superior results on human pose estimation over popular lightweight networks. Moreover, Lite-HRNet can be easily applied to semantic segmentation task in the same lightweight manner. The code and models have been publicly available at https://github.com/HRNet/Lite-HRNet.
1. Introduction
Lite-HRNet targets efficient high-resolution modeling for human pose estimation by combining HRNet's high-resolution design with lightweight shuffle blocks and conditional channel weighting. The resulting network improves the accuracy–efficiency trade-off over several lightweight baselines.
- Human pose estimation benefits from high-resolution representations, motivating efficient high-resolution models under computation-limited resources.
- Combining ShuffleNet shuffle blocks with HRNet produces a naive Lite-HRNet that outperforms ShuffleNet, MobileNet, and Small HRNet.Small HRNet reduces the original HRNet's depth and width.
- Conditional channel weighting replaces costly pointwise 1 × 1 convolutions with channel-wise information exchange whose complexity is linear in channel count rather than quadratic.For 64 × 64 × 40 and 32 × 32 × 80 multi-resolution features, it reduces the shuffle block's total computation complexity by 80%.
- The proposed weights are conditioned on input maps and computed across channels and parallel resolutions, allowing them to exchange information across channels and resolutions.This produces the Lite-HRNet network.
- Lite-HRNet outperforms naive Lite-HRNet, with the authors attributing the advantage to larger complexity reduction than information-exchange loss.
- Lite-HRNet is reported as state-of-the-art in the complexity–accuracy trade-off on COCO and MPII pose estimation and is applicable to semantic segmentation.
2. Related Work
Prior lightweight networks reduce convolutional or spatial costs, but channel communication remains challenging when using grouped or depthwise operations. Lite-HRNet addresses this with conditional channel weighting while preserving HRNet's parallel high-resolution structure.
- MobileNet, ShuffleNet, and related networks reduce convolutional cost using separable or grouped convolutions and other lightweight blocks.
- Grouped and depthwise convolutions block information flow across channels, while pointwise convolutions restore exchange at substantial cost.
- Mediating spatial information loss: Lightweight networks also improve efficiency by reducing spatial resolution and recovering detail through encoder-decoder, multi-resolution, or decoupled sub-networks.
- Conditional weighting: Attention methods generate channel or spatial weights from feature information, including global, local, or combined channel-spatial cues.
- Channel information exchange: Conditional channel weighting provides a lightweight alternative to costly 1 × 1 convolutions and uses conditional weights to exchange information across channels.It can be viewed as a conditional channel-wise 1 × 1 convolution.
- High-resolution architecture: HRNet maintains parallel multi-resolution streams with repeated information exchange, and Lite-HRNet follows this high-resolution pattern.
3. Approach
Lite-HRNet combines HRNet’s parallel multi-resolution design with shuffle blocks, then replaces their costly pointwise convolutions with conditional channel weighting. The weighting exchanges information across channels and resolutions while reducing computational complexity.
- Shuffle blocks: The shuffle block splits channels, processes one partition with pointwise and depthwise convolutions, concatenates both partitions, and shuffles the result.
- Simple combination: Naive Lite-HRNet replaces Small HRNet’s stem and residual convolutions with shuffle blocks and uses separable convolutions for multi-resolution fusion.
- Conditional channel weighting: 1 × 1 convolutions have quadratic complexity Θ(C^2), whereas conditional channel weighting has linear complexity Θ(C) with respect to channel count.
- Conditional channel weighting: Conditional channel weighting computes element-wise channel weights from input maps across parallel resolutions, replacing the 1 × 1 convolution’s information-exchange role.
- Cross-resolution weight computation: The weight-generation function uses pooled multi-resolution maps and lightweight convolutions to produce resolution-specific maps for subsequent channel weighting.
- Cross-resolution weight computation: Each output channel receives information from all input channels at the same position across resolutions, while weight computation runs at small resolution and remains lightweight.
- Instantiation: Lite-HRNet modules contain two conditional channel weighting blocks and one multi-resolution fusion, with branch widths C, 2C, 4C, and 8C.
4. Experiments
Experiments evaluate Lite-HRNet on COCO and MPII human pose estimation, with ablations of conditional channel weighting, then test its transfer to Cityscapes semantic segmentation.
- 4.2. Results: Lite-HRNet-30 reaches 67.2 AP on COCO val at 256 × 192, outperforming lightweight methods while using only 20% of MobileNetV2’s GFLOPs and parameters.It gains 2.6 AP over MobileNetV2, 7.3 points over ShuffleNetV2, and over 10 AP points over Small HRNet-W16.
- 4.2. Results: At 384 × 288 input, Lite-HRNet-18 and Lite-HRNet-30 achieve 67.6 and 70.4 AP, respectively, balancing accuracy with computational complexity.On COCO test-dev, Lite-HRNet-30 achieves 69.7 AP and remains more efficient than competing small networks.
- 4.2. Results: Lite-HRNet-30 achieves 87.0 PCKh @0.5 on MPII, improving over MobileNetV2, MobileNetV3, ShuffleNetV2, and Small HRNet-W16.The respective gains are 1.6, 2.7, 4.2, and 6.8 points.
- 4.3. Ablation Study: Naive Lite-HRNet improves COCO val AP over Small HRNet-W16 by 7.3 points while using less than half its GFLOPs and parameters.This tests whether HRNet’s high-resolution design benefits small models when combined with shuffle blocks.
- 4.3. Ablation Study: Conditional channel weighting improves AP by 3.5 points over removing two 1 × 1 convolutions, while increasing computation by only 16M FLOPs.Its AP is comparable with wider naive Lite-HRNet at 65% of the FLOPs, and deeper Lite-HRNet adds 1.5 AP with similar FLOPs.
- 4.3. Ablation Study: Spatial weights add 1.3 AP and multi-resolution weights add 1.7 AP on COCO val; combining both yields a 3.5-point improvement.The FLOPs of both weighting operations are reported as cheap, with consistent improvements on MPII val.
- 4.4. Application to Semantic Segmentation: Lite-HRNet-18 achieves 72.8% mIoU at 1.95 GFLOPs and Lite-HRNet-30 achieves 75.3% mIoU at 3.02 GFLOPs on Cityscapes.These results use no sliding-window or multi-scale testing tricks and outperform listed hand-crafted and NAS-based methods.