Source-linked AI summary
Quantized Densely Connected U-Nets for Efficient Landmark Localization
Zhiqiang Tang, Xi Peng, Shijie Geng, Lingfei Wu, Shaoting Zhang, Dimitris Metaxas
TL;DR
Visual landmark localization is challenging, and the paper proposes quantized densely connected U-Nets that reuse features across stacked U-Nets. Across human pose estimation and face alignment, it reports comparable localization accuracy alongside reduced model and training costs.
Problem
Locating visual landmarks, including body joints and facial key points, is an important yet challenging problem.
Source: §1 · p. 1Method
DU-Net connects semantically corresponding blocks across U-Nets and combines order-K connectivity, memory-efficient training, iterative refinement, and quantization.
Source: §1 · pp. 1–3Results
Across human pose estimation and face alignment, DU-Net reports comparable localization accuracy; overall, it reports state-of-the-art performance with ∼30% parameters, ∼2% model size, and ∼25% training memory.
Source: §1 · p. 3, §5 · p. 14Takeaways & Limitations
The paper concludes that DU-Net combines state-of-the-art landmark localization performance with reduced parameter, model-size, and training-memory costs.
Source: §5 · p. 14
Abstract
from arXiv · showhide
In this paper, we propose quantized densely connected U-Nets for efficient visual landmark localization. The idea is that features of the same semantic meanings are globally reused across the stacked U-Nets. This dense connectivity largely improves the information flow, yielding improved localization accuracy. However, a vanilla dense design would suffer from critical efficiency issue in both training and testing. To solve this problem, we first propose order-K dense connectivity to trim off long-distance shortcuts; then, we use a memory-efficient implementation to significantly boost the training efficiency and investigate an iterative refinement that may slice the model size in half. Finally, to reduce the memory consumption and high precision operations both in training and testing, we further quantize weights, inputs, and gradients of our localization network to low bit-width numbers. We validate our approach in two tasks: human pose estimation and face alignment. The results show that our approach achieves state-of-the-art localization accuracy, but using ~70% fewer parameters, ~98% less model size and saving ~75% training memory compared with other benchmark localizers. The code is available at https://github.com/zhiqiangdon/CU-Net.
1 Introduction
The paper introduces densely connected U-Nets that reuse features across U-Nets to improve information flow and localization. It addresses the resulting efficiency concerns with order-K connectivity, memory-efficient training, iterative refinement, and quantization.
1 Introduction
- DU-Net connects blocks with the same semantic meanings across U-Nets, reusing features globally to improve information flow and localization accuracy.Source: §1 · pp. 1–2, Figure 1 · p. 2
- A memory-efficient implementation reuses pre-allocated memory across connected blocks, while iterative refinement feeds the first pass output into a second pass supervised by detection or regression loss.Source: §1 · pp. 2–3
- Order-K connectivity cuts long-distance connections to balance fitting accuracy and parameter efficiency, reducing model-size growth from quadratic to linear.Source: §1 · p. 3
- The approach quantizes weights, inputs, and gradients to reduce training memory consumption and high-precision operations.Source: §1 · p. 3
- ∼75% training memory saving is achieved by quantized DU-Net with comparable performance.Source: §1 · p. 3
- In human pose estimation and face alignment, DU-Net achieves comparable localization accuracy while using ∼2% model size compared with state-of-the-art methods.Source: §1 · p. 3
2 Related Work
Related work covers network architectures and quantization, alongside applications to human pose estimation and facial landmark localization.
2 Related Work
- DenseNet motivates extending dense connectivity across multiple U-Nets, while stacked U-Nets build on residual modules.Source: §2 · p. 3
- Network quantization reduces model size by cutting high-precision operations and operands, while BCLL uses XNOR-Net for binarization.Source: §2 · p. 3
- BCLL [5] quantizes weights for inference but retains real-value scaling factors and high-precision training, so it does not save training memory or improve training efficiency.Source: §2 · p. 3
- Recent human pose methods build on stacked hourglasses, modifying residual modules or adding graphical models, adversarial supervision, or data augmentation.Source: §2 · pp. 3–4
- For facial landmark localization, stacked hourglasses achieved state-of-the-art performance in the Menpo Challenge, while order-K U-Nets could perform better with fewer parameters.Source: §2 · p. 4
3 Our Method
The method develops densely connected U-Nets and adds strategies to manage their parameter, training-memory, and precision costs. It covers order-K connectivity, memory-efficient implementation, iterative refinement, and network quantization.
3.1 DU-Net
- DU-Net connects same-location blocks across stacked U-Nets, so blocks receive same-semantic features from preceding U-Nets as well as current-network features.Features are concatenated, extending dense connectivity from layers within a block to semantic blocks across U-Nets.Source: §3.1 · pp. 4–5
3.2 Order-K Connectivity
- Medium-order connectivity reduces parameter growth from quadratic to linear; paired with intermediate supervision, it targets parameter efficiency and prediction accuracy.Intermediate supervision at each U-Net may also alleviate vanishing gradients.Source: §3.2 · pp. 5–6
3.3 Memory Efficient Implementation
- The memory-efficient implementation reuses pre-allocated memory for connected-block concatenations and batch normalization instead of allocating new memory for every connection.Because later features overwrite earlier ones in shared memory, concatenations and batch normalization must be recomputed during the backward phase.Source: §3.3 · p. 6
3.4 Iterative Refinement
- Iterative refinement runs a DU-Net twice, merging the first and last U-Net inputs in a small dense block before feeding the refined input through again.The approach uses only half of a DU-Net and may achieve comparable performance.Source: §3.4 · p. 6
3.5 Network Quantization
- Weights can be quantized to one or two bits, while layerwise inputs and gradients are quantized using linear mapping.The quantization is intended to reduce high-precision operations and parameter costs in training and inference.Source: §3.5 · p. 7
4 Experiments
The experiments evaluate DU-Net against stacked U-Nets, vary connectivity and implementation choices, test iterative refinement and quantization, and compare the final models on pose and face-alignment benchmarks.
4.1 DU-Net vs. Stacked U-Nets
- With the same number of U-Nets, DU-Net achieves comparable or better PCKh with about 70% fewer parameters than stacked U-Nets, enabling 16 U-Nets on a 12G GPU.Training 16 stacked U-Nets was infeasible under the same setting.Source: Table 1 · p. 9, §4.1 · p. 9
4.2 Evaluation of Order-K connectivity
- As connectivity order increases, parameter count grows approximately linearly while PCKh first rises and then falls; very dense connections can overfit.Order-7 has higher training PCKh than order-1 throughout training, but lower validation PCKh in the final epochs.Source: §4.2 · p. 9, Figure 4 · p. 10, Table 3 · p. 10
4.3 Evaluation of Efficient Implementation
- With order-1 connectivity, batch size 16, and a 12GB GPU, memory-efficient training supports 16 U-Nets, versus at most 9 with the naive implementation.Training memory grows more slowly with the memory-efficient implementation.Source: Figure 5 · p. 10, §4.3 · p. 10
4.4 Evaluation of Iterative Refinement
- On 300-W, iterative DU-Net(4) achieves comparable NME to non-iterative DU-Net(8) with only 0.2M additional parameters on DU-Net(4).Iteration lowers localization errors, and regression supervision outperforms detection supervision in both iterative and non-iterative settings.Source: Table 4 · p. 10, §4.4 · p. 11
4.5 Evaluation of Network Quantization
- On 300-W, BW-QIG(818) reduces training memory by more than 4× and model size by 32× while outperforming TSR.BW-α is closest in error to the original DU-Net(4), while ternary weights generally outperform binary weights because they offer more weight-value choices.Source: §4.5 · pp. 11–12, Table 5 · p. 12
4.6 Comparison with State-of-the-art Methods
- On MPII, DU-Net-BW-α(16) has 27%-62% parameters and less than 2% model size of recent methods; on 300-W, it has ∼50× smaller model size with comparable errors.On LSP, DU-Net-BW-α(16) has at most ∼70% fewer parameters and ∼50× smaller model size than other state-of-the-art methods.Source: Table 7 · p. 13, §4.6 · p. 13, Table 8 · p. 14
5 Conclusion
DU-Net extends dense connectivity across stacked U-Nets by linking blocks with the same semantic meanings. Order-K connectivity, iterative refinement, and quantization improve efficiency while retaining comparable localization accuracy.
5 Conclusion
- DU-Net connects blocks with the same semantic meanings across U-Nets; order-K connectivity and iterative refinement improve parameter efficiency.Quantization further reduces training memory and model size; experiments report state-of-the-art performance with ∼30% of the parameters, ∼2% of the model size, and ∼25% of the training memory.Source: §5 · p. 14