Source-linked AI summary
SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation
Yanjie Li, Sen Yang, Peidong Liu, Shoukui Zhang, Yunxiao Wang, Zhicheng Wang, Wankou Yang, Shu-Tao Xia
TL;DR
Heatmap-based HPE suffers from quantization error, costly upsampling, refinement post-processing, and weak low-resolution performance. SimCC reformulates localization as independent horizontal and vertical classification with multiple bins per pixel, and experiments report advantages over heatmap-based counterparts while reducing computation. Its demonstrated setting is top-down pose estimation; bottom-up multi-person matching remains future work.
Problem
2D heatmap HPE suffers from quantization error, costly upsampling layers, refinement post-processing, and limited low-resolution performance.
Method
SimCC reformulates HPE as independent horizontal and vertical coordinate classification, uniformly dividing each pixel into multiple bins and using lightweight classifier heads.
Results
SimCC shows advantages over heatmap-based representations across experiments, including higher performance with reduced computational cost in reported settings.
Takeaways & Limitations
SimCC provides a simpler HPE pipeline that can omit refinement post-processing and, under certain settings, upsampling layers while supporting sub-pixel localization.
Takeaways & Limitations
The paper evaluates SimCC under top-down pose estimation; bottom-up multi-person estimation faces identification ambiguity and may require extra embeddings for coordinate matching.
Abstract
from arXiv · showhide
The 2D heatmap-based approaches have dominated Human Pose Estimation (HPE) for years due to high performance. However, the long-standing quantization error problem in the 2D heatmap-based methods leads to several well-known drawbacks: 1) The performance for the low-resolution inputs is limited; 2) To improve the feature map resolution for higher localization precision, multiple costly upsampling layers are required; 3) Extra post-processing is adopted to reduce the quantization error. To address these issues, we aim to explore a brand new scheme, called \textit{SimCC}, which reformulates HPE as two classification tasks for horizontal and vertical coordinates. The proposed SimCC uniformly divides each pixel into several bins, thus achieving \emph{sub-pixel} localization precision and low quantization error. Benefiting from that, SimCC can omit additional refinement post-processing and exclude upsampling layers under certain settings, resulting in a more simple and effective pipeline for HPE. Extensive experiments conducted over COCO, CrowdPose, and MPII datasets show that SimCC outperforms heatmap-based counterparts, especially in low-resolution settings by a large margin.
1 Introduction
SimCC addresses quantization errors in 2D heatmap HPE by reformulating localization as horizontal and vertical coordinate classification. This design simplifies the pipeline and improves performance, particularly for low-resolution inputs.
- 1 Introduction: 2D heatmap methods discretize continuous coordinates into downscaled maps, causing quantization error and limiting low-resolution performance.This motivates costly upsampling and refinement procedures.
- 1 Introduction: SimCC can omit costly upsampling layers under certain settings and is presented as a simpler alternative to 1D and 2D heatmap pipelines.The comparison includes additional decoupling layers in 1D heatmap regression and refinement stages in 2D heatmaps.
- 1 Introduction: SimCC reformulates HPE as two independent classification tasks for horizontal and vertical coordinates.A CNN- or Transformer-based backbone extracts keypoint representations before coordinate classification.
- 1 Introduction: Uniformly dividing each pixel into multiple bins provides sub-pixel localization precision while avoiding the heatmap pipeline’s extra refinement post-processing.The method uses two lightweight classifier heads, one for each coordinate axis.
- 1 Introduction: Over 55% GFLOPs are reduced for SimBa-Res50 while achieving higher model performance than a heatmap-based counterpart.Experiments cover COCO, CrowdPose, and MPII with different backbones and input sizes.
2 Related Work
Related work spans direct coordinate regression, heatmap-based HPE, quantization-error refinement, facial-landmark 1D heatmaps, and coordinate-classification approaches. SimCC differs by targeting HPE with a lightweight coordinate-classification pipeline.
- 2 Related Work: Regression-based HPE directly predicts keypoint coordinates in a computationally friendly framework but has seen limited adoption because of unsatisfactory performance.
- 2 Related Work: Heatmap-based HPE improved localization through architectures such as hourglass networks, offset aggregation, deconvolutional heads, and high-resolution representations.
- 2 Related Work: Post-processing methods such as DARK reduce quantization error by modeling heatmap activation distributions with Taylor-expansion-based approximation.
- 2 Related Work: Facial-landmark work decouples 2D heatmaps into 1D heatmaps using deconvolution, learnable decoupling layers, and co-attention.
- 2 Related Work: Pix2Seq also uses classification for discrete object representations, whereas SimCC develops coordinate classification specifically for HPE and is designed to combine with CNN-based backbones.
3 SimCC: Reformulating HPE from Classification Perspective
SimCC uses a backbone and two classifier heads to predict horizontal and vertical coordinates independently. Pixel subdivision reduces quantization error and supports sub-pixel precision, while experiments examine robustness, label smoothing, and computational savings.
- 3 SimCC: Reformulating HPE from Classification Perspective: SimCC consists of a backbone network followed by horizontal and vertical classifier heads for coordinate prediction.The backbone may be CNN- or Transformer-based, and CNN features can be flattened before classification.
- 3 SimCC: Reformulating HPE from Classification Perspective: The splitting factor k creates Nx = W · k and Ny = H · k coordinate bins, with larger k reducing quantization error and enabling sub-pixel precision.The coordinate labels are uniformly discretized for the horizontal and vertical axes.
- 3 SimCC: Reformulating HPE from Classification Perspective: SimCC trains coordinate classifiers with Kullback–Leibler divergence and explores equal, Laplace, and Gaussian label smoothing.Equal label smoothing is the default SimCC variant unless otherwise noted.
- 3 SimCC: Reformulating HPE from Classification Perspective: SimCC-based methods outperform heatmap counterparts across 64×64, 128×128, 256×192, and 384×288 inputs, with especially clear gains at low resolutions.The text attributes this robustness to lower quantization error.
4 Experiments
Experiments across COCO, CrowdPose, and MPII evaluate SimCC against heatmap-based methods, with COCO results showing gains across resolutions and reduced computation. Ablations examine post-processing, upsampling, splitting factor, and label smoothing.
- Experimental Setup: Experiments cover COCO, CrowdPose, and MPII using standard benchmark evaluations and selected CNN- and Transformer-based baselines.COCO uses average precision, while MPII uses PCKh evaluation.
- Post-processing: SimCC omits costly refinement post-processing while maintaining higher COCO validation performance than heatmap-based counterparts across input resolutions.Heatmap methods use DARK or empirical shift, whereas SimCC operates without extra refinement post-processing.
- COCO Results: +11.2 AP is achieved by SimCC over heatmap-based counterparts at 64×64 input resolution on COCO validation data.At 256×192, SimCC-based HRNet-W48 also improves by +0.8 AP.
- COCO Results: +0.5 and +1.2 AP are obtained by SimCC-based HRNet-W48 and SimpleBaseline-Res50, respectively, on COCO test-dev at 384×288.The comparisons use heatmap-based counterparts.
- Upsampling Ablation: +0.4 AP and 23.5% faster speed are achieved by deconvolution-free SimCC SimpleBaseline-Res50 versus its heatmap counterpart at 256×192.The reported speeds are 21 versus 17 FPS.
- Ablation Study: The splitting factor k creates a trade-off: larger k lowers quantization error but makes training more difficult, so performance first increases and then decreases.Recommended settings include k = 2 for HRNet-W32 at 128×128 and 256×192, and k = 3 or 2 for SimBa-Res50 at those resolutions.
- Ablation Study: 57.3% fewer GFLOPs are used when SimCC SimpleBaseline-Res50 removes upsampling modules while still outperforming the heatmap counterpart at 256×192.Across input resolutions, removing deconvolution layers consistently reduces computational cost.
5 Limitation and Future Work
SimCC is presented for top-down human pose estimation, while bottom-up multi-person estimation remains unresolved because multiple people create identification ambiguity.
- SimCC introduced in this paper works under the setting of top-down human pose estimation.
- Bottom-up multi-person pose estimation introduces identification ambiguity when multiple people are present.
- Future work may introduce extra embeddings to address matching between candidate coordinate x and y values.
6 Conclusion
SimCC represents keypoint localization as independent horizontal and vertical classification tasks using two one-dimensional coordinate vectors. Experiments report advantages over heatmap representations and suggest implications for lightweight HPE models.
- SimCC represents keypoint localization as two independent classification tasks for horizontal and vertical axes.
- Joint locations are represented by two independent 1D vectors for the x- and y-coordinates.
- Experimental results show that the 2D structure might not be essential for coordinate representations to sustain superior performance.
- SimCC shows performance advantages over heatmap-based representations and may inspire lightweight model designs for HPE.