Source-linked AI summary
GLA-GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video
Bruce X. B. Yu, Zhi Zhang, Yongxu Liu, Sheng-hua Zhong, Yan Liu, Chang Wen Chen
TL;DR
Existing 2D-to-3D lifting methods can underperform on ground-truth 2D poses, motivating a model designed for that setting. GLA-GCN combines adaptive graph-based global spatiotemporal modeling with individually connected local joint estimation, outperforming corresponding state-of-the-art methods on three datasets while remaining less effective on relatively low-quality estimated poses.
Problem
Existing pose-lifting methods often underperform when evaluated with ground-truth 2D poses, despite improvements in estimated 2D pose quality.
Method
GLA-GCN globally represents 2D pose sequences with adaptive graph convolutions and locally estimates 3D joints through an individual connected layer.
Results
GLA-GCN outperforms corresponding state-of-the-art methods with up to 3% and 17% error reductions on Human3.6M and HumanEva-I, respectively, and is reported to outperform them across three datasets.
Takeaways & Limitations
Structured global and local representations support the paper’s ground-truth 2D pose lifting approach across Human3.6M, HumanEva-I, and MPI-INF-3DHP.
Takeaways & Limitations
The method underperforms some recent methods on relatively low-quality estimated 2D poses and uses slightly more parameters than some state-of-the-art methods.
Abstract
from arXiv · showhide
3D human pose estimation has been researched for decades with promising fruits. 3D human pose lifting is one of the promising research directions toward the task where both estimated pose and ground truth pose data are used for training. Existing pose lifting works mainly focus on improving the performance of estimated pose, but they usually underperform when testing on the ground truth pose data. We observe that the performance of the estimated pose can be easily improved by preparing good quality 2D pose, such as fine-tuning the 2D pose or using advanced 2D pose detectors. As such, we concentrate on improving the 3D human pose lifting via ground truth data for the future improvement of more quality estimated pose data. Towards this goal, a simple yet effective model called Global-local Adaptive Graph Convolutional Network (GLA-GCN) is proposed in this work. Our GLA-GCN globally models the spatiotemporal structure via a graph representation and backtraces local joint features for 3D human pose estimation via individually connected layers. To validate our model design, we conduct extensive experiments on three benchmark datasets: Human3.6M, HumanEva-I, and MPI-INF-3DHP. Experimental results show that our GLA-GCN implemented with ground truth 2D poses significantly outperforms state-of-the-art methods (e.g., up to around 3%, 17%, and 14% error reductions on Human3.6M, HumanEva-I, and MPI-INF-3DHP, respectively). GitHub: https://github.com/bruceyo/GLA-GCN.
1. Introduction
The paper targets 2D-to-3D lifting that performs well on ground-truth 2D poses, proposing GLA-GCN to combine global spatiotemporal modeling with local joint estimation. It reports error reductions of up to 3% on Human3.6M and 17% on HumanEva-I against corresponding state-of-the-art methods.
- The work focuses on monocular 3D pose estimation because ordinary RGB cameras are cheaper and more widely applied than specialized motion sensors.
- Existing lifting methods often lag on ground-truth 2D poses even when estimated 2D poses improve through detector fine-tuning.
- GLA-GCN globally models pose sequences with adaptive graph convolutions and locally estimates 3D joints from the resulting representation.
- The model combines global spatiotemporal and local joint representations, using individually connected layers instead of pooled features for joint estimation.
- Up to 3% and 17% error reductions are achieved on Human3.6M and HumanEva-I, respectively, versus corresponding state-of-the-art methods.
2. Related Work
Related work places 2D-to-3D lifting among temporal convolutional, graph convolutional, and Transformer-based approaches. GLA-GCN builds on graph-based structured pose representations to retain joint-specific information during estimation.
- 2D-to-3D lifting approaches are grouped into TCN-, GCN-, and Transformer-based architectures that exploit temporal pose information.
- TCN methods use strided temporal convolutions to shrink pose sequences into embeddings, with a shared fully connected layer predicting all joint positions.
- The architecture is illustrated with AGCN blocks and an individual connected layer that uses separate 1D CNN layers for pose-joint prediction.
- GCN-based methods progress from locally connected and semantic graph models toward temporal architectures that reconstruct a 3D pose from multiple 2D frames.
- GLA-GCN uses structured joint features to locally predict the corresponding 3D pose locations.
3. Method
GLA-GCN combines adaptive global spatiotemporal graph modeling with local joint-specific estimation for lifting 2D pose sequences to 3D. Its strided architecture aggregates temporal context before individually connected layers predict each joint.
- Global Representation: GLA-GCN represents 2D pose sequences with adaptive graph convolutions to reconstruct a global 3D pose sequence.The adaptive design learns connection strengths and joint similarities beyond the physical human-pose graph.
- Global Representation: Strided AGCN modules progressively shrink temporal features from F(96, 243, 17) to F(96, 1, 17), aggregating temporal neighborhoods for central-frame estimation.Each module uses two AGCN blocks with striding in the second block and residual connections.
- Local 3D Pose Joint Estimation: The individual connected layer estimates each joint from its corresponding feature F(96, 1, 1), rather than a pooled representation of all joints.This preserves matching information between joint nodes and their feature vectors after global spatial-temporal aggregation.
- Local 3D Pose Joint Estimation: Shared and unshared individual connected layers are combined by a weighted average controlled by λ.The unshared layer allows joint-specific prediction, while the shared layer captures rules common across joints; λ=0 uses only shared prediction and λ=1 only unshared prediction.
- Training: Training proceeds in two stages: first optimizing Lglobal + Llocal, then optimizing Llocal alone.The first stage supplies global supervision, while the second stage focuses on final 3D pose estimation.
4. Experiments
Experiments evaluate GLA-GCN on three benchmark datasets using standard protocols, comparisons with state-of-the-art methods, qualitative analyses, and ablations. Results show strong performance with ground-truth 2D poses, while performance on lower-quality estimated poses remains weaker.
- Experimental Setup: Experiments use Human3.6M, HumanEva-I, and MPI-INF-3DHP with MPJPE and P-MPJPE as standard evaluation protocols.MPJPE uses root-aligned Euclidean joint error, while P-MPJPE additionally applies rigid alignment.
- Implementation Details: The experiments compare detected and ground-truth 2D poses, using CPN for Human3.6M and MRCNN for HumanEva-I alongside ground-truth inputs.The detected inputs contain 17 joints for CPN and 15 joints for MRCNN.
- Comparison with State-of-the-Art: MPJPE decreases from 32.2mm to 27.76mm on MPI-INF-3DHP versus P-STMO, representing approximately 14% error reduction.The comparison uses Protocol#1 and ground-truth 2D poses.
- Comparison with State-of-the-Art: GLA-GCN underperforms some recent methods on relatively low-quality estimated 2D poses, despite its strong results with ground-truth 2D poses.The authors contrast this behavior with methods that perform well on low-quality estimated poses but generalize less well to ground-truth poses.
- Comparison with State-of-the-Art: HumanEva-I P-MPJPE decreases from 15.4mm to 9.2mm with ground-truth 2D poses, a reported 40% reduction versus the best state-of-the-art algorithm.The paper also reports superiority on Human3.6M and HumanEva-I averaged results across evaluation protocols.
- Ablation Studies: Removing the strided design increases the feature representation from F(Cout, 1, N) to F(Cout, T, N) and reduces 3D pose estimation effectiveness.Average pooling on the temporal dimension is less effective than the strided design.
- Ablation Studies: Individual connected layers improve performance by using structured GCN features and produce more interpretable joint features than fully connected layers.The comparison links joint-specific independence to the prediction behavior of the two layer types.
- Ablation Studies: Breaking pose structure by swapping left and right limbs affects 3D pose estimation, supporting the design of the individual connected layer.The paper reports this comparison in the ablation study.
5. Conclusion
The paper presents GLA-GCN as a structured GCN-based approach for 2D-to-3D human pose lifting and reports improvements over state-of-the-art methods using ground-truth 2D poses.
- GLA-GCN outperforms corresponding state-of-the-art methods with ground-truth 2D poses on Human3.6M, HumanEva-I, and MPI-INF-3DHP.
- The model globally represents 2D pose sequences and locally estimates 3D joints through individual connected layers.
- Future work targets higher parameter efficiency and improved estimated 2D pose data through fine-tuned detectors and additional loss terms.