Source-linked AI summary
Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction
Tiezheng Ma, Yongwei Nie, Chengjiang Long, Qing Zhang, Guiqing Li
TL;DR
Existing motion-prediction approaches make forecasting easier by predicting between extended sequences, but the paper seeks better initial guesses for future poses. It proposes progressively refined multi-stage prediction with graph-convolutional spatiotemporal features, outperforming previous approaches across three public datasets while still requiring complete pose inputs.
Problem
Existing GCN-based approaches predict between extended sequences, motivating better initial guesses for the original future-pose prediction task.
Method
A multi-stage framework recursively smooths target sequences into intermediate targets and uses S-DGCN and T-DGCN networks to extract global spatiotemporal features at each stage.
Results
The method outperforms previous approaches by large margins on three public datasets.
Takeaways & Limitations
Progressively decomposing prediction into simpler subtasks makes each intermediate target a good guess for the next stage.
Takeaways & Limitations
The method requires pose sequences as input, whereas real applications may contain occluded poses.
Abstract
from arXiv · showhide
This paper presents a high-quality human motion prediction method that accurately predicts future human poses given observed ones. Our method is based on the observation that a good initial guess of the future poses is very helpful in improving the forecasting accuracy. This motivates us to propose a novel two-stage prediction framework, including an init-prediction network that just computes the good guess and then a formal-prediction network that predicts the target future poses based on the guess. More importantly, we extend this idea further and design a multi-stage prediction framework where each stage predicts initial guess for the next stage, which brings more performance gain. To fulfill the prediction task at each stage, we propose a network comprising Spatial Dense Graph Convolutional Networks (S-DGCN) and Temporal Dense Graph Convolutional Networks (T-DGCN). Alternatively executing the two networks helps extract spatiotemporal features over the global receptive field of the whole pose sequence. All the above design choices cooperating together make our method outperform previous approaches by large margins: 6%-7% on Human3.6M, 5%-10% on CMU-MoCap, and 13%-16% on 3DPW.
1. Introduction
Human motion prediction supports applications including intelligent security, autonomous driving, and human-robot interaction, but existing approaches face sequence-modeling and pose-structure challenges. This paper addresses these challenges with progressively improved initial guesses, multi-stage supervision, and spatial-temporal graph convolutions.
- Human motion prediction benefits intelligent security, autonomous driving, and human-robot interaction applications.
- RNN-based approaches can produce discontinuities and error accumulation, while CNNs treat inherently irregular poses as images.
- Existing GCN-based preprocessing duplicates the last observed pose and appends it to the observed sequence, making extended-sequence prediction easier than predicting future poses directly.
- The method progressively predicts better initial guesses by using recursively smoothed future sequences as intermediate targets across multiple stages.
- S-DGCN and T-DGCN extract global spatial and temporal features to perform prediction at each stage.
- Experiments report large-margin improvements over previous approaches on three public datasets.
2. Related Work
Related work spans recurrent, convolutional, graph-convolutional, and Transformer-based approaches to human motion prediction. The paper selects GCNs while extending them to model both spatial and temporal interactions.
- RNN-based methods dominate earlier human motion prediction because human motion data are serialized, but they are difficult to train and may capture spatial relationships poorly.
- CNN-based methods improve spatial feature extraction but cannot directly model interactions between every pair of joints.
- Recent methods represent poses as graphs and use GCNs to learn relationships among joints for human motion prediction.
- The proposed framework uses T stages, with each stage receiving an initial guess and the next stage using the previous stage’s predicted future output.
- Transformer approaches can compute pairwise joint relations, while this paper chooses GCNs and reports better running time and accuracy than existing Transformer-based approaches.
3. Methodology
The method progressively refines future-motion guesses across multiple stages, using recursively smoothed targets and stage outputs as intermediate guidance. Each stage uses graph convolutions to capture global spatial and temporal interactions, within an Encoder-Copy-Decoder network.
- 3.1. Multi-Stage Progressive Prediction Framework: The framework pads the observed sequence by repeating the last observed pose, then maps this padded input to the ground-truth extended sequence.The padded sequence has length L = Th + Tf.
- 3.1. Multi-Stage Progressive Prediction Framework: Each stage predicts an output from the observed poses and an initial guess, with later stages using the previous stage’s future output as their initial guess.The first stage uses repeated copies of the last observed pose as its initial guess.
- 3.1. Multi-Stage Progressive Prediction Framework: Recursively smoothed target sequences provide intermediate supervision, guiding successive stages from smoother guesses toward the final future poses.AAS smooths only the future portion while preserving the historical portion, and averages all previous points for each smoothed value.
- 3.2. Encoder-Copy-Decoder Stage Prediction Network Comprising S-DGCN and T-DGCN: S-DGCN models global spatial dependencies among joints, while T-DGCN models global temporal dependencies along joint trajectories.Both use learnable adjacency matrices; their outputs are transposed as needed to process spatial and temporal dimensions.
- 3.2. Encoder-Copy-Decoder Stage Prediction Network Comprising S-DGCN and T-DGCN: The stage predictor combines S-DGCN and T-DGCN in GCLs within an Encoder-Copy-Decoder architecture, retaining the first L decoder poses as the final result.The copy operator duplicates encoder features along the trajectory direction before decoding.
4. Experiments
Experiments evaluate the method across three motion datasets using coordinate-based MPJPE and compare accuracy, efficiency, and architectural choices. Results show strong performance across datasets, with multi-stage learning, intermediate supervision, temporal copying, and AAS contributing to effectiveness, while incomplete observations remain an open limitation.
- Comparisons with previous approaches: Our method generally outperforms previous approaches on Human3.6M in both short-term and long-term prediction, with the largest advantage at 400ms and for walking dog.The per-joint analysis reports especially higher gains on limbs, including hands and feet.
- Comparisons with previous approaches: Our method also outperforms compared approaches on CMU-MoCap and 3DPW, with a particularly significant advantage on the challenging 3DPW dataset.The reported comparisons use average prediction errors at every timestamp.
- Efficiency comparison: The model is smaller than LTD and faster than all other compared methods, although intermediate-loss and AAS computations make it slightly slower than LTD.Both models have 12 GCN blocks, while the proposed model uses latent feature dimension 16 versus LTD’s 256.
- Ablation Analysis: The full four-stage model has average prediction error 65.02, while one stage and removing intermediate supervision increase error to 67.48 and 67.07, respectively.Using ground truth for all intermediate outputs gives 66.11, supporting the contribution of multi-stage learning and intermediate supervision.
- Ablation Analysis: AAS outperforms Gaussian filters, and AAS-based multi-stage intermediate targets outperform mean-based two-stage targets.The full model with T = 4 performs better than the T = 2 comparison.
- Limitations and Future Works: The method still has room to reduce absolute prediction error and requires pose inputs that may be occluded in real applications.The authors identify incomplete observations as a direction for further study.
5. Conclusion
The paper presents a multi-stage human motion prediction framework that decomposes difficult forecasting into simpler subtasks using recursively smoothed target sequences. Its Encoder-Copy-Decoder network combines S-DGCN, T-DGCN, and a Copy operator, with experiments demonstrating effectiveness and advantages.
- The framework decomposes human motion prediction into multiple subtasks using recursively smoothed target pose sequences.The smoothest intermediate target approaches the last observed data, while each stage receives a good guess for the next stage.
- Accumulated Average Smoothing generates intermediate targets that progressively guide later prediction stages.The strategy ensures that the smoothest intermediate target approaches the last observed data.
- The Encoder-Copy-Decoder network uses S-DGCN and T-DGCN to extract spatiotemporal features effectively.The network also includes a Copy operator that enhances decoder capability.
- Experiments and analyses demonstrate the effectiveness and advantages of the proposed method.