Source-linked AI summary
Back to MLP: A Simple Baseline for Human Motion Prediction
Wen Guo, Yuming Du, Xi Shen, Vincent Lepetit, Xavier Alameda-Pineda, Francesc Moreno-Noguer
TL;DR
Human motion prediction asks how to forecast future body poses from observed sequences, a task addressed by increasingly complex neural architectures. This paper proposes siMLPe, a lightweight MLP network combined with DCT, residual displacement prediction, and velocity optimization. It reports state-of-the-art performance across Human3.6M, AMASS, and 3DPW while using substantially fewer parameters.
Problem
Human motion prediction requires forecasting future pose sequences, while recent approaches use complex RNN-, GCN-, or Transformer-based architectures that are difficult to analyze and modify.
Method
siMLPe uses fully connected layers, layer normalization, and transpose operations with DCT, residual joint-displacement prediction, and velocity as an auxiliary loss.
Results
siMLPe achieves state-of-the-art performance on Human3.6M, AMASS, and 3DPW while requiring 20× to 60× fewer parameters than previous state-of-the-art approaches.
Takeaways & Limitations
The results support using a simple MLP-based network as a strong baseline for rethinking human motion prediction.
Takeaways & Limitations
RNN-based alternatives remain constrained by sequential training and inference, memory limitations for distant frames, and complicated large-parameter networks.
Abstract
from arXiv · showhide
This paper tackles the problem of human motion prediction, consisting in forecasting future body poses from historically observed sequences. State-of-the-art approaches provide good results, however, they rely on deep learning architectures of arbitrary complexity, such as Recurrent Neural Networks(RNN), Transformers or Graph Convolutional Networks(GCN), typically requiring multiple training stages and more than 2 million parameters. In this paper, we show that, after combining with a series of standard practices, such as applying Discrete Cosine Transform(DCT), predicting residual displacement of joints and optimizing velocity as an auxiliary loss, a light-weight network based on multi-layer perceptrons(MLPs) with only 0.14 million parameters can surpass the state-of-the-art performance. An exhaustive evaluation on the Human3.6M, AMASS, and 3DPW datasets shows that our method, named siMLPe, consistently outperforms all other approaches. We hope that our simple method could serve as a strong baseline for the community and allow re-thinking of the human motion prediction problem. The code is publicly available at \url{https://github.com/dulucas/siMLPe}.
1. Introduction
Human motion prediction is important for forecasting future poses, but increasingly complex RNN-, GCN-, and Transformer-based approaches motivate a simpler alternative. siMLPe combines a minimal MLP architecture with standard motion-prediction practices and achieves strong benchmark performance with far fewer parameters.
- Human motion prediction forecasts follow-up 3D body poses and supports applications including autonomous-driving safety, people tracking, and human-robot interaction.
- RNN-, GCN-, and Transformer-based methods improve results but often use complicated architectures that are difficult to analyze and modify.
- Repeating the last input pose already achieves reasonable results, while one fully connected layer predicting residuals performs better.These observations suggest that the last pose is close to future poses and that basic layers can model the task effectively.
- siMLPe uses fully connected layers, layer normalization, and transpose operations, omitting commonly used activation layers such as ReLU.Combined with DCT, residual joint displacement prediction, and velocity auxiliary loss, the network remains linear except for layer normalization.
- 20× to 60× fewer parameters accompany state-of-the-art performance on Human3.6M, AMASS, and 3DPW.On Human3.6M, the comparison reports MPJPE at 1,000 ms against network complexity.
- A single fully connected layer can achieve reasonable performance, supporting the paper’s claim that motion prediction can be modeled without explicitly fusing spatial and temporal information.
2. Related Work
Prior work progresses from classical and recurrent models to GCNs and Transformers for encoding human-motion structure. These approaches improve prediction but tend toward complicated, difficult-to-train architectures, motivating the paper’s MLP-based alternative.
- Classical methods predict simple motions but struggle with complex and long-term motion prediction.The related work includes nonlinear Markov models, Gaussian Process dynamical models, and Restricted Boltzmann Machines.
- RNN-based methods model temporal structure but are difficult to parallelize, memory-limited for distant frames, and still complicated with many parameters.
- GCN-based methods represent poses as graphs to encode joint connectivity and commonly combine GCN blocks with nonlinear activation, normalization, and DCT-based temporal encoding.
- Transformer-based methods use attention to model temporal dependencies and, in some cases, pairwise joint relations, progressive decoding, and motion-pattern dictionaries.
- Across recent approaches, improved results accompany architectures that are increasingly complicated and difficult to train.The paper therefore returns to a simple MLP-based network.
3. Our Approach: SIMLPE
siMLPe formulates motion prediction as forecasting future 3D poses from observed sequences, using DCT-based temporal encoding and a simple MLP architecture. It predicts residual motion, combines reconstruction and velocity losses, and supports evaluation across multiple benchmarks.
- Problem formulation: Human motion prediction uses past 3D pose sequences to forecast N future pose frames.Each pose is represented by joint coordinates, with C = 3 × K dimensions for K joints.
- Temporal encoding: DCT encodes temporal information before the network, while IDCT converts network outputs back to the original pose representation.The transformed input is D(x1:T) = Dx1:T, and the prediction is recovered with the inverse transform.
- Network architecture: The network uses fully connected layers, transpose operations, and layer normalization, with temporal blocks merging information across frames.The first fully connected layer operates on the spatial dimension, while subsequent blocks operate on the temporal dimension.
- Residual prediction: Residual prediction estimates future poses relative to the last observed pose rather than predicting absolute poses from scratch.The paper states that this design eases learning and improves performance.
- Evaluation: siMLPe is evaluated on Human3.6M, AMASS, and 3DPW using reported frame-wise and average error protocols.The supplied evaluation descriptions include action-wise Human3.6M results, AMASS-to-3DPW testing, and average-error comparisons under alternative protocols.
- Training objective: The objective combines pose reconstruction loss and velocity loss to match predicted poses and their temporal differences to ground truth.The reconstruction term compares predicted and ground-truth motion with an L2 norm, while the velocity term compares their frame-to-frame differences.
4. Experiments
Experiments evaluate siMLPe across Human3.6M, AMASS, and 3DPW using standardized protocols, quantitative comparisons, qualitative examples, and ablations. The lightweight model achieves strong benchmark performance, while ablations identify temporal fusion, layer normalization, DCT, residual displacement, and velocity loss as important design choices.
- Quantitative results: siMLPe outperforms previous methods on every Human3.6M prediction frame with far fewer parameters.The comparison uses the same testing protocol for all methods and predicts 0–1000ms results with a single model.
- Quantitative results: siMLPe performs consistently better on long-term prediction across AMASS and 3DPW, with a parameter size of approximately 4% of Hisrep.The AMASS-to-3DPW protocol involves a larger train–test difference than Human3.6M, making generalization more challenging.
- Qualitative results: Qualitative predictions closely match ground truth in the short term and globally fit it in the long term, while error increases with prediction horizon.The paper attributes long-horizon error accumulation to autoregressive prediction and notes that motion uncertainty grows with time.
- Ablation study: 48 MLP blocks give the best ablation performance, although 2 blocks already achieve good performance with 0.014M parameters.The ablation is conducted on Human3.6M.
- Ablation study: Temporal feature fusion and layer normalization are vital, DCT provides a slight improvement, and velocity loss improves long-term prediction without changing short-term performance.Operating only spatially degrades results, while temporal-only operation remains comparable; front-back flip augmentation slightly improves performance, and direct absolute-pose prediction drops dramatically relative to residual displacement.
5. Conclusion
The conclusion presents siMLPe as a simple yet effective human motion prediction network built from fully connected layers, layer normalization, and transpose operations. Despite using far fewer parameters, it achieves state-of-the-art performance across benchmarks, with temporal information fusion highlighted as important.
- Conclusion: siMLPe uses fully connected layers, layer normalization, and transpose operations, with layer normalization as its only non-linear operation.The network is presented as a simple-yet-effective architecture for human motion prediction.
- Conclusion: siMLPe achieves state-of-the-art performance on various benchmarks while using much fewer parameters than competing methods.The conclusion also highlights temporal information fusion as an important design choice in the ablation study.