MotionMixer: MLP-based 3D Human Body Pose Forecasting

1 Introduction

• Prior-based methods struggle to forecast reliable 3D poses because of complex biomechanics, and expert knowledge does not ensure generalization to new environments and subjects.Forecasting motion also supports applications such as virtual reality, autonomous driving, and robotics.Source: §1 · p. 1
• RNN-based approaches model temporal correlations between joints, but the passage notes challenges including gradient problems, first-frame discontinuity, and stationary predictions at shorter forecast horizons.The passage also notes difficulty processing longer input sequences.Source: §1 · pp. 1–2
• MotionMixer alternates MLP operations across body poses and time steps to model spatial-temporal dependencies and capture structural and temporal relationships explicitly.The operations allow direct access to current and past information.Source: §1 · p. 2
• The paper proposes a spatial-temporal MLP for jointly modeling body-joint locations and temporal dependencies, and reports state-of-the-art short- and long-term forecasting on three large-scale datasets.The authors also identify reduced computational cost as a contribution.Source: §1 · p. 2

Recurrent-based Motion Prediction

• RNN methods use encoder-decoder structures, residual connections, or motion-context modeling, while recurrent-free backbones for human motion prediction are described as rarely studied.The cited approaches include [Fragkiadaki et al., 2015], [Martinez et al., 2017], and.Source: §2 · p. 2

Convolutional-based Motion Prediction

• CNN-based motion-prediction approaches have drawn interest for their hierarchical structure and effectiveness at capturing spatial and temporal correlations, but convolution-based approaches remain bottlenecked by quite high computational cost.[Mao et al., 2019] converted the joint trajectory to the frequency domain to handle temporal information.Source: §2 · p. 2

Attention- and MLP-based Architectures

• Attention methods model relationships among poses but incur increasing memory overhead with longer histories; MLP-Mixer replaces self-attention with MLPs, while MotionMixer targets human spatial-temporal dynamics.The passage describes attention approaches in [Mao et al., 2020] and [Aksan et al., 2020], and MLP-Mixer in [Tolstikhin et al., 2021].Source: §2 · p. 2

3.1 MotionMixer

• MotionMixer couples pose embedding and spatial-temporal mixing in repeated STMixer blocks, then uses a two-layer fully connected decoder to forecast future motion.Each block contains spatial and temporal MLPs with skip connections.Source: §3.1 · p. 3

Pose Prediction

• The decoder aggregates the mixed features into a global vector and applies an MLP to forecast future motion.The decoder projects each mixed feature to a vector of length Tf using a two-layer nonlinear feedforward network.Source: §3.1 · p. 3, Pose Prediction · p. 3

Pose Embedding

• Each observed skeleton is flattened to length K = 3×J and linearly projected to hidden dimension C to form the learnable pose embedding.The projection is a single fully connected layer.Source: Pose Embedding · p. 3

Spatial-Temporal Mixing

• Spatial mixing models dependencies among body-joint embeddings, while temporal mixing models long-range interactions across frames.The temporal operation is shared across rows representing individual joints; SE blocks reweight the influence of each time step after each mixing operation.Source: Spatial-Temporal Mixing · p. 3

3.2 Training

• Training predicts future joint displacements and adds them to the most recent pose to produce the full-body pose sequence.The stated losses include MPJPE and an angle-based loss comparing predicted joint angles with ground truth in the exponential-map representation.Source: §3.2 · p. 4

4.1 Datasets and Metrics

• Human3.6M uses 22 joints for 3D poses, AMASS uses 18, and the official 3DPW test set measures generalization from AMASS training.Evaluation reports joint-angle distance and 3D MPJPE, including particular-frame and average-over-frames measures.Source: §4.1 · pp. 4–5

4.4 Results

• MotionMixer leads average-frame Human3.6M results, performs best short-term on AMASS, and outperforms prior approaches for long-term AMASS prediction and untuned 3DPW transfer.On Human3.6M, its gain over the second-best approach ranges from 13% at 400ms to 5% at 1000ms; AMASS gains range from 36% at 80ms to 4% at 1000ms.Source: Table 1 · p. 5, §4.4 · pp. 5–6, Table 2 · p. 6

4.2 Implementation Details

• The model uses three STMixer blocks with 60 channels, dropout 0.1, and Adam training for 50 epochs.The learning rate starts at 10^-2 and decays by a factor of 0.1 every 10 epochs; batch sizes are 50 for Human3.6M and 256 for AMASS.Source: §4.2 · p. 5

4.3 Baselines

• Comparisons cover nine recurrent, convolutional, graph-convolutional, and attention-based methods at 400ms and 1000ms prediction horizons.These correspond to 10-frame short-term and 25-frame long-term forecasts.Source: §4.3 · p. 5

4.5 Ablation Studies

• Predicting pose displacement improves performance by 5mm over direct 3D-joint prediction, while removing temporal or spatial mixing raises 1000ms error by 4% or 6%, respectively.The best layer count is N = 3; stacking more layers did not empirically improve performance.Source: §4.5 · p. 6, Figure 3 · p. 6, Table 4 · p. 7
• SE blocks yield a 2mm gain across horizons, and MotionMixer reaches nearly the performance of Mao et al. with 1.4% of its parameters.It also outperforms Sofianos et al. and Dang et al. by 4% and 3%, respectively, using 40% and 0.5% of their parameters.Source: §4.5 · p. 7, Table 4 · p. 7, Table 5 · p. 7

4.6 Limitations

• Long-term motion remains uncertain: the final three predicted WalkDog frames mismatch ground truth, a failure also seen in previous methods.The paper notes that past poses can imply multiple possible futures, making long-term inter-joint and inter-frame dependencies more complex.Source: §4.6 · p. 7

• MotionMixer mixes features across spatial and temporal domains to exploit 3D human body pose dependencies.Source: §5 · p. 7
• The method improves state-of-the-art short- and long-term forecasting on three large-scale benchmark datasets.Source: §5 · p. 7
• Squeeze-and-excitation blocks calibrate the influence of each time step, while the model uses fewer parameters than current best-performing approaches.Source: §5 · p. 7