PaperlayerView exact revision on arXiv

Source-linked AI summary

MoRL: Reinforced Reasoning for Unified Motion Understanding and Generation

Hongpeng Wang, Zeyu Zhang, Wenhao Li, Hao Tang

arXiv:2602.14534v1cs.CV

TL;DR

Motion models remain limited in stepwise reasoning and test-time planning. MoRL unifies motion understanding and generation through supervised fine-tuning, task-specific verifiable rewards, and Chain-of-Motion inference, supported by two CoT datasets. On HumanML3D and KIT-ML, it outperforms state-of-the-art methods while improving logical consistency and perceptual realism.

  • Problem

    Existing motion models have limited fine-grained reasoning and mostly decode outputs in a single pass, constraining step-by-step motion understanding and generation.

  • Method

    MoRL combines SFT, task-specific RLVR rewards, Chain-of-Motion decoding, and MoUnd-CoT-140K and MoGen-CoT-140K for unified motion understanding and generation.

  • Results

    MoRL consistently outperforms state-of-the-art methods on HumanML3D and KIT-ML, improving logical consistency and perceptual realism.

  • Takeaways & Limitations

    Task-specific rewards and CoM provide a unified route to stronger motion reasoning and more perceptually realistic generation.

  • Takeaways & Limitations

    Rule-based rewards may require adaptation to new domains or styles, while CoM adds inference-time computation and the representation omits explicit fine-grained contact dynamics and complex human–object interactions.

Abstract

from arXiv · show

Human motion understanding and generation are crucial for vision and robotics but remain limited in reasoning capability and test-time planning. We propose MoRL, a unified multimodal motion model trained with supervised fine-tuning and reinforcement learning with verifiable rewards. Our task-specific reward design combines semantic alignment and reasoning coherence for understanding with physical plausibility and text-motion consistency for generation, improving both logical reasoning and perceptual realism. To further enhance inference, we introduce Chain-of-Motion (CoM), a test-time reasoning method that enables step-by-step planning and reflection. We also construct two large-scale CoT datasets, MoUnd-CoT-140K and MoGen-CoT-140K, to align motion sequences with reasoning traces and action descriptions. Experiments on HumanML3D and KIT-ML show that MoRL achieves significant gains over state-of-the-art baselines. Code: https://github.com/AIGeeksGroup/MoRL. Website: https://aigeeksgroup.github.io/MoRL.

1 Introduction

MoRL addresses limited stepwise reasoning and single-pass decoding in motion models by unifying understanding and generation with task-specific reinforcement learning, CoM inference, and synthetic CoT data. Experiments on HumanML3D and KIT-ML report consistent gains over state-of-the-art methods.

  • Existing motion models struggle to parse queries into fine-grained steps and typically decode outputs in a single pass.
  • MoRL unifies motion understanding and generation under a reinforcement learning framework with hierarchical post-training.The pipeline combines supervised fine-tuning and reinforcement learning with verifiable rewards.
  • Task-specific rewards target semantic alignment and reasoning coherence for understanding, plus physical plausibility and text–motion consistency for generation.
  • Chain-of-Motion enables step-by-step reasoning and reflection during inference, while MoUnd-CoT-140K and MoGen-CoT-140K align motions with reasoning traces and action descriptions.
  • Experiments on HumanML3D and KIT-ML demonstrate that MoRL consistently outperforms state-of-the-art methods.

2 Related Works

Related work spans unified motion-language models, LLM reasoning methods, and a motion-specific CoT data engine. These efforts motivate MoRL’s use of reasoning traces and reinforcement learning for bidirectional motion tasks.

  • Motion-language research has progressed from specialized models toward LLM-based frameworks unifying perception, reasoning, and text–motion alignment.
  • CoT prompting and step-level supervised fine-tuning are established approaches for improving LLM reasoning on complex multistep tasks.
  • MoRL’s data engine uses Gemini with MotionHubV2 sequences and captions to construct reasoning chains for motion understanding and generation.
  • Rule-based reinforcement learning has shown that carefully designed rewards can elicit complex reasoning without intermediate supervision.

3 Data Synthesis

MoRL constructs a unified CoT resource from MotionHubV2 by generating complementary reasoning data for motion understanding and generation. The two branches support instruction tuning across both directions.

  • The data engine uses Gemini to capture reasoning processes from MotionHubV2 motion sequences, captions, and question–answer pairs.
  • The resulting corpus has complementary Motion Understanding and Motion Generation branches covering diverse dance, performance, interaction, and daily-life motions.
  • MoUnd-CoT-140K: MoUnd-CoT-140K maps SMPL-X motion sequences to reasoning chains and concise textual action descriptions.
  • MoGen-CoT-140K: MoGen-CoT-140K reverses the direction by pairing textual action descriptions and reasoning chains with corresponding SMPL-X motion sequences.
  • Both branches are normalized into a common HumanML3D feature space and form a balanced corpus for unified instruction tuning.

4 The Proposed Method

MoRL unifies motion understanding and generation through a multimodal language-model architecture, staged post-training, task-specific reinforcement rewards, and Chain-of-Motion inference. Its motion tokenizer discretizes continuous motion for autoregressive processing, while CoM plans, evaluates, and refines reasoning-motion outputs.

  • Architecture and training: MoRL combines an MLLM backbone, modality-specific tokenizers, SFT on synthetic CoT data, and reinforcement learning for unified motion understanding and generation.The framework is initialized from Qwen3-4B-Instruct and uses dedicated text and motion tokenizers.
  • Architecture and training: The motion tokenizer converts continuous 3D motion into compact discrete tokens with a VQ-VAE-style encoder-decoder, enabling compatibility with autoregressive LLM generation.Quantized latent sequences are decoded to reconstruct the original motion, while the representation reduces sequence length.
  • Reinforcement learning: RLVR uses task-specific rewards: semantic alignment and reasoning coherence for understanding, and physical plausibility and text-motion consistency for generation.The rewards align captions and reasoning traces with motion semantics while constraining joint-angle violations, abrupt velocity changes, and cross-modal mismatch.
  • Chain-of-Motion: Chain-of-Motion introduces step-by-step planning, candidate selection, and iterative reflection to improve reasoning-based understanding and refine generated motion.Understanding traces explain causal and temporal structure, whereas generation traces outline action primitives before motion-token decoding.
  • Chain-of-Motion: MoUnd-CoT-140K and MoGen-CoT-140K provide explicit reasoning traces that align training with CoM inference.The datasets extend the SFT and RL stages by making reasoning traces available at inference time.

5 Experiments

MoRL is evaluated on HumanML3D and KIT-ML for motion understanding and text-to-motion generation, with results showing strong cross-task performance and benefits from its rewards and CoM. Ablations and controlled reward comparisons identify distinct contributions from semantic, coherence, physical, alignment, and test-time components.

  • Motion understanding: MoRL consistently improves motion-understanding metrics on HumanML3D and KIT-ML over traditional sequence models and recent LLM-based methods.On HumanML3D, CIDEr reaches 35.8 versus 33.74 for Motion Agent; on KIT-ML, MoRL achieves the best balance across precision- and semantic-oriented metrics.
  • Motion generation: MoRL improves HumanML3D RPrecision across Top-1/2/3 and achieves the lowest multimodal distance, while retaining competitive diversity and multimodality.FID is slightly higher than the best diffusion-based models, but KIT-ML results remain comparable to state-of-the-art diffusion models.
  • Qualitative analysis: Figure 1 qualitatively contrasts MotionLLM’s incorrect backflip displacement and abrupt ending with MoRL’s complete, temporally coherent motion.MoRL’s example includes correct takeoff, smooth rotation, stable landing, and natural recovery.
  • Ablation study: Progressively adding RLVR rewards and CoM to the SFT-only baseline consistently improves understanding and generation results.Removing semantic alignment harms BERTScore and CIDEr, removing reasoning coherence affects ROUGE-L and CIDEr, and removing physical plausibility worsens FID.
  • Ablation study: Removing text-motion consistency substantially lowers R-Precision, while excluding CoM causes moderate degradation across metrics.These ablations associate text-motion consistency with cross-modal alignment and CoM with test-time reasoning performance.
  • Reward comparison: On the Complex Motion Subset, MoRL consistently improves R@2 and R@3 and achieves the lowest MM Distance among the compared reward designs.The comparison holds the backbone, data, and optimization fixed while varying only the generation reward.

6 Conclusion

MoRL integrates motion understanding and generation through reinforcement learning, task-specific rewards, and Chain-of-Motion decoding. Experiments on HumanML3D and KIT-ML show improved logical consistency, perceptual realism, and performance over state-of-the-art methods.

  • MoRL integrates motion understanding and generation through reinforcement learning in a unified multimodal model.
  • Task-specific rewards and Chain-of-Motion decoding improve logical consistency and perceptual realism.
  • MoRL includes two large-scale synthetic CoT datasets for motion–language alignment.
  • Experiments on HumanML3D and KIT-ML show that MoRL outperforms state-of-the-art methods.

Limitations

The approach has limitations in reward portability, inference efficiency, and motion representation fidelity.

  • Rule-based reward design may require adaptation for new motion domains or styles.
  • Chain-of-Motion adds inference-time computation that limits real-time applicability.
  • The method uses discretized motion representations without explicitly modeling fine-grained contact dynamics or complex human–object interactions.

B More Implementation Details

The implementation uses a frozen NLI model for reasoning coherence, normalized heterogeneous rewards, and CoM inference with measured efficiency trade-offs.

  • B.1 NLI Model for the Reasoning-Coherence Reward: DeBERTa-v3-large-MNLI evaluates logical consistency between textual reasoning traces and predicted answers as the frozen fNLI model.The entailment probability is used as the coherence score.
  • B.1 NLI Model for the Reasoning-Coherence Reward: Larger NLI models provide more stable entailment probabilities, while DeBERTa-v3-XL increases reward variance and reduces motion quality.
  • B.2 Reward Normalization: Group-wise normalization centers heterogeneous rewards and controls their variance within each GRPO candidate group.This stabilizes advantage computation and reduces gradient variance.
  • B.2 Reward Normalization: Scalar weights λ1 and λ2 balance joint-limit and velocity-smoothness penalties after reward normalization.
  • C Inference Latency and Throughput of CoM: CoM evaluates multiple candidate trajectories with moderate overhead, but parallel sampling makes its cost scale sub-linearly with K × T.The efficiency comparison reports per-sample latency and batch-size-32 throughput.

D Choice of RL Optimizer

GRPO is selected over PPO and DPO because it offers the most stable and effective optimization for motion reasoning and generation.

  • GRPO provides the most stable and effective optimization for motion reasoning and generation under identical settings.
  • PPO becomes unstable with long-horizon, multi-component rewards, while DPO underperforms on structured reward components.
  • GRPO stabilizes credit assignment over long reasoning chains and supports continuous reward shaping, improving understanding and generation metrics.
  • GRPO achieves the best balance of performance, training stability, and efficiency, motivating its adoption as the default optimizer.

E User Study

The user study compares four text-to-motion methods using participant ratings and rankings focused on perceptual and semantic qualities. MoRL is presented as consistently strong across these human-centered evaluations.

  • Study Design: 20 participants evaluated motions from TM2T, AvatarGPT, Motion Agent, and MoRL on 20 HumanML3D prompts.Participants used a four-point rating scale and ranked all four methods from best to worst.
  • Findings: TM2T received lower overall ratings, while AvatarGPT and Motion Agent shifted toward higher scores.The passage attributes TM2T’s weaker ratings to limited modeling of long-term dynamics and complex text–motion relations.
  • Evaluation Criteria: The evaluation focused on physical plausibility, motion smoothness, and semantic consistency.

F More Qualitative Results

Qualitative comparisons examine extended, compositionally constrained prompts requiring temporal ordering, spatial trajectories, repeated actions, and fine-grained motion modifiers. MoRL better preserves these constraints, whereas Motion Agent often produces locally plausible but globally or semantically incomplete motion.

  • F More Qualitative Results: The qualitative prompts test semantic structure, temporal coherence, and spatial constraints over extended motion sequences.They include sequential actions, trajectory following, long-horizon repetition, and complex full-body coordination.
  • F More Qualitative Results: Motion Agent handles simple prompts plausibly but shows systematic limitations when descriptions require ordering, global planning, or fine-grained modifiers.
  • Sequential Actions: For ordered actions, Motion Agent blurs stages, whereas MoRL separates the head turn from the subsequent right-foot kick.
  • Spatial Trajectories: For long-horizon paths, Motion Agent often collapses intended trajectories into locally plausible stepping or near-stationary behavior.The reported pattern prioritizes short-term kinematic validity over long-range spatial constraints.
  • Repeated Actions: Motion Agent often produces incomplete or inconsistent repeated actions, with degraded amplitude and rhythm across three backflips.
  • Fine-Grained Modifiers: Motion Agent frequently defaults to generic walking or repetitive gestures, partially ignoring subtle directional, stylistic, and coordination constraints.
  • MoRL Comparison: MoRL plans long-term trajectories, preserves action ordering, and integrates fine-grained semantic constraints into motion execution.These qualitative results are reported consistently across Tables 7 and 8 for compositional, long-horizon, and structurally constrained prompts.
  • Study Context: The work’s datasets contain motion-capture data and action-level descriptions without personal identifiers or sensitive personal content.
Loading 2602.14534v1…