PartiGen: Part-Aware Representations for Language-Conditioned Humanoid Motion Generation

1 Introduction

• Text-to-motion references must express requested behavior while accounting for robot motion constraints and task timing.Visually plausible motion may still violate joint limits, dynamics, actuator capabilities, or contact constraints.Source: §1 · p. 2
• Part-aware attention processes anatomically masked inputs and combines part-specific features in a shared whole-body representation.The stated contribution uses fixed masks to select joint-angle channels before independent Q/K/V projections, while shared features retain global context.Source: §1 · p. 2
• Text- and history-conditioned motion primitives are generated autoregressively under forward-kinematics supervision, then a learned predictor selects a frame-level endpoint.The endpoint is selected within the complete candidate trajectory.Source: §1 · p. 2
• 81.8% of 500 simulated references pass the posture screen, and 26 of 27 motions attempted after simulation screening and manual selection are judged successful on the physical G1.The real-robot result is limited to the selected motions that were attempted.Source: §1 · p. 2

Text-to-motion generation

• Text-to-motion research spans variational, diffusion, and discrete-token models, with autoregressive generation and interactive control also addressed by DART and TextOp [Zhao et al., 2025, Xie et al., 2026].PartiGen builds on autoregressive primitive generation with anatomical input masks and EOS-based endpoint prediction.Source: §2 · p. 2

Structure-aware modeling

• Structure-aware methods use part-aware attention [Zhong et al., 2023] and skeleton-aware latent diffusion [Hong et al., 2025], while PartiGen applies fixed anatomical masks before part-specific Q/K/V projections.Its head outputs are fused into one shared stream and motion latent rather than separate part latents.Source: §2 · p. 3

Physical plausibility and robot execution

• Physical-plausibility work incorporates constraints into motion or interaction generation [Yuan et al., 2023, Xu et al., 2023]; PartiGen combines FK-supervised generation with a frozen GRIT controller [mrzuang, 2026].Kinematic supervision regularizes references during training, while execution evaluation measures tracking under the adopted controller.Source: §2 · p. 3

Temporal modeling

• PartiGen selects an endpoint at frame resolution after generating the complete candidate sequence, so EOS selection does not stop generation online.This distinguishes sequence termination from continuation during generation.Source: §2 · p. 3

3.1 Problem Formulation

• The task is to generate a future whole-body robot reference from language and motion history, determine its length, and track it using robot-state feedback.The reference is intended to express the requested behavior through kinematically consistent motion with appropriate duration.Source: §3.1 · p. 3

3.2 System Overview: Generation and Execution

• The generator has two stages: BP-MVAE learns a shared 128-dimensional motion latent, and DA-LDM generates motion codes conditioned on text and motion history.The full candidate is decoded before EOS probabilities select its endpoint; execution maps references to the frozen GRIT tracker with robot-state feedback.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3 Body-Part-Aware Motion VAE (BP-MVAE)

• BP-MVAE applies anatomical masks to raw frame features before independent Q/K/V projections, then fuses head outputs into a shared Transformer stream.Root/contact features and joint-angle increments remain shared across heads; the single latent represents whole-body motion.Source: §3.3 · pp. 4–5
• The VAE uses G1 forward-kinematics supervision and temporal smoothness regularization to constrain kinematic state and motion over time.The stated objective combines reconstruction, KL-divergence, and kinematic loss; the kinematic terms include FK-computed state differences and second-order temporal differences.Source: §3.3 · p. 5

3.4 Duration-Adaptive Latent Motion Diffusion (DA-LDM)

• DA-LDM uses latent diffusion to generate successive shared motion codes conditioned on text and history, with a Transformer denoiser predicting the clean latent.The denoiser also receives the diffusion timestep; history is supplied through cross-attention and classifier-free guidance can zero the text condition during training.Source: §3.4 · p. 5

3.5 Adaptive Motion Duration via EOS Prediction

• EOS selects a frame-level endpoint after candidate generation, allowing the reference to end between fixed-length primitive boundaries.The first frame whose EOS probability exceeds 0.9 is retained as the endpoint; if none crosses the threshold, the full 320-frame candidate is retained.Source: §3.5 · p. 6

4.1 Experimental Setup

• The setup compares G1-domain TextOp with human-domain generators retargeted to G1 and evaluates methods with a shared frozen evaluator and identical preprocessing.The evaluation uses a G1-specific text-motion alignment model because human-skeleton evaluators do not directly apply to the 23-DOF G1; reported learned-method mean ± SD summarizes three training-seed runs.Source: §4.1 · p. 6, §4.2 · p. 7

4.2 Quantitative Evaluation on HumanML3D-G1

• FID falls 16.8% (3.802 to 3.165) and R@1 rises 1.4 percentage points (0.287 to 0.301) versus TextOp; PartiGen has the lowest segment-level FID.For transitions, DART+GMR achieves lower FID and PJ, while PartiGen's diversity is closest to the dataset value.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 Compositional Stress Test

• On the compositional stress test, PartiGen has higher R@1 (0.211) than DART+GMR (0.102) and TextOp (0.129), but approximate references limit conclusions about coordinated behavior.The 100-instruction test merges independent part-level trajectories rather than using captured coordinated performances; its FID and Diversity do not directly measure coordination or structural integrity.Source: §4.3 · p. 8, Table 2 · p. 8

4.4 System Evaluation: Closed-Loop Execution

• After simulation screening and manual review, 26 of 27 attempted real-G1 motions were judged successful; in simulation, Full passed 81.8% versus Random-mask's 80.2% posture screen.The posture screen does not assess foot sliding, interpenetration, reference tracking, or semantic completion, and its pass-rate difference alone does not establish overall execution quality.Source: §4.4 · p. 8, Table 3 · p. 8

4.5 Ablation Study

• With group sizes and raw-feature projections held fixed, Full achieves higher mean R@1 (0.301 vs. 0.281) and lower mean FID (3.165 vs. 3.533) than Random-mask.The comparison tests anatomical grouping against random partitions; the table also reports foot-contact agreement.Source: §4.5 · p. 9, Table 4 · p. 9

4.6 Duration Modeling (EOS) Study

• On HumanML3D-G1, with 16-frame primitives and other settings held fixed, Full has lower duration error than Fixed-Length (3.080 vs. 12.471 frames); Oracle Length has higher R@3 and lower FID than Full.Source: §4.6 · p. 9

5 Conclusion

• PartiGen combines part-aware attention and kinematic supervision in a language-conditioned generator, with experiments supporting anatomical grouping and gains in reference quality and duration accuracy.A pretrained, frozen GRIT controller tracks generated references in MuJoCo and selected G1 hardware executions.Source: §5 · p. 10

AI use statement

• The authors used generative AI tools to polish language and improve manuscript clarity and readability, and take responsibility for the final content.Source: AI use statement · p. 10

Appendix Contents

• The appendix is organized into seven sections, from the overview and detailed architecture through evaluation, limitations, video, GRIT deployment, and extended related work.Source: Appendix Contents · p. 13

S1 Overview

• The appendix overview identifies the generator, datasets and evaluation, limitations, and frozen-GRIT tracking and deployment protocol as its contents.It also notes that the appendix includes an extended literature review.Source: S1 Overview · p. 13

• The listed internal configurations describe BP-MVAE and DA-LDM rather than the adopted GRIT policy.Source: S2 Detailed Architecture · p. 13

• Both encoder and decoder use nine-layer Transformer stacks that repeat two masked-feature attention layers followed by one standard attention layer three times.Source: S2.1 BP-MVAE Encoder and Decoder · p. 13
• Each masked Q/K/V branch reads 57-dimensional frame features, distinct from the 512-dimensional hidden residual stream.Source: S2.1 BP-MVAE Encoder and Decoder · p. 13
• The shared hidden dimension is 512, with four attention heads, GELU activations, and Layer Normalization.Source: S2.1 BP-MVAE Encoder and Decoder · p. 13
• The decoder reconstructs future motion from the shared latent and motion history.Source: S2.1 BP-MVAE Encoder and Decoder · p. 13

• The shared hidden dimension is 512 across Transformer layers, Q/K/V matrices, Layer Normalization, and residual connections.Source: S2.2 Hidden Layer Dimensions and Block… · p. 13
• The feedforward network uses a 512 → 1024 → 512 projection with GELU and dropout, a 2× expansion ratio.Source: S2.2 Hidden Layer Dimensions and Block… · p. 13
• Part-aware attention projects masked 57-dimensional frame features into the shared 512-dimensional residual stream; other attention, normalization, residual, and feedforward sublayers remain standard.Source: S2.2 Hidden Layer Dimensions and Block… · p. 13

• The four heads map to left leg, right leg, torso, and both arms; Figure S1's “Upper Body” label denotes both arms, with torso separate.The figure is schematic, and its nodes do not correspond one-to-one with individual DoFs.Source: S2.3 Multi-Head Attention… · p. 14, Figure S1 · p. 14, Table S1 · p. 14
• Fixed part masks select only joint-angle channels, while root/contact features and all joint-angle increments remain visible to every head.The 11 root/contact features and 23 increment channels are shared; the 23 joint-angle channels are gated by part.Source: S2.3 Multi-Head Attention… · pp. 14–15
• Each head independently projects masked 57-feature frames to width 128, and concatenated head outputs are projected back to the shared residual stream.Feature masks apply before projection, not to temporal attention scores; shared channels and output fusion preserve whole-body information rather than enforcing independent part representations.Source: S2.3 Multi-Head Attention… · p. 15
• Random-mask runs permute the 23 joint-angle indices into fixed groups of 6, 6, 3, and 8 while leaving shared channels and the remaining architecture unchanged.The VAE and corresponding LDM use the Full model's 8-frame training and evaluation settings; each run uses a different partition and training run.Source: S2.3 Multi-Head Attention… · p. 15
• Full has higher retrieval and contact accuracy and lower FID and PJ than each random-mask run, while Random B and C have Diversity closer to the dataset reference.The three-run SD describes partition/run combinations, not fixed-partition seed variation or a confidence interval, and does not establish statistical significance or general superiority over arbitrary partitions.Source: S2.3 Multi-Head Attention… · p. 15, Table S2 · p. 15
• The encoded motion primitive uses one shared 128-dimensional latent for root and articulated-body motion, with no dedicated body-part subvectors.Encoder attention features are fused into that latent, and the decoder reconstructs whole-body motion from it with motion history.Source: S2.3 Multi-Head Attention… · pp. 15–16
• Each frame contains 57 features: root orientation, foot contacts, local root displacement, root height, joint angles, and joint-angle increments.Table S3 gives the zero-based inclusive layout; joint angles occupy channels 11–33 and increments channels 34–56.Source: S2.3 Multi-Head Attention… · p. 16, Table S3 · p. 16
• History and future frames share feature-wise Z-score normalization using statistics estimated from sampled training clips and reused for validation.The statistics are not per-sequence normalization or an exhaustive average over every training frame.Source: S2.3 Multi-Head Attention… · p. 16

S2.4 Training Configuration and Loss Decomposition

• BP-MVAE training uses a four-stage, 200,000-step schedule and defines its kinematic objective as weighted geometric and temporal constraints scaled by λkin = 0.8.Both stages use AdamW with batch size 512 and initial learning rate 5.0 × 10−5; BP-MVAE uses cosine annealing, while DA-LDM uses linear annealing. Quantization weights are 0.0 in the final version.Source: S2.4 Training Configuration and Loss… · p. 17

S2.5 Duration-Adaptive Latent Diffusion Model (DA-LDM) Specifications

• DA-LDM uses an 8-layer transformer denoiser conditioned on text and two history frames, trained for 300,000 steps with teacher forcing and text dropout.The shared latent is 128-dimensional; history is never masked, and autoregressive rollout and history perturbation options are disabled in the reported training configuration.Source: S2.5 Duration-Adaptive Latent… · pp. 17–18

S2.6 Frame-Level EOS and Post-Generation Selection

• Frame-level EOS selects the first threshold-crossing endpoint after candidate generation, so retained motion length need not match the primitive block length.The threshold is 0.9, the crossing frame is retained, and if no crossing occurs the full candidate is kept; this post-generation operation does not save diffusion computation already incurred.Source: S2.5 Duration-Adaptive Latent… · pp. 18–19, S2.6 Frame-Level EOS and… · p. 20

S2.5 Duration-Adaptive Latent Diffusion Model (DA-LDM) Specifications

• DA-LDM predicts clean latents with a five-timestep objective and combines latent, future-feature, and weighted geometry losses; its default sampler uses all five DDPM reverse updates.The geometry terms include FK body position and quaternion components, joint angles and angular velocity, contact-masked foot position, and final-frame root yaw and XY drift.Source: S2.5 Duration-Adaptive Latent… · pp. 19–20
• Quaternion-component Huber loss can spike at representation boundaries: a 2 × 10−5 rad yaw difference yielded loss 0.4571 before sign alignment, versus approximately 1.25×10−11 after alignment.The frequency and impact of this effect in training are unquantified, and the diagnostic does not measure downstream performance.Source: S2.5 Duration-Adaptive Latent… · p. 19

S3 Dataset and Evaluation Details

• Evaluation uses motion-disjoint train and test sets and a shared frozen evaluator; the 100-instruction stress test probes coordinated lower-body/root dynamics and upper-body actions.Cross-domain baselines are retargeted to G1 with GMR, so comparisons assess complete pipelines rather than isolating architecture; learned-method error bars summarize three training seeds, not confidence intervals.Source: S3.1 Unified Retargeting Protocol… · p. 20, S3.2 Evaluator and Statistical Protocol · p. 21, S3.3 Compositional Stress Test Prompts · p. 21

S4 Failure Cases and Limitations

• PartiGen uses manually specified body-part structure and lacks independent limb controls; cross-morphology transfer is unverified, and execution results depend on the adopted frozen tracker.Evaluation is primarily limited to the Unitree G1 and selected demonstrations; generator metrics do not measure realized robot motion or isolate execution failures.Source: S4 Failure Cases and Limitations · pp. 21–22

S6.1 Component Interface and Deployment Pipeline

• Hardware examples are selected after simulation screening; execution uses pretrained frozen GRIT in closed-loop MuJoCo, followed by conservative candidate screening and manual review.The supplementary video shows selected simulation and physical-robot examples; screening is a risk-control step, not a safety guarantee.Source: S5 Supplementary Video · p. 22, S6.1 Component Interface and… · pp. 22–23

S6.2 Position Tracking Metrics

• Corrected Global MPJPE means were 308.89 mm for GT (N = 499) and 256.79 mm for PartiGen (N = 496), using a fixed per-motion XYZ pelvis offset without per-frame alignment.Tracking uses 19 body origins and execution-timeline references; each motion’s score averages body points and playback frames, and dataset summaries weight valid motions equally.Source: S6.2 Position Tracking Metrics · pp. 23–24

S6.3 Results and Statistics

• 409 of 500 PartiGen references (81.8%) passed the simulation posture screen; the 500-run evaluation sampled from 2,576 generated motions, and four ERROR records were excluded from tracking metrics.The pass rate is not a measure of instruction completion, and excluded records are not automatically classified as physical failures.Source: S6.3 Results and Statistics · pp. 24–26
• GT passed the posture screen in 427/500 requests (85.4%), versus 395/500 (79.0%) for DART+GMR and 401/500 (80.2%) for Random-mask; these outcomes do not establish a uniform execution advantage.The GT group had 499 valid records, while DART+GMR and Random-mask each had 494; the pass rates use all requests as the denominator.Source: S6.3 Results and Statistics · p. 25
• 92.8% of w/o FK requests passed the posture screen, compared with 81.8% for Full, but this threshold-based ordering does not establish broader execution quality or that FK supervision improves pass rates.The screen checks height and inclination, not reference tracking, semantic completion, or artifacts such as foot sliding and interpenetration; Full has lower all-valid Global and root-relative tracking-error means and medians.Source: Table S8 · p. 25, S6.3 Results and Statistics · pp. 25–26

S6.4 Selected Real-Robot Outcomes

• Under the adopted controller and venue conditions, 26 of 27 attempted manually selected G1 motions succeeded (96.30%); these screened outcomes are not an estimate of success across 500 simulated motions, 100 candidates, or the full test set.Two large-amplitude jumps and one running motion were not attempted because the available venue was too small.Source: S6.4 Selected Real-Robot Outcomes · pp. 26–27

S7.1 Text-to-Motion Generation

• Text-to-motion research spans variational, diffusion, and discrete-token generation, alongside unified representations and controllable priors; PartiGen focuses on how body-part information enters feature processing within a shared motion representation.The review distinguishes text-conditioned motion generators from retargeting and tracking methods, which address other stages of the generation-to-execution pipeline.Source: S7.1 Text-to-Motion Generation · p. 27

S7.2 Structure-Aware and Physically Plausible Motion Generation

• Related structure-aware work includes part-aware attention and skeleton-aware diffusion, while other studies address heterogeneous skeletons and physical plausibility; PartiGen uses fixed anatomical masks with part-specific projections and a shared motion latent.Its FK supervision is applied during generator training without simulator or controller gradients, and pretrained GRIT is used as the frozen execution component.Source: S7.2 Structure-Aware and Physically… · pp. 27–28

S7.3 Temporal Modeling in Motion Generation

• Because one instruction can admit multiple valid durations, PartiGen generates a complete candidate sequence before applying an EOS threshold to select a frame-level endpoint; it does not stop autoregressive generation online.The discussion characterizes its generator as combining part-aware feature learning, a shared 128-dimensional latent, FK-based regularization, and EOS-based duration prediction.Source: S7.3 Temporal Modeling in Motion… · p. 28