Source-linked AI summary
PriMoRetarget: Humanoid Motion Transfer Through Prioritized Motion Retargeting
Nicholas Ioannidis, Dun Yang, Max Smith, Tue Vu, Ziyu Zhang, Shayegan Omidshafiei, Xue Bin Peng
TL;DR
Interaction-rich motion retargeting must preserve contacts and spatial relationships, but conflicting weighted objectives can compromise important requirements. PriMoRetarget uses an explicit hierarchy to constrain lower-priority updates; across four datasets, it substantially reduces artifacts such as foot sliding and contact errors.
Problem
Weighted retargeting objectives can conflict, and relative weights may not preserve intended priorities without careful manual tuning.
Source: §I · p. 2Method
PriMoRetarget replaces soft objective weighting with an explicit hierarchy that constrains lower-priority updates to preserve higher-priority objectives.
Source: §I · p. 2, §III · p. 3, A. Optimizing in the Nullspace · p. 3Results
Across four datasets, PriMoRetarget substantially reduces common retargeting artifacts, including foot sliding and contact errors.
Source: §I · p. 2Takeaways & Limitations
The framework provides a general mechanism for enforcing the relative importance of motion objectives, with priorities currently specified manually.
Source: §VI · p. 7PriMoRetarget primarily preserves source-motion structure, so source artifacts can propagate and object trajectories may not adapt when poorly suited to the target embodiment.
Source: §VI · p. 7
Abstract
from arXiv · showhide
Motion retargeting is a critical preprocessing step for transferring human motion data to humanoid robots. While existing methods are effective for single-character motions, interactions involving multiple characters and objects introduce additional contacts and spatial relationships that must be preserved. Retargeting is commonly formulated as a weighted multi-objective optimization problem, where conflicting objectives can require careful tuning of relative weights and can still compromise important motion characteristics. We introduce PriMoRetarget, a prioritized motion-retargeting framework that replaces relative objective weighting with an explicit priority hierarchy. Lower-priority updates are constrained to preserve higher-priority objectives, allowing critical characteristics such as foot placement and interaction contacts to take precedence over lower-priority pose objectives. The method is evaluated on four datasets spanning single-character motion, character-object manipulation, and character-character interactions. The resulting motions better preserve task-relevant interactions and provide reference trajectories that can be tracked more reliably than those produced by widely used retargeting methods.
I. INTRODUCTION
Interaction-rich motions require preserving contacts, spatial relationships, and coordinated movements alongside body poses, while weighted objectives can fail to preserve intended priorities. PriMoRetarget replaces relative weighting with a priority hierarchy, building on work in motion retargeting and prioritized optimization.
I. INTRODUCTION
- Retargeting object and character interactions must preserve contacts, spatial relationships, and coordinated movement as well as body poses.Errors in these relationships can produce physically infeasible reference motions and increase downstream learning and control burdens.Source: §I · p. 1
- Conflicting objectives make weighted retargeting difficult to tune, and relative weights do not guarantee that higher-priority requirements are preserved.Improving pose fidelity may degrade foot placement or contact preservation, with conflicts growing as interactions add contacts and spatial relationships.Source: §I · pp. 1–2
- PriMoRetarget replaces soft objective weighting with an explicit hierarchy that restricts lower-priority updates to preserve higher-priority objectives.The framework is applied to four datasets and evaluated with kinematic metrics and as reference trajectories for reinforcement-learning policies.Source: §I · p. 2
II. RELATED WORK
- Prior retargeting work covers inverse-kinematics methods, morphology differences, and interaction representations for terrain, objects, and multiple characters.Trajectory optimization and simulation have also been used to improve dynamic feasibility, typically with an IK-based kinematic reference.Source: §II · p. 2
- Data-driven retargeting can learn mappings across morphologies and incorporate contact information, but is generally constrained by its training motions and morphologies.The paper notes that optimization-based methods do not require motion- or morphology-specific training data and have seen wider practical adoption.Source: §II · p. 2
- Adaptive reweighting and gradient-surgery methods address objective conflicts, but generally do not enforce an ordering of importance.PriMoRetarget draws on explicit hierarchies and gradient projection; prior prioritized optimization work has mainly addressed online control and model-predictive settings.Source: §II · p. 2, §I · p. 2
III. PRIORITIZED MOTION RETARGETING
Weighted sums combine retargeting losses but allow conflicts to trade off important requirements. PriMoRetarget orders objectives explicitly and restricts lower-priority optimization to preserve higher-priority objectives.
III. PRIORITIZED MOTION RETARGETING
- Weighted-sum retargeting combines loss terms, but conflicting objectives can trade off even when one requirement matters more.The weights specify relative contributions rather than guaranteeing that a higher-priority requirement is preserved.Source: §III · p. 3
- PriMoRetarget orders objectives from highest to lowest priority and minimizes each over motions that remain optimal for all higher-priority objectives.The hierarchy defines the feasible solution set successively across priority levels.Source: §III · p. 3
- At each optimization step, lower-priority updates are restricted to directions that preserve higher-priority objectives to first order.For two competing objectives, the prioritized update preserves the higher-priority objective rather than compromising between them as a weighted sum does.Source: §III · p. 3, Figure 2 · p. 3
A. Optimizing in the Nullspace
PriMoRetarget preserves higher-priority error while optimizing lower-priority objectives by restricting updates to directions that leave that error unchanged. This replaces weighted compromise with a priority-respecting update.
A. Optimizing in the Nullspace
- Preserving the higher-priority error vector prevents compensation across its components, rather than merely keeping its scalar loss unchanged.The lower-priority update is sought without changing the higher-priority error.Source: A. Optimizing in the Nullspace · p. 3
- The lower-priority gradient is projected into the higher-priority Jacobian’s nullspace, yielding an update that preserves higher-priority error to first order.The higher-priority error is linearized around the current parameters; its Jacobian’s nullspace contains directions that preserve that error to first order.Source: A. Optimizing in the Nullspace · p. 3
- Unlike a weighted sum’s compromise between competing objectives, the projected update preserves higher-priority error to first order.A damped projector is used when the higher-priority Jacobian is rank deficient or poorly conditioned.Source: A. Optimizing in the Nullspace · p. 3
B. Efficient Prioritized Optimization
Efficient prioritized optimization addresses the cost and convergence challenges of applying the hierarchy to full motion sequences. It uses per-frame structure, treats some regularizers as free objectives, and starts with a weighted-sum warmup.
B. Efficient Prioritized Optimization
- Per-frame separability makes the full-sequence projection T independent systems, reducing complexity from O(T^3m^3) to O(Tm^3).The higher-priority Jacobian is block diagonal in time, and the per-frame systems can be evaluated in parallel as a batched operation.Source: B. Efficient Prioritized Optimization · p. 3
- Each iteration projects objective gradients against higher-priority Jacobians, accumulates the projected gradients, and then adds the free-objective gradient.The algorithm appends each objective’s Jacobian to the higher-priority stack as it proceeds through priority levels.Source: B. Efficient Prioritized Optimization · p. 4
- Temporal smoothness and other regularizers are treated as free objectives, with their gradients added directly rather than imposing nullspace constraints.Putting such objectives high in the hierarchy can leave little or no nullspace for lower-priority objectives; putting them low can restrict their influence after projection.Source: B. Efficient Prioritized Optimization · p. 4
- A short uniform weighted-sum warmup moves distant initial motions closer to a suitable solution before hierarchy-constrained optimization begins.The nullspace projection can make large parameter-space moves difficult when the initial motion is far from a suitable solution.Source: B. Efficient Prioritized Optimization · p. 4
IV. EXPERIMENTAL SETUP
The experiments span single-character motion, character-object interaction, and multi-character interactions. They assess kinematic quality and reference-trajectory suitability, with selected motions deployed on real humanoids.
IV. EXPERIMENTAL SETUP
- PriMoRetarget is evaluated in settings ranging from single-character motion to multi-contact interactions involving multiple characters and objects.The experiments primarily use the Unitree G1 as the target embodiment.Source: §IV · p. 4
- Evaluation considers both kinematic quality and suitability of retargeted motions as reference trajectories for reinforcement-learning policies.Source: §IV · p. 4
- Selected retargeted motions are additionally deployed on real humanoid robots.Source: §IV · p. 4
A. Motion Data & Representation
Target-robot motion is represented by root position, root orientation, and joint configuration. The evaluation covers four datasets spanning single-character, object-interaction, assistive, and collaborative manipulation motion.
A. Motion Data & Representation
- Each target-robot frame contains root position, root orientation, and joint configuration, for d = 6 + nθ parameters.For N robots over T frames, all robot motions are jointly optimized.Source: A. Motion Data & Representation · p. 4
- User-defined keypoint and joint correspondences connect source characters to target robots, and forward kinematics maps motion parameters to keypoint positions and orientations.The mappings are defined per frame for corresponding kinematic keypoints.Source: A. Motion Data & Representation · p. 4
- The four datasets cover LAFAN1 single-character motion, OMOMO large-box manipulation, Inter-X assistive interactions, and in-house collaborative carrying and handoffs.The in-house collaborative object-manipulation dataset contains 35 minutes of motion.Source: A. Motion Data & Representation · p. 4
B. Retargeting Configuration
The configuration defines objectives for preserving motion features, scales source motions to the target embodiment or shared scene, and orders objectives so foot placement and applicable contacts take precedence.
B. Retargeting Configuration
- Interaction objectives preserve relative displacements between selected keypoint pairs, while contact objectives apply when source-motion body pairs fall below a distance threshold.Temporal smoothness, joint limits, and ground penetration are treated as free objectives.Source: B. Retargeting Configuration · p. 4
- Single-character motions use a height-based scale, while multi-character or object scenes use a shared scale to preserve spatial relationships.Object interactions use s† = 1; character–character interactions use an anchor character’s source-to-target scale.Source: B. Retargeting Configuration · p. 4
- The hierarchy prioritizes foot-position matching, then applicable object or character contacts, before remaining pose and interaction objectives.The remaining objectives use available degrees of freedom to better match the source; specialized hierarchies can also be used.Source: B. Retargeting Configuration · pp. 4–5
- The box-manipulation comparison shows PriMoRetarget preserving interactions more closely than baselines that distort them or lose box contact.Source: Figure 3 · p. 5
V. RESULTS
The evaluation compares PriMoRetarget with three retargeting baselines, measures kinematic motion quality, and tests retargeted motions as reference trajectories for reinforcement-learning tracking policies.
V. RESULTS
- The comparison includes PHC, GMR [2], and OmniRetarget [5], with each baseline applied independently to each character in multi-character retargeting.PHC uses trajectory-level keypoint fitting, GMR uses frame-wise weighted-sum inverse kinematics, and OmniRetarget uses constrained optimization based on interaction meshes.Source: §V · p. 5
- Kinematic quality is measured by penetration, foot sliding, foot floating, jitter, and preservation of source hand contacts.Where applicable, the same metrics are reported for the source motion as a reference.Source: §V · p. 5
- For RL evaluation, separate PPO policies track each motion on a simulated Unitree G1, with success requiring the full motion without robot or object deviation above 0.5 m.Across 50 motions, the evaluation reports success rate, body tracking error, and, when applicable, object tracking error.Source: §V · p. 5
A. Comparisons
PriMoRetarget better preserves higher-priority motion properties and performs strongly on downstream tracking across datasets. Comparisons with weighted-sum variants show the clearest prioritization differences in interaction settings.
A. Comparisons
- Across all datasets, PriMoRetarget policies achieve the highest success rates and lowest body tracking errors, while contact preservation is highest across interaction settings.It generally reduces foot sliding relative to baselines; retargeted motions remain relatively smooth, and some penetration or foot floating may reflect source-motion artifacts.Source: A. Comparisons · pp. 5–6
B. Effects of Prioritized Optimization
- The dense nullspace-projection solve scales cubically with sequence length, whereas the per-frame formulation remains nearly constant, largely due to GPU overhead.Source: Figure 5 · p. 6
- In interaction settings, uniform weighting worsens foot and contact preservation, while 10× weight separation recovers some properties but shifts trade-offs across priority levels.Single-character locomotion shows similar performance because objective conflicts are limited and only two priority levels are present.Source: B. Effects of Prioritized Optimization · p. 6, Table III · p. 6
- For a representative sequence, PriMoRetarget has the lowest foot-matching and contact-preservation losses, at the expense of larger lower-priority body-objective losses.The weighted formulations reduce body loss further but incur higher-priority losses.Source: B. Effects of Prioritized Optimization · p. 6, Figure 4 · p. 6
C. Ablations
The ablations examine when the warmup helps, how free objectives affect motion quality, and how per-frame projection scales with sequence length.
C. Ablations
- Without warmup, performance changes little from a suitable initialization but degrades when larger corrections are required.The effect is especially pronounced for LAFAN1 and in-house motions, where warmup helps prioritized optimization converge more effectively.Source: C. Ablations · pp. 6–7
- Placing free objectives in the final priority level limits temporal smoothness and increases jitter, while contact gains remain marginal.The Free-last variant improves contact preservation slightly in some cases.Source: C. Ablations · p. 7
- Dense projection cost grows rapidly with sequence length, whereas per-frame projection remains nearly constant over the evaluated range when GPU overhead dominates.Similar behavior is observed for LAFAN1 sequences exceeding 5,000 frames.Source: C. Ablations · p. 7
D. Generalization
PriMoRetarget transfers the same prioritized formulation across different robots, scenes, and object geometries. Its hierarchy is general, but priorities are currently specified manually, and source artifacts or unsuitable object trajectories can constrain retargeted motions.
D. Generalization
- The same priority hierarchy transfers collaborative manipulation across G1 and DR02 robots, box carrying to an H1-2 on a ramp, and OMOMO motion to Booster T1 and G1.Scaling the object shows hand contacts adapting to its geometry to preserve the high-priority contact objective rather than strictly following the source motion.Source: D. Generalization · p. 7
VI. DISCUSSION & FUTURE WORK
- PriMoRetarget provides a general mechanism for objective importance, but the priority hierarchy is currently specified manually.Automatically determining or adapting priorities is identified as future work.Source: §VI · p. 7
- Preserving source-motion structure can propagate its artifacts, while source-following object trajectories limit adaptation when poorly suited to the target embodiment.Suggested future directions include learned motion priors, automatic hierarchy adaptation, and joint robot-object trajectory optimization.Source: §VI · p. 7