PaperlayerView exact revision on arXiv

Source-linked AI summary

DynaRetarget: Dynamically-Feasible Retargeting using Sampling-Based Trajectory Optimization

Victor Dhedin, Ilyass Taouil, Shafeef Omar, Dian Yu, Kun Tao, Angela Dai, Majid Khadiv

arXiv:2602.06827v3cs.RO

TL;DR

Humanoid loco-manipulation retargeting must handle underactuation, complex contacts, and imperfect demonstrations while avoiding difficult exploration. DynaRetarget combines inverse kinematics with SBTO, which incrementally grows the optimization horizon before RL policy training. Across hundreds of motions, it achieves higher success than prior methods and produces trajectories that improve downstream learning and real-hardware transfer.

  • Problem

    Humanoid loco-manipulation requires handling robot and object underactuation, complex contacts, and high-quality demonstrations that are difficult to obtain through existing retargeting and RL approaches.

  • Method

    DynaRetarget combines inverse kinematic retargeting with SBTO, incrementally growing the optimization horizon to refine imperfect trajectories into dynamically feasible motions.

  • Results

    Across hundreds of motions, DynaRetarget achieves substantially higher success rates than prior methods and improves downstream RL tracking and real-hardware transfer.

  • Takeaways & Limitations

    The resulting trajectories support reliable object interaction behaviors without additional curriculum shaping and transfer successfully to real hardware.

Abstract

from arXiv · show

In this paper, we introduce DynaRetarget, a complete pipeline for retargeting human motions to humanoid control policies. The core component of DynaRetarget is a novel Sampling-Based Trajectory Optimization (SBTO) framework that refines imperfect kinematic trajectories into dynamically feasible motions. SBTO incrementally advances the optimization horizon, enabling optimization over the entire trajectory for long-horizon tasks. We validate DynaRetarget by successfully retargeting hundreds of humanoid-object demonstrations and achieving higher success rates than the state of the art. The framework also generalizes across varying object properties, such as mass, size, and geometry, using the same tracking objective. This ability to robustly retarget diverse demonstrations opens the door to generating large-scale synthetic datasets of humanoid loco-manipulation trajectories, addressing a major bottleneck in real-world data collection.

I. INTRODUCTION

DynaRetarget addresses physically inconsistent human-to-humanoid retargeting by combining inverse kinematics with sampling-based trajectory optimization and downstream RL tracking. Its full-horizon refinement improves retargeting success and supports transfer to real humanoids across diverse loco-manipulation motions.

  • Feasible loco-manipulation requires handling robot and object underactuation together with complex contact interactions.
  • Demonstrations reduce RL exploration difficulty and allow simple reward terms shared across motions after retargeting.
  • Kinematic retargeting can introduce physical and geometric inconsistencies, especially in loco-manipulation tasks.
  • DynaRetarget combines inverse kinematic retargeting with SBTO that incrementally increases the optimization horizon for long-horizon problems.
  • The method retargets imperfect kinematic demonstrations into dynamically feasible behaviors while considering the full problem horizon.
  • Hundreds of simulated motions yielded higher retargeting success rates than prior methods, with improved downstream RL learning and robust real-robot transfer.

III. METHOD

The method formulates optimal control as sampling-based trajectory optimization: sampled control sequences are simulated under system dynamics and evaluated by a cost. CEM and MPPI update sampled distributions, while interpolation knots reduce the optimization dimension.

  • Sampling-based optimization minimizes a known function through point-wise evaluations without requiring gradients, supporting non-smooth and non-convex contact-rich control.
  • Optimal control finds a control sequence minimizing cost J while satisfying dynamics xt+1 = fdyn(xt, ut).
  • State trajectories are generated by rolling out controls from the initial state with a simulator treated as a black box.
  • CEM and MPPI commonly operate receding-horizon optimization, often sampling interpolation knots instead of every control input.

B. Issues with existing SBMPC-based retargeting methods

Short-horizon SBMPC retargeting faces three linked problems: myopic planning, greedy irreversible actions, and jerky trajectories. These issues are especially damaging when references are physically inconsistent or tasks span long horizons.

  • Short-horizon limitations: MPC can behave myopically in long-horizon tasks because it repeatedly optimizes only a short horizon.Physically inconsistent references can worsen this problem, such as inaccurate contact geometry causing plans to miss the correct contacts.
  • Short-horizon limitations: SBMPC behaves greedily because executed actions cannot be re-optimized after the system is simulated forward.Early mistakes such as dropping an object are difficult to recover from because substantial deviations from the reference are penalized.
  • Short-horizon limitations: Feedback-generated SBMPC trajectories tend to be jerky, which can harm downstream RL policy training and performance.The issue affects both the motion quality and the policies trained from those motions.
  • Motivation for full-horizon optimization: Full-horizon optimization could avoid these pitfalls, but humanoid loco-manipulation makes simultaneous optimization difficult because the problem is high-dimensional.A single-shooting sampler faces increasing numbers of variables and local minima as the horizon grows.

C. Sampling-Based Trajectory Optimization

SBTO incrementally expands the optimized horizon while repeatedly refining active control variables. It uses partial roll-outs and an adaptive convergence criterion, with an optional caching optimization for long horizons.

  • Incremental optimization: SBTO first optimizes early control knots, then warm-starts progressively larger knot sets until all knots are optimized.This ordering reflects the dependence of later control variables on sufficiently optimized earlier variables.
  • Incremental optimization: The algorithm uses nested loops: an outer loop adds decision variables, while an inner loop repeatedly refines the active variables.This structure is the central mechanism for incrementally solving the long-horizon problem.
  • Partial roll-outs: Each increment performs a partial roll-out only to the last knot being optimized, while the optimization horizon grows over time.The truncated problem is equivalent to Fixed-Horizon Trajectory Optimization with parameters restricted to the active horizon.
  • Convergence criterion: SBTO increases the horizon when the maximum diagonal covariance falls below σmin, indicating sufficient convergence of the current optimization.The adaptive threshold accounts for changing convergence rates as more variables are introduced.
  • Computational improvement: SBTO skip caches converged initial trajectory segments and avoids recomputing them in later roll-outs.This can substantially reduce computation for longer-horizon problems.
  • Scope: SBTO is designed for dense-cost problems where even a short horizon provides a meaningful estimate of the optimal solution.The paper identifies retargeting tasks as satisfying this assumption.

IV. EVALUATION

The evaluation refines challenging kinematic demonstrations in simulation using MuJoCo, full collision modeling, parallel roll-outs, and cost terms covering state, task-space tracking, and collisions.

  • Evaluation setup: SBTO is evaluated on hundreds of G1 humanoid motions involving box pick-and-place, kicking, pushing, and dragging.The input trajectories contain missing contacts, penetrations, or discontinuities that require dynamic refinement.
  • Evaluation setup: The evaluation compares SBTO with state-of-the-art SBMPC and examines optimization behavior, object-property adaptation, and downstream RL policy learning.These components span performance, algorithmic properties, generalization, and transfer.
  • Implementation: MuJoCo simulations use a 0.01 s timestep, the full robot collision model, and 0.25 s knot spacing.The control sequence is represented as a PD target trajectory, and parallel roll-outs are used for SBTO.
  • Implementation: CEM updates the sampling distribution while elite-sample retention and EWMA reduce premature distribution shrinkage.The initial mean follows reference joint positions, and the experiments found that full covariance improved convergence.
  • Objective: The cost function penalizes state-position and velocity deviations, tracks torso, foot, and hand poses, and discourages undesired collisions.The corresponding cost weights are reported in Table II.

B. Performance evaluation

On 285 OmniRetarget motions, SBTO is compared with SPIDER using success, computational efficiency, and trajectory smoothness. SBTO nearly doubles success, produces smoother trajectories, and its skip variant reduces computational cost threefold.

  • Setup: 285 OmniRetarget motions are evaluated against SPIDER, a state-of-the-art SBMPC baseline.SBTO pos provides a configuration-only comparison by omitting velocity terms from the objective.
  • Results: SBTO achieves nearly twice SPIDER’s success rate and produces smoother refined trajectories.The comparison uses algorithm success rate, computational efficiency, and normalized trajectory smoothness.
  • Results: SBTO skip reduces computational cost by a factor of three, making it cheaper than SPIDER while achieving slightly better performance.The unmodified SBTO method is computationally expensive, whereas the skip variant improves its practical efficiency.
  • Computational cost: Refinement takes roughly 20 seconds per second of refined motion on a 112-core Intel(R) Xeon(R) Platinum 8480+ CPU.The reported runtime is hardware-specific, while computational efficiency is defined using simulation steps normalized by reference duration.
  • Failure cases: Failures typically occur for poor-quality references containing sudden hand–object contact changes or abrupt object-orientation flips.This identifies reference quality as the nearest reported boundary on successful refinement.

C. Algorithm analysis

SBTO incrementally expands the optimization horizon while repeatedly refining the active controls, enabling long-horizon optimization from shorter warm starts. Its analysis shows continued refinement of early decisions and lower object-position error than fixed-horizon baselines.

  • SBTO incrementally optimizes controls, warm-starting longer-horizon problems from shorter ones to mitigate FHTO convergence and instability issues.
  • SBTO optimizes over a horizon far longer than SBMPC, addressing its inherent short-sightedness.
  • Trajectory snapshots show SBTO reducing box-position error across successive increments, unlike the compared FHTO and SPIDER trajectories.
  • The analysis tracks box-position error at t0 = 1 s while the optimization horizon grows across iterations.
  • SBTO’s optimization results are compared with baseline performance using algorithm success rate, computational efficiency, and trajectory smoothness.

1) SBTO, an incremental warm-starting process:

SBTO warm-starts the full-horizon problem through incremental optimization, refining early controls over many horizon extensions. This produces a substantially longer effective horizon and succeeds where fixed-horizon alternatives fail on the evaluated contact-rich task.

  • 1) SBTO, an incremental warm-starting process:: SBTO succeeded in all 10 runs, whereas 4.6 s FHTO systematically failed because the robot fell and failed to kick the box correctly.
  • 2) SBTO’s effective horizon:: The box-position error decreased until iteration i1 ≃340, corresponding to an effective horizon of t1 = 3.4 s.
  • 2) SBTO’s effective horizon:: Early control variables continued to be refined over approximately 10 increments, with the tracking objective evaluated over 3 s of future motion.
  • 2) SBTO’s effective horizon:: With a 1 s horizon, FHTO and SPIDER failed because final box-position error remained above 10 cm, despite apparently satisfying joint tracking.
  • 2) SBTO’s effective horizon:: The effective horizon increased with σmin while remaining almost unchanged across αΣ values in the parameter sweep.

D. Demonstration Augmentation

SBTO is evaluated for robustness to changes in manipulated-object properties using the same cost terms and optimization settings. A single demonstration transfers across varied masses, sizes, and geometries.

  • The evaluation uses one motion reference and retains the same cost terms and optimization settings across object variations.
  • SBTO successfully handled box masses from 0.1 to 8 kg and sizes from 0.2 m to 0.4 m.
  • The refinement also succeeded for a cylinder, chair, and shelf, extending beyond the original box geometry.
  • These results show that one demonstration can be refined into dynamically feasible motions across diverse object geometries and physical properties.

E. Motion Tracking using Reinforcement Learning

The resulting physically consistent trajectories are used to train RL tracking controllers with domain randomization and adaptive sampling. Across diverse motions and configurations, these trajectories support reliable object tracking and faster policy convergence than kinematic retargeting alone.

  • RL tracking policies use PPO with residual actions and additional observations of the desired and actual object motion.
  • Training randomizes object poses, velocities, pushes, friction, and mass, while omitting additional object-tracking termination conditions.
  • Evaluation averages 1024 episodes across 8 motions and diverse initial configurations, with success requiring object-pose tracking within threshold at every timestep.
  • Policies trained with SBTO trajectories reliably learn challenging behaviors, including object sliding and leg-based manipulation, without additional curriculum.
  • SBTO trajectories yield significantly faster policy convergence than kinematically retargeted data alone, without additional tuning.

V. CONCLUSIONS AND FUTURE WORK

DynaRetarget’s SBTO refines imperfect kinematic trajectories into dynamically feasible motions by incrementally growing the optimization horizon. Evaluations show stronger downstream learning and smoother, higher-success retargeting, while scalability remains a future concern.

  • Conclusions: Training tracking policies on dynamically consistent SBTO trajectories shows superior performance and sample efficiency compared with references from OmniRetarget.The comparison covers object position and orientation tracking rewards during training.
  • Conclusions: SBTO incrementally grows the optimization horizon, warm-starting full-horizon optimization while refining early decision variables as the horizon expands.This addresses long-horizon convergence challenges and the short-sightedness of short-horizon control.
  • Conclusions: SBTO achieves substantially higher success rates than a state-of-the-art SBMPC baseline while producing smoother trajectories across hundreds of motions.The evaluation compares retargeting performance and trajectory quality at scale.
  • Conclusions: SBTO-generated trajectories improve downstream RL tracking, producing more reliable object interactions without additional curriculum shaping and transferring successfully to real hardware.The result connects retargeting quality to both learning performance and hardware deployment.
  • Future Work: A future scalability direction is replacing the current multivariate Gaussian with multimodal sampling to optimize multiple candidate trajectories within one process.The proposed change could reduce computational cost per refined trajectory.
Loading 2602.06827v3…