Source-linked AI summary
Demonstration-Guided Humanoid Stand-Up on an Emulated Deformable Surface
Aniruddh Kushwah, Vyankatesh Ashtekar, Ashish Dutta
TL;DR
Humanoid robots need reliable stand-up on deformable terrain, where soft-ground deformation changes support and balance. The paper adapts hard-ground human demonstrations through two-stage reference-guided RL with explicit recovery rewards, and successfully achieves standing in simulation on both hard and soft ground.
Problem
Stand-up from a fallen posture on deformable terrain is challenging because terrain deformation alters contact forces, posture, and balance.
Method
A two-stage reference-guided RL framework adapts retargeted hard-ground human demonstrations using residual joint-position control and explicit recovery objectives.
Results
Two policies successfully completed their respective fallen-to-standing motions on hard and soft flat surfaces while preserving the demonstrated movement pattern.
Takeaways & Limitations
Reference tracking alone does not guarantee whole-body recovery, whereas the complete reward with standing-pose, uprightness, and height objectives achieved the standing configuration.
Takeaways & Limitations
The findings are limited to simulation because MuJoCo’s contact model cannot represent normal and tangential compliance or tangential movement resistance in soft terrain.
Abstract
from arXiv · showhide
This paper presents a reference-guided reinforcement learning framework to generate stand-up motion for a 29-DOF Unitree G1 humanoid on deformable soft ground, using a human demonstration recorded on hard ground. The terrain compliance is modelled using solref and solimp parameters from MuJoCo's rigid body soft-contact model. The rewards consists of (i) reference motion tracking through residual joint-position control and (ii) explicit recovery objectives such as pelvis height, torso uprightness, and the final posture. First, the policy is trained with the specified rewards considering hard ground. Next, the terrain stiffness is lowered by updating solref and the nominal surface penetration zone is expanded using solimp. Subsequent training enables the policy to adapt to the delayed support force generation due to significant surface penetration during contact-intensive phases while preserving the original demonstration pattern. The learned policy successfully completes the fallen-to-standing task in simulation, reaching the targeted pelvis height and uprightness, with a maximum contact penetration of approximately 40 mm during the process. The proposed method is demonstrated on two stand-up sequences and successfully achieves the final recovery objective on both hard and soft ground. Ablation studies show that reference tracking alone is insufficient for successful stand-up, and that explicit recovery rewards are essential.
I. INTRODUCTION
Humanoid stand-up on deformable terrain is difficult because soft-ground deformation changes contact and balance, while prior work largely focused on rigid-ground recovery or locomotion. The paper introduces a two-stage reference-guided RL method that adapts hard-ground human demonstrations to soft terrain.
- Stand-up is a contact-rich, non-cyclic task requiring reliable recovery from a lying posture for long-term humanoid autonomy.
- Most motion-imitation studies evaluate locomotion and related skills on rigid ground rather than recovery from fully fallen configurations.
- Soft-ground deformation changes contact geometry, support, wrench, energy dissipation, and balance response during humanoid motion.
- The proposed two-stage RL method adapts hard-ground human demonstrations for a 29-DOF Unitree G1 on deformable soft terrain.
- Training combines reference tracking with pelvis-height, torso-uprightness, and final-posture objectives, then continues after softening MuJoCo contact parameters.
II. METHOD
The method uses reference-guided RL to learn fallen-to-standing motions from retargeted human demonstrations, then fine-tunes each policy on a compliant surface model.
- Two retargeted BONES-SEED stand-up trajectories provide motion references for separate policies controlling a 29-DOF Unitree G1.
- Each policy is first trained from human demonstrations and then fine-tuned using MuJoCo soft-contact parameters for compliant terrain.
- PPO learns residual joint-position commands, which joint-level PD controllers execute.
A. Reinforcement Learning Formulation
The RL formulation uses phase-aware reference tracking with residual actions, state and reference errors, and task-related posture information. PPO trains the policy under a fixed-duration MDP with specified simulation and optimization settings.
- The policy operates at 60 Hz while selecting reference frames from a 120 Hz trajectory according to elapsed simulation time.A frame-skip of 8 is used over a 2 ms integrator step.
- Each action is a bounded 29-dimensional residual adjustment to the corresponding reference joint positions.The resulting commands are clipped to actuator limits and tracked using joint-level PD control.
- Observations include simulated and reference states, tracking errors, final-standing-posture deviation, motion phase, and the previous action.
- The normalized phase ϕ_t ranges from 0 at the first reference frame to 1 at the final frame, providing temporal context.
- Training uses PPO networks with three hidden layers of 512, 256, and 128 units across 20 parallel environments.The discount factor is 0.995, GAE coefficient 0.95, clipping threshold 0.2, batch size 2048, and five optimization epochs.
B. Reference-Guided Training and Adaptation to the Compliant Terrain
The method prepares demonstration data, combines reference tracking with final-standing rewards, and adapts the policy to compliant terrain through MuJoCo soft-contact parameters and staged fine-tuning.
- Reference preparation: The reference trajectory is preconditioned by treating the pelvis as the floating base, rescaling root translations, manually correcting height, and aligning the fallen key frame.Reference joint velocities are estimated by finite differencing motion-capture data.
- Reward design: The reward combines trajectory tracking, final-standing objectives, and regularisation terms, with a phase-dependent weight shifting emphasis toward final standing.The reward components are listed in Table I, including separate tracking and final-standing rewards.
- Training adaptation: Training proceeds for approximately 70 million simulation steps before compliant-ground fine-tuning for another 20 million steps, with reduced learning rate and entropy coefficient.Pre-trained observation and reward normalisation statistics are retained during fine-tuning.
- Training adaptation: Compliant-ground fine-tuning uses progressively increasing Gaussian noise on initial joint positions and generalised velocities while disabling several other randomisations.The initial reference frame is sampled from the first six trajectory frames.
- Compliant terrain model: MuJoCo compliant contact uses solver impedance and spring-damper parameters to soften constraint enforcement as penetration increases.The contact residual and its rate of change determine the normal response, while γ(rc) controls how strongly the constraint is enforced.
- Compliant terrain model: A 0.1 s critically damped response and impedance rising from 0 to 0.95 over 0.02 m approximate compliant ground without imposing a hard penetration limit.Impedance remains approximately constant after the specified depth while the floor geometry stays planar.
- Evaluation: Episodes include the reference duration plus 2 s standing, and evaluation measures pelvis height, uprightness, motion completion, contact penetration, and snapshots.Episodes terminate on non-finite state values or pelvis heights outside [−0.1, 1.8] m.
III. RESULTS AND DISCUSSION
The reference-guided policy completed fallen-to-standing recovery on hard and soft ground while largely preserving the demonstrated sequence. Explicit recovery objectives were necessary because similar joint tracking alone did not ensure whole-body recovery.
- Task completion: Both policies completed their respective retargeted stand-up motions on hard and soft flat surfaces.The evaluation used two demonstrations with different initial positions and stand-up sequences.
- Motion adaptation: The demonstrated movement sequence remained largely unaltered, with small joint-space deviations during contact-intensive phases.These deviations accommodated terrain softness while maintaining balance.
- Task completion: 0.792 m achieved pelvis height closely matched the 0.794 m target, while final uprightness reached 0.991.Together, these measures indicate the policy attained the expected standing posture rather than merely lifting the pelvis.
- Compliant contact: 39.4 mm was the largest recorded floor-contact penetration during policy inference.Active contacts shifted from multiple full-body contacts initially to feet-only contact in the final stage.
- Robustness and limitations: Robustness remained local to the tested initial-noise ranges because yaw, horizontal root displacement, external pushes, gravity variation, and friction randomisation were not included.The policy continued completing the motion up to 0.030 rad initial joint-position noise and 0.100 velocity-noise magnitude.
- Reward ablation: Tracking-only and complete-reward policies had nearly identical aggregate joint-position errors, 0.1571 rad and 0.1560 rad respectively, but substantially different task outcomes.The tracking-only policy ended at 0.059 m pelvis height and 0.543 uprightness, whereas the complete policy ended at 0.792 m and 0.991.
- Robustness and limitations: MuJoCo’s existing contact model cannot represent normal and tangential compliance or tangential movement resistance in terrains such as foam, grass, or leaves.Accordingly, the findings are limited to simulations.
IV. CONCLUSIONS
The paper presents a two-stage reference-guided RL method for adapting humanoid stand-up motions to compliant terrain. Simulation results show standing on both soft and hard ground while preserving reference-motion coordination, though improved compliance modelling and physical validation remain future work.
- Two-stage training adapts a humanoid stand-up motion from rigid to compliant terrain.
- Residual joint-position control combines reference tracking with explicit pelvis-height, uprightness, and final-posture objectives under MuJoCo soft-contact dynamics.
- Learned policies achieved a standing pose on soft and hard ground while preserving the overall coordination of their reference motions.
- Future work includes better terrain-compliance modelling and physical validation of the simulation results.