PaperlayerView exact revision on arXiv

Source-linked AI summary

HuMBLE: Human Motion-Driven Behavior Learning for Embodied Locomotion

Mike Zhang, Dongho Kang, Kevin Bergamin, Nicola Burger, Robin Deits, Jonathan Foster, Bilal Hammoud, Katie Hughes, Francesco Iacobelli, Twan Koolen, M. Eva Mungai, Zach Nobles, Shane Rozen-Levy, Jean Pierre Sleiman, Fangzhou Yu, Yunbo Zhang, Alfred Rizzi, Jessica Hodgins, Scott Kuindersma, Yeuhi Abe, Sylvain Bertrand, Farbod Farshidian

arXiv:2610.10489v1cs.RO

TL;DR

Policies distilled within human-motion datasets can struggle with steering commands that are absent or poorly represented. HuMBLE distills a natural-motion prior and fine-tunes it through multi-task reinforcement learning; evaluations across three humanoid platforms report biological gait features alongside responsive command tracking.

  • Problem

    Policies trained within the support of human-motion datasets can struggle with steering commands that are absent or poorly represented, restricting command coverage.

    Source: INTRODUCTION · p. 2, Stage II: Fine-tuning via Multi-Task RL · p. 15

  • Method

    HuMBLE distills a compact policy from a full-body reference-conditioned teacher, then uses multi-task RL to combine reference-guided imitation with goal-conditioned command tracking.

    Source: INTRODUCTION · pp. 2, 4, Prior Policy Training via… · p. 15

  • Results

    Across three humanoid platforms, HuMBLE reproduces synchronized arm swinging, knee-locking, and heel-strike transitions while mitigating the trade-off between stylistic fidelity and command responsiveness.

    Source: DISCUSSION · p. 12

  • Takeaways & Limitations

    The resulting policy maps proprioception and SE(2) velocity commands directly to joint actions without requiring reference motions at runtime.

    Source: INTRODUCTION · p. 2

    The results are limited to standard humanoid locomotion on flat terrain; broader scenarios require dedicated human-motion references and terrain observations.

    Source: DISCUSSION · p. 12

Abstract

from arXiv · show

Despite recent advances in humanoid locomotion, controllers optimized for command tracking and robustness tend to produce mechanical gaits, whereas controllers tied to human motion data often fail to generalize to commands outside the data distribution. This work introduces a learning framework that balances these competing objectives to synthesize real-time steerable, robust, and biomimetic locomotion policies from human data. Using an in-house curated locomotion dataset covering diverse speeds and directions, we first learn a natural locomotion prior policy through a teacher-student distillation process. Specifically, we train a full-body reference-conditioned policy with Reinforcement Learning (RL), then distill it into a lightweight prior policy conditioned solely on proprioception and a planar torso-velocity steering command. Next, we fine-tune the prior policy with multi-task RL to expand command coverage and robustness beyond the data distribution, pairing a goal-conditioned task that tracks arbitrary commands with a reference-guided task that tracks the human data as an explicit style regularizer. We validate our framework on three humanoid robots: the Boston Dynamics Atlas R1, Atlas D1, and Unitree G1. Experimental results demonstrate robust performance across real-world scenarios, including direct user-controlled locomotion in indoor and outdoor environments, and integration as the locomotion layer within hierarchical control stacks. Benchmarks against Tabula Rasa RL policies trained without human data and ablation studies confirm that our framework yields a lightweight, deployable policy that reconstructs coordinated whole-body behavior from a steering command, retaining the human gait characteristics while remaining robust and fully steerable.

INTRODUCTION

HuMBLE addresses the tension between human-like motion and broad, stable command tracking with a compact policy learned from human motion data. It reports natural full-body movement and command responsiveness comparable to a Tabula Rasa RL baseline, with greater behavioral naturalness.

INTRODUCTION

  • The central challenge is balancing natural gait and transitions with accurate, stable steering across commands underrepresented in human-motion data.Command-tracking and disturbance-recovery controllers can produce mechanical gaits, while data-constrained policies may lose responsiveness to sparse transitions, speeds, or directions.Source: INTRODUCTION · p. 1
  • A full-body reference-conditioned teacher is distilled into a compact policy that maps proprioception and SE(2) velocity commands directly to joint actions without runtime reference motions.The deployed policy avoids latent motion sampling and computationally intensive generative inference.Source: INTRODUCTION · p. 2
  • Hand-designed control objectives can achieve stable tracking and recovery but struggle to reproduce the coupled coordination and stylistic variability of human gait.Human-motion generative approaches model behavior distributions, but VAE-based methods can produce foot slippage and unnatural joint configurations.Source: INTRODUCTION · p. 2
  • Adversarial style rewards offer physically grounded motion learning, but min-max training can be sensitive to design choices and provides indirect fine-grained user control.Diffusion and flow-matching approaches also face onboard latency, computation, and power constraints from iterative sampling or integration.Source: INTRODUCTION · p. 2
  • HuMBLE preserves motion style while expanding command coverage by pairing short-horizon reference-guided imitation with long-horizon goal-conditioned command tracking.Goal-conditioned state initialization further stabilizes training by seeding rollouts from dynamically plausible, command-relevant states.Source: INTRODUCTION · p. 4
  • The framework distills a prior from a curated dataset spanning headings, velocities, and gait modes, then fine-tunes it for broader command coverage while preserving motion fidelity.The dataset includes small stepping, walking, sprinting, sidestepping, and turning.Source: INTRODUCTION · p. 4
  • The policy preserves natural full-body movement and generalizes to sparse or unseen steering inputs, with command responsiveness and robustness comparable to Tabula Rasa RL and greater behavioral naturalness.The evaluation spans human-motion fidelity, command tracking, latency, disturbance rejection, and sim-to-real consistency.Source: INTRODUCTION · p. 4
  • The released dataset contains approximately three hours of human locomotion capture across diverse speeds, directions, and gait modalities.The authors provide it to support reproducibility and broader research progress in humanoid locomotion.Source: INTRODUCTION · p. 4

RESULTS

Across three humanoid platforms, the results assess HuMBLE’s gait fidelity, command control, robustness, sim-to-real transfer, and real-world use. They also examine how Stage II multi-task RL, distillation, and task-allocation ratios affect motion-style preservation, command coverage, reconstruction, and tracking trade-offs.

Qualitative Gait Characteristics

  • HuMBLE shows coordinated arm swing, stance-phase knee extension, heel-strike-like contact, and heel-to-toe rolling, unlike the MPC’s stiff arm motion; similar gait features appear on Atlas D1 and Unitree G1.Compared with Tabula Rasa RL, HuMBLE shows stronger upper- and lower-body coordination and smoother momentum transfer; its gait shifts from shuffling at low speeds toward heel-strike and toe-off at higher speeds.Source: Qualitative Gait Characteristics · p. 4, Figure 2 · p. 5

Fidelity to the Motion Dataset

  • On Unitree G1, deployment MAE was 0.058 m/s torso linear velocity, 0.127 rad/s torso angular velocity, and 0.055 rad joint positions versus teacher values of 0.048 m/s, 0.055 rad/s, and 0.033 rad, respectively.Deployment errors were 0.149σv, 0.257σω, and 0.184σj relative to reference-trajectory standard deviations; the teacher has full-body reference inputs, unlike the command-conditioned deployment policy.Source: Figure 3 · p. 6, Fidelity to the Motion Dataset · p. 7

Command Coverage, Responsiveness, and Robustness

  • On Atlas R1, HuMBLE withstands pushes up to 1400 N along major principal directions, with recovery depending on locomotion direction and gait.Forward walking tolerates stronger longitudinal than lateral pushes, whereas sidestepping’s wider stance increases lateral stability at the expense of another direction.Source: Command Coverage, Responsiveness, and… · p. 7, Figure 4 · p. 8
  • HuMBLE’s command-tracking errors are comparable to Tabula Rasa RL across most evaluated commands, but are largest for underrepresented commands such as fast diagonal walking.It performs favorably near zero velocity, while Tabula Rasa typically reaches large commands faster; HuMBLE responds more smoothly and gradually at high speeds, consistent with human-data acceleration profiles.Source: Command Coverage, Responsiveness, and… · pp. 7–8, Figure 4 · p. 8

Sim-to-Real Analysis

  • Simulation and hardware tracking errors are closely matched, generally differing by no more than 0.05 m/s in linear error and 0.10 rad/s in angular error, with a slightly wider gap at aggressive turn rates.Errors increase at high-speed turning commands in both simulation and hardware, so the increase is not attributed to poor transfer.Source: Sim-to-Real Analysis · p. 9, Table 1 · p. 9

Application of Learned Locomotion Behaviors

  • Joystick deployments show stable, responsive steering with gait transitions from low-speed shuffling to nominal walking and high-speed sprinting; hierarchical demonstrations use HuMBLE for waypoint navigation and box parkour.On Unitree G1, average inference time was 1.1 ms on the Jetson AGX Orin CPU using ONNX Runtime.Source: Application of Learned Locomotion… · p. 9

Role of Teacher–Student Distillation

  • The ablation study evaluates whether teacher–student distillation improves prior-policy fidelity over direct command-conditioned RL, but this supplied part reports the comparison setup rather than its outcome.The evaluation uses simulated Unitree G1 and the same data-tracking fidelity metric as the main analysis.Source: Role of Teacher–Student Distillation · p. 10, Figure 5 · p. 10

Efficacy of Distillation

  • Relative to the HuMBLE prior, the single-stage baseline raises linear-velocity, angular-velocity, and joint-position errors by 16.36%, 54.85%, and 70.87% across three motions, and by 240.87%, 80.54%, and 186.34% for sidestepping.For sidestepping, the baseline settles near standing after a few steps rather than following the steering command.Source: Role of Teacher–Student Distillation · p. 10
  • Teacher–student distillation separates feasible full-body reference learning in simulation from student learning in the deployment observation space, yielding a more reliable command-conditioned prior.Single-stage partial-observation imitation recovers some modes but is less reliable across the motion dataset’s behavioral diversity.Source: Role of Teacher–Student Distillation · p. 11

Efficacy of the Multi-Task RL Stage II

  • Stage II combines long-horizon goal-conditioned learning for broad SE(2) command responsiveness with short-horizon reference-guided imitation to regularize natural gait patterns.The nominal training configuration divides simulation environments equally between the two task groups.Source: Efficacy of the Multi-Task RL · p. 11
  • Figure 5B shows that increasing reference-guided allocation improves motion fidelity, while increasing goal-conditioned allocation improves velocity command tracking; the 50:50 split is a practical middle ground.The allocation ratio is an interpretable deployment hyperparameter; policies were compared across five splits from 0:100 to 100:0.Source: Efficacy of the Multi-Task RL · p. 11
  • The 0:100 policy develops wiggly torsos and unnatural arm swings, whereas the 100:0 policy fails sidestepping and falls when facing sparse lateral commands.These extreme allocations illustrate the trade-off between stylistic consistency and generalization beyond the motion data.Source: Efficacy of the Multi-Task RL · p. 11

Impact of RL Fine-Tuning

  • Across the operational space, the Stage II deployment policy has lower tracking errors and no falls, while the Stage I prior degrades on sparse commands, especially lateral motion with yaw.The Stage I prior can become unstable and fall in high-velocity regimes; the fine-tuned policy follows target commands without collapsing into unnatural gaits.Source: Impact of RL Fine-Tuning · p. 11

DISCUSSION

HuMBLE combines a curated human-locomotion dataset with a two-stage pipeline that learns human movement style before expanding command coverage. Evaluations across three humanoid platforms report stylistically faithful, responsive locomotion, while the discussion identifies limits in terrain, command annotation, and architecture scalability.

DISCUSSION

  • The curated dataset spans varied steering speeds and omnidirectional headings, providing data density for learning naturalistic locomotion.The paper describes the dataset as a large-scale contribution curated for naturalistic locomotion behaviors.Source: DISCUSSION · pp. 11–12
  • The two-stage pipeline first reconstructs full-body behavior from proprioception and velocity commands, then expands command coverage while preserving style through an explicit reference-tracking reward.The explicit reward keeps the balance between style and command tracking directly interpretable.Source: DISCUSSION · pp. 11–12
  • Across three humanoid platforms, HuMBLE reproduces biological gait features and mitigates the trade-off between motion fidelity and command responsiveness in a compact deployment policy.The paper also describes stable, steerable locomotion across physical platforms and real-world scenarios.Source: DISCUSSION · p. 12
  • Evaluation is limited to standard humanoid locomotion on flat terrain; broader tasks such as stair climbing, climbing, and vaulting remain open challenges.Those extensions would require dedicated human-motion references and terrain observations at deployment.Source: DISCUSSION · p. 12
  • Steerable-policy training requires command annotations, which the authors derive from estimated performer intentions but note may not scale easily to more complex steering modes.Semantic inputs such as natural-language commands may require manual labeling or advanced captioning.Source: DISCUSSION · p. 12
  • The modest MLP captures a 528,214-frame locomotion dataset, but its scalability, capacity limits, and performance boundaries remain unassessed.The authors leave rigorous scalability analysis and extension to more diverse behaviors for future work.Source: DISCUSSION · p. 12
  • The framework is proposed as a recipe for other command-driven policies when motion data, frame-level command annotations, and a command-tracking reward are available.The discussion names sparse hand and base inputs for teleoperation as a possible next command modality.Source: DISCUSSION · p. 12

MATERIALS AND METHODS

HuMBLE builds a deployment policy by distilling a reference-conditioned teacher into a proprioception- and command-conditioned prior, then fine-tuning that prior with concurrent imitation and command-tracking tasks. The methods also detail motion-data preparation, deployment, and the constraints of velocity sensing on the G1 outdoors.

Dataset and retargeting

  • Human motion is captured at 120 Hz, processed into BVH, and retargeted through proxy and robot morphologies to create robot-specific reference trajectories.The retargeting pipeline imposes alignment and non-penetration constraints and spans walking through sprinting.Source: Dataset Collection and Command… · pp. 12–13

Dataset and command annotation

  • Each motion frame receives a filtered, acceleration-shifted SE(2) intent command; mirroring broadens coverage and enforces bilateral symmetry.The command comprises longitudinal and lateral velocities and turning rate.Source: Dataset Collection and Command… · p. 14

Dataset distribution

  • Atlas R1 command plots cover 528,214 samples; the visualization emphasizes standard walking commands rather than the dataset’s sprinting maneuvers.Scatter plots show four locomotion regimes, while histograms show marginal proportions for longitudinal velocity, lateral velocity, and turning rate.Source: Dataset Collection and Command… · p. 14, Figure 7 · p. 14

Stage I: prior-policy training

Stage II: multi-task fine-tuning

  • Stage II pairs short-horizon reference-guided imitation with long-horizon goal-conditioned tracking to expand command coverage while preserving style.The authors use concurrent environment groups because arbitrary-command tracking alone can cause style collapse.Source: Stage II: Fine-tuning via Multi-Task RL · pp. 15–16

Reference-guided task

  • Reference-guided environments retain the teacher’s reward and initialization but use deployment observations, with short episodes limiting drift from unobserved references.The short horizon focuses the reinforcement signal on local style preservation.Source: Short-horizon Reference-Guided… · p. 16

Goal-conditioned task

Training and deployment

SUPPLEMENTARY MATERIALS

The supplementary materials list supporting sections on nomenclature, motion data, reward functions, training, a baseline policy, and parkour.

Supplementary contents

Supplementary Material

The supplement describes the motion dataset and processing, fine-tuning rewards, and training procedures, including assistive control and domain randomization. It also reports zero-shot Unitree G1 box-parkour navigation with a hierarchical policy and details the training designs of its experts and the Tabula Rasa policy.

Section S2. Motion Dataset Details

  • The dataset spans five speed categories and reused dance cards covering diverse turns, trajectories, linear maneuvers, and zero-velocity pivots.Choreography included heading-aligned walking and locked-heading strategies.Source: Dataset Composition and Choreography · p. 21
  • Iterative data collection addressed policy capability gaps, while randomized spoken prompts elicited natural transitions among speeds, turns, starts, and stops.Additional takes used constraints such as restricted headings, speed limits, and directional boundaries.Source: Dataset Composition and Choreography · p. 21
  • Trajectories were mirrored and contact-aligned; a 0.8× spatially scaled dataset variant was temporally warped to preserve nominal gravitational acceleration.Contact-aligned foot segments were appended to BVH files; spatial scaling alone degraded tracking fidelity, especially for motions with frequent flight phases.Source: Post-Process · p. 21, Temporal Adjustment · p. 21

Section S4. Reward Function for Multi-Task RL Fine-tuning

Section S5. Other Training Details

  • Teacher training uses a performance-adaptive assistive wrench curriculum that probabilistically increases or decreases assistance according to episodic motion similarity.Assistance is capped below full strength to encourage exploration of the robot’s self-sustained dynamics.Source: Assistive Wrench Curriculum in Teacher… · p. 23
  • Domain randomization varies observations, perturbations, surfaces, and robot properties; added terrain roughness randomization mitigated brittle ankle behavior and stumbling on the Unitree G1.Terrain was introduced after the midpoint of teacher training to preserve initial stylistic learning on flat ground.Source: Domain Randomization · p. 24
  • Tabula Rasa training combines command tracking, regularization, and survival rewards; its extensive regularizers constrain posture, joints, feet, and actuator effort while encouraging natural leg swing.The regularization terms include foot acceleration, jerk, and slide penalties, as well as swing-air-time and foot-clearance rewards targeting a maximum swing height of 0.15 m.Source: Section S6. Tabula Rasa Policy · p. 24
  • Tabula Rasa uses privileged critic observations and a binary command-activation indicator, expands command ranges according to tracking fidelity, and randomizes initial joint position and velocity deviations within ±0.2.Its curriculum omits the reference-dependent assistive-wrench schedule; domain randomization, termination conditions, and PPO settings follow HuMBLE training configurations.Source: Section S6. Tabula Rasa Policy · p. 24

Section S7. Autonomous Box Parkour

  • A perceptive high-level policy outputs 10 Hz SE(2) velocity commands and gating weights to blend three frozen, non-perceptive experts running at 50 Hz.The experts are HuMBLE, box climb-up, and box climb-down; the velocity command is filtered, converted to pose waypoints for the climbing experts, and their joint-target outputs are averaged using normalized gating weights.Source: Section S7. Autonomous Box Parkour · pp. 24–25
  • The Unitree G1 completes long-horizon, zero-shot hardware navigation in one- and two-box environments, smoothly chaining walking, climbing up, and climbing down.The high-level policy selects and blends the behaviors autonomously.Source: Section S7. Autonomous Box Parkour · p. 25

Low-level Policies Training Details

  • Box-climbing experts use noisy SE(2) pose waypoints at a 0.5 s look-ahead, omit full-body reference inputs, and receive critic information including tracking scores and task phase.They are trained with reference-guided imitation and PPO, following HuMBLE teacher training in rewards, curriculum, randomization, and hyperparameters with minor differences.Source: Low-level Policies Training Details · p. 25

High-level Policy Training Details

  • The high-level policy trains first on equally sampled walking and climbing primitives, then adds three compound transitions sampled at half the primitive-task probability.Its observations include proprioception, an SE(2) goal pose, and a local height scan; box-climbing tasks also reward reference torso height for stylistic fidelity.Source: High-level Policy Training Details · p. 25

Related papers

Loading 2610.10489v1…