PaperlayerView exact revision on arXiv

Source-linked AI summary

LocoWM: High-Precision Locomotion through World-Model-Guided Residual Adaptation

Zijie Zhao, Shengqian Chen, Xiaoxu Wang, Han Jiang, Yuanheng Zhu, Dongbin Zhao

arXiv:2609.39179v1cs.RO

TL;DR

High-precision locomotion must track motion commands while regulating task-relevant physical states, but end-to-end and reactive residual approaches can leave precision objectives underoptimized or address deviations only after they appear. LocoWM predicts future task substates to guide additive residual corrections, and experiments report higher task success than end-to-end and reactive residual baselines, with up to 29.6 percentage points improvement over current-state reconstruction.

  • Problem

    High-precision locomotion must track motion commands while regulating task-relevant physical states, yet joint end-to-end optimization can leave precision objectives insufficiently optimized and reactive residual control waits for observable deviations.

    Source: §1 · pp. 1–2

  • Method

    LocoWM uses a base policy and an action-conditioned world model to predict future task substates, then trains a residual adapter to use those forecasts for additive action corrections.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    Experiments on Go2-W report higher task success than end-to-end and reactive residual baselines, with up to 29.6 percentage points higher success than current-state reconstruction across evaluated terrains.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Terrain leveling, acceleration compensation, and push recovery demonstrations show that LocoWM transfers its precision behaviors to hardware without task-specific retraining.

    Source: §1 · p. 2, §4.5 · p. 9

    The main experiments center on the Go2-W wheeled-legged platform and three predefined task settings; the broader Unitree G1 humanoid transport analysis is simulation-based.

    Source: §5 · p. 9

Abstract

from arXiv · show

High-precision locomotion combines motion-command tracking with precise regulation of task-relevant physical states, enabling robots to interact reliably with their surroundings during motion. Joint end-to-end optimization can leave precision objectives insufficiently optimized, while reactive residual control adjusts actions only after deviations become observable. We present \textbf{LocoWM}, a world-model-guided preactive residual adaptation framework for high-precision locomotion. A base policy provides command-following locomotion, while an action-conditioned world model predicts a sequence of future physical states from proprioceptive history and the proposed base action. A residual adapter conditions on this predicted sequence to generate additive action corrections that compensate for anticipated deviations. Two-stage training first learns locomotion and action-conditioned dynamics, then freezes both modules while training the adapter, separating locomotion acquisition from precision adaptation. Experiments spanning terrain leveling, acceleration compensation, and push recovery demonstrate improved control precision and disturbance robustness over end-to-end and reactive residual baselines. Demos and code are available at: https://zhaozijie2022.github.io/LocoWM

1 INTRODUCTION

High-precision locomotion must combine motion-command tracking with regulation of task-relevant physical states, yet end-to-end and reactive approaches leave a gap in precision control. LocoWM addresses this with forecasts of future task substates to guide residual corrections, and reports higher task success than end-to-end and reactive residual baselines.

Motivation

  • High-precision locomotion combines motion-command following with precise regulation of task-relevant body poses or physical states.Examples include stabilizing end effectors while walking and coordinating body orientation with acceleration.Source: §1 · p. 1
  • Joint end-to-end optimization can produce competent locomotion while leaving task-specific precision objectives insufficiently optimized.Separating precision learning from locomotion acquisition allows task-specific compensation for locomotion-induced disturbances.Source: §1 · p. 2
  • Observation-conditioned residual controllers react to measured deviations, whereas forecasting future task states enables corrections before deviations become observable.Analytical-model MPC also predicts future states, but its effectiveness depends on model accuracy and computational resources.Source: §1 · p. 2

Background

  • Learned interaction dynamics can provide forecasts for preactive control without relying on an analytical model.Recent world models learn latent transition dynamics and predict future states for control.Source: §1 · p. 2

Approach

  • LocoWM predicts a short future task-substate sequence from a proposed base action and proprioceptive history, then uses the forecast with current observations to correct anticipated deviations.The paper evaluates terrain leveling, acceleration compensation, and push recovery on a Go2-W wheeled-legged robot.Source: §1 · p. 2

Experiments

  • Across evaluated conditions, LocoWM improves success by up to 29.6 percentage points over the current-state reconstruction baseline, with maximum reductions of 91.8% in roll RMS error and 83.0% in peak vertical acceleration.Source: §1 · p. 2

2 RELATED WORK

Prior work addresses locomotion precision through residual control, analytical-model prediction, and several uses of world models. LocoWM uses learned action-conditioned future task states to guide additive residual corrections without online action-sequence optimization.

High-Precision Locomotion and Residual Adaptation

  • Observation-conditioned residuals respond to current and historical measurements, while LocoWM uses learned short-horizon predictions to anticipate task-substate changes without analytical-model look-ahead.The related work notes that analytical-model look-ahead can be brittle under model mismatch and unmodeled contact or payload dynamics.Source: §2 · pp. 2–3

World Models for Embodied AI

  • World models support imagined rollouts for policy learning, online planning, and predictive representation learning; LocoWM instead conditions a residual controller on predicted task-relevant future states.Its forecast is produced in one forward pass and mapped to an additive action correction without online action-sequence optimization.Source: §2 · p. 3

3 METHODOLOGY

LocoWM combines a locomotion policy, an action-conditioned world model, and a residual adapter, using task substates to link precision objectives, predictions, and corrections. Its training separates locomotion acquisition from precision adaptation, and its forecast supports preactive corrections without online action optimization.

3 METHODOLOGY

  • The world model maps proprioceptive history and the proposed base action to a short task-substate sequence in one forward pass, which the adapter converts into an additive correction without online action optimization.The predicted sequence shifts residual control from observed deviations to anticipated ones.Source: §3 · p. 3

3.1 HIGH-PRECISION LOCOMOTION

  • A task substate is a semantically meaningful projection of full state that defines the precision objective, supplies the world-model prediction target, and conditions residual adaptation.For the tray task, the substate includes back pitch and roll, linear acceleration, and angular velocity.Source: §3.1 · p. 3
  • The tray task regulates back orientation so the mounted tray stays level during steady locomotion and tilts to absorb inertial forces under acceleration.The stated objective aligns the tray normal with effective gravity; zero angular velocity and vertical acceleration also support steady transportation.Source: §3.1 · pp. 3–4

3.2 LOCOWM: WORLD-MODEL GUIDED RESIDUAL ADAPTATION

  • Two-stage training first learns command-following locomotion and task-substate dynamics, then freezes the base policy and world model while training the precision adapter.The stages separate locomotion acquisition from precision adaptation.Source: Figure 2 · p. 4, §3.2 · pp. 4–5
  • The world model predicts future task substates from the base-policy rollout stream using deployable observation–action inputs, task-relevant physical outputs, and a single forward pass.Its training and inference interfaces match, and the prediction gradient updates only the world model rather than the base policy.Source: §3.2 · pp. 4–5
  • At each control step, predicted future task-substate evolution informs an action correction before the resulting deviation becomes observable.The forecast is provided alongside observations and the proposed base action, without iterative rollouts or online action search.Source: §3.2 · p. 5
  • The predicted substate and observation share a physical space, while the adapter applies its correction additively in the base policy’s action space.This composition does not require a shared encoder and keeps the modules independently trainable.Source: §3.2 · p. 5

4 EXPERIMENTS

The experiments assess terrain leveling, acceleration compensation, push recovery, and zero-shot transfer to a physical robot. Across these settings, LocoWM improves task precision and disturbance robustness while retaining command following.

4 EXPERIMENTS

  • The experiment questions cover terrain leveling and tracking, acceleration-linked platform tilt, push-direction robustness, and transfer of precision behaviors to hardware..Source: §4 · p. 6

4.1 EXPERIMENTAL SETUP

  • The evaluation uses a Go2-W robot across four terrain families, comparing staged residual-control variants and measuring platform precision, command tracking, and payload retention.The setup specifies proprioception-only deployment; terrain families are slopes, bumps, a one-sided bridge, and rough terrain.Source: §4.1 · p. 6

4.2 TERRAIN LEVELING

  • LocoWM achieves the most precise leveling on all terrains, including slope pitch RMS 2.61° versus Base Policy 28.18° and bridge roll RMS 0.92°.Its |az|max is 3.97 m/s2 versus Recon’s 10.17 m/s2; reported payload retention reaches 98.1% on the bridge and 87.5% on bumps.Source: §4.2 · pp. 6–7, Table 1 · p. 7

4.3 ACCELERATION COMPENSATION

  • On flat ground, torso velocity follows ±1 m/s step commands, and pitch responds to acceleration changes with tilt-equivalent acceleration matching torso acceleration’s sign.The response is consistent with compensation for predicted in-plane acceleration.Source: Figure 3 · p. 7, §4.3 · pp. 7–8

4.4 PUSH RECOVERY

  • LocoWM maintains the highest direction-averaged payload-retention success across tested push forces and the outermost 50% success-rate envelope in nearly every direction.All methods tolerate larger lateral pushes than forward or backward pushes.Source: Figure 4 · p. 8, §4.4 · p. 8

4.5 REAL-WORLD RESULTS

  • LocoWM deploys zero-shot on the physical robot for terrain leveling, acceleration compensation, and push recovery.On the bridge and bumps, it reconfigures body and legs to keep the platform approximately level while the payload remains upright.Source: §4.5 · pp. 8–9, Figure 5 · p. 8, Figure 6 · p. 9
  • In hardware push recovery, the platform returns toward level after backward and leftward forces, while the unsecured bottle remains on the platform.The bottle tilts transiently during both recovery sequences.Source: Figure 7 · p. 9, §4.5 · p. 9

5 CONCLUSION

LocoWM combines a locomotion policy, action-conditioned world model, and residual adapter, with experiments showing improved task outcomes over end-to-end and reactive residual baselines. The paper notes that its main experiments are centered on one robot platform and three predefined task settings.

5 CONCLUSION

  • LocoWM’s experiments on Go2-W show higher task success rates than end-to-end and reactive residual baselines.Relative to current-state reconstruction, success improves by up to 29.6 percentage points across evaluated terrains, alongside lower posture errors and reduced vertical impacts.Source: §5 · p. 9
  • The framework pairs a base locomotion policy with an action-conditioned model of future task substates and an adapter that refines the proposed action.The model forecasts from proprioceptive history and the base action; the adapter uses the forecast to produce additive corrections.Source: §5 · p. 9
  • Acceleration compensation and push recovery demonstrate coordinated body tilting and stability recovery after external disturbances.Source: §5 · p. 9
  • The main experiments center on the Go2-W wheeled-legged platform and three predefined task settings; the humanoid transport analysis is simulation-based.Source: §5 · p. 9
  • Future work will explore richer task-substate representations and longer-horizon prediction for more complex physical consequences.It will also examine extending established motion skills to broader task settings while preserving command following and precision control.Source: §5 · p. 9

ETHICS STATEMENT

The work involved no human participants, animal subjects, or personal data.

ETHICS STATEMENT

  • Physical Go2-W experiments were conducted in a controlled environment with appropriate safety precautions.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The paper locates its architecture, task definitions, implementation details, evaluation design, and additional results across the main text and appendices.

REPRODUCIBILITY STATEMENT

  • Section 3.2 describes the architecture and training; Appendices A–C provide task, implementation, evaluation, and additional-results details.Source: REPRODUCIBILITY STATEMENT · p. 10

USE OF AI

The authors used AI for writing assistance and language polishing, then reviewed and verified the assisted content.

USE OF AI

  • AI assisted with writing and language polishing; the authors reviewed and verified the content and take responsibility for the paper’s claims and results.Source: USE OF AI · p. 10

A.1 TASK DEFINITION.

This section defines back-mounted payload stabilization during locomotion, details the task representation and training and evaluation setup, and illustrates terrain-leveling and acceleration-compensation behavior. The reported sequences show payload regulation across changing support conditions and platform tilt coordinated with acceleration.

A.1 TASK DEFINITION.

  • The task stabilizes a back-mounted platform by regulating orientation during commanded locomotion; its nine-dimensional substate encodes angular velocity, linear acceleration, and projected gravity.On uneven ground, leg and wheel motion compensate for changing support heights; during acceleration and braking, platform tilt counters inertial loading.Source: §A.1 · p. 14, §A.2 · p. 14, §A.3 · p. 15

B IMPLEMENTATION, TRAINING, AND EVALUATION DETAILS

B.2 WORLD MODEL PREDICTION

  • The world model predicts future task substates from proprioceptive history and base actions, masking labels that would cross episode resets or rollout-batch boundaries.Source: §B.1 · p. 15, §B.2 · pp. 15–16

B.3 REWARDS

  • Stage 1 trains the base policy and world model from shared rollouts; Stage 2 freezes both and trains the residual adapter, using projected-gravity tracking instead of keep-level rewards.Source: §B.3 · p. 16, §B.6 · p. 17

B.7 METRIC DEFINITIONS AND AGGREGATION

  • Continuous-signal metrics are computed per valid trial and averaged across trials; reported peak columns average each trial’s maximum magnitude.Source: §B.7 · p. 18

C.1.1 TERRAIN LEVELING

  • Across slopes, a one-sided bridge, bumps, and rough terrain, the payload remains upright or within indicated toppling thresholds as support conditions change.Source: §C.1.1 · p. 19

C.1.2 ACCELERATION COMPENSATION

  • Step and triangular-wave commands elicit transient and sustained pitch adjustments, with platform tilt changing alongside acceleration and returning toward level as velocity settles.Source: §C.1.2 · p. 20

C.1.3 PUSH RECOVERY

Simulated pushes show LocoWM recovering command tracking and payload orientation, while hardware demonstrations show terrain leveling, push recovery, and acceleration compensation. The displayed trials retain their payloads during terrain traversal and pushes.

C.1.3 PUSH RECOVERY

  • Velocity returns close to the 0.5 m/s command after repeated pushes, while payload pitch and roll settle near zero between disturbances.The responses show recovery of upright payload orientation alongside command tracking.Source: §C.1.3 · p. 21, Figure 14 · p. 21
  • Figure 15 maps payload-retention success by push direction and force magnitude for End-to-End, React, Recon, and LocoWM.The maps complement recovery sequences by showing retention across varying disturbance strengths and directions.Source: Figure 15 · p. 21, §C.1.3 · p. 21
  • Across eight displayed 150 N pushes, the payload remains on the platform and orientation traces stay within the toppling thresholds.Pushes are spaced at 45° intervals in the body frame.Source: §C.1.3 · p. 21, Figure 16 · p. 21

C.2 REAL-WORLD DEMONSTRATIONS

  • On a slope and wave-shaped obstacle, the platform stays approximately level and stacked blocks remain upright as wheel–leg configurations adapt to changing support.The platform orientation remains distinct from local ground inclination near the crest and obstacle exits.Source: Figure 17 · p. 22, §C.2 · p. 22
  • After backward, forward, leftward, and rightward torso pushes, the platform returns toward level and the unsecured bottle remains on it in all four sequences.The bottle tilts transiently without toppling.Source: §C.2 · p. 22, Figure 18 · p. 22
  • During acceleration and deceleration on flat ground, wheel–leg motion adjusts torso and platform tilt, which returns toward level in the later frames.The platform is rigidly attached to the torso.Source: §C.2 · pp. 22–23, Figure 19 · p. 23

D.1 EXPERIMENTAL SETUP

The section evaluates humanoid tray transport across command tracking and push scenarios, with LocoWM achieving the highest success rate in each. Visualizations show start-stop walking and tray retention during bipedal locomotion.

D.1 EXPERIMENTAL SETUP

  • The Isaac Lab setup uses a 29-actuated-joint G1 supporting a tray with an unsecured payload whose radius and height are randomized.The tray measures 25.4 × 35.2 cm; payload radius is randomized within [2.1, 4.5] cm and height within [7.5, 20] cm.Source: §D.1 · p. 23
  • Command Track has no push; Push Robot and Push Object receive 0.1-second XY forces at 5, 10, and 15 seconds, with different per-axis velocity-change bounds.The nominal bounds are 0.5 m/s for Push Robot and 0.3 m/s for Push Object.Source: §D.1 · p. 23
  • The compared executable controllers are Base-Policy, ReST-RL, and LocoWM; metrics include torso-frame velocity error, object-frame gravity tilt, and full-horizon success.Success requires the object to remain above 0.7 m with tilt below 0.7 rad throughout the horizon.Source: §D.1 · p. 23

D.2 COMPARISON RESULTS

  • 94.1%, 82.8%, and 92.6% Success Rate in Command Track, Push Robot, and Push Object exceed ReST-RL by 3.1, 1.9, and 2.4 percentage points, respectively.LocoWM also has the lowest mean TrackLinErr and Grav-XY in Command Track and Push Object, but not in Push Robot.Source: §D.2 · p. 23, Table 6 · p. 24

D.3 VISUALIZATION

  • In simulated start-stop walking, velocity rises after each start and returns close to zero after each stop, while payload pitch and roll oscillations subside after stopping.The command switches between 0 and 1 m/s across two walking intervals.Source: §D.3 · pp. 23–24, Figure 20 · p. 24
  • The G1 visualization shows standing-to-walking transitions and continued tray transport, with the payload remaining on the tray without toppling.The example applies preactive residual control to end-effector stabilization during bipedal locomotion.Source: Figure 21 · p. 24, §D.3 · p. 24
Loading 2609.39179v1…