PaperlayerView exact revision on arXiv

Source-linked AI summary

BeJEPA: A Goal-Conditioned Humanoid Behavior World Model via Unsupervised RL

Liyun Yan, Haoyang Li, Yang Zhang, Jialing Zhu, Zhanxiang Cao, Zihao Zhou, Keqi Zhu, Yizhi Chen, Huang Chuxuan, Shengcheng Fu, Zhong Jiangwei, Qiao Sun, Chen Zhang, Yue Gao

arXiv:2610.23127v1cs.RO

TL;DR

World-coordinate humanoid control must align posture with position and heading, while motion-centered controllers can remain displaced from the target pose. BeJEPA learns a shared representation with physical and goal-conditioned behavioral prediction. It achieves the lowest global MPJPE and root error among learned policy baselines on both datasets, and hardware trials demonstrate joint world-pose control and recovery behaviors.

  • Problem

    Motion tracking does not specify how to reach a new goal, and body-relative control can leave absolute position and heading outside the learned objective.

    Source: §1 · p. 2

  • Method

    BeJEPA uses unsupervised RL to learn shared body-and-world features, combining one-step action prediction with a goal-conditioned summary of future state features.

    Source: §1 · p. 2

  • Results

    BeJEPA has the lowest global MPJPE and root error among learned policy baselines on both datasets, while its G1 deployment demonstrates joint world-pose control, ground recovery, and closed-loop return after disturbances and goal changes.

    Source: §1 · p. 2, §4.1 · p. 7

  • Takeaways & Limitations

    The shared goal representation supports world-placement-aware motion tracking, including corrections that may sacrifice local pose matching to correct position.

    Source: §1 · p. 2, §4.1 · p. 7

Abstract

from arXiv · show

Humanoid control requires coordinating whole-body posture with position and heading in the world frame. Mainstream controllers imitate trajectories in the root frame, but struggle to regain the target world pose after displacement. We introduce BeJEPA, a goal-conditioned Behavior World Model learned through unsupervised reinforcement learning from human motion priors and robot interaction. BeJEPA learns a shared representation of body configuration and world placement through a joint-embedding predictive architecture (JEPA). A Dynamic Predictor predicts the next state embedding under an action, while a Successor Predictor summarizes a goal-conditioned policy's expected discounted future state embeddings in a single vector. During joint training, Dynamic grounds the initial representation in observed physical transitions, while Successor increasingly organizes it around long-term behavior. Their composition guides policy improvement through one physical prediction and one behavioral summary, without long imagined rollouts. In 20-second motion-tracking experiments, BeJEPA with online action refinement using Dynamic outperforms all evaluated baselines in global body-position error, reducing it on AMASS by 61.7% relative to BFM-Zero with world-pose inputs and 58.7% relative to SONIC. Physical deployment on a Unitree G1 demonstrates joint attainment of posture, world position, and heading goals, recovery from the ground, and closed-loop return after disturbances and goal changes. Project page: https://ylyem9x.github.io/BeJEPA-project.

1 Introduction

World-coordinate humanoid control must coordinate posture with position and heading, even when reaching the target requires temporarily worsening posture matching. BeJEPA addresses this with unsupervised, goal-conditioned prediction and demonstrates world-aligned control in simulation and on hardware.

1 Introduction

  • World-coordinate control may require stepping away from the requested posture before reaching it at the correct position and heading.Motion-centered controllers can reproduce posture while remaining displaced from the intended location.Source: §1 · p. 2
  • Trajectory imitation does not specify how to reach a new goal, and BFM-Zero’s body-relative features exclude absolute position and heading.Unsupervised RL can discover intermediate movements and evaluate their contribution to reaching a goal.Source: §1 · p. 2
  • BeJEPA learns a shared representation of body configuration and world placement from human motion priors and robot interaction using unsupervised reinforcement learning.Its goal-independent Dynamic Predictor models one-step action consequences, while its goal-conditioned Successor Predictor summarizes expected future state features.Source: §1 · p. 2
  • Dynamic grounds the representation in observed transitions while Successor organizes it around long-term behavior, composing immediate prediction with a future summary without long imagined rollouts.This guides movement selection even when a step temporarily worsens posture matching.Source: §1 · p. 2
  • BeJEPA’s G1 deployment demonstrates joint posture and world-pose control, ground recovery, and closed-loop return after disturbances and goal changes.In simulation, the learned actor improves global body alignment on AMASS and LAFAN.Source: §1 · p. 2

2 Related Work

Prior work studies motion imitation, unsupervised behavior learning, latent world models, and successor representations. BeJEPA combines physical transition prediction with goal-conditioned future-feature summaries for long-term control.

Humanoid behavioral foundation models

World models and latent-space control

  • Reconstruction-based world models allocate capacity to observation detail, while learned-value methods’ long-term predictions do not automatically transfer to newly specified objectives (Hafner et al., 2019b;a; 2025; Hafner et al., 2019a; Hansen et al., 2022; 2024).JEPAs predict embeddings to abstract from unpredictable observation detail (Assran et al., 2023; 2025).Source: §2 · p. 3
  • BeJEPA combines feature prediction with a directly learned, goal-conditioned summary of future state features, without extended dynamics rollouts or downstream reward supervision.Latent-dynamics planning in DINO-WM and LeWorldModel composes transitions and risks accumulating prediction error (Zhou et al., 2024; Maes et al., 2026).Source: §2 · p. 3

Successor representations and reward-free reinforcement learning

  • Successor features summarize expected discounted future state features separately from reward readout, and Forward–Backward representations use this structure for new rewards and goals (Barreto et al., 2017; Touati and Ollivier, 2021; Touati et al., 2022).BeJEPA jointly learns one-step transitions from observed actions and goal-conditioned continuation as the online policy and long-term targets evolve.Source: §2 · p. 3

3 BeJEPA: A Behavior World Model

BeJEPA combines a shared representation of body configuration and world placement with goal-conditioned behavior prediction. Its predictors guide goal-reaching and motion-tracking policies, while Dynamic also supports online alignment in space and time.

3 BeJEPA: A Behavior World Model

  • Dynamic models one-step action consequences, while Successor summarizes goal-conditioned future state features in a shared representation to guide policy improvement.Their composition evaluates an action through one physical prediction and one prediction of remaining behavior, without recursively rolling out that horizon.Source: §3 · p. 3, §3.3 · p. 5

3.1 Problem formulation

  • The control formulation represents root pose by world position and heading relative to a fixed anchor, with fixed targets for goal reaching and time-indexed references for tracking.Training combines online robot transitions with retargeted human-motion goals and a conditional motion prior.Source: §3.1 · p. 3

3.2 Goal-conditioned control with successor features

  • Goal reward is embedding similarity, and a successor feature summarizes discounted future states whose alignment with the goal evaluates a policy’s behavior.This lets an action be useful even when its immediate posture is less similar to the target, if later behavior moves closer.Source: §3.2 · pp. 3–4

3.3 Learning with Dynamics and Successor Prediction

  • Dynamic predicts the next latent from an action, while Successor predicts continuation under a goal; their Bellman composition separates the immediate action consequence from subsequent policy behavior.Dynamic learns from observed transitions and grounds action-predictable features; Successor organizes the representation around future states under the stored goal.Source: §3.3 · p. 5

3.4 Aligning Behavior in Space and Time

  • Dynamic refines tracking actions by comparing short predicted trajectories with time-indexed references, then executing the first action of the lowest-cost sequence and replanning.The optimization uses fixed network weights and can account for root-position, heading, and wrist-pose errors.Source: §3.4 · p. 6

4 Experiments

Experiments test whole-body tracking, the Dynamic–Successor split, latent-space structure, and physical control, including 20-second AMASS and LAFAN segments and deployment on a Unitree G1.

4.1 Q1: How well does BeJEPA track whole-body motion?

  • BeJEPA has the lowest global MPJPE and root error among learned baselines on both datasets; frozen MPC settings can prioritize root translation or wrist rotation without retraining.Compared with SONIC, BeJEPA improves world placement despite higher MPJPEL.Source: §4.1 · pp. 7–8, Table 1 · p. 7
  • 31.2% on AMASS and 13.4% on LAFAN were the mean relative error reductions for BeJEPA-MPC (Balanced), which cut AMASS mean signed lag from 94.8 to 21.2 ms.Figure 3 compares six normalized errors and joint-tracking lag across reference-time shifts.Source: §4.1 · p. 7, Figure 3 · p. 8

4.2 Q2: Does the Dynamic–Successor split improve world control?

  • Joint training performs best; splitting predictors improves alignment even with Dynamic’s encoder gradient stopped, while the merged predictor has large world-goal errors.Dynamic grounds features in physical transitions, while Successor organizes them for long-term control; local-horizon MPC struggles with distant goals.Source: §4.2 · p. 8, Table 2 · p. 8

4.3 Q3: What structure does BeJEPA give the latent space?

  • World x and y vary mainly along the first two within-body PCA coordinates, while the third partly separates heading with substantial overlap.The representation retains body content while distinguishing possible world placements.Source: §4.3 · p. 9, Figure 4 · p. 9
  • At a common root pose, posture families form characteristic neighborhoods; linear readouts improve for velocity and root height versus world-input BFM-Zero, while position and orientation remain comparable.Lower readout scores do not rule out nonlinear recovery of physical quantities.Source: §4.3 · p. 9, Figure 5 · p. 9

4.4 Q4: How does BeJEPA perform on the physical robot?

  • On the G1, kick recovery returns toward the standing location, ground recovery reaches the commanded standing pose, and sequential goals return to the starting floor mark after posture and heading changes.Figure 6’s gold rings mark the floor references in the kick-recovery and sequential-goal sequences.Source: §4.4 · p. 10, Figure 6 · p. 10

A.1 A distribution over future states

  • The successor measure is a distribution over states visited at a geometrically sampled future time, compressing repeated policy transitions into one feature vector.The geometric sampling probability is Pr(K = k) = (1 − γ)γ^k.Source: §A.1 · p. 13

A.2 From future-feature regression to TD-JEPA

  • Regression onto sampled future embeddings learns their discounted barycenter, while Dynamic translates Successor’s improvement direction into actor action updates.The successor feature is also the unique fixed point of the stated Bellman operator.Source: §A.2 · p. 13, §A.3 · p. 14

A.4 Why separating Dynamic and Successor improves learning

Separating Dynamic from Successor preserves the goal-conditioned control objective while letting physical transition learning be reused across goals and actions. The shared MPC refinement objective then evaluates predicted motion against task-specific geometric priorities.

A.4 Why separating Dynamic and Successor improves learning

  • The decomposition preserves action values and their maximizing actions when the predictors are exact.The composed value is recovered by substituting the action-conditioned future feature identity into the value expression.Source: §A.4 · p. 14
  • Dynamic learns immediate action effects from transitions collected under any goal, while Successor shares continuation predictions across predecessor actions reaching the same latent.Successor learns directly from observed transitions without querying a next action in the bootstrap, reducing duplicated long-term prediction across actions.Source: §A.4 · pp. 14–15
  • Even with Dynamic’s encoder gradient stopped, the split model has lower early global-tracking and fixed-goal heading errors than the merged predictor; full BeJEPA improves further.The reported comparison links the learning advantage to Dynamic also shaping the encoder.Source: §A.4 · p. 15

B.1 A shared prediction and refinement objective

  • MPC averages the two Dynamic predictors’ costs and propagates their mean latent while using the same frozen actor, History Encoder, ensemble, and geometric readouts.The History Encoder initializes the current latent from deployable observations; the state encoder supplies reference-goal embeddings.Source: §B.1 · p. 15
  • Per-step costs with γ = 0.98 score intermediate motion and destination, while latent agreement, geometric readouts, residual penalties, and smoothness shape refinement.The latent term preserves whole-body agreement, and geometric readouts make task-specific precision explicit.Source: §B.1 · p. 15

B.2 Execution priorities within the same controller

The same controller’s refinement settings shift emphasis among whole-body agreement, world translation, and wrist orientation. Reference speed and a filtered latent discrepancy further constrain when and how corrections are applied.

B.2 Execution priorities within the same controller

  • Root-xy refinement increases translation weight and prediction horizon, while wrist-orientation refinement focuses cost on the hands and restricts corrections to arm and wrist joints.Balanced refinement combines latent, root, and wrist terms; wrist refinement uses a short horizon and retains the actor’s lower-body behavior.Source: §B.2 · p. 15, Table 3 · p. 15
  • Balanced and root-xy refinement apply only when reference mean planar speed is at most 0.8 m/s; wrist refinement applies throughout.The activation rule uses the supplied reference, so motion requirements determine when refinement is applied.Source: §B.2 · p. 15
  • For root-xy refinement, anchoring predicted displacement to the observed root pose and filtering latent prediction discrepancy stabilize world coordinates and suppress rapid error fluctuations.The filter retains persistent offsets, while horizon decay limits how far the current correction is extrapolated.Source: §B.2 · p. 16

C.1 Motion data and goal relabeling

The system combines retargeted motion goals and online robot transitions, shared representation learning, and prior-based and physical-execution guidance. In convergence comparisons, full BeJEPA learns world alignment faster and with lower residual goal errors than the ablations.

C.1 Motion data and goal relabeling

  • The 2,000-clip G1 motion prior spans about 4.78 hours at 30 Hz, while robot interaction supplies state-action transitions; rollouts mix moving-reference and fixed-goal behavior.Ground starts occur with probability 0.3, and actor/critic goals are replaced with probability 0.8 using replay-state or motion-reference goals.Source: §C.1 · p. 16

C.2 Shared representation and core parameters

  • The History Encoder supplies the deployment latent, and the actor combines it with the goal and recent action history to produce 29 joint commands.A wrist head preserves fine motion information, while a root-pose head supplies position and heading readouts for MPC.Source: §C.2 · pp. 16–17

C.3 CPR and AUX guidance

  • CPR rewards behavior compatible with the motion prior, while AUX penalizes physical-execution costs; together they complement the Dynamic-Successor goal objective.AUX combines torque, action-smoothness, joint-limit, contact, foot-orientation, and slippage penalties.Source: §C.3 · pp. 17–18

D.1 Convergence of posture and world alignment

  • Full BeJEPA combines physical supervision with goal-conditioned continuation learning in the shared encoder, yielding faster world alignment and lower residual goal errors than the ablations.The split model with Dynamic alone updating the encoder generally converges better than the merged predictor or stopped-Dynamic-gradient model, but physical predictability alone leaves fixed-goal errors.Source: §D.1 · p. 18

D.2 How Dynamic and Successor shape the encoder

Dynamic initially contributes the stronger encoder-prediction signal, then Successor becomes dominant as the representation increasingly reflects long-term behavior. Their gradient directions align more during intermediate learning, before alignment and Dynamic’s relative influence decline late in training.

D.2 How Dynamic and Successor shape the encoder

  • During the first 5 million transitions, median weighted GD/GS is 2.63, then 1.02 during 5–10 million as Successor catches up.Dynamic supplies the dominant initial prediction signal, and the two contributions become comparable within the first 10 million transitions.Source: §D.2 · p. 18
  • After roughly 15 million transitions, median GD/GS falls to 0.39 during 15–30 million and 0.20 during 30–100 million, while mean gradient cosine similarity rises to 0.226 by 30–100 million.Successor increasingly organizes the representation around long-term behavior while Dynamic continues to constrain it through predictable physical change; positive alignment identifies a shared local descent direction.Source: §D.2 · p. 19
  • After 100 million transitions, median GD/GS is 0.0031 and mean cosine similarity is 0.069 as control curves approach a plateau and both prediction errors settle at low values.All logged interval-averaged cosines remain positive; the pattern is consistent with Successor dominating later refinement as behavior stabilizes.Source: §D.2 · p. 19

E.1 Acquiring position, heading, and a simple posture together

The experiments test simultaneous acquisition of displaced world position, heading, and posture, followed by holding near the goal. The approach includes a posture-dependent locomotion transient and gradual alignment without a manually specified walking-to-posing switch.

E.1 Acquiring position, heading, and a simple posture together

  • Joint error rises from zero for standing but falls for arms-out and shallow-squat goals, consistent with adopting a moving configuration before settling.Remaining exactly in the standing target would prevent the leg motion needed to walk.Source: §E.1 · p. 20
  • As the robot approaches, position and heading errors decrease while joint errors settle into a lower bounded range, without a manually specified walking-to-posing switch.The posture transitions gradually from locomotion toward adjustment near the destination.Source: §E.1 · p. 21
  • All three requirements are attained simultaneously, with errors bounded near the goal after acquisition but residual motion, especially for arms out.Table 6 defines acquisition as meeting the position, yaw, and joint-error thresholds together for 0.5 s.Source: Table 6 · p. 21, §E.1 · p. 21

E.2 How far can Dynamic predict without a new observation?

Dynamic is tested by recursively predicting from an encoded hardware state without intermediate observations. Feature error grows approximately linearly, while physical readouts remain local only over a short horizon.

E.2 How far can Dynamic predict without a new observation?

  • The observation-free rollout recursively propagates executed actions from an encoded hardware state and compares predictions with measured future states.Figure 10 evaluates horizons of 1 to 64 steps, with 20 ms per step.Source: §E.2 · p. 21, Figure 10 · p. 21
  • Feature error grows approximately linearly, while physical readouts remain local for roughly eight steps (160 ms) before position and heading errors grow more sharply.At longer horizons, forecasts drift substantially in both position and heading.Source: §E.2 · pp. 21–22, Figure 10 · p. 21
  • Dynamic is a local model of action consequences; Successor represents subsequent behavior through discounted future features learned from observed transitions.Source: §E.2 · p. 22

E.3 Separating destination from arrival heading

The experiment places stationary posture goals around the robot and varies whether target headings point outward or share one world orientation. The paths show that destination and arrival heading can be controlled independently.

E.3 Separating destination from arrival heading

  • Goals lie on a circle around the initial base position, with either radial target headings or a common world heading.Figure 11 marks target positions and yaw, and displays endpoints and base headings along the paths.Source: §E.3 · p. 22, Figure 11 · p. 22
  • With a common target heading, the robot reaches destinations to its sides and behind it while arriving in the same orientation.With radial targets, heading vectors turn toward the requested outward orientation as paths approach the circle.Source: §E.3 · p. 22, Figure 11 · p. 22
  • Travel direction and arrival heading need not coincide; the figure shows approach to a selected endpoint, while sustained holding is assessed separately.Source: §E.3 · p. 22
Loading 2610.23127v1…