PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Behavior Foundation Model for Humanoid Robots

Weishuai Zeng, Kangning Yin, Xiaojie Niu, Shunlin Lu, Weixiang Zhong, Jiahe Chen, Feiyu Jia, Xiao Chen, Zirui Wang, Furui Xu, Ming Zhou, Kailin Li, Weinan Zhang, He Wang, Li Yi, Dahua Lin, Jiangmiao Pang, Jingbo Wang

arXiv:2607.15163v1cs.ROcs.AI

TL;DR

Humanoid control remains challenged by whole-body coordination and generalization, while how to coordinate factors for scaling Behavior Foundation Models is unclear. The paper investigates motion tracking, on-policy data and reference-motion diversity, and the Humanoid Transformer. It reports lower test-set MPKPE than existing humanoid controllers and concludes that BFMs provide a foundation for scalable, general-purpose humanoid control.

  • Problem

    How to coordinate learning paradigm, behavioral data, and model architecture to scale BFMs remains unclear, while humanoid controllers often remain task-specific and generalize poorly across scenarios.

    Source: §1 · p. 2

  • Method

    The paper coordinates global-frame whole-body motion tracking, on-policy rollout quantity with reference-motion diversity, and the Humanoid Transformer architecture.

    Source: §1 · pp. 2–3

  • Results

    Over 10% in local mode and 82% in global mode, test-set Mean Per-Keypoint Position Error (MPKPE) is reduced versus existing humanoid controllers.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors conclude that their BFM provides a principled foundation for scalable, general-purpose humanoid control.

    Source: §1 · p. 3, §5 · p. 19

    Scaling either on-policy data dimension alone does not consistently improve performance, and effective collection may require balancing rollout width and depth.

    Source: §4.2.1 · p. 12

Abstract

from arXiv · show

Humanoid control requires natural whole-body coordination, precise real-time responses to control signals, and robust generalization across diverse environmental contexts, making it a cornerstone for generalist embodied agents. Behavior Foundation Models (BFMs) have recently emerged as a promising solution to address these challenges by leveraging large-scale behavioral data to achieve superior expressiveness, versatility and generalization. However, despite growing interest in scaling BFMs to further improve their capabilities, it remains unclear how key factors, including the learning paradigm, behavioral data and model architecture should be coordinated to enable effective scaling. In this work, we revisit the scaling recipe for BFMs and demonstrate that substantial performance gains can be achieved through the coordination of three core components: 1) the learning paradigm of motion tracking that reformulates diverse humanoid control problems as the reproduction of integrated whole-body behaviors in the global frame; 2) the strategic synergy between on-policy rollout quantity and reference motion diversity; and 3) the expressive and scalable model architecture termed Humanoid Transformer that facilitates the natural emergence of structured behavioral representations. Through extensive experiments in both simulation and real-world deployment, we demonstrate that our approach yields significant improvements in control fidelity and task generalization, reducing Mean Per-Keypoint Position Error (MPKPE) on the test set by over 10% in local mode and 82% in global mode compared with existing humanoid controllers. These results establish BFM as a principled and effective foundation for scalable and general-purpose humanoid control.

1 Introduction

Humanoid control remains difficult to generalize across tasks, while the scaling principles for Behavior Foundation Models (BFMs) are not yet clear. The paper coordinates motion tracking, behavioral data, and model architecture, reporting improved control fidelity and generalization.

1 Introduction

  • BFM scaling principles remain largely unexplored, despite initial studies, and the paper investigates the coordination of learning paradigm, behavioral data, and architecture.Existing humanoid controllers often rely on task-specific reward engineering, limiting versatility and generalization across scenarios.Source: §1 · p. 2
  • Motion tracking trains BFMs to reproduce reference motions as integrated whole-body trajectories in the global frame, providing dense guidance for behavior learning.Tracking root translation helps distinguish behaviors such as walking forward from marching in place; the formulation also avoids separating root evolution from local pose tracking.Source: §1 · pp. 2–3
  • PPO data scaling combines on-policy rollout quantity, shaped by environment count and rollout horizon, with behavioral diversity from a curated reference-motion corpus.The paper distinguishes environment-interaction rollouts as training data from the motion corpus, which shapes the behavioral distribution.Source: §1 · p. 3
  • The Humanoid Transformer is examined as an expressive, scalable architecture for scaling BFMs and enabling structured behavioral representations to emerge without auxiliary objectives.The paper investigates whether those representations support flexible use and efficient transfer of learned behavioral knowledge.Source: §1 · p. 3
  • Over 10% in local mode and 82% in global mode, test-set Mean Per-Keypoint Position Error (MPKPE) is reduced versus existing humanoid controllers.Experiments span simulation and real-world deployment and report improved control fidelity, execution robustness, and generalization.Source: §1 · p. 3

2 Related Work

Prior humanoid control research moved from task-specific objectives toward reference-motion tracking and explored broader command interfaces and reusable behavioral representations. BFM work has developed varied pretraining strategies, while systematic understanding of how scaling factors interact remains limited.

2.1 Humanoid Whole-body Control

2.2 Behavior Foundation Models for Humanoid Robots

  • BFM pretraining spans DAgger, offline supervised learning with classifier-guided diffusion, unsupervised reinforcement learning, and motion tracking with discrete behavioral representations [6] [9] [10] [11].These approaches support different control or adaptation mechanisms, including direct behavior specifications, task-conditioned trajectories, and promptable latent spaces.Source: §2.2 · p. 4
  • Prior scaling evidence from SONIC varies motion count, model capacity, and GPU hours but does not clarify their coupling with learning paradigm and training data [11].This paper revisits motion tracking, PPO training data, and the Humanoid Transformer to investigate those factors together.Source: §2.2 · p. 4

3 Method

The method frames BFM pretraining as goal-conditioned reinforcement learning, using whole-body motion tracking as a shared proxy task and coordinating its control interface, training data, and architecture for scalable behavior learning.

3.1 Problem Formulation

  • Motion tracking serves as a proxy task for behavior learning, while BFMs remain free to accept diverse behavioral specifications rather than only whole-body reference states.The formulation seeks a shared objective and versatile control interface across behaviors and control settings.Source: §3.1 · pp. 5–6

3.2 Learning Paradigm

  • Global-frame tracking of integrated whole-body trajectories provides coherent guidance, while masked root-relative target poses support control from diverse behavioral specifications.The actor receives masked target information, while the critic receives complete targets; training also randomizes physical parameters and robot velocities.Source: §3.2 · pp. 6–7

3.3 Training Data

  • Scalable training requires both more on-policy rollouts and diverse reference motions, since rollout quantity depends on environment parallelism and horizon while motion diversity broadens behavioral coverage.The reference collection contains 102M frames at 50 FPS; adaptive sampling focuses training on difficult motions while retaining corpus coverage.Source: §3.3 · pp. 7–8

3.4 Model Architecture

  • The Humanoid Transformer uses modality-specific tokenizers, temporal context, and cross-attention to condition action or value prediction on behavioral specifications.RMSNorm shapes goal embeddings into a structured latent space through motion-tracking pretraining, without auxiliary regularization; stochastic actor offsets accommodate real-time control-signal delays.Source: §3.4 · pp. 9–10

4 Experiments

The experiments examine how training data and model architecture affect BFM scaling, analyze learned latent representations, and benchmark the resulting controller. They evaluate tracking across two test sets and representative control modes, with training and evaluation performed in IsaacLab and MuJoCo, respectively.

4.1 Experiment Setup

  • The evaluation uses Unitree G1 models trained in IsaacLab and tested in MuJoCo, measuring success and global and local tracking errors over links activated by each control mode.It includes a held-out BONES set and a separate set of previously unseen motions from Xsens and 100Style.Source: §4.1 · pp. 10–11

4.2.1 Scaling On-policy Data Collection

  • Jointly scaling to 64 GPUs and a rollout horizon of 64 gives the best performance in nearly all experiments, whereas scaling either dimension alone is not consistently beneficial.The authors attribute gains to more on-policy experience per update and more reliable estimates for PPO optimization.Source: Figure 4 · p. 11, §4.2.1 · p. 12, Figure 5 · p. 12

4.2.2 Scaling Reference Motions

  • Reference-motion scaling from XXS to S raises motion quantity with nearly unchanged measured coverage, while scaling from S to L expands coverage and improves the novel-source Ours Test Set substantially, but adds little on BONES.The authors report that significant improvements occur only when scaling markedly expands measured behavioral coverage, and only on the benchmark containing motions from novel sources.Source: §4.2.2 · pp. 13–14, Table 1 · p. 13, Figure 6 · p. 13
  • For general BFM capabilities, the authors recommend combining complementary data sources, while noting that more sources do not inherently guarantee broader behavioral coverage.They also argue that adding in-domain motions may yield diminishing returns once dominant target-domain behaviors are represented, and that rare behaviors will remain absent from finite datasets.Source: §4.2.2 · pp. 14–15

4.3 Scaling with Model Architecture

  • The Humanoid Transformer generally achieves higher success rates and lower tracking errors than MLPs, with a medium model matching or exceeding a substantially larger MLP and larger Transformers showing diminishing returns.Increasing Transformer capacity does not consistently improve every control mode; some modes saturate at modest scales.Source: Figure 8 · p. 15, §4.3 · p. 16, Figure 9 · p. 16

4.4 Latent Analysis

  • Latent representations show smooth within-sequence trajectories, separate regions for distinct behavioral directions, and robustness to moderate latent noise.Walking forward and backward occupy opposite sides of one latent dimension, while crouching left and right separate along another.Source: §4.4 · pp. 17–18, Figure 10 · p. 17, Table 2 · p. 17
  • As model capacity increases, latent representations across control modes converge, which may help explain why some modes improve while others slightly degrade.The authors present this relationship as a potential explanation tied to stronger coupling among modes in the shared latent space.Source: §4.4 · p. 18, Figure 11 · p. 18

4.5 Model Performance Benchmarking

  • The proposed BFM outperforms the compared controllers in success rates and tracking errors across local and global whole-body control, while its reward formulation lowers global errors than the ablation.The comparison includes GMT, TWIST, SONIC, and a baseline using BeyondMimic’s reward design.Source: Table 3 · p. 19, §4.5 · p. 19

5 Discussion

The paper investigates BFM scaling across learning paradigm, behavioral data, and architecture, and reports improvements in control fidelity, robustness, and generalization. It identifies unresolved questions about the control interface and constraints in current pretraining infrastructure.

5 Discussion

  • The unified control interface may not be the best abstraction, and its integration with future high-level policies remains unclear.The authors argue BFM design should continue evolving alongside advances in high-level policy learning.Source: §5 · p. 19
  • Current BFM pretraining infrastructure remains preliminary and constrains scaling to broader settings.Source: §5 · p. 19
  • The paper investigates scaling across learning paradigm, behavioral data, and model architecture, reporting gains in fidelity, execution robustness, and generalization.Source: §5 · p. 19

A Method Details

The appendix describes curated control modes, reward components, and domain-randomization parameters as parts of the experimental setup.

A.1 Control Modes

  • Eight representative control modes span different specification granularities and are sampled at environment resets.This lets the same behavior appear under varied specifications and supports inpainting from sparse control signals.Source: Table 4 · p. 25

A.2 Reward Design

  • The reward design is documented in a table of reward components and weights.Reference-derived variables are marked with a superscript, and the environment scales weights according to simulation timestep dt.Source: Table 5 · p. 25

A.3 Domain Randomization

  • The appendix lists domain-randomization parameters and distinguishes randomization applied at training start from periodic root-velocity perturbations.The table defines U[·] as a uniform distribution.Source: Table 6 · p. 26

A.4 Model Architecture

The Humanoid Transformer encodes temporal histories and future goals, then uses attention and a learnable query token to predict actions or values. Its goal embeddings are normalized, with the latent space shaped only by motion tracking.

A.4 Model Architecture

  • Proprioceptive states, actions, and goal states are extended into temporal windows and encoded with modality-specific tokenizers.Proprioception and action tokens are interleaved as context, followed by a learnable query token; goal tokens correspond to future frames.Source: §A.4 · p. 26
  • Self-attention models temporal dependencies in the historical context, while cross-attention conditions on behavioral specifications.The design separates behavioral-manifold modeling from injection of control conditions.Source: §A.4 · p. 27
  • The learnable query token aggregates historical information while remaining masked from other context tokens in self-attention.Source: §A.4 · p. 27
  • RMSNorm maps goal embeddings onto a continuous, bounded hypersphere, and the paper focuses on the unit hypersphere.The latent space is shaped solely by the motion-tracking objective, without auxiliary regularization.Source: §A.4 · p. 27
  • A projection head uses the learnable query-token embedding to predict an action or value.The tokenizers and projection head are linear layers, while the feed-forward module uses SwiGLU.Source: §A.4 · p. 27

B Experiment Details

Real-world experiments use a 29-degree-of-freedom Unitree G1 with separate high- and low-level control processes. The setup supports latency-aware streaming and online switching among curated control modes, with different handling for global and local control.

B Experiment Details

  • Real-world experiments use a 29-degree-of-freedom Unitree G1, with BFM inference separated from the joint-space PD controller.An emergency-stop button enables direct human interference.Source: §B.1 · p. 27
  • The low-level controller runs at 200 Hz and high-level BFM inference at 50 Hz.Inference can run onboard with a Jetson Orin GPU or on a connected local PC.Source: §B.1 · pp. 27–28
  • For streamed control signals, timestamped Wi-Fi data buffers adjust the final future-reference frame to compensate for communication and system latency.Source: §B.1 · p. 28
  • Global control localizes the root with HTC VIVE Ultimate Trackers, while local control preserves heading from the robot's IMU.Both modes align control-signal direction through first-frame calibration; local control treats the specified root position as the robot's actual position.Source: §B.1 · p. 28
  • Any of the eight curated control modes can be activated or switched online, but local control is not directly applicable when a mode omits an explicit root target.Source: §B.1 · p. 28

B.2 Evaluation Protocol

The protocol evaluates success rate and global and root-relative tracking errors over the links activated by each control mode.

B.2 Evaluation Protocol

  • A sequence is unsuccessful if any selected link deviates by more than 0.5 m from its desired global position.The position threshold is evaluated in the global frame.Source: §B.2 · p. 28
  • G-MPKPE and G-MPKRE measure selected-link position and rotation errors in the global frame.Position errors are reported in meters and rotation errors in radians.Source: §B.2 · p. 28
  • L-MPKPE and L-MPKRE measure the corresponding root-relative errors after removing horizontal translation offsets and heading differences.Position errors are reported in meters and rotation errors in radians.Source: §B.2 · p. 28

B.3 Training Hyperparameters

Table 7 lists hyperparameters held fixed across experiments to ensure fair comparisons.

B.3 Training Hyperparameters

  • Table 7 presents hyperparameter details and identifies the terms held fixed across experiments.The caption states that fixing these terms ensures a fair comparison; it does not specify their values.Source: Table 7 · p. 29
  • The table’s stated purpose is to support fair comparisons by keeping the listed terms constant across experiments.Source: Table 7 · p. 29
  • The supplied caption gives no individual hyperparameter values.Source: Table 7 · p. 29

B.4 Scaling On-Policy Data Collection

The scaling studies vary on-policy data quantity and reference-motion dataset size while holding other training or model choices fixed. They also assess behavioral coverage through occupancy in a shared feature-space clustering.

B.4 Scaling On-Policy Data Collection

  • On-policy data collection is scaled by increasing GPU count for rollout width or extending rollout horizon for depth.Experiments hold environments per GPU constant and adjust mini-batches to vary only total collected on-policy data; the medium BONES-SEED dataset and medium Humanoid Transformer are fixed.Source: §B.4 · p. 29
  • The on-policy study evaluates two test sets across four control modes while varying rollout horizon and GPU count over 32, 48, and 64.It evaluates checkpoints at 20,000 epochs; figures identify success rate and tracking errors, but supplied captions do not report outcomes.Source: §B.4 · p. 29, Figure 12 · p. 30, Figure 13 · p. 30

B.5 Scaling Reference Motions

  • Reference-motion scaling uses five nested subsets, from fractions of BONES-SEED through the full collection, with training configuration and model size held fixed.The study adopts the 64-GPU, 64-horizon on-policy configuration and evaluates two test sets across four representative control modes.Source: §B.5 · p. 33, Table 8 · p. 33
  • Behavioral coverage is measured as the fraction of 2,000 shared K-Means clusters occupied by sampled features, with greater occupancy indicating broader coverage.The feature space is fit using 200,000 randomly sampled features from the full partition; the measure depends on that reference space, and finite datasets may omit rare or long-tail behaviors.Source: §B.5 · p. 33

B.6 Scaling with Model Architectures

The architecture study compares MLP and Humanoid Transformer backbones and tests whether increasing Transformer capacity brings further gains. Its scaling conclusions are bounded by training infrastructure and onboard computation constraints.

B.6 Scaling with Model Architectures

  • The study compares MLP and Humanoid Transformer backbones and varies Transformer capacity while holding algorithmic, environmental, data, and on-policy settings fixed.The actor and critic use identical backbone architectures and capacities, differing only in input and output dimensions.Source: §B.6 · p. 37
  • The MLP baselines have 3.05M and 11.86M parameters, while the four Transformer configurations range from 0.41M to 9.91M parameters.Transformer scaling varies embedding and feed-forward dimensions, attention heads, and depth; the L configuration increases depth relative to M while retaining its width.Source: §B.6 · p. 37
  • The authors describe the scaling study as limited by fragile distributed training, simulator instability, immature training infrastructure, and onboard compute constraints.They state that current hardware constraints prevent indefinite BFM parameter growth while maintaining inference above 50 Hz and reserving compute for high-level models.Source: §B.6 · p. 37

Related papers

Loading 2607.15163v1…