Source-linked AI summary
HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL
Langzhe Gu, Chengkai Hou, Meng Li, Xinhua Wang, Jiaming Liu, Xinyuan Lv, Bowei Zhang, Shuanghao Bai, Guangrun Li, Jingyang He, Gaole Dai, Ziluo Ding, Zhiyuan Xu, Kuan Cheng, Jian Tang, Zhengping Che, Shanghang Zhang
TL;DR
Generalist VLAs struggle to coordinate locomotion, posture, and manipulation on humanoids without costly embodiment-specific adaptation. HAF hierarchically generates whole-body actions and refines a frozen VLA through latent offline-to-online reinforcement learning, outperforming standard baselines across seven real-world tasks.
Problem
Adapting generalist VLAs to humanoid whole-body loco-manipulation remains challenging because single-stage generation does not explicitly model locomotion, posture, and manipulation dependencies.
Method
HAF combines hierarchical action-flow generation with cross-stage conditioning and DCT-compressed latent reinforcement learning for frozen VLA adaptation.
Results
Across seven complex household tasks on two physical humanoid robots, HAF outperforms imitation learning and standard VLA baselines, including challenging OOD scenarios.
Takeaways & Limitations
HAF provides a framework for repurposing pretrained generalist VLAs for humanoid whole-body loco-manipulation without training humanoid-specific foundation models from scratch.
Takeaways & Limitations
The hierarchical pipeline increases denoising computation and latency, while latent reinforcement learning may fail to correct erroneous motions in extreme unseen scenarios.
Abstract
from arXiv · showhide
Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet generalist vision-language-action (VLA) foundation models are not readily applicable to humanoid whole-body loco-manipulation. The high dimensionality and interdependence of humanoid motions make it challenging for conventional single-stage VLA architectures to coordinate locomotion, waist posture, and dual-arm manipulation effectively. Moreover, policies trained through offline behavior cloning can remain suboptimal during real-world deployment. Although online reinforcement learning can refine policies through real-world interaction, directly tuning large VLA backbones demands excessive computation and may introduce safety risks during real-robot exploration. To address these bottlenecks, we introduce HAF (Humanoid Adaptation Framework), a two-part framework consisting of HAF-VLA and HAF-Steer that transfers off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation. HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA. It splits full-body action denoising into three sequential stages with stage embeddings and cross-stage KV caches that retain kinematic dependencies, avoiding incoherent whole-body actions from one-shot generation. On top of the frozen HAF-VLA, HAF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction to restrict RL optimization to a compact noise subspace and train a regularized SAC policy. This avoids updating the large VLA backbone and enables efficient real-world policy refinement. Evaluated on seven real-world humanoid loco-manipulation tasks, HAF surpasses vanilla single-stage VLA baselines and improves whole-body coordination and task performance. Project website: https://grange007.github.io/HAF .
1 Introduction
HAF transfers pretrained generalist VLAs to humanoid whole-body loco-manipulation through hierarchical kinematics-aligned generation and compact latent offline-to-online refinement. It addresses coordination instability, deployment distribution shifts, and the cost and safety risks of online adaptation, with validation on two humanoid robots across seven real-world tasks.
- Motivation: Humanoid loco-manipulation requires synchronized locomotion, torso posture, and bimanual manipulation, because unstable base movements can degrade balance and manipulation accuracy.This coordination challenge distinguishes humanoids from fixed-base or wheeled manipulators.
- Motivation: Standard flow-matching VLAs generate all body-part actions in one stage without explicitly modeling dependencies among locomotion, posture, and manipulation.Generalist VLAs therefore remain difficult to adapt to humanoid whole-body control.
- HAF Framework: HAF combines HAF-VLA, a structured whole-body action generator, with HAF-Steer, lightweight refinement of the frozen generator in compact latent noise space.The framework repurposes pretrained generalist VLAs rather than training humanoid-specific foundation models from scratch.
- HAF-VLA: HAF-VLA sequentially generates locomotion and head orientation, torso adjustment, and bimanual manipulation to align action generation with humanoid kinematics.The ordering prioritizes base stabilization to mitigate unstable upper-body compensatory motions.
- HAF-Steer: HAF-Steer recovers demonstrated-action noise by numerically reversing the frozen flow field, applies temporal DCT, and retains the first 8 coefficients for offline-to-online RL.This avoids optimizing raw high-dimensional actions or updating the large VLA backbone.
- Validation: Across two physical humanoid robots and seven complex household tasks, HAF improves performance, whole-body coordination, and robustness under in-distribution and challenging OOD scenarios.HAF-VLA particularly outperforms imitation learning and standard VLA baselines on tasks requiring long-distance travel and coordinated full-body motion.
2 Related Work
Related work spans humanoid VLAs, RL post-training for VLA policies, and whole-body manipulation control. HAF builds on pretrained generalist VLAs while structuring humanoid actions and avoiding the cost and risks of retraining large backbones.
- Humanoid VLAs: Unlike methods based on latent skills, cross-embodiment transfer, or human videos, HAF directly builds on a pretrained generalist VLA to generate structured whole-body actions grounded in the forward-kinematic chain.This approach spares the computational cost of training a new model.
- RL post-training for VLA models: Offline behavior-cloned VLA policies can remain suboptimal during real-world deployment, motivating RL post-training refinement.Existing approaches include directly fine-tuning the entire VLA backbone with PPO, which incurs massive computational overhead and risks unstable, unsafe motion generation.
- Whole-body manipulation: Whole-body control research enables integrated locomotion and manipulation through advanced low-level controllers and RL-based motion pipelines, while language-conditioned controllers support navigation but lack fine-grained bimanual dexterity.The cited work establishes a technical foundation for humanoid whole-body loco-manipulation but leaves a gap in dexterous bimanual manipulation.
3 HAF-VLA: Hierarchical Action Flow for Humanoid Whole-Body Loco-Manipulation
HAF-VLA adapts pretrained flow-matching VLAs for humanoid whole-body loco-manipulation by generating actions hierarchically from locomotion and gaze through waist posture to bimanual control. Shared action-flow weights, stage embeddings, and cross-stage action caches preserve dependencies while enabling real-time receding-horizon execution.
- Hierarchical action decomposition: Three cumulative stages progressively activate locomotion and gaze, waist posture, and then the complete whole-body action space.Later stages can refine earlier action dimensions, unlike rigidly decoupled generation schemes.
- Hierarchical action decomposition: HAF-VLA decomposes each whole-body action into locomotion/skill-mode, head orientation, waist posture, and bimanual manipulation components.This explicit decomposition supports progressive coarse-to-fine generation instead of simultaneous prediction of all action dimensions.
- Cross-stage conditioning: Shared action-flow weights reuse one vision-language prefix KV cache, while trainable stage embeddings and cross-stage action KV caches condition sequential generation.Earlier denoised motion predictions provide coarse prior context to later stages.
- Training and inference: Training applies stage-wise flow matching with equal loss weights and masked inactive dimensions, while teacher-forced ground-truth caches prevent error accumulation.At inference, stages use independent Gaussian noise and sequentially constructed caches.
- Training and inference: Only Stage 3’s denoised output is executed through receding-horizon control, with H = 100, 40 executed steps, and approximately 0.12 seconds for three-stage inference.The deployment uses 10 flow-denoising steps per stage on a single RTX 5090 GPU.
4 HAF-Steer: Spectral Latent RL for Flow-Matching VLAs
HAF-Steer adapts a frozen flow-matching VLA by optimizing compact spectral representations of its initial flow noise rather than the full action space. It combines offline demonstration guidance with mixed offline–online SAC and decodes learned spectral corrections into smooth executable whole-body actions.
- Spectral Latent Space: HAF-Steer maps demonstrated actions back to initial flow noise, compresses them into low-dimensional spectral actions, and trains a stochastic policy in that space.The resulting spectral actions are decoded through the frozen flow generator into executable action chunks.
- Frozen Flow-Matching VLA: The frozen VLA generates whole-body action chunks deterministically from initial noise and unified vision-language-robot conditioning, while its parameters remain fixed during reinforcement learning.HAF-Steer therefore avoids updating the large pretrained backbone during adaptation.
- Spectral Latent Space: Restricting exploration to the first K DCT coefficients reduces temporal noise dimensionality and promotes smoothness, avoiding instability associated with raw noise optimization or repeated chunk-wide noise vectors.At inference, inverse DCT zero-pads truncated coefficients to reconstruct smooth full-horizon temporal noise before flow decoding.
- Offline-to-Online RL: Offline spectral replay stores recovered expert targets with sparse rewards, assigning r = 1 only to successful terminal transitions and r = 0 to intermediate transitions.These targets guide behavior-cloning initialization and task-aligned spectral-noise correction learning.
- Offline-to-Online RL: After behavior-cloning initialization, mixed offline–online SAC trains on demonstration and real-robot transitions, using offline data for early stabilization and online interaction for deployment-distribution adaptation.The offline sampling ratio is gradually decayed during training, while demonstration regularization balances the mixed objective.
- Inference Pipeline: At test time, the spectral actor samples normalized coefficients, reconstructs noise with inverse DCT, and feeds it into the frozen VLA flow model to produce physically plausible whole-body actions.The same spectral parameterization and offline-to-online objective apply to π0.5 and HAF-VLA, with HAF-Steer replacing only HAF-VLA’s Stage 3 initial noise.
5 Experiments
Experiments on seven real-world humanoid loco-manipulation tasks show that HAF-VLA improves overall performance and whole-body coordination over strong baselines. Ablations, disturbance tests, and deployment adaptation further support the hierarchical design, its robustness, and HAF-Steer’s offline-to-online refinement.
- Main Results: HAF-VLA achieves the best or tied-best average normalized score on all seven tasks, raising overall performance from 53.3% with π0.5 to 70.5%.The tasks require long-horizon navigation, whole-body coordination, and precise object interaction.
- Ablation Study: The full hierarchical action-flow design outperforms all ablated variants, showing gains do not simply result from increasing denoising iterations.The comparison includes all-joint denoising, arm-first hierarchy, and vanilla π0.5 with matched total denoising steps.
- Ablation Study: Establishing locomotion and body posture before fine manipulation is important, while increasing vanilla π0.5 from 10 to 30 denoising steps decreases performance.The arm-first variant performs worse, despite reversing the coarse-to-fine generation order.
- Generalization: HAF-VLA maintains higher task performance than vanilla π0.5 under unseen visual distraction and a 20 cm backward initial-position shift.The tests use an unseen black office chair during Laundry Loading and shifted initial positions during Clothes Retrieval.
- HAF-Steer Evaluation: HAF-Steer consistently improves π0.5 and HAF-VLA across in-distribution and out-of-distribution settings, with mixed offline–online RL further increasing real-world success rates.HAF-Steer adapts the frozen policy to deployment-time distribution shifts; applied to HAF-VLA, it improves success in all four evaluated settings and is best or tied-best in three.
6 Conclusion and Limitations
HAF adapts pretrained VLA models to humanoid whole-body loco-manipulation through hierarchical action generation and latent offline-to-online reinforcement learning. Its main limitations are increased denoising latency and limited ability to correct erroneous motions in extreme unseen scenarios.
- Conclusion: HAF combines HAF-VLA’s progressive denoising and cross-stage KV-cache conditioning with HAF-Steer’s SAC-based latent reinforcement learning over DCT-compressed noise coefficients.Together, these components adapt pretrained VLA models to humanoid whole-body loco-manipulation and improve real-world deployment through offline-to-online adaptation.
- Limitations: The hierarchical pipeline increases denoising computation and deployment latency, while latent RL remains constrained by the base VLA’s priors in extreme unseen scenarios.Future work targets efficient denoising schemes and lightweight RL implementations to improve real-time performance and task robustness.