Being-H0.7: A Latent World-Action Model from Egocentric Videos

• Sparse action supervision can push VLAs toward visual-action shortcuts, while pixel-space rollouts waste capacity and add training or inference costs unrelated to control.The paper argues that control needs compact cues such as contact, object motion, affordances, and task progress rather than high-fidelity future frames.Source: §1 · p. 2
• Learnable latent queries attend to instructions, observation history, and robot state, forming a compact interaction-relevant state that conditions action generation.Source: §1 · p. 2
• A training-only posterior replaces the queries with embeddings from subsequent observations, and latent-position hidden-state alignment teaches the deployable prior to infer predictive factors from current context alone.Subsequent observations provide privileged supervision during training, not a deployment-time requirement.Source: §1 · pp. 2–3
• At deployment the posterior is removed and no future frames are generated; norm and rank constraints and a packed Mixture-of-Transformers sequence support stable, scalable dual-branch training.The regularizers prevent magnitude shrinkage and directional collapse, while the dual-branch attention mask shares context computation and keeps reasoning pathways aligned.Source: §1 · p. 3
• Across six simulation benchmarks Being-H0.7 achieves state-of-the-art or comparable performance, leads all five real-world ability suites across 12 tasks, and runs at 3–4 ms/step with UAC.The real-world evaluation spans three robot platforms.Source: §1 · p. 3

Vision-Language-Action Models

• VLAs adapt pretrained vision-language models for generalist robotic control, advancing from tokenized actions to diffusion heads and adding planning or spatial abstractions while still acting from current observations.The cited spatial abstractions include bounding boxes, dense correspondence fields, 3D points, and trajectory traces.Source: §2 · p. 3

World-Action Models

• Video-based world-action work progresses from predictive representations to joint video-action prediction and closed-loop control, whereas Being-H0.7 uses future information to shape latent reasoning without explicit video rollout.DreamZero [12], Cosmos Policy [13], and LingBot-VA [14] carry stronger video priors into embodied policies; Fast-WAM [15] co-trains on video while removing test-time future generation.Source: §2 · pp. 3–4

Latent Future-Aware Action Modeling

• Unlike latent future-prediction approaches that supervise action pathways or stage future-target learning, Being-H0.7 jointly aligns prior and posterior branches on explicit latent reasoning queries before action generation.This lets the queries organize future-aware and context-inferable information for action decoding.Source: §2 · p. 4, Figure 2 · p. 4

Human-Centric Learning

• Human-centric learning spans portable demonstration interfaces, human-video representation and structured supervision, and recovered human actions for VLA pretraining; Being-H0.7 extends this scaling route with latent world-action reasoning.Being-H0 [6] scales reconstructed human hand trajectories, and Being-H0.5 [7] generalizes toward unified cross-embodiment pretraining.Source: §2 · p. 4

3.1 Latent Reasoning: At the Crossroads of VLA and World-Action Model

• Learnable latent queries sit between multimodal context and the noised action chunk, forming an intermediate state that integrates task-relevant information and guides action generation.Action supervision alone may yield weak or shallow latents rather than future prediction, motivating the future-informed alignment mechanism.Source: §3.1 · p. 5

3.2 Latent World-Action Model: Joint Alignment with Future Information

• Future-informed alignment couples a current-context prior with a training-only posterior that replaces its queries with embeddings of future observations, shaping future-relevant, action-oriented latent states.The future observations are encoded by a frozen ViT and aggregated with a Perceiver resampler; hidden states at matched latent positions are aligned across branches.Source: §3.2 · pp. 5–6

3.3 Efficient Dual-Branch Implementation

• A single Mixture-of-Transformers sequence packs both branches with shared context, while an attention mask isolates branch tokens and matched positional IDs preserve structural alignment.The branches are coupled through latent-position alignment rather than direct cross-branch attention.Source: §3.3 · p. 6
• Both branches receive flow-matching action supervision, while norm-preservation and spectral-diversity regularizers discourage latent collapse.Pretraining uses mixed human and robot manipulation data in the UniHand 2.0 [7] sequence format.Source: §3.3 · p. 7

4.1 Training details

• Context images are resized to 224 × 224 and posterior future frames to 256 × 256; V-JEPA2.1 encodes both, with the context encoder trainable.All experiments use RGB-only visual observations.Source: §4.1 · p. 7
• Being-H0.7 builds on Being-H0.5 [7], using InternVL3.5 [108] as the understanding expert and Qwen3 [109] as the action expert.Source: §4.1 · p. 7
• Pretraining uses observation horizon H = 4, action chunk length T = 20, and K = 16 latent queries, matched one-to-one with posterior future embeddings at aligned positions.Prior-posterior latent alignment is applied to the last L = 9 Transformer layers.Source: §4.1 · p. 7
• Pretraining jointly optimizes flow-matching actions, prior–posterior latent alignment, and anti-collapse regularization on mixed human and robot manipulation trajectories in UniHand 2.0 sequence format.The weights are walign = 10^-3 and wnorm = wrank = 10^-4.Source: §4.1 · p. 7
• Downstream post-training optimizes only action generation and latent alignment on task-specific demonstrations, without anti-collapse regularizers.Source: §4.1 · p. 8
• Sequence packing maintains an effective global batch size of approximately 128 trajectory chunks during downstream post-training.Source: §4.1 · p. 8

4.2.1 Experimental Setup

• LIBERO evaluation averages success across four suites over 500 trials per suite; RoboCasa tests 24 tasks with 50 demonstrations and 50 trials per task across held-out scenes.The RoboCasa few-shot setting tests robustness to unseen object instances and novel kitchen styles.Source: §4.2.1 · p. 8
• GR1 covers 24 dexterous bimanual tasks trained with 1,000 demonstrations per task, while LIBERO-plus compares standard-LIBERO training with augmented-data fine-tuning.GR1 evaluation uses 50 trials per task; LIBERO-plus assesses robustness and zero-shot generalization under controlled environmental perturbations.Source: §4.2.1 · p. 8
• RoboTwin 2.0 trains on clean and randomized scenes and evaluates Easy and Hard settings, while CALVIN tests seen- and unseen-environment splits over five-instruction sequences.RoboTwin uses 2,500 clean-scene and 25,000 randomized-scene demonstrations, with 100 rollouts per task. CALVIN evaluates 1,000 sequences on ABCD→D and ABC→D and reports average tasks completed.Source: §4.2.1 · p. 9

4.2.2 Results

• Across six simulation benchmarks, Being-H0.7 achieves state-of-the-art overall performance and the highest average rank, as detailed in Table 1.It reports 99.2% average success on LIBERO, 62.1% on RoboCasa, 49.2% on GR1, and 82.1% zero-shot and 84.8% fine-tuned success on LIBERO-plus. RoboTwin 2.0 success is 90.2% on clean scenes and 89.6% under severe visual domain randomization; CALVIN averages 4.67 tasks on ABCD→D and 4.48 on ABC→D.Source: §4.2.2 · p. 9

4.3.1 Embodiments and Task Suites

• Experiments span PND Adam-U, Unitree G1, and Franka FR3 with Linkerbot O6 hands, evaluating 12 compositional tasks across five ability-oriented suites.The suites cover dynamic scene, physical reasoning, motion reasoning, long-horizon execution, and generalization.Source: §4.3.1 · p. 10, Table 2 · p. 10, Table 3 · p. 10

4.3.2 Evaluation Protocol

• Policies use a shared black-box inference server and randomized rollout order; operators record binary success in blind evaluations of 20 trials per task and method.The shared stack and fixed procedure align comparisons, including on tasks sensitive to reaction timing.Source: §4.3.2 · p. 11

4.3.3 Results Overview

• Dynamic Scene shows the clearest margin, and Being-H0.7 extends Fast-WAM’s strong baseline performance on timing-sensitive Motion Reasoning tasks.The authors attribute the advantage to runtime responsiveness together with a future-aware latent state tracking object motion, relative timing, and contact consequences.Source: §4.3.3 · pp. 11–12
• Being-H0.7 leads all five real-robot suites across three embodiments.Suite-level success rates aggregate tasks by their primary suite and optional overlap tags.Source: §4.3.3 · p. 11, Figure 6 · p. 12
• Being-H0.7 stays ahead of the closest baseline, Being-H0.5, on Physical Reasoning and Long Horizon, where success requires physically grounded, consistent multi-stage manipulation.These suites include tasks involving containment, gravity, deformable contact, tool-mediated force transfer, and subgoals that constrain later stages.Source: §4.3.3 · p. 12
• Being-H0.7 remains most reliable in Generalization across shifted layouts, shelf heights, containers, object instances, and camera geometries.The suite mixes tasks from all three platforms; π0.5 and Being-H0.5 remain reasonably competitive.Source: §4.3.3 · p. 13
• Prior-branch latents paired with current observations yield visualizations suggesting predictive future information, although inference does not reconstruct future frames.The latent representation is described as sufficient to support action generation without pixel-level future modeling.Source: Figure 7 · p. 12, §4.3.3 · p. 13
• UAC locks committed action prefixes while asynchronously updating future suffixes; UAC-enabled Being-H variants reach 3–4 ms/step with unchanged GPU memory.The prefix-lock/suffix-update design absorbs model, transport, and scheduling jitter and supports continuous control across platforms.Source: §4.3.3 · pp. 13–14, Figure 8 · p. 13

5 Conclusion

• Being-H0.7 bridges direct action prediction and world modeling through a compact latent reasoning space.Source: §5 · p. 14
• A deployable prior branch is aligned with a future-aware posterior branch to guide action generation.Source: §5 · p. 14
• The branch alignment injects future-relevant reasoning into action generation.Source: §5 · p. 14
• Inference requires no costly pixel-level rollout.Source: §5 · p. 14
• Combined with large-scale human video pretraining, Being-H0.7 is presented as an effective framework for embodied models.Source: §5 · p. 14
• The authors present Being-H0.7 as a scalable framework for embodied models.Source: §5 · p. 14