Source-linked AI summary
DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation
Junfeng Li, Junjie He, Zhide Zhong, Yangyang Zheng, Pingyue Sheng, Jiayu Dong, Ruixin Li, Haodong Yan, Jiaguan Zhu, Tianran Zhang, Runze Yu, Wen Chen, Liuqing Yang, Yuxiang Gao, Haoang Li
TL;DR
Training one robot-manipulation policy across heterogeneous embodiments remains difficult because existing methods underuse shared dynamics and require manual action alignment. DyPES-VLA learns shared dynamics priors through future prediction and uses embodiment-specific experts to generate native controls, achieving state-of-the-art simulation performance and 75.6% average success across real-world tasks.
Problem
Cross-embodiment VLAs must determine what knowledge to share across robot embodiments while addressing limited dynamics supervision and costly manual action-space alignment.
Method
DyPES-VLA learns shared dynamics priors from future prediction and translates them into native robot controls through an embodiment-specific Mixture-of-Experts action head.
Results
DyPES-VLA achieves state-of-the-art performance across three simulation benchmarks and averages 75.6% success across three real-world tasks and embodiments.
Takeaways & Limitations
The results support shared dynamics priors as a transferable foundation for physical control across distinct robot morphologies and action spaces.
Abstract
from arXiv · showhide
Vision-Language-Action (VLA) models have become a powerful paradigm for robot manipulation, but training a single generalist policy for heterogeneous robot embodiments remains an open problem. Existing methods have two main limitations. First, they underuse dynamics priors shared across diverse visual and interaction data, limiting cross-embodiment transfer. Second, they require extensive manual preprocessing to convert embodiment-specific actions into a common format. To overcome these limitations, we propose DyPES-VLA, a cross-embodiment VLA that learns shared Dynamics Priors and Embodiment-Specific control. First, we learn shared dynamics priors by training the vision-language model (VLM) with a future-prediction objective on cross-embodiment data, driving the shared query representation to capture object motion, contact, and interaction-induced scene changes. Second, an embodiment-specific Mixture-of-Experts (MoE) action head translates these shared dynamics priors into executable controls directly in each embodiment's native action space, without manually pre-aligning heterogeneous actions into a common format. This head shares attention layers to capture common temporal action structures, while its embodiment-specific feed-forward experts resolve the unique kinematic constraints and control semantics of distinct embodiments. As a generalist policy, our \ourmethod achieves state-of-the-art performance across simulation and real-world evaluations, reaching 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin~2.0.
1 Introduction
DyPES-VLA addresses cross-embodiment generalization by learning shared dynamics priors from future prediction and translating them into embodiment-specific native controls without common action-space alignment. A single co-trained checkpoint achieves strong performance across three simulation benchmarks, while real-world experiments support transfer across distinct morphologies and action spaces.
- Motivation: Existing VLA policies often remain tied to specific robots and struggle to generalize across embodiments.
- Limitations: Existing cross-embodiment VLAs underuse manipulation videos because they rely on action prediction alone and manually map heterogeneous controls at the action level.
- Approach: DyPES-VLA learns shared dynamics priors and embodiment-specific control, enabling joint training with arbitrary embodiments without additional action-space alignment.
- Architecture: Future-supervised query states provide a shared interface, while an embodiment-specific MoE action head directly generates action chunks in each robot’s native action space.
- Results: 98.0% success on LIBERO, 59.25% on RoboCasa-GR1, and 89.02% on RoboTwin 2.0 are achieved by one co-trained checkpoint across three simulation benchmarks.
- Results: Real-world experiments show that cross-embodiment co-training provides transferable dynamics priors for physical control across distinct morphologies and action spaces.
2 Related Work
Related work progresses from robot-specific VLAs toward generalist policies trained on heterogeneous robot data, while cross-embodiment methods differ in how they represent embodiment specificity. Predictive learning offers unified supervision across embodiments, with DyPES-VLA using future-supervised query states to learn shared dynamics priors and decode native controls.
- Vision-Language-Action Models: Early VLAs were typically robot-specific, whereas cross-robot corpora enabled increasingly general policies across heterogeneous embodiments.The passage cites Open X-Embodiment, Octo, and the π series as examples of this progression.
- Cross-Embodiment VLA Models: Cross-embodiment VLA methods mainly differ by using common action spaces, separate embodiment context, or embodiment-specific modules within a shared architecture.Examples include shared end-effector frames, soft prompts, motion-transfer training, shared action heads, and per-embodiment stems or heads.
- Predictive Learning for Cross-Embodiment Manipulation: Future prediction supplies unified supervision for cross-embodiment data through coupled prediction-and-control approaches, including world-action models and future-guided VLA planning.A second line uses predictive learning to incorporate action-free videos or scale joint dynamics, forecasting, and policy learning over heterogeneous data.
- Predictive Learning for Cross-Embodiment Manipulation: DyPES-VLA uses future-supervised query states to learn shared dynamics priors and decodes them into native controls.This distinguishes its approach from prior predictive methods described in the passage.
3 Methodology
DyPES-VLA uses shared query states from a multimodal VLM to learn future-oriented dynamics priors and translate them into embodiment-specific native controls. Its two-stage training combines action-free video pretraining with joint cross-embodiment demonstration learning.
- Shared Query Representation: A pretrained VLM encodes visual observations, instructions, and embodiment metadata, while learnable query tokens produce shared query states connecting future prediction and action generation.The policy uses no proprioceptive input, and embodiment metadata explicitly includes fields such as data source, robot type, and control frequency.
- Dynamics-Prior Learning: The future-generation head supervises query states by predicting an embodiment-matched future primary-camera frame from the current context.The temporal offset Δe is set per embodiment to match its action horizon He.
- Embodiment-Specific Control: The action head is a flow-matching DiT whose shared attention captures temporal structure, while metadata-routed experts realize controls for each embodiment.Each embodiment has an encoder–decoder interface mapping native action chunks into the shared DiT width and back to native-space velocities.
- Training Procedure: Stage 1 pretrains the VLM, query tokens, and SANA head on action-free human and simulation videos using the future-prediction objective.The data includes egocentric human manipulation videos from EgoDex and simulation videos from co-training embodiments.
- Training Procedure: Stage 2 jointly trains both heads on action-labeled demonstrations collected from multiple robot embodiments.A weighting coefficient λw balances future prediction against action learning.
- Inference: At deployment, one VLM pass conditions the action head, which integrates flow from Gaussian noise with a few Euler steps to output a native action chunk.The future-generation head is skipped during inference.
4 Experiments
Experiments evaluate DyPES-VLA as a single-checkpoint cross-embodiment policy across simulation and real-world platforms. It matches or surpasses strong baselines on three simulation benchmarks, while ablations support future supervision, embodiment-specific action realization, and embodiment metadata.
- Benchmarks: DyPES-VLA is evaluated on three simulation benchmarks and three real-world robot platforms spanning single-arm, dual-arm, and humanoid embodiments.The simulation platforms are RoboTwin 2.0, RoboCasa-GR1, and LIBERO; physical platforms include FR3, AgileX COBOT Magic, and Unitree G1.
- Simulation results: With a single checkpoint, DyPES-VLA matches or surpasses the strongest methods on all three simulation benchmarks.It is evaluated without per-benchmark finetuning, unlike specialist checkpoints.
- Simulation results: 89.02% average success on RoboTwin 2.0 exceeds Qwen-VLA by 2.37 points, while RoboCasa-GR1 reaches 59.25%, surpassing ABot-M0 by 0.95 points and LDA-1B by 3.85 points.RoboTwin 2.0 results are 88.78% clean and 89.26% randomized; RoboCasa-GR1 uses a 29-DoF humanoid.
- Simulation results: 98.0% success on LIBERO surpasses Fast-WAM by 0.4 points and OpenVLA-OFT by 0.9 points, trailing finetuned X-VLA by only 0.1 points.LIBERO uses a 7-DoF single-arm robot.
- Ablations: Removing future prediction causes drops of 2.4 points on RoboTwin 2.0 and 2.5 points on RoboCasa-GR1, while replacing the MoE head costs 1.2 and 2.1 points, respectively.Removing Stage-1 pretraining also degrades all three benchmarks; removing embodiment metadata causes a consistent drop of up to 0.5 points.
5 Conclusion
DyPES-VLA instantiates a cross-embodiment generalist-manipulation paradigm that combines shared dynamics priors with embodiment-specific control. It learns these priors across heterogeneous data using future prediction and a shared query interface.
- DyPES-VLA combines shared dynamics priors with embodiment-specific control for cross-embodiment generalist manipulation.
- The paradigm is designed to extend to arbitrary embodiments and heterogeneous data.
- It instantiates a single-arm, dual-arm, and humanoid platform, using future prediction across action-free videos and action-labeled demonstrations.
- A shared query interface is supervised to retain regularities of object motion and contact.
Supplementary Material · A Per-Task Results on RoboCasa-GR1
DyPES-VLA’s single co-trained checkpoint was evaluated on all 24 RoboCasa-GR1 tabletop tasks using 50 rollouts per task, totaling 1,200 rollouts. Success rates were 55.3% for pick-and-place-into-articulated-receptacle tasks and 60.6% for container-to-container tasks.
- A Per-Task Results on RoboCasa-GR1: 24 RoboCasa-GR1 tabletop tasks were evaluated with the single co-trained checkpoint.The evaluation covered all tasks in the benchmark.
- A Per-Task Results on RoboCasa-GR1: 50 rollouts per task were used to measure success rates.This produced 1,200 rollouts in total.
- Supplementary Material: 1,200 total rollouts formed the evaluation set across the 24 tasks.The rollout count follows from 50 rollouts for each of 24 tasks.
- A Per-Task Results on RoboCasa-GR1: 55.3% success was achieved on the six pick-and-place-intoarticulated-receptacle tasks.This result summarizes the first task family.
- A Per-Task Results on RoboCasa-GR1: 60.6% success was achieved on the eighteen container-to-container tasks.This result summarizes the second task family.
- A Per-Task Results on RoboCasa-GR1: The reported results were consistent across the two RoboCasa-GR1 task families.The two family-level success rates were 55.3% and 60.6%.
B Per-Task Results on RoboTwin 2.0
DyPES-VLA achieves an overall 89.02% success rate across 50 RoboTwin 2.0 tasks, averaging 88.78% under clean evaluation and 89.26% under randomized evaluation.
- B Per-Task Results on RoboTwin 2.0: 89.02% overall average success across 50 RoboTwin 2.0 tasks combines 88.78% clean and 89.26% randomized evaluation results.The randomized setting perturbs backgrounds, lighting, and object placements; each task uses 100 rollouts in the clean evaluation.
C Additional Real-World Qualitative Results
Figures S1 and S2 show additional real-world qualitative rollouts for water pouring and book placement across the FR3, COBOT Magic, and G1 physical platforms. Each sequence presents six chronological frames under its corresponding task instruction.
- Real-World Qualitative Results: Additional qualitative rollouts cover water pouring and book placement on FR3, COBOT Magic, and G1.Figures S1 and S2 provide these rollouts across all three physical platforms.
- Water Pouring: Water-pouring sequences show six chronological frames for pouring water from the cup into the bowl.The sequences are presented for FR3, COBOT Magic, and G1.
- Book Placement: Book-placement sequences show six chronological frames for placing the book onto the shelf.The sequences are presented for FR3, COBOT Magic, and G1.