Source-linked AI summary
From Foundation to Application: Improving VLA Models in Practice
Wei Wu, Fangjing Wang, Fan Lu, He Sun, Shi Liu, Yunnan Wang, Yibin Yan, Yong Wang, Shuailei Ma, Xinyang Wang, Yibin Liu, Shuai Yang, Tianxiang Zhou, Kejia Zhang, Lei Zhou, Cheng Su, Nan Xue, Bin Tan, Han Zhang, Youchao Zhang, Fei Liao, Xing Zhu, Yujun Shen, Kecheng Zheng
TL;DR
LingBot-VLA 2.0 addresses the gap between laboratory VLA benchmarks and real-world robotic deployment. It combines large-scale heterogeneous pretraining data, whole-body action modeling, and predictive dynamics, and reports improved generalist manipulation plus strong long-horizon mobile manipulation across platforms.
Problem
VLA foundation models remain limited by gaps in embodiment diversity, action-space coverage, and dynamic-environment reasoning between laboratory benchmarks and real-world deployment.
Method
LingBot-VLA 2.0 redesigns data processing, curates around 60,000 hours of robot and egocentric human data, expands control to whole-body degrees of freedom, and uses predictive dynamics modeling.
Results
LingBot-VLA 2.0 improves generalist GM-100 manipulation performance and demonstrates strong cross-embodiment long-horizon mobile manipulation across two robotic platforms.
Takeaways & Limitations
The results support advancing VLA models toward application-oriented usability through combined improvements in generalization, action coverage, and temporal reasoning.
Takeaways & Limitations
Performance gains are not uniform, with partial progress often failing at precise completion steps and disparities remaining between robotic embodiments.
Abstract
from arXiv · showhide
Despite recent progress of VLA foundation models, the disparity between laboratory conditions and real-world applications continues to impede their practical implementation. To bridge this gap, we present LingBot-VLA 2.0, which advances LingBot-VLA through improvements in three functional domains. (1) Generalization across tasks and embodiments. Compared to the previous version, we revamp the data processing pipeline and curate around 60,000 hours of data for pretraining, including 50,000 hours of robot trajectories spanning 20 robot configurations and 10,000 hours of egocentric human videos. (2) Expanded action space in addition to dual-arm hardware platforms. In particular, our system accommodates degrees of freedom for the heads, waists, mobile bases, and dexterous hands, thereby empowering the robots to tackle more complex tasks in practical scenarios. (3) Predictive dynamics modeling for improved temporal reasoning. Specifically, we formulate future prediction as a proxy task, facilitated by a video representation model for semantic priors and a depth estimation model for geometric cues. Evaluations on the GM-100 benchmark, conducted in a generalist setting, validate the beneficial impact of these proposed modifications. Furthermore, benefiting from the expanded pretraining data that covers whole-body degrees of freedom, LingBot-VLA-2.0 demonstrates strong cross-embodiment long-horizon mobile manipulation capability across the two robotic platforms.
1 Introduction
LingBot-VLA 2.0 targets the gap between laboratory benchmarks and real-world deployment by improving data diversity, whole-body action coverage, and temporal reasoning. Evaluations report improved practical capability in generalist manipulation and long-horizon mobile manipulation.
- Motivation: Existing VLA settings often underrepresent heterogeneous embodiments, richer action spaces, and dynamic environments required for practical deployment.These limitations constrain the practical utility of current VLA foundation models.
- Contributions: Future prediction is formulated as a proxy task using video representations for semantic priors and depth estimation for geometric cues.The design targets temporal reasoning in dynamic environments.
- Contributions: LingBot-VLA 2.0 curates around 60,000 hours of pretraining data, including 50,000 hours of robot trajectories and 10,000 hours of egocentric human videos.The robot trajectories span 20 robot configurations.
- Contributions: The model expands beyond dual-arm control to include head, waist, mobile-base, and dexterous-hand degrees of freedom.This expanded action space supports coordinated whole-body interaction for more complex tasks.
- Evaluation: Evaluations cover nine GM-100 dual-arm manipulation tasks and two long-horizon mobile manipulation tasks, with the proposed modifications consistently improving practical capability.The reported results support jointly enhancing generalization, action-space coverage, and predictive dynamics modeling.
2 Related Work
Related VLA research improves generalization, embodiment handling, action modeling, and long-horizon control through heterogeneous data, human-centric signals, unified representations, and mixture-of-experts designs.
- Generalization: Recent VLA efforts scale heterogeneous robot data and use unified pretraining recipes to improve generalization across tasks and embodiments.Human-centric data such as egocentric videos and hand motion is also used to learn transferable embodied priors.
- Embodiment modeling: Other approaches introduce embodiment-aware architectures or unified action spaces to handle heterogeneous robots.These methods address variation in robot configurations and controls.
- System perspective: The data-processing pipeline is a distinct system-level component of LingBot-VLA 2.0's approach to practical VLA development.The supplied related-work passages position practical deployment as depending on more than model scale alone.
- Mixture-of-Experts: Mixture-of-Experts methods scale action-modeling capacity for contact-rich manipulation and long-horizon tasks.Examples route experts using force, tactile, scene-level, or atomic-skill representations.
3 Pre-training Dataset
LingBot-VLA 2.0 builds a filtered, standardized pretraining dataset from robotic and egocentric human data, then maps heterogeneous trajectories and annotations into shared representations.
- Robotic data: Approximately 90,000 hours of data from 20 embodiments are collected, with filtering yielding 50,000 hours of high-quality robotic data.The embodiments include single-arm, dual-arm, and mobile platforms with dexterous hands or grippers.
- Robotic data: Robotic episodes are filtered using jerk and derivative Z-scores, excessive inactivity, state-video misalignment, blur, occlusions, dropped frames, and multi-view misalignment.The inactivity filter discards episodes whose nearly unchanged signals exceed 95% of the episode.
- Egocentric data: Approximately 20,000 hours of egocentric human video are collected and reduced to around 10,000 hours after filtering, reconstruction, standardization, and quality control.The pipeline removes videos lacking egocentric manipulation or clear hand-object interaction before downstream processing.
- Egocentric data: Action-labeled videos undergo metadata organization, timestamp alignment, coordinate transformation, and completeness checks, while action-free videos use egocentric SLAM and hand-pose estimation.Trajectory-level quality control removes insufficient hand-pose coverage, unstable SLAM, and discontinuous trajectories.
- Coordinate standardization: Valid human trajectories are stored in world coordinates and transformed into the sampled frame's camera coordinates during training.This unifies trajectory formats and decouples hand motion from egocentric camera motion.
- Unified representation: A 55-dimensional canonical state-and-action vector shares components for arm joints, end-effector pose, gripper, hand, waist, head, and mobility signals.The representation provides a common format for data collected across embodiments.
- Subtask annotation: Manipulation videos receive temporally aligned task- and subtask-level language annotations, with each subtask assigned an atomic action from an 18-category closed vocabulary.The vocabulary contains 15 primitive actions plus transit, idle, and other auxiliary labels.
4 Method
The method combines a sparse MoE architecture for cross-embodiment VLA pretraining with dual-query distillation for geometric and temporal prediction. It also evaluates scaling behavior and organizes annotated manipulation data by action and object statistics.
- MoE-based VLA model: The sparse MoE architecture uses shared and routed experts to preserve universal priors while adding specialized modeling capacity for cross-embodiment VLA data.Only selected routed experts are activated per token, and the MoE replaces the action expert’s feed-forward network.
- Data annotation: The subtask annotations report action duration composition, frequency, total-time fraction, and mean duration, with idle excluded from training data.Move and transit are frequent, whereas cut, fold, and stir are rare but have longer mean durations.
- MoE-based VLA model: Sigmoid-based routing lets tokens independently activate multiple experts, while bias-based balancing separates expert assignment from routing confidence.The correction bias promotes balanced expert loads, whereas mixture weights retain the original unbiased affinities.
- Scaling experiments: Under matched active parameter counts, MoE models achieve lower training loss and validation action error than dense models.The comparison in Figure 7 attributes the scaling advantage to sparse allocation of model capacity rather than increased total parameter count.
- Spatiotemporal-aware VLA: Dual-query distillation targets current and future observations using complementary depth and video teachers for geometric and temporally grounded representations.The current query represents present geometry, while the future query anticipates geometric configurations relevant to upcoming manipulation.
- Spatiotemporal-aware VLA: DINO-Video extends DINOv3 with causal temporal attention and 3D-RoPE, and is trained on 5M clips from Internet, egocentric, and robotic data.Its representation complements depth distillation by providing motion-aware current and future features.
5 Experiments
LingBot-VLA-2.0 improves generalist bimanual manipulation and long-horizon mobile manipulation across multiple embodiments, while retaining performance advantages under out-of-distribution conditions. Gains are strongest on tasks requiring visual grounding and coordinated, future-aware execution, though final completion and embodiment-specific constraints remain challenging.
- Bimanual manipulation: LingBot-VLA-2.0 achieves the best overall generalist performance on the nine-task GM-100 benchmark across Agilex Cobot Magic and Galaxea R1 Pro.It reaches 66.2 / 34.4 progress score / success rate on Agilex Cobot Magic and 34.6 / 15.6 on Galaxea R1 Pro.
- Bimanual manipulation: 8.0 / 4.4 points separate LingBot-VLA-2.0 from LingBot-VLA-1.0 on Agilex Cobot Magic in progress score / success rate.The model also surpasses π0.5 by 7.1 / 2.2 points on that platform.
- Bimanual manipulation: 100.0 / 100.0 is achieved on Agilex Retrieve keychain, improving from 67.5 / 60.0 with LingBot-VLA-1.0.Comparable gains occur on Agilex Pick out toy bone and Galaxea Pick out toy bone, tasks requiring accurate object grounding and goal-directed execution.
- Limitations: Performance gains are not uniform: gaps between progress score and success rate indicate partial progress followed by failures at precise placement, release, or completion.Differences between Agilex Cobot Magic and Galaxea R1 Pro also indicate continuing challenges from kinematics, camera viewpoints, and action-space alignment.
- Long-horizon mobile manipulation: 77.1 / 60.0 and 84.3 / 66.7 progress score / success rate are achieved in-domain on refrigerator sorting and stove cleaning, respectively.These results improve over π0.5 by 11.8 / 13.3 and 4.4 / 6.7 points and require coordinated base movement, manipulation, and articulated-object interaction.
- Long-horizon mobile manipulation: 37.0 / 13.3 and 67.5 / 40.0 are achieved under OOD evaluation on refrigerator sorting and stove cleaning, maintaining advantages over π0.5 despite performance degradation.OOD trials perturb initial robot poses and introduce unseen manipulated objects; each task-setting pair uses 15 independent trials.
6 Ablation Studies
Ablations show that relative joint actions, MeanStd normalization, and L2 loss improve average action-prediction performance, while task-dependent action-space preferences remain. Dual-query distillation supplies semantic and geometric perceptual supervision for current and future images.
- Action targets: 55.0 average success rate for relative joint actions exceeds 33.7 for absolute joint actions.Across four tasks, relQpos has only 31%–37% of absQpos’s standard deviation, reducing the prediction target’s action scale.
- Action space: 56.0 and 55.0 average success rates are obtained by EEF and joint actions, respectively, but task-level preferences differ.Distribution alignment partly explains behavior: joint actions dominate Barcode Scan, whereas EEF actions dominate Squeeze Ketchup and Scoop Rice.
- Normalization: 55.0 average success rate makes MeanStd the best normalization scheme, versus 47.5 for MinMax and 47.4 for Q01–Q99.MeanStd preserves a normalized standard deviation of 0.95, while MinMax compresses it to 0.15 and Q01–Q99 reaches 0.32.
- Loss function: 55.0 average performance for L2 exceeds 46.4 for L1.Most relQpos targets are small continuous corrections around zero; L1 performs better on Squeeze Ketchup but worse on the other three tasks.
- Perceptual distillation: LingBot-Depth and DINO-Video provide distilled geometric cues and semantic priors for causal perception of current and future images.Figure 13 presents depth and DINO-Video ground truth and predictions for both image timings.
7 Conclusion
LingBot-VLA 2.0 targets the gap between VLA foundation models and real-world deployment through broader generalization, whole-body action modeling, and predictive dynamics. Its reported outcomes include improved GM-100 performance and strong long-horizon mobile manipulation across multiple robotic platforms.
- Conclusion: LingBot-VLA 2.0 narrows the deployment gap by improving generalization, expanding whole-body action modeling, and strengthening temporal reasoning through predictive dynamics.The conclusion identifies these three improvements as the system’s practical focus.
- Conclusion: Expanded pretraining data covering whole-body degrees of freedom supports improved GM-100 performance and strong long-horizon mobile manipulation across multiple robotic platforms.The conclusion attributes these outcomes to the expanded pretraining data and its coverage of whole-body degrees of freedom.