Source-linked AI summary
Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution
Rui Cai, Jun Guo, Xinze He, Piaopiao Jin, Jie Li, Bingxuan Lin, Futeng Liu, Wei Liu, Fei Ma, Kun Ma, Feng Qiu, Heng Qu, Yifei Su, Qiao Sun, Dong Wang, Donghao Wang, Yunhong Wang, Rujie Wu, Diyun Xiang, Yu Yang, Hangjun Ye, Yuan Zhang, Quanyun Zhou
TL;DR
Large VLA models face inference latency that can cause pauses, discontinuous actions, and jerky real-robot motion. Xiaomi-Robotics-0 combines cross-embodiment pre-training, asynchronous-execution post-training, and aligned action-chunk deployment, achieving state-of-the-art simulation performance and fast, smooth real-robot execution.
Problem
Large VLA models can have high inference latency, making consecutive action chunks difficult to chain smoothly during real-robot rollouts.
Method
Xiaomi-Robotics-0 uses cross-embodiment robot trajectories and vision-language data for pre-training, asynchronous-execution post-training, and timestep-aligned action chunks for deployment.
Results
The model achieves state-of-the-art performance across all simulation benchmarks, including 98.7% average success on LIBERO, and reaches 1.2 pcs/min throughput on Towel Folding.
Takeaways & Limitations
Xiaomi-Robotics-0 supports fast, smooth, continuous, and reactive real-time execution on real robots while retaining broad action-generation and vision-language capabilities.
Takeaways & Limitations
Action-prefix conditioning can cause policy learning to copy prior actions instead of attending to visual and language inputs, producing less reactive policies and degraded performance.
Abstract
from arXiv · showhide
In this report, we introduce Xiaomi-Robotics-0, an advanced vision-language-action (VLA) model optimized for high performance and fast and smooth real-time execution. The key to our method lies in a carefully designed training recipe and deployment strategy. Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, endowing it with broad and generalizable action-generation capabilities while avoiding catastrophic forgetting of the visual-semantic knowledge of the underlying pre-trained VLM. During post-training, we propose several techniques for training the VLA model for asynchronous execution to address the inference latency during real-robot rollouts. During deployment, we carefully align the timesteps of consecutive predicted action chunks to ensure continuous and seamless real-time rollouts. We evaluate Xiaomi-Robotics-0 extensively in simulation benchmarks and on two challenging real-robot tasks that require precise and dexterous bimanual manipulation. Results show that our method achieves state-of-the-art performance across all simulation benchmarks. Moreover, Xiaomi-Robotics-0 can roll out fast and smoothly on real robots using a consumer-grade GPU, achieving high success rates and throughput on both real-robot tasks. To facilitate future research, code and model checkpoints are open-sourced at https://xiaomi-robotics-0.github.io
1 Introduction
Xiaomi-Robotics-0 addresses the latency and continuity challenges of large VLA models with a training recipe and deployment strategy for fast, smooth real-robot execution. It combines cross-embodiment robot and vision-language data with asynchronous-execution techniques, achieving strong simulation and real-robot results.
- Large VLA models face inference latency that can cause pauses and jerky, out-of-distribution motions when chaining actions.
- Xiaomi-Robotics-0 pre-trains on large-scale robot trajectories and vision-language data, then uses post-training techniques for asynchronous execution.The recipe aims to retain the underlying VLM’s visual-language capabilities while improving action generation and rollout continuity.
- 98.7% average success rate is achieved on LIBERO, alongside strong results across SimplerEnv and CALVIN benchmarks.SimplerEnv results include 85.5%, 74.7%, and 79.2% under the reported Google Robot and WidowX evaluations; CALVIN improves from 4.54 to 4.75 and 4.67 to 4.80 on two splits.
- Xiaomi-Robotics-0 achieves high success rates and higher throughput than state-of-the-art methods on Lego Disassembly and Towel Folding.The real-robot tasks require precise, dexterous bimanual manipulation and support smooth real-time execution.
- The pre-trained model matches the underlying pre-trained VLM on several general vision-language and embodied-reasoning benchmarks.
2 Xiaomi-Robotics-0
Xiaomi-Robotics-0 combines a pretrained VLM with a diffusion transformer, trained on robot trajectories and vision-language data, then adapts action generation for asynchronous execution. Deployment stitches overlapping action chunks so robots continue moving during inference while preserving continuity.
- Data: Pre-training combines about 200M robot-trajectory timesteps with more than 80M vision-language samples from general and robot-focused sources.The vision-language corpus covers visual grounding, VQA, image captioning, and embodied tasks; in-house trajectories include Lego Disassembly and Towel Folding.
- Architecture: Xiaomi-Robotics-0 maps images, language, and proprioceptive state to bimanual action chunks using a 4.7B-parameter VLM–diffusion-transformer architecture.The VLM supplies conditioning through its KV cache, while the diffusion transformer generates actions via flow-matching.
- Pre-training: The first pre-training step teaches the VLM action prediction while co-training vision-language data, then freezes the VLM and trains the diffusion transformer with flow matching.The VLM predicts multiple action-chunk candidates and scores, while the second stage trains the DiT on robot trajectories.
- Post-training: Post-training prefixes committed actions to noisy action tokens, while the Λ-shaped mask preserves access to visual-language context and recent action history.An offset distinguishes noisy action tokens from clean prefix tokens; the design addresses copying shortcuts that can reduce reactivity.
- Deployment: Asynchronous deployment keeps executing the current chunk while inferring the next, requiring the committed prefix to cover the inference window.The new chunk starts after the inference-latency offset, ensuring actions remain available throughout inference and enabling seamless transitions.
3 Experiments
Xiaomi-Robotics-0 is evaluated across simulation benchmarks and challenging bimanual real-robot tasks, using comparisons with π0.5 and asynchronous ablations. It achieves state-of-the-art simulation performance and improves real-robot throughput, including on precise Lego manipulation and deformable towel folding.
- Simulation benchmarks: 98.7% average success rate is achieved on LIBERO, while SimplerEnv success rates reach 85.5% under Visual Matching, 74.7% under Variant Aggregation, and 79.2% for WidowX.On CALVIN, the average length of completing five tasks rises to 4.75 on ABC→D and 4.80 on ABCD→D.
- Simulation benchmarks: Xiaomi-Robotics-0 achieves state-of-the-art performance across all three simulation benchmarks and surpasses the compared baselines on the reported metrics.The results are reported for LIBERO, CALVIN, and both Google Robot and WidowX evaluations in SimplerEnv.
- Real-robot experiments: Xiaomi-Robotics-0 achieves the highest throughput in Lego Disassembly, while synchronous methods are slightly better in average success rate.The authors attribute lower asynchronous success rates to less reactive motion, less precise grasps, and brick ejection caused by tension between bricks and gripper fingers.
- Real-robot experiments: 1.2 pcs/min throughput is achieved on Towel Folding, exceeding the 1 pcs/min throughput of π0.5, Xiaomi-Robotics-0 (Sync), and Xiaomi-Robotics-0 (Training RTC).The Training RTC variant can repeatedly execute the flinging motion after grasping multiple towel layers, whereas Xiaomi-Robotics-0 avoids such repetitive failures.
Model ERQA SEED POPE AI2D MMBench MME MMMU TextVQA SciQA ChartQA
The model is evaluated on general vision-language and embodied reasoning benchmarks, where it largely preserves the underlying VLM’s capabilities and performs strongly on hallucination and OCR-related tasks.
- Benchmark results: Xiaomi-Robotics-0 outperforms the comparing VLA baselines on all but one evaluated vision-language benchmark.The benchmark suite covers general question answering, hallucination detection, and other vision-language tasks.
- ERQA: 40.8 versus 40.0 is reported for Xiaomi-Robotics-0 and Qwen3-VL-4B-Instruct, respectively, on ERQA.ERQA is included among the evaluated vision-language and embodied reasoning benchmarks.
- Capability retention: The model preserves vision-language capabilities while trailing slightly behind the underlying pre-trained VLM on most general vision-language benchmarks.It also shows strong performance on POPE object-hallucination evaluations and OCR-related tasks such as AI2D.
- Training ablation: Robot trajectories alone fail to retain general-purpose vision-language knowledge, producing severe catastrophic forgetting without explicit vision-language supervision.The Xiaomi-Robotics-0 variant without vision-language data attains zero performance across all evaluated tasks.
4 Related Work
VLA models unify visual-language understanding and robot action generation, but their large parameter counts create inference latency that disrupts smooth real-robot execution.
- Vision-language-action models: VLA models map observations and language instructions directly to actions across diverse tasks, building on pre-trained vision-language models.Large-scale robot data across embodiments, tasks, and environments supports broad generalization to novel environments, instructions, and objects.
- Execution latency: Inference latency from VLA models with up to billions of parameters causes pauses and discontinuous actions during synchronous real-robot rollouts.Synchronous execution leaves the robot idle until the next inference completes; acceleration and asynchronous execution are alternative strategies discussed in the paper.
5 Conclusions
The paper presents Xiaomi-Robotics-0, a VLA model combining multimodal pre-training with asynchronous-execution post-training for high performance and smooth real-time robot control. It reports state-of-the-art simulation results and strong throughput and success rates on two real-robot bimanual tasks.
- Method: Pre-training on large-scale robot trajectories and vision-language data supports action generation while preventing catastrophic forgetting of the underlying VLM’s visual-semantic knowledge.The paper evaluates the retained vision-language capabilities on a broad benchmark suite.
- Deployment: The model uses post-training techniques for asynchronous execution and aligns consecutive action chunks to support continuous, reactive real-time execution.The paper evaluates these capabilities on real robots using a consumer-grade GPU.
- Results: State-of-the-art performance is reported across all simulation benchmarks, alongside high success rates and strong throughput on two real-world bimanual tasks.The real-robot tasks require precise and dexterous manipulation.
- Future work: The paper identifies larger and more diverse robot datasets and improved real-world robustness and generalization as future directions.This is presented as planned future work rather than an evaluated result.
Contributions
The paper lists its authors in alphabetical order, spanning 24 named contributors.
- The authors are explicitly listed in alphabetical order.
- The author list begins with Rui Cai and Jun Guo and continues through Fei Ma.
- The author list concludes with Yuan Zhang, Quanyun Zhou, and other contributors listed before them.
A Case Study for Preservation of Vision-Language Capabilities
The case study presents qualitative vision-language results and comparisons for Xiaomi-Robotics-0, including challenging failure cases involving numerical reasoning and formatting.
- The qualitative materials include dense captioning, fine-grained classification, and embodied grounding tasks.
- Figures 7 and 8 present qualitative results of Xiaomi-Robotics-0 on vision-language tasks.
- Figure 9 provides a qualitative comparison of Xiaomi-Robotics-0 against baseline methods.
- The highlighted failure cases concern complex numerical reasoning on dense charts and minor format-following errors in counting tasks.One example is outputting words instead of digits when counting.
B Detailed results on SimplerEnv
The SimplerEnv results are organized into separate evaluations for WidowX and Google Robot platforms.
- Table 4 reports results on the WidowX evaluations of SimplerEnv.
- Together, the tables organize SimplerEnv results across WidowX and Google Robot evaluations.
- Table 5 reports results on the Google Robot evaluations of SimplerEnv.
C VLM Benchmark Details
The vision-language benchmark summary uses 10 diverse benchmarks to assess broad capability areas, including hallucination, reasoning, and fine-grained perception.
- The benchmark summary selects a diverse set of 10 benchmarks.
- The benchmarks cover comprehensive capabilities, hallucination, reasoning, and fine-grained perception.
- The benchmark suite is used to evaluate the model’s vision-language capabilities.