Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model

• Embodied generation requires multi-view consistency, geometric and physical coherence, robot embodiment constraints, and temporally consistent interactions; robot-only post-training can weaken foundation models' generalization.Generated observations must remain compatible with robot kinematics, camera calibration, and manipulation policies; the paper notes embodied datasets are smaller and less diverse than internet-scale visual corpora.Source: §1 · pp. 1–2
• The model jointly trains on general-domain and embodied data under a unified autoregressive objective across text-to-image generation, image editing, embodied scenes, transfer, and video.This formulation extends foundation image and video generation to embodied synthesis while aiming to preserve visual knowledge and controllability.Source: §1 · pp. 2, 4
• Interleaved embodied sequences and manipulation videos at multiple temporal resolutions support long-horizon task progression, fine-grained interaction dynamics, and coherent rollouts from generated scenes into manipulation videos.The paper frames these rollouts as synthetic trajectories for downstream policy learning.Source: §1 · pp. 2, 4
• Xiaomi-Robotics-U0 outperforms GPT-Image-2.0 in human evaluations of embodied scene generation and transfer, achieves state-of-the-art World Arena performance, and improves downstream policy robustness.The paper also reports zero-shot embodied video generation and improved out-of-distribution robustness using generated data.Source: §1 · p. 4
• Its structured transfer controls workspace, background, irrelevant foreground objects, target objects, and lighting independently to support fine-grained editing while preserving multi-view geometric consistency and interaction dynamics.The framework is described as achieving superior multi-view consistency, controllability, and instruction following.Source: §1 · p. 4

2.1 Problem Formulation

• The model learns the joint distribution of subsequent text, image, and robot-control tokens through standard next-token prediction.This formulation supports a unified multimodal sequence objective.Source: §2.1 · pp. 4–5
• Single-step tasks include text-to-image generation and image editing or generation conditioned on one to three reference images plus text.The latter is categorized as Any-to-Image (X2I).Source: §2.1 · p. 5
• Single-step embodied tasks generate initial multi-view robot observations from embodiment and scene descriptions, or transfer current observations to a target scene.The transfer task predicts multi-view observations conditioned on the current embodied observations and a target scene description.Source: §2.1 · p. 5
• Sequential subtask-subgoal samples interleave each subtask description with the corresponding multi-view observations after completion.Source: §2.1 · p. 5
• Sequential manipulation videos use multiple frame rates to capture both sparse task-level transitions and dense interaction dynamics.Source: §2.1 · p. 5
• All tasks use unified multimodal token sequences and the same autoregressive objective, enabling knowledge sharing across image generation, scene understanding, embodied reasoning, and future prediction.Source: §2.1 · p. 5

2.2 Model Architecture

• Initialized from EMU3.5, Xiaomi-Robotics-U0 uses the Qwen-3-32B decoder-only Transformer and IBQ-tokenized images with 16 × 16 spatial compression; an extended vocabulary represents all modalities as discrete tokens for next-token prediction.The vocabulary extension adds the IBQ codebook to Qwen's original vocabulary.Source: §2.2 · p. 5
• Image generation, embodied scene synthesis and transfer, sequential subtask prediction, and video generation are modeled as one autoregressive sequence for multi-task co-training while preserving inherited visual-generation capabilities.This formulation contrasts with conventional embodied world models that use task-specific prediction heads.Source: §2.2 · pp. 5–6
• Up to 82.9× faster 1024×1024 image generation is achieved with anti-diagonal parallel decoding, optimized KV-cache management, and continuous batching, while preserving most generation quality.The speedup is compared with the original next-token-prediction pattern.Source: §2.2 · p. 6

2.3.1 Dataset Curation

• The six-domain corpus includes general image-text, driving, egocentric, 3D reconstruction, and game data, alongside diverse visual priors.Sources include ShareGPT4V [9], Cosmos-Drive-Dreams [47], EgoWalk [2], ScanNet++ v2 [68] and DL3DV-10K [31], and Open-P2P [71].Source: §2.3.1 · p. 6
• Real and simulated manipulation data, including AgiBotWorld-Beta [8], Open X-Embodiment [15], MiBot, RoboTwin2.0 [11], GenieSim [69], and InternData-A1 [57], is filtered for blur, image quality, and gripper-state alignment.Source: §2.3.1 · p. 6
• 9.5 million single-step samples contain 56.4 billion tokens; sequential-generation data comprises 2.6 million video clips and 49.6 billion tokens.Source: §2.3.1 · p. 6

2.3.2 Dataset Labeling

• Qwen3-VL-235B [3] generates holistic scene captions and five-part embodied scene descriptions; Video Depth Anything [10] supplies temporally consistent inverse-depth conditioning.The five semantic dimensions are workspace, task objects, irrelevant objects, lighting, and background.Source: §2.3.2 · pp. 6, 8
• HDBSCAN clustering segments trajectories at motion and grasp transitions, where interaction-rich keyframes yield temporally aligned subtask plans and scene annotations.Source: §2.3.2 · p. 8, Figure 5 · p. 9
• Task-specific formats train embodied transfer from multi-view depth to RGB, scene generation from descriptions to multi-view RGB, and video generation from instructions and an initial frame to 15–25 RGB frames.Where robot action labels exist, rendered robot-mask sequences provide additional action guidance for video generation.Source: §2.3.2 · p. 8
• Sample-level reweighting by domain and task prioritizes long-tail objects, skills, and interaction phases while downsampling redundant embodiment-task-object combinations.Invalid episodes are excluded, and remaining samples are grouped by task goals, scene context, objects, spatial relations, subtask phases, and manipulation skills.Source: §2.3.2 · p. 8

2.4 Training

• Training uses single-step and sequential continued training to adapt the pre-trained image-generation model while preserving its original visual-generation capability.Source: §2.4 · p. 9

Single-step training

• Single-step training jointly combines T2I, X2I, Embodied Scene Generation, and Embodied Transfer to retain visual and editing capabilities while learning embodied generation.Scene Generation teaches physically plausible initial robot observations conditioned on embodiment and scene descriptions; Transfer supports cross-scene multi-view generation while preserving interaction states and camera geometry.Source: §2.4 · p. 9

Sequential training

• Sequential training extends the model to long-horizon embodied reasoning by interleaving language-described subtasks with post-execution multi-view observations.This format teaches task decomposition and future embodied-observation prediction while aligning the base model’s sequence-modeling capability.Source: §2.4 · p. 9
• Sequential training adds manipulation videos from InternData-A1 [57], GenieSim [69], AgiBotWorld-Beta [8], RoboCOIN [62], and RoboMIND [61] at 1, 3, and 5 FPS.Sparse sequences encourage long-horizon planning and prediction, while denser sequences capture fine-grained manipulation dynamics and continuous interaction, modeling high-level task progression and low-level dynamics.Source: §2.4 · p. 10

2.5.1 FlashAR+ Extension

• Building on FlashAR [73], FlashAR+ extends parallel anti-diagonal image-token generation from T2I to X2I while keeping text and reference inputs as conditioning.X2I includes T2I, I2I, and multi-reference image generation.Source: §2.5.1 · p. 10
• FlashAR+ supervises only the target image, keeping text, references, controls, and its header in the conditional prefix; target tokens attend to prior anti-diagonal steps, not same-step tokens.This mask limits parallel decoding to the target region while preserving standard autoregressive visibility for the conditional prefix.Source: §2.5.1 · pp. 10–11

2.5.2 vLLM Integration

• 450.77 seconds/image for AR fell to 16.56 with FlashAR+ (27.22× speedup), then to 5.44 with vLLM (an additional 3.04× speedup) on 1024 × 1024 T2I with one H20 GPU.vLLM used max_num_seq=28; the comparison reports generation latency per image.Source: Figure 7 · p. 12, §2.5.2 · p. 13

2.5.1 FlashAR+ Extension

• At inference, target-grid positions are generated in increasing r + c order, with same-step positions sampled in parallel; boundary positions use one direction and interior positions use H/V gated fusion.The procedure preserves Any-to-Image conditioning while reducing serial generation cost in the target image region.Source: §2.5 · p. 10, §2.5.1 · p. 12

2.5.2 vLLM Integration

• vLLM handles the conditional prefix, batched execution, and paged KV-cache management, while a diagonal scheduler emits anti-diagonal visual tokens and synchronizes paired guidance requests.The FlashAR+ decoding rule remains unchanged.Source: §2.5.2 · pp. 12–13

3.1 Embodied Transfer

• 36.9% to 63.2%: embodied transfer raises π0.5 success rate and outperforms GPT-Image-2 across objective metrics on both Easy and Hard benchmark splits.Generated views respect input depth and remain consistent in object identity, appearance, and arrangement; zero-shot augmentation also combines diverse generated background, workspace, and lighting descriptions for demonstrations.Source: §3 · p. 13, §3.1 · pp. 13, 15, Table 2 · p. 14

3.2 Embodied Scene Generation

• Xiaomi-Robotics-U0 outperforms GPT-Image-2 in most pairwise comparisons, with clear multi-view consistency advantages and comparable instruction following.It generates photorealistic, geometrically coherent scenes from open-ended prompts across robot arms, including real-world-aligned observations for the AGIBOT G2, which was trained exclusively on simulated data.Source: §3.2 · pp. 18–19, Figure 14 · p. 19

3.3 Real World Experiments

• Under held-out backgrounds and lighting, Xiaomi-Robotics-U0-Aug consistently outperforms Original across tasks, while performing comparably under base conditions.The augmentation style-transfers demonstrations to vary appearance while preserving robot states and action labels; the evaluation includes unseen tablecloths and lighting, including low, colored, and patterned illumination.Source: §3.3 · pp. 21–22, Figure 17 · p. 22

3.4 Video Generation

• 73.64 EWMScore ranks Xiaomi-Robotics-U0 first among over 100 WorldArena submissions; it also leads Instruction Following (93.86), Interaction Quality (87.30), Perspectivity (98.84), and Motion Smoothness (95.51).The benchmark assesses 15 metrics across visual and motion quality, consistency, physics, 3D accuracy, and controllability, using an initial frame, language instruction, and robot-action guidance.Source: §3.4 · p. 24, Figure 21 · p. 25, Table 3 · p. 26
• Generated videos depict plausible robot-object interactions, scene-level dynamics, and material-dependent behavior, including a paper sheet that bends realistically and lags during lateral movement.Examples also show a robot grasping, lifting, and placing a cup, and a simulated person walking toward the table while the robot operates.Source: §3.4 · pp. 24–25, Figure 20 · p. 25

3.5 Text2Image & Image Edit Benchmarks

• Xiaomi-Robotics-U0 retains strong general text-to-image and image-editing capability after embodied post-training, with comparable performance on common generation and local editing tasks.It slightly lags Qwen-Image in highly complex cases involving multi-constraint composition or precise spatial alignment; performance drops are largely confined to complex composition, spatial grounding, or symbolic binding.Source: §3.5 · pp. 25–26, Table 4 · p. 26

4.1 Foundation Generative Models and Embodied World Models

• Robot-centric world models predict future observations but rely on smaller robot-only datasets that can erode foundation models’ semantic richness, controllability, and visual diversity.Xiaomi-Robotics-U0 instead jointly optimizes general-domain generation and embodied synthesis, preserving original generation capabilities while acquiring robot-centric geometric reasoning and interaction modeling.Source: §4.1 · pp. 26–27

4.2 Controllable and Multi-view Embodied Generation

• Prior image-editing [6] [12] [63] and multi-view methods [53, 33, 35, 29, 52, 19, 54] lack robot-aware calibrated interaction, which our unified framework addresses with disentangled scene controls.The controls independently vary workspace layout, background appearance, foreground irrelevant objects, target objects, and lighting while preserving robot geometry, embodiment constraints, and multi-view consistency.Source: §4.2 · p. 27

4.3 Embodied Video Generation and Synthetic Data Engines

• Existing systems separately produce images, simulation assets, or short robot videos, while our autoregressive model unifies embodied scene generation, transfer, interleaved subtask-subgoal prediction, and video generation.Foundation video models [46] [59] [1] [7] [72], robot datasets [40] [39] [58] [25], and embodied data pipelines [61] [62] [69] [57] [8] have advanced the field; multi-resolution video learning captures long-horizon task progression and fine-grained interaction dynamics.Source: §4.3 · p. 27

5 Conclusions

• The model achieves state-of-the-art performance across single-step and sequential embodied generation, outperforming GPT-Image-2.0 in human evaluation and improving downstream policy robustness with generated data.It also achieves state-of-the-art results on the World Arena benchmark.Source: §5 · p. 28
• Built on a world foundation model through lightweight continual training, Xiaomi-Robotics-U0 unifies text-to-image generation, editing, multi-view scenes, embodied transfer, and embodied video in one autoregressive framework.The authors describe it as the first unified embodied synthesis model to support both high-quality multi-view embodied generation and sequential embodied video generation, and the first to introduce structured control for scalable embodied data generation.Source: §5 · p. 28
• Embodied transfer relies on depth estimation, which can cause artifacts and constrain fine-grained control of object textures and local appearance.The authors propose direct multi-view image editing on raw observations as future work.Source: §5 · p. 28
• Generating scenes and videos separately makes long-horizon rollouts susceptible to accumulated errors; jointly generating scenes and subsequent videos is a proposed direction.Source: §5 · p. 28
• The current 32K context window limits long-duration video modeling, while extending context could enable minute-level embodied interaction generation.Source: §5 · p. 28