Source-linked AI summary
RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuanbing Xie, Qiangyu Chen, Kailun Su, Tianling Xu, Guodong Liu, Mengkang Hu, Huan-ang Gao, Kaixuan Wang, Zhixuan Liang, Yusen Qin, Xiaokang Yang, Ping Luo, Yao Mu
TL;DR
Robust bimanual manipulation needs scalable data generation because existing datasets lack sufficient task diversity and realistic environmental variation. RoboTwin 2.0 combines annotated assets, MLLM-driven closed-loop code synthesis, embodiment-aware adaptation, and structured domain randomization, and reports improved policy robustness and generalization across tasks and embodiments.
Problem
Existing synthetic-data pipelines provide insufficient scalable task generation and realistic variation for robust bimanual manipulation.
Method
RoboTwin 2.0 uses RoboTwin-OD, MLLM-based task-code synthesis with simulation feedback, embodiment-aware adaptation, and five-axis domain randomization.
Results
RoboTwin 2.0 improves policy robustness to cluttered environments, generalization to unseen tasks, and cross-embodiment manipulation.
Takeaways & Limitations
The framework provides scalable data generation and unified evaluation resources for robust bimanual manipulation research.
Abstract
from arXiv · showhide
Simulation-based data synthesis has emerged as a powerful paradigm for advancing real-world robotic manipulation. Yet existing datasets remain insufficient for robust bimanual manipulation due to (1) the lack of scalable task generation methods and (2) oversimplified simulation environments. We present RoboTwin 2.0, a scalable framework for automated, large-scale generation of diverse and realistic data, together with unified evaluation protocols for dual-arm manipulation. At its core is RoboTwin-OD, an object library of 731 instances across 147 categories with semantic and manipulation-relevant annotations. Building on this, we design an expert data synthesis pipeline that leverages multimodal language models (MLLMs) and simulation-in-the-loop refinement to automatically generate task-level execution code. To improve sim-to-real transfer, RoboTwin 2.0 applies structured domain randomization along five axes: clutter, lighting, background, tabletop height, and language, enhancing data diversity and policy robustness. The framework is instantiated across 50 dual-arm tasks and five robot embodiments. Empirically, it yields a 10.9% gain in code generation success rate. For downstream policy learning, a VLA model trained with synthetic data plus only 10 real demonstrations achieves a 367% relative improvement over the 10-demo baseline, while zero-shot models trained solely on synthetic data obtain a 228% gain. These results highlight the effectiveness of RoboTwin 2.0 in strengthening sim-to-real transfer and robustness to environmental variations. We release the data generator, benchmark, dataset, and code to support scalable research in robust bimanual manipulation. Project Page: https://robotwin-platform.github.io/, Code: https://github.com/robotwin-Platform/robotwin/.
1 Introduction
RoboTwin 2.0 addresses the limited scalability, diversity, quality control, and embodiment coverage of existing synthetic data pipelines for robust bimanual manipulation. It combines automated expert generation, structured randomization, embodiment-aware adaptation, and shared resources for training and evaluation.
- Existing bimanual datasets lack sufficient variation in objects, clutter, lighting, language, and robot embodiments, causing policies to overfit narrow distributions.
- Simulation pipelines remain limited by weak quality control, superficial domain randomization, and insufficient cross-embodiment variation.
- RoboTwin 2.0 combines MLLM-based code synthesis with simulation feedback, five-axis domain randomization, and embodiment-aware action adaptation.
- 731 annotated object instances span 147 categories, while over 100,000 expert trajectories cover 50 tasks across five dual-arm platforms.
- The released generator, dataset, asset library, and benchmark support training and evaluation of bimanual policies under cluttered, diverse, and open-ended conditions.
2 Method
RoboTwin 2.0 generates expert bimanual manipulation data by combining MLLM code synthesis, simulation-in-the-loop refinement, embodiment adaptation, and structured environmental and language randomization. Its pipeline uses annotated assets and feedback to produce executable trajectories across diverse tasks and conditions.
- The pipeline grounds MLLM-generated task programs in RoboTwin-OD and a skill API, producing domain-randomized trajectories for training and evaluation.
- 2.1 Expert Code Generation via MLLMs and Simulation-in-the-Loop Feedback: A closed-loop code agent and VLM observer synthesize, execute, diagnose, and iteratively repair task programs in simulation.
- 2.1 Expert Code Generation via MLLMs and Simulation-in-the-Loop Feedback: The system supports zero-shot generation of complex dual-arm behaviors beyond primitive pick-and-place actions.
- 2.1 Expert Code Generation via MLLMs and Simulation-in-the-Loop Feedback: Generated programs are evaluated across repeated simulated trials using execution logs and multimodal failure localization to produce robust expert trajectories.
- 2.2 Domain Randomization for Robust Robotic Manipulation: Domain randomization varies clutter, textures, lighting, tabletop heights, and language instructions to broaden training distributions and improve unseen-scenario generalization.
3 RoboTwin 2.0 Data Generator, Benchmark and Large Scale Dataset
RoboTwin 2.0 combines the RoboTwin-OD object library with automated task generation to support diverse, cross-embodiment bimanual manipulation.
- RoboTwin-OD: RoboTwin-OD contains 731 objects across 147 categories with semantic, language, and manipulation-relevant annotations.The dataset includes object descriptions and affordance information such as placement points, functional points, grasp points, and grasp axes.
- RoboTwin-OD: The object library combines in-house reconstructed assets with objects from Objaverse and SAPIEN PartNet-Mobility.These sources provide physically accurate collision models and additional visual, semantic, and articulated-object diversity.
- RoboTwin-OD: RoboTwin-OD annotations and manipulation APIs enable semantically valid clutter placement and generalizable grasp execution in simulation.Placement annotations support collision-aware insertion of distractors, while grasp points and axes explicitly encode object affordances.
- Tasks and benchmark: RoboTwin 2.0 provides 50+ dual-arm collaborative tasks and supports data collection and evaluation across five robot platforms.The benchmark is designed for cross-embodiment evaluation, with representative task keyframes shown in Figure 8.
- Tasks and benchmark: The framework pre-collected over 100,000 dual-arm manipulation trajectories across 50 tasks.These trajectories are released as part of the RoboTwin 2.0 dataset.
4 Experiment
Experiments evaluate automated expert-code generation, policy robustness under domain variation, real-world sim-to-real transfer, and cross-task benchmarking. Feedback-based generation and domain-randomized data consistently improve success and generalization.
- Automated expert code generation: RoboTwin 2.0 multimodal feedback raises ASR from 62.1% to 71.3% over Vanilla generation.In RoboTwin 1.0, the corresponding increase is from 47.4% to 63.9%; RoboTwin 2.0 also converges faster under MM FB, with 1.76 versus 2.42 CR-Iter.
- Automated expert code generation: Feedback narrows the success-rate distribution and centers RoboTwin 2.0 multimodal-feedback results above 80%.The result indicates more compact success behavior across generated programs.
- Automated expert code generation: Combining execution logs with vision-language diagnostics detects failures, guides repairs, and produces more reliable, semantically aligned expert data.The code-generation agent uses quantitative and perceptual feedback to iteratively refine task programs.
- Embodiment-aware grasp augmentation: Embodiment-aware grasp augmentation improves success by 13.5%, 22.7%, and 5.6% on Aloha-AgileX, Piper, and ARX-X5, respectively.The gains are attributed to additional feasible grasp options that mitigate planning limitations of lower-DoF platforms.
- Policy robustness: RDT and Pi0 pretrained with RoboTwin 2.0 achieve relative improvements of 31.9% and 29.3% under domain-randomized evaluation.Clean-data fine-tuning produces negligible average-success improvements, whereas domain-randomized pretraining improves generalization even when downstream training uses clean data.
- Sim-to-real evaluation: Adding 1,000 domain-randomized synthetic trajectories to 10 real demonstrations improves average real-world success by 24.4%.Per-configuration gains are 13.5%, 27.5%, 23.5%, and 33.0%; synthetic-only training improves unseen-background scenarios by 21.0% and 20.5%.
- RoboTwin 2.0 benchmark: Across 50 benchmark tasks, pretrained RDT and Pi0 are more resilient than non-pretrained models under Hard conditions, but their success drops by 20.8% and 30.1% from clean to randomized settings.The benchmark also reports strong few-shot performance from DP3, partly associated with perfect point clouds and clean background segmentation in simulation.
5 Related Work
Prior manipulation platforms and datasets provide scale, interaction, language, or domain diversity, but RoboTwin 2.0 targets automated, dual-arm, cross-embodiment generation with systematic randomization.
- Simulation platforms: Existing simulators span articulated-object interaction, large-scale demonstrations, multitask learning, language conditioning, lifelong learning, and domain randomization.RoboCasa provides large-scale human demonstrations but lacks automation and a dual-arm focus.
- Real-world datasets: Large real-world datasets contribute millions of trajectories across diverse tasks, robots, and environments for sim-to-real research.Examples include AgiBot World, RoboMIND, Open X-Embodiment, and Bridge.
- RoboTwin lineage: RoboTwin 2.0 extends RoboTwin-1.0 with LLM-driven feedback and systematic randomization across visual, physical, and task dimensions.The stated goal is richer corpora for policy robustness and generalization.
- Policy generalization: Task-specific policies often struggle to transfer across embodiments, whereas foundation models trained on multi-robot corpora support stronger zero-shot generalization.The related work contrasts specialized architectures with models such as RT-1 and RT-2.
- RoboTwin 2.0: RoboTwin 2.0 pairs digital-twin data collection with extensive domain randomization to mirror real robot dynamics and train generalizable bimanual policies.This positions the framework as a bridge between simulation fidelity and policy robustness.
- Domain randomization: Prior randomization methods vary visual and physical parameters but are described as isolated and lacking bidirectional digital-twin integration.The paper motivates a more integrated approach spanning visual, physical, and task dimensions.
6 Conclusion
RoboTwin 2.0 presents an automated, scalable simulation framework combining MLLM task generation, embodiment-adaptive synthesis, and domain randomization. The reported experiments associate these resources with improved robustness, unseen-task generalization, and cross-embodiment manipulation.
- Framework: RoboTwin 2.0 integrates MLLM-based task generation, embodiment-adaptive behavior synthesis, and comprehensive domain randomization.The system is presented as addressing limitations in prior synthetic-data generators.
- Data generation: An annotated object library and automated trajectory generation provide visual, linguistic, and physical diversity while reducing manual engineering effort.The conclusion connects these design choices to the framework’s data-generation process.
- Empirical findings: Experiments demonstrate improved policy robustness in cluttered environments, generalization to unseen tasks, and cross-embodiment manipulation.These outcomes summarize the reported evaluation scope rather than a single metric.
- Implications: The paper identifies scalable, automated generation of semantically rich, domain-randomized data as important for robust manipulation policies.It presents RoboTwin 2.0 as a foundation for unified benchmarks and scalable sim-to-real pipelines.
- Future work: Future work focuses on real-world deployment and multi-object task complexity.These directions define the stated scope beyond the current framework.
B Benchmarking RoboTwin 2.0 Against Existing Datasets
RoboTwin 2.0 is benchmarked as a scalable, cross-embodiment platform with broad task, object, and simulation variation. Its object-centric design supports heterogeneous dual-arm systems and standardized comparison across datasets.
- RoboTwin 2.0 is compared with prior benchmarks across task count, domain randomization, automatic data generation, and VLA compatibility.
- Domain randomization spans cluttered scenes, lighting, tabletop height, background textures, and unseen language instructions.Tabletop height variation reaches up to 3 cm.
- RoboTwin 2.0 supports flexible combinations of heterogeneous dual-arm manipulators and relative arm placements.The object-centric, embodiment-agnostic design is intended to extend across diverse hardware setups.
- Curobo provides GPU-accelerated motion planning under varied kinematic constraints.
- The framework currently supports Franka, Piper, UR5, ARX-X5, and Aloha-AgileX arms with multiple gripper types.Successful executions are demonstrated across varied dual-arm pairings.
F Improvements of RoboTwin 2.0 over RoboTwin 1.0 Policy Codebase
RoboTwin 2.0 improves the efficiency, structural quality, and refinement behavior of generated manipulation code over RoboTwin 1.0. Its optional VLM observer adds diagnostic feedback for correcting execution failures.
- 569.4 vs. 1236.6 tokens: RoboTwin 2.0 generates substantially shorter programs than RoboTwin 1.0 in one-shot zero-shot generation.It also reduces prompt length and improves similarity to expert-written code.
- +21.06% AST similarity, +1.08% CodeBERT similarity, and +5.97% Unixcoder alignment indicate closer structural and semantic alignment with expert code.
- The VLM observer detects and localizes failures, then supplies qualitative diagnostics alongside execution logs for code repair.The observer is optional and can be disabled when token efficiency is prioritized.
- RoboTwin 2.0 consistently matches or outperforms RoboTwin 1.0 on code-generation success rates across the majority of tasks.
G.4 Multimodal Observation and Error Localization
The VLM observer evaluates simulated task executions, localizes failures, and diagnoses underlying causes, but its reliability is limited by false positives and subtle or invisible errors.
- 130 execution sequences—101 failed and 29 successful—were curated to evaluate binary error detection and fine-grained failure localization.
- Error Detection: Accuracy = 0.431, Precision = 0.208, Recall = 0.552, and F1-score = 0.302 for VLM execution-error detection.The observer was sensitive to failures but frequently over-predicted errors.
- Error Localization: 30% localization accuracy was achieved among 40 correctly identified failed trials, with 12 failures pinpointed to the correct step.The result indicates room for improvement in reliable error attribution.
- Case Study: A case study identified a hanging failure at Step 4 and linked it to incorrect access of position data from a list-valued pose.The diagnosis connected the failed placement to an AttributeError involving middle_target_pose.p.
- Failure Modes: The observer may overlook box misalignment, incorrect shoe orientation, and grasp-axis errors that are subtle or invisible in visual observations.These failure modes constrain purely vision-based diagnosis of fine-grained manipulation errors.
- LLM-Generated Code and Human-Written Code Case Study: MLLM-generated programs are more verbose than human-written scripts, explicitly logging intermediate visual states and parameters while remaining functionally similar.The structural emphasis on step-by-step clarity supports feedback and repair.
- Prompts for Generating Task Instructions and Object Descriptions: Task instruction prompts vary sentence structure and action verbs, while object-description prompts vary phrasing and include physical properties such as color, shape, size, and texture.The prompt design uses placeholders and separates training and evaluation items.
J RoboTwin 2.0 Benchmark Setting Visualization
The benchmark visualization contrasts clean and domain-randomized evaluation environments for assessing bimanual manipulation under different conditions.
- The Easy setting uses clean environments, whereas the Hard setting uses domain-randomized environments during evaluation.Both settings train models on 50 clean, non-randomized demonstrations per task.
K Full RoboTwin 2.0 Benchmark
RoboTwin 2.0 evaluates policies under Easy and Hard conditions that differ in evaluation environments while keeping training identical.
- Five policies are evaluated on the RoboTwin 2.0 benchmark under Easy and Hard settings with identical training setups.The benchmark covers 50+ tasks, and an online leaderboard is maintained.
L Success Rates of Different Embodiments on RoboTwin 2.0 Tasks
The benchmark reports success rates for five robot embodiments across 50 RoboTwin 2.0 tasks using the same expert programs for data generation.
- Success rates are reported for five robot embodiments across 50 RoboTwin 2.0 tasks.All embodiments use the same set of expert programs for data generation.