Source-linked AI summary
InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policy
Yang Tian, Yuyin Yang, Yiman Xie, Zetao Cai, Xu Shi, Ning Gao, Hangxu Liu, Xuekun Jiang, Zherui Qiu, Feng Yuan, Yaping Li, Ping Wang, Junhao Cai, Jia Zeng, Hao Dong, Jiangmiao Pang
TL;DR
Large-scale real-robot data enables strong VLA generalization, but synthetic data had not shown comparable pre-training effectiveness at scale. The paper introduces InternData-A1 through an autonomous, compositional simulation pipeline and finds synthetic-only pre-training matches the official π0 across broad evaluations, while noting simulator limits on highly dexterous tasks.
Problem
Real-robot data is effective for VLA pre-training but expensive to collect, while existing synthetic datasets have limited diversity and validation at scale.
Method
The paper constructs InternData-A1, a 630k-trajectory, 7,433-hour synthetic dataset using a fully decoupled pipeline for assets, skills, task composition, and rendering.
Results
A π0 model trained exclusively on InternData-A1 matches the official π0 across 49 simulation, 5 real-world, and 4 long-horizon dexterous tasks; ten simulated tasks exceed 50% average direct sim-to-real success.
Takeaways & Limitations
Large-scale simulation can match leading real-world data for VLA pre-training and provide an open, reproducible source for embodied AI research.
Takeaways & Limitations
Physics simulators make highly dexterous tasks such as tying shoelaces or threading a needle challenging to simulate.
Abstract
from arXiv · showhide
Recent works explore how real and synthetic data contribute to Vision-Language-Action (VLA) models' generalization. While current VLA models have shown the strong effectiveness of large-scale real-robot pre-training, synthetic data has not previously demonstrated comparable capability at scale. This paper provides the first evidence that synthetic data alone can match the performance of the strongest $π$-dataset in pre-training a VLA model, revealing the substantial value of large-scale simulation. The resulting model also exhibits surprisingly zero-shot sim-to-real transfer on several challenging tasks. Our synthetic dataset, InternData-A1, contains over 630k trajectories and 7,433 hours across 4 embodiments, 18 skills, 70 tasks, and 227 scenes, covering rigid, articulated, deformable, and fluid-object manipulation. It is generated through a highly autonomous, fully decoupled, and compositional simulation pipeline that enables long-horizon skill composition, flexible task assembly, and heterogeneous embodiments with minimal manual tuning. Using the same architecture as $π_0$, we pre-train a model entirely on InternData-A1 and find that it matches the official $π_0$ across 49 simulation tasks, 5 real-world tasks, and 4 long-horizon dexterous tasks. We release the dataset and will open-source the generation pipeline to broaden access to large-scale robotic data and to lower the barrier to scalable data creation for embodied AI research.
1. Introduction
InternData-A1 addresses whether sufficiently large, diverse, high-fidelity synthetic data can match real-robot data for VLA pre-training. The paper introduces an autonomous compositional dataset and reports comparable performance across simulation and real-world evaluations, plus direct sim-to-real transfer.
- Real-robot data supports strong VLA generalization, but its collection requires skilled operators, specialized hardware, and extensive labor.
- Existing simulated datasets generally cover narrow skills, focus on rigid objects, require human operation, and rarely validate large-scale VLA pre-training.
- 630k trajectories and 7,433 hours span 4 embodiments, 18 skills, 70 tasks, and 227 scenes covering rigid, articulated, deformable, and fluid interactions.
- InternData-A1 uses a fully decoupled, autonomous pipeline that separates asset specification, skill policies, task composition, and rendering.
- A π0 model trained exclusively on InternData-A1 achieves comparable performance to the official π0 trained on the closed-source π-dataset.
- Ten selected simulated tasks achieve an average direct sim-to-real success rate exceeding 50%.
- The dataset and generation pipeline are open-sourced to broaden access to robotic data and support study of data scaling effects.
2. Related Works
Prior robot datasets include simulation-based and real-world collections, while VLA models differ in how they use these sources. InternData-A1 is presented against limitations in manual effort, narrow skill coverage, and scale, with 630k episodes and 401.4M frames.
- Existing large-scale robot datasets are divided into simulation-based and real-world categories.
- Simulation offers controllable variation and automated generation, but teleoperation remains labor-intensive and most automated systems target picking tasks.
- Real-data-only VLA models can achieve strong in-domain performance but often remain tied to specific training environments.
- Simulation-only models benefit from massive trajectories and domain randomization but typically cover narrow skill families.
- InternData-A1 contains 630k episodes and 401.4M frames across 4 embodiments, 70 tasks, 3,185 rigid objects, 321 articulated objects, 20 garments, and 227 rooms.
3. Data Statistics
InternData-A1 is organized for trajectory- and task-level diversity across embodiments, manipulation skills, object types, and indoor scenes. Its task and asset design extends beyond simple object variation through compositional manipulation primitives.
- The dataset statistics are organized around embodiments, tasks, and assets, with overall scale and diversity visualized in Figure 2.
- Embodiments: InternData-A1 includes four robotic embodiments spanning dual-arm and single-arm manipulation.
- Tasks: 70 tasks compose manipulation primitives such as folding, pouring, rotating, and stacking across 4 fluid, 4 deformable, 15 articulated, and 47 rigid-object tasks.
- Assets: The asset collection includes 3,185 rigid objects from 107 categories and 321 articulated objects across 14 categories.
- The pipeline is designed around selected embodiments, scenes, and objects, then task composition, domain randomization, and trajectory generation.
- Assets: The environments contain 20 digitized garments and 227 indoor scenes including kitchens, study rooms, dining rooms, and living rooms.
4. Data Synthesis
The data synthesis system separates environment construction, modular skill composition, randomization, and trajectory generation to produce diverse manipulation demonstrations efficiently. Its decoupled architecture addresses planning-rendering bottlenecks and improves generation throughput.
- Environment Construction: Environment construction retrieves verified robot, scene, and object assets from libraries with annotations for manipulation areas, physical properties, joints, and contacts.
- Skill Composition: Users compose tasks from atomic scripted skills that map robot and object states plus constraints to target end-effector 6D-pose waypoints.
- Domain Randomization: Camera views, lighting, object identities, layouts, positions, and orientations are randomized to increase visual and trajectory diversity.
- Generation and Storage: CuRobo interpolates dense joint-space actions between skill waypoints while the system records multimodal observations, proprioception, actions, and optional annotations.
- System Optimization: Integrating planning and rendering in one stage wastes computation because planning success declines with task complexity and the stages have different computational characteristics.
- System Optimization: The optimized system decouples planning and rendering, pipelines execution, dynamically schedules resources, and parallelizes batch processing.
- System Optimization: 2–3× end-to-end performance improvement over the baseline enables stable, efficient large-scale synthetic data generation.
5. Pre-training Effects
The paper evaluates InternData-A1 pre-training against the official π0 model trained on π-dataset across simulation, regular real-world, and dexterous tasks. InternData-A1 matches or exceeds the baseline across these evaluations, including novel embodiments and open-source dataset comparisons.
- Experimental setup: The evaluation benchmarks the same π0 architecture using InternData-A1-only pre-training against the official π0 checkpoint across simulation and real-world tasks.Simulation uses 49 RoboTwin 2.0 bimanual tasks, while real-world tests cover three embodiments and nine tasks.
- Regular real-world evaluation: Across five regular real-world tasks, InternData-A1 outperforms π-dataset by a 6.2% margin across in-distribution and out-of-distribution settings.The tasks span articulation, pick-and-place, human–robot interaction, and tool use across two embodiments.
- Dexterous evaluation: Four long-horizon dexterous tasks show performance comparable to π-dataset despite novel objects, skills, and the unseen ARX AC One embodiment.The tasks include garment folding, industrial-part sorting, bottle-cap unscrewing, and bag zipping.
- Open-source dataset comparison: InternData-A1 achieves an average improvement of 57.7% over RoboCasa in real-world evaluations.The paper attributes this gain to highly photorealistic rendering and abundant data amounts.
6. Data Analysis
The analysis evaluates simulation-to-real transfer and the contributions of InternData-A1’s task components. Results indicate that diverse, compositional, and articulated data are especially valuable for VLA pre-training and can support real-world transfer without real data.
- Sim-to-Real Evaluation: InternData-A1 is evaluated for transfer across simulation and real-world tasks, including four representative sim-to-real experiments and six additional tasks.The study also examines how the dataset’s composition affects pre-training.
- Sim-to-Real Evaluation: 50%, 63%, 87%, 60%, and 57% success rates are achieved on six additional real-world tasks using 500 simulated episodes.These tasks span pick-and-place, articulation, and bimanual coordination.
- Sim-to-Real Evaluation: Direct simulation-to-real transfer extends beyond single pick tasks to multi-skill operations, articulation, bimanual manipulation, and Flip Package.The paper presents this as the first demonstration of such diverse and complex VLA transfer from simulation to reality.
- Data Component Ablation: Removing any of the PnP, Art, Base, or Long components causes a clear performance drop, while the full-data model achieves the highest success rates.The ablation uses four dataset components: pick-and-place, articulation, base, and long-horizon tasks.
- Data Component Ablation: Removing Base or Long tasks causes a larger decline than removing PnP, and excluding articulation also causes a larger drop than excluding PnP.The authors hypothesize that trajectory diversity may be the core driver of effective pre-training, while noting that rigorous investigation remains future work.
7. Conclusion and Limitation
The paper presents InternData-A1 as a large-scale synthetic dataset whose exclusive use for VLA pre-training can match the leading model. It releases the dataset and generation pipeline, while acknowledging simulator limitations for highly dexterous tasks.
- InternData-A1 is a large-scale, high-fidelity synthetic dataset for robotic manipulation.
- Pre-training a VLA model exclusively on InternData-A1 can match the performance of the leading VLA model.
- Open-sourcing the dataset and generation pipeline aims to lower the barrier to large-scale robotic data access for embodied AI research.
- Physics-simulator limitations make highly dexterous tasks such as tying shoelaces or threading a needle challenging to simulate.
Base Tasks (35.95%) 229,168
The supplied statistics identify pick-and-place tasks as 30.61% of the dataset and report overall trajectory and frame totals. The remaining extracted entries list task names and embodiment-related headers without complete task-specific statistics.
- 30.61% of the dataset comprises pick-and-place tasks, totaling 195,133 trajectories.
- 637,498 overall trajectories are reported in the extracted dataset statistics.
- 401,430,981 overall frames are reported in the extracted dataset statistics.
A. Detailed Data Statistics
InternData-A1 contains 4 embodiments, 70 tasks, 637,498 trajectories, 401,430,981 frames, and 7,433.91 hours of interaction data. The dataset is divided into articulation, long-horizon, base, and pick-and-place categories.
- 637,498 trajectories, 401,430,981 frames, and 7,433.91 hours comprise the complete dataset.
- 4 embodiments and 70 tasks are included in InternData-A1.
- Articulation, Long-horizon, Base, and Pick and Place contain 74,415, 138,782, 229,168, and 195,133 trajectories, respectively.Their shares are 11.67%, 21.77%, 35.95%, and 30.61%, respectively.
B. Detailed Data Synthesis
The synthesis framework builds tasks from configurable environments, embodiments, assets, and composable skills, while randomization broadens trajectory diversity. It supports sequential and parallel skill execution with limited manual tuning for stable manipulation.
- Environment and Task Construction: Tasks retrieve a room, robot, and task-relevant assets from configurable libraries before assigning motion-planning and manipulation specifications.Assets are automatically annotated with gravity parameters, collision properties, and grasp poses.
- Skill Composition: Sequential or parallel composition of modular skills assembles long-horizon tasks from actions such as pick, goto_pose, gripper_action, home, and place.Users can copy and paste skill blocks to construct complete tasks.
- Skill Composition: Manual grasp filtering and offset parameters refine scripted policies for stable grasping and placement while avoiding unsafe motions.Configurable filters include filter_x_dir, filter_y_dir, and filter_z_dir, alongside post-grasp and placement offsets.
- Domain Randomization: Lighting, camera poses, rooms, object replacements, initial poses, joint states, and grasp or placement poses are randomized to increase trajectory diversity.Camera perturbations reach up to 5° in rotation and 5 cm in translation.
C. Policy Training Details
The policy is trained from scratch at the action-expert level on InternData-A1, then post-trained and evaluated across real-world and sim-to-real tasks. Evaluation uses repeated rollouts across multiple settings and reports average success rates.
- Pre-training and Post-training: The model starts from Paligemma weights and a randomly initialized action expert before InternData-A1 pretraining.
- Pre-training and Post-training: Regular and sim-to-real tasks receive 30k post-training iterations on 8 GPUs from the 680k π0(InternData-A1) checkpoint.The supplied passage specifies this post-training setup for 10 sim-to-real experiments and 9 real-world tasks.
- Evaluation Protocol: Each task uses 15 evaluation settings with two trials per setting, producing 30 rollouts and an average success-rate report.
D.2. Sim-to-real Task Description
The sim-to-real evaluation includes multi-step manipulation tasks spanning object handling, transfer, sorting, cleaning, assembly, and placement. Task configurations specify assets, robot embodiments, cameras, instructions, randomization, and motion parameters.
- Task Descriptions: Flip Package requires ARX Lift-2 to flip a package, return it to the conveyor, then transfer it across arms for QR scanning.Success requires all listed steps to be completed correctly.
- Task Descriptions: Wipe Stain requires ARX Lift-2 to remove stains with a towel, while Sandwich requires sequential bread, beef, and bread placement.The sandwich is successful only when assembled correctly and neatly.
- Task Descriptions: Additional tasks cover box closure, microwave closure, packing objects, sweeping, and transferring a long object between arms into a box.These tasks use complete-closure, complete-placement, or successful-transfer criteria.
- Task Configuration: The Sort Tray On Rack configuration specifies a SplitAloha robot, dual-arm joint homes, plate and plate-shelf assets, camera randomization, language instructions, and a 4000-step episode limit.The configuration includes randomized lighting and object placement, grasp filters, gripper actions, and vertical placement constraints.