Source-linked AI summary
Galaxea Open-World Dataset and G0 Dual-System VLA Model
Tao Jiang, Tianyuan Yuan, Yicheng Liu, Chenhao Lu, Jianning Cui, Xiao Liu, Shuiqi Cheng, Jiyang Gao, Huazhe Xu, Hang Zhao
TL;DR
The paper targets limited realism in robot data and unresolved evidence about cross-embodiment pre-training. It introduces a consistently embodied, richly annotated real-world dataset and the G0 dual-system VLM/VLA framework with staged training. Across diverse benchmarks, the authors report strong performance and emphasize the importance of single-embodiment pre-training, while noting that diffusion loss may be unsuitable before stable representations emerge.
Problem
Large-scale robot datasets often lack realistic, environmentally rich data, while the value of cross-embodiment pre-training remains unresolved.
Method
The paper builds Galaxea Open-World Dataset and G0, combining a VLM planner with a VLA executor trained through cross-embodiment, single-embodiment, and task-specific stages.
Results
G0 achieves state-of-the-art performance across diverse benchmarks, with experiments showing strong overall performance and the importance of single-embodiment pre-training.
Takeaways & Limitations
Real-world, consistently embodied data and target-platform pre-training are central to the reported performance of this dual-system approach.
Takeaways & Limitations
Stage-1 training is limited because inconsistent multi-embodiment annotations and actions hinder action-expert learning, while early diffusion loss may disrupt stable representations.
Abstract
from arXiv · showhide
We present Galaxea Open-World Dataset, a large-scale, diverse collection of robot behaviors recorded in authentic human living and working environments. All demonstrations are gathered using a consistent robotic embodiment, paired with precise subtask-level language annotations to facilitate both training and evaluation. Building on this dataset, we introduce G0, a dual-system framework that couples a Vision-Language Model (VLM) for multimodal planning with a Vision-Language-Action (VLA) model for fine-grained execution. G0 is trained using a three-stage curriculum: cross-embodiment pre-training, single-embodiment pre-training, and task-specific post-training. A comprehensive benchmark spanning tabletop manipulation, few-shot learning, and long-horizon mobile manipulation, demonstrates the effectiveness of our approach. In particular, we find that the single-embodiment pre-training stage, together with the Galaxea Open-World Dataset, plays a critical role in achieving strong performance.
1 Introduction
The paper addresses limited realism and environmental richness in robot datasets by introducing the Galaxea Open-World Dataset and G0 dual-system framework. Its benchmark indicates that single-embodiment pre-training is especially important, particularly when pre-training and target robots differ substantially.
- Existing large-scale robot datasets often lack task realism and environmental richness, limiting generalization to diverse real-world contexts.
- Galaxea Open-World Dataset contains 500 hours of data spanning more than 150 tasks and 50 real-world scenes, collected with a single robotic embodiment.Comprehensive filtering and precise language annotations support mobile-manipulation benchmarking.
- G0 couples System 2 multimodal planning with System 1 precise action execution through asynchronously operating VLM and VLA models.Its VLA uses a three-stage curriculum: cross-embodiment pre-training, single-embodiment pre-training, and task-specific post-training.
- Cross-embodiment pre-training benefits diminish or can degrade VLA performance when the pre-training and target embodiments have a large gap.The finding underscores the importance of single-embodiment pre-training.
- The dataset and models are planned for open-source release to support further embodied-AI research.
2 Related Work
Prior work expanded robot-data scale, diversity, or embodiment coverage but often remained controlled or unresolved the value of cross-embodiment pre-training. This paper positions a real-world benchmark as a testbed for analyzing that tradeoff.
- Dual System Designs: Dual-system robotics separates deliberative planning from reactive execution, with VLMs serving as planners and VLAs as low-level executors.The framework follows the System 2/System 1 distinction.
- VLA as the System 1 Executor: Autoregressive VLA generation transfers pretrained VLM knowledge but can be slow, whereas diffusion generation improves throughput but may degrade VLM capabilities.
- VLA as the System 1 Executor: This work uses two training methods similar to π0.5 to combine the benefits of autoregressive and diffusion generation paradigms.
- Large-scale Manipulation Datasets: Existing datasets demonstrated scaling gains through single-embodiment or multi-embodiment collection, but their settings create tradeoffs between platform consistency and task or scenario diversity.
- Large-scale Manipulation Datasets: Many foundational datasets were collected in controlled or artificial settings, creating a domain gap with unstructured real-world environments.Galaxea addresses this gap through large-scale collection in completely unstructured real-world settings.
- Large-scale Manipulation Datasets: The paper uses a high-fidelity real-world benchmark to systematically disentangle how cross-embodiment pre-training contributes to real-world generalization.The efficacy of the common multi-embodiment-to-target-embodiment paradigm remains disputed.
3 Galaxea Open-World Dataset
Galaxea Open-World Dataset is a large, consistently embodied, richly annotated collection spanning diverse real-world scenes, objects, tasks, motions, and skills. Its construction emphasizes observability, quality control, linguistic grounding, and realistic mobile manipulation.
- The dataset contains 100K trajectories across 150 task categories, 50 real-world scenes, more than 1,600 objects, and 58 operational skills.Consistent embodiment aligns perception, action, and language annotations.
- All demonstrations use the mobile, bimanual Galaxea R1 Lite platform with a 23-DoF embodiment and omnidirectional mobile base.The platform includes two arms, a movable torso, and a vector-drive chassis.
- Figure 2 emphasizes collection by identically embodied robots operating across diverse real-world environments.
- Figure 3 organizes diversity by interaction time across Residential, Retail, Catering, and Office scenes, and by trajectory counts across object subcategories.
- The platform combines a stereo RGB head camera with dual wrist RGB-D cameras for scene context and close-range manipulation precision.Its compact dimensions support navigation through narrow spaces.
- Isomorphic teleoperation maps operator movements directly to robot kinematics, keeping arms reachable while avoiding inverse-kinematics failures and morphology retargeting.
- Collection guidelines require observable visual streams, quality-first scaling, and structured subtask-level language annotations.These principles support multimodal alignment for VLA training.
- Figure 4 shows mostly moderate task durations alongside a long tail of complex activities and broad variation in subtask counts.
4 Method
G0 combines a deliberative VLM planner with a reactive VLA executor and trains the executor through a three-stage curriculum. The curriculum progresses from broad cross-embodiment action-token learning to single-embodiment flow-matching and task-specific fine-tuning, while the VLM plans subtasks from high-level instructions.
- Dual System Overview: G0 uses System-2 G0-VLM for high-level planning and System-1 G0-VLA for subtask interpretation and precise action execution.The two models operate asynchronously at different frequencies; G0-VLA conditions continuous action generation on visual observations, language, and proprioceptive state.
- G0-VLA Pre-training Stage-1: Stage 1 trains only the VLM on diverse robot data, using FAST to tokenize continuous action chunks for autoregressive cross-entropy prediction.The training mixture includes approximately 1,000 hours of OXE trajectories, 500 hours of Galaxea data, and 200 hours of in-house data.
- G0-VLA Pre-training Stage-2: Stage 2 adds a newly initialized action expert and trains the VLA on labeled Galaxea data to generate continuous actions with a flow-matching loss.The action expert is conditioned on proprioceptive states and VLM representations; the loss compares predicted and target flows for action chunks.
- G0-VLA Pre-training Stage-2: Galaxea’s single embodiment and subtask-level language-action alignment support more precise actions and stronger grounding during Stage-2 pre-training.Consistent action spaces avoid cross-embodiment adaptation, while fine-grained instruction-trajectory pairs strengthen correspondence between language and actions.
- Task-specific Post-training: Task-specific post-training fine-tunes the VLA with the Stage-2 objective, limiting each downstream task to at most 100 trajectories.The experiment compares VLA models initialized with different pretrained weights to test generalization.
- G0-VLM Training: G0-VLM is instruction-tuned from Qwen2.5-VL using Galaxea episodes, human-annotated subtasks, and synthesized human-style high-level instructions.Key transition frames receive higher sampling weights, and a reasoning LLM generates high-level instructions and robot responses from episode context.
5 Evaluating G0-VLA
The evaluation spans tabletop manipulation, device operation, language following, few-shot transfer, and long-horizon whole-body control. Results consistently identify single-embodiment Stage-2 pre-training as especially valuable, while Stage-1 alone often provides little or negative benefit.
- Benchmark Design: The benchmark covers tabletop manipulation, device operation, blocks stacking, and long-horizon bed making, evaluated using averaged progress scores over 10 trials.Bed Making specifically tests coordinated chassis, torso, and arm control.
- 5.1 Pre-trained Weights: G0 (Full) achieves the highest average progress score, while Stage-2 variants lead language following, action consistency, and whole-body control.G0 (Full) performs particularly well on Table Bussing, Microwave Operation, and Bed Making; G0 (Stage-1) performs worst among pretrained models.
- 5.1 Pre-trained Weights: Stage-1 pre-training enhances simple universal action patterns, whereas Stage-2 grounds the model to the robot platform and improves action stability and instruction following.The comparison distinguishes general action patterns from embodiment-specific specialization.
- 5.2 Few-shot Transfer: Stage-2 pre-training significantly improves few-shot transfer on Table Bussing and Microwave Operation, producing smoother and more stable execution.The few-shot setting uses 20 trajectories per task and identical 10-epoch fine-tuning settings.
- 5.3 Embodiment-specific Actions: Stage-2 substantially improves embodiment-specific chassis and torso skills, while cross-embodiment Stage-1 and π0 can underperform models trained from scratch.The authors attribute this pattern to the embodiment gap between the target robot and cross-embodiment pre-training platforms.
6 Evaluating G0-VLM
The G0-VLM evaluation asks whether fine-tuning improves action grounding beyond pretrained models. Task-specific fine-tuning substantially improves instruction accuracy and produces language instructions that VLAs can execute directly.
- Evaluation Setup: The evaluation compares pretrained Gemini-2.5-pro and Qwen2.5-VL variants with fine-tuned versions to assess action-grounding accuracy.Standardized prompts restrict models to provided options, reducing prompt-design variability.
- Results: Over 50%, fine-tuned-model accuracy surpasses baseline accuracy in benchmark instruction tasks.Table 1 reports instruction accuracy as a percentage.
7 Conclusion
The paper introduces the Galaxea Open-World Dataset and G0, a dual-system model for robotic mobile manipulation. G0 combines VLM planning with VLA execution and achieves state-of-the-art performance across diverse benchmarks.
- Conclusion: The Galaxea Open-World Dataset is a large-scale, high-fidelity, richly annotated resource for robotic mobile manipulation.The conclusion presents it as a resource intended to accelerate research.
- Conclusion: G0 combines a VLM for planning with a VLA model for execution and achieves state-of-the-art performance across diverse benchmarks.The conclusion attributes the system to pre-training on the proposed dataset.
8 Contributors
The contributors are credited across dataset operation, policy training and evaluation, project supervision, and supporting team efforts.
- Contributors: Tao Jiang, Jianning Cui, and Xiao Liu are credited with dataset operation.
- Contributors: Tianyuan Yuan, Yicheng Liu, Chenhao Lu, Shuiqi Cheng, Jianning Cui, Xiao Liu, and Tao Jiang handled policy training and evaluation.
- Contributors: Hang Zhao, Huazhe Xu, and Jiyang Gao provided project supervision.
- Contributors: The authors thank the data collection, hardware, and marketing teams at Galaxea for their support.
A.1 Evaluation Details
The evaluation rubric scores task progress through task-specific subtasks, assigning points to successful actions or stages. Tasks receive different maximum scores based on their required operations.
- The rubric measures progress by decomposing each task into discrete actions or stages with point-based success criteria.
- Table Bussing is scored out of 6 points, with separate points for successful picks and places.
- Microwave Operation is scored out of 5 points across five specified food-handling and microwave steps.
- Bed Making is scored out of 4 points for four movement, grasping, and quilt-flattening stages.
- Blocks Stacking is scored out of 6 points, awarding points for successful picks and stacks.