GR00T N1: An Open Foundation Model for Generalist Humanoid Robots

• The paper frames generalist humanoid autonomy as a full-stack challenge requiring capable hardware, expressive models for open-ended tasks, and scalable training data.Source: §1 · p. 1
• Humanoid data are scarce, and differences across robots fragment cross-embodiment collections into “data islands” rather than a coherent Internet-scale pretraining dataset.Source: §1 · pp. 1–2
• GR00T N1 is a cross-embodiment VLA with a 10Hz VLM System 2 for task interpretation and a jointly optimized Diffusion Transformer System 1 generating closed-loop actions at 120Hz.System 1 uses embodiment-specific encoders and decoders to handle variable state and action dimensions.Source: §1 · p. 2
• Its data pyramid combines large-scale web data and human videos, synthetic data, and real-robot data to provide broad priors and embodied execution grounding.Data quantity decreases and embodiment-specificity increases toward the pyramid’s real-robot top.Source: §1 · p. 2, Figure 1 · p. 2
• A co-training strategy annotates action-less videos using a latent-action codebook and inverse dynamics model, then samples across video, synthetic, and real-robot data for end-to-end pretraining.The unified training format pairs robot state, visual observations, and language instructions with motor actions.Source: §1 · p. 2
• GR00T N1 outperforms state-of-the-art imitation-learning baselines on standard simulation benchmarks and performs strongly in real-world GR-1 humanoid experiments.Source: §1 · p. 2

2. GR00T N1 Foundation Model

• GR00T N1 is a VLA model that couples an Eagle-2 vision-language backbone with a DiT flow-matching policy for high-frequency humanoid actions.The publicly released GR00T-N1-2B has 2.2B parameters, including 1.34B in the VLM, and samples 16 actions in 63.9ms on an L40 GPU using bf16.Source: §2 · p. 3

2.1. Model Architecture

• A DiT processes embodiment-aware state and action inputs, then cross-attends to Eagle-2 vision-language tokens to predict denoised motor actions.The architecture integrates a VLM-based reasoning module (System 2) and a DiT-based action module (System 1) in a unified learning framework.Source: §2.1 · pp. 3–5, §2 · p. 3, Figure 3 · p. 4
• H=16 action chunks are denoised iteratively; GR00T-N1-2B uses 12th-layer VLM representations, and four inference steps work across embodiments.The authors report that middle-layer rather than final-layer embeddings improved inference speed and downstream policy success.Source: §2.1 · pp. 4–5

2.2. Training Data Generation

• A VQ-VAE supplies shared latent-action labels for human videos and neural trajectories, while video generation expands 88 hours of teleoperation data to 827 hours.The generated trajectories use novel language prompts and counterfactual scenarios; the VQ-VAE is trained across heterogeneous data to unify its latent action space.Source: §2.2 · pp. 5–6
• 780,000 simulated manipulation trajectories, equivalent to 6,500 hours of human demonstrations, were generated in 11 hours using DexMimicGen.DexMimicGen expands a small set of human demonstrations through simulation, retaining only successful demonstrations.Source: §2.2 · pp. 6–7

2.3. Training Details

• Pre-training uses flow matching across robot, synthetic, and human data; post-training fine-tunes per embodiment while keeping the VLM language component frozen.For neural-trajectory post-training, the policy is co-trained with real and generated trajectories at a 1:1 sampling ratio; GR00T-N1-2B used roughly 50,000 H100 GPU hours for pre-training.Source: §2.3 · p. 8

3. Pre-Training Datasets

• The corpus distinguishes real-robot, synthetic, and human-video sources, while Table 1 maps data-generation strategies to their applicable sources.Latent-action learning applies broadly to video datasets; neural trajectories can be generated from datasets with robot actions, while simulation trajectories use a physics simulator and DexMimicGen.Source: §3 · p. 9, Table 1 · p. 9

3.1. Real-World Datasets

• Real-world data combines Fourier GR-1 teleoperation, cross-embodiment manipulation datasets, and 140,000 AgiBot-Alpha trajectories.The GR00T N1 dataset records robot actions, head-camera images, human proprioception, and fine- and coarse-grained task annotations at 20Hz. Open X-Embodiment Collaboration et al. (2024) contributes diverse robot datasets; AgiBot-World-Contributors et al. (2025) provides trajectories covering manipulation, tool use, and multi-robot collaboration.Source: §3.1 · p. 9

3.2. Synthetic Datasets

• DexMimicGen produces 540,000 simulated demonstrations across 54 source-target receptacle combinations, with randomized layouts and distractors that require language-guided rearrangement.A few dozen teleoperated demonstrations are segmented around objects and transformed and combined to generate 10,000 demonstrations for each receptacle pair; the tasks use realistic 3D assets in RoboCasa (Nasiriany et al., 2024).Source: §3.2 · p. 10
• Fine-tuned image-to-video models generate around 827 hours of neural trajectories from robot data, with latent-action and IDM-based pseudo-action labels.The models were trained for 100 epochs on 3,000 language-annotated robot samples and generated counterfactual videos for novel prompts, including unseen initial frames, objects, and motion patterns.Source: §3.2 · p. 10

3.3. Human Video Datasets

• Human-video datasets provide unlabeled human-object interactions spanning grasping, tool use, cooking, assembly, and other task-oriented behaviors.The sources include Ego4D, Ego-Exo4D, Assembly-101, EPIC-KITCHENS, HOI4D, HoloAssist, and RH20T-Human, with first-person footage and complementary exocentric views, including detailed assembly and interaction annotations.Source: §3.3 · pp. 10–11

4.1. Simulation Benchmarks

• The simulation suite covers 24 RoboCasa kitchen tasks, nine DexMimicGen cross-embodiment tasks, and 24 GR-1 tabletop tasks, including unseen object or receptacle configurations.DexMimicGen spans three bimanual embodiments, while GR-1 tabletop tasks include rearrangement and articulated-object placement; checkpoints, environments, and datasets are publicly available for reproducibility.Source: §4.1 · pp. 11–12

4.2. Real-World Benchmarks

• Real-world post-training benchmarks comprise five pick-and-place, three articulated-object, three industrial-object, and two multi-agent coordination tasks.They test seen and unseen objects, constrained placement and closure, tool-based workflows, and collaboration; human operators collect task-specific data for 15 minutes to three hours, with low-quality trajectories filtered out.Source: §4.2 · pp. 12–14

4.3. Experiment Setup

• Evaluation post-trains GR00T N1 and BC-Transformer and Diffusion Policy baselines in data-limited settings, using 60k steps and global batch size 1024 except DexMimicGen’s batch size 128.Simulation success is averaged over 100 trials using the best score from the last five checkpoints; real tasks use 10 trials, except Pack Machinery, which uses five trials and counts placed objects under a 30-second limit. Low-data evaluation subsamples 10% of each task dataset.Source: §4.3 · pp. 14–15

4.4. Quantitative Results

• 32.4% (10% Data) and 30.4% (Full Data): GR00T-N1-2B beats Diffusion Policy on real tasks, while GR00T N1 consistently outperforms from-scratch baselines across simulation benchmarks.With 10% of the data, GR00T-N1-2B performs only 3.8% lower than Diffusion Policy trained on the full dataset; GR00T N1 exceeds baselines by more than 17% on the GR-1 simulation task.Source: §4.4 · p. 15, Table 3 · p. 15, Table 2 · p. 15
• +4.2%, +8.8%, and +6.8%: neural-trajectory co-training improves RoboCasa averages across the 30-, 100-, and 300-demonstration regimes, and raises real-world averages by +5.8%.The real-world gain is averaged across eight tasks; LAPA slightly leads IDM at 30 demonstrations, while IDM's advantage grows at 100 and 300, and only IDM actions are used for real-world co-training.Source: §4.4 · p. 16, Figure 9 · p. 16

4.5. Qualitative Results

• 42.2% versus 11.8%: GR00T N1 outperforms the Diffusion Policy baseline on RoboCasa’s Turn Sink Spout task in the 100-sample regime.The pretrained model also transfers an apple between hands when prompted to place it in a basket, whereas the post-trained checkpoint fails after training only on right-hand data; post-trained GR00T N1 is described as smoother and more accurate at grasping than Diffusion Policy.Source: §4.5 · pp. 16–17

4.6. Limitations

• GR00T N1 currently focuses on short-horizon tabletop manipulation; extending to long-horizon loco-manipulation will require hardware, architecture, and training-corpus advances.The paper identifies limits in generating diverse, physics-consistent counterfactual synthetic data and proposes stronger vision-language backbones, improved synthetic-data methods, and new architectures and pre-training strategies.Source: §4.6 · p. 17

Foundation Models in Robotics

• GR00T N1 fine-tunes Eagle-2 with flow-matching action generation and action chunking, using cross-attention instead of a mixture-of-experts bridge and embodiment-specific state/action projectors.The projectors support different embodiments, including latent and IDM-based actions; unlike Octo, GR00T N1 fine-tunes the VLM.Source: §5 · p. 17

Datasets for Robot Learning

• Robot teleoperation yields high-quality demonstrations but requires extensive cost and human effort, while human video is plentiful and easier to collect; GR00T N1 learns from diverse robot, human-video, and synthetic data.Instrumented human demonstrations can be faster to collect but may mismatch the downstream robot; human-video methods also use intermediate representations or latent actions.Source: §5 · pp. 17–18

Synthetic Data Generation in Robotics

• Simulation pipelines can efficiently generate large-scale demonstrations with minimal human effort, but their use is challenging because of the simulation-to-reality gap.GR00T N1 uses MimicGen and DexMimicGen simulation datasets and co-trains synthetic data with real-world data.Source: §5 · p. 18
• ∼300k neural trajectories amounting to 827 hours of robot trajectories are generated with video models, extending prior data augmentation beyond in-painting and text-to-image diffusion.The paper describes this scale of neural trajectory generation as unexplored in prior work.Source: §5 · p. 18

6. Conclusions

• GR00T N1 demonstrates strong generalization across simulation benchmarks and the real GR-1, enabling data-efficient learning of diverse manipulation skills.The paper presents it as a generalist policy evaluated in both settings.Source: §6 · p. 18
• The open GR00T N1 model combines a dual-system design and heterogeneous training data while supporting multiple robot embodiments.Source: §6 · p. 18
• The authors hope the open GR00T-N1-2B model, training datasets, and simulation environments will accelerate progress toward generally capable humanoid robots in the wild.Source: §6 · p. 18

B. Detailed Experiment Results

• Across simulation datasets of 30, 100, and 300 demonstrations and real-world datasets with 10% or full data, both models improve with more data, while GR00T-N1-2B outperforms Diffusion Policy on every benchmark and dataset size.The comparison indicates stronger generalization and sample efficiency for GR00T-N1-2B.Source: B. Detailed Experiment Results · p. 20

C. Additional Qualitative Results

• GR00T-N1-2B demonstrates strong language following and generalization to unseen bimanual manipulation situations.Source: C. Additional Qualitative Results · p. 20
• Before post-training, it places an apple into a basket via a two-handed handover in an unseen setup, albeit with jerkier motion.Source: Figure 11 · p. 21
• After post-training, it places a cucumber in a basket and transfers a lemon to a pan, while Diffusion Policy fails both tasks.Diffusion Policy fails the cucumber task due to an inaccurate grasp and remains stuck during the lemon task.Source: Figure 12 · p. 21

D. Hyperparameters

• Post-training uses smaller batch sizes to avoid overfitting when fine-tuning in data-limited settings.The hyperparameters for both phases are reported in Table 6.Source: D. Hyperparameters · p. 21
• The pre- and post-training phases share the same values for most hyperparameters.Source: D. Hyperparameters · p. 21

E.1. Dataset Formats

• LeRobot provides the open-source foundation for storing, sharing, and using robot demonstrations, with broad adoption and a collection of models and datasets on the Hugging Face hub.Source: §E.1 · p. 21
• The format stores robot states, actions, and metadata in parquet, visual data in MP4 or PNG sequences, and dataset statistics and episode indices in JSON.Parquet files reference the visual data.Source: §E.1 · p. 23
• Demonstrations are organized into episodes of synchronized observation-action pairs, with camera imagery and robot state as observations and robot control commands as actions.This organization supports imitation learning and reinforcement learning.Source: §E.1 · p. 23
• For cross-embodiment data, the extension adds modality.json, semantically named state and action fields with metadata, multiple annotation types, and explicit rotation representations.Fields can specify data types, ranges, and transformations; rotation representations include quaternions, Euler angles, and axis-angle.Source: §E.1 · p. 23
• The extension supports field-specific transformations and clearer dataset validation while retaining backward compatibility with the LeRobot ecosystem.Its fine-grained modality definitions also accommodate visual, state, action, and language data with explicit relationships.Source: §E.1 · p. 23

E.2. Standardized Action Spaces

• State end-effector rotations use a 6D representation, rotation actions use axis-angle, and min-max scaling and a fixed left-to-right vector order standardize spaces across embodiments.The order is end-effector rotation, end-effector position, then gripper closeness; scaling applies to joint and end-effector states and actions.Source: §E.2 · p. 24

Auxiliary Object Detection Loss

• The model adds a squared-loss target-center head using OWL-v2-annotated, normalized bounding-box centers for each trajectory frame; total loss combines flow matching and detection.A linear layer on the final vision-language embedding tokens predicts the 2D coordinates (Minderer et al., 2023).Source: F. Additional Training Details · p. 24

Neural Trajectory Generation

• LoRA-finetuned WAN2.1-I2V-14B generates counterfactual neural trajectories from teleoperation data, with instruction-adherence filtering and re-captioning for failures.Trajectories are downsampled to 81 frames at 480P; a multimodal LLM assesses adherence using eight sampled frames, and failed videos are re-captioned from 16 frames at 256P (Hu et al., 2022).Source: F. Additional Training Details · p. 24

IDM Model Training

• An embodiment-specific inverse dynamics model predicts action chunks from current and future images, then pseudo-labels neural-trajectory actions.It uses System 1 with SigLIP-2 vision embeddings and flow matching, trained for 30K or 60K steps depending on dataset size; adding state or more frames did not significantly improve validation performance.Source: F. Additional Training Details · p. 24

Human Egocentric Video Dataset Samples

• Pre-training uses seven human video datasets with corresponding language annotations.Figure 14 shows sample images from each dataset.Source: Figure 14 · p. 25