PaperlayerView exact revision on arXiv

Source-linked AI summary

World Simulation with Video Foundation Models for Physical AI

NVIDIA, :, Arslan Ali, Junjie Bai, Maciej Bala, Yogesh Balaji, Aaron Blakeman, Tiffany Cai, Jiaxin Cao, Tianshi Cao, Elizabeth Cha, Yu-Wei Chao, Prithvijit Chattopadhyay, Mike Chen, Yongxin Chen, Yu Chen, Shuai Cheng, Yin Cui, Jenna Diamond, Yifan Ding, Jiaojiao Fan, Linxi Fan, Liang Feng, Francesco Ferroni, Sanja Fidler, Xiao Fu, Ruiyuan Gao, Yunhao Ge, Jinwei Gu, Aryaman Gupta, Siddharth Gururani, Imad El Hanafi, Ali Hassani, Zekun Hao, Jacob Huffman, Joel Jang, Pooya Jannaty, Jan Kautz, Grace Lam, Xuan Li, Zhaoshuo Li, Maosheng Liao, Chen-Hsuan Lin, Tsung-Yi Lin, Yen-Chen Lin, Huan Ling, Ming-Yu Liu, Xian Liu, Yifan Lu, Alice Luo, Qianli Ma, Hanzi Mao, Kaichun Mo, Seungjun Nah, Yashraj Narang, Abhijeet Panaskar, Lindsey Pavao, Trung Pham, Morteza Ramezanali, Fitsum Reda, Scott Reed, Xuanchi Ren, Haonan Shao, Yue Shen, Stella Shi, Shuran Song, Bartosz Stefaniak, Shangkun Sun, Shitao Tang, Sameena Tasmeen, Lyne Tchapmi, Wei-Cheng Tseng, Jibin Varghese, Andrew Z. Wang, Hao Wang, Haoxiang Wang, Heng Wang, Ting-Chun Wang, Fangyin Wei, Jiashu Xu, Dinghao Yang, Xiaodong Yang, Haotian Ye, Seonghyeon Ye, Xiaohui Zeng, Jing Zhang, Qinsheng Zhang, Kaiwen Zheng, Andrew Zhu, Yuke Zhu

arXiv:2511.00062v2cs.CVcs.AIcs.LGcs.RO

TL;DR

Training Physical AI in the real world is often slow, costly, and risky. This paper presents unified Cosmos world models and reports leading Physical AI benchmark results with demonstrated robotics and autonomous-driving applications.

  • Problem

    Training Physical AI directly in the real world is often slow, costly, and risky, motivating visual simulation as a safe proxy for acquiring skills before deployment.

    Source: §1 · p. 3

  • Method

    Cosmos-Predict2.5 unifies Text2World, Image2World, and Video2World with richer text grounding, while Cosmos-Transfer2.5 extends the family with control-net models for world translation.

    Source: §1 · p. 3

  • Results

    The models achieve leading results on Physical AI benchmarks and show benefits across synthetic-data generation, policy validation, autonomous-driving simulation, and robotic manipulation.

    Source: §1 · p. 3, §8 · p. 36

  • Takeaways & Limitations

    The work positions high-fidelity synthetic video as a resource for simulation-first Physical AI research in robotics and autonomous driving.

    Source: §8 · p. 36

Abstract

from arXiv · show

We introduce [Cosmos-Predict2.5], the latest generation of the Cosmos World Foundation Models for Physical AI. Built on a flow-based architecture, [Cosmos-Predict2.5] unifies Text2World, Image2World, and Video2World generation in a single model and leverages [Cosmos-Reason1], a Physical AI vision-language model, to provide richer text grounding and finer control of world simulation. Trained on 200M curated video clips and refined with reinforcement learning-based post-training, [Cosmos-Predict2.5] achieves substantial improvements over [Cosmos-Predict1] in video quality and instruction alignment, with models released at 2B and 14B scales. These capabilities enable more reliable synthetic data generation, policy evaluation, and closed-loop simulation for robotics and autonomous systems. We further extend the family with [Cosmos-Transfer2.5], a control-net style framework for Sim2Real and Real2Real world translation. Despite being 3.5$\times$ smaller than [Cosmos-Transfer1], it delivers higher fidelity and robust long-horizon video generation. Together, these advances establish [Cosmos-Predict2.5] and [Cosmos-Transfer2.5] as versatile tools for scaling embodied intelligence. To accelerate research and deployment in Physical AI, we release source code, pretrained checkpoints, and curated benchmarks under the NVIDIA Open Model License at https://github.com/nvidia-cosmos/cosmos-predict2.5 and https://github.com/nvidia-cosmos/cosmos-transfer2.5. We hope these open resources lower the barrier to adoption and foster innovation in building the next generation of embodied intelligence.

1. Introduction

Cosmos-Predict2.5 is a flow-matching world foundation model intended to provide higher-fidelity simulation for Physical AI, while Cosmos-Transfer2.5 extends the family to world translation. The paper also describes downstream simulation uses and releases resources to support adoption.

1. Introduction

  • World simulators can let Physical AI agents develop perception and control skills in silicon before real-world deployment, avoiding the slowness, expense, and risks of direct real-world training.Early-stage system imperfections can lead to unsafe actions that damage the agent or environment; such simulation is described as a safe proxy for the physical world (NVIDIA, 2025).Source: §1 · p. 3
  • Cosmos-Predict2.5 combines Text2World, Image2World, and Video2World, and improves output quality and prompt alignment over Cosmos-Predict1.Its improvements include stronger data filtering, specialized Physical AI post-training data, a simplified architecture, model merging, reinforcement-learning post-training, and Cosmos-Reason1 in place of T5 for richer text representations and finer control.Source: §1 · p. 3
  • Cosmos-Predict2.5 supports VLA training data generation, action-conditioned policy validation, coherent multi-view autonomous-driving simulation, and camera-controllable multi-view robotic-manipulation generation.Synthetic data generation supports VLA training (Jang et al., 2025).Source: §1 · p. 3
  • Cosmos-Transfer2.5 translates worlds for simulator photorealism, real-video augmentation, and realistic multi-view inputs, and is 3.5× smaller than Cosmos-Transfer1 while enabling robust long-horizon translation and closed-loop simulation.The cited applications include simulator-output enhancement (NVIDIA, 2025), real-world video augmentation (Ren et al., 2025), and semantic-to-realistic multi-view conversion (NVIDIA, 2025).Source: §1 · p. 3
  • The authors release source code, pretrained checkpoints, and curated post-training examples to help practitioners adapt models for robotics, autonomous systems, and embodied reasoning.Tab. 1 maps released models to their capabilities.Source: §1 · p. 3

2. Data

The data section strengthens the general video pipeline and adds curated Physical AI data across five target domains. It describes large-scale filtering, annotation, and organization designed to support training across diverse Physical AI use cases.

2.1. Video Curation Pipeline

  • The seven-stage pipeline splits, transcodes, crops, filters, captions, deduplicates, and shards video for large-scale training.Filtering includes aesthetic, motion, OCR, perceptual-quality, semantic-artifact, and final VLM checks; clips receive multi-length captions, semantic deduplication, and structured sharding.Source: §2.1 · pp. 4–5
  • 35 million hours of raw video were processed versus 20 million for Cosmos-Predict1; survival fell from 30% to 4%, leaving 200 million high-quality clips.The pipeline produced over 6 billion clips before retaining the 200 million-clip pre-training dataset.Source: §2.1 · p. 6

2.2. Domain Specific Data

  • Domain-specific data covers Robotics, Autonomous Driving, Smart Spaces, Human Dynamics, and Physics, and is added to general pre-training data.These pipelines omit the VLM filter in favor of domain-specific filters and use a larger VLM with tailored caption prompts.Source: §2.2 · p. 6

2.2.1. Robotics

  • Robotics videos are filtered for low resolution and near-static motion, with overly slow movements sped up for consistent action pacing.Dataset-aware captions describe actions chronologically, normalize viewpoint and embodiment, and use available dataset-specific metadata to improve accuracy and reduce hallucination.Source: §2.2.1 · pp. 6–7

2.2.2. Autonomous Driving

  • Autonomous-driving data uses 3.1 million 20-second surround-view clips recorded by seven synchronized cameras.Sampling targets diverse driving conditions, while captions describe agents, environmental factors, vehicle actions and speeds, object-state changes, and interactions in short, medium, and long forms.Source: §2.2.2 · p. 7

2.2.3. Smart Spaces

  • Smart Spaces curation uses keyword search and VLM relevance checks to produce approximately 40K clips from warehouses, factories, construction sites, and similar settings.A VLM captions the clips with prompts tailored to smart-space content and style.Source: §2.2.3 · p. 7

2.2.4. Human Dynamics

  • Human-dynamics clips require people in over 40% of frames, no more than eight people per frame, and at least one person occupying 3% or more of the image.YOLOX detects people, RTMPose estimates keypoints and facial landmarks, and VLM captions emphasize motion and dynamics for human-behavior simulation.Source: §2.2.4 · p. 7

2.2.5. Physics

  • The Physics dataset organizes videos around observable phenomena in classical and fluid mechanics to improve the physical plausibility of generated videos.Tailored VLM prompts describe physical processes and object interactions, including examples such as shattering glass, colliding balls, and flowing water.Source: §2.2.5 · p. 8

3. Method

[Cosmos-Predict2.5] uses flow matching for velocity-based video generation and combines a revised denoising network with visual and text encoders that support text-, image-, and video-conditioned generation.

3.1. Flow Matching

  • FM trains a velocity-prediction network with MSE against ground-truth velocity, conditioned on inputs such as text embeddings and reference frames.The paper contrasts this velocity target with EDM’s approximately standardized Gaussian inputs and outputs.Source: §3.1 · p. 8
  • A shifted logit-normal timestep distribution biases training toward higher noise, helping the model learn structure when high-resolution pixel correlations are disrupted.The shift is controlled by β; when β = 1, no shift is applied.Source: §3.1 · pp. 8–9

3.2. Network Architecture

  • The model reuses the [Cosmos-Predict1] DiT denoising network but removes absolute positional embeddings, retaining relative embeddings to support higher resolutions and longer video sequences.The paper identifies this as the main architectural change.Source: §3.2 · p. 9
  • WAN2.1 VAE compression is 4 × 8 × 8 across time, height, and width, followed by 1 × 2 × 2 patchification; training generates 93 frames (24 latent frames) at 16 fps.Each generated video is about 5.8 seconds long.Source: §3.2 · p. 9
  • [Cosmos-Reason1] replaces the T5 text encoder; concatenated activations from multiple blocks are projected to 1024 dimensions and incorporated through cross-attention to guide generation.Its vision encoder can support visual conditioning, which the paper leaves for future exploration.Source: §3.2 · p. 9
  • The model supports Text2World from prompts, Image2World from prompts and reference images, and Video2World from video conditioning for coherent continuation or transformation.Image2World and Video2World use frame replacement: conditioned frames substitute for initial generated frames, with adjustable counts that strengthen temporal consistency.Source: §3.2 · pp. 9–10

4. Training

Cosmos-Predict2.5 training progresses from resolution- and task-staged pretraining through domain-specific supervised fine-tuning and model merging to reinforcement-learning post-training. The section also describes timestep distillation and scalable infrastructure for training and reward evaluation.

4. Training

  • The training pipeline combines staged pretraining, curated-domain SFT models merged into one model, and RL post-training to improve generation quality.Source: §4 · p. 10

4.1. Pre-training

  • Pretraining progresses from 256p Text2Image to joint Image2World/Video2World, then 480p and 720p, before adding Text2World with 0–2 conditioning frames.For joint image–video training, 1 or 5 conditioning frames are used to generate the remaining 92 or 88 of 93 pixel frames; only designated frames receive denoising loss.Source: §4.1 · pp. 10–11

4.2.1. Supervised Fine-tuning

  • Every domain-specific SFT model significantly outperformed the pretrained baseline on its target domain; grid search favored model soup as the final post-trained merge.Source: §4.2.1 · pp. 11–13, Figure 3 · p. 12

4.2.2. Reinforcement Learning

4.2.3. Timestep Distillation

4.3. Infrastructure

  • With 4096 H100 GPUs at 720p and 93 frames, MFU was 36.49% for the 2B model and 33.08% for the 14B model, with the drop attributed to context-parallel communication.Infrastructure uses FSDP2 and flexible context parallelism to support long, high-resolution video sequences.Source: §4.3 · pp. 14–15, Table 9 · p. 15

5. Results

On PAI-Bench, post-trained Cosmos-Predict2.5 models perform similarly to Wan2.2 27B-A14B in text-to-world and lead in image-to-world. Human evaluations and qualitative examples further assess preference and physical-AI generation.

Benchmarking

  • PAI-Bench’s Overall Score averages Quality and Domain scores; Quality uses eight adapted VBench metrics, while Domain uses VQA across seven physical-AI domains.The domains are av, common, human, industry, misc, physics, and robotics.Source: §5 · p. 15
  • In T2W, both post-trained Cosmos-Predict2.5 models perform similarly to Wan2.2 27B-A14B; in I2W, they are the best-performing models.The quantitative results are reported in Table 10 for T2W and Table 11 for I2W.Source: §5 · p. 15, Table 10 · p. 16, Table 11 · p. 16

Human Evaluation

  • 30.0% vs. 26.2% of human votes preferred Cosmos-Predict2.5-2B over Wan2.2 5B; it was comparable with Wan 2.1 14B (33.0% vs. 34.8%).The 2B model was 60.0% smaller than Wan2.2 5B and 85.7% smaller than Wan 2.1 14B. Human raters compared realism, visual quality, temporal consistency, and alignment with conditioning inputs.Source: §5 · p. 16, Figure 6 · p. 17
  • 48.6% vs. 31.8% of human votes preferred post-trained Cosmos-Predict2.5-14B over Wan 2.1 14B; it was on par with Wan 2.2 27B-A14B (38.1% vs. 35.9%).The 14B model has half the parameter count of Wan 2.2 27B-A14B. Preference over Wan 2.1 14B increased from 33.0% for the 2B model to 48.6% for the 14B model.Source: §5 · p. 16, Figure 7 · p. 17

Qualitative Examples

  • Qualitative examples show Cosmos-Predict2.5-2B simulating accurate driving behavior, realistic industrial and robotics scenes, and physically coherent motion.The examples are presented as representative physical-AI scenarios in Figure 8.Source: §5 · p. 16

6. Applications · 6.1. Cosmos-Transfer2.5

This section introduces Cosmos-Transfer2.5 as a control-conditioned world-generation model and evaluates its benchmark performance and long-video quality. It also places Transfer2.5 within a broader set of Cosmos-Predict2.5 applications for Physical AI.

6. Applications

  • The broader application portfolio includes Real2Real policy-learning augmentation, map-conditioned multiview driving simulation, camera-pose–controllable multiview generation, VLA training data generation, and action-conditioned policy evaluation.Cosmos-Transfer2.5 is introduced as a control-net style capability within this portfolio.Source: §6 · pp. 17–18

6.1. Cosmos-Transfer2.5

  • Cosmos-Transfer2.5-2B generates world simulations from multiple spatial control modalities, including edges, blurred video, segmentation maps, and depth maps, from simulated or real-world data.The controls may originate in a physics simulator such as NVIDIA IsaacSim or in real-world video.Source: §6.1 · p. 18
  • The model retains four control blocks but distributes them throughout the main branch, inserting one after every seven blocks rather than placing all four at its start.This integrates conditioning information more gradually through the network.Source: §6.1 · p. 18
  • Training data emphasizes Physics AI domains, with depth maps generated for 10 million videos, segmentation for 3 million, and edge and blur conditions curated for 14 million.The curated domains include autonomous driving, robotics, smart spaces, and physics.Source: §6.1 · pp. 18–19

6.1.1. Results

  • Cosmos-Transfer2.5-2B outperforms Cosmos-Transfer1-7B on both control adherence and video quality metrics despite being 3.5 times smaller.The authors attribute the improvement to a stronger Cosmos-Predict2.5-2B base and more carefully curated Physics-AI-focused training data.Source: Table 12 · p. 19, §6.1.1 · p. 19

6.1.2. Long Video Generation

  • The long-video evaluation uses 17 videos of 30 to 120 seconds and introduces averaged RNDS to measure quality degradation across autoregressively generated chunks.RNDS normalizes each chunk's DOVER score against the corresponding ground-truth chunk, so the curve starts at (1, 1), and averages curves over the evaluation videos.Source: §6.1.2 · pp. 19–20
  • Cosmos-Transfer2.5-2B shows far less RNDS reduction over time than Cosmos-Transfer1-7B, indicating fewer accumulated errors, less hallucination, and higher fidelity over long video sequences.Figure 9 also reports better prompt alignment and control-input adherence, alongside less hallucination and error accumulation especially for long videos.Source: §6.1.2 · p. 20, Figure 9 · p. 20

6.2. Cosmos-Transfer2.5 for Robot Policy Learning

Cosmos-Transfer2.5-2B generates controllable visual variations to augment robot-policy training, and the augmented policy shows stronger performance across varied real-robot test scenarios than the reported baselines.

6.2.1. System and Task Settings

  • The study tests bimanual apple-and-bowl pick-and-place policies trained from 100 teleoperation demonstrations, with object positions varied across trials.The UNet-based Diffusion Policy predicts action chunks spanning 8 timesteps at 10 FPS.Source: §6.2.1 · p. 22

6.2.2. Data Augmentation Strategy

  • Cosmos-Transfer2.5-2B generates realistic visual variations while preserving task structure, using text prompts to specify changes and augment training images.The method applies global edge control and restricts blur control to robot pixels; actions and joint states remain unchanged.Source: §6.2.2 · pp. 22–23

6.2.3. Experiments and Results

  • Evaluation covers the base setting and nine novel conditions, including changed objects, tablecloths, lighting, distractors, background edits, and a composite shift.The policy is tested in three trials per scenario with object pose and environment configuration fixed for fair comparisons.Source: §6.2.3 · p. 23
  • 24 of 30 trials succeeded with the Cosmos-Transfer2.5-2B-augmented policy, outperforming both baselines; standard augmentation succeeded in only one case.The base policy also performed poorly, including on the base setting, amid subtle scene variations.Source: Table 13 · p. 23, §6.2.3 · p. 24

6.3. Cosmos-Transfer2.5 for Driving Simulation

Cosmos-Transfer2.5-2B/auto/multiview extends controllable world generation to synchronized driving views, using scenario maps as control and improving visual quality and control adherence over Transfer1 in evaluation.

6.3.1. Model Architecture

  • The multi-view model extends Predict2.5-2B with a control branch, generating consistent scenes across views with separate per-view frame conditioning and control signals.Views are encoded independently and concatenated in the latent temporal dimension, with separately constructed RoPE embeddings for each view.Source: §6.3 · p. 25, §6.3.1 · p. 25

6.3.3. Experiments and Results

  • Up to 2.3x improvement in FVD/FID scores was observed while temporal and cross-camera Sampson error remained competitive.Evaluation used 1,000 held-out multi-view clips with HD maps and human-labeled lanes and cuboids.Source: Table 14 · p. 25, §6.3.3 · p. 28
  • Detection metrics improved by up to 60% over Transfer1-7B-Sample-AV on generated videos.LATR evaluated 3D lane detection and BEVFormer evaluated 3D cuboid detection against ground-truth labels.Source: Table 15 · p. 25, §6.3.3 · p. 28

6.3.2. Training Datasets

  • The world scenario map projects HD maps and dynamic-object boxes onto seven camera views, encoding road elements and motion cues for control.The control-net training set is RDS-HQ, with 140,000 20-second multi-view driving scenes; its maps support fine-grained lane markings and occlusion- and heading-aware boxes.Source: §6.3.2 · pp. 25, 26, 28
  • The model resolves Transfer1’s hallucinated or misaligned vehicles and incorrect lane direction and object class in comparative generations.Examples include a distorted car absent from the prompt and control, wrong-side driving, and a truck rendered instead of a pedestrian.Source: §6.3.2 · p. 27, Figure 15 · p. 27

6.4. Multi-view Generation with Camera Control

This section presents camera-controlled multi-view video generation that synthesizes synchronized target views from a reference view and camera trajectories. Its Transfer2.5 model improves cross-view consistency over a single-view variant while maintaining comparable camera trajectory accuracy.

6.4. Multi-view Generation with Camera Control

  • Given a reference video and camera trajectories, the model generates synchronized videos from target viewpoints, including views that reveal occluded scene regions.The robotic setting includes mapping a humanoid’s head-camera view to gripper views on both hands, supporting perception, planning, and control.Source: §6.4 · p. 28
  • The architecture aligns camera pose with video latents using Plücker raymaps, adding projected raymap tokens before self-attention and freezing other layers during training.Source and target video tokens are concatenated temporally, and camera parameters are sampled every four frames to match the encoder’s temporal downsampling.Source: §6.4 · pp. 28, 31
  • The Agibot variant converts head-view manipulation video into synchronized left- and right-gripper videos.It is fine-tuned on 145,820 Agibot episodes with three views and precise camera poses, sampled from a dataset of approximately one million episodes.Source: Figure 16 · p. 29, §6.4 · p. 31
  • The multiview variant generates synchronized videos under dynamic and static camera transformations from a third-view input.Fine-tuned on MultiCamVideo (136,000 episodes) and SynCamVideo (34,000 episodes), it supports rotations, arcs, zoom, azimuth and elevation shifts, distance changes, and dynamic focal length (Bai et al., 2025).Source: Figure 17 · p. 30, §6.4 · p. 31
  • The multi-view model achieves significantly better cross-view consistency than its single-view counterpart while maintaining comparable camera trajectory accuracy.Evaluation used 80 validation videos across 16 camera trajectories, measuring camera trajectory error and cross-view Sampson error.Source: Table 17 · p. 31, §6.4 · p. 31, Figure 18 · p. 32
  • A separate Predict2.5 variant generates three manipulation videos from single images of two gripper views and one head view, with corresponding camera trajectories.Trained on Agibot data with the multiview architecture, it is intended to generate diverse data for robotic policy training.Source: §6.4 · p. 31

6.5. Synthetic Data Generation for VLA training · 6.6. Action-Conditioned World Generation

The sections present Cosmos-Predict2.5 as both a source of synthetic VLA training data and an action-conditioned video world model. They report strong instruction-following and action-conditioned prediction results, alongside methods for generating and conditioning robot videos.

6.5. Synthetic Data Generation for VLA training

  • Post-trained Cosmos-Predict2.5 can generate realistic videos of robots executing unseen commands, from which latent-action or inverse-dynamics models extract pseudo-actions for vision-language-action training.The resulting video, instruction, and pseudo-action annotations can augment real demonstrations to support VLA generalization.Source: §6.5 · p. 32
  • On DreamGen’s object, behavior, and environment generalization dimensions, Cosmos-Predict2.5-14B/robot/gr00tdream-gr1 achieves the highest instruction-following scores on the GR1 dataset and especially excels at object and environment generalization.The benchmark uses vision-language-model evaluation to assess videos following instructions involving unseen objects, novel behaviors, or new environments.Source: §6.5 · p. 32, Table 18 · p. 34
  • Cosmos-Predict2.5-2B/robot/multiview-agibot generates synchronized manipulation videos from a single-frame input comprising three camera views and their camera trajectories.Source: Figure 19 · p. 33

6.6. Action-Conditioned World Generation

  • Action-conditioned generation takes a conditional image and robot-action sequence to predict future-frame chunks, autoregressively conditioning each new chunk on the last generated frame.This produces full trajectories with Cosmos-Predict2.5-2B/robot/action-cond.Source: §6.6 · p. 33
  • An action embedder MLP maps each action to a tensor that is added to DiT timestamp embeddings, providing conditioning for this modality absent during pre-training.Source: §6.6 · p. 33
  • The action-conditioned experiments use Bridge’s approximately 20,000 kitchen-robot episodes, with 320 × 256 video at 5 FPS and frame-aligned 7-dimensional gripper-coordinate actions.Actions encode relative displacement, rotation, and gripper width; evaluation compares generated videos with ground truth on 100 test episodes.Source: §6.6 · p. 33
  • Cosmos-Predict2.5-2B/robot/action-cond outperforms the Cosmos-Predict1-7B baseline across PSNR, SSIM, Latent L2, and FVD, with better object permanence in the illustrated rollouts.The evaluation uses 100 randomly sampled episodes from the official Bridge test set and compares predictions with ground-truth videos.Source: §6.6 · p. 33, Table 19 · p. 34, Figure 20 · p. 34
  • Among time-embedding conditioning, DiT-block cross-attention, and channel concatenation, time embeddings yield better action-conditioned video-generation performance.Source: §6.6 · p. 34, Table 20 · p. 35

7. Related Work

The section places Cosmos within pixel-space world modeling and open video generation for Physical AI. It describes how the Cosmos family builds on earlier open models to support controllable generation, evaluation, and simulation.

World Models

  • World models span latent-state prediction, pixel-space video generation, and emerging native 3D/4D representations; this work adopts pixel-space prediction for high-fidelity synthetic data and flexible control signals.Pixel-space models can incorporate camera pose, action sequences, and dense inputs such as world scenario maps.Source: §7 · p. 35

Video Generative Models

  • Open video models enable accessible, reproducible research but often lack fine-grained control and physical consistency; this work enhances Cosmos-Predict1 with curated physical-reasoning data and a Cosmos-Reason1-based text encoder.Source: §7 · p. 35

Foundation World Model for Physical AI

  • Cosmos-Predict1 and Cosmos-Transfer1 introduced the first open-source Physical AI world models, supporting open benchmarks and follow-on work in control, relighting, policy-data generation, and autonomous driving.Related benchmarks address general-purpose generation (Duan et al., 2025; Zhou et al., 2025; Zhao et al., 2025), physics (Bansal et al., 2024; Motamed et al., 2025; Guo et al., 2025; Bansal et al., 2025; Bordes et al., 2025), and Embodied AI (Yang et al., 2025; Liao et al., 2025; Yue et al., 2025). Prior post-training enabled camera control (Ren et al., 2025), motion trajectory control (Wang et al., 2025), and video relighting (He et al., 2025); reported applications include robot policy training (Bjorck et al., 2025) and autonomous driving (Ren et al., 2025; Fu et al., 2025).Source: §7 · p. 36
  • Cosmos-Predict2.5 supports VLA training, robot policy training and validation, autonomous-driving simulation, and robotic manipulation; Transfer2.5 improves long-horizon video translation and closed-loop simulation.These are the enhanced capabilities reported for the current models.Source: §7 · p. 36

8. Conclusion · A. Contributors and Acknowledgments

The paper presents Cosmos-Predict2.5 and Cosmos-Transfer2.5 as video world foundation models for Physical AI, reporting leading benchmark results and demonstrating robotics and autonomous-driving applications. The authors release models and code to support a simulation-first ecosystem that bridges simulation and real-world deployment.

8. Conclusion

  • The authors report leading results on Physical AI benchmarks for Cosmos-Predict2.5 and Cosmos-Transfer2.5.The models are described as the latest Cosmos video world foundation models for Physical AI.Source: §8 · p. 36
  • The models combine large-scale curated video datasets, flow-matching training, improved text embedding, domain-specific post-training, and reinforcement learning.Source: §8 · p. 36
  • The authors demonstrate the models’ effectiveness in robotics and autonomous driving, where high-fidelity synthetic video is essential.Source: §8 · p. 36
  • The authors release models and code to support Cosmos as a world foundation model platform for a simulation-first ecosystem.Source: §8 · p. 36
  • The stated aim is for this ecosystem to advance Physical AI and bridge simulation with real-world deployment.Source: §8 · p. 36

Related papers

Loading 2511.00062v2…