PaperlayerView exact revision on arXiv

Source-linked AI summary

World-R1: Reinforcing 3D Constraints for Text-to-Video Generation

Weijie Wang, Xiaoxuan He, Youping Gu, Yifan Yang, Zeyu Zhang, Yefei He, Yanbo Ding, Xirui Hu, Donny Y. Chen, Zhiyuan He, Yuqing Yang, Bohan Zhuang

arXiv:2604.24764v4cs.CV

TL;DR

Video foundation models often lack intrinsic 3D geometric understanding, causing inconsistencies in challenging long-horizon or large-camera-movement scenes. World-R1 uses reinforcement learning with 3D and vision-language feedback to align generation with 3D constraints, substantially improving geometric consistency while preserving visual quality.

  • Problem

    Video foundation models lack intrinsic 3D geometric understanding, while existing constraint-based methods can incur high inference costs and restrict scalability.

  • Method

    World-R1 reformulates 3D alignment as reinforcement learning using composite 3D/VLM rewards, Flow-GRPO, pure-text data, and unchanged architecture and inference.

  • Results

    World-R1 significantly improves geometric consistency, achieving 10.23dB and 7.91dB improvements on PSNR respectively while maintaining high general video benchmark scores.

  • Takeaways & Limitations

    World-R1 provides a scalable way to enhance video models’ world-modeling and 3D reconstruction consistency without expensive 3D supervision or architectural changes.

  • Takeaways & Limitations

    The paper notes that existing static 3D inference-time constraints incur prohibitive costs and restrict generation precision, generalization, and fluid diversity.

Abstract

from arXiv · show

Recent video foundation models demonstrate impressive visual synthesis but frequently suffer from geometric inconsistencies. While existing methods attempt to inject 3D priors via architectural modifications, they often incur high computational costs and limit scalability. We propose World-R1, a framework that aligns video generation with 3D constraints through reinforcement learning. To facilitate this alignment, we introduce a specialized pure text dataset tailored for world simulation. Utilizing Flow-GRPO, we optimize the model using feedback from pre-trained 3D foundation models and vision-language models to enforce structural coherence without altering the underlying architecture. We further employ a periodic decoupled training strategy to balance rigid geometric consistency with dynamic scene fluidity. Extensive evaluations reveal that our approach significantly enhances 3D consistency while preserving the original visual quality of the foundation model, effectively bridging the gap between video generation and scalable world simulation.

1 Introduction

Current video foundation models generate high-quality visuals but lack intrinsic 3D understanding, causing geometric hallucinations and temporal inconsistencies. World-R1 uses reinforcement learning with 3D and vision-language feedback to improve geometric consistency while preserving visual quality and generative diversity.

  • Motivation: Video foundation models focus on image-space generation, leading to geometric hallucinations and temporal inconsistencies during large camera movements or long-horizon driving scenes.The issue arises from lacking intrinsic understanding of real-world 3D geometry.
  • Limitations of prior work: Existing 3D-prior methods improve consistency but impose prohibitive inference costs that restrict precision, generalization, scalability, and the fluid diversity of video models.These methods enforce static 3D constraints at inference time.
  • World-R1: World-R1 aligns video generation with 3D foundation models through reinforcement learning, eliciting latent geometric knowledge without large-scale 3D datasets or expensive inference-time constraints.The framework injects world-modeling capabilities into video models via RL.
  • World-R1: Its reward system combines 3D-model reconstruction and trajectory alignment with VLM semantic criticism of novel views, optimized using Flow-GRPO.The approach provides rigid geometric feedback while evaluating rendered meta-view plausibility.
  • Training data: The method uses a pure text dataset with multi-class, multi-level camera control for post-training video generation models.This dataset supports precise motion control without architectural modification.
  • Results: Finetuned models improve PSNR by 10.23dB and 7.91dB respectively while maintaining high general-video benchmark scores and original generative diversity or visual quality.Additional analyses cover reconstruction-independent multi-view consistency, dataset scaling, long videos, scene complexity, and comparisons with 3D-aware methods.

2 Related Works

Related work spans scalable video foundation models and reinforcement-learning methods for improving controllability and spatial consistency. Recent approaches increasingly favor DiT architectures and efficient Flow-GRPO variants over costly conventional alternatives.

  • Controllable Video Generation: Recent video foundation models have shifted from U-Net-based diffusion to DiT architectures, using internet-scale video data to synthesize high-fidelity visual content.
  • Visual Reinforcement Learning: Reinforcement learning can elicit latent spatial awareness, allowing models to learn geometry and maintain consistency without external modules or explicit 3D-guided inference.
  • Visual Reinforcement Learning: GRPO reduces PPO’s computational burden for high-dimensional visual data by eliminating the critic network, while Flow-GRPO adapts it to flow-matching generators.
  • Visual Reinforcement Learning: Flow-GRPO-Fast accelerates training by injecting noise at randomly selected intermediate steps and switching deterministic ODE sampling to SDE sampling.

3 Preliminaries

Flow-GRPO adapts flow-matching models for online reinforcement learning by introducing stochastic sampling while preserving the pretrained flow model’s marginal distribution. It formulates denoising as an MDP optimized with GRPO and accelerates training through denoise reduction.

  • Flow-GRPO: Flow-GRPO integrates online reinforcement learning into flow-matching models to optimize generation quality.Standard flow matching uses deterministic ODE solvers, whereas reinforcement learning requires stochasticity for exploration and advantage estimation.
  • Stochastic Sampling via SDE: Flow-GRPO reformulates sampling as a stochastic SDE process, preserving the pretrained flow model’s marginal distribution.The resulting stochastic sampler supplies the exploration noise required for reinforcement learning.
  • GRPO: Flow-GRPO formulates denoising as a Markov Decision Process and applies GRPO to groups of sampled trajectories.Rewards are normalized against group statistics to estimate trajectory advantages.
  • GRPO: The GRPO objective includes a KL-divergence constraint that limits deviation from the reference policy.The objective also uses a PPO-style clipped surrogate objective with probability ratios between current and old policies.
  • Denoise Reduction: Flow-GRPO uses fewer denoising timesteps during training to accelerate convergence without sacrificing inference quality.This denoise reduction strategy is applied during training rather than changing the stated inference quality.

4 Methodology

World-R1 injects world-modeling capabilities into a pre-trained video generator without architectural modifications or specialized 3D-aware datasets, using implicit camera conditioning and reinforcement learning. Its methodology combines discrete noise transport, composite 3D and perceptual rewards, a pure-text world-simulation dataset, and periodic dynamic fine-tuning.

  • Framework Overview: World-R1 avoids architectural modifications and specialized 3D-aware datasets by combining noise manipulation with reinforcement learning.The framework is designed to inject world-modeling capabilities into a pre-trained video generation model.
  • Implicit Camera Conditioning: Camera trajectories are converted into optical flow and discretely transported across the initial noise grid while preserving the standard Normal distribution.The method aggregates noise along flow-induced correspondences and normalizes overlapping contributions to prevent variance collapse and missing noise.
  • Reward Design: The composite reward combines physics-grounded 3D consistency with general visual quality, evaluating meta-view geometry, rendering fidelity, trajectory alignment, and HPSv3.Depth Anything 3 reconstructs generated videos as 3D Gaussian Splatting representations, while Qwen3-VL evaluates meta-view renderings and LPIPS measures perceptual fidelity.
  • Pure Text Dataset: The Pure Text Dataset contains approximately 3,000 unique Gemini-generated scene descriptions spanning diverse environments without fixed visual priors.Its categories include natural landscapes, urban structures, and surrealist environments, supporting world simulation independently of specific video distributions.
  • Periodic Decoupled Training: Periodic decoupled training preserves dynamic scene generation by disabling the 3D-aware reward during dynamic fine-tuning after every 100 training steps.The dynamic subset contains approximately 500 prompts describing highly dynamic scenes, while the primary stage uses the full weighted reward.

5 Experiments

Experiments evaluate World-R1’s geometric integrity and general video quality against foundation and auxiliary camera-control models. Results show stronger 3D consistency, preserved generation quality, positive human preference, and identifiable contributions from its reward components.

  • Experimental Setup: World-R1 is evaluated against Wan 2.1, CogVideoX, CameraCtrl, and ReCamMaster using protocols covering geometric integrity and general generative quality.Wan 2.1 and CogVideoX are foundation-model baselines, while CameraCtrl and ReCamMaster provide explicit camera-guidance comparisons.
  • Quantitative Results: 10.23dB and 7.91dB PSNR improvements, alongside substantial SSIM and LPIPS gains, demonstrate stronger 3D consistency than all baselines.The evaluation reconstructs scene geometry with 3DGS and re-renders the test set to assess structural stability and geometric hallucinations.
  • General Generation Quality: World-R1 surpasses the original Wan 2.1 backbone and far exceeds explicit-camera models in Aesthetic Quality, Imaging Quality, and Subject Consistency.These results indicate that geometric alignment improves consistency without sacrificing the reported VBench quality dimensions.
  • User Study: 86% of comparisons favored World-R1 in a blind study spanning 30 complex prompts, 25 participants, and criteria for geometry, camera accuracy, and visual quality.The study compared World-R1-Small and World-R1-Large with their respective Wan 2.1 base models.
  • Ablation Study: The ablation study identifies the 3D-aware reward R3D as fundamental for geometric consistency and the general reward Rgen as indispensable for perceptual fidelity.The analysis systematically isolates reward functions, model conditioning, and data training strategy.

6 Conclusion

World-R1 presents a scalable reinforcement-learning paradigm for equipping video generation models with robust world-modeling capabilities. It elicits latent spatial awareness from pre-trained models without explicit 3D architectural modules or expensive supervised datasets.

  • World-R1 reframes video-generation and 3D-geometry alignment as a reinforcement-learning problem.
  • The approach elicits latent spatial awareness from pre-trained video-generation models without explicit 3D architectural modules.
  • World-R1 avoids requiring expensive supervised datasets while targeting robust world-modeling capabilities.

Impact Statement · A Implementation Details · A.1 Reward Formulation and Details

World-R1 aligns video synthesis with 3D constraints through reinforcement learning and flow-matching optimization, improving geometric consistency for physically accurate applications. Its 3D-aware reward combines meta-view structural integrity, reconstruction fidelity, and camera-trajectory alignment.

  • Impact Statement: The method aligns video synthesis with 3D constraints through reinforcement learning and flow-matching optimization, targeting reliable generation for physical world modeling and autonomous-driving simulation.The stated societal and ethical considerations align with those of existing high-fidelity video generation technologies.
  • A Implementation Details: The 3D-aware reward lifts generated clips into a 3D Gaussian Splatting representation while estimating camera trajectory, then combines Smeta, Srecon, and Straj.The reward is computed as R3D = Smeta + Srecon + Straj, with all rewards limited according to the formulation.
  • A.1 Reward Formulation and Details: Smeta detects 3D hallucinations that appear plausible in 2D by rendering the reconstructed scene from a substantially offset meta-view.This view is designed to expose flat billboard-like objects and other structural failures hidden from the canonical viewpoint.
  • A.1 Reward Formulation and Details: Qwen3-VL evaluates meta-view structural plausibility as a graphics expert, identifying floaters, geometric distortion, and texture stretching.The critic assesses artifacts in the reconstructed 3D representation rather than judging only the original video view.
  • A.1 Reward Formulation and Details: The meta-view score is multiplied by 0.1, placing the final value in [0, 1] and suppressing the cardboard-prop effect revealed in baseline reconstructions.The canonical view may appear structurally sound while the offset meta-view exposes poor 3D consistency.
  • A.1 Reward Formulation and Details: Srecon equals 1 − LPIPS(x, ˆx), rewarding reconstructions whose re-rendered views preserve pixel-level similarity to the generated content.The scene is re-rendered from the estimated trajectory ˆE to obtain ˆx before computing LPIPS.
  • A.1 Reward Formulation and Details: Straj measures deviation between the specified trajectory E and estimated motion ˆE using negative exponential penalties for translation L2 distance and rotation geodesic distance.This term encourages camera movement to follow the user’s trajectory instructions.

A.2 More Training Details

World-R1 training uses Wan 2.1 video foundation models in Small and Large variants, with scaling compute requirements on NVIDIA H200 clusters. Periodic Decoupled Training alternates full reward optimization with dynamic fine-tuning to balance 3D consistency and dynamic preservation.

  • Infrastructure and Base Models: World-R1 builds on Wan 2.1 video foundation models and trains Small and Large variants.World-R1-Small is initialized from Wan2.1-T2V-1.3B, while World-R1-Large is initialized from Wan2.1-T2V-14B.
  • Infrastructure and Base Models: The Small variant uses 48 NVIDIA H200 GPUs, whereas the Large variant uses 96 GPUs for higher memory and computational demands.The optimization is distributed across high-performance compute clusters equipped with NVIDIA H200 GPUs.
  • Periodic Decoupled Training: Periodic Decoupled Training alternates geometric alignment with dynamic preservation through recurring training cycles.The primary phase optimizes the mixed dataset with the composite reward R3D + λgenRgen, while dynamic fine-tuning occurs every 100 training steps with R3D temporarily suspended.

B Dataset Details … C.2 Evaluation Dataset

World-R1’s dataset combines approximately 3,000 Gemini-generated pure-text entries with structured camera controls and diverse geometric, dynamic, and stylistic scenarios. Its evaluation uses a distinct fixed set of 30 complex prompts and a blind 2AFC user study against Wan 2.1.

  • B Dataset Details: Approximately 3,000 unique entries form a proprietary Pure Text Dataset designed to separate geometric constraint learning from biases in open-domain video distributions.The entries were synthesized using Gemini’s instruction-following and creative reasoning capabilities.
  • B.1 Generation Pipeline: A structured Gemini-3 pipeline generates text-video pairs through expert-cinematographer instructions and predefined camera controls, with outputs organized by prompt, camera logic, domain, and layout type.The action space includes push_in, pull_out, lateral, orbit, pan, and fixed movements, while trajectories are categorized as intra-scene, inter-scene, composite, or static.
  • B.2 Dataset Taxonomy and Examples: The taxonomy spans natural landscapes, urban and architectural scenes, micro and still-life observations, fantasy and surrealism, and artistic styles.These categories target terrain and fluids, perspective correctness, depth of field and texture fidelity, non-Euclidean structures, and stylistic diversity.
  • B.2.1 Natural Landscapes: Natural-landscape prompts test large-scale rigid geometry, organic structures, liquid physics, reflections, atmospheric rendering, and dynamic lighting across landforms, water features, and weather-time conditions.Examples combine scenes with camera actions such as orbit, pan, pull_out, fixed, and push_in.
  • B.2.2 Urban and Architectural: Urban and architectural prompts emphasize strict perspective correctness, vanishing points, straight-line preservation, and complex camera transitions through city, infrastructure, and interior scenes.The examples include metro stations, bridges, factories, platforms, and other structured environments.
  • B.2.3 Micro and Still Life: Micro and still-life prompts evaluate depth-of-field handling and texture fidelity through macro-scale objects, surfaces, materials, and microscopic scenes.Examples include keyboards, flowers, flames, food, sand, fabric, corrosion, bubbles, and biological structures.
  • B.2.4 Fantasy and Surrealism; B.2.5 Artistic Styles: Fantasy and surrealism introduce non-Euclidean geometries and physics-defying structures, while artistic-style entries require distinct stylized renderings without reducing aesthetic diversity.Examples include floating libraries, shifting gravity, suspended ruins, watercolor, black-and-white photography, cyberpunk illustration, and fantasy digital art.

C.3 Evaluation Criteria

The evaluation uses pairwise user judgments of geometric consistency, camera-control accuracy, and overall visual quality. World-R1 achieves strong win rates against Wan 2.1, especially on structural stability and physical reasoning.

  • Evaluation Criteria: Participants selected the superior video in each pair using three independent criteria with specific definitions for consistent grading.The criteria were Geometric Consistency, Camera Control Accuracy, and Overall Visual Quality.
  • Evaluation Criteria: World-R1 achieves a 92% win rate over Wan 2.1 for Geometric Consistency, indicating superior structural stability and reduced hallucination.Evaluators assessed whether objects remained solid, backgrounds stayed fixed relative to camera movement, and perspective changes exhibited correct parallax.
  • Evaluation Criteria: World-R1 achieves a 76% win rate for Camera Control Accuracy, reflecting better instruction following for complex trajectories.Evaluators checked adherence to directional commands and penalized drift, static cameras when motion was requested, and incorrect movement directions.
  • Evaluation Criteria: Users give World-R1 an 86% Overall Preference win rate, preferring aligned, consistent videos despite the constraints.Overall visual quality considers fidelity, aesthetics, artifacts, and coherence while capturing the trade-off between physics adherence and artistic quality.

C.4 Detailed Results … D.6 Comparison with 3D-Aware Generation Methods

Across detailed and supplementary evaluations, World-R1 improves perceived and measured 3D consistency across camera control, multi-view agreement, dataset scales, video lengths, scene complexities, and comparisons with 3D-aware methods. These results support its post-training alignment approach while preserving video quality and avoiding inference-time architectural changes.

  • C.4 Detailed Results: World-R1 achieved a 92% win rate in Geometric Consistency against Wan 2.1, whose individual frames were visually pleasing but inconsistent over time.The win rate aggregates participant responses across prompts and denotes instances where World-R1 was selected as superior to the baseline.
  • C.5 Metric Validation Study: The metric-validation study compared majority human preferences with automatic metric rankings across 20 participants and 30 randomized video pairs.Agreement measures how often human preference for 3D consistency matches the automatic metric ranking.
  • D Additional Experimental Analyses; D.1 Camera-Control Accuracy: Supplementary analyses evaluate camera-control accuracy with RotErr, TransErr, and CamMC under a reference-video trajectory protocol.World-R1 remains competitive with specialized camera-control methods while preserving stronger general video quality, despite being a post-training alignment method.
  • D.2 Reconstruction-Independent Multi-View Consistency: World-R1 improves reconstruction-independent MVCS for both small and large backbones, indicating that gains arise from generated videos rather than 3DGS optimization artifacts.The evaluation follows GeoVideo and directly measures cross-view agreement from generated videos.
  • D.3 Dataset Scaling: Increasing the pure-text post-training dataset from 1K to 3K prompts consistently improves both 3D consistency and general video quality.The trend indicates data efficiency at 3K prompts and suggests further prompt-generation scaling could yield additional improvements.
  • D.4 Long-Video Generalization: World-R1-Large substantially improves all 3D-consistency metrics over Wan2.1-T2V-14B on 121-frame generations.Although training uses short clips, the result suggests learned geometric alignment transfers beyond the training horizon.
  • D.5 Scene-Complexity Breakdown: World-R1-Small consistently improves over Wan2.1-T2V-1.3B across static scenes, object dynamics, non-rigid motion, and selected long-horizon dynamics.Long-horizon and non-rigid generation remain the largest challenges because final quality is partly limited by the base model’s compositional and motion-generation capacity.
  • D.6 Comparison with 3D-Aware Generation Methods: Against publicly available 3D-aware video generation methods, World-R1 achieves the strongest 3D-consistency metrics without inference-time architectural changes.The comparison covers camera-control methods with explicit control modules and 3D-conditioned methods using architectural modifications or image-to-video pipelines.

D.7 Reward Hacking Analysis

World-R1 addresses reward hacking by combining multiple reward signals rather than optimizing a narrow 3D metric alone. Component ablations show that the full pipeline provides the best trade-off between 3D consistency and general video quality.

  • Reward hacking risk: Optimizing only a narrow 3D metric could produce near-static clips that reconstruct easily but fail to follow camera motion or preserve dynamic content.This is identified as a potential failure mode of reinforcement-learning-based alignment.
  • Reward design: The composite reward combines meta-view plausibility, reconstruction fidelity, trajectory alignment, and general video-generation quality to avoid reliance on a single metric.The design uses multiple complementary signals to mitigate degenerate shortcuts.
  • Reward ablations: Removing any reward component degrades performance, while the full pipeline achieves the best overall trade-off across 3D consistency and VBench quality.These findings come from the component-wise reward ablations in Table I.
  • Reward ablations: Removing Straj weakens trajectory adherence, whereas removing Smeta or Srecon reduces geometric consistency.The ablation links each decomposed reward signal to its intended behavior.

E More Visualizations

World-R1 preserves dynamic, non-rigid motion while enforcing 3D consistency. Qualitative reconstructions show high-fidelity scene recovery and fewer geometric artifacts than inconsistent videos.

  • Dynamic video results: World-R1 preserves natural motion in dynamic scenes despite strict 3D consistency rewards.Periodic decoupled training and dynamic data help retain high-entropy generation and disentangle camera motion from object motion.
  • 3D consistency via 3D reconstruction: High-fidelity 3DGS re-renderings closely match World-R1’s original videos, indicating 3D-consistent sequences with minimized geometric hallucinations.Because 3DGS optimization assumes static scenes viewed from moving viewpoints, reconstruction quality is sensitive to geometric inconsistencies.
  • 3D consistency via 3D reconstruction: World-R1 recovers dense, accurate 3D scene representations without the floaters and distortion seen in baseline methods.The reconstruction visualizations attribute this recovery to preserved strict 3D consistency.
  • 3D consistency via 3D reconstruction: Wan 2.1 reconstruction fails under temporal warping and object morphing, producing noise, sparse point clouds, and visual artifacts.These failures prevent 3D optimization from forming a coherent structure.
Loading 2604.24764v4…