PaperlayerView exact revision on arXiv

Source-linked AI summary

PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models

Qiyuan Zhang, Biao Gong, Shuai Tan, Zheng Zhang, Yujun Shen, Xing Zhu, Yuyuan Li, Kelu Yao, Chunhua Shen, Changqing Zou

arXiv:2601.11087v1cs.CV

TL;DR

Transformer-based video generators lack explicit physical grounding, especially for rigid-body motion, because training prioritizes perceptual and distributional objectives. PhysRVG introduces physics-aware reinforcement learning with MDcycle and evaluates it using PhysRVGBench, with experiments reporting gains over existing methods and improved training stability.

  • Problem

    Transformer-based video generators lack explicit physical constraints, leading to unstable trajectories, implausible collisions, and limited physical consistency in rigid-body motion.

  • Method

    PhysRVG combines a physics-grounded reward metric with the Mimicry-Discovery Cycle to inject rigid-body physical knowledge through reinforcement learning.

  • Results

    PhysRVG achieves clear gains on VideoPhy-2 and PhysRVGBench, consistently outperforming existing methods on IoU and TO while maintaining high visual quality on VBench.

  • Takeaways & Limitations

    PhysRVG provides a unified framework for improving physical modeling in video generators and a benchmark for evaluating rigid-body motion quality.

  • Takeaways & Limitations

    The study focuses on four rigid-body motion types: collision, pendulum, free fall, and rolling.

Abstract

from arXiv · show

Physical principles are fundamental to realistic visual simulation, but remain a significant oversight in transformer-based video generation. This gap highlights a critical limitation in rendering rigid body motion, a core tenet of classical mechanics. While computer graphics and physics-based simulators can easily model such collisions using Newton formulas, modern pretrain-finetune paradigms discard the concept of object rigidity during pixel-level global denoising. Even perfectly correct mathematical constraints are treated as suboptimal solutions (i.e., conditions) during model optimization in post-training, fundamentally limiting the physical realism of generated videos. Motivated by these considerations, we introduce, for the first time, a physics-aware reinforcement learning paradigm for video generation models that enforces physical collision rules directly in high-dimensional spaces, ensuring the physics knowledge is strictly applied rather than treated as conditions. Subsequently, we extend this paradigm to a unified framework, termed Mimicry-Discovery Cycle (MDcycle), which allows substantial fine-tuning while fully preserving the model's ability to leverage physics-grounded feedback. To validate our approach, we construct new benchmark PhysRVGBench and perform extensive qualitative and quantitative experiments to thoroughly assess its effectiveness.

1. Introduction

Transformer-based video generators achieve data-driven visual realism but lack explicit physical grounding, producing unstable dynamics and implausible rigid-body interactions. PhysRVG addresses this gap with physics-aware reinforcement learning, MDcycle, and PhysRVGBench for quantitative evaluation.

  • 1. Introduction: Physical principles underpin realistic simulation, but data-driven video generation lacks explicit constraints for physically consistent dynamics.Traditional graphics and simulation encode Newtonian mechanics and numerical solvers, whereas transformer-based generators rely primarily on statistical synthesis.
  • 1. Introduction: Pretrain-finetune objectives prioritize pixel reconstruction and perceptual quality, treating object rigidity and physical laws as auxiliary constraints.Distributional alignment can therefore be favored over physical consistency during scaling.
  • 1. Introduction: PhysRVG enforces physical collision rules in high-dimensional video-generation spaces through physics-aware reinforcement learning.Its physics-grounded reward integrates motion masks, trajectory offsets, and collision detection to help models internalize physical knowledge.
  • 1. Introduction: MDcycle alternates Mimicry and Discovery to support substantial adaptation while transitioning from data-driven learning toward physics-consistent generation.Mimicry captures visual patterns and addresses unreliable early reward signals; Discovery progressively internalizes physical rules.
  • 1. Introduction: PhysRVGBench contains 700 videos spanning collision, pendulum, free-fall, and rolling motions, with motion masks and IoU and TO metrics for evaluation.The benchmark combines manually annotated first-frame coordinates with SAM2-generated masks.

2. Related Work

Related work enhances physical fidelity in video generation through explicit knowledge conditioning and reinforcement-learning-based approaches. These methods include simulation-derived motion representations, planning and rendering pipelines, optical-flow cues, and feedback-based optimization.

  • 2. Related Work: Physical-fidelity methods can be categorized by how they inject knowledge into video generation.The paper identifies explicit conditioning as one major class.
  • 2. Related Work: Explicit-conditioning approaches use rigid-body simulation, GPT-4o planning with Blender, or optical flow to provide motion-related guidance.The cited methods derive motion sequences, edge and depth maps, or optical-flow representations.
  • 2. Related Work: Reinforcement-learning work uses human preferences or verifiable outcomes as feedback, with PPO and GRPO providing optimization strategies.GRPO is described as more efficient than PPO because it does not require a separate value model.

3. Methodology

PhysRVG combines physics-grounded reinforcement learning with an adaptive Mimicry-Discovery Cycle to train video generators on rigid-body motion. Its reward measures trajectory discrepancy and emphasizes collision frames, while the unified objective supplements reinforcement learning with Flow Matching supervision for difficult samples.

  • Evaluation: PhysRVG evaluates generated videos against VBench, VideoPhy-2, and PhysRVGBench, whose comparisons include IoU and Trajectory Offset.The framework trains all transformer blocks with full parameters, and PhysRVGBench provides rigid-motion evaluation measures.
  • Task Definition: PhysRVG formulates rigid-body video generation as predicting future frames from observed context frames and a text prompt.The task focuses on rigid-body motion, whose position is represented through measurable coordinate transformations.
  • Reward Modeling: The physics-grounded reward combines trajectory offset and collision detection to measure physical consistency.Object masks extracted with SAM2 yield trajectories, while acceleration changes identify collision timestamps for temporal reweighting.
  • Reward Modeling: The weighted trajectory offset assigns greater importance to collision and adjacent frames, preventing optimization from favoring simple linear motion that avoids interactions.Collision detection uses sudden acceleration changes under Newton’s second law, with temporal weights applied around detected collisions.
  • Reward Modeling: The reinforcement-learning reward is R = −Oc, so higher rewards correspond to smaller trajectory discrepancies and stronger physical consistency.The weighted offset serves as the per-sample score used to guide policy optimization.
  • Mimicry-Discovery Cycle: MDcycle switches between Mimicry and Discovery using group-average trajectory offset, adding Flow Matching loss when a group exceeds a performance threshold.Mimicry stabilizes difficult early cases with pixel-level supervision, while Discovery relies primarily on the reinforcement-learning objective for better-performing groups.

4. Experiment

PhysRVG is evaluated with visual-fidelity and physics-focused benchmarks, where it improves rigid-body motion realism over existing methods. Ablations further show that MDcycle, collision-aware rewards, and tuned training choices improve stability and physical consistency.

  • Quantitative Comparisons: PhysRVG achieves clear gains on VideoPhy-2 and PhysRVGBench while consistently outperforming existing methods on IoU and TO.The comparison also reports higher physical realism for V2V models than I2V models, with PhysRVG improving over the strongest V2V baseline.
  • Qualitative Comparisons: Qualitative comparisons show that competing methods produce incorrect trajectories, static objects, tearing, overlap, and unnatural merging in rolling and collision scenes.PhysRVG preserves physical integrity and motion coherence across these settings, accurately capturing rolling and collision behaviors.
  • Training Strategy Ablation: MDcycle converges more smoothly, reaches higher rewards, increases high-quality samples, and reduces low-quality samples compared with FT and FT+RL.Pure RL remains unstable during its initial 50 steps and settles at a lower reward level, while MDcycle achieves the best results across all metrics.
  • Collision Detection Ablation: Collision-aware rewards eliminate object disappearance and inaccurate trajectories that remain under FT or RL, yielding physically consistent motions.The collision-weight analysis reports stable performance within a reasonable weighting range.
  • Hyperparameter Analysis: A proper threshold balances mimicry-driven stabilization with discovery-driven exploration, while full-parameter fine-tuning benefits saturate around 16k steps.The V2V process mainly provides visual-level data alignment, whereas further physical-knowledge learning relies on MDcycle.

5. Conclusion

The paper presents PhysRVG as a unified reinforcement learning framework for improving the physical modeling ability of video generative models. It introduces physics-grounded motion evaluation, MDcycle, and PhysRVGBench, with experiments demonstrating effectiveness.

  • PhysRVG is a unified reinforcement learning framework designed to enhance physical modeling in video generative models.
  • The framework addresses pretrain–finetune paradigms that prioritize pixel reconstruction and perceptual quality over physical realism.
  • PhysRVG introduces a physics-grounded metric for measuring motion fidelity and integrates abstract physical knowledge through MDcycle.
  • PhysRVGBench evaluates the quality of rigid-body motion in generated videos.
  • Extensive experiments demonstrate the effectiveness of the proposed approach.

Supplementary Material

MDcycle structures reinforcement learning around Mimicry and Discovery branches. The framework alternates between data-driven imitation and physics-aware exploration according to Algorithm 1.

  • MDcycle builds on reinforcement learning and contains complementary Mimicry and Discovery branches.
  • The Mimicry Branch represents data-driven imitation, while the Discovery Branch represents physics-aware exploration.
  • The two branches are integrated into a coherent framework that alternates between their respective learning modes.
  • Algorithm 1 provides a step-by-step description of the MDcycle process.

B. PhysRVG Architecture & Training Details

PhysRVG uses a two-stage training pipeline built on Wan2.2 5B TI2V, followed by MDcycle training for physics-aware generation. The supplementary details cover sampling, motion processing, rewards, dataset construction, and training considerations.

  • Architecture: PhysRVG is built on the pretrained Wan2.2 5B TI2V video generation model.
  • Architecture: Stage-1 fully fine-tunes the pretrained model into a video-to-video model using open-source and proprietary video collections with a fixed 5-frame context length.
  • Architecture: Stage-2 trains MDcycle to add physics-aware generation capability using high-quality rigid-body data.
  • Training: The training algorithm samples groups of videos with the same random initialization noise and extracts motion masks for each sample.
  • Training: Collision detection produces collision weights, which are used to compute weighted trajectory offset and define the reward as its negative.
  • Benchmark: PhysRVGBench covers collision, free fall, rolling, and pendulum motion using data from existing datasets, the Internet, video games, and real-world experiments.
  • Benchmark: The benchmark annotations include active and passive objects in collision scenes, with motion masks extracted using SAM2.

C.2. Evaluation Metrics

PhysRVGBench evaluates rigid-body physical realism using spatial overlap and trajectory deviation. Its evaluation procedure also identifies collision times from sharp acceleration changes inferred from motion masks.

  • PhysRVGBench evaluates rigid-body physical realism with Intersection over Union and Trajectory Offset.
  • IoU: IoU measures the overlap between predicted and ground-truth interaction regions.
  • Trajectory Offset: Trajectory Offset measures the discrepancy between generated and ground-truth motion trajectories by averaging per-frame coordinate distances.
  • IoU: IoU is defined as the area of overlap divided by the area of the union.
  • Collision Detection: Collision detection identifies collision times by locating sharp increases in acceleration derived from motion-mask dynamics.

E. More Generated Results.

PhysRVG generates physically plausible rigid-body motion across collision, pendulum, free-fall, and rolling scenarios, including out-of-distribution examples.

  • The model produces complex motions involving multiple objects and collision events in out-of-distribution collision scenarios.
  • Pendulum behavior generalizes from simple training setups to visually similar scenes such as playground swings.
  • Free-fall generations capture realistic falling motion and reasonably model subsequent impacts with the ground.
  • Rolling generations show repeated acceleration and deceleration along the track, consistent with track geometry and energy conservation.

F. More analysis on Threshold.

The Threshold controls Mimicry-branch participation in MDcycle training, balancing early stability against later RL exploration. Moderate threshold values provide the most stable convergence and highest final reward, though tuning remains empirical and task-dependent.

  • Threshold effects: Small thresholds stabilize early training but can cause premature convergence to a lower reward by restricting later RL exploration.Excessive pixel-level supervision from the Mimicry branch can constrain the model’s performance ceiling.
  • Threshold effects: Large thresholds produce convergence behavior nearly identical to pure RL.
  • Threshold effects: Moderate thresholds yield more stable convergence and a higher final reward.
  • Practical consideration: Threshold selection is largely empirical and must be tuned separately for different tasks.The authors plan to develop an automatic adjustment mechanism in future work.

G.1. Limitation

PhysRVG generates realistic motion trajectories but remains vulnerable to errors weakly correlated with primary motion, while its physics-focused optimization does not inherently encode ethical or safety constraints.

  • Model limitations: The model can produce highly realistic motion trajectories but still makes errors in attributes weakly correlated with the primary motion.
  • Model limitations: Observed failure cases include objects changing color after collision and an extra ball appearing when the original ball turns.
  • Reward scope: These undesirable cases are not supervised by the reward, which is solely related to the primary motion.
  • Responsible use: PhysRVG’s physics-accuracy objective could be misused to create convincing disinformation, including fabricated accidents.
  • Responsible use: The reinforcement learning framework does not inherently encode ethical or safety constraints and is agnostic to generated content.The authors commit to responsible release measures including research labeling, misuse documentation, provenance, and watermarking.
Loading 2601.11087v1…