Source-linked AI summary
Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioning
Zixuan Wang, Yixin Hu, Wen Li, Feng Chen, Yan Liu, Duo Peng, Yinjie Lei
TL;DR
Physically plausible video generation remains difficult because natural-language and chain-of-thought conditioning often describe phenomena holistically without specifying causal progression and intermediate states. The paper models evolution as a chain of physically constrained events and uses shared visual and semantic guidance for diffusion generation. Across four benchmarks, it reports superior physical plausibility, while identifying longer horizons, compositional scenes, and 3D-aware dynamics as future scope.
Problem
Existing PPVG models and prompts often fail to represent the complete causal progression, intermediate states, and transition dynamics of physical phenomena.
Method
The framework decomposes phenomena into scene-graph event chains with physical quantities, routes events to transition-specific keyframe operators, and applies contrastive semantic guidance.
Results
The framework consistently outperforms current PPVG approaches across PhyGenBench, VideoPhy, PhyWorldBench, and Physics-IQ on physics-informed metrics.
Takeaways & Limitations
Generated videos preserve both the chronological order of physical events and the completeness of their intermediate states.
Takeaways & Limitations
Future work must address longer video horizons, complex compositional scenes with multiple interacting objects, and 3D-aware dynamics under occlusion.
Abstract
from arXiv · showhide
Physically Plausible Video Generation (PPVG) seeks to synthesize videos consistent with physical principles, yet remains challenging due to underspecified natural language conditioning. Advanced chain-of-thought (CoT) frameworks augment prompts with physical knowledge. However, such prompts describe physical phenomena holistically, overlooking intermediate states and transition dynamics. In this paper, we reformulate PPVG as event-centric generation by representing physical evolution as a chain of causally connected and physically constrained events. Our framework comprises three key modules: (1) Physics-driven Event Chain Reasoning. This module decomposes physical phenomena into causally connected events represented by evolving scene graphs. Formula-derived physical quantities are bound to relevant objects and interactions, characterizing the direction and magnitude of each event transition. (2) Transition-aware Routed Keyframe Conditioning. This module routes each event to a specialized keyframe synthesis operator for appearance variation or object transformation. Consecutive keyframes are injected as residual guidance during denoising, enabling smooth visual transitions between event-boundary states. (3) Physics-injected Contrastive Semantic Guidance. This module constructs physics-informed positive and counterfactual negative prompts for classifier-free guidance, steering generation toward plausible dynamics and away from physics-violating counterparts. Experiments on PhyGenBench, VideoPhy, PhyWorldBench, and Physics-IQ demonstrate that our framework generates videos with superior physical plausibility across diverse domains.
1 INTRODUCTION
The paper reformulates physically plausible video generation as event-centric modeling because existing prompts and conditioning often fail to represent causal progression and intermediate transitions. It combines physics-driven event reasoning with transition-aware visual and contrastive semantic conditioning, and reports consistent gains across four benchmarks.
- Motivation: Existing text-to-video models can produce photorealistic scenes but often fail to preserve physically plausible dynamics.This limits their reliability for complex physical phenomena.
- Challenges: Prior prompt-enhancement approaches augment descriptions with physical principles but often generate a holistic snapshot instead of complete causal evolution.The missing progression includes intermediate physical states and transitions between events.
- Results: Across PhyGenBench, VideoPhy, PhyWorldBench, and Physics-IQ, the framework consistently outperforms current PPVG approaches on physics-informed metrics.Diagnostic analysis finds preservation of chronological event order and completeness of intermediate states.
- Approach: The framework represents physical phenomena as chains of causally connected events defined by evolving scene graphs and formula-derived physical quantities.These quantities characterize event boundaries and transition dynamics, providing structured causal modeling.
- Approach: Event-adaptive visual conditioning synthesizes keyframes for successive physical transitions, while contrastive semantic conditioning separates plausible dynamics from physics-violating counterfactuals.The visual and semantic cues are combined to guide diffusion-based video generation.
- Contributions: The extended framework adds physics-quantity augmentation, adaptive routing between appearance and object-transformation operators, counterfactual negative prompts, broader benchmarks, and additional ablations.These changes distinguish the extension from the preliminary conference version.
2 RELATED WORKS
Related work studies physical plausibility, chain-of-thought reasoning, and cross-modal conditioning for visual and video generation. These directions provide physical knowledge, step-by-step conditioning, or complementary visual cues, but existing approaches address different aspects of the generation problem.
- Physically Plausible Video Generation: Physically plausible video generation uses graphics engines, differentiable simulators, feedback-driven optimization, or physics-aware prompt enhancement.These approaches aim to make generated videos obey physical laws.
- Physically Plausible Video Generation: Simulation-based methods model motion and interactions through graphics engines or differentiable simulators.They constitute one major category of physics-aware generation approaches.
- Chain-of-Thought in Visual Generation: Chain-of-thought methods either augment conditioning before generation or embed step-by-step reasoning during synthesis.Examples include description refinement, spatial cue generation, and progressive structural assembly.
- Chain-of-Thought in Visual Generation: Visual chain-of-thought approaches organize reasoning around semantic composition or spatial planning rather than physical evolution.Their reasoning targets fine-grained visual synthesis and spatial operations.
- Cross-modal Conditioning in Video Generation: Cross-modal conditioning supplements language with reference images, spatial layouts, or motion priors to convey appearance, geometry, and motion information.Natural language alone often does not specify these visual properties adequately.
3 METHODOLOGY
The framework models physically plausible video generation as causally connected event chains, then converts event transitions into visual and semantic guidance for video synthesis.
- Physics-driven Event Chain Reasoning: PECR decomposes a physical phenomenon into causally ordered scene graphs, with each graph representing an event-boundary configuration.Differences between consecutive graphs define events and encode qualitative or quantitative changes in objects and relations.
- Physics-driven Event Chain Reasoning: PECR retrieves applicable physical formulas and assigns derived quantities to scene-graph nodes and edges to characterize event dynamics.The quantities are checked across consecutive graphs and recalculated when changes violate inferred physical constraints.
- Transition-aware Routed Keyframe Conditioning: TRKC routes each event to a specialized operator for appearance or configuration changes, progressively editing keyframes from the preceding keyframe.The resulting keyframes are converted into smoothly varying visual guidance for video generation.
- Transition-aware Routed Keyframe Conditioning: TRKC interpolates consecutive keyframe features and residually injects accumulated feature deviations into the corresponding denoising latents.The guided features are concatenated along the sequence dimension to form updated noisy video features.
- Physics-injected Contrastive Semantic Guidance: PCSG condenses event descriptions into positive semantic conditions and constructs counterfactual negative conditions by violating event-level physical laws.Both conditions provide contrastive semantic guidance toward plausible progression and away from unphysical dynamics.
4 EXPERIMENTS
Across four benchmarks, the framework achieves leading physical-plausibility performance and preserves chronological event order and intermediate states. Ablations and qualitative comparisons attribute these gains to physics-driven reasoning, transition-aware keyframes, and contrastive semantic guidance.
- Benchmark results: The framework achieves leading performance across PhyGenBench, VideoPhy, PhyWorldBench, and Physics-IQ, covering diverse physical domains and interaction types.Diagnostics further show preservation of reasonable event chronology and complete intermediate states.
- Benchmark results: 72.5% average PCA on PhyGenBench with CogVideoX-5B, with average-score gains of 27.5, 10.0, 30.8, and 18.1 across four evaluated backbones.Domain scores are 70.0% for mechanics, 78.7% for optics, 76.7% for thermal, and 64.2% for material.
- Benchmark results: 58.5% overall SA-PC on VideoPhy with CogVideoX-5B versus 45.9% for PhysHPO, while Wan2.2-14B improves from 39.2% to 57.3%.The approach improves scores for solid-solid, solid-fluid, and fluid-fluid interactions.
- Benchmark results: On PhyWorldBench, CogVideoX-5B rises from 19.2% to 31.4% for fundamental physics and from 0.0% to 5.7% for anti-physics.Wan2.2-14B achieves 36.8% and 7.6% in the two categories, respectively.
- Benchmark results: On Physics-IQ, the overall score increases from 38.3% to 44.9%, with higher Spatial IoU and Weighted Spatial IoU and lower MSE than WMReward.The comparison is conducted under image-conditioned physical continuation.
- Qualitative analysis: Qualitative examples show progressive crumpling, combustion, charring, sinking, deformation, material blending, water displacement, and long-chain melting events.The framework maintains continuous progression rather than delayed events or skipped intermediate states.
- Ablation study: Removing TRKC or PCSG lowers the average score from 72.5% to 66.9% or 69.2%, respectively, disrupting intermediate evolution or introducing counterfactual dynamics.TRKC grounds successive physical states, whereas PCSG suppresses implausible changes.
- Ablation study: Specialized keyframe simulators raise the average score from 68.8% to 71.5%, while full TRKC reaches 72.5%; removing SKGD reduces it to 66.3%.SKGD uses successive-keyframe differences as timestep-varying residual guidance during denoising.
5 CONCLUSION AND FUTURE WORK
The paper reformulates physically plausible video generation around causally ordered event transitions represented by physics-augmented scene graphs. It uses these transitions for visual and semantic guidance, while identifying longer horizons, compositional scenes, and 3D-aware dynamics as future directions.
- Conclusion: The framework decomposes physical phenomena into causally ordered events represented by transitions between consecutive scene graphs augmented with physical quantities.The event chain provides a shared basis for visual and semantic guidance in video diffusion.
- Conclusion: Transition-aware visual conditioning establishes event-boundary keyframes and coherent evolution, while contrastive semantic conditioning steers generation toward plausible dynamics.The two guidance forms address visual transitions and semantic physical plausibility in parallel.
- Future work: Future work targets 1–5 minute physical videos, complex compositional scenes with multiple interacting objects, and 3D-aware dynamics under occlusion.Proposed directions include long- and short-term memory, attention refocusing, and simulator-in-the-loop generation.