PaperlayerView exact revision on arXiv

Source-linked AI summary

ProPhy: Progressive Physical Alignment for Dynamic World Simulation

Zijun Wang, Panwen Hu, Jing Wang, Terry Jingchen Zhang, Yuhao Cheng, Long Chen, Yiqiang Yan, Zutao Jiang, Hanhui Li, Xiaodan Liang

arXiv:2512.05564v2cs.CV

TL;DR

Video generators remain limited in physical consistency and in aligning generation with localized physical cues. ProPhy extracts hierarchical physical priors and aligns refinement experts with VLM-derived localization; experiments report state-of-the-art performance across standard and challenging scenarios.

  • Problem

    Video generators still struggle with physical consistency, while sample-level guidance lacks fine-grained alignment to localized physical cues.

    Source: §1 · p. 2, §2.2 · p. 3

  • Method

    ProPhy uses two-stage physics experts to extract and refine priors, with VLM-guided physical alignment for spatially localized dynamics.

    Source: §1 · p. 2

  • Results

    ProPhy consistently outperforms state-of-the-art methods across standard and challenging physics-related video-generation scenarios, particularly in dynamic physical consistency.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Experiments validate the importance of fine-grained physical modeling for generating physically coherent videos.

    Source: §5 · p. 8

    Annotated physical regions contain noise, and region-level categorization captures only coarse surface patterns.

    Source: §5 · p. 8

Abstract

from arXiv · show

Recent advances in video generation have shown remarkable potential for constructing world simulators. However, current models still struggle to produce physically consistent results, particularly when handling large-scale or complex dynamics. This limitation arises primarily because existing approaches respond isotropically to physical prompts and neglect the fine-grained alignment between generated content and localized physical cues. To address these challenges, we propose ProPhy, a Progressive Physical Alignment Framework that enables explicit physics-aware conditioning and anisotropic generation. ProPhy employs a two-stage Mixture-of-Physics-Experts mechanism for discriminative physical prior extraction, where Semantic Experts infer semantic-level physical principles from textual descriptions, and Refinement Experts capture token-level physical dynamics. This mechanism allows the model to learn fine-grained, physics-aware video representations that better reflect underlying physical laws. Furthermore, we introduce a physical alignment strategy that transfers the physical reasoning capabilities of vision-language models into the Refinement Experts, facilitating a more accurate representation of dynamic physical phenomena. Extensive experiments on physics-aware video generation benchmarks demonstrate that ProPhy produces more realistic, dynamic, and physically coherent results than existing state-of-the-art methods.

1. Introduction

ProPhy targets explicit physical guidance and fine-grained spatial alignment to improve physically consistent video generation. It combines hierarchical physics-prior extraction with VLM-guided alignment and outperforms state-of-the-art methods across standard and challenging scenarios.

  • Video generation models still struggle to produce physically plausible results, limiting their ability to function as world simulators.Source: §1 · p. 2
  • Prior approaches either lack explicit physical guidance at inference or provide global guidance that struggles with coexisting, localized phenomena.WISA identifies physical categories from prompts but its guidance remains largely global.Source: §1 · p. 2
  • The paper identifies two challenges: making physical laws more discriminative and aligning different spatial regions with localized physical cues.Source: §1 · p. 2
  • ProPhy’s two-stage Mixture-of-Physics-Experts first infers physics-specific priors from text, then refines them into token-wise priors fused with visual features.The Semantic Expert Block and Refinement Expert Block progressively extract and refine hierarchical priors.Source: §1 · p. 2
  • Its physical alignment strategy trains refinement experts to match spatial distributions inferred by VLMs, improving generation of dynamic processes, especially in high-motion scenarios.Source: §1 · p. 2
  • ProPhy consistently outperforms state-of-the-art methods on the physics-related video-generation benchmark across standard and challenging scenarios, particularly in dynamic physical consistency.Source: §1 · p. 2

2. Related Work

Related work spans video generators proposed as world simulators and methods that add physical knowledge to video generation. The paper highlights limits in physical realism and in aligning physical guidance with localized phenomena.

2.1. Video Generation as World Simulator

  • Diffusion Transformers and large-scale video models have advanced realistic video generation, but models still struggle to capture physical principles beyond scene appearance.Sora explores extensive video data for a general-purpose world simulator.Source: §2.1 · p. 3

2.2. Physics-Aware Video Generation

  • Physics-simulation approaches estimate and render dynamics but require predefined parameters and explicit rules, limiting generalization to diverse, unconstrained scenarios.Source: §2.2 · p. 3
  • Learning-based methods use interframe relations, while external-prior methods use language or vision-language models to guide generation, including physics-controllable synthesis.Source: §2.2 · p. 3
  • Existing methods largely use sample-level physical information, which can disperse physical awareness instead of focusing on distinct phenomena at their spatial locations.Source: §2.2 · p. 3

3. Method

ProPhy adds a physical branch to a video-diffusion backbone, progressively extracting and injecting video-level and token-level physical priors. Its training aligns semantic routing with global physical categories and refinement routing with VLM-derived localized targets.

3.1. Overall Framework

  • The physical branch adds Semantic Expert, Physical, and Refinement Expert blocks to a video-diffusion backbone, progressively injecting their outputs into video latents while preserving pretrained block weights.The Semantic Expert Block produces video-level priors; the Refinement Expert Block produces token-level priors for spatially anisotropic responses.Source: §3.1 · p. 3

3.2. Progressive Physical Routing

  • The Semantic Expert Block routes text-conditioned physical basis maps into a global prior, while the Refinement Expert Block selects top-k experts per token for token-level physical priors.The Semantic Expert Block uses continuous weighted routing to reduce mode-collapse risk with small training batches; the Refinement Expert Block uses standard top-k routing.Source: §3.2 · p. 4

3.3. Physical Alignment Objectives

  • Semantic alignment encourages samples in the same physical category to share routing weights, while different categories receive more divergent routing distributions.The semantic router weights are aligned with global physical semantics using WISA-80K category annotations.Source: §3.3 · pp. 4–5
  • Fine-grained alignment uses VLM-derived token-localization targets and a masked loss to guide refinement routing toward salient regions of annotated physical phenomena.The targets subtract a background attention map from the phenomenon map; the training objective also includes refinement load balancing and diffusion loss.Source: §3.3 · p. 5

4. Experiment

The experiments compare ProPhy on VideoPhy2 and VBench, then examine qualitative behavior, ablation results, and the physical knowledge captured by its experts.

4.1. Implementation Details

  • ProPhy fine-tunes Wan2.1-1.3B [33] and CogVideoX-5B on 20K randomly sampled WISA-80K videos, and is evaluated on VideoPhy2’s 600 prompts, including 180 more challenging prompts [2].Evaluation also assesses VBench quality-oriented dimensions using the identical set of 600 prompts.Source: §4.1 · pp. 5–6

4.2. Quantitative Comparisons

  • On VideoPhy2 ALL, ProPhy improves CogVideoX’s Joint metric by +19.7%, with consistent PC and SA gains; on HARD, it shows stronger semantic consistency and more faithful physical behaviors.Table 1 reports VideoPhy2 results.Source: Table 1 · p. 6, §4.2 · p. 6
  • On VBench, improvements are most notable in Dynamic Degree, and the aggregated Quality Score is higher overall.Table 2 reports VBench quality scores.Source: §4.2 · p. 6, Table 2 · p. 7

4.3. Qualitative comparison

  • In discus throwing, ProPhy triggers dust when the discus contacts the ground; in iron-ball collisions, it avoids penetration and depicts momentum transfer to the smaller ball.Figure 5 compares ProPhy with base generators and physics-aware approaches.Source: §4.3 · p. 6, Figure 5 · p. 7

4.4. Ablation Study

  • With equal training steps, the Physical Branch yields larger gains than LoRA alone, and adding both SEB and REB further improves physical commonsense and semantic adherence.Table 3 ablates ProPhy on Wan2.1-1.3B; its caption defines LoRA as removing the Physical Branch and applying LoRA to the backbone.Source: §4.4 · p. 6, Table 3 · p. 8
  • The full loss combination performs best across metrics; SEB BCE improves SA but weakens PC and Joint, while either REB loss alone degrades a complementary outcome.SEB BCE weakens PC and Joint; REB absolute-only degrades performance, while load-balance-only improves SA but reduces PC.Source: §4.4 · pp. 6–7, Table 4 · p. 8

4.5. Physics Learned by Experts

  • Semantic-router correlations are high for related categories such as combustion and explosion, and low for unrelated categories such as explosion and refraction.Figure 6 defines r as the Pearson correlation coefficient between distributions; the pattern indicates coherent physical semantics in the SEB.Source: §4.5 · p. 7, Figure 6 · p. 8
  • REB activations localize corresponding physical events, while assigning incorrect experts produces behaviors such as a rigid car door deforming like cloth.Figures 7 and 8 visualize expert activation maps and physical-attribute transfer by expert inversion.Source: §4.5 · pp. 7–8, Figure 7 · p. 8, Figure 8 · p. 8

5. Conclusion

ProPhy achieves state-of-the-art performance in physics-aware video generation by capturing fine-grained, anisotropic physical dynamics. Its conclusion also notes that physical-region annotations remain noisy and coarse.

  • ProPhy achieves state-of-the-art performance and produces physically coherent videos by capturing fine-grained, anisotropic physical dynamics.Source: §5 · p. 8
  • Its two-stage MoPE extracts hierarchical physical priors, while a physical alignment strategy transfers VLM reasoning into generation.Source: §5 · p. 8
  • Physical-region annotations are noisy and region-level categorization captures only coarse surface patterns; future work may integrate governing physical differential equations for more interpretable knowledge.Source: §5 · p. 8
Loading 2512.05564v2…