Source-linked AI summary
PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation
Peiwen Zhang, Yufan Deng, Shangkun Sun, Juncheng Ma, Duomin Wang, Jonas Du, Zilin Pan, Ye Huang, Hao Liang, Songyan Huang, Ruihua Zhang, Enze Xie, Ming-Yu Liu, Daquan Zhou
TL;DR
Current video generators can produce physically implausible contact-rich manipulations, undermining their reliability as visual world simulators. PhysisForcing aligns pixel-level trajectories and semantic relations in interaction-critical regions, consistently improving embodied video generation and raising WorldArena closed-loop success from 16.0% to 24.0%.
Problem
Current video generators produce local motion and global relational violations during contact-rich manipulation, undermining their reliability as visual world simulators.
Method
PhysisForcing focuses hierarchical physics supervision on interaction-critical regions through pixel-level trajectory alignment and semantic-level relational alignment.
Results
Across R-Bench, PAI-Bench, and EZS-Bench, PhysisForcing consistently surpasses strong baselines and raises WorldArena closed-loop success from 16.0% to 24.0%.
Takeaways & Limitations
Physically aligned video models provide stronger representations for robotic manipulation, improving downstream policy success within the evaluated protocols.
Takeaways & Limitations
PhysisForcing inherits its backbone’s capability ceiling, as limited world knowledge and long-horizon temporal reasoning bound achievable physical plausibility.
Abstract
from arXiv · showhide
Video generation models have emerged as a promising paradigm for embodied world simulation. However, both general-domain video generators and robot-specific data fine-tuned models can still produce physically implausible manipulations, including discontinuous motion trajectories and inconsistent robot-object interactions, which limits their reliability as world simulators. Through extensive experiments, we find that such physical instability mainly arises from two factors: deformation of moving objects and implausible spatio-temporal correlations among interacting entities, particularly during contact. Building on this observation, we propose PhysisForcing, a scalable training framework that strengthens physical consistency by focusing supervision on physics-informative regions through joint optimization of pixel-level and semantic-level features. The framework consists of a pixel-level trajectory alignment loss, which supervises DiT features using reference point trajectories, and a semantic-level relational alignment loss, which aligns DiT features with inter-region relations extracted from a frozen video understanding encoder. Extensive experiments on R-Bench, PAI-Bench, and EZS-Bench show that PhysisForcing consistently improves embodied video generation over strong baselines, improving the Wan2.2-I2V-A14B and Cosmos3-Nano base models on R-Bench by 22.3\% and 9.2\% (7.1\% and 3.7\% over vanilla finetuning), with the Cosmos3-Nano variant attaining the best overall score. Beyond generation, as a world model under the WorldArena action-planner protocol it raises the closed-loop success rate from 16.0\% to 24.0\% and further improves downstream policy success, indicating that physically aligned video models yield stronger representations for robotic manipulation.
1 Introduction
PhysisForcing addresses physically implausible contact-rich robotic video generation by applying hierarchical, region-focused alignment to local motion and semantic interactions. It improves embodied video generation across multiple benchmarks and backbones while benefiting world-model action planning and downstream policy learning.
- Motivation: Current generators produce discontinuous gripper motion, object penetration, anti-gravity motion, and inconsistent object responses during contact-rich manipulation.Such local and global relational violations undermine videos as valid visual consequences of robot actions.
- Method: Pixel-level alignment uses tracked points to enforce continuous, contact-compatible DiT feature trajectories, while semantic-level alignment captures evolving inter-region relations.These levels target local motion continuity and interaction semantics such as gripper-object coupling, object displacement, and support-surface contact.
- Method: PhysisForcing focuses supervision on manipulators, manipulated objects, contact areas, and moving parts rather than treating all pixels uniformly.The framework formulates physical plausibility as a hierarchical, region-focused alignment problem.
- Empirical results: 22.3% and 9.2% improvements over base models, and 7.1% and 3.7% over vanilla finetuning, are achieved on R-Bench physical plausibility with Wan2.2-I2V-A14B and Cosmos3-Nano, respectively.The Cosmos3-Nano variant attains the best overall R-Bench score among evaluated models; gains are also reported on PAI-Bench and EZS-Bench.
- Downstream impact: PhysisForcing improves embodied decision making by strengthening world-model action planning and downstream policy success when used as a video backbone.The contribution is evaluated through WorldArena action planning and downstream embodied policy learning.
2 Related Work
Prior work uses video generation for embodied intelligence, but general-domain models struggle with robot–object interaction dynamics, motivating robotics-specific world models and physics-aware alignment. Existing approaches emphasize geometric consistency or preference-based correction, leaving a need for hierarchical, localized supervision that captures both trajectories and semantic interactions.
- 2 Related Work: General-domain video models provide scalable visual simulation for embodied intelligence but often struggle with robot–object interaction dynamics, motivating robotics-specific world models.The related work identifies video generation as a response to robotic data scarcity and distinguishes general-domain models from robotics-specific alternatives.
- 2 Related Work: Geometry-based methods enforce physical consistency through depth prediction, keypoint tracking, or 3D scene reconstruction, improving local spatial structure and motion modeling.RoboScape jointly learns temporal depth prediction and keypoint dynamics, while RoboDreamer and CTRL-World address compositional imagination and controllable world modeling.
- 2 Related Work: Preference-based methods apply post-training alignment to suppress physically implausible generations such as object penetration and anti-gravity motion.ABot-PhysWorld uses DPO with physics-aware discriminators, while MIND-V uses GRPO with a Physical Foresight Coherence reward.
- 2 Related Work: Existing physics-aware video models often supervise a single level or the entire frame, whereas contact-rich manipulation requires hierarchical and localized physical supervision.Pixel-level constraints govern per-point trajectories and contacts, while semantic-level relational cues capture inter-region interactions; informative evidence is concentrated around manipulators and contact interfaces.
3 Method
PhysisForcing strengthens embodied video generation through region-focused hierarchical physics alignment. It combines pixel-level trajectory supervision with semantic-level relational supervision on physics-informative robot-object interaction regions, without adding inference cost.
- Region-focused physics supervision: PhysisForcing identifies contact-rich, physics-informative regions using trajectory motion and depth-aware foreground relevance, then forms a spatiotemporal physics mask.The mask guides both trajectory alignment and semantic relational supervision toward interaction-relevant regions.
- Pixel-level trajectory alignment: Pixel-level trajectory alignment uses intermediate DiT features to predict tracked point locations and applies masked mean squared error against reference trajectories.The supervision directly enforces per-point trajectory continuity for the manipulator and manipulated object within the physics mask.
- Semantic-level relational alignment: Semantic-level relational alignment matches DiT pairwise token relations with those from a frozen video understanding encoder over selected physics-informative tokens.This captures inter-region coupling such as coordinated gripper-object motion and contact-driven object movement.
- Joint training objective: During fine-tuning, PhysisForcing combines standard flow matching with pixel- and semantic-level physics losses on an intermediate DiT layer; auxiliary models are discarded at inference.Consequently, the framework introduces no extra inference cost.
4 Experiments
PhysisForcing consistently improves embodied video generation across R-Bench, PAI-Bench, and EZS-Bench, with PF-Cosmos achieving the strongest overall benchmark results. Its physics-aligned video representations also improve downstream policy execution and WorldArena action-planning success.
- EZS-Bench Evaluation: 81.1 on EZS-Bench, PF-Cosmos improves over vanilla finetuning (80.3→81.1) and surpasses Abot-PhysWorld (80.3) and all other baselines.The benchmark evaluates training-independent zero-shot generalization across 196 unseen robot-task-scene combinations.
- PAI-Bench Evaluation: 85.2 on PAI-Bench, PF-Cosmos surpasses vanilla finetuning (84.0→85.2), Wan2.5 (81.0), and Abot-PhysWorld (84.9).Wan2.2-I2V-A14B also improves from 79.9 to 81.7.
- R-Bench Evaluation: 63.8 on R-Bench, PF-Cosmos improves 9.2% over base and surpasses Wan2.6 (60.7), while PF-Wan reaches 62.0 (+22.3% over base).PhysisForcing improves every evaluated backbone, with PF-Wan ranking second overall.
- Downstream Policy Evaluation: 72.8% average downstream policy success, up from 68.2%, with contact-rich placing rising 41.5%→63.0% and pressing 49.0%→60.0%.PhysisForcing-Wan2.2-TI2V-5B replaces Fast-WAM’s video DiT on six RoboTwin 2.0 tasks, evaluated with 200 rollouts per task.
- World Model as an Action Planner: 24.0% WorldArena closed-loop success, up from 16.0%, surpassing WoW (20.5%) when paired with a shared inverse dynamics model.The protocol decodes predicted world-model rollouts into actions executed in the RoboTwin 2.0 simulator.
5 Conclusion
PhysisForcing improves robotic video generation by aligning pixel-level trajectories and semantic-level relations on interaction-critical regions. It consistently outperforms multiple baselines across three benchmarks, with PF-Cosmos best overall, while improving WorldArena and downstream policy success.
- Conclusion: PhysisForcing improves robotic video generation by aligning pixel-level trajectories and semantic-level relations on interaction-critical regions.This is presented as a training-time framework for strengthening physical consistency.
- Conclusion: PF-Cosmos consistently surpasses base models, vanilla finetuning, and strong open-source, commercial, and robotics-specific baselines, achieving the best overall results on R-Bench, PAI-Bench, and zero-shot EZS-Bench.The framework aligns pixel-level trajectories and semantic-level relations on interaction-critical regions.
- Conclusion: 16.0% to 24.0%: PhysisForcing raises WorldArena closed-loop success and further improves downstream policy success.These gains show that physical plausibility yields concrete benefits for embodied intelligence.
Manipulation · Appendix · A More implementation details
PhysisForcing uses frozen auxiliary models to extract shared physics targets and applies pixel- and semantic-level supervision across three video diffusion backbones. The implementation specializes dynamics-forming components while preserving backbone-specific training configurations.
- A More implementation details: Frozen auxiliary models extract physics targets online from ground-truth clips, with tracker and depth outputs shared by both physics losses.The auxiliary models are used only for target extraction during training.
- A More implementation details: V-JEPA 2 [3] provides semantic supervision through pairwise-cosine relations between aligned DiT and teacher tokens over mask-selected regions.A Wan2.2-I2V-A14B DiT block is mapped into V-JEPA 2’s feature space and resampled onto a shared 32×16×16 grid; relations use up to K=512 tokens.
- A More implementation details: CoTracker3 tracks 625 points from a regular 25×25 first-frame grid, while Depth-Anything-V2 [48] weights motion by normalized relative depth to retain active foreground tracks.The selected trajectories provide pixel-level supervision and define the semantic-loss physics mask after rasterization.
- A More implementation details: For Wan2.2-I2V-A14B [45], PhysisForcing fine-tunes only the high-noise expert because it forms global dynamics and physical structure.Training applies this expert across the full diffusion schedule while leaving the low-noise expert untouched.
- A More implementation details: Cosmos3-Nano [1] is fine-tuned in image-to-video mode on 720p robot-manipulation clips using the Wan2.2-VAE and physics supervision from a mid-depth MoT block.Its latent compression is T×H×W = 4×16×16, with an effective 4×32×32 compression after patchification.
B Evaluation benchmark details
The evaluation covers R-Bench, robot-domain PAI-Bench, and EZS-Bench, spanning physical plausibility, task adherence, and zero-shot cross-embodiment generalization. R-Bench combines five normalized sub-metrics into a human-validated per-prompt score, while PAI-Bench and EZS-Bench target complementary embodied-generation conditions.
- R-Bench: R-Bench evaluates 650 robotic manipulation and locomotion prompts across five task categories and five normalized sub-metrics aggregated into a [0, 1] score.Its metrics assess physical-semantic plausibility, task-adherence consistency, robot-subject stability, motion smoothness, and motion amplitude; scores correlate with human preferences at Spearman ρ=0.96 across 25 video generators.
- R-Bench: R-Bench uses Qwen3-VL temporal-grid inspection to detect physical violations and verify action order, while pixel- and tracking-based statistics measure motion quality.The evaluation penalizes floating components, object penetration, spontaneous appearance or disappearance, non-contact attachment, temporal jitter, and degenerate near-static outputs.
- PAI-Bench: PAI-Bench evaluation is restricted to its robot subset in the PAI-Bench-G generation track, comprising 174 real-world robotic image-prompt pairs with 5–6 physical-semantic QA pairs each.PAI-Bench spans six domains and three tracks, but this study selects the subset most aligned with embodied manipulation.
- EZS-Bench: EZS-Bench evaluates training-independent embodied zero-shot generation under fully out-of-distribution combinations of robot morphologies, environments, and tasks.Its dual-branch construction varies robots, scenes, and tasks while testing cross-embodiment physical fidelity without training-data overlap.
C Detailed benchmark results
PhysisForcing improves both physical-semantic and overall benchmark performance across the full PAI-Bench and EZS-Bench results for both backbones. PF-Cosmos achieves the best overall average, surpassing Abot-PhysWorld, with the largest gains on Domain Score.
- C Detailed benchmark results: Quality Score and Domain Score provide the detailed components underlying the average scores reported in Figures 3 and 4.
- C Detailed benchmark results: PhysisForcing improves Domain Score and overall average on both PAI-Bench and EZS-Bench for both backbones over vanilla finetuning.Tables 7 and 8 provide the full benchmark results.
- C Detailed benchmark results: PF-Cosmos achieves the best overall average across these benchmarks, surpassing the strongest robotics-specific baseline Abot-PhysWorld.
- C Detailed benchmark results: The gains are most pronounced on Domain Score, which measures physical-semantic plausibility.
D More qualitative results · E Broader impacts
PhysisForcing produces more physically consistent robotic manipulation videos than strong commercial and robotics-specific models, while generalizing across embodiments, scenes, and instructions. Its broader impact includes cheaper simulation and evaluation, alongside risks of deceptive synthetic footage and unsafe policy training.
- D More qualitative results: Across single-arm, dual-arm, and humanoid tasks, PhysisForcing maintains more continuous motion and consistent robot-object interactions than competing state-of-the-art models.Wan2.6, Seedance 1.5 Pro, Veo 3.1, Cosmos3-Super, and Abot-PhysWorld frequently drift, break contact, or deform objects.
- D More qualitative results: Qualitative ablations show that PhysisForcing improves physical plausibility beyond vanilla finetuning on both Wan2.2-I2V-A14B and Cosmos3-Nano.PF-Wan stabilizes object shape and grasp-and-place trajectories, while PF-Cosmos better preserves contact dynamics and inter-object relations across task types.
- D More qualitative results: The qualitative comparisons attribute these gains to hierarchical physics alignment rather than additional in-domain finetuning.This conclusion is supported by the matched-backbone comparisons in Figures 12 and 13.
- D More qualitative results: PhysisForcing consistently follows prompts while respecting basic physical regularities across diverse embodiments, scenes, and instructions.These results indicate that its learned physical priors generalize beyond the comparison prompts.
- E Broader impacts: PhysisForcing improves the physical plausibility of generated robotic manipulation videos, lowering barriers to embodied-AI research through cheaper simulation, data augmentation, and pre-deployment policy evaluation.It also provides more reliable training signals for downstream world models.
- E Broader impacts: More realistic robot videos could be misused to fabricate deceptive footage, overstate hardware capabilities, or seed policies trained on synthetic demonstrations.These risks qualify the benefits of improved physical plausibility.
F Limitations and future work
PhysisForcing inherits the capability ceiling of its video-generation backbones: limited world knowledge and long-horizon temporal reasoning constrain attainable physical plausibility. The authors expect it to compound with stronger video and world-model foundation models.
- Limitations: PhysisForcing inherits its backbone’s capability ceiling because current Wan2.2 and Cosmos3 models have limited world knowledge and long-horizon temporal reasoning.These limitations bound the physical plausibility achievable through fine-tuning alone.
- Future work: The authors expect PhysisForcing to compound with stronger video and world-model foundation models as they become available.