Source-linked AI summary
Agentic Physics-Grounded Video Generation via Memory-Augmented In-Context Reasoning
Xikang Guan, Shaocong Yang, Yanqi Bao, Can Wang, Zheng Gu, Zhengqing Chen, Kai Chen, Jing Huo, Yang Gao
TL;DR
Existing pre-synthesis physical guidance can constrain video dynamics, while some errors appear only after generation. PhysRepair diagnoses observed failures and uses causal constraints and prior repair experience to guide localized, adaptive repair; it improves Physics-IQ performance across five physics categories, with a reported overall score of 43.6.
Problem
Pre-synthesis physical guidance can constrain diverse dynamics, while physical errors may emerge only after video generation.
Source: §1 · p. 1Method
PhysRepair combines causal localization of affected states with relevant prior repair experience to guide iterative, localized correction of generated videos.
Source: §1 · p. 2, §3.3 · p. 6Results
PhysRepair scores 43.6 overall on Physics-IQ, 0.9 points above the strongest video-generation baseline, Cosmos3-Super-Image2Video.
Source: §4.2 · p. 8Takeaways & Limitations
On Physics-IQ Verified, PhysRepair improves physical plausibility across solid mechanics, fluid dynamics, optics, thermodynamics, and magnetism.
Source: §1 · pp. 2–3Performance depends partly on the repair agent’s physical reasoning and tool-use capabilities, and multiple repair rounds add inference cost.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Physical plausibility is essential for realistic video generation. Existing methods impose physical guidance before synthesis via simulations or refined prompts, but this can limit dynamic diversity, miss errors that emerge only after generation, and require repeated regeneration when inconsistencies remain. We therefore study a generate-diagnose-repair paradigm that preserves the original generation capability and selectively repairs physical inconsistencies afterward, raising two key questions: what to repair and how to repair? To address them, we introduce PhysRepair, an agentic video generation framework that combines causal reasoning over the observed physical failure with prior repair experience to enable localized and adaptive correction. Specifically, Spatiotemporal Causal Constraints identify the first invalid physical state and its causally affected states, producing a repair specification that guides physically plausible local regeneration while preserving unaffected content. Experience-augmented In-context Learning combines the ongoing repair context with relevant experience from previous repairs, allowing the agent to adapt its repair decisions across diverse physical failures and accumulate completed trajectories for future cases. Experiments on Physics-IQ Verified across solid mechanics, fluid dynamics, optics, thermodynamics, and magnetism demonstrate that PhysRepair consistently improves the physical plausibility of generated videos over existing methods. Our project page is available at https://xxkkxxx.github.io/PhysRepair-Website/.
1 INTRODUCTION
PhysRepair addresses physical failures that remain in generated videos by diagnosing them after synthesis and repairing only affected content. It combines causal localization with prior repair experience, and improves physical plausibility on Physics-IQ Verified across five categories.
1 INTRODUCTION
- Existing approaches add physical knowledge before synthesis, but extensive guidance can constrain diverse, complex dynamics and may miss errors that emerge after generation.The paper instead introduces physical guidance after observing inconsistencies, correcting affected interactions while leaving valid content unchanged.Source: §1 · p. 1
- The repair problem has two parts: identifying temporally and causally affected states, and choosing interventions suited to diverse failures without disrupting valid dynamics.Spatial masks alone cannot capture temporal changes, other objects’ trajectories, or propagation across objects, space, and time.Source: §1 · p. 2
- PhysRepair uses causal reasoning about observed failures and related prior repair processes to localize and adapt interventions without globally modifying valid dynamics.The agent iteratively diagnoses repaired results and refines subsequent interventions.Source: §1 · p. 2
- PhysRepair improves physical plausibility over existing methods on Physics-IQ Verified across solid mechanics, fluid dynamics, optics, thermodynamics, and magnetism.Ablations examine the effects of Spatiotemporal Causal Constraints and Experience-augmented In-context Learning.Source: §1 · pp. 2–3
- The framework’s two components address localization and adaptation: Spatiotemporal Causal Constraints specify affected states, while Experience-augmented In-context Learning retrieves prior repair experience.The latter adapts decisions across diverse physical failures.Source: §1 · pp. 2–3
2 RELATED WORK
Prior work evaluates physical plausibility and applies physical guidance before video synthesis, while agentic verification and accumulated experience offer precedents for localized, adaptive repair.
2.1 PHYSICS-AWARE VIDEO GENERATION AND PHYSICAL EVALUATION
- Pre-synthesis physical guidance can restrict dynamic diversity, miss post-generation errors, and require costly global regeneration; PhysRepair instead applies guidance after detecting specific inconsistencies.Benchmarks assess physical common sense, prediction of valid futures, fine-grained rule adherence, and conservation laws.Source: §2.1 · p. 3
2.2 AGENTIC VERIFICATION, REFINEMENT, AND EXPERIENCE
- Verification systems often use feedback to revise prompts for global regeneration, whereas Reflexion and Voyager reuse feedback and accumulated experience to guide later actions without updating model weights.PhysRepair combines current repair feedback with relevant prior experience in the agent’s context.Source: §2.2 · p. 3
3 METHOD
PhysRepair iteratively diagnoses failures in generated videos and makes localized counterfactual repairs. Causal constraints determine what to change, while retrieved repair experience adapts how to intervene and is distilled for future cases.
3.1 PHYSREPAIR: GENERATE–DIAGNOSE–REFINE FRAMEWORK
- Each iteration diagnoses the current video, specifies a localized repair, simulates a counterfactual state, regenerates the video, and rechecks it for residual violations.Residual or newly introduced violations initialize another repair iteration; successful repairs are accepted.Source: §3.1 · pp. 3–5
3.2 SPATIOTEMPORAL CAUSAL CONSTRAINTS
- Spatiotemporal Causal Constraints trace affected descendants from the first invalid state, editing only affected object–space–time states while preserving unrelated content.The repair specification also guides counterfactual simulation and suppression of contaminated source dynamics during video reconditioning.Source: §3.2 · p. 5
3.3 EXPERIENCE-AUGMENTED IN-CONTEXT LEARNING
- Experience-augmented in-context learning retrieves repair trajectories matched to the current failure and distills successful, distinct repair strategies for later cases.The working memory tracks evidence, hypotheses, repair history, and verification feedback; long-term memory stores successful repairs and reusable skills.Source: §3.3 · pp. 5–6
4 EXPERIMENTS
The experiments evaluate whether PhysRepair improves physical plausibility while preserving content outside the repair scope, using benchmark comparisons, qualitative examples, and analyses of its repair components. PhysRepair leads the reported Physics-IQ comparison, while user-study and ablation results further examine its outputs and repair strategies.
4.1 EXPERIMENTAL SETUP
- The evaluation uses 198 Physics-IQ Verified instances from 66 experiments across five physics categories, with reference continuations and evaluator masks withheld until evaluation.Methods receive the same switch frame and prompt, and failed samples remain in the evaluation.Source: §4.1 · pp. 6–7
4.2 MAIN RESULTS
- 43.6 overall Physics-IQ score places PhysRepair highest among reported methods, 0.9 points above Cosmos3-Super-Image2Video and 3.8 above MiniMax H3 FL2VA.Against MiniMax H3 FL2VA, ST rises from 22.8 to 34.7 (+11.9) and WS from 40.5 to 44.2 (+3.7), while SP and MSE remain stable.Source: §4.2 · p. 8
- In fan–garment and wrench–box interactions, PhysRepair preserves object identities and restores coherent force-response or fall–contact–rebound dynamics.The comparisons contrast PhysRepair with errors including weak force responses, incorrect motion direction, premature box movement, and interpenetration.Source: Figure 3 · p. 7, §4.2 · pp. 8–9
- PhysRepair achieves the best average rank in a user study where participants judged the physical consistency of videos from compared methods.The study's human judgments complement the quantitative evaluation.Source: §4.2 · p. 9, Figure 5 · p. 9
4.3 ABLATION AND ANALYSIS
- Without Spatiotemporal Causal Constraints, repairs can leave downstream effects unresolved or disrupt valid interactions; the full model corrects affected states while preserving valid ones.Examples include residual net or basin-water motion and disrupting a valid foot–ball kick.Source: Figure 4 · p. 8, §4.3 · pp. 9–10
- As long-term experience memory accumulates, moving-average repair rounds decrease, although difficult cases can still exhaust the 20-round budget.The analysis processes 100 initially generated, physically inconsistent cases sequentially; the reported efficiency gain does not require weight updates.Source: Figure 5 · p. 9, §4.3 · p. 10
5 LIMITATIONS AND BROADER IMPACT
PhysRepair depends on a capable multimodal language model for diagnosis, reasoning, tool use, verification, and experience reuse, and repair rounds add inference cost. The paper also notes potential applications and risks associated with increasingly realistic synthetic video.
Limitations
- Repair performance is partly bounded by the agent model's physical reasoning and tool-use capabilities, and weaker backbones may make repair decisions less reliable.Multiple repair rounds also add inference cost compared with one-pass video generation.Source: §5 · p. 10
Broader Impact
- PhysRepair may benefit simulation, embodied AI, and synthetic-data applications requiring more reliable physical dynamics, without retraining the underlying video generator.The paper also warns that realistic synthetic video may become harder to distinguish from real recordings, increasing deceptive-media risks.Source: §5 · p. 10
- The authors identify provenance mechanisms and responsible deployment practices as important complements to further improvements in physically grounded video generation.Source: §5 · p. 10
6 CONCLUSION
PhysRepair improves generated-video physical plausibility through iterative post-generation diagnosis and repair. It localizes repairs to causally affected states, adapts repair decisions using prior experience, and preserves unaffected content.
6 CONCLUSION
- PhysRepair consistently improves physical plausibility on Physics-IQ Verified across five physics categories.The categories are solid mechanics, fluid dynamics, optics, thermodynamics, and magnetism.Source: §6 · p. 10
- PhysRepair uses an iterative generate–diagnose–repair process to correct realized physical failures after generation.The framework is presented as an alternative to imposing all physical guidance before synthesis.Source: §6 · p. 10
- Spatiotemporal Causal Constraints localize repair to causally affected states and guide it with a physically admissible counterfactual evolution.The repair preserves unaffected content.Source: §6 · p. 10
- Experience-augmented In-context Learning adapts repair decisions using current-sample feedback and relevant experience from previous repairs.Source: §6 · p. 10