Source-linked AI summary
PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop
Xinge Peng, Yiting Lu, Tianwu Zhi, Wen Wen, Jianzhao Liu, Xin Li, Zhibo Chen
TL;DR
Physical understanding in VLMs remains difficult to evaluate holistically, particularly across both real-world and AI-generated videos. PhysVista benchmarks a perception–reasoning–assessment loop, and experiments show stronger performance on perception than on reasoning and plausibility assessment. The results indicate a gap between semantic recognition and complete physical intelligence.
Problem
Existing benchmarks fragment evaluation across cognitive stages, while the extent of VLMs’ physical understanding across real-world and generated videos remains unclear.
Source: §1 · pp. 2–3Method
PhysVista evaluates physical state perception, physical dynamics reasoning, and plausibility assessment across real-world and AI-generated videos, distinguishing event- and scale-level reasoning.
Source: §1 · p. 2, §3.1 · pp. 3–4, §3.2.1 · p. 4Results
VLMs perform relatively better on perception than reasoning and assessment, with physical dynamics reasoning the most challenging stage overall.
Source: §4.1 · pp. 7–8Takeaways & Limitations
Experiments indicate that strong semantic perception does not amount to complete physical intelligence across the cognitive loop.
Source: §5 · p. 10
Abstract
from arXiv · showhide
Vision-Language Models (VLMs) have shown strong multimodal reasoning capabilities, yet whether they truly capture the physical consistency underlying real-world dynamics remains unclear. Existing benchmark paradigms often suffer from fragmented evaluation, focusing on isolated cognitive stages while overlooking the inherent synergy between perception, reasoning, and physical judgment. The lack of a holistic perspective limits the ability to diagnose whether VLMs can reliably evaluate the physical authenticity of emerging generative models. To address these issues, we introduce PhysVista, a benchmark designed to evaluate physical intelligence in VLMs through a closed cognitive loop framework inspired by the human seeing-reasoning-assessment process. PhysVista restores this loop by jointly evaluating physical state perception, physical dynamics reasoning, and physical plausibility assessment. It further distinguishes event-level reasoning and scale-level reasoning to enable fine-grained analysis of physical understanding. In addition, PhysVista incorporates both real-world and AI-generated videos, allowing evaluation across diverse domains and emerging generative scenarios. Extensive experiments across a diverse set of VLMs reveal substantial limitations in physical reasoning and plausibility assessment, highlighting a persistent gap between visual recognition and genuine physical understanding, and pointing toward more principled designs for physically grounded multimodal intelligence.
1 Introduction
Existing benchmarks assess physical understanding in fragmented ways and leave important evaluation gaps, especially for generated videos. PhysVista proposes a unified benchmark spanning perception, reasoning, and plausibility assessment across real-world and AI-generated videos.
1 Introduction
- Existing benchmarks cover isolated stages of physical cognition, leaving systematic evaluation of the perception–reasoning–assessment loop incomplete.Prior work focuses on perception or limited reasoning tasks; PAI-Bench includes perception and reasoning but not the final assessment component.Source: §1 · p. 2
- Language shortcuts, binary plausibility judgments, and structured problem-solving settings limit how reliably existing evaluations test visually grounded physical understanding.The cited benchmarks can permit video-blind answers, emphasize shallow physical intuition, or treat visual information as auxiliary.Source: §1 · p. 2
- Evaluation remains limited by simulation-heavy data and underexplored AI-generated videos, whose physical understanding gap relative to real-world videos is unclear.Generated videos can contain implausibilities involving scene changes, object interactions, and human motion.Source: §1 · p. 2
- PhysVista combines perception, temporal and causal reasoning, and plausibility assessment across real-world and AI-generated videos, with event- and scale-level reasoning.Its framework is intended to support holistic diagnosis beyond isolated capability testing.Source: §1 · pp. 2–3
2 Related Work
Prior work on physical understanding spans generative-model evaluation and VLM benchmarks, but often uses isolated measures or limited settings. This section describes those approaches and the gaps they leave.
2.1 Physics-related Benchmarks for Generative Models.
- Generative-model benchmarks assess physical dynamics, causality, or closed-loop behavior, but often reduce evaluation to scalar outputs without continuous state supervision.The cited discussion identifies long-term error propagation as obscured when evaluation relies solely on final generations.Source: §2.1 · p. 3
2.2 Physics-related Benchmarks for VLMs.
- VLM benchmarks separately target intuitive physics, perception, causal reasoning, or applications, while largely neglecting AI-generated content and holistic physical understanding.The approaches include simulation-based tests, minimal video pairs, real-world causal reasoning, structured problem-solving, and embodied-AI evaluations.Source: §2.2 · p. 3
3 PhysVista
PhysVista organizes physical intelligence evaluation as a loop from state perception through causal reasoning to plausibility assessment. It covers real-world and AI-generated videos and distinguishes event-level from scale-level reasoning.
3.1 Overview
- PhysVista evaluates reconstruction of observable states, causal mechanisms, and global plausibility with scores and ranks across real-world and AI-generated videos.The benchmark includes diverse interactions, motion dynamics, and degrees of physical consistency.Source: §3.1 · pp. 3–4
3.2.1 Task Information.
- Perception tasks cover spatial and temporal states, camera motion, quantitative object-scale ratios, and awareness of uncertain physical parameters or outcomes.Scale estimation uses explicit fractional ratios, while uncertainty tasks distinguish latent-parameter uncertainty from indeterminate future outcomes.Source: §3.2.1 · p. 4
3.2.2 Data Collection.
- Perception data use WISA-80K and VideoPhy-2, with Multi-CamVideo for camera motion; task-specific templates and verification aim to ensure quality and reduce language bias.Camera-motion clips are selected for a single dominant motion, and annotators rewrite samples with strong stylistic patterns.Source: §3.2.2 · p. 5
3.3.1 Task Information.
- Physical dynamics reasoning combines shuffled-frame temporal order reconstruction with five causal tasks that test mechanisms, violations, predictions, counterfactuals, and critiques.The tasks require inferring physical dependencies beyond directly observable appearances and include event- and scale-level distinctions.Source: §3.3.1 · pp. 5–6
3.3.2 Data Collection.
- Reasoning videos are curated for rich motion and unambiguous task conditions, and content-agnostic templates are used except for counterfactual questions.Prediction inputs withhold the final outcome; counterfactual conditions are introduced implicitly through modified scenarios.Source: §3.3.2 · p. 6
- Answer options are designed to be plausible, video-aligned, and discriminative; hierarchical variants test event- and scale-level reasoning, followed by human verification.Task-specific construction also uses annotated violations, withheld outcomes, and minimally edited plausible scenarios as answer sources or distractors.Source: §3.3.2 · pp. 6–7
3.4.1 Task Information.
- Assessment requires calibrated 1-to-5 plausibility scores for single videos and relative plausibility rankings between video pairs.The comparison task judges overall coherence and realism, rather than relying on a score-scale evaluation.Source: §3.4.1 · p. 7
3.4.2 Data Collection.
- Assessment uses generated videos sampled across five plausibility-score levels and real-world videos assigned score 5, with comparisons spanning three pair types.Generated–generated pairs control semantic similarity and mostly differ by no more than two plausibility levels.Source: §3.4.2 · p. 7
4 Experiments
PhysVista evaluates physical understanding through perception, reasoning, and assessment, with experiments diagnosing performance across these stages and across real-world and generated videos. The results reveal uneven capabilities, reasoning and fine-grained plausibility judgments remain challenging, and performance often weakens on generated content.
4 Experiments
- The experimental protocol organizes capability diagnosis into physical state perception, physical causal reasoning, and physical plausibility assessment.These stages are framed as seeing, reasoning, and assessment, respectively.Source: §4 · p. 7
4.1 Overall Benchmark Results
- VLMs perform relatively better on perception, especially spatial state perception, than on dynamics reasoning and plausibility assessment, while fine-grained perception remains limited.The reported weak areas include quantitative scale estimation, physical uncertainty awareness, dynamics prediction, counterfactual reasoning, plausibility scoring, and plausibility comparison.Source: §4.1 · pp. 7–8, Table 2 · p. 9, Table 3 · p. 9
4.2 Cognitive Gap Analysis
- Models show within-model ranking inconsistencies across perception, causal reasoning, and plausibility assessment, so strength at one stage does not reliably carry to the next.GPT-5.2 and Kimi K2.5 drop in relative rank from state perception to dynamics reasoning; Gemini 3.1 Pro and Doubao Seed 2.0 Pro drop from dynamics reasoning to plausibility assessment.Source: §4.2 · p. 8
4.3 Subtask Difficulty and Failure Modes in Reasoning
- Temporal order reconstruction is generally easier than causal tasks, while prediction and counterfactual reasoning remain difficult because they require temporal analysis and fine-grained physical inference.Counterfactual reasoning additionally requires hypothetical interventions and implicit simulation of resulting dynamics, unlike prediction, which can often use temporal extrapolation and causal reasoning about observed motion.Source: §4.3 · pp. 8–9, Table 3 · p. 9
4.4 Real-world vs. Generated Content Analysis
- Models generally perform better on real-world than generated videos, with larger gaps for physical uncertainty awareness and dynamics prediction than for basic spatial perception and temporal reasoning.Quantitative scale estimation is an exception, with slightly higher performance on generated videos; the authors suggest cleaner geometry or more controlled object configurations may help.Source: Figure 3 · p. 8, §4.4 · pp. 9–10
4.5 Assessment Task Analysis
- Plausibility scoring can collapse toward a default score, while pairwise comparison is easiest for real-versus-generated videos and harder for similar generated pairs or two valid real-world videos.The authors associate competent scoring with distinct score distributions for real-world and generated videos; low accuracy on real-versus-real pairs indicates difficulty recognizing equivalence.Source: §4.5 · p. 10
5 Conclusion
PhysVista evaluates VLM physical intelligence through a closed loop spanning state perception, dynamics reasoning, and plausibility assessment. Experiments find that physical intelligence remains incomplete across this loop despite strong semantic perception.
5 Conclusion
- PhysVista unifies evaluation of physical state perception, physical dynamics reasoning, and physical plausibility assessment across real-world and AI-generated videos.The authors state that experiments find modern VLMs’ physical intelligence remains incomplete across the full cognitive loop despite strong semantic perception.Source: §5 · p. 10
A.1 Definition of Camera Motions
The appendix points readers to Table 4 for the definitions of camera motions.
A.1 Definition of Camera Motions
- The definitions of camera motions are provided in Table 4.The camera-motion task uses ten motion types, whose definitions are illustrated in that table.Source: §A.1 · p. 14, Table 4 · p. 14
A.2 Benchmark Statistics
PhysVista’s statistics describe video duration, physical principles, scene captions, and task sample counts. Tables 5 and 6 report the number of samples across the benchmark’s task groups.
A.2 Benchmark Statistics
- Figure 4a presents video-duration statistics, while Figure 4b and Figure 4c show word clouds of physical principles and video captions, respectively.The caption word cloud reflects content present in the scenes.Source: §A.2 · p. 14
- Task sample counts are reported for Physical State Perception in Table 5 and for Physical Dynamics Reasoning and Physical Plausibility Assessment in Table 6.The supplied passages identify the tables’ scope but do not give their cell values.Source: Table 5 · p. 14, Table 6 · p. 14
A.3 Human Annotation, Verification, and Evaluation
Six human annotators handled data annotation and verification, while five independent evaluators rated 40% of the benchmark on three quality dimensions. The evaluation results are shown in Figures 5–7.
A.3 Human Annotation, Verification, and Evaluation
- Six human annotators were involved in data annotation and verification.Source: §A.3 · p. 14
- Five independent evaluators rated 40% of the benchmark dataset on a 1-to-5 scale for linguistic neutrality, question-answer precision, and physical reasoning validity.Source: §A.3 · p. 14
- Figures 5, 6, and 7 present the results for linguistic neutrality, question-answer precision, and physical reasoning validity, respectively.Source: §A.3 · p. 14, Figure 5 · p. 15, Figure 6 · p. 15
A.4 More Results
Most models perform relatively poorly on Physical Principle Violation Reasoning, which requires mapping observed dynamics to physical-principle constraints and may involve multiple simultaneous violations.
A.4 More Results
- Most models achieve relatively low performance on Physical Principle Violation Reasoning, suggesting unreliable principle-grounded abstractions even when violations are visually salient.The task reports exact-match accuracy, option-level Micro-F1, and principle-type Macro-F1 in Table 7.Source: §A.4 · p. 16, Table 7 · p. 16
- Physical Principle Violation Reasoning maps observed dynamics to principle-level constraints and often involves multiple simultaneous violations.Source: §A.4 · p. 16
- The benchmark includes Spatial State Perception tasks, illustrated in Figure 8.Source: Figure 8 · p. 17
- Figure 9 illustrates the Temporal Violation Localization task.Source: Figure 9 · p. 17
A.5 Task Illustrations
PhysVista’s task illustrations cover physical state perception, dynamics reasoning, and plausibility assessment. The examples include camera motion, uncertainty, temporal order, physical principles, dynamics prediction, counterfactual reasoning, and plausibility judgments.
A.5 Task Illustrations
- The illustrated task set spans Physical State Perception, Physical Dynamics Reasoning, and Physical Plausibility Assessment.Source: §A.5 · p. 17
- The examples include Camera Motion Recognition and Quantitative Scale Estimation, and Physical Uncertainty Awareness.Source: Figure 10 · p. 17, Figure 11 · p. 18
- Temporal Order Reconstruction and Physical Violation Critique are illustrated alongside Physical Mechanism Reasoning and Physical Principle Violation Reasoning.Source: Figure 12 · p. 18, Figure 13 · p. 18
- The illustrations also include Physical Dynamics Prediction, Counterfactual Physical Reasoning, Physical Plausibility Scoring, and Physical Plausibility Comparison.Source: Figure 14 · p. 19, Figure 15 · p. 19, Figure 16 · p. 19