WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation

• Existing benchmarks cover only selected competencies, limiting fair comparison and failure diagnosis across visual quality, controllability, memory, and physics.Source: §1 · p. 4
• WBENCH evaluates interactive world models across video quality, setting adherence, interaction adherence, consistency, and physics compliance, addressing fragmented prior protocols.Source: §1 · p. 4
• Each test case separates a world setting from a multi-turn interaction sequence spanning diverse scenes, styles, subjects, perspectives, and navigation, subject action, event editing, and perspective switching.Source: §1 · p. 4
• Navigation is represented as text, camera pose, and discrete action, enabling native-interface evaluation through a shared 158-case subset and a 289-case full benchmark.Source: §1 · p. 4
• No model dominates all five dimensions, while navigation, camera control, perspective consistency, physical correctness, and turn-wise degradation exhibit distinct relationships.Source: §1 · p. 4

• Video-generation suites emphasize perceptual quality, text alignment, or physical commonsense without action inputs or multi-turn interaction evaluation.Source: §2 · pp. 4–5
• World-model benchmarks variously target navigation, memory, causal interaction, camera trajectories, embodied agents, or autonomous driving, leaving complementary capabilities fragmented.Source: §2 · pp. 4–5
• Existing benchmarks do not jointly cover open-domain scenes, both perspectives, four interaction types, multi-turn consistency, and physics compliance.Source: §2 · p. 5, Table 1 · p. 5

3 WBENCH Dataset

• Each case decomposes evaluation into a world setting defining the initial state and an interaction sequence specifying control signals across consecutive turns.Source: §3 · p. 5, §3.1 · p. 5

3.1 Dataset Construction

• World settings combine scene, style, perspective, and subject attributes into environment and subject prompts alongside an initial frame.Source: §3.1 · pp. 5–6
• Four composable interaction types cover navigation, subject action, event editing, and perspective switching, with navigation supporting perspective-dependent camera or subject motion.Source: §3.1 · p. 6
• Case construction follows a setting-first principle, enforcing physical executability, semantic coherence, causal ordering, stratified coverage, and manual review.Source: §3.1 · p. 6

3.2 Dataset Statistics

• WBENCH contains 289 cases and 1,058 turns, with first-person cases comprising 62% and third-person cases 38%.Source: §3.2 · p. 6
• Scenes span six categories, explicit subjects are dominated by humans at 64%, and photorealistic rendering accounts for 52% of cases.Source: §3.2 · p. 6
• Cases span 2–9 turns averaging 3.7, with longer sequences typically interleaving subject action and event editing to probe temporal consistency and long-horizon coherence.Source: §3.2 · p. 7

4 WBENCH Evaluation Suite

• The evaluation suite contains 22 fine-grained sub-metrics across five dimensions, with scores rescaled to [0, 100] for comparability.Source: §4 · p. 7
• Video quality measures aesthetics, imaging, flickering, dynamic degree, motion smoothness, and human preference, while setting adherence checks visible, offscreen, appearance, and motion requirements.Source: §4 · p. 7
• Interaction adherence evaluates navigation with pose-based trajectory accuracy and repeated-action consistency, while other interaction types use structured turn-level or categorical VLM checks.Source: §4 · p. 7
• Event editing and subject action receive [0, 5] turn grades from five binary checks, whereas perspective switching succeeds only when transition visibility, target consistency, and viewpoint structure all hold.Source: §4 · p. 7
• Consistency metrics assess spatial, segment, perspective, geometric, photometric, subject, and background stability using poses, perceptual similarity, segmentation, depth, and feature measures.Source: §4 · pp. 7–8, §C.8 · p. 8

C.8 Background Consistency.

• Physical evaluation combines holistic causal-fidelity grading across rendering and causal violations with context-conditioned scores over seven physics sub-dimensions.Source: §C.8 · p. 8

P.2 Visual Plausibility.

• A fine-tuned Qwen3-VL-30B-A3B model scores low-level artifacts such as distortion, penetration, and unnatural deformation to complement causal-fidelity assessment.Source: §P.2 · p. 8

5.1 Evaluated Models and Protocol

• WBENCH evaluates 20 models across paradigms using a 158-case navigation split and full 289-case text-driven benchmark, with native text, pose, or action controls aligned for comparison.Semantic interactions are restricted to text-driven models, while camera-controlled and action-conditioned models are evaluated on navigation.Source: §5.1 · p. 8, Table 2 · p. 9

5.2 Per-Dimension Results

• No model or paradigm dominates uniformly: text-driven models lead setting adherence, native-control models lead navigation, and consistency and physical performance show different leaders.Video quality is near saturation, while perspective switching remains especially difficult.Source: §5.2 · pp. 8–9

5.3 Cross-Dimension Analysis

• Navigation is largely independent of other capabilities, showing near-zero correlation with video quality, consistency, and physical compliance, while physical scores correlate with video quality and consistency.Among text-driven models, semantic interactions align more with setting adherence than navigation, indicating different underlying capabilities.Source: §5.3 · p. 10, Figure 3 · p. 10
• Camera control does not guarantee perspective consistency: top navigation models can rank low on perspective consistency, especially in third-person cases.The result separates accurate camera movement from coherent subject motion.Source: §5.3 · p. 10, Figure 3 · p. 10
• Benchmark difficulty varies systematically by perspective, scene type, and subject category, with first-person, workspace, and robot settings easier than third-person, sports/game, and animal settings.The reported deviations are z=+1.0 for first-person navigation, z=−1.9 for sports/game scenes and animal subjects, and z=+1.6 for workspace scenes.Source: §5.3 · p. 10, Figure 3 · p. 10
• Navigation degrades by 33 points from turn 1 to turn 4+, versus 13 points for event editing and 9 for subject action, while perspective switching changes by only +2.The paper attributes navigation degradation to compounded pose errors and describes dedicated geometric control as more robust for preserving spatial state.Source: §5.3 · pp. 10–11, Figure 4 · p. 10

5.4 Human Preference Alignment

• Automated WBENCH scores align strongly with human preferences: all ten aspects reach Spearman ρ≥0.94, and four reach ρ=1.00.The alignment study uses blind pairwise comparisons from 400 crowdsourced annotators.Source: §5.4 · p. 11, Figure 5 · p. 11

6 Conclusion

• WBENCH concludes that current models have not unified high-fidelity rendering with reliable controllability, consistency, and physics; its test set uses discrete rather than continuous action sequences and partly relies on LMM-based physical evaluation.The authors identify additional domains and real-time evaluation as extensions.Source: §6 · p. 11

• Table 3 compares WBENCH with surveyed benchmarks across perspectives, interaction types, and five evaluation dimensions.Source: Table 3 · p. 19
• Existing benchmark families span perceptual quality, physical reasoning, next-scene generation, instruction following, and interactive control, but use different scopes and primary metrics.Source: §A.1 · p. 19

A.2.1 Scene and Style Gallery

• WBENCH covers six scene categories and seven rendering styles, pairing photorealistic and stylized versions of shared scene specifications.Source: §A.2.1 · p. 21, Figure 7 · p. 21, Figure 8 · p. 22

A.2.2 Perspective and Subject Gallery

• The gallery distinguishes disembodied first-person, embodied first-person, and third-person views, with subjects spanning humans, animals, vehicles, robots, and other objects.Source: Figure 9 · p. 22, §A.2.2 · p. 23, Figure 10 · p. 23

A.2.3 Subject Action and Event Editing Examples

• Subject action and event editing cover eleven temporal sub-types ranging from local manipulation and locomotion to global environmental changes and irreversible physical effects.Source: §A.2.3 · p. 23, Figure 11 · p. 24, Table 4 · p. 24

A.2.4 Perspective Switching Prompts Gallery

• Perspective switching includes same-subject, multi-subject, and scope-mode transitions that re-anchor viewpoints while preserving subject or scene consistency.Source: §A.2.4 · pp. 24–25, Figure 11 · p. 24

A.3 Navigation Design and Distribution

• Navigation uses perspective-dependent WASD and arrow-key semantics, six trajectory types, balanced directional coverage, and text, 6DoF-pose, or discrete-action interfaces.Source: §A.3 · p. 25, Table 5 · p. 25, Table 6 · p. 26

B.1.1 Text-driven Models

• Text-driven models use iterative image-to-video continuation, whereas camera-controlled and action-conditioned models condition generation on pose sequences or action histories.Source: §B.1.1 · p. 27, §B.1.2 · p. 28, §B.1.3 · p. 29

B.1 Per-Model Configuration

• The evaluated models comprise text-driven, camera-controlled, and action-conditioned paradigms, with distinct native controls and different benchmark scopes.Source: §B.1.1 · p. 27, §B.1.2 · p. 28, §B.1.3 · p. 29

B.1.4 Inference Speed Analysis

• Inference analysis reports wall-clock generation time relative to produced-video duration, with lower values indicating faster generation and values below 1 indicating faster-than-real-time output.Source: Table 7 · p. 27, §B.1.4 · p. 30

• The browser-use pipeline fills prompts and images, waits for world loading, executes scripted interactions, and downloads recorded videos.Source: §B.2 · p. 30, Figure 14 · p. 30
• Navigation keys are held for 5 seconds per turn before the generated video is evaluated offline.Source: §B.2 · p. 30, Figure 14 · p. 30

C.1 Metric Quick Reference

• WBENCH’s metric reference organizes 22 sub-metrics and identifies whether each uses an expert model or VLM judge.Source: §C.1 · p. 31, Table 8 · p. 31

C.2 Video Quality

• Video-quality metrics score aesthetic and imaging quality, flickering, dynamic degree, motion smoothness, and normalized HPSv3 rewards.Source: §C.2 · p. 31, §C.2.1 · p. 31, §C.2.2 · p. 31

C.3 Setting Adherence

• Setting adherence decomposes world descriptions into visible, offscreen, appearance, and action components before evaluating scene and subject fidelity.Source: §C.3 · p. 32, §C.3.1 · p. 32, §C.3.2 · p. 33

C.3.1 Scene Adherence

• Scene adherence combines maintenance of visible environment and style with binary appearance of offscreen content after camera movement.Source: §C.3.1 · pp. 32–33

C.3.2 Subject Adherence

• Subject adherence combines a 1–5 appearance-maintenance score with a binary match for the described movement or action pattern.Source: §C.3.2 · p. 33

C.4 Interaction Adherence

• Interaction adherence evaluates navigation trajectories plus event editing, subject action, and perspective switching, with turn-level checks for changes, occurrence, completion, details, and anomalies.Source: §C.4 · p. 33

C.4.1 NavScore

• NavScore combines navigation accuracy and cross-turn consistency after comparing aligned, arc-length-resampled predicted and ground-truth trajectories.Model-level scores average navigation cases and are rescaled to [0, 100].Source: §C.4.1 · pp. 33–34
• Adaptive ground truth matches each model’s predicted motion magnitude, while wrong-direction motion remains a trajectory error.Happy Oyster and HY-World 1.5 navigate correctly at different amplitudes, whereas HY-Video 1.5 reverses rotations and is penalized.Source: §C.4.1 · pp. 34–35, Figure 15 · p. 35, Figure 16 · p. 36

C.4.2 Event Editing Adherence

• Event-editing adherence scores five progressive binary checks per turn, rewarding instructed state transitions and penalizing untouched scenes or unrelated entities.The raw score ranges from 0 to 5 and is rescaled to [0, 100].Source: §C.4.2 · pp. 36–37

C.4.3 Subject Action Adherence

• Subject-action adherence uses the same five-question framework to assess action occurrence, completion, detail accuracy, and unnatural motion.Successful outputs initiate and complete the prescribed action, while failures remain idle or incomplete.Source: §C.4.3 · pp. 37–38

C.5.3 Spatial Consistency

• Spatial consistency compares the initial and return views while a motion gate suppresses high scores from near-static videos.A genuinely revisited scene scores higher than a near-static return with similar endpoint appearance.Source: §C.5.3 · p. 40

C.5.6 Reconstruction Consistency

• Reconstruction consistency combines geometric reprojection displacement with photometric PSNR to assess structural and appearance stability.Both signals use the same depth-based reprojection pipeline.Source: §C.5.6 · p. 41

C.6.1 Causal Fidelity

• Causal fidelity averages global rendering physics with scene-selected physics dimensions when applicable, otherwise using the global score alone.Track 1 and Track 2 use 0–3 ratings, and the resulting case score is normalized to [0, 1].Source: §C.6.1 · pp. 42, 44
• Track 2 selects relevant phenomena from seven dimensions once per case and shares those selections across models for comparable scoring.The dimensions cover fluid, collision, surface tracks, deformation, wind, reflection, and human motion.Source: §C.6.1 · pp. 42–44
• Physics profiles vary substantially by dimension: Reflection & Lighting is nearly saturated, whereas Deformation & Destruction varies more despite activation in only 4 of 50 cases.Aggregate scores can therefore conceal distinct failure axes and trade-offs among similarly ranked models.Source: §C.6.1 · p. 46, Figure 22 · p. 46, Table 9 · p. 47

C.6.2 Visual Plausibility

• Visual plausibility uses a fine-tuned Qwen3-VL-30B-A3B model to predict continuous scores from five ordered human-rating categories.The scorer achieves PLCC 0.92 against ground-truth labels.Source: §C.6.2 · pp. 46, 48

D.1 Full Split Results on Text-Driven Models

• No single text-driven model dominates all evaluation axes; Kling 3.0 and Wan 2.7 lead interaction adherence, while LongCat-Video and HunyuanVideo lead consistency.Perspective switching remains difficult for all models, averaging 30.7, and visual plausibility varies within only 4.6 points.Source: §D.1 · p. 48
• Wan 2.7 reaches 83.3 in causal fidelity versus a 74.7 average, while its dynamic-motion advantage contrasts with consistency-focused models.The results indicate a trade-off between motion magnitude and temporal stability.Source: §D.1 · p. 48

D.2 Human-Preference Annotation Platform and Protocol

• Each human judgment is a blind pairwise comparison with randomized video order, triple redundancy, and majority-vote aggregation.Annotators receive dimension-specific focus and ignore instructions; practice tasks, gold checks, completion-time filtering, and expert review provide additional quality control.Source: Figure 23 · p. 50, §D.2 · pp. 50–51
• Human evaluation covers nine dimension-level sheets spanning 13,515 pairwise comparisons across eight models and ten evaluation dimensions.The study uses 400 crowdsource annotators, and its per-dimension alignment with automatic metrics is reported in Fig. 5.Source: §D.2 · pp. 50–51

E Broader Impact

• WBENCH diagnoses interactive world-model capabilities without generating or distributing synthetic media, releasing evaluation tools and metadata rather than model weights.Its test data use synthetic or openly licensed imagery without identifiable real individuals, while diverse visual styles reduce emphasis on photorealistic identity synthesis.Source: E Broader Impact · p. 51