MindWorldBench: Evaluating Mental-State-to-Behavior Reasoning in Image-to-Video Generation

1 Introduction

• Existing evaluations focus on explicit action alignment or physical continuation, leaving mental reasoning difficult to distinguish from shortcut learning.The paper motivates evaluating behavior conditioned on latent mental states instead of physical instruction-following.Source: §1 · pp. 1–2
• MindWorldBench uses Zero-Action Prompting to test whether models infer behavior from internal states, organized across Perception, Belief, and Desire.The benchmark withholds action verbs so models must deduce intent rather than follow an explicit action directive.Source: §1 · p. 2, Figure 1 · p. 2
• Counterfactual prompt pairs vary a mental state while holding the image fixed, and an automated vision-model pipeline evaluates video quality, plausibility, intention, and world-state maintenance.A study of 100 generated videos found strong agreement between the automated evaluator and judgments aggregated from 10 human experts.Source: §1 · p. 2
• The evaluation covers 744 expert-designed prompts and eleven state-of-the-art image-to-video models, including Veo 3.1, Kling V3, Wan 2.6, and CogVideoX.The benchmark uses a counterfactual design to isolate effects of mental states.Source: §1 · p. 2
• Models maintain physical common sense but struggle with mental reasoning, often generating actions from objective truth rather than a character’s subjective belief.The authors call this failure mode Omniscient Bias.Source: §1 · p. 2

Video Generation Models and World Simulators

• Video models are increasingly viewed as nascent world simulators because they generate high-fidelity, temporally coherent videos and physical dynamics, but modeling why actions begin remains a gap.The related-work discussion distinguishes simulating physical consequences from modeling cognitive intent.Source: §2 · p. 2

Evaluation Benchmarks for Video Generation

• Video-generation benchmarks mainly assess perceptual quality, semantic alignment, or execution of explicit actions, which can encourage mapping textual verbs to visual priors.The cited evaluation landscape includes VBench, EvalCrafter, UI2V-Bench, and FETV.Source: §2 · p. 3

Evaluation Dimensions

• Prior visual Theory-of-Mind work focuses on understanding, while MindWorldBench evaluates the unexplored inverse task of synthesizing behavior from latent mental states.The paper extends the BDI framework into a vision-oriented benchmark for video generation.Source: §2 · p. 3

3.1 Task Definition

• Given an initial world state, belief and desire determine intention, which leads to action; the benchmark maps the image and mental constraints to a video without supplying the action.The model must implicitly infer the latent chain from mental state to intention to action.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2 Mental-Space Taxonomy

• The BDI-grounded taxonomy organizes Perception, Belief, and Desire into 3 primary dimensions, 12 secondary aspects, and 34 fine-grained sub-dimensions.Perception concerns information uptake and attention, Belief tests behavior based on a potentially flawed or outdated understanding, and Desire covers goals and preferences.Source: §3.2 · p. 3

3.3 Dataset Construction

• Experts design and review cognitive graphs, then curate 744 cases over 372 images across the three dimensions, with counterfactual pairs varying mental conditions while holding images fixed.Prompts withhold the ground-truth intention; the dataset spans 12 secondary categories and 34 fine-grained sub-dimensions.Source: §3.3 · pp. 3–4, Figure 3 · p. 4

3.4 Evaluation Protocol

• The evaluation scores visual quality and commonsense plausibility on 1–5 scales, classifies candidate-action occurrence, and reports intention accuracy and world-state maintenance as percentages.On 100 generated videos, the LMM’s agreement with 10 human experts was 92% for Intention Accuracy and 89% for World-State Maintenance.Source: §3.4 · p. 4

4.1 Experimental Setup

• The evaluation covers eleven I2V systems using 744 expert-designed mental-state prompts for 720p resolution and 5–8 second videos.The models span closed-source commercial APIs and open-source foundation models.Source: §4.1 · p. 4

4.2 Overall Benchmark Results

• Visual Quality (3.8–4.4) and Commonsense Plausibility (3.8–4.6) cluster above Intention Accuracy (11.9%–59.5%) and World-State Maintenance (32.5%–68.3%).The reported ranges indicate that current models generate visually plausible videos more reliably than they infer mental-state-conditioned behavior while preserving objective world state.Source: §4.2 · pp. 4–5, Table 1 · p. 5
• Veo3.1 leads Intention Accuracy across Belief, Perception, and Desire at 58.3%, 58.6%, and 61.6%, respectively.Kling is the closest competitor on Belief and Desire, while Wan2.6 is closest on Perception.Source: Figure 4 · p. 5, §4.2 · p. 5

4.3 Diagnostic Analysis of Mental-State Reasoning

• Veo3.1, Wan2.6, and Kling V3 achieve Both Correct rates of 54.2%, 51.1%, and 50.0%, respectively, while LTX2.3 reaches 8.3%.Pair-level success requires the intended behavior in both counterfactual conditions; LTX2.3 also has the highest Neither Correct rate, 47.9%.Source: §4.3 · p. 5, Figure 5 · p. 6
• Kling has the highest joint action–state success at 46.1%, while State Correct Only exceeds Action Correct Only for most models.For example, Veo3.1 scores 45.6% versus 8.9%, and Wan2.6 scores 35.9% versus 15.2%, respectively.Source: Figure 5 · p. 6, §4.3 · p. 6
• Overall, current I2V systems remain unreliable at translating latent mental states into correct actions while maintaining objective world state.The diagnostics indicate that action reasoning and world-state maintenance are related but nonequivalent capabilities.Source: §4.3 · p. 6

• Visual quality and commonsense plausibility remain relatively strong across eleven models, while intention accuracy and world-state maintenance are substantially weaker.Source: §5 · p. 6
• Diagnostic analyses reveal failures in realizing task-relevant actions and tracking counterfactual mental states.Source: §5 · p. 6
• Models also struggle to jointly preserve action correctness and objective world states, highlighting the need for mental-state modeling in future video-generation systems.Source: §5 · p. 6