PaperlayerView exact revision on arXiv

Source-linked AI summary

Mental World Modeling

Hao Fei, Yiran Zhao

arXiv:2607.27201v1cs.CL

TL;DR

Existing world models focus on physical scenes, but predicting human decisions also requires modeling what agents believe, want, intend, feel, and consider permissible. This paper formulates Mental World Modeling and implements MENTIS, finding that explicit mental-state modeling is necessary for predicting human decisions across eight LLM-based world models.

  • Problem

    Existing world models emphasize physical scenes, while decision quality can depend on beliefs, goals, attention, trust, affect, obligations, or norms unavailable from the physical scene alone.

  • Method

    Mental World Modeling couples physical and mental states, renders target-specific observations, and simulates candidate actions’ effects on both through the inspectable MENTIS baseline.

  • Results

    Explicit mental modeling was necessary across all 8 tested LLM-based world models, with full MWM performing best and removing the mental channel degrading every model.

  • Takeaways & Limitations

    MWM is most relevant when hidden mental variables determine which action a person will perceive, choose, accept, resist, or learn from.

  • Takeaways & Limitations

    MWM is not a replacement for domain expertise, clinical judgment, or professional responsibility in care and assistive settings.

Abstract

from arXiv · show

World models enable a predictive substrate for planning and action, yet existing formulations merely answer a physical question: what/where it is, and how will it evolve. Human behavior, however, is driven by hidden mental state (what a person believes, wants, intends, feels, and considers socially permissible), so a model that tracks the physical scene but not what each agent knows and believes about it predicts the wrong action for the right-looking scene. We formulate Mental World Modeling (MWM), a generic theoretical framework that makes mental variables core components of a world model rather than posthoc rationales: MWM aintains a coupled physical-mental world state, renders a target-specific partial observation, and simulates how candidate actions jointly update both components. We instantiate the framework in MENTIS, a training-free and fully inspectable baseline that decomposes the process into state parsing, target-observation generation, action decomposition, coupled physical and mental transition, and branch-level value evaluation. On a manually constructed, quality-controlled dataset of situated decision scenarios spanning text, image, and sounding-video stories, experiments with 8 modern LLM-based world models demonstrate that explicitly modeling the mental state is essential for predicting human decisions. Deeper analyses further expose the bottlenecks of current mental world modeling. We expect MWM as a next stage of world modeling, from simulating physical scenes to simulating the minds that act in them.

1. Introduction

World models support agents in predicting and intervening on evolving worlds, but physical-state modeling alone is insufficient for human-centered intelligence because behavior depends on internal mental-social configurations. Effective agents must therefore account for what is true, how actions affect themselves, and how beliefs, emotions, risks, trust, norms, roles, and relationships shape behavior.

  • Problem: Physical scene tracking can predict the wrong action because human behavior depends on agents’ beliefs about the scene, not only the scene itself.The mug example contrasts object-only prediction with belief-aware prediction when the mug is moved while the person looks away.
  • Motivation: World models give agents a substrate for representing current truths, forecasting action consequences, and anticipating world changes before acting.This motivation follows the need for agents to predict and intervene on evolving worlds rather than merely generate plans or use tools.
  • Motivation: LLMs can write programs, use tools, compose plans, and support autonomous agents, but these capabilities alone do not make them agents that live in, predict, and intervene on evolving worlds.The passage distinguishes useful cognitive capabilities from the broader requirements of world-aware agency.
  • Related work: World models span representation-based latent-state prediction, control-oriented transition learning, and video-generative approaches.The introduction identifies representation world models, including recurrent models, Dreamer, and JEPA, as one relevant technological family before introducing video-generative models.
  • Problem: Human-centered agents must model internal mental-social configurations alongside external environments to interpret behavior in situated interactions.Examples include confusion, impatience, indirect help-seeking, beliefs, risk perception, fear, trust, and socially inappropriate actions.

2. Related Work and Preliminaries

This section positions Mental World Modeling as a world-modeling framework that couples physical and mental dynamics, extending existing physical, visual, and spatial modeling toward socially intelligible behavior. It distinguishes MWM from mental-state inference, BDI reasoning, and classical partial-observation formalisms, and introduces MENTIS as its inspectable implementation.

  • Mental World Modeling: MWM jointly represents physical and mental dynamics, renders a target-specific partial observation, and simulates how candidate actions update the coupled state.Its external simulator is approximate and task-relevant rather than a model of private conscious experience.
  • Mental World Modeling: MENTIS operationalizes MWM through state parsing, target-observation rendering, action decomposition, coupled transition simulation, and branch-level scoring.Branches are evaluated for physical plausibility, mental consistency, and social appropriateness.
  • Existing World Models: Existing world models primarily fall into representation, video-generative, and 3D interactive-spatial families that model latent dynamics, future observations, or persistent navigable environments.These families support prediction, planning, visual simulation, geometry, layout, camera control, and interaction affordances.
  • Mental-State Foundations: MWM addresses a distinct gap by representing latent mental and normative states that physical dynamics alone cannot explain in human behavior.The framework models what a target knows, believes, wants, intends, feels, and considers socially permissible.
  • Mental-State Foundations: Mental-model theory, Theory of Mind, affective computing, BDI agency, Machine Theory of Mind, and social-reasoning benchmarks provide related foundations but use different modeling units.MWM models a joint physical-mental world, target-specific observation, candidate actions, and the next joint state rather than only inferring or evaluating mental states.

3. Theoretical Framework

Mental World Modeling models social decision-making as an objective, target-centric simulator over coupled physical and mental states. It renders target-specific partial observations and predicts action-conditioned transitions in both state components.

  • Observation: The target receives a first-person partial observation rendered from the global state, because observation depends on physical affordances, mental state, social relations, and norms.MWM extends partial observability beyond hidden physical variables to false beliefs, private intentions, emotions, role relations, and social norms.
  • Definition: MWM maintains a joint state S = Sphy × Sment containing physical entities and conditions alongside beliefs, intentions, norms, roles, and other mental-social variables.The framework treats mental variables as causal state variables needed to predict behavior, not posthoc explanations.
  • Definition: MWM is an objective modeling abstraction, not a literal theory of consciousness or a claim to reproduce private conscious experience.The simulator maintains variables needed to predict what a target observes, how it acts, and how the scene evolves.
  • State transition: A coupled physical-mental transition is necessary: physical variables constrain what can happen, while mental variables determine what the same event means to agents.Physical-only and mental-only representations are insufficient in cases where identical scenes differ in beliefs or identical goals differ in physical access.
  • Action representation: An action has inseparable physical and mental dimensions, pairing a physical carrier such as speaking or gesturing with content such as requesting, deceiving, comforting, or threatening.The two components are dimensions of the same action rather than independent actions added together.

4. Where Mental World Modeling Matters

Mental World Modeling matters when physical scenes alone cannot recover the mental variables that determine what people perceive, choose, accept, resist, or learn. Its value is greatest when partial observability, mental-state-dependent decisions, and socially meaningful actions occur together, across collaboration, care, education, and interactive agents.

  • When MWM is worth the cost: MWM is most valuable when partial observability separates global and perceived states, mental variables alter rational or acceptable actions, and actions carry social meaning.If any condition is absent, a simpler physical or task-specific world model may suffice.
  • Embodied collaboration: In embodied collaboration, robots must model whether people noticed them, whether objects are socially available, and whether interruptions or actions are acceptable.The relevant transition includes more than what physically moves next.
  • Care, support, and advising: In care and advising, failures often arise from misrepresenting what people understand, fear, assume, or feel pressured to do rather than from missing factual knowledge.The same instruction may be heard as reassurance, blame, coercion, or permission.
  • Education and training: In education, the same wrong answer can reflect a missing concept, brittle heuristic, low confidence, inattention, or frustration, requiring different interventions.Tutoring targets the learner’s successor state because the immediate physical state is often uninformative.
  • Interactive agents and social worlds: Interactive agents and social worlds require mental transitions to remember promises, hide unobserved information, distinguish friends from strangers, and sustain emotional and normative coherence.Visual continuity alone is insufficient for believable daily behavior in games, digital humans, and agent societies.

5. MENTIS: An Inspectable Baseline Implementation of MWM

MENTIS is a modular, training-free, inspectable baseline that operationalizes MWM by jointly modeling physical and mental states, target-specific observations, action-conditioned futures, and branch selection. Its staged, artifact-producing design makes decisions auditable and failures localizable across parsing, observation, action decomposition, transition, and evaluation.

  • Overview: MENTIS parses situated scenes into a joint physical-mental state, renders the target agent’s partial observation, simulates candidate futures, and selects an answer.The system is designed as a practical testbed for evaluating whether LLMs and MLLMs can behave as a MWM.
  • Inspectability: Because MENTIS exposes its reasoning path, errors can be localized to state parsing, information leakage, action decomposition, transitions, or branch scoring.This component-level diagnosis is a stated advantage over a monolithic direct-answer prompt.
  • Six-stage pipeline: Inference comprises six stages: state parsing, observation generation, action decomposition, parallel branch simulation, value evaluation, and deterministic decision.Each stage instantiates a corresponding formal operator, and the final module selects the highest-value branch under fixed tie-breaking rules.
  • Action decomposition: Each option becomes a candidate action with explicit physical and mental components, preserving both its carrier and intended cognitive-social effect.The pseudo-agent compares supplied options rather than freely inventing actions, maintaining a controlled action space for multiple-choice evaluation.
  • Simulation and evaluation: MENTIS predicts physical and mental successors in parallel from the full joint state, merges them, evaluates each branch on normalized criteria, and records intermediate artifacts.The evaluator receives the state, target observation, decomposed action, successor state, and question; saved traces enable component-level evaluation and ablations.

6 Evaluation Settings

The evaluation uses Menti-Bench, a process-complete, multimodal testbed designed to test target-centric world modeling beyond outcome-only accuracy. Systems select a target agent’s next action while exposing inspectable traces, under layered metrics, controlled comparisons, and a fixed operating point.

  • Evaluation data: Menti-Bench contains 448 records across text, image, and sounding-video modalities, with full process-level annotations and broad scene-category and domain coverage.The dataset includes 320 text, 100 image, and 28 sounding-video records, totaling 2,688 annotated successor states; 78% involve at least two characters.
  • Task protocol: Each task asks systems to choose the target agent’s most likely next action from six options, while MWM systems expose machine-checkable state, observation, action, successor-state, score, and choice artifacts.Direct-answer baselines may output only the chosen letter, whereas MWM systems must provide the full trace.
  • Metric suite: The metric suite separates final-action F1, run-failure rate, and process-structure checks, pairing headline system comparisons on identical records.The layered design distinguishes final-answer correctness from process validity and branch-selection quality.
  • Systems: The system ladder adds one modeling commitment per rung, while information-removal ablations and oracle substitutions isolate specific mechanisms.S2 and S3 implement chain-of-thought and self-consistency baselines; A1–A3 and O1–O4 intervene on the full system.
  • Operating point: All reported runs use medium reasoning effort with batched comparative-rank scoring, fixed before headline evaluation through calibration on a frozen 30-record slice.Eight world models are compared across five OpenAI and three Anthropic models, with fixed target pseudo-agent and judge roles.

7. Experiments and Analysis

Experiments on Menti-Bench show that progressively explicit mental world modeling improves target-action prediction across eight world models, with benefits from state representation, coupled simulation, and value evaluation. Analyses identify transition simulation and state parsing as the main remaining bottlenecks, while gains are largest for interpersonal scenes and consistent across domains.

  • Necessity ladder: All eight world models improve at every ladder rung, with average F1 rising from 31.3 (S0) to 87.9 (S6).The largest increments are +32.0 from reading the story and +11.3 from explicit reasoning.
  • Necessity ladder: The full MWM pipeline outperforms direct answering, while explicit state notes and physical-mental schemas improve performance before simulation.S4 reaches 80.3 versus 77.9 for six-sample self-consistency, and S5 reaches 82.6.
  • Ablations: Removing the mental channel costs 12.1 points, removing the physical channel costs 16.5, and independently predicting transitions costs 6.4.The ordering S6 > A3 > A1 > A2 holds for all eight models, supporting the necessity of physical-mental coupling.
  • Bottleneck localization: Gold transition, state, observation, and action-decomposition interventions gain +3.5, +2.8, +1.7, and +0.7, respectively, making transition simulation the largest bottleneck.The transition oracle alone recovers 45% of the 7.8 human gap.
  • Bottleneck localization: With all four artifacts gold, the pipeline reaches 97.0, attributing 6.3 points (81%) of the 7.8-point human gap to intermediate-stage prediction errors.The remaining 1.5 points (19%) lie in value evaluation, the decision rule, or residual item difficulty.
  • Scenario and domain analysis: Full MWM gains most on interpersonal scenes (+26.4) and least on object/resource scenes (+14.0), while gains across five domains remain within 19.1–22.8.The between-category spread narrows from 7.5 points under S1 to 4.9 under S6.

8. Conclusion … M Future Directions

Mental World Modeling couples physical and mental states, target-specific observation, and joint action simulation to reason about people rather than only objects. Results show that this framework closes text–media performance gaps and that models use multimodal evidence rather than relying mainly on textual priors.

  • 8. Conclusion: 90.5, 91.2, 90.9: S6 closes the text–image–video F1 gap, with S6−S1 gains of +26.1 on video and +24.2 on image.Direct-answering F1 is 70.8 on text, 67.0 on image, and 64.8 on video; the gap disappears under S6.
  • 8. Conclusion: Every channel intervention lowers F1, indicating that the systems use media evidence rather than answering mainly from textual priors.For S6, keeping only audio costs 18.8 points, while shuffling frames costs a further 4.1.
  • 8 Conclusion: Mental World Modeling treats physical and mental states as one coupled world state and simulates how candidate actions change both material and mental-social configurations.The paper formalizes this framework and implements MENTIS as an inspectable baseline.
  • B Extended Description of MENTIS: The appendices cover inspectability and ablations, the multimodal front-end, and baseline limitations.These topics are listed under B.1–B.3.
  • F Menti-Bench: Construction, Composition, and Statistics: The paper documents dataset conversion and quality control, composition and statistics, evaluation objects, and a dataset datasheet.These topics are listed under F.1–F.4.
  • G Evaluation Metrics: Definitions and Computation: Evaluation materials define outcome, structural-validity, semantic-judge, score-alignment, decision-stability, and diagnostic metrics.These categories are listed in the evaluation-metrics appendix.
  • K Extended Results and a Worked Example: The supplementary contents include per-model gap statistics and a worked example from Menti-Bench, alongside run artifacts and reproducibility materials.These items are listed under K.1–K.2 and the reproducibility appendix.
  • M Future Directions: Future-direction materials address implications for the object of study, scaling the modeling machinery, and scaling the world being modeled, while the appendices also include ethics and responsible use.The listed future directions are M.1–M.3; ethics is separately listed as Appendix A.

A Ethics and Responsible Use · B Extended Description of MENTIS

MWM is presented as an auditable, functional simulator of task-relevant mental and social variables, not a model of private consciousness or human-like understanding. Because its inferences concern sensitive and culturally situated attributes, responsible use requires uncertainty, contestability, privacy safeguards, human oversight, and protection against manipulation and contamination.

  • A Ethics and Responsible Use: MWM explicitly simulates approximate, task-relevant mental and social variables to improve target-action prediction and make the process auditable.It does not simulate private conscious experience or demonstrate that language models possess a human Theory of Mind.
  • A Ethics and Responsible Use: Mental-state variables are hypotheses inferred from behavior and context, not measurements, so systems should represent uncertainty and keep assumptions contestable.People should be able to see, question, and correct what a system inferred about them.
  • A Ethics and Responsible Use: The benchmark contains no personal data because its characters, scenes, dialogues, images, and videos are fictional or synthetically generated.Real-user applications would process sensitive beliefs, emotions, and vulnerabilities and require consent, purpose limitation, minimization, retention limits, and legal compliance.
  • A Ethics and Responsible Use: Explicit mental-state modeling can support assistance or manipulation, so targeted persuasion, deception, covert influence, surveillance, and punitive decisions based on inferred states are inappropriate.The distinction between beneficial and harmful use depends on intent and governance.
  • A Ethics and Responsible Use: In care, health, education, and legal settings, MWM-style components require human oversight, escalation, and deferral rather than autonomous judgment.The framework is not validated for psychological assessment or diagnosis.
  • A Ethics and Responsible Use: The benchmark’s norm judgments reflect English-language conventions authored and annotated across everyday settings, not universal cultural standards.Cross-cultural application therefore requires recalibrating normative assumptions.
  • A Ethics and Responsible Use: Benchmark construction used human authoring, auditing, and adjudication of fictional, non-graphic scenes, with synthetic media quality-controlled to depict fictional characters.Human reference judgments were collected under the same task protocol on the same fictional records.
  • A Ethics and Responsible Use: The benchmark is released for research evaluation only, and its gold process annotations must remain out of training corpora to prevent diagnostic contamination.Users are asked not to redistribute gold blocks through training-scrapable channels.

B.1 Inspectability, Ablations, and Experimental Use · B.2 Multimodal Front-End · B.3 Limitations of the Baseline

MENTIS is an inspectable experimental scaffold whose logging and ablations expose where coupled physical-mental modeling succeeds or fails. Its multimodal front-end processes images, adaptively sampled video frames, and audio, while the baseline remains limited by prompted-module weaknesses, annotated action choices, and one-step transitions.

  • B.1 Inspectability, Ablations, and Experimental Use: MENTIS logs predictions, prompts, model responses, parsed artifacts, manifests, traces, summaries, and performance diagrams for inspectable runs.This supports testing at the levels of state representation, target observation, action semantics, physical transition, mental transition, and value evaluation.
  • B.1 Inspectability, Ablations, and Experimental Use: Its ablation variants distinguish failures from invented beliefs, hidden-information leakage, ignored social obligations, or overweighted physical feasibility.Oracle and information-removal settings support mechanism analysis beyond aggregate accuracy.
  • B.1 Inspectability, Ablations, and Experimental Use: Batched comparative scoring grades successful branches in one prompt context, with individual branch scoring as a schema-validation fallback.The evaluator returns per-dimension grades and comparative rankings before branch selection.
  • B.2 Multimodal Front-End: The modality-aware parser treats media ingestion as part of the measured system rather than hidden preprocessing.It builds on instruction-following multimodal LLMs.
  • B.2 Multimodal Front-End: Image stories provide one to five images and scene-anchor text, with prompts requiring verbatim caption quotation, temporal ordering, and visibility-asymmetry tracking.These instructions target records where who can see what determines the scenario.
  • B.2 Multimodal Front-End: Video processing adaptively samples scene-change candidates with uniform coverage anchors, merges near-duplicates, caps the frame budget, and preserves timestamps for event-order reasoning.A uniform-sampling fallback is also provided.
  • B.2 Multimodal Front-End: For sounding videos, MENTIS transcribes audio and injects the transcript alongside timestamped frames, caching per file and degrading gracefully when transcription fails.Dialogue and ambient audio provide evidence that frames alone can miss.
  • B.3 Limitations of the Baseline: MENTIS is a baseline rather than the final MWM architecture, inheriting prompt sensitivity, schema failures, mental-state over-inference, and cross-stage error propagation.Its benchmark constrains actions to annotated options and uses one-step transitions, so it does not yet evaluate open-ended policy generation.

C Theoretical Grounding of MWM … F.2 Composition and Statistics

The appendix formalizes MWM’s theoretical commitments, state taxonomies, and typed module interfaces, then documents Menti-Bench’s construction, quality control, composition, and evaluation coverage. Together, these materials make the framework inspectable and support controlled diagnosis on a compact but stratified benchmark.

  • C Theoretical Grounding of MWM: Table 10 pairs each theoretical source with the specific MWM modeling commitment it supports.It summarizes the arguments of Section 3.1 as theoretical grounding for the formal framework.
  • D Full State Taxonomy: Tables 11 and 12 provide the full physical- and mental-state taxonomies underlying the compact schema and benchmark annotation guidelines.The physical taxonomy appears in Table 11, while Table 12 covers mental state.
  • E Module Interface Schemas: MENTIS modules communicate through typed JSON objects, enabling intermediate artifacts to be validated, compared with gold annotations, and ablated.The abridged contracts follow the Appendix D taxonomies, with optional fields omitted for readability.
  • F Menti-Bench: Construction, Composition, and Statistics: Menti-Bench is designed for paired system comparisons, component-level diagnosis against gold process annotations, and audits of whether decisions can be solved without evidence.The appendix explicitly frames these uses as appropriate for the dataset’s small size.
  • F.1 Conversion Pipeline and Quality Control: 448 records have complete process-level gold covering the current state, target observation, six successor states, and final action, totaling 2,688 annotated successor states.The previously incomplete gold blocks for 26 media records were back-filled, followed by manual re-audit before experiments.
  • F.2 Composition and Statistics: Menti-Bench’s stratification spans interpersonal, object/resource, spatial/perceptual, and risk/norm scenes, matching distinct mental-world-modeling difficulties across modalities.These categories stress variables including belief, intention, emotion, politeness, affordance, ownership, partial observability, occlusion, reachability, viewpoint, risk, and norms.

F.3 Evaluation Objects · F.4 Dataset Datasheet

F.3 defines the benchmark’s evaluable artifacts and supports comparisons across baseline, ablation, oracle, and full-pipeline systems. F.4 presents Menti-Bench as a process-annotated diagnostic dataset for mental world modeling, with explicit scope, construction, intended use, and maintenance constraints.

  • F.3 Evaluation Objects: Section 6.2 accommodates direct LLM baselines, physical-only and mental-only ablations, oracle variants, and the full MENTIS pipeline.The single-task protocol makes these system variants comparable within one evaluation framework.
  • F.3 Evaluation Objects: The evaluation suite lists the artifacts systems may claim to construct and the output fields used to assess them.Intermediate objects are evaluated whenever a system claims to construct them.
  • F.4 Dataset Datasheet: Menti-Bench evaluates whether systems reconstruct a target agent’s state, partial observation, and action-conditioned mental transitions, beyond selecting the correct option.It is explicitly designed as a small, process-annotated diagnostic instrument rather than a large-scale leaderboard.
  • F.4 Dataset Datasheet: 448 single-decision records span 320 textual, 100 image, and 28 sounding-video examples, each with six candidate actions and complete process-level gold.Each record contains a situated decision scene, designated target agent, minimal question, joint current state, target observation, successor states, and final action.
  • F.4 Dataset Datasheet: All characters and scenes are fictional and contain no personal data, defining the dataset’s privacy and content scope.The benchmark’s records are situated scenes cut at decision moments rather than real-world personal cases.
  • F.4 Dataset Datasheet: Scenarios were adapted or authored under the stated construction protocol, while image and sounding-video assets were synthetically generated and human-quality-controlled.Gold labels were re-adjudicated so the gold action was uniquely defensible, and options were balanced against superficial letter-frequency and length cues.
  • F.4 Dataset Datasheet: The benchmark is intended for evaluating and diagnosing MWM systems, and its gold annotations must remain outside training corpora to preserve diagnostic validity.Training on Menti-Bench risks contaminating the process signals the benchmark is designed to measure.
  • F.4 Dataset Datasheet: Menti-Bench is released for research evaluation under a research-only license, with rejected video scripts and keyframes archived for regeneration and extension.The video subset can therefore be regenerated and extended as generation quality improves.

G Evaluation Metrics: Definitions and Computation … K.2 A Worked Example from Menti-Bench

The appendix defines outcome, structural, semantic, diagnostic, alignment, stability, and statistical metrics, then documents calibration, validity probing, reproducibility, per-model gaps, and a worked Menti-Bench example. Together, these sections specify how MENTIS is evaluated and how physical-mental modeling distinguishes candidate decisions.

  • G.1 Outcome Metrics: Outcome metrics report final-action correctness, macro-F1, and run-failure rate, while retaining failed records in accuracy denominators and using Acc-xf only as a separate non-failed measure.Random choice over six options yields Acc = 1/6 ≈0.167; F1 averages per-letter F1 across six option letters.
  • G.2 Structural Validity Metrics: Structural validity measures whether all options receive well-formed successor states, whereas judge metrics separately assess physical, mental, semantic, transition, and coupling fidelity.Judge scores are calibrated against human ratings on a stratified subsample before entering headline claims.
  • G.3 Semantic (Judge) Metrics: Perspective-leakage rate counts target-inaccessible assertions and separately reports cases whose removal changes the selected branch.Unknown gold observations and facts outside the target’s perceptual or epistemic reach define leakage.
  • G.4 Score-Alignment and Decision-Stability Metrics: Score-alignment metrics compare branch values with reference physical, mental, and social scores, while margins and tie rates characterize decision confidence and deterministic tie-breaking.A correct answer with a near-zero margin is flagged as low confidence.
  • G.5 Diagnostic Metrics: Diagnostic metrics compare performance with an options-only floor, quantify losses when story evidence is withheld, and identify correct answers produced by unfaithful processes.Direct-answer systems cannot enter the process-outcome table because they lack an inspectable judged process.
  • G.6 Statistical Testing: Statistical comparisons use exact two-sided McNemar tests for small discordant counts, while Wilson score intervals quantify uncertainty for proportions.Accuracy differences alone are not reported as significance.
  • H Operating-Point Calibration: All headline experiments freeze one operating point selected on a stratified 30-record slice, and the options-only floor is pooled across all 448 records without modality splits.Calibration records accuracy, failure rate, tie rate, and token cost across reasoning-effort and evaluator-scoring configurations.
  • J Run Artifacts and Reproducibility: Run manifests, traces, prompts, responses, parsed artifacts, token accounting, and reproducibility controls support auditability; full MWM beats direct answering and SC@6 for every model, while the worked example shows mental transitions rejecting a physically viable branch.The example records that branch D satisfies the physical goal but violates a norm and causes relational damage, whereas branch A restores access and corrects Lisa’s false belief.

L Error Taxonomy · M Future Directions

The error taxonomy attributes Mental World Modeling failures to specific state, perspective, action, transition, and evaluation modules. Future directions frame MWM as an unfinished research program whose necessity is established, while a 7.8-point gap remains open.

  • L Error Taxonomy: MWM failures are coded at the module level, enabling attribution behind the oracle analysis and qualitative case selection.The taxonomy supports module-level attribution rather than treating failures as undifferentiated errors.
  • L Error Taxonomy: State omission excludes a relevant physical or mental variable, whereas state hallucination inserts an unsupported variable.These errors concern whether the simulated state contains appropriate variables.
  • L Error Taxonomy: Perspective leakage gives the target information unavailable to it, while action misparse decomposes an answer option into incorrect physical or mental actions.Both errors can invalidate decision prediction before or during action simulation.
  • L Error Taxonomy: Physical transition errors predict implausible material consequences, and mental transition errors update beliefs, emotions, intentions, relations, or norms incorrectly.The taxonomy separates failures in material dynamics from failures in mental-social updates.
  • L Error Taxonomy: Evaluator error occurs when branch scores prefer an implausible action despite reasonable simulated states.This isolates decision-branch valuation as a failure source distinct from state simulation.
  • M Future Directions: MWM is presented as a research program, not a finished architecture, requiring simulated world states to include material and mental-social structure.The framework makes this requirement testable for systems intended to reason about people.
  • M Future Directions: The necessity of mental-social structure is no longer open because every commitment of the ladder succeeded across the tested models, but a 7.8-point gap remains open.The passage identifies the gap without specifying what follows after it.

M.1 What the Results Imply for the Object of Study · M.2 Closing the Gap: Scaling the Modeling Machinery · M.3 Scaling the World Being Modeled

The results establish that explicit mental-world structure is necessary in the tested regime, while future progress depends on scaling both the modeling machinery and the modeled world. This requires better transitions, interfaces, uncertainty and recursion handling, longer-horizon grounding, and safety-conscious, inspectable inference.

  • M.1 What the Results Imply for the Object of Study: Explicit structure beats direct answering by 21 to 28 points, while test-time sampling does not close the gap in the tested regime.Full MWM with gold intermediates reaches 97.0 versus 98.5 for humans, and the strongest predictive configuration reaches 90.7.
  • M.1 What the Results Imply for the Object of Study: Benchmark growth should preserve floor and channel audits while using counterfactual pairs, hidden-fact variants, and independently graded process labels to resist shortcuts.The floor experiment shows a flat band near 31 regardless of guesser strength, while channel interventions indicate that media evidence is load-bearing.
  • M.2 Closing the Gap: Scaling the Modeling Machinery: Prompted successor-state simulation is the binding stage, with gold transitions yielding +3.5, the largest single-oracle gain, and every combination containing transitions dominating counterparts.The passage identifies learning physical and mental transitions from annotated successor states, video, dialogue, and interaction traces as the next step.
  • M.2 Closing the Gap: Scaling the Modeling Machinery: Correlated stage errors create an oracle-gain tax, with +8.7 summed gains shrinking to +6.3 combined, motivating confidence flags, re-parsing, and future-observation cross-checks.The proposed interfaces are intended to recover part of the loss inherited by pipelines that merely chain modules.
  • M.2 Closing the Gap: Scaling the Modeling Machinery: Future systems must represent mental-state uncertainty and selectively support asking or deferring, while handling scenes that require first-order or second-order nested belief.Most scenes need no nested belief, a handful depend on first-order false belief, and only a few demand second-order reasoning.
  • M.3 Scaling the World Being Modeled: Scaling the modeled world requires durable state across longer horizons, including commitments, obligations, reputation, and shared history, with evaluation over coherent trajectories.The paper contrasts this regime with its current single-step setting and isolated-decision evaluation.
  • M.3 Scaling the World Being Modeled: Mental inference must remain grounded in physical access and social observability, while multimodal structured state already functions as a modality-agnostic interlingua for image and video stories.The passage notes that frames and audio are only a first step toward deeper grounding.
  • M.3 Scaling the World Being Modeled: Because mental inference can support or manipulate people, it should be treated as sensitive by default: uncertain, task-scoped, logged, and biased toward asking rather than asserting.MWM’s intended endpoint is an inspectable, faithful, and contestable account of why an agent would act, with the remaining 7.8 points mapped stage by stage.
Loading 2607.27201v1…