Source-linked AI summary
Aligning Agentic World Models via Knowledgeable Experience Learning
Baochang Ren, Yunzhi Yao, Rui Sun, Shuofei Qiao, Ningyu Zhang, Huajun Chen
TL;DR
LLMs often produce logically coherent but physically unexecutable plans, while parameter-based alignment struggles to encode open-ended physical dynamics. WorldMind builds an explicit World Knowledge Repository from prediction errors and successful trajectories, combining physical verification with goal guidance. Across EB-ALFRED and EB-Habitat, it reports superior task performance and cross-model transferability, while remaining dependent on visual perception fidelity.
Problem
LLMs possess semantic reasoning but often lack procedural grounding, and static parameter-based alignment may not adapt to dynamic physical environments.
Method
WorldMind autonomously constructs a World Knowledge Repository by combining Process Experience from prediction errors with Goal Experience from successful trajectories.
Results
WorldMind achieves the highest Success Rate across EB-ALFRED and EB-Habitat, including 50.8% on EB-Habitat with GPT-4.1-mini and 48.0% on EB-ALFRED with GPT-3.5-turbo.
Takeaways & Limitations
The constructed world knowledge supports grounded planning and transfers across model architectures, indicating reusable environmental knowledge beyond a single backbone.
Takeaways & Limitations
WorldMind remains contingent on visual-language-model perception and cannot fully compensate for fundamental perceptual hallucinations in highly cluttered scenes.
Abstract
from arXiv · showhide
Current Large Language Models (LLMs) exhibit a critical modal disconnect: they possess vast semantic knowledge but lack the procedural grounding to respect the immutable laws of the physical world. Consequently, while these agents implicitly function as world models, their simulations often suffer from physical hallucinations-generating plans that are logically sound but physically unexecutable. Existing alignment strategies predominantly rely on resource-intensive training or fine-tuning, which attempt to compress dynamic environmental rules into static model parameters. However, such parametric encapsulation is inherently rigid, struggling to adapt to the open-ended variability of physical dynamics without continuous, costly retraining. To bridge this gap, we introduce WorldMind, a framework that autonomously constructs a symbolic World Knowledge Repository by synthesizing environmental feedback. Specifically, it unifies Process Experience to enforce physical feasibility via prediction errors and Goal Experience to guide task optimality through successful trajectories. Experiments on EB-ALFRED and EB-Habitat demonstrate that WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.
1 Introduction
LLMs can reason semantically yet lack procedural grounding in physical laws, producing coherent but unexecutable plans. WorldMind addresses this gap by combining prediction-error feedback and successful trajectories to align simulations with physical plausibility and task accuracy.
- Problem: LLMs often generate semantically coherent plans that violate physical constraints because declarative knowledge does not provide sufficient procedural grounding.The paper illustrates this mismatch with attempting to slice an object without holding a knife.
- Motivation: Fine-tuning and reinforcement learning attempt to encode dynamic physical laws in static parameters, creating limitations for open-ended environmental variability.The paper frames planning as requiring both optimal task achievement and strict physical feasibility.
- Approach: WorldMind treats execution failures as prediction errors that reveal boundaries in the agent’s internal world model.This perspective is inspired by Predictive Coding, where intelligence continuously minimizes discrepancies between expectation and reality.
- Approach: WorldMind’s World Knowledge Repository combines Process Experience for physical feasibility with Goal Experience for efficient convergence toward task objectives.Process Experience comes from prediction errors, whereas Goal Experience is distilled from successful trajectories.
- Results: Experiments on EB-ALFRED and EB-Habitat report state-of-the-art performance, fewer physical hallucinations, and cross-model transferability.The transferred knowledge is reported to capture universal physical laws independent of the backbone model.
2 Background
The paper formulates embodied interaction as a POMDP whose implicit transition modeling can leave physical laws uncaptured. WorldMind instead uses prediction errors as epistemic signals to refine the agent’s belief about environmental dynamics.
- POMDP Formulation: Embodied interaction is modeled as a POMDP with states, actions, transition dynamics, observations, and natural-language goals.The tuple is ⟨S, A, P, Ω, G⟩.
- Background: Standard reward-maximization approaches may learn what actions obtain rewards without explicitly representing immutable physical laws in transition dynamics.The paper identifies this implicit modeling as a source of a reality gap and physically invalid plans in novel scenarios.
- Predictive Coding: WorldMind operationalizes Predictive Coding by using execution failures and prediction errors to refine the internal world model against physical reality.The mechanism parallels biological refinement of motor control through sensorimotor contingencies.
3 The WorldMind Framework
WorldMind augments embodied decision-making with an explicit World Knowledge Repository containing physical rules and procedural heuristics. It builds this repository from prediction errors and successful trajectories, then uses retrieved knowledge for constrained, grounded simulation.
- 3.1 Overview and Problem Formulation: WorldMind externalizes environmental dynamics into explicit memory, enabling planning through goal alignment and continuous reality verification.The framework is formulated as a World Knowledge-Augmented Markov Decision Process.
- 3.1 Overview and Problem Formulation: The World Knowledge Repository augments the state, action, transition, observation, and goal formulation with Process Experience and Goal Experience.Process Experience approximates transition dynamics, while Goal Experience constrains policy search toward the target.
- 3.1 Overview and Problem Formulation: The policy jointly generates an action and predicted future state, with task success linked to minimizing divergence between predicted and actual states.This couples goal-directed action selection with state-prediction accuracy.
- 3.2 Building World Knowledge Repository: Process Experience is built through a Predict-Act-Verify loop that abstracts states, judges prediction discrepancies, and reflects on errors to synthesize corrective causal rules.The mechanism is designed to capture causal dynamics while remaining invariant to trivial environmental noise.
- 3.2 Building World Knowledge Repository: Goal Experience distills high-level procedural heuristics from successful trajectories to complement the physical feasibility provided by Process Experience.These heuristics are intended to guide efficient task completion and constrain future policy search.
- 3.2 Building World Knowledge Repository: During inference, the agent retrieves semantically relevant Process Experience and Goal Experience and uses them to generate grounded actions and predicted future states.Predicted outcomes are gated so the agent simulates them only when the target object is explicitly grounded in the current observation.
4 Experiment
WorldMind is evaluated on EB-ALFRED and EB-Habitat across multiple capability subsets, backbones, baselines, and complementary success metrics. Results show strong task completion, procedural correctness, cross-backbone robustness, and complementary benefits from Goal and Process Experience.
- Experimental Settings: The experiments compare GPT-4.1-mini and GPT-3.5-turbo WorldMind agents with representative open- and closed-source baselines across the two benchmarks.Baselines include Best-of-N, ReAct, Synapse, SimuRA, ReasoningBank, and AWM.
- Main Results: WorldMind achieves the highest Success Rate across both benchmarks, including 50.8% on EB-Habitat with GPT-4.1-mini and an increase from 44.4% to 48.0% on EB-ALFRED with GPT-3.5-turbo.SR measures whether the final goal is fully achieved.
- Main Results: Goal-Conditioned Success improves from 50.4% to 63.0% on EB-ALFRED with GPT-3.5-turbo and reaches 57.2% on EB-Habitat, indicating stronger intermediate-step execution.GC awards partial credit for completed subgoals even when the episode ultimately fails.
- Main Results: The framework remains stable across capability subsets and backbones, reaching 86% Success Rate in the EB-Habitat Base subset for both GPT-3.5-turbo and GPT-4.1-mini.The reported evaluation spans Base, Common Sense or Common, Complex Instruction or Complex, Visual Appearance or Visual, and Spatial Awareness or Spatial subsets.
- Ablation Study: Goal Experience primarily improves Goal Condition, whereas Process Experience produces larger gains in strict Success Rate by guarding against physically infeasible actions.The full model performs best, supporting complementary roles for goal-based heuristics and physical verification.
5 Analysis
WorldMind transfers symbolic experience across model architectures and improves robustness across embodied and hybrid environments. Its error analysis links Process Experience to fewer physical violations and Goal Experience to fewer premature terminations.
- Cross-Model Experience Transfer Analysis: Experience repositories transfer bidirectionally across GPT-3.5-turbo and GPT-4.1-mini, improving EB-ALFRED and EB-Habitat performance over ReAct baselines.GPT-3.5-turbo using GPT-4.1-mini’s repository improves EB-ALFRED SR from 44.4% to 48.8% and GC from 50.4% to 57.0%; GPT-4.1-mini gains 10.3% SR on EB-Habitat.
- Cross-Environment Analysis: The Embodied Web Agent benchmark evaluates domain-specific accuracy, joint Overall Accuracy, and partial-credit Completion Rate across web and physical environments.The benchmark uses 112 Indoor Cooking tasks and measures Embodied Accuracy, Web Accuracy, Overall Accuracy, and Completion Rate.
- Cross-Environment Analysis: 17.02% to 39.99% Completion Rate for GPT-3.5-turbo and 21.88% to 41.50% for GPT-4.1-mini show improved trajectory maintenance during context switching.The results support WorldMind’s generalization to hybrid environments involving web information seeking and embodied execution.
- Error Analysis: WorldMind reduces Invalid Actions for GPT-3.5-turbo in EB-Habitat from 105 to 67, while Timeouts increase from 4 to 30.The redistribution indicates fewer immediate physical failures but more prolonged exploration.
- Error Analysis: WorldMind reduces Wrong Terminations for GPT-4.1-mini in EB-ALFRED from 46 to 19, consistent with Goal Experience providing guidance against premature quitting.Failures are categorized as Invalid Actions, Timeout, and Wrong Termination.
6 Related Work
Related work spans agentic AI, world models, and methods for interacting with external environments. World models support prediction and planning across reinforcement learning, simulation, language models, and robotics.
- Agentic AI: Agentic AI uses reflective, self-correcting loops with tools, memory, and constraints to support open-world robustness and interpretability.Planning mechanisms include chain-of-thought, tree-of-thought, and self-refinement.
- Agentic AI: Researchers have extended agentic systems through memory, tool-calling, embodied control, and collaboration between agents and world models.
- Agentic AI: Related work on agentic systems includes substantial research focused on memory and additional studies spanning other aspects of agent-world interaction.
- World Models: World models learn internal representations of environment dynamics for prediction and planning across model-based reinforcement learning, generative simulation, LLMs, and robotics.They enable agents to imagine possible outcomes.
- World Models: Generative world models predict physical environment changes through high-fidelity video generation, code execution results, and predictions in diverse domains.
7 Conclusion
WorldMind aligns agentic world models through experiential learning, combining process and goal experience to ground LLM-based agents in dynamic environments without gradient updates. Its empirical results support reduced physical hallucination and reusable knowledge across tasks and embodiments.
- Conclusion: WorldMind aligns agentic world models through experiential learning without requiring gradient updates.It combines Process Experience for physical verification with Goal Experience for heuristic guidance.
- Conclusion: Empirical results across diverse benchmarks indicate that WorldMind minimizes physical hallucinations and supports reusable knowledge across tasks and embodiments.
Limitations
WorldMind’s limitations concern perceptual dependence, incomplete mechanistic explanation, and unresolved requirements for real-time multi-agent knowledge sharing.
- The framework depends on the underlying VLM’s visual perception and cannot fully correct semantic object misclassification in highly cluttered scenes.
- The mechanism by which symbolic World Knowledge reshapes latent transition dynamics and decision boundaries remains incompletely understood.
- Extending the framework to simultaneous multi-agent collaboration requires unexplored mechanisms for real-time synchronization, conflict resolution, and consensus building.
A Additional Evaluation on Low-Level Navigation Tasks
WorldMind is additionally evaluated on EB-Navigation, which tests low-level atomic control rather than primarily high-level semantic planning. The evaluation spans two backbones and four capability-oriented subsets, with reported success-rate comparisons against baselines.
- EB-Navigation evaluates precise physical grounding through atomic controls such as moving 0.25m or rotating 90 degrees.
- The evaluation covers GPT-3.5-turbo and GPT-4.1-mini across Base, Common Sense, Complex Instruction, and Visual Appearance subsets.
- Table 4 reports success rates for WorldMind and baseline methods across four capability-oriented subsets and their macro-average.
- +10.0% improvement in the GPT-3.5-turbo Base subset indicates that successful-trajectory retrieval helps avoid oscillatory movements and planning errors.
- Improvements in the Visual Appearance subset for both backbones suggest that Process Experience supports generalization across action spaces and fine-grained kinematic control.
B Prompts
The prompts operationalize WorldMind as a prediction-guided planning system: agents simulate outcomes, validate physical and semantic constraints, and use feedback and experience to choose executable actions.
- B.1 WorldMind Prompt for ALFRED: The ALFRED prompt requires an internal World Model, stepwise reasoning, and an executable action plan with predicted states.
- B.1 WorldMind Prompt for ALFRED: ALFRED action rules enforce visibility, validity conditions, indexed object references, feedback-based instruction extraction, and anti-loop behavior.
- B.1 WorldMind Prompt for ALFRED: The ALFRED prompt also prohibits repeating failed or invisible Pick up or Turn on actions and requires strategy changes.
- B.1 WorldMind Prompt for ALFRED: For visible targets, predicted_state describes environmental and gripper changes; during exploration, it uses a fixed prediction-skipping string.
- B.1 WorldMind Prompt for ALFRED: After prediction is skipped in an ALFRED plan, all subsequent actions must use the same skip string.
- B.2 WorldMind Prompt for Habitat: The Habitat prompt uses a Predict-Act-Verify philosophy combining physical feasibility and semantic plausibility before execution.
- B.2 WorldMind Prompt for Habitat: Habitat action validity rules constrain Navigation, Pick, Place, Open, and Close according to proximity, object possession, visibility, and receptacle state.
- B.2 WorldMind Prompt for Habitat: Habitat planning uses feedback, predicted immediate outcomes, semantic simulation, common-sense location checks, and exhaustive left/right local search.