Source-linked AI summary
Quo Vadis, World Modeling?
Yu Yang, Xuemeng Yang, Licheng Wen, Lingdong Kong, Xiaobin Hu, Dongyue Lu, Wei Chow, Xiyan Huang, Yuxiang Feng, Yue Liao, Jianbiao Mei, Daocheng Fu, Rong Wu, Pinlong Cai, Ran Yi, Ying Tai, Jiangning Zhang, Botian Shi, Yong Liu, Shuicheng Yan
TL;DR
Static supervision and direct real-environment interaction provide insufficiently scalable feedback for continual agent improvement. This paper introduces agent-centric World Proxies that model agent-usable information transitions and organizes their roles into six proxy forms and three levels of agent improvement.
Problem
Static offline supervision cannot provide the new information revealed through active trial and error, while direct environmental interaction is costly, risky, backward-looking, and difficult to parallelize.
Method
The paper defines environment-grounded World Proxies that predict agent-conditioned information transitions and broaden world modeling beyond next-state prediction.
Results
The paper organizes World Proxies into six feedback modalities and analyzes their use across three levels: inference-time guidance, training-time optimization, and agent-proxy co-evolution.
Takeaways & Limitations
World modeling is recast as an agent-facing proxy that can simulate, retrieve, guide, and verify feedback for agents before real-world execution.
Takeaways & Limitations
World Proxies must remain grounded in the real environment and actionable, because proxies that drift from grounding or are confidently wrong can become harmful.
Abstract
from arXiv · showhide
Continually improving agents require dynamic interaction feedback beyond static supervision, yet direct real-environment interaction is costly, slow, unsafe, and hard to parallelize. World modeling offers a natural intermediate proxy that allows agents to query lower-cost, more controllable feedback before committing to real actions. Classical world models instantiate this proxy primarily through future physical-state prediction, a formulation useful yet narrow for agents that require actionable feedback beyond raw state transitions. In this work, we conceptualize Agent-Centric Interactive World Proxies, shifting the fundamental paradigm from physical state transitions to agent-usable information transitions, such as execution outcomes, retrieved experiences or skills, and verification signals, broadening the scope of world modeling to provide versatile feedback for continually improving agents. To systematically map this design space, we organize world proxies into six functional forms based on their feedback modalities: dynamics, spatial, execution, memory/experience, skill, and reward/verification proxies, which together characterize the primary ways world modeling serves agent improvement. We further analyze how these proxies empower agents across three progressive levels: L.1 Inference-Time Guidance, where proxy outputs enrich in-context information for superior decisions; L.2 Training-Time Optimization, where proxy outputs yield rewards, critiques, or synthetic rollouts for policy learning; and L.3 Agent-Proxy Co-Evolution, where real-environment evidence continuously updates both the proxy and the agent for co-evolution. Ultimately, this work recasts world modeling into an agent-centric paradigm, establishing a roadmap for building world proxies that empower agents to plan better, learn faster, and evolve continually.
1 Motivation: Why Improving Agents Need World Modeling
Continually improving agents need active interaction feedback, but static supervision cannot provide the new information revealed by trial and error. Real-environment interaction supplies faithful feedback yet is costly, risky, backward-looking, and difficult to parallelize, motivating agent-centric world proxies that provide cheaper, controllable feedback for planning and improvement.
- Why static supervision is insufficient: Static offline data bounds agent competence to its training distribution and cannot provide information revealed through active trial and error.Such data includes expert trajectories, human annotations, and supervised fine-tuning corpora.
- Why direct interaction is insufficient: Real-environment interaction provides observations, execution results, rewards, and errors, but becomes costly, risky, backward-looking, and difficult to parallelize at scale.These limitations emerge when agents must make thousands of errors, branch across possibilities, and reason several steps ahead before acting.
- World modeling as an intermediate proxy: World modeling serves as an intermediate proxy between agents and reality, providing interaction feedback in a cheaper, more controllable, and more predictable form.The proxy remains grounded in the real environment rather than serving as a photorealistic replacement for it.
- World modeling as an intermediate proxy: A proxy lets agents probe actions, viewpoints, tool calls, prior experiences, and plan safety before making costly or irreversible real-world moves.These queries support planning and decision making by answering what-if questions before commitment.
- Requirements for useful proxies: Agent-centric world modeling should support agent-initiated queries with conditioned feedback, remain grounded in real-environment evidence, and optimize actionable information gain.Its objectives include better context, safer decisions, more effective exploration, and higher-quality training signals rather than only visual realism or prediction accuracy.
2 Definition: From World Models to Agent-Centric World Proxies
The section broadens classical world models from physical state-transition prediction into agent-centric world proxies that model actionable information transitions. These proxies support agent improvement through diverse feedback, agent-initiated closed-loop interaction, and grounded, scalable, actionable operation.
- From World Models to World Proxies: Classical world models map a current physical state and action to a future state, forming a narrow but useful basis for robotics and model-based reinforcement learning.This state-transition formulation also underlies latent-space and raw-pixel prediction.
- From World Models to World Proxies: Agent-centric world proxies broaden feedback beyond next states to execution results, retrieved experiences, reusable skills, rewards, critiques, and verification signals.Their outputs are agent-usable information, including consequences, memories, skills, and judgments that can directly support action.
- Formal Definition: An Agent-Centric World Proxy is an environment-grounded mechanism that predicts information transitions conditioned on agent-initiated interactions to provide information gain for agent improvement.Its generalized information state may include physical state, observation, memory, knowledge, execution results, verification, and guidance.
- Agent-in-the-Loop World Proxy: The proxy operates as a closed loop: the agent proposes an interaction, the proxy returns predicted or generated feedback, and the agent uses it for planning, decision making, policy learning, or continual improvement.Iterating this process turns individual predictions into an improvement trajectory because each response influences subsequent queries.
- Design Principles: Effective world proxies must be grounded, controllable, actionable, scalable, and forward-looking, balancing real-environment grounding with useful feedback and avoiding confidently wrong outputs.The paradigm therefore prioritizes information gain for planning, decision making, learning, and continual improvement over prediction accuracy alone.
3 Empowerment: How World Proxies Improve Agents
This section evaluates world proxies by how much they improve agents, rather than by the world model’s isolated capability. It defines an agent-centric L.1–L.3 axis spanning inference-time context, training signals, and agent–proxy co-evolution.
- World-Proxy-Driven Agent Improvement: The agent-centric L.1–L.3 scale parallels, but does not duplicate, the world model’s intrinsic capability scale.The intrinsic scale describes a Predictor, Simulator, and Evolver, while the agent-centric scale focuses on the resulting improvement to the agent.
- World-Proxy-Driven Agent Improvement: L.1 augments inference-time context, L.2 provides training signals, and L.3 enables agent–proxy co-evolution.These levels organize how world proxies empower agent improvement across progressively broader modes of interaction.
- World-Proxy-Driven Agent Improvement: World proxies are assessed by how much they improve agents, not by how capable the underlying world model is in isolation.The agent-centric axis is intentionally distinct from the world model’s intrinsic competence scale, although both reuse L.1–L.3 shorthand.
L.1 Inference-Time Guidance
Inference-Time Guidance uses world proxies to augment the agent’s inference-time context, enabling better decisions.
- L.1 Inference-Time Guidance: World proxies augment inference-time context to improve agent decisions.
L.2 Training-Time Optimization
Training-Time Optimization uses world proxies to provide reward, verification, or simulation signals that optimize the agent’s policy.
- L.2 Training-Time Optimization: World proxies provide reward, verification, or simulation signals for training-time policy optimization.The proxy outputs directly support optimizing the agent’s policy.
L.3 Agent-Proxy Co-Evolution
L.3 Agent-Proxy Co-Evolution closes the Agent-Proxy-Environment loop, allowing the agent and proxy to reshape each other over time. This deepest level increases capability alongside the burden of proof because proxy mistakes become costlier.
- L.3 Agent-Proxy Co-Evolution: L.3 closes the Agent-Proxy-Environment loop, with the agent and proxy continually co-evolving.The proxy’s role is to complete the interaction loop, while its effect is continual co-evolution of agent and proxy.
- L.3 Agent-Proxy Co-Evolution: Unlike L.1 and L.2, L.3 lets the agent and proxy reshape each other over time.The three levels form a ladder: L.1 informs the next move, L.2 rewrites policy parameters, and L.3 enables mutual reshaping.
- L.3 Agent-Proxy Co-Evolution: As feedback reaches deeper into the agent, capability and the burden of proof both increase because mistakes cost more.L.3 represents the deepest feedback relationship in the ladder and therefore carries the greatest stated cost of proxy errors.
L1 Inference-Time Guidance
At L1, the World Proxy augments the Agent’s inference-time context with retrieved, simulated, or verified information without changing its parameters. This provides cheap, reversible guidance that improves decisions within the current episode but remains bounded by the Agent’s existing competence.
- L1 Inference-Time Guidance: L1 guidance enriches the Agent’s context so it can make better-informed decisions and act more robustly.The proxy’s returned guidance is added to the current context for information augmentation.
- L1 Inference-Time Guidance: The World Proxy supplies extra context, experience, skills, or verification feedback at inference time while leaving the Agent’s parameters unchanged.Because nothing is retrained, L1 is cheap and fully reversible.
- L1 Inference-Time Guidance: The proxy supports memory or skill retrieval, execution simulation, and verification feedback for the current task.Verification checks whether a plan or action is safe, feasible, and constraint-consistent.
- L1 Inference-Time Guidance: Before real execution, an agent can simulate an action’s outcome and revise its plan when the predicted result reveals an error or unintended charge.The example concerns a web agent querying the proxy before clicking Purchase.
L2 Training-Time Optimization
At L2, the World Proxy becomes a training-time judge, teacher, or simulated environment that converts agent rollouts into signals for directly optimizing the policy. It can score, critique, verify, replay, or simulate experience to support scalable policy learning beyond real interaction.
- L2 Training-Time Optimization: The World Proxy evaluates agent-generated rollouts and converts rewards, critiques, preferences, or verification signals into training objectives.This feedback directly optimizes the Agent policy rather than merely guiding an individual decision.
- L2 Training-Time Optimization: Proxy-as-Reward scores trajectories, Proxy-as-Critic diagnoses failures, and Proxy-as-Simulator generates synthetic trajectories or preference pairs for policy learning.These roles support DPO, RLHF, and GRPO, while proxy-guided curricula target tasks matching the agent’s current failure modes.
- L2 Training-Time Optimization: The agent’s rollouts become training fuel as the proxy scores, verifies, or replays them into synthetic trajectories and preference pairs.This enables policy optimization at a scale that collecting real interactions could never reach.
- L2 Training-Time Optimization: L2 upgrades the World Proxy from an inference-time advisor to a training-time judge, teacher, or simulated environment that directly optimizes the Agent policy.In the three-level progression, L2 is defined by the proxy acting as a reward model, critic, verifier, or simulator.
4 Instantiations: Functional Forms of Agent-Centric World Proxies
Section 4 instantiates Agent-Centric World Proxies as six functional forms—dynamics, spatial, execution, memory/experience, skill, and reward/verification—distinguished by the agent question each answers. These functions independently map onto agent-improvement levels, allowing the same proxy to progress from guidance to optimization to co-evolution.
- 4.1 Overview: Six proxy forms answer different agent questions while consuming distinct inputs and returning corresponding feedback: dynamics, spatial, execution, memory/experience, skill, and reward/verification.The forms respectively simulate dynamics, render observations, predict execution outcomes, retrieve experience, suggest reusable skills, or provide judgment feedback.
- 4.2 Dynamics Proxy (World Model): The Dynamics Proxy predicts future state, and optionally reward, from current or historical state information and a proposed action or future query.It is the closest of the six forms to the textbook world model and focuses on how the environment would change after an action.
- 4.3 Spatial Proxy: The Spatial Proxy answers what the agent would observe from another viewpoint or position by generating an observation or spatial representation for a queried pose or location.This lets the agent inspect a potential vantage point before physically relocating.
- 4.4 Execution Proxy: The Execution Proxy predicts post-operation state and feedback for executable interactions such as code, web clicks, API calls, or tool calls.Its feedback can include stdout, stderr, errors, test results, or page changes, modeling brittle digital-system logic.
- 4.5 Memory / Experience Proxy: The Memory / Experience Proxy retrieves task-relevant prior experiences, failures, constraints, or risk hints in response to an agent query and feeds them into decision making.A store becomes such a proxy only when retrieval dynamically returns environment-, task-, or decision-relevant information for planning or improvement.
- 4.6 Skill Proxy: The Skill Proxy recommends reusable skills, tool-use routines, behavioral modules, or action priors based on the current goal, task context, and environment state.It emphasizes what the agent can do now, rather than what happened previously.
- 4.7 Reward / Verification Proxy: Reward / Verification Proxies provide feedback by predicting how the agent’s behavior would be judged by the environment, rules, or evaluation system without simulating the full environment.They are feedback-oriented proxies centered on outcome assessment rather than complete world-state simulation.
- 4.8 Putting It Together: Functions Meet Levels: Proxy functions and agent-improvement levels are orthogonal: one level supports many functions, while one function can advance from advisor to teacher to partner across L1-L3.The mapping is illustrative rather than exclusive, and sparse spatial and reward/verification co-evolution cells identify open territory.
Different proxy functions, one purpose: agent-usable feedback.
World proxies extend beyond future-state prediction to provide diverse agent-usable feedback. Their functions differ by the information transition they approximate, but all aim to support planning, decision-making, learning, verification, or continual improvement.
- Different proxy functions, one purpose: agent-usable feedback.: World proxies can predict dynamics, render spatial observations, simulate execution, retrieve experience, suggest skills, or provide reward and verification feedback.These functions broaden proxy outputs beyond future physical-state prediction.
- Different proxy functions, one purpose: agent-usable feedback.: The six proxy functions—dynamics, spatial, execution, memory, skill, and verification—differ in the information transition they approximate for the Agent.Function determines the type of feedback the proxy provides.
- Different proxy functions, one purpose: agent-usable feedback.: Actionability is the shared criterion: every proxy output should help the Agent plan, decide, learn, verify, or continually improve.The proxy’s purpose is defined by whether its feedback is usable for agent improvement.
5 Conclusion: Quo Vadis, World Modeling? · 6 List of Contributors
The conclusion reframes world modeling as agent-centric proxies that deliver actionable information, while identifying trust, safety, and evaluation as central open problems. The contributors are credited across design, writing, visual design, discussion, and advising.
- 5 Conclusion: Quo Vadis, World Modeling?: World modeling should serve agents through environment-grounded proxies that return needed information transitions, including states, views, execution results, memories, skills, or plan verdicts.This shifts the focus from predicting the world to supporting continual agent improvement.
- 5 Conclusion: Quo Vadis, World Modeling?: The framework organizes proxies by six feedback forms and three improvement levels: dynamics, spatial, execution, memory, skill, reward / verification; L.1, L.2, and L.3.The unifying objective is actionable information gain rather than visual realism alone.
- 5 Conclusion: Quo Vadis, World Modeling?: Trust is the central challenge because proxies must provide grounded, calibrated feedback that remains useful for continual improvement.The conclusion identifies trustworthiness as the hardest property to guarantee.
- 5 Conclusion: Quo Vadis, World Modeling?: Generative proxies can violate modeled dynamics, accumulate errors over long rollouts, and require calibrated uncertainty rather than sharper pixels.The conclusion frames fidelity and uncertainty as limits of imagination.
- 5 Conclusion: Quo Vadis, World Modeling?: Agents must judge online when to trust proxy feedback or return to reality, since treating a proxy as an oracle can cause silent failure.The cited concern is that agents currently make this judgment poorly.
- 5 Conclusion: Quo Vadis, World Modeling?: Using proxies as rewards or verifiers can incentivize reward hacking, while sandboxed exploration introduces a new attack surface.These risks arise when agents exploit proxy blind spots, especially during L.2 optimization.
- 5 Conclusion: Quo Vadis, World Modeling?: Agent-centric benchmarks should measure whether proxy feedback helps agents plan, learn, or improve, rather than grading proxies only on realism, fidelity, controllability, or human-aligned quality.The proposed evaluation target is information gain for the querying agent.
- 6 List of Contributors: Contributors covered concept and design, writing and editing, figures and visual design, discussion and insights, and advising.Concept and design: Yu Yang, Xuemeng Yang, Licheng Wen; advising: Botian Shi, Yong Liu, Shuicheng Yan.