Quo Vadis, World Modeling?

1.1 From Static Supervision to Continual Improvement

• Offline data bounds agent competence to its training distribution because it cannot answer back when agents explore, leaving trial-and-error information unavailable for self-improvement.Continual improvement instead depends on turning successes and failures into feedback for future attempts.Source: §1.1 · p. 3

1.2 Bottlenecks of Direct Real-Environment Interaction

• Real-environment interaction is costly, risky, backward-looking, and hard to parallelize, making it indispensable for grounding but poorly suited as the sole scalable training ground.Agents need an intermediate place to try options, predict outcomes, and reason about what-ifs without paying the full cost of real execution.Source: §1.2 · pp. 3–4

1.3 World Modeling as an Intermediate Proxy

• World modeling should serve as a real-grounded, agent-facing proxy that provides cheaper, controllable feedback for simulation, retrieval, guidance, and verification beyond next-state prediction.This lets agents probe options before committing to costly or irreversible real-world actions.Source: §1.3 · pp. 4–5
• A useful proxy supports agent-initiated queries with conditioned feedback, learns from real-environment evidence, and optimizes actionable information rather than realism or prediction accuracy.Its feedback should improve context, safety, exploration, or training signals.Source: §1.3 · p. 5

2.1 Rethinking World Models

• Classical World Models predict the next physical state from the current state and action, a useful but limited account of agent–world interaction.Even learned-latent variants retain this state-transition backbone.Source: §2.1 · pp. 5–6

2.2 From World Model to World Proxy

• World Proxies broaden this intermediate, environment-grounded feedback mechanism to include execution simulation, experience retrieval, skill guidance, and behavior verification or evaluation.The classical World Model is broadened rather than replaced.Source: §2.2 · p. 6

What new information will the agent obtain after this interaction?

• A memory lookup, code execution, or reward estimate can each provide new agent-usable information, rather than only an environmental state.These outputs are treated as different forms of feedback within the same interaction framing.Source: What new information will the agent… · p. 7

2.3 Agent-Centric World Proxy Definition

• An Agent-Centric World Proxy is an environment-grounded mechanism that models information transitions conditioned on agent-initiated interactions to provide information gain for agent improvement.Its interaction step is not limited to physical time, and its information space can include states, observations, memory, execution results, guidance, and verification.Source: §2.3 · pp. 7–8
• In a closed loop, the agent proposes an interaction, the proxy returns predicted information as feedback, and the agent uses it for planning, decisions, learning, or improvement.Repeated interactions form a trajectory of improvement because each answer can shape the agent’s next question.Source: §2.3 · p. 8

2.4 What Makes a Good World Proxy?

• Effective World Proxies should be grounded, controllable, actionable, scalable, and forward-looking within the agent’s closed loop.They must balance real-environment grounding, scalability, and actionable feedback; confidently wrong proxies can be worse than none [153].Source: §2.4 · p. 9

How can World Proxies help an Agent plan, learn, and evolve?

• Unlike Chu et al.’s [31] Predictor/Simulator/Evolver scale of model competence, these L1-L3 levels measure agent improvement and are related but nonidentical.The paper distinguishes the two readings by setting them side by side in Table 4.Source: How can World Proxies help an Agent… · pp. 9–10
• World-Proxy-Driven Agent Improvement has three levels: L.1 augments inference-time context, L.2 provides training signals, and L.3 enables agent-proxy co-evolution.These levels describe the proxy’s impact on the agent.Source: How can World Proxies help an Agent… · p. 10, Figure 6 · p. 10

L.1 Inference-Time Guidance

• Inference-time proxies augment the agent’s context.Source: §L.1 · p. 10
• The added context supports better agent decisions.Source: §L.1 · p. 10

L.2 Training-Time Optimization

• Training-time proxies provide reward, verification, or simulation signals that agents use to optimize their policies.Source: §L.2 · p. 10

L.3 Agent-Proxy Co-Evolution

• L3 closes the agent–proxy–environment loop, letting agent and proxy continually reshape each other as feedback commitment and the burden of proof increase.The L1–L3 rungs describe agent-centric empowerment and align in spirit, not definition, with a world model’s intrinsic capability.Source: §L.3 · pp. 10–11, Table 4 · p. 11

3.1 L1: Inference-Time Guidance

• At L1, proxies add retrieved context, experience, skills, or verification at inference without changing parameters, cheaply improving decisions but remaining bounded by existing competence [62] [85] [169] [188].For example, a web agent can simulate a purchase page before acting and revise its plan without touching the live site.Source: §3.1 · pp. 11–12

3.2 L2: Training-Time Optimization

• At L2, proxies generate rewards, verification, critiques, or simulated rollouts that train the policy, raising its ceiling while making outcomes vulnerable to biased signals [28] [32] [99] [122].These signals can be converted into SFT, DPO [128], PPO [137], or GRPO [138] objectives.Source: §3.2 · p. 12

L2 Training-Time Optimization

• World Proxies convert evaluated, verified, or simulated rollouts into rewards, critiques, preferences, or verification signals that directly optimize the Agent policy.The proxy can serve as a reward model [10] [122], critic [109] [112] [200], simulator for synthetic trajectories or preference pairs [28] [161] [168], or curriculum generator targeting current failure modes [167] [206].Source: Figure 8 · p. 13, L2 Training-Time Optimization · pp. 13–14
• Agent rollouts become scalable training fuel when the proxy scores, verifies, or replays them into synthetic trajectories and preference pairs [28] [168].This supports policy optimization at a scale that real-interaction collection could never reach.Source: L2 Training-Time Optimization · p. 13

3.3 L3: Agent-Proxy Co-Evolution

• Real trajectories, failures, and discoveries update the proxy, which then guides, verifies, and trains the Agent so both improve together [41] [130] [140] [157].This co-evolution requires machinery to keep the proxy and policy aligned as both change.Source: §3.3 · pp. 14–15
• L3 couples proxy-to-Agent internalization of memory, skills, constraints, verification rules, or rewards with Agent-to-proxy updates from real trajectories, experiences, failures, and environmental feedback [2] [98] [157] [20] [41] [140].Updates can affect the proxy’s memory, skill library, verifier, simulator, or reward model.Source: §3.3 · p. 15

4 Instantiations: Functional Forms of Agent-Centric World Proxies; 4.1 Overview

• Section 4 distinguishes six functional forms, and Table 5 compares the input, represented world slice, and output associated with each.The forms are dynamics, spatial, execution, memory/experience, skill, and reward/verification proxies.Source: §4 · p. 16, Figure 10 · p. 16, §4.1 · p. 16

4.2 Dynamics Proxy (World Model)

• A Dynamics Proxy predicts future state, and optionally reward, from the current state or history and an agent-proposed action, action sequence, or future query.It is the classical, most literal form of a world model, focused on how the environment would change if the agent acted.Source: §4.2 · pp. 16–17

4.3 Spatial Proxy

• A Spatial Proxy returns a predicted or rendered observation for a queried location, pose, or viewpoint, letting the agent inspect a vantage point before moving there.Its input is known visual, geometric, or spatial information; its output represents what another viewpoint would reveal.Source: §4.3 · p. 18

4.4 Execution Proxy

• An Execution Proxy predicts the state and feedback produced by an executable interaction, such as a code command, web click, API call, or tool call.Unlike continuous-physics dynamics, it models discrete, often brittle digital-system logic; feedback can include output, errors, test results, or page changes.Source: §4.4 · pp. 18–19

4.5 Memory / Experience Proxy

• A Memory / Experience Proxy retrieves task-relevant prior interactions, failures, constraints, or risks in response to an agent query for planning or decision making.A static store alone is not a proxy: retrieval must return relevant information and feed it back into planning, decisions, or improvement.Source: §4.5 · p. 19

4.6 Skill Proxy

• A Skill Proxy recommends reusable skills, tool-use routines, behavior modules, or action priors for the current goal and context, emphasizing what the agent can do now.This contrasts with a Memory / Experience Proxy, which emphasizes what happened before.Source: §4.6 · p. 20

4.7 Reward / Verification Proxy

• A Reward / Verification Proxy evaluates an agent’s behavior, trajectory, answer, or plan with a reward, verification result, critique, preference, or failure reason, without simulating the full environment.It predicts how behavior would be judged by the environment, rules, or evaluation system, including whether it is correct, safe, or feasible and how it should improve.Source: §4.7 · p. 21

4.8 Putting It Together: Functions Meet Levels

• Any of the six proxy functions can support any agent-improvement level; functions can progress from advisor to teacher to partner, while spatial and reward proxies at L3 remain open directions.The mapping is illustrative rather than exclusive, since systems may span multiple levels; sparse cells mark underexplored territory.Source: §4.8 · p. 21, Table 6 · p. 22

• World proxies can predict dynamics, render spatial observations, simulate execution, retrieve experience, suggest skills, or provide reward and verification feedback.Source: Different proxy functions, one… · p. 22
• Their functions—dynamics, spatial, execution, memory, skill, and verification—differ in the information transition they approximate for the Agent.Source: Different proxy functions, one… · p. 22
• Regardless of function, a proxy’s output should help the Agent plan, decide, learn, verify, or continually improve.Source: Different proxy functions, one… · p. 22

5 Conclusion: Quo Vadis, World Modeling?

• Historically, world modeling pursued faithful next-frame prediction from a state and action [36].Source: §5 · p. 22
• An Agent-Centric World Proxy is an environment-grounded mechanism that returns the information transition an agent needs for continual improvement.Its outputs may include future states, rendered views, execution results, retrieved memories or skills, and plan verdicts.Source: §5 · p. 22
• The article argues that real environments alone cannot carry continual improvement, motivating proxies that serve agents through interactive information transitions.Source: §5 · p. 22

Agent-centric world proxies

• The framework spans L.2 training-time optimization and L.3 Agent-Proxy co-evolution, using dynamics, spatial, execution, memory, skill, and reward/verification feedback for actionable information gain.The unifying goal is actionable information gain rather than visual realism for its own sake.Source: §L.1 · p. 22

Trustworthiness

• Proxy trust is the hardest guarantee, and agents rarely judge online when to rely on proxy feedback or return to the real environment [125].Treating a proxy as an oracle can invite silent failure.Source: §L.1 · pp. 22–23

Fidelity and imagination

• Convincing generative proxies can violate modeled dynamics [56] [78] [80] [153], while errors compound over long rollouts, making calibrated uncertainty essential.Sharper pixels alone do not address this fidelity limitation.Source: §L.1 · p. 23

Reward hacking and safety

• When a proxy serves as an L.2 reward or verifier, agents can exploit its blind spots, creating an attack surface despite safer sandboxed exploration [94] [101] [178] [180] [198].Source: §L.1 · p. 23

Evaluation

• Existing benchmarks assess realism, fidelity, controllability, or human-aligned quality [87] [91] [127], but agent-centric evaluation should test whether feedback helps agents plan, learn, or improve.Source: §L.1 · p. 23

Agent-Proxy co-evolution

• The proposed L.3 loop continually corrects proxies with real-environment evidence while proxies improve agents, making co-evolution both a learning and safety mechanism.The agenda is to build useful, grounded, calibrated interlocutors rather than perfect mirrors, and judge them by how much they improve agents.Source: §L.1 · p. 23