Source-linked AI summary
Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models
Pengfei Zhou, Hexin Wang, Zhengfeiyang Zhang, Yixing Ma, Zhenglin Wan, Kaipeng Zhang, Wangbo Zhao, Yang You
TL;DR
Spatial world-model scaling is constrained by fuzzy, costly reward signals and expensive real-world spatial annotation. The paper proposes AWoMo and RLHEV, using game-engine checks plus human acceptance in development trajectories; reported results are positive, though larger scaling studies are still needed.
Problem
Spatial generation relies on fuzzy reward proxies and expensive annotations, limiting reliable post-training and scalable correctness feedback.
Method
The paper proposes AWoMo and RLHEV, embedding world models in game-development workflows that combine engine checks, human acceptance, and development trajectories.
Results
AWoMo-generated data improves reported embodied metrics by +0.79% on R2R, +9.96% on Gymnasium MuJoCo, and +48.43% on D4RL Gym-MuJoCo.
Takeaways & Limitations
Game development provides an executable feedback loop in which world construction, verification, and learning can support recursive self-improvement.
Takeaways & Limitations
Current results are positive but diagnostic, and larger scaling studies are required to verify generalization capability.
Abstract
from arXiv · showhide
A common strategy for scaling world models is to train on more crawled video with more compute. We argue that this strategy is inefficient: scaling world models also requires a recursive data engine that offers grounded reward signals. The success of code agents illustrates why this matters. As code is executable, compilers and runtimes can provide high-quality rewards for Reinforcement Learning (RL) post-training of LLMs. By contrast, spatial generation still relies largely on fuzzy proxies such as CLIP scores. These signals are fuzzy and biased, making them hard to support RL post-training. Compared with these, game development provides a missing reward environment for spatial world models. A scene encoded by a game engine is an executable world specification: the engine can efficiently check collision, physics, navigability and bounded playability, while the developer provides the global verification signal by judging whether the scene should be accepted. Game development also provides real-world long-horizon trajectory data for RL post-training. We therefore propose Reinforcement Learning with Human-Engine Verification (RLHEV), a post-training paradigm that combines dense engine signals with implicit human acceptance feedback from the development process.
1. Introduction
World-model scaling is limited by weak spatial reward signals, not simply by insufficient data or compute. The paper proposes game development and AWoMo as a recursive human-engine verification loop that turns development traces into post-training supervision.
- Motivation: Spatial outputs are typically evaluated with noisy, biased, and gameable proxies, while final-output human preferences are too costly and low-bandwidth for iterative post-training.The cited proxies include JSD, FVD, CLIP similarity, and MLLM-as-judge scores.
- AWoMo: AWoMo embeds a world model in a developer workflow that proposes, renders, checks, revises, and learns from complete multimodal world-building trajectories.Stored traces include prompts, scene programs, rendered states, failures, fixes, engine checks, and human decisions.
- Game development as a data engine: Game development supplies executable world specifications that engines can check while developers judge global scene acceptance.This creates complementary structural and human verification signals.
- RLHEV: RLHEV combines dense engine checks with sparse but aligned developer acceptance feedback to support recursive improvement of world models.The loop exposes local mistakes through execution and evaluates global success through human judgment.
- Evidence and agenda: Development trajectories may be more valuable than finished assets because they record intent, expected state, changes, checks, and human acceptance.The paper states this claim is falsifiable through out-of-distribution generalization tests.
- Evidence and agenda: The paper presents controlled studies showing benefits from full human-engine feedback, source pretraining under distribution shift, cross-engine transfer, and AWoMo-generated embodied data.Reported improvements span UnitySceneBench, Unreal, Godot, R2R, Gymnasium MuJoCo, and D4RL Gym-MuJoCo.
2. The Thesis of Verification
The paper argues that scalable progress depends on a reliable feedback channel: executable verification for formalizable properties and human judgment for open-ended spatial quality. RLHEV operationalizes this split by storing engine checks and human acceptance in post-training trajectories.
- Verifiable reward: A verifier maps an input and candidate output to a reward under an explicit correctness criterion at substantially lower cost than generating the output.Examples include program execution, theorem checking, and numerical comparison.
- Verifiable reward: Efficient verifiers ground rewards in task specifications rather than subjective preference proxies, enabling reinforcement learning from verifiable reward.The paper connects this pattern to progress in code, games, and mathematics.
- Human-engine verification: Game engines can detect structural failures such as collider intersections, navmesh disconnection, script errors, and failed bounded playtest goals, but not whether a cutscene has the intended mood.RLHEV therefore combines dense engine labels with human acceptance in unified state-action-check-review trajectories.
- Human-engine verification: RLHEV treats reward as constrained candidate selection: weighted engine penalties are optimized subject to required engine gates passing.The formulation separates soft diagnostic checks from hard runtime constraints.
- Why verification matters: The proposed loop lets models propose, receive grounded feedback, update, and generate stronger candidates while humans focus on final acceptance rather than every low-level failure.This makes expensive human judgment more efficient by grounding it in executable evidence.
- Why verification matters: Proxy reward error contains zero-mean noise and systematic bias, with bias potentially allowing optimization to increase proxy reward while decreasing true quality.The paper argues that scaling a poorly grounded proxy can amplify exploitable directions.
- The thesis: Scaling data and compute without reliable verification can improve imitation and perceptual plausibility but lacks a robust mechanism for ensuring correctness.The paper identifies this absence of correctness feedback as the central obstacle for spatial intelligence.
3. Why Current Spatial Data Fails to Support Scaling
Current spatial world-model progress is constrained by the lack of efficient verifiers for physical and geometric correctness. Fuzzy proxies and costly human judgments leave failures difficult to verify and post-training difficult to scale.
- Spatial world models lack efficient, reliable verifiers for physical and geometric correctness.
- Video generation: Video generation relies on fuzzy proxies and human raters rather than objective feedback for physics, persistence, and perspective coherence.
- Video generation: Distribution-based objectives leave detail misalignment, violated physical constraints, and incoherent long-horizon dynamics unresolved.
- 3D generation: 3D generation faces both limited curated asset data and expensive collection, annotation, and manual checking for high-quality scenes.
- World simulators: World simulators require dense and accurate ground truth for depth, geometry, dynamics, and contact, making verifiable real-world signals expensive to scale.
- The unverifiability tax: The resulting unverifiability tax shifts progress toward more data, compute, scans, and human annotations.
- Game development: Game engines provide grounded checks for physics, collision, navigation, rendering constraints, and bounded playability, while developers judge design fulfillment.
4. Game Development as Human-Engine Verification
Game development supplies a human-engine verification loop around executable scene programs. Engines provide dense structural checks, while developers provide acceptance judgments and development traces that can support reusable post-training data.
- A game project is an executable specification interpreted and partially verified by engines within a broader developer workflow.
- AWoMo design: AWoMo organizes intent, action, verification, and review interfaces around an omni-modal world model.
- Verification: Engine checks cover geometry, collision, physics stability, navigation, script execution, and bounded playability, providing lower-variance structural signals than appearance metrics.
- Rewardable mapping: Games, editors, and modding ecosystems can emit difficulty-graded verifiable spatial tasks whose checks inherit human design intent.
- UWDP: UWDP converts ordinary development into typed multimodal state-action-check-review traces spanning briefs, scene programs, rendered evidence, logs, and reviewer decisions.
- UWDP: The protocol records engine snapshots, rendered evidence, harness checks, typed repairs, iteration, and final reviewer acceptance.
- Dual verification: The engine supplies dense but partial reward, while human review anchors optimization to usefulness and intent.
- Reward ladder: A reward ladder progresses from validity and physical plausibility to functional correctness and playability, with automatic checks reducing human review cost.
5. Experiments on Human-Engine Verification and Generalization
The experiments test human-engine verification, scaling, transfer, and embodied augmentation through controlled UnitySceneBench and diagnostic evaluations. Full RLHEV leads asset classification, source pretraining improves distribution transfer, and AWoMo augmentation improves all reported embodied metrics.
- The validation roadmap uses explicit falsifiers and controlled studies spanning UnitySceneBench, generalization, and embodied diagnostic runs.
- Validation roadmap: The structural-efficiency hypothesis predicts that engine-verifiable rewards outperform fuzzy proxies at matched compute on validity, physics, and playability.
- Validation roadmap: The human-engine hypothesis tests whether adding human acceptance to engine traces improves usefulness, sample efficiency, or robustness over engine-only traces.
- UnitySceneBench: UnitySceneBench evaluates Unity asset classification using multimodal prompts, asset and context features, reference images, and structured layouts.
- UnitySceneBench: Full RLHEV reaches the highest best-of-eight asset-classification performance, beating the strongest non-full baseline by +0.098 primary score and +0.120 accuracy/balanced accuracy.
- Scaling: Figure 5 compares fuzzy proxies, supervised fine-tuning, offline RLHF, engine-based RLVR, and Full RLHEV across training budgets using eight seeds.
- Generalization: Source pretraining improves Unity distribution-shift performance from 0.25 to 0.75 relative to target-only scratch training.
- Generalization: Cross-engine transfer improves Unity-to-Unreal from 0.25 to 0.35 and Unity-to-Godot from 0.15 to 0.35 relative to scratch.
6. Discussion
The pilot studies support human-engine verification as a practical feedback source for agentic world models, with positive results across generalization and embodied diagnostics. Larger scaling studies remain necessary to verify generalization capability.
- Full human-engine feedback achieves the strongest UnitySceneBench asset-classification result, reaching a 0.681 primary score and 0.665 accuracy.The reported breakdown is best-of-eight at full budget; seed-averaged robustness is reported separately.
- Pretraining on source data raises OOD performance from 0.25 to 0.75, while cross-engine transfer improves Unreal from 0.25 to 0.35 and Godot from 0.15 to 0.35.
- The current positive findings are diagnostic rather than definitive, requiring larger scaling studies to verify generalization capability.
7. Related Work
Related work spans neural interactive world models, executable 3D generation, and agentic game-development environments. This paper distinguishes itself by treating development traces as the data engine for training future world builders.
- Neural game engines and interactive video world models: Neural game engines generate action-conditioned interactive rollouts but typically omit explicit state, scripts, collision semantics, and localized failure traces needed for recursive verification.
- Structured and executable 3D generation: Executable 3D-generation benchmarks represent worlds as programs and evaluate code-based, parametric, dynamic, or physics-grounded generation.
- Game development as an agentic verifier and data source: Agentic world-model and game-development systems use interaction, planning, self-revision, multimodal development tasks, runtime assertions, and repair trajectories.
- Novelty: This paper differs by making game development itself the proposed data engine, targeting object-linked edit, check, fix, and review traces rather than only passed games or scores.
- Games as verifiable supervision and open-ended curricula: Games and executable transitions have also served as verifiable training signals for agent and VLM reasoning systems.
8. Conclusion
The paper presents AWoMo as a developer-centered agentic world model embedded in a workflow that builds, verifies, repairs, and reviews executable worlds. Its traces unify generation and understanding supervision, while RLHEV combines engine rewards with human acceptance.
- System: AWoMo is a workflow-embedded system containing a model, agent controller, game-engine verifier, reviewer, and trace store.
- System: Its interfaces receive multimodal intent, emit executable edits, record structural engine checks, and capture reviewer acceptance, rejection, critique, and residual risk.
- Execution loop: The execution loop proposes, renders, verifies, repairs, and reviews executable states, with optional bounded playtests before final review.
- Trace design: UWDP traces preserve intent, stable object identifiers, intermediate states, actions, verifier outputs, rendered evidence, decisions, repairs, costs, and residual risks.
- Training: Stored traces become supervised targets, repair-prediction pairs, and fused RLHEV rewards from engine outputs and reviewer decisions, including rejected traces as negative data.
- RLHEV: RLHEV gates the weighted human-engine reward: engine gates must pass before combining human-review and normalized engine rewards, with optional cost penalties.
- Unified representation: Scene programs make generation a forward map from intent to executable worlds and understanding an inverse map from observations to scene reasoning.
- Scope: Development actions supply shared temporal and human supervision, but generalization beyond games remains an open empirical question.
A.3. Protocol Trace Analysis
Protocol-trace features provide substantially stronger target-engine ranking signals than coarse final-snapshot features under scarce target labels. The analysis treats these results as auditable diagnostic evidence for source-side trace transfer.
- Trace motivation: Final snapshots record terminal outputs, whereas protocol traces retain intent, failed attempts, constraints, verifier history, repair paths, and object-linked process data.
- Protocol: The source-side transfer analysis tests target-engine ranking under a deliberately scarce target-label budget across Unity, Godot, and Unreal splits.
- Representations: Snapshot-only probes use coarse output metadata, while protocol-trace probes add source workflow fields including acceptance, deviation type, target engine, and asset identity.
- Results: 0.719±0.094 Spearman is achieved by protocol traces versus 0.159±0.168 for snapshot-only features; with 720 source instances, the scores are 0.758 ± 0.044 and 0.141 ± 0.084.
- Results: Figure 8 reports held-out Spearman correlations over 8 seeds and two target engines, with additional source instances providing a smaller gain than trace features.
B.2. Full-RLHEV Validation Details
The UnitySceneBench evaluation tests binary acceptance classification for edited Unity assets using human and engine rewards, with full RLHEV evaluated under a controlled best-of-eight protocol. Full RLHEV is strongest in the bounded asset-classification setting, but the result does not establish final game quality or broader preference alignment.
- Task and data: The task predicts whether edited Unity asset candidates are accepted or rejected from multimodal edit, asset, image, and layout inputs.The target label and engine verdict are excluded from the model input; the split contains 720 training, 80 validation, and 200 test examples.
- Rewards and baselines: Full RLHEV combines human and Unity-derived engine rewards with weights 0.65 and 0.35, respectively.Zero-shot CLIP serves as a fuzzy-proxy baseline, while human feedback uses accept/reject labels.
- Evaluation protocol: The learned methods use 720 raw training instances and the same 200-instance test split across eight random seeds.Figure 4 reports the best observed full-budget performance, whereas Figure 5 provides mean±std results across budgets and summarizes seed-level stability.
- Generation protocol: A 720-instance full-RLHEV generation run produced 640 fresh Unity plans without missing render, snapshot, layout, or engine-label artifacts.The generation-side protocol uses the same training budgets rather than a generated-candidate horizontal axis.
- Claim boundary: Full RLHEV is strongest in the best-of-eight full-budget breakdown, gaining +0.098 primary score and +0.120 accuracy/balanced-accuracy over the strongest non-full baseline.The multi-seed scaling curve should be used for seed-averaged robustness; this bounded experiment does not imply final game quality or broader human preference alignment.
B.3. Generalization Experiment Details
The generalization evaluation compares target-only training with source-pretrained target adaptation across Unity distribution shift and Unity-to-Unreal/Godot transfer, alongside embodied-data augmentation. Target adaptation and AWoMo-augmented training improve all reported generalization outcomes.
- Benchmark categories: The benchmark evaluates distribution transfer within Unity and cross-engine transfer from Unity to Unreal and Godot.Distribution transfer uses 5,895 source and 1,254 target examples; each cross-engine split contains 1,000 examples.
- Training conditions: Target-adapted transfer first trains on source data and then adapts on the target training split, with scores measured on the target test split.Target-only scratch trains only on the target-domain split, while zero-shot and source-only conditions are also defined when reported.
- Metrics: The common cross-engine metric is a normalized MLLM-as-a-judge score in [0, 1] because no comparable engine-native scalar exists across Unity, Unreal, and Godot.The rubric judges usability, engine readiness, and task alignment using the accept/reject convention.
- Distribution transfer: Target adaptation raises the Unity distribution-shift score from 0.25 to 0.75.Distribution transfer is reported as the strongest positive case.
- Cross-engine transfer: Target-adapted transfer improves Unity-to-Unreal from 0.25 to 0.35 and Unity-to-Godot from 0.15 to 0.35 relative to scratch.Proxy scores and losses also improve for both target engines.
- Embodied generalization: AWoMo-augmented training improves over the original baseline on R2R, Gymnasium MuJoCo, and D4RL Gym-MuJoCo.The reported metrics are success rate, rollout return, and D4RL normalized score, respectively.
- Embodied results: +48.43% is the largest relative gain, on D4RL Gym-MuJoCo; Gymnasium MuJoCo improves +9.96% and R2R +0.79%.These are direction-normalized relative improvements over the original baseline.
C.1. Limitation Discussion and Future Analysis
The discussion presents game development as a verifier-rich substrate while bounding the evidence: current experiments do not establish sim-to-real transfer, complete game quality, human-subject validity, or closed-loop embodied deployment. It proposes stronger verification, playable-artifact studies, consented human evaluation, and recursive game-building loops as next steps.
- Sim-to-real gap: Current experiments contain no real scans, real robots, or genuine real-to-sim-to-real loop, so engine rewards are not evidence of physical-world transfer.The proposed next test uses real-world environments, limited fine-tuning, and measurement of which executable checks remain useful.
- Video-generation objection: Video generation remains imitation- and compute-bound, with fuzzy evaluation and no self-improvement loop for physical or long-horizon correctness.The discussion distinguishes progress in plausibility from progress in correctness.
- Verifier gaming: Engine rewards are partial verifiers that can be gamed when loose collision, navigation, or budget checks are optimized alone.The proposed safeguards are verifier ensembles, randomized probes, held-out checks, and human review.
- Data versus verification: Cheaper real-world 3D data would not itself provide a verifier: scans show an instance but cannot determine whether a synthesized output is correct.The paper assigns the missing automatic check to the game engine rather than passive capture.
- Engine dependence: Cross-engine transfer is constrained by different import formats, collision semantics, navigation systems, and runtime constraints, making target calibration necessary.The study reports positive Unity-to-Unreal and Unity-to-Godot signal after target adaptation.
- Validation boundary: The evidence is bounded: results do not yet prove complete game quality, human-subject validity, or closed-loop embodied deployment.The next validation step is to implement playable artifacts and conduct consented human studies.
- Future recursive loop: The proposed long-term loop has game agents play executable worlds, identify broken rules and failures, and convert those failures into training signal.Trustworthiness requires executable specifications, hard-to-fool playtest agents, and continued human review of design intent.
- Broader framing: Recursive improvement is framed as an ecosystem property arising from human practice, executable environments, and learning systems, with broader-domain scaling left open.The paper explicitly leaves extension beyond game development to engineering, science, and physical-world interaction as an open research question.