Code2Games: Enabling Coding Agents for Gaming World Generation

1 INTRODUCTION

• Existing systems can generate assets, scenes, scripts, or engine components independently, yet these may conflict with the intended setting, entity roles, or objective locations.A coherent world must jointly combine a structured environment, spatially consistent objects, interactive entities, objectives, and executable rules.Source: §1 · p. 2
• Code2Games uses scene analysis, gameplay planning, and constrained world generation agents coordinated through a shared representation that maintains element correspondence.The framework starts from a Blender base scene generated from the same natural-language intent.Source: §1 · pp. 2–3
• After generation, an engine-customization agent reconstructs actors, configuration, and gameplay logic, while execution feedback helps locate and repair inconsistencies.The feedback includes compiler diagnostics, runtime feedback, and gameplay outcomes; Unreal Engine 5 is the implementation backend.Source: §1 · pp. 2–3
• GameCode4D comprises ten fixed game prompts spanning scene and gameplay complexity, with evaluation across visual quality, interactive fidelity, multimodal artifacts, and playable-game quality.The evaluation dimensions cover appearance, consistency with the specified setting and interactions, artifact quality, and realization of gameplay after engine adaptation and execution.Source: §1 · p. 3

Interactive video and game world models

• Interactive video and game-world models generate action-conditioned observations, whereas Code2Games targets a runnable game project with explicit assets, identities, and executable rules.The cited work evaluates interactive observations through measures including physical plausibility, instruction following, temporal consistency, and action responsiveness.Source: §2 · p. 3

Language-guided 3D world generation

• Language-guided 3D methods support scene generation and planning, but Code2Games grounds objectives, actors, and interactions in scene regions and connects Blender objects to Unreal elements.The passage characterizes prior approaches as stopping at geometry, appearance, or simulation configuration.Source: §2 · p. 3

Agentic game development

• Unlike agentic game-development systems conditioned on an intent alone, Code2Games also uses a paired base world and applies execution feedback to repair affected components.Persistent correspondence between world representation and engine implementation makes repairs local rather than requiring project-wide regeneration.Source: §2 · p. 4

3 METHOD

• Direct Unreal-project generation is difficult because scene constraints, gameplay semantics, and engine code use different forms, so binding or placement errors can propagate into compilation or gameplay failures.Code2Games addresses this with a shared representation, persistent element identifiers, and iterative repair driven by engine diagnostics and gameplay tests.Source: §3 · p. 4

3.1 SCENE AND GAMEPLAY REPRESENTATION

• The shared representation records scene constraints, gameplay objectives, actors, elements, interactions, and each element’s correspondence across scene, engine, and executable logic.Region descriptions include semantic category, spatial boundary, accessibility, and interaction affordances; interactions specify participants, trigger, precondition, and state update.Source: §3.1 · pp. 4–5
• Persistent identifiers connect gameplay elements to their scene and Unreal realizations, interactions, and logic units, allowing affected components to be updated without regenerating the entire project.The correspondence supports propagation of placement changes, asset replacements, and logic repairs.Source: §3.1 · p. 5

3.2 GAME REALIZATION WITH SCENE CONSTRAINTS

• The selected location is instantiated as a Blender object or annotated gameplay region, while each declarative interaction is translated into executable logic bound to its participants.The logic detects the trigger, evaluates its precondition, and performs the specified game-state update.Source: §3.2 · pp. 5–6
• Gameplay elements are placed in regions compatible with their type and affordances, then candidate locations are checked geometrically and ranked for semantic suitability.Checks include surface support, collisions, and reachability; a vision-language model ranks feasible candidates, and an empty candidate set is reported for repair.Source: §3.2 · p. 6

3.3 UNREAL PROJECT RECONSTRUCTION

• Unreal reconstruction restores scene objects, visual settings, gameplay volumes, and logic bindings while preserving the spatial organization and element correspondences.The initial project assembles reconstructed actors and volumes, restores visual and collision configurations, and registers executable logic units.Source: §3.3 · pp. 6–7
• The project is compiled and executed; diagnostics and gameplay failures are localized through correspondence records, affected components are updated, and execution repeats until objectives pass or the repair budget is exhausted.Gameplay failures come from objective assertions and scripted interaction traces derived from the objective and interaction specification.Source: §3.3 · p. 7

4.1 BENCHMARK AND METRICS

• GameCode4D uses ten fixed prompts spanning scene and gameplay complexity, with each prompt specifying genre, viewpoint, and a high-level objective.The benchmark evaluates visual quality, interactive fidelity, multimodal artifact quality, and playable-game quality.Source: §4.1 · p. 7
• Two approximately 50-second rollouts per game yield 20 videos per method; Build measures autonomous engine-check pass rate without human intervention, compilation failure, or runtime crash.Scores are averaged across rollouts and games.Source: §4.1 · pp. 7–8

4.2 MAIN RESULTS

• Across matched backbones, Code2Games improves gaming-world quality; Claude Opus 5 leads 12 of 15 metrics and reaches 97.7% build reliability.The largest gains over the strongest baseline are +15.5 for art and presentation, +15.0 for functional visuals, +13.8 for content depth, and +13.3 for Human.Source: Table 1 · p. 8, §4.2 · pp. 8–9

4.3 ABLATION STUDIES

• Removing any scene-grounding component degrades interactive fidelity and playable-game quality; representation removal mainly reduces Setting, gameplay-specification removal most affects Inter., and constraint removal most affects Physics.Removing scene constraints changes Physics from 73.9 to 66.2; removing gameplay specification changes Inter. from 75.8 to 69.8.Source: §4.3 · p. 9, Table 2 · p. 9
• Progressively adding engine-feedback and gameplay-feedback repair raises Build from 78.5 to 97.7 and Inter. from 58.0 to 75.8.Engine feedback mainly resolves engine-level failures, while gameplay feedback further improves interactive behavior correctness.Source: §4.3 · p. 9, Table 3 · p. 9

5 CONCLUSION

• Code2Games transforms a game intent and paired Code2Worlds scene into a gaming world adapted to Unreal Engine 5.A shared scene-and-gameplay representation, persistent identifiers, and execution-guided repair connect generation stages.Source: §5 · p. 10
• On GameCode4D, Code2Games improves visual quality, interactive fidelity, multimodal artifact quality, and playable-game quality over matched coding agents across multiple backbones.Ablations report complementary contributions from structured game design, scene-aware constraints, and execution-guided reflection, with reflection also improving build reliability.Source: §5 · p. 10
• The results support grounding gameplay generation in explicit scene structure and closing the generation loop with engine feedback as an effective route from generated worlds to playable games.Source: §5 · p. 10

A LIMITATION AND FUTURE WORK

• Unreal reconstruction may not fully preserve the visual appearance of the source Blender scene.The gap can affect color, surface detail, transparency, and atmosphere.Source: A LIMITATION AND FUTURE WORK · p. 14
• Differences in material representations, procedural shader nodes, lighting, and rendering can alter reconstructed-scene appearance.Source: A LIMITATION AND FUTURE WORK · p. 14
• Future work will investigate improved material translation and rendering calibration to reduce these discrepancies.Source: A LIMITATION AND FUTURE WORK · p. 14

B ABLATION STUDY

• The main paper reports component-wise ablations of scene representation, gameplay specification, scene constraints, and execution-guided repair.Source: B ABLATION STUDY · p. 14
• The supplementary material avoids duplicating those main-paper ablation results.Source: B ABLATION STUDY · p. 14
• Table 4 provides a finer-grained decomposition of scene-constrained game realization.Source: B ABLATION STUDY · p. 14

ADDITIONAL ABLATION RESULTS

• All variants use the same backbone model, game-intent prompts, assets, and evaluation protocol as the full Code2Games framework.Source: ADDITIONAL ABLATION RESULTS · p. 14
• Region compatibility restricts gameplay elements to semantically appropriate scene regions, while geometric feasibility checks surface support, collisions, and accessibility.Source: ADDITIONAL ABLATION RESULTS · p. 14
• M improves from 58.0 to 68.8, D from 45.0 to 56.7, V from 52.0 to 61.0, and A from 45.0 to 57.0 with the full constraint combination.The full variant combines both constraint types with scene-constrained selection.Source: Table 4 · p. 14, ADDITIONAL ABLATION RESULTS · p. 14

C IMPLEMENTATION DETAILS

• Qwen-family models handle language and visual reasoning, while Hunyuan3D generates gameplay assets.Source: C IMPLEMENTATION DETAILS · p. 14
• Scene understanding, gameplay-element placement, and visual inspection are performed in Blender 4.2.0 through its Python API.Source: C IMPLEMENTATION DETAILS · p. 14
• The resulting gaming worlds are reconstructed as Unreal Engine 5 projects for executable gameplay.Source: C IMPLEMENTATION DETAILS · p. 15

• The benchmark spans reconnaissance, combat, monster hunting, relic running, racing, skiing, underwater exploration, flight, and resource collection.Source: D BENCHMARK DETAILS · p. 15
• Each prompt specifies an environment, player role, objective, and required interactions.Source: D BENCHMARK DETAILS · p. 15
• Several scenarios include variants with different environments, weather conditions, or routes.Source: D BENCHMARK DETAILS · p. 15

• The prompt examples cover combat, exploration, traversal, racing, underwater exploration, flight, and resource collection.Table 5's IDs match the qualitative figure sequence.Source: BENCHMARK PROMPTS · p. 15, Table 5 · p. 15
• Marine prompts require collecting samples and investigating or scanning marked sites, with one adding sonar monitoring and extraction after completion.Source: BENCHMARK PROMPTS · p. 16
• Flight prompts specify following a corridor, activating four cells, and locking a summit mast.One prompt places the mission over foggy mountains; another specifies an F-104 flight at dawn.Source: BENCHMARK PROMPTS · p. 16
• Archer prompts range from scouting and taking cover to positioning in sparse terrain before eliminating targets with a bow.Source: BENCHMARK PROMPTS · p. 16
• Resource prompts set collection objectives in a red desert canyon or across rocky clearings and forest terrain.The canyon challenge also requires exploring markers and avoiding hazards; the expedition spans multiple regions.Source: BENCHMARK PROMPTS · p. 16

• The protocol covers gameplay rollout recording, automatic metric computation, LLM-as-a-Judge evaluation, and a human user study.Source: E EXPERIMENTAL PROTOCOLS · p. 16
• Identical protocol settings are applied to all compared methods unless otherwise specified.Source: E EXPERIMENTAL PROTOCOLS · p. 16
• Detailed metric definitions, scoring procedures, and evaluation configurations are provided in the supplementary material.Source: E EXPERIMENTAL PROTOCOLS · p. 16

Recording and Aggregation Protocol

• Each game contributes two distinct approximately 50-second rollouts, scored independently, averaged per game, then equally across ten benchmark games.Routes cover different regions and interactions, including an initial state, navigation, a core interaction, a visible state transition, and a goal or progress signal; all methods share recording settings and controller budgets, yielding 20 videos per method.Source: RECORDING AND AGGREGATION PROTOCOL · p. 16

Evaluation Dimensions

• Visual quality uses VBench-derived image quality, dynamic degree, and motion smoothness metrics; interactive fidelity judges setting adherence, interaction adherence, and physics compliance.Image quality uses MUSIQ, motion is measured with RAFT optical flow at 8 fps, and smoothness uses AMT frame interpolation; interactive criteria are judged from observable rollout evidence.Source: METRIC IMPLEMENTATION · pp. 16–17

Multimodal Artifact Quality

• Multimodal artifact quality evaluates code, images, and video separately, with four sub-dimensions scored 0–25 each (0–100 per modality), using Qwen3.8-Max at temperature 0.Code uses source files and the game description; image uses screenshots and the description; video uses the gameplay recording and the description.Source: METRIC IMPLEMENTATION · p. 17

Playable-Game Quality

• Playable-game quality weights content depth and art/presentation most heavily, with its 0–100 weighted rubric multiplied by build reliability.The score is Build/100×(0.15M + 0.35D + 0.15V + 0.35A); Build averages autonomous pass rates for compilation, launch, level loading, and route execution, and projects reached executable state within at most five repair iterations.Source: METRIC IMPLEMENTATION · p. 17

Scoring and Judge Controls

• LLM judges use Qwen3.8-Max at temperature 0 across methods and score only evidence observable in the supplied inputs.Playable-game scoring uses uniform criterion-level rubrics with calibration and caps: absent or broken features score 0, functional but placeholder-level features score 50, and publishable demonstrated features score 100; visible player actions must produce persistent state changes for a 100.Source: LLM-AS-A-JUDGE PROTOCOL · p. 17, PLAYABLE-GAME QUALITY PROMPTS · p. 21

Core mechanics

• Mechanics scoring rewards a clearly demonstrated player-driven loop linking action, persistent state change, consequence, and feedback.Mechanics shown only in scripted sequences or without observable persistent state change are capped at 50; a clearly demonstrated full loop can score 100.Source: PLAYABLE-GAME QUALITY PROMPTS · p. 21

Content depth

• Content-depth scoring assesses meaningful objectives, progression, and interactive content beyond one loop, including multiple visually and functionally distinct content types, visible progress indicators, pursued objectives, and late-game escalation.Source: PLAYABLE-GAME QUALITY PROMPTS · p. 22

Functional visuals

• Functional-visual scoring checks readable HUD elements, identifiable interactive objects, visible action feedback, and the absence of interface or rendering issues.A score of 100 requires players to understand resources, current state, and next action without guessing.Source: PLAYABLE-GAME QUALITY PROMPTS · p. 22

Art and presentation

• Art-and-presentation scoring evaluates coherent assets and lighting, deliberate scene composition, and the absence of placeholders or debug-like content.A score of 100 requires the game to read as a publishable vertical slice with no obvious placeholder elements.Source: PLAYABLE-GAME QUALITY PROMPTS · pp. 22–23

Human evaluation

• The human study used 15 raters, randomized gameplay videos, concealed method identities, and scores across four playable-game dimensions aggregated over participants and benchmark games.The two recordings from each generated game were placed in different rating blocks to reduce direct-comparison and ordering effects.Source: USER-STUDY PROTOCOL · p. 23, Table 6 · p. 23

F ADDITIONAL QUALITATIVE RESULTS

• Representative gameplay frames span Figures 6–25 and diverse GameCode4D scenarios.The examples cover FPS, monster-hunt, racing, skiing, temple-run, third-person shooter, underwater exploration, wingsuit, Assassin’s Creed-style, and Minecraft-style gameplay.Source: F ADDITIONAL QUALITATIVE RESULTS · p. 23, Figure 6 · p. 24, Figure 8 · p. 24

G SYSTEM PROMPT DESIGN

• The supplementary material provides complete Code2Games system prompts to clarify stage responsibilities and support reproducibility.The prompts cover scene understanding, gameplay design, route planning and repair, scene-constrained candidate selection, asset realization, reference-image generation, and LLM-based evaluation.Source: G SYSTEM PROMPT DESIGN · p. 30

SYSTEM PROMPTS

• The prompts separate objective visual scene understanding and rule-relevant visual reading from gameplay-rule design, restricting each stage to its assigned outputs.The visual readers describe visible content or rule-relevant visual notes without designing game rules, routes, coordinates, or code; gameplay planning uses text inputs and specifies a task-based free-navigation challenge.Source: OBJECTIVE SCENE ANALYSIS · p. 31, RULE-RELEVANT VISUAL READING · p. 32, DECLARATIVE GAMEPLAY PLANNING · p. 33
• Route planning analyzes aligned RGB, object-density, and geometry images, then requires a continuous, non-repeating path through edge-adjacent cells on a 16×16 grid.The prompt also directs the planner to compare the visual and geometry evidence when identifying scenic route regions.Source: TOP-DOWN VISUAL REGION AND ROUTE… · p. 35
• Gameplay elements are placed in scenic interior clearings and distributed across the scene rather than grouped in an isolated open field.The prompts treat nearby vegetation as composition context, while requiring clearance for new assets and preventing accidental prop piles.Source: SCENE-CONSTRAINED CANDIDATE SELECTION · p. 38, PHYSICALLY CONSTRAINED CANDIDATE… · p. 40
• Placements must satisfy candidate-specific physical support, walkability, and clearance constraints; deterministic staging rejects unsupported or overlapping layouts rather than silently moving assets.Grounded assets require suitable support for their footprint, and the selector must use physically supported placement modes.Source: PHYSICALLY CONSTRAINED CANDIDATE… · p. 40
• The asset-generation plan keeps placements fixed and assigns visible GLBs to specified gameplay objects while reserving logical anchors for later gameplay systems.Obstacles must be newly generated, self-contained, physically plausible objects; existing scene vegetation cannot fulfill an obstacle placement.Source: ASSET REALIZATION · p. 41
• Props must suit gameplay roles and fixed placements, obey gravity and surface orientation, and match local scene materials, style, scale, and clearance.Scene context informs prop composition and styling but is not itself an asset to reuse; spawn actors and vehicles are implemented as non-GLB placements.Source: ASSET REALIZATION · p. 41
• Assets use Blender [X width, Y depth, Z height] dimensions, declare a justified flat-base policy, and default to unique props unless reuse is physically suitable.For demos with ten or more visible or optionally visible placements, the prompt requires at least ten distinct generated assets; airborne and ground-supported variants should be split when needed.Source: ASSET REALIZATION · pp. 41–42
• Reference images must show one volumetric standalone prop against a uniform white background, without environment, shadows, support geometry, people, text, or UI.The object should be fully visible in an elevated three-quarter product view with empty margin and no invented support beneath airborne props.Source: PER-ASSET REFERENCE-IMAGE GENERATION · p. 42