PaperlayerView exact revision on arXiv

Source-linked AI summary

AI for Games in the Foundation Model Era

Meng Luo, Yanlin Li, Hao Li, Hongzhan Lin, Pengfei Zhou, Tianjie Ju, Ran Zhang, Yeying Jin, Mong-Li Lee, Wynne Hsu

arXiv:2609.16679v1cs.AI

TL;DR

AI for games now spans six lifecycle roles, but it remains unclear which capabilities transfer across games, engines, interfaces, and player populations. This paper synthesizes the roles and their cross-role exchanges, finding that reuse is possible but effectiveness remains conditioned on the target setting and requires renewed validation.

  • Problem

    Research across six AI-for-games roles has evolved separately, leaving evidence about capability transfer across settings and roles limited.

    Source: §1 · p. 5, §12 · p. 80

  • Method

    The paper surveys six roles, compares methods and evidence under operating conditions, and synthesizes exchanges among trajectories, environments, specifications, feedback, and player models.

    Source: §1 · p. 5, §10.3 · p. 75

  • Results

    Across roles, reuse of artifacts and capabilities remains task-dependent, with stronger shared benchmarks for bounded play than for sustained creation, adaptation, and human experience.

    Source: §1 · p. 5, §12 · p. 80

  • Takeaways & Limitations

    Cross-role reuse can support new workflows, but performance must be re-established in the game, engine, interface, player population, and task where systems are evaluated.

    Source: §12 · p. 80

    Transfer evidence requires held-out tasks, rule-sensitive interventions, reference transitions, or sustained play under stated budgets rather than generated demonstrations alone.

    Source: §9.3.1 · p. 69

Abstract

from arXiv · show

Foundation models, alongside advances in learned game-world models, are reshaping AI across the game lifecycle. Beyond playing games, recent systems model players and game dynamics, support design and development, adapt player-facing experiences at runtime, and evaluate resulting artifacts. Yet these directions have evolved largely separately, obscuring which capabilities transfer across settings and which remain tied to particular games, engines, interfaces, or player populations. We organize the literature into six roles according to the immediate use of AI output: playing and acting; modeling players and games; designing games; building and maintaining games; generating and adapting at runtime; and testing and evaluating games. For each role, we examine what structure is supplied by the game or workflow, what AI learns or produces, which capabilities and artifacts transfer across settings and roles, and what evidence supports the claims. We identify cross-role connections: trajectories train world models, learned environments provide experience for agents, design specifications drive executable implementations, and play or testing feedback guides revision. However, control schemes, rules, engine interfaces, state representations, and player contexts often remain setting-specific, so downstream claims require validation in the target setting. Evaluation is most standardized for bounded game playing and selected learned environments, while persistent state in learned worlds, repeated software revision, validated player modeling, sustained runtime adaptation, and representative automated testing remain less established. The central challenge is to reuse or transfer outputs and capabilities across roles while re-establishing evidence for effectiveness in the game-specific contexts where they are used.

1 Introduction

The survey organizes AI for games around six immediate uses of AI output and connects them through transfers, feedback, and evidence. It shows that broad pretraining expands available interfaces but does not remove game-specific structure or uneven support for sustained creation, adaptation, and human experience.

1 Introduction

  • Earlier research developed largely separate traditions for playing, generation, narrative, design assistance, player modeling, and automated playtesting, motivating an integrated comparison.Existing surveys provide foundations for comparing methods within individual areas, but this survey examines their relationships across the lifecycle.Source: §1 · pp. 3–4
  • Foundation models connect established tasks through natural-language design, tool use, rendered-game inspection, constrained interaction, and learned environments that support agent training.These capabilities let systems propose and revise implementations, mediate player commands, and generate imagined experience for policy learning.Source: §1 · p. 3
  • The survey unifies six roles—acting, modeling, design, building, runtime adaptation, and testing—while distinguishing proposals, implementations, player adaptations, and evaluation evidence.This framework compares what structure remains supplied, what capabilities transfer, and how outputs are supported at their point of use.Source: §1 · p. 4, Figure 2 · p. 4
  • Gameplay artifacts can flow across roles as simulator data, learned-environment experience, diagnostic evidence, or repair guidance, but their value depends on the receiving task and its game-specific context.Action semantics, state, requirements, and intended-player behavior may need to accompany reused outputs.Source: §1 · p. 5
  • The survey finds that pretraining broadens interfaces without eliminating game-specific structure, while gameplay increasingly provides data and feedback beyond an agent’s final score.Evidence is stronger for bounded game playing than for sustained creation, runtime adaptation, and human experience.Source: §1 · p. 5

2 Roles Across the Game Lifecycle

The section organizes AI-for-games research into six roles based on the immediate use of AI output, using the foundation-model era as an analytical lens rather than an inclusion criterion. It distinguishes supplied structure, transferable capabilities and reusable artifacts, and evidence at the point of use while assigning systems primary and secondary roles.

2 Roles Across the Game Lifecycle

  • The taxonomy covers playing and acting, modeling players and games, designing games, building and maintaining games, runtime generation and adaptation, and testing and evaluating games.Roles classify outputs by use rather than architecture, and systems may serve multiple roles across lifecycle phases.Source: §2 · pp. 5–6
  • The analysis asks what structure the game or workflow supplies, what AI contributes, what transfers or can be reused, and what evidence supports effectiveness in the target setting.Transfer requires demonstrated competence under changed games, engines, interfaces, player populations, or tasks; sharing backbones or artifacts alone is insufficient.Source: §2 · p. 5

2.1 Assigning Primary and Secondary Roles

  • Systems receive a primary role according to the immediate use of their output and the claim on which the principal empirical evidence rests, with secondary roles recorded separately.Actions map to Play and Act, behavioral predictions to Model Players and Games, evaluated content proposals to Design, executable artifacts to Build and Maintain, live experience effects to Generate and Adapt at Runtime, and test traces or judgments to Test and Evaluate.Source: §2.1 · p. 6
  • Examples clarify role boundaries: MarioGPT is Design, Play2Code is Build and Maintain, and a playtester is Test and Evaluate when the claim concerns coverage or defects.An NPC policy remains in Play and Act when evaluated for action or coordination, whereas session-specific dialogue or behavior belongs to the runtime role when assessed through live player effects.Source: §2.1 · p. 6

3 AI That Plays and Acts

This section examines AI agents that play and act, spanning game-specific specialists, shared policies, hierarchical planners, adaptive memories, and socially coordinated agents. It focuses on how transfer depends on control interfaces, supplied information, adaptation, interaction budgets, coordination demands, and evidence across game-specific changes.

3.1 Player and Generalist Agents

  • Specialist performance and cross-game transfer are distinct: reusing algorithms, parameters, or interfaces tests different forms of generalization.Specialists exploit fixed observations, controls, objectives, and protocols, whereas transfer may involve new layouts, modes, rules, action meanings, or entire games.Source: §3.1 · p. 10, Algorithm reuse · pp. 11–12

Interface constraints.

  • Transfer claims are bounded by the interface: visual encoders may generalize across appearance, but action grammars, controls, objectives, timing, and supplied affordances remain game-specific.Semantic APIs simplify motor control by exposing legal actions or routines, while native-control agents must discover and execute those affordances; evaluation interfaces therefore support different claims.Source: Interface constraints · p. 12

3.2 Learning and Control Hierarchies

  • Hierarchical agents divide long-horizon play between planning and motor execution, linking language goals, executable skills, visual targets, or learned behavior representations.Plan validity does not guarantee grounding: an agent may know a dependency without recognizing the relevant object, reaching it, or verifying completion.Source: Language-based planning · p. 13

Controller training.

  • Controller training combines reinforcement learning, self-play, imitation, behavior tokens, and planner–controller interfaces, with performance depending on reward access, demonstrations, temporal memory, and action latency.ROCKET-1 passes tracked visual regions to a low-level policy, while asynchronous planning and reactive control address different timescales; demonstrations broaden purposeful coverage but inherit their quality and labeling limits.Source: Controller training · pp. 13–14

3.3 Test-Time Adaptation and Memory

  • Test-time adaptation reuses demonstrations, interaction, and memory, but its effectiveness depends on what information is supplied and whether retrieval or summaries preserve the task-relevant state.REGENT adapts in context from held-out-game demonstrations, while S3Gym finds that summaries help some tasks and detailed trajectories help precise state-dependent tasks.Source: §3.3 · p. 14

Dynamics identification.

  • Dynamics-identification systems adapt by retrieving demonstrations, testing executable hypotheses, or translating rules into simulators, while episodic memories and skills preserve experience but can become stale or miss rule changes.Optimus-1 combines a knowledge graph, compressed multimodal episodes, reflection, and a STEVE-1 controller; these experiments concern reuse with a particular controller, not context length alone.Source: Dynamics identification · pp. 15–16

Self-play, league, and cooperative training.

Zero-shot coordination and human–AI teamwork.

  • Zero-shot teamwork and communication are protocol-dependent: coordination may rely on partner inference, explicit task allocation, language actions, or role-specific reasoning, and evaluation must distinguish human compatibility from trained-agent coordination.ProAgent revises inferred partner intentions, MindAgent and VillagerAgent expose more of the joint plan, and dialogue games vary in who can speak, visible history, and how speech maps to legal actions.Source: Zero-shot coordination and human–AI… · p. 17, Communication and social deduction games · p. 18

Communication and social deduction games.

4 AI That Models Players and Games

This section examines AI systems that model players and games through decision-relevant dynamics, persistent state, behavioral patterns, and subjective experience, including generative simulators with action interfaces and real-time rendering. It considers their downstream usefulness and validation, highlighting progress alongside limits in transfer, representation and data coverage, psychological interpretation, persistent state, latency, and long-horizon consistency.

Planning models.

  • Model objectives must match use: planners need action-relevant distinctions, policy learners need reliable changing-policy rollouts, and players need coherent generated observations.Observation reconstruction can preserve texture while missing action-changing transitions, whereas search models may ignore texture; planning quality therefore depends on coupling model objectives to the decision procedure.Source: §4.1 · p. 19, §4 · p. 19, Planning models · p. 20

Learned training environments.

  • 29% of Dreamer 4 evaluation episodes obtained an iron pickaxe and 0.7% obtained diamonds after learning Minecraft behavior from 2,500 hours of offline action-labeled data, exposing difficulty in long-horizon task completion.The result used 60-minute episodes, native mouse/keyboard control, and prescribed task prompts; online model learning, offline policy learning, and deployment in the reference game should be distinguished.Source: Learned training environments · p. 20
  • Interactive learned simulators increasingly support direct control, but evidence commonly covers visual quality and responsiveness rather than persistent mechanics or state fidelity.GameNGen, Genie, GameGen-X, Hunyuan-GameCraft-2, PlayGen, and MineWorld demonstrate game- or domain-specific interaction, while transfer to unfamiliar rules remains unresolved.Source: Learned training environments · pp. 20–21

State representation.

  • Representation, uncertainty, and memory determine which visual, mechanical, and persistent-state regularities survive interaction, and visual diversity alone does not establish rule uncertainty or calibration.Memory systems use compression, retrieval, or geometry-based state; ReWorld reports visual recall over 64-second out-and-back trajectories and 11.95° overall rotation error, but numerical mechanics and inventory persistence require separate tests.Source: §4.2 · p. 22, State representation · p. 22, Aleatoric and epistemic uncertainty · pp. 23–24

Aleatoric and epistemic uncertainty.

  • StatePlay reported normalized state error below 0.06 and an 18.6-point gain over the strongest stateless baseline on mechanics-fidelity judgments in Street Fighter III.WorldMind was preferred to baselines in approximately 70% of one-minute Game A pairwise comparisons for tactical appropriateness and coherence; these results support specific components, not complete game-state correctness.Source: Aleatoric and epistemic uncertainty · p. 24

State supervision from instrumented games.

  • Instrumented games provide precise aligned supervision for observations, actions, geometry, and selected state variables, but access does not automatically reveal health, inventory, quest progress, or other hidden variables.WildWorld aligns RGB, depth, poses, skeletons, actions, and state annotations; Black Myth: Wukong data includes more than 90 hours of frame-aligned inputs, engine states, and observations.Source: State supervision from instrumented… · pp. 24–25

4.3 Player Models and Behavioral Inference

  • Player models must distinguish next-action prediction, skill or style inference, preference estimation, affect modeling, and intervention outcomes because each requires different observations and validation.Behavioral embeddings depend on event encoding and correspondence across games; inferred profiles need calibration and temporal stability, while subjective experience requires dedicated signals such as affect annotations rather than behavioral proxies alone.Source: §4.3 · p. 25, Behavioral representation learning · p. 25, Play-style, skill, and preference… · p. 26

Human action prediction and human-likeness.

  • Maia4All increased held-out individual move accuracy from about 51.4% for Maia-2 to 53.2% using as few as 800 positions, approximately 20 games.This supports more accurate individualized action prediction within chess without requiring an embedding to represent a named personality trait.Source: Human action prediction and… · p. 26

Play-style, skill, and preference profiles

  • Player models predict engagement, affect, actions, and judgments, but zero-shot foundation-model engagement prediction is often weak and stronger optimal-play reasoning does not ensure faithful human-judgment modeling.Multimodal systems reach up to 72% average and 88% best engagement accuracy in one commercial game, whereas three vision–language models across nine shooters often fail to beat majority baselines; judgment agreement varies non-monotonically across 121 strategy games.Source: Play-style, skill, and preference… · pp. 27–28

4.4 Action Interfaces, Real-Time Generation, and Long-Horizon Consistency

  • Action-conditioned and language-controlled simulators improve controllability, but their evaluations target specific interfaces, genres, or short clips rather than general game mechanics.SCOPE provides cross-game action-transfer evidence within first-person shooters, while Incantation reports 89% versus 43% action-control accuracy and 90% versus 0% on out-of-vocabulary probes; H3-World’s 124-frame generation is not evidence of real-time streaming or long-horizon mechanics.Source: §4.4 · p. 29

Real-time generation and serving

  • Autoregressive and diffusion components can be combined for sequential generation, yet action semantics, persistent state, and latency remain open problems because generated observations feed back into later predictions.Diffusion may produce each observation while autoregression continues the rollout, with few-step generation, distillation, and streaming reducing latency; chunking can delay new commands and feedback accumulates errors.Source: Real-time generation and serving · p. 30
  • Real-time neural rendering can achieve playable frame rates when an engine supplies game rules, but reported speed depends on serving design and does not establish learned-rule transfer.AlayaRenderer-Flash reports 31.54 fps on one H200 and 30 fps in live SuperTuxKart after target-game fine-tuning; frame rate should be interpreted with latency, resolution, hardware, and chunk-refresh policy.Source: Real-time generation and serving · p. 30

Long-rollout consistency

  • Long rollouts accumulate visual and state errors that can undermine navigation, trust, action rankings, and policy transfer, while existing visual-drift mitigations do not establish correct game state.CausVid, Self Forcing, and FramePack reduce or address visual drift, but none reports game-state results; synchronized multiplayer worlds additionally require validation of identity, geometry, attribution, outcomes, and reference transitions.Source: Long-rollout consistency · pp. 30–31
  • Useful representations and player predictions are consumer- and task-dependent: visual memory does not replace mechanics state, and better prediction improves player experience only when it changes a useful design decision.MuZero, interactive simulators, ReWorld, StatePlay, Maia4All, and personalized PCG illustrate that planning, rendering, state tracking, player prediction, and adaptation require different evidence.Source: Long-rollout consistency · p. 31

5 AI That Designs Games

AI game design combines foundation-model proposals with representations, constraints, search, simulation, and human revision across content, mechanics, and narrative. The central challenge is preserving playability, executable behavior, and designer control while validating outputs in game-specific settings.

5.1 Assets, Levels, and Worlds

  • Foundation-model design systems broaden language and multimodal control, but playability, narrative coherence, structural validity, and execution require distinct representations and evaluation procedures.Constraint solving, search, learned generators, repair, and human selection provide different sources of validity and variation across assets, levels, and worlds.Source: §5 · p. 31, §5.1 · pp. 32–33
  • MarioGPT solved 88.4% of 250 generated levels with an A* agent, while separate experiments measured prompt adherence, showing that traversability and design control are different properties.Word2Minecraft preserved objective reachability through A* and BFS checks, but its evolutionary baseline scored better on automated playability and path measures while leaving story coherence unmeasured.Source: §5.1 · pp. 32–33
  • Sokoban results show that pretraining offers limited benefit with abundant game data, novelty is difficult with small human-authored datasets, and solution-length control requires solving the puzzle.These findings distinguish learning spatial regularities from learning the conditions that make a level function.Source: §5.1 · p. 33, Granularity of creator control · p. 38

Functional assets and compositional worlds.

Conversational editing and generator control.

  • 95.47% of 6,000 generation attempts were validated after repair in Unity, while the pipeline’s curated libraries and repair process do not establish unassisted language-model generation.The same study imported layouts into Left 4 Dead 2, but its result remains specific to one survival-horror setting.Source: Conversational editing and generator… · p. 34

5.2 Rules and Mechanics

  • Rule and mechanic generators expand the design space through evolutionary search, language models, code mutation, composition, and planning, but executable representations and simulated play determine what can be tested.Fixed grammars improve tractability while flexible code broadens behavior at the cost of harder validation; joint mechanic–level search captures interactions between rules and layouts.Source: Automated game design · p. 35, Language-model rule and mechanic… · p. 35

Checking designs through implementation and play.

  • Compilation, solver checks, and simulated play select different properties, and Mortar’s 14-participant study agreed with agent-based ordering in two game pairs but reversed it in the third.Selection quality therefore depends on the evaluator, objectives, and proposal, repair, and simulation budgets.Source: Checking designs through… · p. 36, Granularity of creator control · p. 38

5.3 Narrative and Co-Creative Design

  • Narrative and co-creative systems preserve author control through constraints, progression graphs, editable plans, direct edits, candidate selection, and staged review rather than relying on isolated plausible outputs.RPGAgent’s 18-participant study reported higher user-experience and creativity-support scores than a GPT-assisted baseline, while DreamGarden users valued interpretable plans but were limited by waiting and insufficient intermediate inspection.Source: §5.3 · p. 36, Coordinating narrative, space, and… · p. 37, Granularity of creator control · p. 37

6 AI That Builds and Maintains Games

AI systems that build and maintain games must coordinate code, scenes, assets, engine configuration, and runtime behavior rather than generate isolated scripts. Evidence shows that execution feedback improves repair, while project scale, integration dependencies, behavioral correctness, revision, and handoff remain substantial challenges.

6 AI That Builds and Maintains Games

  • Project construction requires agreement among code, scenes, assets, engine configuration, object bindings, callbacks, and state, so a visually complete scene may still fail to invoke its behavior.Development agents use repository tools, editor operations, and execution feedback to inspect and repair these dependencies (Yang et al., 2025; Yin et al., 2026a; Huang et al., 2026a).Source: §6 · p. 38, §6.1 · pp. 38–39

Executable environment generation.

  • Generated environments must align observations, actions, transitions, rewards, termination, and reset behavior with consumer expectations, requiring semantic checks beyond structural type validation.Reference transitions, invariants, unit tests, and simulated trajectories help test whether legal actions have their intended consequences (Serapio et al., 2026; Hu et al., 2025).Source: Executable environment generation · p. 40

6.2 Tool-Using Development Agents

  • Repository, editor, and runtime access provide complementary context, while model specialization and reusable workflows address different sources of repeated development work without removing version-dependent project context.OpenGame’s intentalignment score rises from 49.8 for the base model to 54.1 for its fully trained variant, but this comparison should not be combined with fixed-backbone workflow ablations (Jiang et al., 2026b).Source: §6.2 · p. 41

Multi-agent development coordination.

  • Multi-agent systems exchange plans, artifacts, and interaction traces, but their value depends on consistent scene identifiers, requirements, versions, initial states, and comparable models, tools, and execution budgets.Play2Code’s player agent can distinguish an attempted but ineffective jump from an untested screenshot, although control failure and implementation failure may remain ambiguous (Huang et al., 2026a).Source: Multi-agent development coordination · pp. 41–42

Execution feedback.

  • 52.0% versus 41.1%: visual feedback raises GPT-5.4’s GameDevBench pass rate across 333 multimodal tasks, although the richer workflow uses additional tools and inference (Chi et al., 2026).Screenshots or video expose failures that static inspection misses, including broken wiring, absent feedback, unreachable objectives, and interactions that fail during play.Source: Execution feedback · p. 42

Failure modes and project scale.

  • 80.4% versus 5.7%: JAMER’s runtime-pass rate falls from small to large projects, while Mage’s best direct C# generation reaches 43% runtime pass across 858 attempts (Sun et al., 2026; Liu and Tatar, 2026).These measures concern headless startup or runtime execution, not sustained interactive correctness; Mage’s mechanism F1 is near 0.12 for the configuration with the best runtime pass.Source: Failure modes and project scale · p. 42
  • 55.5% of 110 GameEngineBench tasks are solved by the best configuration, while 31 defeat every tested configuration and many successful compilations remain behaviorally incorrect (La et al., 2026).Failures include engine lifecycle, replication, and cross-system behavior, showing that compilation does not establish correct integration.Source: Failure modes and project scale · p. 42

6.4 Revision, Maintenance, and Handoff

  • Maintenance requires update-aware testing that preserves behavior only where requirements leave it unchanged, while evidence for long-lived revision and developer handoff remains limited.GameXpert-Bench evaluates bounded cumulative revisions, and SAGE and SMART connect changes to behaviors that should be re-exercised (Chen et al., 2026b; Cai et al., 2025a; Mu et al., 2025).Source: Regression-aware revision · p. 43, Developer handoff · p. 44

7 AI That Generates and Adapts at Runtime

This section examines AI that generates and adapts game content and experiences at runtime, linking player input and session state to dialogue, narrative, mechanics, memory, and personalization. It addresses the need for game-specific execution constraints, controllability, intelligibility, accurate personalization, and evaluation beyond bounded prototypes, initial deployment, or novelty.

7 AI That Generates and Adapts at Runtime

  • Runtime systems must distinguish presentational language from state-changing or executable output, because player-experience claims depend on integrated generation, validation, timing, and memory.Figure 21 identifies generative content, long-term memory, dynamic rules, engine-side validation, and personalization as connected runtime directions.Source: §7 · p. 44, Figure 21 · p. 44

7.1 Generative Characters, Narrative, and Content

  • Interactive narrative systems distribute interpretation among players, humans, models, and engines, so scenario outcomes and consistency depend on exposed operations and reusable components.CALYPSO retains human adjudication, Craft an Iron Sword invokes game operations through generated actions, and Concordia models the Game Master itself (Volum et al., 2022; Zhu et al., 2023a; Vezhnevets et al., 2025).Source: §7.1 · pp. 44–45

7.2 Runtime Rules, Mechanics, and Worlds

  • Generated quests, behaviors, and rules require parsing, compilation, constraint checks, or engine validation, because fluent content can reference inaccessible entities or alter gameplay incorrectly.Constrained languages expose fewer operations and clearer feedback, but interacting valid rules can still be difficult to diagnose, as shown by IF:CARGO’s multi-robot and periodic-command cases (Hsu et al., 2026b).Source: Spoken dialogue and character voices · p. 46, §7.2 · p. 47

Character and interaction memory.

  • Runtime memory must separate recalled statements, beliefs, and intended actions from events the game has actually recorded, because retrieval and reflection can propagate errors.Generative Agents exposes retrieval failures and invented embellishments, while PANGeA’s reported 98–99% accuracy concerns handling synthetic scenario requests rather than social-event memory (Park et al., 2023).Source: Character and interaction memory · p. 48

Long-horizon and shared-state consistency.

  • Long-horizon and shared play require persistence across sessions, model updates, and multiplayer synchronization, yet existing learned-world evidence covers model-level synchronization more than fairness or live-population persistence.Tests should resume saved quests, correct earlier events, and update character models while preserving revised objectives and shared rewards; MultiGen illustrates synchronized generated viewpoints (Po et al., 2026).Source: Long-horizon and shared-state… · pp. 48–49

Affect- and model-driven adaptation.

  • Affective adaptation remains difficult to validate end to end: a review of 23 empirical studies found complete sensing-to-adaptation loops comparatively uncommon, and model accuracy must be separated from intervention benefits.Policies trained against affect models may exploit model errors, so predicted affect and the effects of resulting changes require distinct checks (Lopes et al., 2025; Barthet et al., 2024).Source: Affect- and model-driven adaptation · p. 49

Difficulty adaptation and balancing.

  • Companion effects vary by role and context: 130 players reported higher autonomy and cognitive load but lower usability and trust without improved overall experience, while 33 novices showed reduced challenges and tension with LeagueBot.These experiments used different games, populations, and outcomes, so their contrast cannot be attributed to companion architecture alone (Hsu et al., 2026a; Lee et al., 2026).Source: Difficulty adaptation and balancing · p. 51
  • In GenFlora, both dynamic item functionality and responsive NPC dialogue improved presence, autonomy, and enjoyment, with no significant interaction between factors in a 72-participant 2 × 2 study (Yin et al., 2026b).The result supports local integration effects within one farming-game prototype rather than a general advantage for generative characters.Source: Difficulty adaptation and balancing · p. 51
  • Adjustment studies vary how personalization choices are represented and offered, while adjustment can affect player confidence; open problems include modeling accuracy, controllability, and players’ mental models (Zhu and Ontañón, 2021).These concerns frame personalization as a player-centered design problem rather than only a generation problem.Source: Difficulty adaptation and balancing · p. 52
  • Language-model agents perform below average humans yet correlate strongly with human-reported difficulty on Wordle and Slay the Spire under generic prompting (Xiao and Yang, 2024).This supports their use as evaluation aids while not establishing human-level play.Source: Difficulty adaptation and balancing · p. 52
  • Deployed systems such as Teammates, PUBG Ally, AI Dungeon, and Fortnite demonstrate runtime integration, but evidence does not establish sustained cooperation, retention, autonomous tactical control, or long-term player benefit.These systems retain game-specific observations, action execution, authored constraints, and hardware budgets; reported playtests and official descriptions do not provide controlled outcome studies. [O’Connor, 2025; KRAFTON, 2026; Singh, 2026b; Latitude, 2026; Epic Games, 2025]Source: Difficulty adaptation and balancing · p. 52
  • Dialogue freedom expands player proposals without guaranteeing executable continuations, so meaningful agency depends on whether the game can act on contributions rather than merely accept their wording (Peng et al., 2024; Rao et al., 2024).Player-Driven Emergence can end improvised events without executable continuations, while Minecraft collaborators compensate for NPC perception gaps.Source: Difficulty adaptation and balancing · p. 52
  • Constrained representations can improve error explanation and player control, but interacting valid rules remain difficult to diagnose, coupling expressiveness with intelligibility (Hsu et al., 2026b).GROMIT exposes generated code, whereas IF:CARGO uses a constrained language with execution feedback.Source: Difficulty adaptation and balancing · p. 52
  • Personalization changes the data used for later personalization, confounding adaptation with generated content difficulty and profile inference (Hafnar and Demšar, 2025; Lu et al., 2026b).A match-three deployment’s first-level difference illustrates why stronger generation, easier initial content, and better profile inference cannot be credited to adaptation as one effect.Source: Difficulty adaptation and balancing · p. 52

8 AI That Tests and Evaluates Games

AI game testing combines exploration, executable or learned oracles, model-based judgement, and human-reference evaluation, but each addresses different bottlenecks and requires validation in its target setting. Evidence is strongest for bounded checks and selected benchmarks, while player relevance, persistent adaptation, and judge reliability remain conditional on appropriate traces, references, and independent validation.

Exploration objectives and coverage.

  • Exploration objectives determine coverage: Wuji balances progress and exploration, while curiosity-driven and multi-agent testers improve game-state coverage but require game-specific oracles and validation.Progress alone can repeatedly follow safe routes, whereas variation alone can miss later content; cMarlTest reports higher coverage than a single-agent variant under three criteria.Source: Exploration objectives and coverage · p. 53
  • Automated testers increasingly operate from pixels, GUIs, and language, reducing dependence on internal instrumentation but making interaction discovery, diagnosis, and trace memory more difficult.Play2Code uses observations and suggested fixes without exposing scoring rubrics, allowing traces to support subsequent software repair.Source: Exploration objectives and coverage · pp. 54–55

Executable oracles and state injection.

  • GameGen-Verifier reaches 92.2% specification-label agreement versus 58.8% for a coverage-enforced baseline, with up to 16.6× lower wall-clock time, while injected states bypass ordinary reachability.These results measure mechanic-check agreement, not fully correct games; injected states must respect invariants or be treated as robustness tests.Source: Executable oracles and state injection · pp. 55–56

Coverage guidance and regression selection.

  • Code-aware and regression methods target functions, changed behavior, or update-relevant tests, but reaching code does not establish that the intended player-visible behavior or fault was exposed.Regression records should retain the build, initial state, action sequence, expected result, and observed failure so defects remain reproducible and diagnosable.Source: Coverage guidance and regression… · p. 56

8.3 Model-Based Judges

  • Model-based judges remain challenged by visual and temporal QA: VideoGameQA-Bench’s best visual-regression accuracy is 45.2%, while PlaytestArena reports 84.2% raw criterion agreement with human judgments and Cohen’s κ = 0.64 on 32 games.Interactive judging grounds verdicts in play traces, whereas passive inspection does not test whether the agent can reach the relevant behavior.Source: §8.3 · pp. 57–58

Judge biases and agreement metrics.

8.4 Player Representativeness and Human Relevance

  • Behavioral diversity broadens failure discovery but is not player representativeness: synthetic testers can be compared with human trajectories, while personality interfaces still require game integration and population validation.Ariyurek et al. measure bug-finding effectiveness and behavioral similarity separately, and Play2Code shows that tester competence can limit which mechanics are reached.Source: Behavioral diversity · p. 59

Human-reference evaluation.

  • Human references must match the evaluation target: anticipated appeal, usability, persistent experience, and NPC bias require different evidence, while executable checks remain appropriate for well-defined rules.Automated judgements are most interpretable when their target and reference are narrowly specified; repair loops additionally need independent checks beyond ordinary agreement.Source: Human-reference evaluation · p. 60

9 Benchmarks and Evidence Across the Six Roles

This section examines how evidence is structured across the six roles, focusing on role-specific targets, evaluation boundaries, and the conditions that limit comparability and interpretation. It organizes evaluation settings by measured targets, evidence sources, protocols, and comparison limits.

9.1.1 Play and Act: Task Performance and Generalization

  • GameWorld and ARC-AGI-3 show that reported performance depends on whether evaluation measures completion or progress and on the configured interface and harness.In GameWorld, both interfaces scored below two human reference participants, with one participant per condition; ARC-AGI-3 reported GPT-6 Astra at 54.82 with the Standard harness and 99.95 with the Provider Adapter at the stated setting.Source: §9.1.1 · p. 61, Table 5 · p. 62, Table 6 · p. 63

9.1.2 Model Players and Games: World-Model Fidelity and Player-Model Validity

  • World-model evaluation must separate perceptual quality, mechanics correctness, persistent state, trajectory validity, and use-specific outcomes such as transferred policy return.PlayWorld reports cases where trajectory validity and rubric-based evolution scores diverge, while Programmable World Model reports 94% count accuracy and 98% death-state accuracy that support coarse state-conditioned rendering rather than complete combat-rule verification.Source: §9.1.2 · p. 64

9.1.3 Design: Design Quality and Designer Control

  • Design evaluation must match the generated object and distinguish artifact validity and diversity from creator control, correction effort, comprehension, agency, and player experience.Formal checks such as standing structures, solvable puzzles, and compilation test different failure modes; human studies are needed for claims about steering, comprehension, enjoyment, or control.Source: §9.1.3 · pp. 64–65

9.1.4 Build and Maintain: Executable Artifacts and Revision

  • Development benchmarks distinguish scoped implementation from complete game construction, and their scores reflect different execution evidence rather than one ranking of game-development ability.GameDevBench reports uncertainty intervals, while GameCraft-Bench shows mechanics scores exceeding content-depth scores across its reported rows; visual feedback creates an inspection opportunity but does not uniformly improve correctness.Source: §9.1.4 · pp. 65–66, Table 9 · p. 66, Table 10 · p. 67

9.1.5 Generate and Adapt at Runtime: System Behavior and Player Outcomes

  • Runtime evaluation must separate narrative, behavior, responsiveness, controllability, latency, and player outcomes, while repeated sessions are needed to distinguish sustained benefits from familiarity or novelty.Component-level studies vary factors such as item functionality, dialogue, response onset, and token delivery; spoken systems must separately measure recognition, generation, synthesis, playback, turn-taking, and interruption recovery.Source: §9.1.5 · pp. 66–67

9.1.6 Test and Evaluate: Coverage and Verdict Reliability

  • AI testers require evidence for coverage, verdict accuracy, diagnostic usefulness, efficiency, oracle independence, and representation of the intended player population.Efficiency must use the same unit as reliability and separate setup cost from repeated execution, since state injection or manual specifications can dominate the apparent cost.Source: §9.1.6 · p. 67

9.2.1 Dimensions for Comparing Evaluations

  • Cross-role comparison is possible through standardization, execution grounding, scope and transfer, horizon and revision, and external validation without collapsing unlike outcomes into a maturity score.Evidence may be standardized yet narrow, execution-grounded yet short-term, or human-centered yet small and system-specific; evidential weight also depends on repeated runs, uncertainty, sample size, and replication.Source: §9.2.1 · pp. 67–68

9.2.2 Statistical Support and Repeated Evaluation

  • Interfaces, harnesses, adaptation budgets, held-out settings, and resampling units determine what a benchmark score or transfer result supports.A benchmark measures the configured system rather than only its base model; transfer should distinguish visual representation, action control, and rule reasoning, while intervals and paired per-task results expose uncertainty and selection effects.Source: §9.3.1 · pp. 68–69, §9.2.2 · p. 68

9.3.1 Interfaces, Adaptation Budgets, and Held-Out Settings

  • Claims remain setting-specific: evidence must test post-action transfer under stated budgets and distinguish zero-shot use, retrieval, and target-game fine-tuning.Generated scenes or rollouts alone do not establish transfer; held-out tasks, rule-sensitive interventions, reference transitions, or sustained play provide stronger evidence.Source: §9.3.1 · p. 69

9.3.2 Measurement Validity and Evaluator Dependence

  • Evaluator agreement cannot replace engine checks: numerical errors, model judgments, programmatic assertions, and human labels measure different aspects of fidelity.Aggregate accuracy may hide false negatives, while text-based narrative audits do not create independently executable game-state models.Source: §9.3.2 · p. 69

9.3.3 Human-Study Validity

  • Human-centered comparisons require independent exposure and controlled baselines, because initial generation-policy differences and deployment duration can confound personalization and longitudinal claims.Keeping the generator fixed while varying profile or adaptation policy separates generated experience from validated player modeling (Hafnar and Demšar, 2025).Source: §9.3.3 · pp. 69–70, §9.3.4 · p. 72

9.3.4 Reproducibility and Data Provenance

  • Reproducibility requires reporting model and engine versions, interfaces, prompts, permissions, randomization, budgets, provenance, access conditions, and representative traces when components cannot be released.These are reporting requirements, not evidence that unavailable components are unreliable.Source: §9.3.4 · p. 70
  • Table 12 maps each role’s benchmarks to measured targets, evidence sources, and interpretation limits, showing that task scope, interfaces, horizons, references, and uncertainty constrain comparisons.Examples include 600+ WorldRoamBench cases spanning action, vision, physics, and memory over 10–60 s, while bounded checks do not establish long-term maintenance.Source: Table 12 · p. 70, §9.3.4 · pp. 70–71
  • Acc@5 averages five runs, and injection bypasses ordinary reachability; coverage is not correctness, while task completion, bug discovery, and deployment claims are distinct evidence targets.Source: §9.3.4 · p. 72

10 Discussion: Cross-Role Connections

Cross-role connections expose how model-facing interfaces, exchanged artifacts, and executable game structure interact, but implementation or reuse alone does not establish capability transfer or downstream benefit. Reliable claims therefore require validating compatibility, actual use, and intended outcomes in the target game, workflow, and player context.

10 Discussion: Cross-Role Connections

  • The discussion synthesizes representative cross-role systems qualitatively, examining their interfaces, exchanged information, and downstream effects without ranking all possible connections.It focuses on how roles connect and what evidence is needed to interpret those connections.Source: §10 · p. 72

10.1 Broader Interfaces and Game-Specific Structure

  • Broader model interfaces connect to constrained plans, tools, or actions, so reliability depends on the receiving component’s game-specific requirements even when dynamics are learned rather than explicitly engineered.DreamGarden, GameWorld, and GameNGen illustrate that expressiveness and executability remain distinct, while learned simulation still requires defined action semantics and state requirements.Source: §10.1 · pp. 72–73

10.2 Artifact Reuse and Capability Transfer

  • Artifact reuse can support capability transfer, but transfer requires specifying what changes and testing whether competence survives across games, engines, interfaces, tasks, or player populations.Implemented exchanges such as compiler feedback, play traces, and inferred profiles do not by themselves demonstrate changed-model competence or generally reusable skills.Source: §10.2 · pp. 73–74, Table 13 · p. 74

Executable grayboxes and generative rendering.

  • Executable grayboxes, engine state, and neural rendering can divide rule execution from visual synthesis, but reliable production workflows remain unestablished and require alignment between executable scenes, displayed outcomes, and inputs.The proposed workflow preserves engine authority over collision and score while testing state alignment, latency, and control errors; the lightweight-engine demonstration does not establish reliable UE/Unity maintenance workflows.Source: Executable grayboxes and generative… · pp. 74–75

10.3 Validating Downstream Benefits

  • Downstream validation must separately test compatibility, actual use, and benefit, using controlled comparisons and outcome measures suited to the connection rather than treating successful exchange as improvement.Relevant outcomes include reference-game behavior, repaired projects preserving unaffected requirements, and player experience; feedback loops and changing opportunities require held-out checks and context-aware records.Source: §10.3 · pp. 75–76

11 Open Challenges and Research Directions

The paper identifies six setting-dependent challenges across game AI roles, each pairing a technical problem with candidate approaches and tests of progress. Cross-role changes can affect downstream components, so claimed progress must be validated in the target setting.

11.1 Play and Act: Transfer Across Interfaces and Rules

  • Play agents must determine which prior skill transfers, which interface mapping requires relearning, and whether failures reflect poor control or unknown rules.Factorial tests should vary layout, appearance, controls, legal actions, objectives, and teammate conventions under fixed interaction budgets.Source: §11.1 · pp. 76–77

11.2 Model Players and Games: Persistent World State and Reliable Player Models

  • Persistent world models should selectively update explicit state while detecting contradictions after moved objects, spent resources, rule changes, and delayed events.Save–restore and synchronized-view tests should measure visual quality alongside rule violations, state disagreement, control errors, memory cost, latency, and correction frequency.Source: §11.2 · p. 77
  • Reliable player models must distinguish stable preferences from session-specific behavior, retain uncertainty under sparse observations, and validate adaptation on held-out behavior.Comparisons should use fixed profiles, recent-history predictors, and updated models under equal observation budgets, including resistance to misleading observations and player correction.Source: §11.2 · pp. 77–78

11.3 Design: Designer Intent, Diversity, and Control

  • Game generators must support iterative designer control by separating hard constraints from revisable preferences, preserving alternatives, and exposing edit effects without silently breaking dependencies.Evaluation should measure artifact validity and variation, creator correction effort and control, and the resulting player experience.Source: §11.3 · p. 78

11.4 Build and Maintain: Reliable Revision and Developer Handoff

  • Reliable game maintenance requires version-aware project context linking requirements, dependencies, reproducible traces, and tests across cumulative revisions and engine-wide effects.Independent checks should assess unchanged requirements, while multi-version benchmarks and developer handoff measure completion time, erroneous edits, and reconstruction effort.Source: §11.4 · pp. 78–79

11.5 Generate and Adapt at Runtime: Cross-Session Consistency and Player Experience

  • Runtime adaptation must convert uncertain predictions into reversible interventions that preserve cross-session continuity, established events, player agency, and shared multiplayer opportunities.Repeated-session trials should compare adaptation policies under fixed generators and starting conditions, measuring benefits, correction costs, continuity, agency, enjoyment, and unequal opportunities.Source: §11.5 · p. 79

11.6 Test and Evaluate: Independent Verification and Behavioral Coverage

  • Automated testing must separate policy coverage, oracle reliability, and population representativeness while preventing optimization against imperfect evaluators from rewarding shortcuts.Independent oracles, complementary exploration, invariant checks, withheld defect families, execution-based checks, and targeted human inspection are proposed to test generalization and verification.Source: §11.6 · p. 79

12 Conclusion

Foundation models enable reuse across AI’s game-lifecycle roles, but capability transfer remains conditioned on explicit game structure and target-specific evidence. Progress therefore requires preserving constraints, developer control, game consistency, and player relevance as rules, projects, and player contexts change over time.

12 Conclusion

  • Automated testing should measure additional validated defects within a fixed budget while separately reporting false positives, reproduction cost, coverage, and human-relevance samples.Novice and expert trajectories, different strategies, and targeted fairness cases (Shi et al., 2026a) answer distinct questions.Source: §12 · p. 80
  • Deployment evaluations should report tool permissions, data retention, version changes, compute, latency, rollback or fallback behavior, profile correction or withdrawal, and multiplayer-state compatibility.These controls connect prototype governance and player-facing systems to reliable live workflows.Source: §12 · p. 80
  • Artifact reuse across game-lifecycle roles does not establish capability transfer, because performance remains conditioned by the game, engine, interface, player population, and task.Useful exchanges include interaction traces, learned environments, specifications, execution and test evidence, and player models or profiles.Source: §12 · p. 80
  • Evidence is strongest for bounded game playing and selected learned environments, while persistent state, evolving software, repeated revision, sustained adaptation, and repeated player encounters remain less established.Progress should be assessed as game rules, project requirements, and player contexts change over time.Source: §12 · p. 80

A System Index

This index organizes game-AI systems across roles, artifacts, mechanisms, interfaces, feedback, data, and evaluation settings, distinguishing systems from related benchmarks, datasets, frameworks, and workflow reports. It surveys evidence across playing, modeling, design, construction, runtime adaptation, personalization, and testing, while noting that evaluation remains heterogeneous and often tied to target-specific environments, controls, representations, protocols, coverage, or deployment conditions.

Index structure

  • The index uses role codes for play, modeling, design, building, runtime adaptation, and testing, while marking additional functions and distinguishing systems from benchmarks, datasets, frameworks, and industry reports.References link indexed rows to their sources, and Section B provides an illustrated guide to selected demonstrations and public resources.Source: A System Index · p. 106

Play and act

  • Game-playing systems span visual, language, symbolic, and structured interfaces, using mechanisms such as diffusion control, retrieval, memory, planning, model construction, and coordination.Examples include Minecraft agents, cross-environment demonstration retrieval, compact-game search, and multi-agent systems for Overcooked, cooperative games, Minecraft, and StarCraft II.Source: A System Index · p. 107
  • 179/183 ARC-AGI-3 levels were evaluated for Code World Models, while REGENT evaluated held-out environments without fine-tuning and other systems used task completion, coordination, memory, or planning measures.These evaluations remain associated with specified games, tools, demonstrations, or offline trajectories rather than a shared score scale.Source: Table 15 · p. 106, A System Index · p. 107

Learned game models

  • Learned game models represent dynamics, state, NPCs, geometry, appearance, spatial memory, or interactive video, with controls ranging from actions and poses to camera inputs, prompts, and engine-derived conditions.Reported evidence includes action or appearance transfer, state and spatial consistency, preference judgments, rendering fidelity, revisiting, and interactive control.Source: Table 16 · p. 108, A System Index · p. 109
  • Model demonstrations report setting-specific measurements such as state error < 0.06, about 70% LLM preference, 31.54 fps on H200, and up to 40 fps, but use different data, hardware, and protocols.The index explicitly notes that reported speed and horizon are not directly comparable across learned game models.Source: Table 16 · p. 108, A System Index · p. 109

Design and development

  • The design and development entries cover mixed-initiative level and map authoring through constraint solving, novelty search, playability checks, and designer edits, with evaluations tied to historical authoring settings.Tanagra, Sentient Sketchbook, ANGELINA, and Mechanic Miner illustrate content, rules, and mixed-initiative design artifacts.Source: Table 18 · p. 111, A System Index · p. 111

Design and generation

  • Game-design systems generate mechanics, levels, assets, scenes, and rules through language models, diffusion, search, or structured editing, with evaluations spanning solver success, fidelity, diversity, and controllability.Reported limitations include stronger within-genre than cross-genre blending, executable structures without complete-game enjoyment, and search baselines remaining competitive.Source: A System Index · p. 111

Executable construction and repair

  • Construction systems translate requirements, plans, or specifications into engine artifacts, code, and scenes, using tool access, retrieved documentation, templates, or iterative repair.Benchmarks evaluate compilation, runtime behavior, screenshots, playtests, and rubric success, while several systems remain limited to prototypes, isolated changes, or non-benchmark demonstrations.Source: A System Index · p. 112

Narrative and content systems

  • Runtime narrative systems generate or adapt encounter prose, quests, dialogue, story graphs, and narrated consequences through memory, constraints, graph managers, and player input.Evaluations include field deployment, authoring studies, text ratings, simulated-history checks, and player studies, but some report no common player-outcome trial or possible unsupported entities.Source: A System Index · p. 113

Evaluation scope

  • Across the index, evaluation ranges from automated validity and compilation checks to raters, authors, developers, and interactive players, but evidence remains uneven across systems and outcomes.Some studies report visual or motion gains without improved state-update accuracy, and several demonstrations quantify language or artifact quality without establishing sustained player effects.Source: A System Index · p. 113, Table 19 · p. 113

Executable behavior and rule generation

  • Executable behavior systems attach generated scripts or constrained rules to game entities, enabling runtime actions, deterministic execution, and language-mediated world crafting.Evidence consists mainly of developer interviews, demonstrations, or small prototypes, and intent following varies with model and prompt when player studies are absent.Source: A System Index · p. 113

Characters, companions, and personalization

  • Personalized and emergent systems support player-directed rule revision, NPC dialogue, narrative events, and timed guidance, with studies exposing both control benefits and interaction failures.Among reported findings, rule revision supports control but adds cognitive load; one NPC system achieved 25% full-quest completion with weak state grounding, while another produced emergent events and dead ends.Source: A System Index · p. 113

Runtime systems

  • Controlled runtime studies report increased autonomy, presence, and enjoyment, but effects on trust, usability, challenge, tension, and interaction vary by system.GenFlora found autonomy/load increased while trust/usability decreased and overall experience was nonsignificant; another study found presence/autonomy/enjoyment increased, with interaction nonsignificant, while personalized match-three generation reduced reported challenge and tension.Source: A System Index · pp. 113–114
  • Runtime latency can harm experience, with sustained delay producing stronger negative effects in simulated RPG conversations.The study exposed 34 participants to simulated RPG conversations and varied first-token and per-token delays for NPC text responses.Source: A System Index · p. 114
  • A completion difference in personalized content followed an initial-level imbalance, underscoring the need to separate adaptive effects from starting-condition differences.The study involved 102 players, 928 starts, and 422 completions.Source: A System Index · p. 114
  • Industry and prototype deployments demonstrate interactive narrative, speech, voice, companion, and functional-object systems, but provide limited controlled efficacy evidence.AI Dungeon had official feature documentation without matched efficacy comparison; Mecha BREAK / ACE, Fortnite Darth Vader [94], Ubisoft Teammates [234], and PUBG Ally [161, 294] reported showcases or prototypes without controlled player outcomes, while Roblox Cube 4D [293] reported creator-enabled live experiences with supplied object and behavior structures.Source: A System Index · p. 114

Testing systems

  • PlaytestArena’s rubric oracle achieved 84.2% raw agreement and κ = 0.64 on 32 games, while GameGen-Verifier reached 92.2% mean Acc@5 but remained vulnerable to injection bypasses.GameGen-Verifier targets rendered browser games through an agent player and model judge, using VLM judgments, assertions, and expert labels.Source: A System Index · p. 115
  • Testing across learned and generated environments uses reference-action adaptation, state injection, and parallel keypoint checks, but tester and verifier choices can influence scores.PlayWorld [81] covered nine learned environments and 171 scenarios; another system targeted 100 generated web games with specified mechanics.Source: A System Index · p. 115
  • Visual and text-game evaluators show narrow strengths: best visual-regression accuracy was 45.2% without exploration, while passive checks identify hallucination, forgetting, and input-misinterpretation bugs.The visual system evaluated nine QA tasks from screenshots/videos, and the text-game system checked DejaBoom interaction logs without active exploration.Source: A System Index · p. 115
  • Downstream repair gains depend on tester coverage, and human alignment can differ from optimal-play accuracy in game evaluation.Play2Code repair used observed failures from coding agents, while another evaluator compared predicted outcomes and anticipated fun against optimal-play estimates and 450+ participants.Source: A System Index · p. 115

B Game Examples and Interactive Demonstrations

This section organizes 23 game examples spanning genres and AI roles, combining commercial titles, generated prototypes, specialist systems, and foundation-model applications. It also provides 12 illustrated cases and public materials distinguished by demonstration, industry, and product context.

B Game Examples and Interactive Demonstrations

B.1 Racing, Team Play, and Learned Simulation

These examples show AI acting within supplied game structures, from specialist racing and language-grounded teamwork to a learned visual simulator. The AI outputs remain tied to each game’s rules, interfaces, and training context.

Gran Turismo 7 / Sophy Learned Opponent

  • Sophy 2.1 applies reinforcement learning to race people in Gran Turismo 7 and selected custom-race configurations [356, 257].Tracks, vehicles, physics, and race rules remain supplied by the game; this is a specialist policy rather than a general language agent.Source: §B.1 · p. 117

PUBG / Ally Duo Co-Playable Character

  • Ella combines language-based communication with grounded teammate behavior, enabling spoken requests during PUBG battle-royale matches [161, 294].PUBG supplies the map, combat mechanics, controls, and action affordances; the PC beta ran from 17 June to 1 July 2026.Source: §B.1 · p. 117

DOOM / GameNGen Learned Environment

  • GameNGen uses an action-conditioned diffusion model to generate subsequent DOOM frames from trajectories supplied by an RL player [333].DOOM supplies the rules and represented states, while the learned simulator produces the visual continuation.Source: §B.1 · p. 117

B.2 Fighting, Platforming, and Physics Puzzles

This section shows AI contributions across fighting, platforming, and physics-puzzle settings, while each system remains grounded in a game-specific simulator, environment, or competition. These systems condition combat behavior, generate playable platform layouts, or construct physics-constrained structures using supplied game rules and elements.

Street Fighter / ReactiveGWM NPC-Aware Simulation

  • Street Fighter’s ReactiveGWM [344] separately conditions generated fights on player controls and high-level NPC strategies, changing opponent reactions across two title-specific mechanics.The system is a research simulator rather than a released Capcom feature.Source: §B.2 · p. 118

Mario Levels / MarioGPT Level Design

  • MarioGPT [302] fine-tunes a language model to propose tile layouts from requests about objects and elevation, producing levels playable or checkable with A*.An existing Mario-style environment supplies the tile vocabulary, movement rules, renderer, and controller; generation changes the level, not the engine.Source: §B.2 · p. 118

Science Birds / LLMs4PCG Physics-Constrained Design

  • LLMs4PCG [198] prompts language models to arrange blocks into letter-shaped structures evaluated for stability, shape resemblance, and diversity.Science Birds supplies executable physics and construction elements, making this an Angry Birds-style research platform rather than an official release.Source: §B.2 · p. 118

B.3 Building Games and Generating Live Content

AI systems support game construction, interactive narrative generation, and functional object creation, while relying on supplied engines, specifications, premises, schemas, scripts, assets, or rules. These systems produce playable projects, responsive story continuations, and objects whose behaviors are adapted to generated geometry.

GameCraft-Bench / Godot Games Project Construction

  • Coding agents implement complete Godot game projects across 15 families, including racing, roguelikes, puzzles, sports, and visual novels [202, 103].The benchmark supplies Godot, task specifications, tools, and available assets; gallery selections illustrate outputs, while full-task comparisons appear in Table 10.Source: §B.3 · p. 119

Book of Infinity: 1001 Nights Interactive Narrative

  • Generated story continuations respond to player-authored tales, and words can become equipment through the game’s narrative mechanic [306, 2].The game supplies the premise and conversion rules, while its current product disclosure identifies core art assets as human-made.Source: §B.3 · p. 119

Roblox / Cube 4D Functional Object Generation

  • Text prompts generate objects with distinct functional parts, while behavior scripts are retargeted to the generated geometry [293].The beta supplies object schemas and scripts; Roblox provides physics and the surrounding creator-configured experience.Source: §B.3 · p. 119

B.4 Character Dialogue and Automated Testing

The section spans character dialogue and automated testing, showing language-and-voice models embedded in authored encounters alongside learned agents that execute or score game tests. These systems depend on supplied characters, rules, interfaces, objectives, and safeguards, with PlayWorld probing generated environments rather than commercial games.

Fortnite / Darth Vader Voiced NPC Dialogue

  • A language model generates Darth Vader’s conversational responses and a voice model delivers them within an authored Fortnite battle-royale encounter [94].Epic supplies the character, combat behavior, game rules, and safety controls; the 2025 seasonal feature is a deployment example, not a current availability promise.Source: §B.4 · p. 120

Battlefield 2042 / EA SEED Production QA

  • A learned helicopter controller executes Battlefield 2042 QA tasks, while the same engineering study integrates zero-gravity navigation into Dead Space tests [108].Scripted frameworks specify objectives and check outcomes, while reinforcement learning supplies difficult control skills inside the QA workflow; this is specialist RL, not an LLM tester.Source: §B.4 · p. 120

PlayWorld / Interactive Worlds Agent-Based Probing

  • PlayWorld’s visual agent probes generated environments for geometric and temporal inconsistencies, and a judge scores trajectories against supplied test rubrics [81].Scenario goals, the action interface, and scoring rules are provided; the demonstrations and code report full results in Table 8.Source: §B.4 · p. 120

Related papers

Loading 2609.16679v1…