PaperlayerView exact revision on arXiv

Source-linked AI summary

VibeGame: Prompt-to-Game Development with AI-Native Engine and Self-Evolving Adversarial Agent Team

Wenbo Hu, Ken Li, Jiazhe Wei, Yukang Cao, Weiyi Hong, Jiayi Dai, Chenjun Bai, Jiajun Liang, Yucheng Liao, Ruichuan An, Zeyu Lou, Haofan Wang, Yueming Lyu, Jingkang Yang, Ziwei Liu, Chenyang Si

arXiv:2610.33106v1cs.MA

TL;DR

Game development requires coordinating interdependent project components and testing their behavior during play, while existing approaches do not fully support complete, inspectable projects. VibeGame combines a structured, runtime-controllable engine with adversarial review and training-free experience reuse; on a GameCraft-Bench subset, it outperforms evaluated harnesses under the same model, including GPT-6.1-Sol.

  • Problem

    Game development requires coordinating code, scenes, assets, animations, physics, and configurations while interactively validating their integrated behavior.

    Source: §1 · pp. 2–3

  • Method

    VibeGame combines an AI-native engine for structured project state and frame-synchronous runtime control with an eight-agent adversarial team and training-free reuse of accepted-project experience.

    Source: §1 · pp. 3–4, §3.1 · p. 4

  • Results

    Under the same underlying model, VibeGame without provided assets outperforms all evaluated agent harnesses; with GPT-6.1-Sol, it improves over the strongest baseline, Codex (P2G), by 4.19 points.

    Source: §1 · pp. 3–4, §5 · p. 10

  • Takeaways & Limitations

    Accepted projects can be distilled into reusable skeletons, modules, and collaboration contracts that subsequent projects adapt and revalidate.

    Source: §1 · pp. 3–4

    The framework is focused on 2D games and has not been validated in more complex 3D scenes or complete commercial games that sustain long-term play.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Large language model agents have enabled software development through natural-language interaction, but extending this paradigm to end-to-end game development remains challenging. Building a playable game requires coordinating code, scenes, assets, animations, physics, and configurations, while allowing developers to validate their integrated behavior through interactive testing. We introduce VibeGame, a framework for creating complete 2D game projects from natural-language prompts and optional reference images. Its AI-native game engine represents project state as schema-validated structured text and provides frame-synchronous runtime control, enabling agents to inspect game states, execute actions, and verify behavior at their own pace. Built upon this interface, we design an Adversarial Agent Team of eight specialized agents to separate implementation from review, so that problems are found and fixed through multi-round loops. For each conducted and accepted project, we further introduce a training-free self-evolution mechanism to distill it into reusable skeletons, modules, and collaboration contracts for future development. We evaluate prompt-assets-to-game (PA2G), where agents are given an asset library, and prompt-to-game (P2G), where agents generate their own art. On a GameCraft-Bench subset, VibeGame in P2G consistently outperforms a range of agent harnesses in either P2G or PA2G with the same underlying model, including GPT-6.1-Sol.

1 Introduction

VibeGame targets the gap between natural-language game requests and complete, editable projects that can be tested during development. It combines an AI-native engine, adversarial review, and reuse of validated experience; the paper reports that it outperforms other evaluated harnesses under the same underlying model.

1 Introduction

  • Game development demands coordination of real-time interaction, visual identity, and balanced rules, traditionally requiring months of collaboration among designers, programmers, and artists.The paper notes that the time and resources required can put game development out of reach for many creators.Source: §1 · p. 2
  • Prior systems either generate interactive experiences without editable source code or leave project state difficult for agents to inspect and modify consistently.The paper frames P2G as producing a complete playable project from a natural-language description and optional reference images, while identifying runtime checks limited to final evaluation or build-and-run verification as a shortcoming of prior efforts.Source: §1 · p. 2
  • Interactive testing matters because many game issues emerge only during play, while fixes to one artifact can create inconsistencies across interdependent project components.The paper identifies agent-operable project state, controllable runtime verification, and preservation of validated development experience as coupled requirements.Source: §1 · p. 3
  • VibeGame connects project construction, runtime verification, and experience reuse through an AI-native engine, an adversarial agent team, and training-free self-evolution.The engine exposes project elements as structured text with explicit references and schema validation, and lets agents inspect states, inject actions, and observe consequences; accepted-project experience is distilled into reusable skeletons, modules, and collaboration contracts.Source: §1 · pp. 3–4
  • With the same underlying model, VibeGame outperforms a range of agent harnesses in both P2G and PA2G settings.The introduction also reports that removing the AI-native engine or adversarial review roles substantially lowers the overall score.Source: §1 · pp. 3–4

2 Related Work

Related work spans repository-level coding agents and vibe-coding frameworks, as well as systems that connect language models or generative models to game engines and 3D tools. VibeGame extends these directions by jointly representing game-project artifacts and verifying running games within development.

2 Related Work

  • Coding agents and vibe-coding frameworks support repository-level development through persistent specifications, tasks, memory, and specialized roles.VibeGame extends these approaches to projects where code, art assets, scene configurations, and runtime behavior must be maintained jointly and verified by running the game.Source: §2 · p. 4
  • Prior engine-integrated systems generate scenes or content through commercial-engine GUI workflows, while Phaser-based systems express projects fully in text.OpenGame uses templates and a debug protocol and repairs build and test failures until the game builds and runs; AutoUE uses play-testing for final evaluation.Source: §2 · p. 4
  • VibeGame adds an agent-oriented project representation, runtime control, visual verification, and independent roles that verify the running game inside the development loop.The related-work discussion contrasts this with OpenGame's build-and-run repair process and with systems that mostly work around graphical interfaces.Source: §2 · p. 4

3 The VibeGame Framework

VibeGame turns natural-language requests and optional reference images into complete 2D game projects through a shared engine interface, specialized development and review roles, and reusable experience from accepted projects. Its workflow combines structured project state and runtime testing with iterative review.

3 The VibeGame Framework

  • The framework accepts natural-language requests and optional reference images, then produces complete 2D projects that can be run, inspected, and modified.Its lifecycle connects the shared engine, an eight-agent team, and experience reuse for future runs.Source: §3 · p. 4

3.1 AI-Native Game Engine

  • The AI-native engine represents projects as structured text, validates scene and asset references, exposes frame-synchronous runtime controls, and includes its uncompiled source.Scenes contain nodes grouping appearance, collision geometry, scripts, and configuration; agents can inspect or intervene in runtime state and actions.Source: §3.1 · pp. 4–5

3.2 Adversarial Agent Team

  • The Adversarial Agent Team assigns eight specialized roles and organizes work into intention alignment, parallel development, and adversarial correction.The orchestrator coordinates roles, while implementation and review responsibilities are separated.Source: §3.2 · p. 6

3.2.1 Intention Alignment

  • Before development, the designer turns the request into a Game Design Document while the artist creates concept art; user-confirmed materials guide downstream work.Design changes during development return to the designer for drafting, user confirmation, and incorporation into the document.Source: §3.2.1 · p. 6

3.2.2 Parallel Development

  • For code tasks, the orchestrator prepares scope and acceptance criteria, then architect and programmer plan and implement work before auditor checks and player play-tests it.A subtask completes only after static auditing and runtime play-testing.Source: §3.2.2 · pp. 6–7
  • The artist turns visual requirements into engine-ready assets using image generation, post-processing, and registration of asset paths, dimensions, and usage.Source: §3.2.2 · p. 7

3.2.3 Adversarial Correction

  • An independent reviewer evaluates functionality, visual quality, and playability, rejecting failures for orchestrated repair until acceptance criteria are met.User feedback can be incorporated into subsequent development.Source: §3.2.3 · p. 7

3.3 Training-free Self-Evolution

  • Self-evolution distills accepted projects into reusable game-type skeletons, behavior modules, and contracts that specify role inputs, outputs, dependencies, and verification.Skeletons preserve runnable structure and tuning while modules can include pre-start checks based on observed runtime failures.Source: §3.3 · p. 7

4 Experiments

On a 30-task GameCraft-Bench subset, VibeGame outperforms the strongest baseline overall for both tested models and wins on most game types. Ablations, user judgments, and qualitative cases further report component effects and gameplay comparisons.

4.1 Experiment Setup

  • The evaluation samples 30 tasks across 15 GameCraft-Bench game types, compares harnesses in PA2G and P2G, and gives each 24 hours per task.A separate 10-task ablation set tests removing the engine or adversarial review and adding stronger-model experience to GPT-5.6-Luna.Source: §4.1 · pp. 7–8

4.2 Results Analysis

  • 72.7% was VibeGame's average overall human win rate; participants preferred it against every GPT-6.1-Sol baseline in every rubric category, as did the judge.The blind study included 30 participants and 300 judgments.Source: Figure 4 · p. 8, §4.2 · p. 9
  • 11.70 points (GPT-5.6-Luna) and 4.19 points (GPT-6.1-Sol) overall gains over strongest baselines; VibeGame also outperforms them on 13 of 15 game types.For GPT-6.1-Sol, gains over Codex (P2G) are 4.76 points in Content Depth and 7.72 points in Art & Presentation.Source: §4.2 · p. 8, Figure 4 · p. 8
  • Removing the AI-native engine lowered overall score by 10.33 points and removing adversarial review lowered it by 4.63; transferred experience raised GPT-5.6-Luna's score by 2.92.Adding the engine alone to Codex with GPT-6.1-Sol raised overall score by 2.69 points.Source: §4.2 · p. 9, Table 2 · p. 9
  • In the skateboard case, VibeGame presents aerial tricks in a consistent style, while Codex (PA2G) uses crude programmatic art and Codex (P2G) has overlapping UI.In the boxing case, VibeGame includes attack wind-up tells; Codex (P2G) lacks them, while Codex (PA2G) represents punches with stretched rectangular arms.Source: Figure 5 · p. 9, §4.2 · p. 10

5 Conclusion

VibeGame combines an AI-native game engine, an adversarial agent team, and training-free self-evolution for end-to-end prompt-to-game development. Without provided assets, it outperforms evaluated harnesses, including a 4.19-point gain over the strongest baseline with GPT-6.1-Sol.

5 Conclusion

  • VibeGame integrates an AI-native game engine, an adversarial agent team, and training-free self-evolution into an end-to-end prompt-to-game framework.The framework targets development from prompts without requiring provided assets.Source: §5 · p. 10
  • Without provided assets, VibeGame outperforms all evaluated agent harnesses under the same underlying model.The conclusion reports this comparison for VibeGame's P2G setting.Source: §5 · p. 10
  • By 4.19 points, VibeGame with GPT-6.1-Sol improves over the strongest baseline, Codex (P2G).Source: §5 · p. 10

Limitations

The framework is currently focused on 2D games and has not been validated for complex 3D scenes or complete commercial games sustaining long-term play. Its AI-native design is implemented on a web-based 2D engine, with support for established engines such as Godot left for future work.

Limitations

  • The framework focuses on 2D games and has not been validated in more complex 3D scenes.Source: Limitations · p. 10
  • Complete commercial games that sustain long-term play have not yet been developed with the framework.Source: Limitations · p. 10
  • The AI-native design is implemented on a web-based 2D engine; bringing it to established engines such as Godot remains future work.That extension would let agents operate existing game projects.Source: Limitations · p. 10

A Evaluation Details

The evaluation ports GameCraft-Bench to Phaser and scores replayed game demonstrations against task rubrics. Scores cover four categories, aggregate requirement scores across demos, and use rubric-specified category weights with a buildability gate.

A Evaluation Details

  • The protocol ports GameCraft-Bench from Godot to Phaser, then checks builds, replays input traces, records games, and has a judge score recordings against the rubric.Requirement scores are aggregated over demos, and scores average three judge passes to reduce judge variance.Source: A Evaluation Details · p. 14
  • Task rubrics group requirements into Core Mechanics, Content Depth, Functional Visuals, and Art & Presentation.Each category score is the mean of its requirement scores.Source: A Evaluation Details · p. 14
  • Overall scoring uses task-rubric-specified category weights and assigns zero when a game cannot be built or played.Source: A Evaluation Details · p. 14
  • Table 3 reports GameCraft-Bench subset results scored by GPT-5.5 using the mean of its passes.Its rows, columns, and marks follow Table 1.Source: Table 3 · p. 14

A.1 Judge Error Cases (GPT-5.5)

The judge model changes absolute scores and rankings, although VibeGame achieves the best overall score under both judges. The examples show GPT-5.5 accepting defects that GPT-6.1-Sol identifies in track layouts, board readability, and production-chain evidence.

A.1 Judge Error Cases (GPT-5.5)

  • Changing the judge alters absolute scores and rankings, although VibeGame achieves the best overall score under both GPT-5.5 and GPT-6.1-Sol.Table 3 reports results scored by GPT-5.5.Source: §A.1 · p. 14, §A.2 · p. 18
  • GPT-5.5 accepts twelve tracks despite repeated layouts, while GPT-6.1-Sol identifies that they reuse four layouts rather than meeting the distinct-circuit requirement.The judged track-select frame shows the repeated layouts across thumbnails.Source: §A.1 · p. 15, Figure 6 · p. 15
  • GPT-5.5 credits a readable shop with a bench, but GPT-6.1-Sol notes that no bench is visible and board labels overlap cards.The board has front and back rows and no bench; cards cover the heading and row labels.Source: §A.1 · p. 16, Figure 7 · p. 16
  • For the production-chain requirement, GPT-5.5 scores the evidence 0.0 / 0.0 / 0.0, while GPT-6.1-Sol scores it 1.0 / 1.0 / 1.0 after observing moving resource particles.The example shows conveyors connecting four machines; the judges received downscaled frames.Source: §A.1 · p. 17, Figure 8 · p. 17

B Benchmark Tasks

The benchmark uses 30 GameCraft-Bench tasks across 15 game types, with a 10-task ablation subset. The setup also specifies a per-task budget and a procedure for transferring evolved artifacts.

B Benchmark Tasks

  • The test set contains 30 GameCraft-Bench tasks, two per game type, and the ablation set is a 10-task subset.Table 4 lists the tasks and marks the ablation tasks.Source: B Benchmark Tasks · p. 18, Table 4 · p. 18

C.1 Setup

  • Each harness receives a 24-hour budget per task, and models run at xhigh reasoning effort.For P2G, harnesses without native image generation receive GPT-Image-2 as an external tool.Source: §C.1 · p. 18
  • For the experience condition, VibeGame with GPT-6.1-Sol self-evolves on five tasks outside the ablation set, and its artifacts are added to VibeGame with GPT-5.6-Luna for ablation tasks.Task rubrics and VLM judge scores stand in for user feedback during self-evolution.Source: §C.1 · p. 18

C.2 Task Instructions

This section describes deterministic, recording-based evaluation and the prescribed development workflow, while demonstrating VibeGame’s creation and selective editing of games from natural-language or multimodal prompts across varied genres and gameplay roles. It notes that Phaser-based results are not claimed to be comparable with the original GameCraft-Bench.

C.2 Task Instructions

  • All scoring uses demo recordings, so demos must show features clearly and use the full 20 seconds; source code is not evaluated.A feature that does not appear in a demo does not count, regardless of how well it is built.Source: §C.2 · p. 18
  • Because the study uses Phaser instead of Godot, it does not claim comparability with results on the original GameCraft-Bench.Source: §C.2 · p. 18

C.2.1 VibeGame

  • The VibeGame prompt specifies a complete autobattler built around drafting creatures, arranging a board, automatic combat, tribal synergies, economy, and eight-player elimination.It calls for a polished, shippable micro-game rather than a prototype.Source: §C.2.1 · p. 19
  • Evaluation requires a playable launch without fatal console errors, plus 1–10 JSON input traces replayed at 1280×720, with recordings capped at 20 seconds.Replay must be deterministic; named scenarios must set up their state deterministically and accept input immediately.Source: §C.2.1 · pp. 19–20
  • For gameplay tasks, the workflow commissions design and checks the GDD against the user's request; visual tasks use an approved in-game reference image consistent with that design.The reference image is reconciled with the GDD before execution if they conflict.Source: §C.2.1 · p. 21
  • The build workflow requires a staged plan with bounded, testable tasks, user approval before execution, parallel work where independent, and a persistent reviewer for final quality review.The lead coordinates and supervises rather than implementing directly except for trivial edits.Source: §C.2.1 · pp. 21–22

C.2.2 Baselines, PA2G

  • The PA2G baseline prompt reuses the autobattler task but supplies read-only asset libraries and requires Phaser 3, without CDN loading or a hand-built canvas renderer.OpenGameArt assets require respecting each subdirectory's license.Source: §C.2.2 · pp. 22–23
  • Baseline demos normally begin at the title screen and show the core gameplay loop; named scenarios instead load specific states immediately and deterministically.Replay must produce the same outcome from the same trace and a fresh launch.Source: §C.2.2 · pp. 23–24

C.2.3 Baselines, P2G (Qwen Code, Kimi Code, DSH); C.2.4 Baselines, P2G (Codex)

  • The P2G baseline task asks for a shippable Phaser 3 autobattler combining drafting, board placement, automatic combat, tribal synergies, economy, and elimination.The specified experience includes at least five tribes and escalating bonuses at 2/4/6 units; the game must run offline without a raw-canvas renderer.Source: §C.2.3 · p. 24, §C.2.4 · pp. 25–26
  • Evaluation scores recordings alone, not source code; submit 1–10 input-trace files, with a fresh game launched per trace and synthetic mouse/keyboard replay at 1280×720; only the first ten by filename count.Recordings longer than 20 s are sampled from a random 20 s window.Source: §C.2.3 · p. 25

D Demonstrations / D.2 Game Edit

  • After user approval, the demonstration workflow continues until reviewer acceptance; the editing examples cover visual-theme, genre, and character-role changes.The paper defines role transfer as changing character roles while reusing their abilities and animations.Source: D Demonstrations · p. 27, §D.2 · p. 29

D.1.1 Text Input

  • A compact Sekiro-style prompt yielded a complete deflection-and-posture combat layer built on an existing action-game skeleton, with 12 scripts, 2 character asset packs, 11 visual-effect entities, and 1 UI asset pack.The system added independent posture gauges, blocking and deflection rules, perilous attacks, and a multi-stage deathblow animation.Source: §D.1.1 · p. 28

D.1.2 Multimodal Input

  • The Hollow Knight-style example used text to specify combat mechanics and a concept image to anchor character, arena, and atmosphere, which the generated art matched.The delivered slice included distance-sensitive boss attacks, the player’s five actions, hit feedback, and a HUD, with art preserving the image’s silhouette language and cold palette.Source: §D.1.2 · pp. 28–29

D.2.1 IP Transfer

  • IP transfer replaced a completed boss fight’s art and adjusted sprite configurations while retaining its combat logic, without rewriting the core logic.The system regenerated characters, effects, UI, and backgrounds for the new theme.Source: §D.2.1 · p. 29

D.2.2 Genre Transfer

  • Genre transfer replaced a turn-based card game’s core play with real-time melee, dodging, and shooting while preserving its cyberpunk visual style and character designs.The example demonstrates selective deep modification: deciding what to rewrite and what to retain for consistency.Source: §D.2.2 · p. 29

D.2.3 Rule Transfer

  • The system swapped player and boss roles while preserving each character’s abilities and animations, reassigning controls, AI, HUD, and win/lose flow without new art or moves.The former boss became player-controlled, while the former player character became an AI-driven boss.Source: §D.2.3 · p. 29

Fruit Ninja

  • A text-only fruit-slicing game featured physically simulated fruit halves, pointer-driven slicing, combo scoring, missed-fruit indicators, and a game-over state.Each fruit type had distinct weight, bounce, and juice effects.Source: Fruit Ninja · p. 30

The King of Fighters

  • A single-sentence, image-free King of Fighters remake featured two fighters trading combos beneath health bars and a round timer on a neon-lit stage.Source: The King of Fighters · p. 30

Dungeon Roguelike

  • A Soul Knight-style roguelike used room-and-corridor maps that seal rooms for combat, with chest weapons, mouse aiming, and health, shield, and mana stats.The player could carry up to two weapons.Source: Dungeon Roguelike · p. 31

Parkour Platformer

  • A cartoon parkour platformer required chaining precise balloon landings, which popped balloons and launched the protagonist upward toward level completion.Source: Parkour Platformer · p. 32

Civilization prototype

  • A simplified Civilization-style 4X prototype combined hex-map city management and unit combat with a separated frontend and backend and multiplayer support.Source: Civilization prototype · p. 33

AI Werewolf

  • An engine-LLM-driven single-player Werewolf game used eleven AI players in a 12-role setup for sheriff elections, day discussions, and night actions, displaying their speech on screen.Source: AI Werewolf · p. 34

Hollow Knight to Demon Slayer

  • An IP transfer restyled a Hollow Knight-style boss fight as a Demon Slayer Mugen Train battle while retaining the underlying combat logic.The recreated battle featured Tanjiro and Upper Rank Three.Source: Hollow Knight to Demon Slayer · p. 35

F Adversarial Agent Team Case Study

The case study shows three review roles identifying distinct gameplay and interface defects, with before-and-after figures documenting repairs.

Auditor

Player

Reviewer

G Codex with the AI-Native Engine

With the same model and asset library, adding VibeGame’s AI-native engine improved Codex’s overall score. The largest category gain was in Content Depth.

G Codex with the AI-Native Engine

H Limitation of Self-Evolution

Experience accumulated across different game types can lower agent performance. The authors therefore evaluate self-evolution separately from their main experiments.

H Limitation of Self-Evolution

  • Experience accumulated while developing different game types can instead lower agent performance.The authors identify this observation as one motivation for the experimental design.Source: H Limitation of Self-Evolution · p. 40
  • The authors propose separating general experience from genre-specific experience and injecting each adaptively as future work.They say the phenomenon requires further analysis.Source: H Limitation of Self-Evolution · p. 40
  • The main experiments use VibeGame without accumulated experience, while self-evolution is evaluated separately in an ablation study.Source: H Limitation of Self-Evolution · p. 40

Related papers

Loading 2610.33106v1…