PaperlayerView exact revision on arXiv

Source-linked AI summary

Diffusion Models Are Real-Time Game Engines

Dani Valevski, Yaniv Leviathan, Moab Arar, Shlomi Fruchter

arXiv:2408.14837v2cs.LGcs.AIcs.CV

TL;DR

Existing neural game simulations have been limited in complexity, speed, long-term stability, or visual quality, motivating real-time simulation of complex environments. GameNGen trains an RL agent and then a diffusion model to predict frames from past frames and actions, demonstrating high-quality interactive DOOM simulation at 20 frames per second on a single TPU. The paper positions this result as a proof of concept for neural game engines while identifying limited context length as an important boundary.

  • Problem

    Prior neural game simulations remain limited in game complexity, speed, long-term stability, or visual quality, motivating the question of real-time high-quality simulation of complex games.

  • Method

    GameNGen first trains an agent to play DOOM and then trains a diffusion model on the recorded actions and observations to generate subsequent frames.

  • Results

    GameNGen demonstrates high-quality real-time DOOM gameplay at 20 frames per second on a single TPU.

  • Takeaways & Limitations

    GameNGen is a proof of concept that complex games can be represented by neural-model weights and run interactively on existing hardware.

  • Takeaways & Limitations

    The model accesses only a little over 3 seconds of history, so some situations exceed its available context and its learned heuristics can be wrong.

Abstract

from arXiv · show

We present GameNGen, the first game engine powered entirely by a neural model that also enables real-time interaction with a complex environment over long trajectories at high quality. When trained on the classic game DOOM, GameNGen extracts gameplay and uses it to generate a playable environment that can interactively simulate new trajectories. GameNGen runs at 20 frames per second on a single TPU and remains stable over extended multi-minute play sessions. Next frame prediction achieves a PSNR of 29.4, comparable to lossy JPEG compression. Human raters are only slightly better than random chance at distinguishing short clips of the game from clips of the simulation, even after 5 minutes of auto-regressive generation. GameNGen is trained in two phases: (1) an RL-agent learns to play the game and the training sessions are recorded, and (2) a diffusion model is trained to produce the next frame, conditioned on the sequence of past frames and actions. Conditioning augmentations help ensure stable auto-regressive generation over long trajectories, and decoder fine-tuning improves the fidelity of visual details and text.

1 INTRODUCTION

GameNGen asks whether a neural model can simulate a complex game in real time at high quality, addressing limitations in prior interactive game simulations. It demonstrates that DOOM can be simulated by a neural network on existing hardware while preserving complex state over long trajectories.

  • Research gap: Interactive game simulation must generate frames autoregressively while conditioning on an ongoing stream of player actions.This requirement distinguishes interactive world simulation from ordinary multimodal video generation.
  • Research gap: Prior neural game simulations are limited by game complexity, simulation speed, long-term stability, or visual quality.The paper identifies these limitations across several earlier approaches.
  • Contribution: GameNGen demonstrates real-time neural simulation of DOOM with visual quality comparable to the original game.The model performs complex state updates, including tracking health and ammo, attacking enemies, damaging objects, and opening doors.
  • Contribution: The system adapts a text-to-image diffusion model for stable autoregressive generation, improves fidelity through latent-decoder fine-tuning, and collects data using an RL agent.These contributions enable training a neural game simulator from gameplay collected at scale.
  • Broader significance: Real-time simulation of complex games on existing hardware supports a possible paradigm in which games are generated by neural models rather than manually programmed.The paper presents this as one step toward automatically generated games, while noting that creating entirely new games remains unresolved.

2 INTERACTIVE WORLD SIMULATION

The paper formalizes interactive world simulation as predicting observations from past observations and actions in a partially observed environment. It distinguishes teacher forcing, which uses environment observations, from autoregressive simulation, which uses the model's own observations.

  • Interactive environment: An interactive environment consists of latent states, observations, a projection function, actions, and a transition probability function.For DOOM, these correspond to dynamic memory, rendered pixels, rendering logic, key presses, and game logic with possible nondeterminism.
  • Simulation objective: An interactive world simulation models a distribution over observations conditioned on past observations and current and past actions.Its objective compares observations sampled from the real environment and the simulation under an agent policy.
  • Training objectives: Conditioning actions are obtained from the agent interacting with the environment, while conditioning observations come either from the environment or the simulation.The two observation sources define teacher-forcing and autoregressive objectives.
  • Training objectives: GameNGen trains with teacher forcing and generates environment trajectories by autoregressively sampling observations.At training time, the model receives environment observations; during simulation, it feeds back its own predictions.

3 GAMENGEN

GameNGen trains a diffusion model on agent-recorded gameplay to predict successive DOOM frames from past actions and observations. Noise-augmented conditioning stabilizes auto-regressive simulation, while decoder fine-tuning improves visual details.

  • Data collection: GameNGen trains a generative diffusion model on trajectories collected from a separately trained gameplay agent.The recorded actions and observations become the generative model’s training dataset.
  • Generative model: The model predicts the next game frame from sequences of past actions and observations using a repurposed Stable Diffusion model.Actions are embedded as tokens, while previous frames provide the observation conditioning.
  • Auto-regressive stability: Noise augmentation addresses error accumulation caused by the mismatch between teacher-forced training and auto-regressive sampling.Context frames receive varying Gaussian noise during training, with the noise level supplied to the model.
  • Latent decoder fine-tuning: Decoder fine-tuning targets artifacts affecting small visual details and the bottom-bar HUD without changing auto-regressive latent generation.Only the latent autoencoder decoder is fine-tuned against target-frame pixels, separately from U-Net training.
  • Denoiser sampling steps: 4 sampling steps preserve quality better than a single step, while distillation enables 50 FPS with some simulation-quality cost.The method therefore uses the 4-step non-distilled version, reaching 20 frames per second in the described hardware setup.

4 EXPERIMENTAL SETUP

The experimental setup trains a PPO agent to play DOOM from visual observations, map information, and recent actions, then trains simulation models with distributed TPU computation.

  • Agent training: The gameplay agent uses PPO with a CNN feature network and MLP actor and critic heads.It processes downscaled 160x120 frame images, an in-game map, and the last 32 actions.
  • Agent training: The agent is trained in ViZDoom using image, map, and action-history inputs to produce gameplay trajectories.The image feature network computes a 512-dimensional representation for each image.
  • Simulation training: Simulation models are fine-tuned from Stable Diffusion 1.4 with a batch size of 128 and learning rate 2e-5 using 128 TPU-v5e devices.Training unfreezes all U-Net parameters and uses Adafactor with gradient clipping of 1.0.

5 RESULTS

GameNGen produces visually similar DOOM trajectories over extended autoregressive play, while its metrics reveal divergence from exact frame-by-frame ground truth. Noise augmentation and longer conditioning contexts improve stability and quality, whereas random-policy data performs unevenly across difficulty levels.

  • Image Quality: 29.43 PSNR and 0.249 LPIPS were achieved on 2048 holdout trajectories across 5 levels under teacher forcing.The PSNR was similar to lossy JPEG compression with quality settings of 20-30.
  • Video Quality: Autoregressive trajectories remain similar in content and image quality despite gradually worsening per-frame PSNR and LPIPS.Small differences in movement velocities accumulate and cause predicted and ground-truth trajectories to diverge after a few steps.
  • Video Quality: 114.02 FVD was obtained for 16-frame trajectories, increasing to 186.23 for 32-frame trajectories.FVD measures the distance between predicted and ground-truth trajectory distributions over a holdout of 512 trajectories.
  • Human Evaluation: Human raters selected the actual game in only 58% or 60% of 1.6- or 3.2-second comparisons, and at chance level after 5 to 10 minutes of gameplay.The extended evaluation used 150 additional three-second side-by-side comparisons, with the real game identified 50% of the time.
  • Context Length: Longer conditioning contexts generally improve PSNR and LPIPS, but efficiently supporting longer contexts remains an important limitation.The context-length ablation varied history from 1 to 64 frames, while the method uses 64.
  • Noise Augmentation: Without noise augmentation, LPIPS distance increases rapidly and PSNR drops, indicating divergence from ground truth during autoregressive generation.The comparison reports average PSNR and LPIPS at each autoregressive step through 64 frames; the figure states that noise augmentation prevents rapid degradation after 10-20 frames.

6 RELATED WORK

Prior work explored neural game and world simulation through diffusion models, latent-state models, transformers, and adversarial architectures, but important limitations remained in complexity, speed, stability, or visual quality.

  • Game Simulation and World Models: Diffusion models have been applied to action-conditioned continuation video and interactive game simulation.Yang et al. trained a diffusion model to predict continuation videos from prior video and textual action descriptions.
  • Game Simulation and World Models: Latent-state approaches used variational autoencoders, recurrent networks, or LSTMs to model game environments and generate frames.These methods included ViZDoom simulation from random rollouts and adversarially trained world-state models.
  • Game Simulation and World Models: Transformer-based methods enabled action-conditioned video generation and latent-token world modeling in games and driving.Examples used discrete VQ-VAE latents for VizDoom or multimodal tokens with a diffusion video decoder for driving data.
  • Game Simulation and World Models: Concurrent work trained diffusion world models jointly with reinforcement-learning models on Atari and later simulated Counter-Strike from human gameplay recordings.The Counter-Strike version used 95 hours of human game-play recordings.
  • Auto-regressive Diffusion Models: Auto-regressive diffusion research investigated variable noise levels and sliding-window denoising, but these approaches were left for future real-time game simulation work.Chen et al. reported improved stability beyond the training horizon, while Ruhe et al. used a sliding-window denoising process.
  • DOOM: DOOM offers an open-source, low-resolution, yet sufficiently complex test case for neural game simulation.Its native resolution supports small models while retaining challenging gameplay complexity.

7 DISCUSSION

GameNGen demonstrates high-quality neural gameplay at 20 frames per second, while its limited memory, imperfect state persistence, and inability to create new games define important boundaries.

  • Summary: 20 frames per second enables high-quality real-time gameplay on a neural model.The discussion presents this as a demonstration using GameNGen.
  • Limitations: GameNGen accesses only a little over 3 seconds of history, limiting its ability to preserve game state in some situations.The model can persist state for much longer trajectories, but its short context can produce incorrect heuristics.
  • Limitations: GameNGen’s state persistence is imperfect because learned heuristics can misinterpret situations such as repeated shooting.The model may infer that an enemy should spawn because the agent usually shoots when enemies are present.
  • Future Work: Future work includes testing other interactive software, improving memory and state accuracy, and reaching higher frame rates on consumer hardware.The authors identify broader environments, more sophisticated architectures, expanded memory, and optimization as future directions.
  • Towards a New Paradigm for Interactive Video Games: The proposed paradigm treats games as neural-model weights rather than manually authored code, with possible editing through textual descriptions or example images.The paper frames GameNGen as a proof-of-concept and states that substantial work remains.

BROADER IMPACT

GameNGen opens possibilities for neural-network-based game development, while emphasizing responsible exploration and reproducible implementation choices.

  • Societal impact: Real-time neural simulation could open new possibilities for game development and interactive experiences.The paper compares this opportunity with other generative technologies and calls for responsible exploration.
  • Reproducibility: The implementation prioritizes reproducibility through Stable Diffusion 1.4, the open-source VizDoom environment, detailed configurations, and shared baseline metrics.These choices are intended to support accessible fine-tuning and future comparison.

CONTRIBUTION

The project combined contributions across proposal, architecture, implementation, stabilization, ablations, agent training, autoencoder fine-tuning, and human-play data creation.

  • CONTRIBUTION: The team distributed work across project design, architecture, coding, parameter tuning, agent training, decoder fine-tuning, stabilization, ablations, and human-play data.The passage attributes these contributions to different authors and collaborators.

A.1 SAMPLES

Figures 8–11 provide selected autoregressive GameNGen samples, pairing context, ground-truth, and predicted frames to assess simulated trajectories.

  • A.1 SAMPLES: Figures 8–11 show four selected samples from GameNGen’s autoregressive evaluation.The samples are identified as Samples #1–#4.
  • A.1 SAMPLES: Each sample arranges context frames above ground-truth frames and model predictions.This layout supports visual comparison between the conditioning history, actual continuation, and generated continuation.

A.2 FINE-TUNING LATENT DECODER EXAMPLES

The supplementary experiments examine decoder fine-tuning, dataset size, sampling, policy data, generalization, alternative games, and session management. Together, they show how GameNGen’s visual fidelity, training data, and interaction settings affect simulation behavior.

  • A.2 FINE-TUNING LATENT DECODER EXAMPLES: Fine-tuning the latent decoder improves visual fidelity relative to the standard decoder.The comparison specifically highlights artifacts in the frozen decoder, including HUD numbers.
  • A.2 FINE-TUNING LATENT DECODER EXAMPLES: Dataset size changes both PSNR performance and when test-set performance peaks.The comparison uses 1M, 5M, 10M, and 70M examples evaluated on 2048 unseen test trajectories.
  • A.2 FINE-TUNING LATENT DECODER EXAMPLES: Edited starting frames are used to test generation of characters and level structures absent from the training data.The setup replicates the edited frame across the history buffer and uses no-key-pressed actions.
  • A.2 FINE-TUNING LATENT DECODER EXAMPLES: Four sampling steps preserve quality, whereas a single sampling step degrades it.This evaluation uses 2048 generated frames on teacher-forced trajectories from 35FPS data.
  • A.2 FINE-TUNING LATENT DECODER EXAMPLES: RL-agent data can produce autoregressive predictions comparable to or better than random-policy data after 3 seconds.Figure 16 compares PSNR against ground truth during a short human play session.
  • A.2 FINE-TUNING LATENT DECODER EXAMPLES: GameNGen was also adapted to Chrome Dino and trained with 2K recorded RL-agent episodes.The system additionally represents termination and automatic replay by sampling contexts spanning concatenated episodes.
Loading 2408.14837v2…