PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playing

Yuguang Yue, Irakli Salia, Samuel Hunt, Chris Green, Wenzhe Shi, Jonathan J Hunt

arXiv:2601.04575v2cs.AI

TL;DR

Behavior cloning can learn policies from collected data, but it is known to face distributional shift and causal confusion. This paper presents an open recipe for real-time video-game play and studies causal behavior as data and model capacity scale. In the reported setting, scaling is associated with more reliable attention to causal signals and higher causality scores in data-abundant regimes.

  • Problem

    Behavior cloning learns from state–action datasets and may generalize across games without reward engineering, but distributional shift and causal confusion can degrade its performance.

    Source: §1 · p. 1

  • Method

    The paper introduces an open, real-time behavior-cloning model for gameplay and studies causal behavior using a toy setting and scaled models.

    Source: §1 · p. 1, §5.2 · p. 8, §7 · p. 9

  • Results

    Scaling model and dataset size improves attention to causal signals in the reported setting, while larger models achieve lower test loss and higher causality scores in data-abundant regimes.

    Source: §1 · p. 2, §7 · pp. 9–10

  • Takeaways & Limitations

    The results suggest scaling model size alongside dataset size and diversity as one possible approach to causal issues in behavior cloning.

    Source: §1 · p. 2, §7 · pp. 9–10

    The model currently follows only simple instructions similar to those in training, reflecting limited quantity and diversity of textual instructions.

    Source: §4.2.1 · p. 7

Abstract

from arXiv · show

Behavior cloning has seen a resurgence as scaling model and data sizes demonstrate strong performance. In this work, we introduce an open recipe for training a video game playing foundation model designed for inference in realtime on a consumer GPU. We release all data (8300+ hours of high quality human gameplay), training and inference code, and pretrained checkpoints under an open license. Empirically, we show that our best model achieves performance competitive with human players across a variety of 3D games. We use this recipe to investigate the scaling laws of behavior cloning, with a focus on causal reasoning. In a controlled toy setting, we first demonstrate that increasing training data and network depth leads to the model learning a more causal policy. We then validate these findings at scale, analyzing models up to 1.2 billion parameters. We observe that the causal improvements seen in the toy domain hold true as model size and training steps increase.

1. Introduction

Behavior cloning offers a data-driven route to policies across games, but it can suffer from distributional shift and causal confusion. The paper presents an open, real-time gameplay model and finds that scaling data and model capacity improves causal behavior in its setting.

1. Introduction

  • Behavior cloning learns policies from state–action pairs and may generalize across games without reward engineering, but distributional shift and causal confusion can degrade performance.The introduction contrasts this with reinforcement-learning systems that are typically tailored to one game and require substantial manual reward design.Source: §1 · p. 1
  • A single model uses raw images to play a range of 3D games in real time on consumer-grade GPUs, and the authors release its data, code, and checkpoints under open licenses.The model demonstrates high competence in simple games that do not require complex planning.Source: §1 · p. 1
  • Scaling both model and dataset size makes models more reliable at attending to causal signals, suggesting a practical way to mitigate causal issues in behavior cloning.The study examines model sizes from 150M to 1.2B parameters and reports a power-law relationship between test loss and dataset size.Source: §1 · pp. 1–2

2. Gameplay Dataset

The dataset combines diverse human gameplay with aligned actions and retrospective text instructions, plus small amounts of human correction data and separate evaluation environments. It also includes a corpus of unlabeled gameplay videos.

2.1. Annotated data

  • After quality filtering, the annotated dataset contains over 8,300 hours of gameplay, around 600 million image–action pairs, recorded at 20 frames per second.Players contribute varied hardware, settings, and styles; the collection is not balanced across games.Source: §2.1 · p. 2

2.1.1. Text-Annotated Data

  • A commercial vision-language model retrospectively generates temporally extended, goal-oriented instructions with start and end timestamps aligned to the gameplay video.The task requires inferring player intent from recorded behavior, which does not have a unique mapping.Source: §2.1.1 · p. 2

2.1.2. Correction Data

  • Human correction trajectories are mixed into training, with loss masked for agent actions; these trajectories account for less than 1% of the annotated dataset.Annotators take control when a deployed policy encounters out-of-distribution situations, then return control to the policy.Source: §2.1.2 · p. 3

2.1.3. Simple Benchmark Environments

  • Two programmatic 3D environments, Hovercraft and Simple-FPS, provide controlled benchmarks for comparing model performance.Their game state and difficulty can be precisely controlled.Source: §2.1.3 · p. 3

2.2. Unlabeled Data

  • The authors also curated a large corpus of unlabeled gameplay videos from public sources.The passage points to an appendix for details on how this corpus is used.Source: §2.2 · p. 3

3. Policy Model

Pixels2Play combines visual observations, optional text, and action history in a transformer policy, with a lightweight decoder for full keyboard and mouse actions. The section also describes training–inference discrepancies and techniques used to reduce them.

3. Policy Model

  • The lightweight action decoder expands one latent action token into the full keyboard and mouse action space, allowing one policy-transformer pass per timestep and about 5× speedup over direct prediction of all action tokens.The model handles up to four simultaneous key presses and two concurrent mouse actions using eight action tokens.Source: §3 · pp. 3–4
  • The policy consumes image, text, ground-truth action, and action-prediction tokens; its custom attention mask prevents the action prediction from accessing the same-step ground-truth action.Ground-truth actions are included because they produce more human-like sustained actions, while an action decoder and larger datasets substantially mitigate observed causal confusion.Source: §3 · p. 4

3.0.1. Leveraging unlabeled data

  • Unlabeled gameplay videos are more abundant than annotated data, motivating their use for training; details and preliminary results are deferred to an appendix.Source: §3.0.1 · p. 5

3.1. Mitigating the Training–Inference Gap

  • Using the same resizing function for training and inference, plus data augmentation, mitigates the input gap and improves online performance.The authors use augmentation in all experiments; they also report smoother control and better online performance from sampling mouse movements using fitted truncated normal distributions.Source: §3.1 · pp. 5–6
  • Compression and resizing create a training–inference input gap, and lossy compression during data collection makes part of that gap irreducible.The gap is smaller with RGB than YUV encoding and grows as compression quality falls.Source: Figure 3 · p. 5, §3.1 · p. 5

4. Evaluation

Evaluation spans controlled Godot environments, human-rated gameplay in commercial games, and a Quake instruction-following maze. Larger models are consistently preferred, and text instructions increase maze success across models.

4. Evaluation

  • The evaluation compares four model sizes using programmatic scores, human preferences, and scaling analyses of test loss and causality.The model sizes range from 150M to 1.2B parameters.Source: §4 · p. 6

4.1. Simple Programmatic Environment

  • The largest model achieves the highest mean score with the lowest variance in the programmatic environments.Hovercraft measures loop-completion time and Simple-FPS measures enemy hits minus hits received; each model is evaluated 16 times per environment.Source: §4.1 · p. 6, Table 1 · p. 6

4.2. Human Evaluation in Real Environments

  • Larger models are consistently preferred in human comparisons, in agreement with the numerical results from the programmatic environments.The human comparison covers 20 game checkpoints per model.Source: §4.2 · p. 6, Figure 4 · p. 7

4.2.1. Instruction-following

  • Text instructions substantially increase maze success for every model, though current instruction following is limited to simple commands similar to those seen during training.The maze task is evaluated from the same checkpoint over five runs; the stated limitation reflects the limited quantity and diversity of training instructions.Source: §4.2.1 · p. 7

5. Causality and scaling laws

The section studies how network depth, model size, training data, and training progress relate to causal behavior in behavior cloning. In both a toy task and scaled models, deeper or larger models and more data generally support more causally grounded behavior, with an exception in extremely data-limited settings.

5. Causality and scaling laws

  • The data-constrained study trained each model/data size for multiple epochs until overfitting or no further improvement, across four parameter scales (1.2B, 600M, 300M, 150M) and five dataset fractions (100%, 50%, 25%, 12%, 6%).For the 1.2B model, the reported estimates were L∞=1.111, α=0.2336, and Dc=17.Source: §5 · p. 7
  • Larger models generally achieve lower test loss, and additional data consistently reduces test loss, with larger models benefiting more in data-abundant regimes.The data-abundant examples include training-data fractions of 50% and 100%.Source: §5 · p. 7

5.1.1. Toy problem

  • In the toy task, randomly initialized linear policies make no causal-learning progress with SGD, while nonlinearity enables progress and greater depth speeds learning.Although a causally optimal policy is representable linearly, the toy observation includes a brake-light correlate that should be ignored.Source: §5.1.1 · p. 8, Figure 6 · p. 8

5.2. Scaled Causality

  • The scaled causality score measures output sensitivity to image frames by comparing predictions on original and perturbed image sequences while conditioning on the action sequence.Frames are perturbed in temporal chunks; training-step scores continue to rise even in overfitting regimes, so the authors recommend interpreting causality scores jointly with test loss.Source: §5.2 · pp. 8–9
  • Causality scores generally rise with model and dataset size, except in the extremely data-limited regime.The reported exception includes 30M frames; Figure 7 summarizes the score across model and dataset sizes.Source: §5.2 · pp. 8–9, Figure 7 · p. 9

6. Related Work

Prior work spans robot-control models and game-playing agents, including systems trained for individual games and generalist systems. The paper positions its work among multi-game approaches while distinguishing its keyboard–mouse PC setting and noting reproducibility barriers in some prior systems.

6. Related Work

  • Robot-control research includes fine-tuned vision–language models and models trained from scratch on robot-centric video and instruction data, with strong results in simulated and real-world tasks.The cited approaches include Driess et al. (2025) and Brohan et al. (2022).Source: §6 · p. 9
  • Prior game-playing work includes agents trained for individual titles and systems using language or vision-language models for planning, code generation, or tool use.Examples span Minecraft, Counter-Strike: Global Offensive, and Genshin Impact.Source: §6 · p. 9
  • Some generalist game systems span multiple environments, but the authors note that several do not release code or datasets, limiting reproduction and further research.The section also cites multi-game agents and more recent general action models.Source: §6 · p. 9
  • The paper's keyboard–mouse PC-game focus differs from Magne et al. (2025), which primarily studies controller-based console games, making the settings complementary.The authors characterize PC games as involving higher-frequency real-time control.Source: §6 · p. 9

7. Conclusion

The paper presents an action- and text-annotated gameplay dataset and a real-time policy model, then studies how model capacity and dataset size relate to causal behavior. Larger models achieve lower test loss and higher causality scores when data are abundant.

7. Conclusion

  • The work presents a large-scale, high-quality dataset of gameplay videos annotated with actions and text.The authors also describe a real-time policy model designed for online gameplay in real-world environments.Source: §7 · p. 9
  • The paper evaluates its policy model through qualitative and quantitative analyses.Source: §7 · p. 9
  • In data-abundant regimes, larger models achieve lower test loss and higher causality scores, with scaling both model size and dataset size and diversity suggested as an approach to causality issues.The paper reports that its causal findings persist as model size and training steps increase.Source: §7 · pp. 9–10

A. Dataset

The section describes the annotated game inventory and its quality-control process, alongside rules for defining and timing macro instructions.

A. Dataset

  • The annotated game inventory includes Call of Duty Zombies, Roblox: Murderers vs. Sheriffs, and Warhammer: Vermintide titles.Source: §A.1 · p. 13, §A.2 · p. 13
  • Automated checks filter recordings for key-hold duration, simultaneous key presses, interaction, and likelihood under a lightweight action policy; flagged videos and a sample are reviewed by humans.The sampling strategy gives higher inspection probability to annotators with fewer accepted hours for a given game.Source: §A.3 · p. 14
  • The authors state that the recording checks were intended to ensure high-quality expert gameplay.Source: §A.3 · p. 14

I. MACRO INSTRUCTIONS￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

  • A macro instruction is defined by an event listed in the GAME-FOCUS RULE table, and its start and end mark when an objective becomes clear and is satisfied.The end is recorded as null if the objective remains unfinished when the video ends.Source: §I · p. 15

II. GAME-FOCUS RULE￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

The instructions define which game events to annotate as macros, which to skip, how frequently and diversely to phrase them, and how to structure the output and evaluate simple test environments.

II. GAME-FOCUS RULE￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

  • The focus rules select strategic events such as major route choices, gear acquisition, tactics, and landmark-based objectives, with game-specific examples.If a game is not listed, annotators map it to the closest family.Source: §II · p. 15

III. GAME-IGNORE RULE￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

  • The ignore rules exclude events deemed insufficiently strategic, including eating small prey and disaster-warning captions.Source: §III · p. 15

IV. EVENT-FREQUENCY RULES￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

  • A global eight-second cooldown separates macros unless a genuine strategy shift appears sooner, and continuous-action games record tactic or target changes rather than repetitive minor events.Source: §IV · p. 16

V. DIVERSITY RULE￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

  • The diversity rule varies verbs, landmark phrases, and sentence forms only after accuracy is assured, and forbids inventing events for variety.Source: §V · p. 16

VI. AVOID RULE￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

  • The avoid rule calls for varied wording and excludes directional instructions such as turning right or left.Source: §VI · p. 16

VII. ON-SCREEN TEXT RULE￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

  • Explicit on-screen text hints or objectives are quoted exactly once and logged as macros only if they meet the four checks.Source: §VII · p. 16

VIII. WORKFLOW￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿

  • The workflow requests a game and win-condition narrative plus a timestamped macro list, and evaluates simple games using lap time or enemy hits minus hits taken.The authors state that they constructed simple games for automated testing with Godot.Source: §VIII · p. 16, §A.5 · p. 17

B. Unlabeled dataset

The section describes a filtered corpus of unlabeled gameplay videos, noting its quality and labeling challenges and its copyright-based non-release.

B. Unlabeled dataset

  • Unlabeled gameplay videos vary in quality, may include non-game content, use different resolutions and frame rates, and lack action labels.Source: §B.1 · p. 17
  • A two-stage commercial-VLM pipeline filters videos using metadata and relevance queries, then segments out non-gameplay intervals from temporally downsampled video.Source: §B.1 · p. 17
  • The authors did not release the unlabeled dataset because they do not hold its copyright.Source: §B.1 · p. 17

C. Policy Model

Jointly training the image tokenizer with the policy model reduces training and validation perplexity. The results indicate that adapting the visual representation is critical for strong downstream policy performance.

C. Policy Model

  • Jointly training the image tokenizer consistently lowers both training and validation perplexity.Figure 11 compares joint training with a frozen tokenizer.Source: §C.1 · p. 18, Figure 11 · p. 19
  • Figure 11 compares training and validation perplexity for a frozen tokenizer and one trained jointly with the policy model.Source: §C.1 · p. 18
  • Adapting the visual representation is critical for achieving strong downstream policy performance.Source: §C.1 · p. 18

D. Evaluation

Evaluation examines scaling trends, causal scores, and models pretrained on pseudo-labeled gameplay. Although pretraining lowers test loss, the pretrained model is not significantly preferred online over its label-only counterpart.

D. Evaluation

  • Human evaluators count six gameplay issues, normalize counts by video length, and multiply by 100; lower scores indicate better performance.Be-a-Shark and Hypershot are excluded from this rubric because their dynamic online environments cannot be reliably controlled across runs.Source: §D.4 · pp. 19–20
  • The Quake text-conditioned maze task requires pressing all three buttons to open the door.Source: §D.5 · p. 22, Figure 12 · p. 22
  • Across four model sizes, test loss closely follows a power law in training frames, and larger models achieve lower loss when dataset size is relatively large.Source: §D.6 · p. 22, Figure 13 · p. 23, Figure 14 · p. 24
  • Causality scores consistently increase during training, so a 600M model trained on 30M samples can exceed the same model trained on 500M at certain steps; lowest-test-loss checkpoint trends align with Section 5.1.Source: §D.7 · p. 25
  • The pretrained 600M policy is fine-tuned on labeled data after one epoch of pretraining on pseudo-labeled unlabeled gameplay, about 10× the annotated dataset size.An inverse dynamics model generates the pseudo-labels; it predicts actions in one pass and may attend to future frames.Source: §D.8.1 · pp. 25–26, §D.8.2 · p. 26
  • IDM metrics on labeled data may overestimate performance and miss effectiveness on unlabeled videos with a substantially different distribution.The authors supplement quantitative evaluation with human inspection of sampled pseudo-labels for plausibility and temporal consistency.Source: §D.8.2 · p. 26
  • On Godot-based programmatic environments, the pretrained 600M model shows only incremental score changes relative to its 600M counterpart.Source: §D.8.3 · p. 26, Table 5 · p. 26
  • Despite substantially lower test loss, the pretrained 600M model was not significantly preferred online over the 600M model trained only on labeled data.The authors hypothesize that unrelated pretraining videos may introduce atypical or game-specific movements that appear less human-like.Source: §D.8.4 · p. 26, §D.8.5 · p. 27, Figure 16 · p. 27

Related papers

Loading 2601.04575v2…