Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds

• Interactive games provide a demanding test of embodied generality because their visuals, rules, physics, action spaces, and goals vary across worlds.Source: §1 · p. 1
• The controlled study spans 120 configurations across four model sizes from 0.8B to 27B, six data budgets from 50 to 1,600 hours, and five mixtures from Minecraft-only to broad multi-game corpora.Source: §1 · p. 2
• More relevant data improves in-corpus performance, diversity improves transfer to related worlds but offers limited gains in distant worlds, and larger models improve data efficiency but generalize worse under single-game training.Source: §1 · p. 2
• An early interface phase improves shared action patterns across worlds, followed by a world phase with gains in trained and unseen related worlds; the fitted scaling law reaches R2 = 0.982 on Minecraft and R2 = 0.995 on in-corpus loss.Source: §1 · p. 2
• Our models are competitive with Minecraft specialists and frontier models in 3D worlds, and outperform frontier models in real-time settings by up to 5×.Source: §1 · p. 2

• CPT has become a common way to turn general backbones into interactive virtual embodied models, following environment-specific specialists and broader pixel-to-action foundation models.Source: §2 · p. 3
• Games offer a controlled testbed across familiar in-corpus worlds, unseen 3D worlds, and visually or mechanically distinct environments, but prior work had not isolated what drives transfer.Source: §2 · p. 3
• Prior CPT studies change model size, training data, and duration together, despite evidence that source composition, task diversity, negative transfer, and capacity-dependent interference can matter.Source: §2 · p. 3

3.1 Data construction

• At each step, the model conditions on prompt context, observation, and interaction history, then samples a possibly empty reasoning trace and an executable action.Source: §3.1 · pp. 3–4
• Actions comprise discretized mouse movement and scroll deltas plus key or button chunks, with observations at 5 Hz and controls at 30 Hz.Source: §3.1 · p. 4
• The study evaluates 120 configurations across four Qwen3.5 backbones (0.8B, 2B, 9B, 27B), six data budgets (50, 100, 200, 400, 800, 1600), and five data-diversity mixtures.Source: §3.1 · p. 4
• Five mixtures vary Minecraft's share of gameplay trajectories from 100% to 0%, with other games filling the remainder; 15% general data is also included.Source: §3.1 · p. 4

3.2 Training and Evaluation

• Training and evaluation use causal next-token prediction over assistant responses, update all parameters, and hold settings fixed apart from data volume, diversity, and base model.Source: §3.2 · p. 4
• Evaluation covers in-corpus Minecraft and DOOM, seven unseen related 3D worlds with familiar action primitives, and unseen distant games with substantially different visuals, physics, controls, and objectives.Source: §3.2 · p. 4

4 Scaling and Generalization in Heterogeneous Virtual Worlds

• The study evaluates three scaling axes across 120 configurations using training and validation losses and benchmarks spanning in-corpus, related, and distant worlds.Figure 3 organizes losses by model size, mixture, and checkpoint, with a final column comparing fitted and observed losses.Source: §4 · p. 4, Figure 3 · p. 5

4.1 Data Scaling

• After 50 hours, validation losses fall below base-model levels across all three validation sets, but later gains depend on relevant training worlds.From 50 to 1,600 hours, mean Minecraft validation loss decreases about 10.4% under S1–S3; more data without sufficient Minecraft trajectories yields negligible further reduction in that loss.Source: §4.1 · pp. 5–6, Figure 4 · p. 6

4.2 Diversity Scaling

• At 1,600 hours, mixed-game training yields a mean OGA score about 2.8× that of Minecraft-only training, but gains remain inconsistent on distant GameWorld.All 16 mixed configurations outperform S1 on OGA at matched model scales; mixed training mitigates, but does not eliminate, GameWorld declines.Source: §4.2 · pp. 6–7

4.3 Model Scaling

• Larger models achieve comparable or superior MCU and OGA performance with up to 32× less gameplay data, yet 27B-S1 falls below its base model on all three non-Minecraft benchmarks.Its maximum GameWorld drop is roughly 17 points; broadening training data mitigates the loss, though distant-world generalization remains challenging.Source: §4.3 · p. 7

4.4 An Empirical Three-Factor Scaling Law on CPT Loss

• The empirical scaling law fits training, Minecraft, and in-corpus losses with R2 values of 0.980, 0.982, and 0.995, respectively, but OOD loss does not follow the law.The fitted law relates validation loss to model size, gameplay hours, and diversity; OOD gains over the base model are already reached at 50 hours, then rise slightly with further training.Source: Figure 3 · p. 5, §4.4 · p. 7

Comparison with Minecraft game models

• GameScaling-0.8B-S1 surpasses 7B Minecraft specialists, while 27B-S1 outperforms the best specialized model by 13 points.The 0.8B-S3 model completes 36% of instruction-conditioned MCU tasks; behavior-cloning models trained on tens of thousands of hours score near zero on those tasks.Source: §5 · p. 8

Cross-benchmark reference

• Under time-paused play, 9B and 27B GameScaling models match roughly 300B open models in 3D worlds, but frontier models score twice as high on distant GameWorld.CPT amplifies the 27B backbone’s score by roughly 6× on MCU and 2.6× on OGA; in GameWorld, CPT improves only the 0.8B and 2B scales.Source: §5 · pp. 8–9, Table 2 · p. 9
• In real-time ViZDoom and MCU, GameScaling 0.8B–9B models retain 88% of time-paused performance and outscore the best frontier models several times over.Frontier models lose approximately 90% of their score on those benchmarks when actions are delayed; the penalty is roughly halved on exploration-heavy OGA.Source: §5 · p. 9, Table 2 · p. 9

6.1 Scalability and Generalization Depend on Virtual Worlds

• At 1,600 hours, diverse training produces a mean OGA score about 2.8× that of Minecraft-only training, but these gains fail to generalize consistently to distant worlds.The discussion concludes that training mixtures should closely mirror target environments when data are limited; a strong in-domain loss fit does not guarantee distant-world play.Source: §6.1 · p. 9

6.2 Games Require Both Ability and Speed

• GameScaling-9B-S3 averages 25.3 across four real-time benchmarks, exceeding the best-tested frontier model’s 19.2.The discussion notes that delayed actions can become obsolete in unpaused worlds and that real-time competence reflects both gameplay ability and response latency.Source: §6.2 · p. 10

6.3 Toward a General Game Model

• CPT uses behavior cloning without direct environmental feedback on task success or failure, and the generalization gap may reflect imitation, limited game coverage, or both.The authors propose investigating broader world diversity, online feedback during play, or a combination of these approaches.Source: §6.3 · p. 10

Ethics Statement

• Training uses VPT, D2E, and OpenP2P trajectories cleaned by providers to protect player privacy; no new gameplay recordings or personal data were collected.The authors identify no additional ethical concerns arising from these procedures.Source: Ethics Statement · p. 10

A.1 Dataset Construction and Mixture Accounting

• S1–S5 shift Minecraft’s gameplay share from 100% to 0% while holding the non-Minecraft pool fixed, separating world proportions from the number of worlds.S2–S4 draw from the same 80 canonical labels; S5 removes residual Minecraft trajectories as a source-free control.Source: §A.1 · p. 14
• D measures mixture diversity as the effective number of equally weighted worlds with the same Shannon entropy, combining world count and exposure balance.S2–S4 increase diversity by reallocating Minecraft’s dominant share; S5 is nearly as diverse as S4.Source: §A.1 · p. 14, Table 3 · p. 15

A.2 Sparse Retrospective Reasoning Annotations and Action Template

• Retrospective rationales describe the player’s apparent observation, intent, and action without changing the original keyboard–mouse supervision.Rationale density and length differ only marginally across S1–S5, so reasoning supervision is not an additional diversity variable.Source: §A.2 · pp. 16–17

A.3 Full Accounting of the 120-Condition Factorial Grid

• The 120-condition grid reuses 20 WSD trunks to produce six completed budget checkpoints per model–mixture pair, with each budget including warmup, stable training, and decay.Shorter-budget checkpoints branch from earlier trunk checkpoints; the 1,600-hour condition uses the final decay endpoint.Source: §A.3 · p. 17

A.4 Training Hyperparameters and Optimization Stability

• Every condition uses bf16 AdamW, a 2,359,296-token target batch, a fixed 15% vision–language share, 854 × 480 frames, 20-frame context, and seed 3407.The language-model peak learning rate is 1 × 10^-5; the unfrozen vision encoder uses 7 × 10^-6.Source: §A.4 · p. 17
• WSD allocates 3% of training to warmup, 87.3% to stable training, and the final 9.7% to cosine decay.All three phases count toward the reported training-hour budget.Source: §A.4 · p. 17, Table 7 · p. 18
• In the illustrated S4 window, only two of five consecutive turns carry rationales; image tokens receive no loss, while thought and action tokens are trained autoregressively.Each action packet contains mouse displacement and keys held across six 33 ms sub-steps.Source: Figure 5 · p. 18

B.1 Benchmark Evaluation Environments

• The benchmark ladder runs from Minecraft and ViZDoom source worlds through seven unseen OGA worlds to distant GameWorld tasks whose styles, physics, and controls depart from training.ViZDoom tests a rare game but common first-person aiming and firing skills; GameWorld is mostly 2D and 2.5D.Source: §B.1 · pp. 18–19

B.2 Evaluation Protocol

• For MCU, ViZDoom, and GameWorld, adapters translate action packets into native events while leaving benchmark task definitions and scoring parameters unchanged.Scores average task means over episodes; Table 2 compares models evaluated by the authors under a shared interface, prompts, resolution, history, and episode counts.Source: §B.1 · p. 19, §B.2 · p. 19

B.3 ViZDoom Scoring Protocol

• ViZDoom’s reported score is the unweighted mean of four per-map scores after each is divided by 100.Battle maps use frag counts and defend maps use cumulative engine reward; each map score averages 10 episodes.Source: §B.3 · p. 19

B.4 Serving and Deployment Configurations

• Real-time scores include decision latency; when latency substantially exceeds 200 ms, performance degrades by more than 5×, and the 27B model shows an obvious drop.The serving layout uses TP = 1 for 0.8B/2B and TP = 4 for 9B/27B; time-paused runs reserve eight H200 GPUs.Source: §B.4 · p. 20, Table 8 · p. 20

C.1 Frontier Progress by Release Date

• Frontier-model gains across the release range can exceed game-specific CPT gains on a fixed backbone, especially on GameWorld, where CPT gains are limited.The models are evaluated under time-paused interaction.Source: §C.1 · p. 20, Figure 6 · p. 21

C.2 Loss Scaling Against Model Size

• Larger models generally have lower Minecraft and mixed-training in-corpus losses, while mixed training limits later OOD-loss increases and single-game training can weaken the size advantage.Figure 7 organizes losses by model size and training duration, with rows for training, Minecraft, in-corpus, and OOD losses.Source: §C.2 · p. 21, Figure 7 · p. 21

C.3 Benchmark Scaling Against Gameplay Hours

• 9B and 27B gains flatten after 100–400 h; S1 declines on OGA and ViZDoom for those sizes, and every 27B mixture declines on GameWorld.Figure 8 plots MCU, ViZDoom, OGA, and GameWorld scores against gameplay hours across four backbone sizes and mixtures S1–S5.Source: §C.3 · p. 22, Figure 8 · p. 22

C.4 Test-Time Scaling on MCU

• At 3,000 steps, GameScaling-0.8B-S1 and 9B-S1 reach 59.3% and 69.3% cumulative success, up from about 42% and 51% at 600 steps.Success rises roughly log-linearly through the tested 3,000-step budget, with no sign of a plateau.Source: §C.4 · pp. 22–23, Figure 9 · p. 23
• The curves remain nearly parallel, preserving an approximately 10-point size gap; untrained Qwen3.5-0.8B completes no tasks, while Qwen3.5-9B reaches only 4.0% at 3,000 steps.The 9B backbone rises from about 2% at 600 steps to 4.0% at 3,000 steps.Source: §C.4 · p. 23

C.5 Inference Latency and Output Length

• GameScaling models take 0.13–0.69 seconds per step, versus 4.0–23.7 seconds for API-served frontier models, and the 0.8B and 2B models decide within the 200 ms action chunk.GameScaling generates 35–39 tokens per step, compared with 123–1,027 for frontier models at their default reasoning efforts.Source: §C.5 · p. 23

C.6 More Experiments on Scaling Law

• OOD loss rises with training hours in every grid cell, so the authors use Eq. (3) only as a negative control rather than modeling OOD dynamics.Across S2–S5, observed loss increases by 0.002–0.003 per doubling; the fitted data term vanishes, consistent with the raw curves.Source: §C.6 · pp. 24, 27
• The fitted law assigns mixture dependence to a floor term and a share-dependent data term, while parameter uncertainty limits interpretation of individual exponents and asymptotic floors.The validation-loss fit has R² = 0.982 on Minecraft and 0.995 in-corpus; α is loosely determined, and β trades off against E along a ridge.Source: §C.6 · pp. 24, 25, 28
• R² = 0.982 for Minecraft and 0.995 for in-corpus loss; Eq. (3) fits all five mixtures and predicts held-out conditions across the grid.Its hold-out errors are 1.3–1.8 times the in-sample errors; predictions extend about one doubling in model size or data budget.Source: §C.6 · pp. 25, 27, Table 13 · p. 27
• The three-factor law outperforms an equal-parameter categorical-mixture model, which has about twice its RMSE on both validation losses and cannot predict unseen mixtures.The categorical model’s RMSE is 0.0044 against 0.0020 on Minecraft and 0.0035 against 0.0012 in-corpus; Eq. (3) predicts held-out S2 or S3.Source: §C.6 · pp. 26–27, Table 12 · p. 27

D Comparison with Related Scaling Studies

• The grid spans 120 configurations across four model sizes, six data budgets, and five mixtures, with about 45,000 H200 GPU-hours supporting a common evaluation protocol.The study connects scaling within training worlds with transfer to unseen related and distant worlds.Source: D Comparison with Related Scaling… · p. 29, §D.2 · p. 29

D.1 Language-Model Scaling Studies

• GameScaling reports 2.2 × 10^22 FLOPs, about 4.7× the estimated CMR budget and of the same order as D-CPT and a dense-equivalent estimate for When Data Is Scarce.The related-work budgets are estimates, not an exact cost ranking.Source: §D.1 · p. 29, Table 15 · p. 30, §D.2 · p. 30

D.2 GUI and Robotics Scaling Studies

• Unlike the listed robotics studies’ fixed model sizes, GameScaling combines controlled variation in model size, data volume, and world mixture.The comparison covers GUI and robotics studies’ model sizes, data coverage, experimental variables, and resources.Source: §D.2 · pp. 29–30, Table 16 · p. 30