PaperlayerView exact revision on arXiv

Source-linked AI summary

GameBoyWorlds: A Testbed for Self-Improvement in Embodied Video Games

Dhananjay Ashok, Adam Shen, Aslan Huo Feng, Chinmay Khanna, Jun Rui Huang, Raghav Sarmukaddam, Surendira Balaji Natarajan, Xiaotong Cui, Xincan Zhang, Thomson Yen, Hongseok Namkoong, Jonathan May, Jesse Thomason

arXiv:2609.32093v1cs.AI

TL;DR

GameBoyWorlds tests whether embodied agents can learn autonomously from their own experience rather than relying on human guidance or pretrained game knowledge. Across short-horizon execution and unfamiliar-game playthrough evaluations, no self-improvement method consistently raises performance across all games, and the playthrough system fails to reach the first major milestone in either game.

  • Problem

    It remains unclear whether embodied agents can acquire capabilities and game knowledge through self-improvement rather than relying on human artifacts or pretrained domain knowledge.

    Source: §1 · pp. 1–2

  • Method

    GameBoyWorlds evaluates self-improvement through reward-free exploration and short-horizon tasks across game series, plus end-to-end playthroughs of two unfamiliar fan-made Pokémon games.

    Source: §1 · pp. 2–3, §2 · p. 3

  • Results

    No method consistently raises performance across all games, and the playthrough system fails to clear the first major milestone in both games.

    Source: §1 · p. 3, §7 · p. 10

  • Takeaways & Limitations

    GameBoyWorlds provides execution and playthrough challenges for evaluating agents expected to acquire capabilities and knowledge from their own game experience.

    Source: §1 · p. 2, §7 · p. 10

    The playthrough framework implementation is Pokémon-specific.

    Source: §5 · p. 9

Abstract

from arXiv · show

Powered by expert guidance, agents can operate in interactive environments; however, it is unclear whether they can learn autonomously from their own experience. To evaluate such self-improvement methods, we introduce GameBoyWorlds, a testbed for agentic self-improvement in video games. GameBoyWorlds-Execution evaluates task execution on a collection of 5 distinct game series. Agents are allowed access to dedicated training games but are provided no demonstrations, documentation, or rewards. Agents must ground themselves in the environment through self-directed exploration and by inferring actionable knowledge from their own experience. At test time, agents must complete short-horizon tasks that evaluate their ability to navigate, interact, and engage with game-specific mechanics in unseen games. Out-of-the-box frontier models complete fewer than 50% of the 500 tasks due to failures in multimodal grounding, establishing that self-improvement methods have room to push performance. We demonstrate that contemporary approaches to self-improvement are lacking, with world modelling and autonomous skill discovery failing, and a novel strategy that uses curiosity-based exploration to write guides achieving only partial success. GameBoyWorlds-Playthrough tests end-to-end game completion in two fan-made Pok\'emon games. We show that while frontier models have been pre-exposed to official releases such as Pok\'emon Red, they lack essential information on the games in our testbed. Instead of relying on their parametric knowledge to succeed, agents must learn from their own experience and autonomously improve over the course of the playthrough. We show that a sophisticated agentic pipeline with multimodal memory and hierarchical subgoals fails to reach even the first major milestone in both games, establishing GameBoyWorlds as an ambitious target for self-improving agents.

1 INTRODUCTION

GameBoyWorlds tests whether embodied agents can acquire capabilities from their own experience rather than relying on human artifacts or pretrained domain knowledge. Its execution and playthrough evaluations show that current agents and self-improvement methods remain challenged by unfamiliar games.

1 INTRODUCTION

  • GameBoyWorlds addresses whether agents can gain capabilities through self-improvement rather than pretrained knowledge, with an execution testbed and a playthrough challenge in unfamiliar fan-made games.Existing game benchmarks are evaluation-only or train with rewards from test environments.Source: §1 · p. 2
  • Execution training provides access to a game but no task exemplars, demonstrations, documentation, or game-specific rewards, requiring self-directed exploration before short-horizon tasks in distinct games.The evaluation spans 500 tasks across five game series, including navigation, interaction, combat, and game-specific control.Source: §1 · p. 2
  • Frontier models complete fewer than 50% of tasks, with consistent failures in perception and understanding game mechanics.The Playthrough leg tests end-to-end play in two fan-made Pokémon games where agents must learn from experience rather than rely on parametric knowledge.Source: §1 · pp. 2–3
  • Curiosity-driven guides improve task completion by 22% on select games, but performance drops by 30% in the worst case, while the playthrough system misses the first major milestone in both games.World modelling and autonomous skill discovery degrade performance; the guide-based approach achieves only partial success.Source: §1 · p. 3
  • The authors release the testbeds with an emulation system that provides unified interfaces for GameBoy games.Figure 1 distinguishes short-horizon execution from deployment-time self-improvement in unseen games.Source: Figure 1 · p. 2, §1 · p. 3

2 GAMEBOYWORLDS-EXECUTION: EMBODIED LEARNING IN GAMES

GameBoyWorlds-Execution evaluates short-horizon task completion after agents explore training games without demonstrations, documentation, or game-specific rewards. Its benchmark models games as partially observable processes and includes 500 tasks across five series.

2 GAMEBOYWORLDS-EXECUTION: EMBODIED LEARNING IN GAMES

  • Games are modelled as partially observable Markov decision processes with emulator states, button actions, screen-frame observations, transition probabilities, and deterministic observation emissions.Agents receive no extrinsic, game-specific rewards, and training is limited to the first installment in each series.Source: §2 · p. 3
  • The benchmark covers 10 games from five series, with 50 tasks and completion functions per game, for 500 test tasks.The series span distinct mechanics, including top-down exploration, point-and-click interaction, command menus, and maze-based play.Source: §2 · p. 3
  • Task success is verified with frame comparisons, and all tasks are executable by humans.Source: §2 · p. 3
  • Models fail to exceed 50% average task completion on GameBoyWorlds-Execution.Figure 2 reports success rate per game and overall success rate.Source: Figure 2 · p. 4

3 EVALUATING BASELINE TASK EXECUTION CAPABILITIES

The baseline uses a hierarchical supervisor–executor agent to evaluate game-agnostic task execution. Frontier models perform unevenly across games and often fail at basic navigation, interaction, and game-specific mechanics.

3 EVALUATING BASELINE TASK EXECUTION CAPABILITIES

  • A supervisor decomposes tasks into subgoals, checks their completion, diagnoses failures, and can revise the subgoal plan after repeated executor failures.The executor chooses one button press at a time; visual history improved performance relative to alternative schemes.Source: §3 · p. 4
  • The evaluation included Gemini-3.6-Flash, Claude-4.5-Haiku, GPT-5-Mini, Gemma-4-31B, and Qwen3-VL-30B-Instruct.The supervisor and executor share the same VLM and use game-agnostic prompts.Source: §3 · p. 4

3.1 RESULTS

  • Gemini-3.6-Flash achieved the best average performance at 49.3%; Deja Vu peaked at 22%, while GPT-5-mini reached 84% on Bomberman Pocket.Source: §3.1 · p. 4
  • Models handled static menu tasks better but struggled with pathfinding, precise screen alignment, and timely interaction.Source: §3.1 · p. 4, Figure 3 · p. 5
  • Other failure modes included repeated action sequences, poor coordination between agent hierarchies, and insufficient knowledge of game-specific rules.Source: §3.1 · p. 5

4 EVALUATING EXISTING SELF-IMPROVEMENT METHODS

The section evaluates world modeling, autonomous skill discovery, and curiosity-guided documentation as self-improvement approaches. The first two degrade task execution, while curiosity yields small average gains but remains inconsistent.

4 EVALUATING EXISTING SELF-IMPROVEMENT METHODS

  • On Gemma-4-31B, world modeling plans by comparing predicted next screens, while PAE proposes tasks, practices perturbed starts, and behavior-clones successful trajectories.These are standard implementations of self-improvement methods evaluated in the section.Source: §4 · p. 5

4.1 CURIOSITY-BASED EXPLORATION

  • Curiosity serves as a search procedure for gameplay snippets ending in novel observations, which may reveal game-specific mechanics useful for later task execution.The approach adapts curiosity-based exploration because these games cannot be cleared through curiosity alone.Source: §4.1 · pp. 5–6

4.1.1 ITERATIVE CURIOSITY-BASED EXPLORATION AS SEARCH

  • Each episode resets its frame buffer; later generations inherit prior frames, and hyperparameter sweeps retain top-k high-reward runs to seed further novelty search.The intrinsic reward increases when the latest screen differs from previously observed screens.Source: §4.1.1 · p. 6
  • Exploration regularly surfaces behaviors using game-specific mechanics, including behaviors the VLA agent does not know or cannot perform consistently.Source: §4.1.1 · p. 6

4.1.2 INSIGHT DISTILLATION

  • VLMs group top trajectories by similar terminal frames, describe actions and screen changes, filter unidentifiable tasks, then merge actionable insights into scenario-specific guidance for the supervisor.Source: §4.1.2 · pp. 6–7

4.2 RESULTS

  • Curiosity documentation is the only partial success: average task completion rises 2.0% (40.7% →41.5%), but no approach improves consistently.Examples include BombermanPocket (72.0% →88.0%) and test-only SwordOfHope2 (66.0% →72.0%), alongside drops on DejaVu1, DejaVu2, and PokémonCrystal.Source: Figure 5 · p. 7, §4.2 · pp. 7–8
  • World modeling lowers average task completion by 3.4%, while skill discovery cuts it 44.3% (40.7% →22.6%), with greater degradation on test-only games.The authors attribute world modeling's weakness to inaccurate next-observation predictions and hypothesize that skill discovery suffers from inaccurate practice completion judgments.Source: §4.2 · p. 7

5 GAMEBOYWORLDS-PLAYTHROUGH: END-TO-END GAMEPLAY

GameBoyWorlds-Playthrough tests whether agents can learn and improve while completing unfamiliar fan-made Pokémon games. A Gemini-3.6-Flash agent progressed through several milestones in contaminated Pokémon Red but made no meaningful progress in Pokémon Brown.

5 GAMEBOYWORLDS-PLAYTHROUGH: END-TO-END GAMEPLAY

  • The benchmark addresses whether models can self-improve during deployment by acquiring the capabilities needed for long-term gameplay.Playing through requires dynamic multimodal memory, active knowledge acquisition, reasoning under uncertainty, and long-term planning.Source: §5 · p. 8
  • Frontier models recover nearly all classic Pokémon progression steps but less than 30% in the fan-made games, highlighting their limited parametric knowledge of those games.Playthrough success also requires short-horizon execution, long-term planning, active knowledge seeking, dynamic multimodal memory, and continual deployment-time improvement.Source: §5 · p. 8
  • The playthrough agent combines action interfaces and perception modules for navigation, mapping, interaction, subgoals, and accumulated knowledge without game-specific knowledge or privileged information.The implementation is Pokémon-specific.Source: §5 · pp. 8–9
  • In Pokémon Red, Gemini-3.6-Flash reached Viridian City and returned with a parcel before a memory failure loop; in Pokémon Brown, it never left the starting town.Similar divergence appeared between Pokémon Crystal and Pokémon Prism, where the agent quickly obtained a starter only in Crystal.Source: §5 · p. 9, Figure 7 · p. 9

6 RELATED WORK

The related-work discussion distinguishes GameBoyWorlds from benchmarks where pretrained knowledge or engineered interfaces can simplify gameplay. It also contrasts the games’ complex mechanics and long-horizon goals with simpler interactive reasoning environments.

6 RELATED WORK

  • Some Pokémon game harnesses read game memory and engineer state spaces to simplify perception and pathfinding, removing core challenges of perception and interaction.The paper says its results indicate frontier models cannot play complex embodied games without such human-crafted scaffolds.Source: §6 · p. 9
  • GameBoyWorlds targets complex mechanics such as Bomberman escape routes under fuse deadlines and Pokémon Brown goals spanning thousands of steps.The discussion contrasts these demands with fully observable handcrafted puzzle environments, where models have achieved 99.9% on ARC-AGI-3.Source: §6 · pp. 9–10

7 CONCLUSION

GameBoyWorlds evaluates self-improvement in embodied video games through execution tasks and unfamiliar-game playthroughs. The authors report that current methods do not consistently improve performance across games and that the benchmark remains largely unsolved.

7 CONCLUSION

  • GameBoyWorlds combines a 500-task execution testbed with unfamiliar-game playthroughs that require agents to acquire knowledge and skills during deployment.Agents train without explicit tasks, human demonstrations, game-specific external knowledge, or task exemplars.Source: §7 · p. 10

A GAMEBOYWORLDS IMPLEMENTATION

The implementation wraps GameBoy games in a reproducible Gymnasium-compatible emulator interface and detects events primarily from screen captures. It supports adding other GameBoy titles through ROM registration and game-specific parsing.

A.1 EMULATION AND ENVIRONMENT INTERFACE

  • The PyBoy-based package exposes games through Gymnasium, separating ROM, save-state, input, rendering, and event tracking from Gym-compatible environment interfaces.Each episode restores a deterministic save state, making task trajectories reproducible for the same action sequence.Source: §A.1 · p. 14
  • Each step presses one button for 5 ticks, releases it, then waits 14 ticks; agents receive the final 144 × 160 grayscale frame.Intermediate frames are retained for event tracking so brief events are not missed.Source: §A.1 · p. 14
  • Adding a GameBoy or GameBoy Color title requires ROM registration, a default save state, and a game-specific state parser.The package distributes save states and screen captures, but not ROMs; users must obtain ROMs legally.Source: §A.1 · p. 14

A.2 SCREEN CAPTURE SYSTEM

  • Screen parsers detect events by comparing named screen regions with human-recorded reference captures, since many titles lack documented memory layouts.A region matches at a mean absolute pixel difference of at most 0.01; documented titles may also use memory values, though benchmark tasks can be verified from screen alone.Source: §A.2 · p. 14

A.3 TASK IMPLEMENTATION

Execution tasks are hand-built from real gameplay and verified against screen captures; playthrough progress is tracked through badge and champion messages.

A.3 TASK IMPLEMENTATION

  • Each execution task starts from a hand-played initial state and is verified against a screen capture recorded at success.The task is replayed from its initial state to confirm the completion check fires at success, not before.Source: §A.3 · pp. 14–15
  • All tasks are achievable from their initial states, and termination always means success because evaluation provides no failure signals.Tasks are annotated with a natural-language description and category.Source: §A.3 · p. 15

A.4 GAMEBOYWORLDS-PLAYTHROUGH PROGRESS MARKERS

  • Playthrough progress uses one subgoal for each of eight gym badges, followed by defeating the Elite Four to become champion.Each badge and the final completion condition are detected by matching their corresponding dialogue-box messages; both games start after the introduction.Source: §A.4 · p. 15

A.5 LIMITATIONS

Screen-based verification limits the benchmark to tasks whose success produces a distinct, reliably recurring visual signal, favoring events with immediate visual feedback.

A.5 LIMITATIONS

  • A task is verifiable only if every successful attempt produces a distinct, reliable screen; outcomes without such a signal are excluded.For example, an internal variable change that is never displayed cannot be checked.Source: §A.5 · p. 15
  • This constraint biases the task distribution toward events with immediate visual feedback, such as dialogue, menus, and map transitions.Source: §A.5 · p. 16

A.6 GAME-SPECIFIC TASKS

The benchmark spans games with substantially different mechanics, while the evaluated subset and agent framework specify the scope and procedure for task execution.

A.6 GAME-SPECIFIC TASKS

  • Game-specific tasks range from navigation and interaction to menu control, combat, survival, and resource management across the covered series.Examples include menu-mediated interactions in Deja Vu and Sword of Hope, bomb-based combat in Bomberman, card battles in Harry Potter, and farming tasks in Harvest Moon.Source: §A.6 · pp. 16–17

A.7 EVALUATED SUBSET

  • Full benchmark evaluation covers 500 tasks across 10 games in five series; another 500 implemented tasks across five other series are left for future evaluation.Source: §A.7 · p. 17

B ZEROSHOT AGENT

  • The zeroshot agent uses a supervisor for planning and progress judgments and an executor that acts one button press at a time, with game-agnostic prompts.The only game-specific information supplied is the game’s name.Source: B ZEROSHOT AGENT · pp. 17–18

B.1 EPISODE STRUCTURE

  • Each episode allows 75 executor steps, counting invalid or unparsed responses, and ends on environment-verified success or budget exhaustion.Supervisor calls do not consume steps.Source: §B.1 · p. 18

B.2 SUPERVISOR

  • The supervisor plans standalone, visually checkable subgoals, judges intermediate attempts, checks for regressions, revises flawed plans, and otherwise supplies corrective hints.Intermediate subgoals receive up to three failed attempts, and a plan may be revised at most twice per episode.Source: §B.2 · p. 18

B.3 EXECUTOR

  • The executor receives the current screen and task, acts once per step, and maintains a visual history of the five most recent actions.For intermediate subgoals, a completion check follows each action.Source: §B.3 · p. 19

B.4 ALTERNATIVE FRAMEWORKS

The framework alternatives vary supervision, action selection, and history; paired comparisons show modest, mixed advantages rather than clear superiority for the selected framework.

B.4 ALTERNATIVE FRAMEWORKS

  • Visual history averages 25.4% task completion, versus 23.3% with action history and 21.8% with no history, but most differences are only 1–4 tasks.It wins 19 of 27 comparisons against no history and loses 6; Table 1 reports the paired comparisons.Source: §B.4 · p. 20, Table 1 · p. 20
  • Subgoal Planning beats the unsupervised executor in 19 of 27 comparisons, but splits evenly against Error Correction and Deja Vu 2 remains below 14.7% under every combination.Table 2 compares supervisors across combinations of action selection and history.Source: §B.4 · p. 20, Table 2 · p. 20
  • The authors’ overall claim is that alternatives produce comparable or worse results, not that the selected framework is fundamentally superior.Source: §B.4 · p. 20

C SELF IMPROVEMENT METHOD IMPLEMENTATIONS

The self-improvement implementations use curiosity-driven exploration to collect experience and surface game-specific insights, while world modelling predicts future observations and autonomous skill discovery builds task guidance and practice data. These methods use agent-generated interaction data rather than human annotations.

C.1 CURIOSITY-BASED EXPLORATION

  • Curiosity exploration trains a PPO agent on visual and text novelty, then extracts high-reward subtrajectories that often surface game-specific mechanics, dialogue, and information-rich gameplay.Text novelty rewards unseen dialogue or menu strips, while the combined reward weights text and observation novelty; subtrajectory endpoints have reward z-scores above 2.5.Source: §C.1 · pp. 20, 22

C.1.1 CONFIGURATION FOR INSIGHT DOCUMENTS

  • Insight-document exploration runs four rounds with text weights α = 0 and α = 0.75, eight agents per training state, and sweeps γ values of 0.95, 0.99, and 0.995 across three seeds.High-reward subtrajectories are retained at a z-score of at least 6.0.Source: §C.1.1 · pp. 23–24

C.2 WORLD MODELLING

  • World modelling has three stages—experience collection, frame embedding, and next-frame latent prediction—and uses only agent-generated interaction data without human annotation.Source: §C.2 · p. 24

C.2.1 EXPERIENCE COLLECTION

  • For each training game and state, experience collection combines random button presses for broad screen coverage with visual-novelty exploration for states random play is unlikely to reach.Both agents' transitions are stored for training; curiosity exploration uses α = 0 for two rounds, without a hyperparameter sweep or subtrajectory extraction.Source: §C.2.1 · p. 24

C.2.2 OBSERVATION EMBEDDER

  • The observation embedder encodes each frame's 360 non-overlapping 8×8 patches into a unit-normalized 1,440-dimensional latent and decodes them back into a frame.It is trained to reconstruct normalized patches from replay-buffer frames, with 10% held out for validation and early stopping after three epochs without improvement.Source: §C.2.2 · p. 24

C.2.3 WORLD MODEL

  • The world model predicts the next frame's latent from the two most recent frame latents and an action, minimizing mean squared error against the embedder's next-frame latent.It uses all replay-buffer transitions except episode-final steps and applies the same 10% validation and three-epoch stopping rule.Source: §C.2.3 · p. 25

C.2.4 WORLD MODEL EXECUTOR

  • At each step, the world-model executor predicts and decodes a frame for every available action, then asks the VLM to choose the prediction that most directly advances the task.The executor maintains the two most recent frames and operates under the same supervisor and step budget as the baseline executor.Source: §C.2.4 · p. 26

C.3 AUTONOMOUS SKILL DISCOVERY

  • Autonomous skill discovery proposes tasks, attempts them and judges success, infers guidance from successful attempts, practices from perturbed starts, then cleans the data and fine-tunes a VLA.Gemma-4-31B is used at every stage, and attempts and practice episodes use the single action executor with visual history.Source: §C.3 · p. 26

C.3.1 TASK PROPOSAL

For each training state, the VLM receives the initial frame and full action space and proposes tasks that can be attempted from that state. The paper includes a sample of proposals from one state per game.

C.3.1 TASK PROPOSAL

  • For each training state, the VLM is shown its initial frame and the full action space.Source: §C.3.1 · p. 26
  • The VLM is asked to propose an exhaustive list of tasks that can be attempted from the displayed state.Source: §C.3.1 · p. 26
  • Table 3 gives a subset of the tasks proposed from one training state of each game.Source: §C.3.1 · p. 26

C.3.2 TASK ATTEMPTS

Proposed tasks are attempted and judged, successful attempts yield step-by-step guidance, and the resulting tasks are practiced from perturbed starting states. Only tasks judged successful proceed to guidance inference and practice.

C.3.2 TASK ATTEMPTS

  • Each proposed task is attempted from its training state for up to five tries, with a 50-step budget and VLM judgments based on windowed trajectory descriptions and the final eight frames.Failed attempts receive a VLM-written hint for the next try; Table 4 reports proposal and judged-success counts by game.Source: §C.3.2 · p. 26, Table 4 · p. 27

C.3.3 GUIDANCE INFERENCE

  • For each successful attempt, the VLM infers step-by-step guidance tied to visual cues and a visual goal condition for confirming completion.The guidance is consolidated from trajectory windows of at most eight frames.Source: §C.3.3 · p. 26

C.3.4 PRACTICE

  • Each successful task is practiced in three episodes from starts perturbed by five random button presses, with inferred guidance shown to the executor but removed from the resulting VLA training data.A strict goal condition is shown only to the judge; failed practice episodes receive one retry from the same perturbation.Source: §C.3.4 · pp. 26–27

C.3.5 DATA CLEANING AND DATASET CONSTRUCTION

Dataset construction filters successful practice episodes into candidate training examples, removes decisions after task completion, and splits episodes by task for training and validation. Validation also uses a held-out task paraphrase.

C.3.5 DATA CLEANING AND DATASET CONSTRUCTION

  • Successful practice episodes supply candidate training examples, with decisions more than two steps after judged success discarded and each decision filtered for clear visual misdescription or failure to advance the task.The filtering prompt strongly favors acceptance.Source: §C.3.5 · p. 28
  • The VLM generates three task paraphrases, reserving one for validation; episodes are split per task with 20% held out, and guidance and step-count blocks are stripped from prompts.All decisions from an episode remain on the same side of the split, and validation examples use the held-out paraphrase.Source: §C.3.5 · p. 28

C.3.6 VLA TRAINING

This section describes fine-tuning separate game-specific VLA executors and building, consolidating, and using insight documents that turn exploratory high-reward trajectories into task-relevant guidance. It covers playthrough games that retain familiar Game Boy controls while introducing new worlds and mechanics, including Pokémon Brown and Pokémon Prism’s game-specific additions.

C.3.6 VLA TRAINING

  • Game-specific LoRA adapters are separately fine-tuned on cleaned data and deployed as executors under the Section B.2 supervisor.The models use supervised fine-tuning on Gemma-4-31B.Source: §C.3.6 · p. 28

C.4 INSIGHT DOCUMENTS

  • Insight documents begin with high-reward subtrajectories that are cycle-cleaned, filtered at reward z-score ≥6.0, and recursively clustered by similarity of their final frames.The pipeline assumes similar ending frames expose similar game mechanics.Source: §C.4 · p. 28, §C.4.1 · p. 28

C.4.2 TASK INFERENCE

  • For each trajectory group, the VLM infers and validates specific multistep tasks, merges tasks with a common core, and discards groups without a valid task.Up to three subtrajectories per group are sampled, and generic tasks such as “explore the area” are rejected.Source: §C.4.2 · p. 29

C.4.3 INSIGHT EXTRACTION

  • For each task–trajectory pair, the VLM extracts non-obvious advice, assigns task and image categories, and discards pairs with no useful insight.The VLM sees sampled frames and actions; the resulting entries include insights and a representative frame.Source: §C.4.3 · p. 29

C.4.4 DOCUMENT MERGING

  • Per-pair documents are merged in a tournament that combines matching lessons but preserves distinct entries, preventing the merge process from dropping an entry.The VLM defaults to no match to avoid collapsing different insights into vague guidance.Source: §C.4.4 · p. 29

C.4.5 DOCUMENT STATISTICS

  • The reported document statistics track pipeline sizes, including valid deduplicated tasks, pairs yielding insights, and separate image- and task-section entries.The two final sections largely overlap in insights, organized under different categories; an example warns that exiting a room can interrupt an unfinished enemy encounter.Source: §C.4.5 · pp. 29, 32, Table 5 · p. 30

C.4.6 TEST-TIME USAGE

  • At test time, the supervisor judges entry relevance from the task and screen, filters unrelated advice, and distills the remaining insights into an ordered list reused throughout the episode.If no entry is relevant, the episode proceeds as with the zeroshot supervisor.Source: §C.4.6 · pp. 32–33

D GAMEBOYWORLDS-PLAYTHROUGH

  • The playthrough games are ROM hacks that retain Game Boy controls and screen formats while adding new worlds, stories, and mechanics.Pokémon Brown is based on Pokémon Red and Pokémon Prism on Pokémon Crystal; the games share a continuous story and overlapping geography.Source: D GAMEBOYWORLDS-PLAYTHROUGH · p. 33
  • Brown is a Generation I hack of Red set in Rijon, where the player earns eight gym badges, drives out Team Rocket, and eventually wins the Rijon League.After the League, a new Team Rocket leader reveals a plan that sets up Prism’s protagonist being stranded in Naljo.Source: D GAMEBOYWORLDS-PLAYTHROUGH · p. 33
  • Brown expands Red’s roster beyond its original 151 Pokémon with Pokémon from Generations II and III, adds the Wood type, and changes some Pokémon’s types and battle matchups.Source: D GAMEBOYWORLDS-PLAYTHROUGH · p. 33
  • Prism, built on Crystal, follows Lance’s child through Naljo to stop the Palette Patrollers, then opens into Rijon, Johto, and Kanto for a 20-badge quest.The plot concerns the group’s plan to awaken and corrupt ancient guardian Pokémon; Crystal has 16 badges.Source: D GAMEBOYWORLDS-PLAYTHROUGH · pp. 33–34
  • Prism adds Gas and Sound types to Crystal’s 17 and lets the player shrink and control their Pokémon in the overworld to reach places and solve puzzles.The Pokémon-controlled mode can use a side-scrolling view; in Crystal, the player walks only as the trainer outside battles.Source: D GAMEBOYWORLDS-PLAYTHROUGH · p. 34

D.1 CONTAMINATION CHECK

This section reports a contamination check comparing factual accuracy on sparsely documented ROM hacks and official titles, and documents the agents’ prompt workflows, planning and supervision, and action/perception components. It also specifies experimental hardware and the authors’ use of LLMs.

D.1 CONTAMINATION CHECK

  • GPT-5.6-sol answered all 200 official-game questions correctly but only 58 of 200 ROM-hack questions; Claude Opus 5 and Gemini 3.5 Flash-Lite scored at least 92% on each official title and 8%–16% on hacks.Strict human accuracy was reported separately for each game.Source: Table 7 · p. 35, §D.1 · p. 35
  • Model accuracy spans 39.5 percentage points on official titles but only 17.0 on ROM hacks, where Claude Opus 5, Gemini 3.5 Flash-Lite, and Gemma 4 31B differ by just two points.Thus, much of the model separation on official games does not persist on the less-documented games.Source: §D.1 · p. 35
  • Pooled official-versus-ROM-hack accuracy gaps ranged from 48.5 to 83.0 percentage points, with Fisher exact tests rejecting independence for every model at p < 10^-24.The authors conclude that the ROM hacks are substantially less contaminated; closely matched terminology, mechanics, and question formats make prior corpus exposure the most consistent explanation for the gap.Source: §D.1 · pp. 35–36, Table 8 · p. 36

D.2 AGENT DESIGN

  • The agent separates short-horizon execution by a supervisor from a strategist that maintains goals, game knowledge, and location across episodes, using only screen-derived experience rather than emulator memory.Its persistent state comprises a goal tree, knowledge tree, location store, and notepad; episode results and end screens update these artifacts for subsequent tasks.Source: §D.2 · pp. 36, 39
  • For movement, the agent classifies screen tiles into a walkability map and uses breadth-first search to route to a visually identified target; its location graph records only crossings already made.The map also surfaces places with few known exits as exploration frontiers, but cannot route to a merely described location before the agent has reached it.Source: §D.2 · pp. 37, 39

E COMPLETE SET OF PROMPTS

E.1 ZEROSHOT AGENT

  • The zero-shot prompts require steps to describe visible objects without naming controls and end in visually checkable conditions; completion and plan judgments rely on screen evidence.Prompts separately assess visible changes, verify completion from the final screen, detect regressions, and distinguish a flawed plan from repeated execution failure.Source: §E.1 · pp. 40–42
  • The zero-shot agent is prompted to diagnose failed actions from the current screen, give control-specific hints with visible stopping cues, and check whether tasks are fully complete.Additional prompts supply hints and recent action effects to the executor and warn it to reconsider repeated actions that produce no change.Source: §E.1 · pp. 42–44

E.2 WORLD MODELLING

  • The world-model executor compares reconstructed post-action screens and selects the predicted outcome that most directly advances the task.The prompt cautions that predictions are blurry and should be judged by overall change rather than fine detail.Source: §E.2 · p. 45

E.3 AUTONOMOUS SKILL DISCOVERY

  • Autonomous skill-discovery prompts enumerate concrete goals from the initial state and judge full trajectories for task success, including the earliest safe success frame.They also elicit failed-attempt critiques and actionable next-attempt hints from trajectory summaries.Source: §E.3 · pp. 46–48
  • Further skill-discovery prompts turn observed trajectories into visually grounded, chronological guidance and reject only decisions that clearly misdescribe frames or hinder progress.Guidance specifies visual cues for when to act and when to move on; decision filtering defaults to accepting uncertain cases.Source: §E.3 · pp. 48–49

E.4 INSIGHT DOCUMENTS

  • Insight-document prompts infer, validate, and distill specific tasks, then extract concrete, non-obvious gameplay insights and merge entries only when their lessons are highly similar.Merged insight lists must preserve distinct actionable details rather than generalizing them into vague guidance.Source: §E.4 · pp. 50–53
  • Insight prompts filter entries by visual fit, retain anything plausibly useful for the full task, and consolidate kept knowledge without adding unsupported claims.Filtering compares current and representative frames; distillation merges specifics, removes redundancy, preserves contradictions, and retains knowledge about later locations.Source: §E.4 · pp. 54–55

E.5.1 EPISODE LEVEL

  • The episode-level strategist dispatches one achievable task with needed guidance, then summarizes the player’s report for its log.The task must be reachable from the current screen within the action budget and the player’s capabilities.Source: §E.5.1 · pp. 55–56

E.5.2 GOAL TREE

  • The goal tree supports querying game knowledge, decomposing goals into ordered subgoals, checking completion, and continuing, updating, or abandoning unfinished goals.Queries must stand alone without situation-specific context; subgoals describe in-game objectives rather than button presses.Source: §E.5.2 · pp. 56–58

E.5.3 KNOWLEDGE TREE

  • The knowledge tree extracts durable episode facts and organizes them through navigation, known-or-update-or-new decisions, entry creation, and relevance-based recall.Facts must stand alone and focus on information likely to matter later, such as locations, items, outcomes, and story developments.Source: §E.5.3 · pp. 58–60

E.5.4 LOCATION STORE

  • The location store identifies places and their kinds, detects place transitions, records destinations and connections, and selects mapped routes fitting a request.It distinguishes major locations, connecting bridges, and internal rooms, and marks a place Unknown when the screen does not show one or enough evidence is unavailable.Source: §E.5.4 · pp. 60–62

E.5.5 NOTEPAD

  • The notepad records working thoughts about goal progress, updating them by appending or rewriting and extracting durable facts when goals close.Notes retain what worked, failed, and what to try next, while excluding step-by-step narration and information no longer useful.Source: §E.5.5 · pp. 62–63

E.5.6 CONTEXT BLOCKS

  • Context blocks supply recent episode history, retries for unparsable replies, goal-status information, optional route lookups, and an optional abandon choice.Route lookups are offered only when the player’s current place is known, while abandonment is available only for goals below seeded milestones.Source: §E.5.6 · pp. 63–64

E.6 PLAYTHROUGH AGENT: SUPERVISOR

  • The supervisor uses a fixed set of action primitives and prompts to break tasks into checkable subgoals, then choose one next action from the current screen.Available primitives include movement, interaction, battle, menu, dialogue, and conditional free actions; menu actions work only in the main menu or an already-open menu.Source: §E.6 · pp. 64–66
  • The supervisor checks subgoal and task completion, records outcomes, replans when stuck, and clarifies ambiguous targets or menu and battle objectives.Its prompts use screen images and recognized tiles, with logs or prior attempts included where relevant.Source: §E.6 · pp. 66–68

E.7 PLAYTHROUGH AGENT: EXECUTORS AND PERCEPTION

  • The action and perception prompts select movement, interaction, battle, and menu actions, while classifying screen tiles and supplying coordinate or neighborhood context.Tile classes include terrain, entrances, objects, items, NPCs, and special movement tiles; interaction prompts restrict selection to adjacent cells.Source: §E.7 · pp. 69–72

F HARDWARE

  • Open-source LLM experiments used nodes with 8xA40 GPUs, with some results on 4xH200 nodes; API-model results used OpenRouter without GPU access, while exploration and model training primarily used 1xV100 nodes.Source: F HARDWARE · p. 73

G USE OF LLMS

  • LLMs assisted with code implementation, some appendix writing, and error and spell checking, but not with project design or interpretation of results.The authors state that no design decisions were made unilaterally by the LLM and that appendix text was checked for veracity.Source: G USE OF LLMS · p. 73
Loading 2609.32093v1…