PaperlayerView exact revision on arXiv

Source-linked AI summary

Mastering PokeGym: Graph-Guided Multimodal Evolution at Test Time

Ruizhi Zhang, Ye Huang, Yuangang Pan, Chuanfu Shen, Zhilin Liu, Ting Xie, Haijun Lei, Lixin Duan

arXiv:2604.08340v2cs.CVcs.AI

TL;DR

Vision-language agents remain limited in visually driven, long-horizon 3D games, while existing test-time learning methods often optimize modalities separately. The paper introduces PokeGym and G-EvoMAC to evaluate and jointly evolve visual perception, strategy, and action macros across episodes, achieving a 60.18% average success rate and surpassing the strongest baseline by 11.76 points.

  • Problem

    Existing benchmarks and test-time learning methods inadequately evaluate or jointly optimize agents’ configuration across consecutive episodes in visually driven, state-free 3D games.

  • Method

    The paper introduces PokeGym and G-EvoMAC, a graph-guided framework that jointly evolves visual perception, strategy, and action-macro sets at test time.

  • Results

    60.18% average success rate on PokeGym, surpassing the strongest baseline by 11.76 points.

  • Takeaways & Limitations

    Across Navigation, Interaction, and Localization, jointly optimizing visual perception, strategy, and action macros outperforms evolving modalities in isolation or relying on hand-designed modules.

  • Takeaways & Limitations

    The paper states that limitations remain and motivate future work.

Abstract

from arXiv · show

While artificial intelligence has mastered structured games like chess and Go, vision-language agents still struggle in visually-driven 3D games without access to game states. Existing game environments typically evaluate a fixed agent configuration, rather than an agent's ability to improve its configuration across consecutive episodes of the same task---a paradigm known as test-time learning (TTL). Furthermore, current TTL methods typically optimize single modalities---such as text prompts or actions---in isolation, ignoring the synergy between perception, reasoning, and control. To bridge these gaps, we first introduce \textbf{PokeGym}, a long-horizon benchmark built upon the 3D open-world game Pokémon Legends: Z-A, where agents act from visual observations without access to game states, designed to evaluate an agent's ability to learn and adapt across consecutive episodes of the task. To tackle this challenging environment, we propose Graph-Guided Evolutionary Multimodal Agent Configuration (\textbf{G-EvoMAC}), a graph-guided framework that jointly optimizes visual perception, strategy, and action set synergistically. Extensive experiments show that G-EvoMAC achieves a 60.18\% average success rate on PokeGym, outperforming the strongest baseline by over 11 percentage points, validating the power of cross-modal co-evolution.

Introduction

The introduction identifies the difficulty of test-time learning for vision-only agents in complex 3D games and presents PokeGym and G-EvoMAC to address it. G-EvoMAC jointly evolves modalities and achieves a 60.18% average success rate on PokeGym.

  • Motivation: Vision-language agents perform sub-humanly in dynamic, perception-rich games requiring visual cues without access to game states.Earlier evaluations focused on text-based or 2D games.
  • Motivation: Online fine-tuning and reinforcement learning are poorly suited to rapid in-session adaptation because they require millions of samples and costly gradient updates.These limitations hinder real-time decision-making within a single playthrough.
  • Research Gap: Existing self-evolving frameworks optimize language or action components separately, neglecting visual perception and cross-modal synergy.This separation fails to address the coupled capabilities required in visually driven games.
  • PokeGym: PokeGym is a visually driven, long-horizon benchmark based on Pokémon Legends: Z-A that evaluates configuration refinement across consecutive episodes.Agents navigate urban environments, interact with humans and Pokémon, and complete multi-stage quests from visual observations without game states.
  • G-EvoMAC: G-EvoMAC jointly optimizes visual perception, strategy, and action-macro sets at test time using graph-guided cross-modal modeling.Its design also includes frame enhancements, a performance predictor, and a selector balancing exploration and exploitation.
  • Results: 60.18% average success rate is achieved by G-EvoMAC on PokeGym, surpassing the strongest baseline by 11.76 points.The result supports the effectiveness of jointly evolving multimodal agent configurations.

Related Work

Prior benchmarks span text-based, 2D, grid-world, and open-ended RPG environments, but often do not capture the challenges of 3D test-time learning. PokeGym addresses this gap, while G-EvoMAC extends agent evolution beyond textual prompts and tool sets to multimodal configuration.

  • Game and RPG Benchmarks: Existing game benchmarks support long-horizon planning, reasoning, progression, resource management, crafting, and exploration across text-based, 2D, grid-world, and open-ended RPG settings.Examples include NetHack, TextWorld, Pokémon Red, StarDojo, MineDojo, and Cradle.
  • Game and RPG Benchmarks: PokeGym is a 3D Pokémon-based benchmark featuring complex urban navigation, dense social interactions, and multi-stage narrative planning.It is designed to push current test-time learning capabilities.
  • Agent Configuration Evolution: Prompt-evolution methods optimize prompts through discrete, black-box, mutation, or crossover-based search rather than changing broader agent configurations.APE and OPRO use discrete or black-box search, while PromptBreeder and EvoPrompt use LLM-driven mutation and crossover.
  • Agent Configuration Evolution: AgentSquare searches modular planning, reasoning, memory, and tool-use combinations, while EvoTest evolves prompts, memory, hyperparameters, and tool-use routines across repeated episodes.These approaches extend optimization from individual prompts to broader agent configurations.
  • Agent Configuration Evolution: G-EvoMAC jointly configures visual enhancement, strategy, and macro actions, using a learned GNN surrogate and selector to avoid exhaustive emulator evaluations.Prior methods primarily optimize textual prompts and tool sets and typically rely on LLM-as-judge rewards or direct evaluation of every candidate.

PokeGym Benchmark

PokeGym is a visual, long-horizon benchmark built on Pokémon Legends: Z-A that evaluates whether agents improve their configurations across consecutive episodes. It covers diverse 3D open-world tasks using multimodal observations, discrete action sequences, and automatic success verification.

  • Environment: Pokémon Legends: Z-A provides a controllable, reproducible 3D urban open-world testbed with dense scenes, dynamic actors, changing viewpoints, and diverse multi-stage quests.These properties create unique visual challenges while preserving clear success signals.
  • Benchmark Objective: PokeGym evaluates test-time learning by measuring agents’ ability to improve performance through configuration adjustments across consecutive episodes of the same task.This differs from game environments that evaluate a fixed agent configuration.
  • Task Design: 114 tasks span navigation, interaction, and target localization, starting from pre-configured saves and ending at success or step exhaustion within 360 environment steps.An automatic evaluator verifies completion by locating task-relevant states.
  • Interface: At each step, agents receive four RGB observations and output an ordered sequence of up to three high-level discrete commands.The observations include current, previous, left-view, and right-view frames; commands include MoveForward, RotateRight, and PressA.
  • Evaluation: Performance is evaluated using success rate and ineffective move rate, with episode-level score labels used to rank configurations and train a GNN predictor.Success rate measures completed episodes, while ineffective move rate measures movement steps without spatial displacement.

The G-EvoMAC Framework

G-EvoMAC jointly evolves visual enhancement, language strategy, and macro-actions, then uses a graph-based surrogate and disagreement-aware selection to guide emulator evaluations across P rounds.

  • Overall Framework: G-EvoMAC represents each agent configuration as c = (V, L, A), covering visual enhancement, language strategy, and macro-action set.Its loop iterates among candidate generation, cross-modal performance prediction, and selective evaluation rather than exhaustively testing every candidate.
  • Multimodal Configuration Evolver: The Multimodal Configuration Evolver summarizes rollout histories and cross-analyzes B trajectories to propose multimodal changes addressing perceptual bottlenecks, reasoning errors, and recurring deadlocks.It generates M candidate configurations per round, including per-frame image processing, high-level strategy guidelines, and executable composite action macros.
  • Cross-Modal Synergy Performance Predictor: The Cross-Modal Synergy Performance Predictor estimates candidate quality without emulator execution by encoding modalities with SigLIP2 and representing configurations as heterogeneous compatibility graphs.The graph contains language, visual, and action nodes initialized with normalized embeddings, with cross-modal edges weighted by pairwise compatibility.
  • Cross-Modal Synergy Performance Predictor: A Graph Attention Network surrogate processes the configuration graph, applies multi-head attention, pools node states globally, and maps the resulting representation to a predicted score.The predictor is continually refined by appending evaluated configurations and observed scores to its training set over P evolutionary rounds.
  • Disagreement-Aware Selector: MC-Dropout disagreement combines predicted mean and uncertainty to select the highest-acquisition-score candidate for emulator execution in each round.The selected configuration’s ground-truth score is added to the training set, while λ ≥0 controls the exploration-exploitation trade-off.

Experiments

Experiments show that G-EvoMAC substantially improves PokeGym performance through multimodal co-evolution, graph-based prediction, and disagreement-aware candidate selection. Ablations and transfer results support complementary modality contributions and generalization beyond PokeGym.

  • Main Results: 60.18% average SR and 5.55% average IMR let G-EvoMAC surpass the strongest baseline by 11.76 percentage points in SR on PokeGym.GPT-5.4, the strongest zero-shot VLM, reaches only 37.19% SR.
  • Main Results: 48.42% SR and 7.40% IMR are achieved by Voyager, while EvoTest and PromptBreeder reach around 47–48% average SR; G-EvoMAC remains more than 11 points ahead.The lead is consistent across Navigation, Interaction, and Localization.
  • Ablations: 60.18% to 39.82% average SR follows removal of visual enhancement, while removing actions yields 50.35% SR and 7.81% IMR and removing strategies yields 47.02% SR and 8.84% IMR.These results indicate complementary benefits, with visual grounding producing the largest drop and action macros reducing ineffective moves.
  • Predictor Analysis: 60.18% average SR with 5.55% IMR is achieved by the GNN predictor, compared with 48.77% SR and 8.65% IMR for an MLP and 34.91% SR and 8.67% IMR for LLM-as-Judge.The LLM-as-Judge approach is described as expensive and brittle because it parses raw configuration descriptions without a compact learned representation.
  • Predictor Analysis: 60.18% to 49.65% average SR and 5.55% to 9.56% IMR result when removing the GNN surrogate under the same simulator-rollout budget.The predictor prioritizes candidates for expensive simulator evaluations rather than merely accelerating evaluation.
  • Selection Strategy: 60.18% SR and 5.55% IMR are reached by the disagreement-aware selector, outperforming pure exploration, pure exploitation, Thompson Sampling, Deep Ensembles, and LinUCB.Calibration associates uncertainty with error using Pearson r = 0.935, p < 10−5, while predicted and true scores align with Pearson r = 0.991 and ECE = 0.027.

Conclusion

The paper introduces PokeGym as a benchmark for test-time learning in long-horizon 3D games and proposes G-EvoMAC, a graph-guided framework for synergistic multimodal optimization at test time.

  • PokeGym benchmarks test-time learning in long-horizon 3D games.
  • G-EvoMAC jointly optimizes visual perception, strategy, and action-macro sets at test time.
  • G-EvoMAC uses a graph-guided framework to coordinate these multimodal components synergistically.

Appendix Extended Environment Comparison

PokeGym addresses a gap in existing game environments by evaluating whether agents can improve their configurations across repeated episodes of the same task. The appendix compares PokeGym with representative environments based on their design.

  • Environment design: PokeGym enables repeated task attempts to evaluate how agents refine their configurations using experience from prior episodes.Existing environments typically evaluate a fixed agent configuration instead.

Comparison with Pokémon Red

Pokémon Red uses a compact, symbolic tile-based 2D representation that simplifies perception, whereas PokeGym presents continuous, perspective-rendered 3D scenes from an egocentric camera. This shift introduces challenges involving depth, scale, occlusion, lighting, and dynamic entities.

  • Pokémon Red: Pokémon Red represents the environment as a grid of discrete, semantically uniform tiles, producing a compact symbolic map.Each tile encodes a small world patch.
  • Pokémon Red: Perception in Pokémon Red is simplified because walkable regions, blocked regions, objects, and characters are explicitly encoded.The global camera also makes the agent’s surroundings directly readable.
  • PokeGym: PokeGym uses a fully 3D open-world urban environment with continuous, perspective-rendered scenes observed through an egocentric camera.The camera follows the avatar rather than presenting a global tile-based view.
  • PokeGym: PokeGym requires estimating depth and scale from a single image while handling occlusions, variable lighting, and dynamic entities.Buildings and crowds create occlusions, and lighting changes across districts and times of day.

Comparison with Minecraft

PokeGym differs from Minecraft in both visual structure and task semantics: it presents smooth, realistic 3D city scenes and evaluates progress through multi-stage narrative quests rather than externally defined resource accumulation.

  • Minecraft: Minecraft uses a voxel-based 3D grid with single-type blocks and simple textures, supporting open-ended tasks such as gathering resources, crafting, and construction.Because Minecraft lacks predefined narrative goals, subgoals are externally defined and progress is measured by accumulating resources.
  • PokeGym: PokeGym uses smooth, realistic 3D city scenes and storyline-driven quests requiring dialogue interpretation, story-context understanding, and grounded interactions such as talking to NPCs and traveling.Its tasks are multi-stage narrative quests that advance the plot.

Experimental Details · ALFRED Benchmark

The study evaluates embodied-AI performance on ALFRED, a household benchmark built on the interactive AI2-THOR simulator. It uses 255 validation-unseen instances and focuses on six challenging task types grouped into three categories.

  • ALFRED Benchmark: ALFRED is a household embodied-AI benchmark introduced by Shridhar et al. in CVPR 2020.
  • ALFRED Benchmark: The benchmark is built on the interactive AI2-THOR simulator.
  • ALFRED Benchmark: ALFRED contains seven task types.
  • ALFRED Benchmark: Evaluation uses 255 validation-unseen task instances.
  • ALFRED Benchmark: The evaluation selects the six most challenging task types.
  • ALFRED Benchmark: The selected task types are grouped into three categories.
  • ALFRED Benchmark: State-Changing tasks require altering an object’s state before placing it in a receptacle.Examples include cleaning, heating, and cooling.

Implementation Details … Prompt 1: Planning

The paper specifies G-EvoMAC’s implementation, comparisons, analyses, complexity, limitations, released artifacts, and prompts for visual, long-horizon Pokémon Legends: Z-A tasks. Across these components, the method combines evolutionary configuration search, graph-guided selection, multimodal co-evolution, and repeated-episode adaptation.

  • Candidate Generation and Evaluation Budget: G-EvoMAC generates M = 3 candidates per round, evaluates only the GNN-selected candidate, caps episodes at 360 steps, and runs evolutionary methods for 10 rounds.The no-predictor variant evaluates all M = 3 candidates, consuming three emulator rollouts per round.
  • Warm-Up Training Set for the GNN Predictor: The GNN predictor begins with directly evaluated warm-up configurations, then receives online supervision from newly evaluated configurations during evolution.Each warm-up configuration contributes graph representations and ground-truth scores.
  • Comparison with Parameter Adaptation: The default selector achieves 60.18% SR and 5.55% IMR, outperforming test-time value-function adaptation and showing complementary benefits from visual, strategy, and action modalities.V-GPS reaches 49.30% SR and 10.94% IMR, while removing Visual reduces SR from 60.18% to 39.82%.
  • Sensitivity of the Disagreement Weight λ: The disagreement selector uses U(c_i) = µ(c_i) + λσ(c_i), with λ = 1.0 as the default and T = 10 stochastic GNN passes for uncertainty estimation.λ = 0 yields 51.93% SR, λ = 2 yields 52.46% SR with higher IMR, and T = 5 lowers SR to 55.61% while raising IMR to 8.02%.
  • Offline Predictor Validation: Offline validation finds Spearman ρ = 0.453, 51.75% Top-1 selection accuracy, and 66.67% pairwise ranking accuracy for the GNN predictor.The corresponding random baselines are 33.3% for Top-1 selection and 50% for pairwise ranking.
  • Evolution Score Trends: G-EvoMAC’s scores rise across 10 evolution rounds and stabilize later, while recurring failures involve spatial traversal, collision-aware control, and task-relevant planning.The simplified spatial state space ranges from 67,320 to 870,840 states, and the evolved action space reaches 1,728 combinations per decision step.
  • Robustness and Practicality of the Evaluator: The evaluator achieves a 100% first-try AOB-scanning hit rate, while G-EvoMAC still consumes around 163k input tokens and costs approximately $0.560 per GPT-5.4 run.Released components include the emulator wrapper, signature-discovery scripts, evaluator, task definitions, implementation, and setup documentation.
  • Prompt 1: Planning: The prompts define planning, trajectory summarization, and cross-trajectory evolution for agents using visual screenshots and discrete actions across long-horizon tasks.Cross-trajectory evolution receives summaries of 5 recent rollouts, while trajectory summarization groups continuous steps into logical Behavioral Phases.
Loading 2604.08340v2…