Gym-V: A Unified Vision Environment System for Agentic Vision Research

1 Introduction

• Zero-shot evaluation reveals substantial model capability gaps: a newer 32B model outperforms a conventional 72B model by 1.8×, while the strongest model still has headroom.This motivates controlled investigation of the factors driving vision-agent performance.Source: §1 · p. 2
• Observation scaffolding is more decisive for learning success than the choice of RL algorithm, with captions improving learning and removing game rules potentially preventing it.Algorithm differences are scenario-dependent and become most visible in long-horizon multi-turn stability.Source: §1 · p. 3
• Gym-V provides a unified Gym-compatible platform connecting interactive training, offline supervision, and benchmark evaluation through one interface.Composable wrappers control received context without changing the underlying environment, while built-in verifiers and distributed rewards support automated scoring.Source: §1 · pp. 1–3
• The suite contains 179 procedurally generated environments across 10 categories with controllable difficulty for curriculum and cross-domain generalization studies.Figure 1 reports 105 single-turn and 74 multi-turn environments.Source: Figure 1 · p. 2, §1 · p. 3
• Training on diverse categories transfers broadly, whereas narrow training can cause negative transfer, and multi-turn interaction amplifies these effects.The same amplification applies to scaffolding and curriculum choices over longer horizons.Source: §1 · p. 3

VLM Benchmarks and Interactive Environments

• Static vision-language benchmarks lack training signals and use one-shot evaluation, while interactive benchmarks often define separate interfaces and cover narrow task families.These limitations hinder unified reinforcement-learning training pipelines.Source: §2 · p. 3

Gym-style Frameworks for Agent Training

• Gym-V extends gym-style training to vision with 179 environments, evaluation-as-a-service, and six interaction workflows under one API.The workflows include single-turn, multi-turn, multi-agent, tool-augmented, VLM evaluation, and image-generation evaluation.Source: §2 · p. 3
• Text gym frameworks offer procedural generation, automatic verification, and standardized workflows but do not address visual observations.Existing visual platforms cover fewer tasks or lack unified RL and tool-augmented integration.Source: §2 · p. 3

3.1 Environment Interface

• Gym-V uses a Gym-compatible reset()/step() interface for single-agent, competitive, cooperative, online, and offline settings, treating batched episodes uniformly.Multi-agent data are indexed by agent identifiers, while offline one-step tasks and online multi-turn games share the same training abstraction.Source: §3.1 · p. 4

3.2 Wrappers as Experimental Variables

• Composable wrappers alter rules, captions, context, outputs, and tools without changing environment dynamics, making observation design reproducible and experimentally controllable.The wrapper layer supports controlled ablations over rule phrasing, caption availability, and history length.Source: §3.2 · p. 4

3.3 Evaluation-as-a-Service and Unified Evaluation

• A Ray Serve reward service provides unified, scalable scoring for generative tasks, while benchmark environments expose the same protocol and reproduce official evaluation results closely.Gym-V delegates generative scoring to heterogeneous reward backends and matches official VLMEvalKit, GenExam, RISE, and GenEval results.Source: §3.3 · pp. 4–5, Figure 2 · p. 5

3.4 Environment Suite and Taxonomy

• The Gym-V suite contains 179 environments across 10 categories, sharing reset/step interfaces and deterministic seeding, with visual observations, automatic verification, and scalable difficulty.It covers single-turn visual reasoning and multi-turn sequential decision making; difficulty levels 0–2 provide standardized complexity sweeps.Source: §3.4 · p. 5, Table 2 · p. 6

3.5 Performance Across Categories

• Qwen3-VL-32B reaches 44.5 single-turn average, ≈1.8× Qwen2.5-VL-72B at 24.3, while Gemini-3-Pro achieves 73.1 overall, leaving substantial model headroom.ARC remains difficult even for Gemini-3-Pro at 40.1, and scaling Qwen2.5-VL from 7B to 72B raises its average from 9.0 to 24.3 (2.7×).Source: §3.5 · p. 7
• Difficulty cliffs expose rapid capability degradation as complexity rises: robustness ρ falls to 0.13 for Gemini-3-Pro on MaxGridPathIntersection and to 0.00 for Claude-Sonnet-4.5 on NQueens.These cliffs identify challenging regimes where continued difficulty scaling can preserve learning signal for reinforcement learning and curriculum studies.Source: Figure 3 · p. 6, §3.5 · p. 7

4.1 Algorithm Comparison

• Across 16 environments, GRPO, GSPO, and SAPO all improve single-turn performance, but no algorithm dominates uniformly; differences are clearest in long-horizon multi-turn stability.GSPO is most consistent across several multi-turn games, whereas SAPO can collapse in harder settings.Source: §4.1 · pp. 7–8, Figure 4 · p. 8

4.2.1 History Length in Multi-Turn Interaction.

• Interaction history benefits multi-turn learning most in complex irreversible environments, where observing recent action outcomes helps agents avoid dead states.In Sokoban, context-equipped agents learn effectively while the current-observation-only agent barely improves.Source: §4.2.1 · p. 8

4.2.2 Explicit Guidance vs. Autonomous Exploration.

• Including game rules generally improves learning, but removing them largely prevents progress in Sokoban and can outperform rule prompting in Minesweeper.The Minesweeper exception may reflect dense visual feedback and distraction or overly cautious behavior from rule text.Source: §4.2.2 · p. 9

4.2.3 Text Scaffolding for Visual Observations.

• Captions consistently improve learning across four environments, with the largest effect on longest_path and substantial gains in multi-turn games.Captioned agents rapidly approach near-optimal return on longest_path, whereas image-only agents make little progress.Source: §4.2.3 · p. 9, Figure 6 · p. 10

4.3 Domain Generaltization

• Broad curricula transfer better than narrow training: Cognition has the greatest transfer breadth at 33.1, while Geometry has the lowest at 17.1 and exhibits negative transfer.Narrow geometry training may encourage domain-specific read-off shortcuts rather than reusable procedures.Source: §4.3 · p. 10
• Transfer is asymmetric and fragile across multi-turn games, with prerequisite-like skill relations, negative transfer from conflicting default strategies, and limited cross-game gains.Logic→Cognition yields +11.0 versus +5.8 in the reverse direction, while FrozenLake→Game2048 yields −2.7.Source: §4.3 · pp. 10–11

5 Conclusion

• Gym-V combines consistent APIs with 179 procedurally seeded environments across 11 categories, supporting single- and multi-turn training, evaluation, and controllable distribution shifts.The platform includes difficulty presets and supports multi-agent interaction, tool use, and generative evaluation.Source: §5 · p. 11
• Scaffolding choices such as rules and image captions are critical, especially in multi-turn settings, while broad curricula transfer better and narrow training can cause negative transfer.Source: §5 · p. 11

Overview of the Appendix

• Section A gives training protocol details, while Sections B and C analyze implicit rule discovery and temporal environments.Source: Overview of the Appendix · p. 15
• Sections D–F provide per-environment examples, evaluation results, and the complete environment catalog.Source: Overview of the Appendix · p. 15

A Training Protocol

• All training experiments use Qwen2.5-VL-7B-Instruct with a frozen vision tower and a learning rate of 1 × 10^-6.Source: A Training Protocol · p. 15
• Single-turn iterations sample 256 instances with 8 rollouts each and apply a KL penalty of 0.01 to limit deviation from the reference policy.Source: A Training Protocol · p. 15
• Multi-turn iterations sample 256 complete trajectories of up to 200 turns, disabling KL regularization because it can overly constrain long-horizon exploration.Source: A Training Protocol · p. 15

Context as Implicit Rule Discovery

• A 5-turn context window generally modestly improves on 3 turns, with the largest gain in Sokoban for diagnosing dead-end states.The experiments compare 3-turn and 5-turn windows across four multi-turn games.Source: B Context as Implicit Rule Discovery · p. 15
• Context without rules consistently outperforms the no-context baseline and approaches the rule-equipped setting in several games.In Game2048 and FrozenLake, context-only nearly matches the rule-equipped agent; in Minesweeper, it exceeds it.Source: B Context as Implicit Rule Discovery · p. 15
• Context-only performance nearly matches rules in Game2048 and FrozenLake, while dense local feedback may substitute for explicit descriptions in Minesweeper.Minesweeper context-only even exceeds the with-rules variant.Source: B Context as Implicit Rule Discovery · p. 15
• Sokoban remains difficult because discovering irreversible push constraints from interaction leaves a persistent gap to explicit rules.Context helps, but the with-rules setting remains better.Source: B Context as Implicit Rule Discovery · p. 15

Context and Temporal Control

• Interaction context without rules substantially outperforms no context and can approach the with-rules condition across multi-turn games.The comparison uses 3-turn and 5-turn context windows across four games.Source: Figure 7 · p. 16
• Retro arcade agents make little meaningful progress after 200 steps, producing near-zero reward across temporal environments.These games require sustained precise inputs over hundreds to thousands of consecutive frames.Source: C Temporal Environment Analysis · pp. 15–16, Figure 8 · p. 16
• Retro arcade games therefore serve as a frontier benchmark for systems capable of fine-grained temporal control.The stated boundary is insufficient action sequences within the allotted step budget.Source: C Temporal Environment Analysis · p. 16

Per-Environment Examples

• The paper presents representative observations and task descriptions, while the repository provides all 202 environments, configuration parameters, difficulty levels, and implementation details.The examples cover 10 Gym-V categories.Source: D Per-Environment Examples · pp. 16–17
• Algorithmic environments include connected-component operations on binary grids and minimum-time computation for rotting oranges via BFS propagation.BinaryMatrix outputs an integer or transformed grid; RottenOranges outputs a single integer.Source: Figure 9 · p. 17
• Cognition environments include answering Rubik’s Cube face-color questions after rotations and identifying an odd polygon by visual properties.RubiksCube-QA uses a rendered 3D cube, while OddOneOutPoly compares polygon properties such as symmetry or number of sides.Source: Figure 10 · p. 17

Per-Environment Evaluation Results

• Figure 18 breaks down zero-shot mean@3 scores for every environment across all 9 evaluated models.The breakdown is per environment rather than a single aggregate score.Source: E Per-Environment Evaluation Results · p. 17
• Single-turn environments are scored by answer correctness, while multi-turn environments use normalized episodic return with negative values clipped to zero.The evaluation metric depends on the interaction format.Source: E Per-Environment Evaluation Results · p. 17
• All reported scores are shown as percentages after multiplying by 100.The passage specifies percentage scaling as ×100.Source: E Per-Environment Evaluation Results · p. 17

• The catalog distinguishes single-turn environments, which require complete answers from one observation, from multi-turn environments requiring sequential decisions over multiple interaction steps.Source: F Complete Environment Catalog · p. 17, Table 5 · p. 22, Table 6 · p. 26

Geometry

• Geometry environments require computational reasoning over points, shapes, areas, connected components, triangles, tangram relations, and upper-envelope visibility.Source: Figure 11 · p. 18, F Complete Environment Catalog · p. 23

Graphs

• Graph environments span shortest and longest paths, connectivity, coloring, cliques, independent sets, matchings, spanning structures, Eulerian circuits, tree optimization, and graph counting.Source: Figure 12 · p. 18, F Complete Environment Catalog · pp. 23–24

Logic

• Logic environments include grid-constraint puzzles, Boolean circuit evaluation, inequality and parity construction, weighted-sum clues, and magic-square completion.Source: Figure 13 · p. 19, F Complete Environment Catalog · p. 24

Puzzles

• Puzzle environments range from Sudoku, N-Queens, path and sequence transformations, solitaire and board games, to Sokoban, word search, Wordle, and Tower of Hanoi.Source: Figure 14 · p. 19, F Complete Environment Catalog · pp. 25–27

Spatial

• Spatial environments include 2D navigation with keys, doors, rooms, obstacles, and hazards, plus 3D MiniWorld tasks involving rooms, mazes, objects, signs, and branching corridors.Source: Figure 16 · p. 20, F Complete Environment Catalog · pp. 27–28

Temporal

• Temporal environments comprise continuous-action arcade tasks such as shooters, platformers, fighting games, beat-em-ups, scrolling games, and hack-and-slash combat.Source: Figure 17 · p. 20, F Complete Environment Catalog · p. 28

Cognition

• Cognition environments cover visual reasoning tasks including rule induction, color patterns, maze traversal, polygon comparison, rectangle counting, Rubik’s Cube state tracking, and tree traversal.Source: F Complete Environment Catalog · p. 23