PaperlayerView exact revision on arXiv

Source-linked AI summary

Gym-Anything-RT: Scaling Real-Time Environments for Vision-Language Agents

Pranjal Aggarwal, Sean Welleck

arXiv:2610.37567v1cs.LGcs.AI

TL;DR

Agents that deliberate between actions face a different challenge when environments keep changing during reasoning. GYM-ANYTHING-RT builds controllable visual minigames and compares the same tasks live, slowed, or paused. Agents perform poorly live: GPT-6 Astra (xHigh) passes 9.0% of real-time tasks, rising to 75.0% when paused.

  • Problem

    Agents may take more than ten seconds to choose an action, while real-time environments keep changing during reasoning; existing evaluations provide limited evidence about performance under that condition.

    Source: §1 · p. 1

  • Method

    GYM-ANYTHING-RT uses coding agents to create visual minigames and compares the same tasks live, slowed, or paused to control the time available for action.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    Current agents perform poorly in real time: GPT-6 Astra (xHigh) has a 9.0% pass rate live versus 75.0% when the environment is paused.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Frontier agents can partially offset slow responses by writing code that perceives and acts between their own responses.

    Source: §1 · p. 3

    Whether gains on these minigames transfer to complete games and physical interfaces remains an open question.

    Source: A LIMITATIONS · p. 18

Abstract

from arXiv · show

AI agents have made substantial progress in domains where they have a non-trivial amount of time to deliberate between actions, such as mathematics, software engineering, and performing tasks on desktop applications. It is less clear how such models fare in settings that demand real-time interaction, such as playing a video game or operating a physical device. In these settings, the environment state may evolve as the agent is reasoning, or even as the underlying model is performing a forward pass. To gain insight into the real-time interaction capabilities of vision-language agents, we introduce Gym-Anything-RT. Our key idea is to construct visual minigames that demand a portfolio of capabilities required for interactive real-time settings. Gym-Anything-RT uses an agentic pipeline to automatically generate such environments based on a taxonomy of six capabilities required for real-time interaction, including the ability to perceive the environment and act under an evolving environment state. A key feature of Gym-Anything-RT is that it allows for controlled comparison of real-time interaction capabilities: we can run the same minigame in real time, or pause or slow down the environment to relax the requirement for real-time actions. We use Gym-Anything-RT to construct 100 real-time minigames and evaluate AI agents on them. We find large capability gaps in real-time interaction. For example, GPT-6 Astra (xHigh) achieves a pass rate of only 9.0% on the real-time minigames, which rises to 75.0% when the environment is paused. We also identify intriguing behaviors by the most powerful frontier agents: for instance, when given the ability to write code, these agents write code that bypasses the need to perceive and act in real time. We publicly release Gym-Anything-RT and the generated minigames to spur further research on AI agents in real-time interactive settings.

1 INTRODUCTION

Agents that deliberate between actions face a different challenge when the environment keeps changing during reasoning. GYM-ANYTHING-RT provides controlled minigames for testing this gap and finds that frontier models perform poorly live, with speed a primary bottleneck.

1 INTRODUCTION

  • Existing computer-use evaluations wait between actions, while game benchmarks combine many capabilities, making it hard to identify why models fail.Separating slowness from other capability gaps requires controlled comparisons that vary response time on the same task.Source: §1 · pp. 1–2
  • GYM-ANYTHING-RT generates visual minigames and runs the same task live, slowed, or paused to distinguish real-time failures from other task difficulty.Its agentic pipeline selects games covering six capabilities and creates oracle solutions, audits, and controlled difficulty and interaction variants.Source: §1 · p. 2
  • GPT-6 Astra (xHigh) passes 9.0% of real-time tasks versus 66.0% of non-real-time tasks; 64× slowdown raises its pass rate to 57.0%, and pausing to 75.0%.Source: §1 · pp. 2–3
  • Temporal reasoning and exploration of unfamiliar interfaces remain additional sources of difficulty beyond response speed.Source: §1 · p. 3
  • When allowed to run code, Astra writes scripts that perceive and act between responses, raising its real-time pass rate from 9.0% to 31.0%.In Rotating On-Screen Keyboard, it uses image template matching to locate moving keys and click them within 200 ms.Source: §1 · p. 3

2 METHODOLOGY

GYM-ANYTHING-RT constructs visual minigames whose timing, difficulty, and interaction can be controlled while the underlying task is held constant. Its construction pipeline selects tasks, creates and audits playable programs, and labels their capability demands.

2 METHODOLOGY

  • Each minigame is an executable environment with internal state, controls, and a clock; real-time interaction requires acting on recent visual evidence despite changes before execution.Source: §2 · p. 3

2.1 TASK DISCOVERY AND SELECTION

  • Candidate minigames are selected for visual grounding, interface actions, and objectively verifiable outcomes, while tasks dominated by OCR, static classification, arithmetic, or question answering are excluded.Sources may be non-real-time initially; the creator turns their mechanics into real-time tasks.Source: §2.1 · pp. 3–4

2.2 CREATOR-AUDITOR CONSTRUCTION

  • A creator implements each minigame as a program and supplies an oracle solution; independent auditors test solvability, rendering, controls, grading, recovery, and whether real-time action is genuinely required.The creator revises the program until an audit passes.Source: §2.2 · p. 4, Figure 2 · p. 4

2.3 NON-REAL-TIME COUNTERPARTS

  • Each non-real-time counterpart preserves the task, controls, difficulty, and available information while minimally changing timing, allowing comparison against its real-time version.Independent audits check that the versions differ only in timing across all ten configurations.Source: §2.3 · p. 4

2.4 CONTROLLED TASK VARIATIONS

  • Difficulty levels vary task demands, while simplified interaction exposes proxy controls and full interaction requires direct mouse or keyboard manipulation.Audits check adjacent difficulty levels and verify that interaction modes preserve the same minigame for a fixed seed and difficulty.Source: §2.4 · p. 5

2.5 CONTROLLING TIME

  • The evaluation stack runs minigames live, slows all game clocks while preserving real-time action delivery, or freezes the environment after an observation until the agent acts.All settings use the same minigame, seed, difficulty, interaction mode, and verifier.Source: §2 · p. 3, §2.5 · p. 5

2.6 CAPABILITY REVIEW

  • Review agents label configurations for visual understanding, temporal reasoning, reasoning and planning, exploration, and interface understanding using code and behavioral evidence.A separate review checks the real-time requirement for each configuration.Source: §2.6 · p. 5

3 GYM-ANYTHING-RT

GYM-ANYTHING-RT comprises real-time minigames and matched counterparts, evaluated under live, slowed, and paused settings. The benchmark spans varied capability requirements and measures success, solve time, and inference cost.

3 GYM-ANYTHING-RT

  • The benchmark contains 100 real-time minigames and 100 non-real-time counterparts, with four easier difficulty levels and a simplified interaction mode, totaling 2,000 configurations.Models are evaluated in live, slowed, and paused settings.Source: §3 · pp. 5–6

3.1 CAPABILITY COVERAGE

  • All real-time minigames require temporal reasoning, 96% require reasoning and planning, 52% require exploration and interface understanding, and 31% require 3D visual understanding at some difficulty level.Counterparts require the same capabilities at similar rates, except temporal reasoning, which drops to 82%.Source: Table 1 · p. 6, §3.1 · p. 6

3.2 EVALUATION PROTOCOL

  • Models are evaluated by pass rate, time to solve, and inference cost across real-time and counterpart games, slowdowns, difficulty levels, interaction modes, and capability requirements.Agents interact through screenshots, mouse actions, and keyboard input without access to program internals or oracle solutions; task-specific verifiers determine success.Source: §3.2 · p. 6

4 RESULTS AND ANALYSIS

The evaluation tests models through screenshots and mouse/keyboard actions at the highest difficulty, then varies timing, coding access, capabilities, interface rules, difficulty, and interaction mode. Across these controlled comparisons, real-time response demands sharply reduce performance, while speed, temporal reasoning, interface exploration, and task complexity each contribute to difficulty.

4.2 CURRENT AGENTS PERFORM POORLY IN REAL TIME

  • 9.0%: GPT-6 Astra (xHigh) solves real-time tasks at the highest difficulty, versus 66.0% on the same minigames’ non-real-time counterparts.Figure 4 compares the models’ pass rates on 100 real-time tasks and their 100 non-real-time counterparts.Source: §4.2 · p. 7, Figure 4 · p. 7

4.3 SPEED IS THE PRIMARY BOTTLENECK FOR FRONTIER MODELS

  • 9% to 57%: Astra (xHigh) improves as the environment slows from normal speed to 64×, reaching 75% when paused.Even at 64× slowdown, performance remains 18 points below paused performance; slowing changes the environment clock while leaving model inference unchanged.Source: §4.3 · pp. 7–8, Figure 5 · p. 7

4.4 STRONG CODING CAPABILITIES PARTIALLY OFFSET WEAK REAL-TIME CAPABILITIES

  • 31.0%: coding raises Astra (xHigh)’s real-time pass rate from 9.0%, while inspected scripts perceive and act between model responses.The gain on non-real-time counterparts is 8.0 points, from 66.0% to 74.0%; scripts can still fall short when relevant visual information cannot be extracted reliably or prepared in advance.Source: §4.4 · p. 8, Table 2 · p. 8

4.5 MODELS ARE SLOW AND EXPENSIVE

  • 9.0%: Astra (xHigh) has the highest model pass rate but takes 7.2 minutes and costs $4.50 per attempt; a human author solves 45.0% in 2.2 minutes per task.The human case study covers 20 randomly sampled tasks; the authors conclude real-time progress requires improving pass rate, speed, and cost.Source: §4.5 · p. 8, Table 3 · p. 8

4.6 PAUSING ALSO HELPS WHEN TEMPORAL REASONING IS REQUIRED

  • 20.4% to 31.4%: pausing raises Luna’s pass rate on non-real-time counterparts requiring temporal reasoning, while the gain without that requirement is not statistically significant.Even paused, Luna solves only 29.8% of real-time tasks, indicating that response speed is not its only bottleneck.Source: §4.6 · p. 9, Figure 6 · p. 9

4.7 PERFORMANCE DECREASES AS MORE CAPABILITIES ARE REQUIRED

  • 37.0% to 7.2%: Luna’s live pass rate falls as the number of required capabilities increases from one to four; under pausing it falls from 55.6% to 23.2%.The trend persists without time pressure, but overlapping capability requirements prevent isolating the effect of any single capability.Source: §4.7 · p. 9, Figure 7 · p. 9

4.8 PROVIDING INTERFACE RULES IMPROVES PERFORMANCE

  • 11.3% to 16.2%: providing Luna an environment-specific rulebook raises its pass rate on tasks requiring exploration and interface understanding.The minigame and controls stay the same, so the comparison tests the benefit of knowing the rules in advance.Source: Figure 8 · p. 9, §4.8 · p. 10

4.9 PERFORMANCE DECREASES WITH DIFFICULTY AND IMPROVES WITH SIMPLIFIED INTERACTION

  • 53.0% to 15.5%: Luna’s paused pass rate falls from difficulty 1 to 5, averaged across interaction modes; simplified controls raise overall performance from 29.8% to 33.2%.Simplified controls yield larger gains at higher difficulties, but performance still declines with difficulty under simplified interaction.Source: Figure 9 · p. 9, §4.9 · p. 10, Table 4 · p. 10

5 RELATED WORK

Prior interactive-agent evaluations often pause environments between actions or combine many capabilities in each task, limiting what low performance reveals. Gym-Anything-RT distinguishes itself by varying timing, difficulty, and interaction interface while preserving the minigame task.

Real-time interaction

  • Existing real-time comparisons cover only a few visual games or text-based environments; Gym-Anything-RT pairs 100 real-time minigames with non-real-time counterparts and controls each minigame’s clock.This permits varying response time on the same task.Source: §5 · p. 10

Benchmarks for interactive visual agents

  • Many computer-use environments wait between actions, while game tasks often combine capabilities, making it difficult to identify what limits an agent.Prior work also evaluates capabilities separately or varies task conditions such as difficulty, initial state, or interface.Source: §5 · p. 11

Capability-specific evaluation and automated environment construction

  • Capability-specific VLM benchmarks commonly ask questions about fixed inputs, while Gym-Anything-RT requires perception and action as the minigame changes.Automated environment construction works have not produced visual minigames whose timing, difficulty, and interface can vary while preserving the underlying task; Gym-Anything-RT extends Gym-Anything’s creator and audit agents.Source: §5 · p. 11

6 CONCLUSION

GYM-ANYTHING-RT provides a controllable benchmark for real-time interaction, where GPT-6 Astra (xHigh) solves 9.0% of real-time tasks versus 66.0% of non-real-time counterparts. The results identify speed as the primary bottleneck for frontier models, with temporal reasoning and unfamiliar interfaces also posing difficulty.

6 CONCLUSION

  • On a benchmark of 100 real-time minigames, GPT-6 Astra (xHigh), the strongest agent, solved only 9.0% of real-time tasks, versus 66.0% of their non-real-time counterparts.Source: §6 · p. 11
  • The framework varies response time, real-time requirements, difficulty, and interaction interface on the same minigame.Coding agents implement and audit each environment as a visual minigame program with an oracle solution, while the evaluation stack controls its clock.Source: §6 · p. 11
  • Speed is the primary bottleneck for frontier models, while temporal reasoning and exploration of unfamiliar interfaces are additional sources of difficulty.Frontier agents can partially offset poor real-time performance by writing code that perceives and acts on their behalf.Source: §6 · p. 11
  • The authors suggest that progress in real-time interaction requires agents that are both capable and fast.They propose faster models, code for time-critical task components, and training on real-time environments as directions for future work.Source: §6 · p. 11

A LIMITATIONS

The paper evaluates existing agents and leaves training effects and transfer to complete games or physical interfaces open. Its construction procedure uses creator and audit agents to implement and verify minigames.

A LIMITATIONS

  • The study evaluates existing agents; whether training on these minigames improves real-time interaction remains future work.The programmatic task generators and automatic verifiers also provide a basis for training.Source: A LIMITATIONS · p. 18
  • Whether improvements on these minigames transfer to complete games and physical interfaces remains an open question.The minigames capture specific demands of real-time environments for controlled comparisons across capabilities, difficulty, and interaction.Source: A LIMITATIONS · p. 18
  • GYM-ANYTHING-RT covers a selected set of capabilities and task mechanics, with the creation–audit framework providing a way to extend coverage.Additional minigames and capability combinations can be added.Source: A LIMITATIONS · p. 19

B.1 CREATOR AND AUDIT AGENTS

  • The creator implements each minigame and tests its configurations, while fresh audit agents inspect the environment and evidence and return a PASS or REVISION REQUIRED verdict.Construction agents use GPT-6 Astra; audits check behavior and evidence rather than relying on written claims.Source: §B.1 · p. 19, B MINIGAME CONSTRUCTION · p. 19

B.2 DIFFICULTY LEVELS

The appendix describes how difficulty levels, interaction modes, and timing counterparts are constructed and reviewed. It also details real-time revisions, the benchmark’s capability coverage, and criteria for annotating task requirements.

B.2 DIFFICULTY LEVELS

  • Difficulty levels change the agent’s decision or control problem, including information, ambiguity, planning depth, motion speed, timing windows, precision, or recovery options.More steps, rounds, or waiting alone do not make a configuration harder.Source: §B.2 · pp. 19–20, Table 5 · p. 20

B.3 INTERACTION MODES

  • For a fixed seed and difficulty, simplified and full interaction use the same world, information, goal, and action effects, but the grader rejects controls from the other mode.The two modes differ in the interaction required to operate the minigame.Source: §B.3 · p. 20, Table 6 · p. 20

B.4 REAL-TIME REVISIONS

  • 64 minigames were rebuilt from the same source as real-time tasks, 23 existing real-time minigames were revised, and 13 retained their original implementation.For 35 rebuilt minigames, exploration was restored after conversion had removed its need.Source: §B.4 · p. 20

B.5 NON-REAL-TIME COUNTERPARTS

  • Each non-real-time counterpart keeps its real-time minigame’s generator, goal, controls, difficulty, information, and grader, changing only how the environment evolves over time.An independent audit compares the two versions across all ten configurations.Source: §B.5 · p. 20

C MINIGAME LIST

  • The benchmark lists 100 real-time minigames and the capabilities required by each minigame and its non-real-time counterpart.Source: C MINIGAME LIST · p. 21

D CAPABILITY ANNOTATION CRITERIA

  • GYM-ANYTHING-RT varies interaction and real-time interaction settings and annotates four other capabilities: visual understanding, temporal reasoning, reasoning and planning, and exploration and interface understanding.Review agents support labels with code or behavioral evidence and apply criteria to each configuration’s requirements.Source: D CAPABILITY ANNOTATION CRITERIA · p. 21

E REAL-TIME EVALUATION PROTOCOL

The evaluation protocol controls when the task clock advances relative to observations and actions. Paused execution provides a chronological frame sequence and stops the clock until the agent acts.

E REAL-TIME EVALUATION PROTOCOL

E.1 ACTION AND SCREENSHOT LATENCY

The appendix describes a direct action-and-screenshot path that avoids routing each exchange through the central server, then measures request-to-response latency. In isolated tests, median latency ranged from 39.2 to 46.8 ms across the reported conditions.

E.1 ACTION AND SCREENSHOT LATENCY

  • The client connects directly to an input service inside its sandbox, while the central server allocates sandboxes and manages their lifecycle.Actions and screenshots bypass the server; screenshots are captured as full-resolution JPEGs and returned without resizing or re-encoding.Source: §E.1 · pp. 21, 23
  • Latency is measured from immediately before an HTTP request until its complete response, including the image, arrives; model inference and subsequent image loading and decoding are excluded.Source: §E.1 · p. 23
  • Median latencies were 39.2 ms for Rotating On-Screen Keyboard, 42.5 ms for an idle LIDAR Blacksite scene, and 46.8 ms while issuing LIDAR scan clicks.The corresponding 95th percentiles were 52.1, 55.3, and 70.7 ms, respectively.Source: §E.1 · p. 23

F EVALUATION DETAILS

The appendix specifies the evaluation models and harnesses, their computer-action interface and prompts, and the attempt budgets and success criteria.

F EVALUATION DETAILS

F.1 MODELS AND HARNESSES

  • GPT-6 Astra and Luna, Claude Opus 5.5, and two Qwen models are evaluated through their specified CLI or API harnesses, with built-in tools disabled and model-specific settings.The Qwen models use temperature 1.0, top-p 0.95, top-k 20, and at most 2,048 output tokens per response.Source: §F.1 · p. 23

F.2 ACTION SPACE AND PROMPT

  • Computer-tool calls contain ordered mouse and keyboard actions and return a screenshot after completion; prompts direct models to use visible controls without writing code or inspecting the page.Figure 10 shows the live prompt, while the paused prompt replaces the paragraph following “Temporal mode” with Figure 11’s text.Source: §F.2 · p. 23

F.3 BUDGETS AND OUTCOMES

  • Each attempt allows at most 100 steps and has a six-hour limit, except specified Astra (xHigh) real-time conditions, which have a one-hour limit; timeouts count as failures.Provider or infrastructure errors are excluded and rerun, and the task verifier determines success.Source: §F.3 · p. 23

F.4 TIME AND COST

This appendix defines attempt time and API-equivalent cost, and describes the coding experiment and interface rulebooks.

F.4 TIME AND COST

  • Time is the wall-clock duration from the minigame clock starting to the episode ending, excluding environment setup.Source: §F.4 · p. 23
  • Cost is computed from recorded token usage using model-specific prices; reasoning tokens count once as output tokens.The appendix uses OpenAI API prices for GPT-6 Astra and Luna, OpenRouter prices for Qwen, and Claude Code’s reported session cost for Claude Opus 5.5.Source: §F.4 · pp. 23, 25
  • Figures 10 and 11 show the prompts for Rotating On-Screen Keyboard in the live and paused settings, respectively.Source: Figure 10 · p. 24, Figure 11 · p. 25

F.5 CODING PROTOCOL

  • In the coding experiment, models can process screenshots and issue actions through an HTTP endpoint using a Python client, but cannot access the task machine, page source, or browser developer tools.Source: §F.5 · p. 25

F.6 INTERFACE RULEBOOKS

  • Interface rulebooks describe controls, action effects, fixed world rules, feedback, submission, and recovery, while excluding generated answers, target coordinates, hidden state, and seed-specific layouts or solutions.The rulebook is appended to the task prompt, with all other settings unchanged.Source: §F.6 · p. 25

F.7 HUMAN BASELINE

A human author completed 9 of 20 randomly sampled difficulty-5 tasks in the live setting, solving them in an average of 132.7 seconds per task.

F.7 HUMAN BASELINE

  • 9 of 20 tasks (45.0%) were solved by one author playing randomly sampled difficulty-5 live tasks for the first time; average time was 132.7 seconds per task.Each task used the same seed received by the models.Source: §F.7 · p. 25

G EXAMPLES OF MODEL-WRITTEN CONTROLLERS

This section presents model-written controllers that convert screenshots into task-specific actions and explains how configurations are classified and reviewed for real-time interaction. It also situates Gym-Anything-RT among benchmarks for interactive visual agents, capability evaluation, and real-time action.

G EXAMPLES OF MODEL-WRITTEN CONTROLLERS

  • At difficulty 5 in live interaction, Astra’s scripts passed three tasks by extracting visual state and converting it into controls, while a similar Gyroscopic Tilt Board attempt failed.The passing scripts located keyboard keys with perspective transforms and image templates, estimated waveform phase error to adjust a slider, and predicted gear-marker angles to time clutch releases; the failed attempt detected a ball and direction indicator, estimated velocity, and followed waypoints.Source: G EXAMPLES OF MODEL-WRITTEN CONTROLLERS · pp. 25–26

H.1 DEFINITION

  • The definition uses visible task information, groups every occurrence of a control mode across seeds and attempts, and excludes histories already unsalvageable after the delay.A fixed wait, later completion alone, or a fixed animation whose result remains available does not establish a degrading opportunity; the visible window must predict the same process used to determine the delayed-optimal action.Source: §H.1 · p. 26
  • A configuration is real-time only when a common visible autonomous process makes recent visual evidence necessary for choosing a delayed-optimal action and causes that action to lose value before it is taken.The definition requires a viable grouped control mode, positive delay, and bounded visual window: an action-history-only policy must fail, a policy using recent visual history must succeed, and the same visible process must degrade an available action through non-clock task-state evolution.Source: §H.1 · p. 26

H.2 REVIEW PROMPTS

  • The review prompts operationalize the shared definition and pre-run exception, with one assigning a configuration per reviewer and the other grouping a game’s configurations for one reviewer.For a configuration to be labeled not real-time, at least 50% of generated instances must admit a successful pre-run solution that sends no further outcome-affecting actions during the autonomous outcome phase.Source: §H.2 · p. 27, §H.1 · p. 27

H.2.1 ONE CONFIGURATION PER REVIEW

  • The single-configuration prompt requires source-based review of the exact setup and a common witness for all three mathematical clauses, returning unresolved when evidence cannot determine the pre-run threshold.Its checks rule out clock-only policies, require a sufficient bounded visual window, and require visible task-state change to degrade an action within the task-scale delay.Source: §H.2.1 · pp. 27–28

H.2.2 CONFIGURATIONS GROUPED BY MINIGAME

  • The grouped-by-game prompt preserves the same definition and threshold while asking one reviewer to judge requested configurations separately, checking mode-specific inputs and difficulty-dependent behavior.The reviewer reads shared implementation files once, applies the definition to each requested configuration, and does not copy judgments across configurations without checking their differences.Source: §H.2.2 · pp. 29–30
  • A real-time label requires one visible autonomous process to make an action depend on a bounded recent visual window and become unavailable or less effective during a delay.The review procedure requires a common witness for all three clauses, checks observation-free and bounded-window policies, and calls for an unresolved label when the source does not determine the result.Source: §H.2.2 · p. 31

I EXTENDED RELATED WORK

  • Interactive visual benchmarks span web, desktop, mobile, games, and embodied environments, but tasks requiring many capabilities at once do not reveal which capability limits performance.Prior work decomposes capabilities or varies task conditions, but GYM-ANYTHING-RT annotates required capabilities and varies difficulty, interface, and real-time setting on the same minigame.Source: I EXTENDED RELATED WORK · pp. 32–33
  • Static-input VLM benchmarks test perception, temporal reasoning, GUI grounding, or image reasoning without requiring action, while GYM-ANYTHING-RT adds temporal reasoning, exploration, interface understanding, real-time interaction, and configuration-level capability annotations.It complements interactive work such as iVISPAR and VisGym.Source: I EXTENDED RELATED WORK · p. 33
  • GYM-ANYTHING-RT compares the same minigame and seed in live, slowed, and paused settings, and pairs each real-time minigame with a non-real-time counterpart.Prior approaches include pausing games during inference, offering paused variants, comparing paused and real-time play, or advancing text games after a fixed number of generated tokens.Source: I EXTENDED RELATED WORK · p. 33
  • GYM-ANYTHING-RT extends environment-generation work to interactive visual minigames with variable difficulty and interfaces, a non-real-time counterpart, five difficulty levels, two interaction modes, and an oracle solution.It also uses CAPTCHA and puzzle challenges as seed minigames, adding controlled capability requirements, difficulty levels, interaction interfaces, and real-time settings.Source: I EXTENDED RELATED WORK · p. 34

Related papers

Loading 2610.37567v1…