PaperlayerView exact revision on arXiv

Source-linked AI summary

The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock

Ningxin Pan, Hanyu Li, Yeqing Shen, Ning Ding, Yehui Tang, Wentao Zhang

arXiv:2610.20665v1cs.AI

TL;DR

LLM agents may make good decisions too slowly for environments that keep changing during inference. The paper introduces RealtimeBench and TORSI, which uses offline teacher practice to give a fast actor reusable guidance. Two teaching rounds improve Jev across all six evaluated game types, while the experiments remain limited in their environments and duration.

  • Problem

    In continuously changing environments, reasoning and tool calls take time, and evaluations that pause during responses do not capture how continued evolution affects agent performance.

    Source: §1 · p. 1

  • Method

    RealtimeBench evaluates agents across game types and clocks, while TORSI uses a slow teacher’s offline practice to create validation-gated instructions for a fast student acting without teacher calls during gameplay.

    Source: §1 · pp. 1–3

  • Results

    13.51 to 24.19: after two teaching rounds, Jev’s average test score rose, improving all six game types; the same instructions improved average scores for three other models without retuning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results suggest that offline LLM reasoning and practice can provide reusable guidance for fast actors during live interaction, including actors beyond Jev in the evaluated settings.

    Source: §6 · p. 10

    Experiments cover six game types and two improvement rounds, with guidance developed separately by game type; transfer was tested in the same environments and does not establish transfer to unseen environments or sustained improvement over many rounds.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Large language model (LLM) agents can solve complex interactive tasks, yet reasoning and tool calls take time. In continuously changing environments, an otherwise correct action may arrive too late. We introduce RealtimeBench, with 180 tasks across six game types, to evaluate agents while games continue running during inference. Paired evaluations show that GPT-6.1-sol outperforms the faster Jev when games are paused, but falls behind when they keep running. We propose Two-Timescale Offline Recursive Self-Improvement (TORSI): a slow teacher studies failures and tests corrections in paused games between attempts, then writes reusable instructions for a fast student. The student acts without teacher calls during gameplay, and validation scores determine which updates are adopted. Two rounds raise Jev's average test score from 13.51 to 24.19 on a 0--100 progress scale, improving all six game types. The instructions also improve average scores for three other models without retuning. These findings point toward real-time intelligence: enabling powerful LLM agents to remain effective in a continuously changing world by organizing deliberation, learning, and action across timescales.

1 Introduction

RealtimeBench tests whether agents can act effectively while games continue changing during inference, and TORSI transfers slow offline reasoning into reusable guidance for a fast actor. The results show both a timing-related performance gap and gains from two rounds of teaching Jev.

1 Introduction

  • An appropriate action can arrive too late because the environment keeps changing during reasoning; Figure 1 illustrates delayed actions colliding with hazards, including a Snake turn applied after the game tick.Action age includes inference and scheduling delays.Source: §1 · p. 1, Figure 1 · p. 2
  • RealtimeBench covers 180 tasks across six game types and compares paused, live, and faster clocks to examine how continued evolution during inference affects performance.The same agent and tasks are compared under different clocks.Source: §1 · pp. 1–2
  • About 19 seconds per response: GPT-6.1-sol outperformed faster Jev in five of six game types with games paused, but live average scores were 1.51 and 13.51, respectively; no model averaged above 16.62 live.Scores measure in-game progress on a 0–100 scale.Source: §1 · p. 2
  • TORSI has a fast student act during gameplay while a slow teacher studies failures between attempts, tests corrections in paused games, and writes reusable instructions adopted only when validation scores improve.The teacher makes no further calls during gameplay; later rounds address the updated student’s remaining failures.Source: §1 · pp. 2–3
  • 13.51 to 24.19: after two teaching rounds, Jev’s average test score rose, improving all six game types; the instructions also helped three other models without retuning, though some game-specific regressions remained.The instructions were adopted after validation-task scores improved.Source: §1 · p. 2

2 Related Work

Prior work studies games and agents in evolving environments, real-time control, and improvement without weight updates. RealtimeBench explicitly controls the clock, while TORSI adds offline practice to iterative guidance updates.

2 Related Work

  • RealtimeBench explicitly controls the clock, supporting consistent comparisons of response speed and performance under real-time conditions alongside step-based and live-or-paused game benchmarks.The cited work includes SmartPlay, BALROG, lmgame-Bench, VideoGameBench, and Real-Time Reasoning Gym.Source: §2 · p. 3
  • Prior approaches span latency–quality trade-offs, time-budget adaptation, asynchronous inference, overlapping action execution, and hybrid fast-control/slow-planning agents.The passage cites Kang et al. (2025), Ma et al. (2026), Riemer et al. (2025), Black et al. (2025), and hybrid-agent work.Source: §2 · p. 3
  • GameSense uses VLMs to develop execution modules, whereas TORSI keeps actor weights and interfaces fixed and revises guidance through offline practice and validation.Decision models such as Jev, Clef, and StartLux score predefined choices and can serve as fast actors.Source: §2 · p. 3
  • Reflection, distilled insights, skill libraries, prompt or context evolution, and validated memory updates are prior weight-free improvement approaches; TORSI adds offline practice for live-clock control.The passage names GEPA, ACE, AHE, Recuris, and RSIAgent among these approaches.Source: §2 · p. 3

3 RealtimeBench

RealtimeBench evaluates agents on games that continue running during inference, using matched clocks and game-specific tasks to study how timing affects performance. Across clock conditions, GPT-6.1-sol leads Jev in five game types when paused, but scores lower under the live clock; sensitivity varies by game type.

3.1 Live-Clock Protocol

  • RealtimeBench keeps games running during inference, with the previous accepted control active until a new action takes effect.Response latency measures waiting for a response; action age also includes any delay before the selected action is applied.Source: §3.1 · p. 4

3.2 Game Types and Tasks

  • The benchmark covers six game types with different control demands, and assigns each game type 30 feedback, validation, and held-out test tasks.Tasks fix the game version, seed or stage, difficulty, action interface, and budget.Source: §3.2 · p. 4

3.3 Scoring

  • Progress scores capture partial achievement, including in games without a single completion condition, on a shared scale across game types.A score of 100 means reaching the reference target, not necessarily winning the native game; normalization does not imply equal difficulty.Source: §3.3 · pp. 4–5

3.4 Clock Sensitivity

  • Under the live clock, GPT-6.1-sol averages 1.51 versus Jev’s 13.51, despite outperforming Jev in five game types when paused.GPT-6.1-sol scores lower live than paused in all six game types; Jev does so in five, with Bird as the exception.Source: §3.4 · p. 5, Figure 4 · p. 6
  • Faster clocks generally reduce scores, but changes are not monotonic and sensitivity varies by game type.Bird’s relative robustness at moderate speeds may reflect its interface for executing preplanned flap schedules; the authors describe this as an interpretation, not an isolated causal test.Source: §3.4 · p. 5, Figure 4 · p. 6

4 TORSI: Two-Timescale Offline Recursive Self-Improvement

TORSI separates live action from offline learning: a fast student acts while a slow teacher studies failures, tests corrections, and turns them into reusable guidance. Validation gates whether guidance replaces the current version.

4 TORSI: Two-Timescale Offline Recursive Self-Improvement

  • A fast student acts without teacher calls during gameplay, while a slow teacher develops guidance offline from the student’s failures.Only the guidance changes; the student’s weights, interfaces, and environment remain fixed.Source: §4 · p. 6, §4.1 · pp. 6–7

4.2 Learning from Feedback and Offline Practice

  • Feedback trajectories capture observations, responses, latency, action age, and outcomes; offline practice tests alternative action sequences from saved states.The teacher’s practice is restricted to offline learning and excludes test states.Source: §4.2 · p. 7
  • Student practice checks draft guidance at live latency, and the teacher revises it using both successful and failed trials.Both types of trials enter the practice evidence.Source: §4.2 · p. 7

4.3 Turning Practice into Reusable Guidance

  • Reusable guidance converts state-specific practice into rules applicable in other states, with fixed fields for skills, actions, and conditions.For example, a Mario rule can tell the student to release jump before initiating another jump.Source: §4.3 · p. 7

4.4 Validation-Gated Rounds

  • A candidate replaces current guidance only when its mean validation score is strictly higher; accepted guidance starts the next round of feedback and learning.Adoption is decided separately for each game type, while test tasks are reserved for final reporting.Source: §4.4 · p. 7

5 Experiments

Across six game types, experiments evaluate baseline performance, TORSI updates, alternative adaptation methods, and transfer of guidance to other models. Validation-selected TORSI guidance improves Jev’s test performance, with gains also observed in other models.

5.2 TORSI Improvement Rounds

  • 24.19 versus 13.51: after two rounds, validation-selected guidance raises Jev’s mean test score by 10.68 points and improves all six game types.Adopted guidance is selected using validation scores; each reported score averages three runs.Source: Table 2 · p. 8, §5.2 · p. 9

5.1 Benchmark Evaluation

  • Under identical clocks and budgets, GPT-6.1-sol scores 1.51 with 18.97-second latency, while faster decision models score 1.65–16.62 with subsecond latency.Table 1 reports test scores on ten tasks per game type, mean latency, and API cost per task.Source: §5.1 · pp. 7–8, Table 1 · p. 8

5.2 TORSI Improvement Rounds

  • Validation selects R1 for Mario and Temple and R2 for Snake, Tetris, River, and Bird; R1 improves test scores in all six game types.Round-2 guidance fails the validation gate for Mario and Temple.Source: §5.2 · p. 9
  • 19.35: combining teacher and student practice yields a higher adopted mean test score than Base’s 13.51, versus 14.14 for teacher-only practice and 13.07 for student-only.Practice budgets are not matched across variants, so the comparison does not establish equal-budget efficiency.Source: §5.2 · p. 9

5.3 Comparison with Other Adaptation Methods

  • 24.19: TORSI has the highest adopted mean test score, compared with 13.22 for ACE, 16.59 for AHE, and 14.52 for GEPA.The adapted configurations differ in editable components and search configurations, and the comparison does not use matched budgets.Source: §5.3 · p. 9

5.4 Cross-Model Generalization

  • Without retuning, StartLux 4B rises from 12.22 to 25.05, StartLux 27B from 16.62 to 20.43, and MiMo from 5.95 to 7.30 in mean score.StartLux 27B regresses in Tetris and Temple; transfer was tested on the same game types, not unseen environments.Source: §5.4 · p. 10

6 Conclusion

Real-time intelligence requires capable decisions delivered in time. RealtimeBench and TORSI illustrate a framework and approach for studying and supporting that requirement.

6 Conclusion

  • RealtimeBench provides a shared framework for comparing approaches to capable decision-making under real-time conditions.TORSI demonstrates offline teacher reasoning and practice becoming reusable guidance for a fast actor during live interaction.Source: §6 · p. 10
  • Improvements across all six game types and gains from transferring guidance to other models suggest reuse beyond a single actor.The fast actor applies the guidance without waiting for teacher inference during gameplay.Source: §6 · p. 10

Limitations

The findings are bounded by structured state inputs, access to offline practice, and a limited experimental scope. In particular, the experiments do not establish transfer to unseen environments or sustained improvement over many rounds.

Structured observations

  • RealtimeBench evaluates structured state descriptions, not raw visual observations, so its findings do not cover end-to-end visual agents.The comparisons remain informative about decision-making under inference delay because evaluated models receive the same state information and legal actions.Source: Limitations · p. 10

Access to offline practice

  • TORSI depends on saved states or resettable environments for offline teacher practice, limiting direct application to physical or irreversible interactions.Extending it to other settings would require a way to test proposed corrections.Source: Limitations · p. 10

Generalization and experimental scope

  • Experiments cover six game types and two improvement rounds, with guidance developed separately for each game type.Cross-model transfer uses the same environments, includes some game-specific regressions, and does not establish transfer to unseen environments or sustained improvement over many rounds.Source: Limitations · p. 10

Ethical Considerations

More capable and responsive agents could create opportunities for misuse, while erroneous or harmful teacher guidance could be repeatedly applied or spread through reuse.

Ethical Considerations

  • More capable and responsive agents could enable unfair gameplay or unauthorized actions on digital services.Teacher-generated guidance may carry model errors or harmful instructions.Source: Ethical Considerations · p. 10
  • Fast actors can apply harmful guidance repeatedly before human intervention, and reusing it across models could spread these risks.Source: Ethical Considerations · pp. 10–11

A Benchmark and Guidance Details

The supplementary materials specify the benchmark’s intended use, task configurations, live-clock interfaces, and evaluation procedures. They also document guidance formats and examples, with figures illustrating full updates rather than isolating individual rules.

Intended use

  • RealtimeBench is intended for research and evaluation of latency-sensitive agents in controlled game environments, and TORSI studies offline teacher guidance for real-time control there.Source: A Benchmark and Guidance Details · p. 15

Guidance examples

  • Table 9 documents guidance formats and representative lessons from both rounds, marking candidate lessons rejected by validation.Figure 7 illustrates behavior before and after guidance, but only Mario shows recorded paused branches; Temple, Snake, and River illustrate rules rather than paired measured trajectories.Source: Table 9 · p. 17, Figure 7 · p. 18, A Benchmark and Guidance Details · p. 18

Benchmark-specific task construction

  • Each benchmark task fixes its game version, seed or stage, difficulty, interfaces, and episode budget to support paired clock comparisons with task identity and physics held fixed.Feedback, validation, and test tasks respectively support experience collection, update selection, and final evaluation; task settings are summarized in Table 7.Source: Table 7 · p. 16, A Benchmark and Guidance Details · p. 17

Live-clock adaptation and timing-aware interfaces

  • The execution framework measures observation, response, and action-application times separately while connecting structured observations to game-specific executable controls.For Snake, the adapter adds configurable presets, grid-topology features, response-delay information, and joint direction-and-turn-timing decisions anchored to the observation tick.Source: Table 8 · p. 16, A Benchmark and Guidance Details · p. 17

Evaluation and traceability

  • Evaluation uses fixed game-specific reference targets and task- and game-level aggregation while retaining each game’s native outcome.Recorded states and action logs support outcome checks; the release includes task definitions, per-episode outcomes, frozen agent specifications, reusable guidance, and replay logs.Source: A Benchmark and Guidance Details · p. 18

Mario guidance update

  • Figure 6 shows two improving student-practice examples and held-out Base-versus-R1 panels on the same task, but illustrates the full update rather than an individual rule’s effect.The archived three-task practice screen left guidance unchanged; Mario poses use native scenes replayed from archived episodes and are aligned using recorded camera positions.Source: Figure 6 · p. 16, A Benchmark and Guidance Details · p. 18

B Additional Experimental Results

The adaptation protocol compares methods through repeated feedback-task rollouts while keeping game mechanics and interfaces fixed. Response-latency results show that decision models are generally faster than GPT-6.1-sol, though some exceed a game tick or have long response tails.

B.1 Adaptation Comparison Protocol

  • All methods use Jev as the real-time student and GPT-6.1-sol with medium reasoning effort as the offline teacher, with ten feedback, validation, and test tasks per game type.The teacher is never queried during scored play, and a candidate is adopted only when its mean validation score strictly exceeds the current agent’s.Source: §B.1 · p. 18
  • Adaptation methods use repeated task-level rollouts on the same ten feedback tasks, while episode budgets count student executions and separate validation and test evaluations.Tables 10 and 11 summarize method-specific adaptations, search configurations, and stopping conditions.Source: §B.1 · p. 19, Table 10 · p. 19, Table 11 · p. 19
  • Table 12 reports mean test scores over ten tasks for paused and faster-clock conditions, while Table 13 defines response latency as observation-to-response wall-clock time.Latency includes network time and all decision questions in a request.Source: Table 12 · p. 20, Table 13 · p. 20

B.2 Response Latency

  • Local models run on a single NVIDIA L20Y GPU without tensor or pipeline parallelism, and their compute cost is reported as wall-clock response time.DeepSeek v4.1 Flash uses thinking mode disabled; MiMo v2.6 Ultra Speed uses the provider’s default decoding settings.Source: §B.2 · p. 20
  • GPT-6.1-sol’s game type medians range from 5 to 31 seconds, while decision models have median response times below two seconds in every game type.In Snake, all decision models except StartLux 27B (0.86 s) stay below the 667 ms tick; MiMo reaches 13.64 s at P95 in Mario.Source: §B.2 · p. 20

B.3 Token Usage and Computational Cost

This section reports token usage, API costs, and GPU compute for recorded runs, then identifies the model and game software sources. The listed implementations include both local models and API-based models.

B.3 Token Usage and Computational Cost

  • Table 14 reports mean token usage and mean API cost per recorded task run, with local-model API fees listed as zero because those models incur no API fees.Zero output tokens reflect decision-interface reports, zero API fees exclude local compute costs, and dashes indicate unavailable usage or billing.Source: §B.3 · p. 20, Table 14 · p. 21
  • During evaluation, serving open-source models used 46.47 GPU-hours; teacher API costs were $2/M input tokens and $10/M output tokens.Source: §B.3 · p. 21

B.4 Model Sources

  • The model-source list identifies evaluated checkpoints and official API documentation for MiMo v2.6 Ultra Speed and DeepSeek v4.1 Flash.The cited model cards include GLiClass, JevK5, Open-Jev-2B, mini-Jev, and StartLux 27B.Source: §B.4 · p. 21, C Software Sources · p. 21

C Software Sources

  • The software sources name the Mario emulator stack and rendering frameworks, including Three.js for Temple and Pygame-based implementations for River and Bird.The listed game implementations also identify the source projects adapted for Snake, Tetris, Temple, River, and Bird.Source: C Software Sources · pp. 21–22

Related papers

Loading 2610.20665v1…