PaperlayerView exact revision on arXiv

Source-linked AI summary

One Rollout Is All You Get: Fully Test-Time Adaptation for GUI Agents

Ziqiang Wang, Li Gu, Zhixiang Chi, Linlian Jiang, Zihuan Jiang, Linqiang Guo, Siobhan Reid, Zhi Liu, Yang Wang

arXiv:2609.34321v1cs.LG

TL;DR

GUI agents discard deployment experience, while many existing improvement approaches depend on resources unavailable in live use. The paper defines fully test-time adaptation and proposes SOLO, which learns from episodes judged and relabeled by auxiliary models. On recurring benchmark streams, SOLO improves over frozen agents and exceeds two memory methods on web streams.

  • Problem

    Deployed GUI agents have fixed weights and discard experience, while existing improvement methods rely on ground truth, extra rollouts, or a learning stage beyond deployment.

    Source: §1 · p. 1

  • Method

    SOLO uses auxiliary models to judge episodes and relabel completed prefixes, then updates a small adapter by top-K self-distillation when its sliding window contains a judged success.

    Source: §1 · p. 2

  • Results

    SOLO exceeds frozen agents by three to six success-rate points across every stream with both agents, and exceeds two memory methods on web streams.

    Source: §4.2 · p. 6

  • Takeaways & Limitations

    The results show that an agent can improve its own weights during deployment from its own episodes, without ground truth and with one attempt per task occurrence.

    Source: §4.2 · p. 6

    The streams repeat benchmark tasks three times rather than using user logs, so real-deployment recurrence and the resulting gain are unmeasured; MobileWorld judged successes are rare.

    Source: §6 · p. 9

Abstract

from arXiv · show

GUI agents are deployed with frozen weights and discard everything they experience on the job. Existing ways to update an agent's weights assume something deployment withholds: ground truth, rollouts beyond the single attempt (retries, samples, practice runs), or a learning phase other than deployment. Because GUI actions can be irreversible, a deployed agent gets one attempt per task occurrence, in arrival order, and every attempt counts. No ground truth is available at any point. We define fully test-time adaptation for GUI agents by these constraints and pair it with a minimal weight-space method, SOLO. Auxiliary models read each episode: a judge selects the episodes it deems successful, and a proposer-verifier pair relabels a failed episode's prefix with the subtask that prefix completed. Admitted episodes enter a short sliding window, and each admission updates a small adapter by top-K self-distillation on the agent's own predictions, provided the window holds a judged success. On recurring task streams built from WebArena, VisualWebArena and MobileWorld, SOLO improves on the frozen agent with both UI-TARS-7B and Qwen3-VL-8B, by three to six points of success rate, and exceeds two in-setting memory methods on the web streams.

1 INTRODUCTION

GUI agents may face recurring tasks in deployment, yet their fixed weights discard what they experience. The paper defines fully test-time adaptation under deployment constraints and proposes SOLO, which learns from episodes read by auxiliary models.

1 INTRODUCTION

  • GUI agents may repeat the same errands and encounter the same apps and accounts, but fixed weights discard experience, while actions can be irreversible and each occurrence gets one attempt.The agent acts in task-arrival order, and every attempt counts.Source: §1 · p. 1
  • Existing improvement methods rely on resources unavailable in live deployment, including ground-truth supervision, extra rollouts, or a non-deployment learning stage.Table 1 places approaches by the resources they use and identifies the setting’s permitted persistent state as memory or weights.Source: §1 · p. 1, Table 1 · p. 2
  • Fully test-time adaptation permits no ground truth, no rollouts beyond one attempt per task occurrence, and no learning phase outside deployment.Automated signals computed from an agent’s own episodes and persistent memory or weights are permitted.Source: §1 · p. 2
  • SOLO uses a judge to select episodes and a proposer–verifier pair to relabel a completed prefix of a judged failure under its completed subtask.The auxiliary models select what the agent learns from and under which instruction; they do not generate actions or token targets.Source: §1 · p. 2
  • Each admission updates a small adapter by top-K self-distillation on the agent’s own predictions, only while the sliding window contains a judged success.A relabeled prefix enters the same window in place of the failed episode.Source: §1 · p. 2
  • With two open agents, SOLO improves success rate by three to six points over the frozen agent on recurring streams built from WebArena, VisualWebArena, and MobileWorld, and exceeds two in-setting memory methods on web streams.Source: §1 · p. 3

2 SETTING: FULLY TEST-TIME ADAPTATION FOR GUI AGENTS

The setting defines fully test-time adaptation for GUI agents as learning during an ordered deployment stream without ground truth or more than one attempt per task occurrence. It evaluates each episode as it is produced and permits persistent state in either memory or weights.

2 SETTING: FULLY TEST-TIME ADAPTATION FOR GUI AGENTS

  • Each task occurrence pairs an instruction with an environment state; tasks sharing a template may recur, and some exact instances may recur.The agent produces one episode per occurrence, with actions that can take effect irreversibly.Source: §2 · p. 3
  • The setting withholds ground truth, retries, alternative samples, branching, resets, practice, and any training phase before or during the stream.Automated models may compute signals from produced episodes, and persistent state may be external memory or the agent’s weights.Source: §2 · p. 3
  • Performance is measured prequentially: each episode is scored as produced, and success rates are compared with the same agent frozen on the identical stream and order.Reported quantities include overall stream success and its profile across rounds.Source: §2 · p. 3
  • In-setting memory systems retain experience externally and return it to context or replay, whereas SOLO changes the agent’s weights.The compared workflow-memory and Darwinian Memory systems satisfy the three constraints.Source: §2 · p. 4
  • Unlike perception-model test-time adaptation, GUI adaptation has a one-attempt constraint because an acted-on occurrence cannot be replayed after an update.The paper relates the episode-reading signal for GUI agents to confidence-based signals for perception models.Source: §2 · p. 4

3 METHOD: SOLO

SOLO adapts a small adapter online from single-attempt episodes, using auxiliary models to select judged successes and verify relabeled prefixes. It trains on the agent’s own top-K predictions in a sliding window anchored by a judged success.

SOLO pipeline

  • SOLO processes tasks in arrival order, admits judged successes or verified relabeled prefixes, and updates its adapter from a sliding window only when that window contains a judged success.The window holds the W most recent admissions; apart from the window, the adapter is the only state carried between tasks.Source: §3 · p. 4

Signals and relabeling

  • For judged failures, the proposer may label a prefix with a completed subtask, which the verifier checks using the subtask, prefix actions, and resulting screen.The proposed subtask must be non-trivial and about the same objects; the prefix must contain at least two actions and avoid repeated-action loops.Source: §3 · p. 4

Objective: top-K self-distillation

  • Top-K self-distillation targets the pre-update policy’s own truncated predictions at every response position, preserving its ranking rather than imitating executed tokens outright.The loss covers reasoning and action tokens; small K moves probability from the distribution’s tail toward its own top-K candidates, while K = |V| is a no-op.Source: §3 · pp. 4–5

Update

  • Each admission triggers one gradient step over the window, and each episode participates in at most W updates before leaving it.The adapter is about 0.2% of the backbone; per task, SOLO uses one judge call, at most two further calls for a judged failure, and at most one gradient step over W episodes.Source: §3 · pp. 4–5

4 EXPERIMENTS

Across three recurring benchmark streams and two open GUI agents, SOLO improves on frozen agents and outperforms both memory baselines on the web streams. Ablations, auxiliary-model substitutions, and round-by-round results characterize which components matter and when gains appear.

4.2 MAIN RESULT: ADAPTATION IS FEASIBLE

  • SOLO beats the frozen agent by three to six success-rate points on every stream with both agents, with each margin exceeding the sum of the two standard deviations.On the web streams it also exceeds the better memory baseline by three to six points with both agents; on MobileWorld it is comparable to AWM-online.Source: Table 2 · p. 6, §4.2 · p. 6

4.1 SETUP AND PROTOCOL

  • The three streams contain recurring tasks, run for three rounds, and are evaluated over 324 WebArena, 411 VisualWebArena, and 120 MobileWorld episodes.The study adapts UI-TARS-7B and Qwen3-VL-8B; all methods use identical streams, and the memory baselines share the judge as their success signal.Source: §4.1 · pp. 5–6

4.3 ABLATIONS

  • Removing any component lowers the mean across the two web streams; hindsight relabeling matters most, while one-hot imitation costs three WebArena points and lengthens episodes toward the step budget.The one-hot variant reaches the frozen level on WebArena by round three, while its VisualWebArena episodes also lengthen but wins hold within three rounds.Source: §4.3 · p. 7, Table 3 · p. 7

4.4 SENSITIVITY TO THE AUXILIARY MODELS

  • Replacing all three auxiliary models keeps SOLO above the frozen agent; with gpt-5 or gpt-5-nano, success rates stay within two points on both web streams.The agent’s own model is three to six points lower on VisualWebArena than gpt-5-mini, and judge precision does not order the success-rate results.Source: §4.4 · p. 7, Table 4 · p. 7

4.5 WHERE THE GAIN APPEARS

  • The margin appears in the first round for two agent-stream pairs, but opens in rounds two and three for the other web pairs; MobileWorld shows one-to-two-task margins each round.The authors interpret this pattern as adaptation to the deployment distribution rather than memorization of particular tasks.Source: §4.5 · p. 8, Figure 2 · p. 8

5 RELATED WORK

Related work adapts models through test-time updates, extra rollouts or training phases, external experiential memory, and learning from an agent’s own outputs or failures. SOLO shares elements with these approaches but operates from one attempt per task and carries its learning in weights.

Test-time adaptation

  • Test-time adaptation updates models on arriving unlabeled inputs; SOLO resembles EATA’s sample selection, CoTTA’s soft pseudo-labeling, and Tent’s restricted parameter updates, but uses episode-level judgments.Tent minimizes prediction entropy and updates normalization parameters; continual methods also use mechanisms to guard against drift.Source: §5 · p. 8

Adaptation through extra rollouts or a training phase

  • Other agent methods use extra candidate rollouts, retries, exploration episodes, or a separate reward-driven training phase, resources unavailable under SOLO’s one-attempt setting.Examples include GTA1, JIT-RL, EvoTest, and GTTA (Li et al., 2026a), and online reinforcement-learning flywheels (Bai et al., 2024; Qi et al., 2025; Wang et al., 2025a; Chen et al., 2026b; Gu et al., 2026).Source: §5 · pp. 8–9

Experiential memory

  • Experiential-memory systems either build experience before tasks count or learn during the stream; SOLO instead uses weights as persistent state and is compared with two in-setting memory systems.Practice-based examples include Reflexion and ExpeL; stream-based examples include AWM-online (Wang et al., 2024; Pan et al., 2024) and Darwinian Memory (Mi et al., 2026).Source: §5 · p. 9

Self-imitation and learning from failures

  • SOLO selects passing episodes online and distills the policy’s truncated distribution, while its failure branch relabels a verified prefix with the subtask it completed.This connects self-imitation and rejection sampling (Yuan et al., 2023) with hindsight relabeling (Ding, 2026).Source: §5 · p. 9

6 DISCUSSION AND LIMITATIONS

SOLO improves agents over three rounds on recurring benchmark streams, but the gains are not measured on user logs or longer deployments. The discussion also identifies dependence on auxiliary models, drift, and adversarial page content as boundaries and risks.

6 DISCUSSION AND LIMITATIONS

  • Within three rounds, SOLO improves deployment-stream performance on three benchmarks with two agents, but the streams use recurring benchmark tasks rather than user logs.How often real deployments repeat errands, and the size of gains there, are not measured.Source: §6 · p. 9
  • The signal depends on auxiliary models, and page content that sways the judge or proposer can train the adapter; adversarial pages are unstudied.Each episode costs one judge call and each judged failure at most two more.Source: §6 · p. 9
  • Under the same judge, SOLO exceeds two prompt-memory methods on web streams and is comparable to AWM-online on mobile.The comparison supports the in-weight route but is not a verdict between memory and weights, which the passage describes as complementary.Source: §6 · p. 9
  • Behavior over longer deployments is untested, and there is no ground truth in the setting to detect degradation.Resetting the adapter to zero restores the frozen agent exactly, but detecting when to reset is unevaluated.Source: §6 · p. 10
  • Episode lengths rise with Qwen3-VL-8B, while they shorten slightly with UI-TARS-7B.For Qwen3-VL-8B, episodes lengthen by two to three steps and about one in six ends at the budget; UI-TARS-7B episodes shorten slightly.Source: §6 · p. 10
  • The experiments use language and vision-language models as auxiliary components and tools, while authors designed, ran, and checked all experiments.The paper states that it will release code, stream files, and run logs, and describes the configuration, streams, runs, and prompts.Source: AI USE STATEMENT · p. 10, REPRODUCIBILITY STATEMENT · p. 10
  • Experiments use self-hosted benchmark websites and Android emulators, not real user accounts, personal data, or live services.The authors regard adversarial-page study as a prerequisite for deployment outside a sandbox.Source: ETHICS STATEMENT · p. 10

A SIGNAL STUDY: ENTROPY MINIMIZATION

The study tests entropy minimization as a GUI agent’s test-time signal against a frozen Qwen3-VL-8B agent on two web streams. It does not help on either stream and collapses the agent on VisualWebArena.

A SIGNAL STUDY: ENTROPY MINIMIZATION

  • Entropy minimization does not help on either stream, and on VisualWebArena it collapses the agent.The table reports wins, first-step endings, and mean response entropy for Qwen3-VL-8B against the frozen agent across both web streams and three runs.Source: A SIGNAL STUDY: ENTROPY MINIMIZATION · p. 15, Table 5 · p. 15
  • Response entropy falls by an order of magnitude along the stream in every run, while WebArena success stays at the frozen level.On VisualWebArena, wins fall from the frozen level in round one to a handful in round three.Source: A SIGNAL STUDY: ENTROPY MINIMIZATION · p. 15
  • On VisualWebArena, episodes ending at their first step rise from about a quarter of a round under the frozen agent to between half and most of the third round.The first-step episodes end with an answer or declaration of completion.Source: A SIGNAL STUDY: ENTROPY MINIMIZATION · p. 15

B METHOD DETAILS

SOLO updates a low-rank adapter through top-K self-distillation over a short episode window. Its failure branch proposes and verifies relabeled prefixes, while the implementation also specifies adapter settings and baseline adaptations.

B METHOD DETAILS

  • The self-distillation gradient moves probability mass from outside the top-K support onto its tokens, and vanishes when that support already holds all the mass.The targets come from the same forward pass with gradients stopped; all response tokens, including reasoning and actions, are supervised.Source: B METHOD DETAILS · p. 15
  • The adapter is rank-16 LoRA with scale α = 32 on attention-output and three MLP projections in the decoder’s upper half; the vision encoder is untouched.It uses AdamW at learning rate 10^-5, a four-episode window, and support K = 4; zero-initialized output makes the adapted policy start identical to the frozen agent.Source: B METHOD DETAILS · p. 15
  • For a judged failure, the proposer names a completed subtask and earliest completion step or abstains; the verifier checks the proposed instruction against the prefix actions and final screenshot.Guards require at least two prefix actions and limit repeated actions to 0.3 of steps; most guard rejections are single-action prefixes, and the verifier rejects between one in thirteen and one in four reaching it.Source: B METHOD DETAILS · p. 16
  • AWM-online re-induces a site workflow after judged successes, while Darwinian Memory stores and retrieves subtask trajectories; both keep the policy frozen and learned content in the prompt.The implementation adapts their action vocabulary and caps the workflow block at 4,000 characters; Darwinian Memory uses IDF-weighted token cosine instead of sentence embeddings and recalibrates its threshold.Source: B METHOD DETAILS · p. 16

C STREAMS AND PROTOCOL

The evaluation uses three fixed task streams, each with three rounds, and benchmark-specific state resets and evaluators. Web streams contain repeated template instances; MobileWorld tasks are individual instances.

C STREAMS AND PROTOCOL

  • WebArena’s stream contains 108 instances from 27 templates, with four instances per template across shopping, GitLab, shopping administration, and a forum.The stream covers ten shopping, nine GitLab, four shopping-admin, and four forum templates.Source: C STREAMS AND PROTOCOL · p. 16
  • VisualWebArena’s stream contains 137 instances from 39 templates, with three or four instances per template across shopping, classifieds, and a forum.The stream covers 28 shopping, nine classifieds, and two forum templates.Source: C STREAMS AND PROTOCOL · p. 16
  • MobileWorld’s stream contains 40 tasks, each treated as one instance because its tasks have no parameterized instances.The tasks cover Mastodon, Gmail, native Android apps, and device settings.Source: C STREAMS AND PROTOCOL · p. 17
  • Every stream has three rounds with seeded permutations that prevent an instance from recurring within eight rows; benchmark procedures reset sites or devices between rounds or tasks.Web episodes have a 30-step budget and MobileWorld episodes a 40-step budget.Source: C STREAMS AND PROTOCOL · p. 17
  • Benchmark evaluators remain hidden from the agent and auxiliary models; VisualWebArena uses gpt-5-mini for its image-query evaluator, and MobileWorld uses benchmark checkers.Source: C STREAMS AND PROTOCOL · p. 17

D FULL RESULTS

Across three runs, results are reported as mean and standard deviation of wins and success rate; episode-length analysis shows different trends for the two agents and the one-hot variant.

D FULL RESULTS

  • Every method and variant was run three times on the identical stream and order; tables report mean and sample standard deviation of wins plus the success rate of the mean.Table 7 lists the runs behind Table 2, with wins split by round, mean ± standard deviation of wins, and success rate.Source: D FULL RESULTS · p. 17, Table 7 · p. 17
  • Table 8 reports Qwen3-VL-8B results on both web streams, including per-run wins by round, mean ± standard deviation of wins, success rate, and mean updates per run.Table 9 lists the runs behind Table 4, including judge precision and success rate.Source: Table 8 · p. 18, Table 9 · p. 18
  • Across three rounds, full-method episodes lengthen by two to three steps, while the share ending at the step budget rises to about one in six.The one-hot variant lengthens episodes far more on both streams; across three runs per stream, its median gradient norm is 0.56 to 0.61, versus 0.02 for top-K self-distillation at the same learning rate.Source: D FULL RESULTS · p. 18
  • With UI-TARS-7B, full-method episodes shorten slightly and fewer end at the step budget than for the frozen agent.Table 10 reports mean steps and the share of episodes ending on the step budget per round.Source: D FULL RESULTS · p. 18, Table 10 · p. 18

E JUDGE AND AUXILIARY MODELS

The judge admits many evaluator successes but also admits episodes the evaluator rejects; replacing it with the evaluator does not improve success rate beyond run-to-run spread. On VisualWebArena, swapping either auxiliary role accounts for most of the observed loss.

E JUDGE AND AUXILIARY MODELS

  • On web streams, the judge admits six to eight of every ten evaluator successes, while a third to a half of its admissions are not evaluator successes.On MobileWorld, it admits nearly every evaluator success, plus about as many rejected episodes with UI-TARS-7B and about half as many with Qwen3-VL-8B.Source: E JUDGE AND AUXILIARY MODELS · p. 19, Table 11 · p. 19
  • Replacing the judge with the benchmark evaluator leaves success rate the same as gpt-5-mini within run-to-run spread, despite eliminating false admissions.The default judge's admissions are one-third to one-half false; removing them does not raise success rate beyond the spread.Source: E JUDGE AND AUXILIARY MODELS · p. 19, Table 12 · p. 19
  • On VisualWebArena, using the agent’s model as judge alone lowers precision to about a third and success rate to 18.2; using it as proposer and verifier alone yields 17.4.Swapping both roles yields 17.3; either role alone accounts for most of the loss, with little additional loss from swapping both.Source: E JUDGE AND AUXILIARY MODELS · p. 19, Table 13 · p. 19

F COMPUTE AND COST

SOLO adds auxiliary-model calls and a gradient step per episode, with extra elapsed time on web streams.

F COMPUTE AND COST

  • SOLO adds one judge call per episode and proposer and verifier calls on most judged failures.The reported runs average about two auxiliary calls per episode.Source: F COMPUTE AND COST · p. 20
  • A proposer call processes every screenshot of a failed episode and uses five to seven thousand tokens; a verifier call uses about one thousand.Source: F COMPUTE AND COST · p. 20
  • On web streams, added calls and updates took about half a minute to a minute per episode, against rollouts of one to one and a half minutes.Each run used one NVIDIA H200 shared with other runs, so absolute times vary with system load and are not compared across runs.Source: F COMPUTE AND COST · p. 20

G PROMPTS

The section documents prompts and instructions for agents, judges, proposer-verifier components, and memory baselines, covering task discipline, tool-call formats, trajectory evaluation, subtask identification, and presentation of prior workflows or retrieved units. Across these prompts, it emphasizes screenshot-verifiable outcomes, task constraints, and limits on what partial evidence can establish.

Agents

  • UI-TARS-7B and Qwen3-VL-8B use different web prompt formats, while both receive shared information-task and step-discipline notes; MobileWorld agents use benchmark prompts.UI-TARS-7B uses its own web prompt, and Qwen3-VL-8B uses its release computer-use prompt with the MobileWorld response format; on MobileWorld, both use benchmark prompts.Source: G PROMPTS · p. 20
  • Agents must answer information-seeking tasks with the answer action, stop once the goal is met, and avoid repeating actions that did not change the page.For non-question tasks, they should finish rather than return an empty answer.Source: G PROMPTS · pp. 20, 21, 23
  • The web and mobile agents use distinct action spaces but share a constrained response format requiring a concise Thought, an imperative Action, and one tool-call block.The web interface includes mouse, keyboard, scroll, wait, answer, and navigation actions; the mobile interface includes touch, swipe, typing, app-opening, and system-button actions.Source: G PROMPTS · pp. 21, 23

Judge

  • The judge evaluates whether the objective was achieved using trajectory evidence, allowing corrected mistakes and mid-trajectory success unless later actions undo it.It should consider screenshots, actions, reasoning, and UI state indicators, while avoiding judgments that are either overly strict or overly lenient.Source: G PROMPTS · p. 24
  • Verification rules require evidence of committed persistent-state changes and reject empty results when the query’s construction cannot be verified.Entered but uncommitted input is not completion; an empty result from a malformed or mis-scoped query is a failure.Source: G PROMPTS · pp. 24–25
  • For collection-wide aggregates or extrema, the judge requires evidence covering the full collection rather than a partial view.The prompt gives exhausted pagination or a visible total/count indicator as examples of adequate coverage.Source: G PROMPTS · p. 25
  • The judge reviews the chronological trajectory and requires visible evidence for the requested item and use of any prescribed route.Membership in the right set alone is insufficient when the task specifies attributes or a particular method.Source: G PROMPTS · p. 25

Proposer

  • A proposed relabeling must be a same-site subtask or prerequisite completed by the earliest qualifying prefix, not an unrelated task or the original task itself.The prefix must show the result after its last action.Source: G PROMPTS · p. 26
  • A valid pseudo-task must be concrete, non-trivial, screenshot-verifiable, and end in a successful state produced by a page-changing action.Unsubmitted text, open menus, highlighted elements, errors, and empty or no-results pages do not qualify.Source: G PROMPTS · p. 26

Verifier

  • The verifier judges completion conservatively from the final screenshot, requiring the correct page, object, or requested state to be evident.It returns false when the screenshot does not demonstrate completion or the task is vague or trivial.Source: G PROMPTS · p. 27

Memory baselines

  • AWM-online prompts induce reusable workflows from prior tasks, while Darwinian Memory presents retrieved units from earlier successes on similar goals and starting states.The workflow prompt asks for non-overlapping routines of at least two steps, with variable elements represented by placeholders; retrieved steps are replayed only where they fit the current screen.Source: G PROMPTS · pp. 27–29
Loading 2609.34321v1…