PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Agents for Computer Use

Gonzalo Gonzalez-Pumariega, Vincent Tu, Chih-Lun Lee, Jiachen Yang, Ang Li, Xin Eric Wang

arXiv:2510.02250v2cs.AIcs.CLcs.CVcs.LG

TL;DR

Long-horizon computer-use tasks remain unreliable, and evaluating multiple rollouts makes effective scaling difficult. BJudge represents agent executions as behavior narratives and compares candidates at that level. It achieves 72.6% success on OSWorld and also demonstrates generalizability on WindowsAgentArena and AndroidWorld.

  • Problem

    Long-horizon computer-use agents remain unreliable, while selecting correctly among multiple rollouts is difficult because trajectories are dense and may admit multiple valid solutions.

    Source: §1 · p. 1

  • Method

    BJudge converts agent trajectories into compact behavior narratives and uses them to compare and select among multiple rollouts.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    72.6% OSWorld success at 100 steps surpasses the previous SoTA by 9.2% absolute and human performance at 72.36%, with generalizability also demonstrated on WindowsAgentArena and AndroidWorld.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The paper reports that selecting among parallel trajectories substantially improves CUA robustness and task success rates.

    Source: §6 · p. 8

    The method assumes independent rollouts from the same initial state; live desktops or shared online resources can introduce cross-run interference.

    Source: §5 · p. 8

Abstract

from arXiv · show

Computer-use agents (CUAs) hold promise for automating everyday digital tasks, but their performance on long-horizon, complex problems remains unreliable. Single-rollout execution is brittle, with small errors compounding over time and leading to high variance in outcomes. While prior work has attempted to scale within a single rollout, such approaches have yielded limited gains. Scaling over multiple rollouts offers a more promising alternative but doing so effectively is challenging due to the difficulty of evaluating and selecting among long-horizon agent behaviors. We introduce Behavior Judge (BJudge), which addresses this challenge by representing agent executions as behavior narratives and comparing candidate behaviors at this level, substantially improving robustness and success rates. Using multiple rollouts, BJudge establishes a new state of the art (SoTA) in OSWorld at 72.6%, significantly outperforming prior methods and surpassing human-level performance at 72.36%, with comprehensive ablations validating key design choices. We further demonstrate strong generalization results to different operating systems on WindowsAgentArena and AndroidWorld. Crucially, our results highlight the strong effectiveness of scaling CUAs, when you do it right: effective scaling requires structured trajectory understanding and selection, and BJudge provides a practical framework to achieve this.

1. Introduction

Long-horizon computer-use tasks remain unreliable, and scaling multiple rollouts makes selecting a correct trajectory a central challenge. Behavior Judge (BJudge) compares rollouts through behavior narratives, achieving strong benchmark results with an improved agent framework.

1. Introduction

  • 72.6% OSWorld success at 100 steps surpasses the previous SoTA by 9.2% absolute and human performance at 72.36%.The reported result uses BJudge together with an improved CUA baseline.Source: Figure 1 · p. 1, §1 · p. 2
  • Long-horizon, complex tasks remain unreliable as small mistakes accumulate and environmental noise destabilizes execution.The resulting outcomes have high variance, with the same agent succeeding on one attempt and failing on another.Source: §1 · p. 1
  • Single-rollout scaling has yielded limited gains, while wide scaling generates parallel rollouts and selects among them.Agents often succeed on complementary task subsets, motivating selection across candidate trajectories.Source: §1 · p. 1
  • Evaluating long-horizon rollouts is difficult because trajectories are information-dense, multimodal, and may have multiple valid solutions.Scripted automatic evaluation can struggle to determine whether a trajectory is correct.Source: §1 · pp. 1–2
  • BJudge converts trajectories into concise behavior narratives that preserve task-relevant action effects and supports selection among multiple rollouts.The paper also reports ablations and zero-shot generalizability on WindowsAgentArena and AndroidWorld.Source: §1 · p. 2

2. Background

The background frames computer-use agents as policies that produce trajectories and contrasts single-trajectory approaches with selecting among multiple candidate rollouts. Wide scaling requires judging long trajectories, a problem BJudge addresses by comparing behavior narratives.

2.1. Computer-Use Agents

  • A CUA can be framed as a POMDP with computer states, observations, actions, transitions, instructions, and rewards.The history h_t denotes the time-ordered observations and actions up to the current observation.Source: §2.1 · p. 2
  • Computer agents are commonly described as policies that produce one trajectory, whereas this work scales candidate trajectories across multiple base models or policies.The authors describe their focus on scaling candidate solution trajectories as novel to their knowledge.Source: §2.1 · p. 2

2.2. Test-Time Scaling

  • Step-wise best-of-N selection can improve local actions but may commit a rollout to a harder route instead of exploring alternative trajectories.Wide scaling instead selects among trajectories generated by multiple base agents or models.Source: §2.2 · p. 2
  • Wide scaling faces an evaluation bottleneck: existing judges are often web-tuned, require human-defined rubrics, and evaluate single trajectories.BJudge compares behavior narratives to distinguish the best among candidate trajectories.Source: §2.2 · p. 3

3. Method

Behavior Judge improves multi-rollout computer-use evaluation by converting trajectories into behavior narratives and selecting among candidates through comparison. Its method also incorporates an improved Agent S3 baseline with programmatic editing and a flat policy.

3. Method

  • The overall framework builds on Agent S2 and combines a Behavior Narrative Generator with a Comparative Behavior Evaluator to select among multiple full-length rollouts.The generator describes action effects while discarding irrelevant details, and the evaluator compares the resulting candidate narratives.Source: §3 · p. 3

3.1. Behavior Narrative Generation

  • Behavior Judge converts each rollout’s transitions into facts describing what the agent did and the task-relevant changes, retaining initial and final screenshots as context.For pointer interactions, the generator uses visual augmentations to help verify intended changes; the after-action screenshot is taken three seconds later to allow delayed changes to appear.Source: §3.1 · p. 3

3.2. Comparative Behavior Evaluator

  • The evaluator compares behavior narratives across candidate rollouts and selects one trajectory, separating long-horizon behavior description from candidate selection.The paper uses a single-round multiple-choice comparison of all narratives, which is described as more informed than independent ranking and more token-efficient and faster than multi-round tournaments.Source: §3.2 · pp. 3–4

3.3. An Improved Agentic Framework Baseline

  • Agent S3 adds programmatic editing when appropriate and replaces manager-worker hierarchy with a flat policy that can replan from the current instruction and history.The GUI policy can invoke a coding agent for tasks such as bulk operations or file transforms; the flat policy is motivated by short-horizon planning in contemporary foundation models and the risk of stale subgoals.Source: §3.3 · pp. 4–5

4. Experiments and Analysis

The experiments evaluate BJudge across benchmarks, rollout budgets, trajectory representations, failure modes, resource allocations, and model mixtures. They report improved OSWorld performance and results on WindowsAgentArena and AndroidWorld.

4.1. Experimental Setup

  • Evaluation covers 361 OSWorld tasks and two additional operating-system benchmarks; the best configuration selects among five GPT-5 and five Opus 4.5 rollouts.Agent S3 rollouts use coding-agent support on OSWorld and WindowsAgentArena, but not AndroidWorld because of emulator constraints.Source: §4.1 · p. 5

4.3. How does Behavior Judge perform under equal rollout budgets?

  • Under equal rollout budgets, BJudge outperforms WebJudge overall and scales better as rollout count grows, while WebJudge plateaus and drops around N=10.WebJudge is adapted to rank trajectories independently; BJudge compares trajectories against each other.Source: §4.3 · p. 6, Figure 4 · p. 6

4.4. How do behavior narratives compare to other trajectory representations?

  • Behavior narratives improve BJudge by 3.4% over the best representation baseline, supporting action-effect facts over screenshots or summaries alone.The comparisons include screenshot-only input, 3–6-sentence trajectory summaries, and captions of individual screenshots.Source: §4.4 · pp. 6–7, Table 3 · p. 7

4.5. BJudge Accuracy and Failure Analysis

  • BJudge reaches 78.4% selection accuracy on 159 judge-subset problems and 92.8% human-evaluated accuracy on 35 remaining problems.Of 12 categorized failures, eight involved behavior-narrative hallucinations and four involved Code-GUI handoffs.Source: §4.5 · p. 7

4.6. How does Behavior Judge perform under varying resource budgets?

  • At budget 200, BJudge with N=4 improves 4.25% absolutely over a single agent; at budget 1000, N=10 yields the largest gain, 6.38%.A single agent performs best at smaller budgets, while larger worker counts become more effective as total resources increase.Source: §4.6 · p. 7, Figure 5 · p. 7

4.8. Generalization to Other Benchmarks

  • With N=3, BJudge improves success by 6.4% on WindowsAgentArena and 3.5% on AndroidWorld over the Agent S3 baseline.The reported comparisons are with Agent S3 on both benchmarks.Source: Table 6 · p. 8, Table 7 · p. 8, §4.8 · p. 8

4.7. How should we select a mixture-of-models ensemble for rollouts?

  • The GPT-5 and Claude Opus 4.5 mixture has the highest success rate at 71.6%, while the All mixture has the highest task coverage at 80.5%.Among single-model mixtures, Claude Opus 4.5 reaches 69.9% success rate, followed by Gemini 3 at 67.7%.Source: §4.7 · pp. 7–8, Table 5 · p. 7

5. Limitations

BJudge assumes independent rollouts from the same initial state. Live desktops and shared online resources can compromise that independence through cross-run interference.

5. Limitations

  • The independence assumption is harder to maintain on live desktops or tasks sharing online state, where concurrent rollouts may interfere.Virtualized environments with snapshotting and duplication can support parallel rollouts with limited additional wall-clock latency.Source: §5 · p. 8

6. Conclusion

The conclusion presents wide scaling and Behavior Judge as a way to improve CUA robustness and task success, reporting a new OSWorld state of the art and generalization to other benchmarks.

6. Conclusion

  • 72.6% success at 100 steps on OSWorld establishes a new state of the art, surpassing prior work by +9.2% and human-level performance at 72.36%.The authors also report extensive ablations validating their design choices.Source: §6 · p. 8
  • BJudge enables wide scaling by generating multiple trajectories in parallel and selecting among them to improve CUA robustness and task success.It transforms dense trajectories into compact behavior narratives for principled selection.Source: §6 · p. 8
  • The authors report strong generalizability on WindowsAgentArena and AndroidWorld.The conclusion describes BJudge as a scalable approach to improving real-world CUAs.Source: §6 · p. 8

Impact Statement

The impact statement says BJudge may improve complex desktop-workflow automation, while warning that errors can have larger effects at scale without safeguards; it also describes the authors’ use of ChatGPT for placeholders.

Impact Statement

  • Errors or unintended behaviors in real computer environments may have larger effects when the system is used at scale without proper safeguards.The statement also says BJudge can improve complex desktop-workflow automation, reduce human effort, and increase task reliability.Source: Impact Statement · p. 8

A. Use of LLMs

  • The authors used ChatGPT to generate structured-sentence and matplotlib-figure placeholders, then paraphrased the text and manually filled in experiment results.Source: A. Use of LLMs · p. 12

B. Summary of costs, time, and total experiment time

The appendix reports the time and setup for Agent S3 with BJudge and discusses cost-conscious efficiency, including ensemble diversity and balancing model cost against performance.

B. Summary of costs, time, and total experiment time

  • The appendix reports average and median per-task cost and time for each module, including medians because API delays create right-skew.Source: Table 8 · p. 12
  • Ten rollouts over OSWorld’s 361 tasks ran in parallel on four c4.8xlarge hosts for 15 hours and 54 minutes.A c4.8xlarge host supported 40 parallel OSWorld-spawned instances.Source: B. Summary of costs, time, and total… · p. 12
  • Narrative generation took about 1 hour and 19 minutes, and comparative judging took about 20 minutes across the 361 tasks.The authors ran both steps after generating all behavior narratives to isolate and monitor each module.Source: B. Summary of costs, time, and total… · p. 12
  • 17 hours and 33 minutes were required to complete Agent S3 with BJudge using N=10.Source: B. Summary of costs, time, and total… · p. 12

C. Efficiency Considerations

C.1. Ensembling Cheap and Expensive Models

  • The ensemble study reports that greater model diversity boosts performance and that mixing cheaper and stronger models can balance cost and performance.Table 9 presents results for ensembles with varying model capacities.Source: §C.1 · p. 12, Table 9 · p. 12

C.2. Cheap Rollouts and Expensive BJudge

The study examines open-source and mixed-model configurations for BJudge, reporting OSWorld results for Qwen3-VL-30B-A3B-Thinking and GPT-5 combinations.

C.2. Cheap Rollouts and Expensive BJudge

  • Ten OSWorld runs using Qwen3-VL-30B-A3B-Thinking achieved an average success rate of 33.3%.Source: §C.2 · p. 13
  • The study was motivated in part by the finding that BJudge modules cost about five times less than trajectory rollouts.Source: §C.2 · p. 13
  • +7.6% improvement with Qwen3-VL-30B-A3B-Thinking for both modules and 18.2% improvement with GPT-5 for both are reported for model combinations.Table 10 covers combinations for behavior narrative generation and comparative behavior evaluation.Source: Table 10 · p. 13, §C.2 · p. 13

D. Agentic Framework Improvements

The appendix specifies coding-agent execution and hand-off details, alongside the operational constraint of flat, online planning. It also reports that MCQ comparison performs similarly to iterative pairwise comparison while being faster and more token-efficient.

D. Agentic Framework Improvements

  • A coding call launches a bounded inner loop that uses the task instruction, current screenshot, and prior execution feedback to generate code or return DONE/FAIL.The loop ends on DONE/FAIL or when its step budget is exhausted.Source: D. Agentic Framework Improvements · p. 13
  • After code execution, a summarizer reports the session’s intent, logic, and observed effects, and gives a verifiable inspection checklist for the GUI worker.The worker receives the post-execution observation and execution history to verify effects before resuming step-by-step planning.Source: D. Agentic Framework Improvements · p. 13
  • Flat planning uses a step-level policy that replans from the current observation and compact history rather than committing to a subgoal list.The paper reports empirical effects on success and efficiency in Table 2, but those results are not supplied here.Source: D. Agentic Framework Improvements · p. 13

E. Iterative vs. MCQ-style Comparison

F. Citing vs. Not Citing Behavior Narratives

The paper tests whether requiring the judge to cite behavior narratives changes comparative selection results. It reports marginal performance improvements of about 1% in the tested GPT-5 and GPT-5 Mini settings.

F. Citing vs. Not Citing Behavior Narratives

  • Table 13 compares BJudge with and without citing behavior narratives using N=10 rollouts.Source: Table 13 · p. 14
  • About 1% marginal performance improvements were observed when the judge cited behavior narratives in the tested GPT-5 and GPT-5 Mini settings.The comparison used GPT-5 as the BJudge judge.Source: F. Citing vs. Not Citing Behavior… · p. 14
  • The citation ablation compared citing with not citing narratives for N=10 GPT-5 rollouts and N=10 GPT-5 Mini rollouts.The reported result describes the improvements as marginal.Source: F. Citing vs. Not Citing Behavior… · p. 14

G. Case Studies

The case studies examine how behavior narratives can mislead, including misread negative values and overlooked GUI–code-agent interactions. They also document prompt mechanisms for screenshot-grounded trajectory comparison, GUI verification, visual-change summaries, and progress reflection using constrained response formats.

G. Case Studies

  • A VLM misread the image’s −17.0 value as 17.0, producing an inaccurate narrative that the action changed vibrancy to 17.0.Source: Figure 6 · p. 15
  • Run A was incorrectly selected over Run B because BJudge favored reasonable-sounding narratives, despite Run B verifying the code changes and completing the task.The example highlights the importance of interaction between the GUI and code agent.Source: Figure 7 · p. 15

H. System Prompts

  • The comparative evaluator prompt requires checking each trajectory against applicable guidelines, comparing how well each meets them, and justifying its choice.It also requires inspecting the chronological screenshot sequence rather than assuming what changed.Source: H. System Prompts · p. 16
  • The GUI policy directs the agent to choose GUI or code agents by task, then verify code-agent file changes through GUI actions before declaring completion.If verification fails, the agent should use GUI actions to finish; modified applications may need to be closed and reopened.Source: H. System Prompts · p. 17
  • The code-agent summarizer is instructed to report factual, neutral summaries and give the GUI agent specific verification guidance for modified files.The code-agent prompt further calls for incremental steps, inspecting files before and after changes, and preserving document structure and formatting.Source: H. System Prompts · pp. 18–19
  • The agent prompt constrains outputs to a single Python block, Bash block, DONE, or FAIL, and requires progress within the step budget.The reflection prompt separately requires tagged thoughts and answer sections, with the answer listing relevant changes.Source: H. System Prompts · p. 20
  • Action-change summaries should describe screenshot evidence precisely, using visual markers and zoomed views while avoiding assumptions that an action succeeded.The prompt says to focus on action-induced changes, account for multiple interactions, and use the visual marker rather than coordinates to locate the action.Source: H. System Prompts · p. 20
  • The reflection prompt classifies trajectories as off-plan, on-plan, or complete, while explaining problems without prescribing a specific next action.Off-plan reflections should identify repeated actions without progress; on-plan reflections should briefly affirm continuation, and completed tasks should be acknowledged as complete.Source: H. System Prompts · p. 21
Loading 2510.02250v2…