PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Agent Training for Computer Use

Yanheng He, Jiahe Jin, Pengfei Liu

arXiv:2505.13909v2cs.AIcs.CLcs.LG

TL;DR

High-quality computer-use trajectories are scarce, limiting the development of human-like agents. PC Agent-E augments a small set of human demonstrations with alternative actions synthesized by Claude 3.7 Sonnet. With 312 augmented trajectories, it improves over its base model and surpasses the teacher on WindowsAgentArena-V2.

  • Problem

    High-quality computer-use trajectory data is scarce, and open-source models remain uncompetitive with leading proprietary systems while current models fall short of human performance.

    Source: §1 · p. 1

  • Method

    PC Agent-E combines human computer-use trajectories with diverse alternative action decisions synthesized by a frontier model.

    Source: §1 · p. 2

  • Results

    With 312 Claude-augmented trajectories, PC Agent-E achieves a 141% relative improvement over Qwen2.5-VL-72B and a 10% relative improvement over Claude 3.7 Sonnet on WindowsAgentArena-V2.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings suggest that complex computer-use capabilities can be elicited from a remarkably small set of high-quality trajectories.

    Source: §6 · p. 10

    Current infeasible-task evaluations do not accurately reflect computer-use agent capabilities.

    Source: §5.6 · p. 10

Abstract

from arXiv · show

Scaling up high-quality trajectory data has long been a critical bottleneck for developing human-like computer use agents. We introduce PC Agent-E, an efficient agent training framework that significantly reduces reliance on large-scale human demonstrations. Starting with just 312 human-annotated computer use trajectories, we further augment them by synthesizing diverse alternative action decisions with Claude 3.7 Sonnet. Trained on these enriched trajectories, our PC Agent-E model achieved a remarkable 141 relative improvement, and even surpassed the Claude 3.7 Sonnet by 10% in relative terms on WindowsAgentArena-V2, an improved benchmark we also released. By integrating robust human computer use skills with automated AI data synthesis capabilities, our method not only brought substantial improvements over training on human trajectories alone, but also significantly surpassed direct distillation from Claude 3.7 Sonnet. Code, data and models are available at https://github.com/GAIR-NLP/PC-Agent-E

1 INTRODUCTION

PC Agent-E addresses the scarcity of high-quality computer-use trajectories by augmenting a small set of human demonstrations with AI-generated action alternatives. With 312 augmented trajectories, it outperforms its base model and Claude 3.7 Sonnet on WindowsAgentArena-V2.

1 INTRODUCTION

  • High-quality computer-use trajectory data is scarce, and open-source models remain uncompetitive with leading proprietary systems while models broadly fall short of human performance.The paper identifies data scarcity as a key factor contributing to these deficiencies.Source: §1 · p. 1
  • PC Agent-E combines real-world human computer-use trajectories with diverse action decisions generated by a frontier model.The framework reconstructs implicit thoughts behind human actions before using trajectory snapshots to sample alternative decisions with Claude 3.7 Sonnet.Source: §1 · p. 2
  • The framework starts from 312 human trajectories collected by two annotators in one day, with task descriptions, screenshots, and keyboard or mouse actions.Human proficiency ensures successful task completion, removing the need for additional verification.Source: §1 · p. 2
  • 141% relative improvement over Qwen2.5-VL-72B, and 10% relative improvement over Claude 3.7 Sonnet on WindowsAgentArena-V2, using 312 Claude-augmented trajectories.The method also outperforms training on human trajectories alone and direct distillation from the teacher model.Source: §1 · p. 2
  • The work releases WindowsAgentArena-V2, an improved benchmark intended to support more robust and fair computer-use evaluations.The benchmark addresses evaluation dependence, infeasible hacking, and other limitations of the original WindowsAgentArena.Source: §1 · p. 2

2 RELATED WORK

Related work covers computer-use agent architectures and data-synthesis approaches. This work focuses on native agent models trained with supervised fine-tuning and synthesizes data-efficient multi-step trajectories from real-world human demonstrations.

2.1 COMPUTER USE AGENT

  • Native agent models act step by step from their history and current state, while modular workflows use specialized modules and multi-agent collaboration.The paper studies efficient supervised fine-tuning for native agent models, which the passage describes as flexible and generalizable.Source: §2.1 · p. 3

2.2 DATA SYNTHESIS

  • General data synthesis includes distillation, where state-of-the-art models generate training data for weaker models, and self-improvement, where a model refines its own data.These are presented as common approaches as large language models grow more powerful.Source: §2.2 · p. 3
  • Prior computer-use data synthesis spans foundational GUI datasets, single-step visual grounding, and multi-step trajectories, while this work synthesizes data-efficient trajectories from real-world human demonstrations.Prior multi-step approaches use web tutorials or agents’ own exploration records.Source: §2.2 · p. 3

3 METHOD

PC Agent-E combines a small set of human computer-use trajectories with reconstructed thoughts and AI-synthesized alternative actions. The resulting augmented action nodes become training samples for a simple end-to-end agent scaffold.

3.1 OVERVIEW

  • The framework combines authentic human-computer interactions with diverse synthesized decisions to create trajectory data with realism and diversity.The overview presents four stages: collect and filter human trajectories, reconstruct thoughts, synthesize alternatives with Claude 3.7 Sonnet, and train PC Agent-E.Source: §3.1 · pp. 3–4

3.2 TRAJECTORY COLLECTION

  • 312 real-world human trajectories were collected across applications, with two annotators completing the process in one day.Annotators averaged roughly three minutes per trajectory; they could discard or revise unsatisfactory trajectories, and rule-based filters removed errors.Source: §3.2 · p. 4, Figure 3 · p. 4

3.3 THOUGHT COMPLETION

  • Claude 3.7 Sonnet reconstructs an implicit thought process for each recorded human action using the task, prior actions and thoughts, current action, and screenshot.This converts raw human trajectories into trajectories with explicit thoughts at each step.Source: §3.3 · p. 4, Figure 4 · p. 5

3.4 TRAJECTORY BOOST

  • Trajectory Boost augments human trajectories with diverse alternative action decisions because computer-use tasks allow multiple valid pathways.Claude 3.7 Sonnet generates single-step alternatives supported by rational thought processes; sampled alternatives are not executed in real environments.Source: §3.4 · p. 5

3.5 AGENT TRAINING

  • PC Agent-E uses a simple end-to-end scaffold that maps a screenshot, task description, and textual action history to a ReAct-style thought and action.Past screenshots are excluded from history; each training sample uses prior human actions on the trajectory’s main trunk, and 312 augmented trajectories yield 27K samples.Source: Figure 5 · p. 5, §3.5 · p. 6

4 WINDOWSAGENTARENA-V2

WindowsAgentArena-V2 updates the original benchmark to improve evaluation reliability and fairness. Its changes address task-state dependence, infeasible-task scoring, unstable VM initialization, and evaluation errors.

4 WINDOWSAGENTARENA-V2

  • WindowsAgentArena-V2 comprises 141 tasks across 11 Windows applications, derived from the original benchmark with evaluation improvements.The original benchmark evaluates computer use in realistic Windows environments through diverse tasks and hand-written evaluation rules.Source: §4 · p. 6
  • VM snapshots are restored before each evaluation to prevent inter-task interference and ensure consistent starting states.The benchmark also installs essential software missing from the original VM snapshot.Source: §4 · p. 6
  • WindowsAgentArena-V2 removes infeasible tasks because agents could otherwise earn success by outputting FAIL, undermining benchmark fairness.The authors describe removal as a temporary solution and say they currently prioritize performance on feasible tasks.Source: §4 · pp. 6–7
  • A rule-based and LLM-based validation framework with up to three restart attempts reduced VM initialization failures from 10%–30% to below 5%.The failure range depended on hardware.Source: §4 · p. 7
  • The authors corrected evaluation-function bugs and used human evaluators for a few complex tasks to improve assessment reliability.One corrected error awarded full scores for deleting all browsing history when asked to clear YouTube history.Source: §4 · p. 7

5 EXPERIMENT

The section introduces experiments intended to evaluate PC Agent-E and validate Trajectory Boost.

5 EXPERIMENT

  • The experiments are designed to evaluate PC Agent-E and validate Trajectory Boost.The passage characterizes the experiments as extensive.Source: §5 · p. 7
  • One stated purpose of the experiments is to assess PC Agent-E.Source: §5 · p. 7
  • The experiments also aim to validate the Trajectory Boost method.Source: §5 · p. 7

1. How does PC Agent-E perform against SOTA methods on computer use tasks? (§5.2)

The section’s stated question concerns whether Trajectory Boost’s data scaling surpasses using human demonstrations alone.

1. How does PC Agent-E perform against SOTA methods on computer use tasks? (§5.2)

  • The section asks how Trajectory Boost’s data scaling compares with using human demonstrations alone.Source: §1 · p. 7

5. How well does PC Agent-E generalize to unseen environments? (§5.6)

PC Agent-E improves WindowsAgentArena-V2 performance over its base model and teacher, while its gains scale with synthesized data and extend to Linux evaluation. The experiments also compare Trajectory Boost with direct distillation and test performance under longer interaction limits.

5.1 SETUP

  • The main evaluation uses WindowsAgentArena-V2; OSWorld is used to test generalization to Linux, and the reported model comparisons include proprietary and open-source baselines.The setup also compares Trajectory Boost with behavior cloning on 312 human trajectories and direct distillation using 3,120 Claude trajectories.Source: §5.1 · p. 7

5.2 MAIN RESULTS

  • 141% relative improvement over Qwen2.5-VL-72B, and 10% relative improvement over Claude 3.7 Sonnet (36.0 vs. 32.6), on WindowsAgentArena-V2.PC Agent-E is reported as the SOTA open-source model for Windows computer use and performs comparably to Claude 3.7 Sonnet with extended thinking.Source: Table 3 · p. 8, §5.2 · p. 8

5.3 DATA SCALING OVER HUMAN DEMONSTRATIONS

  • With 9 synthesized actions and 1 human action per step (s = 10), PC Agent-E reaches 36.0, versus 17.2 from human trajectories alone and 14.9 for the base model.The results are described as scaling with the data factor; the authors attribute the gains primarily to diverse synthesized decisions with thought processes.Source: §5.3 · pp. 8–9

5.4 TRAJECTORY BOOST VS. DIRECT DISTILLATION

  • Trajectory Boost significantly outperforms direct distillation at most training-data scales.The authors attribute this to higher-quality synthesized data grounded in human trajectories and to avoiding error accumulation in end-to-end distillation.Source: Figure 7 · p. 9, §5.4 · p. 9
  • 3 hours versus about 900 hours to collect 3,120 trajectories: Trajectory Boost is reported as a 300-fold speedup under the same hardware conditions.Trajectory Boost uses offline synthesis without real-environment interaction, allowing natural parallelization.Source: §5.4 · p. 9

5.5 TEST TIME SCALING

  • As the step limit rises from 15 to 30, PC Agent-E uses the additional compute effectively and its performance gap over the base model widens.At a 50-step limit, the authors report a slight performance decline.Source: Figure 8 · p. 9, §5.5 · p. 9

5.6 CROSS-PLATFORM EVALUATION

  • 34% relative improvement on Linux OSWorld despite training exclusively on Windows data.The authors report this as evidence of cross-platform generalization.Source: §5.6 · p. 9
  • The weaker Qwen2.5-VL-72B model scores markedly better on infeasible tasks, which the authors say current infeasible-task evaluations do not accurately reflect.They suggest checking agents’ rationale when they declare tasks impossible as one possible criterion.Source: §5.6 · p. 10

6 CONCLUSION

The conclusion presents PC Agent-E as an efficient training framework that augments human demonstrations with frontier-model action alternatives. It reports gains over human-only training and direct distillation, alongside an evaluation on the original WindowsAgentArena benchmark.

6 CONCLUSION

  • PC Agent-E combines human demonstrations with diverse frontier-model action alternatives; the conclusion reports a 141% relative improvement over the base model and better performance than Claude 3.7 Sonnet.The authors state that ablations show advantages over human-only training and direct distillation, and conclude that computer-use capabilities can be elicited from a small set of high-quality trajectories.Source: §6 · p. 10

A TRAINING DETAILS

  • 27k samples, 2 epochs, and approximately 5 hours on 32 NVIDIA GPUs were used to fine-tune Qwen2.5-VL-72B.The visual tower was kept frozen; training used an 8,192-token context length, batch size 128, and learning rate 2e-6.Source: A TRAINING DETAILS · p. 13

B EVALUATION ON ORIGINAL WINDOWSAGENTARENA BENCHMARK

C EXPERIMENT ON SMALLER MODEL

The paper reports that Trajectory Boost improves the smaller Qwen2.5-VL-7B model, though less than the 72B model, and compares PC Agent-E with a human-data-plus-distillation baseline. The authors attribute the smaller model’s more limited gains to knowledge and grounding deficiencies.

C EXPERIMENT ON SMALLER MODEL

  • Knowledge and grounding deficiencies limit the smaller model’s overall gains, while the method mainly enhances planning ability.This is the explanation given for the smaller improvement relative to the 72B model.Source: C EXPERIMENT ON SMALLER MODEL · p. 13
  • Trajectory Boost improves Qwen2.5-VL-7B, but its gains are smaller than those for the 72B model.The authors say the smaller model’s knowledge and grounding deficiencies limit its overall performance gain.Source: Table 6 · p. 13, C EXPERIMENT ON SMALLER MODEL · p. 13

D HUMAN DATA + DIRECT DISTILLATION BASELINE

E EXTENDED TEST-TIME SCALING: 50-STEP EVALUATION

At 50 steps, both models’ performance drops, which the analysis attributes chiefly to weak task-completion recognition. Results support scaling from 15 to 30 steps, but not substantially beyond the training horizon.

E EXTENDED TEST-TIME SCALING: 50-STEP EVALUATION

Related papers

Loading 2505.13909v2…