PaperlayerView exact revision on arXiv

Source-linked AI summary

Agent World Model in the Wild: Computer-Use Agent Environment Synthesis

Zhaoyang Wang, Qianhui Wu, Haorui Wang, Simon Yu, Baolin Peng, Canwen Xu, Hao Cheng, Wenlin Yao, Boyi Liu, Yite Wang, Jianfeng Gao, Huaxiu Yao

arXiv:2610.26828v1cs.AIcs.CLcs.LG

TL;DR

Real-world computer-use agent training is costly and risky, and existing synthesis often focuses on individual interfaces. WildAWM builds open-source executable worlds and cross-interface tasks with execution-based quality checks. Training in these worlds improves two Qwen models across eight out-of-distribution benchmarks.

  • Problem

    Real-world agent training is costly and risky, and scalable synthesis must support connected computer environments with solvable, challenging tasks and reliable verifiers.

    Source: §1 · pp. 1–2

  • Method

    WildAWM synthesizes executable personal-computer worlds and cross-interface tasks with open-source models, using heterogeneous rollouts, judge checks, and counterfactual probes as quality gates.

    Source: §1 · p. 2, §3.2 · pp. 5–6

  • Results

    Training exclusively in WildAWM worlds improves both model scales across all eight out-of-distribution benchmarks, with average gains of +10.58 points for 35B-A3B and +8.21 points for 4B.

    Source: §4 · p. 8

  • Takeaways & Limitations

    Open-source models can synthesize high-quality environments and challenging tasks for general computer-use agent training.

    Source: §1 · p. 2

Abstract

from arXiv · show

Agent learning is shifting from learning from human data toward learning through interaction with the world. However, scaling such learning directly in real-world computer systems is challenging and can introduce substantial security risks. We therefore propose Agent World Model in the Wild (WildAWM), a scalable pipeline for synthesizing controllable environments and tasks for training general computer-use agents (CUAs). Whereas existing approaches typically specialize in a single agent interaction interface, we construct executable environments that jointly support graphical user interfaces, model context protocol services, and command-line interfaces, allowing agents to flexibly combine visual interaction, tool use, and code execution. WildAWM is entirely powered by open-source models, demonstrating that high-quality computer-use environments can be generated without proprietary frontier models. Beyond environments, we address three recurring challenges in task synthesis: ensuring solvability, maintaining sufficient difficulty, and producing reliable verifiers, by introducing execution-based quality gates that combine heterogeneous agent rollouts, cross-judge consistency checks, and counterfactual probes of judgment robustness. The resulting pipeline synthesizes 341 long-horizon environment-task pairs that can require over 100 agent steps to solve and remain challenging even for frontier models, with GPT-6-Astra solving only 60% of them. We train two Qwen models of different sizes exclusively in these synthetic worlds. Both models achieve substantial gains across 8 out-of-distribution benchmarks spanning web, claw, and computer-use agents, demonstrating that learning in WildAWM's synthesized worlds generalizes broadly to unseen computer-use tasks. The code is publicly available at https://github.com/MSR-Orchard/WildAWM.

1 INTRODUCTION

Computer-use agents need scalable, controllable environments because real-world training is costly and risky, while manually built environments are expensive to scale. WildAWM synthesizes open-source, executable personal-computer worlds and cross-interface tasks, with execution-based checks for task quality.

1 INTRODUCTION

  • Real-world agent training is costly and risky, while manually built environments are expensive and difficult to scale.Real systems are hard to reset, change over time, and may be affected by unintended exploratory actions.Source: §1 · p. 1
  • Computer-use tasks span GUI, CLI, and MCP interactions, but existing synthesis often targets individual modes or primarily GUI interaction.Tasks may require agents to select and coordinate interfaces across applications, websites, and local files.Source: §1 · pp. 1–2
  • WildAWM combines GUI, MCP, and CLI in a unified personal-computer setting, with task scope, target modality, and synthesis source compared across methods.The comparison table identifies the dimensions used to contrast related methods.Source: Table 1 · p. 2, §1 · p. 2
  • A realistic synthesized environment must connect websites, applications, MCP services, and local files, while tasks need solvability, difficulty, and reliable verification.These checks should not assume that a frontier model is necessarily correct.Source: §1 · p. 2
  • WildAWM starts from user personas and daily activities, builds executable personal-computer worlds, and synthesizes cross-agent interface tasks using open-source models.Heterogeneous agent rollouts, cross-judge checks, and counterfactual probes test solvability, difficulty, and verifier robustness.Source: §1 · p. 2
  • 341 environment–task pairs require at least three interface types, may take over 100 agent steps, and are solved only about 60% of the time by GPT-6-Astra.Training exclusively in these synthetic worlds yields substantial gains for two Qwen models across eight out-of-distribution benchmarks spanning web, claw, and computer-use agents.Source: §1 · p. 2

2 RELATED WORK

Prior environment synthesis has targeted web, tool, terminal, and software-engineering settings, while CUA work includes reconstructed applications, existing software, and desktop environments. WildAWM combines joint GUI, MCP, and CLI access with task quality controls using open-source models.

Computer-Use Agents

  • Computer-use research increasingly includes CLI, coding, MCP, and harness-based tools alongside GUI interaction, motivating a view of agents that can use multiple modalities.WildAWM considers general agents that jointly access GUI, CLI, and MCP as needed for personal-computer tasks.Source: §2 · p. 3

Environment Synthesis

  • Prior executable-environment synthesis spans web interaction, tool use, terminal interaction, and software engineering, while CUA efforts range from reconstructed apps to existing software and desktop environments.WildAWM instead builds a unified personal-computer world with joint GUI, MCP, and CLI access, combining WebHarbor's visual web substrate with synthesized services, files, and application data.Source: §2 · p. 3

Task Synthesis

  • Task synthesis has moved from standalone instructions to environment-grounded tasks, with prior methods drawing on interactions, exploration, programmatic generation, or executable verification.WildAWM adds heterogeneous rollouts, cross-judge consistency, and counterfactual probes to validate task solvability and difficulty and stress-test verifiers.Source: §2 · p. 3

3 AGENT WORLD MODEL IN THE WILD

WildAWM synthesizes executable personal-computer worlds and cross-interface tasks, then validates the task, environment, and verifier triplets for CUA training. Its process combines persona-driven world construction, agent rollouts and quality gates, and training with validated verifiers.

3.1 WORLD SYNTHESIS

  • A coding agent builds each persona’s multi-day world incrementally, extending prior state with applications, services, user histories, and files.The accumulated state W(c, d) initializes and resets the environment.Source: §3.1 · pp. 3–4, Figure 1 · p. 4

3.2 XAI TASK SYNTHESIS

  • Tasks are grounded in live environment data, require GUI, MCP, and CLI, and receive a specification, verifier, rubric, hint, and ordered sub-tasks.The verifier checks a predefined checklist of binary criteria, and success requires all criteria to pass.Source: §3.2 · p. 5
  • Heterogeneous agent rollouts and three levels of judgment support five gates for solvability, difficulty, and verifier reliability; counterfactual probes test judge stability.A task must have a successful rollout using every declared interface; candidates failing a gate can receive up to two revision rounds and must pass the gates again.Source: §3.2 · pp. 5–6
  • 21.3% mean rollout success; every accepted task was solved by at least one agent, but none was solved by all.This indicates accepted tasks had execution evidence of solvability while remaining difficult for the rollout agents.Source: Table 4 · p. 7, §3.2 · p. 7
  • Tasks average 4.2 sub-tasks and 3.1 interface switches; 74.5% revisit an interface, and GLM-5.2 sometimes needs over 100 steps.Revisiting interfaces requires agents to preserve state across transitions.Source: §3.2 · p. 7

3.3 GENERAL CUA TRAINING

  • GRPO uses validated-verifier rewards with checklist-based partial credit; an adaptive rollout multiplier and reward-diverse groups address zero-variance rewards.When all rollouts have the same format-error reward, the advantage rule supplies a format-related learning signal.Source: §3.3 · pp. 7–8

4 EXPERIMENTS

The experiments test two Qwen model scales trained in WildAWM against frontier models and evaluate transfer across eight out-of-distribution benchmarks. Training exclusively in the synthesized worlds improves both model scales across all eight benchmarks.

Dataset & Benchmark

  • The study splits 341 synthesized tasks into 100 for SFT, 208 for RL, and 33 for in-domain evaluation, alongside eight OOD benchmarks across three agent-task categories.The OOD set includes WebArena, Online-Mind2Web, QwenClawBench, ClawEval, CUA-Gym, OSWorld-Verified, OSWorld 2.0, and Agents’ Last Exam.Source: §4 · p. 8

Model & Baselines

  • Training uses Qwen3.6-35B-A3B and Qwen3.5-4B, with frontier models and prior work included as comparison baselines.The pipeline collects 100 GLM-5.2 SFT trajectories, warm-starts each base model for three epochs, then runs up to 64 RL steps.Source: §4 · p. 8

Main Results

  • +10.58 and +8.21 points on average are the respective gains for Qwen3.6-35B-A3B and Qwen3.5-4B across all eight OOD benchmarks.The benchmarks span web, claw, and computer-use agents.Source: §4 · p. 8
  • The 35B-A3B model’s average score rises from 42.84 to 53.42, matching GLM-5.2 on WebArena at 67.26 and exceeding GPT-5.5 on O2W, 80.77 vs. 72.22.Source: §4 · p. 8

5 ANALYSIS

WildAWM’s analysis examines environment diversity and task quality, the contribution of quality gates and RL components, and the resulting policy behavior. Its findings indicate broad task coverage, complementary quality controls, and gains in cross-interface and long-horizon behavior.

5.1 SYNTHESIS QUALITY

  • WildAWM’s environments have a 52.4K-token median state size and higher diversity than CUA-Gym, with higher Vendi scores and nearest-neighbor cosine distance of 0.25 vs. 0.20.Source: §5.1 · pp. 8–9, Figure 3 · p. 9
  • Across 24 heterogeneous agents, every task is solved by at least one; the three open-source synthesis models solve over 95%, while GPT-6-Astra achieves only about 60% pass@1.On 100 randomly sampled rollouts, the verifier achieves 95% agreement with the majority. After eight attempts, GPT-6-Astra still fails on 27.3% of tasks; with hints, every task is solved by at least one of the three open-source models.Source: §5.1 · p. 9
  • The pipeline reaches 134 domains and 91 task types at a 53% acceptance rate without clear saturation, while mean pairwise cosine distance remains stable as coverage grows.Source: §5.1 · p. 10, Figure 4 · p. 10

5.2 ABLATIONS ON QUALITY GATES

  • Removing any quality gate lowers average performance by 2.60–9.30 points; removing Judge Consistency or Initial Unsatisfaction drops validation from 21.59 to 3.79 or 3.05.Removing Task Difficulty decreases the validation score to 7.58.Source: §5.2 · p. 10, Table 7 · p. 10

5.3 ABLATIONS ON RL

  • Removing format penalty significantly lowers WA and OSV performance, checklist partial credit improves most benchmarks, and removing warm start causes the largest degradation.Without format penalty, CLE pass rate rises but its mean score is 69.89 versus 69.45 for WildAWM.Source: §5.3 · p. 10, Table 8 · p. 11

5.4 POLICY BEHAVIOR ANALYSIS

  • On the second and third OOD difficulty quartiles, WildAWM gains 19.9 and 12.2 points, while the hardest quartile shows no clear gain for any variant.Source: §5.4 · p. 11, Figure 5 · p. 11
  • Interface-switch rates rise from 18.4% to 24.7% after SFT and 25.5% after RL, while WildAWM solves 53.7% within 120 steps versus 47.2% for +RL.Between 40 and 120 steps, WildAWM gains 17.0 points versus 9.7 for SFT; RL adds long-horizon capability most effectively on top of SFT.Source: §5.4 · p. 11, Figure 5 · p. 11

6 CONCLUSION

WildAWM synthesizes executable personal-computer worlds and cross-interface tasks, then uses execution-based gates to assess solvability, difficulty, and verifier reliability. Two Qwen models trained only in these worlds improve across eight out-of-distribution benchmarks.

6 CONCLUSION

  • WildAWM builds personal-computer environments spanning GUI, MCP, and CLI, and grounds cross-interface tasks in accumulated world states.Its execution-based quality gates address task solvability, difficulty, and verifier reliability; synthesis and validation use open-source models.Source: §6 · p. 11
  • The pipeline synthesizes 341 long-horizon tasks and trains two Qwen models exclusively in the synthetic worlds.The resulting agents improve consistently across all eight out-of-distribution benchmarks spanning web, claw, and computer-use agents.Source: §6 · p. 11
  • WildAWM offers a path toward open-source models generating scalable synthetic worlds and learning within them.The conclusion presents this as a promising path toward general computer-use agents.Source: §6 · p. 11

A.1 EVALUATION BENCHMARKS

The evaluation suite covers eight out-of-distribution benchmarks and a 33-task in-domain sample, with benchmark-specific task sets, scoring rules, and pass criteria. Several external benchmarks use subsets, and some scoring procedures account for partial credit or repeated rollouts.

A.1 EVALUATION BENCHMARKS

  • The suite covers eight out-of-distribution benchmarks, with task counts, scoring metrics, and pass criteria summarized for each.Evaluation uses selected subsets of several benchmarks for efficiency; NVFP4 quantization is applied to open-source models, GPT-5.5 judges tasks requiring an LLM judge, and infrastructure errors receive up to three retries.Source: §A.1 · p. 17, Table 9 · p. 18
  • Online-Mind2Web uses 300 live-website tasks and WebArena uses 226 tasks across five self-hosted sites, both reporting pass rates.Online-Mind2Web excludes sites blocked by reCAPTCHA or other access restrictions; WebArena uses its original programmatic evaluator.Source: §A.1 · p. 17, Table 9 · p. 18
  • QwenClawBench reports mean score to capture partial credit, while ClawEval reports the pass rate at a score threshold of 0.75.QwenClawBench uses its full task set; ClawEval's pass rate is used for comparison with OpenForgeRL.Source: §A.1 · p. 17
  • CUA-Gym evaluation randomly samples 200 tasks to test GUI capabilities from a dataset spanning desktop, browser, and OS environments.Its programmatic reward returns a continuous score in [0, 1].Source: §A.1 · pp. 17–18
  • OSWorld-Verified uses a stratified 30% sample of 99 tasks, while OSWorld 2.0 uses its full set and reports credit scores for longer, more complex workflows.OSWorld-Verified reports the original evaluator's mean score; OSWorld 2.0 reports credit rather than pass rate.Source: §A.1 · p. 18
  • Agents’ Last Exam evaluates 99 Linux-compatible tasks, and WildAWM-val samples 33 synthesized tasks with eight rollouts per task.Agents’ Last Exam uses its official continuous grader; WildAWM-val reports an unbiased pass-rate estimate and a stricter all-rollouts-succeed metric.Source: §A.1 · p. 18

A.2 EVALUATION PROTOCOL

The evaluation protocol equips agents to choose among available interfaces and supplies benchmark-matched tools. World synthesis builds persona-specific, multi-day states through sequential planning and deterministic implementation, then checks each materialized day before acceptance.

A.2 EVALUATION PROTOCOL

  • The OpenCode agent runtime lets agents choose native, MCP, browser, shell, and file tools according to each operation rather than a fixed interface priority.The default maximum is 256 turns, increased to 512 for OS-2 and ALE.Source: §A.2 · pp. 18–19
  • GUI access combines browser automation and, for desktop benchmarks, fine-grained desktop controls; a vision tool converts images into text descriptions for policy models.The protocol includes Playwright, Browser-use CLI, desktop actions, and guest execution tools.Source: §A.2 · p. 19

A.3 WORLD SYNTHESIS DETAILS

  • World synthesis uses GLM-5.2 with OpenCode and seeds day-zero worlds with 15 WebHarbor websites.The pipeline proceeds through coding-agent stages governed by constitutions that constrain deliverables and quality criteria.Source: §A.3 · p. 19
  • Persona generation grounds realistic profiles in existing application catalogs and MCP services, then applies a diversity gate against prior personas.Profiles include named contacts, credentials, and multi-day story arcs; rejected candidates may be revised twice.Source: §A.3 · p. 20
  • For each persona, sequential day plans use the cumulative prior world state and specify computer activities, service events, and web actions.A coverage histogram encourages variation in app and site usage across days.Source: §A.3 · p. 20
  • A day implementer materializes each plan as a deterministic, replayable delta over the prior day's state.Constraints include frozen timestamps, derived IDs, idempotent inserts, and fixed-salt password hashes.Source: §A.3 · p. 20
  • Materialized days pass checks for script validity, replay determinism, declared artifacts, and live service and account health.Replay is tested twice for identical content hashes, and an independent reviewer inspects snapshots against the day plan.Source: §A.3 · p. 20
  • The 341 accepted tasks span 36 personas and 640 world states; every persona contributes at least two tasks, including one hard task.Tasks per day track persona world-state coverage (Pearson r = 0.90); 262 tasks come from days 0–13, when at least 33 personas had a world state.Source: §A.3 · pp. 20–21, Table 10 · p. 21, Figure 6 · p. 22

A.4 XAI TASK SYNTHESIS DETAILS

XAI task synthesis moves from diverse cross-interface sketches to authored tasks, multi-agent evaluation, and quality gates. Counterfactual probes test verifier robustness, while failed candidates may be revised and must rerun affected gates.

A.4 XAI TASK SYNTHESIS DETAILS

  • Task synthesis proceeds through sketching, diversity selection, task authoring, multi-agent rollouts, three-level judgment, and five quality gates.Failed candidates may receive diagnostic feedback and undergo up to two fix iterations.Source: §A.4 · p. 22
  • Sketches must require CLI, MCP, and web interfaces, each indispensable, and use a script or command-line program whose result informs a later decision.The sketch agent explores a live world and proposes up to four diverse sketches within a wall-clock budget.Source: §A.4 · p. 22
  • A diversity judge selects sketches that add topical variety and rejects near-duplicates sharing core actions and artifacts..Source: §A.4 · p. 22
  • Task authoring uses a live capability contract and produces seven artifacts, including a reference completion and deterministic verifier scripts.A self-check loop validates the artifacts, while solver-visible task instructions are separated from internal rationale and metadata.Source: §A.4 · p. 22
  • The LLM judge uses rubric intent and verifier evidence to assign completed, failed, or world-imperfect verdicts; an agent judge audits snapshots and classifies false deterministic checks.The agent judge distinguishes brittle verifiers from genuine failures and uses evidence provenance rules.Source: §A.4 · pp. 22–23
  • A counterfactual probe compares a blind re-judge with the ensemble verdict, then tests whether a judge defends its decision with evidence under opposite-verdict pressure.Verifier edits under pressure fail the probe, while evidence that the judge was wrong is routed to review; affected gates are rerun after revisions.Source: §A.4 · p. 23

A.5 TRAINING DETAILS

Training uses supervised fine-tuning on successful teacher trajectories before reinforcement learning on a separate task split. Adaptive rollout selection and trajectory-level loss weighting support learning from tasks that provide useful reward variation.

Training stages

  • SFT uses at most one successful GLM-5.2 trajectory per task across 100 tasks, then RL starts from that checkpoint on 208 tasks.SFT segments trajectories at compaction boundaries, removes segments longer than 102,400 tokens, and trains only on assistant tokens.Source: §A.5 · p. 23

Adaptive rollouts

  • Adaptive sampling expands rollout pools until rewards vary or tasks are mastered, then selects mixed-reward rollouts and updates each task’s rollout multiplier.The multiplier is bounded between 1 and 3; additional rollouts arrive in chunks of 8.Source: §A.5 · pp. 23–25

Compaction-aware training

  • A nine-step rollout with compaction at step 5 yields three segment samples, rather than nine independent samples under per-step training.The segment samples share the trajectory advantage and trajectory-level loss normalization.Source: Figure 7 · p. 26

Policy loss

  • The policy loss weights rollouts equally within groups and groups equally within each step, using trajectory-level normalization by trainable-token count.The objective uses asymmetric clipping ranges of 0.2 and 0.28, entropy coefficient 0.001, and no KL penalty; format failures train only on the failing turn.Source: §A.5 · p. 26

A.6 SEGMENT-WISE TRAINING ACROSS COMPACTION

Automatic context compaction enables long rollouts within a fixed context budget, but breaks a trajectory into separate causal sequences. Segment-wise training groups steps that share a causal prefix.

Segment definition

  • Compaction partitions a rollout into maximal contiguous segments whose contexts remain a causal chain, with each segment sharing a base prefix.Within a segment, the history interleaves earlier policy outputs and environment responses.Source: §A.6 · p. 27

Training formulation

  • Segment-wise training reduces forward passes from Tj to Mj, preserves within-segment dependencies, and balances total gradient weight across trajectories with different compaction frequency.The passage states that Mj is much smaller than Tj in practice.Source: §A.6 · p. 27

B POLICY BEHAVIOR STUDY: DETAILS

The behavioral analysis attributes WildAWM’s main benefit to out-of-distribution transfer rather than in-domain fit. It also reports increased cross-interface use and identifies the hardest tasks as a remaining boundary.

Cross-interface interaction

  • After RL, the model uses 2.68 interfaces per episode and GUI actions account for 1.6% of tool calls, compared with 2.40 and 16.3% for the base model.On WildAWM-val, the switch rate is 35.7% for WildAWM, versus 26.1% for the base model and 25.8% for RFT.Source: B POLICY BEHAVIOR STUDY: DETAILS · p. 27

Transfer versus in-domain fit

  • RL raises net OOD gains over the base model from +93 tasks after SFT to +129, while WildAWM-val pass@1 is 36.0% after RL versus 37.3% after SFT.A turn with at least 8K output tokens occurs in 9.1% of RL episodes, compared with 24.3% after SFT and 7.4% for GLM-5.2.Source: B POLICY BEHAVIOR STUDY: DETAILS · p. 28

Step-budget gains

  • On CUG, OSV, and WA, RL adds 72 net tasks over the base model versus 30 for SFT, and WildAWM gains more than SFT between 40 and 120 steps on each benchmark.The respective WildAWM versus SFT gains are 18.0 vs. 7.5 points on CUG, 16.2 vs. 8.1 on OSV, and 16.4 vs. 12.4 on WA.Source: B POLICY BEHAVIOR STUDY: DETAILS · p. 28

Remaining boundary

  • The hardest quartile, consisting mostly of OS-2 and ALE tasks, remains difficult: the base model and four variants solve 4–7 tasks, while GPT-6-Astra solves 18.The authors identify investigating whether larger models can better exploit the synthesized data as future work, given sufficient compute.Source: B POLICY BEHAVIOR STUDY: DETAILS · p. 28

C CASE STUDY

The case study follows a synthesized persona and task through world construction, generator-skill development, a multi-interface agent trajectory, and verifier checks. It illustrates how a task can require actions and information from several computer interfaces.

Persona and story

  • The example world follows a freelance audio-visual technician across 17 world states, with ongoing storylines and applications added over several days.The persona includes six named contacts, websites and MCP services, and three story arcs; four tasks are accepted on days 5, 7, and 9.Source: C CASE STUDY · p. 28, Table 15 · p. 29

Synthesis skills

  • World synthesis uses a shared library of 83 event-creation skills, while evolving task-synthesis skills record gate outcomes and revise guidance across task-generation events.The task-synthesis process made 802 revisions of 40 final skills; notes are limited to 2,000 characters and 50 files.Source: C CASE STUDY · p. 28, Table 16 · p. 30, Table 18 · p. 32

Agent trajectory

  • The 65-call case-study trajectory resolves a recipient through direct messages, extracts an order from a local invoice, checks product images in the browser, then updates the cart, message, and shortlist file.The task does not name an interface; the agent uses information from the browser to complete later steps.Source: C CASE STUDY · pp. 29–30, Table 20 · p. 34

Verifier

  • The task verifier uses three scripts, one per requested outcome, and succeeds only when all checks pass against changes from the initial world state.The checks use natural keys; the example rejects a rollout that adds the wrong speaker to the cart.Source: C CASE STUDY · p. 30, Table 21 · p. 35

Related papers

Loading 2610.26828v1…