PaperlayerView exact revision on arXiv

Source-linked AI summary

WebWorld: A Large-Scale World Model for Web Agent Training

Zikai Xiao, Jianhong Tu, Chuhang Zou, Yuxin Zuo, Zhi Li, Peng Wang, Bowen Yu, Fei Huang, Junyang Lin, Zuozhu Liu

arXiv:2602.14721v1cs.AI

TL;DR

Web agents need large-scale interaction data, but collecting real-world trajectories is slow, rate-limited, and safety-sensitive. WebWorld addresses this gap with an open-web simulator trained on over one million trajectories and evaluated for simulation and agent-training utility. It achieves performance comparable to leading models, improves downstream web-agent benchmarks, and generalizes beyond web simulation, while showing sycophancy and limited high-quality content generation.

  • Problem

    Real-world web-agent training requires massive trajectories, but network latency, rate limits, access restrictions, and safety considerations constrain collection.

  • Method

    WebWorld trains 8B, 14B, and 32B open-web simulators on 1M+ real-world trajectories using a scalable hierarchical pipeline, multiple formats, and synthesized reasoning data.

  • Results

    +9.9% on MiniWob++ and +10.9% on WebArena follow Qwen3-8B fine-tuning on WebWorld-synthesized trajectories, with the fine-tuned 14B model comparable to GPT-4o.

  • Takeaways & Limitations

    WebWorld supports open-web simulation, inference-time lookahead search, and cross-domain generalization to code, GUI, and game environments.

  • Takeaways & Limitations

    WebWorld can generate overly optimistic outcomes through sycophancy and struggles with high-quality, detailed content such as scientific articles.

Abstract

from arXiv · show

Web agents require massive trajectories to generalize, yet real-world training is constrained by network latency, rate limits, and safety risks. We introduce \textbf{WebWorld} series, the first open-web simulator trained at scale. While existing simulators are restricted to closed environments with thousands of trajectories, WebWorld leverages a scalable data pipeline to train on 1M+ open-web interactions, supporting reasoning, multi-format data, and long-horizon simulations of 30+ steps. For intrinsic evaluation, we introduce WebWorld-Bench with dual metrics spanning nine dimensions, where WebWorld achieves simulation performance comparable to Gemini-3-Pro. For extrinsic evaluation, Qwen3-14B trained on WebWorld-synthesized trajectories improves by +9.2\% on WebArena, reaching performance comparable to GPT-4o. WebWorld enables effective inference-time search, outperforming GPT-5 as a world model. Beyond web simulation, WebWorld exhibits cross-domain generalization to code, GUI, and game environments, providing a replicable recipe for world model construction.

1 Introduction

WebWorld addresses the difficulty of scaling reliable web-agent training by simulating open-web interactions with a large, diverse, reasoning-capable world model. It combines 1M+ real-world trajectories with broad evaluation and shows strong intrinsic and extrinsic utility.

  • Motivation and approach: Existing web world models have poor generalization because their data pipelines are difficult to scale and often rely on closed environments.WebWorld instead targets diverse real-world websites through scalable data collection.
  • Motivation and approach: WebWorld is an open-web world model series trained on 1M+ real-world trajectories, supporting reasoning, 30+ turns, and multiple input formats.The series includes 8B, 14B, and 32B models and uses 100× more trajectories than prior work.
  • Motivation and approach: The training pipeline combines rule-based crawling, autonomous exploration, task-oriented execution, and synthesized reasoning examples.The collected trajectories inject web knowledge, while CoT samples add causal reasoning patterns.
  • Evaluation and results: WebWorld-Bench evaluates factuality and perceptual realism across nine dimensions, including long-horizon simulation and multi-format robustness.WebWorld achieves performance on par with Claude-Opus-4.1 and Gemini-3-Pro across the benchmark’s metrics.
  • Evaluation and results: +9.9% on MiniWob++ and +10.9% on WebArena are achieved by Qwen3-8B fine-tuned on WebWorld-synthesized trajectories.The fine-tuned 14B model reaches performance comparable to GPT-4o; WebWorld also outperforms GPT-5 as a world model for lookahead search.

2 Related Work

Prior web world models use prompting or training on closed benchmark environments to synthesize agent trajectories. WebWorld shifts collection to the open web with a scalable hierarchical strategy aimed at broader generalization.

  • Prompted world models: Early approaches prompted proprietary LLMs as world models to synthesize trajectories for web-agent training.UI-Simulator used retrieval-augmented simulation, while Simia generated trajectories from tool specifications.
  • Trained world models: More recent approaches train world models on offline or agent-explored trajectories from closed environments such as WebArena and WebShop.Methods include experience replay, retrieval augmentation, synthetic tasks, and Monte Carlo Tree Search.
  • WebWorld’s distinction: WebWorld targets the open web rather than closed benchmark environments to improve generalization and capture diverse real-world dynamics.Its hierarchical collection strategy combines randomized exploration, autonomous discovery, and task synthesis.

3 Training WebWorld

WebWorld is trained as an autoregressive browser simulator on a hierarchically collected, filtered, and enriched trajectory corpus. The pipeline adds multiple representations and a small reasoning curriculum to support broad and long-horizon simulation.

  • Model and data pipeline: The simulator predicts the next browser state from an instruction and the history of states and actions, training a causal LLM by maximum likelihood.Trajectories contain an instruction followed by state-action sequences.
  • Model and data pipeline: A three-level collection pipeline combines randomized crawling, autonomous exploration, and task-oriented execution to balance scale, realism, and task relevance.The pipeline harvests websites aligned with pre-training corpora and collects 1.06M trajectories overall.
  • Data quality and enrichment: The data is filtered for website reachability, banned content, and quality, then augmented across A11y Tree, HTML, XML, Markdown, and natural-language formats.A11y Tree is the primary structured representation, while enrichment addresses versatility and catastrophic forgetting.
  • Reasoning curriculum: A two-stage curriculum first trains on full web dynamics and then adds synthesized CoT rationales that explain page structure, intent, and predicted state changes.This injects explicit reasoning patterns into the simulator after broad dynamics training.
  • Reasoning curriculum: 1,000 CoT samples outperform a base model trained on 10× more CoT data when added to robust pre-trained dynamics.The result supports the paper’s knowledge-then-reasoning-pattern injection recipe.
  • Dataset statistics: The dataset spans diverse domains, contexts up to 30k tokens, and trajectories reaching 30 turns.These statistics cover both short interactions and extended web tasks.

4 Benchmarking Web World Model

WebWorld-Bench evaluates web world models across nine dimensions using complementary factuality and perceptual-realism metrics. WebWorld-32B achieves performance comparable to leading proprietary models, with robust rankings across judges.

  • Existing structural and semantic metrics struggle with open-ended web tasks, motivating a broader evaluation framework.
  • WebWorld-Bench combines pointwise Factuality Score with pairwise Web Turing Score across nine evaluation dimensions.Factuality measures functional correctness of predicted state transitions, while Web Turing measures perceptual realism against real browser observations.
  • Table 3 reports paired Factuality and Web Turing scores for proprietary and open-source models, normalized to [0, 1].Higher values indicate better performance, and the best result in each metric is bolded.
  • 71.0% average Factuality Score for WebWorld-32B matches Claude-Opus-4.1 at 71.3%.WebWorld-32B also reaches 77.0% long-horizon consistency and 70–75% multi-format robustness.
  • Relative model rankings remain consistent across GPT-4o and Claude-Opus-4.1 judges despite differences in absolute scores.

5 Extrinsic Evaluation

Extrinsic evaluation tests whether WebWorld improves downstream agents and supports inference-time lookahead. Synthetic training data yields substantial benchmark gains, while WebWorld also outperforms GPT-5 as a world model in search.

  • 8,000 WebWorld-synthesized trajectories improve Qwen3-8B by +9.9% on MiniWob++ and +10.9% on WebArena.The Abstract-and-Instantiate pipeline abstracts concrete seed tasks, executes them in WebWorld, and reinstantiates the resulting trajectories.
  • WebWorld-guided inference-time lookahead search outperforms GPT-5 as a world model.The agent proposes candidate actions, WebWorld simulates next states, and a value model selects the highest-scoring action.
  • Natural-language outputs support deeper planning at k = 5, whereas full HTML is limited to k = 2 by context limits.
  • The bounded gains from inference-time search suggest greater value for synthesizing training data than for search itself.
  • Figure 4 indicates lower evaluation loss for larger models and predicts continued gains for a 72B model.

6 Analysis

Analysis shows that WebWorld benefits from scaling, carefully limited reasoning data, and adaptation beyond web environments. These results support both predictable growth and cross-domain transferability.

  • 6.1 Scaling Law of WebWorld: Larger WebWorld models consistently achieve lower evaluation loss, following a power-law relationship with compute.Predictions for 72B models suggest substantial further improvements with no signs of saturation.
  • 6.2 Ablation of Reasoning Activation: 1,000 CoT samples yield a Total Score of 0.561, exceeding direct reasoning tuning with 10× more data at 0.510.Excessive CoT data can degrade performance, favoring a small curated reasoning set alongside large-scale real-world training.
  • 6.2 Ablation of Reasoning Activation: Table 7 reports superior WebWorld-8B performance with only 1k reasoning samples.
  • 6.3 Cross-Environment Generalization: WebWorld consistently outperforms the baseline after adaptation to API services, code, games, and GUI desktops.The evaluation converts trajectories into (s_t, a_t, s_t+1) transition tuples and reuses the Factuality and Web Turing metrics.
  • 6.3 Cross-Environment Generalization: Table 8 summarizes strong adaptation capability across unseen environments.

7 Conclusions and Limitations

WebWorld is a browser simulator trained on over one million real-world interaction trajectories and improves downstream task performance. The authors identify sycophancy and weak detailed-content generation as limitations.

  • WebWorld is trained on over one million real-world interaction trajectories.
  • WebWorld significantly improves performance on downstream tasks through simulation-based agent training.
  • WebWorld may generate overly optimistic outcomes that cater to the agent’s action.
  • WebWorld struggles to generate high-quality, detailed content such as scientific articles.

Impact Statement

WebWorld aims to enable scalable offline training for web agents while reducing practical constraints and safety risks associated with real-world interaction. The paper also acknowledges dual-use, data-quality, and model-behavior risks that remain despite mitigation efforts.

  • WebWorld enables scalable offline training that circumvents latency, safety constraints, and rate-limiting issues in real-world web interaction.
  • Simulation training can reduce risks from unintended purchases, form submissions, or data modifications during agent exploration.
  • The work seeks to democratize web-agent research by providing an open, high-fidelity simulator trained on diverse real-world trajectories.
  • The authors identify dual-use risks including phishing, credential stuffing, and large-scale scraping that violates terms of service.
  • Web crawls may retain personally identifiable information, toxic content, demographic biases, and sycophancy in simulated predictions despite filtering and safety heuristics.
  • The authors release documentation and ethical guidelines and recommend additional safety checks before high-stakes deployment.

A World Model Training Details

WebWorld’s training uses full supervised fine-tuning with memory- and throughput-oriented infrastructure, while comparing three Qwen3 model scales to study scaling effects.

  • WebWorld uses LLaMA-Factory for supervised fine-tuning with full-parameter training and DeepSpeed ZeRO configurations.Qwen3-32B uses ZeRO-3, while the other models use ZeRO-2.
  • Training runs for 1 epoch with sequence packing, a 20,000-token cutoff, and a cosine learning-rate scheduler.
  • Stage 1 emphasizes large-scale dynamics learning and throughput, whereas Stage 2 refines reasoning with conservative fine-tuning to prevent forgetting.
  • The study trains Qwen3-based world models at 8B, 14B, and 32B parameters to examine model-scale effects.
  • Evaluation uses state-of-the-art proprietary models, with the specific API models and versions listed in Table 10.

C Baseline Implementation Details

The baseline implementations standardize competing datasets or checkpoints around Qwen3-8B where possible, while WebWorld defines broad interaction and representation pipelines for diverse environments.

  • Baseline Implementation Details: WMA data is reformatted and fine-tuned on Qwen3-8B to isolate data and training-objective effects from the foundation model.WMA predicts free-form descriptions of state changes rather than structured state representations.
  • Baseline Implementation Details: WebSynthesis is reproduced from its official 27k-sample dataset and fine-tuned on Qwen3-8B under the main experiment hyperparameters.
  • Baseline Implementation Details: Word2World is evaluated zero-shot using an open-weights WorldModel-Webshop-Llama3.1-8B checkpoint despite substantial format misalignment.
  • F Action Space Definition: WebWorld’s unified action space supports element-based and coordinate-based interactions, browser controls, and meta-actions across websites and mini-games.
  • F Action Space Definition: The A11y Tree is the primary state representation because it provides a structured, information-dense abstraction of interactable UI elements.
  • F Action Space Definition: The system converts web trajectories into multiple representations and constructs cross-environment transition tuples for API, code, games, and GUI domains.
  • F Action Space Definition: LLM-based URL filtering scores accessibility, content suitability, interactivity, and engineering quality before data collection.

J World Model Evaluation Taxonomy

World-model evaluation distinguishes intrinsic quality measurements from extrinsic downstream task performance. WebWorld’s evaluation framework and cross-environment datasets extend assessment across multiple domains and trajectory settings.

  • Intrinsic evaluation explicitly measures world-model quality, whereas extrinsic evaluation assesses downstream task performance.
  • Prior intrinsic approaches evaluate structural correctness, content similarity, semantic consistency, or information coverage against ground-truth web states.
  • Figure 6 compares URL-quality score distributions across four dimensions with a filtering threshold for data collection.
  • WebWorld constructs training and test sets for API services, code, games, and GUI by converting open-source trajectories into state-action-next-state tuples.
  • The evaluation data includes terminal and code/IDE trajectories, GUI/desktop data, game and simulation data, and API or knowledge-base-query tasks.
  • Prior extrinsic approaches measure world models through end-to-end task success, including MCTS-based agent performance.

K Generation Length Analysis

The analysis compares output lengths before and after reasoning activation, showing that reasoning produces more concise simulations despite adding reasoning tokens. The surrounding pipeline uses text-centric representations and provides prompt templates for data synthesis and evaluation.

  • Output Length Comparison: Approximately 49.4% reduction in average output length occurs after Reasoning Activation compared with the Real-World Transition Modeling baseline.The baseline produces longer outputs while modeling comprehensive web states; the reasoning stage yields more concise and structured simulation patterns.
  • Output Length Comparison: Reasoning Activation changes generation patterns beyond adding reasoning tokens, shifting the model toward shorter simulation outputs.The comparison spans the Real-World Transition Modeling and Reasoning Activation stages under varying data scales.
  • Representation Choice: WebWorld uses A11y Tree and HTML rather than visual simulation to prioritize precise interaction-state modeling and compatibility.The text-centric design avoids the rendering limitations and computational overhead associated with visual generation.
  • Prompt Templates: The appendix provides prompt templates for WebWorld, the Actor agent, the Value model, and two-stage agent data synthesis.Additional templates cover factuality and Web Turing evaluations, self-proposed tasks, long-horizon dependencies, composite interactions, and curiosity-driven exploration.
Loading 2602.14721v1…