PaperlayerView exact revision on arXiv

Source-linked AI summary

Qwen-AgentWorld: Language World Models for General Agents

Yuxin Zuo, Zikai Xiao, Li Sheng, Fei Huang, Jianhong Tu, Yuxuan Liu, Tianyi Tang, Xiaomeng Hu, Yang Su, Qingfeng Lan, Yantao Liu, Qin Zhu, Yinger Zhang, Bowen Yu, Haiquan Zhao, Haiyang Xu, Jianxin Yang, Jiayang Cheng, Junyang Wang, Lianghao Deng, Mingfeng Xue, Tianyi Bai, Yang Fan, Yubo Ma, Yucheng Li, Zeyu Cui, Zhihai Wang, Zhihui Xie, Zhuorui Ye, An Yang, Dayiheng Liu, Jingren Zhou, Ning Ding

arXiv:2606.24597v1cs.CL

TL;DR

LLM-agent environments lack a general-purpose world model for predicting subsequent states, despite world modeling being essential alongside policy learning. Qwen-AgentWorld develops native language world models, benchmarks them across seven domains, and evaluates them as simulators and agent warm-ups; its models outperform frontier baselines and improve downstream agent performance.

  • Problem

    LLM-agent research has focused mainly on policies, leaving general-purpose world models for predicting environment state transitions underdeveloped.

  • Method

    Qwen-AgentWorld trains native language world models through CPT, SFT, and RL, and evaluates them with AgentWorldBench across seven agent interaction domains.

  • Results

    Qwen-AgentWorld-397B-A17B achieves the highest seven-domain rubric average at 58.71, surpassing GPT-5.4 at 58.25.

  • Takeaways & Limitations

    As a decoupled simulator and unified agent foundation model, Qwen-AgentWorld improves agentic RL and downstream performance across seven diverse tasks.

  • Takeaways & Limitations

    The three training stages use strictly disjoint data pools, with SFT and RL relying exclusively on internally accumulated trajectories.

Abstract

from arXiv · show

A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigate how world modeling based on language models can further push the boundaries of general agents. (i) We first focus on building foundation models for agentic environment simulation. We introduce Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B, the first language world models capable of simulating agentic environments covering 7 domains via long chain-of-thought reasoning. Leveraging more than 10M environment interaction trajectories of 7 domains in real-world environments, we develop Qwen-AgentWorld through a three-stage training pipeline: CPT injects general-purpose world modeling capabilities from the state transition dynamics and augmented professional corpora, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity through a tailored framework with hybrid rubric-and-rule rewards. To evaluate language world models, we present AgentWorldBench, a comprehensive benchmark constructed from real-world interactions of 5 frontier models on 9 established benchmarks. Empirical results demonstrate that Qwen-AgentWorld significantly outperforms existing frontier models. (ii) Beyond foundation models, we further investigate two complementary paradigms through which world modeling enhances general agents. First, as a decoupled environment simulator, Qwen-AgentWorld supports scalable and controllable simulation of thousands of real-world environments for agentic RL, yielding gains that surpass real-environment training alone. Second, as a unified agent foundation model, world-model training acts as a highly effective warm-up that improves downstream performance across 7 agentic benchmarks. Code: https://github.com/QwenLM/Qwen-AgentWorld

1 Introduction

The introduction argues that general agents need a language-based world model alongside a policy, and presents Qwen-AgentWorld as a unified model for simulating seven agent environments. It further highlights complementary uses as a scalable simulator for agentic RL and as a warm-up for stronger downstream agents.

  • Motivation: LLM-agent research has focused primarily on policies, leaving general-purpose world modeling as a crucial missing component in the agent–environment loop.The policy maps states to actions, while the world model predicts subsequent states from states and actions.
  • Environment Simulator: 4k real-world OpenClaw environments are simulated for agentic RL, with gains on Claw-Eval and QwenClawBench and complementary improvements on Tool Decathlon, MCPMark, and WideSearch.The simulator is described as scalable and controllable, including turn-level scaling without dedicated infrastructure.
  • Agent Foundation Model: LWM training serves as a warm-up or auxiliary stage that acquaints agents with environment dynamics and next-state prediction before downstream agentic RL.The introduction describes this training as a foundation for bootstrapping stronger agent performance across the listed agentic benchmarks.

2 Preliminaries

This section formalizes language world models through a shared textual representation spanning seven domains, defines the unified trajectory schema, and states the world-modeling objective. It also distinguishes LWM training and simulation-based reinforcement learning from real-environment training, while clarifying trajectory terminology.

  • Language world model formalization: The framework unifies seven domains under a shared textual representation to support cross-domain generalization in language world modeling.The section introduces terminology, a unified trajectory schema, and the world-modeling objective.
  • Training terminology: LWM training comprises continual pre-training, supervised fine-tuning, and reinforcement learning, whereas Sim RL and Real RL train agents with simulated and live environments, respectively.Sim RL uses an LWM as its environment simulator; Real RL uses an actual search engine or running terminal.
  • Trajectory terminology: An environment trajectory is a multi-turn dialogue represented as sequential (action, observation) pairs, while an agentic trajectory is a tool-integrated reasoning trace for one task.The agentic trajectory interleaves internal thinking and action selection with environment observations.

2.2 Unified Environment Trajectory Schema

The paper defines a unified trajectory schema for training one language world model across seven domains with heterogeneous state representations. It formalizes actions, observations, prompts, and next-observation prediction while supporting both stateless and stateful environments.

  • Schema design: A shared trajectory format enables one world model to train uniformly across seven domains, despite representations ranging from Terminal file-system snapshots to Android UI view hierarchies.The schema is used across all seven domains and training stages.
  • Schema design: Each trajectory represents an action as the agent’s turn-level output and an observation as the environment’s feedback, including tool responses and command outputs.This action–observation pairing provides the common interaction structure across domains.
  • Prompt structure: The assembled system prompt contains five components: task description, action space, initial state, interaction examples, and simulation instruction.The simulation instruction can impose controllable conditions, such as hiding answers from web_search responses.
  • Next-state prediction: The language world model predicts the next environment observation from the system prompt, full interaction history, and current action, using the ground-truth next observation as its training target.Stateless domains carry state implicitly in conversation history, while stateful domains maintain explicit state that evolves with actions.
  • Cross-domain coverage: The same observation sequence spans diverse domains: SWE requires predicting an out-of-memory traceback after script execution, whereas Android requires predicting a rendered HTML screen after a UI tap.These examples illustrate how next-state prediction exercises domain-specific reasoning under a shared schema.

3 Training Recipe

Qwen-AgentWorld uses a three-stage training recipe in which CPT injects world knowledge, SFT activates next-state-prediction reasoning, and RL sharpens output quality. The stages rely on disjoint data pools and unified, filtered environment-trajectory processing designed to preserve causal state transitions.

  • Three-stage pipeline: The recipe follows “CPT injects, SFT activates, RL sharpens” across continual pre-training, supervised fine-tuning, and reinforcement learning.CPT injects environment world knowledge, SFT instills explicit next-state-prediction thinking, and RL uses hybrid rubric-and-rule rewards to improve output quality.
  • Data sources and separation: The three stages use strictly disjoint data pools, with CPT drawing on infrastructure, open traces, and domain corpora, while SFT and RL use internal trajectories.Because on-policy RL can amplify data artifacts, the downstream data pipeline receives most of the engineering effort.
  • Unified processing: Heterogeneous agent trajectories are normalized into a shared schema of system context, agent actions, and environment observations, then expanded into turn-level next-observation prediction samples.For a trajectory with T turns, each turn can provide a prediction target conditioned on its preceding history, state, and action.
  • Unified processing: Filtering removes short, invalid, failed, or causally uninformative sequences and turns while preserving state continuity through retry-cycle skipping.GUI no-change turns are removed because they can teach the model to copy prior states rather than predict action-induced changes.
  • CPT data curation: Four statistics—Overlap, Novelty, Jaccard, and length ratio—classify turns into seven semantic categories with category-specific keep ratios to retain turns carrying genuine world knowledge.The method targets low-quality gradient noise without deleting turns that later interactions depend on as context.
  • RL refinement: RL stability is studied through reward design and training dynamics, with factuality showing the largest relative improvement at 11.3% while remaining the lowest-scoring rubric dimension.The Turing-Test reward barely converges because judge false negatives create unreliable signals when predictions closely match ground truth.

4 AgentWorldBench

AgentWorldBench evaluates language world models across seven agent-interaction domains using real-environment trajectories derived from frontier agents on established benchmarks. Its long-context, turn-level evaluation compares predicted observations with ground truth using domain-aware scoring across five dimensions.

  • Construction: AgentWorldBench converts trajectories from five frontier agents running nine established benchmark query sets across seven domains into environment trajectories with real-environment ground-truth observations.Queries come from established high-quality agentic benchmarks, and trajectories retain frontier-agent reasoning, tool calls, and error recovery.
  • Turn-level Sampling: The benchmark samples five evaluation turns per text-domain trajectory, retaining the first and last turns plus three uniformly sampled intermediate turns.The first turn anchors recursive state fidelity, while the last turn probes long-context dependence on accumulated prior state.
  • Benchmark Statistics: 2,170 evaluation samples span seven domains, with text-based domains comprising 72.4% and each GUI domain contributing 9.2%.SWE and Search are the largest individual shares at 21.8% and 21.1%, respectively.
  • Evaluation: An LLM judge scores each predicted observation on Format, Factuality, Consistency, Realism, and Quality, with the primary score equal to their mean on a 0–100 scale.Judging is reference-grounded: the predicted observation is compared with the real-environment observation using domain-specific rubrics.
  • Judge Selection: Gemini 3 Flash, Claude Sonnet 4.5, and GPT-5.2 are compared as judge candidates, with Gemini 3 Flash the most lenient and GPT-5.2 the most stringent.Each candidate independently scores predictions from all models across all seven AgentWorldBench domains.

5 Experiments

Qwen-AgentWorld is evaluated across seven domains on AgentWorldBench against 14 diverse baselines, with Qwen-AgentWorld-397B-A17B achieving the highest overall score. World-model training improves both model scales and transfers across held-out domains, although Search remains the most difficult domain.

  • Experimental setup: The evaluation compares Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B against 14 frontier proprietary, open-weight, and non-world-model-trained Qwen baselines.The Qwen baselines share the same base architecture but omit the three-stage CPT→SFT→RL pipeline, isolating its training effect.
  • Main results: Qwen-AgentWorld-397B-A17B achieves the highest overall average, scoring 58.71 versus GPT-5.4 at 58.25 across AgentWorldBench’s seven domains.Scores average Format, Factuality, Consistency, Realism, and Quality on a 1–5 rubric normalized to 0–100.
  • Effect of world-model training: At 35B, world-model training adds 8.66 points, raising the score from 47.73 to 56.39 and surpassing Claude Sonnet 4.6 at 56.04.At 397B, the overall average rises from 54.74 to 58.71; text and GUI averages increase by 3.35 and 4.92, respectively.
  • Cross-domain generalization: Stage 3 training on Terminal data alone improves Terminal by +14.2 points and held-out MCP, SWE, and Search by +5.0, +11.5, and +11.8.This tests generalizable world knowledge by withholding domain-specific training signals from the held-out domains.
  • Domain-level observations: Search is the most challenging domain, with a best score of 37.82 versus 68.49 on SWE and 70.10 on MCP.The passage attributes the difficulty to evolving web content and factual consistency across long retrieval chains.

6 Applications

Qwen-AgentWorld enhances general agents through scalable, controllable environment simulation and unified world-model training. Simulation improves agentic RL across real and fictional environments, while LWM warm-up transfers next-state reasoning across domains and benchmarks.

  • Standalone environment simulator: Qwen-AgentWorld provides scalability and controllability unavailable in real environments, enabling training across domains and conditions beyond real-environment coverage.As a standalone simulator, the policy agent and world model remain separate models.
  • Scalable simulation: 4k simulated OpenClaw environments yield +4.3 on Claw-Eval and +7.1 on QwenClawBench without domain-specific adaptation.Qwen-AgentWorld-397B-A17B improves Claw-Eval from 65.4 to 69.7 and QwenClawBench from 47.9 to 55.0.
  • Controllable simulation: Controllable simulation improves Tool Decathlon by +3.7 and MCPMark by +12.3, whereas standard Sim RL drops Tool Decathlon from 32.4 to 31.5.Without grounded control instructions, the training signal is too noisy to produce gains.
  • Fictional-world construction: In fictional search environments, controllable Sim RL raises 35B F1 by Item from 34.02 to 50.31 and F1 by Row from 13.72 to 24.21.At 397B, gains remain +3.87 and +6.05 despite the base model’s 70.11 F1 by Item.
  • Unified agent foundation model: LWM warm-up transfers single-turn, non-agentic reasoning without tool calls to multi-turn tool-calling tasks across seven benchmarks and five domains.Out-of-distribution gains include +11.3 on Claw-Eval, +9.7 on QwenClawBench, and +9.0 on BFCL v4.
  • Unified agent foundation model: LWM RL improves in-domain benchmarks, including WideSearch F1 by Item from 33.38 to 46.17 (+12.79) and Terminal-Bench 2.0 accuracy from 33.25 to 39.55 (+6.30).Reasoning traces show internalized world modeling predicts environment responses, identifies infeasible approaches, and refines action plans before execution.

7 Analysis

Qwen-AgentWorld’s accurate predictions arise from deliberative, leakage-aware, and multi-step reasoning. RL further improves fidelity at the level of URLs, byte counts, cryptographic invariants, and cross-turn API schemas.

  • 7.1 Reasoning Patterns: Across 129 turns, the model uses 1,347 “Wait!” interrupts, averaging 10.4 per turn and peaking at 56 in one SWE turn.Terminal and MCP show the highest rates, at 16.9 and 12.7 interrupts per turn, respectively.
  • 7.1 Reasoning Patterns: In Search, the model prevents information leakage by distinguishing the agent’s knowledge from what the environment should reveal when queries are unrelated to the reference answer.It identifies topic mismatches and avoids generating snippets that accidentally reveal the target information.
  • 7.1 Reasoning Patterns: The model performs multi-step causal reasoning across system abstractions, tracing a missing Node.js dependency through process, curl, piping, and JSON parsing failures.One Terminal example requires a six-step chain ending in a specific JSONDecodeError.
  • 7.2 Micro-Level Fidelity Improvements: RL progressively improves Search realism, shifting URL identifiers from tt2333444 to tt2988794 while producing natural source rankings and query-specific factual snippets.The trajectory compares checkpoints at Step 100 and Step 200.
  • 7.2 Micro-Level Fidelity Improvements: The model achieves exact Terminal byte arithmetic, including invisible \n bytes, and preserves cryptographic and Unix pipeline invariants.One example reaches an exact count of 53 bytes; identical files produce identical SHA256 hashes, while a failed tee does not affect stdout.
  • 7.2 Micro-Level Fidelity Improvements: In MCP, the model maintains perfect consistency across nine sequential API calls, preserving identifiers, parent references, and a complete Notion schema of ∼20 fields per block.This reflects stateful in-context database behavior with referential integrity across nested JSON objects.

8 Related Work

Qwen-AgentWorld builds on visual, embodied, and emerging language-based world-model research by extending internal environment simulation to structured text-based agent environments. It differs through a native seven-domain foundation model trained with CPT→SFT→RL and through two complementary uses of world modeling for general agents.

  • World Models: World models predict future environment states from actions and support planning or policy learning across visual, embodied, and autonomous-driving settings.Qwen-AgentWorld extends this paradigm to structured text observations, including API responses, terminal outputs, accessibility trees, and UI view hierarchies.
  • Language World Models: Language world-model research has explored fidelity evaluation, action correction, lookahead planning, model-based RL, MCTS, trajectory synthesis, and simulator-based agent training.The cited works include Imagine-then-Plan, DynaWeb, WebSynthesis, Simia, and RWML, among others.
  • Language World Models: Qwen-AgentWorld develops a native language world-model foundation model covering 7 domains through a CPT→SFT→RL pipeline, rather than post-hoc fine-tuning a general-purpose LLM.Its pre-training objective begins with environment modeling, distinguishing the approach from works that add world modeling after general-purpose language-model training.
  • Agent Foundation Models and Continual Pre-Training: Qwen-AgentWorld studies two complementary ways to improve general agents with world modeling: a foundation model for agentic environment simulation and downstream agent enhancement.This framing complements prior agent-foundation-model work using continual pre-training, tool-use simulation, and self-play.
  • Synthetic Environment Generation: Synthetic-environment research programmatically generates code-based environments, trajectories, and reinforcement arenas, including broad and domain-specific systems for web, GUI, mobile, terminal, and software-engineering tasks.Examples include Agent-World’s nearly 2,000 synthesized environments and specialized generators such as WebGym, GUI-Genesis, TermiGen, and SWE-Universe.

9 Conclusion and Future Work

The paper presents Qwen-AgentWorld, native language world models spanning seven agent-interaction domains, trained with a three-stage recipe and evaluated using AgentWorldBench. Future work targets agent–world-model co-evolution, multimodal modeling, adaptive sim-to-real routing, and dynamic tool synthesis.

  • Contributions: Qwen-AgentWorld is a family of native language world models covering seven agent-interaction domains at 35B-A3B and 397B-A17B scales, trained through CPT, SFT, and RL.The recipe progressively injects environment knowledge, activates next-state-prediction reasoning, and sharpens simulation fidelity.
  • Contributions: AgentWorldBench evaluates language world models by pairing every sample with a ground-truth observation from real environments.
  • Future Work: Agent–LWM co-evolution would use self-play to expand discovered states and generate increasingly challenging scenarios.
  • Future Work: A multimodal extension would fuse GUI screenshots with text-based state representations across Android, Web, and OS domains.
  • Future Work: Adaptive sim-to-real routing would choose per query between the world model and real environment to balance cost against fidelity.
  • Future Work: Dynamic tool synthesis would use the world model to create new tools on the fly instead of relying on a predefined tool set.

10 Authors

The paper credits a core contributor team, an alphabetically ordered contributor group, and an external advisor from Tsinghua University.

  • Core Contributors: The core contributors include Yuxin Zuo, Zikai Xiao, Li Sheng, Fei Huang†, Jianhong Tu†, and other listed authors.The full core-contributor list contains 17 names.
  • Contributors: Additional contributors are listed alphabetically, including Haiquan Zhao, Haiyang Xu, Jianxin Yang, and other named collaborators.This contributor group contains 15 names.
  • External Advisor: Ning Ding of Tsinghua University is identified as the external advisor.The paper names one external advisor.

A Domain Interaction Examples

Figures 14 and 15 illustrate representative agent–environment interactions across all seven Qwen-AgentWorld domains, showing each action alongside the simulated observation. In GUI domains, the model predicts the next state in HTML from the current screen and action, then renders it as a screenshot.

  • Interaction examples: Figures 14 and 15 cover representative interaction examples across all seven domains, pairing each agent action with its simulated observation.Each panel shows one agent–environment turn.
  • GUI domains: In GUI domains, Qwen-AgentWorld predicts the next GUI state in HTML from the current screen and agent action, then renders it as a screenshot.Figure 14 shows OS and Web, while Android appears in Section 2.3.
  • Text-based domains: Text-based examples include Terminal, MCP Tool Use, and Search, with Software Engineering shown in Section 2.3.These examples are presented in Figure 15.

B Training Dynamics

Across 440 RL steps, the five evaluation dimensions improve at different rates: surface formatting converges quickly, while consistency and realism require longer training. Factuality achieves the largest relative gain but remains the lowest-scoring dimension.

  • Training setup: Training tracks Format, Factuality, Consistency, Realism, and Quality across 440 RL steps, with evaluations every 10 steps.The analysis uses Qwen-AgentWorld-35B-A3B and follows dimension scores throughout RL training.
  • Dimension-wise dynamics: Consistency gains the most in absolute terms (+0.29, from 2.81 to 3.10), while Quality gains the least (+0.11).Format and Quality reach 90% of their total improvement within 90 steps, whereas Consistency and Realism require ∼250 steps.
  • Dimension-wise dynamics: Factuality improves by 11.3% relative to its initial value (2.03 to 2.26), the largest relative gain among all dimensions.Despite this improvement, Factuality remains the lowest-scoring dimension throughout training.

C Rule-Based Verification

The paper introduces deterministic rule-based verifiers for controllability, error handling, and long-context consistency, using trajectory-grounded test synthesis and validation. These checks corroborate the main findings, with GPT-5.4 ranking first overall and Qwen-AgentWorld-397B-A17B second while leading frontier models on GUI domains.

  • Verification framework: Rule-based verification deterministically tests controllability, error handling, and long-context consistency as targeted capability axes.The axes assess instruction adherence, faithful reproduction of environment failures, and coherent state tracking across long interaction histories.
  • Verification framework: Test cases are derived from the main evaluation set through trajectory-grounded synthesis and validation of deterministic initial states, constraints, and expected observations.Summaries include environment configuration, file-system or database state, active services, tool schemas, and other domain-specific states.
  • Controllability: Controllability tests whether the language world model follows explicit simulation conditions while preserving trajectory context and normal output formats.Examples include forcing terminal-command failures, withholding intermediate information, or predicting screens under different hidden states.
  • Error Handling: Error-handling verification checks deterministic failures such as missing files, malformed tool arguments, and writes to read-only paths instead of fabricated successes.The test cases retain turns with deterministic error outcomes caused by missing preconditions or invalid operations.
  • Long-Context Consistency: Long-context verification checks whether state changes remain coherent across turns, including files, environment variables, and databases.Cases identify cross-turn dependencies in which earlier actions change or reveal state that later observations depend on.
  • Results: 72.93 average score — GPT-5.4 ranks first overall, while Qwen-AgentWorld-397B-A17B scores 67.12, ranks second, and outperforms other frontier models on GUI domains.The rule-based results corroborate improvements in instruction adherence, environment-failure reproduction, and long-history state tracking.

C.1 Per-Axis Breakdown · D Open-Ended Judge Prompts

The appendix breaks down text-based and GUI performance by controllability, error handling, and long-context consistency, then specifies open-ended judge prompts across seven domains. These prompts use five shared dimensions, with content-type-specific verification and domain-specific calibration against ground truth.

  • C.1 Per-Axis Breakdown: Tables 11 and 12 report rule-based accuracy for controllability, error handling, and long-context consistency in text-based and GUI domains.They supplement the per-domain averages in Table 10.
  • D Open-Ended Judge Prompts: Open-ended evaluation prompts cover seven domains and score Format, Factuality, Consistency, Realism, and Quality.Text-domain prompts use content-type classification, while GUI prompts use anchors such as UI elements, navigation state, and visible text.
  • D.1 Terminal Domain: Terminal evaluation compares simulated output with real terminal output, treating the ground truth as the absolute reference.Deterministic and session-established content requires exact matching, while pre-existing state and runtime metadata require plausible formats and values.
  • D.1 Terminal Domain: Terminal prompts separately assess formatting, factual correctness, state consistency, realism, completeness, and conciseness against ground truth.Formatting includes prompt conventions, command echo, spacing, special markers, and output boundaries; factuality prohibits contradictions of known session state.
  • D.2 MCP Domain: MCP evaluation classifies responses into objective facts, numeric data, private or session data, and structural metadata with different verification standards.Objective facts and schema must match exactly, whereas numeric data may vary reasonably and inaccessible data requires validity and realism.
  • D.3 Search Domain: Search evaluation requires simulated web-search and extraction responses to adhere to ground-truth structure, factual content, contextual consistency, realism, and completeness.It emphasizes required result fields, exact matching of ground-truth facts and sources, consistent repeated calls, realistic ranking, and inclusion of top results.
  • D.4 SWE Domain: SWE prompts evaluate tool responses in command-line and filesystem environments using exact checks for objective facts and formatting, alongside semantic checks for context-dependent data.The model is assumed not to access real environment state beyond information shown or created during the current session.
Loading 2606.24597v1…