Source-linked AI summary
Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
Guanting Dong, Junting Lu, Junjie Huang, Wanjun Zhong, Longxiang Liu, Shijue Huang, Zhenyu Li, Yang Zhao, Xiaoshuai Song, Xiaoxi Li, Jiajie Jin, Yutao Zhu, Hanbin Wang, Fangyu Lei, Qinyu Luo, Mingyang Chen, Zehui Chen, Jiazhan Feng, Ji-Rong Wen, Zhicheng Dou
TL;DR
General-purpose agents need realistic, stateful environments and principled mechanisms for continual improvement, but existing environment construction and training approaches leave these gaps unresolved. Agent-World combines scalable environment-task discovery with continuous self-evolving multi-environment training, and reports consistent gains across 23 challenging benchmarks alongside scaling trends tied to environment diversity and self-evolution rounds.
Problem
Training robust general-purpose agents is limited by insufficiently realistic, complex environments and the lack of mechanisms to diagnose weaknesses and drive continual self-improvement.
Method
Agent-World mines topic-aligned databases and executable toolsets, synthesizes verifiable tasks, and combines multi-environment reinforcement learning with a diagnostic arena for targeted iterative learning.
Results
Across 23 challenging agent benchmarks, Agent-World consistently improves performance over strong baselines, with scaling trends linked to environment diversity and self-evolution rounds.
Takeaways & Limitations
Agent-World provides empirical evidence that scalable realistic environments and continuous self-evolution can support more general agent intelligence.
Takeaways & Limitations
Existing environments often mismatch real-world interaction logic or lack sufficient complexity for long-horizon, state-intensive tasks, while prior work lacks principled continual self-improvement mechanisms.
Abstract
from arXiv · showhide
Large language models are increasingly expected to serve as general-purpose agents that interact with external, stateful tool environments. The Model Context Protocol (MCP) and broader agent skills offer a unified interface for connecting agents with scalable real-world services, but training robust agents remains limited by the lack of realistic environments and principled mechanisms for life-long learning. In this paper, we present \textbf{Agent-World}, a self-evolving training arena for advancing general agent intelligence through scalable environments. Agent-World has two main components: (1) Agentic Environment-Task Discovery, which autonomously explores topic-aligned databases and executable tool ecosystems from thousands of real-world environment themes and synthesizes verifiable tasks with controllable difficulty; and (2) Continuous Self-Evolving Agent Training, which combines multi-environment reinforcement learning with a self-evolving agent arena that automatically identifies capability gaps through dynamic task synthesis and drives targeted learning, enabling the co-evolution of agent policies and environments. Across 23 challenging agent benchmarks, Agent-World-8B and 14B consistently outperforms strong proprietary models and environment scaling baselines. Further analyses reveal scaling trends in relation to environment diversity and self-evolution rounds, offering insights for building general agent intelligence.
1 Introduction
Agent-World addresses the difficulty of training general-purpose agents in realistic, stateful environments by combining scalable environment-task discovery with continuous self-evolving training. Across 23 benchmarks, it reports consistent performance advantages and scaling relationships involving environment diversity and self-evolution.
- Motivation: Real-world agent environments are compositional and stateful, requiring multi-tool orchestration and tracking of environment state transitions.Stateless or single-tool settings are described as insufficient for realistic applications.
- Motivation: Existing approaches still struggle to synthesize realistic, complex environments at scale for long-horizon, state-intensive tasks.LLM-generated environments may mismatch real-world interaction logic, while synthetic environments can remain limited in complexity.
- Motivation: Existing work also lacks principled mechanisms for using scalable environments to diagnose agent weaknesses and drive continual self-improvement.The paper identifies this as a separate bottleneck from environment construction and scaling.
- Agent-World: Agent-World unifies scalable real-world environment synthesis with continuous self-evolving training in a closed-loop arena.Its two-stage design couples environment-task discovery with agent training and iterative improvement.
- Agent-World: Agentic Environment-Task Discovery mines executable databases and toolsets from thousands of real-world themes, then synthesizes verifiable tasks with controllable difficulty.The environment ecosystem includes 1978 environments and 19822 tools.
- Agent-World: Continuous Self-Evolving Agent Training applies multi-environment reinforcement learning and executable rewards while synthesizing tasks, identifying capability gaps, and driving targeted learning.This forms a co-evolution loop between agent policies and environments.
- Results: Across 23 challenging benchmarks, Agent-World-8B and 14B consistently outperform strong foundation models and competitive baselines.The evaluation spans agentic tool-use, advanced AI assistants, software engineering, deep research, and general reasoning.
2 Preliminary: Agentic Interaction with Multi-Environments
The paper models multi-turn interaction with multiple tool environments as a POMDP whose state combines external environment state and dialogue state. Agents infer hidden environment state through observations and act through either executable tools or language responses.
- POMDP Formulation: Multi-turn interaction with external environments is modeled as a POMDP represented by (U, S, A, O, P).The tuple captures intent, state, actions, observations, and transition dynamics.
- POMDP Formulation: The intent space U contains the user’s latent intent, which the assistant infers from interaction history and environment feedback.The inferred intent guides action selection.
- State and Environment: The global state is factored as S = S_E × S_H, combining mutable environment state with dialogue state.Environment state covers external databases, files, and services, while dialogue state summarizes conversational context and constraints.
- State and Environment: Each environment is parameterized by a database D and toolset F, where executable tools read or write the database and induce state transitions.The database stores structured records or files that constitute the mutable external world.
- Actions and Observations: The action space combines structured tool-use actions with natural-language response actions.Tool actions query or modify the environment, whereas response actions emit intermediate or final messages.
- Actions and Observations: Observations combine structured tool outputs with dialogue-side signals, while the environment state remains hidden and must be inferred indirectly.Tool observations can include query results, logs, and error codes; dialogue observations include user utterances and termination signals.
- Dynamics: The transition model P maps a state-action pair to the next state and observation after tool execution or response generation.Tool execution occurs against database D and may update the environment state.
3 Methodology
Agent-World builds a scalable, realistic environment ecosystem from real-world themes and uses it to synthesize verifiable tasks and continuously train agents through diagnosis-driven self-evolution. Its methodology combines automated database and tool construction, hierarchical organization, multiple task-generation strategies, and a closed-loop training arena.
- Verifiable task synthesis: Agent-World synthesizes tasks with graph-based DAG walks and programmatic Python solutions, covering sequential dependencies, conditionals, loops, and result aggregation.Tasks include solution code and verifier scripts, while difficulty is expanded through executable validation and longer, more logically complex interactions.
- Environment and tool ecosystem: Agent-World mines real-world themes into databases, generates executable tools, verifies them, and organizes the resulting ecosystem into a hierarchical taxonomy.The taxonomy contains 20 first-tier labels, 50 second-tier labels, and over 2K third-tier labels.
- Verifiable task synthesis: All synthesized tasks contain at least 7 interaction turns, average over 20 turns, and include a non-trivial portion exceeding 40 turns.Pass@10 evaluation with Doubao-Seed-2.0-pro finds that most tasks are solved only once out of ten attempts, with some unsolved.
- Continuous self-evolving training: Continuous self-evolving training uses multi-environment agent–tool–database rollouts with executable rewards, then diagnoses capability gaps on fresh tasks and expands environments accordingly.The resulting arena-driven loop acts as an automated curriculum engine and supports co-evolution between agent policies and environments.
4 Experiment
Across 23 benchmarks, Agent-World consistently outperforms foundation models and environment-scaling baselines, with stronger cross-environment generalization and positive scaling from larger environments and continued self-evolution.
- 4.2 Main Results: Current foundation models remain limited on stateful, long-horizon tool use requiring multi-step planning, orchestration, and state tracking.GPT-5.2 High reaches 53.1% on MCP-Mark, while GPT-OSS-120B and Qwen3-235B-A22B reach 4.7% and 5.8%.
- 4.2 Main Results: Existing environment-scaling methods produce uneven gains across environments, with simulators struggling to capture complex real-world state transitions.Programmatic methods provide broader gains but retain clear weaknesses on specific benchmarks.
- 4.2 Main Results: Agent-World-8B consistently outperforms prior environment-scaling baselines across all three benchmark suites.It achieves 61.8% on τ2-Bench, 51.4% on BFCL V4, and 8.9% on MCP-Mark.
- 4.3 Generalization: Agent-World preserves general reasoning while delivering its largest gains on long-horizon search, coding, knowledge, and MCP tasks.The model improves across iterative planning, software engineering, information retrieval, multi-tool coordination, and heterogeneous external-tool capabilities without degrading core math reasoning.
- 4.3.2 Generalization on Agentic AI Assistant Scenarios: On advanced assistant benchmarks, Agent-World-8B surpasses Qwen3-8B, EnvScaler-8B, and AWM-8B across SkillsBench, ARC-AGI-2, and Claw-Eval.Its scores are 9.2%/6.5%/30.5%, respectively, and improve consistently to 12.6%/8.5%/31.5% at 14B.
- 4.3 Scaling and Self-Evolution: Environment expansion shows positive scaling with diminishing returns at larger scales, while self-evolution produces monotonic gains across evaluation suites.The average score rises from 18.4% to 38.5% (+20.1 points); after two rounds, Agent-World-14B improves from 45.3%/52.4%/29.5% to 50.5%/55.8%/38.1%.
5 Related Work
Related work advances agentic reinforcement learning and environment scaling, but realistic, stateful, and scalable environments remain a central challenge.
- 4 Related Work: Environment-scaling research explores LLM-driven simulation, while real-world services remain restricted and manually constructed sandboxes are costly and difficult to scale.These constraints motivate scalable environment construction for training LLM agents.
- 4 Related Work: Agentic reinforcement learning has expanded from single-tool optimization to long-horizon web agents through verifiable rewards, improved reward design, and policy optimization.The related systems include Tool-Star, ToolRL, OTC, and ARPO.
6 Conclusion
The paper concludes that Agent-World combines realistic environment-task discovery with diagnostic, targeted self-evolving training, and that scaling environments and evolution improves agent performance across challenging benchmarks.
- 6 Conclusion: Agent-World unifies real-world database and executable-tool mining with verifiable task synthesis and continuous diagnostic training.Its two components identify capability gaps and drive targeted iterative data expansion.
- 6 Conclusion: Experiments across 23 challenging benchmarks demonstrate consistent improvement over strong baselines.Further analyses identify scaling trends involving environment diversity, evolution rounds, and task difficulty.
- 6 Conclusion: The paper presents scalable realistic environments as both training data sources and infrastructure for advancing general agent capabilities.This conclusion is framed within the evaluated realistic tool-environment setting.
B Environment Visualizations
The appendix visualization section uses self-contained capsules to document representative environments and their executable interfaces.
- B Environment Visualizations: Each environment capsule summarizes the environment, maps its files, lists tools and intents, and shows one tool implementation.Additional domains follow the same subsection and environment-visualization capsule pattern.
B.1 Arxiv_local
Arxiv_local is a fully offline, file-backed paper library that exposes deterministic tools for listing, metadata retrieval, and title search. Its manifest indexes paper paths and titles, while Markdown cards store canonical metadata fields.
- Environment: The environment stores one Markdown metadata card per paper alongside a JSON manifest for fast title indexing.The repository layout includes manifest.json and papers/*.md cards.
- Tools: list_local_papers returns every paper_id and title discovered through manifest.json, distinguishing an empty manifest from a parse error.
- Tools: get_local_paper_metadata opens a paper card and returns canonical fields, using None for absent values and raising when the file is missing.The fields include title, authors, abstract, subjects, comments, DOI, URL, journal reference, and project page.
- Tools: search_local_papers performs case-insensitive substring matching over manifest titles and returns matching identifier-title rows.It is intended for deterministic planning before opening individual cards.
- Implementation: The reference metadata parser strips lines, treats the first hash-prefixed line as the title, and parses recognized colon-prefixed metadata fields.Recognized fields include authors, abstract, subjects, comments, DOI, URL, journal reference, and project page.
B.2 Emails
The Emails environment is a file-backed mailbox for evaluation, centered on structured synthetic messages and read-only tools for lookup, search, folder inspection, attachment access, sampling, and aggregation. The supplied environment description also specifies analogous JSON-backed Calendar and Hotels services with deterministic query and booking surfaces.
- Emails: The mailbox combines a large historical Enron spam/ham CSV with 2,000 structured synthetic messages stored in emails_export.json.Synthetic records include folders, headers, bodies, labels, and attachment filenames.
- Emails: emails_export.json is the primary corpus for listing, lookup, search, and spam/ham aggregation without requiring a network.
- Emails: Mailbox tools support pagination, folder filtering, message lookup, substring search, attachment verification and retrieval, spam/ham sampling, and per-folder aggregation.The listed tools include list_emails, get_email_by_id, search_emails, get_attachment_content, sample_enron_spam, and get_email_counts_by_folder_and_spam.
- Calendar: The Calendar service stores users, holidays, and 34 timed events as JSON and provides range, identity, attendee, title, and event lookup tools.Date filters use inclusive calendar dates derived from UTC timestamps.
- Hotels: The Hotels service supports end-to-end drills from geocoding and hotel search through room details and booking using structured inventory and booking state.Its files include hotels, hotel details, rates, facilities, places, and bookings data.
C Case Study
The paper presents three representative agent trajectories, each with at least seven interaction turns. Each case reports its environment context, task requirement, tool inventory, rubric criteria, and an abbreviated trajectory.
- Case Study: 3 representative trajectories are presented, each containing at least 7 interaction turns.
- Case Study: Each case reports the environment context, task requirement, tool inventory, rubric criteria, and an abbreviated interaction trajectory.
C.1 Case 1: Ecomm MCP Server
Case 1 uses an e-commerce MCP server to execute a multi-step customer return under policy constraints. The workflow uses four of seventeen tools across nine steps and includes order-state verification.
- C.1 Case 1: Ecomm MCP Server: Case 1 is an Ecomm MCP Server scenario focused on multi-step return execution.
- C.1 Case 1: Ecomm MCP Server: 4/17 tools are used across 9 steps in the e-commerce return workflow.
- C.1 Case 1: Ecomm MCP Server: The task requires identity verification, order enumeration, eligibility checks, item-level return intent, refund-destination confirmation, and policy-constrained submission.
- C.1 Case 1: Ecomm MCP Server: The agent completes the return transaction with iterative order-state verification and user-confirmed actuation.
C.2 Case 2: Slack Workspace Automation Server
Case 2 evaluates Slack compliance triage by combining multiple evidence checks into a deterministic decision and comment-generation workflow.
- The task identifies sender attributes, compares message text and timing with a target user's CSV history, checks login state and contact-graph dependencies, and generates a deterministic triage comment.The workflow uses five of eighteen available tools across seven steps.
- The agent combines message forensics, lexical similarity scoring, temporal reasoning, and contact-graph constraints into the triage decision.
C.3 Case 3: Population Data Server
Case 3 evaluates quantitative city-influence ranking under tool constraints, combining population data, growth adjustments, percentage lookups, and logarithm constants.
- The case is framed as quantitative ranking under tool constraints in the Population Data Server environment.
- The task ranks top-10 population cities plus Karachi using growth-adjusted scoring and outputs the winner with a transformed magnitude metric.
- The workflow integrates sampled population data, category-specific percentage lookups, and logarithm constants under available tool precision limits.It uses five of eleven available tools across ten steps.