PaperlayerView exact revision on arXiv

Source-linked AI summary

CompoWorld: Compositional Environment Scaling for General Agents

Xiao-Wen Yang, Weiyi Xu, Wen Da, Hang Xu, Canwei Li, Hong-Jie You, Pusen Dong, Yucheng Zeng, Zhaokai Luo, Yu-Feng Li, Yao Hu, Mu Chuan

arXiv:2609.33665v1cs.AI

TL;DR

Real-world workflows require agents to connect information and actions across multiple services, while existing environment-generation approaches mainly create tasks within a single environment. CompoWorld composes reusable services into verified cross-environment training tasks and guides RL toward unmet completion criteria. Across eight benchmarks it improves on its backbone, and on AutomationBench it leads the compared agent-specialized 35B-A3B models.

  • Problem

    Real-world workflows couple states across multiple systems, requiring agents to carry information and constraints across services rather than merely make more tool calls.

    Source: §1 · pp. 1–2

  • Method

    CompoWorld composes reusable execution services through task-specific dependency graphs to generate verified cross-environment tasks for SFT and RL.

    Source: §1 · p. 2, §4.2 · p. 5, §4.3 · p. 6

  • Results

    CompoWorld averages a 9.17-point gain over its backbone across eight benchmarks and leads all compared agent-specialized 35B-A3B models on AutomationBench.

    Source: §1 · p. 2, §6 · p. 11

  • Takeaways & Limitations

    Service composition is a promising dimension of environment scaling for increasing training-data complexity and diversity and helping agents coordinate familiar capabilities in new workflows.

    Source: §6 · p. 11

Abstract

from arXiv · show

Automatically generated environments provide a scalable source of interaction data for training general agents. However, existing approaches mainly generate tasks within a single environment, while real-world workflows require agents to connect information and actions across multiple services. We introduce Compositional Environment Scaling (\textbf{CompoWorld}), which expands the task space by composing a finite library of reusable services. Coding agents turn tool specifications into verified services with typed states and shared interfaces, while a world model handles tools that cannot be reliably implemented. A random-walk procedure connects services through dependency graphs, enabling the generation and verification of tasks that require information to flow across services. Verified trajectories support supervised fine-tuning (SFT), while our Completion-Focused Rubric Reward guides reinforcement learning (RL) toward full task completion by emphasizing criteria with lower pass rates within each rollout group. We construct 448 services exposing 10,130 tools and use 3K SFT trajectories and 1K RL tasks to train Qwen3.6-35B-A3B. Experimental results show that CompoWorld improves on its backbone by 9.17 points on average across eight benchmarks. On AutomationBench, it surpasses frontier models such as Claude Opus 4.6 and leads all compared agent-specialized 35B-A3B models.

1 Introduction

Environment scaling has expanded interactive training environments, but real workflows require agents to carry information and constraints across multiple systems. CompoWorld makes reusable-service composition a controllable dimension of task generation and trains agents on verified cross-environment workflows.

Motivation

  • Real-world workflows depend on carrying information and constraints across services, beyond simply increasing tool calls or the number of environments.Earlier environment-scaling efforts construct tool-interaction environments and tasks, while CompoWorld targets service composition itself.Source: §1 · pp. 1–2

CompoWorld

  • CompoWorld builds cross-environment tasks from reusable execution services with typed states, tool sets, and transition logic.Its objective is to train agents to coordinate familiar services in novel combinations.Source: §1 · p. 2

Service and task construction

  • Coding agents build executable mock services from MCP specifications, while a world model simulates long-tail tools that cannot be implemented reliably.A random walk links services into dependency graphs; task generation and verification check transition reachability and success-condition satisfiability.Source: §1 · p. 2

Training

  • Verified successful trajectories support SFT, while Completion-Focused Rubric Reward gives greater weight to lower-pass-rate criteria during RL.The weighting targets unmet requirements when partial workflow completion earns credit but does not satisfy the full task.Source: §1 · p. 2

Results

  • CompoWorld averages a 9.17-point gain over its backbone across eight benchmarks and leads compared agent-specialized 35B-A3B models on AutomationBench.It reaches 32.33% task success on AutomationBench, exceeding Claude Opus 4.6 (25.50%).Source: §1 · p. 2

2 Related Work

Prior work studies environment scaling and compositional generalization through different settings and methods. CompoWorld contributes a training distribution built by recombining independently executable services and their dependencies.

2.1 Environment Scaling for LLM Agents

  • CompoWorld differs from Terminal-Universe [41] by reusing independently executable, stateful services connected through task-specific causal dependency graphs.Typed states and namespaced tools make service combinations and cross-service requirements explicit and controllable in environment generation.Source: §2.1 · pp. 2–3

2.2 Compositional Generalization for LLM Agents

  • CompoWorld generates verified cross-environment SFT and RL tasks by recombining services and dependencies, complementing work on compositional evaluation and reusable skills.The goal is transfer of knowledge about individual services to new workflows by learning how information and state changes connect.Source: §2.2 · p. 3

3 Preliminaries and Formulation

The formulation models each agent environment as a POMDP whose task success is checked by a terminal verifier. Composing environments preserves local service dynamics while allowing information to pass through the agent’s observations and actions.

General Agent Tasks

  • Each general agent task is modeled as a POMDP with state, action, transition, and observation spaces, and success determined by a terminal verifier.The agent samples actions from its interaction history, producing a trajectory without per-step rewards.Source: §3 · p. 3

Compositional Environments

  • Compositional environments combine independently executable environments as a product, with tools and observations associated with their service namespaces.An action updates only the selected environment and returns its local observation.Source: §3 · p. 3
  • Composition preserves each environment’s local dynamics and observation function; information passes between environments through the agent’s observations and subsequent actions..Source: §3 · p. 3

4 CompoWorld: Compositional Environment Scaling

CompoWorld scales general-agent training environments compositionally through service construction, cross-environment task generation and verification, and agent training.

4 CompoWorld: Compositional Environment Scaling

  • CompoWorld’s framework has three parts: building services, generating and verifying cross-environment tasks, and training agents on those tasks.Figure 2 depicts the three stages.Source: §4 · p. 3, Figure 2 · p. 4

4.1 Agent Environment Generation

  • Coding agents build typed-state services from tool specifications and test them independently; unreliable tools use selective world-model simulation, within a corpus of 448 services and 10,130 tools.The corpus spans application domains; no single domain accounts for more than a quarter of services.Source: §4.1 · pp. 3–5

4.2 Cross-Environment Task Generation and Verification

  • A random walk over selected services creates dependency edges, covering every service, avoiding consecutive repeats, and requiring a revisit to structure cross-service information flow.Walk length controls linked service visits without requiring more distinct services.Source: §4.2 · p. 5
  • Task authoring explores services, designs an instruction and initial state, probes a solution to define a reachable goal, and submits artifacts after verification.Executable rubric checks must accept the reference solution; the verifier allows successful trajectories that differ in unrelated state fields.Source: §4.2 · pp. 5–6

4.3 General Agent Training

  • Training first uses supervised fine-tuning on verified trajectories, then reinforcement learning with rubric outcomes that allow partial credit without prescribing tool sequences.Criteria are checked against the final joint state and relevant recorded outputs.Source: §4.3 · p. 6
  • Completion-Focused Rubric Reward weights lower-pass-rate criteria more heavily, while retaining positive weight for every criterion and assigning full reward only when all criteria pass.Reweighting cannot distinguish trajectories on a criterion that every rollout fails.Source: §4.3 · p. 6

5 Experiments

The experiments test performance across agent benchmarks and examine how training data, environment composition, and RL affect results. CompoWorld improves over its backbone across benchmarks, while the scaling experiments compare training on composed environments with training on isolated environments.

5.1 Experimental Settings

  • Evaluation covers eight benchmarks, while training uses 3K successful SFT trajectories followed by 1K RL tasks for Qwen3.6-35B-A3B.The benchmarks span knowledge-grounded interaction, planning, and workflow execution; default tasks compose five services and add five distractor services for tool discovery.Source: §5.1 · p. 7

5.2 Main Results

  • 9.17 points: CompoWorld improves over Qwen3.6-35B-A3B on average across eight benchmarks; AutomationBench success reaches 32.33%, up 22.00 points from the backbone.It leads all six compared agent-specialized 35B-A3B models on AutomationBench.Source: Table 1 · p. 8, §5.2 · p. 8
  • AutomationBench partial-credit mean rises from 41.94 to 72.68, with gains in every domain; Support is the exception when comparing with GPT-5.4 and GLM-5.2.HR has the largest gain (+49.40 points), while Sales remains the lowest-scoring domain at 58.93.Source: §5.2 · p. 8, Table 2 · p. 9

5.3.1 Effect of Environment Scaling

  • All four benchmarks improve over the backbone with as few as 100 SFT examples; SkillsBench peaks at 45.13 at 1K before declining to 40.91 at 3K.At 3K, τ3-Banking rises from 10.65 to 17.87 and DeepPlanning from 26.04 to 35.02; AutomationBench reaches 33.33% at 1K and 32.67% at 3K.Source: Figure 4 · p. 9, §5.3.1 · p. 9
  • With 448 services, composing five yields approximately 1.47 × 10^11 candidate sets and composing seven yields approximately 6.86 × 10^14; task synthesis and verification determine which support meaningful, solvable workflows.The scaling experiment pairs each of 100, 500, 1,000, and 3,000 SFT examples with a distinct composition, using the same backbone, service library, and SFT recipe without RL.Source: §5.3.1 · p. 9

5.3.2 Single vs. Composed Environment Scaling

  • With 1K trajectories and the same number of environments, composed-environment training outperforms single-environment training in all four completed comparisons.The score rises from 9.62 to 17.53 on τ3-Banking and from 12.83 to 33.33 on AutomationBench; single-environment SFT slightly degrades performance on τ3-Banking and DeepPlanning relative to the base model.Source: Figure 5 · p. 10, §5.3.2 · p. 10

5.3.3 RL Training Dynamics

  • RL improves on SFT by 1.2 points on average across eight benchmarks, but gains vary by benchmark, including a slight decline on τ3-Banking.Mean trajectory reward reaches about 0.75 at step 80 and ends near 0.79 at step 150; the paper says overall improvements are primarily driven by SFT.Source: §5.3.3 · p. 10

6 Conclusion

CompoWorld builds reusable executable services and connects them through task-specific causal dependencies. Its automated construction and verification pipeline generates cross-environment tasks for SFT and RL, and experiments across eight agent benchmarks show benefits from training on composed workflows.

6 Conclusion

  • CompoWorld builds reusable executable services and connects them through task-specific causal dependencies, making service composition a dimension of environment scaling.The conclusion presents this approach as a way to increase training-data complexity and diversity and help agents coordinate familiar capabilities in new workflows.Source: §6 · pp. 10–11
  • On AutomationBench, CompoWorld surpasses Claude Opus 4.6 and leads the compared agent-specialized 35B-A3B models.The paper also reports benefits across eight agent benchmarks.Source: §6 · p. 11
  • The automated construction and verification pipeline generates cross-environment tasks for supervised fine-tuning and reinforcement learning.The Completion-Focused Rubric Reward emphasizes unmet requirements to encourage full task completion.Source: §6 · p. 11

A Analysis of Completion-Focused Rubric Reward

The analysis relates completion-focused rubric weighting to policy-gradient allocation and compares evaluation scores with and without reward reweighting. The reweighted run moves ahead during training and maintains a score advantage in the reported evaluations.

A Analysis of Completion-Focused Rubric Reward

  • The analysis subtracts the rollout-group mean reward and examines the policy-reward gradient at the update origin, where the probability ratio is one and clipping is inactive.Reward normalization and group standard deviation are shared across rubric terms.Source: A Analysis of Completion-Focused… · p. 15
  • The update gives greater relative weight to rubric requirements the policy satisfies less reliably, shifting emphasis as completion gaps change.The positive floor λ retains every requirement; gradient-signal norms need not follow the same ordering, and the derivation concerns the policy-reward gradient at the update origin.Source: A Analysis of Completion-Focused… · p. 15
  • At step 140, the reweighted run scores approximately 0.76 versus 0.67 without reweighting, a gap of about 0.09.The reweighted run starts lower, moves ahead around step 50, and remains ahead thereafter; the pattern persists across multiple evaluations.Source: A Analysis of Completion-Focused… · p. 15, Figure 6 · p. 15

B Environment Quality and Selective World-Model Simulation

Spot checks and usability tests indicate that inspected synthesized environments generally support coherent task execution, but finite tests cannot capture the full complexity of real services. The paper also uses world-model simulation selectively for tools that are difficult to implement locally.

B Environment Quality and Selective World-Model Simulation

  • Finite tests cannot cover real services’ full complexity, including complex business rules, implicit dependencies, and uncommon boundary cases.The authors state that current coding agents cannot fully resolve this limitation.Source: B Environment Quality and Selective… · p. 16
  • Manual spot checks found tool responses and state changes generally consistent with intended service behavior and supportive of coherent task execution.Each deterministic implementation must pass automatically generated tests for basic scenarios such as normal invocation and error handling.Source: B Environment Quality and Selective… · p. 16
  • Downstream training gains indirectly indicate that synthesized environments provide effective and usable training signals, but do not prove every instance is completely accurate.The authors say large-scale evaluation of individual environments is difficult.Source: B Environment Quality and Selective… · p. 16
  • World-model simulation covers tools that cannot be directly mocked locally; 7.1% of tools rely on it, and about 10% of samples invoke at least one such tool.The authors caution that inaccurate simulated observations may still affect later judgments and decisions in affected workflows.Source: B Environment Quality and Selective… · p. 16

C Executable Environment Example

The Google Calendar example shows a typed event state and an executable update operation. Its observations expose both successful state changes and explicit failures, allowing a task rubric to inspect the resulting event state.

C Executable Environment Example

  • The example uses typed event records and a Google Calendar mock with an executable update operation to move a community meetup online.The tools retrieve or modify event fields and return structured observations.Source: C Executable Environment Example · p. 17
  • In the shown transition, the event location changes from Khangarid Hall to Online while its date and time remain unchanged, and the result is success: true.The passage labels the comparison as the observed state transition in the training sample.Source: C Executable Environment Example · p. 17
  • The update changes only supplied, supported fields, allowing the task rubric to inspect the resulting event state.The listed updatable fields are summary, description, location, start, and end.Source: C Executable Environment Example · p. 17
  • Missing event identifiers return error 400, while unknown events return error 404.The update operation checks for the required eventId and for whether the event exists.Source: C Executable Environment Example · p. 17

D A Complete Agent Trajectory

This trajectory connects community, roster, weather, calendar, and communication services to carry out a Frostbyte meetup runbook. It moves the meetup online because of snow, updates Reddit, creates a Teams invite for confirmed volunteers, and sends a wrap-up email.

D A Complete Agent Trajectory

  • The meetup task links Google Calendar, Simosphere’s runbook and volunteer roster, Reddit’s announcement thread, and Thinair Geo’s venue forecast.The workflow must account for swag-exempt volunteers and move the event online if snow is forecast, preserving its time and inviting confirmed volunteers.Source: D A Complete Agent Trajectory · pp. 18–19
  • The agent accesses real services indirectly through meta-tools that list services and tools, reveal argument schemas, and execute calls.The prescribed workflow is to list services, inspect relevant tools, describe each tool before use, and then call it with schema-matching arguments.Source: D A Complete Agent Trajectory · p. 18
  • The agent found the January 20 Frostbyte meetup event and searched the mailbox for Dara’s runbook email.The event directs the agent to follow the meetup runbook end to end and reply with a short wrap-up when done.Source: D A Complete Agent Trajectory · p. 26
  • After completing the runbook, the agent must reply to Dara’s email with a short summary of what it did.The message asks that the runbook be followed end to end before sending the wrap-up.Source: D A Complete Agent Trajectory · pp. 26–27
  • The runbook requires posting the pin count from the SharePoint Meetup Details list and replying to confirmed volunteers unless swag_exempt is Yes.The count belongs in a comment on the pinned r/frostbyte announcement; volunteer status is tracked where the post specifies.Source: D A Complete Agent Trajectory · p. 26
  • If the forecast through January 20 shows snow, the agent must move the meetup online; otherwise, it must keep it in person and post an address reminder.The snow branch also requires updating the announcement and calendar event and creating a shared-workspace calendar invite for confirmed volunteers; the in-person branch leaves the event unchanged.Source: D A Complete Agent Trajectory · pp. 26–27
  • The agent locates the announcement, SharePoint lists, and connected mailbox identity in parallel, then retrieves comments, roster entries, and meetup details.The roster and details are obtained through separate SharePoint list-item calls after the list names are discovered.Source: D A Complete Agent Trajectory · pp. 28–31
  • The Reddit announcement identifies Frostbyte’s January 20 meetup at Khangarid Hall in Ulaanbaatar and says confirmed volunteers receive anniversary pins.It also notes catering and lightning talks; a comment gives the venue coordinates and warns that the hill road closes if snow is forecast.Source: D A Complete Agent Trajectory · pp. 29, 31
  • The comments and roster identify Maya Chen and Jonas Weber as confirmed, Tomas Rivera as confirmed but swag-exempt, Sofia Marchetti as declined, and Priya Raman as pending.Maya says she will bring a collector demo; Jonas says he will take a lightning-talk slot.Source: D A Complete Agent Trajectory · pp. 31–32
  • The Meetup Details list reports 12 anniversary pins on hand, a swag shipping address in Ulaanbaatar, and a swag budget code.The retrieved values include Frostbyte Labs, 12 Narniin Zam 1, Ulaanbaatar 14200, Mongolia; the code is redacted in the passage.Source: D A Complete Agent Trajectory · pp. 32–33

A Complete Agent Trajectory

  • Day 4’s forecast showed snow with a 97% precipitation chance, so the agent moved the meetup online under the runbook.The forecast also returned a severe weather advisory for Khangarid Hall.Source: D A Complete Agent Trajectory · p. 36
  • Reddit actions posted the 12-pin count, reserved-pin replies to Maya and Jonas, and an online announcement preserving the meetup’s date and time.The returned observations confirm all four posts, including the announcement that confirmed volunteers would receive an online invite.Source: D A Complete Agent Trajectory · p. 38
  • The agent requested changing the Google Calendar event location to “Online” and began creating an online meeting invite for confirmed volunteers.The supplied passages show the update_event call and the start of a calendar_create action, but not their results.Source: D A Complete Agent Trajectory · pp. 38–39

D A Complete Agent Trajectory

  • The reply to Dara summarized the pin announcement, volunteer reservations and exclusions, weather change, and online meeting invite.Maya Chen and Jonas Weber had pins reserved; Tomas Rivera, Sofia Marchetti, and Alex Kim were skipped for the stated eligibility or confirmation reasons.Source: D A Complete Agent Trajectory · p. 40
  • The final report says the agent found the event and runbook, gathered pin, roster, and map-coordinate details, and posted the pin announcement and volunteer replies.It reports 12 anniversary pins on hand and reservations for Maya Chen and Jonas Weber.Source: D A Complete Agent Trajectory · p. 41
  • Snow prompted an online move: the calendar location was updated and a Teams invite was created for Jan 20, 18:30–21:00 UTC+8, with Maya, Jonas, and Tomas as attendees.The final report also says an online-move announcement was posted on the thread.Source: D A Complete Agent Trajectory · p. 41

Related papers

Loading 2609.33665v1…