PaperlayerView exact revision on arXiv

Source-linked AI summary

WEFT: Scaling Tool-Use Post-Training for General-Purpose Agents

Bo Mao, Hang He, Linting Wang, Lizhi Lin, Maosen Zhou, Guanming Liu, Jinxiu Liu, Tianyu Huai, Chaoyun Zhang, Bingxuan Li, Kepeng Lei, Guanting Dong, Zhou Shao, Rui Zheng, Hang Yan, Jie Zhou, Chengcheng Wan, Tao Gui, Liang He, Xipeng Qiu

arXiv:2609.36887v1cs.AI

TL;DR

Scaling executable environments alone does not ensure reliable learning signals, because outcomes depend on interactions among the environment, task, agent harness, and evaluator. WEFT scales the agentic interaction system, uses execution traces and state evidence to attribute failures and revise components, and supports stable post-training with prefix-preserving sampling, atomic-turn credit assignment, and MegaMCP. WEFT-8B and WEFT-14B outperform all evaluated matched-size environment-scaling baselines on BFCL V4, τ 2-Bench, and Claw-Eval; WEFT-14B exceeds Agent-World-14B by 6.41, 2.23, and 12.27 percentage points, respectively.

  • Problem

    Scaling executable environments alone does not ensure reliable learning signals, because outcomes depend on interactions among the environment, task, agent harness, and evaluator.

    Source: §1 · p. 2

  • Method

    WEFT scales the agentic interaction system, uses execution traces and state evidence to attribute failures and revise components, and supports stable post-training with prefix-preserving sampling, atomic-turn credit assignment, and MegaMCP.

    Source: §1 · pp. 2–3

  • Results

    WEFT-8B and WEFT-14B outperform all evaluated matched-size environment-scaling baselines on BFCL V4, τ 2-Bench, and Claw-Eval; WEFT-14B exceeds Agent-World-14B by 6.41, 2.23, and 12.27 percentage points, respectively.

    Source: §4.2 · p. 10

  • Takeaways & Limitations

    The findings support system-level scaling for reliable tool-use agents, with execution experience serving both policy learning and continual interaction-system improvement.

    Source: §5 · p. 13

Abstract

from arXiv · show

Recent efforts to scale tool-use post-training have largely centered on the synthesis of executable environments, which constitute only one component of a broader agentic interaction system comprising the environment, task, agent harness, and evaluator. Scaling environments in isolation, however, does not guarantee commensurate gains in model performance, because reliable learning signals depend on coherent interactions among all components of the agentic interaction system. To address this problem, we introduce WEFT (Whole-system Evolution For Tool-use Post-training), which couples scalable agentic interaction system construction, execution-driven self-evolution, and stable post-training. WEFT scales agentic interaction system construction across environment breadth, task complexity, and interaction diversity. Execution-driven self-evolution iteratively uses execution traces and state evidence to attribute failures and revise the responsible components, with fresh rollouts evaluating the changes and providing evidence for subsequent evolution rounds. For stable post-training at scale, WEFT addresses both optimization and execution reliability: prefix-preserving sampling retains verified progress and atomic-turn credit assignment localizes learning signals, while MegaMCP maintains isolated, recoverable state across concurrent rollouts over shared tool services. Extensive experiments across various models and benchmarks demonstrate the effectiveness of WEFT for tool-use post-training. WEFT-8B and WEFT-14B outperform all evaluated matched-size environment-scaling baselines on BFCL V4, $\tau^2$-Bench, and Claw-Eval. In particular, WEFT-14B improves over Agent-World-14B by 6.41, 2.23, and 12.27 percentage points. WEFT-35B-A3B further extends these gains to more challenging long-horizon workflow benchmarks, including Toolathlon-Verified and AutomationBench.

1 Introduction

Scaling executable environments alone does not ensure reliable learning signals, because tool-use outcomes depend on interactions among the environment, task, agent harness, and evaluator. WEFT jointly improves these components and uses execution experience for both policy learning and system revision.

1 Introduction

  • Scaling executable environments alone does not ensure reliable learning signals, since failures can stem from faulty tools, infeasible tasks, or verification errors rather than model limitations.Misattributing these failures to the policy can penalize valid behavior and introduce misleading learning signals.Source: §1 · p. 2
  • WEFT couples scalable agentic interaction system construction, execution-driven self-evolution, and stable post-training.It uses execution traces and state evidence to distinguish policy failures from environment, task, or verifier issues, then uses fresh rollouts to assess targeted revisions.Source: §1 · p. 2
  • WEFT-8B and WEFT-14B outperform all evaluated matched-size environment-scaling baselines across BFCL V4, τ²-Bench, and Claw-Eval; WEFT-35B-A3B reaches 45.99% on Toolathlon-Verified and 26.50% on AutomationBench.With the task set and rollout budget fixed, three self-evolution rounds reduce tool-call errors in selected training trajectories by 45.5% and improve three downstream benchmark scores by 3.65–5.25 percentage points.Source: §1 · p. 2
  • WEFT supports stable post-training with prefix-preserving rejection sampling, verifier-reliability task filtering, atomic-turn credit assignment, and MegaMCP’s isolated, recoverable rollout state.Prefix-preserving sampling retains verified progress; MegaMCP uses snapshots to support retries and candidate sampling from identical states.Source: §1 · pp. 2–3

2 Related Work

Prior work scales environments and tasks, studies interaction and rollout systems, and develops self-improvement and post-training methods. WEFT combines these concerns by composing cross-MCP tasks, varying interaction formats and harnesses, and attributing failures to the components that need revision.

Environment and task scaling

  • WEFT composes execution-verified atomic tasks across MCPs through dependencies, entity bindings, and grounded initial state to support a shared long-horizon goal.Prior environment and task work spans multiple domains and uses execution traces, topology-aware sampling, and structured dependencies to support complex interactions.Source: §2 · p. 3

Agent interaction and rollout systems

  • WEFT presents the same grounded task through distinct disclosure formats and native harnesses, while MegaMCP shares tool services across rollouts with isolated, recoverable state.Prior work covers API invocation, interleaved reasoning and action, user interaction, harness design, mixed-harness rollouts, and decoupled execution and optimization.Source: §2 · p. 3

Self-improvement

  • Unlike behavioral and task-adaptation approaches, WEFT attributes execution failures to the components needing revision, distinguishing policy failures from defects in environments, tasks, or verifiers.Prior system-level approaches search over agent programs, revise harnesses using traces, or coordinate synthesis, auditing, and repair.Source: §2 · p. 3

Tool-use post-training

  • WEFT preserves verified prefixes across local failures and applies rejection sampling at atomic-task boundaries when collecting trajectories.For reinforcement learning, it compares candidate atomic turns from identical histories and environment states, using the current task’s binary completion outcome rather than terminal returns.Source: §2 · pp. 3–4

3 Method

WEFT scales the full agentic interaction system through stateful environments, verifiable cross-MCP tasks, varied views and harnesses, and execution-driven revision. It combines these with progress-preserving SFT, atomic-turn RL, and MegaMCP’s isolated, recoverable rollout execution.

3.1.1 Stateful Environment Synthesis

  • Stateful environments are synthesized from tool definitions, schemas, and contracts, then checked with generated calls and revised when structural, mapping, runtime, or state-access errors arise.Passing backends become independently executable MCP servers.Source: §3.1.1 · p. 4

3.1.2 Cross-MCP Task Composition

  • Certified atomic tasks are composed into dependency-grounded cross-MCP workflows with resettable initial states and validated completion conditions.Atomic tasks are independently executable and verifiable units; composition graphs encode information and state dependencies.Source: §3.1.2 · pp. 4–5, Figure 2 · p. 5

3.1.3 Multi-View and Multi-Harness Rollouts

  • Task views vary disclosure while preserving objectives and state, and ReAct, OpenClaw, and Hermes provide different harnesses for sandboxed rollouts.ReAct supports both Agentic and SimUser views; OpenClaw and Hermes use Agentic mode.Source: §3.1.3 · p. 5

3.2 Execution-Driven Self-Evolution

  • Execution traces and state evidence support verification and failure attribution, while fresh rollouts assess targeted revisions and inform subsequent evolution rounds.Environment, task, and verifier issues are routed to the responsible components; policy failures on valid tasks leave tasks unchanged.Source: §3.2 · pp. 5–6, §3.2.2 · p. 6

3.2.1 Verification from Execution Evidence

  • Verification checks atomic-task completion against state changes and tool returns, using deterministic checks and a rubric tested and revised with execution feedback.Trajectory scores average binary checkpoint outcomes covering atomic tasks and the overall objective.Source: §3.2.1 · p. 6

3.3 Stable Tool-Use Post-Training

  • WEFT combines prefix-preserving rejection sampling, task filtering, atomic-turn credit assignment, and MegaMCP to support stable post-training on constructed interactions.MegaMCP supplies scalable rollout support for both SFT and RL.Source: §3.3 · p. 6, §3.3.3 · p. 7

3.3.1 Prefix-Preserving Rejection Sampling

  • SFT rejection operates at the atomic-task-turn level, retaining successful traces and states and resampling only a failed task after restoring its pre-turn state.Once all tasks pass, the retained segments form a complete trajectory.Source: §3.3.1 · p. 6

3.3.2 Reinforcement Learning with Atomic-Turn Credit Assignment

  • RL filters tasks using agreement between rubric-based and executable scores, then samples atomic-task turns from identical states and applies group-relative credit to current-turn tokens.Deterministic checks remove positive credit for flagged undesirable turns while leaving unflagged turns, including later recovery, with their advantages.Source: §3.3.2 · pp. 6–7

4 Experiments

The experiments evaluate WEFT across tool-use benchmarks and analyze task and harness diversity, execution-driven self-evolution, training methods, and rollout infrastructure. WEFT-14B outperforms matched-size environment-scaling baselines, while WEFT-35B-A3B also scores on more demanding workflow benchmarks.

4.1 Experimental Setup

  • Training uses 11,884 composed tasks for SFT and 1,954 filtered tasks for RL, with evaluation on three main benchmarks and additional workflow benchmarks.The constructed resources include 8,172 executable MCPs, 64,755 tools, and 41,695 certified atomic tasks.Source: §4.1 · p. 9

4.2 Main Results

  • WEFT-14B beats Agent-World-14B by 6.41, 2.23, and 12.27 percentage points on BFCL V4, τ 2-Bench, and Claw-Eval.WEFT-35B-A3B scores 45.99% on Toolathlon-Verified and 26.50% on AutomationBench.Source: Table 1 · p. 10, §4.2 · p. 10

4.3 Understanding the Post-Training Gains

  • With tasks fixed, combining SimUser and Agentic views beats either alone, and adding OpenClaw and Hermes yields a further 9.71-point mean gain.The gains are measured under benchmark-native harnesses.Source: §4.3 · p. 10, Table 2 · p. 11
  • Across three self-evolution rounds, tool-error rate falls from 1.76% to 0.96%, while Toolathlon-Verified, AutomationBench, and Claw-Eval improve by 5.25, 4.33, and 3.65 points.The task set and rollout budget are fixed; Toolathlon-Verified continues improving in the third round.Source: Figure 6 · p. 11, §4.3 · p. 11
  • At three retries, prefix-preserving sampling gains 1.85 points on Toolathlon-Verified and 2.17 on AutomationBench over end-to-end sampling, with Claw-Eval nearly tied.Accepted-trace count is fixed, so the gains concern the training value of SFT data rather than more accepted examples.Source: Figure 7 · p. 11, §4.3 · p. 11
  • Atomic-turn GRPO improves all three benchmarks at both model sizes and exceeds trajectory-level GRPO by 5.69 and 4.43 points on τ 2-Bench.Trajectory-level GRPO decreases τ 2-Bench scores relative to SFT by 3.17 and 0.84 points for 8B and 14B, respectively.Source: §4.3 · p. 11, Table 3 · p. 12
  • For WEFT-14B, mean trajectory reward rises overall while policy entropy declines during training.The reported trajectory reward averages binary checkpoint outcomes; training advantages use the current atomic task’s binary reward.Source: Figure 8 · p. 12, §4.3 · p. 12
  • At 1,000 tasks, MegaMCP cuts sandbox upload from 773.5 to 34.8 MiB; shared processes also reduce resident memory by 77.6% or 95.7%, depending on configuration.With both systems cold at 100 tasks, median time to first tool call falls from 10.455 to 4.794 seconds.Source: Figure 9 · p. 12, §4.3 · p. 12

5 Conclusion

WEFT frames tool-use post-training as a systems problem, giving execution experience a dual role in policy learning and interaction-system improvement. Its experiments support system-level scaling for reliable tool-use agents.

5 Conclusion

  • WEFT treats scaling tool-use post-training as a systems problem rather than simply increasing the number of environments.Source: §5 · p. 13
  • Execution experience serves both as policy-training data and as evidence for improving the interaction system that produces it.Source: §5 · p. 13
  • The reported findings support system-level scaling for reliable tool-use agents.Source: §5 · p. 13

A.1 Atomic-Task Construction and Certification

Atomic-task construction certifies executable tasks, removes redundant tool chains, and tracks retained-task tool coverage through iterative synthesis and validation.

A.1 Atomic-Task Construction and Certification

  • E2E validation runs each task from an independently initialized state and checks returned observations and persistent state against completion conditions.Passing tasks are retained; failure diagnostics accumulate for repair.Source: §A.1 · p. 19
  • Tasks are redundant when one ordered tool chain is a subsequence of another; filtering removes the strictly contained chain and retains the earlier task for identical chains.Order is preserved but contiguity is not required, and sharing tools alone does not establish redundancy.Source: §A.1 · p. 19
  • Construction filters validated tasks for orthogonality, measures coverage as the retained chains’ distinct tools divided by all MCP tools, and revises synthesis using failure, redundancy, and coverage feedback.It stops when target coverage is reached, the round budget is exhausted, or no new candidates remain.Source: §A.1 · p. 19

A.2 Cross-MCP Task Composition

Cross-MCP composition controls workflow difficulty with profiles and broadens represented capabilities through coverage-aware selection and semantic checks for coherent shared objectives.

A.2 Cross-MCP Task Composition

  • Five difficulty profiles specify ranges for atomic-task count, MCP count, and domain breadth, along with a target corpus proportion.These dimensions control task length, services coordinated, and business-context breadth.Source: §A.2 · p. 19
  • Largest-remainder allocation turns profile proportions into quotas, while coverage-aware sampling favors underrepresented capabilities and ranks MCP combinations by shared entities, complementary operations, and functional redundancy.The profile mixture controls corpus-level difficulty; coverage-aware selection broadens capabilities within each profile.Source: §A.2 · p. 20
  • A semantic evaluation model checks whether selected MCPs support a coherent shared objective, and rejected combinations are resampled within the same profile.Composition requires surplus candidate atomic tasks because not every local objective fits a shared scenario; ρ supplies the proportional capacity surplus.Source: §A.2 · p. 20

B.1 Training Configuration

Training configures SFT and RL around turn-level sampling and optimization, with token alignment, selective credit masking, and rollout replenishment addressing training and execution details.

B.1 Training Configuration

  • SFT trajectories use prefix-preserving rejection sampling with up to three retries per atomic-task turn, allowing four attempts including the initial attempt.Kimi K3 generates the trajectories.Source: §B.1 · p. 20
  • RL uses 1,954 selected task–verifier pairs for both model sizes, chosen from 5,000 candidates using agreement between rubric-judge and executable-verifier scores across 16 pilot rollouts per task.The 8B and 14B models use atomic-turn GRPO with a fixed reference policy at each model’s RL initialization checkpoint.Source: §B.1 · p. 20
  • TITO preserves sampled token IDs and rollout log-probabilities without retokenizing trajectories; TIS weights policy loss by backend-to-rollout probability ratios and masks weights outside [0.5, 2.0].Prompts, tool observations, and inserted template tokens are excluded from loss; the TIS mask also applies to KL.Source: §B.1 · pp. 20–21
  • For flagged assistant turns, selective credit masking zeros positive advantages on generated tokens while retaining negative advantages; dynamic sampling replenishes batches after discarding incomplete or constant-reward groups.Original rewards, subsequent recovery turns, and KL remain unchanged; physical retries address recoverable infrastructure failures, not low-reward outcomes.Source: §B.1 · p. 21

B.2 Evaluation Details

Evaluation uses each benchmark’s native harness and official aggregate metric, with the main comparison and RL study spanning three benchmarks and data-construction ablations spanning another set.

B.2 Evaluation Details

  • The main comparison and RL study use BFCL V4, τ²-Bench, and Claw-Eval, while data-construction ablations use Toolathlon-Verified, AutomationBench, and Claw-Eval.Evaluation follows each benchmark’s native harness and official aggregate metric.Source: §B.2 · p. 21
  • Qwen3.5-35B-A3B’s BFCL V4 and τ²-Bench scores come from its official model card, while its Claw-Eval average comes from the Qwen3.6-35B-A3B model-card comparison column.These are the external baseline results described for Table 1.Source: §B.2 · p. 21
  • For τ²-Bench, Qwen follows the official setup except in the airline domain, where it applies fixes specified in the Claude Opus 4.5 system card.Source: §B.2 · p. 21

C Additional Experimental Results

Atomic-turn GRPO outperforms trajectory-level GRPO on every benchmark at both tested scales. The training-curve statistic instead aggregates reconstructed full-task trajectories, whereas training advantages use atomic-task rewards.

C.1 Credit-Assignment Granularity

  • Atomic-turn GRPO improves every benchmark at both scales over trajectory-level GRPO, which assigns one trajectory-level advantage to all turns.Atomic-turn advantages are computed within groups using binary task rewards.Source: §C.1 · p. 21

C.2 RL Training Curves

  • Each 14B training step samples 16 tasks and reports rewards from 256 complete root-to-leaf trajectories reconstructed from the final rollout layer.The 256 trajectories comprise 16 per task.Source: §C.2 · p. 21
  • The reward curve averages binary checkpoint outcomes across reconstructed trajectories, task groups, and the batch, while training advantages use the current atomic task’s binary reward.Each trajectory’s reward is the mean of its binary checkpoint outcomes.Source: §C.2 · p. 21

D MegaMCP Serving Evaluation

MegaMCP shares MCP servers while isolating mutable state by session, and the evaluation compares it with local serving across cold-start and rollout-scale workloads. The results show reduced cold-start latency and data transfer, alongside a provisioning cost at rollout scale.

Serving evaluation setup

  • The evaluation used identical task traces and tool interfaces; local serving packaged MCP source, initialized databases, and server processes per sandbox, unlike MegaMCP’s shared servers.The two configurations also used the same initial states, planned calls, and end-of-task cleanup.Source: D MegaMCP Serving Evaluation · p. 21

Provisioning data

Latency and lifecycle completion

  • 54.1% lower median time to first tool call, 23.3% lower median end-to-end latency, and 3.3% lower total wall time under matched cold-start timing.Both configurations completed every task lifecycle.Source: Table 6 · p. 22, D MegaMCP Serving Evaluation · p. 22
  • At 1,000 tasks, MegaMCP completed all lifecycles but had higher latency than prewarmed local servers, with session provisioning included in its timed interval.The larger workload comprised 4,000 sessions and 39,864 planned calls, with up to 1,000 calls in flight.Source: D MegaMCP Serving Evaluation · pp. 21–22

Historical cold and warmed runs

Resident memory

Loading 2609.36887v1…