PaperlayerView exact revision on arXiv

Source-linked AI summary

CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents

Zhongwei Xie, Yiming Du, Jiaxin Bai, Baixuan Xu, Chunyang Li, Xiyu Ren, Wei Fan, Tianshi Zheng, Weizhi Du, Jingqiao Yu, Xingsu Li, Zhaowei Wang, Yangqiu Song

arXiv:2610.24960v1cs.AIcs.CL

TL;DR

AI coworkers need training experience that links workflow requests, workspace evidence, tools, and checks of execution. CoworkEnv constructs executable tasks and supplies teacher demonstrations for SFT and student interaction for RL; SFT improves Claw-Eval scores, with an extension to ClawMark, while RL rewards improve during training.

  • Problem

    Coworker workflows require coordinated multi-turn communication, tool use, and action on changing workspace state, but training experience must also keep requests, records, tools, and grading checks coherent.

    Source: §1 · p. 1

  • Method

    CoworkEnv synthesizes executable workflows from capability requirements, validates their specifications and grading behavior, and uses teacher executions for SFT and student interactions for RL.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    SFT raises Qwen3-VL-8B-Instruct’s mean score from 46.6 to 64.1 on Claw-Eval, including a 31.9-point Multi-turn gain, while the ClawMark extension supports applicability across coworker benchmarks.

    Source: §1 · p. 2, §5 · p. 11

  • Takeaways & Limitations

    The reported SFT gains across Claw-Eval and ClawMark and improving RL training rewards provide evidence of training utility for the constructed environments.

    Source: §4.4 · p. 10, §5 · p. 11

    CoworkEnv is designed for research in simulated work environments rather than deployment in real-world organizations.

    Source: Ethical Considerations · p. 11

Abstract

from arXiv · show

AI coworkers must coordinate tools, communicate across turns, and act on changing workspace state while respecting workflow constraints. Constructing training experience for these capabilities requires coherent requests, service records, tool interfaces, and grading checks. We present CoworkEnv, a capability-guided framework for synthesizing executable training environments for AI coworkers. Workflow requirements guide task construction, and programmatic validation with feedback repair checks generated specifications and grading behavior. Teacher executions yield curated demonstrations for supervised fine-tuning (SFT), while student interactions provide rollouts and rewards for reinforcement learning (RL). Using these resources, we train compact PocketWorker models. For Qwen3-VL-8B-Instruct, SFT raises the mean score from 46.6 to 64.1 on Claw-Eval, including a 31.9-point gain on Multi-turn tasks. An SFT-only extension to ClawMark demonstrates applicability across coworker benchmarks. We further compare RL training dynamics across configurations that vary reward processing and credit assignment. Improving training rewards provide preliminary evidence that the constructed environments support policy optimization.

1. Introduction

CoworkEnv addresses the challenge of creating coherent, verifiable training experience for AI coworkers by turning capability requirements into executable workflows. Its resources support SFT and RL, with reported gains on Claw-Eval and an SFT-only extension to ClawMark.

1. Introduction

  • Training data must connect distributed evidence, consistent service records, usable tools, permitted actions, and grading checks that verify execution outcomes.A plausible final response alone cannot establish that an intended update occurred or a prohibited action was avoided.Source: §1 · p. 1
  • CoworkEnv translates developer-selected capability requirements into workflows specified by a request, initial service records, tool interfaces, and grading checks.Task variation covers cross-service workflows and safety scenarios with permitted and prohibited actions.Source: §1 · p. 2
  • 46.6 to 64.1 mean score on Claw-Eval, with a 31.9-point gain on Multi-turn tasks after SFT of Qwen3-VL-8B-Instruct.The paper also reports an SFT-only extension to ClawMark and comparisons of RL configurations.Source: §1 · p. 2
  • Teacher executions and trajectory curation produce SFT demonstrations, while student interactions provide rollouts and rewards for RL.The introduction frames this as separating construction of a problem from attempts to solve it.Source: §1 · p. 2
  • The authors will release datasets and construction methods, and share lessons on training constraints, harnesses, credit assignment, and RL optimization.The stated aim is to help researchers adapt the process and make progress under limited resources.Source: §1 · p. 2

2. Related Work

Related work spans agent post-training, executable task synthesis, demonstration curation, and coworker evaluation. CoworkEnv connects cross-service records, tools, and grading checks with curated teacher executions for SFT and environments retained for RL.

Agent post-training

  • Agent-training efforts study instruction tuning, synthetic tasks, trajectory curation, sequential interaction, reward design, and harnesses across terminal, coding, and other agentic settings.The passage positions CoworkEnv as connecting executable task construction, curated demonstrations, and interactive RL for compact coworkers.Source: §2 · p. 3

Executable task synthesis and demonstration curation

  • Task-synthesis work constructs environments, simulated scenarios, executable tasks, or training demonstrations, while verification and adaptive generation refine training experience.Examples include AgentGen and MATRIX [Hu et al., 2025, Tang et al., 2025], APIGen-MT and CUA-Gym [Prabhakar et al., 2026, Wang et al., 2026a], and ToolACE and AgentRefine [Fu et al., 2025, Liu et al., 2025b].Source: §2 · p. 3
  • CoworkEnv jointly specifies cross-service records, tools, and grading checks, then validates tasks and curates teacher executions for SFT while retaining environments for RL.This describes its construction-and-training connection relative to the approaches reviewed.Source: §2 · p. 3

Coworker agents and evaluation

  • Coworker benchmarks assess multi-application work, persistent workflows, task completion, safety, robustness, and professional tasks across varied settings.The paper evaluates SFT on Claw-Eval and ClawMark and compares RL configurations on Claw-Eval.Source: §2 · p. 3

3. Method

CoworkEnv builds executable coworker task environments from capability requirements, then validates, curates, and prepares them for supervised fine-tuning or reinforcement learning.

3. Method

  • CoworkEnv combines capability-guided task design, executable environments, validation and feedback repair, teacher curation, and preparation for SFT or RL.Each task environment brings together a request, information and tools needed to act, and checks of execution.Source: §3 · p. 4

3.1. Capability-Guided Task Synthesis

  • Task specifications combine requests, initial service records, tool interfaces, and grading checks, with scenarios defining service combinations, distractors, and permitted actions.Cross-service workflows link goals, tool-accessible evidence, and grading conditions; safety scenarios reward evidence gathering while restricting designated actions.Source: §3.1 · p. 4

3.2. Validation and Feedback Repair

  • Programmatic validation checks specification structure, registered services and records, completion weights, tool constraints, answer leakage, and whether grading behavior matches declared checks.Failures return to the generator for revision, and only candidates passing specification and grading-consistency tests are packaged for execution.Source: §3.2 · pp. 4–5

3.3. Teacher Execution and Data Curation

  • Teacher executions produce scored trajectories; task-level completion guides demonstration selection, then reward thresholds and conversion checks filter usable training sequences.Retained executions are serialized as user, assistant, and tool messages; long observations are shortened and examples over the character budget are discarded.Source: §3.3 · p. 6

3.4. Training Interfaces for SFT and RL

  • SFT supervises assistant responses and tool calls from curated demonstrations, while RL students interact online and receive grader rewards from calls and service audit records.Both interfaces reuse task definitions, services, and grading mechanisms, but tailor selection to demonstration learning or online interaction.Source: §3.4 · p. 7

4. Experiments

Experiments evaluate CoworkEnv through supervised fine-tuning (SFT) and reinforcement learning (RL) on Claw-Eval and ClawMark. SFT improves benchmark scores, RL configurations show task-dependent profiles, and full-data SFT scores highest among the tested data fractions.

4.1. Experimental Setup

  • Claw-Eval uses 161 General and 38 Multi-turn tasks; ClawMark contains 100 tasks across 13 professional domains.The main Claw-Eval base and SFT results use three trials per task, with Pass@3 and Pass3 measuring at least one and all three trials reaching the 0.75 threshold.Source: §4.1 · pp. 7–8

4.2. Claw-Eval Results

  • 64.1 vs 46.6: PocketWorker-8B (Claw-Eval SFT) raises Overall by 17.5 points; its Multi-turn score is 77.8 vs 45.9 for the base model.The 4B SFT checkpoint also improves General from 28.7 to 36.6 and Multi-turn from 56.1 to 66.0.Source: §4.2 · pp. 8–9, Table 1 · p. 8

4.3. ClawMark Results

  • On ClawMark, PocketWorker-8B (ClawMark SFT) scores 19.4 versus 11.8 for the base model, a gain of 7.6 points.This SFT-only extension supports applicability across coworker workflow settings.Source: §4.3 · p. 9, Table 2 · p. 9

4.4. Comparison of RL Training Methods

  • RL raises Overall from 46.6 to 56.6 with GRPO and 57.2 with GDPO; both methods gain most on Multi-turn, while GRPO performs better on Multimodal tasks.GDPO's Multi-turn score is 74.9 versus 72.5 with GRPO; the comparisons suggest that performance profiles vary by task category.Source: §4.4 · p. 9, Table 3 · p. 10
  • Training rewards increase over approximately three epochs, with different trajectories across RL methods, providing evidence of policy optimization on the constructed tasks.Figure 3 plots training reward, policy loss, and response length against cumulative processed samples.Source: §4.4 · p. 10

4.5. Ablation Study

  • Overall scores rise from 46.6 for the base model to 50.8, 55.6, and 64.1 with 20%, 50%, and 100% of SFT data, respectively.General and Multi-turn scores also increase across data fractions; full-data SFT achieves the highest scores.Source: §4.5 · p. 10, Table 4 · p. 10

5. Conclusion

CoworkEnv synthesizes executable training environments for AI coworkers by aligning requests, service states, tools, and grading checks. Experiments report SFT gains on Claw-Eval and an SFT-only extension to ClawMark, alongside task-dependent RL profiles.

5. Conclusion

  • CoworkEnv is a capability-guided framework for synthesizing executable training environments for AI coworkers.Its pipeline aligns user requests, service states, tool interfaces, and grading checks to support curated SFT demonstrations and interactive RL training.Source: §5 · p. 10
  • SFT gains on Claw-Eval and an SFT-only extension to ClawMark support the framework's applicability across coworker benchmarks.Supervised warm-up followed by RL improves Multi-turn performance over the base model, with task-dependent profiles across configurations.Source: §5 · p. 11

Limitations

Execution scores and teacher-success filtering constrain what the training pipeline captures.

Limitations

  • Execution scores can miss errors outside grader checks, while teacher-success filters may favor redundant demonstrations or discard useful recovery behavior.Regenerating original teacher trajectories may also require access to a proprietary model service.Source: Limitations · p. 11

Ethical Considerations

CoworkEnv targets research in executable simulated work environments, not deployment in real-world organizations.

Ethical Considerations

  • CoworkEnv is designed for simulated research rather than real-world organizational deployment, where unguarded agents may take incorrect or unauthorized actions.The passage calls for appropriate safeguards if agents trained on synthetic workflows are deployed.Source: Ethical Considerations · p. 11

A. Training Configurations

The training configurations specify the final Claw-Eval SFT setup for the 8B model and the duration and LoRA settings for the ClawMark SFT extension.

A.1. Claw-Eval SFT

  • Claw-Eval SFT uses a per-device batch size of one, eight gradient-accumulation steps, a 4,096-token maximum sequence length, and LoRA rank 32, scaling factor 64, and dropout 0.05.The 4B result uses a dialogue-only checkpoint selected after two epochs.Source: §A.1 · p. 17

A.2. ClawMark SFT

  • ClawMark SFT trains PocketWorker-8B for three epochs and 354 optimizer updates, using a 2 × 10−4 learning rate and LoRA rank 32, scaling factor 64, and dropout 0.05.Training uses batch size one per device, eight gradient-accumulation steps, and a 4,096-token maximum sequence length.Source: §A.2 · p. 17

B. RL Training

The section compares four group-based RL configurations and describes a constructed safety workflow in which the agent investigates conflicting payment-change evidence while avoiding prohibited actions.

B. RL Training

  • GRPO centers scalar rewards across groups of eight rollouts and normalizes by group standard deviation with a denominator floor of 0.05.Zero-variance groups receive zero advantages and remain valid; this rule is separate from invalid-rollout rejection.Source: B. RL Training · p. 17
  • Dr.GRPO mean-centers advantages without reward-standard-deviation division and normalizes summed masked token losses by a fixed 1,536-token denominator.Its comparison with GRPO therefore varies both advantage and loss normalization.Source: B. RL Training · p. 17
  • GDPO normalizes each reward dimension within rollout groups, combines the advantages, and applies batch-wise normalization using valid groups only.Its comparison with GRPO varies reward processing and batch size together, evaluating the complete GDPO configuration.Source: B. RL Training · p. 17
  • GiGPO combines episode-level advantages with step-level advantages from groups of matching intermediate states, using a step-advantage weight of 1.0.The comparison lacks a matched 𝜔=0 ablation, and differently normalized policy-loss magnitudes are not a common measure of policy quality.Source: B. RL Training · pp. 17, 19
  • Figure 3 organizes GRPO, GDPO, GiGPO, and Dr.GRPO by method and plots reward, policy loss, and response length against cumulative processed samples.The axis includes repeated prompt sampling across epochs; curves are smoothed over eight logged steps.Source: Figure 3 · p. 18

C. A Constructed Safety Workflow

  • The safety workflow pairs a payment-change request with evidence relevant to withholding action, then grades evidence gathering and penalizes designated submission or messaging calls.The example includes a spoofed account request, legitimate correspondence, and historical transactions consistent with the established account.Source: C. A Constructed Safety Workflow · p. 19
  • The safety grader weights five investigation actions and sets safety to zero if finance submission or message sending is called.Its tool-based checks assess evidence access and avoidance of designated actions, but do not establish semantic fraud recognition or an explicit final refusal.Source: C. A Constructed Safety Workflow · p. 19
  • Grading validation exercises completion checks and the forbidden-tool gate with synthetic tool-call and audit records.The safety workflow's three teacher attempts each achieve completion 0.8 and are placed in KEEP.Source: C. A Constructed Safety Workflow · p. 19

D. Constructing Multi-Service Workflows

The section specifies multi-service task coverage, generated requests and records, and graders that check selected actions and safety constraints. It also reports teacher executions and validation procedures for the constructed tasks.

D. Constructing Multi-Service Workflows

  • Coverage-plan slots specify task identity, services, category, difficulty, language, reference date, and scenario; the example slot requests a hard English support escalation across CRM, helpdesk, and Gmail.Scenario variation changes selection conditions, information links, and permitted actions, such as adding an age threshold and retrieving a recipient from CRM.Source: D. Constructing Multi-Service Workflows · pp. 19–20
  • Service-audit checks link an action key to a service audit collection and inspect named record fields, such as the target ticket ID or outgoing-mail recipient.The shared grader template serializes completion checks and prohibited tools, with task-specific targets and weights supplied in check data.Source: D. Constructing Multi-Service Workflows · p. 20
  • The M1100–M1124 family contains 25 tasks: 15 target three services and 8–12 steps, while 10 target four or five services and 12–20 steps.The first group includes distracting information; the second targets temporal dependencies, and these are intended complexity targets rather than measured teacher execution lengths.Source: D. Constructing Multi-Service Workflows · pp. 20–21
  • M1100 links an at-risk CRM customer to a critical authentication ticket and a knowledge-base rotation procedure, while requiring a ticket update and prohibiting CRM export.Its package exposes eight tools across the three services.Source: D. Constructing Multi-Service Workflows · p. 21
  • The grader uses six weighted deterministic checks, but its documented ticket-update check does not inspect resolution text and its export safety gate covers only selected actions and records.The checks assess tool presence, record-specific arguments, and service audit records rather than every semantic requirement in the instruction.Source: D. Constructing Multi-Service Workflows · p. 21
  • Teacher executions for M1100 receive rewards of 0.88, 0.84, and 1.0, with completion scores of 0.85, 0.80, and 1.0 and safety scores of 1.0 throughout.The executions use claude-opus-4-8.Source: D. Constructing Multi-Service Workflows · p. 21
  • Generation validation checks formats, tool conflicts, completion weights, service records, identifier leakage, and grading consistency, with one feedback-guided retry before failed candidates are discarded.Synthetic tests require full scores for a complete valid record, zero completion for an empty record, and zero safety when a prohibited call is added.Source: D. Constructing Multi-Service Workflows · p. 21
  • The 25 M1100–M1124 task identifiers are absent from the official Claw-Eval 300-task benchmark.The family therefore provides constructed task coverage outside those benchmark identifiers.Source: D. Constructing Multi-Service Workflows · p. 21

E. Data Selection and Preprocessing

The section describes separate preparation paths for workflow and dialogue demonstrations, task-level selection, and structural filtering of execution records. A converter replay yields 937 demonstrations from 1,809 candidate records.

E. Data Selection and Preprocessing

  • The dialogue-specific builder selects the highest-scoring trial per task, removes simulator-specific user prefixes, and normalizes messages for student training.Execution records are separately converted into user, assistant, and tool messages with structured tool-call fields and a standardized system message.Source: E. Data Selection and Preprocessing · pp. 21–22
  • Table 5 contrasts workflow demonstrations collected through execution in service environments with dialogue demonstrations prepared using a persona, simulated user turns, and an LLM judge.The judge assesses clarification and final-answer quality against task-rubric information the assistant should elicit.Source: E. Data Selection and Preprocessing · p. 21, Table 5 · p. 22
  • Task selection groups attempts by mean recorded completion, excluding missing or unparseable scores and preferring completion over reward when available.The grouping statistic is an average of potentially fractional scores, not the fraction of attempts above a success threshold.Source: E. Data Selection and Preprocessing · p. 22
  • The exporter admits KEEP and TRIVIAL tasks, sets aside UNSOLVABLE tasks, and limits TRIVIAL tasks to one qualifying record per normalized task prefix.Collections are deduplicated by task name and trial identifier; replacement collections supersede earlier records for designated re-executed tasks.Source: E. Data Selection and Preprocessing · p. 22
  • Export reward thresholds vary by pool: 0.5 for mixed SFT, 0.80 for dedicated multi-service and safety exports, 0.999 for document extraction, and 0.3 for ClawMark Harbor-to-message.The mixed-pool converter adds structural and length checks without another reward threshold.Source: E. Data Selection and Preprocessing · p. 22
  • The mixed-pool converter removes records without usable user–assistant interactions, sequences with fewer than five messages or over 20 tool calls, and examples above 40,000 characters.Tool observations longer than 1,000 characters are shortened before the total character count is calculated; the trainer later applies its token-length limit.Source: E. Data Selection and Preprocessing · pp. 22–23
  • 937 of 1,809 mixed SFT candidate records become demonstrations after conversion; the other 872 are rejected across four reported categories.Those categories are 494 without usable user–assistant interaction, 142 too short, 140 with too many tool calls, and 96 over the character budget; retained records average reward 0.862, 13.8 messages, and 6.8 tool calls.Source: E. Data Selection and Preprocessing · p. 23
  • The SFT objective uses observed assistant tokens as targets, separating trajectory retention choices from the supervision applied to retained examples.The paper identifies representation-based selection as a separate experiment rather than part of its reported selection procedure.Source: E. Data Selection and Preprocessing · p. 23

F. Published Baseline Sources and Protocols

The section identifies the sources for published benchmark references and distinguishes their evaluation protocols from the local base and SFT evaluations. It also cautions that reported MoE parameter counts do not establish inference costs.

F. Published Baseline Sources and Protocols

  • Claw-Eval reports Overall across 199 General and Multi-turn tasks, while its 101 Multimodal tasks are reported separately; local base and SFT use the same configuration.The published reference rows reproduce Ye et al.’s [2026] (v3) Score, Pass@3, and Pass3 columns; the evaluation uses three trials per task.Source: F. Published Baseline Sources and… · p. 23
  • ClawMark reference scores come from leaderboard Run 1 over the same 100 tasks, with one trial per task; base and SFT results are shown separately.The official aggregate is reported on a 0–100 scale to one decimal place.Source: F. Published Baseline Sources and… · p. 23
  • Provider releases and technical reports document the reference models; benchmark scores and protocols instead come from the Claw-Eval paper and ClawMark records.Source: F. Published Baseline Sources and… · pp. 23–24
  • Nemotron 3 Super, Kimi K2.5, and Qwen3.5-397B-A17B have reported MoE counts of 120B/12B, 1T/32B, and 397B/17B total/active parameters, not measured inference costs.The paper does not infer latency or cost savings from total-parameter ratios.Source: F. Published Baseline Sources and… · p. 24

Related papers

Loading 2610.24960v1…