PaperlayerView exact revision on arXiv

Source-linked AI summary

WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution

Hang He, Linting Wang, Guanming Liu, Bo Mao, Jinxiu Liu, Yiran Suo, Maosen Zhou, Zhiheng Xi, Guanting Dong, Zhou Shao, Rui Zheng, Jie Zhou, Chengcheng Wan, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang

arXiv:2610.18067v1cs.AIcs.LG

TL;DR

Existing pipelines offer limited support for revising cowork environments while keeping work contexts, tasks, and verifiers aligned, and terminal rewards give little guidance about which turns made progress. WorkRover combines harness-driven synthesis, recursive repair, and verifier-grounded turn credit; its trained model exceeds its backbone across five benchmarks.

  • Problem

    Existing pipelines offer limited support for repeated component revisions that keep work contexts, tasks, and verifiers aligned, while terminal rewards provide little guidance about turn-level progress.

    Source: §1 · p. 2, §3.3 · p. 6

  • Method

    WorkRover jointly constructs aligned work contexts, executable tasks, and evidence-grounded verifiers, uses dependency-closed replay for repairs, and assigns bounded progress credit to prerequisite turns.

    Source: §1 · pp. 2–3

  • Results

    WorkRover-35B-A3B exceeds Qwen3.5-35B-A3B across five benchmarks, gaining 29.32 percentage points on OfficeQA Pro, 26.27 percentage points on JobBench, and 413.25 Elo points on GDPval v2.

    Source: §1 · p. 2, §4.2 · p. 8

  • Takeaways & Limitations

    The findings support recursive environment evolution as a practical foundation for reliable long-horizon cowork-agent training.

    Source: §5 · p. 10

    VTCA's shaping signals depend on checker coverage and dependency-record quality.

    Source: §2 · p. 4

Abstract

from arXiv · show

Scaling interactive environments is important for training long-horizon agents through reinforcement learning. These environments often require further testing and refinement after initial generation, and correcting defects in a task may also require updating its work context and verifier. However, existing construction pipelines offer limited support for such iterative refinement while keeping these components aligned. We introduce WorkRover, a framework for training long-horizon cowork agents through recursive environment evolution. Its harness-driven synthesis enables construction agents to jointly build and validate semantically aligned work contexts, executable tasks, and evidence-grounded verifiers, adapting their actions to workspace state and execution feedback rather than following a fixed construction workflow. To support environment evolution, we introduce Factorial Compatibility Replay (FCR), which uses rollout feedback to evaluate combinations of component revisions through dependency-closed replay and selects the smallest validated repair among tested candidates while preserving task requirements and evaluation constraints. To provide finer-grained training signals, we propose Verifier-to-Turn Credit Assignment (VTCA), which augments trajectory-level GRPO advantages with signals from deterministic workspace-progress checks and assigns a bounded share of positive-progress credit to prerequisite turns using recorded tool-output and artifact dependencies. Using WorkRover, we post-train Qwen3.5-35B-A3B to obtain WorkRover-35B-A3B and evaluate it on several cowork-agent benchmarks. Its reported scores exceed those of Qwen3.5-35B-A3B by 26.27 percentage points on JobBench, 29.32 percentage points on OfficeQA Pro, and 413.25 Elo points on GDPval v2. These results suggest that WorkRover offers a promising foundation for scalable training of long-horizon cowork agents.

1 Introduction

WorkRover addresses the need to keep cowork tasks, work contexts, and verifiers aligned through repeated environment refinement, while providing finer-grained learning feedback. The paper reports improvements over the Qwen3.5-35B-A3B backbone across five benchmarks.

1 Introduction

  • Existing pipelines offer limited support for repeated revisions that keep work contexts, task specifications, and verifiers aligned; reliable terminal rewards also leave turn-level credit assignment unresolved.Useful steps can occur in failed trajectories, while earlier information-gathering turns may contribute only through later workspace changes.Source: §1 · p. 2
  • WorkRover jointly constructs and validates semantically aligned work contexts, executable tasks, and evidence-grounded verifiers, adapting construction actions to workspace state and execution feedback.Independent reference executions establish feasibility, while verifier validation tests incorrect outcomes and valid alternatives against instruction-grounded requirements.Source: §1 · pp. 2–3
  • WorkRover-35B-A3B exceeds Qwen3.5-35B-A3B on all five benchmarks, including gains of 29.32 percentage points on OfficeQA Pro and 26.27 percentage points on JobBench.It also gains 413.25 Elo points on GDPval v2.Source: §1 · p. 2
  • Under matched repair budget caps, FCR improves task-package synthesis pass rate by 6.0 percentage points over full regeneration.A maintenance evaluation also finds more reliable repairs of naturally occurring defects with fewer unintended changes to task intent.Source: §1 · p. 2
  • With initialization, task and seed schedules, and rollout budget fixed, VTCA outperforms episode-only GRPO across all five benchmarks.Construction ablations also find that following file dependencies and matching files with compatible skills improve task yield and executability.Source: §1 · p. 2

2 Related Work

Related work addresses executable environments and task synthesis, environment repair, and long-horizon credit assignment. WorkRover combines dependency tracking and replay-based repair with workspace-grounded progress signals.

2 Related Work

  • Prior work scales executable environments [2] [7] [30] [36] and validated task synthesis [12] [22]; shared specifications and state align tasks and verifiers [29], while adversarial audits test verifier reliability [47].WorkRover jointly constructs Work Contexts, Tasks, and Verifiers while tracking their dependencies under a versioned Harness.Source: §2 · p. 3
  • Recent work studies task-bundle revision and execution-driven repair [10] [23], while program repair studies patch minimization; FCR tests repair combinations and selects the smallest validated repair among those tested.It preserves task requirements and evaluation constraints and checks affected tasks for regressions.Source: §2 · p. 3
  • Prior long-horizon credit assignment uses state comparisons, planning hierarchies [8] [24], progress estimates, or evidence attribution [9] [34] [42] [43] [44] [46]; VTCA shapes GRPO with frozen workspace checks and bounded credit to prerequisite turns.Its signals are limited by checker coverage and dependency quality.Source: §2 · pp. 3–4

3 Method

WorkRover organizes training around harness-driven synthesis, recursive environment repair, and verifier-grounded turn-level learning. Its method constructs task packages and validates repairs and training signals against fixed requirements and checks.

3.1 Harness-Driven Environment Synthesis

  • The Harness coordinates construction, tracks dependencies, and manages execution across Work Context, Task, and Verifier components.The Work Context supplies files, capabilities, and application environments; the Task specifies goals, deliverables, and constraints; the Verifier checks completion.Source: §3.1 · p. 4

3.1.1 Work Context Construction

  • Work Context construction profiles files and packages skills and plugins into libraries, then instruments applications as executable environments with runtime, observation, interaction, and reset support.The assembled resources include file, capability, and app contexts.Source: §3.1.1 · pp. 4–5

3.1.2 Long-Horizon Task Synthesis

  • Task synthesis assembles compatible files, apps, and capabilities around a job skill, then uses an independent Demonstrator Agent to confirm feasibility and produce a reference state.The task instructions specify goals, materials, deliverables, and constraints; the workflow must not add requirements or exclude other valid solutions.Source: §3.1.2 · p. 5

3.1.3 Verifier Synthesis and Validation

  • Verifier synthesis binds rubric criteria to workspace evidence under a fixed Evaluation Contract, then tests rejection of failures and acceptance of the reference state and valid alternatives.Candidates pass only when weighted outcome thresholds and mandatory checks pass; a one-time admission test rejects failures without feedback for further repair.Source: §3.1.3 · p. 5

3.2 Recursive Environment Evolution

  • FCR diagnoses environment problems, freezes proposals and evaluation constraints, replays dependency-closed combinations under matched conditions, and selects the smallest repair that passes independent checks.Fresh rollouts confirm the selection; failed repairs are discarded for unadmitted candidates or leave the previous admitted version in place.Source: §3.2 · pp. 5–6

3.3 Verifier-to-Turn Credit Assignment

  • VTCA supplements trajectory rewards with deterministic checks of workspace progress and setbacks, allocating bounded positive credit to prerequisite turns identified through recorded dependencies.Missing observations produce no local signal, and checks use recorded states without intermediate Judge calls.Source: §3.3 · p. 6

❸Verifier-to-Turn Credit Assignment

VTCA supplements trajectory-level GRPO feedback with progress signals checked against workspace states. It allocates some positive progress credit to prerequisite turns and subtracts local trace costs.

Verifier-to-Turn Credit Assignment

  • VTCA combines terminal rewards with checked workspace progress and setbacks to shape turn-balanced GRPO advantages.Checks compare supported criteria before and after each turn; missing scores yield no local signal, and checks run on recorded states without intermediate Judge calls.Source: Figure 5 · p. 7, ❸Verifier-to-Turn Credit Assignment · p. 7
  • At least half of each positive increment stays with its producing turn; the remainder is shared equally among earlier turns whose outputs or exact artifact versions directly supported that progress.Merely opening a file does not qualify, and negative increments stay entirely with the producing turn.Source: ❸Verifier-to-Turn Credit Assignment · p. 7
  • Progress credit is added to trajectory advantage while local trace costs penalize tool errors or unproductive repetition; integrity, protected-state, and safety violations disable progress credit for every turn.The shaping signal depends on checker coverage and dependency records.Source: ❸Verifier-to-Turn Credit Assignment · p. 7

4 Experiments

Experiments evaluate transfer across five external benchmarks and test verifier-grounded learning, environment repair, and construction components. WorkRover improves on its backbone across benchmarks, while controlled experiments report gains from VTCA, FCR, and construction rewriting.

4.1 Experimental Setup

  • The evaluation uses five external benchmarks with their official evaluators and aggregate metrics, while published reference scores retain source-specific settings.The benchmarks cover occupational tasks, professional deliverables, cross-application workflows, office workflows, and document-grounded question answering.Source: §4.1 · pp. 7–8

4.2 Main Results

  • WorkRover-35B-A3B improves on its Qwen3.5-35B-A3B backbone across all five external benchmarks.Gains include +29.32 percentage points on OfficeQA Pro, +26.27 on JobBench, +22.67 on AutomationBench, +13.63 on WorkBuddy Bench (Office), and +413.25 Elo on GDPval v2.Source: §4.2 · p. 8, Table 1 · p. 8

4.3.1 Verifier-to-Turn Credit Assignment

  • Under matched rollout budgets, VTCA outperforms episode-only GRPO on all five benchmarks after both methods improve on SFT.The largest percentage-point gains are +5.7 on AutomationBench and +5.4 on OfficeQA Pro; GDPval v2 improves by 43 Elo.Source: §4.3.1 · p. 9, Figure 6 · p. 9

4.3.2 Recursive Environment Evolution

  • On natural defects, FCR raises SRR from 60.7% to 70.3% over full regeneration and reduces task-intent changes from 14.3% to 1.3%.Against applying all fixed edits, FCR raises SRR by 9.0 points on natural defects and 10.4 points on injected defects, reducing cross-task regression from 7.0% to 0.7% at higher model-token cost (1.00 versus 0.59).Source: §4.3.2 · p. 9, Figure 7 · p. 10, Table 2 · p. 10
  • Across three generations, FCR maintains SRR at 70.3–72.4% and audit pass rate at 93.8–94.6%, while task success rises over five repair iterations.Depending on the component pair, 48–74% of safely resolved cases require joint edits; FCR also raises initial-candidate synthesis pass rate to 74.8%, versus 49.8% before repair and 68.8% with full regeneration.Source: §4.3.2 · p. 9, Figure 7 · p. 10

4.3.3 Construction Pipeline

  • Removing relation-aware expansion or validation-guided rewriting lowers task yield from 74.8% to 52.6% or 49.8%, respectively.Removing relation-aware expansion also lowers reference execution from 95.1% to 77.9% and agent success from 58.7% to 41.8%; the semantic-only baseline performs worse on all three measures.Source: §4.3.3 · pp. 9–10, Table 3 · p. 10
  • On the same 10,000 initial task intents, adding rewriting raises admission from 49.8% to 74.8% and end-to-end success from 22.2% to 43.9%.Reference execution rises from 80.0% to 92.0%, and Verifier validity from 70.0% to 87.0%; rejected and budget-exhausted intents remain in the denominator.Source: §4.3.3 · p. 10

5 Conclusion

WorkRover uses execution feedback to improve cowork agents and the environments used to train them. Its Harness, FCR, and VTCA keep work contexts, tasks, and verifiers aligned during construction and repair, and use verified workspace progress to guide learning.

5 Conclusion

  • FCR increases task validation pass rates while repairing defects with fewer unintended changes to task intent and fewer regressions on related tasks.Under matched rollout budgets, VTCA outperforms episode-only GRPO on all five benchmarks.Source: §5 · p. 10

A Extended Related Work

  • WorkRover jointly constructs Tasks and Verifiers within recoverable Work Contexts, tracking dependencies under a versioned Harness.The Harness validates dependencies, permissions, bindings, reset, recovery, and evidence capture.Source: A Extended Related Work · p. 14, §B.1 · p. 15
  • FCR selects the smallest admissible dependency-closed repair among tested combinations, but bounded enumeration does not establish a global minimum.Promotion also checks affected retained tasks for regressions under fixed requirements.Source: A Extended Related Work · p. 14
  • VTCA assigns at most half of positive progress to eligible prerequisite turns using recorded direct use of tool outputs or artifact versions; access alone creates no dependency edge.Incorrect provenance can misallocate credit, while missed links leave prerequisites unrepresented.Source: A Extended Related Work · p. 14
  • VTCA’s workspace-grounded advantage shaping establishes neither policy invariance nor an unbiased policy-gradient estimate.The passage distinguishes this formulation from potential-based reward shaping under stated assumptions.Source: A Extended Related Work · pp. 14–15

B.1 Work Context Construction

  • Indexed Work Context construction costs O(WCindex) + O(JCretrieve) + O(Jk(Cagent + Cmatch)) + O(MCenv), with candidate count M ≤ Jk.Agent inspection covers Jk shortlisted candidates rather than exhaustive J×W job–workspace pairs, and the index can be reused across job-skill queries.Source: §B.1 · p. 15

B.2 Synthesis Resources and Coverage

  • Synthesis resources include 21,444 app skills, 5,918 file skills, and 36,064 job skills covering 317 roles.The skills span 15 app-skill categories, 10 job-skill categories, and 22 file-skill categories.Source: §B.2 · p. 15

B.3 Workspace Construction from Deep Research Sessions

  • Deep Research workspaces use exported files from one research record, while new tasks require fresh Demonstrator execution and independent admission.Branch plans bind eligible files, operations, and deliverables; fresh execution establishes feasibility, and admission tests violating mutants, valid alternatives, and protected state.Source: §B.3 · p. 15, state Violating · p. 16

B.4 Task Synthesis and Quality Control

Task packages combine a resettable workspace with a fresh Demonstrator realization, and construction profiles set task scale and interaction horizon. Admission gates require readable, executable inputs and observable outcomes.

B.4 Task Synthesis and Quality Control

  • Each task package contains an instruction, resettable snapshot, environment specification, metadata, and a fresh Demonstrator realization with available terminal artifacts.Instructions state outcomes and constraints without prescribing tools or action sequences.Source: §B.4 · p. 16
  • Admission gates check instruction clarity, readable inputs, sufficient dependencies, reproducible restoration, and outcome anchors; packages with unobservable core outcomes are rejected.Empirical difficulty is calibrated as the valid-rollout failure rate of a frozen reference panel under matched snapshots, tools, permissions, and budgets.Source: §B.4 · p. 16
  • Construction profiles control task scale and interaction horizon.Profiles also set target edit scope, skill complexity, distractor ratio, and cross-environment dependencies.Source: §B.4 · p. 16, Table 4 · p. 17

B.4.1 Task Design and Review

The design binds task outcomes to inspectable or recomputable evidence and reviews artifact integrity. The recorded quarterly handoff illustrates execution against multiple deliverables, with archived validation passing all four goal groups.

B.4.1 Task Design and Review

  • The Blueprint binds each outcome to extractable or independently recomputable artifact or application evidence.The manifest distinguishes required inputs, history, distractors, and protected artifacts; review checks formulas, layout, version history, and embedded objects, with rendered inspection for visually specified outcomes.Source: §B.4.1 · p. 17, Table 5 · p. 17

B.5 Case Studies: Recorded Deep Research Examples

  • Recorded research workflows are paired with exported file collections, but new tasks still require fresh Demonstrator execution and counterfactual Verifier checks.Raw traces and internal provenance remain outside Solver inputs; source-page references do not establish per-file downloads in the recorded traces.Source: §B.5 · p. 17

B.6 Case Study: Quarterly Executive Handoff

  • All four archived goal groups pass (score 0.946) in the quarterly executive handoff case study.The agent verifies five regional revenue rows and five exports; file validation covers structure and content, while visual review is unavailable.Source: §B.6 · p. 17

C Verifier, Repair, and Training Details

Verifier admission uses instruction-grounded criteria and controls for valid alternatives, violating outcomes, and infrastructure faults. FCR replays dependency-closed repair combinations and promotes only validated edits that preserve requirements.

C Verifier, Repair, and Training Details

  • Recorded cases show policy research using two IPSS workbooks and a quarterly handoff with archived validation and retained follow-ups.The prompt-engineering course-design case reuses captured paper-source archives to produce a comparison workbook and lab briefs with baselines and grading rubrics.Source: Figure 10 · p. 18, Figure 9 · p. 18, Figure 11 · p. 19

C.1 Verifier Specification and Calibration

  • Verifier admission requires a Starter–Reference contrast, protected-state checks, no unresolved invalid controls, and frozen thresholds for criterion agreement, mutant detection, and valid-alternative acceptance.Missing required state implies failure, while infrastructure-unavailable evidence implies invalidity; partial credit cannot offset gates.Source: §C.1 · pp. 19–20

C.2 FCR Mechanics and Promotion

  • FCR freezes one direct proposal per available Context, Task, Harness, and Verifier factor, then tests dependency-closed update sets under matched conditions.Selection requires proposal-independent defect or health predicates, equivalence, no affected-task regression, and calibration; the smallest validated closed edit set among tested cells receives fresh-seed confirmation.Source: §C.2 · p. 20

C.3 FCR Evaluation Protocol

The protocol evaluates FCR repairs with controlled cohorts and matched comparisons, while VTCA assigns progress credit using deterministic checks and recorded dependencies. It specifies how progress and prerequisites are measured and notes that incomplete observations and provenance errors constrain the signals.

C.3 FCR Evaluation Protocol

  • 260 concealed Contract-preserving defects form the controlled cohort, while frozen stratified screening of the 10K pool targets 300 natural mismatches.The protocol also uses five matched seeds per distinct Context–Task–Harness configuration and keeps confirmation seeds disjoint from generation-specific promotion shards.Source: §C.3 · p. 20

C.4 Verifier-to-Turn Credit Assignment

  • Canonical traces preserve call/result IDs, tool definitions, versions, and state metadata; invalid rollouts receive no training signal.SFT supervises agent tokens.Source: §C.4 · p. 20
  • Deterministic progress increments require both endpoints to be evaluable, and prerequisite turns count only when their exact outputs or versions are directly consumed.Unevaluable intervals are recorded with a missing reason, not bridged; stale, ambiguous, inferred, or unrecorded uses are excluded from prerequisite sets.Source: §C.4 · p. 20
  • VTCA assigns a bounded share of positive-progress credit to prerequisite turns, while negative increments and increments with no prerequisites stay local.Weights are not renormalized over observed criteria or centered across turns; GRPO averages tokens within turns, turns within trajectories, and trajectories within task groups.Source: §C.4 · p. 21
  • Write–revert cycles and wrong provenance can still alter updates, and only complete observations telescope.The denominator floor is fixed on development data.Source: §C.4 · pp. 20–21

D.1 Experimental Setup

The setup specifies WorkRover’s training data, post-training procedure, benchmark configurations, and controls for matched VTCA comparisons. Controlled evaluations fix key conditions and keep GDPval Elo separate from percentage scores.

D.1 Experimental Setup

  • Qwen3.5-35B-A3B is the post-training backbone; training uses 30,115 Verifier-accepted SFT trajectories and retains 3,200 RL pairs from 20,000 candidates.RL starts from SFT with turn-balanced GRPO, KL coefficient 0.01, and matched tools, permissions, states, and budgets.Source: §D.1 · p. 21
  • Matched controlled runs fix the release, split, evaluator, Harness, and budget, charge invalid or retried attempts, and cluster tasks by Work Context.Controlled OfficeQA Pro uses 133 questions at 0.0% numerical tolerance; GDPval Elo and percentage metrics are not averaged.Source: §D.1 · p. 21
  • Benchmark configurations include 65 JobBench Main tasks with OpenCode v1.14.18 and Grok 4.3, plus a specified September 12, 2026 GDPval v2 snapshot.For OfficeQA Pro, WorkRover uses Claude Code with temperature 0.7, top-p 0.95, top-k 40, and 1,800 seconds.Source: §D.1 · p. 21

D.2 Matched VTCA Comparison

  • VTCA comparisons match checkpoints, task and seed schedules, groups, budgets, and interaction caps; inference resamples training replicas and task/Context clusters.Controls include aggregation, local-only progress, matched random or verified recipients, and trace cost; GDPval ratings are refit from matched judgments.Source: §D.2 · p. 21

E Data Construction and Case Study Summaries

The section consolidates material boundaries and admission processes, then distinguishes proposed tasks from recorded execution. Its tables cover source materials, task checks, and case-study inputs and evidence scope.

E Data Construction and Case Study Summaries

  • Table 6 summarizes source materials, construction roles, and Solver visibility, while cautioning that eligibility alone does not establish factual authority or evidence sufficiency.The following tables consolidate material boundaries and admission processes from Appendices B.3 and B.4.Source: E Data Construction and Case Study… · p. 21, Table 6 · p. 22
  • Table 7 lists task construction stages and admission checks; every new task requires fresh execution and verification, regardless of historical session completion.The table distinguishes proposed tasks from recorded execution described in Appendices B.5 and B.6.Source: E Data Construction and Case Study… · p. 21, Table 7 · p. 22
  • Table 8 scopes case-study inputs and work products: two Deep Research cases illustrate proposed derivative tasks, while the handoff case is a recorded execution.Source: Table 8 · p. 22

Related papers

Loading 2610.18067v1…