Source-linked AI summary
VACE: Validation-Gated Alternating Co-Evolution of Agent Models and Harnesses
Jiexing Qi, Yu He, Jun Liu, Qichen Huang, Shaohua Hu, Zhan Dang, Guohua Chen, Rui Yang, Wen Jiang, Yang Liu, Tao Lyu, Fangming Li
TL;DR
Agent performance depends on coupled weights and harnesses, but optimizing one while fixing the other leaves their interaction unaddressed. VACE alternates agentic RL with trajectory-driven harness refinement and validates each candidate with the updated model before it can guide further training. It outperforms weight-only RL and ungated alternation on both benchmarks, and exceeds SIA-style by 9.85 pp on AutomationBench.
Problem
Model and harness updates affect one another, while optimizing one component with the other fixed leaves that interaction unaddressed.
Source: §1 · pp. 1–2Method
VACE alternates agentic RL with trajectory-driven harness proposals, adopting a candidate only when it improves validation performance with the updated model held fixed.
Source: §1 · p. 2Results
VACE achieves higher mean test scores than RL-only and WHALE-style on both datasets, exceeding SIA-style by 9.85 pp on AutomationBench (75.19% versus 65.35%).
Source: §5 · p. 7, §4.3 · p. 7Takeaways & Limitations
Because proposed harness revisions can reduce validation performance, an explicit acceptance decision precedes their use in subsequent training.
Source: §6 · p. 8Repeated validation-set use can introduce adaptive selection effects, and the strict-improvement gate relies on noisy point estimates.
Source: §5 · p. 7
Abstract
from arXiv · showhide
Language model agents can be improved by updating their model weights or refining the harness that guides task execution. These components are coupled: weight updates change how the model uses the harness, while harness updates change the trajectories used for training. We propose VACE, Validation-Gated Alternating CoEvolution, which alternates agentic reinforcement learning with trajectory-driven harness refinement. After each RL stage, VACE reuses the collected trajectories to propose a harness revision and evaluates the incumbent and candidate with the updated model held fixed. The candidate guides subsequent training only if it improves validation performance. With Qwen3.5-9B, VACE achieves 45.26% test accuracy on OfficeQA and a mean partial-credit score of 75.19% on AutomationBench, exceeding weight-only RL by 6.43 and 9.09 percentage points and ungated alternation by 4.59 and 6.95 points, respectively. Across 44 harness proposals, 17 reduce validation performance at the updated checkpoint and are rejected before subsequent RL training, highlighting the importance of validation gating.
1 Introduction
Agent performance depends on coupled weights and harnesses, yet optimizing either alone leaves their interaction unaddressed. VACE alternates agentic RL with trajectory-driven harness refinement and validates candidate revisions before they guide subsequent training.
1 Introduction
- RL can improve reasoning over retrieved evidence while a single-retrieval skill still prompts the agent to stop before gathering enough information.Source: §1 · p. 1
- Weight updates alter how the model uses its harness, while harness updates alter the executions available for subsequent learning.Source: §1 · p. 2
- VACE reuses each RL stage’s trajectories to propose a harness revision, then adopts it only if it beats the incumbent on validation with the updated model held fixed.The retained harness guides the next RL stage.Source: §1 · p. 2
- VACE links trajectory feedback, repeated harness refinement, and checkpoint-specific validation in an RL-based co-evolution framework.The introduction situates co-optimization alongside SIA, Co-Harness, and concurrent work WHALE.Source: §1 · p. 2
- 6.43 and 9.09 percentage points above weight-only RL, and 4.59 and 6.95 points above WHALE-style ungated alternation, respectively, on OfficeQA and AutomationBench with Qwen3.5-9B.The comparison reports the OfficeQA and AutomationBench differences in that order.Source: §1 · p. 2
- 17 of 44 harness proposals reduce validation performance at the current checkpoint, motivating separate decisions about proposing and accepting revisions.Source: §1 · p. 3
2 Related work
Prior work studies model training and harness optimization separately, as well as frameworks that co-optimize them using different schedules and training procedures.
Model training
- Agentic RL approaches learn from execution feedback, while VACE also uses that feedback to refine the harness.Examples include Search-R1, WebAgent-R1, AgentCPM-Explore, and Agent Lightning.Source: §2 · p. 3
Harness optimization
- Harness optimization ranges from storing reflections or extracting insights to trajectory-guided prompt revisions and evaluated agent-code changes; VACE couples HarnessEvolve with model training.Source: §2 · p. 3
Related and concurrent co-optimization frameworks
- SIA, Co-Harness, and WHALE use distinct co-optimization schedules; VACE combines agentic RL with trajectory-driven proposals validated at the updated checkpoint.VACE studies document-grounded reasoning and workplace workflows, focusing harness edits on skills and execution guidance.Source: §2 · p. 3
3 VACE
VACE alternates model training with trajectory-driven harness refinement, linking the two through execution feedback. It accepts a proposed harness only when it strictly improves validation performance with the updated model held fixed.
3 VACE
- VACE trains the model under the current harness, proposes revisions from collected trajectories, and retains the better-scoring harness to guide the next round.The overview presents this as a repeating feedback loop between model training and harness refinement.Source: §3 · p. 3, Figure 2 · p. 4
3.1 Model–harness co-optimization
- The objective is expected task reward, and alternating weight and harness updates are coupled because each changes the executions available to the other.Harness edits focus on skills and execution guidance while task evaluation, tool semantics, and environment transition rules remain fixed.Source: §3.1 · p. 4
3.2 Trajectory-driven alternating updates
- Trajectory-driven refinement reuses training executions to identify failures and propose targeted harness edits, avoiding a separate diagnostic-execution collection pass.The harness optimizer proposes one candidate per round, and validation determines whether it guides subsequent training.Source: §3.2 · pp. 4–5
3.3 Validation-gated harness updates
- Harness proposals are compared with the incumbent using the same updated checkpoint, validation tasks, and evaluator; ties retain the incumbent.Only harness revisions are gated; updated model weights are retained either way, and the retained harness governs the next round’s rollouts.Source: §3.3 · p. 5
4 Experiments
The experiments compare VACE with single-component optimization and alternative schedules on OfficeQA and AutomationBench. VACE outperforms these baselines, while proposal-level results and validation trajectories show why checkpoint-specific harness acceptance matters.
4.1 Tasks and experimental setup
- Experiments used Qwen3.5-9B on document-grounded OfficeQA and tool-based AutomationBench, with task-specific splits and validation-selected checkpoints evaluated three times.Source: §4.1 · pp. 5–6
4.2 Compared methods
- Baselines isolate fixed weights, fixed harnesses, harness-first scheduling, and ungated alternation; schedule controls reuse VACE's weight and harness optimizers.Source: §4.2 · p. 6
4.3 Main results
- 45.26% OfficeQA accuracy and 75.19% AutomationBench mean partial credit put VACE 6.43 pp above RL-only (38.83%) and 9.09 pp above RL-only (66.10%), respectively.Source: §4.3 · p. 6, Table 2 · p. 7
- 4.59 pp on OfficeQA (45.26% versus 40.67%) and 6.95 pp on AutomationBench (75.19% versus 68.25%) separate VACE from WHALE-style ungated alternation.Source: §4.3 · p. 6, Table 2 · p. 7
- 9.85 pp separates VACE from SIA-style on AutomationBench (75.19% versus 65.35%); on OfficeQA, VACE scores 45.26% versus 40.67%.Source: Table 2 · p. 7, §4.3 · p. 7
4.4 Why validate harness proposals?
- 25 of 44 proposals were accepted, 17 regressed, and 2 tied; the gate retains the incumbent when a candidate lowers validation performance.Source: Table 3 · p. 7, §4.4 · p. 7
4.5 Alternating optimization dynamics
- Retained validation scores rose from 30.28% to 47.17% on OfficeQA and 53.08% to 81.42% on AutomationBench, but weight stages could lower scores.The highest validation scores were 60.37% on OfficeQA and 81.42% on AutomationBench.Source: §4.5 · p. 7
5 Discussion and limitations
VACE achieves higher mean test scores than RL-only and WHALE-style on both datasets, while its phase-level records include both beneficial and harmful harness proposals. The discussion identifies validation reuse, study scope, computational overhead, and fixed scheduling as limitations.
5 Discussion and limitations
- VACE achieves higher mean test scores than RL-only and WHALE-style on both datasets.The phase-level records contain both beneficial and harmful harness proposals, supporting an explicit acceptance decision before subsequent training.Source: §5 · p. 7
- Repeated validation-set use can introduce adaptive selection effects, and the strict-improvement gate relies on noisy point estimates.Source: §5 · p. 7
- Table 4 presents the first accepted and rejected proposal in each dataset as same-checkpoint trajectory examples.Scores are percentages and changes are percentage points.Source: Table 4 · p. 8
- Figure 3 marks accepted harness updates, rejected candidates, ties, and retained pairs that start the next weight stage.Blue segments connect a retained pair to the next postweight evaluation.Source: Figure 3 · p. 8
- The study covers one model size and two agent benchmarks, with harness edits focused on skills and execution guidance; broader environments and scales remain to be explored.Harness refinement and validation add computational overhead to model training.Source: §5 · p. 8
- VACE uses a fixed alternating schedule; feedback-driven scheduling could instead select update timing based on task outcomes and training progress.Source: §5 · p. 8
6 Conclusion
VACE alternates agentic reinforcement learning with trajectory-driven harness refinement, using validation at the updated checkpoint to select the harness for the next training stage. In experiments on OfficeQA and AutomationBench, it achieves higher mean test scores than weight-only RL and WHALE-style ungated alternation.
6 Conclusion
- VACE alternates model training with trajectory-driven harness refinement, using an updated-checkpoint validation comparison to select the harness for the next training stage.Source: §6 · p. 8
- On OfficeQA and AutomationBench, VACE achieves higher mean test scores than weight-only RL and WHALE-style ungated alternation.Source: §6 · p. 8
- Proposed harness revisions can reduce validation performance, motivating an explicit acceptance decision before further training.Future work will evaluate VACE on more datasets and explore feedback-driven scheduling across additional models and environments.Source: §6 · p. 8
A Optimization settings
The appendix summarizes optimization settings for the compared methods, including single-component conditions, scheduling variants, and evaluation procedures.
A Optimization settings
- Table 5 summarizes optimization settings, with BW and BH denoting the planned RL and harness-proposal caps.Source: A Optimization settings · p. 11, Table 5 · p. 11
- Harness-only optimization fixes W0 and uses diagnostic trajectories, while weight-only RL fixes H0 and applies the common weight optimizer throughout.Harness-only uses the same harness optimizer and validation gate as VACE.Source: A Optimization settings · p. 11
- WHALE-style uses VACE’s optimizers, editable scope, and initial-state specification but commits harness-phase output without the outer validation gate.Source: A Optimization settings · p. 11
- SIA-style runs harness-only optimization before weight training, selecting the best active harness through the validation gate and holding it fixed during RL.Source: A Optimization settings · p. 11
- Each weight-training condition follows its own trajectory, while SIA-style reuses the harness-only phase before weight training.Source: A Optimization settings · p. 11
- Paired evaluation compares incumbent and candidate harnesses on identical task IDs at the same checkpoint with the same evaluator.Source: A Optimization settings · p. 11
B Additional experimental results
The appendix supplies a six-condition score-comparison figure and a validation-trajectory figure, with captions specifying the plotted summaries and markers.
B Additional experimental results
- Figure 4 compares scores across six conditions using means, three test evaluations per selected model–harness pair, and sample-standard-deviation whiskers.Source: Figure 4 · p. 12
- Figure 5 plots validation scores by round, with solid curves for round scores, dashed curves for best-so-far scores, and stars for maxima.Source: Figure 5 · p. 12