Source-linked AI summary
Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails
Zhou Yu, Bin Bi, Shiva Kumar Pentyala, Shubham Mehrotra, Sougata Chaudhuri, Shilpa Bhagavath, Zeyuan Chen, Ran Xu, Phil Mui, James Zhu, Sitaram Asur
TL;DR
The paper asks how harness evolution and lightweight fine-tuning should be combined for economical domain-specific enterprise agents. It develops on-policy expert correction after finding that full expert imitation under an evolved harness regresses performance, and reports a compatibility-preserving co-evolution recipe.
Problem
The central gap is how to combine harness evolution and lightweight model fine-tuning for task-specific domain adaptation under a reasonable budget.
Method
The paper evolves a harness, then uses a meta-level MLE agent to localize failures in the weaker model’s rollouts and obtain expert corrections only for those turns.
Results
−14.9 on average: expert-trajectory imitation under the evolved harness regresses the weaker model, while on-policy correction preserves model–harness fit and adds model-side gains.
Takeaways & Limitations
After harness optimization, model updates should preserve the scaffold’s fit to the model’s planning and execution behavior rather than rely on wholesale expert imitation.
Takeaways & Limitations
Future work must jointly optimize harness evolution and later fine-tuning rather than treating them as sequential steps.
Abstract
from arXiv · showhide
Agent harnesses (the system prompt, tool set, execution hooks, and context-management scaffolding around a model) are a critical determinant of agentic task success. Automated harness evolution can enable smaller models to perform well on domain-specific tasks at a fraction of frontier-model cost. Since both the harness and model weights shape behavior, we ask how harness evolution and lightweight fine-tuning should be combined. Across seven enterprise agent tasks, we first evolve a harness with the weaker model, then find that a stronger expert often uses it more effectively, suggesting expert supervision could close the remaining gap. However, training the weaker model on the expert's complete trajectories under the evolved harness backfires: performance regresses on all seven tasks by 4 to 30 points across Qwen3-Coder and Gemma 4, even though the same procedure helps under the unevolved harness. Our analysis shows that imitation transfers knowledge and increases scaffold usage, but disrupts model-harness fit: the weaker model adopts the expert's planning strategy without the competence to execute it and no longer matches the harness evolved around its native planning style. We therefore develop an on-policy expert-correction pipeline, automated by a meta-level MLE agent, that localizes the failing turn in the weaker model's own rollout and asks the expert to rewrite only that turn. This preserves the model's planning style and combines the gains of harness evolution and model adaptation. Our results identify and resolve a source of contention between harness and weight updates, yielding a compatibility-preserving recipe for economical co-evolution on domain-specific enterprise tasks.
1 Introduction
The paper studies how harness evolution and lightweight model fine-tuning can be combined for economical domain-specific enterprise agents. It finds that full expert imitation can disrupt the evolved model–harness fit, while localized on-policy correction preserves that fit and supports co-evolution.
- Motivation: Harness design and model weights jointly determine agent capability, making their combination central to low-cost enterprise adaptation.The harness includes the system prompt, tools, execution hooks, and context-management scaffolding.
- Upward harness transfer: Across seven enterprise tasks, a stronger expert effectively uses the harness evolved for the weaker model, exposing model-side headroom for expert supervision.The expert outperforms the weaker model on every task under the shared harness.
- Imitation failure: 4 to 30 points: full-trajectory expert imitation reduces weaker-model task success on all seven benchmarks after harness evolution.The same procedure improves performance under the unevolved harness, isolating the failure to the interaction between imitation and harness evolution.
- Imitation failure: Imitation increases knowledge and scaffold usage but causes planning-strategy drift, so the weaker model no longer matches the harness evolved around its native planning style.The resulting problem is model–harness fit rather than lost knowledge or reduced harness usage.
- On-policy correction: On-policy expert correction localizes a failing turn in the weaker model’s rollout and has the expert rewrite only that turn.This preserves the weaker model’s planning style while adding model-side gains on top of the evolved harness.
- Co-evolution: Harness and model adaptation become coupled after harness optimization, so subsequent weight updates should preserve their fit rather than disrupt it.The updated model can re-enter harness evolution in the next co-evolution round.
2 Related Work
Prior work shows that optimizing prompts and broader harnesses can help smaller models perform complex, tool-using agentic tasks. Research is also beginning to examine how harness adaptation interacts with changes to model weights.
- Prompt and harness optimization can enable smaller models to perform complex, multi-step, tool-using agentic tasks.
- Behaviors introduced through external scaffolding can later be absorbed into model weights, making the boundary between harness and model adaptation porous.
- Recent work has begun optimizing harnesses and model weights together rather than treating them as independent adaptation targets.
3 Experiments
Across seven enterprise tasks, harness evolution substantially improves weaker-model performance, but full expert imitation under the evolved harness causes regressions. On-policy correction instead preserves model–harness fit while adding smaller gains.
- Mean test success rises from 29.2% under the base harness to 78.0% under the evolved harness, a gain of 48.8 points across tasks.
- The stronger expert improves from 84.4% to 93.6% under the evolved harness and outperforms the weaker model by 15.6 points on average.The expert triggers nearly every evolved edit, including the domain-computation recipe in 94.2% of rollouts.
- Expert imitation lowers Qwen’s mean test success from 78.0% to 63.1% and regresses all seven tasks by 4.2 to 29.9 points.The same recipe raises baseline-harness performance from 29.2% to 35.5%, isolating the failure to its interaction with harness evolution.
- Imitation increases evolved-scaffold use, including domain-computation usage from 30.8% to 76.1%, but the failure analysis concentrates on implicit-knowledge and planning defects.
- On-policy correction localizes a failed turn in the weaker model’s rollout and uses an expert rewrite while preserving the weaker model’s planning distribution.
- On-policy correction raises mean test success from 78.0% to 79.7% (+1.7), gaining on five of seven tasks while remaining within noise on two saturated tasks.It matches or beats the evolved-harness base on every task and avoids direct imitation’s −14.9-point average regression.
- Under on-policy correction, planning failures move only from 1.1% to 1.8% (+0.7), while knowledge failures fall from 46.2% to 43.2% (−3.0).Overall failure rate also drops from 28.9% to 26.8%.
4 Conclusion
The paper concludes that expert imitation can disrupt the fit between a model and its evolved harness, whereas on-policy correction supports economical co-evolution. Future work will jointly optimize harnesses and model updates and combine correction with reinforcement learning.
- On-policy correction delivers further gains without eroding harness-evolution benefits, offering a lightweight recipe for co-evolving models and harnesses.
- Future work will combine reinforcement learning with on-policy expert guidance and make harness evolution aware of later model fine-tuning.
A Harness Optimization
The study evolves each task’s harness by iteratively testing edits with a weaker model, then selecting the highest-scoring validation candidate. The resulting edits target task-specific failure categories through tools, instructions, hooks, externalized knowledge, or explicit plans.
- Optimization procedure: Each iteration reflects on a minibatch, proposes a harness edit, and retains it only when minibatch performance improves.Retained candidates enter a Pareto pool, and the best harness is selected by validation score, with ties broken by earliest iteration.
- Optimization procedure: The winning harness is selected as the candidate with the highest validation score across the optimization process.Each task uses three random seeds, a $20 budget, and a 600 s per-rollout time limit.
- Harness adaptations: Winning edits address failure categories by creating tools, reinforcing instructions, externalizing implicit knowledge, or adding explicit plans.The ontology maps these interventions to tool-use, instruction-following, implicit-knowledge, and planning adaptations.
B Model SFT
Model adaptation uses LoRA fine-tuning under the evolved harness, comparing expert imitation with on-policy corrections. The correction pipeline trains on the weaker model’s own rollouts while changing only localized failing turns.
- Training setup: All fine-tuning recipes use the same base model and LoRA configuration, with two training epochs at learning rate 1e−4.The setup uses rank 16 or rank 64, bf16, an effective batch size of 8, and extended context when truncation exceeds 5%.
- Imitation training: Expert-imitation data combines successful expert trajectories under the evolved harness with passing rollouts from the base model.Trajectories are converted to the model’s chat format before fine-tuning.
- Correction training: On-policy correction data starts from the base model’s failed rollouts and builds a small training set from localized expert rewrites.An automated failure-locus step identifies one failing turn, while surrounding steps remain unchanged.
- Correction training: For each failing turn, the expert receives the preceding trajectory and failed checkpoint, then rewrites only that turn.Three candidate corrections are sampled and the best is selected by a quality judge.
C Analysis Methodology
The analysis examines failed rollouts under each condition and classifies them with a six-category adaptation-failure ontology. The framework distinguishes failures involving tools, instructions, implicit knowledge, long context, planning, and other causes.
- Data: Failure-composition analysis uses failed rollouts, with hard failure defined as a rollout scoring 0 on binary task success.All failed rollouts for each condition are collected under its relevant harness and classified.
- Method: The ontology contains six categories: other, tool-use, instruction-following, implicit knowledge, long-context, and planning.Categories are numbered 0 through 5.
- Method: Implicit-knowledge failures involve wrong domain values, missed conventions, or omitted practitioner steps.Planning failures involve omitted critical subtasks, unrecovered wrong plans, or related planning errors.
D Case Studies
The case studies show that expert imitation can preserve domain knowledge while disrupting the planning cadence required by an evolved harness. In payroll auditing and Webarena review counting, the weaker model follows a mismatched plan or bypasses a load-bearing scaffold step.
- Cross-case finding: The fine-tuned model retains needed domain knowledge but loses the planning cadence around which the harness was tuned.This pattern appears in both presented rollouts.
- Payroll audit: In payroll auditing, the evolved harness requires employee aggregation, department mean-of-means, fixed rounding, and one final schema check.The base model completes the scaffold cleanly in 14–20 steps, with all three example rollouts scoring 1.0.
- Webarena review count: In Webarena review counting, imitation-fine-tuned Gemma reads the unfiltered 351-record total instead of filtering Approved reviews before counting.The correct total is 346, and the harness’s procedure requires locating the status column, applying the filter, then reading the filtered total.
- Webarena review count: The Webarena failure reflects unsupported answer-early behavior rather than a different surface output style.Gemma navigates to the review grid but never forms the required “filter, then count” sub-goal.