Source-linked AI summary
HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution
Wen Jiang, Mingmin Chu, Yimeng Tian, Qianxin Zhang, Haofei Yang, Rui Yang, Yang Liu, Tao Lv, Fangming Li
TL;DR
Self-evolving agents must overcome ambiguous credit assignment, shortcut learning, and catastrophic forgetting when improving prompts, skills, tools, and execution logic. HarnessEvolve addresses these problems with reference-guided diagnosis, decoupled agent modules, and quality and performance gates. Across open-domain and enterprise benchmarks, it consistently outperforms the evaluated baselines.
Problem
Self-evolving agents struggle with credit assignment failure, shortcut learning, and catastrophic forgetting, limiting reliable harness improvement.
Method
HarnessEvolve decouples execution, evaluation, optimization, and gating, compares failed executions with reference trajectories, clusters error signals, and filters updates through quality and performance gates.
Results
HarnessEvolve consistently outperforms the evaluated baselines across five open-domain and enterprise benchmarks.
Takeaways & Limitations
Reference-guided diagnosis and gated, decoupled evolution provide a framework for reliable self-evolution across open-domain and enterprise tasks.
Abstract
from arXiv · showhide
Self-evolving agents advance toward autonomy by optimizing their harness---prompts, skills, tools, and execution logic---based on environmental feedback. This paradigm, however, is hampered by three challenges: \textit{credit assignment failure}, where terminal success/failure feedback makes it ambiguous which step caused the error; \textit{shortcut learning}, where agents memorize task-specific patterns rather than acquire generalizable capabilities; and \textit{catastrophic forgetting}, where unguarded updates degrade previously acquired competence. In this paper, we introduce HarnessEvolve, a self-evolving framework that learns from reference trajectories to achieve reliable agent self-evolution. HarnessEvolve decouples the execution agent from the evolutionary pipeline, assigning execution, evaluation, optimization, and gating to independent agent modules, enabling generalizable and stable harness improvements. Specifically, HarnessEvolve overcomes credit assignment failure by generating reference trajectories (execution paths produced when given the ground-truth answers) and aligning failed executions against them to extract error signals, which are clustered to reveal systematic failure patterns. To prevent shortcut learning and catastrophic forgetting, candidate harness updates must pass two gates: a quality gate that filters data leakage and prompt bloat, and a performance gate that accepts each update if it improves on the current batch without degrading recent batches, with epoch-end validation on a held-out set selecting the best-performing accepted agent snapshot. We conduct extensive experiments on several benchmarks spanning open-domain and enterprise scenarios, using different models and agent frameworks. Results demonstrate that HarnessEvolve consistently outperforms state-of-the-art baselines across all benchmarks and settings, confirming reliability across task domains.
1 Introduction
HarnessEvolve addresses three obstacles to reliable self-evolution—credit assignment failure, shortcut learning, and catastrophic forgetting—through a decoupled architecture and reference-guided, gated optimization. It consistently outperforms self-evolving baselines across five open-domain and enterprise benchmarks.
- Challenges: Sparse terminal feedback makes it difficult to identify which intermediate action caused failure in long-horizon tasks.Later errors may be downstream consequences of earlier mistakes, making whole-trajectory reflection unreliable.
- Challenges: Shortcut learning causes agents to hardcode answers or add task-specific examples, producing data leakage and prompt bloat without genuine capability gain.
- Challenges: Catastrophic forgetting occurs when updates optimized on subsets of trajectory data conflict with previously acquired competence.
- Framework: HarnessEvolve decouples execution, evaluation, optimization, and gating into independent modules, making optimization decisions auditable and harder for the execution agent to game.
- Results: HarnessEvolve consistently outperforms self-evolving baselines across five benchmarks spanning open-domain tasks and specialized enterprise QA and text-to-SQL tasks.The evaluated baselines optimize only a single harness component, whereas HarnessEvolve optimizes the entire harness.
- Framework: Reference-guided diagnosis compares failed executions with reference trajectories and clusters resulting error signals to identify systematic failure patterns.Reference trajectories are produced when the execution agent receives the question and ground-truth answer.
2 Related Work
Prior agent designs range from human-specified workflows to skill-augmented and self-evolving systems. Existing self-evolving harness methods broaden optimization but leave reference-based root-cause localization and safeguards against shortcut learning unresolved.
- Prior agent paradigms: Handcrafted agents provide high controllability through human-designed workflows but confine agents within static execution structures.
- Prior agent paradigms: Skill-augmented agents enable dynamic trial-and-error through natural-language skills, but performance remains heavily dependent on human-provided skill quality.This dependence creates a scalability bottleneck.
- Self-evolving agents: Self-evolving approaches optimize workflow architecture, prompts, context, tools, or skills, reflecting multiple directions for autonomous harness improvement.
- Self-evolving agents: Self-Harness and Agentic Harness Engineering evolve broader harnesses, yet the described methods leave reference-based root-cause localization and safeguards against shortcut learning unresolved.
3 Methodology
HarnessEvolve separates execution, evaluation, optimization, and gating into independent modules that iteratively improve an agent harness. It uses verified reference trajectories and clustered error diagnoses to localize failures and generate targeted updates, while evaluating candidates across current and recent batches.
- Problem formulation: HarnessEvolve defines self-evolution as iteratively optimizing an execution agent’s prompts, skills, tools, and execution logic on training tasks.
- Architecture: Four independent agents execute tasks, assess accuracy and references, diagnose and cluster errors, and gate harness updates into a snapshot pool.The execution, evaluation, optimization, and gating roles are assigned to Aexec, Aeval, Aopt, and Agate.
- Batch-driven evolution: Training data are shuffled into mini-batches, accepted harnesses are stored as snapshots, and the best snapshot on validation data is selected after each epoch.The loop continues across multiple epochs after epoch-end validation selects the strongest snapshot.
- Reference-guided diagnosis: Reference trajectories are generated with ground-truth answers, verified for genuine reasoning, and compared with failed trajectories to identify the first divergence point.Tasks without verified references fall back to single-trajectory analysis, which is less precise but still contributes an optimization signal.
- Evaluation: The evaluation agent identifies failed trajectories and supplies batch accuracy measurements that determine whether candidate harness updates pass the performance gate.
- Error diagnosis: The optimization agent converts each failure into severity, cause, and fix-hint signals, then clusters them by cause to produce coherent, targeted harness modifications.Cause-based grouping prevents semantically distinct errors from being merged, while root-cause priority emphasizes the first action divergence when a reference exists.
4 Experiments
HarnessEvolve is evaluated across open-domain and enterprise benchmarks, models, and frameworks, with results showing strong accuracy, transferability, and selective self-evolution. Ablations indicate that reference-guided diagnosis, error clustering, and quality gating each contribute to performance.
- Experimental Setup: HarnessEvolve is evaluated on five datasets spanning open-domain QA, spreadsheet manipulation, enterprise document reasoning, and network-domain QA or text-to-SQL tasks.The study uses SearchQA, OfficeQA, SpreadsheetBench, CloudCoreNetwork-QA, and Wireless-QA.
- Main Results: HarnessEvolve consistently achieves the highest accuracy across evaluated settings, including 86.9% on CloudCoreNetwork-QA and 92.8% on Wireless-QA.On CloudCoreNetwork-QA with Qwen3.6-27B, it improves from 43.4% Base and exceeds GEPA’s 65.3% by 21.6 percentage points; on Wireless-QA with DeepSeek-V4-Flash, it exceeds ACE’s 90.1% by 2.7 points.
- Cross-Framework Generalization: HarnessEvolve-optimized skills improve or maintain accuracy over Base across Hermes, OpenCode, LAMAgent, and DeepSeek Harness without re-optimization.The transfer evaluation tests skills optimized on three open-source datasets across four frameworks.
- Self-Evolution Curve: The two-tier gating mechanism rejects many candidate updates while the peak accuracy in the snapshot pool increases to the final harness.Green points represent accepted snapshots passing both gates, gray points represent rejected candidates, and the blue curve tracks peak snapshot-pool accuracy.
- Ablation Study: Removing reference trajectories reduces accuracy from 86.9% to 57.8%, while removing error clustering yields 68.6% and removing the quality gate yields 80.1%.The ablations identify reference-guided diagnosis as the largest contributor among the tested components.
- Qualitative Analysis: The optimized harness makes targeted edits across skills, scripts, prompts, tool specifications, and execution logic rather than a single component.This qualitative analysis contrasts HarnessEvolve with single-component optimization such as SkillOpt.
5 Conclusion
HarnessEvolve learns from reference trajectories and decouples execution, evaluation, optimization, and gating into independent modules. Across five open-domain and enterprise benchmarks, it consistently outperforms state-of-the-art baselines while addressing three identified self-evolution challenges.
- Core Contributions: HarnessEvolve uses reference-guided error diagnosis, a quality gate, and a performance gate with epoch-end validation to address three self-evolution challenges.These mechanisms target credit assignment failure, shortcut learning, and catastrophic forgetting, respectively.
- Results: HarnessEvolve consistently outperforms state-of-the-art baselines across all five evaluated benchmarks.The conclusion reports results across both open-domain and enterprise settings.
A Before-and-After Comparison of Harness Optimization
Figure 3 compares HarnessEvolve’s harness before and after optimization across six cases spanning skills, scripts, prompts, tools, and execution logic.
- Before-and-After Cases: The comparison covers six optimization cases across skill files, an internal skill script, system prompts, tool descriptions, and execution logic.Cases include OfficeQA, CloudCoreNetwork-QA, and SpreadsheetBench.
B Reference-Guided Error Diagnosis and Clustering
HarnessEvolve diagnoses failures by comparing failed executions with cached reference trajectories, locating root-cause divergences, and clustering resulting error signals.
- Reference-guided diagnosis: For each failed instance, the optimization agent retrieves its reference trajectory from the global cache and compares trajectories to identify the first action divergence.When no reference is available, it analyzes the failed trajectory in isolation.
- Error clustering: Error clustering groups the collected signals into error groups represented by summary state, mechanism, error, and harness information.The output is a set of groups C1 through CK, each encoded as a four-part tuple.
- Error collection: The procedure initializes an empty error batch, processes every failed instance, and adds each diagnosed error signal to the batch.This ensures the clustering stage receives one signal for each processed failure.
C HarnessEvolve Self-Evolution Procedure
HarnessEvolve iteratively proposes, filters, and evaluates harness updates across training batches, retaining accepted snapshots and selecting the best validation performer at epoch end.
- Quality and performance gating: Accepted snapshots enter a pool, while performance gating requires improvement margin δ and controls replay degradation ϵ.The pool supports later selection among accepted agent versions.
- Validation and termination: At epoch end, the procedure selects the best-performing snapshot on validation data, and it can terminate when batch- or epoch-level patience is exhausted.The selected output is the optimized agent A* together with the snapshot pool V.
- Scope of harness optimization: Figure 3 shows that HarnessEvolve edits the entire execution agent across skills, scripts, prompts, tool descriptions, and execution logic rather than one isolated component.The six cases span OfficeQA, SpreadsheetBench, and CloudCoreNetwork-QA.
- Batch evolution: Each batch runs the current agent, evaluates failures, diagnoses and clusters errors, and uses them to generate candidate harness updates.The procedure separates execution, evaluation, diagnosis, optimization, and gating stages.
- Quality and performance gating: The gate agent filters candidate updates using leakage and prompt-bloat bounds before performance-based acceptance.The algorithm exposes η_leak and η_blo as quality-filter parameters.