PaperlayerView exact revision on arXiv

Source-linked AI summary

When the Critic Falls Behind: Co-Evolving Rubrics and Harnesses for Continual Agent Self-Improvement

Hao Chen, Yukun Yan, Ziyu Han, Ye He, Chuanzhe Guo, Shangzhan Li, Qi Shi, Qingfu Zhu, Maosong Sun, Wanxiang Che

arXiv:2610.36826v1cs.AI

TL;DR

Harness evolution can improve agents, but static guidance may not track behavioral weaknesses as they shift, a concern for generalization to unseen tasks. RuHarness co-evolves trajectory-based rubrics and targeted harness updates; experiments report gains over strong baselines, particularly on held-out tasks.

  • Problem

    Harness-evolution guidance often remains static, while behavioral weaknesses shift across rounds, and generalization to unseen tasks is a concern.

    Source: §1 · p. 1

  • Method

    RuHarness continually derives and reassesses trajectory-based behavioral criteria, prioritizes unresolved weaknesses, and uses them to guide targeted harness updates while keeping the task objective fixed.

    Source: §1 · p. 2

  • Results

    RuHarness achieves strong gains in held-in performance and generalization to unseen held-out tasks across diverse tasks and backbone models.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The rubric frontier tracks shifting diagnostic relevance and supports sustained harness improvement over repeated evolution rounds.

    Source: §6 · p. 8

    Experiments use a single underlying agent framework and benchmark tasks under a fixed evolution horizon, leaving broader architectures and longer-term, open-ended adaptation for future evaluation.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Autonomous agents increasingly depend on harnesses that govern interaction, memory, tools, and execution around large language models (LLMs). While recent work has begun to evolve these harnesses from execution experience, the supervision used to guide evolution often remains relatively static, making it difficult to track changing behavioral weaknesses across rounds and potentially limiting generalization to unseen tasks. To address this issue, we propose RuHarness, a rubric--harness co-evolution framework for continual agent self-improvement. RuHarness continually discovers and reassesses behavioral criteria from execution trajectories, prioritizes the most relevant unresolved weaknesses, and uses them to guide targeted harness updates while keeping the task objective fixed. Extensive experiments across diverse agent tasks and backbone models show that RuHarness outperforms strong harness-evolution baselines, particularly on unseen held-out tasks. Further analyses show that adaptive rubric prioritization tracks shifting weaknesses and supports more sustained improvement and stable generalization across evolution rounds.

1 Introduction

Harness evolution can improve agents, but static or coarse diagnostic guidance may fail to track shifting weaknesses, raising concerns about generalization to unseen tasks. RuHarness co-evolves rubrics and harnesses, and experiments report stronger performance, especially on held-out tasks.

1 Introduction

  • As harnesses improve, previously dominant failures may recede while subtler weaknesses emerge, making static guidance less useful and leaving changing weakness priorities untracked.The paper identifies adaptive diagnosis and reprioritization, with the task objective fixed, as a need for continual harness evolution.Source: §1 · pp. 1–2
  • RuHarness derives and reassesses behavioral criteria from trajectories, prioritizes unresolved weaknesses, and uses the resulting rubric frontier to guide targeted harness updates.Candidate harnesses are selected using task performance on an independent validation set.Source: §1 · p. 2
  • RuHarness consistently outperforms strong harness-evolution baselines across diverse tasks and backbones, with particularly clear gains on unseen held-out tasks.The analyses also report that rubric prioritization tracks shifting weaknesses and supports more sustained improvement across evolution rounds.Source: §1 · p. 2

2 Related Work

Prior work evolves agent memory, skills, and broader harness infrastructure, while rubric research increasingly adapts criteria alongside policies or responses. Rubric adaptation for continual harness evolution remains largely unexplored.

Self-Evolving Agents

  • Agent self-evolution has expanded from refining interaction experience, memory, and reusable skills to optimizing broader harness infrastructure such as prompts, tools, memory, and control logic.Recent harness methods revise infrastructure using trajectory-based failure diagnosis and increasingly observable or verifiable updates.Source: §2 · p. 2

Rubrics for Self-Improvement

  • Adaptive rubric work co-evolves criteria with policies or responses, but rubric adaptation for continual harness evolution remains largely unexplored.Rubrics are also used as fine-grained supervision in reward modeling and reinforcement learning.Source: §2 · p. 2

3 Method

RuHarness alternates rubric evolution with harness updates, using behavior from each round to inform the next. Its rubric process discovers, assesses, and prioritizes criteria, while its harness process targets and validates coordinated updates.

3.1 Overview of the Framework

  • At each round, the current harness generates trajectories that inform rubric updates and prioritization, which then guide harness changes and produce new behavior for the next cycle.The rubric and harness are coupled through agent behavior: the harness shapes emerging weaknesses, and the rubric determines which are prioritized.Source: §3.1 · p. 3

3.2 Rubric Evolution: Adapting to Shifting Agent Weaknesses

  • Rubric evolution turns recurring trajectory failure patterns into criteria, assesses behavior without revealing task outcomes to the judge, and checks whether criteria provide diagnostic value.New criteria must pass an admission check before entering a lifecycle of probation, active, dormant, and retired states; dormant criteria can be reactivated if failures recur.Source: §3.2 · pp. 3–4
  • Time-varying utility combines behavioral discrimination, current-failure coverage, failure prediction, novel failures, and redundancy to reprioritize criteria as the harness changes.A conservative posterior estimate and top-K selection prioritize unresolved weaknesses that are diagnostically informative and statistically reliable.Source: §3.2 · p. 4

3.3 Harness Evolution: Optimizing against the Moving Frontier

  • The harness is organized into six functional coordinates, and each prioritized weakness can target multiple coordinates for joint intervention.The coordinates cover resource provisioning, instruction, action, observation, context management, and completion control.Source: §3.3 · p. 4
  • RuHarness plans coordinated multicoordinate candidates and verifies their consistency with diagnosed weaknesses before evaluation.The verifier accepts or rejects candidates but does not rewrite them or estimate task performance.Source: §3.3 · pp. 4–5
  • Candidates are compared with the current harness on a separate validation set, and an update is committed only when its improvement exceeds a threshold.The validation set is excluded from evolution and final evaluation; otherwise the current harness is retained.Source: §3.3 · p. 5

4 Experimental Methodology

Experiments use two backbone models in a DeepAgent framework and evaluate agent performance on a five-domain benchmark. Evolution, validation, and held-out test data are separated, and comparisons include agent self-evolution and harness-evolution baselines.

Models and Environment

  • Experiments use Qwen3.5-35B-A3B and GLM-4.7-Flash as fixed backbones within the DeepAgent execution framework.The configurable initial harness is evolved while the underlying model remains fixed.Source: §4 · p. 5

Benchmarks and Metrics

  • Evaluation uses Pass@1 on five EvoAgentBench domains: information retrieval, mathematical reasoning, software engineering, code implementation, and knowledge work.Each domain uses its benchmark evaluator.Source: §4 · p. 5

Baselines

  • RuHarness is compared with ACE and EvoSkill as well as Self-Harness, Meta-Harness, and Agentic Harness Engineering.The baselines cover context, skill, and full-harness evolution.Source: §4 · p. 5

Implementation Details

  • Evolution and validation splits are disjoint, and the held-out test set is reserved for final reporting.Each round generates up to three coordinated candidates, accepts at most one update above a validation improvement threshold of 0.05, and selects among the top 10 rubric criteria.Source: §4 · p. 5

5 Results and Analysis

RuHarness leads the reported held-in and held-out averages across backbones, while ablations and trajectory analyses show the value of adaptive rubric guidance. The analyses also find that frontier priorities track changing diagnostic relevance and that validated harness updates accompany sustained mitigation.

5.1 Overall Performance

  • RuHarness achieves the best average held-in and held-out performance across both backbones, improving the strongest baseline by 1.7 and 4.5 points on Qwen3.5-35B-A3B and 2.7 and 5.3 points on GLM-4.7-Flash, respectively.The table compares performance across five domains and reports macro-averages across them.Source: Table 1 · p. 6, §5.1 · p. 6

5.2 Ablation Study

  • Removing the rubric lowers scores, a fixed rubric performs worse still, and using the entire ranked pool instead of the frontier also degrades performance.Removing the rubric lowers held-in and held-out averages from 62.4 to 57.1 and 53.7 to 45.8; fixed criteria can remain relevant after dominant failures are mitigated, while the full pool adds low-value signals.Source: §5.2 · p. 6

5.3 Analysis of Adaptive Rubric Tracking

  • Frontier similarity to the initial rubric falls sharply and stays low, while similarity between consecutive rounds rises and stabilizes, indicating that diagnostic priorities change over evolution.The analysis compares frontier criterion IDs across rounds in two representative model-task settings.Source: §5.3 · p. 7, Figure 3 · p. 7

5.4 Effectiveness of Adaptive Rubric Prioritization on Harness Evolution

  • Criteria entering the frontier gain discrimination, coverage, and predictive validity while reducing redundancy; exiting criteria show the opposite pattern.Criteria that remain inside or outside the frontier change substantially less, so turnover tracks changing diagnostic relevance rather than arbitrary replacement.Source: Figure 4 · p. 7, §5.4 · p. 8
  • After frontier entry, violation rates stay nearly unchanged in the first round and then decrease over subsequent rounds, consistent with delayed mitigation of prioritized weaknesses.The reported trend links prioritization with later behavioral improvement, without implying immediate resolution.Source: Figure 4 · p. 7, §5.4 · p. 8
  • Committed updates show a sustained increase in violation reduction, whereas non-committed updates fluctuate without a consistent improvement trend.Committed candidates pass the validation gate and are adopted for subsequent evolution.Source: Figure 4 · p. 7, §5.4 · p. 8

5.5 Evolution Progress and Generalization

  • On held-in tasks, baselines often plateau after early gains, while RuHarness continues improving in later rounds and reaches the highest performance.The reported trajectories use the Qwen3.5-35B-A3B model on the knowledge work task.Source: Figure 5 · p. 8, §5.5 · p. 8
  • On held-out tasks, baselines fluctuate more across rounds, while RuHarness trends upward more steadily, indicating more stable transfer beyond held-in tasks.Together, the trajectories indicate sustained held-in improvement and more stable held-out generalization throughout evolution.Source: Figure 5 · p. 8, §5.5 · p. 8

6 Conclusion

RuHarness adapts diagnostic criteria to evolving behavioral weaknesses and uses them to guide harness optimization, with experiments showing gains across tasks and backbone models.

6 Conclusion

  • RuHarness achieves strong gains in held-in performance and generalization to unseen held-out tasks across diverse tasks and backbone models.Further analyses show the rubric frontier tracks shifting diagnostic relevance and supports sustained harness improvement over repeated evolution rounds.Source: §6 · p. 8
  • RuHarness dynamically adapts diagnostic criteria to the agent’s evolving behavioral weaknesses and uses the rubric frontier to guide targeted harness optimization.Source: §6 · p. 8
  • The paper concludes that jointly adapting diagnostic supervision and the harness supports more sustained and generalizable agent evolution.Source: §6 · p. 8

Limitations

The evaluation spans multiple tasks and backbone models but is limited to one agent framework and benchmark tasks under a fixed evolution horizon.

Limitations

  • All experiments use a single underlying agent framework, so evaluation across broader agent architectures remains necessary to establish the six-coordinate harness abstraction’s generality.Source: Limitations · p. 9
  • Computational resource constraints limit experiments to benchmark tasks under a fixed evolution horizon.Source: Limitations · p. 9
  • Longer-term and more open-ended adaptation settings remain unevaluated and would provide a broader view of co-evolution dynamics.Source: Limitations · p. 9

A.1 Experimental Details

The experimental details specify task splits, controlled evaluation conditions, the editable harness abstraction, and prompts used in co-evolution.

Dataset Statistics and Settings

  • Experiments use five EvoAgentBench-derived tasks spanning information retrieval, knowledge work, software engineering, code implementation, and mathematical reasoning.Table 3 summarizes dataset statistics for evolution, validation, and test splits across the five domains.Source: §A.1 · p. 12, Table 3 · p. 12
  • Evolution data drives rubric–harness evolution, validation data selects candidates, and the original test split is reserved for held-out evaluation.All splits remain fixed and identical across baselines; evolution rounds, candidate budgets, and validation-based selection are matched unless specified otherwise.Source: §A.1 · p. 12

Harness Abstraction and Editable Interfaces

  • Six functional harness coordinates map to framework interfaces for skills and tools, prompts, action logic, observation processing, memory, and termination.Table 4 summarizes the coordinates, their functional scopes, and editable contents; methods share the same initial harness while the environment, task specification, APIs, and evaluation protocol remain fixed.Source: §A.1 · p. 12, Table 4 · p. 12

Prompt Templates

  • The co-evolution procedure provides prompts for trajectory analysis, rubric proposal, outcome-blind judging, and harness intervention planning.Source: §A.1 · p. 12

A.2 Details of Diagnostic Utility

Diagnostic utility is built from outcome-blind criterion assessments and five statistics that characterize how criteria relate to task failures and one another.

Assessment counts

  • For each criterion, an outcome-blind judge labels trajectories as violated, satisfied, or not applicable, with counts tracked separately for successful and unsuccessful tasks.Source: §A.2 · p. 12

Discrimination and Coverage

  • Discrimination compares criterion violation rates across unsuccessful and successful trajectories, while coverage measures the proportion of unsuccessful trajectories identified.Source: §A.2 · p. 12

Predictive Validity

  • Predictive validity measures whether a violation raises task-failure likelihood relative to the criterion’s applicable population.It is defined as Pr(fail | violated)−Pr(fail | applicable).Source: §A.2 · p. 13

Novelty and Redundancy

  • Novelty measures the share of failed trajectories uniquely identified by a criterion, while redundancy measures its behavioral overlap with other criteria.Source: §A.2 · p. 13

A.3 Detail of Rubric Admission and Lifecycle

The rubric pool filters new criteria for diagnostic usefulness and maintains admitted criteria through a four-state lifecycle. Its active set turns over as behavioral weaknesses and criterion relevance change.

Birth Check

  • New criteria fail birth checks if over 80% of assessments are N/A, they never violate unsuccessful runs, or violations exceed 90% overall and on successful runs.Criteria passing the checks enter probation and may become active within the same round if they satisfy the promotion condition.Source: §A.3 · p. 13

Lifecycle Transition Mechanism

  • Admitted criteria enter probation, become active at P > 0, and retire if untriggered for four consecutive rounds.Transitions are evaluated after each evolution round using observed diagnostic statistics.Source: §A.3 · p. 13
  • Active criteria become dormant after trigger rates below 5% for three rounds or sustained N/A drift, and retire if predictive validity stays below 0.10 for three rounds.Dormant criteria reactivate when trigger rates reach 15%; otherwise, four consecutive dormant rounds lead to retirement.Source: §A.3 · p. 13

Visualization of Lifecycle Dynamics

  • Active membership repeatedly reorganizes as new criteria enter and others become dormant or retired, rather than accumulating monotonically.Dormant criteria may return when failures re-emerge; the visualization also counts criteria rejected at birth as retired for bookkeeping, though they do not formally enter the lifecycle.Source: §A.3 · pp. 13–14

A.4 Details of Posterior Estimation and Conservative Ranking

Across these analyses, the prioritized frontier concentrates criteria with stronger diagnostic value, while dynamic rubrics retain failure coverage and a smaller generalization gap. Case studies show diagnostic focus and harness interventions becoming more specific across rounds.

A.4 Details of Posterior Estimation and Conservative Ranking

  • A symmetric Jeffreys prior adds 0.5 per contingency-table cell; 2,000 posterior draws yield a lower 0.2 quantile for ranking active criteria.Discrimination, coverage, and predictive validity are recomputed for each draw, while novelty and redundancy remain fixed.Source: §A.4 · p. 14

A.5 Robustness to Random Seeds

  • RuHarness maintains the best average held-in and held-out performance across three seeds in both representative backbone–task settings, with consistent gains and comparable variance.The settings are Qwen3.5-35B-A3B on Knowledge Work and GLM-4.7-Flash on Software Engineering.Source: §A.5 · pp. 14–15, Table 6 · p. 15

A.6 Effectiveness of Dynamic Evolution

  • Dynamic rubrics maintain a smaller held-out–held-in generalization gap and high failure coverage, while static rubrics fluctuate and lose coverage in later rounds.The gap is held-out performance minus held-in performance; frontier coverage is the share of failed trajectories captured by at least one prioritized criterion.Source: §A.6 · p. 15, Figure 7 · p. 15

A.7 Effectiveness of Rubric Frontier

  • 0.442 vs 0.215 median diagnostic value for the frontier and all eligible rubrics, respectively; non-positive diagnostic value occurs in 0.5% vs 29.9%.The full pool contains the frontier, so the shift indicates increased concentration of informative criteria rather than simply a smaller set.Source: §A.7 · p. 16, Figure 8 · p. 16

A.8 Case Study of Rubric–Harness Co-Evolution

  • Knowledge Work progresses from execution checks to diagnosis, executable artifact verification, and pre-access checks; Information Retrieval advances toward auditable constraint matrices.Across rounds, diagnostic focus becomes more specific and corresponding harness updates shift from coarse checks to operational, structured interventions.Source: §A.8 · pp. 16–17, Table 7 · p. 17

Related papers

Loading 2610.36826v1…