PaperlayerView exact revision on arXiv

Source-linked AI summary

HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems

Mingju Chen, Can Lv, Guibin Zhang, Heng Chang, Shiji Zhou

arXiv:2606.01779v1cs.CL

TL;DR

LLM agents face heterogeneous task regimes that fixed systems may not handle optimally, while harnesses and policies are usually adapted separately. HarnessForge co-evolves both components as a coupled pair, improving performance across five benchmarks and two backbones by up to 12.0% over the strongest baselines.

  • Problem

    Existing agent adaptation usually optimizes external harnesses or internal policies separately, leaving harness–policy compatibility and full-system adaptation insufficiently characterized.

  • Method

    HarnessForge co-evolves harnesses and policies through fault-guided harness tailoring and harness-conditioned policy alignment.

  • Results

    3.56% average gains and up to 12.0% over the strongest harness-only and policy-only baselines were achieved across five benchmarks and two backbones.

  • Takeaways & Limitations

    Effective agent-system adaptation depends on optimizing executable compatibility between the external harness and internal reasoning policy.

  • Takeaways & Limitations

    HarnessForge is primarily evaluated with Qwen3-4B and Qwen3-8B, so compatibility gains for substantially larger frontier-scale models remain open.

Abstract

from arXiv · show

LLM agents are increasingly expected to operate across heterogeneous task regimes that require distinct execution paradigms. This challenges fixed agent systems and motivates system-level meta-adaptation beyond isolated component updates. While existing works have adapted external harness or trained underlying reasoning policies, full-system adaptation remains insufficiently characterized. The adaptation space between structure and execution is rarely made explicit, and the compatibility between the external harness and the internal reasoner is not optimized jointly. We propose HarnessForge, a meta-adaptive framework for evolving LLM agent systems. HarnessForge formulates an agent system as a harness--policy pair, defining a stable adaptation space that separates harness-level execution structure from policy-level reasoning behavior. It then performs harness--policy co-evolution through fault-guided harness tailoring and harness-conditioned policy alignment. Experiments across five benchmarks from diverse domains show that HarnessForge consistently improves both Qwen3-4B and Qwen3-8B backbones, outperforming harness-only and policy-only baselines with gains of up to 12.0\% over the strongest baseline and achieving favorable rollout-efficiency tradeoffs, demonstrating that harness--policy co-evolution is effective, and that executable compatibility between the harness and reasoning policy is essential for agent-system adaptation. The code is available at https://github.com/mingju-c/HarnessForge.

1 Introduction

Heterogeneous task regimes impose distinct structural requirements on LLM agent execution, making fixed systems and isolated component adaptation insufficient. HarnessForge addresses this by evolving harness–policy pairs through fault-guided harness tailoring and harness-conditioned policy alignment.

  • Motivation: Heterogeneous task regimes require distinct execution structures, including decomposition, verification, action schemas, tool protocols, memory exposure, and state tracking.These differences suggest that no single fixed agent system is likely to perform optimally across regimes.
  • Limitations of Existing Approaches: Existing methods separately adapt external execution structures or internal policies, leaving full-system meta-adaptation insufficiently addressed.Search-style approaches revise workflows and tool procedures, while training-style approaches adapt policies using supervised, preference, or reinforcement learning on agentic trajectories.
  • HarnessForge: HarnessForge defines the coupled harness–policy pair as the basic adaptation unit, with the harness specifying planning, action, and memory interfaces and the policy executing within them.This formulation targets task-regime-adapted agent systems rather than isolated workflows or policies.
  • HarnessForge: HarnessForge co-evolves pairs through fault-guided harness tailoring and harness-conditioned policy alignment to improve pair-level executable compatibility.The framework coordinates external harness evolution with internal policy adaptation across evolutionary rounds.

2 Related Work

Prior work improves agent systems through external harness design or model-side policy evolution. Harness methods structure reasoning and execution, while agentic RL optimizes policies using interactive trajectories, rewards, feedback, and environment signals.

  • Optimization for Agent System Design: External harnesses strongly shape LLM-agent capabilities by organizing reasoning and execution through traces, plans, actions, observations, roles, protocols, and memory modules.Examples include Chain-of-Thought, Plan-and-Solve, ReAct, and AutoGen.
  • Agentic RL for Policy Evolution: Agentic RL methods evolve model-side policies from interactive trajectories using task rewards, tool feedback, or environment signals.This line includes methods for search, tool use, long-horizon execution, planning, memory management, and broader reasoning.
  • Agentic RL for Policy Evolution: Tool-use RL optimizes when and how models issue external actions, while long-horizon methods address credit assignment and trajectory-level optimization.The cited examples include Search-R1, ToolRL, ToRL, GiGPO, TreeRL, and ARPO.
  • Agentic RL for Policy Evolution: RL has also been extended to planning and memory management, alongside broader reasoning-RL systems.Planner-R1 and Memory-R1 target planning and memory management, while DeepseekMATH is cited as a broader reasoning-RL system.

3 Methodology

HarnessForge represents an agent system as an editable execution harness paired with an adapted reasoning policy, then evolves both components jointly through iterative, compatibility-focused rounds. Harness tailoring uses fault attribution and Pareto filtering, while policy evolution aligns each survivor with its harness using successful rollout traces.

  • Agent-System Formulation: An agent system couples an editable harness H with an adapted reasoner Rδ, separating planning, action, memory, and reasoning components.The harness covers task decomposition and termination, tool interfaces and orchestration, and memory writing, retrieval, summarization, and exposure.
  • Joint Evolution: Each evolution round alternates harness tailoring from execution evidence with harness-conditioned policy alignment over the retained survivor population.The two processes are mutually reinforcing: improved harnesses produce more structured trajectories, while aligned policies execute harness protocols more faithfully.
  • Harness Evolution: Harness tailoring locates execution faults, uses archive-guided improvement, edits only planning, action, and memory components, and applies budgeted Pareto selection.Fault reports attribute failures to harness components, while historical designs and evaluation vectors guide candidate generation and filtering.
  • Policy Evolution: Policy evolution trains a harness-specific LoRA update from the parent lineage to align the inherited reasoner with each evolved harness’s execution conventions.Its objective is compatibility with a particular harness rather than training a universally stronger reasoner.
  • Policy Evolution: Successful rollout traces from harness selection are reused as step-level supervision, avoiding a separate data-collection stage and producing matched next-round harness–policy pairs.The supervision packages task instructions, harness interfaces, observations, memory state, and available actions with the corresponding next behavior.

4 Experiments

HarnessForge is evaluated across diverse tool-use and retrieval benchmarks with Qwen3-4B and Qwen3-8B backbones. It outperforms harness-search and policy-training baselines, while ablations and efficiency analyses support joint harness–policy evolution.

  • Benchmarks and setup: HarnessForge evaluates tool selection, API-grounded execution, and retrieval-heavy multi-hop question answering across ToolHop, RestBench-TMDB, API-Bank, and SearchQA.The implementation uses Qwen3-4B and Qwen3-8B backbones, GPT-5.5 for harness evolution, three evolution rounds, and two retained survivor harnesses per round.
  • Main results: +3.56% average improvement over per-metric strongest baselines is achieved across five agentic benchmark settings and two backbone sizes.HarnessForge reaches state-of-the-art results on most benchmarks, while RLOO and GRPO use larger rollout budgets yet fall behind on most metrics.
  • Main results: 12.00% success improvement on TMDB with Qwen3-4B and 6.00% with Qwen3-8B are achieved over the strongest baseline.On API-Bank, HarnessForge improves API accuracy by average 4.96% across backbones.
  • Ablation analysis: Removing either harness tailoring or policy alignment consistently degrades ToolHop and SearchQA performance, with harness tailoring producing the largest drops.Disabling harness tailoring reduces ToolHop/SearchQA performance from -3.07%/-3.00% in Round 1 to -6.15%/-5.00% in Round 3; removing policy alignment causes a final-round -3.00% SearchQA drop.
  • Efficiency and sensitivity: 3.2% points average improvement results from increasing retained harnesses from k = 1 to k = 2 across ToolHop, SearchQA, TMDB, and API-Bank.Further increasing retention to k = 3 provides only marginal additional gains on most benchmarks, while HarnessForge remains on or near the Pareto frontier across benchmark groups.
  • Compatibility analysis: 77.19% final matched-pair performance rises from 69.30% for the base pair, whereas mismatched final components average only 71.93% or 71.06%.These compatibility gaps indicate that HarnessForge produces jointly adapted harness–policy pairs rather than independently stronger components.

5 Conclusion

HarnessForge reframes LLM agent adaptation as joint evolution of an execution harness and reasoning policy rather than isolated component optimization. Its fault-guided and harness-conditioned co-evolution yields consistent benchmark gains, favorable rollout–performance tradeoffs, and strong matched-pair compatibility.

  • 5 Conclusion: HarnessForge reformulates LLM agent adaptation as evolution of a harness–policy pair.This separates the external execution harness from the internal reasoning policy while adapting them as a system.
  • 5 Conclusion: It co-evolves execution and reasoning through fault-guided harness tailoring and harness-conditioned policy alignment.These mechanisms replace optimizing external workflows or internal policies in isolation.
  • 5 Conclusion: Experiments across diverse agent benchmarks show consistent gains over diverse baselines, favorable rollout–performance tradeoffs, and strong matched-pair compatibility.The conclusion identifies these results as evidence for the effectiveness of joint harness–policy evolution.

Limitations … B Datasets Details

HarnessForge’s limitations concern backbone scale, rollout cost, and the constrained scope of its structured meta-evolution protocol. The appendix describes the co-evolution procedure and notation, while no substantive dataset-detail passage is provided here.

  • Limitations: HarnessForge is evaluated primarily with Qwen3-4B and Qwen3-8B, leaving compatibility gains for substantially larger frontier-scale models unresolved.The setting targets resource-constrained deployment, where harness support and policy execution of the induced paradigm are especially salient.
  • Limitations: Repeated rollouts for harness profiling, selection, and policy alignment can make evolution costly in long-horizon environments.Trajectory reuse improves rollout efficiency, while proxy evaluation, adaptive allocation, and learned early stopping are proposed as future remedies.
  • Limitations: The structured meta-evolution protocol constrains edits to planning, action, and memory components rather than exhaustively exploring agent-system implementations.The passage identifies arbitrary code-level rewrites, new tool abstractions, and learned verifier modules as omitted possibilities.
  • A Algorithm and Notation: Each co-evolution round starts from a parent population and evolution batch, collects traces, generates fault reports, and produces improvement briefs.These steps organize harness evolution around rollout evidence and fault-guided improvement.
  • A Algorithm and Notation: The protocol generates child harnesses, discards invalid candidates through interface and smoke tests, and selects survivors using staged Pareto filtering.Surviving harnesses are paired with independently materialized parent-lineage policies and harness-specific adapters.
  • A.1 Notation: The archive is updated with evaluated harnesses, metrics, reports, and selection logs, while the appendix notation summarizes symbols used in the main text and appendix.The provided passages separately document archival bookkeeping and the purpose of Table 4’s notation summary.

B.1 Evaluation Benchmark and Metrics

HarnessForge is evaluated on five datasets spanning four benchmark families, using benchmark-specific measures of answer quality, trajectory/path fidelity, and API-call correctness. The benchmarks cover tool use, retrieval-heavy QA, REST APIs, and structured API calling.

  • Benchmark coverage: Five datasets span four benchmark families: ToolHop, RestBench-TMDB, API-Bank, and SearchQA, with SearchQA containing HotpotQA and 2WikiMultiHopQA.The paper uses “five datasets” for dataset coverage and “four benchmark families” for the benchmark-group columns.
  • ToolHop: ToolHop evaluates multi-hop tool use on 195 held-out instances using Correct for final-answer accuracy and Path for intermediate-subgoal completion.ToolHop tasks require question decomposition, intermediate tool calls, and a final answer; Path averages the fraction of required subgoals solved.
  • SearchQA: SearchQA measures retrieval-heavy multi-hop QA over HotpotQA and 2WikiMultiHopQA with normalized token-level answer F1 and a macro-averaged Overall score.The local evidence corpora are constructed from the two named datasets, and Overall averages their scores.
  • RestBench-TMDB: RestBench-TMDB evaluates REST-style API use on 100 instances with Success for verifier-satisfied answers and Path for matching the required ordered API execution path.A path matches when the gold API-call sequence is preserved as an ordered subsequence of the predicted path.
  • API-Bank: API-Bank evaluates structured API calling on 114 instances using Success, Path based on ordered API-name overlap, and API Accuracy for exact aligned calls and normalized arguments.Success requires the full call trajectory and final response to satisfy the evaluator; API Accuracy requires correct API names and schema-normalized arguments.

B.2 Training & Evolution Data … E.2 Search-Style Baselines

HarnessForge uses a disjoint 3.8K-task pool to evolve executable harnesses and align policies, while evaluating candidates with Pareto-aware performance and efficiency criteria. Its staged, fault-guided tailoring preserves system boundaries, maintains harness–policy lineage, and supports rollout-efficient policy alignment.

  • B.2 Training & Evolution Data; B.3 Split and Deduplication Protocol: 3.8K training tasks span general tool use, complex environment interaction, and offline retrieval, with held-out test splits reserved for final evaluation.The pool is used for harness evolution, trajectory curation, policy-adapter training, and training-style baselines, and is strictly disjoint from evaluation data.
  • C.1 Evaluator Setting: HarnessForge evaluates executable harness–policy systems jointly across task performance, token usage, and wall-clock latency, preferring candidates on Pareto fronts.The evaluator also retrieves archive cases using fault relevance and Pareto quality, with primary task performance breaking otherwise comparable candidates.
  • C.2 Hyperparameter Configuration; C.3 Harness Representation and Edit Space: Candidates are executable code/configuration bundles with a restricted controller edit scope and a shared evaluator, while lightweight validity checks discard incompatible children before rollout.Checks cover required fields, parseable tool schemas, available action mappings, and defined memory keys; the supplied configuration is otherwise shared across benchmarks and backbones.
  • C.4 Meta Tailoring Operator and Prompt Protocol; C.4.1 Fault-Attribution Operation: The meta-tailoring pipeline stages fault diagnosis, improvement planning, executable generation, and smoke-test repair to reduce context drift and isolate generation noise.Fault attribution localizes failures to planning, action, or memory modules and produces transferable, evidence-grounded diagnoses rather than task-specific patches.
  • C.4.2 Archive-Guided Improvement: Archive-guided improvement retrieves historically strong harnesses with similar module failures, then converts them into transferable repair directions instead of copying complete architectures.The brief assigns fixes to planning, action, memory, or cross-module interfaces while specifying reusable patterns, avoidance constraints, and preservation requirements.
  • C.4.3 Refinement and Generation; C.4.4 Retry Mechanism: Generated candidates may revise internal planning, action, memory, and wiring logic but must preserve the dataset, evaluator, backend model, benchmark runner, and task labels.Candidates are required to be self-contained, importable, builder-compatible, and free of hard-coded benchmark shortcuts; smoke-test failures receive up to 3 repair attempts.
  • D Policy Alignment Details; D.1 Policy Lineage and Adapter Operation: Each survivor retains an independent policy branch: its parent state initializes a new harness-specific LoRA adapter trained on successful trajectories collected under that child harness.Sibling survivors do not share trainable adapter parameters after branching, preserving explicit matched harness–policy lineage.
  • D.2 Trajectory Curation and Success Filtering; D.3 Objective Instantiations; E Baselines and Fairness Protocol; E.1 Shared Fairness Protocol; E.2 Search-Style Baselines: Successful rollout traces are reused for policy alignment, and the framework remains objective-agnostic while using SFT by default to imitate harness-conditioned step-level decisions without new rollout collection.SFT supports learning action formatting, tool-use discipline, memory utilization, verification, and termination behaviors from retained successful trajectories.

E.3 Training-Style Baselines

Training-style baselines keep the harness fixed while updating only model-side LoRA adapters, using supervised or reinforcement-learning objectives. RL variants share a function-based verifier reward and controlled rollout settings for fair comparison.

  • Baseline setup: Training-style baselines freeze the base reasoner, keep the harness fixed, and update only LoRA adapters.Final comparisons use held-out test evaluation; Tab. 8 summarizes adapter training, and Fig. 6 provides online diagnostics for RL baselines.
  • SFT: SFT retains trajectories with valid final answers and positive task feedback, then trains LoRA adapters on harness-conditioned interaction histories.Successful trajectories are reused from harness evolution, avoiding an additional exploration stage.
  • Reward design: RL-style baselines use a function-based verifier reward from complete scaffold-conditioned tool trajectories rather than a learned reward model.For ToolHop, final-answer correctness and intermediate-observation path coverage receive equal weights 0.5/0.5, while malformed answers incur a −0.1 penalty.
  • GRPO: GRPO normalizes group-relative advantages within N = 4 completions, while both RL methods apply an actor-side KL loss with coefficient 10−3.KL is excluded from the reward and handled through the low-variance actor-side loss.
  • RLOO: RLOO samples N = 4 trajectories per prompt and computes each advantage against the mean reward of the other trajectories.Its shared clipped objective, token-mean aggregation, entropy coefficient, KL loss, and rollout budget match GRPO to isolate the advantage estimator.

F Reproducibility Artifacts · G Additional Results and Analysis

HarnessForge supports reproducibility by separating structured, auditable harness-evolution stages from replayable final-system evaluation. Its released schemas and contracts also permit protocol re-instantiation with alternative meta-agents without requiring exact regeneration of intermediate candidates.

  • F Reproducibility Artifacts: HarnessForge implements harness evolution as staged operators with fixed input-output contracts rather than unconstrained generation.Stages consume structured inputs such as parent harnesses, rollout evidence, evaluation summaries, and archive records.
  • F Reproducibility Artifacts: Training-style baselines are documented through a dedicated training-configuration artifact.Table 8 is identified as the training configuration for training-style baselines.
  • F Reproducibility Artifacts: Schema-controlled artifacts document failure diagnoses, responsible modules, evidence, edits, repair priorities, and expected behavioral changes.The artifacts include fault reports, evolution manifests, harness-candidate manifests, and smoke-test logs.
  • F Reproducibility Artifacts: Result-level replay releases final harness–policy pairs, evolved harnesses, adapter checkpoints and configurations, benchmark wrappers, evaluation scripts, and split identifiers.These materials enable evaluation without rerunning meta-agent evolution.
  • F Reproducibility Artifacts: Process-level auditability releases prompts, operator schemas, generated reports, candidate manifests, smoke-test outcomes, and survivor-selection records.This complements replay of the final executable systems by exposing the evolution process artifacts.
  • F Reproducibility Artifacts: Exact regeneration of every intermediate candidate is not assumed because meta-agent outputs may vary across model versions and providers.The artifact release therefore distinguishes replaying final executable systems from re-instantiating the evolution protocol.
  • F Reproducibility Artifacts: Released schemas and stage-wise contracts allow alternative closed- or open-source meta-agents to use the same evolution interface and selection procedure.This supports replacing the proprietary meta-agent while preserving the protocol structure.
  • G Additional Results and Analysis: Fault and improvement signals are summarized across benchmarks by planning-, action-, and memory-related repair categories.Figure 7 normalizes each row within a benchmark and groups columns by repair-signal category.

G.1 Adaptation Necessity Analysis

HarnessForge’s compatibility matrices show that matched harness–policy pairs generally outperform mismatched combinations, while progressive co-evolution improves performance and exposes transfer limitations across non-corresponding components.

  • Compatibility evidence: Matched harness–policy pairs generally achieve stronger performance than mismatched combinations across the compatibility matrices.Diagonal entries represent pairs produced by HarnessForge across co-evolution rounds.
  • Compatibility evidence: Performance improves along the diagonal as evolution proceeds, indicating that gains accumulate through progressive harness–policy co-evolution.The matrices cover API-Bank and ToolHop, with rows representing evolved harnesses and columns representing evolved policies.
  • Compatibility evidence: Off-diagonal entries reveal a compatibility gap because later policies do not always transfer cleanly to earlier or non-corresponding harnesses.Strong harnesses may also underperform when paired with mismatched components.

G.2 Module-Level Repair Statistics

Harness failures are module-specific and benchmark-dependent: API-heavy tasks primarily stress action reliability, retrieval and multi-hop tasks stress planning and evidence coupling, while memory repairs provide cross-cutting stability. These patterns motivate targeted module repairs rather than uniform adaptation.

  • Action-side repair: Action-side failures dominate API-Bank and TMDB, including invalid formats, schema mismatches, missing guards, repeated actions, and incorrect endpoint or tool selection.Stricter format contracts, schema preflight, and loop prevention are the most beneficial action-layer repairs for these benchmarks.
  • Planning-side repair: Planning repairs are most useful when retrieval or multi-hop success depends on preserving intermediate intent across several reasoning or retrieval steps.SearchQA requires current-query repair and tighter search–evidence coupling, whereas ToolHop emphasizes preserving multi-hop tool chains and grounding answers in accumulated evidence.
  • Memory repair: Memory signals are smaller but meaningful, typically appearing as co-repairs with planning or action modules rather than as isolated repair targets.Memory quarantine helps prevent stale or irrelevant traces from degrading execution, functioning as a stabilizing layer.
  • Cross-benchmark pattern: The repair-pressure division is clear: API tasks stress action reliability, search tasks stress query planning and evidence coupling, multi-hop tool-use tasks stress path preservation, and memory repairs stabilize execution.This summarizes the benchmark-dependent module patterns identified by the heatmap analysis.

G.3 Case Study

HarnessForge converts recurring trajectory-level failures into reusable repairs across planning, action, and memory. Five case studies show improvements through evidence interfaces, API contracts, query grounding, schema guards, and endpoint routing rather than task-specific answer patches.

  • Case Study: Five representative parent–child trajectories show HarnessForge turning recurring failures into reusable repairs over planning, action, or memory.The cases span multi-hop comparison, structured API execution, retrieval-heavy QA, tool-chain execution, and REST-style endpoint routing.
  • Case Study: The multi-hop comparison repair replaces repeated unsupported finalization with entity-level evidence slots and support-record verification.This makes finalization evidence-supported rather than repeatedly unsupported.
  • Case Study: The API-route repair enforces the authentication–deletion sequence and emits the expected API-request output format.The repair addresses both execution order and output formatting for account deletion.
  • Case Study: The retrieval repair avoids stale repeated searches by using current-query, question-specific evidence and support-record finalization.This targets retrieval-heavy multi-hop question answering.
  • Case Study: Across the cases, HarnessForge modifies reusable interfaces—including evidence slots, schema guards, support records, API contracts, and memory quarantine—instead of applying task-specific answer patches.Other repairs preserve source–intermediate–transform chains while normalizing invalid date-tool arguments, or route directly to correct TV-detail endpoints while avoiding unnecessary season-level calls.
Loading 2606.01779v1…