PaperlayerView exact revision on arXiv

Source-linked AI summary

HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention

Han Luo, Bingbing Wen, Guang Yang, Zora Zhiruo Wang, Pan Lu, Lucy Lu Wang

arXiv:2610.06563v1cs.AI

TL;DR

LLM agents can fail to recognize when tool-use tasks are infeasible, while harness optimization against fixed tasks reveals only a limited set of failures. HERA creates verified feasible–infeasible task pairs and co-evolves environments and harnesses using execution failures. It improves held-out abstention and feasible-task performance and transfers across other LLMs without model-specific optimization.

  • Problem

    Agents may fail to recognize when tool-use tasks are infeasible, and optimizing harnesses against fixed tasks exposes only a limited set of failures.

    Source: §1 · p. 1

  • Method

    HERA constructs matched feasible and infeasible tasks through controlled environment mutations and uses paired execution failures to guide harness adaptation and new environment-task generation.

    Source: §1 · p. 2, §3.1 · p. 4, §3.2 · p. 5

  • Results

    Co-evolution achieves the highest Abstain (83.3%) and Act (76.7%) among compared methods, improving abstention without sacrificing feasible-task success.

    Source: §5 · p. 7

  • Takeaways & Limitations

    Across 19 additional LLMs, the final harness improves abstention and paired success by averages of 15.3 and 12.5 percentage points without further model-specific optimization.

    Source: §1 · p. 2

Abstract

from arXiv · show

Large language model (LLM) agents are increasingly capable of acting in complex tool-use environments, yet they often fail to recognize when tasks are infeasible and no valid solution exists. Recent work has formalized this reliability gap as the problem of agentic abstention, and existing approaches typically optimize a model or agent harness against a fixed set of tasks, leading to limited generalization to unseen failure modes. We introduce HERA, a framework for harness-environment co-evolution for agentic abstention. HERA consists of (i) a pipeline to automatically construct verifiable pairs of feasible and infeasible tasks by applying controlled environment mutations that transform solvable tasks into cases requiring abstention, and (ii) a co-evolution procedure in which performance failures on previous tasks are used to drive harness adaptation and generate new execution environments and tasks geared towards previous weaknesses. On held-out evaluation tasks, an evolved harness from HERA improves abstention accuracy from 61.7% to 83.3% while improving feasible-task completion from 68.3% to 76.7%, achieving the highest abstention and feasible-task completion among the compared methods. The resulting best harness transfers across 19 other LLMs, improving abstention accuracy by 15.3 percentage points on average without any model-specific optimization, and enabling smaller models to match the performance of more powerful models at an estimated 85% lower cost.

1 INTRODUCTION

Agentic abstention remains underexplored in harness optimization, and fixed task sets provide diminishing evidence about remaining failures. HERA pairs verified feasible and infeasible tasks with failure-driven co-evolution of harnesses and environments, improving held-out performance and transferring across models.

1 INTRODUCTION

  • Fixed task sets expose only a limited set of failures, so continued harness improvement requires task distributions that evolve to challenge current capabilities.HERA uses harness failures to shape new task cases and environment changes.Source: §1 · pp. 1–2
  • HERA constructs matched feasible–infeasible task pairs through controlled environment mutations, then uses execution failures to guide harness adaptation and subsequent environment-task generation.Each pair shares the same request; environment mutation determines whether it can be completed.Source: §1 · p. 2
  • On HERA-BENCH, HERA improves abstention success from 61.7% to 83.3% and paired success from 48.3% to 70.0% relative to the base harness.These held-out results are reported alongside comparisons with abstention prompting, harness-only optimization, and alternative task-expansion strategies.Source: §1 · p. 2
  • On the final training pool, HERA achieves 86.0% abstention success and 78.0% paired success, exceeding the next-best baseline by 10.0 percentage points on both metrics.The reported results support harness and environment/task updates, as well as multiple iterations for both.Source: §1 · p. 2
  • Across 19 additional LLMs, the final harness improves abstention and paired success by averages of 15.3 and 12.5 percentage points without further model-specific optimization.For example, GPT-5.6-Luna with HERA matches base GPT-6-Astra's 70.0% paired success at an estimated 85% lower API cost per episode.Source: §1 · p. 2

2 RELATED WORK

Prior work studies abstention in language models and agents, and optimizes agent harnesses, but typically uses fixed or independently generated environments. HERA instead adapts the environment–task ecosystem to weaknesses revealed by the current harness.

2 RELATED WORK

  • LLM abstention is described as refraining from answering under uncertainty; prior studies group its motivations into insufficient information, knowledge limitations, and safety or value concerns.These categories frame language-model abstention research distinct from the paper's focus on agents stopping in executable environments.Source: §2 · p. 3
  • Earlier agentic-abstention studies evaluate when agents should stop across QA, web, and terminal settings or construct executable pairs, while their environments are fixed or generated independently of agent performance.Related work also studies whether agents can estimate remaining resource requirements and recognize when task completion is unlikely within budget.Source: §2 · p. 3
  • Harness-evolution research ranges from optimizing prompts and demonstrations to searching or jointly evolving full harness components, with some work adapting validation-task selection as the harness changes.The cited literature also raises whether gains generalize beyond tasks repeatedly used during optimization.Source: §2 · p. 3
  • Unlike prior harness-evolution methods that typically optimize against fixed environments, HERA evolves the harness with the environment–task ecosystem to target weaknesses exposed by the current harness.The paper applies this approach to agentic abstention.Source: §2 · p. 3

3 METHODOLOGY: HARNESS-ENVIRONMENT CO-EVOLUTION

HERA constructs verified feasible–infeasible task pairs, then co-evolves task environments and the harness using diagnosed failures. Its paired rollouts and validation procedures target current weaknesses while retaining checks on feasibility and performance.

3.1 VERIFIABLE ABSTENTION ENVIRONMENT–TASK CONSTRUCTION

  • Stage 1 constructs executable environments from themes and structured databases, then validates generated tools against the database schema.The construction procedure synthesizes an executable interface over the resulting data.Source: §3.1 · pp. 3–4
  • An independent solver attempts proposed tasks, retaining only tasks successfully completed and validated in the environment as Qθ.The validated tasks include executable reference trajectories for assessing later mutations.Source: §3.1 · p. 4
  • Multiple proposers generate environment mutations, which semantic-consistency and rescue policies review before accepted proposals are applied to the original environment.Proposals failing review enter a refinement loop; the mutated environment is used for downstream abstention evaluation.Source: §3.1 · p. 4

3.2 HARNESS–ENVIRONMENT CO-EVOLUTION

  • Each round uses paired task rollouts and grouped failure diagnoses to guide environment and harness evolution.Environment designers target diagnosed weaknesses, and proposed task pairs are certified before use.Source: §3.2 · p. 5
  • New feasible–infeasible task batches enter the cumulative pool only when the current harness's failure-to-abstain rate exceeds threshold τ.Batches that miss the threshold are revised or replaced.Source: §3.2 · p. 5
  • Harness candidates are iteratively optimized from updated failure diagnoses while the expanded training pool stays fixed during each update sequence.Candidates build on their predecessors, and final selection adopts only a validation-set candidate that improves at least one metric without decreasing the other.Source: §3.2 · p. 5

4 EXPERIMENTAL SETUP

The experiments evaluate co-evolution against harness and task-generation baselines on cumulative training pools and a held-out benchmark. They examine abstention, feasible-task completion, component contributions, and retention across task expansions.

Experimental questions

  • The experiments ask whether co-evolution improves abstention without reducing feasible-task completion, how each evolution component contributes, and whether performance extends to new tasks while retaining earlier performance.Training-dynamics measurements track new batches from introduction and compare performance before and after harness updates.Source: §4 · pp. 5–6

Training and evaluation data

  • Training starts with 20 task pairs and adds 30 pairs across six rounds, expanding the cumulative pool to 50 pairs.Each iteration adds five feasible and five infeasible tasks; the batch-admission threshold is τ = 0.4.Source: §4 · p. 6
  • HERA-BENCH contains 60 feasible–infeasible task pairs from training-disjoint domains, manually reviewed for plausibility, instruction-environment consistency, and pair validity.The review was conducted by the authors.Source: §4 · p. 6

Harness and models

  • The base harness is a minimal tool-calling agent evaluated with GPT-5.6-Luna and, without model-specific optimization, 19 additional LLMs.The harness alternates model responses and tool calls, with a 30-turn execution limit.Source: §4 · p. 6

Baselines

  • Baselines include an unadapted harness, abstention prompting, harness-only evolution, task-augmented harness evolution, and task generation from initial harness failures.These comparisons isolate harness evolution, environment evolution beyond adding tasks, and continued co-evolution beyond initial failures.Source: §4 · p. 6

Evaluation metrics

  • Abstain measures correct abstention on infeasible tasks, Pair requires success on both matched cases, and Act measures completion of feasible tasks.Table 2 reports Abstain / Act across cumulative pools T0–T6 using each method's same final harness.Source: §4 · p. 6, Table 2 · p. 7

5 RESULTS

HERA co-evolution achieves the highest reported Abstain and Act scores among compared methods. The results also examine performance across evolving task batches, model costs, and harness updates.

5 RESULTS

  • 83.3% Abstain and 76.7% Act are the highest scores among compared methods, with co-evolution improving abstention without sacrificing feasible-task performance.Table 1 compares methods using the same base model and evaluation tasks.Source: §5 · p. 7, Table 1 · p. 7
  • On T6, co-evolution reaches 86.0% Abstain and 88.0% Act, versus 76.0% and 82.0% for task generation from initial failures.The methods match on T0 and T1, then diverge on later rounds.Source: §5 · p. 7
  • Across 19 other models, the fixed harness improves Abstain by 6.7–40.0 percentage points and Pair by 1.7–28.3 points.GPT-6-Astra rises from 81.7% to 91.7% Abstain and from 70.0% to 80.0% Pair.Source: §5 · p. 7, Table 3 · p. 8

5.1 CO-EVOLUTION DYNAMICS

  • New task batches tend to start at lower performance than earlier batches; after B6 is incorporated, H6 raises its Abstain and Act from 60.0% to 80.0%.Figure 2 tracks fixed, non-overlapping batches across harness updates.Source: §5.1 · p. 7, Figure 2 · p. 8
  • Act performance on earlier batches is generally stable across harness evolution, with some fluctuations and a slight overall positive trend.Task generation does not explicitly target Act performance.Source: §5.1 · p. 8

5.2 PERFORMANCE–COST TRADE-OFFS

  • GPT-5.6-Luna with HERA achieves 70.0% paired success at an estimated $0.0091 per episode, matching GPT-6-Astra’s base harness at $0.0610.The estimated API cost is approximately 85% lower; across 14 models, HERA increases API cost and token usage by median factors of 2.09 and 3.31.Source: §5.2 · p. 8, Figure 3 · p. 9

5.3 ANALYSIS OF CHANGES INTRODUCED THROUGH CO-EVOLUTION

  • Successive harness updates add or refine evidence-based stopping, cross-source reconciliation, joint constraint checks, and authorization and privacy checks.Figure 4 presents a representative sequence of updates responding to diagnosed failures.Source: §5.3 · p. 8, Figure 4 · p. 9
  • Evolved tasks vary across rounds: some add interacting constraints or entities, while others change evidence availability, permissions, or whether restrictions apply.The emphasis of these challenge types also shifts across rounds.Source: §5.3 · p. 9

6 CONCLUSION

HERA co-evolves harnesses with their task environments for agentic abstention. The authors report improvements in abstention, feasible-task performance, and transfer across models.

6 CONCLUSION

  • HERA improves abstention while maintaining feasible-task completion, generalizes to held-out task pairs, and transfers across 19 additional LLMs without model-specific optimization.The authors describe broad improvements in abstention and paired success.Source: §6 · p. 9
  • HERA constructs verifiable matched feasible and infeasible task pairs through controlled environment mutations, then uses harness failures to guide task generation and harness adaptation.This lets the training environment evolve alongside the agent rather than remain fixed during optimization.Source: §6 · p. 9
  • The authors present HERA as a step toward more reliable and adaptive evaluation and optimization of LLM agents.Source: §6 · p. 9

AI USE STATEMENT

The authors disclose using LLMs in the proposed framework and generative AI tools during implementation, analysis, and manuscript editing. They report reviewing and verifying all AI-assisted outputs and taking responsibility for the paper.

AI USE STATEMENT

  • LLMs were used as components of the proposed framework, as described in the paper.Source: AI USE STATEMENT · p. 10
  • Generative AI tools assisted with implementation, analysis, and manuscript editing.Source: AI USE STATEMENT · p. 10
  • The authors reviewed and verified all AI-assisted outputs and take full responsibility for the paper’s methodology, results, and final content.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The study examines agentic abstention with synthetic tasks and controlled executable environments. It involves neither human subjects nor private personal data, and the authors report adhering to the ICLR Code of Ethics.

ETHICS STATEMENT

REPRODUCIBILITY STATEMENT

The paper and appendices document framework, construction, validation, evaluation, and implementation details. The authors state these materials are intended to enable reproduction of the reported results.

REPRODUCIBILITY STATEMENT

A.1 ENVIRONMENT–TASK CONSTRUCTION PIPELINE

HERA builds verified feasible–infeasible task pairs through controlled environment changes, then co-evolves task pools and harnesses using failure feedback. The protocol specifies how environments, tasks, harnesses, and evaluation checkpoints are constructed and constrained.

A.1 ENVIRONMENT–TASK CONSTRUCTION PIPELINE

  • Environment construction, task generation, and controlled mutation produce matched feasible–infeasible pairs while preserving the original request.Candidate mutations are checked for alternative solution paths before application.Source: §A.1 · p. 15, Figure 5 · p. 16

A.2 MODELS AND INFERENCE SETTINGS

  • GPT-5.6-Luna supports harness development and main experiments; the frozen final harness is also evaluated on 19 additional models without fine-tuning.Models receive the same task instructions, tool definitions, initial states, and execution limits; episodes start with reset harness state.Source: §A.2 · p. 16

A.3 HERA OPTIMIZATION DETAILS

  • The protocol starts with 20 validated pairs, adds five per round across six rounds, and reaches a 50-pair training pool while keeping validation fixed.The fixed validation set is used for checkpoint selection.Source: §A.3 · p. 16
  • Training failures guide both new task-pair generation and sequential harness candidates; a candidate is retained only if fixed-validation Abstain and Pair do not decrease and one improves.The final selected checkpoint is frozen and evaluated on HERA-BENCH, which is excluded from generation, revision, and selection.Source: §A.3 · pp. 16–17

A.4 HARNESS REPRESENTATION AND EDITABLE COMPONENTS

  • Harnesses are versioned bundles of instructions, configuration, and executable modules; optimization may revise prompts, memory, tools, skills, and control logic.Environment tools, task specifications, scoring rules, data splits, and execution budgets remain outside the editable scope.Source: §A.4 · p. 18

B.1 ENVIRONMENT CONSTRUCTION

  • Environments combine source-grounded entities, relations, workflows, and constraints with generated state and tools whose behavior is checked before acceptance.Validation checks schemas, resource discovery, valid and invalid inputs, unintended writes, and partial updates; accepted components are versioned together.Source: §B.1 · p. 18

B.2 FEASIBLE TASK GENERATION AND VERIFICATION

  • Tasks specify supported user goals without prescribing a solution, and are accepted only after a separate solver succeeds and the result passes verification and replay.Verification rules are authored before solver execution and allow alternative valid execution paths.Source: §B.2 · p. 18

B.3 ENVIRONMENT MUTATION AND PAIR CERTIFICATION

  • Mutation changes a task-relevant environment condition while retaining the task instruction and public interface; certification rejects cases with valid completion paths or unresolved validity issues.Reversing the mutation must restore solvability, and nonessential-change controls test whether irrelevant differences preserve feasibility.Source: §B.3 · pp. 18–19

B.4 FAILURE-GUIDED ENVIRONMENT–TASK EVOLUTION

HERA uses diagnosed training failures to generate and screen new task pairs, while keeping validation and benchmark data separate. The section also defines the dataset splits, evaluation metrics, and review checks used to assess task validity.

B.4 FAILURE-GUIDED ENVIRONMENT–TASK EVOLUTION

  • Failures on solvable and should-abstain tasks are diagnosed separately, then used to propose new environments and matched task pairs targeting identified weaknesses.Only training feedback informs diagnosis and generation; validation and evaluation tasks and trajectories are excluded.Source: §B.4 · p. 19
  • A batch enters the training pool when at least two of its five infeasible instances reveal confirmed abstention failures; all five certified pairs are retained.Screening reviews trajectories for task defects, runtime errors, and scoring ambiguity before admission.Source: §B.4 · p. 19

C.1 DATASETS AND SPLITS

  • The training pool grows from 20 to 50 pairs, validation uses 15 pairs, and HERA-BENCH provides 60 pairs for cross-model comparisons.Both variants of each pair share the public instruction and tool interface, and direct derivatives remain in the same partition.Source: §C.1 · p. 19

C.2 EVALUATION METRICS

  • Abstain and Pair are primary metrics, while Act and Over-abstain provide complementary diagnostics; Over-abstain is not simply the complement of Act.Each pair contains a solvable and a should-abstain instance, and Pair requires correct behavior on both.Source: §C.2 · pp. 19–20

D HERA-BENCH REVIEW AND REVISION

  • Review checks task clarity and realism, verifies a reference solution for feasible instances, and confirms that infeasible instances lack another allowed completion path.A model's failure alone does not establish infeasibility; paired versions must share the goal and differ in feasibility because of the specified environment change.Source: D HERA-BENCH REVIEW AND REVISION · p. 20

E.1 FEASIBLE-TASK SUCCESS AND OVER-ABSTENTION

Across evolution rounds, generated environments shift from interacting constraints toward evidence availability and applicable restrictions. The section also describes supplementary completion and paired-success reporting, and illustrates how a single missing forecast value makes a matched task require abstention.

E.1 FEASIBLE-TASK SUCCESS AND OVER-ABSTENTION

  • Table 5 reports task completion and confirmed over-abstention for H0 and the final harness on HERA-BENCH, complementing the primary metrics.The comparison examines whether stronger abstention coincides with reduced completion or increased refusal on solvable tasks.Source: §E.1 · p. 20

E.2 PAIRED SUCCESS ACROSS CUMULATIVE TRAINING POOLS

  • Table 6 reports Pair for the same final harnesses and cumulative training pools as Table 2, alongside the main-text Abstain and Act results.Source: §E.2 · p. 20, Table 6 · p. 21

E.3 HOW ENVIRONMENTS EVOLVE

  • E3–E4 emphasize interacting constraints, E5 expands existing problems, and E6 focuses on evidence availability and whether apparent restrictions still apply.The 30 retained pairs are grouped by primary change, including new constraint combinations, evidence validity and access, expanded entities or relations, and specification or interface revisions.Source: §E.3 · p. 21, Figure 6 · p. 21

F.1 ENVIRONMENT-TASK EXAMPLES

  • In the weather pair, only the Saturday daytime precipitation probability is unavailable in the should-abstain instance, leaving the postponement threshold unresolved.The appropriate response is to report supported findings and abstain from the unresolved decision.Source: §F.1 · p. 21, Figure 7 · p. 22, Figure 8 · p. 23
Loading 2610.06563v1…