PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning from Research: Toward Lifelong Agent Harness Evolution

Jingbo Yang, Kwei-Herng Lai, Xiaowen Wang, Yaar Harari, Evgeniy Gabrilovich, Shiyu Chang

arXiv:2609.40169v1cs.AI

TL;DR

Harness evolution can improve agents without changing their underlying models, but experience-driven methods may explore narrowly and adapt only after failures are observed. SCHOLAREVOLVE uses research literature to guide module-level harness search and evaluates combinations. It reports gains across AppWorld and τ 2-Bench, including higher Qwen3.5-27B task goal completion on AppWorld Challenge.

  • Problem

    Experience-driven harness evolution can explore narrowly, depend on the meta agent’s existing knowledge, and respond only after relevant failures are observed.

    Source: §1 · p. 1

  • Method

    SCHOLAREVOLVE organizes harness exploration by module and research-derived mechanism family, guides module mutations with papers, and evaluates combinations before selection.

    Source: §1 · p. 2

  • Results

    Across AppWorld and τ 2-Bench Telecom, evolved harnesses improve reported outcomes; on AppWorld Challenge, Qwen3.5-27B task goal completion rises from 49.6% to 63.6%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Evolved harnesses transfer to held-out AppWorld applications, and new publications can drive further gains through lifelong evolution.

    Source: §5 · p. 10

    Search uses Devo and Dval, while Dtest is a finite held-out sample used only for final reporting; the backbone is not updated.

    Source: §3.1 · pp. 3–4

Abstract

from arXiv · show

Language agents are expected to solve increasingly complex tasks, creating a growing need for continual improvement. One promising approach is to evolve the agent harness, the software that governs tool use, memory management, and task execution, while keeping the underlying language model fixed. Recent methods automate this process by using a meta coding agent to modify the harness based on execution feedback. However, relying on that agent's existing knowledge and observed failures can restrict exploration and make adaptation reactive. Inspired by how human experts learn from the research literature for new solutions, we introduce ScholarEvolve, a framework that automatically draws on state-of-the-art research to guide harness evolution. ScholarEvolve organizes the harness evolution directions into functional modules and uses topic modeling to identify distinct improvement strategies for each module. It implements these strategies and evaluates their combinations to improve task performance. Moreover, the framework is designed to incorporate new publications over time, allowing research advances to drive proactive lifelong evolution. Experiments demonstrate improvements on AppWorld and Tau2-Bench. ScholarEvolve raises Qwen3.5-27B task goal completion from 49.6% to 63.6% on AppWorld Challenge, and raises GPT-5.4-mini pass@1 from 72.7% to 81.9% on Tau2-Bench Telecom.

1 INTRODUCTION

Harness evolution offers a way to improve agents around fixed models, but experience-driven methods can explore narrowly, depend on existing design knowledge, and adapt only after failures are observed. SCHOLAREVOLVE draws on research literature to organize module-level exploration and reports gains on AppWorld and τ 2-Bench.

1 INTRODUCTION

  • Harness evolution changes the software around a fixed model—including tool use, context management, skills, memory, and execution workflows—rather than repeatedly training the backbone.The motivation passage notes that repeated large-backbone training can make adaptation computationally expensive.Source: §1 · p. 1
  • Experience-driven harness evolution can explore narrowly, rely on the meta agent’s existing knowledge, and adapt only after relevant failures accumulate.A failure trace may reveal a missing capability without indicating how to build it, while deployment weaknesses may affect users before sufficient evidence motivates an update.Source: §1 · pp. 1–2
  • SCHOLAREVOLVE organizes exploration by harness module and mechanism family, uses research papers to guide mutations, and evaluates recombined modules before selection.The framework also proposes periodic literature refresh and genetic-algorithm evolution, with mutations, crossover, and validation-based champion selection.Source: §1 · p. 2, Figure 1 · p. 2
  • 49.6% to 63.6% task goal completion for Qwen3.5-27B on AppWorld Challenge, with gains also reported on AppWorld Normal and τ 2-Bench Telecom.On AppWorld Normal, task goal completion rises from 69.0% to 81.4%; on Telecom, GPT-5.4-mini pass1 rises from 72.7% to 81.9%.Source: §1 · p. 2

2 RELATED WORK

Prior agent-design research searches executable programs, workflows, and modular harnesses, while related work also studies generalization, continued adaptation, and automated research. SCHOLAREVOLVE distinguishes itself by organizing literature-derived mechanisms into module-level search and recombination.

2 RELATED WORK

  • Agent-design work spans program and workflow search, self-modification, executable harness search, trajectory-based revision, and modular recombination.SCHOLAREVOLVE adds literature-derived mechanisms, using topic modeling to organize directions within modules and distribute exploration across them.Source: §2 · p. 3
  • Generalization work separates search from final evaluation or regularizes evolution, while stream-based methods update harnesses from accumulated task experience.SCHOLAREVOLVE instead proposes new publications as a source of updates before corresponding failures accumulate in an agent’s own interactions.Source: §2 · p. 3
  • Automated research systems generate and share scientific findings, while Paper2Agent turns papers and associated code into callable tools and interactive research agents.SCHOLAREVOLVE makes literature-to-harness improvement an explicit process of mechanism retrieval, topic organization, implementation, recombination, and task evaluation.Source: §2 · p. 3

3 METHOD

SCHOLAREVOLVE evolves harness code around a fixed language-model backbone, using research-derived mechanisms to guide module mutations and evaluating their combinations. Its search is organized to carry useful improvements across cycles while keeping the final test set separate from selection.

3.1 PROBLEM FORMULATION

  • The harness is optimized over a target task distribution; evolution uses disjoint development and validation sets, leaving the held-out test sample for final reporting.The backbone parameters remain unchanged, and the search starts from a base harness within a fixed budget.Source: §3.1 · pp. 3–4

3.2.1 STRUCTURING THE CODE SPACE INTO MODULES

  • Search spans five execution modules and research-mechanism topics; changing one module at a time isolates its effect before crossover tests compatibility.The modules cover tool interfaces, context management, skills, memories, and agentic workflows.Source: §3.2 · p. 4, §3.2.1 · pp. 4–5

3.2.2 CONSTRUCTING THE RESEARCH POOL

  • Trajectories from the evolution set are audited for recurring agent-attributable failures, which are abstracted into capability gaps and literature queries for each module.The resulting research pool contains paper titles and abstracts for mechanism-level topic modeling.Source: §3.2.2 · p. 5

3.2.3 TOPIC MODELING FOR ORTHOGONAL EXPLORATION

  • Topic modeling standardizes papers into distinct mechanism topics, assigns inspectable evidence-backed labels, and allocates selected papers round-robin across nonempty clusters.Refinement merges overlapping labels, broadens overly specific variants, and removes vague or irrelevant topics.Source: §3.2.3 · pp. 5–6

3.2.4 FROM RESEARCH MECHANISMS TO EXECUTABLE MUTATIONS

  • For each selected paper, a research agent produces a provenance-preserving mutation blueprint, which a coding agent adapts into a module implementation.Health probes check imports, interface compatibility, artifact construction and reloading, and valid action production; persistent libraries are built only from evolution trajectories and frozen during evaluation.Source: §3.2.4 · p. 6

3.3 MODULE CROSSOVER AND SELECTION

  • Crossover ranks module combinations by additive predicted validation gains, shortlists them within budget, then evaluates complete harnesses to capture module interactions and reward ceilings.Retaining a base module permits subsets of mutations when an individual gain does not carry over to a combination.Source: §3.3 · p. 6
  • The best evaluated candidate is compared with the champion using paired task gains and a paired-bootstrap interval; cycles stop after consecutive unconfirmed generations or budget exhaustion.If no executable candidate is available, the champion is retained.Source: §3.3 · p. 6

3.4 LIFELONG HARNESS EVOLUTION

  • New publications can initiate a search cycle after convergence, with literature supplying mutation directions, validation guiding selection, and the fixed-backbone champion carrying forward code and artifacts.Available trajectories can refine the research brief as the harness evolves.Source: §3.4 · p. 7

4 EXPERIMENTS

Across AppWorld and Telecom, harness evolution improves task metrics; lifelong updates, ablations, and composition analyses examine performance over time and the contributions of search components.

4.1 EXPERIMENTAL SETUP

  • The study evolves fixed Qwen3.5-27B and GPT-5.4-mini task backbones, comparing initial and Meta Harness baselines over three independent evaluation runs.AppWorld uses separate training, development, Normal, and Challenge sets; Telecom uses training, development, and test splits.Source: §4.1 · p. 7

4.2 MAIN RESULTS

  • 12.4 TGC and 20.2 SGC points are gained by Qwen on AppWorld Normal, while Mini gains 9.2 pass1 and 9.1 pass4 points on Telecom.The main-results table compares ScholarEvolve with the matched initial harness across AppWorld and τ 2-Bench.Source: §4.2 · p. 7, Table 1 · p. 8
  • Challenge TGC gains exceed Normal gains for both backbones (Qwen 14.0 vs 12.4; Mini 9.5 vs 5.3 points) on tasks involving unseen Amazon and Gmail APIs.On Normal, Qwen reaches 81.4% TGC, near Kimi-K2.6’s 81.3%, and closes approximately 74% of its gap to GPT-5.4.Source: §4.2 · p. 7

4.3 LIFELONG EVOLUTION

  • Across three publication-window updates, AppWorld Normal TGC and SGC reach 81.5% and 66.7%, finishing 11.9 and 15.5 points above Meta Harness.The best harness is selected on development tasks and evaluated on Normal; Figure 3 tracks best harness performance by generation.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4 ABLATION STUDIES

  • Topic guidance adds 1.4 TGC and 2.3 SGC points for Qwen, and 2.6 and 5.4 points for Mini, beyond relevance ranking.The cumulative ablation successively removes topic-guided selection, module-wise mutation, and research guidance.Source: §4.4 · pp. 8–9, Table 2 · p. 8
  • Combined modules beat their strongest constituent by 3.5 TGC/12.3 SGC points for Qwen and 4.7/7.0 points for Mini.For Qwen, replacing a 75.4% standalone skills candidate with a 73.7% candidate raises the combination from 80.7% to 84.8%.Source: §4.4 · p. 9, Table 3 · p. 9

4.5 ANALYSIS OF HARNESS EVOLUTION

  • Above the training maximum of 12 reference APIs, Challenge TGC gains remain positive for Qwen and Mini at 9.5 and 7.2 points.Table 4 groups Challenge tasks by the number of reference-solution APIs.Source: §4.5 · p. 9, Table 4 · p. 9
  • Qwen improves on 221 of 585 tasks versus 57 regressions, while invalid-format failures fall from 492 to 206; requirement tracking and evidence coverage remain recurring challenges.Tasks solved in all three runs increase from 205 to 333; the failure taxonomy covers five families and 16 subtypes.Source: Figure 3 · p. 8, §4.5 · p. 9, Figure 4 · p. 9

5 CONCLUSION

SCHOLAREVOLVE evolves agent harnesses around a fixed backbone by learning from research, organizing exploration by module and topic-modeling distinct mechanisms. Its evolved harnesses improve both task backbones and transfer to held-out AppWorld applications, while residual requirement-tracking and evidence-coverage failures remain open problems.

5 CONCLUSION

  • The framework evolves harnesses around a fixed backbone by learning from research and organizing exploration by harness module.It uses topic modeling to cover distinct mechanisms.Source: §5 · pp. 9–10
  • SCHOLAREVOLVE evaluates module combinations before selection.Source: §5 · p. 10
  • The evolved harnesses improve both task backbones on AppWorld and τ 2-Bench and transfer to held-out AppWorld applications.Source: §5 · p. 10
  • New publications drive further gains in lifelong evolution, while requirement tracking and evidence coverage remain open problems.Source: §5 · p. 10

A.1 IMPLEMENTATION DETAILS

The implementation fixes task backbones and environment settings while using development data to select harnesses and test data for evaluation. It specifies the search, metric aggregation, and ablation procedures, alongside figures and a table describing API distributions and lifelong generations.

A.1 IMPLEMENTATION DETAILS

  • AppWorld uses 90 Devo tasks for research auditing and artifact construction, 57 Dval tasks for candidate selection, and DN test to evaluate frozen harnesses.Episodes allow 100 environment steps, and scenario variants remain grouped across splits.Source: §A.1 · p. 16
  • Telecom uses a 250-task experience pool sampled with seed 2026, a 74-task Dval split, and a retained 40-task Dtest split.The environment limits episodes to 200 conversation steps and ten errors, with simulation seed 300.Source: §A.1 · p. 16
  • Qwen3.5-27B and GPT-5.4-mini use fixed task-model configurations across initial and evolved harnesses within each benchmark.Qwen uses vLLM 0.27.1 with tensor parallelism four, temperature zero, and thinking enabled; Mini uses low reasoning effort.Source: §A.1 · p. 16
  • AppWorld searches request four paper-derived candidates per module and Telecom three, measuring single-module candidates before ranking and evaluating combinations.Successful Devo trajectories can build persistent skill and memory artifacts, frozen during Dval and Dtest evaluation; convergence stopping ends a cycle after one generation without confirmed validation gain.Source: §A.1 · p. 16
  • AppWorld reports TGC and SGC averaged over three evaluation runs, while Telecom estimates passk from four internal trials per task and reports means and standard deviations across three runs.Source: §A.1 · p. 16
  • The cumulative AppWorld ablation changes paper selection, module mutation and recombination, then external research guidance, while composition comparisons use the same 57 development tasks over three runs.Qwen combines five modules; Mini combines skills, memory, and workflow while retaining the base tool and context modules.Source: §A.1 · p. 16
  • Table 5 labels t0 as the shared initial harness and t1–t3 as the best harness found in each generation.Source: Table 5 · p. 17
  • Figure 5 shows reference API-count distributions across AppWorld splits, with shaded ranges marking at most nine APIs and more than 12 APIs.The thresholds use the training third quartile and maximum.Source: Figure 5 · p. 17

A.2 LIFELONG EVOLUTION PROTOCOL AND RESULTS

The lifelong protocol evaluates successive harness generations using publication windows while keeping model weights, archived trajectories, and derived artifacts fixed. Each generation selects papers, builds individual mutations, and tests combinations on development tasks before frozen decisions are evaluated on the test set.

A.2 LIFELONG EVOLUTION PROTOCOL AND RESULTS

  • Three generations use disjoint publication windows through December 2025, January–April 2026, and May–August 2026, with search decisions frozen before test evaluation.Windows use arXiv first-submission dates reconstructed from the current index.Source: §A.2 · p. 17
  • At each update, SCHOLAREVOLVE selects at most one paper per module, builds up to five individual mutations, and evaluates up to four combinations on 57 development tasks.The best harness receives three development evaluations in total.Source: §A.2 · p. 17
  • Model weights, archived Devo trajectories, and derived artifacts remain fixed throughout the lifelong evaluation.Research selection uses GPT-5.4 and candidate implementation uses Codex GPT-5.5, both with medium reasoning effort.Source: §A.2 · p. 17

A.3 TRAINING AND TEST API DISTRIBUTIONS

The API-count analysis compares reference-solution API breadth across AppWorld splits using training-derived thresholds. It also describes how Figure 6 encodes success-count transitions and reports uncertainty intervals for gains above the training maximum.

A.3 TRAINING AND TEST API DISTRIBUTIONS

  • 24.5% of Challenge tasks require more APIs than the training maximum of 12, and the median reference API count rises from eight in Train to 8.5 in Normal and ten in Challenge.The training third quartile is nine APIs; the API-count groups contain 168, 147, and 102 Challenge tasks.Source: §A.3 · p. 17
  • Figure 6 organizes task counts by initial and evolved success counts out of three, with colors indicating more, equal, or fewer successes.Darker shades represent larger fractions of each split on a shared square-root scale.Source: Figure 6 · p. 18
  • For tasks above the training maximum, Qwen’s 95% interval is [−0.7, 19.6] percentage points and Mini’s is [0.3, 14.1].Intervals use 20,000 paired scenario bootstrap resamples, preserve three variants per scenario, and condition on saved runs.Source: §A.3 · p. 18

A.4 ANALYSIS PROTOCOL

The analysis combines task-level success transitions with trajectory-level recovery and behavior labels to characterize the evolved harness. Its comparisons use complete test cohorts while retaining limits on what conditional rates and labels establish.

Task-level comparisons

  • Across the complete test cohorts, task success counts are higher, equal, or lower for 57/102/9 Qwen Normal tasks and 164/205/48 Challenge tasks.For Mini, the corresponding counts are 44/94/30 and 156/185/76; heatmaps include all 168 Normal and 417 Challenge tasks.Source: §A.4 · p. 18

Reliability transitions

  • Among tasks with increased success counts, many initially succeeded in only one or two runs, while the numbers solved in all three runs increased for both harnesses.For Qwen, all-three-run tasks rose from 86 to 123 on Normal and 119 to 210 on Challenge; for Mini, from 85 to 96 and 89 to 140.Source: §A.4 · p. 18

Recovery after execution errors

  • 249 of 504 evolved-harness Normal episodes encounter an execution error, and 180 ultimately succeed; error-exposed success rates exceed the initial harness on both splits.Rates are 72.3% versus 64.5% on Normal and 59.9% versus 44.9% on Challenge; the initial and evolved harnesses can encounter errors on different tasks.Source: §A.4 · pp. 18–19

Behavior labels

  • Figure 7 organizes observable trajectory deviations into five families, allows labels to coexist within an episode, and records recovery status separately.The complete-cohort analyses use paired task success counts; run indices are not assumed to represent matched random seeds.Source: §A.4 · p. 18, Figure 7 · p. 19

A.5 FAILURE TAXONOMY AND DIAGNOSTIC CHECKS

The failure analysis combines ordered endpoint checks with overlapping behavior labels, auditing complete Qwen cohorts and preserving uncertainty when evidence does not support a firm label. It reports both diagnostic counts and task-level transitions, while treating examples as mechanisms rather than frequency estimates.

Hierarchical behavior coding

  • The audit covers 1,336 unsuccessful episodes across 396 tasks, using a stratified calibration set before applying frozen definitions to the full cohort.A second pass revisits proposed deviations and subsequent recovery; checks reconcile evidence with verified completion signals, execution limits, and answer rules.Source: §A.5 · pp. 19–20
  • Labels may overlap, and unsupported inferences remain uncertain; omitted requirements are not assigned solely because a run reaches its step budget.Missing per-check outcomes or final-state snapshots remain unobserved, so the labels characterize archived behavior under the evaluated harnesses and search budget.Source: §A.5 · p. 20
  • Sixteen of 17 candidate behavior subtypes have supported instances; 218 failed episodes lack a supported unresolved label and remain uncertain.The missing-answer subtype is unobserved and omitted from both taxonomy figures; Table 6 reports subtype counts, with multilabel counts and recovered occurrences.Source: §A.5 · pp. 19–20, Table 6 · p. 20

Coverage and descriptive comparisons

  • Among initially all-run-failing tasks with invalid response formats, 54 of 111 gain at least one success after evolution, compared with 16 of 42 for constraint violations.These groups can overlap and include other deviations, so the counts are conditional task-level transitions rather than isolated effects.Source: §A.5 · p. 20

Completion and answer diagnostics

  • Total failures fall from 156 to 94 on Normal and from 631 to 455 on Challenge, while the ordered endpoint counts show category-specific changes.The category counts change from (1, 84, 15, 56) to (0, 29, 8, 57) on Normal and (10, 407, 11, 203) to (10, 177, 17, 251) on Challenge; later categories can increase because earlier checks pass.Source: §A.5 · pp. 20–21, Figure 8 · p. 21

Qualitative state examples

  • Two state examples move from zero to three successful runs after changing object handling, while 35 of 71 persistently failing Challenge tasks still show an invalid final response.The examples illustrate mechanisms; frequency estimates come from the complete test cohorts, and residual failures motivate further refinement or targeted training data.Source: §A.5 · p. 21

B QUALITATIVE ANALYSIS

The qualitative analysis maps research-derived alternatives onto decision points in the agent’s execution cycle and illustrates how state-aware action choices can change outcomes. It also identifies remaining response-format failures as a target for further harness refinement.

B QUALITATIVE ANALYSIS

  • The candidate map spans context selection, memory retrieval, skills, workflows, and action representation, with alternatives organized across the agent’s decision process.Figure 9 presents two research-derived mechanisms per module, with arrows showing information and action flow; the alternatives are candidates for selection and recombination.Source: B QUALITATIVE ANALYSIS · p. 21, Figure 9 · p. 22
  • Context candidates prioritize information or score relationships under a length budget, while memory and skill candidates make prior episodes or reusable procedures available during execution.Workflow candidates route among reasoning strategies or aggregate specialist proposals, complementing changes to information selection, experience reuse, and action representation.Source: B QUALITATIVE ANALYSIS · p. 21
  • Two Qwen state-aware action changes each move a task from zero to three successful runs: updating the existing object and clearing pre-existing state before an operation.These examples point to object identity and action preconditions as useful harness-design targets; among 71 persistently failing Challenge tasks, 35 still show an invalid final response in an evolved run.Source: B QUALITATIVE ANALYSIS · pp. 21–22

C.1 REPRESENTATIVE MODULE IMPLEMENTATIONS

The implementations instantiate research-derived changes within modular harness responsibilities, then connect capability gaps to retrieved research and screened mechanisms. Module contracts support executable assembly, validation, and comparison of complete harness configurations.

C.1 REPRESENTATIVE MODULE IMPLEMENTATIONS

  • Representative mutations add documentation-based action inspection and recovery, relevance-and-recency context selection, mined API procedures, structured scene memories, and planner–solver–critic workflows.These examples show distinct implementation choices across tool interface, context, skills, memories, and agentic workflows.Source: §C.1 · pp. 22–23

C.2 RESEARCH POOL CONSTRUCTION AND TOPIC SELECTION

  • For each module, research queries combine broad capability terms with narrower mechanism terms, using the same capability brief, module responsibility, interface, and current harness behavior.An audit can supply the brief from recurring agent-attributable failures; abstracted questions omit benchmark names, domain entities, and specific tool or field names.Source: §C.2 · p. 23
  • Retrieved papers are deduplicated within module pools, assigned to a refined mechanism taxonomy, screened for absent mechanisms, then allocated round-robin across eligible topic clusters.Topic assignments retain supporting abstract excerpts and provenance; the implementation budget is applied after taxonomy construction.Source: §C.2 · p. 23

C.3 MODULE INTERFACES AND EXECUTABLE CHECKS

  • Module output contracts carry memories as evidence and skills as procedures through context into workflow proposals, while health probes check imports, interfaces, artifacts, and valid actions.Persistent artifacts built from Devo trajectories remain fixed on Dval and Dtest, while modules may maintain episode-local state.Source: §C.3 · p. 24

C.4 CROSSOVER DIAGNOSTICS

  • Shortlisted mutations and crossovers are assembled as complete harnesses, ranked by measured validation gain, and retained for further search with their frozen artifacts.The additive score may exceed bounded-reward gains; joint evaluation supplies selection performance, and the residual captures interactions, reward ceilings, and evaluation variation.Source: §C.4 · p. 24

D.1 CANDIDATE POOL FOR QWEN3.5-27B ON APPWORLD

Of 20 paper-derived candidates for Qwen3.5-27B on AppWorld, 17 raise standalone development TGC, while three sharply reduce it and are excluded from the final harness. The selected configurations were fixed on Dval before test evaluation.

D.1 CANDIDATE POOL FOR QWEN3.5-27B ON APPWORLD

  • The candidate pool contains four source papers per module, with the four papers assigned to distinct topics within each module.Table 7 marks which modules are included in the final harness.Source: §D.1 · p. 25, Table 7 · p. 25
  • 17 of 20 candidates raise standalone TGC, eight have a 90% interval above zero, and the three sharp reducers are excluded from the final harness.Table 7 reports standalone gains over 57 development tasks across three runs; each candidate changes one module.Source: Table 7 · p. 25, §D.1 · p. 25

D.2 SELECTED CONFIGURATIONS AND CODE

  • Table 8 lists the selected implementation, source paper, and assigned topic for each module in the four main-experiment harnesses.Selections were fixed on Dval before test evaluation; modules with persistent artifacts build them from Devo during fit and reload them for evaluation.Source: §D.2 · p. 25, Table 8 · p. 26
  • The final Qwen3.5-27B AppWorld harness excerpts show implementations for five modules, each using the interface defined in Appendix C.3.The excerpts cover tool interface, context management, skills, memories, and agentic workflows.Source: §D.2 · pp. 25–29
Loading 2609.40169v1…