PaperlayerView exact revision on arXiv

Source-linked AI summary

PluginRSI: Recursive Improvement of Agent Harnesses with Reusable Plugins

Yaorui Shi, Yuchun Miao, Yuxin Chen, Jiayuan Zhang, Yueqing Sun, Xierui Song, Xiang Wang, An Zhang

arXiv:2609.32423v1cs.AI

TL;DR

Complete-harness optimization makes individual mechanisms difficult to isolate and reuse. PluginRSI improves plugins independently, accumulates beneficial implementations, and recomposes harnesses from a shared library. It improves results across task settings, transfers advantages across solver models, and accelerates subsequent optimization from the initial harness.

  • Problem

    Iterative rewriting of complete harnesses couples mechanisms, limiting independent exploration and making effective mechanisms difficult to preserve and reuse.

    Source: §1 · p. 1

  • Method

    PluginRSI mutates individual plugins while holding the remaining harness fixed, accumulates beneficial variants in a shared library, then recomposes candidates by selecting plugins and revising the workflow.

    Source: §1 · p. 2

  • Results

    PluginRSI improves performance across software engineering, command-line interaction, and question-answering, retains advantages across solver models, and reaches 69% held-out resolve rate after two steps with library reuse versus 65% without reuse.

    Source: §1 · p. 2, §4.2 · p. 6

  • Takeaways & Limitations

    Accumulating reusable mechanisms supports further harness improvement, and reusing the evolved library accelerates optimization from the initial harness.

    Source: §6 · p. 10

    Plugin mutation evaluates each change with the rest of the harness fixed, so it can miss mechanisms requiring coordinated changes to other plugins or the workflow.

    Source: D LIMITATIONS AND FUTURE WORK · p. 16

Abstract

from arXiv · show

The harness surrounding a language model is a central determinant of agent performance. Recent methods optimize harnesses by searching over complete programs, where individual mechanisms are difficult to isolate and reuse. We introduce PluginRSI, which represents a harness as a composition of atomized plugins and organizes harness evolution around these plugins. Individual plugins are improved independently and accumulated in a shared library, then recombined into new harnesses at each iteration. PluginRSI improves over existing harness optimization methods across software engineering, command-line interaction, and question-answering tasks. The resulting harnesses retain their advantage when transferred to other solver models without further optimization. The evolved plugin library accelerates subsequent optimization from the initial harness, which helps faster and higher convergence on unseen tasks. These results show that accumulating reusable mechanisms provides an effective basis for continued harness improvement.

1 INTRODUCTION

Complete-harness rewriting couples mechanisms, making them difficult to explore independently and preserve for reuse. PluginRSI instead mutates and accumulates plugins, then recomposes harnesses; it improves results across task settings and transfers its advantage across solver models.

1 INTRODUCTION

  • Iterative harness rewriting couples mechanisms, limiting independent exploration and making effective mechanisms difficult to preserve and reuse.The authors identify these as constraints on discovering and accumulating mechanisms.Source: §1 · p. 1
  • PluginRSI represents harness mechanisms as plugins that can be developed and evaluated independently, then retains beneficial implementations in a shared library for reuse.The rest of the harness stays unchanged while an individual mechanism is evaluated.Source: §1 · p. 2
  • Each iteration mutates plugins while holding the remaining harness fixed, adds beneficial variants to the library, and recomposes a candidate by selecting plugins and revising the workflow.Recomposition builds on mechanisms discovered in earlier iterations.Source: §1 · p. 2
  • PluginRSI outperforms baselines across software engineering, command-line interaction, and question-answering, with 6–10 percentage-point held-out resolve-rate gains over Meta-Harness on SWE-bench Verified.The gains span two optimization backbones.Source: §1 · p. 2
  • The accumulated library accelerates subsequent optimization: reuse from the initial harness reaches a 69% held-out resolve rate after two evolution steps, versus 65% without reuse.The resulting harnesses also retain their advantage across solver models without further optimization.Source: §1 · p. 2

2 PROBLEM FORMULATION

The paper formulates harness optimization as maximizing expected task reward while holding the policy model fixed. Recursive optimization uses evaluation feedback to propose and select successively improved harnesses.

2.1 OPTIMIZATION OBJECTIVE

  • A harness coordinates tools, skills, and other mechanisms; with a policy model, it defines an agent that generates task trajectories scored by reward.The objective is the harness's expected reward over tasks and trajectories.Source: §2.1 · p. 2
  • Figure 2 depicts plugin mutation in parallel with the remaining harness fixed, followed by recomposition from the updated library and workflow revision.Source: Figure 2 · p. 3
  • Harness optimization seeks the harness H⋆ that maximizes expected reward while keeping the policy model Θ fixed.This frames optimization as harness adaptation rather than changing the policy model.Source: §2.1 · p. 3

2.2 RECURSIVE HARNESS OPTIMIZATION

  • Recursive harness optimization uses a separate proposer agent to inspect implementations and execution trajectories and propose new candidates, while the policy and proposer remain fixed.Source: §2.2 · p. 3
  • The search starts from H0, and at iteration t the proposer constructs a candidate from the previous best harness and its feedback.Source: §2.2 · p. 3
  • Each proposed harness is evaluated on tasks from D, producing rewards and trajectories that inform later iterations.Source: §2.2 · p. 3
  • After T iterations, the procedure returns the highest-scoring harness among those considered.Source: §2.2 · p. 3

3 PLUGINRSI: PLUGIN-ORIENTED HARNESS OPTIMIZATION

PluginRSI represents harnesses as selected plugins coordinated by a workflow and maintains a shared library of beneficial implementations. Its iterative process mutates individual plugins, accumulates improvements, and recomposes harness candidates.

3 PLUGINRSI: PLUGIN-ORIENTED HARNESS OPTIMIZATION

  • PluginRSI represents a harness through plugin implementations and their coordinating workflow, retaining beneficial implementations in a shared library across iterations.The loop explores mechanisms through plugin mutation and then recomposes a harness from the updated library.Source: §3 · p. 3

3.1 PLUGIN-PARAMETERIZED HARNESSES

  • A plugin is an executable harness mechanism exposed through a standardized interface; H = H(W, P) compiles selected plugins P with workflow W.Plugins implement roles, tools, skills, and memory mechanisms; the workflow determines invocation timing and how outputs are used.Source: §3.1 · pp. 3–4

3.2.1 INITIALIZATION AND OVERVIEW.

  • Optimization begins with an initial harness and plugin library, evaluates the harness on validation tasks, and iterates mutation and recomposition before returning the best validation harness.Validation and held-out test datasets are disjoint, and test data is not used during optimization.Source: §3.2.1 · p. 4

3.2.2 PLUGIN MUTATION.

  • Plugin mutation generates parallel candidates using separate balanced minibatches, then replaces one selected plugin while holding the workflow and other plugins fixed.Variants that improve on the same minibatch are added to the library.Source: §3.2.2 · p. 4

3.2.3 HARNESS RECOMPOSITION.

  • Harness recomposition selects promising implementations from the updated library and revises the workflow, including plugin invocation and information flow, to evaluate a new candidate.The algorithm keeps the recomposed candidate only when its validation score exceeds that of the incumbent.Source: §3.2.3 · pp. 4–5

4 EXPERIMENTS

PluginRSI improves harness performance across software engineering, command-line interaction, and question-answering tasks, while retaining advantages across solver models. Its plugin mutation, harness recomposition, and accumulated library support faster optimization and reuse.

4.1 EXPERIMENTAL SETUP

  • Evaluation spans software engineering, command-line interaction, and QA, with held-in-selected harnesses evaluated on held-out tasks; SWE-bench and Terminal-Bench report resolve rates, and QA reports accuracy.Source: §4.1 · p. 5

4.2 MAIN RESULTS

  • On SWE-bench Verified, held-out resolve rates reach 65.00% and 69.00%, beating the best baselines by 10.00 and 6.00 points with GPT-5.6 Terra and Kimi-K3.Source: Table 1 · p. 6, §4.2 · p. 6
  • Transferred harnesses retain advantages without further optimization, although some held-in cross-model results are modestly below baselines.Source: §4.2 · p. 6
  • On Terminal-Bench-2.1, PluginRSI scores 69.5% held-in and 73.3% held-out versus Meta-Harness’s 64.4% and 66.7%; QA overall accuracy rises from 0.638 to 0.675 over baselines.Source: Table 2 · p. 6, Table 3 · p. 6, §4.2 · p. 7

4.3 OPTIMIZATION DYNAMICS

  • PluginRSI reaches a held-in resolve rate Meta-Harness does not attain within the evaluated rollout budget, with held-in gains accompanied by held-out gains.Source: Figure 3 · p. 7, §4.3 · p. 7

4.4 ABLATION STUDY

  • Removing plugin mutation or harness recomposition lowers Kimi-K3 held-out resolve rate from 69.00% to 57.00% or 59.00%; removing initial plugins costs only 1.00–2.00 points across three solvers.Source: Table 4 · p. 8, §4.4 · p. 8

4.5 PLUGIN ACCUMULATION AND REUSE

  • Reusing the evolved library from the initial harness reaches 69% held-out resolve rate after two steps versus 65% without reuse, approaching 70% from continued optimization of the evolved harness after three steps.Source: Figure 4 · p. 8, §4.5 · p. 8
  • The library grows from 75 to 132 plugins over 15 steps, while final candidates reach about 1,300 LoC and the workflow stays below 300 LoC.Source: §4.5 · p. 8, Figure 5 · p. 9

5 RELATED WORK

Prior work improves agents through model updates and external artifacts, while recursive harness optimization searches harness designs. PluginRSI distinguishes itself by decoupling executable mechanisms for independent evaluation and reuse.

Self-Improving Agent

  • Agent self-improvement includes policy learning, prompt optimization, context and memory methods, and reusable skills; PluginRSI targets executable mechanisms and coordinating workflows.Policy learning methods use reinforcement learning or supervised fine-tuning, while artifact methods refine prompts, context, memory, or skills.Source: §5 · p. 9

Recursive Harness Optimization

  • Recursive harness optimization methods search harness designs by rewriting implementations, but entangled mechanisms hinder individual evaluation and reuse.PluginRSI instead decouples mechanisms so they can be evaluated with the rest of the harness fixed and reused across candidates.Source: §5 · p. 9

Modular Harness Design

  • Modular harnesses organize composable components, but PluginRSI studies recursively discovering, accumulating, and reusing plugins rather than only incorporating designed mechanisms.Prior examples include swappable runtime plugins and typed processors for editing harness components and configurations.Source: §5 · p. 9

6 CONCLUSION

PluginRSI improves harnesses by mutating individual plugins and recomposing them from an accumulated library. Experiments report gains across task settings, solver models, and subsequent optimization from the initial harness.

6 CONCLUSION

  • PluginRSI improves individual mechanisms through plugin mutation and recomposes harnesses from an accumulated plugin library.The method organizes recursive harness optimization around plugins.Source: §6 · p. 9
  • PluginRSI improves performance across software engineering, command-line interaction, and QA tasks, with stronger SWE-bench harnesses using fewer solver rollouts and held-out advantages across solver models.Ablations support contributions from both optimization stages.Source: §6 · pp. 9–10
  • Reusing the evolved plugin library accelerates subsequent optimization from the initial harness.The results support accumulating reusable mechanisms as a basis for further harness improvement.Source: §6 · p. 10

A REPRODUCIBILITY STATEMENT

The appendices describe reproducibility materials, ethical considerations, and AI assistance. They also identify limits involving fixed task sets, transfer scope, and plugin mutation with the rest of the harness held fixed.

A REPRODUCIBILITY STATEMENT

  • The reproducibility materials include the optimization procedure, evaluation protocol, model configurations, search hyperparameters, initial harness, seed plugins, prompts, task split details, and plugin interfaces.An anonymous code repository is linked in the abstract.Source: A REPRODUCIBILITY STATEMENT · p. 16

B ETHICS STATEMENT

  • The experiments use existing software engineering, command-line interaction, and question-answering benchmarks; deployment beyond benchmarks warrants code review and appropriate execution permissions.Automatically evolved harnesses can improve software development, but generated code and commands may introduce errors or unintended changes.Source: B ETHICS STATEMENT · p. 16

C USE OF AI STATEMENT

  • Generative AI assisted literature review, code development, and manuscript polishing, and resulting seed-library plugins underwent manual inspection.GPT-5.6 Sol assisted with mechanism extraction for the seed plugin library.Source: C USE OF AI STATEMENT · p. 16

D LIMITATIONS AND FUTURE WORK

  • Evaluation uses fixed task sets over limited iterations, transfer depends on task domain and solver model, and fixed-rest plugin mutation can miss coordinated changes.The authors identify longer-term evolution under changing task distributions, broader transfer, and plugin-workflow joint optimization as future work.Source: D LIMITATIONS AND FUTURE WORK · p. 16

E IMPLEMENTATION DETAILS

Implementation uses specified proposer and solver configurations, mutation minibatches and iteration budgets, and common settings for automatically optimized baselines. A held-out setup compares optimization from the initial harness with different plugin-library states.

E IMPLEMENTATION DETAILS

  • SWE-bench uses GPT-5.6 Terra and Kimi-K3 as proposer and optimization solver in respective settings, with selected harnesses evaluated across GPT-5.6 Terra, Kimi-K3, and GLM-5.2.Kimi-K3 serves as both proposer and optimization solver for Terminal-Bench, QA, and ablations; proposer reasoning effort is xhigh and solver effort is none.Source: E IMPLEMENTATION DETAILS · p. 16
  • PluginRSI generates 8 mutation candidates per iteration, each evaluated on a separate 20-task minibatch, and runs for 15 iterations versus 25 for baselines.On SWE-bench Verified, an iteration uses 560 solver rollouts; recomposition generates one candidate evaluated on all held-in tasks.Source: E IMPLEMENTATION DETAILS · p. 16
  • Automatically optimized harnesses start from ReAct-style harnesses, share task splits, API endpoints, and sandboxes, and are held fixed for held-out and cross-model evaluation.GEPA optimizes solver system prompts, whereas DGM and Meta-Harness revise complete executable harness implementations.Source: E IMPLEMENTATION DETAILS · p. 17
  • The plugin-reuse comparison tests the initial harness with its initial library, the initial harness with the evolved library, and the selected evolved harness with that library on 100 held-out tasks.Table 5 summarizes hyperparameters, and Table 6 lists proposer and solver model configurations.Source: E IMPLEMENTATION DETAILS · p. 17, Table 5 · p. 17, Table 6 · p. 17

F ADDITIONAL RESULTS

On SWE-bench Verified, PluginRSI improves held-out resolve rates over three additional baselines. The appendices describe its seed harness and the proposer’s plugin-mutation and harness-recomposition stages.

F ADDITIONAL RESULTS

G INITIAL HARNESS

  • The seed library contains 75 manually inspected Python plugins across four categories, adapted from prior research and open-source agent systems.Each plugin retains its configuration, supporting resources, and provenance.Source: G INITIAL HARNESS · p. 18, Table 9 · p. 19
  • The initial harness combines a coder role, terminal tool, debugging skill, and experience-bank memory in a workflow that stores tool outputs and runs for up to 20 turns.Memory is initialized separately for each task.Source: G INITIAL HARNESS · p. 18, Figure 6 · p. 19

H SYSTEM PROMPT FOR PROPOSER

  • Plugin mutation uses minibatch feedback to revise one selected plugin while keeping the workflow, other plugins, and selected instance configuration fixed.The proposer is instructed to develop a transferable mechanism for a concrete failure in the supplied trajectories.Source: H SYSTEM PROMPT FOR PROPOSER · pp. 19–20
  • Harness recomposition selects plugins from the updated library and revises the workflow, while plugin implementations remain fixed during this stage.The proposer is directed to integrate useful mechanisms and avoid blindly stacking plugins.Source: H SYSTEM PROMPT FOR PROPOSER · pp. 19–20
  • Positive local plugin gains guide integration but do not guarantee improvement on the full evaluation set.The recomposition instructions also direct the proposer to consider regressions and when each mechanism should help.Source: H SYSTEM PROMPT FOR PROPOSER · p. 20

I DATASET SPLIT

The SWE-bench split supports task-level evaluation within the benchmark but not transfer to unseen repositories. The plugin schema versions components and specifies how workflows load and coordinate them.

I DATASET SPLIT

  • The 400-task held-in and 100-task held-out SWE-bench splits share a fixed partition across methods and backbones, with optimization confined to held-in tasks.The selected harness is kept fixed for held-out and cross-model evaluation.Source: I DATASET SPLIT · p. 21
  • Held-out evaluation measures task-level generalization within the benchmark, not transfer to unseen repositories; Seaborn and Flask have no held-out tasks.The held-in split covers all 12 repositories, while the held-out split covers 10.Source: I DATASET SPLIT · p. 21

J PLUGIN SCHEMA

  • A plugin package contains a manifest, implementation, and supporting resources; its manifest identifies kind, name, and version, specifies an entrypoint, and validates configuration against a schema.Optional provenance records sources, adaptations, and parent references.Source: J PLUGIN SCHEMA · p. 21
  • The harness manifest selects plugin versions and assigns aliases and configurations; the workflow invokes category interfaces and coordinates model calls and plugin execution.Mutation creates a new plugin version while preserving its alias, configuration, and interface; recomposition selects existing versions and revises the workflow.Source: J PLUGIN SCHEMA · pp. 22–23, Figure 8 · p. 22

K DISCOVERED PLUGINS

Selected mutations alter role, skill, memory, and tool mechanisms to address specific solver failures. The examples show changes to repair boundaries, source semantics, evidence retrieval, and terminal feedback.

K DISCOVERED PLUGINS

  • The selected mutations span roles, skills, memory, and tools, with one example from each category illustrated in the section.Table 11 summarizes eight selected mutations and their screening results.Source: K DISCOVERED PLUGINS · p. 23
  • The mutated coder role limits guards, transactions, or cleanup to the smallest persistent-mutation sequence requiring them, resolving a Django repair that the original solver wrapped too broadly.The example is django django-16100.Source: K DISCOVERED PLUGINS · p. 23, Figure 9 · p. 24
  • The mutated debugging skill makes the solver identify the intended value source and define behavior when it is absent, avoiding a fallback that changes source semantics.In pydata xarray-6461, it returns empty attributes for a scalar source and resolves the task.Source: K DISCOVERED PLUGINS · p. 23, Figure 10 · p. 24
  • The mutated memory plugin deduplicates stored outputs and summarizes entries not previously returned, emphasizing newly collected evidence over repeated lexical-overlap retrieval.It also gives guidance for subsequent verification.Source: K DISCOVERED PLUGINS · p. 24, Figure 11 · p. 25
  • The mutated terminal tool detects execution and test failures, including those masked by successful shell status, and prompts inspection when the same focused-test command and first diagnostic recur.On its screening minibatch, the mutation resolved five previously unsolved tasks with one regression.Source: K DISCOVERED PLUGINS · p. 24, Figure 12 · p. 26

L TOKEN COST ANALYSIS

PluginRSI uses more total tokens per solver rollout than Meta-Harness, while solver-token usage remains similar; the added mutation stage contributes to proposer overhead.

L TOKEN COST ANALYSIS

  • Total input and output tokens per solver rollout rise 7.5% and 19.8%, respectively, while solver token usage remains similar.Figure 13 separately reports solver tokens per rollout and proposer tokens per proposal.Source: L TOKEN COST ANALYSIS · p. 27, Figure 13 · p. 27
  • The additional plugin-mutation stage contributes to proposer overhead, which is reported separately from harness recomposition.Figure 13 also reports total token usage per solver rollout, including solver and proposer tokens.Source: L TOKEN COST ANALYSIS · p. 27, Figure 13 · p. 27

Related papers

Loading 2609.32423v1…