PaperlayerView exact revision on arXiv

Source-linked AI summary

ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement

Siwei Wu, Jincheng Ren, Yizhi Li, Haau-Sing Li, Chengran Yang, Yuxuan Zhang, Weicheng Gu, Jian Yang, Riza Batista-Navarro, Chuanyi Zhang, Xianglong Liu, Ming Zhou, Bryan Dai, Chenghua Lin

arXiv:2609.14857v1cs.CL

TL;DR

Generalizable harness RSI is difficult because benchmark-based evolution, single-trajectory evidence, and monolithic harnesses obscure reusable mechanism improvements. ModularRSI addresses this with benchmark-disjoint tasks, contrastive trajectory analysis, and independently evolved functional modules. It consistently improves unseen in-domain and cross-domain performance and transfers across foundation models, while the study does not isolate contrastive analysis in a dedicated ablation and uses only part of its curated dataset.

  • Problem

    Generalizable harness RSI lacks clear, transferable improvement signals because benchmark data, single trajectories, and monolithic harnesses confound adaptation with reusable deficiencies.

  • Method

    ModularRSI contrasts successful and failed trajectories, aggregates evidence across tasks, independently evolves five harness modules, and uses benchmark-disjoint evolution tasks with validation gates.

  • Results

    ModularRSI consistently improves performance on unseen in-domain and cross-domain tasks and transfers across different foundation models.

  • Takeaways & Limitations

    Benchmark-disjoint, contrastive, and modular evolution shows potential for building more generalizable self-improving agent harnesses.

  • Takeaways & Limitations

    The study does not isolate contrastive trajectory analysis in a dedicated ablation and uses only a subset of the 2,000 curated evolution instances because of computational cost.

Abstract

from arXiv · show

Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve execution mechanisms from experience. However, generalizable harness RSI remains challenging. First, evolving harnesses on evaluation benchmarks or their subsets makes it difficult to distinguish reusable improvements from benchmark-specific adaptation. Second, single-trajectory updates can conflate systematic harness deficiencies with instance-specific reasoning and solution details, producing modifications that transfer poorly to unseen tasks. Third, localizing recurring behavioral deficiencies within monolithic harnesses is difficult, while whole-harness optimization can entangle unrelated mechanisms and complicate attribution and validation. We propose ModularRSI, a benchmark-disjoint, contrastive, and modular framework for generalizable harness evolution. ModularRSI contrasts successful and failed trajectories for the same task and aggregates evidence across tasks to identify recurring behavioral deficiencies. It decomposes the evolvable harness into five functional modules: Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection. Each module evolves independently within a restricted modification scope, followed by an integration stage that combines the evolved modules into a unified harness and resolves potential conflicts. To support benchmark-disjoint evolution, we curate 2,000 executable evolution tasks from external sources that are disjoint from downstream evaluation benchmarks. Experiments on TB2.0 and SWE-Bench Verified show consistent improvements on unseen in-domain and cross-domain tasks, with the evolved harness also transferring across different foundation models.

1 INTRODUCTION

Generalizable harness RSI is difficult because benchmark-based evolution, single-trajectory evidence, and monolithic optimization obscure reusable mechanism improvements. ModularRSI addresses these challenges through benchmark-disjoint data, contrastive cross-task analysis, and independently evolved harness modules.

  • 1 INTRODUCTION: Harness RSI often relies on downstream benchmark data, making generalizable improvement difficult to distinguish from benchmark-specific adaptation.High-quality executable long-horizon tasks with reliable environments and correctness feedback are costly to construct.
  • 1 INTRODUCTION: Single or one-sided trajectories entangle systematic harness deficiencies with task-specific reasoning and solution details, causing poor transfer to unseen tasks.This creates a trajectory-level ambiguity about what should be improved.
  • 1 INTRODUCTION: ModularRSI contrasts successful and failed trajectories, aggregates evidence across tasks, and evolves five harness modules within restricted scopes to localize reusable improvements.The modules are Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection.
  • 1 INTRODUCTION: The benchmark-disjoint protocol uses 2,000 independently curated executable evolution tasks and freezes the evolved harness before downstream evaluation.Modifications are retained only after validation for correctness, executability, and task-specific overfitting.
  • 1 INTRODUCTION: Experiments evaluate ModularRSI on TerminalBench 2.0 and SWE-Bench Verified, with controlled studies comparing modular evolution against joint or non-modular evolution.The contribution summary also reports transfer across foundation models and improvements on unseen in-domain and out-of-domain tasks.

2 RELATED WORK

Prior work uses executable experience to improve models, skills, prompts, and agent mechanisms, but harness evolution still lacks fine-grained, broadly transferable improvement. ModularRSI responds by combining benchmark-disjoint data with functional harness decomposition and contrastive trajectory analysis.

  • 2 RELATED WORK: Table 1 compares representative self-evolving agent and harness frameworks, including whether benchmark instances, trajectories, or benchmark-derived rewards are excluded from harness evolution.The table’s comparison criterion is explicitly labeled Do Not Use Benchmark Data.
  • 2 RELATED WORK: Self-improvement methods use executable trajectories to adapt model weights, accumulate reusable skills or context, and optimize prompts or external agent mechanisms.Related evaluations span repository repair, terminal interaction, and online workflows.
  • 2 RELATED WORK: Distinguishing transferable improvements from development-data adaptation makes data selection important, motivating source-aware rubrics and shortcut filtering.These approaches aim to ensure learning signals reflect the intended capability.
  • 2 RELATED WORK: Existing harness-evolution methods often draw evolution data from evaluation benchmarks or treat the harness monolithically, leaving fine-grained generalizable improvement underexplored.ModularRSI instead constructs benchmark-disjoint data and decomposes the harness into functional modules.

3 METHOD

ModularRSI converts task outcomes into localized harness updates by contrasting trajectories, aggregating diagnoses, evolving five modules independently, integrating them, and retaining only validated changes.

  • 3 METHOD: ModularRSI has three stages: contrastive trajectory analysis, module-wise harness evolution, and validation gates for proposed modifications.The stages identify recurring weaknesses, convert them into localized function updates, and filter changes through program, review, and execution checks.
  • 3 METHOD: The harness is decomposed into Agent Loop, Observation Management, Tool Use, Context Management, and Task Completion Detection, each with a restricted behavioral scope.The Agent Loop coordinates the other modules, while the remaining modules process interaction information and completion signals.
  • 3 METHOD: For each evolution instance, multiple rollouts receive binary task-specific rewards, while trajectory memory supplies prior successful evidence when current rollouts are insufficient.The method groups tasks by rollout outcomes and uses historical trajectories across evolution epochs.
  • 3 METHOD: Contrastive analysis pairs successful and failed trajectories from the same task, while all-failure cases use historical successes or diagnose recurring deficiencies such as loops, incorrect tools, ineffective recovery, and premature termination.All-success cases are also analyzed according to available trajectory evidence, and findings are consolidated with supporting evidence and proposed changes.
  • 3 METHOD: Modification candidates are prioritized by support from distinct tasks, and evolution history helps preserve prior functionality while avoiding redundant or conflicting updates.After module evolution, function merging, task-aware composition, and cross-module integration produce a unified frozen function library.
  • 3 METHOD: Validation retains only executable, compatible, and generalization-oriented modifications through static program checks, diff review, and sampled task execution.Failed checks or runtime validation trigger rollback to the previous function or harness version.

4 EVOLUTION DATASET AND PROTOCOL

The evolution dataset is built from external terminal-related resources and filtered for completeness, non-triviality, and evaluator validity. The resulting 2,000 benchmark-disjoint instances are balanced across task categories and support testing transferable harness improvements.

  • 4 EVOLUTION DATASET AND PROTOCOL: Human annotators use benchmark-derived category labels to retrieve relevant resources from GitHub, Hugging Face, Kaggle, and Linux kernel documentation.The sources support diverse terminal-related evolution tasks.
  • 4 EVOLUTION DATASET AND PROTOCOL: Quality control checks environment completeness, practical non-triviality, and evaluator validity so tasks contain required resources, require meaningful interaction, and test functional correctness.These checks form a multi-stage filtering pipeline for constructed instances.
  • 4 EVOLUTION DATASET AND PROTOCOL: The construction pipeline yields 2,000 high-quality evolution instances that are disjoint from downstream benchmarks and relatively balanced across task categories.TB-related and SWE-related instances have broadly comparable distributions.
  • 4 EVOLUTION DATASET AND PROTOCOL: SWE-related tasks emphasize repository-level software engineering from GitHub, whereas TB-related tasks target direct terminal-environment interaction, limiting overlap between groups.The distinct construction sources help maintain separation between the two task families.
  • 4 EVOLUTION DATASET AND PROTOCOL: The standardized evolution set enables evaluation of whether harness RSI learns generalizable improvements rather than downstream benchmark adaptations.It provides a shared foundation for experiments and future harness self-evolution research.

5 EXPERIMENT SETTING

The experiments evaluate ModularRSI against baselines on TerminalBench 2.0 and SWE-Bench-Verified using accuracy, reliability, and interaction-efficiency metrics.

  • Evaluation covers TerminalBench 2.0's 89 long-horizon terminal tasks and SWE-Bench-Verified's 500 human-validated software-engineering tasks.
  • The study reports Accuracy, Pass@3, StepNum, and Pass3 to measure success, repeated-attempt solving, interaction efficiency, and consistency.
  • Evolution runs for 3 epochs, using 120 instances from each TB-related and SWE-related subset of the 2,000-task evolution dataset.

6 RESULTS

Across benchmark-disjoint evolution settings, ModularRSI improves unseen-task performance, transfers across domains and foundation models, and benefits from modular integration and informative difficulty distributions.

  • 6.1 GENERALIZATION BEYOND EVOLUTION EXPERIENCE: 47.57→52.43 and 73.40→76.45 accuracy gains occur in-domain on TerminalBench 2.0 and SWE-Bench-Verified, respectively.The same frozen evolved harnesses are evaluated across held-out benchmarks after evolution on TB-related or SWE-related tasks.
  • 6.1 GENERALIZATION BEYOND EVOLUTION EXPERIENCE: Cross-domain transfer reaches 75.80 SWE-Bench-Verified accuracy after TB-related evolution and 49.40 TerminalBench 2.0 accuracy after SWE-related evolution.The results are presented as evidence of reusable improvements rather than specialization to encountered evolution tasks.
  • 6.1 GENERALIZATION BEYOND EVOLUTION EXPERIENCE: 30.34→35.96 Pass3 on TerminalBench 2.0 indicates improved reliability across three repeated rollout attempts.The reported interpretation is that harness evolution reduces stochastic failures while increasing average task-solving capability.
  • 6.2 CROSS-MODEL GENERALIZATION: The frozen evolved harness improves GLM-5.2 accuracy from 59.55 to 61.80 and MiniMax-2.5 accuracy from 41.57 to 44.94.Acc, Pass@3, and Pass3 improve across all evaluated foundation models.
  • 6.3 EFFECT OF MODULAR EVOLUTION: Independent module evolution followed by integration achieves 52.43 accuracy from 47.57, while joint and non-modular strategies fall below baseline.The result supports restricting modification scope to reduce interference and combining complementary module improvements.
  • 6.3 EFFECT OF MODULAR EVOLUTION: All single-module variants improve accuracy, with Agent Loop yielding the largest gain and Observation Management substantially reducing execution steps.Integrating the five modules further raises Acc to 52.43 and Pass3 to 35.96.
  • 6.4 COMPARING WITH EXISTING RSI METHODS: ModularRSI improves accuracy by more than five points under benchmark-disjoint evolution, whereas AHE and Meta-Harness change by approximately one point.The comparison is conducted under a unified protocol using the same evolution instances and adapted Harbor framework.
  • 6.5 ANALYSIS OF CONTRASTIVE TRAJECTORY ANALYSIS: Contrastive-pair availability decreases across epochs, suggesting that reusable behavioral improvements reduce co-occurring successful and failed trajectories.Case studies show successful-failed trajectory comparisons exposing behavioral differences that can be abstracted into reusable mechanisms.

7 LIMITATIONS AND CONCLUSION

ModularRSI shows promise as a benchmark-disjoint, contrastive, and modular approach to generalizable harness self-improvement, with gains across unseen tasks, domains, and foundation models. The study nevertheless acknowledges missing ablations and limited evolution scale.

  • Consistent gains across unseen tasks, domains, and foundation models were demonstrated on TerminalBench 2.0 and SWE-Bench Verified.
  • The study does not isolate contrastive trajectory analysis in a dedicated ablation.
  • Main evolution experiments use only a subset of the 2,000 curated evolution instances because of computational cost.
  • ModularRSI’s results support the potential of benchmark-disjoint, contrastive, and modular evolution for more generalizable self-improving harnesses.

A HARNESS ARCHITECTURE

The harness is organized as five interacting modules coordinated by an Agent Loop. Each module has defined inputs and outputs, while Context Management, Observation Management, Tool Use, and completion detection handle distinct execution information.

  • The five-module harness comprises Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection.Figure 5 presents the architecture and module interactions.
  • Agent Loop maintains execution state, schedules calls, handles retries, and decides whether execution continues.It coordinates the other modules and constructs prompts for the LLM.
  • Context Management updates and compresses conversation history, whereas Observation Management converts current terminal output into agent-readable feedback.
  • Tool Use parses model responses into commands and completion signals, executes tool calls, and returns results or errors.
  • Task Completion Detection returns a stop recommendation and reason, while Agent Loop maintains the completion count and controls continuation.One example requires two consecutive completion declarations.
  • Listing 1 specifies selected interfaces and shared configuration, environment-access, and recording services through ModuleCtx.

B STRUCTURED ANALYSIS FINDINGS

Structured findings route trajectory evidence to module-specific diagnoses, guide general modifications supported across tasks, and apply reward-blind review plus cross-module integration to limit overfitting and conflicts.

  • Routing buckets select analysis paths without changing trajectory rewards, and infrastructure-only failures are excluded from harness diagnosis.Table 8 maps method-level trajectory groups to implementation routing buckets.
  • Each finding records the task, analysis lens, locked module, causal decision, trajectory divergence, counterfactual outcome impact, and fixability.
  • A successful example recommends an evidence-gated completion variant requiring a successful build, test, or run before termination.The paired failure lacked build or validation commands and repeatedly declared completion.
  • Modification directions use the complete supporting evidence set to implement shared mechanisms rather than vivid single-task patches.
  • Reward-blind review accepts changes only when they are effective and not task-specific or overfit to evolution tasks.
  • Cross-module integration repairs repeated interaction conflicts by preferring redundancy removal, variant merging, or small in-place repairs without adding new capabilities.Static findings are treated as leads rather than proof.

C.1 TRAJECTORY ANALYSIS

Trajectory analysis compares successful and failed rolls of the same task, then uses a counterfactual, module-scoped test to decide whether a concrete harness change could improve the outcome.

  • The contrastive analysis compares successful and failed trajectories for the same task and harness configuration, using the divergence between rolls as evidence.The analysis may also examine successful but wasteful trajectories in efficiency mode.
  • A module is marked as the culprit only when a concrete in-scope change would plausibly move the failing task toward success.Otherwise, the analysis records no culprit and does not fix.
  • The decision process first identifies behavioral divergence, then evaluates the proposed module change counterfactually and records whether it is fixable now.
  • Historical successful trajectories supplied contrastive pairs for 150 all-fail groups, while eight groups had no valid scores.

D.2 CONTRASTIVE: KEEP THE TASK REQUIREMENTS IN VIEW

The cases show how contrastive and negative analyses convert trajectory evidence into localized harness changes, followed by later traces that exhibit the intended behaviors. These examples also show that the comparisons do not isolate any single update’s effect.

  • D.2 CONTRASTIVE: KEEP THE TASK REQUIREMENTS IN VIEW: Two of three rollouts preserve the NULL-filter requirement and pass all 52 external tests after the updated loop prompts another check before completion.The later run produces 47 rows and explicitly follows the written requirement rather than matching the legacy Scala code.
  • D.2 CONTRASTIVE: KEEP THE TASK REQUIREMENTS IN VIEW: All three FFmpeg rollouts pass ten external tests after the completion-integrity guard repeats the acceptance checklist and prompts explicit ldd verification.The later trace lists libavcodec, libavformat, and libx264 after the agent writes and runs a verification script.
  • D.2 CONTRASTIVE: KEEP THE TASK REQUIREMENTS IN VIEW: The tool-use update reduces mean episodes from 36 to 25 while preserving reward 1 across all three rollouts by routing multiline file-writing calls directly to the helper.Episode counts change from [41, 24, 43] to [26, 18, 31].
  • D.2 CONTRASTIVE: KEEP THE TASK REQUIREMENTS IN VIEW: The three cases diagnose distinct deficiencies: preserving requirements, repeating acceptance checks, and avoiding failed helper calls that cause extra work.Each finding contributes to a promoted module update, but the later traces contain other updates, so the comparisons do not isolate a single change.
  • D.2 CONTRASTIVE: KEEP THE TASK REQUIREMENTS IN VIEW: The evolution datasets include SWE-related sets organized by difficulty estimated from eight trajectories generated by four foundation models.The supplied table passage identifies the difficulty-distribution table, while the prose specifies the estimation procedure and model set.

F THE EVOLUTION CURVE

The evolution-curve analysis evaluates successive ModularRSI harness generations on TerminalBench 2.0 using benchmark rewards and LLM-based trajectory scores. Figure 6 presents accuracy variation under different generations, while the evaluation rubric scores observable trajectories across multiple dimensions.

  • F THE EVOLUTION CURVE: Successive harness generations are evaluated on TerminalBench 2.0 using both benchmark-reward task accuracy and LLM-based trajectory scores.Opus-4.8 assesses trajectories with the prompt specified in Listing 10.
  • F THE EVOLUTION CURVE: Figure 6 compares accuracy variation during evolution under different generations on TerminalBench 2.0.The supplied figure passage identifies the comparison but does not provide numerical values or a stated winner.
  • F THE EVOLUTION CURVE: Evaluation is based only on observable actions, tool interactions, environment feedback, and the final outcome.The rubric instructs the evaluator not to infer that all dimensions are good from task success alone.
  • F THE EVOLUTION CURVE: The trajectory evaluator scores task effectiveness, interaction quality, reasoning-process reliability, context and state management, and efficiency and robustness.Each dimension is scored independently from 1 to 5.
Loading 2609.14857v1…