PaperlayerView exact revision on arXiv

Source-linked AI summary

REVERE: Reflective Evolving Research Engineer

Balaji Dinesh Gangireddi, Aniketh Garikaparthi, Manasi Patwardhan, Arman Cohan

arXiv:2603.20667v2cs.SEcs.AI

TL;DR

Research-coding agents face poor cross-task generalization because existing adaptation relies on local signals and weak prompt updates. REVERE combines a Global Training Context with targeted code-based prompt edits, improving performance across three benchmarks while reducing cost and adaptation time.

  • Problem

    Existing prompt adaptation methods rely on local signals and weak update mechanisms, while research-coding workflows involve heterogeneous repositories and weak feedback.

  • Method

    REVERE is a lightweight, self-adapting framework that consolidates experience in a Global Training Context and applies targeted code-based edits to prompts.

  • Results

    REVERE improves over expert-crafted instructions across SUPER, ResearchCodeBench, and ScienceAgentBench, with gains of up to 4.50%, 1.3%, and 4.89%, respectively.

  • Takeaways & Limitations

    REVERE supports consistent adaptation across long-horizon, single-shot, and interactive settings while using nearly 10× less cost and adapting up to 2.7× faster than prior methods.

  • Takeaways & Limitations

    Prompt adaptation alone may not capture task-specific knowledge, and the growing Global Training Context may accumulate stale information over time.

Abstract

from arXiv · show

Existing prompt-optimization techniques rely on local signals, causing poor generalization across tasks. In addition, they also rely on weak update mechanisms, such as full-prompt rewrites or unstructured merges, which cause knowledge loss and unstable adaptation. These limitations are magnified in research-coding workflows, which involve heterogeneous repositories and weak feedback, limiting abstraction and learning across tasks. We introduce Reflective Evolving Research Engineer (REVERE), a lightweight, self-adapting agent framework that learns from a Global Training Context, distills recurring cross-repository failure modes into reusable heuristics, and applies targeted, code-based edits to agent prompts. REVERE is evaluated across settings ranging from long-horizon to single-shot benchmarks, and improves over prior expert-crafted instructions by 4.50% on SUPER, 1.3% on ResearchCodeBench, and 4.89% on ScienceAgentBench. It does so at nearly 10x lower cost and 2.7x faster adaptation than existing prompt-optimization baselines, demonstrating that self-adapting agents with continual learning and global memory consolidation can meaningfully evolve their capabilities over time.

1 Introduction

Research-code reproduction challenges agents with long horizons, weak feedback, tacit assumptions, and heterogeneous repositories. REVERE addresses these limitations with global experience consolidation and targeted prompt adaptation, improving benchmark performance while reducing adaptation overhead.

  • Research-code reproduction requires coordinating long-horizon tasks under weak feedback, inferring tacit assumptions, and accumulating knowledge across heterogeneous frameworks.
  • Local evaluation signals and bounded experience windows can overfit recent outcomes and converge to strategies that do not generalize across tasks.
  • REVERE combines configurable prompt fields, a Global Training Context, and targeted code-based updates to reuse strategies without overfitting to specific tasks.
  • 4.50% gains on SUPER, 1.3% on ResearchCodeBench, and 4.89% on ScienceAgentBench improve over expert-crafted instructions across three research-coding benchmarks.
  • Nearly 10× lower adaptation cost than ACE and GEPA and up to 2.7× faster wall-clock adaptation demonstrate substantially lower overhead.

2 Related Work

Research-coding benchmarks expose persistent gaps in reproducibility, while prompt-optimization methods increasingly use reflective, evolutionary, and runtime-adaptive strategies. However, task-level methods often fail to transfer across tasks.

  • Research-code reproducibility benchmarks reveal persistent performance gaps even in multi-agent and search-based systems.
  • Classical and newer prompt-optimization methods treat prompts as tunable parameters through reinforcement learning, gradient-free search, heuristic search, reflection, or evolution.
  • Task-level adaptation methods improve instance performance but fail to transfer across tasks.

3 Reflective Evolving Research Engineer (REVERE)

REVERE adapts editable agent fields through iterative reflection using local evaluation signals and globally consolidated context. Its Reflector performs targeted code edits while accumulating reusable strategies and cross-task guidance.

  • 3.1 Setup: REVERE formalizes agent behavior through editable system, task, and cheatsheet fields without modifying model weights.
  • 3.2 Method Overview: The adaptation loop evaluates batched tasks, sends outcomes to the Reflector, and iteratively updates the fields.
  • 3.2 Method Overview: The Reflector uses local evaluation outcomes, ground truth when available, prior reflections, and auxiliary task descriptions to guide updates.
  • 3.3 Global Training Context (GTC): The Global Training Context combines a cumulative cheatsheet, reflection history, and auxiliary context to support reusable guidance, coherence, and cross-task generalization.
  • 3.4 Reflection and Update Mechanism: Instead of regenerating full prompts, the Reflector executes short Python programs that modify only relevant field portions.
  • 3.4 Reflection and Update Mechanism: Jinja2 field templates enable conditional rendering, shared instruction reuse, and avoidance of duplicated near-identical heuristics.

4 Experiment Setup

The experiments test REVERE across long-horizon, single-shot, and interactive research-coding benchmarks, using offline and online adaptation regimes with varying supervision. Results compare adaptation methods from a shared baseline under benchmark-specific evaluation protocols.

  • Benchmarks: The evaluation spans long-horizon SUPER, single-shot ResearchCodeBench, and interactive ScienceAgentBench research-coding tasks.SUPER contains 45 tasks, ResearchCodeBench contains 212 tasks, and ScienceAgentBench contains 102 tasks.
  • Adaptation regimes: Offline adaptation uses fixed training and validation tasks before evaluation on held-out test sets, whereas online tasks arrive sequentially without repetition or ground-truth supervision.Offline train/validation/test splits are 9/9/27 for SUPER, 34/34/144 for ResearchCodeBench, and 20/20/62 for ScienceAgentBench.
  • Reporting: Table 1 reports offline-adaptation values averaged over 5 runs, with symbols marking ground-truth conditions and gains or drops relative to Static SOTA.The table uses ✓/✗ for GT hints and no GT, – for inference-only, and +x or −x for changes relative to Static SOTA results marked by †.
  • Supervision: The experiments compare GT supervision, which provides benchmark-specific ground-truth hints during training, with No-GT adaptation from execution traces alone.Online adaptation is inherently No-GT because ground truth is unavailable throughout the sequential task stream.
  • Baselines and implementation: All adaptation methods start from a minimal baseline prompt, while SUPER and ScienceAgentBench use ReAct-style agents equipped with code tools.The baseline prompt contains only the core task description, isolating the effects of adaptation mechanisms.
  • Baselines and implementation: GEPA is designed for offline optimization, whereas REVERE and ACE adapt using training tasks under the reported experimental configurations.GEPA uses a held-out validation set for Pareto candidate selection, while REVERE and ACE adapt using only the training split.

5 Results and Analysis

REVERE improves research-coding adaptation across offline and online settings while preserving performance under limited feedback. Its gains extend across benchmarks and adaptation iterations, with lower cost and faster execution than prior methods.

  • Offline adaptation: REVERE is the only method that consistently benefits from stronger supervision, improving over Static SOTA across all benchmarks and metrics.This pattern is reported for offline adaptation results in Table 1.
  • Method comparison: GEPA improves Output Match while Landmarks collapse to zero, whereas REVERE improves both metrics simultaneously.The contrast is attributed to GEPA’s discard-and-rewrite design, which overfits recent traces and discards stable reasoning structure.
  • Online adaptation: REVERE improves over the baseline across all benchmarks and metrics in the stricter online regime without ground-truth labels.Tasks arrive sequentially, each is encountered once, and future tasks are unseen during adaptation.
  • Adaptation dynamics: REVERE improves steadily across iterations, surpassing Static SOTA and ACE by the final iteration instead of locking onto a weak early prompt.GEPA’s heuristic-based Pareto selection locks in a weaker prompt based on early evaluation.
  • Efficiency and exploration: REVERE converts increased tool usage into stronger performance gains more efficiently than ACE.Baseline tasks cluster in the low-tool-use, low-performance region, while REVERE-GT explores a broader range of outcomes, including efficient wins and informative failures.
  • Efficiency and exploration: REVERE completes adaptation over 2.7x faster than ACE and GEPA while keeping adaptation cost nearly 10× lower.The reported cost comparison excludes shared training cost; REVERE’s SUPER curation cost is $0.16 versus $0.47 for ACE.

6 Conclusion

REVERE is a lightweight, self-adapting framework for research-coding agents that combines global training context with precise prompt code-edits. It improves over strong baselines across long-horizon, single-shot, and interactive settings under supervised and unsupervised learning while reducing adaptation cost and time.

  • REVERE maintains reflection history, auxiliary context, and a cumulative cheatsheet alongside local evaluation signals.
  • REVERE applies precise code-edits to prompts for interpretable and efficient offline and online adaptation.
  • REVERE shows consistent gains over GEPA and ACE across SUPER, ResearchCodeBench, and ScienceAgentBench under supervised and unsupervised learning.
  • REVERE is nearly 10× more cost-effective and 2.7× faster to adapt than existing prompt-updating mechanisms.

A.1 Benchmark datasets and resource limits

The evaluation standardizes tasks, resource limits, agents, tools, execution environments, and models across three benchmarks. Ground-truth content remains incomplete, while model and context choices reflect the extreme demands of research-code adaptation.

  • Benchmark datasets and resource limits: The three benchmarks use fixed task-level LLM-call and execution-time limits, enabling controlled comparisons across methods.SUPER is most expensive because it is long-horizon and interactive, whereas ResearchCodeBench is single-shot and more lightweight.
  • Benchmark datasets and resource limits: Ground-truth hints are deliberately incomplete, preventing direct answer copying while supporting adaptation.SUPER expert trajectories omit exact dependency versions, and ResearchCodeBench provides only partially masked code.
  • Implementation settings: REVERE uses benchmark-specific training epochs, batch sizes, and auxiliary-context sizes, with no separate validation set.Training uses five epochs, while batch and auxiliary-context sizes vary by benchmark.
  • Implementation settings: GEPA and ACE use their official implementations with benchmark-specific optimization settings and step sizes.GEPA uses iterative optimization budgets, while ACE uses a step size of 1 across benchmarks.
  • Controlled evaluation environment: All methods share the same baseline agent, tool interfaces, containerized environment, Azure gpt-4.1 model, and hardware.The shared setup is intended to ensure performance differences reflect methodological distinctions rather than implementation artifacts.
  • Controlled evaluation environment: GPT-4.1 is selected for its 1M-token context window because offline adaptation requires 300k–500k-token Reflector inputs and 40k–120k-token agent inputs.

B Robustness on Adaptation

REVERE’s adaptation gains are statistically supported and generally consistent across runs, especially with ground-truth hints. Without ground-truth supervision, significance weakens on domain-knowledge-heavy tasks, while prompt growth remains controlled relative to competing approaches.

  • Statistical robustness: With ground-truth hints, REVERE significantly outperforms GEPA and ACE across all benchmarks and metrics.All reported comparisons have p < 0.001 and t > 0.
  • Statistical robustness: Without ground-truth hints, significance is largely retained on SUPER but weakens on domain-knowledge-heavy benchmarks, particularly ScienceAgentBench.REVERE versus ACE yields p = 0.050 on Accuracy and p = 0.061 on Success Rate, while means remain directionally higher.
  • Statistical robustness: REVERE’s improvements are generally consistent across runs, with limited confidence-interval overlap against GEPA and ACE on primary metrics.The separation is especially visible on SUPER Overall and ScienceAgentBench Success Rate.
  • Prompt growth: REVERE’s prompt components grow in a controlled, distributed manner, allocating capacity where needed across iterations.
  • Prompt growth: ACE exhibits steep cheatsheet growth, whereas GEPA keeps prompts short by regenerating instructions and discarding accumulated knowledge.REVERE maintains compact prompts across benchmarks, but controlled growth does not remove domain-familiarity constraints.

C.1 Semantic Drift and Edit Operation Analysis

The analysis evaluates whether targeted code-based edits preserve prompt meaning while enabling restructuring. REVERE shows intermediate retention and a diverse mix of string operations, contrasting with GEPA’s rewriting and ACE’s append-only accumulation.

  • Retention metrics: Prompt retention is measured with Levenshtein retention, git-diff retention, and semantic similarity across consecutive adaptation iterations.Higher values indicate greater similarity to the previous iteration, with 1.0 denoting no change.
  • Retention results: GEPA’s near-zero retention reflects semantic drift from full-prompt rewriting, while ACE’s near-perfect retention reflects append-only accumulation.REVERE’s intermediate retention of approximately 0.85–0.95 indicates targeted restructuring.
  • Retention results: ACE’s high retention does not necessarily indicate better adaptation because repeated additions produce unbounded prompt growth.
  • Edit operation analysis: REVERE uses diverse code-based edits, including split, join, replace, insert, and append, indicating restructuring rather than simple accumulation.The operation mix persists across ablation variants and is therefore not attributed to a single Global Training Context component.

D.1 Generalization Across Model Configurations

REVERE’s adaptation gains persist across heterogeneous task-model and Reflector configurations, including cross-family pairings. The results indicate that Reflector quality is more important than task-model capability alone.

  • Generalization Across Model Configurations: Reflector quality predominantly drives adaptation gains, while pairing a strong task model with a weak Reflector yields only marginal improvement.A weaker GPT-5.4-mini task model with a stronger Reflector approaches the unadapted strong-model baseline, whereas the reverse pairing improves only marginally.
  • Generalization Across Model Configurations: Gemini-3-Flash improves substantially over its own unadapted baseline under same-family adaptation, the largest relative gain in Table 8.This further supports adaptation gains within a model family rather than only from a particular task-model configuration.
  • Generalization Across Model Configurations: 55.20% / 54.71% is achieved by GPT-5.4 with Gemini-3-Flash-Preview as Reflector on the Test Subset / Full Set.This cross-family result is comparable to the same-family GPT-5.4 + GPT-5.2 pairing and only marginally below the matched GPT-5.4 + GPT-5.4 ceiling.
  • Generalization Across Model Configurations: Recurring infrastructure and workflow mismatches, rather than algorithmic or model-reasoning errors, dominate the identified failure modes.The recurring mismatches include dependency resolution, environment configuration, script selection, data formatting, and metric extraction across repositories.
  • Generalization Across Model Configurations: Prompt adaptation proceeds through coherent additions, refinements, and localized deletions of semantically grouped instruction units.The pattern indicates structured accumulation and policy sharpening rather than wholesale prompt rewriting.

D.4 Cold-Start Robustness

REVERE remains effective when online adaptation starts from a prompt that contradicts expert-crafted guidance. It recovers most of the performance achieved from standard-prompt adaptation, though initialization retains a small effect.

  • Cold-Start Robustness: Prompt evolution uses incremental additions, modifications, and removals of semantically grouped instruction capsules across adaptation iterations.The snapshots cover the first, twelfth, thirtieth, and final iterations, showing structured accumulation rather than prompt rewriting.
  • Cold-Start Robustness: 56.00% Overall is reached from a counter-instructed prompt, versus 59.30% with standard-prompt adaptation and 47.60% for the unadapted standard baseline.The counter-instructed start substantially exceeds the unadapted standard baseline while leaving a 3-point gap to standard-prompt adaptation.
  • Cold-Start Robustness: REVERE’s gains do not depend on a reasonably designed initial prompt, because adaptation converges toward comparable performance from contradictory instructions.The remaining 3-point gap indicates that initialization quality has a minor but non-negligible effect on final performance.
  • Cold-Start Robustness: Offline SUPER adaptation trajectories classify tasks by outcome groups, including exploratory improvement, declining, and steady improvement.Each task is plotted by Judge Score and total tool calls across baseline and adaptation stages.

D.5 SUPER with LLM-as-Judge Metric

SUPER evaluation supplements its binary pipeline metric with an LLM-as-judge rubric that awards partial credit for completed stages. This exposes intermediate progress while preserving the original metric, alongside task-level adaptation trajectories and context-length considerations.

  • SUPER with LLM-as-Judge Metric: The LLM-as-judge rubric captures partial credit for correctly completed pipeline stages when downstream steps fail.SUPER does not track intermediate progress within a task pipeline, motivating a common evaluation policy across tasks.
  • SUPER with LLM-as-Judge Metric: JudgeScore combines the LLM Rubric Score with the Overall Metric, retaining the original SUPER metric in final evaluation.The composite score surfaces progress that the binary pipeline metric would otherwise obscure.
  • SUPER with LLM-as-Judge Metric: SUPER tasks cover fine-tuning, prompt-based and few-shot learning, parameter-efficient training, sequence-to-sequence generation, structured prediction, retrieval, and domain-specific tasks.The benchmark therefore spans heterogeneous methodological domains and long-horizon research-coding workflows.
  • SUPER with LLM-as-Judge Metric: Fourteen exploratory improving tasks cluster largely in parameter-efficient training, NLI/QA, and prompt-based domains with methodological overlap to training tasks.The graph identifies overlaps such as galore with hype, pet and parallel-context-windows with glee, and team and cet with rah-kbqa.
  • SUPER with LLM-as-Judge Metric: Nine steady-improvement tasks show incremental gains or comparable performance with fewer tool calls, while four declining tasks have limited domain overlap with training data.Domain proximity appears sufficient to prevent regression but not to produce the broader generalization of the exploratory cluster.
  • SUPER with LLM-as-Judge Metric: Growing context length remains a fundamental challenge in prompt-based adaptation, partially mitigated by limiting Reflector additions per training batch.Proposed relief includes periodic flushing with recovery and reducing adaptation frequency as edit sizes shrink.

E.4 ScienceAgentBench: Why CodeBERTScore Drops While Success Rate Improves

ScienceAgentBench’s Success Rate and CodeBERTScore can move in opposite directions because passing programs receive an automatic CodeBERTScore of 1.0, while valid solutions may differ structurally from one reference. REVERE therefore reduces failures even when passing-code similarity is lower.

  • ScienceAgentBench: Why CodeBERTScore Drops While Success Rate Improves: CodeBERTScore averages are largely determined by failing programs because every passing program receives CBS = 1.0.A lower average CBS does not necessarily indicate less reference similarity among passing programs.
  • ScienceAgentBench: Why CodeBERTScore Drops While Success Rate Improves: REVERE reduces failing programs from 19 to 9 while achieving higher Success Rate, despite lower CBS among passing programs than Static SOTA.The lower passing-program CBS reflects structural differences from the single reference file rather than reference-output access or execution control.
  • ScienceAgentBench: Why CodeBERTScore Drops While Success Rate Improves: Passing solutions can differ from the reference in column selection, visualization, and schema handling while still solving the task correctly.Task 7 contrasts fixed-column heatmaps and exact-schema assumptions with automatic column detection, a bar chart, and runtime schema adaptation.
  • ScienceAgentBench: Why CodeBERTScore Drops While Success Rate Improves: REVERE’s Reflector is instructed to make minimal, reversible, behavior-focused edits while preserving schemas, placeholders, and template structure.The prompt also prioritizes structural failures, generalized guidance, conservative recovery, and early exit when evidence is weak.
  • ScienceAgentBench: Why CodeBERTScore Drops While Success Rate Improves: Cross-run learning records task-agnostic Cheat Sheet guidance for recurring patterns and avoids environment- or tool-specific hacks.The prompt distinguishes reusable knowledge from instance-specific strategies and directs generalized improvements into shared fields.
Loading 2603.20667v2…