Source-linked AI summary
Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses
Zhaochen Yu, Yingcheng Wu, Zhenfei Yin, Kaiyuan Chen, Zhe Zhao, Mengdi Wang, Shuicheng Yan, Ling Yang
TL;DR
Long-horizon RSI is hindered by histories that obscure current task state and misalign skill invocation. Recuris couples verified Working Memory with Experiential Memory, uses structured traces for localized validation-gated updates, and improves success across nearly all completed model–benchmark pairs, with gains increasing on longer tasks. The paper concludes that externalized memory control can be an evolving surface around a frozen agent, while noting transfer depends on having unseen failures that the memory can repair.
Problem
Long-horizon RSI lacks a compact, reliable task state for aligning stored experience with evolving execution needs.
Method
Recuris couples verified Working Memory, state-grounded skill invocation, structured traces, localized Meta-Agent patches, and fixed validation while keeping the base model unchanged.
Results
Recuris improves task success in 35 of 37 completed model–benchmark pairs across four long-horizon benchmarks and ten models.
Takeaways & Limitations
The memory-control layer can serve as a trainable surface around a frozen agent, with improvement attributable to components, reversible, and portable across models.
Takeaways & Limitations
Transfer is limited when held-out tasks contain no failures of the kind the evolved memory repairs.
Abstract
from arXiv · showhide
Recursive self-improvement (RSI) remains hard in long-horizon tasks, where growing histories obscure the task state and misalign skill invocation. We introduce Recuris, a recursive Experiential-Working Memory architecture for long-horizon agent harnesses, in which Working Memory tracks task progress and guides skill selection from Experiential Memory, grounding skill use in current needs rather than the full history. This coupling also turns execution into structured evidence that localizes failures to specific memory components. Across tasks, a fixed Meta-Agent turns that evidence into localized, validation-gated updates to Skill Memory that reshape execution and yield new evidence, forming a bounded recursive memory-evolution loop. Across four long-horizon benchmarks and ten models, Recuris improves task success in 35 of the 37 completed model-benchmark pairs, carrying frontier models to SOTA-level task success: on tau-bench it adds +17.8 points to GPT-5.6 Sol and +15.6 to Claude Opus 5, taking Opus 5 to 87.9%, and +16.6/+13.5 points on Qwen3.6-27B/35B on SkillFlow. The advantage widens as the interaction horizon grows, to +32.2 points on the longest tasks, and common long-horizon failures fall by up to 80%. These results position recursively evolving memory as a scalable foundation for RSI, enabling agents to continuously transform accumulated experience into increasingly effective long-horizon behavior. Code: https://github.com/Gen-Verse/Recuris
1. Introduction
Recuris addresses long-horizon RSI by grounding skill use in a verified working state rather than unreliable initial instructions or growing histories. It turns execution into localized evidence for validation-gated memory evolution, improving performance across models and tasks.
- Motivation and approach: Recuris uses Working Memory to track current progress and unresolved goals, guiding skill selection from Experiential Memory.Verified execution feedback updates the task state, creating a closed loop between task state, skill selection, execution, and subsequent state.
- Motivation and approach: EM–WM coupling converts task states, selected skills, actions, and outcomes into structured evidence for component-level failure localization and targeted repair.This avoids rewriting the entire memory system when a failure can be attributed to a specific component.
- Recursive evolution: A fixed Meta-Agent proposes localized Skill Memory updates, and a fixed validation process admits only candidates that pass its checks.The updated memory changes future harness behavior while the underlying LLM and outer improvement procedure remain fixed.
- Results: 35 of 37 completed model–benchmark pairs improve with Recuris across four long-horizon benchmarks and ten models.On τ2-Retail, GPT-5.6 Sol rises from 58.3 to 76.1 and Claude Opus 5 from 72.4 to 87.9.
- Results: +32.2 points on the longest tasks, while common long-horizon failures fall by up to 80%.Structured traces localize failures at 64.8%, compared with 13.0% from task outcomes alone.
2. Method
Recuris separates task execution from cross-task memory evolution: verified working states determine when and which skills are invoked, while structured traces support localized patches behind a fixed gate. The memory-control layer evolves while the base model and outer harness remain unchanged.
- Harness and memory: The harness maintains interaction history and working state while an invocation policy selects experiential skills before the frozen LLM produces actions and receives observations.The resulting trajectory contains actions and observations, while future behavior changes through external memory-control mechanisms.
- Harness and memory: Recuris represents Skill Memory through experiential memory, working-memory specifications, an invocation policy, and checker mechanisms.Together these components define reusable skills, task-state updates, retrieval timing, and evidence-based state-change checks.
- Trace-based evolution: Structured traces link each action and observation to the triggering state, proposed update, and checker evidence used by the cross-task evolution loop.A fixed Meta-Agent uses failed runs to localize a repair target, proposes a component-specific patch, and submits it to a fixed validation gate.
- State-grounded invocation: Skill invocation is state-grounded because retrieval conditions on verified progress and the execution event rather than only the initial instruction or full interaction history.Invocation can occur at defined events such as drafted state-changing calls or turn boundaries, with trigger and retrieval-key behavior itself subject to evolution.
- Validation and recursion: A fixed gate admits a patch only when it repairs the source failure without violating the held-out development criterion.The recursion is bounded by validation, and accepted modifications change the memory-control layer used in later execution.
- Test-time adaptation: Test-time adaptation narrows the evidence and patch space to one task, retrying after localized memory updates against a budget-matched frozen-memory control.The hidden verifier returns only one success bit, and the Meta-Agent never receives the verifier, tests, or expected output.
3. Experiments
The experiments evaluate Recuris across multiple long-horizon settings with frozen models and benchmark-specific memories. Evaluation compares base, initial-memory, and evolved-memory configurations without evolving memory during testing.
- Experimental setup: Four long-horizon benchmarks cover tool-use dialogue, lifelong skill reuse, and terminal work across ten models ranging from a 3B open-source model to frontier models.The tool-use evaluation includes τ2-Retail and τ2-Airline, while SkillFlow tests lifelong skill discovery and procedural reuse.
- Experimental setup: τ2-Retail contains 114 tasks and τ2-Airline contains 50 tasks, with success requiring full environment-verifier reward.Read-action recall and required-write recall are reported separately to distinguish knowing what to do from carrying it out.
- Experimental setup: SkillFlow contains 166 tasks organized into 20 families whose within-family tasks share an execution flow.Each task has its own verifier script and binary reward, making procedural transfer a central test.
- Memory construction: Each benchmark’s Skill Memory is built only from failures generated by the deployment model and read by a fixed Meta-Agent that never sees the test split.The same implementation, prompts, and procedure run at every round and on every benchmark.
- Evaluation protocol: Evaluation compares the benchmark reference agent, Recuris with initial memory, and Recuris with evolved memory while keeping the base model frozen.Nothing is evolved during evaluation, and both Recuris configurations add only the memory-control layer to the reference agent.
3.2. Overall Performance
Recuris improves task success most where benchmark task families share structure, while cross-task evolution yields no patch on Terminal-Bench 2.1. The gains come from evolved memory rather than memory or added context alone.
- 35 of 37 completed model–benchmark pairs improve with Recuris.
- 23.3 and 16.8 points are the largest deployment-model gains on τ2-Retail and SkillFlow, reaching 81.4% and 51.4%.Both intervals exclude zero.
- A fixed Skill Memory adds nothing distinguishable from zero, so the observed gain belongs to memory that revises itself.
- Shared tools, policies, and task families make repairs transferable across τ2-Retail, τ2-Airline, and SkillFlow.Terminal-Bench 2.1 lacks this shared structure.
- Cross-task evolution on Terminal-Bench 2.1 admitted no patch in thirteen evolution-loop runs.Within-task retrying remains the adaptation mechanism there, with or without memory written between attempts.
- Recuris compares the benchmark reference agent with and without the same memory-control layer, using avg@4 task success (%) as the table metric.The caption marks paired task-clustered bootstrap 95% confidence intervals excluding zero with †.
3.3. Analysis of EM–WM Coupling in Long Horizon Tasks
Across long-horizon analyses, Recuris’s advantage comes from verified working state and state-grounded invocation, with the critical harness mechanism differing by domain. Its gains persist as tasks lengthen because failures concentrate on executing required writes rather than retrieving needed information.
- Reliability over longer horizons: Recuris leads the base agent by +17.0 to +44.7 points across all four intrinsic-length quartiles, without a monotone decline as tasks lengthen.Quartiles use a task-intrinsic completion-length map shared across variants, avoiding stratification on each variant’s realized horizon.
- Reliability over longer horizons: Read-action recall remains 88.0–97.9% across variants and quartiles, while Recuris exceeds the base agent by 26.7 points in required-write recall.The length-related separation is therefore on the write path rather than in finding required information.
- Reliability over longer horizons: The base agent executes no required writes in 42% of write-requiring episodes, compared with 16% for Recuris, while the median first-correct-write turn is identical.This re-analysis of held-out episodes supports a consistent execution-coverage pattern, not a controlled manipulation of horizon.
- EM–WM coupling: Adding verified working state is worth +23.9 points against the base agent, whereas experiential memory alone adds +2.0 on 𝜏2-Retail and +1.5 on 𝜏2-Airline, neither excluding zero.Against working state alone, experiential memory adds a further +1.5 with interval [−2.4, +5.7].
- EM–WM coupling: Recuris scores 18.0 points above model-controlled invocation, with interval [+11.6, +24.6], despite the latter exposing the same library and more skill text.The comparison isolates invocation control from skill content because both variants use the same byte-identical ten-skill library.
- Domain-specific critical mechanisms: Removing write review costs 13.5 points on 𝜏2-Airline, while removing the status board costs 17.3 points on 𝜏2-Retail; the other mechanism effects include zero.The domain-specific critical mechanisms form a double dissociation, and later auditing cannot undo an already executed write.
3.4. Analysis of Recursive Skill Memory Evolution
Recursive evolution converts structured execution evidence into localized, validated memory updates whose gains generally transfer to unseen tasks. The analyses also identify boundaries: broken skill binding can mimic diminishing returns, and transfer depends on held-out tasks containing repairable failures.
- Failure localization: 64.8% macro accuracy from structured traces versus 13.0% from task outcomes alone enables more accurate component-level failure localization.Raw trajectories reach 37.0%, while a constant-answer judge scores 33.3%.
- Evolution dynamics: Iteration can compound gains: Run C’s second round adds +6.98 points, while Run A plateaus and Run C’s fourth round reverses.The second-round interval excludes zero.
- Execution mechanism: +17.73 write-recall points accompany +17.44 success points at the peak, indicating gains arise from issuing more required write calls.Across nine final packages, write-recall and task success correlate at r=0.97.
- Evolution dynamics: Run B’s round-4 package was invoked on zero held-out tasks, showing that apparent diminishing returns can reflect broken binding rather than weak evolution.The package scored within the unchanged-package spread, while a same-line round-2 candidate averaged 18.50 invocations per episode.
- Implementation robustness: The two Meta-Agent implementations produce statistically interchangeable gains and converge on the same working-memory, execution-gate, and anti-escalation repair family.The DeepSeek-versus-Claude package difference is −1.45 points with CI [−7.85, +4.65] and p=0.72.
- Validation and safety: Accepted updates break 9.5% of previously solved development tasks, versus 25.9% for rerunning an unchanged package, with none of the four regressions reaching zero.The difference is statistically distinguishable by exact binomial test, p=0.013.
- Held-out transfer: +9.01 to +17.44 points transfer to 86 unseen tasks across three runs, but only when those tasks retain failures the evolved memory repairs.The held-out split is fixed before evolution, and the Meta-Agent never reads it.
- Held-out transfer: Transfer is absent on two τ2-Airline held-out sets, where no unused improvable failures remained and results did not separate from zero.The sets began at 73.0% and 81.0%, and memory invocation was also lower.
3.5. Test-Time Adaptation on Isolated Tasks
Test-time adaptation rebuilds memory from an isolated task’s failed trajectory and applies it on a subsequent attempt. Its headline solved-within-budget advantage is largely attributable to retries, while matched-budget and per-attempt measures show smaller, uncertain learning effects.
- Setup: The regime targets isolated tasks with no shared tools or policies, rebuilding memory from the task’s own failed trajectory for its next attempt.Cross-task evolution has nothing to carry in this setting.
- Results: Test-time adaptation solves 53 of 87 Terminal-Bench 2.1 tasks, or 60.9%, within four attempts.This is +26.4 points over the single-attempt baseline.
- Decomposition: Retrying the seed memory alone also adds +26.4 points, so the headline improvement is not itself evidence of learning.Twenty-three tasks flip to solved and none flip back, with p<10^-4.
- Decomposition: At a matched four-attempt budget, adaptation adds +2.3 points, with 7 tasks won and 5 lost, p=0.774.The memory-control layer without learning is likewise within noise.
- Learning signal: Seven tasks are solved only under adaptation and by no other configuration.Examples include compile-compcert and qemu-alpine-ssh.
- Learning signal: On 56 tasks carrying a learned skill, per-attempt avg@4 rises from 17.4% to 21.9%, a +4.5-point change with every interval containing zero.Across four cuts, two metrics, and two task sets, changes range from +2.3 to +4.5 points.
4. Related Work
Related work spans experiential memory, explicit task-state tracking, and recursive self-improvement across different layers. Recuris distinguishes itself by grounding invocation in verified working state and using structured traces to gate localized memory repairs.
- Experiential memory: Existing experiential-memory systems differ in stored representations, including executable skills, workflows, insights, thought templates, and reusable strategies.Examples include Voyager, AWM, ExpeL, Buffer of Thoughts, and ReasonFlux.
- Skill invocation: Prior work pays less attention to when and how skills are invoked, with fixed injection or unreliable agent-selected retrieval remaining common.Inaccurate injected skills can hurt task completion.
- State grounding: Some methods retrieve by raw observations, whereas Recuris couples experiential memory with working memory to ground invocation in the current verified task state.Observation similarity reflects what the environment looks like, not necessarily what the agent has confirmed achieving.
- State grounding: Explicit-state systems maintain structured states, reflections, ledgers, or constrained flows, but state updates may lack grounding in actual tool returns.Recuris updates progress entries only when supported by tool results.
- Recursive self-improvement: Recursive self-improvement research rewrites different layers, including model runtime, workflow graphs, prompts, policies, and externalized memory.Recuris occupies the memory layer while holding the base model and improvement procedure fixed.
- Recuris distinction: Recuris records state, invoked skill, action, and observation so the Meta-Agent can attribute failures to one of four components and gate repairs against held-out regressions.The four components are experiential memory, working-memory specification, invocation policy, and checkers.
5. Conclusion
Recuris treats recursive self-improvement as evolution of an externalized memory-control layer rather than the model itself. Across benchmarks and models, it improves success broadly and makes failures more localizable, supporting memory as a trainable, portable surface for frozen agents.
- Conclusion: Recuris maintains verified task state, grounds skill invocation, emits structured traces, and applies fixed-gated patches only to implicated memory components.The base model and improvement procedure remain unchanged.
- Conclusion: 35 of 37 completed model-benchmark pairs improve, while horizon-quartile gains range from +17.0 to +44.7 points.The system also adds +17.8 and +15.6 points on τ2-Retail for GPT-5.6 Sol and Claude Opus 5.
- Conclusion: 64.8% fault localization from structured traces versus 13.0% from task outcomes supports component-attributable, reversible, and model-portable improvement.The conclusion frames the memory-control layer as the trainable surface of a frozen agent.
A. Statistical Protocol
The evaluation uses task-clustered paired bootstrap intervals and treats run-to-run variation and cross-population comparisons conservatively.
- Statistical Protocol: 10,000 task-clustered bootstrap resamples produce paired 95% confidence intervals, comparing variants on the same tasks.Tasks, rather than episodes, are resampled because attempts within a task share goals, tools, and environment.
- Statistical Protocol: A difference is not called an effect when its interval includes zero.
- Statistical Protocol: Absolute performance levels are comparable within a table or panel, not across separately drawn evaluation populations.For binary outcomes on the same tasks, the analysis also reports McNemar’s exact two-sided test.
B. Harness Ablation: Where the Gain Comes From
The ablation attributes Recuris’s measurable value to evolved memory content rather than the neutral harness layer alone.
- Harness Ablation: Where the Gain Comes From: Every bare-agent-versus-M0 interval contains zero, with point estimates split between two positive and two negative values.The ablation uses the same harness version, split, task budget, and decoding settings.
- Harness Ablation: Where the Gain Comes From: +10.17 is GPT-OSS-20B’s measurable evolution-term gain, while its harness term remains within noise.The evolution term is the final package minus M0; the harness term is M0 minus the bare agent.
- Harness Ablation: Where the Gain Comes From: The other three evaluated models remain within noise in both harness and evolution terms.
C. Benchmarks and Splits
The benchmarks differ in task organization, adaptation scope, and held-out evaluation, so their results measure different forms of transfer and evolution.
- Benchmarks and Splits: SkillFlow contains 166 tasks organized into 20 families for lifelong skill discovery and evolution.Two duplicated task identifiers yield 164 names after deduplication, but the reported unit is 166 task-family pairs.
- Benchmarks and Splits: Programmatic binary verifiers score tasks without model judges, and reported gains occur under the same verifier.The deployment model moves from 34.6% to 51.4%, while Qwen3.6-27B moves from 42.2% to 58.7%.
- Benchmarks and Splits: Terminal-Bench 2.1 evolves memory from one task’s failed attempt and applies it to that same task on the next attempt.The Meta-Agent receives only the task instruction, failed trajectory, and a hidden-verifier zero bit between attempts.
- Benchmarks and Splits: SkillFlow has no held-out task split: skills are selected within each family and results measure transfer across target models.The benchmark protocol starts each family without skills, processes tasks sequentially, and retains the winning template.
- Benchmarks and Splits: 𝜏2-Retail uses 16 evolve, 12 dev, and 86 test tasks, while the two 𝜏2-Airline lineages use separate evolve, dev, and held-out complements.
D. Compute and Context
The compute and context analysis compares memory-control regimes under matched conditions while acknowledging limits in recoverable accounting data.
- Compute and Context: The four-configuration 𝜏2-Retail comparison holds model, tasks, and budget constant while varying only the memory-control regime.
- Compute and Context: 3,111 additional standing-context prompt tokens produce an 18-point worse result and 46% higher cost per success in the model-controlled configuration.That configuration keeps the whole skill library in context rather than using Recuris’s memory-control regime.
- Compute and Context: Per-turn rendered memory tokens, dollar cost, and aggregated container-harness token records are unavailable from the analyzed artifacts.The authors instead report first-call standing-context increments, static skill sizes, tokens, and an off-parity bounce estimate where available.
E. Case Studies
The case studies examine localized repair, validation-gated updates, within-task adaptation, and procedural memory packages. They show how Recuris uses execution evidence to target repairs while preserving uncertainty when evidence is insufficient.
- Component-Scoped Repair, and a Gate That Waits: Recuris attributes failures to specific components before proposing localized repairs, including experiential memory, procedures, or the harness itself.In the reported round, all repaired clusters targeted experiential memory, while one harness-attributed cluster received no patch.
- Component-Scoped Repair, and a Gate That Waits: The validation gate rejects candidate updates when development evidence cannot distinguish their effect from zero, even when later held-out evaluation supports one rejected candidate.Both candidates improved repair rates on their written-against tasks, but the 12-task development split was too imprecise; the second later exceeded M0 by +11.92 points on 86 held-out tasks.
- Adaptation Inside a Single Task: Within-task adaptation is presented as a mechanism example rather than proof of necessity, because its budget-matched +2.3-point contrast over pure retrying has an interval containing zero.The case study therefore does not settle whether adaptation itself explains the observed outcome.
- Adaptation Inside a Single Task: The within-task information contract limits the Meta-Agent to the task instruction, failed trajectory, and one-bit verifier outcome, excluding tests, expected outputs, and reference solutions.The diagnosis and evidence fields are derived from the failed run rather than hidden verifier information.
- Procedure Is What the Package Carries: SkillFlow isolates the value of procedural descriptions: in weighted-risk-assessment, replacing the canonical template changed performance from zero of eight tasks solved to seven solved.The comparison is in-sample and measures procedural value rather than generalization; transfer to target models is reported separately.
- One Step of EM–WM Coupling: The coupling analysis tests verified working-memory updates by comparing the same 114 tasks with checkers removed and all other conditions held fixed.The case study frames the contrast as isolating the checker component rather than the full architecture.