PaperlayerView exact revision on arXiv

Source-linked AI summary

AReaL-RSI: Adaptive Scaling and Coordination of Recursive Self-Improvement

Wenhao Zhou, Jiajun Song, Haitao Wang, Zipeng Qiu, Yingqi Peng, Ruida Xu, Jiale Li, Wenshuang Li, Shusheng Xu, Wei Dong, Hao Dai, Tongkai Yang, Yi Gao, Binhang Yuan

arXiv:2610.18083v1cs.AI

TL;DR

Persistent-objective RSI faces the challenge of turning additional research effort into progress when early scores can misstate promise, findings can lose their conditions, and procedures may not carry forward. AReaL-RSI coordinates persistent researchers through conditional sharing and post-campaign revision, improving benchmark outcomes under the reported resource allocations.

  • Problem

    Scaling RSI for persistent objectives raises how additional inference and experimental resources should improve outcomes despite misleading early scores, context-losing shared findings, and procedures that may not transfer to later tasks.

    Source: §1 · pp. 1–2

  • Method

    AReaL-RSI combines a resource-aware formulation with leader-guided persistent investigations, conditional memory sharing, and evidence-based post-campaign revisions to researcher orientation, knowledge, and skills.

    Source: §1 · p. 3

  • Results

    AReaL-RSI improves Terminal Bench resolution by 2.2 percentage points and mean KernelBench speedup by 32.2–36.8% over CODEX-GOAL under reported resource allocations.

    Source: §1 · p. 3, §7 · p. 10

  • Takeaways & Limitations

    Across seven comparable task–backbone pairs, experience sharing with specialized researcher roles yields lower kernel latency than independent exploration or sharing alone.

    Source: §7 · p. 10

    The study evaluates configurations at reported parallelism; scaling curves, matched-compute efficiency, and gains from cross-campaign harness evolution remain unmeasured.

    Source: §5.3 · p. 9

Abstract

from arXiv · show

Scaling recursive self-improvement requires additional computation to produce useful research progress in reaching persistent objectives. Intuitively, more researcher agents can explore more hypotheses, but early evaluation scores might misrepresent their promise, shared findings could lose their applicability conditions, and useful procedures could remain confined to completed investigations. In this paper, we present AReaL-RSI, a framework for organizing a persistent research team of agents without updating backbone LLM weights. We formulate the scaling problem in terms of evaluated outcomes and whole-system resource expenditure. Concretely, a leader agent selects task-appropriate research orientations and guides ongoing investigations; conditional memory sharing distinguishes unresolved problems from supported findings and enables recipient adaptation. During recursive evolution, after each campaign, evidence-based review revises researcher orientation, knowledge, and skills for later campaigns. Experiments compare AReaL-RSI with Codex-Goal on downstream harness optimization, GPU kernel optimization, and AI research tasks using two backbones. Under the reported resource allocations, AReaL-RSI improves Terminal Bench performance by 2.2 percentage points for each backbone and increases mean KernelBench speedup by 32.2-36.8%; further scalability comparisons favor the design of AReaL-RSI over native scaling: combining experience sharing with specialized researcher roles yields 1.9-27.5% lower kernel latency than independent exploration and 1.3-33.9% lower latency than sharing alone across all seven comparable task-backbone pairs.

1 Introduction

AReaL-RSI frames test-time scaling as organizing persistent investigations and reusable research procedures, coordinating within-campaign research with evidence-based revisions for future campaigns.

1 Introduction

  • Scaling out can test complementary hypotheses and retain procedures for later tasks, but progress must be weighed against coordination and evaluation costs.Executable artifacts in kernel optimization and machine learning engineering require experimentation to establish their quality.Source: §1 · pp. 1–2
  • Three scaling difficulties are misleading early artifact scores, sharing findings without their conditions, and failing to extract reusable procedures from completed investigations.Early gains may rely on fragile assumptions, while broad sharing can homogenize exploration.Source: §1 · p. 2
  • Existing systems provide coordination, memory, or procedure-evolution mechanisms, but combining them still requires policies for evidence sharing and procedural revision.The paper focuses on coupling evidence interpretation, selective exchange, and reuse of research procedures.Source: §1 · p. 2
  • Within campaigns, a leader selects task-suited harnesses and guides persistent researchers; conditional sharing preserves sources and conditions while leaving recipients to assess applicability.After campaigns, evidence-based review revises orientation, reusable knowledge, and skills for later teams.Source: §1 · pp. 2–3
  • The scaling formulation separates current artifact quality from investigative promise and distinguishes additional-computation opportunities from team-organization efficiency.It connects task-conditioned team composition and leader guidance to adaptation within and across campaigns.Source: §1 · p. 3
  • AReaL-RSI improves Terminal Bench resolution by 2.2 percentage points and mean KernelBench speedup by 32.2–36.8% over CODEX-GOAL under reported resource allocations.Combining experience sharing with specialized roles also yields 1.9–27.5% lower kernel latency than independent exploration and 1.3–33.9% lower latency than sharing alone across seven comparable task–backbone pairs.Source: §1 · p. 3

2 Adaptive RSI Scaling

Adaptive RSI scaling organizes additional computation into useful investigations, with task-conditioned teams and evidence-driven leadership coordinating parallel research under resource costs.

2 Adaptive RSI Scaling

  • Scaling RSI means organizing additional computation into useful investigations, while sharing within campaigns and revising harnesses across campaigns.The objective includes inference, experiments, coordination, and post-campaign review costs.Source: §2 · p. 3

2.1 Why Scale Out RSI?

  • Parallel researchers can test hypotheses under costly or delayed feedback, but duplicate work and indiscriminate sharing waste computation or risk premature convergence.The formulation compares expected task-specific utility under componentwise resource constraints, including researcher, leader, experiment, and review costs.Source: §2.1 · p. 3

2.2 Task-Conditioned Team Composition

  • Each researcher investigates the complete task with persistent local state, while the leader selects task-conditioned harnesses that provide complementary orientations, knowledge, and skills.Harness assignments remain fixed within a campaign, though working files remain editable.Source: §2.2 · pp. 3–4

2.3 Evidence-Driven Coordination

  • The leader uses reports and experimental records to request evidence, revise plans, advise researchers, or await feedback.Observed candidate quality reflects performed checks, not untested correctness or investigative promise; recommendations weigh explanations, unresolved tests, and remaining resources.Source: §2.3 · p. 4

3 Conditional Memory Sharing between Researcher Agents

AReaL-RSI structures shared experiences so recipients can assess their conditions and evidence, while limiting proactive spread of unresolved problems. Recipients can test or adapt findings locally and report results that inform later sharing.

3.1 Experience with Sources and Applicability Conditions

  • Experiences separate observed results from interpretations and record tested conditions, uncertainty, and supporting references so recipients can assess the evidence.Conditions include inputs, acceptance criteria, and evaluation settings.Source: §3.1 · p. 4
  • Each experience has a stable identifier, source, and problem-or-positive label distinguishing unresolved difficulties from findings the sender considers supported and usable.An experience may include outcomes and an uncertain explanation, allowing diagnosis without prematurely rejecting or adopting a modification.Source: §3.1 · pp. 4–5

3.2 Selective Propagation of Problems and Solutions

  • The leader restricts proactive routing of unresolved problems, may recruit one additional relevant researcher, and confines further updates to that recipient and the source; positive transfers have no such recipient bound.Positive findings are routed selectively according to relevance.Source: §3.2 · p. 5

3.3 Recipient Adaptation and Corrective Feedback

  • Recipients assess shared findings against local conditions, may test or adapt advice, and report results under the same identifier so the leader can qualify findings and revise later sharing.Applicability assessment and local verification are prescribed behavior, not an enforced adoption gate.Source: §3.3 · p. 5

4 Experience-Driven Evolution of Researcher Harnesses

AReaL-RSI turns campaign experience into revised researcher harnesses for later campaigns. Reviews use evidence to propose reusable changes, which are published for subsequent teams, while transfer assessment and resource accounting include the costs and boundaries of reuse.

4 Experience-Driven Evolution of Researcher Harnesses

  • The cross-campaign process represents reusable research procedures, revises them from evidence, and publishes revisions for subsequent campaigns.Source: §4 · p. 5

4.1 Evolvable Research Orientation, Knowledge, and Skills

  • A researcher harness combines orientation for choosing hypotheses, knowledge of findings and their boundaries, and skills specifying repeatable procedures; its profile changes without updating model weights or the leader policy.The profile is supplied through initial context and workspace resources.Source: §4.1 · p. 6

4.2 From Experimental Experience to Procedural Revision

  • After exploration and applicable final verification, a separate review uses campaign evidence to propose reusable edits, retaining conditions, uncertainty, and contradictions while excluding complete answers and hidden-test information.Review proposals include a rationale and same-campaign evidence references; publication checks do not require demonstrated improvement on future tasks.Source: §4.2 · p. 6

4.3 Reuse and Specialization in Subsequent Campaigns

  • Accepted revisions update the shared library for new campaigns, where task-conditioned selection specializes existing harness identities; version checks prevent stale reviews from overwriting newer changes.Earlier campaigns retain their snapshots, and new teams do not inherit unrestricted access to past traces or complete answers.Source: §4.3 · pp. 6–7
  • Transfer assessment requires later campaigns with declared task splits, library versions, and feedback permissions, and total resource accounting includes post-campaign review alongside campaign costs.Target held-out outcomes must remain outside updates used to solve that target; resource vectors include additive costs such as tokens, model charges, and experiment compute.Source: §4.3 · p. 7

5 Evaluation

The evaluation compares AREAL-RSI with CODEX-GOAL across downstream harness optimization, GPU kernels, and AI research using two backbones. It reports end-to-end outcomes and compares parallel research configurations.

5.1 Experimental Setup

  • The setup covers Terminal Bench 2.1, seven KernelBench tasks, and four RSI Bench research tasks, comparing two backbones under task-specific resource allocations.Terminal Bench uses 89 tasks with the task-solving model fixed; KernelBench and RSI Bench allow one and two effective hours, respectively, for CODEX-GOAL and each AREAL-RSI researcher agent.Source: §5.1 · p. 7

5.2 End-to-End Performance Compared with CODEX-GOAL

  • AREAL-RSI improves all six reported metrics with both backbones, yielding 12 favorable comparisons across downstream harnesses, kernels, and AI research.The comparisons are under the reported resource allocations; they do not establish equal-compute efficiency.Source: §5.2 · p. 7, Table 1 · p. 8
  • 2.2 percentage points is the Terminal Bench resolution gain for each backbone; mean KernelBench speedup rises 32.2–36.8%.Terminal Bench changes from 75.3% to 77.5% with Luna and 85.4% to 87.6% with Astra; KernelBench changes from 1.74× to 2.30× with Luna and 6.28× to 8.59× with Astra.Source: Table 1 · p. 8, §5.2 · p. 8
  • AI research results improve across all four objectives, although Astra’s jailbreak-score gain is only 0.119.Self-distillation improves by 11.66 percentage points with Luna and 5.56 with Astra; swarm accuracy improves by 13.34 and 13.33 percentage points, respectively; data-curation perplexity decreases by 26.0% and 9.8%.Source: §5.2 · p. 8

5.3 Comparison of Parallel Research Configurations

  • 1.9–27.5% lower kernel latency than independent exploration and 1.3–33.9% lower than sharing alone: specialization with experience sharing wins all seven comparable pairs.Table 2 reports final candidate latencies in milliseconds; Luna’s L3P10 fails under every configuration.Source: Table 2 · p. 8, §5.3 · pp. 8–9
  • Experience sharing alone lowers latency in five of seven pairs, but raises Luna’s L1P63 latency by 30.9% and L2P71 latency by 13.4%.Its outcomes therefore vary by task and backbone.Source: §5.3 · pp. 8–9
  • The comparisons do not isolate individual mechanism effects or measure research-direction diversity; matched-compute efficiency, scaling curves, and cross-campaign evolution remain unmeasured.The study evaluates configurations at the reported parallelism.Source: §5.3 · p. 9

6 Related Work

Related work spans agent evolution, autonomous research and search, and coordination and memory sharing. These approaches retain or adapt procedures, allocate investigative effort, or organize collaboration and reusable knowledge.

Agent Evolution

  • Agent-evolution work ranges from optimizing designs and prompts to revising improvement procedures, including editable self-modification and replay-based exploration policies.Examples include ADAS [15], the Darwin Gödel Machine [1], GEPA [16], Hyperagents [2], and Dream-RSI [3].Source: §6 · p. 9

Autonomous Research and Agentic Search

  • Autonomous research systems use tree search, evolutionary archives, worker reallocation, effort balancing, or proposal screening to organize experimentation and exploration.Examples include AIDE and AI Scientist-v2 [18] [19], AlphaEvolve [13], Population Based Training [11], and Fluid search [12].Source: §6 · p. 9

Agent Coordination and Memory Sharing

  • Coordination and memory systems retain records and reusable knowledge, while prior work also identifies risks from homogenized proposals and evaluator exploits.The literature includes specialized-agent coordination [8] [9], curated collective memory [10], and persistent state or hierarchical experience [27] [28].Source: §6 · p. 9

7 Conclusion

AREAL-RSI frames test-time RSI scaling as organizing persistent investigations and reusable research procedures. Its reported results support coordinated parallel research, while matched-compute scaling and cross-campaign evolution benefits remain unestablished.

7 Conclusion

  • AREAL-RSI combines within-campaign conditional memory sharing with evidence-driven harness evolution across campaigns.Under the evaluated resource allocations, it improves Terminal Bench resolution by 2.2 percentage points and mean KernelBench speedup by 32.2–36.8% over CODEX-GOAL; specialization with experience sharing lowers latency across all seven comparable task–backbone pairs.Source: §7 · p. 10

Related papers

Loading 2610.18083v1…