PaperlayerView exact revision on arXiv

Source-linked AI summary

RSI-Forge: From Research Papers to Environments for Recursive Self-Improvement

Renxiong Wang, Darvin Yi, Abril Herrlein, Anas Mahmoud, Advait Gosai, Lisiman Hua, MohammadHossein Rezaei, Xingang Guo, Anisha Gunjal, Utkarsh Tyagi, David J. Lee, Minglai Yang, Haris Riaz, Chenguang Wang, Huaxiu Yao, Daniel Yue Zhang, Aakash Sabharwal, Tong Zhao, Yunzhong He

arXiv:2610.09426v1cs.AI

TL;DR

Constructing challenging research environments with reliable evaluation still depends on domain expertise, limiting their scale and disciplinary coverage. RSI-Forge turns published papers into executable environments through coordinated construction, reproduction, and review. Across repeated solver attempts, at least one model improved after the first attempt in 84% of environments.

  • Problem

    Constructing research environments with challenging tasks and reliable automated evaluation requires domain knowledge and evaluator engineering, limiting scale and field coverage.

    Source: §1 · p. 2

  • Method

    RSI-Forge uses author, reproducer, and reviewer agents to turn published papers into executable environments with automated evaluators and independently reproduced baselines.

    Source: §1 · pp. 2–3

  • Results

    84% of environments saw at least one model improve after the first attempt, and solver experiments also showed gains beyond reproduced baselines.

    Source: §1 · p. 3, §6 · p. 11

  • Takeaways & Limitations

    RSI-Forge provides a scalable approach to expanding research environments for developing and evaluating self-improving agents.

    Source: §7 · p. 12

    Held-out-data restrictions do not eliminate exploitation opportunities, and some environments left other grading materials accessible.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

Environments are the foundation of recursive self-improvement: they provide the problems agents work on and the feedback used to evaluate progress. Yet constructing challenging research environments with reliable evaluation still depends on domain experts, limiting their scale and disciplinary coverage. We introduce RSI-Forge, a multi-agent pipeline that turns published papers into executable environments for self-improvement. Three agents coordinate construction, reproduction, and review to produce tasks with automated evaluators; each paper's method is independently reimplemented to establish a baseline score. We present 210 environments across 18 fields, including 90 reviewed by independent human domain experts. Both experts and agent judges give high ratings to the potential for improving the provided starting solutions and the evaluators' ability to distinguish solution quality, whereas experts are more critical of shortcut resistance, faithfulness to the source paper, and whether a single idea can exhaust a task. To validate their use for repeated improvement, we evaluate four models over 3 successive attempts on 120 environments, with each attempt inheriting prior code and notes while model weights remain fixed. At least one model improves after the first attempt in 84% of environments. Models also outperform the reproduced paper methods in 68 of the 120 environments, demonstrating room for gains beyond these baselines. Transcript analysis identifies work beyond parameter tuning in 95% of these successful attempts. Analysis of the resulting trajectories shows that models scoring lower on these tasks explore less, more often accept gains smaller than the reported standard error, and rely more heavily on tuning to the development set. RSI-Forge provides a scalable approach to constructing research environments for training and evaluating self-improving agents.

1 Introduction

Research environments need challenging problems and reliable feedback, but constructing them at scale still requires domain expertise and evaluator engineering. RSI-Forge turns published papers into executable environments and tests whether they support repeated improvement.

1 Introduction

  • RSI-Forge converts published papers into executable tasks through author, reproducer, and reviewer agents with distinct responsibilities.The reproducer independently implements the paper method, while the reviewer checks task clarity and evaluator alignment.Source: §1 · p. 2
  • Experts rated improvement potential and evaluator discrimination highly, but were more critical of shortcut resistance, paper fidelity, and task saturation.The rubric also assessed whether tasks posed meaningful research problems.Source: §1 · p. 2
  • 84% of environments saw at least one model improve after the first attempt, and reproduced baselines were surpassed in 68 of 120 environments.Transcript analysis found work beyond parameter tuning in 95% of successful attempts.Source: §1 · p. 3
  • The pipeline produced 210 environments across 18 fields, including 90 independently reviewed by human domain experts.Three agent judges and human experts assessed task quality using a common rubric.Source: §1 · pp. 2–3

2 Related Work

Prior work studies how agents improve outputs, procedures, or environments, while benchmarks draw tasks and evaluation from repositories, competitions, and expert-authored materials. RSI-Forge focuses on constructing executable research tasks from published papers across fields, with reproduced baselines for further improvement.

Related approaches to self-improvement

  • Self-Refine and Reflexion use feedback or retained reflections to improve outputs, while other work optimizes prompts, workflows, agent designs, harnesses, or code.RSI-Forge instead addresses construction of environments that provide challenging problems and measurable feedback.Source: §2 · p. 3

Automated task and environment generation

  • Automated environment-generation work spans curriculum learning, task selection, and executable environments, with some systems coupling task generation to solving or learning.RSI-Forge draws research problems from published papers across diverse fields.Source: §2 · p. 3

Research and engineering benchmarks

  • Benchmarks use repository issues, competitions, and expert-written tasks or rubrics; RSI-Forge constructs cross-field research tasks with independently reproduced baselines.Related efforts include machine-learning experimentation and reproducibility benchmarks, RSI-Exam, and translating published methods into a shared framework [22].Source: §2 · p. 3

3 RSI-Forge

RSI-Forge converts research problems and reference methods from papers into executable tasks, then independently reproduces each method to establish a baseline. Its construction, scoring, and verification procedures define the task inputs, check submissions, and compare results across environments.

3 RSI-Forge

  • RSI-Forge converts research problems and reference methods from published papers into executable environments, with independent method reproduction and reviewer checks.The reviewer assesses whether instructions explain the task and whether the evaluator measures the intended objective, requesting repairs when needed.Source: §3 · p. 4

3.1 Environment Format

  • Each environment specifies an objective, interface, compute budget, starting solution, data-generation code, evaluator, development set, and held-out test set.Its reproduced baseline independently implements the source method under the task definition and resource limits; the target score may be unattainable with solver resources.Source: §3.1 · p. 4

3.2 Multi-Agent Construction

  • The author builds the task and evaluator, automated and reviewer checks assess them, and an information-isolated reproducer must beat the starting solution before acceptance.Failed or unresolved checks return the task for repair; environments that fail after permitted repairs are rejected.Source: §3.2 · pp. 4–5

3.3 Scoring and Verification

  • Raw test scores are normalized between starting and target references; submission checks enforce interfaces and resource limits, but held-out-data restrictions do not eliminate exploitation.The normalized scale maps the starting solution to 0 and the target to 1; some environments left other grading materials accessible.Source: §3.3 · p. 5

4 Building the Corpus

RSI-Forge sources eligible papers, turns research problems into executable environments, and checks their construction through repairs and independent reproduction. The resulting corpus spans diverse fields, includes artifacts for experimentation and held-out grading, and receives expert and agent quality assessments.

4.1 Sourcing

  • Eligible papers had to meet six criteria, including a quantitative objective, measured baseline, deterministic metric, feasible compute demands, room for improvement, and redistribution-permitting licence.All source papers were from 2026; the criteria governed entry into construction but did not guarantee an environment could be built.Source: §4.1 · p. 5

4.2 Construction Funnel

  • Among accepted environments, 79% required at least one revision after failing a construction check; median construction cost was 170 million model tokens per accepted environment.Across construction, 99.6% of input tokens were cache reads.Source: §4.2 · p. 6

4.3 Corpus Composition and Contents

  • The corpus contains 210 environments across 18 fields; the 180-environment main run covered 6–14 environments per field, including robotics, computational physics, statistics, and software engineering.Table 2 showcases task examples and evaluation objectives from six fields.Source: §4.3 · p. 6, Table 2 · p. 6
  • Each environment retains instructions, runnable and data-generation code, a starting solution, reproduced baseline, and evaluation configuration, with development and held-out test sets.Construction records and corpus assessments also include reviewer feedback, written rating justifications, agent transcripts, saved programs, logs, and held-out scores.Source: §4.3 · p. 7
  • In the weather-estimation example, held-out errors were 1.786 for the starting solution and 1.390 for the reproduced paper method.The target score was 1.156; the solver could test changes on 160 development instances, while grading used 324 held-out instances from different simulation conditions.Source: §4.3 · p. 7

4.4 Quality

  • Experts rated room above the starting solution at 4.81 and discrimination between attempts at 4.74; the expert-reviewed environments received an overall mean of 4.24 versus 4.54 for the judge panel.Figure 2 displays ratings from experts and three judges on the same 90 environments.Source: §4.4 · p. 7, Figure 2 · p. 8
  • Experts were more critical of saturation, shortcut resistance, and paper fidelity, with the largest gap on whether one idea could exhaust a task: 3.09 from experts versus 3.98 from agents.Some experts had solver-result summaries unavailable to agent judges; fidelity concerns often involved compute-budget simplifications.Source: §4.4 · p. 7
  • On the same rubric, agent judges rated RSI-Forge above RSI-Exam for distinguishing solution quality and resisting shortcuts, while RSI-Exam scored higher on resistance to saturation.Removing the most generous judge lowered the panel mean to 4.39 without eliminating the largest disagreements; ratings did not select or weight solver outcomes.Source: §4.4 · p. 7, §4.3 · p. 7

5 Experiments

The experiments test whether agents improve on these environments across inherited attempts and how they work toward better solutions. Model performance varied, but repeated attempts produced gains over starting solutions and reproduced baselines, while transcript analysis highlighted differences in exploration and solution-selection behavior.

5.1 Experimental Setup

  • Four models ran three successive sessions on 120 environments, inheriting prior code and notes while keeping model weights fixed; session 3 submissions were the primary outcome.Scores were normalized, and bootstrap intervals resampled environments.Source: §5.1 · p. 8, Table 4 · p. 9

5.2 Performance and Continued Improvement

  • Claude Opus 5 and GPT-5.6-sol had overlapping median-score intervals, both outperformed DeepSeek-V4-Flash, and DeepSeek-V4-Flash outperformed Claude Haiku 4.5.Within-environment ranks gave the same ordering, and paired bootstrap comparisons separated five of six model pairs.Source: §5.2 · p. 9, Table 4 · p. 9
  • At least one model improved by the 0.01 normalized-score threshold after the first session in 84% of environments, and at least one model surpassed the reproduced baseline in 68 of 120 environments.Most score improvement for the three stronger models occurred in the first session, with smaller later gains.Source: §5.2 · p. 9
  • Replaying saved versions revealed that some final submissions scored below an earlier saved solution, despite the earlier version passing submission checks.Held-out material or another run’s graded record was accessed in 22 of 480 chains; excluding those chains changed each model’s median normalized score by at most 0.006 and preserved model ordering.Source: §5.2 · p. 9

5.3 A Three-Session Example

  • In the foothold-selection example, the submitted normalized score rose from 0.15 to 0.33 to 0.58 across sessions as the agent revised its continuation rule and added a ground-support check.The third session removed invalid continuations and returned to the single-best-continuation rule.Source: §5.3 · p. 10, Figure 4 · p. 10

5.4 Transcript Analysis

  • Transcript coding found that Claude Opus 5 explored more than Claude Haiku 4.5, while development-score fitting showed the reverse pattern.The open-coding groups overlap, and “improvement” includes developing and checking solutions without requiring score gains.Source: Figure 3 · p. 9, §5.4 · pp. 10–11
  • Among sessions outperforming the reproduced baseline, 95% involved work beyond parameter tuning; a separate log analysis found sub-standard-error gains accounted for 73% of accepted improvements for Haiku 4.5 versus 44% for Opus 5.The coding describes work in these tasks and does not establish novelty to the source paper’s field.Source: §5.4 · p. 11

6 Discussion

RSI-Forge produces research environments that support repeated improvement, but expert reviews identify challenges in preserving task depth and source-paper fidelity. Solver trajectories expose weaknesses in exploration and experiment selection, while the study’s bounded setup limits what the gains establish.

6 Discussion

  • Most accepted environments required repairs, including fixes for incorrect instructions and checks that could miss failures.The results indicate that automated construction still involves substantial verification and repair.Source: §6 · p. 11
  • Experts rated room to improve and evaluator discrimination highly, but were more critical of shortcut resistance, saturation resistance, and fidelity to source papers.This points to the challenge of preserving research-problem depth when adapting tasks to executable form under a fixed budget.Source: §6 · p. 11
  • Gains over successive attempts and reproduced baselines show improvement, while diminishing gains alone do not establish task saturation.They may also reflect limits in searching for alternatives or using inherited records.Source: §6 · p. 11
  • Lower-scoring models explored less and more often accepted gains smaller than the reported standard error; some final submissions also regressed from saved versions.The study identifies experimental judgment and final selection as targets for future interventions, but its associations do not establish which practices cause higher performance.Source: §6 · p. 11
  • Tasks are adapted to bounded compute, so surpassing a reproduced baseline does not establish an advance on the original research problem.Sessions measure progress on individual tasks, do not isolate inherited records from extra solving time, and do not establish broader research capability.Source: §6 · p. 11
  • Expert review covers 90 of 180 main-run environments, with one expert per environment and no human inter-rater estimate.The construction roles were not ablated, and access restrictions limit independent replication of the full corpus.Source: §6 · pp. 11–12

7 Conclusion

RSI-Forge turns published papers into executable RSI environments with automated evaluators and independently reproduced baselines. The conclusion presents the corpus and solver results, while the appendix specifies selection criteria and acceptance checks.

7 Conclusion

  • RSI-Forge provides a multi-agent pipeline for executable RSI environments with automated evaluators and independently reproduced baselines.The conclusion reports 210 environments across 18 fields and solver gains over successive attempts and beyond reproduced baselines.Source: §7 · p. 12

A.1 Selection and Acceptance

  • Selection requires a quantitative objective, measured baseline, deterministic metric, compute-feasible method, room for improvement, and redistribution-permitting licence.These criteria determine which papers enter construction but do not guarantee that an environment can be built.Source: §A.1 · p. 15
  • Acceptance requires automated checks and reviewer-agent approval at every stage, with development and test sets differing on a documented task condition.Independent reproduction uses the solver execution path and must outperform the starting solution; the author agent has at most three repair attempts per stage.Source: §A.1 · p. 15

A.2 Repairs and Cost

Accepted environments often required repairs, and construction incurred substantial model-token costs. The field-coverage material identifies the composition reported for the main-run corpus.

A.2 Repairs and Cost

  • 79% of 180 accepted main-run environments required repair; the reported stage shares overlap.Repairs occurred during design (53%), build (45%), independent reproduction (21%), and end-to-end testing (20%).Source: §A.2 · p. 15
  • Median construction cost was 170 million model tokens per accepted environment.Median per-environment shares were 82% author, 14% reviewer, and 3% reproducer; these separately summarized medians need not sum to 100%.Source: §A.2 · p. 15

B.1 Field Coverage

  • Table 5 reports field counts for the 180 main-run environments; 30 additional environments were constructed with a second backend.The supplied field-composition caption identifies the table’s scope but gives no field-level counts.Source: Table 5 · p. 15, §B.1 · p. 15

B.2 Review Protocol and Results

The review protocol combines three agent judges with independent domain experts using a shared rubric, though they receive different evidence. Their ratings differ most on resistance to saturation and shortcut resistance, while one judge shows limited harness sensitivity.

B.2 Review Protocol and Results

  • Three agent judges rated all 180 main-run environments after reading instructions and evaluators and could run code with network access.They were not given the author transcript, held-out test data, or solver-result summaries.Source: §B.2 · p. 16
  • Experts reviewed 90 environments, one expert per environment, using the shared 9-dimension rubric; they saw the source paper for the final faithfulness rating.Experts alone received available summaries of solver scores; there was no human inter-rater estimate, so comparisons reflect differences in evidence as well as judgment.Source: §B.2 · p. 16, Table 6 · p. 16
  • Gemini 3.7 Flash’s 1,620 paired ratings differed by a mean absolute 0.13 points between Claude Code and Antigravity; 99% differed by at most one point.Mean ratings were 4.88 and 4.82, respectively, indicating limited sensitivity for this judge under these two harnesses.Source: §B.2 · p. 16
  • Resistance to saturation and shortcut resistance had the largest positive agent-panel gaps over experts, at +0.69 and +0.42, respectively.Removing the highest-mean judge lowered the panel mean from 4.54 to 4.39, versus 4.24 for experts; ratings did not select or weight solver outcomes.Source: Table 6 · p. 16, §B.2 · p. 16

B.2.1 Expert–Agent Disagreements

Experts rated saturation resistance, shortcut resistance, and faithfulness less favorably than agent judges, with concerns illustrated by task exhaustion, access controls, and compute-budget simplifications. The review rubric describes quality scores, while comparisons between task sets do not isolate construction effects.

B.2.1 Expert–Agent Disagreements

  • Agent-panel minus expert gaps were +0.89 for resistance to saturation, +0.58 for shortcut resistance, and +0.50 for faithfulness.The comparison uses the same rubric, but some experts had solver-result summaries that agent judges did not receive.Source: §B.2.1 · p. 16, §B.2.2 · p. 17
  • One expert rated saturation resistance lowest after a supplied search method covered about 69% of the improvement band in roughly five CPU-minutes.Solver results reaching the target score in the first session also informed the expert; reaching a normalization target does not rule out further raw gains or prove exhaustion.Source: §B.2.1 · pp. 16–17
  • A shortcut concern depended on runtime access controls, while exposed parameters and held-out family definitions could let solutions branch instead of estimating the intended quantity.The reported exposure was treated as an environment defect.Source: §B.2.1 · p. 17
  • One task scored 2 from an expert versus 4.5 from the panel after replacing real recordings, pretrained pipelines, and a trimodal model with synthetic low-dimensional data and a linear-Gaussian model.The task retained 50-way zero-shot classification and audio-text semantic comparison.Source: §B.2.1 · p. 17
  • Table 7 compares ratings for 34 manually constructed tasks and 90 RSI-Forge environments, but the unpaired sets do not isolate automated construction’s effect.The comparison excludes faithfulness because the sets lack a common source-paper reference.Source: Table 7 · p. 17

B.2.2 Review Rubric

  • The quality rubric reproduces descriptions for scores 1, 3, and 5, and groups dimensions as in the rating summaries.The human review form places faithfulness to the source paper last.Source: Table 8 · p. 18

C.1 Experimental Setup

The evaluation carries saved solutions and notes across repeated sessions while keeping model weights fixed, and records improvements over reproduced baselines. It also examines how coding procedures characterize solver behavior and checks whether access flags or alternative coders change key patterns.

C.1 Experimental Setup

  • Across three successive attempts, saved code and notes were inherited while model weights remained fixed; final submissions were inherited regardless of score.Sessions ended when the agent stopped or after 60 minutes, and development-evaluator calls had no fixed count limit.Source: §C.1 · p. 18
  • Every saved version and final submission was replayed on held-out data, with scores withheld during solving; sessions lacking a valid measurement remained unscored.Table 9 reports saved versions and output volume for the 120 main evaluation environments.Source: §C.1 · p. 18, Table 9 · p. 19
  • Starting-solution scores map to 0 and target scores to 1, but targets need not be attainable and normalization does not validate them.Targets may use task-specific bounds or idealized solutions; construction checks require revision if the reproduced baseline is too close to the target.Source: §C.1 · p. 18

C.2 Improvements over Reproduced Baselines

  • At least one session surpassed the reproduced baseline in 68 of 120 environments; counts were 65 for GPT-5.6-sol, 63 for Claude Opus 5, 54 for DeepSeek-V4-Flash, and 24 for Claude Haiku 4.5.Counts overlap; the median normalized baseline score was 0.76 where no model surpassed it and 0.46 where at least one did.Source: §C.2 · p. 19
  • Successful sessions were coded as involving work beyond parameter tuning in 95% of cases, while 5% were tuning-only and 0% received neither label.Labels included repairing a non-running solution and evaluating alternatives, so they do not all imply a new method or establish what caused a gain.Source: §C.2 · p. 19

C.3.1 Coding Procedures

  • Open coding produced 62 families grouped as improvement, exploration, and shortcuts, with overlapping session labels and an ungrouped family for unchanged inherited solutions.The process used transcript summaries and required matching quotations for positive assignments; open-coded families appear in Figure 6.Source: §C.3.1 · p. 19, Figure 6 · p. 21
  • A ten-category fixed-practice codebook was proposed from 24 documents and frozen before application; assignments lacking matching quotations were discarded.Quotations support traceability but do not validate the interpretation or narrated action; three coding procedures serve distinct purposes, and labels were not human-validated.Source: §C.3.1 · pp. 19–20, §C.3 · p. 19

C.3.2 Behavior Patterns and Sensitivity Checks

  • Access flags covered 22 of 480 model–environment chains; excluding them left model ordering unchanged and moved no median final normalized score by more than 0.006.Four of 68 baseline-surpassing environments included a successful flagged session, but this overlap does not attribute gains to access; additional coders corroborated exploration and shortcut contrasts without establishing individual-label correctness.Source: §C.3.2 · pp. 20, 22

C.3.3 Gains Smaller than the Reported Standard Error

Among accepted improvements in sessions with recoverable evaluator logs, some gains were smaller than the reported standard error. A second construction backend yielded a larger corpus, with rating and solver-ordering comparisons reported across unpaired sets.

C.3.3 Gains Smaller than the Reported Standard Error

  • Accepted gains were smaller than the reported standard error in 73% of Claude Haiku 4.5 and 44% of Claude Opus 5 improvements in 139 sessions with recoverable logs.The rates use accepted improvements in those sessions as the denominator and describe selection, not a significance test between correlated scores.Source: §C.3.3 · p. 22

C.4 Environments by Backend Model

  • Replacing Claude Opus 5 with GPT-5.6-sol in the same sourcing queue, checks, and repair procedure produced 30 additional environments, for 210 overall.The unpaired construction sets had judge mean-rating differences below 0.2 points, and solvers retained the same ordering; differences may reflect task composition as well as backend choice.Source: §C.4 · p. 23, Table 11 · p. 23, Table 12 · p. 23
Loading 2610.09426v1…