PaperlayerView exact revision on arXiv

Source-linked AI summary

RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement

Fanqing Meng, Lingxiao Du, Qiguang Chen, Ziqi Zhao, Haocheng Lu, Mengkang Hu, Michael Qizhe Shieh

arXiv:2607.25886v1cs.SEcs.CL

TL;DR

Existing benchmarks do not isolate whether LLM agents can automate data-centric post-training research, a capability relevant to turning evidence of model failures into better models. RSIBench-Data evaluates iterative data-strategy research under a fixed post-training stack. Agents improve on their first valid attempt in some settings, but continued search often fails to preserve the best score.

  • Problem

    Existing benchmarks entangle data-centric research decisions with optimization, serving, evaluation, and system implementation, leaving unclear whether LLM agents can reliably automate this research.

    Source: §1 · pp. 1–2

  • Method

    RSIBench-Data evaluates agents that iteratively develop and validate training-data strategies for a fixed target model under shared training, serving, evaluation, and budget controls.

    Source: §1 · p. 2

  • Results

    58.33% of settings improve over the first valid attempt, while 78.26% of searches continuing after their best score finish with a lower-scoring attempt.

    Source: §7 · p. 13

  • Takeaways & Limitations

    The results reveal a discovery–reliability gap: agents make useful data-centric discoveries but do not consistently turn feedback into better training-data strategies.

    Source: §7 · p. 13

    Evaluation uses one representative run per setting and fixed subsets for four benchmarks; checkpoint selection and official evaluation use the same task subset, so held-out-task generalization is not established.

    Source: §6 · p. 13

Abstract

from arXiv · show

Recursive self-improvement requires turning evidence of model failures into better models. Data-centric post-training research entails diagnosing capability gaps, designing and validating training-data strategies, and learning from checkpoint feedback. Can LLM agents automate this loop? Existing benchmarks entangle research decisions with optimization, serving, evaluation, and systems implementation, obscuring agents' research capability. We introduce RSIBench-Data, a controlled benchmark of LLM agents as data-centric researchers with a fixed post-training stack. Agents iteratively revise training-data strategies for a fixed target model; training and serving use Tinker-backed services, official evaluation runs through Harbor and E2B sandboxes, and budgets are fixed across agents. We evaluate four frontier agents on six benchmarks across software engineering, terminal use, scientific question answering, and mathematics. Agents demonstrate core data-centric research capabilities: in 58.33\% of settings, they improve upon the first valid attempt by refining strategies from feedback. However, improvement is inconsistent. Among searches continuing after the best observed score, 78.26\% end with a lower-scoring final attempt, while the rest only recover the same peak. A strong candidate may therefore appear early or midway through a run even as later revisions fail. Trajectory analysis identifies four patterns in stronger runs: accurate hypotheses, validation-grounded supervision, behavior-aligned data, and preservation of strong checkpoints. These findings suggest that current agents can make useful data-centric discoveries but cannot yet translate feedback into consistent improvements. RSIBench-Data provides a measurable, auditable testbed for the research capabilities required for recursive self-improvement. We open-source our code at https://github.com/evolvent-ai/RSIBench-Data.

1. Introduction

Existing benchmarks do not isolate whether LLM agents can automate data-centric post-training research. RSIBench-Data fixes the surrounding stack and evaluates agents’ iterative strategy revisions, finding useful but inconsistent gains from feedback.

1. Introduction

  • Recursive self-improvement requires converting observable evidence about model failures into training experience that can improve models.Tool-use environments expose plans, calls, observations, recovery attempts, and verifier outcomes, but these trajectories are not themselves training experience.Source: §1 · p. 1
  • Existing benchmarks address broad questions across data selection, environment construction, and post-training, but entangle research decisions with optimization, serving, evaluation, and implementation.That entanglement makes it difficult to isolate agents’ research capability.Source: §1 · p. 2
  • RSIBench-Data isolates data-centric research by fixing the surrounding post-training stack and evaluating iterative training-data strategy development from controlled feedback.Each round includes choosing what the target model should learn, validating a strategy, training a checkpoint, and using evaluation feedback to choose the next strategy.Source: §1 · p. 2
  • 58.33% of settings improve over the first valid attempt, but 78.26% of searches continuing after their best score finish lower, revealing a discovery–reliability gap.The remaining searches only recover the same peak; stronger runs feature accurate hypotheses, validation-grounded supervision, behavior-aligned data, and checkpoint preservation.Source: §1 · pp. 2–3
  • The paper frames post-training data synthesis as a research loop of forming capability hypotheses, designing and validating data strategies, learning from checkpoint evidence, and selecting candidates under a fixed budget.Its benchmark protocol compares criteria explicitly required and enforced by each benchmark.Source: §1 · pp. 2–3, Table 1 · p. 3

2. Related Work

Prior work studies data curation, generated experience, or broad automated post-training, but often does not compare agents’ data-centric research under a controlled stack. RSIBench-Data evaluates that role while keeping the surrounding training and evaluation loop fixed.

2. Related Work

  • Data curation benchmarks study selection from fixed corpora, generated datasets, or feedback-driven curricula; RSIBench-Data gives one researcher agent ownership of data-synthesis hypotheses and revisions.DataEnvGym and Curation-Bench introduce student-model or benchmark feedback for adapting curricula and curation strategies.Source: §2 · p. 3
  • Recursive self-improvement systems automate experience acquisition and construct supervision across rounds, but generally evaluate particular methods rather than compare agents in the same controlled researcher role.RSIBench-Data instead isolates a frontier researcher agent improving a separate target model.Source: §2 · p. 3
  • Automated post-training benchmarks grant broad control over data, optimization, hyperparameters, and implementation, measuring end-to-end improvement while obscuring data-centric research decisions.Related evaluation work also covers persistent multi-agent evolution, contamination-resistant tests, and long-horizon tool-use targets.Source: §2 · p. 4
  • RSIBench-Data occupies a middle ground: it evaluates an agent’s data-centric research while keeping training, serving, and official evaluation fixed.It records hypotheses, strategies, feedback-driven revisions, and checkpoint choices, making the research process auditable.Source: §2 · p. 4

3. RSIBench-Data: Evaluation Task and Protocol

RSIBench-Data tests whether an LLM researcher can improve a separate target model through iterative data-centric post-training. It fixes the surrounding services and budget, records the research process, and evaluates the selected checkpoint separately.

3. RSIBench-Data: Evaluation Task and Protocol

  • The benchmark asks whether an agent can hypothesize about a fixed model, design targeted training data, interpret checkpoint evidence, and preserve its best finding under a finite budget.It evaluates both the final checkpoint and the process that produces it, with official performance as the primary outcome.Source: §3 · p. 4, §3.1 · p. 4

3.1. Problem Definition

  • Given a fixed base model, benchmark evidence, and resource budget, the researcher proposes training data and a whitelisted configuration, receives selection feedback, and revises or stops.After its attempts, the agent selects a checkpoint using its history before official evaluation; two separations support attribution: the outer stack is fixed, and iteration uses selection feedback rather than official outcomes.Source: §3.1 · p. 4

3.2. Fixed Infrastructure

  • The base model, optimizer, serving path, sandbox, verifier, and scoring rule are fixed and shared; agents submit training artifacts rather than training code.Each attempt follows the same Tinker-backed LoRA SFT and Harbor–E2B evaluation path, leaving structured records that link strategies to checkpoint behavior.Source: §3.2 · p. 5

3.3. Agent Inputs and Data-Use Constraints

  • Agents receive benchmark descriptions, success criteria, public seed materials, schemas, diagnostics, and a budget, but not a ready-made training corpus.Evaluation-only tasks, labels, and trajectories cannot be used as training supervision.Source: §3.3 · p. 5

3.4. Closed-Loop Data-Synthesis Research

  • The benchmark fixes the outer data–training–evaluation sequence but leaves filtering, verification, curricula, mixtures, failure attribution, and continued search to the researcher.The shared SFT backend trains each submitted distribution; Harbor and E2B return permitted scores, trajectories, verifier outcomes, errors, costs, and elapsed time.Source: §3.4 · p. 6

3.5. Final Checkpoint and Official Evaluation

  • After its attempts, the agent selects one checkpoint, which is evaluated on the target benchmark’s test set using the fixed protocol as its final score.Source: §3.5 · p. 6

4. Experimental Setup

The experimental setup varies researcher agents and benchmarks in a controlled 4 × 6 matrix while holding the target model, training interface, evaluation services, and per-run budget fixed.

4.1. Researcher agents

  • The main matrix compares Claude Code Opus-4.8 and Sonnet-5 with Codex gpt-5.6-sol and gpt-5.6-terra under the same interface, data-use constraints, and budget.Claude Opus 4.8 is the fixed external rollout model; comparisons assess research through a bounded interface that also permits whitelisted configuration choices.Source: §4.1 · p. 6

4.2. Shared training pipeline

  • Each attempt trains a LoRA-adapted Qwen/Qwen3.5-35B-A3B-Base checkpoint through shared Tinker SFT and evaluates it through Harbor in E2B sandboxes.Main runs have a nominal 16-hour wall-clock budget and a $500 Tinker budget.Source: §4.2 · p. 7

4.3. Benchmarks and evaluation

  • Six benchmarks span repository-based software engineering, long-horizon terminal use, and verifiable scientific and mathematical reasoning.Agents share evaluation subsets, runners, verifiers, and parameters; the primary outcome is the selected checkpoint’s official benchmark-native score.Source: §4.3 · p. 7

5. Results and Analysis

Across the benchmark, later attempts often improve the candidate frontier, but post-peak revisions frequently regress. Stronger trajectories reflect specific data-synthesis decisions and disciplined checkpoint selection.

5.1. Main Results

  • 14 of 24 settings improve over the first valid candidate, but 18 of 23 searches continuing past a peak end below it and five only recover the peak.Historical-best checkpoint selection can preserve an earlier discovery despite weaker later attempts.Source: §5.1 · p. 7
  • No agent dominates the suite: the three SWE-style tasks have three different winners, while the best scores on SWE-bench Pro, Multilingual, and Terminal-Bench 2.0 are 9.00%, 22.00%, and 20.22%.Table 2 summarizes official scores and resources; its bold entries mark the best official score within each benchmark.Source: §5.1 · pp. 7–8, Table 2 · p. 8

5.2. Iterative evolution and candidate selection

  • Iteration gains depend on the benchmark and agent: AIME 2026 improves for three of four agents with a 25.42-point median gain, while GPQA Diamond’s median gain is 4.00 points.On GPQA Diamond, all four agents improve their first-candidate score, but the starting frontier is already 56.00%–65.00%.Source: §5.2 · p. 10

5.3. Budget utilization and search efficiency

  • Across 24 settings, runs use a median of 4.5 valid candidates, $62.51 in Tinker cost, and 6.55 hours, with substantial variation in each resource.Terminal-Bench examples show that additional post-peak spending may fail to refresh the candidate frontier; selected checkpoints can nevertheless score higher officially than their selection peak.Source: §5.3 · pp. 10–11

5.4. Mechanism analysis

  • Stronger trajectories are associated with accurate capability diagnoses, validation-grounded data, behavior-aligned supervision, and preservation of strong checkpoints.In the reasoning-effort diagnostic, max effort raises the mean selection score over the first four comparable attempts from 35.5% to 44.0%, alongside larger datasets and higher candidate-loop cost.Source: §5.4 · pp. 11–12

5.5. Early Experiment of RSI

  • In the exploratory SWE-bench Pro run, strategy revisions raise a candidate score from 8% to 21%, but every checkpoint remains below the 33% unadapted-model reference.The sequence includes repaired tool pairing and longer multi-step traces before the 21% result; a near-no-op adapter reaches 22% and is selected.Source: §5.5 · pp. 12–13

6. Discussion and Limitations

The evaluation does not establish adaptive generalization: checkpoint selection and official evaluation use the same task subset, and each setting uses one representative run. The study also compares complete researcher systems rather than isolating their components.

6. Discussion and Limitations

  • One representative run per setting and fixed subsets for four benchmarks limit the evaluation’s breadth.Checkpoint selection and official evaluation use the same task subset.Source: §6 · p. 13
  • Using the same task subset for checkpoint selection and evaluation measures fresh-environment performance, not generalization to statistically held-out tasks.Repeated trials, evaluator-private splits, and refreshed tasks are identified as needed to measure stability and transferability.Source: §6 · p. 13
  • Researcher identity bundles the underlying LLM, agent scaffold, and configured reasoning effort, so comparisons do not isolate any one component.Source: §6 · p. 13

7. Conclusion

RSIBench-Data evaluates data-centric research for recursive self-improvement under a fixed post-training stack. Agents sometimes improve through iteration, but later revisions often regress, revealing a gap between discovery and reliable progress.

7. Conclusion

  • RSIBench-Data is a controlled benchmark for evaluating data-centric research capability under a fixed post-training stack.The authors describe it as making the research process measurable and auditable.Source: §7 · p. 13
  • 58.33% of settings improve on the first valid attempt, while 78.26% of searches continuing after their best score finish lower and the rest only recover that peak.Source: §7 · p. 13
  • Historical-best selection can protect the submitted checkpoint from regressions but does not make the underlying research process reliably progressive.Source: §7 · p. 13

A. Evaluation Suite and Seed Repositories

The evaluation suite aligns agents within each benchmark on evaluation settings and provides public seed sources for constructing training experience. AIME is evaluated with four decoding runs per problem.

A. Evaluation Suite and Seed Repositories

B. Representative Actions Within the Data-Synthesis Research Loop

The described operations elaborate the agent-controlled stages of the data-synthesis loop, while leaving the implementation architecture open. Agents can use subsets of the representative actions, combine them, or introduce alternatives within benchmark contracts.

B. Representative Actions Within the Data-Synthesis Research Loop

B.1. Diagnose the Next Training-Data Need

The agent diagnoses capability gaps by organizing observed failures and estimating where added supervision may transfer. It then uses checkpoint behavior to distinguish competing explanations for an unsuccessful attempt.

B.1. Diagnose the Next Training-Data Need

  • Failure diagnosis can distinguish missing knowledge from reasoning, planning, tool-use, state-tracking, recovery, verification, or completion errors.Failures may also be grouped by capability, error type, task family, environment, tool, interaction length, or execution stage.Source: §B.1 · p. 17
  • Comparing intended lessons with post-training behavior helps distinguish a wrong capability hypothesis from weak examples, invalid data, or mismatched supervision format.These explanations imply different revisions even when aggregate scores are the same.Source: §B.1 · p. 17

B.2. Synthesize Targeted Training Experience

The strategy covers targeted task and trajectory construction, validation and organization of supervision, iterative revision under budget, and benchmark-specific instantiation. Across these choices, the fixed evaluation protocol and current model failures shape what data is useful.

B.2. Synthesize Targeted Training Experience

  • Synthetic tasks can target diagnosed capabilities and construct executable initial states that satisfy the fixed evaluation service’s interaction contract.Task coverage can vary by error type, repository, tool, interaction length, controlled task variants, or permitted public sources.Source: §B.2 · pp. 17–18
  • Training experience may include successful, failed, and recovery trajectories, represented as full interactions, selected decisions, actions, critiques, or repaired continuations.The representation should teach behavior required by the evaluation protocol without overwhelming training with irrelevant interaction history.Source: §B.2 · p. 18

B.3. Validate and Organize Training Experience

  • Validation can filter invalid, unverifiable, contaminated, malformed, or inconsistent examples and remove duplicates; long contexts can be compressed while preserving decision-relevant state and evidence.Difficulty estimates can guide example balance, training order, and mixtures across task families, trajectory types, supervision formats, and difficulty levels.Source: §B.3 · p. 18

B.4. Revise the Strategy and Manage Search

  • After each training and evaluation cycle, the agent can revise data sources, task coverage, representation, filtering, exposure, or mixture based on checkpoint diagnosis.It must allocate remaining budgets among exploration, exploitation, rollback, checkpoint preservation, and stopping; Figure 7 marks selection scores, historical bests, and official evaluations differently.Source: §B.4 · p. 18, Figure 7 · p. 19

B.5. Instantiation Across Benchmark Families

  • Data strategies vary by benchmark: software and terminal tasks emphasize tool-use and task completion, while science and mathematics emphasize reasoning supervision and verification.The appropriate strategy depends on the target benchmark and current failures, while the surrounding SFT and evaluation machinery remains fixed.Source: §B.5 · p. 19

B.6. Representative Evolution Cases

The representative cases trace how strategy changes and checkpoint selection relate to benchmark outcomes. They provide process evidence rather than single-factor causal ablations.

B.6. Representative Evolution Cases

  • The three plotted runs were selected for valid official evaluations, at least three valid candidates, and auditable data changes; Table 3 frames mechanisms as process observations, not causal ablations.Figure 7 distinguishes valid-attempt selection scores, historical-best candidates, and official evaluations.Source: §B.6 · p. 19, Table 3 · p. 21
  • 49.17% official AIME score for the selected tenth checkpoint exceeded the 30.00% base-model reference, after the ninth candidate jumped to 55.00% selection score.The tenth candidate reached 55.83%, the development-stage peak; later candidates did not surpass it, and the score sequence does not isolate which strategy component caused the jump.Source: §B.6 · p. 19
  • 9.00% selection score followed supervision grounded in real successful submissions, versus 7.00% for synthetic finish actions on SWE-bench Pro.The strategy used evaluation failures to build a curriculum around reviewing final diffs, verifying edits, and submitting tasks.Source: §B.6 · pp. 19–20
  • 20.22% official Terminal-Bench 2.0 score (18/89) exceeded the 1.12% base-model reference after focused supervision raised selection performance to 15.73%.A later stop-oriented modification fell to 13.48%, while the selected checkpoint preserved the earlier best.Source: §B.6 · p. 20

Related papers

Loading 2607.25886v1…