PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning

Bingxiang He, Pengqi Sun, Shihao Peng, Shaoxuan Li, Huiyu Xiao, Hongjian Shi, Zixuan Fu, Ziqing Qiao, Yuxin Zuo, Cheng Qian, Xinqiang Yu, Jiayi Li, Xiusi Chen, Yizhen Song, Xiangyang Wang, Iofai Lei, Yudong Wang, Ning Ding, Zhiyuan Liu, Xiaozhi Wang, Chaojun Xiao

arXiv:2610.22194v1cs.AIcs.CL

TL;DR

Existing systems give agents one kind of data control, although data engineers make a broader range of changes. AutoSFT combines execution isolation, complementary analysis, and synthesis; component additions improve benchmark results across instruction-following and code tasks.

  • Problem

    Existing systems give agents a single kind of data control, such as selecting rows from a fixed pool or calling a generator, rather than the broader range used by data engineers.

    Source: §1 · p. 1

  • Method

    AutoSFT uses a meta agent to delegate execution, gather complementary analysis proposals, and synthesize data for capabilities the pool lacks.

    Source: §1 · p. 2, §4 · p. 6

  • Results

    +6.5 IFEval from analysis, +3.4 IFEval and +2.9 IFBench from synthesis, and +3.7 MBPP+ in code accompany component additions.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The results suggest that automating SFT data research depends more on covering diverse data actions than on strengthening a single action.

    Source: §1 · p. 3

    Because AutoSFT optimizes benchmark scores, it could in principle inflate leaderboard numbers; off-target and held-out benchmarks make target overfitting visible.

    Source: Ethics statement · p. 12

Abstract

from arXiv · show

Language model agents increasingly automate post-training, but when given the full training stack they converge on supervised fine-tuning and spend most effort revising training data. Existing systems give the agent one kind of control, selecting rows from a fixed pool or calling a generator, whereas a human data engineer adjusts proportions, prunes, repairs, replaces and synthesizes the data. We design a data action space for these changes and ask whether an automated system can make the full range rather than one of them, and what structure that requires. We first run this loop with a single agent, and it reaches only one action, adjusting category proportions, while its context fills with execution output and is discarded mid-search. We therefore introduce \textbf{AutoSFT}, a multi-agent system where a meta agent separates execution from analysis, gathers proposals from independent angles, and calls a synthesis subagent for capabilities the pool cannot cover, with the model, the trainer, and the budget fixed within each comparison. Adding each component in turn helps: the executor halves context loss, the analysis subagents lift the best IFEval by $+6.5$, and synthesis adds $+3.4$ IFEval and $+2.9$ IFBench. The same loop transfers to code, where analysis adds $+3.7$ MBPP+ over a single agent that does not improve. Every result comes from runs that retrain from the base model each round, so each component's effect is individually attributable. Our findings suggest that automating SFT data research depends more on covering diverse data actions than on a stronger single action.

1. Introduction

Existing systems provide only one kind of data control, while experienced data engineers make a broader range of changes. AutoSFT combines execution isolation, complementary analysis, and data synthesis; controlled component additions improve instruction-following and code results.

1. Introduction

  • Existing systems offer row selection or generation, while data engineers also adjust proportions, prune, repair, replace, synthesize, and order data.The paper frames these operations as six data-action families.Source: §1 · pp. 1–2
  • A single agent collapses onto one action family and spends context on execution output rather than data analysis.Its execution and analysis compete for context, while one proposal stream reinforces recent choices.Source: §1 · p. 2
  • AutoSFT separates execution, gathers proposals from four analysis angles, and synthesizes data for capabilities the pool barely covers.A meta agent integrates proposals into the next recipe; the roles share a candidate pool and skill library.Source: §1 · p. 2, Figure 1 · p. 2
  • +6.5 IFEval from analysis, +3.4 IFEval and +2.9 IFBench from synthesis, and +3.7 MBPP+ in code accompany component-by-component additions.Comparisons fix the base model, initial pool, token budget, training script, and evaluation, retraining from the base model each round.Source: §1 · pp. 2–3
  • The results suggest that automating SFT data research depends more on covering diverse data actions than on strengthening a single action.The contributions include a six-action framing, a multi-agent system, and component-wise evaluation.Source: §1 · p. 3

2. Automated SFT Data Research

The research loop searches over data recipes while retraining from a fixed base checkpoint. Its data actions change recipe rows in distinct ways, while other research actions remain outside the defined action space.

2.1. Research loop

  • Each round trains from the fixed base checkpoint using a recipe that assigns sampling weights to the current pool, which grows when rows are written.The round returns benchmark scores, per-example failures, training dynamics, and comparisons with earlier rounds; the research policy uses feedback and history to choose the next data action.Source: §2.1 · p. 3
  • A fixed three-epoch trainer and equal token cap keep recipe rounds within a comparison on the same training budget.The loop runs until an objective is achieved.Source: §2.1 · p. 3

2.2. Data research actions

  • Data actions specify which rows to train on, their frequency and order, and whether rows are edited or newly written.The six families are classified by their effect on recipe rows and by the level at which the change is decided; labels are applied after a run.Source: §2.2 · p. 4
  • Validation-set design and additional checkpoint evaluations are research actions, not data actions, and remain outside the action space.The actions considered here concern changes to the data recipe.Source: §2.2 · p. 4

3. Pilot study: What limits single agents in SFT data research

The pilot study examines a single agent’s SFT data-research loop and finds it limited by narrow data actions, shrinking per-round training data under a wall-clock budget, and context loss. These observations motivate separating execution from analysis and adding multiple proposal sources.

3. Pilot study: What limits single agents in SFT data research

  • The pilot reproduced an open-ended post-training setting for Qwen3-1.7B-Base on IFEval, then fixed SFT training, supplied an initial pool, added IFBench, and returned richer feedback.The single Codex agent submitted 33 rounds and completed 32 evaluations in 11.5 hours without reaching either target.Source: §3 · p. 4, §3.1 · p. 5

3.1. Three failure modes of the single-agent loop

  • All rounds stayed within category-mixture adjustments; round 9 was best on both benchmarks, and the next 23 rounds produced nothing better.The run used 22 different expressions for the repeated move but never mentioned repairing rows, replacing responses, or writing examples.Source: §3.1 · p. 5, Figure 4 · p. 5
  • After peaking at 264k rows in round four, the recipe shrank every subsequent round to 69k rows, or 26% of its peak.The agent continued submitting rounds; under benchmark targets, this contraction did not occur.Source: §3.1 · p. 5, Figure 5 · p. 5
  • The session reached 94.7% of its 258,400-token context window and was compacted ten times, beginning in round two and then every three to four rounds.Training output, run-record reads, data actions, and evaluation output accounted for large payload shares, while job-status polling accounted for only 1.7% of accumulated bytes.Source: §3.1 · pp. 5–6, Figure 6 · p. 6

3.2. From observations to requirements

  • The observed context and proposal limitations motivate a dedicated executor and multiple analysis subagents; the existing pool’s coverage gap motivates a synthesis component.The executor handles submission, conversion, and evaluation separately, while distinct analysis angles provide multiple proposal streams.Source: §3.2 · p. 6

4. AutoSFT

AutoSFT uses a meta agent to coordinate separate execution, analysis, and synthesis subagents. The executor returns artifacts for later inspection, analysts propose changes from complementary evidence sources, and synthesis adds verified examples for undersupplied capabilities.

4. AutoSFT

  • AutoSFT’s meta agent delegates training and evaluation, gathers data-change proposals from analysis agents, and decides whether to use synthesized data in the next recipe.The system shares the base model, initial pool, and training and evaluation protocol with the single-agent comparison.Source: §4 · p. 6

4.1. Executor subagent

  • The executor runs training, checkpoint conversion, and benchmark evaluation in its own session, then returns artifact paths rather than full logs.The meta agent and other subagents can read relevant files as needed.Source: §4.1 · pp. 6–7

4.2. Analysis subagents

  • Four analysis subagents propose executable data changes using complementary evidence: training data, failure cases, learning dynamics, or research history.Their proposals address data quality and coverage, recurring failures, stalled learning, and interventions that have not been adequately tested.Source: §4.2 · p. 7

4.3. Synthesis subagent

  • The synthesis subagent creates examples for benchmark capabilities absent from or undersupplied in the initial pool, using targets that programmatically verifiers can check.Generated examples pass target verification, independent semantic review, and an n-gram overlap check before entering the pool; the meta agent decides whether to train on them.Source: §4.3 · p. 7

5. Experiment

Component-by-component comparisons test AutoSFT’s executor, analysis, and synthesis components, followed by a code-domain transfer test. Analysis and synthesis improve benchmark results, while the executor reduces context loss.

5. Experiment

  • Each component is compared with its base configuration while other settings are held constant; the code experiment tests whether the structure transfers.Appendix D provides implementation settings and estimates run costs.Source: §5 · p. 7, §5.4 · p. 10

5.1. Executor subagent

  • 10 context compactions fall to five with the executor over comparable elapsed time.Shell commands decline from 58% to 29% of recorded events.Source: Figure 7 · p. 8, §5.1 · p. 8

5.2. Analysis subagent

  • +6.50 IFEval points at equal rounds; analysis shifts actions toward repair and resupervision, while the base remains within mixture adjustments.Repair and resupervision account for 55% of analysis-run actions.Source: §5.2 · p. 8, Figure 8 · p. 9

5.3. Synthesis subagent

  • 81.58 vs 78.15 IFEval and 51.95 vs 49.02 IFBench: synthesis reaches higher best scores than the base configuration.Generated examples are admitted to the pool only after passing a capability-specific verifier.Source: §5.3 · p. 9, Figure 9 · p. 10

5.4. Generalization to code domain

  • 65.8% by round 6 with analysis and synthesis, versus 64.8% by round 13 with analysis; the single agent shows no sustained improvement.The code-domain comparison uses MBPP+, which has no capability breakdown.Source: Figure 9 · p. 10, §5.4 · p. 10

6. Related Work

Prior work covers data curation, automated data optimization, and benchmarks for AI research. AutoSFT is positioned as unifying selection and synthesis within an attributable, repeatedly evaluated loop.

Data selection and synthesis for post-training

  • Curation methods select, reweight, order, or synthesize data, but produce fixed datasets without closing the loop on downstream training performance.The passage also describes curation of preference data.Source: §6 · p. 10

Automated data optimization

  • Automated data-research systems address selection or synthesis, while AutoSFT unifies both and retrains from base each round for attributable actions.The passage distinguishes recipe search, fixed-pool operator search, proxy-reward policies, and synthesis systems.Source: §6 · p. 11

Benchmarks for AI research

  • Related benchmarks cover research tasks, infrastructure, self-improvement, and data-centric recipe search; AutoSFT adds repeated evaluation and a synthesis verification gate.The passage also distinguishes its from-base retraining protocol.Source: §6 · p. 11

7. Conclusion

The conclusion presents AutoSFT as evidence that structure for covering diverse data-engineering changes may matter more than strengthening a single action.

7. Conclusion

  • The same multi-agent structure transfers to coding without a ready-made taxonomy, building its structure from the benchmark and pool.The conclusion frames the bottleneck as potentially lying in coverage of data-engineer changes rather than in any single action.Source: §7 · p. 11

AI use statement

The authors report using generative AI for prose editing, figure refinement, and code implementation assistance, but not for research ideas, methodology, or reported results.

AI use statement

  • Reported numbers come from fixed training and evaluation scripts, and the authors verified claims against the per-round run ledger.The paper identifies the agent-produced recipes, analyses, and prompts as the object of measurement.Source: AI use statement · p. 11

Ethics statement

The paper notes that benchmark optimization could inflate leaderboard numbers and describes checks intended to make target overfitting visible.

Ethics statement

  • The instruction-following and coding pools use openly released datasets and public instruction–code corpora.The paper cites Sections 3 and 5.4 for these sources.Source: Ethics statement · p. 12
  • Because AutoSFT optimizes benchmark scores, it could in principle be used to inflate leaderboard numbers.The paper reports offtarget and held-out benchmarks so a gain from target overfitting is visible rather than hidden.Source: Ethics statement · p. 12
  • Generated data is checked for n-gram overlap with evaluation sets, and the paper reports run compute and API costs.The overlap check is described in Appendix B.4 and costs in Appendix D.Source: Ethics statement · p. 12

Reproducibility statement

The authors plan to release AutoSFT and describe the materials and run records that support reproducibility.

Reproducibility statement

  • The authors state that they will release the AutoSFT implementation as anonymized supplementary material.The planned release includes the training and evaluation script and agent scaffolding.Source: Reproducibility statement · p. 12
  • The paper identifies the training script, token budget, and from-base retraining protocol as described in Sections 2 and 3.Appendix B gives the agent roles, prompts, and skill files.Source: Reproducibility statement · p. 12
  • A run ledger records the initial data pool, per-round recipes, and raw evaluation outputs behind every reported number.Source: Reproducibility statement · p. 12

A.1. Data action distribution in full trajectory

The appendix traces the single-agent run and documents the methods used to organize capabilities, preserve round histories, and support synthesis. It also describes how skills and verification procedures structure the work.

A.1. Data action distribution in full trajectory

  • Round 9 is best on both benchmarks, and no later round improves either score; all rounds also share micro batch size 1 and a minimum learning rate of 10−7.Source: §A.1 · p. 17

A.2. Condensed trace

  • Rounds 19 and 24 show mismatches between the agent’s stated action and its plan, so actions are labeled from consecutive recipe differences.Round 15 accidentally reproduces round 9’s configuration exactly, making the pair useful for measuring the effect of a repeated round.Source: §A.2 · p. 17

A.3. What fills the context

  • The deciding agent reaches 94.74% of its 258,400-token window before its tenth compaction, while the trace breakdown is coarse and does not establish that context pressure stalled scores.Evaluation output read through a generic file read is counted as a file read.Source: §A.3 · pp. 17–18

B.1. Capability taxonomy

  • The capability taxonomy starts from benchmark-provided groupings or observed failures, and its leaves are programmatically decidable requirements used to label pool rows.The taxonomy is rebuilt for each domain and gives the synthesis subagent stable capability names to target.Source: §B.1 · pp. 18–19

B.2. Memory

  • Round history records hypotheses before training, their verdicts and lessons, and proposals not taken; the two-part verdict separates score direction from whether the intended effect appeared.Registered predictions and decided rounds are not edited later, preserving what the run believed at the time.Source: §B.2 · pp. 19–20

B.3. Skill library

  • The four analysis skills return the same kind of proposal but start from different evidence, so their diversity comes from perspective rather than different action menus.Skills are written procedures owned by individual roles and do not call one another.Source: §B.3 · p. 20

B.4. Synthesis constraints and verification

  • Synthesis attributes evaluation requirements to capabilities, builds acceptance criteria and a verifier, then generates and revises candidates until they pass.Generated rows must avoid n-gram overlap with evaluation sets and come from model API calls rather than scripts.Source: §B.4 · pp. 20–21

C.1. Analysis subagents

Analysis changes the actions and examples used during instruction-following research, while synthesis targets capabilities with scarce data. The appendix also reports resource use and limits the scope of the findings.

C.1. Analysis subagents

  • 34.76 points versus 1.22: the analysis run’s varied cue wording and order raises strict repetition accuracy more than the base run’s repeated template.The action-family comparison also shows the base run using only mixture, while other families are used only with analysis.Source: §C.1 · p. 21

C.2. Synthesis subagent

  • Synthesis raises repeat_prompt strict pass rate from near 40% to 75.6% at round 9, but odd_even_syllables stays at zero despite batches in three rounds.Across targeted capabilities, the strongest supported advantages are for repetition, sentence and word counts, and comma avoidance; lower rates mostly occur for capabilities backed by few questions. Targets are selected by data supply, with R9 targets matching only 4 to 161 staged rows each.Source: §C.2 · pp. 21–22

D. Experimental Details and Resource Usage

  • A 20-round synthesis run is estimated at roughly 130 GPU-hours and USD 1,900 in API costs.The estimate assumes about 0.8 GPU-hours for training and 0.15 GPU-hours for evaluation per round, and about USD 95 API cost per round.Source: D. Experimental Details and Resource… · p. 23

E. Limitations

  • All results use one base model at one scale and one search per configuration, so effects may differ at other scales and independent-search variation is unbounded.The generalization study covers one new domain with short runs, demonstrating transfer but not how broadly it applies.Source: E. Limitations · p. 24

Related papers

Loading 2610.22194v1…