PaperlayerView exact revision on arXiv

Source-linked AI summary

From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning

Haodong Wu, Jiahao Zhang, Lijie Hu, Yongqi Zhang

arXiv:2605.12944v1cs.LGcs.CL

TL;DR

SFT data selection is often framed as ranking individual examples, although curation also involves ordered combinations of operators. AutoSelection searches recipes over a fixed pool under a limited full-evaluation budget. It achieves the strongest in-distribution reasoning average across three base models, outperforming the reported baselines.

  • Problem

    SFT data selection is commonly formulated as instance ranking, while practical curation can combine multiple operators whose interactions make recipe search a structured problem under expensive validation.

    Source: §1 · pp. 1–2

  • Method

    AutoSelection searches ordered recipes over a fixed raw pool, using cached signals, warmup probes, local edits, candidate ranking, and reseeding after stagnation.

    Source: §1 · p. 2, §3.3 · pp. 5–6

  • Results

    Reasoning-average gains over the strongest non-AutoSelection baseline were 0.90, 1.69, and 2.89 points on Llama3.2-1B, Qwen2.5-1.5B, and Qwen2.5-3B, respectively.

    Source: §4.2 · pp. 6–7

  • Takeaways & Limitations

    The best single-operator baseline varies across tasks and model scales, supporting recipe-level composition rather than reliance on a universally best selector.

    Source: §4.2 · p. 7

    Evaluation is limited to a moderate-size instruction pool, several model scales, and finite reasoning benchmarks; full evaluations make scaling computationally expensive.

    Source: Limitations and Future Work · p. 10

Abstract

from arXiv · show

Supervised fine-tuning (SFT) data selection is commonly formulated as instance ranking: score each example and retain a top-$k$ subset. However, effective SFT training subsets are often produced through ordered curation recipes, where filtering, mixing, and deduplication operators jointly shape the final data distribution. We formulate this problem as fixed-pool data recipe search: given a raw instruction pool and a library of grounded operators, the goal is to discover an executable recipe that constructs a high-quality selected subset under a limited budget of full SFT evaluations, without generating, rewriting, or augmenting training samples. We introduce AutoSelection, a two-layer solver that decouples fixed-pool materialization based on cached task-, data-, and model-side signals from expensive full evaluation, using warmup probes, realized subset states, local recipe edits, Gaussian-process-assisted ranking, and stagnation-triggered reseeding. Experiments on a 90K instruction pool show that AutoSelection achieves the strongest in-distribution reasoning average across three base models, outperforming full-data training, random recipe search, random top-$k$, and single-operator selectors. Additional Out-of-distribution graph-reasoning results, search-stability analyses, structural ablations, and 1.5B-to-7B transfer checks further show that recipe structure matters beyond individual selection operators. Code is available at https://github.com/w253/AutoSelection.

1 Introduction

SFT data selection is often framed as ranking individual examples, while practical curation combines operators in multi-stage workflows. AutoSelection searches ordered recipes over a fixed pool, aiming to find strong subsets with a limited number of full evaluations.

1 Introduction

  • SFT curation often uses filtering, source mixing, deduplication, and light cleaning as a multi-stage workflow, unlike instance-level score-and-select methods.Instance-level approaches score samples using quality heuristics, influence estimates, learned representations, or iterative utility signals.Source: §1 · p. 1
  • Fixed-pool recipe search makes operator choice, ordering, parameters, and intermediate subset states part of the optimization, rather than retaining a top-k subset from one scoring view.Different ordered recipes transform the same pool into different selected subsets.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • Recipe search requires budget allocation over interacting operator choices because each reliable observation entails recipe execution, fine-tuning, and benchmark evaluation.An operator's value cannot be understood independently of the intermediate subsets produced by the rest of the recipe.Source: §1 · p. 2
  • AutoSelection caches grounded task-, data-, and model-side signals, probes retention regimes, edits seed recipes, ranks candidates, and reseeds after stagnation.Language models, when used, serve as search-side assistants rather than rewriting, synthesizing, or augmenting training samples.Source: §1 · p. 2
  • On a 90K instruction pool across three base models, AutoSelection improves over full-data training, random recipe search, random top-k, and single-operator selectors.The introduction also reports additional OOD, search-stability, structural-ablation, and 1.5B-to-7B transfer analyses.Source: §1 · p. 2

2 Related Works

Prior work commonly selects instruction-tuning data through instance or subset rankings, while newer work also searches workflows and pipelines. AutoSelection instead focuses on controlled, fixed-pool recipe search.

2.1 Instance-level data selection

  • Instance-level methods use varied signals, but typically optimize an instance or subset ranking rather than a jointly optimized multi-step recipe.Examples include task relevance, influence, training-time utility, instruction difficulty, redundancy, and instruction-structure diversity.Source: §2.1 · p. 3

2.2 Recipe and pipeline optimization

  • Workflow and pipeline methods include configurable operator pipelines, broader post-training pipeline search, and recipe automation using open-ended generation or LLM-driven processing.AutoSelection fixes the raw pool, searches sample-level operator compositions, and uses full evaluation as its central search signal.Source: §2.2 · p. 3
  • AutoSelection borrows the budgeted-search perspective of hyperparameter optimization and AutoML, but its configurations are ordered curation programs that change the SFT training subset.Each observation includes recipe execution, fine-tuning, and downstream evaluation, making subset state, operator ordering, and fixed-pool attribution central.Source: §2.2 · p. 3

3 Method

AutoSelection defines recipe search over a fixed canonicalized pool, then uses cached signals and realized subset states to search under a limited full-evaluation budget. Warmup, local edits, ranking, and reseeding guide which recipes receive full evaluation.

3.1 Fixed-pool data recipe search

  • A recipe is an ordered sequence of grounded operators and parameters, executed on the fixed pool to produce a selected subset and downstream utility; a full evaluation consumes the budget.Single-operator selectors are length-one recipes, so the recipe space contains the included selector baselines, without implying finite-budget search finds the global maximum.Source: §3.1 · p. 4

3.2 Making the fixed pool searchable: cached signals and realized subset states

  • AutoSelection canonicalizes the pool, reuses cached task-, data-, and model-side signals, and represents each materialized subset with a state vector for comparison before full evaluation.State summaries include benchmark-conditioned relevance, retained-example and token ratios, and model-side statistics such as IFD, varentropy, and sparse-activation drift.Source: §3.2 · pp. 4–5

3.3 Navigating the recipe space: warmup, local refinement, and reseeding

  • Warmup evaluates probes from low-, medium-, and high-retention regimes to inform the initial seed; later, a Summarizer guides local recipe edits and a Ranker selects one candidate for full evaluation.A GP surrogate supplies a cheap score prior, and stagnation triggers reseeding to explore after consecutive evaluations without incumbent improvement.Source: §3.3 · p. 5
  • The incumbent best result and the seed anchor are tracked separately, keeping the local proposal neighborhood fixed within a phase and refreshing it after stagnation.This preserves local focus for most evaluations while providing exploration after stagnation.Source: §3.3 · p. 6

4 Experiments

The experiments test fixed-pool recipe search on a shared instruction pool under limited full-evaluation budgets. AutoSelection leads in-distribution reasoning averages across three base models, while further analyses examine search stability, component roles, recipe structure, and cross-scale transfer.

4.1 Experimental setup

  • All methods used the same 90K pool across three base models and four benchmarks, with 15 full evaluations per search and no synthetic generation, rewriting, or augmentation.The pool combines 30K samples each from OpenHermes-2.5, the LESS instruction-tuning data pool, and Alpaca-52K.Source: §4.1 · p. 6

4.2 Main results

  • 0.90, 1.69, and 2.89-point reasoning-average gains over each block’s strongest non-AutoSelection baseline made AutoSelection best across Llama3.2-1B, Qwen2.5-1.5B, and Qwen2.5-3B.The reported comparisons are against each model block’s strongest non-AutoSelection baseline.Source: §4.2 · pp. 6–7, Table 1 · p. 7

4.3 Search stability under randomness

  • 42.23, 42.28, and 41.69 were the three 1.5B selected-subset scores (mean 42.07, range 0.59), and even the lowest exceeded the strongest non-AutoSelection baseline at 40.54.The three-run check used the same 15-full-evaluation budget; the authors say the results suggest the improvement was not solely a lucky run.Source: §4.3 · p. 7

4.4 Search-side ablations

  • The full setting reached the highest post-warmup score, while ablations indicate different components trade trajectory smoothness against peak discovery.Removing warmup led to early low-scoring evaluations; removing the Summarizer yielded a weaker best recipe, and removing the Reseeder flattened the best-so-far curve below the full method. State vectors help screen candidates whose recipe form looks promising but whose realized subset is weaker; full SFT evaluation remains necessary to confirm the best recipe.Source: §4.4 · pp. 8–9

4.5 Matched-seed Structural ablations

  • All tested structural variants had positive mean paired drops against the reference, whose mean score was highest under matched random-k seeds.The comparison held operator parameters and the evaluation protocol fixed.Source: Table 3 · p. 9, §4.5 · p. 9

4.6 Transferability to larger models

  • ρ = 0.82 indicates positive but imperfect rank similarity between the selected subsets’ 1.5B and 7B results.The authors treat this single Qwen2.5-1.5B-to-Qwen2.5-7B transfer check as an approximate data-quality signal: subsets from stronger 1.5B runs tended to remain competitive, while weaker-run subsets did not reliably become strong solely at larger scale.Source: §4.6 · p. 9

5 Conclusion

AutoSelection treats SFT data curation as fixed-pool search over ordered combinations of operators rather than a single instance-ranking problem. Across multiple model scales, recipe-level search improves in-distribution reasoning averages over the stated baselines, and analyses indicate that operator ordering and composition can affect performance.

5 Conclusion

  • AutoSelection optimizes ordered combinations of filtering, mixing, deduplication, and selection operators under a limited full-evaluation budget.It caches task-, data-, and model-side signals, uses warmup probes and search-history summaries, ranks candidate recipes, and reseeds after stagnation.Source: §5 · p. 9
  • Across multiple model scales, recipe-level search improves in-distribution reasoning averages over full-data training, single-operator selection, Random top-k, and Random recipe search.It remains competitive on held-out graph benchmarks.Source: §5 · p. 9
  • The analyses indicate that operator ordering and composition can affect downstream SFT performance, supporting data recipes as optimization objects.Source: §5 · p. 9

Limitations and Future Work

The study evaluates fixed-pool recipe search on a moderate-size instruction pool, several model scales, and a finite set of reasoning benchmarks. Full evaluations are computationally expensive, motivating broader evaluation and cheaper search strategies.

Limitations and Future Work

  • The evaluation covers a moderate-size instruction pool, several model scales, and a finite set of reasoning benchmarks.The authors identify larger pools, more heterogeneous sources, and domain-specific SFT tasks as directions for testing generality.Source: Limitations and Future Work · p. 10
  • Each full evaluation requires recipe execution, model fine-tuning, and benchmark evaluation, making larger-model or longer-budget search computationally expensive.The authors identify cheaper proxies, multi-fidelity search, and transferable recipe priors as possible directions for improving scalability.Source: Limitations and Future Work · p. 10
  • The work studies fixed-pool recipe search under a controlled evaluation protocol.Source: Limitations and Future Work · p. 10

A Experimental Protocol and Operator Details

The experimental operator library combines heterogeneous selection signals and recipe-level composition rules rather than relying on a single definition of data quality. Its operators support individual-selector baselines as well as search over multi-step recipes.

A.1 Operator Library

  • The operator library spans task relevance, model-internal difficulty and uncertainty, data-side diversity, redundancy reduction, and recipe-level combinators.The table summarizes operators that provide distinct signals or composition rules for constructing candidate subsets.Source: §A.1 · p. 14, Table 4 · p. 14
  • MONA targets benchmark-conditioned task relevance, while IFD and varentropy capture model-internal difficulty and predictive uncertainty.N-gram entropy and action-object branching represent lexical and instruction-structure diversity, while SemDedup reduces near-duplicates in embedding space.Source: §A.1 · p. 14
  • Random top-k samples uniformly from the current candidate pool, and Mix unions the current subset with a selected historical subset while retaining each sample only once.Random top-k is motivated as an escape from overlapping deterministic subsets; Mix reuses effective data regions while exploring new compositions.Source: §A.1 · pp. 14–15
  • Single-operator baselines test individual signals, while AutoSelection combines the same operator library to search recipes under a fixed validation budget.The controls also include random top-k and random recipe search, which respectively test random subset selection and unguided composition sampling.Source: §A.1 · p. 15

A.2 Implementation Details

The implementation fixes and canonicalizes the raw pool, caches operator-side signals, and applies a shared training and evaluation protocol across candidate recipes. It then uses a recipe representation and staged search procedure to allocate a limited number of full evaluations.

A.2 Implementation Details

  • The protocol canonicalizes one fixed raw pool, caches reusable signals, and evaluates recipes with 15 full evaluations, including three warmup probes across low-, medium-, and high-retention regimes.Candidate recipes do not add synthetic samples, rewrite responses, or query external generators; training hyperparameters are held fixed across recipes.Source: §A.2 · p. 15

A.3 Baseline Parameter Setting

  • Single-operator baseline settings are chosen to keep retained-data scale near the best AutoSelection subset, with size-controlled top-k baselines set to 0.5 and MONA top-k to 0.05.The roughly 45K-example AutoSelection subset motivates the 0.5 operating point; these settings are reused across model sizes.Source: §A.3 · p. 17, Table 6 · p. 17

B.1 Recipe Vector

  • The surrogate recipe feature encodes operator presence and normalized parameters, while the ordered recipe remains available to the controller and Ranker for order-sensitive reasoning.The encoding is compact and fixed-dimensional; order-sensitive reasoning is not claimed to be fully represented by that feature alone.Source: §B.1 · p. 17

B.2 Search Pseudocode

  • After warmup, the search proposes and executes candidate recipes, extracts realized subset states, ranks candidates using GP estimates and other signals, then fully evaluates the selected candidate and reseeds after stagnation.The algorithm returns the best discovered subset, its generating recipe, and its score.Source: §B.2 · p. 18

B.3 Formal Notes and Search-Side Complexity

  • Search-side cost is bounded by candidate recipe materialization and operator applications on intermediate subsets, while full evaluation still requires SFT and benchmark evaluation.In representative 15-evaluation runs, LLM time is 0.37–0.61 hours, compared with 8.25–12.15 hours for recipe execution plus SFT-and-evaluation; Table 7 reports these components jointly.Source: §B.3 · pp. 18–19, Table 7 · p. 19

C.1 Random Select Curve

The extended Random Select run shows that a larger random-search budget can improve its best-so-far score without stabilizing the raw curve. Diagnostics describe realized subset states as screening context and the GP fit as a coarse prioritization signal, not a substitute for full evaluation.

C.1 Random Select Curve

  • 41.17 by step 11 is the Random Select run’s best-so-far score, but its raw scores remain volatile, with drops after strong evaluations and a later collapse after restart.Figure 5 marks warmup probes and restart seeds; extending the budget can raise the best-so-far curve without providing a stable search policy.Source: Figure 5 · p. 20, §C.1 · p. 20

C.2 State Vector Definition and Diagnostics

  • The state vector groups task-, data-, and model-side fields to summarize realized subset scale, distribution shift, difficulty, uncertainty, and task relevance before full evaluation.SNAR records how often each SAE feature appears in the subset, ignoring activation magnitudes.Source: §C.2 · pp. 20–21
  • Across 107 state-score points from six runs, token ratio, retain ratio, GSM8K relevance, and score mean have the strongest all-point Spearman correlations; state vectors are screening context, not standalone replacements for full evaluation.The leave-one-out ridge probe gives Pearson 0.798 and Spearman 0.406.Source: Table 9 · p. 21, §C.2 · p. 21

C.3 GP Fit

  • The rolling GP fit provides a coarse historical signal for prioritizing candidates, but its mixed predictions and few noisy observations mean full evaluation remains necessary.Figure 6 compares saved surrogate predictions with downstream full evaluation scores where both are available.Source: §C.3 · p. 21, Figure 6 · p. 22

C.4 Operator-Composition Case Studies

Run-level diagnostics show that retained-example scale alone does not account for validation scores, while stronger records exhibit recurring adjacent-operator motifs. The stability and ranking analyses provide bounded diagnostics of search behavior.

C.4 Operator-Composition Case Studies

  • The 40K–60K output range spans 12.98 points across 51 records, and the 60K–80K range spans 12.27 points across 67, so retained scale alone does not explain scores.The authors treat retention scale as a useful state variable, while noting that recipes with comparable realized sizes can still differ substantially.Source: §C.4 · p. 22, Table 10 · p. 23
  • MONA→SemDedup and N-gram→MONA are enriched among stronger records, consistent with high scores not being explained by one isolated operator.The motif counts are retrospective diagnostics, not evidence that any pair is universally optimal.Source: §C.4 · pp. 22–23, Table 11 · p. 23

C.5 Search Stability

  • The three-run stability summary uses a 15-full-evaluation budget and reports benchmark scores for each run’s Best@15 recipe.The main text reports the median run in its main result table and gives the three selected scores, mean, and range in the stability discussion.Source: §C.5 · p. 23, Table 13 · p. 24

C.6 Recipe Ranking Case Study

  • Across five audited decision points, the Ranker placed the true best candidate first in 2/5 cases and within the top two in all five.This is a counterfactual diagnostic, not a large-scale ranking benchmark; the authors interpret it as support for allocating full evaluations toward promising candidates.Source: §C.6 · p. 24, Table 14 · p. 24, Table 15 · p. 25

D.1 Comparison Boundary for LLM-Driven Data-Processing Agents

The LLM-AutoDP pilot was stopped before a repeated baseline because of API overhead and domain-specific prompts. Separately, recipes found at 1.5B tended to remain competitive at 7B, but their ordering was not fully preserved.

D.1 Comparison Boundary for LLM-Driven Data-Processing Agents

  • The authors excluded LLM-AutoDP from the main quantitative table because it did not match the fixed-pool protocol, and stopped its pilot before a complete repeated baseline.Large-pool API calls added prohibitive latency and overhead, while several modules used medical-domain prompts that could select for the wrong relevance on the mixed instruction pool.Source: §D.1 · pp. 25–26, Table 16 · p. 25
  • AutoSelection keeps language models on the search side, where they summarize histories and rank grounded edits without rewriting, augmenting, or individually judging training samples.The stated design aims to isolate measured gains to choices among grounded operators on a fixed raw pool rather than prompt engineering or newly introduced samples.Source: §D.1 · p. 26

D.2 Cross-Scale Transfer Checks

  • Recipes strong at 1.5B tended to remain competitive at 7B, but their ordering was not perfectly preserved, so larger-model validation remains necessary.The 7B transfer check is supporting analysis, not the main evidence for AutoSelection.Source: §D.2 · p. 26, Table 17 · p. 26

D.3 Metric-Distribution Comparison

This section compares metric distributions, noting that similar marginal selection-signal distributions can accompany different downstream outcomes, and documents benchmark evaluation-prompt scaffolds. MMLU prompts request reasoning and a final answer letter, while GraphWiz and NLGraph prompts request yes/no answers.

D.3 Metric-Distribution Comparison

  • Recipes with similar marginal MONA, IFD, and entropy distributions obtain different aggregate scores, so individual metric distributions do not explain downstream performance.Figure 8 compares nine evaluated recipes; Table 18 reports their aggregate scores, but the supplied passages do not provide specific score values.Source: §D.3 · p. 27, Table 18 · p. 27, Figure 8 · p. 28
  • Because similar one-dimensional distributions can lead to different outcomes, AutoSelection uses metric views as contextual signals and relies on full evaluation to measure recipe effects.The search jointly considers operators, thresholds, and ordering, while full evaluation observes outcomes after fine-tuning under a limited evaluation budget.Source: §D.3 · p. 27

E.1 Search-agent prompts

  • Four search-agent prompts assign the Summarizer, Proposer, Ranker, and Reseeder roles for guidance, candidate generation, evaluation choice, and stagnation-triggered restarts.The Ranker considers per-task MONA scores, exploration versus exploitation, data-quantity risks, operator interactions, state vectors, and GP uncertainty.Source: §E.1 · pp. 29–30

E.2 Benchmark evaluation prompts

  • The benchmark evaluation prompt suite covers GPQA, GSM8K, BBH, and MMLU, with supplied scaffolds specifying answer formats and few-shot examples for GPQA, GSM8K, and BBH.GPQA requests a multiple-choice letter, GSM8K asks for step-by-step work ending in a #### answer, and BBH requests reasoning followed by an Answer line.Source: §E.2 · pp. 31–32
  • MMLU prompts instruct the assistant to reason through a multiple-choice question and put the answer letter after “Answer:” on the last line.The scaffold also includes example questions and a final-turn template with four choices.Source: §E.2 · p. 33
  • The MMLU scaffold includes few-shot turns with example questions about France’s capital, the Red Planet, and the cell’s powerhouse.Source: §E.2 · p. 33
  • GraphWiz and NLGraph prompts ask a graph-reasoning assistant to output only “Answer: Yes” or “Answer: No” on the last line.Source: §E.2 · p. 33
Loading 2605.12944v1…