PaperlayerView exact revision on arXiv

Source-linked AI summary

Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model-Skill Co-Evolution

Yuhan Liu, Xiyao Ma, Zhongkai Sun, Xu Han, Chengyuan Ma, Benjamin Z. Yao, Chenlei Guo

arXiv:2610.12367v1cs.CL

TL;DR

Individual skill utility is understudied, leaving open which skills are worth distilling and whether internalizing them supports stable co-evolution. SGUID selects skills with consistent training signals; compact selected banks match or exceed full-bank distillation across most models, with further gains after another round.

  • Problem

    Individual skill utility is understudied, and it remains open which skills are worth distilling and whether their internalization supports stable model–skill co-evolution.

    Source: §1 · p. 2

  • Method

    SGUID selects skills that consistently yield verifier-aligned training signals, distills the compact bank into the model, and curates a new candidate bank from updated-model rollouts.

    Source: §3 · p. 5

  • Results

    Six selected skills match or exceed full-bank distillation in mean avg@12 on three of four models; a second round with 3 newly selected skills does so on all four and raises Qwen3-8B from 64.3% to 66.3%.

    Source: §7 · p. 9

  • Takeaways & Limitations

    Skill selection stabilizes model–skill co-evolution by filtering stale, redundant, or conflicting skills before they are reused in later distillation rounds.

    Source: §5 · p. 8

    The study tests only single-turn math reasoning tasks and does not examine skills as inference-time patches.

    Source: §8 · p. 9

Abstract

from arXiv · show

Skills, reusable procedural guidance added at inference, can substantially improve LLM downstream performance (Li et al., 2026). Prior work retrieves skills from a bank by semantic relevance, then uses them as inference-time patches or for model distillation. The individual utility of each skill, however, is largely neglected. We first show that, in on-policy distillation where skill-conditioned policies serve as teachers, fewer than 25% of retrieved skills provide useful distillation signals. We then propose SGUID, a method for selecting a compact subset of skills for distillation. SGUID retains a skill only if it consistently yields effective learning signals during training. The selected skills are then distilled to produce a better model. Our results show that not all skills are worth distilling. Across four models from the Olmo and Qwen families, distilling 6 selected skills matches or exceeds full-bank distillation in mean avg@12 on three of the four models, and on all four after a second round that distills 3 newly selected skills, while the full banks are up to 11x larger. Importantly, SGUID supports stable model-skill co-evolution: after a distillation round, a new candidate bank is curated from the updated model's rollouts, and SGUID selects which skills to internalize next. In the second round, this loop selects 3 new skills and improves Qwen3-8B from 64.3% to 66.3%. The selection step is essential for stability: on Qwen3-4B, naively updating the model with unfiltered skills degrades performance, including a 0.3 percentage point drop on HMMT25, whereas SGUID improves HMMT25 by 0.5 points after the first round and 1.1 points after the second. These results identify skill selection as the key mechanism for stable model-skill co-evolution.

1 INTRODUCTION

The section frames individual skill utility as understudied and asks which skills are worth distilling. It introduces SGUID and reports that compact selected banks can match or exceed full-bank distillation.

1 INTRODUCTION

  • Individual skill utility is understudied, and it remains unclear whether gains come from all skills or only a fraction.Prior SWE work found meaningful improvements from only 7 of 49 coding skills, with others having little effect or degrading performance.Source: §1 · p. 2
  • Prior work selects useful skills for skill-bank maintenance and uses on-policy self-distillation to internalize skill-induced behavior, but which skills are worth distilling remains open.Whether internalization supports stable model–skill co-evolution also remains unresolved in the cited prior work.Source: §1 · p. 2
  • SGUID selects skills that provide consistent learning signals during distillation, then uses the selected compact bank to update the student policy.The updated model's rollouts can be used to curate a new candidate bank for the next co-evolution round.Source: §1 · p. 2
  • A second SGUID round improves Qwen3-8B performance from 64.3% to 66.3%.The second round distills 3 newly selected skills.Source: §1 · p. 2
  • With 6 skills, SGUID matches or exceeds full-bank distillation in mean avg@12 on three of four models, while full banks are up to 11× larger.A second round with 3 newly selected skills achieves this comparison on all four models.Source: §1 · p. 2

2 BACKGROUND

The background defines skill banks curated from verified policy rollouts, describes on-policy self-distillation, and explains how skill-conditioned teachers provide gated signals for selecting skills to internalize.

2.1 PROBLEM SETTING

  • A skill bank can be curated from current-policy rollouts verified against a reference solution, and its skills are compact natural-language procedures, heuristics, or failure-mode descriptions.Retrieved skills can condition training-time teachers or act as inference-time patches.Source: §2.1 · p. 3

2.2 ON-POLICY SELF-DISTILLATION

  • On-policy self-distillation compares a student rollout with teacher predictions, where teacher and student share parameters but differ in conditioning information.The objective minimizes average next-token divergence; the teacher may be frozen or synchronized during training.Source: §2.2 · p. 3

2.3 ON-POLICY SELF-DISTILLATION WITH SKILLS

  • A skill-conditioned teacher's positive log-probability gap means it favors the student's sampled token, and SGSD uses a robust support estimate that masks irrelevant tokens and clips extreme gaps.The reference solution verifies the rollout, while a confidence gate filters weak support signals before skill-guided distillation.Source: §2.3 · pp. 3–4
  • SGUID retains skills that consistently provide verifier-aligned teacher signals across training, then distills the compact subset to obtain an updated policy.The updated policy becomes the starting point for the next model–skill co-evolution round.Source: Figure 1 · p. 4

3 METHOD

SGUID uses training dynamics to select compact skill banks whose signals persist during on-policy self-distillation, then distills them into an updated policy. It repeats selection from the updated policy’s rollouts for co-evolution.

3 METHOD

  • SGUID selects skills that consistently provide verifier-aligned teacher signals, then distills the compact bank to update the policy.The process repeats with the updated policy and a newly curated candidate bank.Source: §3 · pp. 4–5
  • Among 6,400 retrievals per model, 21.2% for Qwen3-1.7B, 19.7% for Qwen3-4B, and 6.4% for Qwen3-8B were used for distillation.The remaining retrieved skills were zeroed out by the gate, motivating selection for persistently active signals.Source: §3 · p. 4, Figure 2 · p. 4
  • SGUID measures each skill’s conditional contribution rate separately in early and late phases, counting positive verifier-aligned signals among retrievals.The rate is the number of positive signals divided by the number of retrievals in that phase.Source: §3 · p. 5
  • The filter requires early contribution, at least h positive late-phase signals, and limited decline; survivors rank by late counts, then late contribution rate.The late-count threshold avoids sparse-event decisions, while the rate constraint removes skills whose useful signal fades during training.Source: §3 · p. 5
  • After selection, training restarts from the round’s original policy and uses only the compact bank; the next round selects skills anew from updated-policy rollouts.The updated policy’s rollouts are sampled and verified before curating the next candidate bank.Source: §3 · p. 5

4 EXPERIMENTS

Experiments compare SGUID with full-bank distillation and answer-conditioned baselines on four models and three math benchmarks. The results show that compact selected banks can match or outperform larger banks, with a second co-evolution round adding gains.

4.1 EXPERIMENTAL SETUP

  • The experiments train Olmo-3-7B, Qwen3-1.7B, Qwen3-4B, and Qwen3-8B on DAPO-Math-17K and evaluate AIME24, AIME25, and HMMT25.Source: §4.1 · p. 5
  • Baselines include GRPO, OPSD, and SGSD; models train for 200 steps, and the best avg@12 checkpoint is reported.Training disables thinking mode for the student and enables it for the teacher; bank sizes are fixed at 6 skills in round 1 and 3 in round 2.Source: §4.1 · pp. 5–6

4.2 MAIN FINDINGS

  • With 6 selected skills, SGUID outperformed full-bank SGSD on all three benchmarks for Olmo-3-7B and Qwen3-4B, raising mean scores from 40.0% to 40.9% and 64.2% to 65.1%, respectively.In round 2, 3 selected skills achieved the best mean score on three models and matched SGSD on Qwen3-1.7B; full banks contain 30–71 skills.Source: Table 1 · p. 6, §4.2 · pp. 6–7
  • On Qwen3-4B, Figure 3 contrasts shrinking mean gains without selection against steady improvement with SGUID across two co-evolution rounds.Source: Figure 3 · p. 7
  • Selecting skills from only the first 25 training steps matched the full-run bank on Qwen3-1.7B but was less reliable on Qwen3-4B.Source: Table 2 · p. 7
  • On Qwen3-8B, second-round SGUID raised the mean from 64.3% to 66.3%, while SGUID outperformed OPSD at every model scale.The second round selects 3 new skills from a candidate bank curated from updated-model rollouts.Source: §4.2 · p. 7

5 ABLATION STUDIES AND DISCUSSION

The ablations show that selecting skills helps preserve gains across distillation rounds, that ranking matters beyond bank size, and that very early selection is not uniformly reliable.

5 ABLATION STUDIES AND DISCUSSION

  • On Qwen3-4B, selected-bank gains hold at +3.0 to +3.1 points across rounds, while full-bank gains fall from +2.1 to +1.6 points and HMMT25 drops 0.3 points.The full-bank mean decline is mainly associated with AIME25 gains falling from 4.5 to 3.1 points.Source: §5 · pp. 7–8
  • Table 3 reports Qwen3-4B avg@12 across retained-skill counts, using K = 6 by default; its last row drops the persistence constraint while keeping the same budget.The table caption specifies the comparison but does not provide cell values in the supplied passage.Source: Table 3 · p. 8
  • With K fixed at 6 on Qwen3-1.7B, the top-ranked bank performs best on the mean, AIME25, and HMMT25, coming within 0.7 points of full-bank SGSD.Random and bottom-ranked banks are harder to distinguish, and neither matches the top-ranked bank on the mean.Source: §5 · p. 8
  • 42.5% versus 42.3% on Qwen3-1.7B, but 63.4% versus 65.1% on Qwen3-4B: selecting from 25 steps is model-dependent relative to the full-run bank.The 25-step bank remains above the Qwen3-4B base model on average, but trails the full-run bank by 1.7 points.Source: §5 · p. 8

6 RELATED WORK

Prior work improves external skill banks or distills skill-conditioned behavior into model weights. SGUID combines these directions by selecting skills during training and using the updated model to seed the next bank.

6 RELATED WORK

  • External-bank methods improve retrieval, trajectory consolidation, validation-guided editing, knowledge accumulation, or joint skill memory, usually optimizing artifacts for inference with a fixed model.SGUID instead internalizes selected skills through distillation, then uses the updated model to curate the next candidate bank.Source: §6 · p. 9
  • On-policy self-distillation uses student-generated trajectories and teacher signals from privileged context; SGUID builds on SGSD but shifts from event-level validation to bank-level curation.It aggregates verifier-aligned utility over training, selects a compact subset for a focused restart, and uses the updated model to seed the next bank.Source: §6 · p. 9

7 CONCLUSION

The conclusion argues that semantic retrieval alone does not identify which skills teach effectively. SGUID selects persistent verifier-aligned contributors, and the reported results link selection to compact distillation and stable co-evolution.

7 CONCLUSION

  • Fewer than 25% of semantically retrieved skills produce active, verifier-aligned distillation signals, motivating compact-bank selection.SGUID retains skills for persistent verifier-aligned contributions and ranks retained candidates by late-phase positive contribution counts.Source: §7 · p. 9
  • Across four models, distilling 6 selected skills matches or exceeds full-bank distillation in mean avg@12 on three models; a second round with 3 new skills does so on all four.The second round improves Qwen3-8B from 64.3% to 66.3%.Source: §7 · p. 9
  • On Qwen3-4B, selected-bank gains persist across rounds (+3.0 to +3.1 points), unlike unfiltered co-evolution (+2.1 to +1.6), which drops below base on HMMT25.The conclusion frames selection as a condition for stable model–skill co-evolution, not merely bank compression.Source: §7 · p. 9

8 LIMITATIONS

The paper evaluates co-evolution only on single-turn math reasoning and does not assess skills as inference-time patches. It reports human-led research with AI text polishing and gives materials and implementation details for reproducibility.

8 LIMITATIONS

  • The evaluation covers single-turn math reasoning; whether skills useful during training also work as inference-time patches is left for future work.The authors also note differing prior findings on inference-time skill use and hypothesize a train-test mismatch in one setting.Source: §8 · p. 9

AI USE STATEMENT

  • Generative AI was used to polish text, while humans wrote the initial draft and developed and checked the research questions, experiments, and conclusions.The authors say all AI-generated content was carefully verified.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The models, training set, and benchmarks are public; selection rules, thresholds, training settings, evaluation protocol, and selected skill banks are specified in the paper or appendices.The authors state they will release source code, curation prompts, and full candidate banks upon acceptance.Source: REPRODUCIBILITY STATEMENT · p. 10

A EXPERIMENTAL DETAILS

The experimental details specify method-specific training hyperparameters and a compact-bank retrieval setup.

A EXPERIMENTAL DETAILS

  • Table 5 lists training hyperparameters for each method, with dashes marking settings that do not apply.LoRA is applied to q proj, k proj, v proj, o proj, gate proj, up proj, and down proj in all runs.Source: Table 5 · p. 13
  • Compact-bank distillation retrieves the K selected skills by cosine similarity between the problem statement and skill description.The run disables dynamic update.Source: A EXPERIMENTAL DETAILS · p. 13

B COMPACT SKILL BANKS

The compact banks are listed by model, with separate candidate banks curated in each co-evolution round and identifiers local to each bank.

B COMPACT SKILL BANKS

  • Each co-evolution round curates its candidate bank from the current policy’s rollouts, and skill identifiers are local to that round’s bank.The same identifier in round-one and round-two table blocks refers to different skills.Source: B COMPACT SKILL BANKS · p. 14
  • Tables 6–9 list compact skill banks selected for Qwen3-1.7B, Qwen3-4B, Qwen3-8B, and Olmo-3-7B.Each table caption identifies the model whose selected banks it presents.Source: Table 6 · p. 14, Table 7 · p. 15, Table 8 · p. 15
Loading 2610.12367v1…