PaperlayerView exact revision on arXiv

Source-linked AI summary

D-Skill: Distilling Teacher Knowledge through Agent Skills

Shuo Yuan, Dacheng Li, Joseph E. Gonzalez

arXiv:2610.28914v1cs.AI

TL;DR

Weight-based knowledge distillation can require student weight access and additional training compute, while storing transferred knowledge implicitly in parameters. D-Skill builds an external natural-language skill from paired teacher–student rollouts and consolidates it offline. Across three benchmarks and multiple model families, it closes 82–95% of the teacher–student gap without weight updates.

  • Problem

    Traditional knowledge distillation updates student parameters, requiring weight access and additional compute while encoding transferred knowledge in parameters that are difficult to inspect or revise.

    Source: §1 · pp. 1–2

  • Method

    D-Skill compares paired teacher and student rollouts for behavioral divergences and agreements, consolidates lessons into a natural-language skill, and provides it to the student at inference time.

    Source: §1 · p. 2

  • Results

    Across SpreadsheetBench Verified, ALFWorld, and chess puzzles and multiple model families, D-Skill closes 82–95% of the teacher–student performance gap without weight updates.

    Source: §7 · p. 10

  • Takeaways & Limitations

    D-Skill offers an interpretable textual alternative to weight-based transfer, with stronger one-pass transfer and performance competitive with multi-epoch SFT in the evaluated setting.

    Source: §1 · p. 2, §7 · p. 10

    D-Skill transfers only capabilities expressible as instructions and executable by the student; it cannot supply fundamental capabilities the student lacks.

    Source: §6 · p. 9

Abstract

from arXiv · show

Frontier language-model agents excel at complex tasks, but high inference costs make transferring their capabilities to smaller models essential. Traditional knowledge distillation achieves this through parameter updates, requiring access to the student's weights and additional training compute while encoding the transferred knowledge implicitly in the parameters. We introduce D-Skill, a framework that treats the teacher as a black box and distills its behavioral advantage into an interpretable natural-language skill by analyzing paired rollouts and identifying behavioral divergences and agreements. The resulting lessons are consolidated offline, with any compatible skill-update backend, into a skill document that is provided to the student at inference time. D-Skill constructs the skill entirely offline from fixed trajectories, without labels, further environment interaction, or weight updates after rollout collection. Across SpreadsheetBench-Verified, ALFWorld, and chess puzzles spanning multiple teacher-student pairs, D-Skill closes 75%-95% of the teacher-student gap without touching model weights. After a single offline pass over the paired traces, D-Skill outperforms one-epoch SFT baselines and remains competitive with multi-epoch SFT. Under the same model family, the resulting student also incurs 5.7x-8.3x lower inference cost per task than the teacher. Our results demonstrate that natural-language skill distillation offers a performant, interpretable, and efficient alternative to weight-based capability transfer.

1 INTRODUCTION

Weight-based distillation can require student weight access and costly training, while existing skill methods do not target the teacher–student relationship. D-Skill compares paired rollouts to build an external skill, closing most of the teacher–student gap across three benchmarks.

1 INTRODUCTION

  • Classical distillation updates student parameters using teacher outputs or trajectories, requiring weight access and backpropagation and encoding transferred knowledge in parameters that are difficult to inspect or revise.The text also identifies a need for sufficient training data for generalization.Source: §1 · pp. 1–2
  • Existing skill methods either compile stronger-model trajectories without conditioning transfer on the target student's behavior or learn from student experience without a matched stronger-agent reference.D-Skill instead constructs a student-targeted skill from teacher and student trajectories on the same tasks.Source: §1 · p. 2
  • D-Skill analyzes paired teacher and student rollouts for divergences and agreements, consolidates lessons into a skill, and provides it to the student at inference time.After rollout collection, it needs no labels, answer keys, or further environment interaction, and does not modify model weights.Source: §1 · p. 2
  • D-Skill closes 83.4%–89.3% of the teacher–student gap on SpreadsheetBench Verified, 84.6%–94.6% on ALFWorld, and 75.3%–87.7% on chess puzzles.The evaluation spans different teacher–student pairs and model families.Source: §1 · p. 2
  • The paper frames skill distillation as an interpretable external alternative to weight-based transfer that requires neither weight access nor gradient-based training.The resulting skill-learning method outperforms evaluated skill-learning baselines while retaining substantially lower inference cost than the teacher.Source: §1 · p. 2

2 RELATED WORK

D-Skill is positioned against parameter-based knowledge distillation, persistent skills and memories, and automatic textual optimization. It differs by using behavioral contrasts between a teacher and target student to construct a skill for capability transfer.

Knowledge distillation

  • Classical knowledge distillation transfers capabilities through parameter updates and distributional signals, whereas D-Skill uses explicit textual skills and behavioral contrasts between teacher and student trajectories.This framing addresses closed-weight students and black-box teachers without requiring shared-tokenizer access.Source: §2 · pp. 2–3

Skill and memory-augmented agents

  • Skill and memory methods include agent memories, skills induced from an agent's own rollouts, and teacher-driven skills; D-Skill instead compares teacher and student trajectories to target their behavioral differences.The passage describes Reflexion, Voyager, SkillGen, Trace2Skill, and AgentDistill as examples of these lines of work.Source: §2 · p. 3

Automatic textual optimization

  • Text-optimization methods refine textual artifacts, typically using task evaluation, while D-Skill derives updates from matched teacher–student behavioral contrasts and can use different skill-update backends.The described methods include prompt search, textual critiques, and skill-file optimization.Source: §2 · p. 3

3 METHOD

D-Skill constructs an external skill from paired teacher and student traces, with offline analysis and consolidation after traces are collected. Its method specifies the transfer objective, rollout collection, structured lesson extraction, and skill updating.

3 METHOD

  • The method has three stages—rollout collection, difference analysis, and skill update—and performs subsequent analysis and skill construction offline on fixed traces.Fresh rollouts require environment interaction, while saved traces allow the process to begin without new interaction.Source: §3 · p. 3

3.1 PROBLEM SETUP

  • Given paired trajectories from unlabeled tasks, the objective is to construct a natural-language skill that improves the student without modifying its parameters.Transfer is quantified as the fraction of the teacher–student performance gap closed.Source: §3.1 · p. 3

3.2 ROLLOUT COLLECTION

  • For each task, the teacher and student independently produce trajectories; they need only share the task and action interface, and saved traces can be used instead of fresh executions.After collection, no further policy calls, environment access, labels, rewards, or evaluation feedback are required.Source: §3.2 · p. 4

3.3 DIFFERENCE ANALYSIS

  • A difference analyst extracts up to three ranked lessons per pair, using behavioral dimensions and trace evidence to identify reusable divergences and agreements.Divergence lessons pair the student's behavior with the teacher's alternative; agreement lessons identify shared behavior worth preserving.Source: §3.3 · p. 4

3.4 SKILL UPDATE

  • The consolidator maps lessons to candidate patches, reduces them by removing duplicates, resolving conflicts, and generalizing rules, then applies the merged patch to the skill.The adapted Trace2Skill backend is used in the experiments, but compatible skill-updating backends can replace it without changing collection or analysis.Source: §3.4 · pp. 4–5

4 EXPERIMENTS

Across spreadsheet, household-task, and chess evaluations, students equipped with D-Skill skills approached teacher performance without weight updates. Ablations show that paired behavioral contrast and a capable analyst matter, while lessons also work with an alternative skill-update backend.

4.1 MAIN RESULTS

  • D-Skill closed 83–95% of the teacher–student gap across eight SpreadsheetBench-Verified and ALFWorld configurations, and 75.3% and 88% on the two chess pairs.On chess, Haiku 4.5 rose from 41.2 to 52.5 and Qwen3.6-35B-A3B from 24.8 to 39.0.Source: §4.1 · p. 6

4.2 ABLATIONS

  • Across three independent skills, each evaluated twice, full paired contrast closed 85–89% of the teacher–student gap, versus 38–45% for student-only and 55–71% for teacher-only analysis.Source: Table 2 · p. 7, §4.2 · p. 7
  • Replacing the analyst with Qwen reduced chess gap closure to 39–48%, while replacing the consolidator cost about five points, from 39.0 to 34.3.The weaker analyst produced fewer confirmed lessons and noisier explanations; the student consolidator yielded a longer skill document.Source: Table 3 · p. 7, §4.2 · p. 7
  • D-Skill separates lesson analysis from consolidation, so structured lessons can be passed to a backend that accepts textual edits.The SkillOpt test replaced only the input to its reflection step; downstream merging, ranking, and validation gating stayed unchanged.Source: §4.2 · p. 7
  • With SkillOpt’s update backend, lesson conditioning reached 36.9 ± 5.7 on chess, 7.3 points above SkillOpt’s original trajectory-reflection analyst.The SkillOpt-backed variant retains its held-out validation gate.Source: Table 4 · p. 7, §4.2 · p. 7
  • One-pass D-Skill outperformed one-epoch SFT and remained competitive with SFT over epochs two through five on SpreadsheetBench Verified and ALFWorld.Both methods used Qwen3.6-35B-A3B and the same teacher-trajectory corpus; the figure tracks teacher tasks processed.Source: Figure 3 · p. 8

5 ANALYSIS

D-Skill substantially outperforms the compared skill-learning pipeline and reaches multi-epoch SFT performance after one offline pass. Its distillation also incurs a one-time cost that scales with the number of training tasks.

5.1 COMPARING WITH OTHER SKILL LEARNING PIPELINES

  • D-Skill scores 71.7 and 73.3, closing 85% and 89% of the gap, versus 10–11% for published Trace2Skill and 16–39% for its strongest variant.Replacing student traces with teacher traces did not improve the stronger baseline and reduced scores in two model settings.Source: §5.1 · p. 8

5.2 COMPARING WITH WEIGHT UPDATES

  • D-Skill reaches 75.5 on SpreadsheetBench Verified and 82.5 on ALFWorld after one pass, remaining competitive with multi-epoch SFT.The comparison uses the same teacher-trajectory corpus; D-Skill requires no gradient updates or access to student weights.Source: §5.2 · p. 9

5.3 COST ANALYSIS

  • Distillation costs about 3.3 cents per SpreadsheetBench training task and 2.6 cents per ALFWorld task, with total cost growing linearly with task count.Figure 4 reports list-price inference costs in cents per episode for SpreadsheetBench Verified and ALFWorld; Table 6 identifies the one-time GPT-5.6 Terra→Luna cost.Source: §5.3 · p. 9, Figure 4 · p. 9, Table 6 · p. 9

6 LIMITATIONS

D-Skill has limitations in analyst requirements, inference overhead, and the kinds of capabilities it can transfer.

6 LIMITATIONS

  • A capable analyst is needed at distillation time; using the weaker student substantially reduces the fraction of the gap closed.This is a one-time rather than per-inference cost.Source: §6 · p. 9
  • The distilled document increases inference cost over the bare student through added input tokens and, in some cases, additional interaction steps.The resulting agent nevertheless remains substantially cheaper than the teacher.Source: §6 · p. 9
  • D-Skill transfers only capabilities expressible as instructions and executable by the student, not missing fundamentals such as deeper calculation, perception, or model capacity.The paper contrasts these limits with procedures such as planning, verification, and tool use that skills can improve.Source: §6 · p. 9

7 CONCLUSION

D-Skill distills behavioral differences between a black-box teacher and student into a natural-language skill using paired task rollouts. Across three evaluated domains, it closes 82–95% of the performance gap and remains cheaper to run than the teacher.

7 CONCLUSION

  • D-Skill contrasts paired teacher and student rollouts on the same tasks to distill their behavioral gap into a natural-language skill.After trajectory collection, it requires no further policy queries, environment interaction, labels, rewards, evaluation feedback, or weight updates.Source: §7 · p. 10
  • Across SpreadsheetBench Verified, ALFWorld, and chess puzzles, D-Skill closes 82–95% of the teacher–student performance gap across multiple model families.The resulting students retain a substantial inference-cost advantage over their teachers.Source: §7 · p. 10
  • Using the same teacher-trajectory corpus, D-Skill provides stronger one-pass transfer and remains competitive with multi-epoch SFT without gradient updates or student-weight access.The paper presents natural-language skill distillation as a practical, interpretable, inference-efficient alternative in the evaluated domains.Source: §7 · p. 10

A CHESS PUZZLE ENVIRONMENT SETUP

The chess environment evaluates agents on Lichess tactics puzzles, requiring complete tactical sequences without correctness feedback. It supplies structured board observations and static analysis tools, then estimates performance using a rating-based solve model.

A CHESS PUZZLE ENVIRONMENT SETUP

  • Each episode is a Lichess tactics puzzle with a scripted opponent move and a one- or two-move solution; the first incorrect move ends the episode.A puzzle is solved only when every solver move is correct, and the agent receives no correctness feedback.Source: A CHESS PUZZLE ENVIRONMENT SETUP · p. 15
  • The evaluation set contains 200 puzzles and the disjoint distillation set contains 100, each filtered for popularity, plays, rating deviation, and solution length.Evaluation ratings span 649–1987; distillation ratings span 612–2357.Source: A CHESS PUZZLE ENVIRONMENT SETUP · p. 15
  • At each solver move, the observation includes the FEN, ASCII board, legal moves, recent moves, material count, halfmove clock, and optional prior plan note.In a two-move puzzle, the second observation clarifies that earlier tool results referred to the previous position.Source: A CHESS PUZZLE ENVIRONMENT SETUP · pp. 15–16
  • Agents may make up to 10 tool calls per solver move, with repeated queries cached and malformed or illegal calls consuming budget.They must commit a move using an action tag; a move outside the legal list is scored incorrect.Source: A CHESS PUZZLE ENVIRONMENT SETUP · p. 16
  • The six in-process chess tools report static position facts without engine search or evaluation, and performance is summarized by a fitted puzzle-rating threshold.The reported threshold is where fitted solve probability reaches one half; document Elo averages two evaluation passes.Source: Table 7 · p. 15, A CHESS PUZZLE ENVIRONMENT SETUP · p. 16

B DIFFERENCE ANALYST PROMPT

The difference analyst compares paired teacher and student traces to identify consequential behavioral differences and shared strategies, then turns them into evidence-grounded, reusable instructions.

Domain-specific prompt slots

  • The same system-prompt template serves all three domains, with slots specifying environment, comparison dimensions, trace units, checks, and generalization criteria.Chess adds a prohibition on asserting a specific move was correct; SpreadsheetBench-Verified asks the analyst to trace answer-range differences to decisions without judging which value is correct.Source: B DIFFERENCE ANALYST PROMPT · p. 16

Inputs and behavioral scope

  • The analyst receives the task instruction and both traces, but no answer key or correctness information, and must assess behavior rather than solution correctness.SpreadsheetBench-Verified inputs may also include the graded answer range and cell contents, without revealing gold values.Source: B DIFFERENCE ANALYST PROMPT · p. 16

Behavioral comparison

  • It compares the student’s behavior with the stronger agent’s to identify differences and shared behaviors worth transferring.The prompt asks for lessons that make future student traces resemble the expert’s.Source: B DIFFERENCE ANALYST PROMPT · pp. 16–17

Evidence and generalization

  • Observations must cite concrete trace evidence, avoid speculation about unseen steps, and generalize across the domain’s specified scope.A domain-specific check and generalization criterion are supplied through prompt slots.Source: B DIFFERENCE ANALYST PROMPT · pp. 16–17

Output format

  • The output allows at most three impact-ordered items: behavioral-gap failure memories and at most one shared-behavior success memory.Each item gives a trace-grounded description and one reusable instruction; its content also includes a concrete runtime check.Source: B DIFFERENCE ANALYST PROMPT · p. 17

C SUPERVISED FINE-TUNING BASELINES

The SFT baselines train the same student on the teacher episodes used by D-Skill and evaluate checkpoints with the paper’s common harness and scorer.

Evaluation

  • Figure 3 tracks SFT checkpoints trained on the same teacher episodes that D-Skill distills and scored with the same evaluation harness and scorer.Checkpoints are saved within epoch 1 after fixed numbers of training episodes and at each epoch end.Source: C SUPERVISED FINE-TUNING BASELINES · p. 17

Training setup

  • The SFT setup uses LoRA rank 16, a 10^-4 peak learning rate, 5% warm-up, linear decay, five epochs, and assistant-token-only loss.Training uses all teacher episodes, including failures, without correctness labels; the teacher’s skill-bearing system prompt is replaced by the student’s harness prompt.Source: C SUPERVISED FINE-TUNING BASELINES · p. 17

Training variants

  • Two SFT target variants train on actions alone or on teacher reasoning plus actions, with reasoning placed in the student’s <think> channel.Checkpoints are served in bf16 with thinking enabled and scored as the mean of two passes.Source: C SUPERVISED FINE-TUNING BASELINES · p. 17

Results tables

  • Tables 8 and 9 present SFT results within epoch 1 and across epochs, respectively.The supplied captions identify their scopes but do not provide cell values.Source: Table 8 · p. 18, Table 9 · p. 18

D DISTILLED SKILL DOCUMENTS

The skill excerpts illustrate task-specific procedures and pitfalls, with rules that prioritize productive actions, verification, and stopping when the task is complete.

Excerpt scope

  • The excerpts show the highest-scoring D-Skill documents for Qwen3.6-35B-A3B, using each document’s opening section and first three Common Pitfalls rules.The complete documents appear in the supplementary material.Source: D DISTILLED SKILL DOCUMENTS · p. 18

SpreadsheetBench-Verified

  • The spreadsheet workflow inspects the whole workbook before modification, then reloads the saved file to verify the answer range and preserved sheet names.It directs agents to modify the existing workbook rather than create a new one.Source: SPREADSHEETBENCH-VERIFIED · p. 18
  • Spreadsheet pitfalls require writing computed values rather than formula strings, type-checking and normalizing strings, and copying source number formats when needed.The instructions also caution against converting arbitrary cell types to strings for matching.Source: SPREADSHEETBENCH-VERIFIED · p. 19

ALFWorld

  • The ALFWorld workflow follows acquire–transform–deliver subgoals and matches each next step to an available admissible action.For multi-object instructions, it treats each object noun as required and tracks objects as they are found.Source: ALFWORLD · p. 19
  • After a placement is confirmed at the goal destination, the ALFWorld rules say to stop acting rather than retrieve items or restart the task.Other pitfalls discourage repeating uninformative actions and revisiting surfaces already found empty.Source: ALFWORLD · pp. 19–20

Chess

  • Chess fast-path rules require a specific trigger and a condition barring the shortcut; one example stops rechecking a confirmed tactical idea unless its first check relied on an unchecked hand-parsed FEN.The chess pitfalls also warn against repeating analysis without a new tool result or revisiting a candidate already refuted by the agent’s calculation.Source: CHESS PUZZLES · pp. 20–21

Related papers

Loading 2610.28914v1…