PaperlayerView exact revision on arXiv

Source-linked AI summary

SKILL-KD: Contrastive Skill Distillation for LLM Agents

Qiming Shi, Yibo Dou, Jiawen Zhu, Yulong Tao, Linbo Jin, Zhaolu Kang, Yunfan Zhou, Di Weng

arXiv:2607.28048v2cs.AI

TL;DR

Existing skill-acquisition methods may not convert student failures or teacher demonstrations into procedural guidance weaker agents can use. SKILL-KD distills teacher-student discrepancies into validated, iteratively refined skill patches and consistently improves frozen students across five benchmarks and two student settings.

  • Problem

    Existing methods derive skills from self-trajectories or stronger-model demonstrations, but may not convert behavioral evidence into procedural guidance usable by the target agent.

  • Method

    SKILL-KD iteratively distills teacher-student discrepancies into validated skill patches and uses trace-linked edit histories for drift-aware consolidation.

  • Results

    SKILL-KD consistently improves both frozen student models across all five evaluated task families, including held-out ALFWorld environments.

  • Takeaways & Limitations

    The framework enables knowledge transfer from strong teachers to weaker frozen students without weight updates, task-specific fine-tuning, or full trajectory replay at inference time.

  • Takeaways & Limitations

    Without trace-linked histories, repeated local revisions can cause case-specific drift or skill bloat, making the skill library less compact.

Abstract

from arXiv · show

Skill-based prompting has become a practical mechanism for improving large language model (LLM) agents, yet existing skill acquisition methods often treat skills as experience summaries, memory entries, or direct summaries of successful demonstrations. This creates a mismatch for weaker student agents: when a student fails because it lacks task knowledge or operational strategy, its failed trajectory may not contain enough evidence to infer the missing behavior, while the teacher trajectory may be too implicit to be internalized as reusable guidance. We propose SKILL-KD, a contrastive skill distillation framework that treats skills as an explicit distillation medium between agents of different capabilities. Given a student failure and the teacher trajectory on the same task, SKILL-KD distills their actionable discrepancy into a textual skill patch, evaluates the patch by re-running the student, and iteratively refines the patch when the student still fails. To prevent repeated local updates from causing skill drift, SKILL-KD further maintains trace-linked edit histories and performs Drift-Aware Skill Consolidation, deciding whether each patch should add a new rule, delete or modify an existing rule, or be skipped. Across five agent benchmarks and two student settings, SKILL-KD consistently improves frozen student agents over fixed-model adaptation baselines.

Introduction

SKILL-KD treats reusable textual skills as a distillation medium for transferring procedural knowledge from stronger to frozen weaker agents. It contrasts teacher and student trajectories, iteratively validates skill patches through student reruns, and consolidates them with trace-linked drift awareness.

  • Motivation: Existing skill-acquisition methods rely on self-experience or stronger-model demonstrations, but behavioral evidence may not reveal the missing knowledge or strategy.Student failures can show incorrect actions without explaining the corrective behavior, while teacher trajectories may remain too implicit for reuse.
  • Core formulation: SKILL-KD formulates skill-level knowledge distillation for frozen LLM agents, using textual skills and teacher-student behavioral discrepancies as the transferable learning signal.The student remains frozen, so the distilled artifact is appended to its prompt-time skill library rather than encoded through parameter updates.
  • Adaptive Skill Distillation: SKILL-KD contrasts student failures with teacher trajectories to identify behavioral gaps and express missing knowledge or strategies as textual skill patches.The framework avoids simply summarizing the failed trajectory or directly transferring the teacher trajectory.
  • Adaptive Skill Distillation: Adaptive Skill Distillation iteratively searches textual skill space by evaluating candidate patches through subsequent student behavior and maintaining refinement records.Each record includes the initial failure, teacher trajectory, candidate patches, and student rollouts after patch application.
  • Drift-Aware Skill Consolidation: Drift-Aware Skill Consolidation uses trace-linked edit histories to prevent local patches from overfitting, adding redundant rules, or overwriting knowledge.The consolidation agent inspects each patch alongside its originating trajectories and the student behavior observed after applying it.
  • Practical contribution: The framework enables teacher-to-student transfer without weight updates, task-specific fine-tuning, or full trajectory replay at inference time.It aims to maintain a compact skill library grounded in teacher-student contrasts and validated through student behavior.

Related Work

Prior work develops reusable skills for adapting LLM agents, optimizes and governs large skill repositories, and transfers stronger-agent behavior through distillation. However, existing approaches primarily evolve skills from trajectories or feedback and use skills or privileged signals mainly as training-time supervision.

  • Reusable skills: Reusable skills adapt LLM agents without changing model weights, with systems evolving skills from trajectories, verification feedback, or validation-gated text edits.Voyager exemplifies persistent skill libraries for open-ended exploration, while Trace2Skill and SkillOpt represent later skill-evolution approaches.
  • Skill lifecycle governance: Skill optimization, retrieval, compression, and lifecycle governance address redundancy, routing burden, and unsupported edits in large skill repositories.Trace2Skill uses grouped trajectories, whereas SkillOpt uses validation-set feedback for consolidation-related control.
  • Knowledge distillation: Knowledge distillation transfers stronger-model behavior through soft targets, rationales, or trajectories, while agent-focused methods distill tool use or reusable external modules.Recent work also uses skills or privileged signals mainly as training-time supervision for policy learning or reinforcement learning.

Method

SKILL-KD transfers procedural knowledge through a trace-linked textual skill library while keeping the student model frozen. It contrasts student and teacher trajectories, adaptively validates candidate patches through student reruns, and consolidates successful edits using persistent histories.

  • Skill representation: SKILL-KD represents skills as trace-linked textual rules in a shared structured Markdown library, exposing instructions while retaining rationales and trajectory links internally.The title and content are visible to both agents; why and trace support consolidation through an internal edit history.
  • Frozen-student optimization: The student model remains fixed, so optimization targets the external skill library K and its effect on student performance across training instances.This distinguishes SKILL-KD from parameter-level distillation.
  • Contrastive distillation: After a student failure, SKILL-KD compares student and teacher trajectories to identify behavioral discrepancies and generate a candidate skill patch.Even imperfect teacher trajectories can provide intermediate-decision evidence for constructing skills.
  • Adaptive validation: The framework treats patching as adaptive textual search: it reruns the student, revises failed patches, and commits only patches that produce successful student behavior.Each revised patch is proposed against the original library; unsuccessful instances produce no update after the maximum adaptive rounds.
  • Drift-aware consolidation: Drift-aware consolidation maintains trace-linked edit history and uses explicit add, modify, delete, or skip operations to update the skill library.The consolidation agent retrieves source trajectories on demand through a trace-link tool before committing a patch operation.

Experiments

Experiments evaluate SKILL-KD on five benchmark task families with two teacher–student configurations and frozen students. The method improves both students across all five task families, including substantial gains on unseen ALFWorld environments.

  • Evaluation: Evaluation covers SearchQA, SpreadsheetBench, DocVQA, LiveMath, and ALFWorld using each benchmark’s native hard-success or exact-match test score.The main experiment evaluates frozen student agents on held-out test partitions across search QA, spreadsheet manipulation, multimodal document QA, mathematical reasoning, and embodied interaction.
  • Experimental settings: Two configurations pair Qwen3.5-4B with Qwen3.7-plus and Qwen3.6-35B-A3B with ChatGPT-5.5, with the teacher also serving as consolidation agent.Training-set evaluator feedback diagnoses failed rollouts and validates candidate patches during skill construction.
  • Overall comparison: SKILL-KD improves both student models across all five task families, with especially pronounced gains for Qwen3.5-4B on SpreadsheetBench, LiveMath, and ALFWorld.The method also yields gains on SearchQA and DocVQA; Qwen3.6-35B-A3B starts from a stronger No Skill baseline.
  • Homogeneous vs. Heterogeneous Distillation: Natural-language skill artifacts let SKILL-KD transfer across model families, covering both homogeneous Qwen pairings and the heterogeneous Qwen–ChatGPT pairing.The method is agnostic to model internals because the distillation carrier is a skill artifact rather than logits or hidden representations.
  • Generalization of Distilled Skills: 30.6 to 86.6 (+56.0) is the Qwen3.5-4B improvement on ALFWorld’s unseen-environment split.Qwen3.6-35B-A3B improves from 59.7 to 96.3 (+36.6) on the same held-out split, supporting reusable interaction strategies rather than trajectory or layout memorization.

Analysis

The analysis attributes SKILL-KD’s effectiveness to contrastive behavioral validation, adaptive refinement, and drift-aware consolidation. Ablations show that these mechanisms improve performance, compactness, stability, and coverage of difficult cases.

  • Skill acquisition variants: SKILL-KD reaches 66.8 (+23.4), gaining 6.7 points over Student-only with only 38 rules, while Pairwise and Batch remain at 59.0 and 59.4.Pairwise uses 6,966 words and Batch 3,849 words, indicating that larger libraries do not ensure stronger performance.
  • Drift-aware consolidation: Removing edit histories and trace-linked trajectories makes skill editing less stable across benchmarks, despite retaining patch operations for adding, modifying, deleting, or skipping rules.The ablated variant ties full consolidation on SearchQA and remains close on ALFWorld, but falls elsewhere.
  • Drift-aware consolidation: Drift-aware consolidation addresses case-specific, skill-bloat, and destructive-update drift caused by repeatedly applying local edits without historical evidence.Examples include a 50-rule, 2,930-word SpreadsheetBench library and later edits that weaken previously useful calculation behavior.
  • Adaptive rounds: The first adaptive round raises average training success from 58.9% to 67.7%, while later rounds increase it to 70.8% and then 72.7%.Later improvements concentrate in SpreadsheetBench, LiveMath, and ALFWorld, supporting a bounded adaptive procedure.
  • Teacher outcomes: Teacher success averages 46.1% on initially failed student cases, with patch acceptance at 45.3% after teacher success and 23.2% after teacher failure.Teacher-failed cases still account for 38.5% of accepted patches on average, so supervision is not oracle-dependent.

Conclusion

SKILL-KD improves frozen LLM agents by distilling teacher–student behavioral discrepancies into reusable skills rather than summarizing demonstrations or failures alone. It validates and consolidates skill patches, consistently improving students across benchmarks while adaptive validation and drift-aware consolidation prove important.

  • SKILL-KD converts teacher–student behavioral discrepancies into reusable skills for improving frozen LLM agents.The framework contrasts teacher and student behavior instead of relying only on teacher demonstrations or isolated student failures.
  • The method distills actionable gaps, validates candidate patches through student reruns, and consolidates accepted patches with trace-linked edit histories.These operations form the framework’s patch-generation and skill-library maintenance process.
  • Across five benchmarks and two teacher–student settings, SKILL-KD consistently improves frozen students over no-skill and skill-learning baselines.Ablations identify adaptive validation and drift-aware consolidation as key components for the resulting skill library.

A Experimental Details · B Consolidation Agent Prompt and Tools

The experiments use controlled benchmark-specific skill libraries, fixed data splits, and a default of three adaptive rounds. The consolidation agent curates compact reusable rules from current skills, edit histories, and trajectories through disciplined add, update, delete, or no-change operations.

  • A Experimental Details: SKILL-KD is evaluated on five benchmarks using native or deterministic train/validation/test partitions, with skill construction restricted to training data.For benchmarks without official splits, the protocol uses seed 42 and a default 2/1/7 train/validation/test ratio.
  • A Experimental Details: Student models run locally on four NVIDIA A100 80GB GPUs, while teacher and consolidation models use official APIs.Each score comes from one complete evaluation run on the fixed test split.
  • A Experimental Details: The default adaptive-training budget is n=3 rounds per task instance, with cumulative training success rates reported by round.Table 6 reports macro-averaged and benchmark-specific sensitivity results.
  • A Experimental Details: Student and teacher share a benchmark-specific skill library, and training and evaluation prompts expose the entire current skill file.Rationale and trace metadata remain internal to consolidation, isolating skill acquisition from retrieval or routing.
  • B Consolidation Agent Prompt and Tools: The consolidation agent acts as a skill curator using current skills, edit history, task context, student/teacher/retry trajectories, and evaluator records.Trajectories are ordered as student_initial, teacher, then same-task retries; the edit log retains rule-version metadata and edit references, with payloads retrieved as needed.
  • B Consolidation Agent Prompt and Tools: Each rule has a short title, a type-level “When <trigger>, do <action>” content sentence, and an audit-only rationale explaining failure versus success.Only the title and content are shown to the student at runtime.
  • B Consolidation Agent Prompt and Tools: The curator classifies rules as cover, partial, or unrelated; prefers updates for partial coverage, adds only reusable triggers, deletes overlap or low-value rules, and submits at most three operations.It avoids cosmetic or case-specific changes and uses [] when no change is needed.

C Representative Final Skills

The appendix presents all final rules in the Group 1 4B skill library, organized by benchmark. Each card shows the student- and teacher-rendered fields, while longer contents are truncated and edit-history metadata remains internal.

  • Final skill library: The cards list all final rules from the Group 1 4B skill library K, grouped by benchmark.This organization structures the representative final skills by benchmark.
  • Final skill library: Long rendered contents are truncated after 28 words to keep the appendix readable.The truncation applies to the card contents shown in the appendix.
  • Final skill library: Each card reports only the fields rendered to the student and teacher, while why and trace remain in the internal edit history.This follows the representation described in Section 3.1.

SearchQA

The SearchQA skills emphasize resolving question structure and referents before answering, then selecting the contextually precise entity, form, title, location, or source material. They also promote concise answers that preserve necessary identifying information while avoiding redundancy.

  • Answer selection: Match answers to the requested semantic form, including categorical subtypes, dominant singular or plural usage, precise terms or numbers, and full location names.These rules cover type questions, number agreement, definition questions, and location codes or abbreviations.
  • Question parsing: Parse “this” correctly by identifying the grammatical role and entity referred to in the described relationship.This skill applies to both “this [noun]” constructions and standalone pronouns.
  • Entity resolution: Prefer the primary entity when a question mentions multiple instances, roles, or items but context emphasizes one underlying entity.The choice should reflect contextual primacy rather than exhaustively listing every candidate.
  • Answer formulation: Avoid redundant context in answers while preserving necessary identity details, including middle initials when they appear in a person’s full name.The answer should be concise without dropping required identifying information.
  • Question formats: Resolve specialized question formats by recognizing shared locations or organizations, Jeopardy category-clue structure, quotes or partial phrases, and possessive-pronoun referents.The skills infer what the question is asking from its structural cues.
  • Work and attribution resolution: Use the original title, original artist, or source material when questions concern translations, tribute works, or adaptations.These rules prioritize the underlying original work or creator over a translation, tribute group, or adapted version.

SpreadsheetBench

SpreadsheetBench skills emphasize type-aware spreadsheet manipulation, dynamic lookup and parsing, direct value replication, rolling workday calculations, and numeric cleaning. These procedures provide concrete rules for reliable spreadsheet operations.

  • SpreadsheetBench: Extract time components by calling .time() on datetime objects and handling string-valued cells separately.The extraction procedure first checks the spreadsheet cell’s value type.
  • SpreadsheetBench: Read lookup values dynamically from spreadsheet cells, identify the relevant columns, and avoid hardcoding list or range values.Dynamic reading keeps lookup behavior tied to the spreadsheet contents.
  • SpreadsheetBench: Parse currency-formatted strings containing $ and commas into numeric values before applying tiered range calculations.Use minimum and maximum bounds when computing tiered amounts.
  • SpreadsheetBench: Replicate a cell by reading its value and writing that value directly to the target, rather than copying a formula.Blank or None source values require corresponding handling.
  • SpreadsheetBench: For rolling workday windows, define workdays as Monday through Friday and collect dates from the specified starting point.Workdays are identified with weekday() < 5 before summing or averaging.
  • SpreadsheetBench: Clean numeric cell strings by retaining only digits and decimal points, excluding commas and other non-numeric characters.Commas should not be preserved in the resulting numeric strings.

DocVQA

DocVQA skills emphasize exact document-field extraction and answer formatting. They also address normalization and visual disambiguation for numbers, names, symbols, and typography.

  • Field extraction: Scan for the exact field label and extract the text immediately adjacent to it, preserving the requested field value.This is the core procedure for structured document field extraction.
  • Answer precision: Return only the specific entities requested, omitting supplementary context such as related product names, usage details, or descriptive labels.The answer should exclude irrelevant surrounding information.
  • Value normalization: Normalize extracted values by removing leading punctuation from numbers, converting Roman numerals to Arabic numerals, and retaining adjacent currency symbols.These formatting rules distinguish the requested value from presentation artifacts while preserving monetary notation.
  • Visual disambiguation: Verify visually ambiguous handwritten digits, printed letters, spaced compound brand names, and quoted terms to avoid misrecognition and normalize quotation marks to straight double quotes.Checks include loop closures and tail shapes for digits, confused letter pairs in names, and treating visually spaced logo text as one brand name.

LiveMath · ALFWorld

The section presents domain-specific skills for selecting stronger provable results in LiveMath and executing object-manipulation tasks in ALFWorld. These skills emphasize checking task structure and following ordered action sequences.

  • LiveMath: For nonlinear PDE blow-up criteria, verify equation-specific energy estimates before defaulting to an L1-in-time gradient norm.The skill targets selecting the strongest provable blow-up criterion.
  • LiveMath: For multiple-choice questions asking for the strongest statement, select a meta-option claiming that a stronger result exists.This applies when such a meta-option is offered.
  • LiveMath: For circular averaging operators in R2 with r = 2, do not assume Lp boundedness for p > 2 from maximal-function results.The skill concerns critical variation unboundedness in harmonic analysis.
  • LiveMath: For block-stable random graph classes, identify the block tree as subcritical when u < uC.This rule concerns tree distribution relative to the critical threshold uC.
  • ALFWorld: When asked to examine an object in lamp light, pick up the target first, navigate to the lamp, and then use it.The prescribed order is pickup, navigation, and lamp use.
  • ALFWorld: When washing an object and putting it away, search likely locations, including the fridge, countertop, and diningtable, before picking it up.The skill specifies an ordered search-and-pickup procedure.
  • ALFWorld: When chilling an object for placement on a receptacle, search countertops and stoveburners to locate and pick up the target.The passage describes the initial search and pickup sequence.
  • ALFWorld: When heating an object for placement in a receptacle, search countertops, diningtables, and the fridge to locate and pick up the target.The passage specifies likely search locations for the heat-then-place task.
Loading 2607.28048v2…