Source-linked AI summary
AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes
Jiahao Qiu, Xinzhe Juan, Yimin Wang, Ling Yang, Xuan Qi, Tongcheng Zhang, Jiacheng Guo, Yifu Lu, Zixin Yao, Hongru Wang, Shilong Liu, Xun Jiang, Liu Leqi, Mengdi Wang
TL;DR
Agent-distillation methods often replay teacher trajectories or imitate tool-use steps, while dynamically planning in new environments remains a challenge. AgentDistill reuses teacher-generated MCPs without training; on biomedical and mathematical benchmarks, its student agents narrow the teacher gap and outperform retrieval-based systems.
Problem
Existing agent distillation often imitates full teacher trajectories or step-by-step tool use, and prior methods are described as costly and limited in adaptability.
Source: §1 · p. 2Method
AgentDistill extracts, abstracts, clusters, and consolidates reusable teacher-generated MCPs into an MCP-Box mounted into student agents at inference time, without gradient updates.
Source: §1 · p. 2, §3.2 · p. 5, §5 · p. 10Results
MCP-equipped students improve across biomedical and mathematical benchmarks, narrow the teacher–student gap, and outperform retrieval-based systems such as OctoTools (GPT-4o).
Source: §1 · p. 2, §5 · p. 10Takeaways & Limitations
The reported results support using structured protocol distillation to transfer task-solving capabilities to small agents without additional training.
Source: §5 · p. 10The student agent receives no further gradient updates, so the described approach does not include post-distillation student training.
Source: §3.1 · p. 4
Abstract
from arXiv · showhide
While knowledge distillation has become a mature field for compressing large language models (LLMs) into smaller ones by aligning their outputs or internal representations, the distillation of LLM-based agents, which involve planning, memory, and tool use, remains relatively underexplored. Existing agent distillation methods typically replay full teacher trajectories or imitate step-by-step teacher tool usage, but they often struggle to train student agents to dynamically plan and act in novel environments. We propose AgentDistill, a novel, training-free agent distillation framework that enables efficient and scalable knowledge transfer via direct reuse of Model-Context-Protocols (MCPs), which are structured and reusable task-solving modules autonomously generated by teacher agents. The reuse of these distilled MCPs enables student agents to generalize their capabilities across domains and solve new problems with minimal supervision or human intervention. Experiments on biomedical and mathematical benchmarks demonstrate that our distilled student agents, built on small language models, can achieve performance comparable to advanced systems using large LLMs such as OctoTools (GPT-4o), highlighting the effectiveness of our framework in building scalable and cost-efficient intelligent agents.
1 Introduction
AgentDistill addresses the cost and adaptability limits of existing agent-distillation approaches by transferring teacher-generated MCPs directly to students without training. Experiments report improved student performance across biomedical and mathematical benchmarks.
1 Introduction
- AgentDistill extends a broader distillation tradition that transfers knowledge through outputs, internal representations, or teacher-generated reasoning traces.Earlier approaches align logits [1] [2], hidden features [3] [4], or attention patterns [5], while CoTD uses teacher rationales [8] [9].Source: §1 · p. 2
- Prior trajectory imitation is costly and adapts poorly, while structure distillation can overlook differences in model capabilities, knowledge, and tool use.Trajectory methods process long sequences and reproduce fixed behavior; structure-based approaches use abstract strategies such as subgoals or interaction graphs.Source: §1 · p. 2
- AgentDistill transfers teacher-generated, self-contained, reusable MCPs to smaller student agents instead of replaying full trajectories or assuming students can execute teacher-defined plans.The MCPs are tailored to task domains and intended to carry transferable problem-solving capabilities.Source: §1 · p. 2
- The framework requires no fine-tuning of teacher or student agents and reuses automatically extracted and abstracted MCPs without gradient updates or handcrafted tool use.Its stated generalization scope is unseen tasks that can be solved with the distilled MCPs.Source: §1 · p. 2
- AgentDistill improves student-agent performance across biomedical and mathematical tasks, narrowing the teacher–student gap with minimal computational overhead.The experiments cover PathVQA, SLAKE, and Game of 24 and report gains across GPT-3.5-turbo, Qwen3-8B, and LLaMA3.1-8B.Source: §1 · p. 2
2 Releated Works
Related work spans MCP infrastructure, model-level distillation, agent-distillation strategies, and generalist or domain-specific agents. AgentDistill is positioned among systems that reuse MCPs and aim to transfer tool use across domains.
2.1 MCP
- MCP-related work describes a standardized interface for language-model context, security measures, and systems that generate or reuse MCPs for adaptable agent capabilities.The section cites MCP, MCP Landscape [18], MCIP [19], and Alita [20].Source: §2.1 · p. 3
2.2 Distillation of Large Language Model
- LLM distillation methods transfer outputs, hidden features, attention, or reasoning patterns, while in-context distillation trains students to internalize few-shot reasoning.The cited approaches include output alignment [1] [2], feature alignment [3] [4], attention distillation [5], architecture-aware methods [6] [7], and ICLD [31] [32] [33] [34].Source: §2.2 · p. 3
2.3 Distillation of LLM Agent
- Agent distillation includes trajectory imitation, structure compression into graphs or subgoals, and transfer from language reasoning to non-linguistic action policies.Examples include SAD [13] and retrieved-evidence distillation [14], MAGDi [15] and Sub-goal Distillation [16], and action-policy work [35] [36].Source: §2.3 · p. 3
2.4 Generalist and Domain-Specific Agents
- Generalist agents coordinate broad tasks through modular systems, while domain-specific agents use specialized tools and prompts across fields; AgentDistill targets reusable MCPs across domains.Examples include OWL [37], AutoAgent [38], Alita [20], and domain-focused systems such as FinRobot [40], ChemAgent [41], and ClinicalAgent [42].Source: §2.4 · p. 4
3 Method
AgentDistill transfers teacher-generated, reusable MCPs to frozen small-model students instead of distilling full trajectories. Teachers create and validate MCPs, which are abstracted, clustered, and consolidated into an MCP-Box that students use as callable tools at inference.
3 Method
- AgentDistill transfers teacher-generated MCPs to smaller student agents through an MCP-Box, without student fine-tuning or trajectory replay.The teacher’s MCPs are abstracted, clustered, and consolidated before being mounted in the student’s tool interface.Source: §3 · p. 4, Figure 3 · p. 4, §3.4 · p. 6
3.1 Problem Formulation
- The objective is to improve a frozen student’s ground-truth accuracy by supplying a distilled MCP-Box built from teacher-generated MCPs.The student’s behavior is evaluated on inputs augmented with MCP-Box guidance.Source: §3.1 · pp. 4–5
3.2 MCP Creation
- Only successful teacher trajectories contribute MCPs, and retained snippets must be syntactically correct and executable.A trajectory may yield multiple self-contained MCPs for distinct subtasks; the collected pool is later organized into an MCP-Box.Source: §3.2 · p. 5
3.3 MCP-Box Construction
- MCP-Box construction rewrites MCPs into parameterized reusable forms, groups them by functionality, and consolidates each group into a general-purpose tool.The outputs include unified parameters, validation, and documentation in FastMCP-compatible Python files.Source: §3.3 · p. 5
3.4 Student Inference with the MCP-Box
- At inference, the student receives every MCP-Box tool as callable and chooses whether to invoke one, without retrieval, reranking, or parameter selection.Tool outputs update the context for subsequent reasoning, and no additional training is required.Source: §3.4 · p. 6
3.5.1 Teacher Agent
- The teacher’s Manager Agent decomposes tasks, delegates MCP creation when tools are needed, and aggregates subtask results; the MCP module brainstorms, searches, scripts, and validates tools.A Basic Image Captioner supplies text descriptions when inputs contain images.Source: §3.5.1 · pp. 6–7
3.5.2 Student Agent
- The student uses a compact language model and a streamlined Manager Agent, which coordinates task decomposition, tool use, and result aggregation with the distilled MCP-Box.The described student configuration also includes a Basic Image Captioner.Source: §3.5.2 · p. 7
4 Experiment
AgentDistill is evaluated on biomedical VQA and mathematical reasoning tasks using frozen student models and comparisons with teacher and tool-based agents. Distilled MCPs improve student performance across datasets and narrow the gap with stronger systems, while providing reusable protocols for varied tasks.
4.1.1 Tasks and Datasets.
- The evaluation covers Game of 24, PathVQA, and SLAKE, with 100 validation examples sampled per dataset for MCP-box generation and task accuracy as the main metric.Game of 24 contains 1,362 arithmetic puzzles; PathVQA has 32,000 questions over 4,998 pathology images, and SLAKE has 642 radiology images with over 14,000 expert-annotated QA pairs.Source: §4.1.1 · p. 7, §4.1.2 · p. 7
4.2 Results and Analysis
- Across the three benchmarks, Table 1 reports teacher-generated MCP counts and the percentage of test cases in which students invoke at least one MCP.The paper treats MCP calling frequency as an indicator of applicability and reuse across inputs.Source: Table 1 · p. 8, §4.2 · p. 8
- MCP-Box improves student accuracy across datasets and models, with especially large Game of 24 gains for GPT-3.5-turbo (+48.4 points, 34.3% to 82.7%) and LLaMA3.1-8B (+42.3 points, 21.7% to 64%).Table 2 summarizes before-and-after performance without additional training; the reported pattern includes moderate but consistent PathVQA improvements and smaller gains for strong Game of 24 models.Source: §4.2 · p. 8, Table 2 · p. 9
- On PathVQA, MCP-equipped students reach 52.7% accuracy, matching the teacher’s 52% and outperforming both retrieval-based variants; on SLAKE, they reach 65.1% versus the teacher’s 66%.Table 3 also compares students with OctoTools and agents using predefined tools; its scope and agent groups are specified in the caption.Source: §4.2 · pp. 8–9, Table 3 · p. 9
- MCP-Boxes act as executable protocol libraries: students still plan whether and which MCP to call and how to fill its arguments, while the library constrains tool-calling options.The protocols encapsulate parameterized tool-level logic, letting students bypass low-level code generation without transferring policy gradients or planning heuristics.Source: §4.2 · p. 9
- Teacher-generated narrow MCPs can be consolidated into a parameterized template, which supports different brain regions and analysis modes by changing arguments rather than code.The example describes reuse across clinical scenarios such as MRI versus CT and left-side versus full-brain analysis.Source: §4.2 · p. 9, Figure 5 · p. 10
5 Conclusion
AgentDistill transfers teacher agents’ task-solving capabilities to students through distilled MCPs mounted at inference time, without trajectory replay or gradient updates. On biomedical and mathematical benchmarks, MCP-equipped students close the performance gap with teacher agents and outperform retrieval-based systems such as OctoTools (GPT-4o).
5 Conclusion
- MCP-equipped students close the performance gap with teacher agents and outperform retrieval-based systems like OctoTools (GPT-4o), even with a strong LLM as the base model.The conclusion reports results on biomedical and mathematical benchmarks.Source: §5 · p. 10
- AgentDistill abstracts, clusters, and consolidates reusable tool-use strategies into an executable MCP-Box mounted in student agents at inference time.The framework transfers task-solving capabilities through distilled Model–Context–Protocols rather than trajectory replay or gradient updates.Source: §5 · p. 10
- The results highlight structured protocol distillation’s potential for efficient, modular, and generalizable agent behavior without additional training or model modifications.Source: §5 · p. 10