Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses

1.1 Motivation: From Model Distillation to Agent Distillation

• Conventional distillation is model-centered, while agent competence is expressed through execution and may remain coupled to a particular agent and task context.Prior experience can be incorporated into successive agent realizations for continual improvement.Source: §1 · p. 2, §1.1 · p. 2
• Agent Distillation transfers teacher-derived task-solving knowledge persistently into a student agent, with improved student competence as its intended outcome.The transfer unit is the complete agent realization rather than only the model.Source: §1.1 · p. 2

1.2 Comparison with Existing Surveys

• Distillation surveys focus mainly on model-level transfer, while agent surveys organize system capabilities; neither centers persistent teacher–student transfer into a complete student agent.The paper positions its survey at the intersection of these two literatures.Source: §1.2 · p. 3

1.3 Our Perspective and Contributions

• The survey classifies approaches by the primary substrate where teacher-derived knowledge persists: the parametric model, reusable artifacts, or execution harness.It also examines migration across substrates and hybrid strategies, and proposes evaluation dimensions of capability preservation, behavioral fidelity, and structural recovery.Source: §1.3 · p. 4

2.1 Agent Realizations and Task Solving

• Agent Distillation takes the complete agent realization—the model, reusable artifacts, and execution harness—as its unit, since behavior can change with the harness even when the model is fixed.Source: §2.1 · p. 5
• Agentic competence is expected task-solving performance of a complete realization under a task distribution, environment, and budget; execution traces are evidence of competence, not competence or automatically reusable knowledge.Source: §2.1 · p. 6

2.2 A Running Example of Agent Distillation

• In the repository example, selection removes transient residue and abstraction separates reusable principles from repository-specific details, yielding a debugging strategy from the teacher’s trace.The strategy is to reproduce the failure, keep edits local, test from targeted to full suite, and roll back regressions.Source: §2.2 · p. 7
• Distillation requires persistent realization of teacher-derived knowledge in the student’s model, artifacts, harness, or jointly across components, enabling reuse beyond the source episode rather than replay of its specific patch.Source: §2.2 · pp. 7–8

2.3 Agent Distillation: Definition and Scope

• The formal definition treats the teacher as the realization from which task-solving knowledge is derived and the student as the realization that persistently incorporates it, covering inter-agent transfer, self-evolution, and learning from multiple teachers.Knowledge may derive from individual teacher components or their joint execution.Source: §2.3 · p. 8
• Agent distillation persistently incorporates teacher knowledge into a student that can reuse it on subsequent tasks, through the model, reusable artifacts, harness, or hybrid realizations.These realizations are not mutually exclusive.Source: §2.3 · p. 8
• The objective is the least complex teacher-derived knowledge sufficient to improve student competence, retaining useful experience while excluding information that does not contribute.Source: §2.3 · pp. 8–9
• The scope requires identifiable teacher-side evidence, transferable task-solving knowledge derived from it, and persistent incorporation that can influence future episodes.The teacher may be another agent, multiple agents, or the same agent in self-evolution.Source: §2.3 · p. 9
• In-context learning, inference-time consultation [64], and multi-agent collaboration [49] [106] are not distillation when effects end in the current episode; persistent changes can make them mechanisms.Feedback-driven adaptation confined to the current episode falls outside the scope [64] [119].Source: §2.3 · p. 9
• Self-evolution qualifies when prior experience supplies teacher-side evidence for reusable knowledge retained across episodes [84] [99] [134]; the framework is agnostic to knowledge form and agent realization.Source: §2.3 · p. 9

2.4 Survey Methodology

• The survey used definition-guided searches because “Agent Distillation” was inconsistently used, querying research databases and preprint sources for studies available through August 31, 2026.Search terms covered agent learning, distillation, experience reuse, memory, tools, skills, and workflow optimization, with backward and forward citation tracing.Source: §2.4 · p. 9
• Core studies were those persistently incorporating teacher knowledge into model, artifact, or harness components; transient prompting and inference-time collaboration served only as context.The researchers coded teacher–student configuration, knowledge form, updated components, mechanism, and evaluation setting to develop the following taxonomy.Source: §2.4 · p. 9

3 Our Lens: a Knowledge-Substrate-Centric Taxonomy

• Parametric, artifact, and harness knowledge are complementary modes of internalizing, externalizing, and operationalizing transferred knowledge.They persist, respectively, in learned model computation, independently addressable reusable objects, and execution or control logic.Source: §3 · pp. 9–10
• Classify a method by how retained knowledge participates in future task solving, not by its physical format or acquisition mechanism.Knowledge that modifies learned computation is parametric; reusable addressable resources are artifact-based; knowledge governing execution sequencing and control belongs to the harness.Source: §3 · p. 10
• A separately stored adapter remains parametric, a callable script is an artifact, and the logic controlling its invocation belongs to the harness.Teacher outputs, rationales, and traces are evidence for distillation, while fine-tuning, memory construction, and workflow induction are transfer mechanisms; neither determines the category independently.Source: §3 · p. 10
• Cross-substrate transfer describes transformation between substrates, whereas hybrid realization describes knowledge jointly retained across multiple substrates.The first characterizes the transfer path; the second characterizes the resulting organization.Source: §3 · p. 10

4 Parametric Distillation: Internalizing Agent Knowledge

• Parametric distillation stores teacher-derived behavioral regularities in the student’s learned computation; answers, token distributions, rationales, and trajectories are evidence, not the retained knowledge itself.The transfer objective focuses on the histories where teacher feedback is supplied, rather than on a fixed catalogue of losses.Source: §4 · pp. 10–11

4.1 From Language Distillation to Policy Internalization

• Agentic task solving requires training beyond fixed prompts because student decisions shape later histories, while response-level distillation offers little supervision for student-generated errors and states.Reasoning traces can provide decomposition and verification patterns, but their value depends on correctness, faithfulness, diversity, and student compatibility; stronger traces may exceed student capacity.Source: §4.1 · p. 12
• Parametric Agent Distillation expands training evidence from response-level reasoning to trajectories recording how a teacher agent reasons, acts, observes, and adapts.This makes transfer relevant to decisions across interaction histories, including tool selection, interpreting observations, plan revision, and recovery from failed actions.Source: §4.1 · pp. 12–13

4.2 Distilling Teacher-Generated Reasoning and Interaction Trajectories

• Teacher interaction trajectories capture state-dependent action consequences and information acquisition, but become transferable parametric knowledge only when reusable regularities are incorporated into the student’s decision process.Trajectory curation can filter failed outcomes or invalid steps, while retaining failures that reveal diagnosis and recovery.Source: §4.2 · p. 13
• High-quality ReAct-style interaction traces can substantially improve agent performance and internalize behavior otherwise supplied by elaborate prompts [10].The learned policy remains tied to the action space, observation format, and prompting conventions represented in its trajectory corpus.Source: §4.2 · pp. 13–14
• Trajectory distillation teaches a coupled pattern of reasoning, tool use, incorporating observations, and continuing toward a goal, while diverse data exposes reusable cross-task regularities.When the student retains those regularities through learned computation rather than retrieving demonstrations, the persistent outcome is parametric knowledge.Source: §4.2 · pp. 13–14
• Offline trajectory distillation mismatches teacher-induced training histories with student-induced deployment histories, allowing early deviations to compound into uncovered failure states.Critical-step selection, structured decomposition, and capacity-aligned construction improve offline evidence relevance but do not eliminate this distribution mismatch.Source: §4.2 · p. 14

4.3 On-Policy Distillation from Student-Induced Experience

• On-policy distillation samples student-generated histories and obtains teacher feedback there, making the teacher an adaptive supervisor at the student's competence boundary.Its defining feature is the source of training states, not whether parameter updates occur after each action; teacher-derived guidance is persistently incorporated into the model.Source: §4.3 · p. 15
• Capacity alignment determines which teacher-derived behavior is selected: forward KL promotes coverage of teacher modes, whereas reverse KL favors modes plausible to the student.Teacher matching can also be combined with task rewards, and the target is behavior sufficient for improved competence and feasible for the student, not reproduction of the full teacher distribution.Source: §4.3 · p. 15
• Feedback granularity and access shape practical on-policy distillation: token distributions require white-box or logits access, while black-box teachers can provide actions, corrections, preferences, or assessments.Querying every generated token is expensive for long outputs, so mixture sampling, cached computations, and replay trade closeness to the current student distribution against reuse of experience.Source: §4.3 · pp. 15–16
• Teacher and student need not be different model identities: privileged information or verified traces can supply teacher evidence retained through parameter updates for future episodes.The key distinction is whether prior experience provides identifiable teacher evidence, not whether teacher and student are separate models.Source: §4.3 · p. 16
• OPD can anchor early learning with dense teacher guidance while reinforcement learning optimizes delayed task utility, but teacher-derived evidence must remain identifiable.Foundational OPD methods largely use linguistic prefixes; agentic tasks require extending supervision to histories shaped by external environment observations and transitions.Source: §4.3 · p. 16

4.4 Extending OPD to Interactive Agent Trajectories

• Agentic OPD supervises decisions within interaction histories generated by student actions and environment transitions, not only unusual language prefixes.An action must be valid for the interface, appropriate to the current state, and useful for future progress.Source: §4.4 · p. 16
• Student-induced states reveal failures absent from successful demonstrations, but erroneous actions can create abnormal or irreversible states where teacher targets are poorly grounded.The challenge is deciding whether guidance remains meaningful, how much to trust it, and whether recovery should start from the current state or an earlier valid prefix.Source: §4.4 · pp. 16–17
• Supervision can operate at token, step, turn, or trajectory level, from local action syntax through the environmental consequences of long-horizon behavior.Temporal curricula can align shorter or earlier trajectory portions before exposing students to longer-horizon dependencies.Source: §4.4 · p. 17
• Selective intervention should assess whether teacher guidance improves the student’s future continuation, since disagreement may signal either a valuable correction or excessive drift.FutureBridge-OPD executes a short teacher bridge and validates the subsequent student continuation, serving both efficiency and supervision reliability.Source: §4.4 · p. 17
• Parametric distillation progresses from response and rationale outputs to offline trajectories and student-generated, environment-grounded histories; its frontier is identifying guidance that is valid, learnable, and consequential.The authors note that explicit experiences, procedures, and executable resources may be more transparent, editable, and economical when retained as reusable artifacts rather than parameters.Source: §4.4 · p. 17

5 Artifact Distillation: Externalizing Reusable Knowledge

• Artifact distillation externalizes teacher-derived knowledge as retrievable student-side objects, which become artifact knowledge when selected, validated, and preserved for reuse beyond the source task.A trajectory used only for fine-tuning is transfer evidence, whereas one retained for inference-time retrieval is an artifact.Source: §5 · p. 18

5.1 From Episodes to Distilled Memories

• Distilled memories selectively retain reusable episode knowledge, and demonstrations count as artifact knowledge only when retrieved in future inference rather than discarded after training.Teacher-derived memories can provide guidance appropriate to the current decision while preserving its provenance.Source: §5.1 · p. 19

5.2 Abstracting Reusable Procedures and Skills

• Reusable procedures abstract repeated trajectory structures into workflows while encoding applicability, steps, completion and recovery criteria, and provenance to reduce negative transfer from context-free advice.Source: §5.2 · p. 20
• Reusable skills abstract task executions into editable procedures, but fine-grained rules increase selection costs while monolithic skills can bind together steps that should vary.Procedure induction should retain transferable competence rather than realization-specific scaffolding, which may overconstrain the student.Source: §5.2 · pp. 20–21
• Because plausible procedures may be invalid, skills should be reapplied to source failures, nearby variants, and outside tasks, with counterexamples narrowing scope or preventing promotion.Validation asks both whether a skill repairs the motivating behavior and whether the repair transfers without unnecessary cost or suppressed exploration.Source: §5.2 · p. 21

5.3 Synthesizing Executable Tools and Programs

• Executable artifacts transfer the created resource itself—not merely tool-use knowledge—and can mediate capability across different models without access to their internal distributions.They must expose usable inputs and outputs, run in the student’s environment, and preserve intended behavior across repeated use.Source: §5.3 · p. 22
• Executable artifacts require construction and deployment checks, and their persistence makes defects consequential while changing APIs or schemas can make even correct artifacts stale.Specialization eases testing but narrows reuse; generality expands coverage while increasing interface and verification complexity, so accumulated libraries require maintenance.Source: §5.3 · pp. 22–23

5.4 Curating and Evolving Artifact Libraries

• Artifact libraries should not retain every candidate: curation must weigh enabled competence against storage, retrieval, context, and execution burdens, including redundancy, inconsistency, and risk.Individually useful artifacts may duplicate one another, prescribe incompatible actions, or compete for limited context.Source: §5.4 · p. 23
• Useful artifact entries expose task and domain, preconditions, schemas, dependencies, provenance, validation, versions, and failure cases so retrieval reflects applicability, not embedding similarity alone.Metadata also helps locate artifacts affected by environment changes without reexamining every trace.Source: §5.4 · p. 23
• Stored artifacts, descriptions, indices, and provenance belong to the artifact substrate; learned retrieval may reside in the model, while search and loading decisions belong to the harness.A retrieval failure need not mean the artifact is poor: the harness may fail to surface valid knowledge, while good retrieval cannot repair an overgeneralized artifact.Source: §5.4 · p. 23
• Artifact libraries should propose, test, revise, merge, or retire entries while balancing plasticity against stability and preserving meaningful exceptions.Consolidation can cluster episodes, generalize local patches, split artifacts with multimodal applicability, or record precedence between conflicting procedures.Source: §5.4 · pp. 23–24
• Rules preserving benchmark-specific successful behavior may transfer less reliably than patches removing unnecessary high-cost operations [123], so a consolidated library should preserve average benefit and the task, student, and cost distributions under which it was observed.Source: §5.4 · p. 23
• Validated instruction, tool, or skill packages can be inspected, edited, rolled back, and shared across model identities without full parameter retraining, but portability is conditional on shared reasoning ability, tokenizer conventions, API schemas, and environment assumptions.AgentDistill transfers teacher-generated MCP modules to smaller students [77], while multi-agent-to-single-agent distillation shows selected capability resources can survive substantial organizational change [112].Source: §5.4 · p. 24
• Artifact distillation progresses from retaining consequential episodes to abstracting reusable procedures and compiling executable resources, then validating and revising the library.Teacher outputs, traces, and feedback provide evidence; the persistent artifact substrate stores knowledge for future retrieval, interpretation, or invocation.Source: §5.4 · p. 24

6 Harness Distillation: Operationalizing Agent Knowledge

• Harness distillation transfers knowledge into persistent execution logic that shapes control decisions, feedback, and responses to outcomes, rather than merely storing reusable objects or teacher trajectories.A trajectory becomes harness knowledge when its organizational regularities are retained as control logic; a stored workflow or skill remains an artifact unless runtime mechanisms retrieve and execute it.Source: §6 · p. 24

6.1 From Demonstrations to Executable Control Flows

• Workflow representations determine which control structures can be expressed and checked, while excessive trace detail makes workflows brittle and omitted preconditions or causal dependencies can invalidate execution.A useful representation exposes operations, bindings, state variables, branching predicates, failure transitions, and stopping criteria, enabling evaluation of structural choices beyond matching the teacher’s surface actions.Source: §6.1 · p. 25
• Workflows distill teacher evidence into reusable control by mining recurring structure, abstracting operations, and searching for effective compositions, when retained in a student harness with identifiable teacher evidence.Trace abstraction separates reusable decisions from task-specific details; aligned successes reveal common subgoals, divergent successes interchangeable implementations, and failures missing guards or recovery branches.Source: §6.1 · pp. 25–26
• Fixed workflows can improve reproducibility and enable static checks, but require adaptation to student capabilities and may fail under changed interfaces, missing recovery behavior, or mismatched budgets.Distillation may map operations to student-accessible components, add adapters, or replace a monolithic step with a scaffolded subworkflow.Source: §6.1 · p. 26

6.2 Distilling Conditional Routing and Resource Allocation

• Conditional routing distills an allocation policy, not a lookup table of observed prompts, preserving the teacher’s allocation principle while adapting it to the student’s components and constraints.Teacher traces show decisions only for visited states, so routing must respond to execution state rather than simply replay observed prompt-to-resource choices.Source: §6.2 · p. 26
• Distilled routing policies determine when retrieval is needed, which artifact or tool fits current conditions, how to invoke it, and whether to combine candidates.Separating the persistent library from harness selection helps distinguish failures caused by missing artifacts, poor routing, or invalid invocation conditions.Source: §6.2 · p. 27
• Context policies select and order observations, artifacts, intermediate results, and messages to preserve decision-sufficient information within student context and latency constraints.Evaluation should intervene on included evidence and test downstream decisions, not rely on textual overlap with teacher prompts.Source: §6.2 · p. 27
• Harness planning distills conditions for decomposing, expanding, parallelizing, merging, postponing, or canceling work into an evolving resource-allocation policy.Teacher plans, timing, dependencies, and outcomes can reveal when concurrency conflicts or early stopping omits necessary work.Source: §6.2 · p. 27
• Capacity-aware harness distillation transfers abstract decision boundaries, then calibrates their implementation to the student’s available models, tools, permissions, context, and budget.When a preferred tool is unavailable, the harness should abstain, choose an allowed substitute, or request assistance rather than follow an incompatible teacher trace.Source: §6.2 · pp. 27–28

6.3 Operationalizing Verification, Feedback, and Recovery

• Harness control distillation learns verifier choice, revision and stopping decisions from teacher executions or outcomes; installing a hand-designed loop alone is engineering, not transfer.Verifier provenance and the risk context of its gates matter because tests and model-based critics have different coverage and trustworthiness.Source: §6.3 · p. 28
• The transferable core of corrective traces is often the intervention boundary: cues for accepting, inspecting, revising, or abandoning outputs, rather than full critiques.Positive and negative pairs and counterfactual queries can reveal which state changes alter the response, reducing context cost for weaker students.Source: §6.3 · p. 29
• Recovery policies handle stateful failures through retry limits, modified arguments, alternatives, checkpoints, restoration, compensation, or escalation, while checking postconditions before retrying.Teacher traces should record environment changes to distinguish harmless failed reads from partial writes, since replaying a successful action may be damaging.Source: §6.3 · p. 29
• Verification and recovery can cost more than the original task or suppress exploration, so scrutiny should reflect action risk and expected failure-loss reduction relative to verification cost.Teacher behavior can calibrate thresholds, but they need adjustment when student error rates or deployment consequences differ.Source: §6.3 · p. 29
• Evaluate utility alongside verification cost and residual risk, and test closed-loop transfer with held-out tasks, trace alignment, and interventions that perturb its mechanisms.Removing a verifier, corrupting a tool result, or injecting a recoverable failure should change behavior when the corresponding mechanism was transferred; black-box probes show functional sensitivity, while gray-box traces support stronger claims when control states are observable.Source: §6.3 · p. 29
• Harness distillation extends from closed-loop recovery to system-level orchestration, with the harness determining how models and artifacts become effective action.The persistent substrate is the harness; trajectories, messages, and outcomes are evidence of transferred knowledge.Source: §6.3 · p. 29, §6.4 · p. 31

6.4 Multi-Agent Orchestration and Adaptive Harnesses

• A transcript records one coordination instance; persistent multi-agent harness knowledge lies in rules for participation, information exchange, stopping, and handling disagreement.A student may preserve competence with a smaller graph by removing redundant communication, so evaluation should specify whether it targets topology reconstruction, functional replication, or cost-aware compression.Source: §6.4 · p. 30
• Agent optimization or graph pruning is not necessarily distillation; the transfer claim requires identifiable task-solving knowledge to persist in a student harness.Teacher-derived coordination may target agent utility, communication necessity, activation order, or early stopping.Source: §6.4 · p. 30
• Black-box recovery identifies an observational equivalence class, not a unique harness; structural claims require traces or interventions that distinguish candidate organizations.Studies should disclose their access model and use functional replication as the default black-box claim, reserving structural recovery for observed or causally identified mechanisms.Source: §6.4 · pp. 30–31

7 Cross-Substrate Distillation and Hybrid Realizations

• Cross-substrate paths are transformations between substrates, not a fourth substrate; a student is hybrid when knowledge persists in more than one destination.The source form affects observable evidence, while the transformation and final allocation affect potential information loss and deployment properties.Source: §7 · p. 31

7.1 Transfer Paths as Representation Translation

• Black-box input–output pairs support source-agnostic evidence, while source-specific attribution requires component access, controlled ablations, provenance logs, or interventions.The source–destination matrix is a conceptual accounting device: systems may contain multiple transfer paths, and the teacher agent may be the only identifiable source.Source: §7.1 · pp. 31–32
• Cross-substrate fidelity is behavioral and conditional, not representational identity: translation should preserve task-relevant decisions, dependencies, error responses, constraints, or outputs.For example, internalizing a debate can preserve comparing alternatives and revising errors without reproducing every message.Source: §7.1 · p. 32
• Substrate changes serve deployment needs: internalization can reduce retrieval latency, while artifacts improve editability and harness control can enforce ordering, but each has liabilities.Parameters can be difficult to audit or patch, artifacts impose selection and context costs, and harnesses can become brittle when interfaces change.Source: §7.1 · pp. 32–33

7.2 Crossing the Model Boundary: Internalization and Externalization

• Artifact-to-model internalization can remove runtime retrieval, but weakens the explicit link to source knowledge; MAGDi distills multi-model discussion graphs into a standalone model [12].Internalization may make common behavior more fluent, while updates become expensive, exact content may be imperfectly recalled, and artifact rules no longer guarantee compliance.Source: §7.2 · p. 33
• Interaction topology can expose rejected alternatives, reasoning changes, and dependencies that final answers omit, providing supervision even when the deployed student has no graph.The graph and multi-agent harness can serve as a training interface without being required for ordinary student inference.Source: §7.2 · pp. 33–34
• Harness-to-model transfer should internalize a scaffold’s functional contribution rather than copy every operation, since redundancy and teacher-specific coordination can waste capacity or entangle the student.A student may retain external support for decisions beyond its capacity, yielding a hybrid realization rather than a fully scaffold-free model.Source: §7.2 · p. 34
• Externalization retains student advantage in callable artifacts rather than copied weights, but artifacts must abstract across instances, specify applicability, expose an interface, and work for the student.Execution tests and student-in-the-loop repair help ensure teacher-authored tools remain usable under student and environment constraints.Source: §7.2 · p. 34
• Teacher-derived knowledge installed in a student controller supports model-to-harness distillation; merely searching for a high-performing architecture is more precisely agent design.The distinction preserves a boundary between transferring competence and optimizing system structure from scratch.Source: §7.2 · p. 34

7.3 Crystallizing and Compiling Operational Knowledge

• Compilation transforms declarative artifacts into persistent execution logic, whereas crystallization turns execution regularities into reusable, inspectable objects; substrate depends on use, not file format.Compilation must bind runtime state, map actions, order dependencies, evaluate guards, check conditions, and define responses to timeouts, refusals, and failures.Source: §7.3 · p. 35
• Compilation can distort soft or context-sensitive guidance into rigid control, while crystallization should retain the smallest resource that predicts beneficial action and is useful across future cases.Repeated success across heterogeneous states supports abstraction; failures reveal missing preconditions or exceptions, and compression quality depends on future utility and applicability calibration.Source: §7.3 · pp. 35–36
• Operational knowledge cycles from artifacts through harness execution and evidence back into revised artifacts, requiring provenance to prevent error amplification or task over-specialization.Revisions should record source episodes, validation scope, interface assumptions, and superseded versions.Source: §7.3 · p. 36

7.4 Hybrid Realizations and Substrate Allocation

• A student is hybrid only when teacher-derived knowledge persists in more than one substrate; discarded intermediate representations or unchanged components do not qualify.A teacher trajectory can update model parameters and install a recovery policy in the harness, while a multi-stage pipeline can end as a purely parametric student.Source: §7.4 · p. 36
• A tool-using deployment is not necessarily hybrid: it qualifies only when teacher-derived knowledge also changes retained resources or invocation policy.If tools and the agent loop remain unchanged while only the model learns from the teacher, the distilled destination is primarily the model.Source: §7.4 · p. 37
• Hybrid components co-adapt: schemas, routers, and workflows may fail across updates or student models, so interfaces need explicit contracts and deployment-matched validation.Contracts should cover inputs, outputs, state, failure signals, permissions, latency, and version compatibility.Source: §7.4 · p. 37
• Allocation should favor the model for stable, frequent, latency-sensitive regularities; artifacts for explicit or changing knowledge; and the harness for conditional, safety-critical, or resource-allocation logic.These are priors, not fixed rules; the best choice depends on student capacity, access, workload, and update horizon.Source: §7.4 · p. 37
• Evaluation should equalize inference budgets, ablate changed substrates, test compatibility, distinguish synergy from duplication, and report migration and maintenance costs alongside fidelity.Cross-substrate distillation is a design problem over where knowledge should live, while hybrid realization describes the chosen endpoint.Source: §7.4 · p. 37

• Evidence is indirect, exposed through responses, artifacts, execution traces, environment states, and controlled interventions selected for the claimed destination.Source: §8 · p. 38
• Substrate-specific tests examine unsupported model behavior, persistent artifact validity and utility, harness changes, and attribution in hybrid transfers.Parametric transfer must persist without teacher-side scaffolding; artifact and harness transfer require persistent resources and execution changes, respectively.Source: §8 · p. 38
• A high end-task score establishes system utility but does not identify what was transferred or where it was retained.Source: §8 · p. 38
• Complete evaluation tests acquisition, on-policy utility, fidelity and generalization, and deployment quality through local diagnostics and system-level outcomes.Source: §8 · p. 38
• The evaluation section maps transfer claims to metrics, specifies controlled evidence-producing protocols, and organizes benchmarks by the deepest transfer claim they support.Source: §8 · p. 38

8.1.1 From Transfer Claims to Observable Evidence.

• A persuasive transfer evaluation links intrinsic destination validity, causal contribution, and end-to-end utility, choosing observables and interventions that justify the claim while reporting deployed utility.A valid component may go unused, a causally active component may reduce performance, and an end-to-end gain may instead reflect an uncontrolled prompt or harness change.Source: §8.1.1 · pp. 38–39

8.1.2 Common Agent-Level Outcomes.

• For executable tasks, evaluate postconditions rather than matching one teacher trace, and supplement binary success with progress, recovery, and repeated-trial measures of reliability.Partial credit can reflect satisfied subgoals or milestones; repeated on-policy trials expose brittle systems whose average success hides frequent failures.Source: §8.1.2 · p. 39
• Distillation gains should be compared with both the teacher and undistilled student, while reporting raw scores, supervision and deployment costs, and utility–cost tradeoffs.Normalized teacher–student gap closure is informative when its denominator is not near zero; raw scores remain necessary when the ratio is unstable.Source: §8.1.2 · p. 39

8.1.3 Evaluating Parametric Destinations.

• To test parametric transfer, hold artifacts and harness fixed, remove training evidence unless it is part of the endpoint, and measure held-out behavior; similarity metrics diagnose fidelity, not competence.Distribution or representation measures support a fidelity claim only alongside held-out behavior and, where feasible, interventions or ablations showing that the aligned component contributes.Source: §8.1.3 · p. 40

8.1.4 Evaluating Artifact Destinations.

• Artifact evaluation follows a chain of intrinsic validity, access, and utility, using matched paired executions with and without the artifact to estimate its marginal effect.Matching tasks, seeds, contexts, and budgets helps distinguish artifact knowledge from stronger routing or additional computation.Source: §8.1.4 · p. 40
• Memory evaluation should cover writing through deletion, operational cost, and privacy risks, since retrieval access alone does not establish utility or attribution.Measures include retention of sensitive items, unauthorized retrieval, cross-user leakage, and deletion compliance.Source: §8.1.4 · p. 40
• Reusable skills require evaluation of acquisition and repeated use, including executability, composition, transfer, portability, and negative transfer rather than extraction-episode performance alone.Matched tasks with deterministic verifiers can estimate an artifact’s marginal effect; portability tests preserve the artifact while swapping the student model or harness.Source: §8.1.4 · p. 41
• Artifact maintenance properties are part of the claimed benefit of externalizing knowledge, alongside whether corrections and revocations behave as intended.Evaluation should assess edit locality, update cost, version compatibility, provenance completeness, and regression after revision; corrected artifacts should improve targeted cases without harming unrelated tasks, and revoked artifacts should no longer be retrievable or executable.Source: §8.1.4 · p. 41

8.1.5 Evaluating Harness Destinations.

• Harness evaluation must inspect runtime control consequences, comparing teacher and student execution under matched states and perturbed observations rather than relying on file format or graph similarity.Compare final states, branch choices, recovery, and resource use; structural fidelity measures do not establish equivalent behavior, and black-box agreement supports functional replication only.Source: §8.1.5 · p. 41
• Harness effects require crossed evaluation of multiple models and harnesses under shared conditions, since interaction terms can reveal that a harness helps some models but hinders others.Harness-Bench compares systems across shared sandboxed tasks, while Exgentic provides a protocol for evaluating agent architectures across models and environments; neither resource alone proves distillation from an identifiable teacher source.Source: §8.1.5 · p. 41
• Multi-agent harness evaluation should measure coordination outcomes and specify whether the goal is topology reconstruction, functional replication, or cost-aware compression.Relevant measures include milestone completion, communication cost, handoff completeness, verification coverage, and termination correctness; a smaller team need not reproduce teacher topology if it preserves coordination function at lower cost.Source: §8.1.5 · pp. 41–42

8.1.6 Cross-Substrate and Hybrid Attribution.

• Cross-substrate claims should report the claimed transfer path and final destinations, then test each changed component’s causal effect on future episodes.Removal and swap interventions distinguish retained knowledge from interface co-adaptation when budgets and interfaces remain compatible; hybrid evaluation should establish multiple destinations’ contributions and account for interaction and lifecycle costs.Source: §8.1.6 · p. 42

8.2.1 Target Declaration and Controlled Baselines.

• A transfer report declares teacher access, inferred source substrate, persistent student changes, and deployment requirements to classify the transfer claim.It also separates temporary training-time objects from changes that remain at deployment.Source: §8.2.1 · pp. 42–43
• Controls should match model size, supervision, prompts, context, tools, action budgets, samples, and verifier calls, or compare utility across a budget curve.The teacher is a reference, not an assumed upper bound; a smaller student may outperform it through additional supervision, regularization, or better deployment components.Source: §8.2.1 · p. 43
• Dataset records should document provenance and generation, while splits must prevent leakage at the unit of claimed generalization.For dynamic environments, use versioned snapshots or temporal splits.Source: §8.2.1 · p. 43

8.2.2 Offline Fidelity, On-Policy Execution, and Interventions.

• Offline replay measures local fidelity, on-policy runs establish deployed competence, and claim-specific counterfactual interventions test whether transferred knowledge controls behavior.Offline action accuracy can coexist with poor closed-loop performance when student deviations lead to unseen states or compound.Source: §8.2.2 · p. 43

8.2.3 Evaluator Hierarchy and Failure Annotation.

• Evaluation combines deterministic, reference-based, model-based, and human judgments, while failure annotation records the first consequential error and terminal outcome.LLM judges and process reward models require calibration; teacher-generated rubrics require disclosure because they may favor the teacher’s style or trajectory.Source: §8.2.3 · pp. 43–44

8.2.4 Statistical, Generalization, and Safety Reporting.

• Reports should quantify uncertainty, test the promised generalization shifts, and assess robustness, safety, and privacy alongside task utility.The final result is a vector spanning task utility, fidelity and validity, causal contribution, efficiency, reliability, OOD retention, robustness, safety, privacy, and maintenance cost; a scalar leaderboard score is meaningful only under fixed weights and protocol.Source: §8.2.4 · p. 44

8.3 Benchmark Landscape and Coverage

• Benchmark scores depend on the environment version, action budget, harness, and evaluator, and availability alone does not identify the source or destination substrate of a gain.Table 3 organizes representative benchmark families by primary observable, strongest supported claim, and principal limitation.Source: §8.3 · p. 45, Table 3 · p. 45

8.3.1 Static and Interactive Evidence.

• Static resources support output, rationale, verifier, and local-action fidelity, but fixed observations cannot reveal post-error states or establish that written reasoning caused an answer.Claims about interactive policy, recovery, or harness behavior require on-policy execution.Source: §8.3.1 · p. 46
• Interactive web, desktop, software, and application benchmarks expose executable consequences and interface sensitivity, but environmental and harness variation can change scores without changing model weights.Studies should pin environment and harness versions, preserve logs, and use state-based evaluators.Source: §8.3.1 · p. 46

8.3.2 Artifact, Workflow, and Harness Evidence.

• Artifact, workflow, and harness benchmarks assess resources, coordination, and model–harness interaction, but do not alone establish teacher-derived knowledge or identify substrate attribution.Distillation studies need provenance and pre/post-transfer controls; artifact gains also require controlling the selecting and executing harness or evaluating it as a second destination.Source: §8.3.2 · p. 46

8.3.3 Safety Coverage and Benchmark Portfolios.

• Safety benchmarks test harmful task completion and attack surfaces; report benign utility alongside security because low attack success can result from refusing benign tasks.No individual benchmark observes every transfer object, so portfolios should match benchmark settings and interventions to the claimed substrate.Source: §8.3.3 · pp. 46–47
• A benchmark portfolio should trace intrinsic destination quality through causal contribution to deployed agent behavior, while preserving distinctions between evidence, knowledge, local fidelity, task utility, and substrate allocation.Few benchmarks jointly provide teacher provenance, editable artifacts, controllable harnesses, stateful execution, and lifecycle costs.Source: §8.3.3 · p. 47

9 Open Challenges and Future Directions

• The section frames five research directions around attribution, environmental reproducibility, joint substrate design, lifecycle governance, and shared foundations.These directions follow from the need to determine where gains reside, whether they survive deployment shifts, and how retained knowledge can be revised or removed.Source: §9 · p. 47

9.1 Identifying What Was Transferred

• Successful end-to-end behavior can have several explanations because models, artifacts, and harnesses can compensate for one another.The authors call for causal attribution using preserved pre-transfer states and component interventions; without access to these, claims should be limited to behavioral transfer or destination attribution.Source: §9.1 · p. 47

9.2 Reproducibility under Environmental Change

• Infrastructure changes can make a component appear better or worse even when its retained knowledge is unchanged.The authors recommend versioned environment snapshots and controlled shifts, with logs that distinguish agent errors from infrastructure failures.Source: §9.2 · pp. 47–48

9.3 Joint Design across Knowledge Substrates

• The appropriate allocation of knowledge across models, artifacts, and harnesses depends on acquisition and deployment costs, portability, update frequency, safety, and maintenance.Frequently reused, latency-sensitive behavior may favor parametric retention; changing content may favor editable artifacts, while constraints and verification may favor the harness.Source: §9.3 · p. 48

9.4 Lifecycle, Safety, and Governance

• Distilled knowledge needs lifecycle controls for provenance, validity, updates, deletion, rollback, and safety.The section warns that transfer can retain hard-to-remove behavior, poisoned artifacts, or over-broad permissions, and calls for policies to merge, revise, or discard knowledge.Source: §9.4 · p. 48

9.5 Shared Foundations for Cumulative Progress

• Shared abstractions, evaluation stacks, and reporting standards are needed to compare heterogeneous realizations and make results cumulative.The proposed infrastructure includes interchangeable models, versioned artifacts, inspectable harnesses, sandboxed environments, and component-level interventions.Source: §9.5 · pp. 48–49

10 Conclusion

• Agent Distillation transfers task-solving knowledge from teacher-agent evidence to a student agent for persistent use in future episodes.This distinguishes it from prompting, inference-time consultation, and collaboration whose effects disappear with the interaction.Source: §10 · p. 49
• The taxonomy distinguishes parametric, artifact, and harness destinations, with cross-substrate transfer and hybrid realization covering changes between and persistence across destinations.It separates teacher evidence and transfer mechanisms from the destination substrates.Source: §10 · p. 49
• End-task success is insufficient evidence; evaluation should connect destination quality, causal component interventions, and on-policy system utility.The framework also accounts for reliability, generalization, efficiency, safety, and lifecycle cost.Source: §10 · p. 49
• The goal is a student that retains appropriate knowledge in appropriate substrates under deployment constraints, not merely a smaller model imitating a stronger one.The conclusion presents Agent Distillation as a systems discipline connecting knowledge transfer, experience reuse, and agent architecture design.Source: §10 · p. 49