DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning

Suyog Khanal; Arun Kumar A; Santu Rana

Source-linked AI summary

DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning

Suyog Khanal, Arun Kumar A, Santu Rana

arXiv:2609.08123v1Sep 8, 2026cs.ROcs.AIcs.LG

TL;DR

Interactive imitation learning still leaves limited budgets to chance by inheriting which failure is corrected and where its demonstration starts. DISEIL explicitly selects a recurring failure mode and feasible starting configuration from failure evidence, and it achieves the highest mean held-out success rate across all 10 evaluated settings, tied once, within a simulated single-round scope.

Problem
Interactive imitation learning decides when to query an expert but usually leaves which failure to correct and where the demonstration starts to the triggering episode, despite demonstrations being scarce.
Method
DISEIL marks failures at their first sustained unreliability, describes and geometrically groups them into modes, selects a target mode and start configuration, and checks the resulting request against task constraints.
Results
DISEIL achieved the highest mean held-out success rate in all 10 settings, with a tie in 1, and its mean margin over the strongest comparison was 2.80 percentage points.
Takeaways & Limitations
Under the evaluated fixed query rule and simulated settings, deliberately choosing the failure mode and demonstration start improved held-out success rate over the comparisons.
Takeaways & Limitations
The evidence is limited to simulation, a single round of failures, and mostly scripted experts, while physical-robot perception and cost-aware human teaching remain untested.

Abstract

from arXiv · show

A robot that can be taught a new task from a handful of demonstrations has to work out for itself what it still cannot do, and then ask for exactly that. Interactive imitation learning takes a step in that direction by letting a policy practice on its own and calling an expert when it goes wrong. Existing methods decide when to interrupt the learner. A further 2 decisions are left to whichever episode happened to trigger the interruption: which failure to correct, and where the demonstration should start. This paper is a first attempt at making both of them deliberately. DISEIL (Demonstration dIstillation for Sample-Efficient Imitation Learning) marks each failed episode at the step where the policy first becomes unreliable, represents that moment with a geometric descriptor, and groups the failures into recurring failure modes. A vision-language model and a language model read the selected mode and write a request for the next demonstration, and a store of task constraints checks that the request can be carried out before any expert time is spent. No model produces a robot action. Across 5 simulated tasks under state and image observations, changing only what the expert is asked for gives the highest mean held-out success rate in all 10 settings, with a tie in 1, and the margin is widest at the smallest budget we tested. The scope is narrow: a single round of practice at a time, in simulation, with experts that are mostly scripted. The longer-term aim is a learner that also tracks what its demonstration set already covers, and that asks a human teacher for the missing behavior in proportion to the effort each request costs them.

1 Introduction

Interactive imitation learning reduces covariate shift but leaves limited demonstration budgets with unresolved choices about which failure to correct and where to begin. DISEIL makes those choices explicit by grouping failures, selecting a recurring weakness, and specifying a feasible starting configuration before expert time is spent.

Interactive imitation learning lets experts correct states the learner actually reaches, but demonstrations remain the scarcest resource.
Existing methods decide when to call the expert, while the triggering episode usually determines both the failure corrected and the demonstration’s starting state.
With small budgets, state-level scoring can repeatedly spend demonstrations on one weakness while leaving other recurring failures untouched.
Starting at the triggering state may teach recovery after accumulated effects rather than avoidance at the point where failure begins.
DISEIL writes failures explicitly, groups recurring modes, selects a mode, and chooses the demonstration start before the expert is called.
The geometric component handles failure geometry while language models describe causes and proposals, with feasibility checked through task constraints.
Across 5 tasks and 2 observation types, DISEIL achieved the highest mean held-out success rate in all 10 settings, with a tie in 1.

2 Background

Prior work mainly selects when to query or chooses among existing data, whereas DISEIL specifies new supervision from observed failures and checks the request symbolically. Its neural components interpret failures, while geometric and constraint-based components handle structure and feasibility.

Interactive imitation-learning baselines differ primarily in the signal used to decide when control passes to the expert.
Active learning, core-set selection, retrieval, and dataset distillation choose or compress data that already exists.
Demonstration distillation instead turns evidence from failed episodes into a specification for supervision that has not yet been collected.
DISEIL uses language models to explain failures, while a robot knowledge base stores workspace, spawn, reachability, and success constraints for feasibility checks.

3 Demonstration distillation

DISEIL treats acquisition as choosing when a query fires, which failure mode to supervise, and where the demonstration starts. It marks failures, describes and geometrically groups them, prioritizes a mode, generates a request, and validates that request against constraints.

A policy is trained on an initial dataset, deployed from fresh resets, and retrained after each round’s expert demonstrations until the budget is exhausted.
The acquisition decision is the triple (t⋆, Ctgt, ξ): query time, target failure mode, and demonstration-start configuration.
DISEIL’s four stages are Perceive, Partition, Prioritize, and Prescribe; VLM and prescription models annotate failures but do not perform grouping.
Perceive: Failures are marked at the first sustained loss-threshold crossing, placing the anchor near failure onset rather than after its effects accumulate.
Perceive: Each descriptor records object position and yaw, failure progress, and end-effector distance, while yaw uses sine and cosine to avoid angular discontinuities.
Partition: When enough failures exist, standardized descriptors are clustered with an agglomerative method and the number of modes is selected by silhouette score.
Prioritize: Modes retain size, centroid, mean peak loss, and a representative failure; prioritization combines prevalence and severity with memory of previously corrected modes.
Prescribe: Prescribe returns either a targeted correction from a marked state or a bridging placement constructed from related failures, while rejected proposals cost no expert time.

4 A controlled study

The controlled study changes only how demonstrations are selected, evaluating DISEIL across 10 simulated task-observation settings. DISEIL achieves the highest mean held-out success rate in every setting, while its advantage is largest at the smallest budget and its allocation preserves informative, diverse corrections.

All methods use the same policy, starting demonstrations, expert, retraining schedule, budget, and held-out evaluation set; only demonstration selection changes.
DISEIL achieves the highest mean final success rate in all 10 settings, tying on Lift with state observations.Its mean margin over the strongest comparison is 2.80 percentage points, ranging from 0.0 to 5.6.
9.07 points is DISEIL’s mean margin at B = 10, compared with 2.87 points at B = 20 and 2.83 points at B = 40.The comparison covers GridWorld with images, Push-T with state observations, and Door with images.
DISEIL acquires demonstrations of higher per-demonstration goodness than Diff-DAgger in all 8 robot settings.Goodness is the policy’s per-step loss before retraining, and the result indicates less repeated correction of similar failures.
Geometry-based clusters show behaviorally distinct Push-T failures, but their labels come from the naming pipeline rather than independent annotation.Figure 5 shows three sampled members per cluster; the figure supports interpretability rather than ground-truth cluster correctness.
Replacing mode-based allocation with greedy highest-loss selection reduces mean performance by 4.37 points, the largest single knockout effect.Supporting component knockouts range from 0.73 to 2.37 points, while per-demonstration goodness changes by only +0.02.

5 What the study does and does not show

The study supports DISEIL’s allocation choices only within a controlled simulation setting. Its main boundaries concern privileged geometric information, nonhuman experts and demonstration-count budgeting, reasoning cost, and incomplete representation of prior coverage.

The evidence supports choosing failure modes and starting configurations to raise held-out success on the 10 evaluated settings, not claims beyond that scope.
The geometric descriptor uses simulator-supplied object pose and end-effector geometry, so physical deployment would require perception and calibrated sensing.The evaluation does not establish how estimation error would affect the partition.
Most experts are learned policies or scripted oracles rather than people, so the study does not exercise human teaching noise, fatigue, or variable skill.The budget counts demonstrations rather than expert-controlled steps, and the cost of longer prescribed demonstrations remains unexplored.
63 to 1,232 seconds and up to about 11,000 tokens are added by reasoning per acquisition round across the instrumented settings.On the three RoboSuite settings, this is smaller than the 783-to-1,491-second retraining-and-evaluation cost, but its value depends on demonstration cost.
DISEIL’s cluster memory records requested corrections rather than the behaviors already contained in collected demonstrations.It therefore cannot distinguish a missing behavior from repeated failure on behavior that was demonstrated but not learned.

6 Conclusion

DISEIL makes failure-mode and starting-configuration selection explicit within a fixed demonstration budget. The authors identify simulation transfer, single-round coverage tracking, and cost-aware human teaching as the main directions needed beyond this study.

DISEIL groups failures geometrically, selects a recurring mode using prevalence and severity, and has task constraints accept the resulting demonstration request.
The study’s three main limitations are simulation-only evaluation, a local single-round view of failures, and treating human demonstrations as equal budget units.Future work must estimate descriptors on physical robots, track collected coverage, and model teacher time as acquisition cost.

A Implementation and Hyperparameters

DISEIL runs a four-stage acquisition loop around a standard interactive imitation-learning harness, with fixed retraining and feasibility-related hyperparameters. Experiments are calibrated to a meaningful starting-competence band and use local language models alongside policy training infrastructure.

Implementation: DISEIL repeats perception, partitioning, prioritization, and prescription within a one-demonstration-per-round loop.The loop rolls out the policy, constructs failure descriptors, annotates failures, clusters when enough failures exist, selects a target, generates or falls back to a feasible request, and retrains.
Hyperparameters: The framework uses a five-attempt re-prescription limit and a context-set cap of 3 in the main Qwen3-32B experiments.A12 supports the context cap, while the re-prescription limit bounds the feasibility loop and can trigger deterministic fallback.
Implementation: Policies are retrained from scratch after every acquired demonstration, with m = 1 throughout all arms.This gives 20 fresh retraining rounds over a budget of 20 and keeps the schedule identical across comparisons.
Calibration: The initial demonstration count is selected by a behavior-cloning scaling sweep so round-zero success lies in a meaningful intermediate band.The resulting round-zero success rates span 43.1 to 67.2 percent, with counts chosen per task rather than per modality.
Compute: Language-model components run locally with Qwen3-VL-32B and Qwen3-32B under 4-bit quantization, while policies use multiple GPU classes.The language models are served with vLLM on three 80GB H100 GPUs; policy training and evaluation use A100, H100, H200, V100/V100L, or L40S GPUs.

B Tasks, Experts and the Geometric Descriptor

The evaluation spans five simulated tasks, their task-specific experts, and two observation modalities. DISEIL uses a six-dimensional geometric descriptor computed from privileged state at the flagged failure step, while model components handle annotation and prescription rather than grouping.

Tasks: The benchmark covers GridWorld, Push-T, Lift, Wipe, and Door under state and image observations.GridWorld is discrete navigation; Push-T is planar manipulation; Lift, Wipe, and Door are UR5/UR5e tasks in simulation.
Experts: Experts are human on GridWorld, scripted oracles on Lift, Wipe, and Door, and a PPO-trained policy on Push-T.The task-specific expert choices support the paper’s claim that demonstrations are correct by construction within the evaluated settings.
Geometric descriptor: The six-dimensional descriptor is identical across a task’s two modalities and uses privileged robot and object state at the flagged step.No foundation-model output enters the descriptor, and object configuration—not end-effector configuration—determines recurrence where an object exists.
Failure anchoring: The first sustained loss crossing anchors failures near the onset of unreliability rather than at a later peak after errors compound.The peak is used only when no qualifying crossing occurs; the crossing requires loss above η for K consecutive steps.
Failure modes: Clustering uses a silhouette-selected k between 2 and 6, but switches to singleton modes when fewer than four failures remain.The cap reflects small failure sets, and the fallback rule handles rounds where the cluster sweep is skipped.
Targeting and prescription: The target mode is selected by severity subject to a size constraint, while feasibility and solvability screens prevent unusable or already-solved prescriptions.The size constraint avoids letting an isolated high-loss failure dominate, and the constraint store checks reachability, spawn, workspace, and success conditions.

E The Ablation program: Scope and Conventions

The ablation program contains component knockouts and diagnostics evaluated across three representative settings. Results are reported per setting and as means over the three settings, with the setting—not an individual trial—as the unit of analysis.

Study structure: The program comprises 17 studies: A1–A12 vary or remove one component, while A13–A17 diagnose running-system properties.A12b additionally tests whether the context-set-size result transfers across prescription models.
Evaluation settings: The three ablation settings are GridWorld with image observations, Push-T with state observations, and Door with image observations.They span three policy classes and both observation modalities; every aggregate is the mean across these settings.
Reporting convention: Each study reports three per-setting values, their shared sign, and the mean across settings.This convention defines the setting as the unit of analysis rather than aggregating over individual runs or tasks without labeling.

F Knockouts, A1 to A8

The knockouts identify geometric clustering as the most consequential allocation component and show that structured failure-mode allocation matters beyond selecting individually high-loss failures. Other components refine targeting or feasibility, while DISEIL and its reduced geometric variant outperform the comparison controls in the reported settings.

Allocation controls: DISEIL reaches 91.3, 96.1, and 88.6 final success rates versus 89.1, 82.3, and 80.0 for uniform-random allocation.The values correspond to GridWorld, Push-T, and Door, respectively; random allocation also trails the strongest uncertainty-gated baseline on both robot settings.
Fallback allocation: The deterministic fallback reaches 89.5, 92.5, and 84.2 final success rates across the three settings but lacks explicit mode severity and correction history.It preserves geometric diversity by avoiding repeated failures, yet does not recover the full allocation-and-prescription benefit.
Clustering knockout: Removing clustering reduces success by 2.2, 4.1, and 6.8 points, a mean of 4.37, with mean margin retained of −53.2 percent.On Push-T and Door, the ablated system falls below its own best baseline.
Mechanism: Clustering removal leaves mean per-demonstration goodness nearly unchanged at +0.02 while lowering final success by 4.37 percentage points on average.Greedy worst-loss selection acquires high-loss demonstrations effectively, but structured allocation distributes supervision across recurring failure modes more effectively under a fixed budget.
Model roles: A4 and A5 each cost a mean 1.33 points, retaining 48.6 and 47.0 percent of the margin, respectively.Their similar effects do not support ranking the prescription and vision-language components; both effects are largest on Push-T and smallest on GridWorld.
Reduced variant: The reduced variant retaining geometric clustering, memory, and a deterministic representative heuristic continues to outperform every evaluated baseline.The result supports a modular interpretation in which geometric allocation selects the region and model components refine its correction.
Supporting components: Disabling bridging costs 1.27 points on average, while disabling cluster memory costs 0.73 points on average.Bridging changes the environment configuration to make prescriptions easier, whereas memory suppresses recently supervised clusters and redirects later rounds.
Knockout ranking: The ranked mean damages are 4.37 points for clustering, 3.27 for fallback-only allocation, 2.37 for the constraint store, 1.33 for each model role, 1.27 for bridging, and 0.73 for memory.The ordering identifies clustering as the largest knockout and places feasibility and model components below allocation structure in mean damage.

G Design Choices, A9 to A12

Studies A9–A12 evaluate how DISEIL represents failures, selects clusters, and supplies context to prescription models. The results favor adaptive clustering and a three-episode context set for the Qwen3-32B main configuration, while showing that context-size effects depend on the prescription model.

A10, descriptor width: The 6-dimensional descriptor is the highest-scoring variant in all 3 settings, with orientation providing the largest single silhouette gain of +0.133.The measured silhouette curve is an inverted U; adding task progress and contact distance yields smaller gains of +0.050 and +0.037.
A9, context composition: The three context-set rules together outperform random selection by 3.6 points, because they combine target-mode representation, loss severity, and diversity.The forced representative, worst-loss seed, and diversity fill each contribute to the prescribed context set.
A12, citation count: At fixed context size, the three-rule construction outperforms peak-loss ranking by 2.13 points, while citing 2 episodes costs 1.93 points and citing 5 adds no measurable gain with Qwen3-32B.These results support a cap of 3 cited episodes for the main Qwen3-32B experiments.
A12b, model sensitivity: Increasing κ from 3 to 5 improves Sonnet 5 by 5.75 points but changes Qwen3-32B by −0.95 points across the 2 tested settings.The comparison indicates a model-by-context-set-size interaction rather than a model-independent optimum.
A11, cluster count: Silhouette-selected clustering beats fixed k alternatives on all 3 settings, with fixed k = 2 costing 4.1 points and fixed k = 5 costing 6.7 points.No single fixed cluster count is best across settings, supporting round-adaptive selection.

H Diagnostics, A13 to A16

Diagnostics characterize how DISEIL’s geometric clusters correspond to semantic failure modes and how its allocation machinery behaves as failures decline. The adaptive system uses varied cluster counts, sometimes bypasses clustering for small failure sets, and is most active early in the budget.

A13, semantic consistency: Geometric separation and semantic purity rise and fall together across the 3 settings: Push-T has silhouette 0.64 and purity 0.91, while Door has 0.56 and 0.84.Door also has the most root causes per cluster at 1.86, compared with 1.35–1.86 across settings.
A14, cluster-count distribution: Across 308 clustered rounds, k = 3 is selected most often at 26.3%, followed by k = 4 at 23.4% and k = 5 at 21.4%.Every candidate count from 2 through 6 occurs in at least 14% of clustered rounds.
A14, small-set transition: Clustering is skipped when fewer than 4 failures remain in 15%–21% of rounds, with each remaining failure becoming its own cluster under the fallback rule.The reported rates are 21% for GridWorld, 15% for Push-T, and 20% for Door.
A15, bridged share: Bridging is used in 24% of accepted GridWorld prescriptions, 28% on Push-T, and 21% on Door.The figure links bridging frequency with its measured success-rate contribution in the bridging knockout study.
A16, failures per round: Failures per round fall from 42 to 2 over the budget, halving by round 8 and declining by an order of magnitude by round 17.The final 3 rounds use the fallback rule on a small number of remaining failures, and the allocation machinery is most active early.

I Computational Cost, A17

Study A17 separates DISEIL’s shared training cost from its selection-time reasoning cost using matched runs against SafeDAgger. The added compute is paid during demonstration selection, while its practical value depends on the cost of expert supervision.

Measurement: DISEIL’s selection pipeline adds 270–700 seconds to a 783–1,491-second shared round cost under P1 on the 3 RoboSuite settings.Under P5, the shared cost is 548–1,333 seconds and the add-on is 233–656 seconds.
Setting differences: Push-T has the largest measured add-on, partly because SafeDAgger stops after one intervened episode while DISEIL screens a fixed 60.Push-T also issues 20 language-model calls, compared with 7 on Door.
Framework implication: DISEIL’s models run only during demonstration selection, so their inference cost is paid once per round and is amortized over retraining and evaluation on the measured RoboSuite settings.The framework trades selection-time inference against expert demonstration time; the reduced-compute heuristic costs 1.33 average success-rate points while remaining above evaluated baselines.

J Prescription Confidence

Prescription confidence is recorded before the demonstration and rollout outcome, then compared with the resulting change in policy success. Across DISEIL runs, it is strongly positively associated with subsequent improvement, but it is used diagnostically rather than to gate demonstrations.

The prescription model emits an integer confidence from 0 to 100 with a rationale estimating whether the resulting demonstration will improve the policy.The score is evaluated against ΔSR, the round-level change in success rate.
0.82 to 0.89 Pearson correlation links reported prescription confidence with subsequent policy improvement across the 10 settings.GridWorld (image) has r = 0.82 over 180 prescriptions.
Confidence is reported before the demonstration is collected, the policy is retrained, or the rollout evaluation is run.The outcome signal arrives only after the round’s budget unit has been spent.
The confidence score is a diagnostic on DISEIL runs, not a confidence-based rule for skipping, deferring, or re-prescribing demonstrations.The reported association therefore does not attribute policy improvement to confidence-based gating.