Source-linked AI summary
DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents
Hangrui Xu, Jiarui Wang, Yang Yang, Chuanbo Zhu, Fangda Chen, Ziqi Wu, Jingming Cai, Yan Song
TL;DR
Full-trajectory imitation can mishandle the diamond-shaped alternatives in multi-turn tool use. DART-SD uses interaction-state topology to identify breakpoints and locally supervise recovery, consistently outperforming distillation and reinforcement-learning baselines across in-domain and out-of-domain benchmarks while reducing redundant tool calls.
Problem
Full-trajectory imitation treats order-independent multi-turn tool interactions as rigid linear sequences, limiting effective agent distillation.
Method
DART-SD builds an Interaction-State Transition Graph, identifies Critical Topological Breakpoints, retrieves recovery references, and locally supervises recovery while preserving valid prefixes.
Results
DART-SD consistently outperforms supervised and reinforcement-learning baselines across five in-domain and out-of-domain benchmarks while reducing redundant tool calls and extending valid interaction prefixes.
Takeaways & Limitations
The findings support guiding agent distillation by interaction-state topology rather than rigid trajectory imitation.
Takeaways & Limitations
ISTG judgment errors may omit informative atoms, leaving the graph sparser, and experiments use a no-thinking configuration.
Abstract
from arXiv · showhide
Equipping Large Language Models (LLMs) with multi-turn tool-calling capabilities is essential for building autonomous agents. However, progress is fundamentally limited by the reliance on full-length trajectory imitation. For tasks involving multiple order-independent sub-goals, the optimal solution space forms a vast combinatorial diamond lattice. Forcing this rich topology into monolithic trajectories causes a severe topological collapse, indiscriminately penalizing valid alternative explorations and severely degrading policy diversity. To address this, we propose DART-SD (Diamond-topology Aware Retrieval and Tuning for Self-Distillation), a novel framework that shifts the paradigm from global forcing to topology-guided localized correction. DART-SD first models the execution process as a converging Interaction-State Transition Graph (ISTG), faithfully capturing the inherent diamond topology of successful and failed exploratory paths. During autonomous rollouts, the framework identifies the Critical Topological Breakpoint (CTB) and retrieves success-supported recovery references. Finally, we introduce a progressive self-distillation paradigm through CTB-guided localized supervision, ensuring that the training loss is calculated exclusively on the generated recovery steps while strictly protecting the valid reasoning prefix from destructive gradient updates. Experiments on complex multi-turn tool-calling benchmarks demonstrate that DART-SD significantly outperforms traditional full-trajectory baselines.
1 Introduction
DART-SD addresses the topological collapse caused by full-trajectory imitation and uniformly distributed reinforcement-learning credit in multi-turn tool-calling agents. It models execution as a diamond-topology Interaction-State Transition Graph and applies CTB-guided localized self-distillation to correct recovery steps while preserving valid prefixes.
- Motivation: Multi-turn tool-calling equips LLMs to retrieve current information, execute code, and manipulate APIs for complex real-world tasks.These capabilities extend beyond static parametric knowledge through sequential environment interactions.
- Limitations: Full-trajectory imitation applies indiscriminate global loss, while standard reinforcement learning spreads rewards uniformly and misassigns credit across intermediate tool calls.These paradigms can overwrite valid exploration and penalize valid steps in failed trajectories.
- Topological formulation: Order-independent sub-goals create intersecting trajectories and a combinatorial diamond lattice that linear-trajectory training fails to represent.DART-SD addresses this mismatch with an Interaction-State Transition Graph built from cumulative interaction states.
- DART-SD: DART-SD identifies Critical Topological Breakpoints by projecting student states onto success-reachable regions, retrieves success-supported recovery references, and supervises only generated recovery steps.The framework strictly preserves valid reasoning prefixes and progressively rolls out the student to extend its capability boundary.
- Results: Extensive experiments across five benchmarks and two model scales show that DART-SD consistently outperforms supervised and reinforcement-learning baselines.The method also reduces redundant tool calls and extends valid interaction prefixes.
2 Related Work
Prior work includes full-trajectory behavior cloning and supervised fine-tuning, dynamic student-centered distillation, and reinforcement-learning methods using environmental feedback. These paradigms face compounding errors, exposure bias, overwritten exploration habits, or credit misassignment from sparse rewards.
- Distillation-Based Paradigms: Behavior Cloning and Supervised Fine-Tuning train autonomous agents by imitating complete expert trajectories.This approach is described as an early paradigm for building autonomous agents.
- Distillation-Based Paradigms: Global sequence forcing causes compounding errors in long-horizon tasks as deviations propagate autoregressively.Indiscriminate token-level imitation also triggers exposure bias and overwrites valid student exploration habits.
- Distillation-Based Paradigms: Recent distillation methods shift toward student-centered and dynamic strategies to address static global forcing.SCoRe-SFT is identified as providing post-hoc scaffolding through student-centered distillation, although the supplied passage is truncated.
- Reinforcement Learning Paradigms: Reinforcement-learning methods such as FTRL-GRPO and ToolRL optimize agents with sparse outcome rewards and verifiable execution feedback.These approaches refine policy optimization beyond distillation by leveraging environment feedback to align agent behaviors.
- Reinforcement Learning Paradigms: Sparse terminal signals can cause severe credit misassignment, inadvertently penalizing valid intermediate steps in failed trajectories.The supplied passage presents this as a limitation of the cited reinforcement-learning methods.
3 Methodology
DART-SD represents tool-calling execution in an information-aware interaction-state graph, then detects the earliest departure of failed student rollouts from a success-supported region. It uses that breakpoint to generate recovery continuations and apply progressive self-distillation only to generated recovery tokens.
- CTB-Guided Self-Distillation: After the CTB, DART-SD generates recovery continuations conditioned on the retained prefix and teacher references, supervising only generated assistant tokens across repeated distillation rounds.The retained student prefix is protected from destructive gradient updates.
- Information Atom Abstraction: Tool responses are canonicalized into task-specific information atoms, grouping semantically equivalent facts across response forms and tools while discarding non-informative outputs.Joint semantic assignment ensures equivalent calls share atoms, while non-informative responses map to no atom.
- Critical Topological Breakpoint (CTB): The CTB is the first transition from a projectable student state to a non-projectable state, marking unsupported information combinations or excessive exploration.Its anchor is the last valid teacher projection before the departure and supports recovery continuation generation.
- Main and Auxiliary Nodes: Each interaction state records acquired information atoms and recent useless operations, distinguishing main acquisition nodes from auxiliary exploration nodes.Main nodes advance the information set, whereas auxiliary nodes capture useless operations after the latest main node.
- Interaction-State Transition Graph (ISTG): The ISTG combines successful and failed teacher rollouts in a shared state space, preserving parallel transitions and diamond-shaped divergence and reconvergence of order-independent acquisition paths.Teacher and student executions can therefore be matched by information acquisition and useless exploration without requiring action-wise correspondence.
- Empirical Success-Reachable Region: DART-SD defines a budget-filtered success-reachable region from successful teacher rollouts, retaining main and auxiliary states near success while excluding failed-only states as recovery anchors.Reachability is measured backward from successful terminals, with the budget allowing task-dependent distance beyond the shortest successful depth.
4 Experiments
Experiments evaluate DART-SD across in-domain and out-of-domain tool-use benchmarks using Qwen3-4B and Qwen3-8B backbones under controlled no-thinking conditions. Results show robust gains, capability transfer, progressively more efficient and recoverable trajectories, and benefits from CTB-guided localized supervision.
- Experimental Setup: DART-SD is trained on FTRL’s 2,000-plus tool-use environments and evaluated on the in-domain test set plus four out-of-domain benchmarks.FTRL includes Single, Para-Single, Multi, and Para-Multi task structures.
- Experimental Setup: The evaluation compares Qwen3-4B and Qwen3-8B backbones against Base, distillation-based methods, and reinforcement-learning methods under no-thinking configurations.Baselines include standard SFT, SCoRe-SFT, OPSD, FTRL-GRPO, ToolRL, and MatchTIR variants.
- Overall Performance: DART-SD achieves robust, superior performance across the in-domain FTRL dataset and four out-of-domain tool-use benchmarks.The reported metrics include FTRL Solve-P, Solve-R, and Solve-F1; BFCL Multi-Turn; ToolHop TS, PI, and CF; and additional benchmark measures.
- Capability Distillation: DART-SD improves over the Qwen3-8B base model on all five benchmarks and surpasses the teacher on FTRL, ToolHop, and τ-bench.The comparison reports FTRL Solve-F1, BFCL average Multi-Turn, ToolHop AC, τ-bench Pass^1, and RoTBench PI.
- Progressive Training: Solve-F1 steadily improves while successful trajectories become shorter across progressive SFT iterations, eventually becoming more efficient than the golden solutions.The analysis tracks task performance and average tool-call length on the FTRL test set.
- Progressive Training: Average CTB position advances from 0.348 at Iter1 to 1.452 at Iter5, indicating that the model executes longer recoverable prefixes before localized recovery.Teacher-guided resampling begins at the first departure point and supervises only the recovery suffix while preserving mastered prefixes.
5 Conclusion
DART-SD is a topology-aware self-distillation framework that models multi-turn tool-calling execution as an Interaction-State Transition Graph and applies Critical Topological Breakpoint-guided localized supervision. Experiments show consistent gains over distillation and reinforcement learning baselines while preserving exploration and improving tool-use efficiency.
- Framework: DART-SD models execution as an Interaction-State Transition Graph that reveals the diamond topology of order-independent valid exploration.The graph supports topology-aware analysis of multi-turn tool-calling behavior.
- Framework: The framework identifies the Critical Topological Breakpoint, retrieves recovery references from the graph, and progressively self-distills with localized supervision that preserves the valid interaction prefix.Training supervision is guided by the CTB rather than applied indiscriminately across the full interaction.
- Results: DART-SD consistently outperforms distillation and reinforcement learning baselines across multiple in-domain and out-of-domain tool-use benchmarks.The reported benefits include improved task success, preserved exploration, fewer redundant tool calls, and progressively more complex tool-use behaviors.