PaperlayerView exact revision on arXiv

Source-linked AI summary

Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translation

Fan Yin, Zifeng Wang, I-Hung Hsu, Jun Yan, Ke Jiang, Yanfei Chen, Jindong Gu, Long T. Le, Kai-Wei Chang, Chen-Yu Lee, Hamid Palangi, Tomas Pfister

arXiv:2503.07826v1cs.CL

TL;DR

Multi-turn function-calling agents still struggle with nested calls, long dependencies, and irrelevant or underspecified requests. Magnet builds graph-based function-signature paths and distills positive and negative training trajectories; its trained models show strong benchmark performance, including a 32.5-point gain on BFCL-v3 multi-turn cases over the base model.

  • Problem

    Current agents still struggle with nested calls, dependencies on conversation history, and missing functions or parameter values in multi-turn interactions.

    Source: §1 · p. 1

  • Method

    Magnet constructs graph-based function-signature paths and uses a teacher model with reference and incorrect hints to synthesize trajectories for SFT and preference optimization.

    Source: §1 · pp. 2–3, §3.2 · p. 5

  • Results

    Magnet-trained models show superior performance on BFCL-v3 and ToolQuery, with Magnet-14B-mDPO adding 32.5 points over the base model on BFCL-v3 multi-turn cases.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors report strong performance on agentic benchmarks from training with Magnet’s synthesized multi-turn trajectories.

    Source: §5 · p. 12

    The studied function signatures are mainly English and plain text, so the authors caution that findings may not generalize to other languages or modalities.

    Source: Limitations · p. 15

Abstract

from arXiv · show

Large language models (LLMs) have exhibited the ability to effectively utilize external tools to address user queries. However, their performance may be limited in complex, multi-turn interactions involving users and multiple tools. To address this, we propose Magnet, a principled framework for synthesizing high-quality training trajectories to enhance the function calling capability of large language model agents in multi-turn conversations with humans. The framework is based on automatic and iterative translations from a function signature path to a sequence of queries and executable function calls. We model the complicated function interactions in multi-turn cases with graph and design novel node operations to build reliable signature paths. Motivated by context distillation, when guiding the generation of positive and negative trajectories using a teacher model, we provide reference function call sequences as positive hints in context and contrastive, incorrect function calls as negative hints. Experiments show that training with the positive trajectories with supervised fine-tuning and preference optimization against negative trajectories, our 14B model, Magnet-14B-mDPO, obtains 68.01 on BFCL-v3 and 73.30 on ToolQuery, surpassing the performance of the teacher model Gemini-1.5-pro-002 by a large margin in function calling.

1. Introduction

Multi-turn function calling remains difficult despite progress on independent calls: agents must handle nested calls, dependencies on earlier outputs, and irrelevant or underspecified requests. Magnet builds graph-structured function-signature paths and synthesizes training trajectories.

1. Introduction

  • Multi-turn agents must handle nested function calls, use conversation history for long dependencies, and ask for clarification when functions or parameter values are missing.The paper distinguishes multi-step interactions within one request from multi-turn exchanges across conversational turns.Source: §1 · p. 1, Figure 1 · p. 2
  • Magnet iteratively translates function-signature paths into user-like queries and executable function-call references, then turns query-reference pairs into training trajectories.The framework targets training public models with supervised fine-tuning and preference optimization.Source: §1 · pp. 2–3
  • A local dependency graph links functions when a source function’s outputs relate to a target function’s inputs, enabling random walks to form multi-turn signature paths.The paper adds Insert, Merge, and Split operations to tailor paths to multi-turn challenges.Source: §1 · pp. 2–3
  • Context distillation uses reference function calls as positive hints and deliberately incorrect hints to construct contrasting negative trajectories for preference optimization.The contribution summary describes trajectories for both SFT and mDPO.Source: §1 · p. 3
  • Magnet-14B-mDPO ranks fourth on BFCL-v3 and adds 32.5 points over the base model on multi-turn cases; the pipeline also demonstrates advantages on ToolQuery.The authors train on 34K trajectories with SFT and 4,556 trajectory pairs with mDPO.Source: §1 · p. 3

2. Related Work

Prior work evaluates tool-use across single-step, multi-step, and multi-turn settings and synthesizes training data in several ways. Magnet differs by organizing function signatures as graphs, adding node operations, and controlling trajectory synthesis.

2. Related Work

  • Tool-use benchmarks span single-step, multi-step, and multi-turn cases; BFCL-v3 covers all three, while many existing datasets are human-curated and Magnet requires minimal human effort.Multi-step cases include parallel, chained, and nested function use.Source: §2 · p. 3
  • Training-data approaches include replacing text segments with API calls, sampling APIs to synthesize queries, and unifying tool-use data formats with other agentic tasks.Other work adds function masking or irrelevant functions, uses multi-task fine-tuning, and adapts composition to improve single-turn calling.Source: §2 · pp. 3–4
  • Unlike prior API-to-query back-translation work, Magnet organizes signatures in graphs, applies node operations for multi-turn complexity, and adds further controls to trajectory synthesis.The pipeline figure divides construction into function-pool and graph setup, node operations, iterative translation, and positive/negative trajectory construction.Source: §2 · p. 4, Figure 2 · p. 4

3. Methodology: Magnet

Magnet synthesizes multi-turn training trajectories by translating function-signature paths into query and executable-call references, then using a teacher LLM to form trajectories. Graph-based path construction and positive and negative hints support this pipeline.

3.1. Training setup and formulation

  • Magnet combines supervised fine-tuning with preference learning, using an mDPO loss plus a maximum-likelihood loss to tune the SFT model.The negative and positive trajectories are used in the mDPO objective.Source: §3.1 · pp. 4–5

3.2. Trajectory Construction Overview

  • The pipeline first creates query and function-call-reference pairs, then uses a teacher LLM to turn them into trajectories.Gemini-1.5-pro-002 is the assistant LLM used unless stated otherwise.Source: §3.2 · p. 5

3.3. Backbone: back-and-forth translation

  • Back-and-forth translation converts each function signature into a synthetic query, then converts that query into an executable call using the signature and previous-round outputs.The process repeats in order through the function-signature path so earlier outputs are available to later rounds.Source: §3.3 · pp. 5–6

3.4. Obtain FSP with function dependency graph random walk and node operations.

  • The method builds function-signature paths by walking a dependency graph and enhances them with Insert, Merge, and Split operations for nested calls, dependencies, and missing information.Insert can add implicit or later-reused functions; Merge creates single-turn multi-call cases; Split introduces missing-function or missing-parameter cases.Source: §3.4 · pp. 6–7

3.5. Hint-based context distillation for positive and negative trajectory sampling

  • Positive trajectory prompts include reference calls as hints, while negative trajectory prompts use misleading calls collected from SFT-model errors.For each instance, ten SFT-model trajectories are examined to identify incorrect calls for negative hints.Source: §3.5 · pp. 7–8

3.6. Post-processing and data mixture

  • Post-processing shuffles available-function order, filters likely failed or malformed calls, and mixes in single-turn and irrelevant-function data.The added single-turn cases include single, parallel, and multiple relevant function calls.Source: §3.6 · p. 8

3.7. Data statistics

  • The final datasets contain 34,000 SFT instances and 4,556 preference-learning instances, totaling 38,556.The paper reports this total as around half the size of APIGen and Hammer datasets.Source: §3.7 · p. 8

4. Experiments

Experiments evaluate Magnet on BFCL-v3 and ToolQuery, where its 14B model ranks fourth on BFCL-v3 and achieves the best reported ToolQuery success rate. Ablations examine pipeline components, training data, base and teacher models, and the data-mixture trade-off.

4. Experiments

  • Evaluation uses BFCL-v3’s 4,751 cases across function-calling categories and ToolQuery’s 60 cases for multi-step and multi-turn calls; training uses 34,000 positive SFT trajectories followed by mDPO.Source: §4 · p. 8, §4.1 · pp. 8–9

4.2. Main results on BFCL-v3

  • #4: Magnet-14B ranks fourth on BFCL-v3, surpasses o1, and is on par with GPT-4-Turbo overall and on multi-turn performance.The 14B mDPO model gains 2.50% multi-turn success over SFT; trained 7B and 14B models also outperform Gemini-1.5-pro-002 on multi-turn cases. Against base models, multi-turn gains are 18.5 for 7B and 30.0 for 14B.Source: Table 2 · p. 9, §4.2 · p. 9

4.3. Main results on ToolQuery

  • 73.3: Magnet-14B achieves a 73.3 success rate on ToolQuery, surpassing GPT-4o and xLAM-8x22b-r on functions unseen during training.The authors report this as evidence of generalization to unseen functions.Source: §4.3 · p. 9, Table 3 · p. 10

4.4. Ablation Study and Analysis

  • The local dependency graph improves base-model multi-turn success by around 8%; merge and insert boost multi-turn performance, while split improves missing-function, missing-parameter, and irrelevance scenarios by 5.5%, 7.5%, and 3.69%.Distilling function-call references into positive trajectories yields a 14.50% multi-turn gain over direct teacher-trajectory distillation; context-distilled negative trajectories improve performance by around 0.5% over randomly sampled rejections.Source: §4.4 · p. 10
  • Magnet-7B-SFT outperforms models trained on APIGen and ToolAce by 20.25 and Hammer2.1-7b by 3 points, especially on multi-turn cases.Source: §4.4 · p. 11
  • Coder models show greater potential to learn from the training data, with 5.38 better multi-turn performance; the data’s function-calling gains also generalize to Mixtral, while self-improvement is possible with slight performance drops.Gemini-1.5-pro-002 and GPT-4o as teachers show comparable performance; hint-based context closes the gap between their original multi-turn performance.Source: §4.4 · p. 11

Discussion: the impact of data mixture

5. Conclusion

Magnet synthesizes multi-turn tool-use trajectories to train LLM agents. The paper reports strong performance on agentic benchmarks.

5. Conclusion

  • Magnet is a pipeline for synthesizing multi-turn trajectories to train tool-use LLM agents.It targets challenges in multi-turn function calling.Source: §5 · p. 12
  • The synthesis method uses graphs to construct multi-turn queries and reference function calls tailored to those challenges.Source: §5 · p. 12
  • The authors report strong performance on agentic benchmarks after training with the synthesized data.The training sequence uses supervised fine-tuning followed by mDPO.Source: §5 · p. 12

Limitations

The paper identifies language and modality coverage, resolving knowledge conflicts, and exploration ability as limitations of the work.

Limitations

  • The studied function signatures are mainly English and plain text, so conclusions may not generalize to other languages or modalities.The authors suggest multilingual and multimodal tools as future extensions.Source: Limitations · p. 15
  • The trained model may follow internal knowledge over conflicting tool-retrieved knowledge, such as returning a fixed date despite a changing date-tool output.The authors describe this as a limitation in resolving conflicts between context and internal knowledge.Source: Limitations · p. 15
  • The model's exploration ability is limited by a lack of such data in its training set, including reflecting on wrong actions and restarting exploration.The authors identify greater exploration ability as future work.Source: Limitations · p. 15

A. Prompts

The appendix gives prompts for function relationships, multi-turn query and call generation, trajectory construction, and model inference.

A. Prompts

  • The system prompt restricts inference responses to function calls when invoked, while a separate prompt classifies functions into listed domains or misc.The system prompt includes a function schema; the domain prompt returns one domain name or misc.Source: A. Prompts · p. 16
  • The dependency prompt builds an adjacency list from functions whose outputs determine or supply all or part of another function's inputs, including across domains.Source: A. Prompts · p. 17
  • The positive-trajectory prompt supplies hinted calls, asks the model to correct imperfect hints, and handles missed functions, parameters, and tool errors.For multiple nested calls, it directs the agent to call one function and wait for feedback before the next.Source: A. Prompts · p. 18
  • The negative-trajectory judge checks responses against references and labels errors such as missing nested calls, dependency mistakes, and wrong summaries.Source: A. Prompts · p. 19
  • The forth-translation prompt fills a candidate function call's parameters from the user query and prior reference outputs, allowing at most three parallel calls.Source: A. Prompts · p. 20
  • The back-translation prompt generates chained, natural queries from candidate functions, using prior outputs and avoiding repeated queries.It instructs the query to include required-parameter information, which may be implicit in prior outputs.Source: A. Prompts · p. 21

B. Training setup

The training setup fine-tunes Qwen2.5-Coder models with SFT and mDPO, using full-parameter SFT and different mDPO tuning strategies by model size.

B. Training setup

  • The experiments start from Qwen2.5-Coder-7B-instruct and Qwen2.5-Coder-14B-instruct, chosen for prior use as function-calling bases and demonstrated potential.Source: B. Training setup · pp. 21–22
  • SFT fully trains both model sizes; the 14B mDPO model uses LoRA, while the 7B model is fully trained.The authors report that one SFT epoch generally worked best and searched over the stated learning rates, batch sizes, and epoch counts.Source: B. Training setup · p. 22
  • SFT uses transformer-trl, and mDPO uses the implementation from Xiong et al. (2024), also based on transformer-trl.Source: B. Training setup · p. 22

C. Data contamination study on BFCL-v3

The study finds that StableToolBench functions contribute substantially to BFCL-v3 multi-turn performance, while exact FSP matches with test cases are rare.

C. Data contamination study on BFCL-v3

Related papers

Loading 2503.07826v1…