PaperlayerView exact revision on arXiv

Source-linked AI summary

ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback

Min Zeng, Yuzhou Liu, Zhenyu Cao, Hanxiu Chen, Heng Li, Caiquan Liu, Yafei Wen, Xiaoxin Chen

arXiv:2609.09072v1cs.CL

TL;DR

High-quality datasets linking user instructions to tool selection remain scarce, and common synthetic methods generate complete samples before static filtering. ToolLoop decomposes synthesis into three stages with dynamic self-feedback, achieving strong performance on BFCL and ACEBench with substantially less training data.

  • Problem

    High-quality datasets capturing links between user instructions and tool selection remain scarce, while synthetic methods commonly generate complete samples before static filtering.

    Source: §1 · p. 1

  • Method

    ToolLoop synthesizes data through ground-truth function plans, user-query derivation, and tool-call instantiation, using dynamic self-feedback to refine each stage.

    Source: §1 · p. 1, §6 · p. 9

  • Results

    ToolLoop achieves strong downstream performance on BFCL and ACEBench with substantially less training data; on BFCL, 11K examples yield 86.40% accuracy.

    Source: §1 · p. 2, §4.2 · p. 6, §6 · p. 9

  • Takeaways & Limitations

    The findings support constructing tool-use data through iterative generate–verify–refine rather than one-shot generation followed by end-stage filtering.

    Source: §6 · p. 9

    Without real-environment feedback, the authors cannot verify whether synthesized data prepares models for practical tool-use challenges or execution-result-driven refinement.

    Source: Limitations · p. 9

Abstract

from arXiv · show

High-quality tool-use data is critical for training language models to interact effectively with external tools. However, existing synthetic approaches typically follow a generate-then-filter paradigm with static post-hoc verification, often yielding inefficient data with imbalanced feature distributions. We propose ToolLoop, a closed-loop framework that decomposes synthesis into three progressive stages: (1) sampling function name combinations as ground truth; (2) backward derivation of user queries; and (3) forward derivation of tool calls. At each stage, dynamic self-feedback iteratively guides the model toward high-quality generation, realizing a transition from generate-then-filter to generate-verify-refine. On the Berkeley Function Calling Leaderboard (BFCL), a 4B parameter model trained with our 11K synthetic examples achieves 86.40% accuracy in non-reasoning mode, while an Isolate variant that removes BFCL-overlapping candidate functions still reaches 86.07\%. Cross-benchmark evaluation on ACEBench further demonstrates strong generalization, with 72.1% overall accuracy using only 18.3% of baseline training data.

1 Introduction

ToolLoop decomposes tool-use data synthesis into three stages and uses dynamic self-feedback to detect problems and guide regeneration. The paper reports strong downstream performance on BFCL and ACEBench with substantially less training data.

1 Introduction

  • Existing generate-then-filter methods struggle with tool selection, discard invalid samples without corrective feedback, and lack intermediate guidance for logical consistency.The introduction links these limitations to biased feature distributions and unmitigated error modes.Source: §1 · p. 1
  • The framework decomposes synthesis into three progressive stages and uses dynamic self-verification to return identified problems to the model as guidance for regeneration.This closed-loop process is intended to improve the efficiency and quality of retained samples.Source: §1 · pp. 1–2
  • ToolLoop achieves strong downstream task performance on BFCL and ACEBench with substantially less training data.The introduction reports strong improvements over existing approaches on both benchmarks, without giving numerical results here.Source: §1 · p. 2
  • The authors construct an 11K-example synthetic tool-use dataset covering four representative function-calling scenarios.Source: §1 · p. 2

2 Related Work

Related work spans synthetic-data generation, from instruction tuning to verifiable function-calling and multi-turn agent data, and LLMs used as automatic evaluators. Prior studies report both scalable evaluation potential and variation in judge performance across tasks and design choices.

Synthetic Data for LLMs

  • Self-Instruct and Alpaca generated instruction data; Persona Hub, APIGen, SynthAgent, and APIGen-MT extended synthesis to reasoning, verifiable function calls, refined trajectories, and multi-turn agents.Self-Instruct used filtering and deduplication, while Alpaca emphasized synthesis diversity and efficiency.Source: §2 · p. 2

LLMs as Automatic Evaluators / Judges

  • LLM-judge studies cover biomedical relation extraction, 20 NLP tasks, rubric-based scientific QA, human-judgment alignment, and consistency for retrieval-augmented generation.Structured output formatting improved judge performance over traditional metrics; findings also report variability by task and model and effects of decoding and criteria formulation.Source: §2 · p. 2
  • These studies suggest LLM assessment can scale as an alternative to traditional evaluation protocols in many scenarios.Source: §2 · p. 3

3 Methods

ToolLoop covers diverse tool-use scenarios, constructs semantically related candidate functions, and synthesizes examples in stages from target function sequences to user queries and tool calls. Stage-specific validation and iterative feedback refine outputs before they proceed.

3.1 Scope of Category

  • The four scenarios—Simple, Multiple, Parallel, and Parallel Multiple—span single-function calls through parallel invocations across multiple candidate functions.Parallel calls are independent; Parallel Multiple also requires selecting relevant functions from multiple candidates.Source: §3.1 · p. 3

3.2 Candidate Function Construction

  • Candidate functions are sampled with LLM guidance from K-means clusters of semantic embeddings, combining semantic grouping with function selection.A vector model embeds function descriptions, and clustering groups functions that typically share a domain or usage scenario.Source: §3.2 · p. 3

3.3 Decomposed Data Synthesis with Dynamic Self-Feedback

  • The synthesis pipeline separates generation into ground-truth sampling, backward query derivation, and forward tool-call instantiation, with dynamic feedback at each stage.This decomposition addresses the difficulty of synthesizing valid tool-use data in a single pass.Source: §3.3 · p. 3

3.3.1 Ground Truth Sampling

  • A ground truth is a sequence of function names for a request: an LLM selects candidates for parallel scenarios, while random sampling selects target invocation sequences for non-parallel scenarios.Parallel and parallel-multiple sequences contain 2–4 invocations, possibly repeating a function name; simple and multiple contain one. Parallel functions have no dependencies and should form a coherent usage context.Source: §3.3.1 · pp. 3–4

3.3.2 Backward Derivation of User Query

  • Queries are derived backward from the ground truth to match its implied calls, use signature-consistent parameters, supply required information, and sound conversational.The query should imply neither more nor fewer calls than the target sequence.Source: §3.3.2 · p. 4

3.3.3 Forward Derivation of Tool Calls

  • Tool calls are generated from the query and candidate functions with signature-compliant parameters and values that fulfill the request, using OpenAI function-calling format.Calls are represented as a complete structured sequence.Source: §3.3.3 · p. 4

3.3.4 Dynamic Self-Feedback

  • Dynamic feedback combines semantic verification, deterministic format and structure checks, and AST syntax parsing, then uses specific validator feedback to refine failed outputs.Checks are stage-specific, and samples are discarded only if they remain invalid after up to three retries; most correctable errors are resolved within one or two.Source: §3.3.4 · pp. 4–5

4 Experiments

ToolLoop reaches 86.40% overall accuracy on BFCL and 72.1% on ACEBench using substantially less training data than the data-centric baselines. Ablations show that dynamic self-feedback and iterative refinement outperform generation without quality control or static filtering.

4.1 Experimental Setup

  • ToolLoop uses Qwen3-4B-Instruct-2507 and synthesizes from 5,281 executable APIs sampled from ToolBench and BFCL, grouped into 26 semantic clusters.The Isolate training set filters BFCL-overlapping candidate functions; models are evaluated in non-reasoning mode with direct benchmark-compatible tool-call outputs, and trained for two epochs with a 16k-token maximum sequence length.Source: §4.1 · p. 5

4.2 Main Results

  • 91.29% non-live accuracy beats APIGen-4B’s 89.90%, while ToolLoop reaches 81.50% live accuracy and exceeds APIGen-4B (76.31%) and ToolMind-4B (77.57%).ToolLoop scores 96.50% on Multiple and 94.50% on Parallel_Multiple; its 79.17% on Live-Parallel_Multiple is one sample below APIGen-4B’s 83.33% on the 24-instance split, so the category result is uncertain. Isolate retains nearly the same overall performance and improves on non-live Multiple; the authors attribute gains to clustering tools, constructing explicit function combinations, and feedback-guided regeneration.Source: §4.2 · p. 6
  • 86.40% overall BFCL accuracy is achieved by ToolLoop-4B with 11K synthetic examples, exceeding APIGen-4B (83.11%) and ToolMind-4B (83.53%).ToolLoop-4B-Isolate, which removes BFCL-overlapping candidate functions, reaches 86.07%, only 0.33 points below the full model.Source: §4.2 · p. 6, Table 1 · p. 7

4.3 Ablation Study

  • 86.40% overall accuracy for full ToolLoop exceeds 82.56% with final filtering and 79.97% without feedback, compared with 82.14% for the stated base model.Final filtering discards invalid outputs, whereas intermediate feedback can correct inconsistencies among queries, target function sequences, and arguments before errors propagate.Source: Table 2 · p. 7, §4.3 · p. 7

4.4 Generalization to Broader Tool-Use Benchmarks

  • 66.5% in Single Turn trails ToolMind-4B’s 69.5%, while ToolLoop’s 60.0% in Profile exceeds APIGen-4B and ToolMind-4B (both 54.0%).All fine-tuned models underperform the base model in Profile, which the authors suggest may reflect the lack of explicit modeling of user preferences.Source: §4.4 · pp. 7–8
  • 72.1% overall ACEBench accuracy exceeds APIGen-4B by 5.1 points using 18.3% of its training data, and exceeds ToolMind-4B by 1.9 points using one fifth of its data.ToolLoop also leads on Atom (84.0%) and Similar API (78.0%), with gains reported beyond BFCL-specific formatting.Source: §4.4 · p. 7, Table 3 · p. 8

5 Analysis

ToolLoop’s analysis finds that iterative refinement concentrates extra synthesis effort on a small share of samples, while Qwen-Max provides a useful but incomplete semantic check. The results also show why semantic verification and deterministic validation are both needed.

5.1 Synthesis Efficiency

  • Synthesis of 11,024 retained examples consumed 33.41M tokens: 26.13M input and 7.28M output, measured with the Qwen tokenizer.Parallel examples accounted for the largest share of cost; simple and multiple examples bypassed LLM-based ground-truth sampling, while Stage 2 received only the sampled tool-chain functions.Source: §5.1 · p. 8
  • Stage 2 requires more retries because query generation must translate function plans into natural language without omitting parameters, adding tool calls, or mismatching the sampled functions.Stage 3 has a relatively low retry rate because deterministic checks can guard argument names and structural validity; resolving ambiguity during synthesis is especially important for multiple and parallel-multiple cases.Source: §5.1 · p. 8
  • Only 1.72% of Stage 1, 8.05% of Stage 2, and 4.25% of Stage 3 samples required two or three retries.Stage 2 had the highest refinement demand, with 18.1% of samples requiring at least one retry; Table 4 reports refinement iterations by stage.Source: Table 4 · p. 8, §5.1 · p. 8
  • Of 280 samples still invalid after three retries, 192 failed semantic verification, 83 failed deterministic validation, and 5 failed both.The discarded set was small relative to the 11,024 retained examples, and most discarded samples had persistent semantic inconsistencies.Source: §5.1 · p. 8

5.2 Verifier Reliability

  • 94% human agreement on 100 annotated instances supports Qwen-Max as a semantic verifier, but deterministic or AST-based checks remain necessary for objective constraints.The manual review was a sanity check, not a full validation; disagreements included fractional values for integer arguments and semantic grounding errors that AST parsing alone may not detect.Source: §5.2 · p. 9

6 Conclusion

ToolLoop decomposes tool-use data synthesis into ground-truth generation, user-query derivation, and tool-call instantiation, using dynamic self-feedback to move from generate-then-filter toward generate-verify-refine. Its reported results include strong BFCL accuracy and the best overall ACEBench score among data-centric methods.

6 Conclusion

  • 86.40% accuracy in BFCL non-reasoning was achieved by a 4B model trained on 11K ToolLoop examples, versus 86.07% for the BFCL-overlap-filtered Isolate variant; ToolLoop also achieved ACEBench’s best overall score among data-centric methods.The paper says the ACEBench result suggests synthesized data can transfer beyond the primary benchmark.Source: §6 · p. 9
  • ToolLoop decomposes synthesis into ground-truth generation, user-query derivation, and tool-call instantiation, with dynamic self-feedback shifting the process toward generate-verify-refine.The framework jointly verifies the intended function sequence, derived user intent, and final executable tool calls.Source: §6 · p. 9
  • ToolLoop’s value lies in improving the internal consistency and alignment of synthetic examples, supporting iterative generate-verify-refine over single-pair generation followed by end-only filtering.The paper presents ToolLoop as one practical implementation and a starting point for more reliable tool-use training data.Source: §6 · p. 9

Limitations

ToolLoop’s static synthesis and evaluation leave practical execution challenges unverified, while its shared semantic verifier may introduce correlated biases despite strong sampled agreement with human annotations.

Limitations

  • Without real environment feedback, ToolLoop cannot verify preparedness for execution failures or iterative refinement, motivating interactive testbeds with executable tools.Current static evaluations do not assess refinement based on execution results or practical challenges such as timeouts, malformed responses, and cascading failures in ground truths.Source: Limitations · p. 9
  • Qwen-Max agrees with human annotations on 94% of 100 sampled instances, but correlated verifier biases remain possible, motivating independent verifiers and stage-level calibration.Deterministic schema and AST checks complement the verifier; future work should also incorporate executable environment feedback.Source: Limitations · p. 9

Ethics Statement

The work uses synthetic tool-use data and public resources, without collecting personally identifiable information or conducting user studies.

A The distribution of synthetic data

ToolLoop’s 11,024 synthetic examples span four query categories, with simple and parallel examples making up the largest shares.

A The distribution of synthetic data

B Prompt Design

ToolLoop provides complete prompt templates for all three stages of its decomposed generation process. Each template includes a History of Feedback section that supports iterative refinement based on issues found in prior attempts.

B Prompt Design

Related papers

Loading 2609.09072v1…