PaperlayerView exact revision on arXiv

Source-linked AI summary

DIVE: Scaling Diversity in Agentic Task Synthesis for Generalizable Tool Use

Aili Chen, Chi Zhang, Junteng Liu, Jiangjie Chen, Chengyu Du, Yunji Li, Ming Zhong, Qin Wang, Zhengmao Zhu, Jiayuan Song, Ke Ji, Junxian He, Pengyu Zhao, Yanghua Xiao

arXiv:2603.11076v1cs.AIcs.SE

TL;DR

Tool-using LLMs struggle to generalize across shifting tasks and toolsets because synthesized training data lacks sufficient diversity while remaining executable and verifiable. DIVE executes diverse real-world tools first and reverse-derives grounded tasks from their traces, scaling tool-pool coverage, per-task toolset variety, and multi-step patterns. Across 9 OOD benchmarks, DIVE improves by +22.2 points per benchmark on average and outperforms the strongest 8B baseline by +68%, while diversity scaling beats quantity scaling even with 4× less data.

  • Problem

    Tool-using LLMs need to generalize across shifting real-world tasks and toolsets, but synthesized training tasks often lack structural diversity while training still requires executability and verifiability.

  • Method

    DIVE executes diverse real-world tools first, then reverse-derives query–answer tasks strictly entailed by observed traces through an Evidence Collection–Task Derivation loop.

  • Results

    +22.2 points per benchmark on average across 9 OOD benchmarks, with DIVE outperforming the strongest 8B baseline by +68%.

  • Takeaways & Limitations

    Diversity scaling consistently outperforms quantity scaling for OOD generalization, and richer tool-use patterns are visible under SFT and amplified by RL.

Abstract

from arXiv · show

Recent work synthesizes agentic tasks for post-training tool-using LLMs, yet robust generalization under shifts in tasks and toolsets remains an open challenge. We trace this brittleness to insufficient diversity in synthesized tasks. Scaling diversity is difficult because training requires tasks to remain executable and verifiable, while generalization demands coverage of diverse tool types, toolset combinations, and heterogeneous tool-use patterns. We propose DIVE, an evidence-driven recipe that inverts synthesis order, executing diverse, real-world tools first and reverse-deriving tasks strictly entailed by the resulting traces, thereby providing grounding by construction. DIVE scales structural diversity along two controllable axes, tool-pool coverage and per-task toolset variety, and an Evidence Collection--Task Derivation loop further induces rich multi-step tool-use patterns across 373 tools in five domains. Training Qwen3-8B on DIVE data (48k SFT + 3.2k RL) improves by +22 average points across 9 OOD benchmarks and outperforms the strongest 8B baseline by +68. Remarkably, controlled scaling analysis reveals that diversity scaling consistently outperforms quantity scaling for OOD generalization, even with 4x less data.

1. Introduction

DIVE addresses poor tool-use generalization by scaling diversity while preserving task executability and verifiability. It inverts synthesis order, using real tool executions and their traces to derive grounded tasks across varied tools and tool-use patterns.

  • Motivation: Synthesized training tasks often remain confined to narrow task families and fixed toolsets, causing poor or negative transfer under new task and toolset shifts.These routines can over-rely on rigid search→browse patterns and fail on scenarios such as clinical diagnosis with PatientLookup.
  • Motivation: Effective agentic training requires both grounded validity—verifiable and executable tasks—and structural diversity beyond template substitutions.Structural diversity includes varied tool types, per-task toolset combinations, and heterogeneous multi-step patterns.
  • DIVE: DIVE executes diverse real-world tools first, then reverse-derives tasks from resulting traces so executability and verifiability follow from observed evidence.The approach expands tool-pool coverage and per-task toolset variety while inducing heterogeneous tool-use patterns.
  • DIVE: DIVE constructs resource pools containing 373 validated tools across general-purpose and four expert domains, domain-specific seed concepts, and query-only exemplars.Each synthesis cycle samples a toolset, seed, and exemplars before evidence collection and task derivation.
  • DIVE: The Evidence Collection–Task Derivation loop gathers logically related evidence through interleaved multi-step tool use and derives query–answer pairs strictly entailed by accumulated traces.As evidence grows across iterations, task derivation refines grounding while increasing diversity.
  • Findings: DIVE improves tool-use generalization, with experiments showing that diversity scaling outperforms quantity scaling and that diverse training data amplifies RL benefits.The contribution claims span grounded task synthesis, structural diversity, and improved generalization.

2. Related Work

Prior work commonly scales tool-use training through synthetic tasks tailored to fixed task types and toolsets. Broader simulation and specialized pipelines improve coverage or scalability but introduce verification risks or costly customization.

  • Tool-Use Agents and Benchmarks: Tool-use benchmarks span web research, software engineering, domain applications, and universal tool suites, reflecting shifts in task distributions and toolsets.This diversity motivates evaluation of generalizable tool use across changing constraints, invocation protocols, and environments.
  • Synthetic Data for Tool-Use Agent Training: Synthetic task and trajectory scaling is prevalent for SFT and RL, but many pipelines target fixed task types and toolsets, limiting diversity for unseen scenarios.Deep-research synthesis with general web search tools is a representative example.
  • Synthetic Data for Tool-Use Agent Training: Specialized pipelines require customization for each task type or environment, making a common data-extraction practice costly and difficult to scale.The limitation follows from each task type or environment demanding its own synthesis pipeline.
  • Synthetic Data for Tool-Use Agent Training: LLM-simulated or generic tools improve scalability but risk unstable mock execution, allowing synthesis-time tasks to fail verification during training.Methods using real toolsets instead follow a contrasting synthesis strategy, though the supplied passage does not complete that comparison.

3. DIVE

DIVE builds diverse, executable, and verifiable agentic tasks by decoupling synthesis resources and deriving tasks from real tool executions. An iterative evidence-driven loop supports heterogeneous tool-use patterns and subsequent SFT/RL training.

  • Diverse Synthesis Resource Preparation: DIVE decomposes its synthesis resources into decoupled pools of tools, seed concepts, and query-only exemplars to broaden task and tool-use diversity.The tool pool defines the action space, the seed pool provides semantic coverage, and exemplars provide heterogeneous structural priors.
  • Diverse Synthesis Resource Preparation: The framework samples a seed, a compatible toolset, and query-only exemplars to create varied synthesis configurations.Each toolset contains 15–50 tools, while each exemplar sample typically contains 3–5 items.
  • Evidence-Driven Task Synthesis: DIVE collects evidence by executing real tools during multi-step reasoning, accumulating validated tool-action and output pairs.The collector operates within the sampled toolset and can perform rollouts of up to T_max steps.
  • Evidence-Driven Task Synthesis: A task generator reverse-derives query–answer pairs from accumulated evidence, varying query forms while keeping content and answers strictly grounded in execution traces.Exemplars guide patterns such as multi-hop retrieval and retrieve–compute pipelines.
  • Evidence-Driven Task Synthesis: The collection–derivation loop iterates over a fixed sampled toolset, increasing task and evidence diversity while preserving executability and verifiability by construction.Derived queries become the basis for subsequent collection steps, and reference answers come from tool-returned outputs.
  • Agentic Training with DIVE Tasks: DIVE trains agents with supervised cold-start demonstrations followed by reinforcement learning over synthesized tasks and their verified trajectories.SFT trajectories are retained only when the teacher’s answer matches the reference answer.

4. Experiments

DIVE is evaluated across in-distribution and multiple out-of-distribution settings, including shifts in tasks, tools, toolsets, protocols, and environments. Results show strong generalization, while controlled scaling analyses compare diversity expansion with quantity scaling.

  • Experimental Setup: The evaluation covers nine benchmarks spanning diverse tasks and toolsets, with L2 and L3 representing distinct out-of-distribution settings.The benchmark suite is designed to test broad generalization beyond DIVE’s training distribution.
  • Experimental Setup: DIVE is evaluated across three benchmark tiers: L1 in-distribution, L2 OOD with general-purpose tools, and L3 OOD with specialized toolsets.The taxonomy also distinguishes task, pool, set, protocol, and environment shifts.
  • Main Results: DIVE improves by +16.2 (SFT) and +22.2 (RL) points per benchmark on average across 9 OOD benchmarks, outperforming the strongest 8B baseline by +68%.It also improves in-distribution performance and transfers across both general and specialized toolsets.
  • Main Results: On TOOLATHLON, DIVE improves from near-zero to 8.3 points under per-task MCP app toolsets and stateful container environments.This zero-shot benchmark combines toolset and environment shifts.
  • Main Results: DIVE matches or surpasses specialist agents on their home benchmarks without task-specific training, while specialists often transfer poorly under unseen shifts.For example, DIVE scores 61.2 versus 50.0 for WebExplorer-8B on GAIA, whereas WebExplorer-8B drops 5.8 points below the base model on L3 benchmarks.
  • Scaling Analysis: Diversity-only scaling expands tool-pool coverage from 1→4 domains at a fixed 12k-task budget, whereas quantity-only scaling grows data from 12k→48k with tools fixed.The figure compares these matched scaling strategies and reports stronger OOD gains from diversity.
  • Scaling Analysis: Pool expansion combined with toolset variety sustains gains as SFT data scales from 12k→48k, unlike variety scaling with a fixed Finance tool pool.The comparison isolates whether new tool types are introduced during scaling.
  • Scaling Analysis: Compared with EnvScaler-8B, DIVE achieves a 3.2× larger OOD lift.This comparison supports the effectiveness of evidence-first synthesis with diverse real tools.

5. Analysis

DIVE’s analysis shows that expanding diversity across tool pools and toolsets improves OOD generalization more effectively than increasing data quantity, while SFT and RL produce richer tool-use structures.

  • Scaling Diversity: Diversity-only scaling consistently outperforms quantity-only scaling across OOD benchmarks, even with 4× less data.Diversity-only uses 12k examples, whereas quantity-only scales from 12k to 48k with task and tool distributions fixed.
  • Scaling Diversity: Pool expansion combined with toolset variety yields faster gains and a higher, slower-saturating generalization ceiling than toolset variety alone.Both approaches increase toolset variants, but only pool expansion introduces new tool types and capabilities.
  • SFT and RL: Across 24 scaling paths, RL amplifies the diversity-scaling trend beyond SFT, with the RL–SFT gap increasing from +4.6 to +5.6 as domains expand from one to four.The supplied passage attributes this pattern to exploration beyond imitation.
  • Structural Diversity: DIVE produces broader tool coverage, toolset variety, and tool-use patterns than Gen-DR, shifting topology mass from PureR toward R+P and PureP.Its tool usage also follows a long-tail distribution across a broad tool pool.
  • Structural Diversity: During RL, accuracy reward improves while unique tool-call graphs and R/P topologies persist and expand over 100 iterations.The analysis interprets this as exploration and reinforcement of multiple effective tool-use structures.

Conclusion

DIVE trains tool-using agents on diverse real-world tools while preserving executability and verifiability through execution-first task synthesis.

  • Conclusion: DIVE inverts synthesis by deriving tasks from real tool traces, scaling diversity while keeping supervision grounded.The framework improves OOD generalization across three benchmark tiers, with tool-pool diversity mattering more than data quantity.

Impact Statement

The work studies grounded data synthesis and post-training for tool-using language agents across diverse real-world tools.

  • Impact Statement: DIVE aims to expand diverse agentic training data, improve generalization across tasks and toolsets, and support more reliable agent evaluation.

A. Synthesized Task Examples

DIVE’s synthesized examples span academic, biological, financial, and medical cases, requiring multi-step reasoning over domain-specific toolsets.

  • Task Design: Each synthesized task requires multi-step reasoning across multiple tools selected from a domain-specific candidate set.The examples illustrate the complexity and diversity achieved by DIVE.
  • Task Design: The tools visualization distinguishes effectively used tools from available but unused tools, highlighting selective tool choice from noisy candidate sets.Stats report total calls, candidate-tool count, and distinct tools used.
  • Academic Domain: The academic example cross-references researcher identity, publication counts, citations, affiliations, dates, and journal metadata to identify a specific paper.Its answer is PANDA: Partitioned Data Security on Outsourced Sensitive and Non-sensitive Data.
  • Biological Domain: The biological example combines taxonomy, enzyme names, database variant counts, catalytic function, sequence identity, modification dates, sequence length, and gene synonyms.
  • Financial Domain: The financial example links fund holdings, sector classification, market value, float ratio, establishment date, trading data, stock-code prefixes, exchange, and company names.The reported stock code is 603658.SH.
  • Medical Domain: The medical example filters drugs by ATC subgroup, mechanism class, contraindication, formulation, tablet strengths, and brand-name counts.It identifies ertugliflozin and reports a 10 MG strength difference, two other ingredients, and empagliflozin’s four brands.

B. Data Synthesis Details

DIVE builds a diverse tool pool spanning Retrieval and Processing primitives, then synthesizes tasks through iterative evidence collection and trace-entailing task derivation. Its resources include 373 tools, domain-specific seeds, and query-only exemplars drawn from agentic benchmarks.

  • The tool pool combines Retrieval tools for information fetching with Processing tools for computation, analysis, or data transformation.
  • DIVE’s complete tool pool contains 373 tools aggregated by category, with official documentation or API endpoints recorded for the used tools.
  • The pool spans general-purpose and expert-domain tools, including Academic, Biological, Financial, and Medical resources.
  • Synthesis samples a toolset, seed concept, and query-only exemplars, then uses an iterative two-stage process to accumulate and reuse evidence.
  • The exemplar sources comprise 3,000 tasks from agentic benchmarks selected for domain-specific knowledge, multi-hop retrieval, and complex processing.
  • Evidence Collection interleaves multi-step reasoning with real tool use, while Task Derivation reorganizes accumulated evidence into query–answer pairs entailed by the traces.

C. Diversity Analysis

The diversity analysis evaluates how iterative synthesis changes structural diversity and task difficulty across trajectory-level metrics. Increasing iteration count expands tool and topology coverage while producing harder, more compositionally demanding tasks.

  • All structural diversity metrics increase substantially from K=1 to K=3 in trajectories generated for 4,000 synthesized tasks.GPT-OSS-120B solves the tasks for each iteration count, and diversity is measured in the resulting trajectories.
  • Tool coverage grows from 58.2% to 92.2%, while unique R/P topologies increase by 341% and topology class coverage expands from 42 to 165 classes.
  • Pass rate decreases from 66.8% to 41.2% as tool calls per task rise from 4.6 to 10.1, indicating that later iterations produce harder tasks with longer solution trajectories.
  • Distinct tools per task increase from 1.89 to 3.15, showing greater reliance on diverse tool combinations rather than repetitive single-tool patterns.
  • The results support the iterative synthesis design by linking additional iterations with broader tool-space coverage, greater structural variety, and more sophisticated reasoning requirements.

D. Topology Class Definition

DIVE categorizes tool-call graph patterns with a mutually exclusive, three-level topology hierarchy. The scheme distinguishes structure types and R/P compositions, enabling coverage comparisons between DIVE and Gen-DR.

  • Topology classes use a three-level hierarchy to systematically categorize tool-call graph patterns.
  • The structure types are checked in priority order and are mutually exclusive when assigning topology classes.
  • Class names encode structure type, using R/P composition with Single, Indep, Phain, Fork, Join, DAG, or Mix forms and associated dimensions.
  • DIVE covers 153 of 222 possible topology classes, compared with 65 for Gen-DR, whose coverage is concentrated in PureR classes.
  • Figure 6 tracks Accuracy, Calls/Task, Tool-call Graphs, and R/P Topologies across Academic, Biological, Financial, and Medical domains over 100 RL steps.Percentages represent relative changes between smoothed start and end values.

F. Training Details

The training setup combines Qwen3-8B supervised fine-tuning with GRPO-based reinforcement learning. The scaling analysis compares expanding data, tool-pool diversity, configurations, and their joint growth using controlled SFT and SFT-to-RL evaluations.

  • SFT fine-tunes Qwen3-8B in bf16 with AdamW, cosine learning-rate scheduling, 5% warmup, peak learning rate 1e-5, and minimum learning rate 0.
  • RL uses GRPO with entropy loss, gradient clipping at 1.0, an off-policy filter threshold of 12.0, and SGLang-generated rollouts.
  • Diversity scaling fixes 12k trajectories while expanding the tool pool from 1 to 4 domains and 174 to 373 tools; quantity scaling instead expands data from 12k to 48k with two fixed search/browse tools.
  • Config scaling expands data from 12k to 48k with a fixed 174-tool Financial pool, while Pool+Config Scaling jointly expands tool-pool diversity from 174 to 373 tools and data quantity.
  • Table 13 reports category scores after SFT and RL, with ΔAvg defined as Avg_RL - Avg_SFT and L2-A and L2-B aggregating specified benchmark groups.
Loading 2603.11076v1…