PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Use Tools via Cooperative and Interactive Agents

Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, Zhaochun Ren

arXiv:2403.03031v4cs.CL

TL;DR

Single-agent tool learning uses predefined workflows and struggles both to revise errors flexibly and to support varied specialized actions. ConAgents assigns tool planning, execution, and review to three agents with automatic or adaptive interaction; experiments report improvements over baselines across three datasets.

  • Problem

    Predefined single-agent workflows constrain flexible correction of tool-use errors, and one general LLM-based agent struggles to perform varied specialized actions.

    Source: §1 · pp. 1–2

  • Method

    ConAgents coordinates specialized agents for tool planning, execution, and review through automatic and adaptive interaction, and uses specialized action distillation to support open-source models.

    Source: §1 · p. 2

  • Results

    Across three datasets, ConAgents outperforms baselines; the conclusion reports a success rate of 77.00 with a 13.2% point improvement.

    Source: §8 · p. 8

  • Takeaways & Limitations

    Specialized action distillation is reported to give open-source models strong performance with only 500 training examples.

    Source: §1 · p. 2

    The LLM-based agent is limited in perceiving multimodal tasks, whose image and speech inputs are represented with URLs.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. Existing methods employ one single LLM-based agent to iteratively select and execute tools, thereafter incorporating execution results into the next action prediction. Despite their progress, these methods suffer from performance degradation when addressing practical tasks due to: (1) the pre-defined pipeline with restricted flexibility to calibrate incorrect actions, and (2) the struggle to adapt a general LLM-based agent to perform a variety of specialized actions. To mitigate these problems, we propose ConAgents, a Cooperative and interactive Agents framework, which coordinates three specialized agents for tool selection, tool execution, and action calibration separately. ConAgents introduces two communication protocols to enable the flexible cooperation of agents. To effectively generalize the ConAgents into open-source models, we also propose specialized action distillation, enhancing their ability to perform specialized actions in our framework. Our extensive experiments on three datasets show that the LLMs, when equipped with the ConAgents, outperform baselines with substantial improvement (i.e., up to 14% higher success rate).

1 Introduction

Single-agent tool learning struggles to revise errors within predefined workflows and to support varied specialized actions. ConAgents assigns tool planning, execution, and review to specialized agents, and its reported results outperform the state-of-the-art baseline with both communication protocols.

1 Introduction

  • Existing methods' predefined pipelines constrain error correction, while one general LLM-based agent struggles to learn varied specialized actions.Practical tasks involve distinct actions such as planning, execution, and reflection.Source: §1 · pp. 1–2
  • ConAgents divides tool use among Grounding, Execution, and Review agents, with automatic real-time review or adaptive feedback triggered by exceptional errors.The review agent gives feedback to revise incorrect planning or execution.Source: §1 · p. 2
  • ConAgents outperforms the state-of-the-art baseline with both communication protocols, averaging a 6% improvement in Success Rate.The evaluation covers ToolBench and RestBench with various LLM backbones.Source: §1 · p. 2
  • SPAN addresses open-source models' difficulty with modular agent flow by distilling GPT-4 ConAgents trajectories into role-specific students, with strong performance reported using 500 training examples.The authors use LoRA to extend distillation to low-resource scenarios.Source: §1 · p. 2

2 Related Work

Related work covers tool-use training and multi-agent approaches, while identifying predefined agent flows as a challenge for tool-learning systems.

2 Related Work

3 Methodology

ConAgents modularizes tool learning across grounding, execution, and review agents. Automatic interaction alternates calibration phases, while adaptive interaction calls the review agent when execution fails.

3.1 Overall Framework

  • The grounding agent MG decomposes a task into stepwise tool plans, using the task, available tools, and prior plans and results as context.Each plan specifies a tool and necessary arguments.Source: §3.1 · p. 3, §3.2 · p. 3

3.2 Specialized Agents

  • The execution agent ME uses tool documentation to turn a plan into executable code; successful runs return targeted information and failures produce error messages.The resulting execution information is incorporated into MG's context for the next planning step.Source: §3.2 · p. 3
  • The review agent MR gives MG feedback on vague plans or nonexistent tools and gives ME feedback on faulty programs or execution results.Its feedback guides revision of incorrect planning or execution.Source: §3.2 · p. 3

3.3 Agent communication protocols

  • Automatic interaction alternates planning-review and execution-review, allowing up to α planning-calibration turns and β execution-calibration turns.The calibrated result returns to MG's context for the next planning generation.Source: §3.3 · p. 4
  • Adaptive interaction follows planning and execution step by step, triggering MR only when generated code fails to execute correctly.MR routes runtime-error feedback to MG or ME according to whether planning or coding caused the error.Source: §3.3 · p. 4

4 Specialization by Agent Distillation

The paper proposes specialized action distillation to adapt open-source LLM agents to ConAgents’ modular workflow. It synthesizes GPT-4 task-solving trajectories and trains separate models for the agents’ functions.

4 Specialization by Agent Distillation

  • SPAN addresses open-source models’ difficulty with ConAgents’ modular agent flow by distilling trajectories from specialized GPT-4 agents.The authors motivate this approach partly by potential privacy issues with commercial-model black boxes.Source: §4 · p. 4, §4.1 · p. 4

4.1 Synthesize the Training Dataset

  • The synthetic dataset contains 500 diverse examples, each pairing a task and candidate toolset with a three-agent task-solving trajectory.The authors sampled 2,919 ToolBench tasks, clustered similar tasks, and retained one instance per cluster before generating solutions.Source: §4.1 · pp. 4–5

4.2 Agent Training

  • The authors separately tune agents for planning, code execution, and review feedback, conditioning their outputs on task history and relevant feedback.They use LoRA to train each specialized agent separately.Source: §4.2 · p. 5

5 Experimental Setup

Experiments use ToolBench and RestBench, with automatic and human evaluations. The setup compares single- and multi-agent baselines using GPT-3.5-turbo and Mistral-8x7B.

5.1 Datasets and Evaluation Metrics

  • RestBench contains TMDB and Spotify subsets, while ToolBench covers practical tasks across diverse scenarios.Success Rate measures successful query completion; Correct Path Rate is the F1 score between generated and ground-truth tool sequences.Source: §5.1 · p. 5
  • Human evaluation rates Executability and Utility on a three-scale rating, using 30 randomly sampled cases per RestBench subset and three volunteers.Executability concerns invoking multiple tools in the correct logical order; Utility concerns whether tool results can support an answer.Source: §5.1 · p. 5

5.2 Baselines

  • Baselines include Chameleon, ReAct, CodeAct, ToolLLM, RestGPT, Reflexion, and α-UMi, plus ReAct@N and ToolLLM@N repeated up to N times.The repeated-run baselines continue until a task is completed.Source: §5.2 · pp. 5–6

5.3 Implementation Details

  • The main experiments use gpt-3.5-turbo at temperature 0, with Mistral-8x7B also tested; all methods receive the same required tools and ten random candidates.The maximum interaction turns are α = 3 and β = 3.Source: §5.3 · p. 6
  • For action distillation, three Mistral-8x7B agents are trained separately with LoRA at a learning rate of 5×10−5.Training takes four hours on three NVIDIA A800-PCIE-80GB GPUs.Source: §5.3 · p. 6

6 Results and Analysis

ConAgents outperforms the baselines across three datasets and all reported automatic metrics, with gains also reported in human evaluation. Ablations and case studies examine the roles of agent feedback and dynamic cooperation.

6.1 Experimental Results

  • ConAgents outperforms all baselines on three datasets across all metrics and achieves about 12% higher Success Rate than RestGPT.The authors attribute the improvement in part to specialized agents reducing the workload on a single agent and to dynamic communication protocols.Source: §6.1 · pp. 6–7
  • With Mistral-8x7B, action distillation raises TMDB Success Rate from 47.00 to 51.00 over directly prompted ConAgents.Direct prompting with ConAgents also performs better than baselines.Source: §6.1 · p. 7

6.2 Human Evaluation

  • ConAgents achieves the best human-evaluation Executability results, with a 0.08~0.12 improvement.The reported overall Kappa statistics are 0.75 for Executability and 0.71 for Utility.Source: §6.2 · p. 7

6.3 Ablation Study

  • Removing MR–MG interaction decreases average Success Rate by 2.50 and Correct Path Rate by 3.05 across two datasets; removing MR–ME interaction also decreases Success Rate.The authors interpret these results as evidence that review feedback helps revise incorrect planning and execution programs.Source: §6.3 · p. 7
  • Replacing dynamic cooperation with static compiler feedback decreases average Success Rate by 4.12, including a TMDB drop from 79.00 to 75.00.Correct Path Rate also decreases by 2.5 on Spotify.Source: §6.3 · p. 7

6.4 Case Study

  • Case studies find the cooperative framework more effective at executing varied tools and handling exceptional errors.The paper provides examples of agent cooperation in Appendix A.5.Source: §6.4 · p. 7

7 Discussion

The discussion examines interaction-turn effects, token use, review quality, and run-to-run consistency. It reports a practical interaction plateau alongside a remaining planning-review difficulty.

Interaction-turn analysis

  • On TMDB, Success Rate rises as maximum interaction turns increase from 1 to 3, then remains relatively stable through 5.The authors infer that agents correct most errors within three turns, though MR may miss plausible plans that omit required tool arguments.Source: §7 · p. 8, Figure 3 · p. 8

Inference efficiency

  • ConAgents consumes fewer tokens than strong baselines RestGPT and ToolLLM@3 while achieving better performance on TMDB.The authors attribute this efficiency to specialized tasks reducing the length of the single-agent exploration trajectory.Source: §7 · p. 8, Figure 4 · p. 8

Review quality

  • In a manual review of 50 trajectories, MR provides useful code or planning feedback in most tasks and hallucinates an incorrect review in fewer than 5%.Examples include advising MG to select tools only from the given list.Source: §7 · p. 8

Runtime consistency

  • Repeated runs show no statistically significant performance difference at significance level α = 0.05 under a two-tailed paired t-test.The comparison is between two randomly conducted experiments with the same settings.Source: §7 · p. 8

8 Conclusions

ConAgents coordinates specialized agents for tool planning, execution, and action calibration through two communication protocols; experiments report a 77.00 success rate with a 13.2% point improvement.

8 Conclusions

  • ConAgents modularizes tool learning across specialized agents for planning, execution, and action calibration, coordinated through two communication protocols.The framework is presented as enabling agents to cooperate on complex tasks.Source: §8 · p. 8
  • 77.00 success rate with a 13.2% point improvement is reported for ConAgents across experiments on three datasets.The conclusion describes this as evidence of ConAgents’ superiority.Source: §8 · p. 8
  • Specialized action distillation is proposed to improve open-source models’ ability to perform specific actions within ConAgents.Source: §8 · p. 8
  • Future work includes extending ConAgents to multimodal foundation models and coordinating cooperation between strong and weak agents.Source: §8 · p. 8

Limitations

The stated limitation concerns the LLM-based agent’s ability to perceive multimodal tasks.

Limitations

  • The LLM-based agent is limited in perceiving multimodal tasks; image and speech inputs are represented by URLs during tool execution.Source: Limitations · p. 9

Ethics Statement

The paper describes its research as avoiding harm, deception, and potential misuse, and emphasizes transparency and reproducibility through open-source tools and models.

Ethics Statement

  • The experiments use open-source platforms for all tools and include an open-source LLM, which the authors connect to transparency, reproducibility, and lower-resource use.Source: Ethics Statement · p. 9
  • The authors state that they made every effort to avoid harm to individuals or groups, deception, and potential misuse of information.Source: Ethics Statement · p. 9

A Appendix

The appendix details SPAN training and data selection, describes datasets and evaluation procedures, and illustrates the framework’s automatic and adaptive communication.

A.1 Details of Action Distillation

  • SPAN separately tunes grounding, execution, and review student models from GPT-4 task-solving trajectories using LoRA.Grounding training uses task context and review feedback; execution training targets executable programs, while review training targets calibration of incorrect actions.Source: §A.1 · p. 11

A.2 Heuristic Strategies for Data Selection

  • Data selection retains tasks with at least 10 candidate tools and removes tasks with uncallable or deprecated tools or inadequate documentation.Documentation must be at least 100 words.Source: §A.2 · p. 12

A.3 Datasets

  • The experiments use RestBench, RestBench-Spotify, and ToolBench; ToolBench evaluation uses 117 sampled complex-task cases and repurposed ground-truth tool lists.Three annotators support the ToolBench repurposing process, with a third resolving disagreements between two annotators.Source: §A.3 · p. 12

A.4 Evaluation Metrics Details

  • Automatic evaluation uses Success Rate for completed queries and Correct Path Rate for tool-sequence F1; human evaluation rates executability and utility on a three-point scale.Human evaluation covers 30 randomly sampled cases from each of the RestBench-TMDB and RestBench-Spotify datasets.Source: §A.4 · p. 12
  • The human rubric distinguishes fully solving a task from partial completion and assesses whether tool results are used to answer the question.The rubric describes three-, two-, and one-point levels for executability and utility.Source: §A.4 · p. 13

A.5 Case Study

  • Case studies report effective tool execution and use of execution results, with Figure 5 illustrating a cooperative-framework workflow.Source: §A.5 · p. 13

Case for our automatic agent communication

  • In the automatic protocol, grounding and review interact around planning before execution, and review feedback supports calibration of incorrect results.Figure 5 highlights the review agent’s feedback.Source: §A.5 · p. 13, Figure 5 · p. 14

Case for our adaptive agent communication

  • In the adaptive protocol, planning and execution alternate, and the review agent is triggered when generated code fails to execute correctly.Figure 6 illustrates review feedback routing an error so the grounding agent can supplement a missing argument instead of moving on with an error response.Source: Case for our adaptive agent… · p. 13, Figure 6 · p. 15
Loading 2403.03031v4…