Source-linked AI summary
AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning
Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Ming Zhu, Juntao Tan, Thai Hoang, Zuxin Liu, Liangwei Yang, Yihao Feng, Shirley Kokane, Tulika Awalgaonkar, Juan Carlos Niebles, Silvio Savarese, Shelby Heinecke, Huan Wang, Caiming Xiong
TL;DR
LLM-agent training is hindered by heterogeneous data sources containing complex multi-turn trajectories. AgentOhana unifies these trajectories, filters them, and supplies a generic loader for distributed training, while xLAM-v0.1 serves as an agent-focused model trained through this pipeline. The resulting model outperforms specified baselines across Webshop and ToolEval settings.
Problem
Heterogeneous multi-turn agent trajectories make data harmonization, training, and quality assessment difficult.
Method
AgentOhana aggregates trajectories from ten environments, converts them to a homogeneous format, filters them with AgentRater, and integrates them into distributed training through a generic dataloader.
Results
xLAM-v0.1 outperforms GPT-3.5-Turbo and GPT-3.5-Turbo-Instruct across Webshop configurations, and surpasses TooLlama V2 and GPT-3.5-Turbo-0125 across all evaluated ToolEval scenarios.
Takeaways & Limitations
AgentOhana provides a unified framework for consolidating diverse agent trajectories and supporting agent-model training across multiple environments.
Abstract
from arXiv · showhide
Autonomous agents powered by large language models (LLMs) have garnered significant research attention. However, fully harnessing the potential of LLMs for agent-based tasks presents inherent challenges due to the heterogeneous nature of diverse data sources featuring multi-turn trajectories. In this paper, we introduce \textbf{AgentOhana} as a comprehensive solution to address these challenges. \textit{AgentOhana} aggregates agent trajectories from distinct environments, spanning a wide array of scenarios. It meticulously standardizes and unifies these trajectories into a consistent format, streamlining the creation of a generic data loader optimized for agent training. Leveraging the data unification, our training pipeline maintains equilibrium across different data sources and preserves independent randomness across devices during dataset partitioning and model training. Additionally, we present \textbf{xLAM-v0.1}, a large action model tailored for AI agents, which demonstrates exceptional performance across various benchmarks. Begin the exploration at \url{https://github.com/SalesforceAIResearch/xLAM}.
1 INTRODUCTION
AgentOhana addresses the difficulty of training LLM agents on heterogeneous, multi-turn trajectories by unifying data collection, standardization, and training. It combines broad environmental coverage with a generic data loader and introduces xLAM-v0.1 as an agent-focused model.
- Motivation: Heterogeneous structures, syntaxes, labels, and processing methods across multi-turn agent datasets complicate harmonization, training, and fine-tuning.These inconsistencies can introduce bias and reduce compatibility across data sources.
- AgentOhana: AgentOhana is a comprehensive pipeline that consolidates heterogeneous multi-turn agent trajectories into a unified training resource.The platform is designed specifically to address data diversity and fragmentation in agent datasets.
- AgentOhana: AgentOhana incorporates data from ten distinct environments, spanning diverse agent scenarios.This coverage supports research into varied agent behaviors and interactions.
- Training pipeline: Standardized trajectories enable a generic data loader that balances data sources and preserves independent randomness across devices during training.The pipeline uses this unified format to support distributed agent training.
- Model: xLAM-v0.1 is a large action model tailored for AI agents and demonstrates strong performance across three benchmarks.The model is presented as evidence that AgentOhana can facilitate training high-performing agents.
2 METHODOLOGY
AgentOhana standardizes heterogeneous multi-turn agent trajectories, evaluates trajectory quality, and integrates datasets through a generic loader for distributed training.
- Workflow: AgentOhana consolidates heterogeneous agent trajectories into a homogeneous multi-turn format and integrates them into distributed training.Its workflow combines trajectory standardization, AgentRater filtering, and a generic data loader.
- 2.1 HETEROGENEITY OF VARIOUS DATASETS: HotpotQA and ToolAlpaca organize trajectories differently, requiring environment-specific parsing and history reconstruction before unification.HotpotQA stores a target trajectory in one prompt string, whereas ToolAlpaca requires matching inputs, outputs, observations, and accumulated history.
- 2.2 HOMOGENEOUS MULTI-TURN AGENT TRAJECTORY STANDARDIZATION: The unified JSON format stores trajectory metadata and represents each interaction as input, output, and next observation.Inputs include the current prompt plus prior interaction history, outputs capture model predictions, and observations record environmental feedback.
- 2.3 AGENTRATER: AgentRater evaluates whole agent trajectories with public models or close-world APIs, assigning scores from 0 to 5 and filtering low-quality data.Whole-trajectory assessment addresses cases where final task rewards do not reveal invalid intermediate actions.
- 2.4 GENERIC DATALOADER: A generic data loader harmonizes diverse datasets for training, while AgentOhana covers 10 environments and reports sampled, filtered, and average-turn statistics.The loader is designed for flexible data ingestion before the training framework.
3 EXPERIMENTS
The experiments evaluate xLAM-v0.1 across four agent benchmarks using balanced data compilation and distributed fine-tuning, with strong results across WebShop, HotpotQA, ToolEval, and MINT-Bench.
- Training setup: AgentOhana compiles a balanced corpus across sources and preserves independent randomness across devices during dataset partitioning and model training.The model was fine-tuned with supervised learning on eight Nvidia H100 GPUs using QLoRA, with each dataset traversed approximately three times on average.
- Benchmarks: The evaluation covers WebShop, HotpotQA, ToolEval, and MINT-Bench, spanning shopping, multi-hop question answering, function calling, and multi-turn tool use.WebShop uses average reward, HotpotQA uses F1, ToolEval uses pass rate, and MINT-Bench measures success across interaction limits from one to five steps.
- Results: xLAM-v0.1 consistently outperforms GPT-3.5-Turbo and GPT-3.5-Turbo-Instruct across WebShop settings, surpasses GPT-4-0613 in five of six settings, and ranks third on MINT-Bench.On ToolEval, it surpasses TooLlama V2 and GPT-3.5-Turbo-0125 in all scenarios and GPT-4-0125-preview in two of three settings.
- Results: On HotpotQA, xLAM outperforms GPT-3.5-Turbo and Mixtral-8x7B-Instruct-v0.1 across all settings, although GPT-4-0613 has a slight performance edge.The analysis reports that GPT-4-0613 typically identifies correct answers within four steps.
4 CONCLUSION
AgentOhana addresses the consolidation of diverse multi-turn LLM agent trajectories through unified data and training pipelines compatible with varied structures, formats, and environments.
- Conclusion: AgentOhana establishes unified data and training pipelines for consolidating diverse multi-turn LLM agent trajectories across varied environments.The framework is intended to provide a comprehensive, high-quality dataset for researchers and practitioners studying autonomous agents.
A HETEROGENEITY OF VARIOUS DATASETS
Figure 4 presents original agent trajectories from four environments, illustrating the range of source settings included in the data landscape.
- Dataset heterogeneity: The figure shows original trajectories from HotpotQA, Webshop, ToolAlpaca, and ToolBench.Panels A through D correspond to HotpotQA, Webshop, ToolAlpaca, and ToolBench, respectively.