Source-linked AI summary
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, Junxian He
TL;DR
LLM-agent evaluation needs a unified way to cover diverse, multi-round, partially observable environments and to capture progress that final success rates miss. AGENTBOARD addresses this gap with a nine-task benchmark, a progress-rate metric, and an open-source analytical toolkit; its evaluations identify differences across proprietary and open-weight agents, with GPT-4 leading overall progress rate. The framework is intended to support more detailed understanding and development of LLM agents.
Problem
Existing LLM-agent evaluations provide limited coverage of diverse, multi-round, partially observable tasks and often fail to capture partial task completion beyond final success rates.
Method
AGENTBOARD combines a nine-task, 1013-environment benchmark with human-annotated subgoals, a unified progress-rate metric, and an open-source analytical visualization toolkit.
Results
GPT-4 achieves the highest average progress rate at 70.0%, while evaluations show proprietary models generally outperform open-weight models and reveal differences in agent abilities.
Takeaways & Limitations
AGENTBOARD provides detailed, interactive analysis intended to improve understanding of LLM-agent capabilities and support further agent development.
Takeaways & Limitations
AGENTBOARD relies on human-annotated subgoals and evaluates agents mainly in simulated environments, limiting scalability and direct real-world coverage.
Abstract
from arXiv · showhide
Evaluating Large Language Models (LLMs) as general-purpose agents is essential for understanding their capabilities and facilitating their integration into practical applications. However, the evaluation process presents substantial challenges. A primary obstacle is the benchmarking of agent performance across diverse scenarios within a unified framework, especially in maintaining partially-observable environments and ensuring multi-round interactions. Moreover, current evaluation frameworks mostly focus on the final success rate, revealing few insights during the process and failing to provide a deep understanding of the model abilities. To address these challenges, we introduce AgentBoard, a pioneering comprehensive benchmark and accompanied open-source evaluation framework tailored to analytical evaluation of LLM agents. AgentBoard offers a fine-grained progress rate metric that captures incremental advancements as well as a comprehensive evaluation toolkit that features easy assessment of agents for multi-faceted analysis. This not only sheds light on the capabilities and limitations of LLM agents but also propels the interpretability of their performance to the forefront. Ultimately, AgentBoard serves as a step towards demystifying agent behaviors and accelerating the development of stronger LLM agents.
1 Introduction
AGENTBOARD addresses the need to evaluate LLM agents across diverse, multi-round, partially observable tasks while revealing progress beyond final success rates. It combines a nine-task benchmark with an open-source analytical toolkit for detailed assessment of agent abilities.
- Evaluation gap: Existing evaluations often rely on final success rate, which provides limited insight into complex interaction processes and obscures distinctions when success rates are near zero.Partial task completion can therefore remain invisible in challenging environments.
- Benchmark scope: AGENTBOARD evaluates agents across diverse tasks using multi-round interactions in partially observable environments.Its benchmark covers embodied AI, game, web, and tool agents.
- Benchmark scope: The benchmark contains 9 diverse tasks and 1013 human-authenticated environments designed to be multi-round and partially observable.Each data sample includes defined or manually annotated subgoals.
- Analytical evaluation: AGENTBOARD introduces a unified progress rate metric that tracks detailed advancement toward subgoals beyond final success rates.The metric is intended to distinguish progress that success rates make negligible.
- Analytical evaluation: Its open-source toolkit provides interactive visualization and analysis of progress, example difficulty, sub-skills, long-range interaction, grounding, and trajectories.The toolkit also offers a unified interface and customization options.
- Reported findings: Evaluations report that GPT-4 outperforms other models, while open-weight models vary in multi-turn interaction and foundational agent abilities.The reported foundational abilities include grounding, world modeling, and self-reflection.
2 AGENTBOARD – Overview
AGENTBOARD is a unified, open-source benchmark and analytical toolkit for evaluating LLM agents across diverse, multi-round, partially observable tasks. It complements final success rates with fine-grained progress tracking and interactive analysis of agent abilities.
- AGENTBOARD follows five principles: task diversity, multi-round interaction, partially observable environments, fine-grained metrics, and analytical evaluation.
- Task Diversity and Uniformity: The benchmark curates nine diverse environments across four scenarios that require multi-round interaction and are primarily partially observable.
- Fine-grained Progress Rate: The progress rate metric tracks intermediate agent progress and distinguishes minimal improvements that final success rates may miss.
- A Unified Multi-Round Reflex Agent: An agent receives textual observations, selects textual actions, and uses goal-conditioned memory of prior actions and observations to predict its next action.
- Fine-grained Progress Rate: The progress rate records the highest matching score between visited states and the goal, reaching 1 when the task is complete.
3 AGENTBOARD – Task Composition
AGENTBOARD combines embodied, game, web, and tool environments into a diverse, text-based benchmark requiring multi-round interaction and partial observability. Its tasks adapt existing environments and use task-specific progress measures, including manually annotated subgoals and state-matching scores.
- Task Scenarios: AGENTBOARD covers four scenarios—embodied, game, web, and tool—with tasks selected for diversity, everyday relevance, multi-round interaction, and partial observability.
- Embodied: ALFWorld, ScienceWorld, and BabyAI are embodied environments adapted or annotated to support text-based LLM interaction and progress measurement.
- Game: Jericho requires exploration and world modeling, while PDDL supplies strategic planning games adapted to text-based observations.
- Web: WebArena progress is computed continuously by combining URL and content matching scores, while WebShop provides a network-based e-commerce simulation.
- Tool: Tool-Query requires planned database access through appropriate trajectories, and Tool-Operation measures progress in todo and spreadsheet tasks.
- Human verification compares automatic progress rates with assessments of 60 trajectories per task generated by GPT-4, GPT-3.5-Turbo, and DeepSeek-67b.
4 Experiments
AGENTBOARD evaluates proprietary, open-weight, and agent LLMs using success and progress rates, then analyzes grounding, difficulty, long-range interaction, and sub-skills. The results show stronger proprietary and agent-tuned models, while progress rate distinguishes models more effectively when success rates are similarly low.
- Main Results: Progress rate separates models with similar success rates, including Llama2-13b at 18.9% and Mistral-7b at 24.6%.Their success rates are both negligible at 2.1% and 3.9%, respectively.
- Main Results: GPT-4 achieves the highest average progress rate at 70.0%, outperforming the best open-weight models.Claude and Gemini follow GPT-4 among the evaluated proprietary models.
- Main Results: Code-oriented training improves agent performance across tool-use and Games tasks, extending beyond code generation.CodeLlama-34b exceeds Llama2-70b by 6.2% in progress rate, while CodeLlama-13b exceeds it by 2%.
- Main Results: Agent instruction tuning improves both progress and success rates, with AgentLM-70b gaining 9.5% and 10.2%, respectively, over Llama2-70b.AgentLM-70b also improves on tasks beyond the AlfWorld and WebShop trajectories used for training.
- Analytical Evaluation: AGENTBOARD provides interactive analyses of progress rates, difficulty breakdowns, sub-skills, long-range interaction, and exploration behavior through an open-source toolkit.The evaluation uses interactive visualizations on a W&B web panel.
- Analytical Evaluation: Grounding accuracy, hard-example performance, and long-range interaction reveal distinct weaknesses beyond aggregate results.Models generally drop on hard cases; open-weight models often stop progressing after about six steps, while GPT-4 and Claude2 progress longer in AlfWorld and PDDL.
5 Visualization Panel for LLM Agent Analysis: A Case Study
The visualization panel supports comparative analysis of agent performance across overall metrics and subskills. In the described case, GPT-4 is compared with six baselines and performs strongly across most assessed dimensions.
- Comparative setup: GPT-4 is evaluated as the current run against six baseline models in the visualization panel.The summary board compares GPT-4 with six other models.
- Overall performance: GPT-4 outperforms all baselines by a large margin on overall metrics.
- Subskill analysis: GPT-4 demonstrates high capability scores across all six subskills.
6 Related Work
Prior work has established LLM-agent benchmarks for web browsing, games, and tool use, while related studies examine interaction ability, agent structures, and generalist challenges. AgentBoard focuses exclusively on text-based environments to analyze textual reasoning and actions in depth.
- Agent benchmarks: Existing LLM-agent benchmarks cover web browsing, games, and tool use.
- Capability-focused studies: Some studies target specific capabilities, including model interaction ability and agent structures.
- Generalist and multimodal evaluation: Recent work also presents generalist challenges for LLM agents and explores multimodal LLM agents.
- AgentBoard scope: AgentBoard evaluates LLM-agent abilities exclusively in text-based environments through textual reasoning and actions.
7 Conclusion
AgentBoard combines a generalist-agent benchmark with an open-source analytical framework and interactive visualization panel. Its stated aim is to support detailed evaluation and understanding of LLM agents, while its progress-rate analysis relies on human-annotated subgoals.
- Conclusion: AgentBoard provides a benchmark for evaluating generalist LLM agents.
- Conclusion: The open-source framework offers customizable unified metrics, comprehensive analysis, and an interactive visualization web panel.
- Conclusion: AgentBoard aims to facilitate detailed evaluation and understanding of LLM agents and further advancement in the field.
- Limitations: The progress rate depends on human-annotated subgoals, while real-world benchmarking remains future work because of changing labels and security risks.
B Limitations
AgentBoard’s limitations concern the human effort required to annotate progress subgoals and the use of simulated rather than real-world environments. Real-world evaluation additionally faces changing ground-truth labels and security risks.
- Human-dependent Annotation: Human-annotated subgoals make progress-rate measurement subjective, labor-intensive, and less scalable as task complexity increases.
- Human-dependent Annotation: Current LLMs underperform on AgentBoard tasks and cannot accurately generate the subgoals needed for annotation.
- Benchmarking Real-World Problems: Standardized benchmarking currently relies on simulated environments, leaving real-world benchmarking for future work.
- Benchmarking Real-World Problems: Real-world benchmarks face variable ground-truth labels and security challenges when agents can alter their environments.
- Ethics and Societal Impact: The tool-operation evaluation permits agents to access and edit online information, while preventing personal-information leakage and online distribution of generated content.
D Confidence Interval of LLM Agents Evaluation
Confidence intervals reveal that proprietary models often vary more across outputs than open-source models, reflecting differences in decoding and temperature settings.
- Proprietary models often exhibit larger output deviations than open-source models.The paper primarily attributes this difference to additional post-processing during proprietary-model decoding.
- Open-source models show smaller deviations under lower-temperature settings chosen for reproducibility.
- Tables 6 and 7 report confidence intervals for progress-rate and success-rate metrics.
E Exploration Behavior Analysis
AgentBoard analyzes exploration by counting locations visited in partially observable environments, linking exploration breadth to the information needed for task completion.
- Exploration counts rooms, containers, and places in BabyAI, AlfWorld, and Jericho, respectively.
- Most models fail to explore the minimum number of locations typically needed to complete the goal.
- GPT-3.5-Turbo performs comparably to GPT-4 on exploration, while Llama2-70b and CodeLlama-34b outperform Vicuna-13b-16k among open-weight models.
- Exploration analysis is not fully implemented in AgentBoard because it is feasible only for some environments.
F Ablation Study of Agent Framework
The ablation study favors a simple Act framework and sliding-window memory for testing agent abilities, particularly under long interactions and limited context.
- Framework design: Act is selected because it requires minimal design and applies to most instruction-following LLMs.
- Framework design: ReAct does not consistently improve GPT-3.5-Turbo over Act in experiments involving interactions of up to 30 turns.The authors hypothesize that interleaved thoughts can pressure context length during long-term interactions.
- Memory design: Cutoff generally performs worse than sliding windows, while summary performance varies across tasks and depends on the LLM's summarization ability.
- Memory design: Memory design barely affects Mistral with 32k context but substantially affects GPT-3.5-Turbo with 4k context.
- Memory design: Sliding-window memory uses recent interaction history and avoids confounding benchmark results with summarization ability.
I.1 Evaluation Setup
The evaluation setup uses standardized inference and model-selection procedures, while documenting the benchmark's memory, sub-skill, and evaluated-model resources.
- Evaluation setup: The experiments use greedy decoding with temperature zero and implement all LLMs through vLLM for reproducibility and accelerated inference.The paper reports 10× acceleration over Hugging Face inference.
- Evaluation setup: Prompts retain the most recent interaction histories within each model's maximum context length.
- Evaluation setup: The setup selects checkpoint versions with the best instruction-following ability, including chat supervised fine-tuning and alignment.
- Evaluation resources: Table 10 compares memory approaches, Table 11 lists task sub-skill scores, and Table 12 records evaluated-model code or API identifiers.
- Evaluation resources: Table 12 lists the evaluated models used in the experiments.
J Data Quality Control
AGENTBOARD uses verified annotations and interactive interfaces to support quality control across diverse, multi-round agent environments. Its benchmark covers embodied, game, web, and tool tasks with task-specific progress measurement.
- Annotation Verification: Three verification rounds review labeled sub-goals through self-checks, sampled inspections, and expert review.Inspectors interact with environments and flag unreasonable reward assignments for further discussion and possible annotation changes.
- Environment Coverage: AGENTBOARD includes nine environments spanning embodied, game, web, and tool-agent scenarios.The benchmark emphasizes planning and strategy in game tasks, including exploration-heavy environments such as Jericho.
- Environment Adaptation: ScienceWorld sub-goals incorporate necessary observations and distribute rewards uniformly throughout tasks.Tool and room restrictions help ensure that annotated sub-goals remain necessary for completing the final goal.
- Environment Adaptation: BabyAI is adapted from image observations and tensor actions to textual observations and textual actions for LLM evaluation.Its re-annotated sub-goals provide denser progress feedback than the original reward structure.
- Environment Adaptation: Jericho goals are rewritten to finish within 15 sub-goals, reducing the original games’ 50-300 interaction requirement.The revised tasks preserve exploration and world-modeling demands while fitting fixed-context LLM evaluation.
- Environment Coverage: WebShop evaluates product-search and purchase behavior using interaction with a website containing 1.18 million products.Agents use search and click actions to find products matching the instructions.
L Details of Progress Rate Metrics
The progress-rate framework decomposes goals into annotated sub-goals or matches current and target states, while adapting scoring to environments such as BabyAI, PDDL, and WebShop. These metrics provide denser or more structured estimates of intermediate progress than sparse final rewards.
- Unique Subgoal Sequences: Multiple inference trajectories can satisfy one sub-goal sequence, so progress-rate scoring does not require a single action path.The framework restricts examples with multiple different sub-goal sets for the same final goal, while allowing diverse trajectories within one set.
- Unique Subgoal Sequences: Single-set sub-goal adaptations affect less than 5% of AgentBoard problems, mainly simplifying multiple viable methods in BabyAI and ScienceWorld.The adaptations specify one method or path when diverse sub-goal sets would otherwise complicate progress measurement.
- Subgoal Annotation: Necessary observations are incorporated into sub-goals, and each environment’s labeled sub-goals are checked through interactive annotation.Annotators interact with environments and record observations needed to complete the task.
- ScienceWorld: ScienceWorld’s original multiple-tool layouts create multiple viable gold paths, making a definitive state-level progress rate difficult to assign.The task descriptions therefore restrict tool locations and methods to ensure unique goal paths and necessary observations.
- Subgoal-Based Progress: The revised BabyAI progress rate produces denser reward increases than the original score, allowing intermediate progress to appear sooner.The metric updates when agents make progress such as entering another room, finding a red ball, or picking it up.
- PDDL: PDDL matching compares properties in the current state with goal-state properties and reaches 100% only when all goal conditions are satisfied.A state matching one of two goal properties receives a matching score of 0.5.
- WebShop: WebShop progress combines URL matching for domain, query, and parameters with content matching across search, product, and order pages.The overall score is a weighted sum of URL and required-content matching components.