PaperlayerView exact revision on arXiv

Source-linked AI summary

AgentBench: Evaluating LLMs as Agents

Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, Jie Tang

arXiv:2308.03688v3cs.AIcs.CLcs.LG

TL;DR

Existing benchmarks provide limited evidence about LLMs acting as agents in challenging interactive environments. AgentBench introduces an eight-environment benchmark and evaluates 29 models, finding strong performance from top commercial systems but a substantial gap versus many OSS models.

  • Problem

    A systematic benchmark is needed to quantitatively evaluate LLMs as agents across challenging interactive environments.

  • Method

    AgentBench evaluates LLM agents across eight code-, game-, and web-grounded environments using an integrated toolkit and Chain-of-Thought prompting.

  • Results

    Top commercial models show strong agent ability, while a significant performance gap separates them from many OSS competitors below 70B.

  • Takeaways & Limitations

    Poor long-term reasoning, decision-making, and instruction following are identified as major obstacles, while alignment data and instruction-following improvements are suggested directions.

  • Takeaways & Limitations

    The OSS evaluation includes only models smaller than 70B, and token counts use a simplified approximation because model tokenizers differ.

Abstract

from arXiv · show

The potential of Large Language Model (LLM) as agents has been widely acknowledged recently. Thus, there is an urgent need to quantitatively \textit{evaluate LLMs as agents} on challenging tasks in interactive environments. We present AgentBench, a multi-dimensional benchmark that consists of 8 distinct environments to assess LLM-as-Agent's reasoning and decision-making abilities. Our extensive test over \num API-based and open-sourced (OSS) LLMs shows that, while top commercial LLMs present a strong ability of acting as agents in complex environments, there is a significant disparity in performance between them and many OSS competitors that are no larger than 70B. We identify the typical reasons of failures in environments and LLMs, showing that poor long-term reasoning, decision-making, and instruction following abilities are the main obstacles for developing usable LLM agents. Improving instruction following and training on high quality multi-round alignment data could improve agent performance. And different from existing assumptions, training on code present ambivalent impacts on different agent tasks. Datasets, environments, and an integrated evaluation package for AgentBench are released at https://github.com/THUDM/AgentBench.

1 INTRODUCTION

AGENTBENCH addresses the lack of a systematic benchmark for evaluating LLM agents by testing them across eight real-world environments. Its evaluation reveals substantial differences between leading commercial and OSS models and identifies reasoning, decision-making, and instruction following as key weaknesses.

  • Benchmark motivation and design: AGENTBENCH standardizes LLM-agent evaluation across eight environments spanning code, games, and web tasks.The benchmark reformulates datasets as interactive environments for text-only autonomous agents.
  • Findings and implications: The benchmark identifies poor long-term reasoning, decision-making, and instruction following as typical causes of agent failures.
  • Findings and implications: Higher-quality alignment data and improved instruction following are highlighted as directions for improving LLM agents.
  • Benchmark motivation and design: 29 LLMs are evaluated, revealing a significant performance gap between leading API-based commercial models and OSS models below 70B.
  • Findings and implications: Code training has ambivalent effects, improving some agent tasks while harming others.

2 LLM-AS-AGENT: DEFINITION AND PRELIMINARY

The paper formalizes interactive LLM-agent evaluation as decision-making in partially observable environments. It uses primitive Chain-of-Thought prompting and categorizes common task termination reasons to diagnose agent weaknesses.

  • Evaluation framework: Interactive evaluation models LLM agents within a partially observable Markov Decision Process containing states, actions, transitions, rewards, instructions, and observations.
  • Reasoning strategy: AGENTBENCH adopts primitive Chain-of-Thought prompting with actions, without multiple trials, repeated generations, or complex reasoning strategies.
  • Failure analysis: The benchmark categorizes agent termination into five typical finish reasons, including context-limit, format, action, and task-limit failures, plus completion.
  • Failure analysis: Invalid-format and invalid-action failures mainly reflect poor instruction following, whereas task-limit failures often indicate weak multi-turn ability.

3 COMPOSITION OF AGENTBENCH: A BRIEF LOOK

AGENTBENCH combines code-, game-, and web-grounded interactive environments to evaluate diverse agent capabilities. Its tasks target coding, reasoning, strategic decision-making, commonsense grounding, and interaction with practical interfaces.

  • Environment composition: AGENTBENCH covers code-grounded, game-grounded, and web-grounded scenarios using Chain-of-Thought prompting.
  • Code-grounded environments: Code environments test operating-system manipulation, database analysis through SQL, and knowledge-graph interaction under partial observability.
  • Game-grounded environments: Game environments assess strategic play, lateral reasoning, instruction following, and commonsense understanding without requiring coding expertise.
  • Game-grounded environments: The Digital Card Game measures win rate while Lateral Thinking Puzzles probe unconventional questioning and reasoning.
  • Web-grounded environments: Web Shopping and Web Browsing evaluate autonomous searching, selection, clicking, typing, and other actions across complex websites.

4 EVALUATION OF AGENTBENCH

AGENTBENCH evaluates 29 LLMs through diverse interactive environments and a unified toolkit, revealing strong commercial-model performance but persistent OSS gaps and identifiable reasoning, formatting, and training-related failure patterns.

  • Evaluation Setup: The evaluation toolkit uses API-oriented model interaction and isolated task environments to simplify testing across diverse agent tasks.Complex environments are encapsulated in Docker images and tasks are subdivided to reduce configuration conflicts.
  • Main Results: gpt-4 achieves the best performance on 6 of 8 datasets and a 78% success rate on House Holding, while all API-based LLMs score above 1.00 overall.These results indicate that leading API-based models can handle a broad range of real-world environmental interactions.
  • Analysis: Task Limit Exceeded is the predominant execution failure, indicating weak long-term reasoning and decision-making in the evaluated LLM agents.Invalid Format errors are frequent in Database and Digital Card Game tasks, while Invalid Action errors occur often in House Holding and Web Browsing.
  • Analysis: Code tuning helps on relatively static procedures such as Web Shopping but can reduce performance on Digital Card Game and Operating System tasks requiring broader thinking.The comparison between codellama and llama-2 indicates a trade-off between procedural following and general reasoning.
  • Analysis: Vicuna-13b outperforms llama-2-13b and performs comparably to the three-times-larger codellama-34b, linking high-quality alignment data with stronger AGENTBENCH performance.Vicuna-13b was aligned using ShareGPT data generated by gpt-4 and gpt-3.5-turbo, whereas llama-2-13b was aligned from scratch.
  • Analysis: Llama-2-13b and llama-2-70b perform similarly despite their size difference, which the authors associate with insufficient pre-training or instruction alignment in llama-2-70b.Both models were pre-trained with 2T tokens, and the authors also report comparatively weaker instruction following for llama-2-70b.

5 RELATED WORK

Prior LLM-agent research spans text games, reasoning strategies, applications, and code-execution environments, but lacks a standard benchmark with broad task and model coverage.

  • LLM-as-Agent: Pre-LLM language-agent research commonly used text-game environments such as TextWorld, Jericho, and LIGHT with BERT-based models and reinforcement learning.
  • LLM-as-Agent: Chain-of-Thought and ReAct helped establish LLM agents that combine reasoning with actions, followed by additional reasoning strategies and multi-agent applications.
  • LLM-as-Agent: Existing LLM-agent research had limited datasets and models and lacked a standard, comprehensive benchmark before AGENTBENCH.
  • Evaluating LLMs in Executive Environments: Code-execution benchmarks such as APPS, HumanEval, and MBPP evaluate functional correctness rather than text similarity in executable environments.

6 CONCLUSION

AGENTBENCH is presented as a multi-dimensional benchmark and unified toolkit for evaluating LLM agents, addressing limitations of narrow or single-task evaluation frameworks.

  • Conclusion: AGENTBENCH establishes a multi-dimensional benchmark and unified testing toolkit for evaluating 29 LLMs as agents.
  • Conclusion: Its environments include code-grounded Operating System and Database tasks with bash and SQL action spaces.
  • Conclusion: Figure 4 presents examples of all AGENTBENCH environments.
  • Conclusion: Traditional single-round frameworks are not designed for complex multi-round interactions, while agent-based frameworks are often task-specific.
  • Conclusion: Existing agent-based frameworks may limit cross-task applicability, require same-device evaluation, and evaluate only one task with one agent at a time.

A.2 OUR DESIGNED EVALUATION FRAMEWORK

The evaluation framework separates task, agent, and evaluation components while supporting flexible multi-agent/task assignments. It uses network-flow scheduling to improve worker utilization and supports recovery of interrupted evaluations.

  • Framework architecture: The framework decouples Task Server, Agent Server, and Evaluation Client components, allowing separate deployment and HTTP communication across devices.This removes the requirement that task and agent components be co-located.
  • Framework architecture: Collaborative evaluation supports testing multiple agents and tasks in varied combinations simultaneously.This enables more comprehensive evaluation scenarios.
  • Operational features: The framework includes resumable evaluation, enabling interrupted evaluations to be recovered and continued.Together with its architecture and scheduling, this supports multi-round evaluation at scale.
  • Evaluation assignment: The Evaluation Client applies max-flow algorithms to allocate evaluation samples across agent-task pairs and maximize worker utilization.The graph contains agents, tasks, and weighted edges representing evaluation assignments.
  • Evaluation assignment: Edge capacities are updated after allocations and completed evaluations, while newly available evaluation triples trigger periodic reapplication of the algorithm.Flow values determine sample allocation for each agent-task pair.

B.1 DATASET DETAILS

The OS evaluation dataset combines human- and model-generated tasks in isolated Docker environments, evaluating both question answering and verifiable system operations. Each task uses interactive shell actions and a checking pipeline to determine success.

  • Task types: OS tasks include Question Answering, which requires final command outputs, and Operation, which requires verifiable system-state changes.A shared checking pipeline evaluates both task types.
  • Dataset construction: About half of the instructions are human-created or collected, while most of the remainder are GPT-4-generated QA problems filtered by unit tests.Human tasks originate from challenging Stack Overflow bash or shell problems selected and annotated by programming-focused annotators.
  • Dataset construction: The OS benchmark contains 144 high-quality, diverse evaluation samples with interactive environments and corresponding checking pipelines.Agents receive 1-shot chain-of-thought prompting to improve response formatting.
  • Evaluation procedure: Each problem proceeds through initialization, interaction, and checking, with failure occurring when the default interaction limit of 8 rounds is exceeded.Checking scripts process the model answer and intermediate outputs, and correctness requires every script to exit with code 0.
  • Interaction protocol: The interaction action space includes bash commands and commit or finish actions, with prompts requiring exactly one action per turn.The prompt exposes the task instruction and interaction trajectory, including truncated shell outputs that agents must handle.
  • Related dataset setup: The dataset also reuses established table-questioning datasets, augments rows, SQL queries, and wording with GPT-3.5-turbo, and evaluates success using a macro average across three categories.Each sample includes instructions, table metadata, table content, and a correct answer or modified-table hash.

C.4 STUDY ON BIAS IN DATA AUGMENTATION

A re-annotation study compared augmented data with the original data using GPT-4 and GPT-3.5-turbo. The scoring patterns remained similar across versions, suggesting limited augmentation-induced bias.

  • Bias validation: Re-annotated data tested on GPT-4 and GPT-3.5-turbo showed similar scoring patterns to the original data.The comparison is summarized in Table 5.
  • Bias validation: GPT-4 performed less effectively on UPDATE operations but showed enhanced proficiency in INSERT tasks in both data versions.Preserving these operation-specific relationships suggests the augmentation approach did not substantially alter them.

D.1 DATASET DETAILS

The KBQA dataset evaluates long-term planning through partially observable knowledge-base interactions. Agents autonomously select sequences of predefined tools to answer questions requiring multiple invocations.

  • Dataset construction: The dataset draws from GrailQA, ComplexWebQuestions, and GraphQuestions to evaluate LLM decision-making and long-term planning.Only questions requiring at least five tool invocations are retained.
  • Dataset construction: Each sample includes an input question, topic entities, a gold tool-action sequence, and a gold answer.Entity linking is omitted so the LLM can focus on planning.
  • Environment assumptions: The task is partially observable because describing the extensive knowledge graph directly to the LLM is infeasible.Unlike database tasks, the database particulars are not integrated into the input.
  • Evaluation setup: The evaluation hosts FREEBASE with Virtuoso and exposes APIs that simplify knowledge-graph querying instead of requiring the LLM to write SPARQL.Agents autonomously invoke tools, accumulate information, and predict a final answer.
  • Evaluation metrics: Performance is measured primarily with F1, alongside answer-set Exact Match and action-sequence Executability.These metrics assess both answer quality and whether generated tool sequences can run.
  • Tool interface: The tool interface supports relation discovery, neighbor retrieval, intersections, attribute lookup, extrema, and counting over entities or variables.Some tools impose sequencing constraints, such as requiring get_relations before get_neighbors and get_attributes before argmax or argmin.
  • Interaction protocol: Agents take one action at a time and may use at most 15 actions to find an answer.A teaching example is provided because zero-shot agents struggle to generate substantively relevant outputs.

E.1 DATASET DETAILS

Aquawar models a two-player hidden-identity fish battle in which agents select attacks, targets, and identity guesses across two game stages. The benchmark combines strategic rules, validity checks, baseline strategies, and gameplay metrics into a final reward.

  • Game Definition: Aquawar is a two-player battle game with four pet fish per team, each having health, attack power, active abilities, and passive abilities.The game uses a ten-fish card pool, with each player selecting four fish before play begins.
  • Game Definition: Each round combines identity assertion with selecting a living fish to perform a normal attack or active skill against an enemy fish.Correct identity guesses reveal the fish and damage the opponent’s living fish.
  • Evaluation Design: Two game stages separately remove or retain assertions, and final performance is the average across both stages.The first stage omits assertions, while the second preserves them to balance engagement and complexity.
  • Baselines: Random action and a greedy strategy prioritizing AOE attacks, one-hit kills, active skills, and normal attacks serve as baselines.The greedy strategy follows a fixed pattern but is not necessarily optimal.
  • Game Definition: Overlapping active and passive skills conceal fish identities and increase the game’s strategic aspects.Examples include shared Counter, Deflect, Heal, and Infight abilities across different fish.

F.1 DATASET DETAILS

The Lateral Thinking Puzzle environment evaluates agents as solvers that infer a hidden truth by asking constrained questions over multiple rounds. It measures progress, efficiency, relevance, and accuracy, while human validation exposes some automatic-evaluation tolerance.

  • Dataset and Roles: Each sample pairs a story with a truth and assigns one of four difficulty levels: easy, medium, hard, or expert.The solver must infer the truth from the story through interaction with a host.
  • Game Procedure: An LLM solver asks one question per round, receives host answers, and synthesizes responses to identify the hidden truth within a maximum round limit.Questions should be answerable with “Yes,” “No,” or “Irrelevant.”
  • Metrics: The evaluation measures Single Game Accuracy, Round Efficiency, Query Relevance, and Game Progress.Game Progress is the main metric and counts the proportion of truth key points reached by the agent.
  • Validation: Automatic evaluation can overestimate Single Game Accuracy and Query Relevance compared with human evaluation, especially for open-sourced models.Human validation checks milestone recognition and fact verification.
  • Evaluation Procedure: Key points are removed after being guessed, along with relevant inferences, to avoid repeated credit for the same deduction.The dataset stores summarized answer keys for each truth.

G.1 DATASET DETAILS

ALFWorld provides text-based household environments requiring agents to decompose high-level objectives into action sequences. AgentBench evaluates these interactions with one-shot examples and task-completion success rate across six task categories.

  • Environment: ALFWorld simulates household scenarios where agents execute multi-step decisions through textual environment interfaces.After each action, the environment returns feedback about changes or observations.
  • Environment: The agent receives an environment description, an objective, and simulated feedback after every action.Objectives commonly require exploration, such as putting a lamp on a table.
  • Dataset: The evaluation uses 134 solvable out-of-distribution problems across pick-and-place, cleaning, heating, cooling, looking, and picking two objects.These are the six categories used for the selected ALFWorld evaluation problems.
  • Evaluation Setup: A one-shot setup supplies one relatively simple successful training example for each task category before interaction begins.The setup follows ReAct prompts, and invalid outputs can be mapped to valid actions using BLEU similarity.
  • Evaluation Setup: Agents generate thoughts and actions from environment feedback, and overall Success Rate equals successfully completed tasks divided by total tasks.The interaction continues by feeding environmental changes or observations back to the model.

H.1 DATASET DETAILS

WebShop is a text-based shopping environment in which agents search and click through product pages to satisfy natural-language purchase instructions. Its matching reward scores attribute, option, price, and text alignment between the chosen and requested products.

  • Environment: WebShop represents a website with textual observations, clickable actions, and roughly one million products annotated with attribute labels.The environment is built from products scraped from amazon.com and linked to 12,087 human instructions.
  • Evaluation Setup: The first 500 of 12,087 instructions form the test set, following the official implementation.Each interaction round includes an instruction, an observation, and available actions.
  • Interaction: The agent alternates between searching and clicking while receiving updated webpage observations after each action.Actions use the forms search[keywords] and click[value].
  • Metric: Matching reward combines selected attributes, options, and price compatibility, with an additional reward-type component.The metric accommodates multiple suitable products for one query.
  • Metric: TextMatch measures overlap of pronouns, nouns, and proper nouns between the chosen product title and the goal product title.The reward therefore evaluates textual similarity alongside attributes and options.
  • Evaluation Setup: A one-shot example and strict Thought/Action formatting guide agents through the shopping interaction.The example demonstrates searching, selecting a product, choosing a size, and purchasing.

I.1 DATASET DETAILS

The section describes Mind2Web’s interactive web-agent setup, evaluation metrics, and selected findings on instruction following and agent planning. It also reports task-completion distributions for rounds and tokens.

  • Mind2Web dataset: Mind2Web targets cross-domain web interaction, using high-level goals and recorded expert action sequences across 912 tasks from 73 websites.The cross-domain set spans domains including housing, jobs, social media, education, health, government, and home services.
  • Evaluation setup: Agents receive webpage observations and action histories, then predict the next target element and operation; evaluation ranks candidates before prompting a multi-choice selection.Type and Select Options operations additionally require specifying the operation argument.
  • Metrics: Step Success Rate is the main metric because LLMs currently struggle to ensure overall task success across complete web tasks.Element Accuracy, Action F1, and Success Rate provide additional measures of element and operation correctness.
  • Instruction following: Open-sourced models show more invalid outputs than commercial API-based models, with Invalid Format at 10.4% versus 6.0% and Invalid Actions at 13.6% versus 4.6%.The comparison is presented as indicative of differences in robustness, generalization, or attention to instruction following.
  • Agent planning: GPT-4 maintained a coherent plan in House Holding by decomposing tasks into Find -> Clean -> Put, searching depth-first, backtracking, and revising assumptions when needed.By contrast, GPT-3.5-turbo could decompose the task but gradually lost sight of its initial plan after failed attempts.
  • Task completion distributions: Completed trajectories had a median of 6.0 rounds and 1850.0 tokens, while the middle 50% required 4.0–9.0 rounds and 761–2709 tokens.The analysis states that the vast majority of tasks finish within 3000 tokens.

J.2.4 PRIMARY CAUSE OF TASK LIMIT EXCEEDED: THE MODELS TEND TO REPEAT THE PREVIOUS CONTENT

Task Limit Exceeded is a predominant non-completion outcome, and the analysis identifies repeated model outputs as its most significant contributor. In database tasks, error feedback can support self-correction, but failures may persist across repeated SQL syntax errors.

  • Primary cause of Task Limit Exceeded: Task Limit Exceeded is a predominant factor in task non-completion, indicating weak reasoning and decision-making abilities.The analysis treats understanding TLE as important for explaining failures and improving performance.
  • Primary cause of Task Limit Exceeded: Models experiencing TLE average 25.5 interaction rounds, with most trajectories forced to terminate because of the round limit.
  • Primary cause of Task Limit Exceeded: The repetition analysis defines P(n, t) as the proportion of TLE trajectories containing two responses among the last n rounds with Rouge-L at least t.
  • Primary cause of Task Limit Exceeded: More than 90% of TLE trajectories contain repeated content within the last 10 rounds, including response pairs with Rouge-L scores of at least 0.8.
  • Influence of code tuning: Code tuning improves CodeLlama completion rates over Llama2 on WebShop tasks but can reduce performance on the more strategically demanding Digital Card Game.WebShop supplies a procedural one-shot template, whereas Digital Card Game requires assessing competitors and devising counter-strategies.
  • Self-correcting ability: In database tasks, models with self-correction ability significantly outperform others, although repeated SQL syntax errors can continue after error feedback.The example shows errors involving unescaped spaces in column and table names.
Loading 2308.03688v3…