BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents

Research gaps

• The paper identifies three gaps: uncertain optimal architectures, limited comparisons of LLM backbones, and little study of multi-agent orchestration.Architecture choice may depend on both the task and its LLM backbone; orchestration may be useful as task complexity increases, especially in open-domain environments.Source: §1 · pp. 1–2

Approach

• To address these gaps, the authors benchmark varied LAA architectures and LLM backbones on environments with different task-complexity levels, and propose BOLAA’s controller for selecting and communicating with labor agents.The benchmark is designed to analyze performance with respect to task complexity and to verify existing architectural design choices.Source: §1 · p. 2

Contributions and results

• The authors develop six LAA architectures spanning prompting, self-thinking, and planning, alongside BOLAA for orchestrating multiple agents.They report that BOLAA enhances solo agents’ action interaction ability.Source: §1 · p. 2
• BOLAA consistently yields the best performance among LAA architectures on WebShop, highlighting the importance of specialist-agent collaboration alongside highly generalizable LLMs.The paper reports experiments on decision-making web navigation and knowledge-reasoning environments, using final sparse rewards and intermediate recalls to inform architecture and compatible LLM choices.Source: §1 · p. 2

2.1 AUGMENTED LANGUAGE AGENT ARCHITECTURE

• CoT breaks challenging reasoning tasks into smaller steps, while ReAct maps observations to reasoning and action traces for interactive engagement with environments.Other approaches address feedback and refinement: Self-refine iteratively refines outputs, while REX and RAP use Monte Carlo Tree Search with environment rewards; Reflexion adds dynamic memory and self-reflection, and Retroformer uses policy-gradient optimization.Source: §2.1 · pp. 2–3

2.2 WEB AGENT

• Web-navigation actions include operations such as clicking, typing, and selecting; prior work includes data-driven agents, task decomposition, and realistic web-task simulation.MIND2Web fine-tunes LLMs on browser data, WebAgent generates executable Python for navigation, and WebArena supports realistic task simulation.Source: §2.2 · p. 3

2.3 TOOL AGENT

• Tool-agent work explores LLMs using external tools, including API-call generation and frameworks that support tool use across many APIs.ToolLLM includes Tool-Bench, an instruction-tuning dataset; work on tool documentation reports that documentation-based zero-shot prompts can rival few-shot prompts.Source: §2.3 · p. 3

3 AGENT ARCHITECTURES

• The architecture section compares solo agents with distinct environment-interaction strategies and presents BOLAA as its multi-agent orchestration design.Figure 1 depicts ZS-LAA, ZST-LAA, and ReAct; Figure 2 depicts PlanAct and PlanReAct; Figure 3 depicts BOLAA.Source: §3 · p. 3, Figure 1 · p. 4, Figure 2 · p. 5

3.1 SOLO AGENTS

• ZS-LAA uses a zero-shot prompt to generate actions, while ZST-LAA adds intermediate self-thinking and ReAct further adds few-shot examples.ZS-LAA requires no examples and is described as easy to generalize to new environments; ReAct’s examples support in-context learning but occupy context length.Source: Figure 1 · p. 4, §3.1 · p. 4
• PlanAct adds a few-shot planning flow before initial action generation, using a plan prompt informed by previously successful plans.The plan is retained and retrieved to construct prompts.Source: §3.1 · pp. 4–5
• PlanReAct extends PlanAct with self-thinking, which the authors say can alleviate hallucination from incorrect plans.Its self-think flow also enables Chain-of-Thought ability.Source: §3.1 · p. 5

3.2 BOLAA: ORCHESTRATING MULTIPLE AGENTS.

• BOLAA combines a labor-agent pool with a controller that selects an agent, constructs its message, parses its response into an executable action, and interacts with the environment.Agents may specialize in action types such as clicking or searching, and can also be designed for thinking or planning.Source: §3.2 · pp. 5–6, Figure 3 · p. 5

4.1 ENVIRONMENT BENCHMARK

• WebShop evaluates 900 shopping tasks across six instruction-complexity levels, while HotPotQA evaluates 300 questions sampled from easy, medium, and hard levels.WebShop agents use SEARCH and CLICK actions; HotPotQA agents use SEARCH, LOOKUP, and FINISH actions.Source: §4.1 · p. 6

4.2 EVALUATION METRICS

• WebShop reward measures attribute overlap with the target product, HotPotQA reward uses answer F1, and WebShop recall tracks whether the target product was retrieved.Recall is averaged across WebShop tasks.Source: §4.2 · p. 6

4.3 LLM UTILIZATION

• The evaluated backbones include open models from 3b to 70b and OpenAI API models, with varying maximum context lengths.The listed families include FastChat, Vicuna, Llama-2, MPT, XGen, LongChat, and GPT-3.5.Source: §4.3 · p. 6

4.4 DECISION-MAKING SIMULATION

• BOLAA performs best among the compared WebShop architectures, especially with high-performing LLMs, and its 3b fastchat-t5 pairing performs comparably to other architectures with more powerful LLMs.The authors suggest orchestrating smaller specialized agents when computing resources are limited.Source: §4.4 · p. 7, Table 1 · p. 7
• Architecture–backbone pairing matters: Llama-2-13b performs best with PlanAct, Llama-2-70b with BOLAA, and LongChat-13b-16K with PlanAct and PlanReAct.Increasing context length alone does not necessarily improve performance; the authors report more hallucinated generations over longer runs.Source: §4.4 · p. 7
• Zero-shot agents perform best for OpenAI API models, whereas less powerful LLMs need few-shot prompts; planning generally improves results for open-source LLMs but can hinder small models.FastChat-T5-3b performs above average with zero-shot prompting but degrades substantially under PlanReAct.Source: §4.4 · pp. 7–8
• In WebShop, BOLAA generally has the highest recall, while recall and reward are not tightly linked because correct clicking is harder than generating a search query.Llama-2-70b has recall 0.3344 and reward 0.0122 under the zero-shot architecture.Source: §4.4 · p. 8, Table 2 · p. 8

4.5 KNOWLEDGE REASONING SIMULATION

• In HotPotQA, ReAct performs best overall; planning can hinder performance, and larger models matter more than context length.Performance declines as question complexity rises; text-davinci-003 outperforms Llama-2-70b at each level, with a smaller gap on hard questions.Source: §4.5 · pp. 8–9, Table 3 · p. 10, Figure 6 · p. 10

• The paper systematically investigates LAA architectures paired with different LLM backbones.Source: §5 · p. 10
• BOLAA is proposed as a novel method for orchestrating multiple agents.Source: §5 · p. 10
• Benchmarking results provide experimental justification for LAA investigation and verify BOLAA’s potential benefits.Source: §5 · p. 10
• Designing BOLAA for environments with compounding actions remains a challenge.Source: §5 · p. 10
• Future work will explore autonomous LLM-based controller selection and communication with labor agents, and evaluate more architectures, LLMs, and environments.Source: §5 · p. 10