Source-linked AI summary
The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey
Tula Masterman, Sandi Besen, Mason Sawtell, Alex Chao
TL;DR
AI agent research needs a clearer account of how architectures support complex goals requiring reasoning, planning, and tool execution. This survey reviews single- and multi-agent designs, their patterns and divergences, and observations about architecture choices. It concludes that both patterns can perform strongly, while feedback, decomposition, iterative refinement, and role definition are recurring features of effective systems.
Problem
The survey addresses limited clarity about the capabilities, limitations, and design trade-offs of emerging AI agent architectures for complex goal execution.
Method
The paper surveys single- and multi-agent architectures, compares their design patterns, and synthesizes observations about leadership, communication, planning, execution, and reflection.
Results
Both single- and multi-agent architectures show compelling performance on complex goal execution, with clear feedback, task decomposition, iterative refinement, and role definition yielding improved performance.
Takeaways & Limitations
Architecture selection should depend on the broader use case, while effective systems commonly incorporate structured prompts, leadership or task division, planning and evaluation phases, feedback, dynamic teams, or message filtering.
Takeaways & Limitations
Multi-agent discussion does not necessarily enhance reasoning when an agent receives a sufficiently robust prompt.
Abstract
from arXiv · showhide
This survey paper examines the recent advancements in AI agent implementations, with a focus on their ability to achieve complex goals that require enhanced reasoning, planning, and tool execution capabilities. The primary objectives of this work are to a) communicate the current capabilities and limitations of existing AI agent implementations, b) share insights gained from our observations of these systems in action, and c) suggest important considerations for future developments in AI agent design. We achieve this by providing overviews of single-agent and multi-agent architectures, identifying key patterns and divergences in design choices, and evaluating their overall impact on accomplishing a provided goal. Our contribution outlines key themes when selecting an agentic architecture, the impact of leadership on agent systems, agent communication styles, and key phases for planning, execution, and reflection that enable robust AI agent systems.
1 Introduction
AI agents extend language models beyond zero-shot prompting through planning, iterative control, reflection, and tool use. The survey frames single- versus multi-agent architecture as a context-dependent design choice, with multi-agent systems organized along a vertical–horizontal spectrum.
- Agent capabilities: Agents use planning, loops, reflection, and tools to orchestrate complex tasks end-to-end.These capabilities support more general-purpose work than zero-shot prompting alone.
- Architecture choice: The survey centers on an ongoing debate over whether single- or multi-agent systems better solve complex tasks.The paper presents this as a current community debate rather than a settled conclusion.
- Single-agent architectures: Single-agent architectures perform reasoning, planning, and tool execution through one language model, with possible human feedback but no feedback from other agents.The agent receives a system prompt and the tools required for its task.
- Multi-agent architectures: Multi-agent architectures use two or more agents that may share or specialize models, tools, and personas.Their organizations range from simple to highly complex.
- Architecture taxonomy: Multi-agent systems span vertical structures with leaders and divided labor and horizontal structures with equal agents sharing discussion and initiative.The survey treats vertical and horizontal organizations as ends of a spectrum, with most architectures between them.
2 Key Considerations for Effective Agents
Effective agents need reasoning and planning to interpret complex tasks, adapt to feedback, and organize actions, while tool calling connects them to external information and services. Architectures address these demands through planning techniques, iterative reasoning, decomposition, and specialized agents working on subtasks.
- Reasoning and planning: Reasoning enables agents to make decisions, solve problems, interact with complex environments, and adjust plans when new information arrives.Without reasoning, agents may interpret queries literally or miss multi-step implications.
- Reasoning and planning: Planning approaches include task decomposition, multi-plan selection, external modules, reflection and refinement, and memory augmentation.These methods break tasks down, select or revise plans, or use external information to improve them.
- Reasoning and planning: Dedicated planning steps create plans before actions execute; PLaG represents plans as directed graphs and supports parallel execution of independent subtasks.The paper associates this design with performance gains on tasks benefiting from asynchronous execution.
- Effective tool calling: Tool calling lets agents retrieve or send information through external data sources and APIs, often alongside problems requiring complex reasoning.Agents may use multiple tools iteratively while solving decomposed subproblems.
- Multi-agent execution: Multi-agent patterns improve parallel-task handling by assigning smaller subtasks to agents that solve them independently with dedicated tools.Single-agent patterns can struggle with the long sequences required after decomposing complex tasks.
3 Single Agent Architectures
Single-agent methods combine reasoning, planning, action, tool use, memory, search, or reflection in different ways to improve goal execution. Across the surveyed approaches, effective planning and self-correction are central, while limitations include looping, hallucination, computational cost, and weak performance on complex tasks.
- Overview: The surveyed single-agent methods each dedicate a reasoning stage before action, covering ReAct, RAISE, Reflexion, AutoGPT+P, and LATS.These methods were selected for their contributions to agent reasoning and tool-calling capabilities.
- Cross-method findings: Successful single-agent goal execution depends on effective planning and self-correction, while single-agent architectures are especially useful for straightforward function calling without inter-agent feedback.Without self-evaluation and effective plans, agents may loop indefinitely or return results that fail to meet user expectations.
- ReAct: ReAct alternates thoughts, actions, and observations until task completion, improving effectiveness over zero-shot prompting and reducing HotpotQA hallucinations to 6% versus 14% for CoT.The recorded thought process also improves human interoperability and trustworthiness.
- RAISE: RAISE adds scratchpad short-term memory and similar-example long-term memory, improving longer-context retention and outperforming ReAct in efficiency and output quality.Fine-tuning produced the best performance for the reported task, even with a smaller model.
- AutoGPT+P: AutoGPT+P combines object detection, Object Affordance Mapping, LLM-driven tool selection, and classical PDDL planning for natural-language robot tasks.The LLM proposes goals and steps while the classical planner executes them, improving on purely language-model-based robotic planning approaches.
- LATS: LATS uses tree search, LM-based heuristics, state evaluation, and self-reflection from environmental and language-model feedback to improve performance across tasks.Its complexity increases computational-resource use and completion time, and evaluation has focused on relatively simple question-answering benchmarks.
4 Multi Agent Architectures
Multi-agent architectures use leadership, dynamic team composition, staged execution, and structured communication to coordinate complex goal completion. The surveyed examples show that organizational choices shape speed, communication cost, collaboration, and tool execution.
- Architectural patterns: Multi-agent teams divide labor by agent skill and dynamically reorganize members across planning, execution, and evaluation phases.Specialized agents can be added or removed as task needs change, matching roles and skills to each round of work.
- Leadership: Teams with an organized leader completed tasks nearly 10% faster than teams without a leader.The architecture combines a vertical leader component with horizontal communication among other agents.
- Leadership: Without a designated leader, agents spent approximately 50% of communication giving orders to one another.With a designated leader, 60% of the leader’s communication involved giving directions, allowing other members to focus more on exchanging and requesting information.
- Leadership: Dynamic teams with rotating leadership achieved the lowest average task-completion time and communication cost.A criticize-reflect step supports plan generation, performance evaluation, feedback, and team reorganization.
- Staged execution: AgentVerse organizes execution into recruitment, collaborative decision making, independent action execution, and evaluation.These phases can repeat until the overall goal is achieved, while recruitment adjusts team membership based on progress.
- Architectural patterns: Horizontal teams generally suit collaborative consulting tasks, whereas vertical teams better support isolated responsibilities for tool calling.This distinction links communication structure to the task’s need for collaboration or responsibility separation.
- Communication: MetaGPT reduces unproductive chatter by requiring structured outputs and using publish-subscribe information sharing.Agents share information centrally while reading only material relevant to their goals and tasks; MetaGPT outperformed single-agent architectures on HumanEval and MBPP.
5 Discussion and Observations
The survey finds that both single- and multi-agent systems can execute complex goals, with performance supported by decomposition, feedback, refinement, and clear roles. Architecture selection depends on task structure, collaboration needs, communication costs, and the availability of human oversight.
- Overview: Both single- and multi-agent architectures demonstrate compelling performance on complex goal execution.Across architectures, clear feedback, task decomposition, iterative refinement, and role definition are associated with improved performance.
- Architecture selection: Single-agent patterns generally suit well-defined tasks with narrowly defined tools and processes.They are easier to implement and avoid poor inter-agent feedback or distracting team chatter, but may become trapped in execution loops.
- Architecture selection: Multi-agent architectures suit tasks benefiting from feedback among personas and parallelization across distinct workflows.The survey notes that multi-agent patterns can perform better than single agents in such settings.
- Architecture selection: A robust prompt can make multi-agent discussion unnecessary for improving reasoning.Architecture choice should therefore reflect the broader use case rather than reasoning capability alone.
- Feedback and oversight: Human validation and feedback produce more reliable and trustworthy results by aligning responses with human expectations.Feedback also helps agents correct course after language models commit prematurely and begin diverging from the goal.
- Communication challenges: Extraneous multi-agent dialogue can impair reasoning, while missing information in vertical teams can cause confusion or hallucination.Explicit access-rights information and prompts discouraging unproductive communication are proposed responses.
- Roles and dynamic teams: Clear role definition keeps agents focused on their responsibilities, while dynamic teams ensure participating agents fit each work round.These design choices apply to both single-agent and multi-agent architectures.
- Key design elements: Effective agent teams benefit from clear leaders, defined planning and refinement phases, intelligent message filtering, and related coordination mechanisms.Single agents similarly benefit from defined personas and tools, human feedback, and iterative work toward the goal.
6 Limitations of Current Research and Considerations for Future Research
Current agent research is constrained by inconsistent evaluation, benchmark contamination and mismatch, uncertain real-world transfer, and safety concerns involving bias and harmful behavior. Future evaluation requires more robust, dynamic, environment-relevant, and human-involved benchmarks.
- Research limitations: Agent research faces major challenges in evaluation, overall reliability, and limitations inherited from language models.These constraints motivate broader consideration of how agent systems are assessed and deployed.
- Benchmarking: Agent benchmarks vary greatly, and team-specific benchmarks make comparisons across implementations difficult.Hand-crafted, manually scored evaluation sets may assess capabilities closely but lack the robustness of larger datasets and risk bias.
- Benchmarking: Modified benchmark questions can substantially reduce model performance, raising doubts about the authenticity of benchmark scores.This pattern supports concerns about data contamination in training data.
- Benchmarking: Static benchmarks can fail to match rapidly evolving model capabilities and may encourage memorization.Dynamic benchmarks resistant to simple memorization and synthetic benchmarks tailored to specific environments are proposed directions.
- Agent-specific evaluation: Single-iteration, no-tool benchmarks such as MMLU and GSM8K are poor proxies for multi-step agent capabilities.Agent evaluation needs measures covering reasoning over multiple steps, external information, and tools beyond training data.
- Real-world applicability: Performance on logic puzzles and video-game benchmarks may not translate clearly to noisy, broad real-world tasks.WildBench uses 570,000 real ChatGPT conversations, while SWE-bench provides narrower evidence for Python software engineering.
- Safety and bias: Agents have been shown to be less robust, more harmful, and capable of stealthier content than language models.As agent complexity increases, robust bias evaluation must include human evaluation despite the scalability challenge.
7 Conclusion and Future Directions
The survey concludes that both single- and multi-agent systems can handle complex multistep reasoning, planning, and tool-calling tasks, but the best architecture depends on the use case. Reliable deployment still requires stronger benchmarks, real-world evaluation, and mitigation of language-model biases.
- Conclusion: Single- and multi-agent patterns both demonstrate the ability to tackle complex multistep problems requiring advanced problem-solving skills.The survey emphasizes that architecture choice should vary according to the use case.
- Future directions: Reliable agents require progress on comprehensive benchmarks, real-world applicability, and mitigation of harmful language-model biases.The survey identifies these as near-term areas for future improvement.