Source-linked AI summary
AgentSquare: Automatic LLM Agent Search in Modular Design Space
Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, Yong Li
TL;DR
LLM-agent research is dominated by manually designed, task-specific systems that struggle to adapt to novel tasks. AgentSquare addresses this gap through Modularized LLM Agent Search, combining a four-module design space with evolution, recombination, and performance prediction. Across six benchmarks, it outperforms the best known human designs and yields interpretable design insights.
Problem
Current LLM-agent research relies on manually designed, task-specific systems that demand substantial expertise and labor and often struggle to adapt to novel tasks.
Method
AgentSquare searches a modular LLM-agent design space of Planning, Reasoning, Tool Use, and Memory through module evolution, module recombination, and an in-context performance predictor.
Results
Across six benchmarks, AgentSquare achieves an average performance gain of 17.2% over the best known human designs.
Takeaways & Limitations
The modular design space enables reuse of prior successful designs and supports interpretable insights into newly discovered agents.
Abstract
from arXiv · showhide
Recent advancements in Large Language Models (LLMs) have led to a rapid growth of agentic systems capable of handling a wide range of complex tasks. However, current research largely relies on manual, task-specific design, limiting their adaptability to novel tasks. In this paper, we introduce a new research problem: Modularized LLM Agent Search (MoLAS). We propose a modular design space that abstracts existing LLM agent designs into four fundamental modules with uniform IO interface: Planning, Reasoning, Tool Use, and Memory. Building on this design space, we present a novel LLM agent search framework called AgentSquare, which introduces two core mechanisms, i.e., module evolution and recombination, to efficiently search for optimized LLM agents. To further accelerate the process, we design a performance predictor that uses in-context surrogate models to skip unpromising agent designs. Extensive experiments across six benchmarks, covering the diverse scenarios of web, embodied, tool use and game applications, show that AgentSquare substantially outperforms hand-crafted agents, achieving an average performance gain of 17.2% against best-known human designs. Moreover, AgentSquare can generate interpretable design insights, enabling a deeper understanding of agentic architecture and its impact on task performance. We believe that the modular design space and AgentSquare search framework offer a platform for fully exploiting the potential of prior successful designs and consolidating the collective efforts of research community. Code repo is available at https://github.com/tsinghua-fib-lab/AgentSquare.
1 INTRODUCTION
AgentSquare frames Modularized LLM Agent Search as a modular alternative to manually designed, task-specific agents. Its framework combines standardized modules with evolution, recombination, and performance prediction, and outperforms human designs across six benchmarks.
- Problem: Current agent research relies heavily on manual, task-specific design that demands expert insight and human labor and often struggles to adapt to novel tasks.Prior approaches also have limited ability to recombine strengths from modules developed in separate agentic systems and codebases.
- Problem and design space: MoLAS automatically optimizes LLM agent designs by leveraging previously published or evaluated modules in a four-category modular design space.The categories are Planning, Reasoning, Tool Use, and Memory, connected through standardized input-output interfaces.
- Framework: AgentSquare searches this space through module evolution and module recombination, while a performance predictor skips unpromising candidates to reduce evaluation costs.Module evolution explores new modules through prompt-level optimization, and recombination searches for promising combinations of existing modules.
- Results: AgentSquare outperforms hand-crafted agents across six benchmarks, with an average performance gain of 17.2% over the best known human designs.The benchmarks cover web, embodied, tool use, and game scenarios.
- Motivation: The modular design space is intended to let researchers reuse successful prior designs and accumulate new discoveries rather than build isolated task-specific agents.The paper presents this as a platform for exploiting prior successful designs and consolidating research efforts.
2 A MODULAR DESIGN SPACE OF LLM AGENTS
The paper formalizes a modular LLM-agent design space around standardized interfaces, enabling reuse and recombination of prior agent components. Its workflow iteratively connects Planning, Reasoning, Tool Use, and Memory modules.
- 2.1 BACKGROUND: Standardized input-output interfaces enable module-level recombination, unlike operation-level frameworks that cannot fully exploit prior successful agent designs.The modularization is intended to support extensibility and seamless integration with existing designs.
- 2 A MODULAR DESIGN SPACE OF LLM AGENTS: The design space abstracts 16 popular LLM agents into 1050 possible combinations and can be extended when new modules are discovered.The space is designed to support recombination of existing agents and discovery of new ones rather than provide a one-size-fits-all design.
- 2.2 WORKFLOW OVERVIEW: The workflow begins with Planning, which decomposes a task into sub-tasks, then passes them sequentially to Reasoning for solution generation.Planning can use optional feedback information, while Reasoning processes each sub-task with optional feedback.
- 2.2 WORKFLOW OVERVIEW: Tool Use selects an appropriate predefined tool when reasoning encounters a problem requiring capabilities beyond the language model’s internal knowledge.The module maps a reasoning-derived problem and tool pool to a best-matched tool.
- 2.2 WORKFLOW OVERVIEW: Memory dynamically writes observations to and retrieves relevant knowledge from a memory database during reasoning.The memory module stores past thoughts, actions, and observations and retrieves knowledge relevant to the current situation.
3 AGENTSQUARE FRAMEWORK
AgentSquare searches modular LLM agent architectures by combining four standardized module types, evolving modules, recombining them, and predicting candidate performance to reduce evaluation cost.
- 3.1 PROBLEM FORMULATION OF MOLAS: AgentSquare formulates MoLAS as optimizing an agent A = (P, R, T, M) over Planning, Reasoning, Tool Use, and Memory modules with standardized interfaces.The objective is to maximize agent performance over the Cartesian product of four design dimensions.
- 3 AGENTSQUARE FRAMEWORK: The framework combines module recombination, module evolution, and an in-context performance predictor to search the modular design space efficiently.Its overall framework is presented as an automatic search process for the MoLAS optimization problem.
- 3.4 MODULE RECOMBINATION: Module recombination uses an LLM to propose promising agents by replacing modules from an initial configuration with alternatives drawn from the module pools and accumulated experience.The proposer considers the task description, module pools, and evaluated combination performance, then generates offspring for predictor-based evaluation.
- 3.5 MODULE EVOLUTION: Module evolution uses a module-programming LLM to create new modules from existing modules, task descriptions, and performance feedback, then real-tests resulting child agents.New modules are appended to the standardized pools, and the best-performing child initializes the subsequent recombination phase.
- 3.6 PERFORMANCE PREDICTOR: The performance predictor estimates novel recombination candidates from task descriptions, module profiles, and in-context results, reducing reliance on costly real-environment evaluations.Newly evolved modules remain subject to real-task testing, while recombination candidates use the predictor; predicted performance closely matches actual performance.
4 EXPERIMENTS
AgentSquare is evaluated across six tasks and four domains against hand-crafted and search-based baselines. It consistently improves performance, search efficiency, and cost-aware agent design, while ablations support the roles of recombination, evolution, and prediction.
- Main results: 14.1% to 30.6% improvements over the best human-designed agents across six tasks establish AgentSquare's strongest empirical result.The gains are 14.1% on Webshop, 26.1% on ALFWorld, 20.5% on SciWorld, 30.6% on M3Tool, 6.0% on Travelplanner, and 6.0% on PDDL.
- Search trajectory: AgentSquare follows a steady convergence trajectory, whereas random, Bayesian, and prompt-level searches encounter bottlenecks and sub-optimal architectures.Random recombination also outperforms prompt-level search, highlighting the value of searching the modular design space.
- Ablation study: Removing either module evolution or recombination reduces performance, with recombination having the larger impact across the six-task ablation.Recombination expands the search space and reduces local-optimum risk, while evolution discovers task-tailored modules.
- Ablation study: Predicted and actual agent performance closely align, while predictor evaluation costs only about 0.025% of full GPT-4o evaluation on ALFWorld.The predictor is validated on randomly sampled agents across all six tasks and supports cheaper search screening.
- Discovered modules: AgentSquare identifies task-adapted agents by combining existing modules with newly programmed ones, such as ALFWorld's Generative Agents memory with new planning and reasoning modules.The discovered planning module is named TD and the reasoning module SF-ToT.
5 RELATED WORK
Related work studies LLM-agent modules, multi-agent organization, and automated optimization, but lacks a unified modular design space and automatic design approach. AgentSquare addresses this gap by organizing agent construction around reusable modules.
- LLM-based autonomous agents: LLM agents commonly use planning, reasoning, tool-use, and memory modules, alongside research on organizing multiple agents.These module families and multi-agent systems form the main surrounding research areas.
- Unified agent design: Existing engineering resources still require human organization of modules and do not support optimization of the resulting agent.The paper contrasts these resources with conceptual unified-design frameworks and its own automatic modular search.
6 CONCLUSION
The paper introduces a modular design space and AgentSquare to automatically optimize LLM agents using prior modules, evolution, recombination, and performance prediction. This shifts research from isolated agent designs toward reusable modular search.
- Conclusion: AgentSquare combines module evolution, module recombination, and an in-context performance predictor to optimize agents within a four-module design space.The framework addresses the large search space by screening novel agents with a surrogate model.
- Conclusion: The modular design space lets researchers build on successful prior designs and accumulate interpretable agent-design insights.The paper frames this as a transition from studying individual agents to studying agents within a shared modular space.
A.1 EXPERIMENTAL SETUP
The experiments span six benchmark tasks across embodied, game, web, and tool domains, comparing AgentSquare with hand-crafted, module-search, prompt-search, and agent-search baselines. Setup details also include task metrics, model settings, cost analyses, search trajectories, predictor validation, and discovered modules.
- Task setup: Six tasks cover embodied, game, web, and tool applications: ALFWorld, ScienceWorld, PDDL, WebShop, TravelPlanner, and M3ToolEval.The tasks respectively test household interaction, scientific experimentation, strategic games, shopping, travel planning, and multi-tool interaction.
- Task setup: Evaluation uses task-specific metrics: success rate, task score, progress rate, and micro pass rate.Success rate applies to ALFWorld and M3ToolEval; task score to Webshop; progress rate to SciWorld and PDDL; micro pass rate to TravelPlanner.
- Baselines: Baselines include 12 hand-crafted agents, random and Bayesian module search, OPRO prompt search, and ADAS code-space agent search.These comparisons cover human-designed agents and three automated search families.
- AgentSquare setup: Experiments use GPT-3.5turbo-0125 and GPT-4o with equal few-shot examples, random initial module combinations, and termination after five non-improving iterations.The setup is designed to keep comparisons consistent across methods.
- Discovered modules: The searched modules include combined reasoning strategies, hierarchical memory, and tool-selection approaches that integrate multiple attempts or similarity retrieval.Examples combine ToT, Self-Consistency, and Self-Refine; organize memory hierarchically; or combine tool retrieval with repeated candidate generation.