EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms

1 Introduction

• Handcrafted roles, task scopes, skills, and prompts require costly human effort, limiting the scaling of multi-agent systems.The paper identifies automatic, generic agent generation as a critical challenge.Source: §1 · pp. 1–2
• EvoAgent treats a specialized agent’s settings as variables to evolve, using selection, crossover, and mutation to generate multiple agents.The method is described as a one-shot way to extend an existing framework without extra human design.Source: §1 · p. 2
• Experiments report that EvoAgent generates agents with diverse skills and consistently improves performance across different scenarios.The evaluated settings include knowledge-based question answering, multimodal reasoning, interactive scientific solving, and real-world planning.Source: §1 · p. 2
• EvoAgent is presented as a generic method applicable to any agent framework for automatically generating specialized agents.The paper also reports exploring scalability in conversational scenarios such as debate.Source: §1 · p. 2

2 Related Work

• Many LLM agent frameworks extend to multi-agent collaboration by designing multiple specific roles, but still rely heavily on handcrafted designs.The paper notes that such human effort can limit adaptability and flexibility when handling unexpected challenges.Source: §2 · p. 3
• Persona-based agent methods often assign roles manually, while some automatic systems still depend on human-designed interventions and fixed architectures.AutoAgents uses a Planner–Agent Observer–Plan Observer framework, and AgentVerse uses a specified recruitment-to-evaluation pipeline.Source: §2 · p. 3
• EvoAgent uses evolutionary algorithms to generate agents with diverse settings and is described as adaptable to various agent frameworks.The paper contrasts this flexibility with the architecture constraints of AutoAgents and AgentVerse.Source: §2 · p. 3

3 Method

• EvoAgent draws on the idea that diverse individuals can combine specialized expertise, and formulates agent generation as an evolutionary process without human labor.Each specialized agent is treated as an individual, with skills as the parts to evolve.Source: §3 · p. 3, §3.2 · p. 3

3.1 Preliminary

• Evolutionary algorithms model reproduction, mutation, recombination, and selection, and can serve as optimization methods for individuals.The method treats EAs as nonparametric and applicable across frameworks when the evolved parts and operators are defined.Source: §3.1 · p. 3

3.2 EVOAGENT

• EvoAgent uses language models to update agent settings and generate agents with distinct skills, treating each skill as an evolvable part.Settings can cover role-playing, planning, tools, and other capabilities.Source: §3.2 · p. 3
• The algorithm specifies an initial agent, population size, iteration count, quality-check module, and crossover, mutation, and update operations.The described pipeline also includes natural-selection-style integration of results.Source: §3.2 · p. 4
• Crossover updates parent settings based on generated results, while mutation creates child agents with new characteristics and selection retains high-quality agents.The pipeline integrates results from selected agents as a natural selection process.Source: §3.2 · p. 4

7 end

• Initialization starts from a predefined agent framework and specifies which parts of it should be upgraded.The paper names MetaGPT and AutoGen as examples of starting frameworks.Source: §7 · p. 4
• Crossover uses parent agents’ results to identify skills for improvement and update their settings; mutation makes child agents distinct while maintaining task-solving capability.The paper describes these operators as generating effective and diverse agents.Source: §7 · p. 4
• An LLM quality-check module checks whether generated agents inherit characteristics and remain distinct from their parents, then selects N child agents per iteration.Source: §7 · p. 4
• Each child agent generates candidate results, which are integrated with the previous iteration’s result into a new result.The paper characterizes this as a natural-selection-like stage.Source: §7 · p. 4
• Repeating crossover, mutation, selection, and result updating generates more agents until the target number is reached, without extra human design.The method is described as applicable to any agent framework without prerequisites.Source: §7 · p. 5

4.1 NLP and Multi-Modal Tasks

• On MMMU, SR and SPP trail CoT, while EVOAGENT improves multimodal models across difficulty levels; it also improves NLP reasoning and beats AgentVerse and AutoAgent.The experiments use NLP knowledge and reasoning tasks alongside MMMU.Source: §4.1 · p. 6, Figure 2 · p. 6

4.2 Interactive Scientific Solving Simulation

• In ScienceWorld, EVOAGENT consistently improves LLM performance on dynamic, open-world scientific tasks, with larger gains on short trajectories than on medium or long ones.The authors describe this as generalizing EVOAGENT to interactive open-world tasks.Source: §4.2 · p. 7

4.3 Real-World Scenarios

• On TravelPlanner, generated culinary and transportation specialists yield plans better aligned with user preferences and commonsense, while existing prompting strategies remain insufficient for complex planning.The authors compare EVOAGENT with Self-Refine and SPP.Source: §4.3 · p. 8

4.4 Ablation Studies

• Ablations favor EVOAGENT over Overgen and prompt refinement, but suggestions and larger populations or iteration counts can trade commonsense adherence for user-preference alignment.PK performs better without quality checks, whereas All-in performs best when quality checks are included.Source: §4.4 · pp. 8–9

• EVOAGENT is an automatic multi-agent generation system that uses evolutionary algorithms.Source: §5 · p. 9
• The method applies to existing agent frameworks and extends them into systems with diverse, effective agents through mutation, crossover, and selection.Source: §5 · p. 9
• Experiments on multiple tasks show that EVOAGENT can significantly improve LLM-based agents’ capabilities on complex tasks.Source: §5 · p. 9

• Generating multiple specialized agents requires more tokens than using a single agent.Source: Limitations · p. 9
• Comparisons with existing multi-agent frameworks are limited, apart from AgentVerse and AutoAgents, because designing benchmark-specific frameworks is beyond the paper’s scope.Source: Limitations · p. 9
• The authors do not manually evaluate generated-agent quality, although they report performance improvements on complex tasks.Source: Limitations · p. 9

Ethics Statement

• The authors state that all authors follow the ACL ARR Code of Ethics and the Code of Conduct; they cannot guarantee public benchmarks are free of harmful or toxic language.They also used ChatGPT to correct grammatical errors.Source: Ethics Statement · p. 9

A.1 Prompt for Baselines and EVOAGENT

• The appendix lists prompts for Direct, CoT, Self-Refine, SPP, EVOAGENT, and its ablation settings.Source: §A.1 · p. 13

A.2 Model Selection

• OpenAI models use specified Azure versions, Gemini-Pro uses its API, and all models run at temperature 0.Source: §A.2 · p. 13

A.3 Human-designed Agent Framework

• AutoAgents and AgentVerse automate agent generation but retain human-designed framework structures, which the paper argues limit scalability and functionality.Source: §A.3 · p. 13

A.4 Experimental Details of ScienceWorld

• For ScienceWorld, REACT gives each LLM a virtual “think” action for generating subgoals during action planning.Source: §A.4 · p. 13

A.5 Evaluation Details of TravelPlanner

• TravelPlanner evaluates plans against user preferences and commonsense rules on 180 user queries with collected information.The setting allows up to 30 steps for plan completion.Source: §A.5 · p. 13
• Delivery Rate measures completion within the step limit, while Final Pass Rate measures viable plans meeting all criteria.Failures on delivery include dead loops, repeated failed attempts, or exceeding the step limit.Source: §A.5 · p. 13
• Commonsense and Hard Constraint Pass Rates assess whether plans incorporate commonsense and meet explicit query constraints, respectively.Hard Constraint Pass Rate tests adaptation to diverse user preferences.Source: §A.5 · p. 13
• Micro scoring measures the ratio of constraints met, whereas macro scoring measures the share of plans meeting all constraints.Both strategies apply to Commonsense and Hard Constraint Pass Rates.Source: §A.5 · p. 13

B.1 Time Efficiency of EVOAGENT

• The time-efficiency experiments use Llama-3.1-70B-Instruct on Logic Grid Puzzle, Trivia Creative Writing, and Codenames Collaborative.The model is deployed on an 8-GPU RTX 3090 setup using bf16.Source: §B.1 · p. 13
• EVOAGENT achieves the best performance at similar time costs to AgentVerse and AutoAgents, while taking more time than a single-agent approach.The comparison uses Logic Grid Puzzle, Trivia Creative Writing, and Codenames Collaborative.Source: §B.1 · pp. 13–14, Table 5 · p. 14

B.2 More Analysis of Ablation Studies

• Removing the quality-check module significantly lowers delivery pass rate under the All-in strategy.Unsuitable agents can produce overly lengthy plans that exceed the LLM context window and prevent final-result generation.Source: §B.2 · p. 14
• Trivia Creative Writing performance improves with more iterations but plateaus beyond three.The authors suggest that simpler traditional NLP tasks yield similar agents after enough iterations, even with quality checking.Source: §B.2 · p. 14

C.1 EVOAGENT Examples of NLP reasoning and knowledge tasks

• The Logic Grid Puzzle, Trivia Creative Writing, and Codenames Collaborative examples are presented as generated multi-agent cases.Source: §C.1 · p. 14

C.2 EVOAGENT Examples of MMMU

• The appendix presents generated-agent examples on the MMMU dataset.Source: §C.2 · p. 14

C.3 EVOAGENT Examples of ScienceWorld

• The appendix presents generated-agent examples on ScienceWorld.Source: §C.3 · p. 14

C.4 EVOAGENT Examples of TravelPlanner

• The appendix presents generated-agent examples on TravelPlanner.Source: §C.4 · p. 14

D Examples of EVOAGENT’s Adaption to Multi-agent Collaboration Frameworks

• The method is applied to existing frameworks to extend them into multi-agent applications and scale the number of agents.The passage describes this as an extension of prior experiments across scenarios.Source: D Examples of EVOAGENT’s Adaption to… · p. 14

D.1 EVOAGENT for MetaGPT

• EVOAGENT extends debate teams with diverse agent settings, increasing opinion variety and debate quality.The MetaGPT example begins with two debaters holding different opinions and producing dull, repetitive content.Source: §D.1 · pp. 14–15
• The accompanying tables cover popularity-selection strategies on TravelPlanner and GPT-3.5 performance across Trivia Creative Writing iterations.Source: Table 6 · p. 15, Table 7 · p. 15

D.2 EVOAGENT for Camel

• EVOAGENT can automatically produce agents from Camel AI assistants for interaction with AI users, bypassing manual role design.Camel initially assigns assistant and user roles through human design.Source: §D.2 · p. 15

D.3 EVOAGENT for AutoGen

• EvoAgent extends a single AutoGen assistant into multiple expert roles, increasing the number of agents in group chats without manual role design.AutoGen uses a user proxy and a group-chat manager to select speakers, gather responses, and disseminate messages.Source: §D.3 · p. 15
• The collaboration prompts call for multiple rounds of critical feedback and suggestions, with the assistant refining its answer while deciding whether to accept an expert’s input.The instructions caution that an expert’s answer may be incorrect, so the assistant should assess it critically.Source: §D.3 · pp. 18–19
• New experts are generated through crossover and mutation, then retained only if they are distinct from existing agents and can effectively assist with the task.Crossover updates skills in prior agents; mutation creates distinct agents while maintaining task-solving capability.Source: §D.3 · p. 19
• In a logic-grid example, an expert supplies deductions and the assistant’s final answer identifies house 1 as the location of the person wearing boots.The example shows the expert giving a hint and the assistant revising its solution before providing the final choice.Source: §D.3 · p. 21
• In a trivia creative-writing example, specialized experts contribute historical and sports references that are incorporated into a Zootopia story.The story includes references to Thatcher, snooker player Cliff Thorburn, cricketer Michael Holding, EastEnders, and Minder.Source: §D.3 · p. 22
• For a fruit-association task, two expert outputs and the assistant’s final response select banana, peach, and bowl.The assistant adopts the shared selection after considering the experts’ reasoning.Source: §D.3 · p. 23
• In the MMMU geometry example, generated mathematician and astronomy agents explain why the Sun could be at focus B and answer E, following an earlier answer of C.The mathematician invokes the definition of an ellipse, while the astronomer refers to orbital eccentricity and Kepler’s laws.Source: §D.3 · p. 24
• For ScienceWorld, EvoAgent generates specialist prompts for determining a fork’s conductivity, boiling gallium, and changing gallium’s state of matter.The example assigns thermodynamics, metallurgy, and chemistry expertise to these tasks, respectively.Source: §D.3 · p. 25
• For travel from Buffalo to Atlanta, the paper presents a three-day itinerary combining culinary, logistics, and tourism expertise with flight details, meals, attractions, and accommodation.Source: §D.3 · p. 26