Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization

1 Introduction

• A single agent must manage the entire code-generation process, and limited context makes understanding, generating, and modifying extensive codebases difficult.As input token counts increase, inference performance decreases, according to the cited work (Levy et al., 2024; Shaham et al., 2023).Source: §1 · pp. 1–2
• SoA assigns different code parts or tasks to independent agents that generate and modify them, allowing the codebase to grow as agents are added.Agents automatically multiply according to problem complexity, while the code handled by each remains constant.Source: §1 · p. 2
• SoA outperformed Reflexion on HumanEval, demonstrating the framework’s effectiveness in generating and modifying code.The introduction also reports that agents multiply with problem complexity, scaling overall code volume while keeping code generation per agent constant.Source: §1 · p. 2
• Agents multiply according to problem complexity, scaling overall code volume while keeping each agent’s code-generation workload constant.The authors present these experimental findings as addressing scalability issues in single-agent approaches.Source: §1 · p. 2

2 Code Generation Task

• Pass@1 counts a problem as solved when any of the k generated code samples passes all test cases.The task supplies a docstring specifying input types, expected outputs, and function requirements; unit tests verify accuracy.Source: §2 · p. 2

3.1 Child Agent

• Child agents generate functions from docstrings, store implementations and unit tests, and revise code using test results and information from their Mother agent.Modification proposals combine the Mother agent’s state with test results and the latest implementation details.Source: §3.1 · pp. 2–3

3.2 Mother Agent

• Mother agents generate a function skeleton, determine subfunctions from problem complexity, and delegate their implementations to newly initialized agents.Agents can work asynchronously, and the Mother’s modification feedback and code changes are used by lower-level agents.Source: §3.2 · pp. 3–4

3 Self-organized Agent Framework

• SoA is a distributed framework in which independent agents repeatedly generate and modify functions, with Mother agents decomposing complex problems into delegated subtasks.The hierarchy allows the number of agents, and therefore total generated code, to scale while each agent manages a small amount.Source: §3 · p. 2, §3.3 · p. 4

3.3 Self-organized Agent Process

• Generation begins with a root Mother that delegates subtasks through a tree, then combined code is tested and feedback propagates downward for iterative modification.The process ends after a predetermined maximum number of iterations.Source: §3.3 · pp. 4–5

4 Experiments

• The evaluation uses Python HumanEval, with six unit tests for Reflexion and one for SoA, and configures SoA with a maximum of eight learning-loop iterations and tree depth two.GPT-3.5-turbo is used for code and feedback generation; GPT-4 was not selected because of experimental cost.Source: §4 · p. 5
• Figure 4 depicts Mother agents delegating function-generation and modification tasks through a hierarchy of Mother and Child agents, each independently implementing a function.The baselines include single-agent and multi-agent systems, with and without self-debugging capabilities.Source: §4 · p. 5, Figure 4 · p. 6

4.1 Main Results

• SoA outperforms Reflexion by 5% in Pass@1 accuracy on HumanEval.Table 1 reports results for SoA and baselines on HumanEval.Source: §4.1 · p. 5, Table 1 · p. 6

4.2 Analysis

• SoA produces more tokens in final code and more average characters per function than Reflexion, despite each SoA agent handling significantly less code.The analysis measures pure code after removing comments and docstrings; it presents greater overall output as evidence of scalability.Source: §4.2 · p. 6, Figure 5 · p. 7

Multi-Agent Collaboration for Software Development

• Unlike role-based software-development agents such as MetaGPT and ChatDev, SoA subdivides capabilities by code functionality and dynamically scales agents with problem difficulty.The authors describe this as a flexible alternative to assigning roles such as programmers, project managers, or QA engineers.Source: §5 · p. 7

Macro vs. Micro Perspectives

• SoA focuses on elemental code generation and modification, complementing other multi-agent methods’ focus on project management and task allocation.The paper presents macro- and micro-level approaches as compatible.Source: §5 · p. 7

Prompt Engineering

• Unlike ToT and SoT, which structure reasoning or answer generation, SoA uses a tree of agents to collaborate on code generation and modification.ToT represents reasoning steps as nodes; SoT generates an answer skeleton and completes it in parallel.Source: §5 · pp. 7–8

• Agents independently generate and modify code components, collaborate on the overall codebase, and multiply automatically with problem complexity while keeping each agent’s code amount constant.The authors state that total code volume can increase indefinitely with the number of agents.Source: §6 · p. 8
• SoA achieved a 5% improvement in Pass@1 over Reflexion on HumanEval, while each agent handled less code than the single-agent baseline despite greater total generated code.Source: §6 · p. 8
• SoA’s performance may depend on the LLM and unit-test quality, and its effectiveness on more complex real-world projects remains uninvestigated.The authors also identify agent communication and collaboration efficiency and fault tolerance as areas for further optimization.Source: §6 · p. 8
• The authors see significant potential for SoA in future automatic software-development research and development.Source: §6 · p. 8

• The root Mother agent creates a skeleton from docstrings and unit tests, then recursively assigns subtasks to Mother or Child agents according to hierarchy depth.Source: A Pseudocode · p. 11
• Agents generate code for their assigned subtasks, while the root combines implementations and evaluates the final implementation with unit tests in each iteration.Source: A Pseudocode · p. 11
• Modification uses test results and upper-agent observations to update code recursively through subagents; the process returns the combined final implementation after the specified iterations.Source: A Pseudocode · p. 11