Source-linked AI summary
Agensh: Scaling Organizational Intelligence to 1,024 Agents
Zhihao Zhan, Ting Song, Li Dong, Shaohan Huang, Jianxun Lian, Yan Xia, Furu Wei
TL;DR
Existing multi-agent harnesses often depend on a central orchestrator whose capacity limits coordination. Agensh replaces that structure with asynchronous, self-organized workers and shared coordination infrastructure; on the five hardest ProgramBench tasks, scaling from 1 to 128 agents raises mean final test-pass rate from 19.31% to 28.78%.
Problem
Existing orchestrator-worker harnesses depend on a main agent to allocate and coordinate work, and their scalability is constrained by the orchestrator’s capacity.
Source: §1 · p. 2Method
Agensh uses an asynchronous cooperation loop and shared workspace, message interface, and context so workers can self-assign sub-tasks, exchange findings, and integrate progress.
Source: §1 · p. 2, §2.2 · p. 4Results
19.31% to 28.78% mean final test-pass rate: scaling from 1 to 128 agents on the five hardest ProgramBench tasks yields an approximately 49% relative improvement.
Source: §3.1 · p. 6, §5 · p. 8Takeaways & Limitations
Agensh identifies agent count as a scaling dimension for multi-agent organizations and offers a practical approach for complex tasks under hard latency constraints or time budgets.
Source: §1 · p. 3
Abstract
from arXiv · showhide
A multi-agent system can reduce latency on complex tasks by executing work concurrently. Several pioneering harness frameworks support multi-agent systems. However, the scalability of current multi-agent harnesses is often constrained by a central orchestrator's capacity to allocate tasks and coordinate workers. To address this limitation, we introduce Agensh, a scalable self-organized multi-agent harness without a central orchestrator: concurrent workers execute a multi-agent cooperation loop, continuously gathering context, claiming and self-assigning sub-tasks, taking action and sharing findings, verifying results, and merging progress in an asynchronous manner. The loop is supported by the agentic organization infrastructure comprising three components: a shared workspace holds proposed, ongoing, and completed work; a message interface lets workers communicate; and shared context retains reusable findings and work intentions. To test the scalability of Agensh, we evaluate it on the five hardest ProgramBench tasks with GPT-5.6-sol (high). Scaling from 1 to 128 agents raises the mean final test-pass rate from 19.31% to 28.78%, an approximately 49% relative improvement. Larger organizations reach comparable test-pass rates earlier. On pandoc, scaling from 1 to 1,024 agents raises the final test-pass rate from 33.89% to 55.06%. Worker trajectories further show that different forms of self-organized cooperation gradually emerges and standardizes as the organization grows. These results reveal the number of agents as a new scaling dimension for multi-agent organizations to expand the frontier of general intelligence, offering a practical solution for complex tasks under hard latency constraints or time budgets.
1 Introduction
Agensh addresses limits in orchestrator-centered multi-agent harnesses with self-organized workers coordinating asynchronously through shared infrastructure. On difficult software-reproduction tasks, scaling the organization improves final test-pass rates and can bring comparable performance earlier.
1 Introduction
- Existing orchestrator-worker harnesses rely on a main agent to decompose and assign tasks, leaving overall scalability constrained by its coordination capacity.Single-agent systems also face limits from one context, action, and memory stream, with sequential execution imposing high latency on complex tasks.Source: §1 · p. 2
- Agensh removes the central orchestrator and uses concurrent, asynchronous workers with shared workspace, messaging, and context to coordinate their work.The workspace holds ongoing and integrated work, messaging supports announcements and direct communication, and shared context retains reusable findings and work intentions.Source: §1 · p. 2
- Workers repeatedly gather context, claim sub-tasks, act, verify results, and merge progress, sharing findings and continuing asynchronously.On pandoc, the final test-pass rate rises from 33.89% with 1 agent to 55.06% with 1,024 agents; trajectories show broader cooperation forms as organizations grow.Source: §1 · pp. 2–3, Figure 3 · p. 3
- 19.31% to 28.78% mean final test-pass rate: scaling from 1 to 128 agents improves results across the five hardest ProgramBench tasks.The increase is an approximately 49% relative improvement; larger organizations also reach comparable test-pass rates earlier.Source: §1 · pp. 2–3
2 Multi-Agent Organization Harness
Agensh organizes concurrent workers through a repeated cooperation loop and shared infrastructure for work, communication, and reusable findings. Its implementation layers organization-level coordination over a single-agent harness.
2.1 Multi-Agent Cooperation Loop
- Each worker asynchronously gathers context, claims a sub-task, acts, verifies it against acceptance criteria, and merges its contribution into shared work.Workers resolve overlapping claims through messages, publish useful findings, and revisit the loop using newly integrated work and peer updates.Source: §2.1 · pp. 3–4
2.2 Agentic Organization Infrastructure
- Three mechanisms coordinate workers: a shared workspace for ongoing and integrated work, a message interface, and shared context for findings and work claims.The workspace supports concurrent contributions and merging; messaging handles announcements and direct communication, while context entries preserve findings and active claims.Source: §2.2 · pp. 4–5, Figure 4 · p. 4
- 19.31% to 28.78% mean final test-pass rate: scaling from 1 to 128 agents on the five hardest ProgramBench tasks improves results under a 6h budget.The figure reports the mean final test-pass rate for the five tasks.Source: Figure 5 · p. 5
2 Multi-Agent Organization Harness
- Together, the coordination mechanisms connect local work to shared state, so workers can reuse findings, address overlap, and integrate contributions.The infrastructure makes accumulated work, findings, and messages available throughout the organization.Source: §2 · p. 3, §2.2 · p. 5
2.3 Implementation
- Agensh layers organization-level coordination over a single-agent harness, while workflow instructions implement the cooperation loop and adapters support different underlying harnesses.The single-agent harness owns each worker’s local loop; Agensh supplies worker identity, coordination protocols, shared services, and recovery and liveness mechanisms.Source: §2.3 · p. 5
3 Scaling Multi-Agent Organization to 1,024 Agents
Scaling agent count improves test-pass rates and shortens time to comparable performance; recorded trajectories show cooperation broadening from peer coordination to specialized roles and standardized workflows.
3 Scaling Multi-Agent Organization to 1,024 Agents
- The evaluation uses the five hardest ProgramBench tasks, requiring reconstruction of reference software within a 6h budget without Internet access.The selected systems span multimedia processing, molecular simulation, document conversion, language interpretation, and code indexing.Source: §3 · p. 6
3.1 Main Results
- Mean final test-pass rate across five tasks rises from 19.31% with 1 agent to 28.78% with 128 agents, a 9.47-point gain.The reported gain is approximately 49% relative, with the agent model, underlying harness, and 6h budget held fixed.Source: §3.1 · p. 6
- On pandoc, 128 agents exceed a 30% test-pass rate at 30 minutes, versus 60 minutes for 32 agents and 90 minutes for 8 agents.The single-agent run stays below that threshold throughout the first two hours; Figure 6 covers the first two hours of the 6h runs.Source: §3.1 · p. 6, Figure 6 · p. 7
- On pandoc, final test-pass rate rises from 33.89% with 1 agent to 55.06% with 1,024 agents.The 1,024-agent result is 4.12 percentage points above the 128-agent result.Source: §3.1 · p. 6
3.2 Self-Organized Cooperation Emerges at Scale
- At 8 agents, workers coordinate interfaces and resolve overlapping claims; at 32, peers jointly review, correct, and integrate contributions.Examples include agreed module interfaces in gromacs, and a PHP-src contribution corrected after a peer found a counterexample.Source: §3.2 · p. 7
- At 128 agents, workers choose reviewers by experience and reuse relationships; pandoc peers also negotiate and reuse integration workflows.After failures, workers revised a protocol to let a peer perform the update, test, check, and merge cycle.Source: §3.2 · p. 7
- At 1,024 agents, specialized workers serve as integrators or take over failed work, avoiding dependence on any individual worker.Workers can contact several candidate integrators, select the first valid responder, and cancel other requests.Source: §3.2 · p. 7
- As organizations grow, cooperation broadens from coordinating implementation to integration management, standardized workflows, and specialized roles, all established through peer interaction.Workers choose collaborators, divide responsibilities, and establish or revise workflows themselves.Source: §3.2 · pp. 7–8
4 Related Work
Prior harnesses commonly rely on a lead agent or orchestrator to assign and coordinate work, while related systems explore shared context, peer communication, and shared workspaces. This paper studies workers sharing responsibility for task discovery, dependency coordination, and integration.
Multi-agent organization and cooperation harnesses
- Claude Code, Codex, and Kimi Agent Swarm use lead-centered or orchestrator-worker structures to delegate, schedule, or consolidate concurrent work [9].The discussion contrasts these structures with persistent self-organized worker collaboration.Source: §4 · p. 8
- This work studies persistent cycles in which self-organized workers share responsibility for discovering tasks, coordinating dependencies, and integrating results.The comparison distinguishes this organization structure from harnesses that route task assignment or consolidation through a lead.Source: §4 · p. 8
Shared context and decentralized coordination
- Related systems combine asynchronous workers and shared context, peer communication tools, or shared workspaces for agent coordination [14] [21] [24].Examples include repository-based coordination without an orchestration agent and a shared workspace designed to detect stale or conflicting edits.Source: §4 · p. 8
5 Conclusion
Agensh combines a cooperation loop with shared infrastructure so workers can integrate progress without a central orchestrator. Experiments report higher test-pass rates and earlier comparable performance as agent count grows, alongside broader self-organized cooperation.
5 Conclusion
- Agensh couples a multi-agent cooperation loop with a shared workspace, message interface, and shared context for asynchronous worker collaboration.Workers discover and claim sub-tasks, exchange findings, and integrate contributions into shared work and knowledge.Source: §5 · p. 8
- With the same model and 6h budget, mean final test-pass rate across five ProgramBench tasks rises from 19.31% with 1 agent to 28.78% with 128 agents.The paper reports an approximately 49% relative improvement; on pandoc, the rate rises from 33.89% with 1 agent to 55.06% with 1,024 agents.Source: §5 · p. 8
- Larger organizations reach comparable test-pass rates earlier, while trajectories show cooperation broadening to standardized workflows and organization-scale role specialization.The reported forms range from peer coordination and multi-worker integration to specialized roles.Source: §5 · p. 8
- The results present agent count as a scaling dimension for multi-agent organizations and a practical approach to complex tasks under hard latency constraints or time budgets.The paper also describes this as a path to expand the frontier of general intelligence.Source: §5 · p. 8
A Worker Prompt
The worker prompt directs autonomous agents to reproduce a reference program from observed behavior, divide work through peer coordination, and deliver a buildable implementation.
A Worker Prompt
- Workers must rebuild a complete codebase that reproduces the pre-compiled reference program’s behavior and passes hidden tests.They can run the binary to inspect behavior but cannot read its bytes; decompiling or dumping it is prohibited.Source: A Worker Prompt · p. 11
- Agents are told to define requirements, divide work, make decisions, and finish autonomously, without waiting for external instructions.They are instructed to keep pushing the rebuild forward.Source: A Worker Prompt · p. 11
- The team works offline using available local tools, while each worker uses Gitea for tracked work, Mattermost for communication, and a shared context board for findings.Direct messages are recommended to resolve imminent editing conflicts; the prompt also permits using or building other tools as needed.Source: A Worker Prompt · pp. 11–12
- Delivery requires a root-level executable compile.sh that creates ./executable, which must pass the hidden tests after the evaluator removes any committed executable.The prompt warns that omitting the root compile.sh yields zero points.Source: A Worker Prompt · pp. 11–12
- The shared workflow has workers claim an area, implement and compare against the reference, merge changes, publish a summary, and repeat.Workers are encouraged to resolve overlapping claims by direct message; they do not need to sync code during implementation.Source: A Worker Prompt · p. 12
B.1 Event-Driven Runtime
The runtime routes repository and chat activity to persistent workers, which process queued events into prompts and have recovery and idle-work mechanisms.
B.1 Event-Driven Runtime
- Gitea activity and Mattermost messages trigger worker events, which are written to durable queues and wake the dispatcher.Gitea notifications are fetched after server-sent events; Mattermost messages arrive over a live connection with catch-up after reconnect, and unique event IDs support deduplication.Source: §B.1 · p. 12
- On each wake, the dispatcher orders and combines queued events into the next prompt, then marks them sent when the worker turn completes.Events arriving during a turn remain queued for a later turn.Source: §B.1 · p. 12
- Failed deliveries are retried, unfinished events are requeued after router restart, and an idle worker receives a continue-working prompt after 10 minutes.A follow-up prompt also reminds workers to publish useful findings if a completed turn produced no shared context entry.Source: §B.1 · p. 12
B.2 Shared Context Delivery
Shared context reaches workers both at turn start and during active work through infrastructure-tool results.
B.2 Shared Context Delivery
- At a new turn, the router appends queued workspace notifications, arriving messages, and recently shared context to the worker prompt.This bundles recent coordination information for the worker at the beginning of its turn.Source: §B.2 · p. 12
- During an active turn, infrastructure-tool results are supplemented with pending direct messages and new shared-context entries.The wrapper operates on MCP tool returns and does not interrupt an arbitrary running process.Source: §B.2 · p. 13
- The active-turn delivery path appends updates to tool results rather than interrupting a running process.It applies to infrastructure tool calls.Source: §B.2 · p. 13
C Experiment Details
The experiments use the five hardest ProgramBench tasks, a fixed model and harness configuration, and a six-hour evaluation budget with staggered worker launches.
C Experiment Details
- The five selected ProgramBench tasks are the hardest by average pass rate of state-of-the-art models in the official extended results.All reported configurations use GPT-5.6-sol with high reasoning effort and the same harness, worker protocol, and evaluation configuration.Source: C Experiment Details · p. 13
- Each run uses a six-hour total budget, follows ProgramBench container preparation, and reports the canonical-kept pass rate under official evaluation.The underlying single-agent harness is Copilot, with a 272,000-token input limit and 128,000-token output limit.Source: C Experiment Details · p. 13
- Workers launch one every 30 seconds during the first hour and one every 3 seconds thereafter, with all agents terminated six hours after the first activation.The experiments stagger activation to reduce scope contention.Source: C Experiment Details · p. 13
- At 45 minutes before termination, workers are told to stop new features and ensure the build works and existing work is merged; five minutes before, they are told to merge ready work and stop.The final reminder asks workers to confirm that bash compile.sh && ./executable works on the default branch.Source: C Experiment Details · p. 13
- The single-agent baseline receives a prompt to continue exploring untried inputs through the six-hour run, while the multi-agent idle detector prompts agents after 10 minutes.The multi-agent prompt asks idle workers to explore untried inputs and close gaps between the reference and team implementation.Source: C Experiment Details · p. 13