Source-linked AI summary
QueenBee Planner: Skill-Evolving Communication Topologies for Token-Efficient LLM Multi-Agent Systems
Congjia Tian, Yuhang Yao, Jiaming Cui
TL;DR
LLM multi-agent systems face a design question of how agents should be organized and how that organization can improve with experience. QueenBee Planner freezes the workers and trains an outer planner to generate temporal communication DAGs using evidence-backed design rules. Its self-evolved structures improve over fixed topologies and cold generation, including a 37.2% RMSE reduction against the strongest fixed topology in CF fulltest.
Problem
As LLM systems move to interacting agents, the design question includes how to organize agents and improve that organization with experience; existing systems often design a topology for a task and stop.
Source: §1 · pp. 1–2Method
QueenBee Planner freezes the worker pool, task adapter, and scoring function while an outer planner generates temporal communication DAGs using evidence-backed rules distilled from execution traces.
Source: §1 · p. 2Results
Self-evolved generation improved over fixed topologies and cold generation; in CF fulltest, it reduced RMSE by about 37.2% relative to the strongest fixed topology.
Source: §1 · p. 2, §4.3 · p. 11Takeaways & Limitations
The findings support treating multi-agent architecture as an explicit learning object, with transferable priors for when to preserve, modify, or avoid communication patterns.
Source: §5 · p. 14The evaluation isolates planner improvement with frozen workers and a fixed grader, so it does not test worker adaptation.
Source: §3.1 · p. 4
Abstract
from arXiv · showhide
Large language model (LLM) multi-agent systems increasingly depend not only on how individual agents reason, but also on how agents are connected. This paper introduces QueenBee Planner, a framework that treats inter-agent communication topology as a retrievable and self-improving design skill. A pool of worker agents, the task adapter, and the scoring function are frozen; only an outer LLM planner learns to generate temporal communication DAGs specifying who sends information to whom, in which round, who merges messages, and who emits the final answer. Execution traces are distilled into evidence-backed design rules with three actions: \emph{Preserve}, \emph{Modify}, and \emph{Avoid}. To prevent self-evolution from turning lucky runs or plausible but false explanations into policy, QueenBee uses held-out acceptance gates, variance-aware credit, motif-level attribution, transfer trust, insight falsification, and structural deduplication. We evaluate the method on Count-Frequency aggregation and Silo-Bench-style distributed coordination tasks. With fixed workers, self-evolved graph generation produces communication structures that improve over fixed topologies and cold generation. In the CF fulltest setting, the best generated graph reduces RMSE from 12.53 for the strongest fixed topology to 7.87 while also reducing messages, model calls, and token cost; Silo-style results show the same direction of improvement over cold and fixed-topology baselines. These results suggest that multi-agent systems can learn reusable architectural design knowledge rather than merely memorizing task answers.
1 Introduction
As multi-agent systems make organization a design question, prior topology optimizers generally do not retain reusable communication-design knowledge across tasks. QueenBee Planner freezes the workers and learns evidence-backed rules for generating temporal communication DAGs, while using safeguards against turning noisy or false lessons into policy.
1 Introduction
- Topology optimizers usually design for a task and stop, while retained experience in skill libraries typically concerns task-solving behavior rather than how agents should be connected.The paper frames the gap as a need for an architect that improves at designing multi-agent systems over its lifetime.Source: §1 · p. 2
- QueenBee Planner freezes workers and learns evidence-backed communication-design rules in an outer planner that generates temporal DAGs specifying message routing, rounds, merging, and final-answer ownership.The task adapter and scoring function also remain fixed.Source: §1 · p. 2
- Self-improvement risks turning lucky runs, plausible but false explanations, or training-only gains into policy, motivating held-out acceptance, variance-aware credit, motif attribution, and falsification.The paper also calls for reusable knowledge to be attributed at the motif level and unverified natural-language insights to be tested before steering generation.Source: §1 · p. 2
- The paper evaluates this architectural-learning goal on communication-sensitive Count Frequency and Silo-Bench tasks where workers see shards and correctness depends on routing and merging information.These exact-answer tasks are presented as a testbed for separating worker competence from organization competence.Source: §1 · p. 2
- The paper reports that disciplined self-evolution improves generated communication structures on held-out tasks while workers remain frozen, targeting learned organization rather than memorized task answers.The contribution also recasts topology as a retrievable, self-improving design skill.Source: §1 · p. 2
2 Related Work
Related work studies optimization of agent workflows and communication graphs, dynamic multi-agent organization, and reusable agent skills. QueenBee differs by storing evidence-backed lessons specifically about communication topology.
Agentic workflow and graph optimization
- Graph-optimization work searches workflow code, agentic systems, or communication graphs, whereas QueenBee optimizes temporal communication DAGs and stores reusable planner skills.The related-work discussion describes GPTSwarm, AFlow, and ADAS as examples of graph or workflow optimization.Source: §2 · p. 3
Multi-agent organization and communication
- Multi-agent systems use roles and dynamic collaboration, but many retain fixed schemes or optimize routing without storing explicit topology lessons.QueenBee requires concrete temporal edges and converts execution evidence into reusable topology-design skills.Source: §2 · p. 3
Agent skills and skill evolution
- Reflection, embodied skill libraries, and workflow memory reuse agent experience, while QueenBee skills guide the planner on when a graph may preserve evidence, distort it, or make merging costly.Its skills concern communication graphs rather than a worker's local task performance.Source: §2 · p. 3
Skill routing, ecosystems, and failure modes
- As reusable memories grow, selection becomes a bottleneck; prior work motivates selective retrieval, while QueenBee uses negative memory, held-out evaluation, and cost-aware comparison with sparse baselines.The CF task also resembles distributed aggregation, but LLM reducers may omit, duplicate, or alter values, making information coverage and merge size relevant to topology.Source: §2 · pp. 3–4
3 Method
QueenBee freezes the worker pool and makes an outer planner responsible for generating temporal communication DAGs. It distills execution evidence into reusable design rules, admitting updates only through validation mechanisms.
3 Method
- The planner, not the workers, is the learning object: it generates executable communication graphs while the workers and task-solving setup remain fixed.The planner generates new graphs rather than merely selecting among named topologies.Source: §3 · p. 4, §3.1 · p. 4
3.1 Planner-Worker Separation
- The planner separates graph design from execution: it chooses information flows and the final answer holder before workers execute the graph.Workers observe private shards, and learning does not change their model, role, prompt, parsing rule, or scoring function.Source: §3.1 · p. 4
3.2 Generated Temporal Communication DAG
- A temporal communication DAG encodes directed messages by round, while feasibility constraints cap rounds, total message edges, and each worker’s per-round fan-in.Time-unrolling keeps cross-round communication acyclic; the constraints bound protocol length, communication cost, and merge load.Source: §3.2 · pp. 5–6
3.3 Skills as Graph-Generation Memory
- The skill bank stores evidence-backed rules that specify when structures should be Preserved, Modified, or Avoided under applicable task conditions.Skills can seed candidate graphs or guide decisions about sinks, fan-in, audit edges, and other structures.Source: §3.3 · pp. 6–7
3.4 Structural Motif-Level Credit Assignment
- Label-invariant motifs assign structural credit across different DAGs, but unfamiliar motifs remain uncertain until supported by probes or execution evidence.The current implementation uses eight hand-crafted features, including message count, fan-in, sinks, audit coverage, and reduction depth.Source: §3.4 · p. 8
3.5 Skill Evolution: From Generated Evidence to Design Rules
- Execution evidence proposes positive rules, counterexamples, and executable scaffolds; candidate insights are hypotheses until they pass falsification and held-out acceptance checks.Updates that explain training runs but fail to improve held-out generation are not admitted to the skill bank.Source: §3.5 · pp. 8–9
- Transfer trust limits deployment to task conditions with direct or related evidence, while structural deduplication merges equivalent graph specifications into shared skill families.Without trusted evidence for a task slot, the planner returns to cold-start generation.Source: §3.5 · p. 9
3.6 Evolution Loop
- Each evolution round executes candidate DAGs, distills evidence into rules, and exports only updates that pass the held-out generation gate.At deployment, the final skill bank guides the planner while workers continue only to execute the generated graph.Source: §3.6 · p. 10
4 Experiments
The experiments test whether communication topology can be learned as a design skill, using exact-evaluation Count-Frequency aggregation and more varied Silo-style coordination tasks with workers and scoring fixed. Self-evolved graph generation outperforms fixed-topology and cold-generation baselines in the reported settings, while round-by-round results track gains as communication-design memory accumulates.
4.1 Evaluation Tasks
- The CF task isolates topology effects by giving workers private shards and measuring exact global-frequency aggregation, while Silo-style tasks test transfer to more varied distributed-coordination cases.Across both settings, comparisons include hand-designed fixed topologies, cold generation, and generation conditioned on the evolved skill bank.Source: §4.1 · p. 10, §4.2 · p. 11
4.3 Count-Frequency Results
- 7.87 RMSE, 8 messages, 15 model calls, and about 19.4k tokens for the self-evolved best graph, versus 12.53 RMSE for the strongest fixed topology.The generated hybrid reduce-audit graph also reduces messages by about 74.2%, model calls by about 54.5%, and token cost by about 59.6% relative to that fixed topology.Source: Table 1 · p. 11, §4.3 · p. 11
- Across representative CF runs, self-evolved graphs have lower RMSE than the best fixed topology under the same setup across DeepSeek, Qwen, GPT-4o, and MiMo models.Examples include GPT-4o iter5 improving from 14.76 to 6.48 RMSE and Qwen3.5-Flash from 16.44 to 11.53 RMSE; successful structures tend to be sparse staged-reduction or audit variants.Source: §4.3 · pp. 11–12
4.4 Silo-Bench-Style Results
- 0.500 EM after three evolution rounds exceeds the best single fixed topology at 0.333 and the per-case fixed-topology oracle at 0.417.The current three-round result improves over cold generation and fixed-topology baselines, but remaining independent runs are needed for mean and variance.Source: Table 3 · p. 12, §4.4 · p. 12
- The Silo-style evaluation uses six held-out cases across three difficulty buckets, each with four seeds, and compares evolved generation with cold, fixed, and per-case oracle topologies.This setting tests whether graph-generation memory transfers to tasks with more varied formats, outputs, and aggregation rules.Source: §4.1 · p. 10, §4.2 · p. 11, §4.4 · p. 12
4.5 Round-by-round Revolution
- The experiments keep the worker pool, task adapter, prompts, and scorer fixed while updating the planner’s skill bank through evaluation feedback.This design isolates the role of accumulated communication-design memory from changes to the underlying agents.Source: §4.5 · pp. 12–13
- In a representative CF run, the best-so-far archive falls from 17.34 to 13.06 RMSE by round 4, ending 15.8% below the fixed-topology reference of 15.50.The per-round proposal mean fluctuates, including a rebound to 15.79, while the Silo trajectory rises from 41.7% held-out exact match to 50.0% and 54.2% as targeted skill cards are added.Source: §4.5 · p. 13, Figure 3 · p. 13
5 Conclusion
QueenBee Planner freezes the worker pool and trains an outer planner to generate temporal communication DAGs using evidence-backed design skills. The conclusion presents communication architecture as a learnable object, while calling for broader evaluation.
5 Conclusion
- The planner learns temporal communication DAGs while the worker pool stays fixed, using execution experience distilled into Preserve, Modify, or Avoid skills.Updates pass a held-out gate and use motif-level credit, variance-aware comparison, transfer trust, insight falsification, and structural deduplication.Source: §5 · p. 14
- On Count-Frequency, self-evolved free-DAG generation outperforms fixed topologies across multiple model settings; Silo-style results improve over cold generation and fixed-topology baselines.The Silo-style result is from three rounds.Source: §5 · p. 14
- QueenBee accumulates transferable priors about when to preserve, modify, or avoid communication patterns, rather than seeking one universally optimal topology.The authors call for evaluation on larger task suites, broader worker configurations, and longer evolution horizons, and for analysis of motifs affecting accuracy, cost, and robustness.Source: §5 · p. 14
A.1 Round 1: Initial Preserve and Avoid Memory
The first accepted batch stores both positive and negative topology lessons. It preserves peer propagation followed by a star sink for accuracy-first Count-Frequency, and records a simple chain as an avoid rule.
A.1 Round 1: Initial Preserve and Avoid Memory
- Peer propagation followed by a star sink is preserved as the accuracy-first Count-Frequency policy.The skill specifies one-peer propagation at distances 1, 2, and 4, followed by a star gather to agent 4.Source: §A.1 · p. 17
- For generated DAG runs, the selected primary is the final sink, and only that answer holder is scored.Source: §A.1 · p. 17
- A simple chain is stored as a negative design under the observed condition, alongside the reusable positive scaffold.The chain routes agents 0→1→2→3→4; the accepted batch records both reusable positive structure and counterexamples.Source: §A.1 · p. 17
A.2 Round 2: Staged Reduction Skills
Round 2 adds generated temporal DAG skills centered on staged reduction and reachability to a selected primary. The recorded patterns include tree and pair reductions.
A.2 Round 2: Staged Reduction Skills
- The staged-tree reduction skill preserves a generated topology whose receivers aggregate partials and forward merged state in a later step.Variants can change fan-in, sink placement, or audit edges while retaining temporal reachability to the selected primary.Source: §A.2 · p. 17
- The staged-tree protocol distributes initial vote counting before a final aggregation step determines the winner.Source: §A.2 · p. 17
- A staged-pair reduction pattern aggregates adjacent agents’ results before final aggregation at the sink.The raw skills emphasize staged reduction, bounded fan-in, and reachability to the selected primary.Source: §A.2 · p. 17
A.3 Round 3: Star-Sink Transfer and Mesh Avoidance
Round 3 preserves a generated star-sink variant with a sink-coverage repair step and marks a dense mesh-star topology for avoidance under the same condition.
A.3 Round 3: Star-Sink Transfer and Mesh Avoidance
- The generated staged-star-sink skill preserves a topology with sink-coverage repair and scores only the final sink as the selected primary.Its protocol includes local distinct counting, first-stage aggregation, final aggregation, and submission of the global distinct count.Source: §A.3 · p. 18
- The dense mesh-star topology becomes an avoid rule under the same condition as the preserved star-sink variant.The mesh broadcasts from all agents to all others, then gathers to agent 4.Source: §A.3 · p. 18
A.4 Round 4: Star Aggregation Variants
Round 4 adds star-style aggregation variants to the bank, making their structural motifs available when a task calls for a single sink or low-depth aggregation.
A.4 Round 4: Star Aggregation Variants
- One preserved variant stages local vote-count distribution, intermediate aggregation, and final decision making.Source: §A.4 · p. 18
- Another preserved variant begins with XOR computation and distribution.Source: §A.4 · p. 18
- The planner can retrieve star-aggregation motifs when a task calls for a single sink or low-depth aggregation.The topology name itself is not treated as universal.Source: §A.4 · p. 18
A.5 Round 5: Negative Memory for a Formerly Positive Family
Round 5 shows that negative evidence can revise a topology family: a broad star-sink motif may be preserved in one evidence context and avoided in another.
A.5 Round 5: Negative Memory for a Formerly Positive Family
- The staged-star-sink topology is marked avoid because it is dominated by stronger CF choices; recommendations include scoring only the final sink and varying fan-in, sink placement, or audit edges.Variants should retain full temporal reachability to the selected primary.Source: §A.5 · p. 18
- The broad star-sink motif is preserved in one evidence context and avoided in another, rather than encoded as one hard-coded choice.Source: §A.5 · p. 19