PaperlayerView exact revision on arXiv

Source-linked AI summary

When Evidence Shapes Collaboration: Knowledge-Conditioned Topology Generation for Multi-Agent Systems

Yangxiao Jiang, Jiarun Fan, Mingcong Xu, Yanxi Guo, Jiwen Feng, Shanqing Xu, Mengchen Qian, Wei Chen, Xiaojin Zhang

arXiv:2608.27984v1cs.AI

TL;DR

Existing dynamic multi-agent systems generate collaboration topologies from query semantics before observing evidence, creating structural mismatch with task evidence requirements. K-GAT addresses this by generating adaptive topologies conditioned on retrieved evidence and improves reasoning accuracy and workflow adaptability across diverse evidence conditions.

  • Problem

    Existing dynamic MAS frameworks generate workflows solely from query semantics before retrieval, which can misalign collaboration structure with actual evidence requirements.

  • Method

    K-GAT generates executable collaboration topologies conditioned on retrieved evidence, learning workflow distributions from evaluated candidate workflows under structural constraints.

  • Results

    K-GAT consistently improves reasoning accuracy and workflow adaptability across diverse evidence conditions.

  • Takeaways & Limitations

    Evidence-conditioned topology generation provides an adaptive workflow-generation paradigm for multi-agent reasoning.

  • Takeaways & Limitations

    K-GAT depends on the quality and coverage of its external knowledge graph, and incomplete or noisy evidence may still produce suboptimal topologies.

Abstract

from arXiv · show

Multi-Agent Systems (MAS) have recently moved from static workflows toward dynamically generated collaboration topologies. However, existing topology generation methods rely primarily on the parametric knowledge of large language models, with external search or retrieval used only as a reactive tool rather than an explicit determinant of collaboration structure. This leads to structure-knowledge misalignment, where systems exhibit redundant interactions or insufficient verification in knowledge-intensive tasks. We propose K-GAT (Knowledge-Guided Agent Topology Generator), a neuro-symbolic framework that formulates collaboration topology design as a knowledge-conditioned structure learning problem, integrating external evidence directly into autoregressive graph generation. Extensive experiments on knowledge-intensive benchmarks demonstrate K-GAT's efficiency and effectiveness: notably on the expert-level GPQA dataset, K-GAT outperforms the LLM-Debate baseline by a substantial margin of +15.7% in accuracy, while consuming less than half the computational tokens.

1 Introduction

Existing dynamic multi-agent workflows can misalign collaboration structure with retrieved evidence because they plan from query semantics before retrieval. K-GAT addresses this Structural Mismatch by conditioning topology generation on external evidence and reports improved accuracy and adaptability across evidence conditions.

  • From Static to Dynamic Collaboration: Existing MAS methods evolved from single-agent reasoning and static collaboration topologies toward dynamically generated workflows for complex reasoning.Single-agent approaches struggle with decomposition, verification, and parallel exploration, while static topologies lack adaptability.
  • Structural Mismatch: Semantic planning can overestimate or underestimate collaboration needs because query complexity does not necessarily reflect evidence requirements.Complex queries may have simple, consistent evidence, whereas sparse or conflicting evidence may require more coordination.
  • Structural Mismatch: Over-planning creates redundant interactions and wasted computation when retrieved evidence is sufficient and consistent.The planner may generate unnecessarily sophisticated debate workflows despite adequate evidence.
  • Structural Mismatch: Under-planning limits verification and exploration when retrieved evidence is sparse, ambiguous, or conflicting.Semantic planning may impose oversimplified sequential structures in these conditions.
  • Evidence-First Collaboration: K-GAT generates collaboration topologies conditioned on retrieved evidence and provenance information rather than planning before retrieval.Its autoregressive graph construction adaptively determines agent instantiation and communication routing.
  • Reported Results: K-GAT consistently improves reasoning accuracy and workflow adaptability across knowledge-intensive benchmarks and diverse evidence conditions.The introduction reports this as the outcome of extensive experiments.

2 Related Works

Research has progressed from single-agent reasoning and predefined multi-agent protocols toward automatically generated workflows, while the related-work passage provides only a partial view of that transition.

  • Agentic Paradigms: LLM-based agents shifted intelligent systems from single-agent reasoning toward collaborative multi-agent workflows.ReAct interleaved reasoning and tool use within one trajectory, while later MAS introduced structured collaboration.
  • Multi-Agent Systems: Early MAS frameworks used predefined communication protocols and role assignments to structure collaboration.These approaches preceded efforts to automate workflow and topology generation.
  • Dynamic Workflow Generation: Recent studies explored automatic workflow and topology generation to overcome the rigidity of manually designed workflows.The supplied related-work passage ends while introducing this line of research.

3 Methodology

K-GAT generates collaboration topologies conditioned on retrieved evidence and provenance, then trains them from execution-derived supervision. Its pipeline combines evidence retrieval, constrained autoregressive graph generation, execution-based evaluation, pruning, and distribution-matching optimization.

  • Evidence-Driven Topology Generation: K-GAT retrieves evidence units with provenance and uses this evidence context to condition which agent roles are instantiated and how messages flow.
  • Evidence-Driven Topology Generation: The workflow is represented as a directed acyclic graph whose nodes are agent roles and whose edges indicate message passing.Smaller graphs are assigned lower communication and computation overhead.
  • Dynamic Inference and Execution: K-GAT autoregressively generates agent nodes and communication edges under acyclicity, bounded-indegree, and execution-validity constraints.Generation terminates at an END token or the maximum node budget, producing the topology Ĝ.
  • Curriculum Learning Framework: The generator learns a soft distribution over retained high-quality topologies through alternating candidate mining and distribution-matching optimization.This provides structural supervision despite datasets containing query–answer pairs rather than gold collaboration workflows.
  • Curriculum Learning Framework: Training mines candidates from templates, random feasible DAGs, and on-policy generator samples, with π controlling exploration versus exploitation.The curriculum framework then evaluates candidates through execution, as illustrated by its candidate-mining stage.
  • Curriculum Learning Framework: Candidates are repeatedly executed to estimate success, scored for structural cost, and pruned to smaller topologies when task success is preserved.Accepted pruning reduces structural cost while maintaining at least the minimum acceptable success rate, yielding compact structures G∗.

4 Experiment

Experiments across seven reasoning benchmarks evaluate K-GAT against single models, static topologies, and multi-agent baselines. Results show strong accuracy, evidence-conditioned adaptation, component benefits, and improved computational efficiency.

  • Overall Performance: K-GAT achieves 78.68% average accuracy, the strongest overall performance among 8B-scale baselines across seven reasoning benchmarks.It remains competitive with substantially larger Qwen-3-32B and Llama-3.1-70B models.
  • Overall Performance: K-GAT's gains are particularly significant on knowledge-intensive tasks, while improvements on symbolic or algorithmic tasks such as GSM8K are comparatively smaller.The paper attributes this difference to the greater reliance of knowledge-intensive tasks on external retrieval.
  • Ablation and Component Analysis: 50.75% is reached when evidence-aware topology generation is combined with the KG-Verifier, compared with 40.19% versus 38.48% for curriculum-trained topology generation without external evidence.The KG-Verifier filters hallucinated paths, while the curriculum-trained generator improves performance even without external evidence.
  • Ablation and Component Analysis: K-GAT gains +15.35% on MMLU-Pro and +10.56% on GPQA when external knowledge is incorporated, exceeding gains from single-agent baselines and query-conditioned G-Designer.These comparisons assess closed-book versus open-book performance across single-agent and dynamic multi-agent settings.
  • Ablation and Component Analysis: 50.00% accuracy with evidence-conditioned topology exceeds 45.34% for query-conditioned topology, while KG-Verifier increases performance further to 50.75% under controlled GPQA conditions.Retrieval alone improves performance from 33.09% to 39.22%; all open-book variants use the same Qwen-3-8B backbone and top-3 evidence.
  • Ablation and Component Analysis: Increasing the topology limit beyond Nmax = 6 does not improve accuracy, while the average generated topology remains around 2.34 nodes.The results suggest that a small node budget is sufficient in this setting.
  • Evidence-Conditioned Adaptation: Under challenging evidence conditions, average topology size increases from 2.02 to 2.66 nodes, indicating additional collaboration and verification.Highly relevant and consistent evidence typically produces compact two-node workflows, whereas conflicting or low-relevance evidence produces larger and more diverse topologies.
  • Computational Efficiency: On GPQA, K-GAT reduces token consumption by over 50% versus LLM-Debate while achieving higher accuracy, and on StrategyQA it avoids Tree's extensive search overhead.K-GAT occupies the optimal high-accuracy, low-token region in the reported trade-off analysis.

5 Conclusion

The paper identifies Structural Mismatch in dynamic multi-agent systems and proposes K-GAT, which generates collaboration topologies conditioned on retrieved evidence.

  • K-GAT addresses Structural Mismatch by generating collaboration topologies conditioned on retrieved evidence rather than query semantics alone.

Limitations

K-GAT depends on the quality and coverage of an external knowledge graph, and noisy or incomplete evidence can produce suboptimal topologies. Its bounded topology focus may also limit scalability for complex long-horizon reasoning.

  • K-GAT's effectiveness depends on the quality and coverage of its underlying external knowledge graph.The framework adds preprocessing and storage costs through a provenance-grounded Wikipedia knowledge graph.
  • Incomplete or noisy retrieved evidence may lead K-GAT to generate suboptimal collaboration topologies.
  • Bounded topology size may limit scalability for more complex long-horizon reasoning scenarios.The current framework mainly focuses on relatively compact collaboration structures.

Ethical Considerations

The work uses public datasets, open-source models, and Wikipedia-based external knowledge sources under their respective licenses, without private user data or human annotations.

  • The study uses publicly available datasets, open-source language models, and external knowledge sources constructed from Wikipedia corpora.
  • The utilized artifacts are restricted to academic research purposes under their respective open-source licenses.
  • The work does not involve private user data or human annotations.

B Complexity Analysis and Token Accounting

K-GAT's inference cost accounts for retrieval, topology decoding, verifier operations, and workflow token consumption. Overall cost scales approximately linearly with generated agents and retrieved evidence units, while bounded indegree limits communication overhead.

  • Inference-cost analysis covers evidence retrieval, topology decoding, verifier operations, and end-to-end token consumption.
  • B.2 Retrieval Complexity: Retrieving Kret evidence units costs Tann(Kret) plus O(Kret) for evidence collection and provenance assembly.The ANN retrieval term depends on the indexed query embedding; provenance spans are precomputed offline.
  • B.3 Topology Decoding Complexity: K-GAT autoregressively decodes agent nodes and communication edges under a bounded indegree constraint, with beam width B affecting decoding complexity.Structural masking introduces an effective edge sparsity factor αd.
  • B.4 Verifier Complexity: Verifier complexity is incurred when generated topologies contain KG-Verifier nodes that extract atomic claims and check them against retrieved evidence.Verification overhead scales with verifier nodes rather than all reasoning steps.
  • Overall inference cost scales approximately linearly with generated agents and retrieved evidence units, while bounded indegree prevents excessive communication overhead.

C Training Cost of Curriculum Optimization

Curriculum optimization incurs substantial execution-based supervision costs, with most wall-clock time spent repeatedly executing and validating candidate workflows rather than optimizing parameters.

  • Each dataset uses 40 stratified curriculum instances, five initial candidate topologies per instance, and two executions per candidate.This produces 400 initial MAS executions before additional pruning and validation executions.
  • The reported four-to-five-hour training time includes candidate construction, execution and scoring, pruning with re-execution, and generator optimization.
  • Pruning is slightly more expensive than initial candidate evaluation because simplified variants require structural checks and re-execution.Multiple feasible variants may be generated and tested for performance preservation.
  • More than 90% of wall-clock time is spent executing and validating initial and pruned multi-agent workflows.Candidate construction and topology-generator optimization contribute only a small fraction of total cost.
  • Reported token counts include prompts and completions from candidate execution, repeated evaluation, pruned-topology re-execution, and KG-Verifier calls.Dense retrieval, graph pruning, connectivity checking, and gradient-based optimization do not consume LLM tokens.

D Additional Experiments with Stronger Backbones

An additional evaluation tests K-GAT with the stronger Qwen3-235B-A22B backbone on HLE Bio/Chem Gold, where evidence-conditioned topology generation remains beneficial.

  • The supplementary evaluation examines whether evidence-conditioned topology generation remains useful when all compared methods use a substantially stronger model.
  • K-GAT achieves 0.1812 accuracy on HLE Bio/Chem Gold with Qwen3-235B-A22B, outperforming all fixed-topology baselines.The benchmark contains 149 samples.
  • K-GAT reaches 0.1812 accuracy versus 0.1342 for K-GAT (Inf w/o KG), showing an improvement with retrieved external evidence.
  • The stronger-backbone results suggest that K-GAT's gains are not merely compensation for weak backbone capacity.The authors position this as a focused supplementary evaluation on harder evidence-intensive reasoning.

F Knowledge Graph Construction

K-GAT constructs a provenance-grounded knowledge graph from Wikipedia and retrieves structured evidence to condition topology generation, while dataset-specific agent roles encode reasoning and verification patterns.

  • Knowledge graph construction: K-GAT adopts StructSense's ontology-guided extraction pipeline to convert unstructured Wikipedia documents into provenance-grounded evidence units.The paper uses the framework without modification because knowledge-graph construction is not its focus.
  • Knowledge graph construction: Extracted entities and relations are aligned with ontology concepts and organized into normalized knowledge units linked to source documents and supporting text spans.
  • Knowledge graph construction: Each retrieved evidence unit records head and tail entities, a relation type, and provenance linked to the original Wikipedia evidence span.
  • Evidence retrieval: During inference, K-GAT retrieves top-K evidence units by dense embedding similarity and uses their context to condition collaboration-topology generation.
  • Dataset-specific roles: Mathematical reasoning roles combine symbolic reasoning, numerical computation, and inspection, with crosschecking between analytical and programmatic processes.The Inspector communicates bidirectionally with all reasoning roles.
  • Dataset-specific roles: Graduate-level scientific, code-generation, multidisciplinary, and StrategyQA workflows pair specialized roles with reviewers, critics, testers, or other verification-oriented agents.The role profiles support cross-validation, debugging, consistency verification, or causal reasoning depending on the dataset.
Loading 2608.27984v1…