Learning to Be A Doctor: Searching for Effective Medical Agent Architectures

Motivation

• Multi-agent systems fit medical workflows because specialized agents can coordinate interdisciplinary tasks while preserving coherent decision-making.They can also help bridge departmental communication gaps and standardize procedures across institutions.Source: §1 · p. 2
• Static, hand-crafted workflows struggle with complex and changing diagnostic scenarios, requiring costly redesign for new techniques, modalities, and applications.This limits scalability and slows deployment across clinical settings.Source: §1 · p. 2

Approach

• The framework represents medical agents as dynamic graph-based workflows that evolve through LLM feedback and operations at node, structural, and framework levels.Operations include node changes, conditional or iterative structures, parallel execution, and established collaboration patterns such as Chain of Thought and Reflexion.Source: §1 · p. 2
• The system begins with a single-node baseline and iteratively adds nodes, structural logic, and framework-level reasoning strategies after diagnosing errors.The authors describe this process as progressively improving diagnostic accuracy, robustness, and efficiency.Source: §1 · p. 2

Evaluation

• Experiments on two medical diagnostic benchmarks report that the method automatically generates more efficient and accurate agent architectures while significantly outperforming baselines.The supplied passage does not provide numerical results.Source: §1 · p. 2

Contributions

• The framework is presented as the first fully automated design method for medical multi-agent systems using LLMs.The contribution includes a hierarchical search space for dynamic workflow evolution and a self-improving search algorithm guided by diagnostic feedback.Source: §1 · pp. 2–3

2.1 LLM-Based Agents in the Medical Domain

• Medical-agent systems such as Agent Hospital, SkinGPT-4, MMedAgent, and MDAgents address training, multimodal diagnosis, tool use, and collaboration, but dynamic workflow adaptation remains underexplored.The limitation is framed specifically around complexity and variability in real-world medical environments.Source: §2.1 · p. 3

2.2 Multi-Agent Collaboration and Evolution

• LLM-based multi-agent research includes peer-to-peer, centralized-controller, and hierarchical architectures, alongside systems designed for collaboration and self-improvement.Examples include MetaGPT, LLM-Blender, and Multi-Agent Debate.Source: §2.2 · p. 3
• Recent self-evolving approaches include EvoMAC’s collaboration network, AutoAgents’ automatic agent generation, and AFLOW’s graph search for task-execution paths.These works motivate automated adaptation beyond fixed multi-agent designs.Source: §2.2 · p. 3

3 Methodology

• The framework represents workflows as graphs of specialized agents, with edges encoding execution order and nodes divided into basic and tool types.The methodology combines graph-based representation, configurable node functions, a constrained search space, and feedback-driven evolution.Source: §3 · p. 3, §3.1 · p. 3

3.1 Workflow Node Definition

• Basic nodes interact directly with the LLM using system and human prompts, while tool nodes invoke external tools that provide supplementary information for reasoning.Basic-node configuration includes node name, system prompt, human prompt, and description; tool nodes include tool-specific attributes.Source: §3.1 · pp. 3–4

3.2 Search Space

• The search space supports node-level addition, removal, and prompt modification, enabling workflows to add functionality, remove redundancy, and align node behavior with task requirements.These operations modify individual nodes rather than the overall execution structure.Source: §3.2 · p. 4
• Structural operations add conditional branches, iterative loops, and parallel paths, transforming linear workflows into more adaptive structures for complex diagnosis.Conditions select paths based on context, loops support error correction and refinement, and parallel paths combine independent clinical perspectives.Source: §3.2 · pp. 4–5
• Framework-level design dynamically composes customized collaboration patterns from basic nodes and structural operations rather than directly adopting existing frameworks.The approach can construct structures functionally similar to Round Table while adapting roles and execution to a diagnostic task.Source: §3.2 · p. 5

3.3 Workflow Evolution

• Workflow evolution classifies failures as image-understanding or diagnostic errors, traces their causes, and converts feedback into structural or prompt modifications.The updated workflow is validated before deployment, and refinement continues across diagnostic iterations until validation accuracy converges.Source: §3.3 · pp. 5–7
• Mermaid code and visual workflow diagrams make execution logic clearer to the LLM when recorded node and edge relationships are difficult to interpret.The visualization is intended to support more accurate analysis and improvement suggestions.Source: §3.3 · pp. 5–6
• Structural modifications undergo validation that rejects duplicate edges, limits iterations, and checks cycle directionality, whereas prompt-only changes can be applied directly.These checks preserve graph integrity, executable paths, and defined loop exits.Source: §3.3 · p. 6

4.1 Experimental Setup

• The framework is evaluated on two dermatological image datasets with balanced training, validation, and testing splits, using Top-1, Top-3, and Top-5 accuracy.The evaluation also measures consensus stability across repeated inputs with cons@64.Source: §4.1 · pp. 7–8
• Experiments use direct LLM diagnosis, chain-of-thought prompting, and Round Table collaboration as baselines, with final workflows evaluated after iterative validation.The framework extends LangGraph with modular nodes and configurable tools, including image similarity search.Source: §4.1 · pp. 7–8

4.2 Results

• The evolved workflow’s Top-k accuracy improves over iterations, initially rising through structural refinement and prompt optimization before converging to a stable configuration.Figure 4 compares the evolving workflow with direct image-to-LLM diagnosis and tracks Top-k accuracy across iterations.Source: Figure 4 · p. 7, §4.2 · p. 8
• Across both datasets, LLMs, and Top-k metrics, the proposed method consistently outperforms IO, CoT, and Round Table baselines.Table 1 reports Top-k diagnostic accuracy for GPT-4o, GPT-4o-mini, and Claude 3.5 Sonnet on Skin Concepts and Skin Conditions.Source: §4.2 · pp. 8–9, Table 1 · p. 9
• The method achieves higher per-disease cons@64 scores across a wide range of conditions, indicating more stable and consistent predictions.Figure 5 evaluates majority-vote consensus over 64 samples per class using GPT-4o on Skin Concepts.Source: §4.2 · p. 9, Figure 5 · p. 9

4.3 Ablation Analysis

• Adding tool nodes and modifying prompts notably improve accuracy, whereas removing nodes has minimal accuracy impact but improves execution efficiency by reducing redundancy.Table 3 reports Top-k accuracy changes when individual operations are disabled.Source: §4.3 · pp. 9–10, Table 3 · p. 10
• Structural-level modifications improve performance, and the full node-plus-structural-plus-framework configuration achieves the best results.The integration ablation compares N, N+S, and N+S+F levels and supports hierarchical workflow evolution.Source: §4.3 · p. 10, Table 4 · p. 10
• Together, the ablations provide interpretable evidence about how individual operations and hierarchical integration levels contribute to workflow performance.The study examines both operation-specific effects and progressively broader integration levels.Source: §4.3 · pp. 9–10

• The framework enables autonomous exploration and iterative refinement of agent-based diagnostic workflows in response to diagnostic errors.Source: §5 · p. 10
• Dynamic adaptability operates at both the structural and prompt levels, allowing workflows to evolve with changing clinical demands and diagnostic requirements.Source: §5 · p. 10
• The approach bridges static expert-defined architectures and dynamic self-improving methods while laying groundwork for future adaptive diagnostic systems.Source: §5 · p. 10