PaperlayerView exact revision on arXiv

Source-linked AI summary

MDTeamGPT: A Self-Evolving LLM-based Multi-Agent Framework for Multi-Disciplinary Team Medical Consultation

Kai Chen, Xinfeng Li, Tianpei Yang, Hewei Wang, Wei Dong, Yang Gao

arXiv:2503.13856v1cs.AI

TL;DR

MDTeamGPT targets long, cognitively burdensome MDT dialogues and limited reflection on diagnostic experience. It combines consensus aggregation, residual discussion, and separate knowledge bases for correct and incorrect cases. The framework achieves 90.1% accuracy on MedQA and 83.9% on PubMedQA, with cross-dataset tests supporting knowledge-base generalization.

  • Problem

    LLM-based MDT consultations face excessively long dialogue histories and methods that store treatment histories without extracting effective experience or reflecting on errors.

    Source: §1 · pp. 1–2

  • Method

    MDTeamGPT uses consensus aggregation and residual multi-round discussion with CorrectKB and ChainKB to accumulate consultation experience.

    Source: §1 · p. 2

  • Results

    90.1% accuracy was achieved on MedQA and 83.9% on PubMedQA, with cross-dataset tests supporting generalizable reasoning skills.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The constructed knowledge bases support continual diagnosis improvement and generalization across test sets from both datasets.

    Source: §1 · p. 2

    Experiments used only MedQA and PubMedQA, motivating evaluation on a broader range of datasets to assess robustness across different data types.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) have made significant progress in various fields. However, challenges remain in Multi-Disciplinary Team (MDT) medical consultations. Current research enhances reasoning through role assignment, task decomposition, and accumulation of medical experience. Multi-role collaboration in MDT consultations often results in excessively long dialogue histories. This increases the model's cognitive burden and degrades both efficiency and accuracy. Some methods only store treatment histories. They do not extract effective experience or reflect on errors. This limits knowledge generalization and system evolution. We propose a multi-agent MDT medical consultation framework based on LLMs to address these issues. Our framework uses consensus aggregation and a residual discussion structure for multi-round consultations. It also employs a Correct Answer Knowledge Base (CorrectKB) and a Chain-of-Thought Knowledge Base (ChainKB) to accumulate consultation experience. These mechanisms enable the framework to evolve and continually improve diagnosis rationality and accuracy. Experimental results on the MedQA and PubMedQA datasets demonstrate that our framework achieves accuracies of 90.1% and 83.9%, respectively, and that the constructed knowledge bases generalize effectively across test sets from both datasets.

1 Introduction

MDTeamGPT addresses inefficiencies and limited experience utilization in LLM-based MDT consultations with structured multi-agent collaboration and self-evolving knowledge bases. It reports 90.1% accuracy on MedQA and 83.9% on PubMedQA, with cross-dataset tests supporting knowledge-base generalization.

  • MDT consultation is motivated by the insufficiency of single-specialty perspectives for comprehensive care, despite the temporal, resource, procedural, and communication costs of interdisciplinary organization.The cited motivation links diversified clinical expertise with more holistic therapeutic strategies while identifying operational vulnerabilities.Source: §1 · p. 1
  • Existing LLM-based MDT approaches face long multi-role dialogues, cognitive burden, hallucinations, scarce consultation data, and limited reflection on errors.These challenges can reduce response efficiency and accuracy and constrain generalization of accumulated medical experience.Source: §1 · pp. 1–2
  • MDTeamGPT combines a lead physician, residual discussion structure, and multi-round specialist consultation to integrate discussion outcomes.The lead physician organizes outcomes into consistency, conflict, independence, and integration categories, while later rounds use residual information.Source: Figure 1 · p. 1, §1 · p. 2
  • The framework stores consultation outcomes in distinct knowledge bases according to correctness and uses correct and incorrect experiences to strengthen transferable reasoning.This adaptive learning mechanism is intended to support continual system evolution.Source: §1 · p. 2
  • 90.1% accuracy on MedQA and 83.9% on PubMedQA were achieved using 600 consultation rounds, with cross-dataset tests supporting generalizable reasoning skills.The results concern the framework’s accumulated consultation experience rather than isolated consultation data.Source: §1 · p. 2

2 Related Work

Related work spans structured LLM-agent collaboration, experience utilization, and discussion aggregation. MDTeamGPT builds on these directions while addressing limited medical applicability, insufficient reflection on errors, and inefficient information exchange.

2.1 LLM-based Multi-Agent Collaboration

  • Prior multi-agent systems use task decomposition, directed acyclic graphs, standard operating procedures, or consensus voting, but often target software development or rely on static structures.Medagents applies doctor roles and consensus voting, yet the cited discussion identifies risks of collective hallucination and limited strategy.Source: §2.1 · p. 2

2.2 Experience Utilization

  • Experience-based methods accumulate successes, heuristics, or iterative refinements, while medical systems use records and experience databases to support agent evolution.The surveyed methods generally lack abstraction, summarization, and reflection on erroneous cases.Source: §2.2 · p. 3

2.3 Discussion and Aggregation

  • Structured discussion methods move agents from individual ideation through sharing and focused discussion toward collective consensus.Nominal Group Technique and LLM Discussion motivate iterative, divergent, and convergent exchanges among multiple agents.Source: §2.3 · p. 3

3 Method

The method selects specialists for each patient, conducts residual multi-round consultation with lead-physician aggregation, and filters the outcome through safety review. It then stores validated or incorrect experiences in separate knowledge bases for later retrieval and prompt enhancement.

3 Method

  • MDTeamGPT organizes specialist and auxiliary roles across specialist assignment, multi-round consultation, and summary/output stages.Specialists include medical doctor roles, while auxiliary roles include the Primary Care Doctor, Lead Physician, Chain-of-Thought Reviewer, and Safety and Ethics Reviewer.Source: §3 · p. 3, Figure 2 · p. 4

3.1 Aranging Specialist Doctors

  • The Primary Care Doctor assigns specialists from patient background and medical problem information, provides selection reasons, and uses few-shot examples for structured outputs.The design aims to avoid information contamination from excessive irrelevant expert agents.Source: §3.1 · p. 3

3.2 Multi-Round Consultations

  • The lead physician transforms specialist statements into consistency, conflict, independence, and integration components before storing them in the Historical Shared Pool.Consistency captures agreement, conflict identifies disagreement, independence preserves unique views, and integration synthesizes all perspectives.Source: §3.2 · p. 4
  • From the second round, specialists use structured statements from recent rounds, continue until consensus or 10 rounds, and otherwise apply majority voting with random tie resolution.The residual structure restricts access to deeper memory layers and is described as reducing information pollution, cognitive load, and hallucination risk to some extent.Source: §3.2 · pp. 4–5

3.3 Summary and Output Stage

  • A Safety and Ethics Reviewer verifies the consultation outcome and determines the final conclusion before validation against the correct result.The reviewer refines the consultation outcome and identifies potential safety concerns.Source: §3.3 · p. 5
  • The system retrieves the five most similar cases from both knowledge bases using cosine similarity to enhance specialist prompts.Knowledge bases are generally withheld from the initial round and consulted from the second round when conflicts arise, or retrospectively after first-round consensus.Source: §3.3 · pp. 5–6
  • Validated outcomes are stored in CorrectKB, whereas incorrect outcomes are summarized with error reflection and stored in ChainKB.The incorrect-case summary includes background, problem, initial hypothesis, analysis process, final conclusion, and reflection on error causes.Source: §3.3 · pp. 5–6

4 Experiments

Experiments evaluate MDTeamGPT on MedQA and PubMedQA, comparing its overall performance, module contributions, self-evolution, cross-model efficacy, and knowledge-base generalization. The framework reaches strong benchmark accuracy, benefits from its combined modules, improves with accumulated consultation experience, and transfers reasoning across datasets.

4.2 Main Results

  • After 600 consultation rounds, experiences were stored in CorrectKB and ChainKB using the respective training sets of MedQA and PubMedQA, while final results used test sets.Source: §4.1 · p. 6, §4.2 · p. 6

4.3 Ablation Studies

  • 5.8% and 3.6% accuracy gains on MedQA and PubMedQA, respectively, result when residual discussion and the lead physician are combined.The combined configuration substantially improves performance over the first ablation setting.Source: §4.3 · pp. 6–7, Table 2 · p. 7
  • CorrectKB alone exceeds ChainKB alone by 2.1% on MedQA, 5.4% on PubMedQA, and 3.8% overall, while combining both knowledge bases performs best.Both knowledge bases contain experiences from 600 consultation rounds, with CorrectKB retaining verified correct consultations and ChainKB providing reflective insights.Source: §4.3 · p. 7

4.4 Self-Evolving

  • Accuracy improves as consultation samples increase across tested LLMs and stabilizes after approximately 600 cases.LLaMA3-8B improves faster than stronger models such as gpt-4-turbo and gpt-4o.Source: §4.4 · p. 7, Figure 4 · p. 8

4.5 Framework Efficacy

  • MDTeamGPT improves accuracy over single-agent baselines for every evaluated language model, although gains vary with static knowledge and instruction-following ability.The comparison covers MedQA and PubMedQA using 600 stored consultation rounds.Source: §4.5 · p. 7, Figure 5 · p. 8

4.6 Generalization of the Knowledge Base

  • Cross-dataset knowledge transfer raises accuracy by 3.6% on MedQA using PubMedQA-derived knowledge and by 2.1% on PubMedQA using MedQA-derived knowledge.CorrectKB and ChainKB are constructed from one dataset and tested on the other, supporting generalization beyond isolated consultation cases.Source: §4.6 · p. 8, Table 3 · p. 8
  • CorrectKB and ChainKB transfer reflective reasoning and thought processes to new cases, not merely specific consultation examples.The knowledge bases preserve reasoning patterns and abstract cognitive strategies derived from accumulated inference and experience.Source: §4.6 · p. 8

5 Conclusion

MDTeamGPT combines residual discussion and consensus aggregation to reduce information redundancy and improve consultation efficiency, while two knowledge bases support accuracy improvement and generalization. Its performance remains partly constrained by the foundational model's ability to store and use consultation experiences.

  • MDTeamGPT uses residual discussion and consensus aggregation to minimize information redundancy and improve discussion efficiency.Source: §5 · p. 8
  • Two knowledge bases dynamically improve overall consultation accuracy and demonstrate some generalization ability.Source: §5 · p. 8
  • The framework's performance largely depends on the foundational model's ability to store and utilize consultation experiences.Source: §5 · p. 9

Ethical Considerations

The study uses publicly available academic datasets and accesses LLMs through official APIs, with stated practices addressing patient-data privacy, policy compliance, and transparency.

  • The study uses publicly released academic datasets intended for research and processes no personal patient data.Source: Ethical Considerations · p. 9
  • LLMs are accessed through official APIs in accordance with API usage policies, fair-use standards, and intellectual-property rights.Source: Ethical Considerations · p. 9
  • The authors describe experimental methods in detail to support reproducibility and transparency in scientific practice.Source: Ethical Considerations · p. 9

Limitations

The authors identify limitations involving simple agent designs, restricted dataset coverage, and limited evaluation in real-world consultation settings. Future work targets richer reasoning mechanisms, broader datasets, and real-world data.

  • The agents are currently defined quite simply, motivating future integration of Tree-of-Thought techniques or external tools and knowledge bases.Source: Limitations · p. 9
  • Experiments use only MedQA and PubMedQA, so broader datasets are needed to assess robustness across different data types.Source: Limitations · p. 9
  • The framework has mainly been evaluated on pre-existing datasets rather than real-world consultation data.Source: Limitations · p. 9
  • Future real-world consultation data could assess practical performance and generalization beyond controlled experimental environments.Source: Limitations · p. 9

A Prompt Templates

This section presents prompt templates that structure coordinated multi-round MDT consultation, including role-specific reasoning, experience extraction, lead-physician synthesis, retrieval-based storage, and safety filtering. It also illustrates how these templates support specialty selection, treatment refinement, and safe final-answer generation across patient cases.

A.1 Primary Care Doctor Agent

  • The Primary Care Doctor Agent selects specialty combinations from patient symptoms, signs, and history, requiring Radiologist, Pathologist, and Pharmacist roles while assigning others conditionally.The template requires reasons for each selection and uses examples involving pregnancy, infant sudden death, and neonatal vomiting.Source: §A.1 · p. 12

A.2 Specialist Doctor Agents

  • Other specialist doctor agents follow the Obstetrician and Gynecologist template, changing the medical background while preserving its logical structure.The shared template includes condition analysis, treatment evaluation, historical-pool storage and retrieval, feedback-based refinement, and final choice formatting.Source: §A.2 · p. 13
  • Specialist agents analyze patient condition, evaluate treatment options, store conclusions, retrieve prior rounds, incorporate feedback, and express an optimal treatment choice.The workflow emphasizes pregnancy-specific medication safety and repeats this process across consultation rounds.Source: §A.2 · p. 13
  • Nitrofurantoin is presented as the relatively safe and effective treatment choice for the pregnant patient’s UTI, remaining confirmed after later-round feedback.The example records the same choice in rounds 1, 2, and at least 3.Source: §A.2 · pp. 14–15

A.3 Lead Physician Agent

  • The Lead Physician Agent organizes each round’s specialist responses and stores the structured information in the Historical Shared Pool without directly diagnosing.Its responsibilities are to keep discussions structured, logical, and coherent across multiple rounds.Source: §A.3 · p. 15
  • The Lead Physician categorizes specialist responses into consistency, conflict, independence, and integration before synthesizing all perspectives.Consistency aggregates shared recommendations, conflict records disagreements, independence extracts unique insights, and integration forms a structured summary.Source: §A.3 · pp. 15–16
  • The lead-agent example collects specialist choices such as “E: Nitrofurantoin” and “B: Cephalexin” before processing them into categorized historical-pool data.The stored format is JSON-based and preserves summarized common aspects among specialist responses.Source: §A.3 · p. 16
  • The Lead Physician Agent identifies conflicting points, extracts unique viewpoints, and integrates all perspectives into a structured summary.Source: §A.3 · pp. 16–17

A.4 Chain-of-Thought Reviewer Agent

  • The Chain-of-Thought Reviewer Agent extracts doctors’ reasoning from the Historical Shared Pool and stores correct and incorrect consultation experiences in a vector database.Source: §A.4 · p. 17
  • The reviewer organizes initial assumptions, analysis, and conclusions into a logically ordered, detailed chain of thought for future analysis and retrieval.Source: §A.4 · p. 17
  • The reviewer analyzes incorrect diagnoses by locating faulty assumptions, biased reasoning, overlooked information, and specific improvements to prevent recurrence.Source: §A.4 · p. 17
  • The reviewer determines consultation correctness by retrieving similar cases from the Historical Shared Pool using cosine similarity, then selects the corresponding execution branch.Source: §A.4 · p. 17

A.5 Safety and Ethics Reviewer

  • The Safety and Ethics Reviewer checks final diagnoses and treatments, removes harmful or unethical suggestions, and confirms an ethically appropriate recommendation.Source: §A.5 · p. 18
  • In the pregnancy example, Ciprofloxacin was filtered as unsafe, Nitrofurantoin was confirmed safest, and the output was Answer ID E.Source: §A.5 · p. 19

A. Step 7: Store the Correct Answer

MDTeamGPT stores correct and incorrect consultation experiences differently: CorrectKB preserves the final correct round, while ChainKB records the full reasoning trajectory and error reflection for incorrect diagnoses.

A. Step 7: Store the Correct Answer

A. Step 8: Store the Incorrect Answer

B.1 Case Studies of the Lead Physician Agent’s Role and Function

The lead physician organizes specialist opinions into consistency, conflict, independence, and integration, using consensus and disagreement to refine consultation outcomes across cases.

Lead Physician Function

  • The lead physician’s workflow explicitly organizes current-round responses into consistency, conflict, independence, and integration categories.Source: §B.1 · p. 19

Case A

  • In Case A, most specialists selected blockade of presynaptic acetylcholine release at the neuromuscular junction (D), while the neurologist selected presynaptic calcium-channel autoantibodies (B).Source: §B.1 · pp. 19–20
  • The Case A integration favored presynaptic acetylcholine-release blockade (D) because pathologists, pharmacists, and pediatricians supported it as the most likely explanation.Source: §B.1 · pp. 20–21

Case B

  • In Case B, all experts selected bullous changes of the lung bases on chest CT (B), producing consistency without conflict or independence.Source: §B.1 · pp. 21–22
  • The Case B integration linked the unanimous imaging choice to jaundice, elevated ALT and AST, and liver-biopsy evidence of liver disease.Source: §B.1 · pp. 21–22

B.2 Specialist Selection Distribution

The Primary Care Doctor Agent selected multiple specialists for both datasets, typically assigning 3–5 specialists on MedQA and about 5 on PubMedQA.

Specialist Selection Distribution

  • On MedQA, the Primary Care Doctor Agent typically assigned around 3 to 5 specialists, while PubMedQA selections centered around 5.Source: §B.2 · p. 22, Figure 6 · p. 22
  • Selections of only 2 or as many as 7 specialists were uncommon, and no case involved selecting exactly one specialist.Source: §B.2 · p. 22

B.3 Human Evaluation

Human evaluation covered ten dimensions across ten questions, with most metrics scoring above 3 but lower ratings for efficiency, diversity, and completeness.

Human Evaluation

  • Five volunteers evaluated ten questions across MedQA and PubMedQA using ten dimensions, including accuracy, explainability, completeness, efficiency, safety, and innovation.Source: §B.3 · p. 23, Table 4 · p. 23
  • Efficiency received relatively low scores because multi-round, multi-role discussions required more time than a Single-Agent approach.Source: §B.3 · pp. 23–24
  • Diversity averaged below 3, while completeness averaged 3 and some feedback reported insufficient consideration of relevant factors.Source: §B.3 · p. 24
  • All other evaluated metrics received scores above 3, indicating potential value for practical medical applications.Source: §B.3 · p. 24

Related papers

Loading 2503.13856v1…