Source-linked AI summary
Adaptive Memory and Reflection Multi-Agent System for Medical Question Answering
Pradeep Murugesan, Luoxiao Yang, Xueli Chen, Xinqi Fan
TL;DR
Medical QA systems often lack adaptability, persistent memory, and structured feedback. AMR addresses this with complexity-aware multi-agent routing, role-specific memory, reflection, and output screening, achieving its strongest results when memory, reflection, and retrieval are combined.
Problem
Existing medical QA approaches remain insufficiently adaptive and feedback-driven, while explicit ethical control mechanisms for clinical outputs are lacking.
Method
AMR combines complexity-aware multi-agent routing, role-specific memory, reflection feedback, collaborative reasoning, and explicit output screening.
Results
93.2% (MedQA) and 90% (MedMCQA) were achieved by full AMR with retrieval, outperforming its baseline and other tested configurations.
Takeaways & Limitations
Structured multi-agent systems show potential for more accurate and adaptive medical question answering.
Takeaways & Limitations
AMR was evaluated on retrospective benchmarks, while its reflection mechanism and ethical overseer remain unvalidated in real clinical workflows and by clinicians.
Abstract
from arXiv · showhide
Accurate and responsible medical question answering (QA) is important in healthcare, where complex cases require factual knowledge and nuanced reasoning. Existing medical QA systems, typically based on single-agent architectures and static retrieval, often lack adaptability, persistent memory, and structured decision-making. This work introduces an adaptive memory and reflection (AMR) agentic system, a multi-agent framework in which specialized agents use dedicated memory and reflection-based feedback to retrieve relevant prior cases and improve subsequent reasoning. Complexity assessment routes questions through solo, collaborative, or escalated workflows, while consensus and ethical overseer modules support reasoning consolidation and output review. Evaluation on MedQA and MedMCQA demonstrates strong performance compared with several baselines. Ablation studies show that combining agent-specific memory, reflection, and external retrieval yields the strongest performance. These findings highlight the potential of structured memory and feedback for developing more trustworthy medical agents. The source code is publicly available at https://github.com/mm-air/AMR-Agent.
I. INTRODUCTION
Medical QA systems support evidence-grounded clinical reasoning but often lack persistent memory, reflection, integrated adaptability, and explicit ethical controls. This work proposes AMR, a graph-based multi-agent framework combining role-specific memory, structured reflection, dynamic routing, consensus reasoning, and safety screening.
- Motivation: Medical QA grounds answers in biomedical knowledge, clinical evidence, and domain-specific reasoning for point-of-care access, literature understanding, education, and assessment.The field has developed from clinical evidence retrieval systems toward modern natural language processing.
- Limitations: Existing medical QA research emphasizes domain-specific LLMs, retrieval-augmented generation, and agent-based clinical decision discussions.Despite these directions, systems often lack persistent memory and reflection over time.
- Limitations: RAG and multi-agent approaches remain insufficient without integrated adaptability and feedback, while absent ethical controls can produce unreliable clinical outputs.These limitations motivate a framework that combines adaptive reasoning with explicit safety oversight.
- Proposed Framework: AMR is a graph-based multi-agent medical QA framework integrating role-specific memory, structured reflection-feedback, complexity-aware routing, consensus reasoning, synthesis, and ethical oversight.Its design is inspired by clinical team processes and aims to support more transparent and controllable workflows.
- Proposed Framework: Agent-specific memories and post-hoc reflection updates enable prior cases to be reused in a role-aware manner rather than through one undifferentiated memory store.This contribution targets adaptive reasoning through role-specific memory and reflection.
- Evaluation: The empirical study evaluates adaptive memory and reflection on MedQA and MedMCQA and reports the combined system design as yielding the strongest gains.The components are examined within the same framework to study their interactions.
II. RELATED WORK
The related work positions medical QA around increasingly capable LLMs, collaborative multi-agent reasoning, and retrieval-augmented generation, while identifying coordination and retrieval-alignment challenges. AMR addresses these gaps through explicit complexity-based routing and the integration of role-specific memory, reflection, and output screening.
- LLMs for Medical NLP: LLMs have advanced medical NLP, with biomedical training improving contextual understanding and structured ontologies enhancing interpretability and reducing hallucination.The passage specifically cites clinical summarization, concept extraction, and question answering as areas of progress.
- Multi-Agent Systems: Multi-agent systems support distributed, collaborative reasoning aligned with the interdisciplinary nature of clinical decision-making.Modern approaches use LLM-based agents that communicate and collaborate, following earlier rule-based coordination systems.
- Retrieval-Augmented Generation: RAG improves factual grounding by integrating external knowledge during inference but remains sensitive to retrieval misalignment, which can degrade reasoning consistency.The related work also identifies agent disagreement, coordination latency, and adaptive collaboration as continuing challenges.
- AMR Framework: AMR treats medical QA as a pipeline design problem by making routing explicit and selecting reasoning depth according to question complexity.This contrasts with applying fixed reasoning depth to every case.
- AMR Framework: AMR combines role-specific memory, post-hoc reflection, and Ethical Overseer output screening to reuse prior cases in a targeted way.These components are integrated within the same framework as a system-level shift in medical QA design.
III. METHOD · A. Overall Architecture · B. Adaptive Routing and Collaborative Reasoning
The AMR system is a modular multi-agent medical QA framework that combines agent-specific memory with reflection-driven feedback and complexity-adaptive routing. Questions follow low-, moderate-, or high-complexity workflows, with collaboration, consensus, iterative revision, ethical review, and final answer selection applied according to difficulty.
- III. METHOD: AMR combines agent-specific memory with reflection-driven feedback to address inconsistent reasoning, limited persistent memory, and unstructured feedback in medical QA.The framework is intended to support continual learning and improved reasoning over time.
- A. Overall Architecture: The framework uses a modular LangGraph-based architecture in which nodes represent agents or processing steps and edges define transitions.A Moderator assesses complexity, and a Recruiter routes each query to the corresponding pathway.
- A. Overall Architecture: Low-complexity questions are handled by a General Practitioner, whereas moderate-complexity questions use multi-agent collaboration and consensus.The routing labels are low, moderate, and high complexity.
- A. Overall Architecture: The execution pipeline consolidates pathway-specific reasoning, submits it to an Ethical Overseer, and maps the screened output to the returned answer option.The Final Answer Picker performs the final mapping after ethical review.
- B. Adaptive Routing and Collaborative Reasoning: Routing assigns direct recall and simple mechanism questions to the low-complexity path, symptom-and-specialty questions to collaboration, and competing interpretations to a longer workflow.The longer workflow includes draft generation, critique, refinement, and final decision.
- B. Adaptive Routing and Collaborative Reasoning: For moderate-complexity questions, specialists produce independent opinions, refine them using the shared transcript, and use a Consensus Facilitator to summarize agreements and disagreements.This pathway supports explicit consolidation of specialist reasoning.
- B. Adaptive Routing and Collaborative Reasoning: For high-complexity questions, intermediate reports enable reasoning revision before final answer selection, aligning reasoning depth with question difficulty.Adaptive routing avoids applying the same multi-agent procedure to every case and serves as both a computational shortcut and a control mechanism.
C. Agent-Specific Memory and Reflective Update
AMR gives each agent a separate, role-specific memory containing prior question-and-answer experience and reflective feedback. Reflection is triggered by incorrect predictions and stores corrective reasoning for future retrieval without changing model parameters.
- Agent-Specific Memory: Each AMR agent maintains a separate memory containing question context, answers, post-hoc notes, role metadata, and timestamps.Role-specific stores let agents retrieve different prior experiences for the same question.
- Reflective Update: Reflection is triggered only when the predicted answer differs from the ground-truth label.Corrective feedback and reasoning summaries are stored as reflective memory rather than used to update model parameters.
- Reflective Update: When an answer is incorrect, the system adds a feedback entry to every agent’s memory.Algorithm 1 applies the update across the team before continuing to the next question.
D. Answer Synthesis and Output Screening · E. Agent Roles and Responsibilities
AMR separates specialist-answer synthesis from final answer release and adds an explicit Ethical Overseer screening stage for potentially unsafe medical content. Its modular agent roles span assessment, recruitment, reasoning, synthesis, and screening, while the screening mechanism remains preliminary rather than formal clinical validation.
- D. Answer Synthesis and Output Screening: The Summarizer condenses specialist rationales into one explanation, and the Final Answer Picker maps that explanation to the returned answer option.AMR separates answer synthesis from answer release.
- D. Answer Synthesis and Output Screening: The Ethical Overseer reviews candidate responses for direct diagnostic statements, harmful advice, and treatment recommendations exceeding the intended educational scope.It screens responses before release for potentially unsafe medical content.
- D. Answer Synthesis and Output Screening: Explicit output screening makes policy decisions inspectable instead of burying them inside the reasoning prompt.A factually grounded answer may still require filtering or abstention under a deployment policy.
- D. Answer Synthesis and Output Screening: The Ethical Overseer evaluates the final response against predefined safety criteria and either approves it or flags it for non-compliance.The layer evaluates outputs rather than modifying the underlying reasoning process.
- D. Answer Synthesis and Output Screening: The screening component is intended as a preliminary safety mechanism to reduce potentially unsafe outputs, not as a substitute for formal clinical validation.This limitation applies to the current implementation.
- E. Agent Roles and Responsibilities: AMR defines modular agents with distinct responsibilities covering assessment, recruitment, reasoning, synthesis, and screening.Table I summarizes the roles and functions of these agents.
IV. EXPERIMENT AND DISCUSSION · A. Datasets · B. Implementation Details
Experiments evaluated the AMR system on MedQA and MedMCQA using separated retrieval and evaluation data. The implementation combined GPT-4o, LangGraph, FAISS-based memory retrieval, role-specialized agents, reflection logging, and sequential batch processing.
- A. Datasets: MedQA and MedMCQA served as established multiple-choice medical QA benchmarks differing in scale, difficulty, and explanation richness.MedQA emphasizes USMLE-style clinical reasoning, whereas MedMCQA offers large-scale exam questions with broader specialty coverage.
- A. Datasets: MedQA used its official training split for retrieval and its official test split exclusively for evaluation.
- A. Datasets: MedMCQA used disjoint training and evaluation subsets, with one subset supporting retrieval and an independent subset reserved for evaluation.
- B. Implementation Details: The system used a Python JSONL pipeline with GPT-4o through the OpenAI API, LangGraph workflow orchestration, and FAISS as the persistent vector store.
- B. Implementation Details: Agent memories were embedded with textembedding-3-large, indexed for semantic retrieval, and queried for the top five role-specific prior cases.
- B. Implementation Details: Questions were routed by the Moderator’s complexity assessment, while incorrect predictions triggered reflection updates and logs recorded routing, retrieval, prediction, and ethical-screening outcomes.
- B. Implementation Details: Role-specific prompts assigned complexity assessment to the Moderator, specialist selection to the Recruiter, independent reasoning to specialists, and final option selection to the Final Answer Picker.
- B. Implementation Details: Retrieval documents came from the training corpus, were chunked into overlapping passages, embedded, and indexed in FAISS; each benchmark was evaluated sequentially in batches of 50 questions.
C. Evaluation Metrics · D. Functionality Comparison with Other Methods
The evaluation reports accuracy and consistency, while the functionality comparison positions AMR as a memory-, feedback-, and screening-enhanced alternative to prior multi-agent methods. The comparison covers functions across multi-agent methods and emphasizes AMR’s robustness and adaptability.
- C. Evaluation Metrics: Accuracy is reported as the fraction of correctly answered questions.
- C. Evaluation Metrics: Consistency is assessed using the mean µ and standard deviation σ.
- D. Functionality Comparison with Other Methods: AMR introduces adaptive agent-specific memory and reflection-driven learning.These mechanisms support continual improvement and context-aware reasoning.
- D. Functionality Comparison with Other Methods: Unlike MDAgent [14], Debate [7], MedAgent [27], and ReConcile [4], AMR integrates memory, feedback, and post-generation screening.The cited prior frameworks rely on static or single-pass reasoning.
- D. Functionality Comparison with Other Methods: AMR’s integrated memory, feedback, and screening are presented as supporting robustness and adaptability.
- D. Functionality Comparison with Other Methods: TABLE III compares functions across multi-agent methods.The table provides the comparison context for AMR and the cited prior frameworks.
E. Ablation Study · F. Quantitative Comparison with Other Methods
The ablation study shows that agent-specific memory, reflection, and external retrieval provide complementary gains, with full AMR performing best on both benchmarks. Compared with representative alternatives, AMR is competitive with or stronger than several automated baselines and approaches the reported human reference on MedMCQA.
- E. Ablation Study: Full AMR with external retrieval achieved 93.2% on MedQA and 90% on MedMCQA, the strongest ablation result.The result demonstrates complementary benefits from retrieval, memory, and feedback.
- E. Ablation Study: AMR without retrieval reached 90% on MedQA and 87.4% on MedMCQA, outperforming the baseline configuration.This configuration combines memory and reflection without external retrieval.
- E. Ablation Study: The baseline, using multi-agent reasoning without memory, reflection, or external retrieval, achieved 80% on MedQA and 78% on MedMCQA.The ablation settings also include Feedback, Memory, and AMR without retrieval.
- E. Ablation Study: Feedback and memory progressively improved results relative to the baseline, supporting incremental contributions from reflection and agent-specific retrieval.Feedback denotes reflection without agent memory retrieval, while Memory denotes agent-specific retrieval without reflection.
- F. Quantitative Comparison with Other Methods: AMR was compared with representative single-agent, human, and prior multi-agent results on both MedQA and MedMCQA.The comparison included automated baselines and a reported human reference level.
- F. Quantitative Comparison with Other Methods: AMR was competitive with or stronger than several automated baselines on both benchmarks.The supplied comparison passage does not provide the individual baseline values.
- F. Quantitative Comparison with Other Methods: On MedMCQA, AMR approached the reported human reference level.The passage characterizes this relationship without specifying the human reference value.
G. Performance Across Different Question Complexity · H. Qualitative Analysis
AMR maintains mean accuracy around 0.85 or higher across question-complexity levels, with its best mean accuracy and lowest variability on moderate-complexity questions without RAG. Qualitative cases show adaptive retrieval supporting consistent reasoning, while reflection and ethical oversight expose and address errors and unsafe recommendations.
- G. Performance Across Different Question Complexity: AMR achieves mean accuracy around 0.85 or higher across all question-complexity levels without enabling RAG.
- G. Performance Across Different Question Complexity: Moderate-complexity questions produce AMR’s highest mean accuracy and lowest standard deviation, potentially reflecting collaborative reasoning and iterative debate.
- H. Qualitative Analysis: The qualitative analysis includes one successful case and one failure case of the AMR system.
- H. Qualitative Analysis: In the positive case, agents retrieve semantically similar prior cases, including surgical error disclosure scenarios, supporting consistent and context-aware reasoning.
- H. Qualitative Analysis: In the error case, AMR selects option “E” instead of “C”, triggering stored feedback containing the incorrect prediction, correct answer, and reasoning correction.
- H. Qualitative Analysis: The Ethical Overseer flags potentially unsafe recommendations and records the reason for review, illustrating a screening mechanism rather than a comprehensive quantitative evaluation.
V. CONCLUSIONS
The AMR framework structures medical QA around adaptive routing, agent-specific memory, collaboration, reflection, and output screening. The paper identifies retrieval, memory scalability, clinical validation, and benchmark coverage as limitations and proposes broader evaluation and improved memory management.
- Contributions: AMR integrates complexity-aware routing, agent-specific memory, collaborative reasoning, reflection, and output screening into a structured medical QA pipeline.The framework enables agents to refine intermediate reasoning and use retrieved knowledge and accumulated experience.
- Limitations: The framework remains constrained by retrieval quality, potentially declining memory effectiveness, retrospective-only reflection evaluation, and an ethically overseer not validated with clinical rules or clinicians.Stored experiences may become duplicated or less relevant as memory grows, while MedQA and MedMCQA do not fully capture clinical settings.
- Future Work: Future work will improve memory management and retrieval through pruning, confidence-based retention, recency-aware retrieval, reranking, and forgetting mechanisms.The authors also plan to extend AMR to open-ended clinical reasoning and real-world decision support, evaluating safety, latency, token consumption, and reasoning-quality tradeoffs.