Source-linked AI summary
MA-RAG: Multi-Agent Retrieval-Augmented Generation for Query-Driven Summarization of Longitudinal Parkinson's Disease Assessments
Sana Alamgeera, Denise Goberta, Muhammad Irshad, Anne H. H. Ngu
TL;DR
Parkinson’s longitudinal assessment is difficult to interpret reliably because clinicians must integrate multimodal scores across visits, while existing LLM and RAG systems can hallucinate or lose temporal consistency. MA-RAG addresses this with metadata retrieval, domain-specialized agents, structured fact extraction, and final verification across four clinical analysis tasks. In offline evaluations, it outperformed the reported baselines, with up to a 122% relative Fact Precision improvement and a 98% Hallucination Rate reduction, while receiving strong clinical-expert ratings.
Problem
Interpreting Parkinson’s multimodal assessments across visits is time-consuming and requires integrating structured motor, non-motor, activities-of-daily-living, and quality-of-life data.
Method
MA-RAG uses metadata-based retrieval, domain-specialized agents operating on structured fact subsets, and final verification against a score table.
Results
MA-RAG consistently outperformed Traditional, RAG-only, and Single-agent RAG baselines across single-visit and longitudinal tasks, with Fact Precision rising from 0.436 to 0.990 and Hallucination Rate falling from 0.564 to 0.010.
Takeaways & Limitations
MA-RAG supports reliable query-driven summarization of structured longitudinal clinical assessments within the evaluated Parkinson’s disease setting.
Takeaways & Limitations
The implementation is tailored to Parkinson’s disease assessments using UPDRS and PDQ-8, with evaluation on additional clinical domains left for future work.
Abstract
from arXiv · showhide
Accurate interpretation of single-visit and longitudinal clinical assessments for Parkinson's disease is time-consuming and often depends on specialist expertise. Although large language models (LLMs) can generate natural language summaries, they frequently lack domain-specific clinical grounding and struggle to produce factually correct and temporally consistent responses for structured longitudinal assessment data. To address these limitations, we propose MA-RAG, a query-driven multi-agent retrieval-augmented generation framework that decomposes clinical reasoning into domain-specialized agents, combines structured fact extraction, and synthesizes clinically grounded summaries through a final verification stage. The framework supports four clinical analysis tasks: single-session, trajectory, comparison, and cohort summarization. We evaluate MA-RAG using objective metrics, namely Fact Precision, Hallucination Rate, Temporal Fidelity, and Semantic Similarity, together with subjective evaluations conducted by clinical experts. Compared to Traditional, RAG-only, and Single-agent RAG baselines, MA-RAG substantially improves factual correctness, achieving up to a 122% relative increase in Fact Precision (from 0.436 to 0.990) and reducing the Hallucination Rate by up to 98% (from 0.564 to 0.010), while consistently receiving top ratings from clinical experts for organization and clinical usefulness. These results demonstrate that domain-specialized multi-agent reasoning enables reliable query-driven summarization of structured longitudinal clinical assessment data.
1. Introduction
Parkinson’s assessments span multiple domains and visits, creating a substantial burden for clinicians who must manually integrate scores and track progression. MA-RAG addresses these challenges with structured retrieval, domain-specialized reasoning, and final verification.
- Clinical challenge: Parkinson’s disease requires continuous multimodal assessment across motor, non-motor, activities-of-daily-living, and quality-of-life domains.UPDRS Parts I–III and PDQ-8 provide complementary structured assessments, with higher ordinal scores indicating greater symptom burden.
- Clinician burden: Manual documentation, score calculation, and cross-domain integration make comprehensive assessment time-consuming during routine clinical encounters.Complementary assessment domains often remain isolated, forcing clinicians to mentally combine single-domain findings.
- Clinician burden: Longitudinal interpretation is difficult because clinicians must compare domain scores across visits without a unified view of disease evolution.Static staging scales such as Hoehn and Yahr may not fully reflect short-term changes.
- Existing limitations: Standard LLMs and unstructured RAG can hallucinate scores, neglect non-motor or quality-of-life indicators, and fail to maintain temporal consistency.These limitations undermine reliable interpretation of structured longitudinal assessments.
- Proposed response: MA-RAG combines metadata-based retrieval, domain-specialized agents, structured fact tables, and final score-table verification for longitudinal clinical summarization.The framework is evaluated with objective and subjective measures of factual and clinical quality.
2. Related Work
Clinical summarization has progressed from rigid templates to flexible language models, retrieval, and agentic systems, but longitudinal multimodal reasoning remains inadequately grounded and verified. MA-RAG is positioned as a unified response to these gaps.
- Prior paradigms: Template-based systems preserve factual consistency but lack flexibility for cross-domain relationships and variable longitudinal patterns.Their fixed textual mappings constrain adaptive clinical synthesis.
- Prior paradigms: Abstractive transformer models improve fluency and domain adaptation but generate without explicit grounding mechanisms, allowing hallucinated clinical content.ClinicalBERT and BioGPT are cited as examples of improved domain adaptation over rule-based methods.
- Prior paradigms: Dense-retrieval RAG improves grounding for single-session summaries but remains inadequate for complex longitudinal multimodal reasoning.The passage identifies failures in extending retrieval-based grounding to longitudinal settings.
- Prior paradigms: Agentic clinical systems reduce arithmetic errors and cross-domain distraction through specialized modules and deterministic tools, yet lack explicit verification.Their decomposition addresses some failure modes without fully checking generated narratives against structured facts.
- Research gap: No existing framework combines metadata retrieval, domain-specialized agents, and fact-table verification for longitudinal multimodal assessment summarization.Common metrics such as ROUGE, BLEU, BERTScore, and SBERT similarity also inadequately capture factual correctness and temporal consistency.
3. Methodology
MA-RAG converts a natural-language clinical query into a structured plan, retrieves exact patient-visit records, partitions facts by domain, and routes analysis to specialized agents. The final pipeline produces summaries for single-session, trajectory, comparison, and cohort tasks.
- Task definition: MA-RAG receives a natural-language query and generates a clinically grounded summary that preserves temporal context and integrates selected assessment domains.The framework is defined over ordered patient visits and domain-specific assessment scores.
- Task definition: The framework supports single-session, trajectory, comparison, and cohort analyses for distinct patient-level and population-level questions.These tasks cover one visit, one patient’s progression, multi-patient contrasts, and cohort-wide extremes or patterns.
- Orchestrator: The Orchestrator converts the query into a plan π=(P, a, M, ϕ) specifying patients, analysis type, instruments, and visit filter.If parsing fails, safe defaults allow the pipeline to proceed with trajectory analysis, both instruments, and no visit filter.
- Retrieve Node: The Retrieve Node gathers exact session records by patient metadata, extracts relevant domain and item scores, and splits the resulting fact table into domain-specific subsets.These subsets are passed to downstream agents so each receives facts relevant to its specialty.
3.4. Router
The Router selects the next analysis branch from the resolved intent, directing comparison, cohort, trajectory, and single-session queries to specialized agents. Domain branches isolate each assessment domain, while comparison and cohort branches aggregate broader patient-level information.
- Intent-based routing: The rule-based Router selects the next stage solely from the resolved intent, without invoking a language model.Its mappings distinguish comparison, cohort or risk, and trajectory or single-session analyses.
- Intent-based routing: Comparison queries receive the full fact table because patient comparisons require access to all domains simultaneously.Trajectory and single-session queries instead invoke domain agents with only their corresponding fact subsets.
- Domain routing: The Domain Agents run sequentially for Motor, ADL, Non-Motor, and QoL data, using domain-specific facts and static reference text to write progress narratives.An empty domain fact subset causes that agent to skip processing.
- Domain routing: Each domain agent computes visit-to-visit and baseline changes, labeling decreases as improvement and increases as worsening.The resulting domain progression summaries are passed to the Final Agent.
- Comparison and cohort branches: The Comparison Agent builds a rule-based table of domain scores and directional changes for every patient and visit before an LLM generates the cross-patient summary.The generated comparison output is then passed to the Final Agent.
- Comparison and cohort branches: The Cohort Agent computes progression slopes and cross-patient maximum, minimum, and average statistics, then selects a query-matched statistic for clinical-language summarization.Its output includes the selected statistic, query, and combined domain reference text before final verification.
3.8. Final Agent
The Final Agent verifies whichever upstream branch produced a narrative against structured facts before generating the final clinical summary. It selects a synthesis mode for cohort, comparison, single-session, or trajectory analyses and formats the response accordingly.
- Verification: The Final Agent receives the upstream narrative from a domain, comparison, or cohort branch together with the fact table.Its role is to verify the narrative against underlying facts before synthesis.
- Verification: The agent builds a structured score table from the fact table and uses it to check reported information before writing the summary.Only confirmed facts are intended to enter the final output.
- Mode selection: A dispatcher selects cohort, comparison, single-session, or trajectory synthesis according to the producing branch and analysis type.The mode determines which upstream narrative is verified and summarized.
- Output formats: Trajectory mode summarizes progression across visits, cross-domain patterns, and domains requiring closest monitoring.It anchors progression to baseline scores and cumulative change.
- Output formats: Comparison mode contrasts patients one domain at a time, then reports cross-patient discordance and divergence points.The final output is returned as the clinical summary for the query.
4. Implementation Details
The implementation stores longitudinal assessment records with patient and visit metadata, retrieves them by exact metadata matching, and evaluates summaries with factual, temporal, and semantic metrics. The protocol reports mean metrics by task type using clinician-assisted gold summaries.
- Data and storage: The dataset contains 44 adults with Parkinson’s disease assessed using UPDRS and PDQ-8, with baseline Hoehn and Yahr stages 1–3.Participants had a mean age of 67.1 years and mean disease duration of 4.3 years.
- Data and storage: Each session-level record stores a patient identifier, standardized visit timestamp, item-level scores, and a chronological visit index with baseline defined as visit 1.These fields support longitudinal retrieval and ordering.
- Data and storage: Visit documents encode four domain scores and item-level scores, and are labeled by instrument and session-versus-longitudinal granularity.The documents are embedded and stored with metadata in a persistent ChromaDB knowledge base.
- Inference configuration: Static domain contexts provide clinical definitions, scoring guidelines, interpretations, and reliability statistics for the four assessment domains.They are initialized directly for agents rather than retrieved from the knowledge base to avoid repeated retrieval and truncation.
- Evaluation: The offline protocol compares generated summaries with clinician-assisted GPT-5.5 gold summaries using Fact precision, hallucination rate, Temporal fidelity, and SBERT similarity.All four metrics are computed per query and averaged by single-session or longitudinal task type.
- Evaluation: Fact precision measures the fraction of generated numeric tokens that also appear in the extracted fact table, whereas hallucination rate measures the fraction absent from it.These metrics directly assess numeric grounding against structured facts.
- Evaluation: Temporal fidelity is one when the visit indices mentioned in a summary appear in increasing order.The metric tests whether generated visit references preserve chronological ordering.
- Evaluation: SBERT similarity is the cosine similarity between generated and gold-summary sentence embeddings computed with all-MiniLM-L6-v2.Both embeddings have dimension d.
5. Results and Discussion
MA-RAG achieved the strongest factual and hallucination performance across single-session and longitudinal evaluations, with gains especially pronounced for longitudinal queries. Clinicians rated summaries positively, while the framework incurred higher inference costs and remains tailored to Parkinson’s disease assessments.
- Comparison Across Different Frameworks: MA-RAG reached Fact Precision values of 0.970 for UPDRS-only and 0.990 for combined UPDRS and PDQ-8 longitudinal queries.The corresponding baseline progressions were 0.436, 0.652, 0.780, and 0.970 for UPDRS-only, and 0.592, 0.652, 0.765, and 0.990 for the combined setting.
- Comparison Across Different Frameworks: MA-RAG consistently achieved the highest Fact Precision and lowest Hallucination Rate across single-session and longitudinal tasks.The reported gains were most pronounced in longitudinal queries, where domain-specialized agents and final verification provided the final performance improvement.
- Comparison Across Different Frameworks: Surface-level metrics did not reliably reflect factual correctness: Single-agent RAG had Temporal Fidelity 0.700 but Hallucination Rate 0.180, versus 0.070 for MA-RAG.Traditional RAG had the highest SBERT Similarity, 0.634, despite lower Fact Precision, 0.639, in the same single-visit UPDRS+PDQ-8 setting.
- Comparison Across Different Frameworks: Paired t-tests found all MA-RAG comparisons against baselines statistically significant at p<0.05 across tasks and metrics.The authors report this as evidence that MA-RAG’s superior factual precision was robust and consistent.
- Comparison Across Different LLMs: Llama-3.1-8B-Instruct produced the highest Fact Precision and lowest Hallucination Rate across all evaluated tasks.For longitudinal queries, its Fact Precision was 0.970 for UPDRS and 0.990 for UPDRS+PDQ-8, with Hallucination Rates of 0.020 and 0.010; single-visit precision scores were 0.940 and 0.930.
- Subjective Evaluation: Clinicians rated all summary types positively, with mean scores ranging from 3.67 to 5.00; Cohort summaries received the highest and most consistent ratings.Open-ended feedback praised factual content and organization, while Trajectory summaries prompted suggestions for treatment context, cross-domain reasoning, terminology, and concision improvements.
- Computational Cost, Ethical Considerations and Limitations: Trajectory and Single-Session analyses required six LLM calls, whereas Comparison and Cohort analyses required three calls.These corresponded to relative costs of 6.0× and 3.0× a single-pass RAG baseline, respectively.
- Computational Cost, Ethical Considerations and Limitations: MA-RAG is currently instantiated for Parkinson’s disease assessments using UPDRS and PDQ-8, although its modular architecture could support adaptation to other diseases.The domain-specific reasoning layer, prompts, and extraction rules require adaptation, with broader clinical evaluation left for future work.
6. Conclusion ad Future Work
MA-RAG combines domain-specialized agents, structured fact extraction, and final verification to summarize longitudinal Parkinson’s assessments. It outperformed the evaluated baselines, while future work targets broader diseases, data sources, adaptation, and prospective validation.
- MA-RAG combines domain-specialized agents, structured fact extraction, and final verification to generate longitudinal multimodal clinical summaries.
- 122% relative improvement in Fact Precision, from 0.436 to 0.990, was achieved compared with the Traditional baseline.
- 98% reduction in Hallucination rates, from 0.564 to 0.010, was achieved compared with the Traditional baseline.
- Clinical experts rated the summaries clinically meaningful and well organized across all four analysis types.
- Future work will extend MA-RAG to additional neurological disorders, heterogeneous clinical data, clinician-feedback adaptation, and larger prospective studies.
B. Static Clinical Context
Static domain contexts provide each clinical agent with assessment definitions, scoring rules, clinical factors, and reliability statistics. These contexts replace Knowledge Base retrieval to maintain complete and consistent domain knowledge during inference.
- Static contexts cover motor, ADL, nonmotor, and QoL domains for their respective agents.
- Each domain context encodes scale definitions, item-level scoring rules, clinical factor domains, and reliability statistics.
- The combined static context is distributed to corresponding agents instead of Knowledge Base retrieval.
- Motor context: UPDRS Part III rates each motor-examination item from 0 to 4, with higher scores indicating worse motor function.
- Non-motor context: UPDRS Part I rates non-motor experiences of daily living on the same 0–4 scale.
- QoL context: The PDQ-8 is treated as a short-form derivative of the PPDQ, with the listed scoring method applied by analogy.
C. System Prompts
The system prompts convert free-text Parkinson’s questions into constrained analysis plans. Rules select participants, analysis type, metrics, and visit filters, with examples covering cohort, single-session, comparison, and trajectory queries.
- The orchestrator prompt converts a user question into a structured clinical analysis plan.
- Plans must specify patient participants, one analysis type, one or both metrics, and a permitted filter.
- Analysis type is selected by query cues: comparison, trajectory, cohort, or single-session.
- Filters distinguish all visits, baseline, latest visit, baseline-versus-latest, and exact first- or last-N visit windows.
- UPDRS is selected for motor, ADL, or non-motor requests, PDQ8 for QoL requests, and both for general queries.
- The planner returns only the structured plan using the required PARTICIPANTS, ANALYSIS_TYPE, METRICS, and FILTER format.
D. Example Outputs
Example outputs illustrate single-session, cohort, comparison, and trajectory analyses across Parkinson’s assessment domains. They report domain scores, fluctuations, cross-domain patterns, monitoring priorities, and progression risks.
- Trajectory analysis: Patient 1’s Motor score changes from 14 at baseline to 18, then 7, 20, and 0 across later visits.
- Trajectory analysis: Patient 1’s ADL score changes from 13 to 22, 3, and 23, reflecting pronounced visit-to-visit fluctuation.
- Trajectory analysis: Patient 1’s NonMotor score changes from 13 to 15, 7, 19, and 0, with hallucinations and psychosis driving the later worsening.
- Single-session analysis: Baseline scores were Motor 14, ADL 13, and Non-Motor 13, with sleep, speech, handwashing, and salivation concerns highlighted.
- Single-session analysis: The baseline interpretation describes a complex motor, ADL, and non-motor profile requiring comprehensive management.
- Single-session analysis: The Non-Motor domain receives closest follow-up because of its high score, sleep problems, and daytime sleepiness.
- Cohort analysis: Patient 20 is identified as high risk because of the largest motor increase of 16 points and fastest QOL worsening slope of 8.0.
- Comparison analysis: Patient 4 shows ADL improvement followed by worsening, whereas Patient 7’s ADL score worsens gradually.