OA-MAP: Evidence-Grounded Multi-Agent Multimodal Framework for Interpretable Knee Osteoarthritis Progression

1 INTRODUCTION

• Agent reliability remains a challenge because agents can misinterpret clinical information, use tools incorrectly, and converge on shared errors rather than independently corroborate one another.These concerns motivate defined responsibilities, verifiable evidence, explicit disagreement handling, and transparent uncertainty assessment.Source: §1 · p. 2
• Specialists ground modality-specific assessments in model outputs and PubMed literature, while a fusion model uses available modalities to produce the final progression label.Reports also include modality-specific predictions, model explanations, literature assessments, and uncertainty indicators.Source: §1 · p. 2
• The framework presents intermediate findings, model explanations, and traceable literature evidence so clinicians can examine the basis of assessments and help limit unsupported agent interpretations.Reports include supporting evidence, conflicts, and evidence gaps.Source: §1 · p. 2
• An uncertainty score combines model-derived indicators, cross-modal disagreement, and literature assessments to evaluate prediction reliability and guide clinician review.Clinicians can correct suspected intermediate errors, triggering recomputation of affected predictions, literature assessments, uncertainty calculations, and the final report.Source: §1 · pp. 2–3
• OA-MAP automates patient-data retrieval, modality- and endpoint-specific tool selection, prediction, and report generation in response to clinical requests.This reduces the need for manual input preparation and coordination across separate processing pipelines.Source: §1 · p. 2

2.1 LLM AGENT

• Prior LLM-agent systems organize specialist reasoning, adapt collaboration to task complexity, or connect language agents with executable medical tools [Wu et al. (2023); Swanson et al. (2025); Tang et al. (2024); Kim et al. (2024)].MMedAgent, MedRAX, and AgentMD select or invoke specialized tools and calculators, while OA-MAP focuses on modality-specific KOA analysis and literature support [Li et al. (2024a); Fallahpour et al. (2025); Jin et al. (2025); Xiong et al. (2024)].Source: §2.1 · p. 3

2.2 KNEE OSTEOARTHRITIS

• KOA prediction studies use multimodal data for structural and symptomatic progression, while interpretability work applies nomograms, Grad-CAM, and feature-importance methods [Hu et al. (2023); Li et al. (2024b); Harari et al. (2025)].OA-MAP extends model-level interpretability by assessing feature–progression relationships against literature and examining agreement and conflicts across modality-specific predictions.Source: §2.2 · p. 3

3.3 COORDINATOR AGENT

• The coordinator combines available modality-specific outputs with fusion-model inference, uncertainty assessment, and a structured final report.It identifies modality disagreements and conflicts with retrieved literature, flags findings for clinician review, and consolidates the assessment.Source: §3 · p. 3, §3.3 · pp. 6, 8

3.1 MEMORY MODULE

• Patient records are indexed by patient ID and retain knee laterality, visit information, and modality availability for task-specific retrieval.Records include clinical data and imaging data such as 3D DESS MRI and X-ray-derived measurements.Source: §3.1 · pp. 3–4

3.2 MODALITY-SPECIFIC AGENT

• Clinical, MRI, and X-ray agents select modality- and endpoint-specific prediction models, then use model feature contributions to guide literature retrieval.The MRI agent can incorporate clinician corrections to ROI abnormality scores and rerun affected prediction and literature-assessment steps.Source: §3.2 · pp. 4–5
• Literature findings are assessed for comparability and directional agreement with model relationships, informing evidence support and uncertainty.The retrieval process prioritizes features with the largest prediction contributions; comparable findings are labeled supporting, conflicting, or inconclusive.Source: §3.2 · pp. 4–5

3.3 COORDINATOR AGENT

• Overall uncertainty combines prediction-boundary proximity, disagreement among modality predictions, and insufficient or conflicting literature evidence.Higher uncertainty indicates lower confidence; a predefined threshold can trigger clinician review, and corrections propagate to affected results.Source: §3.3 · pp. 6–7

4.1 DATASET

• The cohort contains 600 participants, with 297 progressors and 303 non-progressors for each endpoint; structural and persistent pain progression have separate definitions.Structural progression is a ≥0.7 mm reduction in minimum medial tibiofemoral joint-space width; pain progression is a ≥9-point WOMAC pain increase at two or more visits from months 24–60.Source: §4.1 · p. 8

4.2 PROGRESSION PREDICTION PERFORMANCE

• Fusion achieved AUROCs of 0.80 for structural progression and 0.68 for pain progression, the highest values among the reported modality-specific and fusion models.Table 1 covers AUROC for both endpoints across input modalities.Source: §4.2 · p. 8, Table 1 · p. 8

4.3 CASE STUDY: MULTIMODAL ASSESSMENT OF STRUCTURAL PROGRESSION

• The case-study section presents a baseline assessment for structural progression.The supplied case-study passages identify the assessment context but do not provide further case findings.Source: §4.3 · p. 9

• The fusion logistic-regression score is 0.8100, above the 0.4505 training-derived threshold, yielding a progressor classification.Source: §1 · p. 9

• MRI and clinical predictions agree with the final classification, while the X-ray score falls below its threshold; fusion, not majority voting, determines the result.Source: §2 · p. 9

3. Model explanation

• Features are ranked by absolute contribution to model log-odds; βjzj is the feature’s additive contribution, with positive values increasing and negative values decreasing the prediction score.Source: §3 · p. 9

3.1. Fusion model: top five contributions

• Kellgren–Lawrence grade and medial meniscus anterior horn abnormality probability appear among the fusion model’s top five contributions.Source: §3.1 · p. 9

3.2. MRI agent: top five contributions

• The MRI model’s top five contributions include medial meniscus anterior horn abnormality and lateral femoral cartilage bone-scaled volume index.Source: §3.2 · p. 9

3.3. X-ray agent: top five contributions

• The X-ray model’s top five contributions include Kellgren–Lawrence grade, minimum medial joint-space width, and medial joint-space-narrowing grade.Source: §3.3 · p. 9

3.4. Clinical agent: top five contributions

• The clinical model’s top five contributions include female sex indicator, baseline WOMAC physical function, and baseline age.Source: §3.4 · p. 10

• For comparable findings, the assessment aligns literature association direction with the corresponding logistic-regression coefficient sign; agreement supports a feature relationship but not individual progression.Source: §4 · p. 10
• MRI evidence for medial meniscal abnormalities was inconclusive and limited in comparability, while other features lacked clear comparable support, mainly because measurements or endpoints differed.Source: §4 · p. 10
• X-ray evidence supported the model relationship for medial joint-space-narrowing grade but conflicted with minimum medial joint-space width; other feature findings were inconclusive or not comparable.Source: §4 · p. 10
• Clinical BMI findings were inconclusive, while findings for sex, age, physical function, and medication use were not comparable to the corresponding model relationships.Source: §4 · p. 10
• These assessments characterize literature support for modality-specific feature relationships and evidence uncertainty, but do not establish correctness of the patient-level fusion prediction.Source: §4 · p. 10

• Prediction uncertainty was 0.7814, modality-conflict uncertainty was 0.0955, and literature evidence uncertainty was 0.8596.Source: §5 · p. 10
• The reported assessment distinguishes prediction uncertainty from modality-conflict and literature-evidence uncertainty.Source: §5 · p. 10
• Overall uncertainty was 0.5788, alongside complementary confidence of 0.4212.Source: §5 · p. 10

• Overall uncertainty exceeded the predefined 0.5 threshold, so the case was flagged as Clinician Review Required.Source: §6 · p. 10
• Review findings included X-ray cross-modality disagreement, medial meniscal and anterior medial tibial cartilage abnormalities, and feature relationships with conflicting or insufficient evidence.Source: §6 · p. 10
• Clinician corrections can pass through the feature-override workflow to update predictions and the report.Source: §6 · p. 10

5 DISCUSSION

• The framework covers structural and pain progression by integrating MRI, radiographic, and clinical information through modality-specific agents and a coordinator.Source: §5 · p. 10
• It connects automated prediction with interpretable findings, literature evidence, and clinician oversight, while reporting prediction uncertainty and cross-modal disagreement to support review.Source: §5 · p. 10
• Clinicians can correct intermediate MRI findings, with agents propagating corrections through subsequent predictions and reports while retaining automated coordination.Source: §5 · p. 11
• Future work will add analysis tools and prediction models and extend the framework to more clinical endpoints.Source: §5 · p. 11