Source-linked AI summary
FRAC-MAS: A Safe and Explainable Multi-Agent System for Fracture Diagnosis
Hardik Iyer, Tirath Bhathawala, Mihir Panchal, Ying-Jung Chen, Kiran Bhowmick, Pankaj Sonawane, Meera Narvekar
TL;DR
Fracture diagnosis remains difficult because missed injuries and black-box predictions limit safe clinical adoption. FRAC-MAS combines a four-model vision ensemble, conformal differentials, and multi-agent verification to produce explainable, human-supervised reports. The system reports strong external-set performance, improved coverage for ambiguous subtypes, safer triage, and more preferred patient communication.
Problem
Subtle fracture morphology, missed injuries, and limited interpretability create a need for explainable and safely supervised fracture-detection systems.
Method
FRAC-MAS combines four vision models with conformal prediction and a multi-agent workflow for independent verification, clinical retrieval, escalation, and patient-friendly reporting.
Results
96.4% stacking detection rate was achieved on an anatomically matched external set, while conformal calibration yielded 92.0% empirical coverage at α = 0.10.
Takeaways & Limitations
The framework supports auditable, human-in-the-loop fracture triage by combining explicit uncertainty, independent criticism, and clinically grounded explanations.
Takeaways & Limitations
External evaluation found modest OOD performance, with AUC 0.652, indicating limited transfer from wrist and hand X-rays to full-body radiographs.
Abstract
from arXiv · showhide
Fracture detection and its clinical interpretability see notable improvements when deep vision models are integrated with agentic AI architectures. While deep learning models achieve high diagnostic performance, their black-box nature limits clinical adoption. We propose FRAC-MAS, an agentic AI system for automated, explainable, and safe bone fracture detection. The framework combines a stacked ensemble of four vision models with conformal prediction to produce statistically grounded differential diagnoses, while a multi-agent workflow performs independent verification, retrieves clinical guidelines, and generates patient-friendly reports. A pipeline-depth ablation study confirms that our multi-agent critic triages 86.6% of cases into a high-confidence auto-confirmed cohort while escalating uncertain cases, outperforming a single-agent baseline. Patient preference studies against Llama, MedGemma, and Gemini further demonstrate significantly more comprehensible clinical reports. These results suggest that integrating multi-agent critics with conformal guarantees enables safer radiology triage while preserving clinician oversight. More broadly, FRAC-MAS demonstrates how cooperative agentic architectures can serve as auditable, human-in-the-loop decision support systems for safety-critical healthcare. Our code is available at https://github.com/hardik1712/FRAC-MAS, and the website is available at https://frac-mas.vercel.app.
1 Introduction
FRAC-MAS addresses missed and difficult fracture diagnoses by combining explainable vision predictions with independently verifiable, human-supervised agent workflows. Its contributions span modular diagnosis, conformal differentials, and patient-oriented reporting.
- Motivation: Plain-radiograph fracture diagnosis remains difficult because subtle morphologies, inter-observer variability, and high clinical volume contribute to missed injuries.Missed fractures can lead to malunion, avascular necrosis, disability, avoidable surgery, prolonged hospitalization, and economic losses.
- Motivation: 2–3% of orthopedic injuries are missed at first presentation, while radiographs may miss 10–20% of some skull and skeletal fractures.Explainable systems highlight image regions such as cortical discontinuity or trabecular collapse for clinician verification.
- Proposed system: FRAC-MAS decomposes fracture diagnosis into specialized, interpretable components that support human oversight and independent verification.The system is designed to mitigate automation bias and address edge cases where AI alone is unreliable.
- Proposed system: The workflow combines a Patient Interaction Agent with Knowledge, Critic, and Educational agents for evidence grounding, blind verification, escalation, and patient-facing explanations.Grad-CAM overlays connect the generated explanations to image regions used in the diagnostic process.
- Evaluation: The ensemble uses stacking with conformal prediction to disambiguate confused fracture categories and produce differential diagnoses with empirical coverage guarantees.The agent layer is evaluated through pipeline-depth ablation and patient preference comparisons against Llama, MedGemma, and Gemini.
2 Methodology
The methodology combines a four-model vision ensemble and conformal prediction with a four-agent workflow for auditable, clinically grounded fracture assessment. Specialized agents independently verify predictions, retrieve guidance, escalate ambiguity, and generate patient-friendly explanations.
- Data and models: The system trains on augmented HBFMID data containing 1,952 images across 8 fracture categories, with unaugmented validation and test splits.The validation and test sets contain 106 and 112 images, respectively.
- Data and models: Four vision models capture complementary fracture evidence through attention, radiological self-supervision, and preserved low-level edge detail.The models are MaxViT, YOLOv26m-cls, RAD-DINO, and HyperColumn-CBAM DenseNet-169.
- Uncertainty estimation: A split conformal prediction layer converts ensemble probabilities into prediction sets with distribution-free empirical coverage guarantees.Classes exceeding a calibrated threshold enter the set, which communicates uncertainty as a differential diagnosis.
- Multi-agent workflow: A four-agent workflow routes cases through patient interaction, knowledge retrieval, critic verification, and educational reporting roles.The design grounds outputs in verified clinical sources and independently checks diagnostic conclusions.
- Safety and escalation: The Critic Agent performs blind-first triage and escalates cases when it rejects the ensemble, disagrees above a 0.6 confidence threshold, or encounters multi-class conformal ambiguity.These conditions are intended to prevent anchoring bias and enforce human review of uncertain cases.
3 Results and Analysis
FRAC-MAS combines ensemble predictions, conformal differentials, multi-agent triage, and human-facing explanations to improve fracture-detection safety and interpretability. Evaluations cover external datasets, attention behavior, conformal coverage, clinician assessment, and patient preferences.
- Pipeline-depth ablation: 86.6% of test cases were triaged into a high-confidence auto-confirmed cohort by the multi-agent Critic, reducing unnecessary escalations versus the single-agent baseline.The single-agent baseline escalated every test case, while the Critic selectively auto-confirmed cases.
- External dataset evaluation: AUC 0.652 on FracAtlas showed limited transfer from wrist/hand radiographs to full-body radiographs, while the Critic flagged uncertain cases for human escalation.The system uses low ensemble confidence and conformal ambiguity to intercept out-of-distribution uncertainty.
- Conformal prediction: Conformal prediction converted difficult morphologically similar classes, especially Oblique and Transverse fractures, into clinically actionable differentials when argmax predictions were incorrect.Coverage gains concentrated among error-prone categories where the true label was often retained in the conformal set.
- Grad-CAM analysis: MaxViT localized attention to fracture lines or cortical discontinuities, whereas HyperColumn-CBAM captured broader structural context for complementary explanations.Table 4 reports active fractions of 2%-21% for MaxViT and 53%-67% for HyperColumn.
- Human validation: Three orthopedic clinicians found the system’s outputs aligned with clinical standards and were comprehensible for human review.The clinician reader study assessed diagnoses before and after reviewing system-generated outputs.
- Human validation: 55.4% top win rate and 1.77 best average rank showed that evaluators preferred FRAC-MAS reports over Llama, MedGemma, and Gemini outputs.The preference differences were statistically validated with a Friedman test and post-hoc Wilcoxon signed-rank tests.
4 Limitations and Conclusion
The study presents FRAC-MAS as a modular framework for human-verifiable orthopedic diagnosis, combining diverse vision models, explainability, retrieval, critique, and conformal prediction. Its evaluation reports strong detection, coverage, triage, and human-validation results while acknowledging practical limitations requiring future work.
- Conclusion: FRAC-MAS combines diverse vision backbones, Grad-CAM, retrieval-anchored context, a VLM critic, and conformal prediction for human-verifiable orthopedic diagnoses.The framework integrates these components into a modular multi-agent system.
- Conclusion: 96.4% stacking detection rate, 92.0% empirical coverage at α = 0.10, and a 7.5× reduction in escalations summarize the reported system performance.The pipeline-depth ablation also reports a +2.7 percentage point safety margin on confirmed cases.
- Conclusion: Clinician reader studies and patient preference surveys favored the system’s outputs, with mean clinical accuracy rated 4.14/5.The conclusion describes these evaluations as evidence of clinical accuracy and comprehensibility.