Source-linked AI summary
ConsultMind:Towards Automated Diagnostic Consultation via Uncertainty-Aware Reasoning
Xiao Sun, Yuming Yang, Yun Chen, Jiang Zhong, Junnan Zhu, Xinyi Jiang, Haoyang Zeng, Ruirui Chen, Yining Wang, Xinyu Zhou, Rong Tang, Kaiwen Wei
TL;DR
Diagnostic consultation must ground patient responses as evidence and decide what to ask next. AutoDisym constructs a Disorder–Symptom Bayesian Network, and ConsultMind uses evolving diagnostic posteriors to guide inquiry. Across model-specialty combinations, ConsultMind improves diagnostic accuracy over direct prompting.
Problem
Sequential consultation requires mapping patient responses to grounded symptom evidence and selecting subsequent inquiries from the evolving diagnostic state.
Source: Uncertainty Awareness · p. 4, Adaptive Reasoning · p. 4Method
AutoDisym constructs Disorder–Symptom Bayesian Networks from clinical narratives, while ConsultMind updates disorder posteriors and selects inquiries using the diagnostic state.
Source: Uncertainty Awareness · p. 4, Adaptive Reasoning · p. 4, Conclusion · p. 8Results
ConsultMind improves all 21 model-specialty combinations over direct prompting, with gains reaching 22.15 percentage points in Top-1 accuracy and 37.89 points in Top-3 accuracy.
Source: Experiment Results · p. 5Takeaways & Limitations
In an overlapping presentation, ConsultMind directed questions toward discriminative symptoms and yielded stronger evidence for schizoaffective disorder than the narrative-led approach.
Source: Case Study · p. 7
Abstract
from arXiv · showhide
Diagnostic consultation is an online sequential decision-making process in which clinicians gather evidence through patient interaction until a diagnosis is sufficiently supported. Automating this process requires adaptive inquiry and interpretable decisions. Bayesian networks offer a natural foundation by updating diagnostic posteriors as evidence accumulates, but their use in open-ended consultation raises two challenges: linking diagnostic hypotheses to potential inquiries and translating evolving posteriors into consultation decisions. We introduce AutoDisym, an automated pipeline that integrates diagnostic knowledge with heterogeneous diagnosis-labeled clinical narratives to construct a Disorder--Symptom Bayesian Network (DSBN). Building on the DSBN, we propose ConsultMind, an uncertainty-aware framework that updates disorder posteriors after each response and uses posterior uncertainty to guide inquiry and diagnosis. We evaluate both methods across psychiatry, respiratory medicine, fever clinics, and three public datasets. The results show that AutoDisym can automatically construct high-quality DSBNs and that ConsultMind consistently improves diagnostic performance and explanation soundness. For example, AutoDisym achieves macro-averaged F1 scores of 81.37 for canonical symptoms and 72.19 for manifestations using GPT-5.6-Sol. ConsultMind improves Top-1 and Top-3 diagnostic accuracy by up to 22.15 and 37.89 percentage points, respectively. Physician evaluation further shows that ConsultMind improves the quality of ranking explanations, differential diagnoses, and diagnosis rationales across LLMs of different scales. This work offers a promising approach to automatic diagnostic consultation.
Uncertainty Awareness
ConsultMind grounds each patient response as symptom evidence before updating disorder probabilities. It uses the resulting posterior uncertainty to inform subsequent reasoning.
Uncertainty Awareness
- An LLM-based grounding module maps each response to a canonical symptom state, abstaining when no reliable mapping is supported.Explicit negations count as absent; unmentioned symptoms remain unobserved.Source: Uncertainty Awareness · p. 4
- DSBN inference estimates each disorder’s probability given the grounded evidence and normalizes these values into a diagnostic posterior.Source: Uncertainty Awareness · p. 4
- Only grounded observations update the posterior; without evidence, it reduces to normalized disorder priors, and its entropy measures diagnostic uncertainty.That uncertainty informs subsequent reasoning.Source: Uncertainty Awareness · p. 4
Adaptive Reasoning
ConsultMind selects inquiry strategies and symptom targets from the evolving diagnostic state, then uses patient responses to continue or end consultation. The section also describes the clinical data, evaluation setup, and LLM baselines.
Adaptive Reasoning
- ConsultMind encodes the posterior and accumulated evidence in a six-dimensional state, scores three strategy prototypes, and selects the strategy with the highest softmax probability.The strategies are Exploration, Differentiation, and Consolidation.Source: Adaptive Reasoning · p. 4
- Exploration favors symptoms characteristic of a leading disorder, Differentiation favors distributionally distinct symptoms among competitors, and Consolidation seeks likely symptoms of the leading disorder.Source: Adaptive Reasoning · p. 4
- After each posterior update, consultation ends when uncertainty is sufficiently low or stops decreasing; symptom exhaustion and a turn limit are fallback conditions.An LLM turns the selected symptom into a patient-facing question.Source: Adaptive Reasoning · p. 5
- The DSBN corpus comprises 14,541 cases from psychiatry, respiratory medicine, and fever clinics, collected as consultation dialogues, electronic health records, and clinical cases.The psychiatric cases pair dialogue and record narratives for cross-format consistency evaluation.Source: Data Collection · p. 5
- The disjoint evaluation set contains 990 cases across 78 disorders: 287 respiratory, 312 psychiatric, and 391 fever-clinic cases.Source: Data Collection · p. 5
- Virtual standardized patients generate responses from structured records and cite supporting spans; an auditor checks consistency, evidence support, and unsupported claims.Undocumented information must be expressed as uncertain rather than inferred absent.Source: Data Collection · p. 5
- Evaluation records were institutionally deidentified and locally rewritten to protect privacy; clinicians checked preservation of relevant information, and source records will remain restricted.The evaluation set and learned DSBN parameters are slated for release upon acceptance.Source: Data Collection · p. 5
- The evaluation includes small, large, and medical LLM baselines, including Qwen, Llama, GPT, Gemini, DeepSeek, Claude, ClinicalGPT-R1, and HuatuoGPT-O1-7B.Source: Data Collection · p. 5
Experiment Results
Across the evaluated settings and datasets, ConsultMind improves diagnostic accuracy and explanation soundness, while AutoDisym produces high-quality symptom schemas. The experiments also assess virtual-patient reliability and schema agreement.
RQ1: Are Virtual Standardized Patients Reliable?
- Audited virtual standard patients achieve over 88.9% coverage and 93.2% precision, with hallucination below 3.1% across backbone sizes.Auditing raises coverage by up to 16.51 points and reduces hallucination by up to 4.72 points.Source: Experiment Results · p. 5, Table 2 · p. 6
RQ2: Does ConsultMind Improve Diagnostic Accuracy across LLMs and Specialties?
- ConsultMind improves all 21 model–specialty combinations over direct prompting, with gains up to 22.15 percentage points in Top-1 accuracy and 37.89 points in Top-3 accuracy.It also outperforms RAG-enhanced diagnosis across LLMs and specialties.Source: Experiment Results · p. 5, Table 1 · p. 6
RQ3: How Does ConsultMind Compare with Medical LLMs?
- Against medical LLMs on the same Qwen2.5-7B base, ConsultMind reaches 40.42% Top-1 and 66.35% Top-3 accuracy, surpassing Clinical-R1 by 7.56 and 12.06 points, respectively.It also achieves the highest MRR while requiring fewer dialogue turns than either medical LLM.Source: Experiment Results · p. 6
RQ4: Does ConsultMind Generalize across Datasets?
- ConsultMind improves all nine model–dataset combinations, with gains up to 16.78 points in Top-1 accuracy, 24.51 points in Top-3 accuracy, and 0.187 in MRR.The external evaluation covers MentalHospital, MedSP1000, and AIHospital.Source: Experiment Results · p. 6, Table 3 · p. 7
RQ5: Does ConsultMind Improve the Clinical Soundness of Diagnostic Explanations?
- ConsultMind improves physician-rated diagnostic explanations, with Llama-3.1-8B’s overall score rising from 2.30 to 3.74 and its Ranking Explanation score from 2.04 to 3.58.Specialists rated Ranking Explanation, Differential Diagnosis, and Diagnosis Rationale on a five-point scale.Source: Experiment Results · p. 6, Figure 5 · p. 7
RQ6: How Reliable Are AutoDisym-Generated Symptom Schemas?
- AutoDisym performs best with both backbones, reaching 81.37 symptom F1 and 72.19 manifestation F1 with GPT-5.6-Sol.It also reduces the manifestation-F1 gap between backbones from 9.68 to 5.96.Source: Experiment Results · p. 7, Table 4 · p. 7
Case Study
In an overlapping psychiatric presentation, ConsultMind redirects inquiry toward discriminative symptoms and the model predicts schizoaffective disorder rather than following a medication-related narrative toward bipolar I disorder.
Case Study
- Without ConsultMind, GPT-5.6-Sol follows the medication-related narrative and predicts bipolar I disorder; with it, the model probes discriminative symptoms and predicts schizoaffective disorder.The probes include grandiosity, fixed false beliefs, and suspiciousness.Source: Case Study · p. 7
- The case illustrates ConsultMind reducing narrative drift by directing inquiry toward discriminative symptoms in overlapping presentations.Source: Case Study · p. 7
- Across three clinical settings and three public datasets, the paper reports that ConsultMind consistently improves diagnostic performance and explanation soundness.Reported Top-1 and Top-3 accuracy gains reach 22.15 and 37.89 percentage points, respectively.Source: Conclusion · p. 8