Source-linked AI summary
Beyond Information Seeking: Severity-Aware Question Supervision for Proactive Medical Dialogue
Chenxuan Li, Xinrong Chen, Luyan Zhang, Peidong Jia, Zhongyu Zhao, Xuecheng Shang, Peixing Wan
TL;DR
Proactive medical dialogue must select questions from incomplete patient information, but uncertainty reduction does not account for unequal diagnostic consequences. The paper introduces ESR, which ranks questions by expected severity-aware risk reduction under unobserved answers and distills the rankings into a prefix-only policy. Across matched DDxPlus experiments, ESR lowers high-severity misses and improves accuracy with only a small increase in question count.
Problem
Information-seeking question acquisition can reduce diagnostic uncertainty without accounting for the unequal consequences of different diagnostic errors.
Method
ESR scores candidate questions by expected reduction in severity-aware terminal risk, marginalizing over unobserved answers and distilling rankings into a prefix-only language policy.
Results
−29.5%: Across three matched Qwen3-4B seeds on DDxPlus, mean high-severity diagnostic miss falls from .0645 to .0455 and accuracy rises from .9123 to .9320 with 0.14 additional questions per dialogue.
Takeaways & Limitations
Information value and diagnostic consequence can favor different evidence, supporting consequence-aware question supervision beyond purely information-seeking acquisition.
Abstract
from arXiv · showhide
Proactive medical dialogue requires an agent to decide what to ask from incomplete patient information. Existing information-seeking approaches commonly prioritize questions that most reduce diagnostic uncertainty. While effective for acquiring informative evidence, this criterion overlooks an important property of medical diagnosis: different diagnostic errors can carry substantially different consequences. Missing a severe condition may matter more than reducing uncertainty among less consequential alternatives. Question acquisition should therefore consider not only how informative new evidence is, but also how it is expected to affect the downstream diagnostic decision. To this end, we propose Expected-Severity-Risk (ESR), a consequence-aware question-supervision objective that values each candidate by its expected reduction in severity-aware terminal risk. Because questions must be selected before their answers are observed, ESR marginalizes over possible answers using train-only population statistics. Its rankings are then distilled into a prefix-only language policy, so next-question selection requires no teacher-side computation at deployment. Across three Qwen3-4B training seeds on DDxPlus, matched ESR supervision reduces mean high-severity diagnostic miss from .0645 to .0455 (-29.5%) and improves mean diagnostic accuracy from .9123 to .9320 while requiring only 0.14 additional questions per dialogue. Fixed-budget analyses show that the two objectives remain behaviorally distinct when question count is controlled, while a matched expected-0/1-risk control shows that severity-aware weighting improves the high-severity error profile beyond generic decision-aware supervision. These results support moving proactive medical dialogue beyond uncertainty reduction toward consequence-aware evidence acquisition.
1. INTRODUCTION
Proactive medical dialogue must choose questions before answers are observed, yet uncertainty reduction alone may overlook the unequal consequences of diagnostic errors. ESR addresses this gap by ranking questions through expected severity-aware risk reduction and distilling those rankings into a deployable policy.
- Motivation: Information gain can favor separating common alternatives over identifying less likely but more severe conditions.Diagnostic consequence therefore differs from posterior uncertainty reduction.
- Method: ESR ranks candidate questions by expected reduction in severity-aware terminal risk before their answers are observed.The objective marginalizes over possible answers at selection time.
- Method: The rankings are distilled into a prefix-only language policy that requires no teacher-side computation at deployment.This converts selection-time supervision into direct next-question prediction.
- Results: −29.5%: ESR reduces mean high-severity diagnostic miss from .0645 to .0455 across three matched Qwen3-4B seeds on DDxPlus.Mean diagnostic accuracy also rises from .9123 to .9320 with only 0.14 additional questions per dialogue.
- Results: Fixed-budget analyses show that information-seeking and decision-aware objectives remain behaviorally distinct when question count is controlled.A matched expected-0/1-risk control isolates severity-aware weighting from generic decision-aware supervision.
2. DECISION-AWARE QUESTION SUPERVISION
The method treats question acquisition as sequential evidence collection under partial observation, connecting candidate questions to downstream diagnostic consequences. ESR estimates the expected severity-aware risk reduction of each question from possible answer-induced states, then distills the rankings into a prefix-only policy.
- 2.1. Problem Formulation: At turn t, the policy selects an unanswered question from partial dialogue state before observing its answer.The answer updates the dialogue state after selection.
- 2.1. Problem Formulation: A fixed diagnostic model maps each partial state to a disease posterior and serves as the downstream decision maker.Question value is defined relative to the decision induced by the current evidence.
- 2.2. Expected-Severity-Risk Question Supervision: ESR first assigns consequence-aware terminal risk to a partial state, then evaluates each question over next states induced by its unobserved answer.This connects question acquisition to the eventual diagnostic decision.
- 2.2. Expected-Severity-Risk Question Supervision: R01 treats all diagnostic errors equally, whereas Rsev assigns greater consequence to errors involving more severe conditions.Terminal risk therefore characterizes downstream decision consequence rather than posterior concentration alone.
- 2.2. Expected-Severity-Risk Question Supervision: The predictive answer distribution combines a disease-conditioned answer model estimated from training data with the current disease posterior.It weights hypothetical next states using information available before the actual answer is observed.
- 2.2. Expected-Severity-Risk Question Supervision: A high ESR score indicates that asking a question is expected to move the downstream diagnostic decision toward lower severity-aware risk.The score averages terminal risk across possible next states.
- 2.2. Expected-Severity-Risk Question Supervision: E-ENTROPY retains answer marginalization but uses posterior entropy, while EXP.-0/1-RISK uses uniform terminal risk to isolate severity weighting.The comparison separates information seeking from decision awareness and asymmetric severity weighting.
- 2.2. Expected-Severity-Risk Question Supervision: The teacher rankings become next-question labels for a prefix-only student that selects directly from observable context at inference.The student lacks access to the diagnostic posterior, answer model, hidden answers, and teacher utilities.
3. EXPERIMENTS
The experiments evaluate ESR under matched training and interaction conditions, then test whether its behavior differs from entropy-based acquisition and generic expected 0/1-risk supervision. Results show improved high-severity outcomes, distinct evidence preferences under fixed budgets, and a remaining benefit from severity-aware weighting.
- Experimental Setup: The DDxPlus environment uses nine conditions, fixed partial-state features, and a 50-question inventory evaluated with a frozen multinomial logistic classifier.The subset contains 5,000 training, 800 validation, and 1,000 held-out test patients.
- Experimental Setup: Students share Qwen3-4B initialization, training states, optimization settings, and candidate-constrained decoding; only teacher-derived next-question targets differ.The main comparison is repeated across seeds 42, 43, and 44.
- Main Results: Across three matched seeds, ESR reduces mean HSM from .0645 ± .0000 to .0455 ± .0048 (−29.5%) and increases mean accuracy from .9123±.0015 to .9320±.0017.The change requires 0.140±.015 additional questions on average; secondary severity-weighted, population high-severity, and proxy-cost measures also decrease.
- Information and Decision Value: On 2,000 identical partial states and candidate sets, E-ENTROPY yields greater entropy reduction (.2316 versus .1953), whereas ESR yields greater severity-risk reduction (.0902 versus .0810).In 11.4% of states, the objectives explicitly select different questions according to these competing values.
- Controlling for Dialogue Length: Under fixed question budgets, ESR is not uniformly better: entropy is competitive at K = 1, while ESR has lower HSM at K = 10 and K = 15.At K = 15, accuracy is effectively matched (.946 versus .947), while HSM is .0390 for ESR versus .0450 for E-ENTROPY.
- Attributing the Effect to Severity Weighting: Against matched EXP.-0/1-RISK students, ESR increases accuracy from .913 to .931 and reduces HSM from .0690 to .0435 while also decreasing question count from 2.509 to 2.294.Both objectives use the same answer marginalization and training setup; only teacher valuation changes from expected 0/1 risk to severity-aware risk.
4. CONCLUSION
ESR values medical questions by their expected reduction in severity-aware diagnostic risk. On DDxPlus, it reduced high-severity misses while improving accuracy with only a small increase in question count, and analyses linked the improved profile to asymmetric severity weighting.
- 29.5% lower mean high-severity diagnostic miss was achieved across three matched Qwen3-4B seeds on DDxPlus.ESR reduced mean high-severity diagnostic miss by 29.5%.
- ESR improved accuracy with only a small increase in question count.The conclusion describes the question-count increase as small.
- Same-state analysis showed that information value and decision value can prioritize different evidence.This indicates that the two acquisition objectives can behave differently even under matched state conditions.
- A matched distilled 0/1-risk control identified asymmetric severity weighting as a key contributor to the improved high-severity profile.
- The results establish consequence-aware question supervision as a practical approach to aligning proactive evidence acquisition with downstream diagnostic consequence.
5. COMPLIANCE WITH ETHICAL STANDARDS
The study used the publicly available synthetic DDxPlus benchmark and involved no human participants or identifiable patient data. Consequently, no ethical approval was required.
- The study used the publicly available synthetic DDxPlus benchmark.
- No human participants or identifiable patient data were involved.
- No ethical approval was required.