Source-linked AI summary
ClinAlign: Scaling Healthcare Alignment from Clinician Preference
Shiwei Lyu, Xidong Wang, Lei Liu, Hao Zhu, Chaohe Zhang, Jian Wang, Jinjie Gu, Benyou Wang, Yue Shen
TL;DR
Fine-grained clinical alignment remains difficult because coarse objectives and weakly grounded judges do not capture clinician preferences. The paper builds physician-verified HealthRubrics and distills them into reusable HealthPrinciples for scalable training and inference-time self-revision. The framework provides a resource-efficient foundation that enables models to surpass frontier proprietary systems on specialized clinical tasks.
Problem
Aligning open-ended medical responses with fine-grained clinician preferences and professional standards remains difficult because existing objectives and supervision are insufficiently clinically grounded.
Method
The framework collects 7,034 physician-verified rubric examples, distills 119 reusable HealthPrinciples, and applies them to offline supervision and inference-time self-revision.
Results
The framework enables models to surpass substantially larger commercial and open-source systems on HealthBench-Hard and Arena-Hard-v2.
Takeaways & Limitations
Structured clinical logic provides a practical, resource-efficient foundation for specialized healthcare alignment without increasing model size.
Takeaways & Limitations
The method does not substantially improve intrinsic reasoning, with multi-step reasoning gains not consistently significant.
Abstract
from arXiv · showhide
Although large language models (LLMs) demonstrate expert-level medical knowledge, aligning their open-ended outputs with fine-grained clinician preferences remains challenging. Existing methods often rely on coarse objectives or unreliable automated judges that are weakly grounded in professional guidelines. We propose a two-stage framework to address this gap. First, we introduce HealthRubrics, a dataset of 7,034 physician-verified preference examples in which clinicians refine LLM-drafted rubrics to meet rigorous medical standards. Second, we distill these rubrics into HealthPrinciples: 119 broadly reusable, clinically grounded principles organized by clinical dimensions, enabling scalable supervision beyond manual annotation. We use HealthPrinciples for (1) offline alignment by synthesizing rubrics for unlabeled queries and (2) an inference-time tool for guided self-revision. A 30B-A3B model trained with our framework achieves 33.4% on HealthBench-Hard, outperforming much larger models including Deepseek-R1 and o3, establishing a resource-efficient baseline for clinical alignment.
1 Introduction
The paper targets fine-grained clinical alignment, where coarse objectives and scarce clinician rubric data limit reliable supervision. It introduces physician-verified HealthRubrics and reusable HealthPrinciples to scale alignment across scenarios.
- Healthcare alignment requires context-sensitive clinical responses because desired behavior varies with urgency, uncertainty, and user expertise.
- Existing scenario-grounded methods lack scalable access to clinician rubric expertise because physician rubric data are scarce and costly to collect.
- 7,034 physician-verified supervision examples raise Qwen3-4B-Instruct’s HealthBench Hard score from 5.2% to 22.9%, surpassing GPT-5.1-Instant at 20.8%.
- HealthPrinciples distill recurring rubric patterns into 119 reusable, scenario-specific principles organized by urgency, uncertainty, user expertise, and task type.
- Principles add 16,872 rubric-supervised examples and support both scalable offline training and inference-time self-revision.
2 Related Work
Related work is moving medical evaluation and alignment beyond knowledge tests toward scenario-grounded, rubric-based supervision. ClinAlign emphasizes physician-supervised rubrics and reusable principles for offline training and test-time guidance.
- Medical LLM research has expanded from knowledge-intensive evaluations to clinical assistants that integrate structured resources for tasks such as diagnosis and radiology reporting.
- Scenario-grounded benchmarks address consultation behaviors that multiple-choice tests miss, including long-form reasoning, communication quality, and safety compliance.
- Rubric-based evaluation decomposes complex goals into verifiable criteria, but existing approaches face objective conflicts and reward hacking from poorly constrained rubrics.
- ClinAlign differs by combining physician-supervised rubrics with reusable consensus principles that support offline training and rubric referencing at test time.
3 Pilot Study: Generalization Failure of Naive SFT
The pilot tests whether naive supervised fine-tuning can generalize fine-grained healthcare alignment from rubric-aware responses. Training improves quickly, but held-out performance does not improve reliably, motivating more generalizable supervision.
- The pilot evaluates healthcare alignment as context-aware matching of clinical responses to professional standards and clinician preferences across medical scenarios.
- The study splits HealthBench into 3,000 training and 2,000 held-out questions and generates 9,000 rubric-aware training instances for Qwen3-4B-Instruct.
- Training performance rises from 41.3% to about 52% and largely plateaus after the first epoch.
- Held-out performance fluctuates without sustained improvement, indicating that naive SFT does not generalize reliably to unseen questions.
- The generalization gap motivates rubric RL with instance-conditioned, scenario-aware criteria for finer-grained rewards.
4 Methodology
The framework combines physician-revised rubrics from real medical queries with reusable HealthPrinciples for scalable offline supervision and inference-time self-revision.
- HealthRubrics: HealthRubrics produces physician-revised rubrics from real-world medical queries and candidate responses for preference supervision.The workflow begins with medical prompts paired with multiple model answers and uses physician review to ground the resulting rubrics clinically.
- HealthRubrics: Physicians independently relabel response pairs and finalize draft rubrics through revision, audit, and agreement-based review loops.Each response pair receives three physician annotations, while rubric acceptance requires agreement between the revising and auditing physicians.
- HealthRubrics: The physician cohort includes 111 reviewers across specialties and seniority levels, with the annotation process requiring 632.2 person-hours.The reported total cost was $15,172.80 at $24 per hour.
- HealthPrinciples: HealthPrinciples distills recurring rubric patterns into reusable, scenario-specific guidance organized by urgency, uncertainty, user expertise, and task type.The taxonomy includes 21 physician-consolidated task types and yields 119 principles.
- HealthPrinciples: Principle-conditioned rubric generation assigns scenario labels, retrieves matching principles, and converts them into scorable rubrics for 16,872 additional medical questions.The resulting principles support scalable supervision for unseen queries.
- Inference-time guidance: The workflow is also packaged as an inference-time tool that classifies scenarios, extracts matched principles, generates rubric items, and guides self-revision.It can use a question, dialogue context, or draft answer as input.
5 Experiments
Experiments evaluate rubric-supervised training and inference-time guidance across healthcare usefulness, medical evaluation, and open-ended instruction-following benchmarks.
- Experimental setup: The evaluation uses HealthBench, LLMEval-Med, and Arena-Hard-v2 to measure clinical usefulness, medical capabilities, safety, and open-ended performance.HealthBench contains 5,000 multi-turn healthcare conversations, LLMEval-Med contains 2,996 questions, and Arena-Hard-v2 contains 500 challenging prompts.
- Experimental setup: The study compares Draft Rubrics, Doctor Rubrics, Principle Rubrics, and More Query Rubrics alongside open, proprietary, and medical-domain baseline models.The rubric variants differ in whether rubrics are automatically drafted, physician revised, principle derived, or scaled to additional medical queries.
- Main results: Doctor Rubrics improve over Draft Rubrics, with the largest gains on HealthBench Hard and Overall settings and on Arena-Hard-v2.Physician edits remove ambiguity and correct mis-specified criteria, producing higher-fidelity rewards and stronger policy updates.
- Main results: Rubric-trained models transfer gains to Arena-Hard-v2, improving instruction following, actionable guidance, and communication of limitations.LLMEval-Med Reasoning changes little, indicating stronger effects on helpfulness and safety than on problem-solving depth.
- Inference-time scaling: Multiple inference-time calls to the rubric-guidance tool consistently improve HealthBench-hard performance across backbones without additional training.Performance rises with more iterations but gradually saturates after a few calls.
6 Analysis
Scaling analyses show that broader question coverage improves rubric-based reinforcement learning more reliably than repeatedly training on a narrow prompt set.
- Question scaling: Figure 9 evaluates 1k, 2.5k, 5k, 10k, and 20k questions under a fixed training-FLOPs budget.Smaller datasets receive proportionally more epochs so each setting matches training on 20k questions for two epochs.
- Question scaling: Performance improves monotonically as the number of distinct training questions increases, with the largest gains from 1k to 5k and diminishing returns at larger scales.Positive gains continue through 20k questions.
- Question scaling: The analysis indicates that rubric-based reinforcement learning benefits more from supervision diversity than from additional epochs on a narrow prompt set.Broader coverage spans more clinical intents, risk profiles, and failure modes.
- Scoring model choice: Qwen3 rubric-scorer accuracy increases from 76.4% for Qwen3-4B to 87.9% for Qwen3-235B against GPT-4.1 reference judgments.The gain from Qwen3-32B at 87.6% to Qwen3-235B at 87.9% is negligible.
7 Conclusion
The paper presents physician-verified rubrics, reusable clinical principles, and inference-time guidance as a resource-efficient framework for fine-grained medical alignment.
- Conclusion: The framework combines validated data, principle-synthesized rubrics, and inference-time guidance to align medical language models with fine-grained clinical standards.The authors report that these strategies enable models to surpass frontier proprietary systems.
- Conclusion: The authors conclude that structured clinical logic can be as effective as scaling model parameters for specialized tasks.They release the data, principles, and tool as a practical foundation for further healthcare AI research.
Limitations
The method improves response quality without substantially improving intrinsic reasoning, and its inference-time benefits saturate as tool use or sampling increases.
- The method does not substantially improve intrinsic reasoning, with multi-step reasoning gains not consistently significant.The authors attribute improved responses primarily to more effective use of external signals rather than fundamentally stronger internal reasoning.
- Inference-time scaling benefits saturate quickly, so additional tool usage or sampling produces diminishing returns.The authors suggest more agentic planning and tool coordination as a direction for more efficient inference-time computation.
A SFT Data Generation Prompt
The prompt expands each question into rubric-aware targets while using medical-query classification to select clinically relevant conversations for HealthRubrics.
- A SFT Data Generation Prompt: Each training question is paired with the original question, official HealthBench rubrics, a current model draft, and a clinician-written ideal completion.The draft and ideal provide complementary anchors for failure modes and desired coverage, while rubrics define the answer's scoring contract.
- A SFT Data Generation Prompt: Multiple rubric-aware targets reduce brittleness from imitating a single reference completion.
- A SFT Data Generation Prompt: The resulting prompt design combines clinician standards with observed model failures and desired answer coverage.
- B Medical Query Classification for HealthRubrics: GPT-5.1 classifies pooled conversations as medical when responsible responses require clinical or biomedical knowledge.The guideline includes diagnosis, treatment, medication use, prognosis, safety risks, or interpretation of medical information.
- B Medical Query Classification for HealthRubrics: Administrative or loosely health-related conversations without clinical reasoning are labeled non-medical.
- B Medical Query Classification for HealthRubrics: The classification step extracts a high-precision medical subset from pooled preference corpora.
C Subcategory Definitions and Task Type Induction
The HealthPrinciples taxonomy organizes clinical guidance by urgency, information sufficiency, uncertainty, and user expertise, with distinct response implications for each category.
- Urgency: Urgency is divided into non-emergent, conditionally emergent, and emergent cases.Conditionally emergent cases require targeted clarification and conditional escalation guidance, while emergent cases warrant prompt protective action.
- Information sufficiency: Sufficient information means the key details needed for safe and effective guidance are present.
- Uncertainty: Uncertainty is reducible when follow-up questions can resolve missing details and irreducible when examination, measurement, testing, or in-person evaluation is required.
- User expertise: User expertise distinguishes non-professionals, who need plain and actionable language, from professionals, who may receive technical language while safety is maintained.
C.4 Task Type Inventory
The paper uses a physician-informed 21-category task inventory, induced from query summaries and clustering, to route rubric sets and derive reusable principles for varied clinical tasks.
- Task taxonomy: Each query receives a primary task and optional secondary task using readable labels aligned with the taxonomy.
- Task type induction: Task types are induced by summarizing queries, proposing clusterings at multiple granularities, and having physicians reconcile boundaries and select the inventory.
- Task type induction: Physicians added Procedure and perioperative guidance because it was under-represented in early model clusterings but frequent in patient questions and feedback.
- Final task families and definitions: The final inventory contains 21 task families, including emergency triage, symptom assessment, self-care, medication safety, and non-medical queries.
- Final task families and definitions: Emergency triage emphasizes red flags, explicit timelines, and conservative in-person evaluation when key information is missing.
- Final task families and definitions: Symptom assessment outlines plausible etiologies and missing information while avoiding definitive diagnoses.
- Final task families and definitions: Self-care and follow-up guidance defines actionable steps, measurable targets, reassessment windows, and escalation triggers.
- Principle extraction: HealthPrinciples are derived by routing rubric sets into subcategories, then iteratively clustering and compressing them for physician refinement.