Fine-grained Alignment of Large Language Models for General Medication Recommendation without Overprescription

Introduction

• Reliable, general medication recommendation is presented as important for clinical decision-making and equitable access, particularly in resource-constrained regions.The motivation cites South Asia and Africa as examples.Source: Introduction · p. 2
• Existing systems rely on ID-based medication encoding and structured EHR data, limiting adaptation to expanding pharmacopoeias and capture of nuanced patient profiles.Nearly 80% of clinical information in EHR systems resides in unstructured notes without standardized schemas.Source: Introduction · p. 2
• GPT-4 recommends over 80 medications per patient on average—over three times the volume of practicing physicians—while general and medical LLMs underperform.The paper warns that uncalibrated deployment risks higher healthcare expenditures, adverse drug events, and antimicrobial resistance.Source: Introduction · p. 2
• LAMO tailors LLMs with medication-group LoRA experts and outperforms state-of-the-art methods in internal, temporal, and external validation.The framework groups medicines to share LoRAs and is reported to retain accuracy on medications absent from training.Source: Introduction · p. 3

Structured EHR Representation for LLMs

• Discharge summaries are parsed into four components—present illness, past medical history, allergies, and admission medications—to represent patient condition, comorbidities, medication history, and sensitivities.For prior visits, historical diagnoses, procedures, demographics, and previously prescribed medications are included; full prior notes are omitted and discharge medications are excluded to prevent label leakage.Source: Structured EHR Representation for LLMs · p. 3

LLM Fine-Tuning for Medication Recommendation

• LLaMA-2-7B is instruction-tuned using structured clinical context and a candidate medication, with a binary output indicating whether that medication was prescribed.The task formulation is intended to support context-sensitive medication decisions.Source: LLM Fine-Tuning for Medication… · pp. 3–4
• LoRA adapters are trained for distinct medication groups and activated together at inference to produce the recommendation list with limited computational overhead.LoRA adds trainable low-rank matrices to frozen Transformer layers.Source: LLM Fine-Tuning for Medication… · p. 4

Unsatisfactory Accuracy and Overprescription

• On MIMIC-III, GPT-4 achieved the highest accuracy among eight evaluated LLMs, with an F1 score of 0.3542, significantly below traditional SOTA methods.LLaMA-2 and LLaMA-3 were particularly inclined to recommend nearly all candidate medications; ChatGLM3 frequently generated none, resulting in extremely low accuracy.Source: Results · p. 4

Insufficient Medication Knowledge

• Most tested LLMs score below 80% on PrimeKG disease–medicine relation recognition, indicating limitations in nuanced medical relationships.The paper presents this test as a probe into the reason for unsatisfactory recommendation performance.Source: Results · p. 4

Effectiveness of LAMO Fine-tuning

• LAMO outperforms all baselines on MIMIC-III, leads the strongest baseline RAREMed on MIMIC-IV despite an overall temporal performance decrease, and significantly outperforms other models on eICU.The MIMIC-IV evaluation reflects a shift from ICD-9 to ICD-10 coding; the eICU dataset spans diverse hospitals, populations, and clinical practices.Source: Results · p. 5

Generalization to New Medications

• LAMO shows close recommendation accuracy between source and target medications within the same group in zero-shot transfer tests, including target medications absent from training.The authors interpret this as transferability across similar medications.Source: Results · p. 5

In-depth Analysis of LAMO

• Concise titles outperform tested title variants, removing any input factor drops performance, and five- or ten-LoRA variants remain competitive with traditional methods.The analysis also reports performance loss as the number of LoRAs decreases; input-factor variants remove history, note, code, disease, or procedure.Source: Results · p. 5

Discussion

• LAMO clusters medications and trains dedicated LoRA adapters, outperforming baselines in F1 score and generalizing across temporal shifts and datasets.The discussion frames these findings as supporting clinical applicability.Source: Discussion · p. 6
• The authors suggest LAMO could help minimize adverse drug events and support non-specialists in low-resource regions with timely treatment.They also describe possible extension to neglected diseases, including Chagas and leishmaniasis.Source: Discussion · p. 6
• Supervised fine-tuning may scale inefficiently for large drug vocabularies, and drug-pair memorization limits reasoning about synergistic effects and safety constraints such as adverse interactions.The authors propose pharmacological representations and list-wise alignment as future directions.Source: Discussion · p. 6

Figure 1

• Figure 1 gives an overview of LAMO’s data processing and fine-tuning process.Source: Figure 1

Figure 2

• Figure 2 evaluates LLM medication recommendation, including F1 scores, recommendation counts, knowledge-test accuracy, baseline comparisons, and temporal and external validation.Source: Figure 2

Figure 3

• Figure 3 analyzes recommendation precision across disease and procedure representations, input-factor ablations, medication types, and LoRA-adapter counts.Source: Figure 3

• The prompt begins with a task description and a previous-hospitalization section.Source: PROMPT · p. 10
• The previous-hospitalization section lists demographic, diagnosis, procedure, and medication fields.Source: PROMPT · p. 10
• The current-condition section includes diagnosis, procedure, HPI, PMH, allergies, and medications on admission.Source: PROMPT · p. 10
• HPI, PMH, and medications on admission are separately identified in the prompt.Source: PROMPT · p. 10

• Table S1 presents a reformatted raw clinical-note example, whose bold subheadings can vary across patients.Source: Table S1 · p. 13
• Table S2 provides a prompt template for extracting key factors from discharge summaries.Source: Table S2 · p. 14
• Table S3 distinguishes prior-visit history from current-visit input; age and gender are omitted as separate input terms because HPI contains them.Source: Table S3 · p. 15
• Table S4 describes refined prompts that consolidate multiple extracted components when long notes exceed GPT-3.5’s input capacity and are segmented.Source: Table S4 · p. 16
• Tables S5–S10 contain medication-recommendation evaluation prompt templates for the listed LLMs.Source: Table S5 · p. 17, Table S6 · p. 17, Table S7 · p. 18
• Tables S11–S14 give medical-knowledge answering prompt templates for several model groups, with drug and disease placeholders; Tables S15–S16 report GPT-4 and GPT-3.5 medical-knowledge test results across four output-space settings.Source: Table S11 · p. 20, Table S12 · p. 20, Table S13 · p. 20
• Table S23 provides brief descriptions of the compared methods.Source: Table S23 · p. 25

• Table S24 is identified as reporting medication-recommendation accuracy across various models.Source: Table S24 · p. 26
• Higher F1, Jaccard, Recall, and Precision indicate better recommendation accuracy; predicted medication counts are compared with the ground-truth count of 22.93.Source: arXiv preprint arXiv:2304.14454 (2023)… · p. 26
• The out-of-distribution figure reports target-medication performance as a percentage of zero-shot transfer performance relative to original performance.Source: Figure S1 · p. 27
• MIMIC-III data were randomly split 4:1:1, while MIMIC-IV was divided into four temporal cohorts for validation.Source: Table 25 · p. 28