A Data-Centric Approach To Generate Faithful and High Quality Patient Summaries with Large Language Models

• Patients frequently misunderstand hospitalization events and follow-up actions, while producing high-quality patient summaries is difficult and time-consuming for healthcare workers.Source: §1 · p. 1
• LLMs can summarize medical text but may generate unsupported or erroneous facts, especially when training data does not match the information available at care time.Source: §1 · pp. 1–2
• The authors create 100 hallucination-free and 100 improved training examples and evaluate GPT-4 for automatic hallucination detection.Source: §1 · p. 2
• The study introduces a rigorous token-level hallucination annotation protocol for medical texts.Source: §1 · p. 2
• Two expert-labeled datasets contain hallucinations in 100 doctor-written and 100 generated patient summaries, requiring 70 hours per annotator.Source: §1 · p. 2
• Carefully removing hallucinations from training data reduces unsupported content while preserving key information in LLM-generated patient summaries.The study also evaluates improved training examples and GPT-4 hallucination detection.Source: §1 · p. 2

• Clinical summarization approaches span zero- and few-shot prompting and fine-tuning, with GPT-4 sometimes preferred over human summaries for correctness and completeness.Source: §2 · p. 3
• Hallucination annotations in clinical summarization have often relied on proprietary data, coarse categories, or unreleased labels, motivating this work’s public release.Source: §2 · p. 3
• Automated summarization metrics often fail to correlate with manually assessed faithfulness and quality.Source: §2 · p. 3
• Existing hallucination mitigation strategies detect errors after generation, alter the generation model, or improve training data.Source: §2 · p. 3
• For sample-efficient LLMs, the authors argue that revising a small number of training examples manually can be feasible without large-scale data editing.Source: §2 · p. 3

3.1. MIMIC Discharge Instructions Datasets

• 3.1. MIMIC Discharge Instructions Datasets: 331,793 clinical notes were filtered into 100,175 cleaned examples, with a 26,178-entry subset used for human annotation.The authors selected Brief Hospital Course contexts and Discharge Instructions targets, and also released a longer-context version.Source: §3.1 · p. 3
• The dataset overview covers all publicly available datasets used in the work.Source: Table 1 · p. 4

3.2. Hallucination Datasets

• 3.2. Hallucination Datasets: Two medical experts labeled hallucinations in 100 randomly selected doctor-written summaries and 100 summaries generated from 20 held-out contexts by five models.Source: §3.2 · p. 4

3.3. Derived Datasets

• 3.3. Derived Datasets: The Cleaned dataset manually replaces or removes annotated hallucinations from the Original doctor-written summaries.Source: §3.3 · p. 4
• 3.3. Derived Datasets: Cleaned & Improved further corrects mistakes and artifacts for model fine-tuning and prompting in qualitative evaluation.Source: §3.3 · p. 4

4.1. Generation of Patient Summaries

• Patient summaries are generated from Brief Hospital Course contexts, with an additional translation step into layperson language.Contexts average 552.0 words and summaries 113.2 words.Source: §4.1 · p. 4
• The study uses Llama 2, GPT-4, and LED, with 100-example fine-tuning for Llama, five-shot or zero-shot prompting for GPT-4, and large-scale fine-tuning for LED.LED uses 80% of the available training data, whereas Llama and GPT-4 use sample-efficient settings.Source: §4.1 · pp. 4–5

4.2. Protocol for Labeling Hallucinations

• The annotation protocol labels token-level medical errors as unsupported, contradicted, or incorrect facts, with nine subcategories for unsupported facts.The Brief Hospital Course is treated as the only patient-specific ground truth, while general medical advice remains permissible.Source: §4.2 · p. 5

4.3. Hallucinations in MIMIC and Generated Patient Summaries

• Two trained medical students independently annotated 100 doctor-written and 100 generated summaries, resolving disagreements through discussion and recording agreement statistics.Training used twelve examples and final annotation used MedTator.Source: §4.3 · p. 5

4.4. Data-Centric Hallucination Reduction

• The hallucination-reduction experiment compares Llama 70B fine-tuned on 100 Original or Cleaned examples with GPT-4 five-shot and zero-shot prompting across 20 randomly selected contexts.The resulting 100 generated summaries were annotated for hallucinations, missing key facts, and medical jargon.Source: §4.4 · pp. 5–6, Table 2 · p. 7

4.5. Quantitative Evaluation

• Quantitative evaluation uses held-out MIMIC context-summary pairs and reports ROUGE, BERTScore, DeBERT, SARI, and generated-word counts.The evaluation includes 80,140 training examples for LED, 100 for Llama, and five for GPT-4 five-shot.Source: §4.5 · p. 6

4.6. Qualitative Evaluation

• Qualitative evaluation rates relevance, consistency, fluency, coherence, and simplification on five-point scales, while automatic detection frames hallucinations as context-conditioned span classification.Detection compares entity-based baselines with GPT-4 using class-aware or class-agnostic prompts and partial matching.Source: §4.6 · p. 7, §4.7 · pp. 7–8

5.1. Hallucinations in MIMIC and Generated Patient Summaries

• 286 hallucinations were found in 100 doctor-written MIMIC summaries, compared with 114 in 100 generated summaries.Unsupported word was the most common category in both datasets; condition unsupported and time unsupported were also prevalent in MIMIC.Source: §5.1 · p. 8

5.3. Quantitative Evaluation

• LED performed best overall on quantitative patient-summary evaluation, followed by Llama and GPT-4, while GPT-4 zero-shot achieved considerable performance without training data.Llama and GPT-4 showed similar performance only for BERTScore and DeBERT.Source: Table 4 · p. 9, §5.3 · p. 9

5.2. Data-Centric Hallucination Reduction

• 2.60 to 1.55 hallucinations per summary is the reduction for Llama 70B trained on Cleaned rather than Original data, while key-fact coverage remains unchanged.GPT-4 five-shot changes from 0.70 to 0.40 hallucinations per summary, a marginal reduction, while GPT-4 zero-shot produces 0.45.Source: §5.2 · p. 9

5.4. Qualitative Evaluation

• GPT-4 performed significantly better across all five qualitative dimensions, with median ratings of four for relevance and five for simplification, fluency, and coherence.GPT-4 five-shot performed better on consistency, whereas zero-shot performed better on relevance and simplification.Source: §5.4 · p. 9, Figure 4 · p. 10

5.5. Automatic Hallucination Detection

• GPT-4 clearly outperformed MedCat entity-based baselines for automatic hallucination detection, whose F1 scores were 10.2 for MIMIC and 5.8 for generated summaries.GPT-4 nevertheless showed low recall for class-aware detection, indicating remaining detection limitations.Source: Table 5 · p. 10, §5.5 · p. 10

Annotation complexity

• Hallucination annotations took 12–18 minutes per summary, and only 59% of generated-summary annotations were identified by both annotators.Among jointly identified annotations, 56% received the same type; 39% of all hallucinations were unsupported words.Source: §6 · p. 10

Automatic detection

• MedCAT-based UMLS concepts combined with embeddings performed very poorly because many hallucination spans did not align with medical concepts.GPT-4 performed significantly better, but the authors characterize this as an initial step requiring thorough human evaluation.Source: §6 · p. 10

Data-centric hallucination reduction

• Using 100 human-curated examples, fine-tuning made Llama 2 more susceptible to data-centric hallucination reduction, whereas GPT-4 aligned with five examples was less susceptible.Source: §6 · p. 11

Evaluation

• Quantitative metrics did not correlate well with hallucination rates or quality: LED-large outperformed Llama 2 and GPT-4 quantitatively, but quality rankings reversed.LED-large most faithfully reproduced the qualitatively flawed MIMIC summaries, while Llama 2 showed moderate alignment and GPT-4 retained more of its pretrained style.Source: §6 · p. 11

Clinical relevance

• Clinical evidence for the usefulness of automatically generated patient after-visit summaries remains weak, motivating further evidence before deployment.The study notes that GPT-4’s deviation from the original style could potentially support more effective interventions, but more clinical evidence is necessary.Source: §6 · p. 11

Limitations

• Generalizability is limited by one medical-notes dataset, reduced BHC context, two annotators, expert rather than patient ratings, and only two evaluated LLMs.The reduced context likely overestimated hallucinations in doctor-written summaries.Source: §6 · p. 11

Conclusion

• Domain-expert interventions reduced hallucinations and improved generation quality, while LLMs remained sensitive to fine-tuning and prompting examples.GPT-4 showed few hallucinations and omitted key facts while receiving high-quality ratings.Source: §7 · p. 11

Resources

• The work releases a protocol for labeling unsupported evidence and two datasets containing 100 doctor-written and 100 LLM-generated patient summaries.The datasets are intended to support development and assessment of automatic hallucination metrics.Source: §7 · p. 11

Future work

• Future work should examine other summary formats and LLMs while collecting more clinical evidence about interventions that improve patient health literacy.Source: §7 · p. 11

A.1. Methods

• The pipeline prioritized higher precision by removing low-quality summaries rather than retaining all summaries.An author with medical expertise repeatedly compared sampled included and excluded summaries against the preprocessing goals.Source: §A.1 · p. 19
• Two iterative methods guided pipeline refinement: expert review of 100 included and 100 excluded summaries and t-SNE analysis of BERT embeddings.The embedding analysis was used to identify systematic patterns in the summaries.Source: §A.1 · p. 19
• Static patterns and regular expressions performed the final filtering because automatic approaches did not provide sufficient precision.The preprocessing code was made available on GitHub.Source: §A.1 · p. 19

A.2. Results

• The pipeline removed summaries shorter than 350 characters, static prefixes, headings, deidentification patterns, suffixes, templates, and other low-quality content.Discharge Instructions were used to split summaries from the remaining hospital course, and the BHC supplied the context.Source: §A.2 · p. 19
• Table 6 compares datasets using full context versus BHC context, with discharge instructions serving as patient summaries in both.Source: Table 6 · p. 19
• 100,175 context-summary pairs remained after preprocessing the discharge instructions and brief hospital-course contexts.The pipeline began with 296,697 candidate summaries and later retained 119,260 notes before additional quality filtering.Source: §A.2 · p. 19
• Preprocessing significantly decreased LED-large’s discharge-instruction prediction performance, suggesting that easier-to-predict instances were removed.Source: §A.2 · p. 19

A.3. Analyzing 100 Included and Excluded Patient Summaries

• The pipeline was designed to obtain diverse, free-text discharge instructions as patient summaries from MIMIC-IV-Note.Figure 5 presents the preprocessing steps used to create MIMIC-IV-Note-Ext-DI(-BHC).Source: Figure 5 · p. 20
• 33 of 100 retained examples had slight preprocessing problems, while nine lost important content and five of 100 removed examples were problematic.The retained-example issues included incomplete suffix removal, residual greetings or templates, and removal of nonessential follow-up, medication, or procedure content; problematic removed examples were pruned too early.Source: §A.3 · pp. 19–20
• The filtering was still considered useful because it removed short summaries and produced longer summaries with more content.Nine notes lacked a sufficiently long brief hospital course, and 12 useful summaries were 300–350 characters but were removed under the filtering criteria.Source: §A.3 · p. 20
• The LED-large model’s performance decreased after preprocessing, suggesting that easier-to-predict instances were removed.Table 7 reports evaluation on 10,000 test examples before and after preprocessing.Source: Table 7 · p. 21

A.4. Analyzing Embeddings

• After preprocessing, t-SNE embeddings became more homogeneous and medical services were harder to distinguish, unlike the template-associated clusters in unprocessed summaries.The authors used the observed service-specific clusters, which often reflected static templates or copied content, to add filtering steps.Source: §A.4 · p. 21, Figure 6 · p. 21

Appendix B. Annotations Statistics

• Two annotators independently labeled hallucinations before discussing disagreements and agreeing on final annotations.The protocol treated manual reconciliation as more reliable than automatic agreement because the same hallucination could receive different span labels.Source: Appendix B · pp. 21, 23
• Interval Krippendorff’s alpha was 0.629 for 100 doctor-written summaries and 0.826 for generated summaries when counting hallucinations per summary.The higher agreement for generated summaries indicated that annotating them was easier.Source: Appendix B · p. 21
• Span-overlap F1 scores were 0.479 and 0.440 without class labels, versus 0.245 and 0.271 when distinguishing classes, for doctor-written and generated summaries respectively.These results underscored the difficulty of hallucination labeling.Source: Appendix B · p. 22
• Overall qualitative-evaluation agreement was 0.586, with the highest agreement for consistency at 0.778.Agreement was calculated separately for each Likert-rated qualitative dimension and across all ratings.Source: Appendix B · p. 22, Table 10 · p. 23
• Agreement was lower for fluency at 0.431 and coherence at 0.218, consistent with the subjective nature of these ratings.The authors attribute the lower agreement to the subjectivity of fluency and coherence judgments.Source: Appendix B · p. 23

Appendix C. Parameter Tuning

• Llama 2 used LoRA on 100 training examples, whereas LED used full fine-tuning on 80% of either the full or annotated dataset.Both models used 100 validation and 100 test examples; the datasets contained 100,175 and 26,178 examples respectively.Source: Appendix C · p. 23
• LED tuning used 200,000 training steps with grid search, while Llama 2 tuning used 100 steps with validation every 10 steps.The LED models used maximum source and target lengths of 4,096 and 350; Llama 2 used a 4,096-token context and rarely required truncation.Source: Appendix C · p. 23

Appendix D. Prompt Tuning GPT-4

• GPT-4 prompt three produced simplified language matching the in-context examples, and adding five examples plus a “You were admitted” instruction further improved qualitative results.The authors selected the improved variant for GPT-4 summarization experiments.Source: Appendix D · p. 24, Table 12 · p. 25
• Prompt three generally achieved the best quantitative results on ten held-out examples, while additional in-context examples did not degrade performance.The output-start instruction also yielded a quantitative improvement.Source: Appendix D · p. 24

Appendix E. Hallucination Detection with GPT-4

• GPT-4 hallucination detection first prompts the model to mark hallucinated spans in the original summary and then extracts those annotated spans.Because GPT-4 is decoder-only, the pipeline uses HTML-like error tags rather than direct per-token predictions.Source: Appendix E · pp. 24–26
• The detection pipeline supports class-aware and class-agnostic labeling, with optional reasoning before full-summary annotation.The prompts can request specific hallucination types or only generic error tags, and can ask GPT-4 to generate explanations first.Source: Appendix E · p. 26

Appendix F. Qualitative Evaluation

• The evaluation adapted prior summary-quality instructions for medical patient summaries, adding Simplification and requiring hallucination, key-fact, and medical-jargon annotations.Annotators rated summaries on a Likert scale after labeling key facts and medical jargon; hallucinations were labeled using the developed protocol.Source: Appendix F · p. 26, §F.1 · p. 26

F.1. Labeling Key Facts and Medical Jargon

• Key facts were identified in the hospital-course context and summaries to determine omissions, while medical-jargon spans were counted once per summary.These tasks used a subjective procedure without a formal protocol or inter-annotator agreement procedure, limiting their standardization.Source: §F.1 · pp. 26–27, Figure 9 · p. 28

Appendix G. Guidelines for Patient Summary Annotation

• Annotation guidelines classified summary errors as unsupported facts, contradicted facts, or incorrect facts, with unsupported errors further divided by content type.Annotators labeled the smallest span needed to specify each error and separated multiple errors within a sentence for reliable counting.Source: §F.1 · p. 33, Appendix G · pp. 36–37

Addi;onal Clarifications

• The guidelines allowed general medical knowledge and advice without BHC support, while clarifications exempted implied consequences and specialty-based knowledge but required specific department names.They also specified precedence rules for complex mistakes, treatment of deidentified information, grammatical errors, and repeated content.Source: §F.1 · p. 33, Appendix G · p. 36, Addi;onal Clarifica;ons · p. 40

Addi;onal Clarifica;ons

• The clarification rules were developed through agreement procedures on 100 examples and allow the admission reason to come from a later major reason in the BHC.For instance, abdominal pain may be used when ileus is listed first but abdominal pain is also mentioned later.Source: Addi;onal Clarifica;ons · p. 41
• References to absent supplementary materials, such as medication plans or phone numbers, are labeled as unsupported facts.A statement that a medication plan is attached is unsupported when no such plan appears in the BHC.Source: Addi;onal Clarifica;ons · p. 41
• Close generalizations that aid patient understanding are permitted, whereas unrelated or fabricated clinical details remain unsupported.Calling neutropenia “low WBC” is acceptable, but attributing an unmentioned bleeding risk to a lab test is labeled unsupported.Source: Addi;onal Clarifica;ons · p. 41
• If one span contains logically connected error types, it receives a single “unsupported word” label.The example combines an unmentioned lab test with an unmentioned increased bleeding risk into one labeled span.Source: Addi;onal Clarifica;ons · p. 41
• Common daily medication dosages are treated as general knowledge, while unsupported entities referenced again in the next sentence are labeled as errors.Thus, stating Plavix 75 mg daily can be acceptable when only the therapy change is documented, but an unmentioned vascular surgery team remains unsupported when referred to as “They.”Source: Addi;onal Clarifica;ons · p. 41