PaperlayerView exact revision on arXiv

Source-linked AI summary

MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes

Asma Ben Abacha, Wen-wai Yim, Yujuan Fu, Zhaoyi Sun, Meliha Yetisgen, Fei Xia, Thomas Lin

arXiv:2412.19260v2cs.CLcs.AIcs.LG

TL;DR

Medical-text validation lacks a publicly available benchmark, despite concerns about LLM-generated errors in clinical documentation. The paper introduces MEDEC and evaluates LLMs and medical doctors on error detection and correction; the models perform well but are outperformed by doctors. The authors describe MEDEC as a sufficiently challenging benchmark for these tasks.

  • Problem

    The paper addresses the lack of a publicly available benchmark for assessing LLM validation of existing or generated medical text, amid concerns about incorrect or harmful generated information.

    Source: §1 · pp. 1–2

  • Method

    The authors introduce MEDEC and evaluate recent LLMs and medical doctors on medical-error detection and correction in clinical notes.

    Source: §1 · p. 2, §5.4 · p. 7, §6 · p. 10

  • Results

    Recent LLMs perform well on error detection and correction but are still outperformed by medical doctors.

    Source: §5.4 · p. 8, §6 · p. 10

  • Takeaways & Limitations

    The authors conclude that MEDEC is sufficiently challenging for assessing models’ ability to validate existing or generated notes and correct medical errors.

    Source: §6 · p. 10

    The dataset is limited in size and in the types of medical errors it covers, and the paper does not cover all possible detection and correction methods and models.

    Source: §7 · p. 10

Abstract

from arXiv · show

Several studies showed that Large Language Models (LLMs) can answer medical questions correctly, even outperforming the average human score in some medical exams. However, to our knowledge, no study has been conducted to assess the ability of language models to validate existing or generated medical text for correctness and consistency. In this paper, we introduce MEDEC (https://github.com/abachaa/MEDEC), the first publicly available benchmark for medical error detection and correction in clinical notes, covering five types of errors (Diagnosis, Management, Treatment, Pharmacotherapy, and Causal Organism). MEDEC consists of 3,848 clinical texts, including 488 clinical notes from three US hospital systems that were not previously seen by any LLM. The dataset has been used for the MEDIQA-CORR shared task to evaluate seventeen participating systems [Ben Abacha et al., 2024]. In this paper, we describe the data creation methods and we evaluate recent LLMs (e.g., o1-preview, GPT-4, Claude 3.5 Sonnet, and Gemini 2.0 Flash) for the tasks of detecting and correcting medical errors requiring both medical knowledge and reasoning capabilities. We also conducted a comparative study where two medical doctors performed the same task on the MEDEC test set. The results showed that MEDEC is a sufficiently challenging benchmark to assess the ability of models to validate existing or generated notes and to correct medical errors. We also found that although recent LLMs have a good performance in error detection and correction, they are still outperformed by medical doctors in these tasks. We discuss the potential factors behind this gap, the insights from our experiments, the limitations of current evaluation metrics, and share potential pointers for future research.

1 Introduction

Clinical-note mistakes are common, while LLM-generated documentation raises concerns about incorrect or harmful information. The paper introduces MEDEC to address the need for benchmarks assessing medical-error detection and correction.

1 Introduction

  • 1 in 5 patients who read a clinical note reported finding a mistake, and 40% of those perceived it as serious; diagnosis errors were most common.The reported figures come from a survey of US health care organizations.Source: §1 · p. 1
  • LLMs can answer medical exam questions accurately, but their potential to generate incorrect or harmful information remains an obstacle to medical-documentation use.The passage notes that harmful content might alter clinical decision making.Source: §1 · pp. 1–2
  • MEDEC addresses the lack of a publicly available benchmark by targeting medical-error detection and correction in clinical texts, tasks requiring medical expertise and sometimes practical experience.The paper presents MEDEC as the first publicly available benchmark and study of automatic error detection and correction in clinical notes.Source: §1 · p. 2

2 Related Work

Prior work examined general-domain error detection, consistency, medical-answer accuracy, and risks in LLM-assisted messaging. The paper presents MEDEC as a benchmark for medical-text validation and notes its use in a shared task.

2 Related Work

  • Prior consistency studies found language models could be inconsistent across tests and could produce logically inconsistent predictions.The cited work covered semantic, logical, and factual consistency, as well as newer models’ semantic consistency.Source: §2 · p. 2
  • Medical-answer studies found mostly accurate responses but noted limits on complex questions, while GPT models sometimes outperformed average human scores on medical exams.The accuracy and completeness of generated answers were evaluated by physicians in one study.Source: §2 · p. 3
  • LLM-assisted patient messaging showed that doctors might adopt LLM assessments and that some unedited drafts could lead to severe harm or death.The passage describes this as occurring in a minority of drafts.Source: §2 · p. 3
  • MEDEC addresses the stated lack of a public benchmark for validating existing or generated medical text and was used to evaluate seventeen teams in MEDIQA-CORR 2024 [Ben Abacha et al., 2024].The paper describes MEDEC as the first benchmark for medical-error detection and correction in clinical notes.Source: §2 · p. 3

3 MEDEC Dataset

MEDEC contains 3,848 clinical texts covering five medical-error types, with examples created from medical board questions and real hospital notes. Its two creation methods produce correct and error-bearing notes through different procedures.

3 MEDEC Dataset

  • The dataset contains 3,848 clinical texts from different specialties, annotated by eight medical annotators, and covers five error types: diagnosis, management, pharmacotherapy, treatment, and causal organism.The error types were selected after analyzing frequent medical-board-exam question types.Source: §3 · p. 3

3.1 Data Creation Method #1 (MS)

  • In Method #1 (MS), four medically trained annotators used MedQA narratives and multiple-choice questions to inject wrong answers, create correct versions, and manually check the rewritten text.The final dataset randomly selected correct and incorrect versions from scenarios with errors placed in the middle or at the end.Source: §3.1 · p. 3

3.2 Data Creation Method #2 (UW)

  • Method #2 (UW) sampled real diagnosis supports from three University of Washington hospital systems and had medical students introduce errors into 244 notes.Annotators selected or created a medical entity, replaced it with a similar but distinct erroneous concept, and required each error to contradict at least two other parts of the note.Source: §3.2 · p. 4
  • The test set contains 597 MS and 328 UW texts; 51.3% contain errors and 48.7% are correct.The MS subset is publicly available, while access to the UW subset requires a data usage agreement.Source: §3.2 · pp. 4–5
  • Figure 3 presents the dataset’s distribution across diagnosis, management, treatment, pharmacotherapy, and causal-organism errors.The supplied passages identify the categories but do not provide their distribution values.Source: §3.2 · p. 5, Figure 3 · p. 5

4 Medical Error Detection & Correction Approaches

The evaluation divides medical-error work into flagging errors, locating the erroneous sentence, and generating a correction. Two prompt variants solicit these outputs from LLMs.

4 Medical Error Detection & Correction Approaches

  • The approach evaluates three subtasks: predicting whether a text contains an error, extracting its sentence, and generating a corrected sentence for flagged texts.The specified outputs are an error flag, a sentence ID or -1, and a correction or NA, respectively.Source: §4 · p. 5
  • The systems use two prompts to generate the outputs, with P#2 adding a randomly selected training-set input/output example to P#1.P#1 asks for the sentence ID and corrected version when an error is found, or CORRECT when the text has no error.Source: §4 · p. 5
  • P#1 asks the model to check every sentence and use medical knowledge and reasoning to identify and correct errors related to treatment, management, cause, or diagnosis.The prompt presents the clinical narrative one sentence per line, with each line beginning with a sentence ID.Source: §4 · p. 5

5 Experiments & Results

The experiments compare recent language models with physician annotations on medical error detection and correction, using separate detection and correction metrics. Models performed well, but physicians still outperformed them overall; the results also expose task-specific ranking differences and limits of current correction metrics.

5.1 Language Models

  • Experiments span recent small and large models, including Phi-3-7B, Claude 3.5 Sonnet, Gemini 2.0 Flash, GPT-4o, and o1 models; some parameter estimates are uncertain.Source: §5.1 · p. 6

5.2 Evaluation Metrics

  • Detection was evaluated with Accuracy and per-error-type Recall; correction used ROUGE-1, BLEURT, BERTScore, and their average, Aggregate Score.Source: §5.2 · p. 6

5.3 Comparison with Expert Labeling

  • Two doctors annotated 569 of 925 test texts without ground truth, judging error presence and location and providing a correction and error type.Source: §5.3 · p. 6

5.4 Results

  • Claude 3.5 Sonnet led LLMs in error-flag Accuracy (70.16%) and error-sentence Accuracy (65.62%), while o1-preview led correction with 0.698 Aggregate Score.Source: §5.4 · p. 7
  • Recent LLMs performed well in error detection and correction, but medical doctors outperformed them overall.Source: §5.4 · p. 8
  • Detection and correction rankings diverged, while current state-of-the-art generation metrics may miss synonyms and medical-text similarities.Source: §5.4 · pp. 8–9
  • o1-preview had higher error-flag and sentence-detection Recall across all error types, but models showed a substantial Precision problem, often hallucinating error presence.Source: Table 4 · p. 8, §5.4 · p. 8
  • Examples include a valid synonym mismatch—Bruton versus X-linked agammaglobulinemia—and the authors recommend evaluating specialized medical LMs and new clinical-text metrics.Source: §5.4 · p. 9

6 Conclusion

MEDEC is a sufficiently challenging benchmark for assessing medical-error validation and correction, while doctors outperform recent LLMs on these tasks.

  • Although recent LLMs perform well, medical doctors outperform them on medical error detection and correction.Source: §6 · p. 10
  • The doctors’ annotations indicate that MEDEC is sufficiently challenging for assessing validation of existing or generated notes and medical-error correction.Source: §6 · p. 10
  • The authors hope MEDEC will support further research on error detection and correction, clinical reasoning in LLMs, and evaluation metrics for clinical texts and applications.Source: §6 · p. 10

7 Limitations

The paper’s coverage is limited in the methods and models evaluated and in the dataset’s size and error types.

  • The paper does not cover all possible methods and models for detecting and correcting medical errors.Source: §7 · p. 10
  • The dataset is limited in size.Source: §7 · p. 10
  • The dataset covers a limited range of medical error types.Source: §7 · p. 10
Loading 2412.19260v2…