PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Better Assessment of LLMs' Performance in Clinical Error Detection

Yifan Zhang, Rahmatollah Beheshti

arXiv:2608.16643v1cs.CLcs.AIcs.LG

TL;DR

Clinical error-detection benchmarks evaluate notes individually even when erroneous notes have clean counterparts, limiting assessment of true discrimination. This paper evaluates paired notes with pointwise metrics, Both-Correct Rate, and evidence analysis. Most tested models fell below 25% random pairwise discrimination despite reasonable aggregate scores, while F1 rankings typically favored the weakest discriminators.

  • Problem

    Aggregate metrics evaluate clinical notes in isolation and can be inflated by models’ systematic yes- or no-error prediction bias.

  • Method

    The study evaluates paired error-injected and clean notes using pointwise metrics, Both-Correct Rate, and Evidence Contrastive Analysis.

  • Results

    Most tested models fell below 25% random pairwise discrimination despite reasonable aggregate scores, with bias varying by language.

  • Takeaways & Limitations

    F1 and pairwise accuracy can move in opposite directions under the same bias, so F1 rankings typically favor the weakest discriminators.

  • Takeaways & Limitations

    The empirical results are confined to substitution-form errors because the public paired benchmarks do not include insertion or omission errors.

Abstract

from arXiv · show

Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such models rest on benchmarks that evaluate each clinical note in isolation. Error-detection benchmarks are typically constructed by injecting errors into notes, such that each erroneous note has a natural counterpart. Aggregate discriminative metrics (e.g., balanced accuracy or F1) do not exploit this structure. We show that this omission is consequential. In particular, evaluating 15 diverse LLMs on 4 standardized clinical error-detection test sets across 3 languages, we find that 13 of 15 models fall below the level of random pairwise discrimination, even while achieving F1 scores that standard practice would read as moderate. We also observe that the underlying bias patterns differ across languages: the same model can default to "no error" on one language and over-flag errors on another. To diagnose where discrimination breaks down, we further introduce a procedure to score the evidence models cite in their outputs. We find that while models consistently locate error-relevant content, they fail to produce the corresponding correct verdict on the clean counterpart. Finally, we show that F1 and pairwise accuracy are driven in opposite directions by the same underlying bias, so that ranking models by F1 may systematically promote the weakest discriminators. For safety-critical clinical NLP applications, we advocate for supplementing aggregate metrics with paired evaluations in benchmark reporting. Code and analysis scripts are available at https://github.com/healthylaife/paired-clinical-eval.

1. Introduction

Clinical error detection requires models to assess clinical reasoning, but aggregate note-level metrics can reward biased predictions rather than paired discrimination. This paper introduces paired evaluation and evidence analysis to expose pervasive discrimination failures and misleading model rankings.

  • Motivation: Clinical documentation errors can propagate into downstream care decisions, making automated detection important for reducing preventable harm.Examples include incorrect diagnoses, inappropriate treatment plans, and misidentified pathogens.
  • Task difficulty: Error detection requires models to evaluate clinical reasoning and distinguish incorrect decisions from unconventional but clinically appropriate ones.Models must connect symptoms, diagnoses, and management while judging whether the reasoning is sound.
  • Paired evaluation: Paired benchmarks contain error-injected and correct notes differing by one sentence, revealing whether models discriminate or repeat the same label.A truly discriminative model identifies the error in the corrupted note and clears the clean counterpart.
  • Methods: The framework defines Both-Correct Rate and Evidence Contrastive Analysis to measure paired correctness and whether cited evidence overlaps ground-truth error and correction sentences.It applies to substitution, insertion, and omission pairs, while experiments focus on substitution-form errors in public benchmarks.
  • Findings: The vast majority of tested models fall below 25% random pairwise discrimination despite reasonable aggregate scores, with bias varying by language.The same model can show yes-bias in one language and no-bias in another.
  • Findings: Models locate error-relevant evidence yet often issue the same verdict for both notes, while F1 and BCR can rank models in opposite directions.In the majority of yes-bias failures, models locate the error sentence in the erroneous note but label both notes erroneous; models ranked highest by F1 are typically lowest by BCR.

2. Related Work

Prior work has established clinical error-detection benchmarks, including shared-task, public-corpus, and multilingual efforts, but their evaluation paradigm remains aggregate-focused. Related research motivates paired evaluation by separating discrimination from response bias and by showing that models’ internal information and explanations may diverge from their outputs.

  • Clinical error-detection benchmarks: Clinical error-detection benchmarks progressed from MEDIQA-CORR 2024 to the larger MEDEC corpus and subsequent multilingual evaluations.These efforts were motivated by the need for safer clinical documentation.
  • Prediction bias: Systematic prediction bias can make LLMs default to one output class regardless of the input, including yes-bias toward “error present” in clinical error detection.Sycophancy is identified as one mechanism that can produce such bias.
  • Paired evaluation: Signal detection theory distinguishes class discrimination from response bias, while minimal-pair and contrast-set evaluations use minimally contrastive inputs to expose weaknesses.Measured accuracy reflects both discrimination and response bias, so bias alone can produce high accuracy under skewed class priors or single-class evaluation.
  • Internal information and explanations: LLMs may encode correct information internally while generating incorrect outputs, and chain-of-thought explanations may misstate the factors driving predictions.Probing studies require weight access, which is unavailable for closed models that dominate clinical deployment.

3. Methods

The evaluation treats matched error and clean notes as the unit of analysis while still obtaining predictions independently per note. It combines pointwise metrics with pair-level discrimination, evidence diagnostics, and robustness checks across datasets, models, prompts, and decoding settings.

  • Evaluation design: Matched error and clean notes from the same clinical scenarios define the evaluation pairs, with pairing applied post-hoc to independently generated note-level predictions.The binary task flags whether each note contains a medical error, without conflating detection with correction.
  • Evaluation layers: Three parallel layers assess performance: traditional pointwise metrics, pairwise Both-Correct Rate (BCR), and Evidence Contrastive Analysis (ECA).BCR measures the fraction of pairs classified correctly on both members, while ECA examines whether cited evidence overlaps clinically relevant content.
  • Datasets and models: Four clinical error-detection test sets span three languages, and the study evaluates 15 instruction-tuned LLMs across three size tiers and five model families.The datasets are MS-Test, MEB-EN, MEB-CN, and MRT-JA; MedRECT-JA contains 190 error notes sharing 105 unique clean notes.
  • Datasets and models: All models are evaluated zero-shot without task-specific examples or fine-tuning, enabling comparisons of scale within families and medical specialization at matched scale.The study compares model sizes such as Qwen 3 at 4B, 8B, and 32B, and Gemma 3-27B with MedGemma 27B.
  • Robustness analysis: A 2×2 prompt–decoding matrix measures configuration sensitivity by varying instruction wording and sampling, with cross-configuration means and standard deviations separating stable behavior from artifacts.Greedy decoding provides reproducible outputs, while stochastic sampling tests whether sampling variability shifts bias patterns.
  • Pointwise metrics: The framework reports balanced accuracy, F1, precision, recall, specificity, MCC, and error-flag rate, using balanced accuracy and MCC as primary anchors for class imbalance and degenerate classifiers.F1 and recall are included to expose discrepancies between bias-sensitive and bias-resistant metrics.

4. Results

Across 240 runs, aggregate F1 often suggested moderate performance despite near-chance pairwise discrimination, with bias varying across languages and prompting conditions. Models frequently localized error-relevant content yet failed to classify the clean counterpart correctly, while F1 and BCR moved in opposite directions.

  • Pairwise discrimination: Across 240 runs, balanced accuracy ranged from 0.45 to 0.71 and MCC was near zero for most models, indicating near-chance discrimination.F1 reached 0.80, and 178 of 240 runs (74%) had F1 above 0.5 while balanced accuracy was at or below 0.6.
  • Pairwise discrimination: 13 of 15 models fell below the 25% mean BCR expected from random pairwise discrimination; only Qwen 3-32B and UltraMedical 70B exceeded it.Their mean BCRs were 28.0% and 25.7%, respectively, but both still failed to classify both pair members correctly more than 70% of the time.
  • Bias patterns: Bias varied across datasets and prompts: eight of 15 models changed bias category across datasets, while seven switched from yes-bias under neutral prompting to no-bias under conservative prompting.Qwen 3-8B reversed from no-bias on Chinese (0.36) to yes-bias on Japanese (0.69), showing that conservative prompting did not consistently move predictions toward balance.
  • Evidence localization: On MS-Test, all 15 models exceeded random TP-localization baselines, and Both-Hit plus TP-Only accounted for 46–87% of categorized Pred1 pairs.The mean share was 70%, indicating that models often attended to the error sentence despite failing pairwise judgment.
  • Metric deception: Prediction bias inflated F1 (r = +0.85) while suppressing BCR (r = −0.49), producing a near-zero direct F1–BCR correlation (r = −0.06).Thirty-two of 60 entries (53%) had F1 ≥0.6 but BCR < 25%, and all 60 entries fell below the independence baseline.

5. Discussion

The discussion argues that aggregate metrics can reward biased clinical error detectors, making paired discrimination measures necessary for model selection and deployment screening. It also identifies localization–judgment separation, constrained deployment uses, and scope limitations that motivate further evaluation and intervention.

  • Metric selection: 13 of 15 models fell below 25% random BCR despite many exceeding F1 0.6, so F1-based selection favored the weakest discriminators.On MS-Test, F1 ranked Gemma 3-4B, Llama 3.2-3B, and Mistral 7B highest, although their BCRs were only 4.5–5.9%.
  • Metric selection: At ≈50–60% balanced accuracy, models can reach F1 ≈0.68 by always predicting one class, making pairwise evaluation necessary to separate discrimination from bias.At ≈90% balanced accuracy, bias has less room to inflate F1 and BCR is mathematically constrained to be high.
  • Deployment implications: The nearly empty high-F1/high-BCR region shows that high F1 usually arose from bias, supporting paired reporting of BCR alongside traditional metrics.The discussion recommends reporting BCR and the independence ratio next to F1 and MCC so rankings reflect discrimination rather than class tendency.
  • Interventions: Models often produced correct error-relevant evidence but failed on the corresponding verdict, suggesting contrastive fine-tuning or localization-then-judgment pipelines.This localization–judgment separation targets the judgment component while preserving relatively strong localization ability.
  • Deployment implications: Until discrimination improves, high-recall yes-biased models may support pre-filtering only with mandatory human review, adequate error base rates, and alert-fatigue monitoring.The authors state that current evidence may not support standalone clinical error-detection deployment; paired screening is not prospective workflow evaluation.
  • Limitations: The study’s zero-shot design, limited proprietary coverage, possible data leakage, 3B–70B scope, substitution-only benchmarks, and ECA matching constraints limit generalization.Despite these limitations, all 15 models exceeded random baselines and the localization–judgment gap appeared across all four benchmarks.

Appendix A. Pair Construction Details

BCR evaluation constructs matched error–correct note pairs using dataset-specific procedures. The methods include similarity-based matching for MEDEC and MedRECT, direct adjacency pairing for MedErrBench, and a many-to-one structure in MedRECT-JA.

  • Pair construction: BCR evaluation requires matched error–correct pairs from the same clinical scenario, with construction methods varying by dataset.The four datasets differ in release structure, so pairing is dataset-specific.
  • MEDEC MS-Test: 286 of 311 MEDEC MS-Test error notes are paired using one-to-one Jaccard word-overlap matching with a threshold of ≥0.6.Each error note is assigned to its highest-similarity clean note, and 25 remain unmatched.
  • MedErrBench-EN and MedErrBench-CN: 104 MedErrBench-EN pairs and 100 MedErrBench-CN pairs are formed by directly pairing consecutive alternating error and correct samples, with no unmatched samples.In each adjacent pair, one row has Error Flag = 1 and the other has Error Flag = 0.
  • MedRECT-JA: 190 MedRECT-JA pairs are formed by scenario ID and character-level overlap above 0.85, but shared clean notes create a many-to-one structure.MedRECT contains 190 error notes but only 105 unique clean notes, so some clean notes appear in multiple pairs.

Appendix B. An Upper Bound on BCR · Appendix C. Evaluated Models: References and Identifiers · Appendix D. Prompt Templates and Decoding Settings

Appendix B proves that BCR cannot exceed the weaker of sensitivity and specificity, explaining why class-biased predictors can achieve moderate F1 while attaining zero paired discrimination. Appendices C and D document the 15 evaluated models and standardized prompting and decoding configurations.

  • Appendix B. An Upper Bound on BCR: A class-biased predictor can achieve moderate F1 while its BCR is pinned at zero.The bound is governed by the weaker marginal, which class bias drives toward zero.
  • Appendix B. An Upper Bound on BCR: Appendix B establishes BCR ≤ sensitivity and BCR ≤ specificity, so BCR is bounded by the weaker marginal.Both-Correct pairs are the intersection of correctly flagged error notes and correctly cleared correct notes.
  • Appendix B. An Upper Bound on BCR: The always-error predictor has sensitivity = 1 and specificity = 0, yielding BCR = 0 while achieving F1 ≈ 0.67.On a class-balanced set, it has precision = 1/2 and recall = 1.
  • Appendix C. Evaluated Models: References and Identifiers: Appendix C lists references, repository identifiers, model families, medical-domain status, and loading precision for all 15 evaluated models.The models are instruction-tuned and grouped into Small 3–8B, Large 27–32B, and Very Large 70B tiers.
  • Appendix D. Prompt Templates and Decoding Settings: Greedy decoding uses temperature T = 0, whereas stochastic sampling uses T = 0.7 and top-p = 0.9.Each model runs once per prompt × decoding configuration, producing four runs per model–dataset pair.
  • Appendix D. Prompt Templates and Decoding Settings: All prompts require four output lines containing Evidence, Analysis, Confidence, and Error: Yes/No.Evidence is an exact quoted span or NA, Analysis contains 3–5 reasoning sentences, and Confidence ranges from 0–100.
  • Appendix D. Prompt Templates and Decoding Settings: Neutral and conservative prompting differ in system instructions and error-flagging thresholds.The conservative variant flags only clearly supported significant errors, avoids assumptions, and prefers no error for incomplete or ambiguous notes.
  • Appendix D. Prompt Templates and Decoding Settings: The prompt structure is adapted across datasets and languages while retaining English output keys for parsing consistency.MedErrBench-EN expands the error taxonomy to 10 categories; Chinese and Japanese variants translate the structure and use dataset-specific error types.

Appendix E. Robustness Analyses

Robustness analyses show that the main findings persist across parse-failure handling, evaluation thresholds, and evidence-localization criteria. Model rankings are generally stable, although absolute BCR values can vary substantially across configurations.

  • Parse failure handling: 9 of 240 runs exceeded a 5% parse-failure rate, but skip versus random assignment changed every model’s results by < 0.5 pp.Failures concentrated in UltraMedical and Llama 3.2-3B on non-English datasets; 13 of 15 models remained below 25% BCR under both strategies.
  • Threshold sensitivity: < 3 pp shifts in per-model TP-localization rates occurred across tested ECA and MS-Test thresholds, while model rank ordering was preserved.The MRT-JA threshold of 0.85 produced all 190 pairs; lowering it to 0.75 added none.
  • Cross-configuration BCR: 36.2 pp was the largest cross-configuration absolute BCR variation, observed for Llama 3.1-70B, despite largely stable model rankings.Figure 6 reports BCR across all 15 models and four configurations.

Appendix F. Prediction Bias Mediation Analysis

Prediction bias mediates the relationship between F1 and BCR: error-flag rate suppresses BCR while inflating F1 across most datasets. This bias can reverse model rankings, and actual BCR is below the expected sensitivity × specificity for every model–dataset entry.

  • Mediation analysis: Prediction bias, measured by error-flag rate, mediates the relationship between F1 and BCR.Tables 8–10 provide per-dataset mediation details.
  • Mediation analysis: Error-flag rate suppresses BCR across all datasets and strongly inflates F1 on three of four datasets.The inflation is weaker on MEB-EN; direct F1–BCR correlations vary in sign across datasets and are near zero only when pooled.
  • Ranking implications: On 3 of 4 datasets, no model appears in both the top-3 F1 and top-3 BCR lists.The lists use 4-config means.
  • Independence analysis: Actual BCR falls below the expected sensitivity × specificity for all 60 model–dataset entries.The independence ratio is the mean of per-model ratios R_independence from Equation 3, not the quotient of column means.

Appendix G. Detailed Results · G.1. Full Traditional Metrics (All 240 Runs)

Appendix G.1 reports traditional metrics for all 240 inference runs across four configurations, 15 models, and four datasets. Results are organized by dataset and balanced accuracy, with Flag% indicating the fraction of notes flagged as erroneous.

  • G.1. Full Traditional Metrics (All 240 Runs): Tables 11–14 cover 240 runs spanning 15 models, four datasets, and four inference configurations.Each configuration contains 60 model–dataset runs.
  • G.1. Full Traditional Metrics (All 240 Runs): Traditional metrics include BalAcc, F1, precision, recall, specificity, MCC, and Flag%.Flag% is the error-flag rate: the fraction of notes flagged as containing an error.
  • G.1. Full Traditional Metrics (All 240 Runs): Under Conservative-Greedy on MedErrBench-EN, UltraMed-70B achieved BalAcc .692 and F1 .640.The table is sorted by balanced accuracy within each dataset.
  • G.1. Full Traditional Metrics (All 240 Runs): Under Conservative-Greedy on MedErrBench-CN, Qwen 3-32B achieved BalAcc .590 and F1 .474.Its reported precision was .661 and Flag% was 28.0.
  • G.1. Full Traditional Metrics (All 240 Runs): Under Neutral-Greedy on MedRECT-JA, Qwen 3-4B achieved BalAcc .600 and F1 .801.Its recall was .942, specificity was .257, and Flag% was 87.1.
  • G.1. Full Traditional Metrics (All 240 Runs): Under Neutral-Sampling on MedErrBench-EN, UltraMed-8B achieved BalAcc .615 and MCC .245.Its F1 was .667 and Flag% was 67.7.

G.2. Full ECA Category Breakdown

Table 15 provides the complete ECA breakdown for all 15 models across four datasets, pooling category counts across four prompt–decoding configurations. It reports TP localization and FP evidence-hit as four-configuration mean overlap rates and lists all 60 model–dataset combinations.

  • Coverage and aggregation: TP localization and FP evidence-hit are reported as four-configuration mean overlap rates.This matches the convention used in Section 4.4; per-configuration raw outputs are available as supplementary material.
  • Coverage and aggregation: Table 15 covers all 15 models on all four datasets, with pooled ECA category counts across four prompt–decoding configurations.The categories are Both-Hit, TP-Only, FP-Only, Neither-Hit, and Extraction-Fail.
  • Coverage and aggregation: All 60 model–dataset combinations are listed because every combination had at least one configuration with ≥5 Pred1 pairs.Within each dataset, rows are sorted by TP localization.
Loading 2608.16643v1…