PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Majority Vote: Multi-Perspective Adjudication for Medical Hallucination Detection

Joe Cecil, Marjorie Freedman

arXiv:2609.03953v1cs.CL

TL;DR

The paper asks how factual-error benchmarks can reliably capture subtle errors in long-form medical chatbot responses. It studies a multi-perspective pipeline combining first-pass annotation, LaJ candidate discovery, and expert or evidence-based adjudication, finding that first-pass labels miss validated errors, LaJ adds coverage but misses some annotator-found errors, and adjudicators can still disagree.

  • Problem

    Single-pass factual-error annotation can miss subtle claims embedded in mostly correct long-form chatbot responses, limiting confidence in benchmarks used to evaluate detection systems.

  • Method

    The study combines first-pass annotation, LLM-as-a-Judge candidate discovery, and medical-expert and evidence-based fact-checking adjudication.

  • Results

    First-pass annotators miss errors later validated by adjudicators; LaJ improves candidate coverage but misses errors found by annotators, while adjudicators can disagree.

  • Takeaways & Limitations

    Multi-pass adjudication can improve benchmark coverage, but benchmark interpretation remains sensitive to judgment, expertise, and evidence.

  • Takeaways & Limitations

    The strictest interpretation applies only to the study’s data, and the MedExpert analysis does not reproduce that dataset’s original annotation guidelines or task definition.

Abstract

from arXiv · show

Understanding the frequency of factual errors in chatbot-generated text and evaluating systems that detect these errors is critical for determining chatbot safety. Yet factual-error detection is often treated as a single-pass, single-annotator labeling problem. In long-form chatbot responses, factual errors can be subtle and embedded within mostly correct text. We develop a multi-perspective annotation study of medically relevant chatbot responses, combining first-pass annotation, LLM-as-a-Judge (LaJ) candidate discovery, and two forms of adjudication: medical-expert and evidence-based fact-checking. First-pass annotators frequently miss factual errors later validated by adjudicators. LaJ improves candidate discovery, but is insufficient on its own: It misses factual errors that annotators catch. We also find disagreement among adjudicators, suggesting that adjudication over multiple candidate sources can improve benchmark completeness, but does not eliminate the need to apply judgment and expertise. Applied to an existing benchmark, this technique reveals a similar pattern of missing annotations. Together, these results suggest that in the settings examined here, single-pass hallucination benchmarks may achieve scale at the cost of undercounting factual errors. Multi-pass adjudication can improve coverage, but inferences drawn from the benchmarks are still sensitive to the judgment, expertise, and evidence used to determine error presence.

1 Introduction

Factual-error annotation is difficult to scale reliably because subtle errors can be embedded in mostly correct long-form responses. The paper studies whether combining multiple candidate sources with adjudication improves coverage and benchmark reliability.

  • 1 Introduction: The study frames factual-error labeling as error discovery followed by adjudication rather than a single-pass judgment.It combines first-pass annotators, an LLM-as-a-Judge, medical experts, and fact-checkers.
  • 1 Introduction: First-pass annotators can miss factual errors later validated by adjudicators, making singly annotated references incomplete.The problem is especially acute when errors are subtle or embedded within otherwise correct information.
  • 1 Introduction: LaJ improves candidate coverage but misses some errors identified by human annotators, so it complements rather than replaces first-pass annotation.The paper therefore treats multiple perspectives as complementary sources for candidate discovery.
  • 1 Introduction: Disagreement among adjudicators shows that additional adjudication does not eliminate the judgment required to assess factual correctness.Medical expertise, evidence, and interpretation can lead adjudicators to different conclusions.
  • 1 Introduction: Figure 1 illustrates disagreement in which a first-pass annotator finds no error, LaJ identifies a candidate, and two adjudicators disagree.The example makes the multi-perspective pipeline concrete.

2 Related Work

Related work includes medical and general-domain hallucination datasets using different annotation scales, sources, and levels of adjudication. The paper distinguishes its approach by examining disagreement across annotators and adjudicators while using LaJ before adjudication.

  • 2 Related Work: Existing medical datasets differ in whether they use one annotator, multiple judgments, consensus-only outputs, or agreement analyses.Prior work also varies in domain, response format, scale, and access to annotations.
  • 2 Related Work: MedHallu focuses on synthetically injected hallucinations in technical PubMedQA-based responses rather than the annotation process studied here.Its responses are more technical and involve minimal annotator intervention.
  • 2 Related Work: This work incorporates an LLM judge before adjudication to examine errors not found during initial annotation.That ordering distinguishes it from the cited medical benchmarks.
  • 2 Related Work: FELM uses multiple first-pass annotators followed by adjudication, but does not examine automatic augmentation before adjudication or agreement among adjudication decisions.The comparison identifies the specific dimensions expanded by this paper.

3 MDHJudgments and Its Labeling

MDHJudgments contains real English chatbot responses to medically relevant questions and labels them through response- and section-level first-pass annotation followed by targeted adjudication. The pool combines human disagreements with LaJ disagreements, while the initial labels show low agreement.

  • 3 MDHJudgments and Its Labeling: MDHJudgments covers real chatbot responses about cystic fibrosis and pediatric infectious diseases, including medical and care-access questions.The dataset takes a broad view of medical relevance across the two domains.
  • 3 MDHJudgments and Its Labeling: First-pass annotators judge response accuracy globally, then review sections for claims, factual accuracy, related attributes, and comments.Response- and section-level judgments are distinct, and section annotation does not prescribe a single factuality unit.
  • 3 MDHJudgments and Its Labeling: 77% of sections had unanimous first-pass judgments, but only 6 of those sections were coded as having an error; 23% had some annotator disagreement.Table 1 summarizes the first-pass annotation and its agreement pattern.
  • 3 MDHJudgments and Its Labeling: Adjudication targets disagreement rather than random samples, with FP annotators and LaJ serving as candidate discoverers and adjudicators deciding whether an error is present.The design explicitly separates potential-error discovery from adjudication.
  • 3 MDHJudgments and Its Labeling: The study uses medical-expert adjudication for professional judgment and fact-checking adjudication for mismatches with authoritative published resources.Both adjudication styles involve multiple steps and are recruited through certification processes.
  • 3 MDHJudgments and Its Labeling: 95% of the adjudication pool comes from disagreements among FP annotators or between FP annotators and LaJ.Table 2 identifies 402 FP-disagreement sections and 389 FP–LaJ-disagreement sections as the dominant sources.

4 Analyzing has_error Agreement and Adjudication Decisions

Agreement analysis shows that first-pass annotation, LaJ, and adjudication provide complementary but imperfect views of factual errors. First-pass annotators often miss errors validated by adjudicators, LaJ improves recall but also misses errors, and adjudicator disagreement limits any single judgment as ground truth.

  • 4.2 Agreement across Labeling Approaches: Low agreement between adjudicators supports treating adjudication as an additional lens rather than a canonical source of truth.The authors therefore caution against treating any single judgment as definitive, even when adjudication improves candidate coverage.
  • 4.2 Agreement across Labeling Approaches: First-pass annotators have low agreement with adjudicators, with disagreements dominated by annotators labeling as correct sections adjudicators consider incorrect.This indicates that first-pass-only references can omit factual errors later validated through adjudication.
  • 4.2 Agreement across Labeling Approaches: LaJ has higher agreement and recall than individual first-pass annotators, but its disagreements remain skewed toward missed factual errors.LaJ can expand candidate discovery, yet its outputs alone do not provide complete error coverage.
  • 4.3 Impact of Adjudication on has_error Benchmarking: LaJ identifies five of six unanimously labeled error sections, yielding 83% recall but only 2% precision against the first-pass reference.Without adjudication, this combination could make LaJ appear unworkably prone to false alarms.
  • 4.4 Beyond a Binary Adjudication Label: Medical-expert adjudicators mark a major error in 35% of initial judgments after the first expert-adjudication step, while 12% require more research.These judgments show that missed errors can be important and that factual-error labeling is not always simple recognition.
  • 4.5 The Relationship between Response-Level Accuracy and Section-Level Errors: Response-level accuracy correlates only moderately with section-level error frequency, with a correlation of -0.39.Many responses have high response-level accuracy despite containing at least one section labeled has_error, so holistic judgments can obscure detailed errors.

5 Evidence of Missed Factual Errors Elsewhere: MedExpert as a Case Study

Applying LaJ and medical-expert adjudication to MedExpert revealed that many sections implicitly labeled correct contained factual errors, though some reflected differing scope judgments about medical relevance and harm.

  • MedExpert’s missed errors included cases adjudicators viewed as non-medical or minor and unlikely to cause harm under correct treatment.
  • The frequency and character of these missed errors suggest that low-recall annotation extends beyond MDHJudgments to another benchmark.

6 Conclusion

The study concludes that first-pass annotation can miss adjudicator-validated errors, while LaJ broadens candidate coverage without replacing annotators. Multi-pass adjudication improves coverage but leaves judgment and disagreement unresolved.

  • First-pass annotators miss both major and minor factual errors that adjudicators validate.
  • LLM-as-a-Judge improves candidate coverage for adjudication but complements rather than replaces first-pass annotation.
  • Adjudication may still produce disagreement because factual-accuracy decisions require judgment, knowledge, and evidence.
  • Singly annotated data supports scale but is likely incomplete, whereas richer multi-pass annotation does not ensure agreement or remove judgment.

Limitations

The paper’s conclusions are limited by dataset scope, disagreement-focused adjudication, model and annotator choices, and the need to interpret benchmarks as useful but potentially conservative estimates.

  • The strongest interpretation applies to the authors’ data, while the MedExpert analysis uses broader factuality criteria and is not an audit under MedExpert’s original task definition.
  • The study cannot distinguish adjudicator disagreement caused by simple mistakes from genuine differences in factual-accuracy judgment.
  • Because adjudication samples disagreement cases, adjudicator agreement on a random sample remains unknown.
  • Results may change with a different LaJ prompt or underlying model because the study uses one prompt and one model.
  • The responses and judgments come from US-based participants evaluating chatbot answers to surrogate patient questions and are not medical advice.
  • Hallucination benchmarks remain useful for conservative error-rate estimates and system comparisons, despite potentially incomplete annotations.

B Sources of Questions and Chatbot Responses

The study constructs varied medical chatbot responses from researcher-developed questions, prompt augmentations, and multiple language models. It reports sampling and confidence-interval procedures while cautioning that the dataset was not designed to benchmark individual chatbots.

  • Question and response generation: Questions were generated through a researcher-in-the-loop process covering pediatric infectious diseases, cystic fibrosis, intents, and optional user personas.An LLM-anchored pipeline generated candidate questions using domain and contextual instructions.
  • Question and response generation: Prompt augmentations asked chatbots to be brief or provide medical evidence, and some prompts intentionally injected factual errors.These augmentations increased response variety for annotation.
  • Question and response generation: Responses were generated by Gemma3-12b, Qwen3-32b with reasoning, and GPT-4.1.
  • Dataset scope: The study did not balance responses across chatbots because it was not intended to benchmark any particular chatbot’s propensity for factual errors.The full dataset retains augmentation and generating-chatbot information.
  • Uncertainty estimation: Confidence intervals used 10,000 bootstrap samples, with section-level sampling for agreement and response-level sampling for performance metrics.Section-level intervals were at most 3 percentage points narrower in recall bounds than response-level intervals.

C.3 Sensitivity of the Constructed Reference

The constructed-reference sensitivity analysis compares first-pass annotators and LaJ against adjudicated references and tests additional LaJ runs. Results show that LaJ detections remain valuable beyond the contributing run, but performance and interpretation depend on reference construction and model configuration.

  • Reference sensitivity: The reference may be more complete for contributing outputs because adjudication focused on disagreements involving those outputs.This creates a limitation when evaluating a novel detector.
  • Novel LaJ runs: Novel LaJ runs had reduced precision relative to the contributing LaJ and fell below FP precision for the FP+ME reference.Some apparent false positives may instead be detections absent from the constructed reference.
  • Novel LaJ runs: LaJ recall was reduced in novel runs but remained above first-pass annotator recall.The finding that LaJ detects errors missed by first-pass annotators persists outside the exact contributing output.
  • Reference sensitivity: 90% LaJ precision and 71–80% recall exceed first-pass recall of 20–21% against FP+ME and FP+FC references.The exact values are reported for LaJ and FP across both constructed references.
  • Novel LaJ runs: GPT-5’s higher recall may reflect similar detections across repeated runs rather than an intrinsic base-model advantage over Gemini Flash.The analysis holds the prompt fixed while varying the base model.

C.4 Richer Analysis of an Adjudicator’s has_error Label

The adjudicator-label analysis examines how medical experts and fact checkers reach has_error judgments across multiple steps. Both approaches identify substantial error prevalence, and the results show that comments and research can change initial judgments.

  • Medical Expert Adjudication: Medical experts’ comments frequently led to error labels, including for sections initially judged correct.Figure 3 summarizes label changes across the two-step process.
  • Medical Expert Adjudication: 65% of sections were labeled as errors initially by medical experts, with 20% more labeled errors after reviewing comments.Experts indicated that 12% of decisions required information beyond their existing knowledge, most classified as major.
  • Scope of the adjudication pool: The adjudication pool was enriched for suspected errors because it contained sections flagged by first-pass annotators or LaJ.Thus, the observed prevalence should not be interpreted as prevalence in the full dataset.
  • Implications: The findings indicate that experts need research and that first-pass annotation misses both major and minor factual errors.
  • Fact Checking Adjudication: Fact checkers judged 75% of sections in the adjudication pool to contain an accuracy error.They reported needing research to identify the error in 35% of sections.

C.5 Categorizing the Adjudicated Sections

The paper categorizes adjudicated comments and formalizes how multiple adjudication judgments become a single has_error label. It also documents the annotation workflow, adjudicator roles, and the LaJ prompt used for candidate discovery.

  • Comment categories: The MedExpert analysis found “Other Medical Information” most frequent among errors absent from the published answer key, followed by Clarity.All categories appeared among sections assigned has_error.
  • Comment categories: Medical experts applied every comment category to at least some comments from each first-pass annotator group or LaJ, with Other Medical Information most frequent.Table 11 reports category distributions by originating group.
  • has_error construction: For medical-expert adjudication, major or minor errors in the relevant judgment steps produce has_error=true; otherwise the label is false.Figure 3 shows the possible judgment combinations and their frequencies.
  • has_error construction: For fact-checking adjudication, has_error is determined from judgments connecting the retrieved evidence to the section, with Figure 5 mapping response pairs and handling contradictions.
  • LaJ prompt: The GPT-5 LaJ prompt evaluates only the target section for factual problems, using the medical question and surrounding context without judging the context itself.The required output is a reason followed by a True/False contains_problem answer.
  • Annotation workflow: The workflow combines first-pass annotators with medical-expert and fact-checking adjudicators recruited through separate role qualifications.First-pass annotators include medical professionals, trainees, and AI researchers; adjudicators are recruited through Prolific.

F.2 Composition and Performance of the FP Annotation Pool

The FP annotation pool combined heterogeneous annotators with response- and section-level judgments, using references and adjudication to identify factual errors. Performance analysis found a consistent low-recall trend across annotator groups.

  • Pool composition: FP annotators included medical professionals, medical trainees, and AI researchers, creating opportunities for complementary error detection.Annotation proceeded asynchronously without routing items to particular individuals or groups.
  • Performance: Low recall was consistent across all FP annotator groups, although their results are not directly comparable because groups handled arbitrary, nonrandom subsets.Table 13 reports micro-averaged recall and precision over constructed answer keys restricted to multiply annotated sections.
  • Annotation workflow: Each FP workflow began with response-level annotation and then proceeded to per-section annotation.The annotation framework separately captured section-level factual correctness and binary certainty, risk, and urgency attributes.
  • Reference use: References were encouraged during annotation but were not required to be checked, so incorrect citations were often noted without reference-specific searches.Annotators typically recorded one to three trusted references, while the guidelines stated that verifying references was outside the task.
  • Accuracy criteria: Annotators judged medically related content for factual accuracy, treating even one incorrect statement as sufficient to make a response inaccurate.Empathy and style were handled separately, while incorrectly organized correct information was not necessarily inaccurate.

I.1 MDHJudgments Examples

The MDHJudgments examples show that adjudication can revise, confirm, or reject candidate error labels. They also demonstrate that medical experts and fact checkers may disagree, while LaJ and annotator candidates each uncover errors the other source may miss.

  • Example 1: The medical expert changed an initial no-error judgment after reviewing prior reviewer comments.Table 16 presents the adjudicated content and judgments; Table 17 provides comments and evidence.
  • Example 2: Both adjudicators labeled a LaJ-identified error as major, although the medical expert added nuance that was not incorporated into has_error.The example separates the categorical error value from the adjudicator’s qualifying comment.
  • MDHJudgments examples: Adjudication can change, confirm, or reject candidate error labels, and disagreement between adjudicators can remain.The examples include concerns raised by both LaJ and FP annotators.
  • Example 3: Both adjudicators rejected a missing-information concern as insufficient to establish a factual error.The example illustrates that a candidate concern may be rejected rather than converted into an error label.
  • Example 4: The Fact Checker and Medical Expert disagreed about whether a factual error was present.The example shows that independent adjudication does not guarantee a shared conclusion.
  • MedExpert case study: In the MedExpert case study, adjudication confirmed LaJ-detected errors as either minor or major and rejected an error identified in the original dataset.These examples show both newly validated errors and removal of a prior candidate.
Loading 2609.03953v1…