PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Correctness: Validity-Oriented Evaluation of Biomedical LLM Judges

Rodrigo de Oliveira, Federico Pittino, James Gwinnutt, Jay Nanavati

arXiv:2608.29127v1cs.AI

TL;DR

Biomedical LLM judges need evaluation that captures more than agreement with human judgments, particularly when human labels are scarce and outputs can be unstable or non-compliant. The paper builds deterministic, metric-grounded preference pairs and evaluates four Llama-3.1-8B-Instruct training regimes across correctness, robustness, and compliance. SFT→RL performs best overall, with gains concentrated on decomposable tasks and difficulty-sensitive comparisons.

  • Problem

    Existing biomedical judge evaluation emphasizes human or expert agreement, while evidence about operational validity under instability and non-parseable outputs remains limited.

    Source: §1 · p. 1

  • Method

    The paper augments biomedical benchmarks with deterministic metric-grounded mutations and evaluates judges on correctness, repeated-sampling robustness, and output compliance.

    Source: §1 · p. 2, §2.4 · p. 4

  • Results

    SFT→RL is the strongest tested regime across correctness, robustness, and compliance, with gains concentrated on decomposable PICO and MedCalc tasks.

    Source: §6 · p. 6

  • Takeaways & Limitations

    Deterministic metric-grounded pairs can expose calibrated close-call comparisons and support validity-oriented evaluation beyond correctness alone.

    Source: §6 · p. 6

    Experiments use one base model, and calibration methods for LLM judges were not tested.

    Source: §4 · p. 6

Abstract

from arXiv · show

We propose a scalable, validity-oriented pipeline for evaluating biomedical LLM judges when high-quality human judgments are scarce. First, we augment existing human-labelled biomedical benchmarks with deterministic, metric-grounded mutations that produce auditable preference pairs. Second, we evaluate judges beyond aggregate correctness using three deployment-relevant dimensions: correctness against metric-derived gold labels, robustness under repeated stochastic sampling, and compliance with the requested output format. We use this pipeline to assess Llama-3.1-8B-Instruct under four regimes: (1) base, using the instruct model as is; (2) SFT, distillation-based supervised fine-tuning only; (3) RL, GRPO-based reinforcement learning only; and (4) SFT$\rightarrow$RL, SFT followed by RL. The base and single-stage regimes struggle on structured medical discrimination such as PICO extraction and clinical calculations, whereas SFT$\rightarrow$RL performs best across correctness, compliance, and robustness; gains concentrate on decomposable tasks (PICO, MedCalc), at times matching or outperforming frontier models.

1 Introduction

The paper argues that biomedical LLM judges require evaluation beyond agreement with human labels, because operational failures include instability and non-parseable responses. It proposes a validity-oriented protocol and deterministic preference-data generation to study training regimes and difficulty in partially correct tasks.

  • Biomedical judges must discriminate fine-grained PICO spans, calculation steps, and subtle distractors in high-stakes applications.Source: §1 · p. 1
  • Existing evaluations emphasize agreement with human or expert judgments while giving less attention to instability and non-parseable responses.Source: §1 · p. 1
  • The study asks whether disagreements are operational, which training regime works best for small biomedical judges, and whether deterministic data can model partial-task difficulty.Source: §1 · pp. 1–2
  • The proposed protocol combines criterion-linked correctness, resampling robustness, and output compliance.Source: §1 · pp. 1–2
  • Deterministic, metric-grounded preference pairs provide scalable, auditable training and evaluation data without new human-rating campaigns.Source: §1 · p. 2

2 Method

The method constructs auditable preference pairs from benchmark-specific mutations and evaluates judges on correctness, robustness, and compliance across decomposable and atomic biomedical tasks. It compares base, SFT, RL, and SFT→RL regimes using held-out stochastic evaluations and frontier baselines.

Task structure

  • The five tasks comprise decomposable PICO and MedCalc benchmarks plus atomic MedBullets, MedQA, and PubMedQA benchmarks.Source: §2.1 · p. 2

Synthetic preference construction

  • Task-specific mutation operators generate candidate responses, which are scored with macro-F1, MSE, or binary correctness against ground truth.Source: §2.2 · p. 2
  • Candidates are ranked and paired into chosen–rejected examples, with gold verdict l identifying the preferred slot; decomposable pairs are stratified by rank-gap difficulty.Source: §2.2 · pp. 2–3

Training regimes

  • SFT distills teacher completions filtered by Q=3, whereas RL uses the same preference pairs and gold verdicts with verifiable rewards and no stored teacher completions.Source: §2.2 · p. 3
  • The study evaluates base, SFT, RL, and SFT→RL Llama-3.1-8B-Instruct regimes using synthetic data, held-out comparisons, and untuned frontier APIs.Source: §2.3 · p. 3, §2.4 · p. 4

Meta-evaluation axes

  • Meta-evaluation measures majority-vote correctness, verdict variance under repeated stochastic decoding, and compliance conditional on error.Source: §2.4 · pp. 3–4

Figures

  • Figure 1 reports response categories and robustness by dataset, while Figure 2 reports response categories and robustness across hard, medium, and easy decomposable rank-gap tertiles.Source: Figure 1 · p. 4, Figure 2 · p. 5

3 Results

The evaluation finds that operational failures are measurable and that SFT→RL is the strongest training regime for small biomedical judges. Its advantages are largest on decomposable tasks, while deterministic rank gaps produce calibrated difficulty tiers.

  • Operational failures include verdict instability under resampling and non-parseable responses, rather than only substantive judging errors.These failures are measurable and non-negligible in biomedical evaluation.Source: §3 · p. 5
  • SFT→RL beats all other trained regimes on correctness, compliance, and robustness.This establishes the best overall trained regime under the validity-oriented protocol.Source: §3 · p. 5
  • 0.81 pairwise correctness is the highest average correctness achieved by SFT→RL.The result is reported across the evaluated biomedical tasks.Source: §3 · p. 5
  • +0.09 on PICO and MedCalc versus +0.02 on atomic QA shows that SFT→RL gains concentrate on decomposable items.Applying RL directly to the base model is harmful, at −0.06 versus base, suggesting RL benefits from a domain prior established by SFT.Source: §3 · p. 5
  • 0.69 → 0.83 → 0.90 pairwise correctness across hard-to-easy rank-gap tertiles shows that deterministic preference construction models task difficulty.SFT→RL gains persist at every tertile, including close-call pairs in the hardest tier.Source: §3 · p. 5

4 Limitations

The study is bounded by its model scale, correctness-centric synthetic data, and untested calibration methods. Results therefore do not establish how larger models, safety-focused stress tests, or calibration would behave.

  • The experiments use only one base model, Llama-3.1-8B-Instruct.Qwen runs were omitted because many generations were non-compliant with Hugging Face’s TRL package.Source: §4 · p. 6
  • Hardware constraints limited the study to an 8B model, and larger checkpoints such as 70B may behave differently.The authors identify larger-model evaluation as future work.Source: §4 · p. 6
  • The synthetic data pipeline is correctness-centric because it deterministically injects errors into otherwise grounded responses.The authors suggest extending the machinery to safety-centric stress tests by injecting dangerous content into rejected responses.Source: §4 · p. 6
  • Post-hoc calibration methods, which may improve trustworthiness in biomedical language-model applications, were not tested for LLM judges.Calibration remains an unexamined aspect of this evaluation.Source: §4 · p. 6

5 Related work

Prior work studies bias, consistency, and rubric-based judge assessment, while generative judges are also tuned with synthetic preferences and verifiable rewards. This paper combines domain SFT, RL, and metric-derived biomedical evaluation across structured tasks.

  • Recent generative-judge studies use RL on synthetic preferences and verifiable rewards, typically via GRPO; this paper transfers J1 to biomedicine.The study tests whether domain SFT should precede RL rather than proposing a new optimizer.Source: §5 · p. 6
  • EvalBiasBench, Sage, YESciEval, and clinical-judge studies respectively address superficial bias, preference consistency, multi-rubric assessment, and validated instruments.These approaches target different settings and remain separate from tuning studies.Source: §5 · p. 6
  • The paper contributes a three-dimensional protocol measuring correctness, robustness, and compliance on fully verifiable, metric-derived biomedical preferences.It combines domain SFT, RL judge tuning, and systematic pairwise meta-evaluation across structured biomedical tasks.Source: §5 · p. 6

6 Conclusion

The study introduces a validity-oriented pipeline combining auditable metric-grounded preferences with correctness, robustness, and compliance evaluation. Across four regimes and five benchmarks, SFT→RL is strongest overall, with gains concentrated on decomposable tasks and difficulty-sensitive performance.

  • The pipeline combines metric-grounded synthetic preferences with correctness, robustness, and compliance evaluation across four regimes and five biomedical benchmarks.Source: §6 · p. 6
  • SFT→RL is the strongest tested regime across correctness, robustness, and compliance.Source: §6 · p. 6
  • 0.84 versus 0.75 for base on decomposable tasks captures where SFT→RL gains concentrate, while direct RL from base is harmful.Source: §6 · p. 6
  • 0.69 to 0.90 pairwise correctness across rank-gap tertiles shows monotonic improvement from hardest to easiest bins for SFT→RL.Source: §6 · p. 6
  • Auditable preference data and multidimensional evaluation can make small biomedical LLM judges more reliable and operationally useful.Source: §6 · p. 6

A Synthetic data generation

The synthetic-data pipeline mutates source examples into candidate responses, scores them against task-specific ground truth, ranks them, and forms auditable preference pairs. It adapts pair construction to decomposable versus atomic task structure and stratifies decomposable pairs by rank-gap difficulty.

A.1 PICO

The PICO example extracts population, intervention, comparator, and outcome spans from a clinical trial and scores candidates using macro-F1 exact match. Candidate B receives an MSE-style reported score of 0.21 in the supplied example.

  • The source describes a randomized, double-blind, multicentre comparison of budesonide Turbuhaler and budesonide aqua for seasonal allergic rhinitis.Source: §A.1 · p. 10
  • PICO extraction organizes trial information into population, intervention, comparator, and outcome span sets.Source: §A.1 · p. 11
  • Macro-F1 exact match lowercases spans, strips punctuation, and computes set-overlap precision, recall, and F1 for each PICO key.Source: §A.1 · p. 11
  • 0.21 is the reported candidate score m(B, y) for the supplied PICO example.Source: §A.1 · p. 12
  • The MedCalc example uses age, sex, adjusted body weight, and serum creatinine to calculate creatinine clearance with MSE scoring against a scalar target.Source: §A.2 · pp. 12–13

A.3 MedBullets

The MedBullets example presents an emergency case with headache, nausea, and right-eye findings, followed by a multiple-choice prophylaxis question. The extracted answer shown is acetazolamide.

  • The case describes a 64-year-old man with headache, nausea, vomiting, and right-eye visual symptoms.Source: §A.3 · p. 14
  • The clinical findings include a minimally light-responsive pupil, a firm globe, and a right-sided carotid bruit.Source: §A.3 · p. 14
  • The question asks for the most appropriate prophylaxis from five medication options.Source: §A.3 · p. 14
  • The displayed selected answer is A. Acetazolamide, while Clopidogrel appears as option C.Source: §A.3 · p. 14
  • Boolean correctness assigns 1.0 when a candidate exactly matches the reference answer and 0.0 otherwise.Source: §A.3 · p. 14

B Prompt templates

The prompt templates standardize biomedical judge instructions across pairwise and pointwise modes while preserving task-specific formats and rubrics. They require explicit reasoning structure and constrained verdict or score outputs.

B Prompt templates

  • The same prompt templates support distillation-data generation, supervised fine-tuning, reinforcement learning, and meta-evaluation.Source: B Prompt templates · p. 15
  • Every judge receives a system instruction to act as an impartial evaluator of how well responses fulfill user instructions.Source: B Prompt templates · p. 15
  • User messages conceptually contain format instructions, task instructions, source text, and candidate responses.Source: B Prompt templates · p. 15

B.1 Pairwise

  • Pairwise judging compares two assistant responses and selects the one that better follows the instructions and provides the higher-quality answer.Source: §B.1 · p. 15
  • Pairwise prompts require reasoning inside <think> tags and a verdict inside <answer> tags using [[A]] or [[B]].Source: §B.1 · p. 15
  • Pairwise inputs include the source passage, two candidate responses, and task instructions or rubric, with slot assignment varied across batches.Source: §B.1 · pp. 15–16

B.2 Pointwise

  • Pointwise judging evaluates one assistant response against the user question and requires a score from 0 to 10 enclosed in <score> tags.Source: §B.2 · p. 16

B.3 Task instructions

  • PICO instructions require extracting Population, Intervention, Comparison, and Outcome as short verbatim strings in a JSON object with keys P, I, C, and O.Source: §B.3 · p. 17

C Detailed results

The detailed-results section organizes evaluation outputs across correctness, verdict variance, invalid-response share, and rank-gap difficulty. The tables distinguish decomposable and atomic families, summarize averages, and report difficulty-stratified results.

  • Figure 1 summarizes pairwise judging correctness, robustness, and compliance, while Tables 2, 3, and 4 provide exact per-suite values.Source: C Detailed results · p. 18
  • Table 2 reports correctness for decomposable and atomic families, individual benchmarks, and the unweighted average across all five suites.Source: Table 2 · p. 18
  • Table 3 defines verdict variance as the mean |high −low| across datasets over majority-vote and any-/all-sample correctness rates, with lower values preferred.Source: Table 3 · p. 18
  • Table 4 defines invalid-response share as invalid divided by invalid plus legitimate errors, with lower values preferred.Source: Table 4 · p. 18
  • Table 5 reports rank-gap difficulty for PICO and MedCalc using per-example Hard, Medium, and Easy tertiles, where Hard contains the smallest gaps.Source: Table 5 · p. 18
Loading 2608.29127v1…