PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Correctness: Validity-Oriented Evaluation of Biomedical LLM Judges

Rodrigo de Oliveira, Federico Pittino, James Gwinnutt, Jay Nanavati

arXiv:2608.29127v1cs.AI

TL;DR

Biomedical LLM judges need evaluation that captures more than agreement with human judgments, particularly when human labels are scarce and outputs can be unstable or non-compliant. The paper builds deterministic, metric-grounded preference pairs and evaluates four Llama-3.1-8B-Instruct training regimes across correctness, robustness, and compliance. SFT→RL performs best overall, with gains concentrated on decomposable tasks and difficulty-sensitive comparisons.

  • Problem

    Existing biomedical judge evaluation emphasizes human or expert agreement, while evidence about operational validity under instability and non-parseable outputs remains limited.

  • Method

    The paper augments biomedical benchmarks with deterministic metric-grounded mutations and evaluates judges on correctness, repeated-sampling robustness, and output compliance.

  • Results

    SFT→RL is the strongest tested regime across correctness, robustness, and compliance, with gains concentrated on decomposable PICO and MedCalc tasks.

  • Takeaways & Limitations

    Deterministic metric-grounded pairs can expose calibrated close-call comparisons and support validity-oriented evaluation beyond correctness alone.

  • Takeaways & Limitations

    Experiments use one base model, and calibration methods for LLM judges were not tested.

Abstract

from arXiv · show

We propose a scalable, validity-oriented pipeline for evaluating biomedical LLM judges when high-quality human judgments are scarce. First, we augment existing human-labelled biomedical benchmarks with deterministic, metric-grounded mutations that produce auditable preference pairs. Second, we evaluate judges beyond aggregate correctness using three deployment-relevant dimensions: correctness against metric-derived gold labels, robustness under repeated stochastic sampling, and compliance with the requested output format. We use this pipeline to assess Llama-3.1-8B-Instruct under four regimes: (1) base, using the instruct model as is; (2) SFT, distillation-based supervised fine-tuning only; (3) RL, GRPO-based reinforcement learning only; and (4) SFT$\rightarrow$RL, SFT followed by RL. The base and single-stage regimes struggle on structured medical discrimination such as PICO extraction and clinical calculations, whereas SFT$\rightarrow$RL performs best across correctness, compliance, and robustness; gains concentrate on decomposable tasks (PICO, MedCalc), at times matching or outperforming frontier models.

1 Introduction

The paper argues that biomedical LLM judges require evaluation beyond agreement with human labels, because operational failures include instability and non-parseable responses. It proposes a validity-oriented protocol and deterministic preference-data generation to study training regimes and difficulty in partially correct tasks.

  • Biomedical judges must discriminate fine-grained PICO spans, calculation steps, and subtle distractors in high-stakes applications.
  • Existing evaluations emphasize agreement with human or expert judgments while giving less attention to instability and non-parseable responses.
  • The study asks whether disagreements are operational, which training regime works best for small biomedical judges, and whether deterministic data can model partial-task difficulty.
  • The proposed protocol combines criterion-linked correctness, resampling robustness, and output compliance.
  • Deterministic, metric-grounded preference pairs provide scalable, auditable training and evaluation data without new human-rating campaigns.

2 Method

The method constructs auditable preference pairs from benchmark-specific mutations and evaluates judges on correctness, robustness, and compliance across decomposable and atomic biomedical tasks. It compares base, SFT, RL, and SFT→RL regimes using held-out stochastic evaluations and frontier baselines.

  • Task structure: The five tasks comprise decomposable PICO and MedCalc benchmarks plus atomic MedBullets, MedQA, and PubMedQA benchmarks.
  • Synthetic preference construction: Task-specific mutation operators generate candidate responses, which are scored with macro-F1, MSE, or binary correctness against ground truth.
  • Synthetic preference construction: Candidates are ranked and paired into chosen–rejected examples, with gold verdict l identifying the preferred slot; decomposable pairs are stratified by rank-gap difficulty.
  • Training regimes: SFT distills teacher completions filtered by Q=3, whereas RL uses the same preference pairs and gold verdicts with verifiable rewards and no stored teacher completions.
  • Training regimes: The study evaluates base, SFT, RL, and SFT→RL Llama-3.1-8B-Instruct regimes using synthetic data, held-out comparisons, and untuned frontier APIs.
  • Meta-evaluation axes: Meta-evaluation measures majority-vote correctness, verdict variance under repeated stochastic decoding, and compliance conditional on error.
  • Figures: Figure 1 reports response categories and robustness by dataset, while Figure 2 reports response categories and robustness across hard, medium, and easy decomposable rank-gap tertiles.

3 Results

The evaluation finds that operational failures are measurable and that SFT→RL is the strongest training regime for small biomedical judges. Its advantages are largest on decomposable tasks, while deterministic rank gaps produce calibrated difficulty tiers.

  • Operational failures include verdict instability under resampling and non-parseable responses, rather than only substantive judging errors.These failures are measurable and non-negligible in biomedical evaluation.
  • SFT→RL beats all other trained regimes on correctness, compliance, and robustness.This establishes the best overall trained regime under the validity-oriented protocol.
  • 0.81 pairwise correctness is the highest average correctness achieved by SFT→RL.The result is reported across the evaluated biomedical tasks.
  • +0.09 on PICO and MedCalc versus +0.02 on atomic QA shows that SFT→RL gains concentrate on decomposable items.Applying RL directly to the base model is harmful, at −0.06 versus base, suggesting RL benefits from a domain prior established by SFT.
  • 0.69 → 0.83 → 0.90 pairwise correctness across hard-to-easy rank-gap tertiles shows that deterministic preference construction models task difficulty.SFT→RL gains persist at every tertile, including close-call pairs in the hardest tier.

4 Limitations

The study is bounded by its model scale, correctness-centric synthetic data, and untested calibration methods. Results therefore do not establish how larger models, safety-focused stress tests, or calibration would behave.

  • The experiments use only one base model, Llama-3.1-8B-Instruct.Qwen runs were omitted because many generations were non-compliant with Hugging Face’s TRL package.
  • Hardware constraints limited the study to an 8B model, and larger checkpoints such as 70B may behave differently.The authors identify larger-model evaluation as future work.
  • The synthetic data pipeline is correctness-centric because it deterministically injects errors into otherwise grounded responses.The authors suggest extending the machinery to safety-centric stress tests by injecting dangerous content into rejected responses.
  • Post-hoc calibration methods, which may improve trustworthiness in biomedical language-model applications, were not tested for LLM judges.Calibration remains an unexamined aspect of this evaluation.

5 Related work

Prior work studies bias, consistency, and rubric-based judge assessment, while generative judges are also tuned with synthetic preferences and verifiable rewards. This paper combines domain SFT, RL, and metric-derived biomedical evaluation across structured tasks.

  • Recent generative-judge studies use RL on synthetic preferences and verifiable rewards, typically via GRPO; this paper transfers J1 to biomedicine.The study tests whether domain SFT should precede RL rather than proposing a new optimizer.
  • EvalBiasBench, Sage, YESciEval, and clinical-judge studies respectively address superficial bias, preference consistency, multi-rubric assessment, and validated instruments.These approaches target different settings and remain separate from tuning studies.
  • The paper contributes a three-dimensional protocol measuring correctness, robustness, and compliance on fully verifiable, metric-derived biomedical preferences.It combines domain SFT, RL judge tuning, and systematic pairwise meta-evaluation across structured biomedical tasks.

6 Conclusion

The study introduces a validity-oriented pipeline combining auditable metric-grounded preferences with correctness, robustness, and compliance evaluation. Across four regimes and five benchmarks, SFT→RL is strongest overall, with gains concentrated on decomposable tasks and difficulty-sensitive performance.

  • The pipeline combines metric-grounded synthetic preferences with correctness, robustness, and compliance evaluation across four regimes and five biomedical benchmarks.
  • SFT→RL is the strongest tested regime across correctness, robustness, and compliance.
  • 0.84 versus 0.75 for base on decomposable tasks captures where SFT→RL gains concentrate, while direct RL from base is harmful.
  • 0.69 to 0.90 pairwise correctness across rank-gap tertiles shows monotonic improvement from hardest to easiest bins for SFT→RL.
  • Auditable preference data and multidimensional evaluation can make small biomedical LLM judges more reliable and operationally useful.

A Synthetic data generation

The synthetic-data pipeline mutates source examples into candidate responses, scores them against task-specific ground truth, ranks them, and forms auditable preference pairs. It adapts pair construction to decomposable versus atomic task structure and stratifies decomposable pairs by rank-gap difficulty.

  • For each source example, a task-specific operator generates K candidates, scores them against ground truth with a suite-specific metric, and ranks the pool.
  • Task structure determines the ground truth shape, mutation operator, and metric: PICO and MedCalc are decomposable, while MedBullets, MedQA, and PubMedQA are atomic.
  • Atomic pairs contrast the top-ranked correct candidate with each lower-ranked candidate without rank-gap stratification.
  • The pair-selection algorithm uses three rank-gap tertiles and a score-gap threshold τ=0.05 for decomposable tasks.
  • One valid chosen–rejected tuple is sampled per source, bins are balanced, and candidate slots are randomly swapped at a fixed seed for reproducibility.

A.1 PICO

The PICO example extracts population, intervention, comparator, and outcome spans from a clinical trial and scores candidates using macro-F1 exact match. Candidate B receives an MSE-style reported score of 0.21 in the supplied example.

  • The source describes a randomized, double-blind, multicentre comparison of budesonide Turbuhaler and budesonide aqua for seasonal allergic rhinitis.
  • PICO extraction organizes trial information into population, intervention, comparator, and outcome span sets.
  • Macro-F1 exact match lowercases spans, strips punctuation, and computes set-overlap precision, recall, and F1 for each PICO key.
  • 0.21 is the reported candidate score m(B, y) for the supplied PICO example.
  • The MedCalc example uses age, sex, adjusted body weight, and serum creatinine to calculate creatinine clearance with MSE scoring against a scalar target.

A.3 MedBullets

The MedBullets example presents an emergency case with headache, nausea, and right-eye findings, followed by a multiple-choice prophylaxis question. The extracted answer shown is acetazolamide.

  • The case describes a 64-year-old man with headache, nausea, vomiting, and right-eye visual symptoms.
  • The clinical findings include a minimally light-responsive pupil, a firm globe, and a right-sided carotid bruit.
  • The question asks for the most appropriate prophylaxis from five medication options.
  • The displayed selected answer is A. Acetazolamide, while Clopidogrel appears as option C.
  • Boolean correctness assigns 1.0 when a candidate exactly matches the reference answer and 0.0 otherwise.

B Prompt templates

The prompt templates standardize biomedical judge instructions across pairwise and pointwise modes while preserving task-specific formats and rubrics. They require explicit reasoning structure and constrained verdict or score outputs.

  • B Prompt templates: The same prompt templates support distillation-data generation, supervised fine-tuning, reinforcement learning, and meta-evaluation.
  • B Prompt templates: Every judge receives a system instruction to act as an impartial evaluator of how well responses fulfill user instructions.
  • B Prompt templates: User messages conceptually contain format instructions, task instructions, source text, and candidate responses.
  • B.1 Pairwise: Pairwise judging compares two assistant responses and selects the one that better follows the instructions and provides the higher-quality answer.
  • B.1 Pairwise: Pairwise prompts require reasoning inside <think> tags and a verdict inside <answer> tags using [[A]] or [[B]].
  • B.1 Pairwise: Pairwise inputs include the source passage, two candidate responses, and task instructions or rubric, with slot assignment varied across batches.
  • B.2 Pointwise: Pointwise judging evaluates one assistant response against the user question and requires a score from 0 to 10 enclosed in <score> tags.
  • B.3 Task instructions: PICO instructions require extracting Population, Intervention, Comparison, and Outcome as short verbatim strings in a JSON object with keys P, I, C, and O.

C Detailed results

The detailed-results section organizes evaluation outputs across correctness, verdict variance, invalid-response share, and rank-gap difficulty. The tables distinguish decomposable and atomic families, summarize averages, and report difficulty-stratified results.

  • Figure 1 summarizes pairwise judging correctness, robustness, and compliance, while Tables 2, 3, and 4 provide exact per-suite values.
  • Table 2 reports correctness for decomposable and atomic families, individual benchmarks, and the unweighted average across all five suites.
  • Table 3 defines verdict variance as the mean |high −low| across datasets over majority-vote and any-/all-sample correctness rates, with lower values preferred.
  • Table 4 defines invalid-response share as invalid divided by invalid plus legitimate errors, with lower values preferred.
  • Table 5 reports rank-gap difficulty for PICO and MedCalc using per-example Hard, Medium, and Easy tertiles, where Hard contains the smallest gaps.
Loading 2608.29127v1…