PaperlayerView exact revision on arXiv

Source-linked AI summary

Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees

Sher Badshah, Ali Emami, Hassan Sajjad

arXiv:2608.17994v1cs.CL

TL;DR

Reference-free LLM judging remains difficult for objective factual evaluation because judges may lack reliable evidence or require costly tool augmentation. This paper introduces uncertainty-based routing between parametric and retrieval-augmented judging with calibrated risk guarantees, maintaining the target error rate while increasing coverage over single-mode baselines.

  • Problem

    Extending LLM-as-a-judge from subjective evaluation to reliable objective, factual evaluation without explicit references remains an open challenge.

  • Method

    The framework calibrates two uncertainty thresholds, accepts confident parametric verdicts, routes uncertain cases to web retrieval, and abstains when uncertainty persists.

  • Results

    Observed FDR remains at or below the specified target across tested configurations, while adaptive retrieval achieves higher coverage than single-mode abstention.

  • Takeaways & Limitations

    The finite-sample FDR guarantee remains valid under two-threshold routing without additional distributional assumptions.

  • Takeaways & Limitations

    The framework does not implement human review for instances that remain uncertain after retrieval.

Abstract

from arXiv · show

Using LLMs as judges has become standard practice for evaluating model outputs at scale. This is particularly common for subjective, open-ended tasks such as assessing helpfulness or alignment, where no single reference answer exists. However, objective tasks introduce a distinct reliability challenge for reference-free LLM judging. In the absence of a reference answer, the judge evaluates factual correctness either through its parametric knowledge or through tool augmentation. Although the former enables efficient evaluation, the judge may hallucinate or lack sufficient evidence for its verdict. Conversely, tool augmentation can provide additional evidence but introduces extra computational cost and requires an appropriate mechanism to determine when and how that evidence should be used reliably. More importantly, neither approach alone provides formal control over the risk of accepted verdicts or guarantees their reliability at a specified level. We propose a risk-controlled framework that calibrates uncertainty thresholds on a held-out set so that the false discovery rate among accepted verdicts remains below a user-specified level~$α$ with high probability, using finite-sample Clopper--Pearson intervals. When the parametric mode is not sufficiently confident, the instance is routed to a retrieval-augmented mode, where the judge gathers web evidence and re-evaluates the instance under a second calibrated threshold. The finite-sample guarantee carries over to this two-threshold routing without additional assumptions. Across open-domain QA benchmarks and judges of varying scales, the framework maintains the target error rate while achieving substantially higher coverage than single-mode baselines.

1 Introduction

The paper targets reliable reference-free LLM judging for factual QA, where missing knowledge and hallucinations can produce silent, confident errors. It proposes uncertainty-guarded routing between parametric judging, web retrieval, and abstention with formal risk control and improved coverage.

  • Motivation: LLM-as-a-judge scales output evaluation far more cheaply and quickly than human annotation, but existing applications primarily address subjective criteria without single reference answers.This motivates extending LLM judging toward factual correctness, where answers are objectively correct or incorrect.
  • Problem: Factual and open-domain QA require reliable correctness judgments, yet ground-truth labels are costly, expertise-dependent, or unavailable for autonomous real-world queries.Reference-free judging is therefore attractive, but it must operate without pre-collected annotations.
  • Problem: Parametric judges may lack relevant knowledge or hallucinate, producing confident but incorrect verdicts without signaling their unreliability.The problem is especially acute for recent events, rare entities, specialized domains, and plausible fabrications.
  • Method: The proposed framework accepts confident parametric verdicts, routes uncertain cases to web retrieval, and abstains when uncertainty remains.This two-mode design uses external evidence to address parametric knowledge gaps before resorting to abstention.
  • Guarantees and results: The framework calibrates selective evaluation risk with Clopper–Pearson bounds, extends the required Bernoulli structure to two-threshold routing, and improves coverage while controlling error across QA benchmarks and judge scales.The experiments compare adaptive retrieval against single-mode baselines.

2 Related Work

Related work spans LLM-as-a-Judge evaluation, uncertainty quantification, uncertainty-based routing and tool use, and risk-controlled selective evaluation. This framework combines calibrated uncertainty-based retrieval with formal FDR guarantees for accepted verdicts.

  • LLM-as-a-Judge: LLM judges provide a cost-effective alternative to human annotation and have expanded from subjective preference evaluation to objective factual evaluation with web search.Strong LLMs can approximate human preferences, supporting reward modeling and reinforcement learning; SAFE uses an LLM agent with web search for factual evaluation.
  • Uncertainty quantification for LLM evaluation: Uncertainty quantification methods estimate judge reliability either from token probabilities or from repeated-query consistency.White-box methods commonly use predictive entropy, while black-box methods repeatedly query the judge and measure verdict consistency.
  • Uncertainty-based routing and tool use: Prior routing methods use confidence to allocate queries across models or trigger retrieval, trading off cost, capability, and access to external information.FrugalGPT and AutoMix route across models with different capabilities, whereas FLARE uses token-level uncertainty to decide when to retrieve external information during generation.
  • Risk-controlled selective evaluation: Conformal prediction and conformal risk control provide finite-sample guarantees for prediction coverage and user-specified loss bounds, including FDR.These guarantees are calibrated on held-out data and have been applied to question answering, including SConU’s prediction sets.
  • Risk-controlled selective evaluation: The proposed framework selectively activates retrieval using calibrated uncertainty thresholds and provides formal FDR guarantees for accepted verdicts.It differs from unconditional retrieval by making retrieval conditional on calibrated uncertainty.

3 Methodology

The methodology calibrates uncertainty thresholds to control the error rate among accepted LLM-judge verdicts at a user-specified risk level. It extends this guarantee to retrieval-augmented routing, invoking web evidence only when direct judging is uncertain.

  • Risk-controlled selective evaluation: The framework calibrates uncertainty thresholds on held-out human-labeled data so accepted verdicts have error probability at most α.The guarantee holds with high confidence 1−δ over the calibration data.
  • Risk-controlled selective evaluation: Selective evaluation outputs a verdict only when uncertainty U ≤ t, otherwise abstaining to control the false discovery rate among evaluated instances.Better separation of correct and incorrect verdicts increases coverage at a fixed risk level.
  • Threshold calibration: Clopper–Pearson one-sided upper confidence bounds provide rigorous finite-sample risk control, and the largest threshold with UCB below α maximizes coverage.The bound applies because selected failure indicators are i.i.d. Bernoulli under i.i.d. calibration data and deterministic selection.
  • Retrieval-augmented routing: When direct judging is uncertain, the system retrieves top-k web results, appends their evidence to the prompt, and re-evaluates the instance.Confident direct-mode instances incur zero retrieval cost.
  • Guarantee under routing: The two-mode routing guarantee follows because selection and correctness are deterministic functions of each instance and fixed thresholds, preserving the i.i.d. Bernoulli structure.Thus any threshold pair with UCB ≤ α controls error at α with confidence 1−δ.

4 Experiments

The experiments evaluate the uncertainty-guarded judging framework across four open-domain QA benchmarks, multiple candidate and judge models, and calibrated acceptance, retrieval, and abstention mechanisms. They measure error control, coverage, routing, and retrieval cost under repeated calibration/test splits.

  • Datasets and models: Experiments sample 2,000 instances from each of TriviaQA, Natural Questions, HotpotQA, and PopQA using seed 42.Candidate models are Qwen3-8B and LLaMA-3.1-70B; judges include Qwen3-4B, Qwen3-8B, Qwen3-14B, and LLaMA-3.1-8B-Instruct.
  • Admission functions: Ground-truth correctness uses Exact Match, Token F1 with threshold ≥0.5, or a reference-based Qwen2.5-7B-Instruct LLM evaluator.Each judge produces a binary True/False verdict and brief explanation.
  • Test-time evaluation: The online procedure accepts confident Mode 1 verdicts, retrieves web evidence for uncertain cases, accepts sufficiently confident Mode 2 verdicts, and otherwise abstains for human review.Mode 2 retrieves the top k=3 search results by default, concatenates them, and appends them to the judge prompt.
  • Cost analysis: At α = 0.20 with Qwen3-14B, Mode 1 accepts p ≈0.45 of TriviaQA instances, so nearly half incur zero retrieval cost.On harder benchmarks, p decreases, but retrieval remains avoided for the most confident instances.
  • Calibration protocol: Evaluation reports FDR, Coverage, and Routing distribution across Mode 1, Mode 2, and abstention under 100 random 50/50 calibration/test splits.The protocol uses the Clopper–Pearson exact binomial bound, sets δ=0.05, and evaluates α ∈{0.05, 0.10, 0.15, 0.20, 0.25}.

5 Results

The framework empirically maintains FDR at or below the target α across judges, candidates, and datasets, while adaptive retrieval substantially recovers coverage lost to parametric abstention. Retrieval-based acceptance remains risk-controlled, including after retrieval drift.

  • FDR control: FDR stays at or below the target α in every candidate–judge–dataset configuration, satisfying the finite-sample guarantee.All four judges provide valid control, with narrow ±0.01–0.02 confidence bands across 100 random splits.
  • Coverage: Coverage scales with α and depends on dataset difficulty and judge capability.At α=0.20 for the Qwen-8B candidate, Qwen-14B and Llama-8B surpass 80% on NQ-Open and HotpotQA, whereas Qwen-4B remains below 15%.
  • Adaptive retrieval: Retrieval is the dominant acceptance mechanism, converting many Mode 1 abstentions into accepted verdicts after uncertainty falls below the second threshold.At α=0.20 on TriviaQA, Qwen3-14B resolves 45% parametrically versus 9% for Qwen3-4B; on harder tasks, judges rely predominantly on retrieval.
  • Adaptive retrieval: 14–29% of instances become more uncertain after retrieval and are routed to abstention, so only retrieval verdicts meeting the desired risk level are accepted.The second calibrated threshold guards against accepting retrieval-based verdicts whose confidence does not improve.
  • Adaptive retrieval: At α=0.20, adaptive retrieval raises Qwen3-8B coverage on NQ-Open from 7% to 82% and Llama-8B coverage on HotpotQA from 40% to 85%.On TriviaQA, retrieval provides a modest but consistent gain for strong judges, while Qwen3-4B benefits substantially on PopQA.
  • Retrieval drift: After roughly three months of retrieval drift, empirical FDR remains below α and coverage remains largely stable when original calibrated thresholds are redeployed.The evaluation re-queried the same 2,000 questions on TriviaQA and HotpotQA over 100 random 50/50 splits.

6 Conclusion

The paper introduces a risk-controlled selective evaluation framework for LLM-as-a-judge that provides finite-sample reliability guarantees while improving coverage through calibrated two-mode routing. Its Clopper–Pearson FDR guarantee remains valid for the joint two-threshold policy without additional distributional assumptions.

  • The framework provides finite-sample guarantees on the reliability of accepted LLM-judge verdicts.It is designed as a risk-controlled selective evaluation framework.
  • A calibrated two-mode policy routes uncertain instances to a second evaluation mode and achieves higher coverage than single-mode abstention.
  • The Clopper–Pearson FDR guarantee remains valid under the joint two-threshold policy without additional distributional assumptions.

A Proofs … B.1 Notation

The proofs establish Clopper–Pearson upper-confidence validity and the Bernoulli failure structure underlying two-mode routing. The guarantee depends on stable conditional Mode 2 error rates despite retrieval drift, while notation is summarized by a symbol reference.

  • A Proofs: The observed failure count among selected calibration instances follows a Binomial distribution with parameter R(t), enabling Clopper–Pearson upper-confidence analysis.The empirical failure rate is defined as the observed failure count divided by the number of selected instances.
  • A.1 Clopper–Pearson UCB Validity: The Clopper–Pearson upper bound is defined through the supremum of rates whose empirical-rate CDF remains at least δ.Monotonicity of the CDF in the true rate and super-uniformity yield the stated coverage argument.
  • A.2 Bernoulli Structure under Two-Mode Routing: Under fixed thresholds, routing and retrieval are deterministic functions of each input and judge output, making the selection indicator deterministic.This establishes the structure needed to analyze selected instances under two-mode routing.
  • A.2 Bernoulli Structure under Two-Mode Routing: Selected instances are independent and identically distributed after conditioning on selection, with the conditional distribution defined as D_t1,t2.The selection events depend on disjoint instances, and each selected instance follows the same conditional distribution.
  • A.2 Bernoulli Structure under Two-Mode Routing: Failure indicators over the selected subset are i.i.d. Bernoulli(R(t1, t2)), where R(t1, t2) is the expected failure rate conditional on selection.The two routing indicators are mutually exclusive, so the selection indicator is binary.
  • A.2 Bernoulli Structure under Two-Mode Routing: The guarantee is tied to the retrieval snapshot used during calibration and transfers under retrieval drift only if the conditional error rate of Mode 2 remains stable.The paper evaluates this retrieval-drift scenario by re-querying the web in Section 5.5.
  • B.1 Notation: Table 4 provides a complete reference of all symbols used in the paper.This notation reference appears in the experimental-details material.

B.2 Implementation … C.2.1 Coverage Curves

The appendix specifies the implementation, prompt templates, evaluator, and additional results for a two-mode judge that combines calibrated uncertainty with retrieval. Retrieval improves raw verdict accuracy, while the joint framework reports risk-controlled coverage across evaluated configurations.

  • B.2 Implementation: Candidate answers use greedy decoding for Qwen3-8B and LLaMA-3.1-70B, with Qwen3-8B thinking mode disabled to preserve generation budget.Predictive entropy requires only one forward pass at the verdict token and adds no inference beyond greedy decoding.
  • B.2 Implementation: Calibration uses 100 random 50/50 splits and grid-searches threshold pairs subject to a Clopper–Pearson upper-confidence constraint.Cumulative sums reduce grid-search complexity to O(|T1| · |T2|).
  • B.3 Prompt Templates: Mode 1 prompts the judge to assess candidate correctness using its own knowledge, while Mode 2 supplies formatted web search results for evidence-based evaluation.Retrieval queries Serper for the original question, takes the top-k results, and appends their titles, snippets, and URLs to the Mode 2 prompt.
  • B.4 LLM Evaluator Prompt: The Qwen2.5-7B-Instruct evaluator determines semantic equivalence between candidate and reference answers, producing the admission-based ground-truth label e*.Correctness allows matching meaning despite added explanation, synonyms, abbreviations, or alternate names.
  • C Additional Results: The appendix adds raw judge accuracy, joint-framework analyses across risk levels, and per-dataset coverage and FDR curves.These results complement the main paper’s evaluation.
  • C.1 Judge Verdict Accuracy: Retrieval consistently improves pre-calibration judge accuracy across configurations, with gains on PopQA reaching up to 44 percentage points.Table 5 compares parametric Mode 1 with retrieval-augmented Mode 2 using k = 3.
  • C.2 FDR and Coverage Across Risk Levels: Table 6 reports FDR and coverage for the calibrated two-mode routing framework across all evaluated α levels, datasets, and judges.The evaluation covers the Qwen-8B candidate, four datasets, and configurations involving the LLM evaluator, PE, and k=3 retrieval.
  • C.2.1 Coverage Curves: Figure 3 plots coverage as a function of α across all 32 configurations.The curves summarize coverage behavior across the evaluated framework settings.

C.2.2 Per-Dataset FDR

This section examines per-dataset false discovery rates through Figures 4–7 and reports full joint two-mode results across four datasets and all α levels.

  • Figures 4–7 provide per-dataset FDR plots for detailed examination.
  • Table 6 reports full joint two-mode framework results across all α levels and four datasets for the Qwen3-8B LLM evaluator with PE and k=3.

D Ablations and Robustness

Ablation and robustness analyses vary pipeline design, calibration, uncertainty scoring, and the no-retrieval baseline while preserving the finite-sample FDR guarantee. Conservative choices reduce coverage rather than compromising risk control.

  • D Ablations and Robustness: The framework preserves its finite-sample FDR guarantee across pipeline design, calibration procedure, uncertainty score, and no-retrieval baseline choices.These ablations test several dimensions of the framework while retaining formal risk control.
  • D Ablations and Robustness: Conservative design choices reduce coverage rather than compromising risk control.The robustness analysis identifies lower coverage as the cost of conservative choices.

D.1 Sensitivity Analyses … D.4 Conservative Joint Calibration

Sensitivity analyses show that FDR control is robust across admission functions, retrieval depths, confidence levels, calibration-set sizes, and uncertainty scores, while coverage varies substantially. Joint retrieval improves coverage on knowledge-intensive tasks, whereas conservative joint calibration preserves validity but can reduce attainable coverage.

  • D.1 Sensitivity Analyses: At α = 0.15 on NQ-Open, the LLM evaluator reaches 56% coverage while exact match and token F1 remain at 0%.String-matching criteria penalize semantically correct but surface-different answers, inflating apparent error rates; FDR control holds under all three criteria.
  • D.1 Sensitivity Analyses: k=3 already provides sufficient evidence, with coverage and FDR differences within 1–2 percentage points across retrieval depths.The ablation compares k ∈{3, 6, 9} for a Llama-70B candidate and Qwen3-8B judge.
  • D.1 Sensitivity Analyses: Tightening δ from 0.05 to 0.01 trades coverage for confidence, especially at low α and on the harder dataset, while FDR remains controlled.FDR stays controlled under both confidence-level settings.
  • D.1 Sensitivity Analyses: Coverage remains stable on TriviaQA but degrades gracefully on HotpotQA as the calibration fraction shrinks, with FDR controlled at every size.The calibration sweep uses fractions of the 2,000 items per dataset.
  • D.2 Alternative Uncertainty Estimators: LoRA+Prompt raises coverage from 0.04 to 0.44 on HotpotQA at α=0.10 and from 0.00 to 0.71 on PopQA at α=0.05.All three uncertainty scores satisfy the target FDR level across every setting; the Probe improves over PE at NQ-Open α=0.15 from 0.24 to 0.42.
  • D.3 Baseline Comparison: At α=0.20 on HotpotQA, the joint policy reaches 0.74 coverage versus at most 0.07 for no-retrieval controls.Direct, Self-eval, and Empty controls all retain FDR control, but the joint policy dominates nearly every operating point.
  • D.4 Conservative Joint Calibration: Conservative joint calibration corrects the confidence level to δ′ = δ/(|T1| |T2|), yielding simultaneous validity for the selected threshold pair.For data-dependent grids, replacing |T1| |T2| with the data-independent bound N2 preserves the guarantee at negligible cost.
  • D.4 Conservative Joint Calibration: The conservative correction is nearly free on TriviaQA and PopQA but costs substantial coverage on NQ-Open and HotpotQA, while FDR remains controlled under both variants.Bonferroni is loose because adjacent grid thresholds select almost the same instances; attainable jointly-valid coverage lies between conservative and pointwise rows, closer to pointwise figures.

E Analysis … E.3 Failure Modes

The analysis shows that retrieval usually reduces judge uncertainty and can correct confident hallucinations, but it also introduces residual errors from retrieval, reasoning, and answer-admission failures. These limitations motivate abstention for cases that remain unreliable.

  • E Analysis: The analysis examines retrieval’s effects on uncertainty, verdict correction, and residual failure modes beyond aggregate metrics.It focuses on how retrieval changes judge uncertainty, fixes incorrect verdicts, and leaves some errors unresolved.
  • E.1 Effect of Retrieval on Uncertainty: On 14–29% of items, retrieval raises uncertainty rather than lowering it, especially on multi-hop HotpotQA and NQ-Open.Partially relevant snippets and source disagreement concentrate these increases.
  • E.1 Effect of Retrieval on Uncertainty: Retrieval reduces uncertainty on most items, but its effect varies by dataset and instance.Table 14 measures raised and lowered fractions alongside the mean signed change ∆U = U2 − U1.
  • E.2 Qualitative Examples: Mode 2 corrects representative verdicts that Mode 1 got wrong after confidently hallucinating.Retrieved evidence grounds the judge in verifiable facts.
  • E.3 Failure Modes: Residual failures include off-topic retrieval, insufficient evidence, judge reasoning errors, and admission-function disagreement.These arise when queries miss relevant evidence, snippets lack the needed fact, or the judge misinterprets available evidence.
  • E.2 Qualitative Examples: The correction examples use Qwen3-8B as candidate and Qwen3-14B as judge.The retrieved-evidence table reports k=3 results, with the first result directly containing the correct answer.
  • E.3 Failure Modes: The two-threshold policy sends many semantically close but admission-rejected cases to abstention rather than acceptance.This failure mode occurs when the candidate is close to the gold answer but the admission function scores it as different.
Loading 2608.17994v1…