Source-linked AI summary
Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
Pat Verga, Sebastian Hofstatter, Sophia Althammer, Yixuan Su, Aleksandra Piktus, Arkady Arkhangorodsky, Minjie Xu, Naomi White, Patrick Lewis
TL;DR
Evaluating free-form LLM generations remains difficult, and single large LLM judges can be costly and biased. The paper proposes pooling judgements from a panel of smaller, heterogeneous models and finds that PoLL outperforms a single large judge across the tested settings while costing over seven times less.
Problem
Free-form generations are difficult to evaluate because suitable test data and correctness assessment are challenging, while LLM judges can be biased, slow, and costly.
Method
PoLL evaluates generations by pooling scores from evaluator models drawn from different model families instead of using one large judge.
Results
PoLL correlates better with human judgements than GPT-4 while being over seven times cheaper, and reduces intra-model scoring bias.
Takeaways & Limitations
Smaller heterogeneous evaluator panels can provide effective, consistently strong evaluation without relying on a single best judge.
Takeaways & Limitations
The study covers only three evaluator settings and limited judges and panel compositions, leaving broader applicability and panel selection for future work.
Abstract
from arXiv · showhide
As Large Language Models (LLMs) have become more advanced, they have outpaced our abilities to accurately evaluate their quality. Not only is finding data to adequately probe particular model properties difficult, but evaluating the correctness of a model's freeform generation alone is a challenge. To address this, many evaluations now rely on using LLMs themselves as judges to score the quality of outputs from other LLMs. Evaluations most commonly use a single large model like GPT4. While this method has grown in popularity, it is costly, has been shown to introduce intramodel bias, and in this work, we find that very large models are often unnecessary. We propose instead to evaluate models using a Panel of LLm evaluators (PoLL). Across three distinct judge settings and spanning six different datasets, we find that using a PoLL composed of a larger number of smaller models outperforms a single large judge, exhibits less intra-model bias due to its composition of disjoint model families, and does so while being over seven times less expensive.
1 Introduction
Evaluating free-form generations is difficult because suitable test data and correctness assessment remain challenging, while heuristic metrics can misjudge intended properties. The paper proposes PoLL to address bias and cost issues in LLM-based evaluation.
- Motivation: Free-form generation evaluation requires both meaningful test data and reliable correctness assessment.Multiple-choice benchmarks avoid some evaluation difficulty but probe properties that differ from free-form generation.
- Motivation: Heuristic metrics such as exact match commonly produce false positives and false negatives in question answering.False negatives can arise from incomplete reference answers or formatting differences.
- Motivation: LLM evaluators can exhibit intramodel bias and are often costly and slow when using the largest models.Evaluators may prefer their own outputs, limiting the applicability and access of such evaluation.
- Approach: PoLL evaluates LLM generations with evaluators drawn from different model families rather than a single large judge.The experiments span three settings and six datasets.
- Findings: PoLL correlates better with human judgements than GPT-4 while costing over seven times less.The paper also reports high GPT-4 variance under minor prompt changes and reduced intra-model scoring bias with heterogeneous panels.
2 Methods
The paper distinguishes single-point, reference-based, and pair-wise LLM judging settings, then replaces a single judge with pooled scores from multiple evaluators. This pooling is intended to reduce intra-model bias and can use different voting functions.
- Judge settings: In single-point scoring, a judge rates one output independently using its internal model of quality.The score is defined as score = J(a).
- Judge settings: In reference-based scoring, the judge scores an output against a gold reference containing information that should be included.The score is defined as score = J(a, r).
- Judge settings: In pair-wise scoring, an evaluator compares two outputs and produces a preference score such as a > b, a ≈ b, or a < b.The setup is used when choosing which of two model outputs is better.
- Panel scoring: Pooling multiple evaluator models addresses the intra-model bias associated with relying on a single judge.The paper relates this strategy to pooling methods used to reduce variance in human annotations.
- Panel scoring: PoLL has each evaluator independently score an output, then pools those scores through a voting function.The final score is f(j ∈ P : j(a)), where P is the panel and f is the voting function.
3 Experimental Settings
The experiments construct a three-model PoLL from disparate model families and apply voting schemes suited to binary QA or graded pair-wise judgments. They evaluate single-hop QA, multi-hop QA, and Chatbot Arena settings using model and human-reference comparisons.
- Evaluator panel: The PoLL uses Command R, Haiku, and GPT-3.5 from three disparate model families.The panel is designed from models in the Command R, Claude-3, and GPT families.
- Evaluator panel: QA uses max voting for binary correct-or-incorrect judgements, while Chatbot Arena uses average pooling for 1–5 scores.A three-judge panel often lacks a clear majority for graded Chatbot Arena scores.
- Question answering: The QA experiments cover KILT versions of Natural Questions, TriviaQA, and HotpotQA, with multi-hop experiments additionally using Bamboogle and HotpotQA.Open-book QA models retrieve evidence and generate answers evaluated against human-annotated references.
- Evaluation procedure: Max voting with average-voting fallback produced similar results and the same overall ranking.This comparison supports the robustness of the reported aggregation choice in the tested setting.
- Chatbot Arena: Chatbot Arena evaluates anonymized model pairs through user preferences, and Chatbot Arena Hard compares outputs against a baseline using an LLM judge.The study preserves the original pair-wise judge prompts and treats crowdsourced Arena annotations as ground truth.
- Evaluation procedure: QA judges use reference-based prompts with few-shot examples, while the study also tests prompt variations and containment exact match.The containment metric checks whether a normalized reference answer string appears inside the generated response.
4 Results
Across QA and Chatbot Arena evaluations, PoLL generally correlates more closely with human judgements than individual judges, while reducing bias and evaluation cost. Results also show that GPT-4’s agreement can vary substantially with prompting, whereas PoLL produces more consistent rankings and scores.
- 4.1 Correlation to Human Judgements: PoLL has the strongest overall correlation with human judgements across the evaluated single-hop QA tasks, while GPT-4 is among the weaker judges.Agreement is measured with Cohen’s κ on KILT datasets.
- 4.2 Rank Correlation on Chatbot Arena: PoLL rankings correlate best with Chatbot Arena ground-truth rankings, particularly at the top of the ranked list.The comparison uses Pearson and Kendall Tau correlations against crowd-sourced ELO rankings.
- 4.3 Judgement Variance by Prompt Changes: GPT-4’s agreement improves with in-context examples and an explicit instruction not to overthink, but remains below Command-R, Haiku, and PoLL.These prompt changes raise GPT-4 to GPT-3.5’s few-shot agreement level.
- 4.4 Judge Bias and Consistency: PoLL has the smallest spread in multi-hop QA score deviations, with a standard deviation of 2.2 compared with 6.1 for GPT-3.5.The deviations are measured relative to human annotator decisions on HotPotQA and Bamboogle.
- 4.2 Rank Correlation on Chatbot Arena: GPT-4 ranks another GPT-4 variant second instead of its ground-truth fourth-place position, illustrating intra-model preference.The rankings compare GPT-4 and PoLL judgements with original Chatbot Arena ELO rankings.
- 4 Results: Running the three-model PoLL costs seven to eight times less than running a single GPT-4 judge, depending on input-to-output token ratios.The reported prices are $1.25/input + $4.25/output for PoLL versus $10/input + $30/output for GPT-4 Turbo.
- 4 Results: The study did not run explicit latency evaluations, and speed can depend on model choice and serving platform.The authors state that parallel smaller models are generally faster than one large model, but do not provide a measured latency comparison.
5 Conclusions and Limitations
PoLL uses smaller evaluator models to evaluate LLM performance while reducing intra-model bias, latency, and cost. Its broad applicability and optimal panel composition remain open questions.
- PoLL composed of smaller models effectively evaluates LLM performance while reducing intra-model bias, latency, and cost.
- The study evaluates only three evaluator settings and a limited number of judges and panel compositions.
- Further work must assess PoLL in broader domains, including math and reasoning evaluations, and determine how to select panels for quality and cost.
A.1.1 Single-hop QA
Single-hop QA evaluation uses open-domain datasets, retrieved evidence, generated answers, and judge prompts tailored through human-labeled examples. Prompt design materially affects agreement, with few-shot prompting generally outperforming zero-shot evaluation.
- Datasets: KILT evaluates open-domain question answering through retrieval followed by answer generation using retrieved documents.
- Datasets: The single-hop evaluation covers Natural Questions, TriviaQA, and HotpotQA, which differ in question source and complexity.
- Judge Prompts: Judgement prompts are created separately for each KILT dataset using randomly sampled human-labeled examples satisfying positive, negative, and exact-match-disagreement constraints.
- Judge Prompts: Prompt selection sweeps over n shots in {8, 10, 16} and chooses the prompt with strongest average agreement across judges on held-out human labels.
A.1.2 Multihop
The multihop evaluation studies adversarial and evidence-based question answering, using tool-using agents and judge prompts to assess generated answers.
- Datasets: Bamboogle contains 125 adversarial questions requiring multiple internet searches, while HotpotQA uses an accompanying Wikipedia dump.
- Answer Generation: Multihop answer generation follows a REACT Thought-Action-Observation loop in which agents iteratively call tools and reason over their outputs.
- Judge Prompt: The multihop experiments use a dedicated judge prompt documented in Table 10.
A.2.1 Multi-hop QA
The multi-hop QA appendix reports evaluator correlations, Arena-Hard scores, prompts, and annotation procedures across benchmark datasets. These materials document how judgments and human-reference comparisons were constructed.
- Results: Kappa correlations for chatbot models are reported separately on Bamboogle and multi-hop HotpotQA.
- Results: Arena-Hard scores are produced by PoLL using average pooling.
- Annotations: Professional annotators with diverse demographic and professional backgrounds supplied the human judgments used in the experiments.
- Annotations: Triple-annotated datasets use pooled super-annotator decisions based on majority voting, while other listed datasets are singly annotated.