Source-linked AI summary
Robust Conformal Consensus: Multi-Agent LLM-as-a-Judge Interval Evaluation with Conformal Prediction
Lihui Liu
TL;DR
LLM-as-a-Judge evaluation can be unreliable when uncertainty is estimated from a single judge, because different LLM evaluators may assign different scores. This paper adapts conformal prediction to aggregate intervals from multiple judges, and reports more stable intervals with coverage guarantees.
Problem
Single-judge uncertainty estimates overlook variability across LLM evaluators, limiting their representation of evaluation uncertainty.
Method
The framework constructs and aggregates conformal prediction intervals from multiple LLM judges for rating-based evaluations.
Results
Multi-LLM intervals produce more stable uncertainty estimates and maintain strong coverage guarantees across experiments.
Takeaways & Limitations
The findings support multi-LLM, uncertainty-aware evaluation for more robust assessments.
Takeaways & Limitations
The evaluation is limited by training data that primarily covers specific domains rather than broad topic generalization.
Abstract
from arXiv · showhide
LLM-as-a-Judge has emerged as a promising paradigm for evaluating natural language generation. However, the uncertainty associated with such evaluations remains largely unexplored, which limits their reliability in real-world applications. Although conformal prediction offers a principled framework for uncertainty quantification, existing approaches typically apply it to a single LLM judge, overlooking the variability introduced by using different LLM evaluators. In this work, we propose a robust uncertainty estimation framework for multi-agent LLM-as-a-Judge evaluation. Our approach constructs conformal prediction intervals for LLM-based scores from multiple LLMs. By considering intervals from different LLM judges, we obtain more stable and reliable uncertainty estimates. Extensive experiments demonstrate that our method produces valid prediction intervals with coverage guarantees, and that interval-based aggregation across multiple judges leads to more stable evaluation outcomes.
1 Introduction
LLM-as-a-Judge offers flexible, scalable NLG evaluation, but single-judge scores can be unstable and model-dependent. The paper introduces multi-LLM conformal intervals to capture inter-LLM variability and produce more stable uncertainty estimates.
- Motivation: LLM-as-a-Judge provides flexible, scalable evaluation for NLG across diverse domains.Prior studies report agreement with human assessments and competitive performance on ROUGE, BLEU, and BERTScore.
- Motivation: Single-LLM evaluations can be biased or unstable because LLM outputs are inherently random.This unreliability is especially concerning in high-stakes settings such as healthcare and finance.
- Research gap: Single-judge conformal intervals may overlook disagreement caused by differences among LLMs, making uncertainty estimates model-dependent.Different training data, architectures, alignment strategies, and inductive biases can produce different scores for the same input.
- Approach: The proposed framework recalibrates conformal prediction intervals across multiple LLM judges to account for inter-LLM variability.It is applied to nine conformal prediction techniques for rating-based LLM-as-a-Judge evaluations.
- Findings: Experiments show that multi-LLM evaluation yields more stable uncertainty estimates than using a single LLM.The analysis evaluates interval efficiency through average width and validity through coverage of the true rating.
2 Preliminaries
The paper models LLM-generated ratings and uses split conformal prediction to construct statistically guaranteed intervals. The procedure is post-hoc and model-agnostic, with coverage depending on calibration and exchangeability assumptions.
- LLM-as-a-Judge: In rating-based LLM evaluation, a judge assigns a numerical score y0 to candidate text x using a predefined scale.The paper describes scales such as Likert ratings and derives scores from rating-token logits.
- LLM-as-a-Judge: LLM judges can also evaluate pairwise comparisons or rankings by scoring multiple candidate outputs.Relative judgments are derived from the resulting scores.
- Conformal Prediction: Conformal prediction replaces a point estimate with a prediction interval that is guaranteed to contain the true value at a prespecified probability.The method is model-agnostic, works post-hoc, and does not require assumptions about the data distribution.
- Split Conformal Prediction: Split conformal prediction uses a separate calibration set to compute non-conformity scores measuring prediction error.Here, the predicted score is derived from model logits as ˆy = f(z).
- Split Conformal Prediction: The calibration quantile defines an interval for new inputs that contains the true score at least 1 −α of the time under exchangeability.The stated coverage guarantee includes a finite-sample correction term.
3 Methodology
The methodology addresses variability across LLM judges by constructing, aggregating, and recalibrating conformal prediction intervals for rating-based evaluations. Weighted consensus and meta-calibration target a balance between efficiency and formal coverage.
- Motivation: Single-agent conformal prediction can overlook prompt, sampling, and model variability, making uncertainty estimates agent-dependent.Different judges may assign different ratings to identical content, so one-agent intervals may not capture evaluation-process variability.
- Multi-judge conformal evaluation: Multiple judges independently produce ratings and conformal intervals, which are then aggregated into an uncertainty-aware evaluation.The framework targets discrete rating tasks such as Likert scales and treats judge-level outputs as sources of model variability.
- Judge-level intervals: The framework applies regression-based and ordinal conformal methods to generate intervals while respecting the ordered structure of Likert ratings.The methods include CQR variants, CHR, LVD, Boosted methods, R2CCP, Ordinal APS, and Ordinal Risk Control.
- Weighted consensus: Weighted Conformal Consensus assigns greater influence to judges with stronger historical reliability and calibration, producing a consensus interval that balances coverage and efficiency.The weighted interval is formed from individual interval endpoints, while less reliable or outlier-producing agents receive less influence.
- Meta-calibration: Meta-calibration restores formal validity by calibrating the weighted consensus interval on an independent meta-calibration set using meta-nonconformity scores.The resulting final interval is designed to satisfy the desired marginal coverage guarantee.
- Coverage guarantee: Theoretical analysis states that the final WCC interval contains the true rating with probability at least 1 −α under the stated construction.The guarantee uses fixed pre-computed weights and a held-out meta-calibration set.
4 Experiments
Experiments evaluate multi-judge conformal aggregation across summarization, dialogue summarization, and reasoning benchmarks using repeated randomized splits and multiple LLM judges. Results show a coverage–width trade-off, improved robustness from aggregation, and further coverage gains after boundary adjustment.
- Experimental setup: Experiments span summarization, dialogue summarization, and reasoning benchmarks, with human Likert-scale annotations supplying ground-truth labels.SummEval and Dial-Summ use four evaluation dimensions, while reasoning tasks are evaluated with SocREval.
- Experimental setup: A three-way split reserves 40% for judge calibration, 10% for held-out meta-calibration, and 50% for testing.Results average empirical coverage and mean interval width over 30 random seeds.
- Coverage and efficiency: CQR and Asym CQR generally reach the target 90% coverage but produce the widest intervals, whereas CHR and Boosted CQR often fall below 85% with narrower intervals.This demonstrates the reported trade-off between interval width and empirical coverage.
- Coverage and efficiency: On ROSCOE reasoning tasks, CQR and Asym CQR often exceed interval widths of 3.8 while maintaining high coverage, whereas LVD and R2CCP provide a more moderate width–coverage balance.LVD and R2CCP sometimes achieve the smallest interval width while retaining at least 90% coverage.
- Multi-agent robustness: For ROSCOE/CosmosQA with CHR, the Multi-Agent judge reaches 74.0% coverage, compared with 72.65% for Qwen2.5.The authors characterize multi-agent aggregation as more robust against the lowest coverage failures of single models.
- Boundary adjustment: Boundary adjustment consistently improves coverage, including an increase from 85.96% to 95.53% for LVD on e-SNLI with Qwen2.5-72B-Instruct.Multi-LLM aggregation generally preserves or improves coverage while maintaining relatively small interval widths.
5 Related work
Conformal prediction provides distribution-free uncertainty quantification, but prior LLM-as-a-Judge uncertainty methods remain limited and often rely on single-model or costly signals. Existing work has focused mainly on classification-style tasks, leaving robustness in complex decision-making settings underexplored.
- Conformal prediction provides finite-sample marginal coverage guarantees without assumptions about the underlying data distribution.
- Coverage failures under adversarial inputs and uncertainty quantification in complex decision-making remain underexplored areas.
- Recent LLM uncertainty methods use token probabilities, self-reported confidence, or consistency across generations, but may suffer from bias, instability, overconfidence, or high computational cost.
- Conformal prediction has attracted attention as a post-hoc uncertainty tool for LLMs because of its distribution-free guarantees.
- Most existing conformal prediction work targets classification-style tasks that construct unordered prediction sets containing the correct answer with high probability.
6 Conclusion
The paper introduces a robust multi-LLM interval adaptation framework that integrates with conformal prediction and accounts for variability among judges. Experiments across multiple LLMs and conformal techniques report more stable, robust intervals while maintaining strong coverage guarantees.
- The framework moves beyond single-LLM uncertainty estimates by explicitly accounting for inter-LLM variability in rating-based evaluation.
- Experiments across multiple LLMs and conformal techniques show that incorporating multiple judges produces more stable prediction intervals with improved robustness.
- The framework maintains strong coverage guarantees while integrating with existing conformal prediction methods.
7 Ethical Considerations
The paper reports no significant ethical concerns and emphasizes usability, implementation simplicity, transparency, collaboration, and reproducibility through open-source data.
- The authors report no significant ethical concerns associated with the work.
- The framework emphasizes usability and ease of implementation to lower adoption barriers and minimize operational complexity.
- Use of an open-source dataset supports transparency, collaboration, and accessible, reproducible data.
8 Limitations
The paper identifies limited training-data scope and incomplete knowledge-graph coverage as constraints on generalization and reasoning beyond the predefined domains.
- Training data primarily covers specific domains, which may limit performance on queries beyond those predefined areas.
- The knowledge graph is inherently incomplete, with insufficient data or relational links in some regions potentially creating gaps in reasoning and inference.