Source-linked AI summary
When Do Supervised UQ Ensembles Improve LLM Hallucination Detection? A Robustness Study
Mohit Singh Chauhan, Vipin Gyanchandani, Dylan Bouchard
TL;DR
Closed-book hallucination detection lacks trusted inference-time references, and the robustness of learned UQ ensembles across deployment conditions has been limited. The paper trains supervised classifiers over heterogeneous UQ scorer outputs and evaluates them across models, datasets, sample sizes, transfer shifts, and generation regimes. Ensembles outperform the best individual scorer in most settings, while black-box-only combinations nearly match full ensembles and white-box-only combinations provide limited benefit.
Problem
The robustness of supervised UQ ensembles remains underexplored when labels are scarce, test distributions shift, and generation formats vary beyond short-form QA.
Method
The paper trains classifiers over heterogeneous black-box, white-box, and reflexive UQ scorer outputs for out-of-sample closed-book hallucination classification.
Results
30 of 32 settings favor supervised ensembles over the best individual scorer by AUROC, with gains from 100 labeled instances; transfer favors ensembles in 23 of 28 settings.
Takeaways & Limitations
Supervised ensembling is a low-cost default when a small labeled dataset is available, and black-box ensembles are a robust fallback when logprobs are unavailable.
Takeaways & Limitations
The evaluation covers narrow long-form and Python competitive-programming tasks, and cross-dataset transfer is within domains, limiting evidence for broader settings.
Abstract
from arXiv · showhide
Uncertainty quantification (UQ) methods are widely used for hallucination detection in large language models (LLMs) in closed-book settings where ground-truth evidence is unavailable at inference time. Prior work has proposed combining UQ signals via learned ensembles, but empirical investigations into the robustness of these ensembles are limited. We study a supervised ensembling framework that trains a classifier over heterogeneous UQ-based scorer outputs on a small, domain-specific dataset of labeled LLM responses, then applies it to out-of-sample hallucination classification without retrieval, tools, or reference documents. Across four LLMs, nine datasets, and three generation regimes (short-form QA, long-form generation, and code generation), we provide a systematic robustness analysis along three axes: sample efficiency, in-domain dataset transfer, and generation regime dependence. We find that supervised ensembles outperform the best individual scorer in 30 of 32 settings, with gains realized from as few as 100 labeled instances. Ensembles retain most of their advantage in cases of in-domain transfer under distribution shift, outperforming the best non-ensemble scorer in 23 of 28 transfer settings. Sampling-based black-box ensembles are nearly as effective as full ensembles, while single-generation white-box ensembles offer limited benefit.
1 Introduction
Closed-book hallucination detection is difficult because trusted references are often unavailable and performance varies across domains, prompts, and LLMs. This paper studies whether supervised ensembles of heterogeneous UQ scorers remain robust under limited labels, distribution shift, and varied generation regimes.
- Closed-book LLM deployments can produce fluent but incorrect statements, fabricated citations, or incorrect code behaviors.
- Reliable inference-time detection is difficult because systems often lack trusted external references and hallucination rates vary sharply across domains, prompts, and base LLMs.
- Prior UQ approaches include black-box, white-box, and reflexive signals that can often be computed without retrieval.
- Prior supervised ensembles showed gains only in controlled, in-domain short-form QA, leaving robustness across deployment conditions open.
- The study trains a classifier over heterogeneous UQ scorer outputs and evaluates sample efficiency, in-domain transfer, and generation-regime dependence.
- Supervised ensembles outperform the best individual scorer in 30 of 32 settings by AUROC and 29 of 32 by ECE, with gains from 100 labeled instances.
2 Related Work
Related work develops UQ signals for hallucination detection across access types and output formats, alongside both unsupervised and supervised ensemble approaches. The cited literature motivates testing supervised ensembles beyond short-form, in-distribution settings.
- UQ methods differ by access requirement, separating black-box text-output methods from white-box token-probability methods.
- Sampling-based consistency methods compare multiple responses using exact match, lexical or embedding similarity, or semantic-entropy clustering.
- The paper positions its study as extending supervised ensemble evaluation beyond short-form, in-distribution question answering.
- Long-form UQ methods decompose responses into claims and score them using entailment, graph centrality, or question-generation pipelines.
- Prior ensemble work includes unsupervised combinations of consistency and self-reflection signals, panels of LLM evaluators, and supervised weighted averages.
3 Methods
The method frames hallucination detection as binary classification from heterogeneous UQ confidence scores. A supervised classifier combines black-box, white-box, reflexive, and claim-level signals for response- or claim-level prediction.
- 3.1 Problem Formulation: Each response receives a K-dimensional vector of UQ confidence scores, with higher values indicating greater confidence in correctness.
- 3.1 Problem Formulation: The ensemble learns a function mapping scorer vectors to a single confidence score that separates hallucinated from correct outputs.
- 3.1 Problem Formulation: Long-form responses are classified at the claim level, with each extracted claim receiving its own scorer vector and hallucination label.
- UQ Scorers: Black-box scorers measure agreement between an original response and stochastic candidate responses using exact match, NLI, embedding, BERTScore, or semantic-entropy functions.
- UQ Scorers: White-box scorers derive confidence from token probabilities, including sequence probability, minimum token probability, probability margin, and token-level entropy.
- UQ Scorers: Reflexive scorers ask the generating or an external LLM to assess correctness, while claim-level graph scorers use entailment-graph centrality.
- Combination Strategies: The study compares logistic regression, random forest, gradient-boosted trees, and constrained weighted averaging, tuning hyperparameters by 5-fold cross-validation.
4 Experiments
Experiments evaluate supervised UQ ensembles across four LLMs, nine datasets, and short-form, long-form, and code-generation regimes using repeated stratified splits. Ensembles generally improve AUROC and calibration, transfer under distribution shift, and remain effective with black-box access alone.
- Setup: Experiments cover four LLMs from Google and OpenAI across math, factual QA, code, long-form, and reading-comprehension tasks.
- Setup: Labels come from reference-answer grading for short-form QA, test-case execution for code, and FactScore grading of claims for long-form QA.
- Setup: The evaluation uses 25 random stratified 70/30 splits, with shared splits across in-distribution, transfer, and access-constrained analyses.
- Sample Efficiency: Ensembles are trained on 0.1N–0.7N subsamples, evaluated on held-out folds, and compared with a test-set-selected best individual scorer that is an optimistic unavailable baseline.
- In-Distribution Performance: 30 of 32 settings show higher AUROC for the best ensemble than the best individual scorer, including every code-generation and long-form setting.
- Code Generation: Code generation shows ensemble gains for all four LLMs, including AUROC 0.89 vs. 0.87 for Gemini-2.5-Flash and 0.88 vs. 0.86 for GPT-4o.
- Long-Form QA: Long-form ensembles outperform the best individual scorer in all 8 settings, while simpler strategies typically converge by 0.1N–0.2N.
- Calibration: The best ensemble achieves the lowest ECE in 29 of 32 settings, with ECE never exceeding 0.06.
5 Discussion
Supervised UQ ensembles provide robust gains across datasets, access constraints, sample sizes, and generation regimes, although the strongest combination strategy depends on the setting.
- 30 of 32 settings show AUROC gains over the best individual scorer, while 29 of 32 improve calibration by ECE.The study spans four LLMs, nine datasets, and three generation regimes.
- Calibration remains strong: the best ensemble achieves ECE below 0.06 in every setting and below 0.05 in most.Well-calibrated scores support threshold-based deployment decisions without extensive per-dataset threshold tuning.
- The strongest combination strategy varies by regime: random forest and logistic regression lead short-form and code generation, while logistic regression and weighted averaging lead long-form claim detection.Gradient boosting can overfit and degrade with increasing training data in long-form detection.
- 100–200 labeled instances often suffice for logistic regression and weighted-average ensembles to plateau, whereas tree-based strategies typically require 300–500.Random forest may achieve higher final AUROC with larger labeling budgets.
- 23 of 28 transfer settings outperform the best individual scorer, with mean degradation of only 0.02 AUROC points.Underperforming cases are within confidence intervals rather than clearly attributable to a systematic failure mode.
- Black-box ensembles provide nearly the same benefit as full ensembles in 19 of 20 short-form settings, while white-box-only ensembles improve only 11 of 20.Black-box signal diversity includes exact match, NLI, embedding similarity, and semantic clustering; six white-box features provide less diversity.
6 Conclusion
The study evaluates supervised UQ ensembles for hallucination detection across sample efficiency, distribution shift, and generation regimes. It finds broad gains, effective transfer in most settings, and a practical advantage for black-box ensembles when token probabilities are unavailable.
- Supervised ensembles outperform the best individual scorer in 30 of 32 settings by AUROC and 29 of 32 by calibration.
- Ensemble gains appear with as few as 100 labeled instances, while cross-dataset transfer outperforms the best individual scorer in 23 of 28 settings.
- Black-box-only ensembles are nearly as effective as full ensembles, whereas white-box-only ensembles offer limited benefit.
Limitations
The evaluation is constrained to closed-source models and selected generation and transfer slices, limiting how broadly the findings can be generalized.
- The study evaluates four closed-source LLMs from only Google and OpenAI, so results may not generalize to open-weight models or substantially different architectures.Model-family differences cannot be fully characterized with only two providers.
- Long-form tests cover only two narrow factoid-recall domains, while code experiments use Python competitive-programming tasks and self-contained function synthesis.Broader summarization, drafting, dialogue, multilingual, longer-codebase, and other programming settings may behave differently.
- Transfer is tested within domains, while cross-domain and cross-LLM transfer remain unexplored and may show substantially larger degradation.
Ethical Considerations.
The paper frames hallucination detection as supporting safer LLM deployment while warning that detector confidence is not a guarantee of correctness. Human oversight remains essential in high-stakes applications.
- High-performing hallucination detectors can create a false sense of security when confidence scores are treated as guarantees.
- The methods reduce but do not eliminate hallucination risk.
- Human oversight remains essential in high-stakes settings such as clinical or legal applications.
Conflict of Interest
The paper discloses employment and equity relationships involving CVS Health® Corporation, while stating that no conflicts are germane to the work.
- The disclosures include current and former CVS Health® Corporation employment and stock or equity holdings.
- The authors state that no disclosed conflicts are germane to this work.
Disclaimer
The paper disclaims endorsement or affiliation with Google and OpenAI and describes the UQ scorer definitions, inputs, access requirements, and generation-regime adaptations. It also outlines scorer families, computational-cost accounting, and specialized handling for code and long-form generation.
- Disclaimer: The publication is independent and its prompts, Gemini, and GPT references do not imply endorsement, affiliation, authorization, or sponsorship.
- Scorer Definitions: All ensemble input scorers output values in [0, 1] without additional preprocessing or calibration.
- Scorer Families: White-box scorers use token probabilities, while black-box sampling scorers generate m candidate responses and compare them with the original using text-only signals.
- Hybrid Scorers: Hybrid scorers combine token probabilities with sampling-based consistency signals, including average length-normalized sequence probability and CoCoA.
- Code-Specific Scorers: Code-generation adaptations replace NLI semantic equivalence with LLM-based functional equivalence assessment, alongside CodeBLEU consistency.
- Long-Form Claim-Level Scorers: Long-form scoring decomposes responses into claims, scores each claim, and aggregates claim-level confidence, using graph-based centrality measures.
- Computational Cost: Computational costs are reported as additional original-LLM generations, auxiliary-LLM generations, and semantic comparisons.
D Grading Validation
The grading validation compares LLM-generated labels with two human annotators across 400 short-form responses. Agreement is near-perfect overall, remains strong across datasets, and shows no evidence of self-grading bias.
- Validation setup: 400 short-form responses were independently labeled by two human annotators and compared against Gemini-2.5-Flash grader labels.The sample covered five short-form datasets and two generator LLMs, GPT-4o and Gemini-2.5-Flash.
- Overall agreement: κ ≥0.93 across all three pairwise comparisons indicates near-perfect agreement among annotators and between the grader and annotators.The grader agrees with each annotator at least as strongly as the annotators agree with each other: κ = 0.97 and 0.93 versus 0.95.
- Agreement by dataset: κ ≥0.95 on math datasets, while HotpotQA has the lowest human agreement at κ = 0.85.On HotpotQA, the grader agrees with Annotator 1 at κ = 0.97, exceeding human-human agreement.
- Self-grading bias: κ = 0.97 versus 0.98 for Gemini-2.5-Flash and GPT-4o respectively shows comparable grader agreement across generator LLMs.Agreement with Annotator 2 is also comparable: κ = 0.92 versus 0.93.
E Hyperparameters
The ensemble classifiers use cross-validation and model-specific hyperparameter searches, with AUROC as the optimization target. The weighted-average ensemble imposes simplex-constrained weights, while the other approaches use standard classifier-specific search spaces.
- Selection procedure: 5-fold cross-validation on the training fold selects hyperparameters, optimizing AUROC across combination strategies.The weighted-average method uses uqlm, while the other three classifiers use scikit-learn.
- Weighted average: Weighted-average weights lie in [0, 1] and sum to 1, with Optuna using 1,000 trials per configuration.This constrains the ensemble to convex combinations of scorer outputs.
- Logistic regression: Logistic regression searches 18 elastic-net configurations over C ∈ {0.001, 0.01, 0.1, 1, 10, 100} and ℓ1 ratio ∈ {0, 0.5, 1}.The solver is saga with elastic-net regularization.
- Random forest: Random forest searches 96 configurations spanning estimator count, feature selection, depth, split size, and leaf size.The search includes n_estimators ∈ {200, 500}, max_features ∈ {sqrt, log2}, max_depth ∈ {4, 6, 8}, min_samples_split ∈ {2, 5}, and min_samples_leaf ∈ {1, 2}.
- Gradient boosted trees: Gradient boosted trees search 108 configurations over estimator count, learning rate, depth, split size, and subsampling.The search varies n_estimators ∈ {50, 100, 200}, learning_rate ∈ {0.01, 0.1, 0.2}, max_depth ∈ {3, 4, 5}, min_samples_split ∈ {2, 4}, and subsample ∈ {0.8, 1.0}.