PaperlayerView exact revision on arXiv

Source-linked AI summary

Semantic Uncertainty Quantification Needs Factual Equivalence

Joseph Hoche, Quentin Guimard, Gianni Franchi

arXiv:2609.34967v1cs.CLcs.AI

TL;DR

Semantic UQ measures agreement among generated answers, but off-the-shelf comparison operators do not reliably measure whether answers express the same fact for a question. The paper trains a contrastive, question-conditioned operator from synthetic LLM generations and integrates it into existing estimators. It improves 120 of 126 evaluation comparisons across 18 model–dataset combinations, with the conclusion that the operator is a limiting component.

  • Problem

    Off-the-shelf operators used in semantic UQ do not reliably measure factual equivalence between answers to the same question.

    Source: §1 · p. 2

  • Method

    A question-conditioned bi-encoder is trained contrastively on weakly supervised synthetic triplets derived from LLM generations and substituted for existing comparison operators.

    Source: §1 · p. 2, §4.2 · p. 6, §5 · p. 7

  • Results

    120 of 126 comparisons (95%) improved across 18 model–dataset combinations, raising mean AUROC from 0.68 to 0.76.

    Source: §1 · p. 3, §6.2 · p. 8

  • Takeaways & Limitations

    The uniform improvements support the paper’s conclusion that the shared comparison operator, rather than the aggregator, is the limiting component of semantic UQ.

    Source: §7 · p. 9

    The operator was trained on a small synthetic set with a single configuration and might benefit from more data and a finer-grained objective.

    Source: §7 · pp. 9–10

Abstract

from arXiv · show

Semantic uncertainty quantification for large language models rests on a common template: sample several answers, measure how much they agree, and treat disagreement as uncertainty. We first formalize this template as two separate roles: an operator that compares two answers, and an aggregator that combines all pairwise comparisons into a scalar. Existing methods differ almost entirely in how they aggregate, while taking the operator off the shelf, typically an NLI model or a generic sentence encoder. We show that this reliance on off-the-shelf operators is the primary bottleneck of semantic UQ: they do not accurately measure factual equivalence of multiple answers to the same question. We resolve this with a deliberately simple recipe: a single encoder trained contrastively to isolate the targeted fact, utilizing synthetic data generated by an LLM and dataset both disjoint from all evaluation settings. Integrating the resulting operator into existing methods improves performance on 120 of 126 evaluation settings (95%) spanning 18 model dataset combinations across language and vision-language models. The best variant reaches 0.76 mean AUROC against 0.68 for the strongest baseline, while replacing the quadratic cross-encoder comparisons of entailment-based operators with one encoder pass per answer. The uniformity of the improvement supports the view that the operator, not the aggregator, is the limiting factor. The same operator also improves single generation token-level estimators: the norm it assigns to each token measures how much that token bears on the answer, and reweighting token log-likelihoods accordingly sharpens the estimate.

1 INTRODUCTION

Semantic UQ addresses hallucinations by measuring agreement in meaning, but existing approaches inherit comparison operators that can mistake lexical overlap or verbosity for factual agreement. The paper treats the operator as the bottleneck, trains a question-conditioned bi-encoder, and reports broad improvements when it replaces existing operators.

1 INTRODUCTION

  • Token-level scores can mistake lexical diversity for uncertainty even when answers express the same fact, as with “Paris” and “It’s Paris.”The paper motivates semantic UQ as a way to evaluate generations in meaning space rather than vocabulary.Source: §1 · p. 1
  • Off-the-shelf operators miss factual equivalence: sentence encoders can overlook contradictions, while NLI cross-encoders can split equivalent answers and require O(N^2) pairwise inference.Internal hidden states can prioritize shared syntactic suffixes over factual denotation.Source: §1 · p. 2
  • A question-conditioned bi-encoder trained contrastively on predictive variability learns to isolate factual differences without semantic annotations or evaluation-set overlap.The operator serves as a drop-in replacement for existing aggregators.Source: §1 · p. 2, Figure 1 · p. 2
  • 120 of 126 evaluation comparisons (95%) improved, raising mean AUROC from 0.68 to 0.76 across 18 model–dataset combinations spanning LLMs and LVLMs.The learned operator improves existing semantic estimators while avoiding the O(N^2) scaling bottleneck of entailment baselines.Source: §1 · p. 3

2 RELATED WORK

Related work includes uncertainty methods that do not compare the meaning of multiple generations and semantic UQ methods that measure agreement across generations. This paper focuses on the comparison operator rather than proposing a new aggregation method.

2 RELATED WORK

3 THE OPERATOR BOTTLENECK IN SEMANTIC UQ

Semantic UQ methods share a comparison-then-aggregation structure, but inherited operators show failures on factual equivalence and generative text. The section diagnoses those failures and specifies the operator properties needed to address them.

3.1 FORMALIZING SEMANTIC UQ: AN OPERATOR FOLLOWED BY AN AGGREGATOR

  • Semantic UQ is formalized as an operator s that fills an N × N comparison matrix S, followed by an aggregator F that maps S to uncertainty U = F(S).Prior work predominantly designs F, while inheriting operators such as NLI cross-encoders or cosine similarity over internal or external representations.Source: §3.1 · pp. 3–4

3.2 DIAGNOSTIC BENCHMARK: HOW EXISTING OPERATORS FAIL

  • External sentence encoders scored contradictory answers with a changed key fact at 0.91 mean cosine similarity; internal representations scored contradictory answers sharing a suffix at 0.87.The diagnostic benchmark uses 1,000 pairs drawn from sampled and greedy generations of the same pipeline as the training data.Source: §3.2 · p. 4
  • NLI cross-encoders scored equivalent answers of different verbosity at 0.29 mean similarity and a generated sentence versus its ground-truth entity at 0.27.The cited diagnostics attribute these failures to directional entailment and mismatch between human-annotated text and LLM generations.Source: §3.2 · p. 4

3 THE OPERATOR BOTTLENECK IN SEMANTIC UQ

  • The section turns these diagnostic failures into three desiderata and proposes extracting the needed operator from the language model’s own generative distribution.It states that satisfying the desiderata requires abandoning off-the-shelf metric spaces and external NLP datasets.Source: §3.3 · pp. 4–5

3.3 DESIDERATA FOR AN IDEAL OPERATOR

  • An ideal operator must be invariant to generative syntax, measure question-conditioned factual equivalence rather than topical similarity or directional entailment, and scale with O(N) transformer passes.The figure summarizes mining predictive variability, contrastively separating hard negatives from paraphrases, and using the frozen encoder for comparison and token reweighting.Source: §3.3 · p. 4, Figure 2 · p. 5

4 LEARNING SEMANTIC GEOMETRY FROM PREDICTIVE VARIABILITY

The paper learns a question-conditioned comparison operator from an LLM’s predictive variability, using contrastive training to distinguish factual agreement from topical or lexical similarity. The resulting geometry is designed to scale to multiple generations without manual semantic-equivalence annotations.

4.1 THE ARCHITECTURE: QUESTION-CONDITIONED BI-ENCODING

  • The bi-encoder embeds each response together with its question, compares embeddings by inner product, and builds the pairwise matrix with N forward passes rather than O(N^2) cross-attention comparisons.Keeping the question in context anchors comparisons to the point at issue.Source: §4.1 · pp. 5–6

4.2 THE TRAINING DATA: WEAK SUPERVISION VIA PREDICTIVE VARIABILITY

  • Repeated sampling yields correct and incorrect answers that provide weak supervision for constructing positive and negative semantic pairs, without explicit equivalence annotations.The training examples are sampled from Llama-3.1-8B on NQ-OPEN and independently labeled against reference answers.Source: §4.2 · p. 6
  • The training setup assumes that all correct responses to one query share a semantic class, so correctness labels induce pairwise semantic relations.Two correct responses form a positive pair; a correct and an incorrect response form a negative pair.Source: §4.2 · p. 6

4.3 THE OBJECTIVE: CONTRASTIVE FACTUAL EQUIVALENCE

  • A contrastive triplet objective organizes embeddings around targeted factual agreement; same-question incorrect answers serve as hard negatives despite their shared topic and often similar syntax.The margin is m = 0.2, intended to separate factual contradictions while allowing syntactic variation and differing verbosity among positive pairs.Source: §4.3 · pp. 6–7
  • The frozen operator groups generations that agree on the same fact even when that fact is incorrect, matching the structure required by downstream semantic UQ aggregators.The pipeline needs correctness labels to construct triplets but no semantic-equivalence annotations or UQ confidence labels.Source: §4.3 · p. 7

5 A UNIVERSAL DROP-IN FOR SEMANTIC UQ

The learned comparison matrix can replace the operator in existing semantic UQ aggregators while leaving their aggregation procedures unchanged. Its embedding geometry also supplies token-level weights that can reweight a single-generation estimator.

5 A UNIVERSAL DROP-IN FOR SEMANTIC UQ

  • Existing estimators can use the learned symmetric comparison matrix without changing their original aggregators, isolating the operator substitution.The matrix is a positive semidefinite Gram matrix, avoiding redundant directed entailment evaluations and meeting downstream graph and eigenvalue methods’ spectral assumptions.Source: §5 · p. 7, §5.1 · p. 7

5.1 UPGRADING SEMANTIC AGGREGATORS

  • For semantic entropy and cluster-assignment entropy, thresholded similarities form clusters; kernel language entropy instead receives the continuous similarity matrix, while COS uses the question-conditioned embeddings.The original entropy, diffusion, and spectral computations are retained.Source: §5.1 · p. 7

5.2 LIGHTWEIGHT TOKEN-LEVEL REWEIGHTING

  • The embedding norms measure how much each token steers the answer representation, assigning greater semantic importance to tokens carrying the factual claim.The norm weights can reweight token-level entropy; this estimator still uses a single generation and requires tokenizer alignment.Source: §5.2 · pp. 7–8

6 EXPERIMENTS

Across evaluations of language and vision-language models, substituting the learned operator improved most semantic estimators’ results, with the best reported mean AUROC reaching 0.759. The experiments also assess operator cost, question-conditioning, and a single-generation token-level estimator.

6.1 SETUP

  • The evaluation spans six models and six datasets across text-only and visual question answering, with 18 model–dataset combinations; correctness targets use the greedy generation.An LLM judge assigns binary correctness against the reference, and results are reported using AUROC and ECE.Source: §6.1 · p. 8

6.2 MAIN RESULTS

  • Building the comparison matrix took 0.03–0.04 seconds per question with the learned operator versus 1.1–1.2 seconds with conventional NLI, roughly a 30× reduction.The learned operator avoids a forward pass for every pair; COS-int is cheaper still but requires white-box access and had the weakest performance in the table.Source: §6.2 · pp. 8–9
  • 120 of 126 comparisons (95%) improved; mean AUROC rose from 0.682 to 0.756 for KLE-Matern, and the strongest reported variant reached 0.759.Table 1 reports AUROC and ECE for published and operator-substituted estimators, averaged over 18 model–dataset combinations and broken down by dataset and model.Source: §6.2 · p. 8, Table 1 · p. 9
  • Removing the question from COSEnc’s input lowered mean AUROC from 0.759 to 0.742, with the gap present in all 18 model–dataset combinations.The authors describe conditioning as a real, secondary part of the improvement.Source: §6.2 · p. 9
  • Reweighting token-level entropy raised mean AUROC from 0.640 to 0.678 using one generation and one extra encoder pass.This result extends the operator’s use beyond semantic UQ.Source: §6.2 · p. 9

7 CONCLUSION

The paper concludes that replacing the shared comparison operator improves semantic uncertainty estimators across model–dataset combinations, supporting the operator as their limiting component. The authors frame the result as a test of that bottleneck, not a claim that their operator is best.

7 CONCLUSION

  • Across 18 model–dataset combinations, the operator improves SE, CAE, KLE, and COS while requiring N network evaluations instead of O(N^2).The uniform improvement supports the paper’s hypothesis that changing the shared operator improves the baseline estimators built on it.Source: §7 · p. 9
  • The operator uses a contrastive objective and synthetic data disjoint from every evaluation setting.The authors describe the data as generated from the language model’s predictive variability.Source: §7 · p. 9
  • The operator is trained on a small synthetic set with one configuration and might benefit from more data and a finer-grained objective.The authors’ claim is that the operator is the limiting component, not that theirs is the best operator or a new supervised uncertainty method.Source: §7 · pp. 9–10

AI USE STATEMENT

This section documents the paper’s AI use and reproducibility, then reviews uncertainty estimates and the comparison structures semantic estimators use. It distinguishes cluster-, graph-, and spectrum-based ways of summarizing semantic comparisons.

AI USE STATEMENT

  • Generative AI assisted with manuscript phrasing and table formatting; separately, an LLM serves as generator and judge in the training-data pipeline.The authors state that research ideas, experiments, code, results, and interpretation are their own.Source: AI USE STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

  • The paper specifies training-data construction, operator hyperparameters, correctness labeling, evaluation settings, diagnostic benchmarks, and per-cell results with bootstrapped confidence intervals.It states that models, encoders, and datasets are publicly available and that code, trained weights, and generated training data will be released upon acceptance.Source: REPRODUCIBILITY STATEMENT · p. 11

A.1 PRELIMINARIES

  • Token entropy averages uncertainty over generated tokens, but equal weighting treats wording hesitation like uncertainty about the answer itself.The estimate requires neither sampling nor comparison between generations.Source: §A.1 · p. 14

A.2 THE COMPARISON MATRIX

  • Semantic estimators compare sampled generations in a matrix whose operator scores whether two express the same answer; common operators use cosine similarity or bidirectional entailment.The matrix is inherited from components developed for other purposes, while estimators differ in how they summarize it.Source: §A.2 · p. 14

A.3 SEMANTIC ESTIMATORS

  • SE and CAE cluster generations: SE uses normalized sequence probabilities, while CAE uses empirical cluster frequencies and needs no sequence probabilities.Both methods form clusters using bidirectional entailment comparisons.Source: §A.3 · p. 15
  • KLE represents graded comparisons as a weighted semantic graph and summarizes a graph kernel’s entropy; unlike hard clustering, it can distinguish related but non-equivalent generations.The paper describes heat and Matérn kernel variants.Source: §A.3 · p. 15
  • COS-int and COS-ext summarize the spectrum of a cosine-similarity matrix; a dominant eigenvalue indicates concentration in one meaning direction, while a flat spectrum indicates dispersion.The variants differ in whether embeddings come from the generating model’s hidden states or an external sentence encoder.Source: §A.2 · p. 14, §A.3 · p. 15

A.4 DECOMPOSING EXISTING SEMANTIC UQ METHODS INTO AN OPERATOR AND AN AGGREGATOR

Semantic UQ methods can be decomposed into an operator that compares answers and an aggregator that combines those comparisons into an uncertainty score. Across the described methods, the operator determines the answer relations, while each aggregator summarizes them differently.

A.4.1 SEMANTIC ENTROPY

  • Semantic Entropy uses a binary NLI operator to cluster answers, then aggregates sequence-probability mass across the resulting clusters.The operator determines cluster membership; the aggregator measures how sequence probability is distributed across clusters.Source: §A.4.1 · p. 16

A.4.2 CLUSTER-ASSIGNMENT ENTROPY

  • Cluster-Assignment Entropy uses the same binary NLI operator and clustering rule as Semantic Entropy, but aggregates empirical cluster frequencies instead.The two estimators differ only in their final aggregation.Source: §A.4.2 · p. 16

A.4.3 KERNEL LANGUAGE ENTROPY

  • Kernel Language Entropy uses continuous NLI similarities as a weighted adjacency matrix and differs between variants only in the graph kernel used by its aggregator.The variants share the same semantic operator.Source: §A.4.3 · p. 17

A.4.4 COS

  • COS builds a cosine-similarity matrix from answer representations and applies spectral aggregation; its variants differ in whether representations come from the generating model or an external encoder.The operator defines the geometry of generations, while the aggregator measures how that geometry spreads across spectral directions.Source: §A.4.4 · p. 17

B.1 CLUSTERING OF INCORRECT ANSWERS

  • Similarities of 0.975 and 0.980 show the encoder grouping correct and incorrect paraphrase pairs alike when each pair shares meaning.The method uses correctness to construct triplets, not as an input label or prediction target.Source: §B.1 · p. 18, Table B.1 · p. 18

B.2 SENSITIVITY TO BASE ENCODER

  • Across six trained backbones, mean AUROC differed by only 0.009, and even the weakest trained encoder outperformed the best encoder before fine-tuning.The ranking changed after training, and larger models were not consistently better.Source: §B.2 · p. 18, Table B.2 · p. 19

B.3 NEGATIVES ARE NECESSARY, NOT INCIDENTAL

  • 0.759 mean AUROC with same-question hard negatives exceeded 0.705 with random negatives, 0.679 for the untrained encoder, and 0.650 without negatives.Same-question negatives share the topic and often the structure of the anchor while differing mainly in the answer, encouraging factual distinctions.Source: §B.3 · p. 19, Table B.3 · p. 19

B.4 SENSITIVITY TO TRAINING HYPERPARAMETERS

The section examines training-hyperparameter sensitivity, token-level localization of ground-truth answers, and clustering-threshold sensitivity. It reports token localization results and parameter sweeps while retaining the paper’s untuned settings.

B.4 SENSITIVITY TO TRAINING HYPERPARAMETERS

  • COSEnc AUROC varied from 0.749 to 0.765 across tested hyperparameters; margin 0.1 improved AUROC by 0.006, but the authors retained the original settings for fairness.The margin and learning rate were varied separately while other settings stayed fixed.Source: §B.4 · p. 19, Table B.4 · p. 20

B.5 TOKEN-LEVEL LOCALIZATION OF THE GROUND TRUTH

  • 72% of the highest-weighted tokens fell inside the ground-truth span (167/232), versus 3.9% by chance; the encoder received no token-level training supervision.The examples were correct Mistral-7B generations on TriviaQA val selected for containing exactly one annotated ground-truth alias; the figure shows answer tokens receiving most weight across positions.Source: §B.5 · p. 20, Figure B.1 · p. 21

B.6 SENSITIVITY TO THE CLUSTERING THRESHOLD

  • CAEEnc AUROC rose from 0.656 at τ = 0.3 to 0.751 at τ = 0.7; τ = 0.7 was 0.027 above τ = 0.5, and the tested sweep had not reached a maximum.The paper reports results at the principled τ = 0.5 rather than selecting a threshold against evaluation results; the authors say this may understate gains from a tuned threshold.Source: §B.6 · pp. 20–21, Table B.5 · p. 21

C.1 TRAINING HYPERPARAMETERS

The appendix describes the encoder’s training setup, construction of its weakly supervised triplets, and the LLM judging protocol. Training data come from Llama-3.1-8B-Instruct generations, while evaluation uses a separate judge and reports consistent conclusions across labeling strategies.

C.1 TRAINING HYPERPARAMETERS

  • The sentence-T5-large encoder is fine-tuned with triplet loss, cosine distance, margin 0.2, batch size 32, and learning rate 2 × 10^-5 for up to 30 epochs with early stopping.Early stopping follows three epochs without improvement in validation triplet accuracy.Source: §C.1 · p. 21

C.2 CONSTRUCTION OF THE TRAINING SET

  • From 3,160 NQ-OPEN questions, the training set retains 1,720 with at least two correct and one incorrect Llama-3.1-8B-Instruct generations, forming same-question triplets.Two correct answers form the positive pair; an incorrect answer to that question is the negative, and triplets are resampled each epoch.Source: §C.2 · pp. 21–22

D LLM AS A JUDGE

  • A larger external LLM compares generated answers with references; the main results use greedy-answer labels, separate judges label training and test splits, and both labeling strategies yield consistent conclusions.The judge compares against the reference rather than deciding correctness independently; Qwen2.5-32B-Instruct labels training and Llama-3.1-70B-Instruct labels test.Source: D LLM AS A JUDGE · p. 22

E.1 MODELS, DATASETS AND SPLITS

The diagnostic experiments test how off-the-shelf operators handle factual differences, equivalent answers with different verbosity, and syntax shifts. The evaluation also specifies the benchmark scope, sampling setup, and baseline operators.

E.1 MODELS, DATASETS AND SPLITS

  • The evaluation uses six datasets across visual question answering and text question answering, producing 18 model–dataset combinations.Each model is evaluated on the three datasets of its modality.Source: §E.1 · p. 22

E.2 PROMPTS AND DECODING

  • Each question yields 20 naturally varied answers sampled with temperature 1.0, top-p 0.9, and top-k 50.Prompts use a short modality-specific instruction and impose no answer-length or form constraint.Source: §E.2 · p. 22

E.3 ESTIMATOR HYPERPARAMETERS

  • SE, CAE, and KLE use a DeBERTa NLI cross-encoder, while COS-int uses generator hidden states and COS-ext uses sentence-t5-large.COS-ext and COSEnc share architecture and initialization but differ in training.Source: §E.3 · p. 23

F.1 THE CONTEXT TRAP (COS-EXT)

  • 0.91 mean cosine similarity for the base encoder versus 0.45 after training on 1,000 pairs that differ on the fact determining correctness.The pairs were near-identical in phrasing but factually opposed.Source: §F.1 · p. 23

F.2 THE AUTOREGRESSIVE BIAS (COS-INT)

  • 0.869 average cosine similarity for hidden states versus 0.350 for the operator on pairs sharing their final four words but differing in correctness.The matching criterion yielded pairs from 491 queries.Source: §F.2 · pp. 23–24

F.3 THE ENTAILMENT TRAP (NLI CROSS-ENCODERS)

  • 0.291 average bidirectional entailment probability for the NLI cross-encoder versus 0.801 for the operator on 1,000 equivalent pairs spanning response verbosity.The additional correct detail in longer answers blocks entailment in one direction.Source: §F.3 · p. 24

F.4 THE DISTRIBUTION MISMATCH (OOD SYNTAX)

  • The distribution-mismatch test pairs 1,000 generated complete sentences with bare-entity references, with each pair equivalent by construction.The references are typically single entities rather than sentences.Source: §F.4 · p. 24

G.1 PER-CELL RESULTS

Per-cell tables report AUROC and ECE for estimator variants across text-only and vision-language models. The ablations report a consistent benefit from question conditioning and unchanged estimator rankings under two correctness-label strategies.

G.1 PER-CELL RESULTS

  • Tables G.1 and G.4 report per-model AUROC and ECE, with bootstrapped 95% confidence intervals, for textual and visual benchmarks, respectively.The tables show each estimator in its published form and with the operator substituted.Source: §G.1 · p. 25, Table G.1 · p. 26, Table G.4 · p. 28

G.2 QUESTION-CONDITIONING ABLATION

  • Mean AUROC falls from 0.759 to 0.742 when the question is removed from both training and evaluation of COSEnc.The question-conditioned variant gains 0.006 to 0.026 in almost every one of 18 model–dataset combinations; the authors do not claim conditioning is the main source of improvement.Source: §G.2 · p. 25, Table G.2 · p. 27

G.3 GREEDY VS. MAJORITY-VOTE LABELS

  • Across both greedy and majority-vote correctness labels, each operator-substituted variant outperforms its base counterpart, with the same within-block ranking.Absolute scores are slightly higher for all methods with majority-vote labels, while gains are similar and slightly larger.Source: §G.3 · p. 25, Table G.3 · p. 27
Loading 2609.34967v1…