Source-linked AI summary
S-EDL: Eliciting Self-Evidence from Sequence Likelihoods for Semantic Calibration of LLMs
Yawei Li, Jiazheng Li, David Rügamer, Bernd Bischl, Yulan He, Mina Rezaei
TL;DR
Open-ended calibration is challenging because different token sequences can express the same meaning, while existing approaches target fixed labels or single-response correctness. S-EDL derives evidence from a model’s sequence likelihoods to train over prompt-specific semantic classes. Across three models and four benchmarks, it improves accuracy and calibration over supervised fine-tuning, reducing average calibration error by up to roughly 1/3.
Problem
Open-ended generation has many surface forms for the same meaning, and prior calibration targets fixed labels or the binary correctness of a single response rather than distributions over semantic answers.
Source: §1 · p. 1Method
S-EDL derives a differentiable evidence vector from sequence likelihoods and uses it to form a Dirichlet prior over prompt-specific semantic classes for calibration-aware fine-tuning.
Source: §1 · pp. 1–2Results
Across three models and four open-ended benchmarks, S-EDL improves average accuracy and calibration over supervised fine-tuning, reducing average calibration error by up to roughly 1/3.
Source: §1 · p. 2Takeaways & Limitations
The analysis describes probability mass being reallocated across semantic outcomes, raising likelihoods of ground-truth-class sequences and suppressing the most overconfident incorrect alternative.
Source: §6 · p. 10S-EDL adds moderate training overhead; tasks requiring an LLM judge also incur memory overhead and potential clustering noise, and long-context or code-generation settings remain untested.
Source: §6 · p. 10
Abstract
from arXiv · showhide
Calibrating large language models (LLMs) in open-ended generation is uniquely challenging because many distinct token sequences can express the same underlying meaning. Prior calibration techniques target either fixed-set classification confidence or the binary correctness of individual strings, making them inherently ill-suited for the dynamic, multi-string nature of open-ended semantic outcomes. To bridge this gap, we introduce S-EDL, a novel evidential method that natively optimizes LLMs for semantic calibration. By eliciting evidence from the sequence likelihoods of the model itself, S-EDL constructs a differentiable Dirichlet prior over prompt-specific semantic classes. Optimizing the evidential loss under this prior yields a calibration-aware training signal over variable semantic classes, completely bypassing the need for fixed multiple-choice options. Evaluations across four open-ended benchmarks and three models demonstrate that S-EDL substantially reduces semantic calibration error while improving accuracy. By aligning generative likelihoods with semantic correctness, this work establishes a framework for deploying trustworthy LLMs in complex, open-ended applications. Code is available at: https://github.com/sandylaker/s-edl.
1 Introduction
Open-ended calibration must account for distinct strings that express the same meaning, a gap left by fixed-label and single-string correctness methods. S-EDL derives differentiable evidence from sequence likelihoods to train over semantic classes, improving accuracy and calibration across three models and four benchmarks.
1 Introduction
- Open-ended generation challenges calibration because distinct token sequences can express the same answer, so surface-form probability may not reflect semantic confidence.Prior work primarily focused on multiple-choice calibration or confidence in the binary correctness of an individual response, leaving a gap in fine-tuning for semantic calibration.Source: §1 · p. 1
- S-EDL derives a differentiable evidence vector from the model’s sequence likelihoods and uses it to parameterize a Dirichlet prior over semantic outcomes.Highly probable strings receive greater evidential support, yielding a calibration-aware training signal.Source: §1 · pp. 1–2
- S-EDL extends evidential learning to diverse semantic meanings and theoretically unifies post-hoc metrics such as Semantic Entropy [20] under an evidential framework.The approach avoids requiring fixed multiple-choice options.Source: §1 · p. 2
- S-EDL improves average accuracy and calibration over supervised fine-tuning across three models and four benchmarks, reducing average calibration error by up to roughly 1/3.The reported result covers open-ended benchmarks.Source: §1 · p. 2
2 Related work
Prior work addresses open-ended calibration through meaning-based semantic targets, single-response verbal confidence, and token-level calibration. S-EDL follows the semantic line and applies evidential learning to open-ended generation.
From MCQA to semantic calibration
- Semantic uncertainty metrics [6] [11] [20] [34] [36] group sampled generations by meaning, and semantic calibration [33] turns this view into an open-ended calibration target.Earlier calibration work focused on MCQA, where a fixed label set supports classification-style methods, including Bayesian low-rank adaptation [37] [42] [48] [53] and evidential approaches [24].Source: §2 · p. 2
Verbal and token-level calibration
3 Background
Semantic calibration groups responses by meaning and evaluates confidence in the top semantic class. Evidential deep learning instead represents uncertainty over class probabilities with a Dirichlet prior, learned through an evidential objective.
3.1 Semantic calibration in open-ended LLM generation
- B-calibration maps prompt-response pairs to semantic classes, allowing distinct sequences with the same meaning to share a class.The mapping can be implemented by a verification tool or an LLM-as-judge that clusters responses by meaning.Source: §3.1 · p. 2
- Semantic calibration requires confidence in the top class to match its true probability of correctness across prompts with that confidence.The calibration objective therefore concerns both the most likely semantic class and confidence in that belief.Source: §3.1 · p. 2
3.2 Evidential deep learning
- EDL predicts a distribution over class probabilities, representing uncertainty about those probabilities rather than only a point estimate.It places a Dirichlet prior on categorical probabilities, with parameters built from a non-negative evidence vector whose entries represent accumulated class support.Source: §3.2 · p. 3
- The EDL training objective minimizes the Bayes risk of a mean-squared-error objective for a one-hot target.Prior EDL calibration for LLMs was limited to fixed MCQA options such as A/B/C/D; the paper motivates deriving evidence from the LLM’s own generations for open-ended use.Source: §3.2 · p. 3
4 Methodology
S-EDL addresses the mismatch between fixed-label evidence heads and prompt-specific semantic classes by deriving differentiable evidence from the model’s own sequence likelihoods. Its evidential objective supplies a semantic calibration signal whose gradients shift likelihood away from overconfident incorrect classes and toward the ground-truth class.
4.1 Motivation
- A fixed prediction head cannot represent prompt-dependent semantic classes or keep its evidence tied to the distribution being calibrated, while sampled histograms are nondifferentiable and weight samples equally.The desired evidence should favor high-likelihood completions and remain differentiable through sample log-likelihoods.Source: §4.1 · p. 3
4.2 Formulation of self-evidence
- S-EDL estimates a tempered semantic distribution from Monte Carlo generations, using likelihood-based sample weights that remain differentiable through the model’s log-likelihoods.For β > 1, the distribution emphasizes classes supported by high-confidence sequences; β = 1 recovers the standard semantic distribution.Source: §4.2 · p. 4
- As the sample count grows, the estimator recovers the true tempered semantic distribution; S-EDL uses that distribution as a training signal, unlike post-hoc Semantic Entropy.The paper uses β = 1.2 in its experiments, while Semantic Entropy corresponds to β = 2.Source: §4.2 · p. 4
- The estimated distribution allocates sample-budget evidence across semantic classes to form a Dirichlet prior, whose strength is fixed at N + |S|.The evidential MSE is optimized jointly with supervised fine-tuning, and the likelihood-to-loss chain is differentiable with respect to model parameters.Source: §4.2 · pp. 4–5
4.3 Gradient structure of S-EDL
- The gradient weights higher-likelihood samples more, promotes ground-truth-class sequences, and penalizes sequences in the most overconfident incorrect class.The update conservatively reallocates likelihood rather than uniformly increasing or suppressing it, shifting probability mass toward the ground-truth class.Source: §4.3 · p. 5
5 Experiments
Across three models and four open-ended datasets, S-EDL improves semantic calibration and accuracy over baselines. The experiments examine in-domain and cross-domain performance, token-probability profiles, hyperparameter and sample-count ablations, and the cost of semantic clustering.
5.1 Experimental setup
- The experiments train and evaluate on four open-ended STEM datasets, using MATH-500 as the MATH test subset, and report Pass@1, semantic accuracy, and Smooth ECE.They compare S-EDL with SFT, token-level calibration, Bayesian ensembling, and sequence-level correctness optimization; semantic metrics use 50 generations per question.Source: §5.1 · pp. 5–6
5.2 Calibration performance on in-distribution datasets
- S-EDL has the lowest average smECE across all three models, beating the strongest token-level baseline by 2.3, 2.3, and 2.5 points, respectively.The comparisons are Qwen3-8B-Base 10.5 vs. LS 12.8, Ministral3-8B-Base 8.6 vs. RCFT 10.9, and MiMo-7B-Base 9.4 vs. LS 11.9.Source: §5.2 · p. 6, Table 1 · p. 6
- Averaged across four datasets, S-EDL improves Qwen3-8B-Base over SFT by +9.6 Pass@1, +5.7 S-Acc, and −6.0 smECE.S-EDL improves over SFT on all three metrics for every model; it also remains on the S-Acc × smECE Pareto frontier.Source: §5.2 · p. 7
5.3 Cross-domain evaluation
- In cross-domain evaluation, the Pass@1 gap between the better sequence-level method and the strongest token-level or Bayesian baseline widens to 13.8, 16.5, and 13.6 points across the three models.S-EDL exceeds Dr.GRPO in average cross-domain Pass@1 by 7.3/6.3/3.3 points and has lower average smECE by 4.7/3.0/0.1 points across the models.Source: §5.3 · p. 7, Table 2 · p. 8
5.4 Token probability profiles of generated solutions
- S-EDL’s mean sampled-token probabilities stay within 0.06 of SFT across all twelve model-dataset pairs, despite average smECE at least 2 points lower than LS, RCFT, and BLoB.The authors conclude that the gain is not explained by a coarse shift in token concentration; S-EDL instead reallocates probability mass between semantic outcomes.Source: §5.4 · pp. 7–8, Figure 1 · p. 8
5.5 Ablation study
- With β = 1.2, S-EDL improves over SFT on all three metrics for β values 1.1, 1.2, and 1.5; β = 1.2 gives the best balance.At β = 1.01 the calibration signal is attenuated by two orders of magnitude; increasing λMSE raises Pass@1 and mildly lowers smECE at a small cost in S-Acc. The study adopts β = 1.2 and λMSE = 5.0.Source: §5.5 · p. 8, Figure 2 · p. 9
- Increasing Ntrain from 4 to 32 yields monotonic gains in Pass@1 and S-Acc and decreases smECE; even Ntrain = 4 improves all three metrics over SFT.The number of sampled sequences also affects training speed because computing the evidential loss requires clustering them into semantic classes.Source: §5.5 · p. 9, §5.6 · p. 9, Figure 3 · p. 9
5.6 Accelerating semantic clustering by a find-first algorithm
- For N = 50 sequences, find-first clustering averages approximately 65 verifier calls on GSM8K and 165 on MATH, versus 1,225 for naive pairwise clustering.These costs are roughly 1.3N and 3.3N calls, respectively, bringing practical cost closer to O(N); MATH requires more comparisons than GSM8K.Source: §5.6 · p. 9, Figure 4 · p. 10
6 Conclusion
S-EDL derives a differentiable evidential objective from a model’s sequence likelihoods and reallocates probability across semantic outcomes; the paper reports simultaneous gains in accuracy and calibration. The authors also identify training overhead and untested task settings as limitations.
6 Conclusion
- S-EDL builds a Dirichlet prior over prompt-specific semantic classes from sequence likelihoods, then uses it to fine-tune LLMs for semantic calibration.The gradient analysis describes raising likelihoods for ground-truth-class sequences while suppressing the most overconfident incorrect alternative.Source: §6 · p. 10
- The authors report that this probability reallocation produces simultaneous gains in accuracy and calibration.Source: §6 · p. 10
- Training entails moderate overhead over standard SFT, and S-EDL remains untested on long-context tasks and domains where semantic equivalence is difficult to define, such as code generation.For general tasks requiring an LLM judge, the paper also notes memory overhead and potential clustering noise.Source: §6 · p. 10
A.1 Proof of Proposition 1
The proof establishes Proposition 1 by applying the Strong Law of Large Numbers to the numerator and denominator of Eq. (6), then identifying their ratio with πβ(s).
A.1 Proof of Proposition 1
- The proof applies the Strong Law of Large Numbers separately to the numerator and denominator of Eq. (6).Source: §A.1 · p. 15
- The numerator in Eq. (6) is a sample average of i.i.d. terms under z(i) ∼ pθ(·|x).Source: §A.1 · p. 15
- The continuous mapping theorem identifies the ratio of the two limits with πβ(s) in Eq. (4).Source: §A.1 · p. 15
A.2 Proof of Theorem 1
These appendix sections analyze S-EDL’s loss gradients, its relation to GRPO-like updates, handling of unobserved ground-truth classes, and semantic clustering. The supplied proof excerpt for Corollary 2 does not provide a substantive result to summarize.
A.2 Proof of Theorem 1
- Theorem 1’s proof begins by differentiating the MSE loss with respect to the estimated semantic-class probability, then backpropagates through the remaining maps.It uses αs = N ˆπβ(s) + 1 and a constant Dirichlet strength α0 = N + |S| in that derivative.Source: §A.2 · p. 15
A.3 Proof of Corollary 1
- For β > 1, ground-truth-class samples receive nonpositive loss gradients, while samples from the most overconfident incorrect class receive nonnegative gradients.The inequalities are strict under the stated non-collapse conditions for the semantic distribution.Source: §A.3 · p. 16
A.5 Relation to GRPO-like updates
- S-EDL’s negative MSE-loss gradient can be interpreted as a GRPO-like advantage, centered across semantic classes for the same prompt.Unlike GRPO’s arithmetic-mean reward centering, S-EDL uses a semantic-probability-weighted mean and sample-dependent modulation.Source: §A.5 · pp. 16–17
B.1 Construction of semantic class set and handling unobserved ground-truth classes
- If the ground-truth class is absent from sampled generations, S-EDL adds it conditionally so the gradient can still increase its likelihood.During evaluation, the added class has sampled count zero; deterministically adding the ground-truth completion makes the estimator a hybrid of the model distribution and a supervised finite-candidate signal.Source: §B.1 · p. 17
B.2 Find-first algorithm for semantic clustering
- Naive pairwise semantic comparisons scale as O(N^2), whereas listwise LLM-judge clustering reduces temporal complexity to O(N).The find-first algorithm orders existing representatives by string similarity to each new sequence before matching or creating a cluster.Source: §B.2 · pp. 17–18
B.3 Dataset construction
The dataset construction section describes four STEM benchmarks and dataset-specific preparation for ChemistryQA and FormulaReasoning. For FormulaReasoning cross-domain evaluation, it also reports manually adding relevant physical laws as hints.
B.3 Dataset construction
- ChemistryQA training uses GPT-OSS-120B-generated chain-of-thought solutions filtered for correct final answers, yielding 3062 samples.Source: §B.3 · p. 18
- FormulaReasoning training uses GPT-OSS-120B-generated chain-of-thought solutions filtered for correctness, yielding 4096 samples.Source: §B.3 · p. 18
- For cross-domain evaluation on FormulaReasoning, MATH-fine-tuned models showed severe accuracy degradation, so the authors added relevant physical laws as hints to all methods’ evaluation prompts.The hints were used exclusively for cross-domain evaluation to ensure fairness across fine-tuning methods.Source: §B.3 · p. 18
B.4 Training hyperparameters
The appendix specifies semantic-clustering procedures and S-EDL settings, then lists training configurations for the comparison baselines.
B.4 Training hyperparameters
- For MATH and GSM8K, semantic clustering uses math-verify; ChemistryQA and FormulaReasoning use GPT-OSS-20B as an LLM-as-judge.The judge uses the system prompt in Appendix C.5.1.Source: §B.4 · p. 20
B.5 Hyperparameters of baselines
- BLoB uses one Bayesian sample during training and five during evaluation, with bayes_kl_reweighting enabled.Training settings include bayes_epsilon = 0.05, bayes_beta = 0.2, and bayes_kllr = 0.1.Source: §B.5 · p. 20
- RCFT training uses rcft_alpha = 0.5, rcft_calibration_scale = 0.001, rcft_num_bins = 10, and rcft_eps = 0.000001.Source: §B.5 · p. 20
- Label smoothing uses a coefficient of 0.1 during training.Source: §B.5 · p. 20
- Dr.GRPO starts from the supervised checkpoint at epoch 1 and matches S-EDL’s evidential-loss training steps, using group size 8 without a reference-model KL penalty.The no-KL-penalty setting follows Liu et al. [29].Source: §B.5 · p. 20
C.1 In-distribution evaluations across varying temperatures
Appendix evaluations test S-EDL across sampling temperatures and visualize accuracy–calibration outcomes and token-probability profiles. They report a Pareto-frontier position across models, low calibration error across temperatures, and limited shifts in token-probability distributions.
C.1 In-distribution evaluations across varying temperatures
- Across temperatures 0.4–1.0, S-EDL generally achieves low calibration error while maintaining competitive semantic accuracy.S-EDL can be combined with temperature scaling to further boost calibration performance.Source: §C.1 · p. 21, Figure 5 · p. 21
C.2 Visualization of the accuracy-calibration plane
- S-EDL lies on the accuracy–calibration Pareto frontier across all three models, with averages taken over four datasets.The plane’s preferred region combines high S-Acc with low smECE.Source: §C.2 · p. 21, Figure 6 · p. 22
C.3 Extended per-token probability profiles
- Across GSM8K, ChemistryQA, and FormulaReasoning, S-EDL’s mean token probabilities differ from supervised fine-tuning by less than 0.06 for all model–dataset pairs.The authors report that calibration improvements occur without a drastic or systematic shift in overall token-level probability distributions.Source: §C.3 · p. 22, Figure 7 · p. 23
C.4 Additional ablation study on β and λMSE
This section reports an additional MATH ablation supporting the default β and λMSE settings, alongside appendix material on semantic-judging rules, training prompts, reproducibility, societal impacts, and methodological uses of LLMs; the latter passages add no further ablation findings.
C.4 Additional ablation study on β and λMSE
- On MATH-500, β = 1.2 and λMSE = 5.0 improve Pass@1 and semantic accuracy while lowering smECE; β = 1.01 weakens evidential learning and worsens accuracy and calibration versus SFT.The study uses Ministral3-8B-Base trained on MATH, and reports trends consistent with those on GSM8K.Source: Figure 8 · p. 24, §C.4 · p. 24
C.5.1 System prompt of the LLM-as-judge
- The semantic-equivalence judge matches numeric answers across notation and units when values agree, allows rounding only to at least three significant digits, and rejects sign or order-of-magnitude errors.It also treats minor phrasing differences as equivalent for non-numeric answers and returns −1 when no cluster representative matches.Source: §C.5.1 · pp. 24–25
- For chemical equations, the judge accepts equivalent arrow and species-order variants, uniformly scaled coefficients, and omitted state symbols, but rejects conflicting states.The prompt also treats subscript and superscript notation variants as equivalent.Source: §C.5.1 · pp. 24–25
C.6 Examples of training prompts
- Training prompts request step-by-step reasoning followed by a boxed final answer; the authors omit think-block tokens because they triggered random trailing text and repetitive loops during LoRA fine-tuning.The stated issue was that base models had limited exposure to those special tokens.Source: §C.6 · pp. 25–26
NeurIPS Paper Checklist
- The checklist states that theoretical results have explicit assumptions and full proofs in Appendices A.1–A.4, and that experiments used H100/H200 GPUs for about 2,500 GPU-hours overall.Per-run training time is approximately 1–12 hours, depending on the fine-tuning method.Source: NeurIPS Paper Checklist · pp. 29, 31
- The paper reports sufficient reproducibility details, including base models, LoRA settings, evidential hyperparameters, evaluation protocol, dataset construction, optimization, baselines, and clustering implementation.The checklist also notes that training prompts are provided and that datasets are public with preprocessing scripts.Source: NeurIPS Paper Checklist · pp. 29–30
- Main calibration and accuracy tables give single-run point estimates without error bars, which the checklist attributes to compute constraints.Variance information is available for verifier-call counts in Fig. 4.Source: NeurIPS Paper Checklist · p. 31
- The authors say better semantic calibration can help users judge when to trust LLM outputs, while noting that plausible but incorrect outputs could become harder to recognize.They report both accuracy and calibration error so practitioners can audit both dimensions.Source: NeurIPS Paper Checklist · p. 32
- GPT-OSS-120B generates filtered correct chain-of-thought solutions for two training splits, and an LLM-as-judge serves as the equivalence oracle for general semantic clustering.The core S-EDL objective instead uses the model’s own sequence likelihoods and does not depend on LLM-judge supervision.Source: NeurIPS Paper Checklist · p. 34