PaperlayerView exact revision on arXiv

Source-linked AI summary

Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles

Yuxi Xia, Pedro Henrique Luz de Araujo, Klim Zaporojets, Benjamin Roth

arXiv:2501.03991v1cs.CL

TL;DR

Prior work has not comprehensively tested calibration methods across model sizes and prompt styles. This study evaluates calibration across 12 LLMs and four prompt styles, finding benefits from response agreement and focal loss, with few-shot prompts especially useful for auxiliary models.

  • Problem

    Prior studies assess calibration methods in limited settings and overlook generalization across diverse model sizes and prompt styles.

    Source: §6 · p. 8

  • Method

    The study evaluates calibration across 12 LLMs and four prompt styles, examining response agreement and loss functions with Calib-n, an auxiliary confidence model aggregating multiple LLM responses.

    Source: §1 · p. 1, §6 · p. 8

  • Results

    Response agreement and focal loss enhance calibration from baselines, FL-Calib-1 achieves the best performance, and few-shot prompts benefit auxiliary models.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The study provides calibration evidence across 12 LLMs and four prompt styles, while broader LLMs and tasks remain targets for future work.

    Source: §6 · p. 8, §7 · p. 9

    The study uses a common, cost-efficient method to determine generation accuracy, while the optimal solution remains underexplored.

    Source: §7 · p. 9

Abstract

from arXiv · show

Calibration, the alignment between model confidence and prediction accuracy, is critical for the reliable deployment of large language models (LLMs). Existing works neglect to measure the generalization of their methods to other prompt styles and different sizes of LLMs. To address this, we define a controlled experimental setting covering 12 LLMs and four prompt styles. We additionally investigate if incorporating the response agreement of multiple LLMs and an appropriate loss function can improve calibration performance. Concretely, we build Calib-n, a novel framework that trains an auxiliary model for confidence estimation that aggregates responses from multiple LLMs to capture inter-model agreement. To optimize calibration, we integrate focal and AUC surrogate losses alongside binary cross-entropy. Experiments across four datasets demonstrate that both response agreement and focal loss improve calibration from baselines. We find that few-shot prompts are the most effective for auxiliary model-based methods, and auxiliary models demonstrate robust calibration performance across accuracy variations, outperforming LLMs' internal probabilities and verbalized confidences. These insights deepen the understanding of influence factors in LLM calibration, supporting their reliable deployment in diverse applications.

1 Introduction

The introduction frames calibration as aligning confidence with prediction accuracy and notes that prior evaluations covered narrow model and prompt settings. It presents Calib-n and reports that response agreement, focal loss, and few-shot prompts improve calibration, while auxiliary models remain robust across accuracy changes.

  • Prior auxiliary-model evidence covered only two LLMs and two prompt styles, motivating broader generalization testing.Source: §1 · p. 1
  • Calib-n trains one auxiliary confidence model on joint question-answer strings from n LLMs, using their response agreement and BCE, focal, or AUC losses.Source: §1 · p. 1, Figure 1 · p. 2
  • The study examines generalization across 12 LLMs and four prompt styles, with experiments spanning four open-ended question-answering datasets.Source: §1 · pp. 1–2
  • Response agreement lets Calib-n outperform state-of-the-art baselines; focal loss improves calibration over BCE and AUC for Calib-1 and Calib-n.Source: §1 · p. 2
  • Few-shot prompts are most beneficial for training auxiliary models, although method effectiveness is highly influenced by prompt style.Source: §1 · p. 2
  • Auxiliary models maintain robust calibration across accuracy changes, unlike the fluctuating calibration of LLM internal probabilities and verbalized confidences.Source: §1 · p. 2

2 Related Work

Related work studies several approaches to confidence estimation and calibration, as well as the effects of loss functions, response agreement, and prompt styles. The paper identifies comprehensive analysis of these factors as missing from prior work.

  • Prior work compares verbalized confidence with conditional probabilities and explores hidden-state and text-input/output methods for confidence estimation.Source: §2 · p. 2
  • Prior work had not comprehensively analyzed loss functions, response agreement, and prompt styles as factors in LLM calibration.Source: §2 · pp. 2–3
  • Focal loss has been reported to improve neural-network calibration, while AUC surrogate and correctness-ranking losses adjust logit rankings.Source: §2 · p. 3
  • Combining predictions from multiple models has been proposed to mitigate overfitting and reduce overconfidence in calibration (Kim et al., 2023).Source: §2 · p. 3
  • LLM outputs depend on prompts, and different prompt styles can significantly influence performance (Min et al., 2022; Wang et al., 2023; Chen et al., 2024).Source: §2 · p. 3

3 Methodology

The methodology evaluates confidence calibration across prompt styles, uses a judge to label answer correctness, and trains auxiliary models with single- or multi-LLM responses and alternative losses.

3 Methodology

  • The framework combines four prompt styles, correctness assessment, auxiliary-model calibration training, and evaluation of confidence estimates.It is designed to study calibration across different models, datasets, and prompts.Source: §3 · p. 3, §3.4 · p. 4

3.1 Prompt Styles

  • Verbalized, zero-shot, chain-of-thought, and few-shot prompts are compared to address prior evaluations that used inconsistent or limited prompt styles.Generated answers are processed with regular-expression-based text processing.Source: §3.1 · p. 3

3.2 Correctness of LLM Generation

  • Prometheus-8x7bv2.0 assigns each generated answer a binary correctness score based on semantic equivalence to the target answer.The authors selected the open-source judge because its judgments were reported to correlate strongly with human evaluators and large proprietary models.Source: §3.2 · p. 3

3.3 Response Agreements of Multiple LLMs

  • Calib-n trains one auxiliary model on responses from multiple target LLMs to estimate each model’s confidence, using low inter-model consensus as a signal of uncertainty.Its auxiliary model takes joint question-answer strings as input and outputs probabilities for the LLM answers.Source: §3.3 · p. 3

3.4 Loss Functions

  • The study compares BCE, focal, and AUC surrogate losses in Calib-1 and Calib-n variants to optimize calibration.Focal loss emphasizes examples with large confidence–correctness discrepancies, while the AUC surrogate uses logistic loss to separate scores for true and false answers.Source: §3.4 · p. 4

4 Experiments

The experiments compare confidence-estimation methods across four question-answering datasets, 12 LLMs, prompt styles, and calibration metrics, including established baselines.

4 Experiments

  • The experimental setup compares confidence-estimation methods using diverse datasets, LLMs, and state-of-the-art baselines.The aggregate analysis counts method wins across combinations of prompt style, dataset, and model.Source: §4 · p. 4, §4.3 · p. 5

4.1 Datasets

  • Four open-ended question-answering datasets—TriviaQA, SciQ, WikiQA, and NQ—are evaluated with specified training and test sample counts.TriviaQA, SciQ, and NQ use 2k/1k training/test samples each; WikiQA uses 1040/293.Source: §4.1 · p. 4

4.2 Models

  • The evaluation covers 12 models from five families: seven small models with 2–9B parameters and five large models with 27–72B parameters.Calib-n models are trained with all LLMs in the same size group and provide confidence scores for each model in that group.Source: §4.2 · p. 4

4.3 Evaluation Metrics

  • Calibration is evaluated using ECE, temperature-scaled ECE, Brier score, and AUC of selective accuracy and coverage.The aggregate analysis counts method wins across all combinations of prompt style, dataset, and model.Source: §4.3 · p. 5

4.4 Baseline Methods

  • Baselines include LLM probabilities, Platt-scaled probabilities, verbalized confidences, and APRICOT’s auxiliary-model approach.APRICOT trains an auxiliary transformer using per-cluster accuracies derived from a target LLM’s answers.Source: §4.4 · p. 5

5 Results and Analysis

Across the evaluated settings, (FL)Calib-1 has the best overall performance, while calibration outcomes also vary with loss function, prompt style, and accuracy range. Detailed results favor Calib-* methods over baselines on most prompts and models.

What is the best loss function for improving cal-

5.2 Detailed Fraction Results

  • On NQ, (BCE)Calib-n achieves the lowest ECE and Brier scores, including ECE 0.049 on Llama3.1-70b; (BCE)Calib-1 gives ECE-t 0.045 with CoT versus 0.068 with verbalized prompts on Gemma2-27b.On Qwen2-72b, (FL)Calib-n+PS achieves ECE 0.071, compared with 0.095 on Mixtral-8x7b.Source: Table 1 · p. 7, §5.2 · p. 8
  • Reliability diagrams show Platt Scaling produces a narrow confidence range, while LLM probabilities and verbalized confidence often overstate confidence below 40% accuracy.Calib-* confidence levels align more closely with true accuracy.Source: §5.2 · p. 8, Figure 5 · p. 8

6 Conclusion

The study evaluates calibration across 12 LLMs and four prompt styles, finding improvements from response agreement and focal loss. Few-shot prompts and auxiliary methods also show benefits, including stable calibration across accuracy levels.

6 Conclusion

  • Experiments span 12 LLMs ranging from 2B to 72B parameters and four prompt styles, addressing prior evaluations' limited generalization evidence.The study also analyzes four datasets.Source: §6 · p. 8
  • Both response agreement and focal loss enhance calibration from baselines, with (FL)Calib-1 achieving the best performance.The paper also reports that few-shot prompts improve LLM accuracy and calibration.Source: §6 · p. 8
  • Auxiliary-based methods maintain stable calibration across diverse settings, regardless of accuracy levels.The conclusion encourages future methods to re-evaluate the explored factors for confidence estimation.Source: §6 · p. 8

7 Limitations

The study leaves fine-grained and automatically generated prompt variations unexplored, notes that common calibration metrics may miss user-perceived confidence and task utility, and uses a cost-efficient but not established-optimal method for judging answer accuracy.

7 Limitations

  • Fine-grained prompt variations and automatically generated prompts remain unexplored, leaving their interaction with calibration for further investigation.The study covers prompt types including few-shot and Chain-of-Thought prompts.Source: §7 · p. 9
  • ECE, ECE-t, and Brier scores may not capture user-perceived confidence or task-specific utility.Source: §7 · p. 9
  • The study chooses a commonly used, cost-efficient method for judging answer accuracy, while the optimal solution remains underexplored.Existing work uses LLM judges or metrics such as exact match and ROUGE.Source: §7 · p. 9
  • Future work could test broader LLMs and tasks, automate prompt optimization, and develop hybrid approaches for varying accuracy levels and application constraints.Source: §7 · p. 9

A.1 Prompts for target LLM

The appendix specifies four target-LLM prompt styles: verbalized confidence, zero-shot, chain-of-thought, and few-shot.

A.1.1 Verb. prompt

  • The verbalized prompt requests a best guess and a probability from 0.0 to 1.0, with no other explanation.Source: §A.1.1 · p. 14

A.1.2 Zero-shot prompt

  • The zero-shot prompt requests only a short best guess, without explanations.Source: §A.1.2 · p. 14

A.1.3 CoT prompt

  • The CoT prompt asks the LLM to think step by step and provide a minimal final answer prefixed with “Answer:”.Source: §A.1.3 · p. 14

A.1.4 Few-shot prompt

  • The few-shot prompt supplies five question-target examples before the new question and leaves the assistant response to complete.Source: §A.1.4 · p. 14

A.2 Prompt for Judge

The judge prompt evaluates an LLM answer against a reference using a binary semantic-equivalence rubric and a constrained feedback-and-score format.

A.2 Prompt for Judge

  • The judge receives an instruction, response, reference answer, and rubric, then returns rubric-based feedback and a binary score for semantic equivalence.The required output places an integer score of 0 or 1 after the feedback.Source: §A.2 · p. 14

A.3 Technical Details

Across prompt styles, model sizes, and datasets, calibration performance depends on the configuration, with no single method consistently best. Few-shot prompts improve LLM accuracy, while auxiliary-model calibration behavior and reliability vary with accuracy and dataset conditions.

A.3 Technical Details

  • Auxiliary models were trained for five epochs with a learning rate of 1e-5 and batch size 16; one epoch on 2k samples takes around 200 seconds on one GPU.The auxiliary model is BERT-base (110M parameters); experiments use at most two NVIDIA H100 GPUs, and training time varies with dataset size and the number of joint LLMs.Source: §A.3 · p. 14

A.4 Aggregated Results Across Different Configurations

  • Calibration performance varies with prompt style, model size, and dataset, and no method is consistently best across configurations.Figure 6 aggregates method comparisons across these three settings.Source: §A.4 · p. 14, Figure 6 · p. 15
  • With BCE, Calib-n outperforms Calib-1 across prompt styles and datasets, while with focal loss it does so only for few-shot prompts and (FL)Calib-1 performs better for small models.Calib-n outperforms Calib-1 on large models; the preferred method therefore depends on configuration and loss.Source: §A.4 · p. 15

A.5 Accuracy Statistics of LLMs

  • Large models typically achieve higher accuracy than small models, few-shot prompts improve accuracy more than other prompt styles, and NQ is the most challenging dataset.Most LLMs achieve their highest accuracy on Sciq.Source: §A.5 · p. 15

A.6 Reliability Diagrams

  • For high-accuracy TriviaQA and Sciq settings, Calib-* with BCE or focal loss is less conservative and more likely to assign high confidence.The reliability diagrams cover Llama3.1-70b with verbalized prompts.Source: §A.6 · p. 15, Figure 9 · p. 16, Figure 10 · p. 17

A.7 Detailed Results of LLM Calibration

  • Tables 2–8 show the remaining detailed calibration results for different methods, with captions identifying small- and large-size model test results on TriviaQA and Sciq, and small-size model results on WikiQA.Source: §A.7 · p. 15, Table 2 · p. 18, Table 3 · p. 19

Related papers

Loading 2501.03991v1…