PaperlayerView exact revision on arXiv

Source-linked AI summary

Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimation

Vaishnavi Shrivastava, Percy Liang, Ananya Kumar

arXiv:2311.08877v1cs.CLcs.LG

TL;DR

Black-box LLMs such as GPT-4 and Claude-v1.3 do not expose the probabilities commonly used to estimate confidence, despite the need to identify likely errors. The paper evaluates linguistic confidence and surrogate-model probabilities, then combines them. The combined approach reaches 84.6% average AUC for GPT-4.

  • Problem

    GPT-4 and Claude-v1.3 do not expose the internal probabilities commonly used for confidence estimation, although confidence should be low on examples where the model is likely to err.

    Source: §1 · p. 1

  • Method

    The paper elicits linguistic confidence, uses probabilities from another model as surrogate confidence for the original model’s answer, and combines the two scores.

    Source: §1 · pp. 1–2

  • Results

    84.6% average AUC is achieved for GPT-4 by combining the method with self-consistency-based confidence, compared with 82.8% AUC for self-consistency alone.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Probabilities from weaker surrogate models can estimate confidence for stronger black-box models and outperform linguistic confidence, supporting the value of releasing model probabilities.

    Source: §8 · p. 10

Abstract

from arXiv · show

To maintain user trust, large language models (LLMs) should signal low confidence on examples where they are incorrect, instead of misleading the user. The standard approach of estimating confidence is to use the softmax probabilities of these models, but as of November 2023, state-of-the-art LLMs such as GPT-4 and Claude-v1.3 do not provide access to these probabilities. We first study eliciting confidence linguistically -- asking an LLM for its confidence in its answer -- which performs reasonably (80.5% AUC on GPT-4 averaged across 12 question-answering datasets -- 7% above a random baseline) but leaves room for improvement. We then explore using a surrogate confidence model -- using a model where we do have probabilities to evaluate the original model's confidence in a given question. Surprisingly, even though these probabilities come from a different and often weaker model, this method leads to higher AUC than linguistic confidences on 9 out of 12 datasets. Our best method composing linguistic confidences and surrogate model probabilities gives state-of-the-art confidence estimates on all 12 datasets (84.6% average AUC on GPT-4).

1 INTRODUCTION

The paper seeks confidence estimates for black-box LLMs that do not expose internal probabilities. It finds that surrogate probabilities improve on linguistic confidence, and combining approaches gives the strongest reported results.

1 INTRODUCTION

  • 80.5% AUC across 12 datasets is achieved by GPT-4 linguistic confidence, 7% above random guessing, though linguistic confidence still leaves room for improvement.On Llama 2, linguistic confidence has average AUC 10.7% lower than model probabilities.Source: §1 · p. 1
  • Surrogate confidence raises GPT-4’s average selective-classification AUC to 82.1%, and weaker or smaller surrogates can yield comparable or better AUCs for stronger models.Confidence scores can transfer across models even when the model producing them is different or much worse.Source: §1 · pp. 1–2
  • 84.6% average AUC is achieved by combining surrogate probabilities with self-consistency-based confidence, outperforming the 82.8% AUC of self-consistency alone.The combined approach uses five GPT-4 samples per input for self-consistency; Figure 1 reports Mixture++ at 0.85 AUC.Source: Figure 1 · p. 2, §1 · p. 2
  • Combining linguistic confidence with surrogate probabilities improves GPT-4’s selective-classification AUC to 83.4%, and the scores are complementary.The mixture outperforms concurrent self-consistency work, which reports 82.8% AUC and uses five GPT-4 samples per input plus post-processing.Source: §1 · p. 2
  • Linguistic confidence is coarse-grained: GPT-4 gives the same 0.9 confidence on 50% of examples, limiting separation of correct and incorrect answers.The authors suggest that surrogate probabilities can distinguish examples sharing a linguistic confidence score when the scores are mixed.Source: §1 · p. 2

2 SETUP

The setup evaluates confidence scores by how well they prioritize correct answers for selective prediction, using twelve question-answering datasets and several model families. The methods elicit linguistic confidence, use available model probabilities or surrogate-model confidence, and combine linguistic and surrogate scores.

2 SETUP

  • The task is to score answers so correct predictions rank above incorrect ones, allowing low-confidence cases to be deferred to a human or flagged to users.The focus is on models such as GPT-4 and Claude-v1.3, which do not expose softmax probabilities.Source: §2 · pp. 2–3
  • Selective-classification AUC averages accuracy across coverage levels, while AUROC measures how well confidence distinguishes correct from incorrect examples.The authors focus on AUC and AUROC because ECE measures alignment between confidence and accuracy, not discrimination between correct and incorrect examples.Source: §2 · p. 3
  • Evaluation uses twelve question-answering datasets, sampling 250 test questions per dataset or the full test set when it is smaller.The datasets span MMLU subjects, TruthfulQA, CommonsenseQA, OpenbookQA, and MedQA.Source: §2 · p. 3
  • The study includes black-box models GPT-4, Claude-v1.3, and GPT-3.5-Turbo, alongside probability-providing Llama 2 variants and text-davinci-003.Unless otherwise specified, Llama 2 refers to the 70B base model.Source: §2 · p. 3

2.1 CONFIDENCE ELICITATION METHODS

  • Linguistic confidence is elicited by prompting a model to generate an answer and its confidence together; the study tests 24 prompt formats and reports its best prompt.Formats include chain-of-thought, alternative instructions, and fake few-shot examples; results are reported as consistent across prompts.Source: §2.1 · p. 3, Figure 2 · p. 4
  • For models with token probabilities, the answer-choice probability serves as confidence; a surrogate instead supplies confidence for another model’s answer.A mixture combines the main model’s linguistic confidence with the surrogate score as (1 − α)C_M(x) + αC_S(x); the study uses Llama 2 70B as surrogate and sweeps α from 0 to 1.Source: §2.1 · p. 4

3 LINGUISTIC CONFIDENCES: ASKING THE MODEL FOR ITS CONFIDENCE

The section evaluates asking models for linguistic confidence and compares those scores with internal model probabilities where available. Linguistic confidences improve with model scale but remain weaker than model probabilities and vary little across prompts.

3 LINGUISTIC CONFIDENCES: ASKING THE MODEL FOR ITS CONFIDENCE

  • Linguistic confidences range around 50–65% AUROC, compared with 50% for random guessing, and the strongest models’ scores are still described as leaving room for improvement.As of November 2023, GPT-4 and Claude-v1.3 did not provide access to internal model probabilities.Source: §3 · p. 4
  • GPT-4’s linguistic confidence reaches 80.5% AUC, while Claude-v1.3 reaches 73.5% AUC; both remain short of model probabilities where available.For GPT-4, accuracy is 73.5% and AUC is 80.5%; for Claude-v1.3, accuracy is 65.5% and AUC is 73.5%.Source: §3 · p. 4
  • Model probabilities outperform linguistic confidences for Llama 2, with 10.7% higher AUC and 19.0% higher AUROC.Table 1 also reports that model probabilities outperform linguistic confidences across the evaluated datasets for text-davinci-003 and Llama 2.Source: §3 · p. 4, Table 1 · p. 5
  • Across 24 prompts, performance varied little, and the section reports results for the best prompt.The prompts varied the requested confidence format, chain-of-thought, follow-up questions, and instructions.Source: §3 · pp. 4–5
  • Linguistic confidence quality improves with model scale, but GPT-4’s average 65% AUROC remains below model probabilities from Llama 2 70B at 74% and text-davinci-003 at 72%.Selective-classifier utility depends on both accuracy and confidence quality.Source: §3 · p. 5

4 SURROGATE MODELS ARE RELIABLE CONFIDENCE ESTIMATORS

The section evaluates a separate surrogate model’s probabilities as confidence estimates for another model’s answers. Surrogate probabilities outperform the main model’s linguistic confidences, including when the surrogate is smaller or less accurate.

4.1 RESULTS

  • Surrogate probabilities improve AUC over linguistic confidences for all models; with Llama 2 70B probabilities, GPT-4 rises from 80.5% to 82.1%, Claude-v1.3 from 73.5% to 76.3%, and GPT-3.5 from 59.0% to 72.1%.The results also report similar increases in AUROC across models.Source: §4.1 · p. 6, Table 2 · p. 7
  • Llama 2 13B and text-davinci-003 surrogates achieve comparable or better confidence-estimation performance than models’ own linguistic confidences, despite their lower accuracy.Llama 2 13B has average accuracy of 47.2%, compared with 65.5% for Claude-v1.3 and 73.5% for GPT-4.Source: §4.1 · p. 6
  • Llama 2 70B base slightly outperforms Llama 2 70B chat for selective classification with both linguistic confidences and model probabilities, while both perform similarly as surrogates.In general, better models such as Llama 2 70B are better surrogates; GPT-4 linguistic confidences improve GPT-3.5’s AUC by 5.7% over its own.Source: §4.1 · p. 6

5 MIXTURES OF MODELS FOR BETTER CONFIDENCE ESTIMATES

The section combines linguistic confidence scores with surrogate probabilities, whose signals are complementary. It also evaluates adding self-consistency confidence scores to that mixture.

5.1 RESULTS

  • Mixing surrogate and linguistic confidences raises GPT-4’s AUC from 80.5% to 83.4% and Claude-v1.3’s from 73.5% to 77.2%; optimal α is 0.4 for GPT-4 and 0.6 for Claude-v1.3.AUROC also increases by 5.3% for GPT-4 and 5.0% for Claude-v1.3.Source: §5.1 · p. 7
  • Adding only 0.1% of surrogate probabilities as a tie-breaker beats either signal alone and closely matches the optimal mixture; for GPT-4, it captures 86% of the AUC gains and 87% of the AUROC gains.This small weight uses surrogate probabilities to order examples with the same linguistic confidence.Source: §5.1 · p. 7
  • Composing GPT-4 and Claude-v1.3 linguistic confidences boosts GPT-4’s AUC by 2.1% and AUROC by 3%; composing Llama 2 and Llama 2 Chat probabilities raises Llama 2’s AUC from 73.1% to 73.8%.Composing Llama 2 13B probabilities with GPT-4 linguistic confidences retains 66% of the AUC gains from composing GPT-4 with Llama 2 70B.Source: §5.1 · p. 7
  • For GPT-4, mixing surrogate probabilities with self-consistency linguistic confidences gives the best overall result: 84.6% average AUC and 74.1% AUROC, gains of 4.1% and 9.1% over vanilla linguistic confidences.The mixture outperforms all methods, including hybrid self-consistency confidences.Source: §5.1 · p. 7, Table 3 · p. 8

6 ANALYSIS

The analysis links linguistic confidence’s coarse scores to weaker ranking and finds that surrogate models tend to work better when their correctness patterns align with the main model. Shared mistakes and semantic similarity offer an intuition for this transfer, while surrogate probabilities can break ties between identical linguistic scores.

Why Are Vanilla Linguistic Confidences Worse Than Model Probabilities?

  • GPT-4 repeats 0.9 for 50% of examples, and its eight unique linguistic-confidence values contrast with 1,456 unique Llama 2 70B probabilities.The authors report that this clustering hinders relative confidence ordering; increasing linguistic-confidence variation did not improve AUC because accuracy fell.Source: §6 · p. 8

Why Does Surrogate Confidence Estimation Work?

  • A surrogate tends to be better when its correct-answer pattern is more correlated with the main model’s.GPT-4’s correctness correlates more with Llama 2 70B (0.39) than with Llama 2 13B (0.19).Source: §6 · pp. 8–9
  • In two representative tasks, surrogate and mixture curves have higher selective-accuracy-versus-coverage AUC than linguistic-confidence and random-confidence baselines.Figure 4 plots coverage against accuracy among the top-coverage fraction of examples.Source: Figure 4 · p. 9
  • GPT-4 and Llama 2 70B make more overlapping mistakes and show more semantic similarity in their incorrect questions than GPT-4 and Llama 2 13B.The authors reason that both models can assign higher-entropy probabilities to difficult questions and more peaked probabilities to easier ones.Source: §6 · p. 9

Why Is Tiebreaking Sufficient?

  • A small fraction of surrogate probabilities can distinguish examples that share the same linguistic-confidence score.Because surrogate probabilities are nearly unique, they provide an ordering among examples tied by repetitive linguistic scores.Source: §6 · p. 9

7 RELATED WORK

Related work studies confidence estimation, selective classification, out-of-distribution detection, and calibration. This paper distinguishes its focus on confidence for LLMs without exposed probabilities from those neighboring settings and approaches.

Confidence Estimation for LLMs

  • Concurrent work by Xiong et al. (2023) studies calibration and selective classification of prompted confidences, while this paper proposes surrogate and mixture methods for models without log probabilities.The paper describes its methods as less expensive than self-consistency approaches that prompt GPT-4 several times.Source: §7 · p. 9
  • Figure 5 compares incorrect-question embeddings for GPT-4 and Llama 2 70B or 13B; the caption reports more shared mistakes and semantic similarity for the 70B comparison.Blue marks questions only GPT-4 misses, black marks shared misses, and green marks questions only the surrogate misses.Source: Figure 5 · p. 10

Selective Classification and OOD Detection

  • Related out-of-distribution detection targets examples very different from training, whereas this paper focuses on selective classification.Prior OOD detection work includes Pimentel et al. (2014), Liang et al. (2018), and Ovadia et al. (2019).Source: §7 · p. 10

Calibration

  • Calibration differs from selective classification: a model can have zero ECE while failing to distinguish correct from incorrect examples.The passage cites Kuleshov & Liang (2015) for this distinction.Source: §7 · p. 10

8 CONCLUSION AND FUTURE WORK

The paper addresses confidence estimation for state-of-the-art LLMs that do not expose internal probabilities. It finds that weaker surrogate models can estimate stronger models’ confidence, and that combining confidence signals yields more reliable estimates.

8 CONCLUSION AND FUTURE WORK

  • The paper finds that weaker white-box surrogate probabilities can estimate stronger black-box models’ confidence, outperforming linguistic confidences, and that combining signals yields more reliable estimates.The authors identify probability transfer and signal composition as directions for future work.Source: §8 · p. 10

A.1 DATASET DETAILS

The appendix describes evaluation datasets spanning medical knowledge, commonsense, multistep reasoning, and broad subject coverage, alongside TruthfulQA. The method is evaluated on up to 250 examples per dataset.

A.1 DATASET DETAILS

  • TruthfulQA is a multiple-choice benchmark testing truthfulness across 38 categories, including health and politics.Its questions target cases where human responses may reflect implicit biases or incorrect beliefs.Source: §A.1 · p. 14
  • MedQA tests medical knowledge using questions based on USMLE and other medical board exams.Source: §A.1 · p. 14
  • CommonsenseQA tests commonsense reasoning with challenging associations drawn from ConceptNet.The dataset uses many target concepts for a single source concept.Source: §A.1 · p. 14
  • OpenbookQA requires multistep reasoning over common and commonsense knowledge across diverse topics.Source: §A.1 · p. 14
  • MMLU covers 57 subjects across STEM, humanities, and social sciences, testing both rudimentary and advanced knowledge.Source: §A.1 · p. 14
  • The evaluation uses 250 examples from each dataset, or the full test subset when it contains fewer than 250 examples.The method requires no training or model adaptation.Source: §A.1 · p. 14

A.2 MODEL ACCURACIES

GPT-4 has the highest accuracy on all 12 tasks, while text-davinci-003 has the lowest. Claude-v1.3 ranks second, and Llama 2 Chat and Base have accuracy comparable to GPT-3.5.

A.2 MODEL ACCURACIES

  • GPT-4 is the highest-accuracy model on all 12 tasks.Table 4 covers accuracies for all six models across the 12 tasks.Source: §A.2 · p. 14, Table 4 · p. 14
  • Claude-v1.3 ranks second in accuracy, while text-davinci-003 has the lowest accuracy.Source: §A.2 · p. 14
  • Llama 2 Chat and Base have accuracy comparable to GPT-3.5.Source: §A.2 · p. 14

A.3 AUC AND AUROC DEFINITIONS

The appendix defines selective-accuracy and AUROC evaluation, describes confidence-prompt selection, and reports comparisons across chat models, calibration, surrogate models, and model-pair correlations. Its results include improved GPT-4 calibration with the mixture method and better confidence estimates from surrogate probabilities than from a model’s own linguistic confidences.

A.3 AUC AND AUROC DEFINITIONS

  • Randomized classification gives GPT-4 80.5% AUC across 12 datasets, versus 78.8% with deterministic classification.Randomized tie-breaking allows prediction on exactly the requested coverage when examples share confidence scores; AUROC plots true-positive rate against false-positive rate across thresholds.Source: §A.3 · p. 15

A.4 LINGUISTIC CONFIDENCE PROMPTS

  • The study varies 24 confidence-prompt formats, including numerical scores, linguistic certainty categories, and alternative confidence instructions.The formats are assessed across 12 datasets and all models.Source: §A.4 · p. 16, §4 · p. 16

4. Prompting first for an answer and then re-prompting for the confidence in the answer

  • The shared best prompt minimizes the summed AUC drop from each model’s individually optimal prompt.Its fake few-shot examples teach the response format without task-specific information; changing their example answers and confidence scores minimally affects generated confidences.Source: §4 · p. 16

A.5 LLAMA 2 70B CHAT RESULTS

  • Llama 2 Base generally outperforms Llama 2 Chat on both linguistic confidence and model probabilities.Table 5 reports 73.1% versus 70.5% AUC for Base and Chat, respectively, for both confidence types.Source: §A.5 · p. 17, Table 5 · p. 18

A.6 ECE RESULTS

  • For GPT-4, proposed confidence methods yield the lowest ECE on 11 of 12 tasks.Figure 14 compares GPT-4 calibration using linguistic confidences and the mixture method on MedQA, Professional Law, and US Foreign Policy; the mixture improves calibration.Source: §A.6 · p. 19, Figure 14 · p. 21

A.7 SURROGATE MODEL RESULTS

A.8 CORRELATION AND COVARIANCE OF SURROGATE AND MAIN MODELS

  • Higher answer-correctness correlation with a main model generally indicates a better surrogate; similar trends hold for covariance.GPT-4 correlates more with Llama 2 Base, Llama 2 Chat, and text-davinci-003 than with Llama 2 13B.Source: Figure 12 · p. 20, §A.8 · p. 21, Figure 13 · p. 21

A.9 CALIBRATION OF MIXTURE OF MODELS

  • GPT-4’s mixture-method calibration improves on linguistic-confidence calibration across MedQA, Professional Law, and US Foreign Policy.The figure marks perfect calibration as bars aligned with the red line, where bucket-level average confidence equals average accuracy.Source: Figure 14 · p. 21
Loading 2311.08877v1…