Annotation-Efficient Universal Honesty Alignment

1 INTRODUCTION

• Honesty alignment aims to let models answer when confident and abstain or seek external assistance when uncertain.The paper frames this self-assessment as important for trustworthy AI deployment.Source: §1 · p. 1
• Training-based calibration can be more effective, but universal performance across diverse tasks demands substantial, expensive correctness annotations.The introduction asks whether optimal honesty alignment requires so many correctness labels.Source: §1 · p. 1
• Self-consistency confidence and response correctness show high correlation across questions, motivating its use as inexpensive elicitation supervision.EliCal uses self-consistency signals to teach confidence expression before calibrating against correctness labels.Source: Figure 1 · p. 2, §1 · p. 2
• EliCal first elicits confidence from self-consistency supervision, then calibrates that confidence against correctness with a smaller annotation set.The paper describes the stages as resembling pretraining followed by finetuning.Source: §1 · p. 2
• HonestyBench combines ten free-form factual QA datasets with over 560k training samples and 38k in-domain plus 33k out-of-domain evaluation samples.It provides twenty sampled responses and one greedy-search response per model-question pair, with correctness and self-consistency confidence annotations.Source: §1 · p. 2
• With 1k labeled samples (~0.18%), EliCal achieves approximately 98% of the full-supervision upper bound and outperforms Cal-Only on MMLU tasks.Both EliCal and Cal-Only trained with all 560k+ correctness annotations achieve upper-bound alignment across ten QA tasks and outperform the best training-free baseline by over 17%.Source: §1 · p. 2

2 RELATED WORK

• Prior honesty-alignment research broadly separates confidence measurement and calibration into training-free and training-based approaches.Source: §2 · p. 2

2.1 TRAINING-FREE CONFIDENCE INVESTIGATION

• Training-free methods use token probabilities, verbalized confidence, or semantic consistency across generations; self-consistency is reported as the most reliable of these signals.Token probabilities can be miscalibrated in free-form generation, while verbalized confidence can remain overconfident.Source: §2.1 · pp. 2–3

2.2 TRAINING-BASED CONFIDENCE CALIBRATION

• Training-based methods use correctness annotations to calibrate confidence, but optimal performance carries high annotation costs; EliCal instead elicits confidence at scale before calibration with fewer labels.Earlier internal-uncertainty supervision was used to determine abstention rather than teach models to express confidence.Source: §2.2 · p. 3

3.1 TASK FORMULATION OF HONESTY ALIGNMENT

• The target confidence is a 0-to-1 score reflecting the probability of a correct response, represented by expected accuracy over possible outputs and approximated with k sampled responses.Source: §3.1 · p. 3

3.2 CONFIDENCE ESTIMATION BASED ON SELF-CONSISTENCY

• Self-consistency estimates confidence from the probability mass of the most confident response, using semantic agreement with that response across sampled generations.In practice, the estimate uses k responses sampled under the decoding policy.Source: §3.2 · p. 4
• Unlike semantic uncertainty, self-consistency avoids costly explicit clustering while retaining a similar ability to reflect semantic variability across outputs.Semantic uncertainty clusters responses into meaning-based groups and computes entropy across those groups.Source: §3.2 · p. 4

4.1 OVERVIEW

• EliCal first elicits internal confidence using self-consistency supervision, then calibrates that confidence against actual accuracy with a much smaller set of correctness annotations.Source: §4 · p. 4, §4.1 · p. 4
• The model generally overestimates confidence, but self-consistency confidence is highly correlated with its true capabilities.Source: §4.1 · p. 4

4.2 MODEL ARCHITECTURE

• A frozen backbone is augmented with LoRA modules and a linear head on the final question-token state to predict confidence, trained with an MSE objective.Source: §4.2 · p. 5

4.3 TWO STAGES OF ELICAL

• Stage 1 trains on self-consistency targets from a large question set; Stage 2 fine-tunes from that model to predict correctness on a small annotated QA set.Source: §4.3 · pp. 5–6
• Elicitation provides a foundation for calibration, and the resulting model can predict confidence before generation without repeated sampling and consistency checking.Source: §4.3 · p. 6

• HonestyBench consolidates 10 widely used public free-form factual QA datasets, with 560k training samples, 38k in-domain evaluation samples, and 33k out-of-domain evaluation samples.Source: §5 · p. 6
• For ParaRel, 3,000 instances are randomly sampled as the test set, while the remaining instances are used for training.Source: Table 1 · p. 6
• Training combines NQ, TQ, 2Wiki, HotpotQA, and ParaRel, covering single-hop, multi-hop, and template-generated questions across over 560k QA pairs.Source: §5 · p. 6
• Evaluation uses test or development splits from the five training datasets and five additional OOD datasets, with consistency and correctness scores for each question.Source: §5 · p. 7

• Honesty alignment is measured with AUROC, which evaluates discrimination between correct and incorrect predictions; QA performance is measured by answer accuracy.Source: §6 · p. 7
• Comparisons include six training-free and two training-based baselines, alongside Thermometer and DACA temperature-scaling methods.Source: §6 · p. 7
• EliCal and Eli-Only use self-consistency confidence from all HonestyBench-Train questions, while correctness annotation is sampled at sizes from 1k to over 560k.Source: §6 · p. 7

7 RESULTS AND ANALYSIS

• With all HonestyBench annotations, EliCal and Cal-Only achieve very high AUROC, significantly outperforming all training-free methods.The results explore the upper bound of honesty alignment on a large-scale dataset.Source: §7 · p. 8, Table 2 · p. 8
• About 98% of the performance of Cal-Only trained on over 560k annotations is reached by EliCal with only 1k annotated samples.With 1k annotations, EliCal significantly outperforms all baseline methods and achieves the highest AUROC on nearly all datasets.Source: §7 · p. 8, Figure 5 · p. 8
• On MMLU, EliCal outperforms Cal-Only even with over 560k annotations, while the methods converge in standard OOD settings when annotations are sufficient.The paper attributes similar in-domain and standard OOD patterns to their shared free-form question format and Wikipedia-derived QA pairs.Source: §7 · pp. 8–9
• Eli-Only performs on par with Consis-Sem without repeated generations, while EliCal’s confidence can be binarized and significantly outperforms Cal-Only on alignment.For in-domain alignment, Cal-Only is comparable to EliCal when many annotations are available; EliCal consistently leads on MMLU.Source: §7 · p. 9, Figure 6 · p. 9

7.1 ABLATION

• More elicitation training data improves performance with slowing gains toward Consis-Sem, and EliCal (1k) still significantly outperforms Cal-Only (1k) on Qwen2.5-32B-Instruct.The training-size study uses Qwen2.5-7B-Instruct; the larger-model comparison uses Qwen2.5-32B-Instruct.Source: §7.1 · pp. 9–10, Figure 7 · p. 9, Table 4 · p. 10
• With a linear head, EliCal consistently outperforms Cal-Only, especially with limited data, but performs below the Figure 5 results.The head maps the final-layer hidden state of the last question token to a confidence score; the authors attribute lower performance to limited interaction and expressiveness.Source: §7.1 · p. 9
• Across k values of 2, 5, 10, and 20, EliCal is stable; Consis-Sem improves as k increases, while Eli-Only varies only slightly.Even at k = 2, the self-consistency signal remains informative for learning internal confidence from large-scale unlabeled data.Source: §7.1 · p. 10, Table 3 · p. 10

8 CONCLUSION AND FUTURE WORK

• EliCal is an annotation-efficient two-stage honesty-alignment framework, and HonestyBench supports universal honesty training and comprehensive evaluation.The paper reports that EliCal improves model confidence expression with minimal labeled data.Source: §8 · p. 10
• The work focuses primarily on single-turn, text-only, free-form QA tasks.The authors identify multi-turn interactions, multimodal settings, and broader task types as future directions.Source: §8 · p. 10
• The authors describe HonestyBench as supporting models that excel across diverse tasks and scalable honesty alignment in real-world AI applications.Source: §8 · p. 10

ETHICS STATEMENT

• All models used in the paper are open-source, and all datasets are publicly available factual QA datasets.Source: ETHICS STATEMENT · p. 11
• The datasets do not contain harmful information.Source: ETHICS STATEMENT · p. 11
• The work is dedicated to improving model honesty and does not involve generating harmful content.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

• The authors state that LoRA training avoids full-model fine-tuning, does not require extensive GPU resources for reproduction, and that code, HonestyBench, and trained models will be open-sourced.They also state that their models are open-source and datasets come from publicly available sources.Source: REPRODUCIBILITY STATEMENT · p. 11

A RELATED WORK

• Prior work frames honesty evaluation around whether confidence aligns with answer correctness, with training-free and training-based approaches as broad categories.Source: A RELATED WORK · p. 15

A.1 TRAINING-FREE CONFIDENCE INVESTIGATION

• Training-free confidence methods use token probabilities, response consistency, or verbalized confidence, with reported limitations for free-form generation and verbalized judgments.The cited work also explores consistency across models and languages.Source: §A.1 · p. 15

A.2 TRAINING-BASED CONFIDENCE CALIBRATION

• Training-based work investigates correctness cues in internal representations and confidence verbalization, while this paper uses large-scale self-consistency confidence to elicit expression before calibrating it with a small supervised set.Source: §A.2 · p. 15

B FURTHER ANALYSIS USING ECE

• ECE is similarly low for EliCal and Cal-Only in most cases, while the authors say ECE and AUROC together suggest EliCal is superior in calibration and discrimination.They contrast this with Cal-Only's worse AUROC when labeled data is limited and note that temperature scaling can lower ECE without improving discrimination.Source: B FURTHER ANALYSIS USING ECE · pp. 16, 18, Figure 11 · p. 17, Table 6 · p. 18

D DETAILS OF LORA

• The LoRA description applies a low-rank trainable update to linear transformations, training only its A and B parameters while keeping the original weight matrix frozen.The rank is much smaller than the input and output dimensions, and a scaling factor is used.Source: D DETAILS OF LORA · pp. 18–19

E DETAILS OF BASELINES

• For each question, training-free baselines include confidence from greedy-answer token-generation probabilities, implemented as raw Prob and length-normalized N-Prob, and verbalized confidence in zero-shot and few-shot forms; 20 sampled answers are used.Source: E DETAILS OF BASELINES · pp. 19–20

F THE USE OF LARGE LANGUAGE MODELS

• The paper reports using LLMs for grammar correction, sentence polishing, and repetitive plotting code, with human review of model-polished text.Source: F THE USE OF LARGE LANGUAGE MODELS · p. 21

G PROMPTS

• The prompt section points readers to examples for semantic consistency judgments, verbalized confidence, QA, and correctness judgments.Source: Figure 12 · p. 20, Figure 13 · p. 20, Figure 14 · p. 21