PaperlayerView exact revision on arXiv

Source-linked AI summary

NOVA: NOise-aware Verbal Confidence CAlibration for Robust Large Language Models in RAG Systems

Jiayu Liu, Rui Wang, Qing Zong, Yumeng Wang, Cheng Qian, Qingcheng Zeng, Tianshi Zheng, Haochen Shi, Dadi Guo, Baixuan Xu, Chunyang Li, Yangqiu Song

arXiv:2601.11004v3cs.CL

TL;DR

RAG confidence calibration is poorly understood despite the importance of reliable factual answers and the persistence of hallucinations with retrieved evidence. The paper studies this problem, introduces NOVA Rules and a noise-aware SFT framework using approximately 2K examples, and reports 10.9% in-domain and 8.0% out-of-domain ECE improvements. NOVA also improves passage-utility judgments and interpretability by grounding confidence in structured intermediate judgments.

  • Problem

    Confidence calibration under RAG remains insufficiently studied even though retrieved evidence can be noisy and models may hallucinate despite access to additional information.

  • Method

    NOVA uses noise-aware calibration rules to guide supervised fine-tuning on approximately 2K examples without relying on external teacher models.

  • Results

    10.9% in-domain and 8.0% out-of-distribution ECE improvements are reported, alongside improved passage-utility judgment accuracy and interpretability.

  • Takeaways & Limitations

    NOVA bridges retrieval noise and verbal calibration by equipping models with intrinsic noise awareness and more transparent confidence reasoning.

  • Takeaways & Limitations

    Training uses synthetic noise categories, so generalization to nuanced organic retrieval noise in specialized domains remains uncertain.

Abstract

from arXiv · show

Accurately assessing model confidence is essential for deploying large language models (LLMs) in mission-critical factual domains. While retrieval-augmented generation (RAG) is widely adopted to improve grounding, confidence calibration in RAG settings remains poorly understood. We conduct a systematic study across four benchmarks, revealing that LLMs exhibit poor calibration performance especially when noisy contexts are retrieved. Specifically, contradictory or irrelevant evidence tends to exacerbate the model's overconfidence issue. To address this, we propose NOVA Rules (NOise-Aware Verbal Confidence CAlibration Rules) to provide a principled foundation for resolving overconfidence under noise. We further design NOVA, a noise-aware calibration framework that synthesizes supervision from ~2K HotpotQA examples guided by these rules. By performing supervised fine-tuning (SFT) with this data, NOVA equips models with intrinsic noise awareness without relying on stronger teacher models. Empirical results show that NOVA yields substantial gains, improving ECE scores by 10.9% in-domain and 8.0% out-of-domain. By bridging the gap between retrieval noise and verbal calibration, NOVA paves the way for both accurate and epistemically reliable LLMs.

1 Introduction

RAG improves access to external knowledge but does not eliminate hallucinations, making confidence calibration under retrieved evidence an important open problem. NOVA studies this setting and addresses overconfidence caused by noisy retrieval.

  • RAG can remain inaccurate or incomplete, and models may hallucinate despite access to additional information.
  • Verbal confidence calibration in RAG is underexplored because confidence must account for uncertainty in retrieved evidence.
  • Existing methods often overlook external uncertainty from retrieval contents or require inaccessible internal signals and costly sampling.
  • Average ECE exceeds 0.4 across all four datasets, with irrelevant and counterfactual passages especially undermining calibration.
  • NOVA uses rules and approximately 2K examples to train noise-aware confidence calibration without external teacher models.
  • NOVA improves calibration by 10.9% in-domain and 8.0% out-of-distribution while grounding confidence in structured intermediate judgments.

2 Related Work

Prior confidence-estimation work spans white-box and black-box approaches, but existing methods do not adequately address retrieval-specific uncertainty. NOVA targets this gap by modeling noise-aware confidence in RAG.

  • White-box confidence methods use internal signals such as perplexity or logits, limiting their utility for proprietary models.
  • Black-box approaches estimate confidence through sampling consistency, verbalized scores, reinforcement learning, specialized losses, or teacher-distilled critiques.
  • Existing verbal-confidence methods primarily target closed-book internal reasoning rather than uncertainty introduced by retrieved contents.
  • Robustness frameworks address retrieval noise and accuracy but neglect its relationship with confidence calibration.

3 Task Formalization

The paper formalizes RAG confidence calibration as aligning a model’s confidence with answer correctness while accounting for uncertainty introduced by retrieval. It defines passage categories and evaluates calibration and discrimination.

  • A retriever maps a query and corpus to passages, which a RAG model uses to produce an answer and scalar confidence score.
  • A prediction is correct when its normalized response contains at least one normalized gold answer as a substring.
  • Calibration aligns model-reported confidence with empirical correctness while accounting for uncertainty introduced by retrieval.
  • Retrieved passages are categorized as gold, counterfactual, relevant, or irrelevant according to their semantic content.
  • ECE measures the average discrepancy between confidence and accuracy, while AUROC measures confidence’s ability to distinguish correct from incorrect predictions.

4 Experiment

Experiments evaluate four open-source models on four datasets using standardized RAG prompts, Wikipedia retrieval, and robustness checks for prompting and passage ordering.

  • The experiments use four widely used open-source LLMs, excluding proprietary models because internal-state access or fine-tuning is required.
  • Evaluation uses subsets of Natural Questions, Bamboogle, StrategyQA, and HotpotQA as primary benchmarks.
  • Chain-of-Thought prompting is used by default, with additional prompts testing robustness to verbal-confidence instability.
  • The RAG corpus is Wikimedia Wikipedia, and BM25 and Contriever retrieve top-k passages with k = 3 in Table 1.
  • Retrieved passage order is randomized to mitigate position bias, with additional checks evaluating passage positioning.

5 Analysis

Real-world RAG produces severely miscalibrated verbal confidence, and controlled analyses identify retrieval noise—especially counterfactual passages—as a major source of degradation. Relevant and irrelevant noise also worsen calibration, with irrelevant passages provoking substantial AUROC declines and increased confidence.

  • Real-world RAG Calibration: Average ECE exceeds 0.4 across four datasets, indicating poor alignment between verbal confidence and empirical correctness in real-world RAG.DeepSeek-R1-Distill-Qwen-7B reaches an average ECE of 0.542.
  • Controlled Analysis Setup: Gold-plus-noise retrieval is identified as the primary factor driving models’ calibration failures in the controlled analysis.The setup adds two noise passages to the gold passage and compares this condition with Gold Only.
  • Counterfactual Noise: 31.6% and 35.1% average ECE increases accompany 9.1% and 16.1% average AUROC decreases for Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B under counterfactual noise.Average confidence remains similar to Gold-only, indicating persistent confidence despite contradictory evidence.
  • Relevant Noise: Relevant noise increases ECE and decreases AUROC for both models across NQ and Bamboogle relative to Gold-only retrieval.Average AUROC drops by 4.6% for Llama-3.1-8B-Instruct and 10.7% for DeepSeek-R1-Distill-Llama-8B.
  • Irrelevant Noise: 8.6% and 15.7% AUROC declines occur under irrelevant noise for Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B, respectively.Irrelevant noise also moderately increases ECE and raises average confidence, consistent with overconfidence from irrelevant information.

6 Method

NOVA addresses RAG overconfidence by requiring models to distinguish useful, contradictory, and irrelevant retrieved evidence before assigning verbal confidence. Its rule-guided, filtered supervision improves calibration, generalization, passage-utility judgments, and interpretability.

  • NOVA Rules: NOVA Rules require conflict independence, noise invariance, and parametric fallback when retrieved evidence is contradictory, irrelevant, or unhelpful.The rules respectively favor internal knowledge under conflict, ignoring irrelevant passages, and answering from parametric knowledge when no gold passage is retrieved.
  • Data construction: The framework constructs counterfactual, consistent, and irrelevant passage groups to supervise passage-level and group-level utility judgments.These groups provide ground-truth labels for group judgments, while the generated responses include passage judgments, group judgments, answers, and confidence scores.
  • Data filtering and SFT: A multi-stage filtering pipeline retains samples with valid outputs, accurate passage assessments, and explicit adherence to NOVA Rules before SFT.Approximately 2,000 high-quality QA pairs remain for supervised LoRA fine-tuning.
  • In-domain results: NOVA reduces ECE by approximately 11% versus Vanilla and CoT prompting across models while improving AUROC and outperforming training-, white-box-, and test-time baselines.It surpasses Label-only SFT, Ensemble, and P(True) by over 9% in average ECE across four models using one inference pass, while maintaining or improving accuracy for three of four models.
  • Generalization: NOVA reduces average ECE by 8% over Vanilla with five passages at inference and by 39.0% over Vanilla with Contriever retrieval.These results cover increased information load and a real-retriever shift, indicating generalization beyond the fixed training format.
  • Interpretability: NOVA improves passage-utility judgment accuracy by approximately 10% on two instruction-tuned models and about 5% on two DeepSeek-distilled variants.Explicitly verbalizing these judgments links confidence to the model’s assessment of the retrieval environment, supporting interpretability.

7 Conclusion

NOVA identifies retrieval noise as a source of inflated verbal confidence and equips models with intrinsic noise awareness through consistency rules. The resulting framework improves calibration while making the reasoning process more transparent and interpretable.

  • Conclusion: Retrieval noise can inflate false certainty, making standard RAG models overconfident.NOVA addresses this vulnerability by teaching models to distinguish passage utility and decouple confidence from misleading evidence without external teacher models.

Limitations

The paper identifies boundaries in model scale, noise realism, and task/context complexity. Its evaluation is limited to open-source 7B–8B models, synthetic noise, and short-form QA with fixed-depth retrieval.

  • Model Scale and Access: Evaluation is limited to open-source models in the 7B–8B parameter range.The study excludes 70B+ and proprietary models because fine-tuning larger models is computationally prohibitive.
  • Synthetic vs. Real-World Noise: Training uses synthetic counterfactual, relevant, and irrelevant noise, whose fit to nuanced real-world retrieval errors remains uncertain.Generalization to organic noise in specialized biomedical or legal RAG domains remains untested.
  • Scalability to Complex Contexts and Tasks: Evaluation focuses on short-form question answering with fixed-depth retrieval rather than long-form generation or ultra-long contexts.Long-form hallucinations are difficult to capture with a single scalar confidence score, while massive dynamic contexts may require more efficient mechanisms than rule-based scanning.

Ethics Statements

The ethics statement describes the use of public academic datasets and Wikipedia, safety-filtered synthetic passages, open-source models, and licensed resources. AI assistants were used only for language polishing.

  • Personally Identifying or Offensive Content: Experiments use standard public academic datasets and a Wikipedia-based retrieval corpus that generally do not contain sensitive PII.The listed datasets are HotpotQA, Natural Questions, StrategyQA, and Bamboogle.
  • Personally Identifying or Offensive Content: Synthetic training passages were generated with Gemini-2.5-Pro’s safety filters, and manual checks found no offensive material.
  • Data Consent and Licenses: The paper states that all datasets, models, and the Wikipedia corpus follow their licenses and terms of use.The resources are described as open-source and distributed under permissive licenses such as CC BY-SA and Apache 2.0.
  • Implementation: Open-source models ran locally with vLLM, while Gemini-2.5-Pro was accessed through Vertex AI.Experimental settings are reported in the main methodology and appendix.
  • Use of AI Assistants: AI assistants were used solely for language polishing.

Appendices

The appendices document the model implementations and training infrastructure used throughout the experiments. The study uses four open-source models, local inference and training, and deterministic inference settings.

  • Inference Settings: Inference uses a maximum output length of 2048 and temperature 0 to ensure deterministic responses.
  • Infrastructure: vLLM serves as the inference backend and LLaMAFactory handles training.Both inference and training are conducted on four NVIDIA L20 GPUs.

A.3 Dataset Statistics

The appendices describe dataset uncertainty estimates, retrieval and prompting configurations, synthetic-noise construction, training-data filtering, and calibration experiments. They also document that retrieval noise and prompting limitations motivate NOVA’s noise-aware training framework.

  • Dataset Statistics: Table 5 reports dataset sizes with 95% confidence intervals for ECE and AUROC.The authors state that the dataset scale is sufficient for reliable estimates.
  • Prompting: Experiments compare Vanilla, CoT, Multi-Step, and noise-aware prompts, with reasoning models reporting confidence after their reasoning process.Prompt designs are illustrated in Figure 9, while noise-aware prompting incorporates NOVA Rules.
  • Noise Construction: Synthetic retrieval noise includes counterfactual, relevant, and irrelevant passages generated with Gemini 2.5 Pro.The setup selects the last three generated passages as experimental noise passages.
  • Retrieval Configuration: The RAG setup retrieves top-k = 5 passages with BM25 or Contriever and uses no reranking model.Dense-retriever inputs are truncated to 256 tokens during embedding.
  • Training Data: Training data is generated through self-consistency using 16 response paths per query and filtered through format, passage-judgment, rule-following, and alignment stages.An additional balancing step ensures a consistent distribution across models.
  • Reliability Diagrams: The reliability diagram compares accuracy with confidence, using the diagonal as the perfect-calibration reference.The upper row shows CoT with the base model, while the lower row shows SFT models.
  • Baseline Calibration: Average ECE remains above 0.4 across four datasets and prompting strategies, indicating poor baseline calibration.The experiments compare ECE and AUROC across datasets, retrievers, and prompting strategies.
  • Position Bias: Noisy passages significantly degrade verbal confidence quality regardless of the ground-truth passage position.Noise increases ECE and decreases AUROC, indicating that the vulnerability is not merely a positional artifact.

B.7 Accuracy Results under NOVA

NOVA generally preserves or improves answer accuracy while substantially improving confidence calibration under noisy retrieval. Its gains reflect rule-guided, noise-aware reasoning rather than confidence-label fitting alone.

  • 1.67% absolute accuracy gain is achieved by DeepSeek-R1-Distill-Qwen-7B under NOVA versus Vanilla prompting.
  • NOVA maintains or improves generation performance for most evaluated models, although Llama-3.1-8B-Instruct regresses by approximately 5%.
  • In a contradictory retrieval case, Vanilla selects the incorrect answer with 80% confidence, whereas NOVA identifies conflicting evidence and lowers confidence to 10%.
  • NOVA’s improvements are not driven by a single rule; partial rule replacements help, while applying all NOVA Rules achieves the best average performance.
  • Verbal confidence provides an interpretable, directly supervised way to decouple parametric belief from retrieval noise.
  • The framework prioritizes confidence calibration under conflict rather than universally resolving which contradictory source is factually correct.

E Human Evaluation

Human evaluation compares five response qualities across baseline prompting and fine-tuned methods. NOVA achieves the strongest overall ratings, supported by structured evaluation of reasoning, correctness, pertinence, and readability.

  • Four PhD-level annotators independently rate 40 responses on five criteria using a 1–5 scale.The criteria assess transparent reasoning, followability, factual correctness, pertinence, and readability.
  • The evaluation interface presents each query with five candidate responses for rating.
  • 4.56–4.75 mean scores are achieved by NOVA across all five human-evaluation criteria.NOVA significantly outperforms Vanilla, CoT, and Label-only SFT on every criterion with p < 0.001.
  • The case study tests high-conflict retrieval by mixing one ground-truth passage with two passages supporting mutually exclusive incorrect answers.
  • Under this conflict, Vanilla outputs “Omicron Persei 8” with 80% confidence, whereas NOVA detects contradictions and assigns 10% confidence.
  • Prompt comparisons include Vanilla, Chain-of-Thought, Multi-step, noise-aware prompting, and NOVA configurations.
Loading 2601.11004v3…