PaperlayerView exact revision on arXiv

Source-linked AI summary

Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Models

Toghrul Abbasli, Kentaroh Toyoda, Yuan Wang, Li Chen

arXiv:2608.22483v1cs.CL

TL;DR

The paper addresses the mismatch between coarse response-level confidence and the claim-specific decisions required in high-stakes LLM use. It decomposes responses into atomic claims, calibrates inference-time confidence in closed-box settings, and finds improved factual-question calibration but failures on adversarial false premises.

  • Problem

    Response-level confidence is too coarse when one LLM response mixes correct and incorrect claims, limiting accept, reject, and verification decisions.

  • Method

    The framework decomposes responses into atomic claims and applies post-hoc calibration to inference-time consistency and self-verification signals without logits or fine-tuning.

  • Results

    Claim-level decomposition with post-hoc calibration reduces expected calibration error on factual questions, but performs poorly on adversarial false-premise inputs.

  • Takeaways & Limitations

    Closed-box deployments can use calibrated per-claim confidence for selective intervention, while adversarial inputs require active verification such as cross-examination or retrieval.

  • Takeaways & Limitations

    The principal limitation is adversarial false-premise inputs, where models confidently produce incorrect claims and confidence-only signals are insufficient.

Abstract

from arXiv · show

Large Language Models (LLMs) increasingly support decision-making in high-stakes domains, but they often hallucinate and express confidence that is misaligned with factual correctness. Response-level confidence is a coarse signal: a single generation can mix correct and incorrect statements, so a single number is not actionable for users that must accept, reject, or verify individual pieces of information. We study claim-level confidence calibration as a decision-relevant uncertainty signal: each response is decomposed into atomic, verifiable claims, and each claim is assigned a calibrated confidence using inference-time signals from consistency across samples and self-verification. Our framework operates in closed-box settings (no logits, no fine-tuning) and applies post-hoc calibration directly at the claim level, enabling selective intervention such as evidence retrieval or human review for low-confidence claims. Across TriviaQA and TruthfulQA we evaluate seven baselines on six recent models (Llama-3.1, Mistral, Qwen2.5, DeepSeek-R1, GPT-4, GPT-4o), and show that claim-level decomposition combined with post-hoc calibration reduces expected calibration error on factual questions while exposing failure modes on adversarial false-premise questions where decision-makers most need reliable uncertainty estimates.

1 Introduction

The paper frames response-level confidence as too coarse for decisions when responses mix correct and incorrect claims. It proposes claim-level calibration in closed-box settings and evaluates its scope and failure modes.

  • Miscalibrated confidence can drive wrong decisions when an LLM makes a confidently incorrect claim.
  • Response-level scores conflate reliable and unreliable statements within the same generation.A response containing four claims, three correct, cannot identify which individual sentence deserves trust from one score.
  • The paper decomposes responses into atomic claims and attaches per-claim confidence using inference-time signals in a closed-box pipeline.
  • The evaluation compares seven baselines across TriviaQA and TruthfulQA on six modern models.
  • Claim-level calibration is examined in mixed-correctness closed-box settings but fails on adversarial false premises that induce confidently wrong conclusions.
  • Unlike related claim-level methods, this work calibrates continuous per-claim confidence without modifying responses and includes adversarial false-premise evaluation.

2 Claim-Level Confidence at Inference Time

The method decomposes an LLM response into atomic, verifiable claims and estimates confidence for each claim. Post-hoc calibration makes these scores usable for selective decision actions in closed-box deployments.

  • The objective is to produce atomic, verifiable claims and confidence scores aligned with the empirical probability that each claim is evidence-supported.
  • Claim-level confidence supports accepting or rejecting responses, retrieving evidence for low-confidence claims, or escalating cases to a human.
  • The framework combines cross-sample consistency with verification or log-probability signals through a weighted confidence estimate.
  • Temperature scaling and Platt scaling are applied to per-claim scores rather than response-level likelihoods.Calibration parameters are fit on a held-out subset of the training split.
  • The verbalized variant requires only text outputs, making it applicable to API-only models without logits or fine-tuning.

3 Experiments

Experiments evaluate claim-level confidence calibration across factual and adversarial benchmarks, multiple models, and diverse baselines. Calibration improves on factual questions but breaks down when models reach confident conclusions from false premises.

  • What works for decision making: ECE is substantially lower with parametric calibration and the claim-level method than with uncalibrated few-shot baselines on TriviaQA for every model.
  • What works for decision making: 0.123 ECE is achieved by the method on Mistral/TriviaQA at α=0.3, versus 0.270 for LMvsLM and 0.107 for HalluMeasure.
  • Where claim-level decomposition pays off: The method outperforms temperature scaling by 15x on GPT-4o/TriviaQA, while HalluMeasure reaches 0.001 ECE on DeepSeek-R1/TriviaQA.
  • Where claim-level decomposition pays off: Claim-level scoring helps most in mixed-correctness, multi-claim factual responses where response-level confidence averages over heterogeneous content.
  • Where it fails: adversarial false premises: On TruthfulQA, the method’s GPT-4o ECE rises to 0.412 versus 0.118 for TS on adversarial false-premise questions.
  • Figure 2 uses ten-bin reliability diagrams for GPT-4o on TriviaQA, with colors and percentages showing each bin’s point proportion.
  • Decision-relevant takeaways: For adversarial inputs, active verification such as cross-examination or external retrieval is necessary because confidence-only signals are unreliable.

4 Conclusion

The paper presents claim-level confidence calibration as a decision-relevant signal for closed-box LLM deployments, while identifying adversarial false-premise inputs as its main failure mode.

  • 4 Conclusion: Decomposing responses into atomic claims and calibrating each claim yields actionable confidence in closed-box settings.Claims are intended to align with the empirical probability of evidential support.
  • 4 Conclusion: Claim-level calibration reduces expected calibration error on factual questions.
  • 4 Conclusion: Adversarial false-premise inputs remain a main failure mode because models can produce incorrect claims with high confidence.
  • 4 Conclusion: Production pipelines should combine claim-level confidence with active verification such as cross-examination or retrieval.

A.3 Calibration Methods for LLMs

LLM calibration methods differ by access requirements, with open-box approaches using internal model information and closed-box approaches operating without it.

  • A.3 Calibration Methods for LLMs: Open-box calibration fine-tunes models or adapters using internal logits or representations.
  • A.3 Calibration Methods for LLMs: Closed-box calibration requires no internal model access.
  • A.3 Calibration Methods for LLMs: Linguistic calibration maps verbal confidence to calibrated numerical signals.
  • A.3 Calibration Methods for LLMs: Prompt augmentation ensembles paraphrases, permutations, or in-context examples as inference-time strategies.
  • A.3 Calibration Methods for LLMs: Calibration methods can fail under distribution shift.

A.4 Calibration Metrics

The paper evaluates calibration with error, score, ranking, and generation-quality metrics, while emphasizing ECE, smECE, BS, and AUROC for calibration analysis.

  • A.4 Calibration Metrics: ECE measures the gap between bin-wise accuracy and mean confidence across k=10 confidence bins.
  • A.4 Calibration Metrics: SmoothECE replaces binning with Gaussian kernel smoothing to approximate distance from perfect calibration.
  • A.4 Calibration Metrics: Brier Score is the mean squared error between accuracy and confidence.
  • A.4 Calibration Metrics: AUROC measures misprediction-detection performance using confidence scores.
  • A.4 Calibration Metrics: BLEU is included as a generation-quality reference, while calibration analysis relies on ECE, smECE, BS, and AUROC.
  • A.4 Calibration Metrics: Additional metrics include UCE, RCE, cECE, ACE, and AURC.

B.1 Method Comparison Table

Table 2 compares uncertainty estimation methods for LLMs.

  • B.1 Method Comparison Table: Table 2 presents a comparison of uncertainty estimation methods for LLMs.

C.1 Datasets

The evaluation uses TriviaQA and TruthfulQA, with confidence mapping, claim extraction, and inference-cost measurements supporting the experimental setup.

  • Datasets: TriviaQA provides over 650K question–answer–evidence triples and 95K author-written QA pairs; the study samples 12K training and 1.5K test examples.TruthfulQA contains 817 questions across 38 categories and is split evenly between training and testing.
  • Datasets: TruthfulQA uses true and false target answers with supporting sources in a zero-shot setting designed to elicit common misconceptions.
  • Confidence mapping: Verbalized confidence expressions are mapped to numeric scores in [0, 1] using a 14-level decisiveness scale.
  • Inference cost: The method requires approximately four LLM calls per query on average: one generation, one claim-extraction call, and one verification per extracted claim.This cost is comparable to HalluMeasure and substantially cheaper than LMvsLM.
  • Claim extraction: The extractor produces self-contained atomic claims of at most 15 words with pronouns resolved.Manual inspection estimates extraction errors at <8–12% of claims on a 50-example subset.

C.5 𝛼Sensitivity Analysis

The study tunes α on held-out data and reports held-out ECE for model selection, while test-set results are reported separately; reliability diagrams visualize calibration across settings.

  • α tuning: α is tuned over {0.0, 0.1, . . . , 1.0} on a held-out split, with the per-model best α reported alongside held-out ECE.
  • Evaluation protocol: Held-out ECE guides model selection, whereas corresponding test-set ECE values appear in the “Ours” rows of Table 1.For GPT-4o on TriviaQA, held-out ECE is 0.022 and test-set ECE is 0.006 at α=1.0.
  • Evaluation protocol: When held-out tuning is unavailable, the study uses verification-only scoring with α=0.
  • Reliability diagrams: Reliability diagrams for GPT-4o on TriviaQA use 10 bins to display calibration-method behavior.Bar colors and percentage labels indicate the proportion of total points in each bin.
  • Reasoning-model diagnosis: For DeepSeek-R1, extracting only the final-answer segment consistently improves performance relative to extracting the full chain-of-thought trace.Full-trace extraction yields many intermediate reasoning claims with weakly calibrated per-step confidence, diluting the final-answer signal.

E Discussion: When the Method Helps Decision Making

Claim-level calibration is most useful when responses mix correct and incorrect claims, when deployments are closed-box, and when fine-grained intervention matters. Its practical boundaries include consensus errors, verifier or extraction errors, and unverifiable claims.

  • When the method helps: Mixed-correctness responses benefit from claim-level scores because they support targeted intervention on unreliable claims.
  • When the method helps: Closed-box deployments can use verbalized claim-level confidence without requiring logit access for likelihood-based calibration.
  • When the method helps: Claim-level decomposition is most valuable for moderately complex multi-claim responses; response-level confidence may suffice for simple queries.
  • Failure modes: Consensus fails when all sampled responses agree on an incorrect claim, including TruthfulQA cases with GPT-4o.
  • Failure modes: Errors in claim extraction or verification propagate into the resulting confidence estimates.
  • Failure modes: The framework does not support claims that cannot be externally checked, such as future predictions or creative content.
Loading 2608.22483v1…